Scrambling Parameter Generation to Improve Perceptual Information Hiding

Koki Madono · Masayuki Tanaka · Masaki Onishi · Tetsuji Ogawa — Waseda University · AIST · Tokyo Institute of Technology

IS&T International Symposium on Electronic Imaging 2021, Human Vision and Electronic Imaging (HVEI) · Electronic Imaging 33(11), pp. 155-1–155-8 · doi:10.2352/ISSN.2470-1173.2021.11.HVEI-155

Image scramble approach: the client scrambles training and test images with a scramble parameter; a third party trains an adaptation network and a classifier on the scrambled images — **Image scramble approach (Fig. 1 of the paper).** Every scrambler has a *scrambling parameter* (its key). How well the scrambled image hides its content depends on that parameter, yet existing methods simply draw it at random.

Abstract

The present study proposes the method to improve the perceptual information hiding in image scramble approaches. Image scramble approaches have been used to overcome the privacy issues on the cloud-based machine learning approach. The performance of image scramble approaches are depending on the scramble parameters; because it decides the performance of perceptual information hiding. However, in existing image scramble approaches, the performance by scrambling parameters has not been quantitatively evaluated. This may be led to show private information in public. To overcome this issue, a suitable metric is investigated to hide PIH, and then scrambling parameter generation is proposed to combine image scramble approaches. Experimental comparisons using several image quality assessment metrics show that Learned Perceptual Image Patch Similarity (LPIPS) is suitable for PIH. Also, the proposed scrambling parameter generation is experimentally confirmed effective to hide PIH while keeping the classification performance.

What the paper proposes

A metric for perceptual information hiding (PIH). Nine candidates were compared: MSE, PSNR, SSIM, BRISQUE, NIQE, PIQE, VGG-feature MSE, FID and LPIPS. The test used CIFAR-10 images scrambled by pixel-based encryption at intensity-inversion ratios from 0 to 100 %. A good PIH metric should peak at 50 % inversion and fall towards 0 % and 100 %. BRISQUE, FID and LPIPS behave this way, and LPIPS (AlexNet) has the sharpest peak, so it becomes the PIH metric.
Scrambling parameter generation (Algorithm 1). Draw N random scrambling parameters P, score each one by ρ(P) = (1/M) Σ_i LPIPS(x_i, f(x_i; P)) on a set of images, and keep the best. It works with any scrambler; the paper uses N = 20.

Nine image-quality metrics plotted against the intensity inversion ratio from 10 to 90 percent — **Which metric sees perceptual hiding? (Fig. 3).** MSE, PSNR and SSIM change monotonically with the inversion ratio and cannot tell that 50 % hides the most; BRISQUE, FID and LPIPS peak at 50 %, and LPIPS has the sharpest peak.

A CIFAR-10 image scrambled at intensity inversion ratios 10, 30, 50, 70 and 90 percent with the LPIPS score of each — **LPIPS follows perceptual hiding (Table 1).** The image scrambled at a 50 % inversion ratio is the hardest to recognise and has the highest LPIPS (0.3809); 10 % and 90 % inversions stay recognisable and score lower.

Average PIH metric versus the number of trials N for LE, EtC, ELE and PE — **Average PIH metric vs. number of trials N (Fig. 4).**

Standard deviation of the PIH metric versus the number of trials N for LE, EtC, ELE and PE — **Its standard deviation over repeated runs (Fig. 5).** The spread for LE drops quickly as N grows, while PE stays the most variable; the paper uses N = 20.

Main result

CIFAR-10 classification with a ShakeDrop classifier behind the adaptation network of each scrambler, before and after replacing the random scrambling parameter by the generated one. For PE and LE, the generated parameter raises the PIH metric (LPIPS) while the accuracy stays comparable.

	Scrambler	Acc. mean (%)	Acc. SD	PIH mean (LPIPS)	PIH SD
Original (random parameter)	ELE	69.49	1.6205	0.3000	0.0026
	EtC	79.42	2.9198	0.1482	0.0033
	PE	93.40	0.3114	0.3161	0.0038
	LE	93.75	0.2620	0.2880	0.0091
Proposed parameter generation	ELE	71.38	3.1268	0.2999	0.0030
	EtC	78.47	3.3404	0.1513	0.0014
	PE	93.58	0.1201	0.3420	0.0075
	LE	93.05	0.8164	0.3016	0.0046

Values from Table 2 of the paper (mean and standard deviation over repeated runs; batch 64, SGD with Nesterov momentum 0.9, 100 epochs, learning rate 0.1 → 0.01 → 0.001 at epochs 50 and 75). PE: pixel-based encryption; EtC: encryption-then-compression; LE: learnable encryption; ELE: extended learnable encryption.

Use it

Scrambling parameter generation and the LPIPS-based PIH metric are a pip-installable package in the paper repository, together with the experiment scripts.

pip install git+https://github.com/MADONOKOUKI/SPG_EI2020

import torch
import spg

x = torch.rand(16, 3, 32, 32)                    # a batch of images in [0, 1]
res = spg.generate("le", x, trials=20, seed=0)   # Algorithm 1: keep the key with the highest mean LPIPS
print(res.scores[0], "->", res.score)            # random parameter vs. generated parameter
x_s = res.scrambler(x)                           # scramble with the generated parameter

git clone --depth 1 https://github.com/MADONOKOUKI/SPG_EI2020
cd SPG_EI2020 && pip install -e .
python train.py --scheme le --trials 20   # Table 2, "Proposed" (--trials 1: "Original")

The repository also reproduces Fig. 3 / Table 1 (evaluate_iqa.py) and Figs. 4–5 (generate.py --sweep).

Citation

@article{madono2021scrambling,
  title     = {Scrambling Parameter Generation to Improve Perceptual Information Hiding},
  author    = {Madono, Koki and Tanaka, Masayuki and Onishi, Masaki and Ogawa, Tetsuji},
  journal   = {Electronic Imaging (Human Vision and Electronic Imaging)},
  volume    = {33},
  number    = {11},
  pages     = {155-1--155-8},
  year      = {2021},
  publisher = {Society for Imaging Science and Technology},
  doi       = {10.2352/ISSN.2470-1173.2021.11.HVEI-155}
}

Related work in this series

← Back to the publication list