SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Limitation and potential improvement methods for Semsim. One limitation of SemSim is its potential performance decrease when faced with significant distributional shifts (such as using it on medical images). To address this limitation, we can annotate diverse types of data to enhance the adaptability of Semsim to a wider range of domain variations. Additionally, exploring other strategies, such as incorporating local image regions and utilizing multi-valued annotated training data, could also be considered to further enhance the effectiveness of SemSim.
Experimental Setups
- Datasets. We evaluate using the CIFAR-100 [18], Caltech-101 [5], Imagenette  $[2]^{3}$ , CelebA [21], and Stanford Dogs [17] datasets. The first three are for generic object recognition, CelebA is for face recognition, and Stanford Dogs is a fine-grained classification dataset.
- Classification models. We use the following backbones: ResNet20, ResNet50, ResNet152 [9], DenseNet [12] and 8-layer CoveNet [7]. They were trained using different strategies, such as data augmentation [7], gradients with Gaussian/Laplacian noise [43], and layer-wise pruning techniques [4]. In total, there are 70 different models. Details are provided in the supplementary material.
- Reconstruction attack methods. We mainly use InvGrad [8]. In the ablation study, we evaluate SemSim using four additional attack methods, including DLG [43], CAFE [15], and GradAttack [13]. - Correlation strength measurements. We use two rank correlation coefficients: Spearman's rank correlation  $\rho$  [31] and Kendall's rank correlation  $\tau$  [16] to measure the consistency between different metrics with human perception. Values of  $\rho$  and  $\tau$  are between  $[-1, 1]$ . Being closer to -1 or 1 indicates a stronger correlation, and 0 means no correlation. For clarity, the absolute values of  $\rho$  and  $\tau$  are provided in the paper.
- Classification model training. The training of all the models to be evaluated was conducted using the PyTorch framework. The details of the classifier training, such as the specific architectures and hyperparameters used for each model, are provided in the supplementary material. We perform model training with one RTX-2080TI GPU and a 16-core AMD Threadripper CPU @ 3.5Ghz.
SemSim model training. In the main evaluation, SemSim is trained using a learning rate of 0.1 and a batch size of 128 on the ResNet50 architecture for 200 epochs. We use leave-one-out evaluation on the 5 datasets. Some examples of the annotation data are provided in Figure 1 and Figure 5.
3Imagenette is a subset of 10 easily classified classes from ImageNet. https://www.tensorflow.org/datasets/catalog/imagenette
Table 1: Comparison of different metrics on different datasets. For each metric, we rank the 14 models and compute the correlation with rankings made by human assessment. For each test set (Column 1), SemSim is trained on the combination of the rest four datasets. Here, InvGrad [8] attack is used.  $|\rho|$  and  $|\tau|$  are reported. SemSim has a much stronger correlation with human annotations.
Datasets	Metrics	PSNR	MSE	SSIM	LPIPS	FID	SemSim
CIFAR-100	Spearman's |ρ|	0.6703	0.6176	0.3939	0.5127	0.7363	0.8637
Kendall's |τ|	0.4725	0.4286	0.2904	0.3978	0.5604	0.7143
Caltech-101	Spearman's |ρ|	0.6970	0.7349	0.7218	0.5127	0.2242	0.8182
Kendall's |τ|	0.5556	0.5525	0.5244	0.4072	0.1556	0.6889
Imagenette	Spearman's |ρ|	0.5382	0.6395	0.6433	0.6539	0.4791	0.8257
Kendall's |τ|	0.4349	0.5525	0.5108	0.5922	0.4252	0.7012
CelebA	Spearman's |ρ|	0.7495	0.7349	0.6846	0.5824	0.1516	0.8263
Kendall's |τ|	0.5604	0.5525	0.5264	0.4505	0.0989	0.6923
Stanford Dogs	Spearman's |ρ|	0.4023	0.3968	0.4782	0.5031	0.3969	0.7120
Kendall's |τ|	0.3537	0.2743	0.3048	0.3929	0.3196	0.5938
Table 2: Comparison of different metrics under different attacks on the CIFAR-100 dataset. SemSim is trained using human annotations obtained through the InvGrad [8] attack method and evaluated on different attack methods listed in the table.
Attacks	Metrics	PSNR	MSE	SSIM	LPIPS	FID	SemSim
DLG [43]	Spearman's |ρ|	0.6515	0.6367	0.4069	0.5477	0.7268	0.8749
Kendall's |τ|	0.4857	0.4174	0.2858	0.4294	0.5237	0.7342
CAFE [15]	Spearman's |ρ|	0.7104	0.6916	0.5870	0.6793	0.6925	0.8864
Kendall's |τ|	0.5392	0.4259	0.3318	0.4762	0.4735	0.7510
GradAttack [13]	Spearman's |ρ|	0.6831	0.6944	0.5753	0.6841	0.7204	0.8437
Kendall's |τ|	0.4943	0.4980	0.3495	0.4531	0.4819	0.7260
5.1 Main Evaluation
Inconsistency between existing metrics and human perception: more results. On each of the five test sets, we rank the 14 models according to each of the existing metrics as well as human perception. The model ranking of each metric is correlated with that from human assessment. We find that PSNR, MSE, SSIM, LPIPS, and FID do not have a high correlation with human assessment. The worst performing metric is FID: Kendal's  $|\tau|$  is only 0.1556, 0.4252, 0.0989, and 0.3196 between FID and human perception, on the four test sets, respectively. While the rest four metrics exhibit a stronger correlation than FID, Kendall's  $|\tau|$  is generally around 0.5, which is considered only moderate.
Moreover, from Figure 3, we find that the correlation between existing metrics themselves is often weak. For example, In Figure 3 right, Kendall's  $|\tau|$  is only 0.2904 between PSNR and LPIPS. This contradiction also exists between PSNR and LPIPS and others. The above results advocate the study of new metrics that are privacy oriented.
Comparing SemSim with existing metrics in terms of faithfulness to human perception. We utilize SemSim to rank the models and examine its correlation with the ranking based on human perception, as shown in Table 1. We make two key observations.
First, SemSim exhibits a much stronger correlation with human perception. On the five test sets, Kendall's  $|\tau|$  is 0.7143, 0.6889, 0.7012, 0.6923, and 0.5938, respectively, which is 0.2418, 0.1333, 0.2663, 0.1319 and 0.2401 higher than PSNR, for example. The above results suggest the risks of current metrics in the community and advocate the proposed learning-based, privacy-oriented metric.
Second, on Stanford Dogs, while SemSim is still much more faithful to human perception than other metrics, the overall correlation is lower than other datasets. Because dog species are hard to recognize, more noise was introduced to human annotation and thus to the ranking results and correlation. We speculate that fine-grained datasets are harder for privacy interception through reconstruction: humans themselves will find it hard to recognize the private content.
Generalization ability of SemSim. In Table 1, we adopt a leave-one-out setup, where SemSim is trained on four datasets and tested on the fifth dataset. Moreover, for each dataset, the model
Figure 6: Comparing the ranking of reconstruction images using PSNR and SemSim. From the visualizations, we can observe that PSNR exhibits some inconsistencies with human perception, while SemSim consistently aligns with the judgments of human annotators. In the two examples, SemSim correctly ranks all the images with noticeable information leakage (including a laptop or chair) before the ones without or with less information leakage (that are unrecognizable). However, the rankings provided by PSNR are inaccurate for some images.
(A) impact of training data size
(B) impact of backbone
Figure 7: Analysis of SemSim. Evaluating the impact of (A) size of human-annotated training data, (B) variations of SemSim backbones, and (C) different loss functions for SemSim training. Experiments are conducted on the CIFAR-100 dataset.
(C) impact of loss function
architectures are different. For example, when using CelebA as a test set, the tested target models are ResNet50 and DesNet etc, while target models in training are Resnet20, 8-layer CoveNet and ResNet152 etc. As such, the superior results in Table 1 demonstrate the generalization ability of SemSim for test sets and model architectures.
Furthermore, we use SemSim to evaluate model vulnerability to unseen attacks. Results are provided in Table 2, where SemSim is trained using human annotations obtained through the InvGrad [8] attack method and evaluated on other attack methods such as DLG, CAFE, and GradAttack. Remarkably, we consistently observe higher correlation between SemSim and human perception compared to existing metrics. On the CIFAR-100 dataset, we observe significant improvements in Kendall's  $|\tau|$  of 0.7342, 0.7510, and 0.7260, respectively, for DLG, CAFE, and GradAttack. These findings demonstrate the robustness of SemSim in capturing the privacy leakage of reconstructed images across different reconstruction attacks.
Visualization results of SemSim. Figure 6 presents two examples of ranking reconstruction images using PSNR and SemSim. In both cases, SemSim outperforms PSNR and provides better results.
Impact of the number of human annotations on SemSim training. To evaluate this impact, we use the CIFAR100 dataset for testing and randomly select human-annotated training samples from the rest four datasets to train SemSim. Results are shown in Figure 7 (A). We observe a correlation drop between SemSim and human perception as the number of training samples decreases. However, even with as few as 50 training samples (each samples includes 14 reconstructed images), SemSim outperforms existing metrics like PSNR and FID.
Impact of different backbones for SemSim. As mentioned in the implementation details, SemSim uses a simple ResNet50 network. Here, we try several different opinions such as LeNet and ResNet18, and present their correlation with human perception in Figure 7 (B). We show that even a simple LeNet model can achieve  $|\tau|$  scores higher than 0.65, surpassing the best score of 0.5604 obtained by FID. Moreover, we observe that there is a correlation between the complexity of the backbone architectures and the performance of SemSim. This indicates that utilizing more advanced and sophisticated backbone models may be able to further enhance SemSim to capture and represent visual information, leading to improved evaluation of privacy leakage in reconstructed images.
Impact of other loss functions for SemSim training. We further experiment with different loss functions and hyperparameters, including the contrastive loss and the triplet loss (where we set  $\alpha = 1$  in experiments). From the results shown in Figure 7 (C), we observe that the triplet loss shows a comparable correlation strength to the contrastive loss in relation to human assessment.
This paper investigates the suitability of existing evaluation metrics when privacy is leaked by a reconstruction attack. We first collect comprehensive human perception annotations on whether a reconstructed image leaks information from the original image. We find that model vulnerability to such attacks measured by existing metrics such as PSNR has a relatively weak correlation with human perception, which poses a potential risk to the community. We then propose SemSim trained on human annotations to address this problem. On five test sets, we show that SemSim has much stronger faithfulness to human perception than existing metrics. Such faithfulness remains strong when SemSim is used for different model architectures, test categories, and attack methods, thus validating its effectiveness. In future work, we will collect human perception labels from a wider source of datasets and train a more generalizable metric for privacy leakage assessment.
This research was funded by Sony AI. We also gratefully acknowledge partial support from the ARC Discovery Early Career Researcher Award (DE200101283) awarded to Liang Zheng, the ARC Discovery Project (DP210102801), and the ARC Discovery Grant (Grant ID: DP220100800) awarded to Hongdong Li.
[1] Chen Chen, Lingjuan Lyu, Han Yu, and Gang Chen. Practical attribute reconstruction attack against federated learning. IEEE Transactions on Big Data, 2022.
[2] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255, 2009.
[3] DC Dowson and BV666017 Landau. The frechet distance between multivariate normal distributions. Journal of multivariate analysis, 12(3):450-455, 1982.
[4] Aritra Dutta, El Houcine Bergou, Ahmed M Abdelmoniem, Chen-Yu Ho, Atal Narayan Sahu, Marco Canini, and Panos Kalnis. On the discrepancy between the theoretical analysis and practical implementations of compressed communication for distributed deep learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 3817-3824, 2020.
[5] Li Fei-Fei, Rob Fergus, and Pietro Perona. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. Computer Vision and Pattern Recognition Workshop, 2004.
[6] Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model inversion attacks that exploit confidence information and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC conference on computer and communications security, pages 1322-1333, 2015.
[7] Wei Gao, Shangwei Guo, Tianwei Zhang, Han Qiu, Yonggang Wen, and Yang Liu. Privacy-preserving collaborative learning with automatic transformation search. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 114-123, 2021.
[28] Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. Advances in neural information processing systems, 29, 2016.
[29] Umme Sara, Morium Akter, and Mohammad Shorif Uddin. Image quality assessment through fsim, ssim, mse and psnr—a comparative study. Journal of Computer and Communications, 7(3):8-18, 2019.
[30] Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 815-823, 2015.
[31] Charles Spearman. The proof and measurement of association between two things. 1961.
[32] Jingwei Sun, Ang Li, Binghui Wang, Huanrui Yang, Hai Li, and Yiran Chen. Soteria: Provable defense against privacy leakage in federated learning from representation perspective. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9311-9319, 2021.
[33] Zhibo Wang, Mengkai Song, Zhifei Zhang, Yang Song, Qian Wang, and Hairong Qi. Beyond inferring class representatives: User-level privacy leakage from federated learning. In IEEE INFOCOM 2019-IEEE conference on computer communications, pages 2512-2520, 2019.
[34] Zhou Wang and Alan C Bovik. A universal image quality index. IEEE signal processing letters, 9(3): 81-84, 2002.
[35] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600-612, 2004.
[36] Taihong Xiao, Yi-Hsuan Tsai, Kihyuk Sohn, Manmohan Chandraker, and Ming-Hsuan Yang. Adversarial learning of privacy-preserving and task-oriented representations. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 12434-12441, 2020.
[37] Jufeng Yang, Xiaoxiao Sun, Jie Liang, and Paul L Rosin. Clinical skin lesion diagnosis using representations inspired by dermatologist criteria. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1258-1266, 2018.
[38] Yi Zeng, Minzhou Pan, Himanshu Jahagirdar, Ming Jin, Lingjuan Lyu, and Ruoxi Jia. {Meta-Sift}: How to sift out a clean subset in the presence of data poisoning? In 32nd USENIX Security Symposium (USENIX Security 23), pages 1667-1684, 2023.
[39] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586-595, 2018.
[40] Rui Zhang, Song Guo, Junxiao Wang, Xin Xie, and Dacheng Tao. A survey on gradient inversion: Attacks, defenses and future directions. arXiv preprint arXiv:2206.07284, 2022.
[41] Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. idlg: Improved deep leakage from gradients. arXiv preprint arXiv:2001.02610, 2020.
[42] Junyi Zhu and Matthew Blaschko. R-gap: Recursive gradient attack on privacy. Proceedings ICLR 2021, 2021.
[43] Ligeng Zhu, Zhijian Liu, and Song Han. Deep leakage from gradients. Advances in neural information processing systems, 32, 2019.
A Data Annotation
In Section 4.1, we briefly introduced how humans annotate the reconstructed images for different datasets. In the supplementary material, we have included a graphical user interface (GUI) that was utilized by the annotators. Figure 8 displays the GUI, where (A) and (B) were specifically designed for annotating different datasets.
(A) image classification
(B) face recognition and fine-grained image classification
Figure 8: Graphical user interface (GUI) used in our human annotation process. For (A) image classification, we ask annotators to give a category to the reconstructed image. In (B) face recognition and fine-grained classification, we ask annotators to tell whether the original image and its reconstruction have the same or different identity / category.
To minimize the influence of subjective bias, we use a relatively objective formulation: whether the reconstructed image can be correctly labeled. Specifically, for CIFAR-100, Caltech-101, and Imagenette, we provide up to 20 candidate categories and see if the annotators can correctly recognize the reconstructed image; for more difficult tasks like face recognition and fine-grained classification (Celeb-A and Stanford Dogs), we give both the original and the reconstructed images and ask the annotator if they are of the same identity or species.
B Impact of margin value  $\alpha$  in the triplet loss on SemSim
The effect of the margin parameter  $\alpha$  in the triplet loss on the performance of SemSim is depicted in Figure 9. It can be observed that when  $\alpha$  is set to a value close to 1, both Spearman's rank correlation  $(\rho)$  and Kendall's rank correlation  $(\tau)$  coefficients yield better results compared to other values, on CIFAR-100 and Caltech-101 datasets. We think there are two potential reasons for this observation. Firstly, if the value of  $\alpha$  is too small, the model may struggle to effectively learn the discriminative features that distinguish positive (recognizable reconstructed images) and negative (unrecognizable reconstructed images) samples. On the other hand, if  $\alpha$  is set to a value that is too large, the model may become excessively confident in distinguishing between positive and negative samples. However, this can lead to convergence challenges, as the loss function may have difficulty approaching 0.
(A) impact  $a$  of results on CIFAR-100
(B) impact  $a$  of results on Caltech-101
Figure 9: Impact of  $\alpha$  on SemSim when testing on (A) CIFAR-100 and (B) Caltech-101. The margin value  $\alpha$  is used in the triplet loss to ensure that negative samples are kept far apart. When evaluating the reconstructed images of CIFAR-100 or Caltech-101, we trained the ResNet50 model on the four datasets (excluding CIFAR-100 or Caltech-101) using the triplet loss. The training process involved utilizing different values of the margin parameter  $\alpha$  for each dataset.
Table 3: Details of classification models. On each test set, we have two backbones trained without (vanilla,  $2^{nd}$  column) and with different strategies, such as using data augmentation ( $3^{rd} - 6^{th}$  columns) and existing defence methods ( $7^{th} - 8^{th}$  columns).
Datasets	Models
CIFAR-100	ResNet20	+ Random-	+ TranslateX	+ translateY	+ defense	+ defense
CoveNet8
Caltech-101	ResNet20	+ Random-	+ Invert &	+ Autocontrast	+ [7]	Gaussian (10-3)	+ defense
DenseNet	Pruning (70%)
Imagenette	ResNet50	& Random-	& translateY	& Autocontrast
ResNet152
CelebA	ResNet20	HorizontalFlip
DenseNet
Stanford Dogs	ResNet50	HorizontalFlip
ResNet152
In our experiments, we set  $\alpha$  to 1. However, we acknowledge that there is potential for improved performance by carefully selecting the optimal value of  $\alpha$  for different datasets.
C Classification models and training details
We conducted experiments using five datasets, CIFAR-100 [18], Caltech101 [5], CelebA [21], ImageNette [2], and Stanford dogs [17]. In our evaluation process, we considered 14 classification models for each set. Table 3 provides detailed information about these models. They were trained using stochastic gradient descent (SGD) as the optimizer, with a learning rate of 0.1.
D Metrics for reconstruction quality
Mean squared error. Assuming  $x, \bar{x} \in \mathbb{R}^{n \times m}$  are two images to compare, the mean squared error (MSE) is given by,
$$
\operatorname {M S E} (x, \bar {x}) := (1 / m n) \sum_ {i = 1} ^ {m} \sum_ {j = 1} ^ {n} \left(x _ {i j} - \bar {x} _ {i j}\right) ^ {2}. \tag {3}
$$
The value of MSE is between 0 and  $+\infty$ . The lower MSE is, the closer two images are.
Peak-Signal-to-Noise ratio. The Peak-Signal-to-Noise ratio (PSNR) is widely used in image quality assessment, which measures the ratio between the maximal power of a signal and its noise. Its value, expressed in dB, is given by:
$$
\operatorname {P S N R} (x, \bar {x}) = 2 0 \log_ {1 0} \left(\frac {\operatorname {M A X} _ {x}}{\sqrt {\operatorname {M S E} (x , \bar {x})}}\right), \tag {4}
$$
where  $MAX_{x}$  is the maximal value in the image  $x$  (often replaced by 255 for int8 images).
SSIM. Unlike PSNR, the structural similarity index measure (SSIM) [35] is a perception-based metric as it was designed to take into account characteristics of the human vision system through three metrics: luminance, contrast and structure of the image. It is shown that there is an analytical link between PSNR and SSIM and that it is often possible to predict one from the other for controlled perturbations (Gaussian blur, additive Gaussian noise and JPEG compressions) [11]. The above three metrics compute a pixel-wise distance between both images which is very limited when we assess semantic content of an image such as privacy leakage.
LPIPS. LPIPS [39], which stands for learned perceptual image patch similarity, is a perceptual metric based on a neural network aiming at correlating better with perceptual judgments. The authors take inspiration from neuroscience findings, where their model compares the activations between two images as neurons in a human cortex would. As explained in its torchmetrics documentation<sup>4</sup>, a low LPIPS score indicates high similarity. Thus, in the context of privacy assessment, low LPIPS values for an original image and its reconstruction suggest high privacy leakage [13].
Frechet Inception Distance. Aside from LPIPS and the aforementioned hand-crafted metrics calculated at the image level, our works also use Frechet inception distance (FID) [10] to measure information leakage. FID is commonly used to evaluate the domain gap between two distributions, where higher values suggest a larger domain gap. For example, FID is extensively used to evaluate the quality of images generated by generative adversarial networks (GANs) [10], by computing the distribution difference between real and generated images. In this paper, FID may reflect the difference between the original and reconstructed image distributions to reflect privacy leakage. As opposed to the pointwise metrics, FID is computed directly on image sets: InfoLeak  $(\mathcal{X},\bar{\mathcal{X}})\propto \mathrm{FID}(\mathcal{X},\bar{\mathcal{X}})$ .
As defined in [10, 3], given two Gaussian distributions with mean and covariance  $(\pmb{m},\pmb{C})$ , resp.  $(\bar{\pmb{m}},\bar{\pmb{C}})$ , FID is given by:
$$
\operatorname {F I D} \left(\left(\boldsymbol {m}, \boldsymbol {C}\right), \left(\bar {\boldsymbol {m}}, \bar {\boldsymbol {C}}\right)\right) = \left\| \boldsymbol {m} - \bar {\boldsymbol {m}} \right\| _ {2} ^ {2} + \operatorname {T r} \left(\boldsymbol {C} + \bar {\boldsymbol {C}} - 2 \left(\boldsymbol {C} \bar {\boldsymbol {C}}\right) ^ {1 / 2}\right). \tag {5}
$$