Privacy Assessment on Reconstructed Images: Are Existing Evaluation Metrics Faithful to Human Perception?
Authors: Xiaoxiao Sun
Nidham Gazagnadou
Vivek Sharma
Lingjuan Lyu
Hongdong Li
Liang Zheng
Venue: NeurIPS 2023 spotlight (2023)
Keywords: Privacy Assessment
Reconstructed Images
Evaluation Metrics
Human Perception
TL;DR: This paper assesses image quality metrics' faithfulness to human perception of privacy in reconstructed images and proposes SemSim, a learning-based measure with higher correlation to human judgment and generalization

Abstract: Hand-crafted image quality metrics, such as PSNR and SSIM, are commonly used to evaluate model privacy risk under reconstruction attacks. Under these metrics, reconstructed images that are determined to resemble the original one generally indicate more privacy leakage. Images determined as overall dissimilar, on the other hand, indicate higher robustness against attack. However, there is no guarantee that these metrics well reflect human opinions, which offers trustworthy judgement for model privacy leakage.  In this paper, we comprehensively study the faithfulness of these hand-crafted metrics to human perception of privacy information from the reconstructed images. On 5 datasets ranging from natural images, faces, to fine-grained classes, we use 4 existing attack methods to reconstruct images from many different classification models and, for each reconstructed image, we ask multiple human annotators to assess whether this image is recognizable. Our studies reveal that the hand-crafted metrics only have a weak correlation with the human evaluation of privacy leakage and that even these metrics themselves often contradict each other. These observations suggest risks of current metrics  in the community. To address this potential risk, we propose a learning-based measure called SemSim to evaluate the Semantic Similarity between the original and reconstructed images. SemSim is trained with a standard triplet loss, using an original image as an anchor, one of its recognizable reconstructed images as a positive sample, and an unrecognizable one as a negative. By training on human annotations, SemSim exhibits a greater reflection of privacy leakage on the semantic level. We show that SemSim has a significantly higher correlation with human judgment compared with existing metrics. Moreover, this strong correlation generalizes to unseen datasets, models and attack methods. We envision this work as a milestone for image quality evaluation closer to the human level. The project webpage can be accessed at https://sites.google.com/view/semsim.

Privacy Assessment on Reconstructed Images: Are Existing Evaluation Metrics Faithful to Human Perception?
Xiaoxiao Sun
Nidham Gazagnadou
Vivek Sharma Sony AI
Australian National University
Sony Al
xiaoxiao. sun@anu. wan.aan. edu.
nidham, gazagnadouOsony
vivelssharma@sony. com
Lingjuan Lyu E
Hongdong Lit
Liang Zheng Australian National University
Sony AI lingjuan, 1v@sony.c.. com
Australian National University
hongdong 1iiaanu.... edu.
liang zhengeanu. edu.
Abstract
Hand-crafted image quality metrics, such PSNR and SSIM, are commonly used to evaluate model privacy risk under reconstruction attacks. Under these metrics, reconstructed images that determined resemble the original indicate more privacy leakage. Images determined as overall dissimilar, on the other hand, indicate higher robustness against attack. However, there is no guarantce that these metrics well reflect human opinions, which offers trustworthy judgement for model privacy leakage. this paper, we comprehensively study the faithfulness of these hand-crafted metrics to human perception of privacy information from the reconstructed images. On 5 datasets ranging from natural images, faces, fine-grained classes, we use 4 existing attack methods to reconstruct images from many different classification models and, for each reconstructed image, we ask multiple human annotators to assess whether this image is recognizable. Our studies reveal that the hand-crafted metrics only have weak correlation with the human evaluation of privacy leakage and that even these metrics themselves often contradict each other. These observations suggest risks of current metries in the community. To address this potential risk, propose learning- based measure called SemSim to evaluate the Semantic Similarity between the original and reconstructed images. SemSim is trained with standard triplet loss, using an original image as an anchor, one of its recognizable reconstructed images positive sample, and unrecognizable human annotations, SemSim exhibits a greater reflection of privacy leakage on the semantic level. We show that SemSim has significantly higher correlation with human judgment compared with existing metrics. Moreover, this strong correlation generalizes unseen datasets, models and attack methods. We envision this work as a milestone for image quality evaluation closer to the human level. The project webpage can be accessed http/ttt///// //sp::/:///ttss.oooleee cwm/view/ /wwwsmmmmi
generally
as negative. By training on
Introduction
This paper studies the evaluation privacy risks of image classification models, with focus on reconstruction attacks 6 43 During inference, target classifier, reconstruction attack algorithm
37th Conference on Neural Information Processing Systems (NeurIPS 2023).

---

(A) original
reconstructed
original
reconstructed
original
reconstructed
metrics
PSNR
MSE
SSIM
(B) original
roconstructed
original
reconstructed
original
reconstructed
LPIPS
Human
x
Figure 1: Inconsistency between existing metries and human judgements on privacy information leakage. For each original image, we present two reconstructions produced by InvGrad [ Below the reconstructed images, each colored / corresponds corresponding metric evaluates the reconstruction to have more information leakage. In (A), according PSNR, MSE, SSIM and LPIPS, the first reconstructed image evaluated to have more privacy leakage 88 7 than the second one (i.e., the first one has higher PSNR, SSIM values, and lower MSE and LPIPS values). However, human annotators perceive the first image as having less privacy leakage, since they cannot recognise this recognition (in contrast to the second reconstruction, which recognizable and suggested to have more information leakage). Such inconsistency privacy assessment is our key observation and motivation. Moreover, we observe in (B) that even these metries themselves often disagree with each other.
different metric, indicating that the
and test set used. For each original test image, the attack algorithm intercepts gradients of the target model to obtain a reconstructed image [0). The evaluation objective to measure whether the reconstructed image leaks any private information of the original one.
In the literature, objective evaluation metrics [29] 25 such as peak signal-to-noise ratio (PSNR), mean squared error (MSE) and structural similarity index (SSIM) are commonly used. They measure the similarity between two images on the pixel-level. In common practice, the high similarity between the original and reconstructed image indicates good reconstruction attack, thus a more vulnerable classification model. Conversely, the low similarity between the two images means poor reconstruction, which is believed to indicate weak privacy risk.
However, it is often subject to human perception whether privacy is leaked or preserved. In Figure [ we show examples where hand-craft evaluation metrics, such as PSNR and SSIM, and CNN-feature- based metric learned perceptual image patch similarity (LPIPS) 39 give different judgments of privacy assessment reconstructed image that recognizable (privacy-leaked) by annotators evaluated better privacy- preserved by PSNR, SSIM, and LPIPS. In (B), sometimes some of these metrics provide consistent judgments with human annotators, but their evaluation accuracy is still unstable for different images.
reconstructed images from human perception. For example, in (A), the
In light of the above discussions, this paper raises the question: is model privacy preservation ability as measured by existing metrics faithful human perception? ? To answer this question, we conduct extensive experiments to study the correlation of model privacy preserving ability measured by human perception and existing evaluation metrics. Specifically, for each reconstructed image, we ask five independent annotators whether the reconstruction is recognizable. We use the average annotator responses over the test set as human perception privacy information leakage. On a wide range of scenarios (5 datasets of different concepts, many different classification models and 4 reconstruction attack methods), we find that there is only a weak correlation between human perception and existing metrics. It suggests that model determined as less vulnerable to reconstruction attacks by existing metrics may actually reveal more private information as judged by humans.
Recognizing such discrepancy, we propose new learnin-bbased metric, semantic similarity (Sem- Sim), to measure model vulnerability to reconstruction attack. Using binary human labels that indicate whether a reconstructed image is recognizable, we train a simple neural network with standard triplet loss function. For an unseen pair images, we extract their features from the neural network and compute their & distance, which is referred to as the SemSim score. If a model has low (resp. high) average SemSim score, it is considered to have a high (resp. low) risk of privacy leakage, We experimentally show models' vulnerability to reconstruction attack which is ranked by SemSim has much stronger correlation with human perception than existing metrics. Our main contributions are summarized below.

---

We find model privacy leakage against reconstruction attacks measured by existing metrics is often inconsistent with human perception.
We propose SemSim, a saarning-based and generalizable metric to assess model vulnerability reconstruction attack. Its strong correlation with human perception under various datasets, classifiers and attack methods demonstrates its effectiveness.
We collect human perception annotations on whether privacy is preserved for 5 datasets, 14 different architectures of each set, and reconstruction methods. These annotations will become valuable benchmarks for future study.
2
Related Work
Image quality and similarity metrics are usually used to indicate the performance of reconstruc-
tion attack approaches [42] 43 41 and also in privacy assessment 7 36 32 of methods against reconstruction attacks. These metrics can be broadly categorized into pixel-level and perceptual metrics. Pixel-level metrics, such as PSNR [4 29 and MSE [34], evaluate differences between pixel values of the original and reconstructed images 7 36 32), to reflect the degree of privacy leakage. Perceptual metrics, such as SSIM [35] and LPIPS (39] are designed to take into account the perceptual quality images for privacy leakage evaluation [3. This paper examines the effectiveness of these metrics in privacy leakage evaluation and finds they exhibit weak correlation with human annotations.
aim recover the training samples from the shared gradients. Phong et al. 26 show provable reconstruction feasibility on single neuron or single layer networks, which provide theoretical insights into this task. Wang et al. [33] propose an empirical approach to extract single image representations by inverting the gradients of 4-layer network. Meanwhile, Zhu et al. [43 formulate this attack as an optimization process in which the adversarial participant searches for optimal samples in the input space that can best match the gradients. They employed the L-BIGS [19] algorithm to implement this attack. Zhao et al. [4]] extend the approach with a label restoration step, hence improving speed of single image reconstruction. We focus on model privacy assessment against reconstruction attacks and evaluate different metrics using several attack methods.
Reconstruction attacks 43 g
Human perception annotations play an essential role in evaluating machine learning models 22 20 27 38)
Most public test sets, such as the ImageNet [2] dataset from the computer vision, are annotated by humans, allowing for conventional evaluation. Moreover, human feedback has been used to improve machine learning models, such as InstructGPT (24). In fields where human annotations were expensive to obtain, e,g., medical image analysis [37] and image generation [28], there is increasing evidence that the human judgements evaluation is valuable and offers new insights. In our paper, we consider the information leakage of reconstructed images
3 Privacy Assessment Metrics on Reconstructed Images: A Revisit
Pipeline of privacy assessment on the reconstructed images. As shown in Figure 2 the goal of evaluation compare privacy risks of series of K image classification models (Mr)
under
reconstruction attacks. The evaluation process simulates stealing data from gradients 43 41 Its input consists of an original image set X' {x; E IR mxx and reconstruction algorithm A used to attack models {M Given target model M whose parameter weights are denoted by W. its gradients VWx can be calculated using the original data X. The attack algorithm A applied to the target model M and its gradients VWx to obtain a set of reconstructed images denoted by X == A(M, VWx) (x,) Note that, A can access the gradients, but has not access to X. We can evaluate the privacy leakage of target model A over the original set of X' as follows:
ii= Ji=z, where N is the number images,
PL(M) infoLeea((X,,)
infol_ea((X',... A(M, VWx)),
(1)
where InfoLeak(:;) represents the amount of information leakage in reconstructed images. Therefore, it is important to have an effective metric for indicating Infol InfoLeak((,"" e.k(..))
Unless explicitly stated otherwise, the subscript of M is omitted when this does not create ambiguity.
3

---

Input
Output reconstructed Xs from Mr
original images I
which highest privacy rick?
reconstructed I, from M,
reconstructed set Xa from Mr
classifiers
M
reconstruction attack A
M
roconstruct
M
humnan
Infoleak(Xx. (0.0000) IInooLeak(X,3) (0.3333) InfoLeak(2,..)) (0.5000)
Infoleak(XX,)) X2) 1972 InfoLeea((X (9.3424) InfoLeak(Xt (10.1527)
Figure 2: Task definition: privacy leakage assessment on reconstructed images. Given K classification models MI, M2, ...y Mr against the image reconstruction attack (we use K = 3 as an example in this figure) on a set of original images X. For each model, we get set of reconstructed images. The main goal of privacy leakage assessment on reconstructed images is to measure whether semantic information of an original image, is still accessible. We can ask human annotators to evaluate whether they can recognize the image class and then average across the set images to obtain the overall human evaluation score of privacy leakage. In the existing literature, image quality metrics, such as PSNR, are used measure privacy leakage. Here, the evaluation of example images shows again that PSNR deviates from human evaluation.
Information leakage formulation. As introduced in Section information leakage is often as- similated reconstruction quality and based on distance between an original image x; and its reconstructed counterpart &; Under such pointwise metric, Infoleeak(;,)) of an image set X' and its reconstructed set & can be defined as:
InfoLeak (
Σ d(x;, Xz), N
1
2
where d can be a hand-crafted metric, such MSE 34 PSNR 14 29
SSIM B51 or model based, such as LPIPS [39.. Equation (2) averages the distances or similarities over all the original - re- constructed image pairs to obtain the information leakage score of the attacked model M. Apart from these, we can also use Fréchet Inception Distance (FID) [10]. It measures information leakage the distribution difference between original and reconstructed images: Infol.eak(N) X P) x FID(X, X).
4 Diagnosis of Existing Metrics and Our Proposal
4.1 Collecting human assessment of privacy leakage from reconstructed images
To evaluate whether reconstructed image leaks privacy, human perception offers very useful judgement. In the context of image recognition and face recognition, is to determine if the human can still recognize the reconstructed object face.
For image classification, given an image, we provide human annotators with an incomplete list of classes. For example, for the CIFAR-100 dataset, instead of providing annotators with a list of all the 100 classes which are hard to memorize, we provide them with list of the top-20 possible classes that includes the ground truth. We request annotators to annotate the class of a given image. the annotate thinks the images is "incomprehensible" (i.e., severely blurry) or the right class does not appear in the candidate list, then the annotation is 'none". We compare the human annotations between an image and its reconstructed version. If they are the same, privacy not preserved; otherwise, privacy is preserved. The annotation pipeline and more details of the annotation process are provided in the supplementary material.
For face recognition and fine-grained image recognition, because it is by nature very difficult for a human to assign a class label from 20 candidates, we give annotators two images time: an original image and its reconstruction. We then ask the annotator to tell whether the two images

---

0,9955
0.2960
0,5088
0.77
0.6865
0.00
9.00
0.70
0.7956
0.2210
-04111
-0.5344
0.00
280
260
ESIN
8.04
LPIPS
0.74
0.02
0
0.02
220
8.00
200
9.2
9.4
100
0.72
0.73 0.54
0.76
0.77
0.73 0.74 0,75
92 9.4
PSNR
LPPIS
I.PISS
PSNR
(A) correlation between existing metries themseelees
- 0.6703
0.3938
0.5127
0.7363
9.76
300
0,4725
-03978
-0.5604
53IM
LPPS
260
0.00
240
5.0
9.72
220
3.4
034
0.5
0.5
0.7 Human
0.8
0.3
0.4
0.5
0.6
0.7
0.5
0.5
0.70
0.8
0.5
0.5
0.7
0.1
6.9
0.4
0.5
0.6
0.7
0.6
Human
Human
Human
(B) correlation between the existing metries and human pereeption
Figure 3: Correlation between existing metries and their alignment with human perception
in measuring privacy risk. 14 classification models are attacked by InvGrad [8] on the CIFAR-
100 dataset. Each subfigure presents the correlation between the rankings of model privacy Ieakage obtained by two metrics. The correlation strength is measured by Spearman's 's rank correlation (p) [3] and Kendall's 's rank correlation (r) [16]. Between existing metries, (A) indicates that correlation is sometimes very weak. Furthermore, (B) indicates that the correlation between existing metrics and human perception is generally weak.
contain the same person or category. If yes, then privacy is not considered as preserved; otherwise, is. Note that in this procedure, to mitigate the potential bias of annotators, we also give reconstructed images that do not pair with the original image.
In all the above procedures, each image or image pair is labeled by 5 independent annotators. Binary labels, L.e., whether a reconstructed image is recognizable, are obtained via majority voting. this study, we deal with five datasets: CIFAR- 100, Caltech- 101, Imagenette and Celeb-A and Stanford Dog For each classification model being attacked, we annotate 600, 700, and 100 reconstructed images for the 100, Caltech-100 101, and the other three datasets, respectively.
4.2 Correlation analysis between human perception and existing metrics
Examples from Figure motivate to conduct a more comprehensive analysis of the inconsistency between human perception and existing metrics in terms of privacy leakage. To this end, for the reconstructed image set of 14 target models, we plot their privacy risk measured by various metrics against collected human labels in Figure 3 (B). We find that the correlation strength between human evaluation and existing metrics is relatively weak. For example, Kendall's rank correlation that measures rank consistency is only 0,2904 and 0.3978 for SSIM and LPIPS. Even in the best case, i.e FID vs human, correlation only moderate with 0.5604. It signifies that a model identified as more robust against reconstruction attacks based on existing metrics may actually be perceived as highly vulnerable according to human judgment when comparing different models.
The primary issue lies in the fact that existing metrics are computed on either a pixel-wise or patch- wise basis, without considering the semantic understanding of privacy leakage. As a result, these metrics fail to accurately capture the image semantics related to privacy risks, This problem motivates us design privacy-oriented metrics to better assess privacy leakage.
4.3 Proposed metric
To obtain a metric that more faithful to human perception, we propose SemSim, a learning-basee metric using human annotations as training data. The pipeline of SemSim is presented in Figure 4
Training. Using binary human labels whether reconstructed image is recognizable, we train simple neural network fo with a standard triplet loss function. We take the original image x; as an anchor
'The new annotated dataset is distributed under license CC BY-NC 4.0 1, which allows others to share, adapt. and build upon the dataset and restricts its use for non-commercial purposes.
5

---

human annotated data
training:
So
+ tripler
recog.
unreoog.
Test:
anchor
positive (recog.)
negative (unrecog.)
Semsimm
D.fo
Figure 4: Training and inference pipeline of SemSim. Feature extractor fe is trained on human- annotated images with a triplet loss [30]. An original image is used as anchor, and its reconstructions are split into positive (recognizable) and negative (unrecognizable) samples based on human anno- tations (Section[) 4). The goal is to minimize the anchor distance to positive samples and maximize that to negative ones. During inference, given an original image and its reconstruction, we use fo to extract their features and compute the E2 distance between the two features.
and split its reconstructions into positive x and negative x; samples based on human annotations. The loss function = Σ max[d(x; x d(x,, x + a, 0}, where x; is an original image and x (resp. x stands for one of its recognizable (resp. unrecognizable) reconstruction, and the margin. Thus, we obtain our neural network fo trained on
datasets.
Inferenee. During the evaluation, fe is used for extracting features for original and reconstructed images. We calculate the & distance between their feature vectors, that is SemSim (x, x) (o and then average this score over test set as the overall model performance score.
Key Observations. We believe SemSim captures semantic information, which plays a crucial role in privacy preservation. There are several key factors contributing to its effectiveness. (1) Being trained on human annotations enables SemSim to capture privacy leakage semantics better than metrics based on pixel-level similarity patch CNN features. (2) By utilizing a CNN model that extracts relevant higher-level features, SemSim captures visual information related to information leakage effectively. (3) It incorporates the relationship between the original image and recognizble/mneeonnizzbe reconstructions, improving its accuracy in assessing privacy leakage and providing better privacy assessment. SemSim has limitation in that it requires annotated data for training. While we show that it is very generalizable and can work better than existing metrics with limited training data (refer Figure 7. we prioritize our future work annotate more data for even improved generalization.
4.4 Discussions
Are there other ways than human perception to assess privacy leakage? Yes. We can use a classification model trained dataset that contains the same categories as the reconstruction set to classify the reconstructions. If the model accurately predicts the categories, indicates a potential privacy leakage. In our preliminary study, we used two models trained the CIFAR-100 dataset, achieving accuracies of 82% and 65% on the test set, respectively, for recognizing reconstructed images. By using their recognition accuracies indicators of privacy leakage, we obtained Kendall's rank correlation coefficients of 0.7023 and 0.5044 with human evaluation, respectively. These results are considered acceptable. However, there limitations to this approach. The classification model must be trained on a dataset that matches the categories of the task, and it needs to be accurate. These limitations affect the scope of this method. Nonetheless, exploring the use of classifiers evaluate privacy risk offers an alternative viewpoint to human perception, and it merits further investigation
Is privacy leakage reconstructed images binary problem? No. We simplify this problem by binarizing it. It can be continuous, where privacy information is leaked to a greater or lesser degree, depending on various factors such the task and the type and amount of data that is leaked.
How to define privacy leakage on reconstructed images in other vision tasks? The definition depends on the task context. For example, in object counting (23), privacy information can be defined as the number of objects. Therefore, for different tasks, the definition of privacy leakage should be carefully designed and accompanied by a tailored evaluation method.
Relationship between image quality and private leakage of reconstructed images. The relation- ship between the image quality of a reconstructed image and its information leakage is complex. While better image quality can indicate better reconstruction performance, it does not necessarily imply higher privacy leakage. Conversely, a reconstructed image with poor image quality can still contain private information, while an image with higher quality may preserve privacy better. There- fore, the relationship between image quality and privacy leakage is not always straightforward and

---

Figure 5: Sample annotation results. For each original image (leftmost column), its reconstructed images placed left to right by their PSNR values from large small. The red cross denotes that the human annotator fails to recognize the image. We observe that human evaluation is inconsistent with PSNR ranking, e.g., some images that are top-ranked, or equivalently determined as high quality by PSNR are actually not recognizable by humans.
requires careful consideration and evaluation. These discussions also encourage us explore new metrics that incorporate semantic-level information in order to better assess privacy leakage.
Limitation and potential improvement methods for Semsim. One limitation of SemSim is its potential performance decrease when faced with sigmificant distributional shifts (such as using on medical images). To address this limitation, adaptability of Semsim to a wider range of domain variations. Additionally, exploring other strategies, such incorporating local image regions and utilizing multi-valued annotated training data, could also be considered to further enhance the effectiveness of SemSim.
can annotate diverse types of data to enhance the
5 Experiments
Experimental Setups
Datasets. We evaluate using the CIFAR-100 18], Caltech-101 (5]. Imagenette 2 CelebA 21. and Stanford Dogs (17) datasets. The first three are for generic object recognition, CelebA is for face recognition, and Stanford Dogs is a fine-graing grained classification dataset.
Classification models. We use the following backbones: ResNet20, ResNet50, ResNet152 [9]. DenseNet and 8-layer CoveNet (7]. They were trained using different strategies, such as data aug- mentation gradients with Gaussian/Laplacian noise 43], and layer-wise pruning techniques [4]. In total, there are 70 different models. Details are provided in the supplementary material.
Reconstruction attack methods. We mainly InvGrad 8 In the ablation study, we evaluate SemSim using four additional attack methods, including DLG [43, CAFE (15), and GradAttack (13). Correlation strength measurements. We two rank correlation coefficients: Spearman's rank correlation (31] and Kendall's rank correlation т [66] to measure the consistency between
different metrics with human perception. Values of and r are between [-1, 1]. Being closer -1
1 indicates a stronger correlation, and O means no correlation. For clarity, the absolute values of and are provided in the paper.
Implementation Details
Classification model training. The training of all the models to be evaluated was conducted using the PyTorch framework. The details of the classifier training, such as the specific architectures and hyperparameters used for each model, are provided in the supplementary material, We perform model training with one RTX-2080TI GPU and a 16-core AMD Threadripper CPU 3.5Ghz. - SemSim model training. In the main evaluation, SemSim is trained using a learning rate of 0. and batch size 128 on the ResNet50 architecture for 200 epochs. We use leave-one-out evaluation on the 5 datascts. Some examples of the annotation data are provided in Figure I and Figure
'Imagenette is a subset of 10 easily classified classes from ImageNet. https://ww..tnsorff/ datasets/catalog/ hww.o///ggmmggeeeteeemm
7

---

Table 1: Comparison of different metrics different datasets. For each metric, rank the 14 models and compute the correlation with rankings made by human assessment. For each test set (Column 1), SemSim is trained on the combination of the rest four datasets. Here, InvGrad [8] attack is used. |ø] and |r] are reported. SemSim has a much stronger correlation with human annotations.
Datasets
Metrics
PSNR
MSE
SSIM
LPIPS
FID
SemSim
Spearman's s p
0.6703
0.6176
0.3939
0.5127
0.7363
0.8637
CIFAR-I00
Kendall's [-|
0.4725
0.4286
0.2904
0.3978
0.5604
0.7143
Spearman's 's p
0.6970
0.7349
0.7218
0.5127
0.2242
0,8182
Caltech-101
Kendall's |r|
0,5556
0.5525
0.5244
0.4072
0.1556
0.6889
Spearman's |p
0.5382
0,6395
0.6433
0.6539
0.4791
0.8257
Imagenette
Kendall's s |r]
0.4349
0.5525
0,5108
0,5922
0.4252
0,7012
Spearman's ' [p|
0.7495
0.7349
0,6846
0.5824
0.1516
0,8263
CelebA
Kendall's |r|
0.5604
0.5525
0.5264
0.4505
0.0989
0.6923
Spearman's p
0.4023
0.3968
0,4782
0,5031
0.3969
0,7120
Stanford Dogs
Kendall's ' [r]
0.3537
0.2743
0.3048
0.3929
0.3196
0,5938
Table 2: Comparison of different metries under different attacks on the CIFAR-100 dataset. SemSim trained using human annotations obtained through the InvGrad 8 attack method and evaluated on different attack methods listed in the table.
Attacks
Metries
PSNR
MSE
SSIM
LPIPS
FID)
SemSim
Spearman's p
0.6515
0.6367
0.4069
0.5477
0.7268
0.8749
DLG 43
Kendall's s |r|
0.4857
0.4174
0.2858
0.4294
0.5237
0.7342
Spearman s
0.7104
0.6916
0.5870
0.6793
0.6925
0,8864
CAFE 15
Kendall's |
0.5392
0.4259
0.3318
0.4762
0.4735
0.7510
Spearman's
0.6831
0.6944
0.5753
0.6841
0.7204
0,8437
GradAttack
Kendall's |r]
0.4943
0.4980
0,3495
0,4531
0.4819
0.7260
5.1 Main Evaluation
Inconsistency between existing metrics and human perception: more results. On each of the five test sets, rank the 14 models according each of the existing metrics as well human perception. The model ranking of each metric is correlated with that from human assessment. We find that PSNR, MSE, SSIM, LPIPS, and FID do not have high correlation with human assessment. The worst performing metric FID: Kendal's |T] is only 0516. 0.4252, 0.0989, and 0.3196 between FID and human perception, on the four test sets, respectively. While the rest four metrics exhibit stronger correlation than FID, Kendall's |r| is generally around 0.5, which is considered only moderate.
Moreover, from Figure 3 we find that the correlation between existing metrics themselves is often weak. For example, In Figure 3 right, Kendall's | is only 0.2904 between PSNR and LPIPS. This contradiction also exists between PSNR and LPIPS and others. The above results advocate the study of new metrics that are privacy oriented.
Comparing SemSim with existing metrics in terms of faithfulness to human perception. We utilize SemSim to rank the models and examine correlation with the ranking based on human perception, as shown in Table We make two key observations.
First, SemSim exhibits much stronger correlation with human perception. On the five test sets, Kendall's 's | is 0.7 14,, 0.6889, 0.70/2 0.6923, and 0.5938, respectively, which 0.2418, 2,18,10 0.133,, 0.2663. 0. 1319 and 0.2401 higher than PSNR, for example. The above results suggest the risks of current metrics in the community and advocate the proposed learning-based, privacy-orienedd metric.
Second, on Stanford Dogs, while SemSim is still much more faithful to human perception than other metrics, the overall correlation lower than other datasets. Because dog species are hard to recognize, more noise was introduced to human annotation and thus to the ranking results and correlation. We speculate that fine-ge rree-graindd datasets are harder for privacy interception through reconstruction: humans themselves will find it hard recognize the private content.
Generalization ability of SemSim. In Table we adopt a leave-one-out setup, where SemSim is trained on four datasets and tested on the fifth dataset. Moreover, for each dataset, the model
8

---

PSNK
Figure 6: Comparing the ranking of reconstruction images using PSNR and SemSim. From the visualizations, we can observe that PSNR exhibits some inconsistencies with human perception, while SemSim consistently aligns with the judgments human annotators. In the two examples, SemSim correctly ranks all the images with noticeable information leakage (including a laptop or chair) before the ones without or with less information leakage (that are unrecognizable), However, the rankings provided by PSNR are inaccurate for some images.
0.8
0.9
0.8
triplet
contrastive
Kendall's
0.7
Somsim
correlation
0.8
Kendall's
0.7
FID
PSNR
0.6
0.7
0.6
0.5
LeNet
ReaNet18
ReoNet50
DenseNot
ConvNet8
0.5
LeNet
ResNet18
ResNer50
DenseNer
ConvNet8
1000
1500
number of samples
impact training size
impact of backbone
(C) impact loss function
Figure 7: Analysis of SemSim. Evaluating the impact of (A) size of human-annotated training
data, (B) variations of SemSim backbones, and (C) different loss functions for SemSim training. Experiments are conducted on the CIFAR- 100 dataset.
architectures are different. For example, when using CelebA test set, the tested target models are ResNet50 and DesNet etc, while target models in training are Resnet20, 8-layer CoveNet and ResNet152 etc. As such, the superior results in Table I demonstrate the generalization ability of SemSim for test sets and model architectures.
SemSim to evaluate model vulnerability to unseen attacks. Results provided Table where SemSim is trained using human annotations obtained through the InvGrad [ attack method and evaluated other attack methods such DLG, CAFE, and GradAttack. Remarkably, we consistently observe higher correlation between SemSim and human perception compared to existing metries. On the CIFAR- 100 dataset, observe significant improvements in Kendall's |T] of 0.7342, 0.7510, and 0.7260, respectively, for DLG, CAFE, and GradAttack. These findings demonstrate the robustness of SemSim in capturing the privacy leakage of reconstructed images across different reconstruction attacks.
Furthermore,
Visualization results of SemSim. Figure presents two examples of ranking reconstruction images using PSNR and SemSim. both cases, SemSim outperforms PSNR and provides better results.
5.2 Further Analysis
Impact of the number of human annotations on SemSim training. To evaluate this impact, we use the CIFAR 100 dataset for testing and randomly select rest four datasets to train SemSim. Results are shown T (A). observe a correlation drop between SemSim and human perception as the number of training samples decreases. However, even with as few as 50 training samples (each samples includes 14 reconstructed images), SemSim outperforms existing metrics like PSNR and FID.
training samples from

---

Impact of different backbones for SemSim. As mentioned in the implementation details, SemSim uses simple ResNet50 network. Here, we try several different opinions such LeNet and ResNet18, and present their correlation with human perception in Figure 7 (B). We show that even a simple LeNet model can achieve [r] scores higher than 0.65, surpassing the best score of 0.5604 obtained by FID. Moreover, we observe that there is correlation between the complexity of the backbone architectures and the performance of SemSim. This indicates that utilizing more advanced and sophisticated backbone models may be able to further enhance SemSim to capture and represent visual information, leading to improved evaluation of privacy leakage in reconstructed images.
Impact of other loss functions for SemSim training, We further experiment with different loss functions and hyperparameters, including the contrastive loss and the triplet loss (where we set a 1 in experiments). From the results shown in Figure [ (C), we observe that the triplet loss shows comparable correlation strength the contrastive loss in relation to human assessment.
Conclusion
This paper investigates the suitability of existing evaluation metrics when privacy is leaked by reconstruction attack. We first collect comprehensive human perception annotations on whether a reconstructed image leaks information from the original image. We find that model vulnerability to such attacks measured by existing metrics such as PSNR has a relatively weak correlation with human perception, which poses potential risk to the community. We then propose SemSim trained on human annotations to address this problem. On five test sets, we show that SemSim has much stronger faithfulness human perception than existing metrics. Such faithfulness remains strong when SemSim used for different model architectures, test categories, and attack methods, thus validating its effectiveness. In future work, we will collect human perception labels from wider source of datasets and train a more generalizable metric for privacy leakage assessment.
Acknowledgement
This research was funded by Sony AI We also gratefully acknowledge partial support from the ARC Discovery Early Career Researcher Award (DE200101283) awarded to Liang Zheng. the ARC Discovery Project (DP210102801), and the ARC Discovery Grant (Grant ID: DP220100800) awarded to Hongdong Li.
References
[1] Chen Chen, Lingjuan Lyu, Han Yu, and Gang Chen. Practical attribute reconstruction attack against federated learning. IEEE Transactions Big Data, 2022.
[2] Jia Deng. Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pages 248 255, 2009.
[3] DC Dowson and BV666017 Landau. The fréchet distance between multivariate normal distributions. Journal of multivariate analysis, 12(3):450-455, 1982.
[4] Aritra Dutta, El Houcine Bergou, Ahmed M Abdelmoniem, Chen-Yu Yu Ho, Atal Narayan Sahu, Marco Canini, and Panos Kalnis. On the discrepancy between the theoretical analysis and practical implementations of compressed communication for distributed deep learning. Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 3817-3824, 2020.
[5] Li Fei-Fei, Rob Fergus, and Pictro Perona. Learning generative visual models from few training exam- ples: An incremental baycsian approach tested on 101 object categories. Computer Vision and Patten Recognition Workshop, 2004.
[6] Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model inversion attacks that exploit confidence information and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC conference on computer and communications security, pages 1322-1333, 2015.
[7] Wei Gao, Shangwei Guo, Tianwei Zhang, Han Qiu, Yonggang Wen, and Yang Liu. collaborative learning with automatic transformation search. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 2021.
10

---

[8] Jonas Geiping, Hartmut Bauermeister. Hannah Dröge, and Michael Moeller. Inverting gradients-how easy is it to break privacy in lederated learning? Advances in Neural Information Processing Systems, 2020.
[9] Kaiming He, Xiangyu Zhang. Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778. 2016.
[10] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge to local nash equilibrium. In Advances in neural information processing systems, 2017.
[11] Alain Hore and Djemel Ziou. Image quality metrics: Psar ssim. In 2010 20th international conference on pattern recognition, pages 2366- 2366-2369, 2010.
[12] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. pages 4700-4708, 2017.
[13] Yangsibo Huang. Samyak Gupta, Zhao Song, Kai Li, and Sanjeev Arora. Evaluating gradient inversion attacks and defenses federated learning. Advances Neural Information Processing Syxtems, 34: 7232-7241, 2021.
[14] Quan Huynh-Thu and Mohammed Ghanbari. Scope of validity of psnr in image/video quality assessment. Electronics letters 44013)8000-81,, 2008.
[15] Xiao Jin, Pin-Yu Chen, Chia- Yi Hsu, Chia-Mu Yu, and Tianyi Chen. Cafe: vertical federated learning. Advances in Neural Information Processing Systems, 34:994-1006, 2021.
Catastrophic data leakage in
[16] Maurice Kendall. A new measure of rank correlation. Biometrika, 30(1/2)88--9,, 1938.
[17] Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, and Fei-Fei Li. Novel dataset for fine- fine-grained image categorization: Stanford dogs. In Proc. CVPR workshop on fine-grainet visual categorization (FGVC), volume 2, 2011.
[18] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers features from tiny images. 2009.
[19] Dong C Liu and Jorge Nocedal, On the limited memory bigs method for large scale optimization. Mathematical programming, 45(1-3):503-522 1989.
[20] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach, arXiv preprint 1iv: 16600.0..1...2220000 2019.
[21] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), 2015.
[22] David Martin, Charless Fowlkes, Doron Tal, and Jitendra Malik. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, volume 2, pages 2001.
[23] Daniel Onoro-Rubio and Roberto J López-Sastre. Towards perspective-free object counting with deep learning. In Computer Vision-F.CCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11 11-14, 2016, Proceedings, Part VII 14, pages 615-629, 2016
[24] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730-27744, 2022.
[25] Marius Pedersen, Jon Yngve Hardeberg, al. Full-reference image quality metries: Classification and evaluation. Foundations and TrendsS Computer Graphics and Vision 7()::180, 2012.
[26] Le Trieu Phong, Yoshinori Aono, Takuya Hayashi, Lihua Wang, and Shiho Moriai. Privacy-preservinn deep learning: Revisited and enhanced. In Applications and Techniques in Information Security: 8th International Conference, ATIS 2017, Auckland, New Zealand, July 6-7, 2017, Proceedings, pages 100-110, 2017.
[27] Olga Russakovsky, Jia Deng. Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang. Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115:211-25 252, 2015.

---

[28] Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. Advances in neural information processing systems, 29, 2016.
[29]
Umme Sara, Morium Akter, and Mohammad Shorif Uddin. Image quality assessment through fsim, ssim, mse and psnr comparative study. Journal of Computer and Communicaaiinss 7(3):8 18, 2019.
[30] Florian Schroff,, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 815 823, 2015.
[31] Charles Spearman. The proof and measurement of association between two things. 1961.
[32] Jingwei Sun, Ang Li, Binghui Wang, Huanrui Yang, Hai Li, and Yiran Chen. Soteria: Provable defense against privacy leakage in federated learning from representation perspective. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 9311-99111991111 9319, 2021.
[33] Zhibo Wang, Mengkai Song. Zhifei Zhang. Yang Song, Qian Wang, and Hairong Qi. Beyond inferring class representatives: User-level privacy leakage from federated learning. In IEEE INFOCÓM 2019-IEEE conference on computer communications, pages 2512-2520, 2019.
[34] Zhou Wang and Alan C Bovik. A universal image quality index. IEEE signal processing letters, 9(3):
81 81-84, 2002.
[35] Zhou Wang, Alan Bovik, Hamid R Sheikh, and Eero Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600-612, 2004.
[36] Taihong Xiao, Yi-Hsuan Tsai, Kihyuk Sohn, Manmohan Chandraker, and Ming- Hsuan Yang. Adversarial learning of privacy-preservigg and lask-oriented representations. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 12434-12444 2020.
[37] Jufeng Yang. Xiaoxiao Sun, Jie I Liang, and Paul I. Rosin. Clinical skin lesion diagnosis using representations inspired by dermatologist criteria. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pages 1258-1266, 2018
[38] Yi Zeng, Minzhou Pan, Himanshu Jahagirdar, Ming Jin, Lingjuan Lyu, and Ruoxi Jia. (Meta-Sift): How to sift out a clean subset in the presence of data poisoning? In 32nd USENIX Security Symposium (USENIX Security 23), pages (667-1684, 2023.
[39] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechuman, and Oliver Wang. The unreasonable effectiveness deep features as perceptual metric. Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586-595, 2018.
[40] Rui Zhang, Song Guo, Junxiao Wang, Xin Xie, and Dacheng Tao. survey on gradient inversion: Attacks. defenses and future directions. arXiv preprint arXiv:2206.07284, 2022.
[41] Zhao, Konda Reddy Mopuri, and Hakan Bilen. idlg: Improved deep leakage from gradients. arXiv preprint arXiv:2001.022 0220000000.000 2020.
[42] Junyi Zhu and Matthew Blaschko. R-gap: Recursive gradient attack on privacy. Proceedings ICLR 2021. 2021.
[43] Ligeng Zhu, Zhijian Liu, and Song Han. Deep leakage from gradients, Advances in neural information processing systems, 32, 2019.
12

---

A Data Annotation
In Section 4.1, we briefly introduced how humans annotate the reconstructed images for different datasets. In the supplementary material, we have included graphical user interface (GUI) that was utilized by the annotators. Figure g displays the GUI, where (A) and (B) were specifically designed for annotating different datasets.
Labeling options
(1-4) hoider
Lobele
ingac:
C canner boet
Image in
O banal
O boas
O butwef
O aGD
C lobole
WAL_OO
+ gwtstt
Wrddwr_hha
wildi
O howosed
sunfowar
accordoon
(A) image classification
imuge annotated:
original
Image:
(B) face recognition and fine-grained image classification
Figure 8: Graphical user interface (GUI) used in our human annodation process. For (A) image classification, we ask annotators to give category the reconstructed image. In (B) face recognition and fine-grained classification, we ask annotators to tell whether the original image and its reconstruction have the same different identity / category.
To minimize the influence of subjective bias, the reconstructed image can be correctly labeled. Specifically, for CIFAR-100 100, Caltech- 101, and Imagenette, provide up to 20 candidate categories and see if the annotators correctly recognize the reconstructed image; for more difficult tasks like face recognition and fine-grained classification (Celeb-A and Stanford Dogs), we give both the original and the reconstructed images and ask the annotator they are the same identity or species.
relatively objective formulation: whether
B Impact of margin value in the triplet loss on SemSim
The effect of the margin parameter in the triplet loss on the performance of SemSim is depicted in to value close both Spearman's rank correlation
Figure It can be observed that when (p) and Kendall's rank correlation (r) coefficients yield better results compared to other values, on CIFAR-101 100 and Caltech-100 101 datasets. We think there are two potential reasons for this observation. Firstly. if the value of a is too small, the model may struggle to effectively learn the discriminative features that distinguish positive (recognizable reconstructed images) and negative (unrecognizable reconstructed images) samples. On the other hand, if a is set to value that is too large, the model may become excessively confident in distinguishing between positive and negative samples. However, this lead to convergence challenges, the loss function may have difficulty approaching 0.
13

---

0.9
0.9
0.8
correlation
correlation
0.7
P
0.6
P
0.5
0.5
1.5
(A) impact of results CIFAR-100
(B) impact of results Caltech-101
Figure 9: Impact of SemSim when testing on (A) CIFAR-100 and (B) Caltech-101. The margin value a used the triplet loss to ensure that negative samples are kept far apart. When evaluating the reconstructed images of CIFAR-I00 or Caltech-101, 101, we trained the ResNet50 model on the four datasets (excluding CIFAR-100 or Caltech-101) using the triplet loss. The training process involved utilizing different values of the margin parameter for each dataset.
Table Details classification models. On each test set, we have two backbones trained without (vanilla, 2nd column and with different strategies, such using data augmentation (3 6th columns) and existing defence methods (7th 8th columns).
Datasets
Models
ResNet200
CIFAR-100
CaveNer8
ResNet20
Caltech-101
DenseNet
Random- ResizedCrup
Imagenetie
ResNei50
ResNei1
Random-
& Invert
TranslateX
& Autocontrast
ranslate
defense Gaussian (10-
defense Pruning (70%)
ResNet20
HorizontalFii
& missltee
& Autocontrast
CelebA
DenseNet
ReaNet30
Stanford Dogs
RssNet152
In our experiments, we set a to 1. However, we acknowledge that there is potential for improved performance by carefully selecting the optimal value of for different datasets.
C Classification models and training details
We conducted experiments using five datasets, CIFAR-100 18), Caltech101 (5), CelebA [2]] ImageNette [2]. and Stanford dogs [17]. In evaluation process, considered 14 classification models for each set. Table provides detailed information about these models. They were trained using stochastic gradient descent (SGD) the optimizer, with learning of 0.1.
D Metrics for reconstruction quality
Mean squared error. Assuming xr H E Raxm are two images to compare, the mean squared enror (MSE) is given by,
MSE .) = (1/mn)
Dzy Bzz)?.
The value of MSE is between 0 and +oo. The lower MSE is, the closer two images are.
Peak-Signal-to-Noise ratio. The Peak-Signal-to-Noiss ratio (PSNR) is widely used in image quality assessment, which measures the ratio between the maximal power of signal and its noise. Its value, expressed in dB, is given by:
MAX, MSE(2, 3)
PSNR x. 7) = 20 log
(4)
where MAX, is the maximal value in the image æ (often replaced 255 for int8 images).
14

---

SSIM. Unlike PSNR, the structural similarity index measure (SSIM) B5 is a perception-base based metric as it was designed to take into account characteristics of the human vision system through three metrics: luminance, contrast and structure of the image. It is shown that there is an analytical link between PSNR and SSIM and that it is often possible to predict one from the other for controlled perturbations (Gaussian blur, additive Gaussian noise and jpeg compressions) [1] The above three metrics compute a pixel-wise distance between both images which is very limited when we assess semantic content of an image such as privacy leakage.
LPIPS. LPIPS 331, which stands for learned perceptual image patch similarity, a perceptual metric based on neural network aiming at correlating better with perceptual judgments. The authors take inspiration from neuroscience findings, where their model compares the activations between two images as neurons in a human cortex would. As explained in its torchmetric documentation low LPIPS score indicates high similarity. Thus, in the context of privacy assessment, low L.PPSS values for an original image and its reconstruction suggest high privacy leakage [3]
Fréchet Inception Distance. Aside from LPIPS and the aforementioned hand-crafted metrics calculated at the image level, our works also uses Fréchet inception distance (FID) [IO] to measure information leakage. FID commonly used to evaluate the domain gap between two distributions, where higher values suggest larger domain gap. For example, FID is extensively used to evaluate the quality of images generated by generative adversarial networks (GANs) [IO], by computing the distribution difference between real and generated images. In this paper, FID may reflect the difference between the original and reconstructed image distributions to reflect privacy leakage. As opposed to the pointwise metrics, FID is computed directly on image sets: Infol.eak(2' R) (II((X,))
As defined in 100 3) given two Gaussian distributions with mean and covariance (m, C), resp. (m, C), FID is given by:
FID((m, C)2 (m, C)) = |mm ml 2
-T Tr (---T((+--- (C+C-2(C0)1))
(5)
Its evaluation on finite sets X' and X follows verbatim by computing their empirical mean and covariance matrix. The value of FID is between and +oo. The lower the FII) value is, the closer two distributions are.
Relationship between l and cosine similarity.
The f norm can be used as tool to measure the distance between vectors, often embeddings of images like those produced by our SimSem model:
f v) = l v122
(6)
The issue with distances to estimate the similarity between vectors is that they are only bounded below by zero (when u v). This makes hard to set a threshold above which vectors u and v can be considered dissimilar. Thus, cosine similarity is often preferred to εg distance as its values belong to the interval [-1,1]. indicating proportional vectors and -1 vectors of opposite directions. Let u, v be normalized vectors, then the relationship between cosine similarity and lz norm is
fz(u,v) = v 2(1 cossim uu, v))
E
More Discussions
How much extra effort needs to be paid to extend the current approach? The effort required to extend the current method to other tasks also depends on the nature of the tasks. If we evaluate privacy leakage for the counting task, it would be useful to ask human annotators to count the number of objects in the reconstructed image: if it equals the number of objects in the original image. then privacy may be considered leaked. For this particular counting task, speculate that manageable efforts will be needed to extend our current approach. On the other hand, tasks that need specialized or professional annotations will likely require more effort, such as medical image understanding.
hnttps://torccmeerricc r rewdteedocs. http::/:tt...att.....o.//ooo//ea _imaee____________ wwwichhssmilaaitt..oom html
15


Reviews:
[{"review_id": "YgfrS5P236", "reviewer": "Reviewer_AN5f", "summary": "This paper studies the faithfulness of hand-crafted metrics to human perception of privacy information from the reconstructed images, and discovers hand-crafted metrics only have a weak correlation with the human evaluation of privacy leakage. A learning-based measure called SemSim is proposed to evaluate the Semantic Similarity between the original and reconstructed images.", "questions": "- Can we just apply the original classifier on the reconstructed images as a metric for privacy leakage?\n- According to Table 1, how to explain PSNR/MSE is sometimes better than LPIPS/FID, which is used more widely to measure image generation quality?\n", "limitations": "Important details missing for human study, which is centric to the learned metric.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "+ a comprehensive comparison of traditional metrics for privacy leakage of reconstructed image\n+ proposes a learning-based metric with human ratings\n+ improves over all existing image quality metrics\n", "weaknesses": "- The definition of privacy is based on image content classification, which may miss important privacy attributes in local image regions.\n- The details of human rating collection procedure are needed to ensure no subjective bias in the metric training. For example, who are the people giving the rating, how many participants, what questions are asked to the participant.\n- The metric learning method is very simple.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for many insightful comments. We answer the questions in what follows. Please let us know if further clarification is needed.\n\n**Q1: Can we just apply the original classifier on the reconstructed images as a metric for privacy leakage?**\n\nYes, and we have discussed this possible method in Lines 183-193. This method has some limitations. First, it requires the classifier to be trained on a dataset that has the same categories with the task at hand. Second, the classifier accuracy significantly impacts the effectiveness of this approach. These limitations constrain the scope and effectiveness of using the original classifier for privacy leakage measurement. We will add more discussions about this method to highlight its potential benefits and inherent constraints in the revised paper.\n\n\n**Q2: According to Table 1, how to explain PSNR/MSE is sometimes better than LPIPS/FID, which is used more widely to measure image generation quality?**\n\nGreat question. Table 1 is on privacy leakage assessment, which is different from measuring image generation quality. In Lines 201-208 we discussed the connections and differences between these two evaluation problems. Due to the difference, although LPIPS and FID are widely used to **measure image generation quality**, they are not necessarily better than PSNR/MSE in **measuring privacy leakage**. Some visual examples of using LPIPS, PSNR and MSE for privacy leakage assessment are provided in Fig. 1 in the submission: in some cases LPIPS is consistent with human opinion while in others PSNR/MSE is more aligned. \n\n**Q3: The definition of privacy is based on image content classification, which may miss important privacy attributes in local image regions.** \n\nWe agree that image content classification may miss important privacy attributes in local regions. It would be an interesting direction for improvement, along with a few other suggested ways (please refer to our reply in \"general response to all reviewers\" for more discussions).  \n\nNevertheless, as an early attempt to define privacy using image semantics, our current definition based on global image content has proven to be more faithful to human perception than existing metrics in evaluating privacy leakage.  We will add these discussions to our paper. \n\n\n**Q4: The details of human rating collection procedure are needed to ensure no subjective bias in the metric training.**\n\nThanks for the kind suggestion. In Section 4.1, we detailed the number of participants and the annotation procedure. Specifically, each image or image pair is annotated by 5 independent annotators randomly selected from a pool of 23 annotators. These annotators are from various backgrounds, including college students, housewives, and part-time workers from different industries. \n\nTo further minimize the influence of subjective bias, we use a relatively objective formulation: whether the reconstructed image can be correctly labeled. Specifically, for CIFAR-100, Caltech-101, and Imagenette, we provide up to 20 candidate categories and see if the annotators can correctly recognize the reconstructed image; for more difficult tasks like face recognition and fine-grained classification (Celeb-A and Stanford Dogs), we give both the original and the reconstructed images and ask the annotator if they are of the same identity or species. \n\nDetailed graphical user interfaces (GUI) for annotation are shown in Section A of the Supplementary Material.  \n\nWe will add these details in the revised text.\n\n**Q5: The metric learning method is very simple.**\n\nWe agree that the proposed metric is simple and feel that being simple is an advantage. In the comprehensive evaluation, this simple metric outperforms existing metrics *w.r.t* faithfulness to human perception. This conclusion would benefit future explorations in semantic-aware privacy assessment, while the metric itself will create opportunities for further refinement and study. "}, {"review_id": "8njK2NQqZ2", "reviewer": "Reviewer_oRwe", "summary": "The paper establish a system for human to annotate whether a reconstruction (from privacy attacks) is recognizable (thus breaches privacy). Based on human annotation results, a new learning-based metric is proposed, which shows significantly better alignment and faithfulness with human perception.", "questions": "1. The authors claim that the metric generalizes well among several datasets. I am curious whether it can still generalize well enough when the distribution change a lot, for example, on medical imaging datasets (However as you said, the privacy definition of such tasks could be different from a professional view, for example, a doctor may no longer think patch-level similarity is important in privacy for medical images). Readers could be happy to see more discussions on how the definition of \"privacy\" could change for other cases, and figure out how many extra efforts need to be paid to extend the current approach (For example changing ordinary human labelers to professionals like physicians)", "limitations": "1. If we want to make the evaluation metric more robust and more generalizable, in the future, we may consider scaling up the annotations to more images and more datasets. Even if we don't scale up, for new tasks or new datasets, we may need to go through the annoation from scratch. In other words, the scalability of the system is somewhat limited.\n\n2. The metric is designed to evaluate reconstruction similarity. However, if such an evaluation metric is open-sourced and accessible to everyone, attackers may use this metric to train an even stronger privacy attack model. The risk needs to be further discussed and carefully handled.", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1. High originality and novelty. To my knowledge, the work is the first to utilize human annotation to address the faithfulness of evaluation metrics in privacy attacks. \n2. Meaningful for the machine learning privacy community. In terms of breaching privacy, it is highly important and meaningful to align the results with human evaluations. This area currently lacks such faithful and aligned metrics to evaluate whether a privacy attack is indeed successful.\n3. Paper written in good quality: motivation is well-depicted, the approach is described in good clarity. The figures give clear and convincing illustrations.", "weaknesses": "No obvious weakness, good work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for the constructive comments. We address the concerns raised by the reviewer below. Please let us know if further clarification is needed.\n\n**Q1: Can the proposed method generalize well enough when the distribution changed a lot, such as medical images?** \n\nGreat question. The proposed method is shown to perform well on a diverse range of tasks such as generic object recognition, face recognition, and fine-grained classification. For medical images, it is not feasible for us to conduct experiment in a short time to verify the effectiveness of SemSim, because expert annotations are needed to judge whether a reconstructed image still exhibit certain medical conditions. In our best guess, SemSim would be less effective on medical images due to the very different styles (*e.g.,* X-rays and ultrasound image). We discussed potential strategies to improve SemSim's flexibility to broader domain variations through annotating diverse data types (please refer to Lines 177-181), which we will investigate in future work. These discussions will be incorporated into the revised paper. \n\n**Q2: Provide more discussion on 1) how the definition of \"privacy\" could change for other cases, and 2) how much extra efforts need to be paid to extend the current approach *e.g.,* considering professionals for annotation**\n\nThank you for raising these points. In Lines 197-200, we discussed that the definition of privacy may vary in different tasks. For example, for object counting, privacy information may reside in the number of objects instead of individual objects themselves. We will enhance this section by adding insights from our reply to Q1.\n\nThe efforts required to extend the current method to other tasks also depend on the nature of the tasks. If we evaluate privacy leakage for the counting task, it would be useful to ask human annotators to count the number of objects in the reconstructed image: if it equals the number of objects in the original image, then privacy may be considered leaked. For this particular counting task, we speculate that manageable efforts will be needed to extend our current approach. On the other hand, tasks that need specialized or professional annotations will likely require more efforts, such as medical image understanding. These discussions will be integrated into the paper. \n\n**Q3: Scaling up the annotations to more images and datasets.**\n\nGreat suggestion. In Lines 179-181 of the submission, we recognize the importance of scaling up data annotation to more images and datasets and will do so in our future investigation.\n\n**Q4: Risk of open source code.**\n\nInteresting comment. By open-sourcing the metric and training data, we intend to promote transparency and collaboration. We agree on the mentioned risk. But we would like to point out that SemSim will also allow for better development of defence mechanisms and privacy-preserving models. In light of this comment, we will consider releasing SemSim under licenses that only allow for academic uses."}, {"review_id": "Zsufs6E5pa", "reviewer": "Reviewer_j2WQ", "summary": "This paper evaluates the limitations of existing hand-crafted image quality metrics in indicating privacy leakage in reconstructed images. The findings reveal weak correlation and contradictions among these metrics and with human perception, highlighting the risks of solely relying on them. To address this issue, the authors propose SemSim, a learning-based measure that assesses semantic similarity between original and reconstructed images. SemSim shows a higher correlation with human judgment compared to existing metrics when evaluating privacy leakage risk across various models, datasets, and attack methods. The contributions of this paper include valuable insights for privacy leakage assessment and the introduction of SemSim as a more reliable metric.", "questions": "Please address the above weaknesses.", "limitations": "The authors have discussed the limitations, e.g. the requirement of training data.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "\n-\tComprehensive study: The paper conducts a comprehensive evaluation of hand-crafted image quality metrics in privacy leakage assessment across multiple datasets and models, providing valuable insights into privacy assessment for reconstructed images.\n\n-\tClear problem statement: The paper effectively shows the weak correlation between existing metrics and human perception of privacy leakage, highlighting the significance of the problem in privacy assessment.\n\n-\tProposed learning-based measure: SemSim, which considers the semantic similarity between the original and rebuilt images, is useful for a more precise assessment of privacy leakage. In comparison to other metrics, SemSim shows a higher association with human judgement when assessing privacy leaks. \n\n-\tGood results: The authors demonstrate the potential generalizability of SemSim's correlation with human judgment on different datasets.\n\n-\tNew datasets: This paper provides new datasets for further research in privacy assessment.\n\n-\tInsightful Discussion: The paper provides insightful discussions on privacy leakage in reconstructed images.\n", "weaknesses": "\n-\tLack of details about existing metric FID: Providing more information about the specific models used for feature extraction in the FID metric can enhance reproducibility and clarity.\n\n-\tDomain generalization ability of the proposed metric should be discussed, such as only having one set of reconstructed images for training. It is important for the application of proposed measure in real world.\n\n-\tL2 distance is used by Semsim. What will happen to Semsim's performance if alternative distance measures are used? Ablation study should be provided.\n\n-\tTypos: The paper has a few typos, such as in line 241 with the value \"0-.0989,\" which should be corrected.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful review and valuable feedback. Below we address your concerns. Please let us know if further clarification is needed.\n\n**Q1: Details of FID.**\n\nIn lines 118-120 of the submission, we referred to [8] where FID is utilized to assess the similarity between generated and real images. Its calculation uses Eq. 3 in the Supplementary Material. Specifically, we use FID to compute the distance between a set of reconstructed images and a set of original images. Specifically, we use InceptionV3 pre-trained on ImageNet to extract features from both image sets, resulting in two sets of feature vectors. Then, we compute the mean and variance for each set and use Eq. 3 in the Supplementary Material to calculate FID between the reconstructed and original images. We will provide a more detailed introduction to FID computation in the Supplementary Material.\n\n\n**Q2: Domain generalization ability of the proposed metric should be discussed, such as only having one set of reconstructed images for training.**\n\nWe greatly appreciate this suggestion and will add more discussion in the revised paper. In all of our experiments, we apply SemSim on datasets with (mostly) disjoint label spaces from datasets where SemSim is trained. We show that SemSim exhibits higher faithfulness to human perception than existing metrics. Notably, in Section 5.1, SemSim surpasses FID and PSNR on CIFAR-100, even when it is trained with only 50 human annotations from datasets other than CIFAR-100. It illustrates the capacity of SemSim to generalize to unseen data with limited training data.\n\nDuring rebuttal, following the suggestion, we trained SemSim on one dataset (CIFAR-100) and applied it to other datasets than CIFAR-100. Results in the table below indicate that SemSim achieves higher absolute Spearman correlation values on all four test sets, compared with existing metrics. This further demonstrates the generalization capability of SemSim and will be added to the revised paper. \n\n\n| Dataset | Abs(Spearman’s $\\rho$) | SemSim|\n|--------|--------|--------|\n| Caltech-101  | 0.7349 (MSE)   |  0.7517  |\n| Imagenette   | 0.7349 (LPIPS)  | 0.7653  |\n| CelebA       | 0.7495  (PSNR)  | 0.7682  |\n| Stanford Dogs| 0.5031 (LPIPS)   | 0.6194  | \n\nLimitation. We recognize that the effectiveness of SemSim may be influenced when being used for very different tasks from its training domains such as medical images. In Lines 177-181, we discuss potential strategies to improve SemSim's flexibility to broader domain variations through annotating diverse data types, which we will investigate in future work. These discussions will be incorporated into the revised text.\n\n\n**Q3: L2 distance is used by SemSim. What will happen to SemSim's performance if alternative distance measures are used?**\n\n**L2 distance is also used in the triplet loss for SemSim training**, so using L2 distance during inference is a natural choice. Moreover, during both training and inference, L2 distance is computed on the **normalized feature vectors** of reconstructed and original images. So if we use cosine similarity, there will be no change to the correlation observations. We will include this discussion in the revised paper. \n\n\n**Q4: Typo.**\n\nThank you for pointing out the typos. We will correct these typos and double check our paper to ensure accuracy and clarity."}, {"review_id": "qdBA7bMgcW", "reviewer": "Reviewer_Lywc", "summary": "The paper points out that current metrics, such as PSNR, MSE, SSIM, FID, and LPIPS, do not always reflect human's judgements on evaluation of model privacy risk under reconstruction attacks. Therefore, it proposes a learning-based measure called SemSim which is more compatible with human opinions. SemSim was trained on a binary-annotated dataset using a standard triplet loss and demonstrated good generalizability in terms of model and dataset. ", "questions": "1. There is an inconsistency between the claim mentioned between lines 63-65 and the experimental setting mentioned from lines 230 to 231 and 256 to 257. I think the latter is correct, right?\n \n2. Why does the paper use binary annotations for privacy leakage, which is less informative than multiscale ones? Furthermore, using non-binary annotations enable the usage of semi-hard triplets, improving the metric network.\n \n3. Are there any reasons not to use the absolute values of Spearman's and Kendall's correlation? Mixing positive and negative values in the tables may cause readers confusion.\n \n4. Is it possible to use the targeted network or something similar for measurement besides the standard metrics mentioned in the paper? For example, in face recognition, a face recognition system can be used as a metric to judge how similar the two facial images are. Another example is ImageNet classification. We can use an image classifier as a feature extractor, then calculate the distance between the embeddings of the original and the reconstructed image. Or maybe the perceptual loss? The reviewer believes that such scores should be investigated.\n", "limitations": "The authors adequately addressed the limitations.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper has good motivation and is well-written. It is the first in the literature to compare human and machine metrics for measuring privacy leakages under reconstruction attacks. The proposed measure had good generalizability.", "weaknesses": "This paper is based on the assumption that human perception is superior to measuring model privacy preservation ability than the machine.\n \nThe number of human annotators is small and their scores are binary instead of multi-value, e.g., from 0 to 5. The reviewer believes that using non-binary labels for a more advanced triplets loss function during training SemSim will result in better performance of the proposed metric.\n \nIt is easy to guess that a metric network trained using human scores will produce scores close to human scores.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful review and valuable feedback. Below we address your concerns. Please let us know if further clarification is needed.\n\n**Q1: Inconsistency between lines 63-65 and the experimental setting mentioned from lines 230 to 231 and 256 to 257. I think the latter is correct, right?**\n\nThank you very much for pointing out this inconsistency. Our experimental setup aligns with the description in Lines 230-231 and Lines 256-257. During rebuttal, we conducted experiments using the setup described in lines 63-65. Please refer to our reply to Q2/reviewer **j2WQ** for details. We will correct lines 63-65 for consistency in the revised paper.\n\n**Q2: Why does using binary annotations for privacy leakage which is less informative than multiscale ones?  (using non-binary annotations enable the usage of semi-hard triplets, improving the metric network.)**\n\nGreat question. In this work, we use binary annotations mainly for their simplicity: a simple yet effective neural network can be trained with the standard triplet loss, providing a clear and practical methodology. As discussed in lines 194-196, privacy leakage can take continuous values or multiple scales, which has the potential to offer richer supervision for training SemSim. That said, the multi-level scoring might be limited by its subjectivity, where it is non-trivial to design a robust and consistent scoring procedure. We would like to explore this idea in our future investigation.   \n\n**Q3: Are there any reasons not to use absolute values of Spearman's and Kendall's correlation?**\n\nGreat suggestion. We confirm absolute values of Spearman's and Kendall's correlation can be used without causing confusion. We will use these absolute values in the revised paper for easier result interpretation.\n\n\n**Q4: Is it possible to use the targeted network or something similar for measurement besides the standard metrics mentioned in the paper? For example, in face recognition, a face recognition system can be used as a metric to judge how similar the two facial images are. Another example is ImageNet classification. We can use an image classifier as a feature extractor, then calculate the distance between the embeddings of the original and the reconstructed image. Or maybe the perceptual loss? The reviewer believes that such scores should be investigated.**\n\n*Using the target network.*  Yes, it is possible. We discussed this option in detail in Lines 183-193. While this method seems promising and easy to implement, it suffers from a few limitations.  First, it requires the original classifier to be trained on a dataset that aligns with the categories of the task at hand. Second, the accuracy of the original classifier significantly impacts the applicability of this approach, thereby constraining its scope and effectiveness. In comparison, our method can be used in a wider scope, is not dependent on classifier performance, and is demonstrated to have a good correlation with human perception.\n\nIn another consideration, using the target network for privacy leakage evaluation is an alternative to human perception (Lines 183-193). Both give insights into privacy leakage, and we choose to focus on the latter in this paper. \n\n*Using the ImageNet-trained classifier for feature extraction.* Following this suggestion, we computed the ImageNet features for original and reconstructed images during rebuttal and used their feature distance to measure privacy leakage. The resulting absolute Spearman’s ρ values are 0.7495 and 0.4637 on the CIFAR-100 and Caltech-101 datasets, respectively, which are significantly lower than SemSim's 0.8637 and 0.8182. We will incorporate these discussions in the paper and explore more possible methods in future work.\n\n**Q5: This paper is based on the assumption that human perception is superior in measuring model privacy preservation ability to the machine.** \n\nGreat comment. In Lines 183-193, we discuss in length the use of a classifier (machine) for privacy leakage evaluation and recognize that it can be an alternative to human opinions (Lines 192-193). In fact, we do not intend to compare human and machine in privacy leakage evaluation. We simply focus on the former, which should be an important opinion for privacy leakage and forms a nice benchmark to assess the alignment of metrics with human. \n\n**Q6: It is easy to guess that a metric network trained using human scores will produce scores close to human scores.**\n\nWe agree. But please note that human scores are an important source of judging privacy leakage, so we feel it makes sense to train a metric that resembles human opinions. Importantly, through this training scheme, we find that the resulting SemSim metric has very strong generalization ability across datasets, which is critical for its future use.\n"}]

Author rebuttal:
We thank all reviewers for the thoughtful feedback. We are greatly encouraged that the reviewers appreciate this manuscript for its originality and novelty, its implications for the machine learning privacy community (reviewer **oRwe**),  clear motivation (reviewers **oRwe** and **Lywc**), comprehensive comparisons (reviewers **AN5f** and **j2WQ**), and good writing (reviewers **oRwe** and **Lywc**). 

Below we discuss a common question summarized from reviewers' comments. We will then answer the questions of each reviewer separately. 

* **How to further improve the performance of the learning-based metric?** 
As an initial work utilizing human annotation to address the faithfulness of evaluation metrics in privacy attacks (Reviewer **oRwe**), we acknowledge that there is room for further improvement to the proposed SemSim. As suggested by reviewers, potential solutions include using local region information (Reviewer **AN5f**), non-binary labels (Reviewer **Lywc**), and annotating more data to enlarge the application scope (Reviewer **oRwe**). In the current manuscript, we also discussed several potential ways on how to further improve SemSim, *e.g.,* Lines 180-181 and 284-256. With that said, we believe that the potential for further improvement does not impact the contribution of work, which offers valuable insights into privacy assessment and can further stimulate efforts within the community to develop more robust evaluation metrics.

We will answer the questions of ethics reviewers during the discussion session, following the guidelines for rebuttal replies.

Meta-review: The paper received positive scores initially: 1 Strong accept, 1 accept, 2 weak accept. The following issues were identified:
1) the definitely of privacy for image content classification misses privacy in local regions. (AN5f)
2) details missing about the procedure for human rating collection (AN5f)
3) simple metric learning method (AN5f)
4) how well does the metric generalize? what is the scalability? (oRwe, j2WQ)
5) opening up the metric may allow attackers to build stronger attack models (oRwe)
6) missing ablation study on the metric used (j2WQ)
7) would non-binary labels be better? (Lywc)
8) are other similarity measurements possible for specific tasks? (Lywc)
9) an assumption that human perception is superior in measuring privacy preservation. (Lywc)

The authors wrote a response, and all reviewers were satisfied by the responses. The reviewers appreciated the high originality and novelty, as well as the meaningful impact to the ML privacy community, new dataset and good results, and the quality of the paper presentation. 
Thus, the AC recommends accept. The authors should update the paper according to the review, response, and discussion.
Decision: Accept (spotlight)