Membership Attacks on Conditional Generative Models Using Image Difficulty
Authors: Avital Shafran
Shmuel Peleg
Yedid Hoshen
Venue:  (2021)
Keywords: Membership Inference Attack
Image translation

Abstract: Membership inference attacks (MIA) try to detect if data samples were used to train a Neural Network model. As training data is very valuable in machine learning, MIA can be used to detect the use of unauthorized data. Unlike the traditional MIA approaches, addressing classification models, we address conditional image generation models (e.g. image translation). 
Due to overfitting, reconstruction errors are typically lower for images used in training. A simple but effective approach for membership attacks can therefore use the reconstruction error.
However, we observe that some images are "universally" easy, and others are difficult. Reconstruction error alone is less effective at discriminating between difficult images used in training and easy images that were never seen before. To overcome this, we propose to use a novel difficulty score that can be computed for each image, and its computation does not require a training set. Our membership error, obtained by subtracting the difficulty score from the reconstruction error, is shown to achieve high MIA accuracy on an extensive number of benchmarks.

Under review as a conference paper at ICLR 2021
MEMBERSHIP ATTACKS ON CONDITIONAL GENERATIVE MODELS USING IMAGE DIFFICULTY
Anonymous authors Paper under double-blind review
ABSTRACT
Membership inference attacks (MIA) try to detect if data samples were used to train neural network model. As training data is very valuable in machine learn- ing, MIA can be used to detect the use of unauthorized data. Unlike the tradi- tional MIA approaches, addressing classification models, we address conditional image generation models (e.g. image translation). Due to overfitting, reconstruc- tion errors are typically lower for images used in training. A simple but effective approach for membership attacks ever, we observe that some images are "universally" easy, and others are difficult. Reconstruction error alone is less effective at discriminating between difficult im- ages used in training and easy images that were never seen before. To overcome this, we propose
therefore use the reconstruction error. How-
a novel difficulty score that be computed for each im- age, and its computation does not require a training set. Our membership error. obtained by subtracting the difficulty score from the reconstruction error, shown to achieve high MIA accuracy on an extensive number of benchmarks.
INTRODUCTION
Deep neural networks have been widely adopted in various computer vision tasks, e.g. image classi- fication, semantic segmentation, image translation and generation etc. The high sample-complexity of such models requires large amounts of training data. However, obtaining many training images might not be an easy task. In fact, collection and annotation is often an expensive and labor intensive process. In some domains, such as medical imaging, publicly available training data are particularly scarce due to privacy concerns. In such settings, is common to grant access to private sensitive data for training purposes alone, while ensuring not to reveal the data in the inference stage. A common solution is training the model privately and then providing black-box access to the trained model. However, even black-box access may leak sensitive information about the training data.
Membership inference attacks (MIA) are one way to detect such leakage. Given access to a data sample, attacker attempts to find whether or not the sample used in the training process. MIA attacks have been widely studied for image classification models, achieving high success rates (Shokri al., 2017 Salem et al. 2018 Sablayrolles 2020 Choo et al. (020). Due to overfitting in deep neural networks, prediction confidence tends to be higher for images used in training. This difference in prediction confidence helps MIA methods to successfully determine which images were used for training. Therefore, in addition to detecting information leakage, MIA also provide insights on the degree of overfitting in the victim model.
al. 2019 Yeom al., 2018 Li & Zhang
We address MIA for a new domain conditional image generation models, e.g. image translation. While classification models give probability vector over possible classes, generation models give a single color for every pixel. We propose a MIA that uses pixel-wise reconstruction error, as overfit- ting causes lower reconstruction error on images used for training. But we observe that some images are "universally" easy, and others are universally difficult. Reconstruction error alone is therefore less accurate at discriminating between difficult images used in training and previously unseen easy images. To overcome this limitation, we add novel image difficulty score which is computed for each query image. Our image difficulty score uses the accuracy of a linear predictor computed over given image, predicting pixel values from deep features of that image. The reconstruction error together with the difficulty score helps to discriminate between two factors of variation in the recon- struction error, namely (i) The "intrinsic" difficulty of the conditional generation task for each image,

---

Under review as a conference paper at ICLR 2021
based on its difficulty score and (ii) The boost in accuracy due to overfitting to the training images. Defining a membership error that subtracts the difficulty score from the reconstruction error is shown empirically to achieve high success rates in MIA. Differently from other MIA approaches, we do not assume the existence of large number of in-distribution data samples for training a shadow model but rather operate on merely a single image. Our method is evaluated benchmarks demonstrating its effectiveness compared to strong baseline methods.
extensive number of
2
RELATED WORK
2.1
MEMBERSHIP INFERENCE ATTACKS (MIA)
Shokri et al. (2017) were the first to study MIA against classification models in a black-box setting. In black-box setting the attacker can only send queries to the victim model and get the full probability vector response, without being exposed to the model itself. They proposed to train multiple shadow models to mimic the behavior of the victim model, and then use those to train a binary classifier to distinguish between known samples from the train set and unknown samples. They assume the existence of in-distribution new training data and knowledge of the victim model architecture.
Salem al. (2018) further relaxed those assumptions and demonstrated that using only shadow model is sufficient for a successful attack, and proposed using out-of-distribution dataset and differ- shadow model architectures, for a slightly inferior attack. Even more interestingly, they showed that without any training, a simple threshold on the victim model's confidence score is sufficient. This shows that classification models are more confident of samples that appeared in the training process, compared to unseen samples.
Sablayrolles al. (0119) proposed an attack based on applying threshold over the loss value rather then the confidence and showed that Blck-box attacks are as good as white-box attacks. As the naive defense against such attacks to modify the victim model's API to only output the predicted label, other works proposed label-only attacks (Yom et al., 2018 Li & Zhang 2020 Choo et al. 2020).
While most previous work has been around classification models, there has been some effort re- garding MIA on generative models such GANs and VAEs (Chen et al.) 2019 HHaes et al.) 2019 Hilprecht et al. 2019). An attack against semantic segmentation models was proposed by He et al. (2019), where shadow semantic segmentation model is trained, and is used to train binary clas- sifier. The classifier is trained on image patches, and the final decision regarding the query image is set by the aggregation of the per-patch classification scores. The input to the classifier is a structured loss map between the shadow model's output and the ground truth segmentation map. Although this task is the closest ours, our work is the first study of membership inference attacks conditional image generation model.
Besides membership inference attacks, other privacy attacks against neural networks exist. We refer the reader to Sec. J. for more details of such attacks.
2.2 CONDITIONAL IMAGE GENERATION
toage-to-image translation is the task of mapping an image from a source domain to target domain, while preserving the semantic and geometric content of the input image. Over the last decade, with the advent of deep neural network models and increasing dataset sizes, significant progress was made in this field. Currently, the most popular methods for training image-to-image translation models use Generative Adversarial Networks (GANs) (Goodfellow al. 2014) and are currently used in two main scenarios: (i) unsupervised image translation between domains (Zhu et al. 2017a Kim et al. 2017] Liu et al. [2117] Choi al.) [018); (ii) serving perceptual image loss function (Isola et al. [2017] Wang et al. 2018) Zhu et al. 2017b). In this work we introduce the novel task of
MIA on conditional image generation models.
MIA ON CONDITIONAL IMAGE GENERATION MODELS
In membership inference attacks (MIA), an adversary attacks a victim model by attempting to infer whether query data sample used to train a victim model. Such attacks exploit overfitting to
2

---

Under review as a conference paper at ICLR 2021
the training data performed by the victim model. For classification, overfitted model will likely be more confident of the prediction for data samples that were included in the training set.
We focus on MIA in We propose a simple and effective attack. Differently from most previous works (Shokri ct al. 2017 He al., 2019), do not use shadow models train binary classifier, and thus do not require any additional training data. We assume the most restrictive attack setting, where the attacker only has black-box access the victim model V, and has no knowledge of its weights and architecture.
domain: conditional image generation models, e.g. image translation.
Our membership attack is performed on pair of query images (I,, y) where x is an image from the input domain and y is the ground truth from the output domain. The existence of the ground truth image y is in-line with previous works, and is reasonable assumption in conditional generative models. For each query we compute a membership error, Lmem (see Eq. (3). which we apply pre-defined threshold 7, such that all queries where Lmmm y) < T are marked as members of the training data. The membership error has two elements: reconstruction error and difficulty score.
3.1 RECONSTRUCTION ERROR FOR MEMBERSHIP EVALUATION
Typical MIA that operate on classification models consider the probability (or confidence) given by the model the correct class. Image generation models are different they output color value of each pixel. This value is the maximum likelihood estimate, and no probability distribution over possible values given. As the confidence of the model prediction is unknown, we propose to examine the reconstruction error of the model over all pixels.
The reconstruction error described in Eq. ① and computed using the L1 error between the output image predicted by black-box access to the model, V(z), and the ground truth image y- We show that for images in the training set, the model output has mostly lower prediction errors compared to unknown images. See Table [
Liee (x. v) [IV(x) vl12
3.2 EASY IMAGES AND DIFFICULT IMAGES
In this section, we tackle the following question: Given an image, compute a score that measures how "difficult" it is to synthesize it. Consider, for example, the task of supervised segmentation-to- image translation. Le. the task is to "invert" the segmentation process, and recover the original image that gave rise to given segmentation map. It is clear that not all images equally difficult: (i) more difficult images have sharp and detailed textures whereas simpler images have blurrier textures; (ii) images with semantic segmentation maps that contain only few categories provide less guidance than those with more detailed segmentation maps, making the correct prediction less certain. Image difficulty score should quantify these difficulties. In Sec. 4.1 we show that such difficulty score important to increase accuracy of membership inference atlacks on conditional generative models.
We briefly describe two previous approaches for measuring image difficulty:
Human-Supervised: Tudor lonescu et al. (2016) proposed to define image difficulty as the human response time for solving visual search task. For this, they collected human annotations for the PASCAL VOC 2012 dataset (Everingham et al. 2010) and trained regression model, based on pre-trained deep features, to predict the collected difficulty score. The disadvantage of this method is that human-specified difficulty scores may not correlate to the difficulty of image synthesis by neural networks. This demonstrated empirically in Sec. 4.3
Multi-Image: Image: Another approach taken by Chen et al. 2019 is training generative model on set of images similar to the target image distribution. Consider for example training an autoencoder on set of external images. This approach uses the reconstruction error on the target image as its difficulty score larger reconstruction errors correspond to more difficult images. This approach has significant drawback: large number of images, similar to the target image, are required in order to learn a reliable generative model. In many cases, images from the target distribution may not be available. Additionally, training a model for every task is tedious and computationally expensive.
Proposed Single-image difficulty score: We propose a novel method to assign a difficulty score for image generation models. The difficulty score measures the success of linear regression model
3

---

Under review as a conference paper at ICLR 2021
Examples images from the Cityscapes and Maps2sat datasets that received the lowest (first row) and highest (second row) image difficulty scores using our single-imag approach. It can be seen that detailed images with complicated patterns are ranked as difficult, while images with less details and lower contrast ranked easier.
Figure I:
to predict the pixel values from high-level representation of the image. A related approach was proposed by Hacohen Weinshall (2019) for measuring image difficulty for classification models. Our method is significantly different it is trained on a single image rather than large dataset, and that it focuses on generation rather than classification.
Our features Zagoruyko Komodakis 2016), concatenated together, giving 56 56 feature vectors size 3840. We reduce the input image to 56 56 match the spatial dimension of the first Wide-ResNettlle block. We denote the concatenated feature vector for pixel i as (6)). See Sec. A.2 for more details.
the activation values in the first 4 blocks of a pre-trained Wide-ResNet50 x
The linear regression model P is matrix of size 3840x3, 3, multiplied with the feature vector (iii of pixel to give a linear estimate the RGB colors y'. We minimize P over 70% randomly selected pixels. The image difficulty score is the average absolute
the 30% unselected pixels:
N
Laiss( v)
P
where y² is the ground truth value of the izh pixel in the resized ground truth image Fig. presents examples of images that received the highest and lowest difficulty scores.
3.3 MEMBERSHIP ERROR
As observed before, some images "universally" easy to reconstruct, while others universally difficult to reconstruct. While the reconstruction eтror in Eq. aachieess high MIA success rates, has significant limitation it does not discriminate between difficult and easy samples. Difficult training samples might be more difficult for the victim model to generate and therefore can receive high reconstruction error. Similarly, an easy unknown sample can be generated by the victim model with lower error. Such cases can cause wrong classification if only the reconstruction error is used.
Given our image difficulty score Lass in Eq. 2 and the reconstruction error Liee Eq. calculate membership error Lmmen as follows:
Linem (4, y) ()) ar Laifs
Linem is computed by subtracting the difficulty score Laify from the reconstruction error Lrec weighted by (unless specified otherwise, we Lynem for ponder-to-predinn images compared to easier-to-predic images having the same recon- struction error. See Fig. for overview illustration of our method.
0.5). This lowers the membership error
Using the membership error Lmem for MIA substantially improves the success rates in all of our experiments, as shown in Table T and Fig.
4

---

Under review as a conference paper at ICLR 2021
Semantic Segmentation
Generated image
Victim Model
Reconstruction
Enor
Low
Membership Error Love
Feature Extractor
Ground Truth
Difficuity
Wide ResNet50
Score
Linear Regression
Foatures
Figure 2: Illustration of the proposed black-box membership inference attack conditional segmentation-to-imaa generation models. We would like to determine given image and its semantic segmentation based on the segmentation. In the top path the difference between the reconstructed image and the input image gives the reconstruction error Tree- In the bottom path compute the difficulty score L of the input image from the error linear predictor to predict pixel values of the ground- truth image from its deep features. Subtracting Lasss from Lyec gives the membership error.
used in training. The victim model generates reconstructed image
4 EXPERIMENTS AND COMPARISONS
We conduct thorough investigation demonstrating the effectiveness of our approach. First, show its high success rate on various benchmarks. A comparison between our novel single-image, supersup,,,,,ed, difficulty score to two alternative difficulty scores: multi-image and a supervised difficulty scores, shows the superiority of our proposed method. We also compare our attack with the commonly used attack based shadow model and show that attack is not only superior but also requires minimal assumptions over the attacker's knowledge. As MIA attacks are closely related overfitting, we study the effect of overfitting on our attack success rate. Additional results as well as discussion regarding possible defenses can be found in the appendix.
4.1 IMAGE TRANSLATION MEMBERSHIIP INFERENCE ATTACK
We propose novel membership attack on two popular image translation architectures Pix2Pix [Isola et al. 2017) and Pix2PixHD (Wang et al. 2018), as well as three datasets CMP Facades Tyleeek & Sára 2013). Maps2sat (Isola et al. 2017) and Cityscapes (Cordts et al. 2016). All models are trained from scratch, with the exception of the Cityscapes dataset on the Pix2pixHD architecture in which we use the supplied large pre-trained model for computational constraints on the high resolution. In accordance with previous membership attack works, the success rate is measured using the ROC area under the curve (ROCAUC) metric. It can be seen Table that while using the reconstruction error alone achieves high success rate, the membership error further improve these results by to 6%. Fig 3 demonstrates the effect of subtracting the difficulty score from the prediction error. A single threshold on the membership error can separate train and test images. For more results, see Fig. 6 in the appendix.
We study the effect of utilizing common image augmentations, i.c. horizontal flipping and random cropping, over the pair (z, y) in order to construct larger set (Tangy Waug)] and define Lree to be the average reconstruction error over the set. This can improve the accuracy by up to 5% on Pix2pix, see App. A.5 details.
Possible defenses against our attack are discussed and evaluated in Sec. A.6 the appendix.
4.2 SINGLE IMAGE DIFFICULTY SCORE
In this section we compare our single-image difficulty score (Sec. 3.2 with a multi multi-image difficulty score, in which "shadow" model is trained new data and define the difficulty score to be the
5

---

Under review as a conference paper at ICLR 2021
Pix2pix Maps2sat
Pix2pixHD
Facades
Cityscapes
Facades
Maps2sat
Cityscapes
Reconstruction error
93.39%
84.19%
77.44%
98. 88.91% %
95.73%
96.04%
Membership error
96.62% %
90.54%
82.23%
99.02% %
99.89%
99.19% %
Table 1: Membership attack ROCAUC using our (i) reconstruction error Lyee and (ii) member- ship error mmmmn Using the membership error, which subtracts the image difficulty score from the reconstruction error, substantially improves performance.
0.05
0.06
p00.0-
0.08
0.88-
0.06
diffiultty
0.05
0.05
0.00
0.06
trai
train
train
0.00
0.00
0.92
0.02
0.30
0.5
0,30
0.15
029
0.00
0.05
0.10
0.35
93
02
0.3
0.0
0.1
02
Leee
Lmom
Lecc
Amen
(a) Maps2sat
(b) Cityscapes
Figure The proposed membership error Imnem can better separate train and test images by simple threshold (i.e. vertical line) compared to the reconstruction error Tyrec-
average L1 reconstruction error on the new model. order to upper-bound the multi-image diffi- culty score, the same shadow model architecture that of the victim model (as this is the most favorable setting for it). We also ensure the shadow model's 's training data shares the same distribution the victim's 's training data, by randomly sampling 100 images from the test of the corresponding dataset. We did not perform this experiment on the Facades dataset as its test set does not contain sufficient number of samples.
The results are presented in Table In the Pix2pixHD model, the multi-image model badly overfit to able generalize well enough to perform as a difficulty measure for other
its training data and images. It is therefore inferior to our single-image difficulty score. In the simpler pix2pix model, the multi-image difficulty metric seems comparable sufficient number of images. We hypothesize that it is due to the lower capacity of the architecture, which limits overfitting. This however comes at the cost of the often unrealistic requirement of extra training data. In addition, this assumes knowledge of the victim architecture. Our single- image score does not require extra training images or knowledge the architecture, which makes applicable in more cases. Fig 4 presents the effect of number of training images the multi image calibration method. As can be seen, in the pix2pix model, at least 50 of the same distribution images required in order outperform images outperformed our Sngll-image score.
the single-image difficulty score, given
method. For Pix2pixHD none of the evaluated number of
4.2.1 OUT-OI--DITTRIUUTOON MULTI IMAGE
As suggested by He et al. (2019), we also compare our score to the more realistic scenario in which a large amount of similar but oiii---disttiiuutiinn dataset available. For this cause, we train shadow model on 4K images from the BDD dataset (Yu et al., 2018). done by He et al. (2019). We then use this model a shadow model for multi-image difficulty score to the Cityscapes dataset, both datasets consist of street scene images and have compatible label spaces. Note that assumes knowledge of the victim model, which is not always true and is therefore a best-case scenario. Table demonstrates that this approach is inferior to single-image difficulty score.
4.3 SUPERVISED DIFFICULTY SCORE
We compare self-supervised single-image difficulty score with the supervised difficulty score described in Sec. 3.2 The supervised score was proposed by Tudor lonescu et al. (2016), which defined image difficulty be the human response time for solving visual search task. In order to

---

Under review as a conference paper at ICLR 2021
0.00
0.975
ROCAUC
0.53
ROçAUC
ROCAUC
0.22
mut:-map Sing eingge borr)
0.50
0.900
0.39
muliiinage sigleinags (ours)
0.00
Sigle mage lour))
mulli Image
0.500
0.375
0.85
0.80
muGinage
Singl: magi (ouss)
20
50 150
20
to * train Images
60
100
40
60
80
150
43
t0 160
@ train Images
# train Images
train Images
(a) Pixppix-Maps2sat
(b) Pixppix-Cityscapes
(c) Pi22pixHD-Maps2sat (d) PixppixtD-CCtyscape
Figure 4: Comparison of MIA accuracy when using our single image vs. using multi-image diffi- culty scores, a function of the number training images. Note that the multi-image score assumes knowledge of the victim's 's model, as well as the availability of many labeled training images.
Model
Dataset
Single-Image
Multi-Image
Ours
Supervised
BDD Dataset In-Dist.
Pix2pix
Facades
96.62%
94.17%
Pix2pix
Maps2sat
90.54%
86.54%
92.43
Pix2pix
Cityscapes
82.23 %
77.66%
74.43%
82.47%
Pix2pixHD
Facades
99.02 %
98.86%
Pix2pixHD
Maps2sat
99.89 %
98.38%
82.87%
Pix2pixHID
Cityscapes
99.19%
96.86%
66.2%
94.76%
Table MIA accuracy of our method vs. using single and multi-image baselines for the difficulty score. Note that the BDD dataset is only relevant to Cityscapes and that in-distribution multi multi-image requires extra supervision of 100 images.
provide a fair comparison, we replace the pretrained VGG-f (Chatfield et al. 2014) features, used by Tudor lonescu et al. (2016), with the more recent pretrained Wide-ResNetie in model. Fig. 13 the appendix presents samples of im- ages ranked as easy and hard by the supervised score. As can be seen in Table 2 our self-supervised single-image difficulty score outperforms the supervised difficulty score. Fig. compares the lation between the reconstruction error and the supervised score to the relation between the recon- struction error and self-supervised difficulty score, and shows that our score better correlated to the reconstruction error. For comparison on other benchmarks, see Fig. in the appendix.
2 Zagoruyko &
Komodakis 2016) features, as
singlesa imaiiaage atffcultty (ours)
gmg difficutty
difficuity
2.36
0.07
superei wupporvitee difficuity
0.35
2.5
0.08
0.05
0.32
0.04
2004
0.02
train
tain toom.
toin (com. 0.75)
[conr.
0.00
test Icorr. 0.87)
228
test (cort. 0.56)
8.02
test (coer, 0.69)
test (corr, 0.48)
0.00
0u8
ain
als Leee
0.20
ado
ads
oin ois 0.0
03
02
0.3
81
n2
03
Leer
Lee
Leer
(a) Maps2sat
(b) Cityscapes
Figure 5: Comparison of the relation between the reconstruction error and the supervised difficulty score (right) to the relation between the reconstruction eтror and our self self-supervieed difficulty score (left) on Pix2pixHD. Our difficulty score is better correlated to the reconstruction error.
4.4 SHADOW MODELS
We compare our method with shadow wawloowwmoollbbbaeee methods, commonly used membership infer- ence attacks. A shadow model is trained in-distribution data samples create labeled dataset

---

Under review as a conference paper at ICLR 2021
Model
Dataset
Ours
Shadow Model - 100
Shadow Model BDD
ROC
Acc.
ROC
Acc.
Pix2pix
Maps2sat Cityscapes
90.54 % 82.23%
80.15% 78.68%
73.4%
Pix2pix
67.5%
72.57%
56.16%
Pix2pixHD Pix2pixHD
Maps2sat
99.89 %
98. 28.63%
93.7%
Cityscapes
99.19%
96.39%
64.0%
95.78%
56.5%
Table 3: Comparison between our MIA and the commonly used shadow-model- classifier attack, using 100 train and 100 test images, and the BDD shadow model. Our MIA outperforms while not requiring extra training images.
of known (train) and unknown (test) samples. The labeled dataset is then used train binary clas- sifier to distinguish between the two, The assumption is that the classifier trained for membership attacks on the shadow model, will also detect membership of the target model.
i.e. sharing the same architecture between the victim model V and the shadow model S and randomly sampling N images from the test set, referred as shadow train, to train the shadow model S. A labeled dataset is constructed by randomly selecting additional N test images, shadow test. The training procedure detailed App. A.7
We use the shadow models from Sec. 4.2
We compare the ROCAUC of our attack against the classification accuracy and ROCAUC of the confidence score of the classifier C' when applied to images generated by the victim model V in Table 3 Similar to He et al.] (2019), we apply the classifier over several patches from the query image, and average the results over all patches. In order to provide a fair comparison we not evaluate using the 2N images used in the training of the classifier. It can be seen that our attack outperforms the shadow model approach on all experiments. We further investigate the effect of the amount of training data used to train the shadow model and the corresponding classifier. As can be seen in Fig. 11 the success of the shadow model based attack depends on the amount of data used. This shows that in the common scenario of having just one few images in-distribution images, the shadow model approach is not as effective as ours. As in Sec, Sec. 4.2. we compare to the more realistic scenario in which a large amount of similar but -attr--distribution dataset available. We use the shadow model trained on 4K images from the BDD dataset YYu et al., (008)) and train the classifier distinguish between those 4K and additional 4K images that were not used in the training of the shadow model. As can be seen in Table 3 this approach is inferior to our single-image method.
4.5 EFFECT OF OVERFITIIG
Membership inference attacks are closely related to overfitting in the victim model. In order to better understand this relation, we measure the success of our calibrated prediction error based attack under different levels of overfitting, We do so by evaluating our attack on checkpoints saved at different epochs during the training of vietim model. For the purpose of this experiment, we also trained the Pix2pixHD model on the Cityscapes dataset instead of using the pretrained model. The results are presented in Fig. 9 We can observe that as the training process progresses, the victim model overfits more to the training data which results in higher attack success rates.
5 CONCLUSION
In this work, we present black-box membership inference attack on conditional image generation models, e.g. image translation. At first we analyze a training-free attack based on the reconstruction error. We further improve this attack by proposing novel image difficulty score, whose computation does not require an auxiliary training set. By utilizing this score, we can successfully discriminate between difficult images used in the training set, whose reconstruction error is large, and easy previ- ously unseen images that have low reconstruction error. Our novel combined membership error was shown to achieve higher accuracy than baselines that use more supervision multiple benchmarks.
8

---

Under review as a conference paper at ICLR 2021
REFERENCES
Martin Abadi, Andy Chu, lan Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference Computer and Communications Security, pp. 308-318, 2016.
Adrian Bulat and Georgios Tzimiropoulos. How far are we from solving the 2d & 3d face align- ment problem? (and a dataset of 230,000 3d facial landmarks), In International Conference Computer Vision, 2017.
Ken Chatfield, Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Return of the devil in the details: Delving deep into convolutional nets. arXiv preprint ariiv: 2014.
Dingfan Chen, Ning Yu, Yang Zhang, and Mario Fritz. Gan-leaks: A taxonomy of membership inference attacks against gans. arXiv preprint arXiv:1909.03920 2019.
Yunjey Choi, Minje Choi, Munyoung Kim, Jung-Wor Woo Ha, Sunghun Kim, and Jaegul Choo. Star- gan: Unified generative adversarial networks for multi-domain image-to-image translation. In Proceedings of the IEEE conference on computer vision and pultern recognition, pp. 8789-8797, 2018.
Christopher Choquette Choo, Florian Tramer, Nicholas Carlini, and Nicolas Papernot. Label-only membership inference attacks. arXiv preprint arXiv:2007. 14321, 2020.
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
Cynthia Dwork, Aaron Roth, et al. The algorithmic foundations differential privacy. Foundations and Trends in Theoretical Computer Science, 9(3-4):2114400 2014.
M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The pascal visual object classes (voc) challenge. International Journal of Computer Vision, 88(2):303-338 June 2010.
Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model inversion attacks that exploit confi- dence information and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC Confer- ence on Computer and Communications Security, pp. 1322-1333, 2015.
Matthew Fredrikson, Eric Lantz, Somesh Jha, Simon Lin, David Page, and Thomas Ristenpart. Privacy in pharmacogenetics: An end-to-end case study personalized warfarin dosing. In 23rd {USENIX} Security Symposium ((USENIX) Security 14), pp. 17-32, 2014.
Karan Ganju, Qi Wang, Wei Yang, Carl A Gunter, and Nikita Borisov. Property inference attacks on fully connected neural networks using permutation invariant representations. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, pp. 619 633,, 2018.
Justin Gilmer, Nicolas Ford, Nicholas Carlini, and Ekin Cubuk. Adversarial examples are a natural consequence of test error in noise. In International Conference on Machine Learning, pp. 2280- 2289, 2019.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural infor- mation processing systems, pp. 2672-2680, 2014.
Guy Hacohen and Daphna Weinshall. On the power of curriculum learning in training deep net- works. In International Conference on Machine Learning, pp. 2535-2544, 2019.
Jamie Hayes, Luca Melis, George Danezis, and Emiliano De Cristofaro. Logan: Membership infer- ence attacks against generative models. Proceedings on Privacy Enhancing Technologies, 2019 (1):133-152, 2019.

---

Under review as a conference paper at ICLR 2021
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Yang He, Shadi Rahimian, Bernt Schiele, and Mario Fritz. Segmentations-leak Membership in- ference attacks and defenses in semantic image segmentation. arXiv preprint arXiv: 1rXiv:1912009683 2019.
Benjamin Hilprecht, Martin Härterich, and Daniel Bernau. Monte carlo and reconstruction member- ship inference attacks against generative models. Proceedings on Privacy Enhancing Technolo- gies, 2019(4):232-2)) 249, 2019.
Phillip Isola, Jun- Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattem recognition, pp. 1125-1133 2017.
Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex Kurakin, and Nicolas Paperno. High accuracy and high fidelity extraction of neural networks. In 29th {USENIX) Security Symposium ((USENIX) Security 20), 2020.
Taeksoo Kim, Moonsu Cha, Hyunsoo Kim, Jung Kwon Lee, and Jiwon Kim. Learning to discover cross-domain relations with generative adversarial networks. In Proceedings of the 34th Interna- tional Conference on Machine Learning-Volle Volume 70, pp. 1857-1865, 2017.
Zheng Li and Yang Zhang. Label-leaks: Membership inference attack with label. arXiv preprint arXiv:2007.15528, 2020.
Ming- Yu Liu, Thomas Breuel, and Jan Kautz. Unsupervised image-to-image translation networks. In Advances neural information processing systems, pp. 700-708, 2017.
Ziwei Liu, Ping Luo, Xiaogang Wang. and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
Scong Joon Oh, Bernt Schiele, and Mario Fritz. Towards reverse-engineering black-box neural networks. In Explainable Al: Interpreting, Explaining and Visualizing Deep Learning, pp. 121- 144. Springer, 2019.
Alexandre Sablayrolles, Matthijs Douze, Cordelia Schmid, Yann Ollivier, and Hervé Jégou. White- box vs black-box: Bayes optimal strategies for membership inference. In International Confer- ence on Machine Learning, pp. 5558-5567, 2019.
Ahmed Salem, Yang Zhang, Mathias Humbert, Pascal Berrang, Mario Fritz, and Michael Backes. MI-leaks: Model and data independent membership inference attacks and defenses on machine learning models. arXiv preprint arXiv:1806.01244 2018.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference at- tacks against machine learning models. In 20/7 IEEE Symposium on Security and Privacy (SP), pp. 3-18. IEEE, 2017.
Florian Tramèr, Fan Zhang, Ari Juels, Michael K Reiter, and Thomas Ristenpart. Stealing machine learning models via prediction apis. In 25th (USENIX) Security Symposium {USENIX) Security 16), pp. 601-618, 2016.
Radu Tudor Tonescu, Bogdan Alexe, Marius Leordeanu, Marius Popescu, Dim P Papadopoulos, and Vittorio Ferrari. How hard can it be? estimating the difficulty visual search in an image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2157- 2166, 2016.
Radim Tyleček and Radim Šára. Spatial pattern templates for recognition of objects with regular structure. In Proc. GCPR, Saarbrucken, Germany, 2013.
Binghui Wang and Neil Zhenqiang Gong. Stealing hyperparameters in machine learning. In 2018 IEEE Symposium Security and Privacy (SP), pp. 36-52. IEEE, 2018.
10

---

Under review as a conference paper at ICLR 2021
Ting- Ting-Chun Wang, Ming- Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. High- resolution image synthesis and semantic manipulation with conditional gans. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 8798 888888887,-00 2018.
Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. Privacy risk in machine leamn ing: Analyzing the connection to overfitting. In 2018 IEEE 31st Computer Security Foundations Symposium (CSF), pp. 268-2822 IEEE, 2018.
Aron Yu and Kristen Grauman. Fine- fine-grained visual comparisons with local learning. In Proceedings of the IEEE Conference on Computer Vision and Patlern Recognition, pp. 192-199, 2014.
Fisher Yu, Wenqi Xian, Yingying Chen, Fangchen Liu, Mike Liao, Vashisht Madhavan, and Trevor Darrell. Bdd100k: A diverse driving video database with scalable annotation tooling. arXiv preprint arXiv: 04687, 2(5);6, 2018.
Sergey Za and Nikos Komodakis. 1rXiv:1605.07146, 2016.
Wide residual networks.
arXiv preprint
Jun- Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision, pp. 2223-2232, 2017a.
Jun Zhu, Richard Zhang, Deepak Pathak, Trevor Darrell, Alexei A Efros, Oliver Wang, and Eli Shechtman. Toward multimodal image-to-image translation. In Advances in neural information processing systems, pp. 465-476 476, 2017b.
A APPENDIX
A.1 OTHER PRIVACY ATTACKS
Besides membership inference attacks, there exists a wide range of privacy attacks against neural networks. Model inversion attacks, first proposed by Freediikson et al. (2014), aim at reconstructing features of the training data, e.g. recovering an image of person from face recognition models (Fredrikson et al. 2015). Property inference attacks, proposed by Ganju et al. (2018), do not focus on the privacy of individual data samples, as in membership inference and model inversion attacks, but focus at inferring global properties of the training data, such the environment in which the data was produced the fraction of the data that comes from a certain class.
Model extraction attacks, also referred to as model stealing, attack model by constructing substitute model f that is either identical or equivalent to f (Tramèr et al., 2016 Jagiclski et al. 2020). Related line of work (Wang & Gong 2018: Oh et al. (001)) attempts to infer hyperparameters such the optimization proccess, e.g. SGD or ADAM.
A.2 DETAILED DESCRIPTION OUR MIA ALGORITHM
Our MIA consist of computing the two terms in Eq. 3). i.e. Tree and Laif for given query pair (x, y). where z is an image from the input domain and y is the ground truth from the output domain, using only black-box access to the victim conditional generation model V.
Liee is computed using the Li error between the output image predicted by the model, V(x), and the ground truth image y, see step 1 in the Algorithm 1.
Taiss is computed as the average error of a linear regression model, P, predicting pixel values from deep features of the ground truth image.
Our deep features are the activation values in the first blocks of a pre-trained Wide-ResNet50x2 Zagoruyko & Komodakis) 2016). These features are of sizes 56 256. 28 28xx8xx112 14x14x 1024, and 7x7x 2048 We interpolate all features to size 56 using bi-linear interpola- tion (step 2), and also reduce the input image to 56 (step 3). This gives concatenated feature vector of size 3840 for each pixel in the 56x56 image (256+512+ 1024 200--08441111 We denote
the concatenated feature vector for pixel (i).

---

Under review as a conference paper at ICLR 2021
We randomly select 70% of the pixels train set, and compute linear model P to estimate the RGB pixel values Viaain from the corresponding deep features Ptrrai(1) (step 4). The remaining 30% of pixels will be used a test split, (Rrest Voest (step 5). Le. [Wtrain| 2195 x145x |frainl 2195x3 and Mees = 941 x 3840, [ytest] = 941 x3.
The linear regression model P, matrix of size 3840x3, is trained to minimize the Arrain Utrain } (step 6). Laify will be the average absolute error over {tltests Utest (step 7). We found that fitting the linear model 70% of pixels and measuring the error on the remaining 30% gives better results than just measuring the error of the linear fitting.
We compute mmem according Eq. (3) and compare the results with predefined threshold value such that any pair (z, y) for which is holds that Lime (I, v) < is denoted as a member of the victim models' V train set (steps 8-9).
Algorithm 1. Membership Inference Attack Input: Query pair (I, y). victim model V, feature extractor F, scalar a, threshold Output: Membership inference result
1. Trrec [v v|l
F/v\// [bb| 56 x x 3840
y resizely, 56 56 x 3)
{shtroin: Verain 70% {t. y/
fihtest Meest = {2) y} \ (P/train: Mtrain
Train linear regression with (irraains Strain
7. Lasfs = N Σ (PPrrea(()) viestlla //N- 941
Lmem = Lrec a Lasss
if Lmem < then
Return True
else
Return False
A.3 PARAMETER SELECTION
We experimented with different values for the value in Eq. 33. As can be in Fig. 0.5 was the best choice over all benchmarks.
A.4 OVERFITIIG
overfitting is the main reason for the success our method. Fig. shows the accuracy of our method as a function of the number of epochs used for training the victim model, clearly suggesting that overfitting is indeed the vulnerability.
As mentioned Sec. 4.5
A.S EFFECT OF AUGMENTATION
In order further increase the accuracy of our reconstruction-erron based attack, we take advantage of the augmentations that were used as part of the training process of the victim model. We use two common augmentations: random flip and random crop. We randomly sample T augmentations, and estimate the prediction етror as the average I, Le-osssnstitctoon error over all the augmentations. We note that for Pix2pixHD (Wang et al. 2018), we only used the flip augmentation, as random cropping is not used during its training process. More generally, using augmentations at inference time that not used for training produces corrupted results. Therefore in case of Pix2pixHD, T only add augmentation, the horizontally flipped query image. In case of Pix2pix. T 100 was used. As can be seen in Table 4 utilizing augmentations on pix2pix enables us to improve the attack success rate by up 5%.
12

---

Under review as a conference paper at ICLR 2021
013
0:00
dificuityy score
0.6
0.36
difficuity score
0.44
0.34
015
0.15
0.0
0.05
0.05
train
train
0.00
0.00
40.
test
0.35
Loc
Lyoees
Luce
Lnse
(a) Pix2pix-Facadee
(b) Pix2pix-Maps255 Maps2sat
0.14
0.14
0 0.200
0200
0.12
dificuity score
difficuity score
0.75
0175
0.19
0.10
0.08
0.08
0.05
0.000
0.100
0.04
tral test
0.01
trann test
0.75
Train
0.055
train
test
test
0.55
0.10
0.15
0.20
0.25
0.25
0.10
6.15
0.20
0.1
9.2
0.3
0.0
0.1
Lyee
Lye
from
(c) Pix2pix- Pix2pix-Citysena Cityscapes
(d) Pix2pixHD- Facades
Figure 6: Using the membership score, subtracting the difficulty score from the reconstruction error, makes train and test sets better separated by a vertical line on the x axis. Pix2pixHI for Maps2sat and Cityscapes are presented in Fig.
Pix2pix Maps2sat
Pix2pixHD Maps2sat
Facades
Cityscapes
Facades
Cityscapes
Ours-base
66.62%
90.54%
82.23%
99.0 99.02%
99.89% 99.21 %
99. 19%
Ours-Aug
98.46%
92.21
87.81
99.09 %
99.91 %
Table 4: Augmentations improve the accuracy of our attack.
A.6 DEFENSES
Several defenses against membership inference attacks have been previously studied. As most pre- vious works on MIA targeted classification models, not all defenses can be applied to conditional generation models. One such defense is the argmax defense, in which the victim model returns only the predicted label, rather then the full probability vector.
Other defenses attempt reduce overfitting, which is highly correlated with MIA success rates. This can be done for example by changing the dropout or weight normalization ratios during the training of the victim model. Due to the computational resources required for training multiple models with different ratios, we do not evaluate our attack success against this defense, and refer the reader to Sec. A.4 for visualization of the effect of overfitting on our attack, by attacking the victim model at earlier stages of the training process.
Two additional possible defense mechanisms are the differential private SGD (DP-SGD) and Gauss defenses. In DPDSGD (Abadi et al. (2016), the commonly used stochastic gradient descent optimiza- tion algorithm is modified in order to provide a differentially private (Dwork et al. 2014) model. This is done by adding Gaussian noise to clipped gradients for each sample in every training batch. However, this comes at the cost of computational complexity and model generation quality. There exists trade-off between privacy and utility, in which the amount of added noise must be large enough to ensure privacy while not degrading the model's outputs to the point where the model is useless. Training deep model, such as conditional generation models, with DP-SGD is an unsta-
13

---

Under review as a conference paper at ICLR 2021
Single- sis-image difficuity (ours)
single-imi (el---maggeee difficuity (ours)
0.1
difficuity 50.45
0.5
0.20
supervised dirtccuity
0.34
0.44
0.55
0.33
0.2
0.32
0.10
0:1
0.10
0.3
0.66
trair
train
0.9
train (cort. 0.821
0.06
test
0.41)
0.320
0.29)
test
0.861
0.63)
0.00
0.8
6.20
025
0.00
0.20
Lnc
Luc
Loee
Loc
(a) Pix2pix- Facades
(b) Pix2pix-Man Maps2sat
single- sess-mmagg difficulty [ouns]
0.34
single-n 1el--mmageee diftccllty (ours)
supervised dirficuity,
0.35
supee 2at airiccus P
0.36
0.155
0.10
0.35
0.500
0125
0.32
0.04
Train (corr, 0.771
0.22
train (oorr. 0.231
tra toar.
0.49)
train (corr. - 0.27)
test toom. C.67)
text (com. 0.24)
tast [com. 0.20)
0.31
toxt (cor. 0.04)
605
0.10
0.55
5.00
0.25
0.05
510
0.15
0.00
0.2
0.1
02
0.3
0.1
0.2
0,3
Lyec
(c) Pix2pix- Cityspppss
(d) Hr2pixHD-Facadee
Figure 7: Comparison of the relation between the reconstruction error and the supervised difficulty score (right) to the relation between the reconstruction error and our self-supervised difficulty score (left). Maps2sat and Cityscapes presented in Fig. 5 As can be seen, score is better correlated to the reconstruction error.
1.00
0.9
0.99
ROCAUC
0.8
ROCAUC
0.98
0.7
0.97
Pix2plx, Facades
Pix2pbxHD, Facades
0.6
Pix2pix, Maps2sat
Pix2pixDD, Maps2sat
Pix2pix, Cityscapes
0.96
Pix2pixHD Cityscapes
0.0 0.1 0.2 0.3 0.4 0.5 0.6
0.7
0.9
1.0
0.1 0.2
0.4
0.5 α
0.8 0.9 1.0
a
(a) Pix2pix
(b) Pix2pixHD
Figure 8: Effect of a in Eq. 3 over the attack success.
ble process. We experimented with multiple common configurations, i.e. added noise ratios and maximal gradient clipping threshold, and were not able to find a configuration that yields visually satisfying results. Hence, although the DP-SGD defense is theoretically protecting the victim model against membership inference attacks, in practice find it to be impractical against our attack it results with total corruption of the victim model.
In the Gauss defense, add Gaussian noise the image generated by the victim model (Gimmer ct al.) 2019). This attempts to hide specific artifacts of the overfitted model. We evaluate our attack success function of different noise STD. Fig. 10 shows that considerable amount of noise,
14

---

Under review as a conference paper at ICLR 2021
1.0
0.9
ROCAUC
0.8
ROCAUC
0.8
0.7
Facades
Facades
0.6
Maps2sat
0.6
Maps2sat
Cityscapes
Cityscapes
0.5
100
150
200
50
100
150
200
#epochs
# #epochs
(a) Pix2pix
(b) Pix2pixHD
Figure Effect of overfitting on the attack success rate. (a) Pix2pix, (b) Pix2pixHD
which corrupts the generated image, required in order to have a significant effect over our attack success. Moreover, can be seen that even with large amounts of noise, our attack still manages to succeed much better than random guessing. This implies that our attack is robust to the Gauss defense.
1.00
0.95
0.90
ROCAUC
ROCAUC
0.96
0.85
0.94
0.80
0.92
0.75
Px2pix, Facades
0.90
Plx2ptxHD, Facades Plx2pixHD, Mapsžsat Pix2siXHD, Citycapees
0.70
Maps2sat
Pizpix, Ciyscapes
0.88
0.1 0.2 0.3 0.5 0.6 0.7 0.8 0.9 1.0
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 noise
noise
noise STD
noise STD
Pix2pix
(b) Pix2pixHD
Figure 10: Effect of Gauss defense on the attack success rate. (a) pix2pix, (b) Pix2pixHD
A.7 SHADOW MODEL TRAINING
After having selected N images, denoted as shadow _tain, which we used to train the shadow model and another N images, shadow_test, which are seen by the shadow model, we set the labels as:
r + shadow train [0, label(x) = 1, r + shadow test
(4)
The classifier C architecture and training procedure are similar He et al. (201)). For each image, we compute the structured loss map between the ground-truth image and the generated image, and at every epoch randomly crop 15 patches of size 90 90 from the structured loss map. We train ResNet-50 (He et al. 2016) from scratch on the 90 x 90 patches, modified for binary classification. We use batch size of 8, SGD optimizer, weight decay of 1e initial learning rate of 0.1 which reduces by a factor of 0.1 every 15 epochs. As previously mentioned, we do not evaluate this on the Facades dataset, due its size.
15

---

Under review as a conference paper at ICLR 2021
ROCAUC
3.5
ROCAUC
658
ROCAUL
0.79
Shadow
0.5
Snddow
Stadon
Shadow
0.6
#train images
#train images
étrain images
#train mages
(a) Pixppix-Maps2sat
(b) Pixtpix-Cityscapes
(c) PixppixHD-Mapsssat (d) PixppixDD-Cityscape
Figure 11: Effect of the number of training images on shadow-model-based attacks. The shadow- model-based attack depends on the number of images used, we outperform this approach with no additional data.
A.8
ADDITIONAL DATASETS
In In addition the Facades, Maps2sat and Cityscapes datasets, evaluated our attack the Edges2shoes dataset (Yu & Grauman 2014) and the CelebA dataset (Liu et al. 2015)
A.8.1 EDGES2SHOES
The Edges2shoes dataset Yu & Grauman 2014) consists of 50K train images and 200 test images. Due to the large training set, the Pix2pix model has shown less overfitting to the data, and generates results that visually inferior to the other datasets. The lack of overfitting results with attack not managing to do better then a random guess, due to the strong correlation between overfitting and membership inference attacks. As Pix2pixHD has larger capacity, managed to produce better results, in terms of visual quality, over this dataset. This also comes at the cost of more overfitting, as can be observed by the success of our attack over this model: 87 87.52%. This is an improvement of 5.5% from our base attack, using only the reconstruction error, which achieves 81.99%.
A.8.2 CELEBA
The CelebA dataset (Liu 2015) is large-scale face attributes dataset with more than 200K celebrity images. We evaluated our attack on the CelebA dataset - by randomly sampling 10K images, extracting face landmark (Bulat Tzimiropoulos 2017) and training a pix2pixHD model to generate real face images from facial landmarks, see Fig. [2 Our base attack, using reconstruction error alone, achieves a success rate of 98. 83%. By using our difficulty score, our attack further improves to success rate of 99.04%. Faces are considered to be sensitive information, and the ability to accurately determine if a specific facial image was used to train a conditional generative model is a serious and concerning privacy breach.
(a) Facial landmarks
(b) Generated image
Ground truth image
Figure 12: Evaluating our attack on the CelebA dataset, by training pix2pixHD model from facial landmarks to face images, achieves high success rate of 99.04%.
16

---

Under review as a conference paper at ICLR 2021
A.9
SUPERVISED IMAGE DIFFICULTY SCORE
In Fig. 13 we present images ranked from hard to difficult using our implementation of the supervised-imaae difficulty score by Tudor Ionescu al. (2016), for the Cityscapes and Maps datasets. The ranking seems correlated with image sharpness and level-of-detail images.
Figure 13: Examples of images from the Cityscapes (first two rows) and Maps2sat (last two rows) datasets that received the lowest (first and third row) and highest (second and last row) difficulty scores using the supervised difficulty score.
A.10 IMAGENET DIFFICULTY SCORE
Our difficulty score relies learning mapping between feature vectors their corresponding pixel values. We use pre-trained Wide-ResNet50x2 (Zagoruyko & Komodakis (016), which is trained on the ImageNet dataset. We do not make any assumptions regarding overlap between the pre-trained model's training data (i.e. ImageNet) and the data during in the attack. In the scenario which such overlap exists, the concern is that the difficulty score would lose its credibility.
In order to verify this, we computed the difficulty score random subset of 1K train images and 1K test images, from the ImageNet dataset. We do not observe any significant difference between the two both share similar mean and std values: (0.0549, 0.018) for the train images and (0.0556, 0.0191) for the images. A ROCAUC score of 51% further demonstrates that there is no clear difference between the distribution of the difficulty score on seen and unseen images.
Fig. 14 further demonstrates this. The first row presents the train images that received the lowest scores. i.c. marked as easy images, and the second row presents the test images with the lowest scores. Both correspond "plain" images, regardless of whether they are known (train) unknown (test). The same applies to the Difficult images. The third row presents the highest scored train images and the last row presents the highest scored test images. Both contains complex patterns and high variance. This demonstrates that the difficulty score is not affected by the having prior knowledge of the image, and is only measuring the amount of variance and complexity of an image.
A.11 CONDITIONAL DIFFICULTY SCORE
We evaluated the effect of conditioning the difficulty score computation on the data the victim model conditioned on, i.e. training the regression model to predict pixel values from feature vectors extracted from the input the conditional generative model. Both methods achieve very similar results on the Cityscapes dataset and pix2pixHD model: 99.02% when training on ground-truth vs. 98.82% when training the segmentation mask. As the results are similar, with the ground-truth method being slightly superior, we chose to not be dependent of the input image and only use the ground truth image estimate the image's difficulty.
17

---

Under review as a conference paper at ICLR 2021
LAR
LAF
0
Figure 14: Examples of images from the ImageNet dataset that received the lowest and highest difficulty scores. First row lowest scored train images. Second row lowest scored test images. Third row highest scored train images. Last row highest scored test images. As can be seen, the difficulty score is effective even on images that were used for training the feature extractor.
A.12 EFFECT OF DATA PARTITION IN DIFFICULTY SCORE
We evaluated the effect of the different partitions in the training of our difficulty score's regression model. the Cityscapes dataset and pix2pixHD model. As be seen in Fig. [5 training less than 50% of the image pixels results with unstable performance, while using 50% or above is sufficient achieve high success rates, with preference to specific value.
0.975
ROCAUC
0.900
0.850
0.825
Pix2px, Citysrapes
1090 20/50 20/70 4bito 50500 savo 10330 8320 90200
train/test ratio
Figure 15: Effect of different data partitions in the training of the difficulty score's 's regression model, evaluated on the Cityscapes dataset and piX2pixHD model. Using less then 50% of the image pixels results with unstable performance, while all values of 50% or above result in similar attack success rates.
18

---

Under review as a conference paper at ICLR 2021
A.13 EFFECT OF RESIZE METHOD
In the computation of the difficulty score, the ground truth image is resized using bicubic interpola- tion an image of size 56 56 in order match the 56 56 feature vectors extracted by concate- nating the first 4 blocks of a Wide-ResNet50x2 (Zagoruyko & Komodakis 2016). We compared the effect of using different resize methods on the Cityscapes dataset for the pix2pixHD model. As can be seen in Fig. 16 the attack success rate is not very sensitive to the resize method.
1.22
1,80
0.85
0.88
0.015
0.26
1Debuit
Figure 16: We evaluated the effect of different resize methods in the difficulty score on Cityscapes dataset and pix2pixHD model. The attack success rate is not very sensitive the size method.
A.14 EFFECT OF DIFFICULTY SCORE ON CONDITIONAL GENERATIVE MODEL TRAINING
We conducted initial experiments to explore the effect of using the difficulty score during the training process of the conditional generative model. We experimented with training pix2pixHD the Cityscapes dataset while incorporating our difficulty score in the generator loss term.
The generator loss term, L. is composed of three terms: LGAN the GAN loss term, LFM similarity between features of the generator, and LVGG perceptual similarity. We experimented with multiplying each one of these loss term with the difficulty score and multiplying all three term with the difficulty score, i.c. Lg. We compared the effect on our attack success rate, as a function of the progress in the training process, which we use to approximate different levels of overfitting.
Multiplying the loss value by the difficulty score encourages the generator to pay more attention to difficult images. As can be seen in Fig. 17 when this applied on the GAN related loss terms, i.c LGAN and LFM, the model's overfitting slows down (we observe this by the slow down in highly correlated with overfitting) This happens as we pay more
the success of our attack, attention to the generator success against the discriminator and not to the reconstruction, which is mainly affected by LVGG.
19

---

Under review as a conference paper at ICLR 2021
1.0
0.9
ROCAUC
0.8
Orig loss
0.7
Lg
LGAN
0.6
Lsx
LvGG
0.5
50
100
150
200
#epochs
Figure 17: We evaluated the effect of incorporating our difficulty score in the training process of the conditional generative model. This encourages the model to pay more attention difficult images. When we incorporate the difficulty score with the GAN related loss terms, the model's o's overfitting slows down, demonstrated by the decrease our attack success rate.
20


Reviews:
[{"review_id": "mur9FirnXVq", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This submission focuses on the problem of membership inference attack for conditional image synthesis. This is an interesting new application and a meaningful extension of the topic of MIA. It is straightforward to use reconstruction loss as a measure for membership error. However, the authors pointed out that on top of reconstruction loss, the difficulty for the sample should also be taken into account. With that the authors propose a novel method of measuring generation difficulty for each sample by training a regressor to estimate pixel value using the abstract representation by a pre-trained model. The regressor is trained using a portion of randomly selected pixels and tested on the rest. The error of the prediction is use as the indicator of difficulty. This is an intuitive method and from the results of the experiment, the proposed method seems to be quite effective. The reviewer would like the author to clarify a few points: 1) is it an assumption that the pre-trained network has never seen any data used in the testing? If not, what would happen if the data has been seen when training the “feature extractor” but not the victim model? 2) Wouldn’t it be more application/problem-specific to train a regressor using the data the victim model is conditioned on (e.g. the segmentation mask) to predict the pixel values of the ground-truth image in estimating sample difficulty? ", "extra_scores": {}, "extra_text": {}}, {"review_id": "KUZ6gTcvi4-", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper studies membership inference attacks (MIA) in the context of conditional image generation models, i.e., models learned for image-to-image translation. The problem aims to detect if a data sample is used to train a neural network model. The paper observes that some images are \"universally\" easy/difficult to reconstruct, and states using reconstruction error alone is less effective at discriminating between difficult images used in training and easy images that are never seen before. The paper designs a difficulty score to augment reconstruction error based method, and shows MIA results on multiple datasets.\n\n\nThe problem of MIA is meaningful in terms of detecting leakage of privacy-sensitive data, as stated in the introduction. But the motivation of this paper is not clear. The paper lacks an explanation why studying MIA in the context of image-to-image translation.\n\nIn the second paragraph of introduction, it is not clear either why MIA can \"provide insights on the degree of overfitting in the victim mode\"?\n\nSection 3.2: What is the rationale behind the designed function Eq. (2) for measuring the difficulty of an image? The paper resizes all images into 56x56. Does this introduce noises that influences difficulty measurement? What is the resizing methods? Does different resizing methods have different performance?\n\nEq.(3): The paper considers \"difficult\" training images and \"easy\" unseen images, but does not consider \"difficult\" unseen images. For the unseen images that are difficult to reconstruction, does Eq.(3) still work?\n\nTable 3 does not seem complete. Why are there not ROC and ACC for \"ours\"? What do the numbers mean under \"ours\"?\n\nAs the paper mentioned in related work that some prior work also studies MIA in the context of generative models (GANs and VAEs), it is natural to think what the unique conclusions through the study presented in this paper?\n\nFigure 5: How to tell if \"difficult score\" is better correlated to the reconstrunction error than \"supervised difficulty\"?\n\n--------\npost-rebuttal\n--------\n\nI appreciate that authors have provided rebuttal that addresses many of my questions. I've read the updated paper and other reviewers' comments. The rebuttal has largely addressed my questions. However I am still not convinced by the following. Without a clarification I cannot see the significance of this work. So I'd maintain my rating. \n\nI still don't get the motivation why studying MIA in image-to-image translation. I really wanted to solicit a motivational example. The authors merely say \"image-to-image translation has yet to be studied in the context of MIA.\" I'm not convinced by this.\n\nA.4 The authors simply draw curves of MIA performance vs. learning epoch. I still don't know where overfitting happens. From the context, it seems that the proposed method will fail if the victim model is not trained to be overfitting. In practice, surely victim models are not overfitting?", "extra_scores": {}, "extra_text": {}}, {"review_id": "EhVXWmPoK2m", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper aims to solve the problem of membership attack, ie detecting if data samples were used to train a neural network for conditional image generation. The paper first proposes a simple but effective approach by using the reconstruction error. To address the issue that reconstruction error alone is less effective at discriminating between difficult images used in training and easy images that were never seen before, a difficulty score is further proposed, which can be computed for each image, and its computation does not require a training set. The eventual membership error, obtained by subtracting the difficulty score from the reconstruction error, is shown to be effective on several datasets.\n\nMerits:\n+ The paper is well written. I can easily get the main idea of the paper. And this work is well-motivated: some images are universally easy, and others are difficult.\n+ The proposed method looks simple and largely reasonable and is consistent with the motivation.\n+ Experiments have shown good results in terms of segmentation-to-image translation. The ablation study shows the effectiveness of the proposed difficulty score.\n\nIssues:\n- The novelty of the proposed algorithm is limited: the reconstruction error is quite standard, and the difficulty score is not well explained. I wonder why the difficulty score is designed as in (2). Please provide more thorough explanations and clarify how it differs from previous approaches.\n- While the title of the paper is \"conditional generative models\", the experiments are only conducted on segmentation-to-image translation. It will be important to show how it works in other related applications.\n- The membership error relies on a hyper-parameter $\\alpha$ in (3). Why is this parameter universally good for different datasets? Could the author provide more insights into tuning $\\alpha$ for different tasks?\n\n--------------------------------------------------------------------\nI have read the response, and my rating is not changed.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "DtE754P_yE", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper explores the MIA in the scenario of I2I translation and proposed a difficulty scoring function by observing that simply borrowing the MIA techniques from classification task is less effective. The scoring function is to measures the success of a linear regression model trained on part of pixels to predict the unseen pixel values. It is mainly to distinguish difficult-trained vs. easy-untrained examples. The improvement on performance brought by this simple function is obvious and comparisons are comprehensive. Below are some of my concerns and suggestions:\n\n(1) The data used for training the regression model is the same with the one used for training the I2I model? Does it mean we still need to access the original data after I2I model pre-training?\n\n(2) Is the 70/30 partition of pixels for training the regression model working the best? It will be better to analyze this partition ratio and see how the MIA accuracy will be affected.\n\n(3) Generally if the pre-trained model is more generalizable (e.g., trained with more data), I feel the MIA will be less useful. An interesting direction is to explore whether it will be more useful in few-shot case, not only to detect whether overfitting happens, but also to explore how to use MIA to avoid overfitting and make the model trained with less data more robust.\n\n========\n\nUpdate:\n\nThanks for the author's feedback that clarifies my concerns. Adding the difficulty score into training looks like a more promising future direction. ", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The work focuses on detecting whether a certain data sample was used to train a deep network-based conditional image synthesis model. The key idea is not to rely on just reconstruction error but normalizing it via a proposed difficulty score. The reviewers found the problem statement important and the paper easy to follow. However, a common concern in the discussion was the approach is specific to image-to-image translation problems. Many other minor questions were answered by the authors in the rebuttal. Upon discussion post rebuttal, the reviewers decided to maintain their score. AC and reviewers believe that the paper will benefit from better analysis and description of difficulty score and its correlation with reconstruction error. It would be ideal to see how these ideas are useful for a broader problem than image translation. Please refer to the reviews for final feedback and suggestions to strengthen the submission.
Decision: Reject