Original: M EMBERSHIP A TTACKS ON C ONDITIONAL G ENERATIVE M ODELS U SING I MAGE D IFFICULTY
Revised: M EMBERSHIP A TTACKS ON C ONDITIONAL G ENERATIVE M ODELS U SING I MAGE D IFFICULTY

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Membership inference attacks (MIA) try to detect if data samples were used to train a neural network model.
Revised: Membership inference attacks (MIA) try to detect if data samples were used to train a neural network model.

Original: As training data is very valuable in machine learning, MIA can be used to detect the use of unauthorized data.
Revised: As training data is very valuable in machine learning, MIA can be used to detect the use of unauthorized data.

Original: Unlike the traditional MIA approaches, addressing classiﬁcation models, we address conditional image generation models (e.g. image translation).
Revised: Unlike the traditional MIA approaches, addressing classiﬁcation models, we address conditional image generation models (e.g. image translation).

Original: Due to overﬁtting, reconstruction errors are typically lower for images used in training.
Revised: Due to overﬁtting, reconstruction errors are typically lower for images used in training.

Original: A simple but effective approach for membership attacks can therefore use the reconstruction error.
Revised: A simple but effective approach for membership attacks can therefore use the reconstruction error.

Original: However, we observe that some images are ”universally” easy, and others are difﬁcult.
Revised: However, we observe that some images are ”universally” easy, and others are difﬁcult.

Original: Reconstruction error alone is less effective at discriminating between difﬁcult images used in training and easy images that were never seen before.
Revised: Reconstruction error alone is less effective at discriminating between difﬁcult images used in training and easy images that were never seen before.

Original: To overcome this, we propose to use a novel difﬁculty score that can be computed for each image, and its computation does not require a training set.
Revised: To overcome this, we propose to use a novel difﬁculty score that can be computed for each image, and its computation does not require a training set.

Original: Our membership error, obtained by subtracting the difﬁculty score from the reconstruction error, is shown to achieve high MIA accuracy on an extensive number of benchmarks.
Revised: Our membership error, obtained by subtracting the difﬁculty score from the reconstruction error, is shown to achieve high MIA accuracy on an extensive number of benchmarks.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Deep neural networks have been widely adopted in various computer vision tasks, e.g. image classiﬁcation, semantic segmentation, image translation and generation etc.
Revised: Deep neural networks have been widely adopted in various computer vision tasks, e.g. image classiﬁcation, semantic segmentation, image translation and generation etc.

Original: The high sample-complexity of such models requires large amounts of training data.
Revised: The high sample-complexity of such models requires large amounts of training data.

Original: However, obtaining many training images might not be an easy task.
Revised: However, obtaining many training images might not be an easy task.

Original: In fact, collection and annotation is often an expensive and labor intensive process.
Revised: In fact, collection and annotation is often an expensive and labor intensive process.

Original: In some domains, such as medical imaging, publicly available training data are particularly scarce due to privacy concerns.
Revised: In some domains, such as medical imaging, publicly available training data are particularly scarce due to privacy concerns.

Original: In such settings, it is common to grant access to private sensitive data for training purposes alone, while ensuring not to reveal the data in the inference stage.
Revised: In such settings, it is common to grant access to private sensitive data for training purposes alone, while ensuring not to reveal the data in the inference stage.

Original: A common solution is training the model privately and then providing black-box access to the trained model.
Revised: A common solution is training the model privately and then providing black-box access to the trained model.

Original: However, even black-box access may leak sensitive information about the training data.
Revised: However, even black-box access may leak sensitive information about the training data.

Original: Membership inference attacks (MIA) are one way to detect such leakage.
Revised: Membership inference attacks (MIA) are one way to detect such leakage.

Original: Given access to a data sample, an attacker attempts to ﬁnd whether or not the sample was used in the training process.
Revised: Given access to a data sample, an attacker attempts to ﬁnd whether or not the sample was used in the training process.

Original: MIA attacks have been widely studied for image classiﬁcation models, achieving high success rates (Shokri et al., 2017; Salem et al., 2018;
Revised: MIA attacks have been widely studied for image classiﬁcation models, achieving high success rates (Shokri et al., 2017; Salem et al., 2018;

Original: Sablayrolles et al., 2019; Yeom et al., 2018; Li & Zhang, 2020; Choo et al., 2020).
Revised: Sablayrolles et al., 2019; Yeom et al., 2018; Li & Zhang, 2020; Choo et al., 2020).

Original: Due to overﬁtting in deep neural networks, prediction conﬁdence tends to be higher for images used in training.
Revised: Due to overﬁtting in deep neural networks, prediction conﬁdence tends to be higher for images used in training.

Original: This difference in prediction conﬁdence helps MIA methods to successfully determine which images were used for training.
Revised: This difference in prediction conﬁdence helps MIA methods to successfully determine which images were used for training.

Original: Therefore, in addition to detecting information leakage, MIA also provide insights on the degree of overﬁtting in the victim model.
Revised: Therefore, in addition to detecting information leakage, MIA also provide insights on the degree of overﬁtting in the victim model.

Original: We address MIA for a new domain - conditional image generation models, e.g. image translation.
Revised: We address MIA for a new domain - conditional image generation models, e.g. image translation.

Original: While classiﬁcation models give a probability vector over possible classes, generation models give a single color for every pixel.
Revised: While classiﬁcation models give a probability vector over possible classes, generation models give a single color for every pixel.

Original: We propose a MIA that uses pixel-wise reconstruction error, as overﬁtting causes lower reconstruction error on images used for training.
Revised: We propose a MIA that uses pixel-wise reconstruction error, as overﬁtting causes lower reconstruction error on images used for training.

Original: But we observe that some images are ”universally” easy, and others are universally difﬁcult.
Revised: But we observe that some images are ”universally” easy, and others are universally difﬁcult.

Original: Reconstruction error alone is therefore less accurate at discriminating between difﬁcult images used in training and previously unseen easy images.
Revised: Reconstruction error alone is therefore less accurate at discriminating between difﬁcult images used in training and previously unseen easy images.

Original: To overcome this limitation, we add a novel image difﬁculty score which is computed for each query image.
Revised: To overcome this limitation, we add a novel image difﬁculty score which is computed for each query image.

Original: Our image difﬁculty score uses the accuracy of a linear predictor computed over a given image, predicting pixel values from deep features of that image.
Revised: Our image difﬁculty score uses the accuracy of a linear predictor computed over a given image, predicting pixel values from deep features of that image.

Original: The reconstruction error together with the difﬁculty score helps to discriminate between two factors of variation in the reconstruction error, namely (i) The ”intrinsic” difﬁculty of the conditional generation task for each image, based on its difﬁculty score and (ii)
Revised: The reconstruction error together with the difﬁculty score helps to discriminate between two factors of variation in the reconstruction error, namely (i) The ”intrinsic” difﬁculty of the conditional generation task for each image, based on its difﬁculty score and (ii)

Original: The boost in accuracy due to overﬁtting to the training images.
Revised: The boost in accuracy due to overﬁtting to the training images.

Original: Deﬁning a membership error that subtracts the difﬁculty score from the reconstruction error is shown empirically to achieve high success rates in MIA.
Revised: Deﬁning a membership error that subtracts the difﬁculty score from the reconstruction error is shown empirically to achieve high success rates in MIA.

Original: Differently from other MIA approaches, we do not assume the existence of a large number of in-distribution data samples for training a shadow model - but rather operate on merely a single image.
Revised: Differently from other MIA approaches, we do not assume the existence of a large number of in-distribution data samples for training a shadow model - but rather operate on merely a single image.

Original: Our method is evaluated on an extensive number of benchmarks demonstrating its effectiveness compared to strong baseline methods.
Revised: Our method is evaluated on an extensive number of benchmarks demonstrating its effectiveness compared to strong baseline methods.

Original:  R ELATED W ORK 2. M EMBERSHIP I NFERENCE A TTACKS (MIA)
Revised:  R ELATED W ORK 2. M EMBERSHIP I NFERENCE A TTACKS (MIA)

Original: Shokri et al.
Revised: Shokri et al.

Original: (2017) were the ﬁrst to study MIA against classiﬁcation models in a black-box setting.
Revised: (2017) were the ﬁrst to study MIA against classiﬁcation models in a black-box setting.

Original: In black-box setting the attacker can only send queries to the victim model and get the full probability vector response, without being exposed to the model itself.
Revised: In black-box setting the attacker can only send queries to the victim model and get the full probability vector response, without being exposed to the model itself.

Original: They proposed to train multiple shadow models to mimic the behavior of the victim model, and then use those to train a binary classiﬁer to distinguish between known samples from the train set and unknown samples.
Revised: They proposed to train multiple shadow models to mimic the behavior of the victim model, and then use those to train a binary classiﬁer to distinguish between known samples from the train set and unknown samples.

Original: They assume the existence of in-distribution new training data and knowledge of the victim model architecture.
Revised: They assume the existence of in-distribution new training data and knowledge of the victim model architecture.

Original: Salem et al.
Revised: Salem et al.

Original: (2018) further relaxed those assumptions and demonstrated that using only one shadow model is sufﬁcient for a successful attack, and proposed using out-of-distribution dataset and different shadow model architectures, for a slightly inferior attack.
Revised: (2018) further relaxed those assumptions and demonstrated that using only one shadow model is sufﬁcient for a successful attack, and proposed using out-of-distribution dataset and different shadow model architectures, for a slightly inferior attack.

Original: Even more interestingly, they showed that without any training, a simple threshold on the victim model’s conﬁdence score is sufﬁcient.
Revised: Even more interestingly, they showed that without any training, a simple threshold on the victim model’s conﬁdence score is sufﬁcient.

Original: This shows that classiﬁcation models are more conﬁdent of samples that appeared in the training process, compared to unseen samples.
Revised: This shows that classiﬁcation models are more conﬁdent of samples that appeared in the training process, compared to unseen samples.

Original: Sablayrolles et al.
Revised: Sablayrolles et al.

Original: (2019) proposed an attack based on applying a threshold over the loss value rather then the conﬁdence and showed that black-box attacks are as good as white-box attacks.
Revised: (2019) proposed an attack based on applying a threshold over the loss value rather then the conﬁdence and showed that black-box attacks are as good as white-box attacks.

Original: As the naive defense against such attacks is to modify the victim model’s API to only output the predicted label, other works proposed label-only attacks (Yeom et al., 2018; Li & Zhang, 2020; Choo et al., 2020).
Revised: As the naive defense against such attacks is to modify the victim model’s API to only output the predicted label, other works proposed label-only attacks (Yeom et al., 2018; Li & Zhang, 2020; Choo et al., 2020).

Original: While most previous work has been around classiﬁcation models, there has been some effort regarding MIA on generative models such as GANs and VAEs (Chen et al., 2019; Hayes et al., 2019;
Revised: While most previous work has been around classiﬁcation models, there has been some effort regarding MIA on generative models such as GANs and VAEs (Chen et al., 2019; Hayes et al., 2019;

Original: Hilprecht et al., 2019).
Revised: Hilprecht et al., 2019).

Original: An attack against semantic segmentation models was proposed by He et al.
Revised: An attack against semantic segmentation models was proposed by He et al.

Original: (2019), where a shadow semantic segmentation model is trained, and is used to train a binary classiﬁer.
Revised: (2019), where a shadow semantic segmentation model is trained, and is used to train a binary classiﬁer.

Original: The classiﬁer is trained on image patches, and the ﬁnal decision regarding the query image is set by the aggregation of the per-patch classiﬁcation scores.
Revised: The classiﬁer is trained on image patches, and the ﬁnal decision regarding the query image is set by the aggregation of the per-patch classiﬁcation scores.

Original: The input to the classiﬁer is a structured loss map between the shadow model’s output and the ground truth segmentation map.
Revised: The input to the classiﬁer is a structured loss map between the shadow model’s output and the ground truth segmentation map.

Original: Although this task is the closest to ours, our work is the ﬁrst study of membership inference attacks on conditional image generation model.
Revised: Although this task is the closest to ours, our work is the ﬁrst study of membership inference attacks on conditional image generation model.

Original: Besides membership inference attacks, other privacy attacks against neural networks exist.
Revised: Besides membership inference attacks, other privacy attacks against neural networks exist.

Original: We refer the reader to Sec.
Revised: We refer the reader to Sec.

Original: A.1 for more details of such attacks.
Revised: A.1 for more details of such attacks.

Original: 2. C ONDITIONAL I MAGE G ENERATION
Revised: 2. C ONDITIONAL I MAGE G ENERATION

Original: Image-to-image translation is the task of mapping an image from a source domain to a target domain, while preserving the semantic and geometric content of the input image.
Revised: Image-to-image translation is the task of mapping an image from a source domain to a target domain, while preserving the semantic and geometric content of the input image.

Original: Over the last decade, with the advent of deep neural network models and increasing dataset sizes, signiﬁcant progress was made in this ﬁeld.
Revised: Over the last decade, with the advent of deep neural network models and increasing dataset sizes, signiﬁcant progress was made in this ﬁeld.

Original: Currently, the most popular methods for training image-to-image translation models use Generative Adversarial Networks (GANs) (Goodfellow et al., 2014) and are currently used in two main scenarios: (i) unsupervised image translation between domains (Zhu et al., 2017a;
Revised: Currently, the most popular methods for training image-to-image translation models use Generative Adversarial Networks (GANs) (Goodfellow et al., 2014) and are currently used in two main scenarios: (i) unsupervised image translation between domains (Zhu et al., 2017a;

Original: Kim et al., 2017; Liu et al., 2017; Choi et al., 2018); (ii) serving as a perceptual image loss function (Isola et al., 2017; Wang et al., 2018; Zhu et al., 2017b).
Revised: Kim et al., 2017; Liu et al., 2017; Choi et al., 2018); (ii) serving as a perceptual image loss function (Isola et al., 2017; Wang et al., 2018; Zhu et al., 2017b).

Original: In this work we introduce the novel task of
Revised: In this work we introduce the novel task of

Original: MIA on conditional image generation models.
Revised: MIA on conditional image generation models.

Original:  MIA ON C ONDITIONAL I MAGE G ENERATION M ODELS
Revised:  MIA ON C ONDITIONAL I MAGE G ENERATION M ODELS

Original: In membership inference attacks (MIA), an adversary attacks a victim model by attempting to infer whether a query data sample was used to train a victim model.
Revised: In membership inference attacks (MIA), an adversary attacks a victim model by attempting to infer whether a query data sample was used to train a victim model.

Original: Such attacks exploit overﬁtting to the training data performed by the victim model.
Revised: Such attacks exploit overﬁtting to the training data performed by the victim model.

Original: For classiﬁcation, an overﬁtted model will likely be more conﬁdent of the prediction for data samples that were included in the training set.
Revised: For classiﬁcation, an overﬁtted model will likely be more conﬁdent of the prediction for data samples that were included in the training set.

Original: We focus on MIA in a new domain: conditional image generation models, e.g. image translation.
Revised: We focus on MIA in a new domain: conditional image generation models, e.g. image translation.

Original: We propose a simple and effective attack.
Revised: We propose a simple and effective attack.

Original: Differently from most previous works (Shokri et al., 2017;
Revised: Differently from most previous works (Shokri et al., 2017;

Original: He et al., 2019), we do not use shadow models or train a binary classiﬁer, and thus do not require any additional training data.
Revised: He et al., 2019), we do not use shadow models or train a binary classiﬁer, and thus do not require any additional training data.

Original: We assume the most restrictive attack setting, where the attacker only has black-box access to the victim model V , and has no knowledge of its weights and architecture.
Revised: We assume the most restrictive attack setting, where the attacker only has black-box access to the victim model V , and has no knowledge of its weights and architecture.

Original: Our membership attack is performed on a pair of query images ( x, y ) where x is an image from the input domain and y is the ground truth from the output domain.
Revised: Our membership attack is performed on a pair of query images ( x, y ) where x is an image from the input domain and y is the ground truth from the output domain.

Original: The existence of the ground truth image y is in-line with previous works, and is a reasonable assumption in conditional generative models.
Revised: The existence of the ground truth image y is in-line with previous works, and is a reasonable assumption in conditional generative models.

Original: For each query we compute a membership error, L mem (see Eq.
Revised: For each query we compute a membership error, L mem (see Eq.

Original: (3)), to which we apply a pre-deﬁned threshold τ ,
Revised: (3)), to which we apply a pre-deﬁned threshold τ ,

Original: such that all queries where L mem ( x, y ) <
Revised: such that all queries where L mem ( x, y ) <

Original: τ are marked as members of the training data.
Revised: τ are marked as members of the training data.

Original: The membership error has two elements: reconstruction error and difﬁculty score.
Revised: The membership error has two elements: reconstruction error and difﬁculty score.

Original: 3. R ECONSTRUCTION E RROR FOR M EMBERSHIP E VALUATION
Revised: 3. R ECONSTRUCTION E RROR FOR M EMBERSHIP E VALUATION

Original: Typical MIA that operate on classiﬁcation models consider the probability (or conﬁdence) given by the model to the correct class.
Revised: Typical MIA that operate on classiﬁcation models consider the probability (or conﬁdence) given by the model to the correct class.

Original: Image generation models are different as they output a color value of each pixel.
Revised: Image generation models are different as they output a color value of each pixel.

Original: This value is the maximum likelihood estimate, and no probability distribution over possible values is given.
Revised: This value is the maximum likelihood estimate, and no probability distribution over possible values is given.

Original: As the conﬁdence of the model prediction is unknown, we propose to examine the reconstruction error of the model over all pixels.
Revised: As the conﬁdence of the model prediction is unknown, we propose to examine the reconstruction error of the model over all pixels.

Original: The reconstruction error is described in Eq.
Revised: The reconstruction error is described in Eq.

Original: (1), and is computed using the L 1 error between the output image predicted by a black-box access to the model, V ( x ) , and the ground truth image y .
Revised: (1), and is computed using the L 1 error between the output image predicted by a black-box access to the model, V ( x ) , and the ground truth image y .

Original: We show that for images in the training set, the model output has mostly lower prediction errors compared to unknown images.
Revised: We show that for images in the training set, the model output has mostly lower prediction errors compared to unknown images.

Original: See Table 1.
Revised: See Table 1.

Original: [Equation]
Revised: [Equation]

Original: 3. E ASY I MAGES AND D IFFICULT I MAGES
Revised: 3. E ASY I MAGES AND D IFFICULT I MAGES

Original: In this section, we tackle the following question: Given an image, compute a score that measures how ”difﬁcult” it is to synthesize it.
Revised: In this section, we tackle the following question: Given an image, compute a score that measures how ”difﬁcult” it is to synthesize it.

Original: Consider, for example, the task of supervised segmentation-toimage translation.
Revised: Consider, for example, the task of supervised segmentation-toimage translation.

Original: I.e. the task is to ”invert” the segmentation process, and recover the original image that gave rise to a given segmentation map.
Revised: I.e. the task is to ”invert” the segmentation process, and recover the original image that gave rise to a given segmentation map.

Original: It is clear that not all images are equally difﬁcult: (i) more difﬁcult images have sharp and detailed textures whereas simpler images have blurrier textures; (ii) images with semantic segmentation maps that contain only few categories provide less guidance than those with more detailed segmentation maps, making the correct prediction less certain.
Revised: It is clear that not all images are equally difﬁcult: (i) more difﬁcult images have sharp and detailed textures whereas simpler images have blurrier textures; (ii) images with semantic segmentation maps that contain only few categories provide less guidance than those with more detailed segmentation maps, making the correct prediction less certain.

Original: Image difﬁculty score should quantify these difﬁculties.
Revised: Image difﬁculty score should quantify these difﬁculties.

Original: In Sec.
Revised: In Sec.

Original: 4.1 we show that such a difﬁculty score is important to increase accuracy of membership inference attacks on conditional generative models.
Revised: 4.1 we show that such a difﬁculty score is important to increase accuracy of membership inference attacks on conditional generative models.

Original: We brieﬂy describe two previous approaches for measuring image difﬁculty:
Revised: We brieﬂy describe two previous approaches for measuring image difﬁculty:

Original: Human-Supervised: Tudor Ionescu et al.
Revised: Human-Supervised: Tudor Ionescu et al.

Original: (2016) proposed to deﬁne image difﬁculty as the human response time for solving a visual search task.
Revised: (2016) proposed to deﬁne image difﬁculty as the human response time for solving a visual search task.

Original: For this, they collected human annotations for the
Revised: For this, they collected human annotations for the

Original: PASCAL VOC 2012 dataset (Everingham et al., 2010) and trained a regression model, based on pre-trained deep features, to predict the collected difﬁculty score.
Revised: PASCAL VOC 2012 dataset (Everingham et al., 2010) and trained a regression model, based on pre-trained deep features, to predict the collected difﬁculty score.

Original: The disadvantage of this method is that human-speciﬁed difﬁculty scores may not correlate to the difﬁculty of image synthesis by neural networks.
Revised: The disadvantage of this method is that human-speciﬁed difﬁculty scores may not correlate to the difﬁculty of image synthesis by neural networks.

Original: This is demonstrated empirically in Sec. 4.3.
Revised: This is demonstrated empirically in Sec. 4.3.

Original: Multi-Image: Another approach taken by Chen et al.
Revised: Multi-Image: Another approach taken by Chen et al.

Original: (2019) is training a generative model on a set of images similar to the target image distribution.
Revised: (2019) is training a generative model on a set of images similar to the target image distribution.

Original: Consider for example training an autoencoder on a set of external images.
Revised: Consider for example training an autoencoder on a set of external images.

Original: This approach uses the reconstruction error on the target image as its difﬁculty score - larger reconstruction errors correspond to more difﬁcult images.
Revised: This approach uses the reconstruction error on the target image as its difﬁculty score - larger reconstruction errors correspond to more difﬁcult images.

Original: This approach has a signiﬁcant drawback: a large number of images, similar to the target image, are required in order to learn a reliable generative model.
Revised: This approach has a signiﬁcant drawback: a large number of images, similar to the target image, are required in order to learn a reliable generative model.

Original: In many cases, images from the target distribution may not be available.
Revised: In many cases, images from the target distribution may not be available.

Original: Additionally, training a model for every task is tedious and computationally expensive.
Revised: Additionally, training a model for every task is tedious and computationally expensive.

Original: Proposed - Single-image difﬁculty score: We propose a novel method to assign a difﬁculty score for image generation models.
Revised: Proposed - Single-image difﬁculty score: We propose a novel method to assign a difﬁculty score for image generation models.

Original: The difﬁculty score measures the success of a linear regression model to predict the pixel values from a high-level representation of the image.
Revised: The difﬁculty score measures the success of a linear regression model to predict the pixel values from a high-level representation of the image.

Original: A related approach was proposed by Hacohen & Weinshall (2019) for measuring image difﬁculty for classiﬁcation models.
Revised: A related approach was proposed by Hacohen & Weinshall (2019) for measuring image difﬁculty for classiﬁcation models.

Original: Our method is signiﬁcantly different as it is trained on a single image rather than a large dataset, and that it focuses on generation rather than classiﬁcation.
Revised: Our method is signiﬁcantly different as it is trained on a single image rather than a large dataset, and that it focuses on generation rather than classiﬁcation.

Original: Our features are the activation values in the ﬁrst 4 blocks of a pre-trained Wide-ResNet 50 ×(Zagoruyko & Komodakis, 2016), concatenated together, giving 56 × 56 feature vectors of size 3840 .
Revised: Our features are the activation values in the ﬁrst 4 blocks of a pre-trained Wide-ResNet 50 ×(Zagoruyko & Komodakis, 2016), concatenated together, giving 56 × 56 feature vectors of size 3840 .

Original: We reduce the input image to 56 × 56 to match the spatial dimension of the ﬁrst Wide-ResNet 50 ×block.
Revised: We reduce the input image to 56 × 56 to match the spatial dimension of the ﬁrst Wide-ResNet 50 ×block.

Original: We denote the concatenated feature vector for pixel i as ψ ( i ) .
Revised: We denote the concatenated feature vector for pixel i as ψ ( i ) .

Original: See Sec.
Revised: See Sec.

Original: A.2 for more details.
Revised: A.2 for more details.

Original: The linear regression model P is a matrix of size 3840 × 3 , multiplied with the feature vector ψ ( i ) of pixel i to give a linear estimate of the RGB colors
Revised: The linear regression model P is a matrix of size 3840 × 3 , multiplied with the feature vector ψ ( i ) of pixel i to give a linear estimate of the RGB colors

Original: y i .
Revised: y i .

Original: We minimize P over 70% randomly selected pixels.
Revised: We minimize P over 70% randomly selected pixels.

Original: The image difﬁculty score is the average absolute error over the 30% unselected pixels:
Revised: The image difﬁculty score is the average absolute error over the 30% unselected pixels:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: ) where y i is the ground truth value of the i th pixel in the resized ground truth image y .
Revised: ) where y i is the ground truth value of the i th pixel in the resized ground truth image y .

Original: Fig.
Revised: Fig.

Original: 1 presents examples of images that received the highest and lowest difﬁculty scores.
Revised: 1 presents examples of images that received the highest and lowest difﬁculty scores.

Original: 3. M EMBERSHIP E RROR
Revised: 3. M EMBERSHIP E RROR

Original: As observed before, some images are ”universally” easy to reconstruct, while others are universally difﬁcult to reconstruct.
Revised: As observed before, some images are ”universally” easy to reconstruct, while others are universally difﬁcult to reconstruct.

Original: While the reconstruction error in Eq.
Revised: While the reconstruction error in Eq.

Original: 1 achieves high MIA success rates, it has a signiﬁcant limitation - it does not discriminate between difﬁcult and easy samples.
Revised: 1 achieves high MIA success rates, it has a signiﬁcant limitation - it does not discriminate between difﬁcult and easy samples.

Original: Difﬁcult training samples might be more difﬁcult for the victim model to generate and therefore can receive high reconstruction error.
Revised: Difﬁcult training samples might be more difﬁcult for the victim model to generate and therefore can receive high reconstruction error.

Original: Similarly, an easy unknown sample can be generated by the victim model with lower error.
Revised: Similarly, an easy unknown sample can be generated by the victim model with lower error.

Original: Such cases can cause wrong classiﬁcation if only the reconstruction error is used.
Revised: Such cases can cause wrong classiﬁcation if only the reconstruction error is used.

Original: Given our image difﬁculty score L diff in Eq.
Revised: Given our image difﬁculty score L diff in Eq.

Original: (2) and the reconstruction error
Revised: (2) and the reconstruction error

Original: L rec Eq.
Revised: L rec Eq.

Original: (1) we calculate a membership error L mem as follows:
Revised: (1) we calculate a membership error L mem as follows:

Original: [Equation]
Revised: [Equation]

Original: L mem is computed by subtracting the difﬁculty score L diff from the reconstruction error
Revised: L mem is computed by subtracting the difﬁculty score L diff from the reconstruction error

Original: L rec weighted by α (unless speciﬁed otherwise, we use α = 0 . 5 ).
Revised: L rec weighted by α (unless speciﬁed otherwise, we use α = 0 . 5 ).

Original: This lowers the membership error
Revised: This lowers the membership error

Original: L mem for harder-to-predict images compared to easier-to-predict images having the same reconstruction error.
Revised: L mem for harder-to-predict images compared to easier-to-predict images having the same reconstruction error.

Original: See Fig.
Revised: See Fig.

Original: 2 for an overview illustration of our method.
Revised: 2 for an overview illustration of our method.

Original: Using the membership error L mem for MIA substantially improves the success rates in all of our experiments, as shown in Table 1 and Fig.
Revised: Using the membership error L mem for MIA substantially improves the success rates in all of our experiments, as shown in Table 1 and Fig.

Original: [Figure]
Revised: [Figure]

Original:  E XPERIMENTS AND C OMPARISONS
Revised:  E XPERIMENTS AND C OMPARISONS

Original: We conduct a thorough investigation demonstrating the effectiveness of our approach.
Revised: We conduct a thorough investigation demonstrating the effectiveness of our approach.

Original: First, we show its high success rate on various benchmarks.
Revised: First, we show its high success rate on various benchmarks.

Original: A comparison between our novel single-image, self-supervised, difﬁculty score to two alternative difﬁculty scores: a multi-image and a supervised difﬁculty scores, shows the superiority of our proposed method.
Revised: A comparison between our novel single-image, self-supervised, difﬁculty score to two alternative difﬁculty scores: a multi-image and a supervised difﬁculty scores, shows the superiority of our proposed method.

Original: We also compare our attack with the commonly used attack based on a shadow model and show that our attack is not only superior but also requires minimal assumptions over the attacker’s knowledge.
Revised: We also compare our attack with the commonly used attack based on a shadow model and show that our attack is not only superior but also requires minimal assumptions over the attacker’s knowledge.

Original: As MIA attacks are closely related to overﬁtting, we study the effect of overﬁtting on our attack success rate.
Revised: As MIA attacks are closely related to overﬁtting, we study the effect of overﬁtting on our attack success rate.

Original: Additional results as well as a discussion regarding possible defenses can be found in the appendix.
Revised: Additional results as well as a discussion regarding possible defenses can be found in the appendix.

Original: 4. I MAGE TRANSLATION MEMBERSHIP INFERENCE ATTACK
Revised: 4. I MAGE TRANSLATION MEMBERSHIP INFERENCE ATTACK

Original: We propose a novel membership attack on two popular image translation architectures - Pix2Pix (Isola et al., 2017) and Pix2PixHD (Wang et al., 2018), as well as three datasets - CMP Facades (Tyleˇcek & ˇ S´ara, 2013), Maps2sat (Isola et al., 2017) and Cityscapes (Cordts et al., 2016).
Revised: We propose a novel membership attack on two popular image translation architectures - Pix2Pix (Isola et al., 2017) and Pix2PixHD (Wang et al., 2018), as well as three datasets - CMP Facades (Tyleˇcek & ˇ S´ara, 2013), Maps2sat (Isola et al., 2017) and Cityscapes (Cordts et al., 2016).

Original: All models are trained from scratch, with the exception of the Cityscapes dataset on the Pix2pixHD architecture in which we use the supplied large pre-trained model for computational constraints on the high resolution.
Revised: All models are trained from scratch, with the exception of the Cityscapes dataset on the Pix2pixHD architecture in which we use the supplied large pre-trained model for computational constraints on the high resolution.

Original: In accordance with previous membership attack works, the success rate is measured using the ROC area under the curve (ROCAUC) metric.
Revised: In accordance with previous membership attack works, the success rate is measured using the ROC area under the curve (ROCAUC) metric.

Original: It can be seen in Table 1 that while using the reconstruction error alone achieves a high success rate, the membership error further improve these results by up to 6% .
Revised: It can be seen in Table 1 that while using the reconstruction error alone achieves a high success rate, the membership error further improve these results by up to 6% .

Original: Fig 3 demonstrates the effect of subtracting the difﬁculty score from the prediction error.
Revised: Fig 3 demonstrates the effect of subtracting the difﬁculty score from the prediction error.

Original: A single threshold on the membership error can separate train and test images.
Revised: A single threshold on the membership error can separate train and test images.

Original: For more results, see Fig.
Revised: For more results, see Fig.

Original: 6 in the appendix.
Revised: 6 in the appendix.

Original: We study the effect of utilizing common image augmentations, i.e. horizontal ﬂipping and random cropping, over the pair ( x, y ) in order to construct a larger set { ( x aug , y aug ) } and deﬁne L rec to be the average reconstruction error over the set.
Revised: We study the effect of utilizing common image augmentations, i.e. horizontal ﬂipping and random cropping, over the pair ( x, y ) in order to construct a larger set { ( x aug , y aug ) } and deﬁne L rec to be the average reconstruction error over the set.

Original: This can improve the accuracy by up to 5% on Pix2pix, see App.
Revised: This can improve the accuracy by up to 5% on Pix2pix, see App.

Original: A.5 for details.
Revised: A.5 for details.

Original: Possible defenses against our attack are discussed and evaluated in Sec.
Revised: Possible defenses against our attack are discussed and evaluated in Sec.

Original: A.6 in the appendix.
Revised: A.6 in the appendix.

Original: 4. S INGLE I MAGE D IFFICULTY S CORE
Revised: 4. S INGLE I MAGE D IFFICULTY S CORE

Original: In this section we compare our single-image difﬁculty score (Sec. 3.2) with a multi-image difﬁculty score, in which a ”shadow” model is trained on new data and deﬁne the difﬁculty score to be the
Revised: In this section we compare our single-image difﬁculty score (Sec. 3.2) with a multi-image difﬁculty score, in which a ”shadow” model is trained on new data and deﬁne the difﬁculty score to be the

Original: [Table]
Revised: [Table]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: the same distribution as the victim’s training data, by randomly sampling 100 images from the test set of the corresponding dataset.
Revised: the same distribution as the victim’s training data, by randomly sampling 100 images from the test set of the corresponding dataset.

Original: We did not perform this experiment on the Facades dataset as its test set does not contain a sufﬁcient number of samples.
Revised: We did not perform this experiment on the Facades dataset as its test set does not contain a sufﬁcient number of samples.

Original: The results are presented in Table 2.
Revised: The results are presented in Table 2.

Original: In the Pix2pixHD model, the multi-image model badly overﬁt to its training data and is not able to generalize well enough to perform as a difﬁculty measure for other images.
Revised: In the Pix2pixHD model, the multi-image model badly overﬁt to its training data and is not able to generalize well enough to perform as a difﬁculty measure for other images.

Original: It is therefore inferior to our single-image difﬁculty score.
Revised: It is therefore inferior to our single-image difﬁculty score.

Original: In the simpler pix2pix model, the multi-image difﬁculty metric seems comparable to the single-image difﬁculty score, given a sufﬁcient number of images.
Revised: In the simpler pix2pix model, the multi-image difﬁculty metric seems comparable to the single-image difﬁculty score, given a sufﬁcient number of images.

Original: We hypothesize that it is due to the lower capacity of the architecture, which limits overﬁtting.
Revised: We hypothesize that it is due to the lower capacity of the architecture, which limits overﬁtting.

Original: This however comes at the cost of the often unrealistic requirement of extra training data.
Revised: This however comes at the cost of the often unrealistic requirement of extra training data.

Original: In addition, this assumes knowledge of the victim architecture.
Revised: In addition, this assumes knowledge of the victim architecture.

Original: Our singleimage score does not require extra training images or knowledge of the architecture, which makes it applicable in more cases.
Revised: Our singleimage score does not require extra training images or knowledge of the architecture, which makes it applicable in more cases.

Original: Fig 4 presents the effect of number of training images on the multi image calibration method.
Revised: Fig 4 presents the effect of number of training images on the multi image calibration method.

Original: As can be seen, in the pix2pix model, at least 50 of the same distribution images are required in order to outperform our method.
Revised: As can be seen, in the pix2pix model, at least 50 of the same distribution images are required in order to outperform our method.

Original: For Pix2pixHD none of the evaluated number of images outperformed our single-image score.
Revised: For Pix2pixHD none of the evaluated number of images outperformed our single-image score.

Original: 4.2. O UT -O F -D ISTRIBUTION M ULTI I MAGE
Revised: 4.2. O UT -O F -D ISTRIBUTION M ULTI I MAGE

Original: As suggested by He et al.
Revised: As suggested by He et al.

Original: (2019), we also compare our score to the more realistic scenario in which a large amount of similar but out-of-distribuition dataset is available.
Revised: (2019), we also compare our score to the more realistic scenario in which a large amount of similar but out-of-distribuition dataset is available.

Original: For this cause, we train a shadow model on 4 K images from the BDD dataset (Yu et al., 2018), as done by He et al.
Revised: For this cause, we train a shadow model on 4 K images from the BDD dataset (Yu et al., 2018), as done by He et al.

Original: We then use this model as a shadow model for a multi-image difﬁculty score to the Cityscapes dataset, as both datasets consist of street scene images and have compatible label spaces.
Revised: We then use this model as a shadow model for a multi-image difﬁculty score to the Cityscapes dataset, as both datasets consist of street scene images and have compatible label spaces.

Original: Note that it assumes knowledge of the victim model, which is not always true and is therefore a best-case scenario.
Revised: Note that it assumes knowledge of the victim model, which is not always true and is therefore a best-case scenario.

Original: Table 2 demonstrates that this approach is inferior to our single-image difﬁculty score.
Revised: Table 2 demonstrates that this approach is inferior to our single-image difﬁculty score.

Original: 4. S UPERVISED D IFFICULTY S CORE
Revised: 4. S UPERVISED D IFFICULTY S CORE

Original: We compare our self-supervised single-image difﬁculty score with the supervised difﬁculty score described in Sec. 3.2.
Revised: We compare our self-supervised single-image difﬁculty score with the supervised difﬁculty score described in Sec. 3.2.

Original: The supervised score was proposed by Tudor Ionescu et al.
Revised: The supervised score was proposed by Tudor Ionescu et al.

Original: (2016), which deﬁned image difﬁculty to be the human response time for solving a visual search task.
Revised: (2016), which deﬁned image difﬁculty to be the human response time for solving a visual search task.

Original: In order to
Revised: In order to

Original: [Figure]
Revised: [Figure]

Original: Model
Revised: Model

Original: Dataset
Revised: Dataset

Original: [Figure]
Revised: [Figure]

Original: Supervised
Revised: Supervised

Original: BDD Dataset
Revised: BDD Dataset

Original: In-Dist.
Revised: In-Dist.

Original: [Table]
Revised: [Table]

Original: Facades
Revised: Facades

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: Cityscapes
Revised: Cityscapes

Original: [Table]
Revised: [Table]

Original: Pix2pixHD

Original: Facades
Revised: Facades

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Maps2sat
Revised: Maps2sat

Original: [Table]
Revised: [Table]

Original: Cityscapes
Revised: Cityscapes

Original: [Table]
Revised: [Table]

Original: provide a fair comparison, we replace the pretrained VGG-f (Chatﬁeld et al., 2014) features, used by Tudor Ionescu et al.
Revised: provide a fair comparison, we replace the pretrained VGG-f (Chatﬁeld et al., 2014) features, used by Tudor Ionescu et al.

Original: (2016), with the more recent pretrained Wide-ResNet 50 × 2 (Zagoruyko &
Revised: (2016), with the more recent pretrained Wide-ResNet 50 × 2 (Zagoruyko &

Original: Komodakis, 2016) features, as we use in our model.
Revised: Komodakis, 2016) features, as we use in our model.

Original: Fig.
Revised: Fig.

Original: 12 in the appendix presents samples of images ranked as easy and hard by the supervised score.
Revised: 13 in the appendix presents samples of images ranked as easy and hard by the supervised score.

Original: As can be seen in Table 2, our self-supervised single-image difﬁculty score outperforms the supervised difﬁculty score.
Revised: As can be seen in Table 2, our self-supervised single-image difﬁculty score outperforms the supervised difﬁculty score.

Original: Fig.
Revised: Fig.

Original: 5 compares the relation between the reconstruction error and the supervised score to the relation between the reconstruction error and our self-supervised difﬁculty score
Revised: 5 compares the relation between the reconstruction error and the supervised score to the relation between the reconstruction error and our self-supervised difﬁculty score

Original: [Figure]
Revised: [Figure]

Original: and shows that our score is better correlated to the reconstruction error.
Revised: and shows that our score is better correlated to the reconstruction error.

Original: For comparison on other benchmarks, see Fig.
Revised: For comparison on other benchmarks, see Fig.

Original: 7 in the appendix
Revised: 7 in the appendix

Original: [Figure]
Revised: [Figure]

Original: 4. S HADOW M ODELS
Revised: 4. S HADOW M ODELS

Original: We compare our method with shadow-model-based methods, commonly used in membership inference attacks.
Revised: We compare our method with shadow-model-based methods, commonly used in membership inference attacks.

Original: A shadow model is trained on in-distribution data samples to create a labeled dataset
Revised: A shadow model is trained on in-distribution data samples to create a labeled dataset

Original: [Table]
Revised: [Table]

Original: of known (train) and unknown (test) samples.
Revised: of known (train) and unknown (test) samples.

Original: The labeled dataset is then used to train a binary classiﬁer to distinguish between the two.
Revised: The labeled dataset is then used to train a binary classiﬁer to distinguish between the two.

Original: The assumption is that the classiﬁer trained for membership attacks on the shadow model, will also detect membership of the target model.
Revised: The assumption is that the classiﬁer trained for membership attacks on the shadow model, will also detect membership of the target model.

Original: We use the shadow models from Sec.
Revised: We use the shadow models from Sec.

Original: 4.2, i.e. sharing the same architecture between the victim model V and the shadow model S and randomly sampling N images from the test set, referred to as shadow train , to train the shadow model S .
Revised: 4.2, i.e. sharing the same architecture between the victim model V and the shadow model S and randomly sampling N images from the test set, referred to as shadow train , to train the shadow model S .

Original: A labeled dataset is constructed by randomly selecting additional N test images, shadow test .
Revised: A labeled dataset is constructed by randomly selecting additional N test images, shadow test .

Original: The training procedure is detailed in App.
Revised: The training procedure is detailed in App.

Original: A.7.
Revised: A.7.

Original: We compare the ROCAUC of our attack against the classiﬁcation accuracy and ROCAUC of the conﬁdence score of the classiﬁer C when applied to images generated by the victim model V in
Revised: We compare the ROCAUC of our attack against the classiﬁcation accuracy and ROCAUC of the conﬁdence score of the classiﬁer C when applied to images generated by the victim model V in

Original: Table 3.
Revised: Table 3.

Original: Similar to He et al.
Revised: Similar to He et al.

Original: (2019), we apply the classiﬁer over several patches from the query image, and average the results over all patches.
Revised: (2019), we apply the classiﬁer over several patches from the query image, and average the results over all patches.

Original: In order to provide a fair comparison we do not evaluate using the 2 N images used in the training of the classiﬁer.
Revised: In order to provide a fair comparison we do not evaluate using the 2 N images used in the training of the classiﬁer.

Original: It can be seen that our attack outperforms the shadow model approach on all experiments.
Revised: It can be seen that our attack outperforms the shadow model approach on all experiments.

Original: We further investigate the effect of the amount of training data used to train the shadow model and the corresponding classiﬁer.
Revised: We further investigate the effect of the amount of training data used to train the shadow model and the corresponding classiﬁer.

Original: As can be seen in Fig.
Revised: As can be seen in Fig.

Original: 11, the success of the shadow model based attack depends on the amount of data used.
Revised: 11, the success of the shadow model based attack depends on the amount of data used.

Original: This shows that in the common scenario of having just one or few images in-distribution images, the shadow model approach is not as effective as ours.
Revised: This shows that in the common scenario of having just one or few images in-distribution images, the shadow model approach is not as effective as ours.

Original: As in Sec.
Revised: As in Sec.

Original: 4.2.1, we compare to the more realistic scenario in which a large amount of similar but out-of-distribution dataset is available.
Revised: 4.2.1, we compare to the more realistic scenario in which a large amount of similar but out-of-distribution dataset is available.

Original: We use the shadow model trained on 4 K images from the BDD dataset (Yu et al., 2018) and train the classiﬁer to distinguish between those 4 K and additional 4 K images that were not used in the training of the shadow model.
Revised: We use the shadow model trained on 4 K images from the BDD dataset (Yu et al., 2018) and train the classiﬁer to distinguish between those 4 K and additional 4 K images that were not used in the training of the shadow model.

Original: As can be seen in Table 3, this approach is inferior to our single-image method.
Revised: As can be seen in Table 3, this approach is inferior to our single-image method.

Original: 4. E FFECT OF O VERFITTING
Revised: 4. E FFECT OF O VERFITTING

Original: Membership inference attacks are closely related to overﬁtting in the victim model.
Revised: Membership inference attacks are closely related to overﬁtting in the victim model.

Original: In order to better understand this relation, we measure the success of our calibrated prediction error based attack under different levels of overﬁtting.
Revised: In order to better understand this relation, we measure the success of our calibrated prediction error based attack under different levels of overﬁtting.

Original: We do so by evaluating our attack on checkpoints saved at different epochs during the training of victim model.
Revised: We do so by evaluating our attack on checkpoints saved at different epochs during the training of victim model.

Original: For the purpose of this experiment, we also trained the
Revised: For the purpose of this experiment, we also trained the

Original: Pix2pixHD model on the Cityscapes dataset instead of using the pretrained model.
Revised: Pix2pixHD model on the Cityscapes dataset instead of using the pretrained model.

Original: The results are presented in Fig.
Revised: The results are presented in Fig.

Original: We can observe that as the training process progresses, the victim model overﬁts more to the training data which results in higher attack success rates.
Revised: We can observe that as the training process progresses, the victim model overﬁts more to the training data which results in higher attack success rates.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this work, we present a black-box membership inference attack on conditional image generation models, e.g. image translation.
Revised: In this work, we present a black-box membership inference attack on conditional image generation models, e.g. image translation.

Original: At ﬁrst we analyze a training-free attack based on the reconstruction error.
Revised: At ﬁrst we analyze a training-free attack based on the reconstruction error.

Original: We further improve this attack by proposing a novel image difﬁculty score, whose computation does not require an auxiliary training set.
Revised: We further improve this attack by proposing a novel image difﬁculty score, whose computation does not require an auxiliary training set.

Original: By utilizing this score, we can successfully discriminate between difﬁcult images used in the training set, whose reconstruction error is large, and easy previously unseen images that have low reconstruction error.
Revised: By utilizing this score, we can successfully discriminate between difﬁcult images used in the training set, whose reconstruction error is large, and easy previously unseen images that have low reconstruction error.

Original: Our novel combined membership error was shown to achieve higher accuracy than baselines that use more supervision on multiple benchmarks.
Revised: Our novel combined membership error was shown to achieve higher accuracy than baselines that use more supervision on multiple benchmarks.

Original: A A PPENDIX A. O THER P RIVACY A TTACKS
Revised: A A PPENDIX A. O THER P RIVACY A TTACKS

Original: Besides membership inference attacks there exists a wide range of privacy attacks against neural networks.
Revised: Besides membership inference attacks, there exists a wide range of privacy attacks against neural networks.

Original: Model inversion attacks, ﬁrst proposed by Fredrikson et al.
Revised: Model inversion attacks, ﬁrst proposed by Fredrikson et al.

Original: (2014) aim at reconstructing features of the training data, e g. recovering an image of a person from face recognition models (Fredrikson et al., 2015).
Revised: (2014), aim at reconstructing features of the training data, e g. recovering an image of a person from face recognition models (Fredrikson et al., 2015).

Original: Property inference attacks, proposed by Ganju et al.
Revised: Property inference attacks, proposed by Ganju et al.

Original: (2018), do not focus on the privacy of individual data samples, as in membership inference and model inversion attacks, but focus at inferring global properties of the training data, such as the environment in which the data was produced or the fraction of the data that comes from a certain class.
Revised: (2018), do not focus on the privacy of individual data samples, as in membership inference and model inversion attacks, but focus at inferring global properties of the training data, such as the environment in which the data was produced or the fraction of the data that comes from a certain class.

Original: Model extraction attacks, also referred to as model stealing, attack a model f by constructing a substitute model
Revised: Model extraction attacks, also referred to as model stealing, attack a model f by constructing a substitute model

Original: ˆ f that is either identical or equivalent to f (Tram ` er et al., 2016; Jagielski et al., 2020).
Revised: ˆ f that is either identical or equivalent to f (Tram ` er et al., 2016; Jagielski et al., 2020).

Original: Related line of work (Wang & Gong, 2018; Oh et al., 2019) attempts to infer hyperparameters such as the optimization proccess, e.g. SGD or ADAM.
Revised: Related line of work (Wang & Gong, 2018; Oh et al., 2019) attempts to infer hyperparameters such as the optimization proccess, e.g. SGD or ADAM.

Original: A. D ETAILED D ESCRIPTION OF O UR MIA A LGORITHM
Revised: A. D ETAILED D ESCRIPTION OF O UR MIA A LGORITHM

Original: Our MIA consist of computing the two terms in Eq.
Revised: Our MIA consist of computing the two terms in Eq.

Original: (3), i.e. L rec and L diff for a given query pair ( x, y ) , where x is an image from the input domain and y is the ground truth from the output domain, using only a black-box access to the victim conditional generation model V .
Revised: (3), i.e. L rec and L diff for a given query pair ( x, y ) , where x is an image from the input domain and y is the ground truth from the output domain, using only a black-box access to the victim conditional generation model V .

Original: L rec is computed using the L 1 error between the output image predicted by the model, V ( x ) , and the ground truth image y , see step 1 in the Algorithm 1.
Revised: L rec is computed using the L 1 error between the output image predicted by the model, V ( x ) , and the ground truth image y , see step 1 in the Algorithm 1.

Original: L diff is computed as the average error of a linear regression model, P , in predicting pixel values from deep features of the ground truth image.
Revised: L diff is computed as the average error of a linear regression model, P , in predicting pixel values from deep features of the ground truth image.

Original: Our deep features are the activation values in the ﬁrst 4 blocks of a pre-trained Wide-ResNet 50 ×(Zagoruyko & Komodakis, 2016).
Revised: Our deep features are the activation values in the ﬁrst 4 blocks of a pre-trained Wide-ResNet 50 ×(Zagoruyko & Komodakis, 2016).

Original: These features are of sizes 56 × 56 × 256 , 28 × 28 × 512 , 14 × 14 × 1024 , and 7 × 7 × 2048 .
Revised: These features are of sizes 56 × 56 × 256 , 28 × 28 × 512 , 14 × 14 × 1024 , and 7 × 7 × 2048 .

Original: We interpolate all features to size 56 × 56 using bi-linear interpolation (step 2), and also reduce the input image to 56 × 56 (step 3).
Revised: We interpolate all features to size 56 × 56 using bi-linear interpolation (step 2), and also reduce the input image to 56 × 56 (step 3).

Original: This gives a concatenated feature vector of size 3840 for each pixel i in the 56 × 56 image ( 256+512+1024+2048=3840 ).
Revised: This gives a concatenated feature vector of size 3840 for each pixel i in the 56 × 56 image ( 256+512+1024+2048=3840 ).

Original: Wedenote the concatenated feature vector for pixel i as ψ ( i ) .
Revised: We

Original: We randomly select 70% of the pixels as train set, and compute a linear model P to estimate the RGB pixel values y itrain from the corresponding deep features ψ train ( i ) (step 4).
Revised: We randomly select 70% of the pixels as train set, and compute a linear model P to estimate the RGB pixel values y itrain from the corresponding deep features ψ train ( i ) (step 4).

Original: The remaining 30% of pixels will be used as a test split, { ψ test , y test } (step 5).
Revised: The remaining 30% of pixels will be used as a test split, { ψ test , y test } (step 5).

Original: I.e. | ψ train | = 2195 × 3840 , | y train | = 2195 × 3 and | ψ test | = 941 × 3840 , | y test | = 941 × 3 .
Revised: I.e. | ψ train | = 2195 × 3840 , | y train | = 2195 × 3 and | ψ test | = 941 × 3840 , | y test | = 941 × 3 .

Original: The linear regression model P , a matrix of size 3840 × 3 , is trained to minimize the error over { ψ train , y train } (step 6).
Revised: The linear regression model P , a matrix of size 3840 × 3 , is trained to minimize the error over { ψ train , y train } (step 6).

Original: L diff will be the average absolute error over { ψ test , y test } (step 7).
Revised: L diff will be the average absolute error over { ψ test , y test } (step 7).

Original: We found that ﬁtting the linear model to 70% of pixels and measuring the error on the remaining 30% gives better results than just measuring the error of the linear ﬁtting.
Revised: We found that ﬁtting the linear model to 70% of pixels and measuring the error on the remaining 30% gives better results than just measuring the error of the linear ﬁtting.

Original: We compute L mem according to Eq.
Revised: We compute L mem according to Eq.

Original: (3) and compare the results with a predeﬁned threshold value τ , such that any pair ( x, y ) for which is holds that L mem ( x, y ) < τ is denoted as a member of the victim models’ V train set (steps 8-9). 
Revised: (3) and compare the results with a predeﬁned threshold value τ , such that any pair ( x, y ) for which is holds that L mem ( x, y ) < τ is denoted as a member of the victim models’ V train set (steps 8-9). Algorithm Input: Query pair ( x, y ) , victim model V , feature extractor F , scalar α , threshold τ Output: Membership inference result

Original: L rec =
Revised: L rec =

Original: (cid:107) V ( x ) − y (cid:107)2.
Revised: (cid:107) V ( x ) − y (cid:107)2.

Original: ψ = F ( y ) //
Revised: ψ = F ( y ) //

Original: | ψ | = 56 × 56 ×3.
Revised: | ψ | = 56 × 56 ×3.

Original: y =
Revised: y =

Original: resize ( y, 56 × 56 × 3) 4. { ψ train , y train } 70% ←−− { ψ, y } 5. { ψ test , y test } = { ψ, y } \ { ψ train , y train } 6.
Revised: resize ( y, 56 × 56 × 3) 4. { ψ train , y train } 70% ←−− { ψ, y } 5. { ψ test , y test } = { ψ, y } \ { ψ train , y train } 6.

Original: Train linear regression P with { ψ train , y train } 7.
Revised: Train linear regression P with { ψ train , y train } 7.

Original: L diff = 1 N (cid:80) Ni
Revised: L diff = 1 N (cid:80) Ni

Original: =1 (cid:107) P ψ test ( i ) − y itest (cid:107) 1 //N 
Revised: =1 (cid:107) P ψ test ( i ) − y itest (cid:107) 1 //N =8.

Original: L mem =
Revised: L mem =

Original: L rec− α · L diff 9. [Figure] [Figure]
Revised: L rec

Original: A. P ARAMETER S ELECTION
Revised: A. P ARAMETER S ELECTION

Original: We experimented with different values for the α value in Eq.
Revised: We experimented with different values for the α value in Eq.

Original: As can be seen in Fig.
Revised: As can be seen in Fig.

Original: 8, α = 0 .was
Revised: 8, α = 0 .was

Original: the best choice over all benchmarks
Revised: the best choice over all benchmarks.

Original: [Figure]

Original: A. O VERFITTING
Revised: A. O VERFITTING

Original: As mentioned in Sec. 4.5, overﬁtting is the main reason for the success of our method.
Revised: As mentioned in Sec. 4.5, overﬁtting is the main reason for the success of our method.

Original: Fig.
Revised: Fig.

Original: 9 shows the accuracy of our method as a function of the number of epochs used for training the victim model, clearly suggesting that overﬁtting is indeed the vulnerability.[Figure]
Revised: 9 shows the accuracy of our method as a function of the number of epochs used for training the victim model, clearly suggesting that overﬁtting is indeed the vulnerability.

Original: A. E FFECT OF A UGMENTATION
Revised: A. E FFECT OF A UGMENTATION

Original: In order to further increase the accuracy of our reconstruction-error-based attack, we take advantage of the augmentations that were used as part of the training process of the victim model.
Revised: In order to further increase the accuracy of our reconstruction-error-based attack, we take advantage of the augmentations that were used as part of the training process of the victim model.

Original: We use two common augmentations: random ﬂip and random crop.
Revised: We use two common augmentations: random ﬂip and random crop.

Original: We randomly sample T augmentations, and estimate the prediction error as the average L 1 -reconstruction error over all the augmentations.
Revised: We randomly sample T augmentations, and estimate the prediction error as the average L 1 -reconstruction error over all the augmentations.

Original: We note that for Pix2pixHD (Wang et al., 2018), we only used the ﬂip augmentation, as random cropping is not used during its training process.
Revised: We note that for Pix2pixHD (Wang et al., 2018), we only used the ﬂip augmentation, as random cropping is not used during its training process.

Original: More generally, using augmentations at inference time that not used for training produces corrupted results.
Revised: More generally, using augmentations at inference time that not used for training produces corrupted results.

Original: Therefore in case of Pix2pixHD, T =as we only add augmentation, the horizontally ﬂipped query image.
Revised: Therefore in case of Pix2pixHD, T =as we only add augmentation, the horizontally ﬂipped query image.

Original: In case of Pix2pix, T =was used.
Revised: In case of Pix2pix, T =was used.

Original: As can be seen in Table 4, utilizing augmentations on pix2pix enables us to improve the attack success rate by up to 5% . 
Revised: As can be seen in Table 4, utilizing augmentations on pix2pix enables us to improve the attack success rate by up to 5% . [Figure]

Revised: and test sets better separated by a vertical line on the x axis.

Revised: Pix2pixHD for Maps2sat and Cityscapes are presented in Fig.

Original: [Table]
Revised: [Table]

Original: A. D EFENSES
Revised: A. D EFENSES

Original: Several defenses against membership inference attacks have been previously studied.
Revised: Several defenses against membership inference attacks have been previously studied.

Original: As most previous works on MIA targeted classiﬁcation models, not all defenses can be applied to conditional generation models.
Revised: As most previous works on MIA targeted classiﬁcation models, not all defenses can be applied to conditional generation models.

Original: One such defense is the argmax defense, in which the victim model returns only the predicted label, rather then the full probability vector.
Revised: One such defense is the argmax defense, in which the victim model returns only the predicted label, rather then the full probability vector.

Original: Other defenses attempt to reduce overﬁtting, which is highly correlated with MIA success rates.
Revised: Other defenses attempt to reduce overﬁtting, which is highly correlated with MIA success rates.

Original: This can be done for example by changing the dropout or weight normalization ratios during the training of the victim model.
Revised: This can be done for example by changing the dropout or weight normalization ratios during the training of the victim model.

Original: Due to the computational resources required for training multiple models with different ratios, we do not evaluate our attack success against this defense, and refer the reader to
Revised: Due to the computational resources required for training multiple models with different ratios, we do not evaluate our attack success against this defense, and refer the reader to

Original: Sec.
Revised: Sec.

Original: A.4 for a visualization of the effect of overﬁtting on our attack, by attacking the victim model at earlier stages of the training process.
Revised: A.4 for a visualization of the effect of overﬁtting on our attack, by attacking the victim model at earlier stages of the training process.

Original: Two additional possible defense mechanisms are the differential private SGD (DP-SGD) and Gauss defenses.
Revised: Two additional possible defense mechanisms are the differential private SGD (DP-SGD) and Gauss defenses.

Original: In DP-SGD (Abadi et al., 2016), the commonly used stochastic gradient descent optimization algorithm is modiﬁed in order to provide a differentially private (Dwork et al., 2014) model.
Revised: In DP-SGD (Abadi et al., 2016), the commonly used stochastic gradient descent optimization algorithm is modiﬁed in order to provide a differentially private (Dwork et al., 2014) model.

Original: This is done by adding Gaussian noise to clipped gradients for each sample in every training batch.
Revised: This is done by adding Gaussian noise to clipped gradients for each sample in every training batch.

Original: However, this comes at the cost of computational complexity and model generation quality.
Revised: However, this comes at the cost of computational complexity and model generation quality.

Original: There exists a trade-off between privacy and utility, in which the amount of added noise must be large enough to ensure privacy while not degrading the model’s outputs to the point where the model is useless.
Revised: There exists a trade-off between privacy and utility, in which the amount of added noise must be large enough to ensure privacy while not degrading the model’s outputs to the point where the model is useless.

Original: Training a deep model, such as conditional generation models, with DP-SGD is an unstable process.
Revised: Training a deep model, such as conditional generation models, with DP-SGD is an unsta- [Figure] [Figure] ble process.

Original: We experimented with multiple common conﬁgurations, i.e. added noise ratios and maximal gradient clipping threshold, and were not able to ﬁnd a conﬁguration that yields visually satisfying results.
Revised: We experimented with multiple common conﬁgurations, i.e. added noise ratios and maximal gradient clipping threshold, and were not able to ﬁnd a conﬁguration that yields visually satisfying results.

Original: Hence, although the DP-SGD defense is theoretically protecting the victim model against membership inference attacks, in practice we ﬁnd it to be impractical against our attack as it results with total corruption of the victim model.
Revised: Hence, although the DP-SGD defense is theoretically protecting the victim model against membership inference attacks, in practice we ﬁnd it to be impractical against our attack as it results with total corruption of the victim model.

Original: In the Gauss defense, we add Gaussian noise to the image generated by the victim model (Gilmer et al., 2019).
Revised: In the Gauss defense, we add Gaussian noise to the image generated by the victim model (Gilmer et al., 2019).

Original: This attempts to hide speciﬁc artifacts of the overﬁtted model.
Revised: This attempts to hide speciﬁc artifacts of the overﬁtted model.

Original: We evaluate our attack success as a function of different noise STD.
Revised: We evaluate our attack success as a function of different noise STD.

Original: Fig.
Revised: Fig.

Original: 10 shows that a considerable amount of noise,  which corrupts the generated image, is required in order to have a signiﬁcant effect over our attack success.
Revised: 10 shows that a considerable amount of noise, [Figure] which corrupts the generated image, is required in order to have a signiﬁcant effect over our attack success.

Original: Moreover, it can be seen that even with large amounts of noise, our attack still manages to succeed much better than random guessing.
Revised: Moreover, it can be seen that even with large amounts of noise, our attack still manages to succeed much better than random guessing.

Original: This implies that our attack is robust to the Gauss defense.
Revised: This implies that our attack is robust to the Gauss defense

Original: [Figure]
Revised: [Figure]

Original:  S HADOW MODEL TRAINING
Revised: A. S HADOW MODEL TRAINING

Original: After having selected N images, denoted as shadow train , which we used to train the shadow model and another N images, shadow test , which are not seen by the shadow model, we set the labels as:
Revised: After having selected N images, denoted as shadow train [Figure] which we used to train the shadow model and another N images, shadow test , which are not seen by the shadow model, we set the labels as:

Original: [Equation]
Revised: [Equation]

Original: The classiﬁer C architecture and training procedure are similar to He et al.
Revised: The classiﬁer C architecture and training procedure are similar to He et al.

Original: For each image, we compute the structured loss map between the ground-truth image and the generated image, and at every epoch we randomly crop 15 patches of size 90 × 90 from the structured loss map.
Revised: For each image, we compute the structured loss map between the ground-truth image and the generated image, and at every epoch we randomly crop 15 patches of size 90 × 90 from the structured loss map.

Original: We train a
Revised: We train a

Original: ResNet-50 (He et al., 2016) from scratch on the 90 × 90 patches, modiﬁed for binary classiﬁcation.
Revised: ResNet-50 (He et al., 2016) from scratch on the 90 × 90 patches, modiﬁed for binary classiﬁcation.

Original: We use a batch size of 8 , SGD optimizer, weight decay of 1 e − 2 , initial learning rate of 0 . 1 which reduces by a factor of 0 .
Revised: We use a batch size of 8 , SGD optimizer, weight decay of 1 e − 2 , initial learning rate of 0 . 1 which reduces by a factor of 0 .

Original: 1 every 15 epochs.
Revised: 1 every 15 epochs.

Original: As previously mentioned, we do not evaluate this on the
Revised: As previously mentioned, we do not evaluate this on the

Original: Facades dataset, due to its size. 
Revised: Facades dataset, due to its size. [Figure]

Original: A. E DGES 2 SHOES
Revised: A. A DDITIONAL D ATASETS

Original: In  addition to the Facades, Maps2sat and Cityscapes datasets, we evaluated our attack over the
Revised: In In addition to the Facades, Maps2sat and Cityscapes datasets, we evaluated our attack over the

Revised: Edges2shoes dataset (Yu & Grauman, 2014) and the CelebA dataset (Liu et al., 2015)

Original:  Edges2shoes dataset (Yu & Grauman, 2014), which consists of 50 K train images and 200 test[Figure] images.
Revised: The Edges2shoes dataset (Yu & Grauman, 2014) consists of 50 K train images and 200 test images.

Original: Due to the large training set, the Pix2pix model has shown less overﬁtting to the data, and generates results that are visually inferior to the other datasets.
Revised: Due to the large training set, the Pix2pix model has shown less overﬁtting to the data, and generates results that are visually inferior to the other datasets.

Original: The lack of overﬁtting results with our attack not managing to do better then a random guess, due to the strong correlation between overﬁtting and membership inference attacks.
Revised: The lack of overﬁtting results with our attack not managing to do better then a random guess, due to the strong correlation between overﬁtting and membership inference attacks.

Original: As Pix2pixHD has larger capacity, it managed to produce better results, in terms of visual quality, over this dataset.
Revised: As Pix2pixHD has larger capacity, it managed to produce better results, in terms of visual quality, over this dataset.

Original: This also comes at the cost of more overﬁtting, as can be observed by the success of our attack over this model: 87 .
Revised: This also comes at the cost of more overﬁtting, as can be observed by the success of our attack over this model: 87 .

Original: This is an improvement of 5 . 5% from our base attack, using only the reconstruction error, which achieves 81 . 
Revised: This is an improvement of 5 . 5% from our base attack, using only the reconstruction error, which achieves 81 . A.8. C ELEB A

Revised: The CelebA dataset (Liu et al., 2015) is large-scale face attributes dataset with more than 200K celebrity images.

Original: A. S UPERVISED I MAGE D IFFICULTY S CORE
Revised: We evaluated our attack on the CelebA dataset - by randomly sampling 10 K images, extracting face landmark (Bulat & Tzimiropoulos, 2017) and training a pix2pixHD model to generate real face images from facial landmarks, see Fig. Our base attack, using reconstruction error alone, achieves a success rate of 98 . By using our difﬁculty score, our attack further improves to a success rate of 99 . 04% . Faces are considered to be sensitive information, and the ability to accurately determine if a speciﬁc facial image was used to train a conditional generative model is a serious and concerning privacy breach.

Original: In Fig.
Revised: [Figure]

Original: 12, we present images ranked from hard to difﬁcult using our implementation of the supervised-image difﬁculty score by Tudor Ionescu et al.
Revised: [Figure] In Fig. 13, we present images ranked from hard to difﬁcult using our implementation of the supervised-image difﬁculty score by Tudor Ionescu et al.

Original: (2016), for the Cityscapes and Maps datasets.
Revised: (2016), for the Cityscapes and Maps datasets.

Original: The ranking seems correlated with image sharpness and level-of-detail images.
Revised: The ranking seems correlated with image sharpness and level-of-detail images.

Revised: A. I MAGE N ET D IFFICULTY S CORE

Revised: Our difﬁculty score relies on learning a mapping between feature vectors to their corresponding pixel values.

Revised: We use a pre-trained Wide-ResNet 50 × 2 (Zagoruyko & Komodakis, 2016), which is trained on the ImageNet dataset.

Revised: We do not make any assumptions regarding an overlap between the pre-trained model’s training data (i.e. ImageNet) and the data during in the attack.

Revised: In the scenario in which such an overlap exists, the concern is that the difﬁculty score would lose its credibility.

Revised: In order to verify this, we computed the difﬁculty score of a random subset of 1 K train images and 1 K test images, from the ImageNet dataset.

Revised: We do not observe any signiﬁcant difference between the two - both share similar mean and std values: ( 0 . 0549 , 0 . 018 ) for the train images and ( 0 . 0556 , 0 . 0191 ) for the test images.

Revised: A ROCAUC score of 51% further demonstrates that there is no clear difference between the distribution of the difﬁculty score on seen and unseen images.

Revised: Fig.

Revised: 14 further demonstrates this.

Revised: The ﬁrst row presents the train images that received the lowest scores, i.e. marked as easy images, and the second row presents the test images with the lowest scores.

Revised: Both correspond to ”plain” images, regardless of whether they are known (train) or unknown (test).

Revised: The same applies to the Difﬁcult images.

Revised: The third row presents the highest scored train images and the last row presents the highest scored test images.

Revised: Both contains complex patterns and high variance.

Revised: This demonstrates that the difﬁculty score is not affected by the having prior knowledge of the image, and is only measuring the amount of variance and complexity of an image.

Revised: A. C ONDITIONAL D IFFICULTY S CORE

Revised: We evaluated the effect of conditioning the difﬁculty score computation on the data the victim model is conditioned on, i.e. training the regression model to predict pixel values from feature vectors extracted from the input to the conditional generative model.

Revised: Both methods achieve very similar results on the Cityscapes dataset and pix2pixHD model: 99 .

Revised: 02% when training on ground-truth vs.

Revised: 82% when training on the segmentation mask.

Revised: As the results are similar, with the ground-truth method being slightly superior, we chose to not be dependent of the input image and only use the ground truth image to estimate the image’s difﬁculty.

Revised: A. E FFECT OF D ATA P ARTITION IN D IFFICULTY S CORE

Revised: We evaluated the effect of the different partitions in the training of our difﬁculty score’s regression model, on the Cityscapes dataset and pix2pixHD model.

Revised: As can be seen in Fig.

Revised: 15, training on less than 50% of the image pixels results with unstable performance, while using 50% or above is sufﬁcient to achieve high success rates, with no preference to a speciﬁc value

Revised: [Figure]

Revised: A. E FFECT OF R ESIZE M ETHOD

Revised: In the computation of the difﬁculty score, the ground truth image is resized using bicubic interpolation to an image of size 56 × 56 in order to match the 56 × 56 feature vectors extracted by concatenating the ﬁrst 4 blocks of a Wide-ResNet 50 × 2 (Zagoruyko & Komodakis, 2016).

Revised: We compared the effect of using different resize methods on the Cityscapes dataset for the pix2pixHD model.

Revised: As can be seen in Fig.

Revised: 16, the attack success rate is not very sensitive to the resize method.

Revised: [Figure]

Revised: A. E FFECT OF D IFFICULTY S CORE ON C ONDITIONAL G ENERATIVE M ODEL T RAINING

Revised: We conducted initial experiments to explore the effect of using the difﬁculty score during the training process of the conditional generative model.

Revised: We experimented with training pix2pixHD on the

Revised: Cityscapes dataset while incorporating our difﬁculty score in the generator loss term.

Revised: The generator loss term, L G is composed of three terms: L GAN - the GAN loss term, L FMsimilarity between features of the generator, and L V GG - perceptual similarity.

Revised: We experimented with multiplying each one of these loss term with the difﬁculty score and multiplying all three term with the difﬁculty score, i.e. L G .

Revised: We compared the effect on our attack success rate, as a function of the progress in the training process, which we use to approximate different levels of overﬁtting.

Revised: Multiplying the loss value by the difﬁculty score encourages the generator to pay more attention to difﬁcult images.

Revised: As can be seen in Fig.

Revised: 17, when this is applied on the GAN related loss terms, i.e. L GAN and L FM , the model’s overﬁtting slows down (we observe this by the slow down in the success of our attack, as it is highly correlated with overﬁtting).

Revised: This happens as we pay more attention to the generator success against the discriminator and not to the reconstruction, which is mainly affected by L V GG .

Revised: [Figure]