Variational Model Inversion Attacks
Authors: Kuan-Chieh Wang
YAN FU
Ke Li
Ashish J Khisti
Richard Zemel
Alireza Makhzani
Venue: NeurIPS 2021 Poster (2021)
Keywords: model inversion attacks
generative models
variational inference
generative adversarial networks

Abstract: Given the ubiquity of deep neural networks, it is important that these models do not reveal information about sensitive data that they have been trained on. In model inversion attacks, a malicious user attempts to recover the private dataset used to train a supervised neural network. A successful model inversion attack should generate realistic and diverse samples that accurately describe each of the classes in the private dataset. In this work, we provide a probabilistic interpretation of model inversion attacks, and formulate a variational objective that accounts for both diversity and accuracy. In order to optimize this variational objective, we choose a variational family defined in the code space of a deep generative model, trained on a public auxiliary dataset that shares some structural similarity with the target dataset.  Empirically, our method substantially improves performance in terms of target attack accuracy, sample realism, and diversity on datasets of faces and chest X-ray images. 

Variational Model Inversion Attacks
Kuan-Chieh Wang" 122,t Yan Fu'
Ke Li
Ashish Khisti
Richard Z.mel
Alireza Makhzani1223
University of Toronto', Vector Institute", Simon Fraser University
wangkua1@cs toronto.. torggut
edu, "naihhaani@eettoriisstttuu ai
Abstract
Given the ubiquity of deep neural networks, it is important that these models do not reveal information about sensitive data that they have been trained on. model inversion attacks, a malicious user attempts to recover the private dataset used to train a supervised neural network. A successful model inversion attack should generate realistic and diverse samples that accurately describe each of the classes in the private dataset. In this work, we provide a probabílistic interpretation model inversion attacks, and formulate variational objective that accounts for both diversity and accuracy. In order to optimize this variational objective, we choose variational family defined in the code space of deep generative model, trained on a public auxiliary dataset that shares some structural similarity with the target dataset. Empirically, method substantially improves performance in terms of target attack accuracy, sample realism, and diversity on datasets of faces and chest X-ray images.
Introduction
Thanks to recent advances, deep neural networks now widely used in applications including facial recognition [Parkhi et al., 2015], intelligent automated assistants, and personalized medicine [Consor- tium, 2009; Sconce et al., 2005]. Powerful deep neural networks are trained large datasets that could contain sensitive information about individuals. Publishers release only the trained models (e.g., on TensorFlow Hub [TensorFlow]), but not the original dataset to protect the privacy of individuals whose information is in the training set. Regrettably, the published model can leak information about the original training [Geiping et al., 2020; Nasr al., 2019; Shokri et al., 2017]. Attacks that exploit such properties of machine learning models are privacy attacks [Rigaki and Garcia, 2020]. Other real-world scenarios where attacker can access models trained private datasets include interaction with APIs (e.g., Amazon Rekognition [AWS]). or in the growing paradigm of federated learning [Bonawitz al., 2019; Yang et al., 2019a], where malicious insider can try to steal data from other data centers [Hitaj et al., 2017J. Kaissis et al. [2020] provides an overview of how these privacy concerns are hindering the widespread application of deep learning in the medical domain.
This paper focuses on the model inversion (MI) attack, type of privacy attack that tries to recover the training set given access only to trained classifier [Chen et al., 2020; Fredrikson et al., 2015, 201 4; Hidano et al., 2017: Khosravy et al., 2020; Yang et al., 2019b: Zhang et al., 2020]. The classifier under attack is referred to the "target classifier" (see Figure 1). An important relevant application concerns identity recognition from face images, since facial recognition a common approach to biometric identification [Galbally al., 2014; Rathgeb and Uhl. 2011]. Once attacker successfully reconstructs images of the faces of individuals in the private training set, they can use the stolen identity break into otherwise secure systems.
Early studies of MI attacks focused on attacking classifiers whose inputs are tabular data [Fredrikson et al., 2015, 2014; Hidano et al., 2017] by directly performing inference in the input space. However, the same methods fail when the target classifier processes high-dimensional inputs, e.g., in the above
35th Conference on Neural Information Processing Systems (NcurIPS 2021), held virtually.

---

Real Samplex
Attack Samples
Private dataset
Target
Attacker
Classifier
Figure 1: In model inversion attacks, the attacker tries recover the original dataset used to train a classifier with access to only the trained classifier. the two figures on the right, each row corresponds to private identity individual whose images not available to the attacker. The first five identities (not therry-pickedd were visualized. On the left are real data samples, and on the right are generated samples from our proposed method.
example of a face recognizer. Recent methods cope with this high-dimensional search space by performing the attack in the latent space of a neural network generator [Khosravy et al., 2020; Yang et al., 2019b; Zhang et al., 2020]. Introducing a generator works well in practice, but this success is theoretically understood. In this paper, frame MI attacks variational inference (VI) [Blei et al., 2017; Jordan et al., 1999; Kingma and Welling, 2013) problem, and derive practical objective. This view allows us justify existing approaches based a deep generative model, and identify missing ingredients in them. Empirically, we find that our proposed objective leads to significant improvements: more accurate and diverse attack images, which are quantified using standard sample quality metrics (i.e., FID [Heusel et al., 2017], precision, and recall [Kynkäänniemi et al., 2019; Naeem al., 2020]..
In summary, our contributions and findings are:
We view the model inversion (MI) attack problem as a variational inference (VT) problem. This view allows us derive practical objective based on statistical divergence, and provides unified framework for analyzing existing methods.
We provide an implementation of our framework using a set of deep normalizing flows (Dinh et al., 2014, 2016; Kingma and Dhariwal, 2018] in the extended latent space of Style- GAN [Karras et al., 2020J. This implementation can leverage the hierarchy of learned representations, and perform targeted attacks while preserving diversity.
Empirically, on the CelebA [Liu et al., 2015], and ChestX-ray [Wang et al., 2017] datasets, our method results in higher attack accuracy, and more diverse generation compared to existing methods. We provide thorough comparative experiments to understand which components contribute to improved target accuracy, sample realism, and sample diversity, and detail the benefits of the VI perspective.
Code can be found https:///itthi/ cwm com//angkual//ve
2 Problem Setup: Model Inversion Attack
In the problem of model inversion (MI) attack, the attacker has access to a "target classifier":
PTAR (vlx)) _Rd C-1
where C- to denote the (C 1) simplex, representing the C-dimensional probability where C is the number of classes. This target classifier is trained the private target dataset, DTAR = (x,, ) where E Rd and y {1, 2, C). We PTAR (1/x) x) to denote the given target classifier, which an approximation the true conditional probability PTAR (y x) of the underlying data generating distribution.
New
Goal. Given a target classifier Prxx PrAR(xly) without having access to the private training set D
(rx(()x)) (v)x), we wish to approximate the class conditional distribution

---

A good model inversion attack should approximate the Bayes posterior PEAR (xly) x PIAR (rAR(y|x)Pra (x) well, and allow the attacker to generate realistic, accurate, and diverse samples. Notice, our goal shifts slightly from recovering the exact instances in the training set to recovering the data generating distribution of the training data. Let's consider a scenario where the attacker intends to fake a victim's 's identity. A good security system might test the realism of sequence of inputs [Holland and Komogortsev, 2013]. More generally, faking someone's identity requires passing a two-sample test [Holland and Komogortsev, 2013; Lehmann and Romano, 2006], which means the ability to generate samples from PIAR (xly) is necessary. Another scenario can be an attacker trying to steal private/proprietary information. For example, malicious insider in a federated learning setting can try to steal information from another datacenter by accessing the shared model [Hitaj et al., 2017]. 7]. In such a scenario, good PrAx(xly) model can reveal not only the distinguishing features of the data, but also its variations.
3 Related Work
In this section, we first discuss related model inversion attack studies, and then discuss applications of inverting a classifier outside of the privacy attack setting. Lastly, since our attack method relies on using pretrained generator, we discuss existing studies that used a pretrained generator for improving other applications.
Model inversion attacks. MI attacks are one type of privacy attack where the attacker tries to reconstruct the training examples given access only to a target classifier. Other privacy attacks include membership attacks, attribute inference, and model extraction attacks (see Rigaki and Garcia [2020] for a general discussion). Fredrikson et al. [Fredrikson et al., 2015, 2014] were the first to study the MI attack and demonstrated successful attacks on low-capacity models (c.g.. logistic regression, and a shallow MLP network) when partial input information was available to the attacker. The setting where partial input information was not available was studied by Hidano al. [2017]. They termed performing gradient ascent w.r.t. the target classifier in the input space General Model Inversion attack. Though effective on tabular data, this approach failed on deep image classifiers. is well- that directly performing gradient ascent in the image space results in imperceptible and unnatural changes (Szegedy et al., 2013].
To handle high-dimg digh-dimensional observations such as images, MI attacks based on deep generators were proposed [Chen et al., 2020; Yang al., 2019b; Zhang al., 2020]. By training deep generator on an auxiliary dataset, these methods effectively reduce the search space to only the manifold of relevant images. For example, if the target classifier is facial recognition system, then auxiliary dataset of generic faces can be easily obtained. Zhang et al. [2020] pretrain a Generative Adversarial Network (GAN) [Goodfellow et al., 2014] on the auxiliary dataset and performed gradient- attack in the latent space of the generator. Yang et al. [2019b] trained a generator that inverts the prediction of the target classifier, using the architecture of an autoencoder.
Other applications of classifier inversion. Inverting the predictions and activations of a classifier has been studied for applications such model explanation, and model distillation. Motivated by using saliency maps to explain the decision of classifier [Simonyan et al., 2013], Mahendran and Vedaldi [2015] proposed to maximize the activation of target neuron classifier for synthesizing images that best explain that neuron. Their method is based on gradient ascent in the pixel-space. The same idea was recently extended to enable knowledge distillation
setting where the original dataset was too large to be transferred [Yin et al., 2020]. Methods above have the advantage of not requiring the original dataset, but the generated images still appear unnatural. One effective way to improve the realism of the generated images is to optimize in the latent code space of deep generator [Nguyen al., 2015, 2017].
Pretrained generators. Pretrained generators from GANs have other applications. Image inpaint- ing/restoration was one of the early successful applications of using pretrained GAN as image prior. Yeh et al. [2017] optimized the latent code of GANs to minimize the reconstruction error of a partially occluded image for realistic inpainting. The same method formed the basis of AnoGAN, a popular technique for anomaly detection in medical images [Schlegl et al., 2017]. A combination of reconstruction error and discriminator loss was used as the anomaly score. A potential future direction is to apply our proposed method to these adjacent applications.
3

---

4 Model Inversion Attack as Variational Inference
In this section, we derive our Variational Model Inversion ( (VMI) learning objective. For each class label y,, we wish to approximate the true target posterior with a variational distribution q(x) E 2x where 2x is the variational family. This can be achieved by the optimizing the following objectives
o' (x) = arg min { (PRI.(((x) |PAR (xlu))}
qeQx
arg min (Bo(x) (- log PLAR (u|x)] + DRL(((x))ll (x)) + log PrAR (v))
= arg min ( (By(x)(
-oollllooggna Prar (u|x)] + (xx)(((())))xxxxxx)))
qeQx
where in Equation 1 we have used the fact that log Pran(y) is independent of g(x), and thus can be dropped in the optimization.
Auxiliary Image Prior. In order to minimize Eq. (1), we need to compute DKL (q(x)] P (x)).
which requires evaluating the true image prior PrAu (x), that accessible. Similar to recent works on MI attacks [Khosravy et al., 2020; Yang et al., 2019b; Zhang et al., 2020], we assume that public auxiliary dataset DAux. which shares structural similarity with the target dataset, is available to learn an auxiliary image prior PAux (xux(x). This auxiliary image prior Pxux(x) is learned using generative adversarial network (GAN) PAux(x) (x) = (x) [Pa (x|z)), where PAUX (z) NN I) is the GAN prior, and Po (x|z) is the GAN generator that is parameterized by a deep neural network (z),, and a Gaussian observation noise with small standard deviation σ: Px(x (( or equivalently
G(z) + ge, N~, I)-
The learned generator Pa (x z) defines a low dimensional manifold in the observation space x, when z is chosen have fewer dimensions than x. We now make the following assumption about the structural similarity of PAux (x) and PTAR
Assumption 1 (Common Generator). We assume PrAu (x) is concentrated close the low dimensional manifold of Pa 12, learned from the auxiliary dataset D Aux- In other words, we assume there exists target prior Pram (z). possibly different from Paux(Z), such that Pran(x) Expza()) (Pa (x|2)) We refer to Pa (xlz) as the "common generator".
The name "common generator" is inspired by recent progress in theoretical analyses of few-shot and transfer learning, where assumptions about an underlying shared representation have been used to provide eтror bounds. In these works, this shared representation is referred to as the "common representation" " [Du et al., 2020; Tripuraneni et al., 2020]. Assumption I usually holds in the context of model inversion attacks. For example, PAux(x) could be the distribution of all human faces, while PraR(2 (x) could be the distribution of the celebrity faces; or PAuX (x) could be the distribution of natural images, while PTAR (x) could be the distribution of different breeds of dogs. In these examples, both PAux(X; and Prsa (x) approximately live close to the same low dimensional manifold, but the distribution this shared manifold could be different, i.e., PTAR (z) # PAux(Z).
Variational Family. We further consider our variational family Qx to be all distributions that lie on this manifold by assuming that (xx) is of the form (xx = (g(()) [Po [x(( (x/z)), g(z) E 2z. where 22 is a variational family in the code space. In the next section, we further restrict 2₂ to be either Gaussian distributions or normalizing flows.
Proposition 1. We have DKL (q(z) [z)) 2 DKL
(x) [Pa (x(2))
See Appendix A for the proof. Using Assumption 1, we can approximate the objective of Equation 1 with
q (z) = arg min (Fax:((). log (y) + +c)) + DKL (oane } QEC=
(2)
where this objective now defined in terms of (z)) E 2z in the latent space of the GAN. Now using the Proposition 1, can obtain the following upper bound on the objective of Equation 2 (assuming 0)
q" (z) arg min (Bz log Pran P (y) (ogg)))))looogoooooo
PKc.(g(2) (PTA()))
ge Q.

---

Power Posteriors. We now consider the optimization of Equation 3. Since we do not have access to PTAN (zAn we propose to replace with Paux (z), noting that, as argued above, PAUX (z) may not accurately represent PTAR (2). We also propose to replace the unknown likelihood PTAR (y) G(z)) ) with the PIAR (y)) defined by the target classifier, noting that this likelihood could be miscalibrated. Miscalibration is known problem in neural network classifiers [Guo et al., 2017]; namely, the classifier predictions can be over- or under-confident. In order to account for these issues, instead of the standard Bayes posterior GuAVES (z) PAUX (2) TrAR (y) (z)) as the solution of Equation 3, we consider the family of "power posterior" [Bissiri et al., 2016; Holmes and Walker, 2017; Knoblauch et al., 2019; Miller and Dunson, 2018], which raise the likelihood function to power
o (z) x PAux (nx((2) (v))))))))()((())))) (v/G(z)).
and can be characterized as the solution of the following optimization problem:
(z) = arg min (9),
(4)
q=
Ma (9) == Bx-g(x)) |- log TA
7 + (z)).
See Appendix A for the proof. If the prior PAUX importance of the prior. Similarly, by adjusting 7 we can reduce or increase the importance of over- or under-confident likelihood predictions PTAR (z)).
(z) (z) is not reliable, by decreasing 7, we can reduce the
Accuracy Diversity Tradeoff. In the context of model inversion attack, the Bayes posterior (RAVES(Z) can only achieve a particular tradcoff between the accuracy and diversity of the generated samples. However, the family of power posteriors enables between accuracy and diversity by tuning y. If ? = O, then f- (z) = 8(z z²), and the model essentially ignores the image prior and diversity, and finds single point estimate zt on the manifold which maximizes the accuracy PTAE (y) G(z). As we increase 7, the the model achieves larger diversity at the cost of smaller average accuracy. When y 1, we recover the Bayes posterior (==1(2) x Paux(( Frx (y) (z))). In the limit of 7 + oo, we have Th (z) = PAuX (z). In this case, the model ignores the classifier and accuracy, and sample unconditionally from the auxiliary distribution (maximum diversity). We empirically study the effect of in Section 5.
to achieve an arbitrary desired tradeoff
4.1
The Common Generator
In the previous section, we derived practical MI objective from the perspective of variational inference. Similar to previous works [Chen et al., 2020; Zhang et al., 2020] VMI requires a common generator that trained on relevant auxiliary dataset. In this subsection, we first provide brief description of the most common choice, DCGAN. Then, we introduce StyleGAN, which has an architecture that allows for fine-grained control. Finally, we describe how adapt our VMI objective to leverage this architecture by using layer-wise approximate distribution.
For the common generator G' we use GAN due to their ability to synthesize high quality sam- ples [Abdal et al., 2019; Bau et al., 2018; Karras et al., 2019: Shen et al., 2020]. The generator is trained on the auxiliary dataset. A common choice is the DCGAN architecture [Radford et al., 2015]. Though optimizing in the latent space results in more realistic samples, this search space is also more restrictive. Early investigation showed that optimizing at some intermediate layers of pretrained generator was better than either optimizing in the code space, or the pixel space. This motivated the use of generator architecture where activations of intermediate layers can be manipulated.
StyleGAN. A framework that naturally allows for optimization in the intermediate layers is the StyleGAN [Karras et al., 2019]. It consists of a mapping network f + w, and synthesis network S. The synthesis network takes as input one w each layer, S (wi) + x, where T. is the number of layers in S. During training of a StyleGAN, the output from the mapping network is COPY(w, L) before being fed to the synthesis network. This input space of the synthesis network S has been referred to as the "extended w space" [Abdal et al., 2019]. The
// copied L times, {
full generator is G (z) s s(COPY(f(2)) T.))
'These posterior updates are sometimes referred to as "power likelihood" or "power prior" [Bissiri et al.,
2016].
5

---

Layer 0,1,2 hairstyle, bangs, lice contour
Tayer 3,4,5,6
Layer 7,8,9 no effect on identity
eyes, nose, mouth, hair color
Figure 2: Visualizing the factors of variations in each layer of the StyleGAN [Karras et al., 2019] using style mixing. Each image is generated by mixing the the corresponding image in the topmost row, and the image in the leftmost column. The topmost row image provides the features for the indicated set of layers, and the leftmost column image provides features for the remaining layers. We can see that in general, the earlier features contribute more to the facial features, while the later features contribute more to the low-level images statistics such as lighting and general shading. Given this learned layer-wise disentanglement, our VMI attack can adaptively discover layers that contribute the most to the face identity (layers 1, 2, 4, 5, 6), and only manipulates the corresponding features (see Figure 7). This is not facilitated by the generator of DCGAN, as found in existing MI attack methods [Chen et al., 2020; Zhang et al., 2020].
StyleGAN is known for its surprising ability to perform "style mixing". Namely, the synthesis
Head Orientation
Face Shape Nose Shape
Attacking identity
Head Location
network can generate "mixed" image, when be-
S StyleGAN Synthesis Ne
ing fed a mixture of two w's in the extended w
StyleGAN Mapping Net
S
Target Classifier
space, i.e., S([[ 1 (w2)ftto+))) where
Latent Prior
B denotes concatenation. and Lo is some in- termediate layer. Figure 2 visualizes the effect of manipulating different layers of the synthe- sis network. Performing an attack in this ex- tended w space allows us to achieve the desired effect of optimizing the intermediate layers. Be- cause of the lack of explicit prior for w, we instead optimize the extended z space (Figure 3). Suppose q(Z1,-... 21.) is the joint density over 21,..., .L. and q (z:) is the marginal density over 2!. We consider the following objective for the VMI with the StyleGAN generator:
For all layers
Figure 3: Utilizing the StyleGAN architecture, our VMI attack tends to focus on layers whose repre- sentation are relevant for the attack, such as the layer for "face shape""
log PEAR (v(((1((a)Ha)) + (KL(a (5)
C3 vvn = E (x(x).....))) mE.)
4.2 Code Space Variational Family Q2
In the VMI formulations (Equation 4 and 5), the approximate distribution (zz) can be seen generator, or "miner" network [Wang et al., 2020] for the latent space of the common generator. It must belong to distribution family Qz. whose distribution can be sampled from, and can be evaluated in order to optimize the KL-divergence. In this work, we experiment with a powerful class of tractable generative models: deep flow models. When optimizing V (q) (Equation 5), we use L such models, one for each layer in the synthesis network. In our experiments, we used an architecture similar to that of Glow (Kingma and Dhariwal, 2018], where we treated the latent vector as 1x1 images. We removed the original squeezing layers that reduced the image size to account for this change. We also experimented with using Gaussian variational family for (zz).
4.3 Relationships with Existing Methods
VMI provides unified framework that encompasses existing attack methods. Through the lens of VMI, we can obtain insights on the conceptual differences between the existing methods and their pros and cons. "General MI attack [Hidano al., 2017] performs the following optimization:
x" arg max log Piue (vlx).
x

---

Target: MNIST
Audiliary: EMNIST
Target: CelebA
Auxillary: Celebo
Target: ChestX-ray
Auxillary, ChystX-ray
0
O
O
O
m
D 1 Y
V L
1 2 3 4
1
1 2 3 4
1 2 3
f A W d 2 e K I B G N
S R b
2
3 3
3
4
4
4
W
W
X
/
O
Figure 4: Visualization of real samples for all 3 tasks: MNIST, CelebA, and ChestX-ray. For the target datasets, each row corresponds to one class/identity. Best viewed zoomed in.
Note that this objective can be viewed as special case of VMI, where the code space the same as the data space, and we have = in the objective: LVM (g). In this case, as discussed in the previous section, the q(x) distribution collapses a point estimate at which PrAx (v)x)) is maximized. As we will see in the experiment section, since the optimization is in the data space, the solution does not live close the manifold of images, and thus generally does not structurally looks like natural images.
"Generative MI" attack [Zhang et al., 2020] also uses a generator trained on an "auxiliary" dataset. The attack optimizes the following objective:
arg min
x log PrR (o log (D(G(()))
where G. D denotes the generator and discriminator respectively, and a(-) is the sigmoid function. Note that this objective can be viewed special case of VMI, with y x. and with the prior p(z) x a (D(G(z)), and a point estimate for z, instead of optimizing over distributions q(z). As result of this point- point-wise optimization, the Generative MI attack requires re-running the optimization procedure for every new samples. In contrast, VMI learns distribution g(z) using expressive normalizing flows, which allows us to draw multiple attack samples after a single optimization procedure. Recently, Chen et al. [2020] proposed extending Generative MI by considering a Gaussian variational family in the latent space of the pretrained generator in Equation 7. However, we note this method still theoretically results collapsing the Gaussian variational distributions to a point estimate. VMI prevents this by penalizing the entropy of the variational distribution with the KL-divergence term. From the implementation perspective, Chen et al. [2020] uses DCGAN while we use StyleGAN which helps us to achieve better disentanglement. Furthermore, we use a more expressive variational family such as normalizing flows.
5 Experiments
In this section, demonstrate the effectiveness of the proposed VMI attack method on three separate tasks. We will refer to each task by their target dataset: MNIST [LeCun et al., 2010], CelebA [Liu et al., 2015], and ChestX-ray (CXR) [Wang et al., 2017]. Then, showcase the robustness of the VMI attack to different target classifiers on CelebA. More experimental details can be found in Appendix B.
Data. For the MNIST task, we used the 'letters' split (i.e., handwritten English alphabets) of the EMNIST [Cohen et al., 2017] dataset as the auxiliary dataset. For CelebA, we used the 1000 most frequent identities as the target dataset, and the rest as the auxiliary dataset. For ChestX-ray, we used the diseases outlined by Wang et al. [2017] as the target dataset, and randomly selected 50,000 images from the remaining as the auxiliary dataset. Real samples from these six datasets are visualized in Figure 4.
Target classifiers.
The target classifiers used in all tasks were ResNets trained using SGD with momentum. Optimization hyperparameters were selected to maximize accuracy validation set of the private data.
Evaluation metries. An ideal evaluation protocol should measure different aspects of the attack samples such target accuracy, sample realism, and sample diversity.
Target Accuracy: To automate the process of judging the accuracy of an attack, we use an "evaluation" classifier. This metrics measures how well the generated samples resemble the target class/identity. Given a target identity an attack produces set of samples (x', J Each sample is assigned predicted label by the evaluation classifier, and then the top-1 accuracy is computed.
7

---

Accuracyl CelebA
FID +
MNIST
CXR
MNIST
CelebA
CXR
General MI [Hidano et al., 2017]
0 d. uno
0.23 _
376.7 (574)
421.21 (SL.)
499.54 (963)
Generative MI [Zhang et al., 2020]
0.92 002
0.07
0.28 0.25
88.91
43.21
(31.3)
142.66 (963)
VIMI w/ DCGAN
0.95 0.02
0.37 ourr
0.42+ 0.23
77.73
(57.4)
40.89 (31.3)
265.14 (96.))
VMI w/ StyleGAN
0.55 + 0.06
0.69 0.25
17.41 (19.2)
123.17 (57.0)
Table 1: MI attack comparison across tasks. Both VMI methods here used the Flow model for q(z).
Best values are in bold, and values within the 95% confidence interval (+) of the best underlined. The numbers in the braces in the FID columns are FIDs of the pre-trained generators. Our VMI attack results in better accuracy and FID compared baselines all three tasks.
If a good evaluation classifier only generalized to realistic inputs, then this metric would also penalize unrealistic attack samples. To some extent, this was the case in our experiments as evidenced by the extremely low accuracy assigned the General MI attack baseline in Table 1. Figure 5, the General MI attack samples were clearly unrealistic. To ensure fair and informative evaluation, the evaluation classifier should be as accurate and generalizable as possible (details in Appendix B.3).
Sample Realism: We used FID [Heusel et al., 2017] to directly measure the overall sample quality. To compute FID, we aggregated attack samples across classes and compared them to aggregated real images. Following standard practice, the feature extractor used for computing FID was the pretrained Inception network (Szegedy al., 2015].
Sample Diversity: Though FID captures realism and overall diversity, is known to neglect properties such as intra-class diversity, which is an important feature of a good MI attack. To overcome this limitation, we computed the improved precision & recall [Kynkäänniemi et al., 2019], coverage & density [Naeem et al., 2020] on per class basis, capturing the intra-class diversity of the gen- erated samples. The same Inception network was used as the feature extractor. Recall proposed in Kynkäänniemi al. [2019] can be problematic when the generated samples contain outliers. Coverage proposed in Nacem et al. [2020] is less susceptible to outliers, but also less sensitive. For convenience of comparison, we take the average of the two quantities and call the resulting metric "diversity".
Generators. Two generator models were considered: DCGAN [Radford et al., 2015], and Style- GAN [Karras et al., 2019]. For StyleGAN, we used the original implementations. There were 10 and 12 layers in the synthesis networks for the 64x64 CelebA data and 128x 128 ChestX-ra -ray data respectively. We do not use StyleGAN for the MNIST task since the images are simple.
5.1 Results
Comparison with Baselines. Across all three tasks, our VMI formulations outperformed baselines in terms of target accuracy and sample realism (see Table 1), even when using the same generator architecture (DCGAN). Using a flow model for q(z) ) led to improvements in terms of accuracy. Using the formulation in Equation 5 and a StyleGAN further improved performance on all metrics. VMI also improved the overall sample qualities as evidenced by the lower FIDs. The full VMI (Equation 5) was able to improve FID over pre-trained generators including VMI sacrificed sample quality to achieve higher accuracy, suggesting there is still room for improvements for methods that can retain full realism. Detailed evaluation using sample diversity metrics are reported in Table 2. In general, increasing increased diversity and realism while sacrificing accuracy. Our VMI attack outperformed the baseline all metrics multiple settings. The reported improvements in the quantitative metrics can be verified qualitatively in Figure 5. Detailed results for MNIST and ChestX-ray (Table and 5) can be found in the Appendix C. Using StyleGAN with a Gaussian (z) resulted in a slightly lower accuracy, but better diversity.
CelebA. In general, MI attack methods
Our reported results for our baseline [Zhang et al., 2020] are obtained using our own implementation since the reproduction code was not publicly available. The discrepancy with the original result can come from number of factors: dataset split, target classifier checkpoint, and the hyperparameters involved in the method. We used the task setup described in this paper, and tuned the hyperparameters over reasonable grid of values.
Code from: https /tt///gttttt////////e/eeeeehhtet http/////ww/////tyyggg/////////
8

---

CelehA
MNIST
Real Samples
General
Generative
VMI (ours)
General MI
Generative MI
VMI (ours)
MI
StyleGAN Flow
DOGAN Flow
CoQbo
11 1 4
/ /
*2222
22
33333
33333
Hx
MYW44
5 55T
55555
DEEPA
bhbbh.
77777
XXXXX
19999
9 9399
Figure 5: MI attack samples on the first ten identities of CelebA and MNIST. Each row corresponds to different identity/digit. Qualitative differences are best viewed zoomed in.
VMI (ours)
DCGAN
StyleGAN
Generative
x(2) = Gaussian
Gaussian le-3
Flow le.3
[Zhang et al., 2020]
-y00
lo-3
Te-1
le-1
le-1
Accuracy
0.07_re
a.24
0.37 au
0.56 am
0:23
0.58
0.55g
2.39
Precision
0.51=
0.64=
0.48
0.52x
0.40=
0.87= em
U.88 u
0.82
0.87
am
2.89
Density
0.4
0.67
0.492en 0.52 0.38
T.282 2.20
1.14
T.22g
T,22
1.31
Recall
0.03_
e0.00amm
0.01gom
0. 0.13g
00:22
0.25gmm
0.422
0.11
0.15gam
2m
Coverage
a.79
0.37
0.67
0.70
0.98
0.98
0.98..0
0.96
0,97
0.984.00
Diversity
0.52 cas
0.41 a44
0.19 ecs 0.34 wxe 0.41 car
0.600 ezs
0.61 uo
0.70 023
0.54 coz
0.56 no
0.59 ous
FID
43.21
28.98
58.39
40.89
40.74
16.69
16.11
13,49
17.28
17.41
21.35
Table Detailed comparison between attack methods on CelebA.
Attacking Different Target Classifiers. After establishing the effectiveness of our VMI attack on different datasets, we conducted experiments test the effectiveness of our VMI attack against dif- ferent target classifiers. We further tested two target classifier on CelebA: VGG network [Simonyan and Zisserman, 2014] trained from scratch on our target dataset, and an adapted ArcFace classi- fier [Deng et al., 2019], which we denote as ArcFace-NC. The ArcFace a specialized face identity recognizer pretrained on large collection of datasets. We adapt by using the pretrained network as feature encoder, and adding nearest cluster classifier in the feature space. This approach of adapting a powerful feature extractor is popular and effective in few-shot classification [Chen et al., 2019; Snell et al., 2017]. The resulting accuracy held-out test examples in the target dataset were 62.15 1% (VGG) and 96.11 % (ArcFace-NC) Results Table show that the same observations hold.
Accuracy vs. Diversity Tradeoff. Figure 6 shows the accuracy vs. diversity tradeoff in VMI attacks on CelebA. The VMI attacks that used the StyleGAN generally achieved better accuracy vs. diversity tradeoff than the ones used the DCGAN. We conjecture this is because of the better layer-wise disentanglement that the StyleGAN achieves.
191 perpress chee
Effect of Individual Layers. As motivated Section 4.1, the ex- tended latent space in Equation 5 allowed our attack to modify each layer which controlled different aspects of an image. Figure 7 shows values for the KL-divergence and entropy for qı(z) at all layers after VMI optimization in CelebA and ChestX-ray datasets. We can see a general downward trend in KL divergences and upward trend in entropies of q(z) with respect to layers. This shows that in general the q(z) the earlier layers were modified more by the opti- mization than the (z) of the later layers. This is consistent with observation in Figure 2 that the earlier layers have more control over features relevant to the identity of face.
Dawexty
Figure 6: Accuracy and di-
versity tradeoff on CelebA.
Code and model from: http::://gggtuuu..m///rttt/////e

---

VGG
ArcFace-NC
VMI (OIrS))
VMI (ours)
Generative
DCGAN
StyleGAN
Generative MI
DCGAN
StyleGAN
[Zhung et al., 2020]
Gaussian
Flow
Gaussian
Flow
1Zhang et al., 2020
Gausstan
Flow
Gaussian
Flow
Accuracy
0.04
0. 0.
4.
0.
0.3
gere
0.55gom 0.632
0.52201s
0.9
Precision
0.60z
0.74
0.
0.79-mm
0.59
0,52=mz
0.53am
0.
a.84
Density
0.42g0
0.6324m
0776g
0.04g
T.22.04
0.46ge
0.55ges
0.64gmn
9m26
1m
Recall
0.23 are
0.09
0.02+as:
0,28
0.04:
0.1
0.01= wm 0.00:
0.23
0.04:
Coverage
0.79 our
a.75
0.77
0.93
ose
0.88 aio
0.69 on a.69
0.98
om
4.9
Diversity
0.51
0.4
0.39
0.61
0.46
0.4
0.35
0.35
0.60
0.50
FID
59.27
53.85
57.88
32,51
46.13
63.63
59.23
63.37
32.94
40.22
Table 3: Detailed comparison on CelebA for two other target classifiers: VGG, and ArcFace-NC. FID* reported here was computed over 20 identities instead of 100, so the values can not be directly compared with those in Table 2.
KL-divergence
Entropy
Goussian
Flow
Geussian
Flow
11.
701-
723.6
ChystX-ray8
135
210
60
718
706.3
208.2
Layer
Layer
Layer
Layer
Figure 7: KL and entropy q: (z) after VMI attack on CelebA and ChestX-ray at different layers.
6 Limitation & Ethical Concerns
While our VMI attack was effective, relied on the same assumption made in previous works: the attacker had access to a relevant auxiliary dataset [Yang et al., 2019b; Zhang et al., 2020]. A worth- while future direction is to develop methods that do not depend on this assumption. Methodologically, our proposed VMI attack used GAN to narrow down the search space of the variational objective, but GANs might not be equally effective for other data modalities such as text, and tabular data. Lastly, our study focused on the white-box setting where the attacker had full access to the model; extensions of the current VMI attack to the grey-box or black-box settings are also worthy of further investigation.
In this work, we focused on method for improving model inversion attacks, which could have negative societal impacts if it falls into the wrong hands. As discussed in Section 1, malicious users can use attacks like this to break into secured systems and steal information from private data-centers. On the other hand, by pointing out the extent of the vulnerability of current models, we hope to facilitate research that builds better concerns.
algorithms, and raise awareness about privacy
7 Conclusion
An effective model inversion attack method must produce a set of samples that accurately captures the distinguishing features and variations of the underlying identity/class. Our proposed VMI attack optimizes a variational inference objective by using a combination of a StyleGAN generator and a deep flow model as its variational distribution. Our framework also provides insights into existing attack methods. Empirically, we found our VMI attacks were effective on three different tasks, and across different target classifiers.
Acknowledgments and Disclosure of Funding
Acknowledgement. We would like to thank reviewers and ACs for constructive feedback and discussion. Resources used in preparing this research were provided, in part, by the Province of Ontario, the Government of Canada through CIFAR, and companies sponsoring the Vector Institute www www.vectoriistitute ai ai/#partners
10

---

References
Rameen Abdal, Yipeng Qin, and Peter Wonka. Image2stylegan: How to embed images into the stylegan latent space? In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4432-4441, 2019.
Amazon AWS. Amazon rekognition. URI. htRthhtps://aaa www.///wwwaaaaaaaaaaaaaaaaaaaaaa
amazom www.aaaaa...eaaaeeeeaaaa cow//eeoooniiiioo//
David Bau, Yan Zhu, Hendrik Strobelt, Bolei Zhou, Joshua B Tenenbaum, William T Freeman, and Antonio Torralba. Gan dissection: Visualizing and understanding generative adversarial networks. arXiv preprint arXiv:1811.10997 2018.
Pier Giovanni Bissiri, Chris C Holmes, and Stephen G Walker. A general framework for updating belief distributions. Journal of the Royal Statistical Society. Series B, Statistical methodology, 78(5):1 103, 2016.
David M Blei, Alp Kucukelbir, and Jon D McAuliffe. Variational inference: A review for statisticians. Journal of the American statistical Association, 1 12(5188)8559-877 2017.
Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, Vladimir Ivanov, Chloe Kiddon, Jakub Konečný. Stefano Mazzocchi, H Brendan McMahan, et al. Towards federated learning scale: System design. arXiv preprint (Tti::9902.114// 2019.
Si Chen, Ruoxi Jia, and Guo-Jun Qi. Improved techniques for model inversion attack. arXiv preprint axXIv:2010.04092, 2020.
Wu Chen, Yen-Cheng Liu, 7solt Kira, Yu-Chiang Wang, and Jia-Bin Huang. A closer look at few-shot classification. In International Conference on Learning Representations, 2019.
Gregory Cohen, Saced Afshar, Jonathan Tapson, and Andre Van Schaik. Emnist: Extending mnist to handwritten letters. In 2017 International Joint Conference Neural Networks (IJCNN), pages 2921 12226. IEEE, 2017.
International Warfarin Pharmacogenetics Consortium. Estimation of the warfarin dose with clinical and pharmacogenetic data. New England Journal of Medicine, 360(8)7753-764, 2009.
Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos 7afeiriou. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEF/CVF Conference Computer Vision ond Pattern Recognition, pages 4690-4699, 2019.
Terrance Deeries and Graham W Taylor. Improved regularization of convolutional neural networks with cutout. arXiv preprint 17Xiv:1708.04555 2017.
Laurent Dinh, David Krueger, and Yoshua Bengio. Nice: Non-lincar independent components estimation. arXiv preprint arXiv: 241.::/411..///// 2014.
Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using real nvp. arXiv preprint arXiv:1605.00 08800,,..0...00 2016.
Simon S Du, Wei Hu, Sham M Kakade, Jason D Lec, and Qi Lei. Few-shot learning via learning the representa- tion, provably. arXiv preprint arXiv:2002.09433 2020.
Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model inversion attacks that exploit confidence informa- tion and basic counterncasures. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pages 1322 1333, 2015.
Matthew Fredrikson, Fric Lantz, Somesh Jha, Simon Lin, David Page, and Thomas Ristenpart. Privacy in pharmacogenetics: An end-to-end case study of personalized warfarin dosing. In 23rd (USENIX) Security Symposium (USENIX) Security 14), pages 17-32, 2014.
Javier Galbally, Sébastien Marcel, and Julian Fierrez. Biometric antispoofing methods: A survey in face recognition. IEEE Access, 2:1530 1552, 2014.
Jonas Geiping, Hartmut Bauermeister, Hannah Dröge, and Michael Moeller, Inverting gradicnts-hove easy is it 2020.
to break privacy in federated learning? arXiv preprint arXiv:2003.1400
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair. Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27:2672 2680, 2014.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. arXiv preprint 1706 0456:9999999009999..... 2017.

---

Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge local nash equilibrium. arXiv preprini urXiv:1706.08300, 2017.
Seira Hidano, Takao Murakami, Shuichi Katsumata, Shinsaku Kiyomoto, and Goichiro Hanaoka. Model inversion attacks for prediction systems: Without knowledge of non-sensitive attributes. In 2017 15th Annual Conference on Privacy, Security and Trust (PST), pages 115-11509. IEFE, 2017.
Briland Hitaj, Giuseppe Ateniese, and Fernando Berez-Cruz. Deep models under the gan: information leakage from collaborative deep learning. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pages 603-618, 2017.
Corey D Holland and Oleg V Komogortsev. Complex eye movement pattern biometrics: Analyzing fixations and saccades. In 2013 International conference on biometrics (ICB), pages 1-8. IEEE, 2013.
CC Holmes and SG Walker. Assigning value power likelihood in general bayesian model. Biometrika, 104(2):497 503, 2017.
Michael I Jordan, Zoubin Ghahramani, Tommi S Jaakkola, and Lawrence K Saul, An introduction variational methods for graphical models. Machine learning, 37(2):183-233, 1999.
Georgios A Kaissis, Marcus R Makowski, Daniel Rückert, and Rickmer F Braren. Secure, and federated machine learning in medical imaging. Nature Machine Intelligence, 2(6):305-311, 2020.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4401-4410, 2019.
Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila, Analyzing and improving the image quality of stylegan. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 81 811081199 19, 2020.
Mahdi Khosravy. Kazuaki Nakamura, Naoko Nitta, and Noboru Babaguchi. Deep face recognizer privacy attack: Model inversion initialization by a deep generative adversarial data space discriminator. In 2020 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). pages 1400- IEEE, 2020.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
Durk P Kingma and Prafulla Dhariwal. Glow:
Generative flow with invertible 1xl convolutions. In Advances in Neural Information Processing Systems, pages 10215-10224, 2018
Jeremias Knoblauch, Jack Jewson, and Theodoros Damoulas. Generalized variational inference: Three arguments deriving new posteriors. arXiv preprint (tti:1944.00000 2019.
Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models. arXiv preprint arXiv: 1904.06991, 2019.
Yann LeCun, Corinna Cortes, and CJ Burges. Mnist handwritten digit database, 2010.
Erich Lehmann and Joseph Romano. Testing statistical hypotheses. Springer Science Business Media, 2006.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xianou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
Aravindh Mahendran and Andrea Vedaldi. Understanding deep image representations by inverting them. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 5188-5196, 2015.
Jeffrey W Miller and David B Dunson. Robust bayesian inference via coarsening. Journal of the American Statistical Association, 2018
Muhammad Ferjad Nacem, Scong Joon Oh, Youngjung Uh, Yunjey Choi, and Jacjun Yoo. Reliable fidelity and diversity metrics for generative models. In International Conference on Machine Learning, pages 7176-7185. PMLR, 2020.
Milad Nasr, Reza Shokri, and Amir Houmansadr. Comprehensive privacy analysis of deep learning: Passive and active white-bor inference attacks against centralized and federated learning. In 2019 IEEE symposium on security and privacy (SP), pages 739-77 733-753 IEEE, 2019.
12

---

Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Conference on Computer Vision and Pattern Recognition, pages 427-436, 2015.
Anh Nguyen, Jeff Clune, Yoshua Bengio, Alexey Dosovitskiy, and Jason Yosinski. Plug & play generative networks: Conditional iterative generation of images in latent space. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4467-4477, 2017.
Omkar Parkhi, Andrea Vedaldi, and Andrew Zisserman. Deep face recognition. 2015.
Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint urXiv:1511.0645 2015.
Christian Rathgeb and Andreas Uhl. A survey on biometric cryptosystems and cancelable biometrics. EURASIP Journal on Information Security, 201 (1I:1125, 2011.
Maria Rigaki and Scbastian Garcia. survey of privacy attacks in machine Icarning. arXiv preprint arXiv:2007. 07646, 2020.
Thomas Schlegl, Philipp Sechöck, Sehastian M Waldstein, Ursula Schmidt-Frfurth and Georg Langs. Unsuper- vised anomaly detection with generative adversarial networks to guide marker discovery. In International conference on information processing in medical imaging, pages 146-157. Springer, 2017.
Elizabeth Sconce, Tayyaba Khan, Hilary Wynne, Peter Avery, Louise Monkhouse, Barry King, Peter Wood, Patrick Kesteven, Ann K Daly, and Farhad Kamali. The impact of cyp2c9 and vkore genetic polymorphism and patient characteristics upon warfarin dose requirements: proposal for new dosing regimen. Blood, 106(7):2329 (006)))2329-2333,.. 2005.
Yujun Shen, Ceyuan Yang, Xiaoou Tang, and Bolei Zhou. Interfacegan: Interpreting the disentangled face representation learned by gans. IEEE Transactions on Puttern Analysis and Machine Intelligence, 2020.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against. machine learning models. In 2017 IEEE Symposium Security and Privacy (SP), pages 3-18. IEEE, 2017.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: httl://49..155 15500 2014.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv: 1312.6034, 2013.
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In Advances in Neural Information Processing Systems, pages 4077-4087, 2017.
Christian Szegedy, Wojeiech Zaremba, llya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv: 2013.
Christian Szegedy, Wei Liu, Yangging Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEFE conference on computer vision and pattern recognition, pages 1-9, 2015.
Google TensorFlow. Tensorflow hub. URL https:/ /wwp://wwwwww/// http:::/wwww.ennortllle tensorflo org/hub.
Nilesh Tripuraneni. Michael I Jordan, and Chi Jin. On the theory of transfer learning: The importance of task diversity, arXiv preprint arXiv:2006. /6500, /650,
2020.
Xiaosong Wang. Yifan Peng. Le Lu, Zhiyong Lu, Mohammadhadi Bagheri, and Ronald M Summers. Chestx- ray8: Hospital-scale scale chest x-ray database and benchmarks on weakly-supervist classification and localization of common thorax diseases. In Proceedings of the IEEE conference on computer vision and pattern recognition. pages 2097-2106, 2017.
Yaxing Wang, Abel Gonzalez- Gonzalez-Garcia, David Berga, Luis Herranz, Fahad Shahbaz Khan, and Joost van de Weijer. Minegan: effective knowledge transfer from gans to target domains with few images. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 9332-9341, 2020.
Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. Federated machine learning: Concept and applications.
ACM Transactions Intelligent Systems and Technology (TIST), 10(2):1-99, 2019a.
Ziqi Yang, Ee-Chien Chang, and Zhenkai Liang. Adversarial neural network inversion via auxiliary knowledge alignment. arXiv preprint 1902. 2019b.
13

---

Raymond A Yeh, Chen Chen, Teck Yian Lim. Alexander G Schwing, Mark Hasegawa-Jolnson, and Minh N Do. Semantic image inpainting with deep generative models. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 5485-5493, 2017.
Hongxu Yin, Pavlo Molchanov, Jose M Alvarez. Zhizhong Li, Arun Mallya, Derek Hoiem, Niraj K Jha, and Jan Kautz. Dreaming to distill: Data-free knowledge transfer via deepinversion. Proceedings of IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 8715-8724 2020.
Yuheng Zhang, Ruoxi Jia, Hengzhi Pei, Wenxiao Wang, Bo Li, and Dawn Song. The secret revealer: generative model-inversion attacks against deep neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 253 261, 2020.
14


Reviews:
[{"review_id": "hIx-eM5S1fQ", "reviewer": "Reviewer_uEuV", "summary": "This paper frames the model inversion problem as a variational inference problem, where the goal is to approximate the posterior distribution p^{tar}(x|y) from the known target classifier p^{tar}(y|x). From a class of candidate distributions q(x), the aim is to find one that is close to p^{tar}(x|y), and this is achieved by minimizing the KL-divergence between the 2 distributions. Such a formulation allows the authors to make a principled derivation of an objective function that encourages both the optimized distribution to produce images likely to be classified correctly by the target classifier, and the produced images to be realistic. It is also a unified framework for existing methods of generative model inversion.\n\nAdditionally, the authors propose using the StyleGAN architecture as a pre-trained generator for the model inversion task. This allows intermediate layers (as opposed to just the input latent code in previous formulations) of the GAN to be optimized during the search for q*, improving the expressiveness of the GAN. The StyleGAN architecture also means that the attack can focus on layers whose representations control the semantics of the final image. \n", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The techniques proposed in this paper are not entirely novel by themselves (e.g. distributional recovery in model inversion has been proposed in Si Chen, Ruoxi Jia, and Guo-Jun Qi. Improved techniques for model inversion attack.) However, the novelty is the theoretical framework used to motivate such techniques - framing the model inversion problem as approximating the posterior p^{tar}(x|y) distribution. \n\nThe proposal to use StyleGAN as the generator architecture makes sense, since individual layers can potentially control different features related to identity. However, this is not particularly groundbreaking. One can imagine trying out other architectures used in unsupervised disentangled representation learning as well (e.g. InfoGAN).\n\nIn the derivation of the objective function, a number of assumptions are made. Assumption 1 assumes that a generator trained on the auxiliary dataset can capture the distribution of the target data. However, given that the target data and the auxiliary data are meant to be disjoint (should not contain the same classes), it is not immediately obvious that this assumption holds. For example, the generator trained on the ‘letters’ split of the EMNIST dataset is unlikely to capture the distribution of the ‘digits’ images in the MNIST dataset.\n\nThere are also some concerns about the reported results: in table 2, Generative MI on CelebA dataset is reported to have an accuracy of only 6%. However, in the original paper cited (Zhang 2020), the authors reported an accuracy of 44%. What is the reason for the discrepancy? If Generative MI can achieve 44% accuracy, then VMI w/DCGAN which achieves 37% accuracy is not better. The same applies to the Chest X-ray dataset which in the Zhang et al has an accuracy of 71% but is reported to be 28% in this work. Though there are some differences in training data between the implementations, it does not explain such a huge discrepancy in results.\n\n========== after rebuttal ===========\n\n\n\nThere is large discrepancy between the results reported in this paper and the original GMI paper Zhang et al. 2020.\n\n\nGiven such a large discrepancy (6% reported in this submission vs 44% reported in Zhang et al. 2020), and VMI w/ DCGAN actually has poorer accuracy than the original GMI (37% vs 44%), it would be difficult for me to further increase my rating.\n\n\n\n\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "yes, this is discussed in sec 6.\n\n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6 hours"}, "extra_text": {}}, {"review_id": "BneiXG_jCE1", "reviewer": "Reviewer_He1L", "summary": "The paper at hand propose a new model inversion attack that recovers the training dataset that was used to learn a supervised model. The proposed attack uses generative adversarial networks (namely a DCGAN and a StyleGAN) and variational inference to invert the probability distribution $p(y|x)$ that is learned by the classifier. The authors also describe how the proposed VMI framework can be seen as an unified framework of existing model inversion attacks. Experimental evaluation was performed on MNIST, the ChestX-ray dataset and the CelebA dataset.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "REASONS FOR SCORE:\n\n- The reported experimental results look promising, however I have some concerns about the evaluation:\n    1. As the generator is in some cases also trained on the actual input distribution it would be important to also compare the reported FID scores to FID scores of the generator before the attack was performed.\n    2. Related works (e.g., [2] which is also cited by the authors) also evaluate on the CIFAR10 dataset. The authors might consider extending their experiments to also include that dataset.\n    3. What is the test accuracy of the target classifiers before the attack was performed?\n    4. How does VMI compare to General MI in terms of computational runtime?\n\n- The structure of the paper is reasonable, but the mathematical writing can be improved at many places: \n    1. The notation $G : z \\rightarrow x$ (used, e.g., in l. 129 and l. 135) is not correct, as $x$ and $z$ are random variables and not sets.\n    2. In l. 119: What is p? (I suppose $p^\\text{tar}$ is meant.)\n    3. In l. 143f: \"$\\mathcal{J}(q)$ can be rewritten as...\" (I think \"rewritten\" is the wrong word here, because the following inequality is a lower bound!)\n    4. In Eqn. 5: The authors should write $x^*$ instead of $x'$ (which would be more consistent with the notation of the rest of the paper)\n    5. In l. 136: It would be better to write $p^G$ instead of $p_G$ (as it would be more consistent with $p^\\text{tar}$, $p^\\text{aux}$). \n\n- Zhang et al. [1] are able to use auxiliary information about the retrieved images, e.g., blurred versions of the images or partly occluded images. How can auxiliary information be incorporated into this framework? \n\nCONCLUSION:\n\nOverall, I lean towards rejection. The proposed method sure has its merits, but the writing of the paper can be improved and the discussion of the performed experiments lacks some details.\n\nMINOR REMARKS:\n\n- Subsections not always in title-case (for example: \"Relationships with existing methods\" vs. \"Model Class of q(z)\")\n- l.219: \\gamma=0 not in math mode\n\nREFERENCES:\n\n[1] Zhang, Yuheng, et al. \"The secret revealer: Generative model-inversion attacks against deep neural networks.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020.\n\n[2] Si Chen, Ruoxi Jia, and Guo-Jun Qi. Improved techniques for model inversion attack. arXiv preprint arXiv:2010.04092, 2020.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "4mtiG7EgeSi", "reviewer": "Reviewer_Zyff", "summary": "This paper frames model inversion attacks as a variational inference problem, leading to the derivation shown in Equation one. Following prior work, they also assume access to a common generator, than can be learned on auxiliary data and can capture a shared conditional distribution $p_G(x|z)$. With this assumption, they can derive a practical objective as shown in Equation 3, which maximizes (in code space) the attack accuracy (first term) and the realism / diversity of the generated samples (KL term). \n\nFrom an empirical standpoint, they experiment using both DGGAN and StyleGAN as their pretrained generator, and using either a gaussian or flow model to form their $q(z)$. They show that their model outperforms prior work across MNIST, CelebA and CXR datasets in terms of attack accuracy, FID and diversity. \n", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality:\nThis study offers a well-motivated framing of model inversion attacks (i.e as variational inference), and this framing offers a useful lense to unify the prior work and leads to a natural objective. This offers a novel perspective to this line of work. \n\nQuality:\nThe core claims of this paper (i.e improving the performance of model inversion attacks) are well supported by their empirical results across MNIST, CelebA and CXR. \n\nClarity:\nAs a whole, the paper is well written and easy to follow. There were a few areas that I thought could benefit from improved exposition:\n\nSection 4.2:\n      There really isn’t enough detail here to get a concrete idea of what is being used without just looking through the code. If there isn’t enough space, this should maybe be detailed better in an appendix. \n\nSection 4.1:\n      This description was difficult to follow. \n\n\nSignificance:\nThis paper offers a useful new perspective on model inversion attacks, and their constriction leads to improved attacks. I think this study is useful for this ongoing work in better understanding, developing and defending against model inversion attacks. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, the authors noted their primary limitation (assumption 1) and indicated multiple useful areas for future work. They also noted the possible ethical downsides of this line of work (people doing more powerful attacks). ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a well-motivated method with technical novelty, and good performance is shown.  
Reviewer's major concerns, including the questionable assumption and the seemingly inconsistent baseline results with the original papers, have been addressed.  
Mathematical notation should be improved in the camera-ready version.
Decision: Accept (Poster)