Variational Model Inversion Attacks

1 Introduction . Thanks to recent advances , deep neural networks are now widely used in applications including facial recognition [ Parkhi et al. , 2015 ] , intelligent automated assistants , and personalized medicine [ Consortium , 2009 ; Sconce et al. , 2005 ] . Powerful deep neural networks are trained on large datasets that could contain sensitive information about individuals . Publishers release only the trained models ( e.g. , on TensorFlow Hub [ TensorFlow ] ) , but not the original dataset to protect the privacy of individuals whose information is in the training set . Regrettably , the published model can leak information about the original training set [ Geiping et al. , 2020 ; Nasr et al. , 2019 ; Shokri et al. , 2017 ] . Attacks that exploit such properties of machine learning models are privacy attacks [ Rigaki and Garcia , 2020 ] . Other real-world scenarios where an attacker can access models trained on private datasets include interaction with APIs ( e.g. , Amazon Rekognition [ AWS ] ) , or in the growing paradigm of federated learning [ Bonawitz et al. , 2019 ; Yang et al. , 2019a ] , where a malicious insider can try to steal data from other data centers [ Hitaj et al. , 2017 ] . Kaissis et al . [ 2020 ] provides an overview of how these privacy concerns are hindering the widespread application of deep learning in the medical domain . This paper focuses on the model inversion ( MI ) attack , a type of privacy attack that tries to recover the training set given access only to a trained classifier [ Chen et al. , 2020 ; Fredrikson et al. , 2015 , 2014 ; Hidano et al. , 2017 ; Khosravy et al. , 2020 ; Yang et al. , 2019b ; Zhang et al. , 2020 ] . The classifier under attack is referred to as the “ target classifier ” ( see Figure 1 ) . An important relevant application concerns identity recognition from face images , since facial recognition is a common approach to biometric identification [ Galbally et al. , 2014 ; Rathgeb and Uhl , 2011 ] . Once an attacker successfully reconstructs images of the faces of individuals in the private training set , they can use the stolen identity to break into otherwise secure systems . Early studies of MI attacks focused on attacking classifiers whose inputs are tabular data [ Fredrikson et al. , 2015 , 2014 ; Hidano et al. , 2017 ] by directly performing inference in the input space . However , the same methods fail when the target classifier processes high-dimensional inputs , e.g. , in the above 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) , held virtually . example of a face recognizer . Recent methods cope with this high-dimensional search space by performing the attack in the latent space of a neural network generator [ Khosravy et al. , 2020 ; Yang et al. , 2019b ; Zhang et al. , 2020 ] . Introducing a generator works well in practice , but this success is not theoretically understood . In this paper , we frame MI attacks as a variational inference ( VI ) [ Blei et al. , 2017 ; Jordan et al. , 1999 ; Kingma and Welling , 2013 ] problem , and derive a practical objective . This view allows us to justify existing approaches based on a deep generative model , and identify missing ingredients in them . Empirically , we find that our proposed objective leads to significant improvements : more accurate and diverse attack images , which are quantified using standard sample quality metrics ( i.e. , FID [ Heusel et al. , 2017 ] , precision , and recall [ Kynkäänniemi et al. , 2019 ; Naeem et al. , 2020 ] ) . In summary , our contributions and findings are : • We view the model inversion ( MI ) attack problem as a variational inference ( VI ) problem . This view allows us to derive a practical objective based on a statistical divergence , and provides a unified framework for analyzing existing methods . • We provide an implementation of our framework using a set of deep normalizing flows [ Dinh et al. , 2014 , 2016 ; Kingma and Dhariwal , 2018 ] in the extended latent space of a StyleGAN [ Karras et al. , 2020 ] . This implementation can leverage the hierarchy of learned representations , and perform targeted attacks while preserving diversity . • Empirically , on the CelebA [ Liu et al. , 2015 ] , and ChestX-ray [ Wang et al. , 2017 ] datasets , our method results in higher attack accuracy , and more diverse generation compared to existing methods . We provide thorough comparative experiments to understand which components contribute to improved target accuracy , sample realism , and sample diversity , and detail the benefits of the VI perspective . Code can be found at https : //github.com/wangkua1/vmi . 2 Problem Setup : Model Inversion Attack . In the problem of model inversion ( MI ) attack , the attacker has access to a “ target classifier ” : pTAR ( y|x ) : Rd → ∆C−1 where ∆C−1 to denote the ( C − 1 ) -simplex , representing the C-dimensional probability where C is the number of classes . This target classifier is trained on the private target dataset , DTAR = { xi , yi } NTARi=1 where x ∈ Rd and y ∈ { 1 , 2 , ... , C } . We use pTAR ( y|x ) to denote the given target classifier , which is an approximation of the true conditional probability pTAR ( y|x ) of the underlying data generating distribution . Goal . Given a target classifier pTAR ( y|x ) , we wish to approximate the class conditional distribution pTAR ( x|y ) without having access to the private training set DTAR . A good model inversion attack should approximate the Bayes posterior pTAR ( x|y ) ∝ pTAR ( y|x ) pTAR ( x ) well , and allow the attacker to generate realistic , accurate , and diverse samples . Notice , our goal shifts slightly from recovering the exact instances in the training set to recovering the data generating distribution of the training data . Let ’ s consider a scenario where the attacker intends to fake a victim ’ s identity . A good security system might test the realism of a sequence of inputs [ Holland and Komogortsev , 2013 ] . More generally , faking someone ’ s identity requires passing a two-sample test [ Holland and Komogortsev , 2013 ; Lehmann and Romano , 2006 ] , which means the ability to generate samples from pTAR ( x|y ) is necessary . Another scenario can be an attacker trying to steal private/proprietary information . For example , a malicious insider in a federated learning setting can try to steal information from another datacenter by accessing the shared model [ Hitaj et al. , 2017 ] . In such a scenario , a good pTAR ( x|y ) model can reveal not only the distinguishing features of the data , but also its variations . 3 Related Work . In this section , we first discuss related model inversion attack studies , and then discuss applications of inverting a classifier outside of the privacy attack setting . Lastly , since our attack method relies on using a pretrained generator , we discuss existing studies that used a pretrained generator for improving other applications . Model inversion attacks . MI attacks are one type of privacy attack where the attacker tries to reconstruct the training examples given access only to a target classifier . Other privacy attacks include membership attacks , attribute inference , and model extraction attacks ( see Rigaki and Garcia [ 2020 ] for a general discussion ) . Fredrikson et al . [ Fredrikson et al. , 2015 , 2014 ] were the first to study the MI attack and demonstrated successful attacks on low-capacity models ( e.g. , logistic regression , and a shallow MLP network ) when partial input information was available to the attacker . The setting where partial input information was not available was studied by Hidano et al . [ 2017 ] . They termed performing gradient ascent w.r.t . the target classifier in the input space a General Model Inversion attack . Though effective on tabular data , this approach failed on deep image classifiers . It is well-known that directly performing gradient ascent in the image space results in imperceptible and unnatural changes [ Szegedy et al. , 2013 ] . To handle high-dimensional observations such as images , MI attacks based on deep generators were proposed [ Chen et al. , 2020 ; Yang et al. , 2019b ; Zhang et al. , 2020 ] . By training a deep generator on an auxiliary dataset , these methods effectively reduce the search space to only the manifold of relevant images . For example , if the target classifier is a facial recognition system , then an auxiliary dataset of generic faces can be easily obtained . Zhang et al . [ 2020 ] pretrain a Generative Adversarial Network ( GAN ) [ Goodfellow et al. , 2014 ] on the auxiliary dataset and performed gradient-based attack in the latent space of the generator . Yang et al . [ 2019b ] trained a generator that inverts the prediction of the target classifier , using the architecture of an autoencoder . Other applications of classifier inversion . Inverting the predictions and activations of a classifier has been studied for applications such as model explanation , and model distillation . Motivated by using saliency maps to explain the decision of a classifier [ Simonyan et al. , 2013 ] , Mahendran and Vedaldi [ 2015 ] proposed to maximize the activation of a target neuron in a classifier for synthesizing images that best explain that neuron . Their method is based on gradient ascent in the pixel-space . The same idea was recently extended to enable knowledge distillation in a setting where the original dataset was too large to be transferred [ Yin et al. , 2020 ] . Methods above have the advantage of not requiring the original dataset , but the generated images still appear unnatural . One effective way to improve the realism of the generated images is to optimize in the latent code space of a deep generator [ Nguyen et al. , 2015 , 2017 ] . Pretrained generators . Pretrained generators from GANs have other applications . Image inpainting/restoration was one of the early successful applications of using a pretrained GAN as image prior . Yeh et al . [ 2017 ] optimized the latent code of GANs to minimize the reconstruction error of a partially occluded image for realistic inpainting . The same method formed the basis of AnoGAN , a popular technique for anomaly detection in medical images [ Schlegl et al. , 2017 ] . A combination of reconstruction error and discriminator loss was used as the anomaly score . A potential future direction is to apply our proposed method to these adjacent applications . 4 Model Inversion Attack as Variational Inference . In this section , we derive our Variational Model Inversion ( VMI ) learning objective . For each class label y , we wish to approximate the true target posterior with a variational distribution q ( x ) ∈ Qx , where Qx is the variational family . This can be achieved by the optimizing the following objectives q∗ ( x ) = argmin q∈Qx { DKL ( q ( x ) ||pTAR ( x|y ) ) } = argmin q∈Qx { Eq ( x ) [ − log pTAR ( y|x ) ] +DKL ( q ( x ) ||pTAR ( x ) ) + log pTAR ( y ) } = argmin q∈Qx { Eq ( x ) [ − log pTAR ( y|x ) ] +DKL ( q ( x ) ||pTAR ( x ) ) } . ( 1 ) where in Equation 1 we have used the fact that log pTAR ( y ) is independent of q ( x ) , and thus can be dropped in the optimization . Auxiliary Image Prior . In order to minimize Eq . ( 1 ) , we need to compute DKL ( q ( x ) ||pTAR ( x ) ) , which requires evaluating the true image prior pTAR ( x ) , that is not accessible . Similar to recent works on MI attacks [ Khosravy et al. , 2020 ; Yang et al. , 2019b ; Zhang et al. , 2020 ] , we assume that a public auxiliary dataset DAUX , which shares structural similarity with the target dataset , is available to learn an auxiliary image prior pAUX ( x ) . This auxiliary image prior pAUX ( x ) is learned using a generative adversarial network ( GAN ) pAUX ( x ) = EpAUX ( z ) [ pG ( x|z ) ] , where pAUX ( z ) = N ( 0 , I ) is the GAN prior , and pG ( x|z ) is the GAN generator that is parameterized by a deep neural network G ( z ) , and a Gaussian observation noise with small standard deviation σ : pG ( x|z ) = N ( G ( z ) , σ2I ) , or equivalently x = G ( z ) + σϵ , ϵ ∼ N ( 0 , I ) . The learned generator pG ( x|z ) defines a low dimensional manifold in the observation space x , when z is chosen to have fewer dimensions than x . We now make the following assumption about the structural similarity of pAUX ( x ) and pTAR ( x ) . Assumption 1 ( Common Generator ) . We assume pTAR ( x ) is concentrated close to the low dimensional manifold of pG ( x|z ) , learned from the auxiliary dataset DAUX . In other words , we assume there exists a target prior pTAR ( z ) , possibly different from pAUX ( z ) , such that pTAR ( x ) ≈ EpTAR ( z ) [ pG ( x|z ) ] . We refer to pG ( x|z ) as the “ common generator ” . The name “ common generator ” is inspired by recent progress in theoretical analyses of few-shot and transfer learning , where assumptions about an underlying shared representation have been used to provide error bounds . In these works , this shared representation is referred to as the “ common representation ” [ Du et al. , 2020 ; Tripuraneni et al. , 2020 ] . Assumption 1 usually holds in the context of model inversion attacks . For example , pAUX ( x ) could be the distribution of all human faces , while pTAR ( x ) could be the distribution of the celebrity faces ; or pAUX ( x ) could be the distribution of natural images , while pTAR ( x ) could be the distribution of different breeds of dogs . In these examples , both pAUX ( x ) and pTAR ( x ) approximately live close to the same low dimensional manifold , but the distribution on this shared manifold could be different , i.e. , pTAR ( z ) ̸= pAUX ( z ) . Variational Family . We further consider our variational family Qx to be all distributions that lie on this manifold by assuming that q ( x ) is of the form q ( x ) = Eq ( z ) [ pG ( x|z ) ] , q ( z ) ∈ Qz , where Qz is a variational family in the code space . In the next section , we further restrict Qz to be either Gaussian distributions or normalizing flows . Proposition 1 . We have DKL ( q ( z ) ||pTAR ( z ) ) ≥ DKL ( Eq ( z ) [ pG ( x|z ) ] ||EpTAR ( z ) [ pG ( x|z ) ] ) . See Appendix A for the proof . Using Assumption 1 , we can approximate the objective of Equation 1 with q∗ ( z ) = argmin q∈Qz { Ez∼q ( z ) , ϵ∼N ( 0 , I ) [ − log pTAR ( y|G ( z ) + σϵ ) ] +DKL ( Eq ( z ) [ pG ( x|z ) ] ||EpTAR ( z ) [ pG ( x|z ) ] ) } , ( 2 ) where this objective is now defined in terms of q ( z ) ∈ Qz in the latent space of the GAN . Now using the Proposition 1 , we can obtain the following upper bound on the objective of Equation 2 ( assuming σ = 0 ) q∗ ( z ) = argmin q∈Qz { Ez∼q ( z ) [ − log pTAR ( y|G ( z ) ) ] +DKL ( q ( z ) ||pTAR ( z ) ) } . ( 3 ) Power Posteriors . We now consider the optimization of Equation 3 . Since we do not have access to pTAR ( z ) , we propose to replace it with pAUX ( z ) , noting that , as argued above , pAUX ( z ) may not accurately represent pTAR ( z ) . We also propose to replace the unknown likelihood pTAR ( y|G ( z ) ) with the pTAR ( y|G ( z ) ) defined by the target classifier , noting that this likelihood could be miscalibrated . Miscalibration is a known problem in neural network classifiers [ Guo et al. , 2017 ] ; namely , the classifier predictions can be over- or under-confident . In order to account for these issues , instead of the standard Bayes posterior qBAYES ( z ) ∝ pAUX ( z ) pTAR ( y|G ( z ) ) as the solution of Equation 3 , we consider the family of “ power posterior ” 1 [ Bissiri et al. , 2016 ; Holmes and Walker , 2017 ; Knoblauch et al. , 2019 ; Miller and Dunson , 2018 ] , which raise the likelihood function to a power 1γ q∗γ ( z ) ∝ pAUX ( z ) p 1 γ TAR ( y|G ( z ) ) , and can be characterized as the solution of the following optimization problem : q∗γ ( z ) = argmin q∈Qz LγVMI ( q ) , ( 4 ) LγVMI ( q ) : = Ez∼q ( z ) [ − log pTAR ( y|G ( z ) ) ] + γDKL ( q ( z ) ||pAUX ( z ) ) . See Appendix A for the proof . If the prior pAUX ( z ) is not reliable , by decreasing γ , we can reduce the importance of the prior . Similarly , by adjusting γ , we can reduce or increase the importance of overor under-confident likelihood predictions pTAR ( y|G ( z ) ) . Accuracy vs. Diversity Tradeoff . In the context of model inversion attack , the Bayes posterior qBAYES ( z ) can only achieve a particular tradeoff between the accuracy and diversity of the generated samples . However , the family of power posteriors enables us to achieve an arbitrary desired tradeoff between accuracy and diversity by tuning γ . If γ = 0 , then q∗γ=0 ( z ) = δ ( z − z∗ ) , and the model essentially ignores the image prior and diversity , and finds a single point estimate z∗ on the manifold which maximizes the accuracy pTAR ( y|G ( z ) ) . As we increase γ , the the model achieves larger diversity at the cost of smaller average accuracy . When γ = 1 , we recover the Bayes posterior q∗γ=1 ( z ) ∝ pAUX ( z ) pTAR ( y|G ( z ) ) . In the limit of γ → ∞ , we have q∗∞ ( z ) = pAUX ( z ) . In this case , the model ignores the classifier and accuracy , and sample unconditionally from the auxiliary distribution ( maximum diversity ) . We empirically study the effect of γ in Section 5 .

Summary: This paper frames model inversion attacks as a variational inference problem, leading to the derivation shown in Equation one. Following prior work, they also assume access to a common generator, than can be learned on auxiliary data and can capture a shared conditional distribution $p_G(x|z)$. With this assumption, they can derive a practical objective as shown in Equation 3, which maximizes (in code space) the attack accuracy (first term) and the realism / diversity of the generated samples (KL term).   From an empirical standpoint, they experiment using both DGGAN and StyleGAN as their pretrained generator, and using either a gaussian or flow model to form their $q(z)$. They show that their model outperforms prior work across MNIST, CelebA and CXR datasets in terms of attack accuracy, FID and diversity.