Variational Model Inversion Attacks

1 Introduction . Thanks to recent advances , deep neural networks are now widely used in applications including facial recognition [ Parkhi et al. , 2015 ] , intelligent automated assistants , and personalized medicine [ Consortium , 2009 ; Sconce et al. , 2005 ] . Powerful deep neural networks are trained on large datasets that could contain sensitive information about individuals . Publishers release only the trained models ( e.g. , on TensorFlow Hub [ TensorFlow ] ) , but not the original dataset to protect the privacy of individuals whose information is in the training set . Regrettably , the published model can leak information about the original training set [ Geiping et al. , 2020 ; Nasr et al. , 2019 ; Shokri et al. , 2017 ] . Attacks that exploit such properties of machine learning models are privacy attacks [ Rigaki and Garcia , 2020 ] . Other real-world scenarios where an attacker can access models trained on private datasets include interaction with APIs ( e.g. , Amazon Rekognition [ AWS ] ) , or in the growing paradigm of federated learning [ Bonawitz et al. , 2019 ; Yang et al. , 2019a ] , where a malicious insider can try to steal data from other data centers [ Hitaj et al. , 2017 ] . Kaissis et al . [ 2020 ] provides an overview of how these privacy concerns are hindering the widespread application of deep learning in the medical domain . This paper focuses on the model inversion ( MI ) attack , a type of privacy attack that tries to recover the training set given access only to a trained classifier [ Chen et al. , 2020 ; Fredrikson et al. , 2015 , 2014 ; Hidano et al. , 2017 ; Khosravy et al. , 2020 ; Yang et al. , 2019b ; Zhang et al. , 2020 ] . The classifier under attack is referred to as the “ target classifier ” ( see Figure 1 ) . An important relevant application concerns identity recognition from face images , since facial recognition is a common approach to biometric identification [ Galbally et al. , 2014 ; Rathgeb and Uhl , 2011 ] . Once an attacker successfully reconstructs images of the faces of individuals in the private training set , they can use the stolen identity to break into otherwise secure systems . Early studies of MI attacks focused on attacking classifiers whose inputs are tabular data [ Fredrikson et al. , 2015 , 2014 ; Hidano et al. , 2017 ] by directly performing inference in the input space . However , the same methods fail when the target classifier processes high-dimensional inputs , e.g. , in the above 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) , held virtually . example of a face recognizer . Recent methods cope with this high-dimensional search space by performing the attack in the latent space of a neural network generator [ Khosravy et al. , 2020 ; Yang et al. , 2019b ; Zhang et al. , 2020 ] . Introducing a generator works well in practice , but this success is not theoretically understood . In this paper , we frame MI attacks as a variational inference ( VI ) [ Blei et al. , 2017 ; Jordan et al. , 1999 ; Kingma and Welling , 2013 ] problem , and derive a practical objective . This view allows us to justify existing approaches based on a deep generative model , and identify missing ingredients in them . Empirically , we find that our proposed objective leads to significant improvements : more accurate and diverse attack images , which are quantified using standard sample quality metrics ( i.e. , FID [ Heusel et al. , 2017 ] , precision , and recall [ Kynkäänniemi et al. , 2019 ; Naeem et al. , 2020 ] ) . In summary , our contributions and findings are : • We view the model inversion ( MI ) attack problem as a variational inference ( VI ) problem . This view allows us to derive a practical objective based on a statistical divergence , and provides a unified framework for analyzing existing methods . • We provide an implementation of our framework using a set of deep normalizing flows [ Dinh et al. , 2014 , 2016 ; Kingma and Dhariwal , 2018 ] in the extended latent space of a StyleGAN [ Karras et al. , 2020 ] . This implementation can leverage the hierarchy of learned representations , and perform targeted attacks while preserving diversity . • Empirically , on the CelebA [ Liu et al. , 2015 ] , and ChestX-ray [ Wang et al. , 2017 ] datasets , our method results in higher attack accuracy , and more diverse generation compared to existing methods . We provide thorough comparative experiments to understand which components contribute to improved target accuracy , sample realism , and sample diversity , and detail the benefits of the VI perspective . Code can be found at https : //github.com/wangkua1/vmi . 2 Problem Setup : Model Inversion Attack . In the problem of model inversion ( MI ) attack , the attacker has access to a “ target classifier ” : pTAR ( y|x ) : Rd → ∆C−1 where ∆C−1 to denote the ( C − 1 ) -simplex , representing the C-dimensional probability where C is the number of classes . This target classifier is trained on the private target dataset , DTAR = { xi , yi } NTARi=1 where x ∈ Rd and y ∈ { 1 , 2 , ... , C } . We use pTAR ( y|x ) to denote the given target classifier , which is an approximation of the true conditional probability pTAR ( y|x ) of the underlying data generating distribution . Goal . Given a target classifier pTAR ( y|x ) , we wish to approximate the class conditional distribution pTAR ( x|y ) without having access to the private training set DTAR . A good model inversion attack should approximate the Bayes posterior pTAR ( x|y ) ∝ pTAR ( y|x ) pTAR ( x ) well , and allow the attacker to generate realistic , accurate , and diverse samples . Notice , our goal shifts slightly from recovering the exact instances in the training set to recovering the data generating distribution of the training data . Let ’ s consider a scenario where the attacker intends to fake a victim ’ s identity . A good security system might test the realism of a sequence of inputs [ Holland and Komogortsev , 2013 ] . More generally , faking someone ’ s identity requires passing a two-sample test [ Holland and Komogortsev , 2013 ; Lehmann and Romano , 2006 ] , which means the ability to generate samples from pTAR ( x|y ) is necessary . Another scenario can be an attacker trying to steal private/proprietary information . For example , a malicious insider in a federated learning setting can try to steal information from another datacenter by accessing the shared model [ Hitaj et al. , 2017 ] . In such a scenario , a good pTAR ( x|y ) model can reveal not only the distinguishing features of the data , but also its variations . 3 Related Work . In this section , we first discuss related model inversion attack studies , and then discuss applications of inverting a classifier outside of the privacy attack setting . Lastly , since our attack method relies on using a pretrained generator , we discuss existing studies that used a pretrained generator for improving other applications . Model inversion attacks . MI attacks are one type of privacy attack where the attacker tries to reconstruct the training examples given access only to a target classifier . Other privacy attacks include membership attacks , attribute inference , and model extraction attacks ( see Rigaki and Garcia [ 2020 ] for a general discussion ) . Fredrikson et al . [ Fredrikson et al. , 2015 , 2014 ] were the first to study the MI attack and demonstrated successful attacks on low-capacity models ( e.g. , logistic regression , and a shallow MLP network ) when partial input information was available to the attacker . The setting where partial input information was not available was studied by Hidano et al . [ 2017 ] . They termed performing gradient ascent w.r.t . the target classifier in the input space a General Model Inversion attack . Though effective on tabular data , this approach failed on deep image classifiers . It is well-known that directly performing gradient ascent in the image space results in imperceptible and unnatural changes [ Szegedy et al. , 2013 ] . To handle high-dimensional observations such as images , MI attacks based on deep generators were proposed [ Chen et al. , 2020 ; Yang et al. , 2019b ; Zhang et al. , 2020 ] . By training a deep generator on an auxiliary dataset , these methods effectively reduce the search space to only the manifold of relevant images . For example , if the target classifier is a facial recognition system , then an auxiliary dataset of generic faces can be easily obtained . Zhang et al . [ 2020 ] pretrain a Generative Adversarial Network ( GAN ) [ Goodfellow et al. , 2014 ] on the auxiliary dataset and performed gradient-based attack in the latent space of the generator . Yang et al . [ 2019b ] trained a generator that inverts the prediction of the target classifier , using the architecture of an autoencoder . Other applications of classifier inversion . Inverting the predictions and activations of a classifier has been studied for applications such as model explanation , and model distillation . Motivated by using saliency maps to explain the decision of a classifier [ Simonyan et al. , 2013 ] , Mahendran and Vedaldi [ 2015 ] proposed to maximize the activation of a target neuron in a classifier for synthesizing images that best explain that neuron . Their method is based on gradient ascent in the pixel-space . The same idea was recently extended to enable knowledge distillation in a setting where the original dataset was too large to be transferred [ Yin et al. , 2020 ] . Methods above have the advantage of not requiring the original dataset , but the generated images still appear unnatural . One effective way to improve the realism of the generated images is to optimize in the latent code space of a deep generator [ Nguyen et al. , 2015 , 2017 ] . Pretrained generators . Pretrained generators from GANs have other applications . Image inpainting/restoration was one of the early successful applications of using a pretrained GAN as image prior . Yeh et al . [ 2017 ] optimized the latent code of GANs to minimize the reconstruction error of a partially occluded image for realistic inpainting . The same method formed the basis of AnoGAN , a popular technique for anomaly detection in medical images [ Schlegl et al. , 2017 ] . A combination of reconstruction error and discriminator loss was used as the anomaly score . A potential future direction is to apply our proposed method to these adjacent applications . 4 Model Inversion Attack as Variational Inference . In this section , we derive our Variational Model Inversion ( VMI ) learning objective . For each class label y , we wish to approximate the true target posterior with a variational distribution q ( x ) ∈ Qx , where Qx is the variational family . This can be achieved by the optimizing the following objectives q∗ ( x ) = argmin q∈Qx { DKL ( q ( x ) ||pTAR ( x|y ) ) } = argmin q∈Qx { Eq ( x ) [ − log pTAR ( y|x ) ] +DKL ( q ( x ) ||pTAR ( x ) ) + log pTAR ( y ) } = argmin q∈Qx { Eq ( x ) [ − log pTAR ( y|x ) ] +DKL ( q ( x ) ||pTAR ( x ) ) } . ( 1 ) where in Equation 1 we have used the fact that log pTAR ( y ) is independent of q ( x ) , and thus can be dropped in the optimization . Auxiliary Image Prior . In order to minimize Eq . ( 1 ) , we need to compute DKL ( q ( x ) ||pTAR ( x ) ) , which requires evaluating the true image prior pTAR ( x ) , that is not accessible . Similar to recent works on MI attacks [ Khosravy et al. , 2020 ; Yang et al. , 2019b ; Zhang et al. , 2020 ] , we assume that a public auxiliary dataset DAUX , which shares structural similarity with the target dataset , is available to learn an auxiliary image prior pAUX ( x ) . This auxiliary image prior pAUX ( x ) is learned using a generative adversarial network ( GAN ) pAUX ( x ) = EpAUX ( z ) [ pG ( x|z ) ] , where pAUX ( z ) = N ( 0 , I ) is the GAN prior , and pG ( x|z ) is the GAN generator that is parameterized by a deep neural network G ( z ) , and a Gaussian observation noise with small standard deviation σ : pG ( x|z ) = N ( G ( z ) , σ2I ) , or equivalently x = G ( z ) + σϵ , ϵ ∼ N ( 0 , I ) . The learned generator pG ( x|z ) defines a low dimensional manifold in the observation space x , when z is chosen to have fewer dimensions than x . We now make the following assumption about the structural similarity of pAUX ( x ) and pTAR ( x ) . Assumption 1 ( Common Generator ) . We assume pTAR ( x ) is concentrated close to the low dimensional manifold of pG ( x|z ) , learned from the auxiliary dataset DAUX . In other words , we assume there exists a target prior pTAR ( z ) , possibly different from pAUX ( z ) , such that pTAR ( x ) ≈ EpTAR ( z ) [ pG ( x|z ) ] . We refer to pG ( x|z ) as the “ common generator ” . The name “ common generator ” is inspired by recent progress in theoretical analyses of few-shot and transfer learning , where assumptions about an underlying shared representation have been used to provide error bounds . In these works , this shared representation is referred to as the “ common representation ” [ Du et al. , 2020 ; Tripuraneni et al. , 2020 ] . Assumption 1 usually holds in the context of model inversion attacks . For example , pAUX ( x ) could be the distribution of all human faces , while pTAR ( x ) could be the distribution of the celebrity faces ; or pAUX ( x ) could be the distribution of natural images , while pTAR ( x ) could be the distribution of different breeds of dogs . In these examples , both pAUX ( x ) and pTAR ( x ) approximately live close to the same low dimensional manifold , but the distribution on this shared manifold could be different , i.e. , pTAR ( z ) ̸= pAUX ( z ) . Variational Family . We further consider our variational family Qx to be all distributions that lie on this manifold by assuming that q ( x ) is of the form q ( x ) = Eq ( z ) [ pG ( x|z ) ] , q ( z ) ∈ Qz , where Qz is a variational family in the code space . In the next section , we further restrict Qz to be either Gaussian distributions or normalizing flows . Proposition 1 . We have DKL ( q ( z ) ||pTAR ( z ) ) ≥ DKL ( Eq ( z ) [ pG ( x|z ) ] ||EpTAR ( z ) [ pG ( x|z ) ] ) . See Appendix A for the proof . Using Assumption 1 , we can approximate the objective of Equation 1 with q∗ ( z ) = argmin q∈Qz { Ez∼q ( z ) , ϵ∼N ( 0 , I ) [ − log pTAR ( y|G ( z ) + σϵ ) ] +DKL ( Eq ( z ) [ pG ( x|z ) ] ||EpTAR ( z ) [ pG ( x|z ) ] ) } , ( 2 ) where this objective is now defined in terms of q ( z ) ∈ Qz in the latent space of the GAN . Now using the Proposition 1 , we can obtain the following upper bound on the objective of Equation 2 ( assuming σ = 0 ) q∗ ( z ) = argmin q∈Qz { Ez∼q ( z ) [ − log pTAR ( y|G ( z ) ) ] +DKL ( q ( z ) ||pTAR ( z ) ) } . ( 3 ) Power Posteriors . We now consider the optimization of Equation 3 . Since we do not have access to pTAR ( z ) , we propose to replace it with pAUX ( z ) , noting that , as argued above , pAUX ( z ) may not accurately represent pTAR ( z ) . We also propose to replace the unknown likelihood pTAR ( y|G ( z ) ) with the pTAR ( y|G ( z ) ) defined by the target classifier , noting that this likelihood could be miscalibrated . Miscalibration is a known problem in neural network classifiers [ Guo et al. , 2017 ] ; namely , the classifier predictions can be over- or under-confident . In order to account for these issues , instead of the standard Bayes posterior qBAYES ( z ) ∝ pAUX ( z ) pTAR ( y|G ( z ) ) as the solution of Equation 3 , we consider the family of “ power posterior ” 1 [ Bissiri et al. , 2016 ; Holmes and Walker , 2017 ; Knoblauch et al. , 2019 ; Miller and Dunson , 2018 ] , which raise the likelihood function to a power 1γ q∗γ ( z ) ∝ pAUX ( z ) p 1 γ TAR ( y|G ( z ) ) , and can be characterized as the solution of the following optimization problem : q∗γ ( z ) = argmin q∈Qz LγVMI ( q ) , ( 4 ) LγVMI ( q ) : = Ez∼q ( z ) [ − log pTAR ( y|G ( z ) ) ] + γDKL ( q ( z ) ||pAUX ( z ) ) . See Appendix A for the proof . If the prior pAUX ( z ) is not reliable , by decreasing γ , we can reduce the importance of the prior . Similarly , by adjusting γ , we can reduce or increase the importance of overor under-confident likelihood predictions pTAR ( y|G ( z ) ) . Accuracy vs. Diversity Tradeoff . In the context of model inversion attack , the Bayes posterior qBAYES ( z ) can only achieve a particular tradeoff between the accuracy and diversity of the generated samples . However , the family of power posteriors enables us to achieve an arbitrary desired tradeoff between accuracy and diversity by tuning γ . If γ = 0 , then q∗γ=0 ( z ) = δ ( z − z∗ ) , and the model essentially ignores the image prior and diversity , and finds a single point estimate z∗ on the manifold which maximizes the accuracy pTAR ( y|G ( z ) ) . As we increase γ , the the model achieves larger diversity at the cost of smaller average accuracy . When γ = 1 , we recover the Bayes posterior q∗γ=1 ( z ) ∝ pAUX ( z ) pTAR ( y|G ( z ) ) . In the limit of γ → ∞ , we have q∗∞ ( z ) = pAUX ( z ) . In this case , the model ignores the classifier and accuracy , and sample unconditionally from the auxiliary distribution ( maximum diversity ) . We empirically study the effect of γ in Section 5 .

Summary: This paper frames the model inversion problem as a variational inference problem, where the goal is to approximate the posterior distribution p^{tar}(x|y) from the known target classifier p^{tar}(y|x). From a class of candidate distributions q(x), the aim is to find one that is close to p^{tar}(x|y), and this is achieved by minimizing the KL-divergence between the 2 distributions. Such a formulation allows the authors to make a principled derivation of an objective function that encourages both the optimized distribution to produce images likely to be classified correctly by the target classifier, and the produced images to be realistic. It is also a unified framework for existing methods of generative model inversion.  Additionally, the authors propose using the StyleGAN architecture as a pre-trained generator for the model inversion task. This allows intermediate layers (as opposed to just the input latent code in previous formulations) of the GAN to be optimized during the search for q*, improving the expressiveness of the GAN. The StyleGAN architecture also means that the attack can focus on layers whose representations control the semantics of the final image.