Gradient Inversion with Generative Image Prior

1 Introduction . Federated learning ( FL ) is an emerging framework for distributed learning , where central server aggregates model updates , rather than user data , from end users [ 5 , 17 ] . The main premise of federated learning is that this particular way of distributed learning can protect users ’ data privacy as there is no explicit data shared by the end users with the central server . However , a recent line of work [ 34 , 31 , 9 , 29 ] demonstrates that one may recover the private user data used for training by observing the gradients . This process of recovering the training data from gradients , so-called gradient inversion , poses a huge threat to the federated learning community , as it may imply the fundamental flaw of its main premise . Even more worryingly , recent works suggest that such gradient inversion attacks can be made even stronger if certain side-information is available . For instance , Geiping et al . [ 9 ] show that if the attacker knows a prior that user data consists of natural images , then the gradient inversion attack can leverage such prior , achieving a more accurate recovery of the user data . Another instance is when batch norm statistics are available at the attacker in addition to gradients . This can actually happen if the end users share their local batch norm statistics as in [ 17 ] . Yin et al . [ 29 ] show that such batch normalization statistics can significantly improve the strength of the gradient inversion attack , enabling precise recovery of high-resolution images . In this paper , we systematically study how one can maximally utilize and even obtain the prior information when inverting gradients . We first consider the case that the attacker has a generative model pretrained on the exact or approximate distribution of the user data as a prior . For this , we propose an efficient gradient inversion algorithm that utilizes the generative model prior . In ∗ equal contribution 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . particular , the algorithm consists of two steps , in which the first step searches the latent space ( of lower dimension ) defined by the generative model instead of the ambient input space ( of higher dimension ) , and then the second step adapts the generative model to each input given the gradient . Each step provides substantial improvement in the reconstruction . We name the algorithm as gradient inversion in alternative spaces ( GIAS ) . Figure 1 represents reconstruction results with the proposed method and existing one . We then consider a realistic scenario in which the user data distribution is not known in advance , and thus the attacker needs to learn it from gradients . For this scenario , we develop a meta-learning framework , called gradient inversion to meta-learn ( GIML ) , which learns a generative model on user data from observing and inverting multiple gradients computed on the data , e.g . across different FL epochs or participating nodes . Our experimental results demonstrate that one can learn a generative model via GIML and reconstruct data by making use of the learned generative model . This implies a great threat on privacy leakage in FL since our methods can be applied for any data type in most FL scenarios unless a specialized architecture prevents the gradient leakage explicitly , e.g. , [ 18 ] . Our main contributions are as follows : • We introduce GIAS that fully utilizes a pretrained generative model to invert gradient . In addition , we propose GIML which can train generative model from gradients only in FL . • We demonstrate significant privacy leakage occurring by GIAS with a pretrained generative model in various FL scenarios which are challenging to other existing methods , e.g. , [ 9 , 29 ] . • We experimentally show that GIML can learn a generative model on the user data from only gradients , which provides the same level of data recovery with a given pretrained model . To our best knowledge , GIML is the first capable of learning explicit prior on a set of gradient inversion tasks . • We note that a gradient inversion technique defines a standard on defence mechanism in FL for privacy [ 28 ] . By substantiating that our proposed methods are able to break down defense mechanisms that were safe according to the previous standard , we give a strong warning to the FL community to use a higher standard defined by our attack methods , and raise the necessity of a more conservative choice of defense mechanisms . 2 Related work . Privacy attacks in FL . Early works [ 19 , 24 ] investigate membership inference from gradients to check the possibility of privacy leakage in FL . Phong et al . [ 21 ] demonstrate that it is possible to reconstruct detailed input image when FL trains a shallow network such as single-layer perceptron . Fan et al . [ 7 ] and Zhu and Blaschko [ 32 ] consider a wider class of learning model and propose an analytical approach solving a sequence of linear systems to reveal the output of each layer recursively . To study the limit of the gradient inversion in practical scenarios of training deep networks via FL , a sequence of effort has been made formulating optimization problem to minimize discrepancy comparing gradients from true data and reconstructed data [ 9 , 27 , 29 , 31 , 34 ] . Gradient inversion with prior . The optimization-based approaches are particularly useful as one can easily utilize prior knowledge by adding regularization terms , e.g. , total variation [ 27 , 9 ] and BN statistics [ 29 ] , or changing discrepancy measure [ 9 ] . In [ 29 ] , a privacy attack technique using a generative model is introduced . They however require a pretrained model , while we propose a meta learning framework training generative model from gradients only . In addition , our method of inverting gradient maximally exploit a given generative model by alternating search spaces , which are analogous to the state-of-the-art GAN inversion techniques [ 3 , 4 , 33 ] . Generative model revealing private data . Training a generative model with transmitted gradients also demonstrates privacy leakage in FL . Hitaj et al . [ 11 ] introduce an algorithm to train a GAN regarding shared model in FL framework as a discriminator . Wang et al . [ 27 ] use reconstructed data from gradient to train a GAN . Those works require some auxiliary dataset given in advance to enable the training of GAN , while we train a generative model using transmitted gradients only . Also , we not only train a generative model but also utilize it for reconstruction , while the generative models in [ 11 , 27 ] are not used for the reconstruction . Hence , in our approach , the generative model and reconstruction can be improved interactively to each other as shown in Figure 6 . In addition , [ 27 ] is less sample-efficient than ours in a sense that they use gradients to reconstruct images and then train a generative model with the reconstructed images , i.e. , if the reconstruction fails , then the corresponding update of the generative model fails too , whereas we train the generative model directly from gradients . 3 Problem formulation . In this section , we formally describe the gradient inversion ( GI ) problem . Consider a standard supervised learning for classification , which optimizes neural network model fθ parameterized by θ as follows : min θ ∑ ( x , y ) ∈D ` ( fθ ( x ) , y ) , ( 1 ) where ` is a point-wise loss function and D is a dataset of input x ∈ Rm and label y ∈ { 0 , 1 } L ( one-hot vector ) . In federated learning framework , each node reports the gradient of ` ( fθ ( x ) , y ) for sampled data ( x , y ) ’ s instead of directly transferring the data . The problem of inverting gradient is to reconstruct the sampled data used to compute the reported gradient . Specifically , when a node computes the gradient g using a batch { ( x∗1 , y∗1 ) , ... , ( x∗B , y∗B ) } , i.e. , g = 1B ∑B j=1∇ ` ( fθ ( x∗j ) , y∗j ) , we consider the following problem of inverting gradient : min ( x1 , y1 ) , ··· , ( xB , yB ) ∈Rm× { 0,1 } L d 1 B B∑ j=1 ∇ ` ( fθ ( xj ) , yj ) , g , ( 2 ) where d ( · , · ) is a measure of the discrepancy between two gradient , e.g. , ` 2-distance [ 34 , 29 ] or negative cosine similarity [ 9 ] . It is known that label y can be almost accurately recovered by simple methods just observing the gradient at the last layer [ 31 , 29 ] , while reconstructing input x remains still challenging as it is often under-determined even when the true label is given . For simplicity , we hence focus on the following minimization to reveal the inputs from the gradient given the true labels : min x1 , ... , xB∈Rm c ( x1 , ... , xB ; θ , g ) , ( 3 ) where we denote by c ( x1 , ... , xB ; θ , g ) the cost function in ( 2 ) given yj = y∗j for each j = 1 , ... , B . 4 Methods . The key challenge of inverting gradient is that solving ( 2 ) is often under-determined , i.e. , a gradient contains only insufficient information to recover data . Such an issue is observed even when the dimension of gradient is much larger than that of input data . Indeed , Zhu and Blaschko [ 32 ] show that there exist a pair of different data having the same gradient , so called twin data , even when the learning model is large . To alleviate this issue , a set of prior knowledge on the nature of data can be considered . When inverting images , Geiping et al . [ 9 ] propose to add the total variation regularization RTV ( x ) to the cost function in ( 3 ) since neighboring pixels of natural images are likely to have similar values . More formally , RTV ( x ) : = ∑ ( i , j ) ∑ ( i′ , j′ ) ∈∂ ( i , j ) ‖x ( i , j ) − x ( i′ , j′ ) ‖2 , ( 4 ) where ∂ ( i , j ) is the set of neighbors of ( i , j ) . This method is limited to the natural image data . For general type of data , one can consider exploiting the batch normalization ( BN ) statistics from nodes . This is available in the case that the server wants to utilize batch normalization ( BN ) in FL , and thus collects the BN statistics ( mean and variance ) of batch from each node , in addition , with every gradient report [ 17 ] . To be specific , Yin et al . [ 29 ] propose to employ the regularizer RBN ( x1 , ... , xB ; θ ) which quantifies the discrepancy between the BN statistics of estimated xj ’ s and those of true x∗j ’ s on each layer of the learning model . More formally , RBN ( x1 , ... , xB ; θ ) : = ∑ l ‖µl−µl , exact‖2 + ‖σ2l −σ2l , exact‖2 , where µl ( x1 , ... , xB ; θ ) and σ2l ( x1 , ... , xB ; θ ) ( resp . µl , exact ( x ∗ 1 , ... , x ∗ B ; θ ) and σ 2 l , exact ( x ∗ 1 , ... , x ∗ B ; θ ) ) are the mean and variance of l-th layer feature maps for the estimated batch x1 , ... , xB ( resp . the true batch x∗1 , ... , x ∗ B ) given θ . This is available only if clients agree to report their exact BN statistics at every round . But not every FL framework report BN statistics [ 15 , 2 ] . In that case , Yin et al . [ 29 ] also propose to use the BN statistics over the entire data distribution as a proxy of the true BN statistics , and reports that the gain from the approximated BN statistics is comparable to that from the exact ones . The applicability of RBN with the approximated BN statistics is still limited as the proxy needs to be additionally recomputed over the entire data distribution at every change of θ . However , this demonstrates the significant impact of knowing the data distribution in the gradient inversion and motivates our methods using and learning a generative model on the user data , described in what follows .

Summary: The submission "Gradient Inversion with Generative Image Prior" investigates privacy attacks against federated learning which invert gradient information sent by users to uncover private information. They authors propose generative models as tools to improve inversion, utilizing both optimization in latent space and parameter finetuning of such generative models. This approach improves the success rates of gradient inversion attacks against image classification (and especially best-case PSNR).