Gradient Inversion with Generative Image Prior

1 Introduction . Federated learning ( FL ) is an emerging framework for distributed learning , where central server aggregates model updates , rather than user data , from end users [ 5 , 17 ] . The main premise of federated learning is that this particular way of distributed learning can protect users ’ data privacy as there is no explicit data shared by the end users with the central server . However , a recent line of work [ 34 , 31 , 9 , 29 ] demonstrates that one may recover the private user data used for training by observing the gradients . This process of recovering the training data from gradients , so-called gradient inversion , poses a huge threat to the federated learning community , as it may imply the fundamental flaw of its main premise . Even more worryingly , recent works suggest that such gradient inversion attacks can be made even stronger if certain side-information is available . For instance , Geiping et al . [ 9 ] show that if the attacker knows a prior that user data consists of natural images , then the gradient inversion attack can leverage such prior , achieving a more accurate recovery of the user data . Another instance is when batch norm statistics are available at the attacker in addition to gradients . This can actually happen if the end users share their local batch norm statistics as in [ 17 ] . Yin et al . [ 29 ] show that such batch normalization statistics can significantly improve the strength of the gradient inversion attack , enabling precise recovery of high-resolution images . In this paper , we systematically study how one can maximally utilize and even obtain the prior information when inverting gradients . We first consider the case that the attacker has a generative model pretrained on the exact or approximate distribution of the user data as a prior . For this , we propose an efficient gradient inversion algorithm that utilizes the generative model prior . In ∗ equal contribution 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . particular , the algorithm consists of two steps , in which the first step searches the latent space ( of lower dimension ) defined by the generative model instead of the ambient input space ( of higher dimension ) , and then the second step adapts the generative model to each input given the gradient . Each step provides substantial improvement in the reconstruction . We name the algorithm as gradient inversion in alternative spaces ( GIAS ) . Figure 1 represents reconstruction results with the proposed method and existing one . We then consider a realistic scenario in which the user data distribution is not known in advance , and thus the attacker needs to learn it from gradients . For this scenario , we develop a meta-learning framework , called gradient inversion to meta-learn ( GIML ) , which learns a generative model on user data from observing and inverting multiple gradients computed on the data , e.g . across different FL epochs or participating nodes . Our experimental results demonstrate that one can learn a generative model via GIML and reconstruct data by making use of the learned generative model . This implies a great threat on privacy leakage in FL since our methods can be applied for any data type in most FL scenarios unless a specialized architecture prevents the gradient leakage explicitly , e.g. , [ 18 ] . Our main contributions are as follows : • We introduce GIAS that fully utilizes a pretrained generative model to invert gradient . In addition , we propose GIML which can train generative model from gradients only in FL . • We demonstrate significant privacy leakage occurring by GIAS with a pretrained generative model in various FL scenarios which are challenging to other existing methods , e.g. , [ 9 , 29 ] . • We experimentally show that GIML can learn a generative model on the user data from only gradients , which provides the same level of data recovery with a given pretrained model . To our best knowledge , GIML is the first capable of learning explicit prior on a set of gradient inversion tasks . • We note that a gradient inversion technique defines a standard on defence mechanism in FL for privacy [ 28 ] . By substantiating that our proposed methods are able to break down defense mechanisms that were safe according to the previous standard , we give a strong warning to the FL community to use a higher standard defined by our attack methods , and raise the necessity of a more conservative choice of defense mechanisms . 2 Related work . Privacy attacks in FL . Early works [ 19 , 24 ] investigate membership inference from gradients to check the possibility of privacy leakage in FL . Phong et al . [ 21 ] demonstrate that it is possible to reconstruct detailed input image when FL trains a shallow network such as single-layer perceptron . Fan et al . [ 7 ] and Zhu and Blaschko [ 32 ] consider a wider class of learning model and propose an analytical approach solving a sequence of linear systems to reveal the output of each layer recursively . To study the limit of the gradient inversion in practical scenarios of training deep networks via FL , a sequence of effort has been made formulating optimization problem to minimize discrepancy comparing gradients from true data and reconstructed data [ 9 , 27 , 29 , 31 , 34 ] . Gradient inversion with prior . The optimization-based approaches are particularly useful as one can easily utilize prior knowledge by adding regularization terms , e.g. , total variation [ 27 , 9 ] and BN statistics [ 29 ] , or changing discrepancy measure [ 9 ] . In [ 29 ] , a privacy attack technique using a generative model is introduced . They however require a pretrained model , while we propose a meta learning framework training generative model from gradients only . In addition , our method of inverting gradient maximally exploit a given generative model by alternating search spaces , which are analogous to the state-of-the-art GAN inversion techniques [ 3 , 4 , 33 ] . Generative model revealing private data . Training a generative model with transmitted gradients also demonstrates privacy leakage in FL . Hitaj et al . [ 11 ] introduce an algorithm to train a GAN regarding shared model in FL framework as a discriminator . Wang et al . [ 27 ] use reconstructed data from gradient to train a GAN . Those works require some auxiliary dataset given in advance to enable the training of GAN , while we train a generative model using transmitted gradients only . Also , we not only train a generative model but also utilize it for reconstruction , while the generative models in [ 11 , 27 ] are not used for the reconstruction . Hence , in our approach , the generative model and reconstruction can be improved interactively to each other as shown in Figure 6 . In addition , [ 27 ] is less sample-efficient than ours in a sense that they use gradients to reconstruct images and then train a generative model with the reconstructed images , i.e. , if the reconstruction fails , then the corresponding update of the generative model fails too , whereas we train the generative model directly from gradients . 3 Problem formulation . In this section , we formally describe the gradient inversion ( GI ) problem . Consider a standard supervised learning for classification , which optimizes neural network model fθ parameterized by θ as follows : min θ ∑ ( x , y ) ∈D ` ( fθ ( x ) , y ) , ( 1 ) where ` is a point-wise loss function and D is a dataset of input x ∈ Rm and label y ∈ { 0 , 1 } L ( one-hot vector ) . In federated learning framework , each node reports the gradient of ` ( fθ ( x ) , y ) for sampled data ( x , y ) ’ s instead of directly transferring the data . The problem of inverting gradient is to reconstruct the sampled data used to compute the reported gradient . Specifically , when a node computes the gradient g using a batch { ( x∗1 , y∗1 ) , ... , ( x∗B , y∗B ) } , i.e. , g = 1B ∑B j=1∇ ` ( fθ ( x∗j ) , y∗j ) , we consider the following problem of inverting gradient : min ( x1 , y1 ) , ··· , ( xB , yB ) ∈Rm× { 0,1 } L d 1 B B∑ j=1 ∇ ` ( fθ ( xj ) , yj ) , g , ( 2 ) where d ( · , · ) is a measure of the discrepancy between two gradient , e.g. , ` 2-distance [ 34 , 29 ] or negative cosine similarity [ 9 ] . It is known that label y can be almost accurately recovered by simple methods just observing the gradient at the last layer [ 31 , 29 ] , while reconstructing input x remains still challenging as it is often under-determined even when the true label is given . For simplicity , we hence focus on the following minimization to reveal the inputs from the gradient given the true labels : min x1 , ... , xB∈Rm c ( x1 , ... , xB ; θ , g ) , ( 3 ) where we denote by c ( x1 , ... , xB ; θ , g ) the cost function in ( 2 ) given yj = y∗j for each j = 1 , ... , B . 4 Methods . The key challenge of inverting gradient is that solving ( 2 ) is often under-determined , i.e. , a gradient contains only insufficient information to recover data . Such an issue is observed even when the dimension of gradient is much larger than that of input data . Indeed , Zhu and Blaschko [ 32 ] show that there exist a pair of different data having the same gradient , so called twin data , even when the learning model is large . To alleviate this issue , a set of prior knowledge on the nature of data can be considered . When inverting images , Geiping et al . [ 9 ] propose to add the total variation regularization RTV ( x ) to the cost function in ( 3 ) since neighboring pixels of natural images are likely to have similar values . More formally , RTV ( x ) : = ∑ ( i , j ) ∑ ( i′ , j′ ) ∈∂ ( i , j ) ‖x ( i , j ) − x ( i′ , j′ ) ‖2 , ( 4 ) where ∂ ( i , j ) is the set of neighbors of ( i , j ) . This method is limited to the natural image data . For general type of data , one can consider exploiting the batch normalization ( BN ) statistics from nodes . This is available in the case that the server wants to utilize batch normalization ( BN ) in FL , and thus collects the BN statistics ( mean and variance ) of batch from each node , in addition , with every gradient report [ 17 ] . To be specific , Yin et al . [ 29 ] propose to employ the regularizer RBN ( x1 , ... , xB ; θ ) which quantifies the discrepancy between the BN statistics of estimated xj ’ s and those of true x∗j ’ s on each layer of the learning model . More formally , RBN ( x1 , ... , xB ; θ ) : = ∑ l ‖µl−µl , exact‖2 + ‖σ2l −σ2l , exact‖2 , where µl ( x1 , ... , xB ; θ ) and σ2l ( x1 , ... , xB ; θ ) ( resp . µl , exact ( x ∗ 1 , ... , x ∗ B ; θ ) and σ 2 l , exact ( x ∗ 1 , ... , x ∗ B ; θ ) ) are the mean and variance of l-th layer feature maps for the estimated batch x1 , ... , xB ( resp . the true batch x∗1 , ... , x ∗ B ) given θ . This is available only if clients agree to report their exact BN statistics at every round . But not every FL framework report BN statistics [ 15 , 2 ] . In that case , Yin et al . [ 29 ] also propose to use the BN statistics over the entire data distribution as a proxy of the true BN statistics , and reports that the gain from the approximated BN statistics is comparable to that from the exact ones . The applicability of RBN with the approximated BN statistics is still limited as the proxy needs to be additionally recomputed over the entire data distribution at every change of θ . However , this demonstrates the significant impact of knowing the data distribution in the gradient inversion and motivates our methods using and learning a generative model on the user data , described in what follows .

Summary: The paper considers the interesting Federated Learning problem of recovering latent code data using gradient information, in the presence of additional prior knowledge and without. For an unknown dataset $\{(x_i,y_i)\}_{i=1}^B$, where $y_i$ is the label of $x_i$, the task is to find $x_i$ given gradient of $l(f_\theta(x_i),y_i)$ with respect to $\theta$. Here, $f_\theta$, is a classifier with parameters $\theta$ and $l$ is a appropriate loss function.   In the case where a generative prior on the dataset $\{x_i\}$ is known, the paper introduces the Gradient Inversion in Alternate Spaces (GIAS) algorithm that searches the latent code space of the generator to match the known gradients. In the case where a generative prior is not known, the paper introduces the Gradient Inversion to Meta-Learn generative model (GIML) algorithm, which, in two stages, learns weights of a generative model and searches the latent code case of the approximate generative model to match the gradients (up to a regularization term). The paper is primarily focused on the implementation of these algorithms. Experimental results were provided to show the efficiency of the proposed algorithms on real datasets.