Gradient Inversion with Generative Image Prior
Authors: Jinwoo Jeon
Jaechang Kim
Kangwook Lee
Sewoong Oh
Jungseul Ok
Venue: NeurIPS 2021 Poster (2021)
Keywords: Federated Learning
Privacy Leakage
Gradient Inversion
Generative Model

Abstract: Federated Learning (FL) is a distributed learning framework, in which the local data never leaves clients’ devices to preserve privacy, and the server trains models on the data via accessing only the gradients of those local data. Without further privacy mechanisms such as differential privacy, this leaves the system vulnerable against an attacker who inverts those gradients to reveal clients’ sensitive data. However, a gradient is often insufficient to reconstruct the user data without any prior knowledge. By exploiting a generative model pretrained on the data distribution, we demonstrate that data privacy can be easily breached. Further, when such prior knowledge is unavailable, we investigate the possibility of learning the prior from a sequence of gradients seen in the process of FL training. We experimentally show that the prior in a form of generative model is learnable from iterative interactions in FL. Our findings demonstrate that additional mechanisms are necessary to prevent privacy leakage in FL.

Gradient Inversion with Generative Image Prior
Jinwoo Jeon',, Jaechang Kim?*, Kangwook Lee", Sewoong Oh', Jungseul Ok!.2
Department of Computer Science Engineering, Pohang University of Science and Technology Graduate School of Artificial Intelligence. Pohang University of Science and Technology
3 Department of Electrical and Computer Engineering, University of Wisconsin-Madison, Madison 4 Paul G. Allen School of Computer Science & Engineering, University of Washington
Abstract
Federated Learning (FL) a distributed learning framework, which the local data never leaves clients' devices preserve privacy, and the server trains models on the data via accessing only the gradients of those local data. Without further privacy mechanisms such as differential privacy, this leaves the system vulnerable against an attacker who inverts those gradients to reveal clients' sensitive data. However, gradient is often insufficient reconstruct the user data without any prior knowledge. By exploiting a generative model pretrained on the data distribution, we demonstrate that data privacy can be easily breached. Further, when such prior knowledge is unavailable, we investigate the possibility of learming the prior from a sequence of gradients seen the process of FL training. We experimentally show that the prior in a form of generative model is learnable from iterative interactions in FI.. Our findings strongly suggest that additional mechanisms are necessary to prevent privacy leakage in FL..
1 Introduction
Federated learning (FL) is an emerging framework for distributed learning, where central server aggregates model updates, rather than data, from end users [5] 17]. The main premise of federated learning is that this particular way of distributed learning can protect users' data privacy as there is explicit data shared by the end users with the central server.
29 demonstrates that one may recover the private user data used for training by observing the gradients. This process of recovering the training data from gradients, so-called gradient inversion, poses huge threat to the federated learning community, as may imply the fundamental flaw of its main premise.
However, recent line of work
Even more worryingly, recent works suggest that such gradient inversion attacks can be made even stronger if certain iideinformation is available. For instance, Geiping et al. [9] show that if the attacker knows prior that user data consists of natural images, then the gradient inversion attack can leverage such prior, achieving a more accurate recovery of the user data. Another instance is when batch norm statisties are available at the attacker addition to gradients. This can actually happen if the end users share their local batch norm statistics as in [17]. Yin et al. [29] show that such batch normalization statisties can significantly improve the strength the gradient inversion attack, enabling precise recovery of high-resolution images.
In this paper, we systematically study how one can maximally utilize and even obtain the prior information when inverting gradients. We first consider the case that the attacker has a generative model pretrained on the exact or approximate distribution of the user data as prior. For this, we propose an efficient gradient inversion algorithm that utilizes the generative model prior. In
equal contribution
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

ground truth
latent space
parameter space search (GIAS)
latent space search
existing method [9]
Figure 1: An example showing the superiority of GIAS compared to existing method. Images of the authors are reconstructed from gradients by exploiting generative model pretrained on human face images.
particular, the algorithm consists of two steps, in which the first step searches the latent space (of lower dimension) defined by the generative model instead of the ambient input space (of higher dimension), and then the second step adapts the generative model to each input given the gradient. Each step provides substantial improvement in the reconstruction. We name the algorithm as gradient inversion in alternative spaces (GIAS). Figure T represents reconstruction results with the proposed method and existing one.
We then consider a realistic scenario in which the user data distribution is not known in advance, and thus the attacker needs to learn from gradients. For this scenario, develop a meta-learning framework, called gradient inversion to meta-learn (GIML), which learns a generative model on user data from observing and inverting multiple gradients computed the data, e.g. across different FL epochs or participating nodes. Our experimental results demonstrate that one can learn generative model via GIML. and reconstruct data by making of the learned generative model.
This implies a great threat on privacy leakage in FL since our methods can be applied for any data type in most FL scenarios unless specialized architecture prevents the gradient leakage explicitly, e.g., [18].
Our main contributions are as follows:
We introduce GIAS that fully utilizes pretrained generative model to invert gradient. In addition, we propose GIMI. which can train generative model from gradients only in FL..
We demonstrate significant privacy leakage occurring by GIAS with pretrained generative model in various FL scenarios which are challenging to other existing methods, e.g., 9 29
We experimentally show that GIML can learn generative model on the user data from only gradients, which provides the same level data recovery with given pretrained model. To best knowledge, GIML is the first capable of learning explieit prior set of gradient inversion tasks.
We note that gradient inversion technique defines standard defence mechanism in Fl. for privacy (28). By substantiating that our proposed methods are able to break down defense mechanisms that were safe according the previous standard, give strong warning to the FL community to use a higher standard defined by our attack methods, and raise the necessity of more conservative choice of defense mechanisms.

---

2
Related work
Privacy attacks in FL. Early works [99 24 investigate membership inference from gradients to check the possibility of privacy leakage in FL. Phong ct al. [2I] demonstrate that it is possible to reconstruct detailed input image when FL trains shallow network such as single-layer perceptron. Fan et al. 7 and Zhu and Blaschko [2] consider a wider class of learning model and propose an analytical approach solving sequence of linear systems to reveal the output of each layer recursively. To study the limit of the gradient inversion in practical scenarios of training deep networks via FL, sequence of effort has been made formulating optimization problem to minimize discrepancy comparing gradients from true data and reconstructed data 9
Gradient inversion with prior. The optimization- based approaches are particularly useful one can easily utilize prior knowledge by adding regularization terms, e.g., total variation [27 g and BN statistics [29), or changing discrepancy measure [9] In [29]. privacy attack technique using a generative model introduced. They however require a pretrained model, while we propose a meta learning framework training generative model from gradients only. In addition, method of inverting gradient maximally exploit a given generative model by alternating search spaces, which analogous to the state-of-the- of-t-e--rtttttttt GAN inversion techniques
33]
Generative model revealing private data. Training a generative model with transmitted gradients also demonstrates privacy leakage in FL. Hitaj et al. [I]] introduce an algorithm to train GAN regarding shared model in FL framework as a discriminator. Wang al. use reconstructed data from gradient to train a GAN. Those works require some auxiliary dataset given in advance to enable the training of GAN, while we train generative model using transmitted gradients only. Also, we not only train generative model but also utilize it for reconstruction, while the generative models in [1 are not used for the reconstruction. Hence, in our approach, the generative model and reconstruction can be improved interactively to each other as shown in Figure 6 In addition, [27 is less samplc-efficiet than ours in a sense that they use gradients to reconstruct images and then train generative model with the reconstructed images, i.c., if the reconstruction fails, then the corresponding update of the generative model fails too, whereas we train the generative model directly from gradients.
3 Problem formulation
In this section, formally describe the gradient inversion (GI) problem. Consider standard supervised learning for classification, which optimizes neural network model fe parameterized by 0 as follows:
min Σ e(fe(z).v)
(1)
(EN))D
where é is point-wise loss function and D is dataset of input E R and label y {0, 1] (one-hot vector). In federated learning framework, each node reports the gradient ((fo(z) y) for sampled data (x, y)'s instead of directly transferring the data. The problem of inverting gradient is to reconstruct the sampled data used to compute the reported gradient. Specifically, when a node computes the gradient using batch {(x], ).., (ri vE)), i.e., = B Σ Ve(fo(z)). we consider the following problem of inverting gradient:
min ( (a ERTh (0.1)*
Σ B
V(((((a)).u)).. )
J=1
where d(-,)) -) is a measure of the discrepancy between two gradient, e.g., di-dtttance negative cosine similarity
34 29 or
g It known that label y can be almost accurately recovered by simple methods just observing the gradient at the last layer [31] 29), while reconstructing input æ remains still challenging as it is often mnder-determined even when the true label is given. For simplicity, we hence focus on the following minimization to reveal the inputs from the gradient given the true labels:
min
(x1, ..., TB; 0,g)
where we denote by e (Z1, ..., I B; 0, g) the cost function in given y/ V for each 1,..., B.
3

---

4 Methods
The key challenge of inverting gradient is that solving 2 is often under-determined, i.e., gradient contains only insufficient information to recover data. Such an issue is observed even when the dimension of gradient is much larger than that of input data. Indeed, Zhu and Blaschko [2] show that there exist a pair of different data having the same gradient, so called twin data, even when the learning model is large. To alleviate this issue, set of prior knowledge on the nature of data can be considered.
When inverting images, Geiping al. p propose to add the total variation regularization (Tv(F) to the cost function in (3) since neighboring pixels of natural images are likely to have similar values. More formally,
RTv (x)
Σ Σ (r(i.))=(((.0
(3,1) (E(J))
where ji, is the set of neighbors of (i, This method is limited to the natural image data.
For general type of data, one can consider exploiting the batch normalization (BN) statistics from nodes. This is available in the case that the server wants to utilize batch normalization (BN) in FL, and thus collects the BN statisties (mean and variance) of batch from each node, in addition. with every gradient report [17]. To be specific, Yin et al. (29) propose to employ the regularizer RBN (a1, ..., FB; 8) which quantifies the discrepancy between the BN statistics of estimated æ; 's and those of true 2z's 's on each layer of the learning model. More formally,
RBN(II, TB; 0); = Σ ||, + 1107 - o
where (I( 0) and of (21, 0) (resp. PLleenrr (..,..,.... T;; 0) and ("t are the mean and variance of 1-th layer feature maps for the estimated batch I1,-.,, FB (resp. the true
wR;
...,. Ty; 0))
batch =;, ...... 2²g given 0. This is available only if clients agree to report their exact BN statistics at every round. But not every FL. framework report BN statistics [15 2) In that case, Yin et al. (29 also propose to use the BN statistics over the entire data distribution
proxy of the true BN statisties, and reports that the gain from the approximated BN statistics is comparable to that from the exact ones. The applicability of RAN with the approximated BN statistics is still limited as the proxy needs to be additionally recomputed over the entire data distribution at every change of 0. However, this demonstrates the significant impact of knowing the data distribution in the gradient inversion and motivates our methods using and learning a generative model on the user data, described in what follows.
4.1 Gradient inversion with trained generative model
Consider a decent generative model G RR + m trained on the approximate (possibly exact)
distribution of user data D such that x² (2*) for (x",-) E DD and its latent code =' arg ming [==(=) - Il To fully utilize such a pretrained generative model, we propose gradient inversion in alternative spaces (GIAS), of which pseudocode is presented in Appendix [ which performs latent space search over and then parameter space search over w. We also illustrate the
overall procedure of GIAS in Figure 2
Latent space search. Note that the latent space is typically much smaller than the ambient input
space, i.e., k < m. for instances, DCGAN (25) of k = 100 and StyleGAN [12] of k 512 x 16 for image data of m = (width) x (height) x (color) such as 32 x 32 x 3, 256 x 256 x 3, or larger. Using such a pretrained generative model with k & m, the under-determined issues of 33) can be directly mitigated by narrowing down the searching space from I.m to (G (2) E R$ Hence, GIAS first performs the latent space search in the followings:
min $1 ER c (G), Gw
(5)
Considering canonical class of neural network model fo- we can show that the reconstruction of xx by latent space search (5) aligns with that by input space search in (3) the generative model G.,. approximates input data with small enough error.

---

Victim Participant
Ground Truth
Optimize
Optimize
Gw
Averaged Gradient
Shared Model
Gradiient
Gradient Matching
Averaged Gradient
(Loss Computation)
Dummy Gradient
Generative
Dummy
Image Prior
Input
Reconstruction
Result
Figure 2: An overview of GIAS. GIAS optimizes a latent code z and generative model parameters w to reconstruct the data which matches the gradient.
Property 1. For an input data x² E [0, 11 consider the gradient inversion problem of minimizing
cost in B where a canonical form of deep learning for classification considered and the discrepancy measure d is fz-dintance. Suppose that it has the unique global minimizer at x², Let 2 0 be the approximation error bound on x² for generative model G., [0, [0, + [0, 1] Then, there exists B(E) 2 O such that for any z" arg min, (G=.(z))
[[) - =²1l 5(s)
(6)
of which upper bound 8(z) as +
rigorous statement of Property [ and its proof are provided in Appendix where we prove and use that the cost function is continuous around x² under the assumptions. This property justifies solving the latent space search (5) for FL scenarios training neural network model while it requires an accurate generative model.
Parameter space search. Using the latent space search only, there can be inevitable reconstruction error due to the imperfection of generative model. This is mainly because we cannot perfectly prepare the generative model for every plausible data advance, Similar difficulty of the latent space search has been reported even when inverting GAN B3 directly, i.e., min, G (2) x=1| given x², rather than inverting gradient. Bau et al. [3] propose an instance-specific model adaptation, which slightly adjusts the model parameter w to (a part of source image) x" after obtaining a latent code z" for xx Inspired by such an instance-specific adaptation, GIAS performs the following parameter space search over preceded by the latent space search over 2:
for plausible but new data
min cGGu, (21), ..., (2B))
where 21, zB are obtained from 5
Remark. We propose the optimization over followed by that over sequentially This is to maximally utilize the benefit of mitigating the under-determined issue from reducing the searching space the pretrained model. However, the benefit would be degenerated if and optimized jointly or w optimized first. We provide an empirical justification on the proposed searching strategy in Section 5.1
We perform each search in GIAS using a standard gradient method the cost function directly. It is worth noting that those optimizations (5) and (7) with generative model can be tackled in recursive manner as R-GAP [2] reconstructs each layer from output to input. We provide details and performance of the recursive procedure in Appendix C where employing generative model improves the inversion accuracy R-GAP substantially, while R-GAP apparently suffers from an error accumulation issue when fa is deep neural network.
5

---

Lakmn Spazt Sorth
Goood
01M
Glale
Glxx (Geippig
Gl-x
Gl-eix
(GIAS, ous)
9.02
Cost
Gl+e
oGrppng Al
Bentions
(a)
(b)
Figure 3: Comparison of different searching spaces. (a) Each row shows reconstructed images of different optimization domains. The first three rows share the same latent space search of 1, 500 iterations, and GI-z/w is verified to be the best option to fully exploits the knowledge inside the generative model. (b) Cost function over iterations of different optimization domains.
4.2 Gradient inversion to meta-learn generative model
For the case that pretrained generative model is unavailable, we devise an algorithm to train generative model G,, for set {(0,. 9:)} of gradient inversion tasks. Since each inversion task can be considered as small learning task to adapt generative model per data, we hence call it gradient inversions meta-learn (GIML). The detailed procedure of GIML presented Appendix [A] We start with an arbitrary initialization of w. and iteratively update toward w' from variant of GIAS for tasks sub-sampled from S, which different than multiple applications of GIAS for each task in two folds: (i) f /e-regularization in latent space search; and (ii) an integrated optimization on model parameter. The variant first finds optimal latent codes x = (2 ...,,..., 2,B) for each task with respect the same cost function of GIAS but additional eg-regularization. Note that the latent space search with untrained generative model easily diverges. The & 2e-reulaarzzaton the divergence of 27. Once obtained x 's w/ is computed by few steps of gradient descents for an integrated parameter search to minimize Σ, c(G ( Gw' (2āp); 0g, 9:). This is because in GIML, we want meta information w to help GIAS for each task rather than solving individual tasks, while after performing GIML to train w, we perform GIAS to invert gradient with the trained w. This analogous the Reptile in 220
added to prevent
5 Experiments
Setup. Unless stated otherwise, consider the image classification task the validation set of ImageNet 22 dataset scaled down to 64 x 64 pixels (for computational tractability) and randomly initialized ResNet18 [10] for training. For deep generative models in GIAS, we use StyleGAN2 [3 trained on ImageNet. We use batch size B 4 as default and use the negative cosine measure the gradient dissimilarity d(.,-). We present detailed setup in Appendix H Our experiment code is available at httpp:: h/op:::///tthbb com com/ml-postech imagee--mmapppreee prrior
Algorithms. We evaluate several algorithms for the gradient inversion (GI) task in (3. They differ mainly in which spaces each algorithm searches over: the input x, the latent code z, and/or the model parameter w. Each algorithm is denoted by GI-(-), where the suffix indicates the search space(s). For instances, GI-z/w identical the proposed method, GIAS, and GI-z the one proposed by Geiping et al. [9].

---

(Geking
file
68 (Uariiad
(ile and
(oun)
Crurnd
Glaaw
Guo
Glaz
GLx
avergg
(Cisping
(a)
(b)
Figure 4: Comparison of state-of-the-ar models and ours. Replacing GI-r with GI-z/w (GIAS) regardless of using BN 22 or not [9] provides substantial improvement in the reconstruction accuracy. (a) Average PSNR and best PSNR in a batch throughout the experiments. (b) An ablation study and comparison of reconstruction results with our models and state-of-the-art models. We highlight the proposed models in bold.
Table 1: Comparison of our methods with state-of-the-art methods. Adding method makes performance improvement versus two baseline methods. PSNR, SSIM, and LPIPS 330 are used to evaluate reconstruction results. We highlight the best performances in bold.
Method
GI-x 9
GI-z (ours)
GL-w (ours)
GL-z/w (GIAS, ours)
GI-r+BN 29
Gl-z/w+BN (ours)
PSNR f SSIM
13.78
14.27
14.70
15.58
15.52
16.31
0,2542
0.3106
0.3519
0.3895
0.3513
0.4311
LPIPS
0.4376
0.3233
0.5121
0.3023
0.3645
0.2861
5.1
Justification of GIAS design
We first provide an empirical justification of the specific order of searching spaces in GIAS (corre- sponding GI-z/w) to fully utilize pretrained generative model. To do so, provide Figure 4b comparing algorithms with different searching spaces: GI-z/w, GI-z/x. GI-2, and GI-z, of which for the first 1, 500 iterations. As shown in
the first three share the same latent space search Figure Ba), the latent space search over z quickly finds plausible image in a much shorter number of iterations than GI-r, while it does not improve after certain point due the imperfection of pretrained generative model. Such limitation of GI-z is also captured in Figure 3(b) where the cost decreasing after a certain number of optimization steps. To further minimize the cost function, one alternative to GI-2/ (GIAS) is GI-2/z, zz, which can further reduce the loss function whereas the parameter search in GI-z/w w seems to provide more natural reconstruction of the image than 6l. The superiority of GI-z/u w over GI-z/z may come from that the parameter space search exploits an implicit bias from optimizing good architecture for expressing images, c.f., deep image prior [26). In Appendix [ and Figure [] we also present the same comparison on FFHQ (human-face face images) 12 where diversity much smaller than that of ImageNet. On such a less diverse dataset, the distribution can be easily leamned, and the gain from training generative model larger.
function of GI-z
5.2 The gain from fully exploiting pretrained generative model
Comparison with state-of-the-art models. Our method can be casily added to previous methods 29.. In Table and Figure we compare the state-of-the-art methods both with and without the proposed generative modelling. In Table I comparing GI-æ to GI-z/w /w and GI-r + BN to GI-z/w + BN, adding the proposed generative modelling provides additional gain in terms of all the measures (PSNR, SSIM. LPIPS) of reconstruction quality. GI-z/w without BN has lower reconstruction error than GI-z BN, which is the method of [29]. This implies that the gain from the generative model comparable to that from BN statistics. However, while the generative model only requires global (and hence coarse) knowledge on the entire dataset, BN statistics are local to the batch hand and hence requires significantly more detailed information on the exact batch used compute gradient.
7

---

(RIA, wns
Glahk (Raich Strend)
Glow (Ttech Sipee))
Gls LIN (ris etal)
Gles (Baich Supmi)
Gi (Jurch Sirent)
Glix (Gripin
Gl-srir (ztach 3apom32)
Stzee12)
16
86
C4
3.6
0.90
0.3
8000
6391
0.00
01
Dbach Siz
Gradien: Sganityy
Niaic Lovel
(b)
(c)
Figure 5: Comparison of state-of-the-ar models and GI-z/w z/w with varying difficulties. Larger batch size, higher sparsity, and larger gradient noise incraases reconstruction difficulty. GI-z/w w always surpasses GI-z thanks to the pretrained generative model. All subfigures share the y-axis.
As shown in Figure 4 the superiority of our method compared to the others is clear in terms of the best-in-batchh in-batch performance than the average one, where the former is more suitable to show actual privacy threat in the worst case than the latter. It is also interesting to note that GI-w with untrained w provides substantial gain compared to Gl-x. This may imply that there is a gain of the implicit bias, c.f., [6). from training the architecture of deep generative model.
Evaluation against possible defense methods We evaluate the gain of using generative model for various FL. scenarios with varying levels of difficulty in the inversion. As batch size, gradient sparsity (88 and gradient noise level increase, the risk of having under-determined inversion increases and the inversion task becomes more challenging, Figure shows shows that for all the levels of difficulty, the generative model provides significant gain in reconstruction quality. In particular, the averaged PSNR of GI-z with a batch size of 4 is comparable to that of GI-z/w wo with a batch size 32. It is also comparable to that of GI-z/u /w with a gradient sparsity of 99%. To measure the impact of the noisy gradient, we experimented gradient inversion with varying gaussian noise level in aforementioned settings. Figure (5(e) shows that adding enough noise the gradient can mitigate the privacy leakage. GI-z/u /w with a noise level of 0.01, which is relatively large, still surpasses GI-x without noise. large noise of 0.1 can diminish the gain of exploiting a pretrained generative model. However, the fact that adding large noise to the gradient slows down training makes it difficult for FL practitioners to choose suitable hyperparameters. The results imply our method is more robust to defense methods against gradient inversion, but can be blocked by high threshold. Note that our results of gradient sparsity and gradient noise implies the Differential Privacy(DP)) is still a valid defense method, when applied with a more conservative threshold. For more discussion about possible defense methods in FL framework, see Appendix
5.3 Learning generative model from gradients
We demonstrate the possibility of training a generative model only with gradients. For computational tractability, we use DCGAN and images from FFHQ [12] resized to 32x32. We generate set of gradients from 4 rounds of gradient reports from 200 nodes, in which node computes gradient for a classification task based the annotation provided in 6]. From the set of gradients, we perform GIML to train a DCGAN to potentially generate FFHQ data.
Figure shows the evolution of generative model improves the reconstruction quality when perform- ing either GI-z and Gl-z/ui w. We can clearly see the necessity of parameter space search. Figure a) shows that the quality of images from the generative model is evolving in the training process of GIML. As the step t of GIML increases, the generative model G Go (2) for arbitrary 2 outputs more plausible image of human face. When using generative model trained on wrong dataset (CIFAR10), GI-z completely fails at recovering data.
Having gradient sparsity 0,99% implies that we reconstruct data from 1 1 of the gradient after removing 99% elements with the smallest magnitudes at each layer.
8

---

GINE
CIFARIO
Ervend Tress
sampled
Goow ARFA))
Olee (GIMI, um))
(CIFARIO)
66-6
Olw (Unttainaal)
103
Stops
(a)
(b)
Figure 6: Qualitative and quantitative result of GIML. (a) Results validating generative model trained with GIML. Images on the first row are sampled from different GIML training steps. The same latent code 2 was used sample images same rows. Images on the second row and third row are results of GI-2 and GI-z/e /w using generative model trained with GIML and pretrained model which trained with CIFARIO images. Experiments were done with gradient sparsity 0.95 for comparison in difficult setting. Last column represents the ground truth image and result of GI-w with untrained model. (b) comparison of GIAS with meta-learned generative model and GIAS using improper generative model. Proper generative model boosts GIAS performance.
In Figure 6 b). GIML iteration step increases, the performance of GI-z and GI-z/w with GIML surpass GI-z and GI-z/u w with wrong prior knowledge. GI-z/u w using generative model trained on wrong dataset and GI-w which starts with untrained generative model show lower averaged PSNR compared to GI-z/u w with GIML. GI-z/u w with GIML to train generative model on right data shows the best performance in terms of not only quality (Figure 6 but also convergence speed. We provide a comparison of the convergence speed in Appendix
Conclusion
We propose GIAS fully exploit the prior information data from pretrained generative model when inverting gradient. We demonstrate significant privacy leakage using GIAS with pretrained generative model in various challenging scenarios, where our method provides substantial gain additionally to any other existing methods [9] 29). In addition, we propose GIML. which can train a generative model using only the gradients seen in the FL classifier training. We experimentally show that GIML, can meta-learn generative model on the user data from only gradients, which improves the quality of each individual recovered image. To our best knowledge, GIML is the first capable of learning explicit prior on a set of gradient inversion tasks.
Acknowledgments
This work partly supported by Institute of Information & communications Technology Planning Evaluation (ITTP) grant funded by the Korea government (MSIT) (No. 2019-0-01906, Artificial Intelligence Graduate School Program (POSTECH)) and (No. 2021 001--000739, Development of Distributed/Coooerratve AI based 5G+ Network Data Analytics Functions and Control Technology). Jinwoo Jeon and Jaechang Kim were supported by the Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by Korea(MSIT) (2020-0-01 academic joint research and education program). Kangwook Lee was supported by NSF/Intel Partnership on Machine Learning for Wireless Networking Program under Grant No. CNS-2003 and NSF Award DMS-2023233 Sewoong Oh acknowledges funding from NSF IIS-1929955, NSF
PSAI
CCF 2019844, and Google faculty research award.
References
[1] R. Abdal, Qin, and P. Wonka. Image2stylegan++: How to edit the embedded images? In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR),

---

June 2020.
[2] M. Andreux, O. du Terrail, C. Beguier, and E. W. Tramel. Siloed federated learning for multi-centric histopathology datasets. In S. Albarqouni. S. Bakas, K. Kamnitsas, M. Cardoso, B. Landman, W. Li, F. Milletari, N. Ricke, H. Roth, D. Xu, and Z. Xu, editors, Domain Adaptation and Reprexentation Transfer, and Distributed and Collaborative Learning, pages 129-139, Cham, 2020. Springer International Publishing. ISBN 978-3-030-60544--3.
[3] D. Bau, H. Strobelt, W. Peebles, J. Wulff, Zhou, J.-Y. Zhu, and Torralba. Semantic photo manipulation with generative image prior. ACM Trans. Graph., 38(4), July 2019. ISSN 0730-0301. doi: 10. 11453330634663333033. URL (ttpsss/:://do org/ 10. 1144//3063634 3323023
[4] D. Bau, J.-Y. Zhu, J. Wulff, W. Peebles, H. Strobelt, B. Zhou, and A. Torralba. Seeing what gan cannot generate. In Proceedings of the TEEE/CVF International Conference on Computer Vision (ICCV), October 2019.
[5] T.S. Brisimi, R. Chen, T. Mela, A. Olshevsky, I. C. Paschalidis, and W. Shi. Federated learning of predictive models from federated electronic health records. International Journal of Medical Informatics//JMII, 2018.
[6] DCGM. ffng-features-dattsee, 2019.
URL pettp:://iithu... http://ggttthub..coooc//
ffig--eattures-dttas
[7] L. Fan, K. W. Ng. C. Ju, T. Zhang, C. Liu, C. S. Chan, and Q. Yang. Rethinking privacy preserving deep learning: How to evaluate and thwart privacy attacks. In Q. Yang, L. Fan, and H. Yu, editors, Federated Learning: Privucy and Incentive, pages 32-50. Springer, 2020.
[8] C. Finn, P. Abbeel, and S. Levine. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. International Conference on Machine Learning (ICML), 2017.
[9] J. Geiping, H. Bauermeister, H. Dröge, and M. Moeller. Inverting gradients- Howw casy is to break privacy in federated learning? In Neural Information Processing Systems(NeurIPS), 2020.
[10] K. Hc, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
[11] B. Hitaj. G. Ateniese, and F. Perez-Cruz. Deep models under the GAN: Information leakage from collaborative deep learning. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS), 2017.
[12] Karras, S. Laine, and T. Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
[13] T. Karras, S. Laine, M. Aittala, J. Hellsten, J. Lehtinen, and T. Aila. Analyzing and improving the image quality of StyleGAN, In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition(
2020.
[14] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In Y. Bengio and LeCun, editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL http 7/axxiv. orgg org/abe////eeoooooooogooooeeeeae absaaa///aaa a/b////////11/////////////////// 11122 6980
[15] x. Li, M. Jiang, X. Zhang, M. Kamp, and Q. Dou. FedBN: Federated learning on non-llI) features via local batch normalization. In International Conference Learning Representations (ICLR), 2021.
[16] Y. Lin, S. Han, H. Mao, Y. Wang, and W. J. Dally. Deep Gradient Compression: Reducing the communication bandwidth for distributed training. The International Conference on Learning Representations (ICLR), 2018.
10

---

[17] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas. Communication--ffiienn learning of deep networks from decentralized data. A. Singh and J. Zhu, editors, Proceed- ings of the 20th International Conference on Artificial Intelligence and Statistics volume 54 of Proceedings of Machine Learning Research, pages 1273-1282, 2017.
[18] H. B. McMahan, D. Ramage, K. Talwar, and L. Zhang. Learning differentially private recurrent language models. In 6th International Conference on Learning Representations (ICLR), 2018.
[19] L. Melis, C. Song, E. De Cristofaro, and V. Shmatikov. Exploiting unintended feature leakage in collaborative learning. In IEEE Symp. Security and Privacy (SP), pages 691-706, 2019. doi: 10.11...P.....00000 9..0099.
[20] A. Nichol, J. Achiam, and J. Schulman. On first-order meta-learning algorithms. arXiv preprint arXiv:1803.02999, 2018.
[21] L. T. Phong. Y. Aono, I. Hayashi, L. Wang, and S. Moriai. Privacy-preserving deep learning via additively homomorphic encryption. IEEE Transactions on Information Forensics and Security, 13(5):1333-1345, 2018.
[22] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma. Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 115(3):211-252, 2015. doi:
10.1007/111233-05--0811
[23] A. Segal, A. Marcedone, B. Kreuter, D. Ramage, H. B. McMahan, Seth, K. A. Bonawitz, S. Patel, and V. Ivanov. Practical secure aggregation for privacy-preservve machine learning. In CCS, 2017. URL https://pppii iacr org/2017/281 pdff
[24] R. Shokri, M. Stronati, C. Song, and V. Shmatikov. Membership inference attacks against machine learning models. In 20/7 IEEE Symposium on Security and Privacy (SP), pages 3-18, 2017. doi: 10. 109/SP:2017.441.
[25] K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015.
[26] D. Ulyanov, A. Vedaldi, and V. Lempitsky. Deep image prior In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2018
[27] Z. Wang, M. Song, Z. Zhang, Y. Song, Q. Wang, and Qi. Beyond inferring class representa- tives: User-level privacy leakage from federated learning. In IEEE INFOCOM 2019 IEEE Conference on Computer Communicationss, pages 2512-2520, 2019.
[28] W. Wei, L. Liu, M. Loper, K.-H. Chow, M. E. Gursoy, S. Truex, and Y. Wu. A framework for evaluating client privacy leakages in federated learning. In L. Chen, N. Li, K. Liang, and S. Schneider, editors, Computer Security ESORICS 2020, pages 545 Cham, 2020. Springer International Publishing. ISBN
[29] H. Yin, A. Mallya, A. Vahdat, J. M. Alvarez, Kautz, and P. Molchanov. See through gradients: Image batch recovery via gradinversion. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPR), 2021.
[30] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
[31] B. Zhao, K. R. Mopuri, and H. Bilen. iDLG: Improved deep leakage from gradients. arXiv preprint arXiv:2001. 020600 2020.
[32] J. Zhu and M. B. Blaschko. R-GAP: Recursive gradient attack on privacy. In International Conference on Learning Representations (ICLR), 2021.
[33] Zhu, Y. Shen, D. Zhao, and B. Zhou. In-domain gan inversion for real image editing, In Proceedings of European Conference on Computer Vision (ECCV), 2020.
[34] I. 7hu, Z. Liu, and S. Han. Deep leakage from gradients. Annual Conference on Neural Information Processing Systems (NeurIPS), 2019.


Reviews:
[{"review_id": "H0kGzOMDsYS", "reviewer": "Reviewer_ofcZ", "summary": "The paper considers the interesting Federated Learning problem of recovering latent code data using gradient information, in the presence of additional prior knowledge and without. For an unknown dataset $\\{(x_i,y_i)\\}_{i=1}^B$, where $y_i$ is the label of $x_i$, the task is to find $x_i$ given gradient of $l(f_\\theta(x_i),y_i)$ with respect to $\\theta$. Here, $f_\\theta$, is a classifier with parameters $\\theta$ and $l$ is a appropriate loss function. \n\nIn the case where a generative prior on the dataset $\\{x_i\\}$ is known, the paper introduces the Gradient Inversion in Alternate Spaces (GIAS) algorithm that searches the latent code space of the generator to match the known gradients. In the case where a generative prior is not known, the paper introduces the Gradient Inversion to Meta-Learn generative model (GIML) algorithm, which, in two stages, learns weights of a generative model and searches the latent code case of the approximate generative model to match the gradients (up to a regularization term). The paper is primarily focused on the implementation of these algorithms. Experimental results were provided to show the efficiency of the proposed algorithms on real datasets.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper provides a sufficient description of prior work. However, some sentences used in the paper are not clear or have errors. For example: \n1. Lines 53-55\n2. Line 66: propose should be proposed\n3. Is BN in Line 73 Batch Norm? This needs to be made clear.\n4. Lines 75-77 \n5. Lines 94-96\n6. Lines 265-266\n\nTo my knowledge, the paper assumes that the classifier $f_\\theta$ is known (or at least the architecture on $f_\\theta$ is known). Is this a valid and reasonable assumption in practice?\n\nIs there anything we can say about the number of gradients we need to accurately recover/estimate the dataset. For the case when the generative model is not known, is it in the order of $k$ (the latent code dimension)? What about when the generative model is known? A discussion would be helpful.\n\nWhat is the motivation for line 11 in Algorithm 2. \n\nIn practice, how does one assume the architecture of the generative model if such a model is not known a prior?", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The paper discusses its limitation in the conclusion.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "FVEQRqRmIDz", "reviewer": "Reviewer_6jyM", "summary": "The submission \"Gradient Inversion with Generative Image Prior\" investigates privacy attacks against federated learning which invert gradient information sent by users to uncover private information. They authors propose generative models as tools to improve inversion, utilizing both optimization in latent space and parameter finetuning of such generative models. This approach improves the success rates of gradient inversion attacks against image classification (and especially best-case PSNR).", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall I think this work is a reasonable improvement of gradient inversion attacks against federated learning. The idea to utilize prior knowledge via generative models is good and executed well. I have a several main comments which I would like the authors to consider:\n\nComments:\n* I would like to see a larger discussion of [23] (Wang et al., Beyond Inferring Class Representatives) in the related work section, which is an older work in the gradient inversion literature, but also involves training a GAN to aid in the reconstruction (although not quite in the way the authors describe here, but it would be good to delineate the differences of both approaches explicitly)\nTo a lesser degree this comment also applies to works such as [Hitaj et al, Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning]\n* In [9] (Geiping et al, Inverting Gradients), small prior effects are noticed when considering GI-x reconstruction from the gradients of trained models. For the given submission I am not quite clear whether the authors consider a randomly initialized ResNet-18 model are a trained model. In any case, does the beneficial effect of the generative model increase or decrease when switching from random to trained parameters?\n* Feedback from practitioners of federated learning about gradient attacks has often focussed on the perceived security of larger image batches (via e.g secure aggregation) and gradient noise (as main component of differential privacy). It would be great if the authors could plot the success of the proposed reconstruction approach (possibly in the best-PSNR case) over a large range of batch sizes (up to 48 or even 128?) and gradient noise levels. This would be an extension of Fig.3. While there is of course the expectation that all attacks fall off significantly with larger batch sizes, the relative gains in falloff over previous attacks can be instructive for many readers (especially if the authors could show that their method has a noticeably slower falloff rate than others, which is difficult to make out from Fig.3 currently). Alternatively a measure of success along the lines of Fig.8 in [25] (Yin et al, See through gradients) could be investigated.\n* If I understand l236 correctly then this submission does not investigate a comparison to the full attack scheme of [25]? This is understandable, given the recency of that work, but it would be great if the authors could ultimately provide a clearer comparison between both attack schemes to confirm that the improvements of the generative model stack even with the full attack of [25]\n\n\nMinor comments that do not influence my assessment:\n* I think the authors could title this paper \"Gradient Inversion with Generative Image Priors\" (note the added _s_). The usage of \"generative image prior\" in the singular feels a bit off to me grammatically and further brings up associations with works such as \"Deep image prior\" and its successors, even though a deep image prior is not \"the prior\" used in this work (as StyleGAN is used). This could be avoided with a different title.\n* The paper could benefit from another round of proof-reading by the authors to iron out minor grammatical inconsistencies.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors propose an improved attack against privacy in federated learning. They appropriately discuss the societal impact of this measure in a separate subsection.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2.5"}, "extra_text": {}}, {"review_id": "EcVi90VepzF", "reviewer": "Reviewer_A6z3", "summary": "The contributions of this paper are mainly two-fold:\n1) Gradient inversion in alternative spaces (GIAS) that improves reconstruction with the help of the prior from generative models;\n2) Gradient inversion to meta-learn (GIML) which learns a generative model via inverting multiple gradients computed on the data.\nVia experiment results, this paper shows that GIAS with pretrained generative models can result in privacy leakage, and GIML can learn generative models from only gradients.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper proposes GIAS and GIML with improvements from baseline methods, while it might be better to provide more support on their effectiveness.\n\nThis paper compares the proposed GIAS with several other methods thoroughly via experiments. While I wonder if there are more theoretical justifications behind the effectiveness of the proposed searching mechanism in the latent and model parameter spaces.\n\nOn the other hand, how is the generalizability of GIML for more complex tasks? As 32x32 qualitative examples shown in Figure 4, there are substantial gaps between the reconstructions and the ground truths.\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "As the authors acknowledged in the section of \"societal impact and limitation\", this paper mainly focuses on how using generative model prior can facilitate gradient inversion to reveal data. It would nice (and also necessary to some extent) to discuss possible defensive strategies especially against the methods proposed in this paper.", "review_text": "", "extra_scores": {"needs_ethics_review": "Yes", "ethics_review_area": ["Inappropriate Potential Applications & Impact  (e.g., human rights concerns)", "Privacy and Security (e.g., consent)"], "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "DnUPoNSL-rs", "reviewer": "Reviewer_oRD2", "summary": "This paper proposed training a generative model for gradient inversion. Specifically, the authors proposed learning generative models by leveraging a generative image prior and learn a generative model for each data within one gradient inversion task. Empirical results show that the proposed GIAS and GIML outperforms baselines in applications of federated learning.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper proposed a novel method to optimize objective (3) using a generative model by searching latent space and parameter space. In Algorithm 1, the author proposed first searching the latent space then searching the parameter space. However, it is not clear to me why it is not done in an alternate fashion. Firstly, the authors seem to decompose (3) into (6) and (8). With algorithm 1, only (8) is satisfied. There's the z_i's solved by (6) no longer minimize (3) given the solutions in (8). Clarification of why algorithm 1 is the desired method should be stated here.\n\nThe proposed GIML algorithm learns a generative model. In the experiment, a DCGAN is learnt by the proposed algorithm. However, learning a GAN requires real image data. What is used as the real image data here? In the scenario of federated learning, who is doing the gradient inversion? The server or other clients. If the server is inverting the gradient, does that mean the server need to store a public dataset. Moreover, if the public dataset distribution is highly heterogeneous with the local data distribution, what would the performance be. In the experiment section, the heterogeneity of client data distribution is not discussed. I think it would significantly improve the experiment results if the relation between performance and heterogeneity level is presented.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitations and suggestions are listed in the main review.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers all agreed that this is an interesting paper and a worthy improvement over existing gradient inversion attacks against federated learning. I agree with the reviewers: the paper is clearly written and the many experimental results are worthy of publication. The paper will, at the very least, serve as an important motivation for further research on privacy in federated learning. I do caution the authors to take the suggestions made by the reviewers and the changes proposed in their rebuttal seriously and improve the manuscript in terms of related work and additional experiments.

It is also essential for the authors to implement what they promised in their rebuttal to the ethics reviewers in the final version of the paper. Critically, an experiment to demonstrate a potential fundamental defense mechanism and replacing human-face images in the paper's quantitative evaluation should be considered as "mandatory" revisions. I agree with the ethics reviewers that the authors' response helps assuage the ethical concerns (assuming that changes will be made to the final version).
Decision: Accept (Poster)