Evaluating Gradient Inversion Attacks and Defenses in Federated Learning

1 Introduction . Federated learning [ McMahan et al. , 2016 , Kairouz et al. , 2021 ] is a framework that allows multiple clients in a distributed environment to collaboratively train a neural network model at a central server , without moving their data to the central server . At every training step , each client computes a model update —i.e. , gradient— on its local data using the latest copy of the global model , and then sends the gradient to the central server . The server aggregates these updates ( typically by averaging ) to construct a global model , and then sends the new model parameters to all clients . By allowing clients to participate in training without directly sharing their data , such protocols align better with data privacy regulations such as Health Insurance Portability and Accountability Act ( HIPPA ) [ Act , 1996 ] , California Consumer Privacy Act ( CCPA ) [ Legislature , 2018 ] , and General Data Protection Regulation [ Commission , 2018 ] . While sharing gradients was thought to leak little information about the client ’ s private data , recent papers [ Zhu et al. , 2019 , Zhao et al. , 2020 , Geiping et al. , 2020 , Yin et al. , 2021 ] developed a “ gradient inversion attack ” by which an attacker eavesdropping on a client ’ s communications with the server can begin to reconstruct the client ’ s private data . The attacker can also be a malicious participant in the Federated Learning scheme , including a honest-but-curious server who wishes to reconstruct private data of clients , or a honest-but-curious client who wishes to reconstruct private data of other clients . These attacks have been shown to work with batch sizes only up to 100 but even so they have 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . created doubts about the level of privacy ensured in Federated Learning . The current paper seeks to evaluate the risks and suggest ways to minimize them . Several defenses against gradient inversion attacks have been proposed . These include perturbing gradients [ Zhu et al. , 2019 , Wei et al. , 2020 ] and using transformation for training data that clients can apply on the fly [ Zhang et al. , 2018a , Huang et al. , 2020 ] . More traditional cryptographic ideas including secure aggregation [ Bonawitz et al. , 2016 ] or homomorphic encryption [ Phong et al. , 2018 ] for the gradients can also be used and presumably stop any eavesdropping attacks completely . They will not be studied here due to their special setups and overhead . We are not aware of a prior systematic evaluation of the level of risk arising from current attacks and the level of security provided by various defenses , as well as the trade-off ( if any ) between test accuracy , computation overhead , and privacy risks . The paper makes two main contributions . First , we draw attention to two strong assumptions that a current gradient inversion attack [ Geiping et al. , 2020 ] implicitly makes . We show that by nullifying these assumptions , the performance of the attack drops significantly and can only work for lowresolution images . The findings are explored in Section 3 and already imply some more secure configurations in Federated Learning ( Section 6 ) . Second , we summarize various defenses ( Section 4 ) and systematically evaluate ( Section 5 ) some of their performance of defending against a state-of-the-art gradient inversion attack , and present their data utility and privacy leakage trade-offs . We estimate the computation cost of end-to-end recovery of a single image under each evaluated defense . We also experimentally demonstrate the feasibility and effectiveness of combined defenses . Our findings are summarized as strategies to further improve Federated Learning ’ s security against gradient inversion attacks ( Section 6 ) . In Appendix B , we provide theoretical insights for mechanism of each evaluated defense . 2 Gradient Inversion Attacks . Previous studies have shown the feasibility of recovering input from gradient ( i.e . gradient inversion ) for image classification tasks , by formulating it as an optimization problem : given a neural network with parameters θ , and the gradient ∇θLθ ( x∗ , y∗ ) computed with a private data batch ( x∗ , y∗ ) ∈ Rb×d × Rb ( b , d being the batch size , image size ) , the attacker tries to recover x ∈ Rb×d , an approximation of x∗ : argmin x Lgrad ( x ; θ , ∇θLθ ( x∗ , y∗ ) ) + αRaux ( x ) ( 1 ) The optimization goal consists of two parts : Lgrad ( x ; θ , ∇θLθ ( x∗ , y∗ ) ) enforces matching of the gradient of recovered batch x with the provided gradients Lθ ( x∗ , y∗ ) , andRaux ( x ) regularizes the recovered image based on image prior ( s ) . [ Phong et al. , 2017 ] brings theoretical insights on this task by proving that such reconstruction is possible with a single-layer neural network . [ Zhu et al. , 2019 ] is the first to show that accurate pixellevel reconstruction is practical for a maximum batch size of 8 . Their formulation uses ` 2-distance as Lgrad ( · , · ) but no regularization term Raux ( x ) . The approach works for low-resolution CIFAR datasets [ Krizhevsky et al. , 2009 ] , with simple neural networks with sigmoid activations , but can not scale up to high-resolution images , or larger models with ReLU activations . A follow-up [ Zhao et al. , 2020 ] proposes a simple approach to extract the ground-truth labels from the gradient , which improves the attack but still can not overcome its limitations . With a careful choice of Lgrad andRaux ( x ) , [ Geiping et al. , 2020 ] substantially improves the attack and succeeds in recovering a single ImageNet [ Deng et al. , 2009 ] image from gradient : their approach uses cosine distance as Lgrad , and the total variation asRaux ( x ) . Their approach is able to reconstruct low-resolution images with a maximum batch size of 100 or a single high-resolution image . Based on [ Geiping et al. , 2020 ] , [ Wei et al. , 2020 ] analyzes how different configurations in the training may affect attack effectiveness . A more recent work [ Yin et al. , 2021 ] further improves the attack on high-resolution images , by introducing to Raux ( x ) a new image prior term based on batch normalization [ Ioffe and Szegedy , 2015 ] statistics , and a regularization term which enforces consistency across multiple attack trials . An orthogonal line of work [ Zhu and Blaschko , 2021 ] proposes to formulate the gradient inversion attack as a closed-form recursive procedure , instead of an optimization problem . However , their implementation can recover only low-resolution images under the setting where batch size = 1 . 3 Strong Assumptions Made by SOTA Attacks . 3.1 The state-of-the-art attacks . Two recent attacks [ Geiping et al. , 2020 , Yin et al. , 2021 ] achieve best recovery results . Our analysis focuses on the former as the implementation of the latter is not available at the time of writing this paper . We plan to include the analysis for the latter attack in the final version of this paper if its implementation becomes available . [ Geiping et al. , 2020 ] ’ s attack optimizes the following objective function : argmin x 1− 〈∇θLθ ( x , y ) , ∇θLθ ( x ∗ , y∗ ) 〉 ‖∇θLθ ( x , y ) |‖| ∇θLθ ( x∗ , y∗ ) ‖ + αTVRTV ( x ) ( 2 ) where 〈· , ·〉 is the inner-product between vectors , andRTV ( · ) is the total variation of images . We notice that Geiping et al . has made two strong assumptions ( Section 3.2 ) . Changing setups to invalidate those assumptions will substantially weaken the attacks ( Section 3.3 ) . We also summarize whether other attacks have made similar assumptions in Table 1 . 3.2 Strong assumptions . We find that previous gradient inversion attacks have made different assumptions about whether the attacker knows Batch normalization statistics or private labels , as shown in Table 3.2 . Note that [ Geiping et al. , 2020 ] ’ s attack makes both strong assumptions . Assumption 1 : Knowing BatchNorm statistics . Batch normalization ( BatchNorm ) [ Ioffe and Szegedy , 2015 ] is a technique for training neural networks that normalizes the inputs to a layer for every mini-batch . It behaves differently during training and evaluation . Assume the model has L batch normalization layers . Given x∗ , a batch of input images , we use x∗l to denote the input features to the l-th BatchNorm layer , where l ∈ [ L ] . During training , the l-th BatchNorm layer normalizes x∗l based on the batch ’ s mean mean ( x ∗ l ) and variance var ( x ∗ l ) , and keeps a running estimate of mean and variance of all training data points , denoted by µl and σ2l . During inference , { µl } Ll=1 and { σ2l } Ll=1 are used to normalize test images . In the following descriptions , we leave out { · } Ll=1 for simplicity ( i.e . use µ , σ2 to denote { µl } Ll=1 , { σ2l } Ll=1 , and mean ( x∗ ) , var ( x∗ ) to denote { mean ( x∗l ) } Ll=1 , { var ( x∗l ) } Ll=1 ) . We notice that [ Geiping et al. , 2020 ] ’ s implementation1 assumes that BatchNorm statistics of the private batch , i.e. , mean ( x∗ ) , var ( x∗ ) , are jointly provided with the gradient . Knowing BatchNorm statistics would enable the attacker to apply the same batch normalization used by the private batch on his recovered batch , to achieve a better reconstruction . This implicitly increases the power of the attacker , as sharing private BatchNorm statistics are not necessary in Federated learning [ Andreux et al. , 2020 , Li et al. , 2021 ] . Note that this assumption may be realistic in some settings : 1 ) the neural network is shallow , thus does not require using BatchNorm layers , or 2 ) the neural network is deep , but adapts approaches that normalize batch inputs with a fixed mean and variance ( as alternative to BatchNorm ) , e.g . Fixup initialization [ Zhang et al. , 2019 ] . Assumption 2 : Knowing or able to infer private labels . Private labels are not intended to be shared in Federated learning , but knowing them would improve the attack . [ Zhao et al. , 2020 ] finds that label information of a single private image can be inferred from the gradient ( see Section 3.3 for details ) . Based on this , [ Geiping et al. , 2020 ] assumes the attacker knows private labels ( see remark at the end of Section 4 in their paper ) . However , this assumption may not hold true when multiple images in a batch share the same label , as we will show in the next section . 1The official implementation of [ Geiping et al. , 2020 ] : https : //github.com/JonasGeiping/invertinggradients .

Summary: The authors of this paper provide a systematical analysis for the robustness of federated learning. They first show that current strongest gradient inversion attacks cannot perform well under relaxed assumptions and then demonstrate a new defense that can make the attacks less effective, even under their original assumptions. Sufficient experiments provide strong support for their conclusions.