Evaluating Gradient Inversion Attacks and Defenses in Federated Learning

1 Introduction . Federated learning [ McMahan et al. , 2016 , Kairouz et al. , 2021 ] is a framework that allows multiple clients in a distributed environment to collaboratively train a neural network model at a central server , without moving their data to the central server . At every training step , each client computes a model update —i.e. , gradient— on its local data using the latest copy of the global model , and then sends the gradient to the central server . The server aggregates these updates ( typically by averaging ) to construct a global model , and then sends the new model parameters to all clients . By allowing clients to participate in training without directly sharing their data , such protocols align better with data privacy regulations such as Health Insurance Portability and Accountability Act ( HIPPA ) [ Act , 1996 ] , California Consumer Privacy Act ( CCPA ) [ Legislature , 2018 ] , and General Data Protection Regulation [ Commission , 2018 ] . While sharing gradients was thought to leak little information about the client ’ s private data , recent papers [ Zhu et al. , 2019 , Zhao et al. , 2020 , Geiping et al. , 2020 , Yin et al. , 2021 ] developed a “ gradient inversion attack ” by which an attacker eavesdropping on a client ’ s communications with the server can begin to reconstruct the client ’ s private data . The attacker can also be a malicious participant in the Federated Learning scheme , including a honest-but-curious server who wishes to reconstruct private data of clients , or a honest-but-curious client who wishes to reconstruct private data of other clients . These attacks have been shown to work with batch sizes only up to 100 but even so they have 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . created doubts about the level of privacy ensured in Federated Learning . The current paper seeks to evaluate the risks and suggest ways to minimize them . Several defenses against gradient inversion attacks have been proposed . These include perturbing gradients [ Zhu et al. , 2019 , Wei et al. , 2020 ] and using transformation for training data that clients can apply on the fly [ Zhang et al. , 2018a , Huang et al. , 2020 ] . More traditional cryptographic ideas including secure aggregation [ Bonawitz et al. , 2016 ] or homomorphic encryption [ Phong et al. , 2018 ] for the gradients can also be used and presumably stop any eavesdropping attacks completely . They will not be studied here due to their special setups and overhead . We are not aware of a prior systematic evaluation of the level of risk arising from current attacks and the level of security provided by various defenses , as well as the trade-off ( if any ) between test accuracy , computation overhead , and privacy risks . The paper makes two main contributions . First , we draw attention to two strong assumptions that a current gradient inversion attack [ Geiping et al. , 2020 ] implicitly makes . We show that by nullifying these assumptions , the performance of the attack drops significantly and can only work for lowresolution images . The findings are explored in Section 3 and already imply some more secure configurations in Federated Learning ( Section 6 ) . Second , we summarize various defenses ( Section 4 ) and systematically evaluate ( Section 5 ) some of their performance of defending against a state-of-the-art gradient inversion attack , and present their data utility and privacy leakage trade-offs . We estimate the computation cost of end-to-end recovery of a single image under each evaluated defense . We also experimentally demonstrate the feasibility and effectiveness of combined defenses . Our findings are summarized as strategies to further improve Federated Learning ’ s security against gradient inversion attacks ( Section 6 ) . In Appendix B , we provide theoretical insights for mechanism of each evaluated defense . 2 Gradient Inversion Attacks . Previous studies have shown the feasibility of recovering input from gradient ( i.e . gradient inversion ) for image classification tasks , by formulating it as an optimization problem : given a neural network with parameters θ , and the gradient ∇θLθ ( x∗ , y∗ ) computed with a private data batch ( x∗ , y∗ ) ∈ Rb×d × Rb ( b , d being the batch size , image size ) , the attacker tries to recover x ∈ Rb×d , an approximation of x∗ : argmin x Lgrad ( x ; θ , ∇θLθ ( x∗ , y∗ ) ) + αRaux ( x ) ( 1 ) The optimization goal consists of two parts : Lgrad ( x ; θ , ∇θLθ ( x∗ , y∗ ) ) enforces matching of the gradient of recovered batch x with the provided gradients Lθ ( x∗ , y∗ ) , andRaux ( x ) regularizes the recovered image based on image prior ( s ) . [ Phong et al. , 2017 ] brings theoretical insights on this task by proving that such reconstruction is possible with a single-layer neural network . [ Zhu et al. , 2019 ] is the first to show that accurate pixellevel reconstruction is practical for a maximum batch size of 8 . Their formulation uses ` 2-distance as Lgrad ( · , · ) but no regularization term Raux ( x ) . The approach works for low-resolution CIFAR datasets [ Krizhevsky et al. , 2009 ] , with simple neural networks with sigmoid activations , but can not scale up to high-resolution images , or larger models with ReLU activations . A follow-up [ Zhao et al. , 2020 ] proposes a simple approach to extract the ground-truth labels from the gradient , which improves the attack but still can not overcome its limitations . With a careful choice of Lgrad andRaux ( x ) , [ Geiping et al. , 2020 ] substantially improves the attack and succeeds in recovering a single ImageNet [ Deng et al. , 2009 ] image from gradient : their approach uses cosine distance as Lgrad , and the total variation asRaux ( x ) . Their approach is able to reconstruct low-resolution images with a maximum batch size of 100 or a single high-resolution image . Based on [ Geiping et al. , 2020 ] , [ Wei et al. , 2020 ] analyzes how different configurations in the training may affect attack effectiveness . A more recent work [ Yin et al. , 2021 ] further improves the attack on high-resolution images , by introducing to Raux ( x ) a new image prior term based on batch normalization [ Ioffe and Szegedy , 2015 ] statistics , and a regularization term which enforces consistency across multiple attack trials . An orthogonal line of work [ Zhu and Blaschko , 2021 ] proposes to formulate the gradient inversion attack as a closed-form recursive procedure , instead of an optimization problem . However , their implementation can recover only low-resolution images under the setting where batch size = 1 . 3 Strong Assumptions Made by SOTA Attacks . 3.1 The state-of-the-art attacks . Two recent attacks [ Geiping et al. , 2020 , Yin et al. , 2021 ] achieve best recovery results . Our analysis focuses on the former as the implementation of the latter is not available at the time of writing this paper . We plan to include the analysis for the latter attack in the final version of this paper if its implementation becomes available . [ Geiping et al. , 2020 ] ’ s attack optimizes the following objective function : argmin x 1− 〈∇θLθ ( x , y ) , ∇θLθ ( x ∗ , y∗ ) 〉 ‖∇θLθ ( x , y ) |‖| ∇θLθ ( x∗ , y∗ ) ‖ + αTVRTV ( x ) ( 2 ) where 〈· , ·〉 is the inner-product between vectors , andRTV ( · ) is the total variation of images . We notice that Geiping et al . has made two strong assumptions ( Section 3.2 ) . Changing setups to invalidate those assumptions will substantially weaken the attacks ( Section 3.3 ) . We also summarize whether other attacks have made similar assumptions in Table 1 . 3.2 Strong assumptions . We find that previous gradient inversion attacks have made different assumptions about whether the attacker knows Batch normalization statistics or private labels , as shown in Table 3.2 . Note that [ Geiping et al. , 2020 ] ’ s attack makes both strong assumptions . Assumption 1 : Knowing BatchNorm statistics . Batch normalization ( BatchNorm ) [ Ioffe and Szegedy , 2015 ] is a technique for training neural networks that normalizes the inputs to a layer for every mini-batch . It behaves differently during training and evaluation . Assume the model has L batch normalization layers . Given x∗ , a batch of input images , we use x∗l to denote the input features to the l-th BatchNorm layer , where l ∈ [ L ] . During training , the l-th BatchNorm layer normalizes x∗l based on the batch ’ s mean mean ( x ∗ l ) and variance var ( x ∗ l ) , and keeps a running estimate of mean and variance of all training data points , denoted by µl and σ2l . During inference , { µl } Ll=1 and { σ2l } Ll=1 are used to normalize test images . In the following descriptions , we leave out { · } Ll=1 for simplicity ( i.e . use µ , σ2 to denote { µl } Ll=1 , { σ2l } Ll=1 , and mean ( x∗ ) , var ( x∗ ) to denote { mean ( x∗l ) } Ll=1 , { var ( x∗l ) } Ll=1 ) . We notice that [ Geiping et al. , 2020 ] ’ s implementation1 assumes that BatchNorm statistics of the private batch , i.e. , mean ( x∗ ) , var ( x∗ ) , are jointly provided with the gradient . Knowing BatchNorm statistics would enable the attacker to apply the same batch normalization used by the private batch on his recovered batch , to achieve a better reconstruction . This implicitly increases the power of the attacker , as sharing private BatchNorm statistics are not necessary in Federated learning [ Andreux et al. , 2020 , Li et al. , 2021 ] . Note that this assumption may be realistic in some settings : 1 ) the neural network is shallow , thus does not require using BatchNorm layers , or 2 ) the neural network is deep , but adapts approaches that normalize batch inputs with a fixed mean and variance ( as alternative to BatchNorm ) , e.g . Fixup initialization [ Zhang et al. , 2019 ] . Assumption 2 : Knowing or able to infer private labels . Private labels are not intended to be shared in Federated learning , but knowing them would improve the attack . [ Zhao et al. , 2020 ] finds that label information of a single private image can be inferred from the gradient ( see Section 3.3 for details ) . Based on this , [ Geiping et al. , 2020 ] assumes the attacker knows private labels ( see remark at the end of Section 4 in their paper ) . However , this assumption may not hold true when multiple images in a batch share the same label , as we will show in the next section . 1The official implementation of [ Geiping et al. , 2020 ] : https : //github.com/JonasGeiping/invertinggradients .

Summary: This paper has empirical study of federated learning with the following contributions: 1. It evaluates the state-of-the-art gradient attack algorithms under weaker assumptions. For example, attackers do not have access to batch norm statistics and some private labels. Under these weaker assumptions, these attack algorithms perform significantly worse. 2. It experiments various defense algorithms. For example, gradient pruning, MixUp, Weak-InstaHide, with the current state-of-the-art Carlini et al. 2020 attack algorithm. Further, it designs and evaluates a new method which combines various defense algorithms. Through empirical evaluations, it is shown that when attack batch size is 1, W-InstaHide is highly useful, and when attack batch size increases, the combined gradient pruning+W-InstaHide gives the best result.