Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 9: Top 15% of accepted NeurIPS papers, strong accept
review_content: summary: This paper has empirical study of federated learning with the following contributions:
1. It evaluates the state-of-the-art gradient attack algorithms under weaker assumptions. For example, attackers do not have access to batch norm statistics and some private labels. Under these weaker assumptions, these attack algorithms perform significantly worse.
2. It experiments various defense algorithms. For example, gradient pruning, MixUp, Weak-InstaHide, with the current state-of-the-art Carlini et al. 2020 attack algorithm. Further, it designs and evaluates a new method which combines various defense algorithms. Through empirical evaluations, it is shown that when attack batch size is 1, W-InstaHide is highly useful, and when attack batch size increases, the combined gradient pruning+W-InstaHide gives the best result.
main_review: Pros:
1. This paper demonstrates some practical perspective of the state-of-the-art gradient attack algorithms by removing certain strong assumptions and experimenting in more general conditions. Its empirical result is strong and clear that as soon as these strong assumptions are removed, the SOTA attack algorithms no longer perform that well. That sheds lights on how to improve the security of a federated learning system, i.e., do not share statistics like batch norm or certain private labels.
2. From a defense perspective, this paper considers a novel scheme which combines various different defense algorithms, such as gradient pruning, MixUp and InstaHide to achieve a better defense performance compared to any single defense algorithm, especially when attack batch size is large.
3. Previous works either focus on encoding private dataset directly or encoding the gradient information. This paper is a unified review of these two different methods, and combines them to improve the overall defense performance.

Cons:
The line 233 is a bit confusing: “InstaHide further encodes a MixUp image by randomly flipping its pixel signs. We are only mixing private images in our experiments, which is a weaker version of InstaHide”. Does this mean that W-InstaHide does not use random sign flips? If so, what’s the difference between MixUp and W-InstaHide? I believe there should be a difference between MixUp and W-InstaHide. However, based on the write-up, it’s a bit unclear about such differences and hence confusing. It should be further clarified.
limitations_and_societal_impact: I did not realize any potential negative societal impact of the work.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The authors of this paper provide a systematical analysis for the robustness of federated learning. They first show that current strongest gradient inversion attacks cannot perform well under relaxed assumptions and then demonstrate a new defense that can make the attacks less effective, even under their original assumptions. Sufficient experiments provide strong support for their conclusions. 
main_review: 
1. The authors reveal that the strong assumptions in the current strongest attack may not realistic and demonstrate even the current strongest attack cannot perform well under the relaxed assumption. The conclusions encourage that future research on gradient inversion should be evaluated in a more realistic setting.  
2. The authors provide some simple methods to improve the robustness of federated learning against gradient inversion attacks. They get strong results with a combination of several simple practices, which can improve the security against the strongest attack under the strong assumption.  
3. The paper is well organized and easy to understand. The findings and the proposed defenses and be used to evaluate the robustness of future federated learning systems. 
limitations_and_societal_impact: 1. Please fix the citation (geiping2020) in Line 89 of the manuscripts. 
2. I like the presentation of this paper: no fancy tricks but deliver clear and practical messages for future research. But the two parts (contribution 1 vs contribution 2, 3) are kind of separate from each other. I am suggesting the authors find a good way to combine them into a single story. 
3. The experiment and analysis are mainly applied to toy datasets with low-resolution images. I am wondering whether those conclusions are still valid on high-resolution images.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper presents an empirical investigation of existing gradient inversion attacks and defenses in federated learning setting. The authors point out that current attacks make strong assumptions such as knowing batch norm stats or private labels, whose absence would weaken the attacks. The authors also conclude with a list of best practices of doing conducting safe federated learning via comprehensive review and experiments on current defense methods.
main_review: This paper is well written and organized. Advancing state-of-the-art is important, but checking caveats in current methods is critical as well. Summarizing current assumptions made in current gradient inversion attacks makes it easier for future research to design fair and reasonable experiments. The practices lead by experiment results of various defenses are also valuable. However, although the conclusions are interesting, I wonder if the results are strong enough. Instead of doing experiments on a single dataset (CIFAR-10) with a single architecture (ResNet18), I would like to see more diverse settings, either easier (MNIST with MLP) or more complex (ImageNet with deeper ResNets). 
limitations_and_societal_impact: No, the limitations and potential negative societal impact are not clear in Section 6.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper evaluates existing attacks and defenses to gradient inversion attack, where eavesdropper of the protocol could steal private data from the clients. It re-evaluates attacks under relaxed assumptions, the effectiveness of the defenses, and proposes actionable ways of combining defenses under realistic assumptions to have more robust FL. The paper clarifies the state of the art in gradient inversion attacks and sheds light on best practices for good defense deployment
main_review: # Strengths
- Understanding and systematically evaluating strengths and weaknesses in the state of the art is crucial for arising fields
- Authors adjust threat model to support the idea that gradient inversion attacks are much less realistic in practice
- The paper remains always very practical, and contains many actionable points well-summarized in the conclusions.
- Code is planned of being released

# Weaknesses
- Focused only on image classification tasks 
- The experimental testbed is a bit unclear, which datasets are used in which parts
- Only 50 images on CIFAR-10 used for the re-evaluation of the defenses (line 241)

# Detailed comments

The paper studies a very important problem and takes time to re-evaluate the current state of attacks and defenses in gradient inversion attacks. Privacy is of utmost importance, and it is somehow hard to realize what are the pros/cons of existing methods. The authors conduct a very systematic and thorough evaluation, with very good and convincing arguments regarding the revision of existing attack assumptions, to help reason better about more realistic scenarios in-the-wild. Nevertheless, the paper also does a great job at finding interesting properties about more secure deployments, and greatly summarized actionable points for both practitioners and researcher. 

Nevertheless, I do have a couple of major concerns that I'd like the authors to discuss a bit better. 

## Experimental testbed

The datasets used in the experiments are not always fully clear. Of course authors are using CIFAR-10 (mostly) and a bit of ImageNet, but it is not always clear *how much* of it they are using for the evaluation. 

For example, are Figures 1 and 2 contain some images used for the actual attacks and proof-of-concept evaluation of effectiveness in presence of relaxed assumptions. Are these the only adversarial examples generated? This sounds a bit 'small' in terms of chosen images. 

On line 241, you mention:
> We use a subset of 50 CIFAR-10 images to evaluate the attack performance.

This seems quite 'limited', and possibly introducing some unintentional sampling bias in the evaluation that may threaten the empirical validity of the results.

I appreciate that the Appendix has additional settings, but I wonder how much of the study may be affected/threatened by the limited sampling of the dataset for attacks. 


## Task

The paper is focused only in the **image classification task**, so it would be interesting to understand whether the same properties/conclusions would hold also in other domains where deep neural networks are successful, for example speech recognition in smart home devices. 
limitations_and_societal_impact: I feel the authors should emphasize more limitations related to the number of adversarial examples generated.
review_title: null

Meta-review: The reviewers are satisfied by the responses made by the authors. The authors are strongly encouraged to include the additional experiments and results they provided in the rebuttal phase to their final manuscript.
Decision: Accept (Oral)