Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: null
review_content: I would like to thank the authors for providing point-to-point response on my raised concerns. My concerns are mostly addressed by the author rebuttal. Thanks!
review_title: Thanks for the response on raised concerns

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: 
This paper proposes an information leakage attack (i.e., CAFE) that can recover data samples from aggregated gradients against a  FL setting. The proposed attack first recovers gradients of loss with respect to the outputs of the first fully connected (FC) layer. It then recovers the first FC layer's inputs. Finally, it recovers the original data samples by optimizing an objective that consists of the traditional gradient matching objective and two auxiliary regularizers:  a TV norm term and an internal representation regularization term. The paper provides both theoretical analysis and numerical verification about CAFE's performance. Experiments show that the proposed method outperforms some of the previous works. 
main_review: [Strengths]

1. The paper presents an information leakage attack against FL.
2. In addition to numerical results, the paper also presents a theoretical guarantee of the proposed method's performance.
3. The paper also provides a defense mechanism to mitigate the proposed attack.

[Weaknesses]

1. My main concern of the proposed method is that it might not generalize to a typical FL setting where the clients only send their own model updates to the server (without communicating with other workers). The authors might want to clarify how the proposed method can generalize to the traditional FL setting. Otherwise, the contribution of the paper would be quite limited. 
2. In the experiments, only three baselines are selected: DLG, cosine similarity, and SAPAG. More recent and more advanced methods  (e.g., Inverting gradients [Geiping et al., 2020], GradInversion [Yin et al., 2021]) are not included for comparison. 
3. The FL setting in the experiments only includes four workers. In practice, an FL system usually consists of hundreds of or even thousands of clients. The proposed method works in VFL, in which local agents need to exchange updates with other workers to compute the gradients.  
4. Some of the concepts are a bit vague. For instance, the paper mentions "data index alignment." However, it does not provide a clear definition or explanation. 
limitations_and_societal_impact: See the main review.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes a method for the server to recover the batch data from the gradients obtained in vertical federated learning. The recovery process is based on the use of data indices of the batch which are known.  The recovery contains three steps, and the most important nouvel part is to recover the first input of a fully connected layer for every data sample in the batch ($H$ in the paper) and then introduce it as a metric in the final recovery objective function.
main_review: CAFE recovers the whole data as it makes use of the indices of the batch that other previous strategies have no access to. It keeps recovering every image as time goes on i.e., making use of all the collected gradients. If the global model is unchanged (not in a training mode), the authors prove that CAFE recovers the optimal $H$.

Overall the strategy is quite original but with strong assumptions. It may be better to mention in the title that it is for vertical federated learning. As for horizontal federated learning (the one we think about first when we mention FL), it is not realistic that the server would know the batch indices.  Besides, as the authors pointed out, the attack will be weak while training (that’s to say that $\Theta$ changes according to the FL procedure), as it is harder to recover $H$ which depends on $\Theta$. Thus, CAFE works well only when the learning rate in training is small. This limitation is better to be mentioned in the early pages. Although the leakage seems to be catastrophic, there are limitations for the efficiency of this attack.    

Some other comments:
1. Lines 221 - 224: Can you explain why data leakage performance of CAFE is independent of the batch size? Do the experiments in Table 3 share the same T? If the same T is used, from the attack perspective, larger batch size may help the algorithm to approximate $H$ and thus favors the CAFE performance.  
2. I got confused about the names of the algorithm 3 and 4. Should algorithm 3 be the single-loop one and algorithm 4 be the nested one? There are contradictory conclusions on the performance of these algorithms in lines 167-168 and lines 240-241.
3. It may be interesting to show the convergence plot for experiment 4.3 with small learning rate. In addition, as mentioned in the paragraph, it would be great to study as well if CAFE works better on an untrained model.
4. In algorithm 4, line 5 should be “Run step 11-13” and line 6 should be “Run step 11-13”?


limitations_and_societal_impact: Yes, the authors have adequately addressed the limitations and potential negative societal impact of their work
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper studies recovering sensitive training data from vertical federated learning (VFL) process, where same set of training data is shared among the clients, but each has distinct set of features. The authors leveraged the data indices (or batch indices) to reliably recover internal representations, which in turn helps to recover the input training data with high quality. The experimental results validate the effectiveness of the proposed approach. 
main_review: $\textbf{Originality}$: I do not have enough knowledge about the related work, but utilizing the data indices to design stronger model inversion attack seems to be new. To the best of the reviewer's knowledge, related work is adequately cited. 

$\textbf{Quality}$: the proposed approach is technically sound and the claims are well supported by the experimental results. I think it is a clever idea to utilize known data indices (although somewhat limits its application scenario) to recover some intermediate representations, which I believe gives the final optimization process (of recovering the input training data) an even finer control and improves the recovery quality.   

$\textbf{Clarity}$: the paper is very well written and is easy to follow. There are some typos in the paper: in page 3, for the definition of $\mathcal{D}(s^t)$, it should be $s^t[i]$ instead of $s^t[n]$? "betch size" in page 7. 

Regarding the baselines in page 8, the justifications for not including some methods in Table 1 is not satisfactory. For example, the reason "... require certain conditions which may fail to satisfy" is too ambiguous. I am fine with not including certain baselines with justifiable reasons, but they need to be very precise. When experimenting with the fake gradient based defense, it will be better to also report the training and testing accuracy for more intuitive understanding. In terms of writing, I think it will improve the paper if the authors can describe the high level motivation of designing multi-stage attack at the beginning of the paper. My current understanding is, multi-stage attack is preferred because it gives adversaries finer control over the optimization process of data recovery, and a side benefit is, some sensitive intermediate representations are also revealed. If my understanding is correct, then one follow-up question is, why not make the analysis even finer by recovering the inputs to the convolutional layers?  

$\textbf{Significance}$: the method is not general enough and only applies to the special VFL setting where the data indices are known. In other FL settings, data indices are generally unavailable and the proposed method will not work. But I believe VFL is still an important setting and stronger attacks tailored for this specific setting are also valuable. 

In the experiment section, comparison to the baselines might be an apple to orange comparison. The reviewer is only familiar with the two baselines [10], [26], but these two methods are actually very general and do not need the knowledge of data indices. Therefore, it is expected that in VFL setting with large batch sizes, this type of baselines will perform worse. I believe for the general FL setting where data indices are unavailable, larger batch size is still an effective defense. 

When designing a defense in VFL setting, one straightforward option is to apply differential privacy during model training. I think the authors should discuss the difference of their method to differential privacy and it will be very impressive if the proposed method outperforms DP experimentally by providing better model utility while protecting the data privacy. 
limitations_and_societal_impact: The authors addressed the limitations and potential negative impact.
review_title: null

Meta-review: The reviewers were not strongly positive about the paper. The reviewers had a recurring concern over the distinction between Vertical Federated Learning (VFL) and Horizontal Federated Learning, probably the more common one. I would recommend the authors to update the title and make a clear distinction in the introduction itself, to keep the expectation of the reader focussed on VFL. Also, reviewer kAc7 had some concerns about the vagueness in the definition of concepts in the paper, which needs to be addressed.
Decision: Accept (Poster)