Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: The submission considers the problem of reconstructing private data from gradients in a Federated Learning system, which has been recently shown to a threat in distributed learning systems. Two types of federated learning systems are considered. Vertical federated learning (VFL) refers to the case where different agents hold different features of the same data points while  Horizontal federated learning (HFL) refers to the case where different agents how all the features of different subsets of the data.

Previous attacks solve an optimization problem that aims to infer the data by minimizing the mismatch between real gradients and fake gradients. This method suffers difficulty when the number of samples in one round is large. The paper proposes CAFE, which takes advantage of the fact that in vertical federated learning (VFL) systems, the server can identify the indices of the samples that are selected in each round. This extra information help reduce unwanted solutions in the optimization problem and help improve the reconstruction performance. The authors conduct experiments to show that the proposed algorithm outperforms previous works.

The paper shows that data leakage from gradients is a potential threat in VFL systems even when the batch size is large. However, I have the following concerns about the paper.

1. The paper claims that the attack also works for the HFL setting. However, this is no well justified for the following reasons:
(1) The assumption that the server knows the indices of the samples that are selected in each round is not valid in general for the HFL setting since each agent can sample a batch locally.
(2) In HFL settings, it is generally assumed that the number of agents is large and each agent only participates in a few rounds, which is not considered in the experiments in the submission.
2. In the experiments, it is shown that as the number of training epochs grows, better inference on the private data can be made. It would be better if the authors can also include the training error on each epoch in the same plot. It is believable that if the training goes on forever,  enough information can be inferred about the training samples. However, it might be good to see whether the server can infer the train samples before the model has already converged.

I hope the authors can address my above concerns in the response.
review_title: Review for paper #995

final_score: rating: 3: Clear rejection
review_content: This paper studies the data leakage issue in the federated learning. More precisely, when the servers have access to model parameters and gradients. It can recover the input data via gradient matching, and the authors claim that their method performs well even with large training batch sizes, e.g. over 40. Finally, the author also studies the possibility of attacking during learning, where they suggest that multiple updates of fake data helps. However, their contribution seems incremental, gradient matching is used in previous literature [zhu et al 2019], and their main modification is extra two regularization terms: total variation and internal representation regularization, and a data index alignment technique (whose exact meaning is unclear in the paper).

The following are some questions:

What does index alignment mean? Is that the server controls the indexes of samples chosen at each iteration? This seems to be very restrictive in practice, especially for horizontal federated learning.

Does the server have access to the aggregated grads from each worker separately or the workers aggregate all the gradients before sending them back to the server? The second scenario cab be achieved while secure aggregation technique.

In vertical federated learning, the gradients of part 1 of the network does not need to be exchanged with the server, as there is no average  operation needed, even the parameter itself does not need to be transferred to the server for the same reason, will your method work under this setting?

Some terms are not properly defined, such as normalized gradient descent, batch ratio, et. al.

Other questions:

What does the iterations represent in table 1a? Is that the number of iterations need to reach a 35 PSNR?

Using cosine dissimilarity decreases the PSNR, I assume this is because PSNR penalize the scale, is there noticeable degradation visually when using cosine dissimilarity?

In the attack during learning scenario, is there any intuition why optimizing fake data multiple times works better?

review_title: This paper studies the data leakage issue in federated learning, but lacks novelty in methodology and also details in some solutions.

final_score: rating: 4: Ok but not good enough - rejection
review_content: The paper proposes an attack to extract information about training data from gradient updates sent as part of a federated learning setting.

The description of the attack setting and the attack algorithm is provided at a high level and detailed description is missing, making it hard to understand the novelty of the contribution. Experimental results do show that the attack is stronger than previous work. However, the overall presentation of the paper could be improved to be ready for a publication. Some suggestions listed below.

[Attack setting] It seems that the paper departs from some of the related work on information leakage by considering an attacker that can tamper with the federated learning process. Hence, the attacker is malicious and not benign. The authors should make this distinction clear if it is indeed the case. Commenting on why this malicious activity will not be noticed by the workers is important. For example, Figure 5 indeed shows that training accuracy is impacted by the attack.
How does this attack compare to the active attacker in the work by Melis et al?

The paper considers only 4 workers in most experiments. Federated learning usually has many more participants. Is this a problem? Do same workers need to be contacted at every iteration? Are there any assumptions on same data being used in each iteration.

[Attack algorithm]
The algorithm makes use of "data index alignment". Some guidance on what it means would facilitate the reading and understanding of the algorithm. Red part of Algorithm 1 should be expanded. How do these values get computed? Do they replace the blue parts or complement them?

[Presentation]
“As a motivating example, Figure 1 compares”: it would be best to motivate the algorithm key insight and not its improved performance over previous work that was already mentioned.
“However, as shown in Figure 2, a curious server can provide the same legitimate computation as a benign server while simultaneously perform data recovery in a stealthy manner. The server symmetrically generates fake” The figure does not describe how fake parameters are computed and it is not clear how pictorial representation shows this.

Minor details
Figure 1 captions: why 40 vs 10 x 4; “workers participating FL” -> workers participating in FL; please consider a better title for “4.4 ATTACKING WHILE FL”                                         
review_title: Review 3

final_score: rating: 4: Ok but not good enough - rejection
review_content: This work introduces CAFE, a novel training algorithm to leak training data in a federated learning setup. Extending from "deep leakage from gradient" fake images are optimised with respect to the difference observed from the client gradients (i.e. with the real images) and the one observed with the current version of the fake image. However, DLG does not work when the mini-batch size increases due to a messy gradient representation. In this work, the authors propose to keep track of the batch index. Indeed, it may happen that the server decides of the batch index corresponding to the training data that will be used by the client during the local training. Within such conditions, a malicious server can easily store fake images corresponding to specific indices and therefore optimise correctly each fake images w.r.t the corresponding real image. 

It is clear from the obtained results that this method works, and that images are recovered. However, I am unsure about the relevance of the experimental protocol. 1. If the server does not ask for specific indices (and it is pretty common), the method is equivalent to DLG (i.e. does not work well) with large batches. 2. What if we don't have the gradients ? A common way of doing FL is to simply communicate the locally trained weights (with multiple local epochs). As specified in the introduction (point 3), the proposed method wouldn't work in this realistic scenario. 

Then, I found Section 3.3 unclear is some aspects. Are the two proposed regularisation methods relying on the "real" image ? If so, isn't this a strong bias ? (we are not expected to have the input images). I suppose that this comes from the citation to the work of Geipinget al., 2020. However, these two paragraph should be re-written to clearly explain how we can extract the input vector and how it relates to eq. 8. "To promote the smoothness of the fake images, we assume the TV norm of the real images as a constant," -> We can't use the real image here, so it is not valid. 

Pros: 
+ In the given conditions, CAFE clearly outperforms the other approach to leak training data from gradients during FL.
+ Very simple attack to implement.

Cons:
- The conditions necessary to the success of the proposed methods seem to be quite strong and not really connected to a realistic FL framework. 
- Small ideas can lead to drastic changes in the field, but the core idea of the paper is to solely store batch indices. 

Remarks:
- "In this section, we provide necessary background of FL [in this section]."
- Figure 2 should be checked. "Aggreaged", "upload fake gradient" only once .
- What are t and b in Eq. 5. 
review_title: Simple, but unclear.

Meta-review: This paper focuses attacks on federated learning. The reviewers had the following concerns:
- The assumption of knowledge of batch indices is unrealistic in an HFL setting
- The setup only works when doing a single epoch (I believe the authors claim that it is applicable in more general settings, but evidence to that effect has not been provided)
- The novelty of the approach is somewhat limited.
- The description of the algorithm and comparison to prior work could be clearer.

I raised the question of whether the reviewers would be more positive if there were no claimed results on HFL. They still did not seem positive enough to justify acceptance (due to the other reasons mentioned above).
Decision: Reject