Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper delivers an empirical assessment of adversarial transferability between FL trained model and centralized models, between limited data trained model and full data trained model. Based on the empirical evidence, it conjectures that FL models are more robust and their attack possesses high transferability, and believe it is caused by two factors: the decentralized training on distributed data and the averaging operation.
strength_and_weaknesses: Strength: The paper discusses an interesting and important matter. It conducts detailed simulation studies, aiming to reveal how transferability is affected by FL learning.

Weakness: It seems that the results from CNN and ResNe50 are not consistent, hurting the credibility of the author's arguments. Such inconsistency somehow suggests that architecture plays a role.

Definition of transfer accuracy (T.Acc): what is ||x||? Although all simulation presents the results about the T.Acc, there is no discussion on the trend of T.Acc. Then what is the purpose of introducing this notation?

Please double check the legend in figure 2. The paper claims that "T.Rate against the federated model increases till 20 users and decreases.", but the figure shows that T.Rate against the centralized model increases till 20 users and decreases.

As an empirical work, I suppose the current scale of the experiment is not big enough to convince readers.

Figures in Section 5, i.e. Fig 3 & 4, display a complicated trend rough than the claimed monotone changing. It seems some in-depth messages are missing.
clarity,_quality,_novelty_and_reproducibility: The experiment details are clear enough for reproducibility.
The writing is in general good.
summary_of_the_review: I appreciate this attempt to understand adversarial transferability in FL setting. However, I sense the current empirical evidence is not convincing enough.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper investigates black-box adversarial attacks in FL. The paper assumes the attackers control some clients, thus have access to some training data. The attackers can launch transfer-based black-box attack with the training data. The paper has implications for understanding the robustness of federated learning systems and poses a practical question for federated learning applications.
strength_and_weaknesses: Strength: 
- The research topic is important. Defending adversarial attacks in FL is useful in reality.

Weakness：
- The setting is unreasonable. The paper assumes the attacker controls some clients that can take part in the FL training and considers black-box attack where the attacker cannot access the model (i.e., does not know the model parameters). Since the malicious clients can take part in the training, they definitely know the model parameters, which contradicts with black-box attack.

- Table 1 is confusing.
    - (1) The paper says “we early stop at 85% for adversarial training”, but the reported acc of R50 (adv) is 81.23 (centralized) and 80.05 (federated).
    - (2) Since same-acc shows the early-stopped results, it should be lower than regular. However, in federated CNN (adv), the adv. acc of same-acc (25.5) is higher than regular (24.35). Can you explain why?
    - (3) The paper conducts a “fair” comparison (same-acc) and says “when both models reach a comparable clean accuracy, federated model shows greater robustness against white-box attack compared with the centralized model.”I don’t think this experiment is fair: no evidence proves that when both models reach a comparable clean accuracy, they reach the same training stage. Previous studies [1] also show robustness may be at odds with accuracy. Thus, there is no strong relationship between acc and adv. acc.


- The paper conducts extensive experiments but lacks theoretical analysis. 


[1] Tsipras D, Santurkar S, Engstrom L, et al. Robustness may be at odds with accuracy[J]. arXiv preprint arXiv:1805.12152, 2018.
clarity,_quality,_novelty_and_reproducibility: The paper is not novel.
The code is not given, thus I don’t think it is easy to reproduce.

summary_of_the_review: The paper considers transfer-based black-box adversarial attack in FL. However, the setting is unrealistic. The experiments are confusing. Thus， I think the paper is below the bar of ICLR.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper studies the transfer attacks in the context of federated learning (FL) where the attackers don't act maliciously during training but use the information they obtained to perform transfer-based adversarial attacks later on. The paper provides quantitative and qualitative results on the factors that affect the attack success on a simple FL setting.
strength_and_weaknesses: + Covert attackers using FL training for reconnaissance and performing attacks later on is a totally valid threat model that is significant enough to study.

clarity,_quality,_novelty_and_reproducibility: - Poorly motivated threat model, why would the attacker perform a transfer attack when FL protocol when there are malicious clients who observe the victim model at every FL round?

- Experiments are insufficient, vanilla PGD is not particularly transferable.
summary_of_the_review: 1) The proposed threat model doesn't make too much sense to me. Malicious clients don't do anything but they observe the model at every round of FL to be able to send the server the models updates on their data. So, these clients have access to the actual victim model (or something very close to it from the later rounds of FL). Why would they then train another model to launch transfer attacks on the victim model? Can't they just performing an almoist white-box attack? Please tell me if I'm missing something, I'm willing to update my review.

If the setting is that the actual attacker knows the data of some of the clients (without being able to observe the FL models these clients train), then we can't call the clients malicious anymore, if anything they're compromised. In this case, I still can't say the contributions of this paper are significant enough because there's plenty of work studying how training set knowledge affects transferability [1,2]. Although these papers study centralized training, I don't think the results will be different enough to justify a new paper.

2) A more interesting setting would be that the malicious clients are not used throughout the FL training. They participate in the first N round and become inactive afterwards. So, the malicious clients don't have access to final victim model but an intermediate model from the Nth round. Can the attacker leverage this intermediate model + data from the clients to improve the trainsferability to the final victim model? How does the attack success change as a function of N?

3) I would recommend using attacks other than PGD. There's a lot of work on designing transferable attacks in various settings [3,4]. The risk of using PGD is that it probably underestimates the transfer success.


[1] https://www.usenix.org/conference/usenixsecurity18/presentation/suciu
[2] https://www.usenix.org/conference/usenixsecurity19/presentation/demontis
[3] https://openaccess.thecvf.com/content_CVPR_2019/papers/Inkawhich_Feature_Space_Perturbations_Yield_More_Transferable_Adversarial_Examples_CVPR_2019_paper.pdf
[4] https://openreview.net/forum?id=BJlRs34Fvr
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper delivers an empirical study to understand the transferability of adversarial robustness under federated learning. The reviewers found that the presented empirical evidences are not convincing enough and the conclusions may be misleading. For example, "the current scale of the experiment is not big enough to convince readers" and "the attacker controls some clients that can take part in the FL training and considers black-box attack where the attacker cannot access the model" (this concern is actually shared by two reviewers wDFS and bp6y). Thus, we cannot accept it for publication. If the authors would like to revise the paper and try another machine learning conference, I suggest the authors to carefully consider the problem setting to make it more realistic in the revised manuscript.
justification_for_why_not_higher_score: The presented empirical evidences are not convincing enough and the conclusions may be misleading.
Decision: Reject