Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper studies attacks and defenses for federated learning. In particular, 
1) The paper first observes that a “filtering based” attack that uses a few “trusted clients” can help bypass some attacks.
2) Then it presents an attack called “mimic shift” that somehow improves previous attacks known as “mimic” attacks and bypasses the simple defense.
3) Then it improves the defense to add a re-weighting step and shows that it can defend against some standard attacks. The idea is to use a bunch of trusted nodes and use them to approximate the statistics of the gradient, and use that to filter out the outliers. Also, re-weight the nodes to balance out their influence on the gradients.
The paper does some theoretical analysis of their defense (under quite a few assumptions). It also does some experimental study for the 3rd part above.

I have provided more details below, but I think in summary, the paper suffers from an issue that many papers in adversarial/robust learning do: it does not study “adaptive” attacks that are specifically designed with the knowledge of a defense. That is why these works usually fall into a cat and mouse game of defense/attack/defense/etc. By now there are provably approaches in robust (poisoning resistant) learning that apply to federated learning as well. For example look up [1] and other papers on “robust statistics”. This paper ignores that line of work and merely studies heuristic approaches with much weaker guarantees

[1] Jia, Jinyuan, Xiaoyu Cao, and Neil Zhenqiang Gong. "Intrinsic certified robustness of bagging against data poisoning attacks." AAAI’21

In fact, the paper explicitly says “Other attacks, including data poisoning …  and backdoor (..),
are beyond the scope of this work.”
But why is that so? Isn't their framing more general than yours?

strength_and_weaknesses: Strength: aiming to provide solutions for federated learning that resist many corruptions. It is also interesting to combine two different methods as a defense, as they could work well together.

Weakness: the evaluation is not against adaptive attacks (who know the defense). The theoretical part has too many assumptions that are hard to interpret.

clarity,_quality,_novelty_and_reproducibility: As mentioned above, the writing is not great: it ignores provably approaches to resist attacks against general poisoning attacks (that cover attacks in federated learning as a special case). The theory part is also not easy to read and interpret/ understand.
 
You say in the introduction that you need secure hardware. Where exactly do you use secure hardware? Are you only referring to the trusted parties? That is not what other papers mean by secure hardware (enclave) usually.

Some comments:

Why use F for the loss function, and not, say, L?

You say “The Mimic-Shift attack is also difficult to detect because the malicious g′ has the same distance to the reference ¯g_R as the benign ¯g.”
But this is an extremely simple way of detecting things and says nothing about a detector that does something slightly more intelligent.

I strongly recommend rewriting the theory section and explaining/discussing/interpreting your assumptions as you make them. Also, the theorem statements are not clear. With so much math notation and words hanging there, it is hard to keep track of what the theorems are really proving. There are also specific assumptions in the theorem statements that need to be discussed and  explained why making them is reasonable.

summary_of_the_review: In summary, I think the main question of the paper is interesting. But the methods are not fundamentally new, are not adding real guarantees (as they can be broken in the next paper) even though such provable approaches exist, and the paper’s writing in the theory section is not great. Therefore, I cannot support it for ICLR acceptance, even though I acknowledge the merits of the paper.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a new method to build robust FL. The paper performs theoretical analysis and evaluation on multiple datasets and baselines. 
strength_and_weaknesses: Strength

+ A new robust FL method is proposed. 

+ Theoretical analysis is performed on the proposed method to show its robustness. 

Weakness

- Some baseline attacks are not evaluated, e.g., [A]

- Adaptive attacks are not considered. Defending against existing attacks is not hard. But the challenge is that attackers are always adaptive to the defenses. So we should evaluate adaptive attacks that adapt to the defense. 

- FLTrust description seems not correct. I thought the FLTrust does not only consider cosine similarity. It also uses ReLU to clip cosine similarity. 

[A] local model poisoning attacks to byzantine-robust federated learning. In USENIX Security Symposium, 2020. 
clarity,_quality,_novelty_and_reproducibility: The paper will be stronger if more baseline attacks and adaptive attacks are evaluated. 
summary_of_the_review: The paper proposes a new robust FL, performs theoretical analysis on the method, and compares with some baselines. The paper would be stronger if more baseline attacks and adaptive attacks are evaluated.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a new type of attack named Mimic-Shift and designs a two-phase defend framework under majority adversary setting. The proposed framework combines classic filtering method defending standard attack together with a new re-weighting method reducing the influence of Mimic-Shift . In order to weaken the effect of averaged gradients from adversaries while strengthening the effect of average gradient from benign users, the proposed method utilizes projection to judge whether a client is a benign user or an adversary client in the setting where exists majority adversaries. The effectiveness of Mimic-shift and Mimic-shift-Par is shown and compared with other kinds of attacks. And the performance of the proposed method against attacks is demonstrated with the report of accuracies in several classic benchmarks of Federated learning (FEMNIST, CelebA, and Shakespeare), compared with some baselines designed for defending against standard attacks. 
strength_and_weaknesses: **Strength and Weaknesses:**

This paper does well in: (1) clearly explain the intuition behind their proposed methods. (2) the idea of judging benign/adversarial clients with projection on average gradient is innovative. (3) the proposed attack and defend method seem promising under majority adversary setting observed from the results of experiments. 

As for the weaknesses, I have several concerns requiring the authors to clarify: (1) the paper doesn't convincingly clarify the extensiveness of the setting (Majority adversaries can be a common situation in Federated learning systems). (2) some parts of the results of experiments are not clearly analyzed in the paper. (3) The theory does not provide a theoretic guarantee for dropping accuracy caused by Mimic-shift. (4) the effect of filtering is not demonstrated in practice and missing of that may makes the validity of re-weight phase less convincing. 
clarity,_quality,_novelty_and_reproducibility: **Main argument**

For the problem the paper is trying to solve, I cast doubts on whether the majority adversary is universal in FL systems, which may limit the performance of methods designed for fighting against it in real-case scenarios. As the author claims in the introduction, the client availability issue may limit the number of clients participating in a round. However, majority adversaries will still happen with a small probability because the server randomly selects clients, for which it is hard for adversaries to be majority even in a round when the whole system is under minority adversaries setting. 

The idea of measuring good/bad clients with the projection of average gradient on them is innovative; however, I'm not sure why it works even in minority adversary setting where the method using projection should misjudge benign users as an adversary with more detriments on the resulting average gradient. 

The theory misses the part proving that Mimic-shift will damage the accuracy of federated learning under the majority adversary setting, which I think is not fully confirmed with experiments as well because there are not enough different hyper-parameter studies for Mimic-Shift and Mimic-Shift-Par. 

From the experiment, MIMIC-Shift-Par is an effective and practical attack method. However, there is no explanation for why it can attack effectively without knowing the benign users. Randomly eavesdropping 20% of clients per round when the majority of clients are adversary means the clients being eavesdropped on are also mainly adversaries, which leads to great bias when assuming the averaged gradient of eavesdropped clients represents the average gradient of benign users. In that situation, a theory explaining why MIMIC-Shift-Par works is welcomed. 

For the experiments, the following should be addressed.

1. evaluate the damage Mimic-shift can cause when a relatively small proportion of adversaries exist in Federated learning systems can cause (or analysis/measurement of the probability majority of FL systems will be adversaries with random sampling clients)?
2. It is confusing whether "Oracle" means no attacks for all clients or not. If so, in Table 6 the accuracy of "Oracle" is 0.870, different from 0.869 in Table 1; what causes that difference in the same setting?
3. explain why the proposed method also does well in the minority adversary setting.
4. Ablation study of phase 1 and phase 2 demonstrating that filtering and re-weight both play vital roles in defending against attacks. 
5. why the variance of all methods in Table 1 is 0.001, is the effect of attacks really so robust to randomness as observed from the variance?
6. results in Mimic-Shift and Mimic-Shift-Par with different hyper-parameters will help to confirm the effectiveness of these two attacks.

Minor comments:

1. Page 9: In 6.3 DEFENSE AGAINST MIMIC-SHIFT ATTACK, "Our method occationally outperforms the oracle." Does "occationally" here mean "occasionally"?
2. In Table 1, I think the accuracies of standard attacks are suitable to be put here instead of in the appendix. 
3. In Table 2, I think 40% adversary is a little too high for minority adversary setting, maybe 20% adversary is a better choice. 
summary_of_the_review: Overall, this paper proposes a practically effective way of attacking and a method against it; however, there is some flaws in both theory and experiments. Given these clarifications in an author response and some necessary complementary experiments, I would be willing to increase the score.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper focuses on robust federated learning against adversarial clients, which hinder the convergence and performance of federated learning via injecting malicious local updates. Unlike other previous works, this paper considers the majority adversarial setting, where the adversarial clients are more than those benign clients and thus break the existing defenses. Given the assumption of having few trusted clients for reference, this work proposes a two-staged defense combining filtering and re-weighting to defend against a broad class of attacks. Extensive experiments are conducted to verify the effectiveness of the proposed method, and theoretical analyses are provided.
strength_and_weaknesses: Strength:
1. This paper explores a new setting, which differs from the assumption in most previous literature, i.e., the majority adversary setting. It is practical as indicated in Kairouz et al. (2021) [1].
2. The proposed two-staged defense shows empirical effectiveness on several baseline methods.

Weaknesses:
1. The motivation for the proposed second-stage reweighting is not very clear. It could be better improved by adding a more intuitive example or illustration that the reweighting mechanism is the target choice for defending the mimic-shift attack.
2. The technical novelty of the current presentation is limited, as the reweighting mechanism or filtering is commonly used to defend against poisoning attacks. It is encouraged to add more discussion about the rationality of adopting reweighting in this problem.
3. It may lack some empirical evidence to verify the illustration in Figure 2.

Other questions/comments:
1. Is there any empirical evidence supporting "the decentralized nature means that it is relatively straightforward for the adversary to be the majority and thus break existing defense"?
2. It is better to link some major clarification with that empirical evidence or support with reference.
3. Why the performance of aggregators can still outperform the oracle when the adversary is 0% in Table 2?

[1] Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurelien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. Advances and open problems in federated learning. Foundations and Trends® in Machine Learning, 14(1–2):1–210, 2021.
clarity,_quality,_novelty_and_reproducibility: Clarity: Some clarification about the unique problem setting and empirical results can be further added or improved.

Quality: The technical quality of this work is of high quality with solid theoretical results.

Novelty: This work proposed a two-staged defense using filtering and re-weighting to tackle the majority adversary setting. The conceptual novelty of considering the majority adversary setting is high. However, the current presentation does not highlight the technical novelty of the proposed method, as filtering and reweighting seem to be some common strategies used in previous literature about defending.

Reproducibility: The current version of this work does not provide its source code. With limited experimental details, it is hard to adjust the reproducibility of the proposed method. It is encouraged to open-source the code or provide more implementation setups than hyperparameter sets.
summary_of_the_review: Overall, this paper proposes an effective method for the majority adversary setting of the robust federated learning problem. This paper could be further improved to make the clarification clearer and stronger, as well as highlight the technical novelty of the proposed method by adding more conceptual and intuitive comparisons.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper proposes a new type of attack named Mimic-Shift and designs a two-phase defend framework under majority adversary setting. The proposed framework combines classic filtering method defending standard attack together with a new re-weighting method reducing the influence of Mimic-Shift. In order to weaken the effect of averaged gradients from adversaries while strengthening the effect of average gradient from benign users, the proposed method utilizes projection to judge whether a client is a benign user or an adversary client in the setting where exists majority adversaries. The effectiveness of Mimic-shift and Mimic-shift-Par is shown and compared with other kinds of attacks. The performance of the proposed method against attacks is demonstrated with the report of accuracies in several classic benchmarks of Federated learning (FEMNIST, CelebA, and Shakespeare), compared with some baselines designed for defending against standard attacks.

However, there are several obvious weakness: 1) The motivation for the proposed second-stage reweighting is not very clear. It could be better improved by adding a more intuitive example or illustration that the reweighting mechanism is the target choice for defending the mimic-shift attack. 2) The technical novelty of the current presentation is limited, as the reweighting mechanism or filtering is commonly used to defend against poisoning attacks. It is encouraged to add more discussion about the rationality of adopting reweighting in this problem. 3) Regarding adaptivity, you need to define your defense and allow the adversary to know your defense and still show that the defense works. Meanwhile, the setting of AAA21 paper seems more general as a defense. Namely, they allow a number of parties to submit completely malicious data (or models that are completely malicious). Overall, this paper may not be ready for publication at ICLR. The next version must be a strong paper if authors can take comments into consideration.
justification_for_why_not_higher_score: According to my expertise and reviewing process, this paper should belong to a Reject.
justification_for_why_not_lower_score: According to my expertise and reviewing process, this paper should belong to a Reject.
Decision: Reject