Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a dynamic backdoor defenses in federated learning, where both the defender and the attacker can dynamically adjust their strategies as the federated training continues. The defender reverse engineers the potential backdoor trigger so as to finally generate the benign score for each client, and downweights the clients with lower benign scores. Attackers aim to find the proper backdoor poisoning ratio such that a good tradeoff between backdoor effectiveness and benign scores of compromised clients can be reached. 
strength_and_weaknesses: Strength:
The paper is well written and the idea of defending against adaptive attackers is impressive. 
Weakness:
1. the whole framework relies on the defender is able to reverse engineer useful backdoor triggers so as to effectively minimize the impact from compromised clients. However, this design may be exploited by the attacker [1]. Specifically, it is possible that attacker generate backdoors in a way that, either the reversed engineered backdoor samples are not effective or are similarly effective for both the compromised and benign clients. This assumption is the major weakness of this paper. 
2. it is unclear how well those theories (e.g., Theorem 1) imply in practice. For example, the assumption on $r_i^t=0$ seems to not hold in practice and so, the backdoor model cannot be the same the clean global model, even asymptotically as the training rounds continues.  
3. the threat model makes too many restrictions on the attacker capability. For example, the attackers can only augment the training data with backdoored samples while in practice, they may choose to replace some samples with backdoored samples. The attackers are also assumed to be unaware of the benign scores of other clients while in practice, they may be able to simulate the benign scores (given the knowledge that the server is calculating benign scores for each client using reversed engineered triggers). In the worse case scenario, powerful attackers may also be able to get the actual benign scores. It is also assumed that attackers can only use its own backdoor trigger when deciding the proper poisoning ratio while in practice, attackers can also simulate the reverse engineering process of the defender. 
4. there are also some unrealistic experimental settings in the paper. For example, it is assumed that all clients in each round will be selected while in practice, defenders usually randomly selects subset of the clients. Also, the fraction of compromised clients are mostly evaluated under 60%, which, theoretically, should be hopeless to defend against. The lowest fraction of compromised clients is still 20%, which is quite a high number. I am curious to see the comparison among different defenses (not just FLTrust and the proposed method) under even lower compromised clients. 
5. some recent works are missing from the paper. For example, the most recent defense on backdoor attacks in federated learning [2]. 
[1] Veldanda et al., "On Evaluating Neural Network Backdoor Defenses".
[2] Rieger et al., "DeepSight: Mitigating Backdoor Attacks in Federated Learning Through Deep Model Inspection", NDSS 2022. 
clarity,_quality,_novelty_and_reproducibility: The writing quality is adequate and the paper is mostly easy to follow. However, it is worth mentioning in the paper, whether the backdoor attacks corresponding to different defenses are the same or not. For the baselines, I believe the same backdoor attacks can be leveraged. But for the proposed defense, since it involves dynamic game between the defender and attacker, the attacks are actually changing.

As for the originality, this work is not the first to propose defenses based on reputation scores of the clients. In addition, the reputation score computation itself vulnerable to adaptive attacks, and I am concerned that the contribution is limited. 
summary_of_the_review: The idea of defending against adaptive backdoor attacks in federated learning is interesting. However, the proposed framework did not consider this "adaptive" aspect thoroughly and hence, is not convincing. Also, the experimental settings are not practical and some recent baselines are missing. Considering all the weakness mentioned above, rejection is given.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a minimax game model of backdoor attacks between attackers
and defenders in the federated learning (FL) paradigm.  Based on the analysis
of this model, it uses a reverse-engineering technique to defend against
backdoor attacks in the FL process. It provides theoretical analysis and
experimental results to present the effectiveness of its defense mechanism.

strength_and_weaknesses: It is an interesting angle to look at the federated learning backdoor problem.
The analysis is well presented.

Its discussion is limited to strong assumptions about the adversary.
The theoretical analysis makes impractical assumptions about backdoor attacks.
The proposed defense has limited novelty.
clarity,_quality,_novelty_and_reproducibility: I think the analysis of this paper is not very general. The paper restricts
attackers and their capabilities to its designed minimax game model. This
paper proposes a minimax game model between defenders and attackers, where
both optimize a genuine score of backdoored and clean models. However, in a
practical scenario, the attack methods are unavailable, so attackers don't
need to optimize the genuine scores. So this game model considers a restricted
assumption of attackers' attack methods, making its comparison with other
baselines less convincing. 

The theoretical analysis to guarantee the robustness of its method is based on
an impractical assumption. In backdoor attacks, the common triggers are not
bounded by $L_p$ distances. This paper assumes that the Lipschitz continuous
gradient will bound the backdoor updates of the loss function. It requires the
backdoor attacks in the FL model to be as strong as possible to make sure it
can be bounded. Existing works consider a successful backdoor attack (e.g.,
attack success rate over 90%) rather than consider the strongest backdoor
attack. Stronger attacks are more likely to be detected. So I think this
assumption is not practical.

The proposed defense method has limited technical novelty as it mostly applies
a typical reverse-engineering method (Neural Cleanse). There are many existing
backdoor attacks like WaNet [3], the invisible attack [4], which can bypass
Neural Cleanse. These attacks consider trigger patterns other than patches to
break the assumptions of Neural Cleanse. So it is also important to consider
these attacks in this paper, which can help enhance the effectiveness of its
method.

[1] Gu et al. "BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain" IEEE Access 2019
[2] Liu et al. "Trojaning Attack on Neural Networks" NDSS 2018
[3] Nguyen et al. "WaNet - Imperceptible Warping-based Backdoor Attack" ICLR 2021
[4] Li et al. "Invisible Backdoor Attack with Sample-Specific Triggers" ICCV 2021

summary_of_the_review: This paper makes strong assumptions about attackers, and it lacks technical novelty (proposed defense).
review_title: null

final_score: rating: 1: strong reject
review_content: summary_of_the_paper: This paper proposes to formulate the defense against backdoor attacks as a minimax game between the server (defender) and attackers in federated learning. The minmax problem is solved in three steps by 1) construct an auxilary global model; b) reverse-engineer backdoor samples with selected target classes; c) assign to each client a genuine score is used to weigh model update uploaded by the client. 

strength_and_weaknesses: 
Sorry that I don't see any strength in the paper.  

The proposed method is seriously flawed in my view for following reasons: 

- the assumption that the server is able to access and inspect local model updates poses privacy leakage risks on private data in the face of deep leakage type of gradient attacks [1];  this serious privacy risk defeats the purpose of federated learning in the first place;  authors are advised to reconsider the threat model and take into account various privacy-preserving mechanisms such as Differential privacy or Homomorphic Encryption; 

[1] Deep Leakage from Gradients, NeurIPS 2019; 

- the proposed method evaluates local model updates based on the agreement with the reverse-engineered trigger sets and target classes; nevertheless, the approach does not make sense at all since the target class selected by the reverse-engineering process is by no means the same as the target class might be selected by real backdoor attackers; the paper proposes to “find the backdoor trigger and target class such that the genuine scores for benign clients are large but they are small for compromised clients”， however, the compromised clients are a prior unknown, and the proposed method cannot reliably estimate the target class and solve this chicken and egg problem; 

- the proposed method is inefficient in the evaluation of local model update (Line 4-5 in Algo. 1); this method cannot be efficiently applied to a federated learning scenario with large number of clients; that is probably why experiments reported in the paper only used 10 and 20 clients, which are far less than typical cross-devices use cases with millions of clients; 
clarity,_quality,_novelty_and_reproducibility: 
The paper is easy to follow; the proposed method is seriously flawed in my view; 
summary_of_the_review: The proposed method is seriously flawed in my view for following reasons: 

- the assumption that the server is able to access and inspect local model udpates imposes privacy risks on private data in the face of deep leakage type of gradient attacks [1];  this serious privacy risk defeats the purpose of federated learning in the first place;  authors are advised to reconsider the threat model and take into account various privacy-preserving mechanisms such as Differential privacy or Homomorphic Encryption; 

[1] Deep Leakage from Gradients, NeurIPS 2019; 

- the proposed method evluates local model udpates based on the agreement with the reverse-engineered trigger sets and target classes; nevertheless, the approach does not make sense at all since the target class selected by the reverse-engineering process is by no means the same as the target class might be selected by real backdoor attackers; the paper proposes to “find the backdoor trigger and target class such that the genuine scores for benign clients are large but they are small for compromised clients”， however, the compromised clients are a prior unknown, and the proposed method cannot reliably estimate the target class and solve this checken and egg problem; 

- the proposed method is inefficient in the evaluation of local model udpate (Line 4-5 in Algo. 1); this method cannot be efficiently applied to a federated learning scenario with large number of clients; that is probably why experiments reported in the paper only used 10 and 20 clients, which are far less than typcial cross-devices use cases with millions of clients;
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper introduces FLGAME, a defense against adaptive backdoor attacks. It formulates the compromised client attack and server defense as a form of minimax optimization where the clients want to maximize their contribution to the model update and the server wants to minimize malicious contribution. This optimization centers around a genuine score that weights the model update during aggregation. The server computes this through a reverse engineered backdoor trigger and target class. 
strength_and_weaknesses: Strengths: 
•	FLGAME achieves relatively high model accuracy with a very low attack success rate compared to prior defense.
•	The dataset the server has does not need to be in-domain. This is a much easier requirement to achieve that prior work.
•	The introduction of using an adaptive defense is a good contribution.

Weaknesses:
•	The experimental datasets were very simple. I would like to see more complex datasets, such as ImageNet/Tiny Imagenet. 
•	Please expand on the contribution of the compromised clients in the model update. It’s not clear whether the attack success rate is low because the compromised clients have a low genuine score or if their updates result in weak backdoor success.  
•	It is okay to include a few baseline comparisons. However, many of the defenses compared were not intended for backdoor attacks. Please show comparisons to other new defenses aimed for backdoor defense.
•	The paper relies on prior work for reverse engineering backdoor triggers and target class. I would like to see more about this. What limitations does this have? If the compromised clients use larger L1 norm triggers, does this fail? 

clarity,_quality,_novelty_and_reproducibility: The discussion of the attack/defense used was clear. The paper and experiments are reproducible.
summary_of_the_review: The method of using a minimax game and the use of an adaptive defense is a very interesting and important contribution. However, my main issue is the lack of depth in the empirical results. In particular, the datasets used and the analysis on compromised client participation can be made much stronger. Please refer to the weaknesses section for specifics.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: In this work, a minimax game is used to design an interactive defense mechanism called FLGAME which is proven to be theoretically robust and empirically more effective than state-of-the-art baselines. FLGAME can effectively defend against strategic attackers.

the reviewers identified the following strengths and weaknesses 

Strengths: 
• FLGAME achieves relatively good accuracy with a low attack success rate compared to prior defense. 
• The dataset the server has does not need to be in-domain. 
• The use of an adaptive defense mechanism.

Weaknesses: 
• The experimental datasets are very simple. 
• The set of baselines tested against are questionable. 
• The paper relies heavily for reverse engineering backdoor triggers and target class. This may lead to a weak defense mechanism 
• The proposed method evaluates local model updates based on the agreement with the reverse-engineered trigger sets and target classes.  this does not accurately simulate a real-world backdoor attack.
• The proposed method can be inefficient when evaluating model updates. 
• The assumption that the server is able to access/inspects local updates poses privacy risks.

In the rebuttal, the authors evaluated their defense against different trigger sizes, evaluated it when an attacker replaces training samples with backdoored samples, compared it with the state-of-the-art baseline DeepSight, evaluated it under static attacks, and evaluated the running time of evaluating local models of clients for the server. Additionally, the authors clarified and explained assumptions made, results when the fraction of malicious clients is low, the information required for the defender of their defense, their threat model, and the attacks they consider for evaluating their framework.

However, the reviewers' major concern still remained whether the proposed defense could withstand future attacks. One suggested that the authors should concretely define the action space to cover possible actions, and consider the possibility of new attacks once the work is published.

I also stand by the comments made from the reviewers, and believe this paper is not ready for publication, but does have the potential, if the above concerns are thoroughly addressed.
justification_for_why_not_higher_score: The reviewers identified some weaknesses that the authors attempted to address in their rebuttal, but the major concern remains whether the proposed defense is general, and strong enough.
Decision: Reject