IBA: Towards Irreversible Backdoor Attacks in Federated Learning
Authors: Dung Thuy Nguyen
Tuan Minh Nguyen
Anh Tuan Tran
Khoa D Doan
KOK SENG WONG
Venue: NeurIPS 2023 poster (2023)
Keywords: Backdoor Attacks
Federated Learning
Durability
Imperceptibility
Stealthiness
TL;DR: The paper introduces the IBA, a novel framework in federated learning (FL) that stealthily and durably implants imperceptible backdoors into global models, demonstrating higher effectiveness and resilience than conventional backdoor attacks in FL.

Abstract: Federated learning (FL) is a distributed learning approach that enables machine learning models to be trained on decentralized data without compromising end devices' personal, potentially sensitive data. However, the distributed nature and uninvestigated data intuitively introduce new security vulnerabilities, including backdoor attacks. In this scenario, an adversary implants backdoor functionality into the global model during training, which can be activated to cause the desired misbehaviors for any input with a specific adversarial pattern. Despite having remarkable success in triggering and distorting model behavior, prior backdoor attacks in FL often hold impractical assumptions, limited imperceptibility, and durability. Specifically, the adversary needs to control a sufficiently large fraction of clients or know the data distribution of other honest clients. In many cases, the trigger inserted is often visually apparent, and the backdoor effect is quickly diluted if the adversary is removed from the training process. To address these limitations, we propose a novel backdoor attack framework in FL, the Irreversible Backdoor Attack (IBA), that jointly learns the optimal and visually stealthy trigger and then gradually implants the backdoor into a global model. This approach allows the adversary to execute a backdoor attack that can evade both human and machine inspections. Additionally, we enhance the efficiency and durability of the proposed attack by selectively poisoning the model's parameters that are least likely updated by the main task's learning process and constraining the poisoned model update to the vicinity of the global model. Finally, we evaluate the proposed attack framework on several benchmark datasets, including MNIST, CIFAR-10, and Tiny ImageNet, and achieved high success rates while simultaneously bypassing existing backdoor defenses and achieving a more durable backdoor effect compared to other backdoor attacks. Overall, IBA offers a more effective, stealthy, and durable approach to backdoor attacks in FL. The code associated with this paper is available on [GitHub](https://github.com/sail-research/iba).

IBA: Towards Irreversible Backdoor Attacks in Federated Learning
Dung Thuy Nguyen 1.2, Tuan Nguyen 2.3, Tuan Anh Tran', Khoa Doan 2.3, Kok-Seng Wong?* 2:3 Deppatment of Computer Science, Vanderbilt University, Nashville, TN 37212, USA -Vinnni-llinois Smart Health Center, VinUniversity, Hanoi, Vietnam Coolleee of Engineering Computer Science, VinUniversity, Hanoi, Vietnam VinAI Research, Hanoi, Vietnam
dung. t. www.een@aaannrrbll...... Edu, tuan nm@vinuni aeduuuuiiedd...... vn, anhtt152@vinai io, khoa. dd@vinuni edu. vn, wong. ks@vinuni. edu. vn
Abstract
Federated learning (FL. is distributed learning approach that enables machine learning models be trained on decentralized data without compromising end devices' personal, potentially sensitive data. However, the distributed nature and uninvestigated data intuitively introduce new security vulnerabilities, including backdoor attacks. In this scenario, an adversary implants backdoor functionality into the global model during training, which can be activated to cause the desired misbehaviors for any input with specific adversarial pattern. Despite having remarkable success in triggering and distorting model behavior, prior backdoor attacks in FL often hold impractical assumptions, limited imperceptibility, and durability. Specifically, the adversary needs to control a sufficiently large fraction of clients know the data distribution of other honest clients. In many cases, the trigger inserted is often visually apparent, and the backdoor effect is quickly diluted if the adversary removed from the training process. To address these limitations, we propose a novel backdoor attack framework in FL, the Irreversible Backdoor Attack (IBA), that jointly learns the optimal and visually stealthy trigger and then gradually implants the backdoor into global model. This approach allows the adversary to execute backdoor attack that can evade both human and machine inspections. Additionally. we enhance the efficiency and durability of the proposed attack by selectively poisoning the model's parameters that least likely updated by the main task's learning process and constraining the poisoned model update to the vicinity of the global model. Finally, we evaluate the proposed attack framework several benchmark datasets, including MNIST, CIFAR- 10, and Tiny ImageNet, and achieved high success rates while simultancously bypassing existing backdoor defenses and achieving more durable backdoor effect compared to other backdoor attacks. Overall, IBA offers a more effective, stealthy, and durable approach backdoor attacks in FL.
Introduction
Federated Learning (FL) 13
novel distributed learning paradigm that enables multiple parties to train machine learning models collaboratively without centralized data sharing. Specifically, each participant trains local model on their private data, and only the model updates are exchanged with central server for aggregation. However, FL also introduces new security and privacy challenges
"Wok done while DTN was at VinUniversity.
"Code for this paper published htttp::://gttaa...o///alleeeeeee
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

that need to be addressed to ensure the confidentiality and integrity of the learning process B3 24).
such as model poisoning, membership inference, data leakage, and Sybil attacks (9 20
Backdoor attacks are a specific form of model poisoning in which an attacker inserts a backdoor trigger during the training process that can later be exploited manipulate the model's behavior 4 12 17 6
FL is particularly vulnerable to backdoor attacks since each participant trains a local model on their private data, and the central server aggregates the model updates without validating their integrity. An attacker can inject poisoned data or backdoor triggers into the local models of a subset of participants. When the models are aggregated, the backdoor trigger can be activated to compromise the model's behavior 12 28 32].
standard method for backdoors in FL. In artificial backdoor attack, an apparent trigger, such as a specific pixel value, is injected into the local training data [25] 32 Although meaningless in the data context, the apparent trigger causes the trained model to associate signal for the model to optimize during training. When trained on such data, the model will induce misclassification toward target class when making predictions on inputs with the presence of the trigger. Existing artificial backdoor attacks require the involvement of a portion of compromised clients and need assistance from scaling-based methods achieve high attack success rates
Artificial backdoor attacks
This makes these attacks vulnerable to mainstream backdoor defenses [30 31 [5]. Furthermore, most existing attacks are not durable (i.e., when the adversary removed from the FL process, the
backdoor effect is gradually diluted [34)), especially when the size of compromised clients is small, and they do not regularly participate in the FL training.
In this paper, we exploit the principle of adversarial attacks to craft a novel instance-specific backdoor attack, called Irreversible Backdoor Attack (IBA), that only requires small number of compromised clients but achieves more robust attack performance and durability. IBA is a two-phase attack scheme, including Trigger Generating and Backdoor Injection. During training rounds, the former manipulates the attack model, which is a generative model, so the triggers generated by this attack model can deceive the local model into misbehaving. After that, we formulate the Backdoor Injection phase as constrained optimization problem. Then, we propose simple but effective recall to keep the attack model updated regularly. This algorithm allows the adversary achieve high attack success rates even when a small number (i.c., 1) of clients are compromised. In addition, we enhance the durability and effectiveness of the proposed attack by selectively poisoning the model's parameters that are least likely updated by the main task's 's learning process and constraining the poisoned model update to the vicinity of the global model, respectively. This strategy helps IBA bypass mainstream defenses in FL and achieve extended backdoor durability.
Our contributions are summarized below:
Proposal of two-phase backdoor injection mechanism, IBA, in FL: The paper introduces IBA, which combines the manipulation of the trigger generator and the poisoning of the local model into unified two-phase approach. IBA constrains the poisoned model update to be within the vicinity of the global model to improve the practicality and effectiveness of the attack.
Proposal of selective parameter poisoning in the constrained optimization algorithm: The paper enhances the durability of IBA by selectively poisoning the model's 's parameters less likely to be updated during the main task's 's learning process.
State-of-the-art attack performance and stealthiness: The paper demonstrates that IBA achieves state-of-the--rt attack performance and durability and exhibits high stealthiness against existing defense mechanisms. This highlights the robustness and effectiveness of IBA as a practical backdoor attack method.
2 Related Work
Backdoor Attacks in FL. Backdoor attacks in FL have been extensively studied, revealing vulnera- bilities and proposing various attack strategies [16). Backdoor attackers generally aim to incorporate concealed backdoors into Deep Neural Networks (DNNss during the training phase. This allows the compromised DNNs to exhibit normal behavior when presented with benign samples. However, their predictions will be deliberately and consistently altered when triggered by specific patterns defined by the attacker. One of the most commonly employed methods for incorporating backdoor

---

functionality during training is poisoning the training samples 32]. Bagdasaryan et al. 2 also proposed model replacement to implant backdoors into the joint model and demonstrated that even a single-round attack with a small fraction of clients could achieve 100% Attack Success Rate (ASR). Xie et al. al. investigated Distributed Backdoor Attacks (DBA), which decompose the global trigger into non-overlapping local triggers. DBA exhibits higher ASR and increased robustness against FL aggregation methods. Additionally, backdoor triggers in machine learning can be designed to be imperceptible in the input space [0,
17 6 and in the latent space (5), or to possess sophisticated, multi-target payloads 77]: the ground-truth labels of the poisoned samples can also align with the 35). These characteristics contribute the stealthiness of backdoor attacks. Recently, Neurotoxin 34 proposed to enhance backdoor durability by targeting specific coordinates less likely to be updated by benign agents, partially poisoning the model by projecting gradients onto a coordinate-wiss constraint. This extends the backdoor's impact and prevents catas- trophic forgetting, strengthening the effectiveness of the attack. The recent work that is most aligned with ours is PerDoor [. which uses the Basic Iterative Method (BIM) to generate the trigger, which relies on gradients of the model's loss function concerning the input data alter the original inputs into backdoored inputs. However, this method fails to learn underlying patterns and distributions from the training dataset and lacks sharing and collaborating mechanism among multiple malicious clients, which is very beneficial for the adversary in collaborative attacks.
[2
intended target label [26 12
Defense Mechanisms against Backdoor Attacks in FL. The mainstream back attkkk defenses in FL aim to reduce the effect on the global model, as it has been shown that detecting backdoors is challenging without breaking the privacy of participants [14]. For instance, Krum [ aggregates local models by choosing a single local model as the aggregated model with the smallest Euclidean distance certain fraction of other models. Norm difference clipping (NDC) 25 examines the norm-difference between the global model sent to and model updates shipped back from selected clients, using pre-specified threshold to clip model updates that exceed the norm difference. REA [19] aggregates local models by computing a weighted geometric median using the smoothed Weiszfeld's algorithm. Ozdayi al. [8] proposed defense mechanism RLR, that adjusts the learning rate of the aggregation server in FL based on the sign information of agents' updates by carefully considering per dimension and round. These defense mechanisms offer valuable insights and strategies for countering backdoor attacks in FL settings. Another approach is identifying suspicious clients and removing them from the aggregation process. FoolsGold g assumes that benign datasets from different clients differ from each other and assigns low weights to models that are similar to many other models, but it can be vulnerable to sophisticated adversaries submitting poisoned updates without raising suspicion. FLAME [15] uses the HDBSCAN algorithm to detect malicious updates, combines model filtering with poison elimination to detect and remove malicious updates, and is robust against inference attacks.
3 Irreversible Backdoor Attack (IBA) in FL
FL Framework. The training process in the standard FL framework involves the following steps:
Step 0 (FL Initialization): The central server S initializes the weight of the global model to and sets hyperparameters such as the number of FL rounds and local epochs.
Step (Local Model Training and Update): Selected clients C2.... Cx receive the current global weight we from S. Each client C, updates its local model parameters w, using its local dataset D,, After local training, the clients send their local weights to S for model aggregation.
Step 2 (Global Model Aggregation and Update): S aggregates the received local weights and computes the aggregation result. The aggregated result is sent back to the clients for the next round of training. S aggregates the received local model weights using the FedAvg [13] formula:
K
w
Cx
Wk
CR
where [C,l represents the number of data samples held by client Cs.
Attacker Threat. To address the weakness of existing works, i.e., the adversary A controls number of compromised clients, we focus on the scenario that A controls only one client and it will participate in the training for each ff rounds, i.c., fredunnyy attack (28).
3

---

Trigger Generating Function
Wt-1
W
Tw
poison
Target Label: yr
Letean
Lg
Detean
Adversania Nolses
poleon
G
clean
poison
Backdoor Injection Process
Target Yr
GE
Partial model poisoning
alvas
Adversarial Noises
palcom
Optimization path
Input
Figure 1: Irreversible Backdoor Attack Scheme. IEA includes training phases: Trigger Generating Function and (2) Backdoor Injection.
first phase, the trigger generation model Gę is updated to generate adversarial noise. This noise is combined with the original image o to create a backdoor sample to fool the local model w In the second phase, the local model fwy, is updated to perform indifferently on clean sample æ but distorts its prediction the backdoor image (x) the target class yr. then this poisoned model is sent to the aggregator. Model poisoning technique leveraged enhance the stealthiness and durability of attack.
Adversary Capability: We follow the same assumption used in the cxisting works on backdoor attacks in FL 2 [5]. where adversary A has total control over the malicious participant(s). However, A has no control or access to other benign participants' data or local updates. Also, we assume AA thoroughly understands the central server and any potentially deployed defenses but cannot alter the server's s configuration parameters and algorithms.
Adversary Objectives; The goal of A is learning trigger generator Ge such that its outputted triggers can activate backdoor function to mislead the global model F(w) while ensuring the behaviors on clean samples are unaffected. This trigger is expected to bypass human and machine inspections. Moreover, A can bypass configured defenses in FL and prolong the backdoor effect even when AA is removed from the training process. The overall mechanism of our proposed backdoor attack IBA is illustrated Figure]
3.1 The Trigger Generating Function
In FL, when clients actively engage in training round, they have the opportunity to observe and influence the changes the global model w. particular, client is receives the global model Wt-1 and updates it using their local data to produce a new local model, denoted wy. After local training, the learned surrogate model aggregated to partially reflect the subsequent global model. In light of this process, we propose an initial phase called the Trigger Generating. In this phase, the trigger generation model Ge is updated to generate adversarial noise. This noise is combined with the original image to create backdoor sample. This sample remains imperceptibly different from the original, but it effectively misleads the model G5 into making incorrect classifications for the target class. The rationale behind this approach is that manipulating Ge based this local model allows the adversary A to optimally adapt the generative function, even in the face of round-bi huundby-roouu by-round variations in the global model. The transformation function is inspired by adversarial examples, in which model it as a perturbation on the input, as follows:
Ta (x) Ge( || (z)llo S Vx
(I)

---

The function Ge takes an input r and generates adversarial noise on the same input space, guaranteeing the backdoor attack's stealthiness.
To achieve this goal, we utilize the local model as the surrogate model to update the attack model G using the following objective function:
$ - 5
Ce (fwr (Tx E
(2)
JED
in which fw is the trained local model and yr is the targeted label. It's worth noting that the local model fw submitted to the server will be totally vanilla during this phase.
While considering various norms for bounding the generator noise in Equation 1 such as the L norm, it's generally inadvisable for backdoor attacks. This infinity norm guarantees a widespread distribution of the generated trigger across the input image (encompassing all pixels in the trigger); in contrast, the L2 norm can result in localized artifacts within the image (with only select pixels forming the trigger). Employing the T2 norm can make the backdoor attack more susceptible to detection by seeger-sytthesis defenses like Neural Cleanse [27]. Thus, the ll norm a preferred choice.. We can design such generator function However, we observe no significant performance difference between a simple auto-encoder and U-Net by training the generator function and the classifier with the proposed training algorithm.
auto-encoder or the more complex U-Net architecture (21).
In the proposed transformation function, parameter controls the visual stealthiness of the triggers. For instance, even on the gray-scale MNIST dataset, if is less than 0.01, there is usually no discernible difference between the clean and disturbed images. A greater value for 6, i.e., more obvious triggers, simplifies generative model learning. On the other hand, a too-small may make learning generative model challenging since it is heavily dependent on the variance of the surrogate model, which varies across FL training rounds. Therefore, during this phase, we keep the sufficiently large, i.c., ED, until we reach the expected attack success rate.
3.2 The Backdoor Injection Process
At the second phase aims to gradually insert the backdoor function into the global model by ma- nipulating the poisoned local model. Our objective is to learn classifier (i.e., local model) say, that concurrently performs indifferently prediction on the backdoor image Te (x) to the target class yr. The above task can be formulated as the following objective function:
in comparison to the classifier's ''s clean version but distorts its
min
CaLetean (Jrr (3)) Un) + po (Sen, (Te( yr)
RED
The classification model fi in the above problem is trained based on local data and the optimal function Ge learned in the previous phase. When training the classifier, the parameters a and / regulate mixture of the strengths of the loss items from clean and poisoned data, i.e., Lotean and Epoisons respectively. rapidly converges to the optimal performance of the vanilla classifier, according to our experiments. Otherwise, the classifier's s performance on the backdoor data quickly reaches the optimal value. To compensate for the performance of the local model on both the main task and the backdoor task, assume a = 0.5 and = 0.5 in the remaining part of the paper. As previously stated, the value of must be assigned cautiously to overcome the unpredictability of the global model and assist the local model in learning the generated triggers more effectively. To achieve this goal, we propose a self-adjusting value for from the starting round of this phase, i.e., ty:
a is more significant than 3, the classifier's performance on clean data
" max(e, <0* (1.0 - -x),
(4)
where the value of c is gradually decayed until it reaches objective è.
Attack Model Retraining. During this phase, we propose frequently updating the attack model Ge to keep it in sync with the state the global model. The retraining frequency should be chosen (20, 50).
Partial Model Poisoning. To enhance the longevity and effectiveness of the attack, we propose to partially poison the model based on selected poisoning space and poisoning dimension. In which space-based poisoning helps the attack bypass the backdoor defenses because it constrains the
5

---

poisoned model such that it does not go far from the vicinity of the benign model, while dimension- based poisoning helps to reduce dilution effect from benign clients by shrinking the poisoned neurons to infrequently updated neurons.
Poisoning-sye space based. Under this attack, the submitted local model is partially contaminated by being constrained by the magnitude of its parameters. Specifically, the adversary applies projected gradient descent on the losses for D DUD' where D, D' are benign and poisoned datasets, respectively. In other words, it periodically projects the model parameter on the ball centered around the global model of the previous iteration. This enables IBA to eircumvent defenses based on norm and distance investigation by preventing the resulting poisoned model from deviating substantially from the original. Mathematically, this method guarantees that | w| S 8. Note that, this attack strategy can be combined with model replacement [2], where the model parameter is scaled before being The scaled model
sent to the server to cancel the contributions from the other honest clients
28
calculated by the following:
W Wk +
Σ RECVA
(wx w²)
(5)
Poisoning- -iiiiiniiin based. Because the attacker cannot participate in every round of training, directly submitting the poisoned model to the server not effective [3T] To address it, we propose leveraging historical gradient variation to select infrequently updated neurons, which are considered poisoning dimensions for IBA. Then the adversary computes bottom- 1% coordinates of the gradient vector g, obtained by learning the main task on the clean data D. In particular,
p B B,
1 * Be s.t., Br = bottomm (g:), P
(6)
P
in which p refers to the number of rounds up to round / that adversary A has been selected to participating in FL training. After that, when finishing calculating gradient 9 of local model fu using Equation. 3 A projects gradient onto coordinate of 9. As a result, only bottom-k%% &% coordinates are poisoned, which can prolong the backdoor effect since other benign clients rarely update these coordinates. Utilizing historical information aids in determining round-by-roume bottom-1:9 coordinates, and becomes more valuable as rounds advance and local client models converge the global model.
constraint on the top-k% coordinates
4 Experiments
Our empirical study aims to highlight the effectiveness of the IBA attack against the state-of-the-
art (SOTA) FL defenses. We conduct our experiments on real-world datasets and a simulated FL environment. We design three experiment sets: (1) IBA under different attack scenarios, (2)
IBA under different FL defenses, and (3) IBA's imperceptibilitt and durability evaluation. Our results demonstrate that the IBA can achieve significant efficiency, stealthiness, and durability. In particular, IBA attains persistent performance with/without SOTA defenses and maintains higher visual imperceptibiliity and longer durability compared with existing artificial backdoor attacks (22 2) Our implementation is publicly available to reproduce experimental results.
4.1 Experimental Setup
Dataset and Backdoor Tasks. The IBA method is evaluated on three classification datasets: MNIST, CIFAR-10, and Tiny ImageNet. We simulate heterogeneous data partitioning by sampling pa ~
Dirx (0.5) (0rxx(0001 for MNIST, CIFAR- 10/Tiny ImageNet and allocating proportion of each class to participating clients. We follow the established protocol outlined previous works 28 [2). employing stochastic gradient descent (SGD) optimization with E local epochs, a local learning rate of l,,, and a batch size of 32. The number of clients selected in each round is 10/200. The details of the datasets and other parameter configurations are presented in the supplementary material?
Trigger Generation and Backdoored Model Training. The generation of triggers is accomplished through the utilization of the U-Net architecture [21]. incorporating specific parameter configurations.
'The supplementary material nameessupleeenntrry _attiiiaaaaaaallllll
is available at httts:://pperrevieee net http:///www.eeeeeeeeecc/occcccca

---

Table 1: Effectiveness of IBA under different strategies in -reuunnry attacks setting (f 10)
Baseline (wo attack)
FedAvg
FedProx
FedNova
Dataset
MA
BA
MA
BA
MA
BA
MA
BA
CIFAR-10
84.71 %
9.52%
84.49%
87.13%
84.62%
86.59%
83.35%
94.32%
MNIST
99.74%
9.25%
98.11%
99.07%
97.98% 9898
99.97%
98.32%
92.74%
Tiny ImageNet
65.15%
0.56%
64.36%
94.19%
65.02%
92.52%
64.17%
93.11%
The constraint values are set to = 0.3, é = 0.05; while a = 0.5 and / = 0.5 define additional parameter values. During trigger training. the threshold for backdoor accuracy (BA) is established as local = 0.85. To facilitate the generation of updates, the learning rate for the update generation model determined as 7AA 0.0001, and for the update classifier model, the learning rate is set to 0.01. Additionally, the performance of IBA is evaluated in conjunction with prominent FL strategies, namely FedAvg [3. FedProx [ and FedNova 29). This comprehensive evaluation aims to assess the effectiveness of IBA in the context of different FL methodologies.
Participating patterns of attackers. In addition to the frequennyy case described in the threat model, we further evaluate the efficiency of IBA under a more favorable condition known as fixed-pool case (or random sampling). where there is a fixed pool of clients are controlled by A. The result for the fixed-pool case is left in the appendix.
FL Backdoor Defenses. We examine six SOTA defense techniques designed to mitigate backdoor attacks in FL: (i) NDC (25). (ii) KRUM and (iii) Multi-KRUM [3], (iv) RFA [19,, (v) RLR [18) (vi) FoolsGold [8] By evaluating the performance of these defense techniques, we aim to provide insights into their effectiveness in countering backdoor attacks in the FL setting.
4.2 Experimental Results
(1) IBA under different attack scenarios. Firstly, we study the performance of IBA under fixed- frequency attacks with three datasets: MNIST, CIFAR- 10, and Tiny ImageNet. The evaluation of main accuracy (MA) and backdoor accuracy (BA) from Table reveals interesting insights. In the absence of the attack, the MAs of the bascline are 99.74% 84.71%, and 65.15% for MNIST, CIFAR-10 10, and Tiny ImageNet, respectively, indicating accurate classification while the BA remains low at 0.56% to 9%, indicating no backdoor effect on global model. Under IBA attack with FedAvg aggregator, the corresponding MAs are negligibly affected, while the BAs are notably high, i.e., 99.07% for MNIST dataset. This indicates that IBA can stealthily insert a backdoor function without negatively impacting the model's behavior with benign samples.
clean samples,
In FL. many existing backdoor attacks target vulnerabilities within the FedAvg aggregator since the contribution of each client is considered equally. Therefore. examined IBA's re- silience across different aggregation methods, specifically FedAvg, FedProx, and FedNova, as presented in Table I FedProx, with its proximal term, enhances model accuracy and is ben- eficial in scenarios with non-III data distributions. Besides, FedNova, designed to address objective inconsistency in FL, ensures consistent and accurate model updates ing update frequencies from clients, making it suitable for scenarios with heterogeneous data. The results demonstrate the consistent efficacy of IBA with three datasets and three FL aggregators. In particular, IBA persistently obtained sig- nificant BAs, with the lowest BA being 87.13% 13% and the highest being 99.77%, demonstrating its efficacy and potential challenges in diverse FI. settings.
with vary-
BA MA
Accuracy (%)
50
Fixed-pool attacks. We study the performance of IBA under fixed- fixed-pool attack setting (percentage of attackers in the overall clients varies from 5% to 20%). The results are shown Figure demonstrating that higher compromising ratio leads to better backdoor accuracy more substantial backdoor effect.
0 5 10 15
20 25
Compromising Ratio (%)
Figure 2: IBA under fixed- pool attacks with different compromising ratios.
(2) IBA under different FL defenses. We study the effectiveness of stand- alone IBA and hybrid attacks (i.e., IBA and partial model poisoning) against the aforementioned defense techniques. As demonstrated in
7

---

86
85
Main Accurary (96
84
83
Backdoor Accuracy
60
40-
78
77
100
200
300
400
500
100
200
300
400
500
FL traniin round
FL training round
FoolGGold RFA
Krum FoulsGald RFA
(a) Main Accuracy
(b) Backdoor Accuracy
Figure 3: Effectiveness of stand-alone IBA against mainstream defenses for FL CIFAR-10 10 dataset.
PGD Replacement (BA)
PGD Replacement (MA)
PGD Replacement (BA)
PGD Replacement (MA)
IBA
IBA
Acsracy
201
1.0
300
100
201
(a) Krum
(b) Multi-Krum
(c) FoolsGold
(d) RLR
Figure 4: The improved stealthiness IBA when combined with partial model poisoning attacks under various defenses for the CIFAR-10 10 dataset. The attack is conducted from round O with fixed- frequency attacks of 10.
Figure 3 the stand-alone IBA can circumvent NCA, Multi-Krum, and FoolsGold and attain a high backdoor accuracy (i.e., 95.11%........ %, 79.5% and 76.46% for CIFAR-10 dataset, respectively). Krum is the most challenging defense since replaces the global model with only the most representative local model (i.e., the one with the shortest distance its neighbors) in each round. As result, the global model experiences substantial fluctuation over rounds and does not fully capture the knowledge of all participants, i.c., a significant decline primary task accuracy Figure This mechanism hampers
Table Robustness of IBA and the combination of IBA and partial model poisoning under main- stream backdoor defenses in FL
IBA (Standalone)
IBA PGD
IBA + PGD + MR
Dataset
Defense
MA
BA
MA
BA
MA
BA
RFA
81.97%
98.03%
80.71%
96.7%
81.25%
98.17%
NCA
84.64%
95.11%
84.77%
98.73%
84.21%
99.12%
Krum
81.65%
27.07%
81.59%
50.33%
81.44%
65.98%
CIFAR-10
Multi-krum
82.40%
79.5%
83.78%
97.16%
83.25%
95.02%
RLR
83.91% %
63.94%
82.59%
65.14%
82.85%
47.07%
FoolsGold
84.90%
76.46%
85.02 %
76.78%
82.90%
84.36%
No-defense
84.28%
85.63%
84.35%
83.80%
83.81 %
97.79%
RFA 98.04%
100.0 %
98.89%
99.64%
98.04%
100.0%
NCA 98.66%
100.0% %
98.97%
99.69%
98.62%
100.0%
Krum 97.60%
14.13%
97.82%
87.48%
97.44%
17.29%
MNIST
Multi-krum
97.98%
81.27%
98.97%
98.82%
99.01% %
80.18%
RLR 90.91%
12.60%
87.48%
18.74%
90.03%
29.97%
FoolsGold
98.87%
80.08%
99.10%
96.43%
99.05%
99.83%
No-defense
99.98%
98.03%
98.81
99.73%
98.84%
99.97%

---

9
7
6
1
7
9
9
9
2
9
6
7
9
9 9
9
2
9
6
7
9
9
9
6
9
9
(a) IBA, MNIST
(b) DBA, MNIST
(c) IBA, CIFAR-10
(d) DBA, CIFAR-10
Figure 5: Performance under GradCam heat maps of two attack schemes: IBA and Patched- BA with MNIST and CIFAR-10 10 datasets. The first two rows in each image represent the benign samples, and the last two rows represent the backdoored ones.
the attack model's learning process since it is closely associated with global model variance. Figure illustrates the improved performance of IBA against challenging defenses when combined with partial model poisoning. i.e., WGD-based. As observed, the IBA PGD (w./wo.) model replacement can help improve the backdoor accuracy from 10% to 20%. Significantly, the improved IBA can achieve almost 100% of backdoor accuracy for Multi-Krum defense. The more detailed result can be seen in Table 2 The proposed IBA can bypass most existing defenses and achieve significant backdoor accuracy. In most cases, the combination of IBA + PGD/ IBA PGD MR helps improve the backdoor accuracy from 10% to 20%. While Krum and RLR defend the proposed attack to some extent, i.e. we observe the drop in the BAs of IBA. previously unseen information from both adversary and honest clients, which results in low MAs (a drop of 3-10%). Specifically, the proposed IBA causes RLR compensate for the backdoor defense's 's effectiveness with the main accuracy drop in the MNIST dataset by unnecessarily reversing the learning rate of specific dimensions, i.e., the BAs are around 90% (less than other defenses 10%). Therefore, these two defenses raise great concern about its practicability under this scenario.
argue that Krum and RLR tend to reject
D0A-85 18A-M
ORA-BA
Strige J014
Acurao
100
1...angnggg tound
200
492
FL training round
400
(a) MNIST
(b) CIFAR-10
Figure 6: Durability comparison IBA and DBA with MNIST CIFAR- 10 datasets. The adversary is removed from rounds 200 and 400 training, respectively.
Figure 7: Anomaly Indexes of IBA, DBA backdoor attacks in FL.
Table 3:
The decay rate of DBA and IBA after removing the backdoor attack (i.e., from the stopping round). The column (%) indicates the current backdoor effect as a percentage of its peak value at the stopping round.
Stopping round
50 rounds later
100 rounds later
150 rounds later
200 rounds later
250 rounds later
Dataset
Method
BA
(%)
BA
%
BA
(%)
BA
(%)
BA
(%)
BA
(%)
99.94
100.00
99.52
99.57
99.46
99.52
99.18
99.24
99.08
MNIST
DBA
99.70
100.00
98.44
98.74
91.44
91.71
79.88
80.12
65.04
65.24
38.64
38.76
IBA
87.51
100.00
86.22
98.53
85.32
97.50
84.89
97.00
83.19
95.06
83.16
95.03
CIFAR- 10
DBA
69.63
100.00
65.55
94.14
63.59
91.32
55.39
79.55
55.07
71.91
42.55
61.11
(3) IBA's imperceptibiliity and durability evaluation. Besides being imperceptible to the backdoor defenses, IBA is expected to be visually imperceptible and deliver prolonged backdoor durability. The visual imperceptibility of IBA illustrated in Figure We evaluate the behavior of backdoor-injected models against GradCam [23] a widely known visualization tool and helpful in understanding the neural network's behavior. F shows that the visualization heat maps created by IBA attacks
nearly identical across the clean and backdoor images. From Figures we can observe that the

---

backdoored images have negligible deviations from the Grad-CAM behaviors on the clean images for both triggers generate a considerable difference in visualization heat maps (DBA 32 Because IBA is based on input perturbation, the image is formed by adding very little perturbation to the clean image; consequently, the difference in the latent space where the heat maps are generated is likewise minimal. Aside from that, the patch backdoor (such as the pattern in the upper-left corner) is apparent, resulting in significant distinction between the backdoor and clean versions. We also evaluate the backdoor generated by IBA and another attack scheme in FL, i.e. DBA with Neural Cleanse [27], a widely- used backdoor model mitigation method based on the pattern optimization approach. Neural Cleanse assumes that the backdoor trigger is patch-based which makes it suitable for evaluating the proposed method. For each image label, Neural Cleanse identifies if there is a patch pattern that produces misclassification result for that target label. If any class label yields a significantly smaller pattern, Neural Cleanse considers from the optimal patch of each class label by using the Anomaly Index metric. If the Anomaly Index is less than threshold of 2 for class, Neural Cleanse considers that there is backdoor with this class as the target label. Since IBA does not generate the patch-based trigger pattern, it can efficiently bypass this defense by having Anomaly Index much lower than DBA (Figure 7 which indicates the superior stealthiness of backdoor attacks with ad adversarial triggers compared to the patch-based.
and colored images. On the other hand, backdoored images using patched
sign of potential backdoor. Neural Cleanse quantifies such deviations
As previously stated, strong attacks should be successful even if the attacker only participates few times. As a result, we construct a scenario to assess the endurance of IBA compared to a chosen baseline: DBA. We employ the same configuration as the original work DBA but limit each of the four malicious clients' participation frequency to 10 without utilizing any scaling strategy. Regarding IBA, apply the dimension-base based model poisoning mentioned above and use one malicious client with the same frequency. As observed in Figure. [6a] after A is removed from the training (i.e., at round 200), the BA of DBA reduces gradually while the accuracy drop of IBA is almost negligible. As demonstrated in Figure 6b and Table 3 IBA delivers superior backdoor efficiency and durability within the same attacking duration. After round 400, the BA of DBA drops quickly compared to
IBA. We can see the most significant difference towards the end of the comparison duration, i.c., at round 700, the BAs of IBA and DBA are 85% and ~ 45%, respectively. After 250 rounds of elimination, the BA of DBA retains 38.76% 76% of its peak value. but the comparable value of IBA is 99.14% Furthermore, while the BA can preserve more than 60% of its peak value with the CIFAR-10 10 dataset, the highest accuracy it can attain is significantly lower than IBA, i.e., less than 20%, which is consistent with the prior conclusion of the original when the scaling factor is 1. conclusion, IBA can achieve better durability and imperceptibility than SOTA attacks.
5 Conclusion
In conclusion, we have presented the IBA, novel backdoor attack framework for FL that overcomes the limitations of existing approaches. Our framework combines a learnable trigger-generating func- tion with gradual implantation process, resulting in an attack that can evade detection and achieve high success rates. The selective poisoning of model parameters and the constraint on poisoned updates enhance the attack's 's efficiency and durability. Our work demonstrates promising results and suggests several directions for future research. Robust defense mechanisms against FL backdoor attacks must be developed, considering the unique challenges of the distributed and decentralized learning process. Generalizing our approach to diverse FL settings, such as heterogeneous or non- i..d. data distributions, would provide comprehensive understanding of the attack's 's effectiveness. Exploring alternative triggers and their impact on stealthiness and success rates could yield valuable insights. Evaluating the framework on larger-scale FL systems and real-world scenarios will assess its scalability and practicality. Advancing adversarial defense strategies, including proactive detection and mitigation techniques, will contribute to more secure and trustworthy FL systems. These future research directions will contribute to strengthening the security of FL against backdoor attacks.
10

---

References
[1] Manaar Alam, Esha Sarkar, and Michail Maniatakos. Perdoor: Persistent non-uniform backdoors in federated learning using adversarial perturbations. ArXiv preprint, abs/2205.1352)) 2022.
[2] Eugene Bagdasaryan, Andreas Veit, Yiqing Hua, Deborah Estrin, and Vitaly Shmatikov. How to backdoor federated learning. In Silvia Chiappa and Roberto Calandra, editors, The 23rd International Conference on Artificial Intelligence and Statistics, AISTATS 2020, 26-28 August 2020, Online [Palermo, Sicily, Italy]. volume 108 of Proceedings of Machine Learning Research, pages 2938-222880 2948. PMLR, 2020.
[3] Peva Blanchard, EI Mahdi EI Mhamdi, Rachid Guerraoui, and Julien Stainer. Machine learning with adversaries: Byzantine tolerant gradient descent. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett, editors, Advances in Neural Information Processing Systems 30; Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 119-129, 2017.
[4] Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted backdoor attacks on deep learning systems using data poisoning. ArXiv preprint, abs/1712.00522 2017.
[5] Khoa Doan, Yingjie Lao, and Ping Li. Backdoor attack with imperceptible input and latent modification. In Marc' Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan, editors, Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurlPS 2021, December 6-14, 2021, virtual, pages 18944-18957, 2021.
[6] Khoa Doan, Yingjie Lao, Weijie Zhao, and Ping Li. LIRA: learnable, imperceptible and robust backdoor attacks. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, pages 11946-11566 IEEE, 2021.
[7] Khoa D Doan, Yingjie Lao, and Ping Li. Marksman backdoor: Backdoor attacks with arbitrary target class. In Advances in Neural Information Processing Systems, volume 35, pages 38260- 38273, 2022.
[8] Clement Fung, Chris JM Yoon, and Ivan Beschastnikh. The limitations of federated learning in sybil settings. RAID, pages 301-316, 2020.
[9] Malhar S Jere, Tyler Farnan, and Farinaz Koushanfar. A taxonomy of attacks on federated learning. IEEE Security & Privacy, 19(2)20022, 2020.
[10] Shaofeng Li, Minhui Xue, Benjamin Zi Hao Zhao, Haojin Zhu, and Xinpeng Zhang. Invis- ible backdoor attacks on deep neural networks via steganography and regularization. IEEE Transactions on Dependable and Secure Computing, (8(5):2088-2100 2020.
[11] Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. In Inderjit S. Dhillon, Dimitris S. Papailiopoulos, and Vivienne Sze, editors, Proceedings of Machine Learning and Systems 2020, MLSys 2020, Austin, TX, USA, March 2-4, 2020. mlsys.org. 2020.
[12] Yuezun Li, Yiming Li, Baoyuan Wu, Longkang Li, Ran He, and Siwei Lyu. Invisible backdoor attack with sample-specific triggers. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, pages 16443-16452. IEEE, 2021.
[13] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas. Communication-cfiicn learning of deep networks from decentralized data. In Aarti Singh and Xiaojin (Jerry) Zhu, editors, Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, 20-22 April 2017, Fort Lauderdale, FL, USA, volume 54 of Proceedings of Machine Learning Research, pages 1273-1282. PMLR, 2017.
[14] Milad Nasr, Reza Shokri, and Amir Houmansadr. Comprehensive privacy analysis of deep learning. In Proceedings of the 2019 IEFE Symposium on Security and Privacy (SP), pages 1-15, 2018
11

---

[15] Thien Duc Nguyen, Phillip Rieger, Roberta De Viti, Huili Chen, Bjöm B Brandenburg, Hossein Yalame, Helen Möllering, Hossein Fereidooni, Samuel Marchal, Markus Miettinen, et al. (FLAME)]: Taming backdoors in federated learning. In 31st USENIX Security Symposium (USENIX Security 22). pages 1415-1432, 2022.
[16] Thuy Dung Nguyen, Tuan Nguyen, Phi Le Nguyen, Hieu H Pham, Khoa D Doan, and Kok- Lok-Seng Wong. Backdoor attacks and defenses in federated learning: Survey, challenges and future research directions. Engineering Applications of Artificial Intelligence, 127:107166, 2024.
[17] Tuan Anh Nguyen and Anh Tuan Tran. Wanet - imperceptiee warping-based backdoor attack. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
[18] Mustafa Safa Özdayi, Murat Kantarcioglu, and Yulia R. Gel. Defending against backdoors in federated learning with robust learning rate. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intel- ligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence. EAAI 2021, Virtual Event, February 2-9, 2021, pages 9268-9276 AAAI Press, 2021.
[19] Krishna Pillutla, Sham M Kakade, and Zaid Harchaoui. Robust aggregation for federated learning. IEEE Transactions on Signal Processing, 70:1 142-1144,, 2022.
[20] Nuria Rodríguez-BBrroso, Daniel Jiménez-LLópez, M Victoria Luzón, Francisco Herrera, and Eugenio Martinez-Cámara. Survey federated learning threats: Concepts, taxonomy on attacks and defences, experimental study and challenges. Information Fusion, 90:148-173, 2023.
[21] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted DnteventioonMIICAA 2015: 18th International Conference, Munich, Germany, October 5.9, 2015, Proceedings, Part III 18, pages 234-241 Springer, 2015.
[22] Aniruddha Saha, Akshayvarun Subramanya, and Hamed Pirsiavash. Hidden trigger backdoor attacks. In The Thirty-Fourd AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020, pages 1 11957-11965. AAAI Press, 2020.
[23] Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das. Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017, pages 61 618-626. IEEE Computer Society, 2017.
[24] Virat Shejwalkar, Amir Houmansadr, Peter Kairouz, and Daniel Ramage. Back to the drawing board: A critical evaluation of poisoning attacks on production federated learning. 2022 IEEE Symposium on Security and Privacy (SP), pages 1354-1374 IEEE, 2022.
[25] Ziteng Sun, Peter Kairouz, Ananda Theertha Suresh, and H. B. McMahan. Can you really backdoor federated learning? ArXiv preprint, abs/1911.07996 2019.
[26] Alexander Turner, Dimitris Tsipras, and Aleksander Madry. Label-consistent backdoor attacks. ArXiv preprint,
2019.
[27] Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Y. Zhao. Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. 2019 IEEE Symposium on Security and Privacy (SP), pages 707-723, 2019.
[28] Hongyi Wang, Kartik Sreenivasan, Shashank Rajput, Harit Vishwakarma, Saurabh Agarwal, Jy-yong Sohn, Kangwook Lee, and Dimitris S. Papailiopoolos. Attack of the tails: Yes, you really can backdoor federated learning. Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NewrlPS 2020, December 6- 62, 2020, virtual, 2020.
12

---

[29] Jianyu Wang, Qinghua Liu, Hao Liang, Gauri Joshi, and H. Vincent Poor. Tackling the ob- jective inconsistency problem heterogeneous federated optimization. Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin, editors, Ad- vances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-1 12, 2020, virtual, 2020.
[30] Chen Wu, Xian Yang, Sencun Zhu, and Prasenjit Mitra. Mitigating backdoor attacks in federated learning. ArXiv preprint, abs/2011.01767, 2020.
[31] Chulin Xie, Minghao Chen, Pin-Yu Chen, and Bo Li. CRFL: certifiably robust federated learning against backdoor attacks. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pages 11372-11333 11382. PMLR, 2021.
[32] Chulin Xie, Keli Huang, Pin- Yu Chen, and Bo Li. DBA: distributed backdoor attacks against federated learning. In 8th International Conference on Leurning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020.
[33] Jiale Zhang, Junjun Chen, Di Wu, Bing Chen, and Shui Yu. Poisoning attack in federated learning using generative adversarial nets. In 2019 18th IEEE International Conference On Trust, Security And Privacy In Computing And Communications/I1 3th IEEE International Conference On Big Data Science And Engineering (TrustCom/BigDattaE)) pages 374-380. IEEE, 2019.
[34] Zhengming Zhang, Ashwinee Panda, Linyue Song, Yaoqing Yang, Michael W. Mahoney, Prateek Mittal, Kannan Ramchandran, and Joseph Gonzalez. Neurotoxin: Durable backdoors in federated learning. In Kamalika Chaudhuri. Stefanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato, editors, International Conference on Machine Learning. ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA. volume 162 of Proceedings of Machine Learning Research, pages 16429-26446, PMLR, 2022.
[35] Shihao Zhao, Xingjun Ma, Xiang Zheng, James Bailey, Jingjing Chen, and Yu-Gang Jiang. Clean-label backdoor attacks on video recognition models. In 2020 JEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pages 14431-14440 IEEE, 2020.
13


Reviews:
[{"review_id": "6MxcFPpRyf", "reviewer": "Reviewer_gShs", "summary": "**Key Contribution:** This work proposes a two-staged model poisoning attack which works even with only the participation of a small number of malicious clients. Moreover, the proposed method is effective and robust against existing defenses.\nThe attack is evaluated on a variety of existing defenses on CIFAR-10, MNIST, and Tiny-Imagenet.\n", "questions": "- Have you considered different choices of trigger generation models?\n\n- What was the distribution of data considered across the clients during evaluation? Have you considered evaluating with non-iid data?\n", "limitations": "The authors have adequately discussed limitations and potential social impacts of their work.", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "**Novelty and Significance**\nThe paper’s two-stage approach is unique and powerful. Learning a generative model for adversarial noise is additionally alarming as it is difficult to detect and defend against. \n\n**Clarity:**\nThe paper is well-written and clear - the mathematical formulations are simple and easy to follow, with intuitive results. Moreover, the proposed augmentations to the base attack which help evade existing defenses are methodical and clearly motivated.\n", "weaknesses": "**Limited Evaluation:** The paper could be stronger by presenting more results under different numbers of clients. However, despite this weakness, I am overall satisfied with the quality and comprehensiveness of the evaluation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your our time and effort reviewing the paper. We appreciate the review and comments on the paper. We would like to address the concerns as follows.\n\n1. **Results under different number of clients**\n    \n    The standard settings are inherited from other works [1, 31], which are 10 clients. In this work, we consider the case when the number of clients increases and decreases. The performance of IBA is consistent under different number of participating clients (5 and 20 clients).\n    \n    For performance of IBA (stand-alone) under different numbers of participating clients K over 500 rounds, please refer to Figure 1 \n    from in this [**pdf**](https://files.fm/f/r67m5e7a3) file.\n\n2. **Choices of trigger generation models**\n\n    In general, the generator function can be modeled as an autoencoder or the more complex U-Net architecture [18]. However, we observe a negligible performance difference between using an autoencoder and U-Net. We conjecture that since learning to generate trigger noises is a much simpler task than learning to generate images, using the simpler autoencoder is sufficient. Furthermore, training and performing inference with the autoencoder have lower computational overhead. For these reasons, we employ the autoencoder in all experiments in our paper. We will add this discussion to a later version.\n\n3. **Data distribution across the clients during the evaluation**\n\n    FL often presumes non-i.i.d. data distribution across parties. Here, we use a Dirichlet distribution (Minka, 2000) with different hyperparameter $\\alpha$ to generate different data distribution following the setups in (Bagdasaryan et al., 2018). Specifically, we follow an established evaluation protocol in previous works [26, 31] and simulate heterogeneous data partitioning by sampling $p_k \\sim \\text{Dir}_K(0.5)/\\text{Dir}_K(0.01)$ for MNIST, CIFAR-10/Tiny-Imagenet and allocating a proportion of each class to participating clients.\n\n[1] Bagdasaryan, E., Veit, A., Hua, Y., Estrin, D., & Shmatikov, V. (2018). How to backdoor federated learning. arXiv preprint arXiv:1807.00459.\n\n[18] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Nassir Navab, Joachim Hornegger, William M. Wells, and Alejandro F. Frangi, editors, Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, pages 234–241, Cham, 2015. Springer International Publishing.\n\n[26] Hongyi Wang, Kartik Sreenivasan, Shashank Rajput, Harit Vishwakarma, Saurabh Agarwal, Jy-yong Sohn, Kangwook Lee, and Dimitris Papailiopoulos. Attack of the tails: Yes, you really can backdoor federated learning. Advances in Neural Information Processing Systems, 33:16070–16084, 2020.\n\n[31] Chulin Xie, Keli Huang, Pin-Yu Chen, and Bo Li. Dba: Distributed backdoor attacks against federated learning. In International conference on learning representations, 2020.\n"}, {"review_id": "OCgx1q52PN", "reviewer": "Reviewer_S7Me", "summary": "The authors propose a backdoor attack framework (IBA) in Federated Learning for trigger based backdoors that jointly learns a generative model for stealthy visual triggers while also planting the backdoor in the global model. They evaluate their attack on MINST, CIFAR-10, and TinyImageNet and show that it bypasses several known defenses such as KRUM, NC, RFA etc.", "questions": "1. In (2), the adversarial perturbation is bounded in the $\\lVert \\cdot \\rVert_{\\infty}$ norm. Can this be translated to the 2-norm? Naively, speaking the same attack budget will get scaled down by $\\sqrt{d}$, which will reduce the effectiveness of the attack. Have you tried this?\n2. It is quite interesting that the optimal choice of scaling factors is $\\alpha = \\beta = 0.5$. Are the scales of the two losses very similar?\n3. How is $\\lambda_{\\xi}$ chosen?\n4. Please exwhat plain acronyms in the table captions. Such as MA, BA.\n5. Please explain bold entries in the tables mean. It can be inferred, but would be nice not to have it.\n6. Section 4.2 is quite hard to understand especially since accuracies are listed as \"ranging\" across datasets. This is quite unusual and doesn't really make sense. \n7. What is MR?\n8 . It is hard to evaluate the performance of IBA from Table 2 without comparing against other attacks.\n9. It seems like for some defenses (like KRUM), IBA + PGD outperforms IBA + PGD + MR. Whereas in RLR, it gets worse. Can you explain this behavior? Is one supposed to choose the attack based on a defense?\n10. In Figure 3, is the attack at round 0?\n11. Can you specify the choice of $\\varepsilon$ for the PGD attack?\n12. Some citations are duplicated (McMahan et al., Wang et al)", "limitations": "I would strongly recommend the authors include a section listing the limitations of their work (comparison with more baselines, longevity of attack, need for whitebox access etc)", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The experimental setup is quite thorough, considering several datasets, settings, and defenses.\n2. The attack appears to bypass all the considered defenses with good effectiveness and stealth.\n3. The work acknowledges past work well with a good description of existing research.", "weaknesses": "1. While the attack is titled \"Irreversible Backdoor attack\", the longevity of the attack is not well studied. Figure 5 is only for the MNIST dataset and most other experiments consider fixed-frequency attack. I would like to see a better analysis of how long the attack takes to be removed by clean training.\n2. I would like to see a comparison to even trigger-less backdoor attacks since the PGD model replacement attack has already been well studied in Bagdasaryan et al. and Wang et al. I would expect this to be much stronger, but it would act as a good baseline. While there are several ablations, I would like to see comparison to more existing baselines. Perhaps even BadNets [Gu et al.].", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the valuable comments. Additional experiments are at [**EXP**](https://files.fm/f/jh4hy9pg3).\n\n1. **analysis of longevity**\n\nAs presented in Fig 5 (main), Fig 9 (sup.), and Tab 4 (sup.), our method can achieve extended longevity compared to DBA for both MNIST and CIFAR-10. We also conducted additional experiments to compare the durability of IBA, 3DFed [1], and Edge-case [2626], and ours still outperformed the others w.r.t durability (results in Fig 2 of EXP). We leverage the fixed-frequency attack for a fair evaluation of the methods under a low participation rate (i.e., attack frequency is 10 rounds). \n\n2. **Compare to more baselines**\n\nWe provided additional study on the durability of trigger-less backdoor attacks (i.e., the edge-case at Wang et al.) to show that our proposed attack IBA has a longer lifespan (Fig 5 in the provided file).\n\nSince patched triggers used in DBA are relevant to the methodology employed in BadNets, we selected one more SOTA, i.e., 3DFed, to compare with ours in terms of durability. IBA outperformed both 3DFed and DBA, which exhibited a gradual accuracy decline beyond round 450. For further durability evaluation over training rounds, please see Fig 4 and 5 in  `Extended Durability Evaluation` of EXP.\n\n\n3. **Perturbation using $\\|.\\|_{\\infty}$ Norm**\n\nWhile considering various norms for bounding the generator noise, such as the $L_2$ norm, it's generally inadvisable for backdoor attacks. This infinity norm guarantees a widespread distribution of the generated trigger across the input image (encompassing all pixels in the trigger); in contrast, the $L_2$ norm can result in localized artifacts within the image (with only select pixels forming the trigger). In simpler terms, employing the $L_2$ norm can make the backdoor attack more susceptible to detection by trigger-synthesis defenses like Neural Cleanse. Thus, the $\\|.\\|_{\\infty}$ norm stands as a preferred choice. We will enhance the clarity of this aspect.\n\n4. **Optimal choice of scaling factors**\n\nAs discussed in Sec 3.2 (lines 165-169), empirically, if $\\alpha$ is significantly higher than $\\beta$, the classifier's performance on clean data rapidly converges to the optimal performance of the vanilla classifier; on the other hand, if $\\beta$ is significantly higher than $\\alpha$, the classifier's performance on backdoor data quickly reaches the optimal value. Thus, to balance the learning processes of both the main and backdoor tasks, we set $\\alpha = \\beta = 0.5$ in our experiments.\n\n5. **How is $\\lambda_{\\xi}$ chosen?**\n\n$\\lambda_{\\xi}$ is the decay rate appearing in the formula (4) that controls how fast the reduction by round is. The larger $\\lambda_{\\xi}$ is, the faster the backdoor images become stealthy and the harder it is for the generative model to learn to perform well (may overfit with the local data). Therefore, our empirical results suggest to fix the value of $\\lambda_{\\xi}$ to be 0.001 to balance out the two factors mentioned above. Our suggestion is that $(1 - \\lambda_{\\xi})$ should be set in the range $[1\\times (1-\\gamma); 3\\times (1-\\gamma)]$, where $\\gamma$ is the decay factor of learning rate, i.e., in our inherited setup, $\\gamma$ is set to be 0.998.\n\n6. **Acronyms in the table captions**\n\nAs provided in Sec 4.2, MA and BA stand for Main Accuracy and Backdoor Accuracy, respectively. The bolded entries in the table indicate the best MA/BA achieved by each method for a given dataset. MR stands for Model Replacement [1]. We will make the adjustments accordingly.\n\n7. **Unusual writing (`ranging` word)**\n\nHere, we aim to show that the MAs of the poisoned models are similar to those of the corresponding clean models while their BAs are much higher under our attack.\n\n8. **evaluate the IBA performance, Tab 2, with other attacks**\n\nWe focus on providing a comprehensive empirical analysis of the proposed attack, including the properties of the designed trigger function, the attack’s stealthiness and durability,  (main concerns of related federated backdoor-research works). Therefore, regarding evaluating efficiency and stealthiness, we refer to the experimental design in prior works [31, 26, 1] to understand the performance of IBA under different attack schemes (fixed-frequency vs. fixed-pool) and mainstream backdoor defenses. The experiment shows that IBA can obtain equivalent BAs as other backdoor attacks, i.e., >90% for 3 datasets, while ensuring its stealthiness in terms of visualization and defense bypassing and extended durability.\n\n9. **Behavior of IBA+PGD/ IBA+PGD+MR**\n\nIn most cases, the combination of IBA + PGD/ IBA + PGD + MR helps improve the backdoor accuracy from 10% to 20%. However, Krum and RLR are the most challenging to circumvent. For Krum, MR seems not to work well under the MNIST dataset since the poisoned model is scaled before being submitted, and Krum chooses the model with the smallest Euclidean distance to its neighbors as the new global model. \n\nFor RLR, the behavior of attacks is difficult to predict. At some suspicious dimensions of the whole model parameter, the learning rate will be reversed. If the poisoned model is scaled (by MR) and then reversed, the global model will move farther from the optimal point of the backdoor task. \n\nNevertheless, IBA can still be effective against Krum and RLR.\n\n10. **Attack at round 0 in Fig 3**\n\nFig 3's attack is at round 0. Here,  with fixed-frequency attacks of 10, the attacker participates in round 0, 10, 20... \n\n11. **Choice of $\\epsilon$ for PGD**\n\nThe norm-bound for PGD attack should be selected based on the observation of the $L_2$-norm variation of the global model by round. Following the work [1] and [26], the norm bound is set to be 2 for the CIFAR-10/Tiny-Imagenet dataset and 1.5 for the MNIST dataset. We will add this to the updated version.\n\n12. **Duplicate citations**\n\nWe will revise this section. "}, {"review_id": "rYSggDDeih", "reviewer": "Reviewer_yF1b", "summary": "This paper studies backdoor attacks in federated learning setting. They propose a new backdoor attack method that is based on sample-specific trigger, optimized with constraints on weight norm and weight dimension, to achieve more stealthy, harder to detect and more durable. U-Net is trained to generate sample-specific trigger and the stealthiness of the trigger is controlled by $\\epsilon$, which also controls the balance between attack success and trigger stealthiness. To make the attack harder to detect and more durable, the gradients are projected to the ball around the global model weights and optimization is constrained to dimensions that are historically infrequently updated. Experiments on CIFAR10, MNIST and TinyImageNet show that the proposed method is effective against multiple state-of-the-art defense methods.", "questions": "See questions and concerns in the previous part.", "limitations": "The main limitation is the lack of experiments to support the claim. See details in weakness part.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The paper is well organized and the ideas are clearly presented. \n\n- Experiments show that the method can defend against multiple state-of-the-art defense methods.", "weaknesses": "- Even though the proposed method is compared with DBA to show its durability, the paper lacks comparison with other state-of-the-art attack methods. Experiments comparing multiple attack methods against state-of-the-art defense methods on different types of datasets should be conducted to show the effectiveness of the method. \n\n- As for the poisoning dimension restriction, the idea of restricting updates to dimensions that are historically seldomly updated is similar to that of Neurotoxin. Why compare with DBA which does nothing to promote the durability of the attack? Why not compare with Neurotoxin? \n\n-  Generally speaking, the data distribution of clients may affect the performance of attack and defense. Dir(0.5) and Dir(0.01) are used for MNIST, CIFAR10/TinyImagenet. Experiments studying the effect of data distribution could be helpful understand the method better.\n\n- Based on Figure 2 and table 2, it seems that the method does not do a good job against Krum and RLR. The attack accuracy of the method keeps dropping when tested against the two defense methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the comments and suggestions from the reviewer. The following is our response to the raised concerns.\n\n\n1. **Comparison with other state-of-the-art attack methods**\n\n\nAs mentioned, our objective is to design a backdoor attack with efficiency, stealthiness, and durability under more practical assumptions. Therefore, regarding evaluating efficiency and stealthiness, we refer to the experimental design in prior works [1, 26, 31] to understand the performance of IBA under different attack schemes (fixed-frequency vs. fixed-pool) and mainstream backdoor defenses. With respect to durability, we selected DBA (a SOTA backdoor attack in FL) and configured it such that two attacks are under the same participation frequency, and we observed the behavior of backdoor accuracy after the adversary is removed. Based on the two main designs above, we want to draw the conclusion that IBA can achieve equivalent results to other attacks such as Edge-case [26], DBA [31] under practical assumptions but is stealthier in terms of visual representation and more durable. To further demonstrate the superior durability of the proposed IBA, we compared it with 3DFed [34], a new SOTA backdoor attack in FL and proved that IBA can bring much better durability and performance.\n\n\n2. **Comparison with Neurotoxin**\n\n\nWe conducted additional experiments to compare the durability of the proposed attacks with Neurotoxin.\n- IBA + proposed model poisoning (Ours) (1)\n- DBA + Neurotoxin (2)\n- Centralized backdoor attacks + Neurotoxin (3)\n\n\nAs observed, the combination of DBA and Neurotoxin (2) does not have a good effect on extending the backdoor effect. In specific, the BA reduces gradually after round 600, at which point the adversaries leave the training. Moreover, this combination even brings down the main task's accuracy.\n\n\nWe consider the standard centralized backdoor attack [31, 1] combined with Neurotoxin (3) under the same setting as our method (1 attacker participating in each 10 rounds). In the experiment with Neurotoxin, the adversary uses a patched trigger to create the backdoor attack.\nFrom the result, our proposed model poisoning method brings better durability compared to the original Neurotoxin, i.e., after round 400 at which the adversary leaves the training, the IBA+Neurotoxin’s BA drops more quickly.\n\nFor comparison of IBA and Neurotoxin over FL training rounds, please refer to Figure 3 in `Extended Durability Evaluation` section in this [**pdf**](https://files.fm/f/r67m5e7a3) file.\n\n\n3. **Effect of data distribution**\n\n\nWe consider different value of alpha (0.2, 0.5, 0.7) in the Dirichlet distribution to simulate from non-i.i.d to i.i.d distributions for the image datasets. When evaluated under 2 different datasets, the result shows that IBA-BA is stable under various distributions, which exhibits the practicability and robustness of IBA when attacking standard FL. The table below shows the results of IBA-BA under different alpha values in the Dirichlet distribution.\n\n\n**Main Accuracy** (MA) of IBA under different alpha values in the Dirichlet distribution.\n\n\n| Dataset | Alpha = 0.2 | Alpha = 0.5 | Alpha = 0.7 |\n| --- | --- | --- | --- |\n| MNIST | 99.01% | 99.98% | 98.99% |\n| CIFAR-10 | 82.73% | 84.28% | 84.53% |\n\n\n**Backdoor Accuracy** (BA) of IBA under different alpha values in the Dirichlet distribution.\n\n\n| Dataset | Alpha = 0.2 | Alpha = 0.5 | Alpha = 0.7 |\n| --- | --- | --- | --- |\n| MNIST | 98.89% | 98.03% | 98.81% |\n| CIFAR-10 | 83.26% | 85.63% | 88.93% |\n\n\n4. **Effectiveness against Krum and RLR**\n\n\nWhile Krum and RLR defend the proposed attack to some extent, i.e., we observe the drop in the BAs of IBA, we argue that Krum and RLR tend to reject previously unseen information from both adversary and honest clients, which results in low MAs (a drop of 3-5%). Therefore, these two defenses raise a concern for their aplicability under the proposed attacks.\n"}, {"review_id": "PuMDoh07sk", "reviewer": "Reviewer_Avr8", "summary": "The paper introduces a two-phased backdoor injection framework, called IBA, for Federated Learning systems. IBA incorporates an adaptive trigger generation mechanism along with a gradual implantation process to insert stealthy backdoors into the global model. IBA enhances the efficiency and durability of the attack through selective poisoning of specific model parameters. Through evaluation using multiple datasets, IBA demonstrates high success rates and outperforms existing backdoor injection methods even in the presence of several defense techniques.", "questions": "Please address the issues discussed in Weaknesses.", "limitations": "The authors have discussed the limitations and potential negative societal impacts of the paper. However, the paper has several other limitations, as discussed in the review.", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "The paper is well-written and easy to follow. It addresses a crucial problem in federated learning.", "weaknesses": "The paper has several limitations which are discussed below:\n\nThe primary contribution of the paper is to use adversarial examples for generating adaptive backdoor triggers and selective parameter poisoning for injecting durable backdoors. However, a quick search resulted in the following two papers:\n\n[1] M. Alam et al., \"PerDoor: Persistent Non-Uniform Backdoors in Federated Learning using Adversarial Perturbations\", arXiv 2022.\n\n[2] Z. Zhang et al., \"Neurotoxin: Durable Backdoors in Federated Learning\", ICML 2022.\n\nIt is evident from these papers that [1] uses adversarial examples as backdoor triggers, and [2] uses selective parameter poisoning for durable backdoors. On top of that, [2] performs similar operations as the authors do to increase the durability of backdoors. The authors should elaborate on the distinctiveness and novelty of their approach compared to these preceding works.\n\nIBA utilizes adversarial examples as backdoor triggers, which are known for their input specificity. The paper could benefit from a more comprehensive explanation of how IBA addresses this specificity—questions such as whether triggers are unique to each input or shared across a class. Also, it needs to be elaborated more on whether the trigger generation function must be trained at each training round. If so, what is the computational overhead for that?\n\nIBA is not evaluated against important state-of-the-art defenses like FLAME[3], SparseFed[4], etc.\n\nIBA should be compared against other state-of-the-art attacks like 3DFed[5].\n\nIBA should be evaluated using more practical benchmark federated learning datasets in the LEAF project[6].\n\nThe evaluation of IBA considering the fixed-pool case should be included as the main content of the paper.\n\nThe reported accuracy of 98.09% on the CIFAR-10 dataset and the subsequent 14% drop in accuracy in Table 1 raise skepticism, especially in comparison to the marginal drop observed for T-Imagenet. The authors should provide clarification and insights into this discrepancy. In addition, there appears to be a mismatch between the content of Table 1 and its explanation in Section 4.2 (1). The authors need to ensure consistency and clarity in the representation and discussion of results.\n\nThe authors should provide a detailed explanation for specific results, such as why the combination of IBA and PGD yields an accuracy of 18.74% for MNIST and RLR in Table 2 and other similar cases. With such low accuracy, why is IBA deemed to bypass every defense mechanism?\n\nThe authors mention that \"This trigger is expected to bypass human and machine inspections\" but fail to discuss the metrics or criteria used for this assessment.\n\nFigure 1 lacks an adequate explanatory description. A more thorough discussion will enhance the clarity and comprehensibility of the figure.\n\nTo provide more insights, it would be beneficial to depict results on more complex color images in Figure 4 instead of using a monochromatic MNIST dataset.\n\nThe authors should discuss the significance of using FedProx and FedNova, and clearly describe how these differ from FedAvg. This would help in understanding the rationale behind their inclusion.\n\nWhat does the anomaly index represent in Figure 6?\n\nNeuralCleanse is mentioned as a defense mechanism against backdoor attacks in FL, but it was not originally designed for the FL framework. Clarifying its relevance and applicability in this context would be beneficial.\n\n[3] T. Nguyen et al., \"FLAME: Taming Backdoors in Federated Learning\", Usenix Security 2022.\n\n[4] A Panda et al., \"SparseFed: Mitigating Model Poisoning Attacks in Federated Learning with Sparsification\", AISTATS 2022.\n\n[5] H. Li et al., \"3DFed: Adaptive and Extensible Framework for Covert Backdoor Attack in Federated Learning\", IEEE S&P 2023.\n\n[6] https://leaf.cmu.edu/", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the valuable comments. Additional experiments are at [**EXP**](https://files.fm/f/r67m5e7a3).\n\n1. **Compare with PerDoor and Neurotoxin**\n\nPerDoor uses the Basic Iterative Method to generate the trigger and relies on gradients of the loss w.r.t input. Alternatively, we learn a generative model to generate the trigger that blends well with the underlying data distribution. Moreover, the collaborative clients learn their own generative models but aggregate them into one shared model in the next rounds. PerDoor lacks this sharing mechanism, which makes the attack more effective and durable.\n\nWhile the idea of gradient masking is explored in~\\cite[36, 37], our usage of historical information to determine the gradient mask is novel. Our use of a historical gradient mask helps reduce the bad effect of the fluctuation of the global model on finding the optimal gradient mask to poison, especially for non-iid clients with skewed data distributions (may temporarily distort the global behavior). The additional experiments in EXP show the superior performance of IBA's poisoning compared to neurotoxin.\n\n2. **Computation overhead**\n\nAs presented, $T(x) = x+G(x)$, $\\|G(x)\\|$, $x$, for each input $x$, and G learns to generate an input-aware trigger. However, G is updated after the local model’s training is completed, which does not cause a delay in local model submission. In other words, the computational overhead can be easily alleviated in a different thread or outside of FL training.\n\n3. **Evaluation against FLAME/SparseFed**\n\nWe performed additional experiments with the FLAME in EXP, and show that even stand-alone IBA can get around it. IBA achieves 99% BA on MNIST for both cases of K=5 and K=20. Our considered threat model is different from that of  FedSparse, where the server does not send the identical global model to each device.\n\n4. **Comparison with 3DFed**\n\nWe tested IBA against the SOTA 3DFed [5]. We evaluated the method's durability post-attack over 200 rounds (see Figure 3 of EXP). Notably, after 50 epochs and reaching round 250, 3DFed model's ability to learn backdoor tasks deteriorated significantly. Accuracy dropped from 85.98% to 9.69% on MNIST and from 75.92% to 6.06% on CIFAR-10. However, IBA still has superior resilience. Even at round 450, accuracy remained below 1% for MNIST and 5% for CIFAR-10. IBA outperformed both 3DFed and DBA, which exhibited a gradual accuracy decline beyond round 450. \n\n5. **Evaluation with LEAF**\n\nWe employed GitHub's \"Attack of the Tails,\" as the foundation for implementing and evaluating IBA, to ensure a reliable evaluation within an established framework. However, assessing IBA in more FL datasets and benchmark is an interesting future direction of our work, as suggested.\n\n6. **Evaluation of IBA considering the fixed-pool case**\n\nWe will revise this part in the later version, as suggested.\n\n7. **Accuracy drop on CIFAR**\n\nWe confirm that the MA of CIFAR-10 (baseline case w/o. attack) should be 84.71% (a typo in the current version). We will revise it accordingly.\n\n8. **Explanation for specific results**\n\nWhile Krum and RLR defend the proposed attack to some extent, i.e., we observe the drop in the BAs of IBA, we argue that Krum and RLR tend to reject previously unseen information from both adversary and honest clients, which results in low MAs (a drop of 3–10%). Specifically, the proposed IBA causes RLR to compensate for the backdoor defense’s effectiveness with the main accuracy drop in the MNIST dataset by unnecessarily reversing the learning rate of specific dimensions, i.e., the BAs are around 90% (less than other defenses 10%). Therefore, these two defenses raise concerns about their practicality.\n\n9. **Metrics for assessing bypassing human inspections**\n\nWe follow WaNet to perform a similar human evaluation experiment: we tell the testers (a cohort of 50) the mechanism of the attack (e.g., how the trigger is created), present to a tester a clean image and its corresponding backdoor image (with the trigger) without revealing the images’ identities, and ask them to identify the backdoor image. The result shows that the testers cannot distinguish which images are clean and backdoor, suggesting that our trigger is imperceptible against human inspection.\n\n10. **Explanation for Fig 1**\n\nWe added one paragraph to explain Fig 1 in Sec 3: Irreversible Backdoor Attacks (IBA) in FL. In the trigger-generating stage, IBA trains a generative trigger model. In the second stage, the attacker joins the FL process and trains a backdoored model with the triggers. The loss function of IBA is the combination of the loss function of the backdoored data and the loss function of the benign data. The objective of the attacker is to maximize the accuracy of the backdoored model on the training set over the threshold (i.e., 0.85).\n\n11. **Results on color images**\nWe included include the GradCAM analysis for CIFAR-10 (in EXP). We can observe that the backdoored images have negligible deviations from the Grad-CAM behaviors on the clean images. On the other hand, backdoored images using patched triggers generate a considerable difference in visualization heat maps (DBA [31]). \n\n12. **using FedProx and FedNova**\n\nWe will discuss FedProx and FedNova for future extensions of our work in the later version. Evaluating IBA against FedProx and FedNova is an interesting extension of our work.\n\n13, 14. **Anomaly index in Figure 6 and NeuralCleanse in FL**\n\nNC reverse-engineers the assumed patch-based trigger and uses the Anomaly Index to identify an anomalous trigger that is smaller than the rest. If the Anomaly Index < 2 for a class, NC tags the model as backdoor with this class as the target label. NC's Index on IBA is less than 2 or the benign model in T-Imagenet. This means that NC is not effective against IBA.\n\n[36] Zhou et al. \"Deep model poisoning attack on federated learning.\" \n\n[37] Zhang et al. \"Neurotoxin: Durable backdoors in federated learning.\" "}]

Author rebuttal:
Thank you the reviewers for the initial comments and questions. We provide the additional experimental evaluations to address the comments of the authors in this [**EXP**](https://files.fm/f/r67m5e7a3) file, including the following experiments:

- Performance when varying the number of Participating Clients
- Extended Longevity/Durability of IBA (including 3DFed).
- Comparisons with Neurotoxin, Edge-case.
- Grad-cam on colored images (CIFAR10).

In the separate responses to each reviewer, we will use this EXP file as a main reference for any additional experiments.

Meta-review: The authors propose a novel, two-staged poisoning attack on federated learning that can bypass current defensive mechanisms. The authors have addressed most of the negative concerns in the rebuttal phase, and they are invited to include the additional results reported in the rebuttal in the final version of the manuscript.
Decision: Accept (poster)