Trusted Aggregation (TAG): Model Filtering Backdoor Defense In Federated Learning
Authors: Joseph Lavond
Minhao Cheng
Yao Li
Venue: Submitted to ICLR 2023 (2023)
Keywords: federated learning
backdoor attack
robust aggregation

Abstract: Federated Learning is a framework for training machine learning models from multiple local data sets without access to the data in aggregate. A shared model is jointly learned through an interactive process between server and clients that combines locally learned model gradients or weights. However, the lack of data transparency naturally raises concerns about model security. Recently, several state-of-the-art backdoor attacks have been proposed, which achieve high attack success rates while simultaneously being difficult to detect, leading to compromised federated learning models. In this paper, motivated by differences in the output layer distribution between models trained with and without the presence of backdoor attacks, we propose a defense method that can prevent backdoor attacks from influencing the model while maintaining the accuracy of the original classification task. TAG leverages a small validation data set to estimate the largest change that a benign user's local training can make to the output layer of the shared model, which can be used as a cutoff for returning user models. Experimental results on multiple data sets show that TAG defends against backdoor attacks even when 40\% of the user submissions to update the shared model are malicious.

Under review as a conference paper at ICLR 2023
TRUSTED AGGREGATION (TAG): MODEL FILTERING BACKDOOR DEFENSE IN FEDERATED LEARNING
Anonymous authors Paper under double-blind review
ABSTRACT
Federated learning is a framework for training machine learning models from multiple local data sets without access to the data in aggregate. A shared model is jointly learned through an interactive process between server and clients that combines locally learned model gradients or weights. However, the lack of data transparency naturally raises concerns about model security. Recently, several state-of- -fetth-rrttt backdoor attacks have been proposed, which achieve high attack success rates while simultameously being difficult to detect, leading to compromised federated learning models. In this paper, motivated by differences in the outputs of models trained with and without the presence of backdoor attacks, we propose a defense method that can prevent backdoor attacks from influencing the model while maintaining the accuracy of the original classification task. TAG leverages a small validation data set to estimate the largest change that a benign user's 's local training can make to the output layer of the shared model, which can be used as a cutoff for returning user models. Experimental results on multiple data sets show that TAG defends against backdoor attacks even when 40% of the user submissions to update the shared model are malicious.
INTRODUCTION
Federated learning (FL) a potential solution constructing a machine learning model from several local data sources that cannot be exchanged or aggregated. As mentioned in Mahlool & Abed (2022), these restrictions are essential in areas where data privacy or security is critical, including but not limited to healthcare. Also, FL. is valuable for companies that shift computing workloads to local devices. Furthermore, these local data sets are not required be independent and identically distributed. Hence, a shared robust global model is desirable and, in many cases, cannot be produced without some form of collaborative learning. Under the FL setting, local entities (clients) submit their locally learned model gradients and weights to be intelligently combined by some centralized entity (server) to create a shared and robust machine learning model.
Concerns have arisen that the lack of control knowledge regarding the local training procedure could allow a user, with malicious intent, to create an update that compromises the global model for all participating clients. An example of such harm is backdoor attack, where the malicious users try to get the global model to associate a given manipulation of the input data, known as a trigger, with a particular outcome. Some methods (Kurita et al. 2020 Qi al. 2020: Li et al.) (021)) have been proposed to detect the triggers in the training data to defend against backdoor attacks. However, in FL, as only the resulting model gradients or weights are communicated back, such methods cannot be applied to defend against backdoor attacks. Furthermore, since the model update in FL assumes no access all clients' data, there less information available help detect and prevent such malicious intent. Thus backdoor attacks may be easier to perform and harder to detect in FL..
In this paper, we first observe that the output distributions of models created by malicious users are very different from that of benign users. Specifically, there exists a discernible difference between malicious and benign user distributions for the target label class. Therefore, we can leverage this difference to detect backdoor attacks. Figure] | shows clear difference between models trained with and without a backdoor attack in the output scores for the target class on clean data. Therefore using small clean data set, the centralized server produce backdoor-free free locally trained model, which we will refer to as the trusted user. Another candidate user model can be compared on this small clean data set and excluded if they have unusual outputs.

---

Under review as a conference paper at ICLR 2023
Motivated by the finding that the output distributions of a model with and without backdoor different, we propose comparing user and trusted models by the distributional difference between their outputs and the most recent global model to identify malicious updates. We use the trusted user to estimate the most considerable distributional difference benign user's 's local training could produce and eliminate returning models that exceed this distance cutoff. This proposed method is effective against multiple statc-of-the-art backdoor attacks at different strength levels. Even in the unreasonable setting where 40% of the clients are malicious for each update, our proposed method can achieve similar model performance and eliminate backdoor attacks, which dramatically outperforms current alternative methods. In the experiment section, we demonstrate our method's ability on several data sets to prevent backdoor attacks. Additionally, the method performs well even when the attack happens every round and starts at the beginning of the federated learning process. Furthermore, our method does not affect the performance of the global model on clean data, resulting in no decrease in the accuracy of the original classification task.
RELATED WORK
Federated Learning. Federated learning (FL) is an emerging machine learning paradigm that has seen great success in many fields (Ryffel et al. 2018 Hard et al. 2018: Bonawitz et al.] 2019). At high level, FL is an iterative procedure involving rounds of model improvement until it meets some criteria. These rounds send the global model to users and select a subset of users to update the global model. Then those chosen users train their local copy of the model, and their resulting models are communicated back and aggregated to create a new global model. Typically, the final local model's gradients weights transmitted back ensure data privacy.
Backdoor Attack. Recently, several backdoor attacks have been proposed to take advantage of the FL setting. In Xie et al. (2020). the authors show that the multiple-use user nature of FL can be exploitable to make more potent and lasting backdoor attacks. By distributing the backdoor trigger across a few malicious users, they could make the global model exhibit the desired behavior higher rates and for many iterations after the attack had concluded. We will show our threshold's effectiveness holds even when the backdoor attack is more frequently present in federated learning rounds than in the original paper. A recent work (Zhang et al. (022) proposed a projection method, Neurotoxin, which claimed increase the duration that a backdoor association remains present in the shared model after an attack has occurred. The attacker's updates are projected onto dimensions with small absolute values of the weight vector. The authors claim such weights are updated less frequently by other benign users, resulting in greater longevity of successful attacks. We will demonstrate our method's effectiveness against both of the above attacks (Xic et al. 2020 Zhang et al. 2022).
Defense. The most popular aggregation method of FL is FedAvg (McMahan et al 2016). However, Median and Trim-mean, two other robust defense methods for FL, were proposed in Yin et al. (2018). The paper theoretically explores two robust aggregation methods: Median and Trim- mean, which were shown effective in defending against certain attacks in FL. Median is a coordinate wise aggregation rule in which the aggregated weight vector is generated by computing the coordinate-wise median among the weight vectors of selected users. Trim-mean aggregates the weight vectors by computing the coordinate-wel wise mean using trimmed values, meaning that each dimension's 's top and bottom & elements will not be used. We propose a method that can be implemented in addition to other aggregation or model filtering methods. Such other defense methods can be applied to the subset of the randomly selected users to update the model that our method returns. In the experiment, we focus on the original FedAvg (McMahan et al. 2016) aggregation to show the effectiveness of our proposed method without assistance from additional defense techniques.
Few defense methods have been proposed to defend against backdoor attacks in FI.. Prior work (She- jwalkar et al., 2022) claims that norm clipping (Sun et al. 2019) is effective against backdoor attacks in FL but has been broken by the Neurotoxin attack. Recently proposed works for federated learning include (Rieger et al. 2022 Andreina et al. 2020). However, for our experiments, we focus on comparison with FLTrust (Cao et al.] (0200 because it also relies on a small clean training data set. Although their original paper shows that their defense is effective against adaptive attacks where around half of the clients are malicious, we will demonstrate that their method fails against even the most straightforward backdoor attack that we consider in our experiments. Hence, our work is necessary for federated learning and an improvement over the current similar defense methodology.
2

---

Under review as a conference paper at ICLR 2023
3 TRUSTED AGGREGATION
This section describes the motivation and framework for our proposed method, Trusted Aggregation (TAG), which effectively defends against state-of-the-art backdoor attacks in federated learning. We will use the term output to refer to the last model layer's output, typically after several fully connected layers and before any softmax operation has occurred for classífication models.
Motivation. We find that the output distributions of models returned by malicious users are very different from that of benign users. For an m-way classification problem, the outputs of the deep learning model are of the same dimensionality as the classification problem. The softmax operation is commonly performed on these outputs to produce predicted probabilities for each class. Hence, each element of the model output has correspondence with exactly one class. This node-class association specific to the outputs and may not exist for outputs of other hidden layers. For targeted backdoor additional learned association between a given manipulation
attacks, they intend to produce of the input data (trigger) and a specific class label (target). Hence, we believe intuitively that targeted backdoor attacks can be viewed an (m + 1)-way classification problem. We believe such a difference in the model tasks may be exploited to identify models trained with a backdoor attack. In Figure TT we observe that this learned association can come with a distributional change in the output scores specifically for the target class. We conclude that models with a backdoor may produce different distributions of output scores for clean data. Therefore implies that if we had one model believed to be trained without backdoor attack, we might identify whether another candidate model has a backdoor attack by comparing their outputted values on the same clean data.
Thrget: 0
Class
Class:
Class
di31
0.5 Class Score
2.5
5.5
ane Score
Class Score
Class Score
Figure 1:
Output distributions (kernel density estimation based) conditional class label for a backdoor model (red) and clean model (black). Note the obvious difference between distributions the backdoor and clean models for the target label class.
Detection Framework. We assume that there exists a small clean validation data set that can be used for gatekeeping the global model for updates. Note that this data set can either belong to trustworthy existing user or be collected by the centralized server and treated as a new user. Hence, we will refer to this trusted validation data as the trusted user. The detection method leverages the trusted user to evaluate incoming model weights and determine whether each contribution can participate in the global model update procedure. The main idea is to detect user models with an unusually distributed outputs using the clean data set from the trusted user. Our method can be easily extended when several trustworthy data sets or users are present.
See Figure 2 for overview of our proposed detection framework. In each communication round, this validated user completes the following steps to generate a threshold for malicious user detection. First, the validation data is used to update copy of the global model simultaneous the local training of other users. When models are returned by the subset of users chosen to potentially participate in the update of the shared model, all models, including the validation have predicted scores made and stored for the validation data. We denote the saved predicted scores as oc. Or, and o, for the global, validation, and jth user models, respectively.
model and the global model,
Next, we compute the distance between the trusted and other user models from the global model. (c) (c))
Specifically, for each class c, compute the class-conditional distributional distance D(v)) oy
between the global model output (o( and the output (o(e) o by applying distributional difference function. In our experiments, we use the Solmogorov-SSirmo (KS) distance from
estimated CDFs based og (c) (f))) and (c) However, believe there may be other applications where other distance functions should be successful. Suppose there are m classes in total; the process
will result in distance vector (n,, Vy Rm)) for each user, including the trusted user. The distance
3

---

Under review as a conference paper at ICLR 2023
vectors will then determine which can participate in the update. Further details are given in Algorithm
Global Model
5. Aggregated user model replaces global model
4, Accepted user models aggregated
Centralized Server
Trusted Aggregation (TAG)
Global model
Local Data Sets
3a. Compute predictions for user and global models on validation data
Accepted
36. Compute the statistical distance between and global model outputs
Excluded
3c. Use validation distances to estimote the most consideroble distonce possible from benign user's local training
Validation Data
Users
Validation User
3d. Reject user updates with distances exceeding validation cutoff
model locol data
Figure 2: Diagram representation of our trusted aggregation detection framework.
Algorithm Trusted Aggregation
Notation: Let S represent the random subset of that will submit locally trained models Uj to update the global model G, Up to denote the model from the trusted user, X to denote the local data of the trusted user, to represent the distributional difference function, and 0 for the method's scaling coefficient.
1: procedure TRUSTED AGGREGATION(X, G. UT, (U, (E,)
2: Generated outputs: OG = G(X), O7 (x))). ). and o, = Uj(X)
3:
for each class [1,...,,11 mi do
4: Compute the distributional distances between each user and the global model
5:
VT
(e) D(o (c) ) and vf() =D(o (c) (f)) Vje S
(c); output for class
6:
end for
7:
The above procedure produces: UT E RRm,Uu 0 max(UT) F GLOBAL-MIN MEAN SMOOTHING(7)
Uj E Rm
m: total number classes max: maximum element of the vector
8:
Compute threshold:
9:
A Algorithm
10: Select users: S, {j E S max(u,) <)
11: return FedAvg( (U, )jss.)
12:
end procedure
Threshold Construction. In this part, we discuss how to decide the threshold (7) and how to use it to select users. We define the true threshold as the largest possible change a non-malicious user could contribute. Users with distance values exceeding the threshold should be excluded. Naively, this can be thought of as an estimation the maximum given single sample. Hence, the true maximum will be at least as large our observed value. Therefore, parameterize our method by 0 2 1, see Algorithm
In the case that the class-conditional distances (p(e)) are Uniform on [0, b.] for each class c, where be is the maximum possible change the output layer of class through local training by non- malicious user. Hence, estimating the threshold consists of estimating the maximum b, for any class. Let represent the total number of classes. Proposition] gives simple bounds for quantity of interest under arbitrary dependence.
Proposition 1 If p( Uniform(O, be), Ve [1.....m then E V < by E [V] where V max, (f() and arg mnxx)bee) Proof in Appendix C.I
4

---

Under review as a conference paper at ICLR 2023
However, acknowledge that it may be unreasonable to assume that class conditional distances are Uniform as many training paper-prraeetrss and even model choice will impact the distance distributions. We present these results to assist in understanding reasonable magnitudes for & many distributions may not require large scaling values (0 E [ for Uniform). In general, we recommend choosing / based on the setting's 8 prevalence of hackdoor attacks (potentially unknown) and the cost of a successful attack to interested parties, Relatively large values for 0 will allow more users to update the global model but increase the risk of a successful backdoor attack. Conversely, using our threshold 5 without scaling would help to prevent stronger backdoor attacks, but with the potential loss of denying benign users from updating the global model. The goal is choose the smallest that allows benign users to sufficiently create a robust shared machine learning model.
Since the validated user is non-malicious, their distance vector serves as a good representation for other non-malicious users. Therefore, we estimate the threshold 7 with 0 x max(er), where VT E Thm is the distance vector of the validation user and max(-) means getting the maximum value of the vector VT. Then, the maximum distance value (max( (V;)) of each selected user will be compared with the threshold (T) to determine the final list users who can participate in the update. A user with maximum distance smaller than the threshold is considered benign user, while user with a maximum distance larger than or equal to the threshold will be removed. However, this naive threshold is very unstable instability. Therefore, we make an additional modification, global-min mean smoothing, to this basic threshold to address the concern.
lucky malicious can get past it in some rounds due the
Global-Min Mean Smoothing, A naive way to stabilize the threshold value is using a smoothing method, like moving average. However, in the early communication rounds, the naive threshold value rapidly decreases as the model starts making connections between inputs and output classes. Therefore, applying a traditional smoothing method can result in a relatively high threshold early, which may let attackers bypass it. When the naive threshold (T) decreases rapidly, we do not wish to use any previous communication rounds for the smoothing.
Therefore, we propose to use the lowest observed value (Global Min) of r as the starting point of the smoothing window. Let 7, represent the naive estimation of the thresh- old round t. Then the smoothed threshold F at round n is given by
10-
everuge
2
TE
72 tg
where L, is the round that when the global min observed. Details of the global-min mean smoothing are described in Algorithm 2 As Tr shrinks, we observe new global minimums, and the start of the threshold smoothing is reset. In addition, when our estimates do not continue to fall, previous values are leveraged to smooth the cutoff, which keeps lucky malicious users from getting past a volatile threshold. Figure 3 compares our global min- mean smoothing with the naive threshold and various smoothing techniques. The global mean smoothing best captures the naive threshold's early behavior while providing remarkable stability improvements. Most importantly, when our threshold encounters a new global minimum, it provides conservative estimate to prevent malicious users while re-learning cutoff behavior the few rounds.
Commanccaton Round
Figure 3: Comparison of the global min- mean smoothing with the base (naive) thresh- old and various smoothing methods.
Algorithm 2 Global-Min Mean Smoothing
Notation: Let (T1, ... Tn-1. Fn) denote the sequence of values that we wish to smooth.
1: procedure GLOBAL MEAN SMOOTHING(1, Tn-1: Tn)
2: Record the location of global minimum: i arg min 7: se[l....]]
3: Subset to sequence starting with the global min: fTel
4: return average of sequence subset, {
5: end procedure
5

---

Under review as a conference paper at ICLR 2023
4 EXPERIMENTS
In this portion, we provide the following experiments and conclusions. First, in Section 4.2 we show that TAG is the only considered effective defense against several strong backdoor attacks. Secondly, we demonstrate in Section 4.3 that smoothing techniques are in fact necessary for our unstable round-to-round threshold. Next. (..ction|444exteeeee
our main results to when users have non-independent and non-identically distributed data sets, showing the effectiveness of our method for various applications of federated learning. Finally, we include Section 4.3 to emphasize that our method is robust to changes to the distribution and size of the trusted user's validation data. We conclude through extensive experimentation that Trusted Aggregation provides a useful advancement toward model security in the federated learning environment.
4.1 SETTING
Federated Learning. We start by giving further specifications regarding the federated learning environment. Our interest is training a global model over M communication rounds with N users. Each iteration randomly selects K users, using specified proportion of the total users, to participate in the model update. See Section for further experimental specifications.
After local training, the next global model is the average returned model weights by the FedAvg procedure. In our experiments, we use ResNet18, a popular classifier proposed in He et al. (2016). Additionally, in Section B.3]] we demonstrate that our success in not model dependent by repeating main results using VGG16 (Simonyan & Zisserman (2014)). We assume that all users, including malicious, have complete control over all aspects of local training, such as learning rate, the number of epochs, and the model weights they return. For simplicity, we select two main sets of training hyper-parameters for benign and malicious users. The malicious users will poison a given proportion of their local data by adding their backdoor trigger to the input and changing the training label to the target class. They intend for the model to associate the trigger with the target class and hence have the future global model identify any input with the trigger as belonging to the target class.
Attack and Baseline. To show the effectiveness of our method, we choose a setting in which the backdoor attack is powerful. We force the same malicious users to be included in the subset of selected users to update the global model each round after the start of the backdoor attack. Note that the selection of random users is defense against malicious users by making it difficult for them to update the global model repeatedly. Additionally, we do not allow the validation user, a guaranteed benign user, participate in any global model updates. We make these decisions highlight the ability of our threshold to prevent even unreasonably strong backdoor attacks against the global model. For our experiment, we test the proposed method, Median and Trim-mean (Yin et al., 2018 and FLTrust (Cao et al. 2020) against two state-of-tle offe--of-hhe-ar backdoor attacks in FL: Neurotoxin (Zhang al. 2022) and Distributed Backdoor Attacks (DBA) (Xie al. 2020). To further evaluate the effectiveness of the aggregation methods, we also vary the proportion of malicious attackers to comprise between 10% and 40% of the selected users to test the defense methods under different attack strength levels.
Data. The experiments are done on three different data sets: CIFARIO Krizhevsky & Hinton 2009). CIFAR100 100 (Krizhevsky & Hinton] 2009), and STL1O (Coates et al. 2011). In each experiment, we construct user data sets by random sampling from the training data. Again, see Appendix A for further experiment details and hyperparameters. For global model evaluation, we split the test set into two parts. We add the backdoor trigger to images in the second half and remove any target class observations. We measure model performance the proportion of the poisoned half predicted as the target class, known as attack success rate, to measure the extent that the backdoor attack has compromised the model. For a defense method, good performance consists of a low attack success rate and high classification accuracy. In other words, attacks are unsuccessful when the defense method used, and the defense does not negatively influence the classification performance.
the first half using classification accuracy, and
4.2 COMPARISON OF DEFENSE METHOOSS AGAINST BACKDOOR ATTACKS
We begin by considering setting where 10% of the returning user models malicious each communication round. Figure 4 shows the performance of the various methods against backdoor attacks with and without the Neurotoxin projection on three data sets regarding model classification

---

Under review as a conference paper at ICLR 2023
accuracy and attack success rate. For our main results, use the TAG scaling coefficients (0) of 2, 2, and 1.1 for the data sets CIFARIO, CIFARIO0, and STL10, respectively. Our proposed method nullifies the backdoor attack in each case without a meaningful decrease the classification accuracy of the original task in general. However, the other methods, coordinate-wiss Median, Trim-mean, and FLTrust, fail prevent backdoor attacks with without Neurotoxin. Although FL.Trust was shown in its original paper to be successful against an adaptive (attacker knows defense) data poisoning attacks aiming change the global model update along the opposite direction of the global model update under no attacks, FLTrust fails to prevent targeted backdoor attacks in the weakest attack setting we consider. We conclude that our method is a clear improvement to these existing defense methods for federated learning.
CIFAR-10
CIFAR-100
STL-10
1.0
Classification Accuraty
Trusted Aggregation
0.8
0.8
0.8
coordirate Median
Coordirate Thim Mean
0.6
0.6
FLTrust
0.4
0.4
Attack, Success Rate
Trusted Aggragation
0.2
0.2
0.2
Coordir ate Median Coordirate Trim-Mean
0.0
0.0
FLTrust
0
100
200
100
200
200 Communication Round
100
Corr Coniicaaioo Round
Communication Round
(a) Performance Under Backdoor Attack With 10% Malicious Users.
CIFAR-10
CIFAR-100
STL-10
1.0
0.8
0.8
0.6
0.4
0.4
0.4
0.2
0.2
0.0
0
100
200
0
1000
200
200 Communication Round
100
Communication Round
Communication Round
(b) Performance Under Neurotoxin Backdoor Attack With 10% Malicious Users.
Figure 4: Model performance under standard (top) and Neurotoxin (ber) backdoor attacks with 10% malicious users. The proposed method TAG performs well in defending against backdoor attacks the attack success rates are low. Mcanwhile, it does not generally affect the model's classification performance on clean data, except for a slight decrease on STL10 for the DBA attack. However, the other methods do not work well against any backdoor attacks.
TAG can handle more difficult attack settings against state-of-the-e attacks. We observe similar results compared to our other attack settings when testing the defense methods against DBA and Neurotoxin attacks where 20% of updates malicious. Unsurprisingly. the other defenses, which failed against the previous setting, cannot prevent stronger attacks whereas TAG again prevents all backdoor attacks. See Section B. for results.
We instead focus on even stronger DBA and Neurotoxin attacks with 40% malicious users in the selected set. These attacks are catastrophically successful against the current robust aggregation methods, see Figure 5 having nearly perfect attack success rate after round 50 on all our data sets. In some settings FLTrust, able to delay the attack success compared robust aggregation, but still fails to prevent any of the backdoor attacks of this or previous strengths. However, our method, TAG, overcomes the backdoor extent of the initial rounds to prevent the attack against all data sets.
We conclude that even when the federated learning setting updates consist of almost half malicious users, we can use TAG to attempt to eliminate backdoor attacks. We show various additional experi- mentation results in our Appendix to strengthen the evidence for our claims. First, in Section B.2 we offer that our model does not hinder performance for the original classification task even without the presence backdoor attack compared FedAvg. Secondly, Section [B.3] supports that our main results, reproduced with VGG16 instead of ResNet18, are not dependent on model choice. Hence our method leads increased model security negligible cost the shared model, even under lack
7

---

Under review as a conference paper at ICLR 2023
CIFAR-10
CIFAR-100
STL-10
1.0
1.0
1.0
Classification Accuracy
Trusted Aggregation
0.8
0.8
0.8
Coordir Modian
Coordirate
0.6
0.6
0.6
FLTrust
0.4
0.4
0.4
Attack Success Rate
Trusted Aggregation
0.2
0.2
0.7
Coordirate Thim-Mean
0.0
0.0
0.0
100
200
200
Cor munication Round
Communication Round
Communication Round
(a) Performance Under DBA Backdoor Attack With 40% Malicious Users.
10
CIFAR-100
STL-10
1.0
1.0
1.0
0.8
0.8
0.8
0.6
0.6
0.6
0.4
0.4
0.4
0.2
0.2
0.2
0.0
0.0
0.0
100
200
100
200
100
200
Corn munication Round
Communication Round
Corn Communicction Round
(b) Performance Under DBA And Neurotoxin Backdoor Attacks With 40% Malicious Users.
Figure 5: Model performance under DBA without (top) with (bot) Neurotoxin backdoor attacks with 40%
malicious users. TAG is the only method that prevents the backdoor attacks.
of attack. We conclude that Trusted Aggregation (TAG) is an essential advancement toward model security for the federated learning framework.
4.3 NECESSITY OF THRESHOLD SMOOTIING
We revisit the last attack considered the STL10 data highlight the importance of our proposed technique, global-min mean smoothing. Recall that our smoothing is intended to improve the stability of our estimated threshold while preserving its behavior in the initial rounds, smoothing techniques. If we repeat the backdoor attack where 40% of the user subset is malicious each iteration and omit the global-min mean smoothing, our method no longer can prevent the backdoor attacks with Neurotoxin projection, see Figure 6 Hence, without smoothing, we conclude that malicious users may have an opportunity to get past less stable cutoff.
conserved by other
STL-10
STL-10
1.0
Classitictiinn Accuracy
Trusted Aggregation
0.8
Courdinete Medien
Coordinate Mean
0.6
9.6
FLTrust
0.4
9.4
Attakk Success Rate
Trusted Aggregetion
0.2
0.2
Coordinate
Coordinate Trim-Mean
0.0
0.0
Communication Hound
Communicction Hound
Figure 6: Model performance under DBA without (leff) and with (right) Neurotoxin backdoor attacks with 40% malicious STL10 without using global-min smoothing. TAG fails defend against the backdoor attack STL10 without the improved stability our estimate from our proposed smoothing technique.
4.4 EXTENDING RESULTS To IMBALANCED USER DATA SETS
In this section, we show the applicability of our method for imbalanced users focusing on the CIFAR1O data set. Recall that local data sets not required to be independent and identically distributed for federated learning. In this experiment, we use the m-dimensional (number of total classes) Dirichlet distribution with
determine the proportion for each class label to be
8

---

Under review as a conference paper at ICLR 2023
randomly sampled. Recall, we parameterize the Dirichlet distribution by the simplified alm where Im is an m-dimennional vector of ones, and a E [0, 00) is a scalar such that smaller values lead to imbalanced user data.
Regarding attacks with 10% and 20% malicious users, we did not have to tune (using 0 from previ- ously balanced data experiments) have a very successful defense when data is highly imbalanced (a = 1), see results in Section of the Appendix for further discussion. In summary, we repeat the previous experiment without modifying 0 to understand how different defending against backdoor attacks, with TAG, is for imbalanced user data. TAG is reasonably robust to the weaker backdoor attacks without tuning for the new application. However, f should be adjusted for each application to ensure TAG's best possible defense.
CIFAR-10
CIFAR-10
Clessification Accuracy
Trusted Aggregation
Coordinate Mcdian
Coordinate Trim-Medd Mean
FLTrust
Attack Success Rate
Trusted Aggregation
02-
Coordinete Median
Coordinete Trim-Mean
R.Trust
100 200
100
700
Communication Round
Communkcatio Round
Figure 7: Model performance under DBA without (leff) and with (right) Neurotoxin backdoor attacks with 40% malicious imbalanced local data with tuned O. The proposed method, TAG, performs well in defending against backdoor attacks, even when the user data imbalanced. Again, the other defense methods do not prevent any backdoor attack under imbalanced data.
When we do tune 0, TAG is effective for all attack prevalences, including 40%. Recall the trade-off between robustness against backdoor attacks and performance for the original classification task. We attempt to choose the smallest 9 that successfully trains a global model regarding classification accuracy. In Figure 7 with 0 1.25 and 0 = 1.5 respectively, TAG can eliminate the 40% malicious backdoor attacks. With less scaling, the original classification accuracy suffers in early communication rounds as fewer users update the model. However, the global model nears convergence, the round- to-round accuracy stabilizes and becomes similar to or better than the baseline methods. Hence, conclude that our proposed method, TAG, handle backdoor attacks imbalanced data and without losing accuracy for the original task. These experiments support our recommendation to choose the smallest scaling possible that achieves desired performance the original task, and this setting is more than sufficient to show the usefulness of our method for the majority of federated learning applications.
4.5 ROBUSTNESS TO TRUSTED USER DATA SET
Recall that the trusted user either consists of guaranteed non- non-malicious user or the centralized server collects small validation data sets and acts another user. We argue that the centralized server can take precautions to collect a representative clean data set. However, if the trusted user is an actual user or the centralized server fails to obtain such a representative sample, is vital understand how the data distribution of the trusted user impacts our defense. Additionally, we believe that we only need consider the case that the trusted data set is imbalanced if the user data sets are also imbalanced. When the users have access to balanced data, it seems plausible that our trusted user would also likely have access balanced data. For this section, we consider the case where both our users and trusted user have data Dirichlet sampled data sets from CIFAR 10 with a =
In Section B.5.1 we have results on model performance with imbalanced trusted user data without tuning 0 from those obtained for the previous imbalanced user experiments. These additional experiments serve to show the difference defending against backdoor attacks with and without a representative trusted user data set. In summary, we see similar results to robustness of TAG to the data distribution of users where we can defend against our weaker attack settings without tuning A. For TAG to be most effective, however, to provide its best defense. Additionally, Section B.5.2 that dramatically reducing the size of the trusted data set, relative other users, does not impact the success of TAG defense method.
scaling coefficient must be tuned for every application

---

Under review as a conference paper at ICLR 2023
Furthermore, in all these experiments, FLTrust is allowed full sized, representative trusted user data set, and yet is still greatly outperformed by TAG.
Tuning TAG's scaling coefficient for the 40% malicious update setting gives 0 = 1.25 and 2 for DBA without and with Neurotoxin. respectively. In Figure g we observe that even with an imbalanced trusted user data set and 40% of user updates malicious, our proposed method, TAG, can effectively prevent backdoor attacks. We observe that in early communication rounds, classification accuracy for the original task suffers, but the model's performance at convergence is not impacted by the presence of our defense. Therefore, we conclude that although a representative trusted data set is desirable, TAG is an essential tool for backdoor defense even when representative trusted data set is not obtained. TAG can be an effective defense regardless of whether any users, including the trusted user, have data that is non-independent and non-identically distributed.
CIFAR-10
CIFAR-10
1.0
100
Classification Accuracy
Trusted Aggregation
0.8
Courdinate Median
Coordinate Trim Mean
0.6
FLTrust
0.4
Attack Success Rate
Trusted Aggregation
0.2
Coordinare Mediaan
Coordinate Trim-Mean
0.0
FLTrust
0
100
200 Hound
0
100
200
Communication
Communication Round
Figure 8: Model performance under DBA without (left) and with (right) Neurotoxin backdoor attacks with 40% malicious users on imbalanced local user data sets with an equally imbalanced trusted validation data set where 0 is tuned.
5 LIMITATIONS
We demonstrate in Section 4.4 and Section 4.5 the success of TAG for non-independent and non- identically distributed imbalanced (non-iid) user data sets. First, we acknowledge that there exist many more extreme cases of non-iid data sets. For example, we do not test our method in the class partition case where each user holds the entirety of small number of the total classes. However, we note that for more extreme data distributions, may not be reasonable to consider learning a single shared model for all users, which is the main objective in this work. Second, although our framework is extendable to other models (regression, NLP, etc.). we only have experimental results for common classification computer vision models and databases. Future work may need be done modify our method or demonstrate success for other applications.
Most importantly, we do not claim that our defense can successfully defend against an adaptive attack. When the malicious attacker has knowledge that our defense is being used (adaptive attack), they can train two copies of the global model. They can train the first copy without data poisoning and use this model for penalizing the output layer distribution of the second copy while performing backdoor attack. We reemphasize, favor of our work, that as a solution our method can be used in conjunction with other defends methods and also is vastly outperforming similar defense with claims towards adaptive attacks. Regardless, we believe future work is needed to modify our defense to work in the adaptive setting.
6 CONCLUSION
While current defense methods are failing to prevent even mild backdoor attack settings, TAG holds up against state-of- thr- attacks in unreasonably strong settings. Furthermore, TAG works against backdoor attacks when users have heterogeneous and imbalanced data and does not decrease the classification accuracy for the original task regardless of whether or not attacks are present. Furthermore, although our method relies on a some additional data, we show through experimentation that regardless of size distribution TAG is robust many attacks and effective against very strong attacks with proper tuning. Finally, TAG is compatible with other filtering methods or modifications to the choice of aggregation step and can be used in conjunction with other strong defense methods to prevent attacks. We conclude that TAG is an essential advancement toward model security for the federated learning framework.
10

---

Under review as a conference paper at ICLR 2023
REFERENCES
Sebastien Andreina, Giorgia Azzurra Marson, Helen Möllering, and Ghassan Karame, Baffle: Backdoor detection via feedback- based federated learning, 2020. URL /ttppp::/arxxi..oo /bss 2aabb/2011 2011 2011.02167
Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huha, Alex Ingerman, Vladimir Ivanov, Chloc Kiddon, Jakub Konečný. Stefano Mazzocchi, Brendan McMahan, et al. Towards federated learning at scale: System design. Proceedings Machine Learning and Systems, 1:374-388. 2019.
Xiaoyu Cao, Minghong Fang. Jia Liu, and Neil Zhengiang Gong. Fltrust: Byzantine-robust federated learning via trust bootstrapping. 2020. htt.htttss:/axii.orr////////
13995
Adam Coates, Andrew Ng. and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pp. 2011.
Andrew Hard. Kanishka Rao, Rajiv Mathews, Swaroop Ramaswamy, Françoise Beaufays, Sean Augenstein. Hubert Eichner, Chloé Kiddon, and Daniel Ramage. Federated learning for mobile keyboard prediction. arXiv preprint arXiv:1811 1336040 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE Conference Computer Vision and Puttern Recognition (CVPR), pp. 770-778, 2016. doi: 10. N0.1109/CVP.22166.20
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, Citescer, 2009.
Keita Kurita, Paul Michel, and Graham Neubig, Weight poisoning attacks on pretrained models. pp. 2793 2806, July 2020. doi: 10.18655/11200202al maaam..229. /tL/ 200gg///mco..ooooooo/22002000 acl mw..ma....... 249
Linyang Li, Demin Song, Xiaonan Li, Jiehang Zeng. Ruotian Ma. and Xipeng Qiu. Backdoor attacks on pre-trained models by layerwise weight poisoning. pp. 3023-3032, November 2021. doi: 10. 1/2201. emnlp- caappppmmin..2.. 241. URL httttps://aaaansstest
Dhurgham Hassan Mahlool and Mohammed Hamzah Abed. A comprehensive survey on federated learning: Concept and applications. 2022. doi: (0.48550/ARXI1.2221.. 09384. http:tttss///aii..oooo abs/ 22011 09384
H. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas.
Communication- efficiieftttttttiiiee learning deep networks from decentralized data. 2016. doi: 10.48550/ARXIV. 1602.05629.
05629
Fanchao Qi, Yangyi Chen, Mukai Li. Yuan Yao, Zhiyuan Liu, Maosong Sun. Onion: simple and effective defense against textual backdoor attacks. arXiv preprint arXiv:2011.10366 2020.
Phillip Rieger, Thien Due Nguyen, Markus Miettinen, and Ahmad-Reza Sadeghi. DeepSight: Mitigating
backdoor attacks in federated learning through deep model inspection. In Proceedings 2022 Network and Distributed System Security Symposium. Internet Society. 2022. doi: 00.14722/nds..22222355555 URL
Aittps://dci. www.l.rrgglii /0. 14 /22 (22 2022 23155)
Theo Ryffel, Andrew Trask, Morten Dahl, Bobby Wagner, Jason Mancuso, Daniel Rueckert, and Jonathan Passerat-Palmbach. A generic framework for privacy preserving deep learning. arXiv preprint arX1v:1811.04017, 2018.
Virat Shejwalkar, Amir Houmansadr, Peter Kairouz, and Daniel Ramage. Back the drawing board: A critical evaluation poisoning attacks on production federated learning. In 2022 IEEE Symposium on Security and Privacy (SP), pp. 1354-1371 IEBE, 2022.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition, 2014. httph:t/////......
htttttts:t/ /ttp///aaxxii wwtp:/twwwww......././//aaa.aaae http://bs/144..... /11//440911110 1556
Ziteng Sun, Peter Kairouz, Ananda Theertha Suresh, and и Brendan McMahan. Can you really backdoor federated learning? arXiv preprint arXiv:1911. 007663, 2019.
Chulin Xie, Keli Huang, Pin- Yu Chen, and Bo Li. Dba: Distributed backdoor attacks against federated learning. In International Conference on I Learning Representations, 2020. URI. https:: //tp:r/v/eeeevvveeeeee ew net./ forumm Sorkgy

---

Under review as a conference paper at ICLR 2023
Dong Yin. Yudong Chen, Ramchandran Kannan, and Peter Bartlet. Byzantine-robust distributed learning: Towards optimal statistical rates. In Jennifer Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp. 1650-5659 PMLR, 10-15 15 Jul 2018. URL UtLhttps::/ htttp:://procoooingg.. mmr. www.ppsssssee.. v80 y h/n/yinnnll Ba html
Zhengming Zhang, Ashwince Panda, Linyue Song, Yaoqing Yang. Michael Mahoney, Prateek Mittal, Ramchan- dran Kannan, and Joseph Gonzalez. Neurotoxin: Durable backdoors in federated learning. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of 39th International Conference Machine Learning, volume 162 Proceedings of Machine Learning Rexearch, pp. 26429-26446. PMLR, 17-23 Jul 2022.
12

---

Under review as a conference paper at ICLR 2023
A EXPERIMENTAL DETAILS
For reproducibility, offer in Table as a clear description of all arguments used in our experiments. In conjunction with our GitHub code at (LINK TBD), these details enable readers to repeat and expand upon our results.
It is important to note that for the STL10 data set, we inverted the data splits commonly referred to as train and test. We avoid using any unlabeled data, and we use the more extensive labeled data set (800 images per class) for training. Also not specified in the table, local user data sets are randomly sampled from the training data sets where a Dirichlet distribution determines the frequency of each class. Balanced local data dimensionality equal to the total number of classes. In certain experiments, imbalanced user and trusted data sets created by the modification 1.
obtained using 10000 to scale vector of ones with
Hyperparameter
Variable Name
CIFARIO
CIFARI00
STLIO
Users
n_users
100
100
20
Federated Learning
Local Data Size
T1 user _user_data data
500
500
400
User Subset Proportion
p_report
.1
.1
.5
Padding
4
4
12
Data Augmentation
Random Horizontal Flip
NA
.5
.5
Random Crop Size
32
96
Batch Size
batch
64
64
128
All Users
Weight Decay
wd
5e-4
Se-4
5e-4
Local Epochs
n_epochs
10
Benign Users
Learning Rate
Ir
.01
.01
.01
Local Epochs
epochs pois
20 (15)
15
25 (I5)
Malicious Users
Learning Rate
Ir r_poois
.01
.005 (.01)
Poisoning Proportion
P_pois
.1
.1
.1
Data Poisoning
Stamp Pixel Height
row_size
4
4
24
Stamp Pixel Width
col_size
4
24
TAG Scaling (0)
scale
2
1.1
Backdoor Defense
Trim Mean
bela
.2
.2
.2
Table I: Default arguments for all experiments unless otherwise specified. For all experiments, alternative values for B did not seem prevent the backdoor attacks. Any values modified for Neurotoxin attacks are shown in parentheses.
13

---

Under review as a conference paper at ICLR 2023
B SUPPLEMENTARY RESULTS
B.1 COMPARISON OF DEFENSE METHODS AGAINST BACKDOOR ATTACKS (CONTINUED)
TAG can handle difficult attack settings against state-of-the-art attacks. We consider testing the defense methods against DBA and Neurotoxin attacks with 20% malicious users
selected set. Unsurprisingly, the other defenses, which failed against 10% malicious, cannot prevent stronger attacks, see Figure 9 We do observe that FLTrust appears to delay the attack success relative to robust aggregation but still results in compromised model. On the other hand, our method, TAG, again prevents all backdoor attacks. For results demonstrating model performance against 10% and 40% malicious updates, see Section
CIFAR-10
CIFAR- 100
STL-10
Classification Accuracy Trusted Aggregation Median Coordirate Thim-Mean
Attack success Rate
Trusted Aggragation
Cuordirate Median
Coordirate Thim-Mean
FLTrust
300
munication Round
Communication
Communication Round
(a) Performance Under Backdoor Attack With 20% Malicious Users.
CIFAR-10
CIFAR-100
STL-10
0.0
Cor munication Round
Communication Round
Communication Round
(b) Performance Under Neurotoxin Backdoor Attack With 20% Malicious Users.
Figure 9: Model performance under standard and Neurotoxin backdoor attacks with 20% malicious TAG the only method that prevents these backdoor attacks.
14

---

Under review as a conference paper at ICLR 2023
B.2 TAG CLASSIFICATION ACCURACY WITHOUT ATTACKERS
A successful backdoor defense prevent attackers while best preserving the model performance the original task. The model ability for the original task must also be maintained in the absence of attack. In addition to successfully preventing attacks when present, we observe in Figure that our defense does not hinder the classification accuracy of the original task on STL10 compared to the FedAvg procedure. This supports the overall usefulness of proposed method the shared model will have improved security with our defense without cost if the system is not threatened by attacks.
Dalena
Only
0.6
Dieatra on RCCURLY
02
c.omuniiaao
Figure 10: Model classification accuracy on STL10 in the absence of backdoor attacks without (left) and with (right) TAG defense.
15

---

Under review as a conference paper at ICLR 2023
B.3 MODEL GENERALIZABILITT
In this section, we demonstrate that results not architecture dependent by repeating main experiment results on CIFARIO for another off-the-shelf image classification model. The following results are obtained using VGGI6 with batch normalization, originally proposed in (Simonyan & Zisserman 2014). In this experiment, Trim Mean is parameterized by B = 0.1. However, other values for D did not impact defense.
Recall that we attempt to choose the smallest scaling coefficient such that the global model obtains desirable classification accuracy. For CIFAR1O, 0 2 successful for the ResNet18 8 architecture both for preventing backdoor attacks and for good classification accuracy for the original task. However, VGG required 2.5 to train a sufficient global model under Neurotoxin attacks. We observe that the choice of 0 may depend on many hyperparameters of various parts of the federated learning procedure.
all attack settings, as presented in our main results, Section 4.2 our proposed defense TAG is the only method successful in preventing backdoor attacks. We again note that the successful defense of TAG on the original classification task. We conclude that TAG can be effective backdoor defense for federated learning for various model choices.
associated with meaningful change global model performance
CIFAR-10
CIFAR-10
1.0
1.0
Clessifi Accuracy
Trusted Aggregation
0.8
0.8
Coordinate Median
Coordinate Trim-Mean
0.6
0.6
A.Trust
0.4
0.4
Attack Rate
Trusted Aggregation
0.2
0.2
Courdinete
Coordinete Trim-Mean
0.0
0.0
200
Communication Round
Communication Round
Figure 11: VGG model performance under standard (leff) and Neurotoxin (right) backdoor attacks with 10% malicious users.
CIFAR-10
CIFAR-10
Classification Accuraty
trusted Aggregetion
Coordinate
Coordinate trim-Mean
FLTrust
Attack Success Rate
Trusted Aggregation
Coordinete Median
Coordinete Trim-Meen
A.trust
100
200
200 Communication Round
100
Conmunication Round
Figure 12: VGG model performance under DBA without (leff) and with (right) Neurotoxin backdoor attacks
with 20% malicious
CIFAR-10
CIFAR-10
Classification Accuracy
Trusted Aggregation
Coordinate Modiaa
Coordinate trim Mean
Attack Success Rate
Aggregetion
Coordinate Medion
Coordinate Mean
A.Trust
100
200
200 Communication Round
100
Communication
Figure 13: VGG model performance under DBA without (leff) and with (right) Neurotoxin hackdoor attacks with 40% malicious users.
16

---

Under review as a conference paper at ICLR 2023
B.4 EXTENDING RESULTS To IMBALANCED USER DATA SETS (CONTINUED)
We present the omitted figures from Section 4.4 to understand the success of backdoor attacks when local user data sets are imbalanced. Figure 14 and Figure [5 show that even without tuning TAG's scaling coefficient 0, proposed defense is effective for imbalanced user data. Note that are using 9 = as obtained from tuning our defense method on balanced local user data sets for 10. Similar to other experimentation results, TAG is the only method to prevent backdoor attacks without changing model performance for the original classification task.
CIFAR-10
CIFAR-10
10
1.0
Clesaification Accuraty
Aggregation Coordinate Median
0.8
Coordinace Trim- fim-Mean
FU7rust
0.4
Success
Trusted Aggregation
0.2
Coordinate Median
coordinace Trim-Mean
0.0
0.0
RuTrust
100
200
100
200
Concmuniittion Round
Communicatiin Round
Figure 14: Model performance under standard (left) and Neurotoxin (right) backdoor attacks with 10% malicious users and imbalanced local user data sets without tuning O.
CIFAR-10
CIFAR-10
10
10
Classification Accurary
Trusted Aggregation
0.8
0.8
Coordinete Median
Coordinate Trim Mean
0.6
0.6
FUTrust
0.4
0.4
Attack Success Rate
Trusted Aggregation
0.2
Coordinare Mediaan
Coordinate
0.0
FLTrust
100
200
100
200
Communication
Round
Communication Round
Figure 15: Model performance under DBA without (left) and with (right) Neurotoxin backdoor attacks with 20% malicious users and imbalanced local user data sets without tuning 6.
This section's 's results help us understand how different backdoor defense is under other local user data distributions and the implications of misspecified TAG scaling. Consistently TAG scaling is robust to changes under weaker attacks but needs application-specifi tuning to offer its best backdoor defense. If 0 is not tuned, both considered backdoor attacks successful 40% prevalence, see Figure T6 However, in Section 4.4, we observe that with proper tuning of 0, even with imbalanced user data, TAG can prevent the powerful attack where 40% of returning user updates are malicious. TAG is good choice for defense method regardless of the data distribution of its users.
CIFAR-10
CIFAR-10
1.-
1.0
Classification Accuracy
Truszed Aggregation
0.8
0.3
Coordinate Mcdian
Coordinate trim-Kesn
0.6
0.5
FLTrus:
0.4
0.4
Success
Trusted Aggregation
02-
0.-
Coordinate Median
coordinate Trim-Wean
0.6
0.0
FLTrus:
100
200
100
200
Communication Round
Conmmunnattinn Round
Figure 16: Model performance under DBA without (left) and with (right) Neurotoxin backdoor attacks with 40% malicious users and imbalanced local user data sets without tuning 0.
17

---

Under review as a conference paper at ICLR 2023
B.5 ROBUSTNESS To TRUSTED USER DATA (CONTINUED)
B.5.1 ROBUSTNESS TO DISTRIBUTION OF TRUSTED USER DATA
Recall that the trusted user either consists of guaranteed non-malicious user the centralized server collects a small validation data sets and acts as another user. When the trusted user is the centralized server, they should attempt collect balanced representative data However, the trusted user is an actual user or the centralized server fails to obtain such representative sample, it is vital to understand how the data distribution of the trusted user impacts defense.
To better understand the effect of validation data distribution, we first omit finding the smallest scaling coefficient 0 such that the global model obtains desirable classification accuracy. We begin by repeating the imbalanced user data experiments with the same scaling / as before. Without properly tuned 0, TAG still effectively defends against combinations of DBA and Neurotoxin attacks when 10% (Figure 17 and 20% (Figure 18 of the selected user subset are malicious each round.
CIFAR-10
CIFAR-10
1.0
Classification Accuracy
Trusted Aggregation
Coordinace Median
Coordinate Trim- Trim-Mean Mean
R.Trust
Attack Success Kate
Trusted Aggregation
Coordinate Median
Coordinace
FUTustt
100
200
100
200
Communication Round
Communication Round
Figure 17: Model performance under standard (left) and Neurotoxin (right) backdoor attacks with 10% malicious users on imbalanced local user data sets with an equally imbalanced trusted validation data set without tuning 0.
CIFAR-10
CIFAR-10
Clasaification Accuraty
Trusted Aggregation
Coordinate Median
Coordinace Trim-Mean
FUTrutt
Attack: Success Rate
Trusted Aggregation
Coordinace Median
Coordinace
R.Trust
100
280
200 Cormnnnication Round
100
Communication
Figure 18: Model performance under DBA without (left) and with (right) Neurotoxin backdoor attacks with 20% malicious users imbalanced local user data sets with an equally imbalanced trusted validation data set without tuning 8.
TAG can even prevent the DBA and Neurotoxin attack with a 40% attackers but now fails to prevent pure DBA, see Figure 19 We conclude that tuning the scaling coefficient 0 is essential for every TAG defense application. Despite the success of TAG our earlier attack settings, defending against backdoor attacks is different task for differently distributed trusted user data. Careful tuning of will help to ensure the best protection TAG can provide against strong backdoor attacks.
See, Section for proper experimentation with imbalanced users and trusted user using tuned values for TAG's scaling coefficient 0. With against backdoor attacks regardless of whether not any user has imbalanced data.
tuning, TAG can be an effective defense
18

---

Under review as a conference paper at ICLR 2023
CIFAR-10
CIFAR-10
1.0
1.0
Classification Accuracy
Trusted Aggregation
0.8
Coordinate Modian
Coordinace
0.5
FLTrust
0.4
Attack Success Rate
Trusted Aggregation
0.7
0.
Coordinace Trim Trim-Mean Mean
0.0
100
200
200
Concuuniitionn Round
Communication Round
Figure 19: Model performance under DBA without (left) and with (right) Neurotoxin backdoor attacks with 40% malicious users on imbalanced local user data sets with an equally imbalanced trusted validation data set without tuning 0.
B.5.2 ROBUSTNESS To SIZE OF TRUSTED USER DATA
Additionally, we want to determine whether our method depends on such size requirements. Hence revisit the most substantial attack setting for the CIFAR10 R1 data set but only allow the validation user to have a data set that is 20% of the size the other local users. Note that for this experiment, all have balanced and representative data. This experiment most applicable the case where the centralized server needs to collect data, especially for problems where the cost of obtaining data is high. Here the validation set now only allowed 100 images, yet TAG prevents the backdoor attack and can achieve improved accuracy compared to the baseline robust aggregation methods. Hence, we conclude we do not need validation data of the same quantity as other local users discriminate between benign and malicious returning models.
CIFAR-10
CIFAR-10
1.0
Clessification Accuracy
Trusted Aggregation
Coordinate Median
Coordinate Trim-Mean Mean
Attack success Rate
Trusted Aggregation
Cuordinate Median
Coordinate Trim-Mean
0.0
FLTrust
100
200
100
200
Communication Round
Communication Round
Figure 20: Model performance under DBA and Neurotoxin backdoor attacks with 40% malicious users on CIFAR10 where the validation data set is 20% the size of the local users. Still, the proposed method TAG performs well, and the other two aggregation methods do not work well in preventing any backdoor attack.
19

---

Under review as a conference paper at ICLR 2023
C
SUPPLEMENTARY PROOFS
C.1 PROOF OF PROPOSITION
!
Assume Vc E
~ Unif(0, be)
Define V = max (1(c) and
arg (bx)
bj 2
v S V bb bj
E
SBM
S by S E [2
C.2 PROPOSITION 2
If we additionally assume independence between the class conditional distances, additional lower bound, Proposition 2 for our largest possible benign change. Note, Proposition 2
establish
converges asymptotically to the earlier lower bound as the size of the classification problem (m + o0)
grows. Hence, this additional bound is provided to assist with choosing an appropriate 0 for smaller- way classification problems.
Proposition 2 If v(e) A Uniform(0, Ve E [I,..., mm] then E
{
m +
V
S by where
max. ul6) and = arg max, (bc).
Assume Vc E [1,...,m]] (ee Unif(0,bz) by)
Let W max (w()t) where (e()) Unif(0, by) - W
~ Beta(m, 1) - E(W) = m
by
m + 1
m
T7
(v P(V 51)
II Pato (1) N
II
P ii (t) P(W 1) Fw(()
1/ (1
00
=F(V)
Fy (1) S
-1- Fw)()( di F(W)
V
W
[m+1)
v Sb
E
E
E
bj
bj
17 +
m
20


Reviews:
[{"review_id": "xbbHn_HgE0g", "reviewer": "Reviewer_zmnT", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a defense against backdoor attacks to FL. The assumption is that the system has access to some clean validation dataset, which can be used to filter backdoored model updates from malicious clients via analyzing the output layer distribution. Some evaluation is performed to show the effectiveness of the defense against existing backdoor attacks, and comparison is performed for simple baselines. ", "summary_of_the_review": "The paper studies an important topic, but the novelty and quality are limited. ", "strength_and_weaknesses": "+ Backdoor attack is a severe threat to FL, and thus defending against such attack is relevant. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "7zOMDQ2Vu3", "reviewer": "Reviewer_JmoH", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a defense against backdoor attacks on federated learning.\nThe method works by detecting malicious users by analyzing the final layers'\noutput. The proposed method is based on observing that the distributions of\nthe final layer's output between the malicious and benign models differ.\nExperiments on CIFAR-10, CIFAR-100, and STL-10 demonstrate that the proposed\nmethod is effective.", "summary_of_the_review": "Details of the critical experiments are unclear. The evaluation is also weak since\nit only uses a single model and lacks a discussion about adaptive\nattacks. Discussion on existing attacks is also missing.", "strength_and_weaknesses": "Strengths\n\n· Interesting topic.\n\nWeaknesses\n\n· Comparisons with some related works are missing.\n· Generalization to different models is unclear.\n· Discussion about the potential adaptive attacks is missing.\n· Details about the experiments in Figure 1 are unclear.\n· How to select the output layer is unclear.\n· The writing needs improvement. Details of the critical experiments (e.g., Figure\n1) is unclear.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "2BAKvvV6ru", "reviewer": "Reviewer_SvhM", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors introduce a novel robust aggregation method for filtering backdoors in federated learning by using a trusted validation dataset. For this, the authors look at the differences in the distribution of the output layer for the different trusted and untrusted clients, removing those updates that deviate from the distribution in the trusted clients. The authors evaluate the proposed method against two state-of-the-art backdoor attacks in federated learning: Neurotoxin and the Distributed Backdoor Attack, showing the effectiveness of the proposed method to mitigate these attacks. ", "summary_of_the_review": "Although the assumption of having a trusted validation set/clients can be restrictive to certain applications, the algorithms proposed in the paper is interesting and seems to provide good performance against certain backdoor attacks. However, there are certain aspects that require a more thorough analysis: \n+ The comparison with coordinate median and trimmed mean is not really fair as they do not assume having access to a trusted validation set or trusted clients. In this sense it is necessary to compare with existing defenses that make use of this assumption to really validate the benefits of the proposed approach and its possible limitations. \n+ Looking just at the last layer can make the algorithm vulnerable to other attacks. Given this, I think that the evaluation against an adaptive attackers that is aware of this is necessary to have a more realistic evaluation of the proposed defense.  \n+ The description of the experimental settings used should be clearer and more detail. \n+ Finally, the assumptions made in Proposition 1 and 2 about the uniform distribution of the class-conditional distances is not well justified and looks cherry picked. I think that the authors should justify better their assumptions in this sense.  \n", "strength_and_weaknesses": "Strengths: \n+ Defending against backdoors in federated learning is a challenging problem of interest for the research community. The experimental results in the paper show that the proposed method (TAG) can mitigate the effect of state-of-the-art backdoor attacks such as Neurotoxin and DBA (Distributed Backdoor Attack). \n+ Overall, the paper is well written and the authors explain well the intuition about the method they proposed. \n\nWeaknesses: \n+ In the experiments, the authors just compared with Coordinate Median and Trimmed Mean. This comparison is not really fair: on one side, these techniques do not assume the existence of a trusted validation dataset. On the other hand, these techniques are not specifically designed against backdoor attacks, but more general data and/or model poisoning attacks. It would be necessary to compare with existing defenses that rely on a similar set of assumptions, e.g. Fang et al. “Local Model Poisoning Attacks to Byzantine-Robust Federated Learning”, which uses trusted validation datasets.\n+ The proposed method just looks at the last layer of the models. It is true that, for attacks like Neurotoxin and DBA this can be enough to mitigate the attack. However, this opens the door for adaptive attacks where the adversaries are aware of the defensive method applied. Then, the attackers can try to manipulate the parameters in the previous layers to try to evade detection. In this sense, the paper lacks an analysis of the robustness against adaptive attacks and possible limitations. \n+ Some of the experimental settings used are not clearly described in the paper. For instance, the number of participants in the federated learning tasks, learning rates, optimizer, batch size, etc. I think this is especially important in the case of the number of participants, which is a factor that can play a critical role in the performance of the defense. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "-Is_EtUhgnk", "reviewer": "Reviewer_qsGk", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper presents a backdoor defense for federated learning. The proposed defense is motivated by the differences in the output layer with and without backdoor attacks and aims to identify such changes to prevent backdoor attacks. ", "summary_of_the_review": "The paper investigates a timely problem of backdoor defenses in federated learning. The proposed defense is well-motivated. However, the comparison with the state-of-the-art attacks and defenses is missing in the paper. The paper does not consider the strong non-iid settings in federated learning.", "strength_and_weaknesses": "Strengths:\n1. The paper finds an interesting observation that the backdoor attack generates different output distributions compared with clean models.\n2. The proposed defense is effective against multiple backdoor attacks and can eliminate backdoor attacks even when 40% of clients are malicious.\n3. The paper investigates the defensive effectiveness with different sizes of the validation dataset.\n\nWeaknesses:\n1. The proposed defense measures the distributional difference between clients to identify malicious updates. This defense assumes that all clients share similar data distributions.  However, data heterogeneity is a natural property in federated learning, where data distributions vary among clients. The paper only considers the imbalance data distribution with \\alpha=1, which is not a strong non-iid case. It would be better to consider stronger non-iid data distributions. \n2. The paper only considers some basic backdoor attacks and defenses. State-of-the-art attacks (e.g., edge case backdoors [1]) and defenses [2, 3, 4], should be evaluated/compared in the paper. \n3. The proposed defense requires validation data on the server side to update the global model and use the global model to measure the distributional distances. This requires validation data to be similar to the clients’ data. It is nice to see the paper investigate the impact of the validation data size in the defense. It would be great if the paper could also investigate the different data distributions of the validation data.\n\n[1] Wang, Hongyi, et al. \"Attack of the tails: Yes, you really can backdoor federated learning.\" Advances in Neural Information Processing Systems 33 (2020): 16070-16084.\n\n[2] Andreina, Sebastien, et al. \"Baffle: Backdoor detection via feedback-based federated learning.\" 2021 IEEE 41st International Conference on Distributed Computing Systems (ICDCS). IEEE, 2021.\n\n[3] Hayase, Jonathan, et al. \"SPECTRE: defending against backdoor attacks using robust statistics.\" ICML, 2021.\n\n[4] Tang, Di, et al. \"Demon in the Variant: Statistical Analysis of DNNs for Robust Backdoor Contamination Detection.\" 30th USENIX Security Symposium (USENIX Security 21). 2021.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject