FL-WBC: Enhancing Robustness against Model Poisoning Attacks in Federated Learning from a Client Perspective

1 Introduction . Federated learning ( FL ) [ 1 , 2 ] is a popular distributed learning approach that enables a number of edge devices to train a shared model in a federated fashion without transferring their local training data . However , recent works [ 3–12 ] show that it is easy for edge devices to conduct model poisoning attacks by manipulating local training process to pollute the global model through aggregation . Depending on the adversarial goals , model poisoning attacks can be classified as untargeted model poisoning attacks [ 3–6 ] , which aim to make the global model indiscriminately have a high error rate on any test input , or targeted model poisoning attacks [ 7–12 ] , where the goal is to make the global model generate attacker-desired misclassifications for some particular test samples . Our work focuses 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . on the targeted model poisoning attacks introduced in [ 11 , 12 ] . In this attack , malicious devices share a set of data points with dirty labels , and the adversarial goal is to make the global model output the same dirty labels given this set of data as inputs . Our work can be easily extended to many other model poisoning attacks ( e.g. , backdoor attacks ) , which shall be discussed in §4 . Several studies have been done to improve the robustness of FL against model poisoning attacks through robust aggregations [ 13–17 ] , clipping local updates [ 7 ] and leveraging the noisy perturbation [ 7 ] . These defensive methods focus on only preventing the global model from being polluted by model poisoning attacks during the aggregation . However , we empirically show that these serverbased defenses fail to guarantee the robustness when attacks are extremely strong . More importantly , we observe that as long as the global model is polluted , the impact of attacks on the global model will remain in subsequent rounds even if there are no subsequent attacks , and can not be mitigated by these server-based defenses . Therefore , an additional defense is needed to mitigate the poisoning attacks that can not be eliminated by robust aggregation and will pollute the global model , which is the goal of this paper . To achieve this goal , we first propose a quantitative estimator named Attack Effect on Parameter ( AEP ) . It estimates the effect of model poisoning attacks on global model parameters and infers information about the susceptibility of different instantiations of FL to model poisoning attacks . With our quantitative estimator , we explicitly show the long-lasting attack effect on the global model . Based on our analysis , we design a clientbased defense named White Blood Cell for Federated Learning ( FL-WBC ) , as shown in Figure 1 , which can mitigate the model poisoning attacks that have already polluted the global model . FL-WBC differs from previous server-based defenses in mitigating the model poisoning attack that has already broken through the server-based defenses and polluted the global model . Thus , our client-based defense is complementary to current server-based defense and enhances the robustness of FL against the model poisoning attack , especially against the extremely strong attacks that can not be mitigated during the aggregation . We evaluate our defense on Fashion-MNIST [ 18 ] and CIFAR10 [ 19 ] against the model poisoning attack [ 11 ] under IID ( identically independently distributed ) and non-IID settings . The results demonstrate that FL-WBC can effectively mitigate the attack effect on the global model in 1 communication round with nearly no accuracy drop under IID settings , and within 5 communication rounds for non-IID settings , respectively . We also conduct experiments by integrating the robust aggregation with FL-WBC . The results show that even though the robust aggregation is ineffective under extremely strong attacks , the attack can still be efficiently mitigated by applying FL-WBC . Our key contributions are summarized as follows : • To the best of our knowledge , this is the first work to quantitatively assess the effect of model poisoning attack on the global model in FL . Based on our proposed estimator , we reveal the reason for the long-lasting effect of a model poisoning attack on the global model . • We design a defense , which is also the first defense to the best of our knowledge , to effectively mitigate a model poisoning attack that has already polluted the global model . We also derive a robustness guarantee in terms of AEP and a convergence guarantee to FedAvg when applying our defense . • We evaluate our defense on Fashion-MNIST and CIFAR10 against state-of-the-art model poisoning attacks . The results show that our proposed defense can enhance the robustness of FL in an effective and efficient way , i.e. , our defense defends against the attack in fewer communication rounds with less model utility degradation . 2 Related work . Model poisoning attacks in FL Model poisoning attack can be untargeted [ 3–6 ] or targeted [ 7–12 ] . Untargeted model poisoning attacks aim to minimize the accuracy of the global model indiscriminately for any test input . For targeted model poisoning attacks , the malicious goal is to make the global model misclassify the particular test examples as the attacker-desired target class in its prediction . An adversary using this approach can implant hidden backdoors into the global model so that the images with a trojan trigger will be classified as attacker-desired labels , known as a backdoor attack [ 7–10 ] . Another type of targeted model poisoning attack is introduced in [ 11 , 12 ] , which aims to fool the global model to produce adversarial misclassification on a set of chosen inputs with high confidence . Our work focuses on the targeted model poisoning attacks in [ 11 , 12 ] . Mitigate model poisoning attacks in FL A number of robust aggregation approaches have been proposed to mitigate data poisoning attacks while retaining the performance of FL . One typical approach is to detect and down-weight the malicious client ’ s updates on the central server side [ 13– 16 ] , thus the attack effects on training performance can be diminished . The central server calculates coordinate-wise median or coordinate-wise trimmed mean for local model updates before performing aggregation [ 13 ] . Similarly , [ 14 ] suggests applying geometric median to local updates that are uploaded to the server . Meanwhile , some heuristic-based aggregation rules [ 20 , 21 , 3 , 22 , 23 ] have been proposed to cluster participating clients into a benign group and a malicious group , and then perform aggregation on the benign group only . FoolsGold [ 20 ] assumes that benign clients can be distinguished from attackers by observing the similarity between malicious clients ’ gradient updates , but Krum [ 21 , 3 ] utilizes the similarity of benign clients ’ local updates instead . In addition , [ 7 , 24 ] show that applying differential privacy to the aggregated global model can improve the robustness against model poisoning attacks . All these defensive methods are deployed at the server side and their goals are to mitigate model poisoning attacks during aggregation . Unfortunately , often in extreme cases ( e.g . attackers occupy a large proportion of total clients ) , existing robust aggregation methods fail to prevent the aggregation from being polluted by the malicious local updates showing that it is not sufficient to offer defense via aggregation solely . Thus , there is an urgent necessity to design a novel local training method in FL to enhance its robustness against model poisoning attacks at the client side , which is complementary to existing robust aggregation approaches . 3 Motivation . Although current server-based defense approaches can defend against model poisoning attacks under most regular settings , it is not clear whether their robustness can still be guaranteed under extremely strong attacks , i.e. , with significantly larger numbers of malicious devices involved in training . To investigate the robustness of current methods under such challenging but practical settings , we evaluate Coordinate Median aggregation ( CMA ) and Coordinate Trimmed Mean aggregation ( CTMA ) [ 13 ] on the model poisoning attack with Fashion-MNIST dataset , which is performed by following the settings in [ 11 ] . The goal of the attacks is to make the global model misclassify some specified data samples as target classes . In this experiment , we denote a communication round as an adversarial round tadv when malicious devices participate in the training , and Nm malicious devices would participate in training at adversarial rounds . We assume that there are 10 devices involved in training in each round , but increase Nm from 1 to 5 to vary the strength of the attacks . We conduct experiments under IID setting and the training data is uniformly distributed to 100 devices . The model architecture can be found in Table 3 . For training , we set local epoch E as 1 and batch size B as 32 . We apply SGD optimizer and set the learning rate η to 0.01 . The results of confidence that the global model would miss-classify the poisoning data point are shown in Figure 2 . The results show that the effectiveness of both CMA and CTMA dramatically degrades when there are 50 % of malicious devices in the adversarial rounds . It is worthy noting that the attack impact on model performance will remain for subsequent rounds even if no additional attacks occur . We observe the same phenomenon in alternative robust aggregation approaches , and more detailed results are presented in §7 . Therefore , in order to build a more robust FL system , it is necessary to instantly mitigate the impact of model poisoning attack as long as the global model is polluted by malicious devices . This has motivated us to design FL-WBC to ensure sufficient robustness of FL even under extremely strong attacks . 4 Model Poisoning Attack in FL . To better understand the impact of model poisoning attacks in FL scenarios , we first need to theoretically analyze how the poisoning attack affects the learning process and provide a mathematical estimation to quantitatively assess the attack effect on model parameters . During this process we come to a deeper understanding of the reasons for the persistence of the attack effect observed in §3 . Without loss of generality , we employ FedAvg [ 1 ] , the most widely applied FL algorithm as the representative FL method throughout this paper . 4.1 Problem Formulation . The learning objective of FedAvg is defined as : W = min W { F ( W ) , N∑ k=1 pkF k ( W ) } , ( 1 ) where W is the weights of the global model , N represents the number of devices , F k is the local objective of the k-th device , pk is the weight of the k-th device , pk ≥ 0 and ∑N k=1 p k = 1 . Equation 1 is solved in an iterative device-server communication fashion . For a given communication round ( e.g . the t-th ) , the central server first randomly selects K devices to compose a set of participating devices St and then broadcasts the latest global model Wt−1 to these devices . Afterwards , each device ( e.g . the k-th ) in St performs I iterations of local training using their local data . However , the benign devices and malicious devices perform the local training in different manners . Specifically , if the k-th device is benign , in each iteration ( e.g . the i-th ) , the local model W kt , i on the k-th device is updated following : W kt , i+1 ←W kt , i − ηt , i∇F k ( W kt , i , ξkt , i ) , ( 2 ) where ηt , i is the learning rate , ξkt , i is a batch of data samples uniformly chosen from the k-th device and W kt,0 is initialized as Wt−1 . In contrast , if the k-th device is malicious , the local model W k t , i is updated according to : W kt , i+1 ←W kt , i − ηt , i [ α∇F k ( W kt , i , ξkt , i ) + ( 1− α ) ∇FM ( W kt , i , πt , i ) ] , ( 3 ) where FM is the malicious objective shared by all the malicious devices . DM is a malicious dataset that consists of the data samples following the same distribution as the benign training data but with adversarial data labels . All the malicious devices share the same malicious dataset DM and πt , i is a batch of data samples from DM used to optimize the malicious objective . Except that they share a malicious dataset , the malicious attackers have the same background knowledge as the benign clients . The goal of the attackers is to make the global model achieve a good performance on the malicious objective ( i.e . targeted misclassification on DM ) . Considering the obliviousness of attack , the malicious devices also optimize benign objective , and the trade-off between the benign and malicious objectives is controlled by α , where α ∈ [ 0 , 1 ] . Finally , the server averages the local models of the selected K devices and updates the global model as follows : Wt ← N K ∑ k∈St pkW kt , I . ( 4 )

Summary: The paper proposes White Blood Cell for Federated learning (FL-WBC) -- a client-based defense that can mitigate model poisoning attacks that have already polluted the global model.  The authors claim that strong model poisoning attacks that can circumvent server-based defenses can continue to impact the global model even if there are no subsequent attacks. Towards this end, the paper identifies the attack effect on the parameter space (AEP) and observes that the parameter subspace used for the attack is both inaccessible to the server and remains hidden in the kernel of the Hessian matrices on benign agents. Thus, they propose a client-based optimization that is designed to minimize the loss on the benign task while also minimizing the dimensionality of the Hessian kernel. Robustness certificate and convergence guarantees are also provided.  Evaluation is performed on FashionMNIST and CIFAR-10 datasets to demonstrate the ability of the defense to mitigate attacks using three metrics (attack metric, robust metric, and utility metric). Comparison is performed with DP-based defenses, CMA, and CTMA.