FedDiscrete: A Secure Federated Learning Algorithm Against Weight Poisoning

Federated learning ( FL ) is a privacy-aware collaborative learning paradigm that allows multiple parties to jointly train a machine learning model without sharing their private data . However , recent studies have shown that FL is vulnerable to availability poisoning attacks , integrity backdoor attacks and inference attacks via weight poisoning and inference . In this paper , we propose a probabilistic discretization mechanism on the client side , which transforms the client ’ s model weight into a vector that can only have two different values but still guarantees that the server obtains an unbiased estimation of the client ’ s model weight . We theoretically analyze the utility , robustness , and convergence of our proposed discretization mechanism and empirically verify its superior robustness against various weight-based attacks under the cross-device FL setting . 1 INTRODUCTION . Federated learning ( FL ) is an emerging privacy-aware framework that trains a machine learning model across multiple parties without accessing their local private data ( Konečnỳ et al. , 2016 ; McMahan et al. , 2017 ) . In FL , each client first trains the local model using its private data and then sends the model gradients to an honest central server . The central server aggregates all local model gradients to form a global model , which is sent back to the clients for the next round of training . However , sharing model gradients might still leads to security concerns and privacy leakage . Recent studies have shown that FL is vulnerable to various model weight poisoning attacks . The adversarial client ( s ) can stealthily manipulate the global model via modifying the local model updates to achieve attack goals like , preventing model convergence , implanting backdoors into the global model , and inferring the privacy information of clients ’ private data ( Gu et al. , 2017 ; Blanchard et al. , 2017 ; Pyrgelis et al. , 2017 ; Xie et al. , 2019 ; Bagdasaryan et al. , 2020 ; Wang et al. , 2020 ; Tang et al. , 2020 ) . To address such an issue , recent works start to explore different defense techniques to robustify FL against model weight poisoning attacks ( Blanchard et al. , 2017 ; Bagdasaryan et al. , 2020 ; Shen et al. , 2016 ; Geyer et al. , 2017 ; Fung et al. , 2018 ) . However , most existing works apply the defense or robust techniques at the server side , which changes the role of the central server from being honest to trusted . And prior approaches mostly only focus on one type of attacks . Our contributions . In this paper , we propose the FEDDISCRETE , a flexible FL framework that can combine any popular FL algorithms , for example , FedAvg ( McMahan et al. , 2017 ) and FedProx ( Li et al. , 2020 ) , with our probabilistic discretization mechanism . Theoretical analyses on the utility , robustness , and convergence are performed to show that FEDDISCRETE is inherently robust to availability poisoning attacks ( Kurita et al. , 2020 ) , integrity backdoor attacks ( Gu et al. , 2017 ) and server inference attacks ( Shokri et al. , 2017 ) . FEDDISCRETE is evaluated on four popular image classification datasets under both data are i.i.d and non-i.i.d settings . The numerical results indicate that FEDDISCRETE is robust to various weight-based attacks . 2 BACKGROUNDS AND RELATED WORKS . In this section , we provide necessary background information on concepts thorough the paper and formalize the problem to be solved . Federated learning ( FL ) . FL is a privacy-aware collaborative learning , where N clients and one trusted server work together to learn a global model ( McMahan et al. , 2017 ) . Depending on the application scenarios , the number of clients n can range from as small as two to several hundreds in the cross-silo setting or can easily go beyond millions in the cross-device setting . A classical way of formulating FL into an optimization problem is ( Wang et al. , 2021a ) min w∈Rd F ( w ) : = N∑ i=1 piFi ( w ) , where Fi ( w ) = 1 |Di| ∑ ξ∈Di fi ( w ; ξ ) and N∑ i=1 pi = 1 . ( 1 ) In Eq . 1 , w is the global model weight ; Fi is the i-client local objective function ; the local loss functions fi ( w ; ξ ) are often assumed to be the same across all clients ; the local data Di can have different distribution . Following the seminal work of ( McMahan et al. , 2017 ) , extensive researches have been conducted to address various challenges in FL . For instance , ( Li et al. , 2020 ; Reddi et al. , 2020 ) aim to design efficient optimization algorithms ; ( Konečnỳ et al. , 2016 ; Luping et al. , 2019 ) try to improve the communication efficiency ; ( Bagdasaryan et al. , 2020 ; Xie et al. , 2021 ) study the security issues on both attacks and defenses . Adversaries . For any FL attack , the server is assumed to be honest , which faithfully follow the training protocols and any client can not directly get the model weight of other clients . Under these assumptions , there are two adversaries : 1 ) the malicious clients can manipulate the weights for various attack purposes , such as interfering the global model training or implanting backdoors into the global model ; 2 ) the curious server can explore the privacy information from local clients via inference attacks . Availability poisoning attacks . The goal of availability poisoning attacks ( APAs ) is for malicious local client ( s ) to destroy or reduce the global model ’ s utility ( Shen et al. , 2016 ; Sun et al. , 2018 ) . In APAs , the attacker can either control the global model ’ s utility on target tasks or decrease the most of the model ’ s utility as the optimal attack strategy . In practice , adding a large random noise can successfully worsen the global model ’ s utility but can be easily discovered by anomaly detection techniques . In this case , any advanced attack needs to know and tries to bypass the defense techniques used in training . In this work , we assume the attacker has full knowledge of training process and all possible defense techniques . Integrity backdoor attacks . The goals of integrity backdoor attack ( IBAs ) are two-fold : 1 ) the attacker aims to implant backdoors into the global model , through which the attacker can control the prediction results by injecting the trigger to any clean examples ; 2 ) at the same time , the attacker wants the global model can still perform as good as the non-attacked model on all clean inputs ( Bagdasaryan et al. , 2020 ) . Note that , IBAs are also conducted by the malicious clients in FL . Inference attacks . Many studies show that the attacker could explore the private information from the model weights , such as membership and attribute information ( Pyrgelis et al. , 2017 ; Shokri et al. , 2017 ; Ganju et al. , 2018 ; Melis et al. , 2019 ) or even recover the training samples used by clients Zhu et al . ( 2019 ) . When the honest-but-curious server becomes malicious , it can explore privacy information of each client ’ s local dataset through inference attacks ( IAs ) since the server has full access to clients ’ model weight . Both APAs and IBAs can be achieved via weights poisoning that modifies the local model weights . For example , as the global model converges , the deviations of local models start to cancel out , i.e. , ∑nt−1 i=1 ( w t i − wt−1 ) ≈ 0 given as in Eq . 2 of Bagdasaryan et al . ( 2020 ) , where t and i the communication round and client index respectively , wt and wti are the global and local model respectively , and nt is the number of participating clients . If the adversary aims to replace global model wt with a target model X , then it could propose to upload a local model weight wtA = ntX − ( nt − 1 ) wt−1 − ∑nt−1 i=1 ( w t i − wt−1 ) ≈ nt ( X − wt−1 ) + wt−1 as given in Eq . 3 in Bagdasaryan et al . ( 2020 ) . Prior works ( Bagdasaryan et al. , 2020 ; Wang et al. , 2020 ; Xie et al. , 2019 ) demonstrate the effectiveness of both APAs and IBAs when the training algorithm is FedAvg . IAs can be easily achieved by many existing advanced attack methods once they have the full access to the client model Shokri et al . ( 2017 ) ; Hu et al . ( 2021 ) . Related works . To address the various attacks , many prior defense works have been proposed and achieved good performance against the attacks . However , prior defense works only address one aspects of aforementioned three attacks . For availability poisoning attacks , Steinhardt et al . ( 2017 ) proposed a defending framework , which can be applied to defenders that remove outliers and then minimize a margin-based loss on the remaining data . Under assumptions , this framework provides the approximate upper bounds on the efficacy of any data poisoning attack . However , this framework does not fit in the FL setting , as it needs the access to all private data . Blanchard et al . ( 2017 ) studied the presences of Byzantine adversaries and proposed Krum aggregation rule to defend under the assumption that the participants ’ training data are i.i.d , which is not necessarily true in the FL setting . And indeed as argued in ( Bagdasaryan et al. , 2020 ) , Krum can be used by the adversaries to make the attack more effective . FoolsGold ( Fung et al. , 2018 ) can mitigate sybils data poisoning attacks under the assumption that the honest clients can be separated from sybils by the diversity of their gradient updates . As discussed by the authors , FoolsGold is not successful at mitigating attacks initiated by a single adversary . For integrity backdoor attacks , Xie et al . ( 2021 ) provided a general framework that is certifiably robust to the backdoor attacks under the FL setting via the model weight smoothing . However , the convexity assumption is imposed on the loss function , which limits certified robustness to more challenging and widely used deep neural network . Ozdayi et al . ( 2021 ) proposed a defense approach based on adjusting server ’ s learning rate with the guidance of sign information of agents ’ updates . For inference attacks , differential privacy is widely adopted approach to defend , where are judiciously random noise added on either the clients ’ model update or the global model , e.g. , Geyer et al . ( 2017 ) , but it suffers from the trade-off between privacy and accuracy . Compared to this work , neither of these works can address three types of attacks simultaneously . 3 METHOD . We introduce FEDDISCRETE , a federated learning framework with a simply yet effective discrete mechanism and the flexibility to accommodate various FL optimization algorithms . A complete description is given in Algorithm 1 . FEDDISCRETEdifferences FEDDISCRETE consists of three stages , namely , local training , discretization and aggregation , which are discussed in details . Local Training . Since we mainly focus on the cross-device setting , in each communication round t , the server first selects a subset of |St| = K clients to participate the current round of training and broadcasts the global model wt to the selected clients . The clients who receives the global model then performs local training using any appropriate algorithm to generate a new local model , i.e . wt+1i for all i ∈ St . The ith client can 1 ) use the local stochastic gradient descent ( SGD ) method as is considered in FedAVG to perform a fixed number of SGD steps to improve communication efficiency ; 2 ) use adaptive method ( Wang et al. , 2021b ) to improve the convergence ; 3 ) approximately minimize Fi ( w ) + µ2 ‖w−w t‖2 for some µ > 0 as proposed in FedProx to accommodate the system and data heterogeneity ( Li et al. , 2020 ) . Discretization . For all i ∈ St , the ith local client computes the maximum and minimum values of wti and sends noise-perturbed maximum value u t i and minimal value l t i back to the server , where the noise is added to protect the privacy of the local training data and the noise is sampled and added in a way that to guarantee uti and l t i are the valid upper bound and lower bound of w t+1 i elementwise . Once the server received the client-wise upper-bounds and lower-bounds , it computes the minimum of the lower bounds ltmin and the maximum of the upper bounds u t max to prepare the inputs for the discretization mechanism ( see Definition 3.1 ) , which outputs an unbiased estimator of the input ( see Lemma 4.1 ) . For all i ∈ St , the ith client discretizes its continuous model wti into M ( wti ; ltmin , utmax ) and uploadsM ( wti ; ltmin , utmax ) to the server for aggregation . Aggregation . Rather than simply aggregating { M ( wti ; ltmin , utmax ) } i∈St , the sever first inspects whether each coordinate of wti is either l t min or u t max to exclude the any potential malicious adversaries that attempt to bypass the discretization process . And the server only aggregates the local model weights that pass the sanity check . Also when the server computes ltmin and u t max , it can be more conservative by discarding the extreme values in { lti } i∈St and { uti } i∈St by setting thresholds on the lower quantile of { lti } i∈St and upper quantile of { uti } i∈St to prevent adversaries proposing extremely large upper bounds and/or small lower bounds . For the ease of presentation , we do not include this choice in the description of Algorithm 1 . Definition 3.1 ( Discretization Mechanism ) . For any ( w , l , u ) ∈ R×R×R with l ≤ w ≤ u , define the discretization mechanism as M ( w ; l , u ) = { u , w.p . w−lu−l l , w.p . u−wu−l , ( 2 ) where w.p . is the shorthand notation for “ with probability ” . Algorithm 1 FEDDISCRETE 1 : Input : Initial model weight w0 ∈ Rd , total training rounds T , the participants size K , and the learning rate { ηt } T−1t=0 and a positive sequence { σi } Ni=1 . 2 : for t = 0 , 1 , 2 , . . . , T − 1 do 3 : The sever randomly selects an index set St with |St| = K and broadcasts wt to the client i if i ∈ St. 4 : For the ith client , where i ∈ St , it performs the local training to obtain wt+1i and samples two random variables ξti and ζ t i from the truncated Gaussian TN ( 0 , σi ; 0 , 1 ) . Then it computes and uploads lti = minj∈ [ d ] { [ wti ] j −wt } − ξti and uti = maxj∈ [ d ] { [ wti ] j −wt } + ζti to the server . 5 : The sever computes lt = mini∈St { lti } and ut = maxi∈St { uti } then broadcasts lt , ut to clients in St. 6 : For the ith client , where i ∈ St , it applies the discretization mechanismM is an elementwise fashion and uploads the discrete model weightM ( wt+1i −wt ; lt , ut ) to the server . 7 : The server conducts the sanity check for { M ( wt+1i −wt ; lt , ut ) } i∈St , i.e. , validates whether all elements of wt+1i are either l t or ut . Form the set S′t ⊆ St to collect all the clients ’ index passing the sanity check and compute wt+1 = wt+ηt 1|S′t| ∑ i∈S′t M ( wt+1i − wt ; lt , ut ) . 8 : end for We close this section by making following comments . • Discretization is a widely applied technique , for example , in influence maximization Kempe et al . ( 2003 ) , the algorithm needs to simulate the propagation from a probabilistic graph that is consists of a set of edges with activation probabilities in [ 0 , 1 ] . It is also can be regarded as a special form of quantization that is widely used in the distributed optimization ( Alistarh et al. , 2017 ; Reisizadeh et al. , 2020 ) to improve communication efficiency . Compared with prior discretization works , this is the first work to adopt discretization to protect FL from weight poisoning attack . • Compared with the FedAvg , although an additional round of communication is required in FEDDISCRETE to perform the discretization mechanism . We argue that FEDDISCRETE is more communication-efficient than FedAvg . Assume each scalar takes 64bits , then for the t-th round , the total bits communicated are 128dK for FedAvg , while for FEDDISCRETE are 64dK + 128K + dK . For large d , FEDDISCRETE communicates less bits . 1 • Although the discretization mechanism outputs an unbiased estimation of wt+1i , the variance of the estimatorM ( wt+1i ) can be large . To mitigate this issue , one can compute more finely-grained upper bounds and lower bounds . For example , assume each client wants to train a neural network . Then , instead of computing the maximum and minimum values over the entire w , the client could compute layer-wise maximum and minimum values . And the discretization mechanism could be applied layer-wise with tighter upper bounds and lower bounds . This leads to smaller variance in the estimator at the cost of increasing the communicated bits . • Intuitively , the discretization mechanism is effective in defending weight poisoning attacks as the judiciously crafted adversary model is discretized . For example , consider the model replacement attack in Eq . 3 of Bagdasaryan et al . ( 2020 ) , once the malicious model wtA is discretized intoM ( wtA ) , the model replacement becomes ineffective .

Summary: Federated learning (FL) has been shown to be vulnerable to weight poisoning attacks. An attacker who controls malicious clients can poison the clients’ model weights such that a backdoor to perform availability poisoning attacks, integrity backdoor attacks and inference attacks. In this work, the authors proposed a new FL algorithm called FedDiscrete, which probabilistically discretizes the clients’ model weights into two different values. The authors derived the convergence of FedDiscrete and empirically showed its performance against existing attacks. However, I am worried about the theoretical analysis on the robustness, as well as the empirical robustness against adaptive attacks.