FedDiscrete: A Secure Federated Learning Algorithm Against Weight Poisoning

Federated learning ( FL ) is a privacy-aware collaborative learning paradigm that allows multiple parties to jointly train a machine learning model without sharing their private data . However , recent studies have shown that FL is vulnerable to availability poisoning attacks , integrity backdoor attacks and inference attacks via weight poisoning and inference . In this paper , we propose a probabilistic discretization mechanism on the client side , which transforms the client ’ s model weight into a vector that can only have two different values but still guarantees that the server obtains an unbiased estimation of the client ’ s model weight . We theoretically analyze the utility , robustness , and convergence of our proposed discretization mechanism and empirically verify its superior robustness against various weight-based attacks under the cross-device FL setting . 1 INTRODUCTION . Federated learning ( FL ) is an emerging privacy-aware framework that trains a machine learning model across multiple parties without accessing their local private data ( Konečnỳ et al. , 2016 ; McMahan et al. , 2017 ) . In FL , each client first trains the local model using its private data and then sends the model gradients to an honest central server . The central server aggregates all local model gradients to form a global model , which is sent back to the clients for the next round of training . However , sharing model gradients might still leads to security concerns and privacy leakage . Recent studies have shown that FL is vulnerable to various model weight poisoning attacks . The adversarial client ( s ) can stealthily manipulate the global model via modifying the local model updates to achieve attack goals like , preventing model convergence , implanting backdoors into the global model , and inferring the privacy information of clients ’ private data ( Gu et al. , 2017 ; Blanchard et al. , 2017 ; Pyrgelis et al. , 2017 ; Xie et al. , 2019 ; Bagdasaryan et al. , 2020 ; Wang et al. , 2020 ; Tang et al. , 2020 ) . To address such an issue , recent works start to explore different defense techniques to robustify FL against model weight poisoning attacks ( Blanchard et al. , 2017 ; Bagdasaryan et al. , 2020 ; Shen et al. , 2016 ; Geyer et al. , 2017 ; Fung et al. , 2018 ) . However , most existing works apply the defense or robust techniques at the server side , which changes the role of the central server from being honest to trusted . And prior approaches mostly only focus on one type of attacks . Our contributions . In this paper , we propose the FEDDISCRETE , a flexible FL framework that can combine any popular FL algorithms , for example , FedAvg ( McMahan et al. , 2017 ) and FedProx ( Li et al. , 2020 ) , with our probabilistic discretization mechanism . Theoretical analyses on the utility , robustness , and convergence are performed to show that FEDDISCRETE is inherently robust to availability poisoning attacks ( Kurita et al. , 2020 ) , integrity backdoor attacks ( Gu et al. , 2017 ) and server inference attacks ( Shokri et al. , 2017 ) . FEDDISCRETE is evaluated on four popular image classification datasets under both data are i.i.d and non-i.i.d settings . The numerical results indicate that FEDDISCRETE is robust to various weight-based attacks . 2 BACKGROUNDS AND RELATED WORKS . In this section , we provide necessary background information on concepts thorough the paper and formalize the problem to be solved . Federated learning ( FL ) . FL is a privacy-aware collaborative learning , where N clients and one trusted server work together to learn a global model ( McMahan et al. , 2017 ) . Depending on the application scenarios , the number of clients n can range from as small as two to several hundreds in the cross-silo setting or can easily go beyond millions in the cross-device setting . A classical way of formulating FL into an optimization problem is ( Wang et al. , 2021a ) min w∈Rd F ( w ) : = N∑ i=1 piFi ( w ) , where Fi ( w ) = 1 |Di| ∑ ξ∈Di fi ( w ; ξ ) and N∑ i=1 pi = 1 . ( 1 ) In Eq . 1 , w is the global model weight ; Fi is the i-client local objective function ; the local loss functions fi ( w ; ξ ) are often assumed to be the same across all clients ; the local data Di can have different distribution . Following the seminal work of ( McMahan et al. , 2017 ) , extensive researches have been conducted to address various challenges in FL . For instance , ( Li et al. , 2020 ; Reddi et al. , 2020 ) aim to design efficient optimization algorithms ; ( Konečnỳ et al. , 2016 ; Luping et al. , 2019 ) try to improve the communication efficiency ; ( Bagdasaryan et al. , 2020 ; Xie et al. , 2021 ) study the security issues on both attacks and defenses . Adversaries . For any FL attack , the server is assumed to be honest , which faithfully follow the training protocols and any client can not directly get the model weight of other clients . Under these assumptions , there are two adversaries : 1 ) the malicious clients can manipulate the weights for various attack purposes , such as interfering the global model training or implanting backdoors into the global model ; 2 ) the curious server can explore the privacy information from local clients via inference attacks . Availability poisoning attacks . The goal of availability poisoning attacks ( APAs ) is for malicious local client ( s ) to destroy or reduce the global model ’ s utility ( Shen et al. , 2016 ; Sun et al. , 2018 ) . In APAs , the attacker can either control the global model ’ s utility on target tasks or decrease the most of the model ’ s utility as the optimal attack strategy . In practice , adding a large random noise can successfully worsen the global model ’ s utility but can be easily discovered by anomaly detection techniques . In this case , any advanced attack needs to know and tries to bypass the defense techniques used in training . In this work , we assume the attacker has full knowledge of training process and all possible defense techniques . Integrity backdoor attacks . The goals of integrity backdoor attack ( IBAs ) are two-fold : 1 ) the attacker aims to implant backdoors into the global model , through which the attacker can control the prediction results by injecting the trigger to any clean examples ; 2 ) at the same time , the attacker wants the global model can still perform as good as the non-attacked model on all clean inputs ( Bagdasaryan et al. , 2020 ) . Note that , IBAs are also conducted by the malicious clients in FL . Inference attacks . Many studies show that the attacker could explore the private information from the model weights , such as membership and attribute information ( Pyrgelis et al. , 2017 ; Shokri et al. , 2017 ; Ganju et al. , 2018 ; Melis et al. , 2019 ) or even recover the training samples used by clients Zhu et al . ( 2019 ) . When the honest-but-curious server becomes malicious , it can explore privacy information of each client ’ s local dataset through inference attacks ( IAs ) since the server has full access to clients ’ model weight . Both APAs and IBAs can be achieved via weights poisoning that modifies the local model weights . For example , as the global model converges , the deviations of local models start to cancel out , i.e. , ∑nt−1 i=1 ( w t i − wt−1 ) ≈ 0 given as in Eq . 2 of Bagdasaryan et al . ( 2020 ) , where t and i the communication round and client index respectively , wt and wti are the global and local model respectively , and nt is the number of participating clients . If the adversary aims to replace global model wt with a target model X , then it could propose to upload a local model weight wtA = ntX − ( nt − 1 ) wt−1 − ∑nt−1 i=1 ( w t i − wt−1 ) ≈ nt ( X − wt−1 ) + wt−1 as given in Eq . 3 in Bagdasaryan et al . ( 2020 ) . Prior works ( Bagdasaryan et al. , 2020 ; Wang et al. , 2020 ; Xie et al. , 2019 ) demonstrate the effectiveness of both APAs and IBAs when the training algorithm is FedAvg . IAs can be easily achieved by many existing advanced attack methods once they have the full access to the client model Shokri et al . ( 2017 ) ; Hu et al . ( 2021 ) . Related works . To address the various attacks , many prior defense works have been proposed and achieved good performance against the attacks . However , prior defense works only address one aspects of aforementioned three attacks . For availability poisoning attacks , Steinhardt et al . ( 2017 ) proposed a defending framework , which can be applied to defenders that remove outliers and then minimize a margin-based loss on the remaining data . Under assumptions , this framework provides the approximate upper bounds on the efficacy of any data poisoning attack . However , this framework does not fit in the FL setting , as it needs the access to all private data . Blanchard et al . ( 2017 ) studied the presences of Byzantine adversaries and proposed Krum aggregation rule to defend under the assumption that the participants ’ training data are i.i.d , which is not necessarily true in the FL setting . And indeed as argued in ( Bagdasaryan et al. , 2020 ) , Krum can be used by the adversaries to make the attack more effective . FoolsGold ( Fung et al. , 2018 ) can mitigate sybils data poisoning attacks under the assumption that the honest clients can be separated from sybils by the diversity of their gradient updates . As discussed by the authors , FoolsGold is not successful at mitigating attacks initiated by a single adversary . For integrity backdoor attacks , Xie et al . ( 2021 ) provided a general framework that is certifiably robust to the backdoor attacks under the FL setting via the model weight smoothing . However , the convexity assumption is imposed on the loss function , which limits certified robustness to more challenging and widely used deep neural network . Ozdayi et al . ( 2021 ) proposed a defense approach based on adjusting server ’ s learning rate with the guidance of sign information of agents ’ updates . For inference attacks , differential privacy is widely adopted approach to defend , where are judiciously random noise added on either the clients ’ model update or the global model , e.g. , Geyer et al . ( 2017 ) , but it suffers from the trade-off between privacy and accuracy . Compared to this work , neither of these works can address three types of attacks simultaneously . 3 METHOD . We introduce FEDDISCRETE , a federated learning framework with a simply yet effective discrete mechanism and the flexibility to accommodate various FL optimization algorithms . A complete description is given in Algorithm 1 . FEDDISCRETEdifferences FEDDISCRETE consists of three stages , namely , local training , discretization and aggregation , which are discussed in details . Local Training . Since we mainly focus on the cross-device setting , in each communication round t , the server first selects a subset of |St| = K clients to participate the current round of training and broadcasts the global model wt to the selected clients . The clients who receives the global model then performs local training using any appropriate algorithm to generate a new local model , i.e . wt+1i for all i ∈ St . The ith client can 1 ) use the local stochastic gradient descent ( SGD ) method as is considered in FedAVG to perform a fixed number of SGD steps to improve communication efficiency ; 2 ) use adaptive method ( Wang et al. , 2021b ) to improve the convergence ; 3 ) approximately minimize Fi ( w ) + µ2 ‖w−w t‖2 for some µ > 0 as proposed in FedProx to accommodate the system and data heterogeneity ( Li et al. , 2020 ) . Discretization . For all i ∈ St , the ith local client computes the maximum and minimum values of wti and sends noise-perturbed maximum value u t i and minimal value l t i back to the server , where the noise is added to protect the privacy of the local training data and the noise is sampled and added in a way that to guarantee uti and l t i are the valid upper bound and lower bound of w t+1 i elementwise . Once the server received the client-wise upper-bounds and lower-bounds , it computes the minimum of the lower bounds ltmin and the maximum of the upper bounds u t max to prepare the inputs for the discretization mechanism ( see Definition 3.1 ) , which outputs an unbiased estimator of the input ( see Lemma 4.1 ) . For all i ∈ St , the ith client discretizes its continuous model wti into M ( wti ; ltmin , utmax ) and uploadsM ( wti ; ltmin , utmax ) to the server for aggregation . Aggregation . Rather than simply aggregating { M ( wti ; ltmin , utmax ) } i∈St , the sever first inspects whether each coordinate of wti is either l t min or u t max to exclude the any potential malicious adversaries that attempt to bypass the discretization process . And the server only aggregates the local model weights that pass the sanity check . Also when the server computes ltmin and u t max , it can be more conservative by discarding the extreme values in { lti } i∈St and { uti } i∈St by setting thresholds on the lower quantile of { lti } i∈St and upper quantile of { uti } i∈St to prevent adversaries proposing extremely large upper bounds and/or small lower bounds . For the ease of presentation , we do not include this choice in the description of Algorithm 1 . Definition 3.1 ( Discretization Mechanism ) . For any ( w , l , u ) ∈ R×R×R with l ≤ w ≤ u , define the discretization mechanism as M ( w ; l , u ) = { u , w.p . w−lu−l l , w.p . u−wu−l , ( 2 ) where w.p . is the shorthand notation for “ with probability ” . Algorithm 1 FEDDISCRETE 1 : Input : Initial model weight w0 ∈ Rd , total training rounds T , the participants size K , and the learning rate { ηt } T−1t=0 and a positive sequence { σi } Ni=1 . 2 : for t = 0 , 1 , 2 , . . . , T − 1 do 3 : The sever randomly selects an index set St with |St| = K and broadcasts wt to the client i if i ∈ St. 4 : For the ith client , where i ∈ St , it performs the local training to obtain wt+1i and samples two random variables ξti and ζ t i from the truncated Gaussian TN ( 0 , σi ; 0 , 1 ) . Then it computes and uploads lti = minj∈ [ d ] { [ wti ] j −wt } − ξti and uti = maxj∈ [ d ] { [ wti ] j −wt } + ζti to the server . 5 : The sever computes lt = mini∈St { lti } and ut = maxi∈St { uti } then broadcasts lt , ut to clients in St. 6 : For the ith client , where i ∈ St , it applies the discretization mechanismM is an elementwise fashion and uploads the discrete model weightM ( wt+1i −wt ; lt , ut ) to the server . 7 : The server conducts the sanity check for { M ( wt+1i −wt ; lt , ut ) } i∈St , i.e. , validates whether all elements of wt+1i are either l t or ut . Form the set S′t ⊆ St to collect all the clients ’ index passing the sanity check and compute wt+1 = wt+ηt 1|S′t| ∑ i∈S′t M ( wt+1i − wt ; lt , ut ) . 8 : end for We close this section by making following comments . • Discretization is a widely applied technique , for example , in influence maximization Kempe et al . ( 2003 ) , the algorithm needs to simulate the propagation from a probabilistic graph that is consists of a set of edges with activation probabilities in [ 0 , 1 ] . It is also can be regarded as a special form of quantization that is widely used in the distributed optimization ( Alistarh et al. , 2017 ; Reisizadeh et al. , 2020 ) to improve communication efficiency . Compared with prior discretization works , this is the first work to adopt discretization to protect FL from weight poisoning attack . • Compared with the FedAvg , although an additional round of communication is required in FEDDISCRETE to perform the discretization mechanism . We argue that FEDDISCRETE is more communication-efficient than FedAvg . Assume each scalar takes 64bits , then for the t-th round , the total bits communicated are 128dK for FedAvg , while for FEDDISCRETE are 64dK + 128K + dK . For large d , FEDDISCRETE communicates less bits . 1 • Although the discretization mechanism outputs an unbiased estimation of wt+1i , the variance of the estimatorM ( wt+1i ) can be large . To mitigate this issue , one can compute more finely-grained upper bounds and lower bounds . For example , assume each client wants to train a neural network . Then , instead of computing the maximum and minimum values over the entire w , the client could compute layer-wise maximum and minimum values . And the discretization mechanism could be applied layer-wise with tighter upper bounds and lower bounds . This leads to smaller variance in the estimator at the cost of increasing the communicated bits . • Intuitively , the discretization mechanism is effective in defending weight poisoning attacks as the judiciously crafted adversary model is discretized . For example , consider the model replacement attack in Eq . 3 of Bagdasaryan et al . ( 2020 ) , once the malicious model wtA is discretized intoM ( wtA ) , the model replacement becomes ineffective .

Summary: This paper proposes a secure federated learning framework against weight poisoning. The key component in the framework is the discretization mechanism which works well with a sufficient number of clients. Theoretical analysis of the robustness and convergence is provided. Lastly, numerical analysis is provided to verify the performance of the proposed method.