Adversarially Robust Federated Learning for Neural Networks

In federated learning , data is distributed among local clients which collaboratively train a prediction model using secure aggregation . To preserve the privacy of the clients , the federated learning paradigm requires each client to maintain a private local training data set , and only uploads its summarized model updates to the server . In this work , we show that this paradigm could lead to a vulnerable model , which collapses in performance when the corrupted data samples ( under adversarial manipulations ) are used for prediction after model deployment . To improve model robustness , we first decompose the aggregation error of the central server into bias and variance , and then , propose a robust federated learning framework , named Fed BVA , that performs on-device adversarial training using the bias-variance oriented adversarial examples supplied by the server via asymmetrical communications . The experiments are conducted on multiple benchmark data sets using several prevalent neural network models , and the empirical results show that our framework is robust against white-box and black-box adversarial corruptions under both IID and non-IID settings . 1 INTRODUCTION . The explosive amount of decentralized user data collected from the ever-growing usage of smart devices , e.g. , smartphones , wearable devices , home sensors , etc. , has led to a surge of interest in the field of decentralized learning . To protect the privacy-sensitive data of the clients , federated learning ( McMahan et al. , 2017 ; Yang et al. , 2019 ) has been proposed . Federated learning only allows a group of clients to train local models using their own data , and then collectively merges the model updates on a central server using secure aggregation ( Acar et al. , 2018 ) . Due to its high privacy-preserving property , federated learning has attracted much attention in recent years along with the prevalence of efficient light-weight deep models ( Howard et al. , 2017 ) and low-cost network communications ( Wen et al. , 2017 ; Konečnỳ et al. , 2016 ) . In federated learning , the central server only inspects the secure aggregation of the local models as a whole . Consequently , it is susceptible to clients ’ corrupted updates ( e.g. , system failures , etc ) . Recently , multiple robust federated learning models ( Fang et al. , 2019 ; Pillutla et al. , 2019 ; Portnoy & Hendler , 2020 ; Mostafa , 2019 ) have been proposed . These works only focus on performing clientlevel robust training or designing server-level aggregation variants with hyper-parameter tuning for Byzantine failures . However , none of them have the ability to mitigate the federated learning ’ s vulnerability when the adversarial manipulations are present during testing , which as we shown in Section 4.1 that is mainly due to the generalization error in the model aggregation . Our work bridges this gap by investigating the error incurred during the aggregation of federated learning from the perspective of bias-variance decomposition ( Domingos , 2000 ; Valentini & Dietterich , 2004 ) . Specifically , we show that the generalization error of the aggregated model on the central server can be decomposed as the combination of bias ( triggered by the main prediction of these clients ) and variance ( triggered by the variations among clients ’ predictions ) . Next , we propose to perform the local robust training on clients by supplying them with a tiny amount of the bias-variance perturbed examples generated from the central server via asymmetrical communications . The experiments are conducted on neural networks with cross-entropy loss , however , other loss functions are also applicable as long as their gradients w.r.t . bias and variance are tractable to estimate . In this way , any gradient-based adversarial training strategies ( Goodfellow et al. , 2015 ; Madry et al. , 2018 ) could be used . Compared with previous work , our major contributions include : • We provide the exact solution of bias-variance analysis w.r.t . the generalization error which is perfectly suitable for neural network based federated learning . As a comparison , performing adversarial attacks or training with conventional federated learning methods will only focus on the bias of the central model but ignore the variance . • We demonstrate that the conventional federated learning framework is vulnerable to the strong attacking methods with increasing communication rounds even if the adversarial training using the locally generated adversarial examples is performed on each client . • Without violating the clients ’ privacy , we show that providing a tiny amount of bias-variance perturbed data from the central server to the clients through asymmetrical communication could dramatically improve the robustness of the training model under various settings . 2 PRELIMINARIES . 2.1 SETTINGS . In federated learning , there is a central server and K different clients , each with access to a private training set Dk = { ( xki , tki ) } nk i=1 , where x k i , tki , and nk are the features , label , and number of training examples in the kth client ( k = 1 , · · · , K ) . Each data Dk is exclusively owned by client k and will not be shared with the central server or other clients . In addition , there is a small public training set Ds = { ( xsj , tsj ) } ns j=1 with ns training examples from the server that is shared with clients , where ns ⌧ PK k=1 nk . Note that this will not break the privacy constraints , for example , hospitals ( local devices ) that contribute to a federated learned medical image diagnosis system could take a few publicly accessible images as additional inputs . The goal of federated learning is to train a global classifier f ( · ) using knowledge from all the clients such that it generalizes well over test data Dtest . The notation used in this paper is summarized in the Appendix ( see Table 4 ) . 2.2 PROBLEM DEFINITION . In this paper , we study the adversarial robustness of neural networks1 in federated learning setting , and we define robust decentralized learning as follows . Definition 2.1 . ( Adversarially Robust Federated Learning ) Input : ( 1 ) A set of private training data { Dk } Kk=1 on K different clients ; ( 2 ) Tiny amount of training data Ds on the central server ; ( 3 ) Learning algorithm f ( · ) and loss function L ( · , · ) . Output : A trained model on the central server that is robust against adversarial perturbation . We would like to point out that our problem definition has the following properties : Asymmetrical communication : The asymmetrical communication between each client and server cloud is allowed : the server provides both global model parameters and limited shared data to the clients ; while each client only uploads its local model parameters back to the server . Data distribution : All training examples on the clients and the server are assumed to follow the same data distribution . However , the experiments show that our proposed algorithm also achieves outstanding performance under the non-IID setting , which could be common among personalized clients in real scenarios . Shared learning algorithm : All the clients are assumed to use the identical model f ( · ) , including architectures as well as hyper-parameters ( e.g. , learning rate , local epochs , local batch size ) . Remark . The basic assumption of this problem setting is that the learning process is clean ( no malicious behaviors are observed during training ) , however , the intentionally generated adversarial poisoning data will be mixed with clean data during training . The eventual trained model being deployed on the devices will be robust against potential future adversarial attacks . 2.3 BIAS-VARIANCE TRADE-OFF . Following ( Domingos , 2000 ; Valentini & Dietterich , 2004 ) , we define the optimal prediction , main prediction as well as the bias , variance , and noise for any real-valued loss function L ( · , · ) as follows : Definition 2.2 . ( Optimal Prediction and Main Prediction ) Given loss function L ( · , · ) and learning algorithm f ( · ) , optimal prediction y⇤ and main prediction ym for an example are defined as : y⇤ ( x ) = argmin y Et [ L ( y , t ) ] and ym ( x ) = argmin y0 ED [ L ( fD ( x ) , y0 ) ] ( 1 ) where t and D are viewed as the random variables to denote the class label and training set , and fD denotes the model trained on D. In short , the main prediction is the prediction whose average loss relative to all the predictions over data distributions is minimum , e.g. , the main prediction for zeroone loss is the mode of predictions . In this work , we show that the main prediction is the average prediction of client models for mean squared ( MSE ) loss and cross-entropy ( CE ) loss in Section 4.1 . 1Our theoretical contribution mainly focuses on classification using neural networks with cross-entropy loss and mean squared loss . However , the proposed framework is generic to allow the use of other classification loss functions as well . Definition 2.3 . ( Bias , Variance and Noise ) Given a loss function L ( · , · ) and a learning algorithm f ( · ) , the expected loss ED , t [ L ( fD ( x ) , t ) ] for an example x can be decomposed2 into bias , variance and noise as follows : B ( x ) = L ( ym , y⇤ ) and V ( x ) = ED [ L ( fD ( x ) , ym ) ] and N ( x ) = Et [ L ( y⇤ , t ) ] ( 2 ) In short , bias is the loss incurred by the main prediction w.r.t . the optimal prediction , and variance is the average loss incurred by predictions w.r.t . the main prediction . Noise is conventionally assumed to be irreducible and independent to f ( · ) . Remark . Our definitions on optimal prediction , main prediction , bias , variance and noise slightly differ from previous ones ( Domingos , 2000 ; Valentini & Dietterich , 2004 ) . For example , conventional optimal prediction was defined as y⇤ ( x ) = argminy Et [ L ( t , y ) ] , and it is equivalent to our definition when loss function is symmetric over its arguments , i.e. , L ( y1 , y2 ) = L ( y2 , y1 ) . Note that this decomposition holds for any real-valued loss function in the binary setting ( Domingos , 2000 ) with a bias & variance trade-off coefficient that has a closed-form expression . For multi-class set- ting , we inherit their definition of bias & variance directly , and treat the trade-off coefficient as a hyper-parameter to tune because no closed-form expression of is available . 3 THE PROPOSED FRAMEWORK . A typical framework ( Kairouz et al. , 2019 ) of privacy-preserving federated learning can be summarized as follows : ( 1 ) Client Update : Each client updates local model parameters wk by minimizing the empirical loss over its own training set ; ( 2 ) Forward Communication : Each client uploads its model parameter update to the central server ; ( 3 ) Server Update : It synchronously aggregates the received parameters ; ( 4 ) Backward Communication : The global parameters are sent back to the clients . Our framework follows the same paradigm but with substantial modifications as below . Server Update . The server has two components : The first one uses FedAvg ( McMahan et al. , 2017 ) algorithm to aggregate the local models ’ parameters , i.e. , wG = Aggregate ( w1 , · · · , wK ) =PK k=1 nk n wk where n = PK k=1 nk and wk is the model parameters in the k th client . Meanwhile , another component is designed to produce adversarially perturbed examples which could be induced by a poisoning attack algorithm for the usage of robust adversarial training . It has been well studied ( Belkin et al. , 2019 ; Domingos , 2000 ; Valentini & Dietterich , 2004 ) that in the classification setting , the generalization error of a learning algorithm on an example is determined by the bias , variance , and irreducible noise as defined in Eq . ( 2 ) . Similar to the previous work , we also assume a noise-free learning scenario where the class label t is a deterministic function of x ( i.e. , if x is sampled repeatedly , the same values of its class t will be observed ) . This motivates us to generate the adversarial examples by attacking the bias and variance induced by clients ’ models as : max x̂2⌦ ( x ) B ( x̂ ; w1 , · · · , wK ) + V ( x̂ ; w1 , · · · , wK ) 8 ( x , t ) 2 Ds ( 3 ) where B ( x̂ ; w1 , · · · , wK ) and V ( x̂ ; w1 , · · · , wK ) could be empirically estimated from a finite number of clients ’ parameters trained on local training sets { D1 , D2 , · · · , DK } . Here is a hyperparameter to measure the trade-off of bias and variance , and ⌦ ( x ) is the perturbation constraint . Note that Ds ( on the server ) is the candidate subset of all available training examples that would lead to their perturbed counterparts . This is a more feasible setting as compared to generating adversarial examples on clients ’ devices because the server usually has much powerful computational capacity in real scenarios that allows the usage of flexible poisoning attack algorithms . In this case , both poisoned examples and server model parameters would be sent back to each client ( Backward Communication ) , while only clients ’ local parameters would be uploaded to the server ( Forward Communication ) , i.e. , the asymmetrical communication as discussed in Section 2.2 . Client Update . The robust training of one client ’ s prediction model ( i.e. , wk ) can be formulated as the following minimization problem . min wk 0 @ nkX i=1 L ( fDk ( x k i ; wk ) , t k i ) + nsX j=1 L ( fDk ( x̂ s j ; wk ) , t s j ) 1 A ( 4 ) where x̂sj 2 ⌦ ( xsj ) is the perturbed examples that is asymmetrically transmitted from the server . 2This decomposition is based on the weighted sum of bias , variance , and noise . In general , t is a non-deterministic function ( Domingos , 2000 ) of x when the irreducible noise is considered . Namely , if x is sampled repeatedly , different values of t will be observed . Remark . Intuitively , the bias measures the systematic loss of a learning algorithm , and the vari- ance measures the prediction consistency of the learner over different training sets . Therefore , our robust federated learning framework has the following advantages : ( i ) it encourages the clients to consistently produce the optimal prediction for perturbed examples , thereby leading to a better generalization performance ; ( ii ) local adversarial training on perturbed examples allows to learn a robust local model , and thus a robust global model could be aggregated from clients . Theoretically , we could still have another alternative robust federated training strategy : min wk nkX i=1 max x̂ki 2⌦ ( xki ) L ( f ( x̂ki ; wk ) , t k i ) 8k 2 { 1 , 2 , · · · , K } ( 5 ) where the perturbed training examples of each client k is generated on local devices from Dk instead of transmitted from the server . This min-max formula is similar to ( Madry et al. , 2018 ; Tramèr et al. , 2018 ) where the inner maximization problem synthesizes the adversarial counterparts of clean examples , while the outer minimization problem finds the optimal model parameters over perturbed training examples . Thus , each local robust model is trained individually , nevertheless , poisoning attacks on device will largely increase the computational cost and memory usage . Meanwhile , it only considers the client-specific loss and is still vulnerable against adversarial examples with increasing communication rounds . Both phenomena are observed in our experiments ( see Fig . 4 and Fig . 5 ) .

Summary: The paper studies adversarial robustness in the context of federated learning. The authors provide an algorithm for adversarial training that generates adversarial examples on a trusted public dataset and iteratively sends them to the clients, so that they can perform learning on the adversarial examples as well. Notably, the adversarial examples are created by inspecting both the bias and the variance of the current set of models. The method is tested empirically on a wide range of datasets and compared to adversarial training using the local clients' data.