F^2ed-Learning: Good Fences Make Good Neighbors

1 INTRODUCTION . Federated learning ( FL ) has drawn numerous attention in the past few years as a new distributed learning paradigm . In federated learning , the users collaboratively train a model with the help of a centralized server when all the data is held locally to preserve the users ’ privacy . The privacy guarantee can be further enhanced using secure aggregation technique ( Bonawitz et al. , 2017 ) which hides the individual local updates and only reveals the aggregated global update . The graceful balance between utility and privacy popularizes federated learning in a variety of sensitive applications such as Google GBoard , healthcare service and self-driving cars . The above threat model assumes that all the users honestly upload their local updates . However , it is likely that a small number of clients are malicious in a large-scale FL system with tens of thousands of clients . Besides , in most SGD-based FL algorithms used today ( McMahan & Ramage , 2017 ) , the centralized server averages the local updates to obtain the global update , which is vulnerable to even only one malicious client . Therefore , a malicious client can arbitrarily craft its update to either prevent the global model from converging or lead it to a sub-optimal minimum . This kind of attack in federated learning is well-studied by Bhagoji et al . ( 2019 ) ; Fang et al . ( 2019 ) ; Bagdasaryan et al . ( 2020 ) ; Sun et al . ( 2020 ) . To mitigate these attacks , various Byzantine-robust FL protocols ( Blanchard et al. , 2017 ; Yin et al. , 2018 ; Fu et al. , 2019 ; Pillutla et al. , 2019 ) are proposed to reduce the impact of the contaminated updates . These protocols replace trivial averaging with well-designed Byzantine-robust mean estimators . These estimators suppress the influence of the malicious updates and output a mean estimation as accurate as possible . Nevertheless , almost all of these aggregators suffer from the curse of dimensionality . Specifically , the estimation error scales up with the size of the model in a squareroot fashion . As a concrete example , a three-layer MLP on MNIST contains more than 50,000 parameters and leads to a 223-fold increase of the estimation error , which is prohibitive in practice . Draco ( Chen et al. , 2018 ) , BULYAN ( Mhamdi et al. , 2018 ) and ByzatineSGD ( Alistarh et al. , 2018 ) are the only three works that state to yield dimension-free estimation error . However , Draco is designed for distributed learning and is incompatible with federated learning because it requires redundant updates from each worker . On the other hand , although Bulyan ( Mhamdi et al. , 2018 ) and ByzantineSGD ( Alistarh et al. , 2018 ) provide dimension-free estimation error , it is based on much stronger assumptions than other works . When the assumptions are relaxed to the common case , Bulyan ’ s estimation error still scales up with the square root of the model size as discussed in Section 2 . In addition , these robust FL protocols have incompatible implementation with secure aggregation techniques . The robust estimators have to access local updates while secure aggregation hides them from the server . Consequently , the system can not simultaneously protect the server and the clients , but has to place complete trust in either of them . The lack of two-way protection severely harms the people ’ s confidence in the FL system and prevents federated learning from being used in many sensitive applications such as home monitoring and self-driving cars . Contribution . In this paper , we propose FEDERATED LEARNING WITH FENCE , abbreviately F2ED-LEARNING . F2ED-LEARNING integrates a robust mean estimator with dimension-free error ( Steinhardt , 2018 ) and secure aggregation ( Bonawitz et al. , 2017 ) to defend against both the Byzantine malicious clients and the semi-honest server . In particular , F2ED-LEARNING is the first Byzantine-robust FL system with dimension-free estimation error . To address the incompatibility , the clients are split into multiple shards , the local updates from the same shard are securely aggregated at the centralized server , and the robust estimator is run on the aggregated local updates from different shards . Surprisingly , sharding also consolidates the independently and identically distributed ( IID ) assumption required by the robust estimator even under heterogeneous data distribution . According to Lindeberg central limitation theorem ( Lindeberg , 1922 ) , despite the heterogeneity of the individual local updates , the aggregated local updates from the shards will approximately follow an IID Gaussian distribution . 2 LOOPHOLE IN BULYAN AND BYZANTINESGD & RELATED WORK . Byzantine-robust aggregation has drawn enormous attention in the past few years due to the emergence of various distributed attacks in federated learning . Fang et al . ( 2019 ) formalize the attack as an optimization problem and successfully migrate the data poisoning attack to federated learning . The proposed attacks even work under Byzantine-robust federated learning . Sun et al . ( 2020 ) manage to launch data poisoning attack on the multi-task federated learning framework . Bhagoji et al . ( 2019 ) and Bagdasaryan et al . ( 2020 ) even manage to insert backdoor functionalities into the model via local model poisoning or local model replacement . A variety of Byzantine-robust FL protocols are proposed to defend against these attacks . Krum ( Blanchard et al. , 2017 ) picks the subset of updates with enough close neighbors and averages the subset . Yin et al . ( 2018 ) leverage traditional robust estimators like trimmed mean or median to achieve order-optimal statistical error rate under strongly convex assumptions . Yin et al . ( 2019 ) propose to use robust mean estimators to defend against saddle point attack . Mhamdi et al . ( 2018 ) pointed out that Krum , trimmed mean and median all suffers from O ( √ d ) ( d is the model size ) estimation error and proposed a general framework Bulyan to reduce the error to O ( 1 ) . However , we point out that the improvement of Bulyan actually comes from its stronger assumption . In particular , Bulyan assumes that expectation of the distance between two benign updates is bounded by a constant σ1 , while Krum assumes that the distance is bounded by σ2 √ d. We can easily see that if σ1 = σ2 √ d , Bulyan falls back to the same order of estimation error as Krum . The same loophole exists in the analysis of ByzantineSGD ( Alistarh et al. , 2018 ) . Consequently , there is no known federated learning protocol with dimension-free estimation error against Byzantine adversaries . 3 PROBLEM SETUP . In this section , we review the general pipeline of federated learning , introduce the threat model , and establish the notation system . We use bold lower-case letters ( e.g . a , b , c ) to denote vectors , and bold upper-case letters ( e.g . A , B , C ) for matrices . We denote 1 · · ·n with [ n ] . Federated Learning Pipeline . In a federated learning system , there are one server S and n clients Ci , i ∈ [ n ] . Each client holds data samples drawn from some unknown distributionD . Let ` ( w ; z ) be the loss function on the model parameter w ∈ Rd and a data sample z . Let L ( w ) = Ez∼D [ ` ( w ; z ) ] be the population loss function . Our goal is to learn the model w such that the population loss function is minimized : w∗ = arg min w∈W L ( w ) . To learn w∗ , the whole system runs a T -round federated learning protocol . Initially , the server stores a global model w0 . In the tth round , S broadcasts the global model wt−1 to them clients . The clients then run the local optimizers ( e.g . SGD , Adam , and RMSprop ) , compute the difference g ( i ) t between the optimized model and the global model , and upload the difference to S . In the tth round , S takes the average of the differences and update the global model wt = wt−1 + 1n ∑n i=1 g ( i ) t . Threat Model & Defence Goal . We assume that the centralized server S is semi-honest . The server can launch whatever attacks such as inference attack using legitimate updates from the clients as the only inputs . However , the server can not deviate from the protocol for the sake of regulation or reputation pressure . Specifically , we want to emphasize there is no collusion between the server and the clients . On the other hand , we assume that the clients are -Byzantine malicious , which means at most m clients can be malicious . Malicious clients can arbitrarily deviate from the protocol and tamper with their own updates without being detected . In this paper , we aim to achieve a dimension-free error for the mean estimation in each round . Let µ be the true mean of the benign distribution and the output of a protocol with contaminated inputs be µ̂ . The estimation error is defined by the ` 2 distance between the true mean and the estimation ‖µ̂− µ‖2 . 4 F2ED-LEARNING : ROBUST PRIVACY-PRESERVING DISTRIBUTED FL In this section , we formally present our main protocol : F2ED-LEARNING . We first introduce F2EDLEARNING step by step and formally establish the robustness and security guarantees . Then we discuss the effect of sharding on the IID distribution assumption . 4.1 F2ED-LEARNING : BYZANTINE-ROBUST PRIVACY-PRESERVING FEDERATED LEARNING The complete F2ED-LEARNING protocol is presented in Algorithm 1 . F2ED-LEARNING iteratively executes the following steps : ( 1 ) the server broadcasts the global model to the clients ; ( 2 ) clients train the global model with their local data ; ( 3 ) clients in the same shard run secure aggregation protocol to upload the mean of their updates to the server ; ( 4 ) the server aggregates the received updates using robust mean estimation ; ( 5 ) the server updates the global model with the aggregated global update . We highlight step ( 3 ) and ( 4 ) newly proposed in F2ED-LEARNING . Sharded Secure Aggregation ( line 8-10 , 12 , 16 ) . Secure aggregation is developed by Bonawitz et al . ( 2017 ) to defend against the honest but curious server in federated learning . Secure aggregation allows the server to obtain the sum of the clients ’ updates but hides the individual updates cryptographically . We introduce an oversimplified version of secure aggregation as follows for the ease of clarification . As the first step , each client samples random values for the other clients and send the values to the corresponding clients ( line 8-10 ) . After receiving all the values from other clients , each client sums up the received values and subtracts the values generated by itself to produce a random mask ( line 12 ) . Each client blinds its local update with the random mask and sends the blinded update to the server ( line 13 ) . The server then sums up all the blinded updates and obtains the summed update in plaintext ( line 15 ) . Obviously , all the masks cancel out during aggregation and the server receives the plaintext sum . Secure aggregation provides strong privacy guarantee for the clients that the server can not see anything but the aggregated global update and each client is hidden in thousands of other clients . However , in our threat model , vanilla secure aggregation is insufficient since it provides no protection for the server . As the individual updates are completely hidden , there is no way that the server can identify the malicious clients even after detecting the attack . To address the issue , we propose to split the clients into multiple shards and run secure aggregation within each shard . The size of the shards provides a trade-off between the protection for the server or the clients . The smaller the size is , the more information is revealed to the server , thus the easier to defend against Byzantine malicious clients and the harder to fight off the semi-honest server . The trade-off is discussed in detail in Section 4.2 . Algorithm 1 : F2ED-LEARNING : Robust Privacy-Preserving Sharded Federated Learning . 1 for t← [ T ] do 2 Server : 3 Split n clients into p shards { Hj } j∈ [ p ] 4 Broadcast { Hj } j∈ [ p ] and the global model wt−1 to all the clients 5 Client : 6 foreach client i ∈ [ n ] do 7 Locate its own shard j 8 Generate random masks u ( j ) ik , k ∈ Hj/i 9 foreach k ∈ Hj/i do 10 Send uik to k 11 Train the local model w ( i ) t using wt as initialization 12 g ( i ) t = w ( i ) t − wt−1 + ∑ k 6=i , i∈Hj , k∈Hj u ( j ) ik − ∑ k 6=i , i∈Hj , k∈Hj u ( j ) ki 13 Send g ( i ) t to the server 14 Server : 15 foreach Hj ∈ { Hj } j∈ [ p ] do 16 gHjt = ∑ k∈Hj g ( k ) t 17 gt = 1|Hj |FilterL2 ( { g Hj t } j∈ [ p ] ) 18 wt = wt−1 + gt Robust Mean Estimation ( line 17 ) . The core step in Byzantine-robust federated learning is to estimate the true mean of the benign updates as accurate as possible even with some malicious clients . The most commonly used aggregator , averaging , is proven to be vulnerable to even only one malicious client . All other works addressing the issue such as Krum ( Blanchard et al. , 2017 ) and Bulyan ( Mhamdi et al. , 2018 ) suffer from a dimension-dependent estimation error . Such error is unacceptable even for training a 3-layer MLP on MNIST , not to mention more complicated tasks and models such as VGG16 or ResNet50 . Actually , the above problem is well studied in statistics under the name “ robust mean estimation ” and there already exist several robust mean estimators with dimension-free estimation error ( Diakonikolas et al . ( 2019 ) ; Charikar et al . ( 2017 ) ; Steinhardt ( 2018 ) ; Cheng et al . ( 2019 ) ; Dong et al . ( 2019 ) ) . Therefore , instead of reinventing the wheel , we choose to leverage a representative robust mean estimator : FilterL2 ( Algorithm 2 ) . The following formulation is related to the presentation given in Steinhardt ( 2018 ) . Algorithm 2 : FilterL2 : dimension-free robust mean estimation ( Steinhardt ( 2018 ) ) . Input : g1 , · · · , gn ∈ Rd , η > 1 1 Let c1 , · · · , cn = 1 2 µ̂c = ( ∑n i=1 cigi ) / ( ∑n i=1 ci ) 3 Σ̂c = ( ∑n i=1 ci ( gi − µ̂c ) ( gi − µ̂c ) > ) / ( ∑n i=1 ci ) 4 Let v be the maximum eigenvector of Σ̂c , and let σ̂2c = v > Σ̂cv 5 if σ̂2c ≤ ησ2 then return µ̂c 6 else let τ = 〈xi − µ̂c〉2 , and update ci ← ci · ( 1− τi/τmax ) , where τmax = maxi τi 7 Go back to line 2 Specifically , FilterL2 assigns each update a weight and iteratively updates the weights until the weights for the malicious updates are small enough . As mentioned , FilterL2 provides dimensionfree error rate formally presented as follows . Theorem 1 ( Steinhardt ( 2018 ) ) . Let D be the honest dataset and D∗ be the contaminated version of D by inserting malicious samples . Suppose that |D∗| ≤ |D|/ ( 1 − ) , ≤ 112 , and further suppose that MEAN [ D ] = µ and ‖COV [ D ] ‖op ≤ σ2 . Then given D∗ , Algorithm 2 outputs µ̂ s.t . ‖µ̂− µ‖2 = O ( σ √ ) using POLY ( n , d ) time . Although Algorithm 2 only takes polynomial time to run , the per-round time complexity isO ( nd2 ) if implemented with power iteration . Given d is large , the running time is still quite expensive in practice . To address the issue , we cut the update vectors into k sections and apply the robust estimator to each of the sections . The acceleration scheme reduces the per-round running time to O ( nd2/k ) but increases the estimation error to O ( σ √ k ) . For instance , if we take k = √ d , the per-round running time becomes O ( nd ) while the estimation error grows to O ( σ 4 √ σ2d ) . Despite the compromise for acceleration , FilterL2 still gives the known optimal estimation error and outperforms other robust FL protocols by multiple magnitudes .

Summary: The paper considers robustness to poisoning and backdoor attacks in the context of federated learning. It proposes a defence based  on splitting the clients into shards, averaging their updates via secure aggregation and then using a robust mean estimation on top to ensure robustness. The authors point out that controlling the number of shards is a way to trade-off privacy vs robustness, thus potentially dealing with both malicious clients and an honest, but curious server. The paper provides some theoretical justification for the algorithm, as well as an experimental evaluation where its performance is tested against multiple attacks and compared to other existing methods.