MANDERA: Malicious Node Detection in Federated Learning via Ranking

1 INTRODUCTION . Federated learning ( FL ) has observed a steady rise in use across a plethora of applications . FL departs from conventional centralized learning by allowing multiple participating nodes to learn on a local collection of training data , before each respective node ’ s updates are sent to a global coordinator for aggregation . The global model collectively learns from each of these individual nodes before relaying the updated global update back to the participating nodes . With an aggregation of multiple nodes , the resulting model observes greater performance than if each node was to learn on their local subset only . FL presents two key advantages , increased privacy for the contributing node as local data is not communicated to the global coordinator , and a reduction in computation by the global node as the computation is offloaded to contributing nodes . However , the presence of malicious actors in the collaborative process may seek to poison the performance of the global model , to reduce the output performance of the model ( Chen et al. , 2017 ; Fang et al. , 2020 ; Tolpegin et al. , 2020b ) , or to embed hidden back-doors within the model ( Bagdasaryan et al. , 2020 ) . Byzantine attack aims to devastate the performance of the global model by manipulating the gradient values of malicious nodes in a certain fashion . As these attacks emerged , researchers seek to defend FL from the negative impacts of these attacks . In the literature , there are two typical defense strategies : malicious node detection and robust learning . Malicious node detection defenses by detecting malicious nodes and removing them from the aggregation ( Blanchard et al. , 2017 ; Guerraoui et al. , 2018 ; Li et al. , 2020 ; So et al. , 2021 ) . Robust learning ( Blanchard et al. , 2017 ; Yin et al. , 2018 ; Guerraoui et al. , 2018 ; Fang et al. , 2020 ; Cao et al. , 2020 ) , however , withstands a proportion of malicious nodes and defenses by reducing the negative impacts of the malicious nodes via various robust learning methods ( Wu et al. , 2020b ; Xie et al. , 2019 ; 2020 ; Cao et al. , 2021 ) . In this paper , we focus on defensing Byzantine attacks via malicious node detection . In the literature , there have been a collection of efforts along this research line . Blanchard et al . ( 2017 ) propose a defense referred to as Krum that treats local nodes whose update vector is too far away from the aggregated barycenter as malicious nodes and precludes them from the downstream aggregation . Guerraoui et al . ( 2018 ) propose Bulyan , a process that performs aggregation on subsets of node updates ( by iteratively leaving each node out ) to find a set of nodes with the most aligned updates given an aggregation rule . Xie et al . ( 2019 ) compute a Stochastic Descendant Score ( SDS ) based on the estimated descendant of the loss function , and the magnitude of the update submitted to the global node , and only include a predefined number of nodes with the highest SDS in the aggregation . On the other hand , Chen et al . ( 2021 ) propose a zero-knowledge approach to detect and remove malicious nodes by solving a weighted clustering problem . The resulting clusters update the model individually and accuracy against a validation set are checked . All nodes in a cluster with significant negative accuracy impact are rejected and removed from the aggregation step . Although the aforementioned methods try to detect malicious nodes in different ways , they all share a common nature : the detection is based on the gradient updates directly . However , it is usually the case that different dimensions of the gradients remain quite different in the range of values and follow very different distributions . This phenomena makes it very challenging to precisely detect malicious nodes directly based on the node updates , as a few dimensions often dominate the final result . Although the weighted clustering method proposed by Chen et al . ( 2021 ) could avoid this problem partially by re-weighting different update dimensions , it is often not trivial to determine the weights in a principled way . In this paper , we propose to resolve this critical problem from a novel perspective . Instead of working on the node updates directly , we propose to extract information about malicious nodes indirectly by transforming the node updates from numeric gradient values to the rank domain . Compared to the original numeric gradient values , whose distribution is difficult to model , the ranks are much easier to handle both theoretically and practically . Moreover , as ranks are scale-free , we no longer need to worry about the scale difference across different dimensions . We proved under mild conditions that the first two moments of the transformed rank vectors carry key information to detect the malicious nodes under a wide range of Byzantine attacks . Based on these theoretical results , a highly efficient method called MANDERA is proposed to separate the malicious nodes from the benign ones by clustering all local nodes into two groups based on the moments of their rank vectors . With the assumption that malicious nodes are the minority in the node pool , we can simply treat all nodes in the smaller cluster as malicious nodes and remove them from the aggregation . The contributions of this work are as follows . ( 1 ) We propose the first algorithm leveraging the rank domain of model updates to detect malicious nodes ( Figure 1 ) . ( 2 ) We provide theoretical guarantee for the detection of malicious nodes based on the rank domain under Byzantine attacks . ( 3 ) Our method does not assume knowledge on the number of malicious nodes , which is required in the learning process of prior methods . ( 4 ) We experimentally demonstrate the effectiveness and robustness of our defense on Byzantine attacks , including Gaussian attack , Sign Flipping attack and Zero Gradient attack , in addition to a more subtle Label Flipping data poisoning attack . ( 5 ) An experimental comparison between MANDERA and a collection of robust aggregation techniques are provided . The computation times are also compared , demonstrating gains of MANDERA by operating in the rank domain . 2 DEFENSE FORMALIZATION . 2.1 NOTATIONS . Suppose there are n local nodes in the federated learning framework , where n1 nodes are benign nodes whose indices are denoted by Ib and the other n0 = n − n1 nodes are malicious nodes whose indices are denoted by Im . The training model is denoted by f ( θ , D ) , where θ ∈ Rp×1 is a p-dimensional parameter vector and D is a data matrix . Denote the message matrix received from all local nodes by the central server as M ∈ Rn×p , where Mi , : denotes the message received from node i . For a benign node i , let Di be the data matrix on it with Ni as the sample size , we have Mi , : = ∂f ( θ , Di ) ∂θ . A malicious node j ∈ Im , however , tends to attack the learning system by manipulating Mj , : in some way . Hereinafter , we denote N∗ = min ( { Ni } i∈Ib ) to be the minimal sample size of the benign nodes . Given a vector of real numbers a ∈ Rp×1 , define its ranking vector as b = Rank ( a ) ∈ perm { 1 , · · · , p } , where the ranking operator Rank maps the vector a to its permutation space perm { 1 , · · · , p } which is the set of all the permutations of { 1 , · · · , p } . For example , Rank ( 1.1 , −2 , 3.2 ) = ( 2 , 3 , 1 ) . We adopt average ranking , when there are ties . With the Rank operator , we can transfer the message matrix M to a ranking matrix R by replacing its column M : ,j by the corresponding ranking vectorR : ,j = Rank ( M : ,j ) . Further define ei , 1 p p∑ j=1 Ri , j and vi , 1 p p∑ j=1 ( Ri , j − ei ) 2 to be the mean and variance of Ri , : , respectively . As it is shown in later subsections , we can judge whether node i is a malicious node based on ( ei , vi ) under various attack types . In the following , we will highlight the behaviour of the benign nodes first , and then discuss the behaviour of malicious nodes and their interactions with the benign nodes under various Byzantine attacks respectively . 2.2 BEHAVIOUR OF BENIGN NODES . As the behaviour of benign nodes does not depend on the type of Byzantine attack , we can study the statistical properties of ( ei , vi ) for a benign node i ∈ Ib before the specification of a concrete attack type . For any benign node i , the message generated for jth parameter is Mi , j = 1 Ni Ni∑ l=1 ∂f ( θ , Di , l ) ∂θj , ( 1 ) where Di , l denotes the lth sample on it . Throughout this paper , we always assume that Di , ls are independent and identically distributed ( IID ) samples drawn from a data distribution D. Under the independent data assumption , since Equation 1 tells us thatMi , j is the sample mean of IID random variables , i.e. , { ∂f ( θ , Di , l ) ∂θj } Ni l=1 , directly applying the Strong Law of Large Numbers ( SLLN ) and Central Limit Theorem ( CLT ) leads to the lemma below immediately . Lemma 1 . Under the independent data assumption , further denote µj = E ( ∂f ( θ , Di , l ) ∂θj ) and σ 2 j = Var ( ∂f ( θ , Di , l ) ∂θj ) < ∞ , with Ni going to infinity we have for ∀ j ∈ { 1 , · · · , p } Mi , j → µj a.s. and Mi , j →d N ( µj , σ 2 j /Ni ) . ( 2 ) 2.3 BEHAVIOUR OF MALICIOUS NODE UNDER THE GAUSSIAN ATTACK . Definition 1 ( Gaussian attack ) . In a Gaussian attack , the attacker manipulates malicious nodes to send Gaussian random messages to the global coordinator , i.e. , { Mi , : } i∈Im are independent random samples from Gaussian distributionMVN ( mb , : , Σ ) , where mb , : = 1n1 ∑ i∈Ib Mi , : and Σ is the covariance matrix determined by the attacker . Considering that Mi , j → µj almost surely ( a.s. ) with Ni going to infinity for all i ∈ Ib based on Lemma 1 , it is straightforward to see that limN∗→∞mb , j = µj a.s. , and the distribution ofMi , j for each i ∈ Im converges to the Gaussian distribution centered at µj . Lemma 2 provides the details . Lemma 2 . Under the same assumption as in Lemma 1 , with N∗ going to infinity , we have for each malicious node i ∈ Im under the Gaussian attack that Mi , j →d N ( µj , Σj , j ) , 1 ≤ j ≤ p. ( 3 ) Lemma 1 and Lemma 2 tell us that for each parameter dimension j , { Mi , j } ni=1 are independent Gaussian random variables with the same mean ( i.e , µj ) but different variances ( i.e. , σ2j /Ni or Σj , j ) under the Gaussian attack . Due to the symmetry of Gaussian distribution , it is straightforward to see E ( Ri , j ) = n+ 1 2 , 1 ≤ i ≤ n , 1 ≤ j ≤ p. Moreover , the exchangeability of benign nodes and the exchangeability of malicious nodes when N∗ is reasonably large tell us : for each parameter dimension j , there exist two positive constants s2b , j and s 2 m , j such that Var ( Ri , j ) = s 2 b , j , ∀ i ∈ Ib , and Var ( Ri , j ) = s2m , j , ∀ i ∈ Im , where both s2b , j and s 2 m , j are complex functions of σ 2 j , Σj , j and { Ni } i∈Ib . Further assume that Ri , j ’ s are independent of each other , thus ei = 1p ∑p j=1Ri , j is the sum of independent random variables with a common mean . Thus , according to the Kolmogorov Strong Law of Large Numbers ( KSLLN ) , we know that ei converges to a constant almost surely , which in turn indicates that vi also converge some constant almost surely . The Theorem 1 summarizes the results formally , with the detailed proof provided in Appendix C. Theorem 1 . Assuming { R : ,j } 1≤j≤p are independent of each other , under the Gaussian attack , we have for each local node i that lim N∗→∞ lim p→∞ ei = n+ 1 2 a.s. , ( 4 ) lim N∗→∞ lim p→∞ ( vi − s̄2b · I ( i ∈ Ib ) − s̄2m · I ( i ∈ Im ) ) = 0 a.s. , ( 5 ) where I ( · ) stands for the indicator function , s̄2b , 1p ∑p j=1 s 2 b , j and s̄ 2 m , 1 p ∑p j=1 s 2 m , j . Considering that s̄2b = s̄ 2 m if and only if Σj , j ’ s fall into a lower dimensional manifold whose measurement is zero under the Lebesgue measure , we have P ( s̄2b = s̄ 2 m ) = 0 if the attacker specifies the Gaussian variance Σj , j ’ s arbitrarily in the Gaussian attack . Thus , Theorem 1 in fact suggests that the benign nodes and the malicious nodes are different on the value of vi , and therefore provides a guideline to detect the malicious nodes . Although the we do need N∗ and p to go to infinity for getting the theoretical results in Theorem 1 , in practice the malicious node detection algorithm based on the theorem typically works very well when N∗ and p are reasonably large and Ni ’ s are not dramatically far away from each other . The independent rank assumption in Theorem 1 , which assumes that { R : ,j } 1≤j≤p are independent of each other , may look restrictive . However , in fact it is a mild condition that can be easily satisfied in practice due to the following reasons . First , for a benign node i ∈ Ib , Mi , j and Mi , k are often nearly independent , as the correlation between two model parameters θj and θk is often very weak in a larger deep neural network with a huge number of parameters . To verify the statement , we implemented independence tests for 100,000 column pairs randomly chosen from the message matrix M generated from the FASHION-MNIST data . Distribution of the p-values of these tests are demonstrated in Figure 2 via a histogram , which is very close to a uniform distribution , indicating that Mi , j and Mi , k are indeed nearly independent in practice . Second , even some M : ,j and M : ,k shows strong correlation , magnitude of the correlation would be reduced greatly during the transformation from M to R , as the final ranking Ri , j also depends on many other factors . Actually , the independent rank assumption could be relaxed to be uncorrelated rank assumption which assumes the ranks are uncorrelated with each other . Adopting the weaker assumption will result in a change of convergence type of our theorems from the “ almost surely convergence ” to “ convergence in probability ” , but with no essential influence to the our algorithm below .

Summary: The paper focuses on Byzantine defense through malicious node detection in a Federated Learning setting. Namely, by ranking the gradients and then computing the mean/SD, the paper shows that the malicious and benign clients will cluster separately. Assuming that the number of malicious clients is fewer than the number of benign clients, and the clusters correctly separate the malicious from benign, the smaller cluster is removed, and training is done on the gradients in the larger cluster. Appropriate experiments are done to show the ability of the model in malicious node detection along with analysis of performance and computational requirements.