MANDERA: M ALICIOUS N ODE D ETECTION IN F EDERATED L EARNING VIA R ANKING
A BSTRACT
Federated Learning is a distributed learning paradigm which seeks to preserve the privacy of each participating node’s data.
However, federated learning is vulnerable to attacks, speciﬁcally to our interest, model integrity attacks.
In this paper, we propose a novel method for malicious node detection called MANDERA.
By transferring the original message matrix into a ranking matrix whose column shows the relative rankings of all local nodes along different parameter dimensions, our approach seeks to distinguish the malicious nodes from the benign ones with high efﬁciency based on key characteristics of the rank domain.
We have proved, under mild conditions, that MANDERA is guaranteed to detect all malicious nodes under typical Byzantine attacks with no prior knowledge or history about the participating nodes.
The effectiveness of the proposed approach is further conﬁrmed by experiments on three classic datasets, CIFAR-10, FASHIONMNIST and MNIST.
Compared to the state-of-art methods in the literature for defending Byzantine attacks, MANDERA is unique in its way to identify the malicious nodes by ranking and its robustness to effectively defense a wide range of attacks.
I NTRODUCTION
Federated learning (FL) has observed a steady rise in use across a plethora of applications.
FL departs from conventional centralized learning by allowing multiple participating nodes to learn on a local collection of training data, before each respective node’s updates are sent to a global coordinator for aggregation.
The global model collectively learns from each of these individual nodes before relaying the updated global update back to the participating nodes.
With an aggregation of multiple nodes, the resulting model observes greater performance than if each node was to learn on their local subset only.
FL presents two key advantages, increased privacy for the contributing node as local data is not communicated to the global coordinator, and a reduction in computation by the global node as the computation is ofﬂoaded to contributing nodes.
However, the presence of malicious actors in the collaborative process may seek to poison the performance of the global model, to reduce the output performance of the model (Chen et al., 2017;
Fang et al., 2020; Tolpegin et al., 2020b), or to embed hidden back-doors within the model (Bagdasaryan et al., 2020).
Byzantine attack aims to devastate the performance of the global model by manipulating the gradient values of malicious nodes in a certain fashion.
As these attacks emerged, researchers seek to defend FL from the negative impacts of these attacks.
In the literature, there are two typical defense strategies: malicious node detection and robust learning.
Malicious node detection defenses by detecting malicious nodes and removing them from the aggregation (Blanchard et al., 2017; Guerraoui et al., 2018; Li et al., 2020;
So et al., 2021).
Robust learning (Blanchard et al., 2017; Yin et al., 2018; Guerraoui et al., 2018; Fang et al., 2020; Cao et al., 2020), however, withstands a proportion of malicious nodes and defenses by reducing the negative impacts of the malicious nodes via various robust learning methods (Wu et al., 2020b; Xie et al., 2019; 2020; Cao et al., 2021).
In this paper, we focus on defensing Byzantine attacks via malicious node detection.
In the literature, there have been a collection of efforts along this research line.
Blanchard et al.
(2017) propose a defense referred to as Krum that treats local nodes whose update vector is too far away from the aggregated barycenter as malicious nodes and precludes them from the downstream aggregation.
Guerraoui et al.
(2018) propose Bulyan, a process that performs aggregation on subsets of node updates (by iteratively leaving each node out) to ﬁnd a set of nodes with the most aligned updates given an aggregation rule.
Xie et al.
(2019) compute a Stochastic Descendant Score (SDS) based on the estimated descendant of the loss function, and the magnitude of the update submitted to the global node, and only include a predeﬁned number of nodes with the highest SDS in the aggregation.
On the other hand, Chen et al.
(2021) propose a zero-knowledge approach to detect and remove malicious nodes by solving a weighted clustering problem.
The resulting clusters update the model individually and accuracy against a validation set are checked.
All nodes in a cluster with signiﬁcant negative accuracy impact are rejected and removed from the aggregation step.
Although the aforementioned methods try to detect malicious nodes in different ways, they all share a common nature: the detection is based on the gradient updates directly.
However, it is usually the case that different dimensions of the gradients remain quite different in the range of values and follow very different distributions.
This phenomena makes it very challenging to precisely detect malicious nodes directly based on the node updates, as a few dimensions often dominate the ﬁnal result.
Although the weighted clustering method proposed by Chen et al.
(2021) could avoid this problem partially by re-weighting different update dimensions, it is often not trivial to determine the weights in a principled way.
In this paper, we propose to resolve this critical problem from a novel perspective.
Instead of working on the node updates directly, we propose to extract information about malicious nodes indirectly by transforming the node updates from numeric gradient values to the rank domain.
Compared to the original numeric gradient values, whose distribution is difﬁcult to model, the ranks are much easier to handle both theoretically and practically.
Moreover, as ranks are scale-free, we no longer need to worry about the scale difference across different dimensions.
We proved under mild conditions that the ﬁrst two moments of the transformed rank vectors carry key information to detect the malicious nodes under a wide range of Byzantine attacks.
Based on these theoretical results, a highly efﬁcient method called MANDERA is proposed to separate the malicious nodes from the benign ones by clustering all local nodes into two groups based on the moments of their rank vectors.
With the assumption that malicious nodes are the minority in the node pool, we can simply treat all nodes in the smaller cluster as malicious nodes and remove them from the aggregation.
[Figure]
RejectMaliciousUpdate 0.03, 0.12, 0.06, 0.2, 0.
1, 3, 2, 4, 5 Rank Node Parameters 𝜽 𝟏 𝜽 𝟐 0.72, 0.90, 0.69, 0.7, 0.
3, 1, 2, 4, 5 𝜽0.18, 0.2, 0.16, 0.3,
[Figure] [Figure]
[Figure] [Figure]
[Figure]
𝒑
[Figure]
The contributions of this work are as follows.
(1) We propose the ﬁrst algorithm leveraging the rank domain of model updates to detect malicious nodes (Figure 1).
(2) We provide theoretical guarantee for the detection of malicious nodes based on the rank domain under Byzantine attacks.
(3) Our method does not assume knowledge on the number of malicious nodes, which is required in the learning process of prior methods.
(4) We experimentally demonstrate the effectiveness and robustness of our defense on Byzantine attacks, including Gaussian attack, Sign Flipping attack and
Zero Gradient attack, in addition to a more subtle Label Flipping data poisoning attack.
An experimental comparison between MANDERA and a collection of robust aggregation techniques are provided.
The computation times are also compared, demonstrating gains of MANDERA by operating in the rank domain.
D EFENSE F ORMALIZATION 2. N OTATIONS
Suppose there are n local nodes in the federated learning framework, where n 1 nodes are benign nodes whose indices are denoted by I b and the other n 0 = n − n 1 nodes are malicious nodes whose indices are denoted by I m .
The training model is denoted by f ( θ , D ) , where θ ∈ R p ×is a p -dimensional parameter vector and D is a data matrix.
Denote the message matrix received from all local nodes by the central server as M ∈ R n × p , where M i, : denotes the message received from node i .
For a benign node i , let D i be the data matrix on it with N i as the sample size, we have M i, : =
∂f ( θ , D i )
∂ θ .
A malicious node j ∈ I m , however, tends to attack the learning system by manipulating M j, : in some way.
Hereinafter, we denote N ∗ =
min( { N i } i ∈I b ) to be the minimal sample size of the benign nodes.
Given a vector of real numbers a ∈ R p × 1 , deﬁne its ranking vector as b =
Rank ( a ) ∈ perm { 1 , · · · , p } , where the ranking operator Rank maps the vector a to its permutation space perm { 1 , · · · , p } which is the set of all the permutations of { 1 , · · · , p } .
For example,
Rank (1 . 1 , − 2 , 3 . 2) =
We adopt average ranking, when there are ties.
With the Rank operator, we can transfer the message matrix M to a ranking matrix R by replacing its column
M : ,j by the corresponding ranking vector R : ,j = Rank ( M : ,j ) .
Further deﬁne e i (cid:44) 1 p p (cid:88)
j =
R i,j and v i (cid:44) 1 p p (cid:88)
j = ( R i,j − e i )to be the mean and variance of R i, : , respectively.
As it is shown in later subsections, we can judge whether node i is a malicious node based on ( e i , v i ) under various attack types.
In the following, we will highlight the behaviour of the benign nodes ﬁrst, and then discuss the behaviour of malicious nodes and their interactions with the benign nodes under various Byzantine attacks respectively.
2. B EHAVIOUR OF BENIGN NODES
As the behaviour of benign nodes does not depend on the type of Byzantine attack, we can study the statistical properties of ( e i , v i ) for a benign node i ∈ I b before the speciﬁcation of a concrete attack type.
For any benign node i , the message generated for j th parameter is
[Equation]
where D i,l denotes the l th sample on it.
Throughout this paper, we always assume that D i,l s are independent and identically distributed (IID) samples drawn from a data distribution D .
Under the independent data assumption, since Equation 1 tells us that M i,j is the sample mean of IID random variables, i.e., { ∂f ( θ , D i,l ) ∂ θ j } N i l =1 , directly applying the Strong Law of Large Numbers (SLLN) and
Central Limit Theorem (CLT) leads to the lemma below immediately.
Lemma 1.
Under the independent data assumption, further denote µ j = E ( ∂f ( θ , D i,l ) ∂ θ j ) and σ 2 j =
Var( ∂f ( θ , D i,l ) ∂ θ j ) < ∞ , with N i going to inﬁnity we have for ∀ j ∈ { 1 , · · · , p }
M i,j → µ j a.s. and M i,j → d N (cid:0) µ j , σ 2 j /N i (cid:1) .
2. B EHAVIOUR OF MALICIOUS NODE UNDER THE G AUSSIAN ATTACK
Deﬁnition 1 (Gaussian attack) .
In a Gaussian attack, the attacker manipulates malicious nodes to send Gaussian random messages to the global coordinator, i.e., { M i, : } i ∈I m are independent random samples from Gaussian distribution MVN ( m b, : , Σ) , where m b, : = 1 n 1 (cid:80) i ∈I
b M i, : and Σ is the covariance matrix determined by the attacker.
Considering that M i,j → µ j almost surely (a.s.) with N i going to inﬁnity for all i ∈ I b
based on
Lemma 1, it is straightforward to see that lim
N ∗ →∞ m b,j = µ j a.s., and the distribution of M i,j for each
i ∈
I m converges to the Gaussian distribution centered at µ j .
Lemma 2 provides the details.
Lemma 2.
Under the same assumption as in Lemma 1, with N ∗ going to inﬁnity, we have for each malicious node
i ∈
I m under the Gaussian attack that
[Equation]
Lemma 1 and Lemma 2 tell us that for each parameter dimension j , { M i,j } ni =1 are independent
Gaussian random variables with the same mean (i.e, µ j ) but different variances (i.e., σ 2 j /N i or Σ j,j ) under the Gaussian attack.
Due to the symmetry of Gaussian distribution, it is straightforward to see
[Equation]
Moreover, the exchangeability of benign nodes and the exchangeability of malicious nodes when
N ∗ is reasonably large tell us: for each parameter dimension j , there exist two positive constants s 2 b,j and s 2 m,j such that
[Equation]
where both s 2 b,j and s 2 m,j are complex functions of σ 2 j , Σ j,j and { N i } i ∈I b .
Further assume that
R i,j ’s are independent of each other, thus e i = 1 p (cid:80) pj
=1 R
i,j is the sum of independent random variables with a common mean.
Thus, according to the Kolmogorov Strong Law of Large Numbers (KSLLN), we know that e i converges to a constant almost surely, which in turn indicates that v i also converge some constant almost surely.
The Theorem 1 summarizes the results formally, with the detailed proof provided in Appendix C.
Theorem 1.
Assuming { R : ,j } 1 ≤ j ≤ p are independent of each other, under the Gaussian attack, we have for each local node i that
[Equation]
) where I ( · ) stands for the indicator function, ¯ s 2 b (cid:44) 1 p (cid:80) pj =1 s 2 b,j and ¯ s 2 m (cid:44) 1 p (cid:80) pj
=1 s 2 m,j .
Considering that ¯ s 2 b
= ¯ s 2 m if and only if Σ j,j ’s fall into a lower dimensional manifold whose measurement is zero under the Lebesgue measure, we have P (¯ s 2 b = ¯ s 2 m )
=
if the attacker speciﬁes the Gaussian variance Σ j,j ’s arbitrarily in the Gaussian attack.
Thus, Theorem 1 in fact suggests that the benign nodes and the malicious nodes are different on the value of v i , and therefore provides a guideline to detect the malicious nodes.
Although the we do need N ∗ and p to go to inﬁnity for getting the theoretical results in Theorem 1, in practice the malicious node detection algorithm based on the theorem typically works very well when N ∗ and p are reasonably large and N i ’s are not dramatically far away from each other.
The independent rank assumption in Theorem 1, which assumes that { R : ,j } 1 ≤ j ≤ p are independent of each other, may look restrictive.
However, in fact it is a mild condition that can be easily satisﬁed in practice due to the following reasons.
First, for a benign node i ∈ I b , M i,j and M i,k are often nearly independent, as the correlation between two model parameters θ j and θ k is often very weak in a larger deep neural network with a huge number of parameters.
To verify the statement, we implemented independence tests for 100,000 column pairs randomly chosen from the message matrix M generated from the FASHION-MNIST data.
Distribution of the p-values of these tests are demonstrated in Figure 2 via a histogram, which is very close to a uniform distribution, indicating that M i,j and M i,k are indeed nearly independent in practice.
Second, even some M : ,j and
M : ,k shows strong correlation, magnitude of the correlation would be reduced greatly during the transformation from M to R , as the ﬁnal ranking R i,j also depends on many other factors.
Actually, the independent rank assumption could be relaxed to be uncorrelated rank assumption which assumes the ranks are uncorrelated with each other.
Adopting the weaker assumption will result in a change of convergence type of our theorems from the “almost surely convergence” to “convergence in probability”, but with no essential inﬂuence to the our algorithm below.
[Figure]
: Independence tests for 100,000 column pairs randomly chosen from message matrix M generated from FASHION-MNIST data supports the independence assumption made in Theorem 1.
2. M ALICIOUS NODE DETECTION FOR SIGN FLIPPING ATTACK
Deﬁnition 2 (Sign ﬂipping attack) .
Sign ﬂipping attack aims to generate the gradient values of malicious nodes by ﬂipping the sign of the average of all the benign nodes’ gradient at each epoch, i.e., specifying M i, : = − r m b, : for any i ∈
I m , where r > 0 , m b = 1 n 1 (cid:80)
k ∈I b M k, : .
Based on the above deﬁnition, the update message of a malicious node i under the sign ﬂipping attack is
[Equation]
For ﬁxed { M k, : } k ∈I b , M i, : is also a ﬁxed vector without randomness, as it is a deterministic function of { M k, : } k ∈I b .
On the other hand, however, we can also treat M i, : as a random vector, since the randomness of { M k, : } k ∈I b can be transferred to M i, : via the link function in equation 6.
In fact, for any parameter dimension j , considering that M k,j → d N (cid:0) µ j , σ 2 j /N k (cid:1) for any k ∈ I b according to
Lemma 1, it is straightforward to see that M i,j = − rn 1 (cid:80)
k ∈I b M k,j can also be well approximated by a Gaussian distribution.
The lemma 3 summarizes the result formally.
Lemma 3.
Under the sign ﬂipping attack, for each malicious node i ∈ I m and any parameter dimension j , we have M i,j = − rn 1 (cid:80)
k
∈I b M k,j is a deterministic function of { M k,j } k ∈I b , whose limiting distribution when N ∗ goes to inﬁnity is
[Equation]
where µ j ( r ) =
− rµ j , σ 2 j ( r ) =
r 2 · σ 2 j n 1 · ¯ N b , and ¯ N b = n
[Equation]
k ∈I b 1 Nk is the harmonic mean of { N k } k ∈I b .
Lemma 1 and Lemma 3 tell us that for each parameter dimension j , the distribution of { M i,j } ni =1 is a mixture of Gaussian components {N (cid:0) µ j , σ 2 j /N i (cid:1) } i ∈I b centered at µ j plus a point mass located at µ j ( r )
=
− rµ j .
If N i ’s are reasonably large, variances σ 2 j /N
i ’s would be very close to zero, and the probability mass of the mixture distribution would concentrate to two local centers µ j and µ j ( r ) =
− rµ j , one for the benign nodes and the other one for the malicious nodes.
This intuition provides us the guidance to identify the malicious nodes in this attack pattern.
Transforming to the rank domain, the above intuition leads to different behavior patterns of the benign nodes and the malicious nodes in the rank matrix R , which in turn result in different limiting behavior of ( e i , v i ) for the benign and malicious nodes.
The theorem 2 summarizes the results formally, with the detailed proof provided in Appendix D.
Theorem 2.
With the same independent rank assumption as posed in Theorem 1, under the sign ﬂipping attack, we have for each local node i that
[Equation]
where ¯ µ b = n + n 0 +
[Equation]
, ¯ s 2 m and ¯ s 2 b are both quadratic functions of ρ whose concrete form also depends on n 0 and n 1 .
Considering that ¯ µ b = ¯ µ m if and only if ρ = 12 , and ¯ s 2 b
= ¯ s 2 m if and only if ρ is the solution of a quadratic function, the probability of (¯ µ b , ¯ s 2 b ) =
(¯ µ m , ¯ s 2 m ) is zero as p → ∞ .
Such a phenomenon suggests that we can detect the malicious nodes based on the moments ( e i , v i ) to defense the sign ﬂipping attack as well.
Noticeably, we note that the limit behaviour of e i and v i does not dependent on the speciﬁcation of r , which deﬁnes the sign ﬂipping attack.
Although such a fact looks a bit abnormal at the ﬁrst glance, it is totally understandable once we realize that with the variance of
M i,j shrinks to zero with N i goes to inﬁnity for each benign node i , any different between µ j and µ j ( r ) would result in the same rank vector R : ,j in the rank domain.
2. M ALICIOUS NODE DETECTION FOR ZERO GRADIENT ATTACK
Deﬁnition 3 (Zero gradient attack) .
Zero gradient attack aims to make the aggregated message to be zero, i.e., (cid:80) ni
=1 M i, : = 0 , at each epoch, by specifying M i, : = − n 1 n 0 m b, : for all i ∈ I m .
Apparently, the zero gradient attack deﬁned above is a special case of sign ﬂipping attack by specifying r = n 1 n 0 .
Since the conclusions of Theorem 2 keep unchanged for different speciﬁcations of r as we have discussed, we have the following corollary for zero gradient attack.
Corollary 1.
Under the zero gradient attack, e i ’s
and v i
’s follow exactly the same limiting behaviours as described in Theorem 2.
2. MANDERA
Theorem 1, 2 and Corollary 1 imply that, under these three attacks (Gaussian attack, zero gradient attack and sign ﬂipping attack), the ﬁrst two moments of R i, : , i.e., ( e i , v i ) , converge to two different limits for the benign nodes and the malicious nodes, respectively.
Thus, for a real dataset where
N i ’s and p are all ﬁnite but reasonably large numbers, the scatter plot of { ( e i , v i ) } 1 ≤ i ≤ n would demonstrate a clustering structure: one cluster for the benign nodes and the other cluster for the malicious nodes.
Figure 3 illustrates such a scatter plot for the 100 local nodes in a typical epoch of training the FASHION-MNIST dataset under different FL settings (to keep the two dimensions of the scatter plot to the same scale, we replaced v i by its square root s i = √ v i instead).
Clearly, a simple clustering procedure would detect the malicious nodes from the scatter plot.
Based on this intuition, we propose MAlicious Node DEtection via RAnking (MANDERA) to detect the malicious nodes, whose workﬂow is detailed in Algorithm 1. i s i ) i ≤ n
K set of benign nodes the lager cluster denoted
Remark.
MANDERA can be applied to either a single epoch or multiple epochs.
For a single-epoch mode, the input data M is the message matrix received from a single epoch.
For multiple-epoch mode, the data M is the column-concatenation of the message matrices from multiple epochs.
By default, the experiments below all use single epoch to detect the malicious nodes.
The predicted benign nodes ˆ I b obtained by MANDERA naturally leads to an aggregated message ˆ m b, : = 1 #(ˆ I b ) (cid:80) i ∈ ˆ I b M i, : .
The theorem 3 shows that ˆ I b and ˆ m b lead to consistent estimations of I b and m b respectively, indicating that MANDERA enjoys robustness guarantee (Steinhardt, 2018) for typical Byzantine attacks. Theorem 3.
Under the three typical Byzantine attacks, i.e., Gaussian attack, sign ﬂipping attack and zero gradient attack, we have:
[Equation] The proof of Theorem 3 can be found in Appendix E.
E XPERIMENTS
We evaluate the efﬁcacy in detecting malicious nodes within the federated learning framework with the use of three Datasets.
The ﬁrst is the FASHION-MNIST dataset (Xiao et al., 2017), a dataset of 60,000 and 10,000 training and testing samples respectively divided into 10 classes of apparel.
The second is CIFAR-10 (Krizhevsky et al., 2009), a dataset of 60,000 small object images also containing 10 object classes.
The third is MNIST (Deng, 2012) dataset which appears in Appendix H.
In these experiments we mainly adopt implementations of Byzantine attacks released by (Wu et al., 2020b;a) and the label ﬂipping attack (Tolpegin et al., 2020b;a).
The label ﬂipping attack is a data poisoning attack that alters one or more labels of training data to an attacker’s pre-determined target label.
For example, in CIFAR-10’s object labels, an attacker may change the labels of their local cat images to be labelled as dogs .
We use the Label Flipping attack as a comparative poisoning attack that achieves its objective in a more subtle manner.
In our experiments, we set Σ =
I for the Gaussian attack and r = 3 for the sign ﬂipping attack, where I is the identity matrix.
For all experiments we ﬁx n = 100 participating nodes, of which a variable number of nodes are poisoned | n 0 | ∈ { 5 , 10 , 15 , 20 , 25 , 30 } .
The training process is run until 25 epochs have elapsed.
We have described the structure of these networks in Appendix A.
3. I LLUSTRATION OF THE AVERAGE RANKING AND STANDARD DEVIATION OF RANKING
Section 2 speculated that the distribution of parameter ranks differ sufﬁciently for the detection of malicious and benign nodes.
We validate this hypothesis in Figure 3 by illustrating the difference [Figure] [Figure] between the benign nodes and malicious nodes in terms of the mean of gradients’ rankings and the standard deviation of gradients’ ranking.
It can be observed from Figure 3 that, under Gaussian and Label ﬂipping attacks, the average rankings of malicious nodes are of a similar distribution to benign nodes.
It is problematic for distinguishing between the two types of nodes, if only average ranking information is used.
On the other hand, Figure 3 displays a larger separation of distributions for the standard deviation of ranking.
It is noted that all 4 attacks observe a convergence of the distributions as the number of malicious nodes increase, increasing the difﬁculty of defense for both MANDERA and all other defenses.
However, the likelihood of an attacker controlling increasingly large numbers of malicious nodes also decrease.
3. M ALICIOUS NODE DETECTION BY MANDERA
We test the performance of MANDERA on the update gradients of a model under attacks.
In this section, MANDERA acts as an observer without intervening in the learning process to identify malicious nodes with a set of gradients from a single epoch.
Each conﬁguration of 25 training epochs, with a given number of malicious nodes was repeated 20 times.
Figure 4 demonstrates the classiﬁcation performance (Metrics deﬁned in Appendix B) of MANDERA with different settings of participating malicious nodes and the four poisoning attacks of Guassian Attack (GA), Zero
Gradient attack (ZG), Sign Flipping attack (SF) and the Label Flipping attack (LF).
While we have formally demonstrated the efﬁcacy of MANDERA in accurately detecting potentially malicious nodes participating in the federated learning process.
In practice, to leverage an unsupervised K-means clustering algorithm, we must also identify the correct group of nodes as the malicious group.
Our strategy is to identify the group with the most exact gradients, or otherwise the smaller group (we regard a system with over 50% of their nodes compromised as having larger issues than just poisoning attacks) 1 .
We also test other clustering algorithms, such as hierarchical clustering and Gaussian mixture models (Fraley & Raftery, 2002).
It turns out that the performance of MANDERA is quite robust with different choices of clustering methods.
Detailed results can be found in Appendix F.
From Figure 4, it is immediately evident that the recall of the malicious nodes for the Byzantine attacks is exceptional.
However, occasionally benign nodes have also been misclassiﬁed as malicious under a SF, and to a lesser extent the ZG attack for both datasets.
On all attacks, in the presence of more malicious nodes, the recall of malicious nodes trends down.
As for the data poisoning attack of LF, it is consistently more difﬁcult to detect, however we note that the LF attack has a more subtle inﬂuence on the model in contrast to the impact of Byzantine attacks.
3. MANDERA FOR DEFENDING AGAINST POISONING ATTACKS
In this section, we encapsulate MANDERA into a module prior to the the aggregation step, MANDERA has the sole objective of identifying malicious nodes, and excluding their updates from the global aggregation step.
Each conﬁguration of 25 training epochs, a given poisoning attack, defense method [Figure] and a given number of malicious nodes was repeated 10 times.
We compare MANDERA against 5 other robust aggregation defense methods, Krum (Blanchard et al., 2017), Bulyan (Guerraoui et al., 2018), Trimmed Mean (Yin et al., 2018), Median (Yin et al., 2018) and FLTrust (Cao et al., 2020).
Of which the ﬁrst 2 requires an assumed number of malicious nodes [Figure] and the latteronly aggregate robustly.
[Figure]
From Figure [Figure] it is observed that MANDERA performs about the same as the best performing defense mechanisms, close to the performance of a model not under attack. MANDERA’s accuracy
.g.
Figuredisplays less variation of rank variance in malicious cluster compared to benign nodes.
This could robust selection of the malicious group, and enabling selection of malicious groups larger than 50%.
is observed to vary slightly under the LF attack on fashion data with 30 malicious nodes, this is consistent with the larger accuracy ranges previously observed in Figure 4b.
Interestingly, FLTrust as a standalone defense is weak in protecting against the most extreme Byzantine attacks.
However, we highlight that FLtrust is a robust aggregation method against targeted attacks that may thwart defences like Krum, Trimmed mean.
We see FLTrust as a complementary defence that relies on a base method of defence against Byzantine attacks, but expands the protection coverage of the FL system against adaptive attacks.
3. C OMPUTATIONAL E FFICIENCY
We have previously been able to observe that MANDERA can perform at par with the current highest performing poisoning attack defenses.
Another beneﬁt arises with the simpliﬁcation of the mitigation strategy with the introduction of ranking at the core of the algorithm.
Sorting and
Ranking algorithms are fast.
Additionally, we only apply clustering on the two dimensions of rank mean and standard deviation, in contrast to other works that seek to cluster on the entire node update (Chen et al., 2021).
The times in Table 1 for MANDERA, Krum and Bulyan do not include the parameter/gradient aggregation step.
These times were computed on 1 core of a Dual Xeon 14-core E5-2690, with 8 Gb of system RAM and a single Nvidia Tesla P100.
Table 1 demonstrates that MANDERA is able to achieve a faster speed than that of single Krum 2 (by more than half) and
Bulyan (by an order of magnitude).
[Table]
D ISCUSSION AND C ONCLUSION
If attackers create more adaptive attacks unlike Deﬁnition 1, 2 and 3, they may evade MANDERA and achieve model poisoning.
In this work, we have conﬁgured our Federated Learner to use allnodes in the learning process at every round, we acknowledge FL framework may learn the global model only using subset of nodes at each round.
In these settings MANDERA would still function, as we would rank and cluster on the parameters of the participating nodes, without assuming any number of poisoned nodes.
In Algorithm 1, performance could be improved by incorporating higher order moments.
MANDERA is unable to function when gradients are securely aggregated in its current form.
However, malicious nodes can be identiﬁed and excluded from the secure aggregation step, while still protecting the privacy of participating nodes by performing MANDERA through secure ranking (Zhang et al., 2013; Lin & Tzeng, 2005) (recall that MANDERA only requires the ranking matrix to detect poisoned nodes).
It remains to be seen the effectiveness of MANDERA on more advanced poisoning techniques like adversarial poisoning or Evasion attacks.
In conclusion, we have provided theoretical guarantees and experimentally shown efﬁcacy in the use of ranking algorithms for the detection of malicious nodes performing poisoning attacks against federated learning.
Our proposed method MANDERA, is able to achieve high detection accuracy and maintain a model accuracy on par with other seminal, high performing defense mechanisms, but with three notable advantages.
First, provable guarantees for the use of ranking to detect Gaussian,
Zero Gradient and Sign Flipping attacks.
Next, faster detection with the use of ranking algorithms.
Finally, the MANDERA defense does not need a prior estimation of the number of poisoned nodes.
In this work we demonstrate how the rank domain can be useful in applications to defend against malicious actors.
Ethics Statement
The core objective of our research is to provide an additional means of defense against poisoning nodes that target Federated Learning.
To test our defense we have implemented different attacks against the Federated Learning framework.
Attackers may adopt our defense strategy to design new poisoning attacks.
Fortunately, these poisoning attacks can not be leveraged to leak private information from Federated learning models, instead only impact its performance.
Reproducibility Statement
To ensure reproducible research we have supplemented our proposal for MANDERA, by supplying both R and Python implementations of MANDERA used in this paper, uploaded with the remainder of the experiment code.
The three datasets featured in this paper is CIFAR-10 (Krizhevsky et al., 2009), Fasion-MNIST (Xiao et al., 2017), and MNIST (Deng, 2012); we have used each of these dataset unaltered from their respective sources.
We have stated the assumptions in our theorems and their proofs can be found in the Appendix.
But to explain our assumptions in simple terms, (1)
The data samples on each local node are independently drawn from the same distribution.
The gradient value for each parameter is independent to each other.
A N EURAL N ETWORK CONFIGURATIONS
We train these models with a batch size of 10, an SGD optimizer operates with a learning rate of 0. and 0.5 momentum for 25 epochs.
The accuracy of the model is evaluated on a holdout set of 1000 samples.
A. FASHION-MNIST AND MNIST
• Layer 1: 1 ∗ 16 ∗ 5 , 2D Convolution, Batch Normalization, ReLU Activation, Max pooling.
• Layer 2: 16 ∗ 32 ∗ 5 , 2D Convolution, Batch Normalization, ReLU Activation, Max pooling.
• Output: 10 Classes, Linear.
A. C IFAR -
• Layer 1: 1 ∗ 32 ∗ 3 , 2D Convolution, Batch Normalization, ReLU Activation, Max pooling.
• Layer 2: 32 ∗ 32 ∗ 3 , 2D Convolution, Batch Normalization, ReLU Activation, Max pooling.
• Output: 10 Classes, Linear.
B M ETRICS
The metrics observed in Section 3 to evaluate the performance of the defense mechanisms are deﬁned as follows:
[Equation]
C P ROOF OF T HEOREM
Proof.
Because { R i,j } 1 ≤ j ≤ p are independent random variables with a ﬁnite upper bound (since n is ﬁxes) as assumed, direct application of KSLLN leads to
[Equation]
To prove Theorem 1 based on Equation 11 and 12, we need to derive the concrete form of E ( R i,j ) and
Var( R i,j ) .
Fortunately, because M i,j → d N (cid:0) µ j , Σ j,j (cid:1) for ∀ i ∈ I m
and M i,j → d N (cid:0) µ j , σ 2 j /N i (cid:1) for ∀ i ∈
I b when N ∗ → ∞ , it is straightforward to see due to the symmetry of Gaussian distribution that
[Equation]
Moreover, assuming that the sample sizes of different benign nodes approach to each other with N ∗ going to inﬁnity, i.e.,
[Equation]
for each parameter dimension j , { M i,j } i ∈I b would converge to the same Gaussian distribution
N ( µ j , σ 2 j /N ∗ ) with the increase of N ∗ .
Thus, due to the exchangeability of { M i,j } i ∈I b and { M i,j } i ∈I m , it is easy to see that there exist two positive constants s 2 b and s 2 m , such that
[Equation]
where s 2 b,j and s 2 m,j are both complex functions of n 0 , n 1 , σ 2 j , Σ j,j and N ∗ , and s 2 b,j = s 2 m,j if and only if σ 2 j /N ∗ =
Σ j,j .
Combining Equation 11 and 13, we have
[Equation]
i.e., Equation 4, which further indicates that e i and E ( R i,j ) share the same limit when both p and
N ∗ go to inﬁnity.
Thus, we have
[Equation]
Combining Equation 12, 15, and 16, we have
[Equation]
i.e., Equation 5.
Thus, the proof is complete.
D P ROOF OF T HEOREM
Proof.
It is straightforward to see that equation 11 also holds for sign ﬂipping attack under the assumptions of Theorem 2.
But, we need to re-calculate E ( R i,j ) for benign and malicious nodes under the new setting.
Under the sign ﬂipping attack [Equation] because M i,j → d
N (cid:0) µ j ( r ) , σ 2 j ( r ) (cid:1) for ∀ i ∈
I m and M i,j → d
N (cid:0) µ j , σ 2 j /N i (cid:1) for ∀ i ∈ I b when N ∗ → ∞ , and
[Equation]
it is straightforward to see that
[Equation]
which further indicates that
[Equation]
where S 2[ a,b ] =
[Equation]
Combining Equation 11 and 17, we have
[Equation]
where ρ = lim p →∞ (cid:80) pj
[-=
-][Equation]
I [Equation]
[Equation]
p , i.e., Equation 8.
Deﬁne ¯ µ i = ¯ µ m · I ( i ∈ I m )
+ ¯ µ b · I ( i ∈ I b ) .
Based on KSLLN, we have:
[Equation]
As we have proved in Equation 8 that
[Equation]
[Equation]
., we have
[Equation]
which implies that
[Equation]
Considering that
[Equation]
where
[Equation]
we have
[Equation]
It completes the proof of Equation 9 by specifying ¯ s 2 b = ¯ τ b − (¯ µ b [Equation] 2 and ¯ s [Equation]
m = ¯ τ m − (¯ µ
[Equation] [Equation] E P ROOF OF T HEOREM
Proof.
According to Theorem 1, 2 and Corollary 1, when both N ∗ and p are large enough, with probability 1 there exist ( e b , v b ) , ( e m , v m ) and δ > 0 such that || ( e b , v b ) − ( e m , v m ) || 2 > δ , and || ( e i , v i ) − ( e b , v b )
|| 2 ≤ δ 2 for ∀ i ∈ I
b
and || ( e i , v i ) − ( e m , v m ) || 2 ≤ δ 2 for ∀ i ∈ I m .
Therefore, with a reasonable clustering algorithm such as K -mean with K = 2 , we would expect ˆ
I b =
I b with probability 1.
Because we can always ﬁnd a ∆ > 0 such that || M i, : − M j, : || 2 ≤ ∆ for any node pair ( i, j ) in a ﬁxed dataset with a ﬁnite number of nodes, and ˆ m b, : = m b, : when ˆ I b
= I b , we have
[Equation]
and thus
[Equation] It completes the proof. F MANDERA PERFORMANCE WITH DIFFERENT CLUSTERING ALGORITHMS In this section, Figure 6 demonstrate that the discriminating performance of MANDERA when hierarchical clustering and Gaussian mixture models are used in-place of K-means for FASHIONMNIST data set remain robust.
G M ODEL L OSSES ON CIFAR-10 AND FASHION-MNIST DATA
Figure 7 presents the model loss to accompany the model prediction performance of Figure 5 previously seen in Section 3.
H M ODEL PERFORMANCE ON MNIST DATA
In this section we replicate experiments that were previously performed in Section 3 on the
MNIST (Deng, 2012) dataset.
The MNIST dataset is a dataset of 60,000 and 10,000 training and testing samples respectively divided into 10 classes of handwritten digits from multiple authors.
Figure 8 contains the performance characteristics of MANDERA’s defense against the four attacks, whilst Figure 9 contains the comparative accuracy and loss when the different defenses are applied.
Generally speaking, the observations previously observed continue to hold for this dataset.
[Figure]
cc u r
[Figure]
R e c
[Figure] [Figure]
[Figure]
A cc u r a cy
R e c a ll
P r e c
i s i on
F
[Figure] [Figure]
Number of malicious nodes
[Figure]
Metric
Accuracy
Recall
Precision
F
[Figure]