B YZANTINE -R OBUST L EARNING ON H ETEROGENEOUS D ATASETS VIA R ESAMPLING
Anonymous authors
Paper under double-blind review
A BSTRACT
In Byzantine robust distributed optimization, a central server wants to train a machine learning model over data distributed across multiple workers.
However, a fraction of these workers may deviate from the prescribed algorithm and send arbitrary messages to the server.
While this problem has received signiﬁcant attention recently, most current defenses assume that the workers have identical data.
For realistic cases when the data across workers are heterogeneous (non-iid), we design new attacks which circumvent these defenses leading to signiﬁcant loss of performance.
We then propose a simple resampling scheme that adapts existing robust algorithms to heterogeneous datasets at a negligible computational cost.
We theoretically and experimentally validate our approach, showing that combining resampling with existing robust algorithms is effective against challenging attacks.
I NTRODUCTION
Distributed or federated machine learning, where the data is distributed across multiple workers, has become an increasingly important learning paradigm both due to growing sizes of datasets, as well as privacy and security concerns.
In such a setting, the workers collaborate to train a single model without transmitting their data directly over the networks (McMahan et al., 2016;
Bonawitz et al., 2019; Kairouz et al., 2019).
Due to the presence of either actively malicious agents in the network, or simply due to system and network failures, some workers may disobey the protocols and send arbitrary messages; such workers are also known as Byzantine workers (Lamport et al., 2019).
Byzantine robust optimization algorithms combine the gradients received by all workers using robust aggregation rules, to ensure that the training is not impacted by the malicious workers.
While this problem has received signiﬁcant recent attention, (Alistarh et al., 2018; Blanchard et al., 2017; Yin et al., 2018a), most of the current approaches assume that the data present on each different worker has identical distribution.
In this work, we show that existing Byzantine-robust methods catastrophically fail in the realistic setting when the data is distributed
heterogeneously
across
the
workers.
We then propose a simple resampling scheme which can be readily combined with existing aggregation rules to allow robust training on
heterogeneous
data.
Contribution.
Concretely, our contributions in this work are
• We show that when the data across workers is heterogeneous, existing robust rules might not converge, even without any Byzantine adversaries.
• We propose two new attacks, normalized gradient and mimic, which take advantage of data heterogeneity and circumvent median and sign-based defenses (Blanchard et al., 2017; Pillutla et al., 2019;
Li et al., 2019).
• We propose a simple new resampling step which can be used before any existing robust aggregation rule.
We instantiate our scheme with K RUM and theoretically prove that the resampling generalizes it to the setting of heterogeneous data.
• Our experiments evaluate the proposed resampling scheme against known and new attacks and show that it drastically improves the performance of 3 existing schemes on realistic heterogeneously distributed datasets.
Setup and notations.
We study the general distributed optimization problem
[Equation]
where L i : R d → R are the individual loss functions distributed among n workers, each having its own (heterogeneous) data distribution {D i } ni =1 .
The case of empirical risk minimization with m i datapoints ξ i ∼ D i on worker i is obtained when using L i ( x ) :
=
1 m i (cid:80) m i j =1 L i ( x , ξ ji ) .
The (stochastic) gradient computed by a good node i with sample j is given as g i ( x ) :
= ∇L i ( x , ξ ji ) with mean µ i and variance σ 2 i .
We also assume that the heterogeneity (variance across good workers) is bounded i.e.
[Equation]
We write g i instead of g i ( x t ) when there is no ambiguity.
A distributed training step using an aggregation rule is given as
[Equation]
If the aggregation rule is the arithmetic mean, then (2) recovers standard minibatch SGD.
Byzantine attack model.
In each iteration, there is a set Byz of at most f Byzantine workers.
The remaining workers are good , thus follow the described protocol.
A Byzantine worker j ∈ Byz can deviate from protocol and send an arbitrary vecter to the server.
Besides, we also allow that Byzantine workers can collude with each other and know every state of the system.
Unlike martingale-based approaches like (Alistarh et al., 2018), we allow the set Byz to change over time (Blanchard et al., 2017; Chen et al., 2017; Mhamdi et al., 2018).
R ELATED WORK
There has been signiﬁcant recent work of the case when the workers have identical data distributions (Blanchard et al., 2017; Chen et al., 2017; Mhamdi et al., 2018;
Alistarh et al., 2018; Mhamdi et al., 2018;
Yin et al., 2018a;b; Su & Xu, 2018; Damaskinos et al., 2019).
We discuss the most pertinent of these methods next.
Blanchard et al.
(2017) formalize the Byzantine robust setup and propose a distance-based approach K RUM which selects a worker whose gradient is very close to at least half the other workers.
A different approach involves using the median and its variants (Blanchard et al., 2017; Pillutla et al., 2019;
Yin et al., 2018a).
Yin et al.
(2018a) propose to use and analyze the coordinate-wise median method (C M ).
Pillutla et al.
(2019) use a smoothed version of Weiszfeld’s algorithm to iteratively compute an approximate geometric median of the input gradients.
In a third approach, (Bernstein et al., 2018) propose to use the signs of the gradients and then aggregate them by majority vote, however, (Karimireddy et al., 2019) show that it may not always converge.
Finally,
Alistarh et al.
(2018) use a martingale-based aggregation rule which gives a sample complexity optimal algorithm for iid data.
The distance-based approach of K RUM was later extended in Mhamdi et al.
(2018) who propose B ULYAN to overcome the dimensional leeway attack.
This is the so called strong Byzantine resilience and is orthogonal to the question of non-iid-ness we study here.
Recently, (Peng & Ling, 2020; Yang & Bajwa, 2019a;b) studied Byzantine-resilient algorithms in the decentralized setting where there is no central server available.
Extending our techniques to the decentralized setting is an important direction for future work.
In a different line of work, (Lai et al., 2016; Diakonikolas et al., 2019) develop sophisticated spectral techniques to robust estimate the mean of a high dimensional multi-variate standard Gaussian distribution where samples are evenly distributed in all directions and the attackers are concentrated in one direction.
Very recent work (Data & Diggavi, 2020) extend the theoretical analysis to non-convex, strongly-convex and non-i.i.d setup under a gradient dissimilarity assumption and propose a gradient compression scheme on top of it.
Our resampling trick can be combined with it to further reduce gradient dissimilarity.
Many attacks have been devised for distributed training.
For the iid setting, the state-of-the-art attacks are (Baruch et al., 2019; Xie et al., 2019b).
The latter attack is very strong when the fraction of adversaries is large (nearly half), but in this work we focus on settings when this fraction is quite small (e.g. ≤ 0 . 2 ).
Further our normalized mean attack Section 3.2 is inspired by (Xie et al., 2019b).
The former work focuses on attacks which are coordinated across time steps.
Developing strong practical defenses even in the iid case against such time-coordinated attacks remains an open problem.
In this work, we sidestep this issue by restricting ourselves to new attacks made possible by non-iid data and studying how to overcome them.
We focus on schemes which work in the iid setting, but fail with non-iid data.
Once a new method which can defend against (Baruch et al., 2019) is developed, our proposed scheme shows how to adapt such a method to the important non-iid case.
For the non-iid setting, backdoor attacks are designed to take advantage of heavy-tailed data and manipulate model inference on speciﬁc subtask, rather than lower the overall accuracies of training (Bagdasaryan et al., 2018;
Bhagoji et al., 2018).
In contrast, this paper is not intended to address aforementioned challenges but rather to defend the attacks that lower the training accuracies in the non-iid setting.
As far as we are aware, only (Li et al., 2019; Ghosh et al., 2019; Sattler et al., 2020) explicitly investigate Byzantine robustness with non-iid workers.
Li et al.
(2019) proposes an SGD variant (R SA ) which modiﬁes the original objective by adding an (cid:96) 1 penalty.
Ghosh et al.
; Sattler et al.
(2020) assume that all workers belong to an apriori ﬁxed number of clusters and use an outlier-robust clustering method to recover these clusters.
If we assume that the server has the entire training dataset and can control the distribution of samples to good workers, Xie et al. (2019a)
; Chen et al.
Rajput et al.
(2019) show that non-iid-ness can be overcome.
Typical examples of this is distributed training of neural networks on public cloud, or volunteer computing Meeds et al.
; Miura &
Harada (2015).
However, none of these methods are applicable in the standard federated learning setup we consider here.
We aim to minimize the original loss function over workers while respecting the non-iid data locality, i.e. the partition of the given heterogeneous dataset over the workers, without data transfer.
A TTACKS AGAINST EXISTING AGGREGATION SCHEMES
In this section we show that when the data across the workers is heterogeneous (non-iid), then we can design new attacks which take advantage of the heterogeneity, leading to the failure of existing aggregation schemes.
We study three classes of robust aggregation schemes: i) schemes which select a representative worker in each round (e.g. K RUM (Blanchard et al., 2017)), ii) schemes which use normalized means (e.g. R SA (Li et al., 2019)), and iii) those which use the median (e.g. R FA (Pillutla et al., 2019)).
We show realistic settings under which each of these classes would fail when faced with heterogeneous data.
3. F AILURE OF REPRESENTATIVE WORKER SCHEMES ON NON - IID DATA
Algorithms like K RUM select workers who are representative of a majority of the workers, by relying on statistics such as pairwise differences between the various worker updates.
Let ( g 1 , . .
g n ) be the gradients by the workers, f of which are Byzantine (e.g. n ≥ 2 f + 3 for K RUM ).
For i (cid:54) =
j , let i → j denote that g j belongs to the n − f − 2 closest vectors to g i .
Then K RUM is deﬁned as follows
[Equation]
However, when the data across the workers is heterogeneous, there is no ‘representative’ worker.
This is because each worker computes their local gradient over vastly different local data.
Hence, for convergence it is important to not only select a good (non-Byzantine) worker, but also ensure that each of the good workers is selected with roughly equal frequency.
Hence K RUM suffers a signiﬁcant loss in performance with heterogeneous data, even when there are no Byzantine workers .
For example, when K RUM is used for iid datasets without adversary ( f =0 , see left of Figure 1a), the test accuracy is close to simple average and the gap can be ﬁlled by M ULTI - KRUM
(Blanchard et al., 2017).
The right plot of Figure 1a also shows that K RUM ’s selection of gradients is biased towards certain nodes.
When K RUM is applied to non-iid datasets (the middle of Figure 1a), K RUM performs poorly even without any attack.
This is because K RUM mostly selects gradients from a few nodes whose distribution is closer to others (the right of Figure 1a).
This is an example of how robust aggregation rules may fail on realistic non-iid datasets.
3. A TTACKS ON NORMALIZED AGGREGATION SCHEMES
Instead of simply averaging the gradients, some methods ﬁrst normalize them and then average.
This limits the inﬂuence of the Byzantine workers since they cannot output extremely large gradients, and hence is more robust.
For example R FA (Pillutla et al., 2019) with T =1 uses following aggregation rule:
[Equation]
Other methods such as R SA (Li et al., 2019) or signum (Bernstein et al., 2018) normalize entries coordinate-wise before taking a majority vote i.e. update the server model x 0 on server using local model
x i from node i (not gradient) using
[Equation]
[Figure] [Figure]
[Figure]
where f 0 is a strongly convex penalty term and λ > 0 is a relaxation parameter.
However, a Byzantine worker can still craft an “omniscient” attack to foil robust aggregations, using an approach similar to the negative sum for the arithmetic mean (Blanchard et al., 2017; Li et al., 2019):
[Equation]
On the right side of Figure 1b, we can see that this attack lowers the accuracy of R FA -T1 signiﬁcantly, as the number of Byzantine workers increases.
Comparing to its iid counterpart, the normalized mean attack is even more impactful in the non-iid setting.
3. A TTACKS ON MEDIAN - BASED SCHEMES
Geometric median and its variants are popular in robust learning research (Blanchard et al., 2017;
Chen et al., 2017; Pillutla et al., 2019;
Yin et al., 2018a; Mhamdi et al., 2018).
Given gradients { g 1 , . . .
, g n } , we use the estimator
[Equation]
If the vectors { g 1 , . . . , g n } are drawn independently from the same distribution, intuitively most of them would concentrate around their mean.
Then, even if there are some Byzantine outputs, the median would ignore those as outliers and output a ‘central’ point close to the mean.
However, when { g 1 , . .
. , g n } are gradients over heterogeneous data, they may be vastly different from each other and do not concentrate around the mean.
In such a scenario, the median such as (7) can be even less robust than simply taking the mean.
Suppose that worker 0 is Byzantine and the remaining workers { 1 , . .
2 n } are good, with a total of 2 n + 1 workers.
Now suppose that g i =
( − 1) i for all the workers, with half the good workers having − 1 and the other half +1 .
This means that the true mean is 0 , however, the median estimator (7) will output 1.
Mimic attack.
This motivates our mimic attack in which all Byzantine workers collude and agree to always send gradients from the same worker.
We deﬁne a specialized attack, called mimic2 , where half of the good workers have same datasets and send g 1 while the rest good workers send g 2 ; then all Byzantine workers send v = g 1 such that the geometric median of the gradients received by the server is always g 1 .
Therefore, this attack breaks geometric-median-based robust aggregation rules, by leading them to wrong solutions.
The left plot of Figure 1c shows the impact of the mimic2 attack.
Test accuracies of C M and R FA both drop drastically to around 50%.
Setup : n workers, f of which are Byzantine; resampling T times, each time samples s gradients.
A robust learning algorithm
A GGR on iid datasets; γ is the learning rate.
Workers :
Each good worker i randomly samples a datapoint
and
computes a stochastic gradient g i :=
∇ F i ( x , ξ ji ) where ξ ji ∼ D i ; each
worker i sends arbitrary vector g i .
Send g i to server.
Servers :
Receive { g i } ni =1 from
, I
Resampling
{ g i : i ∈ [ n ] } , f , T , s );
x − γ
GGR
Broadcast x
to all workers
Input: { g i : i ∈ [ n ] } , T := n , s , { c [ i ] := 0 : i ∈ [ n ] } for t := 1 , . .
, T do for i := 1 , . . .
, s do while Select j i ∼ Uniform ([ n ]) do if c [ j i ] <
s then c
[ j i ] +=
If c [ j i ] =
= s Break;
Compute average ¯ g t :
= 1 s (cid:80) si =
g j i
Return { ¯ g t : t ∈ [ T ] } , { j ti : t ∈ [ T ] , i ∈
[ s ] }
R OBUST AGGREGATION ON NON - IID DATA
In Section 3 we have demonstrated how existing robust aggregation rules can fail in realistic non-iid scenarios, with and without attackers (Sections 3.2 and 3.3 and Section 3.1 respectively).
To overcome this problem, we propose a simple new resampling-based aggregation rule for training, shown in
Algorithm 1.
More speciﬁcally, we choose s -resampling without replacement in Algorithm 2 where each gradient can be sampled at most s times.
The key property of our rule is that after resampling, the resulting set of averaged gradients { ¯ g t : t ∈ [ T ] } are much more homogeneous (lower variance).
Then these averaged gradients are fed to existing Byzantine robust aggregation schemes, such as
K RUM , see Section 5.
Given an existing aggregation rule A GGR , we denote by A GGR ◦
Resampling the resulting new robust aggregation rule for non-iid input gradients.
In the following proposition, we list the desired properties of Algorithm
Proposition I. Given a population { g i : i ∈ [ n ] } ⊂ R d of mean µ := 1 n (cid:80) ni
=1 g i and variance σ 2 := 1 n (cid:80) ni =1 (cid:107) g
i − µ (cid:107) 2 , let { ¯ g t : t ∈ [ T ] } be the output of Algorithm 2 on { g i : i ∈ [ n ] } .
Then
• If there are no Byzantine workers, then { ¯ g t : t ∈ [ T ] } are identically distributed
[Equation]
• If f of the n inputs are Byzantine, then at least T − sf gradients in { ¯ g t : t ∈ [ T ] } are good; that is, a good ¯ g t is the average of gradients { g j ti : i ∈
[ s ] } ⊂ good ⊂ [ n ] .
Then such good { ¯ g t } are identically distributed with
[Equation]
where
˜ µ :=
1 | good | (cid:80) i ∈
good g i , and ˜
σ 2 :
=
1 |
good |
(cid:80) i ∈
good
(cid:107) g
i
− E
[¯ g t
] (cid:107)
Proof.
Since Algorithm 2 resamples s gradients to estimate a population of sn samples, we can use sampling theory (Middleton, 1988, Ch.
Survey Sampling) to compute the sample mean
[Equation]
and the sample variance
[Equation]
Since the gradients are sampled at most s times, at most sf out of the T gradients are affected by a
Byzantine worker.
Its mean and variance can be calculated in the same way shown above.
Remark 1.
For s = 1 , resampling simply becomes shufﬂing of the input elements, and var (¯ g t )
=
σ 2 is unchanged.
For s > 1 , the resampling scheme reduces the heterogeneity (variance) by approximately 1 /s .
Thus, increasing s leads to the resulting resampled gradients being a better estimator of the population mean, thus improving training convergence speed.
On the other hand, increasing s also increases the number of resampled gradients which can be affected by a Byzantine worker.
In particular, if f workers are Byzantine, then up to fs resampled gradients can be incorrect, which has to be taken into account by the employed robust aggregation rule.
In practice, we found that using a small value s = 2 was already sufﬁcient to overcome heterogeneity.
Remark 2.
A natural question to ask is what happens if we resample with replacement but do not limit on the number of replacements.
We discuss this additional algorithm variant in Appendix C.
Note that the { ¯ g t : t ∈ [ T ] } are identically distributed but not independent.
This does not directly ﬁt into the original assumptions of Byzantine robust algorithms like K RUM and hence the robustness has to be reproved for our more general setting.
C ONVERGENCE ANALYSIS WITH K RUM
In this section, we analyze the convergence of SGD with robust aggregation on non-iid data.
Since the deﬁnition of robustness and other conditions vary from paper to paper, it is not possible to give a uniform proof perfectly ﬁt for all methods.
For example, (Yin et al., 2018a) assumes the gradients have bounded variance and skewness whereas others like K RUM , R FA , B ULYAN does not.
Thus we only analyze K RUM for its simplicity and popularity, and show that analysis is only slightly different from the original version.
For other algorithms, we show by experiments that resampling helps them achieve better performance on heterogeneous data, see Section 6.
Deﬁnition A generalizes the Byzantine resilience of (Blanchard et al., 2017, Deﬁnition 1) to the cases where we have non-iid data.
Let G be an estimator of the good gradients.
Deﬁnition A ( ( α, f ) -Byzantine Resilience.) .
Let 0 ≤ α < π/ 2 be any angular value, and any integer 0 ≤ f ≤ n .
Let B = { j 1 , . . .
, j f : j 1 ≤ j 1 < · · · < j f ≤ n } be the indices of Byzantine workers.
Let { V i ∈
D i : i ∈ [ n ] \B} be independent random vectors in R d .
Let G
=
G ( ξ ) be an independent random variable which randomly selects a good worker i and samples a vector from D i and E G
= g .
Let B 1 , . . .
B f be any Byzantine vectors in R d , possibly dependent on the V i ’s.
An aggregation rule F is said to be ( α, f ) -Byzantine
resilient if
[Equation]
satisﬁes (i) (cid:104) E F, g (cid:105) ≥ (1 − sin α ) · (cid:107) g (cid:107) 2 > 0 and (ii) for r = 2 , 3 , 4 , E (cid:107) F (cid:107) r ≤ E (cid:107) G (cid:107) r .
Then we can conclude the almost sure convergence similar to (Blanchard et al., 2017, Proposition 2)
Theorem II (Resampling K RUM ) .
We assume that (i) the cost function L is three times differentiable with continuous derivatives and non-negative L ( x ) ≥ 0 ; (ii) the learning rates satisfy (cid:80) t γ t = ∞ and (cid:80) t γ 2 t < ∞ .
Let the good workers have stochastic gradients G i ( x , ξ ) for i ∈ good ⊂ [ n ] .
We assume that for a uniformly chosen j ∈ good , the following is true (iii)
[Equation]
and ∀ r ∈ { 2 , 3 , 4 } , E j, ξ (cid:107) G j ( x , ξ ) (cid:107) r ≤ A r + B r (cid:107) x (cid:107) r for some constants A r , B r ; (iv) there exists a constant 0 ≤ α < π/ 2 such that for all x we have η ( T, sf ) · √ d · σ ( x ) ≤ (cid:107)∇L ( x ) (cid:107) · sin α where σ 2 ( x ) :
= n − 1 sn − 1 ˜ σ 2 ( x ) ; (v) ﬁnally, beyond a certain horizon, (cid:107) x (cid:107) 2 ≥ D , there exist ε > 0 and 0 ≤ β < π/ 2 − α such that (cid:107)∇L ( x ) (cid:107) ≥ ε > 0 and (cid:104) x , ∇L ( x ) (cid:105) ≥ cos β (cid:107) x (cid:107) · (cid:107)∇L ( x ) (cid:107) .
If s >and 2 sf + 3 ≤ n , then
• K RUM ◦
Resampling is ( α, sf ) -Byzantine resilient where 0 ≤ α < π/ 2 is deﬁned by
[Equation]
• the sequence of gradients ∇L ( x t ) converges almost surely to zero.
We defer the proof to Appendix A.
The above convergence result for heterogeneous data is nearly identical to (Blanchard et al., 2017, Proposition 2) for iid data, except for the slightly stronger restriction on the number of Byzantine workers 2 sf + 3 ≤ n .
[Figure] [Figure]
[Figure] [Figure]
[Figure] [Figure]
each aggregation we resample and average s gradients for T = n times.
E XPERIMENTS
In this section, we demonstrate the effect of resampling on datasets distributed in a non-iid fashion.
Throughout the section, we illustrate the challenge, attacks, and defense by an example of training an
MLP on the MNIST dataset (LeCun et al., 1998).
In Appendix D, we present the results of similar experiments on Fashion-MNIST (Xiao et al., 2017) and CIFAR-10 (Krizhevsky et al., 2009).
The dataset is sorted by labels and sequentially divided into equal parts among good workers; Byzantine workers have access to the dataset on all good workers.
Implementations are based on PyTorch (Paszke et al., 2019) and will be made publicly available.
6. R ESAMPLING AGAINST THE ATTACKS ON NON - IID DATA
In Section 3 we have presented how heterogeneous data can lead to failure of existing robust aggregation rules.
Here we apply our proposed resampling with T = n , s = 2 to the same aggregation rules, showing that resampling overcomes the described failures.
Results are presented in Figure 2.
In Figure 2a, we show that using resampling helps K RUM to achieve better test accuracy on noniid data.
Since resampling K RUM with s = 2 actually averages 2 gradients, we compare it with
M ULTI K RUM with m = 2 .
The middle of Figure 2a shows that M ULTI K RUM with m = 2 performs better than K RUM , but K RUM with resampling is even better which suggests the resampling step improves the performance on non-iid data.
The selection histogram on the rightmost part of Figure 2a shows that after resampling, K RUM ’s selection is much more evenly distributed between the good workers.
In Figure 2b, we show that resampling ﬁxes R FA with T =1 and allows it to defend against the normalized mean attack.
The resampling-based aggregation can almost reach same accuracy for both iid and non-iid setup.
In Figure 2c, while mimic attack does not work for median-based rules in the iid setting, resampling still slightly improves the performance due to variance reduction.
In the non-iid setting, resampling drastically improves the accuracy to the same level as the iid setting.
6. R ESAMPLING AGAINST GENERAL B YZANTINE ATTACKS
In Figure 3, we present thorough experiments on non-iid data over 10 workers with 2 Byzantine workers.
In each subﬁgure, we compare an aggregation rule with its variant with resampling.
Three aggregation rules are compared: K RUM , C M , R FA .
In particular, we compare to R FA with both T= (normalized mean) and T=
(geometric median).
[Figure]
For R FA , T1, refers to the number of inner iterations of Weiszfeld’s algorithm.
bitﬂipping
gaussian attack, as well as the mimic and mimic2 attacks.
Bitﬂipping :
A Byzantine worker ﬂips the sign bits and sends −∇ f ( x ) instead of ∇ f ( x ) because of problems like hardware failures etc.
Labelﬂipping :
The dataset on workers have corrupted labels.
For the MNIST dataset, we let Byzantine workers transform labels by T ( y ) :
= 9 − y .
Gaussian : A Byzantine worker sends a Gaussian random vector of 0 mean and isotropic covariance matrix with standard deviation 200 (Xie et al., 2018).
• mimic & mimic2 :
Explained in Section 3.3.
From Figure 3 we can see that resampling improves the accuracy on most of the tasks.
The ﬁnal accuracies achieved vary with the aggregation rules we use.
Notice that R FA
-T1 is more robust to the mimic attack than R FA -T8 in Figure 3 because more inner iterations lead to better approximate geometric median and less robust to normalized mean attacks.
The normalized mean attack has been addressed in Section 3.2.
C ONCLUSION
In this paper, we initiated a study of robust distributed learning problem under realistic heterogeneous data.
We showed that many existing Byzantine-robust aggregation rules fail under simple new attacks, or sometimes even without any Byzantine workers.
As a solution, we propose a resampling scheme which effectively adapts existing robust algorithms to heterogeneous datasets at a negligible computational cost.
We believe robustness under heterogeneous conditions has been an overlooked direction of research thus far and hope to inspire more work on this topic.
Extending to the decentralized setting, stronger Byzantine adversaries, as well as obtaining optimal algorithms are other challenging directions for future work.
A C ONVERGENCE ANALYSIS OF K RUM WITH RESAMPLING
Theorem II (Resampling K RUM ) .
We assume that (i) the cost function L is three times differentiable with continuous derivatives and non-negative L ( x ) ≥ 0 ; (ii) the learning rates satisfy (cid:80) t γ t = ∞ and (cid:80) t γ 2 t < ∞ .
Let the good workers have stochastic gradients G i ( x , ξ ) for i ∈ good ⊂ [ n ] .
We assume that for a uniformly chosen j ∈ good , the following is true (iii) E j, ξ [ G j ( x , ξ )]
=
∇L ( x ) and ∀ r ∈ { 2 , 3 , 4 } , E j, ξ (cid:107) G j
( x , ξ ) (cid:107) r ≤
A r + B r (cid:107) x (cid:107) r for some constants A r , B r ; (iv) there exists a constant 0 ≤ α < π/ 2 such that for all x we have η ( T, sf ) ·
√ d · σ ( x ) ≤ (cid:107)∇L ( x ) (cid:107) · sin α where σ 2 ( x ) := n − 1 sn − 1 ˜ σ 2 ( x ) ; (v) ﬁnally, beyond a certain horizon, (cid:107) x (cid:107) 2 ≥ D , there exist ε > 0 and 0 ≤ β < π/ 2 − α such that (cid:107)∇L ( x ) (cid:107) ≥ ε > 0 and (cid:104) x , ∇L ( x ) (cid:105) ≥ cos β (cid:107) x (cid:107) · (cid:107)∇L ( x ) (cid:107) .
If s >and 2 sf + 3 ≤ n , then
• K RUM ◦
Resampling is ( α, sf ) -Byzantine resilient where 0 ≤ α < π/ 2 is deﬁned by
[Equation]
• the sequence of gradients ∇L ( x t ) converges almost surely to zero.
Proof.
We only prove the ﬁrst statement.
The second one follows from applying (Blanchard et al., 2017, Proposition 2) directly.
After resampling, we have ˜ n :=
T gradients, and at most ˜ f :
= sf of them are Byzantine.
Without loss of generality, we assume that Byzantine vectors B 1 , . . .
, B ˜ f occupy the last ˜
f positions in the arguments of K RUM .
We denote as K RUM :
= K RUM ( ˜ V 1 , . . . , ˜ V ˜ n − ˜ f , ˜ B 1 , . . . , ˜ B ˜ f ) .
For each index i , we denote by δ c ( i ) (resp.
δ b ( i ) )
the number of correct (resp.
Byzantine) indices j such that i → j (recall again
that i → j denotes that V j belongs to the n − f − 2 closest vectors to V i .
We have
[Equation]
We focus ﬁrst on the condition (i) of ( α, ˜ f ) -Byzantine resilience as in Deﬁnition A.
We determine an upper bound on the squared distance (cid:107) E K RUM − g (cid:107) 2 .
Note that, for any correct j , E ˜ V j = g .
Let i ∗ be the index chosen by K RUM .
[Equation]
where 1 is the indicator function.
Examine i ∗
=
i for correct index
i
[Equation]
where σ 2 := n − 1 sn − 1 ˜ σ 2 in Proposition I. We now examine the case i ∗ =
k for some Byzantine index k .
The fact that k minimizes the score implies for all correct
i
[Equation]
Then, for all correct indices i
[Equation]
[Table]
Focus on D 2 ( i ) :
= (cid:80) i
→ byz l (cid:107) ˜ V
i − ˜ B l (cid:107) 2 , each correct worker i has ˜ n − ˜ f
− 2 neighbors, and ˜ f +non-neighbors.
Thus there exists a correct ζ ( i ) which is farther from i than any of the neighbors of i .
In particular, for each Byzantine index l such that i → l , (cid:107)
˜ V
i − ˜ B l (cid:107) 2 ≤ (cid:107) ˜
V i − ˜
V ζ ( i ) (cid:107) 2 .
Whence (cid:107) ˜ B
k −
[Table]
[Equation]
[Table]
E
[Equation]
[Table] [Equation] [Table] [Equation]
[Table]
[Equation]
[Table]
[Equation] [Table] [Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
Putting everything together we have
[Equation]
[Table]
[Equation]
[Table]
[Equation]
By assumption η 2 (˜ n, ˜ f ) dσ 2 < (cid:107) g (cid:107) , we know
[Equation]
To sum up, (i) of Byzantine resilience holds.
Now we focuse on (ii),
[Equation]
We think of G as the estimator of gradients among good workers.
More speciﬁcally, G is a random variable which uniformly samples one worker among good workers and samples one gradient from the worker.
Then P ( G = v ) = 1 n − f (cid:80) n − f i =1 P ( V i = v ) , and
E
[Equation]
Thus
[Equation]
We have
[Equation]
Denoting C a generic constant, when i ∗
=
k , we have for all correct indices
i
[Equation]
Now
[Equation]
Take expectation to both sides and apply generalized Young’s inequality
[Equation]
17) where p 1 + · · · + p m = 1 , we have
[Equation]
(18) the last inequality comes from (16).
Thus we have proven property (ii) of ( α, ˜ f ) -Byzantine resilience.
A. A LTERNATIVE CONVERGENCE PROOF OF K RUM WITH RESAMPLING
For completeness we also provide an alternative proof for a slight variation of the Byzantine Resilience deﬁnition, under the same algorithm of K RUM with resampling.
Deﬁnition B ( ( α, f ) -Byzantine
Resilience Alternative.) .
Let 0 ≤ α < π/ 2 be any angular value, and any integer 0 ≤ f ≤ n .
Let V 1 , . .
, V n be any identically distributed random vectors in R d ,
V i ∼ G , with E G
= g .
Let B 1 , . . .
B f be any Byzantine vectors in R d , possibly dependent on the
V i ’s. An aggregation rule F is said to be ( α, f ) -Byzantine
resilient if, for any 1 ≤
j 1 < · · · < j f ≤ n , then
[Equation]
satisﬁes (i) (cid:104) E F, g (cid:105) ≥ (1 − sin α ) · (cid:107) g (cid:107) 2 > 0 and (ii) for r = 2 , 3 , 4 , E (cid:107) F (cid:107) r ≤
[Equation]
(cid:107) G (cid:107) r .
Again, similarly as for the above Theorem II we can obtain almost sure convergence analogous to (Blanchard et al., 2017, Proposition 2)
Theorem III (Resampling K RUM , Alternative) .
We assume that (i) the cost function L is three times differentiable with continuous derivatives and non-negative L ( x ) ≥ 0 ; (ii) the learning rates satisfy (cid:80) t γ t = ∞ and (cid:80) t γ 2 t < ∞ ; (iii) the gradient estimator satisﬁes E G ( x , ξ ) =
∇L ( x ) and ∀ r ∈ { 2 , 3 , 4 } , E (cid:107) G ( x , ξ ) (cid:107) r ≤ A r + B r (cid:107) x (cid:107) r for some constants A r , B r ; (iv) there exists a constant 0 ≤ α < π/ 2 such that for all x we have η ( T, sf ) ·√ d · σ ( x ) ≤ (cid:107)∇L ( x ) (cid:107)· sin α ; (v) ﬁnally, beyond a certain horizon, (cid:107) x (cid:107) 2 ≥ D , there exist ε > 0 and 0 ≤ β < π/ 2 − α such that (cid:107)∇L ( x ) (cid:107) ≥ ε >and (cid:104) x , ∇L ( x ) (cid:105) ≥ cos β (cid:107) x (cid:107) · (cid:107)∇L ( x ) (cid:107) .
If s > 1 and 2 sf + 3 ≤ n , then
• K RUM ◦
Resampling is ( α, sf ) -Byzantine resilient as in Deﬁnition B where 0 ≤ α < π/ 2 is deﬁned by
[Equation]
• the sequence of gradients ∇L ( x t ) converges almost surely to zero.
Proof.
A key difference between Deﬁnition B and (Blanchard et al., 2017, Deﬁnition 1) is that
Deﬁnition B removes the independence requirement for the input good gradients { V i } n − f i =1 .
In (Blanchard et al., 2017), there are two propositions: (Blanchard et al., 2017, Proposition 1) proves that K RUM satisfy (Blanchard et al., 2017, Deﬁnition 1), and (Blanchard et al., 2017, Proposition 2) shows almost surely convergence of the gradient ∇L .
In our case, we only need to show that K RUM ◦
Resampling satisfy Deﬁnition B because the convergence of ∇L is identical to the proof of (Blanchard et al., 2017, Proposition 2).
Since the gradients after resampling is identically distributed according to Proposition I, we can keep all of the proofs of (Blanchard et al., 2017, Proposition 1) except for the last inequality where the independence is used.
[Equation]
By applying expectation to both sides of the inequality and the independence of { V i } n − f i =1 , they conclude that
[Equation]
We can immediately show that E (cid:107) B k (cid:107) r ≤ C (cid:107) G (cid:107) r with the help of a general form of Young’s inequality.
We prove this using the standard Young’s inequality.
The Young’s inequality states that for p > 1 , q > 1 , 1 p + 1 q = 1 , a, b ≥ 0 , we have
[Equation]
Let a = (cid:107) V 1 (cid:107) r 1 , b = (cid:107) V 2 (cid:107) r 2 · · · (cid:107) V n − f (cid:107) r n − f , and p =
rr 1 , q = r r − r 1 , we apply Young’s inequality
[Equation]
We can apply Young’s inequality agin for the second term of right hand side.
Let a = (cid:107) V 2 (cid:107) rr
[Equation] [Equation]
, and p
[Equation]
, we apply Young’s inequality
[Equation]
where the second inequality results from recursively applying Young’s inequality
[Equation]
Then we apply expection to both sizes of the inequality gives
[Equation]
Then applying expectation to Equation (20) gives
[Equation]
where C is a general coefﬁcient.
[Equation]
ONVERGENCE OF B YZANTINE RESILIENT SGD
In this section, we obtain a ﬁnite-time convergence guarantee for any algorithm which satisﬁes ( α, f ) -Byzantine resilience.
As far as we are aware, this is the ﬁrst convergence result for K RUM .
From Theorem II, we know that Suppose that the following condition holds for any iterations k for some constants δ ∈ (0 , 1] and β ≥
such that
[Equation]
where F is the output of the robust-aggregation algorithm and g := ∇ f ( x k ) .
This ﬁrst condition bounds the bias wheras the second part bounds the variance of F .
Convergence.
Theorem IV.
Given any biased stochastic estimator F satisfying (24) , the following holds for the update x k +
= x
k − ηF and an L -smooth potentially non-convex f ( x ) lower-bounded by f (cid:63) :
[Equation]
Proof.
The following holds for any η ≤ 1 L :
[Equation]
The ﬁrst inequality uses the smoothness of f , the second separated and bounded the variance of F by β 2 , third inequality follows from rearranging terms and using η ≤ 1 L , and the ﬁnal inequality used our bound on the bias of F .
Now rearranging the terms and averaging over k gives:
[Equation]
Picking η = min( 1 L , (cid:113)
[Equation]
f
[Equation]
x
[Equation]
−
[Equation]
LKβ) gives the desired rate.
Relation to other conditions.
If F is an unbiased stochastic gradient E [ F | x k ] = g and variance σ 2 , then (24) holds with δ = 1 and β = σ .
In this case, Thm.
IV recovers the standard convergence of SGD for smooth non-convex functions.
If instead, we have (cid:104) E F, g (cid:105) ≥ (1 − sin( α ))
(cid:107) g (cid:107) 2 , and E (cid:107) F (cid:107) 2 ≤ (cid:107) g
(cid:107) 2 ≤ G 2 , (cid:107) E F − g (cid:107)
= (cid:107) g (cid:107) 2 − 2 (cid:104) E
[ F ] , g (cid:105) + (cid:107) E
[ F ] (cid:107)≤(cid:107) g (cid:107) 2 − 2 (cid:104) E
[ F ] , g (cid:105) + E (cid:107) F (cid:107)≤ 2 sin( α ) (cid:107) g (cid:107) 2 .
Thus, in this case (24) holds with δ = 1 − 2 sin( α ) , and β = G .
Here, we only get convergence if sin( α ) ≤ 12 .
Further, the assumption that E (cid:107) F (cid:107) 2 ≤ (cid:107) g (cid:107) 2 ≤ G 2 is extremely strong.
C R ESAMPLING WITH REPLACEMENT FOR ROBUST LEARNING
Algorithm
Input: { g i : i ∈ [ n ] } , T , s for t = 1 , . .
, T do
Uniformly sample { g j i } si =1 from { g i : i ∈ [ n ] }
Compute average ¯ g t :
= 1 s (cid:80) si =
g j i
Return { ¯ g t : t ∈ [ T ] } , { j ti : t ∈ [ T ] , i ∈
[ s ] }
Contrary to resampling without replacement which has an explicit bound for malicious gradients after resampling, its with-replacement version has no guarantee for the honest majority.
Lemma 3.
Given a set of vectors { g i : i ∈ [ n ] } of any distributions, the output of Algorithm{ ¯ g t : t ∈
[ T ] } are identically distributed with expectation
[Equation]
If f of the inputs are Byzantine, then a subset S ⊂ { ¯ g t : t ∈ [ T ] } are not faulty.
The cardinality of S is E |S| = (cid:98) T (1 − f/n ) s (cid:99) and the gradients are identically distributed with expectation
[Equation]
Proof.
E ¯ g t =
E
[Equation]
s (cid:80) si =
[Equation]
g i =
[Equation]
(cid:80)
[Equation]
E
[Equation]
i .
There are (1 − f/n ) chance that a sampled gradient g j i is good and (1 − f/n ) s chance that all of the s sampled gradients are good.
Repeat the resampling for T times gives E |S| = (cid:98) T (1 − f/n ) s (cid:99) .
Remark 4.
The ¯ g t has same expectation as minibatch sgd.
We denote the expected number of Byzantine gradients in { ¯ g t : t ∈ [ T ] } as follows:
[Equation]
Since the vectors in S are identically distributed, we can apply robust aggregation rule A , like
Multik-KRUM, to { ¯ g t : t ∈
[ T ] } .
The convergence of Algorithm 1 for Multi-KRUM is stated below.
Theorem V (Resampled KRUM) .
Assume the dataset is decentralized stored.
Assume other conditions in (Blanchard et al., 2017, Prop. 1 & 2) hold true.
K is the number of aggregations.
If 2 ˜ f + 2 < T , then with a probability p K , KRUM ◦ RSWR is Byzantine robust and the sequence of gradients almost surely converges to zero.
The p is deﬁned as
[Equation]
(28) where q = (1 − f/n ) s
Proof.
The probability of ¯ g t is good is q =
(1 − f/n ) s .
The probability of non-faulty majority after
RSWR is thus
[Equation] [Equation]
(cid:101)
[Equation]
(cid:98)
[Equation]
(cid:99)− i
The output of RSWR are identically distributed by Lemma 3.
Thus we know that the robust aggregation rule converge with probability p K .
Remark 5.
Consider the range of T , s , f .
Since Theorem V requires 2 ˜ f + 2 < T , we need s < log(1 / 2+1 /T )
log(1 − f/n ) .
Furthermore, s ≥ 1 is lower bounded, T > 2 n n − 2 f which only requires 2 f < n .
Remark 6.
Notice that the cardinality |S| is stochastic which means it is possible to have a faulty majority in any round.
if the Byzantine gradients are very large, like gaussian attack, the model diverges as soon as one Byzantine gradient is selected.
On the other hand, in the experiment section, we show that for many attacks (labelﬂipping, bitﬂipping, etc.)
the error introduced by faulty majority rounds are amortized overtime.
To ﬁx this issue, the server can normalize all the gradient by their norm such that a faulty majority round would not lead to catestrophic consequences.
D A DDITIONAL EXPERIMENTS D. F ASHION -MNIST
In this subsection
[Figure]
we demonstrate that our algorithm also works on modern dataset like FashionMNISTXiao et al.
Since the Fashion-MNIST is designed to be a drop-in replacement of
MNIST
[Figure]
we conduct experiments on Fashion-MNIST with same setups as Figure 3.
The results are presented in Figure
[Figure]
algorithm.
D. C IFAR -
We run experiments on CIFAR-10 (Krizhevsky et al., 2009) with ResNet-20 (He et al., 2015).
We train our model on 12 nodes which includes 2 Byzantine attacker with mimic attack.
We use K RUM as the aggregation rule for demonstration.
We choose learning rate to be 0 .
1 and batch size per node to be 120 .
The CIFAR-10 dataset is splitted across good nodes such that 50% of samples on each good node has same distribution as overall distribution, the rest 50% are samples from a single class which is different among good workers.
We present our results in Figure 5.
[Figure]
There are 10 good workers and 2 Byzantine workers.
Note that the accuracy in this example is lower than the normal setting because krum may bias towards certain nodes.
Besides, the batch norm maybe inﬂuenced by the heterogenous distribution of data.
D. R ESAMPLING OR FIXED GROUPING
In (Chen et al., 2017), workers are grouped at the beginning of training, and they are trained on i.i.d datasets.
In contrast, resampling is performed every round, and applies to non-iid datasets.
If a
Byzantine worker can predict the random bits on server, resampling becomes grouping in each round which is still stronger than (Chen et al., 2017).
In Figure
[Figure]
we compare K RUM ◦ resampling with vanilla K RUM and K RUM with ﬁxed grouping.
As we can see, the ﬁxed grouping has better accuracy than vanilla K RUM
[Figure]
but weaker than resampling as we expected.
[Figure]
[Table] [Table]
[Table]
[Figure]
[Table]
[Figure]
[Table] [Table]
[Table]
[Figure] [Figure]
tt a c k : m i m i c
[Figure]
s t
A cc
[Figure]
r a c y
[Figure] [Figure]
D. R ESAMPLING HYPERPARAMETER
The resampling hyperparameter s controls the variance reduction, as has been stated in Proposition I.
In Figure 8, we compare the performance of no resampling and resampling with s = 2 , 3 , 4 on heterogenous MNIST dataset.
There are 10 workers in total and no Byzantine workers.
Each experiment has been run for 5 times.
As we can see from Figure 8, higher s leads to faster convergence.
It matches with the Proposition I that higher s leads to greater variance reduction.
[Figure]