Byzantine-Resilient Non-Convex Stochastic Gradient Descent

1 INTRODUCTION . Motivated by the pervasiveness of large-scale distributed machine learning , there has recently been significant interest in providing distributed optimization algorithms with strong fault-tolerance guarantees . In this context , the strongest , most stringent fault model is that of Byzantine faults ( Lamport et al. , 1982 ) : given m machines , each having access to private data , at most an α fraction of the machines can behave in arbitrary , possibly adversarial ways , with the goal of breaking or slowing down the algorithm . Although extremely harsh , this fault model is the “ gold standard ” in distributed computing ( Lynch , 1996 ; Lamport et al. , 1982 ; Castro et al. , 1999 ) , as algorithms proven to be correct in this setting are guaranteed to converge under arbitrary system behaviour . A setting of particular interest in this context has been that of distributed stochastic optimization . Here , the task is to minimize some stochastic function f ( x ) = Es∼D [ fs ( x ) ] over a distribution D , where fs ( · ) can be viewed as the loss function for sample s ∼ D. We assume there are m machines ( workers ) and an honest master , and α < 1/2 fraction of the workers may be Byzantine . In each iteration t , each worker has access to a version of the global iterate xt , which is maintained by the master . The worker can independently sample s ∼ D , compute ∇fs ( xt ) , and then synchronously send this stochastic gradient to the master . The master aggregates the workers ’ messages , and sends an updated iterate xt+1 to all the workers . Eventually , the master has to output an approximate minimizer of f . Clearly , the above description only applies to honest workers ; Byzantine workers may deviate arbitrarily and return adversarial “ gradient ” vectors to the master in every iteration . This distributed framework is quite general and well studied . One of the first references in this setting studied distributed PCA and regression ( Feng et al. , 2014 ) . Other early approaches ( Blanchard et al. , 2017 ; Chen et al. , 2017 ; Su & Vaidya , 2016a ; b ; Xie et al. , 2018a ) relied on defining generalizations of the geometric median . These approaches can withstand up to half of the nodes being malicious , but can have relatively high local computational cost Ω ( m2d ) ( Blanchard et al. , 2017 ; Chen et al. , 2017 ) , where m is the number of nodes and d is the problem dimension , and usually have suboptimal sample and iteration complexities . Follow-up work resolved this last issue when the objective f ( · ) is convex , leading to tight sample ∗The full and future editions of this paper can be found on https : //arxiv.org/abs/2012.14368 . †Microsoft Research Redmond , zeyuan @ csail.mit.edu ‡University of Waterloo , faezeeb75 @ gmail.com §Microsoft Research Redmond , jerrl @ microsoft.com ¶IST Austria , dan.alistarh @ ist.ac.at complexity bounds . Specifically , Yin et al . ( 2018 ) provided bounds for gradient descent-type algorithms , and showed that the bounds are tight when the dimension is constant . Alistarh et al . ( 2018 ) provided a stochastic gradient descent ( SGD ) type algorithm and showed that its sample and time complexities are asymptotically optimal even when the dimension is large . Non-convex Byzantine-resilient stochastic optimization . In this paper , we focus on the more challenging non-convex setting , and shoot for the strong goal of finding approximate local minima ( a.k.a . second-order critical points ) . In a nutshell , our main result is the following . Fix d to denote the dimension , and let the objective f : Rd → R be Lipschitz smooth and second-order smooth . We have m worker machines , each having access to unbiased , bounded estimators of the gradient of f . Given an initial point x0 , the SafeguardSGD algorithm ensures that , even if at most α < 1/2 fraction of the machines are Byzantine , after T = Õ ( ( α2 + 1m ) d ( f ( x0 ) −min f ( x ) ) ε4 ) parallel iterations , for at least a constant fraction of the indices t ∈ [ T ] , the following hold : ‖∇f ( xt ) ‖ ≤ ε and ∇2f ( xt ) − √ εI . If the goal is simply ‖∇f ( xt ) ‖ ≤ ε , then T = Õ ( ( α2 + 1m ) ( f ( x0 ) −min f ( x ) ) ε4 ) iterations suffice . Here , the Õ notation serves to hide logarithmic factors for readability . We spell out these factors in the detailed analysis . • When α < 1/ √ m , our sample complexity ( = mT ) matches the best known result in the non- Byzantine case ( Jin et al. , 2019 ) without additional assumptions , and enjoys linear parallel speedup : with m workers of which < √ m are Byzantine , the parallel speedup is Ω̃ ( m ) .1 • For α ∈ [ 1/ √ m , 1/2 ) , our parallel time complexity is Õ ( α2 ) times that needed when no parallelism is used . This still gives parallel speedup . This α2 factor appears in convex Byzantine distributed optimization , where it is tight ( Yin et al. , 2018 ; Alistarh et al. , 2018 ) . • The Lipschitz and second-order smoothness assumptions are the minimal assumptions needed to derive convergence rates for finding second-order critical points ( Jin et al. , 2019 ) . Comparison with prior bounds . The closest known bounds are by Yin et al . ( 2019 ) , who derived three gradient descent-type of algorithms ( based on median , mean , and iterative filtering ) to find a weaker type of approximate local minima . Since it relies on full gradients , their algorithm is arguably less practical , and their time complexities are generally higher than ours ( see Section 2.1 ) . Other prior works consider a weaker goal : to find approximate stationary points ‖∇f ( x ) ‖ ≤ ε only : Bulusu et al . ( 2020 ) additionally assumed there is a guaranteed good ( i.e . non-Byzantine ) worker known by the master , Xie et al . ( 2018b ) gave a practical algorithm when the Byzantine attackers have no information about the loss function or its gradient , Yang et al . ( 2019 ) ; Xie et al . ( 2018a ) ; Blanchard et al . ( 2017 ) derived eventual convergence without an explicit complexity bound , and the non-convex result obtained in Yin et al . ( 2018 ) is subsumed by Yin et al . ( 2019 ) , discussed above . Our algorithm and techniques . The structure of our algorithm is deceptively simple . The master node keeps track of the sum of gradients produced by each worker across time . It labels ( allegedly ) good workers as those whose sum of gradients “ concentrate ” well with respect to a surrogate of the median vector , and labels bad workers otherwise . Once a worker is labelled bad , it is removed from consideration forever . The master then performs the vanilla SGD , by moving in the negative direction of the average gradients produced by those workers currently labelled as good . We call our algorithm SafeguardSGD , since it behaves like having a safe guard to filter away bad workers . Its processing overhead at the master is O ( md ) , negligible compared to standard SGD . As the astute reader may have guessed , the key non-trivial technical ingredient is to identify the right quantity to check for concentration , and make it compatible with the task of non-convex optimization . In particular , we manage to construct such quantities so that ( 1 ) good non-Byzantine workers never get mislabelled as bad ones ; ( 2 ) Byzantine workers may be labelled as good ones ( which is inevitable ) but when they do , the convergence rates are not impacted significantly ; and ( 3 ) the notion does not require additional assumptions or running time overhead . The idea of using concentration ( for each worker across time ) to filter out Byzantine machines 1By parallel speedup we mean the reduction in wall-clock time due to sampling gradients in parallel among the m nodes . In each time step , the algorithm generates m new gradients , although some may be corrupted . traces back to the convex setting ( Alistarh et al. , 2018 ) . However , the quantities used in ( Alistarh et al. , 2018 ) to check for concentration are necessarily different from this paper , and our analysis is completely new , as deriving non-convex rates is known to be much more delicate and challenging . Recently , Bulusu et al . ( 2020 ) used similar concentration filters to Alistarh et al . ( 2018 ) in the nonconvex setting , but under stronger assumptions , and for the simpler task of finding stationary points . Many other algorithms do not rely on concentration filters . In each iteration , they ask each worker to compute a batch of stochastic gradients , and then use coordinate-wise median or mean over the batch average ( e.g . Yin et al . ( 2018 ; 2019 ) ; Yang et al . ( 2019 ) ) or iterative filtering ( e.g . Su & Xu ( 2018 ) ; Yin et al . ( 2019 ) ) by the master to derive a “ robust mean. ” These works fundamentally rely on each iteration to calculate an almost precise full gradient , so that they can apply a surrogate of full gradient descent . Such algorithms can introduce higher sample and time complexities ( see Section 2 ) , are less practical than stochastic gradient schemes , require additional restrictions on the resilience factor α , e.g . α < 1/4 ( Su & Xu , 2018 ) , and , critically , have been shown to be vulnerable to recent attacks ( Baruch et al. , 2019 ; Xie et al. , 2020 ) . Attack resilience and experimental validation . There is a growing literature on customized attacks against Byzantine-resilient algorithms , showing that many defenses can be entirely circumvented in real-world scenarios ( Baruch et al. , 2019 ; Xie et al. , 2020 ) . Our algorithm is provably correct against these attacks , a fact we also validate experimentally . We implemented SafeguardSGD to examine its practical performance against a range of prior works ( Xie et al. , 2018b ; Blanchard et al. , 2017 ; Chen et al. , 2017 ; Yin et al. , 2018 ; 2019 ) , and against recent attacks on the distributed task of training deep neural networks . Our experiments show that SafeguardSGD generally outperforms previous methods in convergence speed and final accuracy , sometimes by a wide accuracy margin . This is true not only against known Byzantine attacks , but also against attack variants we fine-crafted to specifically slow down our algorithm , and against transient node failures . 2 STATEMENT OF OUR THEORETICAL RESULT . We denote by ‖ · ‖ the Euclidean norm and [ n ] : = { 1 , 2 , . . . , n } . Given symmetric matrices A , B , we let ‖A‖2 denote the spectral norm of A . We use to denote Loewner ordering , i.e . A B if A−B is positive semi-definite . We denote by λmin ( A ) the minimum eigenvalue of matrix A . We consider arbitrary d-dimensional non-convex functions f : Rd → R satisfying the following : • f ( x ) is L-Lipschitz smooth : meaning ‖∇f ( x ) −∇f ( y ) ‖ ≤ L‖x− y‖ for any x , y ∈ Rd ; • f ( x ) is L2-second-order smooth : ‖∇2f ( x ) −∇2f ( y ) ‖2 ≤ L2 · ‖x− y‖ for any x , y ∈ Rd ; For notational simplicity of the proofs , we assume L = L2 = V = 1.2 Note that we have also assumed the domain of f is the entire space Rd . If instead there is a compact domain X ⊂ Rd , then one can use projected SGD and re-derive similar results of this paper . We choose to present our result in the simplest setting to convey our main ideas . Byzantine non-convex stochastic distributed optimization . We let m be the number of worker machines and assume at most an α fraction of them are Byzantine for α ∈ [ 0 , 12 ) . We denote by good ⊆ [ m ] the set of good ( i.e . non-Byzantine ) machines , and the algorithm does not know good . Assumption 2.1 . In each iteration t , the algorithm ( on the master ) is allowed to specify a point xt and query m machines . Each machine i ∈ [ m ] gives back a vector∇t , i ∈ Rd satisfying • If i ∈ good , the stochastic gradient∇t , i satisfies E [ ∇t , i ] = ∇f ( xt ) and ‖∇f ( xt ) −∇t , i‖ ≤ V .3 • If i ∈ [ m ] \ good , then∇t , i can be arbitrary ( w.l.o.g . we assume ‖∇f ( xt ) −∇t , i‖ ≤ V ) .4 Remark 2.2 . For each t and i 6∈ good , the vector ∇t , i can be adversarially chosen and may depend 2In the literature of convergence analysis for non-convex optimization , the final complexity bounds naturally and polynomially depend on these parameters L , L2 , V , and the way the dependence goes is typically unique ( Allen-Zhu , 2018a ; b ; Fang et al. , 2018 ; Jin et al. , 2019 ) . This is why it suffices to ignore their appearance and only compare the polynomial dependence on ε and d. 3One can instead assume Pr [ ‖∇f ( xt ) − ∇t , i‖ > t ] ≤ 2 exp ( −t2/2V2 ) and the results of this paper continue to hold up to logarithmic factors . To present the simplest theory , we do not include that version in this paper . We refer interested readers to Jin et al . ( 2019 ) for how to deal with such probabilistic assumption ( when there is no Byzantine worker ) . 4This requirement ‖∇f ( xt ) −∇t , i‖ ≤ V is “ without loss of generality ” because it is trivial for the algorithm to catch bad machines if they output∇t , i more than 2V away from the majorities . Algorithm 1 SafeguardSGD : perturbed SGD with double safe guard Input : point x0 ∈ Rd , rate η > 0 , lengths T ≥ T1 ≥ T0 ≥ 1 , threshold T1 > T0 > 0 ; 1 : good0 ← [ m ] ; 2 : for t← 0 to T − 1 do 3 : last1 ← max { t1 ∈ [ t ] : t1 is a multiple of T1 } ; 4 : last0 ← max { t0 ∈ [ t ] : t0 is a multiple of T0 } 5 : for each i ∈ goodt do 6 : receive∇t , i ∈ Rd from machine i ; 7 : Ai ← ∑t k=last1 ∇k , i |goodk| and Bi ← ∑t k=last0 ∇k , i |goodk| ; 8 : Amed ← Ai where i ∈ goodt is any machine s.t . ∣∣ { j ∈ goodt : ‖Aj −Ai‖ ≤ T1 } ∣∣ > m/2 . 9 : Bmed ← Bi where i ∈ goodt is any machine s.t . ∣∣ { j ∈ goodt : ‖Bj −Bi‖ ≤ T0 } ∣∣ > m/2 . 10 : goodt+1 ← { i ∈ goodt : ‖Ai −Amed‖ ≤ 2T1 ∧ ‖Bi −Bmed‖ ≤ 2T0 } ; 11 : xt+1 = xt − η ( ξt + 1 |goodt| ∑ i∈goodt ∇t , i ) ; Gaussian noise ξt ∼ N ( 0 , ν2I ) on { ∇t′ , i } t′≤t , i∈ [ m ] . In particular , the Byzantine machines can even collude during an iteration .

Summary: The paper considers stochastic gradient descent convergence in a distributed setting with m workers, where up to α workers can be Byzantine, i.e. perform in an arbitrarily adversarial way. In this setting, they develop a variant of SGD which finds a second-order stationary point, prevents Byzantine workers from significantly affecting convergence, and achieves α^2 + 1/m speedup compared with the sequential case. The main idea of the algorithm is to measure deviations of gradient updates for a certain number of rounds and detect Byzantine machines which must have a significant deviation to noticeably affect the algorithm’s behavior.