Distributed Momentum for Byzantine-resilient Stochastic Gradient Descent

1 Introduction . Stochastic Gradient Descent ( SGD ) is one of the main optimization algorithm used throughout machine learning . Scaling SGD can mean aggregating more but inevitably less well-sanitized data , and distributing the training over several machines , making SGD even more vulnerable to Byzantine faults : corrupted/malicious training datapoints , software vulnerabilities , etc . Many Byzantine-resilient techniques have been proposed to keep SGD safer from these faults , e.g . Alistarh et al . ( 2018 ) ; Damaskinos et al . ( 2018 ) ; Yang & Bajwa ( 2019b ) ; TianXiang et al . ( 2019 ) ; Bernstein et al . ( 2019 ) ; Yang & Bajwa ( 2019a ) ; Yang et al . ( 2019 ) ; Rajput et al . ( 2019 ) ; Muñoz-González et al . ( 2019 ) . These techniques mainly use the same adversarial model ( Figure 2 ) : a central , trusted parameter server distributing gradient computations to several workers , a minority of which is controlled by an adversary and can submit arbitrary gradients . Two families of defense techniques can be distinguished . The first employs redundancy schemes , inspired by coding theory . This approach has strong resilience guarantees , but * Author list written in alphabetical order , as for all the papers from the DCL at EPFL . its requirement to share data between workers makes this approach unsuitable for several classes of applications , e.g . when data can not be shared for privacy , scalability or legal reasons . The second family uses statistically-robust aggregation schemes , and is the focus of this paper . The underlying idea is simple . At each training step , the server aggregates the stochastic gradients computed by the workers into one gradient , using a function called a Byzantine-resilient Gradient Aggregation Rule ( GAR ) . These statistically-robust GARs are designed to produce at each step a gradient that is expected to decrease the loss . Intuitively , one can think of this second family as different formulations of the multivariate median . In particular , if the non-Byzantine gradients were all equal at each step , any different ( adversarial ) gradient would be rejected by each of these medians , and no attack would succeed . But due to their stochastic nature , the non-Byzantine gradients are different : their variance is strictly positive . Formal guarantees on any given statistically-robust GAR typically require that the variance-norm ratio , the ratio between the variance of the nonByzantine gradients and the norm of the expected non-Byzantine gradient , remains below a certain constant ( constant which depends on the GAR itself and fixed hyperparameters ) . Intuitively , this notion of variance-norm ratio can be comprehended quite analogously to the inverse of the signal-to-noise ratio ( i.e . the “ noise-to-signal ” ratio ) in signal processing . However , Baruch et al . ( 2019 ) noted that an attack could send gradients that are close to non-Byzantine outlier gradients , building an apparent majority of gradients that could be sufficiently far from the expected non-Byzantine gradient to increase the loss . This can happen against most statistically-robust GARs in practice , as the variance-norm ratio is often too large for them . Two recent attacks ( Baruch et al. , 2019 ; Xie et al. , 2019a ) were able to exploit this fact to substantially hamper the training process ( which our experiments confirm ) . The work presented here aims at ( substantially ) improving the resilience of statistically robust GARs “ also in practice ” , by reducing the variance-norm ratio of the gradients received by the server . We do that by taking advantage of an old technique normally used for acceleration : momentum . This technique is regularly applied at the server , but instead we propose to confer it upon each distributed worker , effectively making the Byzantine-resilient GAR aggregate accumulated gradients . Crucially , there is no computational complexity attached to our reformulation : it only reorders operations in existing ( distributed ) algorithms . Contributions . Our main contributions can be summarized as follows : • A reformulation of classical/Nesterov momentum which can significantly improve the effectiveness ( Figure 1 ) of any statistically-robust Gradient Aggregation Rule ( GAR ) . We formally analyze the impact of our reformulation on the variance-norm ratio of the aggregated gradients , ratio on which the studied GARs assume an upper bound . • An extensive and reproducible1 set of experiments substantiating the effectiveness of our reformulation of momentum in improving existing defenses against state-of-the-art attacks . Paper Organization . Section 2 provides the necessary background . Section 3 presents our distributed momentum scheme and provides some intuitions on its effects . Formal developments of these intuitions are given in the appendix . Section 4 describes our experimental settings in details , before presenting and analysing some of our experimental results . The appendix reports on the entirety of our experiments , and details how they can be reproduced ( in one command , graphs included ) . Section 5 discusses related and future work . 2 Background . 2.1 Byzantine Distributed SGD . Stochastic Gradient Descent ( SGD ) . We consider the classical problem of optimizing a non-convex , differentiable loss function Q : Rd → R , where Q ( θt ) , E x∼D [ q ( θt , x ) ] for a fixed data distribution D. Ideally , we seek θ∗ such that θ∗ = arg minθ ( Q ( θ ) ) . We employ mini-batch SGD optimization . Starting from initial parameter θ0 ∈ Rd , at every step t ≥ 0 , b samples ( x ( 1 ) t . . . x ( b ) t ) are sampled from D to estimate one stochastic gradient gt , 1 b ∑b k=1∇q ( θt , x ( k ) t ) ≈ ∇Q ( θt ) . This stochastic gradient is then used to update the parameters θt , with : θt+1 = θt − αt gt . The sequence αt > 0 is called the learning rate . Classical and Nesterov momentum One field-tested amendment to mini-batch SGD is classical momentum ( Polyak , 1964 ) , where each gradient keeps an exponentially-decreasing effect on every subsequent update . Formally : θt+1 = θt − αt ∑t u=0 µ t−ugu , with 0 < µ < 1 . Nesterov ( 1983 ) proposed another revision . Noting vt the velocity vector , v0 = 0 , formally : vt+1 = µ vt + 1 b b∑ k=1 ∇q ( θt − αt µ vt , x ( k ) t ) θt+1 = θt − αt vt+1 Compared to classical momentum , the gradient is estimated at θt − αt µ vt instead of θt . Distributed SGD with Byzantine workers . We follow the parameter server model ( Li et al. , 2014 ) : one single process ( the parameter server ) holding the parameter vector θt ∈ Rd , and n other ( the workers ) estimating gradients . Among these n workers , up to f < n are said Byzantine , i.e . adversarial . Unlike the other n − f honest workers , these f Byzantine workers can submit arbitrary gradients ( Figure 2 ) . At each step t , the parameter server receives n different gradients g ( 1 ) t . . . g ( n ) t , among which f are arbitrary ( submitted by the Byzantine workers ) . So the update equation becomes : θt+1 = θt − αtGt , where : Gt , t∑ u=0 µt−uF ( g ( 1 ) u , . . . , g ( n ) u ) ( 1 ) 1Namely : 736 × 5 = 3680 seeded runs , and one single script to reproduce all of our results . Function F is called a Gradient Aggregation Rule ( GAR ) . In non-Byzantine settings , averaging is used ; formally : F ( g ( 1 ) t , . . . , g ( n ) t ) = 1n ∑n i=1 g ( i ) t . In the presence of Byzantine workers , a more robust aggregation is performed with a Byzantine-resilient GAR . Sections 2.2 and 2.3 respectively describe the 6 existing GARs and 2 attacks studied in this paper . Adversarial Model . The goal of the adversary is to impede the learning process , which is defined as the maximization of the loss Q or , more judiciously for the image classification tasks tackled in this paper , as the minimization2 of the model ’ s top-1 cross-accuracy . The adversary can not directly overwrite θt at the parameter server . The adversary only submits f arbitrary gradients to the server per step , via the f Byzantine workers it controls3 . We assume an omniscient adversary . In particular , the adversary knows the GAR used by the parameter server and , at each step , the adversary can generate Byzantine gradients dependent on the honest gradients submitted at the same step and any previous step . 2.2 Byzantine-resilient GARs . We briefly present below the 6 studied Gradient Aggregation Rules ( GARs ) . These GARs are Byzantine-resilient ( Section A ) , a notion first introduced by Blanchard et al . ( 2017 ) under the name ( α , f ) -Byzantine-resilience . When used within its operating assumptions , a Byzantine-resilient GAR guarantees convergence even in an adversarial setting . Let n be the number of gradients the parameter server received from the n workers ( Figure 2 ) , and let f be the maximum number of Byzantine gradients the GAR must tolerate . Krum ( Blanchard et al. , 2017 ) . Each received gradient is assigned a score . The score of gradient x is the sum of the squared ` 2-distances between x and the n−f−2 closest gradients to x . The aggregated gradient is then the arithmetic mean of the n− f − 2 gradients with the smallest scores . This variant is called Multi-Krum in the original paper . To be proven ( α , f ) -Byzantine resilient , Krum requires the variance of the honest gradients E ‖Gt − EGt‖ 2 to be bounded above as follows : 2 · ( n−f+ f ( n−f−2 ) +f 2 ( n−f−1 ) n−2f−2 ) · E ‖Gt − EGt‖ 2 < ‖EGt‖ 2 ( 2 ) Median ( Yin et al. , 2018 ) . The coordinate-wise median of the n received gradients . Median is proven ( α , f ) -Byzantine resilience with the following condition on the variance-norm ratio : ( n− f ) · E ‖Gt − EGt‖ 2 < ‖EGt‖ 2 ( 3 ) Trimmed Mean ( Yin et al. , 2018 ) . The coordinate-wise trimmed-mean of the n received gradients . The trimmed-mean of n values is the arithmetic mean , after the f smallest and the f largest values have been discarded , of the remaining values . From Theorem 1 of Xie et al . ( 2018b ) , we can derive the following condition on the variance-norm ratio : 2 ( f+1 ) ( n−f ) ( n−2f ) 2 E ‖Gt − EGt‖ 2 < ‖EGt‖ 2 ( 4 ) Phocas ( Xie et al. , 2018b ) . The coordinate-wise arithmetic mean of the n − f closest values to the coordinate-wise trimmed-mean . From Theorem 2 of Xie et al . ( 2018b ) : ( 4 + 12 ( f+1 ) ( n−f ) ( n−2f ) 2 ) E ‖Gt − EGt‖ 2 < ‖EGt‖ 2 ( 5 ) 2Exempli gratia , with 10 classes , the worst possible final accuracy is arguably 0.1 . 3Said otherwise , the f Byzantine workers can collude . MeaMed ( Xie et al. , 2018a ) . Same as Phocas , but with median replacing trimmed-mean . Theorem 5 of Xie et al . ( 2018a ) provides the following condition : 10 ( n− f ) · E ‖Gt − EGt‖ 2 < ‖EGt‖ 2 ( 6 ) Bulyan ( El-Mhamdi et al. , 2018 ) . This is a composite GAR , iterating on another GAR in a first selection phase . In the remaining of this paper , Bulyan will use Krum , so the first phase selects n− 2 f − 2 gradients , at each iteration removing the highest scoring gradient . The aggregated gradient is the coordinate-wise arithmetic mean of the n− 4 f − 2 closest values to the ( coordinate-wise ) median of the selected gradients . The theoretical requirement on the variance-norm ratio are the same as the ones of the underlying GAR . That is , in this paper , they are the same as Krum ( Equation 2 ) .

Summary: The paper describes an approach to counteract Byzantine attacks for distributed stochastic gradient descent by using the momentum of the gradient computed at the workers, which relies only on memory of the previous momentum. This seems to thwart current attacks in the majority of scenarios tested. The theoretical analysis seems appropriate. The empirical results are accompanied by precise details and should facilitate reproducibility (given a week of computation time).