On Stochastic Sign Descent Methods

1 INTRODUCTION . One of the key factors behind the success of modern machine learning models is the availability of large amounts of training data ( Bottou & Le Cun , 2003 ; Krizhevsky et al. , 2012 ; Schmidhuber , 2015 ) . However , the state-of-the-art deep learning models deployed in industry typically rely on datasets too large to fit the memory of a single computer , and hence the training data is typically split and stored across a number of compute nodes capable of working in parallel . Training such models then amounts to solving optimization problems of the form minx∈Rd f ( x ) : = 1 M M∑ m=1 fm ( x ) , ( 1 ) where fm : Rd → R represents the non-convex loss of a deep learning model parameterized by x ∈ Rd associated with data stored on node m. Arguably , stochastic gradient descent ( SGD ) ( Robbins & Monro , 1951 ; Vaswani et al. , 2019 ; Qian et al. , 2019 ) in of its many variants ( Kingma & Ba , 2015 ; Duchi et al. , 2011 ; Schmidt et al. , 2017 ; Zeiler , 2012 ; Ghadimi & Lan , 2013 ) is the most popular algorithm for solving ( 1 ) . In its basic implementation , all workers m ∈ { 1 , 2 , . . . , M } in parallel compute a random approximation ĝm ( xk ) of ∇fm ( xk ) , known as the stochastic gradient . These approximations are then sent to a master node which performs the aggregation ĝ ( xk ) : = 1 M M∑ m=1 ĝm ( xk ) . The aggregated vector is subsequently broadcast back to the nodes , each of which performs an update of the form xk+1 = xk − γkĝ ( xk ) , thus updating their local copies of the parameters of the model . 1.1 GRADIENT COMPRESSION . Typically , communication of the local gradient estimators ĝm ( xk ) to the master forms the bottleneck of such a system ( Seide et al. , 2014 ; Zhang et al. , 2017 ; Lin et al. , 2018 ) . In an attempt to alleviate this communication bottleneck , a number of compression schemes for gradient updates have been proposed and analyzed ( Alistarh et al. , 2017 ; Wang et al. , 2018 ; Wen et al. , 2017 ; Khirirat et al. , 2018 ; Mishchenko et al. , 2019 ) . A compression scheme is a ( possibly randomized ) mapping Q : Rd → Rd , applied by the nodes to ĝm ( xk ) ( and possibly also by the master to aggregated update in situations when broadcasting is expensive as well ) in order to reduce the number of bits of the communicated message . Sign-based compression . Although most of the existing theory is limited to unbiased compression schemes , i.e. , on operators Q satisfying EQ ( x ) = x , biased schemes such as those based on communicating signs of the update entries only often perform much better ( Seide et al. , 2014 ; Strom , 2015 ; Wen et al. , 2017 ; Carlson et al. , 2015 ; Balles & Hennig , 2018 ; Bernstein et al. , 2018 ; 2019 ; Zaheer et al. , 2018 ; Liu et al. , 2019 ) . The simplest among these sign-based methods is signSGD ( see also Algorithm 1 ; Option 1 ) , whose update direction is assembled from the component-wise signs of the stochastic gradient . Adaptive methods . While ADAM is one of the most popular adaptive optimization methods used in deep learning ( Kingma & Ba , 2015 ) , there are issues with its convergence ( Reddi et al. , 2019 ) and generalization ( Wilson et al. , 2017 ) properties . It was noted in Balles & Hennig ( 2018 ) that the behaviour of ADAM is similar to a momentum version of signSGD . Connection between sign-based and adaptive methods has long history , originating at least in Rprop ( Riedmiller & Braun , 1993 ) and RMSprop ( Tieleman & Hinton , 2012 ) . Therefore , investigating the behavior of signSGD can improve our understanding on the convergence of adaptive methods such as ADAM . 1.2 CONTRIBUTIONS . We now summarize the main contributions of this work . Our key results are summarized in Table 1 . 1In fact , bounded variance assumption , being weaker than bounded second moment assumption , is stronger ( or , to be strict , more curtain ) than SPB assumption in the sense of differential entropy , but not in the direct sense . The entropy of probability distribution under the bounded variance assumption is bounded , while under the SPB assumption it could be arbitrarily large . This observation is followed by the fact that for continuous random variables , the Gaussian distribution has the maximum differential entropy for a given variance ( see https : //en.wikipedia.org/wiki/Differential_entropy ) . • 2 methods for 1-node setup . In the M = 1 case , we study two general classes of sign based methods for minimizing a smooth non-convex function f . The first method has the standard form2 xk+1 ← xk − γk sign ĝ ( xk ) , ( 2 ) while the second has a new form not considered in the literature before : xk+1 ← arg min { f ( xk ) , f ( xk − γk sign ĝ ( xk ) ) } . ( 3 ) • Key novelty . The key novelty of our methods is in a substantial relaxation of the requirements that need to be imposed on the gradient estimator ĝ ( xk ) of the true gradient ∇f ( xk ) . In sharp contrast with existing approaches , we allow ĝ ( xk ) to be biased . Remarkably , we only need one additional and rather weak assumption on ĝ ( xk ) for the methods to provably converge : we require the signs of the entries of ĝ ( xk ) to be equal to the signs of the entries of ∇f ( xk ) with a probability strictly larger than 1/2 ( see Section 2 ; Assumption 1 ) . We show through a counterexample ( see Section 2.2 ) that this assumption is necessary . • Geometry . As a byproduct of our analysis , we uncover a mixed l1-l2 geometry of sign descent methods ( see Section 3 ) . • Convergence theory . We perform a complexity analysis of methods ( 2 ) and ( 3 ) ( see Section 4.1 ; Theorem 1 ) . While our complexity bounds have the same O ( 1/√K ) dependence on the number of iterations , they have a better dependence on the smoothness parameters associated with f . Theorem 1 is the first result on signSGD for non-convex functions which does not rely on mini-batching , and which allows for step sizes independent of the total number of iterations K. Finally , Theorem 1 in Bernstein et al . ( 2019 ) can be recovered from our general Theorem 1 . Our bounds are cast in terms of a novel norm-like function , which we call the ρ-norm , which is a weighted l1 norm with positive variable weights . • Distributed setup . We extend our results to the distributed setting with arbitrary M ( Section 4.2 ) , where we also consider sign-based compression of the aggregated gradients . 2 SUCCESS PROBABILITIES AND GRADIENT NOISE . In this section we describe our key ( and weak ) assumption on the gradient estimator ĝ ( x ) of the true gradient∇f ( x ) , and give an example which shows that without this assumption , method ( 2 ) can fail . 2.1 SUCCESS PROBABILITY BOUNDS . Assumption 1 ( SPB : Success Probability Bounds ) . For any x ∈ Rd , we have access to an independent ( and not necessarily unbiased ) estimator ĝ ( x ) of the true gradient g ( x ) : = ∇f ( x ) that satisfies ρi ( x ) : = Prob ( sign ĝi ( x ) = sign gi ( x ) ) > 1 2 , if gi ( x ) 6= 0 ( 4 ) for all x ∈ Rd and all i ∈ { 1 , 2 , . . . , d } . We will refer to the probabilities ρi as success probabilities . As we will see , they play a central role in the convergence of sign based methods . We stress that Assumption 1 is the only assumption on gradient noise in this paper . Moreover , we argue that it is reasonable to require from the sign of stochastic gradient to show true gradient direction more likely than the opposite one . Extreme cases of this assumption are the absence of gradient noise , in which case ρi = 1 , and an overly noisy stochastic gradient , in which case ρi ≈ 12 . Remark 1 . Assumption 1 can be relaxed by replacing bounds ( 4 ) with E [ sign ( ĝi ( x ) · gi ( x ) ) ] > 0 , if gi ( x ) 6= 0 . However , if Prob ( sign ĝi ( x ) = 0 ) = 0 ( e.g . in the case of ĝi ( x ) has continuous distributions ) , then these two bounds are identical . 2sign g is applied element-wise to the entries g1 , g2 , . . . , gd of g ∈ Rd . For t ∈ R we define sign t = 1 if t > 0 , sign t = 0 if t = 0 , and sign t = −1 if t < 0 . Extension to stochastic sign oracle . Notice that we do not require ĝ to be unbiased . Moreover , we do not assume uniform boundedness of the variance , or of the second moment . This observation allows to extend existing theory to more general sign-based methods with a stochastic sign oracle . By a stochastic sign oracle we mean an oracle that takes xk ∈ Rd as an input , and outputs a random vector ŝk ∈ Rd with entries in ±1 . However , for the sake of simplicity , in the rest of the paper we will work with the signSGD formulation , i.e. , we let ŝk = sign ĝ ( xk ) . 2.2 A COUNTEREXAMPLE TO SIGNSGD . Here we analyze a counterexample to signSGD discussed in Karimireddy et al . ( 2019 ) . Consider the following least-squares problem with unique minimizer x∗ = ( 0 , 0 ) : min x∈R2 f ( x ) = 12 [ 〈a1 , x〉2 + 〈a2 , x〉2 ] , a1 = [ 1+ε −1+ε ] , a2 = [ −1+ε 1+ε ] , where ε ∈ ( 0 , 1 ) and stochastic gradient ĝ ( x ) = ∇〈ai , x〉2 = 2〈ai , x〉ai with probabilities 1/2 for i = 1 , 2 . Let us take any point from the line l = { ( z1 , z2 ) : z1 + z2 = 2 } as initial point x0 for the algorithm and notice that sign ĝ ( x ) = ± ( 1 , −1 ) for any x ∈ l. Therefore , signSGD with any step-size sequence remains stuck along the line l , whereas the problem has a unique minimizer at the origin . We now investigate the cause of the divergence . In this counterexample , Assumption 1 is violated . Indeed , note that sign ĝ ( x ) = ( −1 ) i sign〈ai , x〉 [ −1 1 ] with probabilities 12 for i = 1 , 2 . By S : = { x ∈ R2 : 〈a1 , x〉 · 〈a2 , x〉 > 0 } 6= ∅ denote the open cone of points having either an acute or an obtuse angle with both ai ’ s . Then for any x ∈ S , the sign of the stochastic gradient is ± ( 1 , −1 ) with probabilities 1/2 . Hence for any x ∈ S , we have low success probabilities : ρi ( x ) = Prob ( sign ĝi ( x ) = sign gi ( x ) ) ≤ 12 , i = 1 , 2 . So , in this case we have an entire conic region with low success probabilities , which clearly violates ( 4 ) . Furthermore , if we take a point from the complement open cone S̄c , then the sign of stochastic gradient equals to the sign of gradient , which is perpendicular to the axis of S ( thus in the next step of the iteration we get closer to S ) . For example , if 〈a1 , x〉 < 0 and 〈a2 , x〉 > 0 , then sign ĝ ( x ) = ( 1 , −1 ) with probability 1 , in which case x − γ sign ĝ ( x ) gets closer to low success probability region S. In summary , in this counterexample there is a conic region where the sign of the stochastic gradient is useless ( or behaves adversarially ) , and for any point outside that region , moving direction ( which is the opposite of the sign of gradient ) leads toward that conic region .

Summary: The paper presents an improved analysis of the signSGD gradient estimator. The authors propose to relax the requirements on the gradient estimator in Bernstein (2019). The only requirement imposed on the gradient is that it should have the correct sign with probability greater than 1/2. In particular this approach allows the gradient estimate to be biased as opposed to Bernstein (2019) which requires unbiased gradients. The authors also show this condition to be necessary by a small counterexample.