Global Convergence and Stability of Stochastic Gradient Descent

In machine learning , stochastic gradient descent ( SGD ) is widely deployed to train models using highly non-convex objectives with equally complex noise models . Unfortunately , SGD theory often makes restrictive assumptions that fail to capture the non-convexity of real problems , and almost entirely ignore the complex noise models that exist in practice . In this work , we make substantial progress on this shortcoming . First , we establish that SGD ’ s iterates will either globally converge to a stationary point or diverge under nearly arbitrary nonconvexity and noise models . Under a slightly more restrictive assumption on the joint behavior of the non-convexity and noise model that generalizes current assumptions in the literature , we show that the objective function can not diverge , even if the iterates diverge . As a consequence of our results , SGD can be applied to a greater range of stochastic optimization problems with confidence about its global convergence behavior and stability . 1 INTRODUCTION . Stochastic Gradient Descent ( SGD ) is a dominant algorithm for solving stochastic optimization problems that arise in machine learning , and has expanded its reach to more complex problems from estimating Gaussian Processes ( Chen et al. , 2020 ) , covariance estimation in stochastic filters ( Kim et al. , 2021 ) , and systems identification ( Hardt et al. , 2016 ; Zhang & Patel , 2020 ) . Accordingly , understanding SGD ’ s behavior has been crucial to its reliable application in machine learning and beyond . As a result , SGD ’ s theory has greatly advanced from a variety of perspectives : global convergence analysis ( Lei et al. , 2019 ; Gower et al. , 2020 ; Khaled & Richtárik , 2020 ; Mertikopoulos et al. , 2020 ; Patel , 2020 ) , local convergence analysis ( Mertikopoulos et al. , 2020 ) , greedy and global complexity analysis ( Gower et al. , 2020 ; Khaled & Richtárik , 2020 ) , asymptotic weak convergence ( Wang et al. , 2021 ) , and saddle point analysis ( Fang et al. , 2019 ; Mertikopoulos et al. , 2020 ; Jin et al. , 2021 ) . While all of these perspectives add new dimensions to our understanding of SGD , the global convergence analysis of SGD is the foundation as it dictates whether local analyses , complexity analyses or saddle point analyses are even warranted . Unfortunately , current global convergence analyses of SGD make restrictive assumptions that fail to capture actual stochastic optimization problems arising in machine learning . In particular , global convergence analyses of SGD assume : 1. a global Hölder constant for the gradient of the objective function ( e.g. , Reddi et al. , 2016a ; Ma & Klabjan , 2017 ; Zhou et al. , 2018 ; Bassily et al. , 2018 ; Lei et al. , 2019 ; Li & Orabona , 2019 ; Gower et al. , 2020 ; Khaled & Richtárik , 2020 ; Mertikopoulos et al. , 2020 ; Patel , 2020 ; Jin et al. , 2021 ) ; 2. unrealistic noise models ( e.g. , uniformly bounded variance ) for the stochastic gradients ( e.g. , Reddi et al. , 2016b ; Ma & Klabjan , 2017 ; Hu et al. , 2019 ; Bi & Gunn , 2019 ; Zou et al. , 2019 ; Mertikopoulos et al. , 2020 ) . While such assumptions often make the analysis simpler , the resulting global convergence results would not even apply to simple neural network models for binary classification ( see Appendix A ) . To address this gap , we analyze the global convergence of SGD on nonconvex stochastic optimization problems that capture many actual machine learning applications . In particular , we assume local Hölder continuity of the gradient function ( see Assumption 2 ) , which substantially relaxes the global Hölder assumption of all previous works . Second , we assume that the noise model of the stochastic gradients is bounded by an arbitrary upper semi-continuous function of the parameter ( see Assumption 4 ) , and can even have infinite variance ( c.f. , Wang et al. , 2021 ) , which generalizes the assumptions of all previous work . With these two general assumptions , we prove that the iterates of SGD will either converge to a stationary point or diverge to infinity with probability one ( see Theorem 2 ) . Owing to our result , SGD can be applied to actual empirical risk minimization problems with guarantees about its asymptotic behavior . In the process of proving Theorem 2 , we also prove another almost remarkable claim about the behavior of SGD ’ s iterates : that of all the possible asymptotic behaviors of the iterates ( e.g. , convergence to a fixed point , convergence to a manifold , limit cycles , oscillation between points , divergence ) , even with rather arbitrary noise models , the only two possibilities are either the iterates converge to a fixed point or they diverge ( see Theorem 1 ) . Note , we can not expect this outcome apriori even in such a simple context as applying SGD with fixed step sizes to solve a consistent linear system : the iterates may terminate in a cycle and , thus , fail to converge to a fixed point ( Motzkin & Schoenberg , 1954 , Theorem 2 ) . Thus , from a practical perspective , applying SGD to nonconvex problems with exotic noise models , which may initially cause concern , will either result in convergence to a stationary point or divergence of the iterates . While Theorem 2 is patently useful , SGD ’ s iterates diverging may cause some concern , especially when optimizing regularized empirical risk functions that are guaranteed to be coercive ( i.e. , the objective diverges to infinity as the argument tends to infinity ) . To address this issue , we generalize the notion of expected smoothness ( see Khaled & Richtárik , 2020 ) to an assumption about the joint behavior of the gradient function , the noise model , and the local Hölder constant ( see Assumption 5 ) to prove that , regardless SGD ’ s iterates ’ behavior , the objective function will remain finite and the gradient function will converge to zero with probability one ( see Theorem 3 ) . Thus , from a practical perspective , if we can apply Theorem 3 to a coercive objective function , we are guaranteed that the iterates can not diverge , and , consequently , must converge to a stationary point . Furthering the practical value of our results , as alluded to previously , our results enable the use of downstream analyses . Specifically , our results allow for SGD ’ s iterates to converge to stationary point or diverge ; as a result , when SGD ’ s iterates converge to a stationary point , saddle point escape analyses ( e.g. , Fang et al. , 2019 ; Mertikopoulos et al. , 2020 ; Jin et al. , 2021 ) can be applied to ensure that the stationary point is a local minimizer . Moreover , when SGD is converging to a stationary point , local convergence rate analyses can also be supplied , which can inform adaptive step size rules and stopping criteria ( Patel , 2020 ) . Finally , from a theoretical perspective , we innovate two new analysis strategies to deal with the generality of the local Hölder continuity assumption on the gradient , and our general noise model assumption . We term these two strategies the pseudo-global strategy and the local strategy . We develop the pseudo-global strategy to prove global convergence ( i.e. , Theorem 2 ) , while we develop the local strategy to prove stability ( i.e. , Theorem 3 ) . We believe that both of our strategies are of independent interest to theoreticians . Contribution Summary . To summarize , we study the behavior of SGD under much more realistic assumptions than what is currently in the literature ; namely , we consider local Hölder continuity and general noise models . In this context , we are able to show ( 1 ) that the iterates must either converge to a fixed point or that they might diverge ( Theorem 1 ) ; and ( 2 ) , when the iterates converge to a fixed point , it must be a stationary point of the objective function ( Theorem 2 ) . Moreover , under a slightly more restrictive assumption—which still generalizes current assumptions in the literature— , we show that , regardless of iterate behavior , the objective function will converge to a finite random variable ( i.e. , SGD is stable ) and the gradient function will converge to zero ( Theorem 3 ) . Finally , we develop two new analysis strategies , the pseudo-global strategy and the local strategy , that are of independent interest to theoreticians in machine learning and stochastic optimization . Organization . The remainder of the paper is organized as follows . In Section 2 , we specify the stochastic optimization problem that we will study , including a formal statement of all assumptions . In Section 3 , we specify Stochastic Gradient Descent ( SGD ) and the properties of the needed properties of the learning rate . In Section 4 , we prove and discuss our main results and highlight key steps , while leaving the rest to the appendix . In Section 5 , we conclude this work . 2 STOCHASTIC OPTIMIZATION . We consider solving the optimization problem min θ∈Rp { F ( θ ) : = E [ f ( θ , X ) ] } , ( 1 ) where F maps Rp into R ; f maps Rp and the co-domain of the random variable X into R ; and E is the expectation operator . As we require gradients , we take F and f to differentiable in θ , and denote its derivatives with respect to θ by Ḟ ( θ ) and ḟ ( θ , X ) . With this notation , we make the following general assumptions about the deterministic portion of the objective function . Assumption 1 . There exists Fl.b . ∈ R such that ∀θ ∈ Rp , Fl.b . ≤ F ( θ ) . Assumption 2 . There exists α ∈ ( 0 , 1 ] such that Ḟ ( θ ) is locally α-Hölder continuous . Assumptions 1 and 2 would even be considered mild in the context of non-convex deterministic optimization , in which it is also common to assume that the objective function well-behaved level sets ( e.g. , Nocedal & Wright , 2006 , Theorems 3.2 , 3.8 , 4.5 , 4.6 ) . Importantly , Assumption 2 relaxes the common restrictive assumption of globally Hölder continuous gradient functions that is common in other analyses . Our final step is to make some assumptions about the stochastic portion of the objective function . The first assumption requires the stochastic gradients to be unbiased , which can readily be relaxed ( Bottou et al. , 2018 ) . The second assumption allows for a generic noise model for an α-Hölder continuous gradient function , and even allows for the second moment to not exist when α < 1 ( c.f . Wang et al. , 2021 ) . Assumption 3 . For all θ ∈ Rp , Ḟ ( θ ) = E [ ḟ ( θ , X ) ] . Assumption 4 . Let α ∈ ( 0 , 1 ] be as in Assumption 2 . There exists an upper semi-continuous function G ( θ ) such that E [ ‖ḟ ( θ , X ) ‖1+α2 ] ≤ G ( θ ) . We will show that Assumptions 1 to 4 are sufficient for a global convergence result . Remark 1 . It is entirely possible that E [ ‖ḟ ( θ , X ) ‖1+α2 ] is ( at least ) upper semi-continuous , and to set G ( θ ) equal to this function . In the case that E [ ‖ḟ ( θ , X ) ‖1+α2 ] is not upper semi-continuous , it is possible to specify G ( θ ) as the upper envelope of E [ ‖ḟ ( θ , X ) ‖1+α2 ] ( i.e. , the its limit supremum function ) . However , it is unlikely that E [ ‖ḟ ( θ , X ) ‖1+α2 ] nor its upper envelope are easy to specify explicitly , and it is more likely to be able to find an upper bound . In order to show that the objective function can not diverge ( i.e. , to prove stability ) , we will need an additional assumption . This assumption will relate the gradient function , noise model and variation on the local Hölder constant . To begin , we define the variation on the local Hölder constant . Let α ∈ ( 0 , 1 ] be as in Assumption 2 and > 0 be arbitrary , and define L ( θ ) = supϕ { ‖Ḟ ( ϕ ) −Ḟ ( θ ) ‖ 2 ‖ϕ−θ‖α2 : ‖ϕ− θ‖2 ≤ ( G ( θ ) ∨ ) 1 1+α } if this quantity is nonzero otherwise , ( 2 ) where ∨ indicates the maximum between two quantities . Note , the choice of is irrelevant , and they can be distinct for the two cases in the definition of L , but we fix them to be the same for simplicity . Importantly , the quantity , L , is defined at every parameter θ under Assumption 2 . With this quantity , we can state a nonintuitive , technical assumption that is needed to prove stability . Assumption 5 . There exists C1 , C2 , C3 ≥ 0 such that , ∀θ ∈ Rp , L ( θ ) G ( θ ) + α ∥∥∥Ḟ ( θ ) ∥∥∥1+α 2 L ( θ ) 1/α ≤ C1 + C2 ( F ( θ ) − Fl.b . ) + C3 ∥∥∥Ḟ ( θ ) ∥∥∥2 2 . ( 3 ) Assumption 5 generalizes Assumption 4.3 ( c ) of Bottou et al . ( 2018 ) , which is satisfied for a large swath of statistical models . Moreover , Assumption 5 generalizes the notion of expected smoothness ( see Khaled & Richtárik , 2020 , for a history of the assumption ) , which expanded the optimization problems covered by the theory of Bottou et al . ( 2018 ) . Note , Assumption 5 is about the asymptotic properties of the stochastic optimization problem as the left hand side of the inequality in Assumption 5 can be bounded inside of any compact set . Thus , Assumption 5 covers a variety of asymptotic behaviors , such as exp ( ‖θ‖22 ) , exp ( ‖θ‖2 ) , ‖θ‖r2 for r ∈ R , log ( ‖θ‖2+1 ) , and log ( log ( ‖θ‖2+1 ) +1 ) . Therefore , Assumption 5 holds for functions with a variety of different asymptotic behaviors . We will show that Assumptions 1 to 5 are sufficient for a stability result . Now that we have specified the nature of the stochastic optimization problem , we turn our attention to the algorithm used to solve the problem , namely , stochastic gradient descent ( SGD ) .

Summary: In this paper, the authors consider the global convergence and stability of stochastic gradient descent in a fairly general non-convex setting. They are able to remove the often-assumed unrealistic uniform bounded assumption on the noise, and also relax the global Holder assumption in the literature. Their discussions in Appendix A provide an example for which the uniform bounded assumption on the noise commonly assumed in the literature fails. Their global convergence says that under some relatively weak assumptions, SGD either diverges or the objective converges to a finite random variable and the gradient converges to zero. This excludes the bad outcomes, e.g. limit cycle or oscillation. Their stability result says that under a stronger assumption, SGD's objective converges to a finite random variable and the gradient converges to zero with probability one.