Continuized Accelerations of Deterministic and Stochastic Gradient Descents, and of Gossip Algorithms

1 Introduction . In the last decades , the emergence of numerous applications in statistics , machine learning and signal processing has led to a renewed interest in first-order optimization methods [ 10 ] . They enjoy a low iteration cost necessary to the analysis of large datasets . The performance of first-order methods was largely improved thanks to acceleration techniques ( see the review by d ’ Aspremont et al . [ 14 ] and the many references therein ) , starting with the seminal work of Nesterov [ 42 ] . Let f : Rd → R be a convex and differentiable function , minimized at x∗ ∈ Rd . We assume throughout the paper that f is L-smooth , i.e. , ∀x , y ∈ Rd , f ( y ) 6 f ( x ) + 〈∇f ( x ) , y − x〉+ L 2 ‖y − x‖2 . ( 1 ) 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . In addition , we sometimes assume that f is µ-strongly convex for some µ > 0 , i.e. , ∀x , y ∈ Rd , f ( y ) > f ( x ) + 〈∇f ( x ) , y − x〉+ µ 2 ‖y − x‖2 . ( 2 ) For the problem of minimizing f , gradient descent is well-known to achieve a rate f ( xk ) − f ( x∗ ) = O ( k−1 ) in the smooth case , and a rate f ( xk ) − f ( x∗ ) = O ( ( 1− µ/L ) k ) in the smooth and strongly convex case . In both cases , Nesterov introduced an alternative method with essentially the same iteration cost , while achieving faster rates : it converges with rate O ( k−2 ) in the smooth convex case and with rate O ( ( 1− √ µ/L ) k ) in the smooth and strongly convex case [ 43 ] . These rates are then optimal among all black-box first-order methods that access gradients and linearly combine them [ 43 , 41 ] . Nesterov acceleration relies on several sequences of iterates—two or three , depending on the formulation—and on a clever blend of gradient steps and mixing steps between the sequences . Different interpretations and motivations underlying the precise structure of accelerated schemes were approached in many works , including [ 12 , 24 , 3 , 32 , 2 ] . A large number of these works studied continuous time equivalents of Nesterov acceleration , obtained by taking the limit when stepsizes vanish , or from a variational framework . The continuous time index t of the limit allowed to use differential calculus to study the convergence of these equivalents . Examples of studies relying on continuous time interpretation include [ 50 , 33 , 54 , 53 , 9 , 18 , 48 , 49 , 4 , 5 , 57 , 40 ] . Continuized Nesterov acceleration . In this paper , we propose another continuous time equivalent to Nesterov acceleration , which we refer to as the continuized Nesterov acceleration , which avoids vanishing stepsizes . It is built by considering two sequences xt , zt ∈ Rd , t ∈ R > 0 , that continuously mix following a linear ordinary differential equation ( ODE ) , and that take gradient steps at random times T1 , T2 , T3 , . . . . Thus , in this modeling , mixing and gradient steps alternate randomly . Thanks to the continuous index t and some stochastic calculus , one can differentiate averaged quantities ( expectations ) with respect to t. In particular , this leads to simple analytical expressions for the optimal parameters as functions of t , while the optimal parameters of Nesterov accelerations are defined by recurrence relations that are complicated to solve . The discretization x̃k = xTk , z̃k = zTk , k ∈ N , of the continuized process can be computed directly and exactly : the result is a recursion of the same form as Nesterov iteration , but with randomized parameters , and performs similarly to Nesterov original deterministic version both in theory and in simulations . The continuized framework can be adapted to various settings and extensions of Nesterov acceleration . In what follows , we study how the continuized acceleration behaves in the presence of additive and multiplicative noise in the gradients . In the multiplicative noise setting , our acceleration satisfies a convergence rate similar to that of [ 30 ] and depends on the statistical condition number of the problem at hand . The two acceleration schemes are not directly comparable as we work in a continuized setting and only deal with pure multiplicative noise . Our analysis is nevertheless much simpler , as it closely mimics that of Nesterov acceleration . Application to accelerated gossip algorithms . The continuized modeling is natural in asynchronous parallel computing where gradient steps arrive at random times . More importantly , there are situations where the continuized version of Nesterov acceleration can be practically implemented while the original acceleration can not . In distributed settings , for instance , the total number k of gradient steps taken in the network is typically not known to each particular node ; an advantage of the continuized acceleration is that it requires to know only the time t and not k. Gossip algorithms typically feature such asynchronous and distributed behaviors [ 11 ] . In gossip problems , nodes of a network aim at computing the global average of all their values by communicating only locally ( with their neighbors ) , and without centralized coordination . In this set-up , pairs of adjacent nodes communicate at random times , asynchronously , and in parallel , so that the total number of past communications in the network at a given time is unknown to all nodes . In this paper , we formulate the gossip problem as a stochastic optimization problem . Thanks to the continuized formalism , we naturally obtain accelerated gossip algorithms that can be implemented in an asynchronous and distributed fashion . Synchronous gossip algorithms rely on all nodes to communicate simultaneously [ 19 ] . Accelerating synchronous gossip algorithms have been studied in previous works , including SSDA [ 47 ] , Chebyshev acceleration [ 39 ] , Jacobi-Polynomial acceleration [ 7 ] . To that day , acceleration in the asynchronous setting has also been studied in a few works ( see for instance geographic gossip [ 20 ] , shift registers [ 37 ] , ESDAC [ 25 ] , and randomized Kaczmarz methods [ 38 ] ) . However , no algorithm in an asynchronous framework has been rigorously proven to achieve an accelerated rate for general graphs [ 21 ] . Other acceleration schemes [ 25 , 38 ] relied on additional synchronizations between nodes , such as the knowledge of a global iteration counter . This departs from purely asynchronous operations , hence causing practical limitation . Our accelerated randomized gossip algorithm ( Section 6 ) recovers the same accelerated rates , and only requires the knowledge of a common continuous-time t ∈ R > 0 . In this context , the continuized acceleration should be seen as a close approximation to Nesterov acceleration , that features both an insightful and convenient expression as a continuous time process and a direct implementation as a discrete iteration . We thus hope to contribute to the understanding of Nesterov acceleration . In practice , the continuized framework is relevant for handling asynchrony in decentralized optimization , where agents of a network can not share a global iteration counter , preventing accelerated decentralized and asynchronous methods . Notations . The index k always denotes a non-negative integer , while indices t , s always denote non-negative reals . Structure of the paper . In Section 2 , we recall standard results on gradient descent and Nesterov acceleration . In Section 3 , we introduce a continuized variant of Nesterov acceleration . In Section 4 , we show that discretizing the continuized acceleration yields an iterative method similar to that of Nesterov but with random parameters . In Section 5 , we study continuized Nesterov acceleration under pure-multiplicative noise . We finally present accelerated asynchronous algorithms for the gossip problem in Section 6 , as well as for decentralized optimization in Section 7 . 2 Reminders on Nesterov acceleration . For the sake of comparison , let us first recall the classical Nesterov acceleration . To improve the convergence rate of gradient descent , Nesterov introduced iterations of three sequences , parametrized by τk , τ ′k , γk , γ ′ k , k > 0 , of the form yk = xk + τk ( zk − xk ) , ( 3 ) xk+1 = yk − γk∇f ( yk ) , ( 4 ) zk+1 = zk + τ ′ k ( yk − zk ) − γ′k∇f ( yk ) . ( 5 ) Depending on whether the function f is known to be ( 1 ) convex , or ( 2 ) strongly convex with a known strong convexity parameter , Nesterov provided a set of parameter choices for achieving acceleration . Theorem 1 ( Convergence of accelerated gradient descent ) . Nesterov accelerated scheme satisfies : 1 . Choose the parameters τk = 1− AkAk+1 , τ ′ k = 0 , γk = 1 L , γ ′ k = Ak+1−Ak L , k > 0 , where the sequence Ak , k > 0 , is defined by the recurrence relation A0 = 0 , Ak+1 = Ak + 1 2 ( 1 + √ 4Ak + 1 ) . Then f ( xk ) − f ( x∗ ) 6 2L‖x0 − x∗‖2 k2 . 2 . Assume further that f is µ-strongly convex , µ > 0 . Choose the constant parameters τk ≡ √ µ/L 1+ √ µ/L , τ ′k ≡ √ µ L , γk ≡ 1 L , γ ′ k ≡ 1√µL , k > 0 . Then f ( xk ) − f ( x∗ ) 6 ( f ( x0 ) − f ( x∗ ) + µ 2 ‖z0 − x∗‖2 ) ( 1− √ µ L ) k . This result can be found as is in d ’ Aspremont et al . [ 14 , Sections 4.4.1 and 4.5.3 ] . In the convex case , Nesterov acceleration achieves the rate O ( 1/k2 ) , whereas gradient descent achieves a rate O ( 1/k ) ( see [ 43 , Corollary 2.1.2 ] for instance ) . In the strongly convex case , Nesterov acceleration achieves the rate O ( ( 1− √ µ/L ) k ) , where gradient descent achieves a rate O ( ( 1−µ/L ) k ) ( see [ 43 , Theorem 2.1.15 ] for instance ) . In both cases , this results in a significant speedup in practice , see Figure 1 . From a high-level perspective , Nesterov acceleration iterates over several variables , alternating between gradient steps ( always with respect to the gradient at yk ) and mixing steps , where the running value of a variable is replaced by a linear combination of the other variables . However , the precise way gradient and mixing steps are coupled is rather mysterious , and the success of the proof of Theorem 1 relies heavily on the detailed structure of the iterations . In the next section , we try to gain perspective on this structure by developing a continuized version of the acceleration . 3 Continuized version of Nesterov acceleration . This paper uses several mathematical notions related to random processes . The following sections expose the results from heuristic considerations of those notions , rigorously defined in Appendix C. We argue that the accelerated iteration becomes more natural when considering two variables xt , zt indexed by a continuous time t > 0 , that are continuously mixing and that take gradient steps at random times . More precisely , let T1 , T2 , T3 , . . . > 0 be random times such that T1 , T2 − T1 , T3 − T2 , . . . are independent identically distributed ( i.i.d . ) , of law exponential with rate 1 ( any constant rate would do , we choose 1 to make the comparison with discrete time k straightforward ) . By convention , we choose that our stochastic processes t 7→ xt , t 7→ zt are càdlàg almost surely , i.e. , right continuous with well-defined left-limits xt− , zt− ( Definition 6 in Appendix C ) . Our dynamics are parametrized by functions γt , γ′t , τt , τ ′ t , t > 0 . At random times T1 , T2 , . . . , our sequences take gradient steps xTk = xTk− − γTk∇f ( xTk− ) , ( 6 ) zTk = zTk− − γ′Tk∇f ( xTk− ) . ( 7 ) Because of the memoryless property of the exponential distribution , in a infinitesimal time interval [ t , t+ dt ] , the variables take gradients steps with probability dt , independently of the past . Between these random times , the variables mix through a linear , translation-invariant , ordinary differential equation ( ODE ) dxt = ηt ( zt − xt ) dt , ( 8 ) dzt = η ′ t ( xt − zt ) dt . ( 9 ) Following the notation of stochastic calculus , we can write the process more compactly in terms of the Poisson point measure dN ( t ) = ∑ k > 1 δTk ( dt ) , which has intensity the Lebesgue measure dt , dxt = ηt ( zt − xt ) dt− γt∇f ( xt ) dN ( t ) , ( 10 ) dzt = η ′ t ( xt − zt ) dt− γ′t∇f ( xt ) dN ( t ) . ( 11 ) Before giving convergence guarantees for such processes , let us digress quickly on why we can expect an iteration of this form to be mathematically appealing . First , from a Markov chain indexed by a discrete time index k , one can associate the so-called continuized Markov chain , indexed by a continuous time t , that makes transition with the same Markov kernel , but at random times , with independent exponential time intervals [ 1 ] . Following this terminology , we refer to our acceleration ( 10 ) - ( 11 ) as the continuized acceleration . The continuized Markov chain is appreciated for its continuous time parameter t , while keeping many properties of the original Markov chain ; similarly the continuized acceleration is arguably simpler to analyze , while performing similarly to Nesterov acceleration . Second , it can also be compared with coordinate gradient descent methods , that are easier to analyze when coordinates are selected randomly rather than in an ordered way [ 55 ] . Similarly , the continuized acceleration is simpler to analyze because the gradient steps ( 6 ) - ( 7 ) and the mixing steps ( 8 ) - ( 9 ) alternate randomly , due to the randomness of Tk , k > 0 . In analogy with Theorem 1 , we give choices of parameters that lead to accelerated convergence rates , in the convex case ( 1 ) and in the strongly convex case ( 2 ) . Convergence is analyzed as a function of t. As dN ( t ) is a Poisson point process with rate 1 , t is the expected number of gradient steps done by the algorithm . Thus t is analoguous to k in Theorem 1 . In the theorem below , E denotes the expectation with respect to the Poisson point process dN ( t ) , the only source of randomness . Theorem 2 ( Convergence of continuized Nesterov acceleration ) . The continuized Nesterov acceleration satisfies the following two points . 1 . Choose the parameters ηt = 2t , η ′ t = 0 , γt = 1 L , γ ′ t = t 2L . Then Ef ( xt ) − f ( x∗ ) 6 2L‖z0 − x∗‖2 t2 . 2 . Assume further that f is µ-strongly convex , µ > 0 . Choose the constant parameters ηt = η ′ t ≡ √ µ L , γt ≡ 1 L , γ ′ t ≡ 1√µL . Then Ef ( xt ) − f ( x∗ ) 6 ( f ( x0 ) − f ( x∗ ) + µ 2 ‖z0 − x∗‖2 ) exp ( − √ µ L t ) . We give an elementary sketch of proof in Appendix D.1 and a complete proof in Appendix D.2 . Many authors have proposed continuous-time versions of Nesterov acceleration using differential calculus , see the numerous references in the introduction . For instance , in Su et al . [ 50 ] , an ODE is obtained from Nesterov acceleration by taking the joint asymptotic where the stepsizes vanish and the number of iterates is rescaled . The resulting ODE must be discretized to be implemented ; choosing the right discretization is not straightforward as it introduces stability and approximation errors that must be controlled [ 57 , 49 , 46 ] . On the contrary , our continuous time process ( 10 ) - ( 11 ) does not correspond to a limit where the stepsizes vanish . However , in Appendix F , we check that the random continuized acceleration has the same deterministic ODE scaling limit as Nesterov acceleration . This sanity check emphasizes that the continuized acceleration is fundamentally different from previous continuous-time equivalents . Remark 1 . A similar Markovian structure can be obtained in a discrete setting by flipping i.i.d . coins to trigger gradient steps . By denoting p > 0 the probability to trigger a gradient step when flipping a coin , ( i ) p = 1 gives the classical setting , and ( ii ) p → 0 while renormalizing time gives our continuized framework . In fact , this setting with updates triggered randomly is an interpolation between the classical and continuized frameworks , and consists in replacing exponential random variables by geometric random variables of parameter p for the waiting-time between updates . We thus believe the convergence guarantees described here and in the following can be adapted for this discrete scheme .

Summary: This paper introduces a new continuous-time framework for understanding Nesterov’s accelerated gradient descent. The authors consider the following dynamics for two variables (x, z): at the jump times of a Poisson process, each of the two variables x and z takes a step in the negative gradient of the objective function. In between these jumps, the variables x and z mix continuously according to an ODE. The authors show that with appropriate choices for the parameters, this process exhibits similar guarantees as Nesterov’s algorithm.  Continuous-time versions of Nesterov’s algorithm have been considered before, but usually from the perspective of letting the step size of the algorithm tend to zero and thus recovering an ODE. Although it is usually much simpler to study the resulting ODE, it is quite non-trivial to start with an ODE and find a suitable discretization that preserves its convergence guarantees; thus, many of these prior works simply reverse-engineer Nesterov’s algorithm without providing an understanding of how to produce discretized accelerated algorithms. The main novelty in the present approach is that there is no need to discretize the continuous-time dynamics; the gradient steps already happen at discrete times (the jumps of a Poisson process) which can be simulated (with the caveat that the algorithm is now randomized). In between the jumps, the pair (x, z) evolves according to an ODE which does not depend on the objective function (besides the smoothness/convexity parameters). For the parameter choices leading to accelerated rates for convex-smooth and strongly convex-smooth classes, the authors show that in fact this ODE can be solved in closed form. Even if this were not the case, it is noteworthy that this ODE can be implemented in the usual oracle model of optimization.  Besides introducing this framework, the authors provide extensions to stochastic gradients with “additive” or “multiplicative” noise, as well as applications to accelerated gossip.