First-Order Optimization Inspired from Finite-Time Convergent Flows

1 INTRODUCTION . Consider the unconstrained minimization problem for a given cost function f : Rn → R. When f is sufficiently regular , the standard algorithm in continuous time ( dynamical system ) is given by ẋ = FGF ( x ) , −∇f ( x ) ( 1 ) with ẋ , ddtx ( t ) , known as the gradient flow ( GF ) . Generalizing GF , the q-rescaled GF ( qRGF ) Wibisono et al . ( 2016 ) given by ẋ = Fq−RGF ( x ) = −c ∇f ( x ) ‖∇f ( x ) ‖ q−2 q−1 2 ( 2 ) with c > 0 and q ∈ ( 1 , ∞ ] has an asymptotic convergence rate f ( x ( t ) ) − f ( x ? ) = O ( 1 tq−1 ) under mild regularity , for ‖x ( 0 ) − x ? ‖ > 0 small enough , where x ? ∈ Rn denotes a local minimizer of f . However , it has been recently proven Romero & Benosman ( 2020 ) that q-RGF , as well as q-signed GF ( q-SGF ) ẋ = Fq−SGF ( x ) = −c ‖∇f ( x ) ‖ 1 q−1 1 sign ( ∇f ( x ) ) , ( 3 ) where sign ( · ) denotes the sign function ( element-wise ) , are both finite-time convergent ( in continuoustime ) , provided that f is gradient dominated of order p ∈ ( 1 , q ) . In particular , if f is strongly convex , then q-RGF and q-SGF is finite-time convergent for any q ∈ ( 2 , ∞ ] , since f must be gradient dominated of order p = 2 . Considering that many algorithms are inspired from continuous flows with convergence guarantee , e.g. , Muehlebach & Jordan ( 2019 ) ; Fazlyab et al . ( 2017a ) ; Shi et al . ( 2018 ) ; Zhang et al . ( 2018 ) ; França et al . ( 2019b ) ; Wibisono et al . ( 2016 ) , a natural question arises : what is the convergence rate of the corresponding discrete-time algorithms , which are induced by discretization of finite-time convergent continuous-time flows ? 1.1 CONTRIBUTION . In this paper , we investigate the convergence behavior of an Euler discretization for the q-RGF ( equation 2 ) and q-SGF ( equation 3 ) . We provide convergence guarantees in terms of closeness of solutions , using results from hybrid dynamical control theory . Furthermore , we provide iteration / sample complexity upper bounds for both the general ( deterministic ) and stochastic settings . We then test the performance of the proposed algorithms on both synthetic and real-world data in the context of deep learning , namely , on the well-known SVHN dataset . 1.2 RELATED WORK . Propelled by the work of Wang & Elia ( 2011 ) and Su et al . ( 2014 ) , there has been a recent and significant research effort dedicated to analyzing optimization algorithms from the perspective of dynamical systems and control theory , especially in continuous time ( Wibisono et al. , 2016 ; Wilson , 2018 ; Lessard et al. , 2016 ; Fazlyab et al. , 2017b ; Scieur et al. , 2017 ; Franca et al. , 2018 ; Fazlyab et al. , 2018 ; Fazlyab et al. , 2018 ; Taylor et al. , 2018 ; França et al. , 2019a ; Orvieto & Lucchi , 2019 ; Romero et al. , 2019 ; Muehlebach & Jordan , 2019 ) . A major focus within this initiative is in accceleration , both in terms of trying to gain new insight into more traditional optimization algorithms from this perspective , or even to exploit the interplay between continuous-time systems and their potential discretizations for novel algorithm design ( Muehlebach & Jordan , 2019 ; Fazlyab et al. , 2017a ; Shi et al. , 2018 ; Zhang et al. , 2018 ; França et al. , 2019b ; Wilson et al. , 2019 ) . Many of these papers also focus on deriving convergence rates based on the discretization of flows designed in the continuous-time domain . Connecting ordinary differential equations ( ODEs ) and their numerical analysis , with optimization algorithms is a very important topic , which can be dated back to 1970s , see Botsaris ( 1978a ; b ) ; Zghier ( 1981 ) ; Snyman ( 1982 ; 1983 ) ; Brockett ( 1988 ) ; Brown ( 1989 ) . In Helmke & Moore ( 1994 ) , the authors studied relationships between linear programming , ODEs , and general matrix theory . Further , Schropp ( 1995 ) and Schropp & Singer ( 2000 ) explored several aspects linking nonlinear dynamical systems to gradient-based optimization , including nonlinear constraints . Tools from Lyapunov stability theory are often employed for the analysis , mainly because there already exists a rich body of works within the nonlinear systems and control theory community for this purpose . In particular , typically in previous works , one seeks asymptotically Lyapunov stable gradient-based systems with an equilibrium ( stationary point ) at an isolated extremum of the given cost function , thus certifying local convergence . Naturally , the global asymptotic stability leads to global convergence , though such analysis will typically require the cost function to be strongly convex everywhere . For physical systems , a Lyapunov function can often be constructed from first principles via some physically meaningful measure of energy ( e.g. , total energy = potential energy + kinetic energy ) . In optimization , the situation is somewhat similar in the sense that a suitable Lyapunov function may often be constructed by taking simple surrogates of the objective function as candidates . For instance , V ( x ) , f ( x ) − f ( x ? ) can be a good initial candidate . Further , if f is continuously differentiable and x ? is an isolated stationary point , then another alternative is V ( x ) , ‖∇f ( x ) ‖2 . However , most fundamental and applied research conducted in systems and control regarding Lyapunov stability theory deals exclusively with continuous-time systems . Unfortunately , ( dynamical ) stability properties are generally not preserved for simple forward-Euler and sample-and-hold discretizations and control laws ( Stuart & Humphries , 1998 ) . Furthermore , practical implementations of optimization algorithms in modern digital computers demand discrete-time . Nonetheless , it has been extensively noted that a vast amount of general Lyapunov-based results ( perhaps most ) appear to have a discrete-time equivalent . 2 PRELIMINARIES . 2.1 OPTIMIZATION ALGORITHMS AS DISCRETE-TIME SYSTEMS . Generalizing ( 1 ) , ( 2 ) , and ( 3 ) , consider a continuous-time algorithm ( dynamical system ) modeled via an ordinary differential equation ( ODE ) ẋ = F ( x ) ( 4 ) for t ≥ 0 , or , more generally , a differential inclusion ẋ ( t ) ∈ F ( x ( t ) ) ( 5 ) a.e . t ≥ 0 ( e.g . for the q = ∞ case ) , such that x ( t ) → x ? as t → t ? . In the case of the q-RGF ( 2 ) and q-SGF ( 3 ) for f gradient dominated of order p ∈ ( 1 , q ) , we have finite-time convergence , and thus t ? = t ? ( x ( 0 ) ) < ∞ . Most of the popular numerical optimization schemes can be written in a state-space form ( i.e. , recursively ) , as Xk+1 = Fd ( k , Xk ) ( 6a ) xk = G ( Xk ) ( 6b ) for k ∈ Z+ , { 0 , 1 , 2 , . . . } and a given X0 ∈ Rm ( typically m ≥ n ) , where Fd : Z+ × Rm → Rm and G : Rm → Rn . Naturally , ( 6 ) can be seen as a discrete-time dynamical system constructed by discretizing ( 4 ) in time . In particular , we have xk ≈ x ( tk ) , where { 0 = t0 < t1 < t2 < . . . } denotes a time partition and x ( · ) a solution to ( 4 ) or ( 5 ) as appropriate . Therefore , we call Xk and xk , respectively , the state and output at time step k. Example 1 . The standard gradient descent ( GD ) algorithm xk+1 = xk − η∇f ( xk ) ( 7 ) with step size ( learning rate ) η > 0 can be readily written in the form ( 6 ) by taking m = n , Fd ( x ) , x− η∇f ( x ) , and G ( x ) , x . • If the step sizes are adaptive , i.e . if we replace η by a sequence { ηk } with ηk > 0 , then we only need to replace Fd ( k , x ) , x− ηk∇f ( x ) , provided that { ηk } is not computed using feedback from { xk } ( e.g . through a line search method ) . • If we do wish to use time-varying step-size , then we can set m = n + 1 , G ( [ x ; η ] ) , x , and Fd ( [ x ; η ] ) , [ F ( 1 ) d ( [ x ; η ] ) ; F ( 2 ) d ( [ x ; η ] ) ] , where F ( 1 ) d ( [ x ; η ] ) , x − η∇f ( x ) , and F ( 2 ) d is a user-defined function that dictates the updates in the step size . In particular , an open-loop adaptive step size { ηk } may be achieved under this scenario , provided that it is possible to write ηk+1 = F ( 2 ) d ( ηk ) . • If we wish to use individual step sizes for each the n components of { xk } , then it suffices to take ηk as an n-dimensional vector ( thus m = 2n ) , and make appropriate changes in Fd and G. In each of these cases , GD can be seen as a forward-Euler discretization of the GF ( 1 ) , i.e. , xk+1 = xk + ∆tkFGF ( xk ) ( 8 ) with FGF = −∇f and adaptive time step ∆tk , tk+1 − tk chosen as ∆tk = ηk . Example 2 . The proximal point algorithm ( PPA ) xk+1 = arg min x∈Rn { f ( x ) + 1 2ηk ‖x− xk‖22 } ( 9 ) with step size ηk > 0 ( open loop , for simplicity ) can also be written in the form ( 6 ) , by taking m = n , Fd ( k , x ) , arg minx′∈Rn { f ( x′ ) + 12ηk ‖x ′ − x‖22 } , and G ( x ) , x . Naturally , we need to assume sufficient regularity for Fd ( k , x ) to exist and we must design a consistent way to choose Fd ( k , x ) when multiple minimizers exist in the definition of Fd ( k , x ) . Alternatively , these conditions must be satisfied , at the very least , at every ( k , x ) ∈ { ( 0 , x0 ) , ( 1 , x1 ) , ( 2 , x2 ) , . . . } for a particular chosen initial x0 ∈ Rn . By assuming sufficient regularity , we have∇x { f ( x ) + 12ηk ‖x− xk‖ 2 2 } |x=xk+1 = 0 , and thus ∇f ( xk+1 ) + 1 ηk ( xk+1 − xk ) = 0 ⇐⇒ xk+1 = xk + ∆tkFGF ( xk+1 ) ( 10 ) with ∆tk = ηk , which is precisely the backward-Euler discretization of the GF ( 1 ) . 2.2 CONTINUOUS-TIME CONVERGENCE OF q-RGF AND q-SGF In this section , we review the necessary conditions to ensure finite-time convergence of these flows . Here the hyperparameter c > 0 in equation 2 and equation 3 will not be explicitly denoted in Fq−RGF , Fq−SGF . Next , borrowing terminologies from Wilson et al . ( 2019 ) , we define the notion of gradient dominance with order p as following . Assumption 1 ( Gradient Dominance of Order p ) . For a continuously differentiable function f , we assume the function f is µ-gradient dominated of order p ∈ ( 1 , ∞ ] ( µ > 0 ) , i.e. , p− 1 p ‖∇f ( x ) ‖ p p−1 2 ≥ µ 1 p−1 ( f ( x ) − f ( x ? ) ) , ∀ x ∈ Rn , ( 11 ) where x ? ∈ arg minx∈Rn f ( x ) is the local minimizer , also we denote the optimal value f ? , f ( x ? ) . Remark 1 . It can be proved that continuously differentiable strongly convex functions are gradient dominated of order p = 2 . In fact , gradient dominance is usually defined exclusively for order p = 2 , often referred to as the Polyak-Łojasiewicz ( PL ) inequality , which was introduced by Polyak ( 1963 ) to relax the ( strong ) convexity assumption commonly used to show convergence of the GD algorithm ( 7 ) . The PL inequality can also be used to relax convexity assumptions of similar gradient and proximal-gradient methods ( Karimi et al. , 2016 ; Attouch & Bolte , 2009 ) . Furthermore , if f is gradient dominated ( of any order ) w.r.t . x ? , then x ? is an isolated stationary point of f . Our adopted generalized notion of gradient dominance is strongly tied to the Łojasiewicz gradient inequality from real analytic geometry , established by Łojasiewicz ( 1963 ; 1965 ) 1 independently and simultaneously from Polyak ( 1963 ) , and generalizing the PL inequality . More precisely , this inequality is typically written as : for some C > 0 and θ ∈ ( 1 2 , 1 ] , ‖∇f ( x ) ‖2 ≥ C · |f ( x ) − f ? |θ ( 12 ) holds for every x ∈ Rn in a small enough open neighborhood of the stationary point x = x ? . This inequality is guaranteed for analytic functions Łojasiewicz ( 1965 ) . More precisely , when x ? is a local minimizer of f , the aforementioned relationship is explicitly given by C = ( p p− 1 ) p−1 p µ 1 p , θ = p− 1 p . ( 13 ) Therefore , analytic functions are always gradient dominated . However , while analytic functions are always smooth , smoothness is not required to attain gradient dominance . Also recently it is shown that in reinforcement learning , the value functions with softmax parameterization satisfies the above condition ( Mei et al. , 2020 ; 2021 ) , which further rationalizes our settings . The following results in Romero & Benosman ( 2020 ) summarized the finite-time convergence of q-RGF ( equation 2 ) and q-SGF ( equation 3 ) in continuous-time sense , which also motivates the main topic in this paper . Theorem 1 ( Romero & Benosman ( 2020 ) ) . Suppose that f : Rn → R is continuously differentiable and µ-gradient dominated of order p ∈ ( 1 , ∞ ) near a strict local minimizer x ? ∈ Rn . Let c > 0 and q ∈ ( p , ∞ ] . Then , any maximal solution x ( · ) , in the sense of Filippov , to the q-RGF ( 2 ) or q-SGF ( 3 ) will converge in finite time to x ? , provided that ‖x ( 0 ) −x ? ‖2 > 0 is sufficiently small . More precisely , lim t→t ? x ( t ) = x ? , where the convergence time t ? < ∞ may depend on which flow is used , but in both cases is upper bounded by t ? ≤ ‖∇f ( x0 ) ‖ 1 θ− 1 θ′ 2 cC 1 θ ( 1− θθ′ ) , ( 14 ) where x0 = x ( 0 ) , C = ( p p−1 ) p−1 p µ 1 p , θ = p−1p , and θ ′ = q−1q . In particular , given any compact and positively invariant subset S ⊂ D , both flows converge in finite time with the aforementioned convergence time upper bound ( which can be tightened by replacing D with S ) for any x0 ∈ S. Furthermore , if D = Rn , then we have global finite-time convergence , i.e . finite-time convergence to any maximal solution ( in the sense of Filippov ) x ( · ) with arbitrary x0 ∈ Rn . In essence , the analysis introduced in Romero & Benosman ( 2020 ) consists of leveraging the gradient dominance to show that the energy function E ( t ) , f ( x ( t ) ) − f ? satisfies the Lyapunov-like differential inequality Ė ( t ) = O ( E ( t ) α ) for some α < 1 . The detailed proof is recalled in Appendix C for completeness . 1For more modern treatments in English , see Łojasiewicz & Zurro ( 1999 ) ; Bolte et al . ( 2007 )

Summary: This paper studies the convergence rates of the two first-order methods, named $q$-RGF and $q$-SGF. These are constructed by forward Euler discretizing the $q$-rescaled gradient flow ($q$-RGF) [Wibisono et al., 2016] and $q$-signed GF ($q$-SGF) [Romero-Benosman, 2020], respectively. These gradient flows are shown to converge in finite time in [Romero-Benosman, 2020], under the gradient dominant condition of order $q$. The authors show that their forward Euler discretized versions have linear rates, under an additional Lipschitz smoothness of order $q$. The paper also considers their stochastic variants. Numerical experiments on toy examples and practical example illustrate that the proposed method might have practical advantage over existing methods such as GD and ADAM.