First-Order Optimization Inspired from Finite-Time Convergent Flows

1 INTRODUCTION . Consider the unconstrained minimization problem for a given cost function f : Rn → R. When f is sufficiently regular , the standard algorithm in continuous time ( dynamical system ) is given by ẋ = FGF ( x ) , −∇f ( x ) ( 1 ) with ẋ , ddtx ( t ) , known as the gradient flow ( GF ) . Generalizing GF , the q-rescaled GF ( qRGF ) Wibisono et al . ( 2016 ) given by ẋ = Fq−RGF ( x ) = −c ∇f ( x ) ‖∇f ( x ) ‖ q−2 q−1 2 ( 2 ) with c > 0 and q ∈ ( 1 , ∞ ] has an asymptotic convergence rate f ( x ( t ) ) − f ( x ? ) = O ( 1 tq−1 ) under mild regularity , for ‖x ( 0 ) − x ? ‖ > 0 small enough , where x ? ∈ Rn denotes a local minimizer of f . However , it has been recently proven Romero & Benosman ( 2020 ) that q-RGF , as well as q-signed GF ( q-SGF ) ẋ = Fq−SGF ( x ) = −c ‖∇f ( x ) ‖ 1 q−1 1 sign ( ∇f ( x ) ) , ( 3 ) where sign ( · ) denotes the sign function ( element-wise ) , are both finite-time convergent ( in continuoustime ) , provided that f is gradient dominated of order p ∈ ( 1 , q ) . In particular , if f is strongly convex , then q-RGF and q-SGF is finite-time convergent for any q ∈ ( 2 , ∞ ] , since f must be gradient dominated of order p = 2 . Considering that many algorithms are inspired from continuous flows with convergence guarantee , e.g. , Muehlebach & Jordan ( 2019 ) ; Fazlyab et al . ( 2017a ) ; Shi et al . ( 2018 ) ; Zhang et al . ( 2018 ) ; França et al . ( 2019b ) ; Wibisono et al . ( 2016 ) , a natural question arises : what is the convergence rate of the corresponding discrete-time algorithms , which are induced by discretization of finite-time convergent continuous-time flows ? 1.1 CONTRIBUTION . In this paper , we investigate the convergence behavior of an Euler discretization for the q-RGF ( equation 2 ) and q-SGF ( equation 3 ) . We provide convergence guarantees in terms of closeness of solutions , using results from hybrid dynamical control theory . Furthermore , we provide iteration / sample complexity upper bounds for both the general ( deterministic ) and stochastic settings . We then test the performance of the proposed algorithms on both synthetic and real-world data in the context of deep learning , namely , on the well-known SVHN dataset . 1.2 RELATED WORK . Propelled by the work of Wang & Elia ( 2011 ) and Su et al . ( 2014 ) , there has been a recent and significant research effort dedicated to analyzing optimization algorithms from the perspective of dynamical systems and control theory , especially in continuous time ( Wibisono et al. , 2016 ; Wilson , 2018 ; Lessard et al. , 2016 ; Fazlyab et al. , 2017b ; Scieur et al. , 2017 ; Franca et al. , 2018 ; Fazlyab et al. , 2018 ; Fazlyab et al. , 2018 ; Taylor et al. , 2018 ; França et al. , 2019a ; Orvieto & Lucchi , 2019 ; Romero et al. , 2019 ; Muehlebach & Jordan , 2019 ) . A major focus within this initiative is in accceleration , both in terms of trying to gain new insight into more traditional optimization algorithms from this perspective , or even to exploit the interplay between continuous-time systems and their potential discretizations for novel algorithm design ( Muehlebach & Jordan , 2019 ; Fazlyab et al. , 2017a ; Shi et al. , 2018 ; Zhang et al. , 2018 ; França et al. , 2019b ; Wilson et al. , 2019 ) . Many of these papers also focus on deriving convergence rates based on the discretization of flows designed in the continuous-time domain . Connecting ordinary differential equations ( ODEs ) and their numerical analysis , with optimization algorithms is a very important topic , which can be dated back to 1970s , see Botsaris ( 1978a ; b ) ; Zghier ( 1981 ) ; Snyman ( 1982 ; 1983 ) ; Brockett ( 1988 ) ; Brown ( 1989 ) . In Helmke & Moore ( 1994 ) , the authors studied relationships between linear programming , ODEs , and general matrix theory . Further , Schropp ( 1995 ) and Schropp & Singer ( 2000 ) explored several aspects linking nonlinear dynamical systems to gradient-based optimization , including nonlinear constraints . Tools from Lyapunov stability theory are often employed for the analysis , mainly because there already exists a rich body of works within the nonlinear systems and control theory community for this purpose . In particular , typically in previous works , one seeks asymptotically Lyapunov stable gradient-based systems with an equilibrium ( stationary point ) at an isolated extremum of the given cost function , thus certifying local convergence . Naturally , the global asymptotic stability leads to global convergence , though such analysis will typically require the cost function to be strongly convex everywhere . For physical systems , a Lyapunov function can often be constructed from first principles via some physically meaningful measure of energy ( e.g. , total energy = potential energy + kinetic energy ) . In optimization , the situation is somewhat similar in the sense that a suitable Lyapunov function may often be constructed by taking simple surrogates of the objective function as candidates . For instance , V ( x ) , f ( x ) − f ( x ? ) can be a good initial candidate . Further , if f is continuously differentiable and x ? is an isolated stationary point , then another alternative is V ( x ) , ‖∇f ( x ) ‖2 . However , most fundamental and applied research conducted in systems and control regarding Lyapunov stability theory deals exclusively with continuous-time systems . Unfortunately , ( dynamical ) stability properties are generally not preserved for simple forward-Euler and sample-and-hold discretizations and control laws ( Stuart & Humphries , 1998 ) . Furthermore , practical implementations of optimization algorithms in modern digital computers demand discrete-time . Nonetheless , it has been extensively noted that a vast amount of general Lyapunov-based results ( perhaps most ) appear to have a discrete-time equivalent . 2 PRELIMINARIES . 2.1 OPTIMIZATION ALGORITHMS AS DISCRETE-TIME SYSTEMS . Generalizing ( 1 ) , ( 2 ) , and ( 3 ) , consider a continuous-time algorithm ( dynamical system ) modeled via an ordinary differential equation ( ODE ) ẋ = F ( x ) ( 4 ) for t ≥ 0 , or , more generally , a differential inclusion ẋ ( t ) ∈ F ( x ( t ) ) ( 5 ) a.e . t ≥ 0 ( e.g . for the q = ∞ case ) , such that x ( t ) → x ? as t → t ? . In the case of the q-RGF ( 2 ) and q-SGF ( 3 ) for f gradient dominated of order p ∈ ( 1 , q ) , we have finite-time convergence , and thus t ? = t ? ( x ( 0 ) ) < ∞ . Most of the popular numerical optimization schemes can be written in a state-space form ( i.e. , recursively ) , as Xk+1 = Fd ( k , Xk ) ( 6a ) xk = G ( Xk ) ( 6b ) for k ∈ Z+ , { 0 , 1 , 2 , . . . } and a given X0 ∈ Rm ( typically m ≥ n ) , where Fd : Z+ × Rm → Rm and G : Rm → Rn . Naturally , ( 6 ) can be seen as a discrete-time dynamical system constructed by discretizing ( 4 ) in time . In particular , we have xk ≈ x ( tk ) , where { 0 = t0 < t1 < t2 < . . . } denotes a time partition and x ( · ) a solution to ( 4 ) or ( 5 ) as appropriate . Therefore , we call Xk and xk , respectively , the state and output at time step k. Example 1 . The standard gradient descent ( GD ) algorithm xk+1 = xk − η∇f ( xk ) ( 7 ) with step size ( learning rate ) η > 0 can be readily written in the form ( 6 ) by taking m = n , Fd ( x ) , x− η∇f ( x ) , and G ( x ) , x . • If the step sizes are adaptive , i.e . if we replace η by a sequence { ηk } with ηk > 0 , then we only need to replace Fd ( k , x ) , x− ηk∇f ( x ) , provided that { ηk } is not computed using feedback from { xk } ( e.g . through a line search method ) . • If we do wish to use time-varying step-size , then we can set m = n + 1 , G ( [ x ; η ] ) , x , and Fd ( [ x ; η ] ) , [ F ( 1 ) d ( [ x ; η ] ) ; F ( 2 ) d ( [ x ; η ] ) ] , where F ( 1 ) d ( [ x ; η ] ) , x − η∇f ( x ) , and F ( 2 ) d is a user-defined function that dictates the updates in the step size . In particular , an open-loop adaptive step size { ηk } may be achieved under this scenario , provided that it is possible to write ηk+1 = F ( 2 ) d ( ηk ) . • If we wish to use individual step sizes for each the n components of { xk } , then it suffices to take ηk as an n-dimensional vector ( thus m = 2n ) , and make appropriate changes in Fd and G. In each of these cases , GD can be seen as a forward-Euler discretization of the GF ( 1 ) , i.e. , xk+1 = xk + ∆tkFGF ( xk ) ( 8 ) with FGF = −∇f and adaptive time step ∆tk , tk+1 − tk chosen as ∆tk = ηk . Example 2 . The proximal point algorithm ( PPA ) xk+1 = arg min x∈Rn { f ( x ) + 1 2ηk ‖x− xk‖22 } ( 9 ) with step size ηk > 0 ( open loop , for simplicity ) can also be written in the form ( 6 ) , by taking m = n , Fd ( k , x ) , arg minx′∈Rn { f ( x′ ) + 12ηk ‖x ′ − x‖22 } , and G ( x ) , x . Naturally , we need to assume sufficient regularity for Fd ( k , x ) to exist and we must design a consistent way to choose Fd ( k , x ) when multiple minimizers exist in the definition of Fd ( k , x ) . Alternatively , these conditions must be satisfied , at the very least , at every ( k , x ) ∈ { ( 0 , x0 ) , ( 1 , x1 ) , ( 2 , x2 ) , . . . } for a particular chosen initial x0 ∈ Rn . By assuming sufficient regularity , we have∇x { f ( x ) + 12ηk ‖x− xk‖ 2 2 } |x=xk+1 = 0 , and thus ∇f ( xk+1 ) + 1 ηk ( xk+1 − xk ) = 0 ⇐⇒ xk+1 = xk + ∆tkFGF ( xk+1 ) ( 10 ) with ∆tk = ηk , which is precisely the backward-Euler discretization of the GF ( 1 ) . 2.2 CONTINUOUS-TIME CONVERGENCE OF q-RGF AND q-SGF In this section , we review the necessary conditions to ensure finite-time convergence of these flows . Here the hyperparameter c > 0 in equation 2 and equation 3 will not be explicitly denoted in Fq−RGF , Fq−SGF . Next , borrowing terminologies from Wilson et al . ( 2019 ) , we define the notion of gradient dominance with order p as following . Assumption 1 ( Gradient Dominance of Order p ) . For a continuously differentiable function f , we assume the function f is µ-gradient dominated of order p ∈ ( 1 , ∞ ] ( µ > 0 ) , i.e. , p− 1 p ‖∇f ( x ) ‖ p p−1 2 ≥ µ 1 p−1 ( f ( x ) − f ( x ? ) ) , ∀ x ∈ Rn , ( 11 ) where x ? ∈ arg minx∈Rn f ( x ) is the local minimizer , also we denote the optimal value f ? , f ( x ? ) . Remark 1 . It can be proved that continuously differentiable strongly convex functions are gradient dominated of order p = 2 . In fact , gradient dominance is usually defined exclusively for order p = 2 , often referred to as the Polyak-Łojasiewicz ( PL ) inequality , which was introduced by Polyak ( 1963 ) to relax the ( strong ) convexity assumption commonly used to show convergence of the GD algorithm ( 7 ) . The PL inequality can also be used to relax convexity assumptions of similar gradient and proximal-gradient methods ( Karimi et al. , 2016 ; Attouch & Bolte , 2009 ) . Furthermore , if f is gradient dominated ( of any order ) w.r.t . x ? , then x ? is an isolated stationary point of f . Our adopted generalized notion of gradient dominance is strongly tied to the Łojasiewicz gradient inequality from real analytic geometry , established by Łojasiewicz ( 1963 ; 1965 ) 1 independently and simultaneously from Polyak ( 1963 ) , and generalizing the PL inequality . More precisely , this inequality is typically written as : for some C > 0 and θ ∈ ( 1 2 , 1 ] , ‖∇f ( x ) ‖2 ≥ C · |f ( x ) − f ? |θ ( 12 ) holds for every x ∈ Rn in a small enough open neighborhood of the stationary point x = x ? . This inequality is guaranteed for analytic functions Łojasiewicz ( 1965 ) . More precisely , when x ? is a local minimizer of f , the aforementioned relationship is explicitly given by C = ( p p− 1 ) p−1 p µ 1 p , θ = p− 1 p . ( 13 ) Therefore , analytic functions are always gradient dominated . However , while analytic functions are always smooth , smoothness is not required to attain gradient dominance . Also recently it is shown that in reinforcement learning , the value functions with softmax parameterization satisfies the above condition ( Mei et al. , 2020 ; 2021 ) , which further rationalizes our settings . The following results in Romero & Benosman ( 2020 ) summarized the finite-time convergence of q-RGF ( equation 2 ) and q-SGF ( equation 3 ) in continuous-time sense , which also motivates the main topic in this paper . Theorem 1 ( Romero & Benosman ( 2020 ) ) . Suppose that f : Rn → R is continuously differentiable and µ-gradient dominated of order p ∈ ( 1 , ∞ ) near a strict local minimizer x ? ∈ Rn . Let c > 0 and q ∈ ( p , ∞ ] . Then , any maximal solution x ( · ) , in the sense of Filippov , to the q-RGF ( 2 ) or q-SGF ( 3 ) will converge in finite time to x ? , provided that ‖x ( 0 ) −x ? ‖2 > 0 is sufficiently small . More precisely , lim t→t ? x ( t ) = x ? , where the convergence time t ? < ∞ may depend on which flow is used , but in both cases is upper bounded by t ? ≤ ‖∇f ( x0 ) ‖ 1 θ− 1 θ′ 2 cC 1 θ ( 1− θθ′ ) , ( 14 ) where x0 = x ( 0 ) , C = ( p p−1 ) p−1 p µ 1 p , θ = p−1p , and θ ′ = q−1q . In particular , given any compact and positively invariant subset S ⊂ D , both flows converge in finite time with the aforementioned convergence time upper bound ( which can be tightened by replacing D with S ) for any x0 ∈ S. Furthermore , if D = Rn , then we have global finite-time convergence , i.e . finite-time convergence to any maximal solution ( in the sense of Filippov ) x ( · ) with arbitrary x0 ∈ Rn . In essence , the analysis introduced in Romero & Benosman ( 2020 ) consists of leveraging the gradient dominance to show that the energy function E ( t ) , f ( x ( t ) ) − f ? satisfies the Lyapunov-like differential inequality Ė ( t ) = O ( E ( t ) α ) for some α < 1 . The detailed proof is recalled in Appendix C for completeness . 1For more modern treatments in English , see Łojasiewicz & Zurro ( 1999 ) ; Bolte et al . ( 2007 )

Summary: This paper considers the analysis of two discrete time schemes derived from gradient flows named q-RGF and q-SGF. The topic fits more generally into continuous time perspectives for optimization and the relations between ODE theory and optimization. This is an interesting direction with many interesting promises.