Hessian-Free High-Resolution Nesterov Acceleration for Sampling

√ d/ ) iteration complexity in W2 distance , same as underdamped Langevin dynamics , but with a reduced constant . Empirical experiments are conducted to numerically verify our theoretical results . 1 INTRODUCTION . Optimization is a major machinery that drives both the theory and practice of machine learning in recent years . Since the seminal work of Nesterov ( 1983 ) , acceleration has played a key role in gradient-based optimization methods . A notable example is Nesterov ’ s Accelerated Gradient ( NAG ) , which is an instance of a more general family of “ momentum methods ” . NAG in fact consists of multiple methods , including NAG-C and NAG-SC , respectively for convex and strongly convex functions . Both of them provably converge faster than vanilla gradient descent ( GD ) in their corresponding setups ( Nesterov , 1983 ; 2013 ) . Newer perspectives of acceleration continue to be revealed , e.g. , Su et al . ( 2014 ) ; Wibisono et al . ( 2016 ) ; Wilson et al . ( 2021 ) ; Hu & Lessard ( 2017 ) ; Attouch et al . ( 2018 ) ; Shi et al . ( 2021 ) , many based on the interplay between continuous and discrete times . This work aims at turning NAG-SC into a sampler based on this interplay . In fact , approaches for sampling statistical distributions , such as gradient-based Markov Chain Monte Carlo ( MCMC ) methods , are also of great importance in machine learning , for example due to their links to statistical inference and abilities to represent uncertainties lacking in optimizationbased methods . Although not entirely the same thing , optimization and sampling are closely related : besides seeing a large class of sampling dynamics as optimization dynamics with additional noise , viewing sampling as optimization in probability space is another profound perspective that led to fruitful discoveries ( e.g. , Jordan et al . ( 1998 ) ; Liu & Wang ( 2016 ) ; Dalalyan ( 2017a ) ; Wibisono ( 2018 ) ; Zhang et al . ( 2018 ) ; Frogner & Poggio ( 2020 ) ; Chizat & Bach ( 2018 ) ; Chen et al . ( 2018a ) ; Ma et al . ( 2021 ) ; Erdogdu & Hosseinzadeh ( 2021 ) ) . In fact , an unadjusted Euler-Maruyama discretization of overdamped Langevin dynamics ( abbreviated as OLD here ) is commonly considered as the analog of GD in sampling ( although many other discretizations are also possible ) , and often referred to as Unadjusted Langevin Algorithm ( ULA ) ( Roberts et al. , 1996 ) and/or Langevin Monte Carlo ( LMC ) . The convergence properties of the continuous dynamics of OLD , as well as asymptotic and non-asymptotic analyses of its discretizations have been extensively studied ( e.g. , Roberts et al . ( 1996 ) ; Villani ( 2008 ) ; Pavliotis ( 2014 ) ; Dalalyan ( 2017b ) ; Durmus & Moulines ( 2016 ) ; Dalalyan ( 2017a ) ; Durmus et al . ( 2019a ; b ) ; Vempala & Wibisono ( 2019 ) ; Cheng & Bartlett ( 2018 ) ; Dwivedi et al . ( 2019 ) ; Ma et al . ( 2019 ) ; Chewi et al . ( 2021 ) ; Erdogdu & Hosseinzadeh ( 2021 ) ) . Meanwhile , the notion of acceleration is less quantified in sampling compared to that in optimization , although attention has been rapidly building up . Along this direction , one line is based on diffusion processes such as underdamped Langevin dynamics ( ULD ) . For example , the convergence and nonasymptotics of discretized ULD have been studied in Cheng et al . ( 2018 ) ; Dalalyan & RiouDurand ( 2020 ) ; Ma et al . ( 2021 ) , and were demonstrated provably faster than discretized OLD in suitable setups . These are not only great progresses but also forming perspectives complementary to the extensive studies of the convergence of continuous ULD in the mathematical community ( e.g , Mattingly et al . ( 2002 ) ; Cao et al . ( 2019 ) ; Dolbeault et al . ( 2009 ; 2015 ) ; Villani ( 2009 ) ; Eckmann & Hairer ( 2003 ) ; Baudoin ( 2017 ) ; Eberle et al . ( 2019 ) ) . Another important line of research is related to accelerating particle-based approaches for optimization in probability spaces ( Liu et al. , 2019 ; Taghvaei & Mehta , 2019 ; Wang & Li , 2019 ) , although we note there is no clear boundary between these two lines ( e.g. , Leimkuhler et al . ( 2018 ) ) . Additional interesting ideas also include Chen et al . ( 2018b ) ; Deng et al . ( 2020 ) . In general , it has been known that adding an irreversible part to the reversible dynamics of OLD1 accelerates its convergence ( e.g. , Hwang et al . ( 2005 ) ; Lelievre et al . ( 2013 ) ; Ohzeki & Ichiki ( 2015 ) ; Rey-Bellet & Spiliopoulos ( 2015 ) ; Duncan et al . ( 2016 ) ) , and this work can be viewed to be under this umbrella . Note , though , the discretization of an accelerated continuous process is also important , and it will be analyzed . Specifically , we propose an accelerated gradient-based MCMC algorithm termed HFHR . It is motivated by a simple question : how to appropriately inject noise to NAG algorithm in discrete time , so that it is turned into an algorithm for momentum-accelerated sampling ? Note we don ’ t add noise to the learning-rate→ 0 limit of NAG ( this has been well studied in Ma et al . ( 2021 ) ) , because a finite-step-size discretization of this limiting ODE may not converge as fast as NAG with the same learning rate . However , we will still use continuous dynamics as intermediate steps . More precisely , our first step is to combine existing tools to prepare a non-asymptotic formulation for the later steps . The goal is to better account for NAG ’ s behavior when a finite ( not infinitesimal ) learning rate is used . As pointed out in Shi et al . ( 2021 ) , a low-resolution limiting ODE ( Su et al. , 2014 ) , albeit being a milestone leading to a new venue of research ( e.g , Wibisono et al . ( 2016 ) ) , does not fully capture the acceleration enabled by NAG — for example , it can ’ t distinguish between NAG and another momentum method of heavy ball ( Polyak , 1964 ) . The main reason is , the lowresolution ODE describes the h → 0 limit of NAG , but in practice NAG uses a finite ( nonzero ) h. High-resolution ODE was thus proposed to include additionalO ( h ) terms to account for the finite h effect ( Shi et al. , 2021 ) . The original form of high-resolution ODE involves Hessian of the objective function , which is computationally expensive to evaluate and store for high-dimensional problems , but this difficulty can be overcome using techniques introduced in , e.g. , Alvarez et al. , 2002 ; Attouch et al. , 2020 , which allows us to derive a High-Resolution and Hessian-Free limiting ODE for NAG . Then we replace the high-resolution term ’ s coefficient in the HFHR ODE by a hyperparameter α ≥ 0 , and then add noise to the resulting ODE in a specific way , which turns it into an SDE suitable for the sampling purpose . This SDE will be termed as HFHR dynamics . To obtain an actual algorithm , the HFHR SDE is then discretized . We will see , both theoretically and empirically , that nonzero α can lead to accelerated convergence of the sampling algorithm ; this acceleration is not an artificial consequence of time-rescaling , which would not give acceleration after discretization with an appropriate step size . Meanwhile , note our discretization is just one of the many possible schemes . It was known that high-order discretizations can improve statistical accuracy and even the speed of convergence ( see e.g. , Chen et al . ( 2015 ) ; Li et al . ( 2019 ) ; Shen & Lee ( 2019 ) ) , although such improvements often come with more computations per iteration . The discretization considered here is just a simple first-order scheme that uses one ( full- ) gradient evaluation per step , but it better utilizes the structure of HFHR dynamics than Euler-Maruyama . Our presentation will be structured as follows . After detailing the construction of HFHR , we will analyze its convergence , at both the continuous level ( HFHR dynamics ) and the discrete level ( HFHR algorithm ) . For precise theoretical results , we will consider the setup of log-strongly-concave target distributions , which are commonly considered in the literature ( Kim et al. , 2016 ; Bubeck et al. , 2018 ; Dalalyan , 2017b ; Dalalyan & Riou-Durand , 2020 ; Dwivedi et al. , 2019 ; Shen & Lee , 2019 ) . The additional acceleration of HFHR when compared to ULD in continuous time will be demonstrated explicitly in Thm.5.1 . For our discretized HFHR algorithm , a non-asymptotic error bound will 1For irreversible-acceleration not from OLD , see e.g. , Bierkens et al . ( 2019 ) ; Bouchard-Côté et al . ( 2018 ) . be obtained ( Thm.5.2 ) , which confirms that the additional acceleration in continuous time carries through to the discrete territory . Finally , numerical experiments are provided , verifying the validity and tightness of our theoretical results , and empirically showing HFHR remains advantageous for the nonconvex and high-dimensional example of Bayesian Neural Networks . The main contribution of this article is the idea of turning NAG-SC optimization algorithm into a sampler , which also introduces a new dynamics that is neither overdamped or underdamped Langevin . Nevertheless , theoretical analyses ( e.g. , Thm.5.2 , Cor.5.4 & Rmk.5.5 ) and numerical experiments ( Sec.6 ) are also provided to quantify the effectiveness of this idea . 2 BACKGROUND : LANGEVIN DYNAMICS FOR SAMPLING . Consider sampling from Gibbs measure µ whose density is dµ = 1∫ e−f ( y ) dy e−f ( x ) dx , where f : Rd 7→ R will be called the potential function . Two diffusion processes popular for sampling ( and modeling important physical processes too ) are named after Langevin . One is overdamped Langevin dynamics ( OLD ) , and the other is kinetic Langevin dynamics ( abbreviated as ULD to comply with a convention of calling it underdamped Langevin ) . They are respectively given by ( OLD ) dqt = −∇f ( qt ) dt+ √ 2dW t ( ULD ) { dqt = ptdt dpt = −γptdt−∇f ( qt ) dt+ √ 2γdBt where qt , pt ∈ Rd , W t , Bt are i.i.d . Wiener processes in Rd , and γ > 0 is a friction coefficient . Under mild conditions ( e.g. , Pavliotis ( 2014 ) ) , OLD converges to µ and ULD converges to dπ ( q , p ) = dµ ( q ) ν ( p ) dp , where ν ( p ) = ( 2π ) − d 2 e−‖p‖ 2/2 , so its q marginal follows µ . OLD and ULD are closely related . In fact , OLD is the γ → ∞ overdamping limit of ULD after time dilation ( e.g. , Pavliotis ( 2014 ) ) . However , OLD is a reversible Markov process but ULD is irreversible , and thus both their equilibrium and non-equilibrium statistical mechanics are different , although closely related too . We will only focus on the convergence to statistical equilibrium ( see e.g. , Souza & Tao ( 2019 ) for non-equilibrium aspects ) . Many celebrated approaches exist for establishing the exponential convergence ( a.k.a . geometric ergodicity ) of OLD , including the seminal work of Roberts et al . ( 1996 ) , the ones using spectral gap ( e.g. , Dalalyan , 2017b , Lemma 1 ) , synchronous coupling ( Villani , 2008 , p33-35 ) ( Durmus et al. , 2019b , Proposition 1 ) , functional inequalities such as Poincaré ’ s inequality ( Pavliotis , 2014 , Theorem 4.4 ) and log Sobolev inequality ( Vempala & Wibisono , 2019 , Theorem 1 ) . There are also fruitful results for ULD , including the ones leveraging Lyapunov function ( Mattingly et al. , 2002 , Theorem 3.2 ) , hypocoercivity ( Villani , 2009 ; Dolbeault et al. , 2009 ; 2015 ; Roussel & Stoltz , 2018 ) , coupling ( Cheng et al. , 2018 , Theorem 5 ) ( Dalalyan & Riou-Durand , 2020 , Theorem 1 ) ( Eberle et al. , 2019 , Theorem 2.3 ) , LSI ( Ma et al. , 2021 , Section 3.1 ) , modified Poincaré ’ s inequality ( Cao et al. , 2019 , Theorem 1 ) , and spectral analysis ( Kozlov , 1989 ; Eckmann & Hairer , 2003 ) . The study of asymptotic convergence of discretized OLD dates back to at least the 1990s ( Meyn et al. , 1994 ; Roberts et al. , 1996 ) . The non-asymptotic analysis of LMC discretization of OLD can be found in Dalalyan ( 2017b ) and it shows the discretization achieves error , in TV distance , in Õ ( d/ 2 ) steps . Subsequent results include Õ ( d/ 2 ) in W2 ( Durmus & Moulines , 2016 ) , Õ ( d/ ) in KL ( Cheng & Bartlett , 2018 ) , Õ ( d/ ) in W2 under additional 3rd-order regularity ( Durmus et al. , 2019b ) , and Õ ( √ d/ ) in W2 under additional 3rd-order regularity ( Li et al. , 2021 ) . For discretized ULD , one has Õ ( √ d/ ) iteration complexity in W2 ( Cheng et al. , 2018 ; Dalalyan & Riou-Durand , 2020 ) and Õ ( √ d/ √ ) in KL ( Ma et al. , 2021 ) . ULD is still generally conceived to be advantageous over OLD and sometimes understood as its momentum-accelerated version .

Summary: This paper introduces a stochastic process called HFHR. The SDEs of HFHR is derived by rewriting Nesterov’s Accelerated Gradient (NAG) into phase-space representation, formulating it as ODEs, and then injecting noise into both position and momentum variables. HFHR can be used for sampling because it has stationary distribution as the target distribution. A discretization of HFHR is given by operator splitting and Euler integration. A mixing time bound of $\widetilde{O}(\sqrt{d}/\varepsilon)$ is obtained for sampling log-strong-concave-and-smooth target distribution with extra third-order growth condition.