Hessian-Free High-Resolution Nesterov Acceleration for Sampling

√ d/ ) iteration complexity in W2 distance , same as underdamped Langevin dynamics , but with a reduced constant . Empirical experiments are conducted to numerically verify our theoretical results . 1 INTRODUCTION . Optimization is a major machinery that drives both the theory and practice of machine learning in recent years . Since the seminal work of Nesterov ( 1983 ) , acceleration has played a key role in gradient-based optimization methods . A notable example is Nesterov ’ s Accelerated Gradient ( NAG ) , which is an instance of a more general family of “ momentum methods ” . NAG in fact consists of multiple methods , including NAG-C and NAG-SC , respectively for convex and strongly convex functions . Both of them provably converge faster than vanilla gradient descent ( GD ) in their corresponding setups ( Nesterov , 1983 ; 2013 ) . Newer perspectives of acceleration continue to be revealed , e.g. , Su et al . ( 2014 ) ; Wibisono et al . ( 2016 ) ; Wilson et al . ( 2021 ) ; Hu & Lessard ( 2017 ) ; Attouch et al . ( 2018 ) ; Shi et al . ( 2021 ) , many based on the interplay between continuous and discrete times . This work aims at turning NAG-SC into a sampler based on this interplay . In fact , approaches for sampling statistical distributions , such as gradient-based Markov Chain Monte Carlo ( MCMC ) methods , are also of great importance in machine learning , for example due to their links to statistical inference and abilities to represent uncertainties lacking in optimizationbased methods . Although not entirely the same thing , optimization and sampling are closely related : besides seeing a large class of sampling dynamics as optimization dynamics with additional noise , viewing sampling as optimization in probability space is another profound perspective that led to fruitful discoveries ( e.g. , Jordan et al . ( 1998 ) ; Liu & Wang ( 2016 ) ; Dalalyan ( 2017a ) ; Wibisono ( 2018 ) ; Zhang et al . ( 2018 ) ; Frogner & Poggio ( 2020 ) ; Chizat & Bach ( 2018 ) ; Chen et al . ( 2018a ) ; Ma et al . ( 2021 ) ; Erdogdu & Hosseinzadeh ( 2021 ) ) . In fact , an unadjusted Euler-Maruyama discretization of overdamped Langevin dynamics ( abbreviated as OLD here ) is commonly considered as the analog of GD in sampling ( although many other discretizations are also possible ) , and often referred to as Unadjusted Langevin Algorithm ( ULA ) ( Roberts et al. , 1996 ) and/or Langevin Monte Carlo ( LMC ) . The convergence properties of the continuous dynamics of OLD , as well as asymptotic and non-asymptotic analyses of its discretizations have been extensively studied ( e.g. , Roberts et al . ( 1996 ) ; Villani ( 2008 ) ; Pavliotis ( 2014 ) ; Dalalyan ( 2017b ) ; Durmus & Moulines ( 2016 ) ; Dalalyan ( 2017a ) ; Durmus et al . ( 2019a ; b ) ; Vempala & Wibisono ( 2019 ) ; Cheng & Bartlett ( 2018 ) ; Dwivedi et al . ( 2019 ) ; Ma et al . ( 2019 ) ; Chewi et al . ( 2021 ) ; Erdogdu & Hosseinzadeh ( 2021 ) ) . Meanwhile , the notion of acceleration is less quantified in sampling compared to that in optimization , although attention has been rapidly building up . Along this direction , one line is based on diffusion processes such as underdamped Langevin dynamics ( ULD ) . For example , the convergence and nonasymptotics of discretized ULD have been studied in Cheng et al . ( 2018 ) ; Dalalyan & RiouDurand ( 2020 ) ; Ma et al . ( 2021 ) , and were demonstrated provably faster than discretized OLD in suitable setups . These are not only great progresses but also forming perspectives complementary to the extensive studies of the convergence of continuous ULD in the mathematical community ( e.g , Mattingly et al . ( 2002 ) ; Cao et al . ( 2019 ) ; Dolbeault et al . ( 2009 ; 2015 ) ; Villani ( 2009 ) ; Eckmann & Hairer ( 2003 ) ; Baudoin ( 2017 ) ; Eberle et al . ( 2019 ) ) . Another important line of research is related to accelerating particle-based approaches for optimization in probability spaces ( Liu et al. , 2019 ; Taghvaei & Mehta , 2019 ; Wang & Li , 2019 ) , although we note there is no clear boundary between these two lines ( e.g. , Leimkuhler et al . ( 2018 ) ) . Additional interesting ideas also include Chen et al . ( 2018b ) ; Deng et al . ( 2020 ) . In general , it has been known that adding an irreversible part to the reversible dynamics of OLD1 accelerates its convergence ( e.g. , Hwang et al . ( 2005 ) ; Lelievre et al . ( 2013 ) ; Ohzeki & Ichiki ( 2015 ) ; Rey-Bellet & Spiliopoulos ( 2015 ) ; Duncan et al . ( 2016 ) ) , and this work can be viewed to be under this umbrella . Note , though , the discretization of an accelerated continuous process is also important , and it will be analyzed . Specifically , we propose an accelerated gradient-based MCMC algorithm termed HFHR . It is motivated by a simple question : how to appropriately inject noise to NAG algorithm in discrete time , so that it is turned into an algorithm for momentum-accelerated sampling ? Note we don ’ t add noise to the learning-rate→ 0 limit of NAG ( this has been well studied in Ma et al . ( 2021 ) ) , because a finite-step-size discretization of this limiting ODE may not converge as fast as NAG with the same learning rate . However , we will still use continuous dynamics as intermediate steps . More precisely , our first step is to combine existing tools to prepare a non-asymptotic formulation for the later steps . The goal is to better account for NAG ’ s behavior when a finite ( not infinitesimal ) learning rate is used . As pointed out in Shi et al . ( 2021 ) , a low-resolution limiting ODE ( Su et al. , 2014 ) , albeit being a milestone leading to a new venue of research ( e.g , Wibisono et al . ( 2016 ) ) , does not fully capture the acceleration enabled by NAG — for example , it can ’ t distinguish between NAG and another momentum method of heavy ball ( Polyak , 1964 ) . The main reason is , the lowresolution ODE describes the h → 0 limit of NAG , but in practice NAG uses a finite ( nonzero ) h. High-resolution ODE was thus proposed to include additionalO ( h ) terms to account for the finite h effect ( Shi et al. , 2021 ) . The original form of high-resolution ODE involves Hessian of the objective function , which is computationally expensive to evaluate and store for high-dimensional problems , but this difficulty can be overcome using techniques introduced in , e.g. , Alvarez et al. , 2002 ; Attouch et al. , 2020 , which allows us to derive a High-Resolution and Hessian-Free limiting ODE for NAG . Then we replace the high-resolution term ’ s coefficient in the HFHR ODE by a hyperparameter α ≥ 0 , and then add noise to the resulting ODE in a specific way , which turns it into an SDE suitable for the sampling purpose . This SDE will be termed as HFHR dynamics . To obtain an actual algorithm , the HFHR SDE is then discretized . We will see , both theoretically and empirically , that nonzero α can lead to accelerated convergence of the sampling algorithm ; this acceleration is not an artificial consequence of time-rescaling , which would not give acceleration after discretization with an appropriate step size . Meanwhile , note our discretization is just one of the many possible schemes . It was known that high-order discretizations can improve statistical accuracy and even the speed of convergence ( see e.g. , Chen et al . ( 2015 ) ; Li et al . ( 2019 ) ; Shen & Lee ( 2019 ) ) , although such improvements often come with more computations per iteration . The discretization considered here is just a simple first-order scheme that uses one ( full- ) gradient evaluation per step , but it better utilizes the structure of HFHR dynamics than Euler-Maruyama . Our presentation will be structured as follows . After detailing the construction of HFHR , we will analyze its convergence , at both the continuous level ( HFHR dynamics ) and the discrete level ( HFHR algorithm ) . For precise theoretical results , we will consider the setup of log-strongly-concave target distributions , which are commonly considered in the literature ( Kim et al. , 2016 ; Bubeck et al. , 2018 ; Dalalyan , 2017b ; Dalalyan & Riou-Durand , 2020 ; Dwivedi et al. , 2019 ; Shen & Lee , 2019 ) . The additional acceleration of HFHR when compared to ULD in continuous time will be demonstrated explicitly in Thm.5.1 . For our discretized HFHR algorithm , a non-asymptotic error bound will 1For irreversible-acceleration not from OLD , see e.g. , Bierkens et al . ( 2019 ) ; Bouchard-Côté et al . ( 2018 ) . be obtained ( Thm.5.2 ) , which confirms that the additional acceleration in continuous time carries through to the discrete territory . Finally , numerical experiments are provided , verifying the validity and tightness of our theoretical results , and empirically showing HFHR remains advantageous for the nonconvex and high-dimensional example of Bayesian Neural Networks . The main contribution of this article is the idea of turning NAG-SC optimization algorithm into a sampler , which also introduces a new dynamics that is neither overdamped or underdamped Langevin . Nevertheless , theoretical analyses ( e.g. , Thm.5.2 , Cor.5.4 & Rmk.5.5 ) and numerical experiments ( Sec.6 ) are also provided to quantify the effectiveness of this idea . 2 BACKGROUND : LANGEVIN DYNAMICS FOR SAMPLING . Consider sampling from Gibbs measure µ whose density is dµ = 1∫ e−f ( y ) dy e−f ( x ) dx , where f : Rd 7→ R will be called the potential function . Two diffusion processes popular for sampling ( and modeling important physical processes too ) are named after Langevin . One is overdamped Langevin dynamics ( OLD ) , and the other is kinetic Langevin dynamics ( abbreviated as ULD to comply with a convention of calling it underdamped Langevin ) . They are respectively given by ( OLD ) dqt = −∇f ( qt ) dt+ √ 2dW t ( ULD ) { dqt = ptdt dpt = −γptdt−∇f ( qt ) dt+ √ 2γdBt where qt , pt ∈ Rd , W t , Bt are i.i.d . Wiener processes in Rd , and γ > 0 is a friction coefficient . Under mild conditions ( e.g. , Pavliotis ( 2014 ) ) , OLD converges to µ and ULD converges to dπ ( q , p ) = dµ ( q ) ν ( p ) dp , where ν ( p ) = ( 2π ) − d 2 e−‖p‖ 2/2 , so its q marginal follows µ . OLD and ULD are closely related . In fact , OLD is the γ → ∞ overdamping limit of ULD after time dilation ( e.g. , Pavliotis ( 2014 ) ) . However , OLD is a reversible Markov process but ULD is irreversible , and thus both their equilibrium and non-equilibrium statistical mechanics are different , although closely related too . We will only focus on the convergence to statistical equilibrium ( see e.g. , Souza & Tao ( 2019 ) for non-equilibrium aspects ) . Many celebrated approaches exist for establishing the exponential convergence ( a.k.a . geometric ergodicity ) of OLD , including the seminal work of Roberts et al . ( 1996 ) , the ones using spectral gap ( e.g. , Dalalyan , 2017b , Lemma 1 ) , synchronous coupling ( Villani , 2008 , p33-35 ) ( Durmus et al. , 2019b , Proposition 1 ) , functional inequalities such as Poincaré ’ s inequality ( Pavliotis , 2014 , Theorem 4.4 ) and log Sobolev inequality ( Vempala & Wibisono , 2019 , Theorem 1 ) . There are also fruitful results for ULD , including the ones leveraging Lyapunov function ( Mattingly et al. , 2002 , Theorem 3.2 ) , hypocoercivity ( Villani , 2009 ; Dolbeault et al. , 2009 ; 2015 ; Roussel & Stoltz , 2018 ) , coupling ( Cheng et al. , 2018 , Theorem 5 ) ( Dalalyan & Riou-Durand , 2020 , Theorem 1 ) ( Eberle et al. , 2019 , Theorem 2.3 ) , LSI ( Ma et al. , 2021 , Section 3.1 ) , modified Poincaré ’ s inequality ( Cao et al. , 2019 , Theorem 1 ) , and spectral analysis ( Kozlov , 1989 ; Eckmann & Hairer , 2003 ) . The study of asymptotic convergence of discretized OLD dates back to at least the 1990s ( Meyn et al. , 1994 ; Roberts et al. , 1996 ) . The non-asymptotic analysis of LMC discretization of OLD can be found in Dalalyan ( 2017b ) and it shows the discretization achieves error , in TV distance , in Õ ( d/ 2 ) steps . Subsequent results include Õ ( d/ 2 ) in W2 ( Durmus & Moulines , 2016 ) , Õ ( d/ ) in KL ( Cheng & Bartlett , 2018 ) , Õ ( d/ ) in W2 under additional 3rd-order regularity ( Durmus et al. , 2019b ) , and Õ ( √ d/ ) in W2 under additional 3rd-order regularity ( Li et al. , 2021 ) . For discretized ULD , one has Õ ( √ d/ ) iteration complexity in W2 ( Cheng et al. , 2018 ; Dalalyan & Riou-Durand , 2020 ) and Õ ( √ d/ √ ) in KL ( Ma et al. , 2021 ) . ULD is still generally conceived to be advantageous over OLD and sometimes understood as its momentum-accelerated version .

Summary: The author proposed an accelerated-gradient-based MCMC method based on Nesterov's accelerated gradient (NAG). In continuous time, the algorithm is able to achieve a tremendous acceleration over the underdamped Langevin algorithm. Numerical schemes can propose a speed-up with a constant factor.