H ESSIAN -F REE H IGH -R ESOLUTION N ESTEROV A CCELERATION FOR S AMPLING
Anonymous authors
Paper under double-blind review
A BSTRACT
It is known (Shi et al., 2021) that Nesterov’s Accelerated Gradient (NAG) for optimization differs from its continuous time limit (noiseless kinetic Langevin) when its stepsize becomes ﬁnite.
This work explores the sampling counterpart of this phenonemon and proposes an accelerated gradient based MCMC method based on the optimizer of
NAG for strongly convex functions
(NAG-
SC): we reformu late
NAG
SC as a
Hessian
-Free
High
-Resolution ODE, change its high-resolution coefﬁcient to a hyperparameter inject appropriate noise and discretize the resulting diffusion process.
Accelerated sampling enabled by the new hyperparameter is quantiﬁed and it is not a false acceleration created by time-rescaling.
At continuous-time level additional acceleration over underdamped Langevin in Wdistance is proved.
At discrete algorithm level a dedicated discretization is proposed to simulate the Hessian-Free High-Resolution SDE in a cost
-efﬁcient man
- ner
For log
-strong-concave-and-smooth target measures
, the proposed algorithm achieves
˜ O
√ d
/(cid:15)
) iteration complexity in
W
2 distance, same as underdamped
Langevin dynamics, but with a reduced constant.
Empirical experiments are con ducted to numerically verify our theoretical results.
I NTRODUCTION
Optimization is a major machinery that drives both the theory and practice of machine learning in recent years.
Since the seminal work of Nesterov (1983), acceleration has played a key role in gradient-based optimization methods.
A notable example is Nesterov’s Accelerated Gradient (NAG), which is an instance of a more general family of “momentum methods”.
NAG in fact consists of multiple methods, including NAG-C and NAG-SC, respectively for convex and strongly convex functions.
Both of them provably converge faster than vanilla gradient descent (GD) in their corresponding setups (Nesterov, 1983; 2013).
Newer perspectives of acceleration continue to be revealed, e.g., Su et al.
; Wibisono et al.
; Wilson et al.
; Hu & Lessard (2017);
Attouch et al.
; Shi et al.
(2021), many based on the interplay between continuous and discrete times.
This work aims at turning NAG-SC into a sampler based on this interplay.
In fact, approaches for sampling statistical distributions, such as gradient-based Markov Chain
Monte Carlo (MCMC) methods, are also of great importance in machine learning, for example due to their links to statistical inference and abilities to represent uncertainties lacking in optimizationbased methods.
Although not entirely the same thing, optimization and sampling are closely related: besides seeing a large class of sampling dynamics as optimization dynamics with additional noise, viewing sampling as optimization in probability space is another profound perspective that led to fruitful discoveries (e.g., Jordan et al.
; Liu & Wang (2016); Dalalyan (2017a); Wibisono (2018); Zhang et al.
(2018); Frogner & Poggio (2020); Chizat & Bach (2018);
Chen et al.
(2018a);
Ma et al.
(2021); Erdogdu & Hosseinzadeh (2021)).
In fact, an unadjusted Euler-Maruyama discretization of overdamped Langevin dynamics (abbreviated as OLD here) is commonly considered as the analog of GD in sampling (although many other discretizations are also possible), and often referred to as Unadjusted Langevin Algorithm (ULA) (Roberts et al., 1996) and/or Langevin Monte
Carlo (LMC).
The convergence properties of the continuous dynamics of OLD, as well as asymptotic and non-asymptotic analyses of its discretizations have been extensively studied (e.g., Roberts et al.
; Villani (2008); Pavliotis (2014); Dalalyan (2017b); Durmus & Moulines (2016); Dalalyan (2017a
Meanwhile, the notion of acceleration is less quantiﬁed in sampling compared to that in optimization, although attention has been rapidly building up.
Along this direction, one line is based on diffusion processes such as underdamped Langevin dynamics (ULD).
For example, the convergence and nonasymptotics of discretized ULD have been studied in Cheng et al.
; Dalalyan & RiouDurand (2020);
Ma et al.
and were demonstrated provably faster than discretized OLD in suitable setups.
These are not only great progresses but also forming perspectives complementary to the extensive studies of the convergence of continuous ULD in the mathematical community (e.g,
Mattingly et al.
; Cao et al.
; Dolbeault et al.
(2009; 2015); Villani (2009); Eckmann &
Hairer (2003); Baudoin (2017); Eberle et al.
Another important line of research is related to accelerating particle-based approaches for optimization in probability spaces (Liu et al., 2019;
Taghvaei & Mehta, 2019; Wang & Li, 2019), although we note there is no clear boundary between these two lines (e.g., Leimkuhler et al. Additional interesting ideas also include Chen et al. (2018b)
; Deng et al. In general, it has been known that adding an irreversible part to the reversible dynamics of OLD 1 accelerates its convergence (e.g., Hwang et al.
; Lelievre et al.
(2013); Ohzeki & Ichiki (2015); Rey-Bellet & Spiliopoulos (2015); Duncan et al.
and this work can be viewed to be under this umbrella.
Note, though, the discretization of an accelerated continuous process is also important, and it will be analyzed.
Speciﬁcally, we propose an accelerated gradient-based MCMC algorithm termed HFHR.
It is motivated by a simple question: how to appropriately inject noise to NAG algorithm in discrete time , so that it is turned into an algorithm for momentum-accelerated sampling?
Note we don’t add noise to the learning-rate → 0 limit of NAG (this has been well studied in Ma et al.
(2021)), because a ﬁnite-step-size discretization of this limiting ODE may not converge as fast as NAG with the same learning rate.
However, we will still use continuous dynamics as intermediate steps.
More precisely, our ﬁrst step is to combine existing tools to prepare a non-asymptotic formulation for the later steps.
The goal is to better account for NAG’s behavior when a ﬁnite (not inﬁnitesimal) learning rate is used.
As pointed out in Shi et al.
(2021), a low-resolution limiting ODE (Su et al., 2014), albeit being a milestone leading to a new venue of research (e.g, Wibisono et al.
(2016)), does not fully capture the acceleration enabled by NAG — for example, it can’t distinguish between
NAG and another momentum method of heavy ball (Polyak, 1964).
The main reason is, the lowresolution ODE describes the h → 0 limit of NAG, but in practice NAG uses a ﬁnite (nonzero) h .
High-resolution ODE was thus proposed to include additional O ( h ) terms to account for the ﬁnite h effect (Shi et al., 2021).
The original form of high-resolution ODE involves Hessian of the objective function, which is computationally expensive to evaluate and store for high-dimensional problems, but this difﬁculty can be overcome using techniques introduced in, e.g., Alvarez et al., 2002; Attouch et al., 2020, which allows us to derive a High-Resolution and Hessian-Free limiting ODE for NAG.
Then we replace the high-resolution term’s coefﬁcient in the HFHR ODE by a hyperparameter α ≥ 0 , and then add noise to the resulting ODE in a speciﬁc way, which turns it into an SDE suitable for the sampling purpose.
This SDE will be termed as HFHR dynamics.
To obtain an actual algorithm, the HFHR SDE is then discretized.
We will see, both theoretically and empirically, that nonzero α can lead to accelerated convergence of the sampling algorithm; this acceleration is not an artiﬁcial consequence of time-rescaling, which would not give acceleration after discretization with an appropriate step size.
Meanwhile, note our discretization is just one of the many possible schemes.
It was known that high-order discretizations can improve statistical accuracy and even the speed of convergence (see e.g., Chen et al. (2015); Li et al.
; Shen & Lee (2019)), although such improvements often come with more computations per iteration.
The discretization considered here is just a simple ﬁrst-order scheme that uses one (full-)gradient evaluation per step, but it better utilizes the structure of HFHR dynamics than Euler-Maruyama.
Our presentation will be structured as follows.
After detailing the construction of HFHR, we will analyze its convergence, at both the continuous level (HFHR dynamics) and the discrete level (HFHR algorithm).
For precise theoretical results, we will consider the setup of log-strongly-concave target distributions, which are commonly considered in the literature (Kim et al., 2016;
Bubeck et al., 2018; Dalalyan, 2017b; Dalalyan & Riou-Durand, 2020; Dwivedi et al., 2019; Shen & Lee, 2019).
The additional acceleration of HFHR when compared to ULD in continuous time will be demonstrated explicitly in Thm.5.1.
For our discretized HFHR algorithm, a non-asymptotic error bound will be obtained (Thm.5.2), which conﬁrms that the additional acceleration in continuous time carries through to the discrete territory.
Finally, numerical experiments are provided, verifying the validity and tightness of our theoretical results, and empirically showing HFHR remains advantageous for the nonconvex and high-dimensional example of Bayesian Neural Networks.
The main contribution of this article is the idea of turning NAG-SC optimization algorithm into a sampler, which also introduces a new dynamics that is neither overdamped or underdamped
Langevin.
Nevertheless, theoretical analyses (e.g., Thm.5.2, Cor.5.4 & Rmk.5.5) and numerical experiments (Sec.6) are also provided to quantify the effectiveness of this idea.
B ACKGROUND : L ANGEVIN D YNAMICS FOR S AMPLING
Consider sampling from Gibbs measure µ whose density is dµ =
[Equation]
d x , where f :
R d (cid:55)→ R will be called the potential function.
Two diffusion processes popular for sampling (and modeling important physical processes too) are named after Langevin.
One is overdamped Langevin dynamics (OLD), and the other is kinetic Langevin dynamics (abbreviated as ULD to comply with a convention of calling it underdamped Langevin).
They are respectively given by
[Equation]
where q t , p t ∈ R d , W t , B t are i.i.d.
Wiener processes in R d , and γ > 0 is a friction coefﬁcient.
Under mild conditions (e.g., Pavliotis (2014)), OLD converges to µ and ULD converges to dπ ( q , p ) =
dµ ( q ) ν
( p ) d p , where ν ( p ) =
(2 π )
[Equation]
p (cid:107) 2 / 2 , so its q marginal follows µ .
OLD and ULD are closely related.
In fact, OLD is the γ → ∞ overdamping limit of ULD after time dilation (e.g., Pavliotis (2014)).
However, OLD is a reversible Markov process but ULD is irreversible, and thus both their equilibrium and non-equilibrium statistical mechanics are different, although closely related too.
We will only focus on the convergence to statistical equilibrium (see e.g., Souza & Tao (2019) for non-equilibrium aspects).
Many celebrated approaches exist for establishing the exponential convergence (a.k.a.
geometric ergodicity) of OLD, including the seminal work of Roberts et al.
the ones using spectral gap (e.g., Dalalyan, 2017b, Lemma 1), synchronous coupling (Villani, 2008, p33-35)(Durmus et al., 2019b, Proposition 1), functional inequalities such as Poincar´e’s inequality (Pavliotis, 2014, Theorem 4.4) and log Sobolev inequality (Vempala & Wibisono, 2019, Theorem 1).
There are also fruitful results for ULD, including the ones leveraging Lyapunov function (Mattingly et al., 2002,
Theorem 3.2), hypocoercivity (Villani, 2009; Dolbeault et al., 2009; 2015; Roussel & Stoltz, 2018), coupling (Cheng et al., 2018, Theorem 5)(Dalalyan & Riou-Durand, 2020, Theorem 1)(Eberle et al., 2019, Theorem 2.3), LSI (Ma et al., 2021, Section 3.1), modiﬁed Poincar ´ e’s inequality (Cao et al., 2019, Theorem 1), and spectral analysis (Kozlov, 1989; Eckmann & Hairer, 2003).
The study of asymptotic convergence of discretized OLD dates back to at least the 1990s (Meyn et al., 1994; Roberts et al., 1996).
The non-asymptotic analysis of LMC discretization of OLD can be found in Dalalyan (2017b) and it shows the discretization achieves (cid:15) error, in TV distance, in ˜
O ( d / (cid:15) 2 ) steps.
Subsequent results include ˜ O ( d / (cid:15) 2 ) in W 2 (Durmus & Moulines, 2016), ˜ O
( d / (cid:15) ) in
KL (Cheng & Bartlett, 2018), ˜ O
( d / (cid:15) ) in W 2 under additional 3rd-order regularity (Durmus et al., 2019b), and ˜ O ( √ d / (cid:15) ) in W 2 under additional 3rd-order regularity (Li et al., 2021).
For discretized
ULD, one has ˜ O ( √ d / (cid:15) ) iteration complexity in W 2 (Cheng et al., 2018; Dalalyan & Riou-Durand, 2020) and ˜ O ( √ d / √ (cid:15) ) in KL (Ma et al., 2021).
ULD is still generally conceived to be advantageous over OLD and sometimes understood as its momentum-accelerated version.
N OTATIONS AND C ONDITIONS
We will use 2-Wasserstein distance to quantify convergence, i.e
[Equation]
X
[Equation]
Y (cid:107) 2 (cid:17) 12 where Π( µ 1 , µ 2 ) is the set of all couplings of µ 1 and µ 2 .
Assume WLOG that 0 ∈ argmin x ∈ R d f ( x ) .
The following condition will also be frequently used.
A 1.
( Standard Strong-Convexity and Smoothness Condition ) Function f ∈ C 1 ( R d ) : R d (cid:55)→ R is m -stronly-convex and L -smooth, if there exist constants m, L > 0 such that ∀ x , y ∈ R d , we have (cid:107)∇ f ( y ) − ∇ f ( x ) (cid:107) ≤ L (cid:107)
y − x (cid:107) and f ( y ) ≥ f ( x ) + (cid:104)∇ f ( x ) , y − x (cid:105) + m 2 (cid:107) y − x (cid:107)
For f ∈ C 2 , this condition is equivalent to mI (cid:22) ∇ 2 f (cid:22) LI .
T HE C ONSTRUCTION OF HFHR DYNAMICS
HFHR is obtained by formulating NAG-SC as a Hessian free high-resolution ODE, lifting the highresolution term’s coefﬁcient as a free parameter, and adding appropriate noises.
More precisely, let’s start with NAG-SC algorithm:
[Equation]
where s is the learning rate (also known as step size), and c = 1 −√ ms 1+ √ ms is a constant based on s and the strong convexity coefﬁcient m of f ; the method also works for non-strongly-convex f though.
A high-resolution ODE description of Eq.(1) & (2) is obtained in (Shi et al., 2021, Section 2)
[Equation]
which can better account for the effect of non-inﬁnitesimal s than the s → 0 limit (note c depends on s ).
However, in this original form, Hessian of f is involved, which is expensive to compute and store especially for high-dimensional problems.
To obtain a Hessian-free high-resolution ODE description of Equation (1) and (2), we ﬁrst turn the iteration into a ‘mechanical’ version by introducing position variable q k = y k and momentum variable p k
= ( y k − x k ) / h .
Replacing x k +1 in (1) and the ﬁrst x k +1 in (2) by q k +1 and p k +1 , the second x k +1 in (2) by q k − s ∇ f ( q k ) , and the x k in (2) by q k and p k , we obtain
[Equation]
Now, choose γ , α and h as h = √ cs, γ = 1 − ch , α = sh .
It is easy to see that γ > 0 , α > 0 , then
NAG-SC exactly rewrites as
[Equation]
Note the technique for bypassing the Hessian without introducing any approximation is already well studied in the literature (e.g., Alvarez et al.
; Attouch et al.
So far, both h and α are actually determined by the hyperparameter s of NAG-SC.
However, if we now consider α as an independent variable (i.e., ‘lift’ it) and let h → 0 , we see (4) is a 1st-order discretization (with step size h ) of the dynamics
[Equation]
Note α , if inherited from NAG-SC, should be α = (cid:112) s/c
= O ( h ) , which, in a low-resolution ODE will be discarded, and this eventually leads to ULD rather than HFHR.
However, we now allow it to be a free parameter and will see that α (cid:54)
=
O ( h ) can be advantageous.
Before quantifying these advantages later on, we ﬁnish the construction by appropriately injecting
Gaussian noises to Equation (5).
This is just like how OLD can be obtained by adding noise to the gradient ﬂow.
The right amount and structure of noise turn the ODE into a Markov process that can serve the purpose of sampling, and the detailed form of our noise is given by: [Equation]
[Equation]
Here α ≥ 0 , γ > 0 are constant parameters, and W t , B t are independent standard Brownian motions in R d .
This irreversible process will be named as Hessian-Free High-Resolution ( HFHR ) dynamics.
We write it as HFHR ( α, γ ) to emphasize the dependence on α and γ when needed.
Substitution into Fokker-Planck PDE shows HFHR dynamics is unbiased (proof in Appendix B.1):
Theorem 4.1.
π is the invariant distribution of HFHR described in Eq.
(6) , just like ULD.
T HEORETICAL A NALYSIS OF THE HFHR D YNAMICS /A LGORITHM 5. HFHR D YNAMICS IN C ONTINUOUS T IME
Let’s establish the exponential convergence of HFHR dynamics and its additional acceleration when compared to ULD, when the target measure has a strongly-convex and smooth potential.
Theorem 5.1.
Assume Conditions A1 holds and further assume γ 2 > L + m and α ≤ γ 2 − L − m mγ .
Denote the law of q t by µ t .
Then there exists κ (cid:48) > 0 depending only on α and γ , such that
[Equation]
Detailed expression of κ (cid:48) can be found in Appendix A.
Thm. 5.1 state that HFHR converges to the target distribution exponentially fast in log-stronglyconcave and smooth setup.
For ULD, Dalalyan & Riou-Durand (2020, Theorem 1) obtained exponential convergence result in 2-Wasserstein distance with rate √ m √ κ + √ κ − 1 using a simple and elegant coupling approach, and showed this rate is optimal as it is achieved by the bivariate function f ( x, y )
=
m 2 x 2 + L 2 y 2 .
In Thm 5.1, we use the same coupling approach to obtain an (asymptotically) equivalent rate √ m 2 √ κ .
Since ULD is HFHR(0, γ ) and our bound agrees with existing result when α = 0 and shows faster convergence for α > 0 , the acceleration of HFHR in continuous time is evidenced.
For example, if we set γ = 2 √ L and push α to the upper bound speciﬁed in Thm. 5.1, we obtain an O ( √ L ) rate in the log-strongly-concave setup.
Compared with the rate in Dalalyan & Riou-Durand (2020), this is a speed-up of order κ .
5. HFHR A LGORITHM IN D ISCRETE T IME
To obtain an implementable method, we now discretize the time of HFHR dynamics.
As our main goal is to show the acceleration enabled by α won’t disappear after discretization (unlike a fake acceleration due to time rescaling), we’ll just analyze a 1st-order discretization (but a high-accuracy discretization adapted from RMA (Shen & Lee, 2019) will also be provided, in Appendix F).
For simplicity, we’ll work with constant step size h .
Inspired by Strang splitting for differential equations (Strang, 1968; McLachlan & Quispel, 2002), we consider a symmetric composition for updating over each time interval
[ kh, ( k + 1) h ] : x k +1 :
=
φ h 2 ◦ ψ h ◦ φ h 2 ( x k ) where x k = [Equation]
q kh p kh (cid:21) , φ and ψ correspond to solution ﬂows of split SDEs, respectively given by
[Equation]
and φ t ( x 0 ) and ψ t ( x 0 ) mean x ’s value after evolving φ and ψ for t time with initial condition x 0 .
Note that φ ﬂow can be solved explicitly since the second equation is an Ornstein-Unlenbeck process and integrating the second equation followed by integrating the ﬁrst one gives us an explicit solution
[Equation]
For an implementation of the stochastic integral part in Equation 7, denoting X = √ 2 γ
[Equation]
t
[Equation]
d B ( s ) and Y = √ 2 γ (cid:82) t 0 e − γ ( t − s )
d B ( s ) , and the covariance matrix of ( X , Y ) is Cov ( X , Y ) =
[Equation]
  .
As mean and covariance fully determine a Gaussian distribution,
[Equation]
M ξ where M is the Cholesky decomposition of Cov ( X , Y ) , ξ is a 2 d standard Gaussian random vector, i.i.d.
at each step, and φ t can thus be exactly simulated.
However, ψ ﬂow is generally not explicitly solvable unless f is a quadratic function in q .
We simply choose to approximate ψ h ( x 0 ) with one-step Euler-Maruyama integration ψ h ( x 0 )
≈ (cid:101) ψ h ( x 0 ) given by
[Equation]
where η is a standard d -dimensional Gaussian random vector, again i.i.d.
each time ˜ ψ is called.
Altogether, one step of an implementable Strang’s splitting of HFHR is hence φ h 2 ◦ (cid:101)
ψ h ◦ φ h
and we call this numerical scheme the HFHR algorithm, which is summarized in Algorithm 1.
Generate independent standard Gaussian random vectors η k +1 ∈ R d , ξ 1 k +1 , ξ 2 k +1 ∈ R 2 d 6: Run φ h 2 : (cid:20) q [Equation]
p
[Equation]
(cid:21)
[Equation]
Run (cid:101) ψ h : (cid:20) q 2 p
[Equation]
(cid:21) [Equation] (cid:20)
[Equation]
(cid:21) 8:
Run φ h 2 : (cid:20) q 3 p (cid:21) =  
[Equation]
+
[Equation]
9: (cid:20) q ( k +1) h p ( k +1) h (cid:21) ← (cid:20) qp(cid:21) 10: k ← k +11: end while 12: end procedure
As ψ in Strang splitting is replaced by a 1st-order approximation ˜ ψ , the method is of order 1, however with good constant.
This is rigorously established by the following theorem (interested readers are referred to Appendix.
D.5-D.7 and Li et al.
(2021) for more technical details):
Theorem 5.2.
Under Assumption 1, we further assume γ − L + mγ ≥ mα and the function ∇ ∆ f satisﬁes a third-order growth condition, i.e., (cid:13)(cid:13) ∇ ∆ f ( q ) (cid:13)(cid:13) ≤ G (cid:113)
+ (cid:107) q (cid:107) 2 , ∀ q ∈ R d for some G > 0 .
If ( q 0 , p 0 ) ∼ π 0 , then there exists h 0 , C > 0 such that when 0 < h < h 0 , we have
[Equation]
[Equation] where κ (cid:48) is a constant depending only on L, m, γ, α (see Appendix A for detail), µ k is the law of the q marginal of the k -th iterate in Algorithm 1, and µ is the q marginal of the invariant distribution π .
In particular, C = O ( √ d ) and there exists b > 0 , independent of α and is of order O ( √ d ) , s.t.
[Equation]
Remark 5.3.
The linear growth (at inﬁnity) condition on ∇ ∆ f is actually not as restrictive as it appears.
For example, for monomial potentials, i.e., f ( x ) = x p , p ∈
Z
+ , our linear growth condition is met when p ≤ 4 , whereas a standard condition (Pavliotis, 2014, Theorem 3.1) for the existence of SDE solutions holds only when p ≤ 2 .
In addition, our condition is related to the
Hessian Lipschitz condition commonly used in the literature (e.g., Durmus et al.
(2019b)
; Ma et al.
Smoothness and Hessian Lipschitzness imply our condition.
Meanwhile, examples that satisfy linear growth condition but are not Hessian Lipschitz exist, e.g., f ( x ) = x 4 , and thus linear growth condition is not necessarily stronger than Hessian Lipschitzness.
Inspecting the role of α in Equation (8), we see that α clearly increases the rate of exponential decay, but at the same time it can also increase the discretization error (see (9); assuming h is ﬁxed).
However, as the following Corollary 5.4 and its remark will show, the net effect of having a positive α > 0 , at least for some α (cid:63) , is reduced iteration complexity.
Corollary 5.4.
Consider the same assumption as in Thm. 5.2.
If ( q 0 , p 0 ) ∼ π 0 , then there exists h 0 , C > 0 (same as that in Theorem 5.2; recall C = O ( √ d ) ) such that for any target error tolerance (cid:15) > 0 , if we choose h = h (cid:63) (cid:44) min { h 0 , (cid:15) 2 √ 2 C } , then for (cid:15) < 2 √ 2 Ch 0 , after k (cid:63)
[Equation]
steps, we have W 2 ( µ k , µ ) ≤ (cid:15) .
Remark 5.5.
Recall from Thm.5.2 that C ≤ bm ( α 2 − αγ + 1 γ 2 ) , so if we consider the minimizer α (cid:63) of an upper bound of
[Equation]
This suggests that by choosing an optimal α > 0 , one could effectively reduce iteration complexity.
Note, however, that this α (cid:63) may not be the true optimal one as bounds may not be tight.
If they were, k (cid:63)α (cid:63) = (2 √ 3 − 3) k
(cid:63)α =0 ≈ 0 .
46 k (cid:63)α =0 ; i.e., steps needed by ULD (discretized by Alg.1 with α = 0 ) can be halved by HFHR.
Rmk.5.5 shows HFHR algorithm can lead to a similar bound on iteration complexity as ULD algorithm but with an improved constant, and thus is a more efﬁcient algorithm.
This improvement also shows that the acceleration of HFHR can be carried through from continuous time to discrete time.
The same conclusion has been consistently observed in numerical experiments too.
Remark 5.6.
Readers interested in more explicit condition number dependence are referred to Appendix E, where we show, for 2D Gaussian target with condition number κ (cid:29) 1 , the convergences of Euler discretization of ULD under optimal parameters and HFHR under suboptimal parameters are, respectively, given by (1 − 1 /κ
+ o (1 /κ ))
n and (1 − 2 /κ
+ o (1 /κ ))
n , where n is the number of iterations.
The latter (HFHR) is faster despite that its hyperparameters may not be optimal.
N UMERICAL E XPERIMENTS
We now empirically study the acceleration enabled by α (cid:54) = 0 by comparing HFHR algorithm and the popular KLMC discretization of ULD (Dalalyan & Riou-Durand, 2020).
For fairness, discretizations of the same order and cost are compared (Appendix F has an additional comparison).
6. V ERIFICATION OF THEORETICAL RESULTS IN S ECTION 5.
This subsection numerically veriﬁes the d and h dependence of HFHR algorithm as well as the genuine acceleration enabled by α .
For this purpose, we will not use Gaussian targets, because otherwise HFHR will decouple across different (orthogonal) dimensions, and hence its discretization error having a O ( √ d ) dependence would just be a consequence of using 2-Wasserstein distance for quantifying statistical accuracy.
To inspect a more interesting example, we consider a potential that is no longer additive across different dimensions, namely f ( x ) = log ( e x 1 + · · · + e x d ) + 1 2 (cid:107) x (cid:107) 2 .
It’s not hard to see that all dimensions will be coupled in the HFHR dynamics (and ULD too).
Moreover, the new potential f is still a strongly convex function and satisﬁes the assumption in Theorem 5.2.
When the target measure is non-Gaussian, we no longer have a closed form expression for 2-Wasserstein distance and it is computationally expensive to approximate the 2Wasserstein distance by samples.
Therefore, we use the error of mean instead as a surrogate because (cid:13)(cid:13) E µ k q − E µ q (cid:13)(cid:13) ≤ W 2 ( µ k , µ ) and hence the bound in Equation (8) also applies to the error in mean, and so does the iteration complexity bound in Equation (10).
Theorem 5.2 says the ﬁnal sampling error is upper bounded the discretization error that is linear in h and √ d .
To numerically verify the linear dependence on h , we work with d = 2 and ran ULD algorithm for sufﬁciently long time with tiny step size ( h = 0 . 0005 ) to obtain 10 8 independent realizations and use them (as benchmark) to empirically estimate E µ q .
We then set γ = 2 , α =and use Algorithm 1, with h ∈ { 2 k | − 7 ≤ k ≤ 0 } .
For each h , we run Th (with T = 50 ) iterations in Algorithm 1 to ensure the Markov chains are well-mixed and the contribution to ﬁnal error from exponential decay is order-of-magnitude smaller than discretization error.
The results are plotted in Figure 1a.
The observed linear dependence on the step size h is consistent with Theorem 5.2.
[Figure] [Figure]
[Figure] [Figure] To numerically verify the O ( √ d ) dependence, we extensively experiment with d ∈ { 1 , 2 , 5 , 10 , 20 , 50 , 100 , 200 , 500 , 1000 } .
For each d , we run 1,000 independent realizations of ULD algorithm until well converged with tiny step size ( h = 0 . 005 ) and use their empirical average as the ‘true’ mean.
Then we ﬁx γ = 2 , α = 1 , h = 0 .
1 , T = 10 and for each d , we run 1,000 independent realizations of HFHR algorithm for Th = 100 iterations.
Experiment results are plotted in Figure 1b and the linear trend demonstrates that the bound in Thm. 5.2 is tight in terms of d -dependence.
The ﬁnal experiment compares Algorithm 1 with ULD algorithm in terms of iteration complexity.
The goal is to demonstrate the genuine acceleration of HFHR is not an artifact due to time rescaling, which would disappear after discretization as the stability limit changes accordingly.
To do so, we push both ULD and HFHR to their respective largest h values that still allow monotonic convergence at a large scale, and compare their mixing times.
For general nonlinear problems like the one here, Remark 5.5 suggests that with appropriately chosen α , HFHR algorithm effectively reduces the constant factor of the iteration complexity, implying accelerated sampling.
To just provide one empirical veriﬁcation of this improvement, we choose d = 10 and use the error of mean (cid:13)(cid:13)
E µ k q − E µ q (cid:13)(cid:13) to measure sampling accuracy.
The benchmark, i.e., E µ q , is again obtained from 1,000 independent realizations of ULD algorithm with tiny step size ( h = 0 . 005) , ran for long enough to ensure the corresponding Markov chain is wellmixed.
The initial measure is chosen as a Dirac measure at (100 × 1 d , 0 d ) , where 1 d , 0 d are d -dimensional vectors ﬁlled with 1 and 0 respectively.
We pick threshold (cid:15) = 0 .
1 , and for each α ∈ { 0 , 0 .
5 , 1 , 2 , 5 , 10 , 20 , 50 , 100 } , we try all combinations of ( γ, h ) ∈ { 0 . 1 , 0 . 2 , 0 . 5 , 1 , 2 , 5 , 10 , 20 , 50 , 100 } × (cid:8) 0 .
1 × [50] (cid:9) for Algorithm 1 (we also run ULD algorithm when α = 0 ), and empirically ﬁnd the best combination that requires the fewest iterations to meet (cid:13) (cid:13) E µ k q − E µ q (cid:13) (cid:13) ≤ (cid:15) .
We ﬁnd that h = 5 already surpasses the stability limit of ULD algorithm, hence the range of step size covers the largest step size that are practically useable for ULD algorithm.
Experiments are repeated with 100 different seeds to further reduce variance.
The results are shown in Figure 2.
When α > 0 , HFHR algorithm consistently outperforms ULD algorithm (note it also does so when α = 0 because HFHR uses a efﬁciency-wise-comparable but more reﬁned discretization than ULD algorithm).
In particular, when α = 0 .
5 and 1 , which are empirically the best values we found for this experiment, HFHR algorithm achieves the speciﬁed (cid:15) -closeness nearly 6 × times faster than ULD algorithm, and its decreased mixing time (compared to α = 0 for the same algorithm) is consistent with the ≈ 0 .
46 factor in Rmk.5.5).
This empirical study corroborates that the acceleration HFHR dynamics creates also carries through to its discretization, and the acceleration of HFHR algorithm over ULD algorithm can be signiﬁcant.
6. B AYESIAN N EURAL N ETWORK
Now consider Bayesian neural network (BNN) which is a compelling learning model (Wilson, 2020); however, the focus won’t be on its learning capability, and instead we just consider its training, which amounts to a practical, high-dim., multi-modal example of sampling tasks. It no longer satisﬁes the conditions of our analysis, and our goal is to show HFHR still accelerates.
We use fullyconnected network with [22, 10, 2] neurons, ReLU, standard Gaussian prior for all parameters, and compare ULD and HFHR on data set Parkinson from UCI repository (Dua & Graff, 2017).
Choices of hyper-parameter for Algorithm 1 and ULD algorithm are systematically investigated.
For each pair ( γ, α ) ∈ { 0 . 1 , 0 .
5 , 1 , 5 , 10 , 50 , 100 } 2 , we empirically tune the step size to the stability limit of ULD algorithm, simulate 10,000 independent realizations, and use the ensemble to conduct
Bayesian posterior prediction.
HFHR will then use the same step size.
For each γ , we plot the negative log likelihood of HFHR
algorithm (with different α choices) and ULD algorithm on training and test data in Figure 3.
Cases where α is too large for numerical stability are not drawn.
From Figure 3, we ﬁnd that HFHR converges signiﬁcantly faster than ULD in a wide range of setups.
In general, the log-strongly-concave assumption required in Theorem 5.2 does not hold for multimodal target distributions.
However, this numerical result shows that HFHR still accelerates
ULD for highly complex models such as BNN, even when there is no obvious theoretical guarantee.
It showcases the applicability and effectiveness of HFHR as a general sampling algorithm. [Figure] [Table] [Figure]
a i n N
[Figure]
T r a i n N LL
ULD HFHR ( =0.1) HFHR ( =0.5)
[Figure]
[Figure] of [Figure] T r a i n N LL
ULD HFHR ( =0.1)
[Figure]
[Figure]
[Figure]
T r a i n N LL
ULD
[Figure]
C ONCLUSION AND D ISCUSSION
This paper proposes HFHR, an accelerated gradient-based MCMC method for sampling.
To demonstrate the acceleration enabled by HFHR, the geometric ergodicity of HFHR (both the continuous and the discretized versions) is quantiﬁed, and its convergence is provably faster than Underdamped
Langevin Dynamics, which by itself is often already considered as an acceleration of Overdamped
Langevin Dynamics.
As HFHR is based on a new perspective, which is to turn NAG-SC optimizer with ﬁnite learning rate into a sampler, there are a number of interesting directions in which this work can be extended.
Besides further theoretical investigations that aim at reﬁning the error bounds, examples also include the followings: to scale HFHR up to large data sets, full gradient may be replaced by stochastic gradient (SG) — how to quantify, and hence optimize the performance of SG-HFHR?
Can the generalization ability of HFHR trained learning models (e.g., BNN) be quantiﬁed, and how does it compare with that by LMC and/or KLMC?
These will be future work.