Improved Convergence Rate of Stochastic Gradient Langevin Dynamics with Variance Reduction and its Application to Optimization
Abstract
The stochastic gradient Langevin Dynamics is one of the most fundamental algorithms to solve sampling problems and non-convex optimization appearing in several machine learning applications.
Especially, its variance reduced versions have nowadays gained particular attention.
In this paper, we study two variants of this kind, namely, the Stochastic Variance Reduced Gradient Langevin Dynamics and the Stochastic Recursive Gradient Langevin Dynamics.
We prove their convergence to the objective distribution in terms of KL-divergence under the sole assumptions of smoothness and Log-Sobolev inequality which are weaker conditions than those used in prior works for these algorithms.
With the batch size and the inner loop length set to p n , the gradient complexity to achieve an ✏ -precision is ˜ O (( n + dn 1 / 2 ✏  1 )  2 L 2 ↵  2 ) , which is an improvement from any previous analyses.
We also show some essential applications of our result to non-convex optimization.
Introduction 1.1 Background and Organization
Over the past decade, the gradient Langevin Dynamics (GLD) has gained particular attention for providing an effective tool for sampling from a Gibbs distribution, a fundamental task omnipresent in the ﬁeld of machine learning and statistics, and for non-convex optimization, which is nowadays witnessing an unignorable empirical success.
Notably, GLD is a stochastic differential equation (SDE) that can be viewed as the steepest descent ﬂow of the Kullback-Leibler (KL) divergence towards the stationary Gibbs distribution in the space of measures endowed with the 2-Wasserstein metric (Jordan et al., 1998).
As a consequence of the unique properties of GLD, its implementable discrete schemes and their ability to suitably track it have been the subject of a large number of studies.
The Euler-Maruyama scheme of GLD gives rise to an algorithm known as the Langevin Monte
Carlo method (LMC).
This algorithm is biased (Wibisono, 2018): that is, the distribution of the 36th Conference on Neural Information Processing Systems (NeurIPS 2022). discrete scheme does not converge to the same as GLD.
Nonetheless, it has been shown that this bias could be made arbitrarily small under certain assumptions by taking a sufﬁciently small step size (Dalalyan, 2017b; Vempala and Wibisono, 2019).
Dalalyan (2017a,b) provided one of the ﬁrst non-asymptotic rates of convergence of LMC for smooth log-concave distributions.
Assumptions to obtain a non-asymptotic analysis and this controllable bias have been relaxed by further research to dissipativity and smoothness (Raginsky et al., 2017; Xu et al., 2018), and recently to Log-Sobolev inequality (LSI) and smoothness (Vempala and Wibisono, 2019).
This relaxation of conditions is especially meaningful as the objective distribution nowadays tends to become more and more complicated beyond the classical assumption of log-concavity.
However, in the ﬁeld of machine learning, the main function can often be formulated as the average of the loss function of an enormous number of training data points (Welling and Teh, 2011), which subsequently makes it difﬁcult to calculate its full gradient.
As a result, research on stochastic algorithms has been also conducted to avoid this computational burden (Chen et al., 2021; Dubey et al., 2016;
Raginsky et al., 2017; Welling and Teh, 2011; Xu et al., 2018;
Zou et al., 2018, 2019a,b, 2021).
Welling and Teh (2011) introduced the concept of Stochastic Gradient Langevin Dynamics (SGLD) which combines the Stochastic Gradient Descent with LMC.
This has been the subject of successful studies (Raginsky et al., 2017; Welling and Teh, 2011; Xu et al., 2018).
Nevertheless, the variance of its stochastic gradient is too large, which leads to a slow convergence compared to LMC.
Therefore, stochastic gradient Langevin Dynamics algorithms with variance reduction, such as the
Stochastic Variance Reduced Gradient Langevin Dynamics (SVRG-LD), have been considered and their convergence has been thoroughly analyzed for both sampling (Dubey et al., 2016; Zou et al., 2018, 2019a, 2021) and optimization (Huang and Becker, 2021; Xu et al., 2018).
Dubey et al.
ﬁrst united SGLD with variance reduction techniques and proposed two new algorithms, namely, SVRG-LD and SAGA-LD.
Chatterji et al.
(2018) and Zou et al.
(2018) proved the convergence rate of SVRG-LD to the target distribution in 2-Wasserstein distance for smooth log-concave distributions.
Xu et al.
(2018) showed the weak convergence of SVRG-LD under the smoothness and dissipativity conditions.
They expanded the non-asymptotic analysis of Raginsky et al.
(2017) to LMC and SVRG-LD, and improved the result for SGLD.
Few years ago, Zou et al. (2019a) provided the gradient complexity of SVRG-LD to converge to the stationary distribution in 2-Wasserstein distance under the smoothness and dissipativity assumptions.
This convergence can be even improved if we make a warm-start (Zou et al., 2021).
While these works investigated algorithms with ﬁxed hyperparameters, Huang and Becker (2021) additionally assumed a strict saddle and some other minor conditions to study SVRG-LD with a decreasing step size and improved its convergence in high probability to the second order stationary point.
Zou et al.
(2019b) also applied variance reduction techniques to the Hamiltonian Langevin Dynamics, or underdamped Langevin Dynamics in opposition to GLD also known as overdamped Langevin Dynamics.
As we can observe, the current convergence analyses of the stochastic schemes with variance reduction are mostly restricted to log-concavity and dissipativity, and do not enjoy the same broad convergence guarantee with a concrete gradient complexity as LMC does under LSI and smoothness in terms of KL-divergence.
Therefore, in order to bridge this theoretical gap between LMC and stochastic gradient Langevin
Dynamics with variance reduction, we study in this paper the convergence of the latter under the relaxed assumptions of smoothness and LSI.
In Section 3, we study the convergence to the Gibbs distribution of SVRG-LD and the Stochastic Recursive Gradient Langevin Dynamics (SARAH-LD), another variant of stochastic gradient Langevin Dynamics with variance reduction inspired by the
Stochastic Recursive Gradient algorithm (SARAH) of Nguyen et al.
(2017a,b).
On the other hand, optimization and sampling are only two sides of the same coin for GLD.
That is why, in Section 4, we also investigate implications of Section 3 for non-convex optimization.
We prove the convergence of SVRG-LD and SARAH-LD to the global minimum of dissipative functions and we provide their non-asymptotic rate of convergence.
We also consider the additional weak Morse assumption and study its effect.
Finally, we illustrate our main result with a simple experiment.
1.2 Contributions
The major contributions of this paper can be summarized as follows.
We provide a non-asymptotic analysis of the convergence of SVRG-LD and SARAH-LD to the Gibbs distribution in terms of
KL-divergence under smoothness and LSI which are weaker conditions than those used in prior works for these algorithms.
KL-divergence is generally a stronger convergence criterion than both total
[Table]
[Table] This [Table] variation (TV) and 2-Wasserstein distance as they can be controlled by KL-divergence under the LSI condition.
Notably, we prove that, with the batch size and inner loop length set to p n , the gradient complexity to achieve an ✏ -precision in terms of KL-divergence is ˜ O (( n + dn 1 / 2 ✏  1 )  2 L 2 ↵  2 ) , which is better than any previous analyses.
See Table 1 for a comparison with previous research in terms of assumptions, criterion and gradient complexity.
We also prove the convergence of SVRGLD and SARAH-LD to the global minimum under an additional assumption of dissipativity with a gradient complexity of ˜ O
( n + n 1 / 2 ✏  1 dL ↵  1 )  2 L 2 ↵  2 ) which is better than previous work since it has almost all the time a dependence on n of O ( p n ) and does not require the batch size and the inner loop length to depend on the accuracy ✏ .
On the other hand, we import the idea of Li and
Erdogdu (2020) from product manifolds of spheres to the Euclidean space in order to show that under the additional assumption of weak Morse, the convergence in the Euclidean space can be accelerated by eliminating the exponential dependence on 1 / ✏ .
1.3 Other Related Works
The theoretical study of GLD goes back to Chiang et al.
(1987) who showed that global convergence could be achieved with a proper annealing schedule.
This work did not specify how to implement this SDE, but Gelfand and Mitter (1991) ﬁlled this gap.
Later, Borkar and Mitter (1999) proved an asymptotic convergence in terms of relative entropy for the discrete scheme of gradient Langevin
Dynamics when the inverse temperature and the step size are kept constant.
The variance reduction technique, introduced to Langevin Dynamics by Dubey et al.
(2016), was originally presented by Johnson and Zhang (2013) as Stochastic Variance Reduced Gradient (SVRG) to improve the convergence speed of Stochastic Gradient Descent.
Other variance reduction techniques were also considered such as the Stochastic Recursive Gradient Langevin Dynamics (SARAH) from
Nguyen et al.
(2017a,b) which outperforms SVRG in non-convex optimization (Pham et al., 2020) and is used in many algorithms such as SSRGD (Li, 2019) and SpiderBoost (Wang et al., 2019).
Li and Erdogdu (2020) extended Vempala and Wibisono’s result to Riemannian manifolds.
One of the highlights of their work is that they showed the Log-Sobolev constant of the Gibbs distribution for a product manifold of spheres only depends on a polynomial of the inverse temperature under some particular conditions including weak Morse.
We will adapt this result to our situation.
In the concurrent work of Balasubramanian et al. (2022) (especially Section 6), they also studied the convergence of stochastic schemes of GLD with more relaxed conditions than prior analyses.
However, our contributions are not overshadowed by theirs, and we clarify the reasons.
In Subsection 6.1 of their paper, Balasubramanian et al.
(2022) focused on stochastic discrete schemes with ﬁnite variance and bias (which is not the case for SVRG-LD) and provided a ﬁrst-order convergence guarantee in the space of measures equipped with the 2-Wasserstein distance.
Subsection 6.2 proved a global convergence under some other conditions but most of these two analyses did not consider in particular the usual case in machine learning when F is the average of some other functions, which leads to a generally worse gradient complexity than ours.
Concerning this ﬁnite sum setting,
Balasubramanian et al.
(2022) investigated the Variance Reduced LMC
algorithm (slightly different from SVRG-LD in this paper) in Subsection 6.3 and gave a ﬁrst-order convergence under the sole assumption of smoothness.
When restrained in our problem setting, the gradient complexity of
SVRG-LD and SARAH-LD
we provide is still considerably better (see Section 3 for more details).
1.4 Notation
We denote deterministic vectors by a lower case symbol (e.g., x ) and random variables by an upper case symbol (e.g., X ).
The Euclidean norm is denoted by k · k for vectors and the inner product by h · , · i .
For matrices, k · k is the norm induced by the Euclidean norm for vectors.
We only treat distributions absolutely continuous with respect to the Lebesgue measure in R d for simplicity.
Especially, throughout the paper, ⌫ refers to the probability measure with the density function d ⌫ / e   F d x , where F is a function introduced below.
a _ b is equivalent to max { a, b } and a ^ b to min { a, b } .
We also use the shorthand ˜
O to hide logarithmic polynomials.
Preliminaries
In this section, we brieﬂy explain the problem setting, necessary mathematical background and assumptions used in this paper.
2.1 Problem Setting and GLD
In Section 3, we consider sampling from a distribution written in the form d ⌫ / e   F d x where  is a positive constant (which corresponds to the inverse temperature) and F : R d !
R is formulated as
F ( x ) :
= 1 n
P ni [-=
-][Equation]
1 f i ( x ) , the average of the loss function of n training data points { x ( i ) } ni =1 .
Here, f i ( x ) :
= f ( x, x ( i ) ) can be regarded as the loss of data x ( i ) .
For instance, F can be the average of the negative log likelihood of n training data points.
In Section 4, we consider the non-convex optimization (minimization) of the same F as above.
GLD can be described as the following stochastic differential equation (SDE):
[Equation]
where  > 0 is called the inverse temperature parameter and { B ( t ) } t  0 is the standard Brownian motion in R d .
It can be used for sampling since under some reasonable assumptions of F , the distribution ⇢ GLD t of X GLD t governed by SDE (1) converges to the invariant stationary distribution d ⌫ / e   F d x , also known as the Gibbs distribution (Chiang et al., 1987).
Moreover, as previously mentioned, this convergence is efﬁcient in the sense that SDE (1) corresponds to the steepest descent ﬂow of the Kullback-Leibler (KL) divergence towards the stationary distribution in the space of measures endowed with the 2-Wasserstein metric (Jordan et al., 1998).
Alternatively, GLD can be interpreted as the composite optimization problem of a negative entropy and an expected function value as follows (Wibisono, 2018):
[Equation]
The gradient ﬂow is the well-known Fokker-Planck equation associated to SDE (1):
[Equation]
This will be useful in our analysis.
In addition to its potential for sampling, GLD can also be employed for non-convex optimization as the Gibbs distribution concentrates on the global minimum
1 input:
>
, batch
, epoch m , inverse temperature  2 initialization:
3 foreach s = 0 ,
/m ) do
randomly draw ✏ sm ⇠ N
0 , I d ⇥ d
foreach l = 1 , . . , m
1 do = randomly pick a subset I k from { size | I k
B randomly draw k
if
LD then
else if LD then
end end
19 end
2.2 Algorithms of GLD
Applying the Euler-Maruyama scheme to (1), we obtain the Langevin Monte Carlo (LMC)
[Equation] where ⌘ is called the step size. This is similar to the gradient descent except the additional Gaussian noise p 2 ⌘ / ✏ k , where ✏ k ⇠ N (0 , I d ⇥ d ) and I d ⇥ d is the d ⇥ d unit matrix. In the case n is huge and the computation of r F is too difﬁcult, we are incited to use stochastic gradient methods in analogy to stochastic gradient optimization.
This gives
[Equation]
where v ( X k ) is the stochastic gradient.
When v ( X k ) is deﬁned as 1 B
P i k
I k r f i k
( X k ) , where B is called the batch size
and I k is a random subset uniformly chosen from { 1 , . . . , n } such that | I k
| =
B , we obtain the Stochastic Gradient Langevin Dynamics (SGLD).
As this method exhibits a slow convergence, it has been popular to use variance reduction methods such as the Stochastic Variance Reduced Gradient Langevin Dynamics (SVRG-LD) where v ( X k ) = B
{+P i k 
 I k (+} r f i k
( X k )  r f i k
( X ( s ) ))
+ r F ( X ( s ) ) .
Details of this algorithm is stated in Algorithm 1.
X ( s ) is a reference point updated every m steps so that X sm =
X ( s ) .
As we can observe in
Lemma A.4, around the optimal point, the variance of the stochastic gradient is indeed decreased as X ( s ) and X k are both close to each other.
We can also easily extend some successful stochastic gradient algorithms to Langevin Dynamics.
Hence, we are motivated to extend the Stochastic Recursive Gradient Algorithm (SARAH) to Langevin Dynamics since we can expect that some bottlenecks of the analysis of SVRG-LD can be removed in that of SARAH-LD as subtracting the previous stochastic gradient enables a stabler performance than SVRG-LD.
This algorithm can be described as
Algorithm 1 with v ( X k ) =
1 B
[Equation]
Deﬁnition 1.
We deﬁne ⇢ k as the distribution of X k generated at the k th step of SVRG-LD, and similarly  k for SARAH-LD.
2.3 Assumptions
The assumptions used throughout this paper can be summarized as follows.
Assumption 1.
For all i = 1 , . . . , n , r f i is twice differentiable, and 8 x, y 2 R d , kr 2 f i ( x ) k  L .[-
-]
In other words, f i ( i = 1 , . . . , n ) and F are L -smooth.
Assumption 2.
Distribution ⌫ satisﬁes the Log-Sobolev inequality (LSI) with a constant ↵ .
That is, for all probability density functions ⇢ absolutely continuous with respect to ⌫ , the following holds:
[Equation]
where H ⌫ ( ⇢ ) :
= E ⇢ ⇥ log ⇢⌫ ⇤ is the KL-divergence of ⇢ with respect to ⌫ , and J ⌫ ( ⇢ ) :=
E ⇢ h r log ⇢⌫  i is the relative Fisher information of ⇢ with respect to ⌫ .
The recent work of Vempala and Wibisono (2019) motivates us to use the combination of smoothness and LSI for the analysis of SVRG-LD and SARAH-LD.
Indeed, they showed that these conditions were enough to assure for the Euler-Maruyama scheme an exponentially fast convergence and a bias controllable by the step size.
Under smoothness, LSI is not only the necessary condition of logconcavity and dissipativity, but is also robust to bounded perturbation and Lipschitz mapping, contrary to log-concavity (Vempala and Wibisono, 2019).
For example, for any distribution d ⌫ that satisﬁes
LSI and bounded function B : R d !
R , d˜ ⌫ / e B d ⌫ satisﬁes LSI as well (Holley and Stroock, 1986).
Moreover, while KL-divergence is not in general convex with regard to the Wasserstein geodesic, thanks to LSI, the Polyak- Ł ojaciewicz condition is satisﬁed.
It is well-known that LSI sufﬁces to realize an exponential convergence for the case of continuous time Langevin Dynamics (Vempala and Wibisono, 2019).
That is why, it is actually both useful and natural to suppose LSI in this context.
Note that under L -smoothness of F and LSI with constant ↵ for d ⌫ / e   F d x , it holds that ↵   L (Vempala and Wibisono, 2019).
As for optimization, we additionally use the following conditions.
Assumption 3.
F is ( M, b ) -dissipative.
That is, there exist constants M > 0 and b > 0 such that for all x 2 R d the following holds: hr F ( x ) , x i [Equation]
M
[Equation]
Assumption 4 (Li and Erdogdu (2020), Assumption 3.3) .
F satisﬁes the weak Morse condition.
That is, for all non-zero eigenvalues of the Hessian of stationary points, there exists a constant  † 2 (0 , 1] such that
[Equation]
Furthermore, for the set S of stationary points that are not a global minimum, sup x 2 S  min
[Equation]
Assumption 5.
r 2 f i is L 0 -Lipschitz and without loss of generality, we let min x 2 R d F ( x ) = 0 .
Assumption 6.
F has a unique global minimum.
Smoothness and dissipativity are a classical combination of assumptions for this kind of problem setting (Raginsky et al., 2017; Xu et al., 2018; Zou et al., 2019a).
We assume dissipativity instead of
LSI for non-convex optimization in order to obtain an explicit value of the Log-Sobolev constant of d ⌫ / e   F d x in function of the inverse temperature parameter  (see Property C.3), making a non-asymptotic analysis possible.
Furthermore, Assumptions 4 to 6 can ameliorate the exponential dependence of the inverse of the Log-Sobolev constant on the inverse temperature parameter to a polynomial one (see Property C.4).
Main Results
In this section, we state our main results which prove that SVRG-LD and SARAH-LD (Algorithm 1) achieve an exponentially fast convergence to the Gibbs distribution and a controllable bias in terms of KL-divergence under the sole assumptions of LSI and smoothness.
We provide their gradient complexity as well.
The proofs can be found in Appendix A and B respectively.
3.1 Improved Convergence of SVRG-LD
Our analysis shows that the convergence of SVRG-LD to the stationary distribution d ⌫ / e   F d x can be formulated as the theorem below.
Theorem 1.
Under Assumptions 1 and 2, 0 < ⌘ < ↵ 16 p 6 L 2 m  ,   1 and B  m , for all k = 1 , 2 , . . .
, the following holds in the update of SVRG-LD where ⌅ = ( n  B ) B ( n  1) :
[Equation]
We observe that the bias term of the upper bound, which is the second term linearly dependent on ⌘ , can be easily controlled while the ﬁrst term exponentially converges to 0 with k ! 1 .
This is more precisely formulated in the following corollary.
Corollary 1.1.
Under the same assumptions as Theorem 1, for all ✏  0 , if we choose step size ⌘ such that ⌘  [Equation] 2 , then a precision H ⌫ ( ⇢ k )  ✏ is reached after k  ↵⌘ [Equation]
steps. Especially, if we take B = m = p n and the largest permissible step size ⌘ =
[Equation]
then the gradient complexity becomes
[Equation]
This gradient complexity is an improvement compared with prior works for three reasons.
First of all, we provide a non-asymptotic analysis of the convergence of SVRG-LD under smoothness and LogSobolev inequality which are conditions weaker than those (e.g., log-concavity or dissipativity) used in prior works for these algorithms.
Moreover, we prove it in terms of KL-divergence which is generally a stronger convergence criterion than both total variation (TV) and 2-Wasserstein distance as they can both be controlled by KL-divergence under the LSI condition.
For instance, TV was used by Zou et al. (2021) and 2-Wasserstein distance by Dalalyan (2017a) and Zou et al. (2019a).
KL-divergence makes it possible to unify these two different criteria.
Finally, while prior research generally used Girsanov’s theorem which generates a bias term that accumulates through the iteration (see for example Raginsky et al.
(2017) and Xu et al.
, we solve this issue by taking beneﬁt of the exponential convergence of GLD to the Gibbs distribution under LSI and smoothness that enables us to forget about past bias.
That way, with the batch size and inner loop set to p n , the gradient complexity to achieve an ✏ -precision in terms of KL-divergence becomes ˜ O
(( n + dn 1 / 2 ✏  1 )  2 L 2 ↵  2 ) , which is better than previous analyses.
For example, Vempala and Wibisono (2019) provided a gradient complexity of ˜ O
[Equation]
for LMC under Assumptions 1 and 2, and Zou et al. (2019a) a gradient complexity of ˜ O ( n + n 3 / 4 ✏  2 + n 1 / 2 ✏  4 ) ·
e ˜ O
(  + d ) for SVRG-LD under Assumptions 1 and 3.
Note that the dependence on the dimension d is not improved since ↵  1 may exponentially depend on d .
Recently, Zou et al.
(2019b) proposed the Stochastic Gradient Hamiltonian Monte Carlo Methods with Recursive Variance Reduction with a gradient complexity of ˜ O
( n + n 1 / 2 ✏  [Equation]
[Equation] ⇤ ) [Equation] ) in terms of 2-Wasserstein distance.
Even though their algorithm is based on the underdamped
Langevin Dynamics whose discrete schemes use to perform better than those of the overdamped
Langevin Dynamics such as SVRG-LD, our gradient complexity, which applies to a broader family of distributions, is almost the same except for a small interval of ✏ , but we do not require the batch size B and the inner loop length m to depend on ✏ while Zou et al.
(2019b) do, i.e., B .
B 1 / 2 0 , m = O ( B 0 /B ) , where B 0 = ˜
O
[Equation]
.
This strengthens the importance of our result since it shows that adapting this analysis to other stochastic schemes of GLD is promising and could lead to tighter bounds and relaxation of conditions.
See Table 1 for a summary.
Concerning the concurrent work of Balasubramanian et al.
(2022), under the sole assumption of smoothness, they provided a gradient complexity of O
( L 2 d 2 n/ ✏ 2 ) for the Variance Reduced LMC algorithm that updates the stochastic gradient differently as SVRG-LD and SARAH-LD.
This is almost the square of our result, and in some extent, our work can be interpreted as an acceleration of their result with a slightly stronger additional condition than Poincaré inequality.
Proof Sketch Proceeding in a similar way as Vempala and Wibisono (2019), we evaluate how
H ⌫ ( ⇢ k ) decreases at each step as shown in Theorem A.1 of Appendix A.
This is realized by comparing the evolution of the continuous-time GLD for time ⌘ and one step of SVRG-LD.
Since we use a stochastic gradient, we need at the same time to evaluate the variance of the stochastic gradient.
Theorem 1 can be obtained by recursively solving the inequality derived in Theorem A.1.
3.2 Convergence Analysis of SARAH-LD
As for SARAH-LD, its convergence to the stationary distribution d ⌫ / e   F d x can be formulated as the theorem below.
Interestingly, we obtain the same result as SVRG-LD (Theorem 1) but we do not require B  m anymore.
Theorem 2.
Under Assumptions 1 and 2, 0 < ⌘ < ↵ 16 p 2 L 2 m  and   1 , for all k = 1 , 2 , . . .
, the following holds in the update of SARAH-LD where ⌅ = ( n  B ) B ( n  1) :
[Equation]
This is the ﬁrst convergence guarantee of SARAH-LD in this problem setting so far, and it leads to the following gradient complexity.
Corollary 2.1.
Under the same assumptions as Theorem 2, for all ✏  0 , if we choose step size ⌘ such that ⌘ 
[Equation]
, then a precision H ⌫ (  k )  ✏ is reached after
[Equation]
steps.
Especially, if we take B = m = p n and the largest permissible step size ⌘ =
[Equation]
, then the gradient complexity becomes
[Equation]
The reason why we obtain the same gradient complexity for both SARAH-LD and SVRG-LD (except better coefﬁcients for SARAH-LD) is that in our analysis, the Brownian noise added at each step of the Langevin Dynamics plays the role of a fundamental bottleneck that even SARAH-LD could not eliminate, and we still need to set B = m = p n .
We can hypothesize that this order of gradient complexity might be tight for variance-reduced stochastic gradient Langevin Dynamics algorithms.
Some Applications to Non-Convex Optimization
Here, we apply our main results to non-convex optimization.
Thanks to our analysis applicable to a broader family of probability distributions satisfying LSI, the additional conditions we pose in this section are mainly reﬂected in the concrete formulation of the Log-Sobolev constant, which keeps our study simple and clear.
The proofs can be found in Appendix C. Since SVRG-LD and SARAH-LD exhibited almost the same performance in sampling, we can simultaneously analyse them.
We ﬁrst prove the convergence to the global minimum of SVRG-LD and SARAH-LD without clarifying the explicit formulation of the Log-Sobolev constant in function of  .
Theorem 3.
Using SVRG-LD or SARAH-LD, under Assumptions 1 to 3, 0 < ⌘ <
[Equation]
4 d ✏ log  e LM  _ 8 db ✏ 2 _ 1 _ 2 M and B  m , if we take B
= m = p n and the largest permissible step size ⌘ =
[Equation]
16 p 6 L 2 p n  ^ 3 1792 ↵ 2 ✏ L 2 d  , the gradient complexity to reach a precision of
[Equation]
is
[Equation]
where ↵ is a function of  , and X ⇤ is the global minimum of F .
Remark 1.
Under Assumptions 1 and 3, Assumption 2 is negligible as shown in Property C.2.
Under Assumptions 1 to 3 only, this leads to a gradient complexity which exponentially depends on the inverse of the precision level ✏ as shown in the next corollary since the inverse of the Log-Sobolev constant exponentially depends on  .
Corollary 3.1.
Under the same assumptions as Theorem 3, taking  = i ( ✏ ) :
=
4 d ✏ log
[Equation]
1 _ 2 M , we obtain a gradient complexity of
[Equation]
since ↵ =  C 1 e  C 2  (Property C.3).
The second term with n 1 / 2 is almost all the time dominant since it has a factor that exponentially depends on 1 / ✏ and the ﬁrst term not.
This dependence on n of O ( n 1 / 2 ) is the best so far for these algorithms.
Moreover, comparing with the gradient complexity ˜
O
n 1 / 2   4 ✏  5 / ·
e ˜ O
( d ) , also of order n 1 / 2 , provided by Xu et al.
(2018) who used SVRG-LD and the same assumptions, our gradient complexity is an improvement since their analysis required a batch size B and an inner loop length m that strongly depend on ✏ (i.e., B = p n ✏  3 / 2 , m = p n ✏ 3 / 2 ) and ours does not.
Note that the dependence of the gradient complexity of Xu et al.
(2018) on 1 / ✏ is not necessarily better than ours as  is actually the spectral gap of the discrete-time Markov chain generated by (1) and its inverse exponentially depends on 1 / ✏ as well.
Although Xu et al.
(2018) did not investigate the explicit nature of  , this is supported by Raginsky et al.
(2017) who proved this exponential dependence for the spectral gap of the continuous-time SDE and by Mattingly et al.
(2002) who showed the spectral gap of continuous-time SDE and that of discrete-time version are almost the same in this context.
Analysis under the weak Morse condition Now, under the additional Assumptions 4 to 6, it is interesting to note that a polynomial dependence on 1 / ✏ is achieved as the following corollary shows.
Corollary 3.2.
Under the same assumptions as Theorem 3 and Assumptions 4 to 6, taking  = j ( ✏ ) :
=
4 d ✏ log
[Equation]
[Equation] _ 2 M _ C  , where C  is a constant independent of ✏ deﬁned in Property C.4, we obtain a gradient complexity of
[Equation]
since ↵ = C 3 /  (Property C.4).
The crux of this corollary is Property C.4.
To prove this, we show like Li and Erdogdu (2020) that ⌫ satisﬁes the Poincaré inequality with a constant independent of  .
Since it is not hard to show this around the global minimum, we can step by step extend the set where this inequality holds by a
Lyapunov argument (Theorems D.1 and D.2).
The essential difference between this analysis and that of Li and Erdogdu (2020) is that we do not work on compact manifolds anymore.
Some rather minor difﬁculties emerge as we cannot employ the compactness but they can be addressed by supposing dissipativity which assures a quadratic growth for large x .
Remark 2.
These results do not deﬁnitively assert that SARAH-LD and SVRG-LD show the exact same performance in terms of optimization.
Indeed, suppose we are close enough to the global optimum.
Then, a big noise is not necessary anymore since it is more important to stably converge to the global minimum.
Here, we should be able to signiﬁcantly decrease the noise ✏ k , and the bottleneck from the noise should disappear.
In this case, SARAH-LD would perform better than SVRG-LD as we approach the original non-convex optimization setting where SARAH outperforms SVRG.
Remark 3.
We also investigated an annealed version of SVRG-LD and SARAH-LD but could not ameliorate the gradient complexity.
The detailed analysis can be found in Appendix E.
Experiment
In this section, we illustrate our main result with a simple experiment.
1 We follow the same problem setting as that of Welling and Teh (2011) in Section 5.1.
That is, we aim to sample from a non-logconcave posterior distribution p ( ✓ | x ) / p ( ✓ )
Q ni =1 p ( x i | ✓ ) where { x i } ni =1 is sampled from p ( x | ✓ ) , a distribution parameterized by ✓ = ( ✓ 1 , ✓ 2 ) . The prior p ( ✓ ) and the distribution of x parametrized by ✓ are respectively deﬁned as ✓ 1 ⇠ N (0 , 10) , ✓ 2 ⇠ N (0 , 1) and x ⇠ 1 / 2 N ( ✓ 1 , 2) + 1 / 2 N ( ✓ 1 + ✓ 2 , 2) . Here, we set n = 10000 , ✓ 1 = 0 and ✓ 2 = 1 . Using the obtained 10000 samples, we simulatedpoints of SVRG-LD with the inner loop length m = n/B and different batch sizes B , namely, 100, 1000 and 10000 so that B  m as required in Theorem 1.
Evolution of KL-divergence between the true posterior, estimated by the Metropolis-adjusted Langevin algorithm, and that simulated by
SVRG-LD is plotted in Figure 1.
KL-divergence was approximated following Pérez-Cruz (2008).
As we can observe, Figure 1 correctly reproduces the theoretical bound of Theorem 1, with an exponential convergence in the beginning and a persistent bias due to the use of a discrete scheme and mini-batches.
The fastest convergence in terms of gradient complexity under the condition B  m is achieved by SVRG-LD with B = p n , which conﬁrms our main theorem.
Furthermore, with this best batch size, we also simulated 1000 points of SGLD and SARAH-LD as shown in Figure 1 as well.
While SGLD and SVRG-LD have similar convergence speed in the beginning, the latter eventually achieves a higher precision thanks to the variance reduction method adopted in this scheme.
SARAH-LD exhibits a similar performance as SVRG-LD, which agrees with Theorem 2.
Discussion and Conclusion
The main limitations of our work reside in the gap between practice and theory.
Indeed, while our paper supposes assumptions quite standard in the literature of GLD, it cannot explain the whole empirical success that machine learning is currently experiencing.
Some choices of parameters may also seem different than the practical use.
However, compared to previous work, we succeeded in proving convergence of GLD with the popular stochastic gradient with relaxed conditions, and deleting the dependence of batch size and inner loop length on epsilon, which are all more realistic situations than prior work.
The theoretical study in machine learning and deep learning precisely plays the role of ﬁlling as much as possible this large gap, and our work could be regarded as a further step forward to achieve this goal.
Furthermore, in this paper, we focused on the pure sampling and optimization performance of the algorithms, and some of the drawbacks are simply due to this fact.
For example, another limitation is that we did not investigate the generalization error in Section 4, but this was only outside the scope of this work.
In conclusion, we analysed the convergence rate of stochastic gradient Langevin Dynamics with variance reduction under smoothness and LSI and its application to optimization.
In Section 3, we proved the convergence of SVRG-LD in terms of KL-divergence with more relaxed conditions (LSI and smoothness) and with a better gradient complexity than previous works.
We also expanded
SARAH to SARAH-LD and showed that this algorithm enjoyed the same advantages as SVRG-LD with only an improvement in the coefﬁcients of the gradient complexity.
These results led us to apply SVRG-LD and SARAH-LD to non-convex optimization in Section 4.
We provided the global convergence and a non-asymptotic analysis of SVRG-LD and SARAH-LD.
We obtained better conditions than prior works.
Furthermore, we showed that under the additional assumption including weak Morse and Hessian Lipschitzness, the gradient complexity could be ameliorated, eliminating the exponential dependence on the inverse of the required error.
Acknowledgments and Disclosure of Funding
This study was partially supported by Japan Digital Design and JST CREST.
We would like to thank Mufan Li and Murat A. Erdogdu for their valuable comments to ﬁx some issues in the proof in Appendix D and their suggestion to improve it.
We also thank anonymous reviewers for their feedback.