Improved Convergence Rate of Stochastic Gradient Langevin Dynamics with Variance Reduction and its Application to Optimization
Abstract
The stochastic gradient Langevin Dynamics is one of the most fundamental algorithms to solve sampling problems and non-convex optimization appearing inseveral machine learning applications.
Especially, its variance reduced versionshave nowadays gained particular attention.
In this paper, we study two variantsof this kind, namely, the Stochastic Variance Reduced Gradient Langevin Dynamics and the Stochastic Recursive Gradient Langevin Dynamics.
We prove theirconvergence to the objective distribution in terms of KL-divergence under thesole assumptions of smoothness and Log-Sobolev inequality which are weakerconditions than those used in prior works for these algorithms.
With the batchsize and the inner loop length set to p n , the gradient complexity to achieve an✏ -precision is ˜ O (( n + dn 1 / 2 ✏  1 )  2 L 2 ↵  2 ) , which is an improvement from anyprevious analyses.
We also show some essential applications of our result tonon-convex optimization.
Introduction1.1 Background and Organization
Over the past decade, the gradient Langevin Dynamics (GLD) has gained particular attention forproviding an effective tool for sampling from a Gibbs distribution, a fundamental task omnipresentin the ﬁeld of machine learning and statistics, and for non-convex optimization, which is nowadayswitnessing an unignorable empirical success.
Notably, GLD is a stochastic differential equation (SDE)that can be viewed as the steepest descent ﬂow of the Kullback-Leibler (KL) divergence towards thestationary Gibbs distribution in the space of measures endowed with the 2-Wasserstein metric (Jordanet al., 1998).
As a consequence of the unique properties of GLD, its implementable discrete schemesand their ability to suitably track it have been the subject of a large number of studies.
The Euler-Maruyama scheme of GLD gives rise to an algorithm known as the Langevin Monte
Carlo method (LMC).
This algorithm is biased (Wibisono, 2018): that is, the distribution of thediscrete scheme does not converge to the same as GLD.
Nonetheless, it has been shown that thisbias could be made arbitrarily small under certain assumptions by taking a sufﬁciently small stepsize (Dalalyan, 2017b; Vempala and Wibisono, 2019).
Dalalyan (2017a,b) provided one of the ﬁrstnon-asymptotic rates of convergence of LMC for smooth log-concave distributions.
Assumptions toobtain a non-asymptotic analysis and this controllable bias have been relaxed by further research todissipativity and smoothness (Raginsky et al., 2017; Xu et al., 2018), and recently to Log-Sobolevinequality (LSI) and smoothness (Vempala and Wibisono, 2019).
This relaxation of conditionsis especially meaningful as the objective distribution nowadays tends to become more and morecomplicated beyond the classical assumption of log-concavity.
However, in the ﬁeld of machine learning, the main function can often be formulated as the averageof the loss function of an enormous number of training data points (Welling and Teh, 2011), whichsubsequently makes it difﬁcult to calculate its full gradient.
As a result, research on stochasticalgorithms has been also conducted to avoid this computational burden (Chen et al., 2021; Dubeyet al., 2016;
Raginsky et al., 2017; Welling and Teh, 2011; Xu et al., 2018;
Zou et al., 2018, 2019a,b,2021).
Welling and Teh (2011) introduced the concept of Stochastic Gradient Langevin Dynamics(SGLD) which combines the Stochastic Gradient Descent with LMC.
This has been the subject ofsuccessful studies (Raginsky et al., 2017; Welling and Teh, 2011; Xu et al., 2018).
Nevertheless, thevariance of its stochastic gradient is too large, which leads to a slow convergence compared to LMC.
Therefore, stochastic gradient Langevin Dynamics algorithms with variance reduction, such as the
Stochastic Variance Reduced Gradient Langevin Dynamics (SVRG-LD), have been considered andtheir convergence has been thoroughly analyzed for both sampling (Dubey et al., 2016; Zou et al.,2018, 2019a, 2021) and optimization (Huang and Becker, 2021; Xu et al., 2018).
Dubey et al.
ﬁrst united SGLD with variance reduction techniques and proposed two newalgorithms, namely, SVRG-LD and SAGA-LD.
Chatterji et al. (2018) and Zou et al.
(2018) provedthe convergence rate of SVRG-LD to the target distribution in 2-Wasserstein distance for smoothlog-concave distributions.
Xu et al.
(2018) showed the weak convergence of SVRG-LD under thesmoothness and dissipativity conditions.
They expanded the non-asymptotic analysis of Raginskyet al.
(2017) to LMC and SVRG-LD, and improved the result for SGLD.
Few years ago, Zou et al.(2019a) provided the gradient complexity of SVRG-LD to converge to the stationary distribution in2-Wasserstein distance under the smoothness and dissipativity assumptions.
This convergence can beeven improved if we make a warm-start (Zou et al., 2021).
While these works investigated algorithmswith ﬁxed hyperparameters, Huang and Becker (2021) additionally assumed a strict saddle and someother minor conditions to study SVRG-LD with a decreasing step size and improved its convergencein high probability to the second order stationary point.
Zou et al.
(2019b) also applied variancereduction techniques to the Hamiltonian Langevin Dynamics, or underdamped Langevin Dynamicsin opposition to GLD also known as overdamped Langevin Dynamics.
As we can observe, thecurrent convergence analyses of the stochastic schemes with variance reduction are mostly restrictedto log-concavity and dissipativity, and do not enjoy the same broad convergence guarantee with aconcrete gradient complexity as LMC does under LSI and smoothness in terms of KL-divergence.
Therefore, in order to bridge this theoretical gap between LMC and stochastic gradient Langevin
Dynamics with variance reduction, we study in this paper the convergence of the latter under therelaxed assumptions of smoothness and LSI.
In Section 3, we study the convergence to the Gibbsdistribution of SVRG-LD and the Stochastic Recursive Gradient Langevin Dynamics (SARAH-LD),another variant of stochastic gradient Langevin Dynamics with variance reduction inspired by the
Stochastic Recursive Gradient algorithm (SARAH) of Nguyen et al.
(2017a,b).
On the other hand,optimization and sampling are only two sides of the same coin for GLD.
That is why, in Section 4,we also investigate implications of Section 3 for non-convex optimization.
We prove the convergenceof SVRG-LD and SARAH-LD to the global minimum of dissipative functions and we provide theirnon-asymptotic rate of convergence.
We also consider the additional weak Morse assumption andstudy its effect.
1.2 Contributions
The major contributions of this paper can be summarized as follows.
We provide a non-asymptoticanalysis of the convergence of SVRG-LD and SARAH-LD to the Gibbs distribution in terms of
KL-divergence under smoothness and LSI which are weaker conditions than those used in prior worksfor these algorithms.
KL-divergence is generally a stronger convergence criterion than both totalvariation (TV) and 2-Wasserstein distance as they can be controlled by KL-divergence under the LSIcondition.
Notably, we prove that, with the batch size and inner loop length set to p n , the gradientcomplexity to achieve an ✏ -precision in terms of KL-divergence is ˜ O (( n + dn 1 / 2 ✏  1 )  2 L 2 ↵  2 ) ,which is better than any previous analyses.
See Table 1 for a comparison with previous research interms of assumptions, criterion and gradient complexity.
We also prove the convergence of SVRGLD and SARAH-LD to the global minimum under an additional assumption of dissipativity with agradient complexity of ˜ O
( n + n 1 / 2 ✏  1 dL ↵  1 )  2 L 2 ↵  2 ) which is better than previous work sinceit has almost all the time a dependence on n of O ( p n ) and does not require the batch size and theinner loop length to depend on the accuracy ✏ .
On the other hand, we import the idea of Li and
Erdogdu (2020) from product manifolds of spheres to the Euclidean space in order to show that under
[Table]
[Table]
This [Table] [Table] the additional assumption of weak Morse, the convergence in the Euclidean space can be acceleratedby eliminating the exponential dependence on 1 / ✏ .
1.3 Other Related Works
The theoretical study of GLD goes back to Chiang et al.
(1987) who showed that global convergencecould be achieved with a proper annealing schedule.
This work did not specify how to implementthis SDE, but Gelfand and Mitter (1991) ﬁlled this gap.
Later, Borkar and Mitter (1999) proved anasymptotic convergence in terms of relative entropy for the discrete scheme of gradient Langevin
Dynamics when the inverse temperature and the step size are kept constant.
The variance reduction technique, introduced to Langevin Dynamics by Dubey et al.
(2016), was originally presented by Johnson and Zhang (2013) as Stochastic Variance Reduced Gradient (SVRG) toimprove the convergence speed of Stochastic Gradient Descent.
Other variance reduction techniqueswere also considered such as the Stochastic Recursive Gradient Langevin Dynamics (SARAH) from
Nguyen et al.
(2017a,b) which outperforms SVRG in non-convex optimization (Pham et al., 2020)and is used in many algorithms such as SSRGD (Li, 2019) and SpiderBoost (Wang et al., 2019).
Li and Erdogdu (2020) extended Vempala and Wibisono’s result to Riemannian manifolds.
One ofthe highlights of their work is that they showed the Log-Sobolev constant of the Gibbs distributionfor a product manifold of spheres only depends on a polynomial of the inverse temperature undersome particular conditions including weak Morse.
We will adapt this result to our situation.
In the concurrent work of Balasubramanian et al. (2022) (especially Section 6), they also studiedthe convergence of stochastic schemes of GLD with more relaxed conditions than prior analyses.
However, our contributions are not overshadowed by theirs, and we clarify the reasons.
In Subsection6.1 of their paper, Balasubramanian et al.
(2022) focused on stochastic discrete schemes with ﬁnitevariance and bias (which is not the case for SVRG-LD) and provided a ﬁrst-order convergenceguarantee in the space of measures equipped with the 2-Wasserstein distance.
Subsection 6.2 proveda global convergence under some other conditions but most of these two analyses did not considerin particular the usual case in machine learning when F is the average of some other functions,which leads to a generally worse gradient complexity than ours.
Concerning this ﬁnite sum setting,
Balasubramanian et al.
(2022) investigated the Variance Reduced LMC
algorithm (slightly differentfrom SVRG-LD in this paper) in Subsection 6.3 and gave a ﬁrst-order convergence under the soleassumption of smoothness.
When restrained in our problem setting, the gradient complexity of
SVRG-LD and SARAH-LD
we provide is still considerably better (see Section 3 for more details).
1.4 Notation
We denote deterministic vectors by a lower case symbol (e.g., x ) and random variables by an uppercase symbol (e.g., X ).
The Euclidean norm is denoted by k · k for vectors and the inner productby h · , · i .
For matrices, k · k is the norm induced by the Euclidean norm for vectors.
We onlytreat distributions absolutely continuous with respect to the Lebesgue measure in R d for simplicity.
Especially, throughout the paper, ⌫ refers to the probability measure with the density
functiond ⌫ / e   F d x , where F is a function introduced below.
a _ b is equivalent to max { a, b } and a ^ bto min { a, b } .
We also use the shorthand ˜
O to hide logarithmic polynomials.
Preliminaries
In this section, we brieﬂy explain the problem setting, necessary mathematical background andassumptions used in this paper.
2.1 Problem Setting and GLD
In Section 3, we consider sampling from a distribution written in the form d ⌫ / e   F d x where  isa positive constant (which corresponds to the inverse temperature) and F : R d !
R is formulated as
F ( x ) :
= 1 n
P ni =
f i ( x ) , the average of the loss function of n training data points { x ( i ) } ni =1 .
Here,f i ( x ) :
= f ( x, x ( i ) ) can be regarded as the loss of data x ( i ) .
For instance, F can be the averageof the negative log likelihood of n training data points.
In Section 4, we consider the non-convexoptimization (minimization) of the same F as above.
GLD can be described as the following stochastic differential equation (SDE):
[Equation]
where  > 0 is called the inverse temperature parameter and { B ( t ) } t  0 is the standard Brownianmotion in R d .
It can be used for sampling since under some reasonable assumptions of F , thedistribution ⇢ GLD t of X GLD t governed by SDE (1) converges to the invariant stationary distributiond ⌫ / e   F d x , also known as the Gibbs distribution (Chiang et al., 1987).
Moreover, as previouslymentioned, this convergence is efﬁcient in the sense that SDE (1) corresponds to the steepest descentﬂow of the Kullback-Leibler (KL) divergence towards the stationary distribution in the space ofmeasures endowed with the 2-Wasserstein metric (Jordan et al., 1998).
Alternatively, GLD can beinterpreted as the composite optimization problem of a negative entropy and an expected functionvalue as follows (Wibisono, 2018):
[Equation]
The gradient ﬂow is the well-known Fokker-Planck equation associated to SDE (1):
[Equation]
This will be useful in our analysis.
In addition to its potential for sampling, GLD can also be employedfor non-convex optimization as the Gibbs distribution concentrates on the global minimum of F forsufﬁciently large values of  (Hwang, 1980).
2.2 Algorithms of GLD
Applying the Euler-Maruyama scheme to (1), we obtain the Langevin Monte Carlo (LMC)
[Equation]
where ⌘ is called the step size.
This is similar to the gradient descent except the additional Gaussiannoise p 2 ⌘ / ✏ k , where ✏ k ⇠ N (0 , I d ⇥ d ) and I d ⇥ d is the d ⇥ d unit matrix.
In the case n is huge andthe computation of r F is too difﬁcult, we are incited to use stochastic gradient methods in analogyto stochastic gradient optimization.
This gives
[Equation]
, inverse temperature
3 foreach s
/m ) do
randomly draw ✏ sm ⇠ N
0 , I d ⇥ d
foreach l m
1 do l randomly pick a subset I k
B randomly draw k
if
LD
else if then
end
end
19 end where v ( X k ) is the stochastic gradient.
When v ( X k ) is deﬁned as 1 B
P i k
I k r f i k
( X k ) , where 160 B is called the batch size
and I k is a random subset uniformly chosen from { 1 , . . . , n } such 161 that |
I k | =
B , we obtain the Stochastic Gradient Langevin Dynamics (SGLD).
As this methodexhibits a slow convergence, it has been popular to use variance reduction methods such asthe Stochastic Variance Reduced Gradient Langevin Dynamics (SVRG-LD) where v ( X k ) =
[Equation]
r f i k
( X ( s ) ))
+ r F ( X ( s ) ) .
Details of this algorithm is stated in Algorithm 1.
X ( s ) is a reference point updated every m steps so that X sm =
X ( s ) .
As we can observe in
Lemma A.4, around the optimal point, the variance of the stochastic gradient is indeed decreasedas X ( s ) and
X k are both close to each other.
We can also easily extend some successful stochasticgradient algorithms to Langevin Dynamics.
Hence, we are motivated to extend the Stochastic Recursive Gradient Algorithm (SARAH) to Langevin Dynamics since we can expect that some bottlenecksof the analysis of SVRG-LD can be removed in that of SARAH-LD as subtracting the previousstochastic gradient enables a stabler performance than SVRG-LD.
This algorithm can be described as
Algorithm 1 with v ( X k ) =
1 B
[Equation]
Deﬁnition 1.
We deﬁne ⇢ k as the distribution of X k generated at the k th step of SVRG-LD, andsimilarly  k for SARAH-LD.
2.3 Assumptions
The assumptions used throughout this paper can be summarized as follows.
Assumption 1.
For all i = 1 , . . . , n , r f i is twice differentiable, and 8 x, y 2 R d , kr 2 f i ( x ) k  L .
In other words, f i ( i = 1 , . . . , n ) and F are L -smooth.
Assumption 2.
Distribution ⌫ satisﬁes the Log-Sobolev inequality (LSI) with a constant ↵ .
That is,for all probability density functions ⇢ absolutely continuous with respect to ⌫ , the following holds:
[Equation]
where H ⌫ ( ⇢ ) :
= E ⇢ ⇥ log ⇢ ⌫ ⇤ is the KL-divergence of ⇢ with respect to ⌫ , and J ⌫ ( ⇢ ) :=
E ⇢ h [Equation]  i is the relative Fisher information of ⇢ with respect to ⌫ .
The recent work of Vempala and Wibisono (2019) motivates us to use the combination of smoothnessand LSI for the analysis of SVRG-LD and SARAH-LD.
Indeed, they showed that these conditions were enough to assure for the Euler-Maruyama scheme an exponentially fast convergence and a biascontrollable by the step size.
Under smoothness, LSI is not only the necessary condition of logconcavity and dissipativity, but is also robust to bounded perturbation and Lipschitz mapping, contraryto log-concavity (Vempala and Wibisono, 2019).
For example, for any distribution d ⌫ that satisﬁes
LSI and bounded function B : R d !
R , d˜ ⌫ / e B d ⌫ satisﬁes LSI as well (Holley and Stroock, 1986).
Moreover, while KL-divergence is not in general convex with regard to the Wasserstein geodesic,thanks to LSI, the Polyak- Ł ojaciewicz condition is satisﬁed.
It is well-known that LSI sufﬁces torealize an exponential convergence for the case of continuous time Langevin Dynamics (Vempalaand Wibisono, 2019).
That is why, it is actually both useful and natural to suppose LSI in this context.
Note that under L -smoothness of F and LSI with constant ↵ for d ⌫ / e   F d x , it holds that ↵   L(Vempala and Wibisono, 2019).
As for optimization, we additionally use the following conditions.
Assumption 3.
F is ( M, b ) -dissipative.
That is, there exist constants M > 0 and b > 0 such that forall x 2 R d
the following holds: hr F ( x ) , x i  M k x k 2  b .
Assumption 4 (Li and Erdogdu (2020), Assumption 3.3) .
F satisﬁes the weak Morse condition.
Thatis, for all non-zero eigenvalues of the Hessian of stationary points, there exists a constant  † 2 (0 , 1]such that
[Equation]
Furthermore, for the set S of stationary points that are not a global minimum,sup x 2 S  min
[Equation]
Assumption 5.
r 2 f i is L 0 -Lipschitz and without loss of generality, we let min x 2 R d F ( x ) = 0 .
Assumption 6.
F has a unique global minimum.
Smoothness and dissipativity are a classical combination of assumptions for this kind of problemsetting (Raginsky et al., 2017; Xu et al., 2018; Zou et al., 2019a).
We assume dissipativity instead of
LSI for non-convex optimization in order to obtain an explicit value of the Log-Sobolev constantof d ⌫ / e   F d x in function of the inverse temperature parameter  (see Property C.3), making anon-asymptotic analysis possible.
Furthermore, Assumptions 4 to 6 can ameliorate the exponentialdependence of the inverse of the Log-Sobolev constant on the inverse temperature parameter to apolynomial one (see Property C.4).
Main Results
In this section, we state our main results which prove that SVRG-LD and SARAH-LD (Algorithm 1)achieve an exponentially fast convergence to the Gibbs distribution and a controllable bias in termsof KL-divergence under the sole assumptions of LSI and smoothness.
We provide their gradientcomplexity as well.
The proofs can be found in Appendix A and B respectively.
3.1 Improved Convergence of SVRG-LD
Our analysis shows that the convergence of SVRG-LD to the stationary distribution d ⌫ / e   F d xcan be formulated as the theorem below.
Theorem 1.
Under Assumptions 1 and 2, 0 < ⌘ < ↵ 16 p 6 L 2 m  ,   1 and B  m , for allk = 1 , 2 , . . .
, the following holds in the update of SVRG-LD where ⌅ = ( n  B ) B ( n  1) :
[Equation]
2 m ⌅ ) .
We observe that the bias term of the upper bound, which is the second term linearly dependent on ⌘ ,can be easily controlled while the ﬁrst term exponentially converges to 0 with k ! 1 .
This is moreprecisely formulated in the following corollary.
Corollary 1.1.
Under the same assumptions as Theorem 1, for all ✏  0 , if we choose step size⌘ such that ⌘  3 ↵✏ 448  dL 2 , then a precision H ⌫ ( ⇢ k )  ✏ is reached after k  ↵⌘ log 2 H ⌫ ( ⇢ 0 ) ✏ steps.
Especially, if we take B = m = p n and the largest permissible step size ⌘ =
[Equation]
then the gradient complexity becomes
[Equation]
This gradient complexity is an improvement compared with prior works for three reasons.
First of all,we provide a non-asymptotic analysis of the convergence of SVRG-LD under smoothness and LogSobolev inequality which are conditions weaker than those (e.g., log-concavity or dissipativity) used inprior works for these algorithms.
Moreover, we prove it in terms of KL-divergence which is generallya stronger convergence criterion than both total variation (TV) and 2-Wasserstein distance as they canboth be controlled by KL-divergence under the LSI condition.
For instance, TV was used by Zou et al.(2021) and 2-Wasserstein distance by Dalalyan (2017a) and Zou et al. (2019a).
KL-divergence makesit possible to unify these two different criteria.
Finally, while prior research generally used Girsanov’stheorem which generates a bias term that accumulates through the iteration (see for example Raginskyet al.
(2017) and Xu et al.
, we solve this issue by taking beneﬁt of the exponential convergenceof GLD to the Gibbs distribution under LSI and smoothness that enables us to forget about pastbias.
That way, with the batch size and inner loop set to p n , the gradient complexity to achievean ✏ -precision in terms of KL-divergence becomes ˜ O
(( n + dn 1 / 2 ✏  1 )  2 L 2 ↵  2 ) , which is betterthan previous analyses.
For example, Vempala and Wibisono (2019) provided a gradient complexityof ˜
O
n ✏  1 d  2 L 2 ↵  for LMC under Assumptions 1 and 2, and Zou et al. (2019a) a gradientcomplexity of ˜ O ( n + n 3 / 4 ✏  2 + n 1 / 2 ✏  4 ) ·
e ˜ O
(  + d ) for SVRG-LD under Assumptions 1 and 3.
Notethat the dependence on the dimension d is not improved since ↵  1 may exponentially depend on d .
Recently, Zou et al.
(2019b) proposed the Stochastic Gradient Hamiltonian Monte Carlo Methodswith Recursive Variance Reduction with a gradient complexity of ˜ O
( n + n 1 / 2 ✏  2 µ  3 / 2 ⇤ )
^ µ  2 ⇤ ✏  4 ) 249 in terms of 2-Wasserstein distance.
Even though their algorithm is based on the underdamped
Langevin Dynamics whose discrete schemes use to perform better than those of the overdamped
Langevin Dynamics such as SVRG-LD, our gradient complexity, which applies to a broader familyof distributions, is almost the same except for a small interval of ✏ , but we do not require the batchsize B and the inner loop length m to depend on ✏ while Zou et al.
(2019b) do, i.e., B .
B 1 / 2 0 ,m = O ( B 0 /B ) , where B 0 = ˜
O [Equation] ^
[Equation]
.
This strengthens the importance of our result sinceit shows that adapting this analysis to other stochastic schemes of GLD is promising and could leadto tighter bounds and relaxation of conditions.
See Table 1 for a summary.
Concerning the concurrentwork of Balasubramanian et al.
, under the sole assumption of smoothness, they provideda gradient complexity of O
( L 2 d 2 n/ ✏ 2 ) for the Variance Reduced LMC algorithm that updates thestochastic gradient differently as SVRG-LD and SARAH-LD.
This is almost the square of our result,and in some extent, our work can be interpreted as an acceleration of their result with a slightlystronger additional condition than Poincaré inequality.
Proof Sketch Proceeding in a similar way as Vempala and Wibisono (2019), we evaluate how
H ⌫ ( ⇢ k ) decreases at each step as shown in Theorem A.1 of Appendix A.
This is realized by comparingthe evolution of the continuous-time GLD for time ⌘ and one step of SVRG-LD.
Since we use astochastic gradient, we need at the same time to evaluate the variance of the stochastic gradient.
Theorem 1 can be obtained by recursively solving the inequality derived in Theorem A.1.
3.2 Convergence Analysis of SARAH-LD
As for SARAH-LD, its convergence to the stationary distribution d ⌫ / e   F d x can be formulatedas the theorem below.
Interestingly, we obtain the same result as SVRG-LD (Theorem 1) but we donot require B  m anymore.
Theorem 2.
Under Assumptions 1 and 2, 0 < ⌘ < ↵ 16 p 2 L 2 m  and   1 , for all k = 1 , 2 , . . .
, thefollowing holds in the update of SARAH-LD where ⌅ = ( n  B ) B ( n  1) :
[Equation]
This is the ﬁrst convergence guarantee of SARAH-LD in this problem setting so far, and it leads tothe following gradient complexity.
Corollary 2.1.
Under the same assumptions as Theorem 2, for all ✏  0 , if we choose stepsize ⌘ such that ⌘ 
[Equation]
1 , then a precision H ⌫ (  k )  ✏ is reached afterk
[Equation]
log
[Equation]
steps.
Especially, if we take B = m = p n and the largest permissible step size⌘ =
[Equation]
2  , then the gradient complexity becomes
[Equation]
The reason why we obtain the same gradient complexity for both SARAH-LD and SVRG-LD (exceptbetter coefﬁcients for SARAH-LD) is that in our analysis, the Brownian noise added at each stepof the Langevin Dynamics plays the role of a fundamental bottleneck that even SARAH-LD couldnot eliminate, and we still need to set B = m = p n .
We can hypothesize that this order of gradientcomplexity might be tight for variance-reduced stochastic gradient Langevin Dynamics algorithms.
Some Applications to Non-Convex Optimization
Here, we apply our main results to non-convex optimization.
Thanks to our analysis applicable to abroader family of probability distributions satisfying LSI, the additional conditions we pose in thissection are mainly reﬂected in the concrete formulation of the Log-Sobolev constant, which keeps ourstudy simple and clear.
The proofs can be found in Appendix C. Since SVRG-LD and SARAH-LDexhibited almost the same performance in sampling, we can simultaneously analyse them.
We ﬁrstprove the convergence to the global minimum of SVRG-LD and SARAH-LD without clarifying theexplicit formulation of the Log-Sobolev constant in function of  .
Theorem 3.
Using SVRG-LD or SARAH-LD, under Assumptions 1 to 3, 0 < ⌘ < [Equation] [Equation] L 2 m  ,
[Equation]
[Equation]
[Equation]
M and B  m , if we take B = m = p n and the largest permissiblestep size ⌘ = ↵ 16 p
[Equation] 2 p n [Equation] ^ 3 1792 ↵
[Equation]
L 2 d  , the gradient complexity to reach a precision of
[Equation]
is
[Equation]
where ↵ is a function of  , and X ⇤ is the global minimum of F .
Remark 1.
Under Assumptions 1 and 3, Assumption 2 is negligible as shown in Property C.2.
Under Assumptions 1 to 3 only, this leads to a gradient complexity which exponentially depends onthe inverse of the precision level ✏ as shown in the next corollary since the inverse of the Log-Sobolevconstant exponentially depends on  .
Corollary 3.1.
Under the same assumptions as Theorem 3, taking  = i ( ✏ ) : = 4 d ✏ log [Equation]
1 _ 2 M , we obtain a gradient complexity of
[Equation]
since ↵ =  C 1 e  C 2  (Property C.3).
The second term with n 1 / 2 is almost all the time dominant since it has a factor that exponentiallydepends on 1 / ✏ and the ﬁrst term not.
This dependence on n of O ( n 1 / 2 ) is the best so far for thesealgorithms.
Moreover, comparing with the gradient complexity ˜ O  n 1 / 2   4 ✏  5 / ·
e ˜ O ( d ) , also oforder n 1 / 2 , provided by Xu et al.
(2018) who used SVRG-LD and the same assumptions, our gradientcomplexity is an improvement since their analysis required a batch size B and an inner loop lengthm that strongly depend on ✏ (i.e., B = p n ✏  3 / 2 , m = p n ✏ 3 / 2 ) and ours does not.
Note that thedependence of the gradient complexity of Xu et al.
(2018) on 1 / ✏ is not necessarily better than oursas  is actually the spectral gap of the discrete-time Markov chain generated by (1) and its inverse exponentially depends on 1 / ✏ as well.
Although Xu et al.
(2018) did not investigate the explicitnature of  , this is supported by Raginsky et al. (2017) who proved this exponential dependence forthe spectral gap of the continuous-time SDE and by Mattingly et al.
(2002) who showed the spectralgap of continuous-time SDE and that of discrete-time version are almost the same in this context.
Analysis under the weak Morse condition Now, under the additional Assumptions 4 to 6, it isinteresting to note that a polynomial dependence on 1 / ✏ is achieved as the following corollary shows.
Corollary 3.2.
Under the same assumptions as Theorem 3 and Assumptions 4 to 6, taking  =j ( ✏ ) :
=
4 d ✏ log
[Equation]
2 _ 1 _ 2 M _ C  , where C  is a constant independent of ✏ deﬁned in Property 320 C.4, we obtain a gradient complexity of
[Equation]
since ↵ = C 3 /  (Property C.4).
The crux of this corollary is Property C.4.
To prove this, we show like Li and Erdogdu (2020) that⌫ satisﬁes the Poincaré inequality with a constant independent of  .
Since it is not hard to showthis around the global minimum, we can step by step extend the set where this inequality holds by a
Lyapunov argument (Theorems D.1 and D.2).
The essential difference between this analysis and thatof Li and Erdogdu (2020) is that we do not work on compact manifolds anymore.
Some rather minordifﬁculties emerge as we cannot employ the compactness but they can be addressed by supposingdissipativity which assures a quadratic growth for large x .
Remark 2.
These results do not deﬁnitively assert that SARAH-LD and SVRG-LD show the exactsame performance in terms of optimization.
Indeed, suppose we are close enough to the globaloptimum.
Then, a big noise is not necessary anymore since it is more important to stably converge tothe global minimum.
Here, we should be able to signiﬁcantly decrease the noise ✏ k , and the bottleneckfrom the noise should disappear.
In this case, SARAH-LD would perform better than SVRG-LD as weapproach the original non-convex optimization setting where SARAH outperforms SVRG.
Remark 3.
We also investigated an annealed version of SVRG-LD and SARAH-LD but could notameliorate the gradient complexity.
The detailed analysis can be found in Appendix E.
Discussion and Conclusion
The main limitations of our work reside in the gap between practice and theory. Indeed, while ourpaper supposes assumptions quite standard in the literature of GLD, it cannot explain the wholeempirical success that machine learning is currently experiencing. Some choices of parameters mayalso seem different than the practical use.
However, compared to previous work, we succeeded inthe proving convergence of GLD with the popular stochastic gradient with relaxed conditions, anddeleting the dependence of batch size and inner loop length on epsilon, which are all more realisticsituations than prior work.
The theoretical study in machine learning and deep learning preciselyplays the role of ﬁlling as much as possible this large gap, and our work could be regarded as a furtherstep forward to achieve this goal.
Furthermore, in this paper, we focused on the pure sampling andoptimization performance of the algorithms, and some of the drawbacks are simply due to this fact.
For example, another limitation is that we did not investigate the generalization error in Section 4,but this was only outside the scope of this work.
In conclusion, we analysed the convergence rate of stochastic gradient Langevin Dynamics withvariance reduction under smoothness and LSI and its application to optimization.
In Section 3, weproved the convergence of SVRG-LD in terms of KL-divergence with more relaxed conditions (LSIand smoothness) and with a better gradient complexity than previous works.
We also expanded
SARAH to SARAH-LD and showed that this algorithm enjoyed the same advantages as SVRG-LDwith only an improvement in the coefﬁcients of the gradient complexity.
These results led us toapply SVRG-LD and SARAH-LD to non-convex optimization in Section 4.
We provided the globalconvergence and a non-asymptotic analysis of SVRG-LD and SARAH-LD.
We obtained betterconditions than prior works.
Furthermore, we showed that under the additional assumption includingweak Morse and Hessian Lipschitzness, the gradient complexity could be ameliorated, eliminatingthe exponential dependence on the inverse of the required error.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’scontributions and scope?
[Yes](b) Did you describe the limitations of your work?
[Yes](c)
Did you discuss any potential negative societal impacts of your work?
[Yes]
(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] All the
assumptions are summarized in Subsection 2.3 and refered when used.
(b) Did you include complete proofs of all theoretical results?
[Yes] See the appendices.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[N/A]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[N/A]
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)?
[N/A]
(d) Did you include the total amount of compute and the type of resources used (e.g., type
of GPUs, internal cluster, or cloud provider)?
[N/A]
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[N/A]
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots
, if
applicable?
[N/A
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]