L OCAL KL C ONVERGENCE R ATE FOR S TEIN V ARIATIONAL G RADIENT D ESCENT WITH R EWEIGHTED K ERNEL
Anonymous authors
Paper under double-blind review
A BSTRACT
We study the convergence properties of Stein Variational Gradient Descent (SVGD) algorithm for sampling from a non-normalized probabilistic distribution p ∗ ( x ) ∝ exp( − f ∗ ( x )) .
Compared with Kernelized Stein Discrepancy (KSD) convergence analyzed in previous literature, KL convergence as a more convincing criterion can better explain the effectiveness of SVGD in real-world applications.
In the population limit, SVGD performs smoothed gradient descent with kernel integral operator.
Notably, SVGD with smoothing kernels suffers from gradient vanishing in low-density areas, which makes the error term between smoothed gradient and the Wasserstein gradient not controllable.
In this context, we introduce a reweighted kernel to amplify the smoothed gradient in low-density areas, which leads to a bounded error term.
When the p ∗ ( x ) satisfies log-Sobolev inequality, we develop the convergence rate for SVGD in KL divergence with the reweighted kernel.
Our analysis points out the defects of conventional smoothing kernels in SVGD and provides the convergence rate for SVGD in KL divergence.
I NTRODUCTION
Sampling from non-normalized distributions is a crucial task in statistics.
In particular, in Bayesian inference, Markov Chain Monte Carlo (MCMC) and Variational Inference (VI) are considered two mainstream lines to handle the intractable integration of posterior distributions.
On the one hand, although methods based on MCMC, e.g., Langevin Monte Carlo (Durmus & Moulines, 2019; Welling & Teh, 2011) (LMC) and Metropolis-adjusted Langevin algorithm (Xifara et al., 2014) (MALA), are able to provide approximate target distributions with arbitrarily small error (Wibisono, 2018), the sample efficiency is low due to the lack of repulsive force between samples (Duncan et al., 2019;
Korba et al., 2020).
On the other hand, VI-based sampling methods (Blei et al., 2017; Ranganath et al., 2014) can improve the sampling efficiency by reformulating inference as an optimization problem.
However, restricting the search space of the optimization problem to some parametric distributions in VI usually causes a huge gap between its solution and the target distribution p ∗ .
Inspired by conventional VI, a series of recent works analyze LMC as the optimization problem of
Kullback-Leibler (KL) divergence (Wibisono, 2018; Bernton, 2018;
Durmus et al., 2019), i.e.,
[Equation]
where P 2 ( R d ) is the set of Radon-Nikodym derivatives of probability measures ν over Lebesgue measure such that p ( x )
=
d ν ( x ) / d x , (cid:82) ∥ x ∥ 2 p ( x )d x < ∞ .
LMC is considered as a discrete scheme of the gradient flow of the relative entropy by driving particles with stochastic and energy-induced force.
Besides, to take the best of both MCMC and VI, Stein Variational Gradient Descent (Liu &
Wang, 2016) (SVGD) was proposed as a non-parametric VI method.
It replaces the stochastic force in LMC with the interaction between particles and approximates the target distribution by a driving force in Reproducing Kernel Hilbert space (RKHS).
It means the gradient flow of SVGD is defined by the functional derivative projection of Eq.
1 to RKHS.
The empirical performance of SVGD and its variants have been largely demonstrated in various tasks such as learning deep probabilistic models (Liu & Wang, 2016;
Pu et al., 2017), Bayesian inference (Liu & Wang, 2016;
Feng et al., 2017; Detommaso et al., 2018), and reinforcement learning (Liu et al., 2017).
In addition to rich applications, there is a lot of work on the theoretical analysis of SVGD.
For example, Kernelized Stein Discrepancy (KSD) convergence properties of SVGD under asymptotic and non-asymptotic settings are investigated by Liu (2017);
Lu et al.
and
Korba et al.
Salim et al.
(2021; 2022), respectively.
However, different from the convergence of KL divergence in the analysis of LMC (Cheng & Bartlett, 2018; Vempala & Wibisono, 2019), KSD convergence cannot deduce the effectiveness of SVGD in some real-world applications, e.g., posterior sampling (Welling & Teh, 2011) and non-convex learning (Raginsky et al., 2017).
Then, to provide KL convergence, some other works (Duncan et al., 2019; Korba et al., 2020) present a linear convergence of SVGD with Stein log-Sobolev inequality (SLSI)
(Duncan et al., 2019).
Nonetheless, different from the clear meaning and criteria of standard log-Sobolev inequality (LSI) in the analysis of LMC (Vempala & Wibisono, 2019), the establishment of SLSI requires the property of the coupling of designed smoothing kernels and the target distribution, which can hardly be verified in commonly used kernels (Duncan et al., 2019).
In addition, SLSI in higher dimensions is more challenging to hold.
To fill these gaps, in this paper, we aim to provide the convergence rate of SVGD (in the infinite particle regime) in terms of KL objective, when p ∗ =
e − f ∗ satisfies standard LSI.
Specifically, we first point out that the SVGD with smoothing kernel, e.g., RBF kernel, suffers from gradient vanishing in low-density areas due to the extra p t ( x ) scaling.
Then, we denote the importance of reweighted kernels by dividing p t ( x ) or p ∗ ( x ) , where the scaling of smoothed gradients can be normalized.
With the reweighting scaling p − 1 / 2 ∗
[Equation]
( y ) for kernel k ( x , y ) and regularity conditions, SLSI in higher dimensions can be nearly established with an additional term controlled by kernel approximation error.
Finally, by choosing a proper reweighted smoothing kernel, the KL divergence of SVGD dynamics obtains a local linear convergence rate to any neighborhood of p ∗ ( x ) under mild assumptions when the initialization p 0 ( x ) is relatively close to p ∗ ( x ) .
The main contributions of the paper are as follows:
• We introduce reweighted kernels to SVGD which replaces traditional smoothing kernels and overcomes the gradient vanishing problem in low-density areas.
• We study the KL convergence rate of SVGD algorithm.
Under the standard LSI and some mild assumptions, we show SVGD with a reweighted kernel has a local linear convergence rate to any neighborhood of p ∗ ( x ) .
P RELIMINARIES
In this section, we first introduce important notations used in the following sections.
Then, we explain how to optimize functionals on Wasserstein space by continuous updates in the infinite particle regime.
After that, we show that the key condition LSI on the target distribution to obtain the KL convergence rate of LMC.
However, the convergence rate of SVGD dynamics is non-trivial with this assumption.
Notations.
In following sections, bold letters x , y , z denote vectors in R d , and B ( x , r ) means the open ball centered at x with radius r > 0 .
For function f : R d → R , ∇ f ( · ) and ∇ 2 f ( · ) refer to its gradient and Hessian matrix respectively.
For function f : R d → R d , ∇ f ( · ) and ∇ · f ( · ) present
the
Jacobian matrix and divergence.
For function with multiple variables, ∇ i means the gradient w.r.t i -th variable.
The distributions are assumed to be absolutely continuous with respect to the Lebesgue measure, which produces density function p .
The probability density function of the target posterior is denoted by p ∗ .
The density at time t is p t .
Notation ∥ · ∥ denotes 2-norm for both vectors and matrix.
In Hilbert space H equipped with the inner product ⟨· , ·⟩ H , the norm is induced as ∥ · ∥ H .
The set P 2 ( X ) is consist of probability measure µ on X with finite second order moment.
˜
k denotes a function smoother, such as exp( −∥ x ∥ 2 ) , max { 0 , 1 − ∥ x ∥} .
2. O PTIMIZATION IN THE W ASSERSTEIN S PACE
Sampling algorithms can be considered as optimizing some given functionals in the Wasserstein space as Eq.
Generally, they only update particles, which causes the evolution of the particles’ distribution.
Such an evolution finally affects the objective functional.
In particular, given initial distribution x 0 ∼ p 0 ( x ) and function class H , suppose the update of x t is
[Equation]
[Equation]
where ϕ t : R d → R d ∈ H .
By the continuity equation, we have the differential equation for p t ( x ) ,
[Equation]
For any suitable functional F w.r.t.
p t , its evolution can be presented as
[Equation]
where δ F /δp denotes the L 2 ( R d ) -functional derivative (Villani, 2009;
Duncan et al., 2019).
Assuming that F = H p ∗ , we have
Proposition 2.1.
The evolution of KL divergence with Eq.
2 is
[Equation]
Proposition 2.1 is a direct result of the functional derivative for KL divergence, where δ F /δp
= ln p + 1 + ln p ∗ (Chapter 15 of Villani (2009)).
By choosing ϕ t which decreases KL divergence via
Eq.
3, we can optimize p t to approach p ∗ .
2. L OG -S OBOLEV INEQUALITY
In the Wasserstein space optimization literature, LSI is particularly crucial to obtaining the convergence rate, which is an analogue to Polyak-Lojasiewicz (PL) inequality in Euclidean space.
LSI applies to a wider class of measures than log-concave distributions and can be checked by BakryEmery criterion Bakry & Émery (1985).
Specifically, bounded perturbation and Lipschitz mapping can preserve the establishment of LSI Vempala & Wibisono (2019), where log-concavity would be failed.
For example, subtracting some small Gaussians from a strongly log-concave distribution will destroy the log-concavity of the original distribution, while it still satisfies LSI as long as the
Gaussians we subtract are small enough.
When the target distribution p ∗ satisfies µ -LSI, it denotes
[Equation]
for any differentiable function g ∈ L 2 ( ν ) .
Such an inequality usually provides some connection between the sufficient descent of functional evolution and its exact values.
Coupling log-Sobolev inequality with Langevin dynamics.
In particular, the most popular algorithm, Langevin dynamics Vempala & Wibisono (2019) chooses
[Equation]
to decrease the KL functional (Eq. 3) which is equivalent to the particles’ update,
[Equation]
where B t is standard Brownian motion.
The introduction of randomness can also convert ∇ ln p t ( x ) to a tractable form and the dynamics becomes
[Equation]
where the absolute value of RHS in Eq.
6 is called relative Fisher information.
Taking g 2 =
p t /p ∗ , we have
[Equation] [Equation]
Then LSI provides a lower bound for gradient norm, leading to sufficient descent for KL divergence.
Combining Eq.
7 and Eq.
6, we have
[Equation]
for some µ > 0 .
By applying Gronwall’s lemma, Eq. 8 yields H p ∗ ( p t )
≤ H p ∗ ( p 0 ) exp( − 2 µt ) , which indicates the linear convergence rate of Langevin dynamics.
Nonetheless, coupling LSI with
SVGD is challenging due to the introduction of RKHS.
KL divergence descent with asymptotic SVGD.
As shown in Lemma 3.2 of Liu & Wang (2016), the key point of continuous SVGD is to minimize the RHS of Eq.
3 in the unit ball of the RKHS H as follows
[Equation]
such that ∥
[Equation]
where S ( p t , p ∗ ) is called Kernelized Stein’s discrepancy (KSD).
Assume that the RKHS is associated with a kernel k ( x , y ) :
R d × R d → R , such that k ( x , y ) = ⟨ Φ( x ) , Φ( y ) ⟩ H 0 , and H is the d -times
Cartesian product space of H 0 that contains Frobenius-normalized linear functions from H to R d , i.e., ⟨ ϕ t , ϕ t ⟩ H ≤ 1 .
Note that Eq.
10 defines a functional gradient in RKHS, which indicates the steepest KL decreasing direction with a normalized functional vector.
Using integration by parts,
Eq.
9 can be rewritten as
[Equation]
such that ∥ ϕ ∥
[Equation]
[Equation]
The explicit form for ϕ t ( x ) is presented as follows.
Proposition 2.2.
Assume that ϕ t satisfies Eq. 10.
Then we have
[Equation]
where ϕ t ( x ) can be estimated by particle samples from p t .
Proposition 2.2 (as Lemma 3.2 proved in Liu & Wang (2016)) makes Eq.
2 become a practically tractable algorithm by Monte Carlo estimation of Eq.
11, where particle samples are from p t ( y ) .
Note that Eq. 2 and Eq.
11 naturally lead to the algorithm of SVGD.
Combining Proposition 2.1 and 2.2, the dissipation of the KL divergence along continuous SVGD can be obtained,
[Equation]
When the kernel is strictly positive definite, the RHS of Eq.
12 is negative, leading to the decrease of
KL divergence.
Unlike the sufficient descent bounded by the functional value in Eq.
7, LSI cannot be conducted on the RHS of Eq.
12 due to the kernelization, which also causes the KL convergence rate to be unknown.
Instead, previous works (Liu, 2017; Lu et al., 2019) provided an O (1 /t )
KSD convergence as
[Equation]
KL CONVERGENCE OF SVGD
In this section, we first show, compared with KSD convergence proved in most previous works on
SVGD Liu (2017);
Lu et al.
; Korba et al.
; Salim et al.
(2021; 2022), KL convergence is more powerful in explaining the practical performance of real-world applications.
After that, we explain Stein log-Sobolev Inequality (SLSI) the necessary condition for analyzing KL convergence of SVGD can hardly be verified.
In order to investigate more reasonable conditions, our assumptions are proposed and validated empirically in some simple cases.
Finally, we provide the main theorem, i.e., the KL convergence of SVGD under these mild assumptions.
Due to the page limit, we left the comparison with previous works by list in Appendix A.
3. S AMPLING TASKS REQUIRE KL CONVERGENCE
Sampling algorithms are widely used in real-world applications for solving corresponding machine learning problems.
Although different tasks usually require different criteria for convergence analysis, most of these criteria can be deduced by KL convergence.
In Bayesian learning Welling & Teh (2011), people expect to capture parameter uncertainty via
Markov chain Monte Carlo (MCMC) techniques.
Specifically, they prefer to sample the parameter vector w from a posterior distribution presented as
[Equation]
where z := { z i } Ni =1 denotes a given dataset.
The update of w in (Neal et al., 2011)
[Equation]
is equivalent to minimizing the KL divergence w.r.t between p ( w | z ) and p ∗ ( w | z ) where
[Equation]
It means the convergence of Bayesian learning is directly dependent on KL convergence.
Another important application of sampling algorithms is to minimize the expected excess risk as
[Equation]
(15) where F denotes the objective function of the stochastic optimization under the unknown data distribution P
[Equation]
and F ∗ denotes inf w ∈ R d F ( w ) .
When F is L -smooth, previous gradient-based MCMC methods would like to analyze the convergence with the general framework by
[Equation]
where the n -tuple (data) z = { z 1 , z 2 , . . .
, z n } of i.i.d.
samples are drawn from P .
It means the minimization of Wasserstein 2 distance between ˜ p t, z (MCMC samples at time t ) and p ∗ , z ( x ) ∝ p ( w | z ) leads to convergence of expected excess risk.
The aforementioned results can be directly deduced by the KL convergence (Raginsky et al., 2017; Xu et al., 2018) with
[Equation]
Unfortunately, the connection between W 2 (˜ p k, z , p ∗ , z ) and S (˜ p k, z , p ∗ , z ) depends on the choice of
RKHS, which is highly specialized and non-general.
From a theoretical perspective, when the RKHS is over-smooth with a sufficient large bandwidth, k ( x , y ) = σ − d exp( −∥ x − y ∥ 2 / 2 σ 2 ) with large σ , the corresponding kernelized gradient tend to diminish, i.e., lim σ
[Equation]
y [Equation]
[Equation]
That means the KSD can be arbitrarily small with improper RKHS choices.
In this condition, the convergence of KSD does not make much sense about the quality of p t , since it can be simply controlled by some special RKHS.
3. KL CONVERGENCE OF SVGD WITH DIFFERENT ASSUMPTIONS
To investigate KL convergence of SVGD, some previous works (Duncan et al., 2019; Korba et al., 2020; Salim et al., 2021) introduce the following assumption.
Assumption 1.
The probability density p ∗ satisfies Stein log-Sobolev inequality (SLSI) with a constant µ > 0 , if for any p t ∈ P 2 ( R d ) , it has
[Equation]
We can immediately obtain
[Equation]
and a linear KL convergence can be achieved by combining Eq.
12 and Eq.
18 for SVGD
(Duncan et al., 2019; Korba et al., 2020; Salim et al., 2021).
However, the verification of this assumption is highly non-trivial because we cannot test all p t ∈ P 2 ( R d ) .
Only when the designed RKHS is overly regular, the RHS of Eq.
18 can be estimated.
In the meanwhile, an overly regular kernel, i.e.,
[Equation]
will make SLSI fail, which is indicated in (Duncan et al., 2019).
Besides, Eq. 19 holds for the most widely used smoothing kernels, such as Radial basis function (RBF) kernel.
The contradiction between Eq.
18 and Eq.
19 makes the current analysis in KL divergence highly restricted.
In this condition, we expect more reasonable assumptions to investigate KL convergence of SVGD.
Similar to (Arbel et al., 2019), we have additional assumptions on trajectory of p t .
Specifically, we assume the following.
[A 1 ] p ∗ satisfies µ -log-Sobolev Inequality (Eq. 4) and f ∗ is L -smooth, i.e., for any x , y ∈ R d , ∥∇ f ∗ ( x ) − ∇ f ∗ ( y ) ∥
≤ L ∥ x − y ∥ .
[A 2 ]
f t is L -smooth where p t =
e − f t .
[A 3 ] p t is warm : sup x ∈ R d p t ( x ) /
p
∗ ( x ) ≤ β
for some constant
β ≥ Assumption [A 1 ] and [A 2 ] are similar to the convexity geometry and L -smoothness in conventional
Euclidean optimization.
Assumption [A 3 ] restricts the domain of our proof: the tail of p t should be lighter than p ∗ , which is widely used in Langevin dynamics.
Compared with SLSI, due to the decoupling of requirements of the target distribution p ∗ and designed kernels k , we can verify these assumptions in several ways.
The establishment of LSI of the target distribution p ∗ can be checked by the criterion mentioned in Section 2.2.
For the trajectory assumptions, i.e., [A 2 ] and [A 3 ] , we provide the empirical validation by showing the estimation of density ratio and smoothness of p t in some simple cases with the growth of t (Fig 1).
Then, we have the following theorem.
Theorem 3.1.
Suppose Assumption [A 1 ] - [A 3 ] are satisfied, and
chi-square D χ ( p 0 , p ∗ ) ≤ 1 / 4 .
For any ϵ > 0 , if we set reweighted kernel k :
[Equation]
, and
[Equation]
where σ [Equation]
[Equation]
C = (cid:82) (cid:112) p ∗ ( x )d x , then the KL divergence between p t and p ∗ satisfies
[Equation]
Remark 1.
Although we require the trajectory of the algorithm to satisfy some assumptions, the actual requirements are much looser.
For example, we allow the coefficients L of smoothness in
Assumption [A 2 ] and the maximum density ratio β in Assumption [A 3 ] to increase with the number of iteration t growth.
Even if the rate of growth is exponential, the linear convergence can still be preserved if a reweighted kernel with decreasing σ t is utilized to replace the static σ in Eq. 21.
Assumption [A 3 ] is actually introduced to control Rényi divergence between p t and p ∗ will not be infinity in some region near the target, which can be easily obtained in Langevin dynamics.
[Figure]
This theorem demonstrates that, by introducing reweighted kernel k ( x , y ) and controlling the variance σ of smoothing kernel ˜ k , SVGD initialized in a local region will provide a linear convergence rate to any ϵ -neighborhood of the target distribution.
To validate this result, we provide experiments in synthetic data in Appendix D, and illustrate SVGD with reweighted kernels usually achieves a lower KL divergence compared with traditional SVGD.
It should be noticed that commonly used kernels, e.g., RBF kernel and Bump kernel, are proper ˜ k .
Besides, the linear convergence shows all the parameters will not deteriorate the convergence of SVGD when σ is small enough.
R EWEIGHTED KERNEL FOR KL CONVERGENCE
In this section, we mainly explain why we should introduce reweighted kernels in Theorem 3.1.
The intuition can be split into 2 parts: (1) the infeasibility of the usage of LSI due to the kernel approximation error; (2) the tractable kernel approximation error form with a reweighted kernel.
4. K ERNEL A PPROXIMATION
Intuitively, to measure the error between Wasserstein gradient and its kernelized one, the most direct idea is to control the error of relative Fisher information and make use of LSI.
However, this idea will encounter some fatal bottlenecks.
The bottleneck of SVGD analysis with Eq.
7 If we directly upper bound the descent of KL divergence by Eq.
7, we have
[Equation]
where the kernel approximation error can hardly be upper bounded due to p t ( y ) in integration.
It means if we directly plug some smoothing kernels into the iteration of SVGD, kernel approximate error may dominate RHS of Eq. 23 and cause SVGD to converge to a limit different from the target distribution with an uncontrollable bias.
Failures of Smoothing Kernels.
Smoothing Kernels (kernel smoothers), such as radial basis function kernel, are widely used in SVGD, due to their universal approximation capability to smooth functions (Park & Sandberg, 1991; Micchelli et al., 2006).
Assume that k σ ( x , y ) =
˜
k
σ ( x − y )
= σ − d ˜ k
( σ − 1 ( x − y )) is a smoothing kernel with parameter σ > 0 , where σ is called the bandwidth of the kernel.
The variance σ 2 in the smoothing kernel tends to control the smoothness of the estimated gradient.
A large σ makes kernelized gradient well-estimated with finite samples while the kernel approximation error is large.
In the population limit, where randomness from p t ( y ) is ignored, the optimal smoothing kernel should be Dirac delta function δ x ( y ) , where the kernel bandwidth is sufficiently small to estimate Wasserstein gradient.
For each point x , the kernelized gradient becomes p t ( x ) ∇ ln p ∗ ( x ) p t ( x ) , which means that for those low-density areas p t ( x ) → 0 , the kernelization suffers
[Figure]
from gradient vanishing.
Such biased Wasserstein gradient estimation in smoothing kernel SVGD will have an additional p t term, which hampers the sufficient descent of each iteration and even the convergence rate.
This indicates that SVGD is not compatible with smoothing kernels in low-density parts.
Ideally, the kernel should be reweighted by some 1 / (cid:112) p t ( x ) p t ( y ) , which can balance the vanishing scaling in the current SVGD.
However, p t is unknown in general, so this reweighting cannot provide algorithmic insight to improve SVGD.
To solve this problem, a very intuitive idea is to balance the order of p t , we may require kernel k to be related to p t through the following proposition.
Proposition 4.1.
If we use the kernel k ( x , y )
= ( p t ( x )) − 1 / 2 k σ ( x , y )
( p t ( y ))
− 1 / 2 , by choosing delta function as the smoothing kernel k 0 ( x , y )
=
δ x ( y ) , kernel approximation error is 0 and SVGD is equivalent to the Wasserstein gradient flow.
Unfortunately, the reweighting strategy with p t ( x ) makes the iteration of SVGD (Eq. 11) computationally intractable as p t ( x ) and ∇ p t ( x ) are unknown in general.
If we consider the local convergence of SVGD, when p t ( x ) is approaching p ∗ ( x ) , we can expect a lower kernel approximation error by replacing p − 1 / 2 t in Proposition 4.1 with p − 1 / 2 ∗ ( · ) as follows
[Equation]
where k σ denotes the smoothing kernel, and it satisfies
[Equation]
and
[Equation]
Notice that many popular kernels satisfies Eq.
25, e.g., standard RBF kernel, Bump function (Eq. 26), etc.
[Equation]
Therefore, the dynamics of KL divergence in such a reweighted kernel is
[Equation]
Similar to the requirement of a delta function in 4.1, a small σ is preferred in our setting, since our analysis is based on the population limit of p t ( x ) .
Besides, our analysis would indicate the kernel choice to obtain the convergence.
Compared with utilizing smoothing kernel directly, we may expect a smaller kernel approximation error by introducing kernels as Eq.
We also validate this phenomenon by Figure 2, which has shown the gradient vanishing phenomenon with the smoothing kernel.
Figure 2 (a) is the vector field of Wasserstein gradient ∇ ln p ∗ ( x ) /p
t ( x ) , which is a linear function in Gaussian case.
However, when using a smoothing kernel the low-density area in Figure 2 (b) has almost no gradient, which makes the particle in this area stuck.
Our proposed reweighted kernel amplifies the gradient in low-density areas, the resulting gradient is similar to the Wasserstein gradient.
4. E RROR R EWEIGHTING
To achieve a better kernel approximation error with reweighted smoothing kernel Eq.
24, we require a corresponding local version of log-Sobolev inequality to control the sufficient descent of the evolution of the KL divergence (Eq. 27).
Local version of log-Sobolev inequality.
In this context, the sufficient descent term should be reformulated to obtain a well-behaved kernel approximation error.
The choice of g in Eq.
38 may convert the kernel approximation error to a tractable form.
By choosing g ( x ) =
p t ( x ) p ∗ ( x ) , we can find the upper bound of KL divergence by χ 2 version of Rényi information.
Lemma 4.2.
Suppose p ∗ satisfies the µ -log-Sobolev inequality (LSI) with a constant µ > 0 .
When any probability density function p t satisfies D χ [Equation] p t , p [Equation]
) ≤ 1 / 2 , we have
[Equation]
Lemma 4.2 indicates that the KL divergence is also bounded by the Wasserstein gradient of χdivergence.
If p t /p ∗ is bounded, Eq. 28 provides a tighter upper bound of KL divergence compared with that in Eq.
7, especially for the tail part.
Thus, controlling the error term in this form has more potential.
With such a construction, the decreasing of KL divergence satisfies
[Equation]
[Table]
[Equation]
Lemma 4.3.
Assume that ln p ∗ and ln p t are L -smooth; p t is warm: sup x ∈ R d p t ( x ) /p ∗ ( x ) ≤ β and
H p ∗ ( p 0 ) ≤ (2 β ) − 1 .
Then by choosing σ = min (cid:16) 1 , O (cid:16) ϵ
[Equation]
L
[Equation]
βL
[Equation]
(cid:17)(cid:17) , we have
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
where k σ satisfies requirements in Eq.
In this condition, we nearly establish the "Stein log-Sobolev Inequality" with arbitrary small ϵ by combining Eq.
27, Eq. 28, and Eq. 29 as follows
[Equation]
Therefore, reweighted kernels can be considered as a sufficient condition for establishing local "Stein log-Sobolev Inequality" near the target distribution p ∗ .
C ONCLUSIONS
In this paper, we prove the local linear convergence for SVGD with reweighted kernel in KL divergence.
In particular, our analysis is based on the smoothing kernel for SVGD algorithm and we point out that the conventional smoothing kernel fails to provide valid gradient scaling in low-density areas.
Thus, we highlight that the reweighting is necessary for smoothing kernels in SVGD algorithm.
With ( p ∗ ( x ) p ∗ ( y ))
− 1 / 2 weighting for k ( x , y ) , we provides the KL convergence rate for SVGD algorithm locally for log-Sobolev p ∗ ( x ) .
Our analysis provide new insights on the kernel design in
SVGD, especially for gradient amplification in the low-density area.
A C ONVERGENCE RATE COMPARISON IN S ECTION
The assumptions are listed as follows
[AS 1 ] p ∗ satisfies µ -log-Sobolev Inequality (Eq. 4).
[AS 2 ] p ∗ satisfies Talagrand 1 Inequality.
[AS 3 ]
The Stein log-Sobolev inequality with constant λ ,
i
e., KL ( p ∥ p ∗ ) ≤ 12 λ D ( p ∥ p ∗ )
[AS 4 ] f ∗ is L -smooth,
i
e., for any x , y ∈ R d ,
[Equation]
f ∗
x )
− ∇
f ∗
y
) ∥ ≤ L ∥ x
[Equation]
y
[Equation]
[AS 5 ] f t is L -smooth where p t =
e − f t .
[AS 6 ] p t is warm: sup x ∈ R d p t ( x )
p
∗
( x ) ≤ β for some constant β ≥ 1 .
[AS 7 ] p t SVGD satisfy:
[Equation]
[AS 8 ] Kernel regularization assumption 1:
[Equation]
[AS 9 ] Kernel regularization assumption 2:
∥ k ( x , · )
∥
H
≤ B and
∥∇ x
[Equation]
[AS 1 0 ] Bounded moment assumptions:
[Equation]
We compare our theoretical results with all previous work where Stein discrepancy is abbreviated as
SD.
Algorithm
Assumptions
Criterion
Asymptotic
Rate
ULA Vempala & Wibisono (2019)
[AS 1 ] , [AS 4 ]
KL
No
O ( e − t )
SVGD Liu (2017)
[AS 7 ]
KL
Yes
N/A
SVGD Liu (2017)
[AS 4 ] , [AS 7 ] , [AS 8 ]
SD
Yes
O (1 /t )
SVGD Korba et al.
[AS 3 ] , [AS 4 ] , [AS 7 ]
KL
Yes
O ( e − t )
SVGD Korba et al.
[AS 4 ] , [AS 7 ] , [AS 9 ] , [AS 1 0 ]
SD
No
O (1 /t )
SVGD Salim et al.
[AS 4 ] , [AS 2 ] , [AS 7 ] , [AS 9 ]
SD
No
O (1 /t )
Ours [AS 1 ] , [AS 4 ] , [AS 5 ] , [AS 6 ]
KL
Yes
O ( e − t )
B I MPORTANT L EMMAS IN S ECTION
In order to facilitate the lemma in Chapter 4, we first revisit the dynamics of SVGD.
Remark 2.
With a slight abuse of notation, p t of SVGD follows the continuity equation:
[Equation]
35) in the sense of distribution (rather than almost sure), where
[Equation]
Notice that Eq.
35 holds in the sense of distribution here means (cid:90)
[Equation]
(37) for all v ( x ) ∈ C ∞ c ( R d ) .
B. P ROOF OF P ROPOSITION 4.
Proof.
When k ( x , y )
= ( p t ( x )) − 1 / 2 k σ ( x , y )
( p t ( y ))
− 1 / 2 and k 0 ( x , y )
=
δ x ( y ) , we have
[Equation] [Equation]
Thus,
[Equation]
which indicates that ϕ t ( x ) is exactly the Wasserstein gradient and the kernel approximation error is 0 .
B. P ROOF OF L EMMA 4.
Proof.
With LSI, all smooth function g : R d → R with E p ∗ [ g 2 ] < ∞ ,
[Equation]
Suppose g = p t /p ∗ , we have E p ∗ [ g 2 ]
=
D χ 2 ( p t , p ∗ ) + 1 < ∞ and (cid:90)
R d p ∗ ( x ) · (cid:18) p t ( x ) p ∗ ( x ) (cid:19)ln (cid:18) p t ( x ) p ∗ ( x ) (cid:19)d x − (cid:90) R d p ∗ ( x ) · (cid:18) p t ( x ) p ∗ ( x ) (cid:19)d x · ln (cid:34)(cid:90)
R d p ∗ ( x ) · (cid:18) p t ( x ) p ∗ ( x ) (cid:19)d x (cid:35) ≤µ (cid:90)
R d p ∗ ( x ) · (cid:13)(cid:13)(cid:13)(cid:13) ∇ p t ( x ) p ∗ ( x ) (cid:13)(cid:13)(cid:13)(cid:13)
d x = 2 µ (cid:90)
R d p ∗ ( x ) · (cid:13)(cid:13)(cid:13)(cid:13) p t ( x ) p ∗ ( x ) ∇ ln p t ( x ) p ∗ ( x ) (cid:13)(cid:13)(cid:13)(cid:13)
d x .
With the fact ln x ≤ x − 1 and ln x ≥ 1 − 1 x when x ≥ 0 , we have ln (cid:34)(cid:90)
R d p
[Equation]
( x ) · (cid:18) p t ( x ) p ∗ ( x ) (cid:19)d x (cid:35) ≤ (cid:90)
R d p t ( x ) · (cid:18) p t ( x ) p ∗ ( x ) −(cid:19)
d x = (cid:90)
R d p 2 t ( x ) p ∗ ( x ) · p t ( x ) p ∗ ( x ) −p t ( x ) p ∗ ( x ) d x ≤ (cid:90)
R d p 2 t ( x ) p ∗ ( x ) · ln (cid:18) p t ( x ) p ∗ ( x ) (cid:19)
d x .
Plugging the previous inequality into LHS of Eq. 39, we have
LHS ≥ (cid:90)
R d p ∗ ( x ) · (cid:18) p t ( x ) p ∗ ( x ) (cid:19)ln (cid:18) p t ( x ) p ∗ ( x ) (cid:19)d x − (cid:90)
R d p ∗ ( x ) · (cid:18) p t ( x ) p ∗ ( x ) (cid:19)d x · (cid:90)
R d p 2 t ( x ) p ∗ ( x ) · ln (cid:18) p t ( x ) p ∗ ( x ) (cid:19)
d x =2 (cid:90)
R d p ∗ ( x ) · (cid:18) p t ( x ) p ∗ ( x ) (cid:19)ln (cid:18) p t ( x ) p ∗ ( x ) (cid:19) d x − (cid:90) R d p ∗ ( x ) · (cid:18) p t ( x ) p ∗ ( x ) (cid:19)ln (cid:18) p t ( x ) p ∗ ( x ) (cid:19) d x − (cid:90)
R d p ∗ ( x ) · (cid:32)(cid:18) p t ( x ) p ∗ ( x ) (cid:19)−(cid:33) d x · (cid:90)
R d p 2 t ( x ) p ∗ ( x ) · ln (cid:18) p t ( x ) p ∗ ( x ) (cid:19)
d x ≥ (cid:0) 2 − D χ 2 ( p t , p ∗ ) − 1 (cid:1) · (cid:90) R
d p ∗ ( x ) · (cid:18) p t ( x ) p ∗ ( x ) (cid:19)ln (cid:18) p t ( x ) p ∗ ( x ) (cid:19)
d x ≥ (cid:90)
R d p ∗ ( x ) · (cid:18) p t ( x ) p ∗ ( x ) (cid:19)ln (cid:18) p t ( x ) p ∗ ( x ) (cid:19)
[Equation]
[Equation]
where the last inequality follows from D χ 2 ( p t , p ∗ ) ≤ 1 / 2 .
Besides, we have
[Equation]
where the last inequality follows from ( x − 1) ln x ≥ 0 for all x ≥ 0 .
Combining Eq.
39, Eq.
41 and
Eq.
42, we complete the proof
[Equation]
and obtain
[Equation]
B. P ROOF OF L EMMA 4.
Before providing the Kernel Approximation Error (KAE) in Lemma 4.
[Equation]
we need to introduce some lemmas.
Lemma B.1.
Assume that p ( x ) is L -smooth and log-Sobolev, then for any α >
[Equation]
Proof.
By Ledoux (1999), when p ( x ) is log-Sobolev, there exists c > 0 such that
[Equation]
then
[Equation]
Thus, there exists c x > 0 , for sufficient large ∥ x ∥ > c x , ln p ( x ) < − c ∥ x ∥and (cid:90) ∥ x ∥ >c x p
[Equation]
πd cα
For
[Equation]
) where C ( d, c x , αL ) is a constant depend on d, c x , αL .
Lemma B.2.
(A Variant of Lemma.
11 in Vempala & Wibisono (2019))
Suppose p ( x ) =
e − f ( x ) , x ∈ R d , f is L-smooth and p ( x ) satisfies
[Equation]
then we have
[Equation]
Proof.
Since f ( x ) is L -smooth, we have for any x
[Equation]
Using integration by parts, we have
[Equation]
Proof.
In the following
[Equation]
we mainly focus on providing the upper bound of Kernel Approximation
Error (KAE)
[Equation]
and have
[Equation]
where the first equation follows from the change of variable.
With the requirement of k
[Equation]
we have k
[Equation]
and
[Equation] [Equation]
In this condition
[Equation]
suppose y =
σ z in Eq.
47, x z := x − σ z , then we have
[Equation]
p ∗ ( x ) · (cid:13)(cid:13)(cid:13)(cid:13) p t ( x ) p ∗ ( x ) ∇ ln p t ( x ) p ∗ ( x ) (cid:13)(cid:13)(cid:13)(cid:13)
d x .
For any x ∈ R d , we have p t
[Equation]
x
[Equation]
p
[Equation]
p
[Equation]
p
[Equation]
(cid:112)
[Equation]
x
[Equation]
x
[Equation]
where p t ( x ) =
e − f t ( x ) , p ∗ ( x ) =
e − f ∗ ( x ) .
Plugging such an equation into Eq.
49, we have
[Table]
[Equation]
where the first inequality follows from Minkowski inequality, the second inequality follows from
Cauchy–Schwarz inequality
[Equation]
and the third inequality follows from Eq.
[Equation]
Consider Term 1.
[Equation]
we have,
Term 1 .
[Equation]
d z d x + 2 (cid:90)
R d (cid:90)
R d ˜ k
( z ) (cid:13)(cid:13)(cid:13)(cid:13)(cid:13) p t
( x z ) (cid:112) p ∗ ( x z ) ( ∇ f t ( x ) − ∇ f t ( x z ))
(cid:13)(cid:13)(cid:13)(cid:13)(cid:13)
d z d x ≤ 2 (cid:90)
R d (cid:90)
R d ˜ k
( z ) (cid:13)(cid:13)(cid:13)(cid:13)(cid:13)(cid:32) p t ( x ) (cid:112) p ∗ ( x )
− p t ( x z ) (cid:112) p ∗ ( x z ) (cid:33) ∇ f t ( x ) (cid:13)(cid:13)(cid:13)(cid:13)(cid:13)
d z d x + 2 (cid:90)
R d ˜ k
( z ) L 2 σ 2 ∥ z ∥ 2 (cid:90)
R d (cid:32) p t ( x z ) (cid:112) p ∗ ( x z ) (cid:33)d x d z ≤ 2 (cid:90)
R d (cid:90)
R d ˜ k
( z ) (cid:13)(cid:13)(cid:13)(cid:13)(cid:13)(cid:32) p t ( x ) (cid:112) p ∗ ( x )
− p t ( x z ) (cid:112) p ∗ ( x z ) (cid:33) ∇ f t ( x ) (cid:13)(cid:13)(cid:13)(cid:13)(cid:13)
d z d x + 2 σ
[Equation]
( M + 1) , (51) where the second inequality follows from L -smoothness of p t ( x ) and the Fubini’s theorem, and the third inequality follows from p t warm (Assumption 3) and the fact (cid:82) R d ∥ y ∥ 2 · ˜ k
( y )d y ≤ D in Eq.
In the following, we focus on the first term of RHS of Eq. 51, and have
[Equation]
For each x ∈ R d , suppose high dimensional R d can be divided into two parts:
[Equation]
For z ∈ B l ( x ) , we have p t
[Equation] [Equation]
p
[Equation]
≤ p t ( x z ) (cid:112) p ∗ ( x z ) · (cid:20)(cid:18) f
[Equation]
where the first inequality follows from 1 − e − x ≤ x for any x ≥ −
[Equation]
the second and third inequality follows from L-smoothness of f ∗ and f t .
Therefore
[Equation]
we have
[Equation] [Equation]
It can be noticed that the first inequality follows from Eq.
54, the second and the third inequalities follow from Cauchy–Schwarz inequality, the fourth inequality follows from the β -warm during the update, the fourth inequality follows from the β -warm during the update, Lemma B.2 and the following fact
[Equation]
Besides, the constant C is provided by Lemma B.1 as (cid:90)
[Equation]
For z ∈ B u ( x ) , we have
[Equation]
Similar to Eq.
55, we have
[Equation]
[Table]
[Equation]
≤
[Equation]
Plugging Eq.
55, Eq.
into Eq.
51, we have
Term
[Equation] [Equation]
With the same techniques in Eq.
51, we have
Term 1 .
[Equation]
Similar to
[Table]
we have
[Table]
[Equation] [Equation]
where the last inequality utilizes additional β -warm condition comparing with Eq. 55.
Similar to Eq.
[Equation]
when z ∈ B u ( x )
[Equation]
we have
[Equation] [Equation]
Combining Eq.
60, Eq.
61 with Eq.
59, we have
Term
[Equation] [Equation]
[Equation]
Without loss of generality
[Equation]
we suppose σ ≤ 1 and M ≥ 1 .
Plugging Eq.
58 and Eq.
62 into Eq.
50, we have
Term
[Equation] [Equation]
which means if we set
[Equation]
KAE satisfies
[Equation]
and the proof is completed.
C T HE M AIN T HEOREM IN S ECTION
Before providing the main theorem, i.e., Theorem 3.
[Equation]
we need to introduce some lemmas.
Lemma C.1.
Suppose Assumption [A 1 ] – [A 3 ] are satisfied, and p 0 is near to the target p ∗ satisfying
D χ ( p 0 , p ∗ ) ≤ 1 / 4 , for any time T = − C
ln ϵ where ϵ ≤ (16 βC ) − 2 , we have
D χ ( p T , p ∗ ) ≤ 1 / 2 .
Proof.
We denote D χ 2 ( p t , p ∗ ) as chi-square distance between p t and p ∗ .
We have the following functional derivative
[Equation]
Combining the result with Remark
[Equation]
we have
[Equation]
Plugging Eq.
20 to the previous equation, we have
[Equation]
where the last inequality follows from the p t warm assumption.
Suppose we control the KAE by
Eq. 64, which means ∂ t D χ ( p t , p ∗ ) ≤ 4 βϵ , and time T = − C
ln ϵ which leads p t to the target region, i.e., KL( p t ∥ p ∗ ) ≤ ϵ by the linear convergence, when ϵ is small enough, e.g., ϵ ≤ (16 βC ) − 2 and
D χ ( p 0 , p ∗ ) ≤ 1 / 4 , we have
[Equation]
Hence, the proof is completed.
With these Lemmas
[Equation]
we provide the main theorem proof in the following.
Proof.
Suppose H p ∗ ( p t ) :
=
KL( p t ∥ p ∗ ) for abbreviation.
According to the time derivative of KL divergence along any flow, we have
[Equation]
Therefore, along Remark
[Equation]
we have
[Equation]
which follows from Eq.
By taking
[Equation]
Eq.
69 satisfies
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
Kernel Approximation
[Equation]
the decreasing of KL divergence of SVGD at time t satisfies
[Equation]
[Equation]
where the first inequality follows from Lemma 4.3 and the second one follows from Lemma 4.2.
However, Lemma 4.2 requires a local condition of p t which we proved in Lemma C.1.
By applying
Gronwall’s lemma, Eq.
71 implies the desired bound
[Equation]
Hence, the proof is completed.
Remark 3.
In Theorem 3.1, we require the variance of the reweighted kernels to be related to ϵ .
We can also utilize the reweighted kernels with decreasing variance σ t = Ω(1 /t ) .
In this condition, suppose the dynamic variance σ t satisfies
[Equation]
Similar to Eq.
71, we have
[Equation]
It means the KL divergence H p ∗ ( p t ) satisfies
[Equation]
where E⟩ is denoted as exponential integral satisfying
[Equation]
and
[Equation]
Plugging this inequality into Eq.
75, we have
[Equation]
Therefore, the KL divergence will be exponential decay with an additional log -term, which means it can still achieve the ϵ neighborhood of p ∗ in O ( − ln ϵ ) .
In the following, we will show a convergence rate similar to Theorem 3.1 can be obtained by approximating an unknown normalizing constant of the target distribution p ∗ .
Proposition C.2.
Suppose Assumption [A 1 ] - [A 3 ] are satisfied,
chi-square D χ ( p 0 , p ∗ ) ≤ 1 / 4 and
PDF of target distribution p ∗ ( x ) can be estimated by ˆ p ∗ ( x ) satisfying
[Equation]
0 < C ∗ , ˆ C < ∞ .
For any ϵ > 0 , if we set reweighted kernel k :
[Equation]
and
[Equation]
where
[Equation]
then the KL divergence between p t and p ∗ satisfies
[Equation]
Proof.
We suppose the following reweighted kernels
[Equation]
lead to different update rules which are
[Equation]
where the second and the last equations follow from integration by part.
Hence, the dynamic of KL divergence under ˜ ϕ t becomes
[Equation]
where the equation follows from the definition of ˆ k , and the inequality follows from Theorem 3.1.
By applying Gronwall’s lemma, Eq. 81 implies the desired bound
[Equation]
This proposition demonstrates that approximating an unknown normalizing constant will not harm the linear convergence rate of SVGD with reweighted kernels, and only provides an additional factor ˆ
C/C ∗ in total complexity.
The very common question is that it seems that the convergence can be arbitrarily fast when the factor ˆ C/C ∗ is large enough.
Actually, we should notice this convergence rate only establishes in asymptotic analysis, which means the discretization error cannot be controlled without a tiny step size when ˆ C/C ∗ is large.
That means a large ˆ C/C ∗ usually implies a small step size in practice rather than arbitrarily fast convergence.
D E XPERIMENTAL RESULTS
In this section, we conduct the SVGD with reweighted kernels in some synthetic data to validate our claims, i.e., compared with traditional SVGD, SVGD with reweighted kernels can achieve any ϵ -neighborhood with a linear convergence.
To validate our theoretical results in asymptotic settings, we choose different particle sizes and show that sampling by SVGD with reweighted kernels can obtain a lower KL divergence.
[Figure] [Figure]
To demonstrate the efficiency and stability of SVGD, we provide a numerical illustration for these two algorithms in Figure 5.
It is clear that Langevin dynamics suffer from the introduction of the stochasticity, which makes the particle highly unstable.
Thus, it is necessary to use more particles to perform the task to guarantee the stability.
[Figure] [Figure]