A Finite-Particle Convergence Rate for Stein Variational Gradient Descent
Abstract
We provide a first finite-particle convergence rate for Stein variational gradient descent (SVGD).
Specifically, whenever the target distribution satisfies Talagrand’s T 1 inequality, SVGD with n particles and an appropriate step size sequence drives the kernel Stein discrepancy to zero at an order 1 / √ log log n rate.
We suspect that the dependence on n can be improved, and we hope that our explicit, non-asymptotic proof strategy will serve as a template for future refinements
1. Introduction
Stein variational gradient descent [SVGD, 11] is an algorithm for approximating a target probability distribution P on R d with a collection of n particles.
Given an initial particle approximation µ n 0 = 1 n (cid:80) ni =
δ x i with locations x i ∈ R d , SVGD (Algorithm 1) iteratively evolves the particle locations to provide a more faithful approximation of the target P by performing optimization in the space of probability measures.
SVGD has demonstrated encouraging results for a wide variety of inferential tasks, including approximate inference [11, 18, 19], generative modeling [17], and reinforcement learning [8, 13].
Algorithm 1 Stein Variational Gradient Descent [11]: SVGD ( µ 0 )
Input: Target P with density p , approximating distribution µ 0 , kernel k , step sizes ϵ 0: r −for s = 0 , · · · , r − 1 do
Let µ s +1 be the distribution of X s + ϵ s (cid:82)
k
( x, X s ) ∇ log p ( x )+ ∇ x k ( x, X s ) dµ s ( x ) for X s ∼ µ s .
Output: Updated approximation µ r of the target P
Despite the popularity of SVGD, relatively little is known about its approximation quality.
A first analysis by Liu [10, Thm. 3.3] showed that continuous SVGD —that is, Algorithm 1 initialized with a continuous distribution µ ∞ 0 in place of the discrete particle approximation µ n 0 —converges to P in kernel Stein discrepancy [KSD, 2, 6, 12].
Follow-up work by Korba et al.
[9], Salim et al.
[14], Sun et al.
[15] sharpened this result with path-independent constants, weaker smoothness conditions, and explicit rates of convergence.
In addition, Duncan et al.
[4] analyzed the continuoustime limit of continuous SVGD to provide conditions for exponential convergence.
However, each of these analyses applies only to continuous SVGD and not to the finite-particle algorithm used in practice.
To bridge this gap, Liu [10, Thm. 3.2] showed that n -particle SVGD converges to continuous
SVGD in bounded-Lipschitz distance but only under boundedness assumptions violated by most applications of SVGD.
To provide a more broadly applicable proof of convergence, Gorham et al.
[7, Thm. 7] showed that n -particle SVGD converges to continuous SVGD in 1 -Wasserstein distance under assumptions commonly satisfied in SVGD applications.
However, both convergence results are asymptotic, providing neither explicit error bounds nor rates of convergence.
In this work, we derive the first explicit error bounds and rates of convergence for finite-particle
SVGD.
To achieve this, we first bound the 1 -Wasserstein error between each round of finite-particle and continuous SVGD in Theorem 1.
We next bound the KSD in terms of the 1 -Wasserstein distance in Theorem 2.
Finally, Theorem 3 combines our results with the established KSD analysis of continuous SVGD to arrive at an explicit KSD error bound for n -particle SVGD.
2. Notation and Assumptions
Throughout, we fix a reproducing kernel k : R d × R d → R , let H denote its induced reproducing kernel Hilbert space (RKHS), and let H d ≜ (cid:78) di =1 H denote the product RKHS [1].
Let P represent the set of probability measures on R d .
For all µ, ν ∈ P , we let W 1 ( µ, ν ) ≜ inf X ∼ µ,Z ∼ ν E [ ∥ Z − X ∥ 2 ] denote the 1 -Wasserstein distance between them and introduce the shorthand m µ ≜ E µ [ ∥·∥ ] 2 , m µ,P ≜
E X ∼ µ ⊥⊥ Z ∼ P [ ∥
X − Z ∥ 2 ] , and M µ,P ≜ E X ∼ µ ⊥⊥ Z ∼ P
[ ∥ X − Z ∥ 22 ] .
We allow each of these quantities to take on the value ∞ when the random variables are not suitably integrable.
We further define the Kullback-Leibler divergence as KL ( µ ∥ ν ) ≜ E µ [log( dµdν )] when µ is absolutely continuous with respect to ν and as ∞ otherwise.
Our analysis will make use of the following standard assumptions on the SVGD kernel and target distribution.
Assumption 1 (Lipschitz score function)
The target distribution P ∈ P has a differentiable density p on R d and
an L -Lipschitz score function s p ≜ ∇ log p , i.e., ∥ s p ( x ) − s p ( y ) ∥ 2 ≤ L ∥ x − y ∥for any x ̸
= y .
In addition, there exists a point x ∗ satisfying s p ( x ∗ ) = 0 .
Assumption 2 (Bounded kernel derivatives)
For any multi-index I
=
( I 1 , I 2 , . .
, I d ) with | I | ≜ (cid:80) di
=
I i ≤ 2 , we have sup x ∈ R d (cid:0)
D Ix D Iy k ( x, y ) | y
=
x (cid:1) ≤ κ 2 .
Here, D I is the differential operator defined as D Ix =
[Equation]
Assumption 3 (Decaying kernel derivatives)
There exists γ > 0 such that
[Equation]
In line with Salim et al.
[14], we also assume the distribution P satisfies Talagrand’s T 1 inequality [16, Def. 22.1].
We note that this assumption is only necessary for leveraging the continuous
SVGD convergence result of Salim et al.
In case it does not hold, one can alternatively resort to the path-dependent assumptions in Korba et al.
[9], Liu [10] .
Assumption 4 (Talagrand’s T 1 inequality [16, Def. 22.1])
There exists λ > 0 such that for all µ with E µ [ ∥·∥ 2 ] < ∞
[Equation]
Finally we make use of the following notation specific to the SVGD algorithm.
Definition 1 (Stein operator) For any vector-valued function g : R d → R d , the Langevin Stein operator [5] for P satisfying Assumption 1 is defined by
[Equation]
Definition 2 (Vector-valued Stein operator)
For any function h : R d → R , the vector-valued
Langevin Stein operator [12] for P satisfying Assumption 1 is defined by
[Equation]
with components
[Equation]
Definition 3 (SVGD transport map and pushforward)
The SVGD transport map [11] for a kernel k , target P satisfying Assumption 1, approximating µ ∈ P , and step size ϵ is given by
[Equation]
Moreover, the SVGD pushforward Φ ϵ ( µ ) represents the distribution of T µ,ϵ ( X ) when X ∼ µ .
Definition 4 (Kernel Stein discrepancy)
The Langevin KSD [2, 6, 12] with base kernel k , target measure P satisfying Assumption 1, and approximating measure µ ∈ P is given by
[Equation]
3. Discretization Error of SVGD
Our first main result concerns the discretization error of SVGD and shows that n -particle SVGD does not stray too far from its continuous SVGD limit when its step sizes are chosen appropriately.
Theorem 1 (Discretization error of SVGD)
Suppose Assumptions 1, 2, and 3 hold.
For any µ n 0 , µ ∞ 0 ∈ P , the outputs µ n r = SVGD ( µ n 0 ) and µ ∞ r = SVGD ( µ ∞ 0 ) of Algorithm 1 satisfy
W 1 (
[Equation] [Equation]
) ≤ W
[Equation]
[Equation] [Equation]
( b r
[Equation]
1 ( A + B exp( Cb r − 1 ))) for b r − 1 ≜ (cid:80)
r − 1 s =
ϵ s , A
= ( c 1 + c 2 )(1 + m P ) , B = c
1 m µ n 0 ,P + c 2 m µ ∞ 0 ,P , and C = κ 2 (3 L + d ) .
Here c 1 = max( √ dκ 2 L, √ dκ 2 L
∥
x ∗ ∥ 2 + dκ 2 ) and c 2 = κ 2 L + dκ 2 + ( γ + √ dκ 2 )(1 + ∥ x ∗ ∥ 2 ) .
The proof of Theorem 1 in Appendix A relies on two lemmas.
The first, due to Gorham et al.
[7], shows that the one-step SVGD pushforward Φ ϵ (Definition 3) is pseudo-Lipschitz with respect to the 1 -Wasserstein distance whenever the score function ∇ log p and kernel k fulfill a commonlysatisfied pseudo-Lipschitz condition.
Lemma 1 (Wasserstein pseudo-Lipschitzness of SVGD [7, Lem. 12])
For P ∈ P with differentiable density p , suppose that the following pseudo-Lipschitz bounds hold
[Equation]
for some constants c 1 , c 2 > 0 .
Then, for any µ, ν ∈ P ,
[Equation]
where Φ ϵ is the one-step SVGD pushforward (Definition 3) and c µ,ν
=
c 1 (1 + m µ ) + c 2 (1 + m ν ) .
The second lemma, proved in Appendix B, controls the growth of the first and second absolute moments under SVGD.
Lemma 2 (SVGD moment growth) Suppose Assumptions 1 and 2 hold, and let C = κ 2 (3 L + d ) .
Then the SVGD output µ r of Algorithm 1 with b r − 1 ≜ (cid:80) r − 1 s =
ϵ s satisfies
[Equation]
4. From Wasserstein to KSD
Our next result allows us to translate Wasserstein error bounds into KSD error bounds.
Theorem 2 (KSD-Wasserstein bound)
Suppose Assumptions 1 and 2 hold.
For any µ n 0 , µ ∞ 0 ∈ P , the outputs µ nr = SVGD ( µ n 0 ) and µ ∞ r = SVGD ( µ ∞ 0 ) of Algorithm 1 satisfy
[Equation]
Proof We apply Lemma 3, Cauchy-Schwarz, Lemma 2, and the fact min( a, b ) 2 ≤ ab in turn to find KSD
[Equation]
The result now follows by taking the infinimum over the joint distribution of ( X, Y ) .
The proof of Theorem 2 uses the following lemma, a generalization of Gorham and Mackey [6,
Lem.
18] that we prove in Appendix C, which shows that the mean discrepancy between two T P transformed functions is controlled by the 1 -Wasserstein distance.
Lemma 3 (Single function Stein discrepancy bound) Suppose Assumptions 1 and 2 hold.
For any µ, ν ∈ P and g ∈ H d satisfying ∥ g ∥ 2 H d ≜ (cid:80) di =1 ∥ g i ∥ 2 H ≤ 1 , we have
E µ [ T P g ] − E ν [ T P g ] ≤ κ ( d + L ) W 1 ( µ, ν )+ L E ( X ∼ µ,Y ∼ ν ) ⊥⊥ Z ∼ P [ ∥ Y − Z ∥ 2 min(2 κ, √ dκ ∥ X − Y ∥ 2 )] .
5. A Finite-particle Convergence Rate for SVGD
To establish our main SVGD convergence result, we combine Theorems 1 and 2 with the following continuous SVGD error bound due to Salim et al.
[14] (see also Korba et al.
[9], Liu [10]).
Lemma 4 (Continuous SVGD descent lemma [14, Thm. 3.2]) Suppose Assumptions 1, 2, andhold.
For any constant α > 1 and ϵ r < min (cid:16) 2 κ 2 ( L + α 2 ) , R (cid:17) , where R ≜ ( α − 1)(1+ L E µ
[ ∥· − x ∗ ∥ 2 ] + 2 L (cid:112) 2 KL ( µ ∞ [-
-][Equation]
P ) /λ ) , continuous SVGD decreases KL and drives KSD to zero:
[Equation]
By summing (1) over r = 0 , . . .
, t , we have the following corollary:
Corollary 1 Let c ( ϵ ) ≜
ϵ (cid:16) 1 − κ 2 ( L + α [Equation] ϵ (cid:17) .
If ϵ r ≤ min (cid:16) [Equation] (cid:17) , then we have ϵ r 2 ≤ c ( ϵ r ) <
ϵ r and thereby
[Equation]
Finally, we arrive at our main result that shows finite-particle SVGD with appropriately chosen step size sequences drives KSD to zero as W 1 ( µ n 0 , µ ∞ 0 ) → 0 .
Theorem 3 (KSD error of finite-particle SVGD) Suppose Assumptions 1, 2, 3, and 4 hold.
Define π r ≜ c ( ϵ r ) (cid:80) t −r =0 c ( ϵ r ) and w 0 ≜ W 1 ( µ n 0 , µ ∞ 0 ) .
Let b t − 1 = (cid:80) t − 1 r =
ϵ r .
Then,
[Equation]
where a t = κ ( d + L ) w 0 exp( b t − 1 ( A + B exp( Cb t − 1 )))
+ d 1 / 4 κL (cid:112)
2 M µ ∞ 0 ,P w 0 exp(((2 C + ϵ 0 C 2 +
A + B exp( Cb t − 1 )) / 2)
b t − 1 ) , and A, B, C are defined as in Theorem 1.
Proof Let w r ≜
W 1 ( µ nr , µ ∞ r ) .
From Theorem 2, we know
[Equation]
Plugging in Theorem 1’s result w
[Equation]
Cb [Equation] 1 ))) , we have KSD ( µ nr , P ) − a r ≤ KSD ( µ ∞ r , P ) , where
[Equation]
Taking square on both sides and the weighted sum over r = 0 , . .
, t − 1 with weights π r ≜ c ( ϵ r ) (cid:80) t −r =
c ( ϵ r ) , we obtain
[Equation]
where the last inequality follows from Corollary 1.
Moreover, by Cauchy-Schwarz,
[Equation]
Combining (3) and (4), we have
[Equation]
We finish the proof by noticing that (cid:80)
[Equation]
The following corollary, proved in Appendix D, provides an explicit SVGD convergence rate by choosing the step size sequence to balance the three terms on the right-hand side of (2).
Corollary 2 (A finite-particle convergence rate for SVGD)
In the notation of Theorem 3, define the shorthand ψ B,C ( x, y, β ) ≜ 1 C log (cid:16) 1 B ( 1 β log 1 x − y ) (cid:17) .
If max 0 ≤ r<t ϵ r ≤ min (cid:16) 1 κ 2 ( L + α 2 ) , R (cid:17) and b t − 1 = (cid:80) t − 1 r =0 ϵ r = min (cid:16) ψ B,C (cid:0) w 0 (cid:113) log log 1 w 0 , A, β 1 (cid:1) , ψ B,C (cid:0) w 0 log log 1 w 0 , A + 2 C + ϵ 0 C 2 , β 2 (cid:1)(cid:17) , for β 1 ≜ max (cid:16) 1 , ψ B,C (cid:0) w
[Equation]
log log 1 w 0 , A, 1 (cid:1)(cid:17) and β
[Equation]
then,
[Equation]
If in addition µ n 0 = 1 n (cid:80) {+ni =
+} δ x i for x i i
i d. ∼ µ ∞ 0 , then
[Equation]
Appendix A. Proof of Theorem 1: Discretization error of SVGD
We first show that the pseudo-Lipshitz conditions in Lemma 1 holds given our assumptions.
According to Assumption 2, we have
[Equation]
Recall that x ∗ satisfies s p ( x ∗ ) = 0 by Assumption 1.
Then
[Equation]
by triangle inequality
[Equation]
the definition of ∥·∥ op
[Equation]
and Cauchy-Schwartz,
[Equation]
Letting c 1 = max( √ dκ 2 L
[Equation]
√ dκ 2 L ∥ x ∗ ∥ 2 + dκ 2 ) and taking supremum over z proves the first pseudo-Lipschitz condition.
Similarly
[Equation]
we have
[Equation]
where we used
[Equation]
The last inequality is due to
[Equation]
According to Assumption 3, there exists γ > 0 such that ∥∇ x k ( x, z ) ∥ 2 ≤ γ ∥ x − z ∥ − 1 2 .
Therefore, for ∥ x − z ∥ 2 ≥ 1 ,
[Equation]
And for ∥ x − z ∥ 2 < 1 ,
[Equation]
Combining (6) and (7),
[Equation]
Plugging (8) back into (5), we can show the second pseudo Lipschitzness condition holds for c 2 = κ 2 L + dκ
[Equation]
+
[Equation]
+
[Equation]
[Equation]
+
[Equation]
Now that the pseudo Lipschitz conditions hold, by repeated application of Lemma 1 and the inequality (1 + x ) ≤ e x , we have
[Equation]
) for C s = c 1 (1 + m µ ns ) + c 2 (1 + m µ ∞ s ) .
Using the result from Lemma 2, we have
[Equation]
for A = ( c 1 + c 2 )(1 + m P ) and B = c 1 m µ n 0 ,P + c 2 m µ ∞ 0 ,P .
Therefore
[Equation]
Plugging this back into (9) proves the result.
Appendix B. Proof of Lemma 2: SVGD moment growth
From Assumption 2 we know
[Equation]
Let µ be any probability measure.
Using Jensen’s inequality and the facts that E P [ s p ( · )]
= 0 and
E Z ∼ P [( A P k ( · , x ))
( Z )]
= 0 , we have ∥
[Equation]
Let m
[Equation]
r ,
[Equation]
Z ∼ P [ ∥
X − Z ∥ 22 ] , and C = κ 2 (3 L + d ) .
Then,
[Equation]
where the last step used Jensen’s inequality m 2 µ r
[Equation]
P ≤ M µ r ,P .
Then
[Equation]
The final inequality uses the relation 1 + x ≤ e x .
Appendix C. Proof of Lemma 3: Single function
Stein discrepancy bound
From Assumption 2 we know
[Equation]
Suppose X, Y, Z are distributed so that ( X, Y ) is a 1 -Wasserstein optimal coupling of ( µ, ν ) and Z is independent of ( X, Y ) .
Since E P [ s p ( · )] = 0 , repeated use of Cauchy-Schwarz gives
[Equation]
Appendix D. Proof of Corollary 2: A finite-particle convergence rate for SVGD
We first state and prove a useful lemma.
Lemma 5 Suppose f is a monotonic decreasing function.
Let x = f ( β ) for β = max(1 , f (1)) .
Then x ≤ f ( x ) .
Proof If β = 1 ≥ f (1) , then x = f ( β ) =
f (1) ≤ 1 .
Because f is monotonic decreasing, f ( x ) ≥ f (1) = x .
Otherwise, β = f (1) > 1 and hence x
= f ( β ) <
f
(1) =
β .
Again, due to monotonicity, f ( x ) > f ( β ) = x .
Combining the two cases finishes the proof.
Now we are ready to prove Corollary 2.
Let b (1) t − 1 = ψ B,C ( w 0 (cid:113) log log 1 w 0 , A, β 1 ) and b (2) t − 1 = ψ B,C ( w 0 log log 1 w 0 , A + 2 C + ϵ 0 C 2 , β 2 ) .
According to Lemma 5,
[Equation]
b
[Equation]
Rearranging the terms, we have
[Equation]
Similarly, we have b (2) t − 1 ≤ ψ B,C ( w 0 log log 1 w 0 , A + 2 C + ϵ 0 C 2 , b (2) t − 1 ) and √ w 0 exp
[Equation]
b
[Equation]
Since b t − 1 = min( b (1) t − 1 , b (2) t − 1 ) , we can substitute b t − 1 for b (1) t − 1 and b (2) t − 1 in (10) and (11), respectively.
Also, from the definition of b t − 1 we know
[Equation]
Plugging them into (2) proves the first claim.
The second claim follows from the Θ( n − 1 /d ) convergence rate of an empirical measure to its population measure in 1 -Wasserstein distance [3].