Title: A Finite-Particle Convergence Rate for Stein Variational Gradient Descent
Venue: NeurIPS 2023
Accepted: yes
Keywords: Stein Variational Gradient Descent

SVGD

variational inference

sampling

optimization

Stein's method
Abstract: We provide the first finite-particle convergence rate for Stein variational gradient descent (SVGD), a popular algorithm for approximating a probability distribution with a collection of particles. Specifically, whenever the target distribution is sub-Gaussian with a Lipschitz score, SVGD with $n$ particles and an appropriate step size sequence drives the kernel Stein discrepancy to zero at an order ${1/}{\sqrt{\log\log n}}$ rate. We suspect that the dependence on $n$ can be improved, and we hope that our explicit, non-asymptotic proof strategy will serve as a template for future refinements.

A Finite-Particle Convergence Rate for Stein Variational Gradient Descent

Abstract

We provide the first finite-particle convergence rate for Stein variational gradient descent (SVGD), a popular algorithm for approximating a probability distribution with a collection of particles. Specifically, whenever the target distribution is subGaussian with a Lipschitz score, SVGD with n particles and an appropriate step size sequence drives the kernel Stein discrepancy to zero at an order 1 / √ log log n rate. We suspect that the dependence on n can be improved, and we hope that our explicit, non-asymptotic proof strategy will serve as a template for future refinements.

1 Introduction

Stein variational gradient descent [SVGD, 18] is an algorithm for approximating a target probability distribution P on R d with a collection of n particles. Given an initial particle approximation µ n 0 = 1 n ∑ n i =1 δ x i with locations x i ∈ R d , SVGD (Algorithm 1) iteratively evolves the particle locations to provide a more faithful approximation of the target P by performing optimization in the space of probability measures. SVGD has demonstrated encouraging results for a wide variety of inferential tasks, including approximate inference [18, 28, 27], generative modeling [26, 13], and reinforcement learning [11, 21].

Despite the popularity of SVGD, relatively little is known about its approximation quality. A first analysis by Liu [17, Thm. 3.3] showed that continuous SVGD -that is, Algorithm 2 initialized with a continuous distribution µ ∞ 0 in place of the discrete particle approximation µ n 0 -converges to P in kernel Stein discrepancy [KSD, 4, 19, 9]. KSD convergence is also known to imply weak convergence [9, 3, 12, 1] and Wasserstein convergence [14] under various conditions on the target P and the SVGD kernel k . Follow-up work by Korba et al. [15], Salim et al. [22], Sun et al. [24] sharpened the result of Liu with path-independent constants, weaker smoothness conditions, and explicit rates of convergence. In addition, Duncan et al. [6] analyzed the continuous-time limit of continuous SVGD to provide conditions for exponential convergence. However, each of these analyses applies only to continuous SVGD and not to the finite-particle algorithm used in practice.

To bridge this gap, Liu [17, Thm. 3.2] showed that n -particle SVGD converges to continuous SVGD in bounded-Lipschitz distance but only under boundedness assumptions violated by most applications of SVGD. To provide a more broadly applicable proof of convergence, Gorham et al. [10, Thm. 7] showed that n -particle SVGD converges to continuous SVGD in 1 -Wasserstein distance under assumptions commonly satisfied in SVGD applications. However, both convergence results are asymptotic, providing neither explicit error bounds nor rates of convergence. Korba et al. [15, Prop. 7] explicitly bounded the expected squared Wasserstein distance between n -particle and continuous SVGD but only under the assumption of bounded ∇ log p , an assumption that rules out all strongly

Algorithm 1 n -particle Stein Variational Gradient Descent [18]: SVGD ( µ n 0 , r )

Input: Target P with density p , kernel k , step sizes ( ϵ s ) s ≥ 0 , particle approximation µ n 0 = 1 n ∑ n i =1 δ x i , rounds r x i ← x i + ϵ s 1 n ∑ n j =1 k ( x j , x i ) ∇ log p ( x j ) + ∇ x k ( x j , x i ) for i = 1 , . . . , n .

Output: Updated approximation µ n r = 1 n ∑ n i =1 δ x i of the target P

Algorithm 2 Generic Stein Variational Gradient Descent [18]: SVGD ( µ 0 , r )

Input: Target P with density p , kernel k , step sizes ( ϵ s ) s ≥ 0 , approximating measure µ 0 , rounds r

Let µ s +1 be the distribution of X s + ϵ s ∫ k ( x, X s ) ∇ log p ( x ) + ∇ x k ( x, X s ) dµ s ( x ) for X s ∼ µ s . Output: Updated approximation µ r of the target P

log concave or dissipative distributions and all distributions for which the KSD is currently known to control weak convergence [9, 3, 12, 1]. In addition, Korba et al. [15] do not provide a unified bound for the convergence of n -particle SVGD to P and ultimately conclude that 'the convergence rate for SVGD using [ µ n 0 ] remains an open problem.' The same open problem was underscored in the later work of Salim et al. [22], who write 'an important and difficult open problem in the analysis of SVGD is to characterize its complexity with a finite number of particles.'

In this work, we derive the first unified convergence bound for finite-particle SVGD to its target. To achieve this, we first bound the 1 -Wasserstein discretization error between finite-particle and continuous SVGD under assumptions commonly satisfied in SVGD applications and compatible with KSD weak convergence control (see Theorem 1). We next bound KSD in terms of 1 -Wasserstein distance and SVGD moment growth to explicitly control KSD discretization error in Theorem 2. Finally, Theorem 3 combines our results with the established KSD analysis of continuous SVGD to arrive at an explicit KSD error bound for n -particle SVGD.

2 Notation and Assumptions

Throughout, we fix a nonnegative step size sequence ( ϵ s ) s ≥ 0 , a target distribution P in the set P 1 of probability measures on R d with integrable first moments, and a reproducing kernel k -a symmetric positive definite function mapping R d × R d to R -with reproducing kernel Hilbert space (RKHS) H and product RKHS H d ≜ ⊗ d i =1 H [2]. We will use the terms 'kernel' and 'reproducing kernel' interchangeably. For all µ, ν ∈ P 1 , we let Γ( µ, ν ) be the set of all couplings of µ and ν , i.e., joint probability distributions γ over P 1 × P 1 with µ and ν as marginal distributions of the first and second variable respectively. We further let µ ⊗ ν denote the independent coupling, the distribution of ( X,Z ) when X and Z are drawn independently from µ and ν respectively. With this notation in place we define the 1 -Wasserstein distance between µ, ν ∈ P 1 as W 1 ( µ, ν ) ≜ inf γ ∈ Γ( µ,ν ) E ( X,Z ) ∼ γ [ ∥ Z -X ∥ 2 ] and introduce the shorthand m µ,x ⋆ ≜ E µ [ ∥· -x ⋆ ∥ ] 2 for each x ⋆ ∈ R d , m µ,P ≜ E ( X,Z ) ∼ µ ⊗ P [ ∥ X -Z ∥ 2 ] , and M µ,P ≜ E ( X,Z ) ∼ µ ⊗ P [ ∥ X -Z ∥ 2 2 ] . We further define the Kullback-Leibler (KL) divergence as KL ( µ ∥ ν ) ≜ E µ [log( dµ dν )] when µ is absolutely continuous with respect to ν (denoted by µ ≪ ν ) and as ∞ otherwise.

Our analysis will make use of the following assumptions on the SVGD kernel and target distribution.

Assumption 1 (Lipschitz, mean-zero score function) . The target distribution P ∈ P 1 has a differentiable density p with an L -Lipschitz score function s p ≜ ∇ log p , i.e., ∥ s p ( x ) -s p ( y ) ∥ 2 ≤ L ∥ x -y ∥ 2 for all x, y ∈ R d . Moreover, E P [ s p ] = 0 and s p ( x ⋆ ) = 0 for some x ⋆ ∈ R d .

Assumption 2 (Bounded kernel derivatives) . The kernel k is twice differentiable and sup x,y ∈ R d max( | k ( x, y ) | , ∥∇ x k ( x, y ) ∥ 2 , ∥∇ y ∇ x k ( x, y ) ∥ op , ∥ ∥ ∇ 2 x k ( x, y ) ∥ ∥ op ) ≤ κ 2 1 for κ 1 > 0 . Moreover, for all i, j ∈ { 1 , 2 , . . . , d } , sup x ∈ R d ∇ y i ∇ y j ∇ x i ∇ x j k ( x, y ) | y = x ≤ κ 2 2 for κ 2 > 0 .

Assumption 3 (Decaying kernel derivatives) . The kernel k is differentiable and admits a γ ∈ R such that, for all x, y ∈ R d satisfying ∥ x -y ∥ 2 ≥ 1 ,

Assumptions 1, 2, and 3 are both commonly invoked and commonly satisfied in the literature. For example, the Lipschitz score assumption is consistent with prior SVGD convergence analyses [17, 15, 22] and, by Gorham and Mackey [8, Prop. 1], the score s p is mean-zero under the mild integrability condition E X ∼ P [ ∥ s p ( X ) ∥ 2 ] < ∞ . The bounded and decaying derivative assumptions have also been made in prior analyses [15, 9] and, as we detail in Appendix A, are satisfied by the kernels most commonly used in SVGD, like the Gaussian and inverse multiquadric (IMQ) kernels. Notably, in these cases, the bounds κ 1 and κ 2 are independent of the dimension d .

To leverage the continuous SVGD convergence rates of Salim et al. [22], we additionally assume that the target P satisfies Talagrand's T 1 inequality [25, Def. 22.1]. Remarkably, Villani [25, Thm. 22.10] showed that Assumption 4 is equivalent to P being a sub-Gaussian distribution. Hence, this mild assumption holds for all strongly log concave P [23, Def. 2.9], all P satisfying the log Sobolev inequality [25, Thm. 22.17], and all distantly dissipative P for which KSD is known to control weak convergence [9, Def. 4].

Assumption 4 (Talagrand's T 1 inequality [25, Def. 22.1]) . For P ∈ P 1 , there exists λ > 0 such that, for all µ ∈ P 1 ,

Finally we make use of the following notation specific to the SVGD algorithm.

Definition 1 (Stein operator) . For any differentiable vector-valued function g : R d → R d , the Langevin Stein operator [8] for P satisfying Assumption 1 is defined by

Definition 2 (Vector-valued Stein operator) . For any differentiable function h : R d → R , the vector-valued Langevin Stein operator [19] for P satisfying Assumption 1 is defined by

Definition 3 (SVGD transport map and pushforward) . The SVGD transport map [18] for a target P satisfying Assumption 1, a kernel k satisfying Assumption 2, a step size ϵ ≥ 0 , and an approximating distribution µ ∈ P 1 takes the form

Moreover, the SVGD pushforward Φ ϵ ( µ ) represents the distribution of T µ,ϵ ( X ) when X ∼ µ .

Definition 4 (Kernel Stein discrepancy) . The Langevin kernel Stein discrepancy [KSD, 4, 19, 9] for P satisfying Assumption 1, k satisfying Assumption 2, and measures µ, ν ∈ P 1 is given by

Notably, the KSD so-defined is symmetric in its two arguments and satisfies the triangle inequality.

Lemma 1 (KSD symmetry and triangle inequality) . Under Definition 4, for all µ, ν, π ∈ P 1 ,

Proof. Fix any µ, ν, π ∈ P 1 . For symmetry, we note that g ∈ H d ⇔ f = -g ∈ H d , so

To establish the triangle inequality, we write

3 Wasserstein Discretization Error of SVGD

Our first main result concerns the discretization error of SVGD and shows that n -particle SVGD remains close to its continuous SVGD limit whenever the step size sum b r -1 = ∑ r -1 s =0 ϵ s is sufficiently small.

Theorem 1 (Wasserstein discretization error of SVGD) . Suppose Assumptions 1, 2, and 3 hold. For any µ n 0 , µ ∞ 0 ∈ P 1 , the outputs µ n r = SVGD ( µ n 0 , r ) and µ ∞ r = SVGD ( µ ∞ 0 , r ) of Algorithm 2 satisfy

for b r -1 ≜ ∑ r -1 s =0 ϵ s , A = ( c 1 + c 2 )(1 + m P,x ⋆ ) , B = c 1 m µ n 0 ,P + c 2 m µ ∞ 0 ,P , C = κ 2 1 (3 L + 1) , c 1 = max( κ 2 1 L, κ 2 1 ) , and c 2 = κ 2 1 ( L +1) + L max( γ, κ 2 1 ) .

We highlight that Theorem 1 applies to any P 1 initialization of SVGD: the initial particles supporting µ n 0 could, for example, be drawn i.i.d. from a convenient auxiliary distribution µ ∞ 0 or even generated deterministically from some quadrature rule. To marry this result with the continuous SVGD convergence bound of Section 5, we will ultimately require µ ∞ 0 to be a continuous distribution with finite KL ( µ ∞ 0 ∥ P ) . Hence, our primary desideratum for SVGD initialization is that µ n 0 have small Wasserstein distance to some µ ∞ 0 with KL ( µ ∞ 0 ∥ P ) < ∞ . Then, by Theorem 1, the SVGD discretization error W 1 ( µ n r , µ ∞ r ) will remain small whenever the step size sum is not too large.

The proof of Theorem 1 in Section 6 relies on two lemmas. The first, due to Gorham et al. [10], shows that the one-step SVGD pushforward map Φ ϵ (Definition 3) is pseudo-Lipschitz with respect to the 1 -Wasserstein distance 2 whenever the score function ∇ log p and kernel k fulfill a commonlysatisfied pseudo-Lipschitz condition. Here, for any g : R d → R d , we define the Lipschitz constant Lip( g ) ≜ sup R d ∥ g ( x ) -g ( z ) ∥ / ∥ x -z ∥ .

Lemma 2 (Wasserstein pseudo-Lipschitzness of SVGD [10, Lem. 12]) . For P satisfying Assumption 1, suppose that the following pseudo-Lipschitz bounds hold

for some constants c 1 , c 2 ∈ R and all x, z ∈ R d . Then, for any µ, ν ∈ P 1 ,

where Φ ϵ is the one-step SVGD pushforward (Definition 3) and c µ,ν = c 1 (1+ m µ,x ⋆ )+ c 2 (1+ m ν,x ⋆ ) .

In Section 6, we will show that, under Assumptions 1, 2, and 3, the preconditions of Lemma 2 are fulfilled with c 1 and c 2 exactly as in Theorem 1. The second lemma, proved in Section 7, controls the growth of the first and second absolute moments under SVGD.

Lemma 3 (SVGD moment growth) . Suppose Assumptions 1 and 2 hold, and let C = κ 2 1 (3 L +1) . Then the SVGD output µ r of Algorithm 2 with b r -1 ≜ ∑ r -1 s =0 ϵ s satisfies

The key to the proof of Lemma 3 is that we show the norm of any SVGD update, i.e., ∥ T µ,ϵ ( x ) -x ∥ 2 , is controlled by m µ,P , the first absolute moment of µ measured against P . This is mainly due to the Lipschitzness of the score function s p and our assumptions on the boundedness of the kernel and its derivatives. Then, we can use the result to control the growth of m µ r ,P across iterations since m µ r +1 ,P = E ( X,Z ) ∼ µ r ⊗ P [ ∥ T µ r ,ϵ r ( X ) -Z ∥ 2 ] . The same strategy applies to the second absolute moment M µ,P . The proof of Theorem 1 then follows directly from Lemma 2 where we plug in the first moment bound of Lemma 3.

4 KSD Discretization Error of SVGD

Our next result translates the Wasserstein error bounds of Theorem 1 into KSD error bounds.

Theorem 2 (KSD discretization error of SVGD) . Suppose Assumptions 1, 2, and 3 hold. For any µ n 0 , µ ∞ 0 ∈ P 1 , the outputs of Algorithm 2, µ n r = SVGD ( µ n 0 , r ) and µ ∞ r = SVGD ( µ ∞ 0 , r ) , satisfy

Our proof of Theorem 2 relies on the following lemma, proved in Section 8, that shows that the KSD is controlled by the 1 -Wasserstein distance.

Lemma 4 (KSD-Wasserstein bound) . Suppose Assumptions 1 and 2 hold. For any µ, ν ∈ P

Lemma 4 is proved in two steps. We first linearize ( T P g )( x ) in the KSD definition through the Lipschitzness of s p and the boundedness and Lipschitzness of RKHS functions. Then, we assign a 1-Wasserstein optimal coupling of ( µ, ν ) to obtain the Wasserstein bound on the right.

Proof of Theorem 2 The result follows directly from Lemma 4, the second moment bound of Lemma 3, and Theorem 1. ■

5 A Finite-particle Convergence Rate for SVGD

To establish our main SVGD convergence result, we combine Theorems 1 and 2 with the following descent lemma for continuous SVGD error due to Salim et al. [22] which shows that continuous SVGD decreases the KL divergence to P and drives the KSD to P to zero.

Lemma 5 (Continuous SVGD descent lemma [22, Thm. 3.2]) . Suppose Assumptions 1, 2, and 4 hold, and consider the outputs µ ∞ r = SVGD ( µ ∞ 0 , r ) and µ ∞ r +1 = SVGD ( µ ∞ 0 , r +1) of Algorithm 2 with µ ∞ 0 ≪ P . If max 0 ≤ s ≤ r ϵ s ≤ R α, 2 for some α > 1 and

then

By summing the result (2) over r ∈ { 0 , . . . , t } , we obtain the following corollary.

Corollary 1. Under the assumptions and notation of Lemma 5, suppose max 0 ≤ r ≤ t ϵ r ≤ R α, 1 for some α > 1 , and let π r ≜ c ( ϵ r ) ∑ t r =0 c ( ϵ r ) for c ( ϵ ) ≜ ϵ ( 1 -κ 2 1 ( L + α 2 ) 2 ϵ ) . Since ϵ 2 ≤ c ( ϵ ) < ϵ , we have

Finally, we arrive at our main result that bounds the approximation error of n -particle SVGD in terms of the chosen step size sequence and the initial discretization error W 1 ( µ n 0 , µ ∞ 0 ) .

Theorem 3 (KSD error of finite-particle SVGD) . Suppose Assumptions 1, 2, 3, and 4 hold, fix any µ ∞ 0 ≪ P and µ n 0 ∈ P 1 , and let w 0 ,n ≜ W 1 ( µ n 0 , µ ∞ 0 ) . If max 0 ≤ r<t ϵ r ≤ ϵ t ≜ R α, 1 3 for some α > 1 and R α, 1 defined in Lemma 5, then the Algorithm 2 outputs µ n r = SVGD ( µ n 0 , r ) satisfy

for π r as defined in Lemma 5, ( A,B,C ) as defined in Theorem 1, b t -1 ≜ ∑ t -1 ϵ r , and r =0

Proof. By the triangle inequality (Lemma 1) and Theorem 2 we have

for each r . Therefore

where the last inequality follows from Corollary 1. Moreover, by Jensen's inequality,

Combining (5) and (6), we have

We finish the proof by noticing that ∑ π r a r -1 ≤ max 0 ≤ r ≤ t a r -1 = a t -1 .

t r =0

The following corollary, proved in Appendix B, provides an explicit SVGD convergence bound and rate by choosing the step size sum to balance the terms on the right-hand side of (3). In particular, Corollary 2 instantiates an explicit SVGD step size sequence that drives the kernel Stein discrepancy to zero at an order 1 / √ log log( n ) rate.

Corollary 2 (A finite-particle convergence rate for SVGD) . Instantiate the notation and assumptions of Theorem 3, let ( ¯ w 0 ,n , ¯ A, ¯ B, ¯ C ) be any upper bounds on ( w 0 ,n , A, B, C ) respectively, and define the growth functions

If the step size sum b t -1 = ∑ t -1 r =0 ϵ r = s ⋆ n for

then

If, in addition, µ n 0 = 1 n ∑ n i =1 δ x i for x i i.i.d. ∼ µ ∞ 0 with M µ ∞ 0 ≜ E µ ∞ 0 [ ∥·∥ 2 2 ] < ∞ , then

with probability at least 1 -cδ for a universal constant c > 0 . Hence, with this choice of ¯ w 0 ,n ,

with probability at least 1 -cδ .

Specifically, given any upper bounds ( ¯ w 0 ,n , ¯ A, ¯ B, ¯ C ) on the quantities ( w 0 ,n , A, B, C ) defined in Theorem 3, Corollary 2 specifies a recommended step size sum s ⋆ n to achieve an order 1 / √ log log( e e + 1 ¯ w 0 ,n ) rate of SVGD convergence in KSD. Several remarks are in order. First,

the target step size sum s ⋆ n is easily computed given ( ¯ w 0 ,n , ¯ A, ¯ B, ¯ C ) . Second, we note that the target s ⋆ n can equal 0 when the initial Wasserstein upper bound ¯ w 0 ,n is insufficiently small since log( 1 ¯ B max( ¯ B, 1 β log 1 x -y )) = log( ¯ B ¯ B ) = 0 for small arguments x . In this case, the setting b t -1 = 0 amounts to not running SVGD at all or, equivalently, to setting all step sizes to 0 .

Third, Corollary 2 also implies a time complexity for achieving an order 1 / √ log log( e e + 1 ¯ w 0 ,n ) error rate. Recall that Theorem 3 assumes that max 0 ≤ r<t ϵ r ≤ R α, 1 for R α, 1 defined in (1). Hence, t ⋆ ≜ ⌈ s ⋆ n /R α, 1 ⌉ rounds of SVGD are necessary to achieve the recommended setting ∑ t ⋆ -1 r =0 ϵ r = s ⋆ n while also satisfying the constraints of Theorem 3. Moreover, t ⋆ rounds are also sufficient if each step size is chosen equal to s ⋆ n /t ⋆ . In addition, since s ⋆ n = O (log log( e e + 1 ¯ w 0 ,n )) , Corollary 2 implies that SVGD can deliver min 0 ≤ r ≤ t ⋆ KSD P ( µ n r , P ) ≤ ∆ in t ⋆ = O (1 / ∆ 2 ) rounds. Since the computational complexity of each SVGD round is dominated by Θ( n 2 ) kernel gradient evaluations (i.e., evaluating ∇ x k ( x i , x j ) for each pair of particles ( x i , x j ) ), the overall computational complexity to achieve the order 1 / √ log log( e e + 1 ¯ w 0 ,n ) error rate is O ( n 2 ⌈ s ⋆ n /R α, 1 ⌉ ) = O ( n 2 log log( e e + 1 ¯ w 0 ,n )) .

6 Proof of Theorem 1: Wasserstein discretization error of SVGD

In order to leverage Lemma 2, we first show that the pseudo-Lipschitzness conditions of Lemma 2 hold given our Assumptions 1 to 3. Recall that s p is Lipschitz and x ⋆ satisfies s p ( x ⋆ ) = 0 by Assumption 1. Then, by the triangle inequality, the definition of ∥·∥ op , ∥ k ( x, z ) ∥ 2 ≤ κ 2 1 and ∥∇ z ∇ x k ( x, z ) ∥ op ≤ κ 2 1 from Assumption 2, and Cauchy-Schwartz,

Letting c 1 = max( κ 2 1 L, κ 2 1 ) and taking supremum over z proves the first pseudo-Lipschitzness condition. Similarly, we have

where we used the Lipschitzness of s p from Assumption 1 and | k ( x, z ) | ≤ κ 2 1 , ∥ ∥ ∇ 2 x k ( x, z ) ∥ ∥ op ≤ κ 2 1 from Assumption 2. Now we consider two cases separately: when ∥ x -z ∥ 2 ≥ 1 and ∥ x -z ∥ 2 < 1 .

Case 1: ∥ x -z ∥ 2 ≥ 1 . Recall that there exists γ > 0 such that ∥∇ x k ( x, z ) ∥ 2 ≤ γ/ ∥ x -z ∥ 2 by Assumption 3. Then, using this together with the triangle inequality, we have

Case 2: ∥ x -z ∥ 2 < 1 . Then, using ∥∇ x k ( x, z ) ∥ 2 ≤ κ 2 1 from Assumption 2 and by the triangle inequality, we have

Combining (11) and (12) and using the triangle inequality, we get

Plugging (13) back into (10), we can show the second pseudo-Lipschitzness condition holds for c 2 = max( κ 2 1 ( L +1) + L max( γ, κ 2 1 ) , L max( γ, κ 2 1 )) = κ 2 1 ( L +1) + L max( γ, κ 2 1 ) .

Now we have proved that both pseudo-Lipschitzness preconditions of Lemma 2 hold under our Assumptions 1 to 3. By repeated application of Lemma 2 and the inequality (1 + x ) ≤ e x , we have

Using the result from Lemma 3, we have

Plugging this back into (14) proves the result.

7 Proof of Lemma 3: SVGD moment growth

From Assumption 2 we know | k ( y, x ) | ≤ κ 2 1 and ∥ ∥ ∇ 2 y k ( y, x ) ∥ ∥ op ≤ κ 2 1 . The latter implies

Recall that s p is Lipschitz and satisfies E P [ s p ( · )] = 0 by Assumption 1. Let µ be any probability measure. Using the above results, Jensen's inequality and the fact that E Z ∼ P [( A P k ( · , x ))( Z )] = 0 , we have

The last step used the definitions m µ,P ≜ E ( X,Z ) ∼ µ ⊗ P [ ∥ X -Z ∥ 2 ] and C = κ 2 1 (3 L + 1) . Then, applying the triangle inequality and (15), we have

where the second last step used Jensen's inequality m 2 µ r ,P ≤ M µ r ,P . Then, we repeatedly apply (16) and (17) together with the triangle inequality and the bound 1 + x ≤ e x to get

8 Proof of Lemma 4: KSD-Wasserstein bound

Our proof generalizes that of Gorham and Mackey [9, Lem. 18]. Consider any g ∈ H d satisfying ∥ g ∥ 2 H d ≜ ∑ d i =1 ∥ g i ∥ 2 H ≤ 1 . From Assumption 2 we know

Suppose X,Y,Z are distributed so that ( X,Y ) is a 1 -Wasserstein optimal coupling of ( µ, ν ) and Z is independent of ( X,Y ) . Since s p is L -Lipschitz with E P [ s p ] = 0 (Assumption 1), g is bounded (18), and g and ∇· g are Lipschitz (19), repeated use of Cauchy-Schwarz gives

Since our choice of g was arbitrary, the first advertised result now follows from the definition of KSD (Definition 4). The second claim then follows from Cauchy-Schwarz and the inequality min( a, b ) 2 ≤ ab for a, b ≥ 0 , since

9 Conclusions and Limitations

In summary, we have proved the first unified convergence bound and rate for finite-particle SVGD. In particular, our results show that with a suitably chosen step size sequence, SVGD with n -particles drives the KSD to zero at an order 1 / √ log log( n ) rate. The assumptions we have made on the target and kernel are mild and strictly weaker than those used in prior work to establish KSD weak convergence control [9, 3, 12, 1]. However, we suspect that, with additional effort, the Lipschitz score assumption (Assumption 1) can be relaxed to accommodate pseudo-Lipschitz scores as in Erdogdu et al. [7] or weakly-smooth scores as in Sun et al. [24]. A second limitation of this work is that the obtained rate of convergence is quite slow. However, we hope that this initial recipe for explicit, non-asymptotic convergence will serve as both a template and a catalyst for the field to develop refined upper and lower bounds for SVGD error. To this end, we leave the reader with several open challenges. First, can one establish a non-trivial minimax lower bound for the convergence of SVGD? Second, can one identify which types of target distributions lead to worst-case convergence behavior for SVGD? Finally, can one identify commonly met assumptions on the target distribution and kernel under which the guaranteed convergence rate of SVGD can be significantly improved? Promising follow-up work has already begun investigating speed-ups obtainable by focusing on the convergence of a finite set of moments [20] or by modifying the SVGD algorithm [5].

References

Alessandro Barp, Carl-Johann Simon-Gabriel, Mark Girolami, and Lester Mackey. Targeted separation and convergence with kernel discrepancies. arXiv preprint arXiv:2209.12835 , 2022. 1, 2, 9

Alain Berlinet and Christine Thomas-Agnan. Reproducing kernel Hilbert spaces in probability and statistics . Springer Science & Business Media, 2011. 2

Wilson Ye Chen, Lester Mackey, Jackson Gorham, François-Xavier Briol, and Chris Oates. Stein points. In International Conference on Machine Learning , pages 844-853, 2018. 1, 2, 9

Kacper Chwialkowski, Heiko Strathmann, and Arthur Gretton. A kernel test of goodness of fit. In International Conference on Machine Learning , pages 2606-2615, 2016. 1, 3

Aniket Das and Dheeraj Nagaraj. Provably fast finite particle variants of svgd via virtual particle stochastic approximation. arXiv preprint arXiv:2305.17558 , 2023. 9

Andrew Duncan, Nikolas Nüsken, and Lukasz Szpruch. On the geometry of Stein variational gradient descent. arXiv preprint arXiv:1912.00894 , 2019. 1

Murat A. Erdogdu, Lester Mackey, and Ohad Shamir. Global non-convex optimization with discretized diffusions. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett, editors, Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada , pages 9694-9703, 2018. 9

Jackson Gorham and Lester Mackey. Measuring sample quality with Stein's method. In Advances in Neural Information Processing Systems , pages 226-234, 2015. 3

Jackson Gorham and Lester Mackey. Measuring sample quality with kernels. In International Conference on Machine Learning , pages 1292-1301, 2017. 1, 2, 3, 9, 12

Jackson Gorham, Anant Raj, and Lester Mackey. Stochastic Stein discrepancies. Advances in Neural Information Processing Systems , 33:17931-17942, 2020. 1, 4

Tuomas Haarnoja, Haoran Tang, Pieter Abbeel, and Sergey Levine. Reinforcement learning with deep energy-based policies. In International Conference on Machine Learning , pages 1352-1361, 2017. 1

Jonathan Huggins and Lester Mackey. Random feature Stein discrepancies. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems , pages 1903-1913. 2018. 1, 2, 9

Priyank Jaini, Lars Holdijk, and Max Welling. Learning equivariant energy based models with equivariant Stein variational gradient descent. Advances in Neural Information Processing Systems , 34:16727-16737, 2021. 1

Heishiro Kanagawa, Arthur Gretton, and Lester Mackey. Controlling moments with kernel stein discrepancies. arXiv preprint arXiv:2211.05408 , 2022. 1

Anna Korba, Adil Salim, Michael Arbel, Giulia Luise, and Arthur Gretton. A non-asymptotic analysis for Stein variational gradient descent. Advances in Neural Information Processing Systems , 33, 2020. 1, 2, 3

Jing Lei. Convergence and concentration of empirical measures under wasserstein distance in unbounded functional spaces. Bernoulli , 26(1):767-798, 2020. 14

Qiang Liu. Stein variational gradient descent as gradient flow. In Advances in Neural Information Processing Systems , pages 3115-3123, 2017. 1, 3

Qiang Liu and Dilin Wang. Stein variational gradient descent: A general purpose Bayesian inference algorithm. Advances in Neural Information Processing Systems , 29:2378-2386, 2016. 1, 2, 3, 13

Qiang Liu, Jason Lee, and Michael Jordan. A kernelized Stein discrepancy for goodness-of-fit tests. In International Conference on Machine Learning , pages 276-284, 2016. 1, 3

Tianle Liu, Promit Ghosal, Krishnakumar Balasubramanian, and Natesh Pillai. Towards understanding the dynamics of gaussian-stein variational gradient descent. arXiv preprint arXiv:2305.14076 , 2023. 9

Yang Liu, Prajit Ramachandran, Qiang Liu, and Jian Peng. Stein variational policy gradient. In 33rd Conference on Uncertainty in Artificial Intelligence , 2017. 1

Adil Salim, Lukang Sun, and Peter Richtarik. A convergence theory for SVGD in the population limit under Talagrand's inequality T1. In International Conference on Machine Learning , pages 19139-19152. PMLR, 2022. 1, 2, 3, 5

Adrien Saumard and Jon A Wellner. Log-concavity and strong log-concavity: a review. Statistics surveys , 8:45, 2014. 3

Lukang Sun, Avetik Karagulyan, and Peter Richtarik. Convergence of Stein variational gradient descent under a weaker smoothness condition. arXiv preprint arXiv:2206.00508 , 2022. 1, 9

Cédric Villani. Optimal transport: old and new , volume 338. Springer, 2009. 3

Dilin Wang and Qiang Liu. Learning to draw samples: With application to amortized MLE for generative adversarial learning. arXiv preprint arXiv:1611.01722 , 2016. 1

Dilin Wang, Zhe Zeng, and Qiang Liu. Stein variational message passing for continuous graphical models. In International Conference on Machine Learning , pages 5219-5227, 2018. 1

Jingwei Zhuo, Chang Liu, Jiaxin Shi, Jun Zhu, Ning Chen, and Bo Zhang. Message passing Stein variational gradient descent. In International Conference on Machine Learning , pages 6018-6027, 2018. 1

A Kernel Assumptions

To show that Assumptions 2 and 3 are met by the most commonly used SVGD kernels with constants independent of dimension, we begin by bounding the derivatives of any radial kernel of the form k ( x, y ) = ϕ ( ∥ x -y ∥ 2 2 / 2) with ϕ : R → R four times differentiable. By the reproducing property and Cauchy-Schwarz we have

̸

2 2 2 Similarly, the partial derivatives take the form ∇ x j k ( x, y ) = ϕ ′ ( ∥ x -y ∥ 2 2 / 2)( x j -y j ) ∇ y j ∇ x j k ( x, y ) = -ϕ ′ ( ∥ x -y ∥ 2 2 / 2) -ϕ ′′ ( ∥ x -y ∥ 2 2 / 2)( y j -x j ) 2 ∇ x i ∇ y j ∇ x j k ( x, y ) = -ϕ ′′ ( ∥ x -y ∥ 2 2 / 2)( x i -y i ) -ϕ ′′′ ( ∥ x -y ∥ 2 2 / 2)( y j -x j ) 2 ( x i -y i ) -I [ i = j ]2 ϕ ′′ ( ∥ x -y ∥ 2 2 / 2)( x i -y i ) = -ϕ ′′ ( ∥ x -y ∥ 2 2 / 2)( x i -y i ) -I [ i = j ] ϕ ′′′ ( ∥ x -y ∥ 2 2 / 2)( y j -x j ) 2 ( x i -y i ) + I [ i = j ]( ϕ ′′′ ( ∥ x -y ∥ 2 2 / 2)( y i -x i ) 3 -2 ϕ ′′ ( ∥ x -y ∥ 2 2 / 2)( x i -y i )) ∇ y i ∇ x i ∇ y j ∇ x j k ( x, y ) = ϕ ′′′ ( ∥ x -y ∥ 2 2 / 2)( x i -y i ) 2 + ϕ ′′ ( ∥ x -y ∥ 2 2 / 2) + I [ i = j ]( ϕ ′′′′ ( ∥ x -y ∥ 2 2 / 2)( y j -x j ) 2 ( x i -y i ) 2 + ϕ ′′′ ( ∥ x -y ∥ 2 2 / 2)( y j -x j ) 2 ) + I [ i = j ]( ϕ ′′′′ ( ∥ x -y ∥ 2 2 / 2)( y i -x i ) 4 +5 ϕ ′′′ ( ∥ x -y ∥ 2 2 / 2)( y i -x i ) 2 +2 ϕ ′′ ( ∥ x -y ∥ 2 2 / 2)) = ϕ ′′′ ( ∥ x -y ∥ 2 2 / 2)(( x i -y i ) 2 +( x j -y j ) 2 ) + ϕ ′′ ( ∥ x -y ∥ 2 2 / 2) + ϕ ′′′′ ( ∥ x -y ∥ 2 2 / 2)( y j -x j ) 2 ( x i -y i ) 2 + I [ i = j ](4 ϕ ′′′ ( ∥ x -y ∥ 2 2 / 2)( y i -x i ) 2 +2 ϕ ′′ ( ∥ x -y ∥ 2 2 / 2))

so that both | k ( x, y ) | and

are bounded (Assumption 2) by constants independent of dimension.

Gorham and Mackey [9] popularized the use of IMQ kernels for SVGD, by establishing the convergence-determining properties of the associated KSD. The corresponding ϕ satisfies

In this case, ∥∇ y ∇ x k ( x, y ) ∥ op and ∥ ∥ ∇ 2 x k ( x, y ) ∥ ∥ op are bounded (Assumption 2) by constants independent of dimension as

For ∥∇ x k ( x, y ) ∥ 2 , we consider two cases:

̸

When ∥ x -y ∥ ≥ 1 ,

Therefore, ∥∇ x k ( x, y ) ∥ 2 is also bounded (Assumption 2) by constants independent of dimension, and Assumption 3 holds with γ = -2 β .

The original SVGD paper [18] used Gaussian kernels in all experiments, and they remain perhaps the most common choice in the literature. In this case, ϕ satisfies

Using the inequality x ≤ e x -1 , we find that

so that ∥∇ y ∇ x k ( x, y ) ∥ op , ∥ ∥ ∇ 2 x k ( x, y ) ∥ ∥ op , and ∥∇ x k ( x, y ) ∥ 2 are bounded (Assumption 2) by constants independent of dimension, and Assumption 3 holds with γ = 2 /e .

B Proof of Corollary 2: A finite-particle convergence rate for SVGD

We begin by establishing a lower bound on b t -1 . Let

so that b t -1 = min( b (1) t -1 , b (2) t -1 ) . Since β 1 , β 2 , ϕ ( ¯ w 0 ,n ) ≥ 1 , we have

Hence, ϕ ( ¯ w 0 ,n ) ≥ 1 implies that

We divide the remainder of our proof into four parts. First we prove each of the two cases in the generic KSD bound (7) in Appendices B.1 and B.2. Next we show in Appendix B.3 that these two cases yield the generic convergence rate (8). Finally, we prove the high probability upper estimate (9) for w 0 ,n under i.i.d. initialization in Appendix B.4.

B.1 Case b t -1 = 0

In this case, the error bound (7) follows directly from Theorem 3.

B.2 Case b t -1 > 0

We first state and prove a useful lemma.

Lemma 6. Suppose x = f ( β ) for a non-increasing function f : R → R and β = max(1 , f (1)) . Then x ≤ β and x ≤ f ( x ) .

Proof. Because f is non-increasing and β ≥ 1 , x = f ( β ) ≤ f (1) ≤ β . Since x ≤ β and f is non-increasing, we further have f ( x ) ≥ f ( β ) = x as advertised.

Since ψ ¯ B, ¯ C is non-increasing in its third argument, Lemma 6 implies that b (1) t -1 ≤ β 1 and

Rearranging the terms and noting that

since b (1) t -1 ≥ b t -1 > 0 , we have

Similarly, we have b (2) t -1 ≤ ψ ¯ B, ¯ C ( ¯ w 0 ,n log log 1 ¯ w 0 ,n , ¯ A +2 ¯ C,b (2) t -1 ) and

Since b t -1 = min( b (1) t -1 , b (2) t -1 ) , the inequalities (21) and (22) are also satisfied when b t is substituted for b (1) t -1 and b (2) t -1 . Since the error term a t -1 (4) is non-decreasing in each of ( w 0 ,n , A, B, C ) , we have

Since b t -1 = min( b (1) t -1 , b (2) t -1 ) , the claim (7) follows from this estimate, the lower bounds (20), and Theorem 3.

B.3 Generic convergence rate

The generic convergence rate (8) holds as, by the lower bounds (20), b t -1 = min( b (1) t -1 , b (2) t -1 ) > 0 whenever

a condition which occurs whenever ¯ w 0 ,n is sufficiently small since the right-hand side of each inequality converges to zero as ¯ w 0 ,n → 0 .

B.4 Initializing with i.i.d. particles

We begin by restating an expected Wasserstein bound due to Lei [16].

Lemma7 (Lei [16, Thm. 3.1]) . Suppose µ n 0 = 1 n ∑ n i =1 δ x i for x i i.i.d. ∼ µ ∞ 0 with M µ ∞ 0 ≜ E µ ∞ 0 [ ∥·∥ 2 2 ] < ∞ . Then, for a universal constant c > 0 ,

Together, Lemma 7 and Markov's inequality imply that

with probability at least 1 -cδ , proving the high probability upper estimate (9).

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper provides an analysis of the convergence rate of finite-sample Stein Variational Gradient Descent (SVGD) for sub-Gaussian targets with Lipschitz scores. In contrast to previous works such as Liu 2017, Duncan et al. 2019 and Korba et al. 2020, the presented results offer convergence guarantees that hold for finite samples and rely on weaker assumptions. The authors present two key results regarding the discretization error of finite-sample SVGD compared to infinite-sample SVGD, measured in terms of Wasserstein (Theorem 1) and KSD (Theorem 2), respectively. These results are then combined to establish a finite-sample bound on the KSD error between a finite-sample approximation and the target distribution (Theorem 3). By carefully selecting step sizes, the authors demonstrate that this error decays at a rate of $1 / \\sqrt{\\log \\log n}$ (Corollary 2).\n\nSTRENGTHS:\n**Quality**: This paper provides a rigorous study of the convergence of finite-sample SVGD and delivers clear and well-supported results. The main findings (Theorem 1, 2, 3) build upon existing results (Lemma 1 and 4) but require sophisticated combinations and detailed arguments. The authors provide extensive discussions and overview the proof strategy, which appears reasonable and comprehensive (although I did not examine the proofs in depth).\n\n**Novelty**: This paper offers a crucial contribution by bridging the gap between the existing convergence guarantees of infinite-sample SVGD and the practical implementation of finite-sample approximations, which is currently lacking in the literature. Whilst the established convergence rate is slow, this work stands out as the first to provide an explicit, non-asymptotic guarantee for SVGD approximations under reasonably mild assumptions on the target distribution. This, in my view, is the primary novelty and significance of this paper.\n\n**Clarity**: The paper demonstrates clear motivation and objectives. Despite the technical nature of the content, the exposition is highly comprehensible. The intuitive explanations and remarks provided before and after each main theorem are particularly valuable in aiding understanding.\n\nWEAKNESSES:\n**Slow convergence rate**: As mentioned above and in the paper, the established rate $1 / \\sqrt{\\log \\log n}$  is notably slow, suggesting that the bound (3) is likely to be very loose. Although the authors acknowledge the potential of the presented proof strategy as a starting point for refining the bounds, it is not entirely clear whether or not or how the proof could be adapted to achieve an improved bound. Including discussions on potential avenues for enhancing the convergence rate would be valuable and insightful.\n\nQUESTIONS:\n1. There are a few places that say “the Algorithm 2 outputs $\\mu_r^n = SVGD(\\mu_0^n, r)$…”, e.g. L166. Since this output is based on a discrete measure $\\mu_0^n$, should it be “the **Algorithm 1** outputs …” instead?\n2. COuld you elaborate on the suggested step size scheme in Corollary 2? Specifically, what is the dependence on the sample size $n$ and on the dimension $d$? How easy is it to construct the upper bounds $(\\overline{w}_{0, n}, \\overline{A}, \\overline{B}, \\overline{C})$ in practice?\n3. As also mentioned in the paper, the established rate $1 / \\sqrt{\\log \\log n}$ is very slow. Could you provide some insights into which parts of the proof strategy could possibly have led to this slow rate? Is the bound (3) tight? How does this convergence rate compare with empirical performance in numerical simulations?\n\nLIMITATIONS:\nThe assumptions on the target distribution, its score function and the positive definite kernel are summarised in Section 2, accompanied by interpretations and discussions on their connections to related literature. Limitations are also discussed in Section 9. Overall, I think the discussions on assumptions and limitations are adequately covered.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-03"}

comments: SUMMARY:
In this work, the authors present a novel analysis of finite-particle Stein Variational Gradient Descent (SVGD) and derive a unified convergence bound for this algorithm. The convergence bound provides an explicit measure of how close the finite-particle SVGD algorithm gets to its target. To establish this convergence bound, the authors first introduce a bound on the discretization error of the 1-Wasserstein distance between the finite-particle and continuous SVGD. They make certain assumptions that are commonly satisfied in SVGD applications and compatible with Kernelized Stein Discrepancy (KSD) weak convergence control. 

Overall, this work contributes to the understanding of finite-particle SVGD and provides a unified convergence bound that quantifies the algorithm's convergence to its target. The derived bounds enable better control and evaluation of the accuracy of finite-particle SVGD in practical applications.

STRENGTHS:
The strengths are:
* The authors have proved the first unified convergence bound and rate for finite-particle SVGD.
* They show that SVGD with n-particles drives the KSD to zero at an order $1/ \sqrt{\log\log(n)}$ rate.

WEAKNESSES:
In my perspective, this paper would be more suitable for an optimization journal, as it would provide an environment where the technical contributions of the study can undergo a more comprehensive evaluation through an extended rebuttal cycle.

QUESTIONS:
None.

LIMITATIONS:
None.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-07-04

comments: SUMMARY:
The authors provide the first convergence guarantee for finite particle Stein Variational Gradient Descent (SVGD). Although I am not an expert on this topic, I believe this problem remained open for a long time, and it should be the first of many finite particle results to come. 

While $(\log\log n)^{-1/2}$ is a fairly slow rate, I believe we should overlook the rate, but consider the significance of the technical leap taken by the authors to achieve a finite particle result at all. For this reason, I will recommend accept fro this paper.

STRENGTHS:
1. This is the first finite particle convergence guarantee for SVGD. 
2. The contents are well organized and presented. 
3. The proof is concise and clean.

WEAKNESSES:
N/A

QUESTIONS:
Given that I am not an expert on this subject, I would like the authors to clarify a couple of questions for me. 

1. What was the main conceptual challenge in establishing a finite particle guarantee, and how did this work overcome it? For me Theorem 3 reads like a bit of magic, and the desired result just appears without much intuition. I would like to understand how this came to be. 

2. What do the authors perceive as the next challenge preventing improvements to this result? Similar to the previous question, I don't quite see on an intuitive level where the $log log n$ dependence came about, which the authors also believe can be improved. I would appreciate the authors can elaborate further on this topic.

LIMITATIONS:
N/A
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-04

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work studies the non-asymptotic convergence rate of Stein variational gradient descent (SVGD), an algorithm for approximating a target probability distribution with a collection of particles. This work presents a finite-particle convergence rate for SVGD, which provides a measure of how quickly the algorithm converges to the target distribution with a finite number of particles. The convergence rate formula drives the kernel Stein discrepancy to zero at an order 1/√log log n rate, but the authors suspect that the dependence on n can be improved and hope that their proof strategy will serve as a template for future refinements.\n\nSTRENGTHS:\nThis work provides the first finite-particle convergence rate for Stein variational gradient descent (SVGD), which is a popular algorithm for approximating a probability distribution with a collection of particles. This work presents an explicit, non-asymptotic proof strategy for the convergence rate formula --- which drives the kernel Stein discrepancy to zero at an order $1/\\sqrt{\\log \\log n}$ rate providing a measure of how quickly the algorithm converges to the target distribution with a finite $n$ particles. Prior to this work, relatively little is known about SVGD's non-asymptotic approximation quality, despite that SVGD has demonstrated promising results for various inferential tasks. The authors also claims that it serves as a template for future refinements to improve the dependence on $n$. I highly believe in this, due to the soundness of technical tools the work adopted.\n\nThe authors provide a thorough discussion of the assumptions and conditions required for the convergence rate formula to hold, which helps to clarify the limitations and applicability of the formula. Finally, this work also includes a comprehensive list of references to related work, which provides a useful starting point for further research on SVGD and related algorithms.\n\nWEAKNESSES:\nDespite its success in providing the first non-asymptotic convergence rate, this work assumes a level of familiarity with the mathematical concepts and notation for experts, which may make it difficult for readers without a strong background in probability theory and optimization to follow. In addition, this work does not provide any experimental results or comparisons with other algorithms to demonstrate the practical usefulness of the convergence rate formula, and does not provide any specific information on how the dependence on $n$ can be improved, which may limit its usefulness for researchers looking to optimize the performance of SVGD. Lastly, this work focuses exclusively on SVGD and does not provide any insights or comparisons with other algorithms for approximating probability distributions, which may limit its broader relevance to the field.\n\nQUESTIONS:\nI am at an educative level but quite enjoy reading on this topic. I quite like the generic / infinite-particle continuum manner SVGD Algorithm 2 is written instead of the $n$-particle SVGD Algorithm 1. I wondered if this continuous-time approximation is the critical reason for the success of the non-asymptotic rate first established by the authors.\n\nLIMITATIONS:\nThis paper contributes as a theoretical work and does not raise negative social impacts.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-08"}