Improved Finite-Particle Convergence Rates for Stein Variational Gradient Descent
Authors: Sayan Banerjee
Krishna Balasubramanian
PROMIT GHOSAL
Venue: ICLR 2025 Oral (2025)
Keywords: Stein Variational Gradient Descent
Non-asymptotic Rates
Variational Inference
Primary area: learning theory
TL;DR: Near-optimal finite-particle, discrete-time rates for SVGD

Abstract: We provide finite-particle convergence rates for the Stein Variational Gradient Descent (SVGD) algorithm in the Kernelized Stein Discrepancy ($\KSD$) and Wasserstein-2 metrics. Our key insight is that the time derivative of the relative entropy between the joint density of $N$ particle locations and the $N$-fold product target measure, starting from a regular initial distribution, splits into a dominant 'negative part' proportional to $N$ times the expected $\KSD^2$ and a smaller 'positive part'. This observation leads to $\KSD$ rates of order $1/\sqrt{N}$, in both continuous and discrete time, providing a near optimal (in the sense of matching the corresponding i.i.d. rates) double exponential improvement over the recent result by~\cite{shi2024finite}. Under mild assumptions on the kernel and potential, these bounds also grow polynomially in the dimension $d$. By adding a bilinear component to the kernel, the above approach is used to further obtain Wasserstein-2 convergence in continuous time. For the case of `bilinear + Mat\'ern' kernels, we derive Wasserstein-2 rates that exhibit a curse-of-dimensionality similar to the i.i.d. setting. We also obtain marginal convergence and long-time propagation of chaos results for the time-averaged particle laws.

Published as a conference paper at ICLR 2025
IMPROVED FINITE-PARTICLA CONVERGENCE RATES FOR STEIN VARIATIONAL GRADIENT DESCENT
Sayan Banerjee
Krishnakumar Balasubramanian Department of Statistics University of California, Davis kba l aaalaauucviiii edu
Department of Statistics and Operations Research University of North Carolina, Chapel Hill sayangema 11 unc. edu
Promit Ghosal
Department of Statistics
University of Chicago promi tuuch cago. edu
ABSTRACT
We provide finite-particle convergence rates for the Stein Variational Gradi- ent Descent (SVGD) algorithm in the Kernelized Stein Discrepancy (KSD) and Wasserstein-2 metrics. Our key insight is that the time derivative of the relative entropy between the joint density of N particle locations and the N-fold product target measure, starting from regular initial distribution, splits into a dominant 'negative part' proportional to N times the expected KSD and a smaller 'positive part'. This observation leads to KSD rates of order I/vN. in both continuous and discrete time, providing near optimal (in the sense of matching the correspond- ing i.i.d. rates) double exponential improvement over the recent result by [Shi & Mackey (2024). Under mild assumptions on the kernel and potential, these bounds also grow polynomially in the dimension d. By adding bilinear component the kernel, the above approach used to further obtain Wasserstein-2 conver- gence in continuous time. For the case of 'bilinear + Matérn' kernels, we derive Wasserstein-2 rates that exhibit a surse-of--imensioone ting. We also obtain marginal convergence and long-tima propagation of chaos results for the time-averaged particle laws.
similar to the i.i.d. set-
INTRODUCTION
Stein Variational Gradient Descent (SVGD) (Liu & Wang 2016) is widely-used deterministic particle-based algorithm for sampling from a target density и exp(- -V). where Rd + E the potential function. For given symmetric, positive-definit kemel k : Id x Rd + I, discrete time-step n E No, step-size > O, and for sis N. the SVGD algorithm given by
2 (n (1+ (n)
7 Σ (M))N()) (M)) (n), N
(I)
SVGD provides compelling alternative to classical randomized sampling algorithms like Markov Chain Monte Carlo (MCMC) that require additional uncertainty quantification with respect to the algorithmic randomness. It has attracted considerable attention in the machine learning and applied mathematics communities because of its fascinating theoretical properties and broad range of applications (Feng et al., 2017 |Haarnoja al. 2017 Lambert al. 2021 Liu al. 2021 Xu let al. 2022)). Our focus in this work is on deriving rates of convergence of the SVGD algorithm equation| and the corresponding continuous-time, N-particle SVGD dynamics on Rd obtained
by letting + 04, given by
1 Σ N Vek(a, (0, (0),
( = N
k(a (I," +
(2)

---

Published as a conference paper at ICLR 2025
with v denoting the time derivative and V2 represents gradient with respect to the second argument. Throughout the paper, we will implicitly assume the existence of a solution to the above equation
and the completeness of the vector field driving the above dynamics in (Rd)** N' This is satisfied, for example, under standard continuity and linear growth assumptions on the driving vector field.
The motivation for SVGD originates from the gradient flow for the relative entropy (i.e., the KL divergence) on the Wasserstein-2 space of probability measures on Rd More precisely, for proba- bility measure jr on Rd possessing regular enough positive density, the Wasserstein gradient flow is given by the measure-valued trajectory 14, satisfying the continuity equation
OeA2 + V (P(Pt) = O, Po 14.
where = - log(pt/T) and p; is the density of μ4- Under suitable conditions, He can be shown to converge (often with quantifiable fast rates) т. Unfortunately, this approach is not practically implementable via particle discretization as the associated empirical measure approximating μe does not possess density.
In a very influential paper. Liu & Wang (2016) devised a projected gradient descent algorithm by projecting the velocity vector V along a reproducing kernel Hilbert space (RKHS) associated with a symmetric positive definite kernel k. This leads to flow analogous to equation 3 but with e/ Pp, log(pe/T)) where the projection PR. is given by Pzf(x) == J [(a,,, (y) for probability measure v and function f Rd + R for which the integral is well-defined. The key observation Liu & Wang (2066) was that, by applying integration by parts, one obtains
Pa.. V log(pt/T) = / (-k(I, ( V)y + V2k(r, y)) iu (dy).
The right hand side is well-defined even when μr lacks a density and is hence amenable to particle discretization, which leads to the SVGD equations equation | and equation 2 See Korba et al. (2020) for a more detailed description of this approach.
Challenges for finite-particle SVGD: There has been extensive work in quantifying convergence
rates for the mean-field SVGD equation (see 'Past works' below). However, only (Shi & Mackey) (022)) Liu et al. (2024) have made attempts towards obtaining rates for the finite-particle version of (determunistic) SVGD. This has been perceived as a challenging open problem till date. The tractability of the mean-field SVGD equation comes from the observation that it has (projected) gradient structure which leads to the following monotonicity property of the KL-divergence:
O.KL(Mel [T) - KSD (Mell t2 0,
where KSD stands for the Kernelized Stein Discrepancy (Chwialkowski et al. (2016): Liu et al. (016)) Gorbam & Mackey| (2017)) The non-negativity of KL then leads to bounds on the KSD. For the finite-particle versions equation and equation 2 there is no gradient structure the dynamics, which renders the above approach inapplicable. Moreover, the vector field driving the finite-particl dynamics is not globally Lipschitz and lacks suitable convexity properties. This results in double- exponentially growing bounds in time between the particle empirical distribution and the mean-field limit (see Lu et al.] (2019) Prop. 2.6) and [Shi & Mackey (2024) Thm. 1)). a consequence, attempts to demonstrate finite-particle convergence to the target distribution и by relying on the mean-field convergence and the convergence of the mean-fiele fiead equation to the target in the general Gonn-Gassiin) setting lead to a slow convergence rate of 1/vlog log N. In Das & Nagaraj 2023)) the authors intentionally bypassed this approach for the kmite-particle setting, achieving improved convergence rates, but their method requires distinct, albeit related, algorithm that incorporates additional randomness into the dynamics.
Our contributions: A key insight in this paper to work with the joint density of the particle locations, when started from a suitably regular initial distribution, and track the evolution of its relative entropy with respect the f-llld product measure ONN, It turns out that the time derivative of this relative entropy has 'negative part' that is exactly N times the expected KSD2 of the empirical measure at time / with respect to n, and a "positive part' that can be separately handled and shown to be small in comparison the negative part (see equation 9)) This gives novel connection between the joint particle dynamics and the empirical measure evolution.
Our first main result, Theorem pected KSD between J N Jo N (a (t)dt and for the continuous-time SVGD dynamics in
1 exploits this observation to obtain O(1/VN) bounds for the ex-
2

---

Published as a conference paper at ICLR 2025
equation Analogous bounds for the discrete-time SVGD dynamics in equation| are obtained in Theorem Together, these results constitute a double exponential improvement over Shi & Mackey 2024 for the true SVGD algorithm. As discussed in Remark the bounds in Theorem | aare essen- fially optimal when compared with the KSD in the i.i.d. setting, and grow linearly in d (that is, KSD is OddvVN) V))) under mild assumptions on the kemel and the potential. Moreover, unlike previous works even for the mean-field SVGD, we do not require any assumptions on the tail behavior (such as Jub-Gaussianity) of the target π in Theorem | Further, it follows from Gorham & Mackey (2017) that the KSD bound alone does not even guarantee weak convergence of the particle marginal laws as N + 00, unless one establishes tightness of these laws. Our approach gives control on the relative entropy of the joint law in time which, in turn, gives the desired tightness and weak convergence for
the time-averaged particle marginal laws j (.) = * So (() E -)d(, for exchangeable initial conditions, see Theorem
The discrete-time SVGD bound in Theorem 3] although similar in flavor to Theorem D is sub- stantially more involved and requires careful control on the discretization error. In this result, we incorporate a parameter a that lets us interpolate between exponential tails and Gaussian tails as varies from to 1/2. This parameter turns out to be crucial in choosing the step-size 7, which is R d (z1-aa V1) N T- and the number of iterations required R N obtain KSD bounds which are O ( v1) N 11222). Unlike the population limit discrete-time SVGD rates previously obtained Korba et al. (2020); (Salim et al.] (202)) the Hilbert-Schmidt norm of the Jacobian of the transformation associated with each iteration depends non-trivially on the initial configuration and the number of iterations. A key technical ingredient in controlling this is an 'a priori' bound on the functional + N Σ, (Ix (n)) obtained in Lemma 3
In Section 4 we obtain Wasserstein-22 convergence and associated rates. For this purpose, we heav- ily rely the treatise of Kanagawa et al. (2022) which connects KSD convergence to Wasserstein convergence when the kernel has a bilinear component and a translation invariant component of the form (xry) + (x y) (see equation 11.. Such kernels are typically unbounded, in contrast with standard boundedness assumptions in most papers on SVGD (a notable exception is [iu et al. (2024). In Theorem 4 under dissipativity and growth assumptions the potential V. obtain polynomial KSD convergence rates for SVGD finite-particle dynamics with such kernels, which by Kanagawa et al. (2022) imply Wasserstein convergence. When the translation invariant part of the kernel is of Matérn type, we obtain Wasserstein convergence rates in Theorem 5 of the form (1/Na/d) (where a >> O does not depend on d) for the particle SVGD using Theorem in con- junction with results in Kanagawa et al. (2022). This is the first work on Wasserstein convergence for non-Gaune Man-Gausiian SVGD fimite- particle dynamics. Unlike the KSD bound, the d dependence leads to prrse-of-ddmeesiiondd Wasserstein bounds for empirical distribution of i.i.d. random variables (Dudley) 1969, Weed & Bach, 2019). Finally, we obtain a long-time propagation of chaos result in Proposition | namely, we show that the time-averaged marginals of the particle locations over the time interval [O, started from an exchangeable initial configuration, become asymptotically independent as + oo and essentially produce i.i.d samples from Although the results in Sections 4 and are proved for the continuous-time SVGD in equation 2 to highlight the main ideas, analogous results can also be proved in discrete-tin time and is deferred to future work.
in the Wasserstein bound, but this is be expected when compared to
Past works: The following diagram from Liu et al. (0024) highlights the major approaches under- taken in rigorously analyzing the SVGD dynamics:
Initial particles
Evolving partieles
(d)
Equilibrium
N (0) + E 8. (0)
N (t) = * Σx 82,(0)
(0o)
*
Σx 82,(
(a)
(e)
Initial density Po
Evolving donsity
(b)
Target (Fo
M4
(a) Unified convergence of the empirical measure for N < particles to the continuous target as time and N jointly grow to infinity; (b) Convergence of mean-field SVGD the target distribution over time;
3

---

Published as a conference paper at ICLR 2025
(c) Convergence of the empirical measure for finite particles to the meen--ielle distribution at any finite given time t E [0, oo);
(d) Convergence of finite-particle SVGD to equilibrium over time;
(e) Convergence of the empirical measure for finite particles to the continuous target at time
(a) is the ideal outcome that completely defines the algorithmic behavior of SVGD, One approach towards this is to combine (bbberr())nndd (c) or
Practically speaking,
(c) [
and
a quantitative way
to yield
(a)
Regarding (b) Liu (017)) showed the convergence of mean-field SVGD (solution to
equation 3
with it = P fie log Pt/n ) in KSD which is known to imply weak convergence under
appropriate assumptions. Korba et al. 2020); ; Chewi al. (2020); Salim et al. (2022); Sun et al.
2023 Duncan et al.] (2023) sharpened the results with weaker conditions or explicit rates. He et al.
2024 extended the above result to the stronger Fisher information metric and Kullback-Leibler
divergence based on regularization technique. Lu et al. (2019); Gorham et al. (2020): Korba al. (220) obtained time-dependent mean-field convergence (c) under various assumptions using techniques from partial differential equations and from the literature of 'propagation of chaos'. In
particular, Lu et al. (2019) derived the mean-field PDE equation 3 for the evolving density that emerges as the mean-field limit of the finite-particle SVGD systems, and showed the well-posedness
of the PDE solutions. Carrillo & Skrzeczkowski (2023) established refined stability estimates in comparison to Lu et al. (2019) for the mean-field system when the initial distribution is close to the target distribution in a suitable sense. In particular, they increase the length of the time interval in which mean-fiell -ield approximation is meaningful from xe log log N to = VN for such initial data close to the target.
[c) and combined them with (b) to get the first uni- fied convergence (a) in terms of KSD. However, they have rather slow rate of order 1/ /log log N, resulting from the fact that their bounds for [c] still depend on the time (sum of step sizes) double- exponentially, Note that studying the convergence (d) and [e) provides another way characterize the unified convergence (a) for SVGD. Liu et al. (2024) analyzed this strategy for the Gaussian SVGD case where the target distribution and initial distribution μ are both Gaussian and the kernel k is bilinear. In this case, the flow of measures for the mean-field SVGD remains Gaussian for all propagation of chaos
Shi & Mackey (2024) obtained refined results for [c)
time and this fact was exploited obtain detailed rates and 'uniform-in-time results. Das & Nagaraj (2023) obtained polynomial convergence rate (O(N-a) for some a > 0) for a related but different algorithm, which they called SVGD with virtual particles, by adding more randomness to the dynamics and using stochastic approximation techniques. The recent work of Priser et al. (2024) also studies finite-particle asymptotics, albeit for not the original SVGD it- erates (as in equation I1 but for modified one where Langevin-type regularization including Gaussian noise is added at each step. Hence, they leverage existing techniques for Langevin Monte Carlo to establish their results. However, their techniques are not applicable to the deterministic SVGD system in equation
Notation:
We will say a function f is C* it l times continuously differentiable in its arguments.
((X) will denote the law of the random variable X. We let B(TR") denote the Borel sigma-al iigma-algebra on Rdd We use π7 SN N-fold product target measure, i.e., @ (x ( #x) π (x1)
a (xNN) ).
Throughout the article (particularly in the proofs), we will often suppress the superscript N for various objects when it is clear from context. Furthermore, underlined vectors (e.g., 2) denote objects in (Id4) NV
2 CONTINUOUS-TIME FINITE-PARTICLE CONVERGENCE RATES IN KSD METRIC
We first provide rates in the KSD metric. Let Ho denote the reproducing kernel Hilbert space (RKHS) of real-valued functions associated with the positive definite kernel k (Aronszajn 1950). Then H = Ho x x Ho inherits a natural RKHS structure comprising IR -vauued functions. The Langevin- Stein operator (Gorham & Mackey 2015) Ta. associated with क x e acts on
differentiable functions ó: Ed R by
Ta((s) = - VV(x) ②(x) + + ((x), x HERd
4

---

Published as a conference paper at ICLR 2025
The Kernelized Stein Discrepancy! (KSD) (Chwialkowski et al. 2016 Gorham & Mackey 2017 associated with the kernel k, of a probability measure P on IR" with respect to π is defined as
KSD(PI|=) == [Tx((X)) X~
H, | S 11.
The definition of KSD motivated by Stein's identity which says that, for any sufficiently regular 6, Ex-= [T] 0 and thus, the above measures the 'distance' of P from x via the maximum discrepancy of this expectation from 0 when X P. as Φ varies over
The appeal of KSD lies in the fact that, unlike most distances on the space of probability measures, KSD has an explicit tractable expression. The function 6" E for which the above supremum attained has closed form expression ø (x) x Ey-p =((()))VV(() - vik(Y, Using this, we get the following expression for KSD:
(PIIT) = E(X,Y)~PPP (X) (MXX,))VV(Y - VV(()) V2)0(,
-VV(Y) vik(X, + V1 Vek(X, (5)
Before proceeding, we introduce the following regularity conditions, and state an existence and regularity result (proved in Appendix A.I for the joint particle density. Assumption 1. We make the following regularity assumptions.
(a) The maps (x, y) + k(z, v) and x + V(x) are C
(b) xN (0) = (x (00)))))...)())))) has a C² density PD-
Lemma 1. Consider the SVGD dynamics equation 2 Judder Assumption 7 Then the particle loca- tions (Z1(t)...,,xx(t (t) have a joint density p (t, -) for every 2 O, and the map (f, + p (t, 2) C².
The proof of this lemma is deferred to Appendix A.I Now we proceed to bound KL-divergence between the joint density of N-many particles at time / and the N-fold product measure of the target distribution π.
Denote the KL-divergence as
p (t, T
(L (L(p (f)[=TN)
loog
p. (t, 2)22.
(6)
T2N(2)
The following theorem furnishes the key bound on the KSD between the empirical law
N
(1)
5m(())
t t20. 0,
and the target distribution π. Define
C²(2) = V2k(z,) VV(2) + k(z, (z) A2k(Z
2 Rd.
In the above, V2k(z, == V2k(z,)) and A2k(z, = A2k(z, (Z)).
Theorem 1. Let Assumption 7 hold. Then, we have for every T
// EIKSD2 (+)l==)dd < KL(D (0) T
1
SN
N
E
Σ
(xx(t)
dt,
NT
N2T
L==1
where the expectation with respect to p(t). In addition, we have that
kL(p (T))(TT)) m < KL(U" (0)]l
J E
Σ
C
(ca(()) dl.
Moreover, if = suppped C
(* (z) < oo and limsup KL (O)lIT&N)/N N
co, then
(E(KSD
| |=))
N / N
EIKSD2 (n (Dll=))d
sup,
KL(p 784)
L
C*
(8)
N
where u (dx) = Jo N n (t, dr)dt.
See Barp et al. 2022
Section 2.2) for additional technical details regarding the well-definedness of KSD.
5

---

Published as a conference paper at ICLR 2025
The condition C* < 0C holds. for example, when k(u,v) v) = (u - v) for positive- definite C function V R + R. (Bochner 1933), and suPxeR" (v < cc. Examples of such kernels include the radial basis kernel (e.g., Gaussian) and wide class of Matérn kernels. The condition on the potential allows for a large class of non-log-concave densities as well. The condition lim sup KL(p" (0)||=(N)/ )/N < holds, for example, if we set the law of x. (0)
Remark 1.
N N-000
(a .0.... to be N where po is any probability measure on Rd satisfying KL(uol1=) D0.
Remark 2 (Optimality and dimension dependence). According to Sriperumbudar (016)) Ha-
grass al. (2024), we have under mild regularity conditions, that the empirical measure PN
Ei= 8x, where X; P. i.i.d,, satisfies []KSD(Pxx|PP) = O(1/VO)) This points to the fact that our rates in Theorem ① are presumably optimal with respect to N. While there is no curse- df-dimensionaliy in the KSD rates, the dimension factor appears in the rumerator of the bound equation 8 When k(u, v) V( - v) in Remark 7 with supgeRd (V S Cd for some di- mension independent constant C, it can be checked that C I (O) (O),,, which gives a linear in d upper bound on C* for wide range of kernels (including the Gaussian kernel) and potentials. Moreover, as long as mild regularity conditions are assumed about the kernel and the potential function, then according to Vempala & Wibisono (2019 Lemma 1), the initialization de- pendent term could be taken to be linear in d when has Lipschitz gradients. These combine to give an O(d/ /)) bound on the KSD.
Using the particle dynamics equation 2 and integration by parts, it is easy to verify that p(t, 2) is a weak solution of the following N N-body Liouville equation (see, for example, Golse et al. (2013) Pg. 7) and Ambrosio et al. (2005 Chapter 8)) given by
Proof of Theorem 7
We will abbreviate H(t) == (L(p (N
O.plt, 2) +
1
Σ k
div en (p(t, ((),, zt)) 0,
N
where $(2, w) == -k(z, w) (w) + Vek(z, w). Recalling equation 6 and using the density regularity obtained in Lemma D we have that
H'(1) = / 0,p((, 2) + / log
p(1,2)
T TBN Oep(1,
/
2(t,2)
Σ log
div en (p(t, ((t,,))
2) d2)
(2)
k.E
1
Σ
ZE log
2(t,2) (((k, =2))
N
(3N(2)
k.
Σ
VeLp(t,2)
((22 24 dz + N
1
Σ
VV(2)) p(2,, ze)p(t,
Σ /
(-divz, D(26, + VV(=))
Φ zx)) 2)1,
N
Now, observe that
div., (121, 26) = div.,, (K(Ek: 24) (=)))) div:, (V2k(zı, 22))
V,K(Z,, 21) (2 V, Vzk(zz, 24) C'((z) (k-=o)
Similarly,
VV(x)) ((2,, 22) - VV())) (k(2 (2kz 22) VV())) + Vz())) V2k(zk,
Therefore, using the explicit form of KSD in equation 5 we have
div (zx, 21) + (*x)) 2 z)) = KSD? (M(2)] + Σ (*()))
k.
k

---

Published as a conference paper at ICLR 2025
where p12) 2/ Ei= 62.- Hence, have H((t) -NE]KSD²(" (0)1=))
L E ca (cn(t))
(9)
where we recall that (t) t Eim (0,()) is the empirical measure. Hence, we have
1 T h EIKSD2( (t)ll & NT
H(0)
/ E (ax(t)) dt, NºT
1
which completes the first claim. The entropy bound follows from equation 9
To prove the final claim, recall ( = 1 Jo μ² (t, dz)dt and note that the map Q + (SD(QI[T) convex, which follows immediately from the representation of KSD given in equa- tion 4 From this and repeated applications of Jensen's 's inequality, we obtain
N
J. 1 NJ.
1/ N L/ V (MM)))((MM())I
(I ||00]
BIKSD(W (t)l S
1
H(U) supL
L + C*
N EJKSD"(H" (t)ll)]dd
N
This completes the proof of the theorem.
We now address the convergence law of the time averaged marginals of single particle when the initial particle locations are drawn from exchangeable law. We defer its proof to Appendix A.2
Theorem 2. Suppose Assumption 7 holds, C < oo and let k(u, v) = v(x - y). where W
C3 function with non-vanishing generalized Fourier transform. Suppose also that the law Po of the initial particle locations (I1(0), (x(())....NN(()) is exchangeable for each N E N and
lim sup N
1 KL(p" (0]][=*N) c0. Define
N
N N F(21(t) A)dt, for A B(R").
1
(A
Then, N. nT, weakly.
DISCRETE-TIMG HINITE-PARTCIAL RATES IN KSD METRIC
In this section, we obtain KSD rates for the discrete-time SVGD dynamics given by equation I The dynamics can be succinctly represented as
inn + 1) = I(n) - ,T((x())), n E No, (Ti...., TN) with T In this section, use T to denote the number of
where N E, iterations.
the step-size and V2k(z,
T
Although the idea once again track the evolution of the relative entropy of the joint density of particle locations with respect to zNN, one needs careful quantification of the discretization error to obtain an equation similar to equation 9 We will make the following assumptions. All constants appearing in the section will be independent of d.
Assumption 2. We make the following regularity assumptions.
(a) Boundedness: k and all its partial derivatives up to order 2 are uniformly bounded by B E (0, oe).
(b) Growth: inf V(2) 0 and, for some 0. [0,1/2], [VV())] s AV for all x E Idd.
7

---

Published as a conference paper at ICLR 2025
(c) Bounded Hessian: sup,ER" | (2)]lop Cv co, where Hy denotes the Hessian of V and || loop denotes the operator norm.
(d) Recalling equation 7 we assume that supaER" V2k(2,2) VV(2) + 2(2,2)AV(2) constant c² > O.
42k(z, (,,(222) = c² d for
(e) Initial entropy bound: KL(p(0)]| (N S Nd some constant CKL >
Remark 3. Conditions (a) and (c) are essentially motivated by the work of Korba et al. (202)) who in nurn are motivated by standard assumptions made in the stochastic optimization literature. The condition (b) in Assumption 2 captures the growth rate of the potential V which plays a crucial role the step-size selection and number of iterations required (see equation [0 below) to obtain the convergence rate in Theorem 3 As approaches O, approaches the exponential distribution and as a approaches 1/2, a is close to a Gaussian (in tail behavior). Conditions (d) and (e) are explicit refinements of similar conditions required in the continuous analysis.
Theorem 3. Suppose Assumption from
holds. Let the initial locations (I1(0), (I,, [N)) be sampled 2
IN
Pk(0)= = P where SK = f (1O)) N : :N Σ ΣV(E)SK)
for some > O satisfying Jsx p(0,2)dz 2 1/2, where p(O)[Sk represents the restriction of the density to the set Sk- Then there exist positive constants a, b depending only on the constants appearing in Assumption such that with
N 1=
T= T=INT=
(10)
vdKa +
we have
T KSD2 (uN]lmT]
bel
1te d
+ Vdka +d)
E Px (0)
[1
N
0=
In particular, if (z1 (0),
NN (0)) is sampled from p(0), then for any e 0.
T-1
bd (d 1
+ vdka + +d)
P
KSD? (p
sct
p(0, 2)0.
T n=D
Ne
We prove Theorem 3 in Appendix
A.3
Remark 4. The proof of Theorem 3 involves an 'interpolation" in the spirit of Korba al. (2020) between the laws of g(n) and r(n 1) which leads to a Taylor expansion of the relative entropy. The main subtlety in the analysis when dealing with the joint law evolution here comes from the fact that, unlike for the population limit SVGD, the Hilbert-Schmidt norm of the Jacobian of the map T(z) at z = z(n) is not uniformly bounded in n and x()). This requires refined 'a priori' bounds the rate of growth of the path functional result, for initial locations in SK one can fine tune the step-size depending on N. d, K such that the second order term in the Taylor expansion becomes small in comparison the first term. Moreover, the first order term is shown to have the same form as the continuous-time derivative of the joint relative entropy obtained in the proof of Theorem This leads the results in Theorem 3
+ E, (II(n)) (see Lemma 3). As
Marginal convergence results, similar to Theorem 2 in the discrete-time setting can also be deduced from the entropy bounds involved in proving Theorem 3
4 CONTINUOUS-TINE FINITE-PARTICH RATES IN W2 METRIC
We now explore convergence rates for the continuous-time SVGD in the L J.-Sasserstein metric. For s > O, let P. be the set of all Borel measurable probability measures on IR with finite s-moment. For two measures μ,, P.. the L Wassersttii distance (based on the Euclidean distance) is defined as
Wo(,,)) = ( inf E( [|| Y1
1/8
(eEll())))
8

---

Published as a conference paper at ICLR 2025
where [[ v/) denotes the set of all possible couplings of the probability measures j and To address W2 convergence, we consider the SVGD dynamics in equation with kernels the form
k(u, v) = 1 + (u, v) + V ( - v).
(11)
We further assume that the kernel obtained by (u, v) + (r - v) positive-definite, v=()) = (-=)) for all E RR, suPzER4 [=]] < oc, V E C and has non-vanishing and continuous generalized Fourier transform. A classical example of a kernel that satisfies these assumptions the Matérn kernel. We first state the following dissipativity and growth assumptions from Kanagawal et al. (022) along with an additional Laplacian growth condition.
Assumption 3. The following conditions hold:
(a) Dissipativity: The potential V satisfies (z. VV(x)) & -a||=|1+++:1111 + (Bo for some >> 0 and $1. Po 2 0.
(b) Growth: [V _ A1 + [[=])), for some No > 0.
(c) SUPzER4 (v cc.
The above assumptions are widely used in the MCMC literature (e.g.. Raginsky et al. (2017) and are satisfied in many cases including certain Gaussian mixture models, and allow for some degree of non-log-comee wow.ooootyvvttyyoooooooooooooyye æ.
Theorem 4. Let Assumption SKK.suuNNNLLL((OO0 N-+00
3 hold.
Assume that the initialization is such that 0 and let M M(N) [N2+0]. Then, there
co. Fix any
exists a constant Co > 0 such that
Co
E|KSD(P M |
N1+o/2' AA 21
and W2(P M TT) = as N 00,
where recall pM (dx) M Soth (t, dz)dt.
Theorem proven in Appendix A.4 By the results Kanagawa et al. 2022 Section 3.2), we can translate the KSD bound in Theorem 4 into bound in the W₂ metric. While a more abstract result for the choice of kernel in equation
could be obtained by leveraging Kanagawa et al. (0022] Theorem 3.2), for the sake of concreteness,
b restrict ourselves to the Matérn- family of kernels.
Specifically we consider equation 11 with
21-(//2+
(u, 2) = 1 1 (u, + T(d/2+ [[u - uIlkk e))l2).
(12)
(w-v
where I is the Gamma function, a strictly positive definite matrix, and K_,, the modified Bessel function of the second kind of order -v.
To proceed, we also require the following assumption from Kanagawa et al. 2022 which moti- vated by the Langevin diffusion:
dz, (Z,)dd + V2dB,,
(13)
where (B)) is d-dimensional Brownian motion. Note that equation 13 is the stochastic differ- ential equation equivalent of the Wasserstein gradient flow in equation 3 see, for example, Pordan al. (1998;; Bakry al. (2014) for details. The connection between the two perspectives has in particular proved to be extremely useful for analyzing both Markov Chain Monte Carlo algorithms and particle-based methods.
Assumption 4. For s E (1,2), let Ps oo) + [0, be an upper bounding function for the J--assserstein distance in the following sense:
Wa(C(Z)) )(2")) S p.(t))l=- vll2, Vx, y RR, 1 R 0,
(14)
where Z and Z are Langevin diffussin processes in equation 13 with initializations Zo x and log
#1 log palt)
Zo = Assume that p(1) =
11107
is uniformly bounded in and, moreover,
P10 log
P1(D)
[ pl(() (1) (()()) dt co.

---

Published as a conference paper at ICLR 2025
Remark 5. Suppose there exist U > 0 and R, 2 0 such that the potential V satisfies
(vV()) (v) x y) 1 {
U
> R.
(15)
v
L -- S R.
Then, by Eberle 2011
2016
there exists c, C1 0 such that
set P1 (t) ce cet 0.
Moreover, using equation 15 and Grönwall's lemma, we can set p2(t) elt t 2 0. Consequently,
p(t) L+o and hence Assumption holds.
Theorem 5. Consider the SVGD updates in equation 2 with the kernel in equation 12 Suppose As- sumption 4 and the assumptions made in Theorem 4 are satisfied. Then, with o, M as in Theorem 4
there exists constant C(d) >> O such that for any E (0, 1), we have
P Wa(ual T) C(d)
(
(
Co
r(d)
Se
VN Z
Co
240
where Co is the same constant from Theorem 4 2022 Theorem 3.5), and
C(d) is the constant Cp. (1) from Kanagawa et al.
rdd)
(16)
3( 1d+11
+
We prove in Theorem 5 in Appendix A.5
Remark 6. Note that r(d) 18d for large d. Hence, unlike the KSD rates in Theorem the W2 ofrrs--o--immmsionaae dimensionality. However, this is expected, as even in the case of i.i.d. samples, rates have a curse-of
have similar curse-of- of-eiiinssionaliiy (Dudley) 1969; Weed & Bach 2019). Intuitively, this can understood by observing that convergence KSD captures convergence of expectations for class of test functions that is much smuller than that for Wasserstein convergence (see Gorham & Mackey (2017)). The latter class is large enough to be highly sensitive to the effect of growing dimension, thereby exhibiting the curse-of-dingg oiieiiiioiiniiiiiiy.
5 PROPAGATION OF CHAOS
We now exhibit long-time propagation of chaos (POC) for the particle system started from an exchangeable initial configuration and driven by the dynamics equation More precisely, we show in the following proposition that, under the conditions of Theorem 4 the timeraveraged marginals of particle locations over the time interval [0, .V] become asymptotically independent, with distribution π, as N co. This result shows in particular that, unlike traditional MCMC schemes (Brooks al.) 2011), the SVGD algorithm provides multiple i.i.d. approximate samples from the target distribution T. We defer its proof Appendix [
Proposition 1. Suppose that the law Pe of the initial particle locations (x (o)(0),... (O)..... N (0)) is exchangeable for each E N. For S k < N, define the k-dimensional marginal of the time- averoged occupancy measure of particle locations as follows:
1
/ P(xN E A..... x E Ax)dt, for Au...., Ag E B(Rd).
Pk (A1...., Av) N
Recall M = ((N) ( 72 Under the same setting as Theorem we have that, for any fixed k E N. Wi =²k) 4 0, as N + co.
Theorem 2) and Lu et al. 2019 Proposition 2.6) where finite-time POC results shown: the particle marginal laws fixed time become asymptotically independent as > cc. This follows upon observing that the con- vergence of empirical measures shown in these papers implies a fimite-time POC (Chaintron & Diez 2022 Proposition 9 and Theorem 3.21). However, owing to the lack of Lipschitz property of the vec- tor field driving equation this POC can only be extended to growing timest = tN O log N) when the particle marginal laws are not necessarily close to т. In contrast, Proposition | extends to the time interval [0,, N) (hence, long-time POC) and the time-averaged particle trajectories essen- tially produce ii.d. samples from n.
Remark 7. This result should be compared with Shi & Mackey 2024
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENTS
KB was supported in part by National Science Foundation (NSF) grant DMS-2413426. SB was supported in part by NSF CAREER award DMS-2141621 and NSF RTG grant DMS- 2134107. We thank Lester Mackey for his insightful comments on an earlier version of this work and for valuable discussions regarding the broader literature on KSD and SVGD.
REFERENCES
Luigi Ambrosio, Nicola Gigli, and Giuseppe Savare. Gradient Flows: In Metric Spaces and in the Space Probability Measures. Springer Science & Business Media, 2005.
Nachman Aronszajn. Theory of reproducing kernels. Transactions of the American mathematical sociery, 68(3):337-404, 1950.
Dominique Bakry, Ivan Gentil, and Michel Ledoux. Analysis and Geometry of Markov Diffusioo Operators, volume 103. Springer, 2014.
Barp, CJ Simon-Gabriel M Girolami, and L Mackey. Targeted separation and convergence with kernel discrepancies. arXiv preprint arXiv:2209. 12835, 2022.
Salomon Bochner. Monotone funktionen, Stieltjessche integrale und harmonische analyse. Mathe- matische Annalen, 108(1):378-410, 1933.
Steve Brooks, Andrew Gelman, Galin L. Jones, and Xiao-Li Meng (eds.). Handbook of Markov Chain Monte Carlo. Handbooks Modern Statistical Methods. Chapman and Hall/CRC, Boca Raton, FL, 1.1 ISBN 978- 1420007944-8
Amarjit Budhiraja and Paul Dupuis. Analysis and approximation of rare events. Representations and Weak Convergence Methods. Series Prob. Theory and Stoch. Modelling, 94, 2019.
José A Carrillo and Jakub Skrzeczkowski. Convergence and stability results for the particle system in the Stein gradient descent method. arXiv preprint arXiv:2312.16344 2023.
Louis-Pierre Chaintron and Antoine Diez. Propagation of chaos: A review of models, methods and applications. I. Models and methods. Kinetic and Related Models, (5(6))895-1015, 2022.
Sinho Chewi, Thibaut Le Gouic, Chen Lu, Tyler Maunu, and Philippe Rigollet. SVGD as kemel- ized Wasserstein gradient flow of the chi-squared divergence. Advances in Neural Information Processing Systems, 33:2098-211 -2099, 2020.
Kacper Chwialkowski, Heiko Strathmann, and Arthur Gretton. kernel test of goodness of fit. In International Conference Machine Learning, pp. 2606-2615, PMLR, 2016.
Gianluca Crippa. The flow associated to weakly differentiable vector fields. PhD thesis, University of Zurich, 2008.
Aniket Das and Dheeraj Nagaraj. Provably fast finite particle variants of SVGD via virtual particle stochastic approximation. Advances in Neural Information Processing Systems, 36:49748-49766 2023.
Richard Dudley. The speed of mean Glivenko-Cantelli convergence. The Annals of Mathematical Statistics, 40(1):40-50, 1969.
Andrew Duncan, Nikolas Nüsken, and Lukasz Szpruch. On the geometry of Stein variational gradi- ent descent. Journal of Machine Learning Research, 24:1-39, 2023.
Andreas Eberle. Reflection coupling and Wasserstein contractivity without convexity. Comptes Rendus Mathematique, 11t 2011.
Andreas Eberle. Reflection couplings and contraction rates for diffusions. Probability theory and related fields, 166:851-886. 2016.
11

---

Published as a conference paper at ICLR 2025
Yihao Feng, Dilin Wang, and Qiang Liu. Learning to draw samples with amortized Stein variational gradient descent. In Conference on Uncertainty in Artificial Intelligence, 2017.
François Golse, Clément Mouhot, and Valeria Ricci. Empirical measures and Vlasov hierarchies.
Kinetic and Related Models, 6(4):919-943, 2013.
Jackson Gorham and Lester Mackey. Measuring sample quality with Stein's method. Advances in neural information processing systems, 28, 2015.
Jackson Gorham and Lester Mackey. Measuring sample quality with kernels. In International Conference on Machine Learning, pp. 1292-1301. PMLR, 2017.
Jackson Gorham, Anant Raj, and Lester Mackey, Stochastic Stein discrepancies. Advances in Neural Information Processing Systems, 33: 1eee11931117999 2020.
Tuomas Haarnoja, Haoran Tang, Pieter Abbeel, and Sergey Levine. Reinforcement learning with deep energy-based policies. In International conference on machine learning, pp. 1352-1361. PMLR, 2017.
Omar Hagrass, Bharath Sriperumbudur, and Krishnakumar Balasubramanian. Minimax optimal goodness-of-fit testing with kernel Stein discrepancy. arXiv preprint arXiv:2404.08228,, 2024.
Philip Hartman. Ordinary differential equations. SIAM, 2002.
Ye He, Krishnakumar Balasubramanian, Bharath K Sriperumbudur, and Jianfeng Lu. Regularized Stein variational gradient flow. Foundations of Computational Mathematics, pp. 1-59, 2024.
Richard Jordan, David Kinderlehrer, and Felix Otto. The variational formulation of the Fokker- Planck equation. SIAM Journal on Mathematical Analysis, 29(1):1-17, 1998.
Heishiro Kanagawa, Alessandro Barp. Arthur Gretton, and Lester Mackey, Controlling moments with kernel Stein discrepancies. arXiv preprint vrXiv:2211.05408. 2022.
Anna Korba, Adil Salim, Michael Arbel, Giulia Luise, and Arthur Gretton. A non-asymptotic analysis for Stein Variational Gradient Descent. Advances in Neural Information Processing Systems, 33, 2020.
Alexander Lambert, Fabio Ramos, Byron Boots, Dieter Fox, and Adam Fishman. Stein variational model predictive control. Conference on Robot Learning. pp. 1278-1297. PMLR, 2021.
Qiang Liu. Stein Variational Gradient Descent as gradient flow. Advances in Neural Information Processing Systems, 30, 2017.
Qiang Liu and Dilin Wang. Stein Variational Gradient Descent: A general purpose Bayesian infer- ence algorithm. Advances in Neural Information Processing Systems, 29, 2016.
Qiang Liu, Jason Lee, and Michael Jordan. A kernelized Stein discrepancy for goodness-offfr tests. In International Conference on Machine Learning, pp. 276-284. PMLR, 2016.
Tianle Liu, Promit Ghosal, Krishnakumar Balasubramanian, and Natesh Pillai. Towards understand- ing the dynamics of Gaussian-Stein variational gradient descent. Advances in Neural Information Processing Systems, 36, 2024.
Xingchao Liu, Xin Tong, and Qiang Liu. Sampling with trusthworthy constraints: A variational gra- dient framework. Advances in Neural Information Processing Systems, 34:23557-23555 2021.
Jianfeng Lu, Yulong Lu, and James Nolen. Scaling limit of the Stein Variational Gradient Descent: The mean field regime. SIAM Journal on Mathematical Analysis, 51(2):648-67 -711 2019.
Victor Priser, Pascal Bianchi, and Adil Salim. Long-time asymptoties of noisy SVGD outside the population limit. arXiv preprint 24K.:22006.19929 2024.
Maxim Raginsky, Alexander Rakhlin, and Matus Telgarsky. Non-convex learning via stochastic gradient langevin dynamics: a nonasymptotic analysis. In Conference on Learning Theory, pp. 1674-1703. PMLR, 2017.
12

---

Published as a conference paper at ICLR 2025
Adil Salim, Lukang Sun, and Peter Richtarik. A convergence theory for SVGD in the population limit under Talagrand's 's inequality Ti. In International Conference on Machine Learning, pp- 19139-19152. PMLR, 2022.
Jiaxin Shi and Lester Mackey. A finite-particle convergence rate for Stein Variational Gradient Descent. Advances Neural Information Processing Systems, 36, 2024.
Bharath Sriperumbudur. On the optimal estimation of probability measures in weak and strong topologies. Bernoulli, pp. 1839-1893, 2016
Lukang Sun, Avetik Karagulyan, and Peter Richtarik. Convergence of Stein variational gradient de- scent under a weaker smoothness condition. In International Conference on Artificial Intelligence and Statistics, pp. (693-3712 PMLR, 2023.
Santosh Vempala and Andre Wibisono. Rapid convergence of the Unadjusted Langevin Algorithm: Isoperimetry suffices. Advances in Neural Information Processing Systems, 32, 2019.
Jonathan Weed and Francis Bach. Sharp asymptotic and finite-sample rates of convergence of em-
pirical measures in Wasserstein distance. Bernoulli, 25(4A):2620-261) 2019.
Lantian Xu, Anna Korba, and Dejan Slepëev. Accurate quantization measures via interact- ing particle-base based optimization. In International Conference Machine Learning, pp. 24576- 24595. PMLR, 2022.
A PROOFS
A.1 PROOF OF LEMMA
Let F: (Rd)* N + (R1) N be given by
Σ V2k(zz, 24), F(()) N Σ (V(2)) + N
for Sis N. By Assumption I Fi C2 map. Note that the SVGD particle trajectories can be written as (x) 2 0), where the flow x [0, oo) (R-) N (R2) N
is given by
x(1,2) (x 2)), x(0
with x denoting the time derivative. By Hartman (2002 Chapter 5, Cor. 4.1), the map (1,2) + x(t 2), and consequently, the inverse map (t, 2) + xtt. -) (2). are C2 maps on (0, 20) (R.)
A simple change of variable formula gives (see Crippa 2008 Page 21))
Po o x(t, (2) (t. 2) E (0, oo) x (R.)
p(t, 2)
det(Vx((t-)
The existence and regularity of -(: then follows from the above observations.
A.2 PROOF OF THEOREM 2
We will first show tightness of { J.v. By subadditivity of relative entropy (which follows from Budhiraja & Dupuis 2019 Lemma 2.4(b) and Theorem 2.6)), we have
(t))l= s [K(((())+*NN) 3N) ([] ON
C't N
N Hence, there exists C > 0 such that KL(C(z) (t))l== < C for all t E [J. VN > 1. Fix any c <> Let 5 > 0 such that 5C < c/2. Let be a compact subset of Rd such that llgg(1 (eK") 1)) S c/2. By the variational representation of relative entropy (Budhiraja & Dupuis, 2019 Prop. 2.3) and Theorem I we have
(())(1) & K) s o log (fedlm(())())))) + (L((((2((0))
<olog(1 + ((K)) 1)) 50 <c
13

---

Published as a conference paper at ICLR 2025
for all t E [0, AA 1. In particular, {Ø>) IN is tight. Moreover, have
1 N /." ((xr
KSD( KSD
$ N
(tD(
by the convexity of KSD
-M N (SD(B//M))]ll) using exchangeability, where Ell (lI (d)( = BB(u (1. dx))
-N
BJKSD( (+)) +
by KSD convexity and Theorem 7
The result now follows from Gorham & Mackey (2017 Theorem 7).
A.3 PROOF OF THEOREM 3
The proof of Theorem 3 proceeds through the following lemmas. We will assume throughout that Assumption holds.
$ V(r)) + +2NBad ge (R4)
Lemma 2.
Proof of Lemma
Observe that
[T)) N
Σ Σ (k(a,,
(((m),;))VV())) V2k(a;,
<
2
Σ
(VV
V2k(2,, 2,)1 [V
N
ij
B2A=B2N
Σ ((o2))a
+ 2
N
2a
1
V(r;)
+2NB'd
N
where the last step follows by Jensen's inequality noting a S 1/2.
The following lemma gives a key priori' bound on the growth rate of N-1 Σ, V(x(nn) in terms of n and 1) and is crucial to the rest of the proof.
Lemma 3. There exist positive constants M, D depending only on the constants appearing in As- sumption such that for 2 1, 0 S a S T, SIA VDT
Σ V(x((n)) & M
1 Σ [((r(())) [mm) v1] di + N
Proof of Lemma 3 Note that, using Taylor's 's theorem,
1
v(a;(n+1))
Σ V(x((n)) -Va((()) Σ (V((a xz(n))
N
1
Σ ( (1 (1-5) s),(nn+) - (n), Hy ( (n( + -))xx(n)))
+
N 30
1
Cvn²
<
24 (x)1 +
2N [T(z(n))]1*
N
14

---

Published as a conference paper at ICLR 2025
Now, applying Lemma 2 in the above and writing D1 By and D2 BvC,, obtain
Σ (Ix 1)) N
1
Σ (()(nn)
N
2a
$ Σ (V(( -,,,,,))))), + Din²
Σ V(z,(n)) N
D2dn2
7
Σ (VV(a;())) Vok(o.(1))
N2
N2
2a
1 DVvie ([(OS(O))
+Dr
+ D2dn'
N 5
7 N2
Σ
(z), V=((2((1) Din²
N Σ
((z((()))
+ D2dr)"
where, in the last step. we used the n of the right hand side to avoid cumbersome notation, we obtain
of k. Thus, suppressing the dependence on
1
Σ V( 1))
N Σ (zx(n))
N
2a
dn
+
1
Σ v(xs))
+ D2dn?
N
N
2a
s ABVdd
1 *I(a))
1
+ Dio
1
(((s) N
D2dd1
Thus, writing f(n) = # E; (I:())),, recalligg S 1/2 and using fnn)aa S 1 1+ f()), we obtain
f(n + 1) 5 5) (1+ + ABV + Dadn?
(17)
where D3 = D1 + D2. We will now harness the recursive bound equation 17 to obtain the claimed bound the lemma. First, we handle the case S [1/n]. Fix O. Define
Tz = su 2 0: (m) s L} [1/n].
Then for 1 S TL, equation gives for 17 1,
f(n) < (1 + D D ff AB + Dzdn?
(1+ ( + Dzdn?) + (1 ((0))"
1=
< (1+ D112)
+ Dzdn²) + (1+ 211
< 2c Di (ABVaL. + D3d) + (1 D: f(0)-
Hence, taking L M (d f(0)) for some suitably large M 2 1 depending only on the constants appearing in Assumption 2 we conclude from the above bound that f(TL) L and hence 7 [1/m]. Thus,
f(n) M ((O) for all 0 S n S [1/]]
Now, we handle the case 1/n S n $ T. We will proceed by induction. Let
8 M f(0).
where
M M [16 a)) Da)
15

---

Published as a conference paper at ICLR 2025
Take any 7 I 1 4(1-a)DDi7' Then, by the above bound, note that for [1/n], f(n) 1
B(nn) Suppose for some 1/n - n - T, f(n) - (nmm)--- Then, by equation 17
[(n 1) S (1 Div)) - + ABv 3
(mm) + Dzdrj
I-
ABVd
Dydy 1-a
- ( 117)
(1+ Din")
+
B1-an +
n+1
31 --
1
ABVd Dzd
S <B((n + 1)m) 1-
(1+ D
8(1 -
81-an
AB D3d
S BB((n + 1)m) -
(1 + DiTA)
8(1 - =x)
Mn-odn
S B((n + 1)m) -
D
16(1 - 2
S 0((n + 1)m)
where the fifth inequality uses the choice of M and the last inequality uses the bound 17. The claimed bound follows by induction.
We will now use Lemma
3 to obtain bounds on [[T(2()))| and the fillbet-Schhiin norm of the
Jacobian matrix (T(z(m))
Lemma 4, For T 2 1, LLLSSEINELL VDT
we have
2a
Σ v(oo(0))
[(m) vi] + BN
= M
d +
N
Σ v(c(o)) 20
[JT(z())) & 8AABBd(N +2)
M
di
1 + N
(mm))ta
+ & 2 +3) + +82 dCZ.
Proof of Lemma f bound, note that
The first bound follows from Lemma and Lemma
To prove the second
N Σ
(TT(')
Σ +.=1k.1m1
(2)
where for e E R.d, Uak denotes the kth coordinate of i; and 0 denotes the partial derivative with respect to Ezt- We will also write for m = 1, 2, Ömik to denote the partial derivative of k with respect the leth coordinate of the mth variable. Observe that,
N N
Ozz (z)
Ork((z,, 1(i = 5)
+
(O2i(1,; 25 (xx)) - 02ak(sn,z;))
N
1
- 2xck(rr,
N
V - O202nk(1;,
N
16

---

Published as a conference paper at ICLR 2025
Write Sm (i,5) for the math term in the above bound for 1,2,3,4. In similar manner the proof Lemma
Σ kyke
[S((,,) &
Σ 2,, -2,,k(21,
N
suk!
2WV - Fek(21,
N
iul
1
+ +2NB3d. [V(0))
- AN
N
Similarly,
Σ
(s2(1.1)1
1x) - 0zck(or,
N2
Gjki
2B
$
(z;;
N
1
2a
S Bdd
N[[((a)) v(as))
1
+2B28.
N
Moreover,
& 2B2 Σ Sa(t,))
Σ ((0r)() + +1)
N²
Kiki
sjkl
$ 2B
sup =ERd
[H>] +2 S 2
Finally,
20
Σ [SS((,1) <
2B N
Σ +1)) 2A 1
(.v(ss))
+2B²d.
N
ejkt
GjA
Combining the above bounds, we obtain
(
((ass) 2a
|)T())) <4 22 +
2B
N
The claimed bound in the lemma now follows from the above and Lemma
Now, we will complete the proof of Theorem
Proof of Theorem Fix K 0 in the theorem and sample (O).... (0) from pr px(0) supported on Sk- Denote the law of z(n) by (n(n). As we will work with fixed 0 < n S T for the first portion of the proof, we will suppress dependence on n. Set v(0) PK pK(n) and v(n) = (n + 1). Interpolate these laws by defining v(t) = (n), E 7). where (x(z) = x (x), [0, n).. Write SK = (Id- and SK.- = (r()),, e [0.
Set the step-size 7) as
1 Co
A N
N ) 8, where 0 [0.1]
1/2
=
N
fi
will be appropriately chosen later and Co = 2/24A2 B² 32dM2 (di-s + K)2a + 8B2 CC2 + +
By Lemma 4 this choice of 17 ensures that for any SK, t E [0,
0
[T)( S nl|]T(>))l
-
2 $ 2'
17

---

Published as a conference paper at ICLR 2025
Thus, Jo)( is invertible for any such 2, and
[I(J,, [ 2.
In particular, if v0) admits density, then for any t E [0, ny. (t) admits density given by Pk (n, oF (z)),
gk (t, 2: (dct(Jor(or
SK.
and (k) = 0 otherwise. Writing F(()) = (L(xx (0) 2N we obtain the following Taylor expansion on the interval [0, v] along the lines of Korba et al. (2020):
E(n) = E(0) + (0) + /.' ( (t)t))"(ttdr (t)dt.
(18)
clearly, E(O) (L (n)
n @N and F(n) KL(P + +1)11 Moreover, by computations
similar to Korba et al. (2020). writing (V(x) = (VV(x)),.... VV(xx)), we obtain for E [0, n).
(t) = tr ((Jo(()))+ PK (n. + / (VV() (, (,))))))))))))))))))))))))))))))
(1.)))knnn2))))) z)dx
= = (JOO(()))
(VV(or())) T(z))x z)dr.
In particular, recalling ( w) = w(2, + V2k(z,
E'(0) = / (div(T(z)) - (z), T(x)) px (n,
=
div o (x,,
1 Σ
(((a))(
pk pk(n, z)dr
S - (0)
KSDD (m |~] + c'd
along the same lines the proof of Theorem
| Moreover, note that for t [0, n).
E" (t) (t) (1)
where, using Lemma 4 and our choice of step-size 17.
0²
(t) Eu)px( (n) Hvlo) s Cv sup zE8K
[|({)) <
47p2
and
rb2(1) / (JT( PBs
|)T(') [Jo < sup ||)(())) 0
< sup geSk
zeSk
Combining the above observations, we obtain the following key 'descent lemma' for any 0 S n S
T:
(n + 1)][ @N S (L)px(m)] - NT)E [KSD* [|=]] + c² dn + 0².
Hence, for T 2 2,
(0) c'd 0²
Exx (0)
KSD 2 (|T))
T
NTn
N
Note that we have,
Po(x) 9x (Sx)
Po(x) ) log(uo (k)) dx
KL(px (L))(00))780)
log
T&N (x)
/ po(x) log
(o(x) N dr + log
(Sx(
Ho(Sk)
18

---

Published as a conference paper at ICLR 2025
Hence, under assumption that K satisfies (o(Sk) 2 1/2, have that
(L(px (0)p=() - 2KL(p(0)||=8))
+ log 2 ydN,
where 7 = 20 + +lg 2. Using this in the previous display and recalling the choice of we obtain
T-1
N
c²d
Exx (0)
KSD (T)
S Sod
+
Co
T
T8
N
N
n=0
and = /N/T The above expression is "approximately" optimized on taking = [NT
N xu mJ
") which gives the bound
for some KSD (|m) = (rd + 1) + t'd
T Σ
Epx (0)
The theorem follows from the above upon noting that v S c (d + vdka + constant c depending only the constants appearing in Assumption
+d)
A.4 PROOF OF THEOREM 4
The additional bilinear term in equation iis is the key to tackling Wasserstein convergence. It gives uniform control in N, '"" over the second moment of the particle locations in the SVGD dynam- ies equation 2 given in the following lemma.
Lemma 5. Under the same setting of Theorem 4
have that
lim sup sup E
Σ
(t)(t))
< oc.
N T21
T Jo
N
This result is proved using Lyapunov function techniques and plays a key role in the proof of Theo- rem 4
Proof of Lemma tions equation
For this proof, we will abbreviate x: as Note that, using the SVGD equa-
have
d
Σ V(za)
1 N
Vosss)
1
E
dt
N
N²
1=1
Di
VV(x:))
N Σ (V
(V
(19)
(V))
-
N2
The non-negativity claim above a consequence positive-definiteness of the kernel obtained by (u, v) + (u v). Note that
1
(o,, (VV,
Σ
S . (x;)))
N
1
f.
1=1
N
Zz, z4e (VV
t=1
Zz,l VV (zx))e
t=1
( VV(I,))
d
19

---

Published as a conference paper at ICLR 2025
where the penultimate step follows by Cauchy-Schwarte inequality. Using the above inequality in equation
we obtain
d
N
v(a,)
S Σ Js vV(a.))
VV
dt
(20)
1 N
Σ (zz VV(z))
1
(V(x)) (
N2
By Assumption
3 there exists A, ax 3, such that
(a, VV(x)) 2 |||=| 2 for
| A,
VV [x)]] - |||=|| for
|||||| 2 A,
| 00 S%
Using the above in equation 20 and defining
r(t) = Σ | 1 2 A].
we obtain
d V(z.)
S a² d (T(t))
(2d5 + I(t) (I +C',
N Σ
dt
where the constants C² are independent of N (but they depend on A). Thus, choosing picking a sufficiently large constant B > 0 (which is independent of N), we obtain for a constant CB > 0 that, for all T >
a²
(T(+))² 1((()) 2 B)dt S Σ (zx(()) + CB,
2d
L (T(t)) B)dt B²T.
Therefore, we obtain for all T 0,
1 T J N (at(t) < - (' (T())) A?,
T
with
T f (T(I))² di B² CB +
1
Σ (()
NT
Thus, for all T and N and for constant D > 0 (which is independent of N).
Σ v(Er(o))
Σ 1=((
<D+ T
By the variational representation of relative entropy, for (0,
E N Σ
v(a.(o))
< log
(foxporte) x(dz) + No
KL (L)) @0)]
By Assumption
3
Sab-Gaussian. Hence, we have
1 Σ V(Cc(0)))
lim sup N-+00
< <0,
N
from which, the result follows.
20

---

Published as a conference paper at ICLR 2025
Proof of Theorem Recall H(t) KL(p (t(p)(())ll (t) From the general KSD bound obtained in with k, we obtain for every T' >
Theorem
T L
1
H(0) NT
+
1
E
Σ c
N (ma(())
dt,
N
0
kk=1
where C (x) is as defined in equation C > 0 such that, for any E R..
with the kernel k. Now note that we can obtain constant
V2k(z, 2)(z,,)VV(s)) = (z, VV())) < |= & C(1 + 1|
k(z,z)AV(() S C(|==|² +1+ v0)),
-A2k(2, 2) = (V)
Hence, for all z IR, we have that (z) S Ci|lz||2 + Cz, for some constants C₁ C2 > 0. Therefore, we have for any 1 >> 0 and N 2 1, that
T // EIKSD (4) [T)
H(0) NT
C
N
di + C2 N
E
E =[1==((0)'
NT
Now, using Lemma
5 there exists constant C4 such that for any T 2 and N
T J. T
H(0)
Ca
EIKSD* (t)(())) -
NT N
By the convexity of KSD, we have for N 1.
Co
E ElKSD M [T] 2
N1+o/2
Hence, by Borel-Cantelli lemma, we have that KSD [r) 25 0. as N + co. The stated Wasser-
stein convergence result now follows by Kanagawa al. (2022 Thm. 3.1) taking m = Id, Qm 1,9 = 2, (2) and t V.
A.5 PROOF OF THEOREM 5
By Theorem 3.5 in Kanagawa et al. (022). we have that
W2 M T) < Cld)(1 v [=)( (()
where, from Kanagawa al. (2022)
have
3d +
1
d
5
r(d)
where f
+
3
4d+
+ t
2
3
d
resulting in equation
16 Define E = (KSD(PallT)) S
Cp
}.
By Theorem and Markov's
CN14072
inequality, we have that P < e. On the event E for N 2 Co _+ we have
Co
r(d)
Wo(me x) S C(d)
cNI+0/22
thereby proving the claim.
A.6 PROOF OF PROPOSITION
Let P(R") denote the space of probability measures on Rd and denote by P(P(Rd)) the space of probability measures on P(R)). Let L(PM) denote the law of the random measure uad and 8 denote the Dirac measure at æ in ((P((TE))
By Lemma 5 and exchangeability.
V/ || ga
sup E N21
|| 11 M (dx)
sup N21
<o.
(21)
21

---

Published as a conference paper at ICLR 2025
Moreover, by Assumption aa, J. (=()) < Hence, by Chaintron & Diez 2022 Theorem 3.21), conclude that Wi (1 . @rk) 0 if and only if W; (C(mae), o₂) N + co, where Wi is the Wasserstein distance on space ((P(R()) equipped with the distance function W as defined in Chaintron & Diez (2022 Definition 3.5).
Note that Wi (L(1))) 8x)
E
W (M m)] By Theorem 4 and Jensen's 's inequality,
Wi(P T) 3 0
00.
Moreover, observe that
WM an T) S (3 M T) 2 Jad |1r|| Mav (dxx f x,x]1xxdd)))
and hence, by equation 21 and Assumption aa). suPN21 [WP(uau x)] < co. In particular, (Wi r) : N 2 1) is uniformly integrable and thus E [W] (122 T) 0 N 7 00. The result follows.
22


Reviews:
[{"review_id": "ubtqw682wr", "reviewer": "Reviewer_xwur", "summary": "This work provides a convergence analysis of the Stein Variational Gradient Descent (SVGD) algorithm in its full formulation, i.e., using finitely many particles and in discrete time. Such a quantitative convergence proof was long sought after, ever since the algorithm was first proposed in 2016. (The only previous known result, Shi and Mackey (2024), gave a quite poor convergence rate, and had a much more involved proof.) This paper thus improves upon a long line of work reviewed in the introduction. In particular, the behavior of SVGD in the infinite-particle limit is relatively well-understood, so the main difficulty lies in the finite-particle discretization.\n\nThis work's main novel insight is presented in Equation (9) in the proof of Theorem 1, in a continuous-time context: it consists in an identity which cleanly separates the finite-particle error from the infinite-particle behavior of SVGD. The finite-particle error term in this identity can be controlled by making a mild assumption on the kernel and the target distribution (Remark 1). The discrete-time, finite-particle convergence result, Theorem 3, builds upon this insight, as well as on ideas from prior works and on new estimates (Remark 4).\n\nThe Theorems 1 and 3 establish convergence in kernelized Stein discrepancy (KSD). They are complemented by convergence results in Wasserstein distance in section 4. A long-time propagation of chaos-type result for SVGD is established in section 5.", "questions": "- In the proof of Theorem 3, the bounds on the magnitude and the Lipschitz-ness of the update operator (Lemmas 3 and 4) grow with $n$ the iteration count. Is there any way to relate those quantities back to the distance between $\\mu_n^N$ and $\\pi$ instead? Intuitively, this way, the time-discretization error terms could be bounded uniformly for long times...\n- Are there any results on the convergence of SVGD in KL divergence (with infinite particles and/or in continuous time)? If yes, do your results allow to improve upon them? If not, would your proof approach allow to get such a result?\n- Does SVGD also exhibit long-time propagation of chaos for the \"last-iterate\" (as opposed to \"time-averaged\" in Proposition 1) occupancy measure of particle locations?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "This paper establishes quantitative convergence guarantees for finite-particle (and discrete-time) SVGD, with a much better dependency on problem parameters than the only previous known analysis. This is thus arguably the first satisfactory convergence bound, for an algorithm that has attracted considerable attention from theoreticians. So this paper's achievement is highly significant.\n\nThe main novel insight used in this paper is remarkably clean. It is also quite satisfying, in that it allows for an intuitive proof strategy. \n\nThe presentation of the paper is clear, although a bit technical from section 4 onwards.", "weaknesses": "Section 5 on propagation of chaos (POC) could benefit from a little bit more motivation: it is not clear why POC would be a desirable property for SVGD.\n\nPossible typos:\n- In Assumption 1(b) and Lemma 1, p_0^N needs to be C2 (not C1) for p^N(t,.) to be C2\n- in Assumption 2(d), maybe = is meant to be <=\n- typos on line 430", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "L6tiIu8KVA", "reviewer": "Reviewer_Dq3R", "summary": "This paper studies the finite-particle SVGD in both discrete and continuous settings. Their results include convergence results in averaged Kernel Stein Discrepancy and Wasserstein distance.", "questions": "- *Theorems 1 and 2*. Is it easy to choose an initial distribution, such that $\\lim\\sup_{N \\to \\infty} \\frac{1}{N} \\mathrm{KL}\\left(p^{N}(0) \\,\\big\\|\\, \\pi^{\\otimes N}\\right) < \\infty$? If yes, is there a general scheme for it?\n- How easy would it be to replace the averaged KSD error with the 'last iterate' KSD error. In the paper you have this type of results for some time-averaged distributions. But is it possible to avoid averaging completely as in the case of LMC?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "The SVGD algorithm was well studied in practice for several years already. However, its theoretical understanding was rather limited. As well discussed by the authors, past work only tackled particular (simpler) cases discretization mechanisms of the continuous SVGD, leaving the most interesting case open. That is the finite-particle discrete-time setting. In this setting they solve this problem and obtain polynomial convergence guarantees for different types of metrics, under certain conditions.  \nThe paper is generally well-written, although certain mathematical details are omitted.", "weaknesses": "### General comments\n\n- Some of the conditions in the main results are not easy to verify. See the Questions section. \n- The paper is generally well-written, although certain mathematical details are omitted. \n\n### Mathematical comments \n\n- The derivations starting from *line 281* are not well-explained. Why does the first term on the right-hand side vanish on second line? How is the third line derived?\n- In order to obtain (17), the term $(\\sum_j V(x_j)/N)^{2\\alpha}$ is upper bounded  by $\\sum_j V(x_j)/N$. This does not seem to be correct, even with the assumption $2\\alpha < 1$.\n- *line 927*. \"Routine computations give...\". More details on this part would ease the reading. \n\n### Notation\n- *Theorem 2*  $P(x_1(t) ∈ dx)$ is slightly vague from a mathematical point of view. Perhaps explicitly stating what is meant by this notation, would make the theorem clearer. \n-  $p^N (t,\\underline{z})$ is defined in Lemma 1, but later in the paper the notation is changed to $p(t,\\underline{z})$. \n- *notation in the proof of theorem 1* To keep notation self-explanatory I would suggest to use the number of particles in the notation for $H(0)$ in the proof of Theorem 2. That is to use $H^N(0)$, as later in the proof of the last claim you use $\\sup_L H^L(0)/L$.\n\n### typos\n\n* *line 271* equation equation -> equation.\n* *line 291* missing a period at the end of the equation.\n* *line 674* Extra square bracket in the equation. \n* *line 747,749* There is a $\\sum$ sign missing in the first term of the right side and in the definition of $f(n)$. \n* *page 8* literature literature -> literature", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LPKsFN0Rcq", "reviewer": "Reviewer_uGRh", "summary": "This paper performs a new analysis for Stein variational gradient descent (SVGD), achieving double exponential improvement of #particle for the continuous-time and discrete-time convergence rates in Kernel Stein discrepancy (KSD) over the best known previous result. This paper also establishes the convergence in Wasserstein distance using the bilinear Matern kernel and establishes marginal convergence and long-time propagation of chaos results for the time-averaged particle laws.", "questions": "* For time-average distribution $\\frac{1}{N}\\int_0^N\\mu^N(t, \\mathrm{d}x)\\ \\mathrm{d}t$, why do you integrate it from $0$ to $N$ instead of from $0$ to $T$? Since $N$ is the number of particle, the time integration range is confusing to me.\n* What is $\\mathcal{L}$ in Assumption 4?\n* The Growth condition in Assumption 2 is essential for Theorem 3. Can you discuss some relation between this condition with some other common conditions in sampling such as log Sobolev inequality, Poincare inequality and Talagrand inequality (which is also used in some previous works on SVGD)? What kind of distributions satisfy Assumption 2?\n* Also, it would be helpful to collect all related results and their corresponding assumptions in a table for a clear comparison.\n* The smoothness constant in condition (c) could be interesting to the convergence rate. Why don't you include that constant in Thm 3?\n* What is the relation between Wasserstein distance and KSD? Since Thm 1 doesn't exhibit any curse of dimension (CoD), it would be interesting to provide some intuition about where the CoD comes from for Wasserstein distance. Is that due to the relation between Wasserstein distance and KSD? I think the convergence in Wasserstein distance is more interesting than KSD to sampling community, and thus there should be more discussion on this result. Does Matern kernel in SVGD show better performance than some bounded kernels (Gaussian RBF) in practice (maybe do some simulation)?\n* If we choose $\\alpha=1/2$, as remark 3 claims, the distribution will approach Gaussian. According to Thm 3, in order to achieve the averaged $\\mathrm{KSD}^2\\leq\\varepsilon^2$, we need to choose $N\\asymp 1/\\varepsilon^2$, and thus $T=O(N^4)=O(1/\\epsilon^8)$. Is this terrible mixing time comparable to other sampling algorithm for Gaussian targets?\n\nI am happy to raise my score if the authors can answer some of the questions.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "* In KSD, this paper significantly improves the particle dependence over Shi & Mackey (2024) (which is known to be the previous best result) by considering the evolution of the joint distribution of N particles.\n* In Wassertein-2 distance, this is the first convergence rate, albeit showing curse of dimension.\n* The discussion on related works is extensive and informative.", "weaknesses": "* Some notations should be further clarified, and there is not a subsection collecting all the notations which makes certain parts not that readable.\n* This paper is short of some discussion on assumptions.\n* Comparison with previous analysis including assumptions and rates is not that clear.\n* The discussion on the convergence in Wasserstein-2 distance is weak.\n\n\nFor more details, see Questions below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3eHaxoKpBv", "reviewer": "Reviewer_J6ce", "summary": "The paper studies finite-particle convergence rates for the Stein Variational Gradient Descent (SVGD) algorithm in the Kernelized Stein Discrepancy (KSD), deriving results in both continuous and discrete times. It also provides convergence rates for the continuous-time SVGD in the L2-Wasserstein metric for a specific class of kernels.\n\nMinor typo: in line 430, either refer to Assumption 3 in the singular or the conditions in the plural.", "questions": "Theorem 5 is derived for the Matérn-family of kernels (defined by eq. (12)) since it needs Theorem 3.5 in Kanagawa et al. (2022). However, the authors claim in the main text that the restriction to such kernels is only done for \"computational clarity.\" Could the authors add a remark further clarifying if this means that Theorem 5 extends to the broader class of kernels defined in eq. (11)?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The paper is extremely well written, and while I have not checked all the proofs in detail, the results appear to be formally correct. The authors also do a good job explaining the literature.\n\nThe authors bypass the challenges faced by previous attempts at deriving convergence rates for SVGD in the KSD metric by working with the joint density of particle locations and the N-fold product target measure, exploiting a simple and elegant relationship related to the evolution of their KL divergence.", "weaknesses": "A key element in the development is the N-fold product target measure $\\pi^{\\bigotimes N}$. Although it's definition can arguably be derived from its name, I think it would be nice to formally define it too; pressumably, $\\pi^{\\bigotimes N}(x_1,\\cdots,x_N) := \\pi(x_1)\\times\\cdots\\times\\pi(x_N)$.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)