SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Aniket Das
ketd@google.com
Dheeraj Nagaraj
dheerajnagaraj@google.com
Stein Variational Gradient Descent (SVGD) is a popular particle-based variational inference algorithm with impressive empirical performance across various domains. Although the population (i.e., infinite-particle) limit dynamics of SVGD is well characterized, its behavior in the finite-particle regime is far less understood. To this end, our work introduces the notion of virtual particles to develop novel stochastic approximations of population-limit SVGD dynamics in the space of probability measures, that are exactly realizable using finite particles. As a result, we design two computationally efficient variants of SVGD, namely VP-SVGD and GB-SVGD, with provably fast finite-particle convergence rates. Our algorithms can be viewed as specific random-batch approximations of SVGD, which are computationally more efficient than ordinary SVGD. We show that the  $n$  particles output by VP-SVGD and GB-SVGD, run for  $T$  steps with batch-size  $K$ , are at least as good as i.i.d samples from a distribution whose Kernel Stein Discrepancy to the target is at most  $O(d^{1/3} / (KT)^{1/6})$  under standard assumptions. Our results also hold under a mild growth condition on the potential function, which is much weaker than the isoperimetric (e.g. Poincaré Inequality) or information-transport conditions (e.g. Talagrand's Inequality  $\mathsf{T}_1$ ) generally considered in prior works. As a corollary, we analyze the convergence of the empirical measure (of the particles output by VP-SVGD and GB-SVGD) to the target distribution and demonstrate a double exponential improvement over the best known finite-particle analysis of SVGD. Beyond this, our results present the first known oracle complexities for this setting with polynomial dimension dependence, thereby completely eliminating the curse of dimensionality exhibited by previously known finite-particle rates.
Sampling from a distribution over  $\mathbb{R}^d$  whose density  $\pi^{\star}(\mathbf{x})\propto \exp (-F(\mathbf{x}))$  is known only up to a normalizing constant, is a fundamental problem in machine learning [52, 20], statistics [41, 36], theoretical computer science [28, 17] and statistical physics [39, 15]. A popular approach to this is the Stein Variational Gradient Descent (SVGD) algorithm introduced by Liu and Wang [31], which uses a positive definite kernel  $k$  to evolve a set of  $n$  interacting particles  $(\mathbf{x}_t^{(i)})_{i\in [n],t\in \mathbb{N}}$  as follows:
$$
\mathbf {x} _ {t + 1} ^ {(i)} \leftarrow \mathbf {x} _ {t} ^ {(i)} - \frac {\gamma}{n} \sum_ {j = 1} ^ {n} \left[ k \left(\mathbf {x} _ {t} ^ {(i)}, \mathbf {x} _ {t} ^ {(j)}\right) \nabla F \left(\mathbf {x} _ {t} ^ {(j)}\right) - \nabla_ {2} k \left(\mathbf {x} _ {t} ^ {(i)}, \mathbf {x} _ {t} ^ {(j)}\right) \right] \tag {1}
$$
SVGD exhibits remarkable empirical performance in a variety of Bayesian inference, generative modeling and reinforcement learning tasks [31, 51, 54, 22, 50, 34, 45, 19] and usually converges rapidly to the target density while using only a few particles, often outperforming Markov Chain Monte Carlo (MCMC) methods. However, in contrast to its wide practical applicability, theoretical
analysis of the behavior SVGD is a relatively unexplored problem. Prior works on the analysis of SVGD [26, 14, 30, 42, 7] mainly consider the population limit, where the number of particles  $n \to \infty$ . These works assume that the initial distribution of the (infinite number of) particles has a finite KL divergence to the target  $\pi^{\star}$  and subsequently, interpret the dynamics of population-limit SVGD as projected gradient descent updates for the KL divergence on the space of probability measures, equipped with the Wasserstein geometry. Under appropriate assumptions on the target density, one can then use the theory of Wasserstein Gradient Flows to establish non-asymptotic (in time) convergence of population-limit SVGD to  $\pi^{\star}$  in the Kernel Stein Discrepancy (KSD) metric.
While the framework of Wasserstein Gradient Flows suffices to explain the behavior of SVGD in the population limit, the same techniques are insufficient to effectively analyze SVGD in the finite-particle regime. This is primarily due to the fact that the empirical measure  $\hat{\mu}^{(n)}$  of a finite number of particles does not admit a density with respect to the Lebesgue measure, and thus, its KL divergence to the target is always infinite (i.e.  $\mathrm{KL}\left(\hat{\mu}^{(n)}\| \pi^{\star}\right) = \infty$ ). In such a setting, a direct analysis of the dynamics of finite-particle SVGD becomes prohibitively difficult due to complex inter-particle dependencies. To the best of our knowledge, the pioneering work of Shi and Mackey [43] is the only result that obtains an explicit convergence rate of finite-particle SVGD by tracking the deviation between the law of  $n$ -particle SVGD and that of its population-limit. To this end, Shi and Mackey [43] show that for subgaussian target densities, the empirical measure of  $n$ -particle SVGD converges to  $\pi^{\star}$  at a rate of  $O\left(\sqrt{\frac{\mathrm{poly}(d)}{\log\log n^{\Theta(1 / d)}}}\right)$  in KSD<sup>1</sup>. The obtained convergence rate is quite slow and fails to adequately explain the impressive practical performance of SVGD.
Our work takes a starkly different approach to this problem and deliberately deviates from tracking population-limit SVGD using a finite number of particles. Instead, we directly analyze the dynamics of KL divergence along a carefully constructed trajectory in the space of distributions. To this end, our proposed algorithm, Virtual Particle SVGD (VP-SVGD) devises an unbiased stochastic approximation (in the space of measures) of the population-limit dynamics of SVGD. We achieve this by considering additional particles called virtual particles<sup>2</sup> which evolve in time but aren't part of the output (i.e. real particles). These virtual particles are used only to compute information about the current population-level distribution of the real particles, and enable exact implementation of our stochastic approximation to population-limit SVGD, while using only a finite number of particles.
Our analysis is similar in spirit to non-asymptotic analyses of stochastic gradient descent (SGD) that generally do not attempt to track gradient descent (analogous to population-limit SVGD in this case), but instead directly track the evolution of the objective function along the SGD trajectory using appropriate stochastic descent lemmas [24, 21, 11]. The key feature of our proposed stochastic approximation is the fact that it can be implemented using only a finite number of particles. This allows us to design faster variants of SVGD with provably fast finite-particle convergence.
VP-SVGD and GB-SVGD We propose two variants of SVGD that enjoy provably fast finite-particle convergence guarantees to the target distribution: Virtual Particle SVGD (VP-SVGD in Algorithm 1) and Global Batch SVGD (GB-SVGD in Algorithm 2). VP-SVGD is a conceptually elegant stochastic approximation (in the space of probability measures) of population-limit SVGD, and GB-SVGD is a practically efficient version of SVGD which achieves good empirical performance. Our analysis of GB-SVGD builds upon that of VP-SVGD. When the potential  $F$  is smooth and satisfies a quadratic growth condition (which holds under subgaussianity of  $\pi^{\star}$ , a common assumption in prior works [42, 43]), we show that the  $n$  particles output by  $T$  steps of our algorithms, run with batch-size  $K$ , are at least as good as i.i.d draws from a distribution whose Kernel Stein Discrepancy to  $\pi^{\star}$  is at most  $O\left(d^{1/3} / (KT)^{1/6}\right)$ . Our results also hold under a mild subquadratic growth condition for  $F$ , which is much weaker than isoperimetric (e.g. Poincaré Inequality) or information-transport (e.g. Talagrand's Inequality  $\mathsf{T}_1$ ) assumptions generally considered in the sampling literature [47, 42, 43, 8, 2].
State-of-the-art Finite Particle Guarantees As corollaries of the above result, we establish that VP-SVGD and GB-SVGD exhibit the best known finite-particle guarantees in the literature which significantly outperform that of prior works. Our results are summarized in Table 1. In particular,
<sup>1</sup>We explicate the dimension dependence in Shi and Mackey [43] by closely following their analysis
2 (roughly) analogous to virtual particles in quantum field theory that enable interactions between real particles
Result	Algorithm	Assumption	Rate	Oracle Complexity
Korba et al. [26]	Population Limit SVGD	Uniformly Bounded KSDπ*(μt||π*)	poly(d)/√T	Not Implementable
Salim et al. [42]	Population Limit SVGD	Sub-gaussian π*	d3/2√T	Not Implementable
Shi and Mackey [43]	SVGD	Sub-gaussian π*	poly(d)/√log log nΘ(1/d)	poly(d)eΘ(de poly(d)/ε2) / ε2
Ours, Corollary 1	VP-SVGD	Sub-gaussian π*	(d/n)1/4 + (d/n)1/2	d4/ε12
Ours, Corollary 1	GB-SVGD	Sub-gaussian π*	d1/3/n1/12 + (d/n)1/2	d6/ε18
Ours, Corollary 1	VP-SVGD	Sub-exponential π*	d1/3/n1/6 + d/n1/2	d6/ε16
Ours, Corollary 1	GB-SVGD	Sub-exponential π*	d7/8/n1/16 + d/n1/2	d9/ε24
Table 1: Comparison of our results with prior works.  $d$ ,  $T$ , and  $n$  denote the dimension, no. of iterations and no. of output particles respectively. Oracle Complexity denotes number of evaluations of  $\nabla F$  needed to achieve  $\mathrm{KSD}_{\pi^{\star}}(\cdot ||\pi^{\star}) \leq \epsilon$  and Rate denotes convergence rate w.r.t KSD metric. Note that: 1. Population Limit SVGD is not implementable as it requires infinite particles 2. The uniformly bounded  $\mathrm{KSD}_{\pi^{\star}}(\bar{\mu}_t||\pi^{\star})$  assumption cannot be verified apriori and is much stronger than subgaussianity (see [42] Lemma C.1)
under subgaussianity of the target distribution  $\pi^{\star}$ , we show that the empirical measure of the  $n$  particles output by VP-SVGD converges to  $\pi^{\star}$  in KSD at a rate of  $O\left((d/n)^{1/4} + (d/n)^{1/2}\right)$ . Similarly, the empirical measure of the  $n$  output particles of GB-SVGD converges to  $\pi^{\star}$  at a KSD rate of  $O\left(d^{1/3}/n^{1/12} + (d/n)^{1/2}\right)$ . Both these results represent a double exponential improvement over the  $O\left(\frac{\text{poly}(d)}{\sqrt{\log\log n^{\Theta(1/d)}}}\right)$  KSD rate of  $n$ -particle SVGD obtained by Shi and Mackey [43], which, to our knowledge, is the best known prior result for SVGD in the finite particle regime. When benchmarked in terms of gradient oracle complexity, i.e., the number of evaluations of  $\nabla F$  required by an algorithm to achieve  $\text{KSD}_{\pi^{\star}}(\cdot||\pi^{\star}) \leq \epsilon$ , we demonstrate that for subgaussian  $\pi^{\star}$ , the oracle complexity of VP-SVGD is  $O\left(d^{4}/\epsilon^{12}\right)$  while that of GB-SVGD is  $O\left(d^{6}/\epsilon^{18}\right)$ . To the best of our knowledge, our result presents the first known oracle complexity guarantee with polynomial dimension dependence, and consequently, does not suffer from a curse of dimensionality unlike prior works. Furthermore, as discussed above, the conditions under which our result holds are far weaker than subgaussianity of  $\pi^{\star}$ , and as such, includes sub-exponential targets and beyond. In particular, our guarantees for sub-exponential target distributions are (to the best of our knowledge) the first of its kind.
Computational Benefits: VP-SVGD and GB-SVGD can be viewed as specific random batch approximations of SVGD. Our experiments (Section 8) show that GB-SVGD obtains similar performance as SVGD but requires fewer computations. In this context, a different kind of random batch method that divides the particles into random subsets of interacting particles, has been proposed by Li et al. [29]. However, the objective in Li et al. [29] is to approximate finite-particle SVGD dynamics using the random batch method, instead of analyzing convergence of the random batch method itself. Beyond this, their guarantees also suffer from an exponential dependence on the time  $T$ . As explained below, their approach is also conceptually different from our method since we use the same random batch to evolve every particle, allowing us to interpret this as a stochastic approximation in the space of distributions instead of in the path space.
1.2 Technical Challenges
We resolve the following important conceptual challenges, which may be of independent interest.
Stochastic Approximation in the Space of Probability Measures Stochastic approximations are widely used in optimization, control and sampling [27, 52, 23]. In the context of sampling, stochastic approximations are generally implemented in path space, e.g., Stochastic Gradient Langevin Dynamics (SGLD) [52] takes a random batch approximation of the drift term via the update  $\mathbf{x}_{t + 1} = \mathbf{x}_t - \frac{\eta}{K}\sum_{j = 0}^{K - 1}\nabla f(\mathbf{x}_t,\xi_j) + \sqrt{2\eta}\epsilon_t$ ,  $\epsilon_t\sim \mathcal{N}(0,\mathbf{I})$  where  $\mathbb{E}[f(\mathbf{x}_t,\xi_j)|\mathbf{x}_t] = F(\mathbf{x}_t)$ . Such stochastic approximations are then analyzed using the theory of stochastic processes over  $\mathbb{R}^d$  [12, 40, 55, 25]. However, when viewed in the space of probability measures (i.e.,  $\mu_t = \mathrm{Law}(\mathbf{x}_t)$ ), the time-evolution
of these algorithms is deterministic. In contrast, our approach designs stochastic approximations in the space of probability measures. In particular, the time-evolution of the law of any particle in VP-SVGD and GB-SVGD are a stochastic approximation of the dynamics of population-limit SVGD. We ensure that requires only a finite number of particles for exact implementation.
Tracking KL Divergence in the Finite-Particle Regime The population limit  $(n\to \infty)$  ensures that the initial empirical distribution  $(\mu_0)$  of SVGD admits a density (w.r.t the Lebesgue measure). When the KL divergence between  $\mu_0$  and  $\pi^{\star}$  is finite, prior works on population-limit SVGD analyze the time-evolution of the KL divergence to  $\pi^{\star}$ . However, this approach cannot be directly used to analyze finite-particle SVGD since the empirical distribution of a finite number of particles does not admit a density, and thus its KL divergence to  $\pi^{\star}$  is infinite. Our analysis of VP-SVGD and GB-SVGD circumvents this obstacle by considering the dynamics of an infinite number of particles, whose empirical measure then admits a density. However, the careful design ensures that the dynamics of  $n$  of these particles can be computed exactly, using only a finite total number of (real + virtual) particles. When conditioned on the virtual particles, these particles are i.i.d. and their conditional law is close to the target distribution with high probability.
2 Notation and Problem Setup
We use  $\| \cdot \|, \langle \cdot, \cdot \rangle$  to denote the Euclidean norm and inner product over  $\mathbb{R}^d$  respectively. All other norms and inner products are subscripted to indicate their underlying space.  $\mathcal{P}_2(\mathbb{R}^d)$  denotes the space of probability measures on  $\mathbb{R}^d$  with finite second moment, with the Wasserstein-2 metric denoted as  $\mathcal{W}_2(\mu, \nu)$  for  $\mu, \nu \in \mathcal{P}_2(\mathbb{R}^d)$ . For any two probability measures  $\mu, \nu$ , we denote their KL divergence as KL  $(\mu \| \nu)$ . For any function  $f: X \to Y$  and any probability measure  $\mu$  over  $X$ , we let  $f_{\#} \mu$  denote the law of  $f(\mathbf{x}): \mathbf{x} \sim \mu$ . Given a sigma algebra  $\mathcal{F}$  over some space  $\Omega$ , and a measurable space  $\mathcal{X}$ ,  $\mu(\cdot; \cdot): \mathcal{F} \times \mathcal{X} \to \mathbb{R}^+$  is a probability kernel if for every  $\mathbf{x} \in \mathcal{X}$ ,  $\mu(\cdot; \mathbf{x})$  is a measure over  $\mathcal{F}$  and for every  $A \in \mathcal{F}$ , the map  $\mathbf{x} \to \mu(A; \mathbf{x})$  is measurable. We make use of probability measures  $\mu(\cdot; \mathbf{x})$  where  $\mathbf{x}$  is a random element of some appropriate space  $\mathcal{X}$ , resulting in random probability measures. We use  $[m]$  and  $(m)$  to denote the sets  $\{1, \ldots, m\}$  and  $\{0, \ldots, m - 1\}$  respectively. For any finite set  $A$ ,  $\mathbb{S}_A$  denotes the group of all permutations of  $A$ . We use the  $O$  notation to characterize the dependence of our rates on the number of iterations  $T$ , dimension  $d$  and batch-size  $K$ , suppressing numerical and problem-dependent constants. We use  $\lesssim$  to denote  $\leq$  upto universal constants.
We fix a symmetric positive definite reproducing kernel  $k:\mathbb{R}^d\times \mathbb{R}^d\to \mathbb{R}$  and let the corresponding reproducing kernel Hilbert space (RKHS) [44] be denoted as  $\mathcal{H}_0$ . We denote the product RKHS as  $\mathcal{H} = \prod_{i = 1}^{d}\mathcal{H}_{0}$ , equipped with the standard inner product for product spaces. We assume  $k$  is differentiable in both its arguments and let  $\nabla_2k(\mathbf{x},\mathbf{y})$  denote the gradient of  $k(\cdot ,\cdot)$  with respect to the second argument. For any  $\mu \in \mathcal{P}_2(\mathbb{R}^d)$ , we assume  $\mathcal{H}\subset L^2 (\mu)$  and the inclusion map  $i_{\mu}:\mathcal{H}\rightarrow L^{2}(\mu)$  is continuous. We use  $P_{\mu}:L^{2}(\mu)\to \mathcal{H}$  to denote the adjoint of  $i_{\mu}$ , i.e., the unique operator which satisfies  $\langle f,i_\mu g\rangle_{L^2 (\mu)} = \langle P_\mu f,g\rangle_{\mathcal{H}}$  for any  $f\in L^{2}(\mu),g\in \mathcal{H}$ . Carmeli et al. [6] shows that  $P_{\mu}$  can be expressed as a kernel convolution, i.e.,  $(P_{\mu}f)(\mathbf{x}) = \int k(\mathbf{x},\mathbf{y})f(\mathbf{y})\mathrm{d}\mu (\mathbf{y})$
We define the function  $h: \mathbb{R}^d \times \mathbb{R}^d \to \mathbb{R}$  as  $h(\mathbf{x},\mathbf{y}) = k(\mathbf{x},\mathbf{y})\nabla F(\mathbf{y}) - \nabla_2k(\mathbf{x},\mathbf{y})$  and  $h_\mu \in \mathcal{H}$  as  $h_\mu = P_\mu (\nabla_{\mathbf{x}}\log (\frac{\mathrm{d}\mu}{\mathrm{d}\pi^{\star}} (\mathbf{x})))$  for any  $\mu \in \mathcal{P}_2(\mathbb{R}^d)$ . Integration by parts shows that  $h_\mu (\mathbf{x}) = \int h(\mathbf{x},\mathbf{y})\mathrm{d}\mu (\mathbf{y})$ . The convergence metric we use is the Kernel Stein Discrepancy (KSD) metric, which is widely used for comparing probability distributions [32, 9] and analyzing SVGD [42, 26].
Definition 1 (Kernel Stein Discrepancy). Define the Langevin Stein Operator of  $\pi^{\star}$  acting on any differentiable  $g:\mathbb{R}^d\to \mathbb{R}^d$
$$
\left(T _ {\pi^ {*}} g\right) (\mathbf {x}) = \nabla \cdot g (\mathbf {x}) - \langle \nabla F (\mathbf {x}), g (\mathbf {x}) \rangle
$$
For any two probability measures  $\mu, \nu$ , the Kernel Stein Discrepancy between  $\mu$  and  $\nu$  (with respect to  $\pi^{\star}$ ), denoted as  $\mathsf{KSD}_{\pi^{\star}}(\mu||\nu)$  is defined as
$$
\mathrm {K S D} _ {\pi^ {\star}} (\mu | | \nu) = \sup  _ {\| g \| _ {\mathcal {H}} \leq 1} \mathbb {E} _ {\mu} [ T _ {\pi^ {\star}} g ] - \mathbb {E} _ {\nu} [ T _ {\pi^ {\star}} g ]
$$
Using integration by parts (see Chwialkowski et al. [9]), it follows that  $\mathrm{KSD}_{\pi^*}(\mu \| \nu) = \| h_\mu - h_\nu \|_{\mathcal{H}}$
Organization: We review population-limit SVGD in Section 3 and derive VP-SVGD and GB-SVGD in Section 4. We state our technical assumptions in Section 5, main results in Section 6 and provide a proof sketch in Section 7. We present empirical evaluation in Section 8.
3 Background on Population-Limit SVGD
We briefly introduce the analysis of population-limit SVGD using the theory of Wasserstein Gradient Flows and refer the readers to Korba et al. [26] and Salim et al. [42] for a detailed treatment.
The space  $\mathcal{P}_2(\mathbb{R}^d)$  equipped with the 2-Wasserstein metric  $\mathcal{W}_2$  is known as the Wasserstein space, which admits the following Riemannian structure: For any  $\mu \in \mathcal{P}_2(\mathbb{R}^d)$ , the tangent space  $T_{\mu}\mathcal{P}_{2}(\mathbb{R}^{d})$  can be identified with the Hilbert space  $L^2 (\mu)$ . We can then define differentiable functionals  $\mathcal{L}:\mathcal{P}_2(\mathbb{R}^d)\to \mathbb{R}$  and compute their Wasserstein gradients, denoted as  $\nabla_{\mathcal{W}_2}\mathcal{L}$ . Note that the target  $\pi^{\star}$  is the unique minimizer over  $\mathcal{P}_2(\mathbb{R}^d)$  for the functional  $\mathcal{L}[\mu ] = \mathrm{KL}\left(\mu \| \pi^{\star}\right)$ . The Wasserstein Gradient of  $\mathcal{L}[\mu ]$  is  $\nabla_{\mathcal{W}_2}\mathcal{L}[\mu ] = \nabla_{\mathbf{x}}\log (\frac{\mathrm{d}\mu}{\mathrm{d}\pi^{\star}} (\mathbf{x}))$  [1]. This powerful machinery has served as a backbone for the analysis of algorithms such as LMC [53, 3, 2] and population-limit SVGD [14, 26, 42, 45, 7].
The updates of population-limit SVGD can be viewed as Projected Gradient Descent in the Wasserstein space. Recall from Section 2 that the function  $h_{\mu}(\mathbf{x}) = P_{\mu}(\nabla \log (\frac{\mathrm{d}\mu}{\mathrm{d}\pi^*}))(\mathbf{x}) = \int h(\mathbf{x},\mathbf{y})\mathrm{d}\mu (\mathbf{y})$ . Let  $\hat{\mu}_t^n$  denote the empirical measures of the SVGD particles  $(\mathbf{x}_t^{(i)})_{i\in [n]}$  at timestep  $t$ . We note that the SVGD updates in (1) can be recast as  $\hat{\mu}_{t + 1}^n = (I - \gamma h_{\hat{\mu}_t^n})_\# \hat{\mu}_n^t$ . In the limit of infinite particles  $n\to \infty$ , suppose the empirical measure  $\hat{\mu}_t^n$  converges to the population measure  $\bar{\mu}_t$ . In this population limit, the updates of SVGD can be expressed as,
$$
\bar {\mu} _ {t + 1} = \left(I - h _ {\bar {\mu} _ {t}}\right) _ {\#} \bar {\mu} _ {t} = \left(I - \gamma P _ {\bar {\mu} _ {t}} \left(\nabla \log (\frac {d \bar {\mu} _ {t}}{d \pi^ {\star}})\right)\right) _ {\#} \bar {\mu} _ {t} = \left(I - \gamma P _ {\bar {\mu} _ {t}} \left(\nabla_ {\mathcal {W} _ {2}} \mathsf {K L} \left(\bar {\mu} _ {t} \| \pi^ {\star}\right)\right)\right) _ {\#} \bar {\mu} _ {t}
$$
Recall from Section 2 that  $P_{\bar{\mu}_t}: L^2(\bar{\mu}_t) \to \mathcal{H}$  is the Hilbert adjoint of  $i_{\bar{\mu}_t}$ . Since  $\mathcal{H} \subset L^2(\bar{\mu}_t)$ , the updates of SVGD in the population limit can be seen as Projected Wasserstein Gradient Descent for  $\mathcal{L}[\mu] = \mathrm{KL}(\mu \| \pi^\star)$ , with the Wasserstein Gradient at each step being projected onto the RKHS  $\mathcal{H}$ . Assuming  $\mathrm{KL}(\bar{\mu}_0 \| \pi^\star) < \infty$ , convergence of population limit SVGD is then established by tracking the evolution of  $\mathrm{KL}(\bar{\mu}_t \| \pi^\star)$  under appropriate structural assumptions (such as subgaussianity) on  $\pi^\star$ .
4 Algorithm and Intuition
In this section, we derive VP-SVGD (Algorithm 1), and build upon it to obtain GB-SVGD. Consider a countably infinite collection of particles  $\mathbf{x}_0^{(l)}\in \mathbb{R}^d$ ,  $l\in \mathbb{N}\cup \{0\}$ , sampled i.i.d from a measure  $\mu_0$ , having a density w.r.t. the Lebesgue measure. By the strong law of large numbers, the empirical measure of  $\mathbf{x}_0^{(l)}$  is almost surely equal to  $\mu_0$  (see Dudley [13, Theorem 11.4.1]). Let batch size  $K\in \mathbb{N}$  denote the batch size, and  $\mathcal{F}_t$  denote the filtration  $\mathcal{F}_t$ ,  $t\geq 0$  as  $\mathcal{F}_t = \sigma (\{\mathbf{x}_0^{(l)}\mid l\leq Kt - 1\})$ ,  $\forall t\in \mathbb{N}$ , with  $\mathcal{F}_0$  being the trivial  $\sigma$  algebra. For ease of exposition, we discuss the case of  $K = 1$  in this section and present a complete derivation for arbitrary  $K\geq 1$  in Appendix C. Recall from Section 3 that the updates of population-limit SVGD in  $\mathcal{P}_2(\mathbb{R}^d)$  can be expressed as follows:
$$
\bar {\mu} _ {t + 1} = \left(I - \gamma h _ {\bar {\mu} _ {t}}\right) _ {\#} \bar {\mu} _ {t} \tag {2}
$$
We aim to design a stochastic approximation in  $\mathcal{P}_2(\mathbb{R}^d)$  for the updates (2), such that it admits a finite-particle realization. To this end, we propose the following dynamics in  $\mathbb{R}^d$
$$
\mathbf {x} _ {t + 1} ^ {(s)} = \mathbf {x} _ {t} ^ {(s)} - \gamma h \left(\mathbf {x} _ {t} ^ {(s)}, \mathbf {x} _ {t} ^ {(t)}\right), \quad s \in \mathbb {N} \cup \{0 \} \tag {3}
$$