De-randomizing MCMC dynamics with the diffusion Stein operator
Authors: Zheyang Shen
Markus Heinonen
Samuel Kaski
Venue: NeurIPS 2021 Poster (2021)
Keywords: Stein's method
MCMC dynamics
particle variational inference

Abstract: Approximate Bayesian inference estimates descriptors of an intractable target distribution - in essence, an optimization problem within a family of distributions. For example, Langevin dynamics (LD) extracts asymptotically exact samples from a diffusion process because the time evolution of its marginal distributions constitutes a curve that minimizes the KL-divergence via steepest descent in the Wasserstein space. Parallel to LD, Stein variational gradient descent (SVGD) similarly minimizes the KL, albeit endowed with a novel Stein-Wasserstein distance, by deterministically transporting a set of particle samples, thus de-randomizes the stochastic diffusion process. We propose de-randomized kernel-based particle samplers to all diffusion-based samplers known as MCMC dynamics. Following previous work in interpreting MCMC dynamics, we equip the Stein-Wasserstein space with a fiber-Riemannian Poisson structure, with the capacity of characterizing a fiber-gradient Hamiltonian flow that simulates MCMC dynamics. Such dynamics discretizes into generalized SVGD (GSVGD), a Stein-type deterministic particle sampler, with particle updates coinciding with applying the diffusion Stein operator to a kernel function. We demonstrate empirically that GSVGD can de-randomize complex MCMC dynamics, which combine the advantages of auxiliary momentum variables and Riemannian structure, while maintaining the high sample quality from an interacting particle system.

De-randomizing MCMC dynamics with the diffusion Stein operator
Zheyang Shen'
Markus Heinonen' Samuel Kaski
Helsinki Institute for Information Technology, HITT, Department of Computer Science, Aalto University, Finland Depparmentt of Computer Science, University of Manchester {zheyang- shen, markus. o heinonen, samuel. famuel.kakki Caalto.
Abstract
Approximate Bayesian inference estimates descriptors an intractable target distribution in essence, an optimization problem within a family of distribu- tions. For example, Langevin dynamics (LD) extracts asymptotically exact samples from a diffusion process because the time evolution of its marginal distributions constitutes a curve that minimizes the KL-divergence via steepest descent the Wasserstein space. Parallel to LD, Stein variational gradient descent (SVGD) simi- larly minimizes the KL, albeit endowed with a novel Stein-Waan Wasserstein distance, by deterministically transporting set of particle samples. thus de-randomizes the stochastic diffusion process. We propose de-randomized kernel-based particle samplers all diffusion-based samplers known as MCMC dynamies. Following previous work in interpreting MCMC dynamics, we equip the Stein- Wasserstein metric with Riemaaninnnn Poisson structure, with the capacity of character- izing grediett Hamiltonian flow that simulates MCMC dynamics. Such dynamics discretize into generalized SVGD (GSVGD), Stein-type determinis- tic particle sampler, with particle updates coinciding with applying the diffusion Stein operator kernel function. We demonstrate empirically that GSVGD can de-randomize complicated MCMC dynamics, which combine the advantages of auxiliary momentum variables and Riemannian structure, while maintaining the high sample quality from an interacting particle system.
Introduction
Evaluating an un-normalized target distribution æ is a centerpiece of Bayesian inference, due to its ubiquitous presence in posterior distributions. Markov chain Monte Carlo (MCMC) methods fulfill this objective by generating asymptotically exact random samples from the distribution, significant subset of which involves discretization of continuous- time diffusion processes, most notably Langevin diffusion, stochastic gradient Hamiltonian Monte Carlo (HMC) (Chen et al., 2014) and their further generalizations, which we collectively call MCMC dynamics (Ma al., 2015). Despite its simplicity and theoretical soundness, this diffusion-basel sampling often suffers from slow convergence and small effective sample sizes, largely due to the auto-correlation of the samples.
As an alternative to the simulation of stochastic systems, particle-based variational inference (Liu and Wang, 2016; Chen et al., 2018: Liu al., 2019a) partially addresses the shortcomings of MCMC by replacing the Langevin diffusion, the simplest MCMC dynamics, with deterministic interacting particle system that transports set of interacting particles towards the target distribution. Theoretically speaking, Langevin diffusion encodes an evolution of density that minimizes the KL-divergence through steepest descent in the 2- Wasserstein space (Jordan al., 1998), and particle-
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Langevin diffusion
W2 gradient flow
SVGD
Fokker- Planck
projecting H
MCMC dynamics
equation
fGH flow
discretization
G-SVGD
Figure I: A diagram showing the contribution of our paper: we extend and generalize previous work linking Langevin diffusion to gradient flow the 2- 2-Wasserstein metric W2 through the Fokker- Planck equation, and the W2 gradient flow linking to SVGD (Liu and Wang, 2016) through projection onto RKHS; MCMC dynamics (Ma al., 2015) are interpreted as fiber- fiber-gradient Hamiltonian (fGH) flow on W2 (Liu et al., 2019b), and a projection onto an RKHS yields GSVGD.
based variational inference (PARVI) approximates such evolution using reproducing kernel Hilbert space (RKHS) (Liu, 2017; Liu et al., 2019a), thus de-randomizing the Langevin diffusion process.
Given the elegant theoretical link between Langevin diffusion and PARVI, one crucial question arises: can we leverage the advantages of general MCMC dynamics onto de-randomized particle systems? While it is difficult to formulate them direct Wasserstein gradient flow, Liu et al. (2019b) propose an interpretation for "regular" MCMC dynamics that augments the 2- Wasserrstni space with a fiber bundle, forming Hamiltonian flow on the fiber bundle, and a gradient flow on each fiber.
Riemaaiinnnn Poisson manifold, under which MCMC dynamics follows
We show that adapting the Riemaniinnnnnnn Hamiltonian flow to the Stein- Wasserstein metric (Liu, 2017; Duncan et al., 2019) yields a vector field in the form of applying the diffusson Stein operator to the kernel function k(,, 0). A discretization of this vector field gives generalized SVGD (GSVGD), an interacting particle system that generalizes SVGD, with particle updates balancing an attractive force maximizing the log-likelihood and repulsive force preventing "mode collapse" of particles. We further demonstrate that the connection drawn between L.D and SVGD (Liiu and Wang, 2016; Liu, 2017; Liu et al., 2019a) is retraced by GSVGD, reaffirming our claim that GSVGD mirrors SVGD in approximating a larger class of MCMC diffusion processes. Within the generality of our framework, we can develop PARVI algorithms that exploit two key types of possible acceleration in MCMC dynamics: auxiliary momentum variables and an adaptive Riemannian parametrization that allows for fast and efficient exploration of the probability space (Girolami and Calderhead, 2011), 1), as shown in Table 1.
MCMC dynomics
suxiliary waiable
Riemannion
PARVI
SGLD (Welling and Teb, 2011)
Blob (Chen et al, 2018)
SVGD (Las Wang, 2016)
SGRLD (Girolumi and Culdethead, 2011)
Rizmannian SVGD und 2bu,
(Ches il., 2014)
20199)
G
SGRHMC (Ma as al., 2015)
G(#)
((0)
this week (SGRHMC-Ssinn)
(a) LD path
(b) SVGD
(c) SGHMC path
(d) SGHMC Stein
Table I: An overview of MCMC dynamics along with their PARVI approximations. Between (a), (c) and (b), (d), MCMC dynamics simulate diffusion processes (shown by one instance of particle trajectory) that temporally draw samples from the target distribution (samples marked by scatterplots); PARVI approximates the density evolution of MCMC diffusions with of particles with deterministic interactions; Between (a), (b) and (c), (d), the two systems, namely LD (Welling and Teh, 2011) and SGHMC (Chen et al., 2014), follow different density evolutions, (a), (b) describe a gradient flow and (c), (d) describe fiber-gradient Hamiltonian flow.

---

2 MCMC dynamics - the relevant bits
In this paper, consider the problem of extracting samples 0 E TRD Ω2 from un-normalized target distribution x00) X e e- We begin by reviewing the key characteristics of MCMC dynamics, with particular emphasis on Langevin dynamics and its Tokker-Plande equation (FPE), the partial differential equation (PDE) that depicts the time evolution of diffusion process. As the FPE of LD conforms to a gradient flow structure in the 2- Wasserstein metric space of probability measures (P(2), W2) (Jordan et al., 1998), we then briefly cover gradient flow on P(22) infinite-dimeniionll Riemannian manifold. Through the lens of gradient flow, we can see SVGD deterministic interacting particle system that approximates the gradient flow of L.D through (i) gradient flow on a novel Stein- Wasserstein metric (Liu, 2017) or (ii) projecting the gradient flow direction onto an RKHS (Liu al., 2019a). With Wasserstein gradient flow (WGF) neatly linking to both LD and SVGD, we see that diffusion process LD and an interacting particle system SVGD are, respectively, a stochastic instance and deterministic approximation of the same Wasserstein gradient flow.
We use f denote the gradient of scalar-valued function, and V f the divergence of a valued function, V A applies the divergence operator to each row matrix- valued function, Pr the "partial derivative" with respect to f.
Notations:
2.1 Langevin dynamics and its Fokker-Planck equation
In this paper, we consider MCMC dynamics in the form of Itō diffusion processes following the stochastic differential equation (SDE) formula
d0: = f(O.)dl 220(0,)
consisting of drift coefficient f RD - RDD diffusion coefficient V2E and D-dimensional Brownian motion W.. Ma et al. (2015) provide a complete recipe of all Itō diffusion processes converging the target measure The simplest MCMC dynamics takes the form of Langevin diffusion (Langevin, 1908), which moves towards higher densities with f(0) = V log π r0)) perturbed by white noise Σ I. Given initial distribution On po. the Pakker-Plance equation describes the time evolution of the density of O, (Risken, 1996)
P: + V (p: V log π) - (VV): (p.I) = O,
(2)
where Y =tr(XTY)) Given that (VV) M V (V. M), we can rewrite (2) as
8KL [p: || π
òe = (Vpt P log = V. P
(3)
5pt
first variation of KL Jot
a crucial step in developing the Wasserstein gradient flow perspective of LD, as V8E/8p coincides with the differential of functionals induced by the Wasserstein metric.
2.2 Gradient flow on (P(1), W2)
The conventional gradient flow (or the steepest descent curve) that minimizes a smooth function F RDD + R follows the PDE: x, + (x)))) = O. Defining gradient flow on P(2), informally written as Pt + V W Epp)) = 0, requires definition of the diffcrentiation Vw, which requires an understanding of the Wasserstein metric, i.e., the inner product g, defined on its tangent space.
To simplify the discussion, we restrict the discussion on measures with a density function and with finite second-order moments. For each p. the tangent space constitutes smooth functions integrating zero, (2) {flf E C (Q)), (0) 0}. in that curve on PS2)) preserves volume. The cotangent space constitutes an equivalence class of smooth functions in differing constant, noted as the quotient space P()) = C (RR We characterize the inner product space using metric tensor G(p) : TT, + TPP, a one-to-one mapping between elements of the tangent space and those of the cotangent space. The inner product 9p is defined by the inverse of the metric tensor, often denoted as Onsager operators (Onsager, 1931)
f2) / f(G()) /
Φ1 G(p) 02dx &1 = G(pff, &2 G(p)..
(4)
3

---

The Wasserstein Onsager operator takes the form G(p) Wasserstein gradient flow, we define Vw Ww)(pp> G(p) -1 6E(p)
+ V (pV6). In the context of
op leading the formulation:
0 0= G 5p((p))))
5E(pt)
= ir V.
P
(5)
8pt
ôpc
The first variation of KL [p || takes the form of 6KLIP ||
log p/ π 1, leading to the conclusion by Jordan et al. (1998) that the time evolution of the Langevin diffusion follows curve of steepest descent in the Wasserstein space, laying the foundation for approximation using particle interaction.
60
2.3 SVGD as gradient flow
PARVI transports set N of interacting particles {o())
towards the target distribution over USSSN
time. Take SVGD for example, denoting the empirical measure at time I as Pt = * Σ 6 6060
SVGD (Liu and Wang, 2016) follows the update rule
Egr [k,,,, V log n (0') + V2h(0,, vH vx (0.l).)
(6)
where k(;,)) defines a The FPE of SVGD can be written as
kernel, and V2 denotes the gradient of the second argument.
Pt (PxKp 5KI. [pt n]]
= O,
5p:
(e)'d')((((( We shall denote the RKHS where KCp an integral operator: K,)(()) = ((0, 0" (( ddp()).
defined by k as H. While significantly different from LD at first glance, SVGD approximates the WGF of LD by
kernelizing the Wasserstein Onsager operator to give GH(p) - -V (pK,, VΦ), thus forming the Stein-Wass Wasserstein metric WH (Liu, 2017; Duncan et al., 2019);
projecting the gradient flow vector field vp) 4KL Ip
- bp
V log
V log p onto H
(Liu al., 2019a).
The interpretation of SVGD yields valuable insights. While gradient flow (7) (P(2), W2) yields no closed-form energy functional (Chen et al., 2018), it behooves to absorb the operator K, into the definition of the Weee-Wassserttiin metric guarantee tractable energy functional. In the meantime, the kernelization trick transforms a gradient flow (2) simulated by diffusion process into an approximate deterministic transportation of particles in other words, de-randomizes it.
2.4 MCMC dynamics as fiber-gradient Hamiltonian flow
The concept of flow on (P(2), W2) is a generalization of the gradient flow: given a vector field P()) + U UpEP Tp. the flow of is defined as At vp:). Liu et al. (2019b) interpret "regular" MCMC dynamics on (P()), W2) flow, by combining gradient flow and Hamiltonian flow on the Wasserstein space, yielding ther-Riemannian manifold structure of P(Ω), fiber bundle consisting of Riemannian manifolds. In the context of a "regular" MCMC dynamics taking the form of underdamped Langevin dynamics (Chen et al., 2014), the diffusion matrix A determines the gradient flow on each fiber, and the curl matrix C determines the Hamiltonian flow on the fiber bundle. The Hamiltonian flow keeps KI. [p || л constant while encouraging fast exploration of the probability space; the fiber gradient flow minimizes the KL. The fiber- greddita Hamiltonian flow determines a PARVI with the particle update
θ = (A(0,) + C(e.)) (VlogT(O.) log p. = (.A.((p))
(8)
where the intractable & log Pt is approximated via the Blob method (Carrillo et al., 2019).

---

2.5 Stein's method and other relevant works
Our work enriches the practical application of Stein's method (Stein, 1972), which studies the class of operators that maps functions ones with expectation zero w.r.f. a distribution Gorham et al. (2019) draw from the findings of the generator method (Barbour, 1988, 1990; Gotze, 1991) and note the same property with infinitesimal generators of Feller processes and their stationary measures, which further extends into a mapping between diffusion-basee MCMC sampling and Stein operators, noted as diffission Stein operators. In this work, extend beyond sample quality measurement (Gorham et al., 2019) and parameter estimation (Barp et al., 2019) and establish an application the diffusion Stein operator as a deterministic alternative of MCMC dynamics.
Myriad works (Chen et al., 2018; Liu et al., 2019a; Zhang et al., 2020) seek alternatives to approxi- mating the gradient flow (2) through kernelization. Notably, Chen et al. (2018); Liu et al. (2019a) construct PARVI algorithms by approximating the term V Pl in the gradient flow, namely with Blob method (Carrillo et al., 2019), kernel density estimation (Liu et al., 2019a) and Stein underdamped LD connects to
gradient estimator (Li and Turner, 2017). The de-randomization the viewpoint of accelcrating PARVI methods. Ma et al. (2019) demonstrate that underdamped LD (Chen et al., 2014) accelerates the steepest descent steps taken by the overdamped LD, forming an analog of Nesterov acceleration for MCMC methods. Wang and Li (2019) present framework for Nesterov's accelerated gradient method in the Wasserstein space, which consists of augmenting the energy functional with the kinetic energy of an additional momentum variable.
PARVI for MCMC dynamics - a general recipe
In this section, we outline the main contribution of this work, which traces the roadmap delineated by previous works to explore the flow interpretation, as well as the approximation by interacting particle systems of general form MCMC dynamics (Ma et al., 2015) in the form Itō diffusion:
1 (T(0) + C(0)),
((0) π (0)
E(0) (0).
(10)
where A and C are dositive-semidefnnii and -ymmmryimttiic vatrix-valued functions, respectively. This general framework covers all continuous Itō diffusion processes with stationary distribution æ. most notably LD, stochastic gradient HMC (Chen et al., 2014), stochastic gradient Nosé-Hoover thermostat (SGNHT) (Ding et al., 2014) and stochastic gradient Riemannian Hamiltonian Monte Carlo (SGRHMC) (Ma al., 2015).
We demonstrate in this section that applying the fiber-gradient Hamiltonian flow structure of MCMC dynamics the Stein-Wasserstein metric yields flow on (P(S), Wx) that discretizes into GSVGD, a PARVI that updates particles with the diffusion Stein operator (Gorham et al., 2019), suggest- ing that the infinitesimal generator of MCMC diffusion processes offers "kernel smoothing" de- randomization. Apart from the Stein- Wasserstein metric, the analog of GSVGD generalizing SVGD extends other interpretations of SVGD, namely that GSVGD takes steepest descent minimizing the KL-divergence in incremental transformation of particles, and that that GSVGD projects the giber-gradient Hamiltonian flow onto an RKHS.
3.1 Fiber-gradient Hamiltonian flow on (P(2), W+)
Similar to (2), we derive the FPE of MCMC dynamics (9)-(10)
jt - V (p.f) + (VV): (p.A) V . (p:f) + (VV): (P.A) + (VV): :V))((t) (P.C)
(11)
(12)
v. (ps CA P1 V. (p((A
5KL pp
(13)
Öp:
which is curve in P(2) following the continuity equation i. (piv.)
0, v.
=
(A + C) V log pt/T.
5

---

semples
samples
samples
samples
-2
-2
-1
-1
Figure 2: Particle samples of an underlying 2-dimensional correlated mixture a extracted by SVGD (Liu and Wang, 2016), Riemannian SVGD (Liu and Zhu, 2017), SGHMC-Stein and SGRHMC-Stein. While all methods explore the mode nearest to initialization, only RHMC explores all three modes.
SVGD traces
RSVGD traces
SGHMC--Stein traces
SGRHMC-Stein traces
Figure 3: A visualisation of sampler trajectories on the tri- Gaussian density x. With the help of mo- mentum variable and efficient Riemannian parametrization of (A, C). SGRHMC-Stein aggressively explores the space, while other methods cannot escape the local component.
We can derive GSVGD from the 2019b). The Riemannian structure of the Stein- Wasserstein metric induces the tangent space TSP((S) f (2))) 21
manifold perspective of (P(S),, Wx) (Liu et al.,
An orthogonal projection Pp from ( to T,P is uniquely defined as such vector in T.P that satisfies V (p)Cp) = V (ppp(o)). Using Theorem 5 from Liu al. (2019b), we arrive a tibe-ggrddeett Hamiltonian flow
WK.. (p) = P ((A + CJV log क /p).
(14)
straightforward to verify that v A.C(p) = (A + C)V log s/p induces the same evolution of distribution as WKL., (p).
3.2 IRH flow induces the diffusion Stein operator
The Wasserstein flow on (P(1), Wx) following Pr = V. V (prvA. (p:))
requires spatial and temporal discretization suitable for particle update. While v A.C constitutes a vector field on P(2). its calculation does not involve an explicit term V log p inside the expectation.
H (p) = K, [AA + CJV log क /ppl
(15)
Eg'rpp(( (t, CCO') T(0') - (()())))))++
= -Er'p(A((')) C(O)) (0 (O')((,0))) + / V (IA(O) dp
integration by parts
= Egg weighted drift coefficient (9)
((O')k(,,0'
+AA + (2))
(16)
repulsive Force
V (T(A+ V
= Egg' ~p
(17)
diffuusion Stein operator T.A k((-0')

---

where V2 refers to the gradient w.r.t the second argument of the kernel function. Interestingly, the kemelized flow VH coincides with calculating the expectation of the diffusion Stein operator A.A.C (Gorham et al., 2019) applied to the kernel function k(O,-) a family of operators mapping functions to zero-expectation functions under (un-normalized) distribution When P: is approximated by D
A.C
the empirical measure of its particles Pr 1/N
80 i==
NV
we can derive the particle update of
0. - V (p:) = Ep T A(o,, 0') = N
T A TAC B(Cck(O, 0,0)) Po-
(18)
Similar to SVGD, the particle update GSVGD leverages between a weighted average of the drift vector and repulsive force, which drives particles towards the target distribution while preventing a mode collapse. The fiber-gradien Hamiltonian flow on (P(1), Wx) informs a duality between diffusion Stein operator and MCMC dynamics, extending the use of Stein operators beyond generalization of kernel Stein discrepancy (Gorham et al., 2019).
3.3 How does GSVGD approximate MCMC dynamics?
While the derivation of GSVGD originates from the flow interpretation on WH, the analogy between SVGD and GSVGD expresses in equivalent alternative forms discussed by previous literature, further shedding light on the interpretation of GSVGD.
Originally, the SVGD update expressed the form of the functional gradient (Liu and Wang, 2016). For MCMC dynamics, we similarly derive the functional gradient with respect to the push-forward measure Pa- = (id + (A + C) v) P. V HDD
T
V KL KL [p F = EE' TAA.CK.K..O k(- --x A.C (p). (p).
(19)
This presents a generalization of the discussion in Liu and Wang (2016) that GS takes incremental transformation of particles the direction that minimizes KL-divergence, with such transformation warped by (A, C) and constrained by RKHS.
Alternatively, Liu et al. (2019a) explains SVGD as the projection of the original gradient flow direction v(p.) = V log T7 Ps onto an RKHS. We similarly generalize this explanation for SVGD, as it projects v.C. (pt) = (A + C log 7/p: C. onto H!
A.C. HH
(p)
max arg max ( v) VEHP v
(20)
The analogy of GSVGD goes further when we consider the transportation of N particles as inferring a joint distribution 2N which will be covered the supplements.
3.4 GSVGD in practice
We can apply GSVGD updates (16) to de-randomize myriad MCMC dynamics (e.g., Table 1). Notably, GSVGD establishes the previously unexplored duality between Riemannian Langevin diffusion (Girolami and Calderhead, 201 1) and Riemannian SVGD (Liu and Zhu, 2017).
To fully harness the capacity of our framework, we introduce auxiliary (momentum) variables r to help with the exploration of probability space, namely to augment the target distribution as T (0,r)) (OO)N(rOO Σ), therefore achieving de-randomized PARVI variant of SGHMC (Chen et al., 2014). Further leveraging a positive definite G(0) to efficiently explore the target distribution yields PARVI variant of SGRHMC (Ma et al., 2015).
When GSVGD is used in conjunction with auxiliary momentum variables, we can employ symmetric splitting for leapfrog- like steps for GSVGDs de- de-randomizing with momentum,
1/2 kk
(i)
VH
A.C
Tk
( ((0k.rk))
(21)
0 1
(
+ +vv H
A.C
(o(0)
((0,)Tkt3/2))
(22)
r F- k+1/2
(0)
(
+ 2
E A,C
H
(I(0)
(Fx41/2 p(0 ((112),
(23)
7

---

SGRHIMC-Stein
-2-
Figure 4: Evolution of RHMC dynamics (blue lines) over four highlighted time increments (red points) on tri-Gaussian density. The RHMC explores the space by an aggressive outward expansion followed by convergence the tri-Gaussian density
Experiments
4.1 Stein PARVI on toy data
To demonstrate the efficacy of our methods, we use various PARVI to infer 3-component mixture of crescent-shaped target measure inspired by Ma et al. (2015). The parametrizations of Riemannian SVGD and SGRHMC-Stein coincide with those in Ma et al. (2015). While all methods explore the nearest mode well (See figure 2), only the SGRHMC-Stein manage to effectively use the Riemannian formulation of (A, C)) to explore the 2 other modes (see figure 3), showcasing the gain in efficiency with ioplicaation-spcciin dynamics.
4.2 GSVGD on Bayesian neural networks
We apply GSVGD of advanced MCMC dynamics to the inference of Bayesian neural networks, taking a simple structure of one hidden layer, and an output with Gaussian likelihood. We opt for the fully Bayesian specification of BNN, where the precision parameter of the weight prior and the Gaussian likelihood follows Gamma(1, 0.1). The GSVGD variant used in the experiments include SGHMC (Chen al., 2014) and SGNHT (Ding al., 2014), with tar-
and 0
0
get measure (0,r, Z(0N
0 AI
0
0
I
I
0 - (r)
(po²)-Idiagly As a MCMC form suitable when used in conjunc-
0
tion with stochastic gradients, SGNHT takes additional temperature parameter g. The log-likelihoce results in Table demonstrates that the de-randomized MCMC dynamics achieve superior predictive performance than their LD counterparts, largely thanks the exploration of probability space from the momentum variables (See figure 5), with a notable gain from symmetric splitting. SGNHT-Stein methods show robustness to hyperparameter selection and stochastic gradient noise.
Method
boston
concrete
energy
kinSnm
power
yacht
year
protein
-2,52 (0.15)
(0.04)
-2.36 (0.06)
(0.03)
(0.03)
-1.60 (0.08)
(0.01)
(0.01)
SVGD
-2.60 (0.46)
-3.11 (0.10)
-1.95 (0.07)
1.03 (0.25)
-2882 (0.03)
-1.66 (0.17)
(0.00)
(0.01)
Blob
(0.32)
-3.19 (0.07)
-1.67 (0.03)
0.95 (0.03)
-281 (0.04)
-1.34 (0.07)
(0.00)
-2.93 (0.01)
AGHMC-Blub
-2.42
-2.95 (0.01)
-1.36 (0.26)
1.23 (0.02)
-2.77 (0.04)
-0.73 (0.46)
(..02
-3.73 (0.16)
SGNHT
-2.59 (0.13)
33.41 (0.05)
-2.4 (0.04)
0.75 (0.02)
-2.86 (0.02)
.2.44 (0.04)
-3.69 (0.00)
-3.01 (0.01)
-2.68
-2.6 (0.15)
-0.45
-2.80 (0.03)
-1.04 (0.40)
-3.68
-3.00 (0.01)
SGHMC-Stein
-2.81 (0.71)
-3.04 (0.25)
-1.40 (0.82)
1,25 (0.02)
-2.76 (0.04)
-0.86 (0.33)
-3,59 (0.00)
-2.90 (0.01)
SONHT-Stein
(0.30
-2.97 (0.11)
-0.44 (0.10)
(0.03)
-2.78 (0.04)
-0.85 (0.21)
-3.59 (0.00)
-2.85 (0.01)
Table 2: The test log- likelioodd (higher better) results for selected datasets in UCI repository. The
results are reported in mean (standard deviation) form averaged over 20 runs in the first 6 columns and 6 runs in the last 2, with the best performing model marked in boldface.

---

year
power
concrete
kin8nm
-28-
-30
12:
336-
-32
37
-34-
0.6
336
33
SCHMC- Stein
SCHMC Blob
-39
-3.
10000
20000
4000
4000
4000
Figure 5:
The trace plots of test log-likelihoods of BNN experiments: the x-axis determines the iterations. The figures show the effect the momentum variables in SGHMC-Stein and SGNHT- Stein in exploring the posterior space, and quickly converging better optima. The jaggedness of SVGD showcases that while SVGD empirically works with stochastic gradient, is less robust with stochastic gradient noise.
5 Discussion
In this section, we discuss open questions pertinent to the "optimality" of the diffusion Stein operator as PARVI update, and direction for future work regarding the fiber-gradient Hamiltonian flow of the chi-squared divergence, inspired by Chewi et al. (2020).
5.1 Alternatives to the diffusion Stein operator
The vector field (p. = (A + C) V log * is not the unique vector field that induces the time evolution of MCMC dynamics: indeed, two vector fields v1, V2 induce the same curve in P(22) when they differ by divergence-free field: V (p(vi(p) v2(0))) O. Notably, Liu et al. (2019b) propose an alternative vector field v v.A. (p) (p) CV log + C. A projection
p
RKHS yields a Stein PARVI with update
0 = Ep.
(A C) k(,,,)) AV,
(24)
π
consistent with the standard formulation infinitesimal generator. However, the expectation in (24) notably does not converge zero when P: (Gorham et al., 2019), suggesting that the particles will keep rotating along the trajectories of a divergence-free vector field, as opposed to stopping, in the equilibrium state. Interested readers can find experiments with respect to this alternative form in the supplementary materials.
5.2 Generalizing LAWGD with the diffuusio Stein operator
The application of the diffusion Stein operator generalizes Laplacian Adjusted Wasserstein Gradient Descent (LAWGD) (Chewi et al., 2020), which views SVGD as a kernelized Wasserstein gradient flow driven by the chi-squared divergence. Given the first variation of the chi-squared divergence
8x3 Pplll [
A.C = K.p (A 22 we can verify that the vector field VH
C) V log I is rewritten as
2v A.C - Kr (A
C)
V 2T
Krr (A+ CJV
8x2 [pt
m]
(25)
8p:
The chi-squared divergence perspectives enriches GSVGD kernelized fiber- fiber-gradient Hamiltonian flow minimizing the chi-squared divergence. Replacing the vector field Krr
(A + C
+e with
V (Kin (A - C) ), we get the dissipation formula for the KL-divergence
dKl.
π Jpt ||
( π
v (K=r (A C) () E,
PP [
E
KG
(26)
dt
In fact. divergence-fiee free vector field C. V log V Co can be constructed out of an arbitrary skew- symmetric matrix-vall valued function Co

---

where A represents the infinitesimal generator that induces the diffusion Stein operator: AA.C 2n V (T + C)) Following the construction of LAWGD, we choose the kernel so that K ( While it remains be what kernel induces this equality, the diffusion Stein operator yields more flexible formulation, while retaining the superior convergence of LAWGD.
Conclusion
We further the application of the Stein operator and its generalizations by proposing GSVGD, an interacting particle system transporting particle set towards a target distribution in an emulation of the corresponding MCMC dynamics. We showcase theoretically and empirically that GSVGD helps augment the well-researched field of efficient MCMC dynamics with deterministic interacting particle systems with high-quality samples.
Acknowledgments and Disclosure of Funding
We wish to acknowledge the computational resources provided by Aalto Science-IT Project from Computer Science IT and CSC-IT Center for Science, Finland. This work has been supported by the Academy of Finland (Flagship programme: Finnish Center for Artificial Intelligence FCAI and grants no. 292334, 294238, 319264, 334600) and UKRI Turing Al World-Leading Researcher Fellowship, EP/W002973/1.
References
Tianqi Chen, Emily Fox, and Carlos Guestrin. Stochastic gradient Hamiltonian Monte Carlo. In International conference on machine learning, pages 1683-1691, 2014.
Yi-An Ma, Tianqi Chen, and Emily B. Fox. A complete recipe for stochastic gradient MCMC. In Advances in Neural Information Processing Systems, volume 28, pages 2917-2925, 2015.
Qiang Liu and Dilin Wang. Stein variational gradient descent: A general purpose Bayesian inference algorithm. In Advances in Neural Information Processing Systems, volume 29, pages 2378-2386, 2016.
Changyou Chen, Ruiyi Zhang, Wenlin Wang, Bai Li, and Liqun Chen. unified particle-optimization framework for scalable Bayesian sampling. In UAI, pages
2018
Chang Liu, Jingwei Zhuo, Pengyu Cheng. Ruiyi Zhang, and Jun Zhu. Understanding and accelerating particle-based variational inference. In International Conference on Machine Learning, pages 4082-4092 2019a.
Richard Jordan, David Kinderlehrer. and Felix Otto. The variational formulation of the Fokker-Plann equation. SIAM Journal on Mathematical Analysis, 29(1):1-17, 1998.
Qiang Liu. Stein variational gradient descent as gradient flow. In Advances in Neural Information Processing Systems, volume 30. pages 3115 3115-3123, 2017.
Chang Liu, Jingwei Zhuo, and Jun Zhu. Understanding MCMC dynamics as flows the Wasserstein space. In International Conference on Machine Learning, pages 4093-4103, 2019b.
A. Duncan, N. Nuesken, and L. Szpruch. On the geometry of Stein variational gradient descent. arXiv preprint arXiv:1912.000
2019.
Mark Girolami and Ben Calderhead. Riemann manifold langevin and hamiltonian monte carlo methods. Journal of The Royal Statistical Society Series B-statistical Methodology, 1232)12 2011.
Max Welling and Yee W. Teh. Bayesian learning via stochastic gradient Langevin dynamics. In Proceedings of the 28th International Conference on Machine Learning, pages 68 681-688, 2011.
Chang Liu and Jun Zhu. Riemannian Stein variational gradient descent for bayesian inference. In AAAL pages 3627-3634, 2017.
10

---

P. Langevin. Sur la theorie du mouvement Brownien. Compt. Rendus, 146:530-523 533, 1908.
Hannes Risken. Fokker-Planck equation. pages 63-95, 1996.
Lars Onsager. Reciprocal relations in irreversible processes. ii. Physical Review, 37(12))2265-277)
1931.
José Antonio Carrillo, Katy Craig, and Francesco S. Patacchini, A blob method for diffusion. Calculus of Variations and Partial Differential Equations, 58(2):1-53, 2019.
Charles Stein. A bound for the enror in the normal approximation to the distribution of sum of dependent random variables. Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability, Volume 2: Probability Theory, 1972.
Jackson Gorham, Andrew B. Duncan, Sebastian J. Vollmer, and Lester Mackey, Measuring sample quality with diffusions. Annals of Applied Probability, 2985):284--2988, 2019.
A. D. Barbour. Stein's method and Poisson process convergence. Journal of Applied Probability, 25: 175-184, 1988.
A. D. Barbour, Stein's method for diffusion approximations. Probability Theory and Related Fields, 84(3):97-322, 1990.
Gotze. On the rate of convergence the multivariate CLT. Annals of Probability, 19(2):724-739, 1991.
Alessandro Barp, Trançois-Xaviee Briol, Andrew B. Duncan, Mark A. Girolami, and Lester W. Mackey. Minimum stein discrepancy estimators. Advances in Neural Information Processing Systems, volume 32, pages 12964-12976, 2019.
Jianyi Zhang, Ruiyi Zhang, Lawrence Carin, and Changyou Chen. Stochastic particle-optimizatin sampling and the non-asymptotic convergence theory. In International Conference on Artificial Intelligence and Statistics, pages 1877-1887, 2020.
Yingzhen Li and Richard E. Turner. Gradient estimators for implicit models.
International
Conference on Learning Representations, 2017.
Yi-An Ma, Niladri S. Chatterji, Xiang Cheng, Nicolas Flammarion, Peter L. Bartlett, and Michael I. Jordan. Is there an analog of Nesterov acceleration for MCMC. arXiv preprint arXiv: 9900000999, 2019.
Yifei Wang and Wuchen Li. Accelerated information gradient flow. arXiv preprint arXiv: 2019.
Nan Ding, Youhan Fang, Ryan Babbush, Changyou Chen, Robert D Skeel, and Hartmut Neven. Bayesian sampling using stochastic gradient thermostats. In Advances in Neural Information Processing Systems 27, volume 27, pages 3203-3211 2014.
Sinho Chewi, Thibaut Le Gouic, Chen Lu, Tyler Maunu, and Philippe Rigollet. SVGD as a kernelized Wasserstein gradient flow of the chi-squared divergence. In Advances in Neural Information Processing Systems, volume 33, pages 2098-2109, 2020.


Reviews:
[{"review_id": "wyCbJLXWevT", "reviewer": "Reviewer_ubFs", "summary": "In this paper the authors generalise the Stein Variational Gradient Descent (SVGD) algorithm by considering an alternative flow, essentially equipping the Stein Geometry with a Riemmanian Poisson structure.   This enables a particle evolution scheme which is no longer a gradient flow of the KL divergence in the Stein geometry.  \n\nThe resulting flow differs from the original SVGD flow through the introduction of an \"irreversible term\" characterised by a Skew-Symmetric matrix $C$.  \n\nAdopting this approach, the authors are able to consider augmented variable strategies to improve the performance of SVGD by promoting superior exploration, using analogues of underdamped Langevin and Nose-Hoover type dynamics.   This is demonstrated on some toy examples and on a Bayesian Neural Network model.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, I find this paper to be an interesting contribution which provides a new generalisation of SVGD.  From a theoretical point of view, this is a close analogy to previous works such as:\n\n[Liu, Chang, Jingwei Zhuo, and Jun Zhu. \"Understanding mcmc dynamics as flows on the wasserstein space.\" International Conference on Machine Learning. PMLR, 2019.]\nwhere a similar programme is undertaken for the Wasserstein gradient flows.\n\nFollowing on that paper as a parallel study, I feel it was important to study the implication of different choices of the matrices A and C.  In particular, we know that for Langevin gradient flows, degenerate choices of $A$ are possible (both zero or just having zero eigenvalues) are possible, and can yield ergodic dynamics in the right setting, e.g. HMC.   It would be have been interest for the authors to study (at least) numerically the influence of the degeneracy of the matrix $A$, as there are no theoretical results to suggest that $\\pi$ is the unique target distribution for SVGD (in the mean field limit).\n\nThe calculations in the supplementary materials appear to be correct, albeit formal.   In section 2.5 the authors study the mean-field limit of the system of SDEs associated with SVGD, however they provide no justification (formal or otherwise) of the mean field limit.  The authors should provide this, or at least point the readers to a paper where something similar is done for the standard SVGD case.\n\nMinor details:\nThroughout:   The authors frequently write the diffusion part of the Fokker-Planck equation as $\\nabla^\\top \\nabla : (...)$.   Wouldn't $\\nabla^\\top \\nabla$ contract the two gradients to give a Laplacian, in which case the tensor contraction : doesn't make sense?   Wouldn't it be $\\nabla \\nabla : Z = \\sum_{i,j} \\frac{\\partial^2}{dx_i dx_j} Z_{ij}$ ?\n\n\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "They have adequately addressed these. ", "review_text": "", "extra_scores": {"ethical_concerns": "There are no obvious ethical concerns", "needs_ethics_review": "Yes", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "vIFnwiRIXGg", "reviewer": "Reviewer_Kj2w", "summary": "The authors propose a generalization of Stein Variational Gradient\nDescent (SVGD) that hopes to produce more reliable surrogate measures\nof an invariant measure pi. SVGD can be viewed as a procedure\nthat is minimizing the KL divergence by taking (functional) gradient\nsteps in an RKHS; it can similarly be viewed as an approximation to\nthe gradient flow induced by Langevin dynamics by accommodating H^d\nas its function space. This paper studies the generalization of SVGD\nthat occurs by replacing Langevin dynamics by the dynamics induced\nfrom an Ito diffusion with invariant measure pi. Using Ito diffusions\nin lieu of the overdamped Langevin diffusion produces deterministic\nanalogs of SGRLD, SGHMC and SGRHMC. This also permits the\nformulation of GSVGD, a variant of SVGD that incorporates auxiliary\nvariables in a flavor similar to HMC. The authors demonstrate the\nefficacy of this PARVI method on a toy 3-component GMM and on\nBayesian neural nets for a suite of datasets.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The main contribution of the paper is the development of GSVGD: a\ngeneralization of SVGD that applies the mechanics of any Ito process\n(rather than the overdamped Langevin diffusion). There are some nice\nconnections to other related concepts, namely a view of this under\na Wasserstein gradient flow. Overall, I thought the paper was a nice\nread that laid out some complicated concepts clearly. My biggest\ncritique of the paper is the limited empirical evidence; it is great\nto see the method working on a toy model, but it would be great to\nadd another experiment demonstrating its efficacy.\n\nSVGD tends to lead to under-dispersed examples in lower dimensions (D < 50)\nand I wonder if those same limitations apply here. Given the benefit\nof this method is a way to harness both a Riemannian structure and also\nmomentum variables, I would also like to see a comparison of this method\nwith SGRHMC. I'm curious how much the determinism here hurts, as MCMC\nmethods tend to work well in high dimensional inference.\n\nOriginality:\nThis is mostly an extension of previous work, so it not completely novel. But the application of this to other operators is a significant improvement.\n\nValidity:\nThe claims in the paper look well-founded and correct.\n\nClarity:\nThe paper was clearly written and easy to follow. There were a few notational issues; see the detailed comments below.\n\nSignificance:\nIf this yielded a more accurate version of SVGD, that could be a significant improvement for PARVI samplers.\n\nDetailed comments:\n\nEqn 2: It might be helpful to define the notation (grad^T grad) : (rho_t I)\nexplicitly somewhere.\n\nL80: \"Given the property of differential operators\": which property are you\nreferring to?\n\nEqn 6: What's v_H\nL183: One might consider citing \"Minimum Stein Discrepancy Estimators\" by\nBarp et al. as another use of the diffusion Stein operator for the purposes\nof estimation.\n\nL215-L220: At some point, the authors should explain how G(theta) is\ndefined, as this is used in the Riemannian methods.\n\nTable 2: Are LD and SGNHT the only stochastic methods benchmarked? Why\nwas SGHMC not experimented with? I'm curious how the deterministic methods\ncompare to the randomized ones.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "No, it might be helpful to draw out how these ideas could have a negative\nimpact on research, e.g., could these methods lead researchers astray?", "review_text": "", "extra_scores": {"ethical_concerns": "NA", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "sYX5hwbNWjr", "reviewer": "Reviewer_FarY", "summary": "The paper combines the results of two previous works: (Liu, 2017) and (Liu et al., 2019). I'll first describe the contribution of the previous works and then will move to the contribution of the current paper. \n\n(Liu, 2017) provides the theoretical analysis of the Stein Variational Gradient Descent (SVGD). It shows that the evolution of the density under SVGD is described by the Vlasov equation (non-linear analog of the Fokker-Planck equation); moreover, in the space of distributions, SVGD defines the gradient descent minimizing the KL divergence between the current point and the target density (similarly, Langevin Dynamics {LD} defines the gradient descent in the Wasserstein space). Importantly, to derive this result (Liu, 2017) introduces a novel metric in the distribution space (H-Wasserstein distance) based on the Stein operator. \n\nIn a similar fashion, (Liu et al., 2019) provides the analysis of different MCMC dynamics (Langevin, HMC, RMHMC, SGHMC) by considering the time evolution of the densities in the space of distributions. They first show how any MCMC dynamics can be reformulated as a completely deterministic procedure by deriving the corresponding vector field in the state space, which induces a vector field in the Wasserstein space. Further, they derive a kind of a Hodge decomposition for the vector field in the Wasserstein space, which provides many insights into existing MCMC dynamics. To describe SGHMC and SG Nose-Hoover thermostat algorithms, the authors introduce the Fiber-Riemannian manifold, which is a Fiber Bundle, but with the Riemannian structure on each fiber. Unlike (Liu, 2017), this paper works in the Wasserstein space (distributions with a finite second momentum equipped with the 2-Wasserstein distance).\n\nThe current paper goes the same way as (Liu et al., 2019), but uses the H-Wasserstein distance (introduced in {Liu, 2017}) for the space of distributions. To be more precise, it starts with the same family of MCMC dynamics (introduced in (Ma et al., 2015)) and then uses Theorem 5 from (Liu et al., 2019) changing the projection operator onto the tangent space, which changes because of the different metric. Substituting the different metric into Theorem 5 from (Liu et al., 2019) the authors extend the previous results by additionally describing the SGRHMC algorithm. \n\n- (Liu, 2017) Qiang Liu. Stein variational gradient descent as gradient flow. In Advances in Neural Information Processing Systems, volume 30, pages 3115–3123, 2017.\n- (Liu et al., 2019) Chang Liu, Jingwei Zhuo, and Jun Zhu. Understanding MCMC dynamics as flows on the Wasserstein space. In International Conference on Machine Learning, pages 4093–4103, 2019.\n- (Ma et al., 2015) Yi-An Ma, Tianqi Chen, and Emily B. Fox. A complete recipe for stochastic gradient MCMC. In NIPS’15 Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 2, volume 28, pages 2917–2925, 2015.", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I would describe the theoretical contribution of the paper as incremental. The authors marginally improve the developments from (Liu et al., 2019) by including one more algorithm into the \"derandomization framework\". Although this inclusion is of course a positive thing, I would classify the paper as practical, targeting the derandomization of the concrete MCMC dynamics (SGRHMC {Stochastic Gradient Riemannian HMC}). Since it is not clear (and not discussed in the paper) which metric provides better convergence, both metrics must be compared empirically (despite that H-Wasserstein includes 2-Wasserstein). I recall that the metrics yield different derandomization schemes of MCMC dynamics.\n\nThe empirical study in the paper is unsatisfactory. The main implication of the proposed framework is the derandomization of SGRHMC, however, the empirical results for this algorithm are demonstrated only for a toy example. Since the proposed framework is built upon the results of (Liu et al., 2019) it must be compared against it where it is possible. However, the convergence plots (Figure 5) compare the proposed algorithm only against SVGD. The comparison in Table 2 is hard to understand since the names of some methods are not described in the text. I guess, that SGHMC-Blob corresponds to a method from (Liu et al., 2019) and it performs comparably to the proposed SGHMC-Stein. I would also suggest comparing the proposed framework to both derandomization schemes from (Liu et al., 2019): equations 7 and 11.\n\nI think the clarity of the paper could be significantly improved. From the beginning, the paper tricks the reader into thinking that it develops the whole framework of MCMC derandomization from scratch. To avoid this possible confusion I would suggest the authors explicitly mention that they start their developments from (Liu et al., 2019), which is already describing a large body of MCMC dynamics. In general, the structure of the paper is not easy to follow, additionally hindered by many minor flows. I will mention some appearance flaws, which should be considered as suggestions:\n1. Figure 1. \"discretization\" under the second arrow is not so clear since we can discretize in space, time, particles.\n2. Figure under Table 1 is not illustrative, the only thing it says is that LD and HMC are stochastic and the rest are deterministic particles. I think the target audience of the paper knows it.\n3. The notation for the matrix scalar product (colon in the text) is not introduced in the main text.\n4. Notations E, x, H^D are not introduced in the main text.\n5. The derivation of the FP equation in section 3.1 is superfluous for further developments.\n6. The formulation of the tangent space (line 165) is important for the result. Therefore, the definition of H^D here would help the reader a lot.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The limitations of the paper are not addressed properly because of the weak comparison against the previous works.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "15 (I was reading previous papers and I'm not so fluent in differential geometry on probabilistic manifolds)"}, "extra_text": {}}, {"review_id": "f13QA5wMTtQ", "reviewer": "Reviewer_BVWr", "summary": "This paper proposes a generalization of the Stein variational gradient descent (SVGD) algorithm via a discretization of the fiber-gradient Hamiltonian flow. The approach that the authors take to extend the framework of particle-based variational inference is similar to the way in which MCMC dynamics are extended through the \"complete recipe\" framework in Ma et al. (2015). The authors provide background material to establish the connection between MCMC dynamics, the Fokker-Planck equation and gradient flow methods. The GSVGD then follows by applying the fiber-gradient Hamiltonian flow ideas from MCMC dynamics to the Stein-Wasserstein metric, which is then discretized to produce the GSVGD algorithm. The authors test the efficacy of their algorithm on a toy data model and a Bayesian neural network.  ", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality – This paper offers an interesting extension to existing work within the SVGD literature. The authors clearly highlight how this work builds upon previous papers and how it differs from published works. The approach taken by the authors, i.e. via the same conceptual route as the complete recipe framework for stochastic gradient MCMC, allows the authors to produce a natural extension of existing particle-based variational inference algorithms and establish connections between existing algorithms. \nQuality – The paper appears to be technically sound with accurate derivations. The mathematical and experimental claims are well-supported in the paper. However, the simulation study is perhaps a little short compared to other similar papers. In particular, it would have been nice if the authors were able to identify particular machine learning models, aside from toy models, where the GSVGD approach would be expected to perform better than existing algorithms. Altogether, this is a complete piece of work that offers a nice contribution to the existing literature, while also providing interesting avenues for future development. \nClarity – The paper is very well-written, and the structure of the paper is well-organized. The authors provide a thorough and concise review of existing and related techniques which fit into their framework. The paper provides enough detail for a reader to recreate this work and further derivations provided in the Supplementary Material provide the extra steps needed to fill in the gaps in any of the derivations from the main paper. \nSignificance – The mathematical results provided in the paper are significant and are very likely to be used by other researchers to develop this line of research further. Due to the limited experimental results, it isn’t clear at this stage whether the GSVGD framework will lead to significant empirical improvements in approximate Bayesian inference, however, the results provided so far, and the experience from the MCMC dynamics community, would suggest that this new class of algorithm will be beneficial. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitations are partially discussed and no societal impact issues are raised, nor are there expected to be any societal impact issues from this work. ", "review_text": "", "extra_scores": {"ethical_concerns": "None", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3 hours"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes an elegant generalisation of the Stein variational gradient descent method that introduces additional degrees of freedom into how the dynamics are defined, together with empirical evidence in support of this additional flexibility being practically useful. All reviewers agreed on the correctness of the method and that the paper is mostly well-written, but there was one reviewer who viewed the contribution as too incremental.  The paper addresses an important open question - deterministic alternatives to MCMC - which will be of general interest at NeurIPS.
Decision: Accept (Poster)