Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: In this paper the authors generalise the Stein Variational Gradient Descent (SVGD) algorithm by considering an alternative flow, essentially equipping the Stein Geometry with a Riemmanian Poisson structure.   This enables a particle evolution scheme which is no longer a gradient flow of the KL divergence in the Stein geometry.  

The resulting flow differs from the original SVGD flow through the introduction of an "irreversible term" characterised by a Skew-Symmetric matrix $C$.  

Adopting this approach, the authors are able to consider augmented variable strategies to improve the performance of SVGD by promoting superior exploration, using analogues of underdamped Langevin and Nose-Hoover type dynamics.   This is demonstrated on some toy examples and on a Bayesian Neural Network model.

main_review: Overall, I find this paper to be an interesting contribution which provides a new generalisation of SVGD.  From a theoretical point of view, this is a close analogy to previous works such as:

[Liu, Chang, Jingwei Zhuo, and Jun Zhu. "Understanding mcmc dynamics as flows on the wasserstein space." International Conference on Machine Learning. PMLR, 2019.]
where a similar programme is undertaken for the Wasserstein gradient flows.

Following on that paper as a parallel study, I feel it was important to study the implication of different choices of the matrices A and C.  In particular, we know that for Langevin gradient flows, degenerate choices of $A$ are possible (both zero or just having zero eigenvalues) are possible, and can yield ergodic dynamics in the right setting, e.g. HMC.   It would be have been interest for the authors to study (at least) numerically the influence of the degeneracy of the matrix $A$, as there are no theoretical results to suggest that $\pi$ is the unique target distribution for SVGD (in the mean field limit).

The calculations in the supplementary materials appear to be correct, albeit formal.   In section 2.5 the authors study the mean-field limit of the system of SDEs associated with SVGD, however they provide no justification (formal or otherwise) of the mean field limit.  The authors should provide this, or at least point the readers to a paper where something similar is done for the standard SVGD case.

Minor details:
Throughout:   The authors frequently write the diffusion part of the Fokker-Planck equation as $\nabla^\top \nabla : (...)$.   Wouldn't $\nabla^\top \nabla$ contract the two gradients to give a Laplacian, in which case the tensor contraction : doesn't make sense?   Wouldn't it be $\nabla \nabla : Z = \sum_{i,j} \frac{\partial^2}{dx_i dx_j} Z_{ij}$ ?




limitations_and_societal_impact: They have adequately addressed these.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper proposes a generalization of the Stein variational gradient descent (SVGD) algorithm via a discretization of the fiber-gradient Hamiltonian flow. The approach that the authors take to extend the framework of particle-based variational inference is similar to the way in which MCMC dynamics are extended through the "complete recipe" framework in Ma et al. (2015). The authors provide background material to establish the connection between MCMC dynamics, the Fokker-Planck equation and gradient flow methods. The GSVGD then follows by applying the fiber-gradient Hamiltonian flow ideas from MCMC dynamics to the Stein-Wasserstein metric, which is then discretized to produce the GSVGD algorithm. The authors test the efficacy of their algorithm on a toy data model and a Bayesian neural network.  
main_review: Originality – This paper offers an interesting extension to existing work within the SVGD literature. The authors clearly highlight how this work builds upon previous papers and how it differs from published works. The approach taken by the authors, i.e. via the same conceptual route as the complete recipe framework for stochastic gradient MCMC, allows the authors to produce a natural extension of existing particle-based variational inference algorithms and establish connections between existing algorithms. 
Quality – The paper appears to be technically sound with accurate derivations. The mathematical and experimental claims are well-supported in the paper. However, the simulation study is perhaps a little short compared to other similar papers. In particular, it would have been nice if the authors were able to identify particular machine learning models, aside from toy models, where the GSVGD approach would be expected to perform better than existing algorithms. Altogether, this is a complete piece of work that offers a nice contribution to the existing literature, while also providing interesting avenues for future development. 
Clarity – The paper is very well-written, and the structure of the paper is well-organized. The authors provide a thorough and concise review of existing and related techniques which fit into their framework. The paper provides enough detail for a reader to recreate this work and further derivations provided in the Supplementary Material provide the extra steps needed to fill in the gaps in any of the derivations from the main paper. 
Significance – The mathematical results provided in the paper are significant and are very likely to be used by other researchers to develop this line of research further. Due to the limited experimental results, it isn’t clear at this stage whether the GSVGD framework will lead to significant empirical improvements in approximate Bayesian inference, however, the results provided so far, and the experience from the MCMC dynamics community, would suggest that this new class of algorithm will be beneficial. 

limitations_and_societal_impact: Limitations are partially discussed and no societal impact issues are raised, nor are there expected to be any societal impact issues from this work.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The authors propose a generalization of Stein Variational Gradient
Descent (SVGD) that hopes to produce more reliable surrogate measures
of an invariant measure pi. SVGD can be viewed as a procedure
that is minimizing the KL divergence by taking (functional) gradient
steps in an RKHS; it can similarly be viewed as an approximation to
the gradient flow induced by Langevin dynamics by accommodating H^d
as its function space. This paper studies the generalization of SVGD
that occurs by replacing Langevin dynamics by the dynamics induced
from an Ito diffusion with invariant measure pi. Using Ito diffusions
in lieu of the overdamped Langevin diffusion produces deterministic
analogs of SGRLD, SGHMC and SGRHMC. This also permits the
formulation of GSVGD, a variant of SVGD that incorporates auxiliary
variables in a flavor similar to HMC. The authors demonstrate the
efficacy of this PARVI method on a toy 3-component GMM and on
Bayesian neural nets for a suite of datasets.
main_review: The main contribution of the paper is the development of GSVGD: a
generalization of SVGD that applies the mechanics of any Ito process
(rather than the overdamped Langevin diffusion). There are some nice
connections to other related concepts, namely a view of this under
a Wasserstein gradient flow. Overall, I thought the paper was a nice
read that laid out some complicated concepts clearly. My biggest
critique of the paper is the limited empirical evidence; it is great
to see the method working on a toy model, but it would be great to
add another experiment demonstrating its efficacy.

SVGD tends to lead to under-dispersed examples in lower dimensions (D < 50)
and I wonder if those same limitations apply here. Given the benefit
of this method is a way to harness both a Riemannian structure and also
momentum variables, I would also like to see a comparison of this method
with SGRHMC. I'm curious how much the determinism here hurts, as MCMC
methods tend to work well in high dimensional inference.

Originality:
This is mostly an extension of previous work, so it not completely novel. But the application of this to other operators is a significant improvement.

Validity:
The claims in the paper look well-founded and correct.

Clarity:
The paper was clearly written and easy to follow. There were a few notational issues; see the detailed comments below.

Significance:
If this yielded a more accurate version of SVGD, that could be a significant improvement for PARVI samplers.

Detailed comments:

Eqn 2: It might be helpful to define the notation (grad^T grad) : (rho_t I)
explicitly somewhere.

L80: "Given the property of differential operators": which property are you
referring to?

Eqn 6: What's v_H
L183: One might consider citing "Minimum Stein Discrepancy Estimators" by
Barp et al. as another use of the diffusion Stein operator for the purposes
of estimation.

L215-L220: At some point, the authors should explain how G(theta) is
defined, as this is used in the Riemannian methods.

Table 2: Are LD and SGNHT the only stochastic methods benchmarked? Why
was SGHMC not experimented with? I'm curious how the deterministic methods
compare to the randomized ones.
limitations_and_societal_impact: No, it might be helpful to draw out how these ideas could have a negative
impact on research, e.g., could these methods lead researchers astray?
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: The paper combines the results of two previous works: (Liu, 2017) and (Liu et al., 2019). I'll first describe the contribution of the previous works and then will move to the contribution of the current paper. 

(Liu, 2017) provides the theoretical analysis of the Stein Variational Gradient Descent (SVGD). It shows that the evolution of the density under SVGD is described by the Vlasov equation (non-linear analog of the Fokker-Planck equation); moreover, in the space of distributions, SVGD defines the gradient descent minimizing the KL divergence between the current point and the target density (similarly, Langevin Dynamics {LD} defines the gradient descent in the Wasserstein space). Importantly, to derive this result (Liu, 2017) introduces a novel metric in the distribution space (H-Wasserstein distance) based on the Stein operator. 

In a similar fashion, (Liu et al., 2019) provides the analysis of different MCMC dynamics (Langevin, HMC, RMHMC, SGHMC) by considering the time evolution of the densities in the space of distributions. They first show how any MCMC dynamics can be reformulated as a completely deterministic procedure by deriving the corresponding vector field in the state space, which induces a vector field in the Wasserstein space. Further, they derive a kind of a Hodge decomposition for the vector field in the Wasserstein space, which provides many insights into existing MCMC dynamics. To describe SGHMC and SG Nose-Hoover thermostat algorithms, the authors introduce the Fiber-Riemannian manifold, which is a Fiber Bundle, but with the Riemannian structure on each fiber. Unlike (Liu, 2017), this paper works in the Wasserstein space (distributions with a finite second momentum equipped with the 2-Wasserstein distance).

The current paper goes the same way as (Liu et al., 2019), but uses the H-Wasserstein distance (introduced in {Liu, 2017}) for the space of distributions. To be more precise, it starts with the same family of MCMC dynamics (introduced in (Ma et al., 2015)) and then uses Theorem 5 from (Liu et al., 2019) changing the projection operator onto the tangent space, which changes because of the different metric. Substituting the different metric into Theorem 5 from (Liu et al., 2019) the authors extend the previous results by additionally describing the SGRHMC algorithm. 

- (Liu, 2017) Qiang Liu. Stein variational gradient descent as gradient flow. In Advances in Neural Information Processing Systems, volume 30, pages 3115–3123, 2017.
- (Liu et al., 2019) Chang Liu, Jingwei Zhuo, and Jun Zhu. Understanding MCMC dynamics as flows on the Wasserstein space. In International Conference on Machine Learning, pages 4093–4103, 2019.
- (Ma et al., 2015) Yi-An Ma, Tianqi Chen, and Emily B. Fox. A complete recipe for stochastic gradient MCMC. In NIPS’15 Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 2, volume 28, pages 2917–2925, 2015.
main_review: I would describe the theoretical contribution of the paper as incremental. The authors marginally improve the developments from (Liu et al., 2019) by including one more algorithm into the "derandomization framework". Although this inclusion is of course a positive thing, I would classify the paper as practical, targeting the derandomization of the concrete MCMC dynamics (SGRHMC {Stochastic Gradient Riemannian HMC}). Since it is not clear (and not discussed in the paper) which metric provides better convergence, both metrics must be compared empirically (despite that H-Wasserstein includes 2-Wasserstein). I recall that the metrics yield different derandomization schemes of MCMC dynamics.

The empirical study in the paper is unsatisfactory. The main implication of the proposed framework is the derandomization of SGRHMC, however, the empirical results for this algorithm are demonstrated only for a toy example. Since the proposed framework is built upon the results of (Liu et al., 2019) it must be compared against it where it is possible. However, the convergence plots (Figure 5) compare the proposed algorithm only against SVGD. The comparison in Table 2 is hard to understand since the names of some methods are not described in the text. I guess, that SGHMC-Blob corresponds to a method from (Liu et al., 2019) and it performs comparably to the proposed SGHMC-Stein. I would also suggest comparing the proposed framework to both derandomization schemes from (Liu et al., 2019): equations 7 and 11.

I think the clarity of the paper could be significantly improved. From the beginning, the paper tricks the reader into thinking that it develops the whole framework of MCMC derandomization from scratch. To avoid this possible confusion I would suggest the authors explicitly mention that they start their developments from (Liu et al., 2019), which is already describing a large body of MCMC dynamics. In general, the structure of the paper is not easy to follow, additionally hindered by many minor flows. I will mention some appearance flaws, which should be considered as suggestions:
1. Figure 1. "discretization" under the second arrow is not so clear since we can discretize in space, time, particles.
2. Figure under Table 1 is not illustrative, the only thing it says is that LD and HMC are stochastic and the rest are deterministic particles. I think the target audience of the paper knows it.
3. The notation for the matrix scalar product (colon in the text) is not introduced in the main text.
4. Notations E, x, H^D are not introduced in the main text.
5. The derivation of the FP equation in section 3.1 is superfluous for further developments.
6. The formulation of the tangent space (line 165) is important for the result. Therefore, the definition of H^D here would help the reader a lot.
limitations_and_societal_impact: The limitations of the paper are not addressed properly because of the weak comparison against the previous works.
review_title: null

Meta-review: This paper proposes an elegant generalisation of the Stein variational gradient descent method that introduces additional degrees of freedom into how the dynamics are defined, together with empirical evidence in support of this additional flexibility being practically useful. All reviewers agreed on the correctness of the method and that the paper is mostly well-written, but there was one reviewer who viewed the contribution as too incremental.  The paper addresses an important open question - deterministic alternatives to MCMC - which will be of general interest at NeurIPS.
Decision: Accept (Poster)