Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper studies the general gradient flow of KL divergences. The authors consider a generalized metric space and formulate the gradient descent as a least square problem. The metric spaces include L^2, RKHS, and covariance-weighted norms. Some linear and nonlinear families of functions, including neural networks, are used in approximating generalized metric spaces and gradients. Numerical examples are shown the effectiveness of the proposed methods. 
strength_and_weaknesses: Strength: The authors propose general metric space-induced gradient descent to design sampling algorithms. The generality of metrics (Q) may improve the sampling algorithm.  

Weakness: 

1. No convincing examples are provided analytically. I suggest that the authors study a Gaussian target distribution. In this case, the author may illustrate the advantage of general metrics and gradients. 

2. The authors miss many crucial works in this field.  Please illustrate them in the context.

A. Garbuno-Inigo, F. Hoffmann, W.C. Li, A.M. Stuart. Interacting Langevin Diffusions: Gradient Structure And Ensemble Kalman Sampler. SIAM Journal on applied dynamical system, 2019. 

W.C. Li, L.X. Ying. Hessian transport gradient flows. Research in the Mathematical Sciences, 2019. 

Y.F. Wang, W.C. Li. Information Newton flow: second-order optimization method in probability space. 2020. 

Y.F. Wang, W.C. Li. Accelerated Information Gradient flow. 

Y.F. Wang, P. Chen, W.C. Li. Projected Wasserstein gradient descent for high-dimensional Bayesian inference. 

Y.F. Wang, P. Chen, M. Pilanci, W.C. Li. Optimal Neural Network Approximation of Wasserstein Gradient Direction via Convex Optimization. 

W.C. Li, A.T. Lin, G. Montufar. Affine Natural Proximal Learning. GSI, 2019.

A.T. Lin, W.C. Li, S. Osher, G. Montufar. Wasserstein Proximal of GANs. GSI, 2021. 

clarity,_quality,_novelty_and_reproducibility: The quality of this paper is good. 
summary_of_the_review: The paper is written well with clear mathematics. Some analytical examples and important literature are missed.  It will take effect my ratings.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a new variational inference method for sampling from the distribution. The proposed method is a sort of extension of the SVGD method inspired by functional gradient boosting. That is, the drift term is modeled by the neural networks and is optimized to maximize the discrepancy under an appropriate regularization. This work also provides a convergence guarantee under the assumptions regarding the approximation error to the preconditioned Wasserstein gradient. The effectiveness of the method is empirically verified on several tasks: sampling from Gaussian mixture, ill-conditioned Gaussian distribution, and Bayesian and hierarchical logistic regressions.
strength_and_weaknesses: **Strengths**:

The proposed method can be seen as a proper extension of Stein variational gradient descent (SVGD). Indeed, SVGD is a special case of the proposed method through the explicit feature representation of the kernel. In this sense, this work makes a certain contribution to the context by proposing a general form of SVGD. 

**Weaknesses**:

The motivation for using neural networks for representing the drift term is somewhat unclear. Indeed, Langevin dynamics can be directly applied for sampling from Gibbs distribution $p_*$ and this dynamics does not require the approximation. (Note that Langevin dynamics also solves Fokker-Planck equation Eq. (1) with the Wasserstein gradient.) Thus, I think the standard Langevin dynamics is sufficiently powerful in terms of optimization. Moreover, there are also preconditioned variants of Langevin dynamics:

1. Max Welling and Yee Whye Teh. Bayesian Learning via Stochastic Gradient Langevin Dynamics.
2. Gaétan Marceau-Caron and Yann Ollivier. Natural Langevin Dynamics for Neural Networks.
3. C. Li et al., Preconditioned Stochastic Gradient Langevin Dynamics for Deep Neural Networks.
4. A. Wibisono. Proximal Langevin Algorithm: Rapid Convergence Under Isoperimetry

These preconditioned Langevin dynamics can be competitors to the proposed method. Thus, the discussion of these methods would be helpful. In particular,  [WIbisono] (see Section 4.1) considered the dynamics which optimizes Eq. (7) in the submission. 
clarity,_quality,_novelty_and_reproducibility: This paper is well-organized and easy to read. The proof technique is rather straightforward.

Question: 

- Could you elaborate on why $-c \nabla^2 \log p_*$ is better preconditioning? A bit detailed explanation of this would be helpful.
- Is there a connection between Assumption A3 and weak learning conditions for boosting methods?
summary_of_the_review: The proposed method is interesting because it is a proper extension of SVGD. But, the motivation for using neural networks as the drift term is rather weak. More discussion to motivate the use of the method will make the paper stronger.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposed a new partical-based variational inference method by minimization the functional gradient of KL with a regularization where preconditioning seems to be crucial for the performance. The efficiency compared to SVGD is illustrated on several standard benchmark data sets. Relations to previous work were also discussed.
strength_and_weaknesses: Strength:

1. The precondition functional gradient flow for particle-based VI seems to be new.
2. The method has been well motivated with clear theoretical justification.
3. The advantages over SVGD has been clearly stated and illustrated in the experiments.


Weaknesses:

1. The preconditioning seems to be crucial for performance, lack of ablation study except for the toy examples.
2. No theoretical analysis for the precondition matrix H.
3. Discretization error of ODE is not explicitly considered.


clarity,_quality,_novelty_and_reproducibility: The paper is clear and well written. However, learning functional gradient flows for particle-based VI is not new, and similar ideas (Hu et al., 2018; Grathwohl et al., 2020) have been exploited eariler. More specifically, this paper is highly related to a previous workshop paper: neural variational gradient descent (Lauro Langosco et al., 2021). Despite this, the theoretical analysis part and precondition trick is new.
summary_of_the_review: The paper proposed a new precondition functional gradient flow approach for particle-based VI and demonstrated its efficiency on several benchmark problems against SVGD. The method can be viewed as a score matching approach that learns the discrepancy of the target score and current particle scores, and update the particles according to the learned functional gradient flows. In that sense, the paper is highly related to a recent paper called neural variational gradient descent by Lauro Langosco et al., 2021. Despite this, the paper do provide some theoretical analysis regarding convergence given well trained functional gradients and introduce the precondition trick which seems to be crucial for performance. I, therefore, recommend it for a weak acceptance.


minor issue: there is a mistake in the upper bound of dKL/dt term in Theorem 1.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper proposes a new particle based variational inference method that utilizes a preconditioning. The method estimates the gradient direction in the continuity equation by minimizing an L2 distance between the best direction with a preconditioned quadratic loss. The authors show the convergence of the algorithm in the infinite particle limit and the continuous time setting. The proposed method is examined by numerical experiments and its effectiveness is justified. 

The proposed method is indeed new, and the theoretical justification is solid. The particle based variational method is one of the important issues in the literature. This paper gives a new insight to the literature. Overall, the paper is well written.  
On the other hand, there are also some limitations. Convergence of discrete time dynamics and finite particle discretization is not given. The choice of H is not thoroughly exploited. Moreover, the motivation to utilize the neural network is not justified in a fully convincing way. Although the authors gave some additional remarks on the comparison with Langevin dynamics approaches, its comparison is not thorough. The author also stated that the Metropolis-Hastings correction is ad-hoc while it has a proper theoretical justification. However, this method lacks the convergence for discrete time and finite particle approximations. In that sense, comparison to Langevin type MCMC methods is not fully convincing. Finally, the experiments are conducted only on a simple setting.  

However, this paper gives a nice contribution to the literature of the deterministic particle based variational method. The proposed method is sufficiently novel. Hence, I think it can appear in ICLR. 




justification_for_why_not_higher_score: As stated in the summary tab, this paper has some weakness (1) lack of thorough theoretical justification, (2) weak comparison with existing methods such as Metropolis Adjusted Langevin Algorithm, and (3) weak numerical experiments. Although this study is novel and significant, this paper would not be appropriate for spotlight due to these weakness. 
justification_for_why_not_lower_score: The proposed algorithm is sufficiently novel. Introducing the preconditioning actually improves the performance. The theoretical justification is solid, and the paper is well written. Hence, this paper deserves acceptance.
Decision: Accept: poster