Paper: Convergence of SVGD in KL divergence via approximate gradient flow (ICLR-2024)

Reviewer comment (Theory):
The key assumption is not clear and hard to verify. And I think there are some mistakes in the proof (see Question).

Author response:
**For question 1 (related to weakness 1):**  
  - Assumptions 5 and 6 do not contradict. Assumption 6 states that each eigenvalue $\lambda_{i}$ (for all $i$) has a positive upper and lower bound **independent of $t$**.
Let us focus on the RBF kernel $k_{\mathrm{RBF}}$ for simplicity.
In this case, it is known that $\lambda_{i} = \mathcal{O}(e^{-i \log i})$, thereby establishing $\lambda_{i} = \Omega(e^{-i \log i})$ (refer to Example 12.25 in [Wainwright, 2019]). This fact verifies the fulfillment of Assumption 6, as $\lambda_{i}$ is a constant order w.r.t. $t$.
Additionally, given that the RBF kernel satisfies all conditions of Mercer's theorem (refer to Section 12.3 in [Wainwright, 2019]), we can employ the Mercer expansion for $k_{\mathrm{RBF}}$ to derive $k_{\mathrm{RBF}}(x,x) = \sum_{i=1}^{\infty} \lambda_{i}\phi_{i}^{2} < \infty$. This observation confirms the validity of Assumption 5.

- **For question 2 (related to weakness 1):**  Thank you for taking the time to peruse the proofs. This concern is similar to reviewer Spa3's Q.2, so please see our response to it (please see p.20 of the modified version of this paper).

Reviewer comment (Novelty):
Basically, the method is to prove that $I\_{stein} (\mu\_t | \pi) \geq c\_0 \|\nabla \log\frac{\mu\_t}{\pi}\|^2_{L^2(\mu\_t)}$ for some positive constant $c$ and apply LSI. The contribution is not significant enough as an ICLR paper.

Author response:
**For weaknesses 2-3, and question 3:** 
  - The assertion that $\delta=0$ in our AGF is not readily evident from existing knowledge, such as LSI. Instead, it is a theoretical result that becomes apparent through analyzing the gradient flow of SVGD under our AGF, employing spectral decomposition and others. To commence, as [Duncan et al. (2023)] noted, conducting convergence analysis with (stein) LSI on RKHS has proven challenging. In this context, we have shown that we can evaluate convergence through "LSI-like" inequalities by demonstrating that SVGD, under the infinite particle setting, achieves $\delta=0$ in our introduced AGF. Utilizing this finding, we show the sub-linear convergence of SVGD in KL divergence. All our theoretical analysis results are non-trivial and can be attained by introducing our AGF rather than relying solely on conventional LSI.

Reviewer comment (Theory):
In Assumption 6, $\{\lambda_i\}$ have positive lower bounds independent of $t$. However, Assumption 5 implies the kernel has finite trace and $\sum_i \lambda_i< \infty$. This will contradict Assumption 6. How should Assumption 6 be understood here?

Author response:
**For question 1 (related to weakness 1):**  
  - Assumptions 5 and 6 do not contradict. Assumption 6 states that each eigenvalue $\lambda_{i}$ (for all $i$) has a positive upper and lower bound **independent of $t$**.
Let us focus on the RBF kernel $k_{\mathrm{RBF}}$ for simplicity.
In this case, it is known that $\lambda_{i} = \mathcal{O}(e^{-i \log i})$, thereby establishing $\lambda_{i} = \Omega(e^{-i \log i})$ (refer to Example 12.25 in [Wainwright, 2019]). This fact verifies the fulfillment of Assumption 6, as $\lambda_{i}$ is a constant order w.r.t. $t$.
Additionally, given that the RBF kernel satisfies all conditions of Mercer's theorem (refer to Section 12.3 in [Wainwright, 2019]), we can employ the Mercer expansion for $k_{\mathrm{RBF}}$ to derive $k_{\mathrm{RBF}}(x,x) = \sum_{i=1}^{\infty} \lambda_{i}\phi_{i}^{2} < \infty$. This observation confirms the validity of Assumption 5.

Reviewer comment (Theory):
If it indeed holds that $I\_{stein} (\mu\_t | \pi) \geq c\_0 \|\nabla \log\frac{\mu\_t}{\pi}\|^2_{L^2(\mu\_t)}$, then by LSI, $I\_{stein} (\mu\_t | \pi) \geq c KL(\mu_t | \pi)$ for some positive constant $c$. This seems just eq (7), which is not valid in most practical cases by Duncan et al. (2023). What is the difference between the authors' claims and Duncan et al. from this perspective?

Author response:
**For weaknesses 2-3, and question 3:** 
  - Further discussion is needed to elucidate the connection between Eq. (7) outlined in [Duncan et al. (2023)] and our findings.
Indeed, our results are tied to Eq. (7); however, [Duncan et al. (2023)] assert that satisfying Eq. (7) for **ANY** distribution $\mu$ is challenging.
While speculative, the distribution $\mu$ may satisfy such inequality within a context that aligns with the assumptions discussed in our paper.
The demonstration by [Liu et al. (2023)] of linear convergence in the KL divergence for SVGD, with the constrained assumption that the target distribution is Gaussian, may imply that Eq. (7) might be valid under specific and restricted conditions.

Reviewer comment (Writing):
In the first paragraph on page 1, what does the sentence ``While MCMC guarantees asymptotically obtaining unbiassed samples from the target distribution, it suffers from inefficiency due to the randomness'' mean?

Author response:
**To the 1st question:** We apologize for the unclear explanation. What we would like to explain here is that MCMC tends to be computationally intensive. We modified this sentence as follows: 
  - “**While MCMC provides guarantees of producing asymptotically unbiased samples from the target density, it tends to be computationally intensive (Robert & Casella, 2004).**”

Reviewer comment (Novelty):
Although this paper focuses on convergence results for the infinite-particle SVGD, convergence results for finite-particle SVGD should be introduced in Section 1. For related results, I refer to the following two papers:  
[1] Shi, Jiaxin, and Lester Mackey. "A finite-particle convergence rate for stein variational gradient descent." arXiv preprint arXiv:2211.09721 (2022).  
[2] Liu, Tianle, et al. "Towards Understanding the Dynamics of Gaussian--Stein Variational Gradient Descent." arXiv preprint arXiv:2305.14076 (2023).

Author response:
Thank you for pointing that out. We agree that [1] and [2] should have been cited. We will include the following sentences with the references of these papers in the third and fourth paragraphs of Sec. 1 for both papers:
  - …like gradient descent. **Beyond the infinite particle setting, Shi & Mackey (2023) also showed that the SVGD with $n$ finite particles and an appropriate step size converges at the $\mathcal{O}(1/\sqrt{ \log \log n })$ order if the target distribution is sub-Gaussian with a Lipschitz score...**

- …function of SVGD. **Recently, Liu et al. (2023) showed that SVGD with finite particles achieves linear convergence in KL divergence under a very limited setting where the target distribution is Gaussian. However, the analytical approach presented in previous studies makes it difficult to conduct convergence analysis based on KL divergence in a more global setting.** The reason…

Reviewer comment (Writing):
**Outlined proof is hard to grasp:** I found the proof outline (section 4.2) particularly dense and hard to follow. Perhaps having a more intuitive discussion and providing a more qualitative intuition would help people grasp the basic idea.

Author response:
**About outlined proof:** We believe that proof outlines, as in Section 4.2, constitute a vital aspect of this paper and should be articulated rigorously, even if their presentation is somewhat dense. What kind of intuitive or qualitative explanation would facilitate a more precise understanding? We would greatly appreciate it if you could provide an example of the type of explanation you would find helpful, even if it is based on personal opinion.

Reviewer comment (Presentation):
**Conclusions are missing:** The paper does not have a conclusion section where the main findings of the work are summarized. I find this to be a substantial lack of the paper as it prevents the reader from drawing the necessary conclusion from the study.

Author response:
**About the conclusion section:** We agree with your opinion. We have introduced a new section titled "Limitations and Conclusions," specifically addressing the content outlined in the above official comment. Please see the revised version of our paper.

Reviewer comment (Experiments):
**Lack of numerical experiments:** While I appreciate the mathematical rigor and details of the manuscript, I perceive the numerical experiments sections (Sec. 5) to be lacking. The results seem consistent with the theory, but as far as I can tell, these are not sufficient to envision any substantial benefit from the framework presented in this work in a practical setting.

Author response:
**About lack of numerical experiments and take-away message from numerical experiments:** Our study analyzes SVGD in the infinite particle setting, acknowledging its unimplementability as an initial step towards furnishing a realistic convergence guarantee for SVGD. To corroborate the validity of the analytical results derived from the infinite particle setup through numerical experiments with finite particles, it is essential to formulate an experimental design capable of completion within a realistic computation time, even when employing a huge number of particles to simulate a near-infinite particle setup. To attain this goal, employing simple examples like the log-concave distribution experiment proves useful. We believe that the discussion of numerical experiments on convergence in more practical experimental settings will constitute a vital element in our forthcoming study on ensuring the convergence of SVGD in the KL divergence within the finite particle setting detailed in our official comment.

Reviewer comment (Novelty):
In the introduction, the second paragraph introduces SVGD as a method to compensate for the high variance and bias from MCMC and VI methods. I am naively wondering how this would compare to other debiasing schemes often used in the context of VI, such as neural importance sampling [1]. It would be helpful if the authors could elaborate a bit on this.

Author response:
**To the 1st question:** MCMC faces scalability issues in handling big data, while VI efficiently approximates posterior distributions, transforming Bayesian inference into a deterministic or stochastic optimization problem. However, the accuracy of VI is influenced by the choice of the set of variational distributions, introducing bias when approximating complex target distributions with simpler ones. To reduce this bias, methods based on importance sampling have been studied; however, they pose the additional challenge that their effectiveness depends on the accuracy of the composition of the proposal distribution (e.g., $q(x; \theta)$ constructed by the neural network in [1]). In this context, SVGD was developed as a more efficient and less biased algorithm that eliminates the need for the bias-reduction methods described above by allowing the removal of constraints imposed by variational distribution sets (refer to Sec. 1 in [Liu et al., 2016] for details). To clarify this point, we modified the 1st and 2nd paragraphs of Sec. 1 as follows:
  - In the 1st paragraph: “... **While MCMC provides guarantees of producing asymptotically unbiased samples from the target density, it tends to be computationally intensive (Robert & Casella, 2004). On the other hand, VI achieves a computationally efficient approximation of the target distribution through stochastic optimization under a simpler alternative distribution; however, it does not come with a guarantee of obtaining unbiased samples (Blei et al., 2017).**”
  - In the top of the 2nd paragraph: “**To alleviate such sample bias while maintaining computational efficiency of VI as much as possible, Liu & Wang (2016) introduced Stein variational gradient descent (SVGD), which allows the direct approximation of the target distribution without the need for alternative distributions.**”

Reviewer comment (Theory):
I understand the paper deals with the case of infinite particles for most of the discussion. This is relevant from a theoretical standpoint, although it cannot be achieved in practice. When it comes to the final discussion, where numerical experiments are shown, the number of particles is indeed considered to be finite, which prevents the bias $\delta_t$ from vanishing. Moreover, the authors correctly state that the regime of an infinite (or even high) number of particles is practically not feasible from a computational perspective. In light of this, I wonder whether the residual bias one has to expect, resulting from the finite number of particles, can be estimated. Expanding the discussion around this point in the manuscript, I think, would be helpful.

Author response:
**To the 3rd question:** Thank you for bringing up this important point. 
For our responses, please refer to our official comments to all reviewers.
We have incorporated a discussion of this particular point into the newly added "Limitation & Conclusion" section.

Reviewer comment (Presentation):
It would be helpful to have the y-axis sharing the same range for the 2 left-most and 2 right-most plots of Figure 1. The same applies for the first and second rows of Figure 2. This would make it easier to compare visually.

Author response:
Concerning the y-axis range, it's important to note that KSD and KL values are incomparable. These plots aim to assess whether KSD and KL achieve sub-linear convergence in a finite particle setting. Adjusting both scales to match could result in a figure that obscures this behavior. We believe the current state provides the most intuitive means to examine sub-linear convergence.