Title: Gradient-Free Kernel Stein Discrepancy
Venue: NeurIPS 2023
Accepted: yes
Keywords: Bayesian

discrepancy

kernel

sampling

Stein's method
Abstract: Stein discrepancies have emerged as a powerful statistical tool, being applied to fundamental statistical problems including parameter inference, goodness-of-fit testing, and sampling.  The canonical Stein discrepancies require the derivatives of a statistical model to be computed, and in return provide theoretical guarantees of convergence detection and control.  However, for complex statistical models, the stable numerical computation of derivatives can require bespoke algorithmic development and render Stein discrepancies impractical.  This paper focuses on posterior approximation using Stein discrepancies, and introduces a collection of non-canonical Stein discrepancies that are gradient-free, meaning that derivatives of the statistical model are not required.  Sufficient conditions for convergence detection and control are established, and applications to sampling and variational inference are presented.

Gradient-Free Kernel Stein Discrepancy

Abstract

Stein discrepancies have emerged as a powerful statistical tool, being applied to fundamental statistical problems including parameter inference, goodness-of-fit testing, and sampling. The canonical Stein discrepancies require the derivatives of a statistical model to be computed, and in return provide theoretical guarantees of convergence detection and control. However, for complex statistical models, the stable numerical computation of derivatives can require bespoke algorithmic development and render Stein discrepancies impractical. This paper focuses on posterior approximation using Stein discrepancies, and introduces a collection of non-canonical Stein discrepancies that are gradient-free , meaning that derivatives of the statistical model are not required. Sufficient conditions for convergence detection and control are established, and applications to sampling and variational inference are presented.

1 Introduction

Stein discrepancies were introduced in Gorham and Mackey [2015], as a way to measure the quality of an empirical approximation to a continuous statistical model involving an intractable normalisation constant. Rooted in Stein's method [Stein, 1972], the idea is to consider empirical averages of a large collection of test functions, each of which is known to integrate to zero under the statistical model. To date, test functions have been constructed by combining derivatives of the statistical model with reproducing kernels [Chwialkowski et al., 2016, Liu et al., 2016, Gorham and Mackey, 2017, Gong et al., 2021a,b], random features [Huggins and Mackey, 2018], diffusion coefficients and functions with bounded derivatives [Gorham et al., 2019], neural networks [Grathwohl et al., 2020], and polynomials [Chopin and Ducrocq, 2021]. The resulting discrepancies have been shown to be powerful statistical tools, with diverse applications including parameter inference [Barp et al., 2019, Matsubara et al., 2022], goodness-of-fit testing [Jitkrittum et al., 2017, Fernandez et al., 2020], and sampling [Liu and Lee, 2017, Chen et al., 2018, 2019, Riabiz et al., 2022, Hodgkinson et al., 2020, Fisher et al., 2021]. However, one of the main drawbacks of these existing works is the requirement that derivatives both exist and can be computed.

The use of non-differentiable statistical models is somewhat limited but includes, for example, Bayesian analyses where Laplace priors are used [Park and Casella, 2008, Roˇ ckov´ a and George, 2018]. Much more common is the situation where derivatives exist but cannot easily be computed. In particular, for statistical models with parametric differential equations involved, one often requires different, more computationally intensive numerical methods to be used if the sensitivities (i.e. derivatives of the solution with respect to the parameters) are to be stably computed [Cockayne and Duncan, 2021]. For large-scale partial differential equation models, as used in finite element simulation, computation of sensitivities can increase simulation times by several orders of magnitude, if it is practical at all.

The motivation and focus of this paper is on computational methods for posterior approximation, and to this end we propose a collection of non-canonical Stein discrepancies that are gradient free , meaning that computation of the derivatives of the statistical model is not required. Gradient-free

Stein operators were introduced in Han and Liu [2018] in the context of Stein variational gradient descent [Liu and Wang, 2016], but the theoretical properties of the corresponding discrepancy have yet to be investigated. General classes of Stein discrepancies were analysed in Huggins and Mackey [2018], Gorham et al. [2019], but their main results do not cover the gradient-free Stein discrepancies developed in this work, for reasons that will be explained. The combination of gradient-free Stein operators and reproducing kernels is studied in detail, to obtain discrepancies that can be explicitly computed. The usefulness of these discrepancies depends crucially on their ability to detect the convergence and non-convergence of sequences of probability measures to the posterior target, and in both directions positive results are established.

Outline Gradient-free KSD (GF-KSD) is proposed and theoretically analysed in Section 2. The proposed discrepancy involves certain degrees of freedom, including a probability density denoted q in the sequel, and strategies for specifying these degrees of freedom are empirically assessed in Section 3. Two applications are then explored in detail; Stein importance sampling (Section 4.1) and Stein variational inference (Section 4.2). Conclusions are drawn in Section 5.

2 Methods

This section contains our core methodological (Section 2.1) and theoretical (Section 2.2) development. The following notation will be used:

Real Analytic Notation For a twice differentiable function f : R d → R , let ∂ i f denote the partial derivative of f with respect to its i th argument, let ∇ f denote the gradient vector with entries ∂ i f , and let ∇ 2 f denote the Hessian matrix with entries ∂ i ∂ j f . For a sufficiently regular bivariate function f : R d × R d → R , let ( ∂ i ⊗ ∂ j ) f indicate the application of ∂ i to the first argument of f , followed by the application of ∂ j to the second argument. (For derivatives of other orders, the same tensor notation ⊗ will be used.)

Probabilistic Notation Let P ( R d ) denote the set of probability distributions on R d . Let δ ( x ) ∈ P ( R d ) denote an atomic distribution located at x ∈ R d . For π, π 0 ∈ P ( R d ) , let π ≪ π 0 indicate that π is absolutely continuous with respect to π 0 . For π ∈ P ( R d ) and ( π n ) n ∈ N ⊂ P ( R d ) , write π n d → π to indicate weak convergence of π n to π . The symbols p and q are reserved for probability density functions on R d , while π is reserved for a generic element of P ( R d ) . For convenience, the symbols p and q will also be used to refer to the probability distributions that these densities represent.

2.1 Gradient-Free Kernel Stein Discrepancy

The aim of this section is to explain how a GF-KSD can be constructed. Let p ∈ P ( R d ) be a target distribution of interest. Our starting point is a gradient-free Stein operator, introduced in Han and Liu [2018] in the context of Stein variational gradient descent [Liu and Wang, 2016]:

Definition 1 (Gradient-Free Stein Operator) . For p, q ∈ P ( R d ) with q ≪ p and ∇ log q welldefined, the gradient-free Stein operator is defined as

acting on differentiable functions h : R d → R d .

̸

The Langevin Stein operator of Gorham and Mackey [2015] is recovered when p = q , but when q = p the dependence on the derivatives of p is removed. The operator S p,q can still be recognised as a diffusion Stein operator, being related to the infinitesimal generator of a diffusion process that leaves p invariant; however, it falls outside the scope of the theoretical analysis of Huggins and Mackey [2018], Gorham et al. [2019], for reasons explained in Remark 2. It can also be viewed as a non-standard instance of the density method of Diaconis et al. [2004]; see Section 2 of Anastasiou et al. [2023]. The inclusion of q introduces an additional degree of freedom, specific choices for which are discussed in Section 3.

Remark 1. The ratio q/p in Definition 1 could be viewed as an importance weight, but the construction does not fully correspond to importance sampling due to the ∇ log q term, which is q -dependent.

The Stein operator nomenclature derives from the vanishing integral property in Proposition 1 below, which is central to Stein's method [Stein, 1972]:

Proposition 1. In the setting of Definition 1, assume that ∥ x ∥ d -1 q ( x ) → 0 as ∥ x ∥ → ∞ and ∫ ∥∇ log q ∥ d q < ∞ . Then, for any function h : R d → R d whose first derivatives exist and are bounded, it holds that ∫ S p,q h d p = 0 .

All proofs are contained in Appendix A. From Proposition 1, the expectation of S p,q h with respect to π ∈ P ( R d ) will be zero when π and p are equal; conversely, the value of such an expectation can be used to quantify the extent to which π and p are different. Consideration of multiple test functions increases the number and nature of the differences between π and p that may be detected. A discrepancy is obtained by specifying which test functions h are considered, and then taking a supremum over the expectations associated to this set. For computational convenience we take h to be contained in the unit ball of a reproducing kernel Hilbert space, as described next.

For a symmetric positive definite function k : R d × R d → R , called a kernel , denote the associated reproducing kernel Hilbert space as H ( k ) . Let H ( k ) d denote the Cartesian product of d copies of H ( k ) , equipped with the inner product ⟨ h, g ⟩ H ( k ) d := ∑ d i =1 ⟨ h i , g i ⟩ H ( k ) .

Proposition 2. Let π ∈ P ( R d ) . In the setting of Definition 1, assume there is an α > 1 such that ∫ ( q/p ) α d π < ∞ and ∫ ∥∇ log q ∥ α/ ( α -1) d π < ∞ . Let k : R d × R d → R be a continuously differentiable kernel such that both k and its first derivatives x ↦→ ( ∂ i ⊗ ∂ i ) k ( x, x ) , i = 1 , . . . , d , are bounded. Then S p,q is a bounded linear operator from H ( k ) d to L 1 ( π ) .

For discrete distributions π , supported on a finite subset of R d , the moment conditions in Proposition 2 are automatically satisfied. For general distributions π on R d , the exponent α can be taken arbitrarily close to 1 to enable the more stringent moment condition ∫ ( q/p ) α d π < ∞ to hold. An immediate consequence of Proposition 2 is that Definition 2 below is well-defined.

Definition 2 (Gradient-Free Kernel Stein Discrepancy) . For p , q , k and π satisfying the preconditions of Proposition 2, the gradient-free KSD is defined as

The GF-KSD coincides with the canonical kernel Stein discrepancy (KSD) when p = q , and is thus strictly more general. Note that D p,q ( π ) is precisely the operator norm of the linear functional h ↦→ ∫ S p,q h d π , which exists due to Proposition 2. Most common kernels satisfy the assumptions of Proposition 2, and a particularly important example is the inverse multi-quadric kernel

which has bounded derivatives of all orders; see Lemma 4 of Fisher et al. [2021].

The use of reproducing kernels ensures that GF-KSD can be explicitly computed; see Proposition 5 and Corollary 1 in Appendix A. A general spectral characterisation of GF-KSD is provided in Proposition 6 of the supplement, inspired by the recent work of Wynne et al. [2022]. Note that the scale of Equation (6) does not matter when one is interested in the relative performance of different π ∈ P ( R d ) as approximations of a fixed target p ∈ P ( R d ) . In this sense, GF-KSD may be employed with ˜ p in place of p , where p ∝ ˜ p/Z and Z is a normalisation constant. This feature makes GF-KSD applicable to problems of posterior approximation, and will be exploited for both sampling and variational inference in Section 4. On the other hand, GF-KSD is not applicable to problems in which the target distribution p θ involves a parameter θ , such as estimation and composite hypothesis testing, since then the normalisation term Z θ cannot be treated as constant.

Going beyond the discrepancies discussed in Section 1, several non-canonical discrepancies have recently been proposed based on Stein's method, for example sliced [Gong et al., 2021a,b], stochastic [Gorham et al., 2020], and conditional [Singhal et al., 2019] Stein discrepancies (in all cases derivatives of p are required). However, only a subset of these discrepancies have been shown to enjoy important guarantees of convergence detection and control. Convergence control is critical for the posterior approximation task considered in this work, since this guarantees that minimisation of Stein discrepancy will produce a consistent approximation of the posterior target. The aim of the next section is to establish such guarantees for GF-KSD.

2.2 Convergence Detection and Control

̸

The canonical KSD benefits from theoretical guarantees of convergence detection and control [Gorham and Mackey, 2017]. The aim of this section is to establish analogous guarantees for GF-KSD. To set the scene, for a Lipschitz function f : R d → R d , denote its Lipschitz constant L ( f ) := sup x = y ∥ f ( x ) -f ( y ) ∥ / ∥ x -y ∥ . Then, for measurable g : R d → R , we denote the tilted Wasserstein distance as W 1 ( π, p ; g ) := sup L ( f ) ≤ 1 ∣ ∣ ∫ fg d π -∫ fg d p ∣ ∣ whenever this expression is well-defined [Huggins and Mackey, 2018]. Note that the standard 1-Wasserstein distance W 1 ( π, p ) is recovered when g = 1 . There is no dominance relation between W 1 ( · , · ; g ) for different g ; the topologies they induce are different. Tilted Wasserstein distance induces a much weaker topology than, for example, divergences such as Kullback-Leibler or Hellinger, since it does not require absolute continuity of measures.

Theorem 1 (Convergence Detection) . Let p, q ∈ P ( R d ) with q ≪ p , ∇ log q Lipschitz and ∫ ∥∇ log q ∥ 2 d q < ∞ . Assume there is an α > 1 such that the sequence ( π n ) n ∈ N ⊂ P ( R d ) satisfies ∫ ( q/p ) α d π n ∈ (0 , ∞ ) , ∫ ∥∇ log q ∥ α/ ( α -1) d π n < ∞ , ∫ ∥∇ log q ∥ α/ ( α -1) ( q/p ) d π n < ∞ , and ∫ fq/p d π n < ∞ with f ( x ) = ∥ x ∥ , for each n ∈ N . Let k be a kernel such that each of k , ( ∂ i ⊗ ∂ i ) k and ( ∂ i ∂ j ⊗ ∂ i ∂ j ) k exist, are continuous, and are bounded, for i, j ∈ { 1 , . . . , d } . Then W 1 ( π n , p ; q/p ) → 0 implies D p,q ( π n ) → 0 .

Thus the convergence of π n to p , in the sense of the tilted Wasserstein distance with g = q/p , is detected by the GF-KSD D p,q . Note that the conditions on ( π n ) n ∈ N in Theorem 1 are automatically satisfied when each π n has a finite support.

Despite being a natural generalisation of KSD, this GF-KSD does not in general provide weak convergence control in the equivalent theoretical context. Indeed, Gorham and Mackey [2017] established positive results on convergence control for distributions in Q ( R d ) , the set of probability distributions on R d with positive density function q : R d → (0 , ∞ ) for which ∇ log q is Lipschitz and q is distantly dissipative , meaning that

Under the equivalent assumptions, convergence control fails for GF-KSD in general:

Proposition 3 (Convergence Control Fails in General) . Let k be a (non-identically zero) radial kernel of the form k ( x, y ) = ϕ ( x -y ) for some twice continuously differentiable ϕ : R d → R , for which the preconditions of Proposition 2 are satisfied. Then there exist p ∈ P ( R d ) , q ∈ Q ( R d ) and a sequence ( π n ) n ∈ N ⊂ P ( R d ) , also satisfying the preconditions of Proposition 2, such that D p,q ( π n ) → 0 and yet π n d ↛ p .

The purpose of Proposition 3 is to highlight that GF-KSD is not a trivial extension of canonical KSD; it requires a bespoke treatment. This is provided in Theorem 2, next. Indeed, to ensure that GF-KSD provides convergence control, additional condition on the tails of q are required:

Theorem 2 (Convergence Control) . Let p ∈ P ( R d ) , q ∈ Q ( R d ) be such that p is continuous and inf x ∈ R d q ( x ) /p ( x ) > 0 . Assume there is an α > 1 such that the sequence ( π n ) n ∈ N ⊂ P ( R d ) satisfies ∫ ( q/p ) α d π n < ∞ and ∫ ∥∇ log q ∥ α/ ( α -1) ( q/p ) d π n < ∞ , for each n ∈ N . Let k be the inverse multi-quadric kernel in Equation (2) . Then D p,q ( π n ) → 0 implies π n d → p .

The proof of Theorem 2 is based on carefully re-casting GF-KSD between π and p as a canonical KSD between q and a transformed distribution ¯ π (see Proposition 7 in the supplement), then appealing to the analysis of Gorham and Mackey [2017]. Compared to the analysis of KSD in Gorham and Mackey [2017], the distant dissipativity condition now appears on q (a degree of freedom), rather than on p (a distribution determined by the task at hand), offering a realistic opportunity for this condition to be verified. For example, one could take a Gaussian measure q that dominates the target distribution p for use in GF-KSD. Nevertheless, the conditions of Theorem 2 rule out distributions p that are heavy-tailed. Suitable choices for q are considered in Section 3.

Remark2 (Related Work) . Convergence control was established for discrepancies based on general classes of Stein operator in earlier work, but the required assumptions are too stringent when applied in our context. In particular, to use Huggins and Mackey [2018] it is required that the gradient

Inward Convergence

Oblique Convergence

Oblique Non-Convergence

(a)

8

6

4

2

0

2

4

6

8

q

=Prior

q

=Laplace

0

20

40

60

80

100

n

8

6

4

2

0

2

4

6

8

q

=GMM

0

20

40

60

80

100

n

q

=KDE

p

q

p

q

p

q

p

q

(b)

∇ log( q/p ) is bounded 1 , while to use Gorham et al. [2019] it is required that q/p is Lipschitz 2 . In our context, where q must be specified in ignorance of p , such conditions, which require that q is almost as light as p in the tail, cannot be guaranteed to hold. The present paper therefore instead contributes novel analysis for the regime where q may be appreciably heavier than p in the tail.

This completes our theoretical assessment, but the practical performance of GF-KSD remains to be assessed. Suitable choices for both q and the kernel parameters σ and β are proposed and investigated in Section 3, and practical demonstrations of GF-KSD are presented in Section 4.

3 Implementation Detail

The purpose of this section is to empirically explore the effect of varying q , σ and β , aiming to arrive at reasonable default settings. In the absence of an application-specific optimality criterion, we aim to select values that perform well (in a sense to be specified) over a range of scenarios that may be encountered. Here, to assess performance several sequences ( π n ) n ∈ N are considered, some of which converge to a specified limit p and the rest of which converge to an alternative Gaussian target; see Figure 1(a). An effective discrepancy should clearly indicate which of these sequences are convergent and which are not. On this basis, recommendations for q are considered in Section 3.1, and recommendations for σ and b in Section 3.2. Of course, we cannot expect default settings to perform universally well, so in Section 3.3 we highlight scenarios where our defaults may fail. Python code to reproduce the experiments reported below can be downloaded at [blinded] .

3.1 Choice of q

In what follows we cast q as an approximation of p , aiming to inherit the desirable performance of canonical KSD for which q and p are equal. The task to which the discrepancy is being applied will,

in practice, constrain the nature and form of the distributions q that can be implemented. For expository purposes (only), the following qualitatively distinct approaches to choosing q are considered:

Prior In Bayesian settings where p is a posterior distribution, selecting q to be the prior distribution ensures that the condition inf x ∈ R d q ( x ) /p ( x ) > 0 is satisfied.

Laplace If the target p can be differentiated, albeit at a possibly high computational cost, it may be practical to construct a Laplace approximation q to the target [Gelman et al., 2013].

GMM One could take q to be a Gaussian mixture model fitted to approximate samples from p , representing a more flexible alternative to Laplace .

KDE Additional flexibility can be obtained by employing a kernel density estimator as a nonparametric alternative to GMM .

Of course, there is a circularity to GMM and KDE which renders these methods impractical in general. The specific details of how each of the q were constructed are contained in the code that accompanies this paper, but the resulting q are displayed as insets in Figure 1(b). The performance of GF-KSD with these different choices of q is also displayed in Figure 1(b). It was observed that all four choices of q produced a discrepancy that could detect convergence of π n to p , though the detection of convergence was less clear for Prior due to slower convergence of the discrepancy to 0 as n was increased. On the other hand, all approaches were able to clearly detect non-convergence to the target. That Laplace performed comparably with GMM and KDE was surprising, given that the target p is not well-approximated by a single Gaussian component. These results are for a specific choice of target p , but in Appendix C.5.1 a range of p are considered and similar conclusions are obtained. Section 3.3 explores how challenging p must be before the convergence detection and control properties associated to Laplace fail.

3.2 Choice of σ and β

For the investigation in Section 3.1 the parameters of the inverse multi-quadric kernel (2) were fixed to σ = 1 and β = 1 / 2 , the latter being the midpoint of the permitted range β ∈ (0 , 1) . In general, care in the selection of these parameters may be required. The parameter σ captures the scale of the data, and thus standardisation of the data may be employed to arrive at σ = 1 as a natural default. In this paper (with the exception of Section 4.2) the standardisation x ↦→ C -1 x was performed, where C is the covariance matrix of the approximating distribution q being used. In Appendix C.5.2 we reproduce the investigation of Section 3.1 using a range of values for σ and β ; these results indicate the performance of GF-KSD is remarkably insensitive to perturbations around ( σ, β ) = (1 , 1 / 2) .

3.3 Avoidance of Failure Modes

GF-KSD is not a silver bullet, and there are a number of specific failure modes that care may be required to avoid. The four main failure modes are illustrated in Figure 2. These are as follows: (a) q is substantially heavier than p in a tail; (b) q is substantially lighter than p in a tail; (c) the dimension d is too high; (d) p has well-separated high-probability regions. Under both (a) and (c), convergence detection can fail, either because theoretical conditions are violated or because the terms π n must be extremely close to p before convergence begins to be detected. Under (b), the values of GF-KSD at small n can mislead. Point (d) is a well-known pathology of all score-based methods; see Wenliang and Kanagawa [2021], Liu et al. [2023]. These four failure modes inform our recommended usage of GF-KSD, summarised next.

Summary of Recommendations Based on the investigation just reported, the default settings we recommend are Laplace with σ = 1 (post-standardisation) and β = 1 / 2 . Although not universally applicable, Laplace does not require samples from p , and has no settings that must be user-specified. Thus we recommend the use of Laplace in situations where a Laplace approximation can be justified and computed. If Laplace not applicable, then one may attempt to construct an approximation q using techniques available for the task at hand (e.g. in a Bayesian setting, one may obtain q via variational inference, or via inference based on an approximate likelihood). These recommended settings will be used for the application presented in Section 4.1, next.

q

0

20

40

60

80

100

n

2

0

2

4

6

8

log GFKSD

p

q

(a)

0

25

50

75

100

n

100

50

0

50

100

log GFKSD

p

q

(b)

0

20

40

60

80

100

n

12

10

8

6

4

2

0

log GFKSD

d

1

d

=1

d

=10

d

=100

(c)

0

20

40

60

80

100

n

5

0

5

10

15

log GFKSD

p

q

(d)

p

q

p

q

=

p

4 Applications

To demonstrate potential uses of GF-KSD, two applications to posterior approximation are now presented; Stein importance sampling (Section 4.1) and Stein variational inference using measure transport (Section 4.2). In each case, we extend the applicability of existing algorithms to statistical models for which certain derivatives of p are either expensive or non-existent.

4.1 Gradient-Free Stein Importance Sampling

Stein importance sampling [Liu and Lee, 2017, Hodgkinson et al., 2020] operates by first sampling independently from a tractable approximation of the target p and then correcting the bias in the samples so-obtained. To date, applications of Stein importance sampling have been limited to instances where the statistical model p can be differentiated; our contribution is to remove this requirement. In what follows we analyse Stein importance sampling in which independent samples ( x n ) n ∈ N are generated from the same approximating distribution q that is employed within GF-KSD:

Theorem 3 (Gradient-Free Stein Importance Sampling) . Let p ∈ P ( R d ) , q ∈ Q ( R d ) be such that p is continuous and inf x ∈ R d q ( x ) /p ( x ) > 0 . Suppose that ∫ exp { γ ∥∇ log q ∥ 2 } d q < ∞ for some γ > 0 . Let k be the inverse multi-quadric kernel in Equation (2) . Let ( x n ) n ∈ N be independent samples from q . To the sample, assign optimal weights

The proof builds on earlier work in Riabiz et al. [2022]. Note that the optimal weights w ∗ can be computed without the normalisation constant of p , by solving a constrained quadratic programme at cost O ( n 3 ) .

As an illustration, we implemented gradient-free Stein importance sampling to approximate a posterior arising from a discretely observed Lotka-Volterra model

with independent log-normal observations with covariance matrix diag ( σ 2 1 , σ 2 2 ) . The parameters to be inferred are { α, β, γ, δ, u 0 , v 0 , σ 1 , σ 2 } and therefore d = 8 . The data analysed are due to Hewitt [1921], and full details are contained in Appendix C.3. The direct application of Stein importance sampling to this task requires the numerical calculation of sensitivities of the differential equation at each of the n samples that are to be re-weighted. Aside from simple cases where automatic differentiation or adjoint methods can be used, the stable computation of sensitivities can form a major computational bottleneck; see [Riabiz et al., 2022]. In contrast, our approach required a

(a)

(b)

β

8

uo

Vo

01

02

8

3

13

6

.

8:

5:

0.

̸

fixed number of gradient computations to construct a Laplace approximation 3 , independent of the number n of samples required; see Appendix C.3 for detail. In the lower triangular portion of Figure 3(a), biased samples from the Laplace approximation q ( = p ) are displayed, while in the upper triangular portion the same samples are re-weighted using gradient-free Stein importance sampling to form a consistent approximation of p . A visual reduction in bias and improvement in approximation quality can be observed. As a baseline against which to assess the quality of our approximation, we consider self-normalised importance sampling; i.e. the approximation with weights ¯ w such that ¯ w i ∝ p ( x i ) /q ( x i ) . Figure 3(b) reports the accuracy of the approximations to p as quantified using energy distance [Cram´ er, 1928]. These results indicate that the approximations produced using gradient-free Stein importance sampling improve on those constructed using selfnormalised importance sampling. This may be explained by the fact that the optimal weights w ∗ attempt to mitigate both bias due to q = p and Monte Carlo error, while the weights ¯ w only address the bias due to q = p , and do not attempt to mitigate error due to the randomness in Monte Carlo. Additional experiments in Appendix C.5.3 confirm that gradient-free Stein importance sampling achieves comparable performance with gradient-based Stein importance sampling in regimes where the Laplace approximation can be justified.

̸

Although our recommended default settings for GF-KSD were successful in this example, an interesting theoretical question would be to characterise an optimal choice of q in this context. This appears to be a challenging problem but we hope to address it in future work. In addition, although we focused on Stein importance sampling, our methodology offers the possibility to construct gradientfree versions of other related algorithms, including the Stein points algorithms of Chen et al. [2018, 2019], and the Stein thinning algorithm of Riabiz et al. [2022].

4.2 Stein Variational Inference Without Second-Order Gradient

Stein discrepancy was proposed as a variational objective in Ranganath et al. [2016] and has demonstrated comparable performance with the traditional Kullback-Leibler objective in certain application areas, whilst abolishing the requirement that the variational family is absolutely continuous with

̸

GF-KSDOiterations

GF-KSD2000iterations

GF-KSD10,000iterations

GF-KSD20,000iterations

KSD20,000iterations

respect to the statistical model [Fisher et al., 2021]. This offers an exciting, as yet largely unexplored opportunity to construct flexible variational families outside the conventional setting of normalising flows (which are constrained to be diffeomorphisms of R d ). However, gradient-based stochastic optimisation of the canonical Stein discrepancy objective function means that second-order derivatives of the statistical model are required. GF-KSD reduces the order of derivatives that are required from second-order to first-order, and in many cases (such as when differential equations appear in the statistical model) this will correspond to a considerable reduction in computational cost.

In what follows we fix a reference distribution R ∈ P ( R d ) and a parametric class of maps T θ : R d → R d , θ ∈ R p , and consider the variational family ( π θ ) θ ∈ R p ⊂ P ( R d ) whose elements π θ := T θ # R are the pushforwards of R through the maps T θ , θ ∈ R p . The aim is to use stochastic optimisation to minimise θ ↦→ D p,q ( π θ ) (or, equivalently, any strictly increasing transformation thereof). For this purpose a low-cost unbiased estimate of the gradient of the objective function is required; the details and sufficient theoretical conditions are contained in Appendix B.

As an interesting methodological extension, that departs from the usual setting of stochastic optimisation, here we consider interlacing stochastic optimisation over θ and the selection of q , leveraging the current value θ m on the optimisation path to provide a natural candidate π θ m for q in this context 4 . Thus, for example, a vanilla stochastic gradient descent routine becomes θ m +1 = θ m -ϵ ∇ θ D p,π θm ( π θ ) 2 ∣ ∣ θ = θ m for some learning rate ϵ > 0 . (In this iterative setting, to ensure the variational objective remains fixed, we do not perform the standardisation of the data described in Section 3.2.)

To assess the performance of GF-KSD in this context, we re-instantiated an experiment from Fisher et al. [2021]. The results, in Figure 4, concern the approximation of 'banana' and 'sinusoidal' distributions in dimension d = 2 , and were obtained using the reference distribution R = N (0 , 2 I ) and taking T θ to be the inverse autoregressive flow of Kingma et al. [2016]. These are both toy problems, which do not themselves motivate our methodological development, but do enable us to have an explicit ground truth to benchmark performance against. Full experimental detail is contained in Appendix C.4; we highlight that gradient clipping was used, both to avoid extreme values of q/p encountered on the optimisation path, and to accelerate the optimisation itself [Zhang et al., 2019]. The rightmost panel depicts the result of performing variational inference with the standard KSD objective functional. It is interesting to observe that GF-KSD leads to a similar performance in both examples, with the caveat that stochastic optimisation was more prone to occasional failure when GF-KSD was used. The development of a robust optimisation technique in this context requires care and a detailed empirical assessment, and is left as a promising avenue for further research.

5 Conclusion

In this paper GF-KSD was proposed and studied. Theoretical and empirical results support the use of GF-KSD in settings where an initial approximation to the distributional target can readily be constructed, and where the distributional target itself does not contain distant high probability regions, but poor performance can occur outside this context. Nevertheless, for many statistical analyses the principal challenge is the cost of evaluating the statistical model and its derivatives, rather than the complexity of the target itself, and in these settings the proposed discrepancy has the potential to be usefully employed. The focus of this work was on posterior approximation, with illustrative applications to sampling and variational inference being presented. A natural extension to this work would involve a systematic empirical assessment of the performance of GF-KSD across a broad range of applied contexts. However, we note that GF-KSD is not applicable to problems such as estimation and composite hypothesis testing, where the target p θ ranges over a parametric model class, and alternative strategies will be required to circumvent gradient computation in that context.

Acknowledgements MAF was supported by EP/W522387/1. CJO was supported by EP/W019590/1. The authors are grateful to Franc ¸ois-Xavier Briol, Jon Cockayne, Jeremias Knoblauch, Lester Mackey, Marina Riabiz, Rob Salomone, Leah South, and George Wynne for insightful comments on an early draft of the manuscript, as well as to the Reviewers at NeurIPS.

References

Anastasiou, A. Barp, F.-X. Briol, B. Ebner, R. E. Gaunt, F. Ghaderinezhad, J. Gorham, A. Gretton, C. Ley, Q. Liu, L. Mackey, C. J. Oates, G. Reinert, and Y. Swan. Stein's method meets statistics: A review of some recent developments. Statistical Science , 38(1):120-139, 2023.

Barp, F.-X. Briol, A. Duncan, M. Girolami, and L. Mackey. Minimum Stein discrepancy estimators. In Proceedings of the 33rd Conference on Neural Information Processing Systems , volume 32, pages 12964-12976, 2019.

Y. Chen, L. Mackey, J. Gorham, F.-X. Briol, and C. J. Oates. Stein points. In Proceedings of the 35th International Conference on Machine Learning , pages 844-853. PMLR, 2018.

Y. Chen, A. Barp, F.-X. Briol, J. Gorham, M. Girolami, L. Mackey, and C. J. Oates. Stein point Markov chain Monte Carlo. In Proceedings of the 36th International Conference on Machine Learning , pages 1011-1021. PMLR, 2019.

Chopin and G. Ducrocq. Fast compression of MCMC output. Entropy , 23(8):1017, 2021.

Chwialkowski, H. Strathmann, and A. Gretton. A kernel test of goodness of fit. In Proceedings of the 33rd International Conference on Machine Learning , pages 2606-2615. PMLR, 2016.

Cockayne and A. Duncan. Probabilistic gradients for fast calibration of differential equation models. SIAM/ASA Journal on Uncertainty Quantification , 9(4):1643-1672, 2021.

Cram´ er. On the composition of elementary errors. Scandinavian Actuarial Journal , (1):141-180, 1928.

Diaconis, C. Stein, S. Holmes, and G. Reinert. Use of exchangeable pairs in the analysis of simulations. In Stein's Method , pages 1-25. Institute of Mathematical Statistics, 2004.

Fernandez, N. Rivera, W. Xu, and A. Gretton. Kernelized Stein discrepancy tests of goodnessof-fit for time-to-event data. In Proceedings of the 37th International Conference on Machine Learning , pages 3112-3122. PMLR, 2020.

Fisher, T. Nolan, M. Graham, D. Prangle, and C. J. Oates. Measure transport with kernel Stein discrepancy. In Proceedings of the 24th International Conference on Artificial Intelligence and Statistics , pages 1054-1062. PMLR, 2021. (Here we refer to the error-corrected version arXiv:2010.11779.).

Gelman, J. B. Carlin, H. S. Stern, D. B. Dunson, A. Vehtari, and D. B. Rubin. Bayesian Data Analysis . CRC Press, 2013.

Gong, Y. Li, and J. M. Hern´ andez-Lobato. Sliced kernelized Stein discrepancy. In Proceedings of the 9th International Conference on Learning Representations , 2021a.

Gong, K. Zhang, Y. Li, and J. M. Hern´ andez-Lobato. Active slices for sliced Stein discrepancy. In Proceedings of the 38th International Conference on Machine Learning , pages 3766-3776, 2021b.

Gorham and L. Mackey. Measuring sample quality with Stein's method. In Proceedings of the 29th Conference on Neural Information Processing Systems , pages 226-234, 2015.

Gorham and L. Mackey. Measuring sample quality with kernels. In Proceedings of the 34th International Conference on Machine Learning , pages 1292-1301. PMLR, 2017.

Gorham, A. B. Duncan, S. J. Vollmer, and L. Mackey. Measuring sample quality with diffusions. The Annals of Applied Probability , 29(5):2884-2928, 2019.

Gorham, A. Raj, and L. Mackey. Stochastic Stein discrepancies. In Proceedings of the 34th Conference on Neural Information Processing Systems , 2020.

Grathwohl, K.-C. Wang, J.-H. Jacobsen, D. Duvenaud, and R. Zemel. Learning the Stein discrepancy for training and evaluating energy-based models without sampling. In Proceedings of the 37th International Conference on Machine Learning , pages 3732-3747. PMLR, 2020.

Han and Q. Liu. Stein variational gradient descent without gradient. In Proceedings of the 35th International Conference on Machine Learning , pages 1900-1908. PMLR, 2018.

G. Hewitt. The Conservation of the Wild Life of Canada . Charles Scribner's Sons, 1921.

Hodgkinson, R. Salomone, and F. Roosta. The reproducing Stein kernel approach for post-hoc corrected sampling. arXiv:2001.09266 , 2020.

H. Huggins and L. Mackey. Random feature Stein discrepancies. In Proceedings of the 32nd International Conference on Neural Information Processing Systems , pages 1903-1913, 2018.

Jitkrittum, W. Xu, Z. Szab´ o, K. Fukumizu, and A. Gretton. A linear-time kernel goodness-of-fit test. In Proceedings of the 31st Conference on Neural Information Processing Systems . NIPS Foundation, 2017.

P. Kingma and J. Ba. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations , 2015.

P. Kingma, T. Salimans, R. Jozefowicz, X. Chen, I. Sutskever, and M. Welling. Improved variational inference with inverse autoregressive flow. In Proceedings of the 30th Conference on Neural Information Processing Systems , 2016.

C. Liu and J. Nocedal. On the limited memory BFGS method for large scale optimization. Mathematical Programming , 45(3):503-528, 1989.

Liu and J. Lee. Black-box importance sampling. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics , pages 952-961. PMLR, 2017.

Liu and D. Wang. Stein variational gradient descent: A general purpose Bayesian inference algorithm. In Proceedings of the 30th Conference on Neural Information Processing Systems , 2016.

Liu, J. Lee, and M. Jordan. A kernelized Stein discrepancy for goodness-of-fit tests. In Proceedings of the 33rd International Conference on Machine Learning , pages 276-284. PMLR, 2016.

Liu, A. Duncan, and A. Gandy. Using perturbation to improve goodness-of-fit tests based on kernelized Stein discrepancy. In Proceedings of the 40th International Conference on Machine Learning , 2023.

Matsubara, J. Knoblauch, F.-X. Briol, and C. J. Oates. Robust generalised Bayesian inference for intractable likelihoods. Journal of the Royal Statistical Society, Series B , 84(3):997-1022, 2022.

O'Donoghue, E. Chu, N. Parikh, and S. Boyd. Conic optimization via operator splitting and homogeneous self-dual embedding. Journal of Optimization Theory and Applications , 169(3): 1042-1068, June 2016.

Park and G. Casella. The Bayesian lasso. Journal of the American Statistical Association , 103 (482):681-686, 2008.

Prangle and C. Viscardi. Distilling importance sampling for likelihood free inference. Journal of Computational and Graphical Statistics , 2023. To appear.

Ranganath, D. Tran, J. Altosaar, and D. Blei. Operator variational inference. In Proceedings of the 30th Conference on Neural Information Processing Systems , pages 496-504, 2016.

Riabiz, W. Chen, J. Cockayne, P. Swietach, S. A. Niederer, L. Mackey, and C. J. Oates. Optimal thinning of MCMC output. Journal of the Royal Statistical Society, Series B , 84(4):1059-1081, 2022.

Roˇ ckov´ a and E. I. George. The Spike-and-Slab LASSO. Journal of the American Statistical Association , 113(521):431-444, 2018.

Schwarz. Estimating the Dimension of a Model. The Annals of Statistics , 6(2):461 - 464, 1978.

W. Silverman. Density Estimation for Statistics and Data Analysis . Chapman & Hall, 1986.

Singhal, X. Han, S. Lahlou, and R. Ranganath. Kernelized complete conditional Stein discrepancy. arXiv:1904.04478 , 2019.

Stan Development Team. RStan: the R interface to Stan, 2022. URL https://mc-stan.org/ . R package version 2.21.5.

Stein. A bound for the error in the normal approximation to the distribution of a sum of dependent random variables. In Proceedings of the 6th Berkeley Symposium on Mathematical Statistics and Probability , volume 6.2, pages 583-603. University of California Press, 1972.

Steinwart and A. Christmann. Support Vector Machines . Springer Science & Business Media, 2008.

K. Wenliang and H. Kanagawa. Blindness of score-based methods to isolated components and mixing proportions. In Proceedings of the NeurIPS workshop 'Your Model is Wrong' , 2021.

Wynne, M. Kasprzak, and A. B. Duncan. A spectral representation of kernel Stein discrepancy with application to goodness-of-fit tests for measures on infinite dimensional Hilbert spaces. arXiv:2206.04552 , 2022.

Zhang, T. He, S. Sra, and A. Jadbabaie. Why gradient clipping accelerates training: A theoretical justification for adaptivity. In International Conference on Learning Representations , 2019.

Appendices

These appendices supplement the paper Gradient Free Kernel Stein Discrepancy . The proofs for theoretical results stated in the main text are contained in Appendix A. Theoretical analysis of stochastic gradient estimators for use in the context of Stein Variational Inference is contained in Appendix B, as advertised in Section 4.2 of the main text. All additional details related to empirical assessment are contained in Appendix C.

A Proof of Results in the Main Text

This appendix contains proofs for all novel theoretical results reported in the main text.

Proof of Proposition 1. First we show that the integral in the statement is well-defined. Since h and its first derivatives are bounded, we can define the constants C 0 := sup {∥ h ( x ) ∥ : x ∈ R d } and C 1 := sup {|∇ · h ( x ) | : x ∈ R d } . Then

so indeed the integral is well-defined. Now, let B r = { x ∈ R d : ∥ x ∥ ≤ r } , S r = { x ∈ R d : ∥ x ∥ = r } , and t ( r ) := sup { q ( x ) : x ∈ S r } , so that by assumption r d -1 t ( r ) → 0 as r →∞ . Let ✶ B ( x ) = 1 if x ∈ B and 0 if x / ∈ B . Then ∫ S p,q h d p = lim r →∞ ∫ ✶ B r S p,q h d p and, from the divergence theorem,

as required.

Proof of Proposition 2. Since k and its first derivatives are bounded, we can set

From Cauchy-Schwarz and the reproducing property, for any f ∈ H ( k ) it holds that | f ( x ) | = |⟨ f, k ( · , x ) ⟩ H ( k ) | ≤ ∥ f ∥ H ( k ) ∥ k ( · , x ) ∥ H ( k ) = ∥ f ∥ H ( k ) √ ⟨ k ( · , x ) , k ( · , x ) ⟩ H ( k ) = ∥ f ∥ H ( k ) √ k ( x, x ) . Furthermore, using the fact that k is continuously differentiable, it can be shown that | ∂ i f ( x ) | ≤ ∥ f ∥ H ( k ) √ ( ∂ i ⊗ ∂ i ) k ( x, x ) ; see Corollary 4.36 of Steinwart and Christmann [2008]. As a consequence, for all h ∈ H ( k ) d we have that, for all x ∈ R d ,

To use analogous notation as in the proof of Proposition 1, set C 0 := C k 0 ∥ h ∥ H ( k ) d and C 1 := C k 1 ∥ h ∥ H ( k ) d . Then, using H¨ older's inequality and the fact that ( a + b ) β ≤ 2 β -1 ( a β + b β ) with β = α/ ( α -1) , we have

as required.

To prove obtain explicit computable formulae for the GF-KSD, two intermediate results are required: Proposition 4. Let k and π satisfy the preconditions of Proposition 2. Then the function

takes values in H ( k ) d , is Bochner π -integrable and, thus, ξ := ∫ f d π ∈ H ( k ) d .

Proof. Since k has continuous first derivatives x ↦→ ( ∂ i ⊗ ∂ i ) k ( x, x ) , Lemma 4.34 of Steinwart and Christmann [2008] gives that ( ∂ i ⊗ 1) k ( x, · ) ∈ H ( k ) , and, thus f ∈ H ( k ) d . Furthermore, f : R d → H ( k ) d is Bochner π -integrable since

where we have employed the same C k 0 and C k 1 notation as used in the proof of Proposition 2. Thus, from the definition of the Bochner integral, ξ = ∫ f d π exists and is an element of H ( k ) d .

Proposition 5. Let k and π satisfy the preconditions of Proposition 2. Then

where

Proof. Let f be as in Equation (3). From Proposition 4, ξ = ∫ f d π ∈ H ( k ) d . Moreover, since f is Bochner π -integrable and Tf = ⟨ h, f ⟩ H ( k ) d is a continuous linear functional on H ( k ) d , from basic properties of Bochner integrals we have Tξ = T ∫ f d π = ∫ Tf d π . In particular,

which shows that ξ is the Riesz representer of the bounded linear functional h ↦→ ∫ S p,q h d π on H ( k ) d . It follows from Cauchy-Schwarz that the (squared) operator norm of this functional is

as claimed.

For concreteness, we instantiate Proposition 5 in the specific case of the inverse multi-quadric kernel, since this is the kernel that we recommend in Section 2.2:

Corollary 1 (Explicit Form) . For p , q , and π satisfying the preconditions of Proposition 2, and k the inverse multi-quadric kernel in Equation (2) , we have that

Proof of Corollary 1. First we compute derivatives of the kernel k in Equation (2):

Letting u ( x ) := ∇ log q ( x ) form a convenient shorthand, we have that

which, combined with Proposition 5, gives the result.

In addition to the results in the main text, here we present a spectral characterisation of GF-KSD. The following result was inspired by an impressive recent contribution to the literature on kernel Stein discrepancy due to Wynne et al. [2022], and our (informal) proof is based on an essentially identical argument:

Proposition 6 (Spectral Characterisation) . Consider a positive definite isotropic kernel k , and recall that Bochner's theorem guarantees k ( x, y ) = ∫ e -i ⟨ s,x -y ⟩ d µ ( s ) for some µ ∈ P ( R d ) . Then, under regularity conditions that we leave implicit,

The Fourier transform ̂ ∇ q of ∇ q is defined as ∫ e -i ⟨ s,x ⟩ ∇ q ( x ) d x , and a basic property of the Fourier transform is that the transform of a derivative can be computed using the expression is ∫ e -i ⟨ s,x ⟩ q ( x ) d x . This implies that the inner integral in (7) vanishes when π and p are equal. Thus we can interpret GF-KSD as a quantification of the uniformity of d π/ d p , with a weighting function based on the Fourier derivative identity with regard to ∇ q .

Proof of Proposition 6. From direct calculation, and assuming derivatives and integrals can be interchanged, we have that

Now, let

and note through direct calculation and Equations (8) to (10) that

Thus, integrating with respect to π , and assuming that we may interchange the order of integrals, we have that

where the final equality is Proposition 5. This establishes the result.

To prove Theorem 1, two intermediate results are required:

Proposition 7. For an element π ∈ P ( R d ) , assume Z := ∫ ( q/p ) d π ∈ (0 , ∞ ) . Assume that k and π satisfy the preconditions of Proposition 2, and that ∫ ∥∇ log q ∥ α/ ( α -1) ( q/p ) d π < ∞ . Let ¯ π := ( qπ ) / ( pZ ) . Then ¯ π ∈ P ( R d ) and

Proof. The assumption Z ∈ (0 , ∞ ) implies that ¯ π ∈ P ( R d ) . Furthermore, the assumption ∫ ∥∇ log q ∥ α/ ( α -1) ( q/p ) d π < ∞ implies that ∫ ∥∇ log q ∥ α/ ( α -1) d¯ π < ∞ . Thus the assumptions of Proposition 2 are satisfied for both π and ¯ π , and thus both D p,q ( π ) and D q,q (¯ π ) are welldefined. Now, with ξ as in Proposition 4, notice that

as claimed.

Proposition 8. Let f : R d → [0 , ∞ ) and π ∈ P ( R d ) . Then ∫ f α d π > 0 ⇒ ∫ f d π > 0 , for all α ∈ (0 , ∞ ) .

Proof. From the definition of the Lebesgue integral, we have that ∫ f α d π = sup { ∫ s d π : s a simple function with 0 ≤ s ≤ f α } > 0 . Thus there exists a simple function s = ∑ m i =1 s i 1 S i with 0 ≤ s ≤ f α and ∫ s d π > 0 . Here the s i ∈ R and the measurable sets S i ⊂ R d are disjoint. In particular, it must be the case that at least one of the coefficients s i is positive; without loss of generality suppose s 1 > 0 . Then ˜ s := s 1 /α 1 1 S 1 is a simple function with 0 ≤ ˜ s ≤ f and ∫ ˜ s d π > 0 . It follows that ∫ f d π = sup { ∫ s d π : s a simple function with 0 ≤ s ≤ f } > 0 .

Proof of Theorem 1. Since ∫ ( q/p ) α d π n ∈ (0 , ∞ ) and q/p ≥ 0 , from Proposition 8 we have, for each n , that Z n := ∫ q/p d π n > 0 . Thus the assumptions of Proposition 7 are satisfied by k and each π n , which guarantees that D p,q ( π n ) = Z n D q,q (¯ π n ) where ¯ π n := ( qπ n ) / ( pZ n ) ∈ P ( R d ) .

Now, since W 1 ( π n , p ; q/p ) → 0 , taking f = 1 we obtain Z n = ∫ fq/p d π n → ∫ fq/p d p = 1 . In addition, note that

Thus, from the triangle inequality, we obtain the bound

as n →∞ . For ( ∗ ) , since q/p ≥ 0 , the supremum is realised by f ( x ) = ∥ x ∥ and

Thus we have established that W 1 (¯ π n , q ) → 0 . Since ∇ log q is Lipschitz with ∫ ∥∇ log q ∥ 2 d q < ∞ and k has continuous and bounded second derivatives, the standard kernel Stein discrepancy has 1-Wasserstein convergence detection [Proposition 9 of Gorham and Mackey, 2017], meaning that W 1 (¯ π n , q ) → 0 implies that D q,q (¯ π n ) → 0 and thus, since Z n → 1 , D p,q ( π n ) → 0 . This completes the proof.

To prove Proposition 3, an intermediate result is required:

Proposition 9. Let k ( x, y ) = ϕ ( x -y ) be a kernel with ϕ twice differentiable and let q ∈ P ( R d ) with ∇ log q well-defined. Then k q ( x, x ) = -∆ ϕ (0) + ϕ (0) ∥∇ log q ( x ) ∥ 2 , where ∆ = ∇· ∇ and k q was defined in Proposition 5.

Proof. First, note that we must have ∇ ϕ (0) = 0 , else the symmetry property of k would be violated. Now, ∇ x k ( x, y ) = ( ∇ ϕ )( x -y ) , ∇ y k ( x, y ) = -( ∇ ϕ )( x -y ) and ∇ x · ∇ y k ( x, y ) = -∆ ϕ ( x -y ) . Thus ∇ x k ( x, y ) | y = x = ∇ y k ( x, y ) | x = y = 0 and ∇ x · ∇ y k ( x, y ) | x = y = -∆ ϕ (0) . Plugging these expressions into Equation (5) yields the result.

Proof of Proposition 3. Let k q be defined as in Proposition 5. From Cauchy-Schwarz, we have that k q ( x, y ) ≤ √ k q ( x, x ) √ k q ( y, y ) , and plugging this into Proposition 5 we obtain the bound

For a radial kernel k ( x, y ) = ϕ ( x -y ) with ϕ twice differentiable, we have ϕ (0) > 0 (else k must be the zero kernel, since by Cauchy-Schwarz | k ( x, y ) | ≤ √ k ( x, x ) √ k ( y, y ) = ϕ (0) for all x, y ∈ R d ), and k q ( x, x ) = -∆ ϕ (0) + ϕ (0) ∥∇ log q ( x ) ∥ 2 (from Proposition 9). Plugging this expression into Equation (11) and applying Jensen's inequality gives that

Now we may pick a choice of p , q and ( π n ) n ∈ N ( π n d ↛ p ) for which this bound can be made arbitrarily small. One example is q = N (0 , 1) , p = N (0 , σ 2 ) (any fixed σ > 1 ), for which we have

where γ = 1 -σ -2 > 0 . Then it is clear that, for example, the sequence π n = δ ( ne 1 ) (where e 1 = [1 , 0 , . . . , 0] ⊤ ) satisfies the assumptions of Proposition 2 and, for this choice,

and yet π n d ↛ p , as claimed.

Proof of Theorem 2. Since inf x ∈ R d q ( x ) /p ( x ) > 0 , for each n we have Z n := ∫ q/p d π n > 0 and, furthermore, the assumption ∫ ∥∇ log q ∥ α/ ( α -1) ( q/p ) d π n < ∞ implies that ∫ ∥∇ log q ∥ α/ ( α -1) d π n < ∞ . Thus the assumptions of Proposition 7 are satisfied by k and each π n , and thus we have D p,q ( π n ) = Z n D q,q (¯ π n ) where ¯ π n := ( qπ n ) / ( pZ n ) ∈ P ( R d ) .

From assumption, Z n ≥ inf x ∈ R d q ( x ) /p ( x ) is bounded away from 0. Thus if D p,q ( π n ) → 0 then D q,q (¯ π n ) → 0 . Furthermore, since q ∈ Q ( R d ) and the inverse multi-quadric kernel k is used, the standard kernel Stein discrepancy has convergence control, meaning that D q,q (¯ π n ) → 0 implies ¯ π n d → q [Theorem 8 of Gorham and Mackey, 2017]. It therefore suffices to show that ¯ π n d → q implies π n d → p .

From the Portmanteau theorem, π n d → p is equivalent to ∫ g d π n → ∫ g d p for all functions g which are continuous and bounded. Thus, for an arbitrary continuous and bounded function g , consider f = gp/q , which is also continuous and bounded. Then, since ¯ π n d → q , we have (again from the Portmanteau theorem) that Z -1 n ∫ g d π n = ∫ f d¯ π n → ∫ f d q = ∫ g d p . Furthermore, the specific choice g = 1 shows that Z -1 n → 1 , and thus ∫ g d π n → ∫ g d p in general. Since g was arbitrary, we have established that π n d → p , completing the proof.

To prove Theorem 3, an intermediate result is required:

Proposition 10. Let Q ∈ P ( R d ) and let k q : R d × R d → R be a reproducing kernel with ∫ k q ( x, · ) d q = 0 for all x ∈ R d . Let ( x n ) n ∈ N be a sequence of random variables independently sampled from q and assume that ∫ exp { γk q ( x, x ) } d q ( x ) < ∞ for some γ > 0 . Then

almost surely as n →∞ .

̸

Proof. This is Lemma 4 in Riabiz et al. [2022], specialised to the case where samples are independent and identically distributed. Although not identical to the statement in Riabiz et al. [2022], one obtains this result by following an identical argument and noting that the expectation of k q ( x i , x j ) is identically 0 when i = j (due to independence of x i and x j ), so that bounds on these terms are not required.

Proof of Theorem 3. Since π n has finite support, all conditions of Theorem 2 are satisfied. Thus it is sufficient to show that almost surely D p,q ( π n ) → 0 . To this end, we follow Theorem 3 of Riabiz et al. [2022] and introduce the classical importance weights w i = p ( x i ) /q ( x i ) , which are well-defined since q > 0 . The normalised weights ¯ w i = w i /W n , W n := ∑ n j =1 w j satisfy 0 ≤ ¯ w 1 , . . . , ¯ w n and ¯ w 1 + · · · + ¯ w n = 1 , and thus the optimality of w ∗ , together with the integral form of the GF-KSD in Equation (4), gives that

From the strong law of large numbers, almost surely n -1 W n → ∫ p q d q = 1 . Thus it suffices to show that the final term in Equation (12) converges almost surely to 0. To achieve this, we can check the conditions of Proposition 10 are satisfied.

Since q and h ( · ) = S p,q k ( x, · ) satisfy the conditions of Proposition 1, the condition ∫ k q ( x, · ) d q = 0 is satisfied. Let ϕ ( z ) = (1 + ∥ z ∥ 2 ) -β so that k ( x, y ) = ϕ ( x -y ) is the inverse multi-quadric kernel. Note that ϕ (0) = 1 and ∆ ϕ (0) = -d . Then, from Proposition 9, we have that k q ( x, x ) = -∆ ϕ (0) + ϕ (0) ∥∇ log q ∥ 2 = d + ∥∇ log q ∥ 2 . Then

which establishes that the conditions of Proposition 10 are satisfied and completes the proof.

B Stein Variational Inference Without Second-Order Gradient

This section contains sufficient conditions for unbiased stochastic gradient estimators to exist in the context of Stein Variational Inference; see Section 4.2 of the main text. The main result that we prove is as follows:

Proposition 11 (Stochastic Gradients) . Let p, q, R ∈ P ( R d ) and T θ : R d → R d for each θ ∈ R p . Let θ ↦→∇ θ T θ ( x ) be bounded. Assume that for each ϑ ∈ R p there is an open neighbourhood N ϑ ⊂ R p such that

for each of r ∈ { p, q } . Let k be the inverse multi-quadric kernel in Equation (2) and let u ( x, y ) denote the integrand in Equation (6) . Then

̸

where the expectation is taken with respect to independent samples x 1 , . . . , x n ∼ R .

The role of Proposition 11 is to demonstrate how an unbiased gradient estimator may be constructed, whose computation requires first-order derivatives of p only, and whose cost is O ( n 2 ) . Although the assumption that θ ↦→∇ θ T θ ( x ) is bounded seems strong, it can typically be satisfied by reparametrisation of θ ∈ R p .

To prove Proposition 11, we exploit the following general result due to Fisher et al. [2021]:

Proposition 12. Let R ∈ P ( R p ) . Let Θ ⊆ R p be an open set and let u : R d × R d → R , T θ : R d → R d , θ ∈ Θ , be functions such that, for all ϑ ∈ Θ ,

(A1) ∫∫ | u ( T ϑ ( x ) , T ϑ ( y )) | d R ( x )d R ( y ) < ∞ ;

(A2) there exists an open neighbourhood N ϑ ⊂ Θ of ϑ such that

Then F ( θ ) := ∫∫ u ( T θ ( x ) , T θ ( y )) d R ( x )d R ( y ) is well-defined for all θ ∈ Θ and

̸

where the expectation is taken with respect to independent samples x 1 , . . . , x n ∼ R .

Proof. This is Proposition 1 in Fisher et al. [2021].

Proof of Proposition 11. In what follows we aim to verify the conditions of Proposition 12 hold for the choice u ( x, y ) = k q ( x, y ) , where k q was defined in Equation (5).

(A1): From the first line in the proof of Proposition 4, the functions S p,q k ( x, · ) , x ∈ R d , are in H ( k ) d . Since ( x, y ) ↦→ u ( x, y ) = ⟨S p,q k ( x, · ) , S p,q k ( y, · ) ⟩ H ( k ) d is positive semi-definite, from Cauchy-Schwarz, | u ( x, y ) | ≤ √ u ( x, x ) √ u ( y, y ) . Thus

Since k ( x, y ) = ϕ ( x -y ) , we have from Proposition 9 that

and

which is finite by assumption.

(A2): Fix x, y ∈ R d and let R x ( θ ) := q ( T θ ( x )) /p ( T θ ( y )) . From repeated application of the product rule of differentiation, we have that

Let b p ( x ) := ∇ log p ( x ) , b q ( x ) := ∇ log q ( x ) , b ( x ) := b q ( x ) -b p ( x ) , and [ ∇ θ T θ ( x )] i,j = ( ∂/∂θ i ) T θ j ( x ) . In what follows, we employ a matrix norm on R d × d which is consistent with the Euclidean norm on R d , meaning that ∥∇ θ T θ ( x ) b ( T θ ( x )) ∥ ≤ ∥∇ θ T θ ( x ) ∥∥ b ( T θ ( x )) ∥ for each θ ∈ Θ and x ∈ R d . Considering the first term ( ∗ ) , further applications of the chain rule yield that

and from the triangle inequality we obtain a bound

Let ≲ denote inequality up to an implicit multiplicative constant. Since we assumed that ∥∇ θ T θ ( x ) ∥ is bounded, and the inverse multi-quadric kernel k is bounded, we obtain that

Similarly, from Equation (5), and using also the fact that the inverse multi-quadric kernel k has derivatives or all orders [Lemma 4 of Fisher et al., 2021], we obtain a bound

which we multiply by R x ( θ ) R y ( θ ) to obtain a bound on ( ∗∗ ) . Thus we have an overall bound

Substituting this bound into ∫∫ sup θ ∈ N ϑ ∥∇ θ u ( T θ ( x ) , T θ ( y )) ∥ d R ( x )d R ( y ) , and factoring terms into products of single integrals, we obtain an explicit bound on this double integral in terms of the following quantities (where r ∈ { p, q } ):

which we have assumed exist.

Thus the conditions of Proposition 12 hold, and the result immediately follows.

C Experimental Details

These appendices contain the additional empirical results referred to in Section 3, together with full details required to reproduce the experiments described in Section 4 of the main text.

C.1 Detection of Convergence and Non-Convergence

This appendix contains full details for the convergence plots of Figure 1. In Figure 1, we considered the target distribution

where N ( x ; µ, σ 2 ) is the univariate Gaussian density with mean µ and variance σ 2 . The parameter choices used were ( w 1 , w 2 , w 3 ) = (0 . 375 , 0 . 5625 , 0 . 0625) , ( µ 1 , µ 2 , µ 3 ) = ( -0 . 4 , 0 . 3 , 0 . 06) and ( σ 2 1 , σ 2 2 , σ 2 3 ) = (0 . 2 , 0 . 2 , 0 . 9) .

The approximating sequences considered were location-scale sequences of the form L n # u , where L n ( x ) = a n + b n x for some ( a n ) n ∈ N and ( b n ) n ∈ N and u ∈ P ( R ) . For the converging sequences, we set u = p and for the non-converging sequences, we set u = N (0 , 0 . 5) . We considered three different choices of ( a n ) n ∈ N and ( b n ) n ∈ N , one for each colour. The sequences ( a n ) n ∈ N and ( b n ) n ∈ N used are shown in Figure S1. The specification of our choices of q is the following:

Prior : We took q ∼ N (0 , 0 . 75 2 ) .

Laplace : The Laplace approximation computed was q ∼ N (0 . 3 , 0 . 2041 2 ) .

0

20

40

60

80

100

n

0.5

0.0

0.5

1.0

1.5

2.0

2.5

3.0

an

bn

(a)

0

20

40

60

80

100

n

0.0

0.5

1.0

1.5

2.0

2.5

3.0

an

bn

(b)

GMM : The Gaussian mixture model was computed using 100 samples from the target p . The number of components used was 2 , since this value minimised the Bayes information criterion [Schwarz, 1978].

KDE : The kernel density estimate was computed using 100 samples from the target p . We utilised a Gaussian kernel k ( x, y ) = exp( -( x -y ) 2 /ℓ 2 ) with the lengthscale or bandwidth parameter ℓ determined by Silverman's rule of thumb [Silverman, 1986].

The values of GF-KSD reported in Figure 1 were computed using a quasi Monte Carlo approximation to the integral (6), utilising a length 300 low-discrepancy sequence. The low discrepancy sequences were obtained by first specifying a uniform grid over [0 , 1] and then performing the inverse CDF transform for each member of the sequence π n .

C.2 Avoidance of Failure Modes

This appendix contains full details of the experiment reported in Section 3.3 and an explanation of the failure mode reported in Figure 2a. The sequences considered are displayed in Figure S2. Each sequence was a location-scale sequences of the form L n # u , where L n ( x ) = a n + b n x for some ( a n ) n ∈ N and ( b n ) n ∈ N and u ∈ P ( R ) . For the converging sequences, we set u = p . The specification of the settings of each failure mode are as follows:

Failure mode (a) [Figure 2a]: We took p as the target used in Figure 1 and detailed in Appendix C.1 and took q ∼ N (0 , 1 . 5 2 ) . The a n and b n sequences used are displayed in Figure S1a. The values of GF-KSD reported were computed using a quasi Monte Carlo approximation, using a length 300 low discrepancy sequence. The low discrepancy sequences were obtained by first specifying a uniform grid over [0 , 1] and then performing the inverse CDF transform for each member of the sequence π n .

Failure mode (b) [Figure 2b]: We took p ∼ N (0 , 1) and q ∼ N ( -0 . 7 , 0 . 1 2 ) . The a n and b n sequences used are displayed in Figure S1b. The values of GF-KSD reported were computed using a quasi Monte Carlo approximation, using a length 300 low discrepancy sequence. The low discrepancy sequences were obtained by first specifying a uniform grid over [0 , 1] and then performing the inverse CDF transform for each member of the sequence π n .

Failure Mode (c) [Figure 2c]: In each dimension d considered, we took p ∼ N (0 , I ) and q ∼ N (0 , 1 . 1 I ) . The a n and b n sequences 5 used are displayed in Figure S1b. The values of GF-KSD reported were computed using a quasi Monte Carlo approximation, using a length 1 , 024 Sobol sequence in each dimension d .

3

2

1

0

1

2

3

0.0

0.5

1.0

1.5

2.0

2.5

(a)

3

2

1

0

1

2

3

0.0

0.2

0.4

0.6

0.8

1.0

1.2

3

2

1

0

1

2

3

0.0

0.2

0.4

0.6

0.8

1.0

1.2

4

2

0

2

4

0.0

0.2

0.4

0.6

0.8

(b) and (c)

4

2

0

2

4

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

4

2

0

2

4

0.0

0.2

0.4

0.6

0.8

1.0

1.2

3

2

1

0

1

2

3

0

1

2

3

4

Converging Sequence (d)

3

2

1

0

1

2

3

0.00

0.25

0.50

0.75

1.00

1.25

1.50

1.75

2.00

3

2

1

0

1

2

3

0

1

2

3

4

5

6

3

2

1

0

1

2

3

0

2

4

6

8

Non-Converging Sequence (d)

3

2

1

0

1

2

3

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

3

2

1

0

1

2

3

0

2

4

6

8

10

12

Failure mode (d) [Figure 2d]: We took p = q , with p ( x ) = 0 . 5 N ( x ; -1 , 0 . 1 2 ) + 0 . 5 N ( x ; 1 , 0 . 1 2 ) , where N ( x ; µ, σ 2 ) is the univariate Gaussian density with mean µ and variance σ 2 . The a n and b n sequences used are displayed in Figure S1b. For the non-converging sequences we took u = N (1 , 0 . 1 2 ) and used the a n and b n sequences specified in Figure S1b. The values of GF-KSD reported were computed using a quasi Monte Carlo approximation, using a length 300 low discrepancy sequence. The low discrepancy sequences were obtained by first specifying a uniform grid over [0 , 1] and then performing the inverse CDF transform for each member of the sequence π n .

In Figure S3, we provide an account of the degradation of convergence detection between q = Prior considered in Figure 1 and q = N (0 , 1 . 5 2 ) of Failure mode (a). In Figure S3a, it can be seen that the value of the integrals ∫ ( q/p ) 2 d π n are finite for each element of the pink sequence π n . However, in Figure S3b, it can be seen that the values of the integrals ∫ ( q/p ) 2 d π n are infinite for the last members of the sequence π n , thus violating a condition of Theorem 1.

C.3 Gradient-Free Stein Importance Sampling

This appendix contains full details for the experiment reported in Section 4.1. We considered the following Lotka-Volterra dynamical system:

2

4

2

0

2

4

x

0.0

0.5

1.0

1.5

2.0

2.5

q

2

/

p

2

n

q

= Prior

q

2

/

p

(a)

2

0

2

x

0

5

10

15

20

25

q

2

/

p

2

n

q

=

(0, 1.5

)

q

2

/

p

(b)

4

2

0

2

4

x

0

5

10

15

20

25

30

35

q

2

/

p

q

=

(0, 1

2

)

q

=

(0, 1.1

2

)

q

=

(0, 1.2

2

)

q

=

(0, 1.3

2

)

q

=

(0, 1.4

2

)

(c)

Using 21 observations u 1 , . . . , u 21 and v 1 , . . . , v 21 over times t 1 < . . . < t 21 , we considered the probability model

In order to satisfy positivity constraints, we performed inference on the logarithm of the parameters ( α, β, γ, δ, u 0 , v 0 , σ 1 , σ 2 ) = (log α ′ , log β ′ , log γ ′ , log δ ′ , log u ′ 0 , log v ′ 0 , log σ ′ 1 , log σ ′ 2 ) . Wetook the following independent priors on the constrained parameters:

In order to obtain independent samples from the posterior for comparison, we utilised Stan [Stan Development Team, 2022] to obtain 8 , 000 posterior samples using four Markov chain Monte Carlo chains. Each chain was initialised at the prior mode. The data analysed are due to Hewitt [1921] and can be seen, along with a posterior predictive check, in Figure S4.

The Laplace approximation was obtained by the use of 48 iterations of the L-BFGS optimisation algorithm [Liu and Nocedal, 1989] initialised at the prior mode. The Hessian approximation was obtained using Stan's default numeric differentiation of the gradient.

Finally, the quadratic programme defining the optimal weights of gradient-free Stein importance sampling (refer to Theorem 3) was solved using the splitting conic solver of O'Donoghue et al. [2016].

C.4 Stein Variational Inference Without Second-Order Gradient

This appendix contains full details for the experiment reported in Section 4.2. We considered the following bivariate densities

where N ( x ; µ, σ 2 ) is the univariate Gaussian density with mean µ and variance σ 2 . The parameter choices for the sinusoidal experiment p 1 were η 2 1 = 1 . 3 2 , η 2 2 = 0 . 09 2 and a = 1 . 2 . The parameter choices for the banana experiment p 2 were σ 2 1 = 1 , σ 2 2 = 0 . 2 2 and b = 0 . 5 .

The development of a robust stochastic optimisation routine for measure transport with GF-KSD is beyond the scope of this work, and in what follows we simply report one strategy that was successfully used in the setting of the application reported in the main text. This strategy was based on

Hare

Lynx

1900

1905

1910

1915

1920

0

20

40

60

80

0

20

40

60

80

Year

Pelts (thousands)

Data

Prediction

0

2500

5000

7500

10000

12500

15000

17500

20000

m

0.0

0.2

0.4

0.6

0.8

1.0

m

tempering of p , the distributional target, to reduce a possibly rather challenging variational optimisation problem into a sequence of easier problems to be solved. Specifically, we considered tempered distributions p m ∈ P ( R d ) with log density

where ( ϵ m ) m ∈ N ∈ [0 , 1] N is the tempering sequence and p 0 ∈ P ( R d ) is fixed. In this case p 0 was taken to be N (0 , 2 I ) in both the banana and sinusoidal experiment. Then, at iteration m of stochastic optimisation, we considered the variational objective function

where q = π θ m , as explained in the main text. Tempering has been applied in the context of normalising flows in Prangle and Viscardi [2023]. The tempering sequence used ( ϵ m ) m ∈ N for each of the experiments is displayed in Figure S5.

For each experiment, the stochastic optimisation routine used was Adam [Kingma and Ba, 2015] with learning rate 0 . 001 . Due to issues involving exploding gradients due to the q/p term in GFKSD, we utilised gradient clipping in each of the variational inference experiments, with the maximum 2 -norm value taken to be 30 . In both the banana and sinusoidal experiment, the parametric class of transport maps T θ was the inverse autoregressive flow of Kingma et al. [2016]. In the banana experiment, the dimensionality of the hidden units in the underlying autoregressive neural network was taken as 20 . In the sinusoidal experiment, the dimensionality of the hidden units in the underlying autoregressive neural network was taken as 30 . For the comparison with standard kernel Stein discrepancy, the same parametric class T θ and the same initialisations of θ were used.

1.5

1.0

0.5

0.0

0.5

1.0

1.5

x

0

2

4

6

8

(a)

0

20

40

60

80

100

m

0.0

0.2

0.4

0.6

0.8

1.0

(b)

C.5 Additional Experiments

Appendix C.5.1 explores the impact of p on the conclusions drawn in the main text. Appendix C.5.2 investigates the sensitivity of the proposed discrepancy to the choice of the parameters σ and β that appear in the kernel. Appendix C.5.3 compares the performance of GF-KSD importance sampling, KSD importance sampling and self-normalised importance sampling.

C.5.1 Exploring the Effect of p

In this section we investigate the robustness of the convergence detection described in Figure 1 subject to different choices of the target p . We consider two further choices of p :

where N ( x ; µ, σ 2 ) is the univariate Gaussian density with mean µ and variance σ 2 and Student-T ( x ; ν, m, s ) is the univariate Student-T density with degrees of freedom ν , location parameter m and scale parameter s . The parameter choices for p 1 were

The parameter choices for p 2 were ν = 10 and

Instead of using the location-scale sequences of Figure 1, we instead considered tempered sequences of the form

For the converging sequences considered we set u to be the target (either u = p 1 or u = p 2 ) and set u = N ( x ; 0 , 0 . 4 2 ) for each of the non-converging sequences. The different sequences vary in choice of π 0 and tempering sequence ( ϵ n ) n ∈ N . These choices are displayed in Figure S6 and are taken as the same for both of the targets considered.

The specification of our choices of q is the following:

Prior : For p 1 , we took q ∼ N (0 , 0 . 5 2 ) . For p 2 , we took q ∼ Student-T (10 , 0 , 0 . 5) .

Laplace : For p 1 , the Laplace approximation computed was q ∼ N (0 , 0 . 051 2 ) . For p 2 , the Laplace approximation computed was q ∼ N (0 , 0 . 125 2 ) .

GMM : For both targets, the Gaussian mixture model was computed using 100 samples from the target. In both cases, the number of components used was 3 , since this value minimised the Bayes information criterion [Schwarz, 1978].

KDE : For both targets, the kernel density estimate was computed using 100 samples from the target. In both cases, we utilised a Gaussian kernel k ( x, y ) = exp( -( x -y ) 2 /ℓ 2 ) with the lengthscale or bandwidth parameter ℓ determined by Silverman's rule of thumb [Silverman, 1986].

Results for p 1 are displayed in Figure S7 and results for p 2 are displayed in Figure S8. It can be seen that for both target distributions and the different sequences considered, GF-KSD correctly detects convergence in each case. For both targets and for q = Laplace , it can be seen that GF-KSD exhibits the same behaviour of Failure Mode (b), displayed in Figure 2b.

The values of GF-KSD reported in Figure S7 and Figure S8 were computed using a quasi Monte Carlo approximation to the integral (6), utilising a length 300 low-discrepancy sequence. Due to the lack of an easily computable inverse CDF, we performed an importance sampling estimate of GF-KSD as follows

where w is the proposal distribution. For each element of a sequence π n , we used a Gaussian proposal w n of the form:

Since π 0 is Gaussian for each sequence, this construction ensures that each w n is both Gaussian and a good proposal distribution for π n . The low-discrepancy sequences were then obtained by first specifying a uniform grid over [0 , 1] and the performing an inverse CDF transformation using w n .

C.5.2 Exploring the Effect of σ and β

In this section we investigate the effect on convergence detection that results from changing the parameters σ and β in the inverse multi-quadric kernel (2). Utilising the same test sequences and choices of q used in Figure 1, we plot the values of GF-KSD in Figure S9. It can be seen that the convergence detection is robust to changing values of σ and β .

C.5.3 GF-KSD vs. KSD Importance Sampling

In this section we investigate the performance of gradient-free Stein importance sampling, standard Stein importance sampling, and self-normalised importance sampling, as the distribution q varies in quality as an approximation to p . We consider two different regimes:

In both cases, we consider the performance of each approach for varying dimension d and number of samples n . Results are reported in Figure S10 and Figure S11 for each regime respectively. The quadratic programme defining the optimal weights of gradient-free Stein importance sampling and Stein importance sampling (refer to Theorem 3) was solved using the splitting conic solver of O'Donoghue et al. [2016].

Inward Convergence

Oblique Convergence

Oblique Non-Convergence

(a)

10

0

10

20

q

=Prior

q

=Laplace

0

25

50

75

100

n

10

0

10

20

q

=GMM

0

25

50

75

100

n

q

=KDE

p

q

p

q

p

q

p

q

(b)

Inward Convergence

Oblique Convergence

Oblique Non-Convergence

(a)

20

10

0

10

20

q

=Prior

q

=Laplace

0

25

50

75

100

n

20

10

0

10

20

q

=GMM

0

25

50

75

100

n

q

=KDE

p

q

p

q

p

q

p

q

(b)

5

0

5

(

,

) = (0.5,0.5)

q

=Prior

q

=Laplace

q

=GMM

q

=KDE

5

0

5

(

,

) = (1,0.5)

5

0

5

(

,

) = (2,0.5)

5

0

5

(

,

) = (1,0.1)

0

50

100

n

5

0

5

(

,

) = (1,0.9)

0

50

100

n

0

50

100

n

0

50

100

n

p

q

p

q

p

q

p

q

3.25

3.00

2.75

2.50

2.25

2.00

1.75

log ED (

d

=1)

n

=10

6.5

6.0

5.5

5.0

4.5

4.0

n

=100

9

8

7

6

5

4

3

n

=500

GF-KSD

KSD

Self-Normalised

1.6

1.4

1.2

1.0

0.8

log ED (

d

=5)

4.25

4.00

3.75

3.50

3.25

3.00

2.75

6.00

5.75

5.50

5.25

5.00

4.75

0.8

1.0

1.2

0.75

0.50

0.25

0.00

0.25

0.50

log ED (

d

=20)

0.8

1.0

1.2

3

2

1

0

0.8

1.0

1.2

5

4

3

2

1

0.8

1.0

1.2

0.0

0.5

1.0

1.5

log ED (

d

=50)

0.8

1.0

1.2

2

1

0

1

0.8

1.0

1.2

4

3

2

1

0

3.50

3.25

3.00

2.75

2.50

2.25

2.00

log ED (

d

=1)

n

=10

6.5

6.0

5.5

5.0

4.5

4.0

n

=100

9

8

7

6

5

4

3

2

n

=500

GF-KSD

KSD

Self-Normalised

1.5

1.0

0.5

0.0

log ED (

d

=5)

4.0

3.5

3.0

2.5

2.0

1.5

1.0

6.0

5.5

5.0

4.5

4.0

3.5

0.5

0.0

0.5

0.5

0.0

0.5

1.0

1.5

log ED (

d

=20)

0.5

0.0

0.5

3

2

1

0

0.5

0.0

0.5

5

4

3

2

1

0.5

0.0

0.5

c

0.0

0.5

1.0

1.5

2.0

log ED (

d

=50)

0.5

0.0

0.5

c

2

1

0

1

0.5

0.0

0.5

c

4

3

2

1

0

1

Peer reviews:
comments: SUMMARY:
The paper provides a method to estimate the kernel Stein discrepancy without gradients.

STRENGTHS:
The paper provides a method to estimate the kernel Stein discrepancy without gradients.

WEAKNESSES:
The paper is not clearly written for someone who is not familiar with the field. After reading, I'm still confused about whether someone else has published a similar method before.

QUESTIONS:
How limited is the class of distributions that allow gradient free estimation of the kernel Stein discrepancy?

And the paper probably cherry picked good experiments, how often do the mentioned failure modes occur?

LIMITATIONS:
The paper is not clearly written for someone who is not familiar with the field. After reading, I'm still confused about whether someone else has published a similar method before.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-06-19

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper explores the use of Stein discrepancies in scenarios where the score function of the target distribution is unavailable or computationally impractical to evaluate. The authors introduce a novel approach called gradient-free kernelized Stein discrepancy (GF-KSD), which leverages a Stein operator developed by Han and Liu (2018) that does not rely on the score function. The authors establish sufficient conditions for the resulting divergence to control and detect convergence. The empirical evaluation of this divergence extends its application to Stein importance sampling and Stein variational inference, surpassing the scope of the previous work by Han and Liu (2018).\n\nSTRENGTHS:\n**Theories**: Whilst the gradient-free Stein operator has been previously discussed in the community (Han and Liu, 2018), its theoretical properties have not received sufficient attention. This paper established conditions under which the GF-KSD can detect and control convergence of a sequence of empirical distributions to a target probability measure (Theorem 1 and 2). To my knowledge, this contribution fills an important gap in the existing literature and significantly enhances our understanding of the topic.\n\n**Discussions**: Key results on both the theoretical and empirical sides are sufficiently discussed. Limitations of the GF-KSD are thoroughly examined and supported by empirical evidence (Section 3.3). Discussions on the choice of the degree of freedom, the density, are also included (Section 3.1).\n\n**Structure**: This paper exhibits excellent writing with clear motivations throughout.\n\nJ. Han and Q. Liu. Stein variational gradient descent without gradient. In Proceedings of the 35th International Conference on Machine Learning, pages 1900–1908. PMLR, 2018.\n\nWEAKNESSES:\n**Experiments**: My only major concern is over the empirical results. The problems examined in the experiments are primarily toy examples with relatively low dimensionalities. where the one with the highest dimension is a 8-dimensional inference problem for a Lotka-Volterra model. The highest-dimensional problem considered is an 8-dimensional inference problem for a Lotka-Volterra model. To further validate the applicability of the proposed method in more realistic scenarios, additional empirical evidence on higher-dimensional problems would be beneficial.\n\nSpecifically, it would be interesting to investigate the performance of the default choice of Laplace approximation when the dimensionality of the target distribution is high. This analysis could shed light on whether the default approach continues to yield satisfactory results in non-toy scenarios.\n\nMoreover, the reported numerical instability issue in Section 4.2 warrants further investigation, particularly in high-dimensional problems. Understanding the extent to which this instability becomes prominent in higher dimensions is crucial as it may impact the practical viability of the GF-KSD approach.\n\nQUESTIONS:\nIt would be helpful to elaborate on the numerical instability issue noted in Section 4.2. Does this issue only occur when using GF-KSD for Stein variational inference, or does it also occur in the experiments in Section 4.1? Is that an artefact of the specific form of the target densities chosen for this experiment? Can this be avoided by a judicious choice of $q$?\n\nLIMITATIONS:\nExamples where the proposed approach can fail are extensively discussed in Section 3.3.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-21"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposed a posterior approximation using a new Stein discrepancy, which does not require derivatives of the statistical model. For that purpose, the authors derived the new discrepancy, called gradient-free KSD, and studied its statistical and convergence behaviors theoretically.\nThen the authors developed algorithms for differential equations, for which stable computation of derivatives is difficult, and a new sampling algorithm that bypass the Hessian calculations.\n\nSTRENGTHS:\n- A new KSD that does not require gradients, similar to the idea of importance sampling, is proposed. This leads to new algorithms for differential equations where the gradient is difficult to compute and for problems requiring Hessian calculation.\n\n- The authors studied the theoretical property of the proposed GF-KSD by extending the existing KSD theory.\n\n- Not only theoretical analysis but also detailed numerical investigations of the choice of parameters and $q$ are carried out with the actual use in mind.\n\nWEAKNESSES:\n- The writing style is such that the main paper alone is not complete, and it is assumed that the reader will read the Appendix. For example, Eq. 6 of Line 115 does not appear in the main text, and the tilted Wasserstein distance defined in Theorem 1 is introduced without any explanation of its properties in the main text.\n\n- The writing style could be improved since the discussion about existing research and the explanation of the proposed method are mixed, making the paper difficult to read.\n\n-Some parts are mathematically undefined or under-discussed\n - In Definition 2, sup is undefined\n - In Line 158, at last, $\\not \\to$  is undefined.\n - I don't know how widely the tilted Wasserstein distance (TWD) in Theorem 1 is known to the general public, but there is no discussion of the properties of TWD. Therefore I could not understand how important Theorem 1 is, that is, how important it is when it is said that convergence of TWD leads to convergence of GF-KSD; even after reading the proof of Theorem 1, I could only understand that TWD is a convenient form of the usual Wasserstein distance, which is obtained after applying the triangle inequality.\n\n- I could not understand the importance of the proposed method because I am not sure for what problems the proposed method is effective. \nI agree that it may be useful for differential equation problems, but the authors only applied the method to very small models of Lotka-Volterra in the experiments. In such a setting, MCMC is the standard approach, and even if the model is high-dimensional, we can solve it efficiently by variational inference. \nAlso, although the combination with Stein variational inference seems interesting, I wondered if GF-KSD is really flexible enough to generate samples for complex real data under the two restrictions suggested in Section 3. \nOne restriction is that the tail of $q$ should not be far from the target distribution; the other is that it must not be high-dimensional. \nI think the application to differential equations seems promising, so it would be better to find a problem setting where GF-KSD is more useful than MCMC and standard variational inference.\n\nQUESTIONS:\nI would appreciate it if the authors would answer the concerns described in Weakness.\n\nAs for minor questions;\n- What do the dotted and solid lines in Figure 1(b) correspond to?\n\n- Is it required to adjust parameters of the Laplace distribution, KDE, and GMM  for $q$ in some way ? If so, what is the recommended method ?\n\n- Looking at Figure 3 (b), it seems that the number of samples ($n$) must be very large ($\\log n=5$, i.e., $n=150$) even for low-dimensional problems such as d=8 in order for there to be any difference in energy distance. Is my understanding correct?\n\nLIMITATIONS:\nThe limitation of the proposed method is discussed in detail.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors study the kernel Stein discrepancy based on a Stein operator, introduced previously in [Liu 2018], which does not require access to the gradient of the target density. As their main theoretical result, they prove that, under certain assumptions on the target distribution (and the auxiliary distribution q) and the approximating sequence, the discrepancy controls weak convergence. The authors provide recommendations for the choice of the auxiliary distribution q in the construction of the discrepancy and present an experimental study which identifies certain failure modes of the discrepancy. They also discuss two applications to posterior approximation.\n\nSTRENGTHS:\nThe idea of the authors to use the gradient-free Stein operator in order to define a new kind of KSD is interesting and worth studying. There are certainly a lot of situations where the evaluation of the gradient of the model is costly in practice. Giving the practitioners an option to avoid evaluating it when computing the KSD is certainly useful. The discussion of the properties of the new GF-KSD is very thorough and based on both theoretical results and a substantial experimental study. I particularly liked the the experiments revealing failure modes of GF-KSD as those are not necessarily captured by the theoretical results.\n\nWEAKNESSES:\nThe main theoretical contribution of the paper seems to be provided by Theorem 2. However, I am a bit concerned about its real applicability. The authors require that the auxiliary distribution $q$ have tails heavier than the target $p$. On the other hand, they require that $q$ is distantly dissipative, which precludes the use of heavy-tailed $q$. This all means that $p$ must not be heavy-tailed if the assumptions of Theorem 2 are to be satisfied. \n\nMoreover, the authors suggest four potential choices of $q$ (Prior, Laplace, GMM and KDE) but note that GMM and KDE are impractical in general. At the same time, the Prior method does not satisfy the assumptions of Theorem 2 for heavy-tailed priors. The Laplace method, on the other hand, seems to satisfy the assumptions of Theorem 2 only for targets which are sub-Gaussian.\n\nQUESTIONS:\nRelated to what I wrote above, could the authors state clearly what class of targets $p$ their Theorem 2 applies to? Could they also try to characterise the class of targets $p$ for which one can easily construct a useful auxiliary distribution $q$ in practice (using one of the methods of section 3.1), such that $q$ and $p$ satisfy the assumptions of Theorem 2?\n\nLIMITATIONS:\nI believe the real applicabilty of the theoretical results presented by the authors is somewhat limited, to an extent that is not clearly acknowledged in the paper. But I am looking forward to reading the authors' response as perhaps I'm missing something.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-08-07"}