SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Matthew A. Fisher<sup>1</sup>, Chris. J Oates<sup>1,2</sup>
1Newcastle University, UK
$^{2}$ Alan Turing Institute, UK
Stein discrepancies have emerged as a powerful statistical tool, being applied to fundamental statistical problems including parameter inference, goodness-of-fit testing, and sampling. The canonical Stein discrepancies require the derivatives of a statistical model to be computed, and in return provide theoretical guarantees of convergence detection and control. However, for complex statistical models, the stable numerical computation of derivatives can require bespoke algorithmic development and render Stein discrepancies impractical. This paper focuses on posterior approximation using Stein discrepancies, and introduces a collection of non-canonical Stein discrepancies that are gradient-free, meaning that derivatives of the statistical model are not required. Sufficient conditions for convergence detection and control are established, and applications to sampling and variational inference are presented.
Stein discrepancies were introduced in Gorham and Mackey [2015], as a way to measure the quality of an empirical approximation to a continuous statistical model involving an intractable normalisation constant. Rooted in Stein's method [Stein, 1972], the idea is to consider empirical averages of a large collection of test functions, each of which is known to integrate to zero under the statistical model. To date, test functions have been constructed by combining derivatives of the statistical model with reproducing kernels [Chwialkowski et al., 2016, Liu et al., 2016, Gorham and Mackey, 2017, Gong et al., 2021a,b], random features [Huggins and Mackey, 2018], diffusion coefficients and functions with bounded derivatives [Gorham et al., 2019], neural networks [Grathwohl et al., 2020], and polynomials [Chopin and Ducrocq, 2021]. The resulting discrepancies have been shown to be powerful statistical tools, with diverse applications including parameter inference [Barp et al., 2019, Matsubara et al., 2022], goodness-of-fit testing [Jitkrittum et al., 2017, Fernandez et al., 2020], and sampling [Liu and Lee, 2017, Chen et al., 2018, 2019, Riabiz et al., 2022, Hodgkinson et al., 2020, Fisher et al., 2021]. However, one of the main drawbacks of these existing works is the requirement that derivatives both exist and can be computed.
The use of non-differentiable statistical models is somewhat limited but includes, for example, Bayesian analyses where Laplace priors are used [Park and Casella, 2008, Ročková and George, 2018]. Much more common is the situation where derivatives exist but cannot easily be computed. In particular, for statistical models with parametric differential equations involved, one often requires different, more computationally intensive numerical methods to be used if the sensitivities (i.e. derivatives of the solution with respect to the parameters) are to be stably computed [Cockayne and Duncan, 2021]. For large-scale partial differential equation models, as used in finite element simulation, computation of sensitivities can increase simulation times by several orders of magnitude, if it is practical at all.
The motivation and focus of this paper is on computational methods for posterior approximation, and to this end we propose a collection of non-canonical Stein discrepancies that are gradient free, meaning that computation of the derivatives of the statistical model is not required. Gradient-free
Stein operators were introduced in Han and Liu [2018] in the context of Stein variational gradient descent [Liu and Wang, 2016], but the theoretical properties of the corresponding discrepancy have yet to be investigated. General classes of Stein discrepancies were analysed in Huggins and Mackey [2018], Gorham et al. [2019], but their main results do not cover the gradient-free Stein discrepancies developed in this work, for reasons that will be explained. The combination of gradient-free Stein operators and reproducing kernels is studied in detail, to obtain discrepancies that can be explicitly computed. The usefulness of these discrepancies depends crucially on their ability to detect the convergence and non-convergence of sequences of probability measures to the posterior target, and in both directions positive results are established.
Outline Gradient-free KSD (GF-KSD) is proposed and theoretically analysed in Section 2. The proposed discrepancy involves certain degrees of freedom, including a probability density denoted  $q$  in the sequel, and strategies for specifying these degrees of freedom are empirically assessed in Section 3. Two applications are then explored in detail; Stein importance sampling (Section 4.1) and Stein variational inference (Section 4.2). Conclusions are drawn in Section 5.
This section contains our core methodological (Section 2.1) and theoretical (Section 2.2) development. The following notation will be used:
Real Analytic Notation For a twice differentiable function  $f: \mathbb{R}^d \to \mathbb{R}$ , let  $\partial_i f$  denote the partial derivative of  $f$  with respect to its  $i$ th argument, let  $\nabla f$  denote the gradient vector with entries  $\partial_i f$ , and let  $\nabla^2 f$  denote the Hessian matrix with entries  $\partial_i \partial_j f$ . For a sufficiently regular bivariate function  $f: \mathbb{R}^d \times \mathbb{R}^d \to \mathbb{R}$ , let  $(\partial_i \otimes \partial_j) f$  indicate the application of  $\partial_i$  to the first argument of  $f$ , followed by the application of  $\partial_j$  to the second argument. (For derivatives of other orders, the same tensor notation  $\otimes$  will be used.)
Probabilistic Notation Let  $\mathcal{P}(\mathbb{R}^d)$  denote the set of probability distributions on  $\mathbb{R}^d$ . Let  $\delta(x) \in \mathcal{P}(\mathbb{R}^d)$  denote an atomic distribution located at  $x \in \mathbb{R}^d$ . For  $\pi, \pi_0 \in \mathcal{P}(\mathbb{R}^d)$ , let  $\pi \ll \pi_0$  indicate that  $\pi$  is absolutely continuous with respect to  $\pi_0$ . For  $\pi \in \mathcal{P}(\mathbb{R}^d)$  and  $(\pi_n)_{n \in \mathbb{N}} \subset \mathcal{P}(\mathbb{R}^d)$ , write  $\pi_n \stackrel{\mathrm{d}}{\to} \pi$  to indicate weak convergence of  $\pi_n$  to  $\pi$ . The symbols  $p$  and  $q$  are reserved for probability density functions on  $\mathbb{R}^d$ , while  $\pi$  is reserved for a generic element of  $\mathcal{P}(\mathbb{R}^d)$ . For convenience, the symbols  $p$  and  $q$  will also be used to refer to the probability distributions that these densities represent.
2.1 Gradient-Free Kernel Stein Discrepancy
The aim of this section is to explain how a GF-KSD can be constructed. Let  $p \in \mathcal{P}(\mathbb{R}^d)$  be a target distribution of interest. Our starting point is a gradient-free Stein operator, introduced in Han and Liu [2018] in the context of Stein variational gradient descent [Liu and Wang, 2016]:
Definition 1 (Gradient-Free Stein Operator). For  $p, q \in \mathcal{P}(\mathbb{R}^d)$  with  $q \ll p$  and  $\nabla \log q$  well-defined, the gradient-free Stein operator is defined as
$$
\mathcal {S} _ {p, q} h := \frac {q}{p} (\nabla \cdot h + h \cdot \nabla \log q),
$$
acting on differentiable functions  $h:\mathbb{R}^d\to \mathbb{R}^d$
The Langevin Stein operator of Gorham and Mackey [2015] is recovered when  $p = q$ , but when  $q \neq p$  the dependence on the derivatives of  $p$  is removed. The operator  $S_{p,q}$  can still be recognised as a diffusion Stein operator, being related to the infinitesimal generator of a diffusion process that leaves  $p$  invariant; however, it falls outside the scope of the theoretical analysis of Huggins and Mackey [2018], Gorham et al. [2019], for reasons explained in Remark 2. It can also be viewed as a non-standard instance of the density method of Diaconis et al. [2004]; see Section 2 of Anastasiou et al. [2023]. The inclusion of  $q$  introduces an additional degree of freedom, specific choices for which are discussed in Section 3.
Remark 1. The ratio  $q / p$  in Definition 1 could be viewed as an importance weight, but the construction does not fully correspond to importance sampling due to the  $\nabla \log q$  term, which is  $q$ -dependent.
The Stein operator nomenclature derives from the vanishing integral property in Proposition 1 below, which is central to Stein's method [Stein, 1972]:
Proposition 1. In the setting of Definition 1, assume that  $\| x\|^{d - 1}q(x)\to 0$  as  $\| x\| \to \infty$  and  $\int \| \nabla \log q\| \mathrm{d}q < \infty$ . Then, for any function  $h:\mathbb{R}^d\rightarrow \mathbb{R}^d$  whose first derivatives exist and are bounded, it holds that  $\int S_{p,q}h\mathrm{d}p = 0$ .
All proofs are contained in Appendix A. From Proposition 1, the expectation of  $S_{p,q}h$  with respect to  $\pi \in \mathcal{P}(\mathbb{R}^d)$  will be zero when  $\pi$  and  $p$  are equal; conversely, the value of such an expectation can be used to quantify the extent to which  $\pi$  and  $p$  are different. Consideration of multiple test functions increases the number and nature of the differences between  $\pi$  and  $p$  that may be detected. A discrepancy is obtained by specifying which test functions  $h$  are considered, and then taking a supremum over the expectations associated to this set. For computational convenience we take  $h$  to be contained in the unit ball of a reproducing kernel Hilbert space, as described next.
For a symmetric positive definite function  $k:\mathbb{R}^d\times \mathbb{R}^d\to \mathbb{R}$ , called a kernel, denote the associated reproducing kernel Hilbert space as  $\mathcal{H}(k)$ . Let  $\mathcal{H}(k)^d$  denote the Cartesian product of  $d$  copies of  $\mathcal{H}(k)$ , equipped with the inner product  $\langle h,g\rangle_{\mathcal{H}(k)^d}:= \sum_{i = 1}^{d}\langle h_i,g_i\rangle_{\mathcal{H}(k)}$ .
Proposition 2. Let  $\pi \in \mathcal{P}(\mathbb{R}^d)$ . In the setting of Definition 1, assume there is an  $\alpha > 1$  such that  $\int (q / p)^{\alpha} \mathrm{d}\pi < \infty$  and  $\int \| \nabla \log q\|^{\alpha / (\alpha - 1)} \mathrm{d}\pi < \infty$ . Let  $k: \mathbb{R}^d \times \mathbb{R}^d \to \mathbb{R}$  be a continuously differentiable kernel such that both  $k$  and its first derivatives  $x \mapsto (\partial_i \otimes \partial_i) k(x, x)$ ,  $i = 1, \ldots, d$ , are bounded. Then  $S_{p,q}$  is a bounded linear operator from  $\mathcal{H}(k)^d$  to  $L^1(\pi)$ .
For discrete distributions  $\pi$ , supported on a finite subset of  $\mathbb{R}^d$ , the moment conditions in Proposition 2 are automatically satisfied. For general distributions  $\pi$  on  $\mathbb{R}^d$ , the exponent  $\alpha$  can be taken arbitrarily close to 1 to enable the more stringent moment condition  $\int (q / p)^{\alpha}\mathrm{d}\pi < \infty$  to hold. An immediate consequence of Proposition 2 is that Definition 2 below is well-defined.
Definition 2 (Gradient-Free Kernel Stein Discrepancy). For  $p, q, k$  and  $\pi$  satisfying the preconditions of Proposition 2, the gradient-free KSD is defined as
$$
\mathrm {D} _ {p, q} (\pi) = \sup  \left\{\int \mathcal {S} _ {p, q} h \mathrm {d} \pi : \| h \| _ {\mathcal {H} (k) ^ {d}} \leq 1 \right\}. \tag {1}
$$
The GF-KSD coincides with the canonical kernel Stein discrepancy (KSD) when  $p = q$ , and is thus strictly more general. Note that  $\mathrm{D}_{p,q}(\pi)$  is precisely the operator norm of the linear functional  $h \mapsto \int S_{p,q}h \, \mathrm{d}\pi$ , which exists due to Proposition 2. Most common kernels satisfy the assumptions of Proposition 2, and a particularly important example is the inverse multi-quadratic kernel
$$
k (x, y) = \left(\sigma^ {2} + \| x - y \| ^ {2}\right) ^ {- \beta}, \quad \sigma \in (0, \infty), \beta \in (0, 1), \tag {2}
$$
which has bounded derivatives of all orders; see Lemma 4 of Fisher et al. [2021].
The use of reproducing kernels ensures that GF-KSD can be explicitly computed; see Proposition 5 and Corollary 1 in Appendix A. A general spectral characterisation of GF-KSD is provided in Proposition 6 of the supplement, inspired by the recent work of Wynne et al. [2022]. Note that the scale of Equation (6) does not matter when one is interested in the relative performance of different  $\pi \in \mathcal{P}(\mathbb{R}^d)$  as approximations of a fixed target  $p\in \mathcal{P}(\mathbb{R}^d)$ . In this sense, GF-KSD may be employed with  $\tilde{p}$  in place of  $p$ , where  $p\propto \tilde{p} /Z$  and  $Z$  is a normalisation constant. This feature makes GF-KSD applicable to problems of posterior approximation, and will be exploited for both sampling and variational inference in Section 4. On the other hand, GF-KSD is not applicable to problems in which the target distribution  $p_{\theta}$  involves a parameter  $\theta$ , such as estimation and composite hypothesis testing, since then the normalisation term  $Z_{\theta}$  cannot be treated as constant.
Going beyond the discrepancies discussed in Section 1, several non-canonical discrepancies have recently been proposed based on Stein's method, for example sliced [Gong et al., 2021a,b], stochastic [Gorham et al., 2020], and conditional [Singhal et al., 2019] Stein discrepancies (in all cases derivatives of  $p$  are required). However, only a subset of these discrepancies have been shown to enjoy important guarantees of convergence detection and control. Convergence control is critical for the posterior approximation task considered in this work, since this guarantees that minimisation of Stein discrepancy will produce a consistent approximation of the posterior target. The aim of the next section is to establish such guarantees for GF-KSD.
2.2 Convergence Detection and Control
The canonical KSD benefits from theoretical guarantees of convergence detection and control [Gorham and Mackey, 2017]. The aim of this section is to establish analogous guarantees for GF-KSD. To set the scene, for a Lipschitz function  $f: \mathbb{R}^d \to \mathbb{R}^d$ , denote its Lipschitz constant  $L(f) := \sup_{x \neq y} \|f(x) - f(y)\| / \|x - y\|$ . Then, for measurable  $g: \mathbb{R}^d \to \mathbb{R}$ , we denote the tilted Wasserstein distance as  $\mathrm{W}_1(\pi, p; g) := \sup_{L(f) \leq 1} \left| \int fg \, \mathrm{d}\pi - \int fg \, \mathrm{d}p \right|$  whenever this expression is well-defined [Huggins and Mackey, 2018]. Note that the standard 1-Wasserstein distance  $\mathrm{W}_1(\pi, p)$  is recovered when  $g = 1$ . There is no dominance relation between  $\mathrm{W}_1(\cdot, \cdot; g)$  for different  $g$ ; the topologies they induce are different. Tilted Wasserstein distance induces a much weaker topology than, for example, divergences such as Kullback-Leibler or Hellinger, since it does not require absolute continuity of measures.
Theorem 1 (Convergence Detection). Let  $p, q \in \mathcal{P}(\mathbb{R}^d)$  with  $q \ll p$ ,  $\nabla \log q$  Lipschitz and  $\int \| \nabla \log q \|^2 \, \mathrm{d}q < \infty$ . Assume there is an  $\alpha > 1$  such that the sequence  $(\pi_n)_{n \in \mathbb{N}} \subset \mathcal{P}(\mathbb{R}^d)$  satisfies  $\int (q/p)^{\alpha} \, \mathrm{d}\pi_n \in (0, \infty)$ ,  $\int \| \nabla \log q \|^{\alpha/(\alpha-1)} \, \mathrm{d}\pi_n < \infty$ ,  $\int \| \nabla \log q \|^{\alpha/(\alpha-1)} (q/p) \, \mathrm{d}\pi_n < \infty$ , and  $\int fq/p \, \mathrm{d}\pi_n < \infty$  with  $f(x) = \| x \|$ , for each  $n \in \mathbb{N}$ . Let  $k$  be a kernel such that each of  $k, (\partial_i \otimes \partial_i)k$  and  $(\partial_i \partial_j \otimes \partial_i \partial_j)k$  exist, are continuous, and are bounded, for  $i, j \in \{1, \ldots, d\}$ . Then  $W_1(\pi_n, p; q/p) \to 0$  implies  $D_{p,q}(\pi_n) \to 0$ .
Thus the convergence of  $\pi_n$  to  $p$ , in the sense of the tilted Wasserstein distance with  $g = q / p$ , is detected by the GF-KSD  $\mathrm{D}_{p,q}$ . Note that the conditions on  $(\pi_n)_{n\in \mathbb{N}}$  in Theorem 1 are automatically satisfied when each  $\pi_n$  has a finite support.
Despite being a natural generalisation of KSD, this GF-KSD does not in general provide weak convergence control in the equivalent theoretical context. Indeed, Gorham and Mackey [2017] established positive results on convergence control for distributions in  $\mathcal{Q}(\mathbb{R}^d)$ , the set of probability distributions on  $\mathbb{R}^d$  with positive density function  $q: \mathbb{R}^d \to (0, \infty)$  for which  $\nabla \log q$  is Lipschitz and  $q$  is distantly dissipative, meaning that
$$
\operatorname * {l i m i n f} _ {r \to \infty} \inf \left\{- \frac {\langle \nabla \log q (x) - \nabla \log q (y) , x - y \rangle}{\| x - y \| ^ {2}}: \| x - y \| = r \right\} > 0.
$$
Under the equivalent assumptions, convergence control fails for GF-KSD in general:
Proposition 3 (Convergence Control Fails in General). Let  $k$  be a (non-identically zero) radial kernel of the form  $k(x,y) = \phi (x - y)$  for some twice continuously differentiable  $\phi :\mathbb{R}^d\to \mathbb{R}$ , for which the preconditions of Proposition 2 are satisfied. Then there exist  $p\in \mathcal{P}(\mathbb{R}^d)$ ,  $q\in \mathcal{Q}(\mathbb{R}^d)$  and a sequence  $(\pi_n)_{n\in \mathbb{N}}\subsetneq \mathcal{P}(\mathbb{R}^d)$ , also satisfying the preconditions of Proposition 2, such that  $\mathrm{D}_{p,q}(\pi_n)\rightarrow 0$  and yet  $\pi_n\stackrel {\mathrm{d}}{\rightarrow}p$ .
The purpose of Proposition 3 is to highlight that GF-KSD is not a trivial extension of canonical KSD; it requires a bespoke treatment. This is provided in Theorem 2, next. Indeed, to ensure that GF-KSD provides convergence control, additional condition on the tails of  $q$  are required:
Theorem 2 (Convergence Control). Let  $p \in \mathcal{P}(\mathbb{R}^d)$ ,  $q \in \mathcal{Q}(\mathbb{R}^d)$  be such that  $p$  is continuous and  $\inf_{x \in \mathbb{R}^d} q(x) / p(x) > 0$ . Assume there is an  $\alpha > 1$  such that the sequence  $(\pi_n)_{n \in \mathbb{N}} \subset \mathcal{P}(\mathbb{R}^d)$  satisfies  $\int (q/p)^{\alpha} \, \mathrm{d}\pi_n < \infty$  and  $\int \| \nabla \log q \|^{\alpha / (\alpha - 1)}(q/p) \, \mathrm{d}\pi_n < \infty$ , for each  $n \in \mathbb{N}$ . Let  $k$  be the inverse multi-quadric kernel in Equation (2). Then  $D_{p,q}(\pi_n) \to 0$  implies  $\pi_n \stackrel{\mathrm{d}}{\to} p$ .
The proof of Theorem 2 is based on carefully re-casting GF-KSD between  $\pi$  and  $p$  as a canonical KSD between  $q$  and a transformed distribution  $\bar{\pi}$  (see Proposition 7 in the supplement), then appealing to the analysis of Gorham and Mackey [2017]. Compared to the analysis of KSD in Gorham and Mackey [2017], the distant dissipativity condition now appears on  $q$  (a degree of freedom), rather than on  $p$  (a distribution determined by the task at hand), offering a realistic opportunity for this condition to be verified. For example, one could take a Gaussian measure  $q$  that dominates the target distribution  $p$  for use in GF-KSD. Nevertheless, the conditions of Theorem 2 rule out distributions  $p$  that are heavy-tailed. Suitable choices for  $q$  are considered in Section 3.
Remark 2 (Related Work). Convergence control was established for discrepancies based on general classes of Stein operator in earlier work, but the required assumptions are too stringent when applied in our context. In particular, to use Huggins and Mackey [2018] it is required that the gradient
Figure 1: Empirical assessment of gradient-free kernel Stein discrepancy. (a) Test sequences  $(\pi_n)_{n \in \mathbb{N}}$ , defined in Appendix C.1. The first column displays sequences (solid) that converge to the distributional target  $p$  (black), while the second column displays sequences (dashed) which converge instead to a fixed Gaussian target. (b) Performance of gradient-free kernel Stein discrepancy, when approaches to selecting  $q$  (described in the main text) are employed. The colour and style of each curve in (b) indicates which of the sequences in (a) is being considered. [Here we fixed the kernel parameters  $\sigma = 1$  and  $\beta = 1/2$ .]
$\nabla \log (q / p)$  is bounded<sup>1</sup>, while to use Gorham et al. [2019] it is required that  $q / p$  is Lipschitz<sup>2</sup>. In our context, where  $q$  must be specified in ignorance of  $p$ , such conditions, which require that  $q$  is almost as light as  $p$  in the tail, cannot be guaranteed to hold. The present paper therefore instead contributes novel analysis for the regime where  $q$  may be appreciably heavier than  $p$  in the tail.
This completes our theoretical assessment, but the practical performance of GF-KSD remains to be assessed. Suitable choices for both  $q$  and the kernel parameters  $\sigma$  and  $\beta$  are proposed and investigated in Section 3, and practical demonstrations of GF-KSD are presented in Section 4.
3 Implementation Detail
The purpose of this section is to empirically explore the effect of varying  $q$ ,  $\sigma$  and  $\beta$ , aiming to arrive at reasonable default settings. In the absence of an application-specific optimality criterion, we aim to select values that perform well (in a sense to be specified) over a range of scenarios that may be encountered. Here, to assess performance several sequences  $(\pi_n)_{n\in \mathbb{N}}$  are considered, some of which converge to a specified limit  $p$  and the rest of which converge to an alternative Gaussian target; see Figure 1(a). An effective discrepancy should clearly indicate which of these sequences are convergent and which are not. On this basis, recommendations for  $q$  are considered in Section 3.1, and recommendations for  $\sigma$  and  $b$  in Section 3.2. Of course, we cannot expect default settings to perform universally well, so in Section 3.3 we highlight scenarios where our defaults may fail. Python code to reproduce the experiments reported below can be downloaded at [blinded].
3.1 Choice of  $q$
In what follows we cast  $q$  as an approximation of  $p$ , aiming to inherit the desirable performance of canonical KSD for which  $q$  and  $p$  are equal. The task to which the discrepancy is being applied will,
To see this, take  $A = q / p$  in Theorem 3.2 of Huggins and Mackey [2018].
2To see this, take  $m = (q / p)I$  in Theorem 7 and Proposition 8 of Gorham et al. [2019].
in practice, constrain the nature and form of the distributions  $q$  that can be implemented. For expository purposes (only), the following qualitatively distinct approaches to choosing  $q$  are considered:
- Prior In Bayesian settings where  $p$  is a posterior distribution, selecting  $q$  to be the prior distribution ensures that the condition  $\inf_{x \in \mathbb{R}^d} q(x) / p(x) > 0$  is satisfied.
- Laplace If the target  $p$  can be differentiated, albeit at a possibly high computational cost, it may be practical to construct a Laplace approximation  $q$  to the target [Gelman et al., 2013].
- GMM One could take  $q$  to be a Gaussian mixture model fitted to approximate samples from  $p$ , representing a more flexible alternative to Laplace.
- KDE Additional flexibility can be obtained by employing a kernel density estimator as a nonparametric alternative to GMM.
Of course, there is a circularity to GMM and KDE which renders these methods impractical in general. The specific details of how each of the  $q$  were constructed are contained in the code that accompanies this paper, but the resulting  $q$  are displayed as insets in Figure 1(b). The performance of GF-KSD with these different choices of  $q$  is also displayed in Figure 1(b). It was observed that all four choices of  $q$  produced a discrepancy that could detect convergence of  $\pi_n$  to  $p$ , though the detection of convergence was less clear for Prior due to slower convergence of the discrepancy to 0 as  $n$  was increased. On the other hand, all approaches were able to clearly detect non-convergence to the target. That Laplace performed comparably with GMM and KDE was surprising, given that the target  $p$  is not well-approximated by a single Gaussian component. These results are for a specific choice of target  $p$ , but in Appendix C.5.1 a range of  $p$  are considered and similar conclusions are obtained. Section 3.3 explores how challenging  $p$  must be before the convergence detection and control properties associated to Laplace fail.