SlowGuess/NeurIPS_2023_OCR huggingface dataset:
3.2 Choice of  $\sigma$  and  $\beta$
For the investigation in Section 3.1 the parameters of the inverse multi-quadric kernel (2) were fixed to  $\sigma = 1$  and  $\beta = 1/2$ , the latter being the midpoint of the permitted range  $\beta \in (0,1)$ . In general, care in the selection of these parameters may be required. The parameter  $\sigma$  captures the scale of the data, and thus standardisation of the data may be employed to arrive at  $\sigma = 1$  as a natural default. In this paper (with the exception of Section 4.2) the standardisation  $x \mapsto C^{-1}x$  was performed, where  $C$  is the covariance matrix of the approximating distribution  $q$  being used. In Appendix C.5.2 we reproduce the investigation of Section 3.1 using a range of values for  $\sigma$  and  $\beta$ ; these results indicate the performance of GF-KSD is remarkably insensitive to perturbations around  $(\sigma, \beta) = (1,1/2)$ .
3.3 Avoidance of Failure Modes
GF-KSD is not a silver bullet, and there are a number of specific failure modes that care may be required to avoid. The four main failure modes are illustrated in Figure 2. These are as follows: (a)  $q$  is substantially heavier than  $p$  in a tail; (b)  $q$  is substantially lighter than  $p$  in a tail; (c) the dimension  $d$  is too high; (d)  $p$  has well-separated high-probability regions. Under both (a) and (c), convergence detection can fail, either because theoretical conditions are violated or because the terms  $\pi_n$  must be extremely close to  $p$  before convergence begins to be detected. Under (b), the values of GF-KSD at small  $n$  can mislead. Point (d) is a well-known pathology of all score-based methods; see Wenliang and Kanagawa [2021], Liu et al. [2023]. These four failure modes inform our recommended usage of GF-KSD, summarised next.
Summary of Recommendations Based on the investigation just reported, the default settings we recommend are Laplace with  $\sigma = 1$  (post-standardisation) and  $\beta = 1/2$ . Although not universally applicable, Laplace does not require samples from  $p$ , and has no settings that must be user-specified. Thus we recommend the use of Laplace in situations where a Laplace approximation can be justified and computed. If Laplace not applicable, then one may attempt to construct an approximation  $q$  using techniques available for the task at hand (e.g. in a Bayesian setting, one may obtain  $q$  via variational inference, or via inference based on an approximate likelihood). These recommended settings will be used for the application presented in Section 4.1, next.
(c)
Figure 2: Failure modes: (a)  $q$  is substantially heavier than  $p$  in a tail; (b)  $q$  is substantially lighter than  $p$  in a tail; (c) the dimension  $d$  is too high; (d)  $p$  has separated high-probability regions. [For each of (a), (b) and (d), the colour and style of the curves refers to the same sense of convergence or non-convergence (outward/inward/oblique) presented in Figure 1, and we plot the logarithm of the gradient-free kernel Stein discrepancy as a function of the index  $n$  of the sequence  $(\pi_n)_{n \in \mathbb{N}}$ . For (c) we consider convergent sequences  $(\pi_n)_{n \in \mathbb{N}}$  of distributions on  $\mathbb{R}^d$ .
To demonstrate potential uses of GF-KSD, two applications to posterior approximation are now presented; Stein importance sampling (Section 4.1) and Stein variational inference using measure transport (Section 4.2). In each case, we extend the applicability of existing algorithms to statistical models for which certain derivatives of  $p$  are either expensive or non-existent.
4.1 Gradient-Free Stein Importance Sampling
Stein importance sampling [Liu and Lee, 2017, Hodgkinson et al., 2020] operates by first sampling independently from a tractable approximation of the target  $p$  and then correcting the bias in the samples so-obtained. To date, applications of Stein importance sampling have been limited to instances where the statistical model  $p$  can be differentiated; our contribution is to remove this requirement. In what follows we analyse Stein importance sampling in which independent samples  $(x_{n})_{n\in \mathbb{N}}$  are generated from the same approximating distribution  $q$  that is employed within GF-KSD:
Theorem 3 (Gradient-Free Stein Importance Sampling). Let  $p \in \mathcal{P}(\mathbb{R}^d)$ ,  $q \in \mathcal{Q}(\mathbb{R}^d)$  be such that  $p$  is continuous and  $\inf_{x \in \mathbb{R}^d} q(x) / p(x) > 0$ . Suppose that  $\int \exp \{\gamma \| \nabla \log q\|^2\} \, \mathrm{d}q < \infty$  for some  $\gamma > 0$ . Let  $k$  be the inverse multi-quadratic kernel in Equation (2). Let  $(x_n)_{n \in \mathbb{N}}$  be independent samples from  $q$ . To the sample, assign optimal weights
$$
w ^ {*} \in \arg \min  \left\{\mathrm {D} _ {p, q} \left(\sum_ {i = 1} ^ {n} w _ {i} \delta \left(x _ {i}\right)\right): 0 \leq w _ {1}, \dots , w _ {n}, w _ {1} + \dots + w _ {n} = 1 \right\}.
$$
Then  $\pi_n := \sum_{i=1}^n w_i^* \delta(x_i)$  satisfies  $\pi_n \xrightarrow{\mathrm{d}} p$  almost surely as  $n \to \infty$ .
The proof builds on earlier work in Riabiz et al. [2022]. Note that the optimal weights  $w^{*}$  can be computed without the normalisation constant of  $p$ , by solving a constrained quadratic programme at cost  $O(n^3)$ .
As an illustration, we implemented gradient-free Stein importance sampling to approximate a posterior arising from a discretely observed Lotka-Volterra model
$$
\dot {u} (t) = \alpha u (t) - \beta u (t) v (t), \qquad \dot {v} (t) = - \gamma v (t) + \delta u (t) v (t), \qquad (u (0), v (0)) = (u _ {0}, v _ {0}),
$$
with independent log-normal observations with covariance matrix  $\mathrm{diag}(\sigma_1^2,\sigma_2^2)$ . The parameters to be inferred are  $\{\alpha ,\beta ,\gamma ,\delta ,u_0,v_0,\sigma_1,\sigma_2\}$  and therefore  $d = 8$ . The data analysed are due to Hewitt [1921], and full details are contained in Appendix C.3. The direct application of Stein importance sampling to this task requires the numerical calculation of sensitivities of the differential equation at each of the  $n$  samples that are to be re-weighted. Aside from simple cases where automatic differentiation or adjoint methods can be used, the stable computation of sensitivities can form a major computational bottleneck; see [Riabiz et al., 2022]. In contrast, our approach required a
(b)
Figure 3: Gradient-Free Stein Importance Sampling: (a) The lower triangular panels display  $n = 20$  independent (biased) samples from the Laplace approximation  $q$ , while the upper triangular panels display the same number of re-weighted samples obtained using gradient-free Stein importance sampling. [Samples are shown in blue, with their size proportional to the square of their weight, to aid visualisation. The shaded background indicates the high probability regions of  $p$ , the target.] (b) The approximation quality, as a function of the number  $n$  of samples from  $q$ , is measured as the energy distance between the approximation and the target. [The solid line corresponds to the output of gradient-free Stein importance sampling, while the dashed line corresponds to the output of self-normalised importance sampling. Standard error regions are shaded.]
fixed number of gradient computations to construct a Laplace approximation $^3$ , independent of the number  $n$  of samples required; see Appendix C.3 for detail. In the lower triangular portion of Figure 3(a), biased samples from the Laplace approximation  $q (\neq p)$  are displayed, while in the upper triangular portion the same samples are re-weighted using gradient-free Stein importance sampling to form a consistent approximation of  $p$ . A visual reduction in bias and improvement in approximation quality can be observed. As a baseline against which to assess the quality of our approximation, we consider self-normalised importance sampling; i.e. the approximation with weights  $\bar{w}$  such that  $\bar{w}_i \propto p(x_i) / q(x_i)$ . Figure 3(b) reports the accuracy of the approximations to  $p$  as quantified using energy distance [Cramér, 1928]. These results indicate that the approximations produced using gradient-free Stein importance sampling improve on those constructed using self-normalised importance sampling. This may be explained by the fact that the optimal weights  $w^*$  attempt to mitigate both bias due to  $q \neq p$  and Monte Carlo error, while the weights  $\bar{w}$  only address the bias due to  $q \neq p$ , and do not attempt to mitigate error due to the randomness in Monte Carlo. Additional experiments in Appendix C.5.3 confirm that gradient-free Stein importance sampling achieves comparable performance with gradient-based Stein importance sampling in regimes where the Laplace approximation can be justified.
Although our recommended default settings for GF-KSD were successful in this example, an interesting theoretical question would be to characterise an optimal choice of  $q$  in this context. This appears to be a challenging problem but we hope to address it in future work. In addition, although we focused on Stein importance sampling, our methodology offers the possibility to construct gradient-free versions of other related algorithms, including the Stein points algorithms of Chen et al. [2018, 2019], and the Stein thinning algorithm of Riabiz et al. [2022].
4.2 Stein Variational Inference Without Second-Order Gradient
Stein discrepancy was proposed as a variational objective in Ranganath et al. [2016] and has demonstrated comparable performance with the traditional Kullback-Leibler objective in certain application areas, whilst abolishing the requirement that the variational family is absolutely continuous with
In this case, 49 first order derivatives were computed, of which 48 were on the optimisation path and 1 was used to construct a finite difference approximation to the Hessian.
Figure 4: Stein Variational Inference Without Second Order Gradient: The top row concerns approximation of a distributional target  $p$  that is "banana" shaped, while the bottom row concerns a "sinusoidal" target. The first four columns depict the variational approximation  $\pi_{\theta_m}$  to  $p$  constructed using gradient descent applied to gradient-free kernel Stein discrepancy (i.e. first order derivatives of  $p$  required) along the stochastic optimisation sample path ( $m \in \{0, 2 \times 10^3, 10^4, 2 \times 10^4\}$ ), while the final column reports the corresponding approximation ( $m = 2 \times 10^4$ ) constructed using standard kernel Stein discrepancy (i.e. second order derivatives of  $p$  required).
respect to the statistical model [Fisher et al., 2021]. This offers an exciting, as yet largely unexplored opportunity to construct flexible variational families outside the conventional setting of normalising flows (which are constrained to be diffeomorphisms of  $\mathbb{R}^d$ ). However, gradient-based stochastic optimisation of the canonical Stein discrepancy objective function means that second-order derivatives of the statistical model are required. GF-KSD reduces the order of derivatives that are required from second-order to first-order, and in many cases (such as when differential equations appear in the statistical model) this will correspond to a considerable reduction in computational cost.
In what follows we fix a reference distribution  $R \in \mathcal{P}(\mathbb{R}^d)$  and a parametric class of maps  $T^\theta : \mathbb{R}^d \to \mathbb{R}^d$ ,  $\theta \in \mathbb{R}^p$ , and consider the variational family  $(\pi_\theta)_{\theta \in \mathbb{R}^p} \subset \mathcal{P}(\mathbb{R}^d)$  whose elements  $\pi_\theta := T_\#^\theta R$  are the pushforwards of  $R$  through the maps  $T^\theta$ ,  $\theta \in \mathbb{R}^p$ . The aim is to use stochastic optimisation to minimise  $\theta \mapsto \mathrm{D}_{p,q}(\pi_\theta)$  (or, equivalently, any strictly increasing transformation thereof). For this purpose a low-cost unbiased estimate of the gradient of the objective function is required; the details and sufficient theoretical conditions are contained in Appendix B.
As an interesting methodological extension, that departs from the usual setting of stochastic optimisation, here we consider interlacing stochastic optimisation over  $\theta$  and the selection of  $q$ , leveraging the current value  $\theta_{m}$  on the optimisation path to provide a natural candidate  $\pi_{\theta_m}$  for  $q$  in this context. Thus, for example, a vanilla stochastic gradient descent routine becomes  $\theta_{m + 1} = \theta_{m} - \epsilon \nabla_{\theta}\mathrm{D}_{p,\pi_{\theta_{m}}}(\pi_{\theta})^{2}\big|_{\theta = \theta_{m}}$  for some learning rate  $\epsilon >0$ . (In this iterative setting, to ensure the variational objective remains fixed, we do not perform the standardisation of the data described in Section 3.2.)
To assess the performance of GF-KSD in this context, we re-instantiated an experiment from Fisher et al. [2021]. The results, in Figure 4, concern the approximation of "banana" and "sinusoidal" distributions in dimension  $d = 2$ , and were obtained using the reference distribution  $R = \mathcal{N}(0,2I)$  and taking  $T^{\theta}$  to be the inverse autoregressive flow of Kingma et al. [2016]. These are both toy problems, which do not themselves motivate our methodological development, but do enable us to have an explicit ground truth to benchmark performance against. Full experimental detail is contained in Appendix C.4; we highlight that gradient clipping was used, both to avoid extreme values of  $q / p$  encountered on the optimisation path, and to accelerate the optimisation itself [Zhang et al., 2019]. The rightmost panel depicts the result of performing variational inference with the standard KSD objective functional. It is interesting to observe that GF-KSD leads to a similar performance in both examples, with the caveat that stochastic optimisation was more prone to occasional failure when GF-KSD was used. The development of a robust optimisation technique in this context requires care and a detailed empirical assessment, and is left as a promising avenue for further research.
GF-KSD 0 iterations
GF-KSD 2000 iterations
GF-KSD 10,000 iterations
GF-KSD 20,000 iterations
KSD 20,000 iterations
<sup>4</sup>If the density of  $\pi_{\theta_m}$  is not available, for example because  $T$  is a complicated mapping, it can be consistently estimated using independent samples from  $T_{\#}^{\theta_m}$ .
In this paper GF-KSD was proposed and studied. Theoretical and empirical results support the use of GF-KSD in settings where an initial approximation to the distributional target can readily be constructed, and where the distributional target itself does not contain distant high probability regions, but poor performance can occur outside this context. Nevertheless, for many statistical analyses the principal challenge is the cost of evaluating the statistical model and its derivatives, rather than the complexity of the target itself, and in these settings the proposed discrepancy has the potential to be usefully employed. The focus of this work was on posterior approximation, with illustrative applications to sampling and variational inference being presented. A natural extension to this work would involve a systematic empirical assessment of the performance of GF-KSD across a broad range of applied contexts. However, we note that GF-KSD is not applicable to problems such as estimation and composite hypothesis testing, where the target  $p_{\theta}$  ranges over a parametric model class, and alternative strategies will be required to circumvent gradient computation in that context.
Acknowledgements MAF was supported by EP/W522387/1. CJO was supported by EP/W019590/1. The authors are grateful to François-Xavier Briol, Jon Cockayne, Jeremias Knoblauch, Lester Mackey, Marina Riabiz, Rob Salomone, Leah South, and George Wynne for insightful comments on an early draft of the manuscript, as well as to the Reviewers at NeurIPS.
A. Anastasiou, A. Barp, F.-X. Briol, B. Ebner, R. E. Gaunt, F. Ghaderinezhad, J. Gorham, A. Gretton, C. Ley, Q. Liu, L. Mackey, C. J. Oates, G. Reinert, and Y. Swan. Stein's method meets statistics: A review of some recent developments. Statistical Science, 38(1):120-139, 2023.
A. Barp, F.-X. Briol, A. Duncan, M. Girolami, and L. Mackey. Minimum Stein discrepancy estimators. In Proceedings of the 33rd Conference on Neural Information Processing Systems, volume 32, pages 12964-12976, 2019.
W. Y. Chen, L. Mackey, J. Gorham, F.-X. Briol, and C. J. Oates. Stein points. In Proceedings of the 35th International Conference on Machine Learning, pages 844-853. PMLR, 2018.
W. Y. Chen, A. Barp, F.-X. Briol, J. Gorham, M. Girolami, L. Mackey, and C. J. Oates. Stein point Markov chain Monte Carlo. In Proceedings of the 36th International Conference on Machine Learning, pages 1011-1021. PMLR, 2019.
N. Chopin and G. Ducrocq. Fast compression of MCMC output. Entropy, 23(8):1017, 2021.
K. Chwialkowski, H. Strathmann, and A. Gretton. A kernel test of goodness of fit. In Proceedings of the 33rd International Conference on Machine Learning, pages 2606-2615. PMLR, 2016.
J. Cockayne and A. Duncan. Probabilistic gradients for fast calibration of differential equation models. SIAM/ASA Journal on Uncertainty Quantification, 9(4):1643-1672, 2021.
H. Cramér. On the composition of elementary errors. Scandinavian Actuarial Journal, (1):141-180, 1928.
P. Diaconis, C. Stein, S. Holmes, and G. Reinert. Use of exchangeable pairs in the analysis of simulations. In *Stein's Method*, pages 1-25. Institute of Mathematical Statistics, 2004.
T. Fernandez, N. Rivera, W. Xu, and A. Gretton. Kernelized Stein discrepancy tests of goodness-of-fit for time-to-event data. In Proceedings of the 37th International Conference on Machine Learning, pages 3112-3122. PMLR, 2020.
M. Fisher, T. Nolan, M. Graham, D. Prangle, and C. J. Oates. Measure transport with kernel Stein discrepancy. In Proceedings of the 24th International Conference on Artificial Intelligence and Statistics, pages 1054-1062. PMLR, 2021. (Here we refer to the error-corrected version arXiv:2010.11779.).
T. Matsubara, J. Knoblauch, F.-X. Briol, and C. J. Oates. Robust generalised Bayesian inference for intractable likelihoods. Journal of the Royal Statistical Society, Series B, 84(3):997-1022, 2022.
B. O'Donoghue, E. Chu, N. Parikh, and S. Boyd. Conic optimization via operator splitting and homogeneous self-dual embedding. Journal of Optimization Theory and Applications, 169(3): 1042-1068, June 2016.
T. Park and G. Casella. The Bayesian lasso. Journal of the American Statistical Association, 103 (482):681-686, 2008.
D. Prangle and C. Viscardi. Distilling importance sampling for likelihood free inference. Journal of Computational and Graphical Statistics, 2023. To appear.
R. Ranganath, D. Tran, J. Altosaar, and D. Blei. Operator variational inference. In Proceedings of the 30th Conference on Neural Information Processing Systems, pages 496-504, 2016.
M. Riabiz, W. Chen, J. Cockayne, P. Swietach, S. A. Niederer, L. Mackey, and C. J. Oates. Optimal thinning of MCMC output. Journal of the Royal Statistical Society, Series B, 84(4):1059-1081, 2022.
V. Ročková and E. I. George. The Spike-and-Slab LASSO. Journal of the American Statistical Association, 113(521):431-444, 2018.
G. Schwarz. Estimating the Dimension of a Model. The Annals of Statistics, 6(2):461-464, 1978.
B. W. Silverman. Density Estimation for Statistics and Data Analysis. Chapman & Hall, 1986.
R. Singhal, X. Han, S. Lahlou, and R. Ranganath. Kernelized complete conditional Stein discrepancy. arXiv:1904.04478, 2019.
Stan Development Team. RStan: the R interface to Stan, 2022. URL https://mc-stan.org/. R package version 2.21.5.
C. Stein. A bound for the error in the normal approximation to the distribution of a sum of dependent random variables. In Proceedings of the 6th Berkeley Symposium on Mathematical Statistics and Probability, volume 6.2, pages 583-603. University of California Press, 1972.
I. Steinwart and A. Christmann. Support Vector Machines. Springer Science & Business Media, 2008.
L. K. Wenliang and H. Kanagawa. Blindness of score-based methods to isolated components and mixing proportions. In Proceedings of the NeurIPS workshop "Your Model is Wrong", 2021.
G. Wynne, M. Kasprzak, and A. B. Duncan. A spectral representation of kernel Stein discrepancy with application to goodness-of-fit tests for measures on infinite dimensional Hilbert spaces. arXiv:2206.04552, 2022.
J. Zhang, T. He, S. Sra, and A. Jadbabaie. Why gradient clipping accelerates training: A theoretical justification for adaptivity. In International Conference on Learning Representations, 2019.
These appendices supplement the paper Gradient Free Kernel Stein Discrepancy. The proofs for theoretical results stated in the main text are contained in Appendix A. Theoretical analysis of stochastic gradient estimators for use in the context of Stein Variational Inference is contained in Appendix B, as advertised in Section 4.2 of the main text. All additional details related to empirical assessment are contained in Appendix C.
A Proof of Results in the Main Text
This appendix contains proofs for all novel theoretical results reported in the main text.
Proof of Proposition 1. First we show that the integral in the statement is well-defined. Since  $h$  and its first derivatives are bounded, we can define the constants  $C_0 \coloneqq \sup \{ \| h(x) \| : x \in \mathbb{R}^d \}$  and  $C_1 \coloneqq \sup \{ |\nabla \cdot h(x)| : x \in \mathbb{R}^d \}$ . Then
$$
\begin{array}{l} \int \left| \mathcal {S} _ {p, q} h \right| \mathrm {d} p = \int \left| \frac {q}{p} [ \nabla \cdot h + h \cdot \nabla \log q ] \right| \mathrm {d} p = \int | \nabla \cdot h + h \cdot \nabla \log q | \mathrm {d} q \\ \leq C _ {1} + C _ {0} \int \| \nabla \log q \| \mathrm {d} q <   \infty , \\ \end{array}
$$
so indeed the integral is well-defined. Now, let  $B_r = \{x \in \mathbb{R}^d : \| x \| \leq r\}$ ,  $S_r = \{x \in \mathbb{R}^d : \| x \| = r\}$ , and  $t(r) := \sup \{q(x) : x \in S_r\}$ , so that by assumption  $r^{d-1}t(r) \to 0$  as  $r \to \infty$ . Let  $\mathbb{1}_B(x) = 1$  if  $x \in B$  and  $0$  if  $x \notin B$ . Then  $\int S_{p,q}h \, \mathrm{d}p = \lim_{r \to \infty} \int \mathbb{1}_{B_r}S_{p,q}h \, \mathrm{d}p$  and, from the divergence theorem,
$$
\begin{array}{l} \int \mathbb {1} _ {B _ {r}} \mathcal {S} _ {p, q} h \mathrm {d} p = \int \mathbb {1} _ {B _ {r}} \frac {q}{p} [ \nabla \cdot h + h \cdot \nabla \log q ] \mathrm {d} p \\ = \int_ {B _ {r}} [ q \nabla \cdot h + h \cdot \nabla q ] d x \\ = \int_ {B _ {r}} \nabla \cdot (q h) d x \\ = \oint_ {S _ {r}} q h \cdot n \mathrm {d} x \leq t (r) \times \frac {2 \pi^ {d / 2}}{\Gamma (d / 2)} r ^ {d - 1} \stackrel {r \rightarrow \infty} {\rightarrow} 0, \\ \end{array}
$$
Proof of Proposition 2. Since  $k$  and its first derivatives are bounded, we can set
$$
C _ {0} ^ {k} := \sup  _ {x \in \mathbb {R} ^ {d}} \sqrt {k (x , x)}, \qquad C _ {1} ^ {k} := \sup  _ {x \in \mathbb {R} ^ {d}} \sqrt {\sum_ {i = 1} ^ {d} (\partial_ {i} \otimes \partial_ {i}) k (x , x)}.
$$
From Cauchy-Schwarz and the reproducing property, for any  $f \in \mathcal{H}(k)$  it holds that  $|f(x)| = |\langle f, k(\cdot, x) \rangle_{\mathcal{H}(k)}| \leq \| f \|_{\mathcal{H}(k)} \| k(\cdot, x) \|_{\mathcal{H}(k)} = \| f \|_{\mathcal{H}(k)} \sqrt{\langle k(\cdot, x), k(\cdot, x) \rangle_{\mathcal{H}(k)}} = \| f \|_{\mathcal{H}(k)} \sqrt{k(x, x)}$ . Furthermore, using the fact that  $k$  is continuously differentiable, it can be shown that  $|\partial_i f(x)| \leq \| f \|_{\mathcal{H}(k)} \sqrt{(\partial_i \otimes \partial_i) k(x, x)}$ ; see Corollary 4.36 of Steinwart and Christmann [2008]. As a consequence, for all  $h \in \mathcal{H}(k)^d$  we have that, for all  $x \in \mathbb{R}^d$ ,