SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Stein  $\Pi$ -Importance Sampling
Congye Wang<sup>1</sup>, Wilson Ye Chen<sup>2</sup>, Heishiro Kanagawa<sup>1</sup>, Chris. J. Oates<sup>1</sup>
1 Newcastle University, UK
$^{2}$  University of Sydney, Australia
Stein discrepancies have emerged as a powerful tool for retrospective improvement of Markov chain Monte Carlo output. However, the question of how to design Markov chains that are well-suited to such post-processing has yet to be addressed. This paper studies Stein importance sampling, in which weights are assigned to the states visited by a  $\Pi$ -invariant Markov chain to obtain a consistent approximation of  $P$ , the intended target. Surprisingly, the optimal choice of  $\Pi$  is not identical to the target  $P$ ; we therefore propose an explicit construction for  $\Pi$  based on a novel variational argument. Explicit conditions for convergence of Stein  $\Pi$ -Importance Sampling are established. For  $\approx 70\%$  of tasks in the PosteriorDB benchmark, a significant improvement over the analogous post-processing of  $P$ -invariant Markov chains is reported.
Stein discrepancies are a class of statistical divergences that can be computed without access to a normalisation constant. Originally conceived as a tool to measure the performance of sampling methods (Gorham and Mackey, 2015), these discrepancies have since found wide-ranging statistical applications (see the review of Anastasiou et al., 2023). Our focus here is the use of Stein discrepancies for retrospective improvement of Markov chain Monte Carlo (MCMC), and here two main techniques have been proposed: (i) Stein importance sampling (Liu and Lee, 2017; Hodgkinson et al., 2020), and (ii) Stein thinning (Riabiz et al., 2022). In Stein importance sampling (also called black box importance sampling), the samples are assigned weights such that a Stein discrepancy between the weighted empirical measure and the target  $P$  is minimised. Stein thinning constructs a sparse approximation to this optimally weighted measure at a lower computational and storage cost. Together, these techniques provide a powerful set of post-processing tools for MCMC, with subsequent authors proposing a range of generalisations and extensions (Teymur et al., 2021; Chopin and Ducrocq, 2021; Hawkins et al., 2022; Fisher and Oates, 2023; Bénard et al., 2023).
The consistency of these algorithms has been established in the setting of approximate, II-invariant MCMC, motivated by challenging inference problems where only approximate sampling can be performed. In these settings,  $\Pi$  is implicitly an approximation to  $P$  that is as accurate as possible subject to computational budget. However, the critical question of how to design Markov chains that are well-suited to such post-processing has yet to be addressed. This paper provides a solution, in the form of a specific construction for  $\Pi$  derived from a novel variational argument. Surprisingly, we are able to demonstrate a substantial improvement using the proposed  $\Pi$ , compared to the case where  $\Pi$  and  $P$  are equal. The paper proceeds as follows: Section 2 presents an abstract formulation of the task and existing results for optimally-weighted empirical measures are reviewed. Section 3 derives our proposed choice of  $\Pi$  and establishes that Stein post-processing of samples from a II-invariant Metropolis-adjusted Langevin algorithm (MALA) provides a consistent approximation of  $P$ . The approach is stress-tested using the recently released PosteriorDB suite of benchmark tasks in Section 4, before concluding with a discussion in Section 5.
To properly contextualise our discussion we start with an abstract mathematical description of the task. Let  $P$  be a probability measure on a measurable space  $\mathcal{X}$ . Let  $\mathcal{P}(\mathcal{X})$  be the set of all probability measures on  $\mathcal{X}$ . Let  $D_P: \mathcal{P}(\mathcal{X}) \to [0, \infty]$  be a statistical divergence for measuring the quality of an approximation  $Q$  to  $P$ , meaning that  $D_P(Q) = 0$  if and only if  $Q = P$ . In this work we consider approximations whose support is contained in a finite set  $\{x_1, \ldots, x_n\} \subset \mathcal{X}$ , and in particular we consider optimal approximations of the form
$$
P _ {n} ^ {\star} = \sum_ {i = 1} ^ {n} w _ {i} ^ {\star} \delta (x _ {i}), \qquad w ^ {\star} \in \underset {w \geq 0,   1 ^ {\top} w = 1} {\arg \min } D _ {P} \left(\sum_ {i = 1} ^ {n} w _ {i} \delta (x _ {i})\right).
$$
In what follows we restrict attention to statistical divergences for which such approximations can be shown to exist and be well-defined. The question that we then ask is which states  $\{x_{1},\ldots ,x_{n}\}$  minimise the approximation error  $D_P(P_n^\star)$ ? Before specialising to Stein discrepancies, it is helpful to review existing results for some standard statistical divergences  $D_P$ .
2.1 Wasserstein Divergence
Optimal quantisation focuses on the  $r$ -Wasserstein  $(r \geq 1)$  family of statistical divergences  $D_P(Q) = \inf_{\gamma \in \Gamma(P, Q)} \int \|x - y\|^r \mathrm{d}\gamma(x, y)$ , where  $\Gamma(P, Q)$  denotes the set of all couplings<sup>1</sup> of  $P, Q \in \mathcal{P}(\mathbb{R}^d)$ , and the divergence is finite whenever  $P$  and  $Q$  have finite  $r$ -th moment. Assuming the states  $\{x_1, \ldots, x_n\}$  are distinct, the corresponding optimal weights are  $w_i^\star = P(A_i)$  where  $A_i$  is the Voronoi neighbourhood<sup>2</sup> of  $x_i$  in  $\mathbb{R}^d$ . Optimal states achieve the minimal quantisation error for  $P$ ;
$$
e _ {n, r} (P) = \inf  _ {x _ {1}, \dots , x _ {n} \in \mathbb {R} ^ {d}} D _ {P} \left(\sum_ {i = 1} ^ {n} w _ {i} ^ {\star} \delta \left(x _ {i}\right)\right),
$$
the smallest value of the divergence among optimally-weighted distributions supported on at most  $n$  states. Though the dependence of optimal states on  $n$  and  $P$  can be complicated, we can broaden our perspective to consider asymptotically optimal states, whose asymptotic properties can be precisely characterised. To this end, for  $A \subset \mathbb{R}^d$ , let  $\mathcal{U}(A)$  denote the uniform distribution on  $A$ , and define the universal constant  $C_r([0,1]^d) = \inf_{n\geq 1}n^{r / d}e_{n,r}(\mathcal{U}([0,1]^d))$ . Suppose that  $P$  admits a density  $p$  on  $\mathbb{R}^d$ . Then the  $r$ th quantisation coefficient of  $P$  on  $\mathbb{R}^d$ , defined as
$$
C _ {r} (P) = C _ {r} ([ 0, 1 ] ^ {d}) \left(\int p (x) ^ {d / (d + r)} \mathrm {d} x\right) ^ {(d + r) / d},
$$
plays a central role in the classical theory of quantisation, being the rate constant in the asymptotic convergence of the minimal quantisation error;  $\lim_{n\to \infty}n^{r / d}e_{n,r}(P) = C_r(P)$ ; see Theorem 6.2 of Graf and Luschgy (2007). This suggests a natural definition; a collection  $\{x_1,\ldots ,x_n\}$  is called asymptotically optimal if
$$
\lim  _ {n \to \infty} n ^ {r / d} D _ {P} \left(\sum_ {i = 1} ^ {n} w _ {i} ^ {\star} \delta (x _ {i})\right) = C _ {r} (P),
$$
which amounts to  $P_{n}^{\star}$  asymptotically attaining the minimal quantisation error  $e_{n,r}(P)$ . The main result here is that, if  $\{x_1,\ldots ,x_n\}$  are asymptotically optimal, then  $\frac{1}{n}\sum_{i = 1}^{n}\delta (x_i)\to \Pi_r$ , where convergence is in distribution and  $\Pi_r$  is the distribution whose density is  $\pi_r(x)\propto p(x)^{d / (d + r)}$ ; see Theorem 7.5 of Graf and Luschgy (2007). This provides us with a key insight; optimal states are over-dispersed with respect to the intended distributional target. The extent of the over-dispersion here depends both on  $r$ , a parameter of the statistical divergence, and the dimension  $d$  of the space on which distributions are defined.
The  $r$ -Wasserstein divergence is, unfortunately, not well-suited for use in the motivating Bayesian context. In particular, computing the optimal weights  $w_{i} = P(A_{i})$  requires knowledge of  $P$ , which is typically not available when  $P$  is implicitly defined via an intractable normalisation constant. On
${}^{1}$  A coupling  $\gamma  \in  \Gamma \left( {P,Q}\right)$  is a distribution  $\gamma  \in  \mathcal{P}\left( {{\mathbb{R}}^{d} \times  {\mathbb{R}}^{d}}\right)$  whose marginal distributions are  $P$  and  $Q$  .
2The Voronoi neighbourhood of  $x_{i}$  is the set  $A_{i} = \{x\in \mathbb{R}^{d}:||x - x_{i}|| = \min_{j = 1,\dots ,n}||x - x_{j}||\}$
the other hand, the optimal sampling distribution  $\Pi$  is explicit and can be sampled (for example using MCMC); for discussion of random quantisers in this context see Graf and Luschgy (2007, Chapter 9), Cohort (2004, p126) and Sonnleitner (2022, Section 4.5). The simple form of  $\Pi$  is a feature of the classical approach to quantisation that we will attempt to mimic in the sequel.
2.2 Kernel Discrepancies
The theory of quantisation using kernels is less well-developed. A kernel is a measurable, symmetric, positive-definite function  $k: \mathcal{X} \times \mathcal{X} \to \mathbb{R}$ . From the Moore-Aronszajn theorem, there is a unique Hilbert space  $\mathcal{H}(k)$  for which  $k$  is a reproducing kernel, meaning that  $k(\cdot, x) \in \mathcal{H}(k)$  for all  $x \in \mathcal{X}$  and  $\langle f, k(\cdot, x) \rangle_{\mathcal{H}(k)} = f(x)$  for all  $f \in \mathcal{H}(k)$  and all  $x \in \mathcal{X}$ . Assuming that  $\mathcal{H}(k) \subset L^1(P)$ , we can define the weak (or Pettis) integral
$$
\mu_ {P} (\cdot) = \int k (\cdot , x) \mathrm {d} P (x), \tag {1}
$$
called the kernel mean embedding of  $P$  in  $\mathcal{H}(k)$ . The kernel discrepancy is then defined as the norm of the difference between kernel mean embeddings
$$
D _ {P} (Q) = \| \mu_ {Q} - \mu_ {P} \| _ {\mathcal {H} (k)} = \sqrt {\iint k (x , y) \mathrm {d} (Q - P) (x) \mathrm {d} (Q - P) (y)} \tag {2}
$$
where, to be consistent with our earlier notation, we adopt the convention that  $D_P(Q)$  is infinite whenever  $\mathcal{H}(k) \not\subset L^1(Q)$ . The second equality in (2) follows immediately from the stated properties of a reproducing kernel. To satisfy the requirement of a statistical divergence, we assume that the kernel  $k$  is characteristic, meaning that  $\mu_P = \mu_Q$  if and only if  $P = Q$ . In this setting, the properties of optimal states are necessarily dependent on the choice of kernel  $k$ , and are in general not well-understood. Indeed, given distinct states  $\{x_1, \ldots, x_n\}$ , the corresponding optimal weights  $w^\star = (w_1^\star, \ldots, w_n^\star)^\top$  are the solution to the linearly-constrained quadratic program
$$
\underset {w \in \mathbb {R} ^ {d}} {\arg \min } w ^ {\top} K w - 2 z ^ {\top} w \quad \text {s . t .} \quad w \geq 0, 1 ^ {\top} w = 1 \tag {3}
$$
where  $K_{i,j} = k(x_i, x_j)$  and  $z_i = \mu_P(x_i)$ . This program does not admit a closed-form solution, but can be numerically solved. To the best of our knowledge, the only theoretical analysis of approximations based on (3) is due to Hayakawa et al. (2022), who established rates for the convergence of  $P_n^\star$  to  $P$  in the case where states are independently sampled from  $P$ . The question of an optimal sampling distribution was not considered in that work.
Although few results are available concerning (3), relaxations of this program have been well-studied. The simplest relaxation of (3) is to remove both the positivity  $(w\geq 0)$  and normalisation  $(1^{\top}w = 1)$  constraints, in which case the optimal weights have the explicit representation  $w^{*} = K^{-1}z$ . The analysis of optimal states in this context has developed under the dual strands of kernel cubature and Bayesian cubature, where it has been theoretically or empirically demonstrated that (i) if states are randomly sampled, the optimal sampling distribution will be  $n$ -dependent (Bach, 2017) and over-dispersed with respect to the distributional target (Briol et al., 2017), and (ii) space-filling designs are asymptotically optimal for typical stationary kernels on bounded domains  $\mathcal{X}\subset \mathbb{R}^d$  (Briol et al., 2019). Analysis of optimal states on unbounded domains appears to be more difficult; see e.g. Karvonen et al. (2021). Relaxation of either the positivity or normalisation constraints results in approximations that behave similarly to kernel cubature (see, respectively, Ehler et al., 2019; Karvonen et al., 2018). However, relaxation of either constraint can result in the failure of  $P_{n}^{\star}$  to be an element of  $\mathcal{P}(\mathcal{X})$ , limiting the relevance of these results to the posterior approximation task.
Despite relatively little being known about the character of optimal states in this context, kernel discrepancy is widely used. The application of kernel discrepancies to an implicitly defined distributional target, such as a posterior distribution in a Bayesian analysis, is made possible by the use of a Stein kernel; a  $P$ -dependent kernel  $k = k_{P}$  for which  $\mu_{P}(x) = 0$  for all  $x \in \mathcal{X}$  (Oates et al., 2017). The associated kernel discrepancy
$$
D _ {P} (Q) = \| \mu_ {Q} \| _ {\mathcal {H} \left(k _ {P}\right)} = \sqrt {\iint k _ {P} (x , y) \mathrm {d} Q (x) \mathrm {d} Q (y)} \tag {4}
$$
is called a kernel Stein discrepancy (KSD) (Chwialkowski et al., 2016; Liu et al., 2016; Gorham and Mackey, 2017), and this will be a key tool in our methodological development. The corresponding optimally weighted approximation  $P_{n}^{\star}$  is the Stein importance sampling method of Liu and Lee (2017). To retain clarity of presentation in the main text, we defer all details on the construction of Stein kernels to Appendix A.
2.3 Sparse Approximation
If the number  $n$  of states is large, computation of optimal weights can become impractical. This has motivated a range of sparse approximation techniques, which aim to iteratively construct an approximation of the form  $P_{n,m} = \frac{1}{m}\sum_{i = 1}^{m}\delta (y_i)$ , where each  $y_{i}$  is an element from  $\{x_{1},\ldots ,x_{n}\}$ . The canonical example is the greedy algorithm which, at iteration  $j$ , selects a state
$$
y _ {j} \in \underset {y \in \{x _ {1}, \dots , x _ {n} \}} {\arg \min } D _ {P} \left(\frac {1}{j} \delta (y) + \frac {1}{j} \sum_ {i = 1} ^ {j - 1} \delta \left(y _ {i}\right)\right) \tag {5}
$$
for which the statistical divergence is minimised. In the context of kernel discrepancy, the greedy algorithm (5) has computational cost  $O(m^{2}n)$ , which compares favourably with the cost of solving (3) when  $m \ll n$ . Furthermore, under appropriate assumptions, the sparse approximation converges to the optimally weighted approximation;  $D_{P}(P_{n,m}) \to D_{P}(P_{n}^{\star})$  as  $m \to \infty$  with  $n$  fixed. See Teymur et al. (2021) for full details, where non-myopic and mini-batch extensions of the greedy algorithm are also considered. The greedy algorithm can be viewed as a regularised version of the Frank-Wolfe algorithm (also called herding, or the conditional gradient method), for which a similar asymptotic result can be shown to hold (Chen et al., 2010; Bach et al., 2012; Chen et al., 2018). Related work includes Dwivedi and Mackey (2021, 2022); Shetty et al. (2022); Hayakawa et al. (2022). Since in what follows we aim to retrospectively improve MCMC output, where it is not unusual to encounter  $n \approx 10^{4} - 10^{6}$ , sparse approximation will be important.
This completes our overview of background material. In what follows we seek to mimic classical quantisation by deriving a choice for  $\Pi$  that is straight-forward to sample using MCMC and is appropriately over-dispersed relative to  $P$ . This should be achieved while remaining in the framework of kernel discrepancies, so that optimal weights can be explicitly computed, and coupled with a sparse approximation that has low computational and storage cost.
The methods that we consider first sample states  $\{x_{1},\ldots ,x_{n}\}$  using II-invariant MCMC, then postprocess these states using kernel discrepancies (Section 2.2) and sparse approximation (Section 2.3), to obtain an approximation to the target  $P$ . A variational argument, which we present in Section 3.1, provides a suitable  $n$ -independent choice for II (which agrees with our intuition from Section 2.1 that II should be in some appropriate sense over-dispersed with respect to  $P$ ). Sufficient conditions for strong consistency of the approximation are established in Section 3.3.
3.1 Selecting II
Here we present a heuristic argument for a particular choice of  $\Pi$ ; rigorous theoretical support for Stein  $\Pi$ -Importance Sampling is then provided in Section 3.3. Our setting is that of Section 2.2, and the following will additionally be assumed:
Assumption 1. It is assumed that
(A1)  $C_1^2 \coloneqq \inf_{x \in \mathcal{X}} k(x, x) > 0$
(A2)  $C_2 \coloneqq \int \sqrt{k(x, x)} \, \mathrm{d}P(x) < \infty$ .
<sup>3</sup>It is difficult to quantify the complexity of numerically solving (3), since this will depend on details of the solver and the tolerance that are used. On the other hand, if we ignore the non-negativity and normalisation constraints, then we can see that the computational cost of solving the  $n$ -dimensional linear system of equations is  $O(n^{3})$ .
Note that (A2) implies that  $\mathcal{H}(k)\subset L^1 (P)$ , and thus (1) is in fact a strong (or Bochner) integral.
A direct analysis of the optimal states associated to the optimal weights  $w^{\star}$  appears to be challenging due to the fact that the components of  $w^{\star}$  are strongly inter-dependent. Our solution here is to instead consider optimal states associated with weights that, while not optimal, can be expected to perform much better than alternatives, with the advantage that their components are only weakly dependent. Specifically, we will be assuming that  $P$  is absolutely continuous with respect to  $\Pi$  (denoted  $P \ll \Pi$ ), and study convergence of self-normalised importance sampling (SNIS), i.e. the approximation  $P_{n} = \sum_{i=1}^{n} w_{i} \delta(x_{i})$ ,  $w_{i} \propto (\mathrm{d}P / \mathrm{d}\Pi)(x_{i})$ , where  $x_{1}, \ldots, x_{n} \sim \Pi$  are independent. Since  $w \geq 0$  and  $1^{-1}w = 1$ , from the optimality of  $w^{\star}$  under these constraints we have that  $D_{P}(P_{n}^{\star}) \leq D_{P}(\overline{P}_{n})$ . It is emphasised that the SNIS weights are a theoretical device only, and will not be used for computation; indeed, we can demonstrate that the SNIS weights  $w$  perform substantially worse than  $w^{\star}$  in general.
The analysis of SNIS weights  $w$  is tractable when viewed as approximation of the kernel mean embedding  $\mu_P$  in the Hilbert space  $\mathcal{H}(k)$ . Indeed, recall that  $D_P(P_n) = \| \xi_n / \sqrt{n} \|_{\mathcal{H}(k)}$  where  $\xi_n = \sqrt{n} (\mu_{P_n} - \mu_P)$ . Then, following Section 2.3.1 of Agapiou et al. (2017), we observe that
$$
\xi_ {n} = \sqrt {n} \left(\sum_ {i = 1} ^ {n} w _ {i} k (\cdot , x _ {i}) - \mu_ {P}\right) = \frac {\frac {1}{\sqrt {n}} \sum_ {i = 1} ^ {n} \frac {\mathrm {d} P}{\mathrm {d} \Pi} \left(x _ {i}\right) \left[ k (\cdot , x _ {i}) - \mu_ {P} \right]}{\frac {1}{n} \sum_ {i = 1} ^ {n} \frac {\mathrm {d} P}{\mathrm {d} \Pi} \left(x _ {i}\right)}. \tag {6}
$$
The idea is to seek  $\Pi$  for which the asymptotic variance of  $\xi_{n}$  is small. Supposing that
$$
\int \frac {\mathrm {d} P}{\mathrm {d} \Pi} (x) ^ {2} \mathrm {d} \Pi (x) <   \infty , \tag {S1}
$$
from the weak law of large numbers the denominator in (6) converges in probability to 1. Further supposing that
$$
\int \left\| \frac {\mathrm {d} P}{\mathrm {d} \Pi} (x) [ k (\cdot , x) - \mu_ {P} ] \right\| _ {\mathcal {H} (k)} ^ {2} \mathrm {d} \Pi (x) <   \infty , \tag {S2}
$$
from the Hilbert space central limit theorem the numerator in (6) converges in distribution to a Gaussian  $\frac{1}{\sqrt{n}}\sum_{i=1}^{n}(\mathrm{d}P/\mathrm{d}\Pi)(x_i)[k(\cdot,x_i)-\mu_P] \stackrel{\mathrm{d}}{\to}\mathcal{N}(0,\mathcal{C})$  where  $\mathcal{C}:\mathcal{H}(k)\to \mathcal{H}(k)$  is the covariance operator defined via
$$
\langle f, \mathcal {C} g \rangle_ {\mathcal {H} (k)} = \int \left\langle f, \frac {\mathrm {d} P}{\mathrm {d} \Pi} (x) [ k (\cdot , x) - \mu_ {P} ] \right\rangle_ {\mathcal {H} (k)} \left\langle g, \frac {\mathrm {d} P}{\mathrm {d} \Pi} (x) [ k (\cdot , x) - \mu_ {P} ] \right\rangle_ {\mathcal {H} (k)} \mathrm {d} \Pi (x),
$$
see Section 10.1 of Ledoux and Talagrand (1991). Thus, from Slutsky's lemma applied to (6), we conclude that  $\xi_{n}\stackrel {\mathrm{d}}{\to}\mathcal{N}(0,\mathcal{C})$ . Recalling that  $nD_P(P_n)^2 = \| \xi_n\|_{\mathcal{H}(k)}^2$ , and noting that the mean square of the limiting Gaussian random variable is  $\operatorname {tr}(\mathcal{C})$ , a natural idea is to select the sampling distribution  $\Pi$  such that  $\operatorname {tr}(\mathcal{C})$  is minimised.
Fortunately, the trace of  $\mathcal{C}$  can be explicitly computed. It simplifies presentation to restrict attention to a Stein kernel  $k = k_{P}$ , for which  $\mu_{P} = 0$ , giving  $\operatorname{tr}(\mathcal{C}) = \int (\mathrm{d}P / \mathrm{d}\Pi)(x)^{2}k_{P}(x)\mathrm{d}\Pi (x)$ , where for convenience we have let  $k_{P}(x)\coloneqq k_{P}(x,x)$ . Assuming that  $P$  and  $\Pi$  admit densities  $p$  and  $\pi$  on  $\mathcal{X} = \mathbb{R}^d$ , the variational problem we wish to solve is
$$
\underset {\pi \in \mathcal {Q}} {\arg \min } \int \frac {p (x) ^ {2}}{\pi (x)} k _ {P} (x)   \mathrm {d} x \quad \text {s . t .} \quad \int \pi (x)   \mathrm {d} x = 1, \tag {7}
$$
where  $\mathcal{Q}$  be the set of positive measures on  $\mathbb{R}^d$  for which (S1-2) are satisfied. To solve this problem, we first relax the constraints (S1-2) and solve the relaxed problem using the Euler-Lagrange equations, which yield
$$
\pi (x) \propto p (x) \sqrt {k _ {P} (x)}. \tag {8}
$$
Note that the normalisation constant of  $\pi$  is  $C_2$  from (1), whose existence we assumed. Then we verify that (S1-2) in fact hold for this choice of  $\Pi$ . Indeed,
$$
\begin{array}{l} (\mathbb {S} 1) = \int \frac {\mathrm {d} P}{\mathrm {d} \Pi} (x) ^ {2} \mathrm {d} \Pi (x) = C _ {2} \int \frac {1}{k _ {P} (x)} \mathrm {d} \Pi (x) \leq \frac {C _ {2}}{C _ {1} ^ {2}} <   \infty \\ (\mathrm {S 2}) = \int \frac {\mathrm {d} P}{\mathrm {d} \Pi} (x) ^ {2} k _ {P} (x) \mathrm {d} \Pi (x) = C _ {2} \int \sqrt {k _ {P} (x)} \mathrm {d} P (x) = C _ {2} ^ {2} <   \infty , \\ \end{array}
$$
(1.1)
(1.2)
Figure 1: Illustrating our choice of  $\Pi$  in 1D. (a) The univariate target  $P$  (black), and our choice of  $\Pi$  based on the Langevin-Stein kernel (purple), the KGM1-Stein kernel (green), and the KGM3-Stein kernel (blue). (b) The mean kernel Stein discrepancy (KSD) for Stein  $\Pi$ -Importance Sampling using the Stein kernels from (a); in each case, KSD was computed using the same Stein kernel used to construct  $\Pi$ . Solid lines indicate the baseline case of sampling from  $P$ , while dashed lines indicate sampling from  $\Pi$ . (The experiment was repeated 100 times and standard error bars are plotted.)
which shows that we have indeed solved (7). The sampling distribution  $\Pi$  we have obtained is characterised up to a normalisation constant in (8), so just like  $P$  we can sample from  $\Pi$  using techniques such as MCMC. It is interesting to note that  $\Pi$  is also optimal for standard importance sampling (i.e. without self-normalisation); see Lemma 1 of Adachi et al. (2022). The Stein kernel  $k_{P}$  determines the extent to which  $\Pi$  differs from  $P$ , as we illustrate next.
3.2 Illustration