Title: Stein $\Pi$-Importance Sampling
Venue: NeurIPS 2023
Accepted: yes
Keywords: Bayesian

discrepancy

kernel

sampling

Stein's method
Abstract: Stein discrepancies have emerged as a powerful tool for retrospective improvement of Markov chain Monte Carlo output.  However, the question of how to design Markov chains that are well-suited to such post-processing has yet to be addressed.  This paper studies Stein importance sampling, in which weights are assigned to the states visited by a $\Pi$-invariant Markov chain to obtain a consistent approximation of $P$, the intended target.  Surprisingly, the optimal choice of $\Pi$ is not identical to the target $P$; we therefore propose an explicit construction for $\Pi$ based on a novel variational argument.  Explicit conditions for convergence of Stein $\Pi$-Importance Sampling are established.  For $\approx 70$% of tasks in the PosteriorDB benchmark, a significant improvement over the analogous post-processing of $P$-invariant Markov chains is reported.

Stein Π -Importance Sampling

Abstract

Stein discrepancies have emerged as a powerful tool for retrospective improvement of Markov chain Monte Carlo output. However, the question of how to design Markov chains that are well-suited to such post-processing has yet to be addressed. This paper studies Stein importance sampling, in which weights are assigned to the states visited by a Π -invariant Markov chain to obtain a consistent approximation of P , the intended target. Surprisingly, the optimal choice of Π is not identical to the target P ; we therefore propose an explicit construction for Π based on a novel variational argument. Explicit conditions for convergence of Stein Π -Importance Sampling are established. For ≈ 70% of tasks in the PosteriorDB benchmark, a significant improvement over the analogous post-processing of P -invariant Markov chains is reported.

1 Introduction

Stein discrepancies are a class of statistical divergences that can be computed without access to a normalisation constant. Originally conceived as a tool to measure the performance of sampling methods (Gorham and Mackey, 2015), these discrepancies have since found wide-ranging statistical applications (see the review of Anastasiou et al., 2023). Our focus here is the use of Stein discrepancies for retrospective improvement of Markov chain Monte Carlo (MCMC), and here two main techniques have been proposed: (i) Stein importance sampling (Liu and Lee, 2017; Hodgkinson et al., 2020), and (ii) Stein thinning (Riabiz et al., 2022). In Stein importance sampling (also called black box importance sampling ), the samples are assigned weights such that a Stein discrepancy between the weighted empirical measure and the target P is minimised. Stein thinning constructs a sparse approximation to this optimally weighted measure at a lower computational and storage cost. Together, these techniques provide a powerful set of post-processing tools for MCMC, with subsequent authors proposing a range of generalisations and extensions (Teymur et al., 2021; Chopin and Ducrocq, 2021; Hawkins et al., 2022; Fisher and Oates, 2023; Bénard et al., 2023).

The consistency of these algorithms has been established in the setting of approximate, Π -invariant MCMC, motivated by challenging inference problems where only approximate sampling can be performed. In these settings, Π is implicitly an approximation to P that is as accurate as possible subject to computational budget. However, the critical question of how to design Markov chains that are well-suited to such post-processing has yet to be addressed. This paper provides a solution, in the form of a specific construction for Π derived from a novel variational argument. Surprisingly, we are able to demonstrate a substantial improvement using the proposed Π , compared to the case where Π and P are equal. The paper proceeds as follows: Section 2 presents an abstract formulation of the task and existing results for optimally-weighted empirical measures are reviewed. Section 3 derives our proposed choice of Π and establishes that Stein post-processing of samples from a Π -invariant Metropolis-adjusted Langevin algorithm (MALA) provides a consistent approximation of P . The approach is stress-tested using the recently released PosteriorDB suite of benchmark tasks in Section 4, before concluding with a discussion in Section 5.

2 Background

To properly contextualise our discussion we start with an abstract mathematical description of the task. Let P be a probability measure on a measurable space X . Let P ( X ) be the set of all probability measures on X . Let D P : P ( X ) → [0 , ∞ ] be a statistical divergence for measuring the quality of an approximation Q to P , meaning that D P ( Q ) = 0 if and only if Q = P . In this work we consider approximations whose support is contained in a finite set { x 1 , . . . , x n } ⊂ X , and in particular we consider optimal approximations of the form

In what follows we restrict attention to statistical divergences for which such approximations can be shown to exist and be well-defined. The question that we then ask is which states { x 1 , . . . , x n } minimise the approximation error D P ( P ⋆ n ) ? Before specialising to Stein discrepancies, it is helpful to review existing results for some standard statistical divergences D P .

2.1 Wasserstein Divergence

Optimal quantisation focuses on the r -Wasserstein ( r ≥ 1 ) family of statistical divergences D P ( Q ) = inf γ ∈ Γ( P,Q ) ∫ ∥ x -y ∥ r d γ ( x, y ) , where Γ( P, Q ) denotes the set of all couplings 1 of P, Q ∈ P ( R d ) , and the divergence is finite whenever P and Q have finite r -th moment. Assuming the states { x 1 , . . . , x n } are distinct, the corresponding optimal weights are w ⋆ i = P ( A i ) where A i is the Voronoi neighbourhood 2 of x i in R d . Optimal states achieve the minimal quantisation error for P ;

the smallest value of the divergence among optimally-weighted distributions supported on at most n states. Though the dependence of optimal states on n and P can be complicated, we can broaden our perspective to consider asymptotically optimal states, whose asymptotic properties can be precisely characterised. To this end, for A ⊂ R d , let U ( A ) denote the uniform distribution on A , and define the universal constant C r ([0 , 1] d ) = inf n ≥ 1 n r/d e n,r ( U ([0 , 1] d )) . Suppose that P admits a density p on R d . Then the r th quantisation coefficient of P on R d , defined as

plays a central role in the classical theory of quantisation, being the rate constant in the asymptotic convergence of the minimal quantisation error; lim n →∞ n r/d e n,r ( P ) = C r ( P ) ; see Theorem 6.2 of Graf and Luschgy (2007). This suggests a natural definition; a collection { x 1 , . . . , x n } is called asymptotically optimal if

which amounts to P ⋆ n asymptotically attaining the minimal quantisation error e n,r ( P ) . The main result here is that, if { x 1 , . . . , x n } are asymptotically optimal, then 1 n ∑ n i =1 δ ( x i ) → Π r , where convergence is in distribution and Π r is the distribution whose density is π r ( x ) ∝ p ( x ) d/ ( d + r ) ; see Theorem 7.5 of Graf and Luschgy (2007). This provides us with a key insight; optimal states are over-dispersed with respect to the intended distributional target. The extent of the over-dispersion here depends both on r , a parameter of the statistical divergence, and the dimension d of the space on which distributions are defined.

The r -Wasserstein divergence is, unfortunately, not well-suited for use in the motivating Bayesian context. In particular, computing the optimal weights w i = P ( A i ) requires knowledge of P , which is typically not available when P is implicitly defined via an intractable normalisation constant. On

the other hand, the optimal sampling distribution Π is explicit and can be sampled (for example using MCMC); for discussion of random quantisers in this context see Graf and Luschgy (2007, Chapter 9), Cohort (2004, p126) and Sonnleitner (2022, Section 4.5). The simple form of Π is a feature of the classical approach to quantisation that we will attempt to mimic in the sequel.

2.2 Kernel Discrepancies

The theory of quantisation using kernels is less well-developed. A kernel is a measurable, symmetric, positive-definite function k : X × X → R . From the Moore-Aronszajn theorem, there is a unique Hilbert space H ( k ) for which k is a reproducing kernel, meaning that k ( · , x ) ∈ H ( k ) for all x ∈ X and ⟨ f, k ( · , x ) ⟩ H ( k ) = f ( x ) for all f ∈ H ( k ) and all x ∈ X . Assuming that H ( k ) ⊂ L 1 ( P ) , we can define the weak (or Pettis ) integral

called the kernel mean embedding of P in H ( k ) . The kernel discrepancy is then defined as the norm of the difference between kernel mean embeddings

where, to be consistent with our earlier notation, we adopt the convention that D P ( Q ) is infinite whenever H ( k ) ̸⊂ L 1 ( Q ) . The second equality in (2) follows immediately from the stated properties of a reproducing kernel. To satisfy the requirement of a statistical divergence, we assume that the kernel k is characteristic , meaning that µ P = µ Q if and only if P = Q . In this setting, the properties of optimal states are necessarily dependent on the choice of kernel k , and are in general not well-understood. Indeed, given distinct states { x 1 , . . . , x n } , the corresponding optimal weights w ⋆ = ( w ⋆ 1 , . . . , w ⋆ n ) ⊤ are the solution to the linearly-constrained quadratic program

where K i,j = k ( x i , x j ) and z i = µ P ( x i ) . This program does not admit a closed-form solution, but can be numerically solved. To the best of our knowledge, the only theoretical analysis of approximations based on (3) is due to Hayakawa et al. (2022), who established rates for the convergence of P ⋆ n to P in the case where states are independently sampled from P . The question of an optimal sampling distribution was not considered in that work.

Although few results are available concerning (3), relaxations of this program have been well-studied. The simplest relaxation of (3) is to remove both the positivity ( w ≥ 0 ) and normalisation ( 1 ⊤ w = 1 ) constraints, in which case the optimal weights have the explicit representation w ∗ = K -1 z . The analysis of optimal states in this context has developed under the dual strands of kernel cubature and Bayesian cubature , where it has been theoretically or empirically demonstrated that (i) if states are randomly sampled, the optimal sampling distribution will be n -dependent (Bach, 2017) and over-dispersed with respect to the distributional target (Briol et al., 2017), and (ii) space-filling designs are asymptotically optimal for typical stationary kernels on bounded domains X ⊂ R d (Briol et al., 2019). Analysis of optimal states on unbounded domains appears to be more difficult; see e.g. Karvonen et al. (2021). Relaxation of either the positivity or normalisation constraints results in approximations that behave similarly to kernel cubature (see, respectively, Ehler et al., 2019; Karvonen et al., 2018). However, relaxation of either constraint can result in the failure of P ⋆ n to be an element of P ( X ) , limiting the relevance of these results to the posterior approximation task.

Despite relatively little being known about the character of optimal states in this context, kernel discrepancy is widely used. The application of kernel discrepancies to an implicitly defined distributional target, such as a posterior distribution in a Bayesian analysis, is made possible by the use of a Stein kernel ; a P -dependent kernel k = k P for which µ P ( x ) = 0 for all x ∈ X (Oates et al., 2017). The associated kernel discrepancy

is called a kernel Stein discrepancy (KSD) (Chwialkowski et al., 2016; Liu et al., 2016; Gorham and Mackey, 2017), and this will be a key tool in our methodological development. The corresponding optimally weighted approximation P ⋆ n is the Stein importance sampling method of Liu and Lee (2017). To retain clarity of presentation in the main text, we defer all details on the construction of Stein kernels to Appendix A.

2.3 Sparse Approximation

If the number n of states is large, computation of optimal weights can become impractical. This has motivated a range of sparse approximation techniques, which aim to iteratively construct an approximation of the form P n,m = 1 m ∑ m i =1 δ ( y i ) , where each y i is an element from { x 1 , . . . , x n } . The canonical example is the greedy algorithm which, at iteration j , selects a state

for which the statistical divergence is minimised. In the context of kernel discrepancy, the greedy algorithm (5) has computational cost O ( m 2 n ) , which compares favourably 3 with the cost of solving (3) when m ≪ n . Furthermore, under appropriate assumptions, the sparse approximation converges to the optimally weighted approximation; D P ( P n,m ) → D P ( P ⋆ n ) as m → ∞ with n fixed. See Teymur et al. (2021) for full details, where non-myopic and mini-batch extensions of the greedy algorithm are also considered. The greedy algorithm can be viewed as a regularised version of the Frank-Wolfe algorithm (also called herding , or the conditional gradient method), for which a similar asymptotic result can be shown to hold (Chen et al., 2010; Bach et al., 2012; Chen et al., 2018). Related work includes Dwivedi and Mackey (2021, 2022); Shetty et al. (2022); Hayakawa et al. (2022). Since in what follows we aim to retrospectively improve MCMC output, where it is not unusual to encounter n ≈ 10 4 -10 6 , sparse approximation will be important.

This completes our overview of background material. In what follows we seek to mimic classical quantisation by deriving a choice for Π that is straight-forward to sample using MCMC and is appropriately over-dispersed relative to P . This should be achieved while remaining in the framework of kernel discrepancies, so that optimal weights can be explicitly computed, and coupled with a sparse approximation that has low computational and storage cost.

3 Methodology

The methods that we consider first sample states { x 1 , . . . , x n } using Π -invariant MCMC, then postprocess these states using kernel discrepancies (Section 2.2) and sparse approximation (Section 2.3), to obtain an approximation to the target P . A variational argument, which we present in Section 3.1, provides a suitable n -independent choice for Π (which agrees with our intuition from Section 2.1 that Π should be in some appropriate sense over-dispersed with respect to P ). Sufficient conditions for strong consistency of the approximation are established in Section 3.3.

3.1 Selecting Π

Here we present a heuristic argument for a particular choice of Π ; rigorous theoretical support for Stein Π -Importance Sampling is then provided in Section 3.3. Our setting is that of Section 2.2, and the following will additionally be assumed:

Assumption 1. It is assumed that

Note that (A2) implies that H ( k ) ⊂ L 1 ( P ) , and thus (1) is in fact a strong (or Bochner ) integral.

A direct analysis of the optimal states associated to the optimal weights w ⋆ appears to be challenging due to the fact that the components of w ⋆ are strongly inter-dependent. Our solution here is to instead consider optimal states associated with weights that, while not optimal, can be expected to perform much better than alternatives, with the advantage that their components are only weakly dependent. Specifically, we will be assuming that P is absolutely continuous with respect to Π (denoted P ≪ Π ), and study convergence of self-normalised importance sampling (SNIS), i.e. the approximation P n = ∑ n i =1 w i δ ( x i ) , w i ∝ (d P/ dΠ)( x i ) , where x 1 , . . . , x n ∼ Π are independent. Since w ≥ 0 and 1 ⊤ w = 1 , from the optimality of w ⋆ under these constraints we have that D P ( P ⋆ n ) ≤ D P ( P n ) . It is emphasised that the SNIS weights are a theoretical device only, and will not be used for computation; indeed, we can demonstrate that the SNIS weights w perform substantially worse than w ⋆ in general.

The analysis of SNIS weights w is tractable when viewed as approximation of the kernel mean embedding µ P in the Hilbert space H ( k ) . Indeed, recall that D P ( P n ) = ∥ ξ n / √ n ∥ H ( k ) where ξ n = √ n ( µ P n -µ P ) . Then, following Section 2.3.1 of Agapiou et al. (2017), we observe that

The idea is to seek Π for which the asymptotic variance of ξ n is small. Supposing that

from the weak law of large numbers the denominator in (6) converges in probability to 1. Further supposing that

from the Hilbert space central limit theorem the numerator in (6) converges in distribution to a Gaussian 1 √ n ∑ n i =1 (d P/ dΠ)( x i ) [ k ( · , x i ) -µ P ] d →N (0 , C ) where C : H ( k ) → H ( k ) is the covariance operator defined via

see Section 10.1 of Ledoux and Talagrand (1991). Thus, from Slutsky's lemma applied to (6), we conclude that ξ n d →N (0 , C ) . Recalling that nD P ( P n ) 2 = ∥ ξ n ∥ 2 H ( k ) , and noting that the mean square of the limiting Gaussian random variable is tr ( C ) , a natural idea is to select the sampling distribution Π such that tr ( C ) is minimised.

Fortunately, the trace of C can be explicitly computed. It simplifies presentation to restrict attention to a Stein kernel k = k P , for which µ P = 0 , giving tr ( C ) = ∫ (d P/ dΠ)( x ) 2 k P ( x ) dΠ( x ) , where for convenience we have let k P ( x ) := k P ( x, x ) . Assuming that P and Π admit densities p and π on X = R d , the variational problem we wish to solve is where Q be the set of positive measures on R d for which (S1-2) are satisfied. To solve this problem, we first relax the constraints (S1-2) and solve the relaxed problem using the Euler-Lagrange equations, which yield

Note that the normalisation constant of π is C 2 from (1), whose existence we assumed. Then we verify that (S1-2) in fact hold for this choice of Π . Indeed,

(1.1)

(1.2)

which shows that we have indeed solved (7). The sampling distribution Π we have obtained is characterised up to a normalisation constant in (8), so just like P we can sample from Π using techniques such as MCMC. It is interesting to note that Π is also optimal for standard importance sampling (i.e. without self-normalisation); see Lemma 1 of Adachi et al. (2022). The Stein kernel k P determines the extent to which Π differs from P , as we illustrate next.

3.2 Illustration

For illustration, consider the univariate target P (black curve) in Figure 1.1, a 3-component Gaussian mixture model. Our recommended choice of Π in (8) is shown for both the Langevin -Stein kernel (purple curve) and the KGM s -Stein kernels with s ∈ { 1 , 3 } (green and blue curves). The Stein discrepancy corresponding to the Langevin-Stein kernel provides control over weak convergence (i.e. convergence of integrals of functions that are continuous and bounded), while the KGM s -Stein kernel provides additional control over the convergence of polynomial moments up to order s ; full details about the construction of Stein kernels are contained in Appendix A. The Langevin and KGM1-Stein kernels have k P ( x ) ≍ x 2 , while the KGM3-Stein kernel has k P ( x ) ≍ x 6 , in each case as | x |→∞ , and thus greater over-dispersion results from use of the KGM3-Stein kernel. This over-dispersion is less pronounced 4 in higher dimensions; see Appendix D.1.

To illustrate the performance of Stein Π -Importance Sampling, we generated a sequence ( x n ) n ∈ N of independent samples from Π . For each n ∈ { 1 , . . . , 100 } , the samples { x 1 , . . . , x n } were assigned optimal weights w ⋆ by solving (3), and the associated KSD was computed. As a baseline, we performed the same calculation using independent samples from P . Figure 1.2 indicates that, for both Stein kernels, substantial improvement results from the use of samples from Π compared to the use of samples from P . Interestingly, the KGM3-Stein kernel demonstrated a larger improvement compared to the Langevin-Stein kernel, suggesting that the choice of Π may be more critical in settings where KSD enjoys a stronger form of convergence control.

To illustrate a posterior approximation task, consider a simple regression model y i = f i ( x ) + ϵ i with f i ( x ) = x 1 (1 + t i x 2 ) , t i = i -5 , i = 1 , . . . , 10 , with ϵ i independent N (0 , 1) . The parameter x = ( x 1 , x 2 ) was assigned a prior N (0 , I ) . Data were simulated using x = (0 , 0) . The posterior distribution P is depicted in the leftmost panel of Figure 2, while our choice of Π corresponding to the Langevin (centre left), KGM3 (centre right) and Riemann -Stein kernels (right) are also displayed. For the Langevin and KGM3 kernels, the associated Π target their mass toward regions where P varies the most. The reason for this behaviour is clearly seen for the Langevin-Stein kernel since

P

IⅡI (KGM3)

I (Riemann)

3

3

3

3

2

2

2

2

1

1

1

1

0

0

0

0

-1

-1

-1

-1

-2

-2

-2

-2

-3

-3

-3

-3

-2

0

2

-2

0

2

-2

0

2

-2

0

2

Algorithm 1 Π -Invariant Metropolis-Adjusted Langevin Algorithm (MALA)

Require: x 0 (initial state), ϵ (step size), M (preconditioner matrix), n (chain length), k P (Stein kernel)

1: for i = 1 , . . . , n do

4:

5:

if log(

end for

Algorithm 2 Stein Π -Importance Sampling (S Π IS-MALA)

Require: { x 1 , . . . , x n } from Algorithm 1, k P (Stein kernel)

1: w ⋆ ∈ arg min w ∈ R d {⟨ w,K P w ⟩ : w ≥ 0 , 1 ⊤ w = 1 }

▷

[

K

P

▷ U

]

i,j

IID

i

∼ U

=

k

P

([0

(

x

i

,

1])

, x

j

)

k P ( x ) = c 1 + c 2 ∥∇ log p ( x ) ∥ 2 for some c 1 , c 2 > 0 ; see Appendix C for detail. The Riemann-Stein kernel can be viewed as a preconditioned form of the Langevin-Stein kernel which takes into account the geometric structure of P ; see Appendix A for full detail 5 . Results in Figure S2 demonstrate that Stein Π -Importance Sampling improves upon the default Stein importance sampling method (i.e. with Π and P equal) for all choices of kernel.

An additional illustration involving a GARCH model with d = 4 parameters is presented in Appendix D.4, where the effect of varying the order s of the KGM-Stein kernel is explored.

3.3 Theoretical Guarantees

The aim of this section is to establish when post-processing of Π -invariant MCMC produces a strongly consistent approximation of P , for our recommended choice of Π in (8). Our analysis focuses on MALA (Roberts and Stramer, 2002), leveraging the recent work of Durmus and Moulines (2022) to present explicit and verifiable conditions on P for our results to hold. In fact, we consider the more general preconditioned form of MALA, where the symmetric positive definite preconditioner matrix M is to be specified. Our results also allow for (optional) sparse approximation, to circumvent direct solution of (3) (c.f. Section 2.3). The resulting algorithms, which we call Stein Π -Importance Sampling (S Π IS-MALA) and Stein Π -Thinning (S Π T-MALA), are quite straight-forward and contained, respectively, in Algorithms 2 and 3. The linearly-constrained quadratic programme in Algorithm 2 was solved using the Python v3.10.4 packages qpsolvers v3.4.0 and proxsuite v0.3.7 . While it is difficult to analyse the computational complexity associated with these methods, we believe they are at worst O ( n 3 ) .

U

)

< L

then

x

x

′

;

else

x

x

;

end if

i

i

i

-

i

1

Let A ⪯ B indicate that A -B is a positive semi-definite matrix for A,B ∈ R n × n . For a symmetric positive definite matrix A let ∥ z ∥ A := √ z ⊤ A -1 z for z ∈ R d . Let C s ( R d ) denote the set of s -times continuously differentiable real-valued functions on R d .

Theorem 1 (Strong consistency of S Π IS- and S Π T-MALA) . Let Assumption 1 hold where k = k P is a Stein kernel, and let D P : P ( X ) → [0 , ∞ ] denote the associated KSD. Assume also that

(A1) ∇ log p ∈ C 2 ( R d ) with sup x ∈ R d ∥∇ 2 log p ( x ) ∥ < ∞

(A2) ∃ b 1 > 0 , B 1 ≥ 0 such that -∇ 2 log p ( x ) ⪰ b 1 I for all ∥ x ∥ ≥ B 1

(A3) k P ∈ C 2 ( R d ) .

(A4) ∃ 0 < b 2 < 2 b 1 C 2 1 , B 2 ≥ 0 such that ∇ 2 k P ( x ) ⪯ b 2 I for all ∥ x ∥ ≥ B 2

Let P ⋆ n = ∑ n i =1 w ⋆ i δ ( x i ) be the result of running Algorithm 2 and let P n,m = 1 m ∑ m i =1 δ ( y i ) be the result of running Algorithm 3. Let m ≤ n and m = Ω((log n ) δ ) for some δ > 2 . Then there exists ϵ 0 > 0 such that, for all step sizes ϵ ∈ (0 , ϵ 0 ) and all initial states x 0 ∈ R d , D P ( P ⋆ n ) → 0 , D P ( P n,m ) → 0 almost surely as m,n →∞ .

The proof is in Appendix B. Compared to earlier authors 6 , such as Chen et al. (2019); Riabiz et al. (2022), a major novelty here is that our assumptions are explicit and can often be verified (see also Hodgkinson et al., 2020). (A2) is strong log-concavity of P when B 1 = 0 , while for B 1 > 0 this condition is slightly stronger than the related distant dissipativity condition assumed in earlier work (Gorham and Mackey, 2017; Riabiz et al., 2022). (A4) holds for the Langevin-Stein kernel (i.e. weak convergence control) and for the KGM1-Stein kernel (i.e. weak convergence control + control over first moments), but not for the higher-order KGM-Stein kernels. Extending our proof strategy to the higher-order KGM-Stein kernels would require further research into the convergence properties of MALA, and this is expected to be difficult.

4 Benchmarking on PosteriorDB

The area of Bayesian computation has historically lacked a common set of benchmark problems, with classical examples being insufficiently difficult and case-studies being hand-picked (Chopin and Ridgway, 2017). To introduce objectivity into our assessment, we exploited the recently released PosteriorDB benchmark (Magnusson et al., 2022). This project is an attempt toward standardised benchmarking, consisting of a collection of posteriors to be numerically approximated. Here, we systematically compared the performance of S Π IS-MALA against the default Stein importance sampling algorithm (i.e. Π = P ; denoted SIS-MALA), and also against unprocessed P -invariant MALA (i.e. uniform weights), reporting results across the breadth of PosteriorDB . The test problems in PosteriorDB are defined in the Stan probabilistic programming language, and so BridgeStan (Roualdes et al., 2023) was used to directly access posterior densities and their gradients as required. For all instances of MALA, an adaptive algorithm was used to learn a suitable preconditioner matrix M during the warm-up period; see Appendix D.3. All experiments that we report can be reproduced using code available at https://github.com/congyewang/Stein-Pi-Importance-Sampling .

Results are reported in Table 1 for n = 3 × 10 3 samples from MALA. These focus on the LangevinStein kernel, for which our theory holds, and the KGM3-Stein kernel, for which it does not. There was a significant improvement of S Π IS-MALA over SIS-MALA in 73% of test problems for the Langevin-Stein kernel and in 65% of test problems for the KGM3-Stein kernel. Compared to

unprocessed MALA, a significant improvement occurred in 100% and 97% of cases, respectively for each kernel. However, the extent of improvement decreased when the dimension d of the target increased, supporting the intuition that we set out earlier and in Appendix D.1. An in-depth breakdown of results, including varying the number n of samples that were used, and the performance S Π T-MALA, can be found in Appendices D.5 and D.6.

If P and its gradients are cheap to evaluate, the computational cost of MALA is lower than that of SIS-MALA, and one could run more iterations of MALA for an equivalent computational cost. But for more complex P , the computational cost of all algorithms will be gated by the number of times P and its gradients need to be evaluated, making the direct comparison in Table 1 meaningful. Further, if we aim for a compressed representation of P , then some form of post-processing of MALA would be required, which would then entail an additional computational cost.

Our focus is on the development of algorithms for minimisation of KSDs; the properties of KSDs themselves are out of scope for this work 7 . Nonetheless, there is much interest in better understanding the properties of KSDs, and we therefore also report performance of S Π IS-MALA in terms of 1-Wasserstein divergence in Appendix D.7. The main contrast between these results and the results in Table 1 is that, being score-based, KSDs suffer from the blindness to mixing proportions phenomena which has previously been documented in Wenliang and Kanagawa (2021); Koehler et al. (2022); Liu et al. (2023). Caution should therefore be taken when using algorithms based on Stein discrepancies

in the context of posterior distributions with multiple high probability regions that are spatially separated. This is also a failure mode for MCMC algorithms such as MALA, and yet there are still many problems for which MALA has been successfully used.

The alternative choice Π 1 , with π 1 ( x ) ∝ p ( x ) d/ ( d +1) , which provides a generic form of overdispersion and is optimal for approximation in 1-Wasserstein divergence (c.f. Section 2.1), was also considered. Results in Appendix D.8 indicate that, while Π 1 yields an improvement compared to the baseline of using P itself, Π 1 may be less effective than our proposed Π when P is skewed.

5 Discussion

This paper presented Stein Π -Importance Sampling; an algorithm that is simple to implement, admits an end-to-end theoretical treatment, and achieves a significant improvement over existing postprocessing methods based on KSD. On the negative side, second order derivatives of the statistical model are required, and we are ultimately bound to the performance of the KSD on which Stein Π -Importance Sampling is based. Our analysis focused on MALA, but there is in principle no barrier to deriving sufficient conditions for consistent approximation that are applicable to other sampling algorithms, such as the unadjusted Langevin algorithm. Of course, it remains to be seen whether SIS-MALA or any of its variants will stand the test of time compared to continued development in MCMCmethodology, but we believe this line of research merits further investigation. For models for which access to second order derivatives is impractical, our methodology and theoretical analysis are directly applicable to gradient-free KSD (Fisher and Oates, 2023), and this would be an interesting direction for future work. Similarly, alternatives to KSD that are better-suited to high-dimensional P could be considered, such as the sliced KSD of Gong et al. (2021a,b).

Acknowledgements CW was supported by the China Scholarship Council. HK and CJO were supported by EP/W019590/1. The authors are grateful to François-Xavier Briol for feedback on an earlier draft of the manuscript, and to the anonymous Reviewers for their input.

References

Adachi, M., Hayakawa, S., Jørgensen, M., Oberhauser, H., and Osborne, M. A. (2022). Fast Bayesian inference with batch Bayesian quadrature via kernel recombination. In Proceedings of the 35th Conference on Neural Information Processing Systems .

Agapiou, S., Papaspiliopoulos, O., Sanz-Alonso, D., and Stuart, A. M. (2017). Importance sampling: Intrinsic dimension and computational cost. Statistical Science , 32(3):405-431.

Anastasiou, A., Barp, A., Briol, F.-X., Ebner, B., Gaunt, R. E., Ghaderinezhad, F., Gorham, J., Gretton, A., Ley, C., Liu, Q., Mackey, L., Oates, C. J., Reinert, G., and Swan, Y. (2023). Stein's method meets statistics: A review of some recent developments. Statistical Science , (38):120-139.

Bach, F. (2017). On the equivalence between kernel quadrature rules and random feature expansions. The Journal of Machine Learning Research , 18(1):714-751.

Bach, F., Lacoste-Julien, S., and Obozinski, G. (2012). On the equivalence between herding and conditional gradient algorithms. In Proceedings of the 29th International Conference on Machine Learning .

Barbour, A. D. (1988). Stein's method and poisson process convergence. Journal of Applied Probability , 25(A):175-184.

Barbour, A. D. (1990). Stein's method for diffusion approximations. Probability Theory and Related Fields , 84(3):297-322.

Barp, A., Oates, C. J., Porcu, E., and Girolami, M. (2022a). A Riemann-Stein kernel method. Bernoulli , 28(4):2181-2208.

Barp, A., Simon-Gabriel, C.-J., Girolami, M., and Mackey, L. (2022b). Targeted separation and convergence with kernel discrepancies. arXiv preprint arXiv:2209.12835 .

Bénard, C., Staber, B., and Da Veiga, S. (2023). Kernel stein discrepancy thinning: A theoretical perspective of pathologies and a practical fix with regularization. arXiv preprint arXiv:2301.13528 .

Briol, F.-X., Oates, C. J., Cockayne, J., Chen, W. Y., and Girolami, M. (2017). On the sampling problem for kernel quadrature. In Proceedings of the 34th International Conference on Machine Learning , pages 586-595.

Briol, F.-X., Oates, C. J., Girolami, M., Osborne, M. A., and Sejdinovic, D. (2019). Probabilistic integration: A role in statistical computation (with discussion and rejoinder). Statistical Science , 34(1):1-22.

Carmeli, C., De Vito, E., and Toigo, A. (2006). Vector valued reproducing kernel Hilbert spaces of integrable functions and mercer theorem. Analysis and Applications , 4(04):377-408.

Chen, W. Y., Barp, A., Briol, F.-X., Gorham, J., Girolami, M., Mackey, L., and Oates, C. J. (2019). Stein point Markov chain Monte Carlo. In Proceedings of the 36th International Conference on Machine Learning , pages 1011-1021.

Chen, W. Y., Mackey, L., Gorham, J., Briol, F.-X., and Oates, C. J. (2018). Stein points. In Proceedings of the 35th International Conference on Machine Learning , pages 844-853.

Chen, Y., Welling, M., and Smola, A. (2010). Super-samples from kernel herding. In Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence , pages 109-116.

Chopin, N. and Ducrocq, G. (2021). Fast compression of MCMC output. Entropy , 23(8):1017.

Chopin, N. and Ridgway, J. (2017). Leave pima indians alone: Binary regression as a benchmark for Bayesian computation. Statistical Science , 32(1):64-87.

Chwialkowski, K., Strathmann, H., and Gretton, A. (2016). A kernel test of goodness of fit. In Proceedings of the 33rd International Conference on Machine Learning , pages 2606-2615.

Cohort, P. (2004). Limit theorems for random normalized distortion. The Annals of Applied Probability , 14(1):118-143.

Durmus, A. and Moulines, É. (2022). On the geometric convergence for MALA under verifiable conditions. arXiv preprint arXiv:2201.01951 .

Dwivedi, R. and Mackey, L. (2021). Kernel thinning. In Proceedings of 34th Conference on Learning Theory , pages 1753-1753.

Dwivedi, R. and Mackey, L. (2022). Generalized kernel thinning. In Proceedings of the 10th International Conference on Learning Representations .

Ehler, M., Gräf, M., and Oates, C. J. (2019). Optimal Monte Carlo integration on closed manifolds. Statistics and Computing , 29(6):1203-1214.

Fisher, M. A. and Oates, C. J. (2023). Gradient-free kernel Stein discrepancy. In Proceedings of the 37th Conference on Neural Information Processing Systems .

Girolami, M. and Calderhead, B. (2011). Riemann manifold Langevin and Hamiltonian Monte Carlo methods. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 73(2):123-214.

Gong, W., Li, Y., and Hernández-Lobato, J. M. (2021a). Sliced kernelized Stein discrepancy. In Proceedings of the 9th International Conference on Learning Representations .

Gong, W., Zhang, K., Li, Y., and Hernández-Lobato, J. M. (2021b). Active slices for sliced Stein discrepancy. arXiv:2102.03159 .

Gorham, J., Duncan, A. B., Vollmer, S. J., and Mackey, L. (2019). Measuring sample quality with diffusions. The Annals of Applied Probability , 29(5):2884-2928.

Gorham, J. and Mackey, L. (2015). Measuring sample quality with Stein's method. In Proceedings of the 29th Conference on Neural Information Processing Systems , pages 226-234.

Gorham, J. and Mackey, L. (2017). Measuring sample quality with kernels. In Proceedings of the 34th International Conference on Machine Learning , pages 1292-1301.

Gotze, F. (1991). On the rate of convergence in the multivariate CLT. The Annals of Probability , pages 724-739.

Graf, S. and Luschgy, H. (2007). Foundations of Quantization for Probability Distributions . Springer.

Hawkins, C., Koppel, A., and Zhang, Z. (2022). Online, informative MCMC thinning with kernelized Stein discrepancy. arXiv preprint arXiv:2201.07130 .

Hayakawa, S., Oberhauser, H., and Lyons, T. (2022). Positively weighted kernel quadrature via subsampling. In Proceedings of the 35th Conference on Neural Information Processing Systems .

Hodgkinson, L., Salomone, R., and Roosta, F. (2020). The reproducing Stein kernel approach for post-hoc corrected sampling. arXiv preprint arXiv:2001.09266 .

Kanagawa, H., Gretton, A., and Mackey, L. (2022). Controlling moments with kernel Stein discrepancies. arXiv preprint arXiv:2211.05408 (v1) .

Karvonen, T., Oates, C. J., and Girolami, M. (2021). Integration in reproducing kernel Hilbert spaces of Gaussian kernels. Mathematics of Computation , 90(331):2209-2233.

Karvonen, T., Oates, C. J., and Sarkka, S. (2018). A Bayes-Sard cubature method. In Proceedings of the 32nd Conference on Neural Information Processing Systems .

Kent, J. (1978). Time-reversible diffusions. Advances in Applied Probability , 10(4):819-835.

Koehler, F., Heckett, A., and Risteski, A. (2022). Statistical efficiency of score matching: The view from isoperimetry. In Proceedings of the 36th Conference on Neural Information Processing Systems .

Ledoux, M. and Talagrand, M. (1991). Probability in Banach Spaces: Isoperimetry and Processes , volume 23. Springer Science & Business Media.

Liu, Q. and Lee, J. (2017). Black-box importance sampling. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics , pages 952-961.

Liu, Q., Lee, J., and Jordan, M. (2016). A kernelized Stein discrepancy for goodness-of-fit tests. In Proceedings of the 33rd International Conference on Machine Learning , pages 276-284.

Liu, X., Duncan, A., and Gandy, A. (2023). Using perturbation to improve goodness-of-fit tests based on kernelized Stein discrepancy. In Proceedings of the 40th International Conference on Machine Learning .

Magnusson, M., Bürkner, P., and Vehtari, A. (2022). PosteriorDB : A set of posteriors for Bayesian inference and probabilistic programming. https://github.com/stan-dev/posteriordb .

Meyn, S. P. and Tweedie, R. L. (2012). Markov Chains and Stochastic Stability . Springer Science & Business Media.

Oates, C. J., Girolami, M., and Chopin, N. (2017). Control functionals for Monte Carlo integration. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 79(3):695-718.

Riabiz, M., Chen, W., Cockayne, J., Swietach, P., Niederer, S. A., Mackey, L., and Oates, C. J. (2022). Optimal thinning of MCMC output. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 84(4):1059-1081.

Roberts, G. O. and Rosenthal, J. S. (1998). Optimal scaling of discrete approximations to Langevin diffusions. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 60(1):255268.

Roberts, G. O. and Stramer, O. (2002). Langevin diffusions and Metropolis-Hastings algorithms. Methodology and Computing in Applied Probability , 4:337-357.

Roualdes, E., Ward, B., Axen, S., and Carpenter, B. (2023). BridgeStan: Efficient in-memory access to Stan programs through Python, Julia, and R. https://github.com/roualdes/bridgestan .

Shetty, A., Dwivedi, R., and Mackey, L. (2022). Distribution compression in near-linear time. In Proceedings of the 10th International Conference on Learning Representations .

Sonnleitner, M. (2022). The Power of Random Information for Numerical Approximation and Integration . PhD thesis, University of Passau.

Teymur, O., Gorham, J., Riabiz, M., and Oates, C. J. (2021). Optimal quantisation of probability measures using maximum mean discrepancy. In Proceedings of the 24th International Conference on Artificial Intelligence and Statistics , pages 1027-1035.

Wenliang, L. K. and Kanagawa, H. (2021). Blindness of score-based methods to isolated components and mixing proportions. In Proceedings of 'Your Model is Wrong' @ the 35th Conference on Neural Information Processing Systems .

Appendices

These appendices contains supporting material for the paper Stein Π -Importance Sampling . The mathematical background on Stein kernels is contained in Appendix A. The proof of Theorem 1 is contained in Appendix B. For implementation of Stein Π -Importance Sampling without the aid of automatic differentiation, various explicit derivatives are required; the relevant calculations can be found in Appendix C. The empirical protocols and additional empirical results are presented in Appendix D.

A Mathematical Background

This appendix contains mathematical background on reproducing kernels and Stein kernels, as used in the main text. Appendix A.1 introduces matrix-valued reproducing kernels, while Appendix A.2 specialises to Stein kernels by application of a Stein operator to a matrix-valued kernel. A selection of useful Stein kernels are presented in Appendix A.3.

A.1 Matrix-Valued Reproducing Kernels

A matrix-valued kernel is a function K : R d × R d → R d × d , that is both

symmetric; K ( x, y ) = K ( y, x ) for all x, y ∈ R d , and

positive semi-definite; ∑ n i =1 ∑ n j =1 ⟨ c i , K ( x i , x j ) c j ⟩ ≥ 0 for all x 1 , . . . , x n ∈ R d and c 1 , . . . , c n ∈ R d .

Let K x = K ( · , x ) . For vector-valued functions g, g ′ : R d → R d , defined by g = ∑ n i =1 K x i c i and g ′ = ∑ m j =1 K x ′ j c ′ i , define an inner product

There is a unique Hilbert space of such vector-valued functions associated to K , denoted H ( K ) ; see Proposition 2.1 of Carmeli et al. (2006). This space is characterised as

where here the closure is taken with respect to the inner product in (9). It can be shown that H ( K ) is in fact a reproducing kernel Hilbert space (RKHS) which satisfies the reproducing property

for all g ∈ H ( K ) and x, c ∈ R d . Matrix-valued kernels are the natural starting point for construction of KSDs, as described next.

A.2 Stein Kernels

A general construction for Stein kernels is to first identify a matrix-valued RKHS H ( K ) and an operator S P : H ( K ) → L 1 ( P ) for which ∫ S p h d P = 0 for all h ∈ H ( K ) . Such an operator will be called a Stein operator . The collection { S p h : h ∈ H ( K ) } inherits the structure of an RKHS, whose reproducing kernel

is a Stein kernel, meaning that µ P = 0 where µ P is the kernel mean embedding from (1); see Barp et al. (2022b). Explicit calculations for the Stein kernels considered in this work can be found in Appendix C.

For univariate distributions, Barbour (1988) proposed to obtain Stein operators from infinitessimal generators of P -invariant continuous-time Markov processes; see also Barbour (1990); Gotze (1991).

The approach was extended to multivariate distributions in Gorham and Mackey (2015). The starting point is the P -invariant Itô diffusion

where p is the density of P , assumed to be positive, M : R d → R d × d is a symmetric matrix called the diffusion matrix , and W t is a standard Wiener process (Kent, 1978; Roberts and Stramer, 2002). Here the notation [ ∇· A ] i = ∇· ( A ⊤ i, · ) indicates the divergence operator applied to each row of the matrix A ( x ) ∈ R d × d . The infinitessimal generator is

Substituting h ( x ) for 1 2 ∇ u ( x ) , we obtain a Stein operator

called the diffusion Stein operator (Gorham et al., 2019). This is indeed a Stein operator, since under mild integrability conditions on K , the divergence theorem gives that ∫ S p h d P = 0 for all h ∈ H ( K ) ; for full details and a proof see Barp et al. (2022b).

A.3 Selecting a Stein Kernel

There are several choices for a Stein kernel, and which we should use depends on what form of convergence we hope to control (Gorham and Mackey, 2017; Gorham et al., 2019; Hodgkinson et al., 2020; Barp et al., 2022b; Kanagawa et al., 2022). Appendix A.3.1 describes the Langevin-Stein kernel for weak convergence control, Appendix A.3.2 describes the KGM-Stein kernels for additional control over moments, and Appendix A.3.3 presents the Riemann-Stein kernel, whose convergence properties have to-date been less well-studied.

All of the kernels that we consider have length scale parameters that need to be specified, and some also have location parameters to be specified. As a reasonably automatic default we define

as a location and a matrix of characteristic length scales for P that will be used throughout. These values can typically be obtained using gradient-based optimisation, which is usually cheaper to perform compared to full approximation of P . It is assumed that ∇ 2 log p ( x ⋆ ) is positive definite in the sequel.

A.3.1 Weak Convergence Control with Langevin-Stein Kernels

The first kernel we consider, which we called the Langevin-Stein kernel in the main text, was introduced by Gorham and Mackey (2017). This Stein kernel was developed for the purpose of controlling the weak convergence of a sequence ( Q n ) n ∈ N ⊂ P ( R d ) to P . Recall that a sequence ( Q n ) n ∈ N is said to converge weakly (or in distribution ) to P if ∫ f d Q n → ∫ f d P for all continuous bounded functions f : R d → R . This convergence is denoted Q n d → P in shorthand.

The problem considered in Gorham and Mackey (2017) was how to select a combination of matrixvalued kernel K (and, implicitly, a diffusion matrix M ) such that the Stein kernel k P in (10) generates a KSD D P ( Q ) in (4) for which D P ( Q n ) → 0 implies Q n d → P . Their solution was to combine the inverse multi-quadric kernel with an identity diffusion matrix;

for β ∈ (0 , 1) . Provided that P has a density p for which ∇ log p ( x ) is Lipschitz, and that P is distantly dissipative (see Definition 4 of Gorham and Mackey, 2017), the associated KSD enjoys weak convergence control. Technically, the results in Gorham and Mackey (2017) apply only when Σ = I , but Theorem 4 in Chen et al. (2019) demonstrated that they hold also for any positive definite Σ . Following the recommendation of several previous authors, including Chen et al. (2018, 2019); Riabiz et al. (2022), we take β = 1 2 throughout.

A.3.2 Moment Convergence Control with KGM-Stein Kernels

̸

Despite its many elegant properties, weak convergence can be insufficient for applications where we are interested in integrals ∫ f d P for which the integrand f : R d → R is unbounded. In particular, this is the case for moments of the form f ( x ) = x α 1 1 . . . x α d d , 0 = α ∈ N d 0 . In such situations, we may seek also the stronger property of moment convergence control . The development of KSDs for moment convergence control was recently considered by Kanagawa et al. (2022), and we refered to their construction as the KGM-Stein kernels in the main text. (For convenience, we have adopted the initials of the authors in naming the KGM-Stein kernel.)

Asequence ( Q n ) n ∈ N ⊂ P ( R d ) is said to converge to P in the s th order moment if ∫ ∥ x ∥ s d Q n ( x ) → ∫ ∥ x ∥ s d P ( x ) . To establish convergence of moments, we need an additional condition on top of weak convergence control: uniform integrability control. A sequence of measures ( Q n ) n ∈ N is said to have uniformly integrable s th moments if for any ε > 0 , we can take r > 0 such that

This condition essentially states that the tail decay of the measures is well-controlled (so that it has a convergent moment). The KSD convergence D P ( Q n ) → 0 implies uniform integrability if for any ε > 0 , we can take r ε > 0 and f ε ∈ H ( K ) such that

i.e., the Stein-modified RKHS can approximate the (norm-weighted) indicator function arbitrarily well. Such a function f ε can be explicitly constructed (while not guaranteed to be a member of the RKHS). Specifically, the choice f ε = (1 -ι ε ) g satisfies (13) under an appropriate dissipativity condition, where ι ε is a differentiable indicator function vanishing outside a ball, and g ( x ) = -x/ √ 1 + ∥ x ∥ 2 . This motivated Kanagawa et al. (2022) to introduce the s th order KGM-Stein kernel , which is based on the matrix-valued kernel and diffusion matrix

where ( x, y ) ↦→ ϕ ( ∥ x -y ∥ Σ ) is a C 1 0 universal kernel (see Barp et al., 2022b, Theorem 4.8). For comparability of our results, we take ϕ to be the inverse multi-quadric ϕ ( r ) = (1 + r 2 ) -1 / 2 , and

Here the normalised linear kernel κ lin ensures g ∈ H ( K ) , while the C 1 0 universal kernel ϕ allows approximation of S P ι ε g ; see Kanagawa et al. (2022).

A.3.3 Exploiting Geometry with Riemann-Langevin-Stein Kernels

For academic interest only, here we describe the Riemann-Stein kernel that featured in Figure 2 of the main text. This Stein kernel is motivated by the analysis of Gorham et al. (2019), who argued that the use of rapidly mixing Itô diffusions in Stein operators can lead to sharper convergence control. The Riemann-Stein kernel is based on the class of so-called Riemannian diffusions considered in Girolami and Calderhead (2011), who proposed to take the diffusion matrix M in (11) to be M = ( I prior + I Fisher ) -1 , the inverse of the Fisher information matrix, I Fisher, regularised using the Hessian of the negative log-prior, I prior. For the two-dimensional illustration in Section 3.2, this leads to the diffusion matrix

where we recall that y i = f i ( x ) + ϵ i , where the ϵ i are independent with ϵ i ∼ N (0 , 1) , and the prior is x ∼ N (0 , 1) . For the presented experiment we paired the above diffusion matrix with the inverse multi-quadric kernel K ( x, y ) = (1 + ∥ x -y ∥ 2 Σ ) -β for β = 1 2 . The Riemann-Stein kernel extends naturally to distributions P defined on Riemannian manifolds X ; see Barp et al. (2022a) and Example 1 of Hodgkinson et al. (2020).

Unfortunately, the Riemann-Stein kernel is prohibitively expensive in most real applications, since each evaluation of M requires a full scan through the sizen dataset. The computational complexity

of Stein Π -Thinning with the Riemann-Stein kernel is therefore O ( m 2 n 2 ) , which is unfavourable compared to the O ( m 2 n ) complexity in the case where the Stein kernel is not data-dependent. Furthermore, the convergence control properties of the Riemann-Stein kernel have yet to be established. For these reasons we included the Riemann-Stein kernel for illustration only; further groundwork will be required before the Riemann-Stein kernel can be practically used.

B Proof of Theorem 1

This appendix is devoted to the proof of Theorem 1. The proof is based on the recent work of Durmus and Moulines (2022), on the geometric convergence of MALA, and on the analysis of sparse (greedy) approximation of kernel discrepancies performed in Riabiz et al. (2022); these existing results are recalled in Appendix B.1. An additional technical result on preconditioned MALA is contained in Appendix B.2. The proof of Theorem 1 itself is contained in Appendix B.3.

B.1 Auxiliary Results

To precisely describe the results on which our analysis is based, we first need to introduce some notation and terminology. Let V : X → [1 , ∞ ) and, for a function f : X → R and a measure µ on X , let

Recall that a Q -invariant Markov chain ( x i ) i ∈ N ⊂ X with n th step transition kernel Q n is V -uniformly ergodic (see Theorem 16.0.1 of Meyn and Tweedie, 2012) if and only if ∃ R ∈ [0 , ∞ ) , ρ ∈ (0 , 1) such that

for all initial states x ∈ X and all n ∈ N .

Although MALA (Algorithm 1) is classical (Roberts and Stramer, 2002), until recently explicit sufficient conditions for ergodicity of MALA had not been obtained. The first result we will need is due Durmus and Moulines (2022), who presented the first explicit conditions for V -uniform convergence of MALA. It applies only to standard MALA, meaning that the preconditioning matrix M appearing in Algorithm 1 is the identity matrix. The extension of this result to preconditioned MALA will be handled in Appendix B.2.

Theorem 2. Let Q ∈ P ( R d ) admit a density, q , such that

(DM1) there exists x 0 with ∇ log q ( x 0 ) = 0

(DM2) q is twice continuously differentiable with sup x ∈ R d ∥∇ 2 log q ( x -x 0 ) ∥ < ∞

(DM3) there exists b > 0 and B ≥ 0 such that -∇ 2 log q ( x -x 0 ) ⪰ bI for all ∥ x -x 0 ∥ ≥ B .

Then there exists ϵ 0 > 0 such that for all step sizes ϵ ∈ (0 , ϵ 0 ) , standard Q -invariant MALA (i.e. with M = I ) is V -uniformly ergodic for V ( x ) = exp ( b 16 ∥ x -x 0 ∥ 2 ) .

Proof. This is Theorem 1 of Durmus and Moulines (2022).

The next result that we will need establishes consistency of the greedy algorithm applied to samples from a Markov chain that is Q -invariant.

Theorem 3. Let P, Q ∈ P ( X ) with P ≪ Q . Let k P : X × X → R be a Stein kernel and let D P : X × X → [0 , ∞ ] denote the associated KSD. Consider a Q -invariant, time-homogeneous Markov chain ( x i ) i ∈ N ⊂ X such that

Let P n,m be the result of running the greedy algorithm in (5) . If m ≤ n and log( n ) = O ( m γ/ 2 ) for some γ < 1 , then D P ( P n,m ) → 0 almost surely as m,n →∞ .

Proof. This is Theorem 3 of Riabiz et al. (2022).

B.2 Preconditioned MALA

In addition to the auxiliary results in Appendix B.1, which concern standard MALA (i.e. with M = I ), we require an elementary fact about MALA, namely that preconditioned MALA is equivalent to standard MALA under a linear transformation of the state variable. Recall that the M -preconditioned MALA algorithm is a Metropolis-Hastings algorithm whose proposal is the Euler-Maruyama discretisation of the Itô diffusion (11).

Proposition 1. Let M ( x ) ≡ M for a symmetric positive definite and position-independent matrix M ∈ R d × d . Let Q ∈ P ( R d ) admit a probability density function (PDF) q for which the Q -invariant diffusion ( X t ) t ≥ 0 , given by setting p = q in (11) , is well-defined. Then under the change of variables Y t := M 1 / 2 X t ,

where ˜ q ( x ) ∝ q ( M -1 / 2 x ) for all x ∈ R d .

Proof. From the chain rule,

and thus, substituting Y t = M 1 / 2 X t , (15) is equal to

which is identical to (11) in the case where M ( x ) = M is constant.

Let Q and ˜ Q be the distributions referred to in Proposition 1, whose PDFs are respectively q ( x ) and ˜ q ( x ) ∝ q ( M -1 / 2 x ) . Proposition 1 then implies that the M -preconditioned MALA algorithm applied to Q (i.e. Algorithm 1 for Π = Q ) is equivalent to the standard MALA algorithm (i.e. M = I ) applied to ˜ Q . This fact allows us to generalise the result of Theorem 2 as follows:

Corollary 1. Consider a symmetric positive definite matrix M ∈ R d × d . Assume that conditions (DM1-3) in Theorem 2 are satisfied. Then there exists ϵ ′ 0 > 0 and b ′ > 0 such that for all step sizes ϵ ∈ (0 , ϵ ′ 0 ) , the M -preconditioned Q -invariant MALA is V -uniformly ergodic for V ( x ) = exp ( b ′ 16 ∥ x -x 0 ∥ 2 ) .

Proof. From Theorem 2 and Proposition 1, the result follows if we can establish (DM1-3) for ˜ Q , since M -preconditioned MALA is equivalent to standard MALA applied to ˜ Q . For a matrix A ∈ R d × d , let λ min ( A ) and λ max ( A ) respectively denote the minimum and maximum eigenvalues of A . For (DM1) we set y 0 = M 1 / 2 x 0 and observe that

For (DM2) we have that

For (DM3) we have that

where b ′ = bλ max ( M ) -1 , which holds for all ∥ x -x 0 ∥ ≥ B , and in particular for all ∥ y -y 0 ∥ ≥ B ′ where B ′ = Bλ max ( M ) 1 / 2 . Thus (DM1-3) are established for ˜ Q .

Remark 1. The choice M = Σ -1 , which sets the preconditioner matrix M equal to the inverse of the length scale matrix Σ used in the specification of the kernel K (c.f. Appendix A.3), leads to the elegant interpretation that Stein Π -Importance Sampling applied to M -preconditioned MALA is equivalent to the Stein Π -Importance Sampling applied to standard MALA (i.e. with M = I ) for the whitened target ˜ P with PDF ˜ p ( x ) ∝ p ( M -1 / 2 x ) . For our experiments, however, the preconditioner matrix M was learned during a warm-up phase of MALA , since in general the curvature of P (captured by Σ ) and the curvature of Π (captured by M -1 ) may be different.

B.3 Proof of Theorem 1

The route to establishing Theorem 1 has three parts. First, we establish (DM1-3) of Theorem 2 with Q = Π , to deduce from Corollary 1 that Π -invariant M -preconditioned MALA is V -uniformly ergodic. This in turn enables us to establish conditions (R + 1-3) of Theorem 3, again for Q = Π , from which the strong consistency D P ( P n,m ) a.s. → 0 of S Π T-MALA is established. Finally, we note that 0 ≤ D P ( P ⋆ n ) ≤ D P ( P n,m ) , since the support of P n,m is contained in the support of P ⋆ n , and the latter is optimally weighted, whence also the strong consistency of S Π IS-MALA.

Establish (DM1-3) First we establish (DM1-3) for Q = Π . Fix x 0 ∈ R d . For (DM2), first recall that the range of k P is [ C 2 1 , ∞ ) where C 1 > 0 , from Assumption 1. Since log( · ) has bounded second derivatives on [ C 2 1 , ∞ ) , there is a constant C > 0 such that

Thus, using compactness of the set { x : ∥ x -x 0 ∥ ≤ B 2 } ,

Now, π is twice differentiable as it is the product of twice differentiable functions p and k 1 / 2 P from (A1) and (A3), and moreover

so (DM2) is satisfied. For (DM3), first note from the chain and product rules that for all ∥ x ∥ ≥ B 2

Thus, for all ∥ x -x 0 ∥ ≥ B := ∥ x 0 ∥ +max( B 1 , B 2 ) ,

as required. The same argument establishes (DM1); from (18) we have lim ∥ x ∥→∞ π ( x ) = 0 , and since π is a continuously differentiable density there must exist an x 0 at which π is locally minimised. Thus we have established (DM1-3) for Q = Π and we may conclude from Corollary 1 that there is an ϵ ′ 0 > 0 and b ′ > 0 such that, for all ϵ ∈ (0 , ϵ ′ 0 ) , the Π -invariant M -preconditioned MALA chain ( x i ) i ∈ N is V -uniformly ergodic for V ( x ) = C 2 exp ( b ′ 16 ∥ x -x 0 ∥ 2 ) (since if a Markov chain is V -uniformly ergodic, then it is also CV -uniformly ergodic).

Establish (R + 1-3) The aim is now to establish conditions (R + 1-3) of Theorem 3 for Q = Π . By construction d P/ dΠ = C 2 / √ k P ( x ) < C 2 /C 1 < ∞ , where C 1 and C 2 were defined in Assumption 1, so that P ≪ Π . It has already been established that ( x i ) i ∈ N is V -uniformly ergodic, and further

for all x , which establishes (R + 1). Let R and ρ denote constants for which the V -uniform ergodicity property (14) is satisfied. From V -uniform ergodicity, the integral ∫ V dΠ exists and

which establishes (R + 2). Fix γ > 0 . By construction d P/ dΠ ≤ C 2 /C 1 , and thus

where ˜ γ = max(1 , C 2 /C 1 ) γ . Since we have assumed that k P is continuous with, from (A4),

we may take γ such that ˜ γb 3 < b ′ / 16 , so that ∥ x ↦→ exp { ˜ γk P ( x ) }∥ V < ∞ and in particular

which establishes (R + 3). Thus we have established (R + 1-3) for Q = Π , so from Theorem 3 we have strong consistency of S Π T-MALA (i.e. D P ( P n,m ) a.s. → 0 ) provided that m ≤ n with log( n ) = O ( m γ/ 2 ) for some γ < 1 . The latter condition is equivalent to m = Ω((log n ) δ ) for some δ > 2 , which we used for the statement. Since 0 ≤ D P ( P ⋆ n ) ≤ D P ( P n,m ) , the strong consistency of S Π IS-MALA is also established.

C Explicit Calculation of Stein Kernels

This appendix contains explicit calculations for the Langevin-Stein and KGM-Stein kernels k P , which are sufficient to implement Stein Π -Importance Sampling and Stein Π -Thinning. These calculations can also be performed using automatic differentiation, but comparison to the analytic expressions is an important step in validation of computer code.

To proceed, we observe that the diffusion Stein operator S P in (12) applied to a matrix-valued kernel K is equivalent to the Langevin-Stein operator applied to the kernel C ( x, y ) = M ( x ) K ( x, y ) M ( y ) ⊤ . In the case of the Langevin-Stein and KGM-Stein kernels we have K ( x, y ) = κ ( x, y ) I for some κ ( x, y ) and M ( x ) = (1+ ∥ x -x ⋆ ∥ 2 Σ ) ( s -1) / 2 I for some s ∈ { 0 , 1 , 2 , . . . } . Thus C ( x, y ) = c ( x, y ) I where

and

following the calculations in Oates et al. (2017). To evaluate the terms in this formula we start by differentiating c ( x, y ) , to obtain

These expressions involve gradients of κ ( x, y ) , and explicit formulae for these are presented for the choice of κ ( x, y ) corresponding to the Langevin-Stein kernel in Appendix C.1, and to the KGM-Stein kernel in Appendix C.2.

To implement Stein Π -Thinning we require access to both k P ( x ) and ∇ k P ( x ) , the latter for use in the proposal distribution and acceptance probability in MALA. These quantities will now be calculated. In what follows we assume that κ ( x, y ) is continuously differentiable, so that partial derivatives with respect to x and y can be interchanged. Then

so that

Let [ ∇ x c 1 ( x )] i,j = ∂ x i [ c 1 ( x )] j and [ ∇ 2 x log p ( x )] i,j = ∂ x i ∂ x j log p ( x ) . Now we can differentiate (19) to get

In what follows we also derive explicit formulae for c 0 ( x ) , c 1 ( x ) and c 2 ( x ) , and hence for ∇ x c 0 ( x ) , ∇ x c 1 ( x ) and ∇ x c 2 ( x ) , for the case of the Langevin-Stein kernel in Appendix C.1, and the KGMStein kernel in Appendix C.2.

C.1 Explicit Formulae for the Langevin-Stein Kernel

The Langevin-Stein kernel from Appendix A.3.1 corresponds to the choice s = 1 and κ ( x, y ) the inverse multi-quadric kernel, so that

Evaluating on the diagonal:

so that c 0 ( x ) = 1 , c 1 ( x ) = 0 , c 2 ( x ) = 2 β tr (Σ -1 ) . Differentiating these formulae, ∇ x c 0 ( x ) = 0 , ∇ x c 1 ( x ) = 0 , ∇ x c 2 ( x ) = 0 .

C.2 Explicit Formulae for the KGM-Stein Kernel

The KGM kernel of order s from Appendix A.3.2 corresponds to the choice

for which we have

Evaluating on the diagonal:

so that

Differentiating these formulae:

These complete the analytic calculations necessary to compute the Stein kernel k P and its gradient.

D Empirical Assessment

This appendix contains full details of the empirical protocols that were employed and the additional empirical results described in the main text. Appendix D.1 discusses the effect of dimension on our proposed Π . Additional illuatrative results from Section 3.2 are contained in Appendix D.2. The full details for how MALA was implemented are contained in Appendix D.3. An additional illustration using a generalised auto-regressive moving average (GARCH) model is presented in Appendix D.4. The full results for S Π IS-MALA are contained in Appendix D.5, and in Appendix D.6 the convergence of the sparse approximation provided by S Π T-MALA to the optimal weighted approximation is investigated. Finally, the performance of KSDs is quantified using the 1-Wasserstein divergence in Appendix D.7.

D.1 The Effect of Dimension on Π

The improvement of Stein Π -Importance Sampling over the default Stein importance sampling algorithm (i.e. Π = P ) can be expected to reduce as the dimension d of the target P is increased. To see this, consider the Langevin-Stein kernel

for some c 1 , c 2 > 0 ; see Appendix C. Taking P = N (0 , I d × d ) , for which the length scale matrix Σ appearing in Appendix A.3 is Σ = I d × d , we obtain

However, the sampling distribution Π defined in (8) depends on k P only up to an unspecified normalisation constant; we may therefore equally consider the asymptotic behaviour of ˜ k P ( x ) := k P ( x ) /d . Let X ∼ P . Then E [ ˜ k P ( X )] = c 2 is a d -independent constant, and

as d → ∞ . This shows that ˜ k P converges to a constant function in L 2 ( P ) , and thus for 'typical' values of x in the effective support of P ,

so that Π ≈ P in the d → ∞ limit. This intuition is borne out in simulations involving both the Langevin-Stein kernel (as just discussed) and also the KGM3-Stein kernel. Indeed, Figure S1 shows that as the dimension d is increased, the marginal distributions of Π become increasingly similar to those of P .

-

3

-

2

-

1

0

1

2

3

x

1

0

.

00

0

.

05

0

.

10

0

.

15

0

.

20

0

.

25

0

.

30

0

.

35

0

.

40

Density

-

P

Π

(

d

= 1)

Π

(

d

= 2)

Π

(

d

= 10)

(S1.1)

-

3

-

2

-

1

0

1

2

3

x

1

0

.

00

0

.

05

0

.

10

0

.

15

0

.

20

0

.

25

0

.

30

0

.

35

0

.

40

Density

KGM3

-

P

Π

(

d

= 1)

Π

(

d

= 2)

Π

(

d

= 10)

(S1.2)

D.2 2D Illustration from the Main Text

Section 3.2 of the main text contained a 2-dimensional illustration of Stein Π -Importance Sampling and presented the distributions Π corresponding to different choices of Stein kernel. Here, in Figure S2, we present the mean KSDs for Stein Π -Importance Sampling performed using the Langevin-Stein kernel (purple), the KGM3-Stein kernel (blue), and the Riemann-Stein kernel (red), corresponding to the sampling distributions Π displayed in Figure 2 of the main text.

For this experiment, exact sampling from both P and Π was performed using a fine grid on which all probabilities were calculated and appropriately normalised. Results are in broad agreement with the 1-dimensional illustration contained in the main text, in the sense that in all cases Stein Π -Importance Sampling provides a significant improvement over the default Stein importance sampling method with Π equal to P .

Algorithm 4 Adaptive MALA

̸

D.3 Implementation of MALA

For implementation of MALA in Algorithm 4 we are required to specify a step size ϵ and a preconditioner matrix M . In general, suitable values for both of these parameters will be problemdependent. Standard practice is to perform some form of manual or automated tuning to arrive at parameter values for which the average acceptance rate is close to 0.57, motivated by the asymptotic analysis of Roberts and Rosenthal (1998). Adaptive MCMC algorithms, which seek to optimise the parameters of MCMC algorithms such as MALA during the warm-up period, provide an appealing solution, and was the approach taken in this work.

The adaptive MALA algorithm which we used is contained in Algorithm 4, where we have let MALA ( x, ϵ, M, n, k P ) denote the output from the preconditioned MALA with initial state x , step size ϵ , preconditioner matrix M , and chain length n , described in Algorithm 1. In Algorithm 4, we use cov( · ) to denote the sample covariance matrix. The algorithm monitors the average acceptance rate and increases or decreases it according to whether it is below or above, respectively, the 0.57 target. For the preconditioner matrix, the sample covariance matrix of samples obtained from the penultimate tuning run of MALA is used. For all experiments that we report using MALA, we set ϵ 0 = 1 , M 0 = I d , h = 10 , and α 1 = · · · = α 9 = 0 . 3 . The warm-up epoch lengths were n 0 = · · · = n 8 = 1 , 000 and the final epoch length was n 9 = 10 5 . The samples { x h -1 , 1 , . . . , x h -1 ,n i -1 } from the final epoch are returned, and constituted output from MALA for our experimental assessment.

To sample from P instead of Π , we used Algorithm 4 we formally set k P ( x ) = 1 for all x ∈ R d , which recovers Π = P as the target.

D.4 Illustration on a GARCH Model

This appendix contains an additional illustrative experiment, concerning a GARCH model that is a particular instance of a model from the PosteriorDB database discussed in Section 4. The purpose of this illustration is to facilitate an empirical investigation in a slightly higher dimension ( d = 4 ) and to explore the effect of changing the order s of the KGM-Stein kernel defined in Appendix A.3.2.

First we describe the GARCH model that was used. These models are widely-used in econometrics to describe time series data { y t } n t =1 in settings where the volatility process is assumed to be time-varying (but stationary). In particular, we consider the GARCH(1,1) model

where ϕ 2 > 0 , ϕ 3 > 0 , ϕ 4 > 0 , and ϕ 3 + ϕ 4 < 1 are the model parameters, constrained to a subset of R 4 . For ease of sampling, a change of variables τ : ( ϕ 1 , ϕ 2 , ϕ 3 , ϕ 4 ) ↦→ θ is performed in such a way that the parameter θ ∈ R 4 is unconstrained. Assuming an improper flat prior on θ , the log-posterior density for θ is given up to an additive constant by

where | J τ -1 ( θ ) | is the Jacobian determinant of τ -1 .

4

.

6

4

.

8

5

.

0

5

.

2

5

.

4

5

.

6

θ

1

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

2

.

5

3

.

0

3

.

5

Density

P

Π(KGM2)

Π(KGM3)

Π(KGM4)

-

1

.

5

-

1

.

0

-

0

.

5

0

.

0

0

.

5

1

.

0

1

.

5

2

.

0

θ

2

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

-

1

0

1

2

3

θ

3

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

Density

-

5

0

5

10

θ

4

0

.

00

0

.

05

0

.

10

0

.

15

0

.

20

0

.

25

0

.

30

0

.

35

For this illustration, real data were provided within the model description of PosteriorDB , for which the estimated maximum a posteriori parameter is ˆ ϕ = (5 . 04 , 1 . 36 , 0 . 53 , 0 . 31) . The marginal distributions of Π corresponding to the KGM-Stein kernels of orders s ∈ { 2 , 3 , 4 } are compared to the marginals of P in Figure S3. It can be seen that higher orders s correspond to greater overdispersion of Π ; this makes intuitive sense since larger s corresponds to a more stringent KSD (controlling the convergence of moments up to order s ) which places greater emphasis on how the tails of P are approximated. Further, for the final skewed marginal of P , we note that the distribution Π exaggerates the skew, placing more of its mass in the tail of the direction which is positively skewed. Further discussion of skewed targets is contained in Appendix D.8.

D.5 Stein Π -Importance Sampling for PosteriorDB

To introduce objectivity into our assessment, we exploited the PosteriorDB benchmark (Magnusson et al., 2022). This ongoing project is an attempt toward standardised benchmarking, consisting of a collection of posteriors to be numerically approximated. The test problems in PosteriorDB are defined in the Stan probabilistic programming language, and so BridgeStan (Roualdes et al., 2023) was used to directly access posterior densities and their gradients as required. The ambition of PosteriorDB is to provide an extensive set of benchmark tasks; at the time we conducted our research, PosteriorDB was at Version 0.4.0 and contained 149 models, of which 47 came equipped with a gold-standard sample of size n = 10 3 , generated from a long run of Hamiltonian Monte Carlo (the No-U-Turn sampler in Stan ). Of these 47 models, a subset of 40 were found to be compatible with BridgeStan , which was at Version 1.0.2 at the time this research was performed. The version of Stan that we used was Stanc3 Version 2.31.0 (Unix). Thus we used a total of 40 test problems for our empirical assessment.

For each test problem, a total of 10 replicate experiments were performed and standard errors were computed. A sampling method was defined as being significantly better for approximation of a given target, compared to all other methods considered, if had lower mean KSD and the standard error bar did not overlap with the standard error bar of any other method. Table 1 in the main text summarises the performance of S Π IS-MALA, fixing the number of samples to be n = 3 × 10 3 . In this appendix, full empirical results are provided.

For sampling from MALA, we used the adaptive algorithm described in Appendix D.3 with a final epoch of length n max = 10 5 . Then, whenever a set of n ≪ n max consecutive samples from MALA are required for our experimental assessment, these were obtained by selecting at random a consecutive sequence of length n from the total chain of length 10 5 . This ensures that the performance of unprocessed MALA that we report is not negatively affected by burn-in, in so far as is practical to control.

Full results are presented in Figure S4. These results broadly support the interpretation that S Π ISMALA usually outperforms SIS-MALA, or otherwise both methods provide a similar level of performance, for the sufficiently large sample sizes n considered. The sample size threshold at which S Π IS-MALA outperforms SIS-MALA appears to be dimension-dependent. A notable exception is panel 29 of Figure S4, a d = 10 dimensional task for which S Π IS-MALA provided a substantially worse approximation in KSD for the range of values of n considered.

Figure S4: Benchmarking on PosteriorDB . Here we compared raw output from MALA (dotted lines) with the post-processed output provided by the default Stein importance sampling method of Liu and Lee (2017) (SIS-MALA; solid lines) and the proposed Stein Π -Importance Sampling method (S Π IS-MALA; dashed lines). The Langevin (purple) and KGM3-Stein kernels (blue) were used for SIS-MALA and S Π IS-MALA and the associated KSDs are reported as the number n of iterations of MALA is varied. Ten replicates were computed and standard errors were plotted. The name of each model is shown in the title of the corresponding panel, and the dimension d of the parameter vector is given in parentheses. [Langevin-Stein kernel: MALA, SIS-MALA, S Π ISMALA. KGM3-Stein kernel: MALA, SIS-MALA, S Π IS-MALA.]

10

1

10

2

10

3

n

10

-

1

10

0

10

1

10

2

E

[KSD]

earnings

-

earn height(3

D

)

(S4.1)

10

1

10

2

10

3

n

10

-

1

10

0

10

1

E

[KSD]

gp pois regr

-

D

)

(S4.2)

10

1

10

2

10

3

n

10

-

1

10

0

10

1

10

2

E

[KSD]

kidiq

-

kidscore momhs(3

D

)

(S4.3)

10

1

10

2

10

3

n

10

0

10

1

10

2

E

[KSD]

kidiq

-

D

)

(S4.4)

10

1

10

2

10

3

n

10

0

10

1

10

2

E

[KSD]

mesquite

-

logmesquite logvolume(3

D

)

(S4.5)

10

1

10

2

10

3

n

10

0

10

1

10

2

E

[KSD]

arma

-

arma11(4

D

)

(S4.6)

10

1

10

2

10

3

n

10

1

10

2

E

[KSD]

earnings

-

logearn logheight male(4

D

)

(S4.7)

10

1

10

2

10

3

n

10

-

1

10

0

10

1

10

2

E

[KSD]

garch

-

D

)

(S4.8)

10

1

10

2

10

3

n

10

0

10

1

10

2

E

[KSD]

kidiq

-

kidscore momhsiq(4

D

)

(S4.9)

10

1

10

2

10

3

n

10

1

10

2

E

[KSD]

earnings

-

logearn interaction z(5

D

)

(S4.10)

10

1

10

2

10

3

n

10

1

10

2

E

[KSD]

kidiq

-

kidscore interaction(5

D

)

(S4.11)

10

1

10

2

10

3

n

10

0

10

1

10

2

E

[KSD]

kidiq with mom work

-

kidscore interaction c(5

D

)

(S4.12)

10

1

10

2

10

3

n

10

0

10

1

10

2

E

[KSD]

kidiq with mom work

-

kidscore interaction c2(5

D

)

(S4.13)

10

1

10

2

10

3

n

10

0

10

1

10

2

E

[KSD]

kidiq with mom work

-

kidscore interaction z(5

D

)

(S4.14)

10

1

10

2

10

3

n

10

0

10

1

10

2

E

[KSD]

kidiq with mom work

-

kidscore mom work(5

D

)

(S4.15)

10

1

10

2

10

3

n

10

1

10

2

E

[KSD]

-

D

)

(S4.16)

10

1

10

2

10

3

n

10

0

10

1

10

2

E

[KSD]

mesquite

-

logmesquite logva(5

D

)

(S4.17)

10

1

10

2

10

3

n

10

0

10

1

10

2

E

[KSD]

hmm example

-

hmm example(6

D

)

(S4.18)

10

1

10

2

10

3

n

10

3

10

4

E

[KSD]

sblrc

-

blr(6

D

)

(S4.19)

10

1

10

2

10

3

n

10

2

10

3

10

4

E

[KSD]

-

blr(6

D

)

(S4.20)

10

1

10

2

10

3

n

10

1

10

2

E

[KSD]

arK

-

D

)

(S4.21)

10

1

10

2

10

3

n

10

0

10

1

10

2

E

[KSD]

mesquite

-

D

)

(S4.22)

10

1

10

2

10

3

n

10

0

10

1

10

2

E

[KSD]

bball drive event 0

-

hmm drive 0(8

D

)

(S4.23)

10

1

10

2

10

3

n

10

1

10

2

10

3

E

[KSD]

bball drive event 1

-

hmm drive 1(8

D

)

(S4.24)

10

1

10

2

10

3

n

10

1

10

2

E

[KSD]

-

lotka volterra(8

D

)

(S4.25)

10

1

10

2

10

3

n

10

0

10

1

10

2

E

[KSD]

mesquite

-

logmesquite(8

D

)

(S4.26)

10

1

10

2

10

3

n

10

0

10

1

10

2

E

[KSD]

mesquite

-

D

)

(S4.27)

10

1

10

2

10

3

n

10

0

10

1

E

[KSD]

mesquite

-

mesquite(8

D

)

(S4.28)

10

1

10

2

10

3

n

10

-

1

10

0

10

1

10

2

10

3

E

[KSD]

eight schools

-

eight schools centered(10

D

)

(S4.29)

10

1

10

2

10

3

n

10

-

1

10

0

10

1

10

2

E

[KSD]

eight schools

-

eight schools noncentered(10

D

)

(S4.30)

10

1

10

2

10

3

n

10

1

10

2

10

3

E

[KSD]

nes1972

-

nes(10

D

)

(S4.31)

10

1

10

2

10

3

n

10

1

10

2

10

3

E

[KSD]

nes1976

-

nes(10

D

)

(S4.32)

10

1

10

2

10

3

n

10

1

10

2

10

3

E

[KSD]

nes1980

-

nes(10

D

)

(S4.33)

10

1

10

2

10

3

n

10

1

10

2

10

3

E

[KSD]

-

nes(10

D

)

(S4.34)

10

1

10

2

10

3

n

10

1

10

2

10

3

E

[KSD]

nes1988

-

nes(10

D

)

(S4.35)

10

1

10

2

10

3

n

10

1

10

2

10

3

E

[KSD]

nes1992

-

nes(10

D

)

(S4.36)

10

1

10

2

10

3

n

10

1

10

2

10

3

E

[KSD]

nes1996

-

nes(10

D

)

(S4.37)

10

1

10

2

10

3

n

10

1

10

2

E

[KSD]

nes2000

-

nes(10

D

)

10

1

10

2

10

3

n

10

3

10

4

E

[KSD]

diamonds

-

diamonds(26

D

)

(S4.39)

10

1

10

2

10

3

n

10

1

10

2

10

3

10

4

E

[KSD]

mcycle gp

-

accel gp(66

D

)

(S4.40)

D.6 Stein Π -Thinning for PosteriorDB

The results presented in the main text concerned n = 3 × 10 3 samples from MALA, which is near the limit at which the optimal weights w ⋆ can be computed in a few seconds on a laptop PC. For larger values of n , sparse approximation methods are likely to required. In the main text we presented Stein Π -Thinning , which employs a greedy optimisation perspective to obtain a sparse approximation to the optimal weights at cost O ( m 2 n ) , where m are the number of greedy iterations performed. Explicit and verifiable conditions for the strong consistency of the resulting S Π T-MALA algorithm were established in Section 3.3. The purpose of this appendix is to empirically explore the convergence of S Π T-MALA using the PosteriorDB test bed.

In the experiments we report the number of MALA samples was fixed to n = 10 3 and the number of greedy iterations was varied from m = 1 to m = 10 3 . The results, in Figure S5, indicate that for most models in PosteriorDB the minimum value of KSD is approximately reached when m is anywhere from n 10 to n 2 , representing a modest but practically significant reduction in computational cost compared to S Π IS-MALA. This agrees with the qualitative findings reported in the original Stein thinning paper of Riabiz et al. (2022).

10

0

10

1

10

2

10

3

m

10

-

1

10

0

10

1

10

2

E

[KSD]

earnings

-

earn height(3

D

)

(S5.1)

10

0

10

1

10

2

10

3

m

10

0

10

1

E

[KSD]

gp pois regr

-

D

)

(S5.2)

10

0

10

1

10

2

10

3

m

10

0

10

1

E

[KSD]

kidiq

-

kidscore momhs(3

D

)

10

0

10

1

10

2

10

3

m

10

0

10

1

10

2

E

[KSD]

kidiq

-

D

)

(S5.4)

10

0

10

1

10

2

10

3

m

10

0

10

1

E

[KSD]

mesquite

-

logmesquite logvolume(3

D

)

10

0

10

1

10

2

10

3

m

10

1

10

2

E

[KSD]

arma

-

arma11(4

D

)

(S5.6)

10

0

10

1

10

2

10

3

m

10

1

10

2

E

[KSD]

earnings

-

logearn logheight male(4

D

)

(S5.7)

10

0

10

1

10

2

10

3

m

10

0

10

1

E

[KSD]

garch

-

D

)

(S5.8)

10

0

10

1

10

2

10

3

m

10

0

10

1

10

2

E

[KSD]

kidiq

-

kidscore momhsiq(4

D

)

(S5.9)

10

0

10

1

10

2

10

3

m

10

1

10

2

E

[KSD]

earnings

-

logearn interaction z(5

D

)

(S5.10)

10

0

10

1

10

2

10

3

m

10

1

10

2

E

[KSD]

kidiq

-

kidscore interaction(5

D

)

(S5.11)

10

0

10

1

10

2

10

3

m

10

1

E

[KSD]

kidiq with mom work

-

kidscore interaction c(5

D

)

(S5.12)

10

0

10

1

10

2

10

3

m

10

0

10

1

10

2

E

[KSD]

kidiq with mom work

-

kidscore interaction c2(5

D

)

(S5.13)

10

0

10

1

10

2

10

3

m

10

0

10

1

10

2

E

[KSD]

kidiq with mom work

-

kidscore interaction z(5

D

)

(S5.14)

10

0

10

1

10

2

10

3

m

10

0

10

1

10

2

E

[KSD]

kidiq with mom work

-

kidscore mom work(5

D

)

(S5.15)

10

0

10

1

10

2

10

3

m

10

1

10

2

E

[KSD]

-

D

)

(S5.16)

10

0

10

1

10

2

10

3

m

10

1

10

2

E

[KSD]

mesquite

-

logmesquite logva(5

D

)

(S5.17)

10

0

10

1

10

2

10

3

m

10

1

10

2

E

[KSD]

hmm example

-

hmm example(6

D

)

(S5.18)

10

0

10

1

10

2

10

3

m

10

3

10

4

E

[KSD]

sblrc

-

blr(6

D

)

(S5.19)

10

0

10

1

10

2

10

3

m

10

2

10

3

10

4

E

[KSD]

-

blr(6

D

)

(S5.20)

10

0

10

1

10

2

10

3

m

10

1

10

2

E

[KSD]

arK

-

D

)

(S5.21)

10

0

10

1

10

2

10

3

m

10

1

10

2

E

[KSD]

mesquite

-

D

)

(S5.22)

10

0

10

1

10

2

10

3

m

10

1

E

[KSD]

bball drive event 0

-

hmm drive 0(8

D

)

(S5.23)

10

0

10

1

10

2

10

3

m

10

2

10

3

E

[KSD]

bball drive event 1

-

hmm drive 1(8

D

)

(S5.24)

10

0

10

1

10

2

10

3

m

10

1

10

2

E

[KSD]

-

lotka volterra(8

D

)

(S5.25)

10

0

10

1

10

2

10

3

m

10

1

10

2

E

[KSD]

mesquite

-

logmesquite(8

D

)

(S5.26)

10

0

10

1

10

2

10

3

m

10

1

10

2

E

[KSD]

mesquite

-

D

)

(S5.27)

10

0

10

1

10

2

10

3

m

10

-

1

10

0

10

1

E

[KSD]

mesquite

-

mesquite(8

D

)

(S5.28)

10

0

10

1

10

2

10

3

m

10

0

10

1

E

[KSD]

eight schools

-

eight schools centered(10

D

)

(S5.29)

10

0

10

1

10

2

10

3

m

10

0

10

1

10

2

10

3

E

[KSD]

eight schools

-

eight schools noncentered(10

D

)

(S5.30)

10

0

10

1

10

2

10

3

m

10

1

10

2

E

[KSD]

nes1972

-

nes(10

D

)

(S5.31)

10

0

10

1

10

2

10

3

m

10

1

10

2

E

[KSD]

nes1976

-

nes(10

D

)

(S5.32)

10

0

10

1

10

2

10

3

m

10

1

10

2

E

[KSD]

nes1980

-

nes(10

D

)

(S5.33)

10

0

10

1

10

2

10

3

m

10

1

10

2

E

[KSD]

-

nes(10

D

)

(S5.34)

10

0

10

1

10

2

10

3

m

10

1

10

2

E

[KSD]

nes1988

-

nes(10

D

)

(S5.35)

10

0

10

1

10

2

10

3

m

10

1

10

2

10

3

E

[KSD]

nes1992

-

nes(10

D

)

(S5.36)

10

0

10

1

10

2

10

3

m

10

1

10

2

E

[KSD]

nes1996

-

nes(10

D

)

(S5.37)

10

0

10

1

10

2

10

3

m

10

1

10

2

E

[KSD]

nes2000

-

nes(10

D

)

(S5.38)

3

10

0

10

1

10

2

10

3

m

10

2

10

3

10

4

E

[KSD]

diamonds

-

diamonds(26

D

)

(S5.39)

10

0

10

1

10

2

10

m

10

2

10

3

10

4

10

5

E

[KSD]

mcycle gp

-

accel gp(66

D

)

(S5.40)

D.7 Performance of Stein Discrepancies

The properties of Stein discrepancies was out of scope for this work. Nonetheless, there is much interest in better understanding the properties of KSDs, and in this appendix the performance of S Π IS-MALA in terms of 1-Wasserstein divergence is reported. This was made possible since PosteriorDB supplies a set of posterior samples obtained from a long run of Hamiltonian Monte Carlo (the No-U-Turn sampler in Stan ) which we treat as a gold standard.

Full results are presented in Figure S6 and Table 2. Broadly speaking, in most cases the minimisation of KSD seems to be associated with minimisation of 1-Wasserstein distance, and in particular a significant improvement of S Π IS-MALA over SIS-MALA is reported for ≈ 63% of tasks in the PosteriorDB benchmark. However there are some scenarios for which minimisation of KSD is loosely, if at all, related to minimisation of 1-Wasserstein divergence. In these cases, we attribute this performance to a combination of two factors: First, the blindness to mixing proportions phenomena, described in Wenliang and Kanagawa (2021); Koehler et al. (2022); Liu et al. (2023), which is a pathology of KSDs in general. Second, the Langevin-Stein kernel cannot be expected to control convergence in 1-Wasserstein, since convergence in 1-Wasserstein is equivalent to weak convergence plus convergence of the first moment. Focusing therefore on the KGM3-Stein kernel only, it is encouraging to note that S Π IS-MALA outperforms SIS-MALA on 83% of tasks in PosteriorDB in the 1-Wasserstein metric, as shown in Table 2. However, it is interesting to observe that MALA performed well in the 1-Wasserstein sense across the PosteriorDB test bed.

The development of improved Stein discrepancies is an active area of research, and we emphasise that the methodology developed in this work can be applied to any KSDs, including potentially KSDs with better or more direct control over standard notions of convergence (such as 1-Wasserstein) that in the future may be developed.

10

1

10

2

10

3

n

10

4

6

×

10

3

7

×

10

3

8

×

10

3

9

×

10

3

E

[WASD]

earnings

-

earn height(3

D

)

(S6.1)

10

1

10

2

10

3

n

10

-

1

2

×

10

-

1

3

×

10

-

1

4

×

10

-

1

E

[WASD]

gp pois regr

-

D

)

(S6.2)

10

1

10

2

10

3

n

10

0

E

[WASD]

kidiq

-

kidscore momhs(3

D

)

(S6.3)

10

1

10

2

10

3

n

10

0

E

[WASD]

kidiq

-

D

)

(S6.4)

10

1

10

2

10

3

n

10

-

1

3

×

10

-

2

4

×

10

-

2

6

×

10

-

2

E

[WASD]

mesquite

-

logmesquite logvolume(3

D

)

(S6.5)

10

1

10

2

10

3

n

2

×

10

-

2

3

×

10

-

2

4

×

10

-

2

6

×

10

-

2

E

[WASD]

arma

-

arma11(4

D

)

(S6.6)

10

1

10

2

10

3

n

10

0

2

×

10

0

3

×

10

0

4

×

10

0

E

[WASD]

earnings

-

logearn logheight male(4

D

)

(S6.7)

10

1

10

2

10

3

n

10

0

E

[WASD]

garch

-

D

)

(S6.8)

10

1

10

2

10

3

n

2

×

10

0

3

×

10

0

4

×

10

0

6

×

10

0

E

[WASD]

kidiq

-

kidscore momhsiq(4

D

)

(S6.9)

10

1

10

2

10

3

n

10

-

1

3

×

10

-

2

4

×

10

-

2

6

×

10

-

2

E

[WASD]

earnings

-

logearn interaction z(5

D

)

(S6.10)

10

1

10

2

10

3

n

2

×

10

1

3

×

10

1

E

[WASD]

kidiq

-

kidscore interaction(5

D

)

(S6.11)

10

1

10

2

10

3

n

10

0

E

[WASD]

kidiq with mom work

-

kidscore interaction c(5

D

)

(S6.12)

10

1

10

2

10

3

n

10

0

E

[WASD]

kidiq with mom work

-

kidscore interaction c2(5

D

)

(S6.13)

10

1

10

2

10

3

n

10

0

2

×

10

0

3

×

10

0

4

×

10

0

E

[WASD]

kidiq with mom work

-

kidscore interaction z(5

D

)

(S6.14)

10

1

10

2

10

3

n

10

0

2

×

10

0

3

×

10

0

4

×

10

0

6

×

10

0

E

[WASD]

kidiq with mom work

-

kidscore mom work(5

D

)

(S6.15)

10

1

10

2

10

3

n

10

-

1

2

×

10

-

2

3

×

10

-

2

4

×

10

-

2

6

×

10

-

2

E

[WASD]

-

D

)

(S6.16)

10

1

10

2

10

3

n

10

-

1

2

×

10

-

1

3

×

10

-

1

E

[WASD]

mesquite

-

logmesquite logva(5

D

)

(S6.17)

10

1

10

2

10

3

n

10

-

1

2

×

10

-

1

3

×

10

-

1

4

×

10

-

1

6

×

10

-

1

E

[WASD]

hmm example

-

hmm example(6

D

)

(S6.18)

10

1

10

2

10

3

n

5

.

8

×

10

-

2

6

×

10

-

2

6

.

2

×

10

-

2

6

.

4

×

10

-

2

6

.

6

×

10

-

2

E

[WASD]

sblrc

-

blr(6

D

)

(S6.19)

10

1

10

2

10

3

n

10

-

1

7

×

10

-

2

8

×

10

-

2

9

×

10

-

2

E

[WASD]

-

blr(6

D

)

(S6.20)

10

1

10

2

10

3

n

10

-

1

6

×

10

-

2

2

×

10

-

1

E

[WASD]

arK

-

D

)

(S6.21)

10

1

10

2

10

3

n

2

×

10

-

1

3

×

10

-

1

4

×

10

-

1

6

×

10

-

1

E

[WASD]

mesquite

-

D

)

(S6.22)

10

1

10

2

10

3

n

10

0

2

×

10

-

1

3

×

10

-

1

4

×

10

-

1

6

×

10

-

1

E

[WASD]

bball drive event 0

-

hmm drive 0(8

D

)

(S6.23)

10

1

10

2

10

3

n

10

0

8

×

10

-

1

9

×

10

-

1

E

[WASD]

bball drive event 1

-

hmm drive 1(8

D

)

(S6.24)

10

1

10

2

10

3

n

2

×

10

-

1

3

×

10

-

1

4

×

10

-

1

E

[WASD]

-

lotka volterra(8

D

)

(S6.25)

10

1

10

2

10

3

n

2

×

10

-

1

3

×

10

-

1

4

×

10

-

1

6

×

10

-

1

E

[WASD]

mesquite

-

logmesquite(8

D

)

(S6.26)

10

1

10

2

10

3

n

2

×

10

-

1

3

×

10

-

1

4

×

10

-

1

6

×

10

-

1

E

[WASD]

mesquite

-

D

)

(S6.27)

10

1

10

2

10

3

n

2

×

10

2

3

×

10

2

4

×

10

2

E

[WASD]

mesquite

-

mesquite(8

D

)

(S6.28)

10

1

10

2

10

3

n

10

1

E

[WASD]

eight schools

-

eight schools centered(10

D

)

(S6.29)

10

1

10

2

10

3

n

1

.

68

×

10

1

1

.

7

×

10

1

1

.

72

×

10

1

1

.

74

×

10

1

1

.

76

×

10

1

1

.

78

×

10

1

1

.

8

×

10

1

1

.

82

×

10

1

E

[WASD]

eight schools

-

eight schools noncentered(10

D

)

(S6.30)

10

1

10

2

10

3

n

2

×

10

-

1

3

×

10

-

1

4

×

10

-

1

6

×

10

-

1

E

[WASD]

nes1972

-

nes(10

D

)

(S6.31)

10

1

10

2

10

3

n

2

×

10

-

1

3

×

10

-

1

4

×

10

-

1

6

×

10

-

1

E

[WASD]

nes1976

-

nes(10

D

)

(S6.32)

10

1

10

2

10

3

n

3

×

10

-

1

4

×

10

-

1

6

×

10

-

1

E

[WASD]

nes1980

-

nes(10

D

)

(S6.33)

10

1

10

2

10

3

n

2

×

10

-

1

3

×

10

-

1

4

×

10

-

1

6

×

10

-

1

E

[WASD]

-

nes(10

D

)

(S6.34)

10

1

10

2

10

3

n

2

×

10

-

1

3

×

10

-

1

4

×

10

-

1

6

×

10

-

1

E

[WASD]

nes1988

-

nes(10

D

)

(S6.35)

10

1

10

2

10

3

n

2

×

10

-

1

3

×

10

-

1

4

×

10

-

1

6

×

10

-

1

E

[WASD]

nes1992

-

nes(10

D

)

(S6.36)

10

1

10

2

10

3

n

2

×

10

-

1

3

×

10

-

1

4

×

10

-

1

6

×

10

-

1

E

[WASD]

nes1996

-

nes(10

D

)

(S6.37)

10

1

10

2

10

3

n

10

0

3

×

10

-

1

4

×

10

-

1

6

×

10

-

1

E

[WASD]

nes2000

-

nes(10

D

)

(S6.38)

10

1

10

2

10

3

n

6

.

4

×

10

-

1

6

.

5

×

10

-

1

6

.

6

×

10

-

1

6

.

7

×

10

-

1

6

.

8

×

10

-

1

6

.

9

×

10

-

1

E

[WASD]

diamonds

-

diamonds(26

D

)

(S6.39)

10

1

10

2

10

3

n

1

.

54

×

10

1

1

.

56

×

10

1

1

.

58

×

10

1

1

.

6

×

10

1

1

.

62

×

10

1

1

.

64

×

10

1

1

.

66

×

10

1

E

[WASD]

mcycle gp

-

accel gp(66

D

)

(S6.40)

D.8 Investigation for a Skewed Target

This final appendix contrasts the 1-Wasserstein optimal sampling distribution Π 1 (c.f. Section 2.1), with the choice of Π that we recommended in (8). In particular, we focus on the KGM3-Stein kernel under a heavily skewed P , for which Π 1 and Π can be markedly different.

For this investigation a bivariate skew-normal target was constructed, where the density is given by p ( x 1 , x 2 ) = 4 ϕ ( x 1 )Φ(6 x 1 ) ϕ ( x 2 )Φ( -3 x 2 ) , with ϕ and Φ respectively denoting the density and distribution functions of a standard Gaussian. The density p of P , together with the marginal densities of Π 1 and Π , are plotted in Figure S7. It can be seen that, while both Π 1 and Π are over-dispersed with respect to P , our recommended Π assigns proportionally more mass to the tail that is positively skewed.

The performance of Stein Π -Importance Sampling based on Π 1 and Π is compared in Figure S8. Though both choices lead to an improvement relative to Stein importance sampling algorithm with Π = P , the use of Π leads to a significant further reduction (on average) in KSD compared to Π 1 . Based on our investigations, this finding seems general; the use of Π 1 does not realise the full potential of Stein Π -Imporance sampling when the target is skewed.

-

1

0

1

2

3

4

5

6

x

1

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

Density

P

Π(1Wass

.

)

Π(KGM3)

-

6

-

5

-

4

-

3

-

2

-

1

0

1

2

x

2

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

10

1

10

2

10

3

10

-

1

10

0

10

1

E

[KSD]

P

Π(1Wass

.

)

Π(KGM3)

n

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a novel approach for constructing an MCMC target that is specifically designed for post-processing using Stein importance sampling and Stein thinning, where the goal is to assign optimal weights to a subset of sample particles in order to construct the best possible approximations of a distribution $P$.\n\nThe proposed method introduces a new target distribution, denoted as $\\Pi$, which is obtained by tilting the original target density $p(x)$ with the square root of a Stein kernel $k_P(x)$. This construction is derived by solving a variational problem that minimises the trace of the variance of a limiting Gaussian distribution. The use of $\\Pi$ as a target for the Metropolis-Adjusted Langevin Algorithm (MALA) instead of the original target distribution $P$ is justified through an almost sure consistency guarantee (Theorem 1) and numerical experiments conducted on a set of benchmark problems.\n\nSTRENGTHS:\n**Originality**: Whilst previous works have extensively studied post-processing using Stein's discrepancy, this paper introduces a novel perspective on improving Stein importance sampling through the design of a target distribution that is distinct from the original target $P$ but more suitable for post-processing techniques. This novel approach of leveraging the target design to improve post-processing methods is both interesting and original.\n\n**Quality**: The construction of the proposed target distribution $\\Pi$ is clearly explained (Section 3.1). The paper presents both a theoretical guarantee (Theorem 1) and extensive numerical evidence to support the choice of $\\Pi$. The assumptions in Theorem 1 appear to be mild, and the authors provide comprehensive discussions comparing them with similar conditions in related literature. Overall, the significance of the proposed method is convincingly demonstrated.\n\n**Clarity**: This paper exhibits a high level of clarity throughout, including an extensive review of related literature and methodologies (Section 2).\n\nWEAKNESSES:\n**Motivation**: While the authors have effectively justified the use of the proposed target $\\Pi$ instead of the original target for constructing MALA samplers, the rationale for employing S$\\Pi$IS over running MALA **without** post-processing is slightly weak. Specifically:\n\n1. A main advantage of Stein importance sampling (SIS) lies in its ability to provide unbiased estimation when the MCMC sampler used to generate the sample is biased. Consequently, allocating computational resources to post-processing is crucial in such cases, as it is uncertain whether running the chain for a longer time would improve the quality due to the bias. However, for $\\Pi$ sampling, one must set up an unbiased sampler (e.g., MALA) that targets $\\Pi$, raising the question of whether conducting S$\\Pi$IS is more beneficial than simply allocating the same computational budget to running the chain for an extended period. The reported experimental results do not seem to address this question, as they solely compare S$\\Pi$IS with standard SIS. Including experiments comparing S$\\Pi$IS with MALA without post-processing would be valuable in addressing this concern.\n2. S$\\Pi$IS requires the setup of an MCMC sampler targeting a distribution different from $P$. Thus, the generated sample must be used in conjunction with the post-processing step to perform inference on $P$. This contrasts with standard SIS, where samples can be drawn from a sampler targeting $P$, which is typically what practitioners would do regardless of their intention to use post-processing methods. This raises the question of whether setting up a sampler that targets such a specialised distribution and relies on post-processing is more practically attractive than directly targeting $P$. Including discussions addressing these concerns would be beneficial.\n\nQUESTIONS:\nL174: Could you elaborate on what is the relaxation and what one loses due to this relaxation, if any?\n\nFigure 1.2: It seems that for small $n$, targeting $P$ is more beneficial than targeting $\\Pi$. Is it purely coincidental, or does it reflect a specific characteristic or trade-off associated with the methodology? Could you offer some insights into this observation?\n\nL229: Could you explain why the co-domain of $D_P$ is $\\mathcal{X} \\times \\mathcal{X}$ instead of $\\mathcal{P}(\\mathcal{X})$?\n\nTable 1: Are the results averages over multiple repetitions or obtained from a single repetition? t\n\nLIMITATIONS:\nSome limitations of Stein IS and the use of different Stein kernels are addressed in Section 4 and Section 3.2.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-02"}

comments: SUMMARY:
The paper studies the design of the MCMC algorithm which is well suited for post-processing to obtain consistent approximation $P_n^\star$ of the target measure $P$ using Stein kernel discrepancies ($D_P( \cdot)$). The authors suggest the following novel procedure: (1) choose a measure $\Pi$ which differs from the target measure $P$ by the factor $\sqrt{k_p}$, where $k_p$ is the Stein kernel  (solving variational problem) (2); Sample from $\Pi$ using (pre-conditioned) MALA; (3) solve linearly-constrained quadratic problem or construct sparse approximation to obtain $P_n^\star$.  
Theorem 1 provides assumptions to ensure convergence of Stein kernel discrepancies $D(P_n^\star)$ to zero.  Step (1) is based on SNIS procedure and the choice of $\Pi$ which gives the smallest variance of the Stein kernel discrepancies between SNIS estimate P_n and target measure $P$. 
The results are illustrated by numerical experiments.

STRENGTHS:
-Novel variational algorithm for explicit construction of measure \Pi. 
-Theoretical analysis of the algorithm in the case of using MALA as a MCMC sampler

WEAKNESSES:
- Only asymptotic convergence in D_P metric is proved in Theorem 1. 
- Would be good to sketch ideas of construction of Stein's kernel in the main text. 
- It could be difficult to calculate Stein's kernel in more practical problems
- Works in moderate dimensions

QUESTIONS:
It is well known that MALA sampler is not good for mixtures of distributions. Is it possible to replace MALA by some other MCMC sampler as HMC or adaptive MCMC?

LIMITATIONS:
-
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-04

comments: SUMMARY:
This paper proposes a proposal distribution $\Pi$ to generate finite points 
such that a weighted version approximates the target distribution $P$ under 
Stein discrepancy.

STRENGTHS:
The main strength lies in a new proposal for the sampling distribution $\Pi$ that 
is more efficient for follow-up approximation to the original distribution $P$ in the 
sense of Stein discrepancy. 

Asymptotic consistency is established, and extensive simulations on Bayesian computation 
is also conducted to illustrate the benefit of the proposal.

WEAKNESSES:
The authors might consider expanding the section on the actual contribution (the proposal of $\Pi$), 
and be more brief on the background. For instance, the section on Wasserstein distance doesn't seem necessary. 

The second limitation is in the theoretical guarantees. Can one provide a non-asymptotic convergence guarantee 
for the proposed method?

QUESTIONS:
1. Can the authors comment more on the effect of dimensionality on the improvement of $\Pi$ over $P$?

2. Can the same proposal work for other kernel functions ( non Stein)?

LIMITATIONS:
N/A
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-07-04

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper analyses which target distribution to use for MCMC in the situation where a stein-descrepency will be used to post-process it's output samples.\nThey propose to use a different target distribution for the MCMC to the distribution being approximated, and show this improves performance on a variety of posterior inference problems.\n\nSTRENGTHS:\n- The paper identifies and clear question: how to choose the invariant distribution \\pi for MCMC if using a stein-descrepency to post-process the samples.\n- They provide a method for selecting $\\pi$ via a variational problem framing, where $\\pi$ is selected to minimize the variance in the post-processed approximation. This gives a closed form expression for $\\pi$ (up to a normalizing constant) such that it can be easily used within MCMC. Their analysis is agnostic to the choice of stein kernel making their results broadly applicable. \n- Figure 1 nicely illustrates the property of over-dispersion that their choice of $\\pi$ has, and shows that on a simple 1D problem that this results in lower error bars compared to using $P$ for the MCMC. \n- In their experiments their proposed method is shown to consistently improve (better results in 70% of  PosteriorDB tasks) upon the baseline of setting $\\pi$ to the target distribution being approximated. \n- Their presentation is generally very clear, with informative figures provided.\n\nWEAKNESSES:\nI could not see any weaknesses in this paper, however the it's subfield is not within my area of expertise and I did not check the proofs.\n\nQUESTIONS:\nHow does using the stein-discrepency **during** sample generation (e.g. stein variational gradient descent) compare to using it for post-processing? \nI note that this question is not relevant for the contributions of the paper, but an answer would be useful for my understanding of the paper's general usefulness.\n\nLIMITATIONS:\nThe authors address the limitations of their method in the discussion, namely that it requires second order derivatives of the model", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work proposes a design of a probability density that is more over-dispersed than the target density, so that, somewhat surprisingly, the resulting MCMC samples, after optimally reweighted, can achieve lower KSD than MCMC samples from the true target density. Consistency of the two proposed algorithms, SPiIS-MALA and SPiT-MALA, is proved. These two algorithms are benchmarked on PosteriorDB dataset to demonstrate their superior performance over raw MALA and SIS-MALA (MALA plus optimal reweighting).\n\nSTRENGTHS:\n- The paper tackles an interesting yet to my knowledge underexplored problem in sampling, which is how to design a density so that the resulting samples provide a good discrete support for the ensuing optimally reweighting step that finds a weight to minimize KSD.\n- The angle of the attack is, although not new (e.g. Graf and Luschgy 2007), quite surprising (i.e. the density needs to be over-dispersed). The motivational section on the optimal quantization of Wasserstein distance is well-put.\n- The consistency of the two proposed methods is proved which is nice.\n- Many kernels are studied (Langevin-Stein/KGM/Riemann-Langevin-Stein) and are used in producing the empirical results.\n- The writing of the paper is excellent and lots of intuition and toy examples are given to illustrate the points.\n\nWEAKNESSES:\n- The consistency proof of Theorem 1 seems like a rather straightforward application of Theorem 2 (Durmus and Moulines 2022) and Theorem 3(Riabiz et al. 2022). In particular, it seems to me the same proof should go through for quite generic $\\Pi$, not necessarily the one that takes the form in (8). Hence, it is not clear whether the design (8) is theoretically justified, other than the heuristic argument given in Sec. 3.1.\n- There is a considerable gap between the heuristic argument in Sec. 3.1 and the proposed algorithm, which is the weights used in Sec. 3.1 is not optimal but in the algorithms they are. The authors claimed the choice $dP/d\\Pi$ is \"near-optimal\" (without justification), but then noted that using $dP/d\\Pi$ will perform substantially worse than the optimal weight, which seems contradictory to the first claim. \n- The experimental results comparing SIS-MALA and SPiIS-MALA are somewhat mixed. For an end user, there is no provided criterion on whether they should use the proposed method or the baseline SIS-MALA. Moreover, I cannot find standard derivation of the reported numbers.\n  *  There seems to be missing experiments that benchmark the performance of SPiT-MALA compared to baseline (e.g. $\\Pi=P4). The only experiment I found for SPiT-MALA is in D.6 where the consistency is verified. \n- In many places it is hinted that $P$ will be close to $\\Pi$ as the dimension $d$ increases. This implies that the proposed method is only applicable to small dimensions and thus the application value is limited.  Further analysis on the relation to dimension could be helpful. Moreover, there is only one experiment for $d=66$ (last row in Table 1) that corroborate the point that the extent of improvement decreases when the dimension increases; more experiments could be used to strengthen this point.\n\nQUESTIONS:\n- I would like to see a rigorous statement and proof of the heuristic argument from Sec 3.1. The sketched-out argument makes sense (other than one detail --- see below) so I'm wondering why it is not made into a complete proof. \n  * In the paragraph below (S2), it seems to assume that $E_{x \\sim \\Pi}[\\frac{dP}{d\\Pi}(x)(k(\\cdot, x) - \\mu_P)] = 0$. Why is this true? I think $E_{x \\sim \\Pi}[\\frac{dP}{d\\Pi}(x)k(\\cdot, x) - \\mu_P] = 0$ but not when $\\mu_P$ is multiplied by the importance weight. Of course, if $\\mu_P = 0$ then it does not matter.\n- A heuristic argument is given in D.1 for the Langevin-Stein kernel and $P$ is the standard $d$-dimensional Gaussian. Aside from this very simple $P$, is it true that $P \\approx \\Pi$ in general? Can we say anything theoretical about it? \n- How is the simplex-constrained minimization in Algorithm 2 implemented? What is the time complexity? This also seems like the computational bottleneck of Alg. 2 and the reason $n$ is ony a few thousands in all experiments.\n- For the experiment done in D.8, if the plot is for Wasserstein-1 distance, will $\\Pi(1Wass)$ result in better numbers than $\\Pi(KGM3)$?\n\nLIMITATIONS:\nThe authors have adequately addressed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-16"}