SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\mathrm {d} X _ {t} = \frac {1}{2} \frac {1}{p \left(X _ {t}\right)} \nabla \cdot \left[ p \left(X _ {t}\right) M \left(X _ {t}\right) \right] \mathrm {d} t + M \left(X _ {t}\right) ^ {1 / 2} \mathrm {d} W _ {t}, \tag {11}
$$
where  $p$  is the density of  $P$ , assumed to be positive,  $M: \mathbb{R}^d \to \mathbb{R}^{d \times d}$  is a symmetric matrix called the diffusion matrix, and  $W_t$  is a standard Wiener process (Kent, 1978; Roberts and Stramer, 2002). Here the notation  $[\nabla \cdot A]_i = \nabla \cdot (A_{i,\cdot}^\top)$  indicates the divergence operator applied to each row of the matrix  $A(x) \in \mathbb{R}^{d \times d}$ . The infinitesimal generator is
$$
(A _ {P} u) (x) = \frac {1}{2} \frac {1}{p (x)} \nabla \cdot [ p (x) M (x) \nabla u (x) ].
$$
Substituting  $h(x)$  for  $\frac{1}{2}\nabla u(x)$ , we obtain a Stein operator
$$
(S _ {P} h) (x) = \frac {1}{p (x)} \nabla \cdot [ p (x) M (x) h (x) ] \tag {12}
$$
called the diffusion Stein operator (Gorham et al., 2019). This is indeed a Stein operator, since under mild integrability conditions on  $K$ , the divergence theorem gives that  $\int S_{p}h \, \mathrm{d}P = 0$  for all  $h \in \mathcal{H}(K)$ ; for full details and a proof see Barp et al. (2022b).
A.3 Selecting a Stein Kernel
There are several choices for a Stein kernel, and which we should use depends on what form of convergence we hope to control (Gorham and Mackey, 2017; Gorham et al., 2019; Hodgkinson et al., 2020; Barp et al., 2022b; Kanagawa et al., 2022). Appendix A.3.1 describes the Langevin-Stein kernel for weak convergence control, Appendix A.3.2 describes the KGM-Stein kernels for additional control over moments, and Appendix A.3.3 presents the Riemann-Stein kernel, whose convergence properties have to-date been less well-studied.
All of the kernels that we consider have length scale parameters that need to be specified, and some also have location parameters to be specified. As a reasonably automatic default we define
$$
x _ {\star} \in \arg \max  p (x), \quad \Sigma^ {- 1} = - \nabla^ {2} \log p (x _ {\star})
$$
as a location and a matrix of characteristic length scales for  $P$  that will be used throughout. These values can typically be obtained using gradient-based optimisation, which is usually cheaper to perform compared to full approximation of  $P$ . It is assumed that  $\nabla^2\log p(x_\star)$  is positive definite in the sequel.
A.3.1 Weak Convergence Control with Langevin-Stein Kernels
The first kernel we consider, which we called the Langevin-Stein kernel in the main text, was introduced by Gorham and Mackey (2017). This Stein kernel was developed for the purpose of controlling the weak convergence of a sequence  $(Q_{n})_{n\in \mathbb{N}}\subset \mathcal{P}(\mathbb{R}^{d})$  to  $P$ . Recall that a sequence  $(Q_{n})_{n\in \mathbb{N}}$  is said to converge weakly (or in distribution) to  $P$  if  $\int f\mathrm{d}Q_n\to \int f\mathrm{d}P$  for all continuous bounded functions  $f:\mathbb{R}^d\rightarrow \mathbb{R}$ . This convergence is denoted  $Q_{n}\stackrel {\mathrm{d}}{\to}P$  in shorthand.
The problem considered in Gorham and Mackey (2017) was how to select a combination of matrix-valued kernel  $K$  (and, implicitly, a diffusion matrix  $M$ ) such that the Stein kernel  $k_{P}$  in (10) generates a KSD  $D_{P}(Q)$  in (4) for which  $D_{P}(Q_{n}) \to 0$  implies  $Q_{n} \stackrel{\mathrm{d}}{\to} P$ . Their solution was to combine the inverse multi-quadric kernel with an identity diffusion matrix;
$$
K (x, y) = \left(1 + \| x - y \| _ {\Sigma} ^ {2}\right) ^ {- \beta} I, \quad M (x) = I
$$
for  $\beta \in (0,1)$ . Provided that  $P$  has a density  $p$  for which  $\nabla \log p(x)$  is Lipschitz, and that  $P$  is distantly dissipative (see Definition 4 of Gorham and Mackey, 2017), the associated KSD enjoys weak convergence control. Technically, the results in Gorham and Mackey (2017) apply only when  $\Sigma = I$ , but Theorem 4 in Chen et al. (2019) demonstrated that they hold also for any positive definite  $\Sigma$ . Following the recommendation of several previous authors, including Chen et al. (2018, 2019); Riabiz et al. (2022), we take  $\beta = \frac{1}{2}$  throughout.
A.3.2 Moment Convergence Control with KGM-Stein Kernels
Despite its many elegant properties, weak convergence can be insufficient for applications where we are interested in integrals  $\int f\mathrm{d}P$  for which the integrand  $f:\mathbb{R}^d\to \mathbb{R}$  is unbounded. In particular, this is the case for moments of the form  $f(x) = x_1^{\alpha_1}\ldots x_d^{\alpha_d},0\neq \alpha \in \mathbb{N}_0^d$ . In such situations, we may seek also the stronger property of moment convergence control. The development of KSDs for moment convergence control was recently considered by Kanagawa et al. (2022), and we referred to their construction as the KGM-Stein kernels in the main text. (For convenience, we have adopted the initials of the authors in naming the KGM-Stein kernel.)
A sequence  $(Q_{n})_{n\in \mathbb{N}}\subset \mathcal{P}(\mathbb{R}^{d})$  is said to converge to  $P$  in the sth order moment if  $\int \| x\| ^s\mathrm{d}Q_n(x)\to \int \| x\| ^s\mathrm{d}P(x)$ . To establish convergence of moments, we need an additional condition on top of weak convergence control: uniform integrability control. A sequence of measures  $(Q_{n})_{n\in \mathbb{N}}$  is said to have uniformly integrable sth moments if for any  $\varepsilon >0$ , we can take  $r > 0$  such that
$$
\sup  _ {n \in \mathbb {N}} \int_ {\| x \| > r} \| x \| ^ {s} \mathrm {d} Q _ {n} (x) <   \varepsilon .
$$
This condition essentially states that the tail decay of the measures is well-controlled (so that it has a convergent moment). The KSD convergence  $D_P(Q_n) \to 0$  implies uniform integrability if for any  $\varepsilon > 0$ , we can take  $r_{\varepsilon} > 0$  and  $f_{\varepsilon} \in \mathcal{H}(K)$  such that
$$
S _ {P} f _ {\varepsilon} (x) \geq \| x \| ^ {s} 1 \left\{\| x \| > r _ {\varepsilon} \right\} - \varepsilon , \tag {13}
$$
i.e., the Stein-modified RKHS can approximate the (norm-weighted) indicator function arbitrarily well. Such a function  $f_{\varepsilon}$  can be explicitly constructed (while not guaranteed to be a member of the RKHS). Specifically, the choice  $f_{\varepsilon} = (1 - \iota_{\varepsilon})g$  satisfies (13) under an appropriate dissipativity condition, where  $\iota_{\varepsilon}$  is a differentiable indicator function vanishing outside a ball, and  $g(x) = -x / \sqrt{1 + \|x\|^2}$ . This motivated Kanagawa et al. (2022) to introduce the sth order KGM-Stein kernel, which is based on the matrix-valued kernel and diffusion matrix
$$
K (x, y) = \left[ \phi (\| x - y \| _ {\Sigma}) + \kappa_ {\mathrm {l i n}} (x, y) \right] I, \qquad M (x) = \left(1 + \| x - x _ {\star} \| _ {\Sigma} ^ {2}\right) ^ {\frac {s - 1}{2}} I,
$$
where  $(x,y)\mapsto \phi (\| x - y\|_{\Sigma})$  is a  $C_0^1$  universal kernel (see Barp et al., 2022b, Theorem 4.8). For comparability of our results, we take  $\phi$  to be the inverse multi-quadric  $\phi (r) = (1 + r^{2})^{-1 / 2}$ , and
$$
\kappa_ {\mathrm {l i n}} (x, y) = \frac {1 + \left(x - x _ {\star}\right) ^ {\top} \Sigma^ {- 1} \left(y - x _ {\star}\right)}{\sqrt {1 + \left\| x - x _ {\star} \right\| _ {\Sigma} ^ {2}} \sqrt {1 + \left\| y - x _ {\star} \right\| _ {\Sigma} ^ {2}}}.
$$
Here the normalised linear kernel  $\kappa_{\mathrm{lin}}$  ensures  $g\in \mathcal{H}(K)$ , while the  $C_0^1$  universal kernel  $\phi$  allows approximation of  $S_{P}\iota_{\varepsilon}g$ ; see Kanagawa et al. (2022).
A.3.3 Exploiting Geometry with Riemann-Langevin-Stein Kernels
For academic interest only, here we describe the Riemann-Stein kernel that featured in Figure 2 of the main text. This Stein kernel is motivated by the analysis of Gorham et al. (2019), who argued that the use of rapidly mixing Ito diffusions in Stein operators can lead to sharper convergence control. The Riemann-Stein kernel is based on the class of so-called Riemannian diffusions considered in Girolami and Calderhead (2011), who proposed to take the diffusion matrix  $M$  in (11) to be  $M = (\mathcal{I}_{\mathrm{prior}} + \mathcal{I}_{\mathrm{Fisher}})^{-1}$ , the inverse of the Fisher information matrix,  $\mathcal{I}_{\mathrm{Fisher}}$ , regularised using the Hessian of the negative log-prior,  $\mathcal{I}_{\mathrm{prior}}$ . For the two-dimensional illustration in Section 3.2, this leads to the diffusion matrix
$$
M (x) = \left(I + \sum_ {i = 1} ^ {n} [ \nabla f _ {i} (x) ] [ \nabla f _ {i} (x) ] ^ {\top}\right) ^ {- 1},
$$
where we recall that  $y_{i} = f_{i}(x) + \epsilon_{i}$ , where the  $\epsilon_{i}$  are independent with  $\epsilon_{i} \sim \mathcal{N}(0,1)$ , and the prior is  $x \sim \mathcal{N}(0,1)$ . For the presented experiment we paired the above diffusion matrix with the inverse multi-quadric kernel  $K(x,y) = (1 + \| x - y\|_{\Sigma}^{2})^{-\beta}$  for  $\beta = \frac{1}{2}$ . The Riemann-Stein kernel extends naturally to distributions  $P$  defined on Riemannian manifolds  $\mathcal{X}$ ; see Barp et al. (2022a) and Example 1 of Hodgkinson et al. (2020).
Unfortunately, the Riemann-Stein kernel is prohibitively expensive in most real applications, since each evaluation of  $M$  requires a full scan through the size- $n$  dataset. The computational complexity
of Stein  $\Pi$ -Thinning with the Riemann-Stein kernel is therefore  $O(m^{2}n^{2})$ , which is unfavourable compared to the  $O(m^{2}n)$  complexity in the case where the Stein kernel is not data-dependent. Furthermore, the convergence control properties of the Riemann-Stein kernel have yet to be established. For these reasons we included the Riemann-Stein kernel for illustration only; further groundwork will be required before the Riemann-Stein kernel can be practically used.
This appendix is devoted to the proof of Theorem 1. The proof is based on the recent work of Durmus and Moulines (2022), on the geometric convergence of MALA, and on the analysis of sparse (greedy) approximation of kernel discrepancies performed in Riabiz et al. (2022); these existing results are recalled in Appendix B.1. An additional technical result on preconditioned MALA is contained in Appendix B.2. The proof of Theorem 1 itself is contained in Appendix B.3.
B.1 Auxiliary Results
To precisely describe the results on which our analysis is based, we first need to introduce some notation and terminology. Let  $V: \mathcal{X} \to [1, \infty)$  and, for a function  $f: \mathcal{X} \to \mathbb{R}$  and a measure  $\mu$  on  $\mathcal{X}$ , let
$$
\| f \| _ {V} := \sup  _ {x \in \mathcal {X}} \frac {| f (x) |}{V (x)}, \quad \| \mu \| _ {V} := \sup  _ {\| f \| _ {V} \leq 1} \left| \int_ {\mathcal {X}} f \mathrm {d} \mu \right|.
$$
Recall that a  $Q$ -invariant Markov chain  $(x_{i})_{i\in \mathbb{N}}\subset \mathcal{X}$  with  $n^{\mathrm{th}}$  step transition kernel  $\mathrm{Q}^n$  is  $V$ -uniformly ergodic (see Theorem 16.0.1 of Meyn and Tweedie, 2012) if and only if  $\exists R\in [0,\infty),\rho \in (0,1)$  such that
$$
\left\| \mathrm {Q} ^ {n} (x, \cdot) - Q \right\| _ {V} \leq R \rho^ {n} V (x) \tag {14}
$$
for all initial states  $x\in \mathcal{X}$  and all  $n\in \mathbb{N}$
Although MALA (Algorithm 1) is classical (Roberts and Stramer, 2002), until recently explicit sufficient conditions for ergodicity of MALA had not been obtained. The first result we will need is due Durmus and Moulines (2022), who presented the first explicit conditions for  $V$ -uniform convergence of MALA. It applies only to standard MALA, meaning that the preconditioning matrix  $M$  appearing in Algorithm 1 is the identity matrix. The extension of this result to preconditioned MALA will be handled in Appendix B.2.
Theorem 2. Let  $Q \in \mathcal{P}(\mathbb{R}^d)$  admit a density,  $q$ , such that
(DM1) there exists  $x_0$  with  $\nabla \log q(x_0) = 0$
(DM2)  $q$  is twice continuously differentiable with  $\sup_{x\in \mathbb{R}^d}\| \nabla^2\log q(x - x_0)\| <  \infty$
(DM3) there exists  $b > 0$  and  $B \geq 0$  such that  $-\nabla^2 \log q(x - x_0) \succeq bI$  for all  $\| x - x_0 \| \geq B$ .
Then there exists  $\epsilon_0 > 0$  such that for all step sizes  $\epsilon \in (0,\epsilon_0)$ , standard  $Q$ -invariant MALA (i.e. with  $M = I$ ) is  $V$ -uniformly ergodic for  $V(x) = \exp \left(\frac{b}{16}\| x - x_0\|^2\right)$ .
Proof. This is Theorem 1 of Durmus and Moulines (2022).
The next result that we will need establishes consistency of the greedy algorithm applied to samples from a Markov chain that is  $Q$ -invariant.
Theorem 3. Let  $P, Q \in \mathcal{P}(\mathcal{X})$  with  $P \ll Q$ . Let  $k_P: \mathcal{X} \times \mathcal{X} \to \mathbb{R}$  be a Stein kernel and let  $D_P: \mathcal{X} \times \mathcal{X} \to [0, \infty]$  denote the associated KSD. Consider a  $Q$ -invariant, time-homogeneous Markov chain  $(x_i)_{i \in \mathbb{N}} \subset \mathcal{X}$  such that
$(\mathbb{R}^{+}1)$ $(x_{i})_{i\in \mathbb{N}}$  is  $V$  -uniformly ergodic, such that  $V(x)\geq \frac{\mathrm{d}P}{\mathrm{d}Q} (x)\sqrt{k_P(x)}$
$(\mathbf{R}^{+}2)$ $\sup_{i\in \mathbb{N}}\mathbb{E}\left[\frac{\mathrm{d}P}{\mathrm{d}Q} (x_i)\sqrt{k_P(x_i)} V(x_i)\right] <   \infty$
$(\mathbf{R}^{+}3)$  there exists  $\gamma >0$  such that  $\sup_{i\in \mathbb{N}}\mathbb{E}\left[\exp \left\{\gamma \max \left(1,\frac{\mathrm{d}P}{\mathrm{d}Q} (x_i)^2\right)k_P(x_i)\right\} \right] < \infty .$
Let  $P_{n,m}$  be the result of running the greedy algorithm in (5). If  $m \leq n$  and  $\log(n) = O(m^{\gamma/2})$  for some  $\gamma < 1$ , then  $D_P(P_{n,m}) \to 0$  almost surely as  $m, n \to \infty$ .
Proof. This is Theorem 3 of Riabiz et al. (2022).
B.2 Preconditioned MALA
In addition to the auxiliary results in Appendix B.1, which concern standard MALA (i.e. with  $M = I$ ), we require an elementary fact about MALA, namely that preconditioned MALA is equivalent to standard MALA under a linear transformation of the state variable. Recall that the  $M$ -preconditioned MALA algorithm is a Metropolis-Hastings algorithm whose proposal is the Euler-Maruyama discretisation of the Itô diffusion (11).
Proposition 1. Let  $M(x) \equiv M$  for a symmetric positive definite and position-independent matrix  $M \in \mathbb{R}^{d \times d}$ . Let  $Q \in \mathcal{P}(\mathbb{R}^d)$  admit a probability density function (PDF)  $q$  for which the  $Q$ -invariant diffusion  $(X_{t})_{t \geq 0}$ , given by setting  $p = q$  in (11), is well-defined. Then under the change of variables  $Y_{t} \coloneqq M^{1/2} X_{t}$ ,
$$
\mathrm {d} Y _ {t} = \frac {1}{2} (\nabla \log \tilde {q}) (Y _ {t}) \mathrm {d} t + \mathrm {d} W _ {t}, \tag {15}
$$
where  $\tilde{q} (x)\propto q(M^{-1 / 2}x)$  for all  $x\in \mathbb{R}^d$
Proof. From the chain rule,
$$
(\nabla \log \tilde {q}) (y) = \nabla_ {y} \log q (M ^ {- 1 / 2} y) = M ^ {- 1 / 2} (\nabla \log q) (M ^ {- 1 / 2} y),
$$
and thus, substituting  $Y_{t} = M^{1 / 2}X_{t}$ , (15) is equal to
$$
\begin{array}{l} \mathrm {d} X _ {t} = M ^ {- 1 / 2} \left[ \frac {1}{2} M ^ {- 1 / 2} (\nabla \log q) (M ^ {- 1 / 2} M ^ {1 / 2} X _ {t}) + \mathrm {d} W _ {t} \right] \\ = \frac {1}{2} M ^ {- 1} (\nabla \log q) (X _ {t}) + M ^ {- 1 / 2} \mathrm {d} W _ {t}, \\ \end{array}
$$
which is identical to (11) in the case where  $M(x) = M$  is constant.
Let  $Q$  and  $\tilde{Q}$  be the distributions referred to in Proposition 1, whose PDFs are respectively  $q(x)$  and  $\tilde{q}(x) \propto q(M^{-1/2}x)$ . Proposition 1 then implies that the  $M$ -preconditioned MALA algorithm applied to  $Q$  (i.e. Algorithm 1 for  $\Pi = Q$ ) is equivalent to the standard MALA algorithm (i.e.  $M = I$ ) applied to  $\tilde{Q}$ . This fact allows us to generalise the result of Theorem 2 as follows:
Corollary 1. Consider a symmetric positive definite matrix  $M \in \mathbb{R}^{d \times d}$ . Assume that conditions (DM1-3) in Theorem 2 are satisfied. Then there exists  $\epsilon_0' > 0$  and  $b' > 0$  such that for all step sizes  $\epsilon \in (0, \epsilon_0')$ , the  $M$ -preconditioned  $Q$ -invariant MALA is  $V$ -uniformly ergodic for  $V(x) = \exp\left(\frac{b'}{16}\|x - x_0\|^2\right)$ .
Proof. From Theorem 2 and Proposition 1, the result follows if we can establish (DM1-3) for  $\tilde{Q}$ , since  $M$ -preconditioned MALA is equivalent to standard MALA applied to  $\tilde{Q}$ . For a matrix  $A \in \mathbb{R}^{d \times d}$ , let  $\lambda_{\min}(A)$  and  $\lambda_{\max}(A)$  respectively denote the minimum and maximum eigenvalues of  $A$ . For (DM1) we set  $y_0 = M^{1/2}x_0$  and observe that
$$
(\nabla \log \tilde {q}) (y _ {0}) = M ^ {- 1 / 2} (\nabla \log q) (x _ {0}) = 0.
$$
For (DM2) we have that
$$
\begin{array}{l} \sup  _ {y \in \mathbb {R} ^ {d}} \| \nabla^ {2} (\log \tilde {q}) (y - y _ {0}) \| = \sup  _ {y \in \mathbb {R} ^ {d}} \| M ^ {- 1 / 2} (\nabla^ {2} \log q) (M ^ {- 1 / 2} (y - y _ {0})) M ^ {- 1 / 2} \| \\ \leq \lambda_ {\min } (M) ^ {- 1} \sup  _ {x \in \mathbb {R} ^ {d}} \| (\nabla^ {2} \log q) (x - x _ {0}) \| <   \infty . \\ \end{array}
$$
For (DM3) we have that
$$
\begin{array}{l} - (\nabla^ {2} \log \tilde {q}) (y - y _ {0}) = - M ^ {- 1 / 2} (\nabla^ {2} \log q) (M ^ {- 1 / 2} (y - y _ {0})) M ^ {- 1 / 2} \\ = - M ^ {- 1 / 2} \left(\nabla^ {2} \log q\right) \left(x - x _ {0}\right) M ^ {- 1 / 2} \succeq M ^ {1 / 2} (b I) M ^ {1 / 2} = b M ^ {- 1} \succeq b ^ {\prime} I \\ \end{array}
$$
where  $b' = b\lambda_{\max}(M)^{-1}$ , which holds for all  $\| x - x_0 \| \geq B$ , and in particular for all  $\| y - y_0 \| \geq B'$  where  $B' = B\lambda_{\max}(M)^{1/2}$ . Thus (DM1-3) are established for  $\tilde{Q}$ .
Remark 1. The choice  $M = \Sigma^{-1}$ , which sets the preconditioner matrix  $M$  equal to the inverse of the length scale matrix  $\Sigma$  used in the specification of the kernel  $K$  (c.f. Appendix A.3), leads to the elegant interpretation that Stein II-Importance Sampling applied to  $M$ -preconditioned MALA is equivalent to the Stein II-Importance Sampling applied to standard MALA (i.e. with  $M = I$ ) for the whitened target  $\tilde{P}$  with PDF  $\tilde{p}(x) \propto p(M^{-1/2}x)$ . For our experiments, however, the preconditioner matrix  $M$  was learned during a warm-up phase of MALA, since in general the curvature of  $P$  (captured by  $\Sigma$ ) and the curvature of  $\Pi$  (captured by  $M^{-1}$ ) may be different.
The route to establishing Theorem 1 has three parts. First, we establish (DM1-3) of Theorem 2 with  $Q = \Pi$ , to deduce from Corollary 1 that  $\Pi$ -invariant  $M$ -preconditioned MALA is  $V$ -uniformly ergodic. This in turn enables us to establish conditions  $(\mathbb{R}^{+}1 - 3)$  of Theorem 3, again for  $Q = \Pi$ , from which the strong consistency  $D_{P}(P_{n,m}) \stackrel{\mathrm{a.s.}}{\rightarrow} 0$  of SIIT-MALA is established. Finally, we note that  $0 \leq D_{P}(P_{n}^{\star}) \leq D_{P}(P_{n,m})$ , since the support of  $P_{n,m}$  is contained in the support of  $P_{n}^{\star}$ , and the latter is optimally weighted, whence also the strong consistency of SIIS-MALA.
Establish (DM1-3) First we establish (DM1-3) for  $Q = \Pi$ . Fix  $x_0 \in \mathbb{R}^d$ . For (DM2), first recall that the range of  $k_P$  is  $[C_1^2, \infty)$  where  $C_1 > 0$ , from Assumption 1. Since  $\log(\cdot)$  has bounded second derivatives on  $[C_1^2, \infty)$ , there is a constant  $C > 0$  such that
$$
\forall x \in \mathbb {R} ^ {d}, \quad \| \nabla^ {2} \log k _ {P} (x) \| \leq C \| \nabla^ {2} k _ {P} (x) \|.
$$
Thus, using compactness of the set  $\{x: \| x - x_0 \| \leq B_2\}$ ,
$$
\sup  _ {x \in \mathbb {R} ^ {d}} \| \nabla^ {2} \log k _ {P} (x) \| \leq C \max  \left(\underbrace {\sup  _ {\| x - x _ {0} \| \leq B _ {2}} \| \nabla^ {2} k _ {P} (x) \|} _ {<   \infty \text {b y (A 3)}}, \underbrace {\sup  _ {\| x - x _ {0} \| \geq B _ {2}} \| \nabla^ {2} k _ {P} (x) \|} _ {<   b _ {2} \| I \| \text {b y (A 4)}}\right) <   \infty . \tag {16}
$$
Now,  $\pi$  is twice differentiable as it is the product of twice differentiable functions  $p$  and  $k_{P}^{1 / 2}$  from (A1) and (A3), and moreover
$$
\sup_{x\in \mathbb{R}^{d}}\| \nabla^{2}\log \pi (x - x_{0})\| \leq \underbrace{\sup_{x\in\mathbb{R}^{d}}\|\nabla^{2}\log p(x)\|}_{<   \infty \text{by(A1)}} + \frac{1}{2}\underbrace{\sup_{x\in\mathbb{R}^{d}}\|\nabla^{2}\log k_{P}(x)\|}_{<   \infty \text{by(16)}} <   \infty ,
$$
so (DM2) is satisfied. For (DM3), first note from the chain and product rules that for all  $\| x\| \geq B_2$
$$
\nabla^ {2} \log k _ {P} (x - x _ {0}) = \underbrace {\frac {\nabla^ {2} k _ {P} (x - x _ {0})}{k _ {P} (x - x _ {0})}} _ {\preceq \left(b _ {2} / C _ {1} ^ {2}\right) I \text {b y (A 4)}} - \underbrace {\frac {[ \nabla k _ {P} (x - x _ {0}) ] [ \nabla k _ {P} (x - x _ {0}) ] ^ {\top}}{k _ {P} (x - x _ {0}) ^ {2}}} _ {\succeq 0} \preceq \frac {b _ {2}}{C _ {1} ^ {2}} I. \tag {17}
$$
Thus, for all  $\| x - x_0 \| \geq B \coloneqq \| x_0 \| + \max(B_1, B_2)$ ,
$$
- \nabla^ {2} \log \pi (x - x _ {0}) = \underbrace {- \nabla^ {2} \log p (x - x _ {0})} _ {\succeq b _ {1} I \text {b y (A 2)}} - \frac {1}{2} \underbrace {\nabla^ {2} \log k _ {P} (x - x _ {0})} _ {\preceq (b _ {2} / C _ {1} ^ {2}) I \text {b y (1 7)}} \succeq \underbrace {\left(b _ {1} - \frac {b _ {2}}{2 C _ {1} ^ {2}}\right)} _ {=: b > 0} I \tag {18}
$$
as required. The same argument establishes (DM1); from (18) we have  $\lim_{\| x\| \to \infty}\pi (x) = 0$  and since  $\pi$  is a continuously differentiable density there must exist an  $x_0$  at which  $\pi$  is locally minimised. Thus we have established (DM1-3) for  $Q = \Pi$  and we may conclude from Corollary 1 that there is an  $\epsilon_0^{\prime} > 0$  and  $b^{\prime} > 0$  such that, for all  $\epsilon \in (0,\epsilon_0^{\prime})$ , the  $\Pi$ -invariant  $M$ -preconditioned MALA chain  $(x_{i})_{i\in \mathbb{N}}$  is  $V$ -uniformly ergodic for  $V(x) = C_2\exp \Bigl (\frac{b'}{16}\| x - x_0\| ^2\Bigr)$  (since if a Markov chain is  $V$ -uniformly ergodic, then it is also  $CV$ -uniformly ergodic).
Establish  $(\mathbf{R}^{+}\mathbf{1}-3)$  The aim is now to establish conditions  $(\mathbf{R}^{+}1-3)$  of Theorem 3 for  $Q = \Pi$ . By construction  $\mathrm{d}P / \mathrm{d}\Pi = C_2 / \sqrt{k_P(x)} < C_2 / C_1 < \infty$ , where  $C_1$  and  $C_2$  were defined in Assumption 1, so that  $P \ll \Pi$ . It has already been established that  $(x_i)_{i \in \mathbb{N}}$  is  $V$ -uniformly ergodic, and further
$$
V (x) = C _ {2} \exp \left(\frac {b ^ {\prime}}{1 6} \| x - x _ {0} \| ^ {2}\right) \geq C _ {2} = \frac {\mathrm {d} P}{\mathrm {d} \Pi} (x) \sqrt {k _ {P} (x)}
$$
for all  $x$ , which establishes  $(\mathbb{R}^{+}1)$ . Let  $R$  and  $\rho$  denote constants for which the  $V$ -uniform ergodicity property (14) is satisfied. From  $V$ -uniform ergodicity, the integral  $\int V \, \mathrm{d}\Pi$  exists and
$$
\begin{array}{l} \left| \mathbb {E} \left[ \frac {\mathrm {d} P}{\mathrm {d} \Pi} \left(x _ {i}\right) \sqrt {k _ {P} \left(x _ {i}\right)} V \left(x _ {i}\right) \right] - C _ {2} \int V \mathrm {d} \Pi \right| = C _ {2} \left| \mathbb {E} [ V \left(x _ {i}\right) ] - \int V \mathrm {d} \Pi \right| \\ \leq C _ {2} R \rho^ {n} V (x _ {0}) \rightarrow 0 \\ \end{array}
$$
which establishes  $(\mathbb{R}^{+}2)$ . Fix  $\gamma > 0$ . By construction  $\mathrm{d}P / \mathrm{d}\Pi \leq C_2 / C_1$ , and thus
$$
\exp \left\{\gamma \max  \left(1, \frac {\mathrm {d} P}{\mathrm {d} \Pi} (x) ^ {2}\right) k _ {P} (x) \right\} <   \exp \left\{\tilde {\gamma} k _ {P} (x) \right\}
$$
where  $\tilde{\gamma} = \max (1,C_2 / C_1)\gamma$ . Since we have assumed that  $k_{P}$  is continuous with, from (A4),
$$
b _ {3} := \lim  _ {\| x \| \rightarrow \infty} \sup  _ {\| x \| ^ {2}} \frac {k _ {P} (x)}{\| x \| ^ {2}} <   \infty ,
$$
we may take  $\gamma$  such that  $\tilde{\gamma} b_{3} < b^{\prime} / 16$ , so that  $\| x\mapsto \exp \{\tilde{\gamma} k_{P}(x)\} \|_{V} < \infty$  and in particular
$$
\left| \mathbb {E} \left[ \exp \{\tilde {\gamma} k _ {P} (x _ {i}) \} \right] - \int \exp \{\tilde {\gamma} k _ {P} (x) \} d \Pi (x) \right| \leq \| x \mapsto \exp \{\tilde {\gamma} k _ {P} (x) \} \| _ {V} \times R \rho^ {n} V (x _ {0}) \rightarrow 0
$$