SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \| h (x) \| = \sqrt {\sum_ {i = 1} ^ {d} h _ {i} (x) ^ {2}} \leq \sqrt {\sum_ {i = 1} ^ {d} k (x , x) \| h _ {i} \| _ {\mathcal {H} (k)} ^ {2}} = C _ {0} ^ {k} \| h \| _ {\mathcal {H} (k) ^ {d}} \\ | \nabla \cdot h (x) | = \left| \sum_ {i = 1} ^ {d} \partial_ {x _ {i}} h _ {i} (x) \right| \leq \left| \sum_ {i = 1} ^ {d} \sqrt {(\partial_ {i} \otimes \partial_ {i}) k (x , x)} \| h _ {i} \| _ {\mathcal {H} (k)} \right| \leq C _ {1} ^ {k} \| h \| _ {\mathcal {H} (k) ^ {d}}. \\ \end{array}
$$
To use analogous notation as in the proof of Proposition 1, set  $C_0 \coloneqq C_0^k \| h \|_{\mathcal{H}(k)^d}$  and  $C_1 \coloneqq C_1^k \| h \|_{\mathcal{H}(k)^d}$ . Then, using Hölder's inequality and the fact that  $(a + b)^\beta \leq 2^{\beta - 1}(a^\beta + b^\beta)$  with  $\beta = \alpha / (\alpha - 1)$ , we have
$$
\begin{array}{l} \| \mathcal {S} _ {p, q} h \| _ {L ^ {1} (\pi)} = \int | \mathcal {S} _ {p, q} h | \mathrm {d} \pi = \int \left| \frac {q}{p} [ \nabla \cdot h + h \cdot \nabla \log q ] \right| \mathrm {d} \pi \\ \leq \left(\int \left(\frac {q}{p}\right) ^ {\alpha} \mathrm {d} \pi\right) ^ {\frac {1}{\alpha}} \left(\int (\nabla \cdot h + h \cdot \nabla \log q) ^ {\frac {\alpha}{\alpha - 1}} \mathrm {d} \pi\right) ^ {\frac {\alpha - 1}{\alpha}} \\ \leq 2 ^ {\frac {1}{\alpha}} \left(\int \left(\frac {q}{p}\right) ^ {\alpha} \mathrm {d} \pi\right) ^ {\frac {1}{\alpha}} \left(C _ {1} ^ {\frac {\alpha}{\alpha - 1}} + C _ {0} ^ {\frac {\alpha}{\alpha - 1}} \int \| \nabla \log q \| ^ {\frac {\alpha}{\alpha - 1}} \mathrm {d} \pi\right) ^ {\frac {\alpha - 1}{\alpha}} \\ \leq 2 ^ {\frac {1}{\alpha}} \| h \| _ {\mathcal {H} (k) ^ {d}} \left(\int \left(\frac {q}{p}\right) ^ {\alpha} \mathrm {d} \pi\right) ^ {\frac {\alpha - 1}{\alpha}} \left((C _ {1} ^ {k}) ^ {\frac {\alpha}{\alpha - 1}} + (C _ {0} ^ {k}) ^ {\frac {\alpha}{\alpha - 1}} \int \| \nabla \log q \| ^ {\frac {\alpha}{\alpha - 1}} \mathrm {d} \pi\right) ^ {\frac {\alpha - 1}{\alpha}} \\ \end{array}
$$
To prove obtain explicit computable formulae for the GF-KSD, two intermediate results are required:
Proposition 4. Let  $k$  and  $\pi$  satisfy the preconditions of Proposition 2. Then the function
$$
\mathbb {R} ^ {d} \ni x \mapsto f (x) := \frac {q (x)}{p (x)} [ \nabla_ {x} k (x, \cdot) + k (x, \cdot) \nabla \log q (x) ] \tag {3}
$$
takes values in  $\mathcal{H}(k)^d$ , is Bochner  $\pi$ -integrable and, thus,  $\xi := \int f \, \mathrm{d}\pi \in \mathcal{H}(k)^d$ .
Proof. Since  $k$  has continuous first derivatives  $x \mapsto (\partial_i \otimes \partial_i)k(x, x)$ , Lemma 4.34 of Steinwart and Christmann [2008] gives that  $(\partial_i \otimes 1)k(x, \cdot) \in \mathcal{H}(k)$ , and, thus  $f \in \mathcal{H}(k)^d$ . Furthermore,  $f: \mathbb{R}^d \to \mathcal{H}(k)^d$  is Bochner  $\pi$ -integrable since
$$
\begin{array}{l} \int \| f (x) \| _ {\mathcal {H} (k) ^ {d}} \mathrm {d} \pi (x) = \int \frac {q (x)}{p (x)} \| \nabla_ {x} k (x, \cdot) + k (x, \cdot) \nabla \log q (x) \| _ {\mathcal {H} (k) ^ {d}} \mathrm {d} \pi (x) \\ \leq \left(\int \left(\frac {q}{p}\right) ^ {\alpha} \mathrm {d} \pi\right) ^ {\frac {1}{\alpha}} \left(\int \| \nabla_ {x} k (x, \cdot) + k (x, \cdot) \nabla \log q (x) \| _ {\mathcal {H} (k) ^ {d}} ^ {\frac {\alpha}{\alpha - 1}} \mathrm {d} \pi (x)\right) ^ {\frac {\alpha - 1}{\alpha}} \\ \leq 2 ^ {\frac {1}{\alpha}} \left(\int \left(\frac {q}{p}\right) ^ {\alpha} \mathrm {d} \pi\right) ^ {\frac {1}{\alpha}} \left((C _ {1} ^ {k}) ^ {\frac {\alpha}{\alpha - 1}} + (C _ {0} ^ {k}) ^ {\frac {\alpha}{\alpha - 1}} \int \| \nabla \log q \| ^ {\frac {\alpha}{\alpha - 1}} \mathrm {d} \pi\right) ^ {\frac {\alpha - 1}{\alpha}} <   \infty \\ \end{array}
$$
where we have employed the same  $C_0^k$  and  $C_1^k$  notation as used in the proof of Proposition 2. Thus, from the definition of the Bochner integral,  $\xi = \int f\mathrm{d}\pi$  exists and is an element of  $\mathcal{H}(k)^d$ .
Proposition 5. Let  $k$  and  $\pi$  satisfy the preconditions of Proposition 2. Then
$$
\mathrm {D} _ {p, q} (\pi) ^ {2} = \iint \frac {q (x)}{p (x)} \frac {q (y)}{p (y)} k _ {q} (x, y) \mathrm {d} \pi (x) \mathrm {d} \pi (y) \tag {4}
$$
$$
\begin{array}{l} k _ {q} (x, y) = \nabla_ {x} \cdot \nabla_ {y} k (x, y) + \left\langle \nabla_ {x} k (x, y), \nabla_ {y} \log q (y) \right\rangle + \left\langle \nabla_ {y} k (x, y), \nabla_ {x} \log q (x) \right\rangle \\ + k (x, y) \left\langle \nabla_ {x} \log q (x), \nabla_ {y} \log q (y) \right\rangle . \tag {5} \\ \end{array}
$$
Proof. Let  $f$  be as in Equation (3). From Proposition 4,  $\xi = \int f \, \mathrm{d}\pi \in \mathcal{H}(k)^d$ . Moreover, since  $f$  is Bochner  $\pi$ -integrable and  $Tf = \langle h, f \rangle_{\mathcal{H}(k)^d}$  is a continuous linear functional on  $\mathcal{H}(k)^d$ , from basic properties of Bochner integrals we have  $T\xi = T \int f \, \mathrm{d}\pi = \int Tf \, \mathrm{d}\pi$ . In particular,
$$
\begin{array}{l} \langle h, \xi \rangle_ {\mathcal {H} (k) ^ {d}} = \left\langle h, \int \frac {q (x)}{p (x)} \left[ \nabla_ {x} k (x, \cdot) + k (x, \cdot) \nabla \log q (x) \right] \mathrm {d} \pi (x) \right\rangle_ {\mathcal {H} (k) ^ {d}} \\ = \int \frac {q (x)}{p (x)} \left[ \nabla_ {x} \langle h, k (x, \cdot) \rangle_ {\mathcal {H} (k) ^ {d}} + \langle h, k (x, \cdot) \rangle_ {\mathcal {H} (k) ^ {d}} \nabla \log q (x) \right] d \pi (x) \\ = \int \frac {q (x)}{p (x)} [ \nabla \cdot h (x) + h (x) \cdot \nabla \log q (x) ] d \pi (x) = \int \mathcal {S} _ {p, q} h d \pi (x) \\ \end{array}
$$
which shows that  $\xi$  is the Riesz representative of the bounded linear functional  $h\mapsto \int S_{p,q}h  \mathrm{d}\pi$  on  $\mathcal{H}(k)^d$ . It follows from Cauchy-Schwarz that the (squared) operator norm of this functional is
$$
\begin{array}{l} \mathrm {D} _ {p, q} (\pi) ^ {2} = \| \xi \| _ {\mathcal {H} (k) ^ {d}} ^ {2} = \langle \xi , \xi \rangle_ {\mathcal {H} (k) ^ {d}} = \left\langle \int \frac {q (x)}{p (x)} [ \nabla_ {x} k (x, \cdot) + k (x, \cdot) \nabla \log q (x) ]   \mathrm {d} \pi (x), \right. \\ \left. \int \frac {q (y)}{p (y)} \left[ \nabla_ {y} k (y, \cdot) + k (y, \cdot) \nabla \log q (y) \right] \mathrm {d} \pi (y) \right\rangle_ {\mathcal {H} (k) ^ {d}} \\ = \iint \frac {q (x)}{p (x)} \frac {q (y)}{p (y)} k _ {q} (x, y) d \pi (x) d \pi (y) \\ \end{array}
$$
For concreteness, we instantiate Proposition 5 in the specific case of the inverse multi-quadric kernel, since this is the kernel that we recommend in Section 2.2:
Corollary 1 (Explicit Form). For  $p, q,$  and  $\pi$  satisfying the preconditions of Proposition 2, and  $k$  the inverse multi-quadratic kernel in Equation (2), we have that
$$
\begin{array}{l} \mathrm {D} _ {p, q} (\pi) ^ {2} = \iint \frac {q (x) q (y)}{p (x) p (y)} \left\{\frac {4 \beta (\beta + 1) \| x - y \| ^ {2}}{(\sigma^ {2} + \| x - y \| ^ {2}) ^ {\beta + 2}} + 2 \beta \left[ \frac {d + \langle \nabla \log q (x) - \nabla \log q (y) , x - y \rangle}{(\sigma^ {2} + \| x - y \| ^ {2}) ^ {1 + \beta}} \right] \right. \\ \left. + \frac {\left\langle \nabla \log q (x) , \nabla \log q (y) \right\rangle}{\left(\sigma^ {2} + \| x - y \| ^ {2}\right) ^ {\beta}} \right\} \mathrm {d} \pi (x) \mathrm {d} \pi (y) \tag {6} \\ \end{array}
$$
Proof of Corollary 1. First we compute derivatives of the kernel  $k$  in Equation (2):
$$
\nabla_ {x} k (x, y) = - \frac {2 \beta}{(\sigma^ {2} + \| x - y \| ^ {2}) ^ {\beta + 1}} (x - y)
$$
$$
\nabla_ {y} k (x, y) = \frac {2 \beta}{\left(\sigma^ {2} + \| x - y \| ^ {2}\right) ^ {\beta + 1}} (x - y)
$$
$$
\nabla_ {x} \cdot \nabla_ {y} k (x, y) = - \frac {4 \beta (\beta + 1) \| x - y \| ^ {2}}{(\sigma^ {2} + \| x - y \| ^ {2}) ^ {\beta + 2}} + \frac {2 \beta d}{(\sigma^ {2} + \| x - y \| ^ {2}) ^ {\beta + 1}}
$$
Letting  $u(x) \coloneqq \nabla \log q(x)$  form a convenient shorthand, we have that
$$
\begin{array}{l} k _ {q} (x, y) := \nabla_ {x} \cdot \nabla_ {y} k (x, y) + \langle \nabla_ {x} k (x, y), u (y) \rangle + \langle \nabla_ {y} k (x, y), u (x) \rangle + k (x, y) \langle u (x), u (y) \rangle \\ = - \frac {4 \beta (\beta + 1) \| x - y \| ^ {2}}{(\sigma^ {2} + \| x - y \| ^ {2}) ^ {\beta + 2}} + 2 \beta \left[ \frac {d + \langle u (x) - u (y) , x - y \rangle}{(\sigma^ {2} + \| x - y \| ^ {2}) ^ {1 + \beta}} \right] + \frac {\langle u (x) , u (y) \rangle}{(\sigma^ {2} + \| x - y \| ^ {2}) ^ {\beta}} \\ \end{array}
$$
which, combined with Proposition 5, gives the result.
In addition to the results in the main text, here we present a spectral characterisation of GF-KSD. The following result was inspired by an impressive recent contribution to the literature on kernel Stein discrepancy due to Wynne et al. [2022], and our (informal) proof is based on an essentially identical argument:
Proposition 6 (Spectral Characterisation). Consider a positive definite isotropic kernel  $k$ , and recall that Bochner's theorem guarantees  $k(x,y) = \int e^{-i\langle s,x - y\rangle}\mathrm{d}\mu (s)$  for some  $\mu \in \mathcal{P}(\mathbb{R}^d)$ . Then, under regularity conditions that we leave implicit,
$$
\mathrm {D} _ {p, q} (\pi) ^ {2} = \int \left\| \int \frac {1}{p (x)} \left\{e ^ {- i \langle s, x \rangle} \nabla q (x) - i s e ^ {- i \langle s, x \rangle} q (x) \right\} \mathrm {d} \pi (x) \right\| _ {\mathbb {C}} ^ {2} \mathrm {d} \mu (s). \tag {7}
$$
The Fourier transform  $\widehat{\nabla q}$  of  $\nabla q$  is defined as  $\int e^{-i\langle s,x\rangle}\nabla q(x)\mathrm{d}x$ , and a basic property of the Fourier transform is that the transform of a derivative can be computed using the expression  $i\int e^{-i\langle s,x\rangle}q(x)\mathrm{d}x$ . This implies that the inner integral in (7) vanishes when  $\pi$  and  $p$  are equal. Thus we can interpret GF-KSD as a quantification of the uniformity of  $\mathrm{d}\pi/\mathrm{d}p$ , with a weighting function based on the Fourier derivative identity with regard to  $\nabla q$ .
Proof of Proposition 6. From direct calculation, and assuming derivatives and integrals can be interchanged, we have that
$$
\nabla_ {x} k (x, y) = - \int i s e ^ {- i \langle s, x - y \rangle} \mathrm {d} \mu (s), \tag {8}
$$
$$
\nabla_ {y} k (x, y) = \int i s e ^ {- i \langle s, x - y \rangle} d \mu (s), \tag {9}
$$
$$
\nabla_ {x} \cdot \nabla_ {y} k (x, y) = \int \| s \| ^ {2} e ^ {- i \langle s, x - y \rangle} \mathrm {d} \mu (s). \tag {10}
$$
Now, let
$$
\eta (x, s) = \frac {1}{p (x)} \left\{e ^ {- i \langle s, x \rangle} \nabla q (x) - i s e ^ {- i \langle s, x \rangle} q (x) \right\} = \frac {q (x)}{p (x)} \left\{e ^ {- i \langle s, x \rangle} \nabla \log q (x) - i s e ^ {- i \langle s, x \rangle} q (x) \right\}.
$$
and note through direct calculation and Equations (8) to (10) that
$$
\begin{array}{l} \int \eta (x, s) \cdot \overline {{\eta (y , s)}} \mathrm {d} \mu (s) \\ = \frac {q (x)}{p (x)} \frac {q (y)}{p (y)} \int \left\{ \begin{array}{c} \| s \| ^ {2} + i s \cdot \nabla \log q (x) \\ - i s \cdot \nabla \log q (y) + \nabla \log q (x) \cdot \nabla \log q (y) \end{array} \right\} e ^ {- i \langle s, x - y \rangle}   \mathrm {d} \mu (s) \\ = \frac {q (x)}{p (x)} \frac {q (y)}{p (y)} \left\{ \begin{array}{c} \nabla_ {x} \cdot \nabla_ {y} k (x, y) + \nabla_ {y} k (x, y) \cdot \nabla \log q (x) \\ + \nabla_ {x} k (x, y) \cdot \nabla \log q (y) + k (x, y) \nabla \log q (x) \cdot \nabla \log q (y) \end{array} \right\} \\ = \frac {q (x)}{p (x)} \frac {q (y)}{p (y)} k _ {q} (x, y). \\ \end{array}
$$
Thus, integrating with respect to  $\pi$ , and assuming that we may interchange the order of integrals, we have that
$$
\begin{array}{l} \int \left\| \int \eta (x, s) d \pi (x) \right\| _ {\mathbb {C}} ^ {2} d \mu (s) = \int \iint \eta (x, s) \cdot \overline {{\eta (y , s)}} d \pi (x) d \pi (y) d \mu (s) \\ = \iint \int \eta (x, s) \cdot \overline {{\eta (y , s)}} d \mu (s) d \pi (x) d \pi (y) \\ = \iint \frac {q (x)}{p (x)} \frac {q (y)}{p (y)} k _ {q} (x, y) d \pi (x) d \pi (y) = D _ {p, q} (\pi) ^ {2}, \\ \end{array}
$$
where the final equality is Proposition 5. This establishes the result.
To prove Theorem 1, two intermediate results are required:
Proposition 7. For an element  $\pi \in \mathcal{P}(\mathbb{R}^d)$ , assume  $Z\coloneqq \int (q / p)\mathrm{d}\pi \in (0,\infty)$ . Assume that  $k$  and  $\pi$  satisfy the preconditions of Proposition 2, and that  $\int \| \nabla \log q\| ^{\alpha /(\alpha -1)}(q / p)\mathrm{d}\pi < \infty$ . Let  $\bar{\pi}\coloneqq (q\pi) / (pZ)$ . Then  $\bar{\pi}\in \mathcal{P}(\mathbb{R}^d)$  and
$$
\mathrm {D} _ {p, q} (\pi) = Z \mathrm {D} _ {q, q} (\bar {\pi}).
$$
Proof. The assumption  $Z \in (0, \infty)$  implies that  $\bar{\pi} \in \mathcal{P}(\mathbb{R}^d)$ . Furthermore, the assumption  $\int \| \nabla \log q \|^{\alpha / (\alpha - 1)}(q / p) \, \mathrm{d}\pi < \infty$  implies that  $\int \| \nabla \log q \|^{\alpha / (\alpha - 1)} \, \mathrm{d}\bar{\pi} < \infty$ . Thus the assumptions of Proposition 2 are satisfied for both  $\pi$  and  $\bar{\pi}$ , and thus both  $\mathrm{D}_{p,q}(\pi)$  and  $\mathrm{D}_{q,q}(\bar{\pi})$  are well-defined. Now, with  $\xi$  as in Proposition 4, notice that
$$
\begin{array}{l} \mathrm {D} _ {p, q} (\pi) = \| \xi \| _ {\mathcal {H} (k) ^ {s}} = \left\| \int \frac {q (x)}{p (x)} [ \nabla k (x, \cdot) + k (x, \cdot) \nabla \log q (x) ] \mathrm {d} \pi (x) \right\| _ {\mathcal {H} (k) ^ {d}} \\ = \left\| \int [ \nabla k (x, \cdot) + k (x, \cdot) \nabla \log q (x) ] Z \mathrm {d} \bar {\pi} (x) \right\| _ {\mathcal {H} (k) ^ {d}} = Z \mathrm {D} _ {q, q} (\bar {\pi}), \\ \end{array}
$$
Proposition 8. Let  $f: \mathbb{R}^d \to [0, \infty)$  and  $\pi \in \mathcal{P}(\mathbb{R}^d)$ . Then  $\int f^\alpha \, \mathrm{d}\pi > 0 \Rightarrow \int f \, \mathrm{d}\pi > 0$ , for all  $\alpha \in (0, \infty)$ .
Proof. From the definition of the Lebesgue integral, we have that  $\int f^{\alpha} \mathrm{d}\pi = \sup \left\{\int s \mathrm{d}\pi : s \text{ a simple function with } 0 \leq s \leq f^{\alpha}\right\} > 0$ . Thus there exists a simple function  $s = \sum_{i=1}^{m} s_i 1_{S_i}$  with  $0 \leq s \leq f^{\alpha}$  and  $\int s \mathrm{d}\pi > 0$ . Here the  $s_i \in \mathbb{R}$  and the measurable sets  $S_i \subset \mathbb{R}^d$  are disjoint. In particular, it must be the case that at least one of the coefficients  $s_i$  is positive; without loss of generality suppose  $s_1 > 0$ . Then  $\tilde{s} := s_1^{1/\alpha} 1_{S_1}$  is a simple function with  $0 \leq \tilde{s} \leq f$  and  $\int \tilde{s} \mathrm{d}\pi > 0$ . It follows that  $\int f \mathrm{d}\pi = \sup \left\{\int s \mathrm{d}\pi : s \text{ a simple function with } 0 \leq s \leq f\right\} > 0$ .
Proof of Theorem 1. Since  $\int (q / p)^{\alpha} \mathrm{d}\pi_n \in (0, \infty)$  and  $q / p \geq 0$ , from Proposition 8 we have, for each  $n$ , that  $Z_{n} := \int q / p \mathrm{d}\pi_{n} > 0$ . Thus the assumptions of Proposition 7 are satisfied by  $k$  and each  $\pi_{n}$ , which guarantees that  $\mathrm{D}_{p,q}(\pi_n) = Z_n \mathrm{D}_{q,q}(\bar{\pi}_n)$  where  $\bar{\pi}_n := (q\pi_n) / (pZ_n) \in \mathcal{P}(\mathbb{R}^d)$ .
Now, since  $\mathrm{W}_1(\pi_n,p;q / p)\to 0$ , taking  $f = 1$  we obtain  $Z_{n} = \int fq / p\mathrm{d}\pi_{n}\rightarrow \int fq / p\mathrm{d}p = 1$ . In addition, note that
$$
\begin{array}{l} \mathrm {W} _ {1} (\bar {\pi} _ {n}, q) = \sup  _ {L (f) \leq 1} \left| \int f d \bar {\pi} _ {n} - \int f d q \right| \\ = \sup  _ {L (f) \leq 1} \left| \int f (0) + [ f (x) - f (0) ] d \bar {\pi} _ {n} (x) - \int f (0) + [ f (x) - f (0) ] d q (x) \right| \\ = \sup  _ {L (f) \leq 1} \left| \int [ f (x) - f (0) ] d \bar {\pi} _ {n} (x) - \int [ f (x) - f (0) ] d q (x) \right| \\ = \sup_{\substack{L(f)\leq 1\\ f(0) = 0}}\left|\int f  \mathrm{d}\bar{\pi}_{n} - \int f  \mathrm{d}q\right| \\ \end{array}
$$
Thus, from the triangle inequality, we obtain the bound
$$
\begin{array}{l} \mathrm{W}_{1}(\bar{\pi}_{n},q) = \sup_{\substack{L(f)\leq 1\\ f(0) = 0}}\bigg|\int f  \mathrm{d}\bar{\pi}_{n} - \int f  \mathrm{d}q\bigg| \\ = \sup_{\substack{L(f)\leq 1\\ f(0) = 0}}\left|\int \frac{fq}{pZ_{n}}  \mathrm{d}\pi_{n} - \int \frac{fq}{p}  \mathrm{d}p\right| \\ \leq \sup_{\substack{L(f)\leq 1\\ f(0) = 0}}\left|\int \frac{fq}{pZ_{n}}  \mathrm{d}\pi_{n} - \int \frac{fq}{p}  \mathrm{d}\pi_{n}\right| + \sup_{\substack{L(f)\leq 1\\ f(0) = 0}}\left|\int \frac{fq}{p}  \mathrm{d}\pi_{n} - \int \frac{fq}{p}  \mathrm{d}p\right| \\ = \underbrace{\left(\frac{1 - Z_{n}}{Z_{n}}\right)}_{\to 0}\underbrace{\sup_{\substack{L(f)\leq 1\\ f(0) = 0}}\left|\int \frac{fq}{p}  \mathrm{d}\pi_{n}\right|}_{(*)} + \underbrace{\operatorname{W}_{1}\left(\pi_{n},p;\frac{q}{p}\right)}_{\to 0} \\ \end{array}
$$
as  $n\to \infty$  . For  $(*)$  , since  $q / p\geq 0$  , the supremum is realised by  $f(x) = \| x\|$  and
$$
(*) = \int \| x \| \frac {q (x)}{p (x)} d \pi_ {n} (x) <   \infty .
$$
Thus we have established that  $\mathrm{W}_1(\bar{\pi}_n,q)\to 0$ . Since  $\nabla \log q$  is Lipschitz with  $\int \| \nabla \log q\| ^2\mathrm{d}q < \infty$  and  $k$  has continuous and bounded second derivatives, the standard kernel Stein discrepancy has 1-Wasserstein convergence detection [Proposition 9 of Gorham and Mackey, 2017], meaning that  $\mathrm{W}_1(\bar{\pi}_n,q)\to 0$  implies that  $\mathrm{D}_{q,q}(\bar{\pi}_n)\to 0$  and thus, since  $Z_{n}\rightarrow 1$ ,  $\mathrm{D}_{p,q}(\pi_n)\to 0$ . This completes the proof.
To prove Proposition 3, an intermediate result is required:
Proposition 9. Let  $k(x,y) = \phi (x - y)$  be a kernel with  $\phi$  twice differentiable and let  $q\in \mathcal{P}(\mathbb{R}^d)$  with  $\nabla \log q$  well-defined. Then  $k_{q}(x,x) = -\Delta \phi (0) + \phi (0)\| \nabla \log q(x)\|^{2}$ , where  $\Delta = \nabla \cdot \nabla$  and  $k_{q}$  was defined in Proposition 5.
Proof. First, note that we must have  $\nabla \phi(0) = 0$ , else the symmetry property of  $k$  would be violated. Now,  $\nabla_x k(x,y) = (\nabla \phi)(x - y)$ ,  $\nabla_y k(x,y) = -(\nabla \phi)(x - y)$  and  $\nabla_x \cdot \nabla_y k(x,y) = -\Delta \phi(x - y)$ . Thus  $\left.\nabla_x k(x,y)\right|_{y=x} = \left.\nabla_y k(x,y)\right|_{x=y} = 0$  and  $\left.\nabla_x \cdot \nabla_y k(x,y)\right|_{x=y} = -\Delta \phi(0)$ . Plugging these expressions into Equation (5) yields the result.
Proof of Proposition 3. Let  $k_{q}$  be defined as in Proposition 5. From Cauchy-Schwarz, we have that  $k_{q}(x,y) \leq \sqrt{k_{q}(x,x)}\sqrt{k_{q}(y,y)}$ , and plugging this into Proposition 5 we obtain the bound
$$
\mathrm {D} _ {p, q} (\pi) \leq \int \frac {q (x)}{p (x)} \sqrt {k _ {q} (x , x)} \mathrm {d} \pi (x) \tag {11}
$$
For a radial kernel  $k(x,y) = \phi (x - y)$  with  $\phi$  twice differentiable, we have  $\phi (0) > 0$  (else  $k$  must be the zero kernel, since by Cauchy-Schwarz  $|k(x,y)|\leq \sqrt{k(x,x)}\sqrt{k(y,y)} = \phi (0)$  for all  $x,y\in \mathbb{R}^d$ ), and  $k_{q}(x,x) = -\Delta \phi (0) + \phi (0)\| \nabla \log q(x)\|^{2}$  (from Proposition 9). Plugging this expression into Equation (11) and applying Jensen's inequality gives that
$$
\mathrm {D} _ {p, q} (\pi) ^ {2} \leq \int \frac {q (x) ^ {2}}{p (x) ^ {2}} \left[ - \Delta \phi (0) + \phi (0) \| \nabla \log q (x) \| ^ {2} \right] \mathrm {d} \pi (x).
$$
Now we may pick a choice of  $p, q$  and  $(\pi_n)_{n \in \mathbb{N}}$  ( $\pi_n \stackrel{\mathrm{d}}{\to} p$ ) for which this bound can be made arbitrarily small. One example is  $q = \mathcal{N}(0, 1)$ ,  $p = \mathcal{N}(0, \sigma^2)$  (any fixed  $\sigma > 1$ ), for which we have
$$
\mathrm {D} _ {p, q} (\pi) ^ {2} \leq \int \sigma^ {2} \exp (- \gamma \| x \| ^ {2}) \left[ - \Delta \phi (0) + \phi (0) \| x \| ^ {2} \right] \mathrm {d} \pi (x)
$$
where  $\gamma = 1 - \sigma^{-2} > 0$ . Then it is clear that, for example, the sequence  $\pi_n = \delta (ne_1)$  (where  $e_1 = [1,0,\dots ,0]^{\top}$ ) satisfies the assumptions of Proposition 2 and, for this choice,
$$
\mathrm {D} _ {p, q} \left(\pi_ {n}\right) ^ {2} \leq \sigma^ {2} \exp \left(- \gamma n ^ {2}\right)\left[ - \Delta \phi (0) + \phi (0) n ^ {2} \right]\rightarrow 0
$$
and yet  $\pi_n \xrightarrow{\mathrm{d}} p$ , as claimed.
Proof of Theorem 2. Since  $\inf_{x\in \mathbb{R}^d}q(x) / p(x) > 0$ , for each  $n$  we have  $Z_{n}\coloneqq \int q / p\mathrm{d}\pi_{n} > 0$  and, furthermore, the assumption  $\int \| \nabla \log q\| ^{\alpha /(\alpha -1)}(q / p)\mathrm{d}\pi_n < \infty$  implies that  $\int \| \nabla \log q\|^{\alpha /(\alpha -1)}\mathrm{d}\pi_n < \infty$ . Thus the assumptions of Proposition 7 are satisfied by  $k$  and each  $\pi_{n}$ , and thus we have  $\mathrm{D}_{p,q}(\pi_n) = Z_n\mathrm{D}_{q,q}(\bar{\pi}_n)$  where  $\bar{\pi}_n\coloneqq (q\pi_n) / (pZ_n)\in \mathcal{P}(\mathbb{R}^d)$ .
From assumption,  $Z_{n} \geq \inf_{x \in \mathbb{R}^{d}} q(x) / p(x)$  is bounded away from 0. Thus if  $\mathrm{D}_{p,q}(\pi_n) \to 0$  then  $\mathrm{D}_{q,q}(\bar{\pi}_n) \to 0$ . Furthermore, since  $q \in \mathcal{Q}(\mathbb{R}^d)$  and the inverse multi-quadric kernel  $k$  is used, the standard kernel Stein discrepancy has convergence control, meaning that  $\mathrm{D}_{q,q}(\bar{\pi}_n) \to 0$  implies  $\bar{\pi}_n \stackrel{\mathrm{d}}{\to} q$  [Theorem 8 of Gorham and Mackey, 2017]. It therefore suffices to show that  $\bar{\pi}_n \stackrel{\mathrm{d}}{\to} q$  implies  $\pi_n \stackrel{\mathrm{d}}{\to} p$ .
From the Portmanteau theorem,  $\pi_n \stackrel{\mathrm{d}}{\to} p$  is equivalent to  $\int g \, \mathrm{d}\pi_n \to \int g \, \mathrm{d}p$  for all functions  $g$  which are continuous and bounded. Thus, for an arbitrary continuous and bounded function  $g$ , consider  $f = gp / q$ , which is also continuous and bounded. Then, since  $\bar{\pi}_n \stackrel{\mathrm{d}}{\to} q$ , we have (again from the Portmanteau theorem) that  $Z_n^{-1} \int g \, \mathrm{d}\pi_n = \int f \, \mathrm{d}\bar{\pi}_n \to \int f \, \mathrm{d}q = \int g \, \mathrm{d}p$ . Furthermore, the specific choice  $g = 1$  shows that  $Z_n^{-1} \to 1$ , and thus  $\int g \, \mathrm{d}\pi_n \to \int g \, \mathrm{d}p$  in general. Since  $g$  was arbitrary, we have established that  $\pi_n \stackrel{\mathrm{d}}{\to} p$ , completing the proof.
To prove Theorem 3, an intermediate result is required:
Proposition 10. Let  $Q \in \mathcal{P}(\mathbb{R}^d)$  and let  $k_q: \mathbb{R}^d \times \mathbb{R}^d \to \mathbb{R}$  be a reproducing kernel with  $\int k_q(x, \cdot) \, \mathrm{d}q = 0$  for all  $x \in \mathbb{R}^d$ . Let  $(x_n)_{n \in \mathbb{N}}$  be a sequence of random variables independently sampled from  $q$  and assume that  $\int \exp \{\gamma k_q(x, x)\} \, \mathrm{d}q(x) < \infty$  for some  $\gamma > 0$ . Then
$$
\mathrm {D} _ {q, q} \left(\frac {1}{n} \sum_ {i = 1} ^ {n} \delta (x _ {i})\right)\rightarrow 0
$$
almost surely as  $n\to \infty$
Proof. This is Lemma 4 in Riabiz et al. [2022], specialised to the case where samples are independent and identically distributed. Although not identical to the statement in Riabiz et al. [2022], one obtains this result by following an identical argument and noting that the expectation of  $k_{q}(x_{i}, x_{j})$  is identically 0 when  $i \neq j$  (due to independence of  $x_{i}$  and  $x_{j}$ ), so that bounds on these terms are not required.
Proof of Theorem 3. Since  $\pi_{n}$  has finite support, all conditions of Theorem 2 are satisfied. Thus it is sufficient to show that almost surely  $\mathrm{D}_{p,q}(\pi_n)\to 0$ . To this end, we follow Theorem 3 of Riabiz et al. [2022] and introduce the classical importance weights  $w_{i} = p(x_{i}) / q(x_{i})$ , which are well-defined since  $q > 0$ . The normalised weights  $\bar{w}_i = w_i / W_n$ ,  $W_{n}\coloneqq \sum_{j = 1}^{n}w_{j}$  satisfy  $0\leq \bar{w}_1,\ldots ,\bar{w}_n$  and  $\bar{w}_1 + \dots +\bar{w}_n = 1$ , and thus the optimality of  $w^{*}$ , together with the integral form of the GF-KSD in Equation (4), gives that
$$
\begin{array}{l} \mathrm {D} _ {p, q} \left(\sum_ {i = 1} ^ {n} w _ {i} ^ {*} \delta \left(x _ {i}\right)\right) \leq \mathrm {D} _ {p, q} \left(\sum_ {i = 1} ^ {n} \bar {w} _ {i} \delta \left(x _ {i}\right)\right) = \frac {1}{W _ {n}} \sqrt {\sum_ {i = 1} ^ {n} \sum_ {j = 1} ^ {n} k _ {q} \left(x _ {i} , x _ {j}\right)} \\ = \left(\frac {1}{n} W _ {n}\right) ^ {- 1} \mathrm {D} _ {q, q} \left(\frac {1}{n} \sum_ {i = 1} ^ {n} \delta \left(x _ {i}\right)\right) \tag {12} \\ \end{array}
$$