SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\sum_ {\ell \ell^ {\prime}} \lambda_ {\ell} \lambda_ {\ell^ {\prime}} g _ {\ell \ell^ {\prime}} f _ {\ell \ell^ {\prime}} = \sum_ {\ell} \lambda_ {\ell} \left\langle f e _ {\ell} K \left[ g e _ {\ell} \right] \right\rangle . \tag {20}
$$
Suppose that  $f, g$  are probability densities with respect to  $\mu$  that are bounded by  $C$ . Then
$$
0 \leq \sum_ {\ell \ell^ {\prime}} \lambda_ {\ell} \lambda_ {\ell^ {\prime}} g _ {\ell \ell^ {\prime}} f _ {\ell \ell^ {\prime}} \leq C ^ {2} \| \lambda \| _ {2} ^ {2}. \tag {21}
$$
Proof. We prove each claim, starting with (15). Clearly
$$
\begin{array}{l} \| \theta_ {f} \| _ {\mathcal {H} _ {K}} ^ {2} = \| K [ f ] \| _ {\mathcal {H} _ {K}} ^ {2} \\ = \left\| \int K (x, \cdot) f (x) \mathrm {d} \mu (x) \right\| _ {\mathcal {H} _ {K}} ^ {2} \\ = \iint \langle K (x, \cdot), K (y, \cdot) \rangle_ {\mathcal {H} _ {K}} f (x) f (y) d \mu (x) d \mu (y) \\ = \iint K (x, y) f (x) f (y) d \mu (x) d \mu (y) \\ = \sum_ {\ell} \lambda_ {\ell} f _ {\ell} ^ {2} \\ \end{array}
$$
as required. The second claim (16) follows immediately from (15) by definition. For (17) by orthogonality we have
$$
\begin{array}{l} \| K [ f ] \| _ {2} ^ {2} = \| \sum_ {\ell} \lambda_ {\ell} f _ {\ell} e _ {\ell} \| _ {2} ^ {2} \\ = \sum_ {\ell} \lambda_ {\ell} ^ {2} f _ {\ell} ^ {2}. \\ \end{array}
$$
For (18) by the definition of  $K[\cdot ]$  we have
$$
\begin{array}{l} \sum_ {\ell} \lambda_ {\ell} f _ {\ell} g _ {\ell} = \left\langle \left(\sum_ {\ell} \lambda_ {\ell} f _ {\ell} e _ {\ell}\right) g \right\rangle \\ = \langle K [ f ] g \rangle . \\ \end{array}
$$
For (19) we can write
$$
\begin{array}{l} \sum_ {\ell \ell^ {\prime}} \lambda_ {\ell} \lambda_ {\ell^ {\prime}} h _ {\ell \ell^ {\prime}} f _ {\ell} g _ {\ell^ {\prime}} = \sum_ {\ell} \lambda_ {\ell} f _ {\ell} \left\langle \left(\sum_ {\ell^ {\prime}} \lambda_ {\ell^ {\prime}} g _ {\ell^ {\prime}} e _ {\ell^ {\prime}}\right) h e _ {\ell} \right\rangle \\ = \sum_ {\ell} \lambda_ {\ell} f _ {\ell} \left\langle K [ g ] h e _ {\ell} \right\rangle \\ = \langle K [ g ] h K [ f ] \rangle . \\ \end{array}
$$
Finally, for (20) we have
$$
\begin{array}{l} \sum_ {\ell \ell^ {\prime}} \lambda_ {\ell} \lambda_ {\ell^ {\prime}} f _ {\ell \ell^ {\prime}} g _ {\ell \ell^ {\prime}} = \sum_ {\ell} \lambda_ {\ell} \left\langle \left(\sum_ {\ell^ {\prime}} \lambda_ {\ell^ {\prime}} g _ {\ell \ell^ {\prime}} e _ {\ell^ {\prime}}\right) f e _ {\ell} \right\rangle \\ = \sum_ {\ell} \lambda_ {\ell} \left\langle K \left[ g e _ {\ell} \right] f e _ {\ell} \right\rangle . \\ \end{array}
$$
Suppose now that  $f, g$  are probability densities with respect to  $\mu$  that are bounded by  $C > 0$ . Let  $X, Y$  be independent random variables following the densities  $f, g$ . Then
$$
\begin{array}{l} \sum_ {\ell \ell^ {\prime}} \lambda_ {\ell} \lambda_ {\ell^ {\prime}} f _ {\ell \ell^ {\prime}} g _ {\ell \ell^ {\prime}} = \mathbb {E} \left[ \left(\sum_ {\ell} \lambda_ {\ell} e _ {\ell} (X) e _ {\ell} (Y)\right) ^ {2} \right] \\ \leq C ^ {2} \int_ {\mathcal {X}} \int_ {\mathcal {X}} \left(\sum_ {\ell} \lambda_ {\ell} e _ {\ell} (x) e _ {\ell} (y)\right) ^ {2} \mathrm {d} \mu (x) \mathrm {d} \mu (y) \\ = C ^ {2} \| \lambda \| _ {2} ^ {2} \\ \end{array}
$$
as claimed, where we used that the  $e_{\ell}$  are orthonormal.
D.2 Mean and Variance Computation
We take  $\pi = \delta /2$ . Our statistic reads
$$
\begin{array}{l} - T (X, Y, Z) + \gamma (X, Y, \pi) = \left\langle \theta_ {\widehat {P} _ {Z}} - (\bar {\pi} \theta_ {\widehat {P} _ {X}} + \pi \theta_ {\widehat {P} _ {Y}}), \theta_ {\widehat {P} _ {X}} - \theta_ {\widehat {P} _ {Y}} \right\rangle_ {u, \mathcal {H} _ {K}} \\ = \frac {1}{n m} \underbrace {\sum_ {i j} k (X _ {i} , Z _ {j})} _ {\text {I}} - \frac {1}{n m} \underbrace {\sum_ {i j} k (Y _ {i} , Z _ {j})} _ {\text {I I}} - \frac {2 \bar {\pi}}{n (n - 1)} \underbrace {\sum_ {i <   i ^ {\prime}} k (X _ {i} , X _ {i ^ {\prime}})} _ {\text {I I I}} \\ + \frac {2 \pi}{n (n - 1)} \underbrace {\sum_ {i <   i ^ {\prime}} k (Y _ {i} , Y _ {i ^ {\prime}})} _ {\text {I V}} + \frac {\bar {\pi} - \pi}{n ^ {2}} \underbrace {\sum_ {i j} k (X _ {i} , Y _ {j})} _ {\text {V}}. \\ \end{array}
$$
Recall that  $\nu = \arg \min_{\nu' \in \mathbb{R}} \mathrm{MMD}(P_Z, \bar{\nu}' P_X + \nu' P_Y)$ . Let us write  $z = \bar{\nu} x + \nu y + r$  for  $1 - \bar{\nu} = \nu$ , where the residual term is denoted as  $r \in L^2(\mu)$ . Let  $\theta_r = \int r(t) K(t, \cdot) \mu(\mathrm{d}t)$  be the mean embedding of  $r$ . Under both hypotheses we assume that  $\| \theta_r \|_{\mathcal{H}_K} \leq R \cdot \mathrm{MMD}(P_X, P_Y)$ , moreover  $\langle \theta_r, \theta_{P_Y} - \theta_{P_X} \rangle_{\mathcal{H}_K} = 0$  by the definition of  $\nu$ . We look at each of the  $5 + \binom{5}{2} = 15$  terms of the variance separately.
$$
\begin{array}{l} \operatorname {v a r} (1) = \sum_ {\ell \ell^ {\prime}} \lambda_ {\ell} \lambda_ {\ell^ {\prime}} \left\{n (n - 1) m \left(z _ {\ell \ell^ {\prime}} - z _ {\ell} z _ {\ell^ {\prime}}\right) x _ {\ell} x _ {\ell^ {\prime}} + n m (m - 1) \left(x _ {\ell \ell^ {\prime}} - x _ {\ell} x _ {\ell^ {\prime}}\right) z _ {\ell} z _ {\ell^ {\prime}} \right. \\ \left. + n m (x _ {\ell \ell^ {\prime}} z _ {\ell \ell^ {\prime}} - x _ {\ell} x _ {\ell^ {\prime}} z _ {\ell} z _ {\ell^ {\prime}}) \right\} \\ \end{array}
$$
$$
\begin{array}{l} \operatorname {v a r} (\mathrm {I I}) = \sum_ {\ell \ell^ {\prime}} \lambda_ {\ell} \lambda_ {\ell^ {\prime}} \left\{n (n - 1) m \left(z _ {\ell \ell^ {\prime}} - z _ {\ell} z _ {\ell^ {\prime}}\right) y _ {\ell} y _ {\ell^ {\prime}} + n m (m - 1) \left(y _ {\ell \ell^ {\prime}} - y _ {\ell} y _ {\ell^ {\prime}}\right) z _ {\ell} z _ {\ell^ {\prime}} \right. \\ \left. + n m \left(y _ {\ell \ell^ {\prime}} z _ {\ell \ell^ {\prime}} - y _ {\ell} y _ {\ell^ {\prime}} z _ {\ell} z _ {\ell^ {\prime}}\right) \right\} \\ \end{array}
$$
$$
\operatorname {v a r} (\mathrm {I I I}) = \sum_ {\ell \ell^ {\prime}} \lambda_ {\ell} \lambda_ {\ell^ {\prime}} \Big \{\binom {n} {2} (x _ {\ell \ell^ {\prime}} ^ {2} - x _ {\ell} ^ {2} x _ {\ell^ {\prime}} ^ {2}) + (\binom {n} {2} ^ {2} - \binom {n} {2} - \binom {4} {2} \binom {n} {4}) (x _ {\ell \ell^ {\prime}} - x _ {\ell} x _ {\ell^ {\prime}}) x _ {\ell} x _ {\ell^ {\prime}} \Big \}
$$
$$
\operatorname {var}(\mathsf{IV}) = \sum_{\ell \ell^{\prime}}\lambda_{\ell}\lambda_{\ell^{\prime}}\Bigl\{\binom {n}{2}(y_{\ell \ell^{\prime}}^{2} - y_{\ell}^{2}y_{\ell^{\prime}}^{2}) + (\binom {n}{2}^{2} - \binom {n}{2} - \binom {4}{2}\binom {n}{4})(y_{\ell \ell^{\prime}} - y_{\ell}y_{\ell^{\prime}})y_{\ell}y_{\ell^{\prime}}\Bigr \}
$$
$$
\begin{array}{l} \operatorname {v a r} (V) = \sum_ {\ell \ell^ {\prime}} \lambda_ {\ell} \lambda_ {\ell^ {\prime}} \left\{n ^ {2} (n - 1) \left(y _ {\ell \ell^ {\prime}} - y _ {\ell} y _ {\ell^ {\prime}}\right) x _ {\ell} x _ {\ell^ {\prime}} + n ^ {2} (n - 1) \left(x _ {\ell \ell^ {\prime}} - x _ {\ell} x _ {\ell^ {\prime}}\right) y _ {\ell} y _ {\ell^ {\prime}} \right. \\ \left. + n ^ {2} \left(x _ {\ell \ell^ {\prime}} y _ {\ell \ell^ {\prime}} - x _ {\ell} x _ {\ell^ {\prime}} y _ {\ell} y _ {\ell^ {\prime}}\right) \right\} \\ \end{array}
$$
For the cross terms we obtain
$$
\begin{array}{l} \operatorname {c o v} (I, I I) = \sum_ {\ell \ell^ {\prime}} \lambda_ {\ell} \lambda_ {\ell^ {\prime}} n ^ {2} m \left(z _ {\ell \ell^ {\prime}} - z _ {\ell} z _ {\ell^ {\prime}}\right) x _ {\ell} y _ {\ell^ {\prime}} \\ \operatorname {c o v} (1, \mathrm {I I I}) = \sum_ {\ell \ell^ {\prime}} \lambda_ {\ell} \lambda_ {\ell^ {\prime}} n (n - 1) m \left(x _ {\ell \ell^ {\prime}} - x _ {\ell} x _ {\ell^ {\prime}}\right) z _ {\ell} x _ {\ell^ {\prime}} \\ \operatorname {c o v} (I, I V) = 0 \\ \operatorname {c o v} (1, \mathsf {V}) = \sum_ {\ell \ell^ {\prime}} \lambda_ {\ell} \lambda_ {\ell^ {\prime}} n ^ {2} m \left(x _ {\ell \ell^ {\prime}} - x _ {\ell} x _ {\ell^ {\prime}}\right) z _ {\ell} y _ {\ell^ {\prime}} \\ \operatorname {c o v} (\mathrm {I I}, \mathrm {I I I}) = 0 \\ \operatorname {c o v} (\mathbb {I I}, \mathbb {I V}) = \sum_ {\ell \ell^ {\prime}} \lambda_ {\ell} \lambda_ {\ell^ {\prime}} n (n - 1) m \left(y _ {\ell \ell^ {\prime}} - y _ {\ell} y _ {\ell^ {\prime}}\right) z _ {\ell} y _ {\ell^ {\prime}} \\ \operatorname {c o v} (\mathrm {I I}, \mathrm {V}) = \sum_ {\ell \ell^ {\prime}} \lambda_ {\ell} \lambda_ {\ell^ {\prime}} n ^ {2} m \left(y _ {\ell \ell^ {\prime}} - y _ {\ell} y _ {\ell^ {\prime}}\right) z _ {\ell} x _ {\ell^ {\prime}} \\ \operatorname {c o v} (\mathrm {I I I}, \mathrm {I V}) = 0 \\ \operatorname {c o v} (I I I, V) = \sum_ {\ell \ell^ {\prime}} \lambda_ {\ell} \lambda_ {\ell^ {\prime}} n ^ {2} (n - 1) \left(x _ {\ell \ell^ {\prime}} - x _ {\ell} x _ {\ell^ {\prime}}\right) x _ {\ell} y _ {\ell^ {\prime}} \\ \operatorname {c o v} (\mathrm {I V}, \mathrm {V}) = \sum_ {\ell \ell^ {\prime}} \lambda_ {\ell} \lambda_ {\ell^ {\prime}} n ^ {2} (n - 1) (y _ {\ell \ell^ {\prime}} - y _ {\ell} y _ {\ell^ {\prime}}) y _ {\ell} x _ {\ell^ {\prime}}. \\ \end{array}
$$
Note that  $\binom{n}{2}^2 -\binom{n}{2} -\binom{n}{2}\binom{n}{4} = n(n-1)^2 -n(n-1)$ . Collecting terms, and simplifying, we get the coefficient of the  $\frac{1}{n}$  term:
$$
\begin{array}{l} \operatorname {C o e f} \left(\frac {1}{n}\right) = \sum_ {\ell , \ell^ {\prime}} \lambda_ {\ell} \lambda_ {\ell^ {\prime}} \left(\underbrace {(x _ {\ell \ell^ {\prime}} - x _ {\ell} x _ {\ell^ {\prime}}) z _ {\ell} z _ {\ell^ {\prime}}} _ {\operatorname {v a r} (\mathrm {I})} + \underbrace {(y _ {\ell \ell^ {\prime}} - y _ {\ell} y _ {\ell^ {\prime}}) z _ {\ell} z _ {\ell^ {\prime}}} _ {\operatorname {v a r} (\mathrm {I I})} + \underbrace {4 \bar {\pi} ^ {2} (x _ {\ell \ell^ {\prime}} - x _ {\ell} x _ {\ell^ {\prime}}) x _ {\ell} x _ {\ell^ {\prime}}} _ {\operatorname {v a r} (\mathrm {I I I})}\right) \\ + \underbrace {4 \pi^ {2} (y _ {\ell \ell^ {\prime}} - y _ {\ell} y _ {\ell^ {\prime}}) y _ {\ell} y _ {\ell^ {\prime}}} _ {\operatorname {v a r} (\mathrm {I V})} + \underbrace {(\bar {\pi} - \pi) ^ {2} (y _ {\ell \ell^ {\prime}} - y _ {\ell} y _ {\ell^ {\prime}}) x _ {\ell} x _ {\ell^ {\prime}} + (\bar {\pi} - \pi) ^ {2} (x _ {\ell \ell^ {\prime}} - x _ {\ell} x _ {\ell^ {\prime}}) y _ {\ell} y _ {\ell^ {\prime}}} _ {\operatorname {v a r} (\mathrm {V})} \\ - \underbrace {4 \bar {\pi} (x _ {\ell \ell^ {\prime}} - x _ {\ell} x _ {\ell^ {\prime}}) z _ {\ell} x _ {\ell^ {\prime}}} _ {\operatorname {c o v} (\mathrm {I}, \mathrm {I I I})} + \underbrace {2 (\bar {\pi} - \pi) (x _ {\ell \ell^ {\prime}} - x _ {\ell} x _ {\ell^ {\prime}}) z _ {\ell} y _ {\ell^ {\prime}}} _ {\operatorname {c o v} (\mathrm {I}, \mathrm {V})} \\ - \underbrace {4 \pi \left(y _ {\ell \ell^ {\prime}} - y _ {\ell} y _ {\ell^ {\prime}}\right) z _ {\ell} y _ {\ell^ {\prime}}} _ {\operatorname {c o v} (\mathrm {I I}, \mathrm {I V})} - \underbrace {2 (\bar {\pi} - \pi) \left(y _ {\ell \ell^ {\prime}} - y _ {\ell} y _ {\ell^ {\prime}}\right) z _ {\ell} x _ {\ell^ {\prime}}} _ {\operatorname {c o v} (\mathrm {I I}, \mathrm {V})} \\ \left. - \underbrace {4 \bar {\pi} (\bar {\pi} - \pi) (x _ {\ell \ell^ {\prime}} - x _ {\ell} x _ {\ell^ {\prime}}) x _ {\ell} y _ {\ell^ {\prime}}} _ {\operatorname {c o v} (\mathrm {I I I}, \mathrm {V})} + \underbrace {4 \pi (\bar {\pi} - \pi) (y _ {\ell \ell^ {\prime}} - y _ {\ell} y _ {\ell^ {\prime}}) y _ {\ell} x _ {\ell^ {\prime}}} _ {\operatorname {c o v} (\mathrm {I V}, \mathrm {V})}\right). \\ \end{array}
$$
After expanding  $z_{\ell}$  as  $z_{\ell} = \bar{\nu} x_{\ell} + \nu y_{\ell} + r_{\ell}$ , we split the calculation into multiple parts to simplify it. First, we focus on terms that are multiplied by  $(x_{\ell \ell^{\prime}} - x_{\ell}x_{\ell^{\prime}})$  and do not contain  $r_{\ell}$  or  $r_{\ell^{\prime}}$ . Using Lemma D.1 extensively and the fact that  $\bar{\pi} = 1 - \pi$ ,  $\bar{\nu} = 1 - \nu$ , we find that the sum of these terms equals
$$
\begin{array}{l} \bar {\nu} ^ {2} \langle x K [ x ] ^ {2} \rangle + \nu^ {2} \langle x K [ y ] ^ {2} \rangle + 2 \bar {\nu} \nu \langle x K [ x ] K [ y ] \rangle - \bar {\nu} ^ {2} \langle x K [ x ] \rangle^ {2} - \nu^ {2} \langle x K [ y ] \rangle^ {2} - 2 \bar {\nu} \nu \langle x K [ x ] \rangle \langle x K [ y ] \rangle \\ + 4 \bar {\pi} ^ {2} \langle x K [ x ] ^ {2} \rangle - 4 \bar {\pi} ^ {2} \langle x K [ x ] \rangle^ {2} + (\bar {\pi} - \pi) ^ {2} \langle x K [ y ] ^ {2} \rangle - (\bar {\pi} - \pi) ^ {2} \langle x K [ y ] \rangle^ {2} \\ - 4 \bar {\pi} \bar {\nu} \langle x K [ x ] ^ {2} \rangle - 4 \bar {\pi} \nu \langle x K [ x ] K [ y ] \rangle + 4 \bar {\pi} \bar {\nu} \langle x K [ x ] \rangle^ {2} + 4 \bar {\pi} \nu \langle x K [ x ] \rangle \langle x K [ y ] \rangle \\ + 2 (\bar {\pi} - \pi) \bar {\nu} \langle x K [ x ] K [ y ] \rangle + 2 (\bar {\pi} - \pi) \nu \langle x K [ y ] ^ {2} \rangle - 2 (\bar {\pi} - \pi) \bar {\nu} \langle x K [ x ] \rangle \langle x K [ y ] \rangle - 2 (\bar {\pi} - \pi) \nu \langle x K [ y ] \rangle^ {2} \\ - 4 \bar {\pi} (\bar {\pi} - \pi) \langle x K [ x ] K [ y ] \rangle + 4 \bar {\pi} (\bar {\pi} - \pi) \langle x K [ x ] \rangle \langle x K [ y ] \rangle \\ = (\bar {\nu} - 2 \bar {\pi}) ^ {2} \left(\langle x K [ x - y ] ^ {2} \rangle - \langle x K [ x - y ] \rangle^ {2}\right) \\ \leq C \| \lambda \| _ {\infty} \operatorname {M M D} ^ {2} \left(P _ {X}, P _ {Y}\right). \\ \end{array}
$$
Similarly, the terms involving  $(y_{\ell \ell^{\prime}} - y_{\ell}y_{\ell^{\prime}})$  but not  $r_\ell$  or  $r_{\ell '}$  sum up to the quantity
$$
(\nu - 2 \pi) ^ {2} \left(\langle y K [ x - y ] ^ {2} \rangle - \langle y K [ x - y ] \rangle^ {2}\right) \leq C \| \lambda \| _ {\infty} \operatorname {M M D} ^ {2} \left(P _ {X}, P _ {Y}\right).
$$
Next, collecting the terms involving both  $(x_{\ell \ell^{\prime}} - x_{\ell}x_{\ell^{\prime}})$  and  $r_\ell$  or  $r_{\ell '}$  we get
$$
\begin{array}{l} 2 \bar {\nu} \langle x K [ r ] K [ x ] \rangle + 2 \nu \langle x K [ r ] K [ y ] \rangle + \langle x K [ r ] ^ {2} \rangle - 2 \bar {\nu} \langle x K [ x ] \rangle \langle x K [ r ] \rangle - 2 \nu \langle x K [ y ] \rangle \langle x K [ r ] \rangle - \langle x K [ r ] \rangle^ {2} \\ - 4 \bar {\pi} \langle x K [ x ] K [ r ] \rangle + 4 \bar {\pi} \langle x K [ x ] \rangle \langle x K [ r ] \rangle \\ + 2 (\bar {\pi} - \pi) \langle x K [ y ] K [ r ] \rangle - 2 (\bar {\pi} - \pi) \langle x K [ y ] \rangle \langle x K [ r ] \rangle \\ = 2 (\bar {\nu} - 2 \bar {\pi}) \left(\langle x K [ r ] K [ x - y ] \rangle - \langle x K [ r ] \rangle \langle x K [ x - y ] \rangle\right) + \langle x K [ r ] ^ {2} \rangle - \langle x K [ r ] \rangle^ {2} \\ \lesssim C \| \lambda \| _ {\infty} (R + R ^ {2}) \mathrm {M M D} ^ {2} \left(P _ {X}, P _ {Y}\right). \\ \end{array}
$$
Finally, collecting the terms involving both  $(y_{\ell \ell^{\prime}} - y_{\ell}y_{\ell^{\prime}})$  and  $r_\ell$  or  $r_{\ell '}$  we get
$$
\begin{array}{l} 2 (\nu - 2 \pi) \left(\langle y K [ r ] K [ y - x ] \rangle - \langle y K [ r ] \rangle \langle y K [ y - x ] \rangle\right) + \langle y K [ r ] ^ {2} \rangle - \langle y K [ r ] \rangle^ {2} \\ \lesssim C \| \lambda \| _ {\infty} (R + R ^ {2}) \operatorname {M M D} ^ {2} \left(P _ {X}, P _ {Y}\right). \\ \end{array}
$$
Similarly we get
$$
\begin{array}{l} \operatorname {C o e f} \left(\frac {1}{m}\right) = \sum_ {\ell \ell^ {\prime}} \lambda_ {\ell} \lambda_ {\ell^ {\prime}} \left(\underbrace {(z _ {\ell \ell^ {\prime}} - z _ {\ell} z _ {\ell^ {\prime}}) x _ {\ell} x _ {\ell^ {\prime}}} _ {\operatorname {v a r} (\mathrm {I})} + \underbrace {(z _ {\ell \ell^ {\prime}} - z _ {\ell} z _ {\ell^ {\prime}}) y _ {\ell} y _ {\ell^ {\prime}}} _ {\operatorname {v a r} (\mathrm {I})} + \underbrace {2 (z _ {\ell \ell^ {\prime}} - z _ {\ell} z _ {\ell^ {\prime}}) x _ {\ell} y _ {\ell^ {\prime}}} _ {\operatorname {c o v} (\mathrm {I}, \mathrm {I I})}\right) \\ = \langle z K [ x - y ] ^ {2} \rangle - \langle z K [ x - y ] \rangle^ {2} \\ \lesssim C \| \lambda \| _ {\infty} \operatorname {M M D} ^ {2} \left(P _ {X}, P _ {Y}\right). \\ \end{array}
$$
The remaining coefficients don't rely on subtle cancellations, and simple bounds yield
$$
\begin{array}{l} \operatorname {C o e f} \left(\frac {1}{n (n - 1)}\right) = \sum_ {\ell \ell^ {\prime}} \lambda_ {\ell} \lambda_ {\ell^ {\prime}} \left(\underbrace {4 \bar {\pi} ^ {2} \left(\frac {1}{2} \left(x _ {\ell \ell^ {\prime}} ^ {2} - x _ {\ell} ^ {2} x _ {\ell^ {\prime}} ^ {2}\right) - \left(x _ {\ell \ell^ {\prime}} - x _ {\ell} x _ {\ell^ {\prime}}\right) x _ {\ell} x _ {\ell^ {\prime}}\right)} _ {\operatorname {v a r} (\mathrm {I I I})}\right) \\ + \underbrace {4 \pi^ {2} \left(\frac {1}{2} \left(y _ {\ell \ell^ {\prime}} ^ {2} - y _ {\ell} ^ {2} y _ {\ell^ {\prime}} ^ {2}\right) - \left(y _ {\ell \ell^ {\prime}} - y _ {\ell} y _ {\ell^ {\prime}}\right) y _ {\ell} y _ {\ell^ {\prime}}\right)} _ {\text {v a r (I V)}} \\ \lesssim C ^ {2} \| \lambda \| _ {2} ^ {2} \\ \end{array}
$$
$$
\begin{array}{l} \operatorname {C o e f} \left(\frac {1}{n m}\right) = \sum_ {\ell \ell^ {\prime}} \lambda_ {\ell} \lambda_ {\ell^ {\prime}} \left(\underbrace {- \left(z _ {\ell \ell^ {\prime}} - z _ {\ell} z _ {\ell^ {\prime}}\right) x _ {\ell} x _ {\ell^ {\prime}} - \left(x _ {\ell \ell^ {\prime}} - x _ {\ell} x _ {\ell^ {\prime}}\right) z _ {\ell} z _ {\ell^ {\prime}} + \left(x _ {\ell \ell^ {\prime}} z _ {\ell \ell^ {\prime}} - x _ {\ell} x _ {\ell^ {\prime}} z _ {\ell} z _ {\ell^ {\prime}}\right)} _ {\text {v a r (l)}}\right) \\ \left. - \underbrace {(z _ {\ell \ell^ {\prime}} - z _ {\ell} z _ {\ell^ {\prime}}) y _ {\ell} y _ {\ell^ {\prime}} - (y _ {\ell \ell^ {\prime}} - y _ {\ell} y _ {\ell^ {\prime}}) z _ {\ell} z _ {\ell^ {\prime}} + (y _ {\ell \ell^ {\prime}} z _ {\ell \ell^ {\prime}} - y _ {\ell} y _ {\ell^ {\prime}} z _ {\ell} z _ {\ell^ {\prime}})} _ {\operatorname {v a r} (1)}\right) \\ \lesssim C ^ {2} \| \lambda \| _ {2} ^ {2} \\ \end{array}
$$
$$
\begin{array}{l} \operatorname {C o e f} \left(\frac {1}{n ^ {2}}\right) = \sum_ {\ell \ell^ {\prime}} \lambda_ {\ell} \lambda_ {\ell^ {\prime}} \left(\underbrace {(\bar {\pi} - \pi) (- (y _ {\ell \ell^ {\prime}} - y _ {\ell} y _ {\ell^ {\prime}}) x _ {\ell} x _ {\ell^ {\prime}} - (x _ {\ell \ell^ {\prime}} - x _ {\ell} x _ {\ell^ {\prime}}) y _ {\ell} y _ {\ell^ {\prime}} + (x _ {\ell \ell^ {\prime}} y _ {\ell \ell^ {\prime}} - x _ {\ell} x _ {\ell^ {\prime}} y _ {\ell} y _ {\ell^ {\prime}}))} _ {\operatorname {v a r} (\mathsf {V})}\right) \\ \lesssim C ^ {2} \| \lambda \| _ {2} ^ {2}. \\ \end{array}
$$
Summarizing, we've found that
$$
\begin{array}{l} \operatorname {v a r} (T (X, Y, Z) - \gamma (X, Y, \pi)) \lesssim \left(\frac {1}{n} + \frac {1}{m}\right) C \| \lambda \| _ {\infty} (1 + R ^ {2}) \mathrm {M M D} ^ {2} \left(P _ {X}, P _ {Y}\right) \\ + \left(\frac {1}{n ^ {2}} + \frac {1}{n m}\right) C ^ {2} \| \lambda \| _ {2} ^ {2}. \tag {22} \\ \end{array}
$$
Using that  $\langle \theta_r,\theta_{P_Y} - \theta_{P_X}\rangle_{\mathcal{H}_K} = 0$  , we compute the expectation to be
$$
\mathbb {E} \left[ - T (X, Y, Z) + \gamma (X, Y, \pi) \right] = (\pi - \nu) \mathrm {M M D} ^ {2} \left(P _ {X}, P _ {Y}\right).
$$
Taking  $\pi \triangleq \delta / 2$  and applying Chebyshev's inequality shows that there exists a universal constant  $c > 0$ , such that the testing problem is possible at constant error probability (say  $\alpha = 5\%$ ), provided that the sample sizes  $m, n$  satisfy the following inequalities:
$$
\min  \{m, n \} \geq c \frac {C \| \lambda \| _ {\infty} (1 + R ^ {2})}{\delta^ {2} \epsilon^ {2}}
$$
$$
\min  \{n, \sqrt {n m} \} \geq c \frac {C \| \lambda \| _ {2}}{\delta \epsilon^ {2}}.
$$
By repeated sample splitting and majority voting (see Appendix C), we can boost the success probability of this test to the desired level  $1 - \alpha$  by incurring a multiplicative  $\Theta (\log (1 / \alpha))$  factor on the sample sizes  $n,m$ , which yields the desired result.
E.1 Information theoretic tools
Our lower bounds rely on the method of two fuzzy hypotheses [50]. Given a measurable space  $S$ , let  $\mathcal{M}(S)$  denote the set of all probability measures on  $S$ . We call subsets  $H \subseteq \mathcal{M}(S)$  hypotheses. The following is the main technical result that our proofs rely on.
Lemma E.1. Take hypotheses  $H_0, H_1 \subseteq \mathcal{M}(\mathcal{S})$  and  $P_0, P_1 \in \mathcal{M}(\mathcal{S})$  random with  $\mathbb{P}(P_i \in H_i) = 1$ . Then
$$
\inf_{\psi}\max_{i = 0,1}\sup_{P\in H_{i}}P(\psi \neq i)\geq \frac{1}{2}\left(1 - \operatorname {TV}(\mathbb{E}P_{0},\mathbb{E}P_{1})\right),
$$
where the infimum is over all tests  $\psi : \mathcal{X} \to \{0,1\}$ .
Proof. For any  $\psi$
$$
\begin{array}{l} \max  _ {i = 0, 1} \sup  _ {\mathbb {P} _ {i} \in H _ {i}} \mathbb {P} _ {i} (\psi \neq i) \geq \frac {1}{2} \sup  _ {\mathbb {P} _ {i} \in H _ {i}} \left(\mathbb {P} _ {0} (\psi = 1) + \mathbb {P} _ {1} (\psi = 0)\right) \\ \geq \frac {1}{2} \mathbb {E} \left[ P _ {0} (\psi = 1) + P _ {1} (\psi = 0) \right]. \\ \end{array}
$$
Optimizing over  $\psi$  we get that the RHS above is equal to  $\frac{1}{2} (1 - \mathrm{TV}(\mathbb{E}P_0,\mathbb{E}P_1))$  as required.
Therefore, to prove a lower bound on the minimax sample complexity of testing with total error probability  $\alpha$ , we just need to construct two random measures  $P_{i} \in H_{i}$  such that  $1 - \mathrm{TV}(\mathbb{E}P_0,\mathbb{E}P_1) = \Omega (\alpha)$ . In our proofs we also use the following standard results on  $f$ -divergences.
Lemma E.2 ([43, Section 7]). For any probability distributions  $P, Q$  the inequalities
$$
1 - \operatorname {T V} (P, Q) \geq \frac {1}{2} \exp (- \operatorname {K L} (P \| Q)) \geq \frac {1}{2} \frac {1}{1 + \chi^ {2} (P \| Q)}
$$
Lemma E.3 (Chain rule for  $\chi^2$ -divergence). Let  $P_{X,Y}, Q_{X,Y}$  be probability measures such that the marginals on  $X$  are equal ( $P_X = Q_X$ ). Then
$$
\chi^ {2} \left(P _ {X, Y} \| Q _ {X, Y}\right) = \chi^ {2} \left(P _ {Y | X} \| Q _ {Y | X} \mid P _ {X}\right).
$$
Proof. Let  $P_{X,Y}, Q_{X,Y}$  have densities  $p, q$  with respect to some  $\mu$ . Then, by some abuse of notation, we have
$$
\begin{array}{l} \chi^ {2} \left(P _ {X, Y} \| Q _ {X, Y}\right) = - 1 + \int \frac {p (x , y) ^ {2}}{q (x , y)} \mathrm {d} \mu (x, y) \\ = - 1 + \int \frac {p (y | x) ^ {2} p (x)}{q (y | x)} \mathrm {d} \mu (x, y) \\ = \int p (x) \int \left(\frac {p (y | x) ^ {2}}{q (y | x)} - 1\right) \mathrm {d} \mu (y, x) \\ = \chi^ {2} \left(P _ {Y | X} \| Q _ {Y | X} | P _ {X}\right). \\ \end{array}
$$
E.2 Constructing hard instances
Recall that in the statement of Theorem 3.2, we assume that  $\mu (\mathcal{X}) = 1$ ,  $\sup_{x\in \mathcal{X}}K(x,x)\leq 1$  and  $\int K(x,y)\mu (\mathrm{d}x)\equiv \lambda_1$ . Let  $f_{0}\equiv 1$  and for each  $\eta \in \{\pm 1\}^{\mathbb{N}}$  define
$$
f _ {\eta} = 1 + \epsilon \underbrace {\sum_ {j \geq 2} \rho_ {j} \eta_ {j} e _ {j}} _ {\triangleq g _ {\eta}} \tag {23}
$$
where  $\{\rho_j\}_{j\geq 2}$  is chosen as  $\rho_{j} = \mathbb{1}\{2\leq j\leq J\} \sqrt{\lambda_{j}} /\| \lambda \|_{2,J}$ , where we define  $\| \lambda \|_{2,J} = \sqrt{\sum_{2\leq j\leq J}\lambda_j^2}$  for some  $J\geq 2$ . Notice that  $\int f_{\eta}(x)\mu (\mathrm{d}x) = \mu (\mathcal{X}) = 1$  due to orthogonality of the eigenfunctions. Assume from here on that  $J$  is chosen so that for all  $\eta$  we have  $f_{\eta}(x)\geq 1 / 2$  for all  $x\in \mathcal{X}$ . This makes  $f_{\eta}$  into a valid probability density with respect to the base measure  $\mu$ . Before continuing, we prove the following Lemma, which gives a lower bound on the maximal  $J$  for which  $f_{\eta}\geq 1 / 2$  for all  $\eta$ .
Lemma E.4.  $J \leq J_{\epsilon}^{\star}$  holds provided  $2\epsilon \sqrt{J - 1} \leq \| \lambda \|_{2J}$ .
Proof of Lemma E.4. Notice that
$$
\left\| e _ {j} \right\| _ {\infty} = \sup  _ {x \in \mathcal {X}} \langle K (x, \cdot), e _ {j} \rangle_ {\mathcal {H}} \leq \sup  _ {x \in \mathcal {X}} \| K (x, \cdot) \| _ {\mathcal {H}} \| e _ {j} \| _ {\mathcal {H}} \leq \frac {1}{\sqrt {\lambda} _ {j}}, \tag {24}
$$
where we use  $\| K(x,\cdot)\|_{\mathcal{H}} = \sqrt{K(x,x)}$  . We have