SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Same logic applies for  $\mathbb{P}(\hat{\phi}_0 - \phi_0) \leq \frac{1}{1 - \epsilon} \| \pi - \hat{\pi} \| \| \beta_0 - \hat{\beta}_0 \|$ . Therefore  $\| T_2 \|_{\mathcal{H}} = o_{\mathbb{P}}(1 / \sqrt{n})$  and the result follows.
C.4 Proof of Theorem 4.1
Assume that we have access to a sample  $(X_{i},A_{i},Y_{i})_{i = 1}^{2n}\sim (X,A,Y)$ . We denote  $\mathcal{D}_1 = (X_i,A_i,Y_i)_{i = 1}^n$ ,  $\mathcal{D}_2 = (X_j,A_j,Y_j)_{j = n + 1}^{2n}$ . Further,
$$
f _ {h} (z) = \frac {1}{n} \sum_ {j = n + 1} ^ {2 n} \langle \phi (z), \phi (Z _ {j}) \rangle , \quad T _ {h} = \frac {\sqrt {n} \bar {f} _ {h}}{S _ {h}},
$$
where  $\bar{f}_h$  and  $S_h^2$  are the empirical mean and variance respectively:
$$
\bar {f} _ {h} = \frac {1}{n} \sum_ {i = 1} ^ {n} f _ {h} (Z _ {i}), \quad S _ {h} ^ {2} = \frac {1}{n} \sum_ {i = 1} ^ {n} \left(f _ {h} (Z _ {i}) - \bar {f} _ {h}\right) ^ {2}.
$$
In particular, note the indices 1 to  $n$  in the definition of  $\bar{f}_h$ , and the indices  $n + 1$  to  $2n$  when defining  $f_h$ . Similarly, we denote the analogous version for estimated embeddings  $\hat{\phi}$  as follows:
$$
f _ {h} ^ {\dagger} (z) = \frac {1}{n} \sum_ {j = n + 1} ^ {2 n} \langle \hat {\phi} ^ {(1)} (z), \hat {\phi} ^ {(2)} (Z _ {j}) \rangle , T _ {h} ^ {\dagger} = \frac {\sqrt {n} \bar {f} _ {h} ^ {\dagger}}{S _ {h} ^ {\dagger}}, \bar {f} _ {h} ^ {\dagger} = \frac {1}{n} \sum_ {i = 1} ^ {n} f _ {h} ^ {\dagger} (Z _ {i}), S _ {h} ^ {\dagger 2} = \frac {1}{n} \sum_ {i = 1} ^ {n} (f _ {h} ^ {\dagger} (Z _ {i}) - \bar {f} _ {h} ^ {\dagger}) ^ {2}.
$$
For of ease of notation, we also define
$$
\tau_ {1} = \frac {1}{\sqrt {n}} \sum_ {i = 1} ^ {n} \phi (Z _ {i}), \quad \tau_ {2} = \frac {1}{\sqrt {n}} \sum_ {j = n + 1} ^ {2 n} \phi (Z _ {j}), \quad \hat {\tau} _ {1} = \frac {1}{\sqrt {n}} \sum_ {i = 1} ^ {n} \hat {\phi} ^ {(1)} (Z _ {i}), \quad \hat {\tau} _ {2} = \frac {1}{\sqrt {n}} \sum_ {j = n + 1} ^ {2 n} \hat {\phi} ^ {(2)} (Z _ {j}).
$$
Lastly, note that the following equalities hold by definition:
$$
n \bar {f} _ {h} = n \langle \frac {1}{n} \sum_ {i = 1} ^ {n} \phi (Z _ {i}), \frac {1}{n} \sum_ {j = n + 1} ^ {2 n} \phi (Z _ {j}) \rangle = \langle \frac {1}{\sqrt {n}} \sum_ {i = 1} ^ {n} \phi (Z _ {i}), \frac {1}{\sqrt {n}} \sum_ {j = n + 1} ^ {2 n} \phi (Z _ {j}) \rangle = \langle \tau_ {1}, \tau_ {2} \rangle ,
$$
$$
n \bar {f} _ {h} ^ {\dagger} = n \langle \frac {1}{n} \sum_ {i = 1} ^ {n} \hat {\phi} ^ {(1)} (Z _ {i}), \frac {1}{n} \sum_ {j = n + 1} ^ {2 n} \hat {\phi} ^ {(2)} (Z _ {j}) \rangle = \langle \frac {1}{\sqrt {n}} \sum_ {i = 1} ^ {n} \hat {\phi} ^ {(1)} (Z _ {i}), \frac {1}{\sqrt {n}} \sum_ {j = n + 1} ^ {2 n} \hat {\phi} ^ {(2)} (Z _ {j}) \rangle = \langle \hat {\tau} _ {1}, \hat {\tau} _ {2} \rangle ,
$$
$$
(\sqrt {n} S _ {h}) ^ {2} = n \left\{\left(\frac {1}{n} \sum_ {i = 1} ^ {n} \langle \phi (Z _ {i}), \frac {1}{n} \sum_ {j = n + 1} ^ {2 n} \phi (Z _ {j}) \rangle^ {2}\right) - (\bar {f} _ {h}) ^ {2} \right\} = \frac {1}{n} \sum_ {i = 1} ^ {n} \langle \phi (Z _ {i}), \tau_ {2} \rangle^ {2} - (\sqrt {n} \bar {f} _ {h}) ^ {2},
$$
$$
(\sqrt {n} S _ {h} ^ {\dagger}) ^ {2} = n \left\{\left(\frac {1}{n} \sum_ {i = 1} ^ {n} \langle \hat {\phi} ^ {(1)} (Z _ {i}), \frac {1}{n} \sum_ {j = n + 1} ^ {2 n} \hat {\phi} ^ {(2)} (Z _ {j}) \rangle^ {2}\right) - (\bar {f} _ {h} ^ {\dagger}) ^ {2} \right\} = \frac {1}{n} \sum_ {i = 1} ^ {n} \langle \hat {\phi} ^ {(1)} (Z _ {i}), \hat {\tau} _ {2} \rangle^ {2} - (\sqrt {n} \bar {f} _ {h} ^ {\dagger}) ^ {2}.
$$
Proof of Theorem 4.1. We will prove the theorem in four steps:
1. We first prove that  $n\bar{f}_h^\dagger = n\bar{f}_h + o_{\mathbb{P}}(1)$
2. We then show that  $nS_h^\dagger^2 = nS_h^2 +o_{\mathbb{P}}(1)$
3. Further, we prove that  $\frac{1}{\mathbb{E}[n\{f_h(Z)\}^2|\mathcal{D}_2]} = O_{\mathbb{P}}(1)$ .
4. We conclude that  $T_h^\dagger \xrightarrow{d} N(0,1)$ .
It now remains to prove each of the four steps.
Details of step 1. Theorem C.10 may be applied for both  $\hat{\phi}^{(0)},\hat{\phi}^{(1)}$  because its conditions are fulfilled. We obtain that
$$
\hat {\tau} _ {1} = \tau_ {1} + \sqrt {n} \xi_ {n} ^ {(1)}, \quad \hat {\tau} _ {2} = \tau_ {2} + \sqrt {n} \xi_ {n} ^ {(2)},
$$
where  $\| \sqrt{n}\xi_n^{(1)}\|_{\mathcal{H}},\| \sqrt{n}\xi_n^{(2)}\|_{\mathcal{H}} = o_{\mathbb{P}}(1)$ . Rewriting the expression, we note that
$$
\hat {\tau} _ {1} = \tau_ {1} + \chi_ {n} ^ {(1)}, \quad \hat {\tau} _ {2} = \tau_ {2} + \chi_ {n} ^ {(2)},
$$
where  $\| \chi_n^{(1)}\|_{\mathcal{H}},\| \chi_n^{(2)}\|_{\mathcal{H}} = o_{\mathbb{P}}(1)$ . Based on Theorem C.3,
$$
\tau_ {1} \stackrel {{d}} {{\to}} N (0, C), \quad \tau_ {2} \stackrel {{d}} {{\to}} N (0, C),
$$
where  $C = \mathbb{E}[\phi (Z)\otimes \phi (Z)]$ , which implies  $\max (\| \tau_1\|_{\mathcal{H}}^2,\| \tau_2\|_{\mathcal{H}}^2) = O_{\mathbb{P}}(1)$ . Consequently,
$$
\begin{array}{l} n \bar {f} _ {h} ^ {\dagger} = \langle \hat {\tau} _ {1}, \hat {\tau} _ {2} \rangle \\ = \left\langle \tau_ {1} + \chi_ {n} ^ {(1)}, \tau_ {2} + \chi_ {n} ^ {(2)} \right\rangle \\ = \left\langle \tau_ {1}, \tau_ {2} \right\rangle + \left\langle \tau_ {1}, \chi_ {n} ^ {(2)} \right\rangle + \left\langle \chi_ {n} ^ {(1)}, \tau_ {2} \right\rangle + \left\langle \chi_ {n} ^ {(1)}, \chi_ {n} ^ {(2)} \right\rangle \\ = n \bar {f} _ {h} + \left\langle \tau_ {1}, \chi_ {n} ^ {(2)} \right\rangle + \left\langle \chi_ {n} ^ {(1)}, \tau_ {2} \right\rangle + \left\langle \chi_ {n} ^ {(1)}, \chi_ {n} ^ {(2)} \right\rangle . \\ \end{array}
$$
Given that  $|\langle \tau_1,\chi_n^{(2)}\rangle +\langle \chi_n^{(1)},\tau_2\rangle +\langle \chi_n^{(1)},\chi_n^{(2)}\rangle |$  is upper bounded by  $\| \tau_{1}\|_{\mathcal{H}}\| \chi_{n}^{(2)}\|_{\mathcal{H}}+$ $\| \chi_n^{(1)}\|_{\mathcal{H}}\| \tau_2\|_{\mathcal{H}} + \| \chi_n^{(1)}\|_{\mathcal{H}}\| \chi_n^{(2)}\|_{\mathcal{H}}$  (by the triangle inequality and Cauchy-Schwarz inequality), which is  $O_{\mathbb{P}}(1)o_{\mathbb{P}}(1) + o_{\mathbb{P}}(1)O_{\mathbb{P}}(1) + o_{\mathbb{P}}(1)o_{\mathbb{P}}(1) = o_{\mathbb{P}}(1)$ , we deduce that
$$
n \bar {f} _ {h} ^ {\dagger} = n \bar {f} _ {h} + o _ {\mathbb {P}} (1). \tag {8}
$$
Details of step 2. We claim that
$$
\left| \frac {1}{n} \sum_ {i = 1} ^ {n} \langle \hat {\phi} ^ {(1)} \left(Z _ {i}\right), \hat {\tau} _ {2} \rangle^ {2} - \frac {1}{n} \sum_ {i = 1} ^ {n} \langle \phi \left(Z _ {i}\right), \tau_ {2} \rangle^ {2} \right| = o _ {\mathbb {P}} (1), \tag {9}
$$
as well as
$$
\left(\sqrt {n} \bar {f} _ {h} ^ {\dagger}\right) ^ {2} = \left(\sqrt {n} \bar {f} _ {h}\right) ^ {2} + o _ {\mathbb {P}} (1). \tag {10}
$$
If both (9) and (10) hold, we obtain
$$
\begin{array}{l} (\sqrt {n} S _ {h} ^ {\dagger}) ^ {2} - (\sqrt {n} S _ {h}) ^ {2} = \left(\frac {1}{n} \sum_ {i = 1} ^ {n} \langle \hat {\phi} ^ {(1)} (Z _ {i}), \hat {\tau} _ {2} \rangle^ {2} - (\sqrt {n} \bar {f} _ {h} ^ {\dagger}) ^ {2}\right) - \left(\frac {1}{n} \sum_ {i = 1} ^ {n} \langle \phi (Z _ {i}), \tau_ {2} \rangle^ {2} - (\sqrt {n} \bar {f} _ {h}) ^ {2}\right) \\ = \left(\frac {1}{n} \sum_ {i = 1} ^ {n} \langle \hat {\phi} ^ {(1)} (Z _ {i}), \hat {\tau} _ {2} \rangle^ {2} - \frac {1}{n} \sum_ {i = 1} ^ {n} \langle \phi (Z _ {i}), \tau_ {2} \rangle^ {2}\right) - \left((\sqrt {n} \bar {f} _ {h} ^ {\dagger}) ^ {2} - (\sqrt {n} \bar {f} _ {h}) ^ {2}\right) \\ = o _ {\mathbb {P}} (1) + o _ {\mathbb {P}} (1) \\ = o _ {\mathbb {P}} (1), \\ \end{array}
$$
$$
n S _ {h} ^ {\dagger 2} = n S _ {h} ^ {2} + o _ {\mathbb {P}} (1), \tag {11}
$$
thus concluding step 2.
In order to prove (9), we denote  $\epsilon_{i} = \hat{\phi}^{(1)}(Z_{i}) - \phi (Z_{i})$  and consider
$$
\begin{array}{l} \left| \frac {1}{n} \sum_ {i = 1} ^ {n} \langle \hat {\phi} ^ {(1)} (Z _ {i}), \tau_ {2} \rangle^ {2} - \frac {1}{n} \sum_ {i = 1} ^ {n} \langle \phi (Z _ {i}), \tau_ {2} \rangle^ {2} \right| = \left| \frac {1}{n} \sum_ {i = 1} ^ {n} \langle \phi (Z _ {i}) + \epsilon_ {i}, \tau_ {2} \rangle^ {2} - \frac {1}{n} \sum_ {i = 1} ^ {n} \langle \phi (Z _ {i}), \tau_ {2} \rangle^ {2} \right| \\ = \left| \frac {1}{n} \sum_ {i = 1} ^ {n} \langle \epsilon_ {i}, \tau_ {2} \rangle^ {2} + \frac {2}{n} \sum_ {i = 1} ^ {n} \langle \phi (Z _ {i}), \tau_ {2} \rangle \langle \epsilon_ {i}, \tau_ {2} \rangle \right| \\ \stackrel {(i)} {\leq} \left| \frac {1}{n} \sum_ {i = 1} ^ {n} \langle \epsilon_ {i}, \tau_ {2} \rangle^ {2} \right| + \left| \frac {2}{n} \sum_ {i = 1} ^ {n} \langle \phi (Z _ {i}), \tau_ {2} \rangle \langle \epsilon_ {i}, \tau_ {2} \rangle \right| \\ \stackrel {(i i)} {\leq} \frac {1}{n} \sum_ {i = 1} ^ {n} \langle \epsilon_ {i}, \tau_ {2} \rangle^ {2} + 2 \left(\frac {1}{n} \sum_ {i = 1} ^ {n} \langle \phi (Z _ {i}), \tau_ {2} \rangle^ {2}\right) ^ {\frac {1}{2}} \left(\frac {1}{n} \sum_ {i = 1} ^ {n} \langle \epsilon_ {i}, \tau_ {2} \rangle^ {2}\right) ^ {\frac {1}{2}} \\ \stackrel {(i i i)} {\leq} \frac {1}{n} \sum_ {i = 1} ^ {n} \langle \epsilon_ {i}, \tau_ {2} \rangle^ {2} + 2 \left(\frac {1}{n} \sum_ {i = 1} ^ {n} \| \phi (Z _ {i}) \| _ {\mathcal {H}} ^ {2} \| \tau_ {2} \| _ {\mathcal {H}} ^ {2}\right) ^ {\frac {1}{2}} \left(\frac {1}{n} \sum_ {i = 1} ^ {n} \langle \epsilon_ {i}, \tau_ {2} \rangle^ {2}\right) ^ {\frac {1}{2}} \\ \stackrel {(i v)} {\leq} \frac {1}{n} \sum_ {i = 1} ^ {n} \| \epsilon_ {i} \| _ {\mathcal {H}} ^ {2} \| \tau_ {2} \| _ {\mathcal {H}} ^ {2} + \\ + 2 \left(\frac {1}{n} \sum_ {i = 1} ^ {n} \| \phi (Z _ {i}) \| _ {\mathcal {H}} ^ {2} \| \tau_ {2} \| _ {\mathcal {H}} ^ {2}\right) ^ {\frac {1}{2}} \left(\frac {1}{n} \sum_ {i = 1} ^ {n} \| \epsilon_ {i} \| _ {\mathcal {H}} ^ {2} \| \tau_ {2} \| _ {\mathcal {H}} ^ {2}\right) ^ {\frac {1}{2}}, \\ \end{array}
$$
where (i) is obtained by the triangle inequality, and (ii), (iii), (iv) are obtained by Cauchy-Schwarz inequality. We recall that  $\| \tau_2\|_{\mathcal{H}}^2 = O_{\mathbb{P}}(1)$ . Moreover,  $\mathbb{E}\| \phi (Z)\|_{\mathcal{H}}^2\leq \mathbb{E}\| \phi (Z)\|_{\mathcal{H}}^4 < \infty$  implies  $\frac{1}{n}\sum_{i = 1}^{n}\| \phi (Z_i)\|_{\mathcal{H}}^2\stackrel {p}{\to}\mathbb{E}\| \phi (Z)\|_{\mathcal{H}}^2$  by the law of large numbers, so  $\frac{1}{n}\sum_{i = 1}^{n}\| \phi (Z_i)\|_{\mathcal{H}}^2 = O_{\mathbb{P}}(1)$ .
If  $\hat{\phi}$  was constructed independently from  $\mathcal{D}_1$ , then
$$
\mathbb {P} \left[ \frac {1}{n} \sum_ {i = 1} ^ {n} \| \epsilon_ {i} \| _ {\mathcal {H}} ^ {2} \right] = \mathbb {P} \left[ \| \epsilon_ {1} \| _ {\mathcal {H}} ^ {2} \right] = \mathbb {P} \left[ \| \hat {\phi} ^ {(1)} - \phi \| _ {\mathcal {H}} ^ {2} \right] = \| \hat {\phi} ^ {(1)} - \phi \| ^ {2} = o _ {\mathbb {P}} (1),
$$
so by Markov's inequality
$$
\frac {1}{n} \sum_ {i = 1} ^ {n} \| \epsilon_ {i} \| _ {\mathcal {H}} ^ {2} = o _ {\mathbb {P}} (1).
$$
If  $\| \hat{\phi}^{(1)}\|_{\mathcal{H}}^2$  belongs to a Glivenko-Cantelli class, then  $\| \hat{\phi}^{(1)} - \phi \|_{\mathcal{H}}^2$  belongs to a Glivenko-Cantelli class. From this, we conclude that  $(\mathbb{P} - \mathbb{P}_n)\| \widehat{\phi}^{(1)} - \phi \|_{\mathcal{H}}^2 = o_{\mathbb{P}}(1)$ , hence
$$
\begin{array}{l} \frac {1}{n} \sum_ {i = 1} ^ {n} \| \epsilon_ {i} \| _ {\mathcal {H}} ^ {2} = \mathbb {P} _ {n} \| \hat {\phi} ^ {(1)} - \phi \| _ {\mathcal {H}} ^ {2} \\ = \left(\mathbb {P} - \mathbb {P} _ {n}\right) \| \hat {\phi} ^ {(1)} - \phi \| _ {\mathcal {H}} ^ {2} + \mathbb {P} \| \hat {\phi} ^ {(1)} - \phi \| _ {\mathcal {H}} ^ {2} \\ = (\mathbb {P} - \mathbb {P} _ {n}) \| \hat {\phi} ^ {(1)} - \phi \| _ {\mathcal {H}} ^ {2} + \| \hat {\phi} ^ {(1)} - \phi \| ^ {2} \\ = o _ {\mathbb {P}} (1). \\ \end{array}
$$
In either case,
$$
\begin{array}{l} \left| \frac {1}{n} \sum_ {i = 1} ^ {n} \langle \hat {\phi} ^ {(1)} (Z _ {i}), \tau_ {2} \rangle^ {2} - \frac {1}{n} \sum_ {i = 1} ^ {n} \langle \phi (Z _ {i}), \tau_ {2} \rangle^ {2} \right| \leq \frac {1}{n} \sum_ {i = 1} ^ {n} \| \epsilon_ {i} \| _ {\mathcal {H}} ^ {2} \| \tau_ {2} \| _ {\mathcal {H}} ^ {2} + \\ + 2 \left(\frac {1}{n} \sum_ {i = 1} ^ {n} \| \phi (Z _ {i}) \| _ {\mathcal {H}} ^ {2} \| \tau_ {2} \| _ {\mathcal {H}} ^ {2}\right) ^ {\frac {1}{2}} \left(\frac {1}{n} \sum_ {i = 1} ^ {n} \| \epsilon_ {i} \| _ {\mathcal {H}} ^ {2} \| \tau_ {2} \| _ {\mathcal {H}} ^ {2}\right) ^ {\frac {1}{2}} \\ \leq \| \tau_ {2} \| _ {\mathcal {H}} ^ {2} \left(\frac {1}{n} \sum_ {i = 1} ^ {n} \| \epsilon_ {i} \| _ {\mathcal {H}} ^ {2}\right) + \\ + 2 \| \tau_ {2} \| _ {\mathcal {H}} ^ {2} \left(\frac {1}{n} \sum_ {i = 1} ^ {n} \| \phi (Z _ {i}) \| _ {\mathcal {H}} ^ {2}\right) ^ {\frac {1}{2}} \left(\frac {1}{n} \sum_ {i = 1} ^ {n} \| \epsilon_ {i} \| _ {\mathcal {H}} ^ {2}\right) ^ {\frac {1}{2}} \\ = O _ {\mathbb {P}} (1) o _ {\mathbb {P}} (1) + O _ {\mathbb {P}} (1) O _ {\mathbb {P}} (1) o _ {\mathbb {P}} (1) \\ = o _ {\mathbb {P}} (1). \\ \end{array}
$$
Further,
$$
\begin{array}{l} \left| \frac {1}{n} \sum_ {i = 1} ^ {n} \langle \hat {\phi} ^ {(1)} (Z _ {i}), \hat {\tau} _ {2} \rangle^ {2} - \frac {1}{n} \sum_ {i = 1} ^ {n} \langle \hat {\phi} ^ {(1)} (Z _ {i}), \tau_ {2} \rangle^ {2} \right| = \left| \frac {1}{n} \sum_ {i = 1} ^ {n} \langle \hat {\phi} ^ {(1)} (Z _ {i}), \tau_ {2} + \chi_ {n} ^ {(2)} \rangle^ {2} - \frac {1}{n} \sum_ {i = 1} ^ {n} \langle \hat {\phi} ^ {(1)} (Z _ {i}), \tau_ {2} \rangle^ {2} \right| \\ = \left| \frac {1}{n} \sum_ {i = 1} ^ {n} \langle \hat {\phi} ^ {(1)} (Z _ {i}), \chi_ {n} ^ {(2)} \rangle^ {2} + \frac {2}{n} \sum_ {i = 1} ^ {n} \langle \hat {\phi} ^ {(1)} (Z _ {i}), \tau_ {2} \rangle \langle \hat {\phi} ^ {(1)} (Z _ {i}), \chi_ {n} ^ {(2)} \rangle \right| \\ \stackrel {(i)} {\leq} \left| \frac {1}{n} \sum_ {i = 1} ^ {n} \langle \hat {\phi} ^ {(1)} (Z _ {i}), \chi_ {n} ^ {(2)} \rangle^ {2} \right| + \left| \frac {2}{n} \sum_ {i = 1} ^ {n} \langle \hat {\phi} ^ {(1)} (Z _ {i}), \tau_ {2} \rangle \langle \hat {\phi} ^ {(1)} (Z _ {i}), \chi_ {n} ^ {(2)} \rangle \right| \\ \stackrel {(i i)} {\leq} \frac {1}{n} \sum_ {i = 1} ^ {n} \langle \hat {\phi} ^ {(1)} (Z _ {i}), \chi_ {n} ^ {(2)} \rangle^ {2} + \\ + 2 \left(\frac {1}{n} \sum_ {i = 1} ^ {n} \langle \hat {\phi} ^ {(1)} (Z _ {i}), \tau_ {2} \rangle^ {2}\right) ^ {\frac {1}{2}} \left(\frac {1}{n} \sum_ {i = 1} ^ {n} \langle \hat {\phi} ^ {(1)} (Z _ {i}), \chi_ {n} ^ {(2)} \rangle^ {2}\right) ^ {\frac {1}{2}} \\ \stackrel {(i i i)} {\leq} \left(\frac {1}{n} \sum_ {i = 1} ^ {n} \| \hat {\phi} ^ {(1)} (Z _ {i}) \| _ {\mathcal {H}} ^ {2}\right) \left(\| \chi_ {n} ^ {(2)} \| _ {\mathcal {H}} ^ {2} + 2 \| \tau_ {2} \| _ {\mathcal {H}} \| \chi_ {n} ^ {(2)} \| _ {\mathcal {H}}\right) \\ = O _ {\mathbb {P}} (1) \left\{o _ {\mathbb {P}} (1) + O _ {\mathbb {P}} (1) o _ {\mathbb {P}} (1) \right\} \\ = o _ {\mathbb {P}} (1), \\ \end{array}
$$
where we obtain (i) by the triangle inequality, (ii) by Cauchy-Schwarz inequality, and (iii) by Cauchy-Schwarz inequality and grouping terms. By the triangle inequality,  $\left|\frac{1}{n}\sum_{i=1}^{n}\langle\hat{\phi}^{(1)}(Z_i),\hat{\tau}_2\rangle^2-\right.$ $\left.\frac{1}{n}\sum_{i=1}^{n}\langle\phi(Z_i),\tau_2\rangle^2\right|$  is upper bounded by
$$
\left| \frac {1}{n} \sum_ {i = 1} ^ {n} \langle \hat {\phi} ^ {(1)} (Z _ {i}), \hat {\tau} _ {2} \rangle^ {2} - \frac {1}{n} \sum_ {i = 1} ^ {n} \langle \hat {\phi} ^ {(1)} (Z _ {i}), \tau_ {2} \rangle^ {2} \right| + \left| \frac {1}{n} \sum_ {i = 1} ^ {n} \langle \hat {\phi} ^ {(1)} (Z _ {i}), \tau_ {2} \rangle^ {2} - \frac {1}{n} \sum_ {i = 1} ^ {n} \langle \phi (Z _ {i}), \tau_ {2} \rangle^ {2} \right|,
$$
which is  $o_{\mathbb{P}}(1) + o_{\mathbb{P}}(1) = o_{\mathbb{P}}(1)$ , thus (9) holds.
In order to prove (10), we note that (8) implies  $(n\bar{f}_h^\dagger)^2 = (n\bar{f}_h + o_{\mathbb{P}}(1))^2$ . Expanding the second term we obtain
$$
\begin{array}{l} \left(n \bar {f} _ {h} + o _ {\mathbb {P}} (1)\right) ^ {2} = \left(n \bar {f} _ {h}\right) ^ {2} + \left(o _ {\mathbb {P}} (1)\right) ^ {2} + \left(n \bar {f} _ {h}\right) o _ {\mathbb {P}} (1) \\ = (n \bar {f} _ {h}) ^ {2} + (o _ {\mathbb {P}} (1)) ^ {2} + O _ {\mathbb {P}} (1) o _ {\mathbb {P}} (1) \\ = (n \bar {f} _ {h}) ^ {2} + o _ {\mathbb {P}} (1), \\ \end{array}
$$
given that  $|n\bar{f}_h| = |\langle \tau_1,\tau_2\rangle |\leq \| \tau_1\|_{\mathcal{H}}\| \tau_2\|_{\mathcal{H}} = O_{\mathbb{P}}(1)O_{\mathbb{P}}(1) = O_{\mathbb{P}}(1)$ . Consequently,  $(n\bar{f}_h^\dagger)^2 = (n\bar{f}_h)^2 +o_{\mathbb{P}}(1)$ , which implies (10) dividing by  $n$  in both sides.
Details of step 3. We claim that
$$
\frac {1}{\mathbb {E} \left[ n \left\{f _ {h} (Z) \right\} ^ {2} \mid \mathcal {D} _ {2} \right]} = O _ {\mathbb {P}} (1). \tag {12}
$$
To show this, we note that
$$
\mathbb {E} [ n \{f _ {h} (Z) \} ^ {2} | \mathcal {D} _ {2} ] = \mathbb {E} [ \langle \phi (Z), \tau_ {2} \rangle^ {2} | \mathcal {D} _ {2} ] \stackrel {(i)} {=} \langle C \tau_ {2}, \tau_ {2} \rangle = \sum_ {i = 1} ^ {\infty} \lambda_ {j} \beta_ {j} ^ {2},
$$
-  $C = \mathbb{E}[\phi(Z) \otimes \phi(Z)]$ ,  $(\lambda_j, j \geq 1)$  is the sequence of non-negative eigenvalues in decreasing order of the covariance operator  $C$ ,
-  $(v_{j}, j \geq 1)$  is the sequence of orthonormal eigenfunctions of the covariance operator  $C$  (and basis of  $\mathcal{H}$  as well),
-  $\tau_{2} = \sum_{i = j}^{\infty}\beta_{j}v_{j}$ , being  $(\beta_{j},j\geq 1)$  the random coefficients of  $\tau_{2}$  with respect to the basis  $(v_{j},j\geq 1)$ ,
and (i) is obtained from Lemma C.2 and the fact that  $\mathbb{E}[\phi(Z)] = 0$ ,  $\mathbb{E}[\| \phi(Z) \|_{\mathcal{H}}^2] \leq \mathbb{E}[\| \phi(Z) \|_{\mathcal{H}}^4] < \infty$ . Based on Lemma C.4,
$$
\tau_ {2} \xrightarrow {d} \sum_ {j = 1} ^ {\infty} \sqrt {\lambda_ {j}} N _ {j} v _ {j},
$$
so by the continuous mapping theorem for random variables in separable Banach spaces (Bosq, 2000, Equation 2.11)
$$
\beta_ {1} = \left\langle \tau_ {2}, v _ {1} \right\rangle \xrightarrow {d} \left\langle \sum_ {j = 1} ^ {\infty} \sqrt {\lambda_ {j}} N _ {j} v _ {j}, v _ {1} \right\rangle = \sqrt {\lambda_ {1}} N _ {1}.
$$
$$
\mathbb {E} \left[ n \left\{f _ {h} (Z) \right\} ^ {2} \mid \mathcal {D} _ {2} \right] = \sum_ {i = 1} ^ {\infty} \lambda_ {j} \beta_ {j} ^ {2} \geq \lambda_ {1} \beta_ {1} ^ {2} \xrightarrow {d} \lambda_ {1} ^ {2} N _ {1} ^ {2}. \tag {13}
$$
Further, by Cauchy-Schwarz inequality,
$$
\begin{array}{l} 0 <   \mathbb {E} \left[ \langle \phi (Z _ {1}), \phi (Z _ {2}) \rangle^ {2} \right] \leq \mathbb {E} \left[ \| \phi (Z _ {1}) \| _ {\mathcal {H}} ^ {2} \| \phi (Z _ {2}) \| _ {\mathcal {H}} ^ {2} \right] = \mathbb {E} ^ {2} \left[ \| \phi (Z) \| _ {\mathcal {H}} ^ {2} \right] \\ = \left(\mathbb {E} \left[ \sum_ {j = 1} ^ {\infty} \langle \phi (Z), v _ {j} \rangle^ {2} \right]\right) ^ {2} \\ = \left(\sum_ {j = 1} ^ {\infty} \mathbb {E} \langle \phi (Z), v _ {j} \rangle^ {2}\right) ^ {2} \\ = \left(\sum_ {j = 1} ^ {\infty} \langle C v _ {j}, v _ {j} \rangle\right) ^ {2} \\ = \left(\sum_ {j = 1} ^ {\infty} \lambda_ {j}\right) ^ {2}. \\ \end{array}
$$
That is, the sum of non-negative eigenvalues is strictly greater than zero, so at least one of them has to be strictly positive. By the non-increasing order of the eigenvalues, this implies that the first one is strictly positive i.e.,  $\lambda_1 > 0$ . For any  $M > 0$  it holds that
$$
\mathbb {P} \left(\mathbb {E} [ n \{f _ {h} (Z) \} ^ {2} | \mathcal {D} _ {2} ] \leq \frac {1}{M}\right) = \mathbb {P} \left(\sum_ {i = 1} ^ {\infty} \lambda_ {j} \beta_ {j} ^ {2} \leq \frac {1}{M}\right) \leq \mathbb {P} \left(\lambda_ {1} \beta_ {1} ^ {2} \leq \frac {1}{M}\right).
$$
Given  $\lambda_1^2 > 0$  and (13), we also obtain
$$
\mathbb {P} \left(\lambda_ {1} \beta_ {1} ^ {2} \leq \frac {1}{M}\right) \stackrel {n \rightarrow \infty} {\rightarrow} \mathbb {P} \left(N _ {1} ^ {2} \leq \frac {1}{M \lambda_ {1} ^ {2}}\right) = F \left(\frac {1}{M \lambda_ {1} ^ {2}}\right)
$$
where  $F$  is the cdf of a  $\chi_1^2$  distribution. Hence for every  $\epsilon > 0$ , there exists  $m \in \mathbb{N}$  such that
$$
\left| \mathbb {P} \left(\lambda_ {1} ^ {2} \beta_ {1} ^ {2} \leq \frac {1}{M}\right) - F \left(\frac {1}{M \lambda_ {1} ^ {2}}\right) \right| <   \frac {\epsilon}{2}
$$
for  $n\geq m$  , which implies
$$
\mathbb {P} \left(\lambda_ {1} ^ {2} \beta_ {1} ^ {2} \leq \frac {1}{M}\right) <   \frac {\epsilon}{2} + F \left(\frac {1}{M \lambda_ {1} ^ {2}}\right).
$$
Further,  $\lim_{w\to 0}F(w) = 0$ , so there exists  $M^{*} > 0$  such that  $F\left(\frac{1}{M^{*}\lambda_1^2}\right) < \frac{\epsilon}{2}$ . Consequently, for every  $\epsilon >0$ , there exist  $M^{*} > 0$  and  $m\in \mathbb{N}$  such that
$$
\mathbb {P} \left(\lambda_ {1} ^ {2} \beta_ {1} ^ {2} \leq \frac {1}{M ^ {*}}\right) <   \frac {\epsilon}{2} + \frac {\epsilon}{2} = \epsilon ,
$$
for  $n\geq m$  , so
$$
\mathbb {P} \left(\frac {1}{\mathbb {E} [ n \{f _ {h} (Z) \} ^ {2} | \mathcal {D} _ {2} ]} \geq M ^ {*}\right) = \mathbb {P} \left(\mathbb {E} [ n \{f _ {h} (Z) \} ^ {2} | \mathcal {D} _ {2} ] \leq \frac {1}{M}\right) \leq \mathbb {P} \left(\lambda_ {1} ^ {2} \beta_ {1} ^ {2} \leq \frac {1}{M ^ {*}}\right) <   \epsilon ,
$$
thus concluding (12).
Details of step 4. Given Conditions (i), (ii), and (iii), Kim and Ramdas (2023, Proof of Theorem 4.2) proved that
$$
\frac {n \bar {f} _ {h}}{\sqrt {\mathbb {E} [ n \left\{f _ {h} (Z) \right\} ^ {2} | \mathcal {D} _ {2} ]}} = \frac {\sqrt {n} \bar {f} _ {h}}{\sqrt {\mathbb {E} [ f _ {h} ^ {2} (Z) | \mathcal {D} _ {2} ]}} \xrightarrow {d} N (0, 1). \tag {14}
$$
$$
\frac {n \left(S _ {h}\right) ^ {2}}{\mathbb {E} \left[ n \left\{f _ {h} (Z) \right\} ^ {2} \mid \mathcal {D} _ {2} \right]} = \frac {S _ {h} ^ {2}}{\mathbb {E} \left[ f _ {h} ^ {2} (Z) \mid \mathcal {D} _ {2} \right]} \xrightarrow {p} 1. \tag {15}
$$
Combining (8) and (12), we obtain
$$
\begin{array}{l} \frac {n \bar {f} _ {h} ^ {\dagger}}{\sqrt {\mathbb {E} [ n \{f _ {h} (Z) \} ^ {2} | \mathcal {D} _ {2} ]}} = \frac {n \bar {f} _ {h} + o _ {\mathbb {P}} (1)}{\sqrt {\mathbb {E} [ n \{f _ {h} (Z) \} ^ {2} | \mathcal {D} _ {2} ]}} \\ = \frac {n \bar {f} _ {h}}{\sqrt {\mathbb {E} [ n \{f _ {h} (Z) \} ^ {2} | \mathcal {D} _ {2} ]}} + o _ {\mathbb {P}} (1) O _ {\mathbb {P}} (1) \\ = \frac {n \bar {f} _ {h}}{\sqrt {\mathbb {E} [ n \left\{f _ {h} (Z) \right\} ^ {2} \mid \mathcal {D} _ {2} ]}} + o _ {\mathbb {P}} (1), \\ \end{array}
$$
and based on (14) and Slutsky's theorem,
$$
\frac {n \bar {f} _ {h} ^ {\dagger}}{\sqrt {\mathbb {E} [ n \left\{f _ {h} (Z) \right\} ^ {2} \mid \mathcal {D} _ {2} ]}} \xrightarrow {d} N (0, 1). \tag {16}
$$
Combining (11) and (12), we obtain
$$
\begin{array}{l} \frac {n S _ {h} ^ {\dagger 2}}{\mathbb {E} [ n \{f _ {h} (Z) \} ^ {2} | \mathcal {D} _ {2} ]} = \frac {n S _ {h} ^ {2} + o _ {\mathbb {P}} (1)}{\mathbb {E} [ n \{f _ {h} (Z) \} ^ {2} | \mathcal {D} _ {2} ]} \\ = \frac {n S _ {h} ^ {2}}{\mathbb {E} [ n \left\{f _ {h} (Z) \right\} ^ {2} | \mathcal {D} _ {2} ]} + o _ {\mathbb {P}} (1) O _ {\mathbb {P}} (1) \\ = \frac {n S _ {h} ^ {2}}{\mathbb {E} [ n \{f _ {h} (Z) \} ^ {2} | \mathcal {D} _ {2} ]} + o _ {\mathbb {P}} (1). \\ \end{array}
$$
Thus based on (15), we obtain
$$
\begin{array}{c} \frac {n S _ {h} ^ {\dagger} {} ^ {2}}{\mathbb {E} [ n \{f _ {h} (Z) \} ^ {2} | \mathcal {D} _ {2} ]} \xrightarrow {p} 1, \end{array}
$$
and by the continuous mapping theorem
$$
\frac {\sqrt {\mathbb {E} \left[ n \left\{f _ {h} (Z) \right\} ^ {2} \mid \mathcal {D} _ {2} \right]}}{\sqrt {n} S _ {h} ^ {\dagger}} \xrightarrow {p} 1. \tag {17}
$$
Combining (16) and (17) and based on Slutsky's theorem, we obtain
$$
\frac{\sqrt{n}f_{h}^{\dagger}}{S_{h}^{\dagger}} = \underbrace{\frac{n f_{h}^{\dagger}}{\sqrt{\mathbb{E}[n\{f_{h}(Z)\}^{2}|\mathcal{D}_{2}]}}}{\underset{\stackrel{d}{\to}N(0,1)}{\longrightarrow N(0,1)}}\underbrace{\frac{\sqrt{\mathbb{E}[n\{f_{h}(Z)\}^{2}|\mathcal{D}_{2}]}}{\sqrt{n}S_{h}^{\dagger}}}_{\stackrel{p}{\longrightarrow}1}\xrightarrow{d} N(0,1).
$$
$$
T _ {h} ^ {\dagger} = \frac {\sqrt {n} \bar {f} _ {h} ^ {\dagger}}{S _ {h} ^ {\dagger}} \stackrel {d} {\rightarrow} N (0, 1).
$$