SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\left| \left\langle \widehat {\kappa} _ {k} ^ {\frac {p}{2 r} - 1} (F _ {h _ {1}, i} - f _ {i} * \mathcal {K} _ {h _ {1}}) - \kappa_ {k} ^ {\frac {p}{2 r} - 1} (F _ {h _ {2}, i} - f _ {i} * \mathcal {K} _ {h _ {2}}), (f _ {\widehat {\eta} _ {k}} - f _ {\widehat {\eta} _ {k + 1}}) * \mathcal {K} _ {h _ {1}} \right\rangle_ {L _ {2}} \right|
$$
was previously bounded by,
$$
O _ {p} \Big (\frac {1}{\kappa_ {k} ^ {2 - \frac {p}{2 r}}} \Big (\Big (\frac {\log (T)}{\Delta} \Big) ^ {\frac {2 r}{2 r + p}} + \frac {T ^ {\frac {p}{2 r + p}} \log (T)}{\kappa \Delta} \Big) \Big) O _ {p} \Big (\frac {1}{\kappa_ {k} ^ {\frac {p}{2 r} + \frac {1}{2}}} \Big (\Big (\frac {\log (T)}{\Delta} \Big) ^ {\frac {2 r}{2 r + p}} + \frac {T ^ {\frac {p}{2 r + p}} \log (T)}{\kappa \Delta} \Big) ^ {\frac {1}{2}} \Big) \kappa_ {k}.
$$
$$
\begin{array}{l} I _ {2} \leq \frac {1}{\sqrt {S}} \sum_ {i \in S _ {r}} \left| \breve {Y} _ {i} + Y _ {i} \right| \\ = \sqrt {S} \Big (O _ {p} (1) + O _ {p} \Big (\frac {1}{\kappa_ {k} ^ {2 - \frac {p}{2 r}}} \Big (\Big (\frac {\log (T)}{\Delta} \Big) ^ {\frac {2 r}{2 r + p}} + \frac {T ^ {\frac {p}{2 r + p}} \log (T)}{\kappa \Delta} \Big) \Big) O _ {p} \Big (\frac {1}{\kappa_ {k} ^ {\frac {p}{2 r} + \frac {1}{2}}} \Big (\Big (\frac {\log (T)}{\Delta} \Big) ^ {\frac {2 r}{2 r + p}} + \frac {T ^ {\frac {p}{2 r + p}} \log (T)}{\kappa \Delta} \Big) ^ {\frac {1}{2}} \Big) \kappa_ {k} \Big). \\ \end{array}
$$
In consequences,
$$
\begin{array}{l} \left| \widehat {\sigma} _ {\infty} ^ {2} (k) - \check {\sigma} _ {\infty} ^ {2} (k) \right| = \left| \frac {1}{R} \sum_ {r = 1} ^ {R} \left(\frac {1}{\sqrt {S}} \sum_ {i \in S _ {r}} \check {Y} _ {i}\right) ^ {2} - \frac {1}{R} \sum_ {r = 1} ^ {R} \left(\frac {1}{\sqrt {S}} \sum_ {i \in S _ {r}} Y _ {i}\right) ^ {2} \right| \\ = \left| \frac {1}{R} \sum_ {r = 1} ^ {R} I _ {1} I _ {2} \right| \\ = S \left(O _ {p} \left(\frac {1}{\kappa_ {k} ^ {4 - \frac {p}{r}}} \left(\left(\frac {\log (T)}{\Delta}\right) ^ {\frac {2 r}{2 r + p}} + \frac {T ^ {\frac {p}{2 r + p}} \log (T)}{\kappa \Delta}\right) ^ {2}\right) \right. \\ \cdot O _ {p} \left(\frac {1}{\kappa_ {k} ^ {\frac {p}{r} + 1}} \left(\left(\frac {\log (T)}{\Delta}\right) ^ {\frac {2 r}{2 r + p}} + \frac {T ^ {\frac {p}{2 r + p}} \log (T)}{\kappa \Delta}\right)\right) O _ {p} \left(\frac {1}{\kappa_ {k} ^ {\frac {p}{2 r} + \frac {1}{2}}} \left(\left(\frac {\log (T)}{\Delta}\right) ^ {\frac {2 r}{2 r + p}} + \frac {T ^ {\frac {p}{2 r + p}} \log (T)}{\kappa \Delta}\right) ^ {\frac {1}{2}}\right) \kappa_ {k} \\ + O _ {p} \left(\frac {1}{\kappa_ {k} ^ {2 - \frac {p}{2 r}}} \left(\left(\frac {\log (T)}{\Delta}\right) ^ {\frac {2 r}{2 r + p}} + \frac {T ^ {\frac {p}{2 r + p}} \log (T)}{\kappa \Delta}\right)\right) \\ \cdot O _ {p} \left(\frac {1}{\kappa_ {k} ^ {\frac {p}{2 r} + \frac {1}{2}}} \left(\left(\frac {\log (T)}{\Delta}\right) ^ {\frac {2 r}{2 r + p}} + \frac {T ^ {\frac {p}{2 r + p}} \log (T)}{\kappa \Delta}\right) ^ {\frac {1}{2}}\right) O _ {p} \left(\frac {1}{\kappa_ {k} ^ {\frac {p}{2 r} + \frac {1}{2}}} \left(\left(\frac {\log (T)}{\Delta}\right) ^ {\frac {2 r}{2 r + p}} + \frac {T ^ {\frac {p}{2 r + p}} \log (T)}{\kappa \Delta}\right) ^ {\frac {1}{3}}\right) \\ + O _ {p} (\kappa_ {k} ^ {- 1}) O _ {p} \Big (\frac {1}{\kappa_ {k} ^ {\frac {p}{2 r} + \frac {1}{2}}} \Big (\Big (\frac {\log (T)}{\Delta} \Big) ^ {\frac {2 r}{2 r + p}} + \frac {T ^ {\frac {p}{2 r + p}} \log (T)}{\kappa \Delta} \Big) ^ {\frac {1}{2}} \Big) \\ + O _ {p} \left(\kappa_ {k} ^ {- 1}\right) O _ {p} \left(\frac {1}{\kappa_ {k} ^ {\frac {p}{2 r} + \frac {1}{2}}} \left(\left(\frac {\log (T)}{\Delta}\right) ^ {\frac {2 r}{2 r + p}} + \frac {T ^ {\frac {p}{2 r + p}} \log (T)}{\kappa \Delta}\right) ^ {\frac {1}{2}}\right) \\ \cdot O _ {p} \Big (\frac {1}{\kappa_ {k} ^ {2 - \frac {p}{2 r}}} \Big (\Big (\frac {\log (T)}{\Delta} \Big) ^ {\frac {2 r}{2 r + p}} + \frac {T ^ {\frac {p}{2 r + p}} \log (T)}{\kappa \Delta} \Big) \Big) O _ {p} \Big (\frac {1}{\kappa_ {k} ^ {\frac {p}{2 r} + \frac {1}{2}}} \Big (\Big (\frac {\log (T)}{\Delta} \Big) ^ {\frac {2 r}{2 r + p}} + \frac {T ^ {\frac {p}{2 r + p}} \log (T)}{\kappa \Delta} \Big) ^ {\frac {1}{2}} \Big) \kappa_ {k} \Big). \\ \end{array}
$$
In order to conclude (i), we notice that by Assumption 4 and that  $S = O\left(T^{\frac{-p}{2r + p}}\kappa_k^{\frac{p}{2r} +\frac{3}{2}}\right)$ , which implies,
$$
\left| \widehat {\sigma} _ {\infty} ^ {2} (k) - \breve {\sigma} _ {\infty} ^ {2} (k) \right| = o _ {p} (1).
$$
Now, we are going to see that  $\left|\breve{\sigma}_{\infty}^{2}(k) - \widetilde{\sigma}_{\infty}^{2}(k)\right| \xrightarrow{P} 0, \quad T \to \infty$ . To this end, we will show that the estimator is asymptotically unbiased, and its variance  $\to 0$  as  $T \to \infty$ . First, we notice that, by Hölder's inequality and Minkowsky's inequality,
$$
\begin{array}{l} \left| \breve {Y _ {i}} \right| = \left| \kappa_ {k} ^ {\frac {p}{2 r} - 1} \left\langle F _ {h _ {2}, i} - f _ {i} * \mathcal {K} _ {h _ {2}}, \left(f _ {\eta_ {k}} - f _ {\eta_ {k + 1}}\right) * \mathcal {K} _ {h _ {2}} \right\rangle_ {L _ {2}} \right. \\ \leq \kappa_ {k} ^ {\frac {p}{2 r} - 1} \left\| F _ {h _ {2}, i} - f _ {i} * \mathcal {K} _ {h _ {2}} \right\| _ {L _ {2}} \left\| \left(f _ {\eta_ {k}} - f _ {\eta_ {k + 1}}\right) * \mathcal {K} _ {h _ {2}} \right\| _ {L _ {2}} \\ \leq \kappa_ {k} ^ {\frac {p}{2 r} - 1} \kappa_ {k} ^ {- \frac {p}{2 r}} \kappa_ {k} = 1. \\ \end{array}
$$
Now, we analyze the Bias. We observe that,
$$
\mathbb {E} (\check {\sigma} _ {\infty} ^ {2} (k)) = \frac {1}{R} \sum_ {r = 1} ^ {R} \mathbb {E} \Big (\Big (\frac {1}{\sqrt {S}} \sum_ {i \in \mathcal {S} _ {r}} \check {Y} _ {i} \Big) ^ {2} \Big) = \frac {1}{S} \mathbb {E} \Big (\Big (\sum_ {i \in \mathcal {S} _ {r}} \check {Y} _ {i} \Big) ^ {2} \Big) = \sum_ {l = - S + 1} ^ {S + 1} \frac {S - l}{S} \mathbb {E} (\check {Y} _ {i} \check {Y} _ {i + l})
$$
$$
\widetilde {\sigma} _ {\infty} ^ {2} (k) = \sum_ {l = - \infty} ^ {\infty} \mathbb {E} \left(\check {Y} _ {i} \check {Y} _ {i + l}\right).
$$
so that, the bias has the following form,
$$
\widetilde {\sigma} _ {\infty} ^ {2} (k) - \mathbb {E} (\breve {\sigma} _ {\infty} ^ {2} (k)) = 2 \sum_ {l = S} ^ {\infty} \mathbb {E} (\breve {Y} _ {i} \breve {Y} _ {i + l}) + 2 \sum_ {l = 1} ^ {S} \frac {l}{S} \mathbb {E} (\breve {Y} _ {i} \breve {Y} _ {i + l}).
$$
Now, we show that each of the above terms vanishes as  $T \to \infty$ . We have that, by condition (2) and covariance inequality
$$
2 \sum_ {l = S} ^ {\infty} \mathbb {E} \left(\breve {Y _ {i}} \breve {Y _ {i + l}}\right) \leq 8 \sum_ {l = S} ^ {\infty} | | \breve {Y _ {i}} | | _ {L _ {\infty}} ^ {2} \alpha_ {l} \leq 8 \sum_ {l = S} ^ {\infty} \alpha_ {l} \rightarrow 0, \text {a s} T \rightarrow \infty
$$
where  $\alpha_{l}$  is the mixing coefficient. Then,
$$
2 \sum_ {l = 1} ^ {S} \frac {l}{S} \mathbb {E} \left(\breve {Y} _ {i} \breve {Y} _ {i + l}\right) \leq 8 \sum_ {l = 1} ^ {S} \frac {l}{S} | | \breve {Y} _ {i} | | _ {L _ {\infty}} ^ {2} \alpha_ {l} \leq \frac {C}{S} \rightarrow 0,
$$
by condition (2), choice of  $S$  and Assumption 4. Therefore, we conclude that the Bias vanishes as  $T \to \infty$ . To analyze the Variance, we observe that, if  $Y_{r} = \frac{1}{S}\left(\sum_{i\in S_{r}}\check{Y}_{i}\right)^{2}$
$$
\begin{array}{l} V a r \left(\check {\sigma} _ {\infty} ^ {2} (k)\right) = \mathbb {E} \left(\left(\check {\sigma} _ {\infty} ^ {2} (k) - \mathbb {E} \left(\check {\sigma} _ {\infty} ^ {2} (k)\right)\right) ^ {2}\right) \\ = \frac {1}{R ^ {2}} \mathbb {E} \left(\left(\sum_ {r = 1} ^ {R} Y _ {r} - \mathbb {E} \left(Y _ {r}\right)\right) ^ {2}\right) \\ = \frac {1}{R} \sum_ {l = - R + 1} ^ {R - 1} \frac {R - l}{R} \operatorname {c o v} \left(Y _ {r}, Y _ {l + r}\right) \\ \leq \frac {8}{R} | | Y _ {r} | | _ {L _ {\infty}} ^ {2} \sum_ {l = 0} ^ {\infty} \widetilde {\alpha} _ {l} \leq \frac {8 C S}{R} \rightarrow 0, \text {a s}, T \rightarrow \infty . \\ \end{array}
$$
where,  $\widetilde{\alpha}_l$  are the mixing coefficients of  $\{Y_r\}_{r \in \mathbb{Z}}$ , which is bounded by the mixing coefficient  $\alpha_l$ . From here, we conclude the result (ii).
G Large probability events
In this section, we deal with all the large probability events that occurred in the proof of Theorem 1. Recall that, for any  $(s,e]\subseteq (0,T]$
$$
\widetilde {f} _ {t} ^ {s, e} (x) = \sqrt {\frac {e - t}{(e - s) (t - s)}} \sum_ {l = s + 1} ^ {t} f _ {l} (x) - \sqrt {\frac {t - s}{(e - s) (e - t)}} \sum_ {l = t + 1} ^ {e} f _ {l} (x), x \in \mathcal {X}.
$$
Proposition 1. For any  $x$ ,
$$
\mathbb {P} \Big (\max  _ {\rho \leqslant k \leq T - \widetilde {r}} \Big | \frac {1}{\sqrt {k}} \sum_ {t = \widetilde {r} + 1} ^ {\widetilde {r} + k} \left(\mathcal {K} _ {h} (x - X _ {t}) - \int \mathcal {K} _ {h} (x - z) d F _ {t} (z)\right) \Big | \geq C \sqrt {\frac {\log T}{h ^ {p}}} \Big) \leqslant T ^ {- p - 3}.
$$
Proof. We have that the random variables  $\{Z_{t} = \mathcal{K}_{h}\Big(x - X_{t}\Big)\}_{t = 1}^{T}$  satisfies
$$
\sigma \left(\mathcal {K} _ {h} (x - X _ {t})\right) \subset \sigma (X _ {t})
$$
$$
\left| \mathcal {K} _ {h} \left(x - X _ {t}\right) \right| \leqslant \frac {1}{h ^ {p}} C _ {K}.
$$
Moreover, let
$$
V ^ {2} = \sup _ {t > 0} \Big (v a r (\mathcal {K} _ {h} (x - X _ {t}) + 2 \sum_ {j > t} | c o v (Z _ {t}, Z _ {j}) |) \Big).
$$
We observe that,
$$
\begin{array}{l} \operatorname {v a r} \left(\mathcal {K} _ {h} \left(x - X _ {t}\right)\right) \leq E \left(\left(\frac {1}{h ^ {p}} \mathcal {K} \left(\frac {x - X _ {t}}{h}\right)\right) ^ {2}\right) \\ \leq \int \frac {1}{h ^ {2 p}} \mathcal {K} (\frac {x - z}{h}) d F _ {t} (z) \\ \end{array}
$$
making  $\mu = \frac{x - z}{h}$ , the last inequality is equal to
$$
\begin{array}{l} \int \frac {1}{h ^ {2 p}} \mathcal {K} (\frac {x - z}{h}) d F _ {t} (z) \leq \frac {1}{h ^ {p}} \int \mathcal {K} ^ {2} (u) d F _ {t} (z) \\ \leq \frac {1}{h ^ {p}} C _ {k} C _ {f}. \\ \end{array}
$$
Then, by proposition 2.5 on Fan & Yao (2008),  $|cov(Z_1,Z_1 + t)| \leq C\alpha (t)\frac{1}{h^{2p}} C_K^2$ . On the other hand,
$$
\begin{array}{l} c o v \left(Z _ {1}, Z _ {1} + t\right) = \left| E \left(Z _ {1} Z _ {t + 1}\right) - E \left(Z _ {1}\right) ^ {2} \right| \\ \leq \int \int \mathcal {K} _ {h} (x - z _ {1}) \mathcal {K} _ {h} (x - z _ {2}) g _ {t} (z _ {1}, z _ {2}) d z _ {1} d z _ {2} + E (Z _ {1}) ^ {2} \\ \leq \left\| g _ {t} \right\| _ {L _ {\infty}} + E \left(Z _ {1}\right) ^ {2}. \\ \end{array}
$$
Since by assumption Assumption 1b we have that  $||g_t||_{L_\infty} < \infty$ , and
$$
E \left(Z _ {1}\right) = E \left(\mathcal {K} _ {h} \left(x - X _ {1}\right)\right) = \int \frac {1}{h ^ {p}} \mathcal {K} \left(\frac {x - z}{h}\right) d F _ {t} (z) = \int \mathcal {K} (u) f _ {t} (x - h u) d u = O (1),
$$
we obtain that  $|cov(Z_1, Z_{1+t})|$ . Therefore,  $\sum_{t=1}^{h-1} |cov(Z_1, Z_{t+1})| \leq C \frac{1}{h}$  and, using the mixing condition bound, inequality (2),
$$
\begin{array}{l} \sum_ {t = \frac {1}{h ^ {p}}} ^ {T - 1} | c o v (Z _ {1}, Z _ {1} + t) | \leq D \sum_ {t = \frac {1}{h}} ^ {\infty} \frac {e ^ {- 2 C t}}{h ^ {2 p}} \\ \leq D \frac {e ^ {- 2 C \frac {1}{h ^ {p}}}}{h ^ {2 p}} \\ \leq \widetilde {D} \frac {1}{h ^ {2 p}} h ^ {p} = \widetilde {D} h ^ {p} \\ \end{array}
$$
where the last inequality is followed by the fact  $e^{-x} < \frac{1}{x}$  for  $x > -1$ . In consequence,
$$
\begin{array}{l} V ^ {2} = \sup  _ {t > 0} \left(\operatorname {v a r} \left(\mathcal {K} _ {h} \left(x - X _ {t}\right) + 2 \sum_ {j > t} | c o v \left(Z _ {t}, Z _ {j}\right) |\right)\right) \\ = \widetilde {C} \frac {1}{h ^ {p}} + \widetilde {D} \frac {1}{h ^ {p}} = \widetilde {\widetilde {C}} \frac {1}{h ^ {p}}. \\ \end{array}
$$
Then, by Bernstein inequality for mixing dependence, see Merlevède et al. (2009) for more details, letting
$$
\lambda = C _ {p} \Big (\sqrt {\frac {k \log (T)}{h ^ {p}}} + \sqrt {\frac {\log (T)}{h ^ {2 p}}} + \sqrt {\frac {\log (T) \log^ {2} (k)}{h ^ {p}}} \Big)
$$
$$
\mathbb {P} \Big (\Big | \sum_ {t = \widetilde {r} + 1} ^ {\widetilde {r} + k} \left(\mathcal {K} _ {h} (x - X _ {t}) - \int \mathcal {K} _ {h} (x - z) d F _ {t} (z)\right) \Big | > \lambda \Big) \leq T ^ {- p - 3}.
$$
in consequence,
$$
\mathbb {P} \Big (\left| \frac {1}{\sqrt {k}} \sum_ {t = \widetilde {r} + 1} ^ {\widetilde {r} + k} \left(\mathcal {K} _ {h} (x - X _ {t}) - \int \mathcal {K} _ {h} (x - z) d F _ {t} (z)\right) \right| > \frac {\lambda}{\sqrt {k}} \Big) \leq T ^ {- p - 3}.
$$
Since  $kh^p \geq \log(T)$  if  $k > \rho$ , and  $\log^2(k) = O(k)$ ,
$$
\begin{array}{l} \frac {\lambda}{\sqrt {k}} = \frac {C _ {p} \left(\sqrt {\frac {k \log (T)}{h ^ {p}}} + \sqrt {\frac {\log (T)}{h ^ {2 p}}} + \sqrt {\frac {\log (T) \log^ {2} (k)}{h ^ {p}}}\right)}{\sqrt {k}} \\ = C _ {p} \left(\sqrt {\frac {\log (T)}{h ^ {p}}} + \sqrt {\frac {\log (T)}{k h ^ {2 p}}} + \sqrt {\frac {\log (T) \log^ {2} (k)}{k h ^ {p}}}\right) \\ \leq C _ {p} \left(\sqrt {\frac {\log (T)}{h ^ {p}}} + \sqrt {\frac {1}{h ^ {p}}} + \sqrt {\frac {\log (T)}{h ^ {p}}}\right) \\ \leq C _ {1} \sqrt {\frac {\log (T)}{h ^ {p}}}. \\ \end{array}
$$
It follows that,
$$
\mathbb {P} \Big (\Big | \frac {1}{\sqrt {k}} \sum_ {t = \widetilde {r} + 1} ^ {\widetilde {r} + k} \Big (\mathcal {K} _ {h} (x - X _ {t}) - \int \mathcal {K} _ {h} (x - z) d F _ {t} (z) \Big) \Big | > C _ {1} \sqrt {\frac {\log (T)}{h ^ {p}}} \Big) \leq T ^ {- p - 3}.
$$
Proposition 2. Define the events
$$
\mathcal {A} _ {1} = \left\{\max  _ {t = s + \rho + 1} ^ {e - \rho} \sup  _ {x \in \mathbb {R} ^ {p}} \left| \widetilde {F} _ {t, h} ^ {s, e} (x) - \widetilde {f} _ {t} ^ {s, e} (x) \right| \geq 2 C \sqrt {\frac {\log T}{h ^ {p}}} + \frac {2 C _ {1} \sqrt {p}}{h ^ {p}} + 2 C _ {2} \sqrt {T} h ^ {r} \right\}
$$
$$
\mathcal {A} _ {2} = \Big \{\max _ {\rho \leqslant k \leq T - \widetilde {r}} \sup _ {x \in \mathbb {R} ^ {p}} \Big | \frac {1}{\sqrt {k}} \sum_ {t = \widetilde {r} + 1} ^ {\widetilde {r} + k} \Big (\mathcal {K} _ {h} (x - X _ {t}) - f _ {t} (x) \Big) \Big | \geq C \sqrt {\frac {\log T}{h ^ {p}}} + \frac {C _ {1} \sqrt {p}}{h ^ {p}} + C _ {2} \sqrt {T} h ^ {r} \Big \}.
$$
$$
\mathbb {P} \left(\mathcal {A} _ {1}\right) \leqslant 2 R ^ {p} T ^ {- 2} \tag {43}
$$
$$
\mathbb {P} \left(\mathcal {A} _ {2}\right) \leqslant R ^ {p} T ^ {- 2} \tag {44}
$$
where  $R$  is a positive constant.
$$
\begin{array}{l} \max  _ {\rho \leqslant k \leqslant T - \widetilde {r}} \sup  _ {x \in \mathbb {R} ^ {p}} \left| \frac {1}{\sqrt {k}} \sum_ {t = \widetilde {r} + 1} ^ {\widetilde {r} + k} \mathcal {K} _ {h} (x - X _ {t}) - f _ {t} (x) \right| \\ \leq \max  _ {\rho \leqslant k \leq T - \widetilde {r}} \sup  _ {x \in \mathbb {R} ^ {p}} \left| \frac {1}{\sqrt {k}} \sum_ {t = \widetilde {r} + 1} ^ {\widetilde {r} + k} \left(\mathcal {K} _ {h} (x - X _ {t}) - \int \mathcal {K} _ {h} (x - z) d F _ {t} (z)\right) \right| \\ + \max  _ {\rho \leqslant k \leqslant T - \widetilde {r}} \sup  _ {x \in \mathbb {R} ^ {p}} \left| \frac {1}{\sqrt {k}} \sum_ {t = \widehat {r} + 1} ^ {r + k} \left(\int \mathcal {K} _ {h} (x - z) d F _ {z} (z) - f _ {t} (x)\right) \right| = I _ {1} + I _ {2} \\ \end{array}
$$
Now we will bound each of the terms  $I_1, I_2$ . For  $I_1$ , we consider
$$
A = \{x _ {1}, \dots , x _ {(R \sqrt {T} / \sqrt {h ^ {p}} h) ^ {p}} \}
$$
with  $\cup_{x_i\in \{x_1,\ldots ,x_{(R\sqrt{T} /\sqrt{h^{\overline{P}}h})^{p}}}\} Rec(x_i,\frac{\sqrt{h^{\overline{P}}h}}{\sqrt{T}})\supset D$  where  $D$  is the support of  $K$  and  $Rec(x_{i},\frac{\sqrt{h^{p}}h}{\sqrt{T}})$  are boxes centered at  $x_{i}$  of size  $\frac{\sqrt{h^{\overline{P}}h}}{\sqrt{T}}$  and  $R$  is the size of the boxe containing  $D$ . Then by Proposition 1, for any  $x_{i}$ ,
$$
\mathbb {P} \left(\max  _ {\rho \leqslant k \leq T - \tilde {r}} \left| \frac {1}{\sqrt {k}} \sum_ {t = \tilde {r} + 1} ^ {\tilde {r} + k} \left(\mathcal {K} _ {h} \left(x _ {i} - X _ {t}\right) - \int \mathcal {K} _ {h} \left(x _ {i} - z\right) d F _ {t} (z)\right) \right| \geq C \sqrt {\frac {\log T}{h ^ {p}}}\right) \leqslant T ^ {- p - 3}, \tag {45}
$$
by an union bound argument,
$$
\mathbb {P} \left(\max  _ {\rho \leqslant k \leq T - \widetilde {r}} \sup  _ {x \in A} \left| \frac {1}{\sqrt {k}} \sum_ {t = \widetilde {r} + 1} ^ {\widetilde {r} + k} \left(\mathcal {K} _ {h} (x - X _ {t}) - \int \mathcal {K} _ {h} (x - z) d F _ {t} (z)\right) \right| \geq C \sqrt {\frac {\log T}{h ^ {p}}}\right) \leqslant T ^ {- p - 3} | A |. \tag {46}
$$
Let  $I_{1,1} = \{\max_{\rho \leqslant k \leq T - \widetilde{r}} \sup_{x \in A} \left| \frac{1}{\sqrt{k}} \sum_{t = \widetilde{r} + 1}^{\widetilde{r} + k} \left( \mathcal{K}_h(x - X_t) - \int \mathcal{K}_h(x - z) dF_t(z) \right) \right| \}$ . For any  $x \in \mathbb{R}^p$ , there exist  $x_i \in A$  such that
$$
\begin{array}{l} \max  _ {\rho \leqslant k \leq T - \widetilde {r}} \left| \frac {1}{\sqrt {k}} \sum_ {t = \widetilde {r} + 1} ^ {\widetilde {r} + k} \left(\mathcal {K} _ {h} (x - X _ {t}) - \int \mathcal {K} _ {h} (x - z) d F _ {t} (z)\right) \right| \\ \leq \max  _ {\rho \leqslant k \leq T - \widetilde {r}} \left| \frac {1}{\sqrt {k}} \sum_ {t = \widetilde {r} + 1} ^ {\widetilde {r} + k} \left(\mathcal {K} _ {h} \left(x _ {i} - X _ {t}\right) - \int \mathcal {K} _ {h} \left(x _ {i} - z\right) d F _ {t} (z)\right) \right| \\ + \max  _ {\rho \leqslant k \leq T - \widetilde {r}} \left| \frac {1}{\sqrt {k}} \sum_ {t = \widetilde {r} + 1} ^ {\widetilde {r} + k} \left(\mathcal {K} _ {h} (x - X _ {t}) - \mathcal {K} _ {h} (x _ {i} - X _ {t})\right) \right| \\ + \max  _ {\rho \leqslant k \leq T - \widetilde {r}} \left| \frac {1}{\sqrt {k}} \sum_ {t = \widetilde {r} + 1} ^ {\widetilde {r} + k} \left(\int \mathcal {K} _ {h} (x _ {i} - z) d F _ {t} (z) - \int \mathcal {K} _ {h} (x - z) d F _ {t} (z)\right) \right| \\ \leq \max  _ {\rho \leqslant k \leq T - \widetilde {r}} \sup  _ {x \in A} \left| \frac {1}{\sqrt {k}} \sum_ {t = \widetilde {r} + 1} ^ {\widetilde {r} + k} \left(\mathcal {K} _ {h} (x _ {i} - X _ {t}) - \int \mathcal {K} _ {h} (x _ {i} - z) d F _ {t} (z)\right) \right| \\ + \max  _ {\rho \leqslant k \leq T - \widetilde {r}} \left| \frac {1}{\sqrt {k}} \sum_ {t = \widetilde {r} + 1} ^ {\widetilde {r} + k} \left(\mathcal {K} _ {h} (x - X _ {t}) - \mathcal {K} _ {h} (x _ {i} - X _ {t})\right) \right| \\ + \max  _ {\rho \leqslant k \leq T - \widetilde {r}} \left| \frac {1}{\sqrt {k}} \sum_ {t = \widetilde {r} + 1} ^ {\widetilde {r} + k} \left(\int \mathcal {K} _ {h} (x _ {i} - z) d F _ {t} (z) - \int \mathcal {K} _ {h} (x - z) d F _ {t} (z)\right) \right| \\ = I _ {1, 1} + I _ {1, 2} + I _ {1, 3} \\ \end{array}
$$
The term  $I_{1,2}$  is bounded as followed.
$$
\begin{array}{l} \max  _ {\rho \leqslant k \leq T - \widetilde {r}} \left| \frac {1}{\sqrt {k}} \sum_ {t = \widetilde {r} + 1} ^ {\widetilde {r} + k} \left(\mathcal {K} _ {h} (x - X _ {t}) - \mathcal {K} _ {h} (x _ {i} - X _ {t})\right) \right| \\ \leq \max  _ {\rho \leqslant k \leq T - \tilde {r}} \frac {1}{\sqrt {k}} \sum_ {t = \tilde {r} + 1} ^ {\tilde {r} + k} \left| \mathcal {K} _ {h} (x - X _ {t}) - \mathcal {K} _ {h} (x _ {i} - X _ {t}) \right| \\ \leq \max  _ {\rho \leqslant k \leq T - \widetilde {r}} \frac {1}{\sqrt {k}} \sum_ {t = \widetilde {r} + 1} ^ {\widetilde {r} + k} \frac {| x - x _ {i} |}{h ^ {p + 1}} \\ \leq \max  _ {\rho \leqslant k \leq T - \widetilde {r}} \frac {1}{\sqrt {k}} \sum_ {t = \widetilde {r} + 1} ^ {\widetilde {r} + k} \frac {\sqrt {h ^ {p}} h \sqrt {p}}{\sqrt {T} h ^ {p + 1}} \leq \frac {\sqrt {p}}{\sqrt {h ^ {p}}} \\ \end{array}
$$
For the term  $I_{1,3}$ , since the random variables  $\{\mathcal{K}_h(x - X_t)\}_{t = 1}^T$  have bounded expected value for any  $x\in \mathbb{R}^p$
$$
\begin{array}{l} \max  _ {\rho \leqslant k \leq T - \widetilde {r}} \left| \frac {1}{\sqrt {k}} \sum_ {t = \widetilde {r} + 1} ^ {\widetilde {r} + k} \left(\int \mathcal {K} _ {h} (x _ {i} - z) d F _ {t} (z) - \int \mathcal {K} _ {h} (x - z) d F _ {t} (z)\right) \right| \\ \leq \max  _ {\rho \leqslant k \leq T - \widetilde {r}} \left| \frac {1}{\sqrt {k}} \sum_ {t = \widetilde {r} + 1} ^ {\widetilde {r} + k} \left(\frac {\sqrt {h ^ {p}} h 2 C \sqrt {p}}{h ^ {p + 1} \sqrt {T}}\right) \right| \leq \frac {2 C \sqrt {p}}{\sqrt {h ^ {p}}}. \\ \end{array}
$$
$$
\begin{array}{l} \max  _ {\rho \leqslant k \leq T - \widetilde {r}} \sup  _ {x \in \mathbb {R} ^ {p}} \left| \frac {1}{\sqrt {k}} \sum_ {t = \widetilde {r} + 1} ^ {\widetilde {r} + k} \left(\mathcal {K} _ {h} (x - X _ {t}) - \int \mathcal {K} _ {h} (x - z) d F _ {t} (z)\right) \right| \\ \leq I _ {1, 1} + C _ {2} \frac {\sqrt {p}}{\sqrt {h ^ {p}}} \\ \end{array}
$$
From here,
$$
\begin{array}{l} \mathbb {P} \left(I _ {1} > C _ {1} \sqrt {\frac {\log T}{h ^ {p}}} + \frac {C _ {2} \sqrt {p}}{\sqrt {h ^ {p}}}\right) \leq \mathbb {P} \left(I _ {1, 1} + I _ {1, 2} + I _ {1, 3} > C _ {1} \sqrt {\frac {\log T}{h ^ {p}}} + \frac {C _ {2} \sqrt {p}}{\sqrt {h ^ {p}}}\right) (47) \\ \leq \mathbb {P} \left(I _ {1, 1}\right) \leq T ^ {- p - 3} | A | = T ^ {- p - 2} \left(R \sqrt {T} / \sqrt {h ^ {p}} h\right) ^ {p} (48) \\ \leq T ^ {- p - 3} \left(R \sqrt {T} \sqrt {T} T ^ {\frac {1}{p}}\right) ^ {p} = R ^ {p} T ^ {- 2} (49) \\ \end{array}
$$
Finally, we analyze the term  $I_{2}$ . By the adaptive assumption, the following is satisfied,
$$
\begin{array}{l} \max  _ {\rho \leqslant k \leqslant T - \widetilde {r}} \sup  _ {x \in \mathbb {R} ^ {p}} \left| \frac {1}{\sqrt {k}} \sum_ {t = \widehat {r} + 1} ^ {r + k} \left(\int \mathcal {K} _ {h} (x - z) d F _ {z} (z) - f _ {t} (x)\right) \right| \\ \leq \max  _ {\rho \leqslant k \leqslant T - \widetilde {r}} \frac {1}{\sqrt {k}} \sum_ {t = \widetilde {r} + 1} ^ {r + k} \sup  _ {x \in \mathbb {R} ^ {p}} \left| \int \mathcal {K} _ {h} (x - z) d F _ {z} (z) - f _ {t} (x) \right| \\ \leq \max  _ {\rho \leqslant k \leqslant T - \widetilde {r}} \frac {1}{\sqrt {k}} \sum_ {t = \widehat {r} + 1} ^ {r + k} C _ {2} h ^ {r} \\ <   C _ {2} \sqrt {T} h ^ {r} \\ \end{array}
$$
We conclude the bound for event  $\mathcal{A}_2$ . We conclude the bound for event  $\mathcal{A}_2$ . Next, to derive the bound for event  $\mathcal{A}_1$ , by definition of  $\widetilde{F}_{t,h}^{s,e}$  and  $\widetilde{f}_t^{s,e}$ , we have that
$$
\begin{array}{l} \left| \widetilde {F} _ {t, h} ^ {s, e} (x) - \widetilde {f} _ {t} ^ {s, e} (x) \right| \leq \left| \sqrt {\frac {e - t}{(e - s) (t - s)}} \sum_ {l = s + 1} ^ {t} \left(F _ {l, h} (x) - f _ {l, h} (x)\right) \right| \\ + \left| \sqrt {\frac {t - s}{(e - s) (e - t)}} \sum_ {l = t + 1} ^ {e} \left(F _ {l, h} (x) - f _ {l, h} (x)\right) \right|. \\ \end{array}
$$
Then, we observe that,
$$
\sqrt {\frac {e - t}{(e - s) (t - s)}} \leq \sqrt {\frac {1}{t - s}} \mathrm {i f} s \leq t, \mathrm {a n d} \sqrt {\frac {t - s}{(e - s) (e - t)}} \leq \sqrt {\frac {1}{e - t}} \mathrm {i f} t \leq e.
$$
$$
\begin{array}{l} X = \max _ {t = s + \rho + 1} ^ {e - \rho} \left| \widetilde {F} _ {t, h} ^ {(s, e)} (x) - \widetilde {f} _ {t} ^ {s, e} (x) \right| \leq \max _ {t = s + \rho + 1} ^ {e - \rho} \left| \sqrt {\frac {1}{t - s}} \sum_ {l = s + 1} ^ {t} \left(F _ {l, h} (x) - \{f _ {l, h} (x)\}\right) \right| \\ + \max _ {t = s + \rho + 1} ^ {e - \rho} \left| \sqrt {\frac {1}{e - t}} \sum_ {l = t + 1} ^ {e} \left(F _ {l, h} (x) - f _ {l, h} (x)\right) \right| = X _ {1} + X _ {2}. \\ \end{array}
$$
Finally, letting  $\lambda = 2C_1\sqrt{\frac{\log T}{h^p}} +\frac{2C_2\sqrt{p}}{\sqrt{h^p}} +2C_2\sqrt{T} h^r$  , we get that
$$
\begin{array}{l} \mathbb {P} (X \geq \lambda) \leq \mathbb {P} (X _ {1} + X _ {2} \geq \frac {\lambda}{2} + \frac {\lambda}{2}) \\ \leq \mathbb {P} (X _ {1} \geq \frac {\lambda}{2}) + \mathbb {P} (X _ {2} \geq \frac {\lambda}{2}) \\ \leq 2 R ^ {p} T ^ {- 2}, \\ \end{array}
$$
where the last inequality follows from above. This concludes the bound for  $\mathcal{A}_1$ .
Remark 1. On the events  $(\mathcal{A}_1)^c$  and,  $(\mathcal{A}_2)^c$ , by Assumption 1, we have that