SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} I _ {4} = \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} | | F _ {t, h _ {2}} - F _ {(s _ {k}, \eta_ {k} ], h _ {2}} | | _ {L _ {2}} ^ {2} - \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} | | F _ {t, h _ {1}} - F _ {(s _ {k}, \widehat {\eta} _ {k} ], h _ {1}} | | _ {L _ {2}} ^ {2} \\ = \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} \left[ \langle F _ {t, h _ {2}}, F _ {t, h _ {2}} \rangle_ {L _ {2}} - 2 \langle F _ {t, h _ {2}}, F _ {(s _ {k}, \eta_ {k} ], h _ {2}} \rangle_ {L _ {2}} + \langle F _ {(s _ {k}, \eta_ {k} ], h _ {2}}, F _ {(s _ {k}, \eta_ {k} ], h _ {2}} \rangle_ {L _ {2}} \right] \\ + \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} \left[ - \langle F _ {t, h _ {1}}, F _ {t, h _ {1}} \rangle_ {L _ {2}} + 2 \langle F _ {t, h _ {1}}, F _ {(s _ {k}, \widehat {\eta} _ {k} ], h _ {1}} \rangle_ {L _ {2}} - \langle F _ {(s _ {k}, \widehat {\eta} _ {k} ], h _ {1}}, F _ {(s _ {k}, \widehat {\eta} _ {k} ], h _ {1}} \rangle_ {L _ {2}} \right] \\ = I _ {4, 1} + I _ {4, 2} + I _ {4, 3}, \\ \end{array}
$$
$$
I _ {4, 1} = \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} \left\langle F _ {t, h _ {2}}, F _ {t, h _ {2}} \right\rangle_ {L _ {2}} - \left\langle F _ {t, h _ {1}}, F _ {t, h _ {1}} \right\rangle_ {L _ {2}}
$$
$$
I _ {4, 2} = \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} 2 \langle F _ {t, h _ {1}}, F _ {(s _ {k}, \widehat {\eta} _ {k} ], h _ {1}} \rangle_ {L _ {2}} - 2 \langle F _ {t, h _ {2}}, F _ {(s _ {k}, \eta_ {k} ], h _ {2}} \rangle_ {L _ {2}}, \text {a n d ,}
$$
$$
I _ {4, 3} = \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} \left\langle F _ {(s _ {k}, \eta_ {k} ], h _ {2}}, F _ {(s _ {k}, \eta_ {k} ], h _ {2}} \right\rangle_ {L _ {2}} - \left\langle F _ {(s _ {k}, \widehat {\eta} _ {k} ], h _ {1}}, F _ {(s _ {k}, \widehat {\eta} _ {k} ], h _ {1}} \right\rangle_ {L _ {2}}.
$$
Now, we explore each of the terms  $I_{4,1}, I_{4,2}$ , and  $I_{4,3}$ . First,  $I_{4,1}$  can be bounded as follows, we add and subtract  $\langle F_{t,h_1}, F_{t,h_2} \rangle_{L_2}$ , to get
$$
\begin{array}{l} \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} \left\langle F _ {t, h _ {2}}, F _ {t, h _ {2}} \right\rangle_ {L _ {2}} - \left\langle F _ {t, h _ {1}}, F _ {t, h _ {1}} \right\rangle_ {L _ {2}} \\ = \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} \left\langle F _ {t, h _ {2}}, F _ {t, h _ {2}} \right\rangle_ {L _ {2}} - \left\langle F _ {t, h _ {1}}, F _ {t, h _ {1}} \right\rangle_ {L _ {2}} + \left\langle F _ {t, h _ {1}}, F _ {t, h _ {2}} \right\rangle_ {L _ {2}} - \left\langle F _ {t, h _ {1}}, F _ {t, h _ {2}} \right\rangle_ {L _ {2}} \\ = \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} \left\langle F _ {t, h _ {2}} - F _ {t, h _ {1}}, F _ {t, h _ {2}} \right\rangle_ {L _ {2}} + \left\langle F _ {t, h _ {1}}, F _ {t, h _ {2}} - F _ {t, h _ {1}} \right\rangle_ {L _ {2}} \\ \end{array}
$$
which, by Hölder's inequality, is bounded by
$$
\sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} | | F _ {t, h _ {2}} | | _ {L _ {2}} | | F _ {t, h _ {2}} - F _ {t, h _ {1}} | | _ {L _ {2}} + | | F _ {t, h _ {1}} | | _ {L _ {2}} | | F _ {t, h _ {2}} - F _ {t, h _ {1}} | | _ {L _ {2}} = \widetilde {r} O _ {p} (\frac {T ^ {- \frac {r}{2 r + p}}}{\mathcal {K} _ {k} ^ {\frac {p}{2 r} + \frac {1}{2} + \frac {p}{2 r}}} \log^ {\frac {r}{2 r + p}} (T)))
$$
since  $||F_{t,h_1} - F_{t,h_2}||_{L_2} = O\left(\frac{|\kappa - \widehat{\kappa}|^{\frac{1}{2}}}{\kappa_k^{\frac{p}{2r} + \frac{1}{2}}}\right) = O_p\left(\frac{1}{\kappa_k^{\frac{p}{2r} + \frac{1}{2}}}\left(\left(\frac{\log(T)}{\Delta}\right)^{\frac{2r}{2r + p}} + \frac{T^{\frac{p}{2r + p}}\log(T)}{\kappa\Delta}\right)^{\frac{1}{2}}\right)$ , for any  $t$ , see Remark 2 for more detail. Similarly, for  $I_{4,2}$ , we have that adding and subtracting  $2\langle F_{t,h_1},F_{(s_k,\eta_k],h_2}\rangle_{L_2}$ ,
$$
\begin{array}{l} \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} 2 \langle F _ {t, h _ {1}}, F _ {(s _ {k}, \widehat {\eta} _ {k} ], h _ {1}} \rangle_ {L _ {2}} - 2 \langle F _ {t, h _ {2}}, F _ {(s _ {k}, \eta_ {k} ], h _ {2}} \rangle_ {L _ {2}} \\ = \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} 2 \langle F _ {t, h _ {1}}, F _ {(s _ {k}, \widehat {\eta} _ {k} ], h _ {1}} \rangle_ {L _ {2}} - 2 \langle F _ {t, h _ {2}}, F _ {(s _ {k}, \eta_ {k} ], h _ {2}} \rangle_ {L _ {2}} + 2 \langle F _ {t, h _ {1}}, F _ {(s _ {k}, \eta_ {k} ], h _ {2}} \rangle_ {L _ {2}} - 2 \langle F _ {t, h _ {1}}, F _ {(s _ {k}, \eta_ {k} ], h _ {2}} \rangle_ {L _ {2}} \\ = \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} 2 \langle F _ {t, h _ {1}} - F _ {t, h _ {2}}, F _ {(s _ {k}, \eta_ {k} ], h _ {2}} \rangle_ {L _ {2}} + 2 \langle F _ {t, h _ {1}}, F _ {(s _ {k}, \widehat {\eta} _ {k} ], h _ {1}} - F _ {(s _ {k}, \eta_ {k} ], h _ {2}} \rangle_ {L _ {2}}, \\ \end{array}
$$
and by Hölder's inequality and Remark 2, it is bounded by
$$
\begin{array}{l} \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} | | F _ {t, h _ {1}} - F _ {t, h _ {2}} | | _ {L _ {2}} | | F _ {(s _ {k}, \eta_ {k} ], h _ {2}} | | _ {L _ {2}} + | | F _ {t, h _ {1}} | | _ {L _ {2}} | | F _ {(s _ {k}, \eta_ {k} ], h _ {2}} - F _ {(s _ {k}, \widehat {\eta} _ {k} ], h _ {1}} | | _ {L _ {2}} \\ = \widetilde {r} O _ {p} \Big (\frac {1}{\kappa_ {k} ^ {\frac {p}{2 r} + \frac {1}{2} + \frac {p}{2 r}}} \Big (\Big (\frac {\log (T)}{\Delta} \Big) ^ {\frac {2 r}{2 r + p}} + \frac {T ^ {\frac {p}{2 r + p}} \log (T)}{\kappa \Delta} \Big) ^ {\frac {1}{2}} + \frac {T ^ {\frac {p}{2 r + p}} \log (T)}{\kappa^ {2}} \Big) \Big). \\ \end{array}
$$
Finally, for  $I_{4,3}$ , we notice that, adding and subtracting  $\langle F_{(s_k,\widehat{\eta}_k],h_1},F_{(s_k,\eta_k],h_2}\rangle_{L_2}$ , it is written as,
$$
\begin{array}{l} \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} \left\langle F _ {(s _ {k}, \eta_ {k} ], h _ {2}}, F _ {(s _ {k}, \eta_ {k} ], h _ {2}} \right\rangle_ {L _ {2}} - \left\langle F _ {(s _ {k}, \eta_ {k} ], h _ {1}}, F _ {(s _ {k}, \eta_ {k} ], h _ {1}} \right\rangle_ {L _ {2}} \\ = \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} \left\langle F _ {(s _ {k}, \eta_ {k} ], h _ {2}}, F _ {(s _ {k}, \eta_ {k} ], h _ {2}} \right\rangle_ {L _ {2}} - \left\langle F _ {(s _ {k}, \widehat {\eta} _ {k} ], h _ {1}}, F _ {(s _ {k}, \widehat {\eta} _ {k} ], h _ {1}} \right\rangle_ {L _ {2}} \\ + \left\langle F _ {\left(s _ {k}, \widehat {\eta} _ {k} \right], h _ {1}}, F _ {\left(s _ {k}, \eta_ {k} \right], h _ {2}} \right\rangle_ {L 2} - \left\langle F _ {\left(s _ {k}, \widehat {\eta} _ {k} \right], h _ {1}}, F _ {\left(s _ {k}, \eta_ {k} \right], h _ {2}} \right\rangle_ {L 2} \\ = \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} \left\langle F _ {\left(s _ {k}, \eta_ {k} \right], h _ {2}} - F _ {\left(s _ {k}, \widehat {\eta} _ {k} \right], h _ {1}}, F _ {\left(s _ {k}, \eta_ {k} \right], h _ {2}} \right\rangle_ {L _ {2}} + \left\langle F _ {\left(s _ {k}, \widehat {\eta} _ {k} \right], h _ {1}}, F _ {\left(s _ {k}, \eta_ {k} \right], h _ {2}} - F _ {\left(s _ {k}, \widehat {\eta} _ {k} \right], h _ {1}} \right\rangle_ {L _ {2}} \\ \end{array}
$$
which, by Hölder's inequality and Remark 2, is bounded by
$$
\begin{array}{l} \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} | | F _ {(s _ {k}, \eta_ {k} ], h _ {2}} | | _ {L _ {2}} | | F _ {(s _ {k}, \widehat {\eta} _ {k} ], h _ {1}} - F _ {(s _ {k}, \eta_ {k} ], h _ {2}} | | _ {L _ {2}} + | | F _ {(s _ {k}, \widehat {\eta} _ {k} ], h _ {1}} | | _ {L _ {2}} | | F _ {(s _ {k}, \eta_ {k} ], h _ {2}} - F _ {(s _ {k}, \widehat {\eta} _ {k} ], h _ {1}} | | _ {L _ {2}} \\ = \widetilde {r} O _ {p} \left(\frac {1}{\kappa_ {k} ^ {\frac {p}{2 r} + \frac {1}{2} + \frac {p}{2 r}}} \left(\left(\frac {\log (T)}{\Delta}\right) ^ {\frac {2 r}{2 r + p}} + \frac {T ^ {\frac {p}{2 r + p}} \log (T)}{\kappa \Delta}\right) ^ {\frac {1}{2}}\right) \\ \end{array}
$$
Then, by above and Assumption 4, we conclude
$$
I _ {4} = o _ {p} \left(\widetilde {r} \kappa_ {k} ^ {\frac {p}{r} + 2}\right). \tag {31}
$$
From (29), (30) and (31), we find that  $U_{1}$  has the following upper bound,
$$
\sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} \left| \left| F _ {t, h _ {2}} - f _ {\left(s _ {k}, \eta_ {k} \right]} * \mathcal {K} _ {h _ {2}} \right| \right| _ {L _ {2}} ^ {2} - \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} \left| \left| F _ {t, h _ {1}} - F _ {\left(s _ {k}, \widehat {\eta} _ {k} \right], h _ {1}} \right| \right| _ {L _ {2}} ^ {2} = o _ {p} \left(\widetilde {r} \kappa_ {k} ^ {\frac {p}{r} + 2}\right). \tag {32}
$$
Now, making an analogous analysis, we have that  $U_{2}$  is upper bounded by,
$$
\sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} \left| \left| F _ {t, h _ {1}} - F _ {\left(\widehat {\eta} _ {k}, e _ {k} \right], h _ {1}} \right| \right| _ {L _ {2}} ^ {2} - \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} \left| \left| F _ {t, h _ {2}} - f _ {\left(\eta_ {k}, e _ {k} \right]} * \mathcal {K} _ {h _ {2}} \right| \right| _ {L _ {2}} ^ {2} = o _ {p} \left(\widetilde {r} \kappa_ {k} ^ {\frac {p}{r} + 2}\right). \tag {33}
$$
In fact, we observe that
$$
\begin{array}{l} \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} | | F _ {t, h _ {1}} - F _ {(\widehat {\eta} _ {k}, e _ {k} ], h _ {1}} | | _ {L _ {2}} ^ {2} - \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} | | F _ {t, h _ {2}} - f _ {(\eta_ {k}, e _ {k} ]} * \mathcal {K} _ {h _ {2}} | | _ {L _ {2}} ^ {2} \\ = \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} | | F _ {t, h _ {2}} - F _ {(\eta_ {k}, e _ {k} ], h _ {2}} | | _ {L _ {2}} ^ {2} - \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} | | F _ {t, h _ {2}} - f _ {(\eta_ {k}, e _ {k} ]} * \mathcal {K} _ {h _ {2}} | | _ {L _ {2}} ^ {2} \\ + \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} | | F _ {t, h _ {1}} - F _ {(\hat {\eta} _ {k}, e _ {k} ], h _ {1}} | | _ {L _ {2}} ^ {2} - \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} | | F _ {t, h _ {2}} - F _ {(\eta_ {k}, e _ {k} ], h _ {2}} | | _ {L _ {2}} ^ {2} \\ = I _ {5} + I _ {6}, \\ \end{array}
$$
$$
I _ {5} = \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} | | F _ {t, h _ {2}} - F _ {(\eta_ {k}, e _ {k} ], h _ {2}} | | _ {L _ {2}} ^ {2} - \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} | | F _ {t, h _ {2}} - f _ {(\eta_ {k}, e _ {k} ]} * \mathcal {K} _ {h _ {2}} | | _ {L _ {2}} ^ {2}, \text {a n d ,}
$$
$$
I _ {6} = \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} | | F _ {t, h _ {1}} - F _ {(\widehat {\eta} _ {k}, e _ {k} ], h _ {1}} | | _ {L _ {2}} ^ {2} - \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} | | F _ {t, h _ {2}} - F _ {(\eta_ {k}, e _ {k} ], h _ {2}} | | _ {L _ {2}} ^ {2}.
$$
Then,  $I_{5}$  is bounded as follows
$$
I _ {5} = \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} | | f _ {(\eta_ {k}, e _ {k} ]} * \mathcal {K} _ {h _ {2}} - F _ {(\eta_ {k}, e _ {k} ], h _ {2}} | | _ {L _ {2}} ^ {2} + 2 \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} \langle f _ {(\eta_ {k}, e _ {k} ]} * \mathcal {K} _ {h _ {2}} - F _ {(\eta_ {k}, e _ {k} ], h _ {2}}, F _ {t, h _ {2}} - f _ {(\eta_ {k}, e _ {k} ]} * \mathcal {K} _ {h _ {2}} \rangle_ {L _ {2}}
$$
$$
I _ {5, 1} = \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} | | f _ {(\eta_ {k}, e _ {k} ]} * \mathcal {K} _ {h _ {2}} - F _ {(\eta_ {k}, e _ {k} ], h _ {2}} | | _ {L _ {2}} ^ {2}, \mathrm {a n d},
$$
$$
I _ {5, 2} = 2 \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {\tau}} \left\langle f _ {\left(\eta_ {k}, e _ {k} \right]} * \mathcal {K} _ {h _ {2}} - F _ {\left(\eta_ {k}, e _ {k} \right], h _ {2}}, F _ {t, h _ {2}} - f _ {\left(\eta_ {k}, e _ {k} \right]} * \mathcal {K} _ {h _ {2}} \right\rangle_ {L _ {2}}.
$$
The term  $I_{5,1} = O_p\left(\widetilde{r}\frac{1}{T}\frac{\log(T)}{\kappa^{\frac{p}{r}}}\right)$ , using Remark 4, we get
$$
I _ {5, 1} = o _ {p} \left(\widetilde {r} \kappa_ {k} ^ {\frac {p}{r} + 2}\right). \tag {34}
$$
For the term  $I_{5,2}$ , by Cauchy Schwartz inequality,
$$
\begin{array}{l} \left\langle f _ {\left(\eta_ {k}, e _ {k} \right]} * \mathcal {K} _ {h _ {2}} - F _ {\left(\eta_ {k}, e _ {k} \right], h _ {2}}, F _ {t, h _ {2}} - f _ {\left(\eta_ {k}, e _ {k} \right]} * \mathcal {K} _ {h _ {2}} \right\rangle_ {L 2} \\ \leq \left\| f _ {\left(\eta_ {k}, e _ {k} \right]} * \mathcal {K} _ {h _ {2}} - F _ {\left(\eta_ {k}, e _ {k} \right], h _ {2}} \right\| _ {L _ {2}} \left\| F _ {t, h _ {2}} - f _ {\left(\eta_ {k}, e _ {k} \right]} * \mathcal {K} _ {h _ {2}} \right\| _ {L _ {2}} \\ \end{array}
$$
for any  $t \in (\eta_k, \eta_k + \widetilde{r}]$ . By Remark 1, we have that  $||f_{(\eta_k, e_k]} * \mathcal{K}_{h_2} - F_{(\eta_k, e_k], h_2})||_{L_2} = O_p\left(\frac{1}{\sqrt{T}} \sqrt{\frac{\log(T)}{\kappa_k^{\frac{p}{r}}}}\right)$ . Therefore,
$$
I _ {5, 2} \leq O _ {p} \Big (\frac {1}{\sqrt {T}} \sqrt {\frac {\log (T)}{\kappa_ {k} ^ {\frac {p}{r}}}} \Big) \Big (\sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} | | F _ {t, h _ {2}} - f _ {[ \eta_ {k} + 1, e _ {k} ]} * \mathcal {K} _ {h _ {2}} | | _ {L _ {2}} \Big)
$$
Now, similarly to the bound for  $I_{2}$ , we consider the random variable
$$
\bar {Y} _ {i} = \frac {\langle F _ {t , h _ {2}} - f _ {(\eta_ {k} , e _ {k} ]} * \mathcal {K} _ {\kappa} , F _ {t , h _ {2}} - f _ {(\eta_ {k} , e _ {k} ]} * \mathcal {K} _ {h _ {2}} \rangle_ {L _ {2}} ^ {\frac {1}{2}} - \mathbb {E} (| | F _ {t , h _ {2}} - f _ {[ \eta_ {k} + 1 , e _ {k} ]} * \mathcal {K} _ {h _ {2}} | | _ {L _ {2}})}{\mathbb {E} (| | F _ {t , h _ {2}} - f _ {\eta_ {k} + 1} * \mathcal {K} _ {h _ {2}} | | _ {L ^ {2}} ^ {3}) ^ {\frac {1}{3}}}.
$$
In order to use Lemma 3, we observe
$$
\mathbb {E} (| \bar {Y _ {i}} | ^ {3}) = \frac {\mathbb {E} (| | F _ {t , h _ {2}} - f _ {\eta_ {k + 1}} * \mathcal {K} _ {h _ {2}} | | _ {L ^ {2}} ^ {3})}{\mathbb {E} (| | F _ {t , h _ {2}} - f _ {\eta_ {k + 1}} * \mathcal {K} _ {h _ {2}} | | _ {L ^ {2}} ^ {3})} = 1.
$$
so that, by Lemma 3,
$$
I _ {5, 2} \leq O _ {p} \left(\frac {1}{\sqrt {T}} \sqrt {\frac {\log (T)}{\kappa_ {r} ^ {\frac {p}{r}}}}\right) \left(O _ {p} \left(\sqrt {\widetilde {r} \kappa_ {k} ^ {- \frac {p}{r}}} \left(\log \left(\widetilde {r} \kappa_ {k} ^ {\frac {p}{r} + 2}\right) + 1\right)\right) + O _ {p} \left(\kappa_ {k} ^ {\frac {- p}{2 r}}\right)\right) = o _ {p} \left(\widetilde {r} \kappa_ {k} ^ {\frac {p}{r} + 2}\right). \tag {35}
$$
To analyze  $I_{6}$ , we observe that
$$
\begin{array}{l} I _ {6} = \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} | | F _ {t, h _ {2}} - F _ {(\eta_ {k}, e _ {k} ], h _ {2}} | | _ {L _ {2}} ^ {2} - \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} | | F _ {t, h _ {1}} - F _ {(\widehat {\eta} _ {k}, e _ {k} ], h _ {1}} | | _ {L _ {2}} ^ {2} \\ = \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} \left[ \langle F _ {t, h _ {2}}, F _ {t, h _ {2}} \rangle_ {L _ {2}} - 2 \langle F _ {t, h _ {2}}, F _ {\langle \eta_ {k}, e _ {k} ], h _ {2}} \rangle_ {L _ {2}} + \langle F _ {(\eta_ {k}, e _ {k} ], h _ {2}}, F _ {(\eta_ {k}, e _ {k} ], h _ {2}} \rangle_ {L _ {2}} \right] \\ \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} \left[ - \langle F _ {t, h _ {1}}, F _ {t, h _ {1}} \rangle_ {L _ {2}} + 2 \langle F _ {t, h _ {1}}, F _ {(\widehat {\eta} _ {k}, e _ {k} ], h _ {1}} \rangle_ {L _ {2}} - \langle F _ {(\widehat {\eta} _ {k}, e _ {k} ], h _ {1}}, F _ {(\widehat {\eta} _ {k}, e _ {k} ], h _ {1}} \rangle_ {L _ {2}} \right] \\ = I _ {6, 1} + I _ {6, 2} + I _ {6, 3}, \\ \end{array}
$$
$$
\begin{array}{l} I _ {6, 1} = \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} \langle F _ {t, h _ {2}}, F _ {t, h _ {2}} \rangle_ {L _ {2}} - \langle F _ {t, h _ {1}}, F _ {t, h _ {1}} \rangle_ {L _ {2}}, \\ I _ {6, 2} = \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} 2 \langle F _ {t, h _ {1}}, F _ {(\widehat {\eta} _ {k}, e _ {k} ], h _ {1}} \rangle_ {L _ {2}} - 2 \langle F _ {t, h _ {2}}, F _ {(\eta_ {k}, e _ {k} ], h _ {2}} \rangle_ {L _ {2}} \\ I _ {6, 3} = \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} \langle F _ {(\eta_ {k}, e _ {k} ], h _ {2}}, F _ {(\eta_ {k}, e _ {k} ], h _ {2}} \rangle_ {L _ {2}} - \langle F _ {(\widehat {\eta} _ {k}, e _ {k} ], h _ {1}}, F _ {(\widehat {\eta} _ {k}, e _ {k} ], h _ {1}} \rangle_ {L _ {2}}. \\ \end{array}
$$
Then we bound each of these terms. First, we rewrite  $I_{6,1}$ , as
$$
\begin{array}{l} \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} | | F _ {t, h _ {2}} | | _ {L _ {2}} | | F _ {t, h _ {2}} - F _ {t, h _ {1}} | | _ {L _ {2}} + | | F _ {t, h _ {1}} | | _ {L _ {2}} | | F _ {t, h _ {2}} - F _ {t, h _ {1}} | | _ {L _ {2}} \\ = \widetilde {r} O _ {p} \left(\frac {1}{\kappa_ {k} ^ {\frac {p}{2 r} + \frac {1}{2} + \frac {p}{2 r}}}\right) \left(\frac {\log (T)}{\Delta}\right) ^ {\frac {2 r}{2 r + p}} + \left. \frac {T ^ {\frac {p}{2 r + p}} \log (T)}{\kappa \Delta}\right) ^ {\frac {1}{2}} \\ \end{array}
$$
since  $||F_{t,\kappa} - F_{t,\widehat{\kappa}}||_{L_2}^2 = O\left(\frac{|\kappa - \widehat{\kappa}|^{\frac{1}{2}}}{\kappa_k^{\frac{p}{2r} + \frac{1}{2}}}\right) = O_p\left(\frac{1}{\kappa_k^{\frac{p}{2r} + \frac{1}{2}}}\left(\frac{\log(T)}{\Delta}\right)^{\frac{2r}{2r + p}} + \frac{T^{\frac{p}{2r + p}}\log(T)}{\kappa\Delta}\right)^{\frac{1}{2}})$ ), for any  $t$ , see
Remark 2 for more detail. Similarly, for  $I_{6,2}$  we have,
$$
\begin{array}{l} \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} 2 \langle F _ {t, h _ {1}}, F _ {(\widehat {\eta} _ {k}, e _ {k} ], h _ {1}} \rangle_ {L _ {2}} - 2 \langle F _ {t, h _ {2}}, F _ {(\eta_ {k}, e _ {k} ], h _ {2}} \rangle_ {L _ {2}} \\ = \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} 2 \langle F _ {t, h _ {1}}, F _ {(\widehat {\eta} _ {k}, e _ {k} ], h _ {1}} \rangle_ {L _ {2}} - 2 \langle F _ {t, h _ {2}}, F _ {(\eta_ {k}, e _ {k} ], h _ {2}} \rangle_ {L _ {2}} + 2 \langle F _ {t, h _ {1}}, F _ {(\eta_ {k}, e _ {k} ], h _ {2}} \rangle_ {L _ {2}} - 2 \langle F _ {t, h _ {1}}, F _ {(\eta_ {k}, e _ {k} ], h _ {2}} \rangle_ {L _ {2}} \\ = \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} 2 \left\langle F _ {t, h _ {1}} - F _ {t, h _ {2}}, F _ {\left(\eta_ {k}, e _ {k} \right], h _ {2}} \right\rangle_ {L _ {2}} + 2 \left\langle F _ {t, h _ {1}}, F _ {\left(\eta_ {k}, e _ {k} \right], h _ {2}} - F _ {\left(\widehat {\eta} _ {k}, e _ {k} \right], h _ {1}} \right\rangle_ {L _ {2}} \\ \end{array}
$$
$$
\begin{array}{l} \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} | | F _ {t, h _ {1}} - F _ {t, h _ {2}} | | _ {L _ {2}} | | F _ {(\eta_ {k}, e _ {k} ], h _ {2}} | | _ {L _ {2}} + | | F _ {t, h _ {1}} | | _ {L _ {2}} | | F _ {(\eta_ {k}, e _ {k} ], h _ {2}} - F _ {(\widehat {\eta} _ {k}, e _ {k} ], h _ {1}} | | _ {L _ {2}} \\ = \widetilde {r} O _ {p} \Big (\frac {1}{\kappa_ {k} ^ {\frac {p}{2 r} + \frac {1}{2} + \frac {p}{2 r}}} \Big (\Big (\frac {\log (T)}{\Delta} \Big) ^ {\frac {2 r}{2 r + p}} + \Big (\frac {T ^ {\frac {p}{2 r + p}} \log (T)}{\kappa \Delta} \Big) ^ {\frac {1}{2}} \Big) \Big). \\ \end{array}
$$
Now for  $I_{6,3}$ , we write it as
$$
\begin{array}{l} \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} \left\langle F _ {\left(\eta_ {k}, e _ {k} \right], h _ {2}}, F _ {\left(\eta_ {k}, e _ {k} \right], h _ {2}} \right\rangle_ {L _ {2}} - \left\langle F _ {\left(\widehat {\eta} _ {k}, e _ {k} \right], h _ {1}}, F _ {\left(\widehat {\eta} _ {k}, e _ {k} \right], h _ {1}} \right\rangle_ {L _ {2}} \\ = \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} \left\langle F _ {\left(\eta_ {k}, e _ {k} \right], h _ {2}}, F _ {\left(\eta_ {k}, e _ {k} \right], h _ {2}} \right\rangle_ {L _ {2}} - \left\langle F _ {\left(\widehat {\eta} _ {k}, e _ {k} \right], h _ {1}}, F _ {\left(\widehat {\eta} _ {k}, e _ {k} \right], h _ {1}} \right\rangle_ {L _ {2}} + \left\langle F _ {\left(\widehat {\eta} _ {k}, e _ {k} \right], h _ {1}}, F _ {\left(\eta_ {k}, e _ {k} \right], h _ {2}} \right\rangle_ {L _ {2}} - \left\langle F _ {\left(\widehat {\eta} _ {k}, e _ {k} \right], h _ {1}}, F _ {\left(\eta_ {k}, e _ {k} \right], h _ {2}} \right\rangle_ {L _ {2}} \\ = \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} \left\langle F _ {\left(\eta_ {k}, e _ {k} \right], h _ {2}} - F _ {\left(\widehat {\eta} _ {k}, e _ {k} \right], h _ {1}}, F _ {\left(s _ {k}, \eta_ {k} \right], h _ {2}} \right\rangle_ {L _ {2}} + \left\langle F _ {\left(\widehat {\eta} _ {k}, e _ {k} \right], h _ {1}}, F _ {\left(\eta_ {k}, e _ {k} \right], h _ {2}} - F _ {\left(\widehat {\eta} _ {k}, e _ {k} \right], h _ {1}} \right\rangle_ {L _ {2}} \\ \end{array}
$$
$$
\begin{array}{l} \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} | | F _ {(\eta_ {k}, e _ {k} ], h _ {2}} | | _ {L _ {2}} | | F _ {(\eta_ {k}, e _ {k} ], h _ {2}} - F _ {(\widehat {\eta} _ {k}, e _ {k} ], h _ {1}} | | _ {L _ {2}} + | | F _ {(\widehat {\eta} _ {k}, e _ {k} ], h _ {1}} | | _ {L _ {2}} | | F _ {(\eta_ {k}, e _ {k} ], h _ {2}} - F _ {(\widehat {\eta} _ {k}, e _ {k} ], h _ {1}} | | _ {L _ {2}} \\ = \widetilde {r} O _ {p} \left(\frac {1}{\kappa_ {k} ^ {\frac {p}{2 r} + \frac {1}{2} + \frac {p}{2 r}}} \left(\left(\frac {\log (T)}{\Delta}\right) ^ {\frac {2 r}{2 r + p}} + \frac {T ^ {\frac {p}{2 r + p}} \log (T)}{\kappa \Delta}\right) ^ {\frac {1}{2}}\right)\left. \right) \\ \end{array}
$$
By above and Assumption 4, we conclude
$$
I _ {6} = o _ {p} \left(\widetilde {r} \kappa_ {k} ^ {\frac {p}{r} + 2}\right). \tag {36}
$$
From, (34), (35) and (36), we get that  $U_{2}$  is bounded by
$$
\sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} | | F _ {t, \widehat {\kappa}} - F _ {(\widehat {\eta} _ {k}, e _ {k} ], \widehat {\kappa}} | | _ {L _ {2}} ^ {2} - \sum_ {t = \eta_ {k} + 1} ^ {\eta_ {k} + \widetilde {r}} | | F _ {t, \kappa} - f _ {(\eta_ {k}, e _ {k} ]} * \mathcal {K} _ {\kappa} | | _ {L _ {2}} ^ {2} = o _ {p} (\widetilde {r} \kappa_ {k} ^ {\frac {p}{r} + 2})
$$
Therefore, from (32) and (33)
$$
\widehat {Q} _ {k} \left(\eta_ {k}\right) - \widehat {Q} _ {k} \left(\eta_ {k} + \widetilde {r}\right) - Q ^ {*} \left(\eta_ {k}\right) + Q ^ {*} \left(\eta_ {k} + \widetilde {r}\right) = o _ {p} \left(\widetilde {r} \kappa_ {k} ^ {\frac {p}{r} + 2}\right) \tag {37}
$$
Step 4: Combination of all the steps above. Finally, combining (24), (27) and (37), uniformly for any  $\widetilde{r} \geq \frac{1}{\kappa_k^{\frac{p}{r} + 2}}$  we have that
$$
\frac {1}{2} \widetilde {r} \kappa_ {k} ^ {2} - O _ {p} \Big (\sqrt {\widetilde {r}} \kappa_ {k} \kappa_ {k} ^ {- \frac {p}{2 r}} (\log (\widetilde {r} \kappa_ {k} ^ {\frac {p}{r} + 2}) + 1) \Big) - o _ {p} (\widetilde {r} \kappa_ {k} ^ {\frac {p}{r} + 2}) \leq o _ {p} (\widetilde {r} \kappa_ {k} ^ {\frac {p}{r} + 2})
$$
which implies,
$$
\widetilde {r} \kappa_ {k} ^ {\frac {p}{r} + 2} = O _ {p} (1) \tag {38}
$$
and complete the proofs of a.1 and b.1.
Limiting distributions. For any  $k \in \{1, \ldots, K\}$ , due to the uniform tightness of  $\widetilde{r}\kappa_{k}^{\frac{p}{r} + 2}$ , (24) and (37), as  $T \to \infty$
$$
Q ^ {*} (\eta) = \sum_ {t = s _ {k} + 1} ^ {\eta} | | F _ {t, h _ {2}} - f _ {(s _ {k}, \eta_ {k} ]} * \mathcal {K} _ {h _ {2}} | | _ {L _ {2}} ^ {2} + \sum_ {t = \eta + 1} ^ {e _ {k}} | | F _ {t, h _ {2}} - f _ {(\eta_ {k}, e _ {k} ]} * \mathcal {K} _ {h _ {2}} | | _ {L _ {2}} ^ {2},
$$
$$
\left| \widehat {Q} \left(\eta_ {k} + \widetilde {r}\right) - \widehat {Q} \left(\eta_ {k}\right) - \left(Q ^ {*} \left(\eta_ {k} + \widetilde {r}\right) - Q ^ {*} \left(\eta_ {k}\right)\right) \right| \xrightarrow {p} 0.
$$
Therefore, it is sufficient to find the limiting distributions of  $Q^{*}\Big(\eta_{k} + \widetilde{r}\Big) - Q^{*}\Big(\eta_{k}\Big)$  when  $T\to \infty$ . Non-vanishing regime. Observe that for  $\widetilde{r} >0$ , we have that when  $T\rightarrow \infty$