SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \mathbb {E} \left\| \frac {1}{\left| T _ {s} (t - 1) \right|} \sum_ {j \in T _ {s} (t - 1)} \mathbb {I} \left\{Z _ {j} = 1 \right\} \mathbf {w} ^ {*} (\boldsymbol {\alpha} _ {j}) - \frac {1}{\left| T _ {s} (t) \right|} \sum_ {j \in T _ {s} (t)} \mathbf {w} ^ {*} (\boldsymbol {\alpha} _ {j}) \right\| \\ = \mathbb {E} \left\| \sum_ {j \in T _ {s} (t - 1)} \left(\frac {\mathbb {I} \{Z _ {j} = 1 \}}{| T _ {s} (t - 1) |} - \frac {1}{| T _ {s} (t) |}\right) \mathbf {w} ^ {*} (\boldsymbol {\alpha} _ {j}) \right\| \\ \leq \sum_ {j \in T _ {s} (t - 1)} \mathbb {E} _ {Z _ {j}} \left\| \left(\frac {\mathbb {I} \left\{Z _ {j} = 1 \right\}}{\left| T _ {s} (t - 1) \right|} - \frac {1}{\left| T _ {s} (t) \right|}\right) \mathbf {w} ^ {*} (\boldsymbol {\alpha} _ {j}) \right\| \\ \leq \sum_ {j \in T _ {s} (t - 1)} \left(p \frac {1}{| T _ {s} (t - 1) | | T _ {s} (t) |} + (1 - p) \frac {1}{| T _ {s} (t) |}\right) D \\ \leq \left(p \frac {1}{| T _ {s} (t) |} + (1 - p)\right) D \\ \end{array}
$$
where at the second last step we use the fact that  $|T_{s}(t)| = |T_{s}(t - 1)| + 1$ .
$$
\begin{array}{l} \sum_ {t \in T _ {s}} \ell_ {t} (\hat {\mathbf {w}} _ {t}) - \ell_ {t} (\mathbf {w} _ {s} ^ {\star}) \leq \sum_ {t \in T _ {s}} \ell_ {t} (\hat {\mathbf {w}} _ {t}) - \ell_ {t} (\tilde {\mathbf {w}} _ {t}) \\ \leq 4 D \sum_ {t \in T _ {s}} \left(p \frac {1}{| T _ {s} (t) |} + (1 - p)\right) D + 4 p D | T _ {s} | (1 - \frac {1}{\kappa}) ^ {K} D \\ \end{array}
$$
We further sum above statement over all  $s \in S_T$ :
$$
\begin{array}{l} \mathbb {E} \left[ \sum_ {s \in S _ {T}} \sum_ {t \in T _ {s}} \ell_ {t} (\hat {\mathbf {w}} _ {t}) - \ell_ {t} (\mathbf {w} _ {s} ^ {\star}) \right] \leq p \sum_ {s \in S _ {T}} \left(\ln (| T _ {s} |) + (1 - p) T D + 4 p D | T _ {s} | (1 - \frac {1}{\kappa}) ^ {K} D\right) \\ \leq p \left| S _ {T} \right| \ln (T) + (1 - p) T D + 4 p D T \left(1 - \frac {1}{\kappa}\right) ^ {K} D \\ \end{array}
$$
The first inequality is a known bound on the regret under square loss [6, page 43]. We upper bound the size of the final packing  $S_T$  as
$$
\left| S _ {T} \right| \leq \mathcal {M} \left(B, \varepsilon_ {T}, \| \cdot \|\right) \leq C _ {N} \varepsilon_ {T} ^ {- N}
$$
$$
\sum_ {t = 1} ^ {T} \left(\ell_ {t} \left(\hat {\mathbf {w}} _ {t}\right) - \ell_ {t} \left(\mathbf {w} _ {\pi_ {t} (\boldsymbol {\alpha} _ {t})} ^ {\star}\right)\right) \leq p C _ {N} \varepsilon_ {T} ^ {- N} \ln (T) + (1 - p) T D + 4 p D T \left(1 - \frac {1}{\kappa}\right) ^ {K} D \tag {31}
$$
Next, we bound the approximation term. Using (30) we have
$$
\sum_ {t = 1} ^ {T} \left(\ell_ {t} \left(\mathbf {w} _ {\pi_ {t} (\boldsymbol {\alpha} _ {t})} ^ {\star}\right) - \ell_ {t} \left(f (\boldsymbol {\alpha} _ {t})\right)\right) \leq \sum_ {t = 1} ^ {T} \left(\ell_ {t} \left(f \left(\boldsymbol {\alpha} _ {\pi_ {t} \left(\boldsymbol {\alpha} _ {t}\right)}\right)\right) - \ell_ {t} \left(f (\boldsymbol {\alpha} _ {t})\right)\right). \tag {32}
$$
Note that  $\ell_t$  is  $4D$ -Lipschitz because  $\mathbf{y}_t, \hat{\mathbf{y}}_t \in \mathbb{B}_2(D)$ . Hence,
$$
\left. \right. \ell_ {t} \left(f \left(\alpha_ {\pi_ {t} \left(\boldsymbol {\alpha} _ {t}\right)}\right)\right) - \ell_ {t} \left(f (\boldsymbol {\alpha} _ {t})\right) \leq 4 D \left| f \left(\boldsymbol {\alpha} _ {\pi_ {t} \left(\boldsymbol {\alpha} _ {t}\right)}\right) - f (\boldsymbol {\alpha} _ {t}) \right| \leq 4 D \kappa^ {*} \| \boldsymbol {\alpha} _ {\pi_ {t} \left(\boldsymbol {\alpha} _ {t}\right)} - \boldsymbol {\alpha} _ {t} \| \leq 4 D \varepsilon_ {t}
$$
by  $\kappa^{*}$ -Lipschitzness of  $f$ . Thus, putting all together,
$$
\sum_ {t = 1} ^ {T} \ell_ {t} (\hat {\mathbf {w}} _ {t}) - \sum_ {t = 1} ^ {T} \ell_ {t} (f (\boldsymbol {\alpha} _ {t})) \leq 8 \ln (e T) C _ {N} \varepsilon_ {T} ^ {- N} + 4 D \kappa^ {*} \sum_ {t = 1} ^ {T} \varepsilon_ {t}
$$
and recalling that  $\varepsilon_{t} = t^{-\frac{1}{1 + N}}$  we have
$$
\sum_ {t = 1} ^ {T} t ^ {- \frac {1}{1 + N}} \leq \sum_ {t = 1} ^ {T} t ^ {- \frac {1}{1 + N}} \leq \int_ {0} ^ {T} \tau^ {- \frac {1}{1 + N}} d \tau = \left(1 + \frac {1}{N}\right) T ^ {\frac {N}{1 + N}} \leq 2 T ^ {\frac {N}{1 + N}}
$$