SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Theorem H.2 (Ledoux-Talagrand Contraction). Let  $g: \mathbb{R} \to \mathbb{R}$  be a  $L$ -Lipschitz function. For a function class  $\mathcal{F}$ , denote by  $g \circ \mathcal{F}$  the compositions of function in  $\mathcal{F}$  with  $g$ . Then, for all  $n$ ,
$$
\operatorname {R a d} (g \circ \mathcal {F}, n) \leq L \cdot \operatorname {R a d} (\mathcal {F}, n).
$$
Last inequality follows from the fact that the derivative of the log loss is bounded by  $1 / \alpha$  when truncated at level  $\alpha$ . Note that the union bound gives
$$
\Pr \left[ E ^ {c} \right] \leq \sum_ {j = 1} ^ {N / m} \Pr \left[ E _ {j} ^ {c} \right] \leq \frac {N (1 - \sigma) ^ {m}}{m}.
$$
Thus, the law of total expectation gives
$$
\underset {s _ {t}, s _ {t} ^ {\prime} \sim \mathcal {D} _ {t}, R ^ {(t + 1)}} {\mathbb {E}} \left[ \ell \left(h _ {t + 1}, s _ {t} ^ {\prime}\right) - \ell \left(h _ {t + 1}, s _ {t}\right) \right]
$$
$$
\begin{array}{l} \leq \underset {s _ {t}, s _ {t} ^ {\prime} \sim \mathcal {D} _ {t}, R ^ {(t + 1)}} {\mathbb {E}} \left[ \ell \left(h _ {t + 1}, s _ {t} ^ {\prime}\right) - \ell \left(h _ {t + 1}, s _ {t}\right) \mid E \right] + \Pr \left[ E ^ {c} \right] \log (1 / \alpha) \\ \leq \frac {1}{\alpha} \operatorname {R a d} \left(\mathcal {F} _ {\alpha}, N / m\right) + \frac {N (1 - \sigma) ^ {m} \log (1 / \alpha)}{m}. \\ \end{array}
$$
The last equation follows from the fact that the output of the algorithm has loss always bounded by  $\log (1 / \alpha)$ .
We get the desired result by taking the expectation of  $N \sim \mathrm{Poi}(n)$ , and using  $\operatorname*{Pr}[N > n / 2] \geq 1 - e^{-n / 8}$  in the above inequality completes the proof.
I Bound on the Perturbation Term
Lemma I.1 (Perturbation).
$$
\mathbb {E} \left[ \sum_ {i = 1} ^ {N} L (\hat {h}, \tilde {s} _ {t}) - L (h ^ {*}, \tilde {s} _ {t}) \right] \leq n \log \alpha
$$
Proof. Note from the truncation step in Algorithm 1, we have that  $L(\hat{h}, \tilde{s}_t) \leq \log(\alpha)$ . We get the desired bound by taking expectations.