SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \mathbb {E} _ {Z ^ {t - 1}} \inf  _ {p} \mathbb {E} _ {f \sim p} \left[ - \gamma \Delta V _ {f} ^ {*} - \sum_ {h = 1} ^ {H} \sum_ {\iota = 1} ^ {t - 1} \Delta L _ {h} ^ {\iota} (f) + \log \frac {p (f)}{p ^ {0} (f)} \right] \\ \leq \mathbb {E} _ {Z ^ {t - 1}} \mathbb {E} _ {f \sim \widetilde {p}} \left[ - \gamma \Delta V _ {f} ^ {*} - \sum_ {h = 1} ^ {H} \sum_ {\iota = 1} ^ {t - 1} \Delta L _ {h} ^ {\iota} (f) + \log \frac {p (f)}{p ^ {0} (f)} \right] \\ \leq \left(3 H ^ {2} \gamma + \eta H T\right) \varepsilon - \log p ^ {0} (\mathcal {F} (\varepsilon)) \\ \leq \omega (4 H T, p ^ {0}), \\ \end{array}
$$
where the second inequality is by Lemma 13 and Lemma 14 as well as  $\varepsilon \leq 1$ , and the last inequality is by our setting that  $\eta = \frac{1}{2}$  and  $\gamma \leq T$ . Therefore, by combining the above results, we obtain
$$
\begin{array}{l} \mathbb {E} \left[ \operatorname {R e g} ^ {\mathrm {a d v}} (T) \right] \leq \frac {\omega \left(4 H T , p ^ {0}\right) T}{\gamma} + \frac {\gamma d _ {\mathrm {G E C}} H}{4} + 2 H \left(d _ {\mathrm {G E C}} H T\right) ^ {\frac {1}{2}} + \epsilon H T \\ \leq 4 \sqrt {d _ {\mathrm {G E C}} H T \cdot \omega (4 H T , p ^ {0})}, \\ \end{array}
$$
where we set  $HT > d_{\mathrm{GEC}}$ ,  $\epsilon = 1 / \sqrt{HT}$  for adversarial GEC in Definition 3, and  $\gamma = 2\sqrt{\omega(4HT,p^0)T / d_{\mathrm{GEC}}}$ . Note that since we set  $\epsilon = 1 / \sqrt{HT}$ , the adversarial GEC  $d_{\mathrm{GEC}}$  is now calculated associated with  $\epsilon = 1 / \sqrt{HT}$  by Definition 3. Further by our analysis in Appendix B, we know that this setting only introduces  $\log T$  factors in  $d_{\mathrm{GEC}}$  in the worst case. This concludes the proof.
G Other Supporting Lemmas
Lemma 15 (Elliptical Potential Lemma in [1], Lemma 11). Suppose  $\{\phi_t\}_{t\geq 0}$  is a sequence in  $\mathbb{R}^d$  satisfying  $\| \phi_t\| _2\leq 1$ , and  $\Lambda_0$  is a positive definite  $d\times d$  matrix. Let  $\Lambda_{t} = \Lambda_{0} + \sum_{\iota = 1}^{t}\phi_{\iota}\phi_{\iota}^{\top}$ . Then, the following inequalities hold
$$
\log \left(\frac {\operatorname* {d e t} \Lambda_ {t}}{\operatorname* {d e t} \Lambda_ {0}}\right) \leq \sum_ {\iota = 1} ^ {t} \min  \left\{\phi_ {\iota} ^ {\top} (\Lambda_ {\iota - 1}) ^ {- 1} \phi_ {\iota}, 1 \right\} \leq 2 \log \left(\frac {\operatorname* {d e t} \Lambda_ {t}}{\operatorname* {d e t} \Lambda_ {0}}\right).
$$
Lemma 16 (Estimation of Elliptical Potential [38]). Given  $\lambda >0$  and  $\{\phi_t\}_{t\geq 0}$  with  $\| \phi_t\| _2\leq 1$  denoting  $\Lambda_{t} = \lambda \mathrm{I} + \sum_{\iota = 1}^{t}\phi_{\iota}\phi_{\iota}^{\top}$ , then  $\phi_{\iota}^{\top}(\Lambda_{\iota -1})^{-1}\phi_{\iota}$  is upper bounded by
$$
\frac {3 d}{\log 2} \log \left(1 + \frac {1}{\lambda \log 2}\right).
$$
Lemma 17 ( $\ell_2$  Eluder Technique [13, 83]). Suppose that  $\{w_{t,j} \in \mathbb{R}^d\}_{(t,j) \in [T] \times [J]}$ ,  $\{x_{t,i} \in \mathbb{R}^d_{(t,i) \in [T] \times [I]}\}_{(t,i)}$ , and distributions  $\{p_t \in \Delta_{[I]}\}_{[T]}$  satisfy
-  $\sum_{s=1}^{t-1} \mathbb{E}_{i \sim p_s} \left( \sum_{j=1}^J |w_{t,j}^\top x_{s,i}| \right)^2 \leq \gamma_t$ ,
-  $\mathbb{E}_{i\sim p_t}\| x_{t,i}\| _2^2\leq R_x^2$
$$
\cdot \sum_ {j = 1} ^ {J} \| w _ {t, j} \| _ {2} \leq R _ {w},
$$
for any  $1 \leq t \leq T$ . Then, for  $R > 0$ , we have
$$
\sum_ {t = 1} ^ {T} \min  \{R, \mathbb {E} _ {i \sim p _ {t}} \Big (\sum_ {j = 1} ^ {J} | w _ {t, j} ^ {\top} x _ {t, i} | \Big) \} \leq \left[ 2 d \Big (R ^ {2} T + \sum_ {t = 1} ^ {T} \gamma_ {t} \Big) \cdot \log \Big (1 + \frac {T R _ {x} ^ {2} R _ {w} ^ {2}}{R ^ {2}} \Big) \right] ^ {1 / 2}.
$$