SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \mathbb {E} \left[ \| \psi_ {t} ^ {(v)} \| _ {2} ^ {2} | \mathcal {F} _ {t - 1} \right] \stackrel {(a)} {=} \mathbb {E} _ {Z _ {t}} \left[ \| \operatorname {c l i p} \left(\mathcal {L} \left(Z _ {t}, \theta_ {t - 1}\right), \lambda\right) - \mathbb {E} _ {Z _ {t}} \left[ \operatorname {c l i p} \left(\mathcal {L} \left(Z _ {t}, \theta_ {t - 1}\right), \lambda\right) | \mathcal {F} _ {t - 1} \right] \| _ {2} ^ {2} | \mathcal {F} _ {t - 1} \right], \\ \stackrel {(b)} {\leq} \mathbb {E} [ \| \operatorname {c l i p} (\mathcal {L} (Z _ {t}, \theta_ {t - 1}), \lambda) - \nabla \mathcal {R} _ {t} (\theta_ {t - 1}) \| _ {2} ^ {2} | \mathcal {F} _ {t - 1} ]. \tag {64} \\ \end{array}
$$
Inequality  $(a)$  follows from the fact that  $\theta_{t - 1}\in \mathcal{F}_{t - 1}$  and step  $(b)$  follows from the fact that for any random vector  $Z$  and filtration  $\mathcal{F}$  and  $Y\in \mathcal{F}$ ,  $\mathbb{E}_Z[\| Z - \mathbb{E}[Z|\mathcal{F}]\| _2^2 |\mathcal{F}]\leq \mathbb{E}[\| Z - Y\| _2^2 |\mathcal{F}]$  holds almost-surely. To simplify notation, we denote by  $\mathcal{E}_t\coloneqq \mathcal{E}[\cdot |\mathcal{F}_{t - 1}]$ . We now bound Equation (64) to conclude.
$$
\begin{array}{l} \mathbb {E} _ {t} \left[ \| \operatorname {c l i p} \left(\mathcal {L} \left(Z _ {t}, \theta_ {t - 1}\right), \lambda\right) - \nabla \mathcal {R} _ {t} \left(\theta_ {t - 1}\right) \| _ {2} ^ {2} \right] = \mathbb {E} _ {t} \left[ \| \nabla \mathcal {L} \left(Z _ {t}, \theta_ {t - 1}\right) - \nabla \mathcal {R} _ {t} \left(\theta_ {t - 1}\right) \| _ {2} ^ {2} \left(1 - \xi_ {t}\right) ^ {2} \right] + \\ + \mathbb {E} _ {t} \left[ \left\| \frac {\lambda \nabla \mathcal {L} \left(Z _ {t} , \theta_ {t - 1}\right)}{\| \nabla \mathcal {L} \left(Z _ {t} , \theta_ {t - 1}\right) \|} - \nabla \mathcal {R} _ {t} \left(\theta_ {t - 1}\right) \right\| _ {2} ^ {2} \xi_ {t} \right], \\ \stackrel {(a)} {\leq} \sigma^ {2} + \mathbb {E} _ {t} \left[ \left(2 \left\| \frac {\lambda \nabla \mathcal {L} (Z _ {t} , \theta_ {t - 1})}{\| \nabla \mathcal {L} (Z _ {t} , \theta_ {t - 1}) \|} \right\| _ {2} ^ {2} + 2 \| \nabla \mathcal {R} _ {t} (\theta_ {t - 1}) \| _ {2} ^ {2}\right) \xi_ {t} ^ {2} \right], \\ \stackrel {(b)} {\leq} \sigma^ {2} + \frac {5}{2} \lambda^ {2} \mathbb {E} _ {t} [ \xi_ {t} ^ {2} ], \\ \stackrel {(c)} {\leq} \sigma^ {2} + \frac {5}{2} \lambda^ {2} \mathbb {E} _ {t} [ \xi_ {t} ^ {2} ], \\ \stackrel {6 3} {\leq} \sigma^ {2} + \frac {5 \lambda^ {2}}{2}. \frac {4 \sigma^ {2}}{\lambda^ {2}}. \\ \end{array}
$$
Inequality  $(a)$  follows from the fact that for any two vectors  $a, b$ ,  $\|a + b\|_2^2 \leq 2\|a\|_2^2 + 2\|b\|_2^2$ , and the assumption that  $\sup_{\theta \in \Theta} \mathbb{E}_t[\|\mathcal{L}(Z, \theta) - \mathcal{R}_t(\theta)\|_2^2] \leq \sigma^2$ .
20.7 Proof of Corollary 5.4
The finite diameter that  $\mathcal{D} < \infty$  fact is only used in Equation (46) which holds even if  $\mathcal{D} = \infty$ , albeit the bound is vacuous if  $\Lambda_T > 0$ . Thus, the entire proof holds verbatim even if  $\mathcal{D} = \infty$  and gives non-trivial regret guarantees when  $\Lambda_T = 0$ .