SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Simplifying term I from Corollary 19.5 since event  $\pmb{\mathcal{E}}_{\psi}$  holds by assumption of the Lemma, we get that with probability at-least  $1 - 2\delta$
$$
\begin{array}{l} \left\| \theta_ {t} ^ {*} - \theta_ {t} \right\| _ {2} ^ {2} \leq (1 - \eta m) ^ {t - 1} \left\| \theta_ {1} - \theta_ {1} ^ {*} \right\| _ {2} ^ {2} + \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {t - s} \left\| \theta_ {s} ^ {*} - \theta_ {s + 1} ^ {*} \right\| _ {2} ^ {2} \\ + \frac {2 \eta B ^ {2}}{m} \underbrace {- 2 \eta \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {t - s - 1} \langle \theta_ {s} - \theta_ {s} ^ {*}, \psi_ {s + 1} \rangle} _ {\text {T e r m I I}} - 2 \eta \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {t - s - 1} \langle \theta_ {s} ^ {*} - \theta_ {s + 1} ^ {*}, \psi_ {s + 1} \rangle \\ \end{array}
$$
$$
+ 4 \eta^ {2} \lambda^ {2} \sum_ {s = 2} ^ {t - 1} (1 - \eta m) ^ {t - s} \mathbf {1} _ {C _ {s} \neq 0} + 4 \eta \lambda \mathcal {D} \sum_ {s = 2} ^ {t - 1} (1 - \eta m) ^ {t - s} \mathbf {1} _ {C _ {s} \neq 0}. \tag {26}
$$
where  $B = C\sqrt{\nu_{max}(\Sigma)\ln\left(\frac{2T}{\delta}\right)} + \sqrt{\mathrm{Tr}(\Sigma)}$ . Further, simplifying using Cauchy Schwartz inequality, we get the stated result of Lemma 19.6.
$$
\begin{array}{l} \left\| \theta_ {t} ^ {*} - \theta_ {t} \right\| _ {2} ^ {2} \leq (1 - \eta m) ^ {t - 1} \left\| \theta_ {1} - \theta_ {1} ^ {*} \right\| _ {2} ^ {2} + \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {t - s} \left\| \theta_ {s} ^ {*} - \theta_ {s + 1} ^ {*} \right\| _ {2} ^ {2} \\ + \frac {2 \eta B ^ {2}}{m} \underbrace {- 2 \eta \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {t - s - 1} \langle \theta_ {s} - \theta_ {s} ^ {*} , \psi_ {s + 1} \rangle} _ {\text {T e r m I I}} + 2 \eta B \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {t - s - 1} \Phi_ {s} \\ + 4 \eta^ {2} \lambda^ {2} \sum_ {s = 2} ^ {t - 1} (1 - \eta m) ^ {t - s} \mathbf {1} _ {C _ {s} \neq 0} + 4 \eta \lambda \mathcal {D} \sum_ {s = 2} ^ {t - 1} (1 - \eta m) ^ {t - s} \mathbf {1} _ {C _ {s} \neq 0}. \tag {27} \\ \end{array}
$$
19.2 Martingale Analysis of Term II
Denote by the constant
$$
\begin{array}{l} U _ {t} \leq \left[ (1 - \eta m) ^ {t - 1} \| \theta_ {1} - \theta_ {1} ^ {*} \| _ {2} ^ {2} + \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {t - s} \| \theta_ {s} ^ {*} - \theta_ {s + 1} ^ {*} \| _ {2} ^ {2} + 2 \eta B \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {t - s - 1} \Phi_ {s} \right. \\ + \frac {2 \eta B ^ {2}}{m} + 4 \eta^ {2} \lambda^ {2} \sum_ {s = 2} ^ {t - 1} (1 - \eta m) ^ {t - s} \mathbf {1} _ {C _ {s} \neq 0} + 4 \eta \lambda \mathcal {D} \sum_ {s = 2} ^ {t - 1} (1 - \eta m) ^ {t - s} \mathbf {1} _ {C _ {s} \neq 0} ]. \tag {28} \\ \end{array}
$$
The following proposition holds by simple algebraic manipulation.
Proposition 19.7. For every  $t \in [2, T]$  and  $s \leq t$ ,
$$
(1 - \eta m) ^ {t - s} U _ {s} \leq U _ {t}.
$$
Denote by the sequence  $(\xi_s)_{s=1}^T$  of random vectors as
$$
\xi_ {s} = \left\{ \begin{array}{l l} \theta_ {s} ^ {*} - \theta_ {s} & \text {i f} \| \theta_ {s} ^ {*} - \theta_ {s} \| ^ {2} \leq 2 U _ {s} \\ 0 & \text {o t h e r w i s e} \end{array} \right. \tag {29}
$$
$$
\boldsymbol {\mathcal {E}} _ {\xi} = \bigcap_ {t = 1} ^ {T} \left\{\sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {t - s} \left\langle \xi_ {s}, \psi_ {s + 1} \right\rangle \leq \sqrt {\frac {2 0 U _ {t} \nu_ {\max} (\Sigma) \ln \left(\frac {2 T}{\delta}\right)}{\eta m}} \right\} \tag {30}
$$
Lemma 19.8.
$$
\mathbb {P} \left[ \boldsymbol {\mathcal {E}} _ {\xi} \right] \geq 1 - \delta . \tag {31}
$$
Recall the definition of event  $\mathcal{E}_{\psi}$  in Equation (17).
Corollary 19.5 give that  $\mathbb{P}[\pmb{\mathcal{E}}_{\psi}] \geq 1 - 2\delta$ .
Lemma 19.9. Under the events  $\mathcal{E}_{\xi}$  and  $\mathcal{E}_{\psi}$ ,
$$
\left\| \theta_ {t} ^ {*} - \theta_ {t} \right\| _ {2} ^ {2} \leq 2 U _ {t}, \tag {32}
$$
holds for all  $t\in [T]$
19.3 Proof of Theorem 18.1
Proof. Corollary 19.5 gives that  $\mathbb{P}[\pmb{\mathcal{E}}_{\psi}^{C}] \leq 2\delta$ . Lemma 19.8 gives that  $\mathbb{P}[\pmb{\mathcal{E}}_{\xi}^{c}] \leq \delta$ . An union bound argument thus gives that  $\mathbb{P}[\pmb{\mathcal{E}}_{\psi}^{C} \cup \pmb{\mathcal{E}}_{\xi}^{C}] \leq 2\delta$ . Thus, we have that both events  $\pmb{\mathcal{E}}_{\psi}$  and  $\pmb{\mathcal{E}}_{\xi}$  hold with probability at-least  $1 - 3\delta$ , i.e.,
$$
\mathbb {P} \left[ \boldsymbol {\mathcal {E}} _ {\psi} \cap \boldsymbol {\mathcal {E}} _ {\xi} \right] \geq 1 - 3 \delta .
$$
Under the event  $\mathcal{E}_{\psi} \cap \mathcal{E}_{\xi}$ , Lemma 19.9 gives that
$$
\left\| \theta_ {t} - \theta_ {t} ^ {*} \right\| \leq \sqrt {2 U _ {t}}.
$$
Now, applying the fact that  $\sqrt{\sum_{i}x_{i}}\leq \sum_{i}\sqrt{x_{i}}$  , we can simplify  $\sqrt{U_t}$  as
$$
\begin{array}{l} \sqrt {U _ {t}} \leq \sqrt {2} \left[ (1 - \eta m) ^ {\frac {t - 1}{2}} \| \theta_ {1} - \theta_ {1} ^ {*} \| _ {2} + \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {\frac {s}{2}} \| \theta_ {t - s} ^ {*} - \theta_ {t - s + 1} ^ {*} \| \right. \\ + \left(C \sqrt {\nu_ {m a x} (\Sigma) \ln \left(\frac {2 T}{\delta}\right)} + \sqrt {\operatorname {T r} (\Sigma)}\right) \sqrt {\frac {2 \eta}{m}} \\ + \sqrt {2 \eta \left(C \sqrt {\nu_ {m a x} (\Sigma) \ln \left(\frac {2 T}{\delta}\right)} + \sqrt {\operatorname {T r} (\Sigma)}\right)} \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {\frac {t - s - 1}{2}} \sqrt {\| \theta_ {s} ^ {*} - \theta_ {s + 1} ^ {*} \|} \\ + 2 \eta \lambda \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {\frac {s - 1}{2}} \mathbf {1} _ {C _ {t - s + 1} \neq 0} + \sqrt {4 \eta \lambda \mathcal {D}} \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {\frac {s - 1}{2}} \mathbf {1} _ {C _ {t - s + 1} \neq 0} ]. \tag {33} \\ \end{array}
$$
Observe the following deterministic identities that can be proven by switching the order of summations.
$$
\sum_ {s \geq 1} (1 - \eta m) ^ {\frac {s - 1}{2}} \leq \frac {2}{\eta m}, \tag {34}
$$
$$
\sum_ {t = 1} ^ {T} \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {\frac {s - 1}{2}} \mathbf {1} _ {C _ {t - s + 1} \neq 0} \leq \frac {2 \Lambda_ {T}}{\eta m}, \tag {35}
$$
$$
\sum_ {t = 1} ^ {T} \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {\frac {s}{2}} \left\| \theta_ {t - s} ^ {*} - \theta_ {t - s + 1} ^ {*} \right\| \leq \frac {2 \Phi_ {T}}{\eta m}, \tag {36}
$$
$$
\sum_ {t = 1} ^ {T} \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {\frac {t - s - 1}{2}} \sqrt {\left\| \theta_ {s} ^ {*} - \theta_ {s + 1} ^ {*} \right\|} \leq \frac {2 \sum_ {s = 1} ^ {t} \sqrt {\left\| \theta_ {s} ^ {*} - \theta_ {s + 1} ^ {*} \right\|}}{\eta m} \leq \frac {2 \sqrt {T \Phi_ {T}}}{\eta m}. \tag {37}
$$
The first inequality above follows from the fact that for all  $x \in (0,1)$ , we have  $\frac{1}{1 - \sqrt{1 - x}} \leq \frac{2}{x}$  and the last inequality follows from Cauchy-Schwartz inequality that  $\sum_{i=1}^{T} \sqrt{x_i} \leq \sqrt{T \sum_{i=1}^{T} x_i}$ . Recall that the regret  $\mathrm{Reg}_T := \sum_{t=1}^{T} \| \theta_t - \theta_t^* \|$ . Thus we get from Equations (32) and (33) that with probability at-least  $1 - 3\delta$ ,
$$
\begin{array}{l} \operatorname {R e g} _ {T} \leq \frac {2 \left(l _ {1} + \Phi_ {T}\right)}{\eta m} + T \left(C \sqrt {\nu_ {\max } (\Sigma) \ln \left(\frac {2 T}{\delta}\right)} + \sqrt {\operatorname {T r} (\Sigma)}\right) \sqrt {\frac {2 \eta}{m}} \\ + \Lambda_ {T} \left(\frac {2 \lambda}{m} + \sqrt {\frac {4 \lambda \mathcal {D}}{\eta}}\right) + 2 \sqrt {\frac {T \Phi_ {T} \left(C \sqrt {\nu_ {\operatorname* {m a x}} (\Sigma) \ln \left(\frac {2 T}{\delta}\right)} + \sqrt {\operatorname {T r} (\Sigma)}\right)}{\eta m ^ {2}}}, \tag {38} \\ \end{array}
$$
holds with probability at-least  $1 - 3\delta$  . By a change of variables by re-parametrizing  $\delta \rightarrow \frac{\delta}{3}$  yields the stated result.
19.4 Proofs of Lemma 19.9
Proof. We prove this lemma by induction on  $t$ . For the base case of  $t = 1$ , Equation (32) holds with probability 1 trivially by definition. Now for the induction hypothesis, assume Equation (32) holds for all  $s \in \{1, \dots, t - 1\}$  for some  $t > 1$ . Since we are on the event  $\mathcal{E}_{\xi}$ , we Equation (27) holds. Thus,
$$
\begin{array}{l} \| \theta_ {t} ^ {*} - \theta_ {t} \| _ {2} ^ {2} \leq (1 - \eta m) ^ {t - 1} \| \theta_ {1} - \theta_ {1} ^ {*} \| _ {2} ^ {2} + \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {t - s} \| \theta_ {s} ^ {*} - \theta_ {s + 1} ^ {*} \| _ {2} ^ {2} \\ + \frac {2 \eta B ^ {2}}{m} \underbrace {- 2 \eta \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {t - s - 1} \langle \theta_ {s} - \theta_ {s} ^ {*} , \psi_ {s + 1} \rangle} _ {\text {T e r m I I}} + 2 \eta B \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {t - s - 1} \Phi_ {s} \\ + 4 \eta^ {2} \lambda^ {2} \sum_ {s = 2} ^ {t - 1} (1 - \eta m) ^ {t - s} \mathbf {1} _ {C _ {s} \neq 0} + 4 \eta \lambda \mathcal {D} \sum_ {s = 2} ^ {t - 1} (1 - \eta m) ^ {t - s} \mathbf {1} _ {C _ {s} \neq 0}. \tag {39} \\ \end{array}
$$
From the induction hypothesis, we have that  $\| \theta_s^* -\theta_s\| _2^2\leq 2U_s$  holds for all  $s\in \{1,\dots ,t - 1\}$ . Thus,
$$
\sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {t - s} \langle \xi_ {s}, \psi_ {s + 1} \rangle = \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {t - s} \langle \theta_ {s} - \theta_ {s} ^ {*}, \psi_ {s + 1} \rangle .
$$
Further since we are on the event  $\pmb{\varepsilon}_{\xi}$ , we have that
$$
\sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {t - s} \langle \xi_ {s}, \psi_ {s + 1} \rangle \leq \sqrt {\frac {2 0 U _ {t} \nu_ {m a x} (\Sigma) \ln \left(\frac {2 T}{\delta}\right)}{\eta m}}.
$$
Thus, plugging this back into Equation (39), we get that
$$
\| \theta_ {t} ^ {*} - \theta_ {t} \| _ {2} ^ {2} \leq U _ {t} + \sqrt {\frac {8 0 \eta U _ {t} \nu_ {m a x} (\Sigma) \ln \left(\frac {2 T}{\delta}\right)}{m}}.
$$
From Equation (28), we see that  $\frac{2\eta B^2}{m} \leq U_t$ . Substituting this in the above equation, we see that
$$
\begin{array}{l} \| \theta_ {t} ^ {*} - \theta_ {t} \| _ {2} ^ {2} \leq U _ {t} + \frac {U _ {t} \sqrt {4 0 \nu_ {m a x} (\Sigma) \ln \left(\frac {2 T}{\delta}\right)}}{B}, \\ \leq 2 U _ {t} \\ \end{array}
$$
The second inequality follows since  $B = C\sqrt{\nu_{max}(\Sigma)\ln\left(\frac{2T}{\delta}\right)} + \sqrt{\mathrm{Tr}(\Sigma)}$ , where  $C \geq 40$  as given in Proposition 19.1. Thus,  $\frac{\sqrt{40\nu_{max}(\Sigma)\ln\left(\frac{2T}{\delta}\right)}}{B} \leq 1$ .
19.5 Proof of Lemma 19.8
We first reproduce an useful result.
Lemma 19.10 (Sub-gaussian Martingale Azuma-Hoeffding inequality [53]). Suppose  $Y_{1}, \dots, Y_{T}$  is a martingale sequence with respect to a filtration  $(\mathcal{F}_t)_{t=1}^T$ , i.e.,  $\mathbb{E}[Y_t | \mathcal{F}_{t-1}] = 0$  for all  $t$ . Further, suppose there exists deterministic non-negative numbers  $(\sigma_t)_{t=1}^T$  such that for every  $\lambda \in \mathbb{R}$  and  $t \in [T]$ , we have  $\mathbb{E}[\exp(\lambda Y_t) | \mathcal{F}_{t-1}] \leq \exp\left(\frac{\lambda^2 \sigma_t^2}{2}\right)$  almost-surely. Then for every  $a > 0$ ,
$$
\mathbb {P} \left[ \left| \sum_ {t = 1} ^ {T} Y _ {t} \right| \geq a \right] \leq 2 \exp \left(\frac {- 3 a ^ {2}}{2 8 \sum_ {t = 1} ^ {T} \sigma_ {t} ^ {2}}\right)
$$
In order to prove Lemma 19.8, we will show that for each  $t \in [T]$ , the event
$$
\boldsymbol {\mathcal {E}} _ {\xi} ^ {(t)} = \left\{\sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {t - s} \langle \xi_ {s}, \psi_ {s + 1} \rangle \leq 2 B U _ {t} \sqrt {t} \ln \left(\frac {T ^ {2}}{\delta}\right) \right\}, \tag {40}
$$
holds with probability at-least  $1 - \frac{\delta}{T}$ . Then an union bound will conclude the proof of Lemma 19.8.
To do so, first observe that the sum  $\sum_{s=1}^{t-1}(1 - \eta m)^{t-s}\langle \xi_s, \psi_{s+1} \rangle$  is a martingale difference with respect to the filtration  $(\mathcal{F}_t)_{t=1}^T$ . This follows from two facts.
Fact I  $\xi_s \in \mathcal{F}_s$  is measurable with respect to the filtration generated by all the random vectors  $Z_1, \dots, Z_s$  and the corruption vectors  $C_1, \dots, C_s$ . This follows since the adversary is causal and does not have access to future randomness to decide on corruption levels.
Fact II  $\mathbb{E}[\psi_{s + 1}|\mathcal{F}_s] = 0$  is un-biased.
Fact III, The sequence of random variables  $((1 - \eta m)^{t - s}\langle \xi_s,\psi_{s + 1}\rangle)_{s = 1}^t$  satisfies the premise of Lemma 19.10 with  $\sigma_s\leq (1 - \eta m)^{2(t - s)}\| \xi_s\| ^2\nu_{max}(\Sigma)\leq 2(1 - \eta m)^{t - s}U_t\nu_{max}(\Sigma)$ , for all  $1\leq s\leq t$ .
The third fact follows from the definition that conditional on  $\mathcal{F}_s$ , the covariance matrix of  $\psi_{s + 1}$  is upper-bounded by  $\Sigma$  and  $\| \xi_s\|^2\leq 2U_s$  for all  $s$ , and  $(1 - \eta m)^{t - s}U_{s}\leq U_{t}$  holds for all  $s\leq t$ .
In order to apply Lemma 19.10, observe that the sum of variances  $\sum_{s=1}^{t} \sigma_s^2 = 2U_t \nu_{max}(\Sigma) \sum_{s=1}^{t} (1 - \eta m)^{t-s} \leq \frac{2U_t \nu_{max}(\Sigma)}{\eta m}$ . Applying Lemma 19.10, we get that
$$
\mathbb {P} \left[ \sum_ {s = 1} ^ {t} (1 - \eta m) ^ {t - s} \langle \xi_ {s}, \psi_ {s + 1} \rangle \geq \sqrt {\frac {2 0 U _ {t} \nu_ {m a x} (\Sigma) \ln \left(\frac {2 T}{\delta}\right)}{\eta m}} \right] \leq \delta .
$$
19.6 Proof of Corollary 4.5
The finite diameter that  $\mathcal{D} < \infty$  fact is only used in Lemma 19.6. From observing the proof of Lemma 19.6, it holds even if  $\mathcal{D} = \infty$ , albeit the bound is vacuous if  $\Lambda_T > 0$ . Thus, the entire proof holds verbatim even if  $\mathcal{D} = \infty$  and gives non-trivial regret guarantees when  $\Lambda_T = 0$ .
20 Theorem statement and Proofs in the general case from Section 5
Theorem 20.1 (Formal version of Theorem 5.1). For every  $\delta \in (0,1)$ , if Algorithm 1 is run with clipping value  $\lambda \geq 2G$ , and step-size  $\eta > 0$ , then with probability at-least  $1 - \delta$ ,
$$
\begin{array}{l} R e g _ {T} \leq \left(\frac {9 1 \sigma}{\lambda \sqrt {m \eta}} + \frac {2 \lambda \sqrt {m}}{\sigma} + 2\right) \left[ \frac {2 (l _ {1} + \Phi_ {T})}{\eta m} + 4 \sigma T \sqrt {\frac {\eta}{\lambda m}} + 7 \sigma T \frac {\eta}{m} + 6 \eta \lambda T \sqrt {\ln \left(\frac {2 T}{\delta}\right)} \right] \\ + \frac {6 T \eta^ {3 / 4} \sqrt {\lambda \sigma \sqrt {\ln \left(\frac {2 T}{\delta}\right)}}}{2 m ^ {1 / 4}} + 3 \sigma \sqrt {\frac {T \Phi_ {T}}{\eta \lambda m ^ {2}}} + 2 \sqrt {\frac {\lambda T \Phi_ {T}}{\eta}} + \frac {4 \lambda \Lambda_ {T}}{\eta m} + 4 \Lambda_ {T} \sqrt {\frac {\lambda \mathcal {D}}{\eta m ^ {2}}} ]. \tag {41} \\ \end{array}
$$
Corollary 20.2. For every  $\delta \in (0,1)$ , if Algorithm 1 is run with clipping value  $\lambda = (2GT^{\beta})$ , and step-size  $\eta = \frac{1}{mT^{\alpha}}$  for  $\alpha \in [0,1)$ , then with probability at-least  $1 - \delta$ ,
$$
R e g _ {T} \leq \underbrace {\mathcal {O} \left(\left(G T ^ {\alpha + \beta} + \frac {T ^ {\frac {3 \alpha}{2} - \beta}}{G}\right) \frac {\sigma \Phi_ {T}}{m ^ {3 / 2}} + \frac {\sigma}{m ^ {3 / 2}} \sqrt {\frac {\Phi_ {T} T ^ {1 + 2 \alpha - \beta}}{G}}\right)} +
$$
Regret due to distribution shift
$$
\underbrace {\mathcal {O} \left(\left(\sqrt {G} T ^ {1 - \left(\frac {\alpha - \beta}{2}\right)} + T ^ {1 - \frac {\alpha}{2}} + G ^ {2} T ^ {1 + 2 \beta - \alpha}\right) \frac {\sigma^ {2}}{m} \ln \left(\frac {T}{\delta}\right)\right)} _ {\text {}} +
$$
Regret due to finite-sample estimation error
$$
\underbrace {\mathcal {O} \left(\Lambda_ {T} \frac {\sigma \sqrt {\mathcal {D}}}{m} \left(G ^ {2} T ^ {2 \beta + \alpha} + T ^ {\frac {3 \alpha}{2}} + \frac {T ^ {\alpha - \frac {\beta}{2}}}{\sqrt {G}} + G ^ {3 / 2} T ^ {\frac {3 \beta}{2} + \alpha}\right)\right)} _ {\text {O}}. \tag {42}
$$
Regret due to adversarial corruptions
Corollary 20.3. For every  $\delta \in (0,1)$ , if Algorithm 1 is run with clipping value  $\lambda = 2GT^{\frac{\alpha}{3}}$ , and step-size  $\eta = \frac{1}{mT^{\alpha}}$  for  $\alpha \in [0,1]$ , then with probability at-least  $1 - \delta$ ,
$$
R e g _ {T} \leq \underbrace {\mathcal {O} \left(\frac {G \sigma T ^ {\frac {4 \alpha}{3}} \Phi_ {T}}{m ^ {3 / 2}} + \frac {\sigma T ^ {\frac {1}{2} + \frac {5 \alpha}{6}} \sqrt {\Phi_ {T}}}{m ^ {3 / 2} \sqrt {G}}\right)} _ {\text {R e g r e t d u e t o d i s t r i b u t i o n s h i f t}} + \underbrace {\mathcal {O} \left(\frac {T ^ {1 - \frac {\alpha}{3}} (G \sigma) ^ {2} \ln \left(\frac {T}{\delta}\right)}{m}\right)} _ {\text {R e g r e t d u e t o f i n i t e - s a m p l e e s t i m a t i o n e r r o r}} + \underbrace {\mathcal {O} \left(\frac {T ^ {\frac {3 \alpha}{2}} \Lambda_ {T} G ^ {2} \sigma \sqrt {\mathcal {D}}}{m}\right)} _ {\text {R e g r e t d u e t o a d v e r s a r i a l c o r r u p t i o n s}}. \tag {43}
$$
Corollary 20.4. For every  $\delta \in (0,1)$ , if Algorithm 1 is run with clipping value  $\lambda \geq \max(2G, T^{1/5})$ , and step-size  $\eta = \frac{T^{-1/2}}{m}$ , then with probability at-least  $1 - \delta$ ,
$$
\begin{array}{l} R e g _ {T} \leq \underbrace {\mathcal {O} \left(T ^ {5 / 6} \left(\Phi_ {T} + \sqrt {\Phi_ {T}}\right)\right)} _ {\text {R e g r e t d u e t o d i s t r i b u t i o n s h i f t}} + \underbrace {\mathcal {O} \left(T ^ {5 / 6} \mathcal {D} \lambda^ {\frac {1}{2}} \left(\frac {T r (\Sigma)}{m}\right) ^ {\frac {1}{4}} \ln \left(\frac {T}{\delta}\right) ^ {\frac {1}{2}}\right)} _ {\text {R e g r e t d u e t o f i n i t e - s a m p l e e s t i m a t i o n e r r o r}} \\ + \underbrace {\mathcal {O} \left(\Lambda_ {T} \frac {\sqrt {\mathcal {D}}}{m} \left(T ^ {2 \beta + \alpha} + T ^ {\frac {3 \alpha}{2}} + T ^ {\alpha - \frac {\beta}{2}} + T ^ {\frac {3 \beta}{2} + \alpha}\right)\right)} _ {\text {R e g r e t d u e t o a d v e r s a r i a l c o r r u p t i o n s}}. \tag {44} \\ \end{array}
$$
20.1 Proof of Theorem 20.1
We follow a similar proof architecture as that of Theorem 18.1. We define two sequences of random variables  $(\psi_t)_{t \geq 1}$  and  $(\psi)_{t \geq 1}$  as follows.
and by
$$
\bar {\psi} _ {t} := \operatorname {c l i p} \left(\nabla \mathcal {L} \left(X _ {t}, \theta_ {t - 1}\right), \lambda\right) - \nabla \mathcal {R} _ {t} \left(\theta_ {t - 1}\right).
$$
These are random vectors since  $Z_{t}\sim \mathbb{P}_{t}$  and both  $C_t$  and  $\theta_t$  are measurable with respect to the sigma algebra generated by  $\sigma (Z_1,\dots ,Z_t,C_1,\dots ,C_{t - 1})$ . Clearly, for all times  $t\geq 1$ , on the event that  $C_t = 0$ ,  $\psi_t = \bar{\psi}_t$  holds almost-surely. Furthermore, from triangle inequality almost-surely for all time  $t\geq 1$ , we have
$$
\left\| \psi_ {t} \right\| _ {2} ^ {2} \leq \left\| \bar {\psi} _ {t} \right\| _ {2} ^ {2} + 2 \lambda^ {2} \mathbf {1} _ {C _ {t} \neq 0}. \tag {45}
$$
Expanding the one-step recursion
Similar to the analysis carried out in the proof of Theorem 18.1, we consider any time  $t$  and write the recursion as
$$
\left\| \theta_ {t} - \theta_ {t} ^ {*} \right\| _ {2} ^ {2} = \left\| \mathcal {P} _ {\Theta} \left(\theta_ {t - 1} - \eta \operatorname {c l i p} \left(\mathcal {L} \left(X _ {t}, \theta_ {t - 1}\right), \lambda\right)\right) - \theta_ {t} ^ {*} \right\| _ {2} ^ {2}.
$$
Now, this can be expanded verbatim as done in Section 19.1 to yield an identical replica of Equation 24.
$$
\left\| \theta_ {t} ^ {*} - \theta_ {t} \right\| _ {2} ^ {2} \leq (1 - \eta m) ^ {t - 1} \left\| \theta_ {1} - \theta_ {1} ^ {*} \right\| _ {2} ^ {2} + \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {s} \left\| \theta_ {t - s} ^ {*} - \theta_ {t - s + 1} ^ {*} \right\| _ {2} ^ {2}
$$
$$
\begin{array}{l} + 2 \eta^ {2} \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {s - 1} \| \psi_ {t - s + 1} \| _ {2} ^ {2} - 2 \eta \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {s - 1} \langle \theta_ {t - s} - \theta_ {t - s + 1} ^ {*}, \psi_ {t - s + 1} \rangle \\ + 4 \eta^ {2} \lambda^ {2} \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {s - 1} \mathbf {1} _ {C _ {t - s + 1} \neq 0} + 4 \eta \lambda \mathcal {D} \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {s - 1} \mathbf {1} _ {C _ {t - s + 1} \neq 0}. \tag {46} \\ \end{array}
$$
Denote by  $\psi_t \coloneqq \psi_t^{(b)} + \psi_t^{(v)}$ , where  $\psi_t^{(b)} \coloneqq \mathbb{E}_{Z_t}[\psi_t|\mathcal{F}_{t-1}]$  and  $\psi_t^{(v)} \coloneqq \psi_t - \psi_t^{(b)}$ . Using this in the display above and using that fact that  $\|a + b\|_2^2 \leq 2\|a\|_2^2 + 2\|b\|_2^2$ , we get
$$
\begin{array}{l} \left\| \theta_ {t} ^ {*} - \theta_ {t} \right\| _ {2} ^ {2} \leq (1 - \eta m) ^ {t - 1} \left\| \theta_ {1} - \theta_ {1} ^ {*} \right\| _ {2} ^ {2} + \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {s} \left\| \theta_ {t - s} ^ {*} - \theta_ {t - s + 1} ^ {*} \right\| _ {2} ^ {2} \\ + 4 \eta^ {2} \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {s - 1} \| \psi_ {t - s + 1} ^ {(b)} \| _ {2} ^ {2} + 4 \eta^ {2} \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {s - 1} \| \psi_ {t - s + 1} ^ {(v)} \| _ {2} ^ {2} \\ - 2 \eta \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {s - 1} \left\langle \theta_ {t - s} - \theta_ {t - s + 1} ^ {*}, \psi_ {t - s + 1} ^ {(b)} \right\rangle \\ - 2 \eta \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {s - 1} \left\langle \theta_ {t - s} - \theta_ {t - s + 1} ^ {*}, \psi_ {t - s + 1} ^ {(v)} \right\rangle \\ + 4 \eta^ {2} \lambda^ {2} \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {s - 1} \mathbf {1} _ {C _ {t - s + 1} \neq 0} + 4 \eta \lambda \mathcal {D} \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {s - 1} \mathbf {1} _ {C _ {t - s + 1} \neq 0}. \tag {47} \\ \end{array}
$$
Further simplifying by adding and subtracting  $\mathbb{E}_{Z_t}[\| \psi_t^{(v)}\|_2^2 |\mathcal{F}_{t - 1}]$  to be above display, we get
$$
\begin{array}{l} \left\| \theta_ {t} ^ {*} - \theta_ {t} \right\| _ {2} ^ {2} \leq (1 - \eta m) ^ {t - 1} \left\| \theta_ {1} - \theta_ {1} ^ {*} \right\| _ {2} ^ {2} + \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {s} \left\| \theta_ {t - s} ^ {*} - \theta_ {t - s + 1} ^ {*} \right\| _ {2} ^ {2} \\ + 4 \eta^ {2} \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {s - 1} \| \psi_ {t - s + 1} ^ {(b)} \| _ {2} ^ {2} + 4 \eta^ {2} \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {s - 1} \mathbb {E} _ {Z _ {t - s + 1}} [ \| \psi_ {t - s + 1} ^ {(v)} \| _ {2} ^ {2} | \mathcal {F} _ {t - s} ] \\ + 4 \eta^ {2} \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {s - 1} \left(\| \psi_ {t - s + 1} ^ {(v)} \| _ {2} ^ {2} - \mathbb {E} _ {Z _ {t - s + 1}} \left[ \| \psi_ {t - s + 1} ^ {(v)} \| _ {2} ^ {2} \mid \mathcal {F} _ {t - s} \right]\right) \\ - 2 \eta \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {s - 1} \langle \theta_ {t - s} - \theta_ {t - s} ^ {*}, \psi_ {t - s + 1} ^ {(b)} \rangle - 2 \eta \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {s - 1} \langle \theta_ {t - s} ^ {*} - \theta_ {t - s + 1} ^ {*}, \psi_ {t - s + 1} ^ {(b)} \rangle \\ - 2 \eta \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {s - 1} \left\langle \theta_ {t - s} - \theta_ {t - s} ^ {*}, \psi_ {t - s + 1} ^ {(v)} \right\rangle - 2 \eta \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {s - 1} \left\langle \theta_ {t - s} ^ {*} - \theta_ {t - s + 1} ^ {*}, \psi_ {t - s + 1} ^ {(v)} \right\rangle \\ + 4 \eta^ {2} \lambda^ {2} \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {s - 1} \mathbf {1} _ {C _ {t - s + 1} \neq 0} + 4 \eta \lambda \mathcal {D} \sum_ {s = 1} ^ {t - 1} (1 - \eta m) ^ {s - 1} \mathbf {1} _ {C _ {t - s + 1} \neq 0}. \tag {48} \\ \end{array}
$$
Lemma 20.5. If  $\lambda \geq 2\sup_{\theta \in \Theta}\| \nabla \mathcal{R}_t(\theta)\| = 2G$ , the following inequalities hold.
$$
\left\| \psi_ {t} ^ {(v)} \right\| \leq 2 \lambda \tag {49}
$$
$$
\left\| \psi_ {t} ^ {(b)} \right\| _ {2} \leq \frac {4 \sigma^ {2}}{\lambda} \tag {50}
$$
$$
\mathbb {E} _ {Z _ {t}} \left[ \| \psi_ {t} ^ {(v)} \| _ {2} ^ {2} \mid \mathcal {F} _ {t - 1} \right] \leq 1 0 \sigma^ {2} \tag {51}
$$
Simplifying Equation (48) using bounds in Lemma 20.5, we get