SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In (c) we used the fact that  $\kappa \geq \log (T)$ , so  $\delta = R\exp (-\kappa)\geq R$
A.8 Proof of Theorem 4.2
Proof. We bound the regret in every world as followed
$$
\begin{array}{l} \mathcal {R} _ {T} = \operatorname {O P T} \left(\mathcal {P} _ {1: T}\right) - \sum_ {t \in [ T ]} \mathbb {E} \left[ f _ {t} \left(\boldsymbol {x} _ {t}\right) \right] \\ \stackrel {(a)} {\leq} \mathbb {E} \left[ \bar {F} (T - \tau_ {A}) + \sum_ {t \in [ \tau_ {A} ]} \boldsymbol {\lambda} _ {t} ^ {\top} \boldsymbol {g} _ {t} (\boldsymbol {x} _ {t}) + \mathcal {R} _ {\mathrm {B O C O}} (\tau_ {A}) \right] \\ \stackrel {(b)} {\leq} \mathbb {E} \left[ \mathcal {R} _ {\mathrm {B O C O}} (\tau_ {A}) \right] \\ \end{array}
$$
where (a) follows from Lemma 4.3, and (b) follows from Lemma 4.5. Recall  $\mathcal{R}_{\mathrm{BOCO}}(\tau_A)$  is specified in Lemma 4.3 for each world.
In the following we bound  $\mathcal{R}_{\mathrm{BOCO}}(\tau_A)$  for each world.
$$
\mathbb {E} \left[ \mathcal {R} _ {\mathrm {B O C O}} \left(\tau_ {A}\right) \right] = \mathbb {E} \left[ \max  _ {\boldsymbol {x} \in \mathcal {X}} \sum_ {t \in [ \tau_ {A} ]} h _ {t} (\boldsymbol {x}) - h _ {t} \left(\boldsymbol {x} _ {t}\right) \right] \stackrel {(a)} {\leq} \mathcal {O} \left(\frac {(\rho + \alpha) T}{\beta} + \frac {1}{\gamma_ {i}} + \frac {\gamma_ {i} K T}{\beta^ {2} \rho^ {2}} + T \epsilon + \frac {1}{\epsilon}\right) \stackrel {(b)} {=} \mathcal {O} \left(T ^ {\frac {3}{4}}\right), \tag {37}
$$
where (a) follows from Lemma 4.6 by taking the comparator sequence  $\pmb{y}_t = \arg \max_{\pmb{x} \in \mathcal{X}} \sum_{t \in [\tau_A]} h_t(\pmb{x})$  for all  $t \in [\tau_A]$  such that  $P(\pmb{y}_{1:T}) = 1$ , as well as any primal ascent expert  $i \in [N]$ ; (b) follows from taking  $\eta = \frac{1}{\sqrt{KT}}$ ,  $\rho = K^{\frac{1}{3}}T^{-\frac{1}{4}}$ ,  $\epsilon = T^{-\frac{1}{2}}$ ,  $\beta = \frac{1}{\log(T)}$ , and finally choosing  $\gamma_i = K^{-\frac{1}{6}}(1 + DT)^{\frac{1}{2}}T^{-\frac{3}{4}}$ . Recall all primal ascent expert step sizes are  $\{\gamma_1 \dots \gamma_N\} = \{2^{-i}K^{-\frac{1}{6}}(1 + DT)^{\frac{1}{2}}T^{-\frac{3}{4}} : i = 0 \dots N - 1\}$ .
$\delta$ -corrupted, Periodic, and Ergodic. The proof is nearly identical with that of the stochastic world in Eq. (37) given that we still consider the comparator sequence  $\pmb{y}_t = \arg \max_{\pmb{x} \in \mathcal{X}} \sum_{t \in [\tau_A]} h_t(\pmb{x})$  for all  $t \in [\tau_A]$  such that  $P(\pmb{y}_{1:T}) = 1$ . Hence we will omit the proof.
Adversarial. Recall the definition  $\widetilde{\pmb{y}}_t = \arg \max_{\pmb{x} \in \mathcal{X}} f_t(\pmb{x}_t) + \lambda_t^\top \pmb{g}(\pmb{x}_t)$ . Then we have
$$
\begin{array}{l} \mathbb {E} \left[ \mathcal {R} _ {\text {B O C O}} (\tau_ {A}) \right] = \left(1 - \frac {1}{\xi}\right) \operatorname {O P T} \left(\mathcal {P} _ {1: T}\right) + \sum_ {t \in \left[ \tau_ {A} \right]} \mathbb {E} \left[ h _ {t} \left(\widetilde {\boldsymbol {y}} _ {t}\right) - h _ {t} \left(\boldsymbol {x} _ {t}\right) \right] \\ \leq \mathcal {O} \left(\frac {(\rho + \alpha) T}{\beta} + \frac {1 + P \left(\widetilde {\mathbf {y}} _ {1 : T}\right)}{\gamma_ {i}} + \frac {\gamma_ {i} K T}{\beta^ {2} \rho^ {2}} + T \epsilon + \frac {1}{\epsilon}\right) \tag {38} \\ \leq \left(1 - \frac {1}{\xi}\right) \operatorname {O P T} \left(\mathcal {P} _ {1: T}\right) + \tilde {\mathcal {O}} \left(\sqrt {1 + P \left(\tilde {\boldsymbol {y}} _ {1 : T}\right)} \cdot T ^ {\frac {3}{4}}\right), \\ \end{array}
$$
where we chose the primal ascent stepsize  $\gamma_{i}$  s.t.
$$
\frac {1}{2} K ^ {- \frac {1}{6}} (1 + P (\widetilde {\mathbf {y}} _ {1: T})) ^ {\frac {1}{2}} T ^ {- \frac {3}{4}} \leq \gamma_ {i} \leq K ^ {- \frac {1}{6}} (1 + P (\widetilde {\mathbf {y}} _ {1: T})) ^ {\frac {1}{2}} T ^ {- \frac {3}{4}} \tag {39}
$$
We note that such a  $\gamma_{i}$  must exist because  $P(\widetilde{\mathbf{y}}_{1:T})\leq DT$  given all  $\widetilde{\mathbf{y}}_t\in \mathcal{X}$ , so that the largest element in the primal ascent stepsize set, namely  $K^{-\frac{1}{6}}(1 + DT)^{\frac{1}{2}}T^{-\frac{3}{4}}$  is larger than the upper bound above, namely  $K^{-\frac{1}{6}}(1 + P(\widetilde{\mathbf{y}}_{1:T}))^{\frac{1}{2}}T^{-\frac{3}{4}}$ .
A.9 Proof for Lemma A.1
Proof. Recall the definition  $h_t(\pmb{x}) = f_t(\pmb{x}) + \pmb{\lambda}_t^\top \pmb{g}_t(\pmb{x})$  in Eq. (5). Then we have
$$
\begin{array}{l} \left| h _ {t} (\boldsymbol {x}) - h _ {t} \left(\boldsymbol {x} ^ {\prime}\right) \right| \leq \left| f _ {t} (\boldsymbol {x}) - f _ {t} \left(\boldsymbol {x} ^ {\prime}\right) \right| + \| \boldsymbol {\lambda} _ {t} \| \cdot \| \boldsymbol {g} _ {t} (\boldsymbol {x}) - \boldsymbol {g} _ {t} \left(\boldsymbol {x} ^ {\prime}\right) \| \\ \stackrel {(a)} {\leq} L \| \boldsymbol {x} - \boldsymbol {x} ^ {\prime} \| + K \frac {\bar {F}}{\beta} L \| \boldsymbol {x} - \boldsymbol {x} ^ {\prime} \| = (1 + K \frac {\bar {F}}{\beta}) L \cdot \| \boldsymbol {x} - \boldsymbol {x} ^ {\prime} \|, \tag {40} \\ \end{array}
$$
where (a) follows from the fact that any  $(f, g) \in S$  are  $L$ -lipschitz under Assumption 2.1.
On the other hand, recall the definition  $\hat{h}_t(\pmb{x}) = \mathbb{E}_{\pmb{v} \sim U(\mathbb{B})}[\mathcal{L}_t(\pmb{x} + \rho \pmb{v}, \pmb{\lambda}_t)]$  in Eq. (11). Then we have
$$
\left| h _ {t} (\boldsymbol {x}) - \hat {h} _ {t} (\boldsymbol {x}) \right| = \mathbb {E} _ {\boldsymbol {v} \sim U (\mathbb {B})} \left[ h _ {t} (\boldsymbol {x}) - h _ {t} (\boldsymbol {x} + \rho \boldsymbol {v}) \right] \leq (1 + K \frac {\bar {F}}{\beta}) L \rho \cdot E _ {\boldsymbol {v} \sim U (\mathbb {B})} \left[ \| \boldsymbol {v} \| \right] \leq (1 + K \frac {\bar {F}}{\beta}) L \rho , \tag {41}
$$
where the inequality follows from the first part of this lemma.
A.10 Proof of Lemma A.2
Proof. Recall the definitions  $h_t(\pmb{x}) = f_t(\pmb{x}) + \pmb{\lambda}_t^\top \pmb{g}_t(\pmb{x})$  in Eq. (5), and  $\hat{h}_t(\pmb{x}) = \mathbb{E}_{\pmb{v} \sim U(\mathbb{B})}[\mathcal{L}_t(\pmb{x} + \rho \pmb{v}, \pmb{\lambda}_t)]$  in Eq. (11). Then, we have
$$
\hat {h} _ {t} (\boldsymbol {y}) - \hat {h} _ {t} (\widetilde {\boldsymbol {x}} _ {t}) \stackrel {(a)} {\leq} \langle \nabla \hat {h} _ {t} (\widetilde {\boldsymbol {x}} _ {t}), \boldsymbol {y} - \widetilde {\boldsymbol {x}} _ {t} \rangle
$$
$$
\begin{array}{l} \stackrel {(b)} {=} \left\langle \frac {d}{\rho} \cdot \mathbb {E} _ {\boldsymbol {u} \sim U (\mathfrak {S})} \left[ h _ {t} \left(\widetilde {\boldsymbol {x}} _ {t} + \rho \boldsymbol {u}\right) \cdot \boldsymbol {u} \right], \boldsymbol {y} - \widetilde {\boldsymbol {x}} _ {t} \right\rangle \\ = \mathbb {E} _ {\boldsymbol {u} _ {t} \sim U (\mathbb {S})} \left[ \left\langle \frac {d}{\rho} \cdot h _ {t} \left(\widetilde {\boldsymbol {x}} _ {t} + \rho \boldsymbol {u} _ {t}\right) \cdot \boldsymbol {u} _ {t}, \boldsymbol {y} - \widetilde {\boldsymbol {x}} _ {t} \right\rangle \right] \tag {42} \\ \end{array}
$$
$$
\begin{array}{l} \stackrel {(c)} {=} \mathbb {E} _ {\boldsymbol {u} _ {t} \sim U (\mathbb {S})} \left[ \langle \nabla_ {t}, \boldsymbol {y} - \widetilde {\boldsymbol {x}} _ {t} \rangle \right] \\ \stackrel {(d)} {=} \mathbb {E} _ {\boldsymbol {u} _ {t} \sim U (\mathbb {S})} \left[ \ell_ {t} \left(\widetilde {\boldsymbol {x}} _ {t}\right) - \ell_ {t} (\boldsymbol {y}) \right] \\ \end{array}
$$
where (a) follows from concavity of  $\hat{h}_t(\cdot)$ ; (b) follows from Lemma B.2 by taking  $h = -h_t$ , so that in the lemma  $-\nabla_{\pmb{x}}\mathbb{E}_{\pmb{v}\sim U(\mathbb{B})}[h(\pmb{x} + \rho \pmb{v})] = \nabla \hat{h}_t(\pmb{x})$  and  $-\mathbb{E}_{\pmb{u}\sim U(\mathbb{S})}[h(\pmb{x} + \rho \pmb{u})\cdot \pmb{u}] = \mathbb{E}_{\pmb{u}\sim U(\mathbb{S})}[h_t(\pmb{x} + \rho \pmb{u})\cdot \pmb{u}]$ ; (c) follows from the gradient estimate in Eq. (7) where
$$
\nabla_ {t} = \frac {d}{\rho} \left(f _ {t} (\boldsymbol {x} _ {t}) + \boldsymbol {\lambda} _ {t} ^ {\top} \boldsymbol {g} _ {t} (\boldsymbol {x} _ {t})\right) \cdot \boldsymbol {u} _ {t} = \frac {d}{\rho} \cdot h _ {t} (\boldsymbol {x} _ {t}) \cdot \boldsymbol {u} _ {t} = \frac {d}{\rho} \cdot h _ {t} (\widetilde {\boldsymbol {x}} _ {t} + \rho \boldsymbol {u} _ {t}) \cdot \boldsymbol {u} _ {t}
$$
Finally, (d) follows from the definition of surrogate loss functions in Eq. (8).
A.11 Proof of Lemma A.3
Proving (i):
Proof. Let's denote  $\nabla_t = \| \nabla_t\|$ . Since  $\widetilde{\pmb{x}}_{t + 1}^i = \Pi_{(1 - \alpha)\mathcal{X}}(\widetilde{\pmb{x}}_t^i +\gamma_i\nabla_t)$  we have  $\| \pmb {y} - \widetilde{\pmb{x}}_{t + 1}^{i}\| \leq$ $\| \pmb {y} - (\widetilde{\pmb{x}}_t^i +\gamma_i\pmb {\nabla}_t)\|$  for any  $\pmb {y}\in (1 - \alpha)\mathcal{X}$ . Then
$$
\begin{array}{l} \left\| \boldsymbol {y} - \widetilde {\boldsymbol {x}} _ {t + 1} ^ {i} \right\| ^ {2} \leq \left\| \boldsymbol {y} - \widetilde {\boldsymbol {x}} _ {t} ^ {i} \right\| ^ {2} - 2 \gamma_ {i} \nabla_ {t} ^ {\top} (\boldsymbol {y} - \widetilde {\boldsymbol {x}} _ {t} ^ {i}) + \gamma_ {i} ^ {2} \nabla_ {t} ^ {2} \\ \Rightarrow \| \widetilde {\boldsymbol {x}} _ {t + 1} ^ {i} \| ^ {2} \leq \| \widetilde {\boldsymbol {x}} _ {t} ^ {i} \| ^ {2} + 2 \boldsymbol {y} ^ {\top} (\widetilde {\boldsymbol {x}} _ {t + 1} ^ {i} - \widetilde {\boldsymbol {x}} _ {t} ^ {i}) - 2 \gamma_ {i} \nabla_ {t} ^ {\top} (\boldsymbol {y} - \widetilde {\boldsymbol {x}} _ {t} ^ {i}) + \gamma_ {i} ^ {2} \nabla_ {t} ^ {2}. \\ \end{array}
$$
Hence by taking  $\pmb{y} = (1 - \alpha)\pmb{y}_t\in (1 - \alpha)\mathcal{X}$  and rearranging we get
$$
\begin{array}{l} 2 \gamma_ {i} \left(\ell_ {t} \left(\widetilde {\boldsymbol {x}} _ {t} ^ {i}\right) - \ell_ {t} ((1 - \alpha) \boldsymbol {y} _ {t})\right) \\ = 2 \gamma_ {i} \boldsymbol {\nabla} _ {t} ^ {\top} ((1 - \alpha) \boldsymbol {y} _ {t} - \widetilde {\boldsymbol {x}} _ {t} ^ {i}) \tag {43} \\ \leq \left\| \widetilde {\boldsymbol {x}} _ {t} ^ {i} \right\| ^ {2} - \left\| \widetilde {\boldsymbol {x}} _ {t + 1} ^ {i} \right\| ^ {2} + 2 (1 - \alpha) \boldsymbol {y} _ {t} ^ {\top} \left(\widetilde {\boldsymbol {x}} _ {t + 1} ^ {i} - \widetilde {\boldsymbol {x}} _ {t} ^ {i}\right) + \gamma_ {i} ^ {2} \nabla_ {t} ^ {2}. \\ \end{array}
$$
Telescoping with  $\tau = 1\ldots t$  we get
$$
\begin{array}{l} \sum_ {\tau \in [ t ]} \ell_ {\tau} \left(\widetilde {\boldsymbol {x}} _ {\tau} ^ {i}\right) - \sum_ {\tau \in [ t ]} \ell_ {\tau} ((1 - \alpha) \boldsymbol {y} _ {\tau}) \\ \leq \frac {1}{2 \gamma_ {i}} \| \widetilde {\boldsymbol {x}} _ {1} ^ {i} \| ^ {2} + \frac {1 - \alpha}{\gamma_ {i}} \sum_ {\tau \in [ t ]} \boldsymbol {y} _ {\tau} ^ {\top} (\widetilde {\boldsymbol {x}} _ {\tau + 1} ^ {i} - \widetilde {\boldsymbol {x}} _ {\tau} ^ {i}) + \frac {\gamma_ {i}}{2} \sum_ {\tau \in [ t ]} \nabla_ {\tau} ^ {2} \\ = \frac {1}{2 \gamma_ {i}} \| \widetilde {\boldsymbol {x}} _ {1} ^ {i} \| ^ {2} + \frac {1 - \alpha}{\gamma_ {i}} \left(\boldsymbol {y} _ {t} ^ {\top} \widetilde {\boldsymbol {x}} _ {t + 1} ^ {i} + \sum_ {\tau \in [ t - 1 ]} \left(\boldsymbol {y} _ {\tau} - \boldsymbol {y} _ {\tau + 1}\right) ^ {\top} \widetilde {\boldsymbol {x}} _ {\tau + 1} ^ {i}\right) + \frac {\gamma_ {i}}{2} \sum_ {\tau \in [ t ]} \nabla_ {\tau} ^ {2} \tag {44} \\ \leq \frac {1}{2 \gamma_ {i}} \| \widetilde {\boldsymbol {x}} _ {1} ^ {i} \| ^ {2} + \frac {1 - \alpha}{\gamma_ {i}} \left(\| \boldsymbol {y} _ {t} \| \cdot \| \widetilde {\boldsymbol {x}} _ {t + 1} ^ {i} \| + \sum_ {\tau \in [ t - 1 ]} \| \boldsymbol {y} _ {\tau} - \boldsymbol {y} _ {\tau + 1} \| \cdot \| \widetilde {\boldsymbol {x}} _ {\tau + 1} ^ {i} \|\right) + \frac {\gamma_ {i}}{2} \sum_ {\tau \in [ t ]} \nabla_ {\tau} ^ {2} \\ \leq \frac {(1 - \alpha) ^ {2} D ^ {2}}{2 \gamma_ {i}} + \frac {(1 - \alpha) ^ {2} D}{\gamma_ {i}} (P (\boldsymbol {y} _ {1: T}) + D) + \frac {\gamma_ {i} d ^ {2}}{2 \rho^ {2}} \left(\bar {F} + K \frac {\bar {F}}{\beta} \bar {G}\right) ^ {2} t, \\ \end{array}
$$
where  $P(\pmb{y}_{1:T})$  is defined in the lemma statement.
Proving (ii):
Proof. First, we have for any  $t \in [T]$ ,  $i \in [N]$
$$
\left| \ell_ {t} \left(\widetilde {\boldsymbol {x}} _ {t} ^ {i}\right) \right| = \left| \boldsymbol {\nabla} _ {t} ^ {T} \left(\widetilde {\boldsymbol {x}} _ {t} - \widetilde {\boldsymbol {x}} _ {t} ^ {i}\right) \right| \leq \| \boldsymbol {\nabla} _ {t} \| \cdot \| \widetilde {\boldsymbol {x}} _ {t} ^ {i} - \widetilde {\boldsymbol {x}} _ {t} \| \leq \frac {d}{\rho} (\bar {F} + K \frac {\bar {F}}{\beta} \bar {G}) \cdot (1 - \alpha) D, \tag {45}
$$
where we recall  $D = \sup_{\{\pmb{x},\pmb{x}^{\prime}\} \subseteq \mathcal{X}}\| \pmb {x} - \pmb{x}^{\prime}\|$  is the diameter of  $\mathcal{X}$ , and both  $\{\widetilde{\pmb{x}}_t^i,\widetilde{\pmb{x}}_t\} \subseteq (1 - \alpha)\mathcal{X}$ .
Define  $W_{t} = \sum_{i\in [N]}w_{i,t}$  for all  $t\in [T]$ , then
$$
\begin{array}{l} \log \left(\frac {W _ {t + 1}}{W _ {t}}\right) = \log \left(\sum_ {i \in [ N ]} \frac {w _ {i , t} \exp \left(- \epsilon \ell_ {t} \left(\widetilde {\boldsymbol {x}} _ {t} ^ {i}\right)\right)}{W _ {t}}\right) \\ = \log \left(\mathbb {E} _ {I _ {t} \sim \boldsymbol {w} _ {t} / W _ {t}} \left[ \exp \left(- \epsilon \ell_ {t} \left(\widetilde {\boldsymbol {x}} _ {t} ^ {I _ {t}}\right)\right) \right]\right) \\ \stackrel {(a)} {\leq} - \epsilon \mathbb {E} _ {I _ {t} \sim \boldsymbol {w} _ {t} / W _ {t}} \left[ \ell_ {t} \left(\widetilde {\boldsymbol {x}} _ {t} ^ {I _ {t}}\right) \right] + \frac {\epsilon^ {2}}{2} \cdot \left(\frac {d}{\rho} \left(\bar {F} + K \frac {\bar {F}}{\beta} \bar {G}\right) \cdot (1 - \alpha) D\right) ^ {2} \tag {46} \\ \stackrel {(b)} {=} - \epsilon \ell_ {t} \left(\mathbb {E} _ {I _ {t} \sim \boldsymbol {w} _ {t} / W _ {t}} \left[ \widetilde {\boldsymbol {x}} _ {t} ^ {I _ {t}} \right]\right) + \frac {\epsilon^ {2}}{2} \cdot \left(\frac {d}{\rho} \left(\bar {F} + K \frac {\bar {F}}{\beta} \bar {G}\right) \cdot (1 - \alpha) D\right) ^ {2} \\ \stackrel {(c)} {=} - \epsilon \ell_ {t} (\widetilde {\boldsymbol {x}} _ {t}) + \frac {\epsilon^ {2}}{2} \cdot \left(\frac {d}{\rho} \left(\bar {F} + K \frac {\bar {F}}{\beta} \bar {G}\right) \cdot (1 - \alpha) D\right) ^ {2}. \\ \end{array}
$$
Here (a) follows from Hoeffding's Lemma as described in Lemma B.1 where we take  $X = \ell_t(\widetilde{\boldsymbol{x}}_t^{I_t})$ ,  $a = -\frac{d}{\rho}\left(\bar{F} + K\frac{\bar{F}}{\beta}\bar{G}\right)\cdot (1 - \alpha)D$  and  $b = \frac{d}{\rho}\left(\bar{F} + K\frac{\bar{F}}{\beta}\bar{G}\right)\cdot (1 - \alpha)D$ ; (b) follows from the definition that  $\ell_t(\widetilde{\boldsymbol{x}}) = \nabla_t^T (\widetilde{\boldsymbol{x}} - \widetilde{\boldsymbol{x}}_t)$  is a linear function in  $\widetilde{\boldsymbol{x}}$ ; (c) follows from Eq.(6).
Hence, telescoping the above we get
$$
\log \left(\frac {W _ {t + 1}}{W _ {1}}\right) \leq - \epsilon \sum_ {\tau \in [ t ]} \ell_ {\tau} (\widetilde {\boldsymbol {x}} _ {\tau}) + \frac {t \epsilon^ {2}}{2} \cdot \left(\frac {d}{\rho} \left(\bar {F} + K \frac {\bar {F}}{\beta} \bar {G}\right) \cdot (1 - \alpha) D\right) ^ {2}. \tag {47}
$$
$$
\begin{array}{l} \log \left(\frac {W _ {t + 1}}{W _ {1}}\right) = \log \left(W _ {t + 1}\right) - \log \left(W _ {1}\right) \\ \geq \log (\max  _ {i \in [ N ]} w _ {i, t}) - \log (N) \\ = \max  _ {i \in [ N ]} \log \left(w _ {i, t}\right) - \log (N) \tag {48} \\ \stackrel {(a)} {=} \max  _ {i \in [ N ]} \log \left(w _ {i, 1} \exp \left(- \epsilon \sum_ {\tau \in [ t ]} \ell_ {\tau} \left(\widetilde {\boldsymbol {x}} _ {\tau} ^ {i}\right)\right)\right) - \log (N) \\ = - \epsilon \min  _ {i \in [ N ]} \sum_ {\tau \in [ t ]} \ell_ {\tau} \left(\widetilde {\boldsymbol {x}} _ {\tau} ^ {i}\right) - \log (N). \\ \end{array}
$$
Hence, after combining Eqs. (47) and (48), and dividing both sides by  $\epsilon > 0$ , we get
$$
\begin{array}{l} \left. \right. - \sum_ {\tau \in [ t ]} \ell_ {\tau} \left(\tilde {\boldsymbol {x}} _ {\tau}\right) + \frac {t \epsilon}{2} \cdot \left(\frac {d}{\rho} \left(\bar {F} + K \frac {\bar {F}}{\beta} \bar {G}\right) \cdot (1 - \alpha) D\right) ^ {2} \geq - \min  _ {i \in [ N ]} \sum_ {\tau \in [ t ]} \ell_ {\tau} \left(\tilde {\boldsymbol {x}} _ {\tau} ^ {i}\right) - \frac {\log (N)}{\epsilon} \\ \Rightarrow \sum_ {\tau \in [ t ]} \ell_ {\tau} (\widetilde {\boldsymbol {x}} _ {\tau}) - \min  _ {i \in [ N ]} \sum_ {\tau \in [ t ]} \ell_ {\tau} (\widetilde {\boldsymbol {x}} _ {\tau} ^ {i}) \leq \frac {t \epsilon}{2} \cdot \left(\frac {d}{\rho} (\bar {F} + K \frac {\bar {F}}{\beta} \bar {G}) \cdot (1 - \alpha) D\right) ^ {2} + \frac {\log (N)}{\epsilon} \\ \Rightarrow \sum_ {\tau \in [ t ]} \ell_ {\tau} \left(\widetilde {\boldsymbol {x}} _ {\tau}\right) - \sum_ {\tau \in [ t ]} \ell_ {\tau} \left(\widetilde {\boldsymbol {x}} _ {\tau} ^ {i}\right) \leq \frac {t \epsilon}{2} \cdot \left(\frac {d}{\rho} \left(\bar {F} + K \frac {\bar {F}}{\beta} \bar {G}\right) \cdot (1 - \alpha) D\right) ^ {2} + \frac {\log (N)}{\epsilon}, \forall i \in [ N ], \tag {49} \\ \end{array}
$$
B Supplementary lemmas
Lemma B.1 (Hoeffding's lemma). Let  $X$  be some random variable such that  $a \leq X \leq b$  almost surely for some  $a, b \in \mathbb{R}$ . Then for any  $\epsilon \in \mathbb{R}$ , we have  $\mathbb{E}\left[\exp (-\epsilon X)\right] \leq \exp \left(-\epsilon \mathbb{E}\left[X\right] + \frac{\epsilon^2(b - a)^2}{8}\right)$ .
Lemma B.2 ([25] Lemma 2.1). Let  $h: \mathcal{X} \to \mathbb{R}$  be some convex function (not necessarily differentiable). Then for any  $\pmb{x} \in \mathcal{X} \subseteq \mathbb{R}^d$  and  $\delta > 0$  we have
$$
\nabla_ {\boldsymbol {x}} \mathbb {E} _ {\boldsymbol {v} \sim U (\mathbb {B})} [ h (\boldsymbol {x} + \delta \boldsymbol {v}) ] = \frac {d}{\delta} \cdot \mathbb {E} _ {\boldsymbol {u} \sim U (\mathbb {S})} [ h (\boldsymbol {x} + \delta \boldsymbol {u}) \cdot \boldsymbol {u} ]. \tag {50}
$$