SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \gamma \sum_ {t = 1} ^ {T} \frac {\bar {Q} ^ {2} (s , \pi^ {t} , c ^ {t}) - \bar {Q} ^ {2} (s , \pi^ {t - 1} , c ^ {t - 1})}{A (\pi^ {t} (\ell (s) | s) , \tilde {\pi} ^ {t} (\ell (s) | s))} = \gamma \sum_ {t = 1} ^ {T - 1} \left(\frac {\bar {Q} ^ {2} (s , \pi^ {t} , c ^ {t})}{A (\pi^ {t} (\ell (s) | s) , \tilde {\pi} ^ {t} (\ell (s) | s))} - \frac {\bar {Q} ^ {2} (s , \pi^ {t} , c ^ {t})}{A (\pi^ {t + 1} (\ell (s) | s) , \tilde {\pi} ^ {t + 1} (\ell (s) | s))}\right) \\ + \gamma \frac {\bar {Q} ^ {2} (s , \pi^ {T} , c ^ {T})}{A (\pi^ {T} (\ell (s) | s) , \tilde {\pi} ^ {T} (\ell (s) | s))} \\ = \gamma \sum_ {t = 1} ^ {T - 1} \left(\frac {1}{A (\pi^ {t} (\ell (s) | s) , \tilde {\pi} ^ {t} (\ell (s) | s))} - \frac {1}{A (\pi^ {t + 1} (\ell (s) | s) , \tilde {\pi} ^ {t + 1} (\ell (s) | s))}\right) \bar {Q} ^ {2} (s, \pi^ {t}, c ^ {t}) \\ + \gamma \frac {\bar {Q} ^ {2} (s , \pi^ {T} , c ^ {T})}{A (\pi^ {T} (\ell (s) | s) , \tilde {\pi} ^ {T} (\ell (s) | s))} \\ \end{array}
$$
$$
\begin{array}{l} \stackrel {\text {L e m m a A . 4}} {\leq} 1 9 2 \gamma \sum_ {t = 1} ^ {T - 1} \bar {Q} ^ {2} (s, \pi^ {t}, c ^ {t}) \left(\left| \pi^ {t} (\ell (s) | s) - \pi^ {t + 1} (\ell (s) | s) \right| + \left| \tilde {\pi} ^ {t} (\ell (s) | s) - \tilde {\pi} ^ {t + 1} (\ell (s) | s) \right|\right) \\ + \gamma \bar {Q} ^ {2} (s, \pi^ {T}, c ^ {T}) | A ^ {- 1} (\pi^ {T} (\ell (s) | s), \tilde {\pi} ^ {T} (\ell (s) | s)) | \\ \end{array}
$$
$$
\begin{array}{l} \stackrel {\text {L e m m a A . 3}} {\leq} 1 9 2 \gamma \sum_ {t = 1} ^ {T - 1} \bar {Q} ^ {2} (s, \pi^ {t}, c ^ {t}) (2 4 \gamma + 4 \gamma) + \gamma \bar {Q} ^ {2} (s, \pi^ {T}, c ^ {T}) | A ^ {- 1} (\pi^ {T} (\ell (s) | s), \tilde {\pi} ^ {T} (\ell (s) | s)) | \\ \leq 4 \cdot 1 9 2 \cdot 2 8 \gamma^ {2} T + 3 2 \gamma \\ \end{array}
$$
where in the last inequality we used  $\bar{Q}^2 (s,\pi^t,c^t)\leq 4\quad \forall t$  and  $A(\pi^{T}(\ell (s)|s),\tilde{\pi}^{T}(\ell (s)|s))\geq \frac{1}{8}$ . Then, for the second term in Equation (19), we use the second fact of Lemma 3.9. In more details, we have that
$$
\begin{array}{l} \gamma \sum_ {t = 1} ^ {T} \frac {\bar {Q} ^ {2} (s , \pi^ {t - 1} , c ^ {t - 1}) - \bar {Q} ^ {2} (s , \pi^ {t} , c ^ {t - 1})}{A (\pi^ {t} (\ell (s) | s) , \tilde {\pi} ^ {t} (\ell (s) | s))} \\ = \gamma \sum_ {t = 1} ^ {T} \frac {\left(\bar {Q} (s , \pi^ {t - 1} , c ^ {t - 1}) + \bar {Q} (s , \pi^ {t} , c ^ {t - 1})\right) \cdot \left(\bar {Q} (s , \pi^ {t - 1} , c ^ {t - 1}) - \bar {Q} (s , \pi^ {t} , c ^ {t - 1})\right)}{A (\pi^ {t} (\ell (s) | s) , \tilde {\pi} ^ {t} (\ell (s) | s))} \\ \leq \gamma \sum_{t = 1}^{T}\underbrace{\left|\bar{Q}(s,\pi^{t - 1},c^{t - 1}) + \bar{Q}(s,\pi^{t},c^{t - 1})\right|}_{\leq 4}\left|\bar{Q}(s,\pi^{t - 1},c^{t - 1}) - \bar{Q}(s,\pi^{t},c^{t - 1})\right|\underbrace{\left|A^{-1}(\pi^{t}(\ell(s)|s),\tilde{\pi}^{t}(\ell(s)|s))\right|}_{\leq 8} \\ \leq 32 \gamma \sum_ {t = 1} ^ {T} \left| \bar {Q} (s, \pi^ {t - 1}, c ^ {t - 1}) - \bar {Q} (s, \pi^ {t}, c ^ {t - 1}) \right| \\ \begin{array}{c} \text {L e m m a 3 . 9} \\ \leq \end{array} 3 2 \cdot 4 8 \gamma^ {2} T \log n. \\ \end{array}
$$
$$
\mathrm {T e r m} \quad \mathrm {I I} \leq 4 \cdot 1 9 2 \cdot 2 8 \gamma^ {2} T + 3 2 \cdot 4 8 \gamma^ {2} T \log n + 3 2 \gamma .
$$
Therefore, neglecting constants, and plugging in the bounds in Equation (16), we obtain
$$
\mathcal {R} _ {\mathrm {l o c}} ^ {T} (s) \leq \mathcal {O} \left(\frac {\log T}{\gamma} + \gamma^ {2} T \log n\right)
$$
Therefore by our selection of  $\gamma \coloneqq \mathcal{O}\left(\log^{1 / 3}T / (T^{1 / 3}\log^{1 / 3}n)\right)$  we get
$$
\mathcal {R} _ {\mathrm {l o c}} ^ {T} (s) \leq \mathcal {O} \left(\left(\log (T)\right) ^ {\frac {2}{3}} (\log n) ^ {\frac {1}{3}} T ^ {\frac {1}{3}}\right)
$$
B.5 Proof of Therem 2.3
Theorem 2.3. Let  $\mathcal{D}$  be the  $n$ -dimensional simplex,  $\mathcal{D} = \Delta_n$ . There exists an online learning algorithm  $\mathcal{A}$  (Algorithm 3) such that for any cost-vector sequence  $c_1, \ldots, c_T \in [-1, 1]^n$ ,
$$
\sum_ {t = 1} ^ {T} \left(c ^ {t - 1} + c ^ {t}\right) ^ {\top} x ^ {t} - \min  _ {x ^ {*} \in \mathcal {D}} \sum_ {t = 1} ^ {T} \left(c ^ {t - 1} + c ^ {t}\right) ^ {\top} x ^ {*} \leq \mathcal {O} \left(T ^ {1 / 3} \cdot \log^ {4 / 3} (n T)\right)
$$
where  $x^{t} = \mathcal{A}_{t}(c^{1},\ldots ,c^{t - 1})$
Proof. By Lemma 3.8 we obtain that  $\mathcal{R}^{\mathrm{alt}}(T) \leq H \max_{s \in V} \mathcal{R}_{\mathrm{loc}}^T(s)$
Then, recalling that by construction  $H = \log n$  and using the bound on  $\mathcal{R}_{\mathrm{loc}}^T(s)$  in Lemma 3.10 gives
$$
\mathcal {R} ^ {\mathrm {a l t}} (T) \leq (\log n) \cdot \mathcal {O} \left((\log (T)) ^ {\frac {2}{3}} (\log n) ^ {\frac {1}{3}} T ^ {\frac {1}{3}}\right) = \mathcal {O} \left((\log (T)) ^ {\frac {2}{3}} (\log n) ^ {\frac {4}{3}} T ^ {\frac {1}{3}}\right)
$$
C Omitted Proof of Section 4
In this section we present the omitted proofs of Section 4.
C.1 Proof of Lemma 4.1
To simplify notation we denote  $\hat{c}_t\coloneqq c^t +c^{t - 1}$  for  $t\geq 1$  where  $c_{0} = (0,\dots ,0)$ . Moreover we denote with  $\| \cdot \|$  the euclidean norm  $\| \cdot \| _2$ . Adaptive FTRL (Algorithm 5) admits the following equivalent form.
Algorithm 5 Adaptive FTRL
1: for round  $t = 1, \dots, T$  do
2: The learner computes  $r_{0:t - 1}\gets \sqrt{1 + \sum_{s = 1}^{t - 1}\|\hat{c}_s\|}$
3: The learner plays  $w_{t} \gets \operatorname{argmin}_{\| x \| \leq 1} \left[ \sum_{s=1}^{t-1} \hat{c}_{t}^{\top} x + \frac{r_{0:t-1}}{2} \| x \|^{2} \right]$
4: The adversary selects cost  $\hat{c}_t$  with  $\| \hat{c}_t\| _2\leq 2$  and the learner receives cost  $\hat{c}_t^\top \cdot x^t$
5: end for
Lemma C.1 ([5]). Let  $w_{1}, \ldots, w_{T} \in \mathcal{B}(0,1)$  the sequence of points produced by Adaptive FTRL given as input the cost-vector sequence  $\hat{c}_{1}, \ldots, \hat{c}_{T}$  and  $x^{*} \coloneqq \operatorname{argmin}_{x \in \mathcal{B}(0,1)} \left[ \sum_{t=1}^{T} \hat{c}_{t}^{\top} x \right]$ . Then for any index  $S \in [T]$ ,
$$
\begin{array}{l} \sum_ {t = 1} ^ {S} \hat {c} _ {t} ^ {\top} (w _ {S + 1} - x ^ {*}) + \sum_ {t = S + 1} ^ {T} \hat {c} _ {t} ^ {\top} (w _ {t} - x ^ {*}) \leq \frac {r _ {0 : S}}{2} \left(\| x ^ {*} \| ^ {2} - \| w _ {S + 1} \| ^ {2}\right) \\ + \sum_ {t = S + 1} ^ {T} \left[ \frac {r _ {t}}{2} \left(\| x ^ {*} \| ^ {2} - \| w _ {t + 1} \| ^ {2}\right) \right] + \sum_ {t = S + 1} ^ {T} \hat {c} _ {t} ^ {\top} (w _ {t} - w _ {t + 1}) \\ \end{array}
$$
where  $r_t = r_{0:t} - r_{0:t-1}$  for  $t \geq 1$ .
Proof. Let  $f_{t}(x) \coloneqq \hat{c}_{t}^{\top} x + \frac{r_{t}}{2} \| x \|^{2}$  where  $r_0 = 1$  and  $\hat{c}_0 = 0$ . Let us also define  $f_{0:t}(x) \coloneqq \sum_{t=0}^{t} f_{t}(x)$ . Since  $\hat{c}_0 = 0$  we get that  $f_{0:t}(x) = \sum_{s=1}^{t} \hat{c}_{s}^{\top} x + \frac{r_{0:t}}{2} \| x \|^{2}$  and thus  $w_{t+1} \coloneqq \operatorname*{argmin}_{x \in \mathbb{B}(0,1)} f_{0:t}(x)$ . Then,
$$
\begin{array}{l} f _ {0: T} \left(x ^ {*}\right) \geq f _ {0: T} \left(w _ {T + 1}\right) \\ = f _ {T} \left(w _ {T + 1}\right) + f _ {0: T - 1} \left(w _ {T + 1}\right) \\ \geq f _ {T} \left(w _ {T + 1}\right) + f _ {0: T - 1} \left(w _ {T}\right) \\ \geq \sum_ {t = S + 1} ^ {T} f _ {t} \left(w _ {t + 1}\right) + f _ {0: S} \left(w _ {S + 1}\right) \\ \end{array}
$$
As a result we get that,
$$
\sum_ {t = 0} ^ {T} \left(\hat {c} _ {t} ^ {\top} x ^ {*} + \frac {r _ {t}}{2} \| x ^ {*} \| ^ {2}\right) \geq \sum_ {t = S + 1} ^ {T} \left(\hat {c} _ {t} ^ {\top} w _ {t + 1} + \frac {r _ {t}}{2} \| w ^ {t + 1} \| ^ {2}\right) + \sum_ {t = 0} ^ {S} \left(\hat {c} _ {t} ^ {\top} w _ {S + 1} + \frac {r _ {t}}{2} \| w _ {S + 1} \| ^ {2}\right)
$$
By rearranging the terms and using the fact that  $\hat{c}_0 = 0$  and  $r_0 = 1$  we get that,
$$
\begin{array}{l} \sum_ {t = 1} ^ {S} \hat {c} _ {t} ^ {\top} \left(w _ {S + 1} - x ^ {*}\right) + \sum_ {t = S + 1} ^ {T} \hat {c} _ {t} ^ {\top} \left(w _ {t} - x ^ {*}\right) \leq \frac {r _ {0 : S}}{2} \left(\| x ^ {*} \| ^ {2} - \| w _ {S + 1} \| ^ {2}\right) \\ + \sum_ {t = S + 1} ^ {T} \left[ \frac {r _ {t}}{2} \left(\| x ^ {*} \| ^ {2} - \| w _ {t + 1} \| ^ {2}\right) \right] + \sum_ {t = S + 1} ^ {T} \hat {c} _ {t} ^ {\top} (w _ {t} - w _ {t + 1}) \\ \end{array}
$$
Lemma C.2 ([6]). Let  $w_{1}, \ldots, w_{T} \in \mathcal{B}(0,1)$  the sequence of points produced by Adaptive FTRL given as input the cost-vector sequence  $\hat{c}_{1}, \ldots, \hat{c}_{T}$  and  $x^{*} \coloneqq \operatorname{argmin}_{x \in \mathcal{B}(0,1)} \left[ \sum_{t=1}^{T} \hat{c}_{t}^{\top} x \right]$ . Then,
$$
\sum_ {t = 1} ^ {T} \hat {c} _ {t} ^ {\top} w _ {t} - \sum_ {t = 1} ^ {T} \hat {c} _ {t} ^ {\top} x ^ {*} \leq 4. 5 \sqrt {1 + \sum_ {t = 1} ^ {T} \| \hat {c} _ {t} \| ^ {2}}
$$
Proof. Applying Lemma C.1 with  $S = 0$  we get that,
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} \hat {c} _ {t} ^ {\top} \left(w _ {t} - x ^ {*}\right) \leq \sum_ {t = 1} ^ {T} \frac {r _ {t}}{2} \left(\| x ^ {*} \| ^ {2} - \| w _ {t + 1} \| ^ {2}\right) + \sum_ {t = 1} ^ {T} \hat {c} _ {t} ^ {\top} \left(w _ {t} - w _ {t + 1}\right) (20) \\ \leq \frac {r _ {0 : T}}{2} + \sum_ {t = 1} ^ {T} \hat {c} _ {t} ^ {\top} \left(w _ {t} - w _ {t + 1}\right) (21) \\ \leq 0. 5 \sqrt {1 + \sum_ {t = 1} ^ {T} \| \hat {c} _ {t} \| ^ {2}} + \sum_ {t = 1} ^ {T} \hat {c} _ {t} ^ {\top} \left(w _ {t} - w _ {t + 1}\right) (22) \\ \end{array}
$$
Up next we bound the second term. Let  $f_{t}(x) \coloneqq \hat{c}_{t}^{\top} x + \frac{r_{t}}{2}$ . By Lemma 7 in [31] for  $f_{1} \coloneqq f_{0:t-1}$  and  $f_{2} \coloneqq f_{0:t}$ . Since  $f_{1}$  is 1-strongly convex with respect to the norm  $r_{0:t-1} \| x\|^2$  and  $f_{2} - f_{1}$  is convex and  $2\| c^{t}\|$ -Lipschitz. Then since  $w_{t} \coloneqq \operatorname*{argmin}_{x \in \mathcal{B}(0,1)} f_{1}(x)$  and  $w_{t+1} \coloneqq \operatorname*{argmin}_{x \in \mathcal{B}(0,1)} f_{2}(x)$ , Lemma 7 in [31] implies that
$$
\| w _ {t} - w _ {t + 1} \| \leq \frac {2 \| \hat {c} _ {t} \|}{r _ {0 : t - 1}} \leq \frac {2 \| \hat {c} _ {t} \|}{\sqrt {1 + \sum_ {s = 1} ^ {t - 1} \| \hat {c} _ {s} \| ^ {2}}}
$$
$$
\hat {c} _ {t} ^ {\top} (w _ {t} - w _ {t + 1}) \leq \| \hat {c} _ {t} \| \| w _ {t} - w _ {t + 1} \| \leq \frac {2 \| \hat {c} _ {t} \| ^ {2}}{\sqrt {1 + \sum_ {s = 1} ^ {t - 1} \| \hat {c} _ {s} \| ^ {2}}} \leq \frac {2 \| \hat {c} _ {t} \| ^ {2}}{\sqrt {1 + \sum_ {s = 1} ^ {t} \| \hat {c} _ {s} \| ^ {2}}}
$$
Summing from  $t = 1$  to  $T$ , we get that
$$
\sum_ {t = 1} ^ {T} \hat {c} _ {t} ^ {\top} (w _ {t} - w _ {t + 1}) \leq 4 \sqrt {1 + \sum_ {t = 1} ^ {T} \| \hat {c} _ {t} \| ^ {2}}
$$
Lemma C.3. Let  $w_{1}, \ldots, w_{T} \in \mathcal{B}(0, 1)$  the sequence of points produced by Adaptive FTRL given as input the cost-vector sequence  $\hat{c}_{1}, \ldots, \hat{c}_{T}$ . Let any round  $t^{*} \in [T]$  such that for all  $t \geq t^{*} + 1$ ,
$$
\| \sum_ {s = 1} ^ {t} \hat {c} _ {s} \| \geq \frac {1}{4} \| \hat {c} _ {s} \| ^ {2} \quad a n d \quad \sum_ {s = 1} ^ {t} \| \hat {c} _ {s} \| ^ {2} \geq 1 7
$$
Then  $\| w_t \| = 1$  for all  $t \geq t^* + 1$  and additionally,
$$
\sum_ {t = t ^ {*}} ^ {T - 1} \hat {c} _ {t} ^ {\top} \cdot \left(w _ {t} - w _ {t + 1}\right) \leq \log (1 + T).
$$
Proof. To simplify notation we denote  $\hat{\sigma}_t \coloneqq \| \hat{c}_t \|^2$ . Moreover, we denote  $\hat{c}_{1:t} = \sum_{s=1}^{t} \hat{c}_s$  and  $\hat{\sigma}_{1:t} = \sum_{s=1}^{t} \hat{\sigma}_s$ . By the definition of  $t^* \in [T]$  we know that for all  $t \geq t^* + 1$ ,
$$
\frac {\| \hat {c} _ {1 : t} \|}{\sqrt {1 + \hat {\sigma} _ {1 : t}}} \geq \frac {\hat {\sigma} _ {1 : t}}{4 \sqrt {1 + \hat {\sigma} _ {1 : t}}} \geq 1
$$
where the last inequality follows by the fact that  $\sigma_{1:t} \geq 17$ . Since  $w_t \in \mathcal{B}(0,1)$  the latter implies that  $\| w_t \| = 1$  for all  $t \geq t^* + 1$  and thus,
$$
\begin{array}{l} w _ {t} = - \frac {\hat {c} _ {1 : t - 1}}{\| \hat {c} _ {1 : t - 1} \|} \quad \text {a n d} \quad w _ {t + 1} = - \frac {\hat {c} _ {1 : t}}{\| \hat {c} _ {1 : t} \|} \\ \| w _ {t} - w _ {t + 1} \| = \| \frac {\hat {c} _ {1 : t - 1}}{\| \hat {c} _ {1 : t - 1} \|} - \frac {\hat {c} _ {1 : t}}{\| \hat {c} _ {1 : t} \|} \| \\ \leq \left\| \frac {\hat {c} _ {1 : t - 1}}{\| \hat {c} _ {1 : t - 1} \|} - \frac {\hat {c} _ {1 : t - 1}}{\| \hat {c} _ {1 : t} \|} \right\| + \left\| \frac {\hat {c} _ {1 : t - 1}}{\| \hat {c} _ {1 : t} \|} - \frac {\hat {c} _ {1 : t}}{\| \hat {c} _ {1 : t} \|} \right\| \\ \leq \left\| \hat {c} _ {1: t - 1} \right\| \cdot \left\| \frac {1}{\| \hat {c} _ {1 : t - 1} \|} - \frac {1}{\| \hat {c} _ {1 : t} \|} \right\| + \frac {\| \hat {c} _ {t} \|}{\| \hat {c} _ {1 : t} \|} \\ \leq \frac {\| \hat {c} _ {1 : t} \| - \| \hat {c} _ {1 : t - 1} \|}{\| \hat {c} _ {1 : t} \|} + \frac {\| \hat {c} _ {t} \|}{\| \hat {c} _ {1 : t} \|} \\ \leq 2 \frac {\| \hat {c} _ {t} \|}{\| \hat {c} _ {1 : t} \|} \\ \end{array}
$$
where the last inequality follows by the triangle inequality,  $\| \hat{c}_{1:t} \| \leq \| \hat{c}_{1:t-1} \| + \| \hat{c}_t \|$ . As a result,
$$
\left\| w _ {t} - w _ {t + 1} \right\| \leq \frac {2 \| \hat {c} _ {t} \|}{\| \hat {c} _ {1 : t} \|} \leq \frac {8 \| \hat {c} _ {t} \|}{\hat {\sigma} _ {1 : t}}
$$
where the last inequality follows by the fact that  $t \geq t^* + 1$  and thus  $\| \hat{c}_{1:t} \| \geq \frac{1}{4} \hat{\sigma}_{1:t}$ . Finally we get that,
$$
\begin{array}{l} \sum_ {t = t ^ {*} + 1} ^ {T} \hat {c} _ {t} ^ {\top} \left(w _ {t} - w _ {t + 1}\right) \leq \sum_ {t = t ^ {*} + 1} ^ {T} \| \hat {c} _ {t} \| \| w _ {t} - w _ {t + 1} \| \\ \leq \sum_ {t = t ^ {*} + 1} ^ {T} \frac {8 \| \hat {c} _ {t} \| ^ {2}}{1 + \hat {\sigma} _ {1 : t}} \\ \leq \sum_ {t = t ^ {*} + 1} ^ {T} \frac {8 \hat {\sigma} _ {t}}{1 + \hat {\sigma} _ {1 : t}} \\ \leq \log \left(1 + \sum_ {t = t ^ {*} + 1} ^ {T} \hat {\sigma} _ {t}\right) \\ \leq \log (1 + T) \\ \end{array}
$$
We conclude the section with the proof of Lemma 4.1. We restate the theorem so as to be consistent with the notation of the section.
Lemma 4.1. Let  $w_{1}, \ldots, w_{T} \in \mathcal{B}(0,1)$  the sequence of points produced by Adaptive FTRL given as input the cost-vector sequence  $\hat{c}_{1}, \ldots, \hat{c}_{T}$ . Let  $t_{1}$  denote the maximum index such that
$$
\sum_ {t = 1} ^ {t _ {1}} \hat {c} _ {t} ^ {\top} w _ {t} \geq - \frac {1}{4} \sum_ {t = 1} ^ {t _ {1}} \| \hat {c} _ {t} \| ^ {2}.
$$
Then the followig holds,
$$
\sum_ {t = 1} ^ {T} \hat {c} _ {t} ^ {\top} w _ {t} - \min  _ {x \in \mathbb {B} (0, 1)} \sum_ {t = 1} ^ {T} \hat {c} _ {t} ^ {\top} x \leq 4 \sqrt {1 + \sum_ {t = 1} ^ {t _ {1}} \| \hat {c} _ {t} \| ^ {2}} + \mathcal {O} (\log T)
$$
Proof. Let  $t_2$  denotes the maximum index such that  $\sum_{s=1}^{t} \hat{c}_s^\top w_s \leq -\|\hat{c}_{1:t}\|$  and  $t_3$  the maximum index such that  $\hat{\sigma}_{1:t} \leq 17$  (as in the proof of Lemma C.3). We consider the following 3 mutually exclusive case,
$$
\cdot t _ {2} \geq \max  (t _ {1}, t _ {3}):
$$
Due to the fact that  $t_2 \geq t_1$  we have that for any  $t \geq t_2 + 1$ ,
$$
- \| \hat {c} _ {1: t} \| \leq \sum_ {s = 1} ^ {t} \hat {c} _ {s} ^ {\top} w _ {s} \leq - \frac {1}{4} \hat {\sigma} _ {1: t}
$$
where the first inequality follows by the definition of  $t_2$  while the second by the definition of  $t_1$ ,  $\sum_{s=1}^{t} \hat{c}_s^\top w_s \leq -\frac{1}{4} \hat{\sigma}_{1:t}$  for all  $t \geq t_1 + 1$ . Since  $t_2 \geq t_3$  we additionally get that  $\hat{\sigma}_{1:t} \geq 17$  for all  $t \geq t_2 + 1$ . As a result,
$$
\left\| \hat {c} _ {1: t} \right\| \geq \frac {1}{4} \hat {\sigma} _ {1: t} \quad \text {a n d} \quad \hat {\sigma} _ {1: t} \geq 1 7 \quad \text {f o r a l l} t \geq t _ {2} + 1
$$
Meaning that the conditions of Lemma C.3 are satisfied for all  $t \geq t_2 + 1$  and thus
$$
\sum_ {t = t _ {2} + 1} ^ {T} \hat {c} _ {t} ^ {\top} \left(w _ {t} - w _ {t + 1}\right) \leq \log (1 + T) \text {a n d} \| w _ {t} \| = 1 \quad \text {f o r a l l} t \geq t _ {2} + 1 \tag {23}
$$
Up next we analyze the regret of Adaptive FTRL,
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} \hat {c} _ {t} ^ {\top} \left(w _ {t} - x ^ {*}\right) = \sum_ {t = 1} ^ {t _ {2}} \hat {c} _ {t} ^ {\top} \left(w _ {t} - x ^ {*}\right) + \sum_ {t = t _ {2} + 1} ^ {T} \hat {c} _ {t} ^ {\top} \left(w _ {t} - x ^ {*}\right) (24) \\ = \sum_ {t = 1} ^ {t _ {2}} \hat {c} _ {t} ^ {\top} \left(w _ {t} - x _ {t _ {2} + 1}\right) + \sum_ {t = 1} ^ {t _ {2}} \hat {c} _ {t} ^ {\top} \left(w _ {t _ {2} + 1} - x ^ {*}\right) \\ + \sum_ {t = t _ {2} + 1} ^ {T} \hat {c} _ {t} ^ {\top} \left(w _ {t} - x ^ {*}\right) (25) \\ \leq - \left\| \hat {c} _ {1: t _ {2}} \right\| - \sum_ {t = 1} ^ {t _ {2}} \hat {c} _ {t} ^ {\top} x _ {t _ {2} + 1} + \sum_ {t = 1} ^ {t _ {2}} \hat {c} _ {t} ^ {\top} \left(x _ {t _ {2} + 1} - x ^ {*}\right) \\ + \sum_ {t = t _ {2} + 1} ^ {T} \hat {c} _ {t} ^ {\top} \left(w _ {t} - x ^ {*}\right) (26) \\ \leq \sum_ {t = 1} ^ {t _ {2}} \hat {c} _ {t} ^ {\top} \left(w _ {t _ {2} + 1} - x ^ {*}\right) + \sum_ {t = t _ {2} + 1} ^ {T} \hat {c} _ {t} ^ {\top} \left(w _ {t} - x ^ {*}\right) (27) \\ \leq \frac {r _ {0 : t _ {2}}}{2} \left(\| x ^ {*} \| ^ {2} - \| w _ {t _ {2} + 1} \| ^ {2}\right) \\ + \sum_ {t = t _ {2} + 1} ^ {T} \frac {r _ {t}}{2} \left(\| x ^ {*} \| ^ {2} - \| w _ {t + 1} \| ^ {2}\right) + \sum_ {t = t _ {2} + 1} ^ {T} \hat {c} _ {t} ^ {\top} \left(w _ {t} - w _ {t + 1}\right) (28) \\ = \frac {r _ {0 : t _ {2}}}{2} \left(\| x ^ {*} \| ^ {2} - 1\right) + \sum_ {t = t _ {2} + 1} ^ {T} \frac {r _ {t}}{2} \left(\| x ^ {*} \| ^ {2} - 1\right) \\ + \sum_ {t = t _ {2} + 1} ^ {T} \hat {c} _ {t} ^ {\top} \left(w _ {t} - w _ {t + 1}\right) (29) \\ \leq \sum_ {t = t _ {2} + 1} ^ {T} \hat {c} _ {t} ^ {\top} \left(w _ {t} - w _ {t + 1}\right) \leq \log (1 + T) (30) \\ \end{array}
$$
where Inequality (9) follows by the definition of  $t_2$  i.e.  $\sum_{t=1}^{t_2} \hat{c}_t^\top x^t \leq -\|\hat{c}_{1:t_2}\|$ . Inequality (10) follows by the fact that  $\sum_{t=1}^{t_2} \hat{c}_t^\top x_{t_2+1} \geq -\hat{c}_{1:t_2}$ . Inequality (11) follows by applying Lemma C.1 for  $S := t_2$ . Equality (12) and Inequality (13) follow by Equation 23.
-  $t_1 \geq \max(t_2, t_3)$ : By using the exact same arguments as above we can establish that
$$
\sum_ {t = t _ {2} + 1} ^ {T} \hat {c} _ {t} ^ {\top} \left(x ^ {t} - x ^ {t + 1}\right) \leq \log (1 + T) \text {a n d} \| x _ {t} \| _ {2} = 1 \quad \text {f o r a l l} t \geq t _ {1} + 1 \tag {31}
$$
Using the exact same arguments as above we conclude that
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} \hat {c} _ {t} ^ {\top} \left(w _ {t} - x ^ {*}\right) = \sum_ {t = 1} ^ {t _ {1}} \hat {c} _ {t} ^ {\top} \left(w _ {t} - x ^ {*}\right) + \sum_ {t = t _ {1} + 1} ^ {T} \hat {c} _ {t} ^ {\top} \left(w _ {t} - x ^ {*}\right) \\ = \sum_ {t = 1} ^ {t _ {1}} \hat {c} _ {t} ^ {\top} \left(w _ {t} - w _ {t _ {1} + 1}\right) + \sum_ {t = 1} ^ {t _ {1}} \hat {c} _ {t} ^ {\top} \left(w _ {t _ {1} + 1} - x ^ {*}\right) + \sum_ {t = t _ {1} + 1} ^ {T} \hat {c} _ {t} ^ {\top} \left(w _ {t} - x ^ {*}\right) \\ \leq 4. 5 \sqrt {1 + \hat {\sigma} _ {1 : t _ {1}}} + + \sum_ {t = 1} ^ {t _ {1}} \hat {c} _ {t} ^ {\top} (w _ {t _ {1} + 1} - x ^ {*}) + \sum_ {t = t _ {1} + 1} ^ {T} \hat {c} _ {t} ^ {\top} (w _ {t} - x ^ {*}) \\ \leq 4. 5 \sqrt {1 + \sigma_ {1 : t _ {1}}} + \log (1 + T) \\ \end{array}
$$
where the first inequality follows by applying Lemma C.2 for  $T = t_{1}$  and the second by repeating Inequalities (11) - (15).
-  $t_2 \geq \max(t_1, t_3)$ : By the exact same arguments as in the previous case,
$$
\sum_ {t = 1} ^ {T} \hat {c} _ {t} ^ {\top} \left(w _ {t} - x ^ {*}\right) \leq 4. 5 \sqrt {1 + \sigma_ {1 : t _ {3}}} + \log (1 + T) \leq 4. 5 \sqrt {1 8} + \log (1 + T)
$$
where the last inequality follows by the fact that  $\sigma_{1:t_3} \leq 17$  (definition of  $t_3$ ).
As a result, we have established that in any case,
$$
\sum_ {t = 1} ^ {T} \hat {c} _ {t} ^ {\top} \left(w _ {t} - x ^ {*}\right) \leq 4. 5 \sqrt {1 + \sum_ {t = 1} ^ {t _ {1}} \| \hat {c} _ {t} \| _ {2} ^ {2}} + \log (1 + T) + 4. 5 \sqrt {1 8}
$$
C.2 Proof of Lemma 4.3
To simplify notation we summarize the Step 7 of Algorithm 4 in Algorithm 6.
Algorithm 6 OGD with Shrinking Domain
1:  $p_1 \gets 0, D_1 \gets [0,1]$
2: for  $t = 1 \dots T$  do
3: The learner plays  $p_t \in D_t$
4: The adversary selects  $z_{t}$  and  $\sigma_t \leq 1$ .
5: The learner updates the interval  $D_{t} \subseteq [0,1]$  as follows,
$$
D _ {t} \leftarrow \left[ 0, \min  \left(1, \frac {\lambda}{\sqrt {1 + \sum_ {s = 1} ^ {t} \sigma_ {s}}}\right) \right]
$$
and its actions  $p_{t+1} \in [0, 1]$  as follows
$$
p _ {t + 1} \leftarrow \left[ p _ {t} - \eta_ {t} \cdot z _ {t} \right] _ {D _ {t}}
$$
Remark C.4. We remark that Algorithm 6 corresponds to Step 7 of Algorithm 4 once
$$
\lambda := 2 0, z _ {t} := \left(c ^ {t} + c ^ {t - 1}\right) ^ {\top} \cdot \left(w _ {t} + c ^ {t - 1}\right) \text {a n d} \sigma_ {t} := \| c ^ {t} + c ^ {t - 1} \| ^ {2}
$$
Definition C.5. A sequence  $q_{1}, \ldots, q_{T} \in [0, 1]$  is valid in hindsight if and only if there exists a round  $t^{*} \in [T]$  and a  $\delta \in [0, 1]$  such that the following hold,
1.  $q_{t} = \delta \cdot \mathrm{I}\left[t\leq t_{1}\right]$  ( $q_{t} = \delta$  for all  $t\leq t^{*}$  and  $q_{t} = 0$  for all  $t\geq t^{*} + 1$ ).
2. At the switching point  $t^* \in [T]$ ,
$$
\delta^ {2} \leq \frac {\lambda^ {2}}{1 + \sum_ {t = 1} ^ {t ^ {*}} \sigma_ {t}}
$$
In Theorem C.6 we present the payoff guarantees of Algorithm 6 with respect to any sequence  $q_{t}$  that is valid in hindsight.
Theorem C.6 ([5]). Let  $p_1, \ldots, p_T \in [0, 1]$  a sequence of points produced by Algorithm 6 given as input the sequence  $(z_1, \sigma_1), \ldots, (z_T, \sigma_T)$ . In case  $z_t^2 \leq 4\sigma_t$  for all rounds  $t \in [T]$  then for any valid in hindsight sequence  $q_1, \ldots, q_T \in [0, 1]$  (Definition C.5) the following holds,
$$
\sum_ {t = 1} ^ {T} z _ {t} \left(p _ {t} - q _ {t}\right) \leq \lambda \left(1 + 3 \log \left(1 + \sum_ {t = 1} ^ {T} \sigma_ {t}\right)\right)
$$
We conclude the section with the proof of Lemma 4.3.
Lemma 4.3. Let the sequence of cost-vector  $c^1, \ldots, c^T$  given to Algorithm 4 and the produced sequences  $x^1, \ldots, x^t \in \Delta_n$  and  $p_1, \ldots, p_T \in (0,1)$ . Additionally let  $t_1$  denote the maximum time such that
$$
\sum_ {s = 1} ^ {t} \left(c ^ {s} + c ^ {s - 1}\right) ^ {\top} \cdot w _ {s} \geq - \frac {1}{4} \sum_ {s = 1} ^ {t} \| c ^ {s} + c ^ {s - 1} \| _ {2} ^ {2}
$$
and consider the sequence  $q_{t} \coloneqq \mathrm{I}\left[t \leq t_{1}\right] \cdot \left(20 / \sqrt{400 + \sum_{t=1}^{t_{1}}\|c^{t} + c^{t-1}\|_{2}^{2}}\right)$ . Then the following holds,
$$
\sum_ {t = 1} ^ {T} \left(c ^ {t - 1} + c ^ {t}\right) ^ {\top} \left(w _ {t} + c ^ {t - 1}\right) \cdot q _ {t} - \sum_ {t = 1} ^ {T} \left(c ^ {t - 1} + c ^ {t}\right) ^ {\top} \left(w _ {t} + c ^ {t - 1}\right) \cdot p _ {t} \leq \mathcal {O} (\log T)
$$
Proof. The sequence  $q_{t}$  is a valid sequence with switching point  $t^{*} \coloneqq t_{1}$  and
$$
\delta := \frac {2 0}{\sqrt {4 0 0 + \sum_ {t = 1} ^ {t _ {1}} \| c ^ {t} + c ^ {t - 1} \| _ {2} ^ {2}}}
$$
Now the sequence  $p_t$  produced by Algorithm 4 in Steps 7 and Steps 8 can be viewed as the output of Algorithm 6 with the input sequence  $z_t := (c^t + c^{t-1})^\top \cdot (w_t + c^{t-1})$  and  $\sigma_t := \| c^t + c^{t-1} \|^2$ . Since
Lemma 4.3 follows by Theorem C.6.