SlowGuess/NeurIPS_2023_OCR huggingface dataset:
By combining (17) and (18) and noticing that  $\eta_{t - 1}\eta_t\leq 1 / 512$
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} \langle g _ {t} - p _ {t} \oslash x _ {t}, x _ {t} - x _ {t + 1} \rangle - \frac {1}{4 \eta_ {t - 1}} r _ {t} ^ {2} \\ \leq 2 \eta_ {0} + \sum_ {t = 2} ^ {T} 4 \eta_ {t - 1} \| \nabla f _ {t} \left(x _ {t - 1}\right) - g _ {t - 1} \| _ {x _ {t - 1}, *} ^ {2} + \sum_ {t = 1} ^ {T - 1} \left(6 4 \eta_ {t} - \frac {1}{8 \eta_ {t - 1}}\right) r _ {t} ^ {2} \\ \leq 2 \eta_ {0} + \sum_ {t = 2} ^ {T} 4 \eta_ {t - 1} \| \nabla f _ {t} (x _ {t - 1}) - g _ {t - 1} \| _ {x _ {t - 1}, *} ^ {2}. \\ \end{array}
$$
$$
R _ {T} \leq \frac {d \log T}{\eta_ {T}} + 2 \eta_ {0} + \sum_ {t = 2} ^ {T} 4 \eta_ {t - 1} \| \nabla f _ {t} (x _ {t - 1}) - \nabla f _ {t - 1} (x _ {t - 1}) \| _ {x _ {t - 1}, *} ^ {2} + 2.
$$
Plug in the choice of learning rates into the regret bound. We obtain [28, Lemma 4.13]
$$
\begin{array}{l} R _ {T} \leq \sqrt {d} \log T \sqrt {V _ {T} + 5 1 2 d + 2} + 4 \sqrt {d} \sum_ {t = 2} ^ {T} \frac {\left\| \nabla f _ {t} \left(x _ {t - 1}\right) - \nabla f _ {t - 1} \left(x _ {t - 1}\right) \right\| _ {x _ {t - 1} , *} ^ {2}}{\sqrt {5 1 2 d + 2 + V _ {t - 1}}} + 3 \\ \leq \sqrt {d} \log T \sqrt {V _ {T} + 5 1 2 d + 2} + 4 \sqrt {d} \sum_ {t = 2} ^ {T} \frac {\| \nabla f _ {t} (x _ {t - 1}) - \nabla f _ {t - 1} (x _ {t - 1}) \| _ {x _ {t - 1} , *} ^ {2}}{\sqrt {5 1 2 d + V _ {t}}} + 3 \\ \leq \sqrt {d} \log T (\sqrt {V _ {T} + 5 1 2 d} + \sqrt {2}) + 4 \sqrt {d} \int_ {0} ^ {V _ {T}} \frac {\mathrm {d} s}{\sqrt {5 1 2 d + s}} + 3 \\ \leq (\log T + 8) \sqrt {d V _ {T} + 5 1 2 d ^ {2}} + \sqrt {2 d} \log T + 2 - 1 2 8 \sqrt {2 d}. \\ \end{array}
$$
G.2 Proof of Lemma G.1
$$
y _ {t} \in \underset {x \in \Delta} {\arg \min } \left\langle g _ {1: t}, x \right\rangle + \frac {1}{\eta_ {t - 1}} h (x), \quad z _ {t} \in \underset {x \in \Delta} {\arg \min } \left\langle g _ {1: t}, x \right\rangle + \frac {1}{\eta_ {t}} h (x).
$$
$$
\left\| x _ {t} - x _ {t + 1} \right\| _ {x _ {t}} \leq \left\| x _ {t} - y _ {t} \right\| _ {x _ {t}} + \left\| y _ {t} - z _ {t} \right\| _ {x _ {t}} + \left\| z _ {t} - x _ {t + 1} \right\| _ {x _ {t}}. \tag {19}
$$
Define  $d_{1} \coloneqq \| x_{t} - y_{t}\|_{x_{t}}$ ,  $d_{2} \coloneqq \| y_{t} - z_{t}\|_{y_{t}}$ , and  $d_{3} \coloneqq \| z_{t} - x_{t + 1}\|_{x_{t + 1}}$ . Suppose that  $d_{i} \leq 1 / 5$  for all  $1 \leq i \leq 3$ , which we will verify later. By Theorem C.1,
$$
(1 - d _ {1}) ^ {2} \nabla^ {2} h (x _ {t}) \leq \nabla^ {2} h (y _ {t}) \leq \frac {1}{(1 - d _ {1}) ^ {2}} \nabla^ {2} h (x _ {t}),
$$
$$
(1 - d _ {2}) ^ {2} \nabla^ {2} h (y _ {t}) \leq \nabla^ {2} h (z _ {t}) \leq \frac {1}{(1 - d _ {2}) ^ {2}} \nabla^ {2} h (y _ {t}), \tag {20}
$$
$$
(1 - d _ {3}) ^ {2} \nabla^ {2} h (x _ {t + 1}) \leq \nabla^ {2} h (z _ {t}) \leq \frac {1}{(1 - d _ {3}) ^ {2}} \nabla^ {2} h (x _ {t + 1}).
$$
$$
\left\| y _ {t} - z _ {t} \right\| _ {x _ {t}} \leq \frac {d _ {2}}{1 - d _ {1}}, \quad \left\| z _ {t} - x _ {t + 1} \right\| _ {x _ {t}} \leq \frac {d _ {3}}{(1 - d _ {1}) (1 - d _ {2}) (1 - d _ {3})}.
$$
Then, the inequality (19) becomes
$$
\left\| x _ {t} - x _ {t + 1} \right\| _ {x _ {t}} \leq d _ {1} + \frac {d _ {2}}{1 - d _ {1}} + \frac {d _ {3}}{(1 - d _ {1}) (1 - d _ {2}) (1 - d _ {3})}.
$$
It is easily checked that the maximum of the right-hand side is attained at  $d_{1} = d_{2} = d_{3} = 1 / 5$  and the maximum value equals 269/320. This proves the lemma.
Now, we prove that  $d_{i}\leq 1 / 5$  for all  $1\leq i\leq 3$
1. (Upper bound of  $d_{1}$ .) By the optimality conditions of  $x_{t}$  and  $y_{t}$ ,
$$
\langle \eta_ {t - 1} g _ {1: t} + \nabla h (y _ {t}), x _ {t} - y _ {t} \rangle \geq 0
$$
$$
\left\langle \eta_ {t - 1} g _ {1: t - 1} + \eta_ {t - 1} \hat {g} _ {t} + \nabla h (x _ {t}), y _ {t} - x _ {t} \right\rangle \geq 0.
$$
Sum up the two inequalities. We get
$$
\eta_ {t - 1} \left\langle g _ {t} - \hat {g} _ {t}, x _ {t} - y _ {t} \right\rangle \geq \left\langle \nabla h (x _ {t}) - \nabla h (y _ {t}), x _ {t} - y _ {t} \right\rangle
$$
By Hölder's inequality, Lemma 4.4, and Theorem C.2, we obtain
$$
\eta_ {t - 1} \left\| g _ {t} - \hat {g} _ {t} \right\| _ {x _ {t}, *} d _ {1} \geq \frac {d _ {1} ^ {2}}{1 + d _ {1}}.
$$
Then, by the assumption (16), we write
$$
d _ {1} \leq \frac {\eta_ {t - 1} \| g _ {t} - \hat {g} _ {t} \| _ {x _ {t} , *}}{1 - \eta_ {t - 1} \| g _ {t} - \hat {g} _ {t} \| _ {x _ {t} , *}} \leq 1 / 5.
$$
2. (Upper bound of  $d_{2}$ .) By the optimality conditions of  $y_{t}$  and  $z_{t}$ ,
$$
\left\langle g _ {1: t} + \frac {1}{\eta_ {t}} \nabla h \left(z _ {t}\right), y _ {t} - z _ {t} \right\rangle \geq 0
$$
$$
\langle g _ {1: t} + \frac {1}{\eta_ {t - 1}} \nabla h (y _ {t}), z _ {t} - y _ {t} \rangle \geq 0.
$$
$$
\left(\frac {1}{\eta_ {t}} - \frac {1}{\eta_ {t - 1}}\right) \langle \nabla h (y _ {t}), y _ {t} - z _ {t} \rangle \geq \frac {1}{\eta_ {t}} \langle \nabla h (y _ {t}) - \nabla h (z _ {t}), y _ {t} - z _ {t} \rangle .
$$
By Definition C.4, Lemma C.5, and Theorem C.2,
$$
\left(\frac {1}{\eta_ {t}} - \frac {1}{\eta_ {t - 1}}\right) \sqrt {d} d _ {2} \geq \frac {1}{\eta_ {t}} \frac {d _ {2} ^ {2}}{1 + d _ {2}}.
$$
$$
d _ {2} \leq \frac {(1 - \eta_ {t} / \eta_ {t - 1}) \sqrt {d}}{1 - (1 - \eta_ {t} / \eta_ {t - 1}) \sqrt {d}} \leq \frac {1}{5}.
$$
3. (Upper bound of  $d_3$ .) By the optimality conditions of  $z_t$  and  $x_{t+1}$ ,
$$
\left\langle \eta_ {t} g _ {1: t} + \eta_ {t} \hat {g} _ {t + 1} + \nabla h (x _ {t + 1}), z _ {t} - x _ {t + 1} \right\rangle \geq 0
$$
$$
\left\langle \eta_ {t} g _ {1: t} + \nabla h \left(z _ {t}\right), x _ {t + 1} - z _ {t} \right\rangle \geq 0.
$$
$$
\eta_ {t} \left\langle \hat {g} _ {t + 1}, z _ {t} - x _ {t + 1} \right\rangle \geq \left\langle \nabla h (z _ {t}) - \nabla h (x _ {t + 1}), z _ {t} - x _ {t + 1} \right\rangle .
$$
By Hölder's inequality, the assumption (16), and Theorem C.2, we obtain
$$
\eta_ {t} d _ {3} \geq \eta_ {t} \| \hat {g} _ {t + 1} \| _ {x _ {t + 1}, *} d _ {3} \geq \frac {d _ {3} ^ {2}}{1 + d _ {3}}.
$$
$$
d _ {3} \leq \frac {\eta_ {t}}{1 - \eta_ {t}} \leq 1 / 5.
$$
H Proof of Theorem 6.2 (Small-Loss Bound)
Recall that  $\alpha_{t}$  minimizes  $\| g_t + \alpha e\|_{x_t,*}^2$  over all  $\alpha \in \mathbb{R}$  (Remark 4.8). We write
$$
\left\| g _ {t} + \alpha_ {t} e \right\| _ {x _ {t}, *} ^ {2} \leq \left\| g _ {t} \right\| _ {x _ {t}, *} ^ {2} \leq 1,
$$
which, with the observation that  $\eta_t \leq 1/2$  for all  $t \in \mathbb{N}$ , implies that  $\eta_{t-1} \| g_t + \alpha_t e \|_{x_t,*} \leq 1/2$ . Then, since the all-ones vector  $e$  is orthogonal to the set  $\Delta - \Delta$ , by Corollary 5.2 with  $p_t = 0$  and  $u_t = \alpha_t e$ , we get
$$
\begin{array}{l} R _ {T} \leq \frac {d \log T}{\eta_ {T}} + \sum_ {t = 1} ^ {T} \eta_ {t - 1} \| (g _ {t} + \alpha_ {t} e) \odot x _ {t} \| _ {2} ^ {2} + 2 \\ = \frac {d \log T}{\eta_ {T}} + \sum_ {t = 1} ^ {T} \eta_ {t - 1} \| g _ {t} + \alpha_ {t} e \| _ {x _ {t}, *} ^ {2} + 2 \\ \end{array}
$$
Plug in the explicit expressions of the learning rates  $\eta_{t}$ . We write [28, Lemma 4.13]
$$
\begin{array}{l} R _ {T} \leq \frac {d \log T}{\eta_ {T}} + \sqrt {d} \sum_ {t = 1} ^ {T} \frac {\left\| g _ {t} + \alpha_ {t} e \right\| _ {x _ {t} , *} ^ {2}}{\sqrt {4 d + 1 + \sum_ {\tau = 1} ^ {t - 1} \left\| g _ {\tau} + \alpha_ {\tau} e \right\| _ {x _ {\tau} , *} ^ {2}}} + 2 \\ \leq \frac {d \log T}{\eta_ {T}} + \sqrt {d} \sum_ {t = 1} ^ {T} \frac {\left\| g _ {t} + \alpha_ {t} e \right\| _ {x _ {t} , *} ^ {2}}{\sqrt {4 d + \sum_ {\tau = 1} ^ {t} \left\| g _ {\tau} + \alpha_ {\tau} e \right\| _ {x _ {\tau} , *} ^ {2}}} + 2 \\ \leq \sqrt {d} \log T \sqrt {4 d + 1 + \sum_ {t = 1} ^ {T} \| g _ {t} + \alpha_ {t} e \| _ {x _ {t , *}} ^ {2}} + \sqrt {d} \int_ {0} ^ {\sum_ {t = 1} ^ {T} \| g _ {t} + \alpha_ {t} e \| _ {x _ {t, *}} ^ {2}} \frac {\mathrm {d} s}{\sqrt {4 d + s}} + 2 \\ \leq (\log T + 2) \sqrt {d \sum_ {t = 1} ^ {T} \| g _ {t} + \alpha_ {t} e \| _ {x _ {t , *}} ^ {2} + 4 d ^ {2} + d} \\ \leq (\log T + 2) \sqrt {d \sum_ {t = 1} ^ {T} \| g _ {t} + \alpha_ {t} e \| _ {x _ {t} , *} ^ {2} + 4 d ^ {2} + d}. \\ \end{array}
$$
Finally, by Lemma 4.7,
$$
R _ {T} \leq (\log T + 2) \sqrt {4 d \sum_ {t = 1} ^ {T} f _ {t} (x _ {t}) + 4 d ^ {2} + d}.
$$
The theorem then follows from Lemma 4.23 of Orabona [28].
I A Second-Order Regret Bound
In this section, we introduce AA + LB-FTRL with Average-Multiplicative-Gradient Optimism (Algorithm 6) that achieves a second-order regret bound. This bound is characterized by the variance-like quantity
$$
Q _ {T} := \min _ {p \in \mathbb {R} ^ {d}} \sum_ {t = 1} ^ {T} \| x _ {t} \odot \nabla f _ {t} (x _ {t}) - p \| _ {2} ^ {2}.
$$
For comparison, existing second-order bounds are characterized by the quantity
$$
V _ {T} := \min  _ {u \in \mathbb {R} ^ {d}} \sum_ {t = 1} ^ {T} \| \nabla f _ {t} (x _ {t}) - u \| _ {2} ^ {2},
$$
$T$  times the empirical variance of the gradients (see, e.g., Section 7.12.1 in the lecture note of Orabona [28]). Given that every  $p \in \mathbb{R}^d$  can be expressed as  $p = x_t \odot v$  for  $v = p \oslash x_t$ , and by the definition of the dual local norm (3), we obtain
$$
Q _ {T} = \min  _ {v \in \mathbb {R} ^ {d}} \sum_ {t = 1} ^ {T} \| \nabla f _ {t} (x _ {t}) - v \| _ {x _ {t}, *} ^ {2},
$$
showing that  $Q_{T}$  is a local-norm analogue of  $V_{T}$ .
The flexibility introduced by optimizing  $Q_{T}$  over all  $v \in \mathbb{R}^{d}$  allows for a fast regret rate when the data is easy. In particular, Theorem I.2 in Section I.3 shows that when  $Q_{T} \leq 1$ , then Algorithm 6 achieves a fast  $O(d\log T)$  regret rate. However, while  $Q_{T}$  looks reasonable as a mathematical extension of  $V_{T}$ , the empirical variance interpretation for  $V_{T}$  does not extend to  $Q_{T}$ . Interpreting  $Q_{T}$  or seeking a "more natural" variant of  $Q_{T}$  is left as a future research topic.
I.1 The Gradient Estimation Problem
To obtain a better regret bound by using Corollary 5.2, one has to design a good strategy of choosing  $p_t$ . In Section 6.1, we have used  $p_t = x_{t-1} \odot g_{t-1}$ . In this section, we consider a different approach.
Consider the following online learning problem, which we call the Gradient Estimation Problem. It is a multi-round game between LEARNER and REALITY. In the  $t$ -th round, LEARNER chooses a vector  $p_t \in \mathbb{R}^d$ ; then, REALITY announces a convex loss function  $\ell_t(p) = \eta_{t-1} \| x_t \odot g_t - p\|_2^2$ ; finally, LEARNER suffers a loss  $\ell_t(p_t)$ .
I.2 LB-FTRL with Average-Multiplicative-Gradient Optimism
In the gradient estimation problem, the loss functions are strongly-convex. A natural strategy of choosing  $p_t$  is then following the leader (FTL). In the  $t$ -th round, FTL suggests choosing
$$
p _ {t} = \frac {1}{\eta_ {0 : t - 2}} \sum_ {\tau = 1} ^ {t - 1} \eta_ {\tau - 1} x _ {\tau} \odot g _ {\tau} \in \underset {p \in \mathbb {R} ^ {d}} {\arg \min } \sum_ {\tau = 1} ^ {t - 1} \ell_ {\tau} (p).
$$
This strategy leads to Algorithm 5. Theorem I.1 provides a regret bound for Algorithm 5.
Algorithm 5 LB-FTRL with Average-Multiplicative-Gradient Optimism
Input: A sequence of learning rates  $\{\eta_t\}_{t\geq 0}\subseteq \mathbb{R}_{++}$    
1:  $h(x)\coloneqq -d\log d - \sum_{i = 1}^{d}\log x(i).$    
2:  $x_{1}\gets \arg \min_{x\in \Delta}\eta_{0}^{-1}h(x).$    
3: for all  $t\in \mathbb{N}$  do   
4: Announce  $x_{t}$  and receive  $a_{t}$    
5:  $g_{t}\coloneqq \nabla f_{t}(x_{t})$    
6:  $p_{t + 1}\leftarrow (1 / \eta_{0:t - 1})\sum_{\tau = 1}^{t}\eta_{\tau -1}x_{\tau}\odot g_{\tau}.$    
7: Solve  $x_{t + 1}$  and  $\hat{g}_{t + 1}$  satisfying  $\begin{array}{r}\left\{ \begin{array}{ll}x_{t + 1}\odot \hat{g}_{t + 1} = p_{t + 1},\\ x_{t + 1}\in \arg \min_{x\in \Delta}\langle g_{1:t},x\rangle +\langle \hat{g}_{t + 1},x\rangle +\eta_t^{-1}h(x). \end{array} \right. \end{array}$
Theorem I.1. Assume that the sequence of learning rates  $\{\eta_t\}$  is non-increasing and  $\eta_0 \leq 1/(2\sqrt{2})$ . Then, Algorithm 5 achieves
$$
R _ {T} \leq \frac {d \log T}{\eta_ {T}} + 2 \sum_ {t = 1} ^ {T} \frac {\eta_ {t - 1} ^ {2}}{\eta_ {0 : t - 1}} + 2 + \min  _ {p \in \mathbb {R} ^ {d}} \sum_ {t = 1} ^ {T} \eta_ {t - 1} \| x _ {t} \odot \nabla f _ {t} (x _ {t}) - p \| _ {2} ^ {2}.
$$
If  $\eta_t = \eta \leq 1 / (2\sqrt{2})$  is a constant, then
$$
R _ {T} \leq \frac {d \log T}{\eta} + 2 \eta (\log T + 1) + 2 + \eta Q _ {T}.
$$
Proof. Since  $p_t$  is a convex combination of  $x_{\tau} \odot g_{\tau}$ , by Lemma 4.1,  $p_t \in -\Delta$ . We restrict the action set of the gradient estimation problem to be  $-\Delta$ . By the regret bound of FTL for strongly convex losses [28, Corollary 7.24],
$$
\sum_ {t = 1} ^ {T} \ell_ {t} (p _ {t}) - \min _ {p \in - \Delta} \sum_ {t = 1} ^ {T} \ell_ {t} (p) \leq \frac {1}{2} \sum_ {t = 1} ^ {T} \frac {(2 \sqrt {2} \eta_ {t - 1}) ^ {2}}{2 \eta_ {0 : t - 1}} = 2 \sum_ {t = 1} ^ {T} \frac {\eta_ {t - 1} ^ {2}}{\eta_ {0 : t - 1}}.
$$
Then, by Corollary 5.2,
$$
R _ {T} \leq \frac {d \log T}{\eta_ {T}} + 2 + \sum_ {t = 1} ^ {T} \ell_ {t} (p _ {t}) \leq \frac {d \log T}{\eta_ {T}} + 2 + 2 \sum_ {t = 1} ^ {T} \frac {\eta_ {t - 1} ^ {2}}{\eta_ {0 : t - 1}} + \min  _ {p \in - \Delta} \sum_ {t = 1} ^ {T} \ell_ {t} (p).
$$
The first bound in the theorem follows from
$$
\min  _ {p \in - \Delta} \sum_ {t = 1} ^ {T} \ell_ {t} (p) = \min  _ {p \in \mathbb {R} ^ {d}} \sum_ {t = 1} ^ {T} \eta_ {t - 1} \| x _ {t} \odot g _ {t} - p \| _ {2} ^ {2}.
$$
The second bound in the theorem follows from the inequality  $\sum_{t=1}^{T} \eta^2 / (t\eta) \leq \eta (\log T + 1)$ .
Time Complexity. Suppose that a constant learning rate is used. Then, both  $g_{t}$  and  $p_{t+1}$  can be computed in  $O(d)$  arithmetic operations. By Theorem 5.1,  $x_{t+1}$  can be computed in  $\tilde{O}(d)$  time. Hence, the per-round time of Algorithm 5 is  $\tilde{O}(d)$ .
I.3 AA + LB-FTRL with Average-Multiplicative-Gradient Optimism
The constant learning rate that minimizes the regret bound in Theorem I.1 is  $\eta = O(\sqrt{d\log T / Q_T})$ . However, the value of  $Q_{T}$  is not unknown to INVESTOR initially.
In this section, we propose Algorithm 6, which satisfies a second-order regret bound without the need for knowing  $Q_{T}$  in advance. Algorithm 6 uses multiple instances of Algorithm 5, which we call experts, with different learning rates. The outputs of the experts are aggregated by the Aggregating Algorithm (AA) due to Vovk [36].
Algorithm 6 AA + LB-FTRL with Average-Multiplicative-Gradient Optimism
Input: The time horizon  $T$    
1:  $K\coloneqq 1 + \lceil \log_2T\rceil$    
2: for all  $k\in [K]$  do   
3:  $w_{1}^{(k)}\gets 1$    
4:  $q^{(k)}\gets 2^k$    
5:  $\eta^{(k)}\gets \frac{\sqrt{d\log T}}{2\sqrt{2d\log T} + \sqrt{q^{(k)}}}$    
6: Initialize Algorithm 5 with the constant learning rate  $\eta^{(k)}$  as the  $k$  -th expert  $\mathcal{A}_k$    
7: Obtain  $x_{1}^{(k)}$  from  $\mathcal{A}_k$    
8: end for   
9:  $x_{1}\coloneqq \frac{1}{\sum_{k = 1}^{K}w_{1}^{(k)}}\sum_{k = 1}^{K}w_{1}^{(k)}x_{1}^{(k)}$    
10: for all  $t\in [T]$  do   
11: Observe  $a_{t}$    
12: for all  $k\in [K]$  do   
13:  $w_{t + 1}^{(k)} = w_t^{(k)}\langle a_t,x_t^{(k)}\rangle .$    
14: Feed  $a_{t}$  into  $\mathcal{A}_k$  and obtain  $x_{t + 1}^{(k)}$    
15: end for   
16: Output  $x_{t + 1}\coloneqq \frac{1}{\sum_{k = 1}^{K}w_{t + 1}^{(k)}}\sum_{k = 1}^{K}w_{t + 1}^{(k)}x_{t + 1}^{(k)}$    
17: end for
The regret bound of Algorithm 6 is stated in Theorem I.2, which follows from the regret of AA and Theorem I.1.
Theorem 1.2. Algorithm 6 with input  $T \in \mathbb{N}$  satisfies
$$
R _ {T} \leq \left\{ \begin{array}{l l} (1 + \sqrt {2}) \sqrt {d Q _ {T} \log T} + 2 \sqrt {2} d \log T + \frac {1}{\sqrt {2}} \log T + \log (\log_ {2} T + 2) + 3, & i f Q _ {T} \geq 1, \\ 2 \sqrt {2} d \log T + \frac {1}{\sqrt {2}} \log T + \sqrt {2 d \log T} + \log (\log_ {2} T + 2) + 4, & i f Q _ {T} <   1. \end{array} \right.
$$
Remark I.3. Since  $Q_T \leq 2T$ , the regret bound in Theorem I.2 is  $O(\sqrt{dT\log T})$  in the worst case.
Proof.  $K = 1 + \lceil \log_2 T \rceil$  is the number of experts in Algorithm 6. Let  $R_T^{(k)}$  be the regret of the  $k$ -th expert  $\mathcal{A}_k$ . Since the loss functions  $f_t(x) = -\log \langle a_t, x \rangle$  are 1-mixable, by the regret bound of AA [36],
$$
R _ {T} \leq \min _ {k \in [ K ]} R _ {T} ^ {(k)} + \log K,
$$
where  $R_{T}$  and  $R_{T}^{(k)}$  denote the regret of Algorithm 6 and the regret of the  $k$ -th expert, respectively.
The quantity  $q^{(k)}$  in Algorithm 6 is used as an estimate of  $Q_{T}$ . Notice that  $Q_{T} \leq 2T \leq q^{(K)}$ . If  $Q_{T} \geq 1$ , then there exists  $k^{\star} \in [K]$  such that  $q^{(k^{\star} - 1)} \leq Q_{T} \leq q^{(k^{\star})}$ . By Theorem I.1 and some direct calculations,
$$
R _ {T} ^ {(k ^ {\star})} \leq (1 + \sqrt {2}) \sqrt {d Q _ {T} \log T} + 2 \sqrt {2} d \log T + \frac {1}{\sqrt {2}} \log T + 3.
$$
If  $Q_{T} < 1$ , then by Theorem I.1,
$$
R _ {T} ^ {(1)} (x) \leq 2 \sqrt {2} d \log T + \sqrt {2 d \log T} + \frac {1}{\sqrt {2}} \log T + 4.
$$
The theorem follows by combining the inequalities above.
Time Complexity. In each round, Algorithm 6 requires implementing  $O(\log \log T)$  copies of Algorithm 5. By Theorem 5.1, the per-round time is  $\tilde{O}(d\log \log T)$ .