SlowGuess/NeurIPS_2023_OCR huggingface dataset:
where the first step follows the standard analysis of optimistic OMD, e.g., Theorem 1 of Zhao et al. [2021]. One difference of our analysis from the previous one lies in the second step, where previous work dropped the  $\mathcal{D}_{\psi_t}(\pmb{p}_t, \widehat{\pmb{p}}_t)$  term while we keep it to generate the desired negative terms.
To begin with, we require an upper bound of  $\eta_{t,i} \leq 1/32$  for the step sizes. To give a lower bound for TERM (C), we notice that for any  $a, b \in \Delta_d$ ,
$$
\begin{array}{l} \mathcal {D} _ {\psi_ {t}} (\pmb {a}, \pmb {b}) = \sum_ {i = 1} ^ {d} \frac {1}{\eta_ {t , i}} \left(a _ {i} \ln \frac {a _ {i}}{b _ {i}} - a _ {i} + b _ {i}\right) = \sum_ {i = 1} ^ {d} \frac {b _ {i}}{\eta_ {t , i}} \left(\frac {a _ {i}}{b _ {i}} \ln \frac {a _ {i}}{b _ {i}} - \frac {a _ {i}}{b _ {i}} + 1\right) \\ \geq \min  _ {t, i} \frac {1}{\eta_ {t , i}} \sum_ {i = 1} ^ {d} \left(a _ {i} \ln \frac {a _ {i}}{b _ {i}} - a _ {i} + b _ {i}\right) \geq 3 2 \mathrm {K L} (\boldsymbol {a}, \boldsymbol {b}), \quad \left(\text {b y} \eta_ {t, i} \leq^ {1} / 3 2\right) \\ \end{array}
$$
where the first inequality is due to  $x \ln x - x + 1 \geq 0$  for all  $x > 0$ , leading to
$$
\begin{array}{l} \mathrm {T E R M} (\mathrm {C}) \geq 3 2 \sum_ {t = 1} ^ {T} \left(\mathrm {K L} \left(\widehat {\boldsymbol {p}} _ {t + 1}, \boldsymbol {p} _ {t}\right) + \mathrm {K L} \left(\boldsymbol {p} _ {t}, \widehat {\boldsymbol {p}} _ {t}\right)\right) \geq \frac {3 2}{2 \ln 2} \sum_ {t = 1} ^ {T} \left(\| \widehat {\boldsymbol {p}} _ {t + 1} - \boldsymbol {p} _ {t} \| _ {1} ^ {2} + \| \boldsymbol {p} _ {t} - \widehat {\boldsymbol {p}} _ {t} \| _ {1} ^ {2}\right) \\ \geq 1 6 \sum_ {t = 2} ^ {T} \left(\| \widehat {\boldsymbol {p}} _ {t} - \boldsymbol {p} _ {t - 1} \| _ {1} ^ {2} + \| \boldsymbol {p} _ {t} - \widehat {\boldsymbol {p}} _ {t} \| _ {1} ^ {2}\right) \geq 8 \sum_ {t = 2} ^ {T} \| \boldsymbol {p} _ {t} - \boldsymbol {p} _ {t - 1} \| _ {1} ^ {2}, \\ \end{array}
$$
where the first step is from the above derivation, the second step is due to the Pinsker's inequality [Pinsker, 1964]:  $\mathrm{KL}(\pmb {a},\pmb {b})\geq \frac{1}{2\ln 2}\| \pmb {a} - \pmb {b}\| _1^2$  for any  $\pmb {a},\pmb {b}\in \Delta_d$  and the last step is true because of  $2x^{2} + 2y^{2}\geq (x + y)^{2}$  for any  $x,y\in \mathbb{R}$  and  $\| \pmb {a} + \pmb {b}\| \leq \| \pmb {a}\| +\| \pmb {b}\|$  for any  $\pmb {a},\pmb {b}\in \mathbb{R}^d$
For TERM (B), the proof is similar to the previous work, where only some constants are modified. For self-containedness, we give the analysis below. Treating  $\widehat{\pmb{p}}_{t + 1}$  as a free variable and defining
$$
\boldsymbol {p} ^ {\star} \in \underset {\boldsymbol {p}} {\arg \max } \left\langle \boldsymbol {\ell} _ {t} + \boldsymbol {a} _ {t} - \boldsymbol {m} _ {t}, \boldsymbol {p} _ {t} - \boldsymbol {p} \right\rangle - \frac {1}{2} \mathcal {D} _ {\psi_ {t}} (\boldsymbol {p}, \boldsymbol {p} _ {t}),
$$
by the optimality of  $\pmb{p}^{\star}$ , we have
$$
\boldsymbol {\ell} _ {t} + \boldsymbol {a} _ {t} - \boldsymbol {m} _ {t} = \frac {1}{2} (\nabla \psi_ {t} (\boldsymbol {p} _ {t}) - \nabla \psi_ {t} (\boldsymbol {p} ^ {\star})).
$$
Since  $[\nabla \psi_t(\pmb{p})]_i = \frac{1}{\eta_{t,i}} (\ln p_i + 1)$ , it holds that
$$
\ell_ {t, i} - m _ {t, i} + a _ {t, i} = \frac {1}{2 \eta_ {t , i}} \ln \frac {p _ {t , i}}{p _ {i} ^ {\star}} \Leftrightarrow p _ {i} ^ {\star} = p _ {t, i} \exp (- 2 \eta_ {t, i} (\ell_ {t, i} - m _ {t, i} + a _ {t, i})).
$$
$$
\begin{array}{l} \langle \boldsymbol {\ell} _ {t} + \boldsymbol {a} _ {t} - \boldsymbol {m} _ {t}, \boldsymbol {p} _ {t} - \widehat {\boldsymbol {p}} _ {t + 1} \rangle - \frac {1}{2} \mathcal {D} _ {\psi_ {t}} (\widehat {\boldsymbol {p}} _ {t + 1}, \boldsymbol {p} _ {t}) \leq \langle \boldsymbol {\ell} _ {t} + \boldsymbol {a} _ {t} - \boldsymbol {m} _ {t}, \boldsymbol {p} _ {t} - \boldsymbol {p} ^ {\star} \rangle - \frac {1}{2} \mathcal {D} _ {\psi_ {t}} (\boldsymbol {p} ^ {\star}, \boldsymbol {p} _ {t}) \\ = \frac {1}{2} \left\langle \nabla \psi_ {t} \left(\boldsymbol {p} _ {t}\right) - \nabla \psi_ {t} \left(\boldsymbol {p} ^ {\star}\right), \boldsymbol {p} _ {t} - \boldsymbol {p} ^ {\star} \right\rangle - \frac {1}{2} \mathcal {D} _ {\psi_ {t}} \left(\boldsymbol {p} ^ {\star}, \boldsymbol {p} _ {t}\right) = \frac {1}{2} \mathcal {D} _ {\psi_ {t}} \left(\boldsymbol {p} _ {t}, \boldsymbol {p} ^ {\star}\right) \quad (\text {b y}) \\ = \frac {1}{2} \sum_ {i = 1} ^ {d} \frac {1}{\eta_ {t , i}} \left(p _ {t, i} \ln \frac {p _ {t , i}}{p _ {i} ^ {\star}} - p _ {t, i} + p _ {i} ^ {\star}\right) \\ = \frac {1}{2} \sum_ {i = 1} ^ {d} \frac {p _ {t , i}}{\eta_ {t , i}} \left(2 \eta_ {t, i} \left(\ell_ {t, i} - m _ {t, i} + a _ {t, i}\right) - 1 + \exp \left(- 2 \eta_ {t, i} \left(\ell_ {t, i} - m _ {t, i} + a _ {t, i}\right)\right)\right) \\ \leq \frac {1}{2} \sum_ {i = 1} ^ {d} \frac {p _ {t , i}}{\eta_ {t , i}} 4 \eta_ {t, i} ^ {2} (\ell_ {t, i} - m _ {t, i} + a _ {t, i}) ^ {2} = 2 \sum_ {i = 1} ^ {d} \eta_ {t, i} p _ {t, i} (\ell_ {t, i} - m _ {t, i} + a _ {t, i}) ^ {2}, \\ \end{array}
$$
where the first and second steps use the optimality of  $\pmb{p}^{\star}$ , the last inequality uses  $e^{-x} - 1 + x \leq x^{2}$  for all  $x \geq -1$ , requiring  $|2\eta_{t,i}(\ell_{t,i} - m_{t,i} + a_{t,i})| \leq 1$ . It can be satisfied by  $\eta_{t,i} \leq 1/32$  and  $|\ell_{t,i} - m_{t,i} + a_{t,i}| \leq 16$ , where the latter requirement can be satisfied by setting  $a_{t,i} = 16\eta_{t,i}(\ell_{t,i} - m_{t,i})^{2}$ :
$$
\left| \ell_ {t, i} - m _ {t, i} + a _ {t, i} \right| \leq 2 + 1 6 \cdot \frac {1}{3 2} \left(2 \ell_ {t, i} ^ {2} + 2 m _ {t, i} ^ {2}\right) \leq 4 \leq 1 6.
$$
$$
\left(\ell_ {t, i} - m _ {t, i} + a _ {t, i}\right) ^ {2} = \left(\ell_ {t, i} - m _ {t, i} + 1 6 \eta_ {t, i} \left(\ell_ {t, i} - m _ {t, i}\right) ^ {2}\right) ^ {2} \leq 4 \left(\ell_ {t, i} - m _ {t, i}\right) ^ {2},
$$
where the last step holds because  $|\ell_{t,i}|, |m_{t,i}| \leq 1$  and  $\eta_{t,i} \leq 1/32$ . Finally, it holds that
$$
\mathrm {T E R M} (\mathrm {B}) \leq 2 \sum_ {t = 1} ^ {T} \sum_ {i = 1} ^ {d} \eta_ {t, i} p _ {t, i} \left(\ell_ {t, i} - m _ {t, i} + a _ {t, i}\right) ^ {2} \leq 8 \sum_ {t = 1} ^ {T} \sum_ {i = 1} ^ {d} \eta_ {t, i} p _ {t, i} \left(\ell_ {t, i} - m _ {t, i}\right) ^ {2}.
$$
As for TERM (A), following the same argument as Lemma 1 of Chen et al. [2021], we have
$$
\text {T E R M} (\mathrm {A}) \leq \sum_ {i = 1} ^ {d} \frac {1}{\eta_ {1 , i}} f _ {\mathrm {K L}} (u _ {i}, \widehat {p} _ {1, i}) + \sum_ {t = 2} ^ {T} \sum_ {i = 1} ^ {d} \left(\frac {1}{\eta_ {t , i}} - \frac {1}{\eta_ {t - 1 , i}}\right) f _ {\mathrm {K L}} (u _ {i}, \widehat {p} _ {t, i}),
$$
where  $f_{\mathrm{KL}}(a,b) \triangleq a \ln (a / b) - a + b$ . Combining all three terms, we have
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} \langle \boldsymbol {\ell} _ {t} + \boldsymbol {a} _ {t}, \boldsymbol {p} _ {t} - \boldsymbol {u} \rangle \leq \sum_ {i = 1} ^ {d} \frac {1}{\eta_ {1 , i}} f _ {\mathrm {K L}} (u _ {i}, \widehat {p} _ {1, i}) + \sum_ {t = 2} ^ {T} \sum_ {i = 1} ^ {d} \left(\frac {1}{\eta_ {t , i}} - \frac {1}{\eta_ {t - 1 , i}}\right) f _ {\mathrm {K L}} (u _ {i}, \widehat {p} _ {t, i}) \\ + 8 \sum_ {t = 1} ^ {T} \sum_ {i = 1} ^ {d} \eta_ {t, i} p _ {t, i} \left(\ell_ {t, i} - m _ {t, i}\right) ^ {2} - 4 \sum_ {t = 2} ^ {T} \| \boldsymbol {p} _ {t} - \boldsymbol {p} _ {t - 1} \| _ {1} ^ {2}. \\ \end{array}
$$
Moving the correction term  $\sum_{t=1}^{T}\langle\pmb{a}_t,\pmb{p}_t-\pmb{u}\rangle$  to the right-hand side gives:
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} \langle \boldsymbol {\ell} _ {t}, \boldsymbol {p} _ {t} - \boldsymbol {u} \rangle \leq \sum_ {i = 1} ^ {d} \frac {1}{\eta_ {1 , i}} f _ {\mathrm {K L}} (u _ {i}, \widehat {p} _ {1, i}) + \sum_ {t = 2} ^ {T} \sum_ {i = 1} ^ {d} \left(\frac {1}{\eta_ {t , i}} - \frac {1}{\eta_ {t - 1 , i}}\right) f _ {\mathrm {K L}} (u _ {i}, \widehat {p} _ {t, i}) \\ - 8 \sum_ {t = 1} ^ {T} \sum_ {i = 1} ^ {d} \eta_ {t, i} p _ {t, i} \left(\ell_ {t, i} - m _ {t, i}\right) ^ {2} + 1 6 \sum_ {t = 1} ^ {T} \sum_ {i = 1} ^ {d} \eta_ {t, i} u _ {i} \left(\ell_ {t, i} - m _ {t, i}\right) ^ {2} - 4 \sum_ {t = 2} ^ {T} \| \boldsymbol {p} _ {t} - \boldsymbol {p} _ {t - 1} \| _ {1} ^ {2}. \\ \end{array}
$$
Finally, choosing  $\pmb{u} = \pmb{e}_{i^{\star}}$  and  $\eta_{t,i} = \eta_i$  for all  $t\in [T]$  finishes the proof.
In this part, we give a self-contained analysis of the two-layer meta learner, which mainly follows the Theorem 4 and Theorem 5 of Chen et al. [2021], but with additional negative stability terms.
Lemma 3. If  $|\ell_{t,k}|, |m_{t,k}|, |\ell_{t,k,i}|, |m_{t,k,i}| \leq 1$  and  $(\ell_{t,k} - m_{t,k})^2 = \langle \ell_{t,k} - m_{t,k}, p_{t,k} \rangle^2$  for any  $k \in [K]$  and  $i \in [N]$ , then the two-layer MsMwC algorithm satisfies
$$
\sum_ {t = 1} ^ {T} \langle \boldsymbol {\ell} _ {t}, \boldsymbol {q} _ {t} - \boldsymbol {e} _ {k ^ {\star}} \rangle + \sum_ {t = 1} ^ {T} \langle \boldsymbol {\ell} _ {t, k ^ {\star}}, \boldsymbol {p} _ {t, k ^ {\star}} - \boldsymbol {e} _ {i ^ {\star}} \rangle \leq \frac {1}{\eta_ {k ^ {\star}}} \ln \frac {N}{3 C _ {0} ^ {2} \eta_ {k ^ {\star}} ^ {2}} + 3 2 \eta_ {k ^ {\star}} V _ {\star} - \frac {C _ {0}}{2} S _ {T} ^ {\boldsymbol {q}} - \frac {C _ {0}}{4} S _ {T, k ^ {\star}} ^ {\boldsymbol {p}},
$$
where  $V_{\star} \triangleq \sum_{t} (\ell_{t,k^{\star},i^{\star}} - m_{t,k^{\star},i^{\star}})^{2}$ ,  $S_{T}^{q} \triangleq \sum_{t=2}^{T} \| \mathbf{q}_{t} - \mathbf{q}_{t-1} \|_{1}^{2}$  and  $S_{T,k}^{p} \triangleq \sum_{t=2}^{T} \| \mathbf{p}_{t,k} - \mathbf{p}_{t-1,k} \|_{1}^{2}$  measure the stability of MsmwC-TOP and MsmwC-MID.
Note that we leverage a condition of  $(\ell_{t,k} - m_{t,k})^2 = \langle \ell_{t,k} - m_{t,k}, \pmb{p}_{t,k} \rangle^2$  in Lemma 3 only to ensure a self-contained result. When using Lemma 3 (more specifically, in Theorem 2), we will verify that this condition is inherently satisfied by our algorithm.
Proof. Using Lemma 2, the regret of MsMwC-TOP can be bounded as
$$
\sum_ {t = 1} ^ {T} \langle \boldsymbol {\ell} _ {t}, \boldsymbol {q} _ {t} - \boldsymbol {e} _ {k ^ {\star}} \rangle \leq \left(\frac {1}{\eta_ {k ^ {\star}}} \ln \frac {1}{\widehat {q} _ {1 , k ^ {\star}}} + \sum_ {k = 1} ^ {K} \frac {\widehat {q} _ {1 , k}}{\eta_ {k}}\right) + 0 + 1 6 \eta_ {k ^ {\star}} \sum_ {t = 1} ^ {T} \left(\ell_ {t, k ^ {\star}} - m _ {t, k ^ {\star}}\right) ^ {2} - \min  _ {k \in [ K ]} \frac {1}{4 \eta_ {k}} S _ {T} ^ {\boldsymbol {q}},
$$
where the first step comes from  $f_{\mathrm{KL}}(a,b) = a\ln^{a / b} - a + b \leq a\ln^{a / b} + b$  for  $a, b > 0$ . The second term above (corresponding to the second term in Lemma 2) is zero since the step size is time-invariant. The first term above can be further bounded as
$$
\frac {1}{\eta_ {k ^ {\star}}} \ln \frac {1}{\widehat {q} _ {1 , k ^ {\star}}} + \sum_ {k = 1} ^ {K} \frac {\widehat {q} _ {1 , k}}{\eta_ {k}} = \frac {1}{\eta_ {k ^ {\star}}} \ln \frac {\sum_ {k = 1} ^ {K} \eta_ {k} ^ {2}}{\eta_ {k ^ {\star}} ^ {2}} + \frac {\sum_ {k = 1} ^ {K} \eta_ {k}}{\sum_ {k = 1} ^ {K} \eta_ {k} ^ {2}} \leq \frac {1}{\eta_ {k ^ {\star}}} \ln \frac {1}{3 C _ {0} ^ {2} \eta_ {k ^ {\star}} ^ {2}} + 4 C _ {0},
$$
where the first step is due to the initialization of  $\widehat{q}_{1,k} = \eta_k^2 / \sum_{k=1}^{K} \eta_k^2$ . Plugging in the setting of  $\eta_k = 1 / (C_0 \cdot 2^k)$ , the second step holds since
$$
\frac {K ^ {2}}{4 C _ {0} ^ {2}} \leq \sum_ {k = 1} ^ {K} \eta_ {k} ^ {2} \leq \sum_ {k = 1} ^ {K} \frac {1}{C _ {0} ^ {2} \cdot 4 ^ {k}} \leq \frac {1}{3 C _ {0} ^ {2}}, \quad \sum_ {k = 1} ^ {K} \eta_ {k} \leq \frac {1}{C _ {0} \cdot 2 ^ {k}} \leq \frac {1}{C _ {0}}.
$$
Since  $1 / \eta_{k} = C_{0}\cdot 2^{k}\geq 2C_{0}$ , the regret of MsmwC-TOP can be bounded by
$$
\sum_ {t = 1} ^ {T} \langle \boldsymbol {\ell} _ {t}, \boldsymbol {q} _ {t} - \boldsymbol {e} _ {k ^ {\star}} \rangle \leq \frac {1}{\eta_ {k ^ {\star}}} \ln \frac {1}{3 C _ {0} ^ {2} \eta_ {k ^ {\star}} ^ {2}} + 1 6 \eta_ {k ^ {\star}} \sum_ {t = 1} ^ {T} (\ell_ {t, k ^ {\star}} - m _ {t, k ^ {\star}}) ^ {2} - \frac {C _ {0}}{2} S _ {T} ^ {\boldsymbol {q}} + \mathcal {O} (1).
$$
Next, using Lemma 2 again, the regret of the  $k^{\star}$ -th MSMwC-MID, whose step size is  $2\eta_{k^{\star}}$ , can be bounded as
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} \left\langle \boldsymbol {\ell} _ {t, k ^ {\star}}, \boldsymbol {p} _ {t, k ^ {\star}} - \boldsymbol {e} _ {i ^ {\star}} \right\rangle \leq \frac {1}{2 \eta_ {k ^ {\star}}} \ln N + 3 2 \eta_ {k ^ {\star}} \sum_ {t = 1} ^ {T} \left(\left. \boldsymbol {\ell} _ {t, k ^ {\star}, i ^ {\star}} - m _ {t, k ^ {\star}, i ^ {\star}}\right) ^ {2} - \frac {C _ {0}}{4} S _ {T, k ^ {\star}} ^ {\boldsymbol {p}} \right. \\ - 1 6 \eta_ {k ^ {\star}} \sum_ {t = 1} ^ {T} \sum_ {i = 1} ^ {N} p _ {t, k ^ {\star}, i} \left(\ell_ {t, k, i} - m _ {t, k, i}\right) ^ {2}, \\ \end{array}
$$
where the first step is due to the initialization of  $\widehat{p}_{1,k,i} = 1 / N$ . Based on the observation of
$$
\left(\ell_ {t, k ^ {\star}} - m _ {t, k ^ {\star}}\right) ^ {2} = \left\langle \boldsymbol {\ell} _ {t, k ^ {\star}} - \boldsymbol {m} _ {t, k ^ {\star}}, \boldsymbol {p} _ {t, k ^ {\star}} \right\rangle^ {2} \leq \sum_ {i = 1} ^ {N} p _ {t, k ^ {\star}, i} \left(\ell_ {t, k, i} - m _ {t, k, i}\right) ^ {2},
$$
where the last step uses Cauchy-Schwarz inequality, combining the regret of MSMWC-TOP and the  $k^{\star}$ -th MSMWC-MID finishes the proof.
In Appendix C.2, we provide the proof of Theorem 2, the gradient-variation guarantees of the efficient version of Algorithm 1. Therefore, the proof of Theorem 1 can be directly extracted from that of Theorem 2. In the following, we give a proof sketch.
Proof Sketch. The proof of the meta regret is the same as Theorem 2 since the meta learners (top and middle layer) are both optimizing the linearized regret of  $\sum_{t=1}^{T}\langle\nabla f_t(\mathbf{x}_t),\mathbf{x}_t-\mathbf{x}_{t,k^*,i^*}\rangle$ . The only difference lies in the base regret. In Theorem 1, since the base regret is defined on the original function  $f_t$ , the gradient-variation bound of the base learner can be directly obtained via a black-box analysis of the optimistic OMD algorithm, e.g., Theorem B.1 of Zhang et al. [2022a] (strongly convex), Theorem 15 (exp-concave) and Theorem 11 (convex) of Chiang et al. [2012]. The only requirement on the base learners is that they need to cancel the positive term of  $\|\mathbf{x}_{t,k^*,i^*}-\mathbf{x}_{t-1,k^*,i^*}\|^2$  caused by the universal optimism design (Lemma 1) and cascaded correction terms (Figure 1). The above base algorithms satisfy the requirement. The negative stability term in the optimistic OMD analysis is provided in Appendix E.2.
B.5 Proof of Corollary 1
In Appendix C.3, we provide Corollary 2, the small-loss guarantees of the efficient version of Algorithm 1. The proof of Corollary 1 can be directly extracted from that of Corollary 2. We give a proof sketch below.
Proof Sketch. The proof sketch is similar to that of Theorem 1 in Appendix B.4. Specifically, the meta regret can be bounded in the same way. Since gradient-variation bounds naturally imply small-loss ones due to (E.3), the small-loss bound of the base learner can be directly obtained via a black-box analysis of the base algorithms mentioned in Appendix B.4, for different kinds of functions. The requirement of being capable of handling  $\| \mathbf{x}_{t,k^{\star},i^{\star}} - \mathbf{x}_{t - 1,k^{\star},i^{\star}}\|^{2}$  can be still satisfied, which finishes the proof sketch.
In this section, we give the proofs for Section 4, including Proposition 1, Theorem 2 and Corollary 2.
Proof. To handle the meta regret, we use ADAPT-ML-PROD [Gaillard et al., 2014] to optimize the linear loss  $\ell_t = (\ell_{t,1},\dots,\ell_{t,N})$ , where  $\ell_{t,i}\triangleq \langle \nabla f_t(\mathbf{x}_t),\mathbf{x}_{t,i}\rangle$ , and obtain the following second-order bound by Corollary 4 of Gaillard et al. [2014],
$$
\sum_ {t = 1} ^ {T} \langle \nabla f _ {t} (\mathbf {x} _ {t}), \mathbf {x} _ {t} - \mathbf {x} _ {t, i ^ {\star}} \rangle \lesssim \sqrt {\ln \ln T \sum_ {t = 1} ^ {T} \langle \nabla f _ {t} (\mathbf {x} _ {t}) , \mathbf {x} _ {t} - \mathbf {x} _ {t , i ^ {\star}} \rangle^ {2}}.
$$
For  $\alpha$ -exp-concave functions, it holds that
$$
\begin{array}{l} \mathrm {M E T A - R E G} \lesssim \sqrt {\ln \ln T \sum_ {t = 1} ^ {T} \left\langle \nabla f _ {t} \left(\mathbf {x} _ {t}\right) , \mathbf {x} _ {t} - \mathbf {x} _ {t , i ^ {*}} \right\rangle^ {2}} - \frac {\alpha_ {i ^ {*}}}{2} \sum_ {t = 1} ^ {T} \left\langle \nabla f _ {t} \left(\mathbf {x} _ {t}\right), \mathbf {x} _ {t} - \mathbf {x} _ {t , i ^ {*}} \right\rangle^ {2} \\ \leq \frac {\ln \ln T}{2 \alpha_ {i ^ {\star}}} \leq \frac {\ln \ln T}{\alpha}, \quad (\text {b y} \alpha_ {i ^ {\star}} \leq \alpha \leq 2 \alpha_ {i ^ {\star}}) \\ \end{array}
$$
where the second step uses AM-GM inequality:  $\sqrt{xy} \leq \frac{ax}{2} + \frac{y}{2a}$  for any  $x, y, a > 0$  with  $a = \alpha_{i^{\star}}$ . To handle the base regret, by optimizing the surrogate loss function  $h_{t,i^{\star}}^{\exp}$  using online Newton step [Hazan et al., 2007], it holds that
$$
\text {B A S E - R E G} \lesssim \frac {d D G _ {\exp}}{\alpha_ {i ^ {\star}}} \log T \leq \frac {2 d D (G + G D)}{\alpha} \log T,
$$
where  $G_{\mathrm{exp}} \triangleq \max_{\mathbf{x} \in \mathcal{X}, t \in [T], i \in [N]} \| \nabla h_{t,i}^{\mathrm{exp}}(\mathbf{x}) \| \leq G + GD$  represents the maximum gradient norm the last step is because  $\alpha \leq 2\alpha_{i^*}$ . Combining the meta and base regret, the regret can be bounded by  $\mathcal{O}(d\log T)$ . For  $\lambda$ -strongly convex functions, since it is also  $\alpha = \lambda / G^2$  exp-concave under Assumption 1 [Hazan et al., 2007, Section 2.2], the above meta regret analysis is still applicable. To optimize the base regret, by optimizing the surrogate loss function  $h_{t,i^*}^{\mathrm{sc}}$  using online gradient descent [Hazan et al., 2007], it holds that
$$
\text {B A S E - R E G} \leq \frac {G _ {\mathrm {s c}} ^ {2}}{\lambda_ {i ^ {\star}}} (1 + \log T) \leq \frac {2 (G + D) ^ {2}}{\lambda} (1 + \log T),
$$
where  $G_{\mathrm{sc}} \triangleq \max_{\mathbf{x} \in \mathcal{X}, t \in [T], i \in [N]} \| \nabla h_{t,i}^{\mathrm{sc}}(\mathbf{x}) \| \leq G + D$  represents the maximum gradient norm and the last step is because  $\lambda \leq 2\lambda_{i^*}$ . Thus the overall regret can be bounded by  $\mathcal{O}(\log T)$ . For convex functions, the meta regret can be bounded by  $\mathcal{O}(\sqrt{T}\ln \ln T)$ , where the  $\ln \ln T$  factor can be omitted in the  $\mathcal{O}(\cdot)$ -notation, and the base regret can be bounded by  $\mathcal{O}(\sqrt{T})$  using OGD, resulting in an  $\mathcal{O}(\sqrt{T})$  regret overall, which completes the proof.
Proof. We first give different decompositions for the regret, then analyze the meta regret, and finally provide the proofs for different kinds of loss functions. Some abbreviations of the stability terms are defined in (B.1).
Regret Decomposition. Denoting by  $\mathbf{x}^{\star}\in \arg \min_{\mathbf{x}\in \mathcal{X}}\sum_{t = 1}^{T}f_{t}(\mathbf{x})$ , for exp-concave functions,
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} f _ {t} (\mathbf {x} _ {t}) - \sum_ {t = 1} ^ {T} f _ {t} (\mathbf {x} ^ {\star}) \leq \sum_ {t = 1} ^ {T} \langle \nabla f _ {t} (\mathbf {x} _ {t}), \mathbf {x} _ {t} - \mathbf {x} ^ {\star} \rangle - \frac {\alpha}{2} \sum_ {t = 1} ^ {T} \langle \nabla f _ {t} (\mathbf {x} _ {t}), \mathbf {x} _ {t} - \mathbf {x} ^ {\star} \rangle^ {2} \\ \leq \sum_ {t = 1} ^ {T} \left\langle \nabla f _ {t} (\mathbf {x} _ {t}), \mathbf {x} _ {t} - \mathbf {x} ^ {\star} \right\rangle - \frac {\alpha_ {i ^ {\star}}}{2} \sum_ {t = 1} ^ {T} \left\langle \nabla f _ {t} (\mathbf {x} _ {t}), \mathbf {x} _ {t} - \mathbf {x} ^ {\star} \right\rangle^ {2} \quad \left(\text {b y} \alpha_ {i ^ {\star}} \leq \alpha \leq 2 \alpha_ {i ^ {\star}}\right) \\ = \underbrace {\sum_ {t = 1} ^ {T} \langle \nabla f _ {t} (\mathbf {x} _ {t}) , \mathbf {x} _ {t} - \mathbf {x} _ {t , k ^ {\star} , i ^ {\star}} \rangle - \frac {\alpha_ {i ^ {\star}}}{2} \langle \nabla f _ {t} (\mathbf {x} _ {t}) , \mathbf {x} _ {t} - \mathbf {x} _ {t , k ^ {\star} , i ^ {\star}} \rangle^ {2}} _ {\text {M E T A - R E G}} + \underbrace {\sum_ {t = 1} ^ {T} h _ {t , i ^ {\star}} ^ {\exp} (\mathbf {x} _ {t , k ^ {\star} , i ^ {\star}}) - \sum_ {t = 1} ^ {T} h _ {t , i ^ {\star}} ^ {\exp} (\mathbf {x} ^ {\star})} _ {\text {B A S E - R E G}}, \\ \end{array}
$$
where the second step uses the strong exp-concavity and the last step holds by defining surrogate loss functions  $h_{t,i}^{\exp}(\mathbf{x}) \triangleq \langle \nabla f_t(\mathbf{x}_t),\mathbf{x}\rangle +\frac{\alpha_i}{2}\langle \nabla f_t(\mathbf{x}_t),\mathbf{x} - \mathbf{x}_t\rangle^2$ . Similarly, for strongly convex functions, the regret can be upper-bounded by
$$
\underbrace {\sum_ {t = 1} ^ {T} \langle \nabla f _ {t} (\mathbf {x} _ {t}) , \mathbf {x} _ {t} - \mathbf {x} _ {t , k ^ {\star} , i ^ {\star}} \rangle - \frac {\lambda_ {i ^ {\star}}}{2} \| \mathbf {x} _ {t} - \mathbf {x} _ {t , k ^ {\star} , i ^ {\star}} \| ^ {2}} _ {\mathrm {M E T A - R E G}} + \underbrace {\sum_ {t = 1} ^ {T} h _ {t , i ^ {\star}} ^ {\mathrm {s c}} (\mathbf {x} _ {t , k ^ {\star} , i ^ {\star}}) - \sum_ {t = 1} ^ {T} h _ {t , i ^ {\star}} ^ {\mathrm {s c}} (\mathbf {x} ^ {\star})} _ {\mathrm {B A S E - R E G}},
$$
by defining surrogate loss functions  $h_{t,i}^{\mathrm{sc}}(\mathbf{x}) \triangleq \langle \nabla f_t(\mathbf{x}_t),\mathbf{x}\rangle +\frac{\lambda_i}{2}\| \mathbf{x} - \mathbf{x}_t\| ^2$ . For convex functions, the regret can be decomposed as:
$$
\operatorname {R E G} _ {T} \leq \underbrace {\sum_ {t = 1} ^ {T} \langle \nabla f _ {t} (\mathbf {x} _ {t}) , \mathbf {x} _ {t} - \mathbf {x} _ {t , k ^ {\star} , i ^ {\star}} \rangle} _ {\text {M E T A - R E G}} + \underbrace {\sum_ {t = 1} ^ {T} \langle \nabla f _ {t} (\mathbf {x} _ {t}) , \mathbf {x} _ {t , k ^ {\star} , i ^ {\star}} - \mathbf {x} ^ {\star} \rangle} _ {\text {B A S E - R E G}}.
$$
Meta Regret Analysis. For the meta regret, we focus on the linearized term since the negative term by exp-concavity or strong convexity only exists in analysis. Specifically,