SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} \langle \nabla f _ {t} (\mathbf {x} _ {t}), \mathbf {x} _ {t} - \mathbf {x} _ {t, k ^ {\star}, i ^ {\star}} \rangle = \sum_ {t = 1} ^ {T} \langle \nabla f _ {t} (\mathbf {x} _ {t}), \mathbf {x} _ {t} - \mathbf {x} _ {t, k ^ {\star}} \rangle + \sum_ {t = 1} ^ {T} \langle \nabla f _ {t} (\mathbf {x} _ {t}), \mathbf {x} _ {t, k ^ {\star}} - \mathbf {x} _ {t, k ^ {\star}, i ^ {\star}} \rangle \\ = \sum_ {t = 1} ^ {T} \left\langle \boldsymbol {\ell} _ {t}, \boldsymbol {q} _ {t} - \boldsymbol {e} _ {k ^ {\star}} \right\rangle + \sum_ {t = 1} ^ {T} \left\langle \boldsymbol {\ell} _ {t, k ^ {\star}}, \boldsymbol {p} _ {t, k ^ {\star}} - \boldsymbol {e} _ {i ^ {\star}} \right\rangle - \lambda_ {1} \sum_ {t = 1} ^ {T} \sum_ {k = 1} ^ {K} q _ {t, k} \| \mathbf {x} _ {t, k} - \mathbf {x} _ {t - 1, k} \| ^ {2} \\ - \lambda_ {2} \sum_ {t = 1} ^ {T} \sum_ {i = 1} ^ {N} p _ {t, k ^ {*}, i} \| \mathbf {x} _ {t, k ^ {*}, i} - \mathbf {x} _ {t - 1, k ^ {*}, i} \| ^ {2} + \lambda_ {1} S _ {T, k ^ {*}} ^ {\mathbf {x}} + \lambda_ {2} S _ {T, k ^ {*}, i ^ {*}} ^ {\mathbf {x}} \quad (b y \\ \leq \sum_ {t = 1} ^ {T} \left\langle \boldsymbol {\ell} _ {t}, \boldsymbol {q} _ {t} - \boldsymbol {e} _ {k ^ {\star}} \right\rangle + \sum_ {t = 1} ^ {T} \left\langle \boldsymbol {\ell} _ {t, k ^ {\star}}, \boldsymbol {p} _ {t, k ^ {\star}} - \boldsymbol {e} _ {i ^ {\star}} \right\rangle - \lambda_ {1} \sum_ {t = 1} ^ {T} \sum_ {k = 1} ^ {K} q _ {t, k} \| \mathbf {x} _ {t, k} - \mathbf {x} _ {t - 1, k} \| ^ {2} + \lambda_ {2} S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} \\ + \left(2 \lambda_ {1} - \lambda_ {2}\right) \sum_ {t = 1} ^ {T} \sum_ {i = 1} ^ {N} p _ {t, k ^ {\star}, i} \| \mathbf {x} _ {t, k ^ {\star}, i} - \mathbf {x} _ {t - 1, k ^ {\star}, i} \| ^ {2} + 2 \lambda_ {1} D ^ {2} S _ {T, k ^ {\star}} ^ {\boldsymbol {p}} \quad \tag {byLemma6} \\ \leq \sum_ {t = 1} ^ {T} \left\langle \boldsymbol {\ell} _ {t}, \boldsymbol {q} _ {t} - \boldsymbol {e} _ {k ^ {\star}} \right\rangle + \sum_ {t = 1} ^ {T} \left\langle \boldsymbol {\ell} _ {t, k ^ {\star}}, \boldsymbol {p} _ {t, k ^ {\star}} - \boldsymbol {e} _ {i ^ {\star}} \right\rangle - \lambda_ {1} \sum_ {t = 1} ^ {T} \sum_ {k = 1} ^ {K} q _ {t, k} \| \mathbf {x} _ {t, k} - \mathbf {x} _ {t - 1, k} \| ^ {2} \\ + 2 \lambda_ {1} D ^ {2} S _ {T, k ^ {\star}} ^ {\mathbf {p}} + \lambda_ {2} S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}}. \quad \left(\text {r e q u i r i n g} \lambda_ {2} \geq 2 \lambda_ {1}\right) \\ \end{array}
$$
Next, we investigate the regret of the two-layer meta algorithm (i.e., the first two terms above). To begin with, we validate the conditions of Lemma 3. First, since
$$
\left| \ell_ {t, k, i} \right| \leq G D + \lambda_ {2} D ^ {2}, \quad \left| m _ {t, k, i} \right| \leq 2 G D + \lambda_ {2} D ^ {2},
$$
$$
\left| \ell_ {t, k} \right| \leq G D + \lambda_ {1} D ^ {2}, \quad \left| m _ {t, k} \right| \leq 2 G D + (\lambda_ {1} + \lambda_ {2}) D ^ {2},
$$
rescaling the ranges of losses and optimisms to  $[-1, 1]$  only add a constant multiplicative factors on the final result ( $\lambda_1$  and  $\lambda_2$  only consist of constants). Second, by the definition of the losses and the optimisms, it holds that
$$
\begin{array}{l} \left(\ell_ {t, k ^ {*}} - m _ {t, k ^ {*}}\right) ^ {2} = \left(\left\langle \nabla f _ {t} \left(\mathbf {x} _ {t}\right), \mathbf {x} _ {t, k ^ {*}} \right\rangle - \left\langle \widehat {\boldsymbol {m}} _ {t, k ^ {*}}, \boldsymbol {p} _ {t, k ^ {*}} \right\rangle\right) ^ {2} \\ = \left\langle \widehat {\ell} _ {t, k ^ {*}} - \widehat {\boldsymbol {m}} _ {t, k ^ {*}}, \boldsymbol {p} _ {t, k ^ {*}} \right\rangle^ {2} = \left\langle \ell_ {t, k ^ {*}} - \boldsymbol {m} _ {t, k ^ {*}}, \boldsymbol {p} _ {t, k ^ {*}} \right\rangle^ {2} \\ \end{array}
$$
where  $\widehat{\ell}_{t,k,i} \triangleq \langle \nabla f_t(\mathbf{x}_t), \mathbf{x}_{t,k,i} \rangle$ . Using Lemma 3, it holds that
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} \left\langle \nabla f _ {t} \left(\mathbf {x} _ {t}\right), \mathbf {x} _ {t} - \mathbf {x} _ {t, k ^ {\star}, i ^ {\star}} \right\rangle \leq \frac {1}{\eta_ {k ^ {\star}}} \ln \frac {N}{\eta_ {k ^ {\star}} ^ {2}} + 3 2 \eta_ {k ^ {\star}} V _ {\star} + \lambda_ {2} S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} \quad (\text {r e q u i r i n g} C _ {0} \geq 1) \\ - \frac {C _ {0}}{2} S _ {T} ^ {\boldsymbol {q}} - \lambda_ {1} \sum_ {t = 2} ^ {T} \sum_ {k = 1} ^ {K} q _ {t, k} \| \mathbf {x} _ {t, k} - \mathbf {x} _ {t - 1, k} \| ^ {2} + \left(2 \lambda_ {1} D ^ {2} - \frac {C _ {0}}{4}\right) S _ {T, k \star} ^ {\boldsymbol {p}} \\ \leq \frac {1}{\eta_ {k ^ {\star}}} \ln \frac {N}{\eta_ {k ^ {\star}} ^ {2}} + 3 2 \eta_ {k ^ {\star}} V _ {\star} + \lambda_ {2} S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} - \frac {C _ {0}}{2} S _ {T} ^ {\mathbf {q}} - \lambda_ {1} \sum_ {t = 2} ^ {T} \sum_ {k = 1} ^ {K} q _ {t, k} \| \mathbf {x} _ {t, k} - \mathbf {x} _ {t - 1, k} \| ^ {2}, \\ \end{array}
$$
where the last step holds by requiring  $C_0 \geq 4\lambda_1D^2$ .
Exp-concave Functions. Using Lemma 1, it holds that
$$
\frac {1}{\eta_ {k ^ {\star}}} \ln \frac {N}{\eta_ {k ^ {\star}} ^ {2}} + 3 2 \eta_ {k ^ {\star}} V _ {\star} \leq \frac {1}{\eta_ {k ^ {\star}}} \ln \frac {N}{\eta_ {k ^ {\star}} ^ {2}} + 1 2 8 \eta_ {k ^ {\star}} \sum_ {t = 1} ^ {T} \left\langle \nabla f _ {t} (\mathbf {x} _ {t}), \mathbf {x} _ {t, k ^ {\star}, i ^ {\star}} - \mathbf {x} _ {t} \right\rangle^ {2} + \mathcal {O} (1).
$$
As a result, the meta regret can be bounded by
$$
\begin{array}{l} \mathbf {M E T A - R E G} \leq \frac {1}{\eta_ {k ^ {\star}}} \ln \frac {N}{\eta_ {k ^ {\star}} ^ {2}} + \left(1 2 8 \eta_ {k ^ {\star}} - \frac {\alpha_ {i ^ {\star}}}{2}\right) \sum_ {t = 1} ^ {T} \langle \nabla f _ {t} (\mathbf {x} _ {t}), \mathbf {x} _ {t, k ^ {\star}, i ^ {\star}} - \mathbf {x} _ {t} \rangle^ {2} + \lambda_ {2} S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} \\ - \frac {C _ {0}}{2} S _ {T} ^ {\boldsymbol {q}} - \lambda_ {1} \sum_ {t = 2} ^ {T} \sum_ {k = 1} ^ {K} q _ {t, k} \| \mathbf {x} _ {t, k} - \mathbf {x} _ {t - 1, k} \| ^ {2} \\ \leq 2 C _ {0} \ln \left(4 C _ {0} ^ {2} N\right) + \frac {5 1 2}{\alpha} \ln \frac {2 ^ {2 0} N}{\alpha^ {2}} + \lambda_ {2} S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} - \frac {C _ {0}}{2} S _ {T} ^ {\mathbf {q}} \quad (\text {b y}) \\ - \lambda_ {1} \sum_ {t = 2} ^ {T} \sum_ {k = 1} ^ {K} q _ {t, k} \| \mathbf {x} _ {t, k} - \mathbf {x} _ {t - 1, k} \| ^ {2}, \tag {C.1} \\ \end{array}
$$
where the last step holds by  $\alpha_{i^{\star}}\leq \alpha \leq 2\alpha_{i^{\star}}$  . For the base regret, according to Lemma 11, the  $i^{\star}$  -th base learner guarantees
$$
\sum_ {t = 1} ^ {T} h _ {t, i ^ {\star}} ^ {\exp} \left(\mathbf {x} _ {t, k ^ {\star}, i ^ {\star}}\right) - \sum_ {t = 1} ^ {T} h _ {t, i ^ {\star}} ^ {\exp} \left(\mathbf {x} ^ {\star}\right) \leq \frac {1 6 d}{\alpha_ {i ^ {\star}}} \ln \left(1 + \frac {\alpha_ {i ^ {\star}}}{8 \gamma d} \bar {V} _ {T, k ^ {\star}, i ^ {\star}}\right) + \frac {1}{2} \gamma D ^ {2} - \frac {\gamma}{4} S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}}, \tag {C.2}
$$
where  $\bar{V}_{T,k^{\star},i^{\star}} = \sum_{t=2}^{T}\|\nabla h_{t,i^{\star}}^{\exp}(\mathbf{x}_{t,k^{\star},i^{\star}}) - \nabla h_{t-1,i^{\star}}^{\exp}(\mathbf{x}_{t-1,k^{\star},i^{\star}})\|^2$  denotes the empirical gradient variation of the  $i^{\star}$ -th base learner. For simplicity, we denote by  $\mathbf{g}_t \triangleq \nabla f_t(\mathbf{x}_t)$ , and this term can be further decomposed and bounded as:
$$
\begin{array}{l} \bar {V} _ {T, k ^ {\star}, i ^ {\star}} = \sum_ {t = 2} ^ {T} \| (\mathbf {g} _ {t} + \alpha_ {i ^ {\star}} \mathbf {g} _ {t} \mathbf {g} _ {t} ^ {\top} (\mathbf {x} _ {t, k ^ {\star}, i ^ {\star}} - \mathbf {x} _ {t})) - (\mathbf {g} _ {t - 1} + \alpha_ {i ^ {\star}} \mathbf {g} _ {t - 1} \mathbf {g} _ {t - 1} ^ {\top} (\mathbf {x} _ {t - 1, k ^ {\star}, i ^ {\star}} - \mathbf {x} _ {t - 1})) \| ^ {2} \\ \leq 2 \sum_ {t = 2} ^ {T} \| \mathbf {g} _ {t} - \mathbf {g} _ {t - 1} \| ^ {2} + 2 \alpha_ {i ^ {\star}} ^ {2} \sum_ {t = 2} ^ {T} \| \mathbf {g} _ {t} \mathbf {g} _ {t} ^ {\top} (\mathbf {x} _ {t, k ^ {\star}, i ^ {\star}} - \mathbf {x} _ {t}) - \mathbf {g} _ {t - 1} \mathbf {g} _ {t - 1} ^ {\top} (\mathbf {x} _ {t - 1, k ^ {\star}, i ^ {\star}} - \mathbf {x} _ {t - 1}) \| ^ {2} \\ \leq 4 V _ {T} + 4 L ^ {2} S _ {T} ^ {\mathbf {x}} + 4 D ^ {2} \sum_ {t = 2} ^ {T} \left\| \mathbf {g} _ {t} \mathbf {g} _ {t} ^ {\top} - \mathbf {g} _ {t - 1} \mathbf {g} _ {t - 1} ^ {\top} \right\| ^ {2} \tag {by(E.2)} \\ \end{array}
$$
$$
\begin{array}{l} + 4 G ^ {4} \sum_ {t = 2} ^ {T} \| \left(\mathbf {x} _ {t, k ^ {\star}, i ^ {\star}} - \mathbf {x} _ {t}\right) - \left(\mathbf {x} _ {t - 1, k ^ {\star}, i ^ {\star}} - \mathbf {x} _ {t - 1}\right) \| ^ {2} \quad (\text {b y} \alpha \in [ 1 / T, 1 ]) \\ \leq C _ {1} V _ {T} + C _ {2} S _ {T} ^ {\mathbf {x}} + 8 G ^ {4} S _ {T, k ^ {*}, i ^ {*}} ^ {\mathbf {x}}, \\ \end{array}
$$
where the first step uses the definition of  $\nabla h_{t,i}^{\exp}(\mathbf{x}) = \mathbf{g}_t + \alpha_i\mathbf{g}_t\mathbf{g}_t^\top (\mathbf{x} - \mathbf{x}_t)$  and the last step holds by setting  $C_1 = 4 + 32D^2 G^2$  and  $C_2 = 4L^2 +32D^2 G^2 L^2 +8G^4$ . Then we obtain
$$
\begin{array}{l} \ln \left(1 + \frac {\alpha_ {i ^ {\star}}}{8 \gamma d} \bar {V} _ {T, k ^ {\star}, i ^ {\star}}\right) \leq \ln \left(1 + \frac {\alpha_ {i ^ {\star}} C _ {1}}{8 \gamma d} V _ {T} + \frac {\alpha_ {i ^ {\star}} C _ {2}}{8 \gamma d} S _ {T} ^ {\mathbf {x}} + \frac {\alpha_ {i ^ {\star}} G ^ {4}}{\gamma d} S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}}\right) \\ = \ln \left(1 + \frac {\alpha_ {i ^ {\star}} C _ {1}}{8 \gamma d} V _ {T}\right) + \ln \left(1 + \frac {\frac {\alpha_ {i ^ {\star}} C _ {2}}{8 \gamma d} S _ {T} ^ {\mathbf {x}} + \frac {\alpha_ {i ^ {\star}} G ^ {4}}{\gamma d} S _ {T , k ^ {\star} , i ^ {\star}} ^ {\mathbf {x}}}{1 + \frac {\alpha_ {i ^ {\star}} C _ {1}}{8 \gamma d} V _ {T}}\right) \\ \leq \ln \left(1 + \frac {\alpha_ {i ^ {\star}} C _ {1}}{8 \gamma d} V _ {T}\right) + \frac {\alpha_ {i ^ {\star}} C _ {2}}{8 \gamma d} S _ {T} ^ {\mathbf {x}} + \frac {\alpha_ {i ^ {\star}} G ^ {4}}{\gamma d} S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}}. \quad (\ln (1 + x) \leq x \text {f o r} x \geq 0) \\ \end{array}
$$
Plugging the above result and due to  $\alpha_{i^{\star}}\leq \alpha \leq 2\alpha_{i^{\star}}$  , the base regret can be bounded by
$$
\text {B A S E - R E G} \leq \frac {3 2 d}{\alpha} \ln \left(1 + \frac {\alpha C _ {1}}{8 \gamma d} V _ {T}\right) + \frac {2 C _ {2}}{\gamma} S _ {T} ^ {\mathbf {x}} + \left(\frac {1 6 G ^ {4}}{\gamma} - \frac {\gamma}{4}\right) S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} + \frac {1}{2} \gamma D ^ {2}.
$$
Combining the meta regret and base regret, the overall regret can be bounded by
$$
\begin{array}{l} \operatorname {R E G} _ {T} \leq \frac {5 1 2}{\alpha} \ln \frac {2 ^ {2 0} N}{\alpha^ {2}} + 2 C _ {0} \ln \left(4 C _ {0} ^ {2} N\right) + \frac {3 2 d}{\alpha} \ln \left(1 + \frac {\alpha C _ {1}}{8 \gamma d} V _ {T}\right) \\ + \left(\frac {4 C _ {2} D ^ {2}}{\gamma} - \frac {C _ {0}}{2}\right) S _ {T} ^ {\boldsymbol {q}} + \left(\frac {4 C _ {2}}{\gamma} - \lambda_ {1}\right) \sum_ {t = 2} ^ {T} \sum_ {k = 1} ^ {K} q _ {t, k} \| \mathbf {x} _ {t, k} - \mathbf {x} _ {t - 1, k} \| ^ {2} \\ + \left(\lambda_ {2} + \frac {1 6 G ^ {4}}{\gamma} - \frac {\gamma}{4}\right) S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} + \frac {1}{2} \gamma D ^ {2} \\ \leq \mathcal {O} \left(\frac {d}{\alpha} \ln V _ {T}\right). \quad \text {(r e q u i r i n g} C _ {0} \geq 8 C _ {2} D ^ {2} / \gamma , \lambda_ {1} \geq 4 C _ {2} / \gamma , \gamma \geq 8 G ^ {2} + \lambda_ {2}) \\ \end{array}
$$
Strongly Convex Functions. Since a  $\lambda$ -strongly convex function is also  $\alpha = \lambda / G^2$  exp-concave under Assumption 1 [Hazan et al., 2007, Section 2.2], plugging  $\alpha = \lambda / G^2$  into the above analysis, the meta regret can be bounded by
$$
\begin{array}{l} \mathrm {M E T A - R E G} \leq \frac {5 1 2 G ^ {2}}{\lambda} \ln \frac {2 ^ {2 0} N G ^ {4}}{\lambda^ {2}} + 2 C _ {0} \ln \left(4 C _ {0} ^ {2} N\right) + \lambda_ {2} S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} \\ - \frac {C _ {0}}{2} S _ {T} ^ {\boldsymbol {q}} - \lambda_ {1} \sum_ {t = 2} ^ {T} \sum_ {k = 1} ^ {K} q _ {t, k} \| \mathbf {x} _ {t, k} - \mathbf {x} _ {t - 1, k} \| ^ {2}. \tag {C.3} \\ \end{array}
$$
For the base regret, according to Lemma 12, the  $i^{\star}$ -th base learner guarantees
$$
\sum_ {t = 1} ^ {T} h _ {t, i ^ {\star}} ^ {\mathrm {s c}} \left(\mathbf {x} _ {t, k ^ {\star}, i ^ {\star}}\right) - \sum_ {t = 1} ^ {T} h _ {t, i ^ {\star}} ^ {\mathrm {s c}} \left(\mathbf {x} ^ {\star}\right) \leq \frac {1 6 G _ {\mathrm {s c}} ^ {2}}{\lambda_ {i ^ {\star}}} \ln \left(1 + \lambda_ {i ^ {\star}} \bar {V} _ {T, k ^ {\star}, i ^ {\star}}\right) + \frac {1}{4} \gamma D ^ {2} - \frac {\gamma}{8} S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}}, \tag {C.4}
$$
where  $\bar{V}_{T,k^{\star},i^{\star}} = \sum_{t=2}^{T}\|\nabla h_{t,i^{\star}}^{\mathrm{sc}}(\mathbf{x}_{t,k^{\star},i^{\star}}) - \nabla h_{t-1,i^{\star}}^{\mathrm{sc}}(\mathbf{x}_{t-1,k^{\star},i^{\star}})\|^{2}$  is the empirical gradient variation and  $G_{\mathrm{sc}} \triangleq \max_{\mathbf{x} \in \mathcal{X}, t \in [T], i \in [N]}\|\nabla h_{t,i}^{\mathrm{sc}}(\mathbf{x})\| \leq G + D$  represents the maximum gradient norm.
For simplicity, we denote by  $\mathbf{g}_t\triangleq \nabla f_t(\mathbf{x}_t)$ , and the empirical gradient variation  $\bar{V}_{T,k^{\star},i^{\star}}$  can be further decomposed as
$$
\begin{array}{l} \bar {V} _ {T, k ^ {\star}, i ^ {\star}} = \sum_ {t = 2} ^ {T} \left\| \left(\mathbf {g} _ {t} + \lambda_ {i ^ {\star}} \left(\mathbf {x} _ {t, k ^ {\star}, i ^ {\star}} - \mathbf {x} _ {t}\right)\right) - \left(\mathbf {g} _ {t - 1} + \lambda_ {i ^ {\star}} \left(\mathbf {x} _ {t - 1, k ^ {\star}, i ^ {\star}} - \mathbf {x} _ {t - 1}\right)\right) \right\| ^ {2} \\ \leq 2 \sum_ {t = 2} ^ {T} \| \mathbf {g} _ {t} - \mathbf {g} _ {t - 1} \| ^ {2} + 2 \lambda_ {i ^ {\star}} ^ {2} \sum_ {t = 2} ^ {T} \| (\mathbf {x} _ {t, k ^ {\star}, i ^ {\star}} - \mathbf {x} _ {t}) - (\mathbf {x} _ {t - 1, k ^ {\star}, i ^ {\star}} - \mathbf {x} _ {t - 1}) \| ^ {2} \\ \end{array}
$$
$$
\leq 4 V _ {T} + \left(4 + 4 L ^ {2}\right) S _ {T} ^ {\mathbf {x}} + 4 S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}}, \quad (\text {b y} \lambda \in [ 1 / T, 1 ])
$$
where the first step uses the definition of  $\nabla h^{\mathrm{sc}}(\mathbf{x}) = \mathbf{g}_t + \lambda_i(\mathbf{x} - \mathbf{x}_t)$ . Consequently,
$$
\begin{array}{l} \ln \left(1 + \lambda_ {i ^ {\star}} \bar {V} _ {T, k ^ {\star}, i ^ {\star}}\right) \leq \ln (1 + 4 \lambda_ {i ^ {\star}} V _ {T} + (4 + 4 L ^ {2}) \lambda_ {i ^ {\star}} S _ {T} ^ {\mathbf {x}} + 4 \lambda_ {i ^ {\star}} S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}}) \\ = \ln (1 + 4 \lambda_ {i ^ {\star}} V _ {T}) + \ln \left(1 + \frac {(4 + 4 L ^ {2}) \lambda_ {i ^ {\star}} S _ {T} ^ {\mathbf {x}} + 4 \lambda_ {i ^ {\star}} S _ {T , k ^ {\star} , i ^ {\star}} ^ {\mathbf {x}}}{1 + 4 \lambda_ {i ^ {\star}} V _ {T}}\right) \\ \leq \ln \left(1 + 4 \lambda_ {i ^ {*}} V _ {T}\right) + \left(4 + 4 L ^ {2}\right) \lambda_ {i ^ {*}} S _ {T} ^ {\mathbf {x}} + 4 \lambda_ {i ^ {*}} S _ {T, k ^ {*}, i ^ {*}} ^ {\mathbf {x}}. \quad (\ln (1 + x) \leq x \text {f o r} x \geq 0) \\ \end{array}
$$
Plugging the above result and due to  $\lambda_{i^{\star}}\leq \lambda \leq 2\lambda_{i^{\star}}$  , the base regret can be bounded by
$$
\mathrm {B A S E - R E G} \leq \frac {3 2 G _ {\mathrm {s c}} ^ {2}}{\lambda} \ln (1 + 4 \lambda V _ {T}) + 1 6 C _ {3} S _ {T} ^ {\mathbf {x}} + \left(6 4 G _ {\mathrm {s c}} ^ {2} - \frac {\gamma}{8}\right) S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} + \frac {1}{4} \gamma D ^ {2},
$$
where  $C_3 = (4 + 4L^2)G_{\mathrm{sc}}^2$ . Combining the meta regret and base regret, the overall regret satisfies
$$
\begin{array}{l} \mathrm {R E G} _ {T} \leq \frac {5 1 2 G ^ {2}}{\lambda} \ln \frac {2 ^ {2 0} N G ^ {4}}{\lambda^ {2}} + 2 C _ {0} \ln (4 C _ {0} ^ {2} N) + \frac {3 2 (G + D) ^ {2}}{\lambda} \ln (1 + 4 \lambda V _ {T}) \\ + \left(3 2 D ^ {2} C _ {3} - \frac {C _ {0}}{2}\right) S _ {T} ^ {\boldsymbol {q}} + \left(3 2 C _ {3} - \lambda_ {1}\right) \sum_ {t = 2} ^ {T} \sum_ {k = 1} ^ {K} q _ {t, k} \left\| \mathbf {x} _ {t, k} - \mathbf {x} _ {t - 1, k} \right\| ^ {2} \\ + \left(\lambda_ {2} + 6 4 G _ {\mathrm {s c}} ^ {2} - \frac {\gamma}{8}\right) S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} + \frac {1}{4} \gamma D ^ {2} \\ \leq \mathcal {O} \left(\frac {1}{\lambda} \ln V _ {T}\right). \quad \text {(r e q u i r i n g} C _ {0} \geq 6 4 D ^ {2} C _ {3}, \lambda_ {1} \geq 3 2 C _ {3}, \gamma \geq 8 \lambda_ {2} + 5 1 2 G _ {\mathrm {s c}} ^ {2}) \\ \end{array}
$$
Convex Functions. Using Lemma 1, it holds that
$$
\begin{array}{l} \frac {1}{\eta_ {k ^ {\star}}} \ln \frac {N}{\eta_ {k ^ {\star}} ^ {2}} + 3 2 \eta_ {k ^ {\star}} V _ {\star} \leq \frac {1}{\eta_ {k ^ {\star}}} \ln \frac {N}{\eta_ {k ^ {\star}} ^ {2}} + 1 2 8 \eta_ {k ^ {\star}} D ^ {2} V _ {T} + 2 (D ^ {2} L ^ {2} + G ^ {2}) S _ {T} ^ {\mathbf {x}} + 2 G ^ {2} S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} \\ \leq 2 C _ {0} \ln (4 C _ {0} ^ {2} N) + 3 2 D \sqrt {2 V _ {T} \ln (5 1 2 N D ^ {2} V _ {T})} + 2 (D ^ {2} L ^ {2} + G ^ {2}) S _ {T} ^ {\mathbf {x}} + 2 G ^ {2} S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} \\ \end{array}
$$
where the first step is due to  $\eta_{k} = 1 / (C_{0}\cdot 2^{k})\leq 1 / (2C_{0})$  and requiring  $C_0\geq 1$  and the last step is by Lemma 7 and requiring  $C_0\geq 8D$ . Thus the meta regret can be bounded by
$$
\begin{array}{l} \mathrm {M E T A - R E G} \leq 2 C _ {0} \ln \left(4 C _ {0} ^ {2} N\right) + \mathcal {O} \left(\sqrt {V _ {T} \ln V _ {T}}\right) + 2 \left(D ^ {2} L ^ {2} + G ^ {2}\right) S _ {T} ^ {\mathbf {x}} \\ + \left(\lambda_ {2} + 2 G ^ {2}\right) S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} - \frac {C _ {0}}{2} S _ {T} ^ {\boldsymbol {q}} - \lambda_ {1} \sum_ {t = 2} ^ {T} \sum_ {k = 1} ^ {K} q _ {t, k} \left\| \mathbf {x} _ {t, k} - \mathbf {x} _ {t - 1, k} \right\| ^ {2} \\ \end{array}
$$
For the base regret, according to Lemma 10, the convex base learner guarantees
$$
\text {B A S E - R E G} = \sum_ {t = 1} ^ {T} \left\langle \nabla f _ {t} \left(\mathbf {x} _ {t}, \mathbf {x} _ {t, k ^ {\star}, i ^ {\star}} - \mathbf {x} ^ {\star} \right\rangle \leq 5 D \sqrt {1 + \bar {V} _ {T , k ^ {\star} , i ^ {\star}}} + \gamma D ^ {2} - \frac {\gamma}{4} S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}}, \right. \tag {C.5}
$$
where  $\bar{V}_{T,k^{\star},i^{\star}} = \sum_{t = 2}^{T}\| \nabla f_{t}(\mathbf{x}_{t}) - \nabla f_{t - 1}(\mathbf{x}_{t - 1})\|^{2}$  denotes the empirical gradient variation of the base learner. Via (E.2), the base regret can be bounded by
$$
\mathrm {B A S E - R E G} \leq 5 D \sqrt {1 + 2 V _ {T}} + 1 0 D L ^ {2} S _ {T} ^ {\mathbf {x}} + \gamma D ^ {2} - \frac {\gamma}{4} S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}}.
$$
$$
\begin{array}{l} \operatorname {R E G} _ {T} \leq 2 C _ {0} \ln \left(4 C _ {0} ^ {2} N\right) + \mathcal {O} \left(\sqrt {V _ {T} \ln V _ {T}}\right) + 5 D \sqrt {1 + 2 V _ {T}} \\ + \left(2 D C _ {4} - \frac {C _ {0}}{2}\right) S _ {T} ^ {\boldsymbol {q}} + \left(2 C _ {4} - \lambda_ {1}\right) \sum_ {t = 2} ^ {T} \sum_ {k = 1} ^ {K} q _ {t, k} \left\| \mathbf {x} _ {t, k} - \mathbf {x} _ {t - 1, k} \right\| ^ {2} \\ + \left(\lambda_ {2} + 2 G ^ {2} - \frac {\gamma}{4}\right) S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} + \gamma D ^ {2} \\ \end{array}
$$
$$
\leq \mathcal {O} \left(\sqrt {V _ {T} \ln V _ {T}}\right), \quad \text {(r e q u i r i n g} C _ {0} \geq 4 D C _ {4}, \lambda_ {1} \geq 2 C _ {4}, \gamma \geq 4 \lambda_ {2} + 8 G ^ {2})
$$
where  $C_4 = 2D^2 L^2 + 2G^2 + 10DL^2$ .
Overall. At last, we determine the specific values of  $C_0$ ,  $\lambda_1$  and  $\lambda_2$ . These parameters need to satisfy the following requirements:
$$
C _ {0} \geq 1, C _ {0} \geq 4 \lambda_ {1} D ^ {2}, C _ {0} \geq 1 2 8, C _ {0} \geq \frac {8 C _ {2} D ^ {2}}{8 G ^ {2} + \lambda_ {2}}, C _ {0} \geq 6 4 D ^ {2} C _ {3}, C _ {0} \geq 8 D, C _ {0} \geq 4 D C _ {4},
$$
$$
\lambda_ {1} \geq \frac {4 C _ {2}}{8 G ^ {2} + \lambda_ {2}}, \lambda_ {1} \geq 3 2 C _ {3}, \lambda_ {1} \geq 2 C _ {4}, \lambda_ {2} \geq 2 \lambda_ {1}.
$$
As a result, we set  $C_0, \lambda_1, \lambda_2$  to be the minimum constants satisfying the above conditions., which completes the proof.
In this part, we provide Corollary 2, which obtains the same small-loss bounds as Corollary 1, but only requires one gradient query within each round.
Corollary 2. Under Assumptions 1 and 2, if  $f_{t}(\cdot) \geq 0$  for all  $t \in [T]$ , the efficient version of Algorithm 1 obtains  $\mathcal{O}(\log F_T)$ ,  $\mathcal{O}(d\log F_T)$  and  $\hat{\mathcal{O}} (\sqrt{F_T})$  regret bounds for strongly convex, ex-convex and convex functions, using only one gradient per round.
Proof. For simplicity, we define  $F_T^{\mathbf{x}} \triangleq \sum_{t=1}^{T} f_t(\mathbf{x}_t)$ . Some abbreviations are given in (B.1).
Exp-concave Functions. The meta regret can be bounded the same as Theorem 1 (especially (C.1)), and thus we directly move on to the base regret. For simplicity, we denote by  $\mathbf{g}_t\triangleq \nabla f_t(\mathbf{x}_t)$  and give a different decomposition for  $\bar{V}_{T,k^{\star},i^{\star}} = \sum_{t = 2}^{T}\| \nabla h_{t,i^{\star}}^{\exp}(\mathbf{x}_{t,k^{\star},i^{\star}}) - \nabla h_{t - 1,i^{\star}}^{\exp}(\mathbf{x}_{t - 1,k^{\star},i^{\star}})\| ^2$  , the empirical gradient variation of the base learner.
$$
\begin{array}{l} \bar {V} _ {T, k ^ {\star}, i ^ {\star}} = \sum_ {t = 2} ^ {T} \| (\mathbf {g} _ {t} + \alpha_ {i ^ {\star}} \mathbf {g} _ {t} \mathbf {g} _ {t} ^ {\top} (\mathbf {x} _ {t, k ^ {\star}, i ^ {\star}} - \mathbf {x} _ {t})) - (\mathbf {g} _ {t - 1} + \alpha_ {i ^ {\star}} \mathbf {g} _ {t - 1} \mathbf {g} _ {t - 1} ^ {\top} (\mathbf {x} _ {t - 1, k ^ {\star}, i ^ {\star}} - \mathbf {x} _ {t - 1})) \| ^ {2} \\ \leq 2 \sum_ {t = 2} ^ {T} \| \mathbf {g} _ {t} - \mathbf {g} _ {t - 1} \| ^ {2} + 2 \alpha_ {i ^ {\star}} ^ {2} \sum_ {t = 2} ^ {T} \| \mathbf {g} _ {t} \mathbf {g} _ {t} ^ {\top} (\mathbf {x} _ {t, k ^ {\star}, i ^ {\star}} - \mathbf {x} _ {t}) - \mathbf {g} _ {t - 1} \mathbf {g} _ {t - 1} ^ {\top} (\mathbf {x} _ {t - 1, k ^ {\star}, i ^ {\star}} - \mathbf {x} _ {t - 1}) \| ^ {2} \\ \leq 2 \sum_ {t = 2} ^ {T} \| \mathbf {g} _ {t} - \mathbf {g} _ {t - 1} \| ^ {2} + 4 D ^ {2} \sum_ {t = 2} ^ {T} \| \mathbf {g} _ {t} \mathbf {g} _ {t} ^ {\top} - \mathbf {g} _ {t - 1} \mathbf {g} _ {t - 1} ^ {\top} \| ^ {2} + 8 G ^ {4} S _ {T} ^ {\mathbf {x}} + 8 G ^ {4} S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} \\ \leq \left(2 + 1 6 D ^ {2} G ^ {2}\right) \sum_ {t = 2} ^ {T} \left\| \mathbf {g} _ {t} - \mathbf {g} _ {t - 1} \right\| ^ {2} + 8 G ^ {4} S _ {T} ^ {\mathbf {x}} + 8 G ^ {4} S _ {T, k ^ {*}, i ^ {*}} ^ {\mathbf {x}} \\ \leq C _ {5} F _ {T} ^ {\mathbf {x}} + 8 G ^ {4} S _ {T} ^ {\mathbf {x}} + 8 G ^ {4} S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}}, \tag {by(E.3)} \\ \end{array}
$$
where the first step by using the definition of  $\nabla h_{t,i}^{\exp}(\mathbf{x}) = \mathbf{g}_t + \alpha_i\mathbf{g}_t\mathbf{g}_t^\top (\mathbf{x} - \mathbf{x}_t)$ , the third step uses the assumption that  $\alpha \in [1 / T,1]$  without loss of generality, and the last step sets  $C_5 = 16L(2 + 16D^2 G^2)$ . Consequently, it holds that
$$
\ln \left(1 + \frac {\alpha_ {i ^ {\star}}}{8 \gamma d} \bar {V} _ {T, k ^ {\star}, i ^ {\star}}\right) \leq \ln \left(1 + \frac {\alpha_ {i ^ {\star}} C _ {5}}{8 \gamma d} F _ {T} ^ {\mathbf {x}}\right) + \frac {\alpha_ {i ^ {\star}} G ^ {4}}{\gamma d} S _ {T} ^ {\mathbf {x}} + \frac {\alpha_ {i ^ {\star}} G ^ {4}}{\gamma d} S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}}.
$$
Plugging the above result and due to  $\alpha_{i^{\star}}\leq \alpha \leq 2\alpha_{i^{*}}$  , the base regret can be bounded by