SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\mathrm {B A S E - R E G} \leq \frac {3 2 d}{\alpha} \ln \left(1 + \frac {\alpha C _ {5}}{8 \gamma d} F _ {T} ^ {\mathbf {x}}\right) + \frac {1 6 G ^ {4}}{\gamma} S _ {T} ^ {\mathbf {x}} + \left(\frac {1 6 G ^ {4}}{\gamma} - \frac {\gamma}{4}\right) S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} + \frac {1}{2} \gamma D ^ {2}.
$$
Combining the meta regret (C.1) and base regret, the overall regret can be bounded by
$$
\operatorname {R E G} _ {T} \leq \frac {5 1 2}{\alpha} \ln \frac {2 ^ {2 0} N}{\alpha^ {2}} + 2 C _ {0} \ln \left(4 C _ {0} ^ {2} N\right) + \frac {3 2 d}{\alpha} \ln \left(1 + \frac {\alpha C _ {5}}{8 \gamma d} F _ {T} ^ {\mathbf {x}}\right)
$$
$$
\begin{array}{l} + \left(\frac {3 2 D ^ {2} G ^ {4}}{\gamma} - \frac {C _ {0}}{2}\right) S _ {T} ^ {\boldsymbol {q}} + \left(\frac {3 2 G ^ {4}}{\gamma} - \lambda_ {1}\right) \sum_ {t = 2} ^ {T} \sum_ {k = 1} ^ {K} q _ {t, k} \| \mathbf {x} _ {t, k} - \mathbf {x} _ {t - 1, k} \| ^ {2} \\ + \left(\lambda_ {2} + \frac {1 6 G ^ {4}}{\gamma} - \frac {\gamma}{4}\right) S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} \\ \leq \mathcal {O} \left(\frac {d}{\alpha} \ln F _ {T} ^ {\mathbf {x}}\right) \quad (\text {r e q u i r i n g} C _ {0} \geq 6 4 D ^ {2} G ^ {4} / \gamma , \lambda_ {1} \geq 3 2 G ^ {4} / \gamma , \gamma \geq 8 G ^ {2} + \lambda_ {2}) \\ \leq \mathcal {O} \left(\frac {d}{\alpha} \ln F _ {T}\right), \\ \end{array}
$$
where the last uses the following lemma.
Lemma 4 (Corollary 5 of Orabona et al. [2012]). If  $a, b, c, d, x > 0$  satisfy  $x - d \leq a \ln (bx + c)$ ,
$$
x - d \leq a \ln \left(2 a b \ln \frac {2 a b}{e} + 2 b d + 2 c\right).
$$
Strongly Convex Functions. The meta regret can be bounded the same as Theorem 1 (especially (C.3)), and thus we directly move on to the base regret. For simplicity, we denote by  $\mathbf{g}_t \triangleq \nabla f_t(\mathbf{x}_t)$  and give a different decomposition for  $\bar{V}_{T,k^{\star},i^{\star}} = \sum_{t=2}^{T} \|\nabla h_{t,i^{\star}}^{\mathrm{sc}}(\mathbf{x}_{t,k^{\star},i^{\star}}) - \nabla h_{t-1,i^{\star}}^{\mathrm{sc}}(\mathbf{x}_{t-1,k^{\star},i^{\star}})\|^2$ , the empirical gradient variation of the base learner.
$$
\begin{array}{l} \bar {V} _ {T, k ^ {*}, i ^ {*}} = \sum_ {t = 2} ^ {T} \| \left(\mathbf {g} _ {t} + \lambda_ {i ^ {*}} \left(\mathbf {x} _ {t, k ^ {*}, i ^ {*}} - \mathbf {x} _ {t}\right)\right) - \left(\mathbf {g} _ {t - 1} + \lambda_ {i ^ {*}} \left(\mathbf {x} _ {t - 1, k ^ {*}, i ^ {*}} - \mathbf {x} _ {t - 1}\right)\right) \| ^ {2} \\ \leq 2 \sum_ {t = 2} ^ {T} \| \mathbf {g} _ {t} - \mathbf {g} _ {t - 1} \| ^ {2} + 2 \lambda_ {i ^ {\star}} ^ {2} \sum_ {t = 2} ^ {T} \| (\mathbf {x} _ {t, k ^ {\star}, i ^ {\star}} - \mathbf {x} _ {t}) - (\mathbf {x} _ {t - 1, k ^ {\star}, i ^ {\star}} - \mathbf {x} _ {t - 1}) \| ^ {2} \\ \leq 3 2 L F _ {T} ^ {\mathbf {x}} + 4 S _ {T} ^ {\mathbf {x}} + 4 S _ {T, k ^ {*}, i ^ {*}} ^ {\mathbf {x}}, \quad (\text {b y (E . 3) a n d} \lambda \in [ 1 / T, 1 ]) \\ \end{array}
$$
where the first step uses the definition of  $\nabla h_{t,i}^{\mathrm{sc}}(\mathbf{x}) = \mathbf{g}_t + \lambda_i(\mathbf{x} - \mathbf{x}_t)$ . Consequently,
$$
\frac {1 6 G _ {\mathrm {s c}} ^ {2}}{\lambda_ {i ^ {\star}}} \ln \left(1 + \lambda_ {i ^ {\star}} \bar {V} _ {T, k ^ {\star}, i ^ {\star}}\right) \leq \frac {1 6 G _ {\mathrm {s c}} ^ {2}}{\lambda_ {i ^ {\star}}} \ln (1 + 3 2 \lambda_ {i ^ {\star}} L F _ {T} ^ {\mathbf {x}}) + 6 4 G _ {\mathrm {s c}} ^ {2} (S _ {T} ^ {\mathbf {x}} + S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}}).
$$
$$
\text {B A S E - R E G} \leq \frac {3 2 G _ {\mathrm {s c}} ^ {2}}{\lambda} \ln (1 + 3 2 \lambda L F _ {T} ^ {\mathbf {x}}) + 6 4 G _ {\mathrm {s c}} ^ {2} S _ {T} ^ {\mathbf {x}} + \left(6 4 G _ {\mathrm {s c}} ^ {2} - \frac {\gamma}{8}\right) S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} + \frac {1}{4} \gamma D ^ {2}.
$$
Combining the meta regret (C.3) and base regret, the overall regret can be bounded by
$$
\begin{array}{l} \mathrm {R E G} _ {T} \leq \frac {5 1 2 G ^ {2}}{\lambda} \ln \frac {2 ^ {2 0} N G ^ {4}}{\lambda^ {2}} + 2 C _ {0} \ln \left(4 C _ {0} ^ {2} N\right) + \frac {3 2 G _ {\mathrm {s c}} ^ {2}}{\lambda} \ln \left(1 + 3 2 \lambda L F _ {T} ^ {\mathbf {x}}\right) \\ + \left(1 2 8 D ^ {2} G _ {\mathrm {s c}} ^ {2} - \frac {C _ {0}}{2}\right) S _ {T} ^ {\boldsymbol {q}} + \left(1 2 8 G _ {\mathrm {s c}} ^ {2} - \lambda_ {1}\right) \sum_ {t = 2} ^ {T} \sum_ {k = 1} ^ {K} q _ {t, k} \| \mathbf {x} _ {t, k} - \mathbf {x} _ {t - 1, k} \| ^ {2} \\ + \left(\lambda_ {2} + 6 4 G _ {\mathrm {s c}} ^ {2} - \frac {\gamma}{8}\right) S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} \\ \leq \mathcal {O} \left(\frac {1}{\lambda} \ln F _ {T} ^ {\mathbf {x}}\right) \quad (\text {r e q u i r i n g} C _ {0} \geq 2 5 6 D ^ {2} G _ {\mathrm {s c}} ^ {2}, \lambda_ {1} \geq 1 2 8 G _ {\mathrm {s c}} ^ {2}, \gamma \geq 8 \lambda_ {2} + 5 1 2 G _ {\mathrm {s c}} ^ {2}) \\ \leq \mathcal {O} \left(\frac {1}{\lambda} \ln F _ {T}\right). \tag {byLemma4} \\ \end{array}
$$
Convex Functions. We first give a different analysis for  $V_{\star} \triangleq \sum_{t=1}^{T} (\ell_{t,k^{\star},i^{\star}} - m_{t,k^{\star},i^{\star}})^{2}$ . From Lemma 1, it holds that
$$
V _ {\star} \leq 2 D ^ {2} \sum_ {t = 1} ^ {T} \| \nabla f _ {t} (\mathbf {x} _ {t}) - \nabla f _ {t - 1} (\mathbf {x} _ {t - 1}) \| ^ {2} + 2 G ^ {2} \sum_ {t = 1} ^ {T} \| \mathbf {x} _ {t, k ^ {\star}, i ^ {\star}} - \mathbf {x} _ {t - 1, k ^ {\star}, i ^ {\star}} + \mathbf {x} _ {t - 1} - \mathbf {x} _ {t} \| ^ {2}
$$
$$
\leq 3 2 D ^ {2} L F _ {T} ^ {\mathbf {x}} + 4 G ^ {2} S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} + 4 G ^ {2} S _ {T} ^ {\mathbf {x}}. \tag {by(E.3)}
$$
Plugging the above analysis back to the meta regret, we obtain
$$
\begin{array}{l} \mathrm {M E T A - R E G} \leq \frac {1}{\eta_ {k ^ {\star}}} \ln \frac {N}{\eta_ {k ^ {\star}} ^ {2}} + 1 0 2 4 \eta_ {k ^ {\star}} D ^ {2} L F _ {T} ^ {\mathbf {x}} + 2 G ^ {2} S _ {T} ^ {\mathbf {x}} + (\lambda_ {2} + 2 G ^ {2}) S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} \\ - \frac {C _ {0}}{2} S _ {T} ^ {\boldsymbol {q}} - \lambda_ {1} \sum_ {t = 2} ^ {T} \sum_ {k = 1} ^ {K} q _ {t, k} \| \mathbf {x} _ {t, k} - \mathbf {x} _ {t - 1, k} \| ^ {2}. \\ \end{array}
$$
For the base regret, using Lemma 10 and (E.3), it holds that
$$
\text {B A S E - R E G} \leq 2 0 D \sqrt {L F _ {T} ^ {\mathbf {x}}} + \gamma D ^ {2} - \frac {\gamma}{4} S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} + \mathcal {O} (1).
$$
$$
\begin{array}{l} \operatorname {R E G} _ {T} \leq \frac {1}{\eta_ {k ^ {\star}}} \ln \frac {N}{\eta_ {k ^ {\star}} ^ {2}} + 1 0 2 4 \eta_ {k ^ {\star}} D ^ {2} L F _ {T} ^ {\mathbf {x}} + 2 0 D \sqrt {L F _ {T} ^ {\mathbf {x}}} + \left(\lambda_ {2} + 2 G ^ {2} - \frac {\gamma}{4}\right) S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} \\ + \left(4 D ^ {2} G ^ {2} - \frac {C _ {0}}{2}\right) S _ {T} ^ {\boldsymbol {q}} + \left(4 G ^ {2} - \lambda_ {1}\right) \sum_ {t = 2} ^ {T} \sum_ {k = 1} ^ {K} q _ {t, k} \| \mathbf {x} _ {t, k} - \mathbf {x} _ {t - 1, k} \| ^ {2} \\ \leq \frac {1}{\eta_ {k ^ {\star}}} \ln \frac {N}{\eta_ {k ^ {\star}} ^ {2}} + 1 0 2 4 \eta_ {k ^ {\star}} D ^ {2} L F _ {T} ^ {\mathbf {x}} + 2 0 D \sqrt {L F _ {T} ^ {\mathbf {x}}} \leq \frac {1}{\eta_ {k ^ {\star}}} \ln \frac {N e ^ {5}}{\eta_ {k ^ {\star}} ^ {2}} + 1 0 4 4 \eta_ {k ^ {\star}} D ^ {2} L F _ {T} ^ {\mathbf {x}} \\ \leq \frac {2}{\eta_ {k ^ {\star}}} \ln \frac {N e ^ {5}}{\eta_ {k ^ {\star}} ^ {2}} + 2 0 8 8 \eta_ {k ^ {\star}} D ^ {2} L F _ {T} \\ \leq 4 C _ {0} \ln \left(4 C _ {0} ^ {2} N\right) + \mathcal {O} \left(\sqrt {F _ {T} \ln F _ {T}}\right), \tag {by Lemma 7} \\ \end{array}
$$
where the second step holds by requiring  $C_0 \geq 8D^2 G^2$ ,  $\lambda_1 \geq 4G^2$  and  $\gamma \geq 4\lambda_2 + 8G^2$ , and the third step uses AM-GM inequality:  $\sqrt{xy} \leq \frac{ax}{2} + \frac{y}{2a}$  for any  $x, y, a > 0$  with  $a = 1/(2D\eta_{k^\star})$ . The fourth step is by requiring  $1 - 1044\eta_{k^\star}D^2 L \geq 1/2$ , i.e.,  $\eta_k \leq 1/(2088D^2 L)$  for any  $k \in [K]$ , which can be satisfied by requiring  $C_0 \geq 1044D^2 L$ .
$$
C _ {0} \geq 1, C _ {0} \geq 4 \lambda_ {1} D ^ {2}, C _ {0} \geq 1 2 8, C _ {0} \geq \frac {6 4 D ^ {2} G ^ {4}}{8 G ^ {2} + \lambda_ {2}}, C _ {0} \geq 2 5 6 D ^ {2} G _ {\mathrm {s c}} ^ {2}, C _ {0} \geq 8 D ^ {2} G ^ {2},
$$
$$
C _ {0} \geq 1 0 4 4 D ^ {2} L, \lambda_ {1} \geq \frac {3 2 G ^ {4}}{8 G ^ {2} + \lambda_ {2}}, \lambda_ {1} \geq 1 2 8 G _ {\mathrm {s c}} ^ {2}, \lambda_ {1} \geq 4 G ^ {2}, \lambda_ {2} \geq 2 \lambda_ {1}.
$$
As a result, we set  $C_0, \lambda_1, \lambda_2$  to be the minimum constants satisfying the above conditions. Besides, since the absolute values of the surrogate losses and the optimisms are bounded by problem-independent constants, as shown in the proof of Theorem 1, rescaling them to  $[-1,1]$  only add a constant multiplicative factor on the final result.
D Proofs for Appendix A
This section provides the proofs for Appendix A, including Theorem 3 and Theorem 4.
A key result in the work of Chen et al. [2023b] is the following decomposition for the empirical gradient variation  $\bar{V}_T\triangleq \sum_{t = 2}^T\| \nabla f_t(\mathbf{x}_t) - \nabla f_{t - 1}(\mathbf{x}_{t - 1})\|^2$ , restated as follows.
Lemma 5 (Lemma 4 of Chen et al. [2023b]). Under the same assumptions as Chen et al. [2023b],
$$
\mathbb {E} \left[ \sum_ {t = 2} ^ {T} \| \nabla f _ {t} (\mathbf {x} _ {t}) - \nabla f _ {t - 1} (\mathbf {x} _ {t - 1}) \| ^ {2} \right] \leq 4 L ^ {2} \sum_ {t = 2} ^ {T} \| \mathbf {x} _ {t} - \mathbf {x} _ {t - 1} \| ^ {2} + 8 \sigma_ {1: T} ^ {2} + 4 \Sigma_ {1: T} ^ {2} + \mathcal {O} (1).
$$
Proof of Theorem 3. The analysis is almost the same as the proof of Theorem 2. Some abbreviations of the stability terms are defined in (B.1).
Exp-concave Functions. The meta regret remains the same as (C.1) and the only difference is a slightly different decomposition of the empirical gradient variation of the base learner, i.e.,  $\bar{V}_{T,k^{\star},i^{\star}}$  in (C.2). Specifically, it holds that
$$
\begin{array}{l} \bar {V} _ {T, k ^ {\star}, i ^ {\star}} = \sum_ {t = 2} ^ {T} \| (\mathbf {g} _ {t} + \alpha_ {i ^ {\star}} \mathbf {g} _ {t} \mathbf {g} _ {t} ^ {\top} (\mathbf {x} _ {t, k ^ {\star}, i ^ {\star}} - \mathbf {x} _ {t})) - (\mathbf {g} _ {t - 1} + \alpha_ {i ^ {\star}} \mathbf {g} _ {t - 1} \mathbf {g} _ {t - 1} ^ {\top} (\mathbf {x} _ {t - 1, k ^ {\star}, i ^ {\star}} - \mathbf {x} _ {t - 1})) \| ^ {2} \\ \leq \left(2 + 1 6 D ^ {2} G ^ {2}\right) \sum_ {t = 2} ^ {T} \left\| \mathbf {g} _ {t} - \mathbf {g} _ {t - 1} \right\| ^ {2} + 8 G ^ {4} S _ {T} ^ {\mathbf {x}} + 8 G ^ {4} S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} \\ \leq 2 C _ {5} \sigma_ {1: T} ^ {2} + C _ {5} \Sigma_ {1: T} ^ {2} + C _ {6} S _ {T} ^ {\mathbf {x}} + 8 G ^ {4} S _ {T, k ^ {*}, i ^ {*}} ^ {\mathbf {x}}, \\ \end{array}
$$
where  $C_5 = 8(1 + 8D^2 G^2)$ ,  $C_6 = 8L^2 + 64D^2 G^2 L^2 + 8G^4$  and the last step is by taking expectation on both sides and Lemma 5, leading to the following upper bound of the base algorithm:
$$
\mathbb {E} [ \text {B A S E - R E G} ] \leq \mathcal {O} \left(\frac {d}{\alpha} \ln \left(\sigma_ {1: T} ^ {2} + \Sigma_ {1: T} ^ {2}\right)\right) + \frac {2 C _ {6}}{\gamma} S _ {T} ^ {\mathbf {x}} + \left(\frac {1 6 G ^ {4}}{\gamma} - \frac {\gamma}{4}\right) S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} + \frac {1}{2} \gamma D ^ {2}.
$$
$$
\begin{array}{l} \mathbb {E} \left[ \operatorname {R E G} _ {T} \right] \leq \frac {5 1 2}{\alpha} \ln \frac {2 ^ {2 0} N}{\alpha^ {2}} + 2 C _ {0} \ln \left(4 C _ {0} ^ {2} N\right) + \mathcal {O} \left(\frac {d}{\alpha} \ln \left(\sigma_ {1: T} ^ {2} + \Sigma_ {1: T} ^ {2}\right)\right) \\ + \left(\frac {4 C _ {6} D ^ {2}}{\gamma} - \frac {C _ {0}}{2}\right) S _ {T} ^ {\boldsymbol {q}} + \left(\frac {4 C _ {6}}{\gamma} - \lambda_ {1}\right) \sum_ {t = 2} ^ {T} \sum_ {k = 1} ^ {K} q _ {t, k} \| \mathbf {x} _ {t, k} - \mathbf {x} _ {t - 1, k} \| ^ {2} \\ + \left(\lambda_ {2} + \frac {1 6 G ^ {4}}{\gamma} - \frac {\gamma}{4}\right) S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} + \frac {1}{2} \gamma D ^ {2} \leq \mathcal {O} \left(\frac {d}{\alpha} \ln \left(\sigma_ {1: T} ^ {2} + \Sigma_ {1: T} ^ {2}\right)\right), \\ \end{array}
$$
where the last step holds by requiring  $C_0 \geq 8C_6D^2/\gamma$ ,  $\lambda_1 \geq 4C_6/\gamma$  and  $\gamma \geq \lambda_2 + 8G^2$ .
Strongly Convex Functions. The meta regret remains the same as (C.3) and the only difference is a slightly different decomposition of the empirical gradient variation of the base learner:
$$
\begin{array}{l} \mathbb {E} \left[ \| \nabla h _ {t, i ^ {\star}} ^ {\mathrm {s c}} \left(\mathbf {x} _ {t, k ^ {\star}, i ^ {\star}}\right) - \nabla h _ {t - 1, i ^ {\star}} ^ {\mathrm {s c}} \left(\mathbf {x} _ {t - 1, k ^ {\star}, i ^ {\star}}\right) \| ^ {2} \right] \\ = \mathbb {E} \left[ \left\| \left(\mathbf {g} _ {t} + \lambda_ {i ^ {\star}} \left(\mathbf {x} _ {t, k ^ {\star}, i ^ {\star}} - \mathbf {x} _ {t}\right)\right) - \left(\mathbf {g} _ {t - 1} + \lambda_ {i ^ {\star}} \left(\mathbf {x} _ {t - 1, k ^ {\star}, i ^ {\star}} - \mathbf {x} _ {t - 1}\right)\right) \right\| ^ {2} \right] \\ \leq 2 \mathbb {E} \left[ \| \mathbf {g} _ {t} - \mathbf {g} _ {t - 1} \| ^ {2} \right] + 2 \lambda_ {i ^ {*}} ^ {2} \| (\mathbf {x} _ {t, k ^ {*}, i ^ {*}} - \mathbf {x} _ {t}) - (\mathbf {x} _ {t - 1, k ^ {*}, i ^ {*}} - \mathbf {x} _ {t - 1}) \| ^ {2} \\ \leq 1 6 \sigma_ {t} ^ {2} + 8 \Sigma_ {t} ^ {2} + (4 + 8 L ^ {2}) \| \mathbf {x} _ {t} - \mathbf {x} _ {t - 1} \| ^ {2} + 4 \| \mathbf {x} _ {t, k ^ {*}, i ^ {*}} - \mathbf {x} _ {t - 1, k ^ {*}, i ^ {*}} \| ^ {2}, \\ \end{array}
$$
where  $\sigma_t^2 \triangleq \max_{\mathbf{x} \in \mathcal{X}} \mathbb{E}_{f_t \sim \mathcal{D}_t}[\| \nabla f_t(\mathbf{x}) - \nabla F_t(\mathbf{x})\|^2]$ ,  $\Sigma_t^2 \triangleq \mathbb{E}[\sup_{\mathbf{x} \in \mathcal{X}} \| \nabla F_t(\mathbf{x}) - \nabla F_{t-1}(\mathbf{x})\|^2]$ , and the last step is by taking expectation on both sides and Lemma 5. Plugging the above empirical gradient variation decomposition into Lemma 12, we aim to control the following term:
$$
\begin{array}{l} \mathbb {E} \left[ \sum_ {t = 2} ^ {T} \frac {1}{\lambda_ {i ^ {\star}} t} \| \nabla h _ {t, i ^ {\star}} ^ {\mathrm {s c}} \left(\mathbf {x} _ {t, k ^ {\star}, i ^ {\star}}\right) - \nabla h _ {t - 1, i ^ {\star}} ^ {\mathrm {s c}} \left(\mathbf {x} _ {t - 1, k ^ {\star}, i ^ {\star}}\right) \| ^ {2} \right] \\ \leq 8 \sum_ {t = 2} ^ {T} \frac {2 \sigma_ {t} ^ {2} + \Sigma_ {t} ^ {2}}{\lambda_ {i ^ {\star}} t} + 4 \sum_ {t = 2} ^ {T} \frac {(1 + 2 L ^ {2}) \| \mathbf {x} _ {t} - \mathbf {x} _ {t - 1} \| ^ {2} + \| \mathbf {x} _ {t , k ^ {\star} , i ^ {\star}} - \mathbf {x} _ {t - 1 , k ^ {\star} , i ^ {\star}} \| ^ {2}}{\lambda_ {i ^ {\star}} t} \\ \leq \mathcal {O} \left(\frac {1}{\lambda} (\sigma_ {\max } ^ {2} + \Sigma_ {\max } ^ {2}) \ln (\sigma_ {1: T} ^ {2} + \Sigma_ {1: T} ^ {2})\right) + \frac {8 D ^ {2} (L ^ {2} + 1)}{\lambda_ {i ^ {\star}}} \ln (1 + \lambda_ {i ^ {\star}} (1 + 2 L ^ {2}) S _ {T} ^ {\mathbf {x}} + \lambda_ {i ^ {\star}} S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}}). \\ \end{array}
$$
The last step is due to Lemma 9, which is a generalization of Lemma 5 of Chen et al. [2023b]. Combining the meta regret and base regret, the overall regret can be bounded by
$$
\begin{array}{l} \mathbb {E} \left[ \operatorname {R E G} _ {T} \right] \leq \frac {5 1 2 G ^ {2}}{\lambda} \ln \frac {2 ^ {2 0} N G ^ {4}}{\lambda^ {2}} + 2 C _ {0} \ln \left(4 C _ {0} ^ {2} N\right) + \mathcal {O} \left(\frac {1}{\lambda} \left(\sigma_ {\max } ^ {2} + \Sigma_ {\max } ^ {2}\right) \ln \left(\sigma_ {1: T} ^ {2} + \Sigma_ {1: T} ^ {2}\right)\right) \\ + \left(2 C _ {7} D ^ {2} - \frac {C _ {0}}{2}\right) S _ {T} ^ {\boldsymbol {q}} + \left(2 C _ {7} - \lambda_ {1}\right) \sum_ {t = 2} ^ {T} \sum_ {k = 1} ^ {K} q _ {t, k} \| \mathbf {x} _ {t, k} - \mathbf {x} _ {t - 1, k} \| ^ {2} \\ \end{array}
$$
$$
\begin{array}{l} + \left(\lambda_ {2} + 3 2 D ^ {2} \left(L ^ {2} + 1\right) - \frac {\gamma}{8}\right) S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} + \frac {1}{4} \gamma D ^ {2} \\ \leq \mathcal {O} \left(\frac {1}{\lambda} \left(\sigma_ {\max } ^ {2} + \Sigma_ {\max } ^ {2}\right) \ln \left(\sigma_ {1: T} ^ {2} + \Sigma_ {1: T} ^ {2}\right)\right), \\ \end{array}
$$
where  $C_7 = 64D^2 (L^2 +1)^2$  and the last step holds by requiring  $C_0\geq 4C_7D^2$ ,  $\lambda_{1}\geq 2C_{7}$  and  $\gamma \geq 8\lambda_{2} + 256D^{2}(L^{2} + 1)$ .
Convex Functions. We first give a slightly different decomposition for  $V_{\star} \triangleq \sum_{t} (\ell_{t,k^{\star},i^{\star}} - m_{t,k^{\star},i^{\star}})^{2}$ . Starting from Lemma 5, it holds that
$$
\begin{array}{l} V _ {\star} \leq 2 D ^ {2} \sum_ {t = 1} ^ {T} \| \nabla f _ {t} (\mathbf {x} _ {t}) - \nabla f _ {t - 1} (\mathbf {x} _ {t - 1}) \| ^ {2} + 2 G ^ {2} \sum_ {t = 1} ^ {T} \| \mathbf {x} _ {t, k ^ {\star}, i ^ {\star}} - \mathbf {x} _ {t - 1, k ^ {\star}, i ^ {\star}} + \mathbf {x} _ {t - 1} - \mathbf {x} _ {t} \| ^ {2} \\ \leq 8 D ^ {2} \left(2 \sigma_ {1: T} ^ {2} + \Sigma_ {1: T} ^ {2}\right) + \left(8 D ^ {2} L ^ {2} + 4 G ^ {2}\right) S _ {T} ^ {\mathbf {x}} + 4 G ^ {2} S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} + \mathcal {O} (1), \\ \end{array}
$$
where the last step by taking expectation on both sides. Following the same proof as in Theorem 2, taking expectation on both sides, the expected meta regret can be bounded by
$$
\begin{array}{l} \mathbb {E} [ \text {M E T A - R E G} ] \leq \frac {1}{\eta_ {k ^ {*}}} \ln \frac {N}{\eta_ {k ^ {*}} ^ {2}} + 1 2 8 D ^ {2} \eta_ {k ^ {*}} \left(2 \sigma_ {1: T} ^ {2} + \Sigma_ {1: T} ^ {2}\right) + 1 2 8 \left(2 D ^ {2} L ^ {2} + G ^ {2}\right) S _ {T} ^ {\mathbf {x}} \\ + \left(\lambda_ {2} + 1 2 8 G ^ {2}\right) S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} - \frac {C _ {0}}{2} S _ {T} ^ {\mathbf {q}} - \lambda_ {1} \sum_ {t = 2} ^ {T} \sum_ {k = 1} ^ {K} q _ {t, k} \left\| \mathbf {x} _ {t, k} - \mathbf {x} _ {t - 1, k} \right\| ^ {2} \\ \leq 2 C _ {0} \ln \left(4 C _ {0} ^ {2} N\right) + \mathcal {O} \left(\sqrt {\left(\sigma_ {1 : T} ^ {2} + \Sigma_ {1 : T} ^ {2}\right) \ln \left(\sigma_ {1 : T} ^ {2} + \Sigma_ {1 : T} ^ {2}\right)}\right) \\ + 1 2 8 (2 D ^ {2} L ^ {2} + G ^ {2}) S _ {T} ^ {\mathbf {x}} + (\lambda_ {2} + 1 2 8 G ^ {2}) S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} - \frac {C _ {0}}{2} S _ {T} ^ {\boldsymbol {q}} - \lambda_ {1} \sum_ {t = 2} ^ {T} \sum_ {k = 1} ^ {K} q _ {t, k} \| \mathbf {x} _ {t, k} - \mathbf {x} _ {t - 1, k} \| ^ {2}, \\ \end{array}
$$
where the last step is due to Lemma 7 by requiring  $C_0 \geq 8D$ .
For the base regret, the only difference is a slightly different decomposition of the empirical gradient variation of the base learner, i.e.,  $\bar{V}_{T,k^{\star},i^{\star}}$  in (C.5). Specifically, using Lemma 5, it holds that
$$
\mathbb {E} \left[ \text {B A S E - R E G} \right] \leq 1 0 D \sqrt {\left(2 \sigma_ {1 : T} ^ {2} + \Sigma_ {1 : T} ^ {2}\right)} + 1 0 D L S _ {T} ^ {\mathbf {x}} + \gamma D ^ {2} - \frac {\gamma}{4} S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} + \mathcal {O} (1).
$$
$$
\begin{array}{l} \mathbb {E} \left[ \operatorname {R E G} _ {T} \right] \leq \mathcal {O} \left(\sqrt {\left(\sigma_ {1 : T} ^ {2} + \Sigma_ {1 : T} ^ {2}\right) \ln \left(\sigma_ {1 : T} ^ {2} + \Sigma_ {1 : T} ^ {2}\right)}\right) + 2 C _ {0} \ln \left(4 C _ {0} ^ {2} N\right) \\ + \left(2 D ^ {2} C _ {9} - \frac {C _ {0}}{2}\right) S _ {T} ^ {\boldsymbol {q}} + \left(2 C _ {9} - \lambda_ {1}\right) \sum_ {t = 2} ^ {T} \sum_ {k = 1} ^ {K} q _ {t, k} \left\| \mathbf {x} _ {t, k} - \mathbf {x} _ {t - 1, k} \right\| ^ {2} \\ + \left(\lambda_ {2} + 1 2 8 G ^ {2} - \frac {\gamma}{4}\right) S _ {T, k ^ {\star}, i ^ {\star}} ^ {\mathbf {x}} + \gamma D ^ {2} \\ \leq \mathcal {O} \left(\sqrt {\left(\sigma_ {1 : T} ^ {2} + \Sigma_ {1 : T} ^ {2}\right) \ln \left(\sigma_ {1 : T} ^ {2} + \Sigma_ {1 : T} ^ {2}\right)}\right) \\ \end{array}
$$
where  $C_9 = 128(2D^2 L^2 + G^2) + 10DL$  and the last step holds by requiring  $C_0 \geq 4D^2 C_9$ ,  $\lambda_1 \geq 2C_9$  and  $\gamma \geq 4\lambda_2 + 512G^2$ .
$$
C _ {0} \geq 1, C _ {0} \geq 4 \lambda_ {1} D ^ {2}, C _ {0} \geq 1 2 8, C _ {0} \geq \frac {8 C _ {6} D ^ {2}}{8 G ^ {2} + \lambda_ {2}}, C _ {0} \geq 4 C _ {7} D ^ {2}, C _ {0} \geq \frac {4 D ^ {2} C _ {8}}{1 6 + \lambda_ {2}}, C _ {0} \geq 8 D,
$$
$$
C _ {0} \geq 4 D ^ {2} C _ {9}, \lambda_ {1} \geq \frac {4 C _ {6}}{8 G ^ {2} + \lambda_ {2}}, \lambda_ {1} \geq 2 C _ {7}, \lambda_ {1} \geq \frac {2 C _ {8}}{1 6 + \lambda_ {2}}, \lambda_ {1} \geq 2 C _ {9}, \lambda_ {1} \geq 4 G ^ {2}, \lambda_ {2} \geq 2 \lambda_ {1}.
$$
D.2 Proof of Theorem 4
Proof. The proof of the dishonest case is straightforward by directly applying Theorem 2. In the following, we mainly focus on the honest case. Consider a bilinear game of  $f(\mathbf{x}, \mathbf{y}) \triangleq \mathbf{x}^{\top} A \mathbf{y}$  and denote by  $\mathbf{g}_t^{\mathbf{x}} \triangleq A \mathbf{y}_t$ ,  $\mathbf{g}_t^{\mathbf{y}} \triangleq A \mathbf{x}_t$  the gradients received by the x-player and y-player. The only difference from the proof of Theorem 2 is that the gradient variation can be now decomposed
$$
\left\| \mathbf {g} _ {t} ^ {\mathbf {x}} - \mathbf {g} _ {t - 1} ^ {\mathbf {x}} \right\| ^ {2} = \left\| A \mathbf {y} _ {t} - A \mathbf {y} _ {t - 1} \right\| ^ {2} \leq \left\| \mathbf {y} _ {t} - \mathbf {y} _ {t - 1} \right\| ^ {2},
$$
where the last step holds under the mild assumption of  $\| A \| \leq 1$ . The gradient variation of the x-player is associated with the stability term of the y-player. When summing the regret of the players, the negative terms in the x-player's algorithm can be leveraged to cancel the gradient variation of the y-player and vise versa. As a result, all gradient variations can be canceled and the summation of regret is bounded by  $\mathcal{O}(1)$ . As for strongly convex-concave games, since it is a special case of the bilinear games, the above derivations still hold, which completes the proof.
E Supporting Lemmas
In this section, we present several supporting lemmas used in proving our theoretical results. In Appendix E.1, we provide useful lemmas for the decomposition of two combined decisions and the parameter tuning. And in Appendix E.2, we analyze the stability of the base algorithms for different kinds of loss functions.
E.1 Useful Lemmas
In this part, we conclude some useful lemmas for bounding the gap between two combined decisions (Lemma 6), tuning the parameter (Lemma 7 and Lemma 8), and a useful summation (Lemma 9).
Lemma 6. Under Assumption 1, if  $\mathbf{x} = \sum_{i=1}^{N} p_i \mathbf{x}_i$ ,  $\mathbf{y} = \sum_{i=1}^{N} q_i \mathbf{y}_i$ , where  $\mathbf{p}, \mathbf{q} \in \Delta_N, \mathbf{x}_i, \mathbf{y}_i \in \mathcal{X}$  for any  $i \in [N]$ , then it holds that
$$
\left\| \mathbf {x} - \mathbf {y} \right\| ^ {2} \leq 2 \sum_ {i = 1} ^ {N} p _ {i} \left\| \mathbf {x} _ {i} - \mathbf {y} _ {i} \right\| ^ {2} + 2 D ^ {2} \left\| \boldsymbol {p} - \boldsymbol {q} \right\| _ {1} ^ {2}.
$$
Lemma 7. For a step size pool of  $\mathcal{H}_{\eta} = \{\eta_k\}_{k\in [K]}$ , where  $\eta_{1} = \frac{1}{2C_{0}}\geq \ldots \geq \eta_{K} = \frac{1}{2C_{0}T}$ , if  $C_0\geq \frac{\sqrt{X}}{2T}$ , there exists  $\eta \in \mathcal{H}_{\eta}$  such that
$$
\frac {1}{\eta} \ln \frac {Y}{\eta^ {2}} + \eta X \leq 2 C _ {0} \ln \left(4 Y C _ {0} ^ {2}\right) + 4 \sqrt {X \ln (4 X Y)}.
$$
Lemma 8. Denoting by  $\eta_{\star}$  the optimal step size, for a step size pool of  $\mathcal{H}_{\eta} = \{\eta_k\}_{k\in [K]}$ , where  $\eta_1 = \frac{1}{2C_0}\geq \ldots \geq \eta_K = \frac{1}{2C_0T}$ , if  $C_0\geq \frac{1}{2\eta_\star T}$ , there exists  $\eta \in \mathcal{H}_{\eta}$  such that
$$
\frac {1}{\eta} \ln \frac {Y}{\eta^ {2}} \leq 2 C _ {0} \ln \left(4 Y C _ {0} ^ {2}\right) + \frac {2}{\eta_ {\star}} \ln \frac {4 Y}{\eta_ {\star} ^ {2}}.
$$
Lemma 9. For a sequence of  $\{a_t\}_{t=1}^T$  and  $b$ , where  $a_t, b > 0$  for any  $t \in [T]$ , denoting by  $a_{\max} \triangleq \max_t a_t$  and  $A \triangleq \lceil b \sum_{t=1}^T a_t \rceil$ , we have