SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Remark Note that, by setting  $\alpha_{t} = 1$ , according to Theorem 7, we get  $\eta_t = \frac{1}{L(\mathbf{v}_{t - 1})}$ ,  $\beta_{1} = 1$ , and  $\beta_{t} = \frac{1}{\sum_{j = 1}^{t - 1}\alpha_{j}} = \frac{1}{t - 1}$ , which recovers the parameter configuration in Theorem 2.
Corollary 2 shows that, with a step size of order  $O\left(\frac{1}{L(\mathbf{v}_{t - 1})}\right)$ , Algorithm 1 enjoys a faster  $\widetilde{O} (\frac{1}{(q - 1)\gamma^2t})$  margin maximization rate when  $t = \Omega \left(\frac{\log n\log t}{(q - 1)\gamma^2}\right)$ . Next, we also considered using a even larger step size,  $\eta_{t} = \frac{t}{L(\mathbf{v}_{t - 1})}$ , and recover Theorem 3.
Corollary 3 (Theorem 3). Let  $\alpha_{t} = t$ . Then  $\widetilde{\mathbf{v}}_T = \sum_{t=1}^{T} \frac{2}{t+1} \mathbf{v}_t$ . When  $T \geq \sqrt{\frac{8 \left[ \frac{4T}{q-1} + 4 \log n \log T + 1 + 2 \log n \right]}{\gamma^2}}$ , we have we have
$$
\frac {\min  _ {\mathbf {p} \in \Delta^ {n}} \mathbf {p} ^ {\top} \mathbf {A} \widetilde {\mathbf {v}} _ {T}}{\| \widetilde {\mathbf {v}} _ {T} \| _ {q}} \geq \gamma - \frac {3 2}{\gamma^ {2} T (q - 1)} - \frac {8 (4 \log n \log T + 1 + 2 \log n)}{\gamma^ {2} T ^ {2}}, \tag {19}
$$
Note that, when  $\alpha_{t} = t$ , we have  $\eta_{t} = \frac{t}{L(\mathbf{v}_{t - 1})}$ , which recovers Theorem 3. Finally, we note that, if  $T$  is fixed and known before, we can set  $\alpha_{t}$  as  $T$  and get rid of the  $\log T$  term.
C.1 Proof of Theorem 7
We first focus on the algorithm equivalence, and start from the online learning framework. For w-player, we have
$$
\begin{array}{l} \mathbf {w} _ {t} = \underset {\mathbf {w} \in \mathbb {R} ^ {d}} {\operatorname {a r g m i n}} \sum_ {j = 1} ^ {t} \alpha_ {j} h _ {j} (\mathbf {w}) \\ = \underset {\mathbf {w} \in \mathbb {R} ^ {d}} {\operatorname {a r g m i n}} \sum_ {j = 1} ^ {t} - \alpha_ {j} \mathbf {p} _ {j} ^ {\top} \mathbf {A} \mathbf {w} + \frac {\sum_ {j = 1} ^ {t} \alpha_ {j}}{2} \| \mathbf {w} \| _ {q} ^ {2} \tag {20} \\ = [ \nabla \Phi ] ^ {- 1} \left(\frac {1}{\sum_ {j = 1} ^ {t} \alpha_ {j}} \sum_ {j = 1} ^ {t} \alpha_ {j} \mathbf {A} ^ {\top} \mathbf {p} _ {j}\right), \\ \end{array}
$$
$$
\nabla \Phi (\mathbf {w} _ {t}) = \frac {1}{\sum_ {j = 1} ^ {t} \alpha_ {j}} \sum_ {j = 1} ^ {t} \alpha_ {j} \mathbf {A} ^ {\top} \mathbf {p} _ {j} = \frac {\sum_ {j = 1} ^ {t - 1} \alpha_ {j}}{\sum_ {j = 1} ^ {t} \alpha_ {j}} \nabla \Phi (\mathbf {w} _ {t - 1}) + \frac {\alpha_ {t}}{\sum_ {j = 1} ^ {t} \alpha_ {j}} \mathbf {A} ^ {\top} \mathbf {p} _ {t}.
$$
To proceed, note that
$$
\left[ \nabla \Phi (\mathbf {w}) \right] _ {i} = \frac {\mathrm {s i g n} (w _ {i}) | w _ {i} | ^ {q - 1}}{\| \mathbf {w} \| ^ {q - 2}}.
$$
Thus,  $\forall c > 0$ ,  $c\nabla \Phi (\mathbf{w}) = \nabla \Phi (c\mathbf{w})$ . Then, multiplying  $c = \sum_{j = 1}^{t}\alpha_{j}$  on both sides, we get
$$
\nabla \Phi \left(\mathbf {w} _ {t} \sum_ {j = 1} ^ {t} \alpha_ {j}\right) = \nabla \Phi \left(\mathbf {w} _ {t - 1} \sum_ {j = 1} ^ {t - 1} \alpha_ {j}\right) + \alpha_ {t} \mathbf {A} ^ {\top} \mathbf {p} _ {t}.
$$
On the other hand, for the  $\mathbf{p}$ -player, we have
$$
\mathbf {p} _ {t} = \underset {\mathbf {p} \in \Delta^ {n}} {\operatorname {a r g m i n}} \alpha_ {t} \ell_ {t - 1} (\mathbf {p}) + \beta_ {t} D _ {\mathrm {K L}} \left(\mathbf {p}, \frac {1}{n}\right) = \underset {\mathbf {p} \in \Delta^ {n}} {\operatorname {a r g m i n}} \frac {\alpha_ {t}}{\beta_ {t}} \mathbf {p} ^ {\top} \mathbf {A} \mathbf {w} _ {t - 1} + \sum_ {i = 1} ^ {n} p _ {i} \log \frac {p _ {i}}{\frac {1}{n}}.
$$
Based on a standard argument on the relationship between OMD with the negative entropy regularizer on the simplex (see, e.g., Section 6.6 of Orabona, 2019), it is easy to verify that  $\forall i\in [n],t\in [T]$
$$
p _ {t, i} = \frac {\exp \big (- \frac {\alpha_ {t}}{\beta_ {t}} y ^ {(i)} \mathbf {x} ^ {(i) \top} \mathbf {w} _ {t - 1} \big)}{\sum_ {j = 1} ^ {n} \exp \big (- \frac {\alpha_ {t}}{\beta_ {t}} y ^ {(j)} \mathbf {x} ^ {(j) \top} \mathbf {w} _ {t - 1} \big)},
$$
where  $p_{t,i}$  is the  $i$ -th element of  $\mathbf{p}_t$ . Moreover, based on the definition of  $L(\mathbf{w})$ , for any  $\mathbf{w} \in \mathbb{R}^d$ ,
$$
\frac {\nabla L (\mathbf {w})}{L (\mathbf {w})} = - A ^ {\top} \left[ \dots , \frac {\exp (- y ^ {(i)} \mathbf {x} ^ {(i) \top} \mathbf {w})}{\sum_ {j = 1} ^ {n} \exp (- y ^ {(j)} \mathbf {x} ^ {(j) \top} \mathbf {w})}, \dots \right] ^ {\top},
$$
$$
\mathbf {A} ^ {\top} \mathbf {p} _ {t} = - \frac {\nabla L \left(\frac {\alpha_ {t}}{\beta_ {t}} \mathbf {w} _ {t - 1}\right)}{L \left(\frac {\alpha_ {t}}{\beta_ {t}} \mathbf {w} _ {t - 1}\right)}.
$$
Combining the above equations and the definition of  $\beta_{t} = \frac{\alpha_{t}}{\sum_{i = 1}^{t - 1}\alpha_{i}}$ , we get
$$
\nabla \Phi \left(\mathbf {w} _ {t} \sum_ {j = 1} ^ {t} \alpha_ {j}\right) = \nabla \Phi \left(\mathbf {w} _ {t - 1} \sum_ {j = 1} ^ {t - 1} \alpha_ {j}\right) - \alpha_ {t} \frac {\nabla L \left(\mathbf {w} _ {t - 1} \sum_ {j = 1} ^ {t - 1} \alpha_ {j}\right)}{L \left(\mathbf {w} _ {t - 1} \sum_ {j = 1} ^ {t - 1} \alpha_ {j}\right)}.
$$
Substituting  $\mathbf{v}_t = \mathbf{w}_t\cdot \sum_{j = 1}^t\alpha_j$  , we get
$$
\nabla \Phi (\mathbf {v} _ {t}) = \nabla \Phi (\mathbf {v} _ {t - 1}) - \alpha_ {t} \frac {\nabla L (\mathbf {v} _ {t - 1})}{L (\mathbf {v} _ {t - 1})},
$$
and  $\widetilde{\mathbf{w}}_T = \sum_{t=1}^T \alpha_t \mathbf{w}_t = \sum_{t=1}^T \frac{\alpha_t}{\sum_{j=1}^t \alpha_j} \mathbf{v}_t$ . The proof is finished by replacing  $\frac{\alpha_t}{L(\mathbf{v}_{t-1})}$  with  $\eta_t$ .
Next, we focus on bounding the regret of the two players. For the  $\mathbf{p}$ -player, let  $\mathbf{p}^{*,\ell} = \min_{\mathbf{p} \in \Delta^n} \sum_{t=1}^T \alpha_t \mathbf{p}^\top \mathbf{A} \mathbf{w}_t$  be the best decision in hindsight for the online leaning problem. We have
$$
\begin{array}{l} \operatorname {R e g} _ {T} ^ {\mathbf {p}} = \sum_ {t = 1} ^ {T} \alpha_ {t} \mathbf {p} _ {t} ^ {\top} \mathbf {A} \mathbf {w} _ {t} - \sum_ {t = 1} ^ {T} \alpha_ {t} \mathbf {p} ^ {*, \ell , \top} \mathbf {A} \mathbf {w} _ {t} \\ = \sum_ {t = 1} ^ {T} \left(\alpha_ {t} \mathbf {p} _ {t} ^ {\top} \mathbf {A} \mathbf {w} _ {t - 1} + \beta_ {t} D _ {\mathrm {K L}} \left(\mathbf {p} _ {t}, \frac {\mathbf {1}}{n}\right)\right) - \sum_ {t = 1} ^ {T} \alpha_ {t} \mathbf {p} ^ {*}, \ell , \top \mathbf {A} \mathbf {w} _ {t} + \sum_ {t = 1} ^ {T} \alpha_ {t} \mathbf {p} _ {t} ^ {\top} \mathbf {A} (\mathbf {w} _ {t} - \mathbf {w} _ {t - 1}) \\ - \sum_ {t = 1} ^ {T} \beta_ {t} D _ {E} \left(\mathbf {p} _ {t}, \frac {\mathbf {1}}{n}\right) \\ \end{array}
$$
$$
\begin{array}{l} \stackrel {(a)} {\leq} \sum_ {t = 1} ^ {T} \left(\alpha_ {t} \mathbf {p} ^ {*, \ell , \top} \mathbf {A} \mathbf {w} _ {t - 1} + \beta_ {t} D _ {\mathrm {K L}} \left(\mathbf {p} ^ {*, \ell}, \frac {\mathbf {1}}{n}\right)\right) - \sum_ {t = 1} ^ {T} \alpha_ {t} \mathbf {p} ^ {*, \ell , \top} \mathbf {A} \mathbf {w} _ {t} + \sum_ {t = 1} ^ {T} \alpha_ {t} \mathbf {p} _ {t} ^ {\top} \mathbf {A} (\mathbf {w} _ {t} - \mathbf {w} _ {t - 1}) \\ - \sum_ {t = 1} ^ {T} \beta_ {t} D _ {E} \left(\mathbf {p} _ {t}, \frac {\mathbf {1}}{n}\right) \\ \end{array}
$$
$$
\begin{array}{l} \stackrel {(b)} {\leq} \sum_ {t = 1} ^ {T} \alpha_ {t} \mathbf {p} ^ {*, \ell , \top} \mathbf {A} (\mathbf {w} _ {t - 1} - \mathbf {w} _ {t}) + \sum_ {t = 1} ^ {T} \alpha_ {t} \mathbf {p} _ {t} ^ {\top} \mathbf {A} (\mathbf {w} _ {t} - \mathbf {w} _ {t - 1}) + 2 \log n \sum_ {t = 1} ^ {T} \beta_ {t} \\ \stackrel {(c)} {\leq} \sum_ {t = 1} ^ {T} \alpha_ {t} \| \mathbf {p} ^ {*, \ell} \| _ {1} \| \mathbf {A} (\mathbf {w} _ {t - 1} - \mathbf {w} _ {t}) \| _ {\infty} + \sum_ {t = 1} ^ {T} \alpha_ {t} \| \mathbf {p} _ {t} \| _ {1} \| \mathbf {A} (\mathbf {w} _ {t} - \mathbf {w} _ {t - 1}) \| _ {\infty} + 2 \log n \sum_ {t = 1} ^ {T} \beta_ {t} \\ \stackrel {(d)} {\leq} 2 \sum_ {t = 1} ^ {T} \alpha_ {t} \| \mathbf {A} (\mathbf {w} _ {t - 1} - \mathbf {w} _ {t}) \| _ {\infty} + 2 \log n \sum_ {t = 1} ^ {T} \beta_ {t} \\ = 2 \sum_ {t = 1} ^ {T} \alpha_ {t} \underset {i \in [ n ]} {\operatorname {a r g m a x}} \left| y ^ {(i)} \mathbf {x} ^ {(i) \top} \left(\mathbf {w} _ {t - 1} - \mathbf {w} _ {t}\right) \right| + 2 \log n \sum_ {t = 1} ^ {T} \beta_ {t} \\ \stackrel {(e)} {\leq} 2 \sum_ {t = 1} ^ {T} \alpha_ {t} \| \mathbf {w} _ {t - 1} - \mathbf {w} _ {t} \| _ {q} + 2 \log n \sum_ {t = 1} ^ {T} \beta_ {t} \\ = 2 \sum_ {t = 2} ^ {T} \alpha_ {t} \| \mathbf {w} _ {t - 1} - \mathbf {w} _ {t} \| _ {q} + 2 \log n \sum_ {t = 1} ^ {T} \beta_ {t} + \alpha_ {1} \| \mathbf {w} _ {1} \| _ {q} \\ \stackrel {(f)} {\leq} 2 \sum_ {t = 2} ^ {T} \frac {2 \alpha_ {t} ^ {2}}{2 \sum_ {j = 1} ^ {t - 1} \alpha_ {j} (q - 1)} + 2 \sum_ {t = 2} ^ {T} \frac {\sum_ {j = 1} ^ {t - 1} \alpha_ {j} (q - 1)}{2 \cdot 2} \| \mathbf {w} _ {t} - \mathbf {w} _ {t - 1} \| _ {q} ^ {2} + 2 \log n \sum_ {t = 1} ^ {T} \beta_ {t} + \alpha_ {1} \| \mathbf {w} _ {1} \| _ {q} \\ = 2 \sum_ {t = 2} ^ {T} \frac {\alpha_ {t} ^ {2}}{\sum_ {j = 1} ^ {t - 1} \alpha_ {j} (q - 1)} + \sum_ {t = 2} ^ {T} \frac {\sum_ {j = 1} ^ {t - 1} \alpha_ {j} (q - 1)}{2} \| \mathbf {w} _ {t} - \mathbf {w} _ {t - 1} \| _ {q} ^ {2} + 2 \log n \sum_ {t = 1} ^ {T} \beta_ {t} + \alpha_ {1} \| \mathbf {w} _ {1} \| _ {q}. \tag {21} \\ \end{array}
$$
Here, inequality  $(a)$  is based on the optimality of  $\mathbf{p}_t$ , inequality  $(b)$  is due to the fact that the negative entropy regularizer is upper bounded, inequality  $(c)$  is because of the Hölder's inequality, inequality  $(d)$  is derived from  $\mathbf{p}_t$ ,  $\mathbf{p} \in \Delta^n$ , inequality  $(e)$  is based on the Hölder's inequality and  $\| \mathbf{x}^{(i)} \|_p \leq 1$  for all  $i \in [n]$ , and inequality  $(f)$  is based on Young's inequality:
$$
\sum_ {t = 2} ^ {T} \alpha_ {t} \| \mathbf {w} _ {t - 1} - \mathbf {w} _ {t} \| _ {q} \leq \sum_ {t = 2} ^ {T} \frac {2 \alpha_ {t} ^ {2}}{2 \sum_ {j = 1} ^ {t - 1} \alpha_ {j} (q - 1)} + \sum_ {t = 2} ^ {T} \frac {\sum_ {j = 1} ^ {t - 1} \alpha_ {j} (q - 1)}{2 \cdot 2} \| \mathbf {w} _ {t} - \mathbf {w} _ {t - 1} \| _ {q} ^ {2},
$$
where we pick  $\frac{\sum_{j=1}^{t-1} \alpha_i(q-1)}{2}$  as the constant of Young's inequality. Finally, note that  $\mathbf{w}_1 = [\nabla \Phi]^{-1} (\mathbf{A}^\top \mathbf{p}_1)$ , so based on the property of  $p$ -norm, we have
$$
\alpha_ {1} \| \mathbf {w} _ {1} \| _ {q} = \alpha_ {1} \| [ \nabla \Phi ] ^ {- 1} (\mathbf {A} ^ {\top} \mathbf {p} _ {1}) \| _ {q} = \alpha_ {1} \| \mathbf {A} ^ {\top} \mathbf {p} _ {1} \| _ {p} \leq \alpha_ {1}.
$$
For the  $\mathbf{w}$ -player, note that  $h_t(\mathbf{w})$  is  $(q - 1)$ -strongly convex wrt the  $\| \cdot \|_{q}$ -norm. Thus, based on Lemma 3, we have
$$
\operatorname {R e g} _ {T} ^ {\mathbf {w}} \leq - \sum_ {t = 1} ^ {T} \left(\frac {(q - 1) \sum_ {s = 1} ^ {t - 1} \alpha_ {s}}{2}\right) \| \mathbf {w} _ {t} - \mathbf {w} _ {t - 1} \| _ {q} ^ {2}. \tag {22}
$$
Note that  $C_T \leq \frac{\gamma^2}{4}$  is equivalent to  $\frac{\gamma^2}{4} \sum_{t=1}^{T} \alpha_t \geq (\operatorname{Reg}_T^{\mathbf{p}} + \operatorname{Reg}_T^{\mathbf{w}})$ . We have  $\frac{\gamma^2}{4} \sum_{t=1}^{T} \alpha_t = \frac{\gamma^2}{4} \sum_{t=1}^{T} \frac{1}{\sqrt{t}} \geq \frac{\gamma^2}{4} \sqrt{T}$ , and
$$
\begin{array}{l} \operatorname {R e g} _ {T} ^ {\mathbf {p}} + \operatorname {R e g} _ {T} ^ {\mathbf {w}} = \left(2 \sum_ {t = 2} ^ {T} \frac {\alpha_ {t} ^ {2}}{\sum_ {j = 1} ^ {t - 1} \alpha_ {j} (q - 1)} + 2 \log n \sum_ {t = 2} ^ {T} \frac {\alpha_ {t}}{\sum_ {i = 1} ^ {t - 1} \alpha_ {i}}\right) + \alpha_ {1} (1 + 2 \log n) \\ = 2 \sum_ {t = 2} ^ {T} \frac {1 / t}{(q - 1) \sum_ {j = 1} ^ {t - 1} 1 / \sqrt {j}} + 2 \log n \sum_ {t = 2} ^ {T} \frac {1 / \sqrt {t}}{\sum_ {i = 1} ^ {t - 1} 1 / \sqrt {j}} + 1 + 2 \log n \\ \leq 2 \sum_ {t = 2} ^ {T} \frac {1}{(q - 1) t \sqrt {t - 1}} + \log n \sum_ {t = 2} ^ {T} \frac {1}{\sqrt {t} \sqrt {t - 1}} + 1 + 2 \log n \tag {23} \\ \leq \frac {2}{q - 1} (\log T + 1) + \log n (\log T + 1) + 1 + 2 \log n \\ \leq \left(\frac {2}{q - 1} + \log n\right) \log T + 4 \log n + \frac {4}{q - 1}. \\ \end{array}
$$
Therefore,  $C_T \leq \frac{\gamma^2}{4}$  when
$$
\sqrt {T} \geq \frac {4 \left(\left(\frac {2}{q - 1} + \log n\right) \log T + 4 \log n + \frac {4}{q - 1}\right)}{\gamma^ {2}},
$$
and in this case
$$
\frac {\min  _ {\mathbf {p} \in \Delta^ {n}} \mathbf {p} ^ {\top} \mathbf {A} \widetilde {\mathbf {v}} _ {T}}{\left\| \widetilde {\mathbf {v}} _ {T} \right\| _ {q}} \geq \gamma - \frac {4 \left(\left(\frac {2}{q - 1} + \log n\right) \log T + 4 \log n + \frac {4}{q - 1}\right)}{\gamma^ {2} \sqrt {T}}. \tag {24}
$$
C.3 Proof of Corollary 2
We have  $\frac{\gamma^2}{4}\sum_{t = 1}^{T}\alpha_t = \frac{\gamma^2}{4} T$ , and
$$
\begin{array}{l} \operatorname {R e g} _ {T} ^ {\mathbf {p}} + \operatorname {R e g} _ {T} ^ {\mathbf {w}} = \left(2 \sum_ {t = 2} ^ {T} \frac {\alpha_ {t} ^ {2}}{\sum_ {j = 1} ^ {t - 1} \alpha_ {j} (q - 1)} + 2 \log n \sum_ {t = 2} ^ {T} \frac {\alpha_ {t}}{\sum_ {i = 1} ^ {t - 1} \alpha_ {i}}\right) + \alpha_ {1} (1 + 2 \log n) \\ = 2 \sum_ {t = 2} ^ {T} \frac {1}{(t - 1) (q - 1)} + 2 \log n \sum_ {t = 2} ^ {T} \frac {1}{t - 1} + 1 + 2 \log n \tag {25} \\ = 1 + 2 \log n + \left(\frac {2}{q - 1} + 2 \log n\right) \sum_ {t = 1} ^ {T - 1} \frac {1}{t} \\ \leq \left(\frac {2}{q - 1} + 2 \log n\right) (\log T + 2). \\ \end{array}
$$
$$
T \geq \frac {4 \left(\frac {2}{q - 1} + 2 \log n\right) (\log T + 2)}{\gamma^ {2}},
$$
$$
\frac {\min  _ {\mathbf {p} \in \Delta^ {n}} \mathbf {p} ^ {\top} \mathbf {A} \widetilde {\mathbf {v}} _ {T}}{\| \widetilde {\mathbf {v}} _ {T} \| _ {q}} \geq \gamma - \frac {4 \left(\frac {2}{q - 1} + 2 \log n\right) (\log T + 2)}{\gamma^ {2} T}. \tag {26}
$$
C.4 Proof of Corollary 3
We have  $\frac{\gamma^2}{4}\sum_{t = 1}^{T}\alpha_t = \frac{\gamma^2}{4}\frac{T(T + 1)}{2}\geq \frac{\gamma^2}{8} T^2$ , and
$$
\begin{array}{l} \left(2 \sum_ {t = 2} ^ {T} \frac {\alpha_ {t} ^ {2}}{\sum_ {j = 1} ^ {t - 1} \alpha_ {j} (q - 1)} + 2 \log n \sum_ {t = 2} ^ {T} \frac {\alpha_ {t}}{\sum_ {i = 1} ^ {t - 1} \alpha_ {i}}\right) + \alpha_ {1} (1 + 2 \log n) \\ = 2 \sum_ {t = 2} ^ {T} \frac {2 t ^ {2}}{t (t - 1) (q - 1)} + 2 \log n \sum_ {t = 2} ^ {T} \frac {2 t}{t (t - 1)} + 1 + 2 \log n \tag {27} \\ \leq \frac {8 T}{q - 1} + 4 \log n \log T + 1 + 2 \log n. \\ \end{array}
$$
$$
T \geq \sqrt {\frac {8 \left[ \frac {8 T}{q - 1} + 4 \log n \log T + 1 + 2 \log n \right]}{\gamma^ {2}}},
$$
$$
\frac {\min  _ {\mathbf {p} \in \Delta^ {n}} \mathbf {p} ^ {\top} \mathbf {A} \widetilde {\mathbf {v}} _ {T}}{\| \widetilde {\mathbf {v}} _ {T} \| _ {q}} \geq \gamma - \frac {3 2}{\gamma^ {2} T (q - 1)} - \frac {8 (4 \log n \log T + 1 + 2 \log n)}{\gamma^ {2} T ^ {2}}. \tag {28}
$$
C.5 Proof of Theorem 4
We first focus on the algorithm equivalence. We have
$$
\begin{array}{l} \mathbf {w} _ {t} = \underset {\mathbf {w} \in \mathbb {R} ^ {d}} {\operatorname {a r g m i n}} - \sum_ {i = 1} ^ {t - 1} \alpha_ {i} \mathbf {p} _ {i} ^ {\top} \mathbf {A} \mathbf {w} - \alpha_ {t} \mathbf {p} _ {t - 1} ^ {\top} \mathbf {A} \mathbf {w} + \sum_ {i = 1} ^ {t} \alpha_ {i} \Phi (\mathbf {w}) \tag {29} \\ = \left[ \nabla \Phi \right] ^ {- 1} \left(\frac {1}{\sum_ {i = 1} ^ {t} \alpha_ {i}} \left(\sum_ {i = 1} ^ {t - 1} \alpha_ {i} \mathbf {A} ^ {\top} \mathbf {p} _ {i} + \alpha_ {t} \mathbf {A} ^ {\top} \mathbf {p} _ {t - 1}\right)\right), \\ \end{array}
$$
$$
\begin{array}{l} \nabla \Phi (\mathbf {w} _ {t}) = \frac {1}{\sum_ {i = 1} ^ {t} \alpha_ {i}} \left(\sum_ {i = 1} ^ {t - 1} \alpha_ {i} \mathbf {A} ^ {\top} \mathbf {p} _ {i} + \alpha_ {t} \mathbf {A} ^ {\top} \mathbf {p} _ {t - 1}\right) \\ = \frac {1}{\sum_ {i = 1} ^ {t} \alpha_ {i}} \left(\frac {\sum_ {i = 1} ^ {t - 1} \alpha_ {i}}{\sum_ {i = 1} ^ {t - 1} \alpha_ {i}} \left(\sum_ {i = 1} ^ {t - 2} \alpha_ {i} \mathbf {A} ^ {\top} \mathbf {p} _ {i} + \alpha_ {t - 1} \mathbf {A} ^ {\top} \mathbf {p} _ {t - 2}\right) + \alpha_ {t} \mathbf {A} ^ {\top} \mathbf {p} _ {t - 1} + \alpha_ {t - 1} \mathbf {A} ^ {\top} (\mathbf {p} _ {t - 1} - \mathbf {p} _ {t - 2})\right) \\ = \frac {1}{\sum_ {i = 1} ^ {t} \alpha_ {i}} \left(\left[ \sum_ {i = 1} ^ {t - 1} \alpha_ {i} \right] \nabla \Phi \left(\mathbf {w} _ {t - 1}\right) + \alpha_ {t} \mathbf {A} ^ {\top} \mathbf {p} _ {t - 1} + \alpha_ {t - 1} \mathbf {A} ^ {\top} \left(\mathbf {p} _ {t - 1} - \mathbf {p} _ {t - 2}\right)\right). \tag {30} \\ \end{array}
$$
$$
\nabla \Phi \left(\mathbf {w} _ {t} \sum_ {i = 1} ^ {t} \alpha_ {i}\right) = \nabla \Phi \left(\mathbf {w} _ {t - 1} \sum_ {i = 1} ^ {t - 1} \alpha_ {i}\right) + \alpha_ {t} \mathbf {A} ^ {\top} \mathbf {p} _ {t - 1} + \alpha_ {t - 1} \mathbf {A} ^ {\top} \left(\mathbf {p} _ {t - 1} - \mathbf {p} _ {t - 2}\right). \tag {31}
$$
$$
\mathbf{p}_{t} = \operatorname *{argmin}_{\mathbf{p}\in \Delta^{n}}\alpha_{t}\ell_{t}(\mathbf{p}) + \beta_{t}D_{\mathrm{KL}}\left(\mathbf{p},\frac{\mathbf{1}}{n}\right).
$$
Based on the relationship between OMD with the negative entropy regularizer on the simplex (Hazan, 2016), it is easy to verify that  $\forall i\in [n],t\in [T]$
$$
p _ {t, i} = \frac {\exp \left(- \frac {\alpha_ {t}}{\beta_ {t}} y ^ {(i)} \mathbf {x} ^ {(i) \top} \mathbf {w} _ {t}\right)}{\sum_ {j = 1} ^ {n} \exp \left(- \frac {\alpha_ {t}}{\beta_ {t}} y ^ {(j)} \mathbf {x} ^ {(j) \top} \mathbf {w} _ {t}\right)},
$$
$$
\mathbf {A} ^ {\top} \mathbf {p} _ {t} = - \frac {\nabla L \left(\frac {\alpha_ {t}}{\beta_ {t}} \mathbf {w} _ {t}\right)}{L \left(\frac {\alpha_ {t}}{\beta_ {t}} \mathbf {w} _ {t}\right)}.
$$
Combining the equations above and replace  $\sum_{k=1}^{t} \alpha_k \mathbf{w}_k$  with  $\mathbf{v}_t$ , we get
$$
\nabla \Phi (\mathbf {v} _ {t}) = \nabla \Phi (\mathbf {v} _ {t - 1}) - \alpha_ {t} \frac {\nabla L (\mathbf {v} _ {t - 1})}{L (\mathbf {v} _ {t - 1})} - \alpha_ {t - 1} \left(\frac {\nabla L (\mathbf {v} _ {t - 1})}{L (\mathbf {v} _ {t - 1})} - \frac {\nabla L (\mathbf {v} _ {t - 2})}{L (\mathbf {v} _ {t - 2})}\right).
$$
The proof is finished by setting  $\alpha_{t} = t$
Next, we focus on the regret. For the  $\mathbf{w}$ -player, Note that  $h_t(\mathbf{w})$  is  $(q - 1)$ -strongly convex wrt the  $\| \cdot \|$ -norm. Let  $\widehat{\mathbf{w}}_t = \underset {\mathbf{w} \in \mathbb{R}^d}{\mathrm{argmin}} \sum_{i=1}^{t-1} \alpha_i h_i(\mathbf{w})$ . Then, based on Lemma 4, we have
$$
\begin{array}{l} \operatorname {R e g} _ {T} ^ {\mathbf {w}} \leq \sum_ {t = 1} ^ {T} \alpha_ {t} \left(h _ {t} (\mathbf {w} _ {t}) - h _ {t} \left(\widehat {\mathbf {w}} _ {t + 1}\right) - h _ {t - 1} (\mathbf {w} _ {t}) + h _ {t - 1} \left(\widehat {\mathbf {w}} _ {t + 1}\right)\right) \\ - \sum_ {t = 1} ^ {T} \frac {\sum_ {i = 1} ^ {t} \alpha_ {i} (q - 1)}{2} \| \mathbf {w} _ {t} - \widehat {\mathbf {w}} _ {t + 1} \| _ {q} ^ {2} \\ = \sum_ {t = 1} ^ {T} \alpha_ {t} (\mathbf {p} _ {t} - \mathbf {p} _ {t - 1}) ^ {\top} \mathbf {A} (\mathbf {w} _ {t} - \widehat {\mathbf {w}} _ {t + 1}) - \sum_ {t = 1} ^ {T} \frac {\sum_ {i = 1} ^ {t} \alpha_ {i} (q - 1)}{2} \| \mathbf {w} _ {t} - \widehat {\mathbf {w}} _ {t + 1} \| _ {q} ^ {2} \\ \leq \sum_ {t = 1} ^ {T} \alpha_ {t} \| \left(\mathbf {p} _ {t} - \mathbf {p} _ {t - 1}\right) ^ {\top} \mathbf {A} \| _ {p} \| \mathbf {w} _ {t} - \widehat {\mathbf {w}} _ {t + 1} \| _ {q} - \sum_ {t = 1} ^ {T} \frac {\sum_ {i = 1} ^ {t} \alpha_ {i} (q - 1)}{2} \| \mathbf {w} _ {t} - \widehat {\mathbf {w}} _ {t + 1} \| _ {q} ^ {2} \tag {32} \\ \leq \sum_ {t = 1} ^ {T} \frac {\alpha_ {t} ^ {2}}{2 \sum_ {i = 1} ^ {t} \alpha_ {i} (q - 1)} \| \mathbf {A} ^ {\top} (\mathbf {p} _ {t} - \mathbf {p} _ {t - 1}) \| _ {p} ^ {2} + \frac {\sum_ {i = 1} ^ {t} \alpha_ {i} (q - 1)}{2} \| \mathbf {w} _ {t} - \widehat {\mathbf {w}} _ {t + 1} \| _ {q} ^ {2} \\ - \sum_ {t = 1} ^ {T} \frac {\sum_ {i = 1} ^ {t} \alpha_ {i} (q - 1)}{2} \| \mathbf {w} _ {t} - \widehat {\mathbf {w}} _ {t + 1} \| _ {q} ^ {2} \\ = \frac {1}{2 (q - 1)} \sum_ {t = 1} ^ {T} \frac {\alpha_ {t} ^ {2}}{\sum_ {i = 1} ^ {t} \alpha_ {i}} \left\| \sum_ {i = 1} ^ {n} \left(p _ {t, i} - p _ {t - 1, i}\right) y ^ {(i)} \mathbf {x} ^ {(i)} \right\| _ {p} ^ {2} \\ \leq \frac {1}{2 (q - 1)} \sum_ {t = 1} ^ {T} \frac {\alpha_ {t} ^ {2}}{\sum_ {i = 1} ^ {t} \alpha_ {i}} \| \mathbf {p} _ {t} - \mathbf {p} _ {t - 1} \| _ {1} ^ {2}, \\ \end{array}
$$
where the first inequality is based on Hölder's inequality, the second inequality is based on Young's inequality, i.e.,
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} \alpha_ {t} \| (\mathbf {p} _ {t} - \mathbf {p} _ {t - 1}) ^ {\top} \mathbf {A} \| _ {p} \| \mathbf {w} _ {t} - \widehat {\mathbf {w}} _ {t + 1} \| _ {q} \\ \leq \sum_ {t = 1} ^ {T} \frac {\alpha_ {t} ^ {2}}{2 \sum_ {i = 1} ^ {t} \alpha_ {i} (q - 1)} \| \mathbf {A} ^ {\top} \left(\mathbf {p} _ {t} - \mathbf {p} _ {t - 1}\right) \| _ {p} ^ {2} + \frac {\sum_ {i = 1} ^ {t} \alpha_ {i} (q - 1)}{2} \| \mathbf {w} _ {t} - \widehat {\mathbf {w}} _ {t + 1} \| _ {q} ^ {2} \tag {33} \\ \end{array}
$$
For the  $\mathbf{p}$  -player, we have
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} \alpha_ {t} \mathbf {p} _ {t} ^ {T} \mathbf {A} \mathbf {w} _ {t} - \min  _ {\mathbf {p} \in \Delta^ {n}} \alpha_ {t} \mathbf {p} ^ {\top} \mathbf {A} \mathbf {w} _ {t} \\ = \sum_ {t = 1} ^ {T} \left(\alpha_ {t} \mathbf {p} _ {t} ^ {\top} \mathbf {A} \mathbf {w} _ {t} + \beta_ {t} D _ {\mathrm {K L}} \left(\mathbf {p} _ {t}, \frac {1}{n}\right) - \min  _ {\mathbf {p} \in \Delta^ {n}} \sum_ {t = 1} ^ {T} \alpha_ {t} \mathbf {p} ^ {\top} \mathbf {A} \mathbf {w} _ {t} - \sum_ {t = 1} ^ {T} \beta_ {t} D _ {E} \left(\mathbf {p} _ {t}, \frac {1}{n}\right)\right. \tag {34} \\ \leq 2 \sum_ {t = 1} ^ {T} \beta_ {t} \log n = 2 \sum_ {t = 1} ^ {T} \frac {\alpha_ {t}}{\sum_ {i = 1} ^ {t} \alpha_ {i}} \log n. \\ \end{array}
$$
Finally, we focus on the margin and implicit bias. Since  $\alpha_{t} = t$ , for the w-player's regret, we have
$$
\frac {1}{2 (q - 1)} \sum_ {t = 1} ^ {T} \frac {\alpha_ {t} ^ {2}}{\sum_ {i = 1} ^ {t} \alpha_ {i}} \| \mathbf {p} _ {t} - \mathbf {p} _ {t - 1} \| _ {1} ^ {2} \leq \frac {1}{(q - 1)} \sum_ {t = 1} ^ {T} \| \mathbf {p} _ {t} - \mathbf {p} _ {t - 1} \| _ {1} ^ {2},
$$
and for the  $\mathbf{p}$  -player, we have
$$
\sum_ {t = 1} ^ {T} \frac {\alpha_ {t}}{\sum_ {i = 1} ^ {t} \alpha_ {i}} \log n \leq 4 \log T \log n.
$$
Following Theorem 1, we have the margin and implicit bounds when
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} \alpha_ {t} = \frac {T (T + 1)}{2} \geq \frac {T ^ {2}}{2} \geq \frac {4}{\gamma^ {2}} \left(4 \log n \log T + \frac {2 T}{(q - 1)}\right) \tag {35} \\ \geq \frac {4}{\gamma^ {2}} \left(4 \log n \log T + \frac {1}{(q - 1)} \sum_ {t = 1} ^ {T} \| \mathbf {p} _ {t} - \mathbf {p} _ {t - 1} \| _ {1} ^ {2}\right), \\ \end{array}
$$
since the RHS is exactly  $\frac{\gamma^2}{4} (\mathrm{Reg}_T^{\mathbf{p}} + \mathrm{Reg}_T^{\mathbf{w}})$ .
D Omitted Proof in Section 4.2
In this section, we provide the proof related to the steepest descent algorithm. We first restate Algorithm 3, which is presented in Algorithm 6. Here, we provide two online dynamic under the game framework. They both are equivalent to the steepest descent algorithm in the left box, in the sense that  $\mathbf{v}_T = \widetilde{\mathbf{w}}_T$ . The left one is good for recovering the results in Nacson et al. (2019), while the bottom one is more suitable for analysing our accelerated rates.
We first recover the results of Nacson et al. (2019) in Theorem 8, and then proof our Theorem 5.
Algorithm 6 Steepest Descent [Recall  $\ell_t(\mathbf{p}) = g(\mathbf{p},\mathbf{w}_t)$ , and  $h_t(\mathbf{w}) = -g(\mathbf{p}_t,\mathbf{w})$ ]
1: for  $t = 1, \dots, T$  do
2:  $\mathbf{s}_{t - 1} = \mathrm{argmin}_{\| \mathbf{s}\| \leq 1}\mathbf{s}^{\top}\nabla L(\mathbf{v}_{t - 1})$
3:  $\mathbf{v}_t = \mathbf{v}_{t - 1} + \eta_{t - 1}\mathbf{s}_{t - 1}$
4: end for
5: Output:  $\mathbf{v}_T$
$\mathbf{p}$  -player:  $\mathbf{p}_t = \underset {\mathbf{p}\in \Delta^n}{\mathrm{argmin}}\sum_{i = 1}^{t - 1}\alpha_i\ell_i(\mathbf{p}) + D_{\mathrm{KL}}\left(\mathbf{p},\frac{\mathbf{1}}{n}\right)$
w-player:  $\mathbf{w}_t = \mathrm{argmin}_{\mathbf{w}\in \mathbb{R}^d}\alpha_th_t(\mathbf{w})$