SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In this section, we show that average mirror descent and steepest descent can find their equivalent online learning forms under Protocol 1. Thus, their margin maximization rates can be directly characterized by the corresponding average regret  $C_T$ . For clarity, we use  $\mathbf{v}_t$  to denote the classifier updates in the original methods, and  $\mathbf{w}_t$  the update under the game framework. Note that Theorem 1 clearly implies that the convergence rate of the directional error is always a square-root worse than that of the margin maximization rate. Due to space limitations, we only present the margin maximization rates, while the corresponding rates on directional errors are presented in the appendix.
4.1 Mirror-Descent-Type of Methods
First, we consider minimizing (1) by applying the following mirror descent algorithm:
$$
\mathbf {v} _ {t} = \underset {\mathbf {v} \in \mathbb {R} ^ {d}} {\operatorname {a r g m i n}} \eta_ {t} \left\langle \nabla L \left(\mathbf {v} _ {t - 1}\right), \mathbf {v} \right\rangle + D _ {\Phi} (\mathbf {v}, \mathbf {v} _ {t - 1}), \tag {4}
$$
where  $D_{\Phi}(\mathbf{v}, \mathbf{v}_{t-1})$  is the Bregman divergence between  $\mathbf{v}$  and  $\mathbf{v}_{t-1}$ , and  $\Phi(\mathbf{v})$  is a strongly convex potential function that defines the mirror map. Note that since the feasible domain in (4) is unbounded, we can rewrite the algorithm in the following form:
$$
\nabla \Phi (\mathbf {v} _ {t}) = \nabla \Phi (\mathbf {v} _ {t - 1}) - \eta_ {t} \nabla L (\mathbf {v} _ {t - 1}).
$$
In this paper, we consider weighted-average mirror descent with the squared  $q$ -norm, i.e.,  $\Phi(\mathbf{w}) = \frac{1}{2} \| \mathbf{w} \|_q^2$ , where  $q \in (1, 2]$ , and demonstrate that this optimization algorithm can enable faster  $\| \cdot \|$ -margin maximization rates. The detailed update rule is summarized in the left box of Algorithm 1. It is worth noting that this type of regularizer is  $(q - 1)$ -strongly convex with respect to  $\| \cdot \|_q$ , and can be updated efficiently in closed form as below<sup>1</sup>: for each coordinate  $i \in [d]$ , we have
$$
\begin{array}{l} \widehat {v} _ {t, i} = \mathrm {s i g n} (v _ {t - 1, i}) | v _ {t - 1, i} | ^ {q - 1} \| \mathbf {v} _ {t - 1} \| _ {q} ^ {2 - q} - \eta_ {t} [ \nabla L (\mathbf {v} _ {t - 1}) ] _ {i}, \\ v _ {t, i} = \operatorname {s i g n} \left(\widehat {v} _ {t, i}\right) | \widehat {v} _ {t, i} | ^ {p - 1} \| \widehat {\mathbf {v}} _ {t} \| _ {p} ^ {2 - p}. \\ \end{array}
$$
We make a few final observations about this algorithm: 1) Instead of using the weighted sum  $\widetilde{\mathbf{v}}_T$ , we could output the weighted average  $\frac{\widetilde{\mathbf{v}}_t}{\sum_{s=1}^t \alpha_s}$  without altering the margin or directional convergence rate. This is attributed to the scale-invariance of the margin, i.e.,  $\forall c > 0$ ,  $\mathbf{w} \in \mathbb{R}^d$ ,  $\widetilde{\gamma}(\mathbf{w}) = \widetilde{\gamma}(c\mathbf{w})$ . The same argument applies to the directional error. 2) The use of the weighted average is standard in the analysis of mirror descent (e.g., Section 4.2 of Bubeck et al. (2015)). This paper shows that using non-uniform weights is advantageous for achieving rapid margin maximization rates; 3) The per-round computational complexity of (5) is  $O(d)$ , which is similar to that of  $p$ -mirror-descent of Sun et al. (2022). However, we note that the  $p$ -MD algorithm of Sun et al. (2022) does not need to compute the norm of the decision at each round, which could be more efficient in real-world applications where parallel or distributed computation is desired.
For Algorithm 1, we have the following theorem. We present its proof in Appendix C, along with a more general theorem that allows a general configuration of the parameters  $\eta_t$ ,  $\alpha_t$  and  $\beta_t$ .
Theorem 2. Suppose Assumption 1 holds wrt  $\| \cdot \| _q$ -norm for  $q\in (1,2]$ . For the left box of Algorithm 1, let  $\eta_t = \frac{1}{L(\mathbf{v}_{t - 1})}$ . For the right box, let  $\alpha_{t} = 1$ , and  $\beta_{1} = 1$ ,  $\beta_{t} = \frac{1}{t - 1}$ . Then the methods in the two boxes of Algorithm 1 are identical, in the sense that  $\widetilde{\mathbf{v}}_T = \widetilde{\mathbf{w}}_T$ . Moreover, we have the average
This expression appears in (Section 6.7, Arabona, 2019) and we reproduce it for completeness.
Algorithm 1 Mirror Descent [Recall  $\ell_t(\mathbf{p}) = g(\mathbf{p},\mathbf{w}_t)$ , and  $h_t(\mathbf{w}) = -g(\mathbf{p}_t,\mathbf{w})$ ]
1: for  $t = 1, \dots, T$  do
2:  $\nabla \Phi (\mathbf{v}_t) = \nabla \Phi (\mathbf{v}_{t - 1}) - \eta_t\nabla L(\mathbf{v}_{t - 1})$
3: end for
4: Output:  $\widetilde{\mathbf{v}}_T = \sum_{t=1}^{T} \frac{1}{t} \mathbf{v}_t$
$\mathbf{p}$  -player:  $\mathbf{p}_t = \underset {\mathbf{p}\in \Delta^n}{\mathrm{argmin}}\alpha_t\ell_{t - 1}(\mathbf{p}) + \beta_tD_{\mathrm{KL}}\left(\mathbf{p},\frac{\mathbf{1}}{n}\right)$
w-player:  $\mathbf{W}_t = \underset {\mathbf{w}\in \mathbb{R}^d}{\mathrm{argmin}}\sum_{j = 1}^{t}\alpha_jh_j(\mathbf{w})$
Output:  $\widetilde{\mathbf{w}}_T = \sum_{t=1}^T \alpha_t \mathbf{w}_t$
regret upper bound  $C_T = \frac{\left(\frac{2}{q - 1} + 2\log n\right)(\log T + 2)}{T}$ . Therefore, the algorithm achieves a positive margin when  $T$  is sufficiently large such that  $T \geq \frac{4\left(\frac{2}{q - 1} + 2\log n\right)(\log T + 2)}{\gamma^2}$ . We have the convergence rate
$$
\frac {\min  _ {\mathbf {p} \in \Delta^ {n}} \mathbf {p} ^ {\top} \mathbf {A} \widetilde {\mathbf {v}} _ {T}}{\| \widetilde {\mathbf {v}} _ {T} \| _ {q}} \geq \gamma - \frac {4 \left(\frac {2}{q - 1} + 2 \log n\right) (\log T + 2)}{\gamma^ {2} T} = \gamma - O \left(\frac {\log n \log T}{(q - 1) \gamma^ {2} T}\right), \tag {6}
$$
$$
\left\| \frac {\widetilde {\mathbf {v}} _ {T}}{\| \widetilde {\mathbf {v}} _ {T} \| _ {q}} - \mathbf {w} _ {\| \cdot \| _ {q}} ^ {*} \right\| _ {q} \leq \frac {8 \sqrt {2}}{\gamma^ {2} \sqrt {(q - 1)}} \sqrt {\frac {\left(\frac {2}{q - 1} + 2 \log n\right) (\log T + 2)}{T}} = O \left(\frac {\sqrt {\log n \log T}}{\gamma^ {2} (q - 1) \sqrt {T}}\right).
$$
The first part of Theorem 2 indicates that the mirror descent algorithm can be described as two players using certain cleverly designed online learning algorithms to solve the regularized bilinear game in (2). More specifically, for the  $\mathbf{p}$ -player, we propose a new and unusual online learning algorithm, which we call regularized greedy.
$$
\mathbf {p} _ {t} = \operatorname * {a r g m i n} _ {\mathbf {p} \in \Delta^ {n}} \alpha_ {t} \ell_ {t - 1} (\mathbf {p}) + \beta_ {t} D _ {\mathrm {K L}} \left(\mathbf {p}, \frac {\mathbf {1}}{n}\right).
$$
Essentially, in round  $t$ , the p-player minimizes the previous round's loss function,  $\ell_{t-1}$ , plus a regularizer at round  $t$ , and the two terms are balanced by the parameters  $\alpha_t$  and  $\beta_t$ . On the other hand, we select the follow-the-leader+ algorithm for the w-player:
$$
\mathbf{w}_{t} = \operatorname *{argmin}_{\mathbf{w}\in \mathbb{R}^{d}}\sum_{j = 1}^{t}\alpha_{j}h_{j}(\mathbf{w}),
$$
which returns the solution that minimizes the cumulative loss so far. The  $+$  sign in the name is because the algorithm can pick the decision  $\mathbf{w}_t$  after seeing its loss function. This is an interesting and unusual design because the regularized greedy algorithm will clearly suffer a worst-case linear regret for the p-player. Fortunately, for our specific case we are able to prove a sharper data-dependent regret bound for the p-player as below:
$$
\operatorname {R e g} _ {T} ^ {\mathbf {p}} = O \left(\sum_ {t = 2} ^ {T} \frac {(t - 1) (q - 1)}{2} \| \mathbf {w} _ {t} - \mathbf {w} _ {t - 1} \| _ {q} ^ {2} + \log n \log T\right),
$$
Therefore, the dominating term (i.e., the first term above) of the p-player's regret bound can be canceled by the w-player's regret bound, given by:
$$
\operatorname {R e g} _ {T} ^ {\mathbf {w}} = O \left(- \sum_ {t = 2} ^ {T} \frac {(t - 1) (q - 1)}{2} \| \mathbf {w} _ {t} - \mathbf {w} _ {t - 1} \| _ {q} ^ {2}\right).
$$
Note that the w-player's regret bound is negative as the corresponding algorithm used is clairvoyant, i.e. can see the current loss  $\ell_t$  before making a decision at round  $t$ . This ensures that sublinear (and more generally fast) rates are possible. Note that  $\beta_{t}$  and  $\alpha_{t}$  will influence both the regret bound and the algorithm equivalence analysis, so finding the right parameter configuration that works for both is a non-trivial task. We make the choice  $\beta_{t} = \frac{\alpha_{t}}{\sum_{i = 1}^{t - 1}\alpha_{i}}$ , which ensures both algorithmic equivalence and sublinear regret bounds.
Algorithm 2 Momentum-based MD [Recall  $\ell_t(\mathbf{p}) = g(\mathbf{p},\mathbf{w}_t)$  , and  $h_t(\mathbf{w}) = -g(\mathbf{p}_t,\mathbf{w})]$
$$
- \left(\widehat {\eta} _ {t} \nabla L \left(\mathbf {v} _ {t - 1}\right) - \eta_ {t - 1} \nabla L \left(\mathbf {v} _ {t - 2}\right)\right)
$$
1: for  $t = 1, \dots, T$  do
2:  $\nabla \Phi (\mathbf{v}_t) = \nabla \Phi (\mathbf{v}_{t - 1}) - \eta_t\nabla L(\mathbf{v}_{t - 1})$
3: end for
4: Output:  $\widetilde{\mathbf{v}}_T = \sum_{t=1}^T \frac{1}{t} \mathbf{v}_t$
w-player:
$$
\mathbf {w} _ {t} = \operatorname {a r g m i n} _ {\mathbf {w} \in \mathbb {R} ^ {d}} \sum_ {i = 1} ^ {t - 1} \alpha_ {i} h _ {i} (\mathbf {w}) + \alpha_ {t} h _ {t - 1} (\mathbf {w})
$$
p-player:
$$
\mathbf {p} _ {t} = \operatorname * {a r g m i n} _ {\mathbf {p} \in \Delta^ {n}} \alpha_ {t} \ell_ {t} (\mathbf {p}) + \beta_ {t} D _ {\mathrm {K L}} (\mathbf {p}, \mathbf {p} _ {0})
$$
The second part of Theorem 2 shows that the average regret  $C_T$  of Algorithm 1 is on the order of  $O\left(\frac{\log n\log T}{(q - 1)\gamma^2T}\right)$ . Therefore, by plugging in Theorem 1, we observe that the margin shrinks on the order of  $\gamma - O\left(\frac{\log n\log T}{\gamma^2(q - 1)T}\right)$ , and the implicit bias convergence rate is  $O\left(\frac{\log n\log T}{\gamma^2(q - 1)\sqrt{T}}\right)$ . Next, we show an improved rate with a more aggressive step size on the order of  $O\left(\frac{t}{L(\mathbf{v}_t)}\right)$  instead of  $O\left(\frac{1}{L(\mathbf{v}_t)}\right)$ . The proof of this result is given in Appendix C.
Theorem 3. Suppose Assumption 1 holds wrt  $\| \cdot \|_q$ -norm for  $q \in (1,2]$ . For the left box of Algorithm 1, let  $\eta_t = \frac{t}{L(\mathbf{v}_{t-1})}$ , and let the final output be  $\widetilde{\mathbf{v}}_T = \sum_{t=1}^T \frac{2}{t+1} \mathbf{v}_t$ . For the right box, let  $\alpha_t = t$ , and  $\beta_1 = 1$ ,  $\beta_t = \frac{2}{t-1}$ . Then the two algorithms are identical, in the sense that  $\widetilde{\mathbf{v}}_T = \widetilde{\mathbf{w}}_T$ . Moreover, when  $T \geq \sqrt{\frac{8\left[\frac{4T}{q-1} + 4 \log n \log T + 1 + 2 \log n\right]}{\gamma^2}}$ , we have
$$
\frac {\underset {\mathbf {p} \in \Delta^ {n}} {\min } \mathbf {p} ^ {\top} \mathbf {A} \widetilde {\mathbf {v}} _ {T}}{\| \widetilde {\mathbf {v}} _ {T} \| _ {q}} \geq \gamma - \frac {3 2}{\gamma^ {2} T (q - 1)} - \frac {8 (4 \log n \log T + 1 + 2 \log n)}{\gamma^ {2} T ^ {2}}, \tag {7}
$$
$$
\left\| \frac {\widetilde {\mathbf {v}} _ {T}}{\| \widetilde {\mathbf {v}} _ {T} \| _ {q}} - \mathbf {w} _ {\| \cdot \| _ {q}} ^ {*} \right\| _ {q} \leq \frac {8 \sqrt {2}}{\gamma^ {2} \sqrt {q - 1}} \sqrt {\frac {8}{(q - 1) T} + \frac {4 \log n \log T + 2 \log n + 1}{T ^ {2}}}.
$$
Observe that the margin maximization rate in Theorem 3 is  $O\left(\frac{1}{(q - 1)\gamma^2T}\right) + O\left(\frac{\log n\log T}{\gamma^2T^2}\right)$ . Compared to (6), it has a better dependence on  $\log n$  and  $\log T$ .
Finally, we focus on a momentum-based mirror descent algorithm, which is given in Algorithm 2. For this algorithm, we have the following guarantee.
Theorem 4. Suppose Assumption 1 holds wrt  $\| \cdot \|_q$ -norm for  $q \in (1,2]$ . For the left box of Algorithm 2, let  $\eta_t = \frac{t}{L(\mathbf{v}_{t-1})}$ , and  $\widehat{\eta}_t = \frac{t-1}{L(\mathbf{v}_{t-1})}$ . For the second box, let  $\alpha_t = t$ , and  $\beta_t = \frac{2}{t+1}$ . Then the methods in the two boxes of Algorithm 2 are identical, in the sense that  $\widetilde{\mathbf{v}}_T = \widetilde{\mathbf{w}}_T$ . Moreover, when  $T \geq \frac{\sqrt{8(4\log n\log T + \frac{2T}{q-1})}}{\gamma}$ , we have
$$
\frac {\underset {\mathbf {p} \in \Delta^ {n}} {\min } \mathbf {p} ^ {\top} \mathbf {A} \widetilde {\mathbf {v}} _ {T}}{\| \widetilde {\mathbf {v}} _ {T} \| _ {q}} \geq \gamma - \frac {\sum_ {t = 1} ^ {T} \| \mathbf {p} _ {t} - \mathbf {p} _ {t - 1} \| _ {1} ^ {2}}{\gamma^ {2} (q - 1) T ^ {2}} - \frac {3 2 \log n \log T}{\gamma^ {2} T ^ {2}} \tag {8}
$$
$$
\left\| \frac {\widetilde {\mathbf {v}} _ {T}}{\| \widetilde {\mathbf {v}} _ {T} \| _ {q}} - \mathbf {w} _ {\| \cdot \| _ {q}} ^ {*} \right\| _ {q} \leq \frac {8 \sqrt {2}}{\gamma^ {2} \sqrt {(q - 1)}} \sqrt {\frac {\sum_ {t = 1} ^ {T} \| \mathbf {p} _ {t} - \mathbf {p} _ {t - 1} \| _ {1} ^ {2}}{(q - 1) T ^ {2}} + \frac {3 2 \log n \log T}{T ^ {2}}}.
$$
The above theorem shows that, for sufficiently large  $T$ , the margin maximization rate can be data-dependent. Note that  $\sum_{t=1}^{T} \| \mathbf{p}_t - \mathbf{p}_{t-1} \|_1^2 \leq 2T$ , so in the worst case, the bound reduces to the results in Theorem 3, but it can become significantly better when  $\sum_{t=1}^{T} \| \mathbf{p}_t - \mathbf{p}_{t-1} \|_1^2$  is small. We expect that when  $T$  is very large,  $\mathbf{p}_T$  will change very slowly as we already know that the direction of  $\widetilde{\mathbf{v}}_T$  will converge — however, turning this into a precise faster rate dependent on the original training data geometry, i.e.  $\mathbf{A}$ , is an intriguing open question.
Algorithm 3 Steepest Descent [Recall  $\ell_t(\mathbf{p}) = g(\mathbf{p},\mathbf{w}_t)$ , and  $h_t(\mathbf{w}) = -g(\mathbf{p}_t,\mathbf{w})$ ]
1: for  $t = 1, \dots, T$  do
2:  $\mathbf{s}_{t - 1} = \underset {\| \mathbf{s}\| \leq 1}{\mathrm{argmin}}\mathbf{s}^{\top}\nabla L(\mathbf{v}_{t - 1})$
3:  $\mathbf{v}_t = \mathbf{v}_{t - 1} + \eta_{t - 1}\mathbf{s}_{t - 1}$
4: end for
5: Output:  $\mathbf{v}_T$
$\mathbf{w}$  -player:  $\mathbf{w}_t = \underset {\mathbf{w}\in \mathbb{R}^d}{\mathrm{argmin}}\langle \delta_{t - 1}\alpha_{t - 1}\nabla h_{t - 1}(\mathbf{w}_{t - 1}),\mathbf{w}\rangle$
$$
+ D _ {\frac {1}{2} \parallel . \parallel 2} (\mathbf {w}, \mathbf {w} _ {t - 1})
$$
p-player:  $\mathbf{p}_t = \underset {\mathbf{p}\in \Delta^n}{\mathrm{argmin}}\sum_{i = 1}^t\alpha_i\ell_i(\mathbf{p}) + D_{\mathrm{KL}}\left(\mathbf{p},\frac{\mathbf{1}}{n}\right)$
Output:  $\widetilde{\mathbf{w}}_T = \sum_{t=1}^{T} \alpha_t \mathbf{w}_t$ .
4.2 Steepest Descent
Next, we consider the steepest descent method under a given general norm  $\| \cdot \|$ . For a succinct description of this algorithm see (Boyd & Vandenberghe, 2004). For completeness, we have also described this algorithm in the left box of Algorithm 3. In each iteration  $t$ , the algorithm first identifies the steepest direction with respect to the norm  $\| \cdot \|$  (Step 2). It then adjusts the decision towards this direction using a specific step size  $\eta_t$  (Step 3). After  $T$  iterations, the algorithm yields the final iteration  $\mathbf{v}_T$ . In the following, we show that an  $O\left(\frac{\lambda + \log n}{\gamma^2\lambda T}\right)\| \cdot \|$ -margin maximization rate can be achieved when the squared-norm, i.e.  $\frac{1}{2}\| \cdot \| ^2$  is  $\lambda$ -strongly convex (e.g.,  $\frac{1}{2}\| \cdot \| _q^2$  is  $(q - 1)$ -strongly convex wrt  $\| \cdot \| _q$ ). The proof of this result provided in Appendix D. Moreover, we recover the slower  $O\left(\frac{\log n + \log T}{\sqrt{T}}\right)$  of Nacson et al. (2019) rate as a special case for norms that are not necessarily strongly convex.
Theorem 5. Suppose Assumption 1 holds wrt a general norm  $\| \cdot \|$ , and  $\frac{1}{2} \| \cdot \|^{2}$  is  $\lambda$ -strongly convex wrt  $\| \cdot \|$ . Let  $\eta_{t} = \frac{\alpha_{t} \| \nabla L(\mathbf{w}_{t}) \|}{L(\mathbf{w}_{t})}$ , and  $\delta_{t-1} = \alpha_{t-1}$ . Then the methods in the two boxes of Algorithm 3 are equivalent, in the sense that  $\mathbf{v}_{T} = \widetilde{\mathbf{w}}_{T}$ . Moreover, let  $\alpha_{t} = \frac{\lambda}{2}$ . Then  $C_{T} = \frac{\frac{\lambda}{4} + \log n}{T\lambda}$ . Therefore, when  $T \geq \frac{\lambda + 4\log n}{\lambda\gamma^{2}}$ , we have
$$
\frac {\underset {\mathbf {p} \in \Delta^ {n}} {\min } \mathbf {p} ^ {\top} \mathbf {A} \mathbf {v} _ {T}}{\| \mathbf {v} _ {T} \|} \geq \gamma - \frac {\lambda + 4 \log n}{\gamma^ {2} T \lambda},
$$
$$
\left\| \frac {\mathbf {v} _ {T}}{\| \mathbf {v} _ {T} \|} - \mathbf {w} _ {\| \cdot \|} ^ {*} \right\| \leq \frac {4 \sqrt {2}}{\sqrt {\lambda} \gamma^ {2}} \sqrt {\frac {\lambda + 4 \log n}{T \lambda}}.
$$
The first part of Theorem 5 elucidates the equivalent online dynamic of the steepest descent algorithm, which is also depicted in the right box of Algorithm 3. The  $\mathbf{w}$ -player employs the standard online mirror descent (OMD) algorithm (Hazan, 2016), while the  $\mathbf{p}$ -player utilizes  $\mathrm{FTRL}^+$ , i.e.,  $\mathbf{p}_t$  is selected by minimizing the cumulative loss observed so far, coupled with a regularization term. The crux of our algorithm equivalence analysis lies in evaluating the output of the  $\mathbf{w}$ -player. For this, we initially prove that given  $\delta_t = \frac{1}{\alpha_t}$ , the OMD algorithm condenses to best-response (BR), that is,  $\mathbf{w}_t = \operatorname*{argmin}_{\mathbf{w} \in \mathbb{R}^d} \alpha_t h_{t-1}(\mathbf{w})$ . We then prove that BR's output coincides with the steepest descent direction. The second part of Theorem 5 shows that the average regret of this online learning dynamic is  $O\left(\frac{\lambda + \log n}{\gamma^2 \lambda T}\right)$ , which leads to the corresponding fast margin maximization/small direction error. We note that the favorable average regret is made possible by allowing the two players to play against each other, rather than plugging in worst-case regret bounds.
4.3 Even Faster Rates with Accelerated Generic Methods
In the preceding subsections, we showed that, with suitable step sizes, steepest descent and average mirror descent can achieve an  $O\left(\frac{\log n\log T}{T}\right)$  margin maximization rate. We now aim to derive even faster rates using two approaches, as illustrated in the top two boxes of Algorithm 4. The left box introduces a Nesterov-acceleration-based mirror descent (Nesterov, 1988; Tseng, 2008): In each iteration  $t$ , the algorithm initially performs an extra update to yield  $\mathbf{v}_t$  (Step 2), then executes a mirror descent step with the gradient at  $\mathbf{v}_t$  (Step 3), and finally calculates a moving average (Step 4). On the other hand, the right box depicts a momentum-based steepest descent algorithm: In each iteration,
Algorithm 4 Accelerated Methods [Recall  $\ell_t(\mathbf{p}) = g(\mathbf{p},\mathbf{w}_t)$ , and  $h_t(\mathbf{w}) = -g(\mathbf{p}_t,\mathbf{w})$ ]
1: for  $t = 1,\dots ,T$  do 1: for  $t = 1,\dots ,T$  do 2:  $\mathbf{g}_t = \beta_{t,3}\mathbf{g}_{t - 1} + \beta_{t,3}'\nabla L(\beta_{t,4}\mathbf{v}_{t - 1} + \beta_{t,4}'\mathbf{s}_{t - 1})$  3:  $\nabla \Phi (\mathbf{z}_t) = \nabla \Phi (\mathbf{z}_{t - 1}) - \eta_t\nabla L(\mathbf{v}_t)$  4:  $\widetilde{\mathbf{v}}_t = \beta_{t,2}\widetilde{\mathbf{v}}_{t - 1} + \beta_{t,2}'\mathbf{z}_t$  5: end for 6: Output:  $\widetilde{\mathbf{v}}_T$  p-player:  $\mathbf{p}_t = \mathrm{argmin}_{\mathbf{p}\in \Delta^n}\sum_{i = 1}^{t - 1}\alpha_i\ell_i(\mathbf{p}) + \alpha_t\ell_{t - 1}(\mathbf{p}) + \frac{1}{c} D_{\mathrm{KL}}(\mathbf{p},\mathbf{p}_0)$  w-player:  $\mathbf{w}_t = \mathrm{argmin}_{\mathbf{w}\in \mathbb{R}^d}\sum_{i = 1}^{t}\alpha_ih_i(\mathbf{w})$  Output:  $\widetilde{\mathbf{w}}_T = \sum_{t = 1}^T\alpha_tw_t$
the method maintains a momentum term  $\mathbf{g}_t$  with an additional gradient (Step 2), then identifies the steepest direction with respect to  $\mathbf{g}_t$  (Step 3), and applies this direction to update the decision (Step 4). At first glance, these two algorithms appear markedly different. However, we show that with appropriately chosen parameters, they are actually equivalent, in the sense that they both correspond to the online dynamic in the bottom box of Algorithm 4. More specifically, we provide the following theoretical guarantee. The proof is given in Appendix E.
Theorem 6. Suppose Assumption 1 holds wrt a general norm  $\| \cdot \|$ , and  $\frac{1}{2} \| \cdot \|^2$  is  $\lambda$ -strongly convex wrt  $\| \cdot \|$ . For the left box, let  $\beta_{t,1} = \frac{\lambda}{4}$ ,  $\beta_{t,1}' = \frac{\lambda}{2(t - 1)}$ ,  $\beta_{t,2} = 1$ ,  $\beta_{t,2}' = \frac{2}{t + 1}$ , and  $\eta_t = \frac{t}{L(\mathbf{v}_t)}$ . For the right box, let  $\beta_{t,3} = \frac{t - 1}{t + 1}$ ,  $\beta_{t,4} = \frac{\lambda}{4}$ ,  $\beta_{t,4}' = \frac{\lambda t \| \mathbf{g}_{t - 1} \|_*}{4}$ ,  $\beta_{t,3}' = \frac{2}{(t + 1)L(\beta_{t,4} \mathbf{v}_{t - 1} + \beta_{t,4}^{\prime} \mathbf{s}_{t - 1})}$ , and  $\eta_t = t \| \mathbf{g}_t \|_*$ . For the bottom box, let  $c = \frac{\lambda}{4}$ ,  $\alpha_t = t$ . Then all three methods in Algorithm 4 are identical, in the sense that  $\widetilde{\mathbf{v}}_T = \mathbf{v}_T = \widetilde{\mathbf{w}}_T$ . Moreover, when  $T \geq \frac{4\sqrt{2\log n}}{\sqrt{\lambda\gamma}}$ , we have
$$
\frac {\min  _ {\mathbf {p} \in \Delta^ {n}} \mathbf {p} ^ {\top} A \widetilde {\mathbf {w}} _ {T}}{\| \widetilde {\mathbf {w}} _ {T} \|} \geq \gamma - \frac {3 2 \log n}{\gamma^ {2} T ^ {2} \lambda},
$$
$$
\left\| \frac {\mathbf {v} _ {T}}{\| \mathbf {v} _ {T} \|} - \mathbf {w} _ {\| \cdot \|} ^ {*} \right\| \leq \frac {3 2 \sqrt {\log n}}{\gamma^ {2} \lambda T}.
$$
Remark Theorem 6 reveals that the two strategies implemented in Algorithm 4 yield an optimal  $O(\log n / [\gamma^2 T^2])$  rate. It is worth noting that a similar online dynamic to the one detailed in the bottom box of Algorithm 4 was also considered by (Algorithm 5, Wang et al., 2022b). Nonetheless, there are some crucial distinctions: 1) Their work only demonstrated that this dynamic could achieve a positive margin, leaving open questions regarding whether the margin can be maximized (i.e., converge to  $\gamma$ ), and if so, what the margin maximization rate would be; 2) They only presented the online dynamic, without its equivalent optimization form. Finally, we note that, Theorem 6 requires the norm to be strongly convex, which is satisfied for, e.g. the  $q$ -norm when  $q \in (1,2]$ .
This paper examines the implicit bias of generic optimization methods, delivering accelerated margin maximization and directional errors for average mirror descent and steepest descent. Our approach translates generic optimization methods for ERM into online learning dynamics for a regularized bilinear game, offering a simpler analysis and a fresh perspective on implicit bias. Despite the effectiveness of the game framework in handling generic methods and accelerated techniques, it presently holds some limitations: 1) the framework is currently operational only for exponential loss, making its extension to handle more general losses a vital area for future research; 2) identifying algorithmic equivalence is nuanced and non-trivial, and it is as yet unresolved whether this framework can elucidate other methods, such as the last-iterate of MD; 3) Finally, it remains to be seen whether more advanced online learning algorithms are beneficial under our framework, such as parameter-free online learning (Orabona & Pál, 2016; Cutkosky & Orabona, 2018) or adaptive online learning methods (van Erven & Koolen, 2016; Wang et al., 2019; Zhang et al., 2019; Wang et al., 2021a).
Acknowledgments. GW was supported by a ARC-ACO fellowship provided by Georgia Tech. VM was supported by the NSF (through CAREER award CCF-2239151 and award IIS-2212182), an Adobe Data Science Research Award, an Amazon Research Award and a Google Research Colabs Award. JA was supported by the AI4OPT Institute, as part of NSF Award 2112533, and NSF through Award IIS-1910077.