SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$\mathbf{w}$  -player:  $\mathbf{w}_t = \underset {\mathbf{w}\in \mathbb{R}^d}{\mathrm{argmin}}\left\langle \delta_{t - 1}\alpha_{t - 1}\nabla h_{t - 1}(\mathbf{w}_{t - 1}),\mathbf{w}\right\rangle +D_{\frac{1}{2}\| \cdot \| ^2}(\mathbf{w},\mathbf{w}_{t - 1})$
Output:  $\widetilde{\mathbf{w}}_T = \sum_{t=1}^T \alpha_t \mathbf{w}_t$ .
Theorem 8. Suppose Assumption 1 holds wrt a general norm  $\| \cdot \|$ . Let  $\eta_t = \frac{\alpha_t\|\nabla L(\mathbf{w}_{t-1})\|_*}{L(\mathbf{w}_{t-1})}$ . Then, the methods in the top two boxes of Algorithm 6 are equivalent, in the sense that  $\mathbf{v}_T = \widetilde{\mathbf{w}}_T$ . Moreover, let  $\alpha_t = \frac{1}{\sqrt{t}}$ . Then  $C_T = \frac{\log n + 2\log T + 2}{\sqrt{T}}$ . Therefore, when  $T$  is sufficiently large such that  $\sqrt{T} \geq \frac{4(\log n + 2\log T + 2)}{\gamma^2}$ , we have
$$
\frac {\min  _ {\mathbf {p} \in \Delta^ {n}} \mathbf {p} ^ {\top} \mathbf {A} \widetilde {\mathbf {w}} _ {T}}{\| \widetilde {\mathbf {w}} _ {T} \|} \geq \gamma - \frac {4 (2 + \log n + 2 \log T)}{\gamma^ {2} \sqrt {T}}. \tag {36}
$$
The first part of Theorem 8 shows that the steepest descent algorithm can be seen as an online learning dynamic where the in each round  $\mathbf{p}$ -player performs the standard follow-the-regularized-leader algorithm (Hazan, 2016), while the w-player uses best-response $^+$  (Wang et al., 2021b), meaning it picks the decision by directly minimizing the loss of the round. The second part of the theorem shows that the margin convergence rate, which recovers the  $\gamma - O\left(\frac{\log n + \log T}{\sqrt{T}}\right)$  rate of Nacson et al. (2019), while the convergence in terms of distance is new. Note that the strongly convex condition is not required for the margin maximization analysis, but is needed for the distance convergence analysis. Next, we restate Theorem 5, where an improved margin maximization rate when the strong convexity condition is met.
Theorem 9 (Restate of Theorem 5). Suppose Assumption 1 holds wrt a general norm  $\| \cdot \|$ , and  $\frac{1}{2}\| \cdot \| ^2$  is  $\lambda$ -strongly convex wrt  $\| \cdot \|$ . Let  $\eta_t = \frac{\alpha_t\|\nabla L(\mathbf{w}_t)\|}{L(\mathbf{w}_t)}$ . Then the methods in the first and third boxes of Algorithm 6 are equivalent, in the sense that  $\mathbf{v}_T = \widetilde{\mathbf{w}}_T$ . Moreover, let  $\alpha_{t} = \frac{\lambda}{2}$ . Then  $C_T = \frac{\frac{\lambda}{4} + \log n}{T\lambda}$ . Therefore, when  $T\geq \frac{\lambda + 4\log n}{\lambda\gamma^2}$ , we have
$$
\frac {\min _ {\mathbf {p} \in \Delta^ {n}} \mathbf {p} ^ {\top} \mathbf {A} \mathbf {v} _ {T}}{\| \mathbf {v} _ {T} \|} \geq \gamma - \frac {\lambda + 4 \log n}{\gamma^ {2} T \lambda}. \mathrm {a n d} \left\| \frac {\mathbf {v} _ {T}}{\| \mathbf {v} _ {T} \|} - \mathbf {w} _ {\| \cdot \|} ^ {*} \right\| ^ {2} \leq \frac {8}{\sqrt {\lambda} \gamma^ {2}} \sqrt {\frac {\lambda + 2 \log n}{T \lambda}}.
$$
D.1 Proof of Theorem 8
We first focus on the algorithm equivalence between the top two boxes. Based on the relationship between FTRL and EWA (Orabona, 2019), one can verify that
$$
p _ {t, i} = \frac {\exp (- y ^ {(i)} \mathbf {x} ^ {(i) \top} (\sum_ {k = 1} ^ {t - 1} \alpha_ {k} \mathbf {w} _ {k}))}{\sum_ {j = 1} ^ {n} \exp (- y ^ {(j)} \mathbf {x} ^ {(j) \top} (\sum_ {k = 1} ^ {t - 1} \alpha_ {k} \mathbf {w} _ {k}))}.
$$
Combining with the definition of  $L$  and  $\widetilde{\mathbf{w}}_t$ , it implies that
$$
\frac {\nabla L (\widetilde {\mathbf {w}} _ {t - 1})}{L (\widetilde {\mathbf {w}} _ {t - 1})} = - \mathbf {A} ^ {\top} \mathbf {p} _ {t},
$$
That is,  $\nabla L(\widetilde{\mathbf{w}}_{t - 1}) = -L(\widetilde{\mathbf{w}}_{t - 1})\mathbf{A}^{\top}\mathbf{p}_{t}$ . Let  $\widetilde{\mathbf{s}}_t = \mathrm{argmin}_{\| \mathbf{s}\| \leq 1}\mathbf{s}^\top \frac{\nabla L(\widetilde{\mathbf{w}}_{t - 1})}{L(\widetilde{\mathbf{w}}_{t - 1})}$ . Note that, on one hand, we have
$$
\widetilde {\mathbf {s}} _ {t} = \underset {\| \mathbf {s} \| \leq 1} {\operatorname {a r g m i n}} \mathbf {s} ^ {\top} \frac {\nabla L (\widetilde {\mathbf {w}} _ {t - 1})}{L (\widetilde {\mathbf {w}} _ {t - 1})} = \underset {\| \mathbf {s} \| \leq 1} {\operatorname {a r g m i n}} \mathbf {s} ^ {\top} \nabla L (\widetilde {\mathbf {w}} _ {t - 1}), \tag {37}
$$
where the second equality is because because the argmin does not change if we scale the objective functions. On the other hand, we have
$$
\widetilde {\mathbf {s}} _ {t} = \underset {\| \mathbf {s} \| \leq 1} {\operatorname {a r g m i n}} \mathbf {s} ^ {\top} \frac {\nabla L (\widetilde {\mathbf {w}} _ {t - 1})}{L (\widetilde {\mathbf {w}} _ {t - 1})} = \underset {\| \mathbf {s} \| \leq 1} {\operatorname {a r g m a x}} \mathbf {s} ^ {\top} \left(- \frac {\nabla L (\widetilde {\mathbf {w}} _ {t - 1})}{L (\widetilde {\mathbf {w}} _ {t - 1})}\right). \tag {38}
$$
To proceed, we introduce the following lemma.
Lemma 2. Let  $\| \cdot \|$  be any norm in  $\mathbb{R}^d$ . Let  $\mathbf{a}\in \mathbb{R}^d$ , and
$$
\mathbf {s} = \underset {\| \mathbf {s} ^ {\prime} \| \leq 1} {\operatorname {a r g m a x}} \mathbf {s} ^ {\prime \top} \mathbf {a}. \tag {39}
$$
$$
\left\| \mathbf {a} \right\| _ {*} \mathbf {s} = \underset {\mathbf {x} \in \mathbb {R} ^ {d}} {\operatorname {a r g m i n}} - \mathbf {a} ^ {\top} \mathbf {x} + \frac {1}{2} \| \mathbf {x} \| ^ {2}. \tag {40}
$$
Proof. We first focus on (40). Note that the objective has two terms. For the first term, based on Hölder's inequality, we have
$$
- \mathbf {a} ^ {\top} \mathbf {x} \geq - \| \mathbf {a} \| _ {*} \| \mathbf {x} \|.
$$
Let  $\| \mathbf{x} \| = c$ , where  $c > 0$  is a constant, then the equality is achieved (and thus the first term of the objective function is minimized) when
$$
\mathbf {x} = \underset {\| \mathbf {x} ^ {\prime} \| \leq c} {\operatorname {a r g m i n}} - \mathbf {x} ^ {' \top} \mathbf {a} = \underset {\| \mathbf {x} ^ {\prime} \| \leq c} {\operatorname {a r g m a x}} \mathbf {x} ^ {' \top} \mathbf {a}. \tag {41}
$$
In this case, for the objective function of (40), we have  $-\mathbf{a}^\top \mathbf{x} + \frac{1}{2}\| \mathbf{x}\|^2 = -c\| \mathbf{a}\|_* + \frac{1}{2} c^2$ . It's easy to see that the objective function is minimized when  $c = \| \mathbf{a}\|_*$ . The proof is finished by combining (39) and (41).
This lemma shows that the best response direction (under our game) is the steepest direction. Thus, we have
$$
\begin{array}{l} \mathbf {w} _ {t} = \underset {\mathbf {w} \in \mathbb {R} ^ {d}} {\operatorname {a r g m i n}} - \mathbf {p} _ {t} ^ {\top} \mathbf {A} \mathbf {w} + \frac {1}{2} \| \mathbf {w} \| ^ {2} = \underset {\mathbf {w} \in \mathbb {R} ^ {d}} {\operatorname {a r g m i n}} \mathbf {w} ^ {\top} \frac {\nabla L (\widetilde {\mathbf {w}} _ {t - 1})}{L (\widetilde {\mathbf {w}} _ {t - 1})} + \frac {1}{2} \| \mathbf {w} \| ^ {2} \\ = \underset {\mathbf {w} \in \mathbb {R} ^ {d}} {\operatorname {a r g m i n}} - \mathbf {w} ^ {\top} \left(- \frac {\nabla L (\widetilde {\mathbf {w}} _ {t - 1})}{L (\widetilde {\mathbf {w}} _ {t - 1})}\right) + \frac {1}{2} \| \mathbf {w} \| ^ {2} \tag {42} \\ = \operatorname * {a r g m a x} _ {\| \mathbf {s} ^ {\prime} \| \leq 1} \mathbf {w} ^ {\top} \left(- \frac {\nabla L (\widetilde {\mathbf {w}} _ {t - 1})}{L (\widetilde {\mathbf {w}} _ {t - 1})}\right) = \frac {\| \nabla L (\widetilde {\mathbf {w}} _ {t - 1}) \| _ {*}}{L (\widetilde {\mathbf {w}} _ {t - 1})} \widetilde {\mathbf {s}} _ {t}. \\ \end{array}
$$
where the first equality is based on the update rule of  $\mathbf{w}_t$  at the right box of Algorithm 6, the final inequality is based on Lemma 2 and (38). Note that, for the first equality, we dropped  $\alpha_{t}$  as it is a scaling parameter and does not have a influence on argmin. Thus, combining with (37), we have
$$
\mathbf {w} _ {t} = \frac {\| \nabla L (\widetilde {\mathbf {w}} _ {t - 1}) \| _ {*} \widetilde {\mathbf {s}}}{L (\widetilde {\mathbf {w}} _ {t - 1})} \widetilde {\mathbf {s}} _ {t} = \frac {\| \nabla L (\widetilde {\mathbf {w}} _ {t - 1}) \| _ {*}}{\left(L (\widetilde {\mathbf {w}} _ {t - 1})\right)} \underset {\| \mathbf {s} \| \leq 1} {\operatorname * {a r g m i n}} \mathbf {s} ^ {\top} \nabla L (\widetilde {\mathbf {w}} _ {t - 1}).
$$
$$
\widetilde {\mathbf {w}} _ {t} = \widetilde {\mathbf {w}} _ {t - 1} + \alpha_ {t} \mathbf {w} _ {t} = \widetilde {\mathbf {w}} _ {t} + \alpha_ {t} \frac {\| \nabla L (\widetilde {\mathbf {w}} _ {t - 1}) \| _ {*}}{L (\widetilde {\mathbf {w}} _ {t - 1})} \underset {\| \mathbf {s} \| \leq 1} {\operatorname * {a r g m i n}} \mathbf {s} ^ {\top} \nabla L (\widetilde {\mathbf {w}} _ {t - 1}).
$$
We can finish the first part of the proof by replacing  $\widetilde{\mathbf{w}}_t$  with  $\mathbf{v}_t$ ,  $\alpha_t \frac{\|\nabla L(\widetilde{\mathbf{w}}_{t-1})\|_*}{L(\widetilde{\mathbf{w}}_{t-1})}$  with  $\eta_t$ , and noticing  $\operatorname*{argmin}_{\|\mathbf{s}\| \leq 1} \mathbf{s}^\top \nabla L(\widetilde{\mathbf{w}}_{t-1})$  is  $\mathbf{s}_{t-1}$ .
Next, we study the regret bound. Note that the  $\mathbf{p}$ -player plays the FTRL algorithm on the simplex with a 1-strongly convex regularizer  $D_{\mathrm{KL}}\left(\mathbf{p},\frac{\mathbf{1}}{n}\right)$  (wrt  $\ell_1$ -norm). Therefore, based on Lemma 5, the regret can be upper bounded by
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} \alpha_ {t} \ell_ {t} (\mathbf {p} _ {t}) - \sum_ {t = 1} ^ {T} \alpha_ {t} \ell_ {t} (\mathbf {p} ^ {*}) \leq \log n + 2 \sum_ {t = 1} ^ {T} \alpha_ {t} ^ {2} \| \mathbf {A} ^ {\top} \mathbf {w} _ {t} \| _ {\infty} ^ {2} \\ = \log n + 2 \sum_ {t = 1} ^ {T} \frac {1}{t} \left(\max _ {i \in n} | y ^ {(i)} \mathbf {x} ^ {(i) \top} \mathbf {w} _ {t} |\right) \left(\max _ {i \in n} | y ^ {(i)} \mathbf {x} ^ {(i) \top} \mathbf {w} _ {t} |\right) \\ \leq \log n + 2 \sum_ {t = 1} ^ {T} \frac {1}{t} \| \mathbf {w} _ {t} \| ^ {2} \\ = \log n + 2 \sum_ {t = 1} ^ {T} \frac {1}{t} \| \mathbf {A} ^ {\top} \mathbf {p} _ {t} \| _ {*} ^ {2} \leq \log n + 2 \log T + 2, \tag {43} \\ \end{array}
$$
where the equality is because
$$
\left\| \mathbf {w} _ {t} \right\| = \left\| \left\| \frac {\nabla L (\mathbf {w} _ {t})}{L (\mathbf {w} _ {t})} \right\| _ {*} \widetilde {\mathbf {s}} _ {t} \right\| = \left\| \frac {\nabla L (\mathbf {w} _ {t})}{L (\mathbf {w} _ {t})} \right\| _ {*} = \left\| \mathbf {A} ^ {\top} \mathbf {p} _ {t} \right\| _ {*}.
$$
On the other hand, the w-player uses the best response $^+$  algorithm, and one can easily observe that the regret is upper bounded by 0. Finally, we have  $\sum_{t=1}^{T} \alpha_t = \sum_{t=1}^{T} \frac{1}{\sqrt{t}} \geq \sqrt{T}$ .
D.2 Proof of Theorem 9
We first focus on the bottom box. For the w-player, we show that, if we set  $\delta_{t - 1} = \frac{1}{\alpha_{t - 1}}$ , then this OMD algorithm is equivalent to the best response algorithm.
Specifically, note that  $\Phi (\mathbf{w}) = \frac{1}{2}\| \mathbf{w}\|^2$  is now  $\lambda$ -strongly convex. Thus the corresponding mirror map is well-defined and unique, and the function  $\nabla \Phi (\cdot)$  is invertible. Therefore, the solution for best response is
$$
\widehat {\mathbf {w}} _ {t} = \underset {\mathbf {w}} {\operatorname {a r g m i n}} \alpha_ {t - 1} h _ {t - 1} (\mathbf {w}) = \underset {\mathbf {w}} {\operatorname {a r g m i n}} h _ {t - 1} (\mathbf {w}) = \nabla \Phi^ {- 1} \left(\mathbf {A} ^ {\top} \mathbf {p} _ {t - 1}\right). \tag {44}
$$
On the other hand, since the w-player uses OMD, and the decision set is unbounded, we have
$$
\nabla \Phi (\mathbf {w} _ {t}) = \nabla \Phi (\mathbf {w} _ {t - 1}) - \delta_ {t - 1} \alpha_ {t - 1} \nabla h _ {t - 1} (\mathbf {w} _ {t - 1}) = \nabla \Phi (\mathbf {w} _ {t - 1}) + \mathbf {A} ^ {\top} \mathbf {p} _ {t - 1} - \nabla \Phi (\mathbf {w} _ {t - 1}) = \mathbf {A} ^ {\top} \mathbf {p} _ {t - 1}. \tag {45}
$$
Note that  $\delta_{t - 1}\alpha_{t - 1} = 1$ . Combining (44) and (45), we can draw the conclusion that  $\mathbf{w}_t$  and  $\widehat{\mathbf{w}}_t$  are identical, which shows OMD (with  $\delta_{t - 1} = \frac{1}{\alpha_{t - 1}}$ ) and BR (at round  $t - 1$ ) here are the same. We use the BR form the algorithm equivalence analysis, and OMD form for the regret analysis.
Next, we prove the algorithm equivalence of the left and bottom boxes in Algorithm 6. The arguments are similar to that in Appendix D.1. The difference is that the order of the w-player and the p-player is switched. Firstly, For the p-player, based on the connection between FTRL and EWA, we have
$$
p _ {t, i} \propto \exp \left(- y ^ {(i)} \mathbf {x} ^ {(i) \top} \left(\sum_ {j = 1} ^ {t} \alpha_ {j} \mathbf {w} _ {j}\right)\right) = \exp \left(- y ^ {(i)} \mathbf {x} ^ {(i) \top} \widetilde {\mathbf {w}} _ {t}\right).
$$
Combining with the definition of  $L$ , it implies that
$$
\frac {\nabla L (\widetilde {\mathbf {w}} _ {t})}{L (\widetilde {\mathbf {w}} _ {t})} = - \mathbf {A} ^ {\top} \mathbf {p} _ {t},
$$
That is,  $\nabla L(\widetilde{\mathbf{w}}_t) = -L(\widetilde{\mathbf{w}}_t)\mathbf{A}^\top \mathbf{p}_t$ . Let
$$
\widetilde {\mathbf {s}} _ {t} = \underset {\| \mathbf {s} \| \leq 1} {\operatorname {a r g m a x}} - \mathbf {s} ^ {\top} \frac {\nabla L (\widetilde {\mathbf {w}} _ {t})}{L (\widetilde {\mathbf {w}} _ {t})} = \underset {\| \mathbf {s} \| \leq 1} {\operatorname {a r g m i n}} \mathbf {s} ^ {\top} \nabla L (\widetilde {\mathbf {w}} _ {t}), \tag {46}
$$
Combining the first equality in (46) and Lemma 2, we have
$$
\frac {\left\| \nabla L (\widetilde {\mathbf {w}} _ {t}) \right\| _ {*}}{L (\widetilde {\mathbf {w}} _ {t})} \widetilde {\mathbf {s}} _ {t} = \operatorname * {a r g m i n} _ {\mathbf {w} \in \mathbb {R} ^ {d}} \mathbf {w} ^ {\top} \frac {\nabla L (\widetilde {\mathbf {w}} _ {t})}{L (\widetilde {\mathbf {w}} _ {t})} + \frac {1}{2} \| \mathbf {w} \| ^ {2} = \operatorname * {a r g m i n} _ {\mathbf {w} \in \mathbb {R} ^ {d}} - \mathbf {p} _ {t} ^ {\top} \mathbf {A} \mathbf {w} + \frac {1}{2} \| \mathbf {w} \| ^ {2} = \mathbf {w} _ {t + 1}.
$$
$$
\mathbf {w} _ {t} = \frac {\| \nabla L (\widetilde {\mathbf {w}} _ {t - 1}) \| _ {*} \widetilde {\mathbf {s}} _ {t - 1}}{L (\widetilde {\mathbf {w}} _ {t - 1})} = \frac {\| \nabla L (\widetilde {\mathbf {w}} _ {t - 1}) \| _ {*}}{\left(L (\widetilde {\mathbf {w}} _ {t - 1})\right)} \underset {\| \mathbf {s} \| \leq 1} {\operatorname * {a r g m i n}} \mathbf {s} ^ {\top} \nabla L (\widetilde {\mathbf {w}} _ {t - 1}).
$$
We can finish the first part of the proof by replacing  $\widetilde{\mathbf{w}}_t$  with  $\mathbf{v}_t$ ,  $\alpha_t \frac{\|\nabla L(\widetilde{\mathbf{w}}_{t-1})\|_*}{L(\widetilde{\mathbf{w}}_{t-1})}$  with  $\eta_{t-1}$ , and  $\operatorname*{argmin}_{\|\mathbf{s}\| \leq 1} \mathbf{s}^\top \nabla L(\widetilde{\mathbf{w}}_{t-1})$  with  $\mathbf{s}_t$ .
Next, we focus on regret. For the w-player, it uses the OMD algorithm, and we set the initial point  $\mathbf{w}_0 = \mathbf{0}$ . Note that we fixed  $\alpha_{t} = \frac{\lambda}{4}$  for all  $t$ , and thus step size  $\delta_{t-1} = \frac{1}{\alpha_{t-1}} = \frac{4}{\lambda}$  is also fixed.
Therefore, Lemma 7 can be applied. Define  $\mathbf{u} = \underset {\mathbf{w}\in \mathbb{R}^d}{\mathrm{argmin}}\sum_{t = 1}^{T}\alpha_th_t(\mathbf{w})$  , we have
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} \alpha_ {t} h _ {t} (\mathbf {w} _ {t}) - \min _ {\mathbf {w} \in \mathbb {R} ^ {d}} \sum_ {t = 1} ^ {T} \alpha_ {t} h _ {t} (\mathbf {w}) = \sum_ {t = 1} ^ {T} \alpha_ {t} h _ {t} (\mathbf {w} _ {t}) - \sum_ {t = 1} ^ {T} \alpha_ {t} h _ {t} (\mathbf {u}) \\ \leq \frac {\Phi (\mathbf {u})}{\delta} + \sum_ {t = 1} ^ {T} \frac {\delta \alpha_ {t} ^ {2}}{\lambda} \| \nabla h _ {t} (\mathbf {w} _ {t}) \| _ {*} ^ {2} \\ = \frac {\Phi (\mathbf {u})}{\delta} + \sum_ {t = 1} ^ {T} \frac {\alpha_ {t} ^ {2} \delta}{\lambda} \left\| - \mathbf {A} ^ {\top} \mathbf {p} _ {t} + \nabla \Phi (\mathbf {w} _ {t}) \right\| _ {*} ^ {2} \\ = \frac {\Phi (\mathbf {u})}{\delta} + \sum_ {t = 1} ^ {T} \frac {\alpha_ {t} ^ {2} \delta}{\lambda} \left\| - \mathbf {A} ^ {\top} \mathbf {p} _ {t} + \mathbf {A} ^ {\top} \mathbf {p} _ {t - 1} \right\| _ {*} ^ {2} \tag {47} \\ = \alpha_ {T} \Phi (\mathbf {u}) + \sum_ {t = 1} ^ {T} \frac {\alpha_ {t}}{\lambda} \left\| - \mathbf {A} ^ {\top} \mathbf {p} _ {t} + \mathbf {A} ^ {\top} \mathbf {p} _ {t - 1} \right\| _ {*} ^ {2} \\ = \alpha_ {T} \Phi (\mathbf {u}) + \sum_ {t = 1} ^ {T} \frac {\alpha_ {t}}{\lambda} \left\| \sum_ {i = 1} ^ {n} y ^ {(i)} \mathbf {x} ^ {(i)} \left(p _ {t, i} - p _ {t - 1, i}\right) \right\| _ {*} ^ {2} \\ \leq \alpha_ {T} \Phi (\mathbf {u}) + \sum_ {t = 1} ^ {T} \frac {\alpha_ {t}}{\lambda} \left(\sum_ {i = 1} ^ {n} \left| p _ {t, i} - p _ {t - 1, i} \right|\right) ^ {2} \\ \leq \alpha_ {T} \Phi (\mathbf {u}) + \sum_ {t = 1} ^ {T} \frac {\alpha_ {t}}{\lambda} \left\| \mathbf {p} _ {t} - \mathbf {p} _ {t - 1} \right\| _ {1} ^ {2}, \\ \end{array}
$$
where the second-to-last inequality is derived using triangle inequality and the assumption that  $\| \mathbf{x}^{(i)}\|_{*}$  is upper bounded by 1. Next, for  $\mathbf{u}$ , note that
$$
\begin{array}{l} \underset {\mathbf {w} \in \mathbb {R} ^ {d}} {\operatorname {a r g m i n}} \sum_ {t = 1} ^ {T} \alpha_ {t} h _ {t} (\mathbf {w}) = \underset {\mathbf {w} \in \mathbb {R} ^ {d}} {\operatorname {a r g m i n}} - \sum_ {t = 1} ^ {T} \alpha_ {t} \mathbf {p} _ {t} ^ {\top} \mathbf {A} \mathbf {w} + \frac {\sum_ {t = 1} ^ {T} \alpha_ {t}}{2} \| \mathbf {w} \| ^ {2} \tag {48} \\ = \operatorname *{argmin}_{\mathbf{w}\in \mathbb{R}^{d}} - \frac{1}{\sum_{t = 1}^{T}\alpha_{t}}\sum_{t = 1}^{T}\alpha_{t}\mathbf{p}_{t}^{\top}\mathbf{A}\mathbf{w} + \frac{1}{2}\| \mathbf{w}\|^{2}. \\ \end{array}
$$
Based on Lemma 2, we have
$$
\mathbf {u} = \left\| \mathbf {A} ^ {\top} \left(\frac {1}{\sum_ {t = 1} ^ {T} \alpha_ {t}} \sum_ {t = 1} ^ {T} \alpha_ {t} \mathbf {p} _ {t}\right) \right\| _ {*} \mathbf {s},
$$
$$
\mathbf {s} = - \operatorname * {a r g m a x} _ {\| \mathbf {s} \| \leq 1} \mathbf {s} ^ {\top} \left(\mathbf {A} ^ {\top} \left(\frac {1}{\sum_ {t = 1} ^ {T} \alpha_ {t}} \sum_ {t = 1} ^ {T} \alpha_ {t} \mathbf {p} _ {t}\right)\right).
$$
$$
\Phi (\mathbf {u}) = \frac {1}{2} \| \mathbf {u} \| ^ {2} = \frac {1}{2} \left\| \mathbf {A} ^ {\top} \left(\frac {1}{\sum_ {t = 1} ^ {T} \alpha_ {t}} \sum_ {t = 1} ^ {T} \alpha_ {t} \mathbf {p} _ {t}\right) \right\| _ {*} ^ {2} \leq \frac {1}{2},
$$
where the inequality is based on the triangle-inequality and the assumption that the dual norm of data is upper bounded by 1. Finally, for the  $\mathbf{p}$ -player, since it uses  $\mathrm{FTRL}^{+}$ , based on Lemma 6, we have
$$
\sum_ {t = 1} ^ {T} \alpha_ {t} \ell_ {t} (\mathbf {p} _ {t}) - \sum_ {t = 1} ^ {T} \ell_ {t} (\mathbf {p} ^ {*}) \leq \log n - \sum_ {t = 1} ^ {T} \frac {1}{2} \| \mathbf {p} _ {t} - \mathbf {p} _ {t - 1} \| _ {1} ^ {2}.
$$
To summarize, and let  $\alpha_{t} = \frac{\lambda}{2}$ , we have
$$
\frac {\operatorname {R e g} _ {T} ^ {\mathbf {w}} + \operatorname {R e g} _ {T} ^ {\mathbf {p}}}{\sum_ {t = 1} ^ {T} \alpha_ {t}} = \frac {\frac {\lambda}{4} + \log n}{T \lambda}.
$$
We first focus on the equivalence between the left and bottom boxes of Algorithm 4. For the w-player, similar to the proof of Theorem 7, we have
$$
\begin{array}{l} \mathbf {w} _ {t} = \underset {\mathbf {w} \in \mathbb {R} ^ {d}} {\operatorname {a r g m i n}} \sum_ {j = 1} ^ {t} \alpha_ {j} h _ {j} (\mathbf {w}) = \underset {\mathbf {w} \in \mathbb {R} ^ {d}} {\operatorname {a r g m i n}} \sum_ {j = 1} ^ {t} - \alpha_ {j} \mathbf {p} _ {j} ^ {\top} \mathbf {A} \mathbf {w} + \frac {\sum_ {j = 1} ^ {t} \alpha_ {j}}{2} \| \mathbf {w} \| ^ {2} \\ = \left[ \nabla \Phi \right] ^ {- 1} \left(\frac {1}{\sum_ {j = 1} ^ {t} \alpha_ {j}} \sum_ {j = 1} ^ {t} \alpha_ {j} \mathbf {A} ^ {\top} \mathbf {p} _ {j}\right), \tag {49} \\ \end{array}
$$
$$
\nabla \Phi (\mathbf {w} _ {t}) = \frac {\sum_ {j = 1} ^ {t - 1} \alpha_ {j}}{\sum_ {j = 1} ^ {t} \alpha_ {j}} \nabla \Phi (\mathbf {w} _ {t - 1}) + \frac {\alpha_ {t}}{\sum_ {j = 1} ^ {t} \alpha_ {j}} \mathbf {A} ^ {\top} \mathbf {p} _ {t},
$$
On the other hand, for the  $\mathbf{p}$ -player, since it performs Optimistic FTRL on a simplex with the negative entropy regularizer, we have
$$
\begin{array}{l} p _ {t, i} \propto \exp \left(- c \left(\sum_ {i = 1} ^ {t - 1} \alpha_ {i} \mathbf {w} _ {i} + \alpha_ {t} \mathbf {w} _ {t - 1}\right) ^ {\top} \mathbf {x} ^ {(i)} y ^ {(i)}\right) \tag {50} \\ = \exp \left(- c \left(\widetilde {\mathbf {w}} _ {t - 1} + \alpha_ {t} \mathbf {w} _ {t - 1}\right) ^ {\top} \mathbf {x} ^ {(i)} y ^ {(i)}\right), \\ \end{array}
$$
$$
\frac {\nabla L \left(c \widetilde {\mathbf {w}} _ {t - 1} + c \alpha_ {t} \mathbf {w} _ {t - 1}\right)}{L \left(c \widetilde {\mathbf {w}} _ {t - 1} + c \alpha_ {t} \mathbf {w} _ {t - 1}\right)} = - \mathbf {A} ^ {\top} \mathbf {p} _ {t}.
$$
Let  $\mathbf{z}_t = \mathbf{w}_t\sum_{i = 1}^t\alpha_i$  , then we have
$$
\nabla \Phi (\mathbf {z} _ {t}) = \nabla \Phi (\mathbf {z} _ {t - 1}) - \alpha_ {t} \frac {\nabla L (c \widetilde {\mathbf {w}} _ {t - 1} + c \alpha_ {t} \mathbf {w} _ {t - 1})}{L (c \widetilde {\mathbf {w}} _ {t - 1} + c \alpha_ {t} \mathbf {w} _ {t - 1})} = \nabla \Phi (\mathbf {z} _ {t - 1}) - \alpha_ {t} \frac {\nabla L (c \widetilde {\mathbf {w}} _ {t - 1} + \frac {c \alpha_ {t}}{\sum_ {i = 1} ^ {t - 1} \alpha_ {i}} \mathbf {z} _ {t - 1})}{L (c \widetilde {\mathbf {w}} _ {t - 1} + \frac {c \alpha_ {t}}{\sum_ {i = 1} ^ {t - 1} \alpha_ {i}} \mathbf {z} _ {t - 1})}.
$$
Finally, notice that  $\widetilde{\mathbf{w}}_t = \widetilde{\mathbf{w}}_{t - 1} + \alpha_t\mathbf{w}_t = \widetilde{\mathbf{w}}_{t - 1} + \frac{\alpha_t}{\sum_{i = 1}^t\alpha_i}\mathbf{z}_t$ . The proof is finished by replacing  $\widetilde{\mathbf{w}}_t$  with  $\widetilde{\mathbf{v}}_t$ ,  $\mathbf{w}_t$  with  $\frac{\mathbf{z}_t}{\sum_{i = 1}^t\alpha_i}$ ,  $c\widetilde{\mathbf{w}}_{t - 1} + c\alpha_t\mathbf{w}_{t - 1}$  with  $\mathbf{v}_t$ , configuring  $\beta_{t,1} = c = \frac{\lambda}{4}$ ,  $\beta_{t,1}' = \frac{c\alpha_t}{\sum_{i = 1}^{t - 1}\alpha_i} = \frac{\lambda}{2(t - 1)}$ ,  $\beta_{2,t} = 1$ ,  $\beta_{2,t}' = \frac{\alpha_t}{\sum_{i = 1}^t\alpha_t} = \frac{2}{t + 1}$ ,  $\eta_t = \frac{t}{L(\mathbf{v}_t)}$ .
Next, we focus on the equivalence between the right and bottom boxes. Note that for the w-player, we also have
$$
\begin{array}{l} \mathbf {w} _ {t} = \underset {\mathbf {w} \in \mathbb {R} ^ {d}} {\operatorname {a r g m i n}} \sum_ {j = 1} ^ {t} \alpha_ {j} h _ {j} (\mathbf {w}) = \underset {\mathbf {w} \in \mathbb {R} ^ {d}} {\operatorname {a r g m i n}} \sum_ {j = 1} ^ {t} - \alpha_ {j} \mathbf {p} _ {j} ^ {\top} \mathbf {A} \mathbf {w} + \frac {\sum_ {j = 1} ^ {t} \alpha_ {j}}{2} \| \mathbf {w} \| ^ {2} \tag {51} \\ = \operatorname * {a r g m i n} _ {\mathbf {w} \in \mathbb {R} ^ {d}} - \frac {1}{\sum_ {j = 1} ^ {t} \alpha_ {j}} \sum_ {j = 1} ^ {t} \alpha_ {j} \mathbf {p} _ {j} ^ {\top} \mathbf {A} \mathbf {w} + \frac {1}{2} \| \mathbf {w} \| ^ {2}. \\ \end{array}
$$
$$
\mathbf {s} _ {t} = - \underset {\| \mathbf {s} \| \leq 1} {\operatorname {a r g m a x}} \mathbf {s} ^ {\top} \left[ \frac {1}{\sum_ {j = 1} ^ {t} \alpha_ {j}} \sum_ {j = 1} ^ {t} \alpha_ {j} \mathbf {A} ^ {\top} \mathbf {p} _ {j} \right].
$$
$$
\left\| \frac {1}{\sum_ {j = 1} ^ {t} \alpha_ {j}} \sum_ {j = 1} ^ {t} \alpha_ {j} \mathbf {A} ^ {\top} \mathbf {p} _ {j} \right\| _ {*} \mathbf {s} _ {t} = \mathbf {w} _ {t}.
$$
Next, let  $\mathbf{g}_t = -\frac{1}{\sum_{j=1}^t \alpha_j} \sum_{j=1}^t \alpha_j \mathbf{A}^\top \mathbf{p}_j$ , we know
$$
\mathbf {g} _ {t} = \frac {\sum_ {j = 1} ^ {t - 1} \alpha_ {j}}{\sum_ {j = 1} ^ {t} \alpha_ {j}} \mathbf {g} _ {t} + \left(- \frac {\alpha_ {t}}{\sum_ {j = 1} ^ {t} \alpha_ {j}} \mathbf {A} ^ {\top} \mathbf {p} _ {t}\right).
$$
For the  $\mathbf{p}$ -player, it is clear that due to the optimistic term, we have
$$
- \mathbf {A} ^ {\top} \mathbf {p} _ {t} = \frac {\nabla L \left(c \widetilde {\mathbf {w}} _ {t - 1} + c \alpha_ {t} \mathbf {w} _ {t - 1}\right)}{L \left(c \widetilde {\mathbf {w}} _ {t - 1} + c \alpha_ {t} \mathbf {w} _ {t - 1}\right)}.
$$
To summarize, and let  $\alpha_{t} = t$ , we can conclude the proof by the following algorithm: