SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Lemma 3.2. Let  $y^1, \ldots, y^T \in \Delta_2$  where  $y^t \coloneqq \min_{x \in \Delta_2} \left[ (c^t + c^{t-1})^\top x + \sum_{s=1}^{t-1} (c^s + c^{s-1})^\top x + R(x) / \gamma \right]$ .
Then,  $\sum_{t=1}^{T}(c^t + c^{t-1})^\top y^t - \min_{i \in [n]} \sum_{t=1}^{T}(c_i^t + c_i^{t-1}) \leq 2\log T / \gamma + 2.$
In Lemma 3.3 we provide a closed formula capturing the difference between the output  $x^t \in \Delta_2$  of Algorithm 2 and the output  $y^t \in \Delta_2$  of Be the Regularized Leader algorithm defined in Lemma 3.2.
Lemma 3.3. Let  $x^{t} = (x_{1}^{t},x_{2}^{t})\in \Delta_{2}$  as in Algorithm 2 and  $y^{t} = (y_{1}^{t},y_{2}^{t})\in \Delta_{2}$  as in Lemma 3.2. Then,
$$
x _ {1} ^ {t} - y _ {1} ^ {t} = \gamma A ^ {- 1} \left(x _ {1} ^ {t}, y _ {1} ^ {t}\right) \cdot \left(\left(c _ {1} ^ {t} - c _ {2} ^ {t}\right) - \left(c _ {1} ^ {t - 1} - c _ {2} ^ {t - 1}\right)\right)
$$
with  $A(x_{1},y_{1})\coloneqq (x_{1}y_{1})^{-1} + (1 - x_{1})^{-1}(1 - y_{1})^{-1}$  and  $|A^{-1}(x_1^t,y_1^t) - A^{-1}(x_1^{t + 1},y_1^{t + 1})|\leq \mathcal{O}(\gamma)$
Up next we use Lemma 3.2 and Lemma 3.3 to establish that Algorithm 2 admits  $\mathcal{O}(\log^{2/3} T \cdot T^{1/3})$  alternating regret.
Theorem 3.4. Let  $x^{1}, \ldots, x^{T} \in \Delta_{2}$  the sequence produced by Algorithm 2 for the cost sequence  $c^{1}, \ldots, c^{T} \in [-1,1]^{2}$  with  $\gamma = \mathcal{O}\left(\log^{1/3} T \cdot T^{-1/3}\right)$  then  $\mathcal{R}^{\mathrm{alt}}(T) = \mathcal{O}\left(\log^{2/3} T \cdot T^{1/3}\right)$ .
Proof. By Lemma 3.2 then  $\sum_{t\in [T]}(c^t +c^{t - 1})^\top x^t -\min_{i\in [n]}\sum_{t\in [T]}(c_i^t +c_i^{t - 1})\leq \mathcal{O}(\log T / \gamma) +$ $\sum_{t\in [T]}(c^t +c^{t - 1})^\top (x^t -y^t)$  where  $y^{t}\in \Delta_{2}$  as in Lemma 3.2. Using Lemma 3.3 we get that
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} \left(c ^ {t} + c ^ {t - 1}\right) ^ {\top} \left(x ^ {t} - y ^ {t}\right) = \sum_ {t = 1} ^ {T} \left(\left(c _ {1} ^ {t} - c _ {2} ^ {t}\right) + \left(c _ {1} ^ {t - 1} - c _ {2} ^ {t - 1}\right)\right) \left(x _ {1} ^ {t} - y _ {1} ^ {t}\right) \\ = \gamma \sum_ {t = 1} ^ {T} \left(\left(c _ {1} ^ {t} - c _ {2} ^ {t}\right) + \left(c _ {1} ^ {t - 1} - c _ {2} ^ {t - 1}\right)\right) A ^ {- 1} \left(x _ {1} ^ {t}, y _ {1} ^ {t}\right) \cdot \left(\left(c _ {1} ^ {t} - c _ {2} ^ {t}\right) - \left(c _ {1} ^ {t - 1} - c _ {2} ^ {t - 1}\right)\right) \\ = \gamma \sum_ {t = 1} ^ {T} A ^ {- 1} \left(x _ {1} ^ {t}, y _ {1} ^ {t}\right) \left(\left(c _ {1} ^ {t} - c _ {2} ^ {t}\right) ^ {2} - \left(c _ {1} ^ {t - 1} - c _ {2} ^ {t - 1}\right) ^ {2}\right) \\ = \gamma \sum_ {t = 1} ^ {T} \left(c _ {1} ^ {t} - c _ {2} ^ {t}\right) ^ {2} \cdot \left(A ^ {- 1} \left(x _ {1} ^ {t}, y _ {1} ^ {t}\right) - A ^ {- 1} \left(x _ {1} ^ {t + 1}, y _ {1} ^ {t + 1}\right)\right) \leq \mathcal {O} \left(\gamma^ {2} T\right) \\ \end{array}
$$
Hence  $\mathcal{R}_{alt}(T) \leq \mathcal{O}\left(\log T / \gamma + \gamma^2 T\right) \leq \mathcal{O}\left(\log^{2/3} T \cdot T^{1/3}\right)$  for  $\gamma := \mathcal{O}\left(\log^{1/3} T / T^{1/3}\right)$ .
3.1 The  $n$ -Dimensional Simplex
In this section we extend Algorithm 2 to the case of the  $n$ -dimensional simplex. Our extension is motivated and builds upon the CFR algorithm developed in the context of EFGs [44].
Without loss of generality we assume that  $n = 2^H$ . We consider a complete binary tree  $T(V, E)$  of height  $H = \log n$  where the leaves  $L \subseteq V$  corresponds to the  $n$  actions,  $|L| = n$ . Each node  $s \in V / L$  admits exactly two children with  $\ell(s), r(s)$  respectively denoting the left and right child. Moreover,  $\operatorname{Level}(\mathrm{h}) \subseteq V$  denotes the nodes lying at depth  $h$  from the root ( $\operatorname{Level}(1) = \{\text{root}\}$  and  $\operatorname{Level}(\log n) = L$ ). Up next we present the notion of policy on the nodes of  $T(V, E)$ .
Definition 3.5.  $\cdot$  A policy over the nodes  $\pi : V / L \mapsto \Delta_2$  encodes the probability of selecting the left/right child at node  $s \in V$ . Specifically  $\pi(s) = (\pi(\ell(s)|s), \pi(r(s)|s))$  where  $\pi(\ell(s)|s) + \pi(r(s)|s) = 1$  and  $\pi(\ell(s)|s)$  is the probability of selecting  $\ell(s)$  (resp. for  $r(s)$ ).
-  $\operatorname*{Pr}(s,i,\pi)$  denotes the probability of reaching leaf  $i\in L$  starting from node  $s\in V / L$  and following  $\pi (\cdot)$  at each step.
-  $x^{\pi} \in \Delta_{n}$  denotes the probability distribution over the leaves/actions induced by  $\pi(\cdot)$ . Formally, we have  $x_{i}^{\pi} \coloneqq \operatorname{Pr}(\text{root}, i, \pi)$  for each leaf  $i \in L$ .
Definition 3.6. Given a cost vector  $c \in [-1,1]^n$  for the leaves/actions, the virtual cost of a node  $s \in V$  under policy  $\pi(\cdot)$ , denoted as  $Q(s,\pi,c)$ , equals
$$
Q (s, \pi , c) := \left\{ \begin{array}{l l} c _ {s} & s \in L \\ \sum_ {i \in L} \operatorname * {P r} (s, i, \pi) \cdot c _ {i} & s \notin L \end{array} \right.
$$
The virtual cost vector of  $s \in V$  under  $\pi(\cdot)$  is defined as  $q(s, \pi, c) := (Q(\ell(s), \pi, c), Q(r(s), \pi, c))$ .
We remark that  $Q(s, \pi, c)$  is the expected cost of the random walk starting from  $s \in V$  and following policy  $\pi(\cdot)$  until a leaf  $i \in L$  is reached in which case cost  $c_i$  is occurred.
Our online learning algorithm for the  $n$ -dimensional simplex is illustrated in Algorithm 3.
Algorithm 3 An Online Learning Algorithm for the  $n$ -Dimensional Simplex
1: Input: A sequence of cost vectors  $c^1, \ldots, c^T \in [-1, 1]^n$   
2: The learner constructs a complete binary tree  $T(V, E)$  with  $L = \mathcal{A}$ .  
3: for each round  $t = 1, \ldots, T$  do  
4: for each  $h = \log n$  to 1 do  
5: for every node  $s \in \text{Level}(h)$  do  
6: The learner computes  $q(s, \pi^t, c^{t-1}) := (Q(\ell(s), \pi^t, c^{t-1}), Q(r(s), \pi^t, c^{t-1}))$  and sets  $\pi^t(s) := \arg \min_{x \in \Delta_2} [2q(s, \pi^t, c^{t-1})^\top x + \sum_{\tau=1}^{t-1} (q(s, \pi^\tau, c^{\tau-1}) + q(s, \pi^\tau, c^\tau))^\top x + R(x)/\gamma]$   
7: end for  
8: end for  
9: The learner selects  $x^t := x^{\pi^t} \in \Delta_n$  (induced by policy  $\pi_t$ , Definition 3.5).  
10: The adversary selects cost vector  $c^t \in [0, 1]^n$   
11: The learner suffers cost  $(c^t + c^{t-1})^\top y^t$   
12: end for
We remark that at each round  $t$ , the learner computes a policy  $\pi^t (\cdot)$  as an intermediate step (Step 6) that then uses to select the probability distribution  $x^{t} := x^{\pi^{t}} \in \Delta_{n}$  (Step 9). Notice that the computation of policy  $\pi^t (\cdot)$  is performed in Steps (4)-(8). Since nodes are processed in decreasing order (with respect to their level), during Step 6  $\pi^t (\cdot)$  has already been determined for nodes  $\ell(s), r(s)$  and thus  $Q(\ell(s), \pi^t, c^{t-1}), Q(r(s), \pi^t, c^{t-1})$  are well-defined.
Up next we present the main steps for establishing Theorem 2.3. A key notion in the analysis of Algorithm 3 is that of local alternating regret of a node  $s \in V$  presented in Definition 3.7. As established in Lemma 3.8 the overall alternating regret of Algorithm 3 can be upper bounded by the sum of the local alternating regrets of the nodes lying in the path of the best fixed leaf/action.
Definition 3.7. For any sequence  $c^1, \ldots, c^T \in [-1,1]^n$  the alternating local regret of a node  $s \in V$ , denoted as  $\mathcal{R}_{loc}^T(s)$ , is defined as
$$
\mathcal {R} _ {l o c} ^ {T} (s) := \sum_ {t \in [ T ]} \left(q (s, \pi^ {t}, c ^ {t}) + q (s, \pi^ {t}, c ^ {t - 1})\right) ^ {\top} \pi^ {t} (s) - \min _ {\alpha \in \{\ell (s), r (s) \}} \sum_ {t \in [ T ]} \left(Q (\alpha , \pi^ {t}, c ^ {t}) + Q (\alpha , \pi^ {t}, c ^ {t - 1})\right)
$$
Lemma 3.8. Let a leaf/action  $i \in L$  and consider the path  $p = (\mathrm{root} = s_1, \ldots, s_H = i)$  from the root to the leaf  $i \in L$ . Then,  $\sum_{t=1}^{T} (c^t + c^{t-1})^\top x^{\pi^t} - 2 \sum_{t=1}^{T} c_i^t \leq \sum_{\ell=1}^{H} \mathcal{R}_{loc}(s_\ell)$ .
Up to this point, it is evident that in order to bound the overall alternating regret of Algorithm 3, we just need to bound the local alternating regret of any node  $s \in V$ . Using Theorem 3.4 we can bound the local regret of leaves  $i \in L$  for which  $q(i, \pi^t, c^{t-1}) = q(i, \pi^{t-1}, c^{t-1})$ . However this approach does apply for nodes  $s \in V / L$  since the local regret does not have the alternating structure,  $q(s, \pi^t, c^{t-1}) \neq q(s, \pi^{t-1}, c^{t-1})$ . To overcome the latter in Lemma 3.9 we establish that  $q(s, \pi^t, c^{t-1}), q(s, \pi^{t-1}, c^{t-1})$  are in distance  $\mathcal{O}(\gamma)$  which permits us to bound  $\mathcal{R}_{loc}^T(s)$  for  $s \in V / L$  by tweaking the proof of Theorem 3.4.
Lemma 3.9. Let  $\pi^1, \ldots, \pi^T$  the policies produced by Algorithm 3 then for any node  $s \in V$ ,  $i)$ $\| \pi^t(s) - \pi^{t-1}(s) \|_1 \leq 48\gamma$  and  $ii)$ $\| q(s, \pi^t, c^{t-1}) - q(s, \pi^{t-1}, c^{t-1}) \|_\infty \leq 48\gamma \log n$ .
Using Lemma 3.9 we can establish an upper bound on the local regret of any actions  $s \in V$ . The proof of Lemma 3.10 lies in Appendix B and follows a similar structure with the proof of Theorem 3.4.
Lemma 3.10. Let  $\gamma := \mathcal{O}\left(\log^{1/3} T / (T^{1/3} \log^{1/3} n)\right)$  in Algorithm 3 then  $\mathcal{R}_{loc}^T(s) \leq \mathcal{O}\left(\log^{2/3} T \cdot \log^{1/3} n \cdot T^{1/3}\right)$  for all  $s \in V$ .
Theorem 2.3 directly follows by combining Lemma 3.10, Lemma 3.8 and  $H = \log n$ .
4 The Ball case
In Algorithm 4 we present an online learning algorithm with  $\mathcal{O}(\log T)$  for  $\mathcal{D} = \mathbb{B}(0,1)$  and  $\| c^t\| _2\leq 1$ . Then through the transformation  $\hat{x}_t\coloneqq c + \rho x^t$  with  $x^{t}\in \mathbb{B}(0,1)$ , Algorithm 4 can be transformed to a  $\mathcal{O}$ $(\rho \log T)$ -alternating regret algorithm for  $\mathcal{D} = \mathbb{B}(c,\rho)$ .
Algorithm 4 may seem complicated at the first sight however it is composed by two basic algorithmic primitives. At Step 4 Algorithm 4 computes the output  $w_{t} \in \mathcal{B}(0,1)$  of the Follow the Regularized Leader (FTRL) with Euclidean regularization and adaptive step-size  $r_{0:t-1}$  (Step 3 of Algorithm 4). At Step 5, it mixes the output  $w_{t} \in \mathcal{B}(0,1)$  of FTRL with the unnormalized best-response  $-c^{t-1} \in \mathcal{B}(0,1)$ . The selection of the mixing coefficient  $p_{t}$  is adaptively updated at Step 7.
4.1 Proof of Theorem 2.4
In this section we present the main steps of the proof of Theorem 2.4. In Lemma 4.1 we provide a first upper bound on the alternating regret of Adaptive FTRL.
Lemma 4.1. Let  $w_{1},\ldots ,w_{T}\in \mathbb{B}(0,1)$  the sequence produced by Adaptive FTRL (Step 4 of Algorithm 4) given as input the cost-vector sequence  $c^1,\dots ,c^T\in \mathbb{B}(0,1)$ . Let  $t_1$  denote the maximum time-index such that  $\sum_{s = 1}^{t}(c^{s} + c^{s - 1})^{\top}w_{t}\geq -\sum_{s = 1}^{t}\| c^{s} + c^{s - 1}\|_{2}^{2} / 4$ . Then,
$$
\sum_ {t = 1} ^ {T} \left(c ^ {t} + c ^ {t - 1}\right) ^ {\top} w _ {t} - \min  _ {x \in \mathbb {B} (0, 1)} \sum_ {t = 1} ^ {T} \left(c ^ {t} + c ^ {t - 1}\right) ^ {\top} x \leq 4 \sqrt {1 + \sum_ {t = 1} ^ {t _ {1}} \| c ^ {t} + c ^ {t - 1} \| _ {2} ^ {2}} + \mathcal {O} (\log T)
$$
Lemma 4.1 guarantees that Adaptive FTRL admits only  $o(\sqrt{T})$  alternating regret in case  $t_1 = o(T)$ . Using Lemma 4.1, we establish Lemma 4.2 which is the cornerstone of our algorithm and guarantees that once Adaptive FTRL is appropriately mixed with unnormalized best-response  $(-c^{t - 1})$ , then the resulting algorithm always admits  $\mathcal{O}(\log T)$  regret.
Lemma 4.2. Let  $w_{1},\ldots ,w_{T}\in \mathbb{B}(0,1)$  be produced by Adaptive FTRL given as input  $c^1,\dots ,c^T\in$ $\mathbb{B}(0,1)$  and  $t_1$  be the maximum round such that  $\begin{array}{r}\sum_{s = 1}^{t}(c^{s} + c^{s - 1})^{\top}w_{s}\geq -\sum_{s = 1}^{t}\| c^{s} + c^{s - 1}\|_{2}^{2} / 4. \end{array}$
Algorithm 4 Online Learning Algorithm for Unit Ball
1:  $p_1 \gets 0, D_1 \gets [0,1]$  and  $c^0 \gets (0,\dots,0)$ .
2: for each round  $\bar{t} = 1, \dots, T$  do
3: The learner computes the coefficient  $r_{0:t-1} \gets \sqrt{1 + \sum_{s=1}^{t-1} \|c^s + c^{s-1}\|_2^2}$ .
4: The learner computes the output of FTRL,
$$
w _ {t} \leftarrow \operatorname {a r g m i n} _ {\| x \| \leq 1} \left[ \sum_ {s = 1} ^ {t - 1} \left(c ^ {s} + c ^ {s - 1}\right) ^ {\top} x + \frac {r _ {0 : t - 1}}{2} \| x \| _ {2} ^ {2} \right] \quad \# \text {A d a p t i v e F T R L}
$$
5: The learner selects the action  $x^{t} \gets (1 - p_{t})w_{t} + p_{t}(-c^{t - 1})$  # Mixing Adaptive FTRL with Unnormalized Best-Response
6: The adversary selects cost  $c^t$  with  $\| c^t \|_2 \leq 1$  and the learner suffers cost  $(c^{t-1} + c^t)^{\top} x^t$ .
7: The learner updates the interval  $D_{t} \subseteq [0,1]$  as follows,
$$
D _ {t} \leftarrow \left[ 0, \min  \left(1, \frac {2 0}{\sqrt {1 + \sum_ {s = 1} ^ {t} \| c ^ {s} + c ^ {s - 1} \| _ {2} ^ {2}}}\right) \right]
$$
and then updates the coefficient  $p_t \in [0,1]$  as follows,
$$
p _ {t + 1} \leftarrow \left[ p _ {t} + \frac {2 0 (c ^ {t} + c ^ {t - 1}) ^ {\top} \cdot (x ^ {t} + c ^ {t - 1})}{1 + \sum_ {s = 1} ^ {t} \| c ^ {s} + c ^ {s - 1} \| _ {2} ^ {2}} \right] _ {D _ {t}}
$$
Let  $p := 20 / \sqrt{400 + \sum_{t=1}^{t_1} \|c^t + c^{t-1}\|_2^2}$  and let  $y_t := (1 - p)w_t - pc^{t-1}$  for  $t \leq t_1$  and  $y_t := w_t$  for  $t \geq t_1 + 1$ . Then  $\sum_{t=1}^{T}(c^t + c^{t-1})^\top y_t - \min_{x \in \mathbb{B}(0,1)} \sum_{t=1}^{T}(c^t + c^{t-1})^\top x \leq \mathcal{O}(\log T)$ .
Lemma 4.2 establishes that in case at Step 5, Algorithm 4 mixed the output  $w_{t}$  of Adaptive FTRL with the unnormalized best-response  $(-c^{t - 1}\in \mathcal{B}(0,1))$  as follows,
$$
y _ {t} := \left(1 - q _ {t}\right) \cdot w _ {t} + q _ {t} \cdot \left(- c ^ {t - 1}\right) \text {w i t h} q _ {t} := \frac {2 0 \mathrm {I} [ t \leq t _ {1} ]}{\sqrt {4 0 0 + \sum_ {t = 1} ^ {t _ {1}} \| c ^ {t} + c ^ {t - 1} \| _ {2} ^ {2}}}, \tag {3}
$$
then it would admit  $\mathcal{O}(\log T)$  alternating regret. Obviously, Algorithm 4 does not know a-priori neither  $t_1$  nor  $\sum_{t=1}^{t_1} \|c^t + c^{t-1}\|_2^2$ . However by using the recent result of [5] for Online Gradient Descent in Shrinking Domains, we can establish that the mixing coefficients  $p_t \in [0,1]$  selected by Algorithm 4 at Step 7, admit the exact same result as selecting  $q_t \in [0,1]$  described in Equation 3. The latter is formalized in Lemma 4.3.
Lemma 4.3. Let the sequences  $w_{1},\ldots ,w_{T}\in \mathbb{B}(0,1)$  and  $p_1,\dots ,p_T\in (0,1)$  produced by Algorithm 4 given as input  $c^1,\ldots ,c^T\in \mathcal{B}(0,1)$ . Additionally let  $t_1$  denote the maximum time such that  $\sum_{s = 1}^{t}(c^{s} + c^{s - 1})^{\top}w_{s}\geq -\sum_{s = 1}^{t}\| c^{s} + c^{s - 1}\|_{2}^{2} / 4$  and consider the sequence  $q_{t}\coloneqq \mathrm{I}\left[t\leq t_{1}\right]\cdot \left(20 / \sqrt{400 + \sum_{t = 1}^{t_{1}}\|c^{t} + c^{t - 1}\|_{2}^{2}}\right)$ . Then,
$$
\sum_ {t \in [ T ]} \left(c ^ {t - 1} + c ^ {t}\right) ^ {\top} \left(w _ {t} + c ^ {t - 1}\right) \cdot q _ {t} - \sum_ {t \in [ T ]} \left(c ^ {t - 1} + c ^ {t}\right) ^ {\top} \left(w _ {t} + c ^ {t - 1}\right) \cdot p _ {t} \leq \mathcal {O} (\log T)
$$
In this paper we introduced a variant of the Online Linear Optimization that we call Alternating Online Linear Optimization for which we developed the first online learning algorithms with  $o(\sqrt{T})$  regret guarantees. Our work is motivated by the popular setting of alternating play in two-player games and raises some interesting open questions. The most natural ones is understanding whether
$\tilde{\mathcal{O}}(1)$  regret guarantees can be established the  $n$ -dimensional simplex as well as establishing  $o(\sqrt{T})$  for general convex losses.
Limitations: The current work is limited to the linear losses setting. Notice that the classic reduction from convex to linear losses in Standard OLM no longer holds in Alternating OLM. Therefore the generalization to general convex losses seems to require new techniques. We defer this study for future work.
This work was supported by the Swiss National Science Foundation (SNSF) under grant number 200021_205011, by Hasler Foundation Program: Hasler Responsible AI (project number 21043) and Innovation project supported by Innosuisse (contract agreement 100.960 IP-ICT). Luca Viano is funded through a PhD fellowship of the Swiss Data Science Center, a joint venture between EPFL and ETH Zurich. This research was also supported in part by the National Research Foundation, Singapore and DSO National Laboratories under its AI Singapore Program (AISG Award No: AISG2-RP-2020-016), grant PIESGP-AI-2020-01, AME Programmatic Fund (Grant No.A20H6b0151) from A*STAR and Provost's Chair Professorship grant RGEPPV2101.
[1] Ioannis Anagnostides, Constantinos Daskalakis, Gabriele Farina, Maxwell Fishelson, Noah Golowich, and Tuomas Sandholm. Near-optimal no-regret learning for correlated equilibria in multi-player general-sum games. In Stefano Leonardi and Anupam Gupta, editors, STOC '22: 54th Annual ACM SIGACT Symposium on Theory of Computing, pages 736-749. ACM, 2022.
[2] Ioannis Anagnostides, Gabriele Farina, Christian Kroer, Chung-Wei Lee, Haipeng Luo, and Tuomas Sandholm. Uncoupled learning dynamics with o(log T) swap regret in multiplayer games. CoRR, abs/2204.11417, 2022.
[3] James P. Bailey, Gauthier Gidel, and Georgios Piliouras. Finite regret and cycles with fixed step-size via alternating gradient descent-ascent. In Jacob D. Abernethy and Shivani Agarwal, editors, Conference on Learning Theory, COLT 2020, 9-12 July 2020, Virtual Event [Graz, Austria], volume 125 of Proceedings of Machine Learning Research, pages 391-407. PMLR, 2020.
[4] Aditya Bhaskara, Ashok Cutkosky, Ravi Kumar, and Manish Purohit. Online learning with imperfect hints. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pages 822-831. PMLR, 2020.
[5] Aditya Bhaskara, Ashok Cutkosky, Ravi Kumar, and Manish Purohit. Logarithmic regret from sublinear hints. In Marc'Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan, editors, Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, pages 28222-28232, 2021.
[6] Aditya Bhaskara, Ashok Cutkosky, Ravi Kumar, and Manish Purohit. Power of hints for online learning with movement costs. In Arindam Banerjee and Kenji Fukumizu, editors, The 24th International Conference on Artificial Intelligence and Statistics, AISTATS 2021, volume 130 of Proceedings of Machine Learning Research, pages 2818-2826. PMLR, 2021.
[7] Nicolo Cesa-Bianchi and Gabor Lugosi. Prediction, learning, and games. Cambridge university press, 2006.
[8] Antonin Chambolle and Thomas Pock. A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis., 40(1):120-145, 2011.
[9] Tatjana Chavdarova, Gauthier Gidel, François Fleuret, and Simon Lacoste-Julien. Reducing noise in GAN training with variance reduced extragradient. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d'Alché-Buc, Emily B. Fox, and Roman Garnett,
[41] Guodong Zhang, Yuanhao Wang, Laurent Lessard, and Roger B. Grosse. Near-optimal local convergence of alternating gradient descent-ascent for minimax optimization. In Gustau Camps-Valls, Francisco J. R. Ruiz, and Isabel Valera, editors, International Conference on Artificial Intelligence and Statistics, AISTATS 2022, volume 151 of Proceedings of Machine Learning Research, pages 7659-7679. PMLR, 2022.
[42] Guojun Zhang and Yaoliang Yu. Convergence of gradient methods on bilinear zero-sum games. In 8th International Conference on Learning Representations, ICLR 2020. OpenReview.net, 2020.
[43] Martin Zinkevich. Online convex programming and generalized infinitesimal gradient ascent. In Tom Fawcett and Nina Mishra, editors, Machine Learning, Proceedings of the Twentieth International Conference (ICML 2003), August 21-24, 2003, Washington, DC, USA, pages 928-936. AAAI Press, 2003.
[44] Martin Zinkevich, Michael Johanson, Michael H. Bowling, and Carmelo Piccione. Regret minimization in games with incomplete information. In Advances in Neural Information Processing Systems 20, Proceedings of the Twenty-First Annual Conference on Neural Information Processing Systems, Vancouver, NIPS 2007, pages 1729–1736. Curran Associates, Inc., 2007.
A Omitted Proofs of Section 3
A.1 Auxiliary Lemmas
Lemma A.1. The log-barrier function  $R(x) = -\log x - \log (1 - x)$  is 1-strongly convex in [0, 1]. More precisely, for all  $x, y \in [0,1]$
$$
R (y) \geq R (x) + R ^ {\prime} (x) ^ {\top} (y - x) + \frac {1}{2} | x - y | ^ {2}
$$
Proof. Let  $f(x) \coloneqq -\log x$  then  $f'(x) = -\frac{1}{x}$  and  $f''(x) = \frac{1}{x^2}$ . Since  $x \leq 1$  we get that  $f''(x) \geq 1$  and thus
$$
f (y) \geq f (x) + f ^ {\prime} (x) (y - x) + \frac {1}{2} (x - y) ^ {2}
$$
At the same time the function  $f(x) = -\log (1 - x)$  is convex in [0, 1]. This concludes the proof.
Lemma A.2. Let  $x \coloneqq \operatorname{argmin}_{z \in [0,1]} [\gamma c \cdot z + R(z)]$  and  $y \coloneqq \operatorname{argmin}_{z \in [0,1]} [\gamma \hat{c} \cdot z + R(z)]$  where  $R(\cdot)$  is an 1-strongly convex function in  $\mathbb{R}$ . Then,
$$
| x - y | \leq 2 \gamma | c - \hat {c} |
$$
Proof. By the strong convexity of the function  $\gamma c^T z + R(z)$  and first order optimality conditions for  $x$ , we get that
$$
\gamma c ^ {\top} y + R (y) \geq \gamma c ^ {\top} x + R (x) + \frac {1}{2} | x - y | ^ {2}
$$
As a result, we get that
$$
\begin{array}{l} \frac {1}{2} | x - y | ^ {2} \leq \gamma c \cdot (y - x) + R (y) - R (x) \\ = \gamma \hat {c} \cdot (y - x) + \gamma (c - \hat {c}) \cdot (y - x) + R (y) - R (x) \\ \leq \gamma (c - \hat {c}) \cdot (y - x) \\ \end{array}
$$
which implies that  $|x - y| \leq 2\gamma |c - \hat{c}|$ .
A.2 Proof of Lemma 3.2
Lemma 3.2. Let  $y^1, \ldots, y^T \in \Delta_2$  where  $y^t \coloneqq \min_{x \in \Delta_2} \left[(c^t + c^{t-1})^\top x + \sum_{s=1}^{t-1}(c^s + c^{s-1})^\top x + R(x) / \gamma\right]$ .
Then,  $\sum_{t=1}^{T}(c^t + c^{t-1})^\top x^t - \min_{i\in[2]}\sum_{t=1}^{T}(c_i^t + c_i^{t-1}) \leq 2\log T / \gamma + 2$ .
Proof. We start by rewrite the regret minimization problem over  $\Delta_{2}$  as an equivalent one over  $[0,1]$ , that is
$$
\sum_ {t = 1} ^ {T} \left(c ^ {t} + c ^ {t - 1}\right) ^ {\top} \left(x ^ {t} - x ^ {\star}\right) = \sum_ {t = 1} ^ {T} \left(\hat {c} ^ {t} + \hat {c} ^ {t - 1}\right) ^ {\top} \left(x _ {1} ^ {t} - x _ {1} ^ {\star}\right)
$$
where  $\hat{c}^t = c_1^t -c_2^t$  . Moreover notice that
$$
y _ {1} ^ {t} := \underset {p \in [ 0, 1 ]} {\arg \min } \left[ \sum_ {\tau = 1} ^ {t} \left(\hat {c} ^ {\tau} + \hat {c} ^ {\tau - 1}\right) p - \frac {\log p + \log (1 - p)}{\gamma} \right] \tag {4}
$$
By the "Follow the Leader/Be the Leader" Lemma [7, Lemma 3.1], we have that
$$
\left[ \sum_ {t = 1} ^ {T} (\hat {c} ^ {t} + \hat {c} ^ {t - 1}) y _ {1} ^ {t} - \frac {\log y _ {1} ^ {t} + \log (1 - y _ {1} ^ {t})}{\gamma} \right] \leq \min _ {p \in [ 0, 1 ]} \left[ \sum_ {t = 1} ^ {T} (\hat {c} ^ {t} + \hat {c} ^ {t - 1}) p - \frac {\log p + \log (1 - p)}{\gamma} \right].
$$
That implies
$$
\left[ \sum_ {t = 1} ^ {T} (\hat {c} ^ {t} + \hat {c} ^ {t - 1}) y _ {1} ^ {t} \right] \leq \min _ {p \in [ 0, 1 ]} \left[ \sum_ {t = 1} ^ {T} (\hat {c} ^ {t} + \hat {c} ^ {t - 1}) p - \frac {\log p + \log (1 - p)}{\gamma} \right]
$$
Now let  $x^{\star} = \arg \min_{p\in [0,1]}\sum_{t = 1}^{T}(\hat{c}^{t} + \hat{c}^{t - 1})p$  and let us subtract  $\sum_{t = 1}^{T}(\hat{c}^{t} + \hat{c}^{t - 1})x^{\star}$  from both sides
$$
\left[ \sum_ {t = 1} ^ {T} \left(\hat {c} ^ {t} + \hat {c} ^ {t - 1}\right) \left(y _ {1} ^ {t} - x ^ {\star}\right) \right] \leq \min  _ {p \in [ 0, 1 ]} \left[ \sum_ {t = 1} ^ {T} \left(\hat {c} ^ {t} + \hat {c} ^ {t - 1}\right) p - \frac {\log p + \log (1 - p)}{\gamma} \right] - \sum_ {t = 1} ^ {T} \left(\hat {c} ^ {t} + \hat {c} ^ {t - 1}\right) x ^ {\star}
$$