SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Volkan Cevher
LIONS, EPFL
volkan.cevher@epfl.ch
Ashok Cutkosky
ashok@cutkosky.com
Ali Kavis
ali.kavis@epfl.ch
Georgios Piliouras
SUTD
georgios@sutd.edu.sg
Stratis Skoulakis
efstratios.skoulakis@epfl.ch
Luca Viano
luca.viano@epfl.ch
Motivated by alternating game-play in two-player games, we study an alternating variant of the Online Linear Optimization (OLO). In alternating OLO, a learner at each round  $t \in [n]$  selects a vector  $x^t$  and then an adversary selects a cost-vector  $c^t \in [-1, 1]^n$ . The learner then experiences cost  $(c^t + c^{t-1})^\top x^t$  instead of  $(c^t)^\top x^t$  as in standard OLO. We establish that under this small twist, the  $\Omega(\sqrt{T})$  lower bound on the regret is no longer valid. More precisely, we present two online learning algorithms for alternating OLO that respectively admit  $\mathcal{O}((\log n)^{4/3} T^{1/3})$  regret for the  $n$ -dimensional simplex and  $\mathcal{O}(\rho \log T)$  regret for the ball of radius  $\rho > 0$ . Our results imply that in alternating game-play, an agent can always guarantee  $\tilde{\mathcal{O}}((\log n)^{4/3} T^{1/3})$  regardless the strategies of the other agent while the regret bound improves to  $\mathcal{O}(\log T)$  in case the agent admits only two actions.
Game-dynamics study settings at which a set of selfish agents engaged in a repeated game update their strategies over time in their attempt to minimize their overall individual cost. In simultaneous play all agents simultaneously update their strategies, while in alternating play only one agent updates its strategy at each round while all the other agents stand still. Intuitively, each agent only updates its strategy in response to an observed change in another agent.
Alternating game-play captures interactions arising in various contexts such as animal behavior, social behavior, traffic networks etc. (see [33] for various interesting examples) and thus has received considerable attention from a game-theoretic point of view [11, 3, 33, 41, 40]. At the same time, alternation has been proven a valuable tool in tackling min-max problems arising in modern machine learning applications (e.g. training GANs, adversarial examples etc.) and thus has also been studied from an offline optimization perspective [37, 35, 21, 42, 9, 8, 10].
In the context of two-players, alternating game-play admits the following form: Alice (odd player) and Bob (even player) respectively update their strategies on odd and even rounds. Alice (resp. Bob) should select her strategy at an odd round so as to exploit Bob's strategy of the previous (even) round while at the same time protecting herself from Bob's response in the next (even) round. As a result, the following question arises:
Q1: How should Alice (resp. Bob) update her actions in the odd rounds so that, regardless of Bob's strategies, her overall cost (over the  $T$  rounds of play) is minimized?
1.1 Standard and Alternating Online Linear Minimization
Motivated by the above question and building on the recent line of research studying online learning settings with restricted adversaries [15, 25, 4, 5, 6, 34], we study an online linear optimization setting [43], called alternating online linear optimization. We use the term "alternating" to highlight the connection with alternating game-play that we subsequently present in Section 1.2.
In Algorithm 1 we jointly present both standard and alternating OLO so as to better illustrate the differences of the two settings.
Algorithm 1 Standard and Alternating Online Linear Minimization
1: Input: A feasibility set  $\mathcal{D} \subseteq \mathbb{R}^n$  and  $c^0 \gets (0, \dots, 0)$ .
2: for each round  $t = 1,\dots ,T$  do
3: The learner selects a vector  $x^{t} \in \mathcal{D}$  based on  $c^1, \ldots, c^{t-1} \in [-1, 1]^n$ .
4: The adversary learns  $x^{t} \in \mathcal{D}$  and selects a cost vector  $c^{t} \in [-1, 1]^{n}$  (based on  $x^{1}, \ldots, x^{t}$ ).
5: The learner learns  $c^t \in [-1, 1]^n$  and receives cost,
$$
\left(c ^ {t}\right) ^ {\top} x ^ {t}
$$
Standard OLM
$$
\left(c ^ {t} + c ^ {t - 1}\right) ^ {\top} x ^ {t}
$$
Alternating OLM
6: end for
In both standard and alternating OLO, the adversary selects  $c^t$  after the learner's selection of  $x^t$ . The only difference between standard and alternating OLM is that in the first case the learner admits cost  $(c^t)^{\top}x^{t}$  while in the second its cost is  $(c^{t} + c^{t - 1})^{\top}x^{t}$ . An online learning algorithm<sup>1</sup> selects  $x^{t}\in \mathcal{D}$  solely based on the previous cost-vector sequence  $c^1,\ldots ,c^{t - 1}\in [-1,1]^n$  with the goal minimizing the overall cost that is slightly different in standard and alternating OLO.
The quality of an online learning algorithm  $\mathcal{A}$  in standard OLO is captured through the notion of regret [23], comparing  $\mathcal{A}$ 's overall cost with the overall cost of the best fixed action,
$$
\mathcal {R} _ {\mathcal {A}} (T) := \max  _ {c ^ {1}, \dots , c ^ {T}} \left[ \sum_ {t = 1} ^ {T} \left(c ^ {t}\right) ^ {\top} x ^ {t} - \min  _ {x \in \mathcal {D}} \sum_ {t = 1} ^ {T} \left(c ^ {t}\right) ^ {\top} x \right]. \tag {1}
$$
When  $\mathcal{R}_{\mathcal{A}}(T) = o(T)$ , the algorithm  $\mathcal{A}$  is called no-regret since it ensures that regardless of the cost-vector sequence  $c^1, \ldots, c^T$ , the time-averaged overall cost of  $\mathcal{A}$  approaches the time-averaged overall cost of the best fixed action with rate  $o(T)/T \to 0$ . Correspondingly, the quality of an online learning algorithm  $\mathcal{A}$  in alternating OLO is captured through the notion of alternating regret,
$$
\mathcal {R} _ {\mathcal {A}} ^ {\mathrm {a l t}} (T) := \max  _ {c ^ {1}, \dots , c ^ {T}} \left[ \sum_ {t = 1} ^ {T} \left(c ^ {t} + c ^ {t - 1}\right) ^ {\top} x ^ {t} - \min  _ {x \in \mathcal {D}} \sum_ {t = 1} ^ {T} \left(c ^ {t} + c ^ {t - 1}\right) ^ {\top} x \right]. \tag {2}
$$
Over the years various no-regret algorithms have been proposed for different OLO settings $^2$  achieving  $\mathcal{R}_{\mathcal{A}}(T) = \tilde{\mathcal{O}}\left(\sqrt{T}\right)$  regret [28, 20, 43]. The latter regret bounds are optimal since there is a simple probabilistic construction establishing that any online learning algorithm  $\mathcal{A}$  admits  $\mathcal{R}_{\mathcal{A}}(T) = \Omega (\sqrt{T})$  even when  $\mathcal{D}$  is the 2-dimensional simplex. This negative results comes from the fact that the adversary has access to the action  $x^{t}$  of the algorithm and can appropriately select  $c^t$  to maximize  $\mathcal{A}$ 's regret.
At a first sight, it may seem that the adversary can still enforce  $\Omega\left(\sqrt{T}\right)$  alternating regret to any online learning algorithm  $\mathcal{A}$  by appropriately selecting  $c^t$  based on  $x^t$  and possibly on  $c^{t-1}$ . Interestingly enough the construction establishing  $\Omega(\sqrt{T})$  regret, fails in the case of alternating regret (see Section 2). As a result, the following question naturally arises,
1the notion of an online learning algorithm is exactly the same in standard and alternating OLO.
the difference concerns the feasibility set  $\mathcal{D}$
Q2: Are there online learning algorithms with  $o\left(\sqrt{T}\right)$  alternating regret?
Apart from its interest in the context of online learning, answering  $Q2$  implies a very sound answer to  $Q1$ . In Section 1.2 we present the connection between Alternating OLO and Alternating Game-Play.
1.2 Alternating OLO and Alternating Game-Play
Alternating game-play in the context of two-player games can be described formally as follows: Let  $(A,B)$  be a game played between Alice and Bob. The matrix  $A\in [-1,1]^{n\times m}$  represents Alice's costs,  $A_{ij}$  is the cost of Alice if she selects action  $i\in [n]$  and Bob selects action  $j\in [m]$  (respectively  $B\in [-1,1]^{m\times n}$  for Bob). Initially Alice selects a mixed strategy  $x^{1}\in \Delta_{n}$ . Then,
- At the even rounds  $t = 2,4,6,\ldots ,{2k}$  : Bob plays a new mixed strategy  ${y}^{t} \in  {\Delta }_{m}$  and Alice plays  ${x}^{t - 1} \in  {\Delta }_{n}$  . Alice and Bob incur costs  ${\left( {x}^{t - 1}\right) }^{\top }A{y}^{t}$  and  ${\left( {y}^{t}\right) }^{\top }B{x}^{t - 1}$  respectively.
- At the odd rounds  $t = 3,5,\ldots ,{2k} - 1$  : Alice plays a new mixed strategy  ${x}^{t} \in  {\Delta }_{n}$  and Bob plays  ${y}^{t - 1} \in  {\Delta }_{m}$  . Alice and Bob incur costs  ${\left( {x}^{t}\right) }^{\top }A{y}^{t - 1}$  and  ${\left( {y}^{t - 1}\right) }^{\top }B{x}^{t}$  respectively.
From the perspective of Alice (resp. Bob), the question is how to select her mixed strategies  $x^{1}, x^{3}, \ldots, x^{2k-1} \in \Delta_{n}$  so as to minimize her overall cost
$$
\left(x ^ {1}\right) ^ {\top} A y ^ {2} + \sum_ {k = 1} ^ {T / 2 - 1} \left(x ^ {2 k + 1}\right) ^ {\top} A \left(y ^ {2 k} + y ^ {2 k + 2}\right).
$$
In Corollary 1.1 we establish that if Alice uses an online learning algorithm  $\mathcal{A}$  then her overall regret (over the course of  $T$  rounds of play) is at most  $\mathcal{R}_{\mathcal{A}}^{\mathrm{alt}}(T / 2)$ . As a result, in case  $\mathbf{Q2}$  admits a positive answer, then Alice can guarantee at most  $o(\sqrt{T})$  regret and improve over the  $\tilde{\mathcal{O}}\left(\sqrt{T}\right)$  regret bound provided by standard no-regret algorithms [28, 20, 43, 23].
Corollary 1.1. In case Alice (resp. Bob) uses an online learning algorithm  $\mathcal{A}$  to update her strategies in the odd rounds,  $x^{2k + 1} \coloneqq \mathcal{A}(Ay^2, Ay^4, \ldots, Ay^{2k})$  for  $k = 1, \ldots, T/2 - 1$ . Then no matter Bob's selected sequence  $y^2, y^4, \ldots, y^T \in \Delta_m$ ,
$$
\left(x ^ {1}\right) ^ {\top} A y ^ {2} + \sum_ {k = 1} ^ {T / 2 - 1} \left(x ^ {2 k + 1}\right) ^ {\top} A \left(y ^ {2 k} + y ^ {2 k + 2}\right) - \min  _ {x \in \Delta_ {n}} \left[ x ^ {\top} A y ^ {2} + \sum_ {k = 1} ^ {T / 2 - 1} x ^ {\top} A \left(y ^ {2 k} + y ^ {2 k + 2}\right) \right] \leq \mathcal {R} _ {\mathcal {A}} ^ {\mathrm {a l t}} (T / 2)
$$
Remark 1.2. We remark that Corollary 1.1 refers to the standard notion of regret [23] and  $\mathcal{R}_{\mathcal{A}}^{\mathrm{alt}}(T / 2)$  appears only as an upper bound. We additionally remark that if both Alice and Bob respectively use algorithms  $\mathcal{A}$  and  $\mathcal{B}$  in the context of alternating play, then the time-average strategy vector converges with rate  $\mathcal{O}(\max (\mathcal{R}_{\mathcal{A}}(T),\mathcal{R}_{\mathcal{B}}(T)) / T)$  to Nash Equilibrium in case of zero-sum games  $(A = -B^{\top})$  and to Coarse Correlated Equilibrium for general two-player games [32]. Our objective is more general: we focus on optimizing the performance of a single player regardless of the actions of the other player.
1.3 Our Contribution and Techniques
In this work we answer  $Q2$  on the affirmative. More precisely we establish that,
- There exists an online learning algorithm (Algorithm 3) with alternating regret  $\tilde{\mathcal{O}}\left((\log n)^{4/3}T^{1/3}\right)$  for  $\mathcal{D} = \Delta_n$  ( $n$ -dimensional simplex).
- There exists an online learning algorithm (Algorithm 4) with alternating regret  $\mathcal{O}(\rho \log T)$  for  $\mathcal{D} = \mathbb{B}(c, \rho)$  (ball of radius  $\rho$ ).
- There exists an online learning algorithm with alternating regret  $\mathcal{O}(\log T)$  for  $\mathcal{D} = \Delta_2$  (2-dimensional simplex), through a straight-forward reduction from  $\mathcal{D} = \mathbb{B}(c,\rho)$ .
Due to Corollary 1.1 our results provide a non-trivial answer to  $Q1$  and establish that Alice can substantially improve over the  $\mathcal{O}(\sqrt{T})$  regret guarantees of standard no-regret algorithms.
Corollary 1.3. In the context of alternating game play, Alice can always guarantee at most  $\tilde{\mathcal{O}}\left((\log n)^{4/3}T^{1/3}\right)$  regret regardless the actions of Bob. Moreover in case Alice admits only 2 actions ( $n = 2$ ), the regret bound improves to  $\mathcal{O}(\log T)$ .
Bailey et al. [3] studied alternating game-play in unconstrained two-player games (the strategy space is  $\mathbb{R}^n$  instead of  $\Delta_{n}$ ). They established that if the  $x$ -player (resp. the  $y$ -player) uses Online Gradient Descent (OGD) with constant step-size  $\gamma > 0$  ( $x^{2k} := x^{2k-2} - \gamma Ay^{2k-1}$ ) then it experiences at most  $\mathcal{O}(1/\gamma)$  regret regardless the actions of the  $y$ -player. In the context of alternating OLM this result implies that OGD admits  $\mathcal{O}(1/\gamma)$  alternating regret as long as it always stays in the interior of  $\mathcal{D}$ . However the latter cannot be guaranteed for bounded domains (simplex, ball). In fact there is a simple example for  $\mathcal{D} = \Delta_2$  at which OGD with  $\gamma$  step-size admits  $\Omega(1/\gamma + \gamma T)$  alternating regret. More recently, [40] studied alternating game-play in zero-sum games ( $B = -A^\top$ ). They established that if both players adopt Online Mirror Descent (OMD) the individual regret of each player is at most  $\mathcal{O}(T^{1/3})$  and thus the time-averaged strategies converge to Nash Equilibrium with  $\mathcal{O}(T^{-2/3})$  rate. The setting considered in this works differs because where the  $y$ -player can behave adversarily.
In order to achieve  $\tilde{\mathcal{O}}\left((\log n)^{4/3}T^{1/3}\right)$  alternating regret in case  $\mathcal{D} = \Delta_n$ , we first propose an  $\tilde{\mathcal{O}}(T^{1/3})$  algorithm for the special case of  $\mathcal{D} = \Delta_2$ . For this special case our proposed algorithm is an optimistic-type of Follow the Regularized Leader (FTRL) with log-barrier regularization. Using the latter as an algorithmic primitive, we derive the  $\tilde{\mathcal{O}}\left((\log n)^{4/3}T^{1/3}\right)$  alternating regret algorithm for  $\mathcal{D} = \Delta_n$ , by upper bounding the overall alternating regret by the sum of local alternating regret of 2-actions decision points on a binary tree at which the leafs correspond to the actual  $n$  actions.
In order to achieve  $\mathcal{O}(\rho \log T)$  alternating regret for  $\mathcal{D} = \mathbb{B}(c,\rho)$  we follow a relatively different path. The major primitive of our algorithm is FTRL with adaptive step-size [16, 5]. The cornerstone of our approach is to establish that in case Adaptive FTRL admits more than  $\mathcal{O}(\rho \log T)$  alternating regret, then unnormalized best-response  $(-c^{t - 1})$  can compensate for the additional cost. By using a recent result on Online Gradient Descent with Shrinking Domains [5], we provide an algorithm interpolating between Adaptive FTRL and  $-c^{t - 1}$  that achieves  $\mathcal{O}(\rho \log T)$  alternating regret.
1.4 Further Related Work
The question of going beyond  $\mathcal{O}(\sqrt{T})$  regret in the context of simultaneous game-play has received a lot of attention. A recent line of work establishes that if both agents simultaneously use the same no-regret algorithm (in most cases Optimistic Hedge) to update their strategies, then the individual regret of each agent is  $\tilde{\mathcal{O}}(1)$  [1, 14, 13, 2, 36, 26, 17].
Our work also relates with the more recent works in establishing improved regret bounds parametrized by the cost-vector sequence  $c^1, \ldots, c^T$ , sometimes also called "adaptive" regret bounds [16, 29, 38, 30, 12]. However these parametrized upper bounds focus on finding "easy" instances while still maintaining  $\mathcal{O}(\sqrt{T})$  in the worst case. Alternating OLO can be considered as providing a slight "hint" to the learner that fundamentally changes the worst-case behavior, since its cost is  $(c^t + c^{t-1})^\top x^t$  with the learner being aware of  $c^{t-1}$  prior to selecting  $x_t$ . Improved regret bounds under different notions of hints have been established in [4, 5, 15, 34, 24, 39].
Finally our work also relates with the research line of no-regret learning in the context of Extensive Form Games [44, 18, 19] and Stackelberg Games [27, 22].
We denote with  $\Delta_n \subseteq \mathbb{R}^n$  the  $n$ -dimensional simplex,  $\Delta_n := \{x \in \mathbb{R}^n : x_i \geq 0 \text{ and } \sum_{i=1}^n x_i = 1\}$ .  $\mathbb{B}(c, \rho)$  denotes the ball of radius  $\rho > 0$  centered at  $c \in \mathbb{R}^n$ ,  $\mathbb{B}(c, \rho) := \{x \in \mathbb{R}^n : \|x - c\|_2 \leq \rho\}$ . We also denote with  $[x]_{\mathcal{D}} := \arg \min_{z \in \mathcal{D}} \|z - x\|^2$  the projection operator to set  $\mathcal{D}$ .
2.1 Standard and Alternating Online Linear Minimization
As depicted in Algorithm 1 the only difference between standard and Alternating OLM is the cost of the learner,  $(c^t)^\top x^t$  (OLM) and  $(c^t + c^{t-1})^\top x^t$  (Alternating OLM). Thus, the notion of an online learning algorithm is exactly the same in both settings.
Definition 2.1. An online learning algorithm  $\mathcal{A}$ , for an Online Linear Optimization setting with  $\mathcal{D} \subseteq \mathbb{R}^n$ , is a sequence of functions  $\mathcal{A} := (\mathcal{A}_1, \dots, \mathcal{A}_t, \dots)$  where  $\mathcal{A}_t: \underline{\mathbb{R}^d \times \dots \times \mathbb{R}^d} \mapsto \mathcal{D}$ .
As Definition 2.1 reveals, the notion of an online learning algorithm depends only on the feasibility set  $\mathcal{D}$ . As a result, an online learning algorithm  $\mathcal{A}$  simultaneously admits both standard  $\mathcal{R}_{\mathcal{A}}(T)$  and alternating regret  $\mathcal{R}_{\mathcal{A}}^{\mathrm{alt}}(T)$  (see Equations 1 and 2 for the respective definitions). In Theorem 2.2, we present the well-known lower bound establishing that any online learning algorithm  $\mathcal{A}$  admits  $\mathcal{R}_{\mathcal{A}}(T) = \Omega (\sqrt{T})$  and explain why it fails in the case of alternating regret  $\mathcal{R}_{\mathcal{A}}^{\mathrm{alt}}(T)$ .
Proposition 2.2. Any online learning algorithm  $\mathcal{A}$  for  $\mathcal{D} = \Delta_2$ , admits regret  $\mathcal{R}_{\mathcal{A}}(T) \geq \Omega \left(\sqrt{T}\right)$ .
Proof. Let  $c^t$  be independently selected between  $(-1, 1)$  and  $(1, -1)$  with probability  $1/2$ . Since  $c^t$  is independent of  $(c^1, \ldots, c^{t-1})$  then  $\sum_{t=1}^{T} \operatorname{E}\left[(c^t)^\top x^t\right] = 0$  where  $x^t \coloneqq \mathcal{A}_t(c^1, \ldots, c^{t-1})$ . At the same time,  $\operatorname{E}\left[-\min_{x \in \Delta_2} \sum_{t=1}^{T} (c^t)^\top x\right] \leq \mathcal{O}(\sqrt{T})$ . As a result,  $\mathcal{R}_{\mathcal{A}}(T) \geq \Omega(\sqrt{T})$ .
We now explain why the above randomized construction does not apply for alternating regret  $\mathcal{R}_{\mathcal{A}}^{\mathrm{alt}}(T)$ . Let  $\mathcal{A}$  be the best-response algorithm,  $A_{t}(c^{1},\ldots ,c^{t - 1})\coloneqq \operatorname *{argmin}_{x\in \Delta_{2}}(c^{t - 1})^{\top}x$ . Since  $c^t = (1, - 1)$  or  $c^t = (-1,1)$  we get that  $\min_{x\in \Delta_2}(c^{t - 1})^\top x = -1$  while  $\operatorname {E}\left[(c^t)^\top x^t\right] = 0$  since  $x^{t}\coloneqq \operatorname *{argmin}_{x\in \Delta_{2}}(c^{t - 1})^{\top}x$  and  $c^t$  is independent of  $c^{t - 1}$ . As a result,
$$
\mathrm {E} \left[ \sum_ {t = 1} ^ {T} \left(c ^ {t} + c ^ {t - 1}\right) ^ {\top} x ^ {t} - \sum_ {t = 1} ^ {T} \min  _ {x \in \Delta_ {2}} \left(c ^ {t} + c ^ {t - 1}\right) ^ {\top} x \right] = - T + \Omega (\sqrt {T}).
$$
The latter implies that there exists at least one online learning algorithm (Best-Response) that admits  $\Theta(-T)$  alternating regret in the above randomized construction. However, the latter is not very informative since there is a simple construction at which Best-Response admits linear alternating regret.
We conclude this section with the formal statement of our results. First, for the case that  $\mathcal{D}$  is the simplex, we show  $\tilde{O}(T^{1/3})$  alternating regret (Section 3):
Theorem 2.3. Let  $\mathcal{D}$  be the  $n$ -dimensional simplex,  $\mathcal{D} = \Delta_n$ . There exists an online learning algorithm  $\mathcal{A}$  (Algorithm 3), such that for any cost-vector sequence  $c^1, \ldots, c^T \in [-1,1]^n$ ,
$$
\sum_ {t = 1} ^ {T} \left(c ^ {t - 1} + c ^ {t}\right) ^ {\top} x ^ {t} - \min  _ {x ^ {*} \in \mathcal {D}} \sum_ {t = 1} ^ {T} \left(c ^ {t - 1} + c ^ {t}\right) ^ {\top} x ^ {*} \leq \mathcal {O} \left(T ^ {1 / 3} \cdot \log^ {4 / 3} (n T)\right) \text {w h e r e} x ^ {t} = \mathcal {A} _ {t} \left(c ^ {1}, \dots , c ^ {t - 1}\right).
$$
Next, when  $\mathcal{D}$  is a ball of radius  $\rho$ , we can improve to  $\tilde{O}(1)$  alternating regret (Section 4):
Theorem 2.4. Let  $\mathcal{D}$  be a ball of radius  $\rho$ ,  $\mathcal{D} = \mathbb{B}(c, \rho)$ . There exists an online learning algorithm  $\mathcal{A}$  (Algorithm 4) such that for any  $\mathcal{T}$  cost-vector sequence  $c^1, \ldots, c^T$  where  $\|c^t\|_2 \leq 1$ ,
$$
\sum_ {t = 1} \left(c ^ {t - 1} + c ^ {t}\right) ^ {\top} x ^ {t} - \min  _ {x ^ {*} \in \mathcal {D}} \sum_ {t = 1} \left(c ^ {t - 1} + c ^ {t}\right) ^ {\top} x ^ {*} \leq \mathcal {O} (\rho \log T) \quad w h e r e \quad x ^ {t} = \mathcal {A} _ {t} \left(c ^ {1}, \dots , c ^ {t - 1}\right).
$$
Remark 2.5. Using Algorithm 4 we directly get an online learning algorithm with  $\mathcal{O}(\log T)$  alternating regret for  $\mathcal{D} = \Delta_2$ .
2.2 Alternating Game-Play
A two-player normal form game  $(A,B)$  is defined by the payoff matrix  $A\in [-1,1]^{n\times m}$  denoting the payoff of Alice and the matrix  $B\in [-1,1]^{m\times n}$  denoting the payoff of Bob. Once the Alice selects a mixed strategy  $x\in \Delta_{n}$  (prob. distr. over [n]) and Bob selects a mixed strategy  $y\in \Delta_{m}$  (prob. distr. over [n]). Then Alice suffers (expected) cost  $x^{\top}Ay$  and Bob  $y^{\top}Bx$
In alternating game-play, Alice updates her mixed strategy in the even rounds while Bob updates in the odd rounds. As a result, a sequence of alternating play for  $T = 2K$  rounds (resp. for  $T = 2K + 1$ ) admits the form  $(x^{1},y^{2}),(x^{3},y^{2}),\ldots ,(x^{2k + 1},y^{2k})$ ,  $(x^{2k + 1},y^{2k + 2}),\ldots ,(x^{2K - 1},y^{2K})$ . Thus, the regret of Alice in the above sequence of play equals the difference between her overall cost and the cost of the best-fixed action,
$$
\mathcal{R}_{x}(T):= \underbrace{(x^{1})^{\top}Ay^{2} + \sum_{k = 1}^{T / 2 - 1}(x^{2k + 1})^{\top}A(y^{2k} + y^{2k + 2})}_{\text{Alice's cost}} - \underbrace{\min_{x\in\Delta_{n}}\left[x^{\top}Ay^{1} + \sum_{k = 1}^{T / 2 - 1}x^{\top}A(y^{2k} + y^{2k + 2})\right]}_{\text{cost of Alice's best action}}
$$
If Alice selects  $x^{2k + 1} \coloneqq \mathcal{A}_k(Ay^2, Ay^4, \ldots, Ay^{2k - 2}, Ay^{2k})$  for  $k \in [K - 1]$  and  $x_1 = \mathcal{A}_1(\cdot)$  then by the definition of alternating regret in Equation 2, we get that
$$
\left(x ^ {1}\right) ^ {\top} A y ^ {2} + \sum_ {k = 1} ^ {K - 1} \left(x ^ {2 k + 1}\right) ^ {\top} \left(A y ^ {2 k} + A y ^ {2 k + 2}\right) - \min  _ {x \in \Delta_ {n}} \left[ x ^ {\top} A y ^ {2} + \sum_ {k = 1} ^ {K - 1} x ^ {\top} \left(A y ^ {2 k} + A y ^ {2 k + 2}\right) \right] \leq \mathcal {R} _ {\mathcal {A}} ^ {\mathrm {a l t}} (K)
$$
which establishes Corollary 1.1. The proof for  $T = 2K + 1$  is the same by considering  $Ay^{2K + 2} = 0$ .
3 The Simplex case
Before presenting our algorithm for the  $n$ -dimensional simplex, we present Algorithm 2 that admits  $\mathcal{O}(\log^{2/3} T \cdot T^{1/3})$  alternating regret for the 2-simplex and is the basis of our algorithm for  $\Delta_n$ .
Definition 3.1 (Log-Barrier Regularization). Let the function  $R: \Delta_2 \mapsto \mathbb{R}_{\geq 0}$  where  $R(x) := -\log x_1 - \log x_2$ .
Algorithm 2 Online Learning Algorithm for 2D-Simplex
1: Input:  $c^0 \gets (0, 0)$
2: for rounds  $t = 1, \dots, T$  do
3: The learner selects  $x^t \coloneqq \min_{x \in \Delta_2} [2(c^{t-1})^\top x + \sum_{\tau=1}^{t-1} (c^\tau + c^{\tau-1})^\top x + R(x) / \gamma]$ .
4: The adversary selects cost vector  $c^t \in [0,1]^n$
5: The learner suffers cost  $(c^t + c^{t-1})^\top x^t$
6: end for
In order to analyze Algorithm 2 we will compare its performance with the performance of the Be the Regularized Leader algorithm with log-barrier regularization that is ensured to achieve  $\mathcal{O}(\log T / \gamma)$  alternating regret [23]. The latter is formally stated and established in Lemma 3.2.