SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We are now ready to state the main result of this subsection. Below, we use the notation  $\mathrm{EQGAP}^{(t)}(\pmb{z}^{(t)})$  to represent the Nash equilibrium gap of  $(\pmb{x}^{(t)},\pmb{y}^{(t)}) = \pmb{z}^{(t)}\in \mathcal{Z}$  at the  $t$ -th game. More precisely,  $\mathrm{EQGAP}^{(t)}(\pmb{z}^{(t)})\coloneqq \max \{\mathrm{BR}_x^{(t)}(\pmb{x}^{(t)}),\mathrm{BR}_y^{(t)}(\pmb{y}^{(t)})\}$ , where  $\mathrm{BR}_x^{(t)}(\pmb{x}^{(t)})\coloneqq \max_{\pmb {x}\in \mathcal{X}}\{\langle \pmb {x},\pmb{u}_x^{(t)}\rangle \} -\langle \pmb {x}^{(t)},\pmb{u}_x^{(t)}\rangle$  is the best response gap of Player  $x$  (and analogously for Player  $y$ ).
Theorem 3.3 (Detailed version in Theorem A.11). Suppose that both players employ  $\mathcal{OGD}$  with learning rate  $\eta = \frac{1}{4L}$  in a sequence of time-varying BSPPs, where  $L \coloneqq \max_{1 \leq t \leq T} \| \mathbf{A}^{(t)} \|_2$ . Then,  $\sum_{t=1}^{T} \left( \mathrm{EQGAP}^{(t)}(\boldsymbol{z}^{(t)}) \right)^2 = O\left( 1 + \mathcal{V}_{\epsilon - NE}^{(T)} + \mathcal{V}_{\mathbf{A}}^{(T)} \right)$ , where  $(\boldsymbol{z}^{(t)})_{1 \leq t \leq T}$  is the sequence of joint strategy profiles produced by  $\mathcal{OGD}$ .
The proof of this theorem is based on upper bounding the second-order path length of  $\mathsf{OGD}$ ,  $\sum_{t=1}^{T}\left(\|z^{(t)}-\hat{z}^{(t)}\|_{2}^{2}+\|z^{(t)}-\hat{z}^{(t+1)}\|_{2}^{2}\right)$ , as a function of the variation measures  $\nu_{\epsilon-\mathrm{NE}}^{(T)}$  and  $\nu_{\mathbf{A}}^{(T)}$ . This is shown via a dynamic RVU bound [93] (Lemmas A.1 and A.2) in conjunction with Property 3.1.
It is worth noting that when the deviation of the payoff matrices is controlled by the deviation of the players' strategies, in the sense that  $\sum_{t=1}^{T-1}\|\mathbf{A}^{(t+1)} - \mathbf{A}^{(t)}\|_2^2 \leq W^2\sum_{t=1}^{T-1}\|z^{(t+1)} - z^{(t)}\|_2^2$  for some parameter  $W \in \mathbb{R}_{>0}$ , the variation measure  $\mathcal{V}_{\mathbf{A}}^{(T)}$  in Theorem 3.3 can be entirely eliminated; see Corollary A.9. The same in fact applies under an improved prediction mechanism (see Remark A.13); while that prediction is not implementable in the online learning setting, it can be used, for example, when the sequence of games is known in advance (as is the case in certain applications).
We next state some immediate consequences of Theorem 3.3. (Item 2 below follows from Theorem 3.3 by Jensen's inequality.)
Corollary 3.4. In the setting of Theorem 3.3,
1. If at least a  $\delta$ -fraction of the iterates have  $\epsilon > 0$  NE gap,  $\epsilon^2\delta \leq O\left(\frac{1}{T}\left(1 + \mathcal{V}_{\epsilon - NE}^{(T)} + \mathcal{V}_{\mathbf{A}}^{(T)}\right)\right)$ .
2. The average NE gap is bounded as  $O\left(\sqrt{\frac{1}{T}\left(1 + \mathcal{V}_{\epsilon - NE}^{(T)} + \mathcal{V}_{\mathbf{A}}^{(T)}\right)}\right)$ .
In particular, in terms of asymptotic implications, if  $\lim_{T\to +\infty}\frac{\mathcal{V}_{\epsilon - \mathrm{NE}}^{(T)}}{T},\lim_{T\to +\infty}\frac{\mathcal{V}_{\mathbf{A}}^{(T)}}{T} = 0$  then (i) for any  $\epsilon >0$  the fraction of iterates of OGD with at least an  $\epsilon$  Nash equilibrium gap converges to 0; and (ii) the average Nash equilibrium gap of the iterates of OGD converges to 0.
In the special case where  $\mathcal{V}_{\epsilon -\mathrm{NE}}^{(T)},\mathcal{V}_{\mathbf{A}}^{(T)} = O(1)$ , Theorem 3.3 recovers the  $T^{-1 / 2}$  iterate-convergence rate of OG in static bilinear saddle-point problems.
Meta-learning Our results also have immediate applications in the meta-learning setting. More precisely, meta-learning in games is a special case of time-varying games which consists of a sequence of  $H \in \mathbb{N}$  separate games, each of which is repeated for  $m \in \mathbb{N}$  consecutive rounds, so that  $T \coloneqq m \times H$ . The central goal in meta-learning is to obtain convergence bounds parameterized by the similarity of the games; identifying suitable similarity metrics is a central question in that line of work.
In this context, we highlight that Theorem 3.3 shown above readily provides a meta-learning guarantee parameterized by the following notion of similarity between the Nash equilibria:  $\inf_{\boldsymbol{z}^{(h,\star)}\in \mathcal{Z}^{(h,\star)},\forall h\in [H]}\sum_{h = 1}^{H - 1}\| \boldsymbol{z}^{(h + 1,\star)} - \boldsymbol{z}^{(h,\star)}\| _2$ , where  $\mathcal{Z}^{(h,\star)}$  is the set of Nash equilibria of the  $h$ -th game in the meta-learning sequence, as well as the similarity of the payoff matrices—corresponding to the term  $\nu_{\mathbf{A}}^{(T)}$ . In fact, under a suitable prediction—the one used by Harris et al. [48]—the dependence on  $\nu_{\mathbf{A}}^{(T)}$  can be entirely removed; see Proposition A.14 for our formal result. A compelling aspect of our meta-learning guarantee is that the considered algorithm is oblivious to the boundaries of the meta-learning. It is also worth noting that our similarity metric can be arbitrarily
2In accordance with Theorem 3.3, this similarity metric can be refined using a sequence of approximate NE.
smaller than the one considered by Harris et al. [48] (Proposition A.15). We further provide some novel results on meta-learning in general-sum games in Section 3.3.
3.1.1 Beyond bilinear saddle-point problems
As we alluded to earlier, our approach can be generalized beyond time-varying bilinear saddle-point problems to a more general class of time-varying variational inequality (VIs) as follows. Let  $F^{(t)}: \mathcal{Z} \to \mathcal{Z}$  be the (single-valued) operator of the VI problem at time  $t \in \mathbb{N}$ .  $F^{(t)}$  is said to satisfy the MVI property [61, 36] if there exists a point  $z^{(t,\star)} \in \mathcal{Z}$  such that  $\langle z - z^{(t,\star)}, F^{(t)}(z) \rangle \geq 0$  for any  $z \in \mathcal{Z}$ . For example, in the special case of a bilinear saddle-point problem we have that  $F: z := (x,y) \mapsto (\mathbf{A}y, -\mathbf{A}^\top x)$ , and the MVI property is satisfied by virtue of Von Neumann's minimax theorem. It is direct to see that Property 3.1 applies to any time-varying sequence of VIs with respect to  $(z^{(t,\star)})_{1 \leq t \leq T}$  as long as every operator in the sequence  $(F^{(1)}, \ldots, F^{(T)})$  satisfies the MVI property. (Even more broadly, it would suffice if almost all operators in the sequence—in that their fraction converges to 1 as  $T \to +\infty$ —satisfy the MVI property.) This observation enables extending Theorem 3.3 to a more general class of problems. As a concrete example, we provide a generalization of Theorem 3.3 (Theorem A.16) to polymatrix zero-sum games [15] in Appendix A.1.7. By contrast, in problems where the MVI property fails it appears that a much different approach is called for.
We finally point out that our framework could have certain implications for solving (static) general VIs, as we discuss in Appendix A.1.8.
3.2 Strongly convex-concave games
In this subsection, we show that under additional structure we can significantly improve the variation measures established in Theorem 3.3. More precisely, we first assume that each objective function  $f(\pmb{x},\pmb{y})$  is  $\mu$ -strongly convex with respect to  $\pmb{x}$  and  $\mu$ -strongly concave with respect to  $\pmb{y}$ . Our second assumption is that each game is played for multiple rounds  $m \in \mathbb{N}$ , instead of only a single round; this is akin to the meta-learning setting. The key insight is that as long as  $m$  is large enough,  $m = \Omega(1/\mu)$ , those two assumptions suffice to obtain a second-order variation bound in terms of the sequence of Nash equilibria,  $S_{\mathrm{NE}}^{(H)} := \sum_{h=1}^{H-1} \| \pmb{z}^{(h+1,\star)} - \pmb{z}^{(h,\star)} \|_2^2$ , where  $\pmb{z}^{(h,\star)}$  is a Nash equilibrium of the  $h$ -th game. This significantly refines the result of Theorem 3.3, and is inspired by the improved dynamic regret bounds obtained by Zhang et al. [92]. Below we sketch the key ideas of the improvement; full proofs are deferred to Appendix A.2.
In this setting, it is assumed that Player  $x$  obtains the utility  $\boldsymbol{u}_x^{(t)} \coloneqq -\nabla_{\boldsymbol{x}}f^{(t)}(\boldsymbol{x}^{(t)},\boldsymbol{y}^{(t)})$  at every time  $t \in [[T]]$ , while its regret will be denoted by  $\mathrm{Reg}_{\mathcal{L},y}^{(T)}$ ; similar notation applies for Player  $y$ . The first observation is that, focusing on a single (static) game, under strong convexity-concavity the sum of the players' regrets are strongly nonnegative (Lemma A.18):
$$
\operatorname {R e g} _ {\mathcal {L}, x} ^ {(m)} \left(\boldsymbol {x} ^ {\star}\right) + \operatorname {R e g} _ {\mathcal {L}, y} ^ {(m)} \left(\boldsymbol {y} ^ {\star}\right) \geq \frac {\mu}{2} \sum_ {t = 1} ^ {m} \| \boldsymbol {z} ^ {(t)} - \boldsymbol {z} ^ {\star} \| _ {2} ^ {2}, \tag {3}
$$
for any Nash equilibrium  $z^{\star} \in \mathcal{Z}$  of the game. In turn, this can be cast in terms of dynamic regret over the sequence of the  $h$  games (Lemma A.19). Next, combining those dynamic-regret lower bounds with a suitable RVU-type property leads to a refined second-order path length bound as long as  $m = \Omega(1/\mu)$ , which in turn leads to our main result below. Before we present its statement, let us introduce the following measure of variation of the gradients:  $\mathcal{V}_{\nabla f}^{(H)} := \sum_{h=1}^{H-1} \max_{z \in \mathcal{Z}} \| F^{(h+1)}(z) - F^{(h)}(z) \|_2^2$ , where let  $F: z \coloneqq (x, y) \mapsto (\nabla_x f(x, y), -\nabla_y f(x, y))$ . This variation measure is analogous to  $\mathcal{V}_{\mathbf{A}}^{(T)}$  we introduced earlier for time-varying BSPPs.
Theorem 3.5 (Detailed version in Theorem A.21). Let  $f^{(h)}: \mathcal{X} \times \mathcal{Y}$  be a  $\mu$ -strongly convex-concave and  $L$ -smooth function, for all  $h \in [[H]]$ . Suppose further that both players employ OGDP with learning rate  $\eta = \min \left\{\frac{1}{8L}, \frac{1}{2\mu}\right\}$  for  $T \in \mathbb{N}$  repetitions, where  $T = m \times H$  and  $m \geq \frac{2}{\eta\mu}$ . Then,  $\sum_{t=1}^{T} \left(\mathrm{EQGAP}^{(t)}(\boldsymbol{z}^{(t)})\right)^2 = O(1 + S_{NE}^{(H)} + \mathcal{V}_{\nabla f}^{(H)})$ .
Our techniques also imply the improved regret bounds  $\mathrm{Reg}_{\mathcal{L},x}^{(T)}, \mathrm{Reg}_{\mathcal{L},y}^{(T)} = O\left(\sqrt{1 + \mathcal{S}_{\mathrm{NE}}^{(H)} + \mathcal{V}_{\nabla f}^{(H)}}\right)$ , under suitable tuning of the learning rate (see Corollary A.22).
There is another immediate but important implication of (3): any no-regret algorithm in a (static) strongly convex-concave setting ought to be approaching the Nash equilibrium; in contrast, this property is spectacularly false in (general) monotone settings [60].
Proposition 3.6. Let  $f: \mathcal{X} \times \mathcal{Y} \to \mathbb{R}$  be a  $\mu$ -strongly convex-concave function. If players incur regrets such that  $\operatorname{Reg}_{\mathcal{L},x}^{(T)} + \operatorname{Reg}_{\mathcal{L},y}^{(T)} \leq CT^{1 - \omega}$ , for some parameters  $C > 0$  and  $\omega \in (0,1]$ , then for any  $\epsilon > 0$  and  $T > \left(\frac{2C}{\mu\epsilon^2}\right)^{1 / \omega}$  there is a pair of strategies  $z^{(t)} \in \mathcal{Z}$  such that  $\| z^{(t)} - z^{\star}\|_2 \leq \epsilon$ , where  $z^{\star} \in \mathcal{Z}$  is a Nash equilibrium.
The insights of this subsection are also of interest in general monotone settings by incorporating a strongly convex regularizer; tuning its magnitude allows us to trade off between a better approximation and the benefits of strong convexity-concavity revealed in this subsection.
3.3 General-sum multi-player games
Next, we turn our attention to general-sum multi-player games. For simplicity, in this subsection we posit that the game is represented in normal form, so that each player  $i \in [[n]]$  has a finite set of available actions  $\mathcal{A}_i$ , and  $\mathcal{X}_i \coloneqq \Delta(\mathcal{A}_i)$ . The proofs of this subsection are included in Appendix A.3.
Potential games First, we study the convergence of (online) gradient descent (GD) in time-varying potential games (see Definition A.23 for the formal description); we recall that unlike two-player zero-sum games, gradient descent is known to approach Nash equilibria in potential games. In our time-varying setup, it is assumed that each round  $t \in [[T]]$  corresponds to a different potential game described with a potential function  $\Phi^{(t)}$ . We further let  $d: (\Phi, \Phi') \mapsto \max_{z \in X_{i=1}^n} \chi_i(\Phi(z) - \Phi'(z))$ , so that  $\mathcal{V}_{\Phi}^{(T)} := \sum_{t=1}^{T-1} d(\Phi^{(t)}, \Phi^{(t+1)})$ ; we call attention to the fact that  $d(\cdot, \cdot)$  is not symmetric. Analogously to Theorem 3.3, we use  $\mathrm{EQGAP}^{(t)}(\boldsymbol{z}^{(t)}) \in \mathbb{R}_{\geq 0}$  to represent the NE gap of the joint strategy profile  $\boldsymbol{z}^{(t)} := (\boldsymbol{x}_1^{(t)}, \ldots, \boldsymbol{x}_n^{(t)})$  at the  $t$ -th game.
Theorem 3.7. Suppose that each player employs (online)  $GD$  with a sufficiently small learning rate in a sequence of time-varying potential games. Then,  $\sum_{t=1}^{T} \left( \mathrm{EQGAP}^{(t)}(z^{(t)}) \right)^2 = O(\Phi_{max} + \mathcal{V}_{\Phi}^{(T)})$ , where  $\Phi_{max}$  is such that  $|\Phi^{(t)}(\cdot)| \leq \Phi_{max}$ .
We refer to Appendix B for some illustrative experiments related to Theorem 3.7.
General games Unfortunately, unlike the settings considered thus far, computing Nash equilibria in general games is computationally hard even under a crude approximation  $\epsilon = \Theta(1)$  [25, 19, 30]. Instead, learning algorithms are known to converge—in a time-average sense—to relaxations of the Nash equilibrium, known as (coarse) correlated equilibria. For our purposes, we will employ a bilinear formulation of correlated equilibria, which dates back to the seminal work of Hart and Schmeidler [50] (see also [84, Chapter 12] for an excellent exposition). This will allow us to translate the results of Section 3.1 to general multi-player games.
Specifically, correlated equilibria<sup>4</sup> can be phrased via a game between the  $n$  players and a mediator, an additional agent. At a high level, the mediator is endeavoring to identify a correlated strategy  $\pmb{\mu} \in \Xi := \Delta \left( \bigotimes_{i=1}^{n} \mathcal{A}_i \right)$  for which no player has an incentive to deviate from the recommendation. In contrast, the players are trying to optimally deviate so as to maximize their own utility. More precisely, there exist matrices  $\mathbf{A}_1, \ldots, \mathbf{A}_n$ , with each matrix  $\mathbf{A}_i$  depending solely on the utility of Player  $i$ , for which the bilinear saddle-point problem can be expressed as
$$
\min  _ {\boldsymbol {\mu} \in \Xi} \max  _ {(\bar {\boldsymbol {x}} _ {1}, \dots , \bar {\boldsymbol {x}} _ {n}) \in \times_ {i = 1} ^ {n}} \bar {\mathcal {X}} _ {i} \sum_ {i = 1} ^ {n} \boldsymbol {\mu} ^ {\top} \mathbf {A} _ {i} \bar {\boldsymbol {x}} _ {i}, \tag {4}
$$
<sup>3</sup>Such observations were independently documented by Wang et al. [85].
There is also a bilinear formulation tailored to coarse correlated equilibria, but we will focus solely on the stronger variant (CE) for concreteness.
where  $\bar{\mathcal{X}}_i$  above is a suitable set of strategies; we elaborate more on this formulation in Appendix A.3. This zero-sum game has the property that there exists a strategy  $\pmb{\mu}^{\star}\in \Xi$  such that  $\max_{\bar{\pmb{x}}_i\in \bar{\mathcal{X}}_i}(\pmb {\mu}^\star)^\top \mathbf{A}_i\bar{\pmb{x}}_i\leq 0$ , for any player  $i\in [[n]]$ , which is precisely a correlated equilibrium.
Before we proceed, it is important to note that the learning paradigm considered here deviates from the traditional one in that orchestrating the protocol requires an additional learning agent, resulting in a less decentralized protocol. Yet, the dynamics induced by solving (4) via online algorithms remain uncoupled [49], in the sense that each player obtains feedback—corresponding to the deviation benefit—that depends solely on its own utility.
Now in the time-varying setting, the matrices  $\mathbf{A}_1,\ldots ,\mathbf{A}_n$  that capture the players' utilities can change in each repetition. Crucially, we show that the structure of the induced bilinear problem (4) is such that there is a sequence of correlated equilibria that guarantee nonnegative dynamic regret; this refines Property 3.1 in that only one player's strategies suffice to guarantee nonnegativity, even if the strategy of the other player remains invariant. Below, we denote by  $\mathrm{DReg}_{\mu}^{(T)}$  the dynamic regret of Player min in (4), and by  $\mathrm{Reg}_i^{(T)}$  the regret of each player  $i\in [[n]]$  up to time  $T\in \mathbb{N}$ , so that the regret of Player max in (4) can be expressed as  $\sum_{i = 1}^{n}\mathrm{Reg}_i^{(T)}$ .
Property 3.8. Suppose that  $\Xi \ni \pmb{\mu}^{(t,\star)}$  is a correlated equilibrium of the game at any time  $t\in [[T]]$ . Then,  $\mathrm{DReg}_{\mu}^{(T)}(\pmb{\mu}^{(1,\star)},\dots,\pmb{\mu}^{(T,\star)}) + \sum_{i = 1}^{n}\mathrm{Reg}_{i}^{(T)}\geq 0$ .
As a result, this enables us to apply Theorem 3.3 parameterized on (i) the variation of the CE  $\mathcal{V}_{\mathrm{CE}}^{(T)}\coloneqq \inf_{\pmb{\mu}^{(t,\star)}\in \Xi^{(t,\star)},\forall t\in [[T]]}\sum_{t = 1}^{T - 1}\| \pmb{\mu}^{(t + 1,\star)} - \pmb{\mu}^{(t,\star)}\| _2$  where  $\Xi^{(t,\star)}$  denotes the set of CE of the  $t$ -th game, and (ii) the variation in the players' utilities  $\mathcal{V}_{\mathbf{A}}^{(T)}\coloneqq \sum_{i = 1}^{n}\sum_{t = 1}^{T - 1}\| \mathbf{A}_i^{(t + 1)} - \mathbf{A}_i^{(t)}\| _2^2;$  below, we denote by  $\mathrm{CEGAP}^{(t)}(\pmb{\mu}^{(t)})$  the CE gap of  $\pmb{\mu}^{(t)}\in \Xi$  at the  $t$ -th game.
Theorem 3.9. Suppose that each player employs OGDP in a sequence of time-varying BSPPs (4) with a sufficiently small learning rate. Then,  $\sum_{t=1}^{T}\left(\mathbf{CEGAP}^{(t)}(\boldsymbol{\mu}^{(t)})\right)^2 = O(1 + \mathcal{V}_{CE}^{(T)} + \mathcal{V}_{\mathbf{A}}^{(T)})$ .
There are further interesting implications of our framework that are worth highlighting. First, we obtain meta-learning guarantees for general games that depend on the (algorithm-independent) similarity of the correlated equilibria (Corollary A.25); that was left as an open question by Harris et al. [48], where instead algorithm-dependent similarity metrics were derived. Further, by applying Corollary A.12, we derive natural variation-dependent per-player regret bounds in general games (Corollary A.26); this addresses a question left by Zhang et al. [93], albeit under a different learning paradigm. We suspect that obtaining such results—parameterized on the variation of the CE—are not possible without the presence of the additional agent, as in (4).
3.4 Dynamic regret bounds in static games
Finally, in this subsection we switch gears by investigating dynamic regret guarantees when learning in static games. The proofs of this subsection are included in Appendix A.4.
First, we point out that while traditional no-regret learning algorithms guarantee  $O(\sqrt{T})$  external regret, instances of 0MD—a generalization of 0GD; see (5) in Appendix A—in fact guarantee  $O(\sqrt{T})$  dynamic regret in two-player zero-sum games, which is a much stronger performance measure:
Proposition 3.10. Suppose that both players in a (static) two-player zero-sum game employ  $\mathcal{O}MD$  with a smooth regularizer. Then,  $\mathrm{DReg}_x^{(T)}$ ,  $\mathrm{DReg}_y^{(T)} = O(\sqrt{T})$ .
In proof, the dynamic regret of each player under OMD with a smooth regularizer can be bounded by the first-order path length of that player's strategies, which in turn can be bounded by  $O(\sqrt{T})$  given that the second-order path length is  $O(1)$  (Theorem A.7). In fact, Theorem A.7 readily extends Proposition 3.10 to time-varying zero-sum games as well, implying that  $\mathrm{DReg}_x^{(T)},\mathrm{DReg}_y^{(T)} = O\left(\sqrt{T(1 + \mathcal{V}_{\epsilon - \mathrm{NE}}^{(T)} + \mathcal{V}_{\mathbf{A}}^{(T)})}\right)$ .
A question that arises from Proposition 3.10 is whether the  $O(\sqrt{T})$  guarantee for dynamic regret of OMD can be improved in the online learning setting. Below, we point out a significant improvement to
$O(\log T)$ , but under a stronger two-point feedback model; namely, we posit that in every round each player can select an additional auxiliary strategy, and each player then gets to additionally observe the utility corresponding to the auxiliary strategies. Notably, this is akin to how the extra-gradient method works [52] (also cf. [72, Section 4.2] for multi-point feedback models in the bandit setting).
Observation 3.11. Under two-point feedback, there exist learning algorithms that guarantee  $\mathrm{DReg}_x^{(T)},\mathrm{DReg}_y^{(T)} = O(\log T)$  in two-player zero-sum games.
In particular, it suffices for each player to employ OMD, but with the twist that the first strategy in each round is the time-average of OMD; the auxiliary strategy is the standard output of OMD. Then, the dynamic regret of each player will grow as  $O\left(\sum_{t=1}^{T} \frac{1}{t}\right) = O(\log T)$  since the duality gap of the average strategies is decreasing with a rate of  $T^{-1}$  [72]. It is an interesting question whether the bound of Observation 3.11 can be improved to  $O(1)$  [14]; we conjecture that there is a lower bound of  $\Omega(\log T)$ .
General-sum games In stark contrast, no (computationally efficient) sublinear dynamic regret guarantees are possible in general-sum games:
Proposition 3.12. Any polynomial-time algorithm incurs  $\sum_{i=1}^{n} \mathrm{DReg}_i^{(T)} \geq CT$  for any polynomial  $T \in \mathbb{N}$ , even if  $n = 2$  and  $C > 0$  is an absolute constant, unless ETH for PPAD is false [73].
Indeed, this follows immediately since computing a Nash equilibrium to  $O(1)$  precision in two-player games requires superpolynomial time [73]. As such, Proposition 3.12 applies beyond the online learning setting. This motivates considering a relaxation of dynamic regret, wherein the sequence of comparators is subject to the constraint  $\sum_{t=1}^{T-1} \mathbb{1}\{\pmb{x}_i^{(t+1,\star)} \neq \pmb{x}_i^{(t,\star)}\} \leq K-1$ , for some parameter  $K \in \mathbb{N}$  [22]; this will be referred to as  $K\text{-DReg}_i^{(T)}$ . Naturally, external regret coincides with  $K\text{-DReg}_i^{(T)}$  under  $K = 1$ . In this context, we employ Lemma A.1 to bound  $K\text{-DReg}^{(T)}$  under 0GD:
Theorem 3.13 (Precise version in Theorem A.28). Suppose that all  $n$  players employ OGD with a suitable learning rate in an  $L$ -smooth game. Then, for any  $K \in \mathbb{N}$ ,
1.  $\sum_{i=1}^{n} K\text{-}\mathrm{DReg}_i^{(T)} = O(K\sqrt{n}LD_{\mathcal{Z}}^2)$ ;
2.  $K$ - $\mathrm{DReg}_i^{(T)} = O(K^{3/4}T^{1/4}n^{1/4}L^{1/2}D_{\mathcal{X}_i}^{3/2})$ , for any player  $i \in [[n]]$ .
One question that arises here is whether the per-player bound of  $O(K^{3/4}T^{1/4})$  (Item 2) can be improved to  $\tilde{O}(K)$ , where  $\tilde{O}(\cdot)$  hides logarithmic factors. The main challenge is that, even for  $K = 1$ , all known methods that obtain  $\tilde{O}(1)$  [29, 68, 1, 38] rely on non-smooth regularizers that violate the preconditions of Lemma A.2—our dynamic RVU bound beyond (squared) Euclidean regularization; yet, we point out that it is possible under a slightly stronger feedback model (Remark A.29). We finally highlight that the game-theoretic significance of the solution concept that arises as the limit point of no-regret learners when  $K\text{-DReg} = o(T)$  was recently investigated by Crippa et al. [22].
4 Conclusions and future work