SlowGuess/NeurIPS_2023_OCR huggingface dataset:
A.1.6 Meta-learning
We next provide the implication of Theorem 3.3 in the meta-learning setting. We first make a remark regarding the effect of the prediction of OGD to Theorem 3.3, and how that relates to an assumption present in earlier work on this problem [48].
Remark A.13 (Improved predictions). Throughout Section 3.1, we have considered the standard prediction  $\boldsymbol{m}_x^{(t)} \coloneqq \boldsymbol{u}_x^{(t-1)} = -\mathbf{A}^{(t-1)} \boldsymbol{y}^{(t-1)}$  for  $t \geq 2$ , and similarly for Player  $y$ . It is easy to see that using the predictions
$$
\boldsymbol {m} _ {x} ^ {(t)} := - \mathbf {A} ^ {(t)} \boldsymbol {y} ^ {(t - 1)} \text {a n d} \boldsymbol {m} _ {y} ^ {(t)} := (\mathbf {A} ^ {(t)}) ^ {\top} \boldsymbol {x} ^ {(t - 1)} \tag {37}
$$
for  $t \geq 1$  (where  $z^{(0)} \coloneqq \hat{z}^{(1)}$ ) entirely removes the dependency on  $\mathcal{V}_{\mathbf{A}}^{(T)}$  on all our convergence bounds. While such a prediction cannot be implemented in the standard online learning model, there are settings in which we might, for example, know the sequence of matrices in advance; the meta-learning setting offers such examples, and indeed, Harris et al. [48] use the improved prediction of (37).
Proposition A.14 (Meta-learning). Suppose that both players employ OGDP with learning rate  $\eta = \frac{1}{4L}$  where  $L\coloneqq \max_{1\leq h\leq H}\| \mathbf{A}^{(h)}\| _2$  and the prediction of (37) in a meta-learning bilinear saddle-point problem with  $H\in \mathbb{N}$  games, each repeated for  $m\in \mathbb{N}$  consecutive iterations. Then, for an average game,
$$
\left[ \frac {P}{H \epsilon^ {2}} + \frac {P ^ {\prime} \mathcal {V} _ {N E} ^ {(H)}}{H \epsilon^ {2}} \right] + 1 \tag {38}
$$
iterations suffice to reach an  $\epsilon$ -approximate Nash equilibrium, where  $P := 4L^2(4D_{\mathcal{Z}} + \| \mathcal{Z} \|_2)^2 D_{\mathcal{Z}}^2$ ,  $P' := 8L^2(4D_{\mathcal{Z}} + \| \mathcal{Z} \|_2)^2 D_{\mathcal{Z}}$ , and
$$
\mathcal{V}_{NE}^{(H)}:= \inf_{\boldsymbol{z}^{(h,\star)}\in \mathcal{Z}^{(h,\star)},\forall h\in \llbracket H\rrbracket}\sum_{h = 1}^{H - 1}\| \boldsymbol{z}^{(h + 1,\star)} - \boldsymbol{z}^{(h,\star)}\|_{2}.
$$
The proof is a direct application of Theorem A.11, where we remark that the term depending on  $\mathcal{V}_{\mathbf{A}}^{(T)}$  and the term  $4\eta^2 L^2\| \mathcal{Z}\| _2^2$  from Theorem A.11 are eliminated because of the improved prediction of Remark A.13. More precisely, if we let  $m^{(h)}\in \mathbb{N}$  be the number of iterations required so that the equilibrium gap is at most  $\epsilon$  at the  $h$ -th game, then Theorem A.11 implies that
$$
P + P ^ {\prime} \mathcal {V} _ {\mathrm {N E}} ^ {(H)} \geq \sum_ {t = 1} ^ {T} \left(\operatorname {E Q G A P} ^ {(t)} \left(\boldsymbol {z} ^ {(t)}\right)\right) ^ {2} > \epsilon^ {2} \sum_ {h = 1} ^ {H} (m ^ {(h)} - 1).
$$
Solving with respect to  $\frac{1}{H}\sum_{h = 1}^{H}m^{(h)}$  thus yields Proposition A.14.
The first term in the iteration complexity bound (38) vanishes in the meta-learning regime—as the number of games increases  $H \gg 1$ —while the second term is proportional to  $\frac{\mathcal{V}_{\mathrm{NE}}^{(H)}}{H}$ , a natural similarity measure; (38) always recovers the  $m^{-1/2}$  rate, but offers significant gains if the games as similar, in the sense that  $\frac{\mathcal{V}_{\mathrm{NE}}^{(H)}}{H} \ll 1$ . It is worth noting that, unlike the similarity measure derived by Harris et al. [48],  $\frac{\mathcal{V}_{\mathrm{NE}}^{(H)}}{H}$  depends on the order of the games. As a result, a natural problem that arises in settings where the sequence of games is known in advance is to optimize the order of the games so as to minimize  $\mathcal{V}_{\mathrm{NE}}^{(H)}$ ; the key challenge of course is to minimize  $\mathcal{V}_{\mathrm{NE}}^{(H)}$  without actually solving each game since that without defeat the purpose. We further remark that Proposition A.14 can be readily extended even if each game in the meta-learning sequence is not repeated for the same number of iterations.
Another natural question in this context is to compare the similarity metric of Proposition A.14 compared to that obtained by Harris et al. [48]—namely  $\min_{\boldsymbol{z} \in \mathcal{Z}} \sum_{h=1}^{H} \| \boldsymbol{z}^{(h,\star)} - \boldsymbol{z} \|_2^2$ , where we can assume here that  $z^{(h,\star)}$  is the unique Nash equilibrium of the  $h$ -th game. Below, we observe that  $\mathcal{V}_{\mathrm{NE}}^{(H)}$  can be arbitrarily smaller.
Proposition A.15. There exists a sequence of  $H$  games such that  $\mathcal{V}_{NE}^{(H)} = O(1)$  while  $\min_{\boldsymbol{z}\in \mathcal{Z}}\sum_{h = 1}^{H}\| \boldsymbol{z}^{(h,\star)} - \boldsymbol {z}\| _2^2 = \Omega (H)$ .
Proof. We let  $\mathcal{X},\mathcal{Y} = \Delta^3$ . We consider a sequence of  $H$  games so that  $\pmb{x}^{(h,\star)} = \pmb{y}^{(h,\star)}$  for all  $h\in \llbracket H\rrbracket$ , and the sequence of points  $(\pmb{x}^{(h,\star)})_{1\leq h\leq H}$  forms a regular  $H$ -gon within the relative interior of  $\Delta^3$  with a constant radius. We note that a unique fully-mixed equilibrium  $((\alpha ,\beta ,\gamma),(\alpha ,\beta ,\gamma))\in \Delta^3\times \Delta^3$  can be obtained by considering the diagonal payoff matrix
$$
\left[ \begin{array}{c c c} 1 / \alpha & 0 & 0 \\ 0 & 1 / \beta & 0 \\ 0 & 0 & 1 / \gamma \end{array} \right].
$$
It is easy to see that the above sequence of games establishes the claim.
A.1.7 Polymatrix zero-sum games
As we claimed earlier in Section 3.1.1, Theorem 3.3 can be extended to more general time-varying VIs as long as the MVI property holds. For concreteness, here we focus on a multi-player generalization of two-player zero-sum games, namely polymatrix zero-sum games [15]. More precisely, here it is assumed that there is an underlying  $n$ -node graph so that each player is uniquely associated with a node in the graph. The utility of each player  $i \in \mathbb{I}[n]$  can be expressed as  $\sum_{i' \in \mathcal{N}_i} x_i^\top \mathbf{A}_{i,i'} x_{i'}$ , where  $\mathcal{N}_i$  denotes the set of neighbors of Player  $i$  and  $\mathbf{A}_{i,i'} \in \mathbb{R}^{d_i \times d_{i'}}$ . It is also assumed that  $\mathbf{A}_{i,i'}^\top = -\mathbf{A}_{i',i}$ , for any edge  $\{i, i'\}$ . If  $z := (x_1, \ldots, x_n) \in \mathcal{Z}$ , the corresponding operator is defined as
$$
F (\boldsymbol {z}) = - \left(\sum_ {i \in \mathcal {N} _ {1}} \mathbf {A} _ {1, i} \boldsymbol {x} _ {i}, \dots , \sum_ {i \in \mathcal {N} _ {n}} \mathbf {A} _ {n, i} \boldsymbol {x} _ {i}\right).
$$
It is easy to see that  $F$  is monotone, in that  $\langle z - z', F(z) - F(z') \rangle \geq 0$ , for any  $z, z' \in \mathcal{Z}$ . Now, for any  $\epsilon$ -approximate Nash equilibrium  $z^{\star}$  it holds, by definition, that  $\langle z - z^{\star}, F(z^{\star}) \rangle \geq -n\epsilon$ , for any  $z \in \mathcal{Z}$ . Thus, by monotonicity,  $\langle z - z^{\star}, F(z) \rangle \geq \langle z - z^{\star}, F(z^{\star}) \rangle \geq -n\epsilon$ , for any  $z \in \mathcal{Z}$ . In other words, approximate Nash equilibria approximately satisfy the MVI property. This suffices to directly extend Property 3.1 to time-varying polymatrix zero-sum games. The rest of the argument of Theorem 3.3 is analogous, leading to the following result. Below, we define  $V_{\epsilon - \mathrm{NE}}^{(T)}$  as in (2), for a suitable parameter  $C > 0$ , and  $V_{\mathbf{A}}^{(T)} := \sum_{t=1}^{T-1} \sum_{i=1}^{n} \sum_{i' \in \mathcal{N}_{i'}} \| \mathbf{A}_{i,i'}^{(t+1)} - \mathbf{A}_{i,i'}^{(t)} \|_2^2$ .
Theorem A.16. Suppose that each player employs OG with a sufficiently small learning rate in a sequence of time-varying polymatrix zero-sum games. Then,  $\sum_{t=1}^{T} \left( \mathrm{EQGAP}^{(t)}(\boldsymbol{z}^{(t)}) \right)^2 = O\left(1 + \mathcal{V}_{\epsilon - NE}^{(T)} + \mathcal{V}_{\mathbf{A}}^{(T)} \right)$ , where  $(\boldsymbol{z}^{(t)})_{1 \leq t \leq T}$  is the sequence of joint strategies produced by OG.
A.1.8 General variational inequalities
Although our main focus in this paper is on the convergence of learning algorithms in time-varying games, our techniques could also be of interest for solving (static) general variational inequality (VI) problems.
In particular, let  $F: \mathcal{Z} \to \mathcal{Z}$  be a single-valued operator. Solving general VIs is well-known to be computationally intractable, and so instead focus has been on identifying broad subclasses that elude those intractability barriers (see the references below). Our framework in Section 3.1 motivates introducing the following measure of complexity for a VI problem:
$$
\mathfrak {C} (F) := \inf  _ {\boldsymbol {z} ^ {(1, \star)}, \dots , \boldsymbol {z} ^ {(T, \star)} \in \mathcal {Z}} \sum_ {t = 1} ^ {T - 1} \| \boldsymbol {z} ^ {(t + 1, \star)} - \boldsymbol {z} ^ {(t, \star)} \| _ {2}, \tag {39}
$$
$$
\mathrm {D R e g} _ {z} ^ {(T)} \left(\boldsymbol {z} ^ {(1, \star)}, \dots , \boldsymbol {z} ^ {(T, \star)}\right) \geq 0 \Longleftrightarrow \sum_ {t = 1} ^ {T} \langle \boldsymbol {z} ^ {(t)} - \boldsymbol {z} ^ {(t, \star)}, F (\boldsymbol {z} ^ {(t)}) \rangle \geq 0. \tag {40}
$$
In words, (39) expresses the infimum first-order variation that a sequence of comparators must have in order to guarantee nonnegative dynamic regret (40); it is evident that (40) always admits a feasible sequence, namely  $s_{z}^{(T)} \coloneqq (z^{(t)})_{1 \leq t \leq T}$ . We note that, in accordance to our results in Section 3.1, one can also consider an approximate version of the complexity measure (39), which could behave much more favorably (recall Proposition 3.2).
Now in a (static) bilinear saddle-point problem, it holds that  $\mathfrak{C}(F) = 0$  given that there exists a static comparator that guarantees nonnegativity of the dynamic regret. More broadly, our techniques imply  $O(\mathrm{poly}(1 / \epsilon))$  iteration-complexity bounds for any VI problem such that  $\mathfrak{C}(F)\leq CT^{1 - \omega}$ , for a time-independent parameter  $C > 0$  and  $\omega \in (0,1]$ :
Proposition A.17. Consider a variational inequality problem described with the operator  $F: \mathcal{Z} \to \mathcal{Z}$  such that  $F$  is L-Lipschitz continuous, in the sense that  $\| F(z) - F(z')\|_2 \leq L\| z - z'\|_2$ , and  $\mathfrak{C}(F) \leq CT^{1 - \omega}$  for  $C > 0$  and  $\omega \in (0,1]$ . Then, OGDP with learning rate  $\eta = \frac{1}{4L}$  reaches an  $\epsilon$ -strong solution  $z^{\star} \in \mathcal{Z}$  in  $O(\epsilon^{-2/\omega})$  iterations; that is,  $\langle z - z^{\star}, F(z^{\star}) \rangle \geq -\epsilon$  for any  $z \in \mathcal{Z}$ .
This result should be viewed as part of an ongoing effort to characterize the class of variational inequalities (VIs) that are amenable to efficient algorithms; see [33, 13, 4, 7, 21, 23, 56, 57, 66, 76, 87, 24], and the many references therein.
It is worth comparing (39) with another natural complexity measure, namely  $\inf_{\boldsymbol{z}^{\star}\in \mathcal{Z}}\sum_{t = 1}^{T}\langle \boldsymbol{z}^{(t)} - \boldsymbol{z}^{\star},F(\boldsymbol{z}^{(t)})\rangle$ ; the latter measures how negative (external) regret can be, and has already proven useful in certain settings that go bilinear saddle-point problems [88], although unlike (39) it does not appear to be of much use in characterizing time-varying bilinear saddle-point problems. In this context,  $O(\mathrm{poly}(1 / \epsilon))$  iteration-complexity bounds can also be established whenever
-  $\inf_{\pmb{z}^{\star}\in \mathcal{Z}}\sum_{t = 1}^{T}\langle \pmb{z}^{(t)} - \pmb{z}^{\star},F(\pmb{z}^{(t)})\rangle \geq -CT^{1 - \omega}$  for a time-invariant  $C > 0$  , or
-  $\inf_{\boldsymbol{z}^{\star}\in \mathcal{Z}}\sum_{t = 1}^{T}\langle \boldsymbol{z}^{(t)} - \boldsymbol{z}^{\star},F(\boldsymbol{z}^{(t)})\rangle \geq -C\sum_{t = 1}^{T - 1}\| \boldsymbol{z}^{(t + 1)} - \boldsymbol{z}^{(t)}\| _2^2,$  for a sufficiently small  $C > 0$
Following [88], identifying VIs that satisfy those relaxed conditions but not the MVI property is an interesting direction. In particular, it is important to understand if those relaxations can shed more light into the convergence properties of OG in Shapley's two-player zero-sum stochastic games.
A.2 Proofs from Section 3.2
In this subsection, we provide the proofs from Section 3.2, leading to our main result in Theorem 3.5.
Let us first introduce some additional notation. We let  $f^{(t)}: \mathcal{X} \times \mathcal{Y} \to \mathbb{R}$  be a continuously differentiable function for any  $t \in [[T]]$ . We recall that in Section 3.2 it is assumed that the objective
In case the interior  $(\mathcal{X}\times \mathcal{Y})^{\circ}$  is empty, here and throughout this paper we tacitly posit differentiability on a closed and convex neighborhood  $\tilde{\mathcal{X}}\times \tilde{\mathcal{Y}}$  such that  $\mathcal{X}\subseteq \tilde{\mathcal{X}}^{\circ}$  and  $\mathcal{V}\subseteq \tilde{\mathcal{V}}^{\circ}$ . This assumption suffices for our arguments; see, for example, the treatment of Daskalakis et al. [28].
function changes only after  $m \in \mathbb{N}$  (consecutive) repetitions, which is akin to the meta-learning setting. Analogously to our setup for bilinear saddle-point problems (Section 3.1), it is assumed that Player  $x$  is endeavoring to minimizing the objective function, while Player  $y$  is trying to maximize it. We will denote by  $\mathrm{Reg}_{\mathcal{L},x}^{(T)}(\pmb{x}^{\star}) \coloneqq \sum_{t=1}^{T} \langle \pmb{x}^{(t)} - \pmb{x}^{\star}, -\pmb{u}_x^{(t)} \rangle$  and  $\mathrm{Reg}_{\mathcal{L},y}^{(T)}(\pmb{y}^{\star}) \coloneqq \sum_{t=1}^{T} \langle \pmb{y}^{\star} - \pmb{y}^{(t)}, \pmb{u}_y^{(t)} \rangle$ , where  $\pmb{u}_x^{(t)} \coloneqq -\nabla_x f(\pmb{x}^{(t)}, \pmb{y}^{(t)})$  and  $\pmb{u}_y^{(t)} \coloneqq \nabla_y f(\pmb{x}^{(t)}, \pmb{y}^{(t)})$  for any  $t \in [T]$ ; similar notation is used for  $\mathrm{DReg}_{\mathcal{L},x}^{(T)}$ ,  $\mathrm{DReg}_{\mathcal{L},y}^{(T)}$ .
Furthermore, we let  $s_z^{(T)} = ((\pmb{x}^{(t,\star)}, \pmb{y}^{(t,\star)}))_{1 \leq t \leq T}$ , so that  $\pmb{x}^{(t,\star)} = \pmb{x}^{(h,\star)}$  and  $\pmb{y}^{(t,\star)} = \pmb{y}^{(h,\star)}$  for any  $t \in [T]$  such that  $\lfloor (t - 1) / m \rfloor = h \in [H]$ . The first important step in our analysis is that, following the proof of Lemma A.1,
$$
\begin{array}{l} \mathrm {D R e g} _ {\mathcal {L}, x} ^ {(T)} \left(s _ {x} ^ {(T)}\right) \leq \frac {1}{2 \eta} \sum_ {h = 1} ^ {H} \left\| \hat {\boldsymbol {x}} ^ {(h, 1)} - \boldsymbol {x} ^ {(h, \star)} \right\| _ {2} ^ {2} - \left\| \hat {\boldsymbol {x}} ^ {(h, m + 1)} - \boldsymbol {x} ^ {(h, \star)} \right\| _ {2} ^ {2} + \eta \sum_ {t = 1} ^ {T} \left\| \boldsymbol {u} _ {x} ^ {(t)} - \boldsymbol {m} _ {x} ^ {(t)} \right\| _ {2} ^ {2} \\ - \frac {1}{2 \eta} \sum_ {t = 1} ^ {T} \left(\| \boldsymbol {x} ^ {(t)} - \hat {\boldsymbol {x}} ^ {(t)} \| _ {2} ^ {2} + \| \boldsymbol {x} ^ {(t)} - \hat {\boldsymbol {x}} ^ {(t + 1)} \| _ {2} ^ {2}\right), \tag {41} \\ \end{array}
$$
where  $\hat{\pmb{x}}^{(h,k)}\coloneqq \hat{\pmb{x}}^{((h - 1)\times m) + k)}$  for any  $(h,k)\in [[H]]\times [[m]]$ $\hat{\pmb{x}}^{(h,m + 1)}\coloneqq \hat{\pmb{x}}^{(h + 1,1)}$  for  $h\in [[H - 1]]$  and  $\hat{\pmb{x}}^{(H,m + 1)}\coloneqq \hat{\pmb{x}}^{(T + 1)}$ . Similarly,
$$
\begin{array}{l} \mathrm {D R e g} _ {\mathcal {L}, y} ^ {(T)} \left(s _ {y} ^ {(T)}\right) \leq \frac {1}{2 \eta} \sum_ {h = 1} ^ {H} \left\| \hat {\boldsymbol {y}} ^ {(h, 1)} - \boldsymbol {y} ^ {(h, \star)} \right\| _ {2} ^ {2} - \left\| \hat {\boldsymbol {y}} ^ {(h, m + 1)} - \boldsymbol {y} ^ {(h, \star)} \right\| _ {2} ^ {2} + \eta \sum_ {t = 1} ^ {T} \left\| \boldsymbol {u} _ {y} ^ {(t)} - \boldsymbol {m} _ {y} ^ {(t)} \right\| _ {2} ^ {2} \\ - \frac {1}{2 \eta} \sum_ {t = 1} ^ {T} \left(\| \boldsymbol {y} ^ {(t)} - \hat {\boldsymbol {y}} ^ {(t)} \| _ {2} ^ {2} + \| \boldsymbol {y} ^ {(t)} - \hat {\boldsymbol {y}} ^ {(t + 1)} \| _ {2} ^ {2}\right). \tag {42} \\ \end{array}
$$
Next, we will use the following key observation, which lower bounds the sum of the players' (external) regrets under strong convexity-concavity.
Lemma A.18. Suppose that  $f:\mathcal{X}\times \mathcal{Y}\to \mathbb{R}$  is a  $\mu$ -strongly convex-concave function with respect to  $\| \cdot \| _2$ . Then, for any Nash equilibrium  $\pmb {z}^{\star} = (\pmb{x}^{\star},\pmb{y}^{\star})\in \mathcal{Z}$ ,
$$
\operatorname {R e g} _ {\mathcal {L}, x} ^ {(m)} \left(\boldsymbol {x} ^ {\star}\right) + \operatorname {R e g} _ {\mathcal {L}, y} ^ {(m)} \left(\boldsymbol {y} ^ {\star}\right) \geq \frac {\mu}{2} \sum_ {t = 1} ^ {m} \| \boldsymbol {z} ^ {(t)} - \boldsymbol {z} ^ {\star} \| _ {2} ^ {2}.
$$
Proof. First, by  $\mu$ -strong convexity of  $f(x,\cdot)$ , we have that for any time  $t\in [[m]]$
$$
\langle \boldsymbol {x} ^ {(t)} - \boldsymbol {x} ^ {\star}, \nabla_ {\boldsymbol {x}} f (\boldsymbol {x} ^ {(t)}, \boldsymbol {y} ^ {(t)}) \rangle \geq f (\boldsymbol {x} ^ {(t)}, \boldsymbol {y} ^ {(t)}) - f (\boldsymbol {x} ^ {\star}, \boldsymbol {y} ^ {(t)}) + \frac {\mu}{2} \| \boldsymbol {x} ^ {(t)} - \boldsymbol {x} ^ {\star} \| _ {2} ^ {2}. \tag {43}
$$
Similarly, by  $\mu$ -strong concavity of  $f(\cdot, \pmb{y})$ , we have that for any time  $t \in [[m]]$ ,
$$
\langle \boldsymbol {y} ^ {\star} - \boldsymbol {y} ^ {(t)}, \nabla_ {\boldsymbol {y}} f (\boldsymbol {x} ^ {(t)}, \boldsymbol {y} ^ {(t)}) \rangle \geq f (\boldsymbol {x} ^ {(t)}, \boldsymbol {y} ^ {\star}) - f (\boldsymbol {x} ^ {(t)}, \boldsymbol {y} ^ {(t)}) + \frac {\mu}{2} \| \boldsymbol {y} ^ {(t)} - \boldsymbol {y} ^ {\star} \| _ {2} ^ {2}. \tag {44}
$$
Further, for any Nash equilibrium  $(\pmb{x}^{\star},\pmb{y}^{\star})\in \mathcal{Z}$  it holds that  $f(\pmb{x}^{(t)},\pmb{y}^{\star})\geq f(\pmb{x}^{(t)},\pmb{y}^{(t)})\geq$ $f(\pmb{x}^{\star},\pmb{y}^{(t)})$ . Combining this fact with (43) and (44) and summing over all  $t\in \llbracket m\rrbracket$  gives the statement.
In turn, this readily implies the following lower bound for the dynamic regret.
Lemma A.19. Suppose that  $f^{(h)}: \mathcal{X} \times \mathcal{Y} \to \mathbb{R}$  is a  $\mu$ -strongly convex-concave function with respect to  $\| \cdot \|_2$ , for any  $h \in [[H]]$ . Consider a sequence  $s_z^{(T)} = ((\pmb{x}^{(t,\star)}, \pmb{y}^{(t,\star)})_{1 \leq t \leq T})$ , so that  $\pmb{x}^{(t,\star)} = \pmb{x}^{(h,\star)}$  and  $\pmb{y}^{(t,\star)} = \pmb{y}^{(h,\star)}$  for any  $t \in [[T]]$  such that  $\lfloor (t - 1)/m \rfloor = h \in [[H]]$ . If  $(\pmb{x}^{(h,\star)}, \pmb{y}^{(h,\star)}) \in \mathcal{Z}$  is a Nash equilibrium of  $f^{(h)}$ ,
$$
\mathrm {D R e g} _ {\mathcal {L}, x} ^ {(T)} \left(s _ {x} ^ {(T)}\right) + \mathrm {D R e g} _ {\mathcal {L}, y} ^ {(T)} \left(s _ {y} ^ {(T)}\right) \geq \frac {\mu}{2} \sum_ {h = 1} ^ {H} \sum_ {k = 1} ^ {m} \left\| \boldsymbol {z} ^ {(h, k)} - \boldsymbol {z} ^ {(h, \star)} \right\| _ {2} ^ {2},
$$
where  $\pmb{z}^{(h,k)}\coloneqq \pmb{z}^{((h - 1)\times m) + k)}$  for any  $(h,k)\in [[H]]\times [[m]]$
We next combine this with the following monotonicity property of OGD: If  $z^{\star}$  is a Nash equilibrium,  $\| \hat{z}^{(t)} - z^{\star}\|_{2}$  is a decreasing function in  $t$  [48, Proposition C.10]. This leads to the following refinement of Lemma A.19.
Lemma A.20. Under the assumptions of Lemma A.19, if additionally  $\eta \leq \frac{1}{2\mu}$ ,
$$
\mathrm {D R e g} _ {\mathcal {L}, x} ^ {(T)} (s _ {x} ^ {(T)}) + \mathrm {D R e g} _ {\mathcal {L}, y} ^ {(T)} (s _ {y} ^ {(T)}) + \frac {1}{4 \eta} \sum_ {t = 1} ^ {T} \| \pmb {z} ^ {(t)} - \hat {\pmb {z}} ^ {(t + 1)} \| _ {2} ^ {2} \geq \frac {\mu m}{4} \sum_ {h = 1} ^ {H} \| \hat {\pmb {z}} ^ {(h, m + 1)} - \pmb {z} ^ {(h, \star)} \| _ {2} ^ {2}.
$$
Proof. By Lemma A.19,
$$
\begin{array}{l} \mathrm {D R e g} _ {\mathcal {L}, x} ^ {(T)} (s _ {x} ^ {(T)}) + \mathrm {D R e g} _ {\mathcal {L}, y} ^ {(T)} (s _ {y} ^ {(T)}) + \frac {1}{4 \eta} \sum_ {t = 1} ^ {T} \| \boldsymbol {z} ^ {(t)} - \hat {\boldsymbol {z}} ^ {(t + 1)} \| _ {2} ^ {2} \\ \geq \frac {\mu}{2} \sum_ {h = 1} ^ {H} \sum_ {k = 1} ^ {m} \| \boldsymbol {z} ^ {(h, k)} - \boldsymbol {z} ^ {(h, \star)} \| _ {2} ^ {2} + \frac {1}{4 \eta} \sum_ {t = 1} ^ {T} \| \boldsymbol {z} ^ {(t)} - \hat {\boldsymbol {z}} ^ {(t + 1)} \| _ {2} ^ {2} \\ \geq \frac {\mu}{4} \sum_ {h = 1} ^ {H} \sum_ {k = 1} ^ {m} \| \hat {\boldsymbol {z}} ^ {(h, k + 1)} - \boldsymbol {z} ^ {(h, \star)} \| _ {2} ^ {2} (45) \\ \geq \frac {\mu m}{4} \sum_ {h = 1} ^ {H} \| \hat {z} ^ {(h, m + 1)} - z ^ {(h, \star)} \| _ {2} ^ {2}, (46) \\ \end{array}
$$
where (45) uses that  $\frac{1}{4\eta} \geq \frac{\mu}{2}$  along with Young's inequality and triangle inequality, and (46) follows from [48, Proposition C.10].
Armed with this important lemma, we are ready to establish our main result (Theorem 3.5), the detailed version of which is given below. We first recall that the function  $f: \mathcal{X} \times \mathcal{Y} \to \mathbb{R}$  is said to be  $L$ -smooth if  $\| F(z) - F(z') \|_2 \leq L \| z - z' \|_2$ , where  $F(z) := (\nabla_x f(x, y), -\nabla_y f(x, y))$ .
Theorem A.21 (Detailed version of Theorem 3.5). Let  $f^{(h)}: \mathcal{X} \times \mathcal{Y}$  be a  $\mu$ -strongly convex-concave and  $L$ -smooth function, for  $h \in [[H]]$ . Suppose that both players employ OGDP with learning rate  $\eta = \min \left\{\frac{1}{8L}, \frac{1}{2\mu}\right\}$  for  $T$  repetitions, where  $T = m \times H$  and  $m \geq \frac{2}{\eta\mu}$ . Then,
$$
\sum_ {t = 1} ^ {T} \left(\| \pmb {z} ^ {(t)} - \hat {\pmb {z}} ^ {(t)} \| _ {2} ^ {2} + \| \pmb {z} ^ {(t)} - \hat {\pmb {z}} ^ {(t + 1)} \| _ {2} ^ {2}\right) \leq 4 D _ {\mathcal {Z}} ^ {2} + 8 \eta^ {2} \| F (\pmb {z} ^ {(1)}) \| _ {2} ^ {2} + 8 \mathcal {S} _ {N E} ^ {(H)} + 1 6 \eta^ {2} \mathcal {V} _ {\nabla f} ^ {(H)}.
$$
Thus,  $\sum_{t=1}^{T}\left(\operatorname{EQGAP}^{(t)}(\boldsymbol{z}^{(t)})\right)^2 = O(1 + \mathcal{S}_{NE}^{(H)} + \mathcal{V}_{\nabla f}^{(H)}).$
Proof. Combining Lemma A.20 with (41) and (42),
$$
\begin{array}{l} 0 \leq \frac {1}{2 \eta} \sum_ {h = 1} ^ {H} \left(\| \hat {\boldsymbol {z}} ^ {(h, 1)} - \boldsymbol {z} ^ {(h, \star)} \| _ {2} ^ {2} - \frac {2 + \eta \mu m}{2} \| \hat {\boldsymbol {z}} ^ {(h, m + 1)} - \boldsymbol {z} ^ {(h, \star)} \| _ {2} ^ {2}\right) \\ + \eta \sum_ {t = 1} ^ {T} \| \boldsymbol {u} _ {z} ^ {(t)} - \boldsymbol {m} _ {z} ^ {(t)} \| _ {2} ^ {2} - \frac {1}{4 \eta} \sum_ {t = 1} ^ {T} \left(\| \boldsymbol {z} ^ {(t)} - \hat {\boldsymbol {z}} ^ {(t)} \| _ {2} ^ {2} + \| \boldsymbol {z} ^ {(t)} - \hat {\boldsymbol {z}} ^ {(t + 1)} \| _ {2} ^ {2}\right), \\ \end{array}
$$
for a sequence of Nash equilibria  $(\pmb{z}^{(h,\star)})_{1\leq h\leq H}$ , where we used the notation  $\pmb{u}_z^{(t)} := (\pmb{u}_x^{(t)},\pmb{u}_y^{(t)})$  and  $\pmb{m}_z^{(t)} := (\pmb{m}_x^{(t)},\pmb{m}_y^{(t)})$ . Now we bound the first term of the right-hand side above as
$$
\begin{array}{l} \frac {1}{2 \eta} \sum_ {h = 1} ^ {H} \left(\| \hat {\boldsymbol {z}} ^ {(h, 1)} - \boldsymbol {z} ^ {(h, \star)} \| _ {2} ^ {2} - 2 \| \hat {\boldsymbol {z}} ^ {(h, m + 1)} - \boldsymbol {z} ^ {(h, \star)} \| _ {2} ^ {2}\right) \leq \\ \frac {1}{2 \eta} \| \hat {\boldsymbol {z}} ^ {(1, 1)} - \boldsymbol {z} ^ {(1, \star)} \| _ {2} ^ {2} + \frac {1}{2 \eta} \sum_ {h = 1} ^ {H - 1} \left(\| \hat {\boldsymbol {z}} ^ {(h + 1, 1)} - \boldsymbol {z} ^ {(h + 1, \star)} \| _ {2} ^ {2} - 2 \| \hat {\boldsymbol {z}} ^ {(h + 1, 1)} - \boldsymbol {z} ^ {(h, \star)} \| _ {2} ^ {2}\right), \\ \end{array}
$$
where we used the fact that  $m \geq \frac{2}{\eta \mu}$  and  $\hat{\pmb{z}}^{(h,m + 1)} = \hat{\pmb{z}}^{(h + 1,1)}$ , for  $h \in [[H - 1]]$ . Hence, continuing from above,
$$
\begin{array}{l} \frac {1}{2 \eta} \sum_ {h = 1} ^ {H} \left(\| \hat {\boldsymbol {z}} ^ {(h, 1)} - \boldsymbol {z} ^ {(h, \star)} \| _ {2} ^ {2} - 2 \| \hat {\boldsymbol {z}} ^ {(h, m + 1)} - \boldsymbol {z} ^ {(h, \star)} \| _ {2} ^ {2}\right) \leq \frac {1}{2 \eta} \| \hat {\boldsymbol {z}} ^ {(1, 1)} - \boldsymbol {z} ^ {(1, \star)} \| _ {2} ^ {2} \\ + \frac {1}{\eta} \sum_ {h = 1} ^ {H - 1} \| z ^ {(h + 1, \star)} - z ^ {(h, \star)} \| _ {2} ^ {2}, \\ \end{array}
$$