SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Let also  $\pmb{x}^{(t,\star)}\in \arg \min_{\pmb{x}\in \mathcal{X}}\max_{\pmb{y}\in \mathcal{Y}}f^{(t)}(\pmb {x},\pmb {y})$  and  $\pmb{y}^{(t,\star)}\in \arg \max_{\pmb {y}\in \mathcal{Y}}\min_{\pmb {x}\in \mathcal{X}}f^{(t)}(\pmb {x},\pmb {y}),$  for any  $t\in \llbracket T\rrbracket$  . Then, for  $s_x^{(T)} = (\pmb{x}^{(t,\star)})_{1\leq t\leq T}$  and  $s_y^{(T)} = (\pmb{y}^{(t,\star)})_{1\leq t\leq T}$
$$
\mathrm {D R e g} _ {x} ^ {(T)} \left(s _ {x} ^ {(T)}\right) + \mathrm {D R e g} _ {y} ^ {(T)} \left(s _ {y} ^ {(T)}\right) \geq 0.
$$
Proof. By definition of dynamic regret (16), it suffices to show that  $f^{(t)}(\pmb{x}^{(t)}, \pmb{y}^{(t,\star)}) \geq f^{(t)}(\pmb{x}^{(t,\star)}, \pmb{y}^{(t)})$ , for any time  $t \in [T]$ . Indeed,
$$
\begin{array}{l} f ^ {(t)} \left(\boldsymbol {x} ^ {(t)}, \boldsymbol {y} ^ {(t, \star)}\right) \geq \min  _ {\boldsymbol {x} \in \mathcal {X}} f ^ {(t)} \left(\boldsymbol {x}, \boldsymbol {y} ^ {(t, \star)}\right) (17) \\ = \max  _ {\boldsymbol {y} \in \mathcal {Y}} \min  _ {\boldsymbol {x} \in \mathcal {X}} f ^ {(t)} (\boldsymbol {x}, \boldsymbol {y}) (18) \\ = \min  _ {\boldsymbol {x} \in \mathcal {X}} \max  _ {\boldsymbol {y} \in \mathcal {Y}} f ^ {(t)} (\boldsymbol {x}, \boldsymbol {y}) (19) \\ = \max  _ {\boldsymbol {y} \in \mathcal {Y}} f ^ {(t)} \left(\boldsymbol {x} ^ {(t, \star)}, \boldsymbol {y}\right) (20) \\ \geq f ^ {(t)} \left(\boldsymbol {x} ^ {(t, \star)}, \boldsymbol {y} ^ {(t)}\right), (21) \\ \end{array}
$$
where (17) and (21) are obvious; (18) and (20) follow from the definition of  $\pmb{y}^{(t,\star)}\in \mathcal{V}$  and  $\pmb{x}^{(t,\star)}\in \mathcal{X}$ , respectively; and (19) holds by assumption. This concludes the proof.
A.1.3 Variation of the Nash equilibria
In our next example, we point out the standard observation that an arbitrarily small change in the entries of the payoff matrix can lead to a substantial deviation in the Nash equilibrium.
Example A.5. Consider a  $2 \times 2$  (two-player) zero-sum game, where  $\mathcal{X} \coloneqq \Delta^2$ ,  $\mathcal{Y} \coloneqq \Delta^2$ , described by the payoff matrix
$$
\mathbf {A} := \left[ \begin{array}{l l} 2 \delta & 0 \\ 0 & \delta \end{array} \right], \tag {22}
$$
for some  $\delta > 0$ . Then, it is easy to see that the unique Nash equilibrium of this game is such that  $\pmb{x}^{\star}, \pmb{y}^{\star} \coloneqq \left(\frac{1}{3}, \frac{2}{3}\right) \in \Delta^{2}$ . Suppose now that the original payoff matrix (22) is perturbed to a new matrix
$$
\mathbf {A} ^ {\prime} := \left[ \begin{array}{c c} \delta & 0 \\ 0 & 2 \delta \end{array} \right]. \tag {23}
$$
The new (unique) Nash equilibrium now reads  $\boldsymbol{x}^{\star}, \boldsymbol{y}^{\star} \coloneqq \left(\frac{2}{3}, \frac{1}{3}\right) \in \Delta^{2}$ . We conclude that an arbitrarily small deviation in the entries of the payoff matrix can lead to a non-trivial change in the Nash equilibrium.
Next, we leverage the simple observation of the example above to establish Proposition 3.2, the statement of which is recalled below.
Proof. We consider a sequence of  $T \geq 4$  games such that  $\mathcal{X}, \mathcal{Y} \coloneqq \Delta^2$  and
$$
\mathbf {A} ^ {(t)} = \left\{ \begin{array}{l l} \mathbf {A} & \text {i f t \mod 2 = 1}, \\ \mathbf {A} ^ {\prime} & \text {i f t \mod 2 = 0}, \end{array} \right.
$$
where  $\mathbf{A},\mathbf{A}^{\prime}$  are the payoff matrices defined in (22) and (23), and are parameterized by  $\delta >0$  (Example A.5). Then, the exact Nash equilibria read
$$
\boldsymbol{x}^{(t,\star)},\boldsymbol{y}^{(t,\star)} = \left\{ \begin{array}{ll}\bigl(\frac{1}{3},\frac{2}{3}\bigr) & \text{\rm if $t$ mod 2 = 1},\\ \bigl(\frac{2}{3},\frac{1}{3}\bigr) & \text{\rm if $t$ mod 2 = 0}. \end{array} \right.
$$
As a result, it follows that  $\mathcal{V}_{\mathrm{NE}}^{(T)} := \sum_{t=1}^{T-1} \|z^{(t+1,\star)} - z^{(t,\star)}\|_2 = \frac{2}{3}(T-1) \geq \frac{T}{2}$ , for  $T \geq 4$ . In contrast, it is clear that  $\mathcal{V}_{\epsilon-\mathrm{NE}}^{(T)} \leq C \delta T$ , which follows by simply considering the sequence of strategies wherein both players are always selecting actions uniformly at random; we recall that  $C > 0$  here is the value that parameterizes  $\mathcal{V}_{\epsilon-\mathrm{NE}}^{(T)}$ . Thus, taking  $\delta := \frac{\delta'}{CT}$ , for some arbitrarily small  $\delta' > 0$ , concludes the proof.
In the above sequence of games, the variation measure  $\mathcal{W}_{\mathbf{A}}^{(T)}$  is in fact also arbitrarily small as  $\delta \rightarrow 0$ ; this reflects the fact that, although the exact Nash equilibria change abruptly, the sequence of payoff matrices exhibits small variation. Nevertheless, we next point out a modified sequence of games where  $\nu_{\epsilon -\mathrm{NE}}$  remains small even though both  $\mathcal{W}_{\mathbf{A}}^{(T)}$  and  $\nu_{\mathrm{NE}}^{(T)}$  can be unbounded.
Proposition A.6. For any  $T \geq 4$ , there is a sequence of games such that  $\mathcal{V}_{NE}^{(T)}, \mathcal{W}_{\mathbf{A}}^{(T)} = \Omega(T)$  while  $\mathcal{V}_{\epsilon - NE}^{(T)} \leq \delta$ , for any  $\delta > 0$ .
$$
\mathbf {A} ^ {(t)} = \left\{ \begin{array}{l l} \mathbf {A} + \mathbf {1} _ {2 \times 2} & \text {i f t \mod 2 = 1}, \\ \mathbf {A} ^ {\prime} & \text {i f t \mod 2 = 0}, \end{array} \right.
$$
where  $\mathbf{A}$ ,  $\mathbf{A}'$  are the payoff matrices defined in (22) and (23), and  $\mathbf{1}_{2\times 2}$  denotes the all-ones  $2\times 2$  matrix. Given that  $\boldsymbol{x}^{\top}(\mathbf{A} + \mathbf{1}_{2\times 2})\boldsymbol{y} = \boldsymbol{x}^{\top}\mathbf{A}\boldsymbol{y} + 1$ , for any  $\boldsymbol{x},\boldsymbol{y}\in \Delta^{2}$ , it follows that the set of  $\epsilon$ -approximate Nash equilibria of each game coincides with the set of  $\epsilon$ -approximate Nash equilibria of the corresponding game in Proposition 3.2. Thus, by the same argument as in Proposition 3.2 earlier, we have that  $\mathcal{V}_{\mathrm{NE}}^{(T)}\geq \frac{T}{2}$  and  $\mathcal{V}_{\epsilon -\mathrm{NE}}^{(T)}\leq C\delta T$ . It is also clear that  $\mathcal{W}_{\mathbf{A}}^{(T)} = \Omega (T)$ , concluding the proof.
In words, the phenomenon identified above occurs because  $\mathcal{W}_{\mathbf{A}}^{(T)}$  is affected by perturbations in the payoff matrices that do not strategically alter the game—namely, the addition of the matrix  $\mathbf{1}_{2\times 2}$ .
A.1.4 Proof of Theorem 3.3
Next, we proceed with the proof of one of our main results, namely Theorem 3.3. The key ingredient is Theorem A.7 below, which bounds the second-order path length of OGD in terms of the considered variation measures.
Theorem A.7. Suppose that both players employ OGDP with learning rate  $\eta \leq \frac{1}{4L}$  in a time-varying bilinear saddle-point problem, where  $L \coloneqq \max_{1 \leq t \leq T} \| \mathbf{A}^{(t)}\|_2$ . Then, for any time horizon  $T \in \mathbb{N}$ ,
$$
\sum_ {t = 1} ^ {T} \left(\| \boldsymbol {z} ^ {(t)} - \hat {\boldsymbol {z}} ^ {(t)} \| _ {2} ^ {2} + \| \boldsymbol {z} ^ {(t)} - \hat {\boldsymbol {z}} ^ {(t + 1)} \| _ {2} ^ {2}\right) \leq 2 D _ {\mathcal {Z}} ^ {2} + 4 \eta^ {2} L ^ {2} \| \mathcal {Z} \| _ {2} ^ {2} + 4 D _ {\mathcal {Z}} \mathcal {V} _ {\epsilon - N E} ^ {(T)} + 8 \eta^ {2} \| \mathcal {Z} \| _ {2} ^ {2} \mathcal {V} _ {\mathbf {A}} ^ {(T)}.
$$
Proof of Theorem A.7. First, for any  $t \geq 2$  we have that  $\| \pmb{u}_x^{(t)} - \pmb{m}_x^{(t)} \|_2^2$  is equal to
$$
\begin{array}{l} \left\| \mathbf {A} ^ {(t)} \boldsymbol {y} ^ {(t)} - \mathbf {A} ^ {(t - 1)} \boldsymbol {y} ^ {(t - 1)} \right\| _ {2} ^ {2} \leq 2 \left\| \mathbf {A} ^ {(t)} \left(\boldsymbol {y} ^ {(t)} - \boldsymbol {y} ^ {(t - 1)}\right) \right\| _ {2} ^ {2} + 2 \left\| \left(\mathbf {A} ^ {(t)} - \mathbf {A} ^ {(t - 1)}\right) \boldsymbol {y} ^ {(t - 1)} \right\| _ {2} ^ {2} (24) \\ \leq 2 \| \mathbf {A} ^ {(t)} \| _ {2} ^ {2} \| \boldsymbol {y} ^ {(t)} - \boldsymbol {y} ^ {(t - 1)} \| _ {2} ^ {2} + 2 \| \mathbf {A} ^ {(t)} - \mathbf {A} ^ {(t - 1)} \| _ {2} ^ {2} \| \boldsymbol {y} ^ {(t - 1)} \| _ {2} ^ {2} (25) \\ \leq 2 L ^ {2} \| \boldsymbol {y} ^ {(t)} - \boldsymbol {y} ^ {(t - 1)} \| _ {2} ^ {2} + 2 \| \mathcal {Y} \| _ {2} ^ {2} \| \mathbf {A} ^ {(t)} - \mathbf {A} ^ {(t - 1)} \| _ {2} ^ {2}, (26) \\ \end{array}
$$
where (24) uses the triangle inequality for the norm  $\| \cdot \| _2$  along with the inequality  $2ab\leq a^{2} + b^{2}$  for any  $a,b\in \mathbb{R}$ ; (25) follows from the definition of the operator norm; and (26) uses the assumption that  $\| \mathbf{A}^{(t)}\|_2\leq L$  and  $\| \pmb {y}\| _2\leq \| \mathcal{V}\| _2$  for any  $\pmb {y}\in \mathcal{V}$ . A similar derivation shows that for  $t\geq 2$
$$
\left\| \boldsymbol {u} _ {y} ^ {(t)} - \boldsymbol {m} _ {y} ^ {(t)} \right\| _ {2} ^ {2} \leq 2 L ^ {2} \left\| \boldsymbol {x} ^ {(t)} - \boldsymbol {x} ^ {(t - 1)} \right\| _ {2} ^ {2} + 2 \| \mathcal {X} \| _ {2} ^ {2} \| \mathbf {A} ^ {(t)} - \mathbf {A} ^ {(t - 1)} \| _ {2} ^ {2}. \tag {27}
$$
Further, for  $t = 1$  we have that  $\| \pmb{u}_x^{(1)} - \pmb{m}_x^{(1)}\|_2 = \| \pmb{u}_x^{(1)}\|_2 = \| -\mathbf{A}^{(1)}\pmb{y}^{(1)}\|_2 \leq L\|\mathcal{Y}\|_2$ , and  $\| \pmb{u}_y^{(1)} - \pmb{m}_y^{(1)}\|_2 = \| \pmb{u}_y^{(1)}\|_2 = (\mathbf{A}^{(1)})^\top \pmb{x}^{(1)}\|_2 \leq L\|\mathcal{X}\|_2$ . Next, we will use the following simple corollary, which follows similarly to Lemma A.1.
Corollary A.8. For any sequence  $s_z^{(T)} \coloneqq (z^{(t,\star)})_{1\leq t\leq T}$ , the dynamic regret  $\mathrm{DReg}_z^{(T)}(s_z^{(T)}) \coloneqq \mathrm{DReg}_x^{(T)}(s_x^{(T)}) + \mathrm{DReg}_y^{(T)}(s_y^{(T)})$  can be bounded by
$$
\begin{array}{l} \frac {D _ {\mathcal {Z}} ^ {2}}{2 \eta} + \frac {D _ {\mathcal {Z}}}{\eta} \sum_ {t = 1} ^ {T - 1} \| \boldsymbol {z} ^ {(t + 1, \star)} - \boldsymbol {z} ^ {(t, \star)} \| _ {2} + \eta \sum_ {t = 1} ^ {T} \| \boldsymbol {u} _ {\tilde {z}} ^ {(t)} - \boldsymbol {m} _ {\tilde {z}} ^ {(t)} \| _ {2} ^ {2} \\ - \frac {1}{2 \eta} \sum_ {t = 1} ^ {T} \left(\| \pmb {z} ^ {(t)} - \hat {\pmb {z}} ^ {(t)} \| _ {2} ^ {2} + \| \pmb {z} ^ {(t)} - \hat {\pmb {z}} ^ {(t + 1)} \| _ {2} ^ {2}\right), \\ \end{array}
$$
where  $\pmb{m}_{z}^{(t)}\coloneqq (\pmb{m}_{x}^{(t)},\pmb{m}_{y}^{(t)})$  and  $\pmb{u}_{z}^{(t)}\coloneqq (\pmb{u}_{x}^{(t)},\pmb{u}_{y}^{(t)})$  for any  $t\in [[T]]$
As a result, combining (27) and (26) with Corollary A.8 applied for the dynamic regret of both players with respect to the sequence of comparators  $((\pmb{x}^{(t,\star)},\pmb{y}^{(t,\star)}))_{1\leq t\leq T}$  yields that  $\mathrm{DReg}_x^{(T)}(\pmb{x}^{(1,\star)},\dots ,\pmb{x}^{(T,\star)}) + \mathrm{DReg}_y^{(T)}(\pmb{y}^{(1,\star)},\dots ,\pmb{y}^{(T,\star)})$  is upper bounded by
$$
\begin{array}{l} \frac {D _ {\mathcal {Z}} ^ {2}}{2 \eta} + \eta L ^ {2} \| \mathcal {Z} \| _ {2} ^ {2} + \frac {D _ {\mathcal {Z}}}{\eta} \sum_ {t = 1} ^ {T - 1} \| \boldsymbol {z} ^ {(t + 1, \star)} - \boldsymbol {z} ^ {(t, \star)} \| _ {2} + 2 \eta \| \mathcal {Z} \| _ {2} ^ {2} \mathcal {V} _ {\mathbf {A}} ^ {(T)} \\ - \frac {1}{4 \eta} \sum_ {t = 1} ^ {T} \left(\| \boldsymbol {z} ^ {(t)} - \hat {\boldsymbol {z}} ^ {(t)} \| _ {2} ^ {2} + \| \boldsymbol {z} ^ {(t)} - \hat {\boldsymbol {z}} ^ {(t + 1)} \| _ {2} ^ {2}\right), \\ \end{array}
$$
$$
\begin{array}{l} 2 \eta L ^ {2} \sum_ {t = 2} ^ {T} \| \boldsymbol {z} ^ {(t)} - \boldsymbol {z} ^ {(t - 1)} \| _ {2} ^ {2} - \frac {1}{4 \eta} \sum_ {t = 1} ^ {T} \left(\| \boldsymbol {z} ^ {(t)} - \hat {\boldsymbol {z}} ^ {(t)} \| _ {2} ^ {2} + \| \boldsymbol {z} ^ {(t)} - \hat {\boldsymbol {z}} ^ {(t + 1)} \| _ {2} ^ {2}\right) \\ \leq \left(2 \eta L ^ {2} - \frac {1}{8 \eta}\right) \sum_ {t = 2} ^ {T} \| \boldsymbol {z} ^ {(t)} - \boldsymbol {z} ^ {(t - 1)} \| _ {2} ^ {2} \leq 0, \\ \end{array}
$$
for any leaning rate  $\eta \leq \frac{1}{4L}$ . Finally, using the fact that  $\mathrm{DReg}_x^{(T)}(\pmb{x}^{(1,\star)},\dots,\pmb{x}^{(T,\star)}) + \mathrm{DReg}_y^{(T)}(\pmb{y}^{(1,\star)},\dots,\pmb{y}^{(T,\star)}) \geq -2\sum_{t=1}^T \epsilon^{(t)}$  for a suitable sequence of  $\epsilon^{(t)}$ -approximate Nash equilibria (Property 3.1)—one that attains the variation measure  $\mathcal{V}_{\epsilon-\mathrm{NE}}^{(T)}$ —yields that
$$
0 \leq \frac {D _ {\mathcal {Z}} ^ {2}}{2 \eta} + \eta L ^ {2} \| \mathcal {Z} \| _ {2} ^ {2} + \frac {D _ {\mathcal {Z}}}{\eta} \mathcal {V} _ {\epsilon - \mathrm {N E}} ^ {(T)} + 2 \eta \| \mathcal {Z} \| _ {2} ^ {2} \mathcal {V} _ {\mathbf {A}} ^ {(T)} - \frac {1}{4 \eta} \sum_ {t = 1} ^ {T} \left(\| \boldsymbol {z} ^ {(t)} - \hat {\boldsymbol {z}} ^ {(t)} \| _ {2} ^ {2} + \| \boldsymbol {z} ^ {(t)} - \hat {\boldsymbol {z}} ^ {(t + 1)} \| _ {2} ^ {2}\right),
$$
where in the above derivation it suffices if the parameter  $C$  of  $\mathcal{V}_{\epsilon -\mathrm{NE}}^{(T)}$  is such that
$$
2 \leq \frac {D _ {\mathcal {Z}}}{\eta} C \Longleftrightarrow C = C (\eta) \geq \frac {2 \eta}{D _ {\mathcal {Z}}}. \tag {28}
$$
Thus, rearranging the last displayed inequality concludes the proof.
Next, we refine this theorem in time-varying games in which the deviation of the payoff matrices is bounded by the deviation of the players' strategies, in the following formal sense.
Corollary A.9. Suppose that both players employ OGD with learning rate  $\eta \leq \min \left\{\frac{1}{4L}, \frac{1}{8W\|\mathcal{Z}\|}\right\}$  in a time-varying bilinear saddle-point problem, where  $L \coloneqq \max_{1 \leq t \leq T} \| \mathbf{A}^{(t)}\|_2$  and  $\mathcal{V}_{\mathbf{A}}^{(T)} \leq W^2 \sum_{t=1}^{T-1} \| \pmb{z}^{(t+1)} - \pmb{z}^{(t)} \|_2^2$ , for some parameter  $W \in \mathbb{R}_{>0}$ . Then, for any time horizon  $T \in \mathbb{N}$ ,
$$
\sum_ {t = 1} ^ {T} \left(\| \boldsymbol {z} ^ {(t)} - \hat {\boldsymbol {z}} ^ {(t)} \| _ {2} ^ {2} + \| \hat {\boldsymbol {z}} ^ {(t)} - \hat {\boldsymbol {z}} ^ {(t + 1)} \| _ {2} ^ {2}\right) \leq 4 D _ {\mathcal {Z}} ^ {2} + 8 \eta^ {2} L ^ {2} \| \mathcal {Z} \| _ {2} ^ {2} + 8 D _ {\mathcal {Z}} \mathcal {V} _ {N E} ^ {(T)}.
$$
Proof. Following the proof of Theorem A.7, we have that for any  $\eta \leq \frac{1}{4L}$
$$
0 \leq \frac {D _ {\mathcal {Z}} ^ {2}}{2 \eta} + \eta L ^ {2} \| \mathcal {Z} \| _ {2} ^ {2} + \frac {D _ {\mathcal {Z}}}{\eta} \mathcal {V} _ {\mathrm {N E}} ^ {(T)} + 2 \eta \| \mathcal {Z} \| _ {2} ^ {2} \mathcal {V} _ {\mathbf {A}} ^ {(T)} - \frac {1}{4 \eta} \sum_ {t = 1} ^ {T} \left(\| \boldsymbol {z} ^ {(t)} - \hat {\boldsymbol {z}} ^ {(t)} \| _ {2} ^ {2} + \| \boldsymbol {z} ^ {(t)} - \hat {\boldsymbol {z}} ^ {(t + 1)} \| _ {2} ^ {2}\right).
$$
Further, for  $\eta \leq \frac{1}{8W\|\mathcal{Z}\|_2}$
$$
\begin{array}{l} 2 \eta \| \mathcal {Z} \| _ {2} ^ {2} \mathcal {V} _ {\mathbf {A}} ^ {(T)} - \frac {1}{8 \eta} \sum_ {t = 1} ^ {T} \left(\| \boldsymbol {z} ^ {(t)} - \hat {\boldsymbol {z}} ^ {(t)} \| _ {2} ^ {2} + \| \boldsymbol {z} ^ {(t)} - \hat {\boldsymbol {z}} ^ {(t + 1)} \| _ {2} ^ {2}\right) \\ \leq \left(2 \eta \| \mathcal {Z} \| _ {2} ^ {2} W ^ {2} - \frac {1}{1 6 \eta}\right) \sum_ {t = 1} ^ {T - 1} \| \boldsymbol {z} ^ {(t + 1)} - \boldsymbol {z} ^ {(t)} \| _ {2} ^ {2} \leq 0. \\ \end{array}
$$
Thus, we have shown that
$$
0 \leq \frac {D _ {\mathcal {Z}} ^ {2}}{2 \eta} + \eta L ^ {2} \| \mathcal {Z} \| _ {2} ^ {2} + \frac {D _ {\mathcal {Z}}}{\eta} \mathcal {V} _ {\mathrm {N E}} ^ {(T)} - \frac {1}{8 \eta} \sum_ {t = 1} ^ {T} \left(\| \boldsymbol {z} ^ {(t)} - \hat {\boldsymbol {z}} ^ {(t)} \| _ {2} ^ {2} + \| \boldsymbol {z} ^ {(t)} - \hat {\boldsymbol {z}} ^ {(t + 1)} \| _ {2} ^ {2}\right),
$$
and rearranging concludes the proof.
Thus, in such time-varying games it is the first-order variation term,  $\mathcal{V}_{\mathrm{NE}}^{(T)}$ , that will drive our convergence bounds.
Now before proving Theorem 3.3, we state the connection between the equilibrium gap and the deviation of the players' strategies  $\left(\| \boldsymbol{z}^{(t)} - \hat{\boldsymbol{z}}^{(t)}\|_2 + \| \boldsymbol{z}^{(t)} - \hat{\boldsymbol{z}}^{(t + 1)}\|_2\right)$ . In particular, the following claim can be extracted by [2, Claim A.14]. (We caution that we use a slightly different indexing for the secondary sequence  $(\hat{\boldsymbol{x}}_i^{(t)})$  in the definition of OMD (5) compared to [2].)
Claim A.10. Suppose that the sequences  $(\pmb{x}^{(t)})_{1\leq t\leq T}$  and  $(\hat{\pmb{x}}^{(t)})_{1\leq t\leq T + 1}$  are produced by OMD under a  $G$  -smooth regularizer 1-strongly convex with respect to a norm  $\| \cdot \|$ . Then, for any time  $t\in [[T]]$  and any  $\pmb {x}\in \mathcal{X}$ ,
$$
\langle \boldsymbol {x} ^ {(t)}, \boldsymbol {u} _ {x} ^ {(t)} \rangle \geq \langle \boldsymbol {x}, \boldsymbol {u} _ {x} ^ {(t)} \rangle - \frac {G D \chi}{\eta} \| \hat {\boldsymbol {x}} ^ {(t + 1)} - \hat {\boldsymbol {x}} ^ {(t)} \| - \| \boldsymbol {u} _ {x} ^ {(t)} \| _ {*} \| \boldsymbol {x} ^ {(t)} - \hat {\boldsymbol {x}} ^ {(t + 1)} \|.
$$
We are now ready to prove Theorem 3.3, the precise version of which is stated below.
Theorem A.11 (Detailed version of Theorem 3.3). Suppose that both players employ  $OGD$  with learning rate  $\eta = \frac{1}{4L}$  in a time-varying bilinear saddle-point problem, where  $L\coloneqq \max_{1\leq t\leq T}\| \mathbf{A}^{(t)}\| _2$ . Then,
$$
\sum_ {t = 1} ^ {T} \left(\operatorname {E Q G A P} ^ {(t)} (\boldsymbol {z} ^ {(t)})\right) ^ {2} \leq 2 L ^ {2} \left(4 D _ {\mathcal {Z}} + \| \mathcal {Z} \| _ {2}\right) ^ {2} \left(2 D _ {\mathcal {Z}} ^ {2} + 4 \eta^ {2} L ^ {2} \| \mathcal {Z} \| _ {2} ^ {2} + 4 D _ {\mathcal {Z}} \mathcal {V} _ {\epsilon - N E} ^ {(T)} + 8 \eta^ {2} \| \mathcal {Z} \| _ {2} ^ {2} \mathcal {V} _ {\mathbf {A}} ^ {(T)}\right),
$$
where  $(\pmb{z}^{(t)})_{1\leq t\leq T}$  is the sequence of joint strategy profiles produced by OG.
Proof. Let us fix  $t \in [[T]]$ . For convenience, we denote by  $\mathrm{BR}_x^{(t)}(\pmb{x}^{(t)}) \coloneqq \max_{\pmb{x} \in \mathcal{X}} \{\langle \pmb{x}, \pmb{u}_x^{(t)} \rangle\} - \langle \pmb{x}^{(t)}, \pmb{u}_x^{(t)} \rangle$ , the best response gap of Player's  $x$  strategy  $\pmb{x}^{(t)} \in \mathcal{X}$ , and similarly for  $\mathrm{BR}_y^{(t)}(\pmb{y}^{(t)})$ .
By definition, it holds that  $\mathrm{EQGAP}^{(t)}(\pmb{z}^{(t)})\coloneqq \max \{\mathrm{BR}_x^{(t)}(\pmb{x}^{(t)}),\mathrm{BR}_y^{(t)}(\pmb{y}^{(t)})\}$ . By Claim A.10, we have that
$$
\begin{array}{l} \mathbf {B R} _ {x} ^ {(t)} \left(\boldsymbol {x} ^ {(t)}\right) \leq \frac {D \chi}{\eta} \| \hat {\boldsymbol {x}} ^ {(t + 1)} - \hat {\boldsymbol {x}} ^ {(t)} \| _ {2} + \| \boldsymbol {u} _ {x} ^ {(t)} \| _ {2} \| \boldsymbol {x} ^ {(t)} - \hat {\boldsymbol {x}} ^ {(t + 1)} \| _ {2} (29) \\ \leq 4 L D _ {\mathcal {X}} \| \hat {\boldsymbol {x}} ^ {(t + 1)} - \hat {\boldsymbol {x}} ^ {(t)} \| _ {2} + L \| \mathcal {Y} \| _ {2} \| \boldsymbol {x} ^ {(t)} - \hat {\boldsymbol {x}} ^ {(t + 1)} \| _ {2} (30) \\ \leq L \left(4 D _ {\mathcal {Z}} + \| \mathcal {Z} \| _ {2}\right) \left(\| \boldsymbol {x} ^ {(t)} - \hat {\boldsymbol {x}} ^ {(t)} \| _ {2} + \| \boldsymbol {x} ^ {(t)} - \hat {\boldsymbol {x}} ^ {(t + 1)} \| _ {2}\right), (31) \\ \end{array}
$$
where (29) follows from Claim A.10 for  $G = 1$  (since the squared Euclidean regularizer  $\phi_x : \pmb{x} \mapsto \frac{1}{2} \| \pmb{x} \|_2^2$ ) is trivially 1-smooth; (30) uses the fact that  $\eta := \frac{1}{4L}$  and  $\| \pmb{u}_x^{(t)} \|_2 = \| -\mathbf{A}^{(t)} \pmb{y}^{(t)} \|_2 \leq L \| \mathcal{Y} \|$ ; and (31) follows from the triangle inequality. A similar derivation shows that
$$
\mathrm {B R} _ {y} ^ {(t)} \left(\boldsymbol {y} ^ {(t)}\right) \leq L \left(4 D _ {\mathcal {Z}} + \| \mathcal {Z} \| _ {2}\right) \left(\| \boldsymbol {y} ^ {(t)} - \hat {\boldsymbol {y}} ^ {(t)} \| _ {2} + \| \boldsymbol {y} ^ {(t)} - \hat {\boldsymbol {y}} ^ {(t + 1)} \| _ {2}\right). \tag {32}
$$
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} \left(\operatorname {E Q G A P} ^ {(t)} (\boldsymbol {z} ^ {(t)})\right) ^ {2} = \sum_ {t = 1} ^ {T} \left(\max  \left\{\mathbf {B R} _ {x} ^ {(t)} (\boldsymbol {x} ^ {(t)}), \mathbf {B R} _ {y} ^ {(t)} (\boldsymbol {y} ^ {(t)}) \right\}\right) ^ {2} \\ \leq \sum_ {t = 1} ^ {T} \left(\left(\mathbf {B R} _ {x} ^ {(t)} (\boldsymbol {x} ^ {(t)})\right) ^ {2} + \left(\mathbf {B R} _ {y} ^ {(t)} (\boldsymbol {y} ^ {(t)})\right) ^ {2}\right) \\ \leq 2 L ^ {2} \left(4 D _ {\mathcal {Z}} + \| \mathcal {Z} \| _ {2}\right) ^ {2} \sum_ {t = 1} ^ {T} \left(\| \boldsymbol {z} ^ {(t)} - \hat {\boldsymbol {z}} ^ {(t)} \| _ {2} ^ {2} + \| \boldsymbol {z} ^ {(t)} - \hat {\boldsymbol {z}} ^ {(t + 1)} \| _ {2} ^ {2}\right), \tag {33} \\ \end{array}
$$
where the last bound uses (31) and (32). Combining (33) with Theorem A.7 concludes the proof.
A.1.5 Variation-dependent regret bounds
Here we state for completeness an implication of Theorem 3.3 for deriving variation-dependent regret bounds in time-varying bilinear saddle-point problems; cf. [93].
Corollary A.12. In the setup of Theorem A.7, it holds that
$$
\begin{array}{l} \operatorname {R e g} _ {x} ^ {(T)} \leq \frac {D _ {\mathcal {X}} ^ {2}}{\eta} + 8 \eta L ^ {2} D _ {\mathcal {Z}} ^ {2} + \eta L ^ {2} \| \mathcal {Y} \| _ {2} ^ {2} + 1 6 \eta^ {3} L ^ {4} \| \mathcal {Z} \| _ {2} ^ {2} + 1 6 \eta L ^ {2} D _ {\mathcal {Z}} \mathcal {V} _ {N E} ^ {(T)} \\ + (2 \eta \| \mathcal {Y} \| _ {2} ^ {2} + 3 2 \eta^ {3} L ^ {2} \| \mathcal {Z} \| _ {2} ^ {2}) \mathcal {V} _ {\mathbf {A}} ^ {(T)}, \\ \end{array}
$$
$$
\begin{array}{l} \operatorname {R e g} _ {y} ^ {(T)} \leq \frac {D _ {y} ^ {2}}{\eta} + 8 \eta L ^ {2} D _ {\mathcal {Z}} ^ {2} + \eta L ^ {2} \| \mathcal {X} \| _ {2} ^ {2} + 1 6 \eta^ {3} L ^ {4} \| \mathcal {Z} \| _ {2} ^ {2} + 1 6 \eta L ^ {2} D _ {\mathcal {Z}} \mathcal {V} _ {N E} ^ {(T)} \\ + \left(2 \eta \| \mathcal {X} \| _ {2} ^ {2} + 3 2 \eta^ {3} L ^ {2} \| \mathcal {Z} \| _ {2} ^ {2}\right) \mathcal {V} _ {\mathbf {A}} ^ {(T)}. \\ \end{array}
$$
Proof. First, applying Lemma A.1 under  $\pmb{x}^{(1,\star)} = \dots = \pmb{x}^{(T,\star)}$ , we have
$$
\operatorname {R e g} _ {x} ^ {(T)} \leq \frac {D _ {\mathcal {X}} ^ {2}}{\eta} + \eta L ^ {2} \| \mathcal {Y} \| _ {2} ^ {2} + 2 \eta L ^ {2} \sum_ {t = 2} ^ {T} \| \boldsymbol {y} ^ {(t)} - \boldsymbol {y} ^ {(t - 1)} \| _ {2} ^ {2} + 2 \eta \| \mathcal {Y} \| _ {2} ^ {2} \sum_ {t = 2} ^ {T} \| \mathbf {A} ^ {(t)} - \mathbf {A} ^ {(t - 1)} \| _ {2} ^ {2}, \tag {34}
$$
$$
\operatorname {R e g} _ {y} ^ {(T)} \leq \frac {D _ {y} ^ {2}}{\eta} + \eta L ^ {2} \| \mathcal {X} \| _ {2} ^ {2} + 2 \eta L ^ {2} \sum_ {t = 2} ^ {T} \| \boldsymbol {x} ^ {(t)} - \boldsymbol {x} ^ {(t - 1)} \| _ {2} ^ {2} + 2 \eta \| \mathcal {X} \| _ {2} ^ {2} \sum_ {t = 2} ^ {T} \| \mathbf {A} ^ {(t)} - \mathbf {A} ^ {(t - 1)} \| _ {2} ^ {2}. \tag {35}
$$
Now, by Theorem A.7 we have
$$
\sum_ {t = 1} ^ {T} \left(\| \boldsymbol {z} ^ {(t)} - \hat {\boldsymbol {z}} ^ {(t)} \| _ {2} ^ {2} + \| \boldsymbol {z} ^ {(t)} - \hat {\boldsymbol {z}} ^ {(t + 1)} \| _ {2} ^ {2}\right) \leq 2 D _ {\mathcal {Z}} ^ {2} + 4 \eta^ {2} L ^ {2} \| \mathcal {Z} \| _ {2} ^ {2} + 4 D _ {\mathcal {Z}} \mathcal {V} _ {\mathrm {N E}} ^ {(T)} + 8 \eta^ {2} \| \mathcal {Z} \| _ {2} ^ {2} \mathcal {V} _ {\mathbf {A}} ^ {(T)}. \tag {36}
$$
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} \left(\| \boldsymbol {z} ^ {(t)} - \hat {\boldsymbol {z}} ^ {(t)} \| _ {2} ^ {2} + \| \boldsymbol {z} ^ {(t)} - \hat {\boldsymbol {z}} ^ {(t + 1)} \| _ {2} ^ {2}\right) \geq \sum_ {t = 1} ^ {T} \left(\| \boldsymbol {x} ^ {(t)} - \hat {\boldsymbol {x}} ^ {(t)} \| _ {2} ^ {2} + \| \boldsymbol {x} ^ {(t)} - \hat {\boldsymbol {x}} ^ {(t + 1)} \| _ {2} ^ {2}\right) \\ \geq \frac {1}{2} \sum_ {t = 2} ^ {T} \| \boldsymbol {x} ^ {(t)} - \boldsymbol {x} ^ {(t - 1)} \| _ {2} ^ {2}. \\ \end{array}
$$
Combining this bound with (36) and (35) gives the claimed regret bound on  $\mathrm{Reg}_y^{(T)}$ , and a similar derivation also gives the claimed bound on  $\mathrm{Reg}_x^{(T)}$ .
Hence, selecting optimally the learning rate gives an  $O\left(\sqrt{1 + \mathcal{V}_{\mathrm{NE}}^{(T)} + \mathcal{V}_{\mathbf{A}}^{(T)}}\right)$  bound on the individual regret of each player; while that optimal value depends on the variation measures, which are not known to the learners, there are techniques that would allow bypassing this [93]. Corollary A.12 can also be readily parameterized in terms of the improved variation measure  $\mathcal{V}_{\epsilon - \mathrm{NE}}^{(T)}$ .