SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \mathbb {E} _ {\xi_ {t}, \xi_ {t} ^ {\prime}} \| z _ {t + 1} - z _ {t + 1} ^ {\prime} \| ^ {2} \\ \leq \mathbb {E} \| \left(z _ {t} - z _ {t} ^ {\prime}\right) - \alpha (\tilde {\nabla} f (z _ {t}; \xi_ {t}) - \tilde {\nabla} f (z _ {t} ^ {\prime}; \xi_ {t} ^ {\prime})) \| ^ {2} \\ = \| z _ {t} - z _ {t} ^ {\prime} \| ^ {2} - 2 \alpha (\tilde {\nabla} F (z _ {t}) - \tilde {\nabla} F (z _ {t} ^ {\prime})) ^ {\top} (z _ {t} - z _ {t} ^ {\prime}) + \alpha^ {2} \mathbb {E} \| \tilde {\nabla} f (z _ {t}; \xi_ {t}) - \tilde {\nabla} f (z _ {t} ^ {\prime}; \xi_ {t} ^ {\prime}) \| ^ {2} \\ = \| z _ {t} - z _ {t} ^ {\prime} \| ^ {2} - 2 \alpha (\tilde {\nabla} F (z _ {t}) - \tilde {\nabla} F (z _ {t} ^ {\prime})) ^ {\top} (z _ {t} - z _ {t} ^ {\prime}) + \alpha^ {2} \mathbb {E} \| \tilde {\nabla} F (z _ {t}) - \tilde {\nabla} F (z _ {t} ^ {\prime}) \| ^ {2} \\ + \alpha^ {2} \mathbb {E} \| (\tilde {\nabla} f (z _ {t}; \xi_ {t}) - \tilde {\nabla} f (z _ {t} ^ {\prime}; \xi_ {t} ^ {\prime})) - (\tilde {\nabla} F (z _ {t}) - \tilde {\nabla} F (z _ {t} ^ {\prime})) \| ^ {2} \\ \leq \left(1 + \alpha^ {2} \ell^ {2}\right) \| z _ {t} - z _ {t} ^ {\prime} \| ^ {2} + 4 \alpha^ {2} \delta^ {2}. \\ \end{array}
$$
Unrolling the recursion, noticing  $z_0 = z_0'$ , we have that for any  $t = 0,1,\dots ,T - 1$
$$
\mathbb {E} \| z _ {t} - z _ {t} ^ {\prime} \| ^ {2} \leq 4 \alpha^ {2} \delta^ {2} \sum_ {i = 0} ^ {t - 1} (1 + \alpha^ {2} \ell^ {2}) ^ {i}.
$$
Since  $T \geq 1 / \epsilon^2$ , we know  $\alpha = 1 / (\ell \epsilon T) \leq 1 / (\ell \sqrt{T})$ . The reproducibility is thus
$$
\begin{array}{l} \mathbb {E} \left[ \| \bar {x} _ {T} - \bar {x} _ {T} ^ {\prime} \| ^ {2} + \| \bar {y} _ {T} - \bar {y} _ {T} ^ {\prime} \| ^ {2} \right] \leq \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} \left[ \| x _ {t} - x _ {t} ^ {\prime} \| ^ {2} + \| y _ {t} - y _ {t} ^ {\prime} \| ^ {2} \right] \\ \leq \frac {4 \alpha^ {2} \delta^ {2}}{T} \sum_ {t = 1} ^ {T - 1} \sum_ {i = 0} ^ {t - 1} (1 + \alpha^ {2} \ell^ {2}) ^ {i} \\ \leq \frac {4 \alpha^ {2} \delta^ {2}}{T} \sum_ {t = 1} ^ {T - 1} t \left(1 + \frac {1}{T}\right) ^ {T} \\ \leq 2 e \delta^ {2} \alpha^ {2} T \\ = \frac {2 e}{\ell^ {2}} \cdot \frac {\delta^ {2}}{\epsilon^ {2} T}. \\ \end{array}
$$
The last step uses the choice of  $\alpha$  such that  $\alpha^2 T = 1 / (\ell^2\epsilon^2 T)$
B.3 Guarantees of Extragradient
This section provides proof of Theorem 4.3 for the sub-optimal guarantees of Extragradient (EG).
B.3.1 General Analysis
Algorithm 5 Extragradius
$$
y _ {t + 1 / 2} = \Pi_ {\mathcal {Y}} \left(y _ {t} + \alpha \nabla_ {y} F \left(x _ {t}, y _ {t}\right)\right),
$$
$$
x _ {t + 1 / 2} = \Pi_ {\mathcal {X}} \left(x _ {t} - \alpha \nabla_ {x} F \left(x _ {t}, y _ {t}\right)\right).
$$
$$
y _ {t + 1} = \Pi_ {\mathcal {Y}} \left(y _ {t} + \alpha \nabla_ {y} F \left(x _ {t + 1 / 2}, y _ {t + 1 / 2}\right)\right),
$$
$$
x _ {t + 1} = \Pi_ {\mathcal {X}} (x _ {t} - \alpha \nabla_ {x} F (x _ {t + 1 / 2}, y _ {t + 1 / 2})).
$$
Output:  $(\bar{x}_{T + 1 / 2},\bar{y}_{T + 1 / 2}) = (1 / T)\sum_{t = 0}^{T - 1}(x_{t + 1 / 2},y_{t + 1 / 2}).$
For deterministic smooth convex-concave minimax optimization, Extragradient [48, 72] (EG), summarized in Algorithm 5, achieves the optimal  $\mathcal{O}(1 / \epsilon)$  convergence rate. When only given inexact gradients or stochastic gradients, the true gradients are just replaced by  $G(x_{t},y_{t})$  or  $\nabla f(x_{t},y_{t};\xi_{t})$ .
We provide proof of its  $\mathcal{O}(1 / \epsilon)$  convergence for completeness. The proof is standard in the literature, e.g., see Nemirovski [59] or Section 4.5 of Bubeck [20].
Lemma B.8. Under Assumption 4.1. When setting the stepsize to  $\alpha = 1 / \ell$ , the average iterates  $(\bar{x}_{T + 1 / 2},\bar{y}_{T + 1 / 2})$  of  $EG$  converges with
$$
\max  _ {y \in \mathcal {Y}} F (\bar {x} _ {T + 1 / 2}, y) - \min  _ {x \in \mathcal {X}} F (x, \bar {y} _ {T + 1 / 2}) \leq \frac {\ell D ^ {2}}{T}.
$$
This suggests  $\mathcal{O}(1 / \epsilon)$  gradient complexity is required to achieve  $\epsilon$ -saddle point.
Proof. Recall  $z_{t} = (x_{t},y_{t})$  and  $\tilde{\nabla} F(z_{t}) = (\nabla_{x}F(x_{t},y_{t}), - \nabla_{y}F(x_{t},y_{t}))$ . The EG updates in Algorithm 5 can be simplified to
$$
z _ {t + 1 / 2} = \Pi_ {\mathcal {X} \times \mathcal {Y}} \left(z _ {t} - \alpha \vec {\nabla} F (z _ {t})\right),
$$
$$
z _ {t + 1} = \Pi_ {\mathcal {X} \times \mathcal {Y}} \left(z _ {t} - \alpha \tilde {\nabla} F \left(z _ {t + 1 / 2}\right)\right). \tag {14}
$$
By fact  $(i)$  in Lemma A.1, we have that for any  $z\in \mathcal{X}\times \mathcal{Y}$
$$
\begin{array}{l} \left\| z _ {t + 1} - z \right\| ^ {2} + \left\| z _ {t + 1} - z _ {t} \right\| ^ {2} - \left\| z _ {t} - z \right\| ^ {2} = 2 \left(z _ {t + 1} - z _ {t}\right) ^ {\top} \left(z _ {t + 1} - z\right) \\ \leq 2 \alpha \tilde {\nabla} F (z _ {t + 1 / 2}) ^ {\top} (z - z _ {t + 1}), \\ \end{array}
$$
where we use the optimality condition of the projection step such that  $(\Pi_{\mathcal{C}}(u) - u)^{\top}(v - \Pi_{\mathcal{C}}(u)) \geq 0, \forall v \in \mathcal{C}$ . For the same reason, we can obtain that
$$
\begin{array}{l} \left\| z _ {t + 1 / 2} - z _ {t} \right\| ^ {2} + \left\| z _ {t + 1 / 2} - z _ {t + 1} \right\| ^ {2} - \left\| z _ {t} - z _ {t + 1} \right\| ^ {2} = 2 \left(z _ {t + 1 / 2} - z _ {t}\right) ^ {\top} \left(z _ {t + 1 / 2} - z _ {t + 1}\right) \\ \leq 2 \alpha \tilde {\nabla} F (z _ {t}) ^ {\top} (z _ {t + 1} - z _ {t + 1 / 2}). \\ \end{array}
$$
Summing up the above two inequalities, we get
$$
\begin{array}{l} \left\| z _ {t + 1} - z \right\| ^ {2} \leq \left\| z _ {t} - z \right\| ^ {2} - \left\| z _ {t + 1 / 2} - z _ {t} \right\| ^ {2} - \left\| z _ {t + 1 / 2} - z _ {t + 1} \right\| ^ {2} + 2 \alpha \tilde {\nabla} F (z _ {t + 1 / 2}) ^ {\top} (z - z _ {t + 1}) \\ + 2 \alpha \bar {\nabla} F (z _ {t}) ^ {\top} \left(z _ {t + 1} - z _ {t + 1 / 2}\right) \\ = \| z _ {t} - z \| ^ {2} - \| z _ {t + 1 / 2} - z _ {t} \| ^ {2} - \| z _ {t + 1 / 2} - z _ {t + 1} \| ^ {2} + 2 \alpha \tilde {\nabla} F (z _ {t + 1 / 2}) ^ {\top} (z - z _ {t + 1 / 2}) \\ + 2 \alpha \left(\tilde {\nabla} F \left(z _ {t}\right) - \tilde {\nabla} F \left(z _ {t + 1 / 2}\right)\right) ^ {\top} \left(z _ {t + 1} - z _ {t + 1 / 2}\right). \tag {15} \\ \end{array}
$$
According to Lemma B.1, we can obtain
$$
\begin{array}{l} \big (\tilde {\nabla} F (z _ {t}) - \tilde {\nabla} F (z _ {t + 1 / 2}) \big) ^ {\top} (z _ {t + 1} - z _ {t + 1 / 2}) \leq \ell \| z _ {t} - z _ {t + 1 / 2} \| \| z _ {t + 1} - z _ {t + 1 / 2} \| \\ \leq \frac {\ell}{2} \| z _ {t} - z _ {t + 1 / 2} \| ^ {2} + \frac {\ell}{2} \| z _ {t + 1} - z _ {t + 1 / 2} \| ^ {2}. \\ \end{array}
$$
Therefore, rearranging terms, by the choice of stepsize  $\alpha \leq 1 / \ell$ , we have  $\forall z \in \mathcal{X} \times \mathcal{Y}$
$$
\begin{array}{l} \tilde {\nabla} F \left(z _ {t + 1 / 2}\right) ^ {\top} \left(z _ {t + 1 / 2} - z\right) \\ \leq \frac {1}{2 \alpha} \left(\| z _ {t} - z \| ^ {2} - \| z _ {t + 1} - z \| ^ {2}\right) - \frac {1}{2 \alpha} \| z _ {t + 1 / 2} - z _ {t} \| ^ {2} - \frac {1}{2 \alpha} \| z _ {t + 1 / 2} - z _ {t + 1} \| ^ {2} \\ + \left(\tilde {\nabla} F (z _ {t}) - \tilde {\nabla} F (z _ {t + 1 / 2})\right) ^ {\top} (z _ {t + 1} - z _ {t + 1 / 2}) \\ \leq \frac {1}{2 \alpha} \left(\| z _ {t} - z \| ^ {2} - \| z _ {t + 1} - z \| ^ {2}\right) - \left(\frac {1}{2 \alpha} - \frac {\ell}{2}\right) \| z _ {t + 1 / 2} - z _ {t} \| ^ {2} - \left(\frac {1}{2 \alpha} - \frac {\ell}{2}\right) \| z _ {t + 1 / 2} - z _ {t + 1} \| ^ {2} \\ \leq \frac {1}{2 \alpha} \left(\| z _ {t} - z \| ^ {2} - \| z _ {t + 1} - z \| ^ {2}\right). \tag {16} \\ \end{array}
$$
Taking summation from  $t = 0$  to  $T - 1$ , by Lemma B.2, we have
$$
\max  _ {y \in \mathcal {Y}} F (\bar {x} _ {T + 1 / 2}, y) - \min  _ {x \in \mathcal {X}} F (x, \bar {y} _ {T + 1 / 2}) \leq \frac {\| z _ {0} - z \| ^ {2}}{2 \alpha T}.
$$
Since  $\| z_0 - z \|^2 \leq 2D^2$  and  $\alpha = 1 / \ell$ , the proof is complete.
The following results are motivated from Boob and Guzmán [16].
Lemma B.9. Under Assumption 4.1. Let  $z_{t + 1} = (x_{t + 1},y_{t + 1})$  be obtained through 1-step of EG update (14) given  $z_{t} = (x_{t},y_{t})$ , and  $z_{t + 1}^{\prime}$  is obtained given  $z_{t}^{\prime}$ . Setting  $\alpha \leq 1 / \ell$ , then we have
$$
\left\| z _ {t + 1} - z _ {t + 1} ^ {\prime} \right\| \leq \left\| z _ {t} - z _ {t} ^ {\prime} \right\| + 2 L \ell^ {2} \alpha^ {3}.
$$
Proof. For any  $z = (x,y) \in \mathcal{X} \times \mathcal{Y}$ , we define an operator  $P_{z_t}(\cdot): \mathcal{X} \times \mathcal{Y} \to \mathcal{X} \times \mathcal{Y}$  as  $P_{z_t}(z) = \Pi_{\mathcal{X} \times \mathcal{Y}}(z_t - \alpha \bar{\nabla} F(z))$ , and the EG updates can be written as  $z_{t+1} = P_{z_t}(P_{z_t}(z_t))$ . When the stepsize  $\alpha \leq 1/\ell$ , the operator  $P_{z_t}(\cdot)$  is nonexpansive, i.e.,  $\forall z_1, z_2 \in \mathcal{X} \times \mathcal{Y}$ ,
$$
\begin{array}{l} \left\| P _ {z _ {t}} \left(z _ {1}\right) - P _ {z _ {t}} \left(z _ {2}\right) \right\| \leq \alpha \| \tilde {\nabla} F (z _ {1}) - \tilde {\nabla} F (z _ {2}) \| \\ \leq \alpha \ell \| z _ {1} - z _ {2} \| \\ \leq \left\| z _ {1} - z _ {2} \right\|. \\ \end{array}
$$
Since the domain  $\mathcal{X} \times \mathcal{Y}$  is a nonempty bounded closed convex set, by Theorem 4.19 in Bauschke et al. [12], the nonexpansive operator  $P_{z_t}(\cdot)$  admits fixed points. Denote one fixed point as  $u_t \in \mathcal{X} \times \mathcal{Y}$  such that  $u_t = \Pi_{\mathcal{X} \times \mathcal{Y}}(z_t - \alpha \tilde{\nabla} F(u_t)) = P_{z_t}(u_t)$ . The nonexpansiveness of  $P_{z_t}(\cdot)$  implies
$$
\begin{array}{l} \left\| z _ {t + 1} - u _ {t} \right\| = \left\| P _ {z _ {t}} \left(P _ {z _ {t}} (z _ {t})\right) - P _ {z _ {t}} \left(P _ {z _ {t}} (u _ {t})\right) \right\| \\ \leq (\alpha \ell) ^ {2} \| z _ {t} - u _ {t} \| \\ \leq \alpha^ {2} \ell^ {2} \cdot \alpha \| \tilde {\nabla} F (u _ {t}) \| \tag {17} \\ <   \alpha^ {3} \ell^ {2} L. \\ \end{array}
$$
The same holds true for  $z_{t+1}^{\prime}$  and  $u_{t}^{\prime} = P_{z_{t}^{\prime}}(u_{t}^{\prime})$  defined for  $z_{t}^{\prime}$ . As a result, we can obtain that
$$
\begin{array}{l} \left\| z _ {t + 1} - z _ {t + 1} ^ {\prime} \right\| \leq \left\| z _ {t + 1} - u _ {t} \right\| + \left\| u _ {t} - u _ {t} ^ {\prime} \right\| + \left\| u _ {t} ^ {\prime} - z _ {t + 1} ^ {\prime} \right\| \tag {18} \\ \leq \left\| u _ {t} - u _ {t} ^ {\prime} \right\| + 2 L \ell^ {2} \alpha^ {3}. \\ \end{array}
$$
By optimality conditions of  $u_{t} = \Pi_{\mathcal{X}\times \mathcal{Y}}(z_{t} - \alpha \tilde{\nabla} F(u_{t}))$  and  $u_{t}^{\prime} = \Pi_{\mathcal{X}\times \mathcal{Y}}(z_{t}^{\prime} - \alpha \tilde{\nabla} F(u_{t}^{\prime}))$ , we obtain that for any  $z, z^{\prime}\in \mathcal{X}\times \mathcal{Y}$
$$
\left(u _ {t} - z _ {t} + \alpha \tilde {\nabla} F (u _ {t})\right) ^ {\top} (z - u _ {t}) \geq 0,
$$
$$
\left(u _ {t} ^ {\prime} - z _ {t} ^ {\prime} + \alpha \tilde {\nabla} F \left(u _ {t} ^ {\prime}\right)\right) ^ {\top} \left(z ^ {\prime} - u _ {t} ^ {\prime}\right) \geq 0.
$$
Taking  $z = u_t'$  and  $z' = u_t$  and using the fact that  $\bar{\nabla} F$  is monotone by Lemma B.1, we obtain that
$$
\begin{array}{l} \left\| u _ {t} - u _ {t} ^ {\prime} \right\| ^ {2} \leq \left(u _ {t} - u _ {t} ^ {\prime}\right) ^ {\top} \left(z _ {t} - z _ {t} ^ {\prime}\right) - \alpha \left(\tilde {\nabla} F (u _ {t}) - \tilde {\nabla} F (u _ {t} ^ {\prime})\right) ^ {\top} \left(u _ {t} - u _ {t} ^ {\prime}\right) \\ \leq \| u _ {t} - u _ {t} \| \| z _ {t} - z _ {t} ^ {\prime} \|. \\ \end{array}
$$
Combined with (18), the proof is complete since  $\| u_t - u_t' \| \leq \| z_t - z_t' \|$ .
Remark 6. We can alternatively derive the relation between  $\| z_{t + 1} - z_{t + 1}' \|$  and  $\| z_t - z_t' \|$  as follows:
$$
\begin{array}{l} \left\| z _ {t + 1} - z _ {t + 1} ^ {\prime} \right\| ^ {2} \\ \leq \| z _ {t} - z _ {t} ^ {\prime} \| ^ {2} - 2 \alpha (z _ {t} - z _ {t} ^ {\prime}) ^ {\top} (\tilde {\nabla} F (z _ {t + 1 / 2}) - \tilde {\nabla} F (z _ {t + 1 / 2} ^ {\prime})) + \alpha^ {2} \| \tilde {\nabla} F (z _ {t + 1 / 2}) - \tilde {\nabla} F (z _ {t + 1 / 2} ^ {\prime}) \| ^ {2} \\ \leq \left\| z _ {t} - z _ {t} ^ {\prime} \right\| ^ {2} + 2 \alpha \ell \left\| z _ {t} - z _ {t} ^ {\prime} \right\| \left\| z _ {t + 1 / 2} - z _ {t + 1 / 2} ^ {\prime} \right\| + \alpha^ {2} \ell^ {2} \left\| z _ {t + 1 / 2} - z _ {t + 1 / 2} ^ {\prime} \right\| ^ {2} \\ \leq \left(1 + 2 \alpha \ell \sqrt {1 + \alpha^ {2} \ell^ {2}} + \alpha^ {2} \ell^ {2} \left(1 + \alpha^ {2} \ell^ {2}\right)\right) \| z _ {t} - z _ {t} ^ {\prime} \| ^ {2} \\ = \left(1 + \alpha \ell \sqrt {1 + \alpha^ {2} \ell^ {2}}\right) ^ {2} \| z _ {t} - z _ {t} ^ {\prime} \| ^ {2}. \\ \end{array}
$$
Here, we use Lemma B.1 and B.5. The above results will lead to reproducibility that grows with  $\mathcal{O}(e^T)$ , which is similar to the results of AGD for the minimization setting [6].
B.3.2 Inexact Initialization Oracle
Theorem B.10 (Restate Theorem 4.3, part (i)). Under Assumptions 4.1. The average iterate  $(\bar{x}_{T + 1 / 2},\bar{y}_{T + 1 / 2})$  of  $EG$  satisfies  $\max_{y\in \mathcal{Y}}F(\bar{x}_{T + 1 / 2},y) - \min_{x\in \mathcal{X}}F(x,\bar{y}_{T + 1 / 2})\leq \mathcal{O}(\epsilon)$  with complexity  $T = \mathcal{O}(1 / \epsilon)$  if setting stepsize  $\alpha = 1 / \ell$ . Furthermore, the reproducibility, i.e.,  $(\epsilon ,\delta)$ -deviation between outputs of two independent runs of  $EG$  given different initialization is  $\| \bar{x}_{T + 1 / 2} - \bar{x}_{T + 1 / 2}^{\prime}\|^{2} + \| \bar{y}_{T + 1 / 2} - \bar{y}_{T + 1 / 2}^{\prime}\|^{2}\leq \mathcal{O}(\min \{\delta^{2}e^{1 / \epsilon},\delta^{2} + 1 / \epsilon^{2},D^{2}\})$ .
Proof. The convergence part directly follows from Lemma B.8 with  $T = c / \epsilon$  for some constant  $c \geq \ell D^2$ . For reproducibility, by Lemma B.5, B.9 and the stepsize  $\alpha = 1 / \ell$ , we have that for  $t = 1, 2, \dots, T - 1$ ,
$$
\begin{array}{l} \left\| x _ {t + 1 / 2} - x _ {t + 1 / 2} ^ {\prime} \right\| ^ {2} + \left\| y _ {t + 1 / 2} - y _ {t + 1 / 2} ^ {\prime} \right\| ^ {2} \leq \left(1 + \alpha^ {2} \ell^ {2}\right) \left(\left\| x _ {t} - x _ {t} ^ {\prime} \right\| ^ {2} + \left\| y _ {t} - y _ {t} ^ {\prime} \right\| ^ {2}\right) \\ \leq 2 \left(\left\| z _ {0} - z _ {0} ^ {\prime} \right\| + 2 L \ell^ {2} \alpha^ {3} t\right) ^ {2} \\ \leq 2 \left(\delta + \frac {2 L}{\ell} t\right) ^ {2}. \\ \end{array}
$$
The above also holds for  $t = 0$ . Therefore, by Jensen's inequality, we obtain
$$
\begin{array}{l} \| \bar {x} _ {T + 1 / 2} - \bar {x} _ {T + 1 / 2} ^ {\prime} \| ^ {2} + \| \bar {y} _ {T + 1 / 2} - \bar {y} _ {T + 1 / 2} ^ {\prime} \| ^ {2} \leq \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \Big (\| x _ {t + 1 / 2} - x _ {t + 1 / 2} ^ {\prime} \| ^ {2} + \| y _ {t + 1 / 2} - y _ {t + 1 / 2} ^ {\prime} \| ^ {2} \Big) \\ \leq \frac {2}{T} \sum_ {t = 0} ^ {T - 1} \left(\delta + \frac {2 L}{\ell} t\right) ^ {2} \\ \leq 4 \delta^ {2} + \frac {1 6 L ^ {2}}{3 \ell^ {2}} T ^ {2}. \\ \end{array}
$$
Alternatively, by Remark 6, we know that  $\| z_{t + 1 / 2} - z_{t + 1 / 2}'\| ^2\leq 2(1 + \sqrt{2})^{2t}\delta^2$  , and thus the reproducibility is  $\| \bar{x}_{T + 1 / 2} - \bar{x}_{T + 1 / 2}'\| ^2 +\| \bar{y}_{T + 1 / 2} - \bar{y}_{T + 1 / 2}'\| ^2\leq \mathcal{O}(e^T\delta^2)$  . The proof is complete by taking the minimum between the two results and replacing  $T$  with  $c / \epsilon$
B.3.3 Inexact Deterministic Gradient Oracle
When only given inexact gradient  $(G_{x}(x_{t},y_{t}),G_{y}(x_{t},y_{t}))$ , the updates of EG become
$$
z _ {t + 1 / 2} = \Pi_ {\mathcal {X} \times \mathcal {Y}} \left(z _ {t} - \alpha \tilde {G} (z _ {t})\right),
$$
$$
z _ {t + 1} = \Pi_ {\mathcal {X} \times \mathcal {Y}} (z _ {t} - \alpha \tilde {G} (z _ {t + 1 / 2})),
$$
where exact gradients  $\tilde{\nabla} F(z_{t})$  in (14) are replaced by  $\tilde{G} (z_t) = (G_x(x_t,y_t), - G_y(x_t,y_t))$ .
Theorem B.11 (Restate Theorem 4.3, part (ii)). Under Assumptions 4.1. Given an inexact deterministic gradient oracle in Definition 4 with  $\delta \leq \mathcal{O}(\epsilon)$ . The average iterate  $(\bar{x}_{T + 1 / 2},\bar{y}_{T + 1 / 2})$  of  $EG$  satisfies  $\max_{y\in \mathcal{Y}}F(\bar{x}_{T + 1 / 2},y) - \min_{x\in \mathcal{X}}F(x,\bar{y}_{T + 1 / 2})\leq \mathcal{O}(\epsilon)$  with complexity  $T = \mathcal{O}(1 / \epsilon)$  if setting stepsize  $\alpha = 1 / \ell$ . Furthermore, the reproducibility is  $\mathcal{O}(\min \{\delta^2 e^{1 / \epsilon},1 / \epsilon^2,D^2\})$ .
Proof. Let  $\Delta(z_t) = \tilde{G}(z_t) - \tilde{\nabla} F(z_t)$ . We know  $\|\Delta(z_t)\| \leq \delta$  by Definition 4. Using (15) in the proof of Lemma B.8, we have that  $\forall z \in \mathcal{X} \times \mathcal{Y}$ ,
$$
\begin{array}{l} \left\| z _ {t + 1} - z \right\| ^ {2} \leq \left\| z _ {t} - z \right\| ^ {2} - \left\| z _ {t + 1 / 2} - z _ {t} \right\| ^ {2} - \left\| z _ {t + 1 / 2} - z _ {t + 1} \right\| ^ {2} + 2 \alpha \tilde {G} \left(z _ {t + 1 / 2}\right) ^ {\top} \left(z - z _ {t + 1 / 2}\right) \\ + 2 \alpha (\tilde {G} (z _ {t}) - \tilde {G} (z _ {t + 1 / 2})) ^ {\top} (z _ {t + 1} - z _ {t + 1 / 2}) \\ = \| z _ {t} - z \| ^ {2} - \| z _ {t + 1 / 2} - z _ {t} \| ^ {2} - \| z _ {t + 1 / 2} - z _ {t + 1} \| ^ {2} + 2 \alpha \tilde {\nabla} F (z _ {t + 1 / 2}) ^ {\top} (z - z _ {t + 1 / 2}) \\ + 2 \alpha (\bar {\nabla} F (z _ {t}) - \bar {\nabla} F (z _ {t + 1 / 2})) ^ {\top} (z _ {t + 1} - z _ {t + 1 / 2}) + 2 \alpha \Delta (z _ {t + 1 / 2}) ^ {\top} (z - z _ {t + 1 / 2}) \\ + 2 \alpha \left(\Delta \left(z _ {t}\right) - \Delta \left(z _ {t + 1 / 2}\right)\right) ^ {\top} \left(z _ {t + 1} - z _ {t + 1 / 2}\right) \\ \leq \left\| z _ {t} - z \right\| ^ {2} - \left\| z _ {t + 1 / 2} - z _ {t} \right\| ^ {2} - \left\| z _ {t + 1 / 2} - z _ {t + 1} \right\| ^ {2} + 2 \alpha \tilde {\nabla} F (z _ {t + 1 / 2}) ^ {\top} (z - z _ {t + 1 / 2}) \\ + 2 \alpha (\tilde {\nabla} F (z _ {t}) - \tilde {\nabla} F (z _ {t + 1 / 2})) ^ {\top} (z _ {t + 1} - z _ {t + 1 / 2}) + 6 \sqrt {2} \alpha \delta D. \tag {19} \\ \end{array}
$$
The above is the same as (15) up to an additional error in  $\mathcal{O}(\delta)$ . Following the same proof after (15), with  $\alpha = 1 / \ell$ , we obtain that
$$
\max  _ {y \in \mathcal {Y}} F (\bar {x} _ {T + 1 / 2}, y) - \min  _ {x \in \mathcal {X}} F (x, \bar {y} _ {T + 1 / 2}) \leq \frac {\ell D ^ {2}}{T} + 3 \sqrt {2} \delta D.
$$
When  $\delta \leq \epsilon /(6\sqrt{2} D)$  and  $T = c / \epsilon$  for some constant  $c\geq 2\ell D^2 /\epsilon$ , we get  $\epsilon$ -saddle point.
We then show the reproducibility guarantee. Let  $u_{t} = \Pi_{\mathcal{X}\times \mathcal{Y}}(z_{t} - \alpha \tilde{\nabla} F(u_{t}))$  be the same as in the proof of Lemma B.9. Similarly to (17), we have that
$$
\begin{array}{l} \left\| z _ {t + 1} - u _ {t} \right\| \leq \alpha \| \tilde {G} \left(z _ {t + 1 / 2}\right) - \tilde {\nabla} F (u _ {t}) \| \\ \leq \alpha \| \tilde {\nabla} F (z _ {t + 1 / 2}) - \tilde {\nabla} F (u _ {t}) \| + \alpha \| \tilde {G} (z _ {t + 1 / 2}) - \tilde {\nabla} F (z _ {t + 1 / 2}) \| \\ \leq \alpha \ell \| z _ {t + 1 / 2} - u _ {t} \| + \alpha \delta \\ \leq \alpha^ {2} \ell \| \tilde {G} (z _ {t}) - \tilde {\nabla} F (u _ {t}) \| + \alpha \delta \\ \leq \alpha^ {2} \ell^ {2} \| z _ {t} - u _ {t} \| + (1 + \alpha \ell) \alpha \delta \\ \leq \alpha^ {3} \ell^ {2} L + (1 + \alpha \ell) \alpha \delta . \\ \end{array}
$$
As a result, the same as (18), since  $\alpha = 1 / \ell$ , we can obtain that  $\forall t = 0,1,\dots ,T - 1$
$$
\begin{array}{l} \left\| z _ {t} - z _ {t} ^ {\prime} \right\| \leq \left\| z _ {t - 1} - z _ {t - 1} ^ {\prime} \right\| + 2 \alpha^ {3} \ell^ {2} L + 2 (1 + \alpha \ell) \alpha \delta \\ \leq t \left(2 \alpha^ {3} \ell^ {2} L + 2 (1 + \alpha \ell) \alpha \delta\right) \\ \leq \frac {2 t}{\ell} (L + 2 \delta). \\ \end{array}
$$
Therefore, by Jensen's inequality and (12) in Section B.2.3 for the guarantee of GDA, we know
$$
\begin{array}{l} \left\| \bar {x} _ {T + 1 / 2} - \bar {x} _ {T + 1 / 2} ^ {\prime} \right\| ^ {2} + \left\| \bar {y} _ {T + 1 / 2} - \bar {y} _ {T + 1 / 2} ^ {\prime} \right\| ^ {2} \leq \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \left(\left\| x _ {t + 1 / 2} - x _ {t + 1 / 2} ^ {\prime} \right\| ^ {2} + \left\| y _ {t + 1 / 2} - y _ {t + 1 / 2} ^ {\prime} \right\| ^ {2}\right) \\ \leq \frac {1}{T} \sum_ {t = 0} ^ {T - 1} 2 \left((1 + \alpha^ {2} \ell^ {2}) \| z _ {t} - z _ {t} ^ {\prime} \| ^ {2} + 4 \alpha^ {2} \delta^ {2}\right) \\ \leq \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \frac {8}{\ell^ {2}} \left(2 t ^ {2} (L + 2 \delta) ^ {2} + \delta^ {2}\right) \\ \leq \frac {1 2 8}{3 \ell^ {2}} \delta^ {2} T ^ {2} + \frac {3 2 L ^ {2}}{3 \ell^ {2}} T ^ {2} + \frac {8}{\ell^ {2}} \delta^ {2}. \\ \end{array}
$$
Note that  $T = c / \epsilon$  and  $\delta \leq \mathcal{O}(\epsilon)$ . Thus the reproducibility is  $\mathcal{O}(1 / \epsilon^2)$ . Alternatively, by Remark 6 and similarly to (12), we have that
$$
\begin{array}{l} \| z _ {t + 1} - z _ {t + 1} ^ {\prime} \| \\ \leq \sqrt {\| z _ {t} - z _ {t} ^ {\prime} \| ^ {2} + 2 \alpha \ell \| z _ {t} - z _ {t} ^ {\prime} \| \| z _ {t + 1 / 2} - z _ {t + 1 / 2} ^ {\prime} \| + \alpha^ {2} \ell^ {2} \| z _ {t + 1 / 2} - z _ {t + 1 / 2} ^ {\prime} \|} + 2 \alpha \delta \\ \leq \sqrt {\left(1 + \alpha \ell \sqrt {1 + \alpha^ {2} \ell^ {2}}\right) ^ {2} \| z _ {t} - z _ {t} ^ {\prime} \| ^ {2} + 4 \alpha^ {2} \ell \delta \left(1 + \alpha \ell \sqrt {1 + \alpha^ {2} \ell^ {2}}\right) \| z _ {t} - z _ {t} ^ {\prime} \| + 4 \alpha^ {4} \ell^ {2} \delta^ {2}} + 2 \alpha \delta \\ = \left(1 + \alpha \ell \sqrt {1 + \alpha^ {2} \ell^ {2}}\right) \| z _ {t} - z _ {t} ^ {\prime} \| + 2 \alpha \delta (1 + \alpha \ell) \\ = (1 + \sqrt {2}) \| z _ {t} - z _ {t} ^ {\prime} \| + \frac {4 \delta}{\ell}. \\ \end{array}
$$
Thus  $\| z_{t + 1 / 2} - z_{t + 1 / 2}^{\prime}\| \leq \sqrt{2}\| z_{t} - z_{t}^{\prime}\| +2\delta /\ell \leq \mathcal{O}(e^{T}\delta /\ell)$  and the reproducibility is  $\mathcal{O}(\delta^2 e^{1 / \epsilon})$  The proof is complete by taking the minimum between the two results.
B.3.4 More Discussions
In this section, we show that Extragradient can also be optimally reproducible by a different selection of parameters. Although it will suffer from a sub-optimal convergence rate  $\mathcal{O}(1 / \epsilon^{3 / 2})$  instead of  $\mathcal{O}(1 / \epsilon)$ , this is still an improvement on the  $\mathcal{O}(1 / \epsilon^2)$  rate of GDA.
Theorem B.12. Under Assumptions 4.1. The average iterate  $(\bar{x}_{T + 1 / 2},\bar{y}_{T + 1 / 2})$  of  $EG$  satisfies  $\max_{y\in \mathcal{Y}}F(\bar{x}_{T + 1 / 2},y) - \min_{x\in \mathcal{X}}F(x,\bar{y}_{T + 1 / 2})\leq \mathcal{O}(\epsilon)$  with complexity  $T = \mathcal{O}(1 / (\delta^{1 / 2}\epsilon^{3 / 2}))$  if setting stepsize  $\alpha = \min \{1 / \ell ,(\delta /(2\ell^2 T))^{1 / 3}\}$ . The reproducibility is  $\mathcal{O}(\delta^2)$ .
Proof. The same as Section B.3.2, by the choice of stepsize  $\alpha$  such that  $\alpha^3 T\leq \delta /2\ell^2$ , we obtain
$$
\begin{array}{l} \left\| \bar {x} _ {T + 1 / 2} - \bar {x} _ {T + 1 / 2} ^ {\prime} \right\| ^ {2} + \left\| \bar {y} _ {T + 1 / 2} - \bar {y} _ {T + 1 / 2} ^ {\prime} \right\| ^ {2} \leq \frac {2}{T} \sum_ {t = 0} ^ {T - 1} \left(\delta + 2 L \ell^ {2} \alpha^ {3} t\right) ^ {2} \\ \leq 4 \delta^ {2} + 4 (2 L \ell^ {2} \alpha^ {3} T) ^ {2} \\ \leq 4 (L ^ {2} + 1) \delta^ {2}. \\ \end{array}
$$
By Lemma B.8, when the stepsize  $\alpha \leq 1 / \ell$ , we have that
$$
\begin{array}{l} \max  _ {y \in \mathcal {Y}} F (\bar {x} _ {T + 1 / 2}, y) - \min  _ {x \in \mathcal {X}} F (x, \bar {y} _ {T + 1 / 2}) \leq \frac {D ^ {2}}{\alpha T} \\ \leq \frac {\ell D ^ {2}}{T} + \frac {D ^ {2} (2 \ell^ {2} / \delta) ^ {1 / 3}}{T ^ {2 / 3}}. \\ \end{array}
$$
This means a  $\mathcal{O}(1 / (\delta^{1 / 2}\epsilon^{3 / 2}))$  convergence rate with reproducibility  $\mathcal{O}(\delta^2)$ . In the case  $\delta = \mathcal{O}(1)$ , the gradient complexity is  $\mathcal{O}(1 / \epsilon^{3 / 2})$ .