SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proof of Theorem 3.5. For the convergence guarantee, similarly to the perturbed initialization case in (4), for  $x^{*} \in \arg \min_{x \in \mathcal{X}} F(x)$  and  $x_{r}^{*} = \arg \min_{x \in \mathcal{X}} F_{r}(x)$ , we have that
$$
F (x _ {r}) - F (x ^ {*}) \leq F _ {r} (x _ {r}) - F _ {r} (x _ {r} ^ {*}) + \frac {r D ^ {2}}{2}.
$$
For the reproducibility guarantee, using  $r$ -strong-convexity of  $F_{r}(x)$ , we can obtain that
$$
\begin{array}{l} \left\| x _ {r} - x _ {r} ^ {\prime} \right\| \leq \left\| x _ {r} - x _ {r} ^ {*} \right\| + \left\| x _ {r} ^ {*} - x _ {r} ^ {\prime} \right\| \\ \leq \sqrt {\frac {2 \left(F _ {r} \left(x _ {r}\right) - F _ {r} \left(x _ {r} ^ {*}\right)\right)}{r}} + \sqrt {\frac {2 \left(F _ {r} \left(x _ {r} ^ {\prime}\right) - F _ {r} \left(x _ {r} ^ {*}\right)\right)}{r}}. \\ \end{array}
$$
Applying Lemma 3.4, if (Inexact-AGD) is used as the base algorithm  $\mathcal{A}$  and  $x_{r}$  is the output given initialization  $y_0 = x_0$  after  $T$  iterations, since  $r = \epsilon / D^2 \leq \ell$ , we know that
$$
\begin{array}{l} F _ {r} (x _ {r}) - F _ {r} (x _ {r} ^ {*}) \\ \leq \exp \left(- \frac {T}{2} \sqrt {\frac {r}{\ell + r}}\right) \left(F _ {r} \left(x _ {0}\right) - F _ {r} \left(x _ {r} ^ {*}\right) + \frac {r}{4} \left\| x _ {0} - x _ {r} ^ {*} \right\| ^ {2}\right) + \sqrt {\frac {\ell + r}{r}} \left(\frac {1}{\ell + r} + \frac {2}{r}\right) \delta^ {2} \\ \leq \exp \left(- \frac {T}{2} \sqrt {\frac {r}{2 \ell}}\right) \left(F _ {r} \left(x _ {0}\right) - F _ {r} \left(x _ {r} ^ {*}\right) + \frac {r}{4} \left\| x _ {0} - x _ {r} ^ {*} \right\| ^ {2}\right) + 5 \delta^ {2} \sqrt {\frac {\ell}{2 r ^ {3}}}. \\ \end{array}
$$
When setting  $T = \mathcal{O}(\sqrt{\ell / r}\log (r^{3 / 2} / \delta^2))$ , this means the algorithm converges to  $F_{r}(x_{r}) - F_{r}(x_{r}^{*})\leq 6\delta^{2}\sqrt{\ell / (2r^{3})}$  and  $\| x_r - x_r^*\| ^2\leq 12\delta^2\sqrt{\ell / (2r^5)}$ . Therefore, since  $r = \epsilon /D^2$ , we have that
$$
F \left(x _ {r}\right) - F \left(x ^ {*}\right) \leq \mathcal {O} \left(\frac {\delta^ {2}}{\epsilon^ {3 / 2}} + \epsilon\right),
$$
and the reproducibility is  $\| x_r - x_r' \|^2 \leq \mathcal{O}(\delta^2 / \epsilon^{5/2})$ .
The results suggest that to achieve  $\epsilon$ -approximation error on the function value gap, we need to set  $\delta \leq \mathcal{O}(\epsilon^{5/4})$ , which is a smaller regime compared to  $\delta \leq \mathcal{O}(\epsilon)$  in the previous work [1] when  $\epsilon \leq 1$ . Furthermore, optimal reproducibility  $\mathcal{O}(\delta^2/\epsilon^2)$  is not attained. We observe from the proof that the additional  $\mathcal{O}(\sqrt{\kappa})$ -factor in the last term of the error bound in Lemma A.4 leads to this degradation. Since we set  $r = \mathcal{O}(\epsilon)$  to balance the convergence rate and approximation error introduced through regularization, this factor can be  $\mathcal{O}(\sqrt{1/\epsilon})$ . Based on the lower-bound in Devolder et al. [27] for  $(\delta, \ell, \mu)$ -oracle such that this  $\mathcal{O}(\sqrt{\kappa})$ -factor is unavoidable for an accelerated convergence rate and the transformation between the two inexact oracles in Lemma A.3, we thus make the conjecture here that the above results cannot be further improved. Algorithms that achieve optimal convergence and reproducibility under this setting require better designs and we leave it for future work.
B Preliminary Results in the Minimax Case
In this section, we provide proof of some preliminary results in the minimax setting. We start with a proof of the lower-bounds in Lemma B.3. Sub-optimal guarantees of gradient descent ascent (GDA) in the deterministic case, as well as optimal guarantees of stochastic gradient descent ascent (SGDA), are provided in Section B.2. Sub-optimal results of Extragradient (EG) are proved in Section B.3.
Before that, we introduce some notations and helpful lemmas that will be used in the analysis. We let  $z = (x,y)$  and  $\tilde{\nabla} F(z) = (\nabla_xF(x,y), - \nabla_yF(x,y))$  for simplicity of the notation in the remaining of the paper. The following results will be frequently used.
Lemma B.1. Under Assumption 4.1, the operator  $\tilde{\nabla} F$  is monotone and  $\ell$ -Lipschitz. That is,  $\forall z_1, z_2 \in \mathcal{X} \times \mathcal{Y}$ ,  $\| \tilde{\nabla} F(z_1) - \tilde{\nabla} F(z_2) \| \leq \ell \| z_1 - z_2 \|$  and  $(\tilde{\nabla} F(z_1) - \tilde{\nabla} F(z_2))^{\top} (z_1 - z_2) \geq 0$ . Moreover,  $\forall z \in \mathcal{X} \times \mathcal{Y}$ ,  $\| \tilde{\nabla} F(z) \| \leq L$  where we define  $L := \min \| \tilde{\nabla} F(z^*) \| + \sqrt{2} \ell D$  for minimum taking w.r.t. any saddle point  $z^* = (x^*, y^*)$  of  $F(x, y)$ .
Proof. Lipschitzness of  $\tilde{\nabla} F$  directly follows from  $\ell$ -smoothness of  $F(x,y)$ . The fact that  $\tilde{\nabla} F$  is monotone when  $F(x,y)$  is convex-concave is well-known in the literature (e.g., see Theorem 1 in Rockafellar [65]). For the last statement, taking any saddle point  $z^*$ , we have that  $\forall z \in \mathcal{X} \times \mathcal{Y}$ ,
$$
\begin{array}{l} \| \tilde {\nabla} F (z) \| \leq \| \tilde {\nabla} F (z ^ {*}) \| + \| \tilde {\nabla} F (z) - \tilde {\nabla} F (z ^ {*}) \| \\ \leq \left\| \bar {\nabla} F \left(z ^ {*}\right) \right\| + \ell \| z - z ^ {*} \|. \\ \end{array}
$$
The proof is complete since the domain  $\mathcal{X}$  and  $\mathcal{Y}$  have a diameter of  $D$ .
Lemma B.2. Under Assumption 4.1. For some integer  $T \geq 1$ , let  $z_{t} = (x_{t},y_{t})$  for  $t = 0,1,\dots ,T - 1$  and  $\bar{z}_T = (\bar{x}_T,\bar{y}_T) = (1 / T)\sum_{t = 0}^{T - 1}(x_t,y_t)$ . If  $\forall z\in \mathcal{X}\times \mathcal{Y}$ ,  $(1 / T)\sum_{t = 0}^{T - 1}\tilde{\nabla} F(z_t)^\top (z_t - z)\leq \epsilon$ , then it satisfies that  $\max_{y\in \mathcal{Y}}F(\bar{x}_T,y) - \min_{x\in \mathcal{X}}F(x,\bar{y}_T)\leq \epsilon$ .
Proof. Since  $F(x,y)$  is convex-concave, we get that  $\forall z = (x,y) \in \mathcal{X} \times \mathcal{Y}$ ,
$$
\begin{array}{l} F \left(x _ {t}, y\right) - F (x, y _ {t}) = F \left(x _ {t}, y\right) - F \left(x _ {t}, y _ {t}\right) + F \left(x _ {t}, y _ {t}\right) - F (x, y _ {t}) \\ \leq \nabla_ {x} F (x _ {t}, y _ {t}) ^ {\top} (x _ {t} - x) - \nabla_ {y} F (x _ {t}, y _ {t}) ^ {\top} (y _ {t} - y) \\ = \bar {\nabla} F (z _ {t}) ^ {\top} (z _ {t} - z). \\ \end{array}
$$
Summing up from  $t = 0$  to  $T - 1$  and dividing both sides by  $T$ , by Jensen's inequality, we have that
$$
\begin{array}{l} F (\bar {x} _ {T}, y) - F (x, \bar {y} _ {T}) \leq \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \tilde {\nabla} F (z _ {t}) ^ {\top} (z _ {t} - z) \\ \leq \epsilon . \\ \end{array}
$$
Taking  $y = \arg \max_{v\in \mathcal{Y}}F(\bar{x}_T,v)$  and  $x = \arg \min_{u\in \mathcal{X}}F(u,\bar{y}_T)$ , we conclude the proof.
B.1 Lower-bounds for Reproducibility
The lower-bounds follow from the minimization setting [1].
Lemma B.3. For smooth convex-concave minimax optimization under Assumption 4.1, the reproducibility, i.e.,  $(\epsilon, \delta)$ -deviation, of any algorithm  $\mathcal{A}$  is at least (i)  $\Omega(\delta^2)$  for the inexact initialization oracle; (ii)  $\Omega(\delta^2/\epsilon^2)$  for the deterministic inexact gradient oracle; (iii)  $\Omega(\delta^2/(T\epsilon^2))$  for the stochastic gradient oracle, where  $T$  is the total number of iterations of the algorithm.
Proof. The lower-bound of reproducibility in Ahn et al. [1] for smooth convex minimization problems is also a valid lower-bound for smooth convex-concave minimax problems. To show this, we consider a special case of the minimax problem (1) where the domain  $\mathcal{V}$  is a singleton, i.e.,  $\mathcal{V} = \{y_0\}$  for some  $y_0$ . Then the original smooth convex-concave minimax problem  $\min_{x \in \mathcal{X}} \max_{y \in \mathcal{Y}} F(x, y)$  is equivalent to the smooth convex minimization problem  $\min_{x \in \mathcal{X}} F(x, y_0)$ . For all three inexact oracles, let  $(\hat{x}, \hat{y})$  and  $(\hat{x}', \hat{y}')$  be the  $\epsilon$ -approximate outputs of independent two runs of the same algorithm, i.e., the duality gap can be upper-bounded by  $\epsilon$ , then the reproducibility  $\| \hat{x} - \hat{x}' \|^2 + \| \hat{y} - \hat{y}' \|^2 = \| \hat{x} - \hat{x}' \|^2$  since  $\hat{y} = \hat{y}' = y_0$ . Moreover,  $\hat{x}$  and  $\hat{x}'$  are also  $\epsilon$ -approximate solutions of the function  $F(x, y_0)$  based on the definition of duality gap. Thus the lower-bound in the minimization setting [1] directly implies the lower-bound in the minimax setting. To be specific, the lower-bound is: (i)  $\Omega(\delta^2)$  for the inexact initialization case; (ii)  $\Omega(\delta^2 / \epsilon^2)$  for the inexact deterministic gradient case; and (iii)  $\Omega(\delta^2 / (T\epsilon^2))$  for the stochastic gradient case.
B.2 Guarantees of Gradient Descent Ascent
This section provides proof of Theorem 4.2 for the sub-optimal guarantees of GDA in the deterministic setting and Theorem 5.1 for the optimal guarantees of SGDA in the stochastic setting. We first provide a general analysis and then expand it for three different inexact oracles in subsequent sections.
B.2.1 General Analysis
Algorithm 4 Gradient Descent Ascent
Input: Stepsize  $\alpha > 0$ , initialization  $(x_0, y_0)$ , number of iterations  $T > 0$ .
$$
y _ {t + 1} = \Pi_ {\mathcal {Y}} \left(y _ {t} + \alpha \nabla_ {y} F \left(x _ {t}, y _ {t}\right)\right),
$$
$$
x _ {t + 1} = \Pi_ {\mathcal {X}} \left(x _ {t} - \alpha \nabla_ {x} F \left(x _ {t}, y _ {t}\right)\right).
$$
Output:  $(\bar{x}_T,\bar{y}_T) = (1 / T)\sum_{t = 0}^{T - 1}(x_t,y_t)$ .
We consider (stochastic) gradient descent ascent (GDA/SGDA) outlined in Algorithm 4 for solving minimax problems (1) or (3). The algorithm iteratively updates the variables  $x_{t}$  and  $y_{t}$  using exact gradients  $\nabla F(x_{t},y_{t})$ , or inexact gradients  $G(x_{t},y_{t})$ , or stochastic gradients  $\nabla f(x_{t},y_{t};\xi_{t})$  based on different types of the inexact oracles in Definition 4.
We first analyze the behavior of GDA with access to exact gradients. It is well-known that the last iterate of GDA can diverge even for bilinear functions [55, 9, 36], and the average iterates converge with a sub-optimal rate  $\mathcal{O}(1 / \sqrt{T})$ . We provide proof for completeness.
Lemma B.4. Under Assumption 4.1. When setting the stepsize to  $\alpha = 1 / (\ell \sqrt{T})$ , the average iterates  $(\bar{x}_T, \bar{y}_T)$  of GDA converges with
$$
\max  _ {y \in \mathcal {Y}} F (\bar {x} _ {T}, y) - \min  _ {x \in \mathcal {X}} F (x, \bar {y} _ {T}) \leq \frac {\ell D ^ {2} + L ^ {2} / (2 \ell)}{\sqrt {T}},
$$
This suggests  $\mathcal{O}(1 / \epsilon^2)$  gradient complexity is required to achieve  $\epsilon$ -saddle point.
Proof. Recall  $z_{t} = (x_{t},y_{t})$  and  $\tilde{\nabla} F(z_{t}) = (\nabla_{x}F(x_{t},y_{t}), - \nabla_{y}F(x_{t},y_{t}))$ . The GDA updates in Algorithm 4 can be simplified to
$$
z _ {t + 1} = \Pi_ {\mathcal {X} \times \mathcal {Y}} \left(z _ {t} - \alpha \tilde {\nabla} F \left(z _ {t}\right)\right). \tag {9}
$$
Since the projection step is nonexpansive [12], we have that  $\forall z = (x,y)\in \mathcal{X}\times \mathcal{Y},$
$$
\begin{array}{l} \left\| z _ {t + 1} - z \right\| ^ {2} \leq \left\| z _ {t} - \alpha \tilde {\nabla} F (z _ {t}) - z \right\| ^ {2} \\ = \| z _ {t} - z \| ^ {2} - 2 \alpha \tilde {\nabla} F (z _ {t}) ^ {\top} (z _ {t} - z) + \alpha^ {2} \| \tilde {\nabla} F (z _ {t}) \| ^ {2}. \\ \end{array}
$$
Rearranging terms and using Lemma B.1, we can obtain that
$$
\tilde {\nabla} F (z _ {t}) ^ {\top} (z _ {t} - z) \leq \frac {1}{2 \alpha} \big (\| z _ {t} - z \| ^ {2} - \| z _ {t + 1} - z \| ^ {2} \big) + \frac {\alpha L ^ {2}}{2}.
$$
Taking summation from  $t = 0$  to  $T - 1$  and dividing both sides by  $T$ , by Lemma B.2, we thus have
$$
\max _ {y \in \mathcal {Y}} F (\bar {x} _ {T}, y) - \min _ {x \in \mathcal {X}} F (x, \bar {y} _ {T}) \leq \frac {D ^ {2}}{\alpha T} + \frac {\alpha L ^ {2}}{2}.
$$
When setting  $\alpha = 1 / (\ell \sqrt{T})$ , this means the complexity is required to be  $T \geq (\ell D^2 + L^2 / (2\ell))^2 / \epsilon^2$  to achieve an  $\epsilon$ -saddle point such that  $\max_{y \in \mathcal{Y}} F(\bar{x}_T, y) - \min_{x \in \mathcal{X}} F(x, \bar{y}_T) \leq \epsilon$ .
Lemma B.5. Under Assumption 4.1, the GDA update (9) is  $(1 + \alpha^{2}\ell^{2})$ -expansive. That is, if  $(x_{t+1}, y_{t+1})$  is obtained through 1-step of the update given  $(x_{t}, y_{t})$ , and  $(x_{t+1}^{\prime}, y_{t+1}^{\prime})$  is obtained given  $(x_{t}^{\prime}, y_{t}^{\prime})$ , we have that
$$
\left\| x _ {t + 1} - x _ {t + 1} ^ {\prime} \right\| ^ {2} + \left\| y _ {t + 1} - y _ {t + 1} ^ {\prime} \right\| ^ {2} \leq \left(1 + \alpha^ {2} \ell^ {2}\right) \big (\left\| x _ {t} - x _ {t} ^ {\prime} \right\| ^ {2} + \left\| y _ {t} - y _ {t} ^ {\prime} \right\| ^ {2} \big).
$$
Proof. Recall  $z_{t} = (x_{t},y_{t})$  and  $z_{t}^{\prime} = (x_{t}^{\prime},y_{t}^{\prime})$ . By the updates of GDA (9), we get that
$$
\begin{array}{l} \left\| z _ {t + 1} - z _ {t + 1} ^ {\prime} \right\| ^ {2} \leq \left\| \left(z _ {t} - z _ {t} ^ {\prime}\right) - \alpha (\tilde {\nabla} F (z _ {t}) - \tilde {\nabla} F (z _ {t} ^ {\prime})) \right\| ^ {2} \\ \leq \| z _ {t} - z _ {t} ^ {\prime} \| ^ {2} + \alpha^ {2} \| \tilde {\nabla} F (z _ {t}) - \tilde {\nabla} F (z _ {t} ^ {\prime}) \| ^ {2} - 2 \alpha (\tilde {\nabla} F (z _ {t}) - \tilde {\nabla} F (z _ {t} ^ {\prime})) ^ {\top} (z _ {t} - z _ {t} ^ {\prime}) \\ \leq \left(1 + \alpha^ {2} \ell^ {2}\right) \left(\| x _ {t} - x _ {t} ^ {\prime} \| ^ {2} + \| y _ {t} - y _ {t} ^ {\prime} \| ^ {2}\right), \\ \end{array}
$$
where we use the fact that the projection step is nonexpansive and Lemma B.1.
B.2.2 Inexact Initialization Oracle
Theorem B.6 (Restate Theorem 4.2, part (i)). Under Assumptions 4.1. The average iterate  $(\bar{x}_T,\bar{y}_T)$  of GDA satisfies  $\max_{y\in \mathcal{Y}}F(\bar{x}_T,y) - \min_{x\in \mathcal{X}}F(x,\bar{y}_T)\leq \mathcal{O}(\epsilon)$  with complexity  $T = \mathcal{O}(1 / \epsilon^2)$  if setting stepsize  $\alpha = 1 / (\ell \sqrt{T})$ . The reproducibility, i.e.,  $(\epsilon ,\delta)$ -deviation between outputs  $(\bar{x}_T,\bar{y}_T)$  and  $(\bar{x}_T',\bar{y}_T')$  of two independent runs given different initialization is  $\| \bar{x}_T - \bar{x}_T'\| ^2 +\| \bar{y}_T - \bar{y}_T'\| ^2\leq \mathcal{O}(\delta^2)$ .
Proof. The convergence analysis directly follows from Lemma B.4. For the reproducibility analysis, by Lemma B.5 and the choice that  $\alpha = 1 / (\ell \sqrt{T})$ , we have that for  $t = 1,2,\dots ,T - 1$
$$
\begin{array}{l} \left\| x _ {t} - x _ {t} ^ {\prime} \right\| ^ {2} + \left\| y _ {t} - y _ {t} ^ {\prime} \right\| ^ {2} \leq (1 + \alpha^ {2} \ell^ {2}) \left(\left\| x _ {t - 1} - x _ {t - 1} ^ {\prime} \right\| ^ {2} + \left\| y _ {t - 1} - y _ {t - 1} ^ {\prime} \right\| ^ {2}\right) \\ = \left(1 + \frac {1}{T}\right) \left(\left\| x _ {t - 1} - x _ {t - 1} ^ {\prime} \right\| ^ {2} + \left\| y _ {t - 1} - y _ {t - 1} ^ {\prime} \right\| ^ {2}\right) \\ \leq \left(1 + \frac {1}{T}\right) ^ {t} \left(\| x _ {0} - x _ {0} ^ {\prime} \| ^ {2} + \| y _ {0} - y _ {0} ^ {\prime} \| ^ {2}\right). \\ \end{array}
$$
The above also trivially holds for  $t = 0$ . Therefore, by Jensen's inequality, we can obtain that
$$
\begin{array}{l} \| \bar {x} _ {T} - \bar {x} _ {T} ^ {\prime} \| ^ {2} + \| \bar {y} _ {T} - \bar {y} _ {T} ^ {\prime} \| ^ {2} \leq \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \left(\| x _ {t} - x _ {t} ^ {\prime} \| ^ {2} + \| y _ {t} - y _ {t} ^ {\prime} \| ^ {2}\right) \\ \leq \delta^ {2} \cdot \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \left(1 + \frac {1}{T}\right) ^ {t} \\ \leq e \delta^ {2}. \\ \end{array}
$$
The choice of  $\alpha$  is to avoid exponential dependence on  $\ell$  in the reproducibility bound.
B.2.3 Inexact Deterministic Gradient Oracle
When only given an inexact gradient oracle in Definition 4, the updates of GDA become
$$
z _ {t + 1} = \Pi_ {\mathcal {X} \times \mathcal {Y}} \left(z _ {t} - \alpha \tilde {G} \left(z _ {t}\right)\right), \tag {10}
$$
where we let  $\tilde{G}(z_t) = (G_x(x_t, y_t), -G_y(x_t, y_t))$  for the inexact gradients.
Theorem B.7 (Restate Theorem 4.2, part (ii)). Under Assumptions 4.1. Given an inexact deterministic gradient oracle in Definition 4 with  $\delta \leq \mathcal{O}(\epsilon)$ . The average iterate  $(\bar{x}_T,\bar{y}_T)$  of GDA satisfies  $\max_{y\in \mathcal{Y}}F(\bar{x}_T,y) - \min_{x\in \mathcal{X}}F(x,\bar{y}_T)\leq \mathcal{O}(\epsilon)$  with complexity  $T = \mathcal{O}(1 / \epsilon^2)$  if setting stepsize  $\alpha = 1 / (\ell \sqrt{T})$ . Furthermore, the reproducibility is  $\| \bar{x}_T - \bar{x}_T'\|^2 +\| \bar{y}_T - \bar{y}_T'\|^2\leq \mathcal{O}(\delta^2 /\epsilon^2)$ .
Proof. We first show the optimization guarantee. By the GDA updates in (10) and Definition 4 such that  $\| \tilde{\nabla} F(z_t) - \tilde{G} (z_t)\| ^2\leq \delta^2$ , we have that for any  $z = (x,y)\in \mathcal{X}\times \mathcal{Y}$
$$
\begin{array}{l} \left\| z _ {t + 1} - z \right\| ^ {2} \leq \left\| z _ {t} - z \right\| ^ {2} - 2 \alpha \tilde {G} (z _ {t}) ^ {\top} (z _ {t} - z) + \alpha^ {2} \| \tilde {G} (z _ {t}) \| ^ {2} \\ \leq \left\| z _ {t} - z \right\| ^ {2} - 2 \alpha \tilde {\nabla} F \left(z _ {t}\right) ^ {\top} \left(z _ {t} - z\right) + 2 \alpha^ {2} \left\| \tilde {\nabla} F \left(z _ {t}\right) \right\| ^ {2} \tag {11} \\ + 2 \alpha (\tilde {\nabla} F (z _ {t}) - \tilde {G} (z _ {t})) ^ {\top} (z _ {t} - z) + 2 \alpha^ {2} \| \tilde {G} (z _ {t}) - \tilde {\nabla} F (z _ {t}) \| ^ {2} \\ \leq \left\| z _ {t} - z \right\| ^ {2} - 2 \alpha \tilde {\nabla} F (z _ {t}) ^ {\top} (z _ {t} - z) + 2 \alpha^ {2} L ^ {2} + 2 \sqrt {2} \alpha \delta D + 2 \alpha^ {2} \delta^ {2}. \\ \end{array}
$$
Taking summation from  $t = 0$  to  $T - 1$ , we obtain that
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \tilde {\nabla} F (z _ {t}) ^ {\top} (z _ {t} - z) \leq \frac {\| z _ {0} - z \| ^ {2}}{2 \alpha T} + \alpha (L ^ {2} + \delta^ {2}) + \sqrt {2} \delta D.
$$
Supposing  $\delta \leq \epsilon /(2\sqrt{2} D)$  and setting  $\alpha = 1 / (\ell \sqrt{T})$  , by Lemma B.2, this means
$$
\max  _ {y \in \mathcal {Y}} F (\bar {x} _ {T}, y) - \min  _ {x \in \mathcal {X}} F (x, \bar {y} _ {T}) \leq \frac {\ell D ^ {2} + (L ^ {2} + \delta^ {2}) / \ell}{\sqrt {T}} + \frac {\epsilon}{2}.
$$
$\epsilon$ -saddle point is guaranteed when  $T = c / \epsilon^2$  for some constant  $c \geq 4(\ell D^2 + (L^2 + \delta^2) / \ell)^2$ .
We then prove the reproducibility guarantee. Let  $\{z_t\}_{t=1}^T$  and  $\{z_t'\}_{t=1}^T$  be the trajectories of two independent runs of GDA with the same initial point  $z_0 \in \mathcal{X} \times \mathcal{Y}$  and stepsize  $\alpha > 0$ . By the GDA updates (10) and Lemma B.5, we have that
$$
\begin{array}{l} \left\| z _ {t + 1} - z _ {t + 1} ^ {\prime} \right\| \leq \left\| \left(z _ {t} - z _ {t} ^ {\prime}\right) - \alpha (\tilde {G} (z _ {t}) - \tilde {G} (z _ {t} ^ {\prime})) \right\| \\ \leq \left\| \left(z _ {t} - z _ {t} ^ {\prime}\right) - \alpha \left(\tilde {\nabla} F \left(z _ {t}\right) - \tilde {\nabla} F \left(z _ {t} ^ {\prime}\right)\right) \right\| + 2 \alpha \delta \tag {12} \\ \leq \sqrt {1 + \alpha^ {2} \ell^ {2}} \| z _ {t} - z _ {t} ^ {\prime} \| + 2 \alpha \delta . \\ \end{array}
$$
Since the initialization  $z_0 = z_0'$  is the same, we obtain that for any  $t = 1,2,\dots ,T - 1$
$$
\begin{array}{l} \left\| z _ {t} - z _ {t} ^ {\prime} \right\| \leq \left(\sqrt {1 + \alpha^ {2} \ell^ {2}}\right) ^ {t} \| z _ {0} - z _ {0} ^ {\prime} \| + 2 \alpha \delta \left(1 + \sqrt {1 + \alpha^ {2} \ell^ {2}} + \dots + \left(\sqrt {1 + \alpha^ {2} \ell^ {2}}\right) ^ {t - 1}\right) \\ = 2 \alpha \delta \sum_ {i = 0} ^ {t - 1} (1 + \alpha^ {2} \ell^ {2}) ^ {i / 2} \\ \leq 2 \alpha \delta \cdot t \left(1 + \alpha^ {2} \ell^ {2}\right) ^ {T / 2}. \\ \end{array}
$$
The above also holds for  $t = 0$  denoting  $\sum_{i=0}^{-1} = 0$ . Setting  $\alpha = 1 / (\ell \sqrt{T})$ , the reproducibility is
$$
\begin{array}{l} \left\| \bar {x} _ {T} - \bar {x} _ {T} ^ {\prime} \right\| ^ {2} + \left\| \bar {y} _ {T} - \bar {y} _ {T} ^ {\prime} \right\| ^ {2} \leq \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \left(\left\| x _ {t} - x _ {t} ^ {\prime} \right\| ^ {2} + \left\| y _ {t} - y _ {t} ^ {\prime} \right\| ^ {2}\right) \\ \leq \frac {4 \alpha^ {2} \delta^ {2}}{T} (1 + \alpha^ {2} \ell^ {2}) ^ {T} \sum_ {t = 0} ^ {T - 1} t ^ {2} \\ \leq \frac {4 e}{3 \ell^ {2}} \cdot \delta^ {2} T, \\ \end{array}
$$
which is  $\mathcal{O}(\delta^2 /\epsilon^2)$  when  $T = c / \epsilon^2$  as required in the convergence analysis of GDA.
B.2.4 Stochastic Gradient Oracle
For the stochastic minimax problem (3), with access to a stochastic gradient oracle in Definition 4, SGDA updates for  $t = 0,1,\dots ,T - 1$
$$
z _ {t + 1} = \Pi_ {\mathcal {X} \times \mathcal {Y}} \left(z _ {t} - \alpha \tilde {\nabla} f \left(z _ {t}; \xi_ {t}\right)\right), \tag {13}
$$
where  $\tilde{\nabla} f(z_{t};\xi_{t}) = (\nabla_{x}f(x_{t},y_{t};\xi_{t}), - \nabla_{y}f(x_{t},y_{t};\xi_{t}))$  and  $\{\xi_t\}_{t = 0}^{T - 1}$  are i.i.d. samples.
Proof of Theorem 5.1. We first show the convergence guarantee. By the SGDA updates in (13), given all the information up to iteration  $t$  and taking expectation with respect to  $\xi_t$ , we have  $\forall z \in \mathcal{X} \times \mathcal{Y}$
$$
\begin{array}{l} \mathbb {E} _ {\xi_ {t}} \| z _ {t + 1} - z \| ^ {2} \leq \| z _ {t} - z \| ^ {2} - 2 \alpha \mathbb {E} [ \tilde {\nabla} f (z _ {t}; \xi_ {t}) ^ {\top} (z _ {t} - z) ] + \alpha^ {2} \mathbb {E} \| \tilde {\nabla} f (z _ {t}; \xi_ {t}) \| ^ {2} \\ = \| z _ {t} - z \| ^ {2} - 2 \alpha \tilde {\nabla} F (z _ {t}) ^ {\top} (z _ {t} - z) + \alpha^ {2} \mathbb {E} \| \tilde {\nabla} f (z _ {t}; \xi_ {t}) \| ^ {2}. \\ \end{array}
$$
Taking full expectation, rearranging terms, and summing up from  $t = 0$  to  $T - 1$ , we have that
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} \left[ \tilde {\nabla} F (z _ {t}) ^ {\top} (z _ {t} - z) \right] \leq \frac {\| z _ {0} - z \| ^ {2}}{2 \alpha T} + \frac {\alpha (L ^ {2} + \delta^ {2})}{2}.
$$
Therefore, by slightly modifying the proof of Lemma B.2 through taking expectations, and then setting  $x = \arg \min_{u\in \mathcal{X}}\mathbb{E}[F(u,\bar{y}_T)]$  and  $y = \arg \max_{v\in \mathcal{Y}}\mathbb{E}[F(\bar{x}_T,v)]$ , we get
$$
\max _ {y \in \mathcal {Y}} \mathbb {E} [ F (\bar {x} _ {T}, y) ] - \min _ {x \in \mathcal {X}} \mathbb {E} [ F (x, \bar {y} _ {T}) ] \leq \frac {D ^ {2}}{\alpha T} + \frac {\alpha (L ^ {2} + \delta^ {2})}{2}.
$$
We obtain that  $\max_{y\in \mathcal{Y}}\mathbb{E}[F(\bar{x}_T,y)] - \min_{x\in \mathcal{X}}\mathbb{E}[F(x,\bar{y}_T)]\leq (\ell D^2 +(L^2 +\delta^2) / (2\ell))\epsilon$  if the inexactness  $\delta = \mathcal{O}(1)$ , and we set  $\alpha = 1 / (\ell \epsilon T),T\geq 1 / \epsilon^{2}$ .
We then show the reproducibility guarantee. For two independent runs of SGDA (13) with output  $\{z_t\}_{t=1}^T$  and  $\{z_t'\}_{t=1}^T$ , by Lemma B.5, we have that for any  $t = 0,1,\dots,T-1$ ,