SlowGuess/NeurIPS_2023_OCR huggingface dataset:
"Bilinearly-coupled minimax" example The first one is the "Bilinearly-coupled Minimax" example (3) with  $A = 10$ . It is a representative example to showcase the limit cycle phenomenon as it breaks the  $\alpha$ -dominant condition. When the bilinear intersection term between primal and dual variables, i.e.,  $x$  and  $y$ , is moderate, it becomes uncertain which variable, either the primal or dual, holds dominance over the other. As a result, this particular example poses a great challenge in terms of ensuring convergence.
"Forsaken" example The second one is the "Forsaken" example considered in [32, Example 5.2], i.e.,
$$
\min  _ {x \in \mathcal {X}} \max  _ {y \in \mathcal {Y}} x (y - 0. 4 5) + \phi (x) - \phi (y), \tag {8}
$$
where  $\phi(z) = \frac{1}{4}z^2 - \frac{1}{2}z^4 + \frac{1}{6}z^6$  and  $\mathcal{X} = \mathcal{Y} = \{z : -1.5 \leq z \leq 1.5\}$ . This example serves as a representative case, highlighting the limitations of min-max optimization algorithms. It demonstrates
Optimization problems	Function values	Optimal solutions
minF(x,y,z,v)	d(y,z,v)	x(y,z,v)
maxF(x,y,z,v)	h(x,z,v)	y(x,z,v)
minmaxF(x,y,z,v)	p(z,v)
minh(x,z,v)	p(z,v)	x(z,v) = x(y(z,v),z,v)
maxd(y,z,v)	p(z,v)	y(z,v) = y(x(z,v),z,v)
minz∈Rn p(z,v)	g(v)	z(v)
maxp(z,v)	q(z)	v(z)
Table 1: Notation
situations where standard algorithms fail to converge to the desired critical points but instead converge to spurious, non-critical points. Specifically, for problem (8), two spurious limit cycles exist across the entire domain. Worse, the one closer to the optimal solution  $[x^{\star};y^{\star}]\simeq [0.08;0.4]$  is unstable, which potentially pushes the trajectories to fall into the recurrent orbit.
"Sixth-order polynomial" example The third one is a sixth-order polynomial scaled by an exponential function. It is studied in [18, 56, 13], i.e.,
$$
\min_{x\in \mathcal{X}}\max_{y\in \mathcal{Y}}(4x^{2} - (y - 3x + 0.05x^{3})^{2} - 0.1y^{4})\exp (-0.01(x^{2} + y^{2})),
$$
where  $\mathcal{X} = \mathcal{Y} = \{z: -2 \leq z \leq 2\}$ . It is shown in [56] that existing first-order methods will suffer from limit cycles around  $[x^{*}; y^{*}] = [0; 0]$ . As far as we know, all existing convergence results rely on second-order information. Therefore, it is intriguing to investigate the potential for ensuring convergence using first-order methods.
"PolarGame" example The last example is constructed in [50, Example 3]:
$$
G (u) = \left[ \nabla_ {x} f (x, y); - \nabla_ {y} f (x, y) \right] = \left[ \Phi (x, y) - y, \Phi (y, x) + x \right],
$$
where  $u = [x; y]$  satisfy  $u \in \mathcal{X} \times \mathcal{Y}$  with  $\mathcal{X} = \mathcal{Y} = \{z : -1 \leq z \leq 1\}$  and  $\Phi(x, y) = x(-1 + x^2 + y^2)(-9 + 16x^2 + 16y^2)$ . It has been verified that there exist limit cycles at  $\|u\| = 1^2$  and  $\|u\| = \frac{3}{4}$ , which definitely make the iterates hard to converge to  $[x^*; y^*] = [0; 0]$ . To further demonstrate the effectiveness of our DS-GDA, we intentionally initialize the algorithm on the limit cycle  $\|u\| = 1$ .
C Notation and Useful Lemmas
We first list some useful notations in Table 1. In the following parts, some technical lemmas are presented. Recall that  $r_1 > L_x$  and  $r_2 > L_y$ .
Lemma 1 For any  $x, x' \in \mathcal{X}$ ,  $y, y' \in \mathcal{Y}$ ,  $z \in \mathbb{R}^n$  and  $v \in \mathbb{R}^d$ , it follows that
$$
\begin{array}{l} \frac {r _ {1} - L _ {x}}{2} \| x - x ^ {\prime} \| ^ {2} \leq F (x ^ {\prime}, y, z, v) - F (x, y, z, v) - \langle \nabla_ {x} F (x, y, z, v), x ^ {\prime} - x \rangle \leq \frac {L _ {x} + r _ {1}}{2} \| x - x ^ {\prime} \| ^ {2}, \\ - \frac {L _ {y} + r _ {2}}{2} \| y - y ^ {\prime} \| ^ {2} \leq F (x, y ^ {\prime}, z, v) - F (x, y, z, v) - \langle \nabla_ {y} F (x, y, z, v), y ^ {\prime} - y \rangle \leq \frac {L _ {y} - r _ {2}}{2} \| y - y ^ {\prime} \| ^ {2}. \\ \end{array}
$$
Proof Since  $f$  is  $L$ -smooth (from the Assumption 1), we have
$$
\begin{array}{l} - \frac {L _ {x}}{2} \| x - x ^ {\prime} \| ^ {2} \leq f \left(x ^ {\prime}, y\right) - f (x, y) - \left\langle \nabla_ {x} f (x, y), x ^ {\prime} - x \right\rangle \leq \frac {L _ {x}}{2} \| x - x ^ {\prime} \| ^ {2}, \tag {9} \\ - \frac {L _ {y}}{2} \| y - y ^ {\prime} \| ^ {2} \leq f (x, y ^ {\prime}) - f (x, y) - \langle \nabla_ {y} f (x, y), y ^ {\prime} - y \rangle \leq \frac {L _ {y}}{2} \| y - y ^ {\prime} \| ^ {2}. \\ \end{array}
$$
$\| \cdot \|$  represents the  $\ell_2$  -norm.
On the other hand, we know that
$$
\begin{array}{l} F \left(x ^ {\prime}, y, z, v\right) - F (x, y, z, v) - \langle \nabla_ {x} F (x, y, z, v), x ^ {\prime} - x \rangle \\ = f \left(x ^ {\prime}, y\right) - f (x, y) - \left\langle \nabla_ {x} f (x, y) + r _ {1} (x - z), x ^ {\prime} - x \right\rangle + \frac {r _ {1}}{2} \| x ^ {\prime} - z \| ^ {2} - \frac {r _ {1}}{2} \| x - z \| ^ {2} \tag {10} \\ = f \left(x ^ {\prime}, y\right) - f (x, y) - \left\langle \nabla_ {x} f (x, y), x ^ {\prime} - x \right\rangle + \frac {r _ {1}}{2} \| x ^ {\prime} - x \| ^ {2} \\ \end{array}
$$
$$
\begin{array}{l} F (x, y ^ {\prime}, z, v) - F (x, y, z, v) - \langle \nabla_ {y} F (x, y, z, v), y ^ {\prime} - y \rangle \\ = f (x, y ^ {\prime}) - f (x, y) - \left\langle \nabla_ {y} f (x, y) - r _ {2} (y - v), y ^ {\prime} - y \right\rangle - \frac {r _ {2}}{2} \| y ^ {\prime} - v \| ^ {2} + \frac {r _ {2}}{2} \| y - v \| ^ {2} \tag {11} \\ = f (x, y ^ {\prime}) - f (x, y) - \langle \nabla_ {y} f (x, y), y ^ {\prime} - y \rangle - \frac {r _ {2}}{2} \| y ^ {\prime} - y \| ^ {2}. \\ \end{array}
$$
Combing (9), (10) and (11), we directly obtain the desired results.
Lemma 2 (Lipschitz type error bound conditions) Suppose that  $r_2 > \left(\frac{L_y}{r_1 - L_x} + 2\right)L_y$ , then for any  $x, x' \in \mathcal{X}$ ,  $y, y' \in \mathcal{Y}$ ,  $z, z' \in \mathbb{R}^n$  and  $v, v' \in \mathbb{R}^d$ . Then the following inequalities hold:
(i)  $\| x(y', z, v) - x(y, z, v) \| \leq \sigma_1 \| y' - y \|$ ,
(ii)  $\| x(y,z',v) - x(y,z,v)\| \leq \sigma_2\| z - z'\|$
(iii)  $\| x(z', v) - x(z, v) \| \leq \sigma_2 \| z - z' \|$ ,
(iv)  $\| y(z,v) - y(z^{\prime},v)\| \leq \sigma_{3}\| z - z^{\prime}\|$
(v)  $\| y(x,z,v) - y(x',z,v)\| \leq \sigma_4\| x - x'\|$
(vi)  $\| y(x,z,v) - y(x,z,v')\| \leq \sigma_5\| v - v'\|$
(vii)  $\| y(z,v) - y(z,v^{\prime})\| \leq \sigma_{5}\| v - v^{\prime}\|$
$$
w h e r e \sigma_ {1} = \frac {L _ {y} + r _ {1} - L _ {x}}{r _ {1} - L _ {x}}, \sigma_ {2} = \frac {r _ {1}}{r _ {1} - L _ {x}}, \sigma_ {3} = \frac {r _ {1} \sigma_ {1}}{r _ {2} - L _ {y}} + \frac {\sigma_ {2}}{\sigma_ {1}}, \sigma_ {4} = \frac {L _ {x} + r _ {2} - L _ {y}}{r _ {2} - L _ {y}}, a n d \sigma_ {5} = \frac {r _ {2}}{r _ {2} - L _ {y}}.
$$
Proof (i) From Lemma 1, we know that
$$
F (x (y, z, v), y ^ {\prime}, z, v) - F (x (y ^ {\prime}, z, v), y ^ {\prime}, z, v) \geq \frac {r _ {1} - L _ {x}}{2} \| x (y, z, v) - x (y ^ {\prime}, z, v) \| ^ {2},
$$
$$
F (x (y, z, v), y ^ {\prime}, z, v) - F (x (y, z, v), y, z, v) \leq \langle \nabla_ {y} F (x (y, z, v), y, z, v), y ^ {\prime} - y \rangle + \frac {L _ {y} - r _ {2}}{2} \| y - y ^ {\prime} \| ^ {2},
$$
$$
F (x (y ^ {\prime}, z, v), y, z, v) - F (x (y ^ {\prime}, z, v), y ^ {\prime}, z, v) \leq \langle \nabla_ {y} F (x (y ^ {\prime}, z, v), y, z, v), y - y ^ {\prime} \rangle + \frac {L _ {y} + r _ {2}}{2} \| y - y ^ {\prime} \| ^ {2},
$$
$$
F (x (y, z, v), y, z, v) - F (x (y ^ {\prime}, z, v), y, z, v) \leq \frac {L _ {x} - r _ {1}}{2} \| x (y, z, v) - x (y ^ {\prime}, z, v) \| ^ {2}.
$$
Combining above inequalities, one has that
$$
\begin{array}{l} \left(r _ {1} - L _ {x}\right) \left\| x (y, z, v) - x \left(y ^ {\prime}, z, v\right) \right\| ^ {2} \\ \leq \left\langle \nabla_ {y} F (x (y, z, v), y, z, v) - \nabla_ {y} F (x (y ^ {\prime}, z, v), y, z, v), y ^ {\prime} - y \right\rangle + L _ {y} \| y - y ^ {\prime} \| ^ {2} \\ \leq L _ {y} \| x (y ^ {\prime}, z, v) - x (y, z, v) \| \| y ^ {\prime} - y \| + L _ {y} \| y - y ^ {\prime} \| ^ {2}, \\ \end{array}
$$
where the second inequality is from Cauchy-Schwarz inequality and  $L$ -smooth property. Let  $\xi \coloneqq \| x(y', z, v) - x(y, z, v) \| / \| y - y' \|$ . Then it follows that
$$
\xi^ {2} \leq \frac {L _ {y}}{r _ {1} - L _ {x}} + \frac {L _ {y}}{r _ {1} - L _ {x}} \xi .
$$
Consequently, utilizing AM-GM inequality we derive  $\xi \leq \frac{\sqrt{L_y^2 + 2r_1L_y - 2L_xL_y}}{r_1 - L_x} \leq \frac{L_y + r_1 - L_x}{r_1 - L_x} \coloneqq \sigma_1$ . (ii-iii) Again from Lemma 1, we know that
$$
\begin{array}{l} F (x (y, z, v), y, z ^ {\prime}, v) - F (x (y, z ^ {\prime}, v), y, z ^ {\prime}, v) \geq \frac {r _ {1} - L _ {x}}{2} \| x (y, z, v) - x (y, z ^ {\prime}, v) \| ^ {2}, \tag {12} \\ F (x (y, z, v), y, z, v) - F (x (y, z ^ {\prime}, v), y, z, v) \leq \frac {L _ {x} - r _ {1}}{2} \| x (y, z, v) - x (y, z ^ {\prime}, v) \| ^ {2}, \\ \end{array}
$$
From the definition of  $F$ , we know that
$$
F (x (y, z, v), y, z ^ {\prime}, v) - F (x (y, z, v), y, z, v) = \frac {r _ {1}}{2} \left\langle z ^ {\prime} + z - 2 x (y, z, v), z ^ {\prime} - z \right\rangle , \tag {13}
$$
$$
F (x (y, z ^ {\prime}, v), y, z, v) - F (x (y, z ^ {\prime}, v), y, z ^ {\prime}, v) = \frac {r _ {1}}{2} \langle z + z ^ {\prime} - 2 x (y, z ^ {\prime}, v), z - z ^ {\prime} \rangle .
$$
Incorporating (12), (13) and using the Cauchy-Schwarz inequality, we have
$$
\begin{array}{l} \left(r _ {1} - L _ {x}\right) \left\| x (y, z, v) - x \left(y, z ^ {\prime}, v\right) \right\| ^ {2} \leq r _ {1} \langle x \left(y, z ^ {\prime}, v\right) - x (y, z, v), z ^ {\prime} - z \rangle \\ \leq r _ {1} \| x (y, z ^ {\prime}, v) - x (y, z, v) \| \| z ^ {\prime} - z \|, \\ \end{array}
$$
which completes the proof of (ii). Moreover, since  $\max_{y\in \mathcal{Y}}F(\cdot ,y,\cdot ,\cdot)$  is  $(r_1 - L_x)$ -strongly convex in  $x$ , the similar argument leads to (iii).
(iv) We will now proceed to prove inequality (iv). From Lemma 1, we know that
$$
\begin{array}{l} d (y (z, v), z, v) - d (y \left(z ^ {\prime}, v\right), z, v) \geq \frac {r _ {2} - L _ {y}}{2} \| y (z, v) - y \left(z ^ {\prime}, v\right) \| ^ {2}, \\ d (y (z, v), z ^ {\prime}, v) - d (y (z ^ {\prime}, v), z ^ {\prime}, v) \leq \frac {L _ {y} - r _ {2}}{2} \| y (z, v) - y (z ^ {\prime}, v) \| ^ {2}. \\ \end{array}
$$
$$
\begin{array}{l} d (y (z, v), z, v) - d (y (z, v), z ^ {\prime}, v) \\ \leq F (x (y (z, v), z ^ {\prime}, v), y (z, v), z, v) - F (x (y (z, v), z ^ {\prime}, v), y (z, v), z ^ {\prime}, v) \\ = \frac {r _ {1}}{2} \left\langle z + z ^ {\prime} - 2 x (y (z, v), z ^ {\prime}, v), z - z ^ {\prime} \right\rangle \\ \end{array}
$$
$$
\begin{array}{l} d (y \left(z ^ {\prime}, v\right), z ^ {\prime}, v) - d (y \left(z ^ {\prime}, v\right), z, v) \\ \leq F (x (y \left(z ^ {\prime}, v\right), z, v), y \left(z ^ {\prime}, v\right), z ^ {\prime}, v) - F (x (y \left(z ^ {\prime}, v\right), z, v), y \left(z ^ {\prime}, v\right), z, v) \\ = \frac {r _ {1}}{2} \langle z ^ {\prime} + z - 2 x (y (z ^ {\prime}, v), z, v), z ^ {\prime} - z \rangle . \\ \end{array}
$$
Armed with these inequalities, we conclude that
$$
\begin{array}{l} \left(r _ {2} - L _ {y}\right) \left\| y (z, v) - y \left(z ^ {\prime}, v\right) \right\| ^ {2} \leq r _ {1} \left\langle x \left(y \left(z ^ {\prime}, v\right), z, v\right) - x \left(y (z, v), z ^ {\prime}, v\right), z - z ^ {\prime} \right\rangle \\ \leq r _ {1} \| z - z ^ {\prime} \| \left(\sigma_ {1} \| y (z, v) - y \left(z ^ {\prime}, v\right) \| + \sigma_ {2} \| z - z ^ {\prime} \|\right) \\ \leq r _ {1} \sigma_ {1} \| z - z ^ {\prime} \| \| y (z, v) - y \left(z ^ {\prime}, v\right) \| + r _ {1} \sigma_ {2} \| z - z ^ {\prime} \| ^ {2}, \\ \end{array}
$$
where the last inequality is from error bounds (i) and (ii). Let  $\xi := \frac{\|y(z,v) - y(z',v)\|}{\|z - z'\|}$ , we have  $\xi^2 \leq \frac{r_1\sigma_1}{r_2 - L_y}\xi + \frac{r_1\sigma_2}{r_2 - L_y}$ . Using AM-GM inequality, we obtain
$$
\frac {\| y (z , v) - y (z ^ {\prime} , v) \|}{\| z - z ^ {\prime} \|} \leq \frac {r _ {1} \sigma_ {1}}{r _ {2} - L _ {y}} + \frac {\sigma_ {2}}{\sigma_ {1}} := \sigma_ {3}.
$$
Next, we consider (v). Still from Lemma 1, we have the following inequalities:
$$
F (x, y (x, z, v), z, v) - F (x, y (x ^ {\prime}, z, v), z, v) \geq \frac {r _ {2} - L _ {y}}{2} \| y (x, z, v) - y (x ^ {\prime}, z, v) \| ^ {2},
$$
$$
F (x ^ {\prime}, y (x, z, v), z, v) - F (x ^ {\prime}, y (x ^ {\prime}, z, v), z, v) \leq \frac {L _ {y} - r _ {2}}{2} \| y (x, z, v) - y (x ^ {\prime}, z, v) \| ^ {2},
$$
$$
F (x, y (x, z, v), z, v) - F (x ^ {\prime}, y (x, z, v), z, v) \leq \langle \nabla_ {x} F (x ^ {\prime}, y (x, z, v), z, v), x - x ^ {\prime} \rangle + \frac {L _ {x} + r _ {1}}{2} \| x - x ^ {\prime} \| ^ {2},
$$
$$
F (x ^ {\prime}, y (x ^ {\prime}, z, v), z, v) - F (x, y (x ^ {\prime}, z, v), z, v) \leq \langle \nabla_ {x} F (x ^ {\prime}, y (x ^ {\prime}, z, v), z, v), x ^ {\prime} - x \rangle + \frac {L _ {x} - r _ {1}}{2} \| x - x ^ {\prime} \| ^ {2}.
$$
Summing them up, we derive that
$$
\left(r _ {2} - L _ {y}\right) \left\| y (x, z, v) - y \left(x ^ {\prime}, z, v\right) \right\| ^ {2} \leq L _ {x} \left\| x - x ^ {\prime} \right\| ^ {2} + L _ {x} \left\| x - x ^ {\prime} \right\| \left\| y (x, z, v) - y \left(x ^ {\prime}, z, v\right) \right\|.
$$
Let  $\xi := \frac{\|y(x,z,v) - y(x',z,v)\|}{\|x - x'\|}$ . Then, we have  $\xi^2 \leq \frac{L_x}{r_2 - L_y} + \frac{L_x}{r_2 - L_y} \xi$  and consequently  $\xi \leq \frac{L_x + r_2 - L_y}{r_2 - L_y} = \sigma_4$ .
(vi-viii) From the definition of  $F$ , we get
$$
F (x, y (x, z, v), z, v) - F (x, y (x, z, v), z, v ^ {\prime}) = \frac {r _ {2}}{2} \langle v ^ {\prime} + v - 2 y (x, z, v), v ^ {\prime} - v \rangle ,
$$
$$
F (x, y (x, z, v ^ {\prime}), z, v ^ {\prime}) - F (x, y (x, z, v ^ {\prime}), z, v) = \frac {r _ {2}}{2} \langle v + v ^ {\prime} - 2 y (x, z, v ^ {\prime}), v - v ^ {\prime} \rangle .
$$
Moreover, by the strong concavity of  $F(x,\cdot ,z,v)$ , we have
$$
F (x, y (x, z, v), z, v) - F (x, y (x, z, v ^ {\prime}), z, v) \geq \frac {r _ {2} - L _ {y}}{2} \| y (x, z, v) - y (x, z, v ^ {\prime}) \| ^ {2},
$$
$$
F (x, y (x, z, v), z, v ^ {\prime}) - F (x, y (x, z, v ^ {\prime}), z, v ^ {\prime}) \leq \frac {L _ {y} - r _ {2}}{2} \| y (x, z, v) - y (x, z, v ^ {\prime}) \| ^ {2}.
$$
Armed with these inequalities and Cauchy-Schwarz inequality, we conclude that
$$
\begin{array}{l} \left(r _ {2} - L _ {y}\right) \left\| y (x, z, v) - y (x, z, v ^ {\prime}) \right\| ^ {2} \leq r _ {2} \langle y (x, z, v) ^ {\prime} - y (x, z, v), v ^ {\prime} - v \rangle \\ \leq r _ {2} \| y (x, z, v ^ {\prime}) - y (x, z, v) \| \| v - v ^ {\prime} \|, \\ \end{array}
$$
which gives the inequality (vi). Since  $d(\cdot, z, v) = \min_{x \in \mathcal{X}} F(x, \cdot, z, v)$  is  $(r_2 - L_y)$ -strongly concave, similarly we can derive the Lipschitz property of  $y(z, v)$ , as shown in (vii).
Lemma 3 (L-smooth property of dual function) For any fixed  $z \in \mathbb{R}^n$ ,  $v \in \mathbb{R}^d$ , the dual function  $d(\cdot, z, v)$  is continuously differentiable with the gradient  $\nabla_y d(y, z, v) = \nabla_y F(x(y, z, v), y, z, v)$  and
$$
\left\| \nabla_ {y} d (y, z, v) - \nabla_ {y} d \left(y ^ {\prime}, z, v\right) \right\| \leq L _ {d} \| y - y ^ {\prime} \|,
$$
where  $L_{d}\coloneqq L_{y}\sigma_{1} + L_{y} + r_{2}$
Proof Using Danskin's theorem, we know that  $d(\cdot, z, v)$  is differentiable with  $\nabla_y d(y, z, v) = \nabla_y F(x(y, z, v), y, z, v)$ . Also, we know from the  $L$ -smooth property of  $f$  that
$$
\begin{array}{l} \| \nabla_ {y} d (y, z, v) - \nabla_ {y} d \left(y ^ {\prime}, z, v\right) \| = \| \nabla_ {y} F (x (y, z, v), y, z, v) - \nabla_ {y} F (x \left(y ^ {\prime}, z, v\right), y ^ {\prime}, z, v) \| \\ \leq \left\| \nabla_ {y} F (x (y, z, v), y, z, v) - \nabla_ {y} F (x \left(y ^ {\prime}, z, v\right), y, z, v) \right\| + \\ \| \nabla_ {y} F (x (y ^ {\prime}, z, v), y, z, v) - \nabla_ {y} F (x (y ^ {\prime}, z, v), y ^ {\prime}, z, v) \| \\ \leq L _ {y} \| x \left(y ^ {\prime}, z, v\right) - x (y, z, v) \| + \left(L _ {y} + r _ {2}\right) \| y - y ^ {\prime} \| \\ \leq \left(L _ {y} \sigma_ {1} + L _ {y} + r _ {2}\right) \| y - y ^ {\prime} \| = L _ {d} \| y - y ^ {\prime} \|, \\ \end{array}
$$
where the last inequality is due to the error bound in Lemma 2 (i).
Recall that  $y_{+}(z,v) = \mathrm{proj}_{\mathcal{Y}}(y + \alpha \nabla_yF(x(y,z,v),y,z,v))$ . Incorporating the iterates of DS-GDA, we have the following error bounds:
Lemma 4 For any  $t \geq 0$ , the following inequalities hold:
(i)  $\| x^{t + 1} - x(y^t,z^t,v^t)\| \leq \sigma_6\| x^{t + 1} - x^t\|$
(ii)  $\| y^{t + 1} - y(x^{t + 1},z^t,v^t)\| \leq \sigma_7\| y^{t + 1} - y^t\| ,$
(iii)  $\| y(z^t,v^t) - y^t\| \leq \sigma_8\| y^t -y_{+}^t (z^t,v^t)\|$
(iv)  $\| y^{t + 1} - y_{+}^{t}(z^{t},v^{t})\| \leq L_{y}\alpha \sigma_{6}\| x^{t} - x^{t + 1}\|$
where  $\sigma_{6} = \frac{2cr_{1} + 1}{cr_{1} - cL_{x}}$ ,  $\sigma_{7} = \frac{2\alpha r_{2} + 1}{\alpha r_{2} - \alpha L_{y}}$ , and  $\sigma_{8} = \frac{1 + \alpha L_{d}}{\alpha(r_{2} - L_{y})}$ .
Proof (i-iii) First, we consider (i), which is also called "primal error bound". From Lemma 1, we know that the mapping  $\nabla_x F(\cdot, y, z, v)$  is  $(r_1 - L_x)$ -strongly monotone and Lipschitz continuous with constant  $(r_1 + L_x)$  on set  $\mathcal{X}$  for all  $y \in \mathcal{Y}, z \in \mathbb{R}^n, v \in \mathbb{R}^d$ . Adopting the proof in [49, Theorem 3.1], we can easily derive that
$$
\left\| x ^ {t} - x \left(y ^ {t}, z ^ {t}, v ^ {t}\right) \right\| \leq \frac {c L _ {x} + c r _ {1} + 1}{c r _ {1} - c L _ {x}} \left\| x ^ {t + 1} - x ^ {t} \right\|,
$$
$$
\| x ^ {t + 1} - x (y ^ {t}, z ^ {t}, v ^ {t}) \| \leq \| x ^ {t + 1} - x ^ {t} \| + \| x ^ {t} - x (y ^ {t}, z ^ {t}, v ^ {t}) \| \leq \frac {2 c r _ {1} + 1}{c r _ {1} - c L _ {x}} \| x ^ {t + 1} - x ^ {t} \|.
$$
Similarly, since  $-\nabla_y F(x,\cdot ,z,v)$  is  $(r_2 - L_y)$ -strongly monotone and Lipschitz continuous with constant  $(r_2 + L_y)$  on  $\mathcal{V}$  for all  $x\in \mathcal{X},z\in \mathbb{R}^n,v\in \mathbb{R}^d$  we can derive the "primal error bound" for  $y_{t}$  in the inequality (ii). As for (iii), notice that  $y(z,v) = \operatorname{argmax}_{y\in \mathcal{Y}}d(y,z,v)$ , the "primal error bound" is defined with operator  $\nabla_y d(\cdot ,z,v)$ . Since  $d(\cdot ,z,v) = \min_{x\in \mathcal{X}}F(x,\cdot ,z,v)$  is  $(r_2 - L_y)$ -strongly concave, the operator  $-\nabla_y d(\cdot ,z,v)$  is  $(r_2 - L_y)$ -strongly monotone. Moreover, from the Lemma 3, we find  $\nabla_y d(\cdot ,z,v)$  is  $L_{d}$ -Lipschitz continuous on  $\mathcal{V}$  for all  $z\in \mathbb{R}^n,v\in \mathbb{R}^d$ . Thus, (iii) can be derived correspondingly.
(iv) Utilizing the inequality (i), we can further bound the desired term
$$
\begin{array}{l} \left\| y ^ {t + 1} - y _ {+} ^ {t} \left(z ^ {t}, v ^ {t}\right) \right\| \\ = \| \operatorname {p r o j} _ {\mathcal {Y}} \left(y ^ {t} + \alpha \nabla_ {y} F \left(x ^ {t + 1}, y ^ {t}, z ^ {t}, v ^ {t}\right)\right) - \operatorname {p r o j} _ {\mathcal {Y}} \left(y ^ {t} + \alpha \nabla_ {y} F \left(x \left(y ^ {t}, z ^ {t}, v ^ {t}\right), y ^ {t}, z ^ {t}, v ^ {t}\right)\right) \| \\ \leq \alpha \| \nabla_ {y} F (x ^ {t + 1}, y ^ {t}, z ^ {t}, v ^ {t}) - \nabla_ {y} F (x (y ^ {t}, z ^ {t}, v ^ {t}), y ^ {t}, z ^ {t}, v ^ {t}) \| \\ \leq \alpha L _ {y} \| x ^ {t + 1} - x (y ^ {t}, z ^ {t}, v ^ {t}) \| \leq L _ {y} \alpha \sigma_ {6} \| x ^ {t} - x ^ {t + 1} \|, \\ \end{array}
$$
where the first inequality follows from the non-expansivity of the projection operator.
D Basic Descent Lemmas
Lemma 5 (Primal descent) For any  $t \geq 0$ , the following inequality holds:
$$
\begin{array}{l} F (x ^ {t}, y ^ {t}, z ^ {t}, v ^ {t}) \geq F (x ^ {t + 1}, y ^ {t + 1}, z ^ {t + 1}, v ^ {t + 1}) + \left(\frac {1}{c} - \frac {L _ {x} + r _ {1}}{2}\right) \| x ^ {t + 1} - x ^ {t} \| ^ {2} + \\ \langle \nabla_ {y} F (x ^ {t + 1}, y ^ {t}, z ^ {t}, v ^ {t}), y ^ {t} - y ^ {t + 1} \rangle + \frac {r _ {2} - L _ {y}}{2} \| y ^ {t + 1} - y ^ {t} \| ^ {2} + \\ \frac {2 - \beta}{2 \beta} r _ {1} \| z ^ {t + 1} - z ^ {t} \| ^ {2} + \frac {\mu - 2}{2 \mu} r _ {2} \| v ^ {t + 1} - v ^ {t} \| ^ {2}. \\ \end{array}
$$
Proof We firstly split the target into four parts as follows:
$$
\begin{array}{l} F \left(x ^ {t}, y ^ {t}, z ^ {t}, v ^ {t}\right) - F \left(x ^ {t + 1}, y ^ {t + 1}, z ^ {t + 1}, v ^ {t + 1}\right) \\ = \underbrace {F (x ^ {t} , y ^ {t} , z ^ {t} , v ^ {t}) - F (x ^ {t + 1} , y ^ {t} , z ^ {t} , v ^ {t})} _ {①} + \underbrace {F (x ^ {t + 1} , y ^ {t} , z ^ {t} , v ^ {t}) - F (x ^ {t + 1} , y ^ {t + 1} , z ^ {t} , v ^ {t})} _ {②} + \\ \underbrace {F (x ^ {t + 1} , y ^ {t + 1} , z ^ {t} , v ^ {t}) - F (x ^ {t + 1} , y ^ {t + 1} , z ^ {t + 1} , v ^ {t})} _ {\text {③}} + \underbrace {F (x ^ {t + 1} , y ^ {t + 1} , z ^ {t + 1} , v ^ {t}) - F (x ^ {t + 1} , y ^ {t + 1} , z ^ {t + 1} , v ^ {t + 1})} _ {\text {④}}. \\ \end{array}
$$
As for ①, we have
$$
\begin{array}{l} F (x ^ {t}, y ^ {t}, z ^ {t}, v ^ {t}) - F (x ^ {t + 1}, y ^ {t}, z ^ {t}, v ^ {t}) \geq \langle \nabla_ {x} F (x ^ {t}, y ^ {t}, z ^ {t}, v ^ {t}), x ^ {t} - x ^ {t + 1} \rangle - \frac {L _ {x} + r _ {1}}{2} \| x ^ {t + 1} - x ^ {t} \| ^ {2} \\ \geq \left(\frac {1}{c} - \frac {L _ {x} + r _ {1}}{2}\right) \| x ^ {t + 1} - x ^ {t} \| ^ {2}, \\ \end{array}
$$
where the first inequality is from Lemma 1 and the second is due to the projection update of  $x^{t + 1}$ , i.e.,  $\langle x^t - c\nabla_xF(x^t, y^t, z^t, v^t) - x^{t + 1}, x^t - x^{t + 1} \rangle \leq 0$ . Next, from Lemma 1, one has for the inequality (2) that
$$
F (x ^ {t + 1}, y ^ {t}, z ^ {t}, v ^ {t}) - F (x ^ {t + 1}, y ^ {t + 1}, z ^ {t}, v ^ {t}) \geq \langle \nabla_ {y} F (x ^ {t + 1}, y ^ {t}, z ^ {t}, v ^ {t}), y ^ {t} - y ^ {t + 1} \rangle + \frac {r _ {2} - L _ {y}}{2} \| y ^ {t + 1} - y ^ {t} \| ^ {2}.
$$
For ③, on top of the update of  $z^{t+1}$ , i.e.,  $z^{t+1} = z^t + \beta (x^{t+1} - z^t)$ , we obtain
$$
F (x ^ {t + 1}, y ^ {t + 1}, z ^ {t}, v ^ {t}) - F (x ^ {t + 1}, y ^ {t + 1}, z ^ {t + 1}, v ^ {t}) = \frac {2 - \beta}{2 \beta} r _ {1} \| z ^ {t + 1} - z ^ {t} \| ^ {2}.
$$
Similarly, as for  $④$ , following the update of  $v^{t + 1}$ , i.e,  $v^{t + 1} = v^t + \mu (y^{t + 1} - v^t)$ , we can verify that
$$
F (x ^ {t + 1}, y ^ {t + 1}, z ^ {t + 1}, v ^ {t}) - F (x ^ {t + 1}, y ^ {t + 1}, z ^ {t + 1}, v ^ {t + 1}) = \frac {\mu - 2}{2 \mu} r _ {2} \| v ^ {t + 1} - v ^ {t} \| ^ {2}.
$$
Combining all the above bounds leads to the conclusion.
Lemma 6 (Dual ascent) For any  $t \geq 0$ , the following inequality holds:
$$
\begin{array}{l} d (y ^ {t + 1}, z ^ {t + 1}, v ^ {t + 1}) \geq d (y ^ {t}, z ^ {t}, v ^ {t}) + \frac {(2 - \mu) r _ {2}}{2 \mu} \| v ^ {t + 1} - v ^ {t} \| ^ {2} + \\ \frac {r _ {1}}{2} \left\langle z ^ {t + 1} + z ^ {t} - 2 x \left(y ^ {t + 1}, z ^ {t + 1}, v ^ {t}\right), z ^ {t + 1} - z ^ {t} \right\rangle + \\ \langle \nabla_ {y} F (x (y ^ {t}, z ^ {t}, v ^ {t}), y ^ {t}, z ^ {t}, v ^ {t}), y ^ {t + 1} - y ^ {t} \rangle - \frac {L _ {d}}{2} \| y ^ {t + 1} - y ^ {t} \| ^ {2}. \\ \end{array}
$$
Proof The difference of the update for the dual function is controlled by the following three parts:
$$
\begin{array}{l} d \left(y ^ {t + 1}, z ^ {t + 1}, v ^ {t + 1}\right) - d \left(y ^ {t}, z ^ {t}, v ^ {t}\right) \\ = \underbrace {d (y ^ {t + 1} , z ^ {t + 1} , v ^ {t + 1}) - d (y ^ {t + 1} , z ^ {t + 1} , v ^ {t})} _ {\text {①}} + \underbrace {d (y ^ {t + 1} , z ^ {t + 1} , v ^ {t}) - d (y ^ {t + 1} , z ^ {t} , v ^ {t})} _ {\text {②}} + \\ \underbrace {d (y ^ {t + 1} , z ^ {t} , v ^ {t}) - d (y ^ {t} , z ^ {t} , v ^ {t})} _ {\text {③}}. \\ \end{array}
$$
For the first part, following from the update of  $v^{t + 1}$ , we have
$$
\circledast = \frac {r _ {2}}{2} \left(\| y ^ {t + 1} - v ^ {t} \| ^ {2} - \| y ^ {t + 1} - v ^ {t + 1} \| ^ {2}\right) = \frac {(2 - \mu) r _ {2}}{2 \mu} \| v ^ {t + 1} - v ^ {t} \| ^ {2}.
$$