SlowGuess/NeurIPS_2023_OCR huggingface dataset:
we specialize the notation  $\mathcal{X} \coloneqq [-1, 1]^n$ , and for any  $v \in \mathbb{R}^n$  we let  $\Pi_{\mathcal{X}}(v) \coloneqq \mathrm{med}(v, -1, 1)$  be truncation applied entrywise. We also use  $\mathcal{Y} \coloneqq \Delta^d$  and for any  $v \in \mathbb{R}_{\geq 0}^d$  we let  $\Pi_{\mathcal{Y}}(v) \coloneqq v / \|v\|_1$  normalize onto  $\mathcal{Y}$ . We let  $\mathcal{Z} \coloneqq \mathcal{X} \times \mathcal{Y}$ , and define the gradient operator of (1):
$$
g (x, y) := \left(\mathbf {A} y + c, b - \mathbf {A} ^ {\top} x\right). \tag {5}
$$
We refer to the  $x$  and  $y$  components of  $g(x,y)$  by  $g^{\times}(x,y) \coloneqq \mathbf{A}y + c$  and  $g^{\mathcal{V}}(x,y) \coloneqq b - \mathbf{A}^{\top}x$ . Finally, to minimize clutter we denote the Bregman divergence in  $r_{\mathbf{A}}^{(\alpha)}$  by  $V^{(\alpha)} \coloneqq V^{r_{\mathbf{A}}^{(\alpha)}}$ .
1. Section 2.1 presents our unification of the convergence analysis in [She17] with more standard analyses, by way of interpreting area convexity as Bregman domination (6).
2. Section 2.2 presents several helper results from convex analysis which demonstrates how area convexity implies relative smoothness of the subproblems encountered by [She17], leading to simpler subproblem solvers. This also gives way to developments which generalize to the matrix setting (and are crucial for our proof of Theorem 2).
Ultimately, our proof of Theorem 1 in Section 3 will build upon both of these sets of insights.
2.1 Area convexity as Bregman domination
In Appendix B of the supplement, we analyze extragradient algorithms for approximately solving variational inequalities in an operator  $g$ , i.e. which find  $z$  with small  $\langle g(z), z - u \rangle$  for all  $u$  in the domain (in the minimax optimization setting where  $g$  is taken to be the gradient operator, e.g. the operator in (5), this corresponds to duality gap). Specifically, we analyze convergence under the following condition, weaker than previous notions in [She17, CST21].
Definition 1 (Relaxed relative Lipschitzness). We say an operator  $g: \mathcal{Z} \to \mathcal{Z}^*$  is  $\frac{1}{\eta}$ -relaxed relatively Lipschitz with respect to  $r: \mathcal{Z} \to \mathbb{R}$  if for all  $(z, z', z^+) \in \mathcal{Z} \times \mathcal{Z} \times \mathcal{Z}$ ,
$$
\eta \left\langle g (z ^ {\prime}) - g (z), z ^ {\prime} - z ^ {+} \right\rangle \leq V _ {z} ^ {r} (z ^ {\prime}) + V _ {z ^ {\prime}} ^ {r} (z ^ {+}) + V _ {z} ^ {r} (z ^ {+}).
$$
This notion is related to and subsumes the notions of relative Lipschitzness [CST21] and area convexity [She17] which have recently been proposed to analyze extragradient methods, which we define below. In particular, Definitions 2 and 3 were motivated by designing solvers for (1).
Definition 2 (Relative Lipschitzness [CST21]). We say an operator  $g: \mathcal{Z} \to \mathcal{Z}^*$  is  $\frac{1}{\eta}$ -relatively Lipschitz with respect to  $r: \mathcal{Z} \to \mathbb{R}$  if for all  $(z, z', z^+) \in \mathcal{Z} \times \mathcal{Z} \times \mathcal{Z}$ ,
$$
\eta \left\langle g (z ^ {\prime}) - g (z), z ^ {\prime} - z ^ {+} \right\rangle \leq V _ {z} ^ {r} (z ^ {\prime}) + V _ {z ^ {\prime}} ^ {r} (z ^ {+}).
$$
Definition 3 (Area convexity [She17]). We say convex  $r: \mathcal{Z} \to \mathbb{R}$  is  $\eta$ -area convex with respect to an operator  $g: \mathcal{Z} \to \mathcal{Z}^*$  if for all  $(z, z', z^{+}) \in \mathcal{Z} \times \mathcal{Z} \times \mathcal{Z}$ , defining  $c := \frac{1}{3}(z + z' + z^{+})$ ,
$$
\eta \left\langle g \left(z ^ {\prime}\right) - g (z), z ^ {\prime} - z ^ {+} \right\rangle \leq r (z) + r \left(z ^ {\prime}\right) + r \left(z ^ {+}\right) - 3 r (c).
$$
Area convexity is monotone in  $\eta$  as the right-hand side is nonnegative for any  $z, z', z^+$ .
Relaxed relative Lipschitzness simultaneously generalizes relative Lipschitzness and area convexity. The relationship is obvious for Definition 2, but the generalization of Definition 3 relies on the following simple observation, which has not previously appeared explicitly to our knowledge:
$$
r (z) + r \left(z ^ {\prime}\right) + r \left(z ^ {+}\right) - 3 r (c) = V _ {z} ^ {r} \left(z ^ {+}\right) + V _ {z} ^ {r} \left(z ^ {\prime}\right) - 3 V _ {z} ^ {r} (c) \leq V _ {z} ^ {r} \left(z ^ {+}\right) + V _ {z} ^ {r} \left(z ^ {\prime}\right). \tag {6}
$$
In Appendix B of the supplement we give a framework unifying previous analyses of [Nem04, Nes07, She17, CST21], possibly of further utility. The realization (6) that area convexity implies a Bregman divergence bound lets us reinterpret the [She17] outer loop to tolerate larger amounts of inexactness in the subproblems, where inexactness is measured in Bregman divergence. We combine our improved outer loop analysis with tools leveraging relative smoothness (to be discussed) to remove alternating minimization from [She17]. We will use the following known fact.
Fact 1 (Lemma 2, [CST21]). For convex functions  $f, r$ , if  $f$  is  $L$ -relatively smooth with respect to  $r$ , then  $\partial f$  is  $L$ -relatively Lipschitz with respect to  $r$ .
We also summarize some useful properties of  $r_{\mathbf{A}}^{(\alpha)}$  adapted from [JST19].
Lemma 1. For  $\alpha \geq \frac{1}{2}$ ,  $r_{\mathbf{A}}^{(\alpha)}$  is jointly convex; for  $\alpha \geq 2$ ,  $r_{\mathbf{A}}^{(\alpha)}$  is  $\frac{1}{3}$ -area convex with respect to  $g$  (5).
2.2 Area convexity as relative smoothness
In [She17], the second property in Lemma 1 was used with an ad hoc outer loop analysis (unified with existing analyses in Appendix B of the supplement) to prove convergence. However, it remains to discuss how to implement steps of the outer loop method, each of which solves a regularized subproblem of the form, for  $(g^{\mathsf{x}}, g^{\mathsf{y}}) \in \mathcal{X}^{*} \times \mathcal{Y}^{*}$  (and recalling our regularizer (4))
$$
\min  _ {x \in [ - 1, 1 ] ^ {n}, y \in \Delta^ {d}} F (x, y) := \langle g ^ {\times}, x \rangle + \langle g ^ {\Upsilon}, y \rangle + \left\langle | \mathbf {A} | y, x ^ {2} \right\rangle + \alpha h (y).
$$
These subproblem forms are standard for mirror descent-based methods (including extragradient methods). In contrast to schemes using separable regularizers (admitting closed-form solutions), the subproblems induced by (4) themselves require an iterative method to solve. In [She17], a linearly-convergent minimization subroutine for  $F$  was given based on multiplicative stability of simplex variables, using a fairly nonstandard analysis (see Lemma 6, [JST19]). Our next observation is that a linearly-convergent subproblem solver follows off-the-shelf from [LFN18] and viewing area convexity under the lens of relative smoothness, a more standard condition in optimization theory. We use the following convex analysis facts, deferring proofs to Section 3.2 of the supplement.
Lemma 2. Let  $\mathcal{X} \subseteq \mathbb{R}^m$  and  $\mathcal{Y} \subseteq \mathbb{R}^n$  be convex compact subsets. Suppose  $F: \mathcal{X} \times \mathcal{Y} \to \mathbb{R}$  is jointly convex over its argument  $(x, y) \in \mathcal{X} \times \mathcal{Y}$ . For  $y \in \mathcal{Y}$ , define  $x_{\mathrm{br}}(y) := \operatorname*{argmin}_{x \in \mathcal{X}} F(x, y)$  and  $f(y) := F(x_{\mathrm{br}}(y), y)$ . Then for all  $y \in \mathcal{Y}$ ,  $\partial_y F(x_{\mathrm{br}}(y), y) \subset \partial f(y)$ .
Lemma 3. In the setting of Lemma 2, suppose for any  $x \in \mathcal{X}$ ,  $F(x, \cdot)$  (as a function over  $\mathcal{Y}$ ) always is  $r: \mathcal{Y} \to \mathbb{R}$  plus a linear function (where the linear function may depend on  $x$ ). Then  $r - f$  is convex, and  $f - q$  is convex for any  $q: \mathcal{Y} \to \mathbb{R}$  such that  $F - q: \mathcal{X} \times \mathcal{Y} \to \mathbb{R}$  is jointly convex.
Recall from Lemma 1 that  $F$  is jointly convex over  $(x,y)$  for any  $\alpha \geq \frac{1}{2}$ . Hence, for  $\alpha = 2$ , we may apply Lemma 3 with  $r(y) \coloneqq 2h(y)$  to conclude that  $f(y) \coloneqq \min_{x \in [-1,1]^n} F(x,y)$  is 2-relatively smooth with respect to  $h$ , as a function over  $\mathcal{V} = \Delta^d$ . Moreover, applying Lemma 3 with  $q(y) \coloneqq h(y)$ , and again using the joint convexity fact in Lemma 1, shows that  $f$  is further 1-relatively strongly convex with respect to  $h$ . At this point, a direct application of Theorem 3.1 in [LFN18] (which gives an algorithm for optimization under relative smoothness and strong convexity) yields a linearly-convergent algorithm for minimizing  $F$ . We remark that in light of Lemma 2, we can implement gradient queries to  $f$  by computing the best response argument for a given  $y$ .
Interestingly, this argument used nothing more than Lemma 3, joint convexity of  $r_{\mathbf{A}}^{(\alpha)}$ , and the ability to tune  $\alpha$  to induce relative strong convexity (in contrast to [She17], which requires multiplicative stability properties). An important consequence is that the same technique generalizes to the matrix setting via new joint convexity facts we prove in Section 4 of the supplement, where multiplicative stability breaks due to non-monotonicity of matrix exponentials; the corresponding subproblem analysis in [She17] hence does not apply. This gives a simple proof-of-concept matching Theorem 2 up to logarithmic factors, which we improve via our approximation-tolerant extragradient methods.
3 Box-simplex games without alternating minimization
In this section, we show how to combine the insights regarding area convexity from Section 2, along with a careful implementation of our inner loop subproblem solvers, to prove Theorem 1.
3.1 Approximation-tolerant extragradient method
We begin by stating two oracles whose guarantees, when combined with Definition 3, give our conceptual algorithm for (1). These oracles can be viewed as approximately implementing steps of the extragradient method framework in Appendix B of the supplement. We develop subroutines which satisfy these relaxed definitions in the following Section 3.2. In the following definitions (and throughout), we let  $V^{(\alpha)} \coloneqq V^{r_{\mathbf{A}}^{(\alpha)}}$  be the divergence associated with  $r_{\mathbf{A}}^{(\alpha)}$  for notational convenience.
Definition 4 (Gradient step oracle). For a problem (1), we say  $\mathcal{O}_{\mathrm{grad}}: \mathcal{Z} \times \mathcal{Z}^* \to \mathcal{Z}$  is an  $(\alpha, \beta)$ -gradient step oracle if on input  $(z, v)$ , it returns  $z'$  such that
$$
\langle v, z ^ {\prime} - u \rangle \leq V _ {z} ^ {(\alpha + \beta)} (u) - V _ {z ^ {\prime}} ^ {(\alpha)} (u) - V _ {z} ^ {(\alpha)} \left(z ^ {\prime}\right) f o r a l l u \in \mathcal {Z}.
$$
Definition 5 (Extragradient step oracle). For a problem (1), we say  $\mathcal{O}_{\mathrm{xgrad}}: \mathcal{Z} \times \mathcal{Z}^* \times \mathcal{Y} \to \mathcal{Z}$  is an  $(\alpha, \beta)$ -extragradient step oracle if on input  $(z, v, \bar{y})$ , it returns  $(z^+, \bar{y}^+)$  such that
$$
\left\langle v, z ^ {+} - u \right\rangle \leq V _ {z} ^ {(\alpha)} (u) - V _ {z ^ {+}} ^ {(\alpha)} (u) - V _ {z} ^ {(\alpha)} (z ^ {+}) + V _ {\bar {y}} ^ {\beta h} (u ^ {\mathcal {Y}}) - V _ {\bar {y} ^ {+}} ^ {\beta h} (u ^ {\mathcal {Y}}) \text {f o r a l l} u = \left(u ^ {\times}, u ^ {\mathcal {Y}}\right) \in \mathcal {Z}.
$$
When  $\beta = 0$ , Definitions 4 and 5 reduce to the conventional proximal oracle steps used by the extragradient method of [Nem04]. In our solver for (1), we use  $\beta > 0$  to compensate for our inexact subproblem solves. The asymmetry in Definitions 4 and 5 reflects an asymmetry in the analyses of extragradient methods. In typical analyses, the regret is bounded for the "gradient oracle" points, but the regret upper bound is stated in terms of the divergences of the "extragradient oracle" points (which our inexact oracles need to compensate for).
The utility of our Definitions 3, 4, and 5 reveals itself through the following lemma.
Lemma 4. Let  $z \in \mathcal{Z}$ ,  $\bar{y} \in \mathcal{V}$ ,  $\alpha \geq 2$ ,  $\beta, \gamma \geq 0$  and  $0 \leq \eta \leq \frac{1}{3}$ . Let  $z' \gets \mathcal{O}_{\mathrm{grad}}(z, \eta g(z))$  and  $(z^{+}, \bar{y}^{+}) \gets \mathcal{O}_{\mathrm{xgrad}}(z, \frac{\eta}{2} g(z'), \bar{y})$ , where  $\mathcal{O}_{\mathrm{grad}}$  is an  $(\alpha, \beta)$ -gradient step oracle and  $\mathcal{O}_{\mathrm{xgrad}}$  is an  $(\alpha + \beta, \gamma)$ -extragradient step oracle. Then for all  $u \in \mathcal{Z}$
$$
\langle \eta g (z ^ {\prime}), z ^ {\prime} - u \rangle \leq 2 V _ {z} ^ {(\alpha + \beta)} (u) - 2 V _ {z ^ {+}} ^ {(\alpha + \beta)} (u) + 2 V _ {\bar {y}} ^ {\gamma h} (u ^ {\mathsf {y}}) - 2 V _ {\bar {y} ^ {+}} ^ {\gamma h} (u ^ {\mathsf {y}}).
$$
Proof. By definition of  $\mathcal{O}_{\mathrm{grad}}$  (with  $u \gets z^{+}$ ) and  $\mathcal{O}_{\mathrm{xgrad}}$ , we have
$$
\begin{array}{l} \left\langle \eta g (z), z ^ {\prime} - z ^ {+} \right\rangle \leq V _ {z} ^ {(\alpha + \beta)} (z ^ {+}) - V _ {z ^ {\prime}} ^ {(\alpha)} (z ^ {+}) - V _ {z} ^ {(\alpha)} (z ^ {\prime}), \\ \left\langle \eta g (z ^ {\prime}), z ^ {+} - u \right\rangle \leq 2 V _ {z} ^ {(\alpha + \beta)} (u) - 2 V _ {z ^ {+}} ^ {(\alpha + \beta)} (u) - 2 V _ {z} ^ {(\alpha + \beta)} (z ^ {+}) + 2 V _ {\bar {y}} ^ {\gamma h} (u ^ {y}) - 2 V _ {\bar {y} ^ {+}} ^ {\gamma h} (u ^ {y}). \tag {7} \\ \end{array}
$$
Combining yields
$$
\begin{array}{l} \langle \eta g (z ^ {\prime}), z ^ {\prime} - u \rangle \leq 2 V _ {z} ^ {(\alpha + \beta)} (u) - 2 V _ {z ^ {+}} ^ {(\alpha + \beta)} (u) + 2 V _ {\bar {y}} ^ {\gamma h} (u ^ {y}) - 2 V _ {\bar {y} ^ {+}} ^ {\gamma h} (u ^ {y}) \\ + \eta \left\langle g (z ^ {\prime}) - g (z), z ^ {\prime} - z ^ {+} \right\rangle - V _ {z} ^ {(\alpha)} (z ^ {\prime}) - V _ {z} ^ {(\alpha + \beta)} (z ^ {+}) - V _ {z ^ {\prime}} ^ {(\alpha)} (z ^ {+}) \\ \leq 2 V _ {z} ^ {(\alpha + \beta)} (u) - 2 V _ {z ^ {+}} ^ {(\alpha + \beta)} (u) + 2 V _ {\bar {y}} ^ {\gamma h} (u ^ {y}) - 2 V _ {\bar {y} ^ {+}} ^ {\gamma h} (u ^ {y}) \\ + \eta \left\langle g (z ^ {\prime}) - g (z), z ^ {\prime} - z ^ {+} \right\rangle - V _ {z} ^ {(\alpha)} (z ^ {\prime}) - V _ {z} ^ {(\alpha)} (z ^ {+}), \\ \end{array}
$$
where in the second inequality we used that  $V^{(\alpha +\beta)}$  dominates  $V^{(\alpha)}$ , and  $V_{z'}^{(\alpha)}(z^+) \geq 0$  by Lemma 1. The conclusion follows by applying Definition 3 (see (6)) and the second fact in Lemma 1.
When  $\beta = \gamma = 0$ , Lemma 4 is the same as Appendix B of the supplement, and hence yields similar implications as standard extragradient methods: a scaling of the left-hand side upper bounds duality gap of  $z'$ , and the right-hand side telescopes (and is bounded using the following standard fact).
Lemma 5. Let  $\alpha, \gamma \geq 0$ , and let  $z_0 = (x_0, y_0)$  where  $x_0 = \mathbb{0}_n$  and  $y_0 = \frac{1}{d}\mathbb{1}_d$ . Then  $z_0$  is the minimizer of  $r_{\mathbf{A}}^{(\alpha)}$  over  $\mathcal{Z}$ , and  $V_{z_0}^{(\alpha)}(u) \leq 1 + \alpha \log d$ ,  $V_{y_0}^{\gamma h}(u^{\mathsf{y}}) \leq \gamma \log d$  for all  $u = (u^{\mathsf{x}}, u^{\mathsf{y}}) \in \mathcal{Z}$ .
Finally, for convenience to the reader, we put together Lemma 4 and 5 to obtain an analysis of the following conceptual "outer loop" extragradient algorithm (subject to the implementation of gradient and extragradient step oracles), Algorithm 1. Our end-to-end algorithm will be an explicit implementation of the framework in Algorithm 1; we provide a runtime analysis and error guarantee for our complete algorithm in Theorem 1, as well as pseudocode in Algorithm 4 of the supplement.
Algorithm 1: ConceptualBoxSimplex(A, b, c,  $\mathcal{O}_{\mathrm{grad}}$ ,  $\mathcal{O}_{\mathrm{xgrad}}$ )
1 Input:  $\mathbf{A} \in \mathbb{R}^{n \times d}$  with  $L \coloneqq \| \mathbf{A} \|_{1 \to 1}$ , desired accuracy  $\epsilon \in (0, L)$ ,  $\mathcal{O}_{\mathrm{grad}}$  a (2, 2)-gradient step oracle,  $\mathcal{O}_{\mathrm{xgrad}}$  a (4, 4)-extragradient step oracle  
2 Initialize  $x_0 \gets \mathbb{0}_n$ ,  $y_0 \gets \frac{1}{d} \mathbb{1}_d$ ,  $\bar{y}_0 \gets \frac{1}{d} \mathbb{1}_d$ ,  $\hat{x} \gets \mathbb{0}_n$ ,  $\hat{y} \gets \mathbb{0}_d$ ,  $T \gets \lceil \frac{6(8 \log d + 1)L}{\epsilon} \rceil$ ,  $\eta \gets \frac{1}{3}$   
3 Rescale  $\mathbf{A} \gets \frac{1}{L} \mathbf{A}$ ,  $b \gets \frac{1}{L} b$ ,  $c \gets \frac{1}{L} c$   
4 for  $t = 0$  to  $T - 1$  do  
5  $g_t \gets (\mathbf{A} y_t + c, b - \mathbf{A}^\top x_t)$   
6  $z_t' := (x_t', y_t') \gets \mathcal{O}_{\mathrm{grad}}(z_t, \eta g_t)$   
7  $g_t' \gets (\mathbf{A} y_t' + c, b - \mathbf{A}^\top x_t')$   
8  $(z_{t+1}, \bar{y}_{t+1}) := (x_{t+1}, y_{t+1}, \bar{y}_{t+1}) \gets \mathcal{O}_{\mathrm{xgrad}}(z_t, \frac{\eta}{2} g_t', \bar{y}_t)$   
9 end  
10 Return:  $(\hat{x}, \hat{y}) \gets \frac{1}{T} \sum_{t=0}^{T-1}(x_t', y_t')$
Corollary 1. Algorithm 1 deterministically computes an  $\epsilon$ -approximate saddle point to (1).
Proof. First, clearly the rescaling in Line 3 multiplies the entire problem (1) by  $\frac{1}{L}$ , so an  $\frac{\epsilon}{L}$ -approximate saddle point to the new problem becomes an  $\epsilon$ -approximate saddle point for the original. Throughout the rest of the proof it suffices to treat  $L = 1$ . Next, by telescoping and averaging Lemma 4 with  $\alpha = \beta = 2$ ,  $\gamma = 4$ , and  $\eta = \frac{1}{3}$ , we have for  $z_0 = (x_0, y_0)$  and any  $u = (u^{\times}, u^{\mathcal{Y}}) \in \mathcal{Z}$
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \langle g (z _ {t} ^ {\prime}), z _ {t} ^ {\prime} - u \rangle \leq \frac {6 \left(V _ {z _ {0}} ^ {(4)} (u) + V _ {\bar {y} _ {0}} ^ {4 L h} (u ^ {y})\right)}{T} \leq \epsilon .
$$
The last inequality used the bounds in Lemma 5 and the definition of  $T$ . Moreover since  $g$  is bilinear, and  $\hat{z} \coloneqq (\hat{x}, \hat{y})$  is the average of the  $z_t'$  iterates, we have  $\langle g(\hat{z}), \hat{z} - u \rangle \leq \epsilon$ . Taking the supremum over  $u \in \mathcal{Z}$  bounds the duality gap of  $\hat{z}$  and gives the conclusion.
3.2 Implementing oracles
In this section, we give generic constructions of gradient and extragradient step oracles. $^{10}$
Algorithm 2: GradStepOracle  $(z,v,\alpha ,\beta)$
1 Input:  $z = (x,y)\in \mathcal{Z},v = (v^{\times},v^{\vee})\in \mathcal{Z}^{*},\alpha ,\beta \geq 0$    
2  $y^\prime \gets \mathrm{argmin}_{\hat{y}\in \mathcal{Y}}\left\langle v^{\mathbf{y}} + \nabla_{y}r_{\mathbf{A}}^{(\alpha)}(x_{\mathrm{br}}(y),y) - \nabla_{y}r_{\mathbf{A}}^{(\alpha)}(z),\hat{y}\right\rangle +V_{y}^{\beta h}(\hat{y}),$  where for all  $\hat{y}\in \mathcal{V}$
$$
x _ {\mathrm {b r}} (\hat {y}) := \operatorname {a r g m i n} _ {\hat {x} \in \mathcal {X}} \left\langle v ^ {\times} - \nabla_ {x} r _ {\mathbf {A}} ^ {(\alpha)} (z), \hat {x} \right\rangle + r _ {\mathbf {A}} ^ {(\alpha)} (\hat {z}) \text {w h e r e} \hat {z} := (\hat {x}, \hat {y}) \tag {8}
$$
3  $x^{\prime}\gets x_{\mathrm{br}}(y^{\prime})$
4 Return:  $(x', y')$
Lemma 6. For  $\beta \geq \alpha \geq \frac{1}{2}$ , Algorithm 2 is an  $(\alpha, \beta)$ -gradient step oracle.
Proof. By (3) and the first-order optimality condition for  $y'$ , we have for any  $u = (u^{\times}, u^{\vee}) \in \mathcal{Z}$ ,
$$
\left\langle v ^ {y} + \nabla_ {y} r _ {\mathbf {A}} ^ {(\alpha)} \left(x _ {\mathrm {b r}} (y), y\right) - \nabla_ {y} r _ {\mathbf {A}} ^ {(\alpha)} (z), y ^ {\prime} - u ^ {y} \right\rangle \leq V _ {y} ^ {\beta h} \left(u ^ {y}\right) - V _ {y ^ {\prime}} ^ {\beta h} \left(u ^ {y}\right) - V _ {y} ^ {\beta h} \left(y ^ {\prime}\right). \tag {9}
$$
Further, define for any  $\hat{y}\in \mathcal{V}$
$$
f (\hat {y}) := \langle v, \hat {z} \rangle + V _ {z} ^ {(\alpha)} (\hat {z}) \text {w h e r e} \hat {z} := \left(x _ {\mathrm {b r}} (\hat {y}), \hat {y}\right). \tag {10}
$$
Note that  $f$  is a partial minimization of a function on two variables which is a linear term plus  $r_{\mathbf{A}}^{(\alpha)}$ , which is convex by Lemma 1. For any fixed  $x$ ,  $r_{\mathbf{A}}^{(\alpha)}(x,y)$  is itself  $\alpha h(y)$  plus a linear function.
<sup>10</sup>We slightly abuse notation and use  $x_{\mathrm{br}}$  in (8) in a consistent way with how it is used in Lemma 2, where  $F$  in Lemma 2 is taken to be the jointly convex function of  $(x,y)$  in (8) before minimizing over  $\mathcal{X}$ .
Lemma 3 then shows  $f$  is  $\alpha$ -relatively smooth with respect to  $h$ . We then have for all  $\hat{y}, u^{\mathcal{V}} \in \mathcal{V}$ ,
$$
\begin{array}{l} \left\langle \nabla_ {y} r _ {\mathbf {A}} ^ {(\alpha)} \left(x _ {\mathrm {b r}} (\hat {y}), \hat {y}\right) - \nabla_ {y} r _ {\mathbf {A}} ^ {(\alpha)} \left(x _ {\mathrm {b r}} (y), y\right), \hat {y} - u ^ {\mathrm {y}} \right\rangle = \left\langle \partial f (\hat {y}) - \partial f (y), \hat {y} - u ^ {\mathrm {y}} \right\rangle \tag {11} \\ \leq V _ {y} ^ {\alpha h} (\hat {y}) + V _ {\hat {y}} ^ {\alpha h} (u ^ {\mathcal {Y}}). \\ \end{array}
$$
Here the equality used Lemma 2 where the linear shift between  $r_{\mathbf{A}}^{(\alpha)}$  and the minimization problem inducing  $f$  cancels in the expression  $\partial f(\hat{y}) - \partial f(y)$ , and the inequality used Fact 1. Combining (9) and (11) (with  $\hat{y} \gets y'$ ) and using  $V^{\beta h}$  dominates  $V^{\alpha h}$  yields
$$
\left\langle v ^ {y} + \nabla_ {y} r _ {\mathbf {A}} ^ {(\alpha)} \left(z ^ {\prime}\right) - \nabla_ {y} r _ {\mathbf {A}} ^ {(\alpha)} (z), y ^ {\prime} - u ^ {y} \right\rangle \leq V _ {y} ^ {\beta h} \left(u ^ {y}\right). \tag {12}
$$
Finally, first-order optimality of  $x'$  with respect to the objective induced by  $y'$  implies for all  $u^{\times} \in \mathcal{X}$ ,
$$
\left\langle v ^ {\times} + \nabla_ {x} r _ {\mathbf {A}} ^ {(\alpha)} \left(z ^ {\prime}\right) - \nabla_ {x} r _ {\mathbf {A}} ^ {(\alpha)} (z), x ^ {\prime} - u ^ {\times} \right\rangle \leq 0, \tag {13}
$$
so combining with (12) we have for  $u = (u^{\times}, u^{\mathcal{Y}})$ ,  $\langle v + \nabla r_{\mathbf{A}}^{(\alpha)}(z') - \nabla r_{\mathbf{A}}^{(\alpha)}(z), z' - u \rangle \leq V_y^{\beta h}(u^{\mathcal{Y}})$ . The conclusion follows by using the identity (3) to rewrite  $\langle \nabla r_{\mathbf{A}}^{(\alpha)}(z') - \nabla r_{\mathbf{A}}^{(\alpha)}(z), u - z' \rangle$ .
Algorithm 3: XGradStepOracle  $(z, v, \bar{y}, \alpha, \beta)$
1 Input:  $z = (x,y)\in \mathcal{Z},v = (v^{\times},v^{\mathcal{Y}})\in \mathcal{Z}^{*},\bar{y}\in \mathcal{V},\alpha ,\beta \geq 0$
2  $y^{+}\gets \operatorname *{argmin}_{\hat{y}\in \mathcal{Y}}\left\langle v^{\mathsf{y}} + \nabla_{y}r_{\mathbf{A}}^{(\alpha)}(x_{\mathrm{br}}(\bar{y}),\bar{y}) - \nabla_{y}r_{\mathbf{A}}^{(\alpha)}(z),\hat{y}\right\rangle +V_{\bar{y}}^{\beta h}(\hat{y})$  (following (8))
3  $x^{+}\gets x_{\mathrm{br}}(y^{+})$
4  $\bar{y}^{+}\gets \operatorname *{argmin}_{\hat{y}\in \mathcal{Y}}\left\langle v^{\mathsf{y}} + \nabla_{y}r_{\mathbf{A}}^{(\alpha)}(x^{+},y^{+}) - \nabla_{y}r_{\mathbf{A}}^{(\alpha)}(z),\hat{y}\right\rangle +V_{\bar{y}}^{\beta h}(\hat{y})$
5 Return:  $(x^{+},y^{+},\bar{y}^{+})$
We defer a proof of the following to Section 3.2 of the supplement, as it is similar to Lemma 6. We remark that the auxiliary sequence  $\bar{y},\bar{y}^{+}$  used in the extragradient step oracle definition is itself inspired by running a few steps of a descent method on the subproblem, inducing a telescoping divergence sequence used by the outer loop to pay for inaccuracy in the subproblem solution.
Lemma 7. For  $\beta \geq \alpha \geq \frac{1}{2}$ , Algorithm 3 is an  $(\alpha, \beta)$ -extragradient step oracle.