SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Interestingly, when the spectrum of  $S$  is sparse, the typical behavior of the leading term in the expansion of  $\tilde{\mu}_{M_{\alpha}}$ ,  $\min_j u_j^\top Q u_j + v_j^\top R v_j$ , is quite different. In this case, that quantity depends on the geometric mean of the non-zero eigenvalues of  $S$ , rather than the arithmetic mean, as formalized in the following proposition. The proof is placed in the Appendix, along with a high-probability version of the estimate.
Proposition 3.3. Suppose  $Q, R$  are fixed and  $U, V$  are independently distributed uniformly on the set of  $n \times n$  orthonormal matrices. Let  $s_1 \geq \ldots \geq s_r > 0 = s_{r+1} = \ldots = s_{2n}$  the eigenvalues of  $S$ .
<sup>4</sup>If  $P = U\Sigma V^{\top}$  has i.i.d. Gaussian entries, then  $P$  has the same law as  $\widetilde{U}^{\top}P\widetilde{V}$ , for any  $\widetilde{U},\widetilde{V}\in \mathcal{O}_n$  the set of  $n\times n$  orthonormal matrices. So  $(U,V)$  has the same law as  $(\widetilde{U}^{\top}U,\widetilde{V}^{\top}V)$ . This shows that  $(U,V)$  is distributed according to the Haar measure on the product group  $\mathcal{O}_n\times \mathcal{O}_n$ , that is,  $U$  and  $V$  are independently distributed uniformly on  $\mathcal{O}_n$ .
Table 1: Expansions of  $\rho (\nabla T(z^{*}))$  in  $\alpha$  and  $\eta$  for classical gradient methods
Algorithm	T(z)	∇T(z*)	ρ(∇T(z*))2 = maxj≤2n[...] + O(ηα3 + η2α2)
Sim-GDA	z - ηg(z)	I - ηM	1 - 2αη (wj⊥ Swj) + η2σj2
Alt-GDA	see text	I - η(I - η/2A)M + O(η3)	1 - 2αη (wj⊥ Swj) + O(η3)
EG	z - ηg(z - ηg(z))	I - η(I - ηM)M	1 - 2αη (wj⊥ Swj) - η2σj2 + O(η3)
$$
\mathbb {E} \left[ \min  _ {j \leq n} u _ {j} ^ {\top} Q u _ {j} + v _ {j} ^ {\top} R v _ {j} \right] \geq \max  _ {\mathcal {S} \subset \{1, \dots , r \}} \frac {1}{e} \frac {| \mathcal {S} |}{n} n ^ {- \frac {2}{| \mathcal {S} |}} \left[ \prod_ {l \in \mathcal {S}} s _ {l} \right] ^ {\frac {1}{| \mathcal {S} |}}.
$$
In particular,  $\mathbb{E}\left[\min_{j\leq n}u_j^\top Q u_j + v_j^\top R v_j\right] \gtrsim n^{-\frac{2}{r} -1}$  when  $n\to \infty$  and  $r$  and  $s\in \mathbb{R}^r$  are fixed.
Numerically, the quantity  $\min_{j\leq n}u_j^\top Qu_j + v_j^\top Rv_j$  indeed scales as  $n^{-\frac{2}{r} -1}$  under the conditions of the proposition, suggesting that our lower estimate could be tight.
3.2 Convergence rate of discrete-time algorithms
Prop. 3.1 gave an expansion of the normalized convergence rate  $r / \eta$  of gradient methods (for a game with Jacobian at optimum  $M_{\alpha}$ ), non-asymptotically in  $\alpha$  and in the asymptotic limit  $\eta \rightarrow 0$ . In this subsection, we give expansions of the convergence rate  $r$  that are non-asymptotic in  $\alpha$  and  $\eta$ .
The algorithms we will consider can be written in the form  $z^{k + 1} = T(z^k)$  with the update operator  $T$  dependent only on  $\nabla f$  and on step-size  $\eta$ , and satisfying  $z^{*} = T(z^{*})$ . It is well-known that local convergence of such methods is determined by  $\rho(\nabla T(z^{*}))$ , where  $\nabla T$  is the Jacobian of  $T$  and  $\rho(\cdot)$  denotes spectral radius. Namely, if  $\rho(\nabla T(z^{*})) < 1$  then the iterates converge locally with  $\| z^k - z^* \| = O\left((\rho(\nabla T(z^*)) + \varepsilon)^k\right)$ , with  $\varepsilon > 0$  an arbitrarily small constant [Ber97, Proposition 4.4.1].
In Table 1 we give an expansion for  $\rho (\nabla T(z^{*}))$  for three classical gradient methods: simultaneous GDA (Sim-GDA), alternating GDA (Alt-GDA), and Extra-Gradient (EG). Let us clarify immediately that throughout this paper, statements made about "GDA" without further specification apply to both Sim-GDA and Alt-GDA. In the second column we denoted by  $g(z) = \left( \begin{array}{c}\nabla_{x}f\\ -\nabla_{y}f \end{array} \right)(z)$  the skewed gradient field of the game, and in the third column we wrote  $M$  instead of  $M_{\alpha}$  for concision. In the fourth column,  $\pm i\sigma_{j}$  denotes the eigenvalues of  $A$  assumed distinct and  $w_{j}$  are the associated eigenvectors - equivalently, the singular value decomposition of  $P$  is  $P = \sum_{j = 1}^{n}\sigma_{j}u_{j}v_{j}^{\top}$  and for each  $j\leq n$ ,  $\overline{w}_j^\top Sw_j = \overline{w}_{j + n}^\top Sw_{j + n} = \frac{1}{2}\left(u_j^\top Qu_j + v_j^\top Rv_j\right)$ . We refer to the Appendix for the derivation of this table and for explicit bounds on the "O(\cdot)" terms.
Informally, as one can directly see from the fourth column, Sim-GDA requires a very small step-size for the first term in  $\eta$  to overcome the terms  $+\eta^2\sigma_j^2$ , while for EG those terms actually appear with a favorable sign, and Alt-GDA neither benefits nor suffers from those terms. We also see that Alt-GDA is quite faithful to GF, in that their normalized convergence rates coincide up to terms of order  $\eta^3 + \alpha^4$ . All of these insights are in line with common intuition in the min-max optimization literature [BGP20; Lu22], as well as with our numerical experiments for the next section, Fig. 2.
A symmetrized formulation of Alt-GDA. In order to derive the rate for Alt-GDA, we used the following symmetrized formulation of it: we let  $(x^0,y^{1 / 2})\in \mathbb{R}^d\times \mathbb{R}^d$  and
$$
\left\{ \begin{array}{l l} \forall k \in \mathbb {N}, & x ^ {k + 1} = x ^ {k} - \eta \nabla_ {x} f (x ^ {k}, y ^ {k + 1 / 2}) \\ \forall k \in \mathbb {N} + 1 / 2, & y ^ {k + 1} = y ^ {k} + \eta \nabla_ {y} f (x ^ {k + 1 / 2}, y ^ {k}) \end{array} \right. \quad \text {a n d} \quad \left\{ \begin{array}{l l} \forall k \in \mathbb {N}, & x ^ {k + 1 / 2} = \frac {x ^ {k + 1} + x ^ {k}}{2} \\ \forall k \in \mathbb {N} + 1 / 2, & y ^ {k + 1 / 2} = \frac {y ^ {k + 1} + y ^ {k}}{2}. \end{array} \right.
$$
That is,  $x$  gets updated with the gradient rule at integer time-steps,  $y$  at half-integer time-steps, and we define  $x^k$ ,  $y^k$  at non-updating time-steps as the average of the preceding and following updating time-steps. Assuming  $\eta \leq \left\| \nabla_{xx}^2 f \right\|_\infty^{-1} \wedge \left\| \nabla_{yy}^2 f \right\|_\infty^{-1}$ , we show in the Appendix that  $z^{k+1}$  is indeed
entirely determined by  $z^k = (x^k, y^k)$  for each  $k \in \mathbb{N}$ , and that the associated update operator  $T$  satisfies
$$
T (z) = z - \eta g (z) + \frac {\eta^ {2}}{2} A (z) g (z) + O (\eta^ {3} \| g (z) \|) \quad \mathrm {w h e r e} \quad A (z) = \frac {\nabla g (z) - \nabla g (z) ^ {\top}}{2}.
$$
For comparison, the usual formulation of Alt-GDA considers as the iterates  $(\tilde{x}^k,\tilde{y}^k) = (x^k,y^{k + 1 / 2})$  We emphasize that  $(\tilde{x}^k,\tilde{y}^k)$  and  $(x^{k},y^{k})$  have the same convergence rate if they converge exponentially, as one can check directly from the definition.
4 Illustration: sparse mixed Nash equilibria of continuous games
In this section we apply the above considerations to a particular class of min-max problems, which is of its own interest in game theory. Namely we consider the classical problem of finding the mixed Nash equilibria (MNE) of two-player zero-sum games, that is, given strategy spaces  $\mathcal{X},\mathcal{Y}$  and a payoff function  $f:\mathcal{X}\times \mathcal{Y}\to \mathbb{R}$ , solving the min-max problem
$$
\min  _ {\mu \in \mathcal {P} (\mathcal {X})} \max  _ {\nu \in \mathcal {P} (\mathcal {Y})} \left\{F (\mu , \nu) = \mathbb {E} _ {x \sim \mu , y \sim \nu} [ f (x, y) ] \right\}.
$$
Here  $\mathcal{P}(\mathcal{X})$  denotes the space of probability measures - representing mixed strategies - over  $\mathcal{X}$ . Let us focus on cases where  $\mathcal{X}$  and  $\mathcal{Y}$  are continuous sets, say,  $\mathcal{X} = \mathcal{Y} = \mathbb{T}^1$  the one-dimensional Euclidean torus, and  $f: \mathbb{T}^1 \times \mathbb{T}^1 \to \mathbb{R}$  is smooth. The above min-max problem is then infinite-dimensional, and algorithms to solve it explicitly must be based on reparameterized formulations. More specifically, suppose that the MNE  $(\mu^{*}, \nu^{*})$  is unique and "sparse", i.e., has finite supports:  $\mathrm{supp}(\mu^{*}) = \{x_{I}^{*}, 1 \leq I \leq N\}$ ,  $\mathrm{supp}(\nu^{*}) = \{y_{J}^{*}, 1 \leq J \leq M\}$  (this is the case for example if  $f$  is a sum of separable functions [SOP08]).
In this setting there are two natural reparameterizations and associated algorithms:
1. If the optimal support points  $\{x_I^*\}_{I},\{y_J^*\}_{J}$  are known, then we may reparameterize by  $\mu = \sum_{I = 1}^{N}a_{I}\delta_{x_{I}^{*}},\nu = \sum_{J = 1}^{M}b_{J}\delta_{y_{J}^{*}}$  and optimize over the  $a_{I},b_{J}$ . The problem reduces to a constrained bilinear game
$$
\min  _ {a \in \Delta_ {N}} \max  _ {b \in \Delta_ {M}} \left\{F _ {1} (a, b) = \sum_ {I = 1} ^ {N} \sum_ {J = 1} ^ {M} a _ {I} b _ {J} f \left(x _ {I} ^ {*}, y _ {J} ^ {*}\right) =: a ^ {\top} P b \right\}
$$
where  $\Delta_N$  denotes the standard simplex. A classical approach is then to apply Mirror Prox (MP) with entropy link function [Nem04] (MP is the Bregman-geometry analog of EG).
2. If only the number of optimal support points is known, then we may reparameterize by  $\mu = \sum_{I=1}^{N} a_{I} \delta_{x_{I}}$ ,  $\nu = \sum_{J=1}^{M} b_{J} \delta_{y_{J}}$  and optimize over both the weights  $(a_{I}, b_{J})$  and the support points  $(x_{I}, y_{J})$ . The problem reduces to
$$
\min  _ {(a, x) \in \Delta_ {N} \times (\mathbb {T} ^ {1}) ^ {N}} \max  _ {(b, y) \in \Delta_ {M} \times (\mathbb {T} ^ {1}) ^ {M}} \left\{F _ {2} (a, x, b, y) = \sum_ {I = 1} ^ {N} \sum_ {J = 1} ^ {M} a _ {I} b _ {J} f (x _ {I}, y _ {J}) \right\}.
$$
A possible approach is to iteratively update (simultaneously) the  $a, b$  using MP steps with step-size  $\eta$  and the  $x, y$  using EG steps with step-size  $\gamma \eta$ , for some parameter  $\gamma > 0$ . This algorithm is called Conic Particle Mirror Prox in [WC22], but for concision we will refer to it simply as "EG" in this section. Note that the main challenge in that reference is to deal with the case where  $N$  and  $M$  are unknown, but here we assume they are known for the sake of simplicity.
In Fig. 2, we show the dependency on  $\eta$  and  $\gamma$  of the local convergence rate of MP and EG, as well as that of the analogs of Sim-GDA and Alt-GDA and GF for the problem  $\min_{(a,x)}\max_{(b,y)}F_2(a,x,b,y)$  in order to illustrate the insights from Sec. 3.2. We used a randomly generated payoff function  $f$ , and the convergence is measured by the iterates'  $\ell_2$  distance to the solution; see the Appendix for details.
The choice of  $\mathcal{X} = \mathcal{Y} = \mathbb{T}^1$  is made for simplicity of exposition. The discussion below extends straightforwardly to toruses of any dimension, and could be extended to  $\mathcal{X},\mathcal{Y}$  Riemannian manifolds without boundaries at the cost of more technical notation.
(a) Convergence rate  $r$  vs.  $\gamma$  (fixed  $\eta = 10^{-2}$ )
(b) Convergence rate vs.  $\eta$  (fixed  $\gamma = 10^{-2}$ )
Figure 2: Observed local convergence rates  $r$  (i.e.,  $\left\| z^k - z^* \right\| = \Theta((1 - r)^k)$ ) for various conic particle methods using step-size  $\eta$  for the weights  $(a, b)$  and  $\gamma \eta$  for the positions  $(x, y)$  and for a fixed random draw of  $f$ . (left) Fixing  $\eta = 10^{-2}$ , we observe a rate for GF (we plot  $\eta \cdot \tilde{\mu}_{\gamma}$ ) scaling as  $\gamma^2$  as predicted in Prop. 4.1. Interestingly, Alt-GDA has exactly the same behavior, while the behavior of Sim-GDA and EG differ for small  $\gamma$ , due to the corrective terms shown in Table 1. (right) Fixing  $\gamma = 10^{-2}$ , we observe a rate scaling as  $\eta$  for EG and its variants, and as  $\eta^2$  for MP; the convergence of EG is mostly faster than MP.
MP (the algorithm based on the reparameterization  $F_{1}$ ) converges to the MNE for any small enough  $\eta$  with an exponential rate at least proportional to  $\eta^2$  [WLZL21], and this scaling is tight numerically (green dots in Fig. 2b). Its explicit variant (Mirror Descent-Ascent) and its continuous-time flow are known to diverge for any  $\eta$  [BP18] [MPP18].
EG is known to converge locally to the MNE for any small enough  $\eta$  (and any  $\gamma > 0$ ) with an exponential rate at least proportional to  $\eta^2$ , despite the non-convexity of  $F_2$ , under some nondegeneracy assumptions [WC22, Sec. 3.1]. However numerically the convergence rate of EG typically scales as  $\eta$ , not  $\eta^2$  (orange dots in Fig. 2b) – a fact which we will explain below. The explicit variant of EG and its continuous-time flow have not previously been analyzed; the discussion below will give a characterization of when they converge locally.
Note that, at least for the particular  $f$  and  $\gamma$  used for Fig. 2b, EG converges locally faster than MP for the same  $\eta$  (in number of iterations, the per-iteration costs differing only by a constant factor), even though the former does not use the knowledge of the  $\{x_{I}^{*}\}_{I}, \{y_{J}^{*}\}!$ . In other words, even when the optimal support points are known, it is beneficial to use the overparameterized formulation  $F_{2}$  where we also vary the support points.
Overparameterization induces partial curvature. Let us inspect the "Jacobian" at optimum for the two dynamics, MP vs. EG with parameter  $\gamma$ . Due to the simplex constraints and the non-Euclidean nature of the updates for  $a, b$ , the relevant matrices are  $M_{\mathrm{MP}}$  and  $M_{\gamma}$  defined below, in the sense that the exponential convergence rate of each algorithm is  $\tilde{\mu}_M\eta + O(\eta^2)$  (see the Appendix). Namely, omitting half of the antisymmetric off-diagonals for readability,
$$
M _ {\mathrm {M P}} = \left[ \begin{array}{c c} \mathbf {0} & \Pi_ {a} D _ {a} P D _ {b} \Pi_ {b} ^ {\top} \\ - (\ast) ^ {\top} & \mathbf {0} \end{array} \right]
$$
where  $D_{a} = \mathrm{Diag}(\sqrt{a^{*}})$  (square roots being taken component-wise) and  $\Pi_{a}\in \mathbb{R}^{(N - 1)\times N}$  is any matrix such that  $\Pi_a\Pi_a^\top = I_{N - 1}$  and  $\Pi_a^\top \Pi_a = I_N - \sqrt{a^*\sqrt{a^*}}^\top$ , and likewise for  $D_{b}$  and  $\Pi_b\in \mathbb{R}^{(M - 1)\times M}$ ; and for EG,
$$
M _ {\gamma} = \left[ \begin{array}{c c c c} \mathbf {0} & \mathbf {0} & \Pi_ {a} D _ {a} P D _ {b} \Pi_ {b} ^ {\top} & \sqrt {\gamma}   \Pi_ {a} D _ {a} [ \partial_ {y} P ] D _ {b} \\ \mathbf {0} & \gamma \mathrm {D i a g} (\partial_ {x x} ^ {2} P b ^ {*}) & \sqrt {\gamma} D _ {a} [ \partial_ {x} P ] D _ {b} \Pi_ {b} ^ {\top} & \gamma D _ {a} [ \partial_ {x y} ^ {2} P ] D _ {b} \\ - (*) ^ {\top} & - (*) ^ {\top} & \mathbf {0} & \mathbf {0} \\ - (*) ^ {\top} & - (*) ^ {\top} & \mathbf {0} & - \gamma \mathrm {D i a g} (\partial_ {y y} ^ {2} P ^ {\top} a ^ {*}) \end{array} \right]
$$
where  $[\partial_xP]_{IJ} = \partial_xf(x_I^*,y_J^*)$ , and likewise for  $\partial_yP, \partial_{xx}^2 P, \partial_{yy}^2 P, \partial_{xy}^2 P$ .
For MP, it is clear that the equivalent conditions of Thm. 2.1 are violated for any payoff function  $f$  and so  $\tilde{\mu}_{M_{\mathrm{MP}}} = 0$ . For EG, depending on  $f$  and  $\gamma$ , they may or may not be violated. For all of the random payoff functions we considered in our experiments, we observed that  $\tilde{\mu}_{M_{\gamma}} > 0$ , suggesting
that the conditions hold generically. They are violated for certain  $f$ 's however, as shown in the Appendix, so that the scaling in  $\eta^2$  of the convergence rate proved in [WC22] is tight in the worst case.
More precisely as we show in the next proposition, assuming that the blocks of  $M_{\gamma}$  are in general position, we expect  $\tilde{\mu}_{M_{\gamma}}$  to scale as  $\gamma^2$ , which is indeed observed in the numerical experiment reported in Fig. 2a. The proof, placed in the Appendix, relies on the same tools as Prop. 3.1, that is, on the asymptotic expansions of the eigenvalues of perturbed matrices.
Proposition 4.1. Let  $S_{2}$  symmetric and  $A_0, A_1, A_2$  antisymmetric real matrices of the form
$$
S _ {2} = \left[ \begin{array}{c c c} \mathbf {0} & & \\ & * & \\ \hline & & \mathbf {0} & \\ & & & * \end{array} \right], A _ {0} = \left[ \begin{array}{c c c} & * & \mathbf {0} \\ & \mathbf {0} & \mathbf {0} \\ \hline * & \mathbf {0} \\ \mathbf {0} & \mathbf {0} \end{array} \right], A _ {1} = \left[ \begin{array}{c c c} & \mathbf {0} & * \\ \hline \mathbf {0} & * & \mathbf {0} \\ \hline * & \mathbf {0} \end{array} \right], A _ {2} = \left[ \begin{array}{c c c} & \mathbf {0} & \mathbf {0} \\ & \mathbf {0} & * \\ \hline \mathbf {0} & \mathbf {0} \\ \mathbf {0} & * \end{array} \right]
$$
and  $M_{\gamma} = \gamma S_2 + A_0 + \sqrt{\gamma} A_1 + \gamma A_2$  for all  $\gamma > 0$ . Then  $\tilde{\mu}_{M_{\gamma}} = O(\gamma^2)$  as  $\gamma \to 0$ .
We have investigated the local convergence of gradient methods for min-max games and found that they converge generically under partial curvature. In more specific settings, we have obtained quantitative estimates of the local convergence rate which exhibit the average of the eigenvalues of  $S$  as the driving quantity for typical problems. For the computation of mixed Nash equilibria of continuous games, this leads to a behavior of conic particle gradient methods that is more favorable than that described by the worst-case bounds.
More generally, our analysis leads to the following insights: (i) worst-case bounds might be looser in min-max optimization (compared to minimization) as they fail to capture the interplay between interaction and potential parts; (ii) the addition of new degrees of freedom with curvature typically accelerates the local convergence, as we illustrated in Sec. 4.
We note that the phenomenon described in this paper is fundamentally a consequence of the fact that the skewed gradient field's Jacobian at optimum has a positive-semidefinite symmetric part. This property is satisfied at local Nash equilibria of min-max games, i.e., of two-player zero-sum differentiable games, but is not true for general differentiable games.
[ALW21] Jacob Abernethy, Kevin A Lai, and Andre Wibisono. "Last-iterate convergence rates for min-max optimization: Convergence of hamiltonian gradient descent and consensus optimization". In: Algorithmic Learning Theory. PMLR. 2021, pp. 3-47.
[AW20] Ehsan Amid and Manfred KK Warmuth. "Reparameterizing mirror descent as gradient descent". In: Advances in Neural Information Processing Systems 33 (2020), pp. 8430-8439.
[AMLG20] Waiss Azizian, Ioannis Mitlagkas, Simon Lacoste-Julien, and Gauthier Gidel. "A tight and unified analysis of gradient-based methods for a whole spectrum of differentiable games". In: International Conference on Artificial Intelligence and Statistics. PMLR. 2020, pp. 2863-2873.
[BGP20] James P Bailey, Gauthier Gidel, and Georgios Piliouras. “Finite regret and cycles with fixed step-size via alternating gradient descent-ascent”. In: Conference on Learning Theory. PMLR. 2020, pp. 391–407.
[BP18] James P Bailey and Georgios Piliouras. "Multiplicative weights update in zero-sum games". In: Proceedings of the 2018 ACM Conference on Economics and Computation. 2018, pp. 321-338.
[Ber97] Dimitri P Bertsekas. "Nonlinear programming". In: Journal of the Operational Research Society 48.3 (1997), pp. 334-334.
[Doa22] Thinh Doan. “Convergence rates of two-time-scale gradient descent-ascent dynamics for solving nonconvex min-max problems”. In: Learning for Dynamics and Control Conference. PMLR. 2022, pp. 192–206.
[VTM07] Nico Van der Aa, H Ter Morsche, and R Matthaeij. "Computation of eigenvalue and eigenvector derivatives for a general complex-valued eigensystem". In: The electronic journal of linear algebra 16 (2007), pp. 300-314.
[Vil09] Cédric Villani. Hypocoercivity. Vol. 202. 950. American Mathematical Society, 2009.
[Wai19] Martin J Wainwright. High-dimensional statistics: A non-asymptotic viewpoint. Vol. 48. Cambridge university press, 2019.
[WC22] Guillaume Wang and Lénaïc Chizat. “An Exponentially Converging Particle Method for the Mixed Nash Equilibrium of Continuous Games”. In: arXiv preprint arXiv:2211.01280 (2022).
[WLZL21] Chen-Yu Wei, Chung-Wei Lee, Mengxiao Zhang, and Haipeng Luo. "Linear Last-iterate Convergence in Constrained Saddle-point Optimization". In: International Conference on Learning Representations. 2021.
[ZWLG22] Guodong Zhang, Yuanhao Wang, Laurent Lessard, and Roger B Grosse. "Near-optimal local convergence of alternating gradient descent-ascent for minimax optimization". In: International Conference on Artificial Intelligence and Statistics. PMLR. 2022, pp. 7659-7679.