SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Remark 2 It is worth noting that our results are more general compared to AGP in [57], where a unified analysis for convex-nonconcave and nonconvex-concave problems is provided. Here, our algorithm can also be applied to the nonconvex-nonconcave problem with one-sided KL property. Moreover, our algorithm is universal, meaning that one single set of parameters can ensure convergence across all these scenarios. By contrast, different choices of parameters are required to guarantee optimal convergence in AGP.
4 Empirical Validation of DS-GDA
4.1 Universality of DS-GDA
To validate the universality of DS-GDA, we commence by providing a graphical description of feasible regions for the choice of parameters. Subsequently, employing a set of symmetric parameters, we will show that the KL-nonconvex problems can converge to the desired stationary point, which supports our universality results.
Without loss of generality, we consider the case where  $\lambda = 1$ . In the pursuit of symmetric parameter selection, we initially fix  $\beta = \mu = 1 / 5000$ , thus reducing the problem to determining only two remaining parameters:  $c$  and  $r$ . We then explore their relationships by setting  $r_1 = r_2 = t_2L$  and  $c = \alpha = 1 / (t_1r)$ . Restricting  $0 \leq t_1, t_2 \leq 100$ , the feasible choices of  $t_1$  and  $t_2$  are selected to guarantee the first four coefficients in basic descent estimate (1) are positive. As visually depicted in Figure 1a, it becomes apparent that a large number of choices for  $r$  and  $c$  are available to ensure convergence.
Next, we test our algorithm on a nonconvex-nonconcave problem that satisfies the one-sided KL property. With a set of symmetric parameters, we find that DS-GDA can easily converge to the desired stationary point. We first introduce the KL-nonconcave problems as follows:
KL-Nonconcave Example The following example satisfies two-sided KL property with exponent  $\frac{1}{2}$ , which is studied in [58]:
$$
\min  _ {x \in \mathcal {X}} \max  _ {y \in \mathcal {Y}} x ^ {2} + 3 \sin (x) ^ {2} \sin (y) ^ {2} - 4 y ^ {2} - 1 0 \sin (y) ^ {2}, \tag {2}
$$
where  $\mathcal{X} = \mathcal{Y} = \{z: -1 \leq z \leq 1\}$ . The only saddle point is  $u^{*} = [x^{*}; y^{*}] = [0; 0]$ .
From Figure 1b, we can observe that symmetric parameter selection is effective for addressing KL-nonconcave problems. To be specific, by setting  $r_1 = r_2 = 0.125$ ,  $c = \alpha = 0.04$ ,  $\beta = \mu = 0.8$ , DS-GDA directly converges to the saddle point, which validates the universal results in Theorem 3.
4.2 Robustness of DS-GDA
In this section, we compare the convergence performance of DS-GDA with the closely related algorithm S-GDA on some illustrative examples. Additionally, we report the range of parameters for these two algorithms to demonstrate the robustness of DS-GDA. To begin, we present some simple polynomial examples.
(a) Feasible region of different  $t_1$  and  $t_2$
(b) Convergence of KL-nonconcave problem
Figure 1: In Figure (a), the blue region indicates where convergence cannot be guaranteed. The green region indicates a series of parameters that can be chosen to guarantee convergence. Figure (b) demonstrates the effectiveness of symmetric parameter selection for (non)convex-(non)concave problems.
Convex-Nonconcave Example The following example is convex-nonconcave, which is studied in [13, 1]:
$$
\min_{x\in \mathcal{X}}\max_{y\in \mathcal{Y}}2x^{2} - y^{2} + 4xy + 4y^{3} / 3 - y^{4} / 4,
$$
where  $\mathcal{X} = \mathcal{Y} = \{z: -1 \leq z \leq 1\}$  and  $u^{*} = [x^{*}; y^{*}] = [0; 0]$  is the only stationary point.
KL-Nonconcave Example The KL-nonconcave example considered here is mentioned in Section 4.1 (see equation (2)). The  $u^{*} = [x^{*};y^{*}] = [0;0]$  is a saddle point and the only stationary point.
Nonconvex-Nonconcave Example The nonconvex-nonconcave example considered here is the "Bilinearly-coupled Minimax" example (3) discussed in [29]:
$$
\min  _ {x \in \mathcal {X}} \max  _ {y \in \mathcal {Y}} f (x) + A x y - f (y), \tag {3}
$$
where  $f(z) = (z + 1)(z - 1)(z + 3)(z - 3)$ ,  $A = 11$ , and  $\mathcal{X} = \mathcal{Y} = \{z : -4 \leq z \leq 4\}$ . It does not satisfy any existing regularity condition, and the point  $u^{*} = [x^{*}; y^{*}] = [0; 0]$  is the only stationary point.
The extrapolation parameters  $z$  and  $v$  are initialized as  $x$  and  $y$ , respectively. According to Lemma 8, our algorithm terminates when  $\| z - v \|$  and  $\| y - v \|$  are less than  $10^{-6}$  or when the number of iterations exceeds  $10^{7}$ . To ensure a fair comparison, we use the same initializations for DS-GDA and S-GDA in each test. The algorithm parameters are tuned so that both DS-GDA and S-GDA are optimal. In other words, they can converge to a stationary point in a minimum number of iterations. We compare the convergence of two algorithms by plotting the iteration gap  $\| u^{k} - u^{*} \|$  against the number of iterations for each example, where  $u^{*}$  denotes the stationary point. Our results show that DS-GDA and S-GDA have similar convergence performance when the primal function is convex or satisfies the KL property, as depicted in Figure 2a and 2b. However, for the nonconvex-nonconcave example where no regularity condition is satisfied, DS-GDA achieves much faster convergence than S-GDA, as shown in Figure 2c.
To demonstrate the robustness of the proposed DS-GDA, we present feasible regions of all common hyperparameters in Figure 3. We tune the lower and upper bounds of each parameter while keeping other parameters fixed at their optimal values. As illustrated in Figures 3a and 3b, for examples that satisfy the one-sided KL property or with a convex primal function, the implementable ranges of DS-GDA and S-GDA for common parameters are roughly similar. In addition, the two auxiliary parameters for DS-GDA, i.e.,  $r_2$  and  $\mu$ , have relatively wide ranges of values, indicating that they allow for flexibility in their selection. However, for nonconvex-nonconcave problems, DS-GDA exhibits a wider range of viable parameter values compared to S-GDA. This observation highlights the robustness of DS-GDA when it comes to parameter selection (refer to Figure 3c).
(a) Convex-Nonconcave
(b) KL -Nonconcave
(c) Nonconvex-Nonconcave
Figure 2: Convergence performance of different problems.
(a) Convex-Nonconcave
Figure 3: Range of parameters for different problems.
4.3 Effectiveness of Getting Rid of Limit Cycle
In this section, we demonstrate the effectiveness of the proposed DS-GDA on some illustrative examples that are widely used in literature. Notably, they do not satisfy any of the regularity conditions in previous literature (i.e., KL condition, weak MVI, and  $\alpha$ -dominant condition). We refer the readers to Appendix L for details on how to check the failure of these conditions for these examples. In addition to the violation of regularity conditions, it has been verified that none of the existing first-order algorithms can achieve convergence for all four examples. The detailed description of the four examples can be found in Appendix B.
To showcase the convergence behavior and effectiveness of the proposed DS-GDA, we compare it with three other state-of-the-art methods, that is, damped extragradiant method (Damped EGM) [30], S-GDA [40, 62], and generalized curvature extragradiant method (CurvatureEG+) [50]. Damped EGM is guaranteed to converge under  $\alpha$ -dominant condition, and S-GDA converges when the dual function is concave. CurvatureEG+ could converge under weak MVI condition, which is the weakest variational inequality-based condition as far as we know in the literature.
For the experiments, we use the same initializations of primal and dual variables for all four methods to ensure a fair comparison. For DS-GDA, the exponentially weighted variables are initialized as the same values of the primal and dual variables, respectively. We stop DS-GDA when the differences between primal-dual variables and the corresponding exponentially weighted variables are all less than  $10^{-6}$  or when the number of iterations exceeds  $10^{5}$ . For other baseline methods, we terminate them when either the number of iterates reaches  $10^{5}$  or the difference between two consecutive iterates is less than  $10^{-6}$ .
Figure 4 shows the trajectories of different methods with various initializations for the aforementioned four examples. We observe from the first column that our DS-GDA successfully gets rid of limit cycles in all examples. While S-GDA exhibits similar performance as DS-GDA, it is still not as potent in terms of its overall effectiveness. Specifically, in the case of the "Blinearly-coupled minimax" example, S-GDA gets trapped in a limit cycle, while our DS-GDA successfully avoids it and achieves convergence. The figure in the second column provides more details on this comparison. With the violation of the of  $\alpha$ -interaction dominance condition, damped EGM either suffers from the spurious cycling convergence phenomenon or diverges to a point on the boundary (see the third row in the fourth column). It converges only when the initialization is very close to the stationary point (see the fourth row in the last column). Similar results are observed for CurvatureEG+ (see Figures in the third column for details). Thus, the proposed DS-GDA outperforms other methods. It is the only
Figure 4: Trajectories of different methods with various initialization for the aforementioned four examples. The initialization of every trajectory is marked as a black star, the blue (resp. red) line represents the path (resp. limit cycle) of the methods, and the red rhombus is the stationary point.
algorithm that can get rid of the limit cycle and enjoy global convergence for all these challenging examples.
In this paper, we propose a single-loop algorithm called the doubly smoothed gradient descent ascent (DS-GDA) algorithm, which offers a natural balance between primal-dual updates for constrained nonconvex-nonconcave minimax problems. This is the first simple and universal algorithm for nonconvex-concave, convex-nonconcave, and nonconvex-nonconcave problems with one-sided KL property. By employing a single set of parameters, DS-GDA achieves convergence with an iteration complexity of  $\mathcal{O}(\epsilon^{-4})$  across all these scenarios. Sharper iteration complexity can be obtained when the one-sided KL property is satisfied with an exponent  $\theta \in (0,1)$ , matching the state-of-the-art results. We further conduct experiments to validate the universality and efficiency of DS-GDA for avoiding the limit cycle phenomenon, which commonly occurs in challenging nonconvex-nonconcave examples. There is still a gap between theory and practice, and it would be intriguing to explore the possibility of achieving global convergence for DS-GDA without any regularity conditions. This opens up new avenues for research on nonconvex-nonconcave minimax problems.
Anthony Man-Cho So is supported in part by the Hong Kong Research Grants Council (RGC) General Research Fund (GRF) projects CUHK 14203920 and 14216122. Jose Blanchet and Jiajin Li are supported by the Air Force Office of Scientific Research under award number FA9550-20-1-0397 and NSF 1915967, 2118199, 2229012, 2312204.
[1] Leonard Adolphs, Hadi Daneshmand, Aurelien Lucchi, and Thomas Hofmann. Local saddle point optimization: A curvature exploitation approach. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 486-495. PMLR, 2019.
[2] Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In International Conference on Machine Learning, pages 214-223. PMLR, 2017.
[3] Hédy Attouch, Jérôme Bolte, Patrick Redont, and Antoine Soubeyran. Proximal alternating minimization and projection methods for nonconvex problems: An approach based on the kurdyka-Lojasiewicz inequality. Mathematics of Operations Research, 35(2):438-457, 2010.
[4] Hedy Attouch, Jérôme Bolte, and Benar Fux Svaiter. Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods. Mathematical Programming, 137(1-2):91-129, 2013.
[5] Hédy Attouch and Roger J-B Wets. A convergence theory for saddle functions. Transactions of the American Mathematical Society, 280(1):1-41, 1983.
[6] Aharon Ben-Tal, Laurent El Ghaoui, and Arkadi Nemirovski. Robust optimization, volume 28. Princeton University Press, 2009.
[7] Dimitris Bertsimas, David B Brown, and Constantine Caramanis. Theory and applications of robust optimization. SIAM review, 53(3):464-501, 2011.
[8] Axel Böhm. Solving nonconvex-nonconcave min-max problems exhibiting weak minty solutions. arXiv preprint arXiv:2201.12247, 2022.
[9] Jérôme Bolte, Aris Daniilidis, and Adrian Lewis. The Łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems. SIAM Journal on Optimization, 17(4):1205-1223, 2007.
[10] Jérôme Bolte, Shoham Sabach, and Marc Teboulle. Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Mathematical Programming, 146(1-2):459-494, 2014.
[11] Yang Cai, Argyris Oikonomou, and Weiqiang Zheng. Accelerated algorithms for monotone inclusions and constrained nonconvex-nonconcave min-max optimization. arXiv preprint arXiv:2206.05248, 2022.
[12] Yang Cai and Weiqiang Zheng. Accelerated single-call methods for constrained min-max optimization. arXiv preprint arXiv:2210.03096, 2022.
[13] Raphael Chinchilla, Guosong Yang, and Joao P Hespanha. Newton and interior-point methods for (constrained) nonconvex-nonconcave minmax optimization with stability guarantees. arXiv preprint arXiv:2205.08038, 2022.
[14] Djork-Arné Clevert, Thomas Unterthiner, and Sepp Hochreiter. Fast and accurate deep network learning by exponential linear units (elus). arXiv preprint arXiv:1511.07289, 2015.
[15] Michel Coste. An introduction to o-minimal geometry. Istituti editoriali e poligrafici internazionali Pisa, 2000.
[16] Bo Dai, Albert Shaw, Lihong Li, Lin Xiao, Niao He, Zhen Liu, Jianshu Chen, and Le Song. Sbeed: Convergent reinforcement learning with nonlinear function approximation. In International Conference on Machine Learning, pages 1125-1134. PMLR, 2018.
[51] Boris T Polyak. Gradient methods for solving equations and inequalities. USSR Computational Mathematics and Mathematical Physics, 4(6):17-32, 1964.
[52] R Tyrrell Rockafellar and Roger J-B Wets. Variational analysis, volume 317. Springer Science & Business Media, 2009.
[53] Aman Sinha, Hongseok Namkoong, Riccardo Volpi, and John Duchi. Certifying some distributional robustness with principled adversarial training. arXiv preprint arXiv:1710.10571, 2017.
[54] Maurice Sion. On general minimax theorems. Pacific Journal of Mathematics, 8(1):171-176, 1958.
[55] Chaobing Song, Zhengyuan Zhou, Yichao Zhou, Yong Jiang, and Yi Ma. Optimistic dual extrapolation for coherent non-monotone variational inequalities. Advances in Neural Information Processing Systems, 33:14303-14314, 2020.
[56] Yuanhao Wang, Guodong Zhang, and Jimmy Ba. On solving minimax optimization locally: A follow-the-ridge approach. arXiv preprint arXiv:1910.07512, 2019.
[57] Zi Xu, Huiling Zhang, Yang Xu, and Guanghui Lan. A unified single-loop alternating gradient projection algorithm for nonconvex-concave and convex-nonconcave minimax problems. arXiv preprint arXiv:2006.02032, 2020.
[58] Junchi Yang, Negar Kiyavash, and Niao He. Global convergence and variance-reduced optimization for a class of nonconvex-nonconcave minimax problems. arXiv preprint arXiv:2002.09621, 2020.
[59] Junchi Yang, Antonio Orvieto, Aurelien Lucchi, and Niao He. Faster single-loop algorithms for minimax optimization without strong concavity. In International Conference on Artificial Intelligence and Statistics, pages 5485-5517. PMLR, 2022.
[60] TaeHo Yoon and Ernest K Ryu. Accelerated algorithms for smooth convex-concave minimax problems with  $o(1 / k^2)$  rate on squared gradient norm. In International Conference on Machine Learning, pages 12098-12109. PMLR, 2021.
[61] Peiran Yu, Guoyin Li, and Ting Kei Pong. Kurdyka-Lojasiewicz exponent via inf-projection. Foundations of Computational Mathematics, 22(4):1171-1217, 2022.
[62] Jiawei Zhang, Peijun Xiao, Ruoyu Sun, and Zhiquan Luo. A single-loop smoothed gradient descent-ascent algorithm for nonconvex-concave min-max problems. Advances in Neural Information Processing Systems, 33:7377-7389, 2020.
- The related work and the four challenging nonconvex-nonconcave examples are listed in Section B;
- Notations and some useful lemmas such as Lipschitz error bounds are provided in Section C;
- The characterization of changes in the Lyapunov function between successive iterations is established in Section D;
- The proof of Proposition 1 is given in Section E;
- The proof of proximal and dual error bounds are given in Section F;
- The proof of Theorem 1 is given in Section G;
- The convergence proof for nonconvex(convex)-nonconcave problem is provided in Section H.
- The proof of Theorem 2 is provided in Section I;
- The proof of Theorem 3 is provided in Section J;
- The quantitative relationship between different notions of the stationary point is provided in Section K;
- The properties of examples mentioned in Section 4.3 are checked in Section L. We also show that the wrong selection of smoothing sides will result in slow convergence for S-GDA.
There are three representative types of regularity conditions in the literature to restrict the problem class that algorithms are developed to get rid of the limit cycle.
Polyak-Lojasiewicz (PL) condition The PL condition (4) was originally proposed by [51] and is a crucial tool for establishing linear convergence of first-order algorithms for pure minimization problems [35]. Suppose that the problem  $\max_{x\in \mathbb{R}^d}h(x)$  has a nonempty solution set and a finite optimal value. The PL condition states that there exist a constant  $\mu >0$  such that for any  $x\in \mathbb{R}^d$
$$
\frac {1}{2} \| \nabla h (x) \| ^ {2} \geq \mu \left(h (x) - \min  _ {x \in \mathbb {R} ^ {d}} h (x)\right). \tag {4}
$$
There is a host of works trying to invoke the PL condition on the dual function  $f(x, \cdot)$  [59, 47, 22, 58]. Unfortunately, we would like to point out that this condition is too restrictive and inherently avoid the main difficulty in addressing general nonconvex-nonconcave minimax problems. With PL condition imposed on the dual function, the inner maximization value function  $\phi(\cdot) = \max_{y \in \mathcal{Y}} f(\cdot, y)$  is  $L$ -smooth [47, Lemma A.5]. Thus, the dual update can naturally be controlled by the primal since we can regard minimax problems as pure smooth (weakly convex) minimization problems over  $x$ . However, for general cases, the inner value function  $\phi$  may not even be Lipschitz. Recently, [47] proposed a so-called multi-step GDA method with an iteration complexity of  $O(\log (\epsilon^{-1})\epsilon^{-2})$ . [22] further developed the single-loop two-timescale GDA method to better take the computational tractability into account and the complexity is improved to  $O(\epsilon^{-2})$ . Following the smoothing (extrapolation) technique developed in [62], [59] extended the proposed smoothed GDA to the stochastic setting and obtained an iteration complexity of  $O(\epsilon^{-4})$ .
Varitional Inequality (VI) Variational inequalities can be regarded as generalizations of minimax optimization problems [20]. In convex-concave minimax optimization, finding a saddle point is equivalent to solving the Stampacchia Variational Inequality (SVI):
$$
\langle G (u ^ {\star}), u - u ^ {\star} \rangle \geq 0, \quad \forall u \in \mathcal {U}. \tag {5}
$$
Here  $u \coloneqq [x; y]$ ,  $u^{\star}$  is the optimal solution, and the operator  $G$  is a gradient operator:  $G(u) \coloneqq [\nabla_{x}f(x,y); -\nabla_{y}f(x,y)]$  with  $\mathcal{U} = \mathcal{X} \times \mathcal{Y}$ . The solution of (5) is referred to as a strong solution of
the VI corresponding to  $G$  and  $\mathcal{U}$  [31]. For the nonconvex-nonconcave minimax problem, without the monotonicity of  $G$ , the solution of SVI may not even exist. One alternative condition is to assume the existence of solutions  $u^{\star}$  for the Minty Variational Inequality (MVI):
$$
\langle G (u), u - u ^ {\star} \rangle \geq 0, \quad \forall u \in \mathcal {U}. \tag {6}
$$
The solution of (6) is called a weak solution of the VI [24]. In the setting where  $G$  is continuous and monotone, the solution sets of (5) and (6) are equivalent. However, these two solution sets are different in general and a weak solution may not exist when a strong solution exists. Many works have established the convergence results under the MVI condition or its variants [21, 28, 45, 43, 42, 17, 55, 8, 23]. Although MVI leads to convergence, it is hard to check in practice and is inapplicable to many functions (see examples in Section 4.3). A natural question is: Can we further relax the MVI condition to ensure convergence? One possible way is to relax the nonnegative lower bound to a negative one [33, 37, 11, 12, 21], i.e., the so-called weak MVI condition:
$$
\langle G (u), u - u ^ {\star} \rangle \geq - \frac {\rho}{2} \| G (u) \| ^ {2}, \quad \forall u \in \mathcal {U}.
$$
Here, we restricted  $\rho \in [0,\frac{1}{4L})$  and [21] proposed a Generalized extragradient method (Generalized EGM) with  $\mathcal{O}(\epsilon^{-2})$  iteration complexity. To include a wider function class, [50] enlarged the range of  $\rho$  to  $[0,\frac{1}{L})$  and  $\rho$  can be larger if more curvature information of  $f$  is involved. However, for general smooth nonconvex-nonconcave problems, various VI conditions are hard to check and  $\rho$  would easily violate the constraints. In this case, the proposed CurvatureEG+ still suffers from the limit cycle issue.
$\alpha$ -interaction dominant condition Another line of work is to impose the  $\alpha$ -interaction dominant conditions (7a), (7b) on  $f$ , i.e.,
$$
\begin{array}{l} \nabla_ {x x} ^ {2} f (x, y) + \nabla_ {x y} ^ {2} f (x, y) (\eta I - \nabla_ {y y} ^ {2} f (x, y)) ^ {- 1} \nabla_ {y x} ^ {2} f (x, y) \succeq \alpha I, (7a) \\ - \nabla_ {y y} ^ {2} f (x, y) + \nabla_ {y x} ^ {2} f (x, y) (\eta I + \nabla_ {x x} ^ {2} f (x, y)) ^ {- 1} \nabla_ {x y} ^ {2} f (x, y) \succeq \alpha I. (7b) \\ \end{array}
$$
Intuitively, this condition is to characterize how the interaction part of  $f$  affects the landscape of saddle envelope  $f_{\eta}(x,y) = \min_{z\in \mathcal{X}}\max_{v\in \mathcal{Y}}f(z,v) + \frac{\eta}{2}\| x - z\| ^2 -\frac{\eta}{2}\| y - v\| ^2$  [5]. We say  $\alpha$  is in the interaction dominant regime if  $\alpha$  in (7a), (7b) is a sufficiently large positive number and in the interaction weak regime when  $\alpha$  is a small but nonzero positive number. Convergence results for the damped proximal point method (Damped PPM) can only be obtained for these two regimes [29]. Otherwise, the method may fall into the limit cycle or even diverge. Unfortunately, conditions (7a) and (7b) only hold with  $\alpha = -L < 0$  for general  $L$ -smooth nonconvex-nonconcave function, which will dramatically restrict the problem class. Moreover, second-order information of  $f$  is required. For instance, if we choose Exponential Linear Units (ELU) with  $a = 1$  [14] as the activation function in neural networks,  $f$  is  $L$ -smooth but not twice differentiable. [29] studied the convergence of Damped PPM and showed that in the interaction dominant regime, their method converges with only one-sided dominance. In the interaction weak regime, their method also enjoys a local convergence rate of  $\mathcal{O}(\log (\epsilon^{-1}))$ . Taking computational efficiency into consideration, [30] developed the Damped EGM, which has an iteration complexity of  $\mathcal{O}(\log (\epsilon^{-1}))$  under two-sided dominance conditions.
There are four representative challenging nonconvex-nonconcave examples in the literature, which have been mentioned in Section 4.3.