Title: Universal Gradient Descent Ascent Method for Nonconvex-Nonconcave Minimax Optimization
Venue: NeurIPS 2023
Accepted: yes
Keywords: Nonconvex-Nonconcave Minimax Optimization

Limit Cycle
Abstract: Nonconvex-nonconcave minimax optimization has received intense attention over the last decade due to its broad applications in machine learning. Most existing algorithms rely on one-sided information, such as the convexity (resp. concavity) of the primal (resp. dual) functions, or other specific structures, such as the Polyak-Łojasiewicz (PŁ) and Kurdyka-Łojasiewicz (KŁ) conditions. However, verifying these regularity conditions is challenging in practice. To meet this challenge, we propose a novel universally applicable single-loop algorithm, the doubly smoothed gradient descent ascent method (DS-GDA), which naturally balances the primal and dual updates. That is, DS-GDA with the same hyperparameters is able to uniformly solve nonconvex-concave, convex-nonconcave, and nonconvex-nonconcave problems with one-sided KŁ properties, achieving convergence with $\mathcal{O}(\epsilon^{-4})$ complexity. Sharper (even optimal) iteration complexity can be obtained when the KŁ exponent is known. Specifically, under the one-sided KŁ condition with exponent $\theta\in(0,1)$, DS-GDA converges with an iteration complexity of $\mathcal{O}(\epsilon^{-2\max\\{2\theta,1\\}})$. They all match the corresponding best results in the literature. Moreover, we show that DS-GDA is practically applicable to general nonconvex-nonconcave problems even without any regularity conditions, such as the PŁ condition, KŁ condition, or weak Minty variational inequalities condition.  For various challenging nonconvex-nonconcave examples in the literature, including *Forsaken*, *Bilinearly-coupled minimax*, *Sixth-order polynomial*, and *PolarGame*, the proposed DS-GDA can all get rid of limit cycles. To the best of our knowledge, this is the first first-order algorithm to achieve convergence on all of these formidable problems.

Universal Gradient Descent Ascent Method for Nonconvex-Nonconcave Minimax Optimization

Abstract

Nonconvex-nonconcave minimax optimization has received intense attention over the last decade due to its broad applications in machine learning. Most existing algorithms rely on one-sided information, such as the convexity (resp. concavity) of the primal (resp. dual) functions, or other specific structures, such as the Polyak-Łojasiewicz (PŁ) and Kurdyka-Łojasiewicz (KŁ) conditions. However, verifying these regularity conditions is challenging in practice. To meet this challenge, we propose a novel universally applicable single-loop algorithm, the doubly smoothed gradient descent ascent method (DS-GDA), which naturally balances the primal and dual updates. That is, DS-GDA with the same hyperparameters is able to uniformly solve nonconvex-concave, convex-nonconcave, and nonconvex-nonconcave problems with one-sided KŁ properties, achieving convergence with O ( ϵ -4 ) complexity. Sharper (even optimal) iteration complexity can be obtained when the KŁ exponent is known. Specifically, under the one-sided KŁ condition with exponent θ ∈ (0 , 1) , DS-GDA converges with an iteration complexity of O ( ϵ -2 max { 2 θ, 1 } ) . They all match the corresponding best results in the literature. Moreover, we show that DS-GDA is practically applicable to general nonconvex-nonconcave problems even without any regularity conditions, such as the PŁ condition, KŁ condition, or weak Minty variational inequalities condition. For various challenging nonconvex-nonconcave examples in the literature, including ' Forsaken ', ' Bilinearly-coupled minimax ', ' Sixth-order polynomial ', and ' PolarGame ', the proposed DS-GDA can all get rid of limit cycles. To the best of our knowledge, this is the first first-order algorithm to achieve convergence on all of these formidable problems.

1 Introduction

In this paper, we are interested in studying nonconvex-nonconcave minimax problems of the form

where f : R n × R d → R is nonconvex in x and nonconcave in y , and X ⊆ R n , Y ⊆ R d are convex compact sets. Such problems have found significant applications in machine learning and operation research, including generative adversarial networks [27, 2], adversarial training [44, 53], multi-agent

Jiajin Li ∗

reinforcement learning [16, 48], and (distributionally) robust optimization [6, 19, 38, 25, 7], to name a few.

For smooth functions, one natural idea is to use gradient descent ascent (GDA) [41], which applies gradient descent on the primal function and gradient ascent on the dual function. However, GDA is originally designed for the strongly-convex-strongly-concave problem, where either primal or dual players can dominate the game. When applying it to the nonconvex-concave case, a so-called two-timescale method can make it converge. In this scenario, the primal player is the dominant player in the game. We can regard this one-step GDA scheme as an inexact subgradient descent on the inner max function max y ∈Y f ( x, y ) , thus it is necessary for the dual update to be relatively faster than the primal update at each iteration. However, this two-timescale GDA yields a high iteration complexity of O ( ϵ -6 ) . To achieve a lower iteration complexity, smoothed GDA (S-GDA) [62, 59] employs the Moreau-Yosida smoothing technique to stabilize the primal sequence. The resulting stabilized sequence can help S-GDA achieve a lower iteration complexity of O ( ϵ -4 ) . Alternating Gradient Projection (AGP) [57] can also achieve this lower iteration complexity by adding regularizers to both primal and dual functions. It is worth noting that the convergence of all these algorithms is heavily contingent on the convexity/concavity of primal/dual functions, leading to asymmetric updates. In the convex-nonconcave scenario, the roles are reversed, and the dual player takes dominance in the game. To apply the aforementioned algorithms, their updates should be modified accordingly. Specifically, for SGDA and AGP, the smoothing and regularized sides also need to be changed to guarantee the optimal convergence rate. However, when dealing with nonconvex-nonconcave problems, no player inherently dominates the other and all existing first-order algorithms cannot be guaranteed to converge to stationary points (see Definition 1), and they can even suffer from limit cycles . That is, the generated trajectories of all these algorithms will converge to cycling orbits that do not contain any stationary point of f . Such spurious convergence phenomena arise from the inherent minimax structure of (P) and have no counterpart in pure minimization problems. Conceptually, nonconvex-nonconcave minimax optimization problems can be understood as a seesaw game, which means no player inherently dominates the other. More explicitly, the key difficulty lies in adjusting the primal and dual updates to achieve a good balance. Most existing works address this challenge by adding additional regularity conditions to ensure the automatic domination of one player over the other. Specifically, research works either impose the global Polyak-Łojasiewicz (PŁ) condition on the dual function f ( x, · ) [59, 47, 22, 58] or assume the satisfaction of α -dominance condition [29, 30]. Both approaches fall into this line of research, enabling the adoption of algorithms and analysis for nonconvex-concave minimax problems. Although the recent work [40] extends the PŁ condition to the one-sided Kurdyka-Łojasiewicz (KŁ) property, it is still hard to provide the explicit KŁ exponent, and prior knowledge regarding which side satisfies the KŁ condition is required to determine the appropriate side to employ extrapolation. If we choose the wrong side, it will result in a slow convergence or even divergence (see Figure 5). On another front, variational inequality (VI) provides a unified framework for the study of equilibrium/minimax problems [46, 36, 26, 60, 45]. However, VI-related conditions are usually hard to check in practice. Hence, the convergence of the existing algorithms all highly rely on prior knowledge of the primal/dual function, which makes it paramount to design a universal algorithm for convex-nonconcave, nonconvex-concave, and nonconvex-nonconcave minimax problems.

We propose a new algorithm called the Doubly Smoothed Gradient Descent Ascent (DS-GDA) algorithm. DS-GDA builds upon S-GDA by applying the Moreau-Yosida smoothing technique to both the primal and dual variables, which allows a better trade-off between primal and dual updates. All hyperparameters, including the stepsize for gradient descent and ascent steps, and extrapolation parameters are carefully and explicitly controlled to ensure the sufficient descent property of a novel Lyapunov function that we introduce in our paper. The carefully selected variables automatically decide the dominant player, thereby achieving the balance of primal and dual updates by the interaction of parameters. Furthermore, the doubly smoothing technique enables the use of a set of symmetric parameters to achieve universality. This stands in sharp contrast to S-GDA, where only primal/dual function is smoothed. Specifically, the regularized function and the primal-dual updates in DS-GDA are inherently symmetric, which provides the possibility of applying the DS-GDA without prior information on the primal and dual functions.

To validate our idea and demonstrate the universality of DS-GDA, we provide a visual representation of the feasible symmetric parameter selections by relating the regularizer to Lipschitz constants and step sizes. This graphical illustration, depicted in Figure 1a, reveals that numerous parameter settings

can be chosen to guarantee convergence, which showcases the flexibility of DS-GDA. An experiment for a nonconvex-nonconcave problem has also been done to test the efficiency of using symmetric parameters.

Wealso evaluate the performance of DS-GDA on a range of representative and challenging nonconvexnonconcave problems from the literature, which violate all known regularity conditions. These include the 'Forsaken' example [32, Example 5.2], the 'Bilinearly-coupled minimax' example [29], the 'Sixth-order polynomial' example [18, 56, 13], and the 'PolarGame' example [50]. In all cases, DS-GDA successfully escapes limit cycles and converges to the desired stationary point, while other methods either suffer from the recurrence behavior or diverge. Moreover, our algorithm exhibits robustness to parameter selection (see Section 4.2), offering practical flexibility.

To corroborate its superior performance and have a better understanding of its convergence behaviors, we demonstrate that DS-GDA converges to a stationary point for nonconvex-concave, convex-nonconcave, and nonconvex-nonconcave problems that satisfy a one-sided KŁ property. By employing a single set of parameters, DS-GDA converges with an iteration complexity of O ( ϵ -4 ) across all these scenarios. Remarkably, DS-GDA achieves this without prior verification of these conditions. However, if we have access to a one-sided KŁ condition or knowledge of the convexity (concavity) of the primal (dual) function, the range of allowable parameters can be expanded. What is more, DS-GDA attains a lower or even optimal iteration complexity of O ( ϵ -2 max { 2 θ, 1 } ) when the one-sided KŁ property with exponent θ ∈ (0 , 1) is satisfied. Notably, these convergence results match the best results for single-loop algorithms when the dual function is concave [62, 40] or satisfies KŁ condition [40]. To the best of our knowledge, our work demonstrates, for the first time, the possibility of having a simple and unified single-loop algorithm for solving nonconvex-nonconcave, nonconvexconcave, and convex-nonconcave minimax problems. However, it remains an open question whether convergence results can be derived without any regularity conditions. This question is intriguing and warrants further theoretical investigation of nonconvex-nonconcave minimax problems in the future.

Our main contributions are summarized as follows:

(i) We present DS-GDA, the first universal algorithm for convex-nonconcave, nonconvex-concave, and nonconvex-nonconcave problems with one-sided KŁ property. A single set of parameters can be applied across all these scenarios, guaranteeing an iteration complexity of O ( ϵ -4 ) . With the KŁ exponent θ ∈ (0 , 1) of the primal or dual function, we improve the complexity to O ( ϵ -2 max { 2 θ, 1 } ) . Our current convergence analysis achieves the best-known results in the literature.

(ii) We demonstrate that DS-GDA converges on various challenging nonconvex-nonconcave problems, even when no regularity conditions are satisfied. This makes DS-GDA the first algorithm capable of escaping limit cycles in all these hard examples.

2 Doubly Smoothed GDA

In this section, we propose our algorithm (i.e., DS-GDA) for solving (P). To start with, we introduce the blanket assumption, which is needed throughout the paper.

Assumption 1 (Lipschitz gradient) The function f is continuously differentiable and there exist positive constant L x , L y > 0 such that for all x, x ′ ∈ X and y, y ′ ∈ Y

For simplicity, we assume L y = λL x = λL with λ > 0 .

For general smooth nonconvex-concave problems, a simple and natural algorithm is GDA, which suffers from oscillation even for the bilinear problem min x ∈ [ -1 , 1] max y ∈ [ -1 , 1] xy . To address this issue, a smoothed GDA algorithm that uses Moreau-Yosida smoothing techniques is proposed in [62]. Specifically, they introduced an auxiliary variable z and defined a regularized function as follows:

The additional quadratic term smooths the primal update. Consequently, the algorithm can achieve a better trade-off between primal and dual updates. We adapt the smoothing technique to the nonconvexnonconcave setting, where the balance of primal and dual updates is not a trivial task. To tackle this

problem, we also smooth the dual update by subtracting a quadratic term of dual variable and propose a new regularized function F : R n × R d × R n × R d → R as

with different smoothed parameters r 1 > L x , r 2 > L y for x and y , respectively. Then, our DS-GDA is formally presented in Algorithm 1.

Algorithm 1: Doubly Smoothed GDA (DS-GDA)

The choice of r 1 and r 2 is crucial for the convergence of the algorithm in both theoretical and practical senses. In particular, when r 1 = r 2 , it reduces to the proximal-point mapping proposed in [43] and inexact proximal point method (PPM) is only known to be convergent under certain VI conditions. Even with the exact computation of proximal mapping, PPM will diverge in the absence of regularity conditions [29]. By contrast, with an unbalanced r 1 and r 2 , our algorithm can always converge. The key insight here is to carefully adjust r 1 and r 2 to balance the primal-dual updates, ensuring the sufficient descent property of a novel Lyapunov function introduced in our paper. In fact, as we will show later in Section 3, r 1 and r 2 are typically not equal theoretically and practically. The two auxiliary variables z and v , which are updated by averaging steps, are also indispensable in our convergence proof. Intuitively, the exponential averaging applied to proximal variables z and v ensures they do not deviate too much from x and y , contributing to sequence stability.

We would like to highlight that the way we use the Moreau-Yosida smoothing technique is a notable departure from the usual approach. Smoothing techniques are commonly invoked when solving nonconvex-concave problems to achieve better iteration complexity [62, 40, 59]. However, we target at smoothing both the primal and dual variables with different magnitudes to ensure global convergence.

3 Convergence Analysis

The convergence result of the proposed DS-GDA (i.e., Algorithm 1) will be discussed in this section. To illustrate the main result, we first provide the stationarity measure in Definition 1.

Definition 1 (Stationarity measure) The point (ˆ x, ˆ y ) ∈ X × Y is said to be an

(i) ϵ -game stationary point (GS) if

(ii) ϵ -optimization stationary point (OS) if

3.1 Complexity under Nonconvex-(Non)concave Setting

Inspired by [62, 40], we consider a novel Lyapunov function Φ : R n × R d × R n × R d → R defined as follows:

where d ( y, z, v ) := min x ∈X F ( x, y, z, v ) , p ( z, v ) := max y ∈Y d ( y, z, v ) , q ( z ) := max v ∈ R d p ( z, v ) , and F := min z ∈ R n q ( z ) . Obviously, this Lyapunov function is lower bounded, that is, Φ ≥ F . To gain a better understanding of the rationale behind the construction of Φ , it is observed that the Lyapunov function has a strong connection to the updates of the iterations. The primal update corresponds to 'primal descent' and gradient ascent in dual variable corresponds to the 'dual ascent'. The averaging updates of proximal variables could be understood as an approximate gradient descent of p ( z, v ) and an approximate gradient ascent of g ( v ) , resulting in the 'proximal descent' and 'proximal ascent' terms in the Lyapunov function. Compared with that in [62, 40], we have an additional 'proximal ascent' term. It is introduced by the regularized term for dual variable in F and the update of proximal variable v . Essentially, the 'nonconcavity' of f ( x, · ) brings the additional term. With this Lyapunov function, we can establish the following basic descent property as our first important result.

Proposition 1 (Basic descent estimate) Suppose that Assumption 1 holds and r 1 ≥ 2 L , r 2 ≥ 2 λL with the parameters

Then for any t ≥ 0 ,

where σ := 2 cr 1 +1 c ( r 1 -L ) and L d := ( λL r 1 -L +2 ) λL + r 2 . Moreover, we have y + ( z, v ) := proj Y ( y + α ∇ y F ( x ( y, z, v ) , y, z, v )) and v + ( z ) := v + µ ( y ( x ( z, v ) , z, v ) -v ) with the following definitions: (i) x ( y, z, v ) := argmin x ∈X F ( x, y, z, v ) , (ii) y ( x, z, v ) := argmax y ∈Y F ( x, y, z, v ) , (iii) x ( z, v ) := argmin x ∈X max y ∈Y F ( x, y, z, v ) , (iv) v ( z ) := argmax v ∈ R d P ( z, v ) .

The lower bound of Φ by F is established by its construction, so the crux of proving subsequence convergence is to establish the decreasing property of the Lyapunov function. Although Proposition 1 quantifies the variation of the Lyapunov function values between two consecutive iterates, there is a negative error term ∥ x ( z t +1 , v ( z t +1 )) -x ( z t +1 , v t + ( z t +1 )) ∥ that makes the decreasing property of Φ unclear.

Next, we characterize the negative error term in terms of other positive terms and then exhibit the sufficient descent property by bounding the coefficients. Conceptually, the error term is related to ∥ v t + ( z t +1 ) -v ( z t +1 ) ∥ by the Lipschitz property of the solution mapping x ( z, · ) . However, ∥ v t + ( z t +1 ) -v ( z t +1 ) ∥ may not be a suitable surrogate since it includes the information about the optimal solution v ( z t +1 ) . Fortunately, with the help of the global KŁ property or concavity for the dual function (see Assumption 2 and 3), we can bound the negative error term by ∥ v t + ( z t +1 ) -v ( z t +1 ) ∥ (called the proximal error bound). The explicit form of this bound is provided in the following Propositions 2.

Assumption 2 (KŁ property with exponent θ of the dual function) For any fixed point x ∈ X , the problem max y ∈Y f ( x, y ) has a nonempty solution set and a finite optimal value. Moreover, there

exist τ > 0 , θ ∈ (0 , 1) such that for any x ∈ X , y ∈ Y

Assumption 3 (Concavity of the dual function) For any fixed point x ∈ X , f ( x, · ) is concave.

Proposition 2 (Proximal error bound) Under the setting of Proposition 1 with Assumption 2 or 3, for any z ∈ R n , v ∈ R d one has (i) KŁ exponent θ ∈ (0 , 1) :

(ii) Concave:

where ω 0 := 2 ( r 1 -L ) τ ( r 2 (1 -µ ) µ + r 2 2 r 2 -λL ) 1 θ and ω 1 := 4 r 2 diam( Y ) r 1 -L ( 1 -µ µ + r 2 r 2 -λL ) . Here, diam( Y ) denotes the diameter of the set Y .

Armed with Proposition 1 and Proposition 2, we establish the main theorem concerning the iteration complexity of DS-GDA with respect to the above-mentioned standard stationarity measure for (P).

Theorem 1 (Iteration complexity for nonconvex-(non)concave problems) Under the setting of Theorem 1 and Proposition 2, for any T > 0 , there exists a t ∈ { 1 , 2 , . . . , T } such that (i) KŁ exponent θ ∈ ( 1 2 , 1) : ( x t +1 , y t +1 ) is an O ( T -1 4 θ ) -GS and z t +1 is an O ( T -1 4 θ ) -OS if β ≤ O ( T -2 θ -1 2 θ ) ;

Moreover, when the problem (P) equipped with the widely fulfilled semi-algebraic structure [9, 4, 15] and the dual function satisfies the KŁ property with θ ∈ (0 , 1 2 ] , we additionally have the following sequential convergence result of the DS-GDA.

Theorem 2 (Last-iterate convergence of DS-GDA) Consider the setting of Theorem 1 and suppose that Assumption 2 holds with θ ∈ (0 , 1 2 ] . Suppose that f ( · , y ) is semi-algebraic and X , Y are semialgebraic sets. Then, the sequence { ( x t , y t , z t , v t ) } converges to ( x ∗ , y ∗ , z ∗ , v ∗ ) , where ( x ∗ , y ∗ ) is a GS and z ∗ is an OS.

3.2 Universal Results

For convex-nonconcave or nonconvex-nonconcave minimax problems in which the primal function satisfies the KŁ property, analogous results to Theorem 1 can be established. This can be accomplished by introducing an alternative Lyapunov function Ψ : R n × R d × R n × R d → R as follows:

It is worth noting that our Lyapunov function exhibits symmetry with respect to nonconvex(non)concave problems, since the adjustment only entails interchanging the position between ( d ( y, z, v ) , q ( z )) and ( h ( x, z, v ) , g ( v )) . Therefore, similar convergence results as Theorem 1 could be derived without any effort. A more detailed proof can be found in Appendix H.

Based on these results, we are ready to show that our DS-GDA is a universal algorithm. By incorporating the choices of parameters, we can identify a consistent set of parameters that ensures the convergence of DS-GDA in nonconvex-nonconcave, nonconvex-concave, and convex-nonconcave problems. The universal convergence rate is stated as follows:

Theorem 3 (Universal convergence of DS-GDA) Without loss of generality, we consider the case where λ = 1 , implying L x = L y = L . For convex-nonconcave, nonconvex-concave, and nonconvexnonconcave minimax problems that satisfy the one-sided KŁ property, if we further set r 1 = r 2 = r ≥ 20 L , DS-GDA converges provided certain parameter conditions are met:

When KŁ exponent θ ∈ (0 , 1 2 ] , we further require

where ω 0 and ω 2 are the coefficients in Propositions 2 and 6. Then, ( x t +1 , y t +1 ) is an O ( T -1 4 ) -GS.

Remark 2 It is worth noting that our results are more general compared to AGP in [57], where a unified analysis for convex-nonconcave and nonconvex-concave problems is provided. Here, our algorithm can also be applied to the nonconvex-nonconcave problem with one-sided KŁ property. Moreover, our algorithm is universal, meaning that one single set of parameters can ensure convergence across all these scenarios. By contrast, different choices of parameters are required to guarantee optimal convergence in AGP.

4 Empirical Validation of DS-GDA

4.1 Universality of DS-GDA

To validate the universality of DS-GDA, we commence by providing a graphical description of feasible regions for the choice of parameters. Subsequently, employing a set of symmetric parameters, we will show that the KŁ-nonconvex problems can converge to the desired stationary point, which supports our universality results.

Without loss of generality, we consider the case where λ = 1 . In the pursuit of symmetric parameter selection, we initially fix β = µ = 1 / 5000 , thus reducing the problem to determining only two remaining parameters: c and r . We then explore their relationships by setting r 1 = r 2 = t 2 L and c = α = 1 / ( t 1 r ) . Restricting 0 ≤ t 1 , t 2 ≤ 100 , the feasible choices of t 1 and t 2 are selected to guarantee the first four coefficients in basic descent estimate (1) are positive. As visually depicted in Figure 1a, it becomes apparent that a large number of choices for r and c are available to ensure convergence.

Next, we test our algorithm on a nonconvex-nonconcave problem that satisfies the one-sided KŁ property. With a set of symmetric parameters, we find that DS-GDA can easily converge to the desired stationary point. We first introduce the KŁ-nonconcave problems as follows:

KŁ-Nonconcave Example The following example satisfies two-sided KŁ property with exponent 1 2 , which is studied in [58]:

From Figure 1b, we can observe that symmetric parameter selection is effective for addressing KŁ-nonconcave problems. To be specific, by setting r 1 = r 2 = 0 . 125 , c = α = 0 . 04 , β = µ = 0 . 8 , DS-GDA directly converges to the saddle point, which validates the universal results in Theorem 3.

4.2 Robustness of DS-GDA

In this section, we compare the convergence performance of DS-GDA with the closely related algorithm S-GDA on some illustrative examples. Additionally, we report the range of parameters for these two algorithms to demonstrate the robustness of DS-GDA. To begin, we present some simple polynomial examples.

(a) Feasible region of different

t

1

and

t

2

(b) Convergence of KŁ-nonconcave problem

100

90

80

70

Feasible

60

t2

50

Infeasible

40

30

20

10

0

0

10

20

30

40

50

60

70

80

90

100

t1

0.5

Initialization

Trajectories

0

-0.5

-1

-0.5

0

0.5

Convex-Nonconcave Example The following example is convex-nonconcave, which is studied in [13, 1]:

where X = Y = { z : -1 ≤ z ≤ 1 } and u ∗ = [ x ∗ ; y ∗ ] = [0; 0] is the only stationary point.

KŁ-Nonconcave Example The KŁ-nonconcave example considered here is mentioned in Section 4.1 (see equation (2)). The u ∗ = [ x ∗ ; y ∗ ] = [0; 0] is a saddle point and the only stationary point.

Nonconvex-Nonconcave Example The nonconvex-nonconcave example considered here is the 'Bilinearly-coupled Minimax' example (3) discussed in [29]:

where f ( z ) = ( z +1)( z -1)( z +3)( z -3) , A = 11 , and X = Y = { z : -4 ≤ z ≤ 4 } . It does not satisfy any existing regularity condition, and the point u ∗ = [ x ∗ ; y ∗ ] = [0; 0] is the only stationary point.

The extrapolation parameters z and v are initialized as x and y , respectively. According to Lemma 8, our algorithm terminates when ∥ z -v ∥ and ∥ y -v ∥ are less than 10 -6 or when the number of iterations exceeds 10 7 . To ensure a fair comparison, we use the same initializations for DS-GDA and S-GDA in each test. The algorithm parameters are tuned so that both DS-GDA and S-GDA are optimal. In other words, they can converge to a stationary point in a minimum number of iterations. We compare the convergence of two algorithms by plotting the iteration gap ∥ u k -u ∗ ∥ against the number of iterations for each example, where u ∗ denotes the stationary point. Our results show that DS-GDA and S-GDA have similar convergence performance when the primal function is convex or satisfies the KŁ property, as depicted in Figure 2a and 2b. However, for the nonconvex-nonconcave example where no regularity condition is satisfied, DS-GDA achieves much faster convergence than S-GDA, as shown in Figure 2c.

To demonstrate the robustness of the proposed DS-GDA, we present feasible regions of all common hyperparameters in Figure 3. We tune the lower and upper bounds of each parameter while keeping other parameters fixed at their optimal values. As illustrated in Figures 3a and 3b, for examples that satisfy the one-sided KŁ property or with a convex primal function, the implementable ranges of DS-GDA and S-GDA for common parameters are roughly similar. In addition, the two auxiliary parameters for DS-GDA, i.e., r 2 and µ , have relatively wide ranges of values, indicating that they allow for flexibility in their selection. However, for nonconvex-nonconcave problems, DS-GDA exhibits a wider range of viable parameter values compared to S-GDA. This observation highlights the robustness of DS-GDA when it comes to parameter selection (refer to Figure 3c).

10 0

10 1

Iterations

10 -6

10 -4

10 -2

10 0

(a) Convex-Nonconcave

1

2

3

4

5

6

7

8

Iterations

10 -6

10 -5

10 -4

10 -3

10 -2

10 -1

10 0

10 1

(b) KŁ -Nonconcave

10 0

10 2

10 4

10 6

Iterations

10 -6

10 -4

10 -2

10 0

DSGDA

SGDA

(c) Nonconvex-Nonconcave

r

1

c

Parameters

0

2

4

6

8

Range of parameters

(a) Convex-Nonconcave

r

1

c

Parameters

0

2

4

6

8

10

Range of parameters

Parameters

0

0.2

0.4

0.6

Range of parameters

(b) KŁ -Nonconcave

r

1

c

Parameters

10

-6

10

-4

10

-2

10

0

10

2

Range of parameters

DSGDA

SGDA

(c) Nonconvex-Nonconcave

4.3 Effectiveness of Getting Rid of Limit Cycle

In this section, we demonstrate the effectiveness of the proposed DS-GDA on some illustrative examples that are widely used in literature. Notably, they do not satisfy any of the regularity conditions in previous literature (i.e., KŁ condition, weak MVI, and α -dominant condition). We refer the readers to Appendix L for details on how to check the failure of these conditions for these examples. In addition to the violation of regularity conditions, it has been verified that none of the existing first-order algorithms can achieve convergence for all four examples. The detailed description of the four examples can be found in Appendix B.

To showcase the convergence behavior and effectiveness of the proposed DS-GDA, we compare it with three other state-of-the-art methods, that is, damped extragradient method (Damped EGM) [30], S-GDA [40, 62], and generalized curvature extragradient method (CurvatureEG+) [50]. Damped EGM is guaranteed to converge under α -dominant condition, and S-GDA converges when the dual function is concave. CurvatureEG+ could converge under weak MVI condition, which is the weakest variational inequality-based condition as far as we know in the literature.

For the experiments, we use the same initializations of primal and dual variables for all four methods to ensure a fair comparison. For DS-GDA, the exponentially weighted variables are initialized as the same values of the primal and dual variables, respectively. We stop DS-GDA when the differences between primal-dual variables and the corresponding exponentially weighted variables are all less than 10 -6 or when the number of iterations exceeds 10 5 . For other baseline methods, we terminate them when either the number of iterates reaches 10 5 or the difference between two consecutive iterates is less than 10 -6 .

Figure 4 shows the trajectories of different methods with various initializations for the aforementioned four examples. We observe from the first column that our DS-GDA successfully gets rid of limit cycles in all examples. While S-GDA exhibits similar performance as DS-GDA, it is still not as potent in terms of its overall effectiveness. Specifically, in the case of the 'Bilinearly-coupled minimax' example, S-GDA gets trapped in a limit cycle, while our DS-GDA successfully avoids it and achieves convergence. The figure in the second column provides more details on this comparison. With the violation of the of α -interaction dominance condition, damped EGM either suffers from the spurious cycling convergence phenomenon or diverges to a point on the boundary (see the third row in the fourth column). It converges only when the initialization is very close to the stationary point (see the fourth row in the last column). Similar results are observed for CurvatureEG+ (see Figures in the third column for details). Thus, the proposed DS-GDA outperforms other methods. It is the only

0

DS-GDA

S-GDA

CurvatureEG+

Damped EGM

Bilinear-coupled

6-order poly

PolarGame

-2

0

2

-2

0

2

-2

2

1.5

1.5

1.5

1.5

0.5

-0.5

0.5

0.5

0.5

-1.5

-1.5

-1.5

-1.5

-1.5

S'0-

0.5

1.5-1.5

-0.5

0.5

1.5

-1.5

-0.5

0.5

1.5-1.5

-0.5

0.5

1.5

2F

2.5F

2.5F

1.25

1.25

0

0

-1.25

-1.25

-2.5

-2.5

0

-2.5

-1.25

0

1.25

2.5

-2.5

-1.25

0

1.25

2.5

Initialization

Trajectories

0.5

0.5

0.5

-0.5

0.5

0.5

0.5

-0.5

0

0.5

L-

-0.5

0

0.5

1

-1

-0.5

0

0.5

-1

-0.5

0

0.5

algorithm that can get rid of the limit cycle and enjoy global convergence for all these challenging examples.

5 Conclusion

In this paper, we propose a single-loop algorithm called the doubly smoothed gradient descent ascent (DS-GDA) algorithm, which offers a natural balance between primal-dual updates for constrained nonconvex-nonconcave minimax problems. This is the first simple and universal algorithm for nonconvex-concave, convex-nonconcave, and nonconvex-nonconcave problems with one-sided KŁ property. By employing a single set of parameters, DS-GDA achieves convergence with an iteration complexity of O ( ϵ -4 ) across all these scenarios. Sharper iteration complexity can be obtained when the one-sided KŁ property is satisfied with an exponent θ ∈ (0 , 1) , matching the state-of-the-art results. We further conduct experiments to validate the universality and efficiency of DS-GDA for avoiding the limit cycle phenomenon, which commonly occurs in challenging nonconvex-nonconcave examples. There is still a gap between theory and practice, and it would be intriguing to explore the possibility of achieving global convergence for DS-GDA without any regularity conditions. This opens up new avenues for research on nonconvex-nonconcave minimax problems.

Acknowledgments and Disclosure of Funding

Anthony Man-Cho So is supported in part by the Hong Kong Research Grants Council (RGC) General Research Fund (GRF) projects CUHK 14203920 and 14216122. Jose Blanchet and Jiajin Li are supported by the Air Force Office of Scientific Research under award number FA9550-20-1-0397 and NSF 1915967, 2118199, 2229012, 2312204.

References

Leonard Adolphs, Hadi Daneshmand, Aurelien Lucchi, and Thomas Hofmann. Local saddle point optimization: A curvature exploitation approach. In The 22nd International Conference on Artificial Intelligence and Statistics , pages 486-495. PMLR, 2019.

Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In International Conference on Machine Learning , pages 214-223. PMLR, 2017.

Hédy Attouch, Jérôme Bolte, Patrick Redont, and Antoine Soubeyran. Proximal alternating minimization and projection methods for nonconvex problems: An approach based on the kurdyka-Łojasiewicz inequality. Mathematics of Operations Research , 35(2):438-457, 2010.

Hedy Attouch, Jérôme Bolte, and Benar Fux Svaiter. Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods. Mathematical Programming , 137(1-2):91-129, 2013.

Hédy Attouch and Roger J-B Wets. A convergence theory for saddle functions. Transactions of the American Mathematical Society , 280(1):1-41, 1983.

Aharon Ben-Tal, Laurent El Ghaoui, and Arkadi Nemirovski. Robust optimization , volume 28. Princeton University Press, 2009.

Dimitris Bertsimas, David B Brown, and Constantine Caramanis. Theory and applications of robust optimization. SIAM review , 53(3):464-501, 2011.

Axel Böhm. Solving nonconvex-nonconcave min-max problems exhibiting weak minty solutions. arXiv preprint arXiv:2201.12247 , 2022.

Jérôme Bolte, Aris Daniilidis, and Adrian Lewis. The Łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems. SIAM Journal on Optimization , 17(4):1205-1223, 2007.

Jérôme Bolte, Shoham Sabach, and Marc Teboulle. Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Mathematical Programming , 146(1-2):459-494, 2014.

Yang Cai, Argyris Oikonomou, and Weiqiang Zheng. Accelerated algorithms for monotone inclusions and constrained nonconvex-nonconcave min-max optimization. arXiv preprint arXiv:2206.05248 , 2022.

Yang Cai and Weiqiang Zheng. Accelerated single-call methods for constrained min-max optimization. arXiv preprint arXiv:2210.03096 , 2022.

Raphael Chinchilla, Guosong Yang, and Joao P Hespanha. Newton and interior-point methods for (constrained) nonconvex-nonconcave minmax optimization with stability guarantees. arXiv preprint arXiv:2205.08038 , 2022.

Djork-Arné Clevert, Thomas Unterthiner, and Sepp Hochreiter. Fast and accurate deep network learning by exponential linear units (elus). arXiv preprint arXiv:1511.07289 , 2015.

Michel Coste. An introduction to o-minimal geometry . Istituti editoriali e poligrafici internazionali Pisa, 2000.

Bo Dai, Albert Shaw, Lihong Li, Lin Xiao, Niao He, Zhen Liu, Jianshu Chen, and Le Song. Sbeed: Convergent reinforcement learning with nonlinear function approximation. In International Conference on Machine Learning , pages 1125-1134. PMLR, 2018.

Cong D Dang and Guanghui Lan. On the convergence properties of non-euclidean extragradient methods for variational inequalities with generalized monotone operators. Computational Optimization and Applications , 60(2):277-310, 2015.

Constantinos Daskalakis, Noah Golowich, Stratis Skoulakis, and Manolis Zampetakis. Stayon-the-ridge: Guaranteed convergence to local minimax equilibrium in nonconvex-nonconcave games. arXiv preprint arXiv:2210.09769 , 2022.

Erick Delage and Yinyu Ye. Distributionally robust optimization under moment uncertainty with application to data-driven problems. Operations Research , 58(3):595-612, 2010.

Vladimir Fedorovich Dem'yanov and Aleksandr Borisovich Pevnyi. Numerical methods for finding saddle points. USSR Computational Mathematics and Mathematical Physics , 12(5):1152, 1972.

Jelena Diakonikolas, Constantinos Daskalakis, and Michael I Jordan. Efficient methods for structured nonconvex-nonconcave min-max optimization. In International Conference on Artificial Intelligence and Statistics , pages 2746-2754. PMLR, 2021.

Thinh Doan. Convergence rates of two-time-scale gradient descent-ascent dynamics for solving nonconvex min-max problems. In Learning for Dynamics and Control Conference , pages 192-206. PMLR, 2022.

Zehao Dou and Yuanzhi Li. On the one-sided convergence of adam-type algorithms in nonconvex non-concave min-max optimization. arXiv preprint arXiv:2109.14213 , 2021.

Francisco Facchinei and Christian Kanzow. Generalized nash equilibrium problems. 4or , 5(3):173-210, 2007.

Rui Gao, Xi Chen, and Anton J Kleywegt. Wasserstein distributionally robust optimization and variation regularization. Operations Research , 2022.

Gauthier Gidel, Hugo Berard, Gaëtan Vignoud, Pascal Vincent, and Simon Lacoste-Julien. A variational inequality perspective on generative adversarial networks. arXiv preprint arXiv:1802.10551 , 2018.

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM , 63(11):139-144, 2020.

Eduard Gorbunov, Adrien Taylor, Samuel Horváth, and Gauthier Gidel. Convergence of proximal point and extragradient-based methods beyond monotonicity: the case of negative comonotonicity. arXiv preprint arXiv:2210.13831 , 2022.

Benjamin Grimmer, Haihao Lu, Pratik Worah, and Vahab Mirrokni. The landscape of the proximal point method for nonconvex-nonconcave minimax optimization. arXiv preprint arXiv:2006.08667 , 2020.

Saeed Hajizadeh, Haihao Lu, and Benjamin Grimmer. On the linear convergence of extra-gradient methods for nonconvex-nonconcave minimax problems. arXiv preprint arXiv:2201.06167 , 2022.

Philip Hartman and Guido Stampacchia. On some non-linear elliptic differential-functional equations. Acta mathematica , 115:271-310, 1966.

Ya-Ping Hsieh, Panayotis Mertikopoulos, and Volkan Cevher. The limits of min-max optimization algorithms: Convergence to spurious non-critical sets. In International Conference on Machine Learning , pages 4337-4348. PMLR, 2021.

Alfredo N Iusem, Alejandro Jofré, Roberto Imbuzeiro Oliveira, and Philip Thompson. Extragradient method with variance reduction for stochastic variational inequalities. SIAM Journal on Optimization , 27(2):686-724, 2017.

Chi Jin, Praneeth Netrapalli, and Michael Jordan. What is local optimality in nonconvexnonconcave minimax optimization? In International conference on machine learning , pages 4880-4889. PMLR, 2020.

Hamed Karimi, Julie Nutini, and Mark Schmidt. Linear convergence of gradient and proximalgradient methods under the Polyak-Łojasiewicz condition. In Joint European conference on machine learning and knowledge discovery in databases , pages 795-811. Springer, 2016.

Galina M Korpelevich. The extragradient method for finding saddle points and other problems. Matecon , 12:747-756, 1976.

Sucheol Lee and Donghwan Kim. Fast extra gradient methods for smooth structured nonconvexnonconcave minimax problems. Advances in Neural Information Processing Systems , 34:2258822600, 2021.

Daniel Levy, Yair Carmon, John C Duchi, and Aaron Sidford. Large-scale methods for distributionally robust optimization. Advances in Neural Information Processing Systems , 33:8847-8860, 2020.

Guoyin Li and Ting Kei Pong. Calculus of the exponent of Kurdyka-Łojasiewicz inequality and its applications to linear convergence of first-order methods. Foundations of Computational Mathematics , 18(5):1199-1232, 2018.

Jiajin Li, Linglingzhi Zhu, and Anthony Man-Cho So. Nonsmooth nonconvex-nonconcave minimax optimization: Primal-dual balancing and iteration complexity analysis. arXiv preprint arXiv:2209.10825 , 2022.

Tianyi Lin, Chi Jin, and Michael Jordan. On gradient descent ascent for nonconvex-concave minimax problems. In International Conference on Machine Learning , pages 6083-6093. PMLR, 2020.

Mingrui Liu, Youssef Mroueh, Jerret Ross, Wei Zhang, Xiaodong Cui, Payel Das, and Tianbao Yang. Towards better understanding of adaptive gradient algorithms in generative adversarial nets. arXiv preprint arXiv:1912.11940 , 2019.

Mingrui Liu, Hassan Rafique, Qihang Lin, and Tianbao Yang. First-order convergence theory for weakly-convex-weakly-concave min-max problems. Journal of Machine Learning Research , 22:169-1, 2021.

Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 , 2017.

Panayotis Mertikopoulos, Bruno Lecouat, Houssam Zenati, Chuan-Sheng Foo, Vijay Chandrasekhar, and Georgios Piliouras. Optimistic mirror descent in saddle-point problems: Going the extra (gradient) mile. arXiv preprint arXiv:1807.02629 , 2018.

Arkadi Nemirovski. Prox-method with rate of convergence o (1 /t ) for variational inequalities with lipschitz continuous monotone operators and smooth convex-concave saddle point problems. SIAM Journal on Optimization , 15(1):229-251, 2004.

Maher Nouiehed, Maziar Sanjabi, Tianjian Huang, Jason D Lee, and Meisam Razaviyayn. Solving a class of non-convex min-max games using iterative first order methods. Advances in Neural Information Processing Systems , 32, 2019.

Shayegan Omidshafiei, Jason Pazis, Christopher Amato, Jonathan P How, and John Vian. Deep decentralized multi-task multi-agent reinforcement learning under partial observability. In International Conference on Machine Learning , pages 2681-2690. PMLR, 2017.

Jong-Shi Pang. A posteriori error bounds for the linearly-constrained variational inequality problem. Mathematics of Operations Research , 12(3):474-484, 1987.

Thomas Pethick, Panagiotis Patrinos, Olivier Fercoq, Volkan Cevherå, et al. Escaping limit cycles: Global convergence for constrained nonconvex-nonconcave minimax problems. In International Conference on Learning Representations , 2022.

Boris T Polyak. Gradient methods for solving equations and inequalities. USSR Computational Mathematics and Mathematical Physics , 4(6):17-32, 1964.

R Tyrrell Rockafellar and Roger J-B Wets. Variational analysis , volume 317. Springer Science &Business Media, 2009.

Aman Sinha, Hongseok Namkoong, Riccardo Volpi, and John Duchi. Certifying some distributional robustness with principled adversarial training. arXiv preprint arXiv:1710.10571 , 2017.

Maurice Sion. On general minimax theorems. Pacific Journal of Mathematics , 8(1):171-176, 1958.

Chaobing Song, Zhengyuan Zhou, Yichao Zhou, Yong Jiang, and Yi Ma. Optimistic dual extrapolation for coherent non-monotone variational inequalities. Advances in Neural Information Processing Systems , 33:14303-14314, 2020.

Yuanhao Wang, Guodong Zhang, and Jimmy Ba. On solving minimax optimization locally: A follow-the-ridge approach. arXiv preprint arXiv:1910.07512 , 2019.

Zi Xu, Huiling Zhang, Yang Xu, and Guanghui Lan. A unified single-loop alternating gradient projection algorithm for nonconvex-concave and convex-nonconcave minimax problems. arXiv preprint arXiv:2006.02032 , 2020.

Junchi Yang, Negar Kiyavash, and Niao He. Global convergence and variance-reduced optimization for a class of nonconvex-nonconcave minimax problems. arXiv preprint arXiv:2002.09621 , 2020.

Junchi Yang, Antonio Orvieto, Aurelien Lucchi, and Niao He. Faster single-loop algorithms for minimax optimization without strong concavity. In International Conference on Artificial Intelligence and Statistics , pages 5485-5517. PMLR, 2022.

TaeHo Yoon and Ernest K Ryu. Accelerated algorithms for smooth convex-concave minimax problems with o (1 /k 2 ) rate on squared gradient norm. In International Conference on Machine Learning , pages 12098-12109. PMLR, 2021.

Peiran Yu, Guoyin Li, and Ting Kei Pong. Kurdyka-Łojasiewicz exponent via inf-projection. Foundations of Computational Mathematics , 22(4):1171-1217, 2022.

Jiawei Zhang, Peijun Xiao, Ruoyu Sun, and Zhiquan Luo. A single-loop smoothed gradient descent-ascent algorithm for nonconvex-concave min-max problems. Advances in Neural Information Processing Systems , 33:7377-7389, 2020.

A Organization of the Appendix

We organize the appendix as follows:

The related work and the four challenging nonconvex-nonconcave examples are listed in Section B;

Notations and some useful lemmas such as Lipschitz error bounds are provided in Section C;

The characterization of changes in the Lyapunov function between successive iterations is established in Section D;

The proof of Proposition 1 is given in Section E;

The proof of proximal and dual error bounds are given in Section F;

The proof of Theorem 1 is given in Section G;

The convergence proof for nonconvex(convex)-nonconcave problem is provided in Section H.

The proof of Theorem 2 is provided in Section I;

The proof of Theorem 3 is provided in Section J;

The quantitative relationship between different notions of the stationary point is provided in Section K;

The properties of examples mentioned in Section 4.3 are checked in Section L. We also show that the wrong selection of smoothing sides will result in slow convergence for S-GDA.

B Related Works

There are three representative types of regularity conditions in the literature to restrict the problem class that algorithms are developed to get rid of the limit cycle.

Polyak-Łojasiewicz (PŁ) condition The PŁ condition (4) was originally proposed by [51] and is a crucial tool for establishing linear convergence of first-order algorithms for pure minimization problems [35]. Suppose that the problem max x ∈ R d h ( x ) has a nonempty solution set and a finite optimal value. The PŁ condition states that there exist a constant µ > 0 such that for any x ∈ R d ,

There is a host of works trying to invoke the PŁ condition on the dual function f ( x, · ) [59, 47, 22, 58]. Unfortunately, we would like to point out that this condition is too restrictive and inherently avoid the main difficulty in addressing general nonconvex-nonconcave minimax problems. With PŁ condition imposed on the dual function, the inner maximization value function ϕ ( · ) = max y ∈Y f ( · , y ) is L -smooth [47, Lemma A.5]. Thus, the dual update can naturally be controlled by the primal since we can regard minimax problems as pure smooth (weakly convex) minimization problems over x . However, for general cases, the inner value function ϕ may not even be Lipschitz. Recently, [47] proposed a so-called multi-step GDA method with an iteration complexity of O (log( ϵ -1 ) ϵ -2 ) . [22] further developed the single-loop two-timescale GDA method to better take the computational tractability into account and the complexity is improved to O ( ϵ -2 ) . Following the smoothing (extrapolation) technique developed in [62], [59] extended the proposed smoothed GDA to the stochastic setting and obtained an iteration complexity of O ( ϵ -4 ) .

Varitional Inequality (VI) Variational inequalities can be regarded as generalizations of minimax optimization problems [20]. In convex-concave minimax optimization, finding a saddle point is equivalent to solving the Stampacchia Variational Inequality (SVI):

Here u := [ x ; y ] , u ⋆ is the optimal solution, and the operator G is a gradient operator: G ( u ) := [ ∇ x f ( x, y ); -∇ y f ( x, y )] with U = X × Y . The solution of (5) is referred to as a strong solution of

the VI corresponding to G and U [31]. For the nonconvex-nonconcave minimax problem, without the monotonicity of G , the solution of SVI may not even exist. One alternative condition is to assume the existence of solutions u ⋆ for the Minty Variational Inequality (MVI):

The solution of (6) is called a weak solution of the VI [24]. In the setting where G is continuous and monotone, the solution sets of (5) and (6) are equivalent. However, these two solution sets are different in general and a weak solution may not exist when a strong solution exists. Many works have established the convergence results under the MVI condition or its variants [21, 28, 45, 43, 42, 17, 55, 8, 23]. Although MVI leads to convergence, it is hard to check in practice and is inapplicable to many functions (see examples in Section 4.3 ). A natural question is: Can we further relax the MVI condition to ensure convergence ? One possible way is to relax the nonnegative lower bound to a negative one [33, 37, 11, 12, 21], i.e., the so-called weak MVI condition:

Here, we restricted ρ ∈ [0 , 1 4 L ) and [21] proposed a Generalized extragradient method (Generalized EGM) with O ( ϵ -2 ) iteration complexity. To include a wider function class, [50] enlarged the range of ρ to [0 , 1 L ) and ρ can be larger if more curvature information of f is involved. However, for general smooth nonconvex-nonconcave problems, various VI conditions are hard to check and ρ would easily violate the constraints. In this case, the proposed CurvatureEG+ still suffers from the limit cycle issue.

α -interaction dominant condition Another line of work is to impose the α -interaction dominant conditions (7a), (7b) on f , i.e.,

Intuitively, this condition is to characterize how the interaction part of f affects the landscape of saddle envelope f η ( x, y ) = min z ∈X max v ∈Y f ( z, v ) + η 2 ∥ x -z ∥ 2 -η 2 ∥ y -v ∥ 2 [5]. We say α is in the interaction dominant regime if α in (7a), (7b) is a sufficiently large positive number and in the interaction weak regime when α is a small but nonzero positive number. Convergence results for the damped proximal point method (Damped PPM) can only be obtained for these two regimes [29]. Otherwise, the method may fall into the limit cycle or even diverge. Unfortunately, conditions (7a) and (7b) only hold with α = -L < 0 for general L -smooth nonconvex-nonconcave function, which will dramatically restrict the problem class. Moreover, second-order information of f is required. For instance, if we choose Exponential Linear Units (ELU) with a = 1 [14] as the activation function in neural networks, f is L -smooth but not twice differentiable. [29] studied the convergence of Damped PPM and showed that in the interaction dominant regime, their method converges with only one-sided dominance. In the interaction weak regime, their method also enjoys a local convergence rate of O (log( ϵ -1 )) . Taking computational efficiency into consideration, [30] developed the Damped EGM, which has an iteration complexity of O (log( ϵ -1 )) under two-sided dominance conditions.

There are four representative challenging nonconvex-nonconcave examples in the literature, which have been mentioned in Section 4.3.

'Bilinearly-coupled minimax' example The first one is the 'Bilinearly-coupled Minimax' example (3) with A = 10 . It is a representative example to showcase the limit cycle phenomenon as it breaks the α -dominant condition. When the bilinear intersection term between primal and dual variables, i.e., x and y , is moderate, it becomes uncertain which variable, either the primal or dual, holds dominance over the other. As a result, this particular example poses a great challenge in terms of ensuring convergence.

'Forsaken' example The second one is the 'Forsaken' example considered in [32, Example 5.2], i.e.,

where ϕ ( z ) = 1 4 z 2 -1 2 z 4 + 1 6 z 6 and X = Y = { z : -1 . 5 ≤ z ≤ 1 . 5 } . This example serves as a representative case, highlighting the limitations of min-max optimization algorithms. It demonstrates

situations where standard algorithms fail to converge to the desired critical points but instead converge to spurious, non-critical points. Specifically, for problem (8), two spurious limit cycles exist across the entire domain. Worse, the one closer to the optimal solution [ x ⋆ ; y ⋆ ] ≃ [0 . 08; 0 . 4] is unstable, which potentially pushes the trajectories to fall into the recurrent orbit.

'Sixth-order polynomial' example The third one is a sixth-order polynomial scaled by an exponential function. It is studied in [18, 56, 13], i.e.,

where X = Y = { z : -2 ≤ z ≤ 2 } . It is shown in [56] that existing first-order methods will suffer from limit cycles around [ x ∗ ; y ∗ ] = [0; 0] . As far as we know, all existing convergence results rely on second-order information. Therefore, it is intriguing to investigate the potential for ensuring convergence using first-order methods.

'PolarGame' example The last example is constructed in [50, Example 3]:

where u = [ x ; y ] satisfy u ∈ X × Y with X = Y = { z : -1 ≤ z ≤ 1 } and Φ( x, y ) = x ( -1 + x 2 + y 2 )( -9 + 16 x 2 +16 y 2 ) . It has been verified that there exist limit cycles at ∥ u ∥ = 1 2 and ∥ u ∥ = 3 4 , which definitely make the iterates hard to converge to [ x ∗ ; y ∗ ] = [0; 0] . To further demonstrate the effectiveness of our DS-GDA, we intentionally initialize the algorithm on the limit cycle ∥ u ∥ = 1 .

C Notation and Useful Lemmas

We first list some useful notations in Table 1. In the following parts, some technical lemmas are presented. Recall that r 1 > L x and r 2 > L y .

Lemma 1 For any x, x ′ ∈ X , y, y ′ ∈ Y , z ∈ R n and v ∈ R d , it follows that

Proof Since f is L -smooth (from the Assumption 1), we have

On the other hand, we know that

and similarly

Combing (9), (10) and (11), we directly obtain the desired results.

⊓ ⊔

Lemma 2 (Lipschitz type error bound conditions) Suppose that r 2 > ( L y r 1 -L x + 2) L y , then for any x, x ′ ∈ X , y, y ′ ∈ Y , z, z ′ ∈ R n and v, v ′ ∈ R d . Then the following inequalities hold:

Proof (i) From Lemma 1, we know that

Combining above inequalities, one has that

where the second inequality is from Cauchy-Schwarz inequality and L -smooth property. Let ξ := ∥ x ( y ′ , z, v ) -x ( y, z, v ) ∥ / ∥ y -y ′ ∥ . Then it follows that

Consequently, utilizing AM-GM inequality we derive ξ ≤ L 2 y +2 r 1 L y -2 L x L y r 1 -L x ≤ L y + r 1 -L x r 1 -L x := σ 1 . (ii-iii) Again from Lemma 1, we know that

From the definition of F , we know that

Incorporating (12), (13) and using the Cauchy-Schwarz inequality, we have

which completes the proof of (ii). Moreover, since max y ∈Y F ( · , y, · , · ) is ( r 1 -L x ) -strongly convex in x , the similar argument leads to (iii).

(iv) We will now proceed to prove inequality (iv). From Lemma 1, we know that

On the other hand, we have

and

Armed with these inequalities, we conclude that

where the last inequality is from error bounds (i) and (ii). Let ξ := ∥ y ( z,v ) -y ( z ′ ,v ) ∥ ∥ z -z ′ ∥ , we have ξ 2 ≤ r 1 σ 1 r 2 -L y ξ + r 1 σ 2 r 2 -L y . Using AM-GM inequality, we obtain

Next, we consider (v). Still from Lemma 1, we have the following inequalities:

Summing them up, we derive that

Let ξ := ∥ y ( x,z,v ) -y ( x ′ ,z,v ) ∥ ∥ x -x ′ ∥ . Then, we have ξ 2 ≤ L x r 2 -L y + L x r 2 -L y ξ and consequently ξ ≤ L x + r 2 -L y r 2 -L y = σ 4 .

(vi-viii) From the definition of F , we get

Moreover, by the strong concavity of F ( x, · , z, v ) , we have

Armed with these inequalities and Cauchy-Schwarz inequality, we conclude that

which gives the inequality (vi). Since d ( · , z, v ) = min x ∈X F ( x, · , z, v ) is ( r 2 -L y ) -strongly concave, similarly we can derive the Lipschitz property of y ( z, v ) , as shown in (vii). ⊓ ⊔

Lemma 3 ( L -smooth property of dual function) For any fixed z ∈ R n , v ∈ R d , the dual function d ( · , z, v ) is continuously differentiable with the gradient ∇ y d ( y, z, v ) = ∇ y F ( x ( y, z, v ) , y, z, v ) and

Proof Using Danskin's theorem, we know that d ( · , z, v ) is differentiable with ∇ y d ( y, z, v ) = ∇ y F ( x ( y, z, v ) , y, z, v ) . Also, we know from the L -smooth property of f that

where the last inequality is due to the error bound in Lemma 2 (i).

⊓ ⊔

Recall that y + ( z, v ) = proj Y ( y + α ∇ y F ( x ( y, z, v ) , y, z, v )) . Incorporating the iterates of DS-GDA, we have the following error bounds:

Lemma 4 For any t ≥ 0 , the following inequalities hold:

Proof (i-iii) First, we consider (i), which is also called 'primal error bound'. From Lemma 1, we know that the mapping ∇ x F ( · , y, z, v ) is ( r 1 -L x ) -strongly monotone and Lipschitz continuous with constant ( r 1 + L x ) on set X for all y ∈ Y , z ∈ R n , v ∈ R d . Adopting the proof in [49, Theorem 3.1], we can easily derive that

which implies that

Similarly, since -∇ y F ( x, · , z, v ) is ( r 2 -L y ) -strongly monotone and Lipschitz continuous with constant ( r 2 + L y ) on Y for all x ∈ X , z ∈ R n , v ∈ R d , we can derive the 'primal error bound' for y t in the inequality (ii). As for (iii), notice that y ( z, v ) = argmax y ∈Y d ( y, z, v ) , the 'primal error bound' is defined with operator ∇ y d ( · , z, v ) . Since d ( · , z, v ) = min x ∈X F ( x, · , z, v ) is ( r 2 -L y ) -strongly concave, the operator -∇ y d ( · , z, v ) is ( r 2 -L y ) -strongly monotone. Moreover, from the Lemma 3, we find ∇ y d ( · , z, v ) is L d -Lipschitz continuous on Y for all z ∈ R n , v ∈ R d . Thus, (iii) can be derived correspondingly.

(iv) Utilizing the inequality (i), we can further bound the desired term

where the first inequality follows from the non-expansivity of the projection operator. ⊓ ⊔

D Basic Descent Lemmas

Lemma 5 (Primal descent) For any t ≥ 0 , the following inequality holds:

Proof We firstly split the target into four parts as follows:

As for 1 ❧ , we have where the first inequality is from Lemma 1 and the second is due to the projection update of x t +1 , i.e., ⟨ x t -c ∇ x F ( x t , y t , z t , v t ) -x t +1 , x t -x t +1 ⟩ ≤ 0 . Next, from Lemma 1, one has for the inequality 2 ❧ that

For 3 ❧ , on top of the update of z t +1 , i.e, z t +1 = z t + β ( x t +1 -z t ) , we obtain

Similarly, as for 4 ❧ , following the update of v t +1 , i.e, v t +1 = v t + µ ( y t +1 -v t ) , we can verify that

Combining all the above bounds leads to the conclusion.

Lemma 6 (Dual ascent) For any t ≥ 0 , the following inequality holds:

Proof The difference of the update for the dual function is controlled by the following three parts:

For the first part, following from the update of v t +1 , we have

For the second part,

Finally, consider the third part, from the Lemma 3, we get

Combining the above inequalities finishes the proof.

Lemma 7 (Proximal descent) For all t ≥ 0 , the following inequality holds:

Proof From Sion's minimax theorem [54], we have

Thus, it follows that

The proof is complete.

⊓ ⊔

⊓ ⊔

⊓ ⊔

E Proof of Proposition 1

From the results in Lemmas 5, 6, and 7, we know that

For the part 1 ❧ , using the projection update of y t +1 , we have

The part 2 ❧ is due to the Lipschitz gradient property (see Assumption 1) and error bound (i) in Lemma 4:

As for the part 3 ❧ , for any κ > 0 it follows that

where the inequality is from the Cauchy-Schwarz inequality, AM-GM inequality, and error bound (iii) in Lemma 2. Hence,

Next, we focus on the negative term. From the fact x ( z, v ) = x ( y ( z, v ) , z, v ) and x ( y, z, v ) = x ( y, z, v ′ ) , the inequality 4 ❧ is bounded as follows:

Here, the second inequality is due to the error bound (i) in Lemma 2. We can further simplify the second error term by (iii)-(iv) in Lemma 4 and (iv), (vii) in Lemma 2:

Plugging (15) into (14), we have

Summing the above inequalities and letting s 1 := 1 c -L x + r 1 2 -L y , s 2 := 1 α + r 2 -L y 2 -L := d -L y σ 2 6 , and s 3 := r 1 ( 2 -β 2 β -2 σ 2 -1 κ ) , we have

Moreover, to make the terms in the upper bound consistent, by error bound (iv) in Lemma 4, we have

Similarly, we provide a lower bound for ∥ v t +1 -v t ∥ 2 as follows:

Recalling that v t + ( z t +1 ) = v t + µ ( y ( z t +1 , v t ) -v t ) , the last inequality can be obtained by using error bounds (iv) in Lemma 2 and (iii), (iv) in Lemma 4, i.e.,

Substituting (17) and (18) to (16) yields

Recalling that L y = λL x , we have the following results:

As r 1 ≥ 2 L x , we have σ 1 = L y r 1 -L x +1 ≤ L y L x +1 = λ +1 and σ 2 = r 1 r 1 -L x ≤ 2 . As r 2 ≥ 2 L y , similarly, we find σ 3 = r 1 σ 1 r 2 -L y + σ 2 σ 1 ≤ r 1 ( λ +1) L y +2 and σ 5 = r 2 r 2 -L y ≤ 2 . With these

bounds and set κ := 2 β with 0 < β ≤ min { 24 r 1 360 r 1 +5 r 2 1 λ +(2 L y +5 r 1 ) 2 , αL 2 y 384 r 1 ( λ +5)( λ +1) 2 } and 0 < µ ≤ min { 2( λ +5) 2( λ +5)+ L 2 y , αL 2 y 64 r 2 ( λ +5) } we derive that

and

With the upper bound (19), we can further bound the coefficient of ∥ z t +1 -z t ∥ 2 by

Let 1 c -L x + r 1 2 ≥ 1 3 c and 1 3 c -L y ≥ 1 6 c . Then, we have 1 c ≥ max { 3 4 ( L x + r 1 ) , 6 L y } , which implies that s 1 ≥ 1 6 c and σ 6 = 2 r 1 + 1 c r 1 -L x ≥ 2 + 1 cr 1 ≥ 11 4 .

Let 1 α -L y σ 2 6 ≥ 1 3 α and 1 3 α -L d ≥ 1 6 α . Then, we have 1 α ≥ max { 3 2 L y σ 2 6 , 6 L d } and s 2 ≥ 1 6 α + r 2 -L y 2 . σ 8 = 1 α + L d r 2 -L y ≤ 2 αr 2 + λ +4 . With these bounds, if we further assume that 1 α ≥ 5 √ λ +5 L y , we obtain

and

where the first inequality is due to 2 3 α ≥ L y σ 2 6 and

Putting together all the pieces, we get

The proof is complete.

F Proof of Proximal Error Bounds and Dual Error Bounds

F.1 Proof of Proposition 2 under Assumption 2

On the other side, by leveraging the KŁ Assumption 2, h ( x, z, · ) satisfies the KŁ property with same exponent [61, Theorem 5.2]. Thus, we get

where the first inequality is because

Next, we further bound the right-hand part. Recalling the definition of v t + ( z t +1 ) and with the bound (vii) in Lemma 2, we obtain

Combining (20) (21), and (22), we have

The proof is complete.

F.2 Proof of Proposition 2 under Assumption 3

By the strong convexity of h ( · , z, v ) = max y ∈Y F ( · , y, z, v ) , we have

On the other side,

Here, the first inequality is because h ( x, z, · ) = max y ∈Y f ( x, y ) -r 2 2 ∥ y - ·∥ 2 is concave [52, Theorem 2.26]. The last inequality is from (22) and the fact that ∥ v t ∥ ≤ diam( Y ) , which could be derived from mathematical induction. Moreover, v ( z ) = argmax v { max y ∈Y f ( x ( y, z, v ) , y ) + r 1 2 ∥ x ( y, z, v ) -z ∥ 2 -r 2 2 ∥ y -v ∥ 2 } should satisfy v ( z ) = prox f ( x ( · ,z,v ) , · )+ r 1 2 ∥ x ( · ,z,v ) -z ∥ 2 + 1 Y ( · ) r 2 ( z ( v ))

[52, Theorem 2.26] , which indicates that ∥ v ( z ) ∥ ≤ diam( Y ) . Thus,

F.3 Dual Error Bounds

Before presenting the results, we first introduce some useful notation:

Proposition 3 Suppose that Assumption 2 holds. Then, we have

Proof By the strong convexity of ϕ ( · , z ) = max y ∈Y f ( · , y ) + r 1 ∥ · -z ∥ 2

On the other hand, we have

Recall that x ( z, v ) = argmin x ∈X max y ∈Y F ( x, y, z, v ) . Then, the first inequality is from the fact that

Under Assumption 2, we find that

where the last inequality is due to the first optimality condition of y ( z, v ) = argmax y ∈Y f ( x ( y, z, v ) , y ) -r 2 2 ∥ y -v ∥ 2 . ⊓ ⊔

Proposition 4 Suppose that Assumption 3 holds. Then, we have

Proof We first define ψ ( x, y, z ) = f ( x, y ) + r 1 2 ∥ x -z ∥ 2 and Y ( z ) = argmax y ∈Y min x ∈X ψ ( x, y, z ) = argmax y ∈Y ψ ( x r 1 ( y, z ) , y, z ) . Let y ∗ ( z ) be an arbitrary solution in Y ( z ) . By the strong convexity of ψ ( · , y, z ) = f ( · , y ) + r 1 2 ∥ · -z ∥ 2 , we have

Here, the first inequality is beacuse ψ ( x ∗ ( z ) , y ∗ ( z ) , z ) = max y ∈Y ψ ( x r 1 ( y, z ) , y, z ) ≥ ψ ( x r 1 ( y ( z, v ) , z ) , y ( z, v ) , z ) = ψ ( x ( z, v ) , y ( z, v ) , z ) . On the other hand, by the concavity of Ψ( x, · , z ) ,

The second inequality is due to ⟨∇ y f ( x ( z, v ) , y ( z, v )) -r 1 ( y ( z, v ) -v ) , y ∗ ( z ) -y ( z, v ) ⟩ ≤ 0 and the last inequality is because both y ∗ ( z ) and y ( z, v ) lie in set Y . ⊓ ⊔

G Proof of Theorem 1

Before presenting the proof of Theorem 1, we first introduce the following lemma.

Lemma 8 Let ϵ ≥ 0 . Suppose that

Then, there exists a ρ > 0 such that ( x t +1 , y t +1 ) is a ρϵ -GS .

Proof In accordance with definition 1, we just need to evaluate the two terms dist( 0 , ∇ x f ( x t +1 , y t +1 ) + ∂ 1 X ( x t +1 )) and dist( 0 , -∇ y f ( x t +1 , y t +1 ) + ∂ 1 Y ( y t +1 )) . We first note that

Its optimality condition yields

which implies that

Then, we are ready to simplify the stationarity measure as follows:

The last inequality is because max { ∥ x t -x t +1 ∥ c , ∥ y t -y t + ( z t ,v t ) ∥ α , ∥ z t -z t +1 ∥ β } ≤ ϵ . Similarly, we have

which can be directly derived from the update of y t +1 . Then, we can simplify the expression

The last inequality is due to max { ∥ x t -x t +1 ∥ c , ∥ y t -y t + ( z t ,v t ) ∥ α , ∥ v t -v t +1 ∥ µ } ≤ ϵ . The proof is complete. ⊓ ⊔

Proof of Theorem 1 Firstly, it is easy to check that Φ( x, y, z, v ) is lower bounded by ¯ F . Let

Then, we consider the following two cases separately:

We first consider the first case. If the KŁ exponent θ ∈ ( 1 2 , 1) , then from Proposition 2,

Then, we have ∥ v t + ( z t +1 ) -v t ∥ ≤ ρ 1 β θ 2 θ -1 , where ρ 1 := ( 32 r 1 µω 0 r 2 ) θ 2 θ -1 . Armed with this, we can bound other terms as follows:

where ρ 2 := 256 ω 0 c 2 ρ 1 θ 1 , ρ 3 := 120 r 1 ω 0 r 2 α 2 ρ 1 θ 1 , ρ 4 := 40 ω 0 ρ 1 θ 1 and ρ 5 := 2 ρ 2 1 µ 2 +8 L 2 y α 2 σ 2 6 c 2 ρ 2 +8( σ 8 + 1) 2 α 2 ρ 3 +4 σ 2 3 ρ 3 . According to Lemma 8, there exists a ρ > 0 such that ( x t +1 , y t +1 ) is a ρϵ -GS, where ϵ = max { √ ρ 2 β θ 2 θ -1 , √ ρ 3 β θ 2 θ -1 , √ ρ 4 β θ 2 θ -1 , √ ρ 5 β 1 4 θ -2 } . For the general concave case, the above analysis holds with θ = 1 and ( x t +1 , y t +1 ) is a ρ max { √ ρ 2 β √ ρ 4 β, √ ρ 5 β, √ ρ 3 β 1 2 } -GS by replacing ω 0 with ω 1 .

One remaining thing is to prove that z t +1 is an O ( β 1 4 θ -2 ) -OS. By the dual error bounds in Proposition 3 and error bounds in Lemmas 2 and 4, we have

Now, we consider the second case. Since

holds for t ∈ { 0 , 1 , . . . , T -1 } and Φ( x, y, z, v ) ≥ ¯ F , we know that

Since Φ( x, y, z, v ) ≥ ¯ F , there exists a t ∈ { 0 , 1 , . . . , T -1 } such that

Note that

where ρ 6 = η ( 2 + 8 L 2 y α 2 σ 2 6 c 2 +8( σ 8 +1) 2 α 2 +4 σ 2 3 ) . Thus, we know there exists a ρ > 0 such that ( x t +1 , y t +1 ) is a ρϵ -GS, where ϵ = √ ρ 6 Tβ . Similar argument as (23) shows that z t +1 is an O ( 1 √ Tβ ) -OS. Moreover, if we choose β ≤ O ( T -2 θ -1 2 θ ) when the KŁ exponent θ ∈ ( 1 2 , 1) and β = O ( T -1 2 ) for general concave case, the two cases coincide. We conclude that under KŁ

assumption, ( x t +1 , y t +1 ) is an O ( T -1 4 θ ) -GS and z t +1 is an O ( T -1 4 θ ) -OS. When the dual function is concave, ( x t +1 , y t +1 ) is an O ( T -1 4 ) -GS and z t +1 is an O ( T -1 4 ) -OS.

Next, suppose that the KŁ exponent θ ∈ (0 , 1 2 ] . From Proposition 2, we obtain

The last inequality is from ∥ v t + ( z t +1 ) -v t ∥ = β ∥ y ( z t +1 , v t ) -v t ∥ ≤ 2 diam( Y ) since ∥ v t ∥ ≤ diam( Y ) . Armed with the homogeneous proximal error bound (24), we have

where the last inequality is because β ≤ r 2 32 r 1 µω 0 (2 diam( Y )) 1 θ -2 . The rest of the proof is similar to that in the second case when θ ∈ ( 1 2 , 1) , and we conclude that ( x t +1 , y t +1 ) is a O ( T -1 2 ) -GS and z t +1 is an O ( T -1 2 ) -OS.

H Convergence Anaylsis for (Non)convex-Nonconcave Problems

Similar convergence results for (non)convex-nonconcave problems can be established with the symmetric property of our Lyapunov function. Specifically, we can derive the following Proposition 5 by doing an easy transformation of ( x, z, r 1 , c, β, L x ) to ( y, v, r 2 , α, µ, L y ) :

Proposition 5 (Basic descent estimate) Suppose that Assumption 1 holds and r 1 ≥ 2 L , r 2 ≥ 2 λL with the parameters

Then, for any t ≥ 0 ,

where σ ′ := 2 αr 2 +1 α ( r 2 -λL ) and L h := ( L r 2 -λL +2 ) L + r 1 . Moreover, we set x + ( z, v ) := proj X ( z + c ∇ x F ( x, y ( x, z, v ) , z, v )) and z + ( v ) := z + µ ( x ( y ( z, v ) , z, v ) -z ) with the following definitions: (i) y ( z, v ) := argmin y ∈Y max x ∈X F ( x, y, z, v ) , (ii) z ( v ) := argmin z ∈ R n P ( z, v ) .

To handle the negative term in the basic decrease estimate (Proposition 5), we would impose the following assumptions on primal function:

Assumption 4 (KŁ property for primal function) For any fixed point y ∈ Y , z ∈ R n , and v ∈ R d , the problem min x ∈X F ( x, y, z, v ) has a nonempty solution set and a finite optimal value. Moreover, there exist τ 1 > 0 , θ 1 ∈ (0 , 1) such that

holds for any y ∈ Y and x ∈ X .

Assumption 5 (Convexity of the primal function) For any fixed point y ∈ Y , f ( · , y ) is convex.

Proposition 6 (Proximal error bound) Under the setting of Proposition 5 with Assumption 4 or 5, for any z ∈ R n , v ∈ R d , we have (i) KŁ exponent θ 1 ∈ (0 , 1) :

(ii) Convex:

where ω 2 := 2 ( r 2 -L y ) τ 1 ( r 1 (1 -β ) β + r 2 1 r 1 -L x ) 1 θ 1 and ω 3 := 4 r 1 diam( X ) r 2 -L y ( 1 -β β + σ 2 ) . Here, diam( X ) denotes the diameter of the set X .

Equipped with these results, we could also derive similar complexity results for (non)convexnonconcave minimax problems as Theorem 1.

I Proof of Theorem 2

In general, analyzing the local convergence rate of sequences could be challenging. Fortunately, when θ ∈ (0 , 1 2 ] , the homogeneous sufficient descent property holds as follows:

Consequently, we can use the unified analysis framework in [3]. Next, we prove the 'Safeguard' property as required there.

Proposition 7 ('Safeguard' property) There exists a series of parameters ν 1 , ν 2 , ν 3 , ν 4 > 0 such that for all t ∈ { 0 , 1 , . . . , T -1 } , the following holds:

Proof We observe from the update of x t +1 = argmin x ∈X 1 2 ∥ x -( x t -c ∇ x F ( x t , y t , z t , v t )) ∥ 2 and y t +1 = argmin y ∈Y 1 2 ∥ y -( y t + α ∇ y F ( x t +1 , y t , z t , v t )) ∥ 2 that

Moreover, we have

This, together with (25), gives

Under Assumption 1 and utilizing the error bounds in Lemmas 2 and 4, we can simplify (26). When z t reaches the local region that satisfies β ∥ x ( z t , v t +1 ) -z t ∥ ≤ 1 , we can show that the right-hand side is actually upper bounded by the linear combination of the desired four terms:

Here, ν 1 = 1 c + L x + r 1 + 2 L x σ 6 + 2 r 1 σ 1 L y ασ 6 + ( L x + 1 α + L y + 2 r 2 + 2 L x σ 1 ) L y ασ 6 , ν 2 = L x + 1 α + L y + r 2 +2 L x σ 1 + r 1 (2 σ 1 +1)( σ 8 +1) , ν 3 = r 1 β +2 L x σ 2 +2 r 1 σ 1 σ 3 + r 2 σ 3 , and ν 4 = r 2 µ +2 r 1 σ 1 σ 5 +2 r 1 √ ω 1 . Moreover, since f ( · , y ) and f ( x, · ) is semi-algebraic [10, Theorem 3], using results in [10, Example 2], we find that Φ( x, y, z, v ) = F ( x, y, z, v ) -2 d ( y, z, v ) + 2 q ( z ) is also semi-algebraic in x, y, z, v . Based on Assumption 1, Φ( x, y, z, v ) is continuous in x, y, z, v . Building on the unified convergence analysis framework in [4], we can conclude that { ( x t , y t , z t , v t ) } converges to one stationary point ( x ∗ , y ∗ , z ∗ , v ∗ ) . On top of Lemma 8 and lim t →∞ ∥ x t -x t +1 ∥ =

Since lim t →∞ ∥ z t -z t +1 ∥ = 0 , lim t →∞ ∥ x t -x t +1 ∥ = 0 , lim t →∞ ∥ y t -y t + ( z t , v t ) ∥ = 0 , lim t →∞ ∥ v t -v t + ( z t +1 ) ∥ = 0 , we conclude { z t } convergs to an OS. ⊓ ⊔

J Proof of Theorem 3

Here, without the generality, we assume L x = L y = L and r 1 = r 2 = r . Then from the Proposition 1 and Proposition 5, we can easily derive the bounds for all the parameters as follows:

There are numerous feasible choices for β and µ that satisfy equation (27b). Next, we will demonstrate the existence of a viable solution for c and α by establishing the intersection of the last two constraints in equation (27a). Solving these constraints under the condition r ≥ 2 L yields

where r should further satisfy r ≥ 14 L . Combing these with the penultimate constraint in equation (27a), we can provide a set of feasible parameters as follows:

Armed with equation (27b) and equation (28), we finish the proof of Theorem 3.

K Relationship Between Different Notions of Stationary Points

In this section, we illustrate the quantitative relationship among several notions of stationary measure.

Definition 2 The point ( x, y ) ∈ X × Y is said to be an

ϵ -optimization game stationary point (OS) if

ϵ -game stationary point (GS) if

Recalling that x ∗ ( z ) = argmin x ∈X max y ∈Y f ( x, y ) + r 1 2 ∥ x -z ∥ 2 , we have ∥ prox max y ∈Y f ( · ,y ) r 1 + 1 X ( x ) -x ∥ = ∥ x ∗ ( x ) -x ∥ . With Proposition 3, we are ready to prove the relationship between OS and GS.

Proposition 8 Suppose that Assumption 2 holds. If ( x, y ) ∈ X × Y is a ϵ -GS, then it is a O ( ϵ min { 1 , 1 2 θ } ) -OS.

Building on the proposition 3, we have

By the nonexpansiveness of projection operator and error bounds in Lemmas 3 and 4, we can further bound ∥ y -y ( z, v ) ∥ as follows:

Plugging this bound into (29), we get

Next, we will explore the relationship between ∥ x -x ( y, x, v ) ∥ and dist( 0 , ∇ x f ( x, y ) + ∂ 1 X ( x )) . Let x + ( y, x, v ) := proj X ( x -c ∇ x F ( x, y, x, v )) . Then, from the primal error bound (see [49]), we know that

Moreover, since ∇ x F ( x, y, x, v ) = ∇ x f ( x, y ) , it follows from [39, Lemma 4.1] that

A similar analysis can be applied to derive the bounds for ∥ y -y ( x, z, y ) ∥ . Thus, if ( x, y ) is a ϵ -GS, then it is an O ( ϵ min { 1 , 1 2 θ } ) -OS.

L Details about Examples in Section 4.3 and Behaviors of Wrong Selection of Smoothing Sides in S-GDA

This section will check all the regularity conditions for the examples mentioned in Section 4.3. The violation of the global KŁ condition can be easily vertified by plotting the figures for these examples. Therefore, we mainly discuss whether they satisfy 'weak MVI' and ' α -interaction dominant' conditions, which are two representative classes of conditions in the nonconvex-nonconcave setting. Moreover, we will also give an example showing the slow convergence caused by the wrong selection of S-GDA.

L.1 Proof of Proposition for 'Forsaken' Example

This subsection considers the 'Forsaken' example in [32, Example 5.2] on the constraint sets X = Y = { z : -1 . 5 ≤ z ≤ 1 . 5 } . In [50], it is vertified that the 'Forsaken' example violates 'weak MVI' condition with ρ < -1 2 L . Therefore, we only consider the α -interaction dominant condition here. By a simple calculation, we get ∇ 2 xx f ( x, y ) = 1 2 -6 x 2 +5 x 4 , ∇ 2 xy f ( x, y ) = ∇ 2 yx f ( x, y ) = 1 , and ∇ 2 yy f ( x, y ) = -1 2 +6 y 2 -5 y 4 . Armed with these, α can be found globally by minimizing the following:

It is less than zero when [ x ; y ] = [1; 0] . That is, α < 0 in the constraint set X , which means the α -interaction dominant condition is violated for the primal variable x . Similar proof could be adapted for the dual variable. This rules out the convergence guarantees of damped PPM, which is validated in Figure 4.

L.2 Proof of Proposition for 'Bilinearly-coupled Minimax' Example

This 'Bilinearly-coupled Minimax' example is mentioned as a representative example where α is in the interaction moderate regime [29]. Experiments also validate that the solution path will be globally trapped into a limit cycle (see Figure 4). For this reason, we only check the 'weak MVI' condition. In this example, X = Y = { z : -4 ≤ z ≤ 4 } , G ( u ) = [ ∇ x f ( x, y ); -∇ y f ( x, y )] = [4 x 3 -20 x +10 y ; 4 y 3 -20 y -10 x ] , and u ⋆ = [0; 0] . Then, ρ can be found by globally minimizing ρ ( u ) := ⟨ G ( u ) ,u -u ⋆ ⟩ 2 for all u ∈ X × Y . Notice that

We have ρ ( u ) = ⟨ G ( u ) ,u -u ⋆ ⟩ ∥ G ( u ) ∥ 2 = -4 89 when u = [ x ; y ] = [0; 1] , which implies that ρ < -4 89 . Moreover, we find L = 172 , so ρ < -4 89 < -1 344 = -1 2 L . We conclude that this example does not satisfy the 'weak MVI' condition and the limit cycle phenomenon is actually observed in Figure 4.

1

DSGDA

0

SGDA with dual smoothing

SGDAwithprimal smoothing

-1

-2

E-

#

-4

-5

-6

-7

8-

0

5

10

15

20

25

30

35

40

45

Iterations

L.3 Proof of Proposition for 'Sixth-order polynomial' Example

We demonstrate that this example violates both the 'weak MVI' and ' α -interaction' conditions. To provide evidence of this violation, we follow the same approach used to prove violations in Sections L.1 and L.2 and present counter-examples. On the one hand, we found that ρ ≤ ρ (ˆ u ) = -0 . 0795 < -0 . 0368 = -1 2 L with ˆ u = [ x ; y ] = [ -1; 0 . 5] , which implies the violation of 'weak MVI' condition. On the other hand, we have

It is less than zero when η ≥ L , which indicates the violation of the α -interaction condition.

L.4 Proof of Proposition for 'PolarGame' Example

In this subsection, we check the two conditions for the 'PolarGame' example. Firstly, we have ρ ≤ ρ (ˆ u ) = -0 . 3722 < -0 . 0039 = -1 2 L with ˆ u = [0 . 8; 0] . Thus, it does not satisfy the 'weak MVI' condition. Next, consider the following at [ x ; y ] = [0 . 8; 0] :

It is less than zero when η ≥ L . Thus, the 'PolarGame' violates the α -interaction condition.

L.5 Wrong Smoothing Side of S-GDA

For S-GDA, we show that if we choose the wrong side, it will result in a slow convergence. To validate this, we conduct a new experiment for the KL-NC problem

where X = Y = { z : -1 ≤ z ≤ 1 } . With a wrong smoothing side, S-GDA (with primal smoothing) leads to a slower convergence compared with dual smoothing (see Figure 5).

Peer reviews:
comments: SUMMARY:
The paper proposes the doubly smoothed gradient descent ascent method and studies its convergence under nonconvex-KL and nonconvex-concave minimax problems.

STRENGTHS:
This paper proposes a novel model to address the limiting cycle phenomena in nonconvex-nonconcave minimax optimization. The theoretal analysis match the best-known results for other methods.

WEAKNESSES:
The algorithm proposed in this paper comes naturally. From my experience, I can expect that DSGDA performs well in practice. My main concern is the theoretical guarantee of DSGDA. However, it seems that the results given by this paper do not show any superiority of DSGDA compared with other methods.

QUESTIONS:
Are there any differences between the convergence of Doubly Smoothed GDA and Smoothed GDA (Zhang et.al 2020) / Smoothed PLDA (Li et. al 2022) in theory? From the theoretical viewpoint, why is  Doubly Smoothed GDA better than Smoothed GDA? Can we show that the Doubly Smoothed GDA provably gets rid of limiting cycles?

The authors claimed that "our work demonstrates, for the first time, the possibility of having a
simple and unified single-loop algorithm for solving nonconvex-nonconcave, nonconvex-concave,
and convex-nonconcave minimax problems."  Can the authors explain that why the Smoothed GDA cannot have such a "unified analysis"?
By the way, I think it is better to use "nonconvex-KL" instead of "nonconvex-nonconcave" since the difficulties of these two problems seem to be very different from my point of view.

LIMITATIONS:
n/a
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-06-25

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the nontrivial general minimax problem by proposing a novel algorithm called Doubly Smoothed Gradient Descent Ascent (DSGDA). The authors provide both experimental and theoretical evidence to support their claims.\n\nSTRENGTHS:\n1. Overview I enjoyed reading this work. It's well written and organized, although some refernece and comparison is missing (see below). \n\n2. The proposed algorithm is not complicated and the intuition is sound. \n\n3.  Convergence results are sound.\n\nWEAKNESSES:\nWhile I acknowledge the soundness of the algorithm and the obtained results, I do have several concerns about this paper.\n\n### W1: \nThe theoretical convergence results demonstrating that the proposed method converges to a stationary point do not necessarily showcase its effectiveness in addressing the issue of **general** minimax problems. Although it is generally difficult for first-order methods to derive other convergence properties, such as local minimax points, for nonconvex-nonconcave problems, could the authors show the superiority of the algorithm under other settings?\n\n### W2: \nThe authors claim that their work demonstrates, for the first time, the possibility of having a simple and unified single-loop algorithm for solving nonconvex-nonconcave, nonconvex-concave, and convex-nonconcave minimax problems. However, to my knowledge, GDA-AM (ICLR 2022) also demonstrated such theoretical results (Thm C.4 and C.9) and showed convergent experiments. Additionally, the fast extragradient (NeurIPS 2021) made similar claims, as mentioned in Remark 1. Could the authors discuss the differences in contributions between their work and these existing papers?\n\n### W3: \nExperiments\nThe authors only compared their algorithm with SGDA. However, what about other methods? For example, how does it compare to fast extragradient or GDA-AM on these problems?\n\n1. He et.al. GDA-AM: On the effectiveness of solving minimax optimization via Anderson Acceleration, ICLR 2022\n2. Lee et.al. Fast Extra Gradient Methods for Smooth Structured Nonconvex-Nonconcave Minimax Problems, Neurips 2021\n\n\nExcept several concerns unaddressed, I tend to accept. I look forward to the authors' response.\n\nQUESTIONS:\n### Explainantion on Theorem 1\nCould authors make additional comments are conditions of theorem 1.\n\nLIMITATIONS:\nAuthors discussed limitations", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel single-loop algorithm called the doubly smoothed gradient descent ascent method (DSGDA). DSGDA effectively balances the primal and dual updates, eliminating limit cycles in various challenging nonconvex-nonconcave scenarios in the literature. The paper establishes that under a one-sided Kurdyka-Łojasiewicz condition, with an exponent $\\theta\\in (0,1)$ (or for convex primal/concave dual functions), DSGDA can discover a game-stationary point with an iteration complexity of $O(\\epsilon^{-2\\max\\{(2\\theta, 1)\\}})$ (or $O(\\epsilon^{-4})$), respectively. These complexity results match the best outcomes achieved by single-loop algorithms for solving nonconvex-concave or convex-nonconcave minimax problems, as well as problems satisfying the restrictive one-sided Polyak-Łojasiewicz condition.\n\nSTRENGTHS:\n**Originality**: The paper provides a comprehensive discussion of existing algorithms for mini-max optimization problems and introduces a novel algorithm aimed at avoiding limit cycles. The proposed algorithm is supported by both theoretical results and empirical validation, strengthening its efficacy.\n\n**Quality and Clarity**: The paper exhibits a high level of writing quality, featuring a well-structured presentation that is easy to follow. It effectively employs examples and explanations to enhance comprehension of both the algorithm and the underlying theory.\n\nWEAKNESSES:\n**Significance**: While I acknowledge the valuable contributions made by this paper in terms of algorithmic and theoretical aspects, it is important to note that the current theory relies on one-sided KL or one-sided convex/concave conditions. Considering the paper's objective of addressing limit cycles, it would be beneficial to establish a clearer connection between the existing theory and the role of DSGDA in avoiding such cycles. Elaborating on this relationship would significantly enhance the paper's overall strength and clarity.\n\nQUESTIONS:\n**Main Questions**\n\n1. Relation between OS and GS: I would suggest including further discussions in the paper regarding the relationship between OS and GS notations. It appears that there exists a translation method between these two notations, as mentioned in [1]. Elaborating on this relationship can greatly enhance the paper's quality and comprehensibility.\n\n2. Odd constants in Theorem 1:The extrapolation step in the algorithm introduces exceedingly large constants, which might seem counter-intuitive. Although the calculations on Page 23 provide some insight into their derivation, further explanation is required to clarify the reasoning behind these values and make them more understandable to readers.\n\n3. Connection between theory and practice: Considering the paper's objective of addressing limit cycles, it would be beneficial to establish a clearer connection between the existing theory and the role of DSGDA in avoiding such cycles. Elaborating on this relationship would significantly enhance the paper's overall strength and clarity.\n\n4. Unconstrained setting: While the paper focuses on the two-sided constrained setting, it is worth noting that some existing papers tackle the scenario where only the domain of the max variable is bounded, as demonstrated in [1]. To enhance the paper, it could be valuable to discuss the disparities and challenges encountered in the unconstrained setting, providing a comparative analysis between the two scenarios.\n\n5. Stochastic setting: It would be interesting to see if similar theoretical results can be obtained in the stochastic setting as provided in [2]\n\n**Minor issues**\n\n6. I would recommend not directly using “nonconvex-nonconcave” in the title as the paper does not consider the general nonconvex-nonconcave setting.\n\n7. I recommend not using “SGDA” to represent the smoothed GDA as SGDA is well-known for stochastic gradient descent ascent algorithms. Using “S-GDA” and “DS-GDA” instead would avoid confusion.\n\n8. Line 77: The size of the right parenthesis\n\n**Reference**\n\n[1]  Lin, Tianyi, Chi Jin, and Michael Jordan. \"On gradient descent ascent for nonconvex-concave minimax problems.\" International Conference on Machine Learning. PMLR, 2020.\n\n[2] Yang, Junchi, et al. \"Faster single-loop algorithms for minimax optimization without strong concavity.\" International Conference on Artificial Intelligence and Statistics. PMLR, 2022.\n\nLIMITATIONS:\nThe paper is theoretical and does not have any potential negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nConstructing a method that finds an optimal point of nonconvex-nonconcave problems is of interest. Built upon the \"one-sided\" smoothed GDA (SGDA) that exploits the Moreau-Yosida smoothing technique, this paper studies the \"doubly\" smoothed GDA (DSGDA), applying the smoothing to both the primal and dual variables. As an illustration, this paper considers four challenging nonconvex-nonconcave minimax problems that do not satisfy any of the regularity conditions considered in the literatures and that none of existing methods work. It turns out that the DSGDA is the first known method that works for all four difficult problems. To complement this empirical success, the authors theoretically show that the DSGDA converges to a stationary point under an one-sided KL condition. (The aforementioned four problems do not satisfy the one-sided KL condition.) The authors also present that the DSGDA works for a wider range of hyperparameters than the SGDA for nonconvex-nonconcave problems.\n\nSTRENGTHS:\n- The proposed DSGDA methods empirically converges to stationary points of four notoriously difficult minimax problems, while other existing methods do not, which is impressive.\n- The authors shows that the DSGDA converges to a stationary point of nonconvex-nonconcave problems with one-sided KL property. The method achieves the best known rate for a more restrictive setting that the dual function satisfies either the concavity or the PL condition.\n\nWEAKNESSES:\n* The DSGDA does not outperform SGDA under the one-sided KL setting, where the proposed DSGDA is theoretically shown to work well. Would it be possible to show that the SGDA also converges under the one-sided KL setting? Or instead, would it be possible to show that it does not converge? Are there other methods that work under the one-sided KL setting?\n* As mentioned by the authors, there remains a gap between the theoretical claim and practical claim.\n* The robustness claim is only based on one nonconvex-nonconcave experiment. In addition, the region of convergence of DSGDA for the nonconvex-nonconcave case is relatively small, so it does not seem to be sufficient to claim that DSGDA is robust (even though it is relatively robust than the SGDA).\n* The choice of hyperparameters that guarantees convergence in Theorem 1 is complicated, so it is not easy to find an optimal choice in terms of the rate, making it difficult for the users to choose.\n\nQUESTIONS:\n* Line 33: Why does no player inherently dominate the other in nonconvex-nonconcave problems? Later in line 50, the authors claim that the primal player becomes dominant for the nonconvex-concave problem. Then, consider a nonconvex-nonconcave problem that is locally nonconvex-concave at some stationary points. Then will the primal player suddenly become dominant locally?\n\n* Line 34: How do you achieve a good balance between primal and dual updates in this paper? Of course, applying smoothing to both variables improve the balance, as claimed by the authors. However, the choice of regularization parameters $r_1$ and $r_2$ and step sizes $\\alpha$, $c$, $\\beta$, $\\mu$ are not the same for both variables. At this point, I am not following what it really means by the good balance.\n\n* line 83: In what sense, is this work the best known convergence analysis?\n\n* Are there practical examples that satisfy the one-sided KL property but not the one-sided PL property?\n\nLIMITATIONS:\nThis work has potential but the gap between the practical claim and the theoretical claim seems not negligible, and I think that the paper could have been better if such gap was further reduced. For example, an experiment with one-sided KL where DSGDA outperforms SGDA could have better supported the paper's claim.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-07"}