Title: A Single-Loop Accelerated Extra-Gradient Difference Algorithm with Improved Complexity Bounds for Constrained Minimax Optimization
Venue: NeurIPS 2023
Accepted: yes
Keywords: Constrained Minimax Optimization; nonconvex- nonconcave
Abstract: In this paper, we propose a novel extra-gradient difference acceleration algorithm for solving constrained nonconvex-nonconcave (NC-NC) minimax problems. In particular, we design a new extra-gradient difference step to obtain an important quasi-cocoercivity property, which plays a key role to significantly improve the convergence rate in the constrained NC-NC setting without additional structural assumption. Then momentum acceleration is also introduced into our dual accelerating update step. Moreover, we prove that, to find an $\epsilon$-stationary point of the function $f$, our algorithm attains the complexity $\mathcal{O}(\epsilon^{-2})$ in the constrained NC-NC setting, while the best-known complexity bound is $\widetilde{\mathcal{O}}(\epsilon^{-4})$, where $\widetilde{\mathcal{O}}(\cdot)$ hides logarithmic factors compared to $\mathcal{O}(\cdot)$. As the special cases of the constrained NC-NC setting, our algorithm can also obtain the same complexity $\mathcal{O}(\epsilon^{-2})$ for both the nonconvex-concave (NC-C) and convex-nonconcave (C-NC) cases, while the best-known complexity bounds are $\widetilde{\mathcal{O}}(\epsilon^{-2.5})$ for the NC-C case and $\widetilde{\mathcal{O}}(\epsilon^{-4})$ for the C-NC case. For fair comparison with existing algorithms, we also analyze the complexity bound to find $\epsilon$-stationary point of the primal function $\phi$ for the constrained NC-C problem, which shows that our algorithm can improve the complexity bound from $\widetilde{\mathcal{O}}(\epsilon^{-3})$ to $\mathcal{O}(\epsilon^{-2})$. To the best of our knowledge, this is the first time that the proposed algorithm improves the best-known complexity bounds from $\mathcal{O}(\epsilon^{-4})$ and $\widetilde{\mathcal{O}}(\epsilon^{-3})$ to $\mathcal{O}(\epsilon^{-2})$ in both the NC-NC and NC-C settings.

A Single-Loop Accelerated Extra-Gradient Difference Algorithm with Improved Complexity Bounds for Constrained Minimax Optimization

Abstract

In this paper, we propose a novel extra-gradient difference acceleration algorithm for solving constrained nonconvex-nonconcave (NC-NC) minimax problems. In particular, we design a new extra-gradient difference step to obtain an important quasi-cocoercivity property, which plays a key role to significantly improve the convergence rate in the constrained NC-NC setting without additional structural assumption. Then momentum acceleration is also introduced into our dual accelerating update step. Moreover, we prove that, to find an glyph[epsilon1] -stationary point of the function f , the proposed algorithm attains the complexity O ( glyph[epsilon1] -2 ) in the constrained NC-NC setting, while the best-known complexity bound is ˜ O ( glyph[epsilon1] -4 ) , where ˜ O ( · ) hides logarithmic factors compared to O ( · ) . As the special cases of the constrained NC-NC setting, the proposed algorithm can also obtain the same complexity O ( glyph[epsilon1] -2 ) for both the nonconvex-concave (NC-C) and convex-nonconcave (C-NC) cases, while the best-known complexity bounds are ˜ O ( glyph[epsilon1] -2 . 5 ) for the NC-C case and ˜ O ( glyph[epsilon1] -4 ) for the C-NC case. For fair comparison with existing algorithms, we also analyze the complexity bound to find glyph[epsilon1] -stationary point of the primal function φ for the constrained NC-C problem, which shows that our algorithm can improve the complexity bound from ˜ O ( glyph[epsilon1] -3 ) to O ( glyph[epsilon1] -2 ) . To the best of our knowledge, this is the first time that the proposed algorithm improves the best-known complexity bounds from ˜ O ( glyph[epsilon1] -4 ) and ˜ O ( glyph[epsilon1] -3 ) to O ( glyph[epsilon1] -2 ) in both the NC-NC and NC-C settings.

1 Introduction

This paper considers the following smooth minimax optimization problem, min x ∈X max y ∈Y f ( x, y ) ,

where X ⊆ R m and Y ⊆ R n are nonempty closed and convex feasible sets, and f : R m × R n → R is a smooth function. In recent years, this problem has drawn considerable interest from machine learning and other engineering communities such as generative adversarial networks [14], adversarial machine learning [15, 28], game theory [3], reinforcement learning [12, 41], empirical risk minimization [54, 39], and robust optimization [36, 13, 5]. While there is an extensive body of literature on minimax optimization, most prior works such as [54, 39, 42, 33, 50] focus on the convex-concave setting, where f ( x, y ) is convex in x and concave in y . However, in many nonconvex minimax machine learning problems as in [35, 6], f ( x, y ) is nonconvex in x and (strongly) concave in y , or f ( x, y ) is (strongly) convex in x and nonconcave in y . For nonconvex-strongly-concave (NC-SC) minimax problems, a number of efficient algorithms such as [34, 25, 27] were proposed, and their complexity can be ˜ O ( κ 2 y glyph[epsilon1] -2 ) for achieving an glyph[epsilon1] -stationary point ˆ x (e.g., ‖∇ φ (ˆ x ) ‖ ≤ glyph[epsilon1] ) of the primal function φ ( · ) := max y ∈Y f ( · , y ) , where κ y is the condition number for f ( x, · ) , and ˜ O hides logarithmic factors. More recently, [24] proposed an accelerated algorithm, which improves the gradient complexity to ˜ O ( √ κ y glyph[epsilon1] -2 ) , which exactly matches the lower complexity bound in [52, 23]. Therefore, we mainly consider the problem (1) in nonconvex-nonconcave (NC-NC), nonconvexconcave (NC-C) and convex-nonconcave (C-NC) settings.

1.1 Algorithms in NC-C and C-NC Settings

For nonconvex-concave (NC-C, but not strongly concave) and convex-nonconcave (C-NC) problems, there are two types of algorithms (i.e., multi-loop (including double-loop and triple-loop) and single-loop algorithms), and most of them are multi-loop algorithms such as [18, 31, 40, 24]. [31] proposed a multi-step framework that finds an glyph[epsilon1] -first order Nash equilibrium of f ( x, y ) with the complexity ˜ O ( glyph[epsilon1] -3 . 5 ) . [40] designed a proximal dual implicit accelerated algorithm and proved that their algorithm finds an glyph[epsilon1] -stationary point of φ with the complexity ˜ O ( glyph[epsilon1] -3 ) . More recently, [24] proposed an accelerated algorithm, which achieves the complexity ˜ O ( glyph[epsilon1] -2 . 5 ) to find an glyph[epsilon1] -stationary point of f . These multi-loop algorithms require at least O ( glyph[epsilon1] -2 ) outer iterations, and thus their complexities are more than O ( glyph[epsilon1] -2 ) . Even though single-loop methods are more popular in practice due to their simplicity, few single-loop algorithms have been proposed for NC-C setting. The most natural approach is the gradient descent-ascent (GDA) method, which performs a gradient descent step on x and a gradient ascent step on y at each iteration. However, GDA fails to converge even for simple bilinear zero-sum games [22]. Subsequently, several improved GDA algorithms such as [8, 25, 27, 44] were proposed. For instance, [25] proved that the complexity of their two-time-scale GDA to find an glyph[epsilon1] -stationary point of φ is O ( glyph[epsilon1] -6 ) for NC-C problems. Moreover, [27] presented an efficient algorithm, which obtains an glyph[epsilon1] -stationary point of f with the complexity O ( glyph[epsilon1] -4 ) . In fact, the complexity only counts the number of times the maximization subproblem is solved, and does not consider the complexity of solving this subproblem. [44] proposed a unified algorithm, and proved that the complexity of their algorithm to find an glyph[epsilon1] -stationary point of f is O ( glyph[epsilon1] -4 ) for both NC-C and C-NC problems. More recently, [51] presented a smoothed-GDA algorithm and proved that the complexity can be improved to O ( glyph[epsilon1] -2 ) for optimizing a special case of NC-C problems (i.e., minimizing the pointwise maximum of a finite collection of nonconvex functions). However, its complexity is still O ( glyph[epsilon1] -4 ) for both NC-C and C-NC problems. One natural question is: can we design a single-loop accelerated algorithm with the optimal complexity bound O ( glyph[epsilon1] -2 ) for both NC-C and C-NC problems ?

1.2 Algorithms in NC-NC Settings

This paper mainly considers constrained NC-NC minimax problems, i.e., f ( x, y ) is nonconvex in x and nonconcave in y in the constrained setting (called constrained NC-NC). In recent years, some works such as [7, 31, 46, 38, 20] focus on structured NC-NC problems. That is, the saddle gradient operator of such minimax problems or their objectives must satisfy one of the structured assumptions: the minty variational inequality (MVI) condition, the weak MVI condition, or negative comonotone condition and the Polyak-Łojasiewicz condition. However, structured NC-NC problems have limited practical applications because they are required to satisfy strong structural assumptions. For more practical constrained NC-NC problems or more general NC-NC problems, the convergence guarantee of the algorithms is still a challenge. In this paper, we also focus on convergence analysis

for solving more practical constrained NC-NC problems. Another natural question is: can we design a single-loop accelerated algorithm to further improve the bound in the constrained NC-NC setting?

1.3 Motivations and Our Contributions

Motivations: For NC-C minimax problems, can we design a single-loop directly accelerated algorithmwith the gradient complexity lower than the best-known result ˜ O ( glyph[epsilon1] -2 . 5 ) ? Though Smoothed-GDA [51] can obtain the complexity O ( glyph[epsilon1] -2 ) for a special case of Problem (1), it only attains the complexity O ( glyph[epsilon1] -4 ) for NC-C minimax problems. For C-NC minimax problems, for any given x , to solve the nonconcave maximization subproblem with respect to y is NP-hard. As a result, all existing multi-loop algorithms will lose their theoretical guarantees as discussed in [44]. Can we propose a single-loop directly accelerated algorithm with the complexity lower than the best-known result O ( glyph[epsilon1] -4 ) for C-NC and NC-NC minimax problems?

Our Contributions: This paper proposes a novel single-loop extra-gradient difference acceleration algorithm to push towards optimal gradient complexities for constrained NC-NC minimax problems (1), and answer the above-mentioned problems.

We summarize the major contributions of this paper.

· We design a new single-loop accelerating algorithm for solving constrained NC-NC problems. In the proposed algorithm, we design a new extra-gradient difference scheme, and combine the gradient ascent and momentum acceleration steps for the dual variable update. In our algorithm, we present an important quasi-cocoercivity property. By leveraging the quasi-cocoercivity property, we can improve the complexity bound in our theoretical analysis.

· We analyze the convergence properties of the proposed algorithm for constrained NC-NC problems. Theorem 1 shows that to find an glyph[epsilon1] -stationary point of f , the proposed algorithm can obtain the gradient complexity O ( glyph[epsilon1] -2 ) , which is the first time to attains the complexity bound in constrained NC-NC setting. The constrained NC-C and C-NC problems can be viewed as two special cases of the constrained NC-NC problem, and the proposed algorithm is also applicable to these two special problems. And its complexity is still O ( glyph[epsilon1] -2 ) for both NC-C and C-NC problems, which significantly improves the gradient complexity from O ( glyph[epsilon1] -4 ) of existing single-loop algorithms or ˜ O ( glyph[epsilon1] -2 . 5 ) of existing multi-loop algorithms to O ( glyph[epsilon1] -2 ) . The complexities of some recently proposed algorithms are listed in Table 1.

· In order to make a comprehensive comparison with existing algorithms, we also provide the theoretical analysis of our algorithm in terms of another convergence criteria (i.e., an glyph[epsilon1] -stationary point of φ ) for constrained NC-C problems. The result shows that our algorithm improves the best-known result as in [40, 24, 55] from ˜ O ( glyph[epsilon1] -3 ) to O ( glyph[epsilon1] -2 ) , as shown in Table 2.

2 Preliminaries and Related Works

Notation: Throughout this paper, we use lower-case letters to denote vectors such as x, y , and calligraphic upper-case letters to denote sets such as X , Y . For a differentiable function f , ∇ f ( x ) is the gradient of f at x . For a function f ( · , · ) of two variables, ∇ x f ( x, y ) (or ∇ y f ( x, y ) ) is the partial gradient of f with respect to the first variable (or the second variable) at ( x, y ) . For a vector x , ‖ x ‖ denotes its glyph[lscript] 2 -norm. We use P X and P Y to denote projections onto the sets X and Y .

Assumption 1 (Smoothness) . f ( · , · ) is continuously differentiable, and there exists a positive constant L such that

holds for all x 1 , x 2 ∈ R , y 1 , y 2 ∈ R

Definitions of the monotone operators: A operator F ( · ) : R n → R n is monotone, if [ F ( s ) -F ( t )] T ( s -t ) ≥ 0 , ∀ s, t ∈ R n . If [ F ( s ) -F ( t )] T ( s -t ) ≤ 0 , F is negative monotone.

A mapping F ( · ) is co-coercive if there is a positive constant α , such that

Nonconvex-Concave Minimax Optimization: Due to the nonconvex nature of these minimax problems, finding the global solution is NP-hard in general. The recently proposed algorithms aim to find stationary solutions to such problems. For instance, the first-order Nash equilibrium condition (called game stationary) is used as an optimality condition in [31]. Besides game stationary, there are two main optimality criteria (i.e., an glyph[epsilon1] -stationary point of f ( · , · ) or φ ( · ) := max y ∈Y f ( · , y ) ) for the convergence analysis of the algorithms such as [4, 44, 24].

For solving NC-C minimax problems, there exist a number of efficient multi-loop and single-loop algorithms such as [31, 25, 4, 44, 24]. Most of them are multi-loop algorithms, which either employ an accelerated update rule of x by adding regularization terms to its subproblem, or use multiple gradient ascent steps for the update of y to solve the subproblem exactly or inexactly. Compared with multi-loop algorithms, single-loop algorithms are easier to implement. One of the most popular single-loop algorithms is GDA. However, GDA with a constant stepsize can fail to converge even for a simple bilinear minimax problem [29].

To address this issue, only a few single-loop algorithms such as [25, 27, 44] were proposed, and most of them employed a smoothing or proximal point technique. For instance, Smoothed-GDA [51] introduces a smooth function ϕ ( x, y, z )= f ( x, y ) + a 2 ‖ x -z ‖ 2 for the update of the primal variable x , where z is an auxiliary variable and a is a constant, and its main update steps are

where η x , η y > 0 are two stepsizes, and 0 <β ≤ 1 . Smoothed-GDA can obtain the gradient complexity, O ( glyph[epsilon1] -4 ) , for nonconvex-concave minimax problems.

Extra-Gradient Methods: There are some extra-gradient methods such as [48] for solving minimax problems. Let η 0 > 0 and u t be the t -th iterate, the extra-gradient method [53] has the following projection-type prediction-correction step:

In this paper, we call u t +1 / 2 as the prediction point at the t -iteration.

3 Single-Loop Extra-Gradient Difference Acceleration Algorithm

In this section, we propose a single-loop Extra-Gradient Difference Acceleration algorithm (EGDA) for solving constrained NC-NC minimax problems.

Algorithm 1 EGDA for NC-NC minimax problems

1: Initialize: x 0 , y 0 , u 0 , u -1 / 2 , τ > 0 . 5 , η x , η t y , β .

2: for t = 0 , 1 , . . . , T -1 do

3: x t +1 = P X [ x t -η x ∇ x f ( x t , y t )]

; u t +1 / 2 = y t + β [ ∇ y f ( x t , u t -1 / 2 ) -∇ y f ( x t , y t -1 )] ;

4: u t +1 = P Y [ y t + η t y ∇ y f ( x t , u t -1 / 2 ) ] ; y t +1 = τy t +(1 -τ ) u t +1 ;

5: end for

6: Output: ( x T , y T ) .

3.1 Extra-Gradient Difference Acceleration

In recent years, many algorithms such as [27, 31, 32, 24, 51, 44] have been proposed for solving NC-C minimax problems. In essence, most of them are 'indirect' acceleration algorithms, which are used to optimize the surrogate functions with a smoothing or proximal point term instead of the original function. However, this may hurt the performance of these algorithms both in theory and in practice [2, 1]. To address this issue, we propose a single-loop directly accelerating algorithm to find an glyph[epsilon1] -stationary points of f and φ with a significantly improved complexity O ( glyph[epsilon1] -2 ) . The main update steps of our EGDA algorithm are designed as follows.

Gradient descent:

Extra-gradient difference prediction:

Gradient ascent correction:

Momentum acceleration:

Here, 0 < β < 1 L , η x , η t y > 0 are two stepsizes, u t +1 / 2 , u t +1 are auxiliary variables, u t +1 / 2 is a prediction point, and 1 / 2 <τ ≤ 1 is a momentum parameter. Our EGDA algorithm is formally presented in Algorithm 1. Our EGDA algorithm first performs one proximal gradient descent step on the primal variable x , and then we design a new dual-accelerating scheme in (3), (4) and (5) for the dual variable y .

3.2 Advantages of Our Algorithm and Comparison to Related Work

We first design a new prediction point u t +1 / 2 in Eq. (3). Compared with extra-gradient-type methods such as [7, 16, 20], one of the main differences is that the proposed prediction point u t +1 / 2 in (3) is updated by the gradient difference (i.e., ∇ y f ( x t , u t -1 / 2 ) -∇ y f ( x t , y t -1 ) ), while the prediction point in other extra-gradient-type algorithms is updated only by using the gradient information at the correction point u t . Then the gradient at the new prediction point u t +1 / 2 is used in the gradient ascent step (4). And the dual variable y is update by the momentum acceleration step in (5).

Prediction Point: The monotonicity and co-coercivity properties of gradient operators play a crucial role for convergence analysis. However, these important properties do not always hold for nonconvex problems. Some researchers have made great progress in some special nonconvex settings, such as structured nonconvex and weakly convex settings, which require a weaker condition such as weakly monotone [26], pseudo-monotone [16], and MVI [7]. However, such conditions seriously limit the application scope of Problem (1).

To address this challenge, we design a new prediction point scheme in (3), which can help us obtain a useful quasi-cocoercivity property. As a result, it does not require any monotone or structural assumption. Specifically, we find that we only require a weaker property in our theoretical analysis, that is, the co-coercivity is required at some special points { u t +1 / 2 , y t } ( 〈∇ y f ( x t , u t +1 / 2 ) -∇ y f ( x t , y t ) , u t +1 / 2 -y t 〉 ≥ β ‖∇ y f ( x t , u t +1 / 2 ) -∇ y f ( x t , y t ) ‖ 2 with β > 0 ). Thus, we develop a

decoupling idea to construct the prediction point u t +1 / 2 . That is, we use the gradients with respect to y at u t -1 / 2 , y t -1 instead of those at the points u t +1 / 2 , y t . We can obtain a property (called the quasi-co-coercivity) in Section 4.2, which plays a key role in our theoretical analysis.

Gradient Difference: We also briefly discuss the underlying intuition of the proposed gradient difference in (3). We find that the proposed update in (3) is similar to the forms in [43] (see Eqs. (12) and (14) in [43]). [43] proposed a first-order procedure (i.e., difference of gradients) to achieve the negative curvature of a Hessian matrix. Therefore, our algorithm has a similar procedure, which contains second-order information. Moreover, we use the difference of gradients in the gradient ascent procedure for the dual update. But our EGDA algorithm only requires the Lipschitz gradient assumption for minimax problems to find first-order stationary points, while [43] requires both the Lipschitz gradient and Lipschitz Hessian assumptions for solving second-order stationary points of nonconvex minimization problems.

Momentum Acceleration: We design a dual-accelerating update rule in (4) for the dual variable y in our EGDA algorithm, which is different from standard momentum acceleration schemes as in [31, 40, 24]. That is, the accelerated rules of existing algorithms are for the primal variable x , while our accelerated rule is designed for the dual variable y .

Therefore, the proposed new dual-accelerating step (including the gradient different prediction step in (3), the gradient ascent correction in (4) and momentum acceleration in (5)) is a key accelerated technique for our EGDA algorithm, and can help to improve the complexity bound from O ( glyph[epsilon1] -4 ) to O ( glyph[epsilon1] -2 ) . In particular, our EGDA algorithm performs both gradient descent and ascent steps to the original function f . In contrast, many existing algorithms such as [51, 47, 44, 18] optimize their surrogate functions with smoothing terms instead of the original function. In particular, their smoothing parameters need to tune by repeatedly executing the algorithms, which may make them impractical [1]. As in our theoretical guarantees below, the proposed single-loop algorithm is able to significantly improve the best-known gradient complexity, O ( glyph[epsilon1] -4 ) , of existing single-loop algorithms such as [27, 51, 44] to O ( glyph[epsilon1] -2 ) .

4 Convergence Guarantees

In this section, we provide the convergence guarantees of our EGDA algorithm (i.e., Algorithm 1) for solving constrained NC-NC and NC-C problems. We first present the definitions of the two optimality criteria (i.e., an glyph[epsilon1] -stationary point of f or φ ). All the proofs of the lemmas, properties and theorems below are included in the Supplementary Material.

4.1 Optimality Criteria and Key Property

Since finding a global minimum of a nonconvex optimization problem is NP-hard, finding a global saddle point (or Nash equilibrium) of a NC-NC function f is intractable [30]. As in the literature in the NC-NC setting, we introduce the local surrogates (i.e., the stationary point of f ) and in the NC-C setting , we introduce the local surrogates (i.e., the stationary point of f or φ ), whose gradient mappings are equal to zero. Below we define the following two optimality measures (i.e., an glyph[epsilon1] -stationary points of f or φ ) for our theoretical analysis.

Definition 1 (An glyph[epsilon1] -stationary point of f [27]) . A point ( x, y ) ∈ X × Y is an glyph[epsilon1] -stationary point of f ( · , · ) if ‖ π ( x, y ) ‖ ≤ glyph[epsilon1] , where η x and η y are two constants, and

If glyph[epsilon1] = 0 , then ( x, y ) is a stationary point of f .

For the NC-C setting, we also present another convergence criterion used in [40, 24]. Let φ ( x ) := max y ∈Y f ( x, y ) , ˆ x is called an glyph[epsilon1] -stationary point of a smooth primal function φ : R m → R , if ‖∇ φ (ˆ x ) ‖ ≤ glyph[epsilon1] . However, the function φ is not necessarily differentiable for minimax problems. Following [9, 40, 24], we introduce the Moreau envelope of φ for the optimality criterion, especially when φ is a weakly convex function, i.e., φ is L -weakly convex if the function φ ( · ) + L 2 ‖ · ‖ 2 is convex. We refer readers to [9, 24] for the comparison of these two criteria.

Definition 2 (An glyph[epsilon1] -stationary point of L -weakly convex function φ ) . ˆ x is an glyph[epsilon1] -stationary point of an L -weakly convex function φ : R m → R , if ‖∇ φ 1 / (2 L ) (ˆ x ) ‖≤ glyph[epsilon1] , where φ 1 / (2 L ) is the Moreau envelope of φ and is defined as: φ ρ ( x ):=min z φ ( z )+(1 / 2 ρ ) ‖ z -x ‖ 2 . If glyph[epsilon1] = 0 , then ˆ x is a stationary point.

Then we give the following important property for the analysis of the proposed algorithm. By leveraging the property, we can obtain the complexity bound of the proposed algorithm.

Property 1 ( Quasi-Cocoercivity ) . Let u t +1 / 2 be updated in Eq. (3), then

4.2 Core Lemma

Our theoretical results mainly rely on Lemma 1 below, which plays a key role in the proofs of Theorems 1 and 2. Let { ( x t , y t , u t , u t -1 / 2 ) } be a sequence generated by Algorithm 1, and we define the potential function

Next, we need to prove that our defined potential function can make sufficient decrease at each iteration, i.e., G t -G t +1 > 0 as in Lemma 1 below. To prove Lemma 1, we will provide and prove the following upper bounds.

Proposition 1 (Upper bound of primal-dual updates) . Suppose Assumption 1 holds. Let { ( x t , y t , u t ) } be a sequence generated by Algorithm 1 with η t y =min { β ‖∇ y f ( x t , u t -1 / 2 ) -∇ y f ( x t ,y t -1 ) ‖ 2 2 ‖∇ y f ( x t ,u t -1 / 2 ) ‖ 2 , 1 28 L , η x } .

Proposition 2 (Upper bound of dual updates) . Suppose Assumption 1 holds. Let { ( x t , y t , u t ) } be a sequence generated by Algorithm 1, then

Using the optimal condition of Problem (4) and our quasi-cocoercivity in Property 1, we can further bound A 1 + A 2 + A 3 . The proof sketch of Lemma 1 is listed as follows:

Combining the above results and the definition of G t , we can get the descent estimate of the potential function in Lemma 1, which is a main lemma for Theorems 1 and 2 below.

Lemma 1 (Descent estimate of G ) . Suppose Assumption 1 holds. Let { ( x t , y t , u t ) } be a sequence generated by Algorithm 1, then

4.3 Convergence Results

In this subsection, by using the optimality measure in Definitions 1 and 2, we present the following theoretical results in Theorem 1 and Theorem 2 for the constrained NC-NC setting and NC-C setting, respectively.

Assumption 2. Y is a closed, convex and compact set with a diameter D Y > 0 , and X is a closed and convex set.

Furthermore, using Lemma 1 and the optimality measure in Definition 1, we will study the relation between π ( x t , u t ) and the difference (i.e., G t -G t +1 ) to obtain the gradient complexity in Theorem 1 by computing the number of iterations to achieve an glyph[epsilon1] -stationary point of f .

Theorem 1 (Stationarity of f in constrained NC-NC settings) . Suppose Assumptions 1 and 2 hold. Let the two stepsizes η x ≤ 1 4 L , η t y =min { β ‖∇ y f ( x t ,u t -1 / 2 ) -∇ y f ( x t ,y t -1 ) ‖ 2 2 ‖∇ y f ( x t ,u t -1 / 2 ) ‖ 2 , 1 28 L , η x } , β ≤ 1 60 L and τ ≥ 1 / 2 , then the complexity of Algorithm 1 to find an glyph[epsilon1] -stationary point of f is bounded by

where G 0 := G ( x 0 , y 0 ) , and G := min x ∈X φ ( x ) .

Remark 1. For constrained NC-NC problems, the gradient complexity of our EGDA algorithm to find an glyph[epsilon1] -stationary point of f is O ( glyph[epsilon1] -2 ) . That is, our EGDA algorithm is first to obtain the gradient complexity in constrained NC-NC setting. In addition, our method can achieve the same complexity, O ( glyph[epsilon1] -2 ) , as the algorithm with an additional structured assumption for NC-NC problems. However, different from existing algorithms, our algorithm is more practical. That is, it only requires that Y is a compact set, while existing algorithms need some stronger structured assumptions in the structured NC-NC setting. The detailed comparison is shown in Table 1. As two special cases of the constrained NC-NC problem, this theoretical result in Theorem 1 can be extended to the constrained NC-C and C-NC settings. For NC-C problems, the gradient complexity of our EGDA algorithm to find an glyph[epsilon1] -stationary point of f is O ( glyph[epsilon1] -2 ) , while the best-known result of single-loop algorithms such as [27, 51, 44] is O ( glyph[epsilon1] -4 ) , and the best-known result of multi-loop algorithms such as [24] is ˜ O ( glyph[epsilon1] -2 . 5 ) . That is, our EGDA algorithm improves the best-known gradient complexity from ˜ O ( glyph[epsilon1] -2 . 5 ) to O ( glyph[epsilon1] -2 ) . Smoothed-GDA [51] can also obtain the complexity O ( glyph[epsilon1] -2 ) for a special case of Problem (1) and O ( glyph[epsilon1] -4 ) for general NC-C minimax problems, while our algorithm attains the optimal result O ( glyph[epsilon1] -2 ) for all NC-C minimax problems. Existing algorithms such as HiBSA [27] and AGP [44] require the compactness of the domain X in the NC-C setting, while our EGDA algorithm does not, which significantly extends its applicability. For C-NC minimax problems, the proposed algorithm can obtain the complexity, O ( glyph[epsilon1] -2 ) , while the best-known complexity as in [44] is O ( glyph[epsilon1] -4 ) . In other words, the proposed algorithm can improve the best-known result from O ( glyph[epsilon1] -4 ) to O ( glyph[epsilon1] -2 ) .

By using the criterion in Definition 2 as the optimality measure, we use the definition of φ 1 / 2 L and introduce the property of ∇ φ 1 / 2 L as in [24]. With a similar setting for varying stepsizes as in [24], we study the relation between ∇ φ 1 / 2 L and the basic descent estimation of the potential function in Lemma 1 by using the property of ∇ φ 1 / 2 L , and compute the number of iterations to achieve an glyph[epsilon1] -stationary point of φ . We provide the following theoretical result and its detailed proof in the Supplementary Material.

Theorem 2 (Stationarity of φ for constrained NC-C settings) . Using the same notation as in Theorem 1, and f is concave with respect to y . Let { ( x t , y t , u t ) } be a sequence generated by Algorithm 1 with the stepsizes η t y =min { βa 2 t 2 b 2 t , 1 28 L , βa 2 t 2 b t , βc 2 t 2 d t } , where a t := ‖∇ y f ( x t ,u t -1 / 2 ) -∇ y f ( x t ,y t -1 ) ‖ , b t := ‖∇ y f ( x t , u t -1 / 2 ) ‖ , c t := ‖∇ y f ( x t , u t -1 / 2 ) ‖ , d t := ‖∇ y f ( x t , u t ) -∇ y f ( x t , u t -1 / 2 ) ‖ . Then the gradient complexity of Algorithm 1 to find an glyph[epsilon1] -stationary point of φ is bounded by

Remark 2. From Theorem 2, it is clear that the gradient complexity of the proposed algorithm is O ( glyph[epsilon1] -2 ) . That is, Algorithm 1 can improve the best-known gradient complexity from ˜ O ( glyph[epsilon1] -3 ) as in [24] to O ( glyph[epsilon1] -2 ) . Therefore, Algorithm 1 is the first algorithm, which attains the gradient complexity O ( glyph[epsilon1] -2 ) to find an glyph[epsilon1] -stationary point of φ for NC-C minimax problems.

0

100

200

300

400

500

Number of iterations

10 -30

10 -20

10 -10

10 0

GDA

EG

FEG

EGDA

0

100

200

300

400

500

Number of iterations

10 -30

10 -20

10 -10

10 0

GDA

EG

FEG

EGDA

5 Numerical Results

We conduct many experiments to illustrate the performance of the proposed algorithm for solving NC-NC, NC-C and convex-concave problems.

NC-NC Problems. We conduct some experiments to illustrate the performance of the proposed algorithm, EGDA, for solving NC-NC problems. Moreover, we compare it against existing methods such as GDA [46], EG [10], EAG [49] and FEG [20]. Detailed setup is provided in the Supplementary Material.

We compare EGDA with GDA, EG and FEG for solving a simple NC-NC minimax problem (i.e., f ( x, y )= x 2 +3sin 2 x sin 2 y -4 y 2 -10 sin 2 y ), which satisfies the Polyak-Łojasiewicz condition, as shown in Fig. 1. All the results show that EGDA indeed converges to the global saddle point and is significantly faster than other methods including GDA and FEG in terms of both ‖ x t -x ∗ ‖ 2 + ‖ y t -y ∗ ‖ 2 and ‖∇ x f ‖ 2 + ‖∇ y f ‖ 2 . Although FEG has a fast theoretical rate, O (1 /t 2 ) , with a negative monotone assumption, it converges much slower than EGDA in practice.

0

100

200

300

400

500

Number of iterations

10 -2

10 0

GDA

EAG

EGDA

8

6

2

4

2

n

0

十

1

-2

GDA

···..·EAG

-4

EGDA

-6

米

saddlepoint

o

Initial point

-8

-6

-4

-2

0

2

4

6

Convex-Concave Problems. Fig. 2 shows the convergence results of the methods including GDA, EAG [49] and EGDA for solving the convex-concave problem, f ( x, y ) = log(1 + e x ) + 3 xy -log(1 + e y ) . It is clear that EGDA converges much faster than other methods such as EAG. We also observe empirically when the same step-size is used, even if small, GDA may not converge to stationary points, and it is proven to always have bounded iterates.

NC-C Problems. We also apply our EGDA algorithm to train robust neural networks against adversarial attacks on Fashion MNIST and MNIST, and verify our theoretical results. In [15, 28, 31],

10°

-GDA

100

-GDA

1111

MGDA

111111

MGDA

Smoothed-GDA

Smoothed-GDA

EGDA

EGDA

10

10~1

W

Loss

Loss

10~2

10

2

MIA

10

-3

0

20

40

60

80

100

0

20

40

60

80

100

Epoch

Epoch

the robust training procedure can be formulated into a NC-C minimax optimization problem. It is clear that the minimax problem is nonconvex in x , but concave in y .

FGSM [15] and PGD [19] are two popular methods for generating adversarial examples. To obtain the targeted attack ˆ x ij , we use the same procedure as in [31, 51] as follows. The perturbation level ε is chosen from { 0 . 0 , 0 . 1 , 0 . 2 , 0 . 3 , 0 . 4 } , and the stepsize is 0 . 01 . Note that the number of iterations is set to 40 for FGSM and 10 for PGD, respectively. The details of the network are included in the Supplementary Material. We compare our EGDA method with GDA [25], MGDA [31] and Smoothed-GDA [51], and illustrate the convergence of all the algorithms on the loss function in Fig. 3. The results show that Smoothed-GDA and EGDA converge significantly faster than other algorithms. This verifies that they have a faster convergence rate, O ( glyph[epsilon1] -2 ) . Moreover, EGDA converges much faster than Smoothed-GDA.

6 Conclusions and Future Work

In this paper, we proposed a new single-loop accelerated algorithm for solving various constrained NC-NC minimax problems. In the proposed algorithm, we designed a novel extra-gradient difference scheme for dual updates. Moreover, we provided the convergence guarantees for the proposed algorithm, and the theoretical results show that to find an glyph[epsilon1] -stationary point of f , the proposed algorithm obtains the complexity bound O ( glyph[epsilon1] -2 ) for constrained NC-NC problems. That is, this is the first time that the proposed algorithm attains the complexity bound O ( glyph[epsilon1] -2 ) in constrained NC-NC settings (including constrained NC-C and C-NC). For NC-C problems, we provided the theoretical guarantees of the proposed algorithm under the stationarity of φ , which show that our algorithm improves the complexity bound from ˜ O ( glyph[epsilon1] -3 ) to O ( glyph[epsilon1] -2 ) . Experimental results also verified the theoretical results of the proposed algorithm, which have the factors of glyph[epsilon1] -1 and glyph[epsilon1] -2 faster than existing algorithms, respectively. For further work, we will extend our directly accelerating algorithm to stochastic, non-smooth, nonconvex-nonconcave and federated learning settings as in [47, 11, 4, 55, 45, 21, 37].

Acknowledgments

We want to thank the anonymous reviewers for their valuable suggestions and comments. This work was supported by the National Key R&D Program of China (No. 2022ZD0160302), the National Natural Science Foundation of China (Nos. 6227071567, 61976164, 62276004 and 61836009), the National Science Basic Research Plan in Shaanxi Province of China (No. 2022GY-061), the major key project of PCL, China (No. PCL2021A12), and Qualcomm.

References

Z. Allen-Zhu. Katyusha: The first direct acceleration of stochastic gradient methods. In STOC , 2017.

Z. Allen-Zhu and Y. Yuan. Improved SVRG for non-strongly-convex or sum-of-non-convex objectives. In ICML , pages 1080-1089, 2016.

J. P. Bailey, G. Gidel, and G. Piliouras. Finite regret and cycles with fixed step-size via alternating gradient descent-ascent. In The Annual Conference on Learning Theory (COLT) , pages 391-407, 2020.

R. I. Bot and A. Bohm. Alternating proximal-gradient steps for (stochastic) nonconvex-concave minimax problems. SIAM J. Optim. , 2023.

Y. Cai and W. Zheng. Accelerated single-call methods for constrained min-max optimization. In International Conference on Learning Representations (ICLR) , 2023.

Z. Chen, Z. Hu, Q. Li, Z. Wang, and Y. Zhou. A cubic regularization approach for finding local minimax points in nonconvex minimax optimization. Transactions on Machine Learning Research , pages 1-43, 2023.

C. D. Dang and G. Lan. On the convergence properties of non-euclidean extragradient methods for variational inequalities with generalized monotone operators. Comput. Optim. Appl. , 60(2):277-310, 2015.

C. Daskalakis and I. Panageas. The limit points of (optimistic) gradient descent in min-max optimization. In The Annual Conference on Neural Information Processing Systems (NeurIPS) , pages 9256-9266, 2018.

D. Davis and D. Drusvyatskiy. Stochastic model-based minimization of weakly convex functions. SIAM J. Optim. , 29(1):207-239, 2019.

J. Diakonikolas, C. Daskalakis, and M. I. Jordan. Efficient methods for structured nonconvex-nonconcave min-max optimization. In AISTATS , pages 2746-2754, 2021.

J. Diakonikolas, C. Daskalakis, and M. I. Jordan. Efficient methods for structured nonconvex-nonconcave min-max optimization. In The International Conference on Artificial Intelligence and Statistics (AISTATS) , pages 2746-2754, 2021.

S. S. Du, J. Chen, L. Li, L. Xiao, and D. Zhou. Stochastic variance reduction methods for policy evaluation. In International Conference on Machine Learning (ICML) , pages 1049-1058, 2017.

J. C. Duchi and H. Namkoong. Variance-based regularization with convex objectives. J. Mach. Learn. Res. , 20:1-55, 2019.

I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In The Annual Conference on Neural Information Processing Systems (NIPS) , pages 2672-2680, 2014.

I. Goodfellow, J. Shlens, and C. Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations (ICLR) , 2015.

A. N. Iusem, A. Jofr´ e, R. I. Oliveira, and P. Thompson. Extragradient method with variance reduction for stochastic variational inequalities. SIAM J. Optim. , 27(2):686-724, 2017.

C. Jin, P. Netrapalli, and M. I. Jordan. What is local optimality in nonconvex-nonconcave minimax optimization? In International Conference on Machine Learning (ICML) , pages 4880-4889, 2020.

W. Kong and R. D. Monteiro. An accelerated inexact proximal point method for solving nonconvex-concave min-max problems. In arXiv preprint arXiv:1905.13433v3 , 2021.

A. Kurakin, I. Goodfellow, and S. Bengio. Adversarial machine learning at scale. In International Conference on Learning Representations (ICLR) , 2017.

S. Lee and D. Kim. Fast extra gradient methods for smooth structured nonconvex-nonconcave minimax problems. In NeurIPS , 2021.

Y. Lei, Z. Yang, T. Yang, and Y. Ying. Stability and generalization of stochastic gradient methods for minimax problems. In International Conference on Machine Learning (ICML) , pages 6175-6186, 2021.

A. Letcher, D. Balduzzi, S. Racaniere, J. Martens, J. N. Foerster, K. Tuyls, and T. Graepel. Differentiable game mechanics. J. Mach. Learn. Res. , 20:1-40, 2019.

H. Li, Y. Tian, J. Zhang, and A. Jadbabaie. Complexity lower bounds for nonconvex-strongly-concave min-max optimization. In arXiv preprint arXiv:2104.08708v1 , 2021.

T. Lin, C. Jin, and M. I. Jordan. Near-optimal algorithms for minimax optimization. In The Annual Conference on Learning Theory (COLT) , pages 2738-2779, 2020.

T. Lin, C. Jin, and M. I. Jordan. On gradient descent ascent for nonconvex-concave minimax problems. In International Conference on Machine Learning (ICML) , pages 6083-6093, 2020.

M. Liu, H. Rafique, Q. Lin, and T. Yang. First-order convergence theory for weakly-convex-weakly-concave min-max problems. J. Mach. Learn. Res. , 22:169:1-169:34, 2021.

S. Lu, I. Tsaknakis, M. Hong, and Y. Chen. Hybrid block successive approximation for one-sided nonconvex min-max problems: algorithms and applications. IEEE Trans. Signal Process. , 68:3676-3691, 2020.

A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations (ICLR) , 2018.

A. Mokhtari, A. Ozdaglar, and S. Pattathil. A unified analysis of extra-gradient and optimistic gradient methods for saddle point problems: Proximal point approach. In The International Conference on Artificial Intelligence and Statistics (AISTATS) , pages 1497-1507, 2020.

K. G. Murty and S. N. Kabadi. Some NP-complete problems in quadratic and nonlinear programming. Mathematical Programming , 39(2):117-129, 1987.

M. Nouiehed, M. Sanjabi, T. Huang, J. D. Lee, and M. Razaviyayn. Solving a class of non-convex min-max games using iterative first order methods. In NeurIPS , pages 14905-14916, 2019.

D. M. Ostrovskii, A. Lowy, and M. Razaviyayn. Efficient search of first-order Nash equilibria in nonconvexconcave smooth min-max problems. SIAM J. Optim. , 2021.

Y. Ouyang and Y. Xu. Lower complexity bounds of first-order methods for convex-concave bilinear saddle-point problems. Mathematical Programming , 185:1-35, 2021.

H. Rafique, M. Liu, Q. Lin, and T. Yang. Weakly-convex concave min-max optimization: Provable algorithms and applications in machine learning. Optim. Method Softw. , 2022.

M. Razaviyayn, T. Huang, S. Lu, M. Nouiehed, M. Sanjabi, and M. Hong. Nonconvex min-max optimization: Applications, challenges, and recent theoretical advances. IEEE Signal Process. Mag. , 37(5):55-66, 2020.

S. Shafieezadeh-Abadeh, P. M. Esfahani, and D. Kuhn. Distributionally robust logistic regression. In The Annual Conference on Neural Information Processing Systems (NIPS) , pages 1576-1584, 2015.

P. Sharma, R. Panda, G. Joshi, and P. K. Varshney. Federated minimax optimization: Improved convergence analyses and algorithms. In International Conference on Machine Learning (ICML) , pages 19683-19730, 2022.

C. Song, Z. Zhou, Y. Zhou, Y. Jiang, and Y. Ma. Optimistic dual extrapolation for coherent non-monotone variational inequalities. In NeurIPS , 2020.

C. Tan, T. Zhang, S. Ma, and J. Liu. Stochastic primal-dual method for empirical risk minimization with o (1) per-iteration complexity. In The Annual Conference on Neural Information Processing Systems (NeurIPS) , pages 8376-8385, 2018.

K. K. Thekumprampil, P. Jain, P. Netrapalli, and S. Oh. Efficient algorithms for smooth minimax optimization. In The Annual Conference on Neural Information Processing Systems (NeurIPS) , pages 12659-12670, 2019.

H.-T. Wai, Z. Yang, Z. Wang, and M. Hong. Multi-agent reinforcement learning via double averaging primal-dual optimization. In The Annual Conference on Neural Information Processing Systems (NeurIPS) , pages 9672-9683, 2018.

G. Xie, L. Luo, Y. Lian, and Z. Zhang. Lower complexity bounds for finite-sum convex-concave minimax optimization problems. In International Conference on Machine Learning (ICML) , pages 10504-10513, 2020.

Y. Xu, R. Jin, and T. Yang. First-order stochastic algorithms for escaping from saddle points in almost linear time. In NeurIPS , pages 5535-5545, 2018.

Z. Xu, H. Zhang, Y. Xu, and G. Lan. A unified single-loop alternating gradient projection algorithm for nonconvex-concave and convex-nonconcave minimax problems. Mathematical Programming , 2023.

J. Yang, N. Kiyavash, and N. He. Global convergence and variance-reduced optimization for a class of nonconvex-nonconcave minimax problems. In The Annual Conference on Neural Information Processing Systems (NeurIPS) , 2020.

J. Yang, N. Kiyavash, and N. He. Global convergence and variance reduction for a class of nonconvexnonconcave minimax problems. In NeurIPS , 2020.

J. Yang, S. Zhang, N. Kiyavash, and N. He. A catalyst framework for minimax optimization. In The Annual Conference on Neural Information Processing Systems (NeurIPS) , 2020.

T. Yoon and E. K. Ryu. Accelerated algorithms for smooth convex-concave minimax problems with o(1/kˆ2) rate on squared gradient norm. In M. Meila and T. Zhang, editors, ICML , pages 12098-12109, 2021.

T. Yoon and E. K. Ryu. Accelerated algorithms for smooth convex-concave minimax problems with o (1 /k 2 ) rate on squared gradient norm. In ICML , pages 12098-12109, 2021.

J. Zhang, M. Hong, and S. Zhang. On lower iteration complexity bounds for the saddle point problems. Mathematical Programming , 194:901-935, 2022.

J. Zhang, P. Xiao, R. Sun, and Z. Luo. A single-loop smoothed gradient descent-ascent algorithm for nonconvex-concave min-max problems. In The Annual Conference on Neural Information Processing Systems (NeurIPS) , pages 7377-7389, 2020.

S. Zhang, J. Yang, C. Guzman, N. Kiyavash, and N. He. The complexity of nonconvex-strongly-concave minimax optimization. In The Conference on Uncertainty in Artificial Intelligence (UAI) , 2021.

W. Zhang and D. Han. A new alternating direction method for co-coercive variational inequality problems. Comput. Math. Appl. , 57(7):1168-1178, 2009.

Y. Zhang and L. Xiao. Stochastic primal-dual coordinate method for regularized empirical risk minimization. J. Mach. Learn. Res. , 18:1-42, 2017.

R. Zhao. A primal dual smoothing framework for max-structured nonconvex optimization. Mathematics of Operations Research , 2023.

Peer reviews:
comments: SUMMARY:
Authors propose method of solving nonconvex-nonconcave saddle point problems with convergence rate O(eps^-2) by using gradient difference prediction and momentum acceleration to improve extragradient descent-ascent method. Proposed method is state-of-the-art in theory and leading in practice, including neural network learning with adversarial attacks task.

STRENGTHS:
Quite elegant construction of the algorithm, which also allows one to obtain best-known convergence rate guarantees. Algorithms allows practitioners to address the most practically important setting of nonconvex-nonconcave problems efficiently using easy to implement algorithm which will surely replace analogous methods, judging from empirical study.

WEAKNESSES:
No significant weaknesses

QUESTIONS:
Authors could provide more extensive empirical study, because algorithm seems to be candidate for being widely-accepted in practice and it would be good to have more justifications of its efficiency.

LIMITATIONS:
Everything is okay
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-06-28

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors have designed a single-loop accelerated algorithm for constrained min-max optimization problems of the form $\\min_{x\\in X}\\max_{y\\in Y} f(x,y)$. The algorithm provably converges in an approximate local stationary point in three particular setting: \n1. Non-convex non-concave min-max optimization, where the stationarity is measured for the function $f(x,y)$.\n2. Convex non-concave min-max optimization, and non-convex concave min-max optimization, where the stationarity is measured for function $\\phi(x)=\\max_{y'\\in Y} f(x,y')$. \n\nThe authors showed that their algorithm computes an $\\epsilon$-stationary point, in $O(1/\\epsilon^2)$ iterations. Finally, they experimentally verify their proposed algorithm.\n\n\n\nThe authors' rebuttal addressed my concerns, and their additional empirical evidence complemented their already compelling results. For these reasons, I decided to increase my score.\n\nSTRENGTHS:\nThe design and analysis of algorithms for non-convex non-concave minimax optimization is a fundamental problem, and the convergent results are indeed compelling. Moreover, the authors get the state-of-the-art for convex non-concave, and concave non-convex for the merit function they consider. Furthermore the main paper is well-written and easy to follow, and the algorithm seems to combine several interesting ideas.\n\nI verified the proofs of proposition 1 and 2, and the rest of the statement seems reasonable. I found the proofs of proposition 1 and 2 to be a bit dense, which made verifying them somewhat taxing.\n\nWEAKNESSES:\nOverall, I did not find some important weakness in the paper. As a suggestion, improving the readability and verifiability of the proofs could greatly benefit readers. Lastly, I came across a couple of typos:\n- In line 242, I think \"conference\" was meant to be \"convergence\".\n- When considering the proof of proposition 2 in the appendix, is $\\widehat{u}_{t+1/2}$ identical to the one referred to in line 706? If so, clarifying this might prevent confusion.\n- In line 3 in Algorithm 1, do you also need to initialize $y_{-1}$ for the first iteration of the algorithm?\n\nQUESTIONS:\nThe recent work in [1], shows that the computation of an approximate stationary point is PPAD-complete when the action space of the two players is a joint. Notably, the findings in your paper seem to suggest a different outlook when the strategy space of the two players is a product space, which negates the hardness results. A comment from the authors addressing this observation would be greatly appreciated.\n\nFurthermore, the proofs of the propositions and lemmas presented appear to be quite opaque, and verifying them poses a bit of a challenge. It would be truly beneficial if the authors could share some additional insights into the process behind the design and analysis of the algorithm. I think it would be good if the authors could provide a better commentary about the derivations of the proofs.\n\n[1] \"The Complexity of Constrained Min-Max Optimization\" by Constantinos Daskalakis, Stratis Skoulakis, and Manolis Zampetakis\n\nLIMITATIONS:\nAny assumptions needed for the theorems to hold are listed. I do not believe this work can have negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 9, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work proposes a single-loop extra-gradient difference acceleration algorithm to find an \\epsilon-stationary point for constrained minimax optimization, which pushes forward the best complexity bounds of NC-NC, C-NC, NC-C problems to \\mathcal{O}(\\epsilon^{-2}). The proposed approach can deal with more general problems as it does not require monotone or structural assumption. Moreover, for the NC-C problem, the authors prove that the proposed method has better complexity bound under the stationarity of \\phi. Experiments are conducted to validate the method empirically. The results show that it can achieve better convergence rate when comparing with the related methods.\n\nSTRENGTHS:\n1.\tThe theoretical contributions are significant. The method employs a novel prediction point scheme to obtain the quasi-cocoercivity property, which relaxes the assumption requirements. Additionally, the paper provides a thorough analysis of the convergence complexity bound, demonstrating its superiority over the current state-of-the-art approaches.\n\n2.\tThe paper is well organized and easy to follow. The logical flow of ideas is well-structured, enhancing the overall readability and comprehension of the presented contents.\n\n3.\tEmpirical studies are conducted to validate the method in both synthetic and real tasks.\n\nWEAKNESSES:\nThere are some possible limitations where the paper could be further improved.\n\n1.\tI suggest the authors to undertake additional analysis of the algorithm's time complexities, both theoretically and empirically. This deeper exploration would provide valuable insights, particularly for potential industrial applications.\n\n2.\tThe absence of experiments conducted on the C-NC problem should be explained within the paper.\n\n3.\tThere are some empirical evidences that seem to be inconsistent with the theoretical result, for example, FEG has a fast theoretical rate but is less effective in practice; GDA may not converge to stationary points. The authors may explain more about these in the paper.\n\nQUESTIONS:\nplease see above.\n\nLIMITATIONS:\nno negative societal impact", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper discusses a new extra-gradient difference acceleration algorithm for solving constrained nonconvex-nonconcave minimax problems. The algorithm introduces a \"quasi-cocoercivity property\" and momentum acceleration to significantly improve the convergence rate in the constrained NC-NC setting. The algorithm attains a complexity of $O(\\epsilon^{-2})$ for finding an $\\epsilon$-stationary point of the function $f$, which outperforms the best-known complexity bounds. The paper also provides theoretical analysis and comparisons with existing algorithms.\n\nSTRENGTHS:\nAs a person who works in minimax optimization, I can make a fair judgment of this work. This paper presents a novel extra-gradient difference acceleration algorithm for solving constrained nonconvex-nonconcave minimax problems, which improves the existing convergence rate and outperforms the best-known complexity bounds to $O(\\epsilon^{-2})$. The paper also provides a comprehensive comparison with existing algorithms and a theoretical analysis of the algorithm's performance. I understand the \"extra-gradient difference prediction\" step as the key to the success of convergence rate improvements. In addition, I went through the proofs of Theorems 1 and 2 in detail. Overall, this paper provides valuable contributions to the field of minimax optimization and presents a promising algorithm for solving constrained NC-NC problems.\n\nWEAKNESSES:\nThe paper assumes that the objective function satisfies certain structural assumptions, which may limit its practical applications. Also the writing style might not be as friendly for readers unfamiliar with the topic (I found it sufficiently clear though).\n\nQUESTIONS:\n---Can you explain more about the \"quasi-cocoercivity\" property and discuss how it improves the convergence rate in the constrained NC-NC setting? Is this an absolutely necessary property for the improved convergence rate to hold?\n\n---Can your algorithm or its variants be applied to other minimax optimization problems beyond the constrained NC-NC setting?\n\n---There do exist some typos. For example in Eq. (4) in Line 215, missing factor 2 in the denominator. Should be minor but the authors should check more and correct them.\n\nLIMITATIONS:\nThis paper is purely theoretical and does not admit negative social impacts to my best knowledge.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-10"}