Tight Analysis of Extra-gradient and Optimistic Gradient Methods For Nonconvex Minimax Problems
Authors: Pouria Mahdavinia
Yuyang Deng
Haochuan Li
Mehrdad Mahdavi
Venue: NeurIPS 2022 Accept (2022)
Keywords: Minimax Optimization
Nonconvex Optimization

Abstract: Despite the established convergence theory of Optimistic Gradient Descent Ascent (OGDA) and Extragradient (EG) methods for the convex-concave minimax problems, little is known about the theoretical guarantees of these methods in nonconvex settings. To bridge this gap, for the first time, this paper establishes the convergence of OGDA and EG methods under the nonconvex-strongly-concave (NC-SC) and nonconvex-concave (NC-C) settings by providing a unified analysis through the lens of single-call extra-gradient methods. We further establish lower bounds on the convergence of GDA/OGDA/EG, shedding light on the tightness of our analysis. We also conduct experiments supporting our theoretical results. We believe our results will advance the theoretical understanding of OGDA and EG methods for solving complicated nonconvex minimax real-world problems, e.g., Generative Adversarial Networks (GANs) or robust neural networks training.

Tight Analysis of Extra-gradient and Optimistic Gradient Methods For Nonconvex Minimax Problems
Pouria Mahdavinia Pennsylvania State University pxm54260psu. edu
Yuyang Deng Pennsylvania State University yzd820psu. edu
Haochuan Li Massachusetts Institute of Technology haochuan@mit.. edu
Mehrdad Mahdavi Pennsylvania State University mzm61 60psu edu
Abstract
Despite the established convergence theory of Optimistic Gradient Descent As- cent (OGDA) and Extragradient (EG) methods for the problems, little known about the theoretical guarantees of these methods in nonconvex settings. To bridge this gap, for the first time, this paper establishes the convergence of OGDA and EG methods under the nonconvex wwwptueee--stooyy-oooo conccvve (NC-SC) and through the lens of single-call extra- extra-gradien gradient methods. We further establish lower bounds on the convergence of GDA/OGDA/EG, shedding light on the tightness of our analysis. We also conduct experiments supporting our theoretical results. We believe our results will advance the theoretical understanding of OGDA and EG methods for solving complicated nonconvex minimax real-world problems, c.g., Generative Adversarial Networks (GANs) robust neural networks training.
minimax
(NC-C) settings by providing a unified analysis
Introduction
In this paper, we consider the following minimax problem:
min max f(x, y) æERd wEy
where y could be bounded convex unbounded
and the function f Rd ) R is smooth and ctoonlly-omccav//concaa with respect to y. but possibly nonconvex in æ. Minimax optimization (Problem has been explored
variety of fields, including classical game theory, online learning. and control theory [2, 50, 21]. Minimax has emerged as key optimization framework for machine learning applications such as generative adversarial networks (GANs) [14], robust and adversarial machine learning [46, 37, 15], and reinforcement learning [54, 43].
Gradient descent ascent (GDA) is well-known algorithm for solving minimax problems, and it widely used to optimize generative adversarial networks. GDA performs gradient descent step on the primal variable and gradient ascent step the dual variable y simultaneously in each iteration. GDA with equal step sizes for both variables converges linearly to Nash equilibrium under the strongly-conve strongly-concave (SC-SC) assumption [28, 12], but diverges under the convex-concave (C-C) setting for functions such as bilinear [22, 38).
Given the high nonconvexity of practical applications such GANs, exploring convergence guar- antees of minimax optimization algorithms beyond the convex-concave (C-C) setting is one of the canonical research directions in minimax optimization. Several algorithms with convergence guaran- tees beyond the C-C domain have been explored the literature. Alternating Gradient Descent Ascent
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

NC-C
NC-SC
Algorithm
Deterministic
Stochastic
Deterministic
Stochastic
PG-SVRG [44]
0
HiBSA [36]
0
Prox-DIAG [48]
Ō
Minimax-PPA [31]
O(c
0
ALSET [4]
0 ((
Smoothed-AGDA [52]
(
0
GDA [30]
O(e 9) O(e-9)
Ofe ")
((#)
OGDA/EG (Theorems 4.2, 4.4, 4.8, 4.9)
O(e-))
0
0
Table I: A summary of prior and our convergence rates in nonconvex-concat (NC-C) and nonconvex- strongly-concave (NC-SC) minimax optimization. For NC-C, assume f(x y) f-smooth, G- Lipschitz in æ, and concave in y, and for NC-SC we assume Ê-smoothness, and j--strng concavity y, where K f/ denote the condition number.
(AGDA) is one of these methods demonstrated to have excellent convergence properties beyond the C-C setting [51, 52, 63. Additionally, two alternative powerful algorithms are Extragradient (EG) and Optimistic GDA (OGDA), which have recently acquired prominence due to their superior empirical performance in optimizing GANs compared to other minimax optimization algorithms [28, 8, 38]. Spurred by the empirical success EG and OGDA methods, there has been a tremendous amount of work in theoretical understanding of their convergence rate under different sets of assumptions. Specifically, recently the convergence properties of EG and OGDA were investigated for SC-SC and C-C settings, where it has been shown that they tend to converge significantly faster than GDA in both deterministic and stochastic settings [39, 12, 40J. Despite these remarkable advances, there is a dearth of theoretical understanding of the convergence of OGDA and EG methods in the nonconvex setting. This naturally motivates us to rigorously examine the convergence of these methods in nonconvex minimax optimization that we aim investigate. Thus, we emphasize that our focus on vanilla variants of OGDA/EG, and improved rates in NC-C and NC-SC problems have already been obtained with novel algorithms as mentioned in Section 2.
Contributions. We propose a unified framework for analyzing and establishing the convergence of OGDA and EG methods for solving NC-SC and NC-C minimax problems. To the best of our knowledge, our analysis provides the first theoretical guarantees for such problems. Our contribution can be summarized follows:
For NC-SC objectives, we demonstrate that OGDA and EG iterates converge to the
e etstationary point, with gradient complexity of O + for deterministic case, and O ((+)) for the stochastic setting, matching the gradient complexity of GDA [30].
For NC-C objectives, we establish the gradient complexity of O(c- for the deterministic and (EE-8) for stochastic oracles, respectively. Compared to the most analogous work on GDA [30], our rate matches the gradient complexity of GDA our results show that OGDA and EG have the advantage of shaving off significant term related to primal function gap
(Âo (xx)) min (x))).
We establish impossibility results on the achievable rates by providing an 2 ), and 22
(c==)) lower bounds based on the common choice of parameters for both OGDA and EG in deterministic NC-SC and NC-C settings, respectively, thus demonstrating the tightness of our analysis of upper bounds.
By carefully designing hard instances, establish a general lower bound of O(5), inde- pendent of the learning rate, for GDAJOGDA/EG methods in deterministic NC-SC setting- demonstrating the optimality of obtained upper bound up to factor of x.
2 Related Work
Extra-gradient (EG), and OGDA methods. Under smooth SC-SC assumption, deterministic OGDA and EG have been shown to converge to an O(e) neighborhood the optimal solution with rate of Ok log( E)) [39, 49]. Fallah et al. [12] improved upon the previous rates by proposing multistage

---

OGDA, which achieved the best-known rate of (max log( (لحي)) )) for the stochastic OGDA in SC-SC setting. Under monotone and gradient Lipschitzness assumption (a slightly weaker notion of smooth convex-concave problems), Cai et al. [3] established the tight last iterate convergence of 0 for OGDA and EG, and similar results for EG has been achieved in [17, 16]. Furthermore, To the best of our knowledge, OGDA and EG methods have not been extensively explored in nonconvex- nonconcave settings except in a few recent works on structured conconvex-nonconee
problems in which the analysis is done through the lens of a variational incquality. This line of work discussed in section. Moreover, recently, Guo et al. [18] established the convergence rate of OGDA in NC-SC, however, they have J--LL assumption on + (z),, which is a strong assumption and further allows them to show the convergence rate in terms of the objective gap. However, we did not make such an assumption on the primal function, and hence unlike [18], we measure the convergence by the gradient norm of the primal function.
the manconvex-nonconcnn
(NC-SC) problems. In deterministic setting, Lin et al. [30] demon- strated the first asn-asymt convergence of GDA to e-stationary point of ((x),, with the gradient complexity of 0 ( Lin et al. [31] and Zhang al. [55] proposed triple loop algorithms achieving
Nonconvex-strongly-ee
gradient complexity of O ((2)) by leveraging ideas from catalyst methods (adding ar|Eæ zoll2 to the objective function), and inexact proximal point methods, which nearly match the existing lower bound [27, 55, 20J. Approximating the inner loop optimization of catalyst idea by one step of GDA, Yang et al [52] developed a single loop algorithm called smoothed AGDA, which provably con- verges to e-stationary point, with gradient complexity of O ( ). For stochastic setting, Lin et al [30] showed that Stochastic GDA, with choosing dual and primal learning rate ratio of O ((+)), converges
to c-stationary point with gradient complexity of 0 Chen et al. [4] proposed a double loop algorithm whose outer loop performs one step of gradient descent on the primal variable, and inner loop performs multiple steps of gradient ascent. Using this idea, they achieved gradient complexity of ((2+) with fixed batch size. However, their algorithm is double loop, and the iteration complexity of the inner loop is (k). Yang et al [52] also introduced the stochastic version of smoothed AGDA we mentioned earlier. They showed gradient complexity of O ( using fixed batch size. They achieved the best-known rate for NC-PL. problems, which is an even weaker assumption than NC-SC.
Nonconvex-concave Recently, due to the surge of GANs [14] and adversarially robust neural network training, a line of researches are focusing on nonconvex -concve or even nonconvex -oonneee-oononccveee minimax optimization problems [36, 29, 41, 44, 48, 13, 32, 33, 24]. For nonconvex-conee setting, to our best knowledge, Rafique et al [44] is the pioneer to propose provable nonconvex- concave minimax algorithm, where they proposed Proximally Guided Stochastic Mirror Descent Method, which achieves ((c=9) gradient complexity to find stationary point. Nouiehed et al [41] presented a double-loop algorithm to solve nonconvex-conean with constraint on both æ and y. and achieved O(- rate. Lin et al [30] provided the first analysis of the classic algorithm (S)GDA on and nonconvex-coneet
functions, and in nonconvex-concave setting they achieve 66) for GDA and ((c-)) ) for SGDA. Zhang et al [53] proposed smoothed-GDA and also achieve O(E- rate. Thekumparampil et al. [48] proposed Proximal Dual Implicit Accelerated Gradient method and achieved the best known rate O for cecooonve--coccae problem. Kong and Monteiro [26] proposed an accelerated inexact proximal point method and also achieve O(E 3) rate. Lin et al [31] designed near-optimal algorithm using an acceleration method with O(c-a) rate. However, their algorithms require double or triple loops and are not as easy to implement as GDA, OGDA, or EG methods
nonconvex wwwpeeee-sstooyyy-ooeeee
Nonconvex-noneoneen Minimax optimization problems can be cast as one of the special cases of variational inequality problems (VIPs) [1, 34]. Thus, one way of studying the convergence in Nonconvex-nonconcaan problems is to leverage some variants of Variational Inequality properties such as Monotone variational inequality, Minty variational inequality (MVI), weak MVI, and negative comonotone, which are weaker assumptions compared to convex-concave problems. For instance, Loizou et al. [35] showed the linear convergence of SGDA under expected co-coercivity. a condition that potentially holds for the non-monotone problem. Moreover, it has been shown that deterministic EG obtains gradient complexity of O( (,)) for the aforementioned settings [7, 10, 47,47, 42]. Alternatively, another line of works established the convergence under the weaker notions of strong convexity such as the convergence of the AGDA algorithm assuming the two-sided PI. condition. Hajizadeh et al [19] achieved the same results for EG under the weakly-convex, weakly-concave assumption.
(PL) condition, or p- p-weakly convex. Yang et al [51] established the linear
3

---

3 Problem setup and preliminaries
We use lower-case boldface letters such z denote vectors and let || denote the f--nomm of vectors. In Problem I, we refer to a as the primal variable and to y as the dual variable. For function f: IR. x X - R2 we a, and Fx, y) to denote the gradient of f(x, y) with respect to dual variable y. In stochastic setting, we Iz,t to be the unbiased estimator of ( B/)). computed by a minibatch of size Mr and gy to be the unbiased estimator of v (2r, Mt), computed by minibatch of size My. where at and y, are the ith iterates of the algorithms. Particularly, In.t M. Em ( Vr, St,)) and yy m. E Vz, Ve. (,,;), where { Ma and (4% My are i.i.d minibatch samples utilized to compute stochastic gradients at each iteration E (1...., 7}.
V= y) denote the gradient of far y) with respect primal variable
Definition 3.1 (Primal Function). We introduce (xx) maxy (ax y) as the primal function, and defin y (z) arg maxyey fxx y) as the optimal dual variable at point a:
Definition 3.2 (Smoothness). A function (ax y) is E-smooth in both æ. and if it is differentiable, and the following inequalities hold: [[ v1) V.f222 32)1 S e G x211 y 32ll?.
Definition 3.3. A function g is p-stronggy-coonvex if for any æ1, IRd the following holds: g(x2) 2 ( ) + (Vg(z1), Z2 - x1) | - x2||²
Definition 3.4. We say is is e-stationary point for differentiable function & if ( S
We note that e-stationary point common optimality criterion used in the NC-SC setting. As pointed out in [30], considering ((x) as convergence measure is natural since in many application scenarios, mainly care about the value of the objective (ar y) under the maximized y, e.g..
adversarial training or distributionally robust learning.
When f(r y) is merely concave in y, (xx) could be non-differentiable. Hence, following the routine of nonsmooth nonconvex minimization [9], we consider the following Moreau envelope function:
Definition 3.5 (Moreau envelope). A function px(x) is the p-Moreau envelope of a function & if (xx) == eR (ax") + 2p I - ==
We will utilize the following property of the Moreau envelope of a nonsmooth function:
Lemma 3.6 (Davis and Drusvyatskiy [9]). Let a arg migginger eR (<x') + 1 | æ1I² then the 2p
following inequalities hold: | x S pl (x))), min |||||| S V (z))).
Lemma 3.6 suggests that, if we can find a with a small ( then æ is near some point which near-stationary point of क. We will use 1/2ê-Moreau envelope of Φ. following the setting in [30, 45], and establish the convergence rates in terms of ||
We also define two quantities + 11/2e(ar0) - min $ 11/22(2)) (z) and Å = (( - min ((x) that appear in convergence bounds. Before presenting results on EG and OGDA, we briefly revisit the most related algorithm, Gradient Descent Ascent (GDA).
3.1 Gradient Descent Ascent (GDA) algorithm
Algorithm 1 GDA
The GDA method, as detailed in Algorithm 1, per- forms simultaneous gradient descent and ascent updates on primal and dual variables, respectively. This simple algorithm has been deployed exten- sively for minimax optimization applications such as Generative Adversarial Networks (GANs). Un- der Assumptions 4.1, and 4.3, Lin et al. [30] es- tablished the convergence of GDA by choosing
Input: (Io, yo). stepsizes (Ne my)
for 1 1,2,... ................................ do
2; - 21-1 7z V f( (x., 9:-1): y: + Py + Thy vyJ (x. V 1)
end for Randomly choose from ,,.... ØT Output:2
7 = O ) and ny @ In particular, they showed that deterministic GDA requires 0 0(4) calls to a gradient oracle, and stochastic GDA requires O ( calls using the minibatch size of ( to find an e-stationary point of the primal function.

---

3.2 Optimistic Gradient Descent Ascent (OGDA) and Extra-gradient (EG) Method
We now turn to reviewing the algorithms we study this paper: Optimistic GDA (OGDA) and Extra- Extra-gradiens gradient (EG) methods. To optimize Problem (1), at each iteration t = 1....... T, OGDA performs the following updates on the primal and dual variables:
3t++1 2t The V a (z y.) 7h (V fz,, v.) V [ 11
(OGDA)
Ve+1 = Py y Fp V v. ( v.) + Fy Vv De) (.
where correction terms (c.g. v= -.) V y are added to the updates of the GDA.
EG method performs the following updates:
1/+1/2 3t 1/ V
a (
Mt+1/2 = Py (y, + Bly V (ar.
(EG)
D++1 I; 1/2 V v (2 1141/2)
Vi+1 = Py y, + fly V (2 Wx-122)
where the gradient the current point used to find mid-point, and then the gradient at the mid-point is used to find the next iterate. We also consider stochastic variants of the two algorithms where we replace full gradients with unbiased stochastic estimations. The detailed versions of these algorithms are provided in Algorithm 2, and Algorithm 3 in Appendix A.
4
Main Results
We provide upper bounds on the gradient complexity and iteration complexity of OGDA and EG methods for NC-C and NC-SC objectives in both deterministic and stochastic settings. We also show the tightness of obtained bounds for the choice of learning rates made. We will derive general stepsize-independent lower bounds in Section 5.
4.1
Norconver-strongly-oommm minimax problems
We start by establishing the convergence of deterministic OGDA/EG in the NC-SC setting by making the following standard assumption the loss function.
Assumption 4.1. We assume
RT
R" + R. is E-smooth, and f(x..) )) is p-stronglyy-concave.
Moreover, we assume the initial primal optimality gap is bounded. i.e., A = max(#((1)))...m...mm ((xa)) min ((x).
Theorem 4.2. Let v be output of OGDA/EG algorithms and choose The S T3y
S. For OGDA.
let e₁ C2 3. and for EG, let Ci converges to an e-stationary point, i.e., [=) < with iteration number T bounded by:
75: C2 7. Then under Assumption 4.1, OGDA/EG
1:2 kll Do
2²
where Do max (||= 2ol12.112 10 1 1i113
-3012)
To establish the convergence rate in stochastic setting, we will make the following assumption on the stochastic gradient oracle.
Assumption 4.3. Let Vaf(æ, 5") and Vyf(æ, ()) to be the unbiased estimator of the Vef(zz y) and vyf(x, y). respectively. Then, the stochastic gradient oracle satisfies the following:
Unbiasedness:
Ee. [V.f(x y. ( vzf(w, y) and Eşs Wf,f(x, (") V fz ((x..)) y).
Bounded variance: We assume the variance of stochastic gradients o² and
are bounded, i.e. Eş [[]Vz] 5T (z) Eşy [ y. (*)
fa, !) o².
We now turn to establishing the convergence rate stochastic setting.
Theorem 4.4. Let i be output of stochastic OGDA/EG algorithms and let nh and to be chosen as Theorem 4.2. For EG, choose minibulch size M = max {1, " and for OGDA choose
5

---

primal minibatch size M.. m1 } and dual minibatch size My max (1, 60" Then under
2
Assumptions 4.1, and 4.3, OGDA/EG converges e-stationary point, i.e., E ( [2 < e,, with the iteration number T bounded by:
L2 + DO
2
where Do = max (I|z) zoll?, : |1y1
yoll yoll, l1 yr1?, 1lyo ------1 30.
The proofs of Theorems 4.2 and 4.4 are deferred to Appendix A. Our iteration complexity matches with the complexity two-scale GDA obtained in [30]. However, we improve primal gradient oracle complexity for OGDA by factor of к as analysis works for smaller primal batch size M2 compared GDA [30]. This paper establishes primal gradient oracle complexity of O ((5)) ), while the analysis for GDA in [30], requires gradient oracle complexity of O ( for primal variable.
In previous theorems, we established upper bounds on the convergence OGDA and EG algorithms. In the following results, we turn to examining the tightness of obtained rates. To this end, we first consider simple GDA algorithm and will extend the analysis to OGDA/EG. Note that in this section, we only consider the stepsize choice in our upper bound results.
Theorem 4.5 (Tightness GDA). Consider GDA method (Algorithm 1) with step sizes chosen as in Theorem 4.4 in [30], and let a be the returned solution after T iterations. Then, there exists function f(.-)) that E-gradient Lipschitz and j ju-strongly concave in y, and an initialization (æo. yo).
such that Algorithm requires at least T 52
iterations guarantee [[V(())]| S.
Theorem 4.6 (Tightness of EG/OGDA). Consider deterministic EG and OGDA methods with step sizes chosen as in Theorem 4.2 and let ā be the returned solution after T iterations. Then, there exists a function f(.-)) that is E- E-gradient Lipschitz and ju-strongly concave in y, and an initialization (æv. yo).
such that both methods require at least T -=
iterations to guarantee [V S.
The proofs of Theorems 4.5 and 4.6 are deferred to Appendix A.3.1 and A.3.2, respectively. Theo- rems 4.6 show that to achieve stationary point Φ, EG and OGDA need at least 0 gradient evaluations, which match with our upper bound results (Theorems 4.2). These impossibility results demonstrate the tightness of our analysis. It would also be interesting to see such analysis for stochastic setting, which leave valuable future work.
4.2 Nonconvex-concavy minimax problems
now turn to establishing the convergence rate of (stochastic) OGDA/EG in the NC-C setting. We make the following assumption throughout this subsection:
Assumption 4.7. We assume f I x y - IR ê-smooth in æ, y. G-Lipschitz in æ and y is bounded convex set with diameter D, and also f f(x,.) is concave.
From the above assumption, we note when f merely concave in y. we have to assume the dual variable domain is bounded since otherwise, the Moreau envelope function will not be well-defined (This is shown in Lemma 3.6 [30]. Therefore, the update rule for y requires projection as follows:
y: Py (y:-1 My V y f ( 1 (V / (a:,-1, 1) (2)
(OGDA)
Vi+1/2 Py (v. Thy V VyJ
D141 Py (y, my V v (2+1/12 21+1/2)
(EG)
The following theorem establishes the convergence OGDA/EG for NC-C objectives. 2
Theorem 4.8. Let na = O (min foe i D'E }). and Thy 27. By convention, set
1
2-1/2 = Eo. y 1/2 1/2 = Yo- Under Assumption 4.7, OGDA/EG converges to an c-stationary point, iee.
1 +. (2=)))))) < è for OGDA and T+1 ΣT (= S 2 for EG, with
T+1 the gradient complexity bounded by:
1G do
max ( D2A
1, 2 1)

---

Theorem 4.9. Let 1a O (mmin(704407) [ DReGVGG402 D }), and Tly O(minf 1 2²
1E ts }). By convention, we set 1-1/2 Vo, 9-1/2 yo. Under Assump- tions 4.3 and 4.7, stochastic OGDA/EG algorithmss converge an e-stationary point, ie., T+I Ei BlI ( e² for OGDA and E E [ S e² for EG, with the gradient complexity bounded by:
( G& A
+o max (1, 1, - ( =
66
The proofs of Theorems 4.8 and 4.9 are deferred to Appendix B. Here we show that OGDA/EG need most O D gradient evaluations in deterministic setting and pa (D gradient evaluations in stochastic setting to visit c-stationary point.
Our stepsize choices for dual variable match the optimal analysis in convex-concave setting, ( in deterministic setting [40] and O ((2z) in stochastic setting [23], so we suppose our dual stepsize choice optimal. The stepsize ratio te O(c)) in both settings, same as Lin et al. [30]'s results on apply-
fyy
ing GDA to a nonconvex-concav objective, which reveals some connection and similarity between
OGDA and GDA. However, compared to GDA [30], where they get an DDeGG DD
el
D' in stochastic setting, we shave rate in deterministic setting, and O D'
off the significant terms with dependency on Åg. As we will show in the proof, this acceleration is mainly due to the fact that OGDA/EG enjoys an inherent nice descent property on concave function, which is more elaborated in Section 4.3. In the stochastic setting, we observe similar superiority.
Now, we switch to examining the tightness of obtained rates. Similar the NC-SC setting, we first consider a simple GDA algorithm and will extend the analysis to OGDA/EG.
Theorem 4.10 (Tightness of GDA ). Consider GDA that runs T iterations on solving (1), and let FT be the returned solution. Then, there exists a function that is G-Lipschitz in x. L-gradient Lipschitz and concave in y, and an initialization point (æo, Yo) such that GDA requires at least iterations to guarantee (/ (/11)(((TT]] S.
T= 22 @CCDDPd
Theorem 4.11 (Tightness of OGDA/EG). Consider OGDA/EG that runs T iterations on solving (I), and let ør be the returned solution. Then, there exists function f that is G G-Lipschitz in E-gradient Lipschitz and concave in y, and an initialization point (Fo, Mo) such that to achieve
|| 1/20((x) S OGDA/EG requires least = @ (
The proof of Theorems 4.10 and 4.11 deferred to Appendix B.3.1 and B.3.2, respectively. Theorems 4.11 demonstrates that to find an e stationary point of 4 1 OGDA and EG with our stepsize choices need at least O gradient evaluations, which verifies the tightness of upper bound.
4.3 Discussion
Key technical challenges. Here, we present the key technical challenges that arise in the nonconvex setting, which makes the analysis much more involved compared to the previous analysis of these algorithms in convex settings. Our proofs are mainly based on NC-C and NC-SC GDA analysis in [30], and SC-SC OGDA/EG analysis in [39]. In the sonconvex-stronglyyccmm
setting, finding an upper bound for D y - y²xx))l l| is one of the key steps to establish the convergence rate, however bounding this term is much more complicated for OGDA and EG than GDA due to difference
in updating rules. Note that in GDA analysis [30], Σ lly: y (xx))] can be bounded by deriving simple recursive equation for |, - y [12 while extending it to OGDA quite complicated. Hence, we propose to bound r, ||=++1 y² (xx + 1 l1,,, y/-1ll 2| and establish the upper bound on Σ lyy; - y (x:
[x))]] )|² in terms of Σi= r,. In nonconvex-concave setting, we have to bound 1/ - y, v,-1ll 2| so we reduce it the primal function gap: ((x,) - f(x,, y.). To bound this gap, we utilize the benign descent property of OGDA and EG on concave function and shave off significant term Åg which yields a better upper complexity bound than GDA.
7

---

On descent property of concave function for OGDA/EG Take OGDA, for example. The key step in NC-C analysis is to bound (xx,) f(x,, Mr). OGDA proof, we split this into bounding the following:
((x,) - fare, v.) S fa y (a:4) - I (a (xx)) + f( s y (2r,) [m,, y (a: + (m, y (xx)) f( Ur)-
(2)
y.). OGDA can guarantee its convergence without bounded gradient assumption y. However, for GDA, it requires bounded gradient assumption y to show the convergence of this term, and without such assumption, we can only show the convergence of fx y" (x+)) - f( V1+1), Lin et al. [30] split the (xx)) f(z,, y,) as follow:
For the last term f(x, y² (x:)))
((x,) f( v.) S (a (xx)) fa y (xx)) + f Vc+1) f( (.) + f Mr41) - [( Ue+1) + f11,, y² (ar,) fa Me+1)
(3)
Hence they reduce the problem to bounding f(,,, yy (x.)) )) - f( Me41). Therefore, they have to pay the price for the extra term f (I++1, 3e+1) f(z yz)
Generalized OGDA, Generalized OGDA algorithm is variant of OGDA in which different learning rates are used for current gradient (xt, y.), and the correction term (1 y.) - (. The update rule for this algorithm is as follows:
Ft+1 1 = 3t. - 1/,,1 V (ax, y2) - 11 V ( y.) V. (z. 91-))
Ve+1 = Py (v+- + n7.1 v.ff(zz. v.) + 7y.2 vff y:) - () (OGDA+)
Mokhtari et al. [39] introduced this algorithm and established the convergence bound for the bilinear setting while analysis beyond this setting remained as an open problem. In Appendix D, we show that our analysis can be adapted to establish the convergence of the generalized OGDA algorithm. In Section 6, the empirical ad advantage of generalized OGDA over the state of art optimization algorithms shown, and it seems this algorithm is better alternative OGDA in practice. We also define the correction term ratios $1
n..3 B2 Ya.2 and empirically study the effect of these parameters
1s.2
Vs.
on convergence. Note that if [3 B2 1, generalized OGDA would be same OGDA. It would also be an interesting future direction to analyze this algorithm for C-C and SC-SC problems to understand its superior performance better.
Projected OGDA/EG for NC-SC. Here, we highlight that while our analysis for NC-SC assumes that = R", it can be easily extended to a constrained setting, where the dual update is performed under projection onto a convex bounded set V. In the following, we provide a proof sketch for extending our analysis of OGDA to its projected variant, in which we do the same primal update as unconstrained OGDA and a projected (Optimistic gradient) OG update, as defined in [23]. on the dual variable. The main idea behind our dual descent lemma, Lemma A.6, is interpreting OGDA as an extension of the PEG/OG method and then using Theorem 5 of [23] for PEG/OG analysis, which already considers the projected gradient updates. Thus, our Lemma A.6 could be immediately adapted to the projected update. Lemma A.5 can also be extended to projected setting by leveraging Lemma A. in [23]. Combining the projected variant of the mentioned lemmas, the convergence could be easily established for projected OGDA/EG.
5
Lower Bounds
So far, we have established upper bounds and tightness results given specific stepsize choices. In this section, we turn to establishing general iensize-indeeendeett lower bound results in the NC-SC setting.
Theorem 5.1 (Lower complexity bound for GDA). Consider deterministic GDA method (Algorithm 1) with any arbitrary choice of learning rates, and let a be the returned solution. Then, there exists a function f satisfying Assumption 4.1. and initialization (æo. yo), such that Algorithm I requires at least T = Ω iterations to guarantee [Vb( S
Theorem 5.1 implies that GDA algorithm can not find e stationary point of NC-SC problem with less than with (($) many gradient evaluations. This result provides the first known lower bound for the
8

---

GDA algorithm in NC-SC, showing that the rate obtained in [30] for the convergence of GDA is tight up to factor of R. The general proof idea is to consider the following quadratic NC-SC function f IR x IR + R, which is strongly-concat in both a: and y::
f(z, y) == Ex + bry 2/y
By construction, is nonconvex in x (it is actually concave in x) and p-strongly-concave in y- Assume K : f/ 2 4 and choose
v Vu(e + Me) for some 0 Me S f/2 to be chosen later. Then we know b E 8/2, and it is easy to verify that f is smooth. Note that the primal function
& (x) = max f(z, v) _22
is actually strongly convex. This also justifies the symbol for Ma- We use GDA to find the solution for ming max, fx, v). Indeed for this problem, the optimal solution is achieved at the origin. The stepsizes ratio is chosen as r = Us and TJy & for some numerical constants c. Then the GDA update rule can be written as
FR+1 yk-
= (I+ M))
4h y/
M:
rb
-b -ur
(4)
Note that (4) is linear time-invariant system, and due to the simplicity of quadratic form, we are able to track the dynamic of primal and dual variables. By iterating this linear system and analyzing the eigenvalues of the transition matrix, we are able to lower bound the gradient at final iterations.
Now we turn to the extension the lower bound analysis of GDA to OGDA/EG as stated below.
Theorem 5.2 (Lower complexity bound for OGDA/EG). Consider the deterministic OGDA/EG method with any arbitrary choice of learning rates and let æ be the returned solution. Then, there exists a function I satisfying Assumption 4,1, and an initialization (æo; Yo), such that OGDA/EG method requires at least = 92 (RA.) iterations to guarantee [[V[(>)] S.
Theorem 5.2 shows that OGDA/EG methods can not find e-stationary point for any choice of learning rates with less than S2 + gradient evaluations. Given the upper bounds we derived for deterministic OGDA/EG in section 4.1, our result indicates that our upper bounds is tight up to factor of however, we highlight that according to Theorem 4.6, given our choice of the learning rate, our upper bound is exactly tight. The complete proof of Theorems 5.1 and 5.2 are deferred to Appendix C.
6 Experiments
In this section, we empirically evaluate the performance of the OGDA algorithm. In particular, we follow [52] and optimize Wasserstein GAN (WGAN) synthetic dataset generated from Gaussian distribution. We mainly follow the setting of [52, 34] to conduct our experiment. We consider optimizing the following WGAN loss, where the generator approximates one-dimensional Gaussian distribution:
min max We WD
Ez (Dup (x)] - Ez (G. - A \|
(5)
Where WG and wD correspond to generator and discriminator parameters, respectively. We define discriminator to be Dxx) Φιz + Ø2x², and generator to be neural network with one hidden layer with 5 neurons with ReLU activation function, same as the setup considered in [52]. We assume that real data comes from Gaussian N(u,, o²) distribution, and the generator tries to approximate and o² using a neural network. We set μ = 0, and a = 0.1. is the regularization parameter which set 0.001. Note that À makes the function parameters, so the problem becomes NC-SC/NC-C.
in terms of discriminator
Performance of fine-tuned stochastic OGDA is depicted in Figure la, in comparison to ADAM [25]. RMSprop, SGDA [30], SAGDA [52], and Smooth SAath--SAAAAAAA [52], which are well-known minimax optimization methods. Our evaluation shows that OGDA outperforms all of these methods, supporting the empirical advantage of OGDA as seen relevant studies [28, 8J. While our theoretical results show that OGDA/EG might not outperform GDA in terms of convergence rate, comparing the empirical result suggests that OGDA might converge faster. Figure Ic, the evolution of the Wasserstein distance metric during the training has been shown. While GDA and OGDA

---

yD
Wasserstein distance loss
10
10
10"
0006 6-6L.
10
10
OGDA PL 8
10-
OODA F--
10-3
10"4
10 0
10-
10"
30
10-1
10-5
10
10"4
5000
15000
35000
herations
Perations
Ieeations
(a)
(b)
(c)
Figure I: Figure la demonstrates the best performance of different algorithms on optimizing NC-SC
objective in WGAN, where V|V y)1l V=. y)1 + || g (x. y) 1| For GDA, and OGDA, Ne. and fyy chosen from the set {5e 5, le 4,5e 1e 3, 5e 3. le 2, 5e - 2) using grid search. For OGDA, we choose correction term ratios from the set {0, 0.01, 0.1, 0.5, 1}. The optimal learning rates are follows. For both OGDA, and GDA, we 1/2- 7y 0.05, and for OGDA 31 = 32 = 0.01. For other algorithms, we used the same hyperparameters as reported in [52], using the same random seed. Figure 1b indicates effect of tuning correction term ratio / on the performance of generalized OGDA algorithm. Figure Ic indicates the evaluation of the Wasserstein distance metric during the training for the best hyperparameter configuration.
0.5.1).
stabilized faster than other algorithms, it seems that they converge suboptimal solution, which incurs a higher Wasserstein distance. Thus, our study suggests that comparing different minimax algorithms only based on the convergence of gradient norm may not be that insightful in practice, as they might converge to suboptimal equilibrium. This observation naturally leads to an interesting future direction to theoretically understand how different notions of equilibrium in first-order minimax optimization algorithms are related to the realistic performance of practical methods such as GANs or WGANs.
The common version of OGDA, as depicted in Algorithm 2 in Appendix A, uses the same learning rate for the current gradient and correction term (difference between gradient). Empirically, we observed that using different learning rates for those terms (which we call generalized OGDA) makes the convergence faster and more stable. Hence in the following, we investigate the effect of using different correction term ratios in OGDA, which we refer them as 31 and 32 as defined in Subsection 4.3. The results in Figure 1 demonstrate that small values of these parameters benefit the convergence rate, and larger values degrade the performance. We further observe that using correction term ratios larger than 0.5 makes the algorithm diverge and become unstable. Hence, this corroborates the practical importance of the generalized OGDA algorithm compared to OGDA, as we are restricted to choosing the same learning rate OGDA (i.e., 31 = fB =1)
Conclusion
In this paper, we established the convergence of Optimistic Gradient Descent Ascent (OGDA) and Extra-gradiem (EG) methods in solving nonconvex minimax optimization problems. We demonstrated that both methods exhibit the same convergence rate that is achievable by GDA in both stochastic and deterministic settings. We also derived matching lower bounds for the choice of parameters that indicate the tightness of obtained rates. Further, we established general lower bounds (i.e, learning rate-independent) for GDA/EG/OGDA in the NC-SC setting, indicating the optimality of obtained upper bounds up to the factor of K. remains an interesting future work to extend the lower bound results to the stochastic setting and also derive the general lower bound for GDA/EG/OGDA in the NC-C setting. Moreover, there is a gap by a factor of λ between our lower and upper bounds for NC-SC problems, which would also be an interesting future work to close this gap.
Acknowledgements
This work was supported part by NSF grant CNS 1956276. We also would like to thank Mohammad Mahdi Kamani for his help on conducting the experiments.
10

---

References
[1] J. Abernethy, K. A. Lai, and A. Wibisono. Last-iterate convergence rates for min-max optimiza- tion: Convergence of hamiltonian gradient descent and consensus optimization. In Algorithmic Learning Theory, pages 3-47. PMLR, 2021.
[2] T. Basar and G. Olsder. Dynamic noncooperative game theory, vol. 23 (siam, philadelphia). 1999.
[3] Y. Cai, A. Oikonomou, and W. Zheng. Tight last-iterate convergence of the extragradient and the optimistic gradient descent-ascent algorithm for constrained monotone variational inequalities. arXiv preprint orXhv:2204.09223 2022.
[4] T. Chen, Y. Sun, and W. Yin. Tighter analysis of alternating stochastic gradient method for stochastic nested problems. arXiv preprint urXiv:2106. 13781, 2021.
[5] Y. Chen, G. Lan, and Ouyang. Accelerated schemes for a class of variational inequalities. Mathematical Programming, 165(1):113-149, 2017.
[6] Z. Chen, S. Ma, and Y. Zhou. Accelerated proximal alternating gradient-desse pesscent-ascent for nonconvex minimax machine learning. arXiv preprint arXiv:2112./16633 2021.
[7] C. D. Dang and G. Lan. On the convergence properties of non-euclidean extragradient methods for variational inequalities with generalized monotone operators. Computational Optimization and applications. 60(2):277-310, 2015.
[8] C. Daskalakis, A. Ilyas, V. Syrgkanis, and H. Zeng. Training gans with optimism. arXiv preprint 1711v:1711.000 1,00141, 2017.
[9] D. Davis and D. Drusvyatskiy. Stochastic -base- minimization of weakly convex functions. SIAM Journal on Optimization, 19(1):2772239 2019.
[10] J. Diakonikolas, C. Daskalakis, and M. Jordan. Efficient methods for structured nonconvex- nonconcave min-max optimization. In International Conference on Artificial Intelligence and Statistics, pages 2746-2754. PMLR, 2021.
[11] Y. Drori and O. Shamir. The complexity of finding stationary points with stochastic gradient descent. In International Conference on Machine Learning, pages 2658-2667. PMLR, 2020.
[12] A. Fallah, Ozdaglar, and S. Pattathil. An optimal multistage stochastic gradient method for minimax problems. In 2020 59th IEEE Conference Decision and Control (CDC), pages 3573-3579. IEEE, 2020.
[13] G. Gidel, H. Berard, Vignoud, P. Vincent, and S. Lacoste-Julien. variational inequality perspective generative adversarial networks. arXiv preprint arXiv: 1802. 2018.
[14] I. Goodfellow, J. Pbbdee-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Bengio. Generative adversarial nets. Advances neural information processing systems, 27, 2014.
[15] I. J. Goodfellow, J. Shlens, and C. Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv: 1412.6441..66555 2.672, 2014.
[16] E. Gorbunov, H. Berard, G. Gidel, and N. Loizou. Stochastic extragradient: General analysis and improved rates. In International Conference on Artificial Intelligence and Statistics, pages 7865-7901. PMLR, 2022.
[17] E. Gorbunov, N. Loizou, and G. Gidel. Extragradient method: O (1/k) last-iterate convergence for monotone variational inequalities and connections with cocoercivity. In International Conference Artificial Intelligence and Statistics, pages 366-402. PMLR, 2022.
[18) Z. Guo, Z. Yuan, Y. Yan, and T. Yang. Fast objective & duality gap convergence for nonconvex- strongly-concave min-max problems. arXiv preprint 2rXiv:2006.06889/ 2020.
[19] S. Hajizadeh, H. Lu, and B. Grimmer. On the linear convergence of extra -atriengradeett methods for minimax problems. arXiv preprint arXiv:2201.06166 2022.
nonconvex-nonconee
[20] Y. Han, G. Xie, and Z. Zhang. Lower complexity bounds of finite-sum optimization problems: The results and construction. ArXiv, 0bb/22033.02288 2021.
[21] M. Hast, K. J. Åström, B. Bernhardsson, and Boyd. Pid design by convex-concave optimiza- tion. In 2013 European Control Conference (ECC), pages 4460-4466 IEEE, 2013.
11

---

[22] C. H. Hommes and M. I. Ochea. Multiple equilibria and limit cycles in evolutionary games with logit dynamics. Games and Economic Behavior, 74(1):434 441, 2012.
[23] Y.-G. Hsieh, F. lutzeler, J. Malick, and P. Mertikopoulos. On the convergence of single-call stochastic extra-gradient methods. arXiv preprint arxiv::908:0800
2019.
[24] C. Jin, P. Netrapalli, and M. I. Jordan. What is local optimality in nonconvex-nonconean minimax optimization? arXiv preprint arXiv: 1902.00618, 2019.
[25] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint
arXiv:1412.6988 2014.
[26] W. Kong and R. D. Monteiro. An accelerated inexact proximal point method for solving nonconvex-coneave min-max problems. arXiv preprint 1xXiv:/905.13444 2019.
[27] H. Li, Y. Tian, J. Zhang, and A. Jadbabaie. Complexity lower bounds for nonconvex-strongly concave min-max optimization. ArXiv, abs/2104.0870$. 2021.
[28] T. Liang and J. Stokes. Interaction matters: A note non-asymptotic local convergence of generative adversarial networks. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 907-915. PMLR, 2019.
[29] T. Lin, C. Jin, and M. I. Jordan. On gradient descent ascent for nonconvex-coneavy minimax problems. arXiv preprint arXiv: 1906. 00331, 2019.
[30] T. Lin, C. Jin, and M. Jordan. On gradient descent ascent for nonconvex-concave minimax problems. International Conference on Machine Learning, pages 6083-6093. PMLR, 2020.
[31] T. Lin, C. Jin, and M. I. Jordan. Near-optimal algorithms for minimax optimization. In Conference on Learning Theory, pages 2738-2779. PMLR, 2020.
[32] M. Liu, Y. Mroueh, J. Ross. W. Zhang, X. Cui, P. Das, and Yang. Towards better understanding of udaptive gradient algorithms in generative adversarial nets. arXiv preprint arXiv:1912.11940 2019.
[33] M. Liu, Y. Mrouch, W. Zhang, X. Cui, T. Yang. and P. Das. Decentralized parallel algorithm for training generative adversarial nets. arXiv preprint arXiv: 12999, 2019.
[34] N. Loizou, H. Berard, A. Jolicneur-Martineau, P. Vincent, S. Lacoste-Julien, and I. Mitliagkas. Stochastic hamiltonian gradient methods for smooth games. In International Conference on Machine Learning, pages 6370-6381. PMLR, 2020.
[35] N. Loizou, H. Berard, G. Gidel, I. Mitliagkas, and S. Lacoste-Julien. Stochastic gradient descent-ascent and consensus optimization for smooth games: Convergence analysis under expected co-coercivity. Advances in Neural Information Processing Systems, 34:19095-19108,
2021.
[36] S. Lu, I. Tsaknakis, M. Hong, and Y. Chen. Hybrid block successive approximation for one- sided non-convex min-max problems: algorithms and applications. IEEE Transactions on Signal Processing, 68:3676-3691, 2020.
[37] A. Madry, A. Makelov, I. Schmidt, D. Tsipras, and A. Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv: ToxXi::/00..000 176.70006..333 2017.
[38] P. Mertikopoulos, B. Lecouat, H. Zenati, C.-S. Foo, V. Chandrasekhar, and G. Piliouras. Opti- mistic mirror descent in saddle-pone point problems: Going the extra (gradient) mile. arXiv preprint 1877v21807.00622, 2018.
[39] A. Mokhtari, A. Ozdaglar, and S. Pattathil. A unified analysis of extra- astra-gradient and opti- mistic gradient methods for saddle point problems: Proximal point approach. In International Conference on Artificial Intelligence and Statistics, pages 1497-1507. PMLR, 2020.
[40] A. Mokhtari, A. E. Ozdaglar, and Pattathil. Convergence rate of o( ((1kk) for optimistic gradient and extragradient methods in smooth convex-concave saddle point problems. SIAM Journal on Optimization, 30(4):323-33255 2020.
[41] M. Nouiehed, M. Sanjabi, T. Huang, J. D. Lee, and M. Razaviyayn. Solving class of non- convex min-max games using iterative first order methods. arXiv preprint 1902.00290 2019.
[42] T. Pethick, P. Patrinos, O. Fercoq, V. Cevherå, et al. Escaping limit cycles: Global convergence for constrained Learning Representations, 2022.
minimax problems. In International Conference on
12

---

[43] S. Qiu, Z. Yang, X. Wei, J. Ye, and Z. Wang.
stochastic canconvex-coneanee optimization for smooth nonlinear learning. arXiv preprint arXiv:2008. 10103, 2020.
[44] H. Rafique. M. Liu, Q. Lin, and T. Yang. Non-convex min-max optimization: Provable algorithms and applications in machine learning. arXiv preprint arXiv: /8/00020600 2018.
[45] H. Rafique, M. Liu, Q. Lin, and Yang. Weakly-conver conell--ccoxxxccccaa... concave man--mx optimization: provable algorithms and applications in machine learning. Optimization Methods and Software, pages 1-35, 2021.
[46] A. Sinha, Namkoong, R. Volpi, and J. Duchi, Certifying some distributional robustness with principled adversarial training. arXiv preprint arXiv: 1710:1171000 2017.
[47] C. Song, Z. Zhou, Y. Zhou, Y. Jiang, and Y. Ma. Optimistic dual extrapolation for coherent non-monotone variational inequalities. arXiv preprint arXiv:2103. 04..00 2021.
[48] K. K. Thekumparampil, P. Jain, P. Netrapalli, and S. Oh. Efficient algorithms for smooth minimax optimization. In Advances in Neural Information Processing Systems, pages 12659-
12670, 2019.
[49] P. Tseng. On linear convergence of iterative methods for the variational inequality problem. Journal of Computational and Applied Mathematics, 60(1-2):237-253 1995.
[50] Von Neumann and O. Morgenstem. Theory of games and economic behavior. Princeton university press, 2007.
[51] Yang, N. Kiyavash, and N. He. Global convergence and variance-reduced optimization for a class of nonconvex-nomconenn minimax problems. arXiv preprint arXiv:2002. 09621, 2020.
[52] J. Yang, A. Orvieto, A. Lucchi, and N. He. Faster single-loop algorithms for minimax optimiza- tion without strong concavity. arXiv preprint 1
2021.
[53] Zhang, Xiao, Sun, and Luo. single-lopp smoothed gradient descent- algorithm for nonconvex-concan min-max problems. Advances in Neural Information Processing Systems, 33:7377-7389, 2020.
[54] K. Zhang, Z. Yang, and T. Başar. Multi-agent reinforcement learning: A selective overview of theories and algorithms. Handbook of Reinforcement Learning and Control, pages 321-384, 2021.
[55] S. Zhang, J. Yang, C. Guzmán, N. Kiyavash, and N. He. The complexity of nonconvex-strongly- concave minimax optimization. arXiv preprint arXiv:2103.15888, 2021.
13


Reviews:
[{"review_id": "l7wol_e9s9b", "reviewer": "Reviewer_8N1p", "summary": "The paper, proposes a convergence analysis of Optimistic gradient descent ascent (OGDA) and Extragradient (EG) methods for two classes of min-max optimization problems the nonconvex-strongly-concave (NC-SC) and nonconvex-concave (NC-C). The authors also provide lower bounds showing the tightness of their convergence guarantees. ", "questions": "Below i include some questions related to the weaknesses of the paper: \n\n1. if I did not miss its definition, i believe $\\bar{x}$ is only defined in Algorithm 1 (GDA). However the authors provide $\\bar{x}$ as the output of OGDA and EG as well. Can the authors elaborate more on this? Why one need to have $\\bar{x}$ as the output of OGDA and EG and not just simply use the last iterate of these methods (note that in the Algorithms 2 and 3 no output is given)?\n\n2. In section 4.2 what is the point of mentioning the following sentence?\n\"From the above assumption we note when f is merely concave in y, we have to assume the domain for dual variable is bounded, and hence it requires projection in the updating rule of y:\" The two algorithmic updates are exactly the same with the update rules first introduced in section 3.1, right?\n\n3. In the current problem (1), it is assumed that there is unbounded x, and y \\in Y. How the paper's theoretical results would be affected in the scenario that y \\in R^d ( no constraints) or in the scenario that x \\in X where X is bounded convex set (classical constrained min-max optimization problem). In other words, is there any particular reason (in terms of proof techniques) that the specific setting is considered?\n\n4. On Experiments: \nIn the experiments you use the setting from [29] but you did not compare against the algorithms from [29]. Was there any particular reason for this? In addition i believe that parameter alpha and beta were never properly defined in the main paper but they are referenced extensively in the experiments related to generalized OGDA?", "limitations": "Yes limitations have been addressed. ", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "\nStrengths:\n\nThe paper is well written and the main contributions of the work are well presented. I went through the main proofs and the important steps looks also correct. I really like the idea of including Proof sketches, before the actual proofs in the appendix. This add pedagogical value to the presentation and makes it easier to follow.\n\nIn terms of presentation I also like Table 1 with the summary of main theoretical contributions and comparison of the proposed results with previous papers. In this table  would suggest to include also a line with the lower bounds. This would highlight better the tightness of your results as one will be able to see early in the paper the two main contributions of the work (convergence rates and lower bounds), in the a single table. \n\nWeaknesses:\n\nIn general i like this paper and i believe its contributions could be valuable to the ML community. To the best of my knowledge this is the first paper that analyze OGDA and EG for NC-SC and NC-C min-max problems. However i notice some issues/weaknesses that will require further clarification. \n\nThere are inconsistencies between the main paper and the appendix, in terms of notation and presentation of the results. These inconsistencies raised some questions the robustness of the theoretical results. Let me add a few details below:\n\n1. In the statement of the theorems in the main paper the quantity $\\Phi(x_1) -min_x \\Phi(x)$ is defined as $\\Delta_\\Phi$ while in the appendix the same quantity is $\\delta$. I would suggest the authors do a thorough pass and fix these issues. Statements of Theorems and their proofs need to use the same notation. \n\n2. The main theoretical results in the main theorems are presented in terms of gradient complexity, however the proofs of main Theorems  stop in equations (49) and (75). The authors should explain (even if this might be trivial) in more detail and provide the exact derivation of how one can obtain the main Theorem (the gradient complexity) from the end of their proofs.\n\n3. As we can see for the proof to work the algorithms should be run with specific step-size selection. In the current statement of the Theorems the exact values of the step-size for OGDA and EG are hidden in the big $\\Theta$ expression. The authors should add the exact step-size values that used in the proofs (at the moment the Theorems are more abstract than what should be). \n\n4. The step-sizes of the main theorems in the section 4.2 (Nonconvex-concave minimax problems) are not part of the statement of the theorems. This should be added in the updated version. \n\n5. More clarifications are needed for some parts. I include a few questions about these points below. \n\n6. Minor comments:\n\n(a)  Definition 3.3. and stochastic definition of in line 120 use both \"g\". This might be confusing for some readers.\n(b) Theorem 4.5 is about GDA but in the statement the author refer to Theorem 4.2which is a bit confusing.\n(c) The title could be updated to include \"nonconvex-concave\" instead of simple nonconvex as it is not. At the moment the reader expects that the methods proposed solve nonconvex nonconave problem (potentially with extra structured).\n\n6.There are several recent closely related missing references (could be added in corresponding paragraphs of Section 2):\n\n[1] Abernethy, J., Lai, K. A., Wibisono, A.. Last-iterate convergence rates for min-max optimization: Convergence of hamiltonian gradient descent and consensus optimization. In Algorithmic Learning Theory (pp. 3-47). PMLR \n\n[2] Loizou, N., Berard, H., Jolicoeur-Martineau, A., Vincent, P., Lacoste-Julien, S., Mitliagkas, I. . Stochastic hamiltonian gradient methods for smooth games. In International Conference on Machine Learning (pp. 6370-6381). PMLR. \n\n[3] Gorbunov, E., Berard, H., Gidel, G., Loizou, N. . Stochastic extragradient: General analysis and improved rates. In International Conference on Artificial Intelligence and Statistics (pp. 7865-7901). PMLR.\n\n[4] Gorbunov, E., Loizou, N., Gidel, G.. Extragradient method: O (1/K) last-iterate convergence for monotone variational inequalities and connections with cocoercivity. In International Conference on Artificial Intelligence and Statistics (pp. 366-402). PMLR.\n\n[5] Hsieh, Y. G., Iutzeler, F., Malick, J., Mertikopoulos, P. . Explore aggressively, update conservatively: Stochastic extragradient methods with variable stepsize scaling. Advances in Neural Information Processing Systems, 33, 16223-16234.\n\n[6] Cai, Y., Oikonomou, A.,  Zheng, W.. Tight Last-Iterate Convergence of the Extragradient and the Optimistic Gradient Descent-Ascent Algorithm for Constrained Monotone Variational Inequalities. arXiv preprint arXiv:2204.09228.\n\n[7] Loizou, N., Berard, H., Gidel, G., Mitliagkas, I.,  Lacoste-Julien, S.. Stochastic gradient descent-ascent and consensus optimization for smooth games: Convergence analysis under expected co-coercivity. Advances in Neural Information Processing Systems, 34, 19095-19108.\n\n[8] Beznosikov, A., Gorbunov, E., Berard, H.,  Loizou, N. . Stochastic gradient descent-ascent: Unified theory and new efficient methods. arXiv preprint arXiv:2202.07262.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "cXcaaJpZqnj", "reviewer": "Reviewer_8dwY", "summary": "This paper analyzes the convergence rates of the two-time-scale versions of extra-gradient (EG) and optimistic gradient (OGDA) methods\nfor nonconvex-strongly-concave (NC-SC) and nonconvex-concave (NC-C) settings. Similar rate analysis has been already studied for a two-time-scale variant of GDA in [Lin et al., 2020], and this paper's rates of two-time-scale EG and OGDA methods are found to be (almost) equivalent to those of GDA. This equivalence is new and interesting, but might look disappointing in some sense, especially considering that OGDA performs well in practice as illustrated in Section 6. The authors further provide lower bounds of GDA/OGDA/EG, and confirm that [Lin et al., 2020] and this paper's rates are tight.\n\nA simple WGAN experiment illustrate that OGDA might outperform existing algorithms in some practical NC-(S)C problems. Further empirical investigation on different stepsizes for the current gradient and correction term of OGDA suggests that certain different stepsizes might empirically further accelerate OGDA.", "questions": "- Table 1: ALSET [2] and [Nouiehed et al., 34] are basically similar, so I think [34] should also be included, especially considering that [34] also has a result on the NC-C setting. Let me know if I am missing something.\n- Line 39: Among [32,9,33], [9] is the one that discusses the stochastic setting, and it seems to only consider the SC-SC case. I suggest adding another reference on the stochastic C-C case, if there is any.\n- Line 49: The rates in the paper are in terms of the randomly chosen point $\\bar{x}$ (or the best iterate for the NC-C setting), but here it is called the ergodic average.\n- Contributions: I believe that the contributions part can be parsed so that each contribution is more clearly given to the readers.\n- Line 76: the the\n- Line 86: preforming\n- Line 97: The nonconvex-\"non\"concave setting does not seem to be discussed here.\n- Line 188-192: Could you explain this in more details? I was also not able to find its related part in [23].\n- Line 235: I am not following why OGDA/EG has an inherent nice descent property on concave function, which GDA does not have. Could you write down such property explicitly? (If it is in the appendix, let me know where it is.) Why don't we see such difference on the descent property (and its resulting rates) in NC-SC setting? In my viewpoint, the $\\hat{\\Delta}_0$ terms are not the \"significant\" ones, compared to the other.\n- Line 277: leaning rate (this also appears in other places)\n- Figure 1: What is the step sizes for OGDA? Do they match the theoretically chosen step size in theorems?\n- Line 340: generlaztion", "limitations": "- In conclusion, the authors state that the general lower bound for OGDA in NC-C setting and stochastic setting is missing. Also, the authors point out that there is a gap between the lower and upper bounds for NC-SC problems. I think the authors reasonably addressed this work's limitations, including the rate equivalence between GDA and OGDA/EG.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "- S1: It is the first rate analysis of EG and OGDA for NC-SC and NC-C settings.\n- S2: Both stepsize-dependent and stepsize-independent lower bounds of GDA/EG/OGDA are new, and provide how much tight the derived upper bounds are.\n\n- W1: Theorems state that OGDA/EG converge to an $\\epsilon$-stationary point, but they only guarantee finding it. This is not a weakness, and it just needs correction. A minor weakness of this paper's analysis of OGDA/EG that I want to point out is that they do not guarantee finding an \"exact\" solution as we further iterate, similar to GDA [23].\n- W2: Theoretically, this work only suggests that OGDA/EG is as much good as GDA.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "aTy7CiH9ZQ8", "reviewer": "Reviewer_C83W", "summary": "This paper shows the convergence of Extragradients methods in the context of nonconvex-(strongly)concave minimax optimization. \n", "questions": "- In your experiments: Can you report the performance of each methods (in terms of Wasserstein distance between the generated and the data distribution)?\n- It seems to me that reporting the actual norm of the gradient (i.e the sum of the norm of the gradients of the discriminator and the generator) would be a more relevant metric of convergence than both norms independently. \n", "limitations": "The authors mention the slight gap between the lower and the upper bound for NS-SC problems and the missing lower bound for OGDA", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strenghts:\n- The authors propose tight upper and lower bounds for their analysis \n- They study the Extragradient and the Optimistic method. \n- They propose some non-trivial experiments (e.g. GANs on a toyish dataset)  to compare different optimization methods. The experiment is non-trivial because the architectures considered are non-convex. \n\nweaknesses: \n- The lower bounds shown in the paper only regard the methods considered and not a class of methods (e.g. first-order methods or p-SCLI methods) \n- Minor issues: There are some formatting issues (page 2). The page number is overlapping the text\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "KquMzAGQL2t", "reviewer": "Reviewer_yYTE", "summary": "This paper provides theoretical analysis of optimistic gradient descent ascent (OGDA) and extragradient (EG) on the convergence and its tightness. For both OGDA and EG, two families of problems are considered, i.e., nonconvex-strongly-concave (NC-SC) and nonconvex-concave (NC-C), and two settings are analyzed, i.e., deterministic and stochastic (minibatch), respectively, leading to totally four specific conditions. \n\nThe convergence analysis of both algorithms looks under a unified framework. For example, Lemma A.4 and Lemma A.8 create similar telescoping summation to start analysis of OGDA and EG under NC-SC, respectively. Similar situations can be seen in Lemma B.2, Lemma B.8, Lemma B.15 and Lemma B.20, where Moreau Envelope is first used to start for deterministic OGDA, stochastic OGDA, deterministic EG and Stochastic EG, respectively.\n\nCompared to previous convergence rate of GDA in the same four conditions, this paper achieve the same results of OGDA/EG in terms of gradient complexity (Table 1). In addition, tightness of these convergence results is analyzed by comparing with the lower bounds deriving from the designed hard function instances, showing that convergence results are tight.\n", "questions": "If I did not miss anything, there may be something inconsistent (and minor if it is) when using Moreau Envelope for NC-C analysis. In Lemma B.2, after the final inequality, the proof says that one can use $\\|| \\hat x_{t-1} - x_{t-1} \\|^2 = 1 / (2\\ell) \\||\\nabla \\Phi_{1/(2\\ell)} (x_{t-1}) \\||^2$. Given the definition of $\\hat x_{t-1}$ in the beginning of Proof for Lemma B.2, I think one has $\\hat x_{t-1} – x_{t-1}=(1/2\\ell) (\\nabla \\Phi_{1/(2\\ell)} (x_{t-1}))$ (this result can also be found in [7], the equation right above Figure 1 therein), so $\\|| \\hat x_{t-1} – x_{t-1} \\||^2=(1/4\\ell^2) \\|| \\nabla \\Phi_{1/(2\\ell)}(x_{t-1}) \\||^2$, or $\\|| \\hat x_{t-1} – x_{t-1} \\||=(1/2\\ell) \\|| \\nabla \\Phi_{1/(2\\ell)}(x_{t-1}) \\||$. In fact, the latter equation is shown in Proof of Lemma B.20, Line 982.\n\nSimilar concern occurs in Lemma B.8, Lemma B.15 and Lemma B.20. If the above dependency of $\\ell$ changes, then it may change a little bit on the lower bound for $\\eta_x$ in Theorem B.6 (NC-C, deterministic OGDA), Theorem B.12 (NC-C, stochastic OGDA), Theorem B.18 (NC-C, deterministic EG) and Theorem B.23 (NC-C, stochastic EG).\n\n\n", "limitations": "I think the authors adequately addressed the limitations and potential negative societal impact of their work", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Originality: \nThis paper provides convergence analysis of OGDA and EG for nonconvex problems, i.e., NC-SC and NC-C, matching that of GDA in previous studies, which offers new understanding to these algorithms. Tightness analysis also gives further understanding to convergence results. \n\nSpecifically, to deal with the gap between OGDA/EG and existing analysis such as [24] of GDA for NC-SC and NC-C, as well as [33] of OGDA/EG for SC-SC, this paper constructs several new bounds to derive the convergence result. For example, since the update is different from GDA, the bound of $\\||y_i-y^*(x_i)\\| |^2$ developed in [24] is not available. Instead, this paper crafts new upper bound for term $r_t$, which involves $\\||z_{t+1}-y^*(x_t)\\| |^2$ and $\\||y_t-y_{t-1}\\||^2$. These ideas are particularly discussed in Section 4.3 conceptually. Proof part is complete and self-contained.\n\nQuality: \nThis paper is technically sound. Proofs look almost (please see ‘Questions’ part for my concern) correct to me.\n\n\nClarity:\nThis paper is clear and easy to follow. However, it looks a little bit long (65 pages totally) and some results may be further merged/organized, so that it could be somehow explicitly show the unified theoretical framework of both algorithms.\n\n\nSignificance:\nThis paper provides significant insights to OGDA and EG algorithms for nonconvex (strongly) concave min-max problems.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept