Faster Stochastic Algorithms for Minimax Optimization under Polyak-Łojasiewicz Conditions
Anonymous Author(s)
AffiliationAddress email
Abstract
This paper considers stochastic first-order algorithms for minimax optimizationunder Polyak-Łojasiewicz (PL) conditions.
We propose SPIDER-GDA for solvingthe finite-sum problem of the form min x max y f
( x, y ) ≜ (cid:80) ni =
f i ( x, y ) , wherethe objective function f ( x, y ) is µ x -PL in x and µ y -PL in y ; and each f i ( x, y )is L -smooth.
We prove SPIDER-GDA could find an ϵ -approximate solutionwithin O (cid:0) ( n + √ n κ x κ 2 y ) log(1 /ϵ ) (cid:1) stochastic first-order oracle (SFO) complexity, which is better than the state-of-the-art method whose SFO upper bound is O (cid:0) ( n + n 2 / 3 κ x κ 2 y ) log(1 /ϵ ) (cid:1) , where κ x ≜ L/µ x and κ y ≜ L/µ y .
For the illconditioned case, we provide an accelerated algorithm to reduce the computationalcost further.
It achieves ˜ O (cid:0) ( n + √ n κ x κ y ) log(1 /ϵ ) (cid:1) SFO upper bound whenκ x ≥ √ n .
Our ideas also can be applied to the more general setting that the objective function only satisfies PL condition for one variable.
Numerical experimentsvalidate the superiority of proposed methods.
Introduction
This paper focuses on smooth minimax optimization problem of the form
[Equation]
which covers a lot of important applications in machine learning such as reinforcement learning [10,39], AUC maximization [13, 22, 45], imitation learning [5, 29], robust optimization [11], causalinference [26], game theory [6, 27] and so on.
We are interested in the minimax problems under PL conditions [9, 29, 42].
The PL condition [32]was originally proposed to relax the strong convexity in minimization problem that is sufficient forachieving the global linear convergence rate for first-order methods.
In machine learning community,it has been successfully used to analyze the convergence behavior for overparameterized neuralnetworks [21], robust phase retrieval [37] and a plenty of fundamental models [17].
There are manypopular minimax formulations only satisfy PL condition, but lack strong convexity (or strong concavity).
The examples include PL-game [29], robust least square [42], deep AUC maximization [22] andgenerative adversarial imitation learning of LQR [5, 29].
et al.
[42] showed that the alternating gradient descent ascent (AGDA) algorithm linearly converges to the saddle point when the objective function satisfies two-sided PL condition
They also
[Table] [Table]
[Table]
Theorem C.
[Table]
Theorem 4.
[Table]
Corollary 5.
proposed the SVRG-AGDA method 1 for the finite-sum problem (1), which could find ϵ -approximatesolution within O (cid:0) ( n + n 2 / 3 κ x κ 2 y ) log(1 /ϵ ) (cid:1) stochastic first-order oracle (SFO) calls, 2 where κ xand κ y are the condition numbers with respect to PL condition for x and y respectively.
The variancereduced technique in the SVRG-AGDA leads to better a convergence rate than full batch AGDAwhose SFO complexity is O (cid:0) nκ x κ 2 y log(1 /ϵ ) (cid:1) .
However, there are still some open questions left.
Firstly, Yang et al.
[42]’s theoretical analysis heavily relies on the alternating update rules.
It remainsinteresting whether a simultaneous version of GDA (or its stochastic variants) also has similarconvergence results.
Secondly, it is unclear whether the SFO upper bound obtain by SVRG-AGDAcan be improved by designing more efficient algorithms.
For one-sided PL condition, we desire to find the stationary point of g ( x ) = max y ∈ R dy f
( x, y ) ,since the saddle point may not exist.
Nouiehed et al.
[29] proposed the multi-step GDA methodthat achieves the ϵ -stationary point within ˜ O
( κ 2 y Lϵ − 2 ) numbers of full gradient iterations.
Thesimilar complexity also can be obtained by AGDA [42].
Recently, Yang et al.
[44] proposed thesmoothed-AGDA that improves the upper bound into O ( κ y Lϵ − 2 ) .
Both multi-step GDA Nouiehedet al.
[29] and smoothed-AGDA [43] can be extended to online setting, but the formulation (1) withfinite-sum structure has not been explored.
In this paper, we introduce a variance reduced first-order method, called SPIDER-GDA, whichconstructs the gradient estimator by stochastic recursive gradient and the iterations are based onsimultaneous gradient descent ascent.
We prove that SPIDER-GDA could achieve ϵ -approximatesolution of the two-sided PL problem of the form (1) within O (cid:0) ( n + √ n κ x κ 2 y ) log(1 /ϵ ) (cid:1) SFOcalls, which has better dependency on n than SVRG-AGDA [42].
We also provide an accelerationframework to improve first-order methods for solving ill-conditioned minimax problems under PL conditions.
The accelerated SPIDER-GDA (AccSPIDER-GDA) could achieve ϵ -approximatesolution within ˜ O
(cid:0) ( n + √ n κ x κ y ) log(1 /ϵ ) (cid:1) SFO calls when κ x ≥ √ n , which is the best known SFO upper bound for this problem.
We summarize our main results and compare them with relatedwork in Table 1.
Without loss of generality, we always suppose κ x ≥ κ y .
Furthermore, the proposedalgorithms also work for minimax problem with one-sided PL condition.
We present the results forthis case in Table 2.
Related Work
The minimax optimization problem (1) can be viewed as the following minimization problem
[Equation]
(cid:0) proof
[Table]
[Table]
Theorem B.2,
[Table] [Table]
[Table] [Table]
Theorem 6.
[Table]
Theorem 6.
A natural way to solve such problem is the multi-step GDA algorithm
[19, 23, 29, 33] that containsdouble-loop iterations in which the outer loop can be regarded as running inexact gradient descent ong ( x ) and the inner loop finds the approximate solution to max y ∈ R dy
f ( x, y ) for a given x .
Anotherclass of methods is the two-timescale (alternating) GDA algorithm
[9, 19, 41, 42] that only has singleloop iterations which update two variables with different stepsizes.
The two-timescale GDA methodcan be implemented more easily and typically performs better than multi-step GDA empirically.
Its convergence rate also can be established by analyzing function g ( x ) but the analysis is morechallenging than the multi-step GDA.
The variance reduction is a popular technique to improve the efficiency of stochastic optimizationalgorithms [2–4, 7, 8, 12, 15, 16, 25, 30, 31, 34–36, 40, 46, 47].
It is shown that solving nonconvexminimization problems with stochastic recursive gradient estimator [12, 15, 31, 40, 48] has the optimal
SFO complexity.
In the context of minimax optimization, the variance reduced algorithms also obtainthe best-known SFO complexities in several settings [1, 14, 23, 24, 38, 42].
Specifically, the (near)optimal SFO algorithm for several convex-concave minimax problem has been proposed [14, 24], butthe optimality for the more general case is still unclear [23, 42].
The Catalyst acceleration [18] is a useful approach to reduce the computational cost of ill-conditionedoptimization problems, which is based on a sequence of inexact proximal point iterations.
Linet al.
[20] first introduced Catalyst into minimax optimization.
Later, Luo et al.
[24], Tomininet al.
[38], Yang et al.
[43] designed the accelerated stochastic algorithms for convex-concave andnonconvex-concave problem.
Recently, Yang et al.
[44] also applied this technique to one-sided PLsetting.
Notation and Preliminaries
First of all, we present the definition of saddle point.
Definition 3.1.
We say ( x ∗ , y ∗ ) ∈ R d x × R d y is a saddle point of function f : R d x × R d y
→ R if itholds that f ( x ∗ , y ) ≤ f ( x ∗ , y ∗ ) ≤ f ( x, y ∗ ) for any x ∈ R d x and x ∈ R d y .
Then we formally define the Polyak-Łojasiewicz (PL) condition [32] as follows.
Definition 3.2.
{+We say a differentiable function h : R d → R satisfies µ -PL for some µ > 
+}
if∥∇ h ( z )
∥ 2 ≥ 2 µ (cid:0) h ( z ) − min z ′ ∈ R d h ( z ′ ) (cid:1) holds for any z ∈ R d .
Note that the PL condition does not require the strongly convexity and it can be satisfied even if thefunction is nonconvex [17].
We are interested in the finite-sum minimax optimization problem (1) under following assumptions.
Assumption 3.1.
We suppose each component f i : R d x × R d y → R is L -smooth, i.e., there exists aconstant L > 0 such that ∥∇ f i ( x, y ) − ∇ f i ( x ′ , y ′ )
∥ 2 ≤ L 2 (cid:0) ∥ x − x ′ ∥ 2 + L 2 ∥ y − y ′
∥ 2 (cid:1) holds forany x, x ′ ∈ R d x and y, y
′ ∈
R d y .
Assumption 3.2.
We suppose the differentiable function f : R d x × R d y
→ R satisfies two-sided PLcondition, i.e., there exist constants µ x > 0 and µ y > 0 such that f ( · , y ) is µ x -PL for any y ∈ R d yand − f ( x, · ) is µ y -PL for any x ∈ R d x .
Under Assumption 3.1 and 3.2, we define the condition numbers of problem (1) with respect to PLconditions for x and y as κ x ≜ L/µ x and κ y ≜ L/µ y respectively.
We also introduce the following assumption for the existence of saddle point.
Assumption 3.3 (Yang et al.
We suppose the function f : R d x × R d y
→ R has at least one saddle point ( x ∗ , y ∗ ) .
We also suppose that for any fixed y ∈ R d y , the problem min x ∈ R
dx f ( x, y )has a nonempty solution set and an optimal value; and for any fixed x ∈ R d x , the problemmax y ∈ R dy
f ( x, y ) has a nonempty solution set and a finite optimal value.
The goal of solving minimax problem under two-sided PL condition is finding an ϵ -approximatesolution or ϵ -saddle point that is defined as follows.
Definition 3.3.
We say x is an ϵ -approximate solution of problem (1) if it holds that g ( x ) − g ( x ∗ ) ≤ ϵ ,where g ( x ) = max y ∈ R dy f ( x, y ) .
Definition 3.4.
Under Assumption 3.3, we say ( x, y ) is an ϵ -saddle point of problem (1) if it holdsthat ∥ x − x ∗ ∥ 2 + ∥ y − y ∗ ∥ 2 ≤ ϵ .
We allow the saddle point does not exist for the problem with one-sided PL condition.
In such case, itis guaranteed that g ( x ) ≜ max y ∈ R dy
f
( x, y ) is differentiable [29, Lemma A.5] and we target to findan ϵ -stationary point of g ( x ) .
Definition 3.5.
If the function g : R d x → R is differentiable, we say x is an ϵ -stationary point of g ifit holds that ∥∇ g ( x ) ∥ ≤ ϵ .
A Faster Algorithm for the Two-Sided PL Condition
We first consider the two-sided PL conditioned minimax problem of the finite-sum form (1) under
Assumption 3.1, 3.2 and 3.3.
We propose a novel stochastic algorithm, which we refer to as SPIDERGDA.
The detailed procedure of our method is presented in Algorithm 1.
SPIDER-GDA constructsthe stochastic recursive gradient estimators as follows:
[Equation]
It simultaneously updates two variables x and y by estimators G x and G y with different stepsizesτ x = Θ(1 / ( κ 2 y L )) and τ y = Θ(1 /L ) respectively.
Huang et al.
[15], Luo et al.
[23], Xian et al.
[41]have studied the SPIDER-type algorithm for nonconvex-strongly-concave problem and showed itconverges to the stationary point of g ( x ) ≜ max y ∈ R dy
f ( x, y ) sublinearly.
However, solving theproblem minimax problems with two-sided PL condition desires stronger linear convergence rate,which leads to our theoretical analysis be different from previous work.
We measure the convergence of SPIDER-GDA by the following Lyapunov function
[Equation]
where x ∗ ∈ arg min x ∈ R
dx g ( x ) and λ =
Θ( κ 2 y ) .
We can establish recursion for V t,k as follows
[Equation]
1: ˜ x 0 = x 0 , ˜ y t
= y2: for t = 0 , 1 , . .
, T − 1 do 3: x t, 0 = ˜ x t , y t, 0 = ˜
y t 4: for k
, K − 1 do 5:
if mod ( k, M ) = 0 then 6:
G x ( x t,k , y t,k ) = ∇ x f ( x t,k , y t,k ) 7:
G y ( x t,k , y t,k ) = ∇ y f
( x t,k , y t,k ) 8: else 9: draw mini-batches S x and S y independently with sizes of B .
G x ( x t,k , y t,k ) =
1 B (cid:80)
i
[Equation]
S x
[ ∇ x f i ( x t,k , y t,k ) −
[Equation]
x f i ( x t,k − 1 , y t,k − 1 ) + G x ( x t,k − 1 , y t,k − 1 )] 11:
G y ( x t,k , y t,k ) =
1 B (cid:80) i ∈
S y [ ∇ y f i ( x t,k , y t,k ) − ∇ y f i ( x t,k − 1 , y t,k − 1 )
+ G y ( x t,k − 1 , y t,k − 1 )] 12: end if 13: x t,k +
= x t,k − τ x G x ( x t,k , y t,k ) 14: y t,k +
= x y,k
+ τ y G y
( x t,k , y t,k ) 15: end for 16: choose
17: end for 18: return
Using the above inequality by setting M = B = √ n leads to the estimators G x (˜ x t , ˜ y t ) and
G y (˜ x t , ˜ y t ) be sufficiently close to the exact gradient and converge to zero linearly, which indicatesg (˜ x t ) also converges to g ( x ∗ ) linearly.
We formally provide the convergence result for SPIDER-GDAin the following theorem and its detailed proof is shown in appendix.
Theorem 4.1.
Under Assumption 3.1, 3.2 and 3.3, we run Algorithm 1 with M = B = √ n ,τ y = 1 / (5 L ) , λ =
L 2 /µ
y , τ x = τ y / (24 λ ) , K = ⌈ 4224 / ( µ x τ x ) ⌉ and T =
⌈ log(1 /ϵ )
Then theoutput (˜ x T , ˜ y T ) satisfies g (˜ x T ) − g ( x ∗ ) ≤ ϵ and g (˜ x T ) − f (˜ x T , ˜ y T ) ≤ ϵ in expectation; and it takesno more than O (cid:0) ( n + √ nκ x κ 2 y ) log(1 /ϵ ) (cid:1) SFO calls.
Corollary 4.1.
Under the setting of Theorem 4.1, the output (˜ x T , ˜ y T ) is also a (16 L 2 ϵ/ ( µ 2 x µ y ))
-saddle point in expectation.
Our results provide an SFO upper bound of O (( n + √ nκ x κ 2 y ) log(1 /ϵ )) for finding an ε -approximatesolution that is better than the complexity O (( n + n 2 / 3 κ x κ 2 y ) log(1 /ϵ )) derived from SVRGAGDA [42].
It is possible to use SVRG-type [16, 46] estimators to replace the stochastic recursiveestimators in Algorithm 1, which results the algorithm SVRG-GDA.
We can prove that SVRGGDA also has O (( n + n 2 / 3 κ x κ 2 y ) log(1 /ϵ ))
SFO upper bound that matches the theoretical result of
SVRG-AGDA.
We provide the details in Appendix C.
Further Acceleration with Catalyst
Both the proposed SPIDER-GDA (Algorithm 1) and existing SVRG-AGDA [42] have the complexities more heavily depend on the condition number of y than the condition number of x .
It isnatural to ask can we make the dependency of two condition numbers balanced like the results inthe strongly-convex-strongly-concave case [20, 23, 38].
In this section, we show it is possible byintroducing the Catalyst acceleration.
1: u 0 = x2: for k = 0 , 1 , . .
, K − 1 do 3: ( x k +1 , y k +1 )
=
SPIDER-GDA (cid:0) f ( x, y ) + β 2 ∥ x − u k ∥ 2 , ( x k , y k ) , T, K, M, B, τ x , τ y (cid:1) 4:
u k +
=
x k +1 + γ ( x k +1 − x k ) 5: end for 6: option I (two-sided PL): return ( x K , y K ) 7: option II (one-sided PL): return (ˆ x, ˆ y ) chosen uniformly at random from { ( x k , y k ) } K − 1 k =
We proposed the accelerated SPIDER-GDA (AccSPIDER-GDA) in Algorithm 2 for reducing thecomputational cost further.
Each iteration of the algorithm solve the following sub-problem
[Equation]
by SPIDER-GDA (Algorithm 1).
AccSPIDER-GDA has the following convergence result if thesub-problem attain the required accuracy.
Lemma 5.1.
Under Assumption 3.1, 3.2 and 3.3, we run Algorithm 2 by β = 2 L , γ = 0 and theappropriate setting for the sub-problem solver such that ∥ x k − ˜ x k ∥ 2 + ∥
y k − ˜ y k
∥ 2 ≤ δ , where(˜ x k , ˜ y k ) is a saddle point of F k − 1 ( k ≥ 1) and δ = µ x ϵ/ (11( µ x + 4 L ) L ) .
Then it holds that
[Equation]
The setting β = Θ( L ) in Lemma 5.1 guarantees the sub-problem (2) has condition number ˜ κ x = O (1)for x .
It is more well-conditioned on x , we prefer to address the following equivalent problem
[Equation]
Using Corollary 4.1 (in the view of changing the roles of x and y ), we can find a δ -approximatesaddle point of − F k ( x, y ) within O (( n + √ nκ y ˜ κ 2 x ) log(1 /δ ))
=
O (( n + √ nκ y ) log(1 /δ ))
SFOcalls in expectation.
Additionally, Lemma 5.1 means Algorithm 2 requires O ( κ x log(1 /ϵ )) numberof iterations to find an ϵ -approximate solution of the problem.
Thus, the total complexity for AccSPIDER-GDA becomes O (( nκ x + √ nκ x κ y ) log(1 /ϵ ) log(1 /δ )) .
Theorem 5.1.
Under Assumption 3.1, 3.2 and 3.3, if we let γ = 0 , β = 2 L and use Algorithm1 to solve each sub-problem max y ∈ R dy min x ∈ R dx F k ( x, y ) with M = B = √ n , τ x = 1 / (10 L ) ,λ = 128 , τ y =
τ x / (24 λ ) , K = ⌈ 4224 / ( µ y τ y )
⌉ , T =
⌈ log(1 / ˆ δ ) ⌉ and precision ˆ δ is defined by
[Equation]
where δ is followed by the definition in Lemma 5.1.
Then Algorithm 2 can return x K such that [Equation] ϵ in expectation with no more than O (( nκ x + √ nκ x κ y ) log(1 /ϵ ) log( κ x κ y /ϵ ))
SFO calls.
Furthermore, Theorem 5.1 implies the more general result as follows.
Corollary 5.1.
Under Assumption 3.2, 3.1 and 3.3, running Algorithm 2 by appropriate parameterscould achieve the output x K such that g ( x K ) − g ( x ∗ ) ≤ ϵ in expectation within the following SFOcomplexity
[Equation]
Lemma 5.1 does not rely on the choice of sub-problem solver, we can apply the accelerationframework in Algorithm 2 by replacing SPIDER-GDA with other algorithms.
We summarize the
SFO complexities for the acceleration of different algorithms in Table 3. [Table]
  ˜ O ( n 2 / 3 κ x κ y log 2 (1 /ϵ )) , 0 < n 1 / 3 ≤ κ x ;
[Table] O (cid:0) [Table] y log 2 (1 /ϵ ) (cid:1)
[Table]
κ x < n 1 / 3 ≤ κ x κ y
[Table]

[Table]
κ
[Table]  Extension to One-Sided PL Condition In this section, we show the idea if SPIDER-GDA and its Catalyst acceleration also work for one-sided PL condition.
We relax Assumption 3.2 and 3.3 to the following one.
Assumption 6.1.
We suppose that − f ( x, · ) is µ y -PL for any x ∈ R d x ; the problem max y ∈ R dy f ( x, y )has a nonempty solution set and an optimal value ; g ( x ) ≜ max y ∈ R dy
f
( x, y ) is lower bounded, i.e.,we have g ∗ =
inf x ∈ R dx g ( x ) > −∞ .
We first show that the SFO complexity of SPIDER-GDA outperforms SVRG-GDA by a factor of
O ( n 1 / 6 ) in Theorem 6.1.
Theorem 6.1.
Under Assumption 3.1 and 6.1 , Let T = 1 and M, B, τ x , τ y , λ as defined in Theorem 4.1 and K = ⌈ 64 / ( τ x ϵ 2 )
⌉ , then Algorithm 1 can guarantee the output ˆ x to satisfy ∥∇ g (ˆ x ) ∥ ≤ ϵin expectation with no more than O ( n + √ nκ 2 y Lϵ − 2 ) SFO calls.
The AccSPIDER-GDA also performs better than SPIDER-GDA in one-sided PL condition for illconditioned case.
In the following lemma, we show that AccSPIDER-GDA could find an approximatestationary point if we solve the sub-problem sufficiently accurate.
Lemma 6.1.
Under Assumption 3.1 and 6.1, if it holds true that ∥ x k − ˜ x k ∥ 2 + ∥
y k − ˜ y k
∥ 2 ≤ δ forsome saddle point (˜ x k , ˜ y k ) of F k − 1 ( k ≥ 1) , where δ =
ϵ 2 / (16 L 2 (1 / 2 µ y + 11)) .
Let β = 2 L , thenfor the output (ˆ x, ˆ y ) of Algorithm 2, it holds true that
[Equation]
Compared with two-sided PL condition, the analysis of AccSPIDER-GDA is more complicated sincethe precision δ k at each round are different.
By choosing the parameters of the algorithm carefully,we obtain the following results.
Theorem 6.2.
Under Assumption 3.1 and 6.1, if we run Algorithm 2 by γ = 0 , β = 2 L and usethe sub-problem Algorithm 1 to solve with M, B, τ x , τ y , λ, K as Theorem 5.1 and T = ⌈ log(1 /δ
k ) ⌉ ,where
[Equation]
and δ is followed by the definition in Lemma 6.1, then Algorithm 2 can find ˆ x such that ∥∇ g (ˆ x ) ∥ ≤ ϵin expectation within O (( n + √ nκ y Lϵ − 2 ) log( κ y /ϵ ))
SFO calls.
We can directly set β = 0 for Algorithm 2 in the case of very large n .
The comparison the complexitiesof algorithm for one-sided PL condition in Table 2.
Besides, the algorithms SPIDER-GDA, GDAand SVRG-GDA can also be accelerated with Catalyst framework and we present the correspondingresults in Table 4. [Table]
Experiments
In this section, we conduct the numerical experiments to show the advantage of proposed algorithms.
We consider the following two player Polyak-Łojasiewicz game
[Equation]
where
P = 1 n n (cid:88) i
= p i p ⊤ i , Q = 1 n n (cid:88) i =
q i q ⊤ i and R = 1 n n (cid:88) i =
r i r ⊤ i .
We independently sample p i , q i and r i from N (0 , Σ P ) , N (0 , Σ Q ) and N (0 , Σ R ) respectively.
Weset the covariance matrix Σ P as the form of UDU ⊤ such that U ∈ R
d
× r is column orthogonalmatrix and D ∈ R r × r is diagonal with r < d .
The diagonal elements of D are distributed uniformlyin the interval [ µ, L ] with 0 < µ < L .
The matrix Σ Q is set by the similar way to Σ P .
We alsolet Σ R = 0 .
1 V V ⊤ , where each element of V is sampled from N (0 , 1) independently.
Since thecovariance matrices Σ P and Σ Q are rank-deficient, it is guarantee that both P and Q are singular.
Hence, the objective function is not strongly-convex and not strongly-concave, but it satisfies thetwo-sided PL-condition [17].
We set n = 6000 , d = 10 , r = 5 and conduct two cases for differentcondition numbers.
We compare the proposed SPIDER-GDA (Algorithm 1) and AccSPIDER-GDA (Algorithm 2)
withthe baseline algorithm SVRG-AGDA [42].
We let B = 1 and M = n for all of these algorithms andboth of the stepsizes for x and y are tuned from { 10 − 1 , 10 − 2 , 10 − 3 , 10 − 4 , 10 − 5 } .
For AccSPIDER,we set β = L/ (20 n ) and γ = 0 .
We present the results of the number of SFO calls against thenorm of gradient and the distance to the saddle point in Figure 1 and Figure 2.
It is clear that ouralgorithms outperform than baselines.
Conclusion and Future Works
In this paper, we have investigated stochastic GDA algorithms for PL conditioned minimax problems.
We have proposed the SPIDER-GDA algorithm that reduce the dependency of the sample numbersin SFO complexity.
Moreover, we have introduced Catalyst scheme for further acceleration forill-conditioned problems.
Our ideas both work for the two-sided PL and one-sided cases.
However, itis unclear whether our algorithms obtain the optimal SFO complexity. The tightness for the minimaxproblem under PL conditions is still an open problem.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] See Section 8.
(c) Did you discuss any potential negative societal impacts of your work?
[No] The
purpose of this work is for to provide a better understanding of GDA on a class of
nonconvex-nonconcave minimax optimization
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Section 3.
(b) Did you include complete proofs of all theoretical results?
[Yes] See Appendix for
details.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] We include thecodes
In the supplemental materials.
(b) Did you specify all the training details (e g., data splits, hyperparameters, how they were chosen)?
[Yes] See Section 7.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[No] We wants to compare the training dynamic, and differenttrials may cause different numerical results, which can not be observed clearly in one
graph.
(d) Did you include the total amount of compute and the type of resources used (e
g., typeof GPUs, internal cluster, or cloud provider)?
[No] The experiments is certainly simple
and easy to run under CPUs.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[Yes]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[No] These datasets are common.
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[No] These datasets are common.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]