Nest Your Adaptive Algorithm for Parameter-Agnostic Nonconvex Minimax Optimization
Anonymous Author(s)
AfﬁliationAddress email
Abstract
Adaptive algorithms like AdaGrad and AMSGrad are successful in nonconvexoptimization owing to their parameter-agnostic ability – requiring no a prioriknowledge about problem-speciﬁc parameters nor tuning of learning rates.
However, when it comes to nonconvex minimax optimization, direct extensions ofsuch adaptive optimizers without proper time-scale separation may fail to workin practice.
We provide such an example proving that the simple combinationof Gradient Descent Ascent (GDA) with adaptive stepsizes can diverge if theprimal-dual stepsize ratio is not carefully chosen; hence, a fortiori, such adaptiveextensions are not parameter-agnostic.
To address the issue, we formally introducea Nested Adaptive framework, NeAda for short, that carries an inner loop foradaptively maximizing the dual variable with controllable stopping criteria and anouter loop for adaptively minimizing the primal variable.
Such mechanism canbe equipped with off-the-shelf adaptive optimizers and automatically balance theprogress in the primal and dual variables.
Theoretically, for nonconvex-stronglyconcave minimax problems, we show that NeAda with AdaGrad stepsizes canachieve the near-optimal e
O ( ✏  2 ) and e
O ( ✏  4 ) gradient complexities respectively inthe deterministic and stochastic settings, without prior information on the problem’ssmoothness and strong concavity parameters.
To the best of our knowledge, this isthe ﬁrst algorithm that simultaneously achieves near-optimal convergence rates andparameter-agnostic adaptation in the nonconvex minimax setting.
Numerically, wefurther illustrate the robustness of the NeAda family with experiments on simpletest functions and a real-world application.
Introduction
Adaptive gradient methods, whose stepsizes and search directions are adjusted based on past gradients,have received phenomenal popularity and are proven successful in a variety of large-scale machinelearning applications.
Prominent examples include AdaGrad [17], RMSProp [31], AdaDelta [81],
Adam [41], and AMSGrad [67], just to name a few.
Their empirical success is especially pronouncedfor nonconvex optimization such as training deep neural networks.
Besides improved performance,being parameter-agnostic is another important trait of adaptive methods.
Unlike (stochastic) gradientdescent, adaptive methods often do not require a priori knowledge about problem-speciﬁc parameters(such as Lipschitz constants, smoothness, etc.).
1 On the theoretical front, some adaptive methods canachieve nearly the same convergence guarantees as (stochastic) gradient descent [17, 76, 67].
Recently, adaptive methods have sprung up for minimax optimization:
[Equation]
where f is l -Lipschitz smooth jointly in x and y , Y is closed and convex, and ⇠ is a random vector.
Such problems have found numerous applications in generative adversarial networks (GANs) [25, 4],
Wasserstein GANs [4], generative adversarial imitation learning [32], reinforcement learning [14, 58],adversarial training [71], domain-adversarial training of neural networks [21] , etc.
A common practice is to simply combine adaptive stepsizes with popular minimax optimizationalgorithms such as Gradient Descent Ascent (GDA), extragradient method (EG) and the like; seee.g.
It is worth noting that these methods are reported successful in some applicationsyet at other times can suffer from training instability.
In recent years, theoretical behaviors ofsuch adaptive methods are extensively studied for convex-concave minimax optimization; see e.g.,[6, 3, 1, 18, 70, 22, 57, 15].
However, for minimax optimization in the important nonconvex regime,little theory related to adaptive methods is known.
Unlike the convex-concave setting, a key challenge for nonconvex minimax optimization lies in thenecessity of a problem-speciﬁc time-scale separation of the learning rates between the min-playerand max-player when GDA or EG methods are applied, as proven in [79, 49, 68, 8].
This makesthe design of adaptive methods fundamentally different from and more challenging than nonconvexminimization.
Several recent attempts [28, 33, 34] studied adaptive methods for nonconvex-stronglyconcave minimax problems; yet, they all require explicit knowledge of the problems’ smoothnessand strong concavity parameters to maintain a stepsize ratio proportional to the condition number.
Such a requirement evidently undermines the parameter-agnostic trait of adaptive methods.
This thenraises a couple of interesting questions: (1) Without a problem-dependent stepsize ratio, does simplecombination of GDA and adaptive stepsizes still converge?
(2) Can we design an adaptive algorithmfor nonconvex minimax optimization that is truly parameter-agnostic and provably convergent?
In this paper, we address these questions and make the following key contributions:
• We investigate two generic frameworks for adaptive minimax optimization: one is a simple
(non-nested) adaptive framework, which performs one step of update of x and y simulta-
neously with adaptive gradients; the other is Nested Adaptive (NeAda) framework, which performs multiple updates of y after one update of x , each with adaptive gradients.
Both
frameworks allow ﬂexible choices of adaptive mechanisms such as Adam, AMSGrad and
AdaGrad.
We provide an example proving that the simple adaptive framework can fail
to converge without setting an appropriate stepsize ratio; this applies to any of the adap-
tive mechanisms mentioned above, even in the noiseless setting.
In contrast, the NeAda
framework is less sensitive to the stepsize ratio, as numerically illustrated in Figure 1.
• We provide the convergence analysis for a representative of NeAda that uses AdaGrad
stepsizes for x and a convergent adaptive optimizer for y , in terms of nonconvex-strongly-
concave minimax problems.
Notably, the convergence of this general scheme does not
require to know any problem parameters and does not assume the bounded gradients.
We
demonstrate that NeAda is able to achieve e O ( ✏  2 ) oracle complexity for the deterministic
setting and e O ( ✏  4 ) for the stochastic setting to converge to ✏ -stationary point, matching best
known bounds.
To the best of our knowledge, this seems to be the ﬁrst adaptive framework
for nonconvex minimax optimization that is provably convergent and parameter-agnostic.
• We further make two complementary contributions, which can be of independent interest.
First, we propose a general AdaGrad-type stepsize for strongly-convex problems without
knowing the strong convexity parameters, and derive a convergence rate comparable to SGD. It can serve as a subroutine for NeAda.
Second, we provide a high probability convergence
result for the primal variable of NeAda under a subGaussian assumption.
• Finally, we numerically validate the robustness of the NeAda
framework on several testfunctions compared to the non-nested adaptive framework, and demonstrate the effectivenessof the NeAda framework on distributionally robust optimization task with a real dataset.
1.1 Related work
Adaptive algorithms.
Duchi et al.
[17] introduce AdaGrad for convex online learning and achieve
O ( p T ) regrets.
Li and Orabona [47] and Ward et al.
[76] show an e
O ( ✏  4 ) complexity for AdaGradin the nonconvex stochastic optimization.
There are an extensive number of works on AdaGrad-typemethods; to list a few, [45, 2, 39, 63].
Another family of algorithms uses more aggressive stepsizes of [Figure] exponential moving average of the past gradients, such as Adam [41] and RMSProp [31].
Reddi et al.[67] point out the non-convergence of Adam and provide a remedy with non-increasing stepsizes.
There is a surge in the study of Adam-type algorithms due to their popularity in the deep neuralnetwork training [80, 11, 50].
Some work provides the convergence results for adaptive methods in thestrongly-convex optimization [75, 44, 60].
Line search and stochastic line search are another effectivestrategy that can detect the objective’s curvature and have received much attention [72, 74, 73].
Notably, many adaptive algorithms are parameter-agnostic [17, 67, 76].
Nonconvex minimax optimization.
Stationary convergence of GDA in NC-SC setting was ﬁrstprovided by Lin et al.
[49], showing O ( ✏  2 ) oracle complexity and O ( ✏  4 ) sample complexitywith minibatch.
Recently, Chen et al.
[10] and Yang et al.
[79] achieve this sample complexity inthe stochastic setting without minibatch.
GDmax is a double loop algorithm that maximizes thedual variable to a certain accuracy.
It achieves nearly the same complexity as GDA [62].
Sebbouhet al.
[68] recently discuss the relation between the two-time-scale and number of inner steps for GDmax.
Besides NC-SC setting, some work provides convergent algorithms when the objective is(non-strongly) concave about the dual variable[82, 54, 78].
Nonconvex-nonconcave regime is onlyexplored under some special structure [52, 15], such as Polyak- Ł ojasiewicz (PL) condition [19].
Allalgorithms mentioned above require prior knowledge about problem parameters, such as smoothnessmodulus, strong concavity modulus, and noise variance.
Adaptive algorithms in minimax optimization.
There exist many adaptive and parameteragnostic methods designed for convex-concave minimax optimization as a special case of monotonevariational inequality [6, 3, 1, 18, 70, 22, 57, 15].
Most of them combine extragradient method, mirrorprox [61] or the like, with AdaGrad mechanism.
Liu et al.
[51] and Dou and Li [16] relax convexityconcavity assumption to the regime where Minty variational inequality (MVI) has a solution.
In thesesettings, time-scale separation of learning rates is not required even for non-adaptive algorithms.
Fornonconvex-strongly-concave problems, Huang and Huang [33], Huang et al.
[34], Guo et al.
[28]propose adaptive methods, which set the learning rates based on knowledge about smoothness andstrong-concavity modulus and the bounds for adaptive stepsizes.
Non-nested and nested adaptive methods
In this section, we investigate two generic frameworks that can incorporate most existing adaptivemethods into minimax optimization.
We remark that many variants encapsulated in these twofamilies are already widely used in practice, such as training of GAN [24], distributionally robustoptimization [69], etc.
These two frameworks, coined as non-nested and nested adaptive methods,can be viewed as adaptive counterparts of GDA and GDmax.
We aim to illustrate the differencebetween these two adaptive families, even though GDA and GDmax are often considered “twins”.
Non-nested adaptive methods.
In Algorithm 1, non-nested methods update the primal and dualvariables in a symmetric way.
Weighted gradients m xt and m yt are the moving average of the paststochastic gradients with the momentum parameters  x and  y .
The effective stepsizes of x and yare ⌘ x / p v xt and ⌘ y / p v yt , where the division is taken coordinate-wise.
We refer to ⌘ x and ⌘ y as learning rates, and v xt , v yt are some average of squared-past gradients through function .
Manypopular choices of adaptive stepsizes are captured in this framework, see also [67]:(GDA)
[Equation]
= 0;
[Equation]
(Adam)
[Equation]
1: Input: x 0 and y2: for t = 0 , 1 , 2 , ... do 3: sample ⇠ t and let g xt =
r x F ( x t , y t ; ⇠ t ) and g yt =
r y F ( x t , y t ; ⇠ t ) 4: // update the first moment m xt
{++
 =+}
x m xt + (1   x ) g xt and m yt +1 =
y m yt + (1   y )
g yt 5: // update the second moment v xt +
=  v x 0 , { ( g xi ) 2 } ti =
and v yt +1 =
v y 0 , { ( g yi ) 2 } ti =
6: // update variables x t +1 = x
t  ⌘ x p v xt +1 m xt +
and y t +1 =
y t + ⌘ y p v yt
+1 m yt
+ 7: end for
2: for t = 0 , 1 , 2 , ... do 3: for k = 0 , 1 , 2 , ... until a stopping criterion is satisﬁed
do 4: sample ˆ ⇠ kt and g yt,k = r y F ( x t , y kt ; ˆ ⇠ kt ) 5: m y t,k +1 =
y m y t,k + (1   y ) g y t,k 6: v yt,k
+1 =
y  v yt, 0 , { ( g yt,i ) 2 } ki =  7: y k +1 t = y kt + ⌘ y p v yt,
k +1 m yt,k + 8: end for 9: v yt +1 ,
= v yt,k +
and m yt +1 ,
= m yt,k + 10: sample ⇠ t and g xt =
r x F ( x t , y k +1 t ; ⇠ t ) 11: m
xt +1 =
x m xt + (1   x ) g xt 12: v xt +
= x  v x 0 , { ( g xi ) 2 } ti =
13: x t +1 = x
t  ⌘ x p v xt +1 m xt + 14: end for
Nested adaptive (NeAda) methods.
NeAda, presented in Algorithm 2, has a nesting inner loopto maximize y until some stopping criterion is reached (see details in Section 3).
Instead of using aﬁxed number of inner iterations or a ﬁxed target accuracy as in GDmax [49, 62], NeAda graduallyincreases the accuracy of the inner loop as the outer loop proceeds to make it fully adaptive.
We refer to the ratio between two learning rates, i.e. ⌘ y / ⌘ x , as the two-time-scale.
The currentanalysis of GDA in nonconvex-strongly-concave setting requires two-time-scale to be proportionalwith the condition number  = l/µ , where l and µ are Lipschitz smoothness and strongly-concavitymodulus [49, 79].
We provide an example showing that the problem-dependent two-time-scale isnecessary for GDA and most non-nested methods even in the deterministic setting.
Lemma 2.1. Consider the function [- 12 y 
-]f
{+( x, y ) =  12 y 
+}
+ Lxy  L 2 2 x 2 in the deterministic setting.
Letr ⌘ x = ⌘ y .
(1) GDA will not converge to the stationary point when r  L 2 :
[Equation]
(2) Assume the averaging function x and y are the same, and satisfy that for any ⌧ , if v xt = ⌧ v ytand ( g xt ) 2 = ⌧ ( g yt ) 2 then v xt +
= ⌧ v yt
+1 .
With  x =  y , v x 0 = v y 0 =
and m x 0 = m y 
 {+= 
+}
(whichare commonly used in practice), non-nested adaptive method will not converge when r  L :
[Equation]
When r = L , r x f ( x t , y t ) =
r x f ( x 0 , y 0 ) for all t .
Remark 1. Most popular adaptive stepsizes we mentioned before, such as Adam, AMSGrad and
AdaGrad, have averaging functions satisfying the assumption in the lemma.
Any point on the line y =
Lx is a stationary point for the above function, and the distance from a point to this line isproportional to its gradient norm, so the divergence in gradient norm will also implies that of iterates. In the proof, we will also show that the averaged or best iterate will still diverge under the samecondition.
The lemma implies that for any given time-scale r , there exists a problem for which thenon-nested algorithm does not converge to the stationary point, so they are not parameter-agnostic.
We compare non-nested and nested methods combined with different stepsizes schemes: Adam,
AMSGrad, AdaGrad and ﬁxed stepsize, on the function:  12 y 2 + 2 xy  2 x 2 .
In the experimentsof this section, we halt the inner loop when the (stochastic) gradient about y is smaller than 1 /tor
the number iteration is greater than t .
We observe from Figure 1 that the thresholds for thenon-convergence of non-nested methods ( r = 2 for adaptive methods and r = 4 for GDA) are exactlyas predicted by the lemma.
Although the adaptive methods admit a smaller two-time-scale thresholdthan GDA in this example, it is not a universal phenomenon from our experiments in Section 4.
Interestingly, nested adaptive methods are robust to different two-time-scales and always have thetrend to converge to the stationary point.
Convergence Analysis of NeAda-AdaGrad
In this section, we reveal the secret behind the robust performance of NeAda by providing theconvergence guarantee for a representative member in the family.
For sake of simplicity and clarity,we mainly focus on NeAda with AdaGrad.
Adam-type mechanism can suffer from non-convergencealready for nonconvex minimization despite its good performance in practice.
Our result also shedslight on the analysis of other more sophisticated members such as AMSGrad in the family.
NeAda-AdaGrad: Presented in Algorithm 3, NeAda-AdaGrad adopts the scalar AdaGrad schemefor the x -update in the outer loop and uses mini-batch in the stochastic setting.
For the inner loopfor maximizing y , we run some adaptive algorithm for maximizing y until some easily checkablestopping criterion is satisﬁed.
We suggest two criteria here: at t -th outer loop: (I) the squard gradientmapping norm about y is smaller than 1 / ( t + 1) in the deterministic setting, (II) the number of innerloop iterations reaches t + 1 in the stochastic setting.
Algorithm
For the purpose of theoretical analysis, we mainly focus on the minimax problem of the form (1) underthe nonconvex-strongly-concave (NC-SC) setting 2 , formally stated in the following assumptions.
Assumption 3.1 (Lipschitz smoothness) .
There exists a positive constant l > 0 such thatmax
[Equation]
holds for all x 1 , x 2 2 R d , y 1 , y 2 2 Y .
Assumption 3.2 (Strong-concavity in y ) .
There exists µ > 0 such that: f ( x, y 1 )  f ( x, y 2 ) +hr y f ( x, y 2 ) , y 1  y 2 i + µ 2 k y 1  y 2 k 2 , 8 x 2 R d , y 1 , y 2 2 Y .
For simplicity of notation, deﬁne  =
l/µ as the condition number,  ( x ) = max y
2 Y f ( x, y ) asthe primal function, and y ⇤ ( x ) =
arg max Y f
( x, y ) as the optimal y w.r.t x .
Since the objective isnonconvex about x , we aim at ﬁnding an ✏ -stationary point ( x t , y t )
such that E
kr x f ( x t , y t ) k  ✏and E k y t  y ⇤ ( x t )
k  ✏ , where the expectation is taken over the randomness in the algorithm.
3.1 Convergence in deterministic and stochastic settings
Assumption 3.3 (Stochastic gradients) . r x F ( x, y ; ⇠ ) and r y F ( x, y ; ⇠ ) are unbiased stochasticestimators of r x f ( x, y ) and r y f ( x, y ) and have variances bounded by  2  0 .
We assume the unbiased stochastic gradients have the variance  2 , and the problem reduces to thedeterministic setting when  = 0 .
Now we provide a general analysis of the convergence for anyadaptive optimizer used in the inner loop.
Theorem 3.1. Deﬁne the expected cumulative suboptimality of inner loops as E =
[Equation]
i .
Under Assumptions 3.1, 3.2 and 3.3, the output from Algorithm 3 satisﬁes
[Equation]
where A = e
O ⇣  ( x 0 )  min x  ( x ) ⌘ + 2  p M +  l ⌘ ⌘ .
Remark [-
-]2.
The general analysis is built upon milder assumptions than existing work on AdaGradin nonconvex optimization, not requiring either bounded gradient in [76] or prior knowledge aboutthe smoothness modulus in [47].
This theorem implies the algorithm attains convergence for thenonconvex variable x with any constant ⌘ > 0 and v 0 > 0 that does not depend on any problemparameter, so it is parameter-agnostic.
Remark [-
-]3.
Another beneﬁt of this analysis is that the variance  appears in the leading term T  14 ,which means the convergence rate can interpolate between the deterministic and stochastic settings.
It implies a complexity of e
O ( ✏  2 ) in the deterministic setting and e
O ( ✏  4 ) in the stochastic settingfor the primal variable as long as the accumulated suboptimality for the inner-loops E is e O (1) , 197 regardless of the batch size M .
However, M can control the number of outer loops and there affectthe sample complexity for the dual variable.
In the next two theorems, we derive the total complexities, in the deterministic and stochastic settings,of ﬁnding ✏ -stationary point by controlling the cumulative suboptimality E in Theorem 3.1 forsubroutine A with speciﬁc convergence rate.
In fact, we can also use any off-the-shelf adaptiveoptimizer for solving the inner maximization problem up to the desired accuracy.
Note that (stochastic)
GDmax ﬁxes each inner-loop’s accuracy or steps to be related with µ , ` and ✏ so that E can be easilybounded [49, 62].
In contrast, since we do not have access to the problem parameters and ✏ , Algorithm3 gradually increases the inner-loop accuracy.
In the proof of the following theorems, we will showthat with our proposed stopping criteria and desired subroutines, E is bounded by O (log T ) . Theorem 3.2 (deterministic) . Suppose we have a linearly-convergent subroutine A for maximizing any strongly concave function h ( · ) : [Equation]
where y k is k -th iterate, y ⇤ is the optimal solution, and a 1 > 0 and 0 < a 2 < 1 are constantsthat can depend on the parameters of h .
Under the same setting as Theorem 3.1 with  = 0 , for
Algorithm 3 with M = 1 and a subroutine A under stopping criterion I, there exists t ⇤  e
O  ✏  such that ( x t ⇤ , y t ⇤ ) is an ✏ -stationary point.
Therefore, the total gradient complexity is e
O  ✏  .
Remark [-
-]4.
This complexity is optimal in ✏ up to logarithmic term [83], similar to GDA [49].
Notethat many adaptive and parameter-agnostic algorithms can achieve the linear rate when solvingsmooth and strongly concave maximization problems; to list a few, gradient ascent with backtrackingline-search [72], SC-AdaNGD [44] and polyak stepsize [30, 53, 64] 3 .
Here we can also pick moregeneral subproblem accuracy in criterion I that only needs to scale with 1 /t .
Theorem 3.3 (stochastic) . Suppose we have a sub-linearly-convergent subroutine A for maximizing any strongly concave function h ( · ) : after K = k + 1 iterations
[Equation] where y k is k -th iterate, y ⇤ [-( x t ) k 
-]is the optimal solution, and b 1 , b 2 > 0 are constants that can depend onthe parameters of h .
Under the same setting as Theorem 3.1, for Algorithm 3 with M = ✏  2 andsubroutine A under the stopping criterion II, there exists t ⇤  e
O  ✏  such that ( x t ⇤ , y t ⇤ ) is an✏ -stationary point.
Therefore, the total stochastic gradient complexity is e
O  ✏  .
Remark [-
-]5.
This e
O  ✏  complexity is nearly optimal in the dependence of ✏ for stochastic NC-SCproblems [46].
Here we set M = ✏  2 for the simplicity of exposition, and a similar result also holdsfor gradually increasing M .
The sublinear rate speciﬁed above for solving the stochastic stronglyconvex subproblem can be achieved by several existing parameter-agnostic algorithms under someadditional assumptions, such as F REE R EX M OMENTUM [12] and Coin-Betting [13] 4 .
Parameter-free
SGD [9] is partially parameter-agnostic that only requires the stochastic gradient bound rather thanthe strongly-convexity parameter.
Mukkamala and Hein [60] and Wang et al.
[75] introduce thevariants of AdaGrad, RMSProp and Adam for strongly-convex online learning, but they need to knowboth gradient bounds and strongly-convexity parameter for setting stepsizes.
We will show in thenext subsection that AdaGrad with a slower decaying rate is parameter-agnostic.
We note that theanalysis of this theorem is not the simple gluing of the outer loop and inner loop complexity, butrequires more sophisticated control of the cumulative suboptimality E .
With the popularity of computational resource demanding deep neural networks, in both minimizationand minimax applications, people ﬁnd high probability guarantees for a single run of an algorithmuseful [40, 48].
Given the lack of such guarantee in the minimax optimization, we provide a highprobability convergence result for NeAda-AdaGrad in Appendix C, which shows a similar samplecomplexity as Theorem 3.3 under the subGaussian noise.
3.2 Generalized AdaGrad for strongly-convex subproblem
We now introduce the generalized AdaGrad for minimizing strongly convex objectives, which canserve as an adaptive subroutine for Algorithm 3, without requiring knowledge on the strongly convexparameter.
We analyze it for the more general online convex optimization setting: at each round t , thelearner updates its decision x t , then it suffers a loss f t ( x t ) and receives the sub-gradient of f t .
Thegeneralized AdaGrad, described in Algorithm 4, keeps the cumulative gradient norm v t and takes thestepsize ⌘ /v ↵ t with a decaying rate ↵ 2 (0 , 1] .
When ↵ = 1 / 2 , it reduces to the scalar version of theoriginal AdaGrad [17]; when ↵ = 1 , it reduced to the scalar version of SC-AdaGrad [60].
Theorem 3.4. Consider Algorithm 4 for online convex optimization and assume that (i) f t iscontinuous and µ -strongly convex, (ii) X is convex and compact with diameter D ; (iii) k g t k  G for 247 every t .
Then for 0 < ↵ < 1 with any ⌘ > 0 , the regret of Algorithm 4 satisﬁes:
[Equation]
and for ↵ = 1 with ⌘  G 2 2 µ , [Equation]
[Figure]
where c ↵ and d ↵ are constants depending on the problem parameters, ↵ and ⌘ .
The theorem implies a logarithmic regret for the case ↵ = 1 , but the stepsize needs knowledge aboutproblem’s parameters µ and G ; similar results are shown for SC-AdaGrad [60] and SAdam [75].
When ↵ < 1 , the algorithm becomes parameter-agnostic and attains an O ( T 1  ↵ ) regret.
Suchparameter-agnostic phenomenon for smaller decaying rates is also observed for SGD in stochasticoptimization [20].
Proving the regret bound for the generalized AdaGrad with ↵ < 1 in the onlinesetting is much more challenging, since the adversarial g t can lead to a “sudden” change in thestepsize.
In the proof, we bound the possible number of times such “sudden” change could happen.
To the best of our knowledge, this is the ﬁrst regret bound for adaptive methods with general decayingrates in the strongly convex setting.
By online-to-batch conversion [38], it can be converted to
O ( T  ↵ ) rate in the strongly convex stochastic optimization.
Xie et al.
[77] prove the O (1 /T )convergence rate for AdaGrad with ↵ = 1 / 2 in this setting, but under a strong assumption, Restricted
Uniform Inequality of Gradients (RUIG), that requires the loss function with respect to each sample⇠ to satisfy the error bound condition with some probability.
Experiments
To evaluate the performance of NeAda, we conducted experiments on simple test functions and areal-world application of distributional robustness optimization (DRO).
In all cases, we compare
NeAda with the non-nested adaptive methods using the same adaptive schemes.
For notationalsimplicity, in all ﬁgure legends, we label the non-nested methods with the names of the adaptivemechanisms used.
We observe from all our experiments that: 1) while non-nested adaptive methodscan diverge without the proper two-time-scale, NeAda with adaptive subroutine always converges; 2)when the non-nested method converges, NeAda can achieve comparable or even better performance.
4.1 Test functions
In Section 2, we have compared NeAda with non-nested methods on a quadratic function in Figure 1and the observations match Lemma 2.1.
Now we consider a more complicated function that iscomposed of McCormick function in x , a bilinear term, and a quadratic term in y ,
[Equation]
For this function, we compare the adaptive frameworks in the stochastic setting with Gaussian noise.
As demonstrated in Figure 2, non-nested methods are sensitive to the selection of the two-time-scale.
When the learning rate ratio is too small, e.g., ⌘ y / ⌘ x = 0 .
01 , non-nested Adam, AMSGrad and GDAall fail to converge.
We observe that GDA converges when the ratio reaches 0.03, while non-nested
Adam and AMSGrad still diverge until 0.05.
Although non-nested adaptive methods require a smallerratio than GDA in Lemma 2.1, this example illustrates that adaptive algorithms sometimes canbe more sensitive to the time separation.
In comparison, NeAda with adaptive subroutine alwaysconverges regardless of the learning rate ratio.
[Figure]
4.2 Distributional robustness optimization
[Figure]
To justify the effectiveness of NeAda on real-world applications, we carried out experiments ondistributionally robust optimization [69], where the primal variable is the model weights to be learnedby minimizing the empirical loss while the dual variable is the adversarial perturbed inputs.
The dualvariable problem targets ﬁnding perturbations that maximize the empirical loss but not far away fromthe original inputs.
Formally, for model weights x and adversarial samples y , we have:
[Equation]
where n is the total number of training samples, v i is the i -th original input and f i is the loss functionfor the i -th sample.
is a trade-off parameter between the empirical loss and the magnitude ofperturbations.
When  is large enough, this problem is nonconvex-strongly-concave, and followingthe same setting as [69, 68], we set  = 1 . 3 .
For NeAda, we use both stopping criterion I withstochastic gradient and criterion II in our experiments.
For the results, we report the training lossand the test accuracy on adversarial samples generated from fast gradient sign method (FGSM)
FGSM can be formulated as x adv = x + ✏ · sign ( r x f ( x )) , where ✏ is the noise level.
To getreasonable test accuracy, NeAda with Adam as subroutine is compared with Adam with ﬁxed 15inner loop iterations, which is consistent with the choice of inner loop steps in [69], and such choiceobtains much better test accuracy than the completely non-nested Adam.
Our experiments include asynthetic dataset and MNIST [43] with code modiﬁed from [55]. We show the results of MNIST herewhile leave the discussion of the synthetic dataset in Appendix D. For MNIST, we use a convolutionalneural network with three convolutional layers and one ﬁnal fully-connected layer.
Following eachconvolutional layer, ELU activation and batch normalization are used.
We compare NeAda with Adam under three different noise levels and the accuracy is shown in
Figures 3(a) to 3(c).
Under all noise levels, NeAda outperforms Adam with the same learning rates. When we have proper time-scale separation (the red curves), both methods achieve good test accuracy,and NeAda achieves higher accuracy and converges faster.
After we change to the same learning ratesfor the primal and dual variables (the blue curves), the accuracy drop of NeAda is slighter comparedto Adam, especially when ✏ = 0 .
As for the training loss shown in Figure 3(d), NeAda (the solidcurves) is always faster at the beginning.
We also observed that with proper time-scale separation, NeAda reaches a lower loss.
Conclusion
Both non-nested and nested adaptive methods are popular in nonconvex minimax problems, e.g., thetraining of GANs.
In this paper, we demonstrate that non-nested algorithms may fail to convergewhen the time-scale separation is ignorant of the problem parameter even when the objective isstrongly-concave in the dual variable with noiseless gradients information.
We propose ﬁxes to thisproblem with a family of nested algorithms–NeAda, that nests the max oracle of the dual variableunder an inner loop stopping criterion.
The proper stopping criterion will help to balance the outerloop progress and inner loop accuracy.
NeAda-AdaGrad attains the near-optimal complexity withouta priori knowledge of problem parameters in the nonconvex-strongly-concave setting.
It can be afuture direction to design parameter-agnostic algorithms for nonconvex-concave minimax problemsor more general regimes by leveraging recent progress in nonconvex minimax optimization and theadaptive analysis in this paper.
Another interesting direction is to investigate the convergence behaviorof Adam-type algorithms with general decaying rates in the strongly convex online optimization.