P ROXIMAL G RADIENT D ESCENT -A SCENT : V ARIABLE C ONVERGENCE UNDER KŁ G EOMETRY
Anonymous authors
Paper under double-blind review
A BSTRACT
The gradient descent-ascent (GDA) algorithm has been widely applied to solve minimax optimization problems.
In order to achieve convergent policy parameters for minimax optimization, it is important that GDA generates convergent variable sequences rather than convergent sequences of function value or gradient norm.
However, the variable convergence of GDA has been proved only under convexity geometries, and it lacks understanding in general nonconvex minimax optimization.
This paper ﬁlls such a gap by studying the convergence of a more general proximal-GDA for regularized nonconvex-strongly-concave minimax optimization.
Speciﬁcally, we show that proximal-GDA admits a novel Lyapunov function, which monotonically decreases in the minimax optimization process and drives the variable sequences to a critical point.
By leveraging this Lyapunov function and the
KŁ geometry that parameterizes the local geometries of general nonconvex functions, we formally establish the variable convergence of proximal-GDA to a certain critical point x ∗ , i.e., x t → x ∗ , y t → y ∗ ( x ∗ ) .
Furthermore, over the full spectrum of the KŁ-parameterized geometry, we show that proximal-GDA achieves different types of convergence rates ranging from sublinear convergence up to ﬁnite-step convergence, depending on the geometry associated with the KŁ parameter.
This is the ﬁrst theoretical result on the variable convergence for nonconvex minimax optimization.
I NTRODUCTION
Minimax optimization is a classical optimization framework that has been widely applied in various modern machine learning applications, including game theory Ferreira et al.
(2012), generative adversarial networks (GANs) Goodfellow et al.
adversarial training Sinha et al.
(2017), reinforcement learning Qiu et al.
imitation learning Ho and Ermon (2016); Song et al.
(2018), etc.
A typical minimax optimization problem is shown below, where f is a differentiable function.
[Equation]
A popular algorithm for solving the above minimax problem is gradient descent-ascent (GDA), which performs a gradient descent update on the variable x and a gradient ascent update on the variable y alternatively in each iteration.
Under the alternation between descent and ascent updates, it is much desired that GDA generates sequences of variables that converge to a certain optimal point, i.e., the minimax players obtain convergent optimal policies.
In the existing literature, many studies have established the convergence of GDA-type algorithms under various global geometries of the objective function, e.g., convex-concave geometry ( f is convex in x and concave in y ) Nedi´c
and
Ozdaglar (2009), bi-linear geometry Neumann (1928); Robinson (1951) and Polyak-Łojasiewicz (PŁ) geometry Nouiehed et al.
; Yang et al.
Some other work studied GDA under stronger global geometric conditions of f such as convex-strongly-concave geometry Du and Hu (2019) and strongly-convex-strongly-concave geometry Mokhtari et al.
; Zhang and Wang (2020), under which GDA is shown to generate convergent variable sequences.
However, these special global function geometries do not cover modern machine learning problems that usually have complex models and nonconvex geometry.
Recently, many studies characterized the convergence of GDA in nonconvex minimax optimization, where the objective function is nonconvex in x .
Speciﬁcally, Lin et al.
Nouiehed et al.
; Xu et al. (2020b) studied the convergence of GDA in the nonconvex-concave setting whereas Lin et al.
Xu et al.
(2020b) focused on the nonconvex-strongly-concave setting.
In these general nonconvex settings, it is shown that GDA converges to a certain stationary point at a sublinear rate, i.e., (cid:107) G ( x t ) (cid:107) ≤ t − α for some α > 0 , where G ( x t ) corresponds to a certain notion of gradient.
Although such a gradient convergence result implies the stability of the algorithm, namely, lim t →∞ (cid:107) x t +1 − x t (cid:107) = 0 , it does not guarantee the convergence of the variable sequences { x t } t , { y t } t generated by GDA.
So far, the variable convergence of GDA has not been established for nonconvex problems, but only under (strongly) convex function geometries that are mentioned previously Du and Hu (2019); Mokhtari et al.
; Zhang and Wang (2020).
Therefore, we want to ask the following fundamental question:
• Q1: Does GDA have guaranteed variable convergence in nonconvex minimax optimization?
If so, where do they converge to?
In fact, proving the variable convergence of GDA in the nonconvex setting is highly nontrivial due to the following reasons: 1) the algorithm alternates between a minimization step and a maximization step; 2) It is well understood that strong global function geometry leads to the convergence of GDA.
However, in general nonconvex setting, the objective functions typically do not have an amenable global geometry.
Instead, they may satisfy different types of local geometries around the critical points.
Hence, it is natural and much desired to exploit the local geometries of functions in analyzing the convergence of GDA.
The Kurdyka-Łojasiewicz (KŁ) geometry provides a broad characterization of such a local geometry for nonconvex functions.
The Kurdyka-Łojasiewicz (KŁ) geometry (see Section 2 for details)
Bolte et al.
(2007; 2014) parameterizes a broad spectrum of the local nonconvex geometries and has been shown to hold for a broad class of practical functions.
Moreover, it also generalizes other global geometries such as strong convexity and PŁ geometry.
In the existing literature, the KŁ geometry has been exploited extensively to analyze the convergence rate of various gradient-based algorithms in nonconvex optimization, e.g., gradient descent Attouch and Bolte (2009); Li et al.
(2017) and its accelerated version Zhou et al.
(2020) as well as the distributed version Zhou et al.
(2016a).
Hence, we are highly motivated to study the convergence rate of variable convergence of GDA in nonconvex minimax optimization under the KŁ geometry.
In particular, we want to address the following question:
• Q2:
How does the local function geometry captured by the KŁ parameter affects the variable convergence rate of GDA?
In this paper, we provide comprehensive answers to these questions.
We develop a new analysis framework to study the variable convergence of GDA in nonconvex-strongly-concave minimax optimization under the KŁ geometry.
We also characterize the convergence rates of GDA in the full spectrum of the parameterization of the KŁ geometry.
1. O UR CONTRIBUTIONS
We consider the following regularized nonconvex-strongly-concave minimax optimization problem
[Equation]
where f is a differentiable and nonconvex-strongly concave function, g is a general nonconvex regularizer and h is a convex regularizer.
Both g and h can be possibly nonsmooth.
To solve the above regularized minimax problem, we study a proximal-GDA algorithm that leverages the forward-backward splitting update Lions and Mercier (1979); Attouch et al.
We study the variable convergence property of proximal-GDA in solving the minimax problem (P).
Speciﬁcally, we show that proximal-GDA admits a novel Lyapunov function H ( x, y )
(see Proposition 2), which is monotonically decreasing along the trajectory of proximal GDA, i e.,
H ( x t +1 , y t +1 ) <
H ( x t , y t ) .
Based on the monotonicity of this Lyapunov function, we show that every limit point of the variable sequences generated by proximal-GDA is a critical point of the objective function
Moreover, by exploiting the ubiquitous KŁ geometry of the Lyapunov function, we prove that the entire variable sequence of proximal-GDA has a unique limit point, or equivalently speaking, it converges to a certain critical point x ∗ , i.e., x t → x ∗ , y t → y ∗ ( x ∗ ) (see the deﬁnition of y ∗ in Section
To the best of our knowledge, this is the ﬁrst variable convergence result of GDA-type algorithms in nonconvex minimax optimization.
Furthermore, we characterize the asymptotic convergence rates of both the variable sequences and function value of proximal-GDA in different parameterization regimes of the KŁ geometry.
Depending on the value of the KŁ parameter θ [Table] we show that proximal-GDA achieves different types of convergence rates ranging from sublinear convergence up to ﬁnite-step convergence, as we summarize in Table 1 below
[Table]
[Table]
1. R ELATED WORK
Deterministic GDA algorithms: Yang et al.
(2020) studied an alternating gradient descent-ascent (AGDA) algorithm in which the gradient ascent step uses the current variable x t +1 instead of x t .
Xu et al.
(2020b) studied an alternating gradient projection algorithm which applies (cid:96) 2 regularizer to the local objective function of GDA followed by projection onto the constraint sets.
Daskalakis and Panageas (2018); Mokhtari et al.
; Zhang and Wang (2020) analyzed optimistic gradient descent-ascent (OGDA) which applies negative momentum to accelerate GDA.
Mokhtari et al.
(2020) also studied an extra-gradient algorithm which applies two-step GDA in each iteration.
Nouiehed et al.
(2019) studied multi-step GDA where multiple gradient ascent steps are performed, and they also studied the momentum-accelerated version.
Cherukuri et al.
; Daskalakis and Panageas (2018);
Jin et al.
(2020) studied GDA in continuous time dynamics using differential equations.
Adolphs et al.
(2019) analyzed a second-order variant of the GDA algorithm.
Stochastic GDA algorithms: Lin et al.
; Yang et al.
(2020) analyzed stochastic GDA and stochastic AGDA, which are direct extension of GDA and AGDA to the stochastic setting.
Variance reduction techniques have been applied to stochastic minimax optimization, including SVRG-based
Du and Hu (2019); Yang et al.
SPIDER-based Xu et al.
(2020a)
, STORM Qiu et al.
(2020) and its gradient free version Huang et al.
Xie et al.
(2020) studied the complexity lower bound of ﬁrst-order stochastic algorithms for ﬁnite-sum minimax problem.
KŁ geometry: The KŁ geometry was deﬁned in Bolte et al.
The KŁ geometry has been exploited to study the convergence of various ﬁrst-order algorithms for solving minimization problems, including gradient descent Attouch and Bolte (2009), alternating gradient descent Bolte et al.
(2014), distributed gradient descent Zhou et al.
(2016a; 2018a), accelerated gradient descent Li et al.
It has also been exploited to study the convergence of second-order algorithms such as Newton’s method Noll and Rondepierre ( [Table] [Table] [Table]
[Table]
P ROBLEM F ORMULATION AND KŁ G EOMETRY
In this section, we introduce the problem formulation, technical assumptions and the KurdykaŁojasiewicz (KŁ) geometry.
We consider the following regularized minimax optimization problem.
[Equation]
P) where f : R m × R n → R is a differentiable and nonconvex-strongly concave loss function, Y ⊂ R n is a compact and convex set, and g, h are the regularizers that are possibly non-smooth.
In particular, deﬁne Φ( x ) :
=
max y ∈Y f ( x, y ) − h ( y ) , and then the problem (P) is equivalent to the minimization problem min x ∈ R m Φ( x ) + g ( x ) .
Throughout the paper, we adopt the following standard assumptions on the problem (P).
Assumption 1.
The objective function of the problem (P) satisﬁes:
Function f ( · , · ) is L -smooth and function f ( x, · ) is µ -strongly concave; 2.
Function (Φ + g )( x ) is bounded below,
i
e., inf x ∈ R m (Φ + g )( x ) >
−∞ ; 3.
For any α ∈ R , the sub-level set { x : (Φ + g )( x ) ≤ α } is compact; 4.
Function h is proper and convex, and function g is proper and lower semi-continuous.
To elaborate, item 1 considers the class of nonconvex-strongly-concave functions f that has been widely studied in the existing literature Lin et al.
(2020); Jin et al.
Xu et al. (2020b;a); Lu et al. Items 2 and 3 guarantee that the minimax problem (P) has at least one solution, and the variable sequences generated by the proximal-GDA algorithm
(See Algorithm 1) are bounded. Item 4 requires the regularizer h to be convex (possibly non-smooth), which includes many normbased popular regularizers such as (cid:96) p ( p ≥ 1 ), elastic net, nuclear norm, spectral norm, etc.
On the other hand, the other regularizer g can be nonconvex but lower semi-continuous, which includes all the aforementioned convex regularizers, (cid:96) p ( 0 ≤ p < 1 ), Schatten- p norm, rank, etc.
Hence, our formulation of the problem (P) covers a rich class of nonconvex objective functions and regularizers and is more general than the existing nonconvex minimax formulation in Lin et al.
(2020), which does not consider any regularizer.
Remark 1.
We note that the strong concavity of f ( x, · ) in item 1 can be relaxed to concavity, provided that the regularizer h ( y ) is µ -strongly convex.
In this case, we can add − µ 2 (cid:107) y (cid:107) 2 to both f ( x, y ) and h ( y ) such that Assumption 1 still holds.
For simplicity, we will omit the discussion on this case.
By strong concavity of f ( x, · ) , it is clear that the mapping y ∗ ( x ) :=
arg max y ∈Y f ( x, y ) − h
( y ) is uniquely deﬁned for every x ∈ R m .
In particular, if x ∗ is the desired minimizer of Φ( x ) , then ( x ∗ , y ∗ ( x ∗ )) is the desired solution of the minimax problem (P).
Next, we present some important properties regarding the function Φ( x ) and the mapping y ∗ ( x ) .
The following proposition generalizes the Lemma 4.3 of Lin et al.
(2020) to the regularized setting.
The proof can be found in Appendix A.
Throughout, we denote κ = L/µ as the condition number and denote ∇ 1 f ( x, y ) , ∇ 2 f ( x, y ) as the gradients with respect to the ﬁrst and the second input argument, respectively.
For example, with this notation, ∇
f ( x, y ∗ ( x )) denotes the gradient of f ( x, y ∗ ( x )) with respect to only the ﬁrst input argument x , and the x in the second input argument y ∗ ( x ) is treated as a constant.
Proposition 1 (Lipschitz continuity of y ∗ ( x ) and ∇ Φ( x ) ) .
Let Assumption 1 hold.
Then, the mapping y ∗ ( x ) and the function Φ( x ) satisfy
Mapping y ∗ ( x ) is κ -Lipschitz continuous; 2.
Function Φ( x ) is L (1 + κ ) -smooth with ∇ Φ( x ) = ∇ 1 f ( x, y ∗ ( x )) .
As an intuitive explanation of Proposition 1, since the function f ( x, y ) − h
( y ) is L -smooth with respect to x , both the maximizer y ∗ ( x ) and the corresponding maximum function value Φ( x ) should not change substantially with regard to a small change of x .
Recall that the minimax problem (P) is equivalent to the standard minimization problem min x ∈ R m Φ( x ) + g ( x ) , which, according to item 2 of Proposition 1, includes a smooth nonconvex function Φ( x ) and a lower semi-continuous regularizer g ( x ) .
Hence, we can deﬁne the optimization goal of the minimax problem (P) as ﬁnding a critical point x ∗ of the nonconvex function Φ( x ) + g ( x ) that satisﬁes the necessary optimality condition 0 ∈ ∂ (Φ + g )( x ∗ ) for minimizing nonconvex functions.
Here, ∂ denotes the notion of subdifferential as we elaborate below.
Deﬁnition 1.
(Subdifferential and critical point, Rockafellar and Wets (2009))
The Frechét subdifferential (cid:98) ∂h of function h at x ∈ dom h is the set of u ∈ R d deﬁned as
[Equation]
and the limiting subdifferential ∂h at x ∈ dom h is the graphical closure of (cid:98) ∂h deﬁned as:
[Equation]
The set of critical points of h is deﬁned as crit h :=
{ x : 0 ∈ ∂h
( x ) } .
Throughout, we refer to the limiting subdifferential as subdifferential.
We note that subdifferential is a generalization of gradient (when h is differentiable) and subgradient (when h is convex) to the nonconvex setting.
In particular, any local minimizer of h must be a critical point.
Next, we introduce the Kurdyka-Łojasiewicz (KŁ) geometry of a function h .
Throughout, the point-to-set distance is denoted as dist Ω ( x ) :=
inf u ∈ Ω (cid:107) x − u (cid:107) .
Deﬁnition 2 (KŁ geometry, Bolte et al.
A proper and lower semi-continuous function h is said to have the KŁ geometry if for every compact set Ω ⊂ dom h on which h takes a constant value h Ω ∈ R , there exist ε, λ > 0 such that for all ¯ x ∈ Ω and all x ∈ { z ∈ R m : dist Ω ( z ) <
ε, h Ω < h ( z ) <
h Ω + λ } , the following condition holds:
[Equation]
where ϕ (cid:48) is the derivative of function ϕ : [0 , λ ) → R + , which takes the form ϕ ( t )
=
cθ t θ for certain universal constant c > 0 and KŁ parameter θ ∈ (0 , 1] .
The KŁ geometry characterizes the local geometry of a nonconvex function around the set of critical points.
To explain, consider the case where h is a differentiable function so that ∂h ( x ) = ∇ h ( x ) .
Then, the KŁ inequality in eq.
(1) becomes h ( x ) − h Ω ≤ O ( (cid:107)∇ h ( x ) (cid:107) 11 − θ ) , which generalizes the
Polyak-Łojasiewicz (PL) condition h ( x )
− h Ω ≤ O ( (cid:107)∇ h ( x ) (cid:107) 2 ) Łojasiewicz (1963); Karimi et al.
(2016) (i.e., KŁ parameter θ = 12 ).
Moreover, the KŁ geometry has been shown to hold for a large class of functions including sub-analytic functions, logarithm and exponential functions and semi-algebraic functions.
These function classes cover most of the nonconvex objective functions encountered in practical machine learning applications Zhou et al.
(2016b)
; Yue et al.
; Zhou and Liang (2017)
; Zhou et al.
(2018b).
The KŁ geometry has been exploited extensively to analyze the convergence of various ﬁrst-order algorithms, e.g., gradient descent Attouch and Bolte (2009); Li et al.
(2017), alternating minimization
Bolte et al.
and distributed gradient methods Zhou et al.
(2016a).
It has also been exploited to study the convergence of second-order algorithms such cubic regularization Zhou et al.
(2018b).
In these works, it has been shown that the variable sequences generated by these algorithms converge to a desired critical point in nonconvex optimization, and the convergence rates critically depend on the parameterization θ of the KŁ geometry.
In the subsequent sections, we provide a comprehensive understanding of the convergence and convergence rate of proximal-GDA under the KŁ geometry.
P ROXIMAL -GDA AND G LOBAL C ONVERGENCE A NALYSIS
In this section, we study the following proximal-GDA algorithm that leverages the forward-backward splitting updates Lions and Mercier (1979); Attouch et al. (2013) to solve the regularized minimax problem (P) and analyze its global convergence properties.
In particular, the proximal-GDA algorithm is a generalization of the GDA Du and Hu (2019) and projected GDA Nedi´c and Ozdaglar (2009) algorithms.
The algorithm update rule is speciﬁed in Algorithm 1, where the two proximal gradient steps are formally deﬁned as
[Equation]
Recall that our goal is to obtain a critical point of the minimization problem min x ∈ R m Φ( x ) + g ( x ) .
Unlike the gradient descent algorithm which generates a sequence of monotonically decreasing function values, the function value (Φ + g )( x k ) along the variable sequence generated by proximalGDA is generally oscillating due to the alternation between the gradient descent and gradient ascent steps.
Hence, it seems that proximal-GDA is less stable than gradient descent.
However, our next result shows that, for the problem (P), the proximal-GDA admits a special Lyapunov function that monotonically decreases in the optimization process.
The proof of Proposition 2 is in Appendix B.
Input: Initialization x 0 , y 0 , learning rates η x , η y .
for t = 0 , 1 , 2 , . . .
, T − 1 do
[Equation]
endOutput: x T , y T .
Proposition 2.
Let Assumption 1 hold and deﬁne the Lyapunov function H ( z ) :
=
Φ( x ) + g ( x ) + (1 −
[Equation]
2 ) (cid:107) y − y ∗ ( x ) (cid:107) 2 with z := ( x, y ) .
Choose the learning rates such that η x ≤
[Equation]
Then, the variables z t = ( x t , y t ) generated by proximal-GDA satisfy, for all t = 0 , 1 , 2 , ... [Equation]
[Equation]
We ﬁrst explain how this Lyapunov function is introduced in the proof.
By eq.
(19) in the supplementary material, we established a recursive inequality on the objective function (Φ + g )( x t +1 ) . One can see that the right hand side of eq. (19) contains a negative term −(cid:107) x t +1 − x t (cid:107) 2 and an undesired positive term (cid:107) y ∗ ( x t ) − y t (cid:107) 2 .
Hence, the objective function (Φ + g )( x t +1 ) may be oscillating and cannot serve as a proper Lyapunov function.
In the subsequent analysis, we break this positive term into a difference of two terms (cid:107) y ∗ ( x t ) − y t (cid:107) 2 −(cid:107)
y ∗ ( x t +1 )
− y t +1 (cid:107) 2 , by leveraging the update of y t + for solving the strongly-concave maximization problem.
After proper rearranging, this difference term contributes to the quadratic term in the Lyapunov function.
We note that the Lyapunov function H ( z ) is the objective function Φ( x ) + g ( x ) regularized by the additional quadratic term (1 − 14 κ 2 ) (cid:107) y − y ∗ ( x ) (cid:107) 2 , and such a Lyapunov function clearly characterizes our optimization goal.
To elaborate, consider a desired case where the sequence x t converges to a certain critical point x ∗ and the sequence y t converges to the corresponding point y ∗ ( x ∗ ) .
In this case, it can be seen that the Lyapunov function H ( z t ) converges to the desired function value (Φ + g )( x ∗ ) .
Hence, solving the minimax problem (P) is equivalent to minimizing the Lyapunov function.
More importantly, Proposition 2 shows that the Lyapunov function value sequence { H ( z t ) } t is monotonically decreasing in the optimization process of proximal-GDA, implying that the algorithm continuously makes optimization progress.
We also note that the coefﬁcient (1 − 14 κ 2 ) in the Lyapunov function is chosen in a way so that eq.
(4) can be proven to be strictly decreasing.
This monotonic property is the core of our analysis of proximal-GDA.
Based on Proposition 2, we obtain the following asymptotic properties of the variable sequences generated by proximal-GDA.
The proof can be found in Appendix C.
Corollary 1.
Based on Proposition 2, the sequences { x t , y t } t generated by proximal-GDA satisfy
[Equation]
The above result shows that the variable sequences generated by proximal-GDA in solving the problem (P) are asymptotically stable.
In particular, the last two equations show that y t asymptotically approaches the corresponding maximizer y ∗ ( x t ) of the objective function f ( x t , y )
+ g ( x t ) − h
( y ) .
Hence, if x t converges to a certain critical point, y t will converge to the corresponding maximizer.
Discussion: We note that the monotonicity property in Proposition 2 further implies the convergence rate result min 0 ≤ k ≤ t (cid:107) x k +1 − x k (cid:107) ≤ O ( t − 1 / 2 ) (by telescoping over t ).
When there is no regularizer, this convergence rate result can be shown to further imply that min 0 ≤ k ≤ t (cid:107)∇ Φ( x k ) (cid:107) ≤ O ( t − 1 / 2 ) , which reduces to the Theorem 4.4 of Lin et al.
However, such a convergence rate result does not imply the convergence of the variable sequences { x t } t , { y t } t .
To explain, we can apply the convergence rate result (cid:107) x t +1 − x t (cid:107) ≤ O ( t − 1 / 2 ) to bound the trajectory norm as (cid:107) x T (cid:107) ≤ (cid:107) x 0 (cid:107) + (cid:80) T − 1 t =
(cid:107) x
t +1 − x t (cid:107) ≈ √ T , which diverges to + ∞ as T → ∞ .
Therefore, such a type of convergence rate does not even imply the boundedness of the trajectory.
In this paper, our focus is to establish the convergence of the variable sequences generated by proximal-GDA.
All the results in Corollary 1 imply that the alternating proximal gradient descent & ascent updates of proximal-GDA can achieve stationary points, which we show below to be critical points.
Theorem 1 (Global convergence) .
Let Assumption 1 hold and choose the learning rates
η x ≤κ 3 ( L +3) 2 , η y ≤ 1 L .
Then, proximal-GDA satisﬁes the following properties.
The function value sequence { (Φ + g )( x t ) } t converges to a ﬁnite limit H ∗ > −∞ ; 2.
The sequences { x t } t , { y t } t are bounded and have compact sets of limit points.
Moreover, (Φ + g )( x ∗ ) ≡ H ∗ for any limit point x ∗ of { x t } t ; 3.
Every limit point of { x t } t is a critical point of (Φ + g )( x ) .
The proof of Theorem 1 is presented in Appendix D.
The above theorem establishes the global convergence property of proximal-GDA.
Speciﬁcally, item 1 shows that the function value sequence { (Φ + g )( x t ) } t converges to a ﬁnite limit H ∗ , which is also the limit of the Lyapunov function sequence { H ( z t ) } t .
Moreover, items 2 & 3 further show that all the converging subsequences of { x t } t converge to critical points of the problem, at which the function Φ + g achieves the constant value H ∗ .
These results show that proximal-GDA can properly ﬁnd critical points of the minimax problem (P).
Furthermore, based on these results, the variable sequences generated by proximal-GDA are guaranteed to enter a local parameter region where the Kurdyka-Łojasiewicz geometry holds, which we exploit in the next section to establish stronger convergence results of the algorithm.
V ARIABLE C ONVERGENCE OF P ROXIMAL -GDA UNDER KŁ G EOMETRY
We note that Theorem 1 only shows that every limit point of { x t } t is a critical point, and the sequences { x t , y t } t may not necessarily be convergent.
In this section, we exploit the local KŁ geometry of the
Lyapunov function to formally prove the convergence of these sequences.
Throughout this section, we adopt the following assumption. Assumption 2. Regarding the mapping y ∗ ( x ) , the function (cid:107) y ∗ ( x ) − y (cid:107) 2 has a non-empty subdifferential, i.e., ∂ x ( (cid:107) y ∗ ( x ) − y (cid:107) 2 ) (cid:54)
= ∅ .
Note that in many practical scenarios y ∗ ( x ) is sub-differentiable.
In addition, Assumption 2 ensures the sub-differentiability of the Lyapunov function H ( z ) :
=
Φ( x ) + g ( x ) + (1 − 14 κ 2 ) (cid:107) y − y ∗ ( x ) (cid:107) 2 .
We obtain the following variable convergence result of proximal-GDA under the KŁ geometry.
The proof is presented in Appendix E.
Theorem 2 (Variable convergence) .
Let Assumption 1 & 2 hold and assume that H has the KŁ geometry.
Choose the learning rates η x ≤ 1 κ 3 ( L +3) 2 and η y ≤ 1 L .
Then, the sequence { ( x t , y t ) } t generated by proximal-GDA converges to a certain critical point ( x ∗ , y ∗ ( x ∗ )) of (Φ + g )( x ) , i.e., x t t → x ∗ , y t t → y ∗ ( x ∗ ) .
Theorem 2 formally shows that proximal-GDA is guaranteed to converge to a certain critical point ( x ∗ , y ∗ ( x ∗ )) of the minimax problem (P), provided that the Lyapunov function belongs to the large class of KŁ functions.
To the best of our knowledge, this is the ﬁrst variable convergence result of
GDA-type algorithms in nonconvex minimax optimization.
The proof logic of Theorem 2 can be summarized as the following two key steps.
Step 1:
By leveraging the monotonicity property of the Lyapunov function in Proposition 2, we ﬁrst show that the variable sequences of proximal-GDA eventually enter a local region where the
KŁ geometry holds;
Step 2: Then, combining the KŁ inequality in eq.
(1) and the monotonicity property of the Lyapunov function in eq.
(4), we show that the variable sequences of proximal-GDA are Cauchy sequences and hence converge to a certain critical point.
C ONVERGENCE R ATE OF P ROXIMAL -GDA UNDER KŁ G EOMETRY
In this section, we exploit the parameterization of the KŁ geometry to establish various types of asymptotic convergence rates of proximal-GDA.
We obtain the following asymptotic convergence rates of proximal-GDA under different parameter regimes of the KŁ geometry.
The proof is presented in Appendix F.
In the sequel, we denote t 0 as a sufﬁciently large positive integer, denote c > 0 as the constant in Deﬁnition 2 and also deﬁne
[Equation]
Theorem 3 (Funtion value convergence rate) .
Under the same conditions as those of Theorem 2, the
Lyapunov function value sequence { H ( z t ) } t converges to the limit H ∗ at the following rates.
If KŁ geometry holds with θ = 1 , then H ( z t ) ↓
H ∗ within ﬁnite number of iterations ; 2.
If KŁ geometry holds with θ ∈ ( 12 , 1) , then H ( z t ) ↓
H ∗ super-linearly as
[Equation]
If KŁ geometry holds with θ = 12 , then H ( z t ) ↓ H ∗ linearly as
[Equation]
If KŁ geometry holds with θ ∈ (0 , 12 ) , then H ( z t ) ↓
H ∗ sub-linearly as
[Equation]
where C = min (cid:104) 1 −
[Equation] d
[Equation]
It can be seen from the above theorem that the convergence rate of the Lyapunov function of proximal-GDA is determined by the KŁ parameter θ .
A larger θ implies that the local geometry of H is ‘sharper’, and hence the corresponding convergence rate is orderwise faster.
In particular, the algorithm converges at a linear rate when the KŁ geometry holds with θ = 12 (see the item 3), which is a generalization of the Polyak-Łojasiewicz (PL) geometry.
As a comparison, in the existing analysis of GDA, such a linear convergence result is established under stronger geometries, e.g., convex-strongly concave Du and Hu (2019), strongly convex-strongly concave Mokhtari et al.
Zhang and Wang (2020) and two-sided PL condition Yang et al.
In summary, the above theorem provides a full characterization of the fast convergence rates of proximal-GDA in the full spectrum of the KŁ geometry.
Moreover, we also obtain the following asymptotic convergence rates of the variable sequences that are generated by proximal-GDA under different parameterization of the KŁ geometry.
The proof is presented in Appendix G.
Theorem 4 (Variable convergence rate) .
Under the same conditions as those of Theorem 2, the sequences { x t , y t } t converge to their limits x ∗ , y ∗ ( x ∗ ) respectively at the following rates.
If KŁ geometry holds with θ = 1 , then ( x t , y t )
→ ( x ∗ , y ∗ ( x ∗ )) within ﬁnite number of iterations ; 2.
If KŁ geometry holds with θ ∈ ( 12 , 1) , then ( x t , y t ) → ( x ∗ , y ∗ ( x ∗ )) super-linearly as max (cid:8) (cid:107) x t − x ∗ (cid:107) , (cid:107) y t − y ∗ ( x ∗ ) (cid:107) (cid:9)
≤
O (cid:16) exp (cid:16) − (cid:0)
[Equation]
If KŁ geometry holds with θ = 12 , then ( x t , y t ) → ( x ∗ , y ∗ ( x ∗ )) linearly as
[Equation]
If KŁ geometry holds with θ ∈ (0 , 12 ) , then ( x t , y t ) → ( x ∗ , y ∗ ( x ∗ )) sub-linearly as
[Equation]
To the best of our knowledge
[Equation]
this is the ﬁrst characterization of the variable convergence rates of proximal-GDA in the full spectrum of the KŁ geometry.
It can be seen that, similar to the convergence rate results of the function value sequence, the convergence rate of the variable sequences is also affected by the parameterization of the KŁ geometry.
C ONCLUSION
In this paper, we develop a new analysis framework for the proximal-GDA algorithm in nonconvexstrongly-concave optimization.
Our key observation is that proximal-GDA has a intrinsic Lyapunov function that monotonically decreases in the minimax optimization process.
Such a property demonstrates the stability of the algorithm.
Moreover, we establish the formal variable convergence of proximal-GDA to a critical point of the objective function under the ubiquitous KŁ geometry.
Our results fully characterize the impact of the parameterization of the KŁ geometry on the convergence rate of the algorithm.
In the future study, we will leverage such an analysis framework to explore the convergence of stochastic GDA algorithms and their variance-reduced variants.