Proximal Gradient Descent-Ascent: Variable Convergence under KŁ Geometry

1 INTRODUCTION . Minimax optimization is a classical optimization framework that has been widely applied in various modern machine learning applications , including game theory Ferreira et al . ( 2012 ) , generative adversarial networks ( GANs ) Goodfellow et al . ( 2014 ) , adversarial training Sinha et al . ( 2017 ) , reinforcement learning Qiu et al . ( 2020 ) , imitation learning Ho and Ermon ( 2016 ) ; Song et al . ( 2018 ) , etc . A typical minimax optimization problem is shown below , where f is a differentiable function . min x∈X max y∈Y f ( x , y ) . A popular algorithm for solving the above minimax problem is gradient descent-ascent ( GDA ) , which performs a gradient descent update on the variable x and a gradient ascent update on the variable y alternatively in each iteration . Under the alternation between descent and ascent updates , it is much desired that GDA generates sequences of variables that converge to a certain optimal point , i.e. , the minimax players obtain convergent optimal policies . In the existing literature , many studies have established the convergence of GDA-type algorithms under various global geometries of the objective function , e.g. , convex-concave geometry ( f is convex in x and concave in y ) Nedić and Ozdaglar ( 2009 ) , bi-linear geometry Neumann ( 1928 ) ; Robinson ( 1951 ) and Polyak-Łojasiewicz ( PŁ ) geometry Nouiehed et al . ( 2019 ) ; Yang et al . ( 2020 ) . Some other work studied GDA under stronger global geometric conditions of f such as convex-strongly-concave geometry Du and Hu ( 2019 ) and strongly-convex-strongly-concave geometry Mokhtari et al . ( 2020 ) ; Zhang and Wang ( 2020 ) , under which GDA is shown to generate convergent variable sequences . However , these special global function geometries do not hold for modern machine learning problems that usually have complex models and nonconvex geometry . Recently , many studies characterized the convergence of GDA in nonconvex minimax optimization , where the objective function is nonconvex in x . Specifically , Lin et al . ( 2020 ) ; Nouiehed et al . ( 2019 ) ; Xu et al . ( 2020b ) ; Boţ and Böhm ( 2020 ) studied the convergence of GDA in the nonconvex-concave setting and Lin et al . ( 2020 ) ; Xu et al . ( 2020b ) studied the nonconvex-strongly-concave setting . In these general nonconvex settings , it has been shown that GDA converges to a certain stationary point at a sublinear rate , i.e. , ‖G ( xt ) ‖ ≤ t−α for some α > 0 , where G ( xt ) corresponds to a certain notion of gradient . Although such a gradient convergence result implies the stability of the algorithm , namely , limt→∞ ‖xt+1 − xt‖ = 0 , it does not guarantee the convergence of the variable sequences { xt } t , { yt } t generated by GDA . So far , the variable convergence of GDA has not been established for nonconvex problems , but only under ( strongly ) convex function geometries that are mentioned previously Du and Hu ( 2019 ) ; Mokhtari et al . ( 2020 ) ; Zhang and Wang ( 2020 ) . Therefore , we want to ask the following fundamental question : • Q1 : Does GDA have guaranteed variable convergence in nonconvex minimax optimization ? If so , where do they converge to ? In fact , proving the variable convergence of GDA in the nonconvex setting is highly nontrivial due to the following reasons : 1 ) the algorithm alternates between a minimization step and a maximization step ; 2 ) It is well understood that strong global function geometry leads to the convergence of GDA . However , in general nonconvex setting , the objective functions typically do not have an amenable global geometry . Instead , they may satisfy different types of local geometries around the critical points . Hence , it is natural and much desired to exploit the local geometries of functions in analyzing the convergence of GDA . The Kurdyka-Łojasiewicz ( KŁ ) geometry provides a broad characterization of such local geometries for nonconvex functions . The Kurdyka-Łojasiewicz ( KŁ ) geometry ( see Section 2 for details ) Bolte et al . ( 2007 ; 2014 ) parameterizes a broad spectrum of the local nonconvex geometries and has been shown to hold for a broad class of practical functions . Moreover , it also generalizes other global geometries such as strong convexity and PŁ geometry . In the existing literature , the KŁ geometry has been exploited extensively to analyze the convergence rate of various gradient-based algorithms in nonconvex optimization , e.g. , gradient descent Attouch and Bolte ( 2009 ) ; Li et al . ( 2017 ) and its accelerated version Zhou et al . ( 2020 ) as well as the distributed version Zhou et al . ( 2016a ) . Hence , we are highly motivated to study the convergence rate of variable convergence of GDA in nonconvex minimax optimization under the KŁ geometry . In particular , we want to address the following question : • Q2 : How does the local function geometry captured by the KŁ parameter affects the variable convergence rate of GDA ? In this paper , we provide comprehensive answers to these questions . We develop a new analysis framework to study the variable convergence of GDA in nonconvex-strongly-concave minimax optimization under the KŁ geometry . We also characterize the convergence rates of GDA in the full spectrum of the parameterization of the KŁ geometry . 1.1 OUR CONTRIBUTIONS . We consider the following regularized nonconvex-strongly-concave minimax optimization problem min x∈Rm max y∈Y f ( x , y ) + g ( x ) − h ( y ) , ( P ) where f is a differentiable and nonconvex-strongly-concave function , g is a general nonconvex regularizer and h is a convex regularizer . Both g and h can be possibly nonsmooth . To solve the above regularized minimax problem , we study a proximal-GDA algorithm that leverages the forward-backward splitting update Lions and Mercier ( 1979 ) ; Attouch et al . ( 2013 ) . We study the variable convergence property of proximal-GDA in solving the minimax problem ( P ) . Specifically , we show that proximal-GDA admits a novel Lyapunov function H ( x , y ) ( see Proposition 2 ) , which is monotonically decreasing along the trajectory of proximal GDA , i.e. , H ( xt+1 , yt+1 ) < H ( xt , yt ) . Based on the monotonicity of this Lyapunov function , we show that every limit point of the variable sequences generated by proximal-GDA is a critical point of the objective function . Moreover , by exploiting the ubiquitous KŁ geometry of the Lyapunov function , we prove that the entire variable sequence of proximal-GDA has a unique limit point , or equivalently speaking , it converges to a certain critical point x∗ , i.e. , xt → x∗ , yt → y∗ ( x∗ ) ( see the definition of y∗ in Section 2 ) . To the best of our knowledge , this is the first variable convergence result of GDA-type algorithms in nonconvex minimax optimization . Furthermore , we characterize the asymptotic convergence rates of both the variable sequences and the function values of proximal-GDA in different parameterization regimes of the KŁ geometry . Depending on the value of the KŁ parameter θ , we show that proximal-GDA achieves different types of convergence rates ranging from sublinear convergence up to finite-step convergence , as we summarize in Table 1 below . 1.2 RELATED WORK . Deterministic GDA algorithms : Yang et al . ( 2020 ) studied an alternating gradient descent-ascent ( AGDA ) algorithm in which the gradient ascent step uses the current variable xt+1 instead of xt . Boţ and Böhm ( 2020 ) extended the AGDA algorithm to an alternating proximal-GDA ( APGDA ) algorithm for a regularized minimax optimization . Xu et al . ( 2020b ) studied an alternating gradient projection algorithm which applies ` 2 regularizer to the local objective function of GDA followed by projection onto the constraint sets . Daskalakis and Panageas ( 2018 ) ; Mokhtari et al . ( 2020 ) ; Zhang and Wang ( 2020 ) analyzed optimistic gradient descent-ascent ( OGDA ) which applies negative momentum to accelerate GDA . Mokhtari et al . ( 2020 ) also studied an extra-gradient algorithm which applies two-step GDA in each iteration . Nouiehed et al . ( 2019 ) studied multi-step GDA where multiple gradient ascent steps are performed , and they also studied the momentum-accelerated version . Cherukuri et al . ( 2017 ) ; Daskalakis and Panageas ( 2018 ) ; Jin et al . ( 2020 ) studied GDA in continuous time dynamics using differential equations . Adolphs et al . ( 2019 ) analyzed a second-order variant of the GDA algorithm . Stochastic GDA algorithms : Lin et al . ( 2020 ) ; Yang et al . ( 2020 ) ; Boţ and Böhm ( 2020 ) analyzed stochastic GDA , stochastic AGDA and stochastic APGDA , which are direct extensions of GDA , AGDA and APGDA to the stochastic setting respectively . Variance reduction techniques have been applied to stochastic minimax optimization , including SVRG-based Du and Hu ( 2019 ) ; Yang et al . ( 2020 ) , SPIDER-based Xu et al . ( 2020a ) , STORM Qiu et al . ( 2020 ) and its gradient free version Huang et al . ( 2020 ) . Xie et al . ( 2020 ) studied the complexity lower bound of first-order stochastic algorithms for finite-sum minimax problem . KŁ geometry : The KŁ geometry was defined in Bolte et al . ( 2007 ) . The KŁ geometry has been exploited to study the convergence of various first-order algorithms for solving minimization problems , including gradient descent Attouch and Bolte ( 2009 ) , alternating gradient descent Bolte et al . ( 2014 ) , distributed gradient descent Zhou et al . ( 2016a ; 2018a ) , accelerated gradient descent Li et al . ( 2017 ) . It has also been exploited to study the convergence of second-order algorithms such as Newton ’ s method Noll and Rondepierre ( 2013 ) ; Frankel et al . ( 2015 ) and cubic regularization method Zhou et al . ( 2018b ) . 2 PROBLEM FORMULATION AND KŁ GEOMETRY . In this section , we introduce the problem formulation , technical assumptions and the KurdykaŁojasiewicz ( KŁ ) geometry . We consider the following regularized minimax optimization problem . min x∈Rm max y∈Y f ( x , y ) + g ( x ) − h ( y ) , ( P ) where f : Rm × Rn → R is a differentiable and nonconvex-strongly-concave loss function , Y ⊂ Rn is a compact and convex set , and g , h are the regularizers that are possibly non-smooth . In particular , define Φ ( x ) : = maxy∈Y f ( x , y ) − h ( y ) , and then the problem ( P ) is equivalent to the minimization problem minx∈Rm Φ ( x ) + g ( x ) . Throughout the paper , we adopt the following standard assumptions on the problem ( P ) . Assumption 1 . The objective function of the problem ( P ) satisfies : 1 . Function f ( · , · ) is L-smooth and function f ( x , · ) is µ-strongly concave ; 2 . Function ( Φ + g ) ( x ) is bounded below , i.e. , infx∈Rm ( Φ + g ) ( x ) > −∞ ; 3 . For any α ∈ R , the sub-level set { x : ( Φ + g ) ( x ) ≤ α } is compact ; 4 . Function h is proper and convex , and function g is proper and lower semi-continuous . To elaborate , item 1 considers the class of nonconvex-strongly-concave functions f that has been widely studied in the existing literature Lin et al . ( 2020 ) ; Jin et al . ( 2020 ) ; Xu et al . ( 2020b ; a ) ; Lu et al . ( 2020 ) . Items 2 and 3 guarantee that the minimax problem ( P ) has at least one solution , and the variable sequences generated by the proximal-GDA algorithm ( See Algorithm 1 ) are bounded . Item 4 requires the regularizer h to be convex ( possibly non-smooth ) , which includes many normbased popular regularizers such as ` p ( p ≥ 1 ) , elastic net , nuclear norm , spectral norm , etc . On the other hand , the other regularizer g can be nonconvex but lower semi-continuous , which includes all the aforementioned convex regularizers , ` p ( 0 ≤ p < 1 ) , Schatten-p norm , rank , etc . Hence , our formulation of the problem ( P ) covers a rich class of nonconvex objective functions and regularizers and is more general than the existing nonconvex minimax formulation in Lin et al . ( 2020 ) , which does not consider any regularizer . Remark 1 . We note that the strong concavity of f ( x , · ) in item 1 can be relaxed to concavity , provided that the regularizer h ( y ) is µ-strongly convex . In this case , we can add −µ2 ‖y‖ 2 to both f ( x , y ) and h ( y ) such that Assumption 1 still holds . For simplicity , we will omit the discussion on this case . By strong concavity of f ( x , · ) , it is clear that the mapping y∗ ( x ) : = arg maxy∈Y f ( x , y ) − h ( y ) is uniquely defined for every x ∈ Rm . In particular , if x∗ is the desired minimizer of Φ ( x ) , then ( x∗ , y∗ ( x∗ ) ) is the desired solution of the minimax problem ( P ) . Next , we present some important properties regarding the function Φ ( x ) and the mapping y∗ ( x ) . The following proposition from Boţ and Böhm ( 2020 ) generalizes the Lemma 4.3 of Lin et al . ( 2020 ) to the regularized setting . The proof can be found in Appendix A . Throughout , we denote κ = L/µ as the condition number and denote∇1f ( x , y ) , ∇2f ( x , y ) as the gradients with respect to the first and the second input argument , respectively . For example , with this notation , ∇1f ( x , y∗ ( x ) ) denotes the gradient of f ( x , y∗ ( x ) ) with respect to only the first input argument x , and the x in the second input argument y∗ ( x ) is treated as a constant . Proposition 1 ( Lipschitz continuity of y∗ ( x ) and∇Φ ( x ) ) . Let Assumption 1 hold . Then , the mapping y∗ ( x ) and the function Φ ( x ) satisfy 1 . Mapping y∗ ( x ) is κ-Lipschitz continuous ; 2 . Function Φ ( x ) is L ( 1 + κ ) -smooth with∇Φ ( x ) = ∇1f ( x , y∗ ( x ) ) . As an intuitive explanation of Proposition 1 , since the function f ( x , y ) − h ( y ) is L-smooth with respect to x , both the maximizer y∗ ( x ) and the corresponding maximum function value Φ ( x ) should not change substantially with regard to a small change of x . Recall that the minimax problem ( P ) is equivalent to the standard minimization problem minx∈Rm Φ ( x ) + g ( x ) , which , according to item 2 of Proposition 1 , includes a smooth nonconvex function Φ ( x ) and a lower semi-continuous regularizer g ( x ) . Hence , we can define the optimization goal of the minimax problem ( P ) as finding a critical point x∗ of the nonconvex function Φ ( x ) + g ( x ) that satisfies the necessary optimality condition 0 ∈ ∂ ( Φ + g ) ( x∗ ) for minimizing nonconvex functions . Here , ∂ denotes the notion of subdifferential as we elaborate below . Definition 1 . ( Subdifferential and critical point , Rockafellar and Wets ( 2009 ) ) The Frechét subdifferential ∂̂h of function h at x ∈ domh is the set of u ∈ Rd defined as ∂̂h ( x ) : = { u : lim inf z 6=x , z→x h ( z ) − h ( x ) − uᵀ ( z − x ) ‖z − x‖ ≥ 0 } , and the limiting subdifferential ∂h at x ∈ domh is the graphical closure of ∂̂h defined as : ∂h ( x ) : = { u : ∃xk → x , h ( xk ) → h ( x ) , uk ∈ ∂̂h ( xk ) , uk → u } . The set of critical points of h is defined as crith : = { x : 0 ∈ ∂h ( x ) } . Throughout , we refer to the limiting subdifferential as subdifferential . We note that subdifferential is a generalization of gradient ( when h is differentiable ) and subgradient ( when h is convex ) to the nonconvex setting . In particular , any local minimizer of h must be a critical point . Next , we introduce the Kurdyka-Łojasiewicz ( KŁ ) geometry of a function h. Throughout , the point-to-set distance is denoted as distΩ ( x ) : = infu∈Ω ‖x− u‖ . Definition 2 ( KŁ geometry , Bolte et al . ( 2014 ) ) . A proper and lower semi-continuous function h is said to have the KŁ geometry if for every compact set Ω ⊂ domh on which h takes a constant value hΩ ∈ R , there exist ε , λ > 0 such that for all x̄ ∈ Ω and all x ∈ { z ∈ Rm : distΩ ( z ) < ε , hΩ < h ( z ) < hΩ + λ } , the following condition holds : ϕ′ ( h ( x ) − hΩ ) · dist∂h ( x ) ( 0 ) ≥ 1 , ( 1 ) where ϕ′ is the derivative of function ϕ : [ 0 , λ ) → R+ , which takes the form ϕ ( t ) = cθ t θ for certain universal constant c > 0 and KŁ parameter θ ∈ ( 0 , 1 ] . The KŁ geometry characterizes the local geometry of a nonconvex function around the set of critical points . To explain , consider the case where h is a differentiable function so that ∂h ( x ) = ∇h ( x ) . Then , the KŁ inequality in eq . ( 1 ) becomes h ( x ) − hΩ ≤ O ( ‖∇h ( x ) ‖ 1 1−θ ) , which generalizes the Polyak-Łojasiewicz ( PL ) condition h ( x ) − hΩ ≤ O ( ‖∇h ( x ) ‖2 ) Łojasiewicz ( 1963 ) ; Karimi et al . ( 2016 ) ( i.e. , KŁ parameter θ = 12 ) . Moreover , the KŁ geometry has been shown to hold for a large class of functions including sub-analytic functions , logarithm and exponential functions and semi-algebraic functions . These function classes cover most of the nonconvex objective functions encountered in practical machine learning applications Zhou et al . ( 2016b ) ; Yue et al . ( 2018 ) ; Zhou and Liang ( 2017 ) ; Zhou et al . ( 2018b ) . The KŁ geometry has been exploited extensively to analyze the convergence of various first-order algorithms , e.g. , gradient descent Attouch and Bolte ( 2009 ) ; Li et al . ( 2017 ) , alternating minimization Bolte et al . ( 2014 ) and distributed gradient methods Zhou et al . ( 2016a ) . It has also been exploited to study the convergence of second-order algorithms such cubic regularization Zhou et al . ( 2018b ) . In these works , it has been shown that the variable sequences generated by these algorithms converge to a desired critical point in nonconvex optimization , and the convergence rates critically depend on the parameterization θ of the KŁ geometry . In the subsequent sections , we provide a comprehensive understanding of the convergence and convergence rate of proximal-GDA under the KŁ geometry .

Summary: In this paper, the authors analyze the convergence of a proximal gradient descent ascent (GDA) method when applied to non-convex strongly concave functions. To establish convergence results, the authors show that proximal-GDA admits a novel Lyapunov function that monotonically decreases at every iteration. Along with KL-parametrized local geometry, the Lyapunov function was used to establish the convergence of decision variables to a critical point. Moreover, the rate of convergence of the algorithm was computed for various ranges of KL-parameter.