GDA-AM: ON THE EFFECTIVENESS OF SOLVING MIN-IMAX OPTIMIZATION VIA ANDERSON MIXING

1 INTRODUCTION . Minimax optimization has received a surge of interest due to its wide range of applications in modern machine learning , such as generative adversarial networks ( GAN ) , adversarial training and multiagent reinforcement learning ( Goodfellow et al. , 2014 ; Madry et al. , 2018 ; Li et al. , 2019 ) . Formally , given a bivariate function f ( x , y ) , the objective is to find a stable solution where the players can not improve their objective , i.e. , to find the Nash equilibrium of the underlying game ( von Neumann & Morgenstern , 1944 ) : arg min x∈X arg max y∈Y f ( x , y ) . ( 1 ) It is commonplace to use simple algorithms such as gradient descent ascent ( GDA ) to solve such problems , where both players take a gradient update simultaneously or alternatively . Despite its simplicity , GDA is known to suffer from a generic issue for minimax optimization : it may cycle around a stable point , exhibit divergent behavior , or converge very slowly since it requires very small learning rates ( Gidel et al. , 2019a ; Mertikopoulos et al. , 2019 ) . Given the widespread usage of gradient-based methods for solving machine learning problems , first-order optimization algorithms to solve minimax problems have gained considerable popularity in the last few years . Algorithms such as optimistic Gradient Descent Ascent ( OG ) ( Daskalakis et al. , 2018 ; Mertikopoulos et al. , 2019 ) and extra-gradient ( EG ) ( Gidel et al. , 2019a ) can alleviate the issue of GDA for some problems . Yet , it has been shown that these methods can still diverge or cycle around a stable point ( Adolphs et al . ; Mazumdar et al. , 2019 ; Parker-Holder et al. , 2020 ) . For example , these algorithms even fail to find a local minimax ( the set of local minimax is a superset of local Nash ( Jin et al. , 2020 ; Wang et al. , 2020 ) ) as shown in Figure 1 . This leads to the following question : Can we design better algorithms for minimax problems ? We answer this in the affirmative , by introducing GDA-AM . We cast the GDA dynamics as a fixed-point iteration problem and compute the iterates effectively using an advanced nonlinear extrapolation method . We show that indeed our algorithm has theoretical and empirical guarantees across a broad range of minimax problems , including GANs . Our contributions : In this paper , we propose a different approach to solve minimax optimization . Our starting point is to cast the GDA dynamics as a fixed-point iteration . We then highlight that the fixed-point iteration can be solved effectively by using advanced non-linear extrapolation methods such as Anderson Mixing ( Anderson , 1965 ) , which we name as GDA-AM . Although first mentioned in Azizian et al . ( 2020 ) , to our best knowledge , this is still the first work to investigate and improve the GDA dynamics by tapping into advanced fixed-point algorithms . We demonstrate that GDA dynamics can benefit from Anderson Mixing . In particular , we study bilinear games and give a systematic analysis of GDA-AM for both simultaneous and alternating versions of GDA . We theoretically show that GDA-AM can achieve global convergence guarantees under mild conditions . We complement our theoretical results with numerical simulations across a variety of minimax problems . We show that for some convex-concave and non-convex-concave functions , GDA-AM can converge to the optimal point with little hyper-parameter tuning whereas existing first-order methods are prone to divergence and cycling behaviors . We also provide empirical results for GAN training across two different datasets , CIFAR10 and CelebA . Given the limited computational overhead of our method , the results suggest that an extrapolation add-on to GDA can lead to significant performance gains . Moreover , the convergence behavior across a variety of problems and the ease-of-use demonstrate the potential of GDA-AM to become the minimax optimization workhorse . 2 PRELIMINARIES AND BACKGROUND . 2.1 MINIMAX OPTIMIZATION . Definition 1 . Point ( x∗ , y∗ ) is a local Nash equilibrium of f if there exists δ > 0 such that for any ( x , y ) satisfying ‖x− x∗‖ ≤ δ and ‖y − y∗‖ ≤ δ we have : f ( x∗ , y ) ≤ f ( x∗ , y∗ ) ≤ f ( x , y∗ ) . To find the Nash equilibria , common algorithms including GDA , EG and OG , can be formulated as follows . For the two variants of GDA , simultaneous GDA ( SimGDA ) and alternating GDA ( AltGDA ) , the updates have the following forms : Simultaneous : xt+1 = xt − η∇xf ( xt , yt ) , yt+1 = yt + η∇yf ( xt , yt ) Alternating : xt+1 = xt − η∇xf ( xt , yt ) , yt+1 = yt + η∇yf ( xt+1 , yt ) . ( 2 ) The EG update has the following form : xt+ 12 = xt − η∇xf ( xt , yt ) , yt+ 12 = yt + η∇yf ( xt , yt ) xt+1 = xt − η∇xf ( xt+ 12 , yt+ 12 ) , yt+1 = yt + η∇yf ( xt+ 12 , yt+ 12 ) . ( 3 ) The OG update has the following form : xt+1 = xt−η∇xf ( xt , yt ) + η 2 ∇xf ( xt−1 , yt−1 ) , yt+1 = yt+η∇yf ( xt , yt ) − η 2 ∇yf ( xt−1 , yt−1 ) . ( 4 ) 2.2 FIXED-POINT ITERATION AND ANDERSON MIXING ( AM ) . Definition 2. w ? is a fixed point of the mapping g if w ? = g ( w ? ) . Consider the simple fixed-point iteration wt+1 = g ( wt ) which produces a sequence of iterates { w0 , w1 , · · · , wN } . In most cases , this converges to the fixed-point , w∗ = g ( w∗ ) . Take gradient descent as an example , it can be viewed as iteratively applying the operation : wt+1 = g ( wt ) , wt−αt∇f ( wt ) , where the limit is the fixed-point w ? = g ( w ? ) ( i.e.∇f ( wt = 0 ) . SimGDA updates can be defined as the repeated application of a nonlinear operator : wt+1 = G ( sim ) η ( wt ) , wt − ηV ( wt ) with w = [ x y ] , V ( w ) = [ ∇xf ( x , y ) −∇yf ( x , y ) ] Similarly , we can write AltGDA updates as wt+1 = G ( alt ) η ( wt ) . An issue with fixed-point iteration is that it does not always converge , and even in the cases where it does converge , it might do so very slowly . GDA is one example that it could result in the possibility of the operator converging to a limit cycle instead of a single point for the GDA dynamic . A way of dealing with these problems is to use acceleration methods , which can potentially speed up the convergence process and in some cases even decrease the likelihood for divergence . There are many different acceleration methods , but we will put our focus on an algorithm which we refer to as Anderson Mixing ( or Anderson Acceleration ) . In short , Anderson Mixing ( AM ) shares the same idea as Nesterov ’ s acceleration . Given a fixed-point iteration wt = g ( wt−1 ) , Anderson Mixing argues that a good approximation to the final solution w∗ can be obtained as a linear combination of the previous p iterates wt+1 = ∑p i=0 βig ( wt−pt+i ) . Since obtaining the proper coefficients βi is a nonlinear procedure , Anderson Mixing is also known as a nonlinear extrapolation method . The general form of Anderson Mixing is shown in Algorithm 1 . For efficiency , we prefer a ‘ restarted ’ version with a small table size p that cleans up the table F every p iterations because it avoids solving a linear system of increasing size . Algorithm 1 : Anderson Mixing Prototype ( truncated version ) Input : Initial point w0 , Anderson restart dimension p , fixed-point mapping g : Rn → Rn . Output : wt+1 for t = 0 , 1 , . . . do Set pt = min { t , p } . Set Ft = [ ft−pt , . . . , ft ] , where fi = g ( wi ) − wi for each i ∈ [ t− pt . . t ] . Determine weights β = ( β0 , . . . , βpt ) T that solves minβ ‖Ftβ‖2 , s. t. ∑pt i=0 βi = 1 . Set wt+1 = ∑pt i=0 βig ( wt−pt+i ) . end 2.3 AM AND GENERALIZED MINIMAL RESIDUAL ( GMRES ) . Developed by Saad & Schultz ( 1986 ) , Generalized Minimal Residual method ( GMRES ) is a Krylov subspace method for solving linear system equations . The method approximates the solution by the vector in a Krylov subspace with minimal residual , which is described below . Definition 3 . Assume we have the linear system of equations Ax = b with A ∈ Rn×n , b ∈ Rn and an initial guess x0 . Then we denote the initial residual by r0 = b−Ax0 and define the tth Krylov subspace as Kt = span { r0 , Ar0 , · · · , At−1r0 } . The tth iterate xt of GMRES minimizes the norm of the residual rt = b −Axt in Kt , that is , xt solves min xt∈x0+Kt ‖b−Axt‖2 . The following formulation is equivalent to GMRES minimization problem and more convenient for implementation . It computes x̂t such that x̂t = arg min x̂t∈Kt ‖b−A ( x0 + x̂t ) ‖2 = arg min x̂t∈Kt ‖r0 −Ax̂t‖2 . Using a larger Krylov dimension will improve the convergence of the method , but will require more memory . For this reason , a smaller Krylov subspace dimension t and ‘ restarted ’ versions of the method are used in practice Saad ( 2003 ) . The convergence of GMRES can be studied through the magnitude of the residual polynomial . Theorem 2.1 ( Lemma 6.31 of Saad ( 2003 ) ) . Let x̂t be the approximate solution obtained at the t-th iteration of GMRES being applied to solve Ax = b , and denote the residual as rt = b−Ax̂t . Then , rt is of the form rt = ft ( A ) r0 , ( 5 ) where ‖rt‖2 = ‖ft ( A ) r0‖2 = min ft∈Pt ‖ft ( A ) r0‖2 , ( 6 ) where Pp is the family of polynomials with degree p such that fp ( 0 ) = 1 , ∀fp ∈ Pp , which are usually called residual polynomials . Although GMRES is applied to a system of linear equations not a fixed-point problem , there is a strong connection between Anderson Mixing and GMRES . In AM we are looking for a fixed-point x such that Gx− b− x = 0 and by rearranging this equation we get b + ( G− I ) x = 0⇔ ( I−G ) x = b. Theorem 2.2 shows that if GMRES is applied to the system ( I −G ) x = b and AM is applied to g ( x ) = Gx + b with the same initial guess and I−G is non-singular , then these are equivalent in the sense that the iterates of each algorithm can be obtained directly from the iterates of the other algorithm . Theorem 2.2 ( Equivalence between AM with restart and GMRES ( Walker & Ni , 2011a ) ) . Consider the fixed point iteration x = g ( x ) where g ( x ) = Gx + b for G ∈ Rn×n and b ∈ Rn . If I −G is non-singular , Algorithm 1 produces exactly the same iterates as GMRES being applied to solve ( I−G ) x = b when both algorithms start with the same initial guess . Theorem 2.2 can also be generalized to the restart version of AM an GMRES as well . 3 GDA-AM : GDA WITH ANDERSON MIXING We propose a novel minimax optimizer , called GDA-AM , that is inspired by recent advances in parameter ( or weight ) averaging ( Wu et al. , 2020 ; Yazici et al. , 2019 ) . We argue that a nonlinear adaptive average ( combination ) is a more appropriate choice for minimax optimization .

Summary: Minimax optimization is a central problem in machine learning. The gradient descent ascent method is the most commonly used algorithm to solve this problem. This paper views the gradient descent ascent method as a fixed-point iteration and solves it using Anderson Mixing to converge to the local minimax. They show theoretically that the algorithm can achieve global convergence for bilinear problems under mild conditions. Some numerical experiments have been conducted.