GDA-AM: ON THE EFFECTIVENESS OF SOLVING MIN-IMAX OPTIMIZATION VIA ANDERSON MIXING

1 INTRODUCTION . Minimax optimization has received a surge of interest due to its wide range of applications in modern machine learning , such as generative adversarial networks ( GAN ) , adversarial training and multiagent reinforcement learning ( Goodfellow et al. , 2014 ; Madry et al. , 2018 ; Li et al. , 2019 ) . Formally , given a bivariate function f ( x , y ) , the objective is to find a stable solution where the players can not improve their objective , i.e. , to find the Nash equilibrium of the underlying game ( von Neumann & Morgenstern , 1944 ) : arg min x∈X arg max y∈Y f ( x , y ) . ( 1 ) It is commonplace to use simple algorithms such as gradient descent ascent ( GDA ) to solve such problems , where both players take a gradient update simultaneously or alternatively . Despite its simplicity , GDA is known to suffer from a generic issue for minimax optimization : it may cycle around a stable point , exhibit divergent behavior , or converge very slowly since it requires very small learning rates ( Gidel et al. , 2019a ; Mertikopoulos et al. , 2019 ) . Given the widespread usage of gradient-based methods for solving machine learning problems , first-order optimization algorithms to solve minimax problems have gained considerable popularity in the last few years . Algorithms such as optimistic Gradient Descent Ascent ( OG ) ( Daskalakis et al. , 2018 ; Mertikopoulos et al. , 2019 ) and extra-gradient ( EG ) ( Gidel et al. , 2019a ) can alleviate the issue of GDA for some problems . Yet , it has been shown that these methods can still diverge or cycle around a stable point ( Adolphs et al . ; Mazumdar et al. , 2019 ; Parker-Holder et al. , 2020 ) . For example , these algorithms even fail to find a local minimax ( the set of local minimax is a superset of local Nash ( Jin et al. , 2020 ; Wang et al. , 2020 ) ) as shown in Figure 1 . This leads to the following question : Can we design better algorithms for minimax problems ? We answer this in the affirmative , by introducing GDA-AM . We cast the GDA dynamics as a fixed-point iteration problem and compute the iterates effectively using an advanced nonlinear extrapolation method . We show that indeed our algorithm has theoretical and empirical guarantees across a broad range of minimax problems , including GANs . Our contributions : In this paper , we propose a different approach to solve minimax optimization . Our starting point is to cast the GDA dynamics as a fixed-point iteration . We then highlight that the fixed-point iteration can be solved effectively by using advanced non-linear extrapolation methods such as Anderson Mixing ( Anderson , 1965 ) , which we name as GDA-AM . Although first mentioned in Azizian et al . ( 2020 ) , to our best knowledge , this is still the first work to investigate and improve the GDA dynamics by tapping into advanced fixed-point algorithms . We demonstrate that GDA dynamics can benefit from Anderson Mixing . In particular , we study bilinear games and give a systematic analysis of GDA-AM for both simultaneous and alternating versions of GDA . We theoretically show that GDA-AM can achieve global convergence guarantees under mild conditions . We complement our theoretical results with numerical simulations across a variety of minimax problems . We show that for some convex-concave and non-convex-concave functions , GDA-AM can converge to the optimal point with little hyper-parameter tuning whereas existing first-order methods are prone to divergence and cycling behaviors . We also provide empirical results for GAN training across two different datasets , CIFAR10 and CelebA . Given the limited computational overhead of our method , the results suggest that an extrapolation add-on to GDA can lead to significant performance gains . Moreover , the convergence behavior across a variety of problems and the ease-of-use demonstrate the potential of GDA-AM to become the minimax optimization workhorse . 2 PRELIMINARIES AND BACKGROUND . 2.1 MINIMAX OPTIMIZATION . Definition 1 . Point ( x∗ , y∗ ) is a local Nash equilibrium of f if there exists δ > 0 such that for any ( x , y ) satisfying ‖x− x∗‖ ≤ δ and ‖y − y∗‖ ≤ δ we have : f ( x∗ , y ) ≤ f ( x∗ , y∗ ) ≤ f ( x , y∗ ) . To find the Nash equilibria , common algorithms including GDA , EG and OG , can be formulated as follows . For the two variants of GDA , simultaneous GDA ( SimGDA ) and alternating GDA ( AltGDA ) , the updates have the following forms : Simultaneous : xt+1 = xt − η∇xf ( xt , yt ) , yt+1 = yt + η∇yf ( xt , yt ) Alternating : xt+1 = xt − η∇xf ( xt , yt ) , yt+1 = yt + η∇yf ( xt+1 , yt ) . ( 2 ) The EG update has the following form : xt+ 12 = xt − η∇xf ( xt , yt ) , yt+ 12 = yt + η∇yf ( xt , yt ) xt+1 = xt − η∇xf ( xt+ 12 , yt+ 12 ) , yt+1 = yt + η∇yf ( xt+ 12 , yt+ 12 ) . ( 3 ) The OG update has the following form : xt+1 = xt−η∇xf ( xt , yt ) + η 2 ∇xf ( xt−1 , yt−1 ) , yt+1 = yt+η∇yf ( xt , yt ) − η 2 ∇yf ( xt−1 , yt−1 ) . ( 4 ) 2.2 FIXED-POINT ITERATION AND ANDERSON MIXING ( AM ) . Definition 2. w ? is a fixed point of the mapping g if w ? = g ( w ? ) . Consider the simple fixed-point iteration wt+1 = g ( wt ) which produces a sequence of iterates { w0 , w1 , · · · , wN } . In most cases , this converges to the fixed-point , w∗ = g ( w∗ ) . Take gradient descent as an example , it can be viewed as iteratively applying the operation : wt+1 = g ( wt ) , wt−αt∇f ( wt ) , where the limit is the fixed-point w ? = g ( w ? ) ( i.e.∇f ( wt = 0 ) . SimGDA updates can be defined as the repeated application of a nonlinear operator : wt+1 = G ( sim ) η ( wt ) , wt − ηV ( wt ) with w = [ x y ] , V ( w ) = [ ∇xf ( x , y ) −∇yf ( x , y ) ] Similarly , we can write AltGDA updates as wt+1 = G ( alt ) η ( wt ) . An issue with fixed-point iteration is that it does not always converge , and even in the cases where it does converge , it might do so very slowly . GDA is one example that it could result in the possibility of the operator converging to a limit cycle instead of a single point for the GDA dynamic . A way of dealing with these problems is to use acceleration methods , which can potentially speed up the convergence process and in some cases even decrease the likelihood for divergence . There are many different acceleration methods , but we will put our focus on an algorithm which we refer to as Anderson Mixing ( or Anderson Acceleration ) . In short , Anderson Mixing ( AM ) shares the same idea as Nesterov ’ s acceleration . Given a fixed-point iteration wt = g ( wt−1 ) , Anderson Mixing argues that a good approximation to the final solution w∗ can be obtained as a linear combination of the previous p iterates wt+1 = ∑p i=0 βig ( wt−pt+i ) . Since obtaining the proper coefficients βi is a nonlinear procedure , Anderson Mixing is also known as a nonlinear extrapolation method . The general form of Anderson Mixing is shown in Algorithm 1 . For efficiency , we prefer a ‘ restarted ’ version with a small table size p that cleans up the table F every p iterations because it avoids solving a linear system of increasing size . Algorithm 1 : Anderson Mixing Prototype ( truncated version ) Input : Initial point w0 , Anderson restart dimension p , fixed-point mapping g : Rn → Rn . Output : wt+1 for t = 0 , 1 , . . . do Set pt = min { t , p } . Set Ft = [ ft−pt , . . . , ft ] , where fi = g ( wi ) − wi for each i ∈ [ t− pt . . t ] . Determine weights β = ( β0 , . . . , βpt ) T that solves minβ ‖Ftβ‖2 , s. t. ∑pt i=0 βi = 1 . Set wt+1 = ∑pt i=0 βig ( wt−pt+i ) . end 2.3 AM AND GENERALIZED MINIMAL RESIDUAL ( GMRES ) . Developed by Saad & Schultz ( 1986 ) , Generalized Minimal Residual method ( GMRES ) is a Krylov subspace method for solving linear system equations . The method approximates the solution by the vector in a Krylov subspace with minimal residual , which is described below . Definition 3 . Assume we have the linear system of equations Ax = b with A ∈ Rn×n , b ∈ Rn and an initial guess x0 . Then we denote the initial residual by r0 = b−Ax0 and define the tth Krylov subspace as Kt = span { r0 , Ar0 , · · · , At−1r0 } . The tth iterate xt of GMRES minimizes the norm of the residual rt = b −Axt in Kt , that is , xt solves min xt∈x0+Kt ‖b−Axt‖2 . The following formulation is equivalent to GMRES minimization problem and more convenient for implementation . It computes x̂t such that x̂t = arg min x̂t∈Kt ‖b−A ( x0 + x̂t ) ‖2 = arg min x̂t∈Kt ‖r0 −Ax̂t‖2 . Using a larger Krylov dimension will improve the convergence of the method , but will require more memory . For this reason , a smaller Krylov subspace dimension t and ‘ restarted ’ versions of the method are used in practice Saad ( 2003 ) . The convergence of GMRES can be studied through the magnitude of the residual polynomial . Theorem 2.1 ( Lemma 6.31 of Saad ( 2003 ) ) . Let x̂t be the approximate solution obtained at the t-th iteration of GMRES being applied to solve Ax = b , and denote the residual as rt = b−Ax̂t . Then , rt is of the form rt = ft ( A ) r0 , ( 5 ) where ‖rt‖2 = ‖ft ( A ) r0‖2 = min ft∈Pt ‖ft ( A ) r0‖2 , ( 6 ) where Pp is the family of polynomials with degree p such that fp ( 0 ) = 1 , ∀fp ∈ Pp , which are usually called residual polynomials . Although GMRES is applied to a system of linear equations not a fixed-point problem , there is a strong connection between Anderson Mixing and GMRES . In AM we are looking for a fixed-point x such that Gx− b− x = 0 and by rearranging this equation we get b + ( G− I ) x = 0⇔ ( I−G ) x = b. Theorem 2.2 shows that if GMRES is applied to the system ( I −G ) x = b and AM is applied to g ( x ) = Gx + b with the same initial guess and I−G is non-singular , then these are equivalent in the sense that the iterates of each algorithm can be obtained directly from the iterates of the other algorithm . Theorem 2.2 ( Equivalence between AM with restart and GMRES ( Walker & Ni , 2011a ) ) . Consider the fixed point iteration x = g ( x ) where g ( x ) = Gx + b for G ∈ Rn×n and b ∈ Rn . If I −G is non-singular , Algorithm 1 produces exactly the same iterates as GMRES being applied to solve ( I−G ) x = b when both algorithms start with the same initial guess . Theorem 2.2 can also be generalized to the restart version of AM an GMRES as well . 3 GDA-AM : GDA WITH ANDERSON MIXING We propose a novel minimax optimizer , called GDA-AM , that is inspired by recent advances in parameter ( or weight ) averaging ( Wu et al. , 2020 ; Yazici et al. , 2019 ) . We argue that a nonlinear adaptive average ( combination ) is a more appropriate choice for minimax optimization .

Summary: This paper proposes a new optimization method for min-max optimization (That could actually be generalized to any variational inequality problem) based on Anderson Mixing. They prove the convergence of their algorithm in the case of bilinear min-max games for both alternated and simultaneous updates operator. They eventually try their algorithm on toy bilnear problems and GANs.