Two steps at a time --- taking GAN training in stride with Tseng's method

1 INTRODUCTION . Generative Adversarial Networks ( GANs ) ( Goodfellow et al. , 2014 ) have proven to be a powerful class of generative models , producing for example unseen realistic images . Two neural networks , called generator and discriminator , compete against each other in a game . In the special case of a zero sum game this task can be formulated as a minimax ( aka saddle point ) problem . Conventionally , GANs are trained using variants of ( stochastic ) Gradient Descent Ascent ( GDA ) which are known to exhibit oscillatory behavior and thus fail to converge even for simple bilinear saddle point problems , see Goodfellow ( 2016 ) . We therefore propose the use of methods with provable convergence guarantees for ( stochastic ) convex-concave minimax problems , even though GANs are well known to not warrant these properties . Along similar considerations an adaptation of the Extragradient method ( EG ) ( Korpelevich , 1976 ) for the training of GANs was suggested in Gidel et al . ( 2019 ) , whereas Daskalakis et al . ( 2018 ) ; Daskalakis & Panageas ( 2018 ) ; Liang & Stokes ( 2019 ) studied Optimistic Gradient Descent Ascent ( OGDA ) based on optimistic mirror descent ( Rakhlin & Sridharan , 2013a ; b ) . We however investigate the Forward-Backward-Forward ( FBF ) method ( Tseng , 1991 ) from monotone operator theory , which uses two gradient evaluations per update , similar to EG , in order to circumvent the aforementioned issues . Instead of trying to improve GAN performance via new architectures , loss functions , etc. , we contribute to the theoretical foundation of their training from the point of view of optimization . Contribution . Establishing the connection between GAN training and monotone inclusions motivates to use the FBF method , originally designed to solve this type of problems . This approach allows to naturally extend the constrained setting to a regularized one making use of the proximal operator . We also propose a variant of FBF reusing previous gradients to reduce the computational cost per iteration , which turns out to be a known method , related to OGDA . By developing a unifying scheme that captures FBF and a generalization of OGDA , we reveal a hitherto unknown connection . Using this approach we prove novel non asymptotic convergence statements in terms of the minimax gap for both methods in the context of saddle point problems . In the deterministic and stochastic setting we obtain rates of O ( 1/k ) and O ( 1/√k ) , respectively . Concluding , we highlight the relevance of our proposed method as well as the role of regularizers by showing empirical improvements in the training of Wasserstein GANs on the CIFAR10 dataset . Organization . This paper is structured as follows . In Section 2 we highlight the connection of GAN training and monotone inclusions and give an extensive review of methods with convergence guarantees for the latter . The main results as well as a precise definition of the measure of optimality are discussed in Section 3 . Concluding , Section 4 illustrates the empirical performance in the training of GANs as well as solving bilinear problems . 2 GAN TRAINING AS MONOTONE INCLUSION . The GAN objective was originally cast as a two-player zero-sum game between the discriminator Dy and the generator Gx ( Goodfellow et al. , 2014 ) given by min x max y Eρ∼q [ log ( Dy ( ρ ) ) ] + Eζ∼p [ log ( 1−Dy ( Gx ( ζ ) ) ) ] , exhibiting the aforementioned minimax structure . Due to problems with vanishing gradients in the training of such models , a successful alternative formulation called Wasserstein GAN ( WGAN ) ( Arjovsky et al. , 2017 ) has been proposed . In this case the minimization tries to reduce the Wasserstein distance between the true distribution q and the one learned by the generator . Reformulating this distance via the Kantorovich Rubinstein duality leads to an inner maximization over 1-Lipschitz functions which are approximated via neural networks , yielding the saddle point problem min x max y : ‖Dy‖Lip≤1 Eρ∼q [ Dy ( ρ ) ] − Eζ∼p [ Dy ( Gx ( ζ ) ) ] . 2.1 CONVEX-CONCAVE MINIMAX PROBLEMS . Due to the observations made in the previous paragraph we study the following abstract minimax problem min x∈Rd max y∈Rn Ψ ( x , y ) : = f ( x ) + Eξ∼Q [ Φ ( x , y ; ξ ) ] − h ( y ) , ( 1 ) where the convex-concave coupling function Φ ( x , y ) : = Eξ∼Q [ Φ ( x , y ; ξ ) ] , which hides the stochasticity for ease of notation , is differentiable with L-Lipschitz continuous gradient . The proper , convex and lower semicontinuous functions f : Rd → R ∪ { +∞ } and h : Rn → R ∪ { +∞ } act as regularizers . A solution of ( 1 ) is given by a so-called saddle point ( x∗ , y∗ ) fulfilling for all x and y Ψ ( x∗ , y ) ≤ Ψ ( x∗ , y∗ ) ≤ Ψ ( x , y∗ ) . In the context of two-player games this corresponds to a pair of strategies , where no player can be better off by changing just their own strategy . For the purpose of this motivating section , we will restrict ourselves for now to the special case of the deterministic constrained version of ( 1 ) , given by min x∈X max y∈Y Φ ( x , y ) , where f and h are given by indicator functions of closed convex sets X and Y , respectively . The indicator function δC of a set C is defined as δC ( z ) = 0 for z ∈ C and δC ( z ) = +∞ otherwise . 2.2 MINIMAX PROBLEMS AS MONOTONE INCLUSIONS . If the coupling function Φ is convex-concave and differentiable then solving ( 1 ) is equivalent to solving the first order optimality conditions which can be written as a so-called monotone inclusion with w = ( x , y ) ∈ Rm and m = d+ n , given by 0 ∈ F ( w ) +NΩ ( w ) . ( 2 ) The entities involved are F ( x , y ) : = ( ∇xΦ ( x , y ) , −∇yΦ ( x , y ) ) , ( 3 ) and the normal cone NΩ of the convex set Ω : = X × Y . The normal cone mapping is given by NΩ ( w ) = { v ∈ Rm : 〈v , w′ − w〉 ≤ 0 ∀w′ ∈ Ω } , for w ∈ Ω and NΩ ( w ) = ∅ for w /∈ Ω . Here , the operators F and NΩ satisfy well known properties from convex analysis ( Bauschke & Combettes , 2011 ) , in particular the first one is monotone ( and Lipschitz if ∇Φ is so ) whereas the latter one is maximal monotone . We call a , possibly set-valued , operator A from Rm to itself monotone if 〈u− u′ , z − z′〉 ≥ 0 ∀u ∈ A ( z ) , u′ ∈ A ( z′ ) . We say A is maximal monotone , if there exists no monotone operator A′ such that the graph of A is properly contained in the graph of A′ . Problems of type ( 2 ) have been studied thoroughly in convex optimization , with the most established solution methods being Extragradient ( Korpelevich , 1976 ) and Forward-Backward-Forward ( Tseng , 1991 ) . Both methods are known to generate sequences of iterates converging to a solution of ( 2 ) . Note that in the unconstrained setting ( i.e . if Ω is the entire space ) both of these algorithms even produce the same iterates . 2.3 SOLVING MONOTONE INCLUSIONS . The connection between monotone inclusions and saddle point problems is of course not new . The application of Extragradient ( EG ) to minimax problems has been studied in the seminal paper Nemirovski ( 2004 ) under the name of Mirror Prox and a convergence rate of O ( 1/k ) in terms of the function values has been proven . Even a stochastic version of the Mirror Prox algorithm has been studied in Juditsky et al . ( 2011 ) with a convergence rate of O ( 1/√k ) . Applied to problem ( 2 ) , with PΩ being the projection onto Ω , it iterates EG : ⌊ wk = PΩ [ zk − αkF ( zk ) ] zk+1 = PΩ [ zk − αkF ( wk ) ] . The Forward-Backward-Forward ( FBF ) method , introduced in Tseng ( 1991 ) , has not been studied rigorously for minimax problems in terms of function values yet , despite promising applications in Boţ et al . ( 2020 ) and its advantage of it only requiring one projection , whereas EG needs two . It is given by FBF : ⌊ wk = PΩ [ zk − αkF ( zk ) ] zk+1 = wk + αk ( F ( zk ) − F ( wk ) ) . ( 4 ) Both , EG and FBF , have the “ disadvantage ” of needing two gradient evaluations per iteration . A possible remedy — suggested in Gidel et al . ( 2019 ) for EG under the name of extrapolation from the past — is to recycle previous gradients . In a similar fashion we consider FBFp : ⌊ wk = PΩ [ zk − αkF ( wk−1 ) ] zk+1 = wk + αk ( F ( wk−1 ) − F ( wk ) ) , ( 5 ) where we replaced F ( zk ) by F ( wk−1 ) twice in ( 4 ) . As a matter of fact , the above method can be written exclusively in terms of the first variable wk by incrementing the index k in the first update and then substituting in the second line . This results in wk+1 = PΩ [ wk − αk+1F ( wk ) + αk ( F ( wk−1 ) − F ( wk ) ) ] . ( 6 ) This way we rediscover a known method which was studied in Malitsky & Tam ( 2020 ) for general monotone inclusions under the name of forward-reflected-backward . It reduces to optimistic mirror descent ( Rakhlin & Sridharan , 2013a ; b ) in the unconstrained case with constant step size αk = α , giving wk+1 = wk − α ( 2F ( wk ) − F ( wk−1 ) ) ( 7 ) which has been proposed for the training of GANs under the name of Optimistic Gradient Descent Ascent ( OGDA ) , see Daskalakis et al . ( 2018 ) ; Daskalakis & Panageas ( 2018 ) ; Liang & Stokes ( 2019 ) . All of the above methods and extensions rely solely on the monotone operator formulation of the saddle point problem where the two components x and y play a symmetric role . Taking the special minimax structure into consideration , Hamedani & Aybat ( 2018 ) showed convergence of a method that uses an optimistic step ( 7 ) in one component and a regular gradient step in the other , thus requiring less storing of past gradients in comparison to ( 6 ) . On the downside , however , by reducing the number of required gradient evaluations per iteration , the largest possible step size is reduced from 1/L ( see Korpelevich ( 1976 ) or Section 3 ) to 1/2L ( see Gidel et al . ( 2019 ) ; Malitsky & Tam ( 2020 ) ; Malitsky ( 2015 ) or Section 3 ) . To summarize , the number of required gradient evaluations is halved , but so is the step size , resulting in no clear net gain .

Summary: The authors in this paper, inspired by the applications of min-max optimization in GANs, study the problem of min-max optimization for convex-concave functions. The main contribution of the paper is proving novel convergence results for Forward-Backward-Forward (FBF) algorithms as well as Optimistic Gradient Descent Ascent (OGDA) based on tools from monotone inclusion problems. Their convergence results cover both deterministic and stochastic settings and the rates of convergence for suitably chosen gap function are non-asymptotic. Finally, they apply their algorithms both on toy problems but also on training GANs on CIFAR-10.