Two steps at a time --- taking GAN training in stride with Tseng's method

1 INTRODUCTION . Generative Adversarial Networks ( GANs ) ( Goodfellow et al. , 2014 ) have proven to be a powerful class of generative models , producing for example unseen realistic images . Two neural networks , called generator and discriminator , compete against each other in a game . In the special case of a zero sum game this task can be formulated as a minimax ( aka saddle point ) problem . Conventionally , GANs are trained using variants of ( stochastic ) Gradient Descent Ascent ( GDA ) which are known to exhibit oscillatory behavior and thus fail to converge even for simple bilinear saddle point problems , see Goodfellow ( 2016 ) . We therefore propose the use of methods with provable convergence guarantees for ( stochastic ) convex-concave minimax problems , even though GANs are well known to not warrant these properties . Along similar considerations an adaptation of the Extragradient method ( EG ) ( Korpelevich , 1976 ) for the training of GANs was suggested in Gidel et al . ( 2019 ) , whereas Daskalakis et al . ( 2018 ) ; Daskalakis & Panageas ( 2018 ) ; Liang & Stokes ( 2019 ) studied Optimistic Gradient Descent Ascent ( OGDA ) based on optimistic mirror descent ( Rakhlin & Sridharan , 2013a ; b ) . We however investigate the Forward-Backward-Forward ( FBF ) method ( Tseng , 1991 ) from monotone operator theory , which uses two gradient evaluations per update , similar to EG , in order to circumvent the aforementioned issues . Instead of trying to improve GAN performance via new architectures , loss functions , etc. , we contribute to the theoretical foundation of their training from the point of view of optimization . Contribution . Establishing the connection between GAN training and monotone inclusions motivates to use the FBF method , originally designed to solve this type of problems . This approach allows to naturally extend the constrained setting to a regularized one making use of the proximal operator . We also propose a variant of FBF reusing previous gradients to reduce the computational cost per iteration , which turns out to be a known method , related to OGDA . By developing a unifying scheme that captures FBF and a generalization of OGDA , we reveal a hitherto unknown connection . Using this approach we prove novel non asymptotic convergence statements in terms of the minimax gap for both methods in the context of saddle point problems . In the deterministic and stochastic setting we obtain rates of O ( 1/k ) and O ( 1/√k ) , respectively . Concluding , we highlight the relevance of our proposed method as well as the role of regularizers by showing empirical improvements in the training of Wasserstein GANs on the CIFAR10 dataset . Organization . This paper is structured as follows . In Section 2 we highlight the connection of GAN training and monotone inclusions and give an extensive review of methods with convergence guarantees for the latter . The main results as well as a precise definition of the measure of optimality are discussed in Section 3 . Concluding , Section 4 illustrates the empirical performance in the training of GANs as well as solving bilinear problems . 2 GAN TRAINING AS MONOTONE INCLUSION . The GAN objective was originally cast as a two-player zero-sum game between the discriminator Dy and the generator Gx ( Goodfellow et al. , 2014 ) given by min x max y Eρ∼q [ log ( Dy ( ρ ) ) ] + Eζ∼p [ log ( 1−Dy ( Gx ( ζ ) ) ) ] , exhibiting the aforementioned minimax structure . Due to problems with vanishing gradients in the training of such models , a successful alternative formulation called Wasserstein GAN ( WGAN ) ( Arjovsky et al. , 2017 ) has been proposed . In this case the minimization tries to reduce the Wasserstein distance between the true distribution q and the one learned by the generator . Reformulating this distance via the Kantorovich Rubinstein duality leads to an inner maximization over 1-Lipschitz functions which are approximated via neural networks , yielding the saddle point problem min x max y : ‖Dy‖Lip≤1 Eρ∼q [ Dy ( ρ ) ] − Eζ∼p [ Dy ( Gx ( ζ ) ) ] . 2.1 CONVEX-CONCAVE MINIMAX PROBLEMS . Due to the observations made in the previous paragraph we study the following abstract minimax problem min x∈Rd max y∈Rn Ψ ( x , y ) : = f ( x ) + Eξ∼Q [ Φ ( x , y ; ξ ) ] − h ( y ) , ( 1 ) where the convex-concave coupling function Φ ( x , y ) : = Eξ∼Q [ Φ ( x , y ; ξ ) ] , which hides the stochasticity for ease of notation , is differentiable with L-Lipschitz continuous gradient . The proper , convex and lower semicontinuous functions f : Rd → R ∪ { +∞ } and h : Rn → R ∪ { +∞ } act as regularizers . A solution of ( 1 ) is given by a so-called saddle point ( x∗ , y∗ ) fulfilling for all x and y Ψ ( x∗ , y ) ≤ Ψ ( x∗ , y∗ ) ≤ Ψ ( x , y∗ ) . In the context of two-player games this corresponds to a pair of strategies , where no player can be better off by changing just their own strategy . For the purpose of this motivating section , we will restrict ourselves for now to the special case of the deterministic constrained version of ( 1 ) , given by min x∈X max y∈Y Φ ( x , y ) , where f and h are given by indicator functions of closed convex sets X and Y , respectively . The indicator function δC of a set C is defined as δC ( z ) = 0 for z ∈ C and δC ( z ) = +∞ otherwise . 2.2 MINIMAX PROBLEMS AS MONOTONE INCLUSIONS . If the coupling function Φ is convex-concave and differentiable then solving ( 1 ) is equivalent to solving the first order optimality conditions which can be written as a so-called monotone inclusion with w = ( x , y ) ∈ Rm and m = d+ n , given by 0 ∈ F ( w ) +NΩ ( w ) . ( 2 ) The entities involved are F ( x , y ) : = ( ∇xΦ ( x , y ) , −∇yΦ ( x , y ) ) , ( 3 ) and the normal cone NΩ of the convex set Ω : = X × Y . The normal cone mapping is given by NΩ ( w ) = { v ∈ Rm : 〈v , w′ − w〉 ≤ 0 ∀w′ ∈ Ω } , for w ∈ Ω and NΩ ( w ) = ∅ for w /∈ Ω . Here , the operators F and NΩ satisfy well known properties from convex analysis ( Bauschke & Combettes , 2011 ) , in particular the first one is monotone ( and Lipschitz if ∇Φ is so ) whereas the latter one is maximal monotone . We call a , possibly set-valued , operator A from Rm to itself monotone if 〈u− u′ , z − z′〉 ≥ 0 ∀u ∈ A ( z ) , u′ ∈ A ( z′ ) . We say A is maximal monotone , if there exists no monotone operator A′ such that the graph of A is properly contained in the graph of A′ . Problems of type ( 2 ) have been studied thoroughly in convex optimization , with the most established solution methods being Extragradient ( Korpelevich , 1976 ) and Forward-Backward-Forward ( Tseng , 1991 ) . Both methods are known to generate sequences of iterates converging to a solution of ( 2 ) . Note that in the unconstrained setting ( i.e . if Ω is the entire space ) both of these algorithms even produce the same iterates . 2.3 SOLVING MONOTONE INCLUSIONS . The connection between monotone inclusions and saddle point problems is of course not new . The application of Extragradient ( EG ) to minimax problems has been studied in the seminal paper Nemirovski ( 2004 ) under the name of Mirror Prox and a convergence rate of O ( 1/k ) in terms of the function values has been proven . Even a stochastic version of the Mirror Prox algorithm has been studied in Juditsky et al . ( 2011 ) with a convergence rate of O ( 1/√k ) . Applied to problem ( 2 ) , with PΩ being the projection onto Ω , it iterates EG : ⌊ wk = PΩ [ zk − αkF ( zk ) ] zk+1 = PΩ [ zk − αkF ( wk ) ] . The Forward-Backward-Forward ( FBF ) method , introduced in Tseng ( 1991 ) , has not been studied rigorously for minimax problems in terms of function values yet , despite promising applications in Boţ et al . ( 2020 ) and its advantage of it only requiring one projection , whereas EG needs two . It is given by FBF : ⌊ wk = PΩ [ zk − αkF ( zk ) ] zk+1 = wk + αk ( F ( zk ) − F ( wk ) ) . ( 4 ) Both , EG and FBF , have the “ disadvantage ” of needing two gradient evaluations per iteration . A possible remedy — suggested in Gidel et al . ( 2019 ) for EG under the name of extrapolation from the past — is to recycle previous gradients . In a similar fashion we consider FBFp : ⌊ wk = PΩ [ zk − αkF ( wk−1 ) ] zk+1 = wk + αk ( F ( wk−1 ) − F ( wk ) ) , ( 5 ) where we replaced F ( zk ) by F ( wk−1 ) twice in ( 4 ) . As a matter of fact , the above method can be written exclusively in terms of the first variable wk by incrementing the index k in the first update and then substituting in the second line . This results in wk+1 = PΩ [ wk − αk+1F ( wk ) + αk ( F ( wk−1 ) − F ( wk ) ) ] . ( 6 ) This way we rediscover a known method which was studied in Malitsky & Tam ( 2020 ) for general monotone inclusions under the name of forward-reflected-backward . It reduces to optimistic mirror descent ( Rakhlin & Sridharan , 2013a ; b ) in the unconstrained case with constant step size αk = α , giving wk+1 = wk − α ( 2F ( wk ) − F ( wk−1 ) ) ( 7 ) which has been proposed for the training of GANs under the name of Optimistic Gradient Descent Ascent ( OGDA ) , see Daskalakis et al . ( 2018 ) ; Daskalakis & Panageas ( 2018 ) ; Liang & Stokes ( 2019 ) . All of the above methods and extensions rely solely on the monotone operator formulation of the saddle point problem where the two components x and y play a symmetric role . Taking the special minimax structure into consideration , Hamedani & Aybat ( 2018 ) showed convergence of a method that uses an optimistic step ( 7 ) in one component and a regular gradient step in the other , thus requiring less storing of past gradients in comparison to ( 6 ) . On the downside , however , by reducing the number of required gradient evaluations per iteration , the largest possible step size is reduced from 1/L ( see Korpelevich ( 1976 ) or Section 3 ) to 1/2L ( see Gidel et al . ( 2019 ) ; Malitsky & Tam ( 2020 ) ; Malitsky ( 2015 ) or Section 3 ) . To summarize , the number of required gradient evaluations is halved , but so is the step size , resulting in no clear net gain .

Summary: This work studies minimax optimization (a.k.a saddle-point problems) with nonsmooth regularizers. By leveraging the monotone operator theory, the authors propose to use the forward-backward-forward method so as to avoids the notorious limit cycling problem. The classical FBF method requires two gradient evaluations per step, the authors introduce a new algorithm which reuses the past gradient in the same way as OGDA. In the setting of convex-concave minimax optimization, the authors claim to prove novel convergence rates for both methods.