A Variational Inequality Perspective on Generative Adversarial Networks

1 INTRODUCTION . Generative adversarial networks ( GANs ) ( Goodfellow et al. , 2014 ) form a generative modeling approach known for producing realistic natural images ( Karras et al. , 2018 ) as well as high quality super-resolution ( Ledig et al. , 2017 ) and style transfer ( Zhu et al. , 2017 ) . Nevertheless , GANs are also known to be difficult to train , often displaying an unstable behavior ( Goodfellow , 2016 ) . Much recent work has tried to tackle these training difficulties , usually by proposing new formulations of the GAN objective ( Nowozin et al. , 2016 ; Arjovsky et al. , 2017 ) . Each of these formulations can be understood as a two-player game , in the sense of game theory ( Von Neumann and Morgenstern , 1944 ) , and can be addressed as a variational inequality problem ( VIP ) ( Harker and Pang , 1990 ) , a framework that encompasses traditional saddle point optimization algorithms ( Korpelevich , 1976 ) . Solving such GAN games is traditionally approached by running variants of stochastic gradient descent ( SGD ) initially developed for optimizing supervised neural network objectives . Yet it is known that for some games ( Goodfellow , 2016 , §8.2 ) SGD exhibits oscillatory behavior and fails to converge . This oscillatory behavior , which does not arise from stochasticity , highlights a fundamental problem : while a direct application of basic gradient descent is an appropriate method for regular minimization problems , it is not a sound optimization algorithm for the kind of two-player games of GANs . This constitutes a fundamental issue for GAN training , and calls for the use of more principled methods with more reassuring convergence guarantees . Contributions . We point out that multi-player games can be cast as variational inequality problems ( VIPs ) and consequently the same applies to any GAN formulation posed as a minimax or non-zerosum game . We present two techniques from this literature , namely averaging and extrapolation , widely used to solve VIPs but which have not been explored in the context of GANs before.1 We extend standard GAN training methods such as SGD or Adam into variants that incorporate these techniques ( Alg . 4 is new ) . We also explain that the oscillations of basic SGD for GAN training previously noticed ( Goodfellow , 2016 ) can be explained by standard variational inequality optimization results and we illustrate how averaging and extrapolation can fix this issue . ∗Equal contribution , correspondence to firstname.lastname @ umontreal.ca . 1The preprints for ( Mertikopoulos et al. , 2019 ) and ( Yazıcı et al. , 2019 ) , which respectively explored extrapolation and averaging for GANs , appeared after our initial preprint . See also the related work section §6 . We introduce a technique , called extrapolation from the past , that only requires one gradient computation per update compared to extrapolation which requires to compute the gradient twice , rediscovering , with a VIP perspective , a particular case of optimistic mirror descent ( Rakhlin and Sridharan , 2013 ) . We prove its convergence for strongly monotone operators and in the stochastic VIP setting . Finally , we test these techniques in the context of GAN training . We observe a 4-6 % improvement over Miyato et al . ( 2018 ) on the inception score and the Fréchet inception distance on the CIFAR-10 dataset using a WGAN-GP ( Gulrajani et al. , 2017 ) and a ResNet generator.2 Outline . §2 presents the background on GAN and optimization , and shows how to cast this optimization as a VIP . §3 presents standard techniques and extrapolation from the past to optimize variational inequalities in a batch setting . §4 considers these methods in the stochastic setting , yielding three corresponding variants of SGD , and provides their respective convergence rates . §5 develops how to combine these techniques with already existing algorithms . §6 discusses the related work and §7 presents experimental results . 2 GAN OPTIMIZATION AS A VARIATIONAL INEQUALITY PROBLEM . 2.1 GAN FORMULATIONS . The purpose of generative modeling is to generate samples from a distribution qθ that matches best the true distribution p of the data . The generative adversarial network training strategy can be understood as a game between two players called generator and discriminator . The former produces a sample that the latter has to classify between real or fake data . The final goal is to build a generator able to produce sufficiently realistic samples to fool the discriminator . In the original GAN paper ( Goodfellow et al. , 2014 ) , the GAN objective is formulated as a zero-sum game where the cost function of the discriminator Dϕ is given by the negative log-likelihood of the binary classification task between real or fake data generated from qθ by the generator , min θ max ϕ L ( θ , ϕ ) where L ( θ , ϕ ) def= −E x∼p [ logDϕ ( x ) ] − E x′∼qθ [ log ( 1−Dϕ ( x′ ) ) ] . ( 1 ) However Goodfellow et al . ( 2014 ) recommends to use in practice a second formulation , called non-saturating GAN . This formulation is a non-zero-sum game where the aim is to jointly minimize : LG ( θ , ϕ ) def = − E x′∼qθ logDϕ ( x ′ ) and LD ( θ , ϕ ) def = − E x∼p logDϕ ( x ) − E x′∼qθ log ( 1−Dϕ ( x′ ) ) . ( 2 ) The dynamics of this formulation has the same stationary points as the zero-sum one ( 1 ) but is claimed to provide “ much stronger gradients early in learning ” ( Goodfellow et al. , 2014 ) . 2.2 EQUILIBRIUM . The minimax formulation ( 1 ) is theoretically convenient because a large literature on games studies this problem and provides guarantees on the existence of equilibria . Nevertheless , practical considerations lead the GAN literature to consider a different objective for each player as formulated in ( 2 ) . In that case , the two-player game problem ( Von Neumann and Morgenstern , 1944 ) consists in finding the following Nash equilibrium : θ∗ ∈ arg min θ∈Θ LG ( θ , ϕ∗ ) and ϕ∗ ∈ arg min ϕ∈Φ LD ( θ∗ , ϕ ) . ( 3 ) Only when LG = −LD is the game called a zero-sum game and ( 3 ) can be formulated as a minimax problem . One important point to notice is that the two optimization problems in ( 3 ) are coupled and have to be considered jointly from an optimization point of view . Standard GAN objectives are non-convex ( i.e . each cost function is non-convex ) , and thus such ( pure ) equilibria may not exist . As far as we know , not much is known about the existence of these equilibria for non-convex losses ( see Heusel et al . ( 2017 ) and references therein for some results ) . In 2Code available at https : //gauthiergidel.github.io/projects/vip-gan.html . our theoretical analysis in §4 , our assumptions ( monotonicity ( 24 ) of the operator and convexity of the constraint set ) imply the existence of an equilibrium . In this paper , we focus on ways to optimize these games , assuming that an equilibrium exists . As is often standard in non-convex optimization , we also focus on finding points satisfying the necessary stationary conditions . As we mentioned previously , one difficulty that emerges in the optimization of such games is that the two different cost functions of ( 3 ) have to be minimized jointly in θ and ϕ. Fortunately , the optimization literature has for a long time studied so-called variational inequality problems , which generalize the stationary conditions for two-player game problems . 2.3 VARIATIONAL INEQUALITY PROBLEM FORMULATION . We first consider the local necessary conditions that characterize the solution of the smooth two-player game ( 3 ) , defining stationary points , which will motivate the definition of a variational inequality . In the unconstrained setting , a stationary point is a couple ( θ∗ , ϕ∗ ) with zero gradient : ‖∇θLG ( θ∗ , ϕ∗ ) ‖ = ‖∇ϕLD ( θ∗ , ϕ∗ ) ‖ = 0 . ( 4 ) When constraints are present,3 a stationary point ( θ∗ , ϕ∗ ) is such that the directional derivative of each cost function is non-negative in any feasible direction ( i.e . there is no feasible descent direction ) : ∇θLG ( θ∗ , ϕ∗ ) > ( θ − θ∗ ) ≥ 0 and ∇ϕLD ( θ∗ , ϕ∗ ) > ( ϕ−ϕ∗ ) ≥ 0 , ∀ ( θ , ϕ ) ∈ Θ× Φ . ( 5 ) Defining ω def= ( θ , ϕ ) , ω∗ def= ( θ∗ , ϕ∗ ) , Ω def= Θ× Φ , Eq . ( 5 ) can be compactly formulated as : F ( ω∗ ) > ( ω − ω∗ ) ≥ 0 , ∀ω ∈ Ω where F ( ω ) def= [ ∇θLG ( θ , ϕ ) ∇ϕLD ( θ , ϕ ) ] > . ( 6 ) These stationary conditions can be generalized to any continuous vector field : let Ω ⊂ Rd and F : Ω→ Rd be a continuous mapping . The variational inequality problem ( Harker and Pang , 1990 ) ( depending on F and Ω ) is : find ω∗ ∈ Ω such that F ( ω∗ ) > ( ω − ω∗ ) ≥ 0 , ∀ω ∈ Ω . ( VIP ) We call optimal set the set Ω∗ of ω ∈ Ω verifying ( VIP ) . The intuition behind it is that any ω∗ ∈ Ω∗ is a fixed point of the constrained dynamic of F ( constrained to Ω ) . We have thus showed that both saddle point optimization and non-zero sum game optimization , which encompass the large majority of GAN variants proposed in the literature , can be cast as VIPs . In the next section , we turn to suitable optimization techniques for such problems . 3 OPTIMIZATION OF VARIATIONAL INEQUALITIES ( BATCH SETTING ) . Let us begin by looking at techniques that were developed in the optimization literature to solve VIPs . We present the intuitions behind them as well as their performance on a simple bilinear problem ( see Fig . 1 ) . Our goal is to provide mathematical insights on averaging ( §3.1 ) and extrapolation ( §3.2 ) and propose a novel variant of the extrapolation technique that we called extrapolation from the past ( §3.3 ) . We consider the batch setting , i.e. , the operator F ( ω ) defined in Eq . 6 yields an exact full gradient . We present extensions of these techniques to the stochastic setting later in §4 . The two standard methods studied in the VIP literature are the gradient method ( Bruck , 1977 ) and the extragradient method ( Korpelevich , 1976 ) . The iterates of the basic gradient method are given by ωt+1 = PΩ [ ωt − ηF ( ωt ) ] where PΩ [ · ] is the projection onto the constraint set ( if constraints are present ) associated to ( VIP ) . These iterates are known to converge linearly under an additional assumption on the operator4 ( Chen and Rockafellar , 1997 ) , but oscillate for a bilinear operator as shown in Fig . 1 . On the other hand , the uniform average of these iterates converge for any bounded monotone operator with a O ( 1/ √ t ) rate ( Nedić and Ozdaglar , 2009 ) , motivating the presentation of averaging in §3.1 . By contrast , the extragradient method ( extrapolated gradient ) does not require any averaging to converge for monotone operators ( in the batch setting ) , and can even converge at the faster O ( 1/t ) rate ( Nesterov , 2007 ) . The idea of this method is to compute a lookahead step ( see intuition on extrapolation in §3.2 ) in order to compute a more stable direction to follow . 3An example of constraint for GANs is to clip the parameters of the discriminator ( Arjovsky et al. , 2017 ) . 4Strong monotonicity , a generalization of strong convexity . See §A .

Summary: The authors take a variational inequality perspective to the study of the saddle point problem that defines a GAN. By doing so, they are able to profit from the corresponding literature and propose a few methods that are variants of SGD. The authors show in a simple example (a bilinear function) these exhibit better performance than Adam and a basic gradient method. After showing theoretical guarantees of these methods (linear convergence) the authors propose to combine them with existing techniques, and show in fact this leads to better results.