Understand the dynamics of GANs via Primal-Dual Optimization

1 INTRODUCTION . Since it was first invented by Ian Goodfellow in his seminal work Goodfellow et al . ( 2014 ) , generative adversarial networks ( GANs ) have been considered as one of the greatest discoveries in machine learning community . It is an extremely powerful tool to estimate data distributions and generate realistic samples . To train its implicit generative model , GAN uses a discriminator since traditional Bayesian methods that require analytic density functions are no longer applicable . This novel approach inspired by zero sum game theory leads to a significant performance boost ; GANs are able to generate samples in a fidelity level that is way beyond traditional Bayesian methods . During the last few years , there have been numerous research articles in this area aiming at improving its performance ( Radford et al. , 2015 ; Zhao et al. , 2016 ; Nowozin et al. , 2016 ; Arjovsky et al. , 2017 ; Mao et al. , 2017 ) . GANs have now become one of most recognized unsupervised learning techniques and have been widely used in a variety of domains such as image generation ( Nguyen et al. , 2017 ) , image super resolution ( Ledig et al. , 2017 ) , imitation learning ( Ho & Ermon , 2016 ) . Despite the great progress of GANs , many essential problems remain unsolved . Why is GAN so hard to train ? How to tune the hyper-parameters to reduce instability in GAN training ? How to eliminate mode collapse and fake images that show up frequently in training ( Arjovsky & Bottou , 2017 ) ? Comparing with many other machine learning techniques , the properties of GANs are far from being well understood . It is quite likely that the theoretical foundation of GANs will become a longstanding problem . The theoretical difficulty of GANs mainly lies in the following several aspects . First , it is a non-convex optimization problem with a complicated landscape . It is unclear how to solve such optimization problems efficiently . The first-order method widely used in the literature via updating the generator and discriminator along descent/ascent direction does not seem to converge all the time . Although some techniques were proposed to stabilize the training performance of the network , e.g. , spectral normalization Miyato et al . ( 2018 ) , in fact , there is no evidence that these algorithms guarantee even local optimality . Second , even if there were an efficient algorithm to solve this optimization problem , we do not know how well they generalize . After all , the optimization formulation is based only on the samples generated by the underlying distribution but our goal is to recover this underlying distribution . Of course , this is a problem faced by all machine learning techniques . Last , there are no reliable ways to evaluate the quality of trained models . There are a number of works in this topic ( Salimans et al. , 2016 ; Heusel et al. , 2017 ) , but human eyes inspection remains the primary approach to judge a GAN model . In the present work , we focus on the first problem and analyze the dynamics of GANs from an optimization point of view . More precisely , we study the convergence properties of the first-order method in GAN training . Our contributions can be summarized as follows . 1 ) We formulate a large class of GAN problems as a primal-dual optimization problem with a coupling term that is linear over discriminator ( see Section 2 for the exact formulation ) ; 2 ) We prove that the simple primal-dual first-order algorithm converges to a stationary solution with a sublinear convergent rate O ( 1/t ) . There have been a number of papers that study the dynamics of GANs from an optimization viewpoint . These works can be roughly divided into three categories . In the first category , the authors focus on high level idea using nonparametric models . This includes the original GAN paper Goodfellow et al . ( 2014 ) , the Wasserstein GAN papers Arjovsky & Bottou ( 2017 ) ; Arjovsky et al . ( 2017 ) and many other works proposing new GAN structures . In the second category , the authors consider the unrolled dynamics ( Metz et al. , 2016 ) , that is , the discriminator remains optimal or almost optimal during the optimization processes . This is considerably different to the first-order iterative algorithm widely used in GAN training . Recent works Heusel et al . ( 2017 ) ; Li et al . ( 2017 ) ; Sanjabi et al . ( 2018 ) provide global convergence analysis for this algorithm . The last category is on the first-order primal-dual algorithm , in which both the discriminator and the generator update via ( stochastic ) gradient descent . However , most of the convergence analysis are local ( Daskalakis et al. , 2017 ; Mescheder et al. , 2017 ; Nagarajan & Kolter , 2017 ; Li et al. , 2018 ) . Other related work including the following : In Qian et al . ( 2018 ) the authors consider a gradient descent/ascent algorithm for a special min-max problem arising from robust learning ( min problem is unconstrained , max problem has simplex constraints ) ; In Yadav et al . ( 2018 ) the GANs are treated as convex-concave primal-dual optimization problems . This formulation is considerably different to our setup where GANs , as they should be , are formulated as nonconvex saddle point problems . In Farnia & Tse ( 2018 ) , the authors investigated the properties of the optimal solutions , which is also different from our work focusing on convergence analysis of the first-order primal-dual algorithm . In Zhao et al . ( 2018 ) , some unified framework covering several generative models , e.g. , VAE , infoGAN , were proposed in the Lagrangian framework . However , the dual variable in their problem is a Lagrangian multiplier , while in our problem , it is the discriminator of GAN . Besides , the focus of their paper is not the optimization algorithm . In Chen et al . ( 2018 ) , the authors related a class of GANs to constrained convex optimization problems . More specifically , such GANs can be viewed as Lagrangian forms of these convex optimization problems . The optimization variables in their formulation are the probability density of the generator and the function values of the discriminator . Many issues like nonconvexity do not show up . This is essentially a nonparametric model , which doesn ’ t apply to cases when the discriminator and the generator are represented by parametric models . On the other hand , our analysis is carried out on the parametric models directly and we have to deal with the nonconvexity of neural networks . In Hajinezhad & Hong ( 2017 ) a primal-dual algorithm has been studied for a non-convex linearly constrained problem ( which can be reformulated into a min-max problem , with the max problem being linear and unconstrained , and with linear coupling between variables ) ; In Hamedani & Aybat ( 2018 ) , ( Chen et al. , 2013 ) and the references therein , first-order methods have been developed for convex-concave saddle point problems . Compared to these works , our considered problem is more general , allowing non-convexity and non-smoothness in the objective , non-convex coupling between variables , and can further include constraints . Moreover , we provide global convergence rate analysis , which is much stronger than the local analysis mentioned above . It turns out that the primal-dual framework we study in this paper can also be applied to the distributional robust machine learning problems ( Namkoong & Duchi , 2016 ) and the multi-task learning problems ( Qian et al. , 2018 ) . In multi-task learning , the goal is to train a single neural network that would work for several different machine learning tasks . Similarly , in distributional robust learning , the purpose is to have a single model that would work for a set of data distributions . In both problems , an adversarial layer is utilized to improve the worst case performance , which leads to a primal-dual optimization structure that falls into the scope of problems we consider . The rest of the paper is structured as follows . In Section 2 we introduce GAN and its primal-dual formulation . We provide details of the algorithms with proof sketches in Section 3 . The full proofs are relegated to the appendix . We highlight our theoretical results in Section 4 via several numerical examples , with both synthetic and real datasets . 2 GENERATIVE ADVERSARIAL NETWORKS AND MIN-MAX PROBLEMS . GAN is a type of deep generative model with implicit density functions . It consists of two critical components : a generator and a discriminator . The generator takes random variables with known distribution as input and outputs fake samples . The discriminator is trained to distinguish real samples and fake samples . 2.1 GANS . In the original form of GAN , the generator is a neural network and the discriminator is a standard classifier with cross entropy cost . Denote the underlying probability distributions of the data sample { yi } ny i=1 ⊂ Rd and the random seed x ( a random variable in Rk ) of the generative model by Py and Px respectively . The original or vanilla GAN is of the form ( Goodfellow et al. , 2014 ) min G∈G max F∈F { E ( log ( F ( y ) ) ) −E ( log ( 1− F ( G ( x ) ) ) ) } . ( 1 ) Here G stands for the set of possible generators and F the set of possible discriminators . There have been numerous modifications of the original GAN , among which the Wasserstein GAN ( Arjovsky et al. , 2017 ) attracts a lot of attention . It has the form min G∈G max F∈F { E ( F ( y ) ) −E ( F ( G ( x ) ) ) } , ( 2 ) where G is the set of parameterized generators and F is the set of Lipschitz functions with Lipschitz constant 1 . This is a special case of a more general class of GANs with the same structure but different constraint set F . The inner maximization loop defines different notions of integral probability metrics ( Müller , 1997 ) depending on the choices of F . Other than Wasserstein GAN , one interesting example with this structure is the generative moment matching networks ( Li et al. , 2015 ; Arjovsky et al. , 2017 ) . Wasserstein GAN can be extended to general optimal transport cost c ( x , y ) , which results in saddle point formulation min G∈G max ψ , φ { E ( ψ ( y ) ) + E ( φ ( G ( x ) ) | ψ ( y ) + φ ( x ) ≤ c ( x , y ) , ∀x , y } . ( 3 ) Note that the discriminator now becomes two functions ψ , φ instead of one . When c ( x , y ) = ‖x−y‖ , the above reduces to the standard Wasserstein GAN in equation 2 . 2.2 NONCONVEX MIN-MAX PROBLEMS . We observe that , compared to vanilla GAN in equation 1 , the Wasserstein GAN in equation 2 and equation 3 has a special structure . In the nonparametric form , the coupling between the generator G and the discriminator F is linear on F in Wasserstein GAN while nonlinear in vanilla GAN . Indeed , replacing F by αF1 + βF2 in the coupling term E ( F ( G ( x ) ) ) yields E ( F ( G ( x ) ) ) = E ( ( αF1 + βF2 ) ( G ( x ) ) ) = αE ( F1 ( G ( x ) ) ) + βE ( F2 ( G ( x ) ) ) . This structure motivates us to study the following min-max primal-dual optimization problem min X∈X max Y ∈Y h ( X ) + 〈g ( X ) , Y 〉 − l ( Y ) ( 4 ) with l being strictly convex . the other two functions h and g can be non-convex . Here Y represents the discriminator F and X represents the generator G. Note that the coupling term 〈g ( X ) , Y 〉 is linear over the discriminator Y , but nonlinear in X .In real applications , we need to parameterize the discriminator and generator , which may lead to the loss of the property that the coupling is linear over discriminator . Next , we present several cases where this linear structure stays .

Summary: This paper analyses the learning dynamics of GANs by formulating the problem as a primal-dual optimisation problem. This formulation assumes a limited class of models -- Wasserstein GANs with discriminators using linear combinations of base functions. Although this setting is limited, it advanced our understanding of a central problem related to GANs, and provides intuition for more general cases. The paper further shows the same analysis can be applied to multi-task learning and distributed learning.