Stackelberg GAN: Towards Provable Minimax Equilibrium via Multi-Generator Architectures

1 INTRODUCTION . Generative Adversarial Nets ( GANs ) are emerging objects of study in machine learning , computer vision , natural language processing , and many other domains . In machine learning , study of such a framework has led to significant advances in adversarial defenses ( Xiao et al. , 2018 ; Samangouei et al. , 2018 ) and machine security ( Athalye et al. , 2018 ; Samangouei et al. , 2018 ) . In computer vision and natural language processing , GANs have resulted in improved performance over standard generative models for images and texts ( Goodfellow et al. , 2014 ) , such as variational autoencoder ( Kingma & Welling , 2013 ) and deep Boltzmann machine ( Salakhutdinov & Larochelle , 2010 ) . A main technique to achieve this goal is to play a minimax two-player game between generator and discriminator under the design that the generator tries to confuse the discriminator with its generated contents and the discriminator tries to distinguish real images/texts from what the generator creates . Despite a large amount of variants of GANs , many fundamental questions remain unresolved . One of the long-standing challenges is designing universal , easy-to-implement architectures that alleviate the instability issue of GANs training . Ideally , GANs are supposed to solve the minimax optimization problem ( Goodfellow et al. , 2014 ) , but in practice alternating gradient descent methods do not clearly privilege minimax over maximin or vice versa ( page 35 , Goodfellow ( 2016 ) ) , which may lead to instability in training if there exists a large discrepancy between the minimax and maximin objective values . The focus of this work is on improving the stability of such minimax game in the training process of GANs . To alleviate the issues caused by the large minimax gap , our study is motivated by the so-called Stackelberg competition in the domain of game theory . In the Stackelberg leadership model , the players of this game are one leader and multiple followers , where the leader firm moves first and then the follower firms move sequentially . It is known that the Stackelberg model can be solved to find a subgame perfect Nash equilibrium . We apply this idea of Stackelberg leadership model to the architecture design of GANs . That is , we design an improved GAN architecture with multiple generators ( followers ) which team up to play against the discriminator ( leader ) . We therefore name our model Stackelberg GAN . Our theoretical and experimental results establish that : GANs with multi-generator architecture have smaller minimax gap , and enjoy more stable training performances . Our Contributions . This paper tackles the problem of instability during the GAN training procedure with both theoretical and experimental results . We study this problem by new architecture design . Under review as a conference paper at ICLR 2019 Under review as a conference paper at ICLR 2019 ( a ) Step 0 . ( b ) Step 6k . ( c ) Step 19k . ( d ) Step 25k . ( e ) Step 0 . ( f ) Step 6k . ( g ) Step 19k . ( h ) Step 25k . Figure 1 : Stackelberg GAN stabilizes the training procedure on a toy 2D mixture of 8 Gaussians . Top Row : Standard GAN training . It shows that several modes are dropped . Bottom Row : Stackelberg GAN training with 8 generator ensembles , each of which is denoted by one color . We can see that each generator exactly learns one mode of the distribution without any mode being dropped . Under review as a conference paper at ICLR 2019 ( a ) Step 0 . ( b ) Step 6k . ( c ) Step 13k . ( d ) Step 19k . ( e ) Step 25k . ( f ) Step 0 . ( g ) Step 6k . ( h ) Step 13k . ( i ) Step 19k . ( j ) Step 25k . Figure 1 : Stackelberg GAN stabiliz s the training procedure on a toy 2D mixture of 8 G ussians . Top Row : Standard GAN trai ing . It shows that several modes are dropp d. Bottom Row : Stack lberg GAN training with 8 generator ensembles , each of which is denoted by one color . We can see that each generator exactly learns one mode of the distribution without any mode being dropped . • We propose Stackelberg GAN framework of having multiple generators in the GAN archi- tecture . Our framework is general that can be applied to all variants of GANs , e.g. , vanilla Figure 1 : Left Figure , Top Row : Standard GAN training on a toy 2D mixture of 8 Gaussians . Left Figure , Bottom Row : Stackelberg GAN training with 8 generator ensembles , each of which is denoted by one color . Right Figure : Stackelberg GAN training with 10 generator ensembles on fashion-MNIST dataset without cherry pick , where each row corresponds to one generator . probabil stic models with assumption that the g nerator and discriminator have infinite ca- pacity . Also , they require that h generators sha e common network parameters . In contrast , in e Stackelberg GAN model we allow various sampling schemes beyond the mixture model , e.g. , each generator samples a fixed but unequal number of data points independently . Fur ermore , each gene ator has free par meters . We also make no assumption on the model 2 STACKELBERG GAN . Before proceeding , we define some notations and formalize our model setup in this section . Notations . We will use bold lower-case letter to represent vector and lower-case letter to represent scalar . Specifically , we denote by θ ∈ Rt the parameter vector of discriminator and γ ∈ Rg the parameter vector of generator . Let Dθ ( x ) be the output probability of discriminator given input x , and let Gγ ( z ) represent the generated vector given random input z . For any function f ( u ) , we denote by f∗ ( v ) : = supu { uTv − f ( u ) } the conjugate function of f . Let c̆lf be the convex closure of f , which is defined as the function whose epigraph is the convex closed hull of that of function f . We define ĉlf : = −c̆l ( −f ) . We will use I to represent the number of generators . 2.1 MODEL SETUP . Preliminaries . The key ingredient in the standard GAN is to play a zero-sum two-player game between a discriminator and a generator — which are often parametrized by deep neural networks in practice — such that the goal of the generator is to map random noise z to some plausible images/texts Gγ ( z ) and the discriminator Dθ ( · ) aims at distinguishing the real images/texts from what the generator creates . For every parameter implementations γ and θ of generator and discriminator , respectively , denote by the payoff value φ ( γ ; θ ) : = Ex∼Pdf ( Dθ ( x ) ) + Ez∼Pzf ( 1−Dθ ( Gγ ( z ) ) ) , where f ( · ) is some concave , increasing function . Hereby , Pd is the distribution of true images/texts and Pz is a noise distribution such as Gaussian or uniform distribution . The standard GAN thus solves the following saddle point problems : inf γ∈Rg sup θ∈Rt φ ( γ ; θ ) , or sup θ∈Rt inf γ∈Rg φ ( γ ; θ ) . ( 1 ) For different choices of function f , problem ( 1 ) leads to various variants of GAN . For example , when f ( t ) = log t , problem ( 1 ) is the classic GAN ; when f ( t ) = t , it reduces to the Wasserstein GAN . We refer interested readers to the paper of Nowozin et al . ( 2016 ) for more variants of GANs . Stackelberg GAN . Our model of Stackelberg GAN is inspired from the Stackelberg competition in the domain of game theory . Instead of playing a two-player game as in the standard GAN , in Stackelberg GAN there are I + 1 players with two firms — one discriminator and I generators . One can make an analogy between the discriminator ( generators ) in the Stackelberg GAN and the leader ( followers ) in the Stackelberg competition . Stackelberg GAN is a general framework which can be built on top of all variants of standard GANs . The objective function is simply an ensemble of losses w.r.t . all possible pairs of generators and discriminator : Φ ( γ1 , ... , γI ; θ ) : = ∑I i=1 φ ( γi ; θ ) . Thus it is very easy to implement . The Stackelberg GAN therefore solves the following saddle point problems : w∗ : = inf γ1 , ... , γI∈Rg sup θ∈Rt 1 I Φ ( γ1 , ... , γI ; θ ) , or q∗ : = sup θ∈Rt inf γ1 , ... , γI∈Rg 1 I Φ ( γ1 , ... , γI ; θ ) . We term w∗ − q∗ the minimax ( duality ) gap . We note that there are key differences between the naïve ensembling model and ours . In the naïve ensembling model , one trains multiple GAN models independently and averages their outputs . In contrast , our Stackelberg GAN shares a unique discriminator for various generators , thus requires jointly training . Figure 2 shows the architecture of our Stackelberg GAN . How to generate samples from Stackelberg GAN ? In the Stackelberg GAN , we expect that each generator learns only a few modes . In order to generate a sample that may come from all modes , we use a mixed model . In particular , we generate a uniformly random value i from 1 to I and use the i-th generator to obtain a new sample . Note that this procedure in independent of the training procedure . 3 ANALYSIS OF STACKELBERG GAN . In this section , we develop our theoretical contributions and compare our results with the prior work . 3.1 MINIMAX DUALITY GAP . We begin with studying the minimax gap of Stackelberg GAN . Our main results show that the minimax gap shrinks as the number of generators increases . To proceed , denote by hi ( ui ) : = infγi∈Rg ( −φ ( γi ; · ) ) ∗ ( ui ) , where the conjugate operation is w.r.t . the second argument of φ ( γi ; · ) . We clarify here that the subscript i in hi indicates that the function hi is derived from the i-th generator . The argument of hi should depend on i , so we denote it by ui . Intuitively , hi serves as an approximate convexification of −φ ( γi , · ) w.r.t the second argument due to the conjugate operation . Denote by c̆lhi the convex closure of hi : c̆lhi ( ũ ) : = inf { aj } , { uji } t+2∑ j=1 ajhi ( u j i ) : ũ = t+2∑ j=1 ajuji , t+2∑ j=1 aj = 1 , aj ≥ 0 . c̆lhi represents the convex relaxation of hi because the epigraph of c̆lhi is exactly the convex hull of epigraph of hi by the definition of c̆lhi . Let ∆minimaxθ = infγ1 , ... , γI∈Rg supθ∈Rt 1 IΦ ( γ1 , ... , γI ; θ ) − infγ1 , ... , γI∈Rg supθ∈Rt 1 I Φ̃ ( γ1 , ... , γI ; θ ) , and ∆ maximin θ = supθ∈Rt infγ1 , ... , γI∈Rg 1 I Φ̃ ( γ1 , ... , γI ; θ ) − supθ∈Rt infγ1 , ... , γI∈Rg 1 IΦ ( γ1 , ... , γI ; θ ) , where Φ̃ ( γ1 , ... , γI ; θ ) : = ∑I i=1 ĉlφ ( γi ; θ ) and −ĉlφ ( γi ; θ ) is the convex closure of −φ ( γi ; θ ) w.r.t . argument θ . Therefore , ∆maximinθ + ∆minimaxθ measures the non-convexity of objective function w.r.t . argument θ . For example , it is equal to 0 if and only if φ ( γi ; θ ) is concave and closed w.r.t . discriminator parameter θ . We have the following guarantees on the minimax gap of Stackelberg GAN . Theorem 1 . Let ∆iγ : = supu∈Rt { hi ( u ) − c̆lhi ( u ) } ≥ 0 and ∆worstγ : = maxi∈ [ I ] ∆iγ . Denote by t the number of parameters of discriminator , i.e. , θ ∈ Rt . Suppose that hi ( · ) is continuous and domhi is compact and convex . Then the duality gap can be bounded by 0 ≤ w∗ − q∗ ≤ ∆minimaxθ + ∆maximinθ + , provided that the number of generators I > t+1 ∆ worst γ . Remark 1 . Theorem 1 makes mild assumption on the continuity of loss and no assumption on the model capacity of discriminator and generators . The analysis instead depends on their nonconvexity as being parametrized by deep neural networks . In particular , ∆iγ measures the divergence between the function value of hi and its convex relaxation c̆lhi ; When φ ( γi ; θ ) is convex w.r.t . argument γi , ∆iγ is exactly 0 . The constant ∆ worst γ is the maximal divergence among all generators , which does not grow with the increase of I . This is because ∆worstγ measures the divergence of only one generator and when each generator for example has the same architecture , we have ∆worstγ = ∆ 1 γ = ... = ∆ I γ . Similarly , the terms ∆ minimax θ and ∆ maximin θ characterize the non-convexity of discriminator . When the discriminator is concave such as logistic regression and support vector machine , ∆minimaxθ = ∆ maximin θ = 0 and we have the following straightforward corollary about the minimax duality gap of Stackelberg GAN . Corollary 2 . Under the settings of Theorem 1 , when φ ( γi ; θ ) is concave and closed w.r.t . discriminator parameter θ and the number of generators I > t+1 ∆ worst γ , we have 0 ≤ w∗ − q∗ ≤ .

Summary: This paper proposes the Stackelberg GAN framework of multiple generators in the GAN architecture. The architecture is similar with previous multiple-generator GANs (MAD-GAN and MGAN). In fact, it's even simpler in the sense that Stackelberg GAN has simpler loss function for the discriminator compared with the previous two. The authors prove that the minimax duality gap shrinks as the number of generators increases. And this proof has no assumption on the expressive power of generators and discriminator. With this proof, the authors argues that because the duality gap shrinks as the number of generators increases, the training of GANs gets more stable.