Incremental training of multi-generative adversarial networks

1 INTRODUCTION . Generative Adversarial Nets ( GAN ) Goodfellow et al . ( 2014 ) is a framework of estimating generative models . The main idea Goodfellow ( 2017 ) is to train two target network models simultaneously , in which one , called the generator , aims to generate samples that resemble those from the data distribution , while the other , called the discriminator , aims to distinguish the samples by the generator from the real data . Naturally , this type of training framework admits a nice interpretation as a twoperson zero-sum game and interesting game theoretical properties , such as uniqueness of the optimal solution , have been derived Goodfellow et al . ( 2014 ) . It is further proved that such adversarial process minimizes certain divergences , such as Shannon divergence , between the generated distribution and the data distribution . Simply put , the goal of training a GAN is to search for a distribution in the range of the generator that best approximates the data distribution . The range is often defined by the input latent variable z and its specific architecture , i.e. , Π = { G ( z , θ ) , θ ∈ Θ } . When the range is general enough , one could possibly find the real data distribution . However , in practice , the range is usually insufficient to perfectly describe the real data , which is typically of high dimension . As a result , what we search for is in fact the I-projection Csiszár & Shields ( 2004 ) of the real data distribution on Π . Consider two cases : 1 . The range of the generator Π is convex ( see figure 1 ( a ) ) , or it is not convex but the projection of the real data distribution on Π ’ s convex hull ( CONV Π ) is in Π ( see figure 1 ( b ) ) . 2 . The range of the generator is non-convex and the projection of the real data distribution in CONV Π is not in the range Π ( see figure 1 ( c ) ) . In case 1 , one can find the optimal distribution in Π to approximate real data set in CONV Π . But in case 2 , using standard GANs with a single generator , one can only find the distribution in Π that is nearest to the projection . It then makes sense to train multiple generators and use a convex combination of them to better approximate the data distribution ( than using a single generator in the non-convex case ( see figure 1 ( c ) ) ) . The above argument is based on the assumption that one could achieve global optimality by training , while this is not the case in general . When reaching a local optimal distribution , in order to improve performance , do we need to add more generators and restart training ? In this paper , we put forward a sequential training procedure that adds generators one by one to improve the performance , without retraining the previously added generators . Our contributions can be summarized as follows . • We derive an objective function tailored for such a incremental training process . The objective function takes both the real data distribution and the pre-learned distribution into consideration . We show that with this new objective , we actually maximize marginal contribution when adding a new generator . We also put forward an incremental training algorithm based on the new objective function . • We prove that our algorithm always converges to the projection of real data distribution to the convex hull of the ranges of generators , which is the optimal solution with multiple generators . This property continues to hold in online settings where target distribution changes dynamically . • Our experiments show that our algorithm can overcome the local optimal issue mentioned above . We perform experiments on a synthetic dataset as well as two real world datasets , e.g. , CelebA and MNIST , and conclude that our algorithm could improve the mixture distribution even in the case where the range is not sufficient enough . • Experiments also show that , compared with previous methods , our algorithm is fast and stable in reducing the divergence between mixture distribution and the real data . 1.1 RELATED WORKS . Recently , there have been intensive researches on improving the performance of generative adversarial neural networks . Two lines of works are closely related to our paper . They focus mainly on improving the discriminator and the generator respectively . 1.1.1 IMPROVING THE DISCRIMINATOR . The Unrolled GAN introduced by Metz et al . ( 2016 ) improves the discriminator by unrolling optimizing the objective during training , which stabilizes training and effectively reduces the mode collapse . D2GAN proposed by Nguyen et al . ( 2017 ) utilizes two discriminators to minimize the KL-divergence and the reverse KL-divergence respectively . It treats different modes more fairly , and thus avoids mode collapse . DFM introduced by Warde-Farley & Bengio ( 2016 ) brings a Denoising AutoEncoder ( DAE ) into the generator ’ s objective to minimize the reconstruction error in order to get more information from the target manifold . Mroueh et al . ( 2017 ) proposed McGan based on mean and covariance feature matching to stabilize the training of GANs . Finally , WGAN introduced by Arjovsky et al . ( 2017 ) employs the Wasserstein distance , which is a more appropriate measure of performance , and achieves more stable performance . These works are different from ours since they focus on the discriminator by measuring the divergence between the generated data and the real data more precisely . However , our work fixes the discriminator and tries to enrich the expressiveness of the generator by combining multiple generators . 1.1.2 IMPROVING THE GENERATOR . Wang et al . ( 2016 ) proposes two methods to improve the training process . The first is selfensembling GANs , which assembles the generators from different epochs to stabilize training . The other is Cascade GAN , where the authors train new generator using the data points with highest values from the discriminator . These two methods are heuristic ways to improve training , but with no theoretical guarantee . Hoang et al . ( 2017 ) and Ghosh et al . ( 2017 ) proposed the methods called MGAN and multi-agent GANs respectively . The former introduces a classifier into the discriminator to catch different modes , while the later employ a new component into the generators ’ objective to promote diversity . Arora et al . ( 2017 ) introduces a new metric on distributions and proposes a MIX+GAN to search for an equilibrium . But all these methods need to train the multiple generators simultaneously , and none of them can deal with the case when the training process reaches a local optima . Also , these models lack flexibility , in the sense that when one tries to change the number of generators , all the generators need to be retrained . Another closely related work is Tolstikhin et al . ( 2017 ) , in which the authors propose a method called AdaGAN , which is based on a robust reweighting scheme on the data set inspired from boosting . The idea is that the new generators should focus more on the previous bad training data . But AdaGAN and other boosting-like algorithms are based on the assumption that one generator could catch some modes precisely , which may not be reasonable since the generator always learns to generate the average samples among the real data set in order to obtain low divergence , especially when the generator ’ s range is under condition of Figure 1 ( c ) . In Section 5 , we compare our algorithm with AdaGAN with different dataset . 2 PRELIMINARIES . A GAN Goodfellow et al . ( 2014 ) takes samples ( a.k.a . latent variables z ) from a simple and standard distribution as its input and generates samples in a high dimensional space to approximate the target distribution . This is done by training a generative neural network and an auxiliary discriminative neural network alternatively . An f-GAN Nowozin et al . ( 2016 ) generalizes the adversarial training as minimizing the f-divergence between the real data distribution and the generated distribution , defined as Df ( p‖q ) = ∫ x q ( x ) f ( p ( x ) q ( x ) ) dx . A GAN is a special f-GAN that minimizes the JensenShannon divergence . The general objective function of an f-GAN can be defined as follows : min θ max ξ F ( θ , ξ ) = Ex∼pdata [ Tξ ( x ) ] + Ex∼pθ [ −f∗ ( Tξ ( x ) ) ] . Here f∗ is the conjugate function of f in f-divergence ; T represents a neural network regarded as the corresponding discriminator ; finally , θ and ξ denote the parameters of the generator and the discriminator , respectively . 2.1 GENERATOR GROUP . The adversarial training method proposed by Goodfellow et al . ( 2014 ) is playing a minimax game between the generator and the discriminator . Such a method can be caught in local optima and thus is undesirable ( e.g. , mode collapse ) . In this paper we propose a novel framework to train multiple generators sequentially : We maintain a group of generators ( empty at the beginning ) as well as their corresponding weights , then add new generators into the group one by one and rebalance the weights . In particular , only the newly added generator at each step is trained . The purpose here is to augment the capacity of the group of generators and mitigate the local optima issue . Define the distribution range of a generator as Π = { p | p = G ( z , θ ) , θ ∈ Θ } , i.e. , the set of distributions that the generator can produce with different parameter θ . The distribution range is determined by the distribution of input z and the architecture of the generative network . Define a generator group as G = { G1 , G2 , , . . . , Gn } , where Gi is the generator added in step i . We associate each generator with a weight ωi > 0 . Then the mixed distribution of the group is : ppre ( x ) = 1Φn ∑n i=1 ωipi ( x ) , where pi ( x ) = Gi ( z ) and Φn = ∑n i=1 ωi is the sum of weights . When a new generator Gn+1 joins in the group G , the group becomes G′ = G ∪ { Gn+1 } and the mixed distribution becomes pnow ( x ) = 1 Φn+1 n+1∑ i=1 ωipi ( x ) = 1 Φn+1 ( Φn · ppre ( x ) + ωn+1 · pn+1 ( x ) ) . 3 AN INCREMENTAL TRAINING FRAMEWORK . In this section , we describe how we use a generator group to improve the performance and tackle the local optima issue mentioned previously . To train such a generator group , we propose an incremental training algorithm ( algorithm 1 ) adding generators to the group sequentially . In algorithm 1 , we use Algorithm 1 Incremental Training Algorithm Predetermined : preal , pz , Gi ( z ) , i ∈ { 1 , 2 , . . . , N } ωi , i ∈ { 1 , 2 , . . . , N + 1 } Input preal , pz , ωi , i > 0 . Output generator group G. Initialize i← 1. repeat Build and initialize generator Gi using the same network structure . Set target distribution for Gi to be ptarget = Φi·preal− ∑ j < i ωjpj ωi . Train Gi to minimize D ( ptarget , pi ) . i← i+ 1. until Convergence D ( · , · ) to denote the “ distance ” between two distributions , which can be any divergence ( e.g. , fdivergence or Wasserstein distance ) or a general norm . The key step in algorithm 1 is the choice of the target distribution for training Gi . Ideally , if D ( ptarget , pi ) = 0 , then we have pi = ptarget . In this case , preal = 1Φi ∑i j=1 ωjpj and after adding Gi , the generator group G can perfectly produce the desired distribution preal . However , in general , we have D ( ptarget , pi ) 6= 0 and our algorithm proceeds in a greedy fashion , i.e. , it always maximizes the marginal contribution of Gi to the generator group G. We devote the rest of this section to proving the above statement .

Summary: In this work, the authors propose to use multiple generators to estimate the target distribution. Especially, it assumes the case that the range of generators is non-convex and the target distribution doesn't fall into it. To solve this issue, the multiple generators are convex combined to do better approximation and an incremental training process is proposed to train multiple generators one by one.