Generative Adversarial Networks (GAN) [6] are an effective method for training generative models of complex data such as natural images. However, they are notoriously hard to train and can suffer from the problem of missing modes where the model is not able to produce examples in certain regions of the space. We propose an iterative procedure, called AdaGAN, where at every step we add a new component into a mixture model by running a GAN algorithm on a reweighted sample. This is inspired by boosting algorithms, where many potentially weak individual predictors are greedily aggregated to form a strong composite predictor. We prove that such an incremental procedure leads to convergence to the true distribution in a finite number of steps if each step is optimal, and convergence at an exponential rate otherwise. We also illustrate experimentally that this procedure addresses the problem of missing modes. Algorithm 1: AdaGAN, a meta-algorithm to construct a "strong" mixture of T individual GANs, trained sequentially. The mixture weight schedule ChooseMixtureWeight and the training set reweighting schedule UpdateTrainingWeights should be provided by the user. Section 3 gives a complete instance of this family. Input: Training sample S N := {X 1 , . . . , X N }. Output: Mixture generative model G = G T . W 1 = (1/N, . . . , 1/N ) for t = 2, . . . , T do #Choose a mixture weight for the next component β t = ChooseMixtureWeight(t) #Update weights of training examples #Update the overall generative model #Notation below means forming a mixture of G t−1 and G c t . G t = (1 − β t )G t−1 + β t G c t end for
Inspired by boosting algorithms, AdaGAN REF proposes an iterative procedure, which incrementally addresses uncovered data modes by introducing new GAN components using the sample reweighting technique.
AdaGAN: Boosting Generative Models
ArXiv
ArXiv
Mathematics Computer Science