Assisting the Adversary to Improve GAN Training

1 INTRODUCTION . The generative adversarial network ( GAN ) framework ( Goodfellow et al. , 2014 ) trains a neural network known as a generator which maps from a random vector to an output such as an image . Key to training is another neural network , the adversary ( sometimes called a discriminator or critic ) , which is trained to distinguish between “ true ” and generated data . This is done by maximizing one of the many different objectives proposed in the literature ; see for instance Goodfellow et al . ( 2014 ) ; Arjovsky et al . ( 2017 ) ; Nowozin et al . ( 2016 ) . The generator directly competes against the adversary : it is trained to minimize the same objective , which it does by making the generated data more similar to the true data . GANs are efficient to sample from , requiring a single pass through a deep network , and highly flexible , as they do not require an explicit likelihood . They are especially suited to producing photo-realistic images ( Zhou et al. , 2019 ) compared to competing methods like normalizing flows , which impose strict requirements on the neural network architecture ( Kobyzev et al. , 2020 ; Rezende & Mohamed , 2015 ) and VAEs ( Kingma & Welling , 2014 ; Razavi et al. , 2019 ; Vahdat & Kautz , 2020 ) . Counterbalancing their appealing properties , GANs can have unstable training dynamics ( Kurach et al. , 2019 ; Goodfellow , 2017 ; Kodali et al. , 2017 ; Mescheder et al. , 2018 ) . Substantial research effort has been directed towards improving the training of GANs . These endeavors can generally be divided into two camps , albeit with significant overlap . The first develops better learning objectives for the generator/adversary to minimize/maximize . These are designed to have properties which improve training ( Arjovsky et al. , 2017 ; Li et al. , 2017 ; Nowozin et al. , 2016 ) . The other camp develops techniques to regularize the adversary and improve its training dynamics ( Kodali et al. , 2017 ; Roth et al. , 2017 ; Miyato et al. , 2018 ) . The adversary can then provide a better learning signal for the generator . Despite these contributions , stabilizing the training of GANs remains unsolved and continues to be an active research area . An overlooked approach is to train the generator in a way that accounts for the adversary not being trained to convergence . One such approach was introduced by Mescheder et al . ( 2017 ) and later built on by Nagarajan & Kolter ( 2017 ) . The proposed method is a regularization term based on the norm of the gradients used to train the adversary . This is motivated as a means to improve the convergence properties of the minimax game . The purpose of this paper is to provide a new perspective as to why this regularizer is appropriate . Our perspective differs in that we view it as promoting updates that lead to a solution that satisfies a sufficient condition for the adversary to be optimal . To be precise , it encourages the generator to move towards points where the adversary ’ s current parameters are optimal . Informally , this regularizer “ assists ” the adversary , and for this reason we refer to this regularization method as the Adversary ’ s Assistant ( AdvAs ) . We additionally propose a version of AdvAs which is hyperparameter-free . Furthermore , we release a library which makes it simple to integrate into existing code . We demonstrate its application to a standard architecture with the WGAN-GP objective ( Arjovsky et al. , 2017 ; Gulrajani et al. , 2017 ) ; the state-of-the-art StyleGAN2 architecture and objective introduced by Karras et al . ( 2020 ) ; and the AutoGAN architecture and objective introduced by Gong et al . ( 2019 ) . We test these on the MNIST ( Lecun et al. , 1998 ) , CelebA ( Liu et al. , 2015 ) , CIFAR10 ( Krizhevsky et al. , 2009 ) datasets respectively . We show that AdvAs improves training on all datasets , as measured by the Fréchet Inception Distance ( FID ) ( Heusel et al. , 2017 ) , and the inception score ( Salimans et al. , 2016 ) where applicable . 2 BACKGROUND . A generator is a neural network g : Z → X ⊆ Rdx which maps from a random vector z ∈ Z to an output x ∈ X ( e.g. , an image ) . Due to the distribution over z , the function g induces a distribution over its output x = g ( z ) . If g is invertible and differentiable , the probability density function ( PDF ) over x from this “ change of variables ” could be computed . This is not necessary for training GANs , meaning that no such restrictions need to be placed on the neural network g. We denote this distribution pθ ( x ) where θ ∈ Θ ⊆ Rdg denotes the generator ’ s parameters . The GAN is trained on a dataset x1 , . . . , xN , where each xi is in X . We assume that this is sampled i.i.d . from a data-generating distribution ptrue . Then the aim of training is to learn θ so that pθ is as close as possible to ptrue . Section 2.1 will make precise what is meant by “ close. ” The adversary aφ : X → A has parameters φ ∈ Φ ⊆ Rda which are typically trained alternately with the generator . It receives as input either the data or the generator ’ s outputs . The set that it maps to , A , is dependent on the GAN type . For example , Goodfellow et al . ( 2014 ) define an adversary which maps from x ∈ X to the probability that x is a “ real ” data point from the dataset , as opposed to a “ fake ” from the generator . They therefore choose A to be [ 0 , 1 ] and train the adversary by maximizing the associated log-likelihood objective , h ( pθ , aφ ) = Ex∼ptrue [ log aφ ( x ) ] + Ex∼pθ [ log ( 1− aφ ( x ) ) ] . ( 1 ) Using the intuition that the generator should generate samples that seem real and therefore “ fool ” the adversary , the generator is trained to minimize h ( pθ , aφ ) . Since we find θ to minimize this objective while fitting φ to maximize it , training a GAN is equivalent to solving the minimax game , min θ max φ h ( pθ , aφ ) . ( 2 ) Eq . ( 1 ) gives the original form for h ( pθ , aφ ) used by Goodfellow et al . ( 2014 ) but this form varies between different GANs , as we will discuss in Section 2.1 . The minimization and maximization in Eq . ( 2 ) are performed with gradient descent in practice . To be precise , we define Lgen ( θ , φ ) = h ( pθ , aφ ) and Ladv ( θ , φ ) = −h ( pθ , aφ ) . These are treated as losses for the generator and adversary respectively , and both are minimized . In other words , we turn the maximization of h ( pθ , aφ ) w.r.t . φ into a minimization of Ladv ( θ , φ ) . Then on each iteration , θ and φ are updated one after the other using gradient descent steps along their respective gradients : ∇θLgen ( θ , φ ) = ∇θh ( pθ , aφ ) , ( 3 ) ∇φLadv ( θ , φ ) = −∇φh ( pθ , aφ ) . ( 4 ) 2.1 GANS MINIMIZE DIVERGENCES . A common theme in the GAN literature is analysis based on what we call the optimal adversary assumption . This is the assumption that , before each generator update , we have found the adversary aφ which maximizes h ( pθ , aφ ) given the current value of θ . To be precise , we define a class of permissible adversary functions F . This is often simply the space of all functions mapping X → A ( Goodfellow et al. , 2014 ) , but is in some GAN variants constrained by , e.g. , a Lipschitz constant ( Arjovsky et al. , 2017 ) . Then we call the adversary aφ optimal for a particular value of θ if and only if h ( pθ , aφ ) = maxa∈F h ( pθ , a ) . In practice , the neural network aφ can not represent every a ∈ F and so it may not be able to parameterize an optimal adversary for a given θ . As is common in the literature , we assume that the neural network is expressive enough that this is not an issue , i.e. , we assume that for any θ , there exists at least one φ ∈ Φ resulting in an optimal adversary . Then , noting that there may be multiple such φ ∈ Φ , we define Φ∗ ( θ ) to be the set of all optimal adversary parameters . That is , Φ∗ ( θ ) = { φ ∈ Φ | h ( pθ , aφ ) = maxa∈F h ( pθ , a ) } and the optimal adversary assumptions says that before each update of θ we have found φ ∈ Φ∗ ( θ ) . We emphasize that , in part due to the limited number of gradient updates performed on φ , this assumption is essentially never true in practice . This paper presents a method to improve the training of GANs by addressing this issue . The optimal adversary assumption simplifies analysis of GAN training considerably . Instead of being a two-player game , it turns into a case of minimizing an objective with respect to θ alone . We denote this objective M ( pθ ) = max a∈F h ( pθ , a ) = h ( pθ , aφ∗ ) where φ∗ ∈ Φ∗ ( θ ) . ( 5 ) For example , Goodfellow et al . ( 2014 ) showed that using the objective presented in Eq . ( 1 ) results in M ( pθ ) = 2 · JSD ( ptrue||pθ ) − log 4 , where JSD is the Jensen-Shannon divergence . By making the optimal adversary assumption , they could prove that their GAN training procedure would converge , and would minimize the Jensen-Shannon divergence between ptrue and pθ . A spate of research following the introduction of the original GAN objective has similarly made use of the optimal adversary assumption to propose GANs which minimize different divergences . For example , Wasserstein GANs ( WGANs ) ( Arjovsky et al. , 2017 ) minimize a Wasserstein distance . MMD GANs ( Li et al. , 2017 ) minimize a distance known as the maximum mean discrepancy . Nowozin et al . ( 2016 ) introduce f-GANs which minimize f-divergences , a class including the Kullback-Leibler and Jensen-Shannon divergences . We emphasize that this is by no means an exhaustive list . Like these studies , this paper is motivated by the perspective that , under the optimal adversary assumption , GANs minimize a divergence . However , the GAN framework can also be viewed from a more game-theoretic perspective ( Kodali et al. , 2017 ; Grnarova et al. , 2018 ) .

Summary: This paper proposes a new regularizer to improve GAN training. By noticing that the discriminator does not always reach optimum at each iteration, this paper proposes Adversary's Assistant (AdvAs) for helping the discriminator to satisfy this condition. Interestingly, compared to the previous methods for improving GAN training, this work applies the regularizer at the generator (rather than the discriminator) and is theoretical motivated. Experiments on several GAN objectives, datasets and network architectures are provided to support the effectiveness of AdvAs.