Assisting the Adversary to Improve GAN Training

1 INTRODUCTION . The generative adversarial network ( GAN ) framework ( Goodfellow et al. , 2014 ) trains a neural network known as a generator which maps from a random vector to an output such as an image . Key to training is another neural network , the adversary ( sometimes called a discriminator or critic ) , which is trained to distinguish between “ true ” and generated data . This is done by maximizing one of the many different objectives proposed in the literature ; see for instance Goodfellow et al . ( 2014 ) ; Arjovsky et al . ( 2017 ) ; Nowozin et al . ( 2016 ) . The generator directly competes against the adversary : it is trained to minimize the same objective , which it does by making the generated data more similar to the true data . GANs are efficient to sample from , requiring a single pass through a deep network , and highly flexible , as they do not require an explicit likelihood . They are especially suited to producing photo-realistic images ( Zhou et al. , 2019 ) compared to competing methods like normalizing flows , which impose strict requirements on the neural network architecture ( Kobyzev et al. , 2020 ; Rezende & Mohamed , 2015 ) and VAEs ( Kingma & Welling , 2014 ; Razavi et al. , 2019 ; Vahdat & Kautz , 2020 ) . Counterbalancing their appealing properties , GANs can have unstable training dynamics ( Kurach et al. , 2019 ; Goodfellow , 2017 ; Kodali et al. , 2017 ; Mescheder et al. , 2018 ) . Substantial research effort has been directed towards improving the training of GANs . These endeavors can generally be divided into two camps , albeit with significant overlap . The first develops better learning objectives for the generator/adversary to minimize/maximize . These are designed to have properties which improve training ( Arjovsky et al. , 2017 ; Li et al. , 2017 ; Nowozin et al. , 2016 ) . The other camp develops techniques to regularize the adversary and improve its training dynamics ( Kodali et al. , 2017 ; Roth et al. , 2017 ; Miyato et al. , 2018 ) . The adversary can then provide a better learning signal for the generator . Despite these contributions , stabilizing the training of GANs remains unsolved and continues to be an active research area . An overlooked approach is to train the generator in a way that accounts for the adversary not being trained to convergence . One such approach was introduced by Mescheder et al . ( 2017 ) and later built on by Nagarajan & Kolter ( 2017 ) . The proposed method is a regularization term based on the norm of the gradients used to train the adversary . This is motivated as a means to improve the convergence properties of the minimax game . The purpose of this paper is to provide a new perspective as to why this regularizer is appropriate . Our perspective differs in that we view it as promoting updates that lead to a solution that satisfies a sufficient condition for the adversary to be optimal . To be precise , it encourages the generator to move towards points where the adversary ’ s current parameters are optimal . Informally , this regularizer “ assists ” the adversary , and for this reason we refer to this regularization method as the Adversary ’ s Assistant ( AdvAs ) . We additionally propose a version of AdvAs which is hyperparameter-free . Furthermore , we release a library which makes it simple to integrate into existing code . We demonstrate its application to a standard architecture with the WGAN-GP objective ( Arjovsky et al. , 2017 ; Gulrajani et al. , 2017 ) ; the state-of-the-art StyleGAN2 architecture and objective introduced by Karras et al . ( 2020 ) ; and the AutoGAN architecture and objective introduced by Gong et al . ( 2019 ) . We test these on the MNIST ( Lecun et al. , 1998 ) , CelebA ( Liu et al. , 2015 ) , CIFAR10 ( Krizhevsky et al. , 2009 ) datasets respectively . We show that AdvAs improves training on all datasets , as measured by the Fréchet Inception Distance ( FID ) ( Heusel et al. , 2017 ) , and the inception score ( Salimans et al. , 2016 ) where applicable . 2 BACKGROUND . A generator is a neural network g : Z → X ⊆ Rdx which maps from a random vector z ∈ Z to an output x ∈ X ( e.g. , an image ) . Due to the distribution over z , the function g induces a distribution over its output x = g ( z ) . If g is invertible and differentiable , the probability density function ( PDF ) over x from this “ change of variables ” could be computed . This is not necessary for training GANs , meaning that no such restrictions need to be placed on the neural network g. We denote this distribution pθ ( x ) where θ ∈ Θ ⊆ Rdg denotes the generator ’ s parameters . The GAN is trained on a dataset x1 , . . . , xN , where each xi is in X . We assume that this is sampled i.i.d . from a data-generating distribution ptrue . Then the aim of training is to learn θ so that pθ is as close as possible to ptrue . Section 2.1 will make precise what is meant by “ close. ” The adversary aφ : X → A has parameters φ ∈ Φ ⊆ Rda which are typically trained alternately with the generator . It receives as input either the data or the generator ’ s outputs . The set that it maps to , A , is dependent on the GAN type . For example , Goodfellow et al . ( 2014 ) define an adversary which maps from x ∈ X to the probability that x is a “ real ” data point from the dataset , as opposed to a “ fake ” from the generator . They therefore choose A to be [ 0 , 1 ] and train the adversary by maximizing the associated log-likelihood objective , h ( pθ , aφ ) = Ex∼ptrue [ log aφ ( x ) ] + Ex∼pθ [ log ( 1− aφ ( x ) ) ] . ( 1 ) Using the intuition that the generator should generate samples that seem real and therefore “ fool ” the adversary , the generator is trained to minimize h ( pθ , aφ ) . Since we find θ to minimize this objective while fitting φ to maximize it , training a GAN is equivalent to solving the minimax game , min θ max φ h ( pθ , aφ ) . ( 2 ) Eq . ( 1 ) gives the original form for h ( pθ , aφ ) used by Goodfellow et al . ( 2014 ) but this form varies between different GANs , as we will discuss in Section 2.1 . The minimization and maximization in Eq . ( 2 ) are performed with gradient descent in practice . To be precise , we define Lgen ( θ , φ ) = h ( pθ , aφ ) and Ladv ( θ , φ ) = −h ( pθ , aφ ) . These are treated as losses for the generator and adversary respectively , and both are minimized . In other words , we turn the maximization of h ( pθ , aφ ) w.r.t . φ into a minimization of Ladv ( θ , φ ) . Then on each iteration , θ and φ are updated one after the other using gradient descent steps along their respective gradients : ∇θLgen ( θ , φ ) = ∇θh ( pθ , aφ ) , ( 3 ) ∇φLadv ( θ , φ ) = −∇φh ( pθ , aφ ) . ( 4 ) 2.1 GANS MINIMIZE DIVERGENCES . A common theme in the GAN literature is analysis based on what we call the optimal adversary assumption . This is the assumption that , before each generator update , we have found the adversary aφ which maximizes h ( pθ , aφ ) given the current value of θ . To be precise , we define a class of permissible adversary functions F . This is often simply the space of all functions mapping X → A ( Goodfellow et al. , 2014 ) , but is in some GAN variants constrained by , e.g. , a Lipschitz constant ( Arjovsky et al. , 2017 ) . Then we call the adversary aφ optimal for a particular value of θ if and only if h ( pθ , aφ ) = maxa∈F h ( pθ , a ) . In practice , the neural network aφ can not represent every a ∈ F and so it may not be able to parameterize an optimal adversary for a given θ . As is common in the literature , we assume that the neural network is expressive enough that this is not an issue , i.e. , we assume that for any θ , there exists at least one φ ∈ Φ resulting in an optimal adversary . Then , noting that there may be multiple such φ ∈ Φ , we define Φ∗ ( θ ) to be the set of all optimal adversary parameters . That is , Φ∗ ( θ ) = { φ ∈ Φ | h ( pθ , aφ ) = maxa∈F h ( pθ , a ) } and the optimal adversary assumptions says that before each update of θ we have found φ ∈ Φ∗ ( θ ) . We emphasize that , in part due to the limited number of gradient updates performed on φ , this assumption is essentially never true in practice . This paper presents a method to improve the training of GANs by addressing this issue . The optimal adversary assumption simplifies analysis of GAN training considerably . Instead of being a two-player game , it turns into a case of minimizing an objective with respect to θ alone . We denote this objective M ( pθ ) = max a∈F h ( pθ , a ) = h ( pθ , aφ∗ ) where φ∗ ∈ Φ∗ ( θ ) . ( 5 ) For example , Goodfellow et al . ( 2014 ) showed that using the objective presented in Eq . ( 1 ) results in M ( pθ ) = 2 · JSD ( ptrue||pθ ) − log 4 , where JSD is the Jensen-Shannon divergence . By making the optimal adversary assumption , they could prove that their GAN training procedure would converge , and would minimize the Jensen-Shannon divergence between ptrue and pθ . A spate of research following the introduction of the original GAN objective has similarly made use of the optimal adversary assumption to propose GANs which minimize different divergences . For example , Wasserstein GANs ( WGANs ) ( Arjovsky et al. , 2017 ) minimize a Wasserstein distance . MMD GANs ( Li et al. , 2017 ) minimize a distance known as the maximum mean discrepancy . Nowozin et al . ( 2016 ) introduce f-GANs which minimize f-divergences , a class including the Kullback-Leibler and Jensen-Shannon divergences . We emphasize that this is by no means an exhaustive list . Like these studies , this paper is motivated by the perspective that , under the optimal adversary assumption , GANs minimize a divergence . However , the GAN framework can also be viewed from a more game-theoretic perspective ( Kodali et al. , 2017 ; Grnarova et al. , 2018 ) .

Summary: This paper concerns how to efficiently regularize the generatior for training generative adversarial networks (GANs). A new regularizer for the generator loss is proposed to penalize the norm of the gradient with respect to discriminator’s parameters ($\phi$). In other words, the generator learns to encourage small norm of the discriminator’s gradient w.r.t $\phi$. The author(s) also propose a heuristic to remove the introduction of a further hyperparameter. The author(s) applied the proposed regularizer to WGAN-GP, AutoGAN, and StyleGAN2 to validate its effectiveness. Their experiments reveal that the new regularizer is promising.