Discriminator Rejection Sampling

1 INTRODUCTION . Generative Adversarial Networks ( GANs ) ( Goodfellow et al. , 2014 ) are a powerful tool for image synthesis . They have also been applied successfully to semi-supervised and unsupervised learning ( Springenberg , 2015 ; Odena , 2016 ; Kumar et al. , 2017 ) , image editing ( Yu et al. , 2018 ; Ledig et al. , 2017 ) , and image style transfer ( Zhu et al. , 2017 ; Isola et al. , 2017 ; Yi et al. , 2017 ; Azadi et al. , 2018 ) . Informally , the GAN training procedure pits two neural networks against each other , a generator and a discriminator . The discriminator is trained to distinguish between samples from the target distribution and samples from the generator . The generator is trained to fool the discriminator into thinking its outputs are real . The GAN training procedure is thus a two-player differentiable game , and the game dynamics are largely what distinguishes the study of GANs from the study of other generative models . These game dynamics have well-known and heavily studied stability issues . Addressing these issues is an active area of research ( Mao et al. , 2017 ; Arjovsky et al. , 2017 ; Gulrajani et al. , 2017 ; Odena et al. , 2018 ; Li et al. , 2017 ) . However , we are interested in studying something different : Instead of trying to improve the training procedure , we ( temporarily ) accept its flaws and attempt to improve the quality of trained generators by post-processing their samples using information from the trained discriminator . It ’ s well known that ( under certain very strict assumptions ) the equilibrium of this training procedure is reached when sampling from the generator is identical to sampling from the target distribution and the discriminator always outputs 1/2 . However , these assumptions don ’ t hold in practice . In particular , GANs as presently trained don ’ t learn to reproduce the target distribution ( Arora & Zhang , 2017 ) . Moreover , trained GAN discriminators aren ’ t just identically 1/2 — they can even be used to perform chess-type skill ratings of other trained generators ( Olsson et al. , 2018 ) . We ask if the information retained in the weights of the discriminator at the end of the training procedure can be used to “ improve ” the generator . At face value , this might seem unlikely . After all , if there is useful information left in the discriminator , why doesn ’ t it find its way into the generator via the training procedure ? Further reflection reveals that there are many possible reasons . First , the assumptions made in various analyses of the training procedure surely don ’ t hold in practice ( e.g . the discriminator and generator have finite capacity and are optimized in parameter space rather than density-space ) . Second , due to the concrete realization of the discriminator and the generator as ∗Correspondence to sazadi @ cs.berkeley.edu neural networks , it may be that it is harder for the generator to model a given distribution than it is for the discriminator to tell that this distribution is not being modeled precisely . Finally , we may simply not train GANs long enough in practice for computational reasons . In this paper , we focus on using the discriminator as part of a probabilistic rejection sampling scheme . In particular , this paper makes the following contributions : • We propose a rejection sampling scheme using the GAN discriminator to approximately correct errors in the GAN generator distribution . • We show that under quite strict assumptions , this scheme allows us to recover the data distribution exactly . • We then examine where those strict assumptions break down and design a practical algorithm – called DRS – that takes this into account . • We conduct experiments demonstrating the effectiveness of DRS . First , as a baseline , we train an improved version of the Self-Attention GAN , improving its performance from the best published Inception Score of 52.52 up to 62.36 , and from a Fréchet Inception Distance of 18.65 down to 14.79 . We then show that DRS yields further improvement over this baseline , increasing the Inception Score to 76.08 and decreasing the Fréchet Inception Distance to 13.75 . 2 BACKGROUND . 2.1 GENERATIVE ADVERSARIAL NETWORKS . A generative adversarial network ( GAN ) ( Goodfellow et al. , 2014 ) consists of two separate neural networks — a generator , and a discriminator — trained in tandem . The generator G takes as input a sample from the prior z ∈ Z ∼ pz and produces a sample G ( z ) ∈ X . The discriminator takes an observation x ∈ X as input and produces a probability D ( x ) that the observation is real . The observation is sampled either according to the density pd ( the data generating distribution ) or pg ( the implicit density given by the generator and the prior ) . Using the standard non-saturating variant , the discriminator and generator are then trained using the following loss functions : LD = −Ex∼pdata [ logD ( x ) ] − Ez∼pz [ 1− logD ( G ( z ) ) ] LG = −Ez∼pz [ logD ( G ( z ) ) ] ( 1 ) 2.2 EVALUATION METRICS : INCEPTION SCORE ( IS ) AND FRÉCHET INCEPTION DISTANCE . ( FID ) The two most popular techniques for evaluating GANs on image synthesis tasks are the Inception Score and the Fréchet Inception Distance . The Inception Score ( Salimans et al. , 2016 ) is given by exp ( ExKL ( p ( y|x ) ||p ( y ) ) ) , where p ( y|x ) is the output of a pre-trained Inception classifier ( Szegedy et al. , 2014 ) . This measures the ability of the GAN to generate samples that the pre-trained classifier confidently assigns to a particular class , and also the ability of the GAN to generate samples from all classes . The Fréchet Inception Distance ( FID ) ( Heusel et al. , 2017 ) , is computed by passing samples through an Inception network to yield “ semantic embeddings ” , after which the Fréchet distance is computed between Gaussians with moments given by these embeddings . 2.3 SELF-ATTENTION GAN . We use a Self-Attention GAN ( SAGAN ) ( Zhang et al. , 2018 ) in our experiments . We do so because SAGAN is considered state of the art on the ImageNet conditional-image-synthesis task ( in which images are synthesized conditioned on class identity ) . SAGAN differs from a vanilla GAN in the following ways : First , it uses large residual networks ( He et al. , 2016 ) instead of normal convolutional layers . Second , it uses spectral normalization ( Miyato et al. , 2018 ) in the generator and the discriminator and a much lower learning rate for the generator than is conventional ( Heusel et al. , 2017 ) . Third , SAGAN makes use of self-attention layers ( Wang et al . ) , in order to better model long range dependencies in natural images . Finally , this whole model is trained using a special hinge version of the adversarial loss ( Lim & Ye , 2017 ; Miyato & Koyama , 2018 ; Tran et al. , 2017 ) : LD = −E ( x , y ) ∼pdata [ min ( 0 , −1 +D ( x , y ) ) ] − Ez∼pz , y∼pdata [ min ( 0 , −1−D ( G ( z ) , y ) ) ] LG = −Ez∼pz , y∼pdata [ D ( G ( z ) , y ) ) ] ( 2 ) 2.4 REJECTION SAMPLING . Rejection sampling is a method for sampling from a target distribution pd ( x ) which may be hard to sample from directly . Samples are instead drawn from a proposal distribution pg ( x ) , which is easier to sample from , and which is chosen such that there exists a finite value M such that Mpg ( x ) > pd ( x ) for ∀x ∈ domain ( pd ( x ) ) . A given sample y drawn from pg is kept with acceptance probability pd ( y ) /Mpg ( y ) , and rejected otherwise . See the blue points in Figure 1 ( Left ) for a visualization . Ideally , pg ( x ) should be close to pd ( x ) , otherwise many samples will be rejected , reducing the efficiency of the algorithm ( MacKay , 2003 ) . In Section 3 , we explain how to apply this rejection sampling algorithm to the GAN framework : in brief , we draw samples from the trained generator , pg ( x ) , and then reject some of those samples using the discriminator to attain a closer approximation to the true data distribution , pd ( x ) . An independent rejection sampling approach was proposed by Grover et al . ( 2018 ) in the latent space of variational autoencoders for improving samples from the variational posterior . 3 REJECTION SAMPLING FOR GANS . In this section we introduce our proposed rejection sampling scheme for GANs ( which we call Discriminator Rejection Sampling , or DRS ) . We ’ ll first derive an idealized version of the algorithm that will rely on assumptions that don ’ t necessarily hold in realistic settings . We ’ ll then discuss the various ways in which these assumptions might break down . Finally , we ’ ll describe the modifications we made to the idealized version in order to overcome these challenges . 3.1 REJECTION SAMPLING FOR GANS : THE IDEALIZED VERSION . Suppose that we have a GAN and our generator has been trained to the point that pg and pd have the same support . That is , for all x ∈ X , pg ( x ) 6= 0 if and only if pd ( x ) 6= 0 . If desired , we can make pd and pg have support everywhere inX if we add low-variance Gaussian noise to the observations . Now further suppose that we have some way to compute pd ( x ) /pg ( x ) . Then , if M = maxx pd ( x ) /pg ( x ) , then Mpg ( x ) > pd ( x ) for all x , so we can perform rejection sampling with pg as the proposal distribution and pd as the target distribution as long as we can evaluate the quantity pd ( x ) /Mpg ( x ) 1 . In this case , we can exactly sample from pd ( Casella et al. , 2004 ) , though we may have to reject many samples to do so . But how can we evaluate pd ( x ) /Mpg ( x ) ? pg is defined only implicitly . One thing we can do is to borrow an analysis from the original GAN paper ( Goodfellow et al. , 2014 ) , which assumes that we can optimize the discriminator in the space of density functions rather than via changing its parameters . If we make this assumption , as well as the assumption that the discriminator is defined by a sigmoid applied to some function of x and trained with a cross-entropy loss , then by Proposition 1 of that paper , we have that , for any fixed generator and in particular for the generator G that we have when we stop training , training the discriminator to completely minimize its own loss yields D∗ ( x ) = pd ( x ) pd ( x ) + pg ( x ) ( 3 ) We will discuss the validity of these assumptions later , but for now consider that this allows us to solve for pd ( x ) /pg ( x ) as follows : As noted above , we can assume the discriminator is defined as : D ( x ) = σ ( x ) = 1 1 + e−D̃ ( x ) , ( 4 ) where D ( x ) is the final discriminator output after the sigmoid , and D̃ ( x ) is the logit . Thus , D∗ ( x ) = 1 1 + e−D̃∗ ( x ) = pd ( x ) pd ( x ) + pg ( x ) 1 + e−D̃ ∗ ( x ) = pd ( x ) + pg ( x ) pd ( x ) pd ( x ) + pd ( x ) e −D̃∗ ( x ) = pd ( x ) + pg ( x ) pd ( x ) e −D̃∗ ( x ) = pg ( x ) pd ( x ) pg ( x ) = eD̃ ∗ ( x ) ( 5 ) Now suppose one last thing , which is that we can tractably compute M = maxx pd ( x ) /pg ( x ) . We would find thatM = pd ( x∗ ) /pg ( x∗ ) = eD̃ ∗ ( x∗ ) for some ( not necessarily unique ) x∗ . Given all these assumptions , we can now perform rejection sampling as promised . If we define D̃∗M : = D̃ ∗ ( x∗ ) , then for any input x , the acceptance probability pd ( x ) /Mpg ( x ) can be written as eD̃ ∗ ( x ) −D̃∗M ∈ [ 0 , 1 ] . To decide whether to keep any particular example , we can just draw a random number ψ uniformly from [ 0 , 1 ] and accept the sample if ψ < eD̃ ∗ ( x ) −D̃∗M .

Summary: This paper proposed a post-processing rejection sampling scheme for GANs, named Discriminator Rejection Sampling (DRS), to help filter ‘good’ samples from GANs’ generator. More specifically, after training GANs’ generator and discriminator are fixed; GANs’ discriminator is further exploited to design a rejection sampler, which is used to reject the ‘bad’ samples generated from the fixed generator; accordingly, the accepted generated samples have good quality (better IS and FID results). Experiments of SAGAN model on GMM toys and ImageNet dataset show that DRS helps further increases the IS and reduces the FID.