The relativistic discriminator: a key element missing from standard GAN

1 INTRODUCTION . Generative adversarial networks ( GANs ) ( Hong et al. , 2017 ) form a broad class of generative models in which a game is played between two competing neural networks , the discriminator D and the generator G. D is trained to discriminate real from fake data , while G is trained to generate fake data that D will mistakenly recognize as real . In the original GAN by Goodfellow et al . ( 2014 ) , which we refer to as Standard GAN ( SGAN ) , D is a classifier , thus it is predicting the probability that the input data is real . When D is optimal , the loss function of SGAN is approximately equal to the Jensen–Shannon divergence ( JSD ) ( Goodfellow et al. , 2014 ) . SGAN has two variants for the generator loss functions : saturating and non-saturating . In practice , the former has been found to be very unstable , while the latter has been found to more stable ( Goodfellow et al. , 2014 ) . Under certain conditions , Arjovsky & Bottou ( 2017 ) proved that , if real and fake data are perfectly classified , the saturating loss has zero gradient and the non-saturating loss has non-zero , but volatile gradient . In practice , this means that the discriminator in SGAN often can not be trained to optimality or with a too high learning rate ; otherwise , gradients may vanish and , if so , training will stop . This problem is generally more noticeable in high-dimensional setting ( e.g. , high resolution images and discriminator architectures with high expressive power ) given that there are enough degrees of freedom available to perfectly classify the training set . To improve on SGAN , many GAN variants have been suggested using different loss functions and discriminators that are not classifiers ( e.g. , LSGAN ( Mao et al. , 2017 ) , WGAN ( Arjovsky et al. , 2017 ) ) . Although these approaches have partially succeeded in improving stability and data quality , the large-scale study by Lucic et al . ( 2017 ) suggests that these approaches do not consistently improve on SGAN . Additionally , some of the most successful approaches , such as WGAN-GP ( Gulrajani et al. , 2017 ) , are much more computationally demanding than SGAN . Many of the recent successful GANs variants have been based on Integral probability metrics ( IPMs ) ( Müller , 1997 ) ( e.g. , WGAN , WGAN-GP , Fisher GAN ( Mroueh & Sercu , 2017 ) , Sobolev GAN ( Mroueh et al. , 2017 ) ) . In IPM-based GANs , the discriminator is real-valued and constrained to a specific class of function which regularize the discriminator . See Mroueh et al . ( 2017 ) for a review of the different IPMs . These IPM constraints have been shown to be beneficial even in non-IPM based GANs . Spectral normalization ( Miyato et al. , 2018 ) improves the stability of various GANs and it consists in making the discriminator Lipschitz-1 , which is the constraint of WGAN . Similarly , the gradient penalty of WGAN-GP also provides improve the stability of SGAN ( Fedus et al. , 2017 ) . Although this shows that certain IPM constraints improve the stability of GANs , it does not explain why IPM-based GANs generally provide increased stability over other metrics/divergences in GANs ( e.g. , JSD for SGAN , f -divergences for f -GANs ( Nowozin et al. , 2016 ) ) . Note that although powerful , IPM-based GANs tend to more computationally demanding than other GANs . Certain IPM-based GANs use a gradient penalty ( e.g . WGAN-GP , Sobolev GAN ) which is very computationally costly and most IPM-based GANs need more than one discriminator update per generator update ( WGAN-GP requires at least 5 ( Gulrajani et al. , 2017 ) ) . Assuming equal training time for D and G , every additional discriminator update increase training time by a significant 50 % . In this paper , we argue that non-IPM-based GANs are missing a key ingredient , a relativistic discriminator , which IPM-based GANs already possess . We show that a relativistic discriminator is necessary to make GANs analogous to divergence minimization and produce sensible predictions based on the a priori knowledge that half of the samples in the mini-batch are fake . We provide empirical evidence showing that GANs with a relativistic discriminator are more stable and produce data of higher quality . 2 BACKGROUND . 2.1 GENERATIVE ADVERSARIAL NETWORKS . GANs can be defined very generally in terms of the discriminator in the following way : LD = Exr∼P [ f̃1 ( D ( xr ) ) ] + Ez∼Pz [ f̃2 ( D ( G ( z ) ) ) ] , ( 1 ) and LG = Exr∼P [ g̃1 ( D ( xr ) ) ] + Ez∼Pz [ g̃2 ( D ( G ( z ) ) ) ] , ( 2 ) where f̃1 , f̃2 , g̃1 , g̃2 are scalar-to-scalar functions , P is the distribution of real data , Pz is generally a multivariate normal distribution centered at 0 with variance 1 , D ( x ) is the discriminator evaluated at x , G ( z ) is the generator evaluated at z ( Q is the distribution of fake data , thus of G ( z ) ) . Note that , through the paper , we refer to real data as xr and fake data as xf . Without loss of generality , we assume that both LD and LG are loss functions to be minimized . Most GANs can be separated into two classes : non-saturating and saturating loss functions . GANs with the saturating loss are such that g̃1=−f̃1 and g̃2=−f̃2 , while GANs with the non-saturating loss are such that g̃1=f̃2 and g̃2=f̃1 . Saturating GANs are intuitive as they can be interpreted as alternating between maximizing and minimizing the same loss function . After training D to optimality , the loss function is generally an approximation of a divergence ( e.g. , Jensen–Shannon divergence ( JSD ) for SGAN ( Goodfellow et al. , 2014 ) , f -divergences for F-GANs ( Nowozin et al. , 2016 ) , and Wassertein distance for WGAN ( Arjovsky et al. , 2017 ) ) . Thus , training G to minimize LG can be roughly interpreted as minimizing the approximated divergence ( although this is not technically true ; see Jolicoeur-Martineau ( 2018 ) ) . On the other hand , non-saturating GANs can be thought as optimizing the same loss function , but swapping real data with fake data ( and vice-versa ) . In this article , unless otherwise specified , we assume a non-saturating loss for all GANs . SGAN assumes a cross-entropy loss , i.e. , f̃1 ( D ( x ) ) = − log ( D ( x ) ) and f̃2 ( D ( x ) ) = − log ( 1 − D ( x ) ) , whereD ( x ) = sigmoid ( C ( x ) ) , and C ( x ) is the non-transformed discriminator output ( which we call the critic as per Arjovsky et al . ( 2017 ) ) . In most GANs , C ( x ) can be interpreted as how realistic the input data is ; a negative number means that the input data looks fake ( e.g. , in SGAN , D ( x ) = sigmoid ( −5 ) = 0 ) , while a positive number means that the input data looks real ( e.g. , in SGAN , D ( x ) = sigmoid ( 5 ) = 1 ) . 2.2 INTEGRAL PROBABILITY METRICS . IPMs are statistical divergences represented mathematically as : IPMF ( P||Q ) = sup C∈F Ex∼P [ C ( x ) ] − Ex∼Q [ C ( x ) ] , where F is a class of real-valued functions . IPM-based GANs can be defined using equation 1 and 2 where f̃1 ( D ( x ) ) = g̃2 ( D ( x ) ) = −D ( x ) and f̃2 ( D ( x ) ) = g̃1 ( D ( x ) ) = D ( x ) , where D ( x ) = C ( x ) ( i.e. , no transformation is applied ) . It can be observed that both discriminator and generator loss functions are unbounded and would diverge to −∞ if optimized directly . However , IPMs assume that the discriminator is of a certain class of function that does not grow too quickly which prevent the loss functions from diverging . Each IPM applies a different constraint to the discriminator ( e.g. , WGAN assumes a Lipschitz D , WGAN-GP assumes that D has a gradient norm equal to 1 around real and fake data ) . 3 MISSING PROPERTY OF SGAN . We argue that the key missing property of SGAN is that the probability of real data being real ( D ( xr ) ) should decrease as the probability of fake data being real ( D ( xf ) ) increase . We provide three arguments suggesting that SGAN should have this property . 3.1 PRIOR KNOWLEDGE ARGUMENT . Assuming a rational human was shown half real data and half fake data . If they perceived all samples shown as equally real ( C ( xf ) ≈ C ( xr ) for most xr and xf ) , they would assume that each sample has probability .50 of being real . However , this is not the case for the discriminator in SGAN . If all samples looked real ( C ( xf ) ≈ C ( xr ) ≥ 3 ) , D would assume incorrectly that they are indeed all real ( D ( x ) ≈ 1 for all x ) . Of course , once trained with the labels , D would decrease D ( xf ) and thus would obtain more reasonable estimates . However , if D ( xr ) decreased as D ( xf ) increased , we would have had that D ( x ) ≈ .50 for all x before even retraining D. A rational human would not require retraining . IPM-based GANs implicitly account for the fact that some of the samples must be fake because they compare how realistic real data is compared to fake data . This is the behavior that we would want . 3.2 DIVERGENCE MINIMIZATION ARGUMENT . In SGAN , when optimized , we have that the discriminator loss function is equal to the Jensen–Shannon divergence ( JSD ) ( Goodfellow et al. , 2014 ) . The JSD is minimized ( JSD ( P||Q ) = 0 ) when D ( xr ) = D ( xf ) = 12 for all xr ∈ P and xf ∈ Q and maximized ( JSD ( P||Q ) = log ( 2 ) ) when D ( xr ) = 1 , D ( xf ) = 0 for all xr ∈ P and xf ∈ Q . Thus , if we were directly minimizing the divergence from maximum to minimum , we would expect D ( xr ) to smoothly decrease from 1 to .50 for most xr and D ( xf ) to smoothly increase from 0 to .50 for most xf ( Figure 1a ) . However , when minimizing the saturating loss in SGAN , we are only increasing D ( xf ) , we are not decreasing D ( xr ) ( Figure 1b ) . Furthermore , we are bringing D ( xf ) closer to 1 rather than .50 . This means that SGAN dynamics are very different from the minimization of the JSD . To bring SGAN closer to divergence minimization , training the generator should not only increase D ( xf ) but also decrease D ( xr ) ( Figure 1c ) . Note that although specific to the JSD , similar dynamics are true for other divergences ; when the divergence is maximal , D ( xr ) and D ( xf ) are very far from one another , but they converge to the same value as the divergence approach zero . Thus , this argument applies to other divergences .

Summary: In this work, the authors considers a variation of GAN by consider simultaneously decrease the probability that real data is real for the generator. To include such a property, the authors propose a relativistic discriminator which estimate the probability that the given real data is more realistic than the fake data. Numerical results are performed to show that the proposed methods are effective, and the resulting GANs are relatively more stable and generate higher quality data samples than their non-relativistic counterparts.