 The relativistic discriminator: a key element missing from standard GAN
In standard generative adversarial network, the discriminator estimates the probability that the input data is real.The generator is trained to increase the probability that fake data is real.We argue that it should also simultaneously decrease the probability that real data is real because1) this would account for a priori knowledge that half of the data in the mini-batch is fake,2) this would be observed with divergence minimization, and3) in optimal settings, SGAN would be equivalent to integral probability metric GANs.We show that this property can be induced by using a relativistic discriminator which estimate the probability that the given real data is more realistic than a randomly sampled fake data.We also present a variant in which the discriminator estimate the probability that the given real data is more realistic than fake data, on average.We generalize both approaches to non-standard GAN loss functions and we refer to them respectively as Relativistic GANs and Relativistic average GANs.We show that IPM-based GANs are a subset of RGANs which use the identity function.Empirically, we observe that1) RGANs and RaGANs are significantly more stable and generate higher quality data samples than their non-relativistic counterparts,2) Standard RaGAN with gradient penalty generate data of better quality than WGAN-GP while only requiring a single discriminator update per generator update, and3) RaGANs are able to generate plausible high resolutions images from a very small sample, while GAN and LSGAN cannot; these images are of significantly better quality than the ones generated by WGAN-GP and SGAN with spectral normalization.The code is freely available on https://github.com/AlexiaJM/RelativisticGAN.
Improving the quality and stability of GANs using a relativistic discriminator; IPM GANs (such as WGAN-GP) are a special case.The paper proposes a “relativistic discriminator”, whic helps in some settings, although a bit sensitive to hyperparameters, architectures, and datasets.In this work, the authors considers a variation of GAN by simultaneously decreasing the probability that real data is real for the generator.