Title:  The relativistic discriminator: a key element missing from standard GAN
Authors: Alexia Jolicoeur-Martineau
Venue: ICLR 2019
Keywords: AI
deep learning
generative models
GAN

Abstract: In standard generative adversarial network (SGAN), the discriminator estimates the probability that the input data is real. The generator is trained to increase the probability that fake data is real. We argue that it should also simultaneously decrease the probability that real data is real because 1) this would account for a priori knowledge that half of the data in the mini-batch is fake, 2) this would be observed with divergence minimization, and 3) in optimal settings, SGAN would be equivalent to integral probability metric (IPM) GANs. 

We show that this property can be induced by using a relativistic discriminator which estimate the probability that the given real data is more realistic than a randomly sampled fake data. We also present a variant in which the discriminator estimate the probability that the given real data is more realistic than fake data, on average. We generalize both approaches to non-standard GAN loss functions and we refer to them respectively as Relativistic GANs (RGANs) and Relativistic average GANs (RaGANs). We show that IPM-based GANs are a subset of RGANs which use the identity function. 

Empirically, we observe that 1) RGANs and RaGANs are significantly more stable and generate higher quality data samples than their non-relativistic counterparts, 2) Standard RaGAN with gradient penalty generate data of better quality than WGAN-GP while only requiring a single discriminator update per generator update (reducing the time taken for reaching the state-of-the-art by 400%), and 3) RaGANs are able to generate plausible high resolutions images (256x256) from a very small sample (N=2011), while GAN and LSGAN cannot; these images are of significantly better quality than the ones generated by WGAN-GP and SGAN with spectral normalization.

The code is freely available on https://github.com/AlexiaJM/RelativisticGAN.

TL;DR: Improving the quality and stability of GANs using a relativistic discriminator; IPM GANs (such as WGAN-GP) are a special case.

Review discussion:
In this work, the authors considers a variation of GAN by consider simultaneously decrease the probability that real data is real for the generator. To include such a property, the authors propose a relativistic discriminator which estimate the probability that the given real data is more realistic than the fake data. Numerical results are performed to show that the proposed methods are effective, and the resulting GANs are relatively more stable and generate higher quality data samples than their non-relativistic counterparts.
Overall the paper is well written and the rationale behind the proposed modification is clear.
In particular, the authors use three different perspective, (the prior knowledge, the divergence minimization, and the gradient expressions), to explain what they thought is missing in the state-of-the-art. By proposing to utilize the information about both real and fake data in the discriminator definition, the authors’ have (to some extent) alleviated the above shortcoming of the state-of-the-art.  Unfortunately, like almost all papers related to the field,  there has been no rigorously justification behind the proposed methods.
The English of the paper has to be significantly improved.
For example, grammar errors like “this mean….”, “didn’t converge, …”
Unfortunately, the codes of the paper is not released, I will encourage the authors to do so.
Dear Reviewer 3,
Thank you for your comments.
We reviewed the paper to correct for spelling mistakes and to make it less familiar (by removing contractions). According to Reviewer 1 suggestions, we also revised Section 3 to improve the wording and explanations.
The code has already been released through GitHub. To retain anonymity, we re-uploaded the GitHub repository without any information relating to the authors: https://github.com/anonymousconference/RGAN.
The paper proposes a “relativistic discriminator” which has the property that the probability of real data being real decreases as the probability of fake data being real increases.
The paper is very well-written. I particularly liked Section 3 which motivates the key idea through multiple viewpoints. The experiments show that the relativistic discriminator helps in some settings, although it does seem a bit sensitive to hyperparameters, architectures and datasets.
I found the argument about connections to IPM-GANs a bit confusing. In a couple of places in Section 4, the relativistic loss is motivated by showing that the relativistic discriminator makes SGANs more like IPM-GANs.
However, not all IPM-GANs are the same, e.g. the experiments show performance gaps between RSGAN, RaSGAN, and WGAN-GP, which suggests there could be other confounding factors.
Could you devise experiments on synthetic datasets where the different hypotheses in Section 3 might lead to different solutions? Would be very interesting to see which hypothesis best explains why relativistic discriminator helps!
Section 4.3: How do you justify the averaging?
While the relativistic GAN is well-explained, section 4.3 only briefly mentions the averaging idea. Given that averaging seems to help a lot in some of the experiments, it’d be great to see further discussion of why this helps.
Dear Reviewer 2,
Thank you for your comments.
We are in agreement about the fact that IPM-based GANs are different from Relativistic GANs. They are similar, but yet different enough that they are not of the same class.
Although our paper mentioned the similarity, it did not mention the difference which could lead to readers thinking that IPM-Based GANs are a subset of Relativistic GANs (and we talked to people who thought this was the case after reading our paper). In section 4.2 p5, we now highlight better the differences and similarities:
“If one use the identity function (i.e., f_1(y)=g_2(y)=-y, f_2(y)=g_1(y)=y), this results in a degenerate case since there is no supremum/maximum.
However, if one adds a constraint so that C(x_r)-C(x_f) is bounded, then there is a supremum and one arrives at IPM-based GANs.
Thus, although different, IPM-based GANs share a very similar loss function focused on the difference in critics.”
As you suggested, we ran some additional experiments focused on testing the gradient argument (see Appendix E, p13-14).
Although the gradient argument applies if we train G to optimality;
in practice, we do not train G to optimality.
Thus, we observed that RSGAN/RaSGAN are not equivalent to IPM-based GANs in real-world scenarios.
However, they act in a way that is somewhere in-between the dynamics of SGAN and IPM-based GANs.
In addition to Appendix E, we now also mention that training G to optimality is an unrealistic assumption in Section 3.3 p4.
The main intuition that led to Relativistic average GANs was actually in our initial paper version, but it was removed due to space constraints (8 pages max). Given your comment, we decided to relay it to the Appendix rather than completely removing it; it is now in Appendix B p11-12.
Additionally, we added the following sentence at the beginning of section 4.3 p5:
“The discriminator has a very different interpretation in SGAN compared to RSGAN. In SGAN, D(x) estimates the probability that x is real, while in RGANs, D(x_r,x_f) estimates the probability that x_r is more realistic than x_f. As a middle ground, we developed an alternative to the Relativistic Discriminator, which retains approximately the same interpretation as the discriminator in SGAN while still being relativistic.”
This explains why we created RaGANs, but it does not explain why they generally perform better than RGANs. We are still uncertain as to why RGANs perform less well than RaGANs, given that both approaches improve stability.
The paper describes an interesting tweak of the standard GAN model (inspired by IPM based GANs) where both the generator and the discriminator optimize relative realness (and fakeness) of the (real, fake) image pairs. The authors give some intuition for this tweak and ran experiments with CIFAR10 and CAT datasets. Different variants of the standard GAN and the new tweak were compared under the FID metric. The experimental setup and details are provided; and the code is made publicly available.
The results are good and their tweak seems to help in most of the cases. The paper, however, is not very well written and is not of publication quality.  All the insights given in Section 3 are wrong, incomplete and unsatisfying.
For example, in Section 3.4, the authors suggest that gradient dynamics of the tweaked model (with some unrealistic and infeasible assumptions) is same as that of an IPM-GAN and contribute to stability. This is wrong. Similar dynamics (even under the unrealistic assumption), does not imply similar performance.
In fact, if one is trying to move towards IPM dynamics, then one should try to tweak an IPM model directly. Section 3.2 also seems wrong from my understanding of GAN training. Section 3.3 could also be improved.
In fact, any explanations based on minimizing JS divergence is incomplete without answering as to why JS divergence minimizing is the best thing to do.
The author should have provided more comparison images to rule out the fact that the tweak is not overfitting for the FID metric. The benchmarks are also weak and more experiments need to be done (Eg, CelebA).
Dear Reviewer 1,
Thank you for your comments.
We hope that this message will find you well. We really took the time to review all your comments and in doing so we significantly improved the paper. As you suggested, one aspect (the gradient argument) was relying on unrealistic assumptions (that G would be trained to optimality). We believe that we were able to make the paper of much higher quality so please consider this response in your assessment of the paper.
You mention that the paper is “not well written” and a lot of your emphasis is on Section 3. To remedy your concerns, we spent a lot of time to rewrite parts of it in a way that is much clearer.
Also, as suggested by Reviewer 3, we reviewed corrected spelling mistakes and removed contractions to make it less familiar.
Note that we removed section 3.1 since it was not a real subsection.
Regarding Section 3.2 (which is now section 3.1), we rewrote it because it was somewhat unclear after we removed so much text to fit the 8 pages limit.
Regarding Section (3.3, which is now section 3.2), we clarified that JSD is not the only divergence where we see something like Figure 1a, this is true for most divergences.
Thus, our explanation is not incomplete. See below:
“Note that although specific to the JSD, similar dynamics are true for other divergences;
when the divergence is maximal, D(x_r) and D(x_f) are very far from one another, but they converge to the same value as the divergence approach zero.
Thus, this argument applies to other divergences.”
Regarding Section (3.4, which is now section 3.3), we agree that one assumption was unrealistic. The problematic assumption was assuming that both D and G are trained to optimality.
In practice, certain GANs (mostly IPM-based GANs) train D multiple times.
However, no GANs to our knowledge train G multiple times since GANs do not converge when doing so. G can only take a small step at a time;
otherwise, the generator will collapse early on. Note that Reviewer 3 suggested that we do some experiments regarding the gradient argument and we did (the full experiment described below is in Appendix E). We observed that we do not reach D(x_r)=0 using relativistic GANs when n_G = 1 (the number of generator update per critic’s updates). If using n_G = 2, it does sometimes happen that D(x_r)=0.
Either way, we have that RSGAN significantly increase the proportion of low D(x_r) even if it rarely reaches 0.
Thus, although we cannot make SGAN equivalent to IPM-based GANs, we can make them more similar. We rectify this in p4.
To respond to your comments about IPMs, we seek to find a GAN with a similar dynamic to IPM-based GANs without actually using IPMs. We want this because IPM-based GANs have an important drawback: they tend to be very computationally demanding (not always, but more often than not). In the introduction, we now mention that IPM-based GANs tend to be longer to train.
Thus, finding an approach with similar stability, but which requires less training time would be useful.
The added paragraph is:
" Note that although powerful, IPM-based GANs tend to more computationally demanding than other GANs. Certain IPM-based GANs use a gradient penalty (e.g. WGAN-GP, Sobolev GAN) which is very computationally costly and most IPM-based GANs need more than one discriminator update per generator update (WGAN-GP requires at least 5 \citep{WGAN-GP}). Assuming equal training time for D and G, every additional discriminator update increase training time by a significant 50\%.”
We do provide more comparison images in the linked GitHub.
However, the link (the footnote on p18) is hidden to retain anonymity for the review process. We transferred the GitHub to an anonymous version for the reviewers. Here are the full minibatch for the models generating 256x256 cats:
https://github.com/anonymousconference/RGAN/tree/master/images/full_minibatch.
We would like to note that we ran additional stability analyses for CIFAR-10 in the appendix. We will consider doing using more benchmarks next time. We are very limited in our computing capability, thus we decided to only use CIFAR-10 and CAT.
Next time, we will consider using CAT and CelebA instead.

Meta-review:
role: AC
data: metareview: All authors agree that the relativistic discriminator is an interesting idea, and a useful proposal to improve the stability and sample quality of GANs. In earlier drafts there were some clarity issues and missing details, but those have been fixed to the satisfaction of the reviewers. Both R1 and R3 expressed a desire for a more theoretical justification of why the relativistic discriminator should work better, but the empirical results are strong enough that this can be left for future work.

Decision: Accept (Poster)