Venue: ICLR 2017 conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: The paper presents an approach for tackling the instability problem that is present in generative adversarial networks. The general idea is to allow the generator to "peek ahead" at how the discriminator will evolve its decision boundary over-time with the premise that this information should prevent the generator from collapsing to produce only samples from a single mode of the data distribution.

This is a very well written paper that clearly motivates its attack on an important open issue. The experiments are well carried out and strongly support the presented idea. The pursued approach is substantially more elegant than current existing "hacks" that are commonly used to make GANs work in practice. I however have three main issues that let me partly doubt the success of the method. If these can be resolved this paper is a clear candidate for acceptance.

1) I am not entirely convinced that the same effect cannot be obtained by the following procedure: simply train the discriminator for an extended number of K steps when updating the generator (say a number equivalent to the unrolling steps used in the current experiments) then, after the generator was updated undo the K updates to the discriminator and do 1 new update step instead. I only briefly glanced at your response to Reviewer2 which seems to imply you now tried something similar to this setup by stopping gradient flow at an appropriate point (although I think this is not exactly equivalent).
2) I tried to reproduce the simple MNIST example but using a fully connected network instead of an RNN generator without much success. Even when unrolling the discriminator for 30-40 steps the generator still engages in mode seeking behavior or does not train at all. This could either be because of a bug in my implementation or because of some peculiarities of the RNN generator or because I did not use batch normalization anywhere. If it is one of the latter two this would entail a dependence of the proposed approach on specific forms of the discriminator and generator and should be discussed. My code can be found here https://github.com/iclrreproducer/unrolled_gan -- see the comments in the file for switching on unrolling and batch normalization in the generator. This issue could also be addressed simply by opening up the code and setting up a fully connected network example on your side.
3) For the more complicated data distributions the method is only used in combination with many of the existing tricks for training GANs. As a result the experiments are much less convincing. I personally cannot see a strong difference between the samples generated by any of the CIFAR-10 models. Why did you not try to remove batch-normalization etc. for these examples ? Does training then again become unstable ?

---- UPDATE -----
I am leaving the initial review intact to preserve the conversation. After the authors response all points have been resolved, including a fix to my re-implementation of the ideas presented in the paper. As a result I have increased my score and believe the paper should be accepted. Please also see my response to the rebuttal below.
review_title: A nice idea for stabilizing GANs

final_score: rating: 7: Good paper, accept
review_content: This work introduces a novel method for training GANs by displacing simultaneous SGD, and unrolling the inner optimization in the minmax game as a computational graph. The paper is very clearly written, and explains the justification very well. The problem being attacked is very significant and important. The approach is novel, however, similar ideas have been tried to solve problems unrelated to GANs.

The first quantitative experiment is section 3.3.1, where the authors attempt to find the best z which can generate training examples. This is done by using L-BFGS on |G(z) - x|. The claim is that if we're able to find such a z, then the generator can generate this particular training example. It's demonstrated that 0-step GANs are not able to generate many training examples, while unrolled GANs do. However, I find this experiment unreasonable. Being able to find a certain z, which generates a certain sample does not guarantee that this particular mode is high probability. In fact, an identity function can potentially beat all the GAN models in the proposed metric. And due to Cantor's proof of equivalence between all powers of real spaces, this applies to smaller dimension of z as well. More realistically, it should be possible to generate *any* image from a generator by finding a very specific z. That a certain z exists which can generate a sample does not prove that the generator is not missing modes. It just proves that the generator is similar enough to an identity function to be able to generate any possible image. This metric is thus measuring something potentially tangential to diversity or mode-dropping. Another problem with this metric is that that showing that the optimization is not able to find a z for a specific training examples does not prove that such a z does not exist, only that it's harder to find. So, this comparison might just be showing that unrolled GANs have a smoother function than 0-step GANs, and thus easier to optimize for z.

The second quantitative experiment considers mean pairwise distance between generated samples, and between data samples. The first number is likely to be small in the case of a mode-dropping GAN. The authors argue that the two numbers being closer to each other is an indication of the generated samples being as diverse as the data. Once again, this metric is not convincing. 1. The distances are being measured in pixel-space. 2. A GAN model could be generating garbage, and yet still perform very well in this metric.

There are no other quantitative results in the paper. Even though the method is optimizing diversity, for a sanity check, scores for quality such as Inception scores or SSL performance would have been useful. Another metric that the authors can consider is training GAN using this approach on the tri-MNIST dataset (concatenation of 3 MNIST digits), which results in 1000 easily-identifiable modes. Then, demonstrate that the GAN is able to generate all the 1000 modes with equal probability. This is not a perfect metric either, but arguably much better than the metrics in this paper. This metric is used in this ICLR submission: https://openreview.net/pdf?id=HJKkY35le

Whether this paper is accepted or not, I encourage the authors to investigate this approach further, since the method is promising and interesting.

# Post-rebuttal review

The authors have incorporated changed in the paper by adding more experiments. These experiments now demonstrate the claims of the paper better. The paper was already well-written and introduced a novel idea and addressed an important problem. The only thing holding this paper back was unconvincing experiments, which now has been corrected. Thus, I would increase my score by 2 points, and recommend accepting the paper.
review_title: Interesting approach, but evaluation leaving a lot to be desired

final_score: rating: null
review_content: I missed the deadline for the pre-review question. Please see main review for remaining issues.
review_title: None

final_score: rating: 9: Top 15% of accepted papers, strong accept
review_content: The paper introduces a technique for stabilizing the training of Generative Adversrial Networks by unrolling the inner (discriminator) optimization in the GAN loss function several steps and optimizing the generator with respect to the final state of this optimization process.
The experimental evidence that this actually helps is very compelling: the 2d example shows a toy problem where this technique helps substantially, the LSTM MNIST generator example shows that the procedure helps with stabilizing the training of an unusual architecture of generator, and the image generation experiment, while not being definitive, is very convincing.
For future work it would be interesting to see whether a method with smaller memory requirements could be devised based on similar principles.
I strongly recommend to accept this paper.
review_title: Excellent and important paper

final_score: rating: null
review_content: Have you considered the relation between this approach with other simpler approaches like historical averaging (3.3 in [1]), expert of discriminators (eg. [2])?

Section 2.4 does a good job of explaining why this approach seems to be working well. However, I'd like to decouple the effect of just using multiple discriminators and the proposed effect (G taking a step taking into account how D will react). To make the narrative in section 2.4 more convincing, it would be useful to do a baseline, where instead of optimizing over the unrolled optimization, we just move G in accordance with a mixture of the last K discriminators (or something similar).


[1]: https://arxiv.org/pdf/1606.03498v1.pdf
[2]: https://arxiv.org/abs/1611.01673
review_title: multi-discriminator baseline

Meta-review: The main idea of this paper is clearly presented, and its empirical claims well-demonstrated. It improves our understanding of how to train GANs.
Decision: Accept (Poster)