Title: Towards Principled Methods for Training Generative Adversarial Networks
Authors: Martin Arjovsky
Leon Bottou
Venue: ICLR 2017

Abstract: The goal of this paper is not to introduce a single algorithm or method, but to make theoretical steps towards fully understanding the training dynamics of gen- erative adversarial networks. In order to substantiate our theoretical analysis, we perform targeted experiments to verify our assumptions, illustrate our claims, and quantify the phenomena. This paper is divided into three sections. The first sec- tion introduces the problem at hand. The second section is dedicated to studying and proving rigorously the problems including instability and saturation that arize when training generative adversarial networks. The third section examines a prac- tical and theoretically grounded direction towards solving these problems, while introducing new tools to study them.

TL;DR: We introduce a theory about generative adversarial networks and their issues.

Review discussion:
In Theorem 2.2. what is the definition of an `optimal' discriminator?
Closed manifolds are intended to mean manifolds without boundary or manifolds with boundary?
1) Thanks for the comments! By optimal discriminator we meant that it is optimal with respect to its objective E_x~Pr [ log D(x)] + E_x~Pg[ log(1 - D)]. Clearly we forgot to define this properly so we'll add it in the revision :).
2) This is a very important question (which we forgot to properly address). As is, the theorem assumes a manifold without boundary. We will extend the theorem to manifolds with boundary in the revision.
This is quite simple, since the definition of perfect alignment can be extended by considering the (as now defined) perfect alignment between the interior and boundary of M with the interior and boundary of P, and we can easily prove Lemma 2 for this new definition.
After that, the missing step is to prove that L = intersection(M, P) has measure 0 on both M and P (analogous to Lemma 3).
But L is the union of intersection(interior(M), interior(P)), intersection(interior(M), boundary(P)), intesection(boundary(M), interior(P)), intersection(boundary(M), boundary(P)), which is the union of 4 intersections of not perfectly aligned manifolds without boundary (so union of strictly lower dimensional manifolds inside P and M), so it will have to have measure 0 in both M and P.
The rest of the proof will follow in the same way as now :)
Again, a million thanks for the important observations!
SUMMARY
This paper addresses important questions about the difficulties in training generative adversarial networks. It discusses consequences of using an asymmetric divergence function and sources of instability in training GANs.
Then it proposes an alternative using a smoothening approach.
PROS
Theory, good questions, nice answers.
Makes an interesting use of concepts form analysis and differential topology.
Proposes avenues to avoid instability in GANs.
CONS
A bit too long, technical. Some parts and consequences still need to be further developed (which is perfectly fine for future work).
MINOR COMMENTS
- Section 2.1 Maybe shorten this section a bit. E.g., move all proofs to the appendix.
- Section 3 provides a nice, intuitive, simple solution.
- On page 2 second bullet. This also means that P_g is smaller than the data distribution in some other x, which in turn will make the KL divergence non zero.
- On page 2, ``for not generating plausibly looking pictures'' should be ``for generating not plausibly looking pictures''.
- Lemma 1 would also hold in more generality.
- Theorem 2.1 seems to be basic analysis. (In other words, a reference could spare the proof).
- In Theorem 2.4, it would be good to remind the reader about p(z).
- Lemma 2 seems to be basic analysis. (In other words, a reference could spare the proof).
Specify the domain of the random variables.
- relly - > rely
- Theorem 2.2 the closed manifolds have boundary or not? (already in the questions)
- Corollary 2.1, ``assumptions of Theorem 1.3''. I could not find Theorem 1.3.
- Theorem 2.5 ``Therefore'' -> `Then'?
- Theorem 2.6 ``Is a... '' -> `is a' ?
- The number of the theorems is confusing.
Thanks for the review! We agree with everything said. We did however choose to leave some of the proofs in the main text. We think it may be helpful for people that don't have such a strong analytical background to get acquainted with the ideas so that it doesn't appear as a black box result.
- We extended the definitions in section 2 and the statements and proofs of Lemmas 2 and 3 to cover manifolds with and without boundary.
- We would love to get a more general version of Lemma 1, but we couldn't come up with one. If you have any pointers on directions or a reference to look into that would be super helpful!
- We fixed all the typos and the numbering of the theorems :)
If you have any more suggestions or questions please let us know!
Best,
Martin
This is a strong submission regarding one of the most important and recently introduced methods in neural networks - generative adversarial networks. The authors analyze theoretically the convergence of GANs and discuss the stability of GANs. Both are very important. To the best of my knowledge, this is one of the first theoretical papers about GANs and the paper, contrary to most of the submissions in the field, actually provides deep theoretical insight into this architecture. The stability issues regarding GANs are extremely important since the first proposed versions of GANs architecture were very unstable and did not work well in practice. Theorems 2.4-2.6 are novel and introduces mathematical techniques are interesting. I have some technical questions regarding the proof of Theorem 2.5 but these are pretty minor.
Thank you for the amazing review!!! If you have any questions please send them our way and we'll be glad to answer them :)
Best!
Martin
I'd like to ask a question what you think is the next research direction that should be taken regarding theoretical understanding of GANs.
Here's a few questions and problems that we think are important to be studied (and hopefully, tackled!)
- We have yet to characterize what the distances between distributions with a suboptimal training of the discriminator behave like in a precise mathematical sense. It is clear that GANs do something if the discriminator is far from optimal (indeed, our paper shows that we cannot learn from an optimal one in the current setting, which is what causes most of the problems we have now). Intuitively this behaves as a smoothened very imperfect version of the JSD or the reverse KL, depending on which update is used, but how this smoothening process works and what properties it imposes to the learned generator is far from being understood.
- How these divergences behave in comparison to other distances such as Wasserstein ones, which are properly defined and well behaved even on the manifold case.
- Wasserstein is symmetric so we cannot in principle tune it to care more about mode dropping or generating plausible samples (as we can do with the KLs), so figuring out weak analogs to the f-divergences would be massively interesting.
- The terms of generating plausible samples and mode dropping still need to be precisely understood in a mathematical setting that covers the manifold case. I think the comments of section 5.6 of https://arxiv.org/abs/1610.04490 is a very important starting point towards asking the right questions.
- Finding out which is the optimal variance or form for the added noise. It is likely that too little noise won’t be enough to fully ameliorate these issues, and too high noise will make the variance of the gradients too high, and give too little signal. Trying to define the problem for example of finding the optimal variance according to some quantitative estimate (such as making a bound like Theorem 3.3 the tightest possible, or minimizing the variance of the gradients) would be quite interesting.
- I'd like to get a general statement of the form of Lemma 1, ideally with necessary and sufficient conditions.
We are considering adding these items to an appendix C as possible further directions of research, do you think this would be valuable to the readers?
This paper makes a valuable contribution to provide a more clear understanding of generative adversarial network (GAN) training procedure.
With the new insight of the training dynamics of GAN, as well as its variant, the authors reveal the reason that why the gradient is either vanishing in original GAN or unstable in its variant. More importantly, they also provide a way to avoid such difficulties by introducing perturbation. I believe this paper will inspire more principled research in this direction.
I am very interested in the perturbation trick to avoid the gradient instability and vanishment.
In fact, this is quite related to dropout trick in where the perturbation can be viewed as Bernoulli distribution. It will be great if the connection can be discussed.
Besides the theoretical analysis, is there any empirical study to justify this trick? Could you please add some experiments like Fig 2 and 3 for the perturbated GAN for comparison?
Thanks for the review! The perturbation trick definitely seems like a good direction, and we are already starting to see empirical evidence in other groups using it successfully (e. g: https://openreview. net/pdf? id=S1RP6GLle in image super resolution and http://mrdrozdov. com/papers/infogan. pdf in infogan variants). That being said, the variance of the noise can be important, since too little noise will likely not be enough to ameliorate all issues, and too much noise will likely lead to high variance of the gradients. The structure of the noise can be quite important as well, since the covariance matrix of images is quite complex (even if it is known), so N(0, sigma * I) noise is probably not the best we can do.
We'd like to get a more complete understanding of how different types of noise behave, and in the end finish the story by properly stabilizing GANs, but due to time constraints and the paper already being quite long we chose to keep this for further research. This way we also give the possibility to other researchers to start experimenting on this issues (with some understanding) right away, which we in the end considered more valuable.
The connection with dropout seems quite interesting!
In fact, adding noise at higher layers (similar to dropout) could be quite a good idea, since as the factors of variations get disentangled, a more simple noise will have the right metric structure on hidden unit space. We'll be glad to continue exploring this direction in future work :)
If you have more questions or comments please let us know!
Best,
Martin
Hi! We would first like to thank the reviewers for your comments. We will aim to make all the suggestions fit into a revision. We will update the paper shortly and provide individual responses to the reviews :)
Best!
Martin
Hi! We would like to comment that we added a revision with the following changes at its core:
- We extended the proofs and definitions in section 2 to work with manifolds with boundary.
- We included a one page Appendix B with further clarifications for the things that were suggested in the comments, such as a small comment on continuity vs absolute continuity of random variables, and how it is relevant to our paper.
- We did some minor rewriting to take into account the suggestions from the reviewers and commenters.
- We fixed all the typos :)
Best!
Martin
Very good paper, I hope it will be accepted. I keep my original evaluation.

Meta-review:
role: PC
data: comment: The paper provides a detailed analysis of the instability issues surrounding the training of GANs. They demonstrate how perturbations can help with improving stability. Given the popularity of GANs, this paper is expected to have a significant impact.

Decision: Accept (Oral)