Venue: ICLR 2017 workshop

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: This paper presents two tricks to prevent the mode-missing behavior of GANs. The first trick is to use a multimodal noise at the input in order to make it easier for the generator to generate multi-modal data. The second trick is to initialize the generator with a "covering initialization" so that the mean and covariance of the generated and real data match. One of the reasons that GAN training fails is that typically the support of the real and generated distributions are disjoint, in which case there will be a perfect discriminator between them which makes the training dynamic unstable. It is argued in the paper that the second trick addresses this problem. It is further shown that both of these tricks are necessary for covering all the modes of a mixture of Gaussian toy dataset with a GAN.

While both of these tricks do make sense, I am not convinced that these trick will actually resolve the mode-missing behavior of GANs in high-dimensional data. The GAN training dynamic of a 2D toy dataset is very different from that of a 1000 dimensional dataset, and it is likely that heuristic tricks like these do not generalize to high dimensional datasets. I think the quality of this paper can be substantially improved, if the authors show that these trick will actually help on a more realistic dataset.
review_title: Review

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary: This paper proposed two techniques to help train GANs, in particular, by improving their stability. Multimodal noise essentially replaces the typically Standard Normal distributions with a multimodal distribution. Although theoretically unimodal noise can allow modeling any data distribution, but it's believed that it would be easier to model real-world data using multimodal noise distributions since real-world data also resembles islands separated by low-probability regions. Covering initialization pretrains the generator such that the mean and covariance of the model distribution and data distribution matches. This avoids mode collapse by avoiding initializing the generator as a many-to-one function (which is harmful). The experiments are shown on a low-dimensional toy dataset.

Novelty: The ideas described have not been described in prior work, AFAIK. But, the ideas have been well-known by practitioners of GANs, but never studied properly, or found to be significantly useful in large-dimensional datasets and tasks.

Clarity: The paper is written clearly.
Significance: Improving techniques for training GANs is a significant problem.
Quality: Overall, the lack of experimentation on a high-dimensional dataset (even MNIST) hurts the quality of the paper.

Pros: The main reason for accepting the paper would be to have a written description of these two techniques, so that these ideas can be discussed and cited.
Cons: The paper provides little to no evidence on whether these techniques are actually useful in real high-dimensional tasks.
review_title: Review

Decision: Reject