Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: Authors propose a  modification to the original GAN formulation, by coupling the generated samples and the true samples to avoid mode collapse.

I have some concerns about the analysis and the experiments of the paper: Most of the analysis is tailored for a very simple linear discriminator case which for the WGAN means just matching the first moments. Even in this simple setup, they consider d=1 (the scalar case). I am not sure how one can generalize this analysis to a more realistic case. Also the experimental gains seem incremental which makes me worried about such generalization. Finally, there are a few works in the literature about understanding the optimization landscape of GANs. For a sample, see https://arxiv.org/abs/1706.04156 and https://arxiv.org/abs/1710.10793. The later uses a Lyp function to analysis the global convergence of a GAN.  Also there is a few papers about the mode collapse issue in GANs. See for example https://arxiv.org/abs/1712.04086
 
review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: The paper attempts to perform global analysis of GAN on the issue of sub-optimal strict local minima and mode collapse, and proposes a new GAN formulation (CoupleGAN) that enjoys nice global properties. The paper is overall well written and conveys an interesting new formulation of GANs. However, the reviewer is concerned with the following questions:
The paper is mainly on analyzing the case when the true data has n points instead of on a continuous support. It would be more interesting to see theoretical guarantee on even Gaussian mixture model. Also since GANs are mostly known for generalizing what is seen to generate new data, whether converging only to the n points are good or not still worth debating.
In claim 4.2 and 4.3, what if the initialization of y is completely random? Then the claim cannot say anything on mode collapse. So is the formulation in the paper the real characterization of mode collapse?

review_title: Official Blind Review #2

final_score: rating: 8: Accept
review_content: In this paper, the authors introduce a new training loss for GANs.  This loss allows the outer optimization problem to have no spurious local minima, under an appropriate finite sample analysis.  In contrast, the authors establish that there are exponentially many spurious local minima under the conventional GAN training loss.  Under a linear discriminator model, the authors show that a standard GAN can not escape from collapsed modes in a finite sample analysis, whereas the new trining loss allows for such an escape (due to the presence of a Lyapunov functional with favorable properties).  The authors use this new training loss to train GANS on MNIST, CIFAR10, CelebA, and LSUN datasets, and observe mild improvements in Inception Scores and Frechet Inception Distances of the resulting generated images.

I recommend the paper be accepted because it provides a new formulation for training GANs that both demonstrates improved empirical performance while also allowing theoretically favorable properties (on spurious local minima and avoidance of mode collapse) that specifically do not hold for a standard GAN.

The primary question I am left with after reading the paper is: is there a probabilistic interpretation of the new loss function (equation 4a).  The authors justify this formulation because it allows analysis via Lyapunov functions, but it would be very useful to know if it itself is the maximum likelihood estimate under an alternate data model.  Such an explanation would improve the understandability of this method.

Minor comment: 

The fourth bullet point under the contributions section should specific the sense in which the new GAN "performs better"

review_title: Official Blind Review #3

Meta-review: The paper is proposed a rejection based on majority reviews.
Decision: Reject