Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: This paper deals with an important problem of mode collapse and mode mixture. In order to
tackle the both problems, the paper proposes to separate the manifold embedding and the
optimal transportation problems; the first part being carried out using an autoencoder to map the
images onto the latent space and the second part is accomplished using a GPU-based
convex optimization to find the discontinuous transportation maps.

I have some doubts about moving from the "semi-discrete OT map" to the piece-wise linear extension. The illustration in Fig. 3, and implicit in all the explanation charts is the fact that discontinuity can be found by a linear separation. This seems to be an extremely simplifying assumption, which leads to not so great visual results from the paper. Although the numerical results seems promising, I feel that fewer images, but larger in size, and analysis of mode collapse phenomenon in real images would have been much better.

Singular set detection seems to be the most tricky part in this paper, which should have been explained further. The Simplex projection assumption, renders this part not that tricky, but that is where I feel the biggest doubt about this paper lies.

The authors themselves mention the need for a high quality auto encoder model to encode celebA dataset, which has been improved upon by numerous other papers, the claims seems not too strong. Also, the method does not have any adversarial training and hence, it studies the GAN idea from only fixing the generator point of view. 
review_title: Official Blind Review #4

final_score: rating: 8: Accept
review_content: General Comments:  The generator in Generative Adversarial Networks (GANS) computes an optimal transportation from the noise distribution to the data distribution.  However, such maps are in general discontinuous.  Since deep neural networks can only represent continuous maps, this brings two problems: mode collapse and mode mixture. This paper approaches both problems using Figalli's regularity theory. They separate the manifold embedding (here an autoencoder maps input data to a latent space) from the optimal transportation (this map is found by convex optimization). Composing these two steps yields the proposed method. Their method basically avoids representing discontinuous maps by the generator. Empirically, the proposed method performs similar or better than state-of-the-art.

I think the idea of the paper is nice, and an interesting perspective  on GANs is presented. A new method is proposed. The numerical contributions are certainly significant. Therefore, I believe the paper deserves publication.

Nevertheless, I have some comments below.

1) Although this paper brings a new perspective, based on optimal transport theory, as far as I can understand this paper does not establish formal new results. Thus I think some strong claims about providing deep theoretical explanation should be more moderate. In essence, it seems that the paper verifies *numerically* (in section B.3) that Figalli's theorem (stated in Appendix B) holds in this context.

2) This is just a suggestion. I think in some parts a lighter notation and a more intuitive explanation could help.

3) After Eq. (5) in the Appendix the authors mention Newton's method, and Thm 3 is also specific to Newton's method. Then they mention that *Gradient Descent* is used (and in the main part of the paper they mentioned Adam). This is confusing. All these algorithms are different, and Newton's method does not imply convergence results for gradient descent. I don't see how Thm 3 is relevant.

4) This is a simple doubt. To avoid non-differentiability of the gradient, the OT step computes the Brenier potential and is able to locate the singularities. I wonder if using a simpler approach through optimization for nosmooth problems (such as Moreau envelopes or proximal methods) could resolve this issue? In the negative case, why not?

5) Some Minor comments:
1. Define OT in the abstract (Optimal Transportation?) 
2. What is AE? (not defined also; Auto Encoder?)
3. There are lots of typos through the text, such as missing "the", "a", etc. 
and a couple mispelled words. I suggest the authors proofread the draft
more carefully.
4. pp. 4 ... what is a "PL convex function". PL is not defined.

review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: Contributions:
1. This paper proposes a new problem in GAN distribution mapping: the concavity of support problem.
2. This paper provides a solution to the concave support together with mode collapsed problem in GAN, via a discrete-continuous optimal transport model, given some post-processing techniques to rule out "singular points".
3. Empirical results show the effectiveness of the proposed method.

To summarize their method. First, they fit a good auto-encoder model to get embeddings for the observed data as an empirical distribution \nu on space Z. Second, they use a semi-discrete OT to map a noise distribution \mu to \nv. Since OT will be aware of all modes in \nu, singular points can be detected by checking the angle between "shards" and those points that are around the "ridge" should be rejected. Thus, the proposed method could handle both the concave support problem and the mode collapse problem.

My concern is whether the proposed method is overkill because the singular point detection can be very tricky and relies on heavy linear programming. Could you explain why not using the following substitute: 
Step 1. Fit an auto-encoder just as you did in the paper and get an empirical distribution \nu.
Step 2. Fit a Gaussian mixture model on \nu and do model selection over # clusters.
Step 3. Sample from the Gaussian mixture model to generate fresh images.

Since this method relies on a high-quality auto-encoder model, it is hard to say this paper makes progress in fixing the GAN's mode collapsed problem. Besides, the paper does not involve an adversarial training module. So I will not treat it as a satisfactory improvement over GAN. Overall, the proposed problem in GAN indeed exists. But the solution seems to deviate from the goal the paper aim to achieve.
review_title: Official Blind Review #1

Meta-review: The authors present a different perspective on the mode collapse and mode mixture problems in GAN based on some recent theoretical results. 

This is an interesting work. However, two reviewers have raised some concerns about the results and hence given a low rating of the paper. After reading the reviews and the rebuttal carefully I feel that the authors have addressed all the concerns of the reviewers. In particular, at least for one reviewer I felt that there was a slight misunderstanding on the reviewer's part which was clarified in the rebuttal. The concerns of R1 about a simpler baseline have also been addressed by the authors with the help of additional experiments. I am convinced that the original concerns of the reviewers are addressed. Hence, I recommend that this paper be accepted. 

Having said that, I strongly recommend that in the final version, the authors should be a bit more clear in motivating the problem. In particular, please make it clear that you are only dealing with the generator and do not have an adversarial component in the training. Also, as suggested by R3 add more intuitive descriptions to make the paper accessible to a wider audience.


Decision: Accept (Poster)