Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper proposes a new method for learning an optimal pushforward (for the quadratic cost) from a distribution to another distribution based on samples of both distributions. The optimal transport problem is first written equivalently as a minimax problem over set of convex functions, as in Makkuva et al. 19. Then, the convex functions are parametrized as Input Convex Neural Networks (ICNN). To my understanding, the novelty of the approach is to replace the minimax problem over the parameters of the ICNN by an easier to solve a regularized minimization problem. The authors prove the consistency of their approach: loosely speaking, they show that true minimizers of their approximate problem are epsilon minimizers of the original optimal transport problem. They also provide promising numerical experiments.



The paper proposes a simple fix to drawbacks of previous works of on the topic, which e.g. propose minimax approaches. Moreover, the consistency of the approach is rigorously proven.
On the other hand, one could argue that the novelty is marginal (replacing a minimax problem by a regularized minimization). Moreover, the consistency result assumes access to a true minimizer of the regularized problem (which is highly non convex). Finally, the writing of the paper can be improved.

    
I rate the paper as marginally below.

    
Regarding the novelty of the approach. This can be addressed by SOTA numerical experiments, but this is not the case (although the numerical experiments are well explained and detailed). 
Regarding the consistency result. The authors provide an approximation result that does not take into account the optimization of the regularized problem. It is a difficult a problem but the authors could at least mention this difficulty. 
Regarding the overall quality of the paper. 
- The structure of the introduction is strange. Several paragraphs are used to explain points which cannot be understood at this stage or provide too much details. Several paragraphs are used to explain that we look for well structured pushforward mappings. Why not directly mention the equivalent optimal transport problem ? 
In dimension one all continuous invertible mappings sending P to Q are monotone? I am not sure. What is a maximal monotone mapping (eq 2)? Considerations about the dimension of the space should not be in the intro in my opinion (the reader has the feeling that the authors are defending themselves). Considerations on minimax vs non minimax and end to end are too difficult to understand at this stage (same in Section 2).
-Section 2. The notation for the Fenchel transform is misleading. It should be with a star...
-Section 3. Why do we need a positive density? Optimal pushforward mappings exist (in both directions) if both distributions admit a density wrt Lebesgue measure. What is the point of the footnote 4? The set "Convex" is not defined. Convex potentials are called discriminators. Why this name? I guess that there is a connection with GANs but I don't see (actually this work is about fitting an optimal pushforward mapping, which is something independent from the GAN setup). 
-Section 4. Check Eq 10 (symbol = and alignment). Consideration on why non minimax is better than minimax should be explained before.  "Fully-connected ICNNs satisfy universal approximation property". What does it imply for the assumptions of Th 4.2? Are they satisfied or not?



MINOR:
- Question: Are gradients wrt to x of the ICNN easy to compute?
- parenthesis in the equation after Forward Generative Property
- "diffirentiable" (several times)
- I have seen similar considerations (using ICNN to fit gradient of convex functions) in https://openreview.net/pdf?id=rklx-gSYPS
review_title: Reasonable approach but writing to be improved

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: From my perspective, this is a much needed and love-to-see work for the line of neural generative modeling. Previous approaches were dominated by GAN based approaches which require solving a minimax optimization which has technical hurdles in practice. This paper reviews the literature of OT theory, and provides a comprehensive explanation about the relation between W_2 and cyclical monotonic map. In particular, it motivates the opportunity to approximate Eq. (8) with Eq.(12), as which a non-minimax formulation exist. 

Pros:

- a non-minimax formulation was proposed for neural generative modeling
- theoretical properties are derived for the proposed formulation Eq.(12)
- experiments are preliminary but promising. 

Cons:

- The paper misses a discussion about how gradient-based optimization is done for Eq.(12) in implementation. For example, it is not clear what functions are supposed to be parameterized as a neural network. Is it \varphi_\theta and \bar{\varphi}_\omega? If so, how the gradient is calculated for \Delta \bar{\varphi_\omega}? Would it involve high order auto-differentiation? 
- The authors have not outlined how computational feasible their approach is? This makes me less confident about their approach regarding processing more difficult datasets or tasks. 
- I strongly suggest the authors to re-organize their presentations, and focus more on what was actually calculated in practice. This allows others to follow their work and reproducing their experiments. Some mathematical introduction is nice, but they have to be directly related to the approach of this paper. 



---------------
post revision:
I read authors' revision, and it is indeed an improved version with more readabilities. Since I am familiar with the OT literature, the mathematical presentation is clear enough for me to follow. The motivation of this paper is clear. The presentation of algorithm is also clear enough in the revision. But I can not say much for wider audience this paper may target. 

Given the significance and popularity of GAN related work, this paper seems a big deal to the community. This's why I intend to give 8 instead of 7 for the revised paper.




review_title: An end-to-end algorithm with a non-minimax objective for training a OT map for quadratic cost

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper proposes Wasserstein-2 Generative Networks (W2GNs) which is an optimal transport framework for learning generative models. Unlike minimax problems of Wasserstein GANs, the proposed approach which is based on minimizing the 2-Wasserstein distance reduces to a single-level optimization problem. The paper numerically shows that the new approach enjoys faster convergence and improves upon the performance scores of Wasserstein GANs and other optimal transport baselines. While the paper's idea on applying optimal transport tools for training generative models seems interesting, the discussed theoretical and numerical results are not supportive enough to show that the proposed approach indeed improves upon WGANs. Also, the theoretical sections have been written in a convoluted way with several weakly supported claims and the final algorithm has not been stated clearly. I, therefore, do not recommend the paper for acceptance.

To further explain my concerns, let me start with section 3 which reviews the dual formulation to 2-Wasserstein distance in Eq. (8) and also the connection to the convex conjugate optimization in Eq. (9). Here, the paper vaguely mentions that the optimization problem for computing the convex conjugate is "convex and very complex" followed by a brief explanation which I do not find satisfactory. Specifically, the paper's way of reasoning does not convince me why it is necessary to switch from Eq. (8)  to the paper's formulation in (12). Both the issues mentioned for problems (8) and (9) also apply to the formulation in (12) as (12) still requires taking the gradient of the convex conjugate \psi_w. Also, I highly recommend replacing the terms "very complex" and "impossible" with more solid theoretically or numerically supported statements. 

Next, let me refer to the final sentences of the first paragraph of section 4.1: " Yet such a problem is still minimax. Thus, it suffers from typical problems such as convergence to local saddle points, instabilities during training and usually requires non-trivial hyperparameters choice." These sentences argue that every minimax problem suffers from instability and convergence to local solutions. However, the paper's own formulation in (12) also leads to a non-convex optimization problem for which the authors show no convergence guarantees to a global solution. The paper should either remove these sentences or precisely explain why the proposed non-convex problem enjoys better convergence properties than the minimax problems. Let me also add that while Eq. (12) states an optimization problem, it still does not completely characterize a learning algorithm, because it is unclear how one wants to take the gradient of \psi_w's convex conjugate. I think the paper should include an algorithm clearly stating the steps of learning the generative model.

Finally, the theoretical guarantees in Theorem 4.1 and 4.2 do not analyze the algorithm's performance for the class of input convex neural nets. Theorem 4.1 connects the optimization error to the closeness of the generative and underlying distributions. Yet, it does not provide any guarantee on how large the optimization error could be for convex neural nets. The result of Theorem 4.2 also immediately follows from the assumptions and offers little understanding of the algorithm's performance with convex networks, since it considers the set of all differentiable functions instead. The theoretical guarantees should somehow analyze the algorithms' convergence and approximation properties for convex neural nets rather than all differentiable convex functions. Overall, the paper seems to carry several nice ideas, but the theoretical discussion needs to be significantly improved.  

*****
Review update: I thank the authors for their response and for revising the paper based on the comments. The revision addresses several of my concerns. I still think the theoretical guarantees should be stronger and therefore change my score to borderline 5.
review_title: Official Review

Meta-review: 
The reviewers have  different views on the papers but agreed that the paper can be accepted. However, they suggested
some points of improvements including the writing (clarity and style) and experiments showing strong improvements
compared to WGAN.
Decision: Accept (Poster)