Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: ######## Updated Review ############

The author(s) have presented a sincere rebuttal, which I really appreciate. Although I still don't quite agree with all the points made by the author(s), I have changed my mind to be more or less borderline about this submission, given that the author(s) have gone through great length to clarify and improve their manuscript. 


#################################

This paper proposed a generative modeling framework called potential flow generator. Instead of deriving new matching criteria between distributions, the authors considered redefining the generative process via simulating a continuous flow that is constrained by the optimality conditions on the flow potential field derived based on L2 optimal transport. This is certainly an interesting direction to explore, however, while the points made are valid, they are not well justified. My major criticism is that too much compromise needs to be made in order to construct such a flow generator. In practical terms, it's computationally costly and sacrifices too much of the network's flexibility. My overall evaluation for this work is a straightforward/brute-force application of well-known (but less practical) results, without proposing any remedies to the real challenges that underlie. My detailed comments are listed below:

1. It is assumed that the input distribution should have the same ambient dimensionality as the target distribution, and the continuity constraints mean that the deforming to the target can take an excruciatingly slow pace, the main obstacle faced by all flow-based constructions. This point is partly evidenced by the experiment section where none of the input distributions is far from the target. It's questionable whether this framework can efficiently perform "generative modeling", in which a simple noise distribution is pushed to a more sophisticated target distribution. 

2. Relations to the Neural ODE literature is not sufficiently discussed, which I believe is closest to this work. A major drawback of Neural ODE is slow computations. 

3. While the author(s) have criticized an intuitive construction of L2 transport penalty in Eqn (13), their objective Eqn (17) suffers a similar issue. 

4. The experiments are weak and not convincing. First, ss mentioned in earlier comments, 2D toy transport and image translation are fairly easy tasks. Second, only qualitative results are reported, and there is no baseline model to compare with. Third, without ablation study, We can hardly verify the fact the gains are actually coming from the flow part, as vanilla GANs can also perform a similar task. 

Minors: Additionally, language issues can be spotted here and there. The author(s) should more carefully proofread this manuscript. And I find it confusing that Fig 3 (a) and (b) uses different examples for WGAN and CNF. 
review_title: Official Blind Review #3

final_score: rating: 8: Accept
review_content: This is a great paper using optimal transport theory for generative and implicit models. Instead of using general vector fields, the authors apply the potential vector fields in optimal transport theory to design neural networks. The mathematics is correct with convincing examples. This brings an important mathematical connection between fluid dynamics and GANs or implicit models. 

I suggest the acceptance of this paper after addressing the following minor questions. 

1. Would the authors provide slightly more details about the design of networks? 

2. In the literature, the author may need to cite 

"A. Lin, W. Li, S. Osher, G. Montufar, Wasserstein proximal of GANs, 2018."
"W. Li, G. Montufar, Natural gradient via optimal transport, 2018"

The Wasserstein natural gradient method there may improve the computational speed of the proposed models. 

In all, this is an exciting paper with many potentials in future neural network designs. 

review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: The paper proposes a ‘potential flow generator’ that can be seen as a regularizer for traditional GAN losses. It is based on the idea that samples flowing from one distribution to another should follow a minimum travel cost path. This regularization is expressed as an optimal transport problem with a squared Euclidean cost. Authors rely on the dynamic formulation of OT proposed by Benamou and Brenier, 2000. They propose to learn a time-dependent potential field which gradient defines the velocity fields used to drive samples from a source distribution toward a target one. Experiments on a simple 1D case (where the optimal transport map is known), and on images with an MNIST / CelebA qualitative example.
The use of this dynamic formulation is well known in the OT community. See as a good examples:
Trigila, G., & Tabak, E. G. (2016). Data‐driven optimal transport. Communications on Pure and Applied Mathematics, 69(4), 613-648.
and more generally Chapter 7 of the ‘Computational Optimal Transport’ book by Peyré and Cuturi.

The novelty arises from the use of neural networks to represent the potentials. However, the claim that the obtained map is the optimal transport map seems wrong to me, because:
The class of potential functions over which the optimization is performed is not the whole class of functions, leading to approximations;
The optimality conditions (a.k.a continuity or preservation of mass equations) are only enforced on sampled trajectories, not on the entire space. 
While this claim should definitely be lowered, it is nonetheless still acceptable provided that the proposed model is performing good. On this part, the paper strength could be improved provided that comparisons with existing methods computing a Monge map could be given. Notably, a comparison with the approach from Seguy et al. 2018 is missing. On a same level, a qualitative comparison with cycleGAN  in Figure 4 is missing. 
There are some unclear elements in the paper. The final, total, optimization problem is never clearly expressed. I believe a general algorithm presentation could help in understanding the general picture of the method. Notably, for instance, It is still not clear if the generator G is disconnected from the potential definition (following Eq. 10 I assume not). How are the trajectories sampled ? Is the discriminator trained on the same sampled trajectories or different ones ?  In the potential generator, it seems that the time is considered the same way as the feature space. Can you comment on this point ? Is it possible to evaluate the flow of different time stamps that the ones used for training ? 

As a summary,
Pros:
An interesting way to represent time-dependent potentials with a network for regularizing generative models
Cons:
Not much theoretical novelties in the paper, nor a good analysis on the source of errors of the model (e.g. impact of discretization on the problem)
There are some unclear aspects in the paper (see comments)
The potential benefits of the approach over the state-of-the-art should be more clearly discussed. 


Minor comment:
 P3. Uniqueness of Monge problem for the squared Euclidean cost should be attributed to Brenier 91 and his polar factorization theorem. McCann generalized it to Riemannian manifold.


review_title: Official Blind Review #1

Meta-review: This paper proposes applying potential flow generators in conjunction with L2 optimal transport regularity to favor solutions that "move" input points as little as possible to output points drawn from the target distribution.  The resulting pipeline can be effective in dealing with, among other things, image-to-image translation tasks with unpaired data.  Overall, one of the appeals of this methodology is that it can be integrated within a number of existing generative modeling paradigms (e.g., GANs, etc.).

After the rebuttal and discussion period, two reviewers maintained weak reject scores while one favored strong acceptance.  With these borderline/mixed scores, this paper was discussed at the meta-review level and the final decision was to side with the majority, noting that a revision which fully addresses reviewer comments could likely be successful at a future venue.  As one important lingering issue, R1 pointed out that the optimality conditions of the proposed approach are only enforced on sampled trajectories, not actually on the entire space.  The rebuttal concedes this point, but suggests that the method still seems to work.  But as an improvement, the suggestion is made that randomly perturbed trajectories could help to mitigate this issue.  However, no experiments were conducted using this modification, which could be helpful in building confidence in the reliability of the overall methodology.

Additionally, from my perspective the empirical validation could also be improved to help solidify the contribution in a revision.  For example, the image-to-image translation experiments with CelebA were based on a linear (PCA) embedding and feedforward networks.  It would have been nice to have seen a more sophisticated setup for this purpose (as discussed in Section 5), especially for a non-theoretical paper with an ostensibly practically-relevant algorithmic proposal.  And consistent with reviewer comments, the paper definitely needs another pass to clean up a number of small grammatical mistakes.
Decision: Reject