Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes to solve the continuous Optimal Mass Transport problem as a convex mapping function estimation problem based on Brenier's theorem using input convex neural networks.
The proposed method is compared with other DNN-based methods in the ubiquitous density estimation and generative modeling tasks in statistics, and good results are obtained.
main_review: The proposed method in this paper has the potential to be a very useful method for solving the continuous optimal transport problem using DNNs.
In addition, the manuscript is clear and well constructed.
On the other hand, the effectiveness of the method for application problems in machine learning cannot be confirmed because there are few implications about real applications and few experiments.
In order for the paper to be accepted at this conference, it might be necessary to apply the method to a real problem such as the image generation problem of natural images, and then conduct experiments to compare the method with existing methods which solved the application problem.
In addition to that, the following points were unclear and I would like to ask for your response.

Does ICNN have the ability to represent all convex functions?

On page5, it says "L^2-norm used in equation 5", but isn't it a mistake of "equation 4"?

In the experiment of method comparison, is there any evidence that the parameters of DNN in the previous study are optimized? (If there is no evidence, it would be better to specify the possibility that it is not optimized?　That would be positively evaluated as a scientific paper, as it would present the limits of verification).
summary_of_the_review: The proposed method in this paper has the potential to be a very useful method for solving the continuous optimal transport problem using DNNs.
In addition, the manuscript is clear and well constructed.
On the other hand, the effectiveness of the method for application problems in machine learning cannot be confirmed because there are few implications about real applications and few experiments.
In order for the paper to be accepted at this conference, it might be necessary to apply the method to a real problem such as the image generation problem of natural images, and then conduct experiments to compare the method with existing methods which solved the application problem.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a new physics informed neural architecture for efficiently learning and optimizing optimal transport maps using techniques from partial differential equations.
main_review: Strength: + The probabilistic universal approximation framework can be much more efficient than the finite element or difference methods which is known to have exponential scaling with respect to dimensions.
+ The paper provides sufficient details in terms of network, hyperparameters, evaluation methods in the experimental section.

Weakness: - The main required definitions are provided, however, the main relevant technical concepts are explained heuristically. For example, while relevant background for optimal transport theory is provided in detail, there is little to no background on partial differential equations.
- The main loss functions (4) and (5) proposed do have exponential scaling with respect to dimensions of the domain of integration \Omega.
- Density estimation experiments are preliminary.

summary_of_the_review: Justification: The paper makes a solid contribution in terms of technical content. The main high level idea is that solving optimal transport maps under some conditions can be posed as a PDE. While this is not something new, for instance, sinkhorn algorithm does the same -- turn a nonsmooth optimization problem, a linear program, into a smooth version, as given in equations (4) and (5). However, many technical details regarding the algorithm are missing -- (i) what is the overall procedure?; (ii) near equation (7), there is some discussion regarding representing u again using a neural network, is the overall architecture different from what is shown in Figure (1)? It seems like some of these questions regarding how and what PDE is exactly being solved can be answered by reading various other papers cited in the experiments, but it is not clear from the submission. In this sense, I feel like the paper spends too much time on background material (which the reader can pick up from any standard book on Applied PDE, for example, Introduction in Optimal
Transport for Applied Mathematicians by Filippo Santambrogio, 2015) rather than focusing on the key insights of the paper. 

The network details are provided in abundant details for verification of the main idea. However, the main task density estimation which the paper mentions quite in detail about in the beginning of the paper is only minimally experimented. For example, the paper claims that finite elements solvers are slow in high dimensions. However, the paper contains no comparisons with these methods on the problems considered in the paper, which are also low dimensional.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper introduces a new method for density estimation that is based on the idea of optimal mass transport (OMP), i.e., finding some function that transports a probability density into another probability density while minimizing the transportation cost. To do so, the authors leverage Brenier's theorem which guarantees the existence and uniqueness of the optimal transport map. As a consequence of this theorem,  the optimal transport map of the OMT problem can be obtained as the solution of a special nonlinear partial differential equation (PDE). The authors then show that the approximate solution can be computed with help of physics informed neural networks (PINNs). The approach is demonstrated for several canonical examples.
main_review: The paper combines a range of interesting ideas to solve density estimation problems. The authors reformulate the original OMP problem as a special PDE. In turn, PINNs can be used to approximate the solution of this special PDE. In addition, the authors use input convex neural networks (ICNNs) to  incorporate the Brenier potential into the PINN. This approach bypasses some of the challenges of directly minimizing the original OMP problem.

* _Originality:_ The presented ideas in this paper are original and well suited for this conference. I not aware of another paper that uses PINNs for density estimation. 

* _Quality:_ The proposed framework seems plausible and is demonstrated on a few experiments. There is no theory that backups some of the claims, and the amount of experiments is on the lower end of the spectrum for an empirical ML conference paper. Further, it would help to better discuss the assumption that have been made and some of the limitations. I assume that the authors have a fair understanding for these details and they should make better use of the unlimited space of the appendix to provide additional discussions and experiments. 

* _Clarity:_ The proposed ideas are well organized and clearly presented. The detailed discussion of the background materials helped reading the paper. 

* _Significance:_ This paper is of interest for two communities within ML. First, it is of interest for the PINN community and the paper has the potential to motivate future research in the intersection of PINNs and density estimation. On the other hand, density estimation is an important topic in ML and the presented approach performs well on the presented tasks.However, it is not clear whether the presented approach is competitive with other state-of-the-art methods in this area, or whether it just does better on the selected tasks compared to 2 baselines. A better discussion of the advantages and limitations and additional baselines would help to strengthen the paper and improve its significance. 


Here are a few comments that might help to improve the paper:

* The weakness of this paper is the experimental section. Only two other algorithms are used for comparison. The authors state that these two algorithms performed best on their benchmark problems, still it would be interesting to show results for other algorithms that have been tested. Also, how does your approach compare to state-of-the-art GANs or neural ordinary differential equations (NODEs) for density estimation methods? At least, it would be helpful to discuss some qualitative advantages / disadvantages of your proposed approach in comparison to other methods such as GANs or NODEs. In which situation should one use your approach, and which situation does your approach fail? 

* You state that you only consider the simple case of a quadratic cost function. Does Brenier's Theorem apply to other cost functions?

* You state that ICNNs introduce implicit regularization. Can you provide theory to back this claim up, or is this only an educated guess?

* The results in Figure 6 look interesting. Why don't you include these results in the main text? I would rather move the network details to the appendix.   

* Can you comment on the computational costs of your framework as compared to the other methods that you used for comparison.

* On page 2: I assume that there is some typo in:  'the convex function itself should be indenity helps in avoiding'.

* On page 9: word repetition in 'a density estimation estimation'.

* The quality of the figures is poor. It would be nice to better format the figures so that they eat up less space and use vector graphics instead of low-resolution pngs. 
summary_of_the_review: This paper presents interesting and novel ideas for density estimation, and to the best of my knowledge this is the first paper that uses PINNs for density estimation. The paper is clearly written and provides a good overview of related work and background to put the proposed ideas into context. However, since this paper provides only empirical results and no theory, I only think that the paper is slightly above the acceptance threshold. Additional results would help to make the story more compelling, and given additional results I am happy to reconsider my rating.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper presented an input convex neural networks (ICNN)-based methods to approximate the optimal transport maps. The learning objective is based on the Monge-Ampere equation, i.e. zero loss corresponds to the solution to the Monge-Ampere equation. Physics Informed Neural Networks (PINNs) are adopted to minimise the loss. The author(s) also extended the work to density estimation. The problem is interesting as estimation of optimal maps has a wide range of applications. Experiment results look promising when compared with other PINNs in one experiment setup, but only qualitative results are provided for the density estimation experiment with no comparison of other baselines.
main_review: The paper is in general well-written and the idea is clearly presented. Yet, I have a few concerns as listed below:

1. The authors stated that "We will, however, see, that in our situation, i.e., for absolutely continuous measures, the existence and uniqueness is indeed guaranteed". This refers to the Brenier's theorem concerning that a unique optimal transport map exists which is given by the gradient of a convex function. This consequently leads to the Monge-Ampere equation that help formulate the loss function in the proposed work. Yet, the physics informed neural networks adopted can only be shown to be a convex function, but there is no guarantee that the optimized network is indeed the solution of the Monge-Ampere equation. Hence, the uniqueness of the map does not seem to be guaranteed. If the alternative is true, a proof is needed here.

2. The presentation of the density estimation in Section 2.3 is a bit confusing. What does it mean to be "or equivalently $\bigtriangledown u)_* \mu = \nu$"? A bigger question is whether the proposed work can be used in density estimation, given that the uniqueness of the mapping is not guaranteed as in Comment 1 above.

3. Although literature review covers other density estimation techniques such as normalizing flow, there is no comparison with any of these algorithms in Section 3.3. The authors stated in Section 1 that "Although these methods (normalizing flow) have shown to perform well in density estimation applications, the interpretability of the obtained transformation is less clear". But it is not clear to me how the interpretability of the proposed method is clearer, especially given the fact that the computed density may not be exact/correct (due to the lack of the proof of the uniqueness of the obtained approximate mapping), while normalizing flow admits exact density estimation.
summary_of_the_review: The paper provides an interesting approach to incorporate ICNN and Brenier's Theorem to approximate optimal transport between distributions. Experiment results are encouraging. But there is no proof to support the claim on the uniqueness of the mapping. Therefore, its application on density estimation is also not fully supported.
review_title: null

Meta-review: This work proposes to define densities via the pushforward of a base density through the gradient field of a convex potential as studied in OT theory and, in particular, inspired by Brenier's theorem.

More concretely, it proposes to use ICNNs to parametrize the convex potentials and considers two mechanisms to match a target density: 1) with a known (normalized) target approximately solve the Monge-Ampere equation via optimization; 2) with only samples available, they propose to use the maximum-likelihood approach.

While the paper is overall well-written, the idea is very close to existing work that was not mentioned or discussed in the paper. The paper would benefit from a substantial revision to incorporate the missing references and emphasize the relative novelty.
Decision: Reject