Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors propose a neural network architecture that can be used to parametrize divergence-free vector fields. As applications, they show how this method can be used to enforce the continuity equation in PDE simulations of incompressible fluid flow, and also used to enforce both continuity and non-negativity when optimizing time-dependent solutions of the dynamical optimal transport problem. 

The authors propose two parametrizations in this work, which realize a tradeoff between model size and computational cost. 
1. Matrix-field: the vector field $v$ is computed by taking row wise divergences of $n$ matrix fields, ie., $v$ is parametrized by $n$ matrix fields. 
2. Vector-field: each matrix field is written in terms of the pointwise Jacobians of $n$ vector fields, ie., $v$ is parametrized by $n$ vector fields at the additional cost of evaluating these Jacobians. 

The authors show that, possibly up to a constant vector field, any divergence-free field on the $n$-dim Torus admits a representation in either of these two forms. Lastly, to apply this representation in experiments, the corresponding matrix/vector fields are approximated by trained neural network functions. 
strengths_and_weaknesses: The main strength of this work is that the proposed parametrization allows to exactly satisfy the divergence-free constraint, addressing one of the main drawbacks of penalty-based approaches such as in existing PINN methods. 

While this is an interesting idea, there are some significant weaknesses in the experimental evaluation:

1. Insufficient quantitative evaluation: for simulating incompressible fluid flow, there is no quantitative evaluation at all of the numerical accuracy of the proposed method. It's not clear from qualitative plots in Figures 1 & 2 whether this method is method is accurate relative to the performance of competing methods. For optimal transport, the authors show comparable performance with one baseline optimal transport solver on toy models. However, since there is also no discussion of the computational cost of NCL, it's highly unclear if it represents a practical contribution to either of these two problem domains. 

2. No discussion of computational cost: related to the last point, there is no discussion of the computational cost of either variant of NCL. From the descriptions of the matrix-field and vector-field algorithms, it seems this could be very costly compared to eg. evaluating a feedforward network. Even if this method can achieve quantitative benefits over alternatives, it's not clear whether it can do so in a computationally feasible manner.


_Edit_: I have changed my evaluation of this work based on feedback from the authors and I have raised my score to 7. 
questions: What is a rough estimate of the wall clock time required to generate Figures 1, 2, and 3? In the case of Figure 3, how does this compare to the time required by the baseline method to compute the OT map?

limitations: The proposed method relies heavily on expensive differentiation operations, such as evaluating the Jacobian of many NNs and evaluating the divergence at points of corresponding matrix field.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper describes a method for parameterizing divergence-free vector fields in N-dimensions. The method is demonstrated with application to physics-informed neural networks (PINNs) for modeling the 2D Euler Equations with non-constant density, and to 2D optimal transport.
strengths_and_weaknesses: My experience is more on the PDE side, so I'll focus my review there. (Hopefully other reviewers can cover the optimal transport example.)

Strengths:

1. The paper describes two general approaches to parameterizing N-D divergence free fields. This will be a good references for the physics/ML community.
2. The paper demonstrates that the approach is workable in several machine learning applications.

Weaknesses:

1. The proof is for N-dimensional divergence free fields, but the examples are all for 2+1 or 3+1 space+time dimensions. Exactly one example (the 3D ball) requires going beyond the very well approach of parameterizing a vector field and taking the curl. A lack of high-dimensional examples (or even hypothetical high-dimensional applications) makes the discussion about $O(n)$ vs $O(n^2)$ scaling with dimensionality seem a little silly.
2. There are no quantitative comparisons between the two different parameterizations of divergence-free fields (matrix-free and vector-free). This would strengthen the paper.
3. I am concerned about the applicability for PDE problems. The paper describes avoiding "expensive numerical simulations" by parameterizing solutions with a neural network as a strength, but in truth PINNs are orders of magnitude [slower and less accurate](https://arxiv.org/abs/2205.14249) than state of the art simulation methods that parameterize solutions on grids. PINNs have indeed received a fair amount of recent attention from the physics community, but are not yet really a practical method, e.g., as demonstrated by the poor results in this paper for modeling the 2D flow shown in Figure 2.
4. The application to the Euler equation with non-constant density is also rather niche. In my experience, it much more common to either enforce constant density, in which case the curl formulation for vector fields is sufficient (as in Rao 2020), or to solve the full compressible Navier-Stokes via an equation of state. The technique in this paper would be application for this later case, which would probably make more compelling examples.
questions: 1. Any there any applications you can think of that would need more than 4 dimensions?
2. Can you add quantitative comparisons between the matrix-field and vector-field formulations?
limitations: No concerns.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper introduces two parametrizations for Divergence-Free Vector Fields in arbitrary dimensions. In principle, this offers an extension to the well-known method of using the curl of a vector field as a divergence-free vector field in $\mathbb{R}^3$.
The derivation of such Divergence-Free vector fields is based on Differential Forms and seemingly follows from first principles within this area.
The two different parametrizations were similarly derived but come with different properties.
a) is based on a Matrix representation, which scales quadratically with the input dimension, and b) requires 2nd order derivatives but only relies on a more compact vector representation, and further allows "building in" constraints such as positivity.

Significant applications for such a parameterization include the modeling of compressible fluids and the Maxwell Equations.
The velocity $\vec{v}$ and density $\rho$ of a mass-conserving fluid can be reformulated in terms of a divergence-free vector-field $\vec{v}' = [\rho, \rho \cdot \vec{v}]$ with $\vec{v}\in\mathbb{R}^3$ using the augmented arguments $[t, x, y, z]$.
Consequently, the proposed parameterization enables building a Neural Network, which by construction conserves mass instead of relying on soft constraints enforced similar to Physics Informed Neural Networks (PINNs).
A similar construction can be made for e.g. the Maxwell equation.

As a second application setting, the modeling of probability densities in an optimal transport setting is showcased.
For this, one of the introduced parameterizations is adapted to conform to the density constraints by construction (positivity, within 0 and 1).

Mostly qualitative experiments in simple settings are shown for both settings, providing a proof of concept for modeling Euler equations on a Ball and Torus, and learning optimal transport maps.
For the Euler equations, improved modeling of the flow compared to a baseline PINN is shown.

strengths_and_weaknesses: ### Significance:
1. _**[Medium]**_ A general parametrization for Divergence-Free Vector Fields in arbitrary Dimensions is proposed.
2.  _**[Medium]**_ The authors introduce to the ML community an elegant reformulation of continuity equations, that allows reformulating the problem to finding divergence-free vector fields.
3.  _**[High]**_ Combining (1.) and (2.), a Neural Network Architecture that provides conservation of mass by construction is provided. Enforcing this conservation constraint in Physics Informed Neural Networks (PINNs) is a difficult problem, limiting in my experience the number of publications of PINNs made within the area of compressible fluid dynamics. This work may open up PINNs to difficult problem settings such as Compressible Navier Stokes equations.  
4.  _**[High]**_ Combining (1.) and (2.), allows for solving optimal transport optimization problems, while by construction enforcement of the mass conservation constraints.  Other methods often require additional enforcement of these constraints, with non-adherence leading to a meaningless optimization objective.

### Originality:
1. The proposed construction of Divergence-free Vector Fields (of arbitrary dimensionality) with Neural Networks is novel.
2. The insight of reformulating the conservation of mass equation $\frac{\delta \rho}{\delta t} + div( \rho \cdot \vec{v} )=0$ to $div(\vec{v_{aug}})=0$ is to the best of my knowledge novel to the Machine Learning community.

### Quality:
- To the best of my knowledge, the proposed approach is technically sound and complete. 
- The experiments are sound and provide a qualitative proof of concept of the provided method.
- The universality proof provided in section 3.1 seems sound, however, I am rather limited in my familiarity with differential forms.
- No code is provided

### Clarity:
- The general story and flow of the paper is well organized and readable.
- The preliminaries on differential forms are not well written and structured. 
Although the mathematical tools for the derivation of the divergence-free vector fields are provided, they are not referenced when used and intermediate steps are skipped. 
Instead, they are handwaved in the form of "direct calculation shows[...]" (l. 96, l. 106).
When not that familiar with differential forms, it will be difficult to follow the paper in the current form - as it is the case for me.
Differential forms are in my opinion not a well-known area within ML, and such derivations should be made more clearly - A more step-wise derivation in the Appendix would help significantly.
- The notation is at times confusing, e.g. with overloaded variables (l.103), or missing equation numbers (e.g. between l.96 and l.97)
- (Small note on Terminology) "Incompressible Flow". A short clarification of what incompressible flows are and how they are not the same as incompressible fluids is probably helpful to avoid confusion in a wider audience. 


To summarize, this is a very promising work that is in my opinion a significant step for applying Physics Informed Neural Networks to challenging compressible fluid dynamics problems.
While the contribution itself is of high quality, novelty and significance, I see issues in terms of the clarity of writing mainly in the provided background in differential forms. 
questions: - Considering fluid flows in 2D, the augmented vector field would be in 3D and your equation would correspond to the curl: $*d\mu = \text{curl}(\vec{v})$, with a 3D vector representation $\vec{v}\in\mathbb{R}^3$ requiring only first-order gradients. How would this compare to the proposed matrix- and vector-field representations in praxis? Would the matrix representation still be favorable, as it is again more flexible in low dimensions? 
This would be interesting since many real applications consider a simplified problem flattened to 2-dimensional domains.


limitations: - The authors mention possible scaling limitations of the individual proposed parametrization, namely the required number of entries for the matrix representations and the higher-order derivatives needed for the vector representation. 

- Possible stability issues due to the division of $\rho$ for some losses are indirectly addressed by reformulating the loss terms. When access to the velocity is directly needed (e.g. observations of the velocity are provided for the training), training instabilities might still be an issue.

- It is not totally clear how the performance and flexibility of the two proposed parametrizations differ in praxis or how they behave during training. The authors provide a vague statement that the matrix representation is more flexible in lower-dimensional settings. The vector field representation offers some options for adding additional constraints such as positivity.
review_title: null

Meta-review: metareview: This paper presents two parameterizations for divergence-free vector fields as outputs of neural networks.  These parameterizations allow for modeling of compressible fluids and Maxwell's equations, as they automatically enforce the continuity equations.  In contrast, existing approaches are based on extra penalty terms and or expensive numerical simulation.  While the approach studied does not beat state of the art finite element approaches, it does provide a significant enhancement for physics inspired neural networks, which will provide a platform for further work to continue advancing neural network approaches for physics problems.
confidence: Certain
Decision: Accept