Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes a new technique for moving the vertices of a mesh that is used as the underlying discretization for solving a PDE in order to better capture relevant details without increasing complexity. This is achieved by designing a neural spline-based, and also a graph attention network-based model to output the optimized node positions. Direct supervision coming from the Monge-Ampère method is utilized.
strengths_and_weaknesses: Strenghths:

- To my knowledge, this is the first method that proposes to optimize for node positions with direct supervision in the context of solving PDEs.
- Can potentially inspire a line of research for re-meshing.

Weaknesses:

- Although the general technique is well-motivated, it is not clear why the particular choices of network architectures make sense. Most of the methods section is about listing the various design choices without justifications.
questions: - As far as I understand, with the GAT-based variant, it is not possible to guarantee non-self-intersecting meshes/mesh tangling. I was expecting an analysis of the behaviour of the generated maps for this characteristic, but there is none.

- I was a bit disappointed by the irregular domain experiments, which is the main motivation for using GAT. Heptagon is not that irregular. I suggest including more examples with ideally real-world-like domains.
limitations: Please see above.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The paper proposes learning-based mesh movement methods for improving the accuracy of PDE solutions. The key idea is to train neural networks to approximate the solutions generated using the traditional Monge-Ampère method. The models are evaluated using two types of PDE problems on different domains, where the proposed model achieved performance close to the traditional method while being much faster.

The main contribution of the paper is the proposed neural models that can quickly produce a mesh movement result close to the traditional Monge-Ampère method.
strengths_and_weaknesses: Strength:
- The proposed models can achieve performance close to the traditional Monge-Ampère method while being much faster.
- The models are evaluated on two types of PDE problems to show their performance.

Weakness:
- According to the literature cited in the paper, the Monge-Ampère method has been applied to non-planar domains such as spheres. But the evaluation in this paper is only performed on planar domains. It is unclear how well the method performs on domains with non-zero Gaussian curvature.
- Whilst the speed-up reported in the paper is impressive, they can also be misleading. The traditional MA method is run on the CPU, whilst the proposed models are run on the GPU. In the paper there is no information on the CPU model. Moreover, it is unclear what types of underlying linear algebra library the Firedrake code uses (NumPy can use different underlying BLAS libraries, which may have very different performance). So it is difficult to assess the actual speedup.
- The proposed models are formulated purely based on closeness to the Monge-Ampère solution. However, Monge-Ampère is not the ultimate goal of the mesh movement problem, but rather just one particular way to generate high-quality meshes. The paper does not provide much information on what types of meshes are good for PDE solving, which makes it difficult to judge the quality of the result beyond its similarity to the Monge-Ampère solution. This somewhat limits the novelty of the proposed approaches.
- Since the loss function is purely based on closeness to the Monge-Ampère solution rather than specific quality measures of the meshes, it cannot totally prevent mesh tangling, as mentioned in the paper. 
- The evaluation is done on small meshes. It is unclear how well the method performs on larger meshes (say more than 10K vertices).
questions: 1) What type of CPU were the experiments run on? Are the implementation of the MA method single-threaded or paralellized?

2) What types of meshes are good for PDE solving? Why is not possible to encode these quality meshes into the loss function?

3) How do the models perform on large meshes and non-planar domains?
limitations: The discussion is sufficient.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper presents two neural architectures for warping meshes to improve finite element modeling performance (reduce error). The models are trained in a supervised manner to match the maps produced by a Monge-Ampere (MA) mesh movement method. Error reduction is comparable to the classical MA method, but with a speed improvement of several orders of magnitude.
strengths_and_weaknesses: The overall idea of using a normalizing flow to move a mesh around seems sound, and allows for a nice tradeoff between the speed advantages of neural methods and the reliability of traditional FEM. The neural spline model can only natively preserve boundaries that are aligned to the axes. Also, while the neural spline model may produce a diffeomorphism, this does not guarantee avoiding mesh tangling because the discrete mesh might not be able to represent a sufficiently complicated smooth diffeomorphism. For the graph attention model, the use of neighborhood attention to keep vertices within the convex hulls of their neighbors is a nice observation, and could potentially guarantee injectivity under suitable conditions (as with Tutte embedding).
questions: The examples shown in the paper are on relatively coarse, high-quality meshes of domains with convex boundaries. How does the method perform on larger meshes, or when the initial mesh is less regular? Can it be applied to other types of PDE? 
limitations: 1. The models are only tested on two PDEs on simple domains. It would be good to see at least a few more evaluations, especially on larger/finer-grained meshes.
2. Even though comparison to h-adaptation methods may not be useful to evaluate the performance of this method, it might still be useful for practitioners to choose which type of adaptation to use.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The authors have presented a novel approach for r-adaptivity based adaption for PDEs using two different network architectures. The Neural spline-based model serves well to avoid mesh tangling while handling meshes with large deformation, and the GAT-based model can generalize for domains with arbitrary shapes. The authors use the Monge-Ampere method to train using supervised learning, and demonstrate the effectiveness of their method on stationary and time-dependent, linear and non-linear equations, as well as regularly and irregularly shaped domains.
strengths_and_weaknesses: Strength:
(1) The proposed framework is truly the first learning-based approach for r-adaptivity of meshes.
(2) The authors have considered two different approaches in Neural splines and GAT, and demonstrated their effectiveness along with their limitations.

Weaknesses:
The scope of the work seems somewhat limited in the sense of:
(1) the authors consider  only the Monge Ampere method for r-adaptivity. See "Simulation-driven optimization of high-order meshes in ALE hydrodynamics", "rp-adaptation for compressible flows", and "Defining a stretching and alignment aware quality measure for linear and curved 2D meshes", which have been used for r-adaptivity of meshes to PDEs.
(2) the problems considered are too simple; element count in each direction is ~20 and the simulations are 2D only.
questions: (1) In irregular domains, the sampling is done by using a bounding box and values outside the actual domain are set to 0. We have found that such an approach is problematic if it used for solutions that have non-smooth (shocks) features. It would be good to note this limitation.
(2) Can the boundary nodes move with the neural spline model? Node movement along the boundary can be crucial for certain applications.
(3) Regarding the last two lines right before Section (4): Two adjacent nodes can move within the convex hull of their neighbors in such a way that the resulting mesh is tangled. Do you mean something else (e.g., half the distance from the closest neighbor)?
limitations: While this work is a step in the right direction, the proposed methods must really be demonstrated to work for large scale 2D/3D problems if they are to be claimed to be competitive against existing methods.
review_title: null

Meta-review: metareview: Reviews for this paper were somewhat mixed, but overall the AC agrees with reviewer h6GN and others that the idea in this work is creative and could inspire future work.  In particular, while the experiments do not display diversity in terms of mesh density/shape, the overall approach is compelling and fairly well verified by the experiments.  It seems reasonable for NeurIPS to take a risk on publishing this work in hopes that it will be valuable for future PDE solution machinery that can address some of the missing finer points.

In the revised camera-ready version of the paper, please acknowledge the shortcomings/limitations of the work (e.g., use of Monge-Ampere in contrast to other methods and use of simple 2D problems) and say something about how you might address these limitations.
confidence: Less certain
Decision: Accept