Towards Universal Mesh Movement Networks
Authors: Mingrui Zhang
Chunyang Wang
Stephan C. Kramer
Joseph Gregory Wallwork
Siyi Li
Jiancheng Liu
Xiang Chen
Matthew D Piggott
Venue: NeurIPS 2024 spotlight (2024)
Keywords: PDE
Physical Simulation
Mesh Adaptation
Physical Science
Primary area: machine_learning_for_physical_sciences
TL;DR: We propose Universal Mesh Movement Networks (UM2N), which once trained, can be applied in a non-intrusive, zero-shot manner to move meshes with different sizes and structure, for solvers applicable to different PDE types and boundary geometries.

Abstract: Solving complex Partial Differential Equations (PDEs) accurately and efficiently is an essential and challenging problem in all scientific and engineering disciplines. Mesh movement methods provide the capability to improve the accuracy of the numerical solution without increasing the overall mesh degree of freedom count. Conventional sophisticated mesh movement methods are extremely expensive and struggle to handle scenarios with complex boundary geometries. However, existing learning-based methods require re-training from scratch given a different PDE type or boundary geometry, which limits their applicability, and also often suffer from robustness issues in the form of inverted elements. In this paper, we introduce the Universal Mesh Movement Network (UM2N), which -- once trained -- can be applied in a non-intrusive, zero-shot manner to move meshes with different size distributions and structures, for solvers applicable to different PDE types and boundary geometries. UM2N consists of a Graph Transformer (GT) encoder for extracting features and a Graph Attention Network (GAT) based decoder for moving the mesh. We evaluate our method on advection and Navier-Stokes based examples, as well as a real-world tsunami simulation case. Our method out-performs existing learning-based mesh movement methods in terms of the benchmarks described above. In comparison to the conventional sophisticated Monge-Ampère PDE-solver based method, our approach not only significantly accelerates mesh movement, but also proves effective in scenarios where the conventional method fails. Our project page can be found at https://erizmr.github.io/UM2N/.

Towards Universal Mesh Movement Networks
Mingrui Zhang'
Chunyang Wang' Jiancheng Liu
Stephan Kramer' Joseph G. Wallwork2 Xiang Chen Matthew D. Piggott
Styi Li
Imperiil College London University of Cambridge Noah's Ark Lab, Huawei {mingrui zhang18, chunyang. wang22, s. kramer) ac. uk {siyi. 1120,,20,,, m. piggott) ac..uk
j=24236cam. ac.uk xiangchen.c all..aiiioo ai@outlook. com 1jcc09300gmail.com com
Abstract
Solving complex Partial Differential Equations (PDEs) accurately and efficiently is an essential and challenging problem in all scientific and engineering disciplines. Mesh movement methods provide the capability to improve the accuracy of the numerical solution without increasing the overall mesh degree freedom count, Conventional sophisticated mesh movement methods extremely expensive and struggle to handle scenarios with complex boundary geometries. However, existing learning-bas based methods require re- re-ttraning from scratch given different PDE type or boundary geometry, which limits their applicability, and also often suffer from robustness issues in the form of inverted elements. In this paper, introduce the Universal Mesh Movement Network (UM2N), which once trained can meshes with different
be applied in non-intrusive, zero-shot manner size distributions and structures, for solvers applicable to different PDE types and boundary geometries. UM2N consists of a Graph Transformer (GT) encoder for extracting features and a Graph Attention Network (GAT) based decoder for moving the mesh. We evaluate our method on advection and Wavier-Stoker based examples, as well as a real-world tsunami simulation case. Our method out-performs existing learning-basee brased mesh movement methods in terms of the benchmarks described above. In comparison to the conventional sophisticated Monge- Ampère PDE-solver based method, our approach not only significantly accelerates mesh movement, but also proves effective in scenarios where the conventional method fails. Our project page can be found wtthtps:// :/te:://eriie imrr
ww........ttitt. -igggtttuub..... 10/UM2N/
Introduction
Various natural phenomena are modeled by Partial Differential Equations (PDE). The accurate and efficient approximation of the solutions to these complex and often nonlinear equations represents fundamental challenge across scientific and engineering disciplines. To solve real-world PDEs, many numerical methods require computational mesh grid to discretize the spatial domain. The quality of this mesh significantly impacts the balance between numerical solution's s accuracy and the computational cost required to obtain it. To maintain high-resolution everywhere in the domain computationally expensive. Many systems modeled by PDEs however are multi relti-rrsolltion in nature. For example, a small part of the system may be highly dynamic, while other regions quasi-stationary; alternatively, some locations may more important while others less important to the question being considered by the calculation [144] While a uniform, high-resolution mesh can be utilized to capture the dynamics or what most important accurately, this is often wasteful of
finite, precious computational resources and comes with sustainability implications (5).
An active body of research is focused on developing deep learning based methods to accelerate the PDE-solving process by learning surrogate neural solvers [6-8] A learned solver can output PDE
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

solutions directly given the physical parameters, and boundary conditions of a PDE. However, these methods encounter challenges such as not being able to guarantee physical plausibility of the PDE solution, weak generalization ability, and low data efficiency.
An alternative approach improve the efficiency of a PDE solver is utilize mesh adaptation, which is a technique for distributing the mesh according to numerical accuracy requirements. Two main categories of mesh adaptation techniques can be identified: /t-adaptation and r-adaptation. h-adaptation refines or coarsens the mesh resolution dynamically through topological operations such as adding/deleting nodes and swapping element edges/faces. In contrast, r-adaptation (or mesh movement) relocates or moves mesh nodes, keeping the mesh topology and thus the number of elements and vertices in the mesh unchanged [9]. These traditional mesh adaptation techniques can help reduce PDE-solving costs, but the mesh adaptation process itself may come at the cost of significant computational overhead.
Deep learning based methods have been proposed to accelerate mesh adaptation. [0-16]. Most previous methods focus on h-adaptive refinement [НН [77. Some existing works focus on learning error indicators or Riemannian metrics (which account for element shape and anisotropy. as well as size) 13-16] to guide the mesh adaptation process. The learned indicator or metrie is then fed into traditional remesher for mesh generation, or a mesh optimizer, which overall limits the performance of these works to being no better than traditional methods, especially in terms of efficiency. Reinforcement learning based methods show potential to improve the mesh adaptation task, but are difficult train with low data efficiency [IT [8] Only a small number of works focus on r-adaptation. [19 20 investigated supervised and unsupervised learning based mesh movement methods. However, the proposed methods require re-training from scratch given different PDE or geometry, which limits their applicability. In addition, the unsupervised learning method requires significant GPU memory and suffers from long training times, which makes re-training even more prohibitive for large scale problems.
Targeting the generalization ability and efficiency of mesh movement, here we introduce the Universal Mesh Movement Network (UM2N), two stage, computational mesh guided by an optimal-transport approach driven by solutions of equations of Monge-Ampére (MA) type. Given an underlying PDE to solve numerically, the process of MA based mesh movement includes choosing a suitable monitor function (i.e., a measure for the desired mesh density) and solving an auxiliary PDE whose solution prescribes the movement of the mesh. The Monge-Ampère equation is an example of such an auxiliary PDE, which results in mesh movement with attractive theoretical properties [9] but whose solution comes at a significant computational cost. Therefore, we decouple the underlying PDE solve and the mesh movement process. The learning of the mesh movement process is essentially learning to solve the auxiliary Monge-Ampère equation.
based model that learns to move the
The proposed UM2N consists of a Graph Transformer based encoder and a Graph Attention Network (GAT) 21 based decoder.
Element volume loss is selected within the training loss, instead of the coordinates loss used in
previous works 19 20 This offers the advantage that element volume loss not only provides supervision signals but also penalizes negative volumes (i.e., inverted elements), thereby reducing mesh tangling and enhancing robustness. We construct a PDE--independent dataset by generating random generic fields for model training. The trained model can be applied in zero-shot manner to move meshes with different sizes and structure, for solvers applicable to different PDE types and boundary geometries. We comprehensively demonstrate the effectiveness of our methods on different cases including advection. Savier-Stokes examples. as well as real-world tsunami simulations on meshes with different sizes and structures. Our UM2N outperforms existing learning- based mesh movement methods in terms of the benchmarks described above. In comparison to the sophisticated Monge-Amp PDE-solver based method, our approach not only significantly accelerates mesh movement, but also proves effective in scenarios where the conventional method fails. The trained model can be directly integrated into any mesh based numerical solvers for error reduction and acceleration in non-intrusive way, which can benefit various engineering applications suffering trade-offs between accuracy and computational cost.
Our main contributions are listed as follows:
2

---

We present the UM2N framework, which once trained can be applied in a non non-intrusive, shot manner to move meshes with different sizes and structures, for solvers applicable to different PDE types and boundary geometries.
We propose PDE-independent training strategies and mesh tangling aware loss functions for our Graph Transformer and GAT-based architecture.
We demonstrate the universal ability of UM2N on different PDEs and boundary geometries: Advection, Navier-Stokes examples, and a real-world tsunami simulation case.
2
Related Works
Neural PDE solver. Neural PDE surrogate models can significantly accelerate the PDE solving process, There are three main groups of existing works. Physics-informed Neural Networks (PINNs) 22 23 are the first group, modeling the PDEs through implicit representations by neural networks. PINNS require knowledge of the governing equations and train neural networks with equation residuals from the PDEs with given boundary conditions and initial conditions. Neural operators are another group of methods for learning to solve PDEs which seek to learn a mapping from a function describing the problem to the corresponding solution function 7 g 26). The third group is mesh-base PDE solvers with deep learning, including the utilization of CNNs and GNNs for processing structured and unstructured meshes [15 27 32) Our method can be considered be related to the neural operator approach, although it learns a mapping from monitor function values to the potential field describing the mesh movement instead of the PDE solution directly. In addition, the networks in the proposed model share similar ideas to mesh-base PDE solvers for processing unstructured meshes.
Learning for mesh generation and adaptation. Deep learning methods have been advanced for applications in mesh generation and adaptation. Two tasks need to be differentiated here, the first involves modeling the ideal mesh resolution or density, e.g. through learning a metric or error indicator. The second involves the step that actually adapts the mesh in response, where we can distinguish between h-adaptivity, in which the connectivity structure of the mesh is changed to refine coarsen the mesh, and r-adaptivity which maintains the connectivity and only moves the vertices.
An example of the first task is MeshingNet (IO] which employs a neural network to establish the requisite local mesh density, subsequently utilized by a standard Delaunay triangulation-baset mesh generator. Similarly, the determination of optimal local mesh density for the purpose of mesh refinement is described in [12). In [13], the authors propose a learning-basee model to determine optimal anisotropy to guide h-adaptation. [15] introduces a
approach for predicting the sizing field for use with external remesher. Instead of learning metric, some works learn to manipulate the mesh directly. [ [8] formulate the h-adaptation process as a reinforcement learning paradigm, learning policy that guides to adjust the mesh elements.
r-adaptation (or mesh movement) is generally formulated in fundamentally different manner to h-adaptation. Leading r-adaptation methods are based on mapping between a fixed reference computational domain and the physical domain encompassing the adapted mesh, which established through the solution of nonlinear auxiliary PDE. [19] first proposed the use of a GNN based mesh movement network. 220) investigated the unsupervised learning of mesh movement for improving performance of neural PDEs solvers. [33] introduced a learning based mesh movement method specially tailored for computation fluid dynamics simulation of airfoil design.
Our work focuses specifically on r-adaptation, i.e., mesh movement methods. The approaches proposed in previous work require re-training the model when applied to different problems, or can only generalize within a limited range of PDE parameters. We propose a mesh movement network that aims to be universal across applications to different PDEs without retraining.
3 Preliminaries and Problem Statement
Monge-Ampère mesh movement. A mesh movement problem can be defined as the search for the transformation between fixed computational domain Pe. and a physical domain Ωp. Coordinates in these spaces are denoted & and x, respectively. The mesh Hc defined in the computational domain is the original, often uniform mesh. The mesh Hp defined in the physical domain is the
3

---

adapted (or moved) mesh The goal of mesh movement is to find a mapping a: = (() between the continuous coordinate fields of f2g and Slp, although in practice we deduce mapping of the discrete vertex sets of Hc and Hp. While the coordinate transformation can also be used to accommodate time-dependent moving boundaries [34] or the optimisation of the shape of the domain, here we focus on its application to achieve variable resolution such that the solution Me of PDE solved on the adapted mesh has higher accuracy than the solution dgg solved on the original mesh.
A monitor function m over the spatial domain is used to specify the desired mesh density, which can be based on various characteristics of the solution field u of the PDE. It prescribes where the mesh should be refined or coarsened, i.e. large monitor values indicate where high mesh resolution is required. Therefore, the goal of the mesh movement process can be rephrased as finding a mapping so that m is equidistributed over the adapted (i.e., physical) mesh [35
(1) where J is the Jacobian of the map x f(E) with respect to the computational coordinates &. det(J) denotes the determinant of the Jacobian which corresponds to the relative change in volume under the transformation, and is a normalization constant. By using concepts from optimal transport theory [36], the problem can be constrained to have unique solution, with the deformation of the map expressed in terms of the gradient of a scalar potential & such that:
vu(a) del(J) = 0,
xx{)) { + Ve((g)-
Substituting the additional constraint described by equation into equation ① gives nonlinear PDE of Longe-Ampère type:
m(x)) det(I+ (((E)) = 0,
where I is the identity matrix and (Ø) is the Hessian of o, with derivatives taken with respect 82 to E. i., H H()); = ,, It should be noted that guarantees for existence and uniqueness, and convergence of solution methods for the Monge-Ampère equation generally rely on the domain being diffeomorphic with convex domain (e.g. 377. In practice, the method is known to break down in some cases where the domain is not simply-connected or has non-smooth boundaries.
The MA equation 3 is an auxiliary PDE which is purely associated with the the mesh movement process, and is independent of the underlying PDE or physical problem we wish to solve, i.c., the MA equation keeps the same form for different PDEs. This decoupling benefits learning-bass based methods in terms of generalization properties, i.e., a well-learned MA neural solver has the potential to be applied to different problems with a small cost of fine-tuning or even without re-training.
4
UM2N: Universal Mesh Movement Network
4.1 Framework overview.
The proposed UM2N framework is shown in Fig. [ Given input mesh, vertex features and edge features are col- lected separately. The coordinates and monitor function values are gathered from the vertices and input into graph transformer to extract embeddings. The embedding vector z obtained from the graph transformer encoder is then con- catenated with the extracted edge fea- tures e to serve as the input for the Graph Attention Network (GAT) decoder. The decoder processes this combined imput along with a mesh query to ultimately produce an adapted mesh.
Tansomer Frooder
Sdi-Mtrtoo
Vettow
Excoding
Gkp fuaus
NN Blocke
MP
GAT Dwonter
Figure 1: Overview of Universal Mesh Movement Network.
38 are a popular architecture which have been They have strong expressivity and can naturally handle irregular data structures such as unstructured meshes. Their attention mechanism can capture both the local and global information for a vertex of interest in a mesh. Here we use
Graph Transformer encoder. Transformers successfully applied for neural PDE solvers 25
26 39
In what follows, we abuse notation slightly by also using& & to refer the the (discrete) vertices of He, rather than the (continuous) coordinate field. Similarly, we also use denote the vertices of Hr.

---

graph transformer encoder as feature extractor. The input features include the coordinates of the original mesh & and monitor values mg. The coordinates serve as the positional encoding. Features are concatenated and encoded into Q (query), K (key) and V (value) as inputs using ML.s for a self-attention module. The encoder output embedding 2 are fed into a downstream GAT based deformer for mesh movement.
Graph Attention Network (GAT) based decoder. We choose GATs to construct our decoder. Its attention mechanism can help constrain the mesh vertex movement to within one hop of its neighbors which assists in alleviation of mesh tangling issues. The decoder consists of N GAT blocks. The first GAT block takes mesh query &, and the embedding from the transformer encoder as well as the edge features as inputs. For each following h block, the imputs consist of the coordinates of the initial mesh &, the coordinates of the intermediate moved mesh g (kk-1) and extracted features from the previous layer (K--1).
4.2 PDE-independent dataset and training.
In the existing work of neural mesh movement 19 20 proposed models are trained on solutions of PDEs, with the training data generated by solving one specific type of PDE, limiting generalizability of the trained model. Aiming here to train universal mesh movement networks, we construct a PDE-independent training dataset D {d (E. me: V,,Tr) J,)). where &. mg denotes the original mesh and monitor values as the model's input, and V T, as the pre-caleulated (reference) mesh's 's vertex coordinates and elements ground truth. To build the dataset, randomly generate generic solution fields, which are composed as the summation of a random number of Gaussian distribution functions centred in random locations, and with random widths in different directions to introduce anisotropy. samples are shown in Figure [AT] refer to Appx.B]tt Bor a detailed description. The generated fields can be interpreted as representing solutions of any PDE. For mesh movement, widely used method to translate the PDE solution u(x) into a monitor function, is the Hessian based formula [6]
[|H(
mxx) 1++
(4)
max ([Hll))l'
where [u]] indicates the Frobenius norm of the Hessian of u and a is a user chosen constant. This choice results in small cells where the curvature of the solution is high, with a ratio of + a between the largest and smallest cell volumes. Our network is trained on the values of the monitor function only, which can be computed relatively cheaply from the PDE solution. The expected movement of the mesh vertices is computed by solving the Monge-Ampère equation (3) for each of the monitor functions based on the randomly generated PDE solutions. After training, the model can then be applied using only the monitor values as input, independent of the PDE that is being solved. Other choices of formulae for the monitor function in terms of u may be appropriate in different cases, but again these can be applied without re- re-training the model, e.g., the flow-past wow-past--cyye past-a-cylinder and tsunami test cases the following section use monitor function based on the gradient of the solution.
4.3 Loss functions.
Given the dataset DD defined above, the final objective is find model parameters 0, by minimizing the total loss Lrota-- Thus, we can formalize the process as
arg Lrocat (0) := Lool (0) + (e).
(5)
where Leot; Led represents the element volume loss and Chamfer distance loss respectively, which are defined later. Xed, Apol > 0 represent paremm-prameters balancing these two effects.
We denote the modified (adapted) mesh with model parameters A as mc; where V. = {m represents the coordinates of vertices and T. = {tc ) represents the elements with x,, t; as the 1t vertex and element. For ease of presentation, we omit the here. Similarly, we define y, 9, V,','s 7t vertex and element.
(Va, a. T.).
Element volume loss. In contrast to the coordinate loss used in previous work 19 20
here we
use the element volume loss for training. Element volume loss is computed as the averaged volume difference between each element in the adapted mesh (T.) a)) and the reference mesh (T,) computed from the MA method. It is defined as
5

---

Luoi me V..DT.JE J.JED
vol(e)) -
(6)
T
where Vol is the function computing the volume of given element t.
The element volume loss is inspired by the equidistribution relation in equation Intuitively, the equidistribution relation enforces deformed mesh elements such that their volumes correspond to rescaling by the monitor function value m(x). Therefore, the element volume loss is utilised to encourage the model to learn move the mesh so as to conform to the equidistribution relation,
under the guidance of MA method, given that the MA method provides one good way to achieve this relation. In addition, the element volume loss penalizes negative Jacobian determinants, which helps prevent element inversion, i.e., mesh tangling.
Chamfer distance loss. Chamfer distance finds for all vertices in the first mesh, the nearest vertex in second mesh and sums the square of these distances. It encourages the model to output a mesh which has a similar spatial distribution of vertices to that of the reference mesh. To achieve parte-too-verter alignment, the bidirectional Chamfer distance utilized without additional sampling. It defined as
Lea 0) V.
Σ =.EVa
min || - -,,lla + ujeV.
Σ RyEV, [
min || v,lla (7) *.EV.
[V
5 Experiments
5.1 Experiment setups.
Unlike existing approaches mesh movement, such as 19 20 by-case basis, our study aims to explore the universal applicability of the proposed UM2N. Therefore, all examples shown in this section are tested in zero-shot generalization manner, i.e., without re-training. In addition to PDE type, our training meshes are of small size typically comprising 500 vertices, while our test meshes have far more vertices: Advection (2, 052 vertices), Cylinder (4, 993 vertices), Tsunami (8, 117 vertices), which also tests the model's scalability. There are examples with more complicated settings, please refer to Appx.
which re-train their model on a case-
Training. The training dataset consists 600 randomly generated generic solution fields and original meshes with 463 and 513 vertices. The model is trained using the Adam optimizer. The training and experiments are performed on an Nvidia RTX 3090 GPU.
Metrics. The main metric for evaluating mesh movement quality is underlying PDE solution error reduction (ER) ratio. Here PDE error is approximated by the difference between solutions obtained coarse mesh and accurate solutions which obtained The PDE error reduction ratio is the difference between the PDE error from an original mesh and that from an adapted mesh. Another important metric for mesh movement is mesh tangling. Once mesh tangling happens, the mesh is invalid for a numerical solver, which breaks the simulation.
very high resolution mesh.
5.2 Benchmarking mesh movement.
Table 1: Quantitative results across mesh movement methods. ER indicates PDE error reduction ratio. "Fail" indicates that mesh tangling happens during the simulation.
Swirl
Cylinder
Helmholtz
Methods
(56) 1
Time (ms) J
ER (%)
Time (ms) J
ER (%)
Time (ms) 1
MA 36
37.43
5615.78
Fail
17.95
3265 .33
M2N
14.15
25.01
16.32
UM2N (Ours)
35.93
17.66
61.29
19.25
17.08
4,86
In the following section, we benchmark the mesh movement using both non-learned Monge-Ampéns (MA) mesh movement implemented Movement [40] and learning-basee based baseline M2N [99 over three different scenarios to compare with UM2N. Table shows the quantitative results in three different scenarios, Swirl, Cylinder, and Helmholtz. An additional qualitative result of Tōhoku Tsunami simulation, is provided to show our model can handle highly complex boundaries in real- world scenarios. Moreover, to give fair comparison, we re-trained the model M2N on our training dataset before evaluation.

---

Initial State
Drs Tecaction
t= 0.0s Initial State (Targel)
High Res
Original
M2N
UM2N
MA
Evior L2 L2-eem
1.0s End State
522
100
Figure 2: Results for swirl case. The plot shows the end state of the swirl simulation (bottom row) and corresponding meshes (top row). As the process is reversible, the end state should be consistent to the initial state. can be observed that the solution obtained high resolution almost recover the initial state. The proposed UM2N outperform M2N and achieve comparable performance to MA method. Quantitatively in the rightmost plot, the orange line (UM2N) and blue line (MA) best suppress error accumulation along timesteps.
Helmholtz. The Helmholtz case is time-independdnt problem solving 2D Helmholtz equation as in (A2). The source terms of the equation are generated use formula {AT). Our model achieves better error reduction than M2N and comparable to MA method in significantly less time.
Swirl. The swirl case considers a time-dependent pure advection equation. In this scenario, the initial tracer field takes a narrow ring shape and advected within swirling velocity field, see Eq. (A3) and (A4) in Appx.[] The velocity swirling direction is anti-clockwise in the first half of the simulation and clockwise for the second half. Given this setting. as the advection process is reversible, the final state should be consistent with the initial state. This constitutes problem of significant multi-resolution complexity, as the extremely narrow ring requires high resolution to accurately resolve the dynamics at the sharply defined interfaces of both inner and outer sides.
The results are shown in Fig. 2 As shown in Table our UM2N shows an average 35.93% error reduction which clearly outperforms M2N's 14.15%. It is close to that of the full Monge-Ami Ampère (MA) mesh movement approach which achieves an error reduction of 37.43% but with significantly lower computational time.
Flow past cylinder. Simulation of flow past cylinder in channel is classic and challenging multi- resolution case of computational fluid dynamics, solving the Navier-Stoker equations. The cylinder and the channel's top-bottom surfaces require high resolution mesh to resolve the boundary layers [1 Given appropriate Reynolds number, the well-known von Kármán vortex street phenomena can be observed within the wake flow which also benefits from higher resolution resolve the time-varying evolution of vorticity structures. The drag On and lift Cz. coefficients for the cylinder other quantities of interest in this problem. Here
a benchmark configuration from 42
please refer to Appx C.3 for more details.
The simulation is run for 8, 000 time steps of size 0.001 s, i.e., 8 physical time for the whole simulation. The original mesh a uniform mesh with triangular elements and 4, 993 vertices. The reference solutions obtained on a high resolution mesh with 260, 610 vertices.
We investigate the dynamics the area around the cylinder and in the wake flow respectively. For the former, we use the measure of On as shown in Fig. 3 The results for Cy. is shown in Fig (A10
Appx. E For the wake flow, we use cross-section probes at x [0.5, 1.0, 1.5, 2.0] to measure the
intensity of vorticity spatially along the y-axis as shown in Fig. 4
Qualitatively, it can be observed that the proposed method moves the mesh manner that captures the fluid dynamics for the whole domain in general (see upper part of Fig. 33.. At the cylinder and top-bottom surface, the vertices are moved increase the resolution in order to resolve the boundary layers as expected. Quantitatively, the drag CD and lift Ct coefficients computed on our moved mesh show 61. 31.29% average error reduction compared to that of original mesh as shown in lower part of Fig.3] The proposed UM2N approach not only improves the accuracy of these coefficients in magnitude but also prevents their periodic variation from phase shifting. For the wake flow, as shown in Fig. 4 we selected a single snapshot at / = 4.0s for qualitative investigation. Our UM2N also shows an accuracy improvement for vorticity intensity on data sampled from all four cross-section probes comparing to the that original mesh
7

---

Original Mesh
UM2N
High Res
Original
UM22
4.00
t= 4.05
t= 4.10
- 4.15
t= 4:20
Drag Coefficient (Cp)
Res 1260610 verticesi
2.8
Original
29
Co UM2N (4933 vertices)
-32
-33
Figure 3: Results for the flow past cylinder in time. the upper part, visualize 5 snapshots of adapted meshes vorticity intensity from 4.00s to 4.20s. Please refer the video in the supplementary materials. shows that our UM2N can output meshes which adapt to the dynamics around the cylinder. can be observed from the zoom-in in mini-plots at the top-left that UM2N reduces errors/noise in vorticity intensity compared to the original mesh. In the lower part, the bluc, orange and green lines show the drag Cp coefficients obtained on high resolution fixed mesh, UM2N adapted mesh and the original coarse mesh the first seconds (i.e., UM2N output mesh improves the accuracy of Cp in magnitude and periodicity compared to the original mesh. Adopted Mesh (UMEN)
6000 steps). can observed
Vorticity Frror Map (Original Mesh)
Vorticity
Vorticity Map (UM2N)
Vorticity at cross-sed sessisseciion (x=0.5)
Vorticity cross sess-section (x=1.0)
Vorticity at cress-saction (x=1.5)
Vorticity crass section (x=22.0)
10
13
-10
-10
-20
-30
0.0
0.1
0.2
0.5
0.6
0.0
0.1
0.2
0.5
0.4
0.0
01.1
0.2
C.5
3.4
0.0
0.1
0.2
0.5
0.4
High
Original
Figure 4; Analysis wake flow vorticity for the cylinder The adapted mesh output by UM2N (sec top-lcft part of the plot). Qualitatively, the maps the top-right part indicate that UM2N reduce the vorticity difference to the high resolution mesh result in general. are shown in the lower part of the figure. There are four cross-section probes place on x [0.5, 1.0, 1.5, 2.0] marked by the black dash line in the middle left plot. The green line (our UM2N) is much more consistent to the blue dash line (high resolution) compared to the orange line (original mesh), which further verifies that method can reduce the PDE emoos.
quantitative comparison results along the probes
Note that intended to compare UM2N to M2N and the conventional full MA method. However, both MA and M2N fail i.e., the solver diverges, within 30 time steps due to mesh tangling and thus provide insufficient data points for comparison. For comparisons with a non-tangled M2N case, please refer to the simplified laminar flow wow-past-cylinder example presented in Appendix [.I
Töhoku tsunami simulation. Here we present the case of simulation of the 2011 Töhoku Tsunami, to show that our methods can be applied to world scenarios with boundaries describing highly complex geometries. The tsunami is simulated by solving the shallow water equations using the Thetis framework 43 For details on the datasets and software used set up the Tōhoku case study,
Appx. B.2 Note that the mesh (8, 117 vertices) has arbitrarily irregular boundary. It can be

---

Step
Step = 10
Step-20 -0
Step
Tōhoku Tsunami
Simulation
0.90
(eotav)s)
Stop
Step
Figure 5: Qualitative results for the Töhoku tsunami simulation. The boundary of simulation is generated based on the real coastline data (see mesh overlaid satellite map). show snapshots the wave clevation from 80 steps of the tsunami simulation enhanced hy our UM2N. In these visualizations, red and blue hues indicate wave elevations that respectively, ahove below mean sea level. Regions exhibiting significant elevation magnitudes necessitate increased resolution accurately resolve the underlying dynamics. Our method dynamically robustly adjusts the mesh increase resolution at the wave front, thereby effectively tracking its propagation.
Mesh Tangling
UM2N (Ours)
Monge-Ampère
Figure Handling complex boundaries. Monge-Ampère method generates inverted elements, resulting in mesh tangling near the coastline as shown in the areas delineated by red rectangles. Our UM2N can smoothly handle the elements near the coastline without mesh tangling.
observed that the mesh moves in manner that helps track the propagation with enhanced resolution, as shown in Fig.5) Our model can also robustly handle the highly complex boundary without mesh tangling near the coastline. As comparison, the conventional Monge-AApénn (MA) mesh movement approach struggles to handle the complex geometries, i.e., fails to converge when solving the non-linear MA equation, resulting in highly tangled elements the coastline as shown in Fig. Therefore, our method not only provides a more efficient approach but also show advantages dealing with complex boundaries compared to the full MA mesh movement approach. Please the full video of this case in the supplementary materials.
5.3 Tangled results of the UM2N model
There are in fact still cases where UM2N might fail. To investigate these, we perform additional experiments as shown in Figure Starting from rectangle, we gradually distort the geometry to more and more non-convex shape, i.e., the case becomes more challenging from top bottom. The UM2N finally fails in the extreme case at sudden jumps in the boundary accompanied by large variation in the required resolution as shown in the bottom row of Figure RI: sudden constriction of the flow leads to tangling with UM2N in the two left corners of the channel. In less extreme cases, UM2N does produce valid mesh whereas the original MA method still fails seen in the second to bottom rows in Figure 7).

---

Non-tangled
Tangled
H MA
Original
UM2N
Monitors
Figure 7: Tangled Case of UM2N and MA. Red/blue indicate high/low monitor values, i.e. the mesh is expected stretch towards the right direction. The is more and more challenging from top to bottom.
5.4
Ablation study
Volume loss vs coordinate loss. We denote the
Table 2:
Ablation Study for design choices of learning monitors and element volume loss.
UM2N variant trained with coordinate loss [99]
UM2N-coord. As shown in Table UM2N-coord achieves comparable error reduction (31.21%
Helmholtz ER (%)
Swirl ER (%)
Cylinder ER (%) f
Method
the proposed UM2N (35.93%) on the Swirl case.
However, fails (i.e., encounters mesh tangling) on the Cylinder case which has more complex boundary geometry. This indicates that the ele-
UM2N-coord
16.64
31.21
Fail
UM2N-sol
13.52
-0.35
39,94
UM2N (Ours)
17.08
35.93
61.29
ment volume loss is mesh-tangling aware, i.e,, helps to prevent producing inverted elements.
Monitors PDE solutions inputs. We denote the UM2N variant using the PDE solution input, instead of monitor function values, as UM2N-sol. The UM2N-sol is trained on solutions of the Helmholtz PDE. The UM2N-sol shows much inferior performance (-0.35%) compared to UM2N (3593%) on the Swirl case as well as on the Cylinder case. This indicates that learning mesh movement from monitors directly improves the generalisability of the model.
Conclusions
We introduce the Universal Mesh Movement Network (UM2N), which once trained on our generated PDE-independent dataset, can be applied in a non-intrusive, zero-shot manner to move meshes with different sizes and structures, for solvers applicable different PDE types and boundary geometries. Our method demonstrates superior mesh movement results on Advection and Navier- Stokes PDE examples as well as real-world tsunami simulation case. Our UM2N outperforms the existing learning based method and achieves comparable PDE error reduction to the far more costly Monge-Ampére (MA) PDE based approach. Our UM2N demonstrates effectiveness cases with complex boundary geometries where existing learning and conventional MA based methods fail. Our method also shows significant acceleration compared MA based mesh adaptation. See Appx. for discussions limitations and broader impacts.
10

---

Acknowledgment
Thanks to Jinghan Jía for testing UM2N performance Nvidia A100.
References
[1] Joseph G. Wallwork, Nicolas Barral, Stephan C. Kramer, David A. Ham, and Matthew D. Piggott. Goal-oriented enror estimation and mesh adaptation for shallow water modelling. SN Applied Sciences, 2(6):1053, 2020. ISSN 2523-3971.
doi: 10. 10.1077/44422--00--45555
[2] Pantelis R Vlachas, Georgios Arampatzis, Caroline Uhler, and Petros Koumoutsakos. Multi- scale simulations of complex systems by learning their effective dynamics. Nature Machine Intelligence, 4(4):359-366, 2022.
[3] Chi Zhang, Massoud Rezavand, and Xiangyu Hu. A multi-resolluionn SPH method for fluid- structure interactions. Journal of Computational Physics, 429: 10028, 2021. ISSN 0021-9991. doi: iftp://ooorrrr.111/..........pp
100028.
[4] Michael Athanasopoulos, Hassan Ugail, and Gabriela González Castro. Parametric design of aircraft geometry using partial differential equations. Advances in Engineering Software, 40(7): 479 486, 2009.
[5] The carbon footprint of computational research. Nature Computational Science, 388):659-659, 2023.
[6] Maziar Raissi, Paris Perdikaris, and George E Karniadakis. deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686-707, 2019.
neural networks:
[7] Lu Lu, Pengzhan Jin, and George Em Karniadakis. Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators. arXiv preprint arXiv; 1910.03110 2019.
[8] Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differen- tial equations. International Conference on Learning Representations (ICLR), 2021.
[9] Weizhang Huang and Robert D. Russell. Adaptive Moving Mesh Methods. Applied Mathemati- cal Sciences. Springer Science+Business Media, LLC, 2 edition, 2011.
[10] Zheyan Zhang. Yongxing Wang, Peter K. Jimack, and He Wang. MeshingNet: A new mesh generation method based on deep learning, 2020.
[11] Jiachen Yang. Tarik Dzanic, Brenden K. Petersen, Jun Kudo, Ketan Mittal, Vladimir Z. Tomov, Jean-Sylvain Camier, Tuo Zhao, Hongyuan Zha, Tzanio V. Kolev, Robert W. Anderson, and Daniel Faissol. Reinforcement learning for adaptive mesh refinement. CoRR. abs/2103.01342..
2021. URL wtt.ttttt//aarittia org/ abs 2203 2134.01342
[12] Keefe Huang, Moritz Krügener, Alistair Brown, Friedrich Menhorn, Hans-Joachim Bungartz, and Dirk Hartmann. Machine learning-based optimal mesh generation in computational fluid dynamics. arXiv preprint arXiv:2 /2923, 2021.
[13] Krzysztof J. Fidkowski and Guodong Chen. Metric-based, goal-oriented mesh adaptation using machine learning. Journal of Computational Physics, 426: 109957, 2021. ISSN 0021-9991. doi: http:://oi..rrg//..o0doij...pppp 109977 URL http::tts////// http://wwww.cennnnd.rcce com/
[14] Wu Tingfan, Liu Xuejun, An Wei, Zenghui Huang, and Lyu Hongqiang. A mesh optimization method using machine learning technique and variational mesh adaptation. Chinese Journal of Aeronautics, 2021.
[15] Tobias Pfaff, Meire Fortunato, Alvaro Sanzhez-Gonzalez, and Peter W Battaglia. Learning mesh-based simulation with graph networks. International Conference on Machine Learning (ICLR), 2021.
11

---

[16]
Joseph G Wallwork, Jingyi Lu, Mingrui Zhang, and Matthew D Piggott. E2N: error estimation networks for goal-oriented mesh adaptation. arXiv preprint arXiv:2207.11233, 2022.
[17]
Tailin Wu, Takashi Maruyama. Qingging Zhao, Gordon Wetzstein, and Jure Leskovee. Learning controllable adaptive simulation for multi-resolution physics. In The Eleventh International Conference on Learning Representations, 2023. URL htttp:://ppenrevie. net.ieruuee//rmm
id id-PbfgkZHHbbE
[18] Corbin Foucart, Aaron Charous, and Pierre FJJ. Lermusiaux. Deep reinforcement learning for adaptive mesh refinement. Journal of Computationnl Physics, 491:112381, 2023. ISSN (021- 9991. doi:
WRL.https://wwa http::/://ww..ssceeedrrre.. scw.scedireccrcct
com http:///saacceearrreeee////////9
[19] Wenbin Song, Mingrui Zhang, Joseph G Wallwork, Junpeng Gao, Zheng Tian, Fanglei Sun, Matthew D Piggott, Junqing Chen, Zuoqiang Shi, Xiang Chen, and Jun Wang. M2N: Mesh movement networks for PDE solvers. Advances in Neural Information Processing Systems, 35, 2022.
[20] Peiyan Hu, Yue Wang, and Zhi-Ming Ma. Better neural PDE solvers through data-free mesh movers. In The Twelfth International Conference on Learning Representations, 2024.
[21] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint 1711v11710.10003, 2017
[22] Shengze Cai, Zhiping Mao, Zhicheng Wang, Minglang Yin, and George Em Kamiadakis. Physics-informed neural networks (pinns) for fluid mechanics: A review. Acta Mechanica, 37 (12):1727-1738, 2021.
[23] Jiequn Han, Amulf Jentzen, and Weinan E. Solving high-dimensional partial differential equations using deep learning. Proceedings of the National Academy of Sciences, 115(34): 8505-88 100,8500,200 2018.
[24] Zijie Li, Kazem Meidani, and Amir Barati Farimani. Transformer for partial differential equations' operator learning. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. httplhttss:/oppereeeeeeeeee hwwpperrevieeenee/eeeeeeeeeeeeee
[25] Zijie Li, Dule Shu, and Amir Barati Farimani. Scalable transformer for PDE surrogate modeling. Advances in neural information processing systems (NeurIPS), 36, 2023.
[26] Zhongkai Hao, Chengyang Ying, Zhengyi Wang, Hang Su, Yinpeng Dong, Songming Liu, Ze Cheng, Jun Zhu, and Jian Song. Gnot: A general neural operator transformer for operator learning. International Conference on Machine Learning (ICMI), 2023.
[27] Nils Thuerey, Konstantin Weissenow, Lukas Prantl, and Xiangyu Y. Hu. Deep learning methods for Reynolds-averaged Navier-Stokes simulations airfoil flows. AIAA Journal, 2018. URL
https://gpp...mmmttccccchll.l hwwp:aaoo///opooreeee hww.a/aaa..rgr/ooossssS:////. 67855877
(28) Alvaro Sanchez--Gonzzler,, Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, and Peter Battaglia. Learning to simulate complex physics with graph networks. International Conference on Machine Learning (ICML), 2020.
[29] Johannes Brandstetter, Daniel Worrall, and Max Welling. Message passing neural PDE solvers. International Conference on Learning Representations (ICLR), 2022.
[30] Steeven JANNY, Aurèlien Bênèteau, Madiha Nadri, Julie Digne, and Nicolas THOME. Ea- gle: Large-scale learning of turbulent fluid dynamics with mesh transformers. International Conference on Leurning Representations, 2023.
[31] Lukas Prantl, Benjamin Ummenhofer, Vladlen Koltun, and Nils Thuerey. Guaranteed con- servation of momentum for learning particle-based fluid dynamics. In Conference on Neural Information Processing Systems, 2022.
[32] Xu Han, Han Gao, Tobias Pfaff, Jian-Xun Wang, and Li-Ping Liu. Predicting physics in mesh- reduced space with temporal attention. International Conference on Learning Representations, 2022.
12

---

[33] Jian Yu, Hongqiang Lyu, Ran Xu, Wenxuan Ouyang, and Xuejun Liu. Flow2Mesh: A flow- guided data-driven mesh adaptation framework. Physics of Fluids, 36(3):037124, 03 2024.
[34] J.M. Nunez Rattia, J.R. Percival, S.J. Neethling, and M.D. Piggott. Modelling local scour near structures with combined mesh movement and mesh optimisation. Journal of Computational Physics, 375:1220-12333 2018.
[35] Chris J Budd, Weizhang Huang, and Robert D Russell. Adaptivity with moving grids. Acta Numerica, 18:111-241, 2009.
[36] Andrew T. T. McRae, Colin Cotter, and Chris J. Budd. adaptivity on the plane and sphere using finite elements. SIAM Journal on Scientific Computing, 40(2):A1221-A114) 2018. doi: 10. 1137/166M1005111
mesh
[37] Mohamed M. Sulman, J.F. Williams, and Robert D. Russell. An efficient approach for the numerical solution of the Monge-Ampère equation. Applied Numerical Mathematics, 61): 298-307, 2011.
(38) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, tukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances neural information processing systems (NeurIPS), 30, 2017.
[39] Shuhao Cao. Choose a transformer: Fourier Galerkin. Advances in neural information processing systems (NeurIPS), 34, 2021.
[40] Joseph G. Wallwork, Stephan C. Kramer, Mingrui Zhang, and Davor Dundovic. Movement, May 2024. URL/ataps://aoiorrg//0 http://1..522/zeooddoo 11239647
[41] Hermann Schlichting and Klaus Gersten. Boundary-Layer Theory. Springer, Berlin Heidelberg,
9 edition, 2017. ISBN 978-3-662-52917-15 doi: 10. (0.1007/978-3--62-522999
[42] M. Schäfer, S. Turek, F. Durst, E. Krause, and R. Rannacher. Benchmark Computations of Laminar Flow Around a Cylinder, pages 547-566. Vieweg+ Teubner Verlag, Wiesbaden, 1996. ISBN 978-3-322-89849--4 doi:
URL httph://too........ org/
10. N0.1007/998--3200 --0-888-------------------9-----
[43] T. Kärnä, S. C. Kramer, L. Mitchell, D. A. Ham, M. D. Piggott, and A. M. Baptista. Thetis coastal ocean model: discontinuous Galerkin discretization for the three-dimensional hydro- static equations. Geoscientific Model Development, 11(11):33594433 gmd- 11-4359-20-1 URL https:///m//d coperni cus .wt. wwtggarticles/1//// 21/tc///eee///////00209
2018 doi: 10.5194/
[44] Florian Rathgeber, David A Ham, Lawrence Mitchell, Michael Lange, Fabio Luporini, An- drew TT McRae, Gheorghe- Teodor Bercea, Graham R Markall, and Paul HJ Kelly. Firedrake: automating the finite element method by composing abstractions. ACM Transactions on Mathe- matical Software (TOMS), 43(3):1-27, 2016.
[45] Martin Alnæs, Anders Logg, Kristian Ølgaard, Marie Rognes, and Garth N Wells. Unified form language: A domain- language for weak formulations of partial differential equations. ACM Transactions on Mathematical Software (TOMS), 40(2):1-37, 2014.
[46] Satish Balay, Shrirang Abhyankar, Mark F. Adams, Steven Benson, Jed Brown, Peter Brune, Kris Buschelman, Emil Constantineseu, Lisandro Dalcin, Alp Dener, Victor Eijkhout, Jacob Faibussowitsch, William D. Gropp, Václav Hapla, Tobin Isaac, Pierre Jolivet, Dmitry Karpeev, Dinesh Kaushik, Matthew G. Knepley, Fande Kong, Scott Kruger, Dave A. May, Lois Curfman McInnes, Richard Tran Mills, Lawrence Mitchell, Todd Munson, Jose E. Roman, Karl Rupp, Patrick Sanan, Jason Sarich, Barry F. Smith, Stefano Zampini, Hong Zhang, Hong Zhang, and Junchao Zhang. PETSc/TAO users manual. Technical Report ANI-211/39 Argonne National Laboratory. 2024.
Revision 3.21,
[47] Christophe Geuzaine and Jean- FraannFs Remacle. Gmsh: A 3-d finite element mesh generator with built-in pre- and post-processing facilities. International Journal for Numerical Methods Engineering, 79(11)) 2009. doi: https://doi.coo www.:///.000000mmee5555555 URL httpss // //onlinelibrary. wttp://cccooo.a/aaeeeeee 100 1002/ nme. .2799
13

---

[48] Alexandros Avdis, Adam S. Candy, Jon Hill, Stephan C. Kramer, and Matthew D. Piggott.
Efficient unstructured mesh generation for marine renewable energy applicationn.. Renewable En- ergy, 11::842 856, 2018. ISSN 0980- doi: http:://hoo...////o/oooooeeeeeee
001.0 .088.
URI. http:///www....eeooeoooc///////a
[49] Mingrui Zhang, Chunyang Wang, Stephan C. Kramer, and Joseph G. Wallwork. Um2n, October 2024. URL http::://ai.oorgg///// 000.5581/222 2281//enodo.....
http:////ooooorr////.//a//aeooo/ 14009265
[50] Päl Wessel and Walter H. F. Smith. A global, self-consistent, hierarchical, high-resolution shoreline database. Journal of Geophysical Research: Solid Earth, 101((4):8741-8743, 1996. doi: https://doi.orr/////// http:///0000000///0/0000000000
https:tt:///guuuutt http:////aaapppppaaaaeeeeeeeeeee inwiinnbrrrrr......o willy
10. 10299999999900444 L000
[51] Christopher Amante and Barry W Eakins. ETOPO1 are-minute global relief model: procedures, data sources and analysis, 2022. Accessed: 2022-03-22.
[52] Yoshimitsu Okada. Surface deformation due to shear and tensile faults in half-space. Bulletin of the seismological society of America, 75(4): 135-1154, 1985.
14

---

Appendix
We include additional experimental details and results in the appendix. Detailed information on the software used is provided in Appendix A A description of the dataset generation process is presented in Appendix B We provide the details of experimental settings cases the paper Appendix C To provide more holistic analysis for the proposed UM2N, include evaluations across various test cases, performance benchmarks, more ablation studies, and assessments on low-quality initial meshes in Appendix [ To demonstrate the the universal abilitt of the UM2N, additional experiments conducted under diverse settings are detailed in Appendix B We introduce the mesh movement- enhanced simulation, illustrating the non-intrusive integration of UM2N into numerical simulation pipeline in Appendix F provided in Appendix
discussion about the limitations and broader impacts of our approach is
A Software used in the paper.
In this paper, PDEs are solved using finite element methods with Firedrake in Python, uses the Unified Form Language (UFL) [45] domain -eeaai-specific language to represent finite element forms, and automatically generates C kernels for computational efficiency. Firedrake uses PETSc 46) for solving linear and nonlinear equations that result from discretisation, as well as for its underlying unstructured mesh representation.
44
Firedrake written
Original meshes (i.e., meshes before adaptation) are generated by either Firedrake or Gmsh (47] Gmsh meshing software especially designed for finite element methods, providing access to several meshing algorithms. In the case of the Töhoku tsunami simulation, the mesh of the Japan Sea generated using qmesh 48 a meshing tool specifically designed for coastal ocean modelling applications, which manages GIS (Geoscientific Information System)- datasets to construct valid input geometry used by Gmsh generate the mesh.
The conventional mesh movement strategy used to generate training data is implemented in Movement 40). which is itself written in UFL and Firedrake. Movement implements several mesh movement approaches, including two based on solutions of Monge-Ampère type equations. In this paper, we use the 'relaxation' approach, which solves the nonlinear MA equation in iterative fashion by introducing a pseudo-timestep (see [36] for details).
The code used to produce the results presented in this paper is publicly available at https:// gi github. com/mesh adaptation/UM211 (Release (4].) .) Code documentation hosted at lhttps
wwwm/mmeh-adapatttiiooo/o - awaphhhhddpttiiio.. wwaeeriiiinntttuuu..oo github.
B Dataset description.
B.1 Dataset generation pipeline.
We developed a pipeline for generating PDE-independent training data and further research. Through our pipeline, generic fields are generated and can be utilized solutions of any PDE. The monitor values computed from the generic field are used to guide the MA-based method to perform mesh adaption on the original mesh
We stack N randomly generated Gaussian distributions to form the generic field u. Concretely, they can be generated by the formula:
(x (x- -z
(y - 22
exp
(A1)
Ogy
where Ma and My denote the means of the distribution along two orthogonal directions in 2D domain, while oN and o represent the respective standard deviations.
All training data are generated within 2-D unit square domain. Equivalent size meshes (n {18, 20)) are generated for model training. Here we show samples of the generated fields in Fig. Al
15

---

02
26
Figure A1: Samples of generated generic fields.
B.2 Evaluation data for the Töhoku tsunami simulation.
As mentioned in Sec. the mesh of the Japan Sea is generated using qmesh [8] For this, provide qmesh with coastline data from (50). Bathymetry (sea bed topography) data is taken from the ETOPO1 dataset [5]] A highly idealised tsunami source condition used, comprised of sum of Okada functions 52)
C Details of experimental settings.
C.1 Helmholtz.
The Helmholtz equation used in the paper is defined as:
2
Vu = on r,
(A2)
where f is the source term and r denotes the boundary.
C.2 Swirl.
We use a similar setting to the unsteady 2D linear advection case in 18
Initial Condition.
1 (v 2
exp
20 + (y 30) ro)²
(A3)
The initial condition defines a ring centered at (No, yo) with an inner radius ro- The thickness of the ring 3o. In this paper, ro 0.2, (Io, Mo) (0.30, 0.30), and a 0.05/3.
Velocity Field.
el,,y,t)
9
(1)) sin? (Tar)) ( sin()
(1) sin (Tr) sin sin(2rr))
(A4)
10
10
In this paper, 0 < < 1, if < 0.5 then a(t) = if <= a(t) = =-1, iee, the velocity
direction is reversed / 0.5
Monitor Function.
[H(I]]
|G(w]
m(x) + max
(AS)
max (u)]] max [[((
where we set a = and = The [w)] and [|G(u)] are Hessian norm and gradient norm respectively.
16

---

Table A2: Performance benchmark on shown in Figure. A3
MA
UM2N CPU
UM2N
UM2N GPU (Nvida A100)
Hardware
CPU
GPU (Nvida RTX 3090)
Avg. inference time (8)
114.3
1.113
0.0487
0.0342
Specdup
102x
2347x
3312x
C.3 Flow past cylinder.
Setting The Reynolds number is 100, a quadratic velocity profile is imposed at the left boundary as inflow condition. At the right boundary zero pressure outflow condition applied. The cylinder and top-bottom surface are imposed with non-slip boundaries.
D More Evaluations.
High Res
UM2N
M2N
Figure A2: Non-tangled results M2N on Now-past-a-cclindee case.
D.1 Non-tangled results of other methods on the flow-past-cylinder.
We additionally perform a non-tangled now-past-cylinder shown in AA2 We simplified the setting: lower the Rrenynolds number by reducing the inflow velocity and set the top and bottom slip boundary, which finally makes it laminar flow. As shown in Fig[A2] both the UM2N and M2N give non-tangled meshes and it can be observed that UM2N better captures the dynamics i.e., adapts to the PDE solution of stable state compared to the High
Table A1: Measure hetween UM2N
and M2N model output mesh the flow flow-past www-past-a-cylinder Fig.
UM2N ER 1 (%)
M2N
Property
ER (%)
Vorticity
27.43 %
10.22 %
Res solution. The quantitative results shown in the Table AT also indicate that UM2N perform better than M2N regarding to error reduction.
D.2 Performance benchmark.
To evaluate the runtime performance of UM2N, we constructed a benchmark case based on the Swirl scenario, consisting of 11,833 vertices and 23,668 triangular elements. We extended the one-ring setup to a nine-ring configuration. In this benchmark, we compare the UM2N output mesh and the reference mesh generated by the MA method, well as the solutions obtained on these meshes, against those computed on high-resolution and original meshes as shown in Fig. A3
We tested the inference time of UM2N two GPUs (Nvidia GeForce RTX 3090 and Nvidia A100) and compared its performance to the MA method. For fair comparison, we also included CPU benchmarks, the current state-of-the-an MA implementation is limited to CPU execution. The CPU used for testing was an 11th Gen Intel(R) Core(TM) 19-1 1900K @ 3.50GHz. The performance results presented in Table
The results indicate that UM2N achieved more than 2000x speedup on the Nvidia RTX 3090 and over 3000x on the Nvidia A 100, compared to the MA method. Even the CPU, UM2N demonstrated an approximate 100x speedup. In summary, UM2N provides comparable solution accuracy while delivering significant performance improvement.
17

---

Adtette Moch
Meen (USO))
High Resolution
Adepped (MA)
Adapted (UNZNN)
UMZN
Figure A3: Visualization of UM2N output mesh and reference mesh from MA. Comparison between solutions obtained high resolution, original, UM2N and MA meshes.
D.3 Boundary layer case.
X-0.1
x-0.2
Navier-Stoliec
0.08
0.06
0.04
0.04
0.02
0.00
0.000
0.025
0.050
0.075
3.000
0.000
3.025
0.050
0.075
0.10
High Res (260610 vartices)
Original (4023 vertices)
UMMNN (4033 varticas]
Figure A4: Boundary Layer. The two plots at left visualize the my from boundary to 0.1. It can be observed that UM2N improves the accuracy of the velocity obtained uniform mesh compared to High Res solution.
In our flow past the cylinder case, at the top and bottom we imposed non-slip boundary conditions which already promotes boundary layer similar the suggested flow over a flat plate. For clearer illustration, we have now additionally conducted a flow past two parallel plates experiment. We observe that the velocity parallel to the boundary (u,,) obtained from UM2N moved mesh better aligns with the high resolution results compared to the results uniform mesh as shown in the Fig. A4
D.4 Poor quality initial mesh.
We include an additional experiment starting from a poor quality initial mesh as shown in Fig. A5 The input mesh has highly anisotropic elements and several vertices with valence This can certainly be considered a "low quality" mesh. Consider the monitor function to be the - of the recovered gradient of the solution of anisotropic Helmholtz problem. With this monitor function, we tried applying conventional MA solvers and found that both the quasi-Newton and relaxation approaches failed converge and/or resulted in tangled meshes, even though the elements are already aligned with the anisotropy of the Helmholtz solution. The UM2N approach, however, was able to successfully apply mesh movement without tangling. We would also like to clarify that our work targeting mesh movement that dynamically adapts to PDE solution, which is not replacement for mesh generation in general, so we would always expect reasonable input mesh when applying our model.
18

---

Original Mesh
UM2N Output
Helmholtz solution
Monitors
Figure A5: A very poor initial mesh case. UM2N perform mesh movement anisotropic Helmholtz problem.
Swirl ER (%)
Method
UM2N-w/o-eccode
UM2N-wooGT
12.35
UM2N (Ours)
35.93
MA (reference)
37.43
Original
w/ Decoder
wio GT
UM2N
MA
Error reduction of UM2N variants on
Swirl case.
(b) Moved from UM2N variants and refereace mesh from MA Swirl case.
Figure A6: Ablation of network components. Full UM2N gives the best error reduction.
D.5 Further Ablation studies.
We perform an additional ablation study of our network architecture on the Swirl case. We construct two models, which replace the GAT Decoder with one layer GAT denoted as remove the graph transformer denoted as UM2N-w/o-GT. The results are shown in the Table|A6a) can be observed that both variants give worse results compared to the full UM2N. A visualization also shown in A6b It can be observed that without the Decoder, the model distorts the shape of the ring, i.e., missing relative information between vertices; without the graph transformer, the model fails to capture the details of the ring shape.
-ecoder and
E
Additional experimental results.
Flow past multiple cylinders. The Aig. shows a challenge scenario with 5 cylinders in the domain. The proposed UM2N tangling.
mesh capturing the complex dynamics without mesh
Flow past V-formation cylinders. The Fig. A8 shows bio-inspired V-formation setting. The proposed UM2N can move mesh capturing the wake flows.
Flow past cylinders and squares. The Fig, A9 shows the proposed UM2N handle another challenging scenario with a mixture of cylinders and squares of different sizes.
Lift coefficient (C₂) of flow past cylinder. In Fig. A10 visualized the lift coefficient timestep across different methods.
19

---

t= 1s
t= 3s
t= 5s
7s
Figure A7: Multiple Cylinders
Acspeted Meah (302M)
Figure A8: V-formation
F Mesh movement enhanced simulation.
The Mesh Movement Enhanced Simulation Algorithm, as outlined in Alg I integrates our mesh adaption strategies into the simulation process to enhance the quality and accuracy of computational solutions. The proposed MMES procedure can be adapted into almost any time-dependent numerical simulation processes non-intrusively.
For given time-dependent PDE P and an initial mesh Eimi the algorithm evaluates the monitor function based on the solution field of the PDE at each time step The monitor value is then used by our proposed model to guide mesh movement. For robustness, a mesh integrity check is performed before time stepping (a tangled mesh will interrupt the procedure).
20

---

Nauide Aasiie Viriee
Adapsted Nach LPGN Curout
Figure A9: Cylinder and Square
Lift Confficient (C;)
High
Original 10933 vertices) UR2N (4933 veriies
0.5
0.0
Figure A10: Results of CT. coefficients. The blue, orange and green lines show the drag Cr coefficients obtained on a high resolution fixed mesh, UM2N adapted mesh and the original coarse mesh for the first 6 seconds (i.c., 6000 steps). It can be observed that the UM2N output mesh improves the accuracy of Cz in magnitude and periodicity compared to the original mesh.
G Limitations and broader impacts.
Limitations. 1) The GAT to extent constrains the mesh movement to lie in restricted range, which may be less effective in scenarios requiring substantial large deformation within a single iteration of the mesh mover. 2) There is no theoretical guarantee that mesh tangling will be completely prevented. 3) The model's best performance relies on a manually chosen monitor function, though gradient based monitor function can yield reasonable results. 3) Given low-quality or extremely coarse original mesh, the efficacy of the proposed method may be constrained by the inherent
Algorithm Mesh Movement Enhanced Simulation
1:
procedure SIMULATION(P, Eimit ?)
2:
E, f Einit
3:
while < do
4:
PDESoover(P,, E,)
5:
me +- MonitorFunc(u, E.)
6:
E,-1 UM2N(E,, mg)
7:
MeshIsValid <- ChecKMessNNrTTaglllCC...
8:
Meshls then
9:
E, + Et+1
else
11:
Break
end if TimeStepUpdate0
13:
end while
15:
end procedure
21

---

limitation on the degree of freedom count, as it redistributes rather than increases the number of mesh vertices. A promising future direction could be exploring the integration of mesh movement with h-adaptation, potentially developing hybrid hr-adaptation approach that combines the advantages of both techniques.
Broader Impacts. This research contributes to the field of AI for physical simulation by introducing an advanced learning-based mesh movement method that enhances the accuracy and efficiency of numerical simulations. The primary application of this method lies in areas requiring simulations of physical phenomena, such as geophysics, renewable energy, climate modeling, and aerodynamics. This method can aid in better weather or hazards forecasting, more efficient energy harvesting, fast prototyping of aircraft design etc.
22

---

NeurIPS 2024 Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Please see abstract and introduction
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Please see section G
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
23

---

Justification:
Please see the section 3
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification:
We would like to release the code for reproducibility. Please refer to Appendix
A
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
24

---

Answer: [Yes]
Justification:
We would like to release the code for reproducibilityyPles. refer to Appendix
A
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cmm ec/
for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https for more details.
//nips. //nips.cc/ppl wwcppbblc/guuiicuccccccccoecoopp /etgguddsse/CeSbbbiiiinn
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please section 5.1 and appendix C
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification:
We perform the experiments multiple times and compute the average.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
25

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please see section 5.1
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
The reserach in the paper conform with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Please section G
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
26

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Please the appendix
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
27

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We would provide README alongside the assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
28


Reviews:
[{"review_id": "RGhxM0MVaI", "reviewer": "Reviewer_xMNp", "summary": "In the present work, authors tackle the challenging task of accelerating PDE-solving process with deep learning, focusing on mesh movement problem. They suggest a new architecture that combines graph transformer encoder and graph attention decoder to significantly accelerate the PDE-solving. Authors propose a way to decouple the underlying PDE solving from the mesh movement process itself. It makes the approach universal with regard to solvers as well as to boundary geometries and mesh structures. They demonstrate the superiority of their method over conventional solver, and the ability on various benchmark tasks.", "questions": "I would appreciate if authors discuss some of the issues I raised above in the Weaknesses. \n\nMainly: non-tangled results of other methods on the flow-past-cylinder; tangled results of the UM2N model itself", "limitations": "-", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 4, "contribution": 4, "strengths": "The work is well written and concise. The work focuses specifically on the r-adaptation, i.e. mesh movement problem. Authors carefully describe what the complete problem is and which domain their method attributes to. This might look excessive but overall leaves good impression and facilitates understanding. \n\nThe approach itself of generating the dataset of random fields and train the model to learn an auxiliary PDE sounds compelling and indeed universal. It allows them to effectively decouple the monitor values from the PDE solved, which is one of the main strengths of the work. \n\nThe use of data, the architecture, the losses used and other deep learning-related details are well explained; the losses and the input varioations are justified in the convincing ablation study.\n\nAuthors claim that they would like to provide the code for reproducibility, which is always for the better.", "weaknesses": "Authors do not demonstrate a single experiment, where their model provides mesh tangling. Authors explain such effect by the use of the volume loss (line 326) calling it \"tangling-aware\". In general, it might be good, since one can be sure that any use of UM2N is reliable. \nYet, without proper evaluation (or at least, clarification for such perfection), It cannot be regarded as a virtue. It implies either that the experiment setups are chosen to be weak or the model has the inductive bias that might limit its universality. In other words, there might exist some setups that the model won't be able to estimate correctly, because of oversmoothness. As for example, one can imagine the domain with a tight bottleneck, the flow might not propagate to the other side at all. \n\nAnother side of the same issue can be seen in the flow-past-cylinder experiment. Both MA and M2N provide some tangling and hence considered as \"Failed\", so entire discussion in this experiment is in favor of the proposed UM2N. Authors call the setup \"classic and challenging\", yet there is not a single baseline that provides at least some solution. Either another baseline that actually gives some number should be provided or the setup should be simplified, e.g. choosing the object that provides less turbulence, such as the airfoil.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We like to thank the reviewer for their kind words on the presentation and level of contribution of our paper. Below, we present our detailed responses to the comments, indicating **[W]** for weaknesses and **[Q]** for questions.\n\nWe appreciate the feedback regarding the amount of explanation we were able to provide regarding limitations of our approach and validation of the flow past a cylinder case. As the reviewer correctly identifies our paper focuses specifically on r-adaptation which has benefits over other mesh adaptation methods - maintaining mesh topology, and, in the Monge-Ampere (MA) approach, minimising the amount of change - but also drawbacks: solving the MA equations is very costly, and it puts limitations on the physical geometry, demanding smooth boundaries and convex domains. In our paper, we demonstrate a significant reduction in the cost through the UM2N acceleration. We also demonstrate a relaxation to some extent of the geometric limitations, but this is indeed a bit harder to outline and quantify. \n\n**Response to W1/Q1:‘tangled results of the UM2N model’:** There are in fact still cases where UM2N might fail, we perform additional experiments as shown in Figure R1. Starting from a rectangle, we gradually distort the geometry to a more non-convex shape i.e., the case is more and more challenging from top to bottom. The UM2N finally fails in the case surrounding sudden jumps in the boundary accompanied with a large variation in the required resolution as shown in the bottom row of Figure R1: a sudden constriction of the flow leads to tangling with UM2N in the two left corners of the channel. In a more smoothed version UM2N does produce a valid mesh, where the original MA method still fails (second to bottom rows in Figure R1). We agree that this warrants a more extensive discussion in our paper in the revision.\n\n**Response to W2/Q2: ‘non-tangled results of other methods on the flow-past-cylinder’:**\nWe additionally perform a non-tangled flow-past-cylinder shown in Figure.R6 as mentioned by the reviewer xMNp. We simplified the setting: lower the Rrenynolds number by reducing the inflow velocity and set the top and bottom slip boundary, which finally makes it a laminar flow. As shown in Figure.R6, both the UM2N and M2N give non-tangled meshes and it can be observed that UM2N better captures the dynamics i.e., adapts to the PDE solution of stable state compared to the High Res solution. The quantitative results shown in the Tab.R3 also indicate that UM2N perform better than M2N regarding to error reduction.\n\n\nThe aim of the flow past a cylinder case was to demonstrate that mesh movement can now readily be applied, which is not feasible with a classical approach like MA for several reasons: the non-convexity of the domain means the equations are fundamentally ill-posed, and, even if that could be overcome, e.g. by applying mesh movement only in the right half of the domain, the prohibitive costs to apply this every timestep. This case is based on the so called DFG-benchmark 2D-2 [1], which has been modelled with a variety of other frameworks, including those that incorporate other mesh adaptation approaches [see e.g. 2-5] The results in our paper were validated by comparison with a high resolution uniform-mesh model, which corresponds well with the reference value in [2,3]. Note that convergence to the reference depends on various discretisation choices, finite-element pair, type of boundary condition, etc., which are not the focus of our paper. We therefore decided to only compare with a high resolution case, using the same discretisation details, to demonstrate the improvement in accuracy through mesh movement alone. We would like to clarify this however in our paper, and make better reference to and comparison with published results for this case.\n\n\n[1] https://wwwold.mathematik.tu-dortmund.de/~featflow/en/benchmarks/cfdbenchmarking/flow/dfg_benchmark2_re100.html\n\n[2] D. Capatina et al. ‘22 https://doi.org/10.1016/j.cma.2019.112775\n\n[3] V. John ‘04 https://onlinelibrary.wiley.com/doi/abs/10.1002/fld.679\n\n[4] T. Coupez et al. ‘13 https://doi.org/10.1016/j.jcp.2012.12.010\n\n[5] Hachem et al ‘13 https://doi.org/10.1002/nme.4481", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rp0qrodRD6", "reviewer": "Reviewer_yaVX", "summary": "The paper introduces the Universal Mesh Movement Network (UM2N), a deep-learning model that enhances solving Partial Differential Equations (PDEs) through adaptive mesh movement. UM2N, with a Graph Transformer encoder and a Graph Attention Network (GAT) based decoder, is trained on a PDE-independent dataset, allowing zero-shot application to various PDEs and geometries without retraining. The model uses element volume loss to improve robustness and reduce mesh tangling.", "questions": "1/ How does UM2N handle extremely poor-quality initial meshes and highly irregular geometries? \n\nSuggestion: \n\nConduct additional experiments with initial meshes characterized by highly skewed aspect ratios, large variations in element sizes, and complex boundary geometries. Include quantitative metrics on mesh quality before and after adaptation to highlight the robustness of UM2N.\n\n2/  How does UM2N ensure mesh continuity and high-quality element shapes when dealing with both local and global mesh deformations?\n\n3/ How does UM2N's learned mesh configuration compare to known optimal solutions or high-resolution reference meshes? Are there benchmarks or validation cases included to demonstrate the accuracy and efficiency of the mesh adaptation?\n\nSuggestion: \n\nBenchmark the learned mesh configurations against known optimal solutions or high-resolution reference meshes. Include comparisons using quantitative metrics such as error norms and gradient capture to highlight the accuracy and efficiency of UM2N in approaching the optimal mesh configuration.\n\n4/ How does UM2N perform in resolving boundary layers and turbulent flows? Could you provide experimental results or benchmarks that validate its effectiveness in these critical fluid dynamics applications?\n\nSuggestion:\n\nConduct targeted experiments on canonical boundary layer cases (e.g., flow over a flat plate) and turbulent flow benchmarks (e.g., turbulent channel flow). Use quantitative metrics like wall shear stress and turbulence intensity to evaluate performance and compare with high-fidelity simulations or experimental data.\n\n5/  Can you elaborate on the rationale behind selecting Graph Attention Networks (GAT) over other Graph Neural Network (GNN) methods for mesh movement? \nAdditionally, have you considered or benchmarked against hierarchical GNNs, Octree-based methods, or point cloud-based methods, such as those utilizing multi-scale architectures like U-Net? \nExploring these alternatives could provide insights into potential improvements in handling mesh adaptation across different scales and complex geometries.\n\nif the authors address the weaknesses and questions outlined above, I would be happy to increase my score. Doing so will significantly enhance the robustness and applicability of the proposed work.", "limitations": "Yes.", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1/ Originality: \n\nThe paper presents a novel approach that advances the field of adaptive mesh movement for solving Partial Differential Equations (PDEs). By combining a Graph Transformer encoder with a Graph Attention Network (GAT) decoder, the authors develop a unique method that can be applied zero-shot to various PDE types and boundary geometries without retraining. This approach overcomes the limitations of traditional and existing learning-based methods, which often require costly retraining and struggle with complex geometries.\n\n2/ Quality: \n\nThe quality of the paper is good, demonstrated through comprehensive evaluations on different PDE types, including advection, Navier-Stokes, and a real-world tsunami simulation. The experiments clearly show that UM2N outperforms existing methods in terms of accuracy, efficiency, and robustness. The use of a PDE-independent dataset for training and the adoption of element volume loss for reducing mesh tangling further highlight the thoroughness and rigor of the approach.\n\n3/ Clarity: \n\nThe methodology is clearly explained, and the results are presented in a way that effectively supports the claims made. \n\n4/ Significance: \n\nThe significance of this work is substantial, as it addresses a fundamental challenge in scientific and engineering simulations: \nsolving PDEs efficiently and accurately. By enabling zero-shot application to various PDEs and geometries, UM2N offers a versatile tool that can be integrated into a wide range of numerical solvers, potentially benefiting numerous applications in geophysics, renewable energy, climate modeling, and aerodynamics. The improvements in mesh movement accuracy and reduction in computational costs have broad implications for advancing simulation technologies and their applications.\n\n5/ Learning-based mesh generation and adaptation:\n\nOne of the primary advancements is the use of a PDE-independent dataset for training. This allows UM2N to generalize across various PDE types and boundary geometries without the need for retraining. This is a significant improvement over existing methods that typically require retraining for new scenarios, making UM2N more versatile and practical for real-world applications. Another key strength is the incorporation of element volume loss in the training process. Unlike coordinate loss used in previous methods, element volume loss not only supervises the mesh movement but also penalizes negative volumes. This effectively reduces mesh tangling and enhances robustness.\n\nThe architectural choice of combining a Graph Transformer encoder with a Graph Attention Network (GAT) decoder is particularly well-suited for handling the irregular structures of unstructured meshes. This graph-based architecture allows UM2N to capture both local and global information more effectively, leading to more accurate and efficient mesh movements. The attention mechanisms within these networks help in managing the complex dependencies and interactions within the mesh elements.\n\nMoreover, UM2N's ability to perform zero-shot generalization is another significant strength. The model can be applied to new, unseen problems without the need for additional training. This capability is particularly beneficial in practical applications where retraining models is computationally expensive and time-consuming. \n\nBy using a learning-based approach for mesh movement, UM2N significantly improves the accuracy of numerical solutions while also reducing computational costs. This balance between accuracy and efficiency is crucial for advancing simulation technologies in various scientific and engineering domains. The method's ability to enhance accuracy without incurring additional computational overhead makes it an important tool for high-fidelity simulations.\n\n\n6/ Monge-Ampère PDE-solver point of view:\n\nFrom the perspective of Monge-Ampère, UM2N enhances practical applicability by addressing computational and robustness challenges through a novel learning-based approach. Traditionally, Monge-Ampère-based methods offer attractive theoretical properties such as equidistribution and optimal transport but are computationally expensive and struggle with complex geometries. UM2N mitigates these limitations by decoupling the underlying PDE solve from the mesh movement process, focusing on learning the auxiliary Monge-Ampère equation.\n\nThe integration of a Graph Transformer and GAT-based architecture allows UM2N to handle the complexities of mesh movement efficiently. The Graph Transformer captures both local and global information from the mesh, which is essential for accurately solving the Monge-Ampère equation. It processes the mesh as a graph, taking into account the positional relationships and interactions between mesh elements. The GAT-based decoder then uses this rich feature set to move the mesh nodes in a way that aligns with the desired mesh density distribution.\n\nUM2N's use of the element volume loss function is a significant technical advancement. This loss function is inspired by the equidistribution principle of the Monge-Ampère method, which aims to distribute the monitor function values uniformly across the mesh. By focusing on the volumes of the mesh elements rather than their coordinates, the element volume loss function ensures that the mesh adapts according to the desired density distribution. This approach also penalizes negative volumes, which helps prevent mesh tangling a common issue in traditional Monge-Ampère-based methods.\n\nAnother important detail is UM2N's training on a PDE-independent dataset, which allows it to generalize well to different PDEs and boundary conditions without retraining. This is a significant improvement over traditional Monge-Ampère methods, which often require problem-specific adjustments. UM2N's ability to perform zero-shot generalization makes it highly versatile and practical for a wide range of applications.\n\nUM2N's robustness is further demonstrated through its successful application to scenarios with complex boundary geometries, such as the Tohoku tsunami simulation. Traditional Monge-Ampère methods often fail in such scenarios due to the challenges in solving the non-linear Monge-Ampère equation under these conditions. UM2N, however, effectively manages these complexities, maintaining high-quality mesh adaptation even in challenging geometries.", "weaknesses": "1/ Comparison with State-of-the-art methods:\n\nWhile UM2N is compared with a few existing methods, the comparison could be broadened to include more recent state-of-the-art approaches. This would provide a clearer context for UM2N's performance and highlight its relative advantages and limitations more comprehensively. Expanding the experimental section to include comparisons with more recent and relevant learning-based mesh movement methods would be valuable. If new experiments are not feasible during the rebuttal, a detailed qualitative comparison discussing potential strengths and weaknesses based on the literature can be provided.\n\n2/ Ablation Studies:\n\nThe ablation studies are somewhat limited. A more comprehensive analysis of different components of UM2N, such as the impact of various hyperparameters, the contribution of each network component (Graph Transformer vs. GAT), and the sensitivity to different types of monitor functions, would provide deeper insights into the model's workings. The authors could add more ablation studies focusing on hyperparameter tuning, the role of different network components, and the sensitivity analysis to different monitor functions.\n\n\n3/ Handling extreme geometries and mesh qualities:\n\nThe current paper does not extensively explore how UM2N handles extremely poor-quality initial meshes or highly irregular and extreme geometries. These scenarios are crucial for practical applications, as real-world problems often involve complex and challenging geometrical domains. Poor-quality initial meshes, characterized by elements with highly skewed aspect ratios, large variations in element sizes, or even degenerate elements, can significantly complicate the mesh movement process. For instance, elements with aspect ratios far from unity can lead to numerical instabilities and inaccurate solutions due to poor approximation properties. Large variations in element sizes can cause localized errors to propagate or lead to uneven error distribution across the mesh, while degenerate elements with nearly zero area or volume can invert or fold during movement, leading to invalid meshes.\n\nSimilarly, highly irregular geometries, such as those with sharp corners, narrow channels, intricate boundary shapes, or complex topological features like holes and multiple connected components, can test the robustness and adaptability of mesh movement algorithms. Sharp corners and narrow channels require fine mesh resolution to capture the geometric details accurately, while avoiding mesh element inversion or tangling. Intricate boundary shapes and complex topological features increase the difficulty of maintaining mesh quality and conformity to the domain boundaries during adaptation.\n\nA. Aspect Ratio and skewness:\n\nMaintaining aspect ratios close to one is crucial to minimize numerical errors. High aspect ratios can lead to poor interpolation properties, making it challenging to preserve or improve aspect ratios during significant mesh adjustments.\n\nHigh skewness causes poor approximation and solver instability. Controlling skewness during mesh movement to prevent highly distorted elements is critical.\n\nSuggestion: implement a loss term that penalizes high aspect ratios and skewness during training to encourage well-shaped elements.\n\nB. Element size variation:\n\nUniformity and adaptivity: \nBalancing element sizes to ensure smooth transitions between refined and coarse regions is complex. Small elements can lead to computational overhead, while large elements might miss critical solution features.\n\nDynamic adaptation:\nEfficiently refining in regions of high gradients and coarsening elsewhere requires dynamic strategies, balancing computational cost and accuracy.\n\nSuggestion: \nUse adaptive mesh refinement algorithms that dynamically adjust element sizes based on local error estimates and monitor function gradients.\n\nC. Mesh tangling:\n\nInversion and Overlap:\n\nPreventing elements from inverting or overlapping during movement is essential. This requires strategies like regularization terms in the loss function to penalize negative volumes and ensure coherent adaptation.\n\nLocal and Global Coherence:\nEnsuring local coherence in element displacement to prevent tangling, especially in regions with high gradient monitor functions, is challenging.\n\nSuggestion:\nIntroduce a regularization term in the loss function to penalize negative volumes and use local displacement constraints to maintain element integrity.\n\nD. Boundary conformance:\n\nComplex boundaries: \nAccurately adapting the mesh to complex geometries with sharp features and varying topologies without introducing distortions requires careful refinement near boundaries.\n\nBoundary preservation: \nTechniques such as boundary snapping and boundary layer refinement are necessary to maintain accurate geometric fidelity.\n    \nSuggestion:\nFor instance applying boundary snapping techniques and boundary layer refinement to ensure elements conform accurately to the physical boundaries.\n\n\nE .Adaptive refinement and coarsening:\n\nDynamic adaptation:\n\nAdapting the mesh based on evolving solution features, such as moving fronts, involves adding or removing elements dynamically.\n\nSmooth transitions: \n\nEnsuring smooth transitions between different resolution regions without creating poorly shaped elements is crucial. Hierarchical refinement and mesh smoothing techniques are often employed.\n\n\n\n4/ Theoretical guarantees and limitations:\n\nWhile the paper mentions that there are no theoretical guarantees that mesh tangling will be completely prevented, it would be beneficial to delve deeper into this limitation. Understanding the theoretical bounds of UM2N's performance and the conditions under which it might fail is important for setting realistic expectations.\n\n\n5/ Real-world application scenarios:\n\nAlthough the tsunami simulation is an excellent example, including more diverse real-world application scenarios would further demonstrate UM2N's versatility and robustness. For instance, applications in aerodynamics, biomechanics, or climate modeling could be explored.\n\nSuggestion: \n\nAdd more case studies or at least detailed discussions of how UM2N could be applied to other real-world problems, emphasizing its practical benefits and potential challenges in these domains.\n\n\n6/ Computational efficiency and scalability:\n\nThe paper discusses the reduction in computational costs compared to traditional methods but does not provide detailed benchmarks or discussions on the scalability of UM2N for very large meshes or real-time applications.\n\nSuggestion:\n\nInclude more detailed benchmarks of computational efficiency and scalability, comparing UM2N's performance on large-scale problems with other methods. \nDiscuss strategies for optimizing performance and potential bottlenecks in the current implementation.\n\n\n7/ Standpoint from Multi-Scale phenomena:\n\nThe paper does not address the specific challenges posed by multi-scale phenomena, where different parts of the domain may require vastly different resolutions to capture fine-scale features accurately while maintaining computational efficiency. Multi-scale phenomena often involve a wide range of spatial and temporal scales, making it difficult to balance resolution and computational cost effectively.\n\nSuggestion:\n\nIntroduce experiments that specifically target multi-scale phenomena, demonstrating UM2N's ability to adapt the mesh dynamically to capture fine-scale features in high-gradient regions while coarsening in less critical areas. This could involve benchmarks on problems known for their multi-scale nature, such as turbulent flows or geophysical simulations. Additionally, discuss potential enhancements to the model, such as multi-grid techniques or hybrid approaches that combine UM2N with other multi-scale modeling strategies, to better handle these complex scenarios.\n\n8/ Analysis from mesh continuity, local and global deformation viewpoints:\n\nThe paper does not thoroughly address how UM2N manages local and global deformations of the mesh, which are essential for accurately capturing complex physical phenomena. Effective mesh adaptation requires the ability to handle fine-scale local deformations to capture detailed features and large-scale global deformations to adapt to overall changes in the domain. Additionally, ensuring mesh continuity during these deformations is crucial to maintain the fidelity and stability of numerical simulations. Inadequate handling of these deformations can lead to poor resolution of critical areas, excessive computational costs, and potential discontinuities that degrade the accuracy of the simulation.\n\nA/ Coupling local and global deformations: \n\nBalancing local and global deformations is crucial for maintaining overall mesh quality and continuity. This involves ensuring that local refinements do not introduce excessive computational overhead and that global adaptations do not degrade the resolution of critical regions. Achieving this balance is technically challenging and requires sophisticated mesh adaptation algorithms that can handle both fine-scale and large-scale changes seamlessly.\n\nB/ Mesh quality and continuity maintenance:\n\nDuring both local and global deformations, maintaining high-quality elements (in terms of aspect ratio, skewness, and smoothness) is essential to avoid numerical instability and ensure accurate simulations. Poorly shaped elements can significantly degrade the performance of numerical solvers. Additionally, ensuring continuity in the mesh, where element shapes and sizes transition smoothly, is vital for maintaining numerical stability and accuracy.\n\n\n9/ Analyze the optima of the learned meshes:\n\nThe paper does not address the issue of ensuring that the learned mesh configuration is globally optimal, nor does it consider the potential existence of multiple local optima in the optimization landscape. This oversight can result in suboptimal mesh configurations, which may not fully capture the desired features of the simulation domain or may introduce unnecessary computational overhead. In complex adaptive meshing scenarios, the risk of converging to local optima rather than the global optimum can significantly impact the accuracy and efficiency of the numerical solutions.\n\nThe optimization process for adaptive mesh movement often involves a highly non-convex loss landscape with many local minima. A single global optimum represents the best possible configuration of the mesh in terms of accuracy and computational efficiency. However, due to the complex nature of the problem, the optimization algorithm may converge to multiple local optima, each providing a suboptimal solution that fails to maximize the potential benefits of the adaptive meshing process.\n\nThe presence of multiple local optima can lead to inconsistent mesh configurations across different runs or simulations. This variability can make it difficult to ensure that the mesh is optimally adapted to the specific features of the simulation domain. Inconsistent convergence can also lead to variability in simulation results, reducing the reliability and robustness of the numerical methods.\n\nSuggestion:\n\nBenchmark the learned mesh against known optimal solutions or high-resolution reference meshes. This benchmarking can help assess how close the learned meshes are to the ideal configuration and identify specific areas where the model falls short.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer yaVX providing a detailed summary of our strengths and valuable suggestions. Below, we present our detailed responses to the comments, indicating [W] for weaknesses and [Q] for questions.\n\n**Response to W1:**\nAs the reviewer correctly identifies we do compare our method with existing methods, focussing on the M2N method. The reason for the limited number of comparisons is because there really has not been a lot of research so far on learning-based mesh-movement methods, most of the literature has focused on acceleration of other mesh adaptation strategies. We do cite the very recent flow2mesh paper by Jian Ju et al. (ref. [32] in our paper) who confirm our view: “However, these works all focused on mesh refinement rather than the mesh movement\"\n \n**Response to W2:** We perform an additional ablation study of our network architecture on the Swirl case, analysing the contribution of each network component. We construct two models, which replace the GAT Decoder with one layer GAT denoted as UM2N-w/o-Decoder and remove the graph transformer denoted as UM2N-w/o-GT. The results are shown in Table R1. Both variants give worse results compared to the full UM2N. A visualization is also shown in Figure R2. It can be observed that without Decoder, the model distorts the shape of the ring, i.e., missing relative information between vertices; without the GT, the model fails to capture the details of the ring shape.\n\n**Response to W3/W4/Q1:**\nThanks for raising this question. We include an additional experiment dealing with poor quality initial meshes as shown in Figure R4 and an additional experiment for irregular geometries. Also note the real-world Tsunami case shown in Figure 5 of our paper is also a good example for highly irregular geometries. And more cases with complex geometries are shown in our supplementary video.\nFigure R4 shows a low-quality input mesh with highly anisotropic elements. Using a monitor function based on the l2-norm of the gradient of the solution of an anisotropic Helmholtz problem, we tried applying conventional MA solvers and found that both the quasi-Newton and relaxation approaches failed to converge and/or resulted in tangled meshes, even though the elements are already aligned with the anisotropy of the Helmholtz solution. The UM2N approach, however, was able to successfully apply mesh movement without tangling.\nWe would also like to clarify that our work targets mesh movement that dynamically adapts to a PDE solution, which is not a replacement for mesh generation in general, so we would always expect a reasonable input mesh when applying our model.\nIn Figure R1, we show a stress test for both MA and UM2N. It can be observed that MA soon fails in the presence of non-convexity in the geometry while UM2N only gets tangled in the most challenging right angle case. It indicates that our method allows a relaxation of the geometric limitations compared to MA, although we admit that there  is no theoretical guarantee for no mesh tangling as we mentioned in the limitation. \n\n**Response to W5/W6:** \nWe appreciate that the reviewer acknowledges that our Tsunami case is an excellent real-world scenario application. We also kindly remind that there are also more applications shown in our supplementary video. We do agree more examples for different areas would better demonstrate the versatility of UM2N. Due to the time limit in the rebuttal period, we would like to leave this to our revisions.\nFor some limited scalability, we further apply our trained model on a flow past cylinder case with ~11k vertices and ~22k triangles as shown in Figure R5. It is shown that our UM2N works well on this case with more degrees of freedom compared to the one in the paper.\nFor an even larger case, we perform stress tests using a time-independent Helmholtz case on our RTX 3090 24 GB GPU and observe that the maximum scale of the problem that can be run on this GPU has ~50k number of elements. The inference time is ~760 ms, the MA method on the same problem requires ~37800 ms with a residual threshold 1e-4. Considering this limitation, applying the memory efficient transformer (linear-attention transformer for further improvement of the inference efficiency) is targeted as future work.\n\n**Response to Q2/W8:** There are two types of mesh continuity to consider: continuity in space (i.e. mesh smoothness) and continuity in time (avoiding large changes between timesteps). The latter is promoted by training with samples based on  Monge Ampere solutions. Based on optimal transport theory, these represent the unique minimal change transformation that satisfies the equidistribution principle provided certain smoothness and convexity criteria are satisfied. Given that this is already the unique optimal mesh that with these properties, quality of elements in terms of their aspect ratio is not explicitly enforced, but the minimal change property does mean a high quality input mesh is transformed to a close-in-quality output mesh as long as the required transformation in terms of the monitor function is not too demanding (e.g. non-smooth). It should be noted it is not the case that equal-aspect triangles always provide the highest quality mesh. This in fact depends on the anisotropy of the PDE solution, and in cases where more control over the aspect ratio is required, other mesh movement methods allow for a tensorial input metric, but this necessarily means the minimal change (continuity) property is no longer satisfied in general. There is thus always a trade-off between different desired properties. Likewise, when asking for a considerable global redistribution of degrees of freedom this may come at the cost of the local mesh quality. Any mesh movement method will therefore be a compromise with UM2N focusing on maintaining minimal movement and maintaining desired output cells areas.\n\nPlease refer to **Supplement Response to yaVX** in general response for more responses **Q3/W9, Q4, Q5**.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jBsHIQHTPN", "reviewer": "Reviewer_jEN5", "summary": "This paper introduces a learnable model for mesh movement - which is a method for improving efficiency of a PDE solver by moving mesh nodes, while keeping the topology fixed. The proposed model itself is a two-stage neural architecture comprising a transformer encoder - taking a representation of the input mesh - and a graph attention network as a decoder outputting an updated mesh. The resulting model is independent of specific PDE type and thus can be applied to novel problem types in zero-shot manner. Quantitative evaluation is conducted on multiple synthetic and one realistic (tsunami prediction) tasks.", "questions": "- In figure 6, you demonstrate that um2n is better at handling complex boundaries. From what I can see there are no guarantess for your purely data-driven method to produce non-tangled meshes? Do you have an intuition to why this is the case?\n- Would this model work for larger scale problems with >1M?", "limitations": "Limitations are discussed.", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Proposed method provides a way to improve PDE solvers agnostic to the underlying PDE type. A zero-shot model like this could be extremely useful in a lot of engineering fields. \n- Quantitative results indicate that the method is outperforming existing method for mesh movement and leads to improved surrogate modeling.", "weaknesses": "- Although overall the architecture makes sense, there is no ablation study conducted on the individual parts of the architecture.  \n- Similarly, there are no baselines to off-the-shelf graph convolutional or transformer methods (any recent method e.g. for semantic mesh segmentation would work here?). \n- The model is only concerned with vertex movement of the model, which means there is no guarantee that there won't be any degraded triangles, which can influence the quality of the dowstream solutions. This could be a fundamental flaw of purely data-driven methods since there is not simple way to enforce constraints at test time. \n- The scale of the dataset seems to be very small for the model to be directly applicable in realistic scenarios - both in terms of number of samples and number of vertices. \n- Model relies on a transformer encoder, which will not scale to meshes with large number of vertices.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer jEN5 for acknowledging the contributions, soundness, and presentation quality of our paper. And greatly appreciate Reviewer jEN5 for proposing these insightful questions. Below, we provide our point-to-point responses to the comments, denoted by **[W]** for weaknesses and **[Q]** for questions.\n\n\n**Response to W1**: We perform an additional ablation study of our network architecture on the Swirl case. We construct two models, which replace the GAT Decoder with one layer GAT denoted as UM2N-w/o-Decoder and remove the graph transformer denoted as UM2N-w/o-GT. The results are shown in the Table R1.It can be observed that both variants give worse results compared to the full UM2N. A visualization is also shown in Figure R2. It can be observed that without the Decoder, the model distorts the shape of the ring, i.e., missing relative information between vertices; without the graph transformer, the model fails to capture the details of the ring shape.\n\n\n**Response to W2**: To the best of our knowledge, there are not a lot of works addressing the end-to-end learning-based r-adaptation problem. Many existing works focus on end-to-end surrogate neural PDE solvers. The models proposed in these works try to predict the PDE solutions directly, therefore it is hard to make a fair comparison as our UM2N predicts the mesh i.e., PDE discretization. To compare with them qualitatively, these methods have advantages in efficiency. On the other hand, our approach has the advantage that PDE solutions obtained are physically plausible based on guarantees from classical numerical analysis regarding the PDE discretization, whereas such guarantees are absent or enforced in a weaker sense in directly learned PDE solutions.\n\nFor the network architecture, we apply the graph transformer as encoder for its powerful expressivity, and apply the graph attention network as decoder because it can naturally help alleviate mesh tangling. There are existing works such as efficient transformers from natural language processing or computer vision community, which provide promising future directions to improve the efficiency of our model. The semantic mesh segmentation mentioned by the Reviewer jEN5, although not directly applicable to our problem, would also be an interesting direction to explore.\n\n**Response to W3/Q1**: We agree that there are no guarantees for our method to produce non-tangled meshes as we have also noted in the limitations section of our paper. In Figure 6, we claim that UM2N is better at handling complex boundaries compared to the MA method. The MA method requires smooth boundaries and convex domains. Its behaviour can be observed from our additional experiments shown in Figure R1. It produces good results for the rectangle domain (shown in the first row) but outputs a highly tangled mesh given a non-convex domain (shown in the following rows).\n\nThe intuitions behind why our data-driven UM2N can handle complex boundaries can be divided into two aspects. The first is that we use Graph Attention Network (GAT) to build the mesh deformer. Therefore, the coordinate of each vertex is updated by the weighted sum of its neighbors, the weights (or coefficients) of which are determined by the GAT module. This guarantees each vertex to only move within the convex hull of its neighboring vertices, hence effectively alleviating mesh tangling issues. In addition, we utilize element volume loss penalizing negative element volumes (i.e., inverted elements), which helps reduce mesh tangling.\n\n\n**Response to W4/W5/Q2**: \nAs for the scale of the dataset, we consider the test case shown in the paper -  Cylinder (~5k vertices, ~10k triangles), Tsunami ( ~8k vertices, ~16k triangles) - to have moderate degrees of freedom. The tsunami case is already a real-world scenario application in the ocean modelling community. To show a case with more degree of freedom, we further apply our trained model on a flow past cylinder case with ~11k vertices and ~22k triangles as shown in Figure R5. It can be shown that our UM2N works well in this case.\n\nAs for the large case such as >1 million vertices/triangles mentioned by the reviewer, the model can be naturally applied to larger scale problems as there is no limitation for the input length of graph transformer and graph neural network but the GPU memory is the limit. We performed stress tests using a time-independent Helmholtz case on our RTX 3090 24 GB GPU and observe that the maximum scale of the problem that can be run on this GPU has ~50k number of elements. The inference time is ~760 ms, the MA method on the same problem requires ~37800 ms with a residual threshold 1e-4. Therefore, given our current limited computational resource, it is hard for us to construct a case with 1 > million vertices. Considering this limitation, applying the memory efficient transformer (linear-attention transformer for further improvement of the inference efficiency) is targeted as future work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4Ps4jYDT4S", "reviewer": "Reviewer_Jzyt", "summary": "The authors propose a method for general mesh movement. They design a network that takes the original mesh and the monitor values as input to predict the mapping for the adapted mesh. This makes it able to handle any mesh without the need of training PDE-type-specific models.", "questions": "1. The network takes the original mesh and the monitor values $m$ as input. $m$ is the values for the original mesh or the target mesh? If it is the value of the original mesh, then how does the network know the target? If it is the value of the target mesh, then the symbol of $m_\\xi$ is a bit misleading, since $\\xi$ denotes the original mesh.\n2. When using chamfer distance for the optimization of mesh matching, the result is easy to be stuck at the local minimum. Is it the case for the training of the proposed method?", "limitations": "In the section of Limitation, the authors mentioned that a low-quality original mesh can lead to poor prediction. A visualization of it will be nice.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed method can handle mesh with different geometries and PDE with different type using a single model. The model only needs to be trained once.\n2. Another advantage is its time cost is much lower than the traditional non-learned Monge-Ampère without the sacrifice of the movement accuracy.\n3. Compared with prior works, ie, MA and M2N, it is more robust and easier to converge.\n4. The paper is well written and easy to follow.", "weaknesses": "The idea of this paper is interesting to me, and the results, especially those shown in the supplementary video, are pretty nice. I believe the proposed method can have many potential application in mesh formation and the modeling of physical phenomena. I didn't see any obvious weaknesses from this paper. However, I have to admit that I'm not the expert in this field.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We greatly appreciate your insightful comments that precisely recognize the strengths of our work!  We are also very happy the Reviewer Jzyt checked our supplementary video and found it nice and useful. Below, we provide our point-to-point responses to the comments, denoted by **[W]** for weaknesses and **[Q]** for questions.\n\n**Response to Q1**:\nThe monitor values 𝑚 correspond to the original mesh. Given a time-dependent PDE, the monitor values $𝑚$ at the first time step are computed based on the initial condition on the original mesh. Then the model takes in the monitor values $𝑚$ as input (with other features) and outputs a moved mesh. The PDE is solved based on this moved mesh and the solution obtained will be used for computing monitor values for next time-step.\n\nThe target meshes only come in during the training process: the moved mesh is compared to the target mesh with element volume loss and Chamfer distance. The networks get to know the target during the training. In other words, the monitor values are part of input features and are defined in the nodes of the input  meshes and not those of the target meshes. This indeed explains the notation $m_ξ$ with the monitor values being associated with the input mesh. The training process can be explained to teach the model to learn to solve an optimal transport problem according to equidistribution theory given the monitor values and other related features.\n\n**Response to Q2**: \nThanks for raising this question. We acknowledge that Chamfer distance-based optimization for mesh matching or point cloud registration can potentially lead to convergence at local minima. However, in our case, we did not observe significant issues of being trapped in poor local minima during the training process, though we cannot guarantee that a global minimum was reached.\nWe believe there are two primary reasons for this. First, the initial alignment between the original uniform meshes and the target meshes in our training dataset is not significantly different. This likely increases the chances of finding well-matched vertices when searching for nearest points using Chamfer distance. Second, in addition to the Chamfer distance, we employed an element volume loss, which provides additional information beneficial for vertex matching.\n\n**Response to limitation**:\nThanks for the suggestions. We include an additional experiment about poor quality initial mesh as shown in Figure R4.\nThe input mesh has highly anisotropic elements and several vertices with valence 6. This can certainly be considered a “low quality” mesh. Consider the monitor function to be the l2-norm of the recovered gradient of the solution of an anisotropic Helmholtz problem. With this monitor function, we tried applying conventional MA solvers and found that both the quasi-Newton and relaxation approaches failed to converge and/or resulted in tangled meshes, even though the elements are already aligned with the anisotropy of the Helmholtz solution. The UM2N approach, however, was able to successfully apply mesh movement without tangling.\nWe would also like to clarify that our work targeting mesh movement that dynamically adapts to a PDE solution, which is not a replacement for mesh generation in general, so we would always expect a reasonable input mesh when applying our model.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We are glad to receive valuable and constructive comments from all the reviewers. We have made a substantial effort to clarify reviewers' doubts and enrich our experiments in the rebuttal phase. In our responses, Tab. Rxx or Fig. Rxx
refers to the new Rebuttal results in the attached PDF. Below is a summary of our responses:

**Reviewer xMNp:**
1. We perform additional experiments to demonstrate a failing case for UM2N and to explore the limits to its ability to avoid mesh tangling (Figure. R1)
2. We demonstrate how UM2N provides a relaxation to some extent of the geometric limitations compared to the Monge-Ampère (MA) based method. (Figure. R1)
3. We conduct additional experiments to show non-tangled results of other methods on the flow-past-cylinder (Figure. R6), and provide an explanation on why MA inherently fails on flow-past-cylinder.

**Reviewer yaVX:**
1. We perform an additional ablation study on network components (Table.R1/Figure.R2) to demonstrate the effectiveness of both Graph Transformer based encoder and Graph Attention Network based deformer.
2. We conduct an additional experiment dealing with poor quality initial meshes (Figure R4).
3. We provide a detailed discussion about mesh continuity in mesh movement.
4. We provide a clarification for optimality of mesh movement and explain our decision on the metric we use for evaluating mesh movement. We also provide measures between model output mesh and target mesh. (Tab. R2)
5. We perform an additional boundary layer experiment. (Figure.R3) 
6. We explain the rationale for selecting Graph Transformer and Graph Attention Network and discuss the other related networks as potential future work.

**Supplement Response to yaVX:**

**Response to Q3/W9:** 
Here we would like to discuss both the comparisons to optimal mesh and reference PDE solutions. It is generally hard to strictly prove the optimality of a mesh in cases with more than one dimension, and, as argued previously, there are various definitions of optimal that can be considered, and, in a sense, the meshes obtained from the MA method can be regarded as optimal meshes already. In Table R2, we provide the coordinate loss, element volume loss and Chamfer distance during training. Note that these metrics for measuring the difference between model output mesh and MA reference mesh are intermediates. The ultimate metric we care about is the PDE solution error reduction rate (ER), which is why we focus on ER in our paper.

We provided reference solutions for both Swirl and Flow past cylinder cases in our paper which are computed on high-resolution meshes (~10k for swirl, ~260k for flow-past-cylinder), indicated as ‘High Res’. As shown in Figures 2 and 3 of the paper, the solution obtained on a UM2N moved mesh has a smaller L2 error (plots in right part of Figure 2) and more accurate drag coefficient (bottom part of Figure 3) compared to that obtained on an original uniform mesh. These errors are computed against the reference, high-res PDE solution.

**Response to Q4:**  In our flow past the cylinder case, at the top and bottom we imposed non-slip boundary conditions which already promotes a boundary layer similar to the suggested flow over a flat plate.
For a clearer illustration, we have now additionally conducted a flow past two parallel plates experiment. We observe that the velocity parallel to the boundary ($u_y$) obtained from UM2N moved mesh better aligns with the high resolution results compared to the results on uniform mesh as shown in the Figure R3.

**Response to Q5:** Thank you for this inspiring question. Graphs are a natural way to represent unstructured meshes. We apply the graph transformer as encoder for its powerful expressivity, and apply the graph attention network as decoder because it can naturally help alleviate mesh tangling with some extra design. More specifically, we update the coordinate of each vertex with the combination of its neighbours, the coefficients of which are determined by the GAT module. This guarantees each vertex to only move within the convex hull of its neighbouring vertices, hence effectively alleviating mesh tangling issues.

We thank the reviewer for their suggestions for other benchmark methods. We agree these methods have their strengths but not all are equally relevant to our fully unstructured mesh based approach. For example, octree-based methods can be only applied to octree-based hierarchical meshes; point-cloud-based models ignore the topological relationship between vertices. Hierarchical models or multi-scale architectures such as U-Net is a very interesting idea, as the input and output of the model share the same structure, and capturing and sharing local and global information with hierarchical model structures is appealing, hence is definitely worth further investigation. In general, we agree with the reviewer that exploring these methods for mesh adaptation are interesting directions for future work.

**Reviewer jEN5:**
1. We perform an additional ablation study on network components (Table.R1/Figure.R2) to demonstrate the effectiveness of both Graph Transformer based encoder and Graph Attention Network based deformer.
2. We compare our method to related work qualitative and discuss the potential future direction.
3. We discuss the intuition and limitation of our model handing boundary with complex geometries.
4. We conduct additional experiment on a larger case (Figure.R5) and investigate the scalability based on our computational resources.

**Reviewer Jzyt:**
1. We provided clarification about the computation of monitor values as well as our pipeline.
2. We provide explanation about the chamfer distance issue the reviewer concerns.
3. We conduct an additional experiment dealing with poor quality initial meshes and visualize the results. (Figure R4).

Meta-review: Initially all reviewers gave a weak or borderline accept rating. The authors wrote a strong rebuttal that convinced two of the reviewers to upgrade to strong accept with the third one maintaining his weak accept.
Decision: Accept (spotlight)