Towards Universal Mesh Movement Networks

Authors: mingrui  zhang
chunyang  wang
stephan  kramer
joseph g wallwork
siyi  li
jiancheng  liu
xiang  chen
matthew d piggott
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-13 10:52:25

Abstract: Solving complex Partial Differential Equations (PDEs) accurately and efficiently is an essential and challenging problem in all scientific and engineering disciplines. Mesh movement methods provide the capability to improve the accuracy of the numerical solution without increasing the overall mesh degree of freedom count. Conventional sophisticated mesh movement methods are extremely expensive and struggle to handle scenarios with complex boundary geometries. However, existing learning-based methods require re-training from scratch given a different PDE type or boundary geometry, which limits their applicability, and also often suffer from robustness issues in the form of inverted elements. In this paper, we introduce the Universal Mesh Movement Network (UM2N), which -- once trained -- can be applied in a non-intrusive, zero-shot manner to move meshes with different size distributions and structures, for solvers applicable to different PDE types and boundary geometries. UM2N consists of a Graph Transformer (GT) encoder for extracting features and a Graph Attention Network (GAT) based decoder for moving the mesh. We evaluate our method on advection and Navier-Stokes based examples, as well as a real-world tsunami simulation case. Our method out-performs existing learning-based mesh movement methods in terms of the benchmarks described above. In comparison to the conventional sophisticated Monge-Ampère PDE-solver based method, our approach not only significantly accelerates mesh movement, but also proves effective in scenarios where the conventional method fails. Our project page can be found at https://erizmr.github.io/UM2N/.

Main paper: 
 introduction: 
 Various natural phenomena are modeled by Partial Differential Equations (PDE). The accurate and efficient approximation of the solutions to these complex and often nonlinear equations represents a fundamental challenge across all scientific and engineering disciplines. To solve real-world PDEs, many numerical methods require a computational mesh or grid to discretize the spatial domain. The quality of this mesh significantly impacts the balance between a numerical solution's accuracy and the computational cost required to obtain it. To maintain high-resolution everywhere in the domain is computationally expensive. Many systems modeled by PDEs however are multi-resolution in nature. For example, a small part of the system may be highly dynamic, while other regions are quasi-stationary; alternatively, some locations may be more important while others less important to the question being considered by the calculation [1] [2] [3] [4] . While a uniform, high-resolution mesh can be utilized to capture the dynamics or what is most important accurately, this is often wasteful of finite, precious computational resources and comes with sustainability implications [5] .An active body of research is focused on developing deep learning based methods to accelerate the PDE-solving process by learning surrogate neural solvers [6] [7] [8] . A learned solver can output PDE solutions directly given the physical parameters, and boundary conditions of a PDE. However, these methods encounter challenges such as not being able to guarantee physical plausibility of the PDE solution, weak generalization ability, and low data efficiency.An alternative approach to improve the efficiency of a PDE solver is to utilize mesh adaptation, which is a technique for distributing the mesh according to numerical accuracy requirements. Two main categories of mesh adaptation techniques can be identified: h-adaptation and r-adaptation. h-adaptation refines or coarsens the mesh resolution dynamically through topological operations such as adding/deleting nodes and swapping element edges/faces. In contrast, r-adaptation (or mesh movement) relocates or moves mesh nodes, keeping the mesh topology and thus the number of elements and vertices in the mesh unchanged [9] . These traditional mesh adaptation techniques can help reduce PDE-solving costs, but the mesh adaptation process itself may come at the cost of significant computational overhead.Deep learning based methods have been proposed to accelerate mesh adaptation. [10] [11] [12] [13] [14] [15] [16] . Most previous methods focus on h-adaptive refinement [11] [12] [13] 17] . Some existing works focus on learning error indicators or Riemannian metrics (which account for element shape and anisotropy, as well as size) [13] [14] [15] [16] to guide the mesh adaptation process. The learned indicator or metric is then fed into a traditional remesher for mesh generation, or a mesh optimizer, which overall limits the performance of these works to being no better than traditional methods, especially in terms of efficiency. Reinforcement learning based methods show potential to improve the mesh adaptation task, but are difficult to train with low data efficiency [11, 17, 18] . Only a small number of works focus on r-adaptation. [19, 20] investigated supervised and unsupervised learning based mesh movement methods. However, the proposed methods require re-training from scratch given a different PDE or geometry, which limits their applicability. In addition, the unsupervised learning method requires significant GPU memory and suffers from long training times, which makes re-training even more prohibitive for large scale problems.Targeting the generalization ability and efficiency of mesh movement, here we introduce the Universal Mesh Movement Network (UM2N), a two stage, deep-learning based model that learns to move the computational mesh guided by an optimal-transport approach driven by solutions of equations of Monge-Ampère (MA) type. Given an underlying PDE to solve numerically, the process of MA based mesh movement includes choosing a suitable monitor function (i.e., a measure for the desired mesh density) and solving an auxiliary PDE whose solution prescribes the movement of the mesh. The Monge-Ampère equation is an example of such an auxiliary PDE, which results in mesh movement with attractive theoretical properties [9] , but whose solution comes at a significant computational cost. Therefore, we decouple the underlying PDE solve and the mesh movement process. The learning of the mesh movement process is essentially learning to solve the auxiliary Monge-Ampère equation.The proposed UM2N consists of a Graph Transformer based encoder and a Graph Attention Network (GAT) [21] based decoder.Element volume loss is selected within the training loss, instead of the coordinates loss used in previous works [19, 20] . This offers the advantage that element volume loss not only provides supervision signals but also penalizes negative volumes (i.e., inverted elements), thereby reducing mesh tangling and enhancing robustness. We construct a PDE-independent dataset by generating random generic fields for model training. The trained model can be applied in a zero-shot manner to move meshes with different sizes and structure, for solvers applicable to different PDE types and boundary geometries. We comprehensively demonstrate the effectiveness of our methods on different cases including advection, Navier-Stokes examples, as well as a real-world tsunami simulations on meshes with different sizes and structures. Our UM2N outperforms existing learning-based mesh movement methods in terms of the benchmarks described above. In comparison to the sophisticated Monge-Ampère PDE-solver based method, our approach not only significantly accelerates mesh movement, but also proves effective in scenarios where the conventional method fails. The trained model can be directly integrated into any mesh based numerical solvers for error reduction and acceleration in a non-intrusive way, which can benefit various engineering applications suffering trade-offs between accuracy and computational cost.Our main contributions are listed as follows:• We present the UM2N framework, which once trained can be applied in a non-intrusive, zero-shot manner to move meshes with different sizes and structures, for solvers applicable to different PDE types and boundary geometries.• We propose PDE-independent training strategies and mesh tangling aware loss functions for our Graph Transformer and GAT-based architecture.• We demonstrate the universal ability of UM2N on different PDEs and boundary geometries: Advection, Navier-Stokes examples, and a real-world tsunami simulation case. 
related works: 
 Neural PDE solver. Neural PDE surrogate models can significantly accelerate the PDE solving process. There are three main groups of existing works. Physics-informed Neural Networks (PINNs) [6, 22, 23] are the first group, modeling the PDEs through implicit representations by neural networks. PINNs require knowledge of the governing equations and train neural networks with equation residuals from the PDEs with given boundary conditions and initial conditions. Neural operators are another group of methods for learning to solve PDEs which seek to learn a mapping from a function describing the problem to the corresponding solution function [7, 8, [24] [25] [26] . The third group is mesh-based PDE solvers with deep learning, including the utilization of CNNs and GNNs for processing structured and unstructured meshes [15, [27] [28] [29] [30] [31] [32] . Our method can be considered to be related to the neural operator approach, although it learns a mapping from monitor function values to the potential field describing the mesh movement instead of the PDE solution directly. In addition, the networks in the proposed model share similar ideas to mesh-based PDE solvers for processing unstructured meshes.Learning for mesh generation and adaptation. Deep learning methods have been advanced for applications in mesh generation and adaptation. Two tasks need to be differentiated here, the first involves modeling the ideal mesh resolution or density, e.g. through learning a metric or error indicator. The second involves the step that actually adapts the mesh in response, where we can distinguish between h-adaptivity, in which the connectivity structure of the mesh is changed to refine or coarsen the mesh, and r-adaptivity which maintains the connectivity and only moves the vertices.An example of the first task is MeshingNet [10] which employs a neural network to establish the requisite local mesh density, subsequently utilized by a standard Delaunay triangulation-based mesh generator. Similarly, the determination of optimal local mesh density for the purpose of mesh refinement is described in [12] . In [13] , the authors propose a learning-based model to determine optimal anisotropy to guide h-adaptation. [15] introduces a neural-network-based approach for predicting the sizing field for use with an external remesher. Instead of learning a metric, some works learn to manipulate the mesh directly. [11, 17, 18] formulate the h-adaptation process as a reinforcement learning paradigm, learning a policy that guides to adjust the mesh elements.r-adaptation (or mesh movement) is generally formulated in a fundamentally different manner to h-adaptation. Leading r-adaptation methods are based on a mapping between a fixed reference computational domain and the physical domain encompassing the adapted mesh, which is established through the solution of a nonlinear auxiliary PDE. [19] first proposed the use of a GNN based mesh movement network. [20] investigated the unsupervised learning of mesh movement for improving performance of neural PDEs solvers. [33] introduced a learning based mesh movement method specially tailored for computation fluid dynamics simulation of airfoil design.Our work focuses specifically on r-adaptation, i.e., mesh movement methods. The approaches proposed in previous work require re-training the model when applied to different problems, or can only generalize within a limited range of PDE parameters. We propose a mesh movement network that aims to be universal across applications to different PDEs without retraining.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: Please see section G Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
preliminaries and problem statement: 
 Monge-Ampère mesh movement. A mesh movement problem can be defined as the search for the transformation between a fixed computational domain Ω C , and a physical domain Ω P . Coordinates in these spaces are denoted ξ and x, respectively. The mesh H C defined in the computational domain is the original, often uniform mesh. The mesh H P defined in the physical domain is the adapted (or moved) mesh. 1 The goal of mesh movement is to find a mapping x = f (ξ) between the continuous coordinate fields of Ω C and Ω P , although in practice we deduce a mapping of the discrete vertex sets of H C and H P . While the coordinate transformation can also be used to accommodate time-dependent moving boundaries [34] or the optimisation of the shape of the domain, here we focus on its application to achieve variable resolution such that the solution u x of a PDE solved on the adapted mesh has higher accuracy than the solution u ξ solved on the original mesh.A monitor function m over the spatial domain is used to specify the desired mesh density, which can be based on various characteristics of the solution field u of the PDE. It prescribes where the mesh should be refined or coarsened, i.e. large monitor values indicate where high mesh resolution is required. Therefore, the goal of the mesh movement process can be rephrased as finding a mapping so that m is equidistributed over the adapted (i.e., physical) mesh [35] :EQUATIONwhere J is the Jacobian of the map x = f (ξ) with respect to the computational coordinates ξ, det(J ) denotes the determinant of the Jacobian which corresponds to the relative change in volume under the transformation, and θ is a normalization constant. By using concepts from optimal transport theory [36] , the problem can be constrained to have a unique solution, with the deformation of the map expressed in terms of the gradient of a scalar potential ϕ such that:EQUATIONSubstituting the additional constraint described by equation (2) into equation ( 1) gives a nonlinear PDE of Monge-Ampère type:EQUATIONwhere I is the identity matrix and H(ϕ) is the Hessian of ϕ, with derivatives taken with respect to ξ, i.e, H(ϕ) ij = ∂ 2 ϕ ∂ξi∂ξj .It should be noted that guarantees for existence and uniqueness, and convergence of solution methods for the Monge-Ampère equation generally rely on the domain being diffeomorphic with a convex domain (e.g. [37] ). In practice, the method is known to break down in some cases where the domain is not simply-connected or has non-smooth boundaries.The MA equation ( 3) is an auxiliary PDE which is purely associated with the the mesh movement process, and is independent of the underlying PDE or physical problem we wish to solve, i.e., the MA equation keeps the same form for different PDEs. This decoupling benefits learning-based methods in terms of generalization properties, i.e., a well-learned MA neural solver has the potential to be applied to different problems with a small cost of fine-tuning or even without re-training. The proposed UM2N framework is shown in Fig. 1 . Given an input mesh, vertex features and edge features are collected separately. The coordinates and monitor function values are gathered from the vertices and input into a graph transformer to extract embeddings. The embedding vector z obtained from the graph transformer encoder is then concatenated with the extracted edge features e to serve as the input for the Graph Attention Network (GAT) decoder. The decoder processes this combined input along with a mesh query to ultimately produce an adapted mesh.Graph Transformer encoder. Transformers [38] are a popular architecture which have been successfully applied for neural PDE solvers [25, 26, 39] . They have strong expressivity and can naturally handle irregular data structures such as unstructured meshes. Their attention mechanism can capture both the local and global information for a vertex of interest in a mesh. Here we use a graph transformer encoder as a feature extractor. The input features include the coordinates of the original mesh ξ and monitor values m ξ . The coordinates serve as the positional encoding. Features are concatenated and encoded into Q (query), K (key) and V (value) as inputs using MLPs for a self-attention module. The encoder output embedding z are fed into a downstream GAT based deformer for mesh movement.Graph Attention Network (GAT) based decoder. We choose GATs to construct our decoder. Its attention mechanism can help constrain the mesh vertex movement to within one hop of its neighbors which assists in alleviation of mesh tangling issues. The decoder consists of N GAT blocks. The first GAT block takes a mesh query ξ, and the embedding z from the transformer encoder as well as the edge features e as inputs. For each following k th block, the inputs consist of the coordinates of the initial mesh ξ, the coordinates of the intermediate moved mesh ξ (k-1) and extracted features from the previous layer f (k-1) .Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer: [Yes]• It should be clear whether the error bar is the standard deviation or the standard error of the mean. Justification: The reserach in the paper conform with the NeurIPS Code of Ethics.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
pde-independent dataset and training.: 
 In the existing work of neural mesh movement [19, 20] , proposed models are trained on solutions of PDEs, with the training data generated by solving one specific type of PDE, limiting generalizability of the trained model. Aiming here to train universal mesh movement networks, we construct a PDE-independent training dataset D = {d = (ξ, m ξ ; V r , T r )}, where ξ, m ξ denotes the original mesh and monitor values as the model's input, and V r , T r as the pre-calculated (reference) mesh's vertex coordinates and elements as ground truth. To build the dataset, we randomly generate generic solution fields, which are composed as the summation of a random number of Gaussian distribution functions centred in random locations, and with random widths in different directions to introduce anisotropy, samples are shown in Figure A1 ; refer to Appx. B for a detailed description. The generated fields can be interpreted as representing solutions of any PDE. For mesh movement, a widely used method to translate the PDE solution u(x) into a monitor function, is the Hessian based formula [36] :EQUATIONwhere ∥H(u)∥ indicates the Frobenius norm of the Hessian of u and α is a user chosen constant. This choice results in small cells where the curvature of the solution is high, with a ratio of 1 + α between the largest and smallest cell volumes. Our network is trained on the values of the monitor function only, which can be computed relatively cheaply from the PDE solution. The expected movement of the mesh vertices is computed by solving the Monge-Ampère equation (3) for each of the monitor functions based on the randomly generated PDE solutions. After training, the model can then be applied using only the monitor values as input, independent of the PDE that is being solved. Other choices of formulae for the monitor function in terms of u may be appropriate in different cases, but again these can be applied without re-training the model, e.g., the flow-past-a-cylinder and tsunami test cases in the following section use a monitor function based on the gradient of the solution.Given the dataset D defined above, the final objective is to find model parameters θ, by minimizing the total loss L total . Thus, we can formalize the process asEQUATIONwhere L vol , L cd represents the element volume loss and Chamfer distance loss respectively, which are defined later. λ cd , λ vol > 0 represent hyper-parameters balancing these two effects.We denote the modified (adapted) mesh with model parameters θ as M(ξ, m ξ ; θ) = {V a , T a }, where V a = {x i } n1 i=1 represents the coordinates of vertices and T a = {t i } n2 i=1 represents the elements with x i , t i as the i th vertex and element. For ease of presentation, we omit the θ here. Similarly, we define y i , q i as V r , T r 's i th vertex and element.Element volume loss. In contrast to the coordinate loss used in previous work [19, 20] , here we use the element volume loss for training. Element volume loss is computed as the averaged volume difference between each element in the adapted mesh (T a ) and the reference mesh (T r ) computed from the MA method. It is defined asEQUATION)where Vol is the function computing the volume of a given element t.The element volume loss is inspired by the equidistribution relation in equation (1) . Intuitively, the equidistribution relation enforces deformed mesh elements such that their volumes correspond to rescaling by the monitor function value m(x). Therefore, the element volume loss is utilised to encourage the model to learn to move the mesh so as to conform to the equidistribution relation, under the guidance of MA method, given that the MA method provides one good way to achieve this relation. In addition, the element volume loss penalizes negative Jacobian determinants, which helps prevent element inversion, i.e., mesh tangling.Chamfer distance loss. Chamfer distance finds for all vertices in the first mesh, the nearest vertex in a second mesh and sums the square of these distances. It encourages the model to output a mesh which has a similar spatial distribution of vertices to that of the reference mesh. To achieve vertex-to-vertex alignment, the bidirectional Chamfer distance is utilized without additional sampling. It is defined asL cd (θ) = E (ξ,m ξ ;V r ,T r )∈D   1 |V a| x i ∈V a min y j ∈V r ∥xi -y j ∥2 + 1 |V r | y j ∈V r min x i ∈V a ∥xi -y j ∥2   . (7)5 Experiments 
experiment setups.: 
 Unlike existing approaches to mesh movement, such as [19, 20] , which re-train their model on a caseby-case basis, our study aims to explore the universal applicability of the proposed UM2N. Therefore, all examples shown in this section are tested in a zero-shot generalization manner, i.e., without re-training. In addition to PDE type, our training meshes are of small size typically comprising 500 vertices, while our test meshes have far more vertices: Advection ( Metrics. The main metric for evaluating mesh movement quality is underlying PDE solution error reduction (ER) ratio. Here PDE error is approximated by the difference between the solutions obtained on a coarse mesh and accurate solutions which are obtained on a very high resolution mesh. The PDE error reduction ratio is the difference between the PDE error from an original mesh and that from an adapted mesh. Another important metric for mesh movement is mesh tangling. Once mesh tangling happens, the mesh is invalid for a numerical solver, which breaks the simulation. In the following section, we benchmark the mesh movement using both non-learned Monge-Ampère (MA) mesh movement implemented in Movement [40] and learning-based baseline M2N [19] over three different scenarios to compare with UM2N. Helmholtz. The Helmholtz case is a time-independent problem solving a 2D Helmholtz equation as in (A2). The source terms of the equation are generated use formula (A1). Our model achieves better error reduction than M2N and comparable to MA method in significantly less time.Swirl. The swirl case considers a time-dependent pure advection equation. In this scenario, the initial tracer field takes a narrow ring shape and is advected within a swirling velocity field, see Eq. ( A3) and (A4) in Appx. C. The velocity swirling direction is anti-clockwise in the first half of the simulation and clockwise for the second half. Given this setting, as the advection process is reversible, the final state should be consistent with the initial state. This constitutes a problem of significant multi-resolution complexity, as the extremely narrow ring requires a high resolution to accurately resolve the dynamics at the sharply defined interfaces of both inner and outer sides.The results are shown in Fig. 2 . As shown in Table 1 , our UM2N shows an average 35.93% error reduction which clearly outperforms M2N's 14.15%. It is close to that of the full Monge-Ampère (MA) mesh movement approach which achieves an error reduction of 37.43%, but with significantly lower computational time.Flow past cylinder. Simulation of flow past a cylinder in a channel is a classic and challenging multiresolution case of computational fluid dynamics, solving the Navier-Stokes equations. The cylinder and the channel's top-bottom surfaces require a high resolution mesh to resolve the boundary layers [41] . Given appropriate Reynolds number, the well-known von Kármán vortex street phenomena can be observed within the wake flow which also benefits from higher resolution to resolve the time-varying evolution of vorticity structures. The drag C D and lift C L coefficients for the cylinder are other quantities of interest in this problem. Here we use a benchmark configuration from [42] , please refer to Appx.C.3 for more details.The simulation is run for 8, 000 time steps of size 0.001 s, i.e., 8 s physical time for the whole simulation. The original mesh is a uniform mesh with triangular elements and 4, 993 vertices. The reference solutions are obtained on a high resolution mesh with 260, 610 vertices.We investigate the dynamics in the area around the cylinder and in the wake flow respectively. For the former, we use the measure of C D as shown in Fig. 3 . The results for C L is shown in Fig. A10 in Appx. E. For the wake flow, we use cross-section probes at x = [0.5, 1.0, 1.5, 2.0] to measure the intensity of vorticity spatially along the y-axis as shown in Fig. 4 .Qualitatively, it can be observed that the proposed method moves the mesh in a manner that captures the fluid dynamics for the whole domain in general (see upper part of Fig. 3 ). At the cylinder and top-bottom surface, the vertices are moved to increase the resolution in order to resolve the boundary layers as expected. Quantitatively, the drag C D and lift C L coefficients computed on our moved mesh show a 61.29% average error reduction compared to that of original mesh as shown in lower part of Fig. 3 . The proposed UM2N approach not only improves the accuracy of these coefficients in magnitude but also prevents their periodic variation from phase shifting. For the wake flow, as shown in Fig. 4 , we selected a single snapshot at t = 4.0s for qualitative investigation. Our UM2N also shows an accuracy improvement for vorticity intensity on data sampled from all four cross-section probes comparing to the that on original mesh. It shows that our UM2N can output meshes which adapt to the dynamics around the cylinder. It can be observed from the zoom-in mini-plots at the top-left that UM2N reduces errors/noise in vorticity intensity compared to the original mesh. In the lower part, the blue, orange and green lines show the drag CD coefficients obtained on a high resolution fixed mesh, UM2N adapted mesh and the original coarse mesh for the first 6 seconds (i.e., 6000 steps). It can be observed that the UM2N output mesh improves the accuracy of CD in magnitude and periodicity compared to the original mesh. Note that we intended to compare our UM2N to M2N and the conventional full MA method. However, both MA and M2N fail i.e., the solver diverges, within 30 time steps due to mesh tangling and thus provide insufficient data points for comparison. For comparisons with a non-tangled M2N case, please refer to the simplified laminar flow-past-cylinder example presented in Appendix D.1.Here we present the case of a simulation of the 2011 Tōhoku Tsunami, to show that our methods can be applied to real world scenarios with boundaries describing highly complex geometries. The tsunami is simulated by solving the shallow water equations using the Thetis framework [43] . For details on the datasets and software used to set up the Tōhoku case study, see Appx. B.2. Note that the mesh (8, 117 vertices) has an arbitrarily irregular boundary. It can be Figure 5 : Qualitative results for the Tōhoku tsunami simulation. The boundary of the simulation is generated based on the real coastline data (see the mesh overlaid on the satellite map). We show 9 snapshots of the wave elevation from 80 steps of the tsunami simulation enhanced by our UM2N. In these visualizations, red and blue hues indicate wave elevations that are, respectively, above or below mean sea level. Regions exhibiting significant elevation magnitudes necessitate increased resolution to accurately resolve the underlying dynamics. Our method dynamically and robustly adjusts the mesh to increase resolution at the wave front, thereby effectively tracking its propagation.Monge-AmpèreFigure 6 : Handling complex boundaries. The Monge-Ampère method generates inverted elements, resulting in mesh tangling near the coastline as shown in the areas delineated by red rectangles. Our UM2N can smoothly handle the elements near the coastline without mesh tangling.observed that the mesh moves in a manner that helps track the wave propagation with enhanced resolution, as shown in Fig. 5 . Our model can also robustly handle the highly complex boundary without mesh tangling near the coastline. As a comparison, the conventional Monge-Ampère (MA) mesh movement approach struggles to handle the complex geometries, i.e., fails to converge when solving the non-linear MA equation, resulting in highly tangled elements near the coastline as shown in Fig. 6 . Therefore, our method not only provides a more efficient approach but also show advantages dealing with complex boundaries compared to the full MA mesh movement approach. Please see the full video of this case in the supplementary materials.There are in fact still cases where UM2N might fail. To investigate these, we perform additional experiments as shown in Figure 7 . Starting from a rectangle, we gradually distort the geometry to a more and more non-convex shape, i.e., the case becomes more challenging from top to bottom. The UM2N finally fails in the extreme case at sudden jumps in the boundary accompanied by a large variation in the required resolution as shown in the bottom row of Figure R1 : a sudden constriction of the flow leads to tangling with UM2N in the two left corners of the channel. In less extreme cases, UM2N does produce a valid mesh, whereas the original MA method still fails (as seen in the second to bottom rows in Figure 7 ). Volume loss vs coordinate loss. We denote the UM2N variant trained with coordinate loss [19] as UM2N-coord. As shown in Table 2 , UM2N-coord achieves comparable error reduction (31.21%) to the proposed UM2N (35.93%) on the Swirl case. However, it fails (i.e., encounters mesh tangling) on the Cylinder case which has a more complex boundary geometry. This indicates that the element volume loss is mesh-tangling aware, i.e., helps to prevent producing inverted elements.Monitors vs PDE solutions as inputs. We denote the UM2N variant using the PDE solution as input, instead of monitor function values, as UM2N-sol. The UM2N-sol is trained on solutions of the Helmholtz PDE. The UM2N-sol shows a much inferior performance (-0.35%) compared to UM2N (35.93%) on the Swirl case as well as on the Cylinder case. This indicates that learning mesh movement from monitors directly improves the generalisability of the model. 
conclusions: 
 We introduce the Universal Mesh Movement Network (UM2N), which once trained on our generated PDE-independent dataset, can be applied in a non-intrusive, zero-shot manner to move meshes with different sizes and structures, for solvers applicable to different PDE types and boundary geometries. Our method demonstrates superior mesh movement results on Advection and Navier-Stokes PDE examples as well as a real-world tsunami simulation case. Our UM2N outperforms the existing learning based method and achieves comparable PDE error reduction to the far more costly Monge-Ampére (MA) PDE based approach. Our UM2N demonstrates effectiveness in cases with complex boundary geometries where existing learning and conventional MA based methods fail. Our method also shows significant acceleration compared to MA based mesh adaptation. See Appx. G for discussions on limitations and broader impacts. As mentioned in Sec. A, the mesh of the Japan Sea is generated using qmesh [48] . For this, we provide qmesh with coastline data from [50] . Bathymetry (sea bed topography) data is taken from the ETOPO1 dataset [51] . A highly idealised tsunami source condition is used, comprised of a sum of Okada functions [52] .  
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: Please see section G Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA]Justification: [NA] Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: Please see the appendix A Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. 
 
 Appendix: To evaluate the runtime performance of UM2N, we constructed a benchmark case based on the Swirl scenario, consisting of 11,833 vertices and 23,668 triangular elements. We extended the one-ring setup to a nine-ring configuration. In this benchmark, we compare the UM2N output mesh and the reference mesh generated by the MA method, as well as the solutions obtained on these meshes, against those computed on high-resolution and original meshes as shown in Fig. A3 .We tested the inference time of UM2N on two GPUs (Nvidia GeForce RTX 3090 and Nvidia A100) and compared its performance to the MA method. For a fair comparison, we also included CPU benchmarks, as the current state-of-the-art MA implementation is limited to CPU execution. The CPU used for testing was an 11th Gen Intel(R) Core(TM) i9-11900K @ 3.50GHz. The performance results are presented in Table A2 .The results indicate that UM2N achieved more than 2000x speedup on the Nvidia RTX 3090 and over 3000x on the Nvidia A100, compared to the MA method. Even on the CPU, UM2N demonstrated an approximate 100x speedup. In summary, UM2N provides comparable solution accuracy while delivering a significant performance improvement. In our flow past the cylinder case, at the top and bottom we imposed non-slip boundary conditions which already promotes a boundary layer similar to the suggested flow over a flat plate. For a clearer illustration, we have now additionally conducted a flow past two parallel plates experiment. We observe that the velocity parallel to the boundary (u y ) obtained from UM2N moved mesh better aligns with the high resolution results compared to the results on uniform mesh as shown in the Fig. A4 .We include an additional experiment starting from a poor quality initial mesh as shown in Fig. A5 .The input mesh has highly anisotropic elements and several vertices with valence 6. This can certainly be considered a "low quality" mesh. Consider the monitor function to be the L 2 -norm of the recovered gradient of the solution of an anisotropic Helmholtz problem. With this monitor function, we tried applying conventional MA solvers and found that both the quasi-Newton and relaxation approaches failed to converge and/or resulted in tangled meshes, even though the elements are already aligned with the anisotropy of the Helmholtz solution. The UM2N approach, however, was able to successfully apply mesh movement without tangling. We would also like to clarify that our work targeting mesh movement that dynamically adapts to a PDE solution, which is not a replacement for mesh generation in general, so we would always expect a reasonable input mesh when applying our model.Original MeshFigure R4 . A very poor initial mesh case. UM2N perform mesh movement for an anisotropic Helmholtz problem. We perform an additional ablation study of our network architecture on the Swirl case. We construct two models, which replace the GAT Decoder with one layer GAT denoted as UM2N-w/o-Decoder and remove the graph transformer denoted as UM2N-w/o-GT. The results are shown in the Table A6a .It can be observed that both variants give worse results compared to the full UM2N. A visualization is also shown in Fig. A6b . It can be observed that without the Decoder, the model distorts the shape of the ring, i.e., missing relative information between vertices; without the graph transformer, the model fails to capture the details of the ring shape.Flow past multiple cylinders. The Fig. A7 shows a challenge scenario with 5 cylinders in the domain. The proposed UM2N can move mesh capturing the complex dynamics without mesh tangling.Flow past V-formation cylinders. The Fig. A8 shows a bio-inspired V-formation setting. The proposed UM2N can move mesh capturing the wake flows.Flow past cylinders and squares. The Fig. A9 shows the proposed UM2N can handle another challenging scenario with a mixture of cylinders and squares of different sizes.Lift coefficient (C L ) of flow past cylinder. In Fig. A10 , we visualized the lift coefficient over timestep across different methods. The Mesh Movement Enhanced Simulation Algorithm, as outlined in Alg. 1, integrates our mesh adaption strategies into the simulation process to enhance the quality and accuracy of computational solutions. The proposed MMES procedure can be adapted into almost any time-dependent numerical simulation processes non-intrusively.For a given time-dependent PDE P and an initial mesh ξ init , the algorithm evaluates the monitor function m based on the solution field of the PDE at each time step t. The monitor value m is then used by our proposed model to guide mesh movement. For robustness, a mesh integrity check is performed before time stepping (a tangled mesh will interrupt the procedure). The blue, orange and green lines show the drag CL coefficients obtained on a high resolution fixed mesh, UM2N adapted mesh and the original coarse mesh for the first 6 seconds (i.e., 6000 steps). It can be observed that the UM2N output mesh improves the accuracy of CL in magnitude and periodicity compared to the original mesh.Limitations. 1) The GAT to an extent constrains the mesh movement to lie in a restricted range, which may be less effective in scenarios requiring substantial large deformation within a single iteration of the mesh mover. 2) There is no theoretical guarantee that mesh tangling will be completely prevented.3) The model's best performance relies on a manually chosen monitor function, though a gradient based monitor function can yield reasonable results. 3) Given a low-quality or extremely coarse original mesh, the efficacy of the proposed method may be constrained by the inherent Algorithm 1 Mesh Movement Enhanced Simulation end while 15: end procedure limitation on the degree of freedom count, as it redistributes rather than increases the number of mesh vertices. A promising future direction could be exploring the integration of mesh movement with h-adaptation, potentially developing a hybrid hr-adaptation approach that combines the advantages of both techniques.Broader Impacts. This research contributes to the field of AI for physical simulation by introducing an advanced learning-based mesh movement method that enhances the accuracy and efficiency of numerical simulations. The primary application of this method lies in areas requiring high-fidelity simulations of physical phenomena, such as geophysics, renewable energy, climate modeling, and aerodynamics. This method can aid in better weather or hazards forecasting, more efficient energy harvesting, fast prototyping of aircraft design etc.NeurIPS 2024 Checklist • The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.In what follows, we abuse notation slightly by also using ξ to refer the the (discrete) vertices of HC , rather than the (continuous) coordinate field. Similarly, we also use x to denote the vertices of HP . 


Decision: Initially all reviewers gave a weak or borderline accept rating. The authors wrote a strong rebuttal that convinced two of the reviewers to upgrade to strong accept with the third one maintaining his weak accept.

Reviews: review: paper_summary: paper_summary: In the present work, authors tackle the challenging task of accelerating PDE-solving process with deep learning, focusing on mesh movement problem. They suggest a new architecture that combines graph transformer encoder and graph attention decoder to significantly accelerate the PDE-solving. Authors propose a way to decouple the underlying PDE solving from the mesh movement process itself. It makes the approach universal with regard to solvers as well as to boundary geometries and mesh structures. They demonstrate the superiority of their method over conventional solver, and the ability on various benchmark tasks.
strength_weakness: strength_weakness: Authors do not demonstrate a single experiment, where their model provides mesh tangling. Authors explain such effect by the use of the volume loss (line 326) calling it "tangling-aware". In general, it might be good, since one can be sure that any use of UM2N is reliable. 
Yet, without proper evaluation (or at least, clarification for such perfection), It cannot be regarded as a virtue. It implies either that the experiment setups are chosen to be weak or the model has the inductive bias that might limit its universality. In other words, there might exist some setups that the model won't be able to estimate correctly, because of oversmoothness. As for example, one can imagine the domain with a tight bottleneck, the flow might not propagate to the other side at all. 

Another side of the same issue can be seen in the flow-past-cylinder experiment. Both MA and M2N provide some tangling and hence considered as "Failed", so entire discussion in this experiment is in favor of the proposed UM2N. Authors call the setup "classic and challenging", yet there is not a single baseline that provides at least some solution. Either another baseline that actually gives some number should be provided or the setup should be simplified, e.g. choosing the object that provides less turbulence, such as the airfoil.
questions: questions: I would appreciate if authors discuss some of the issues I raised above in the Weaknesses. 

Mainly: non-tangled results of other methods on the flow-past-cylinder; tangled results of the UM2N model itself
limitations: limitations: -
paper_summary: paper_summary: The paper introduces the Universal Mesh Movement Network (UM2N), a deep-learning model that enhances solving Partial Differential Equations (PDEs) through adaptive mesh movement. UM2N, with a Graph Transformer encoder and a Graph Attention Network (GAT) based decoder, is trained on a PDE-independent dataset, allowing zero-shot application to various PDEs and geometries without retraining. The model uses element volume loss to improve robustness and reduce mesh tangling.
strength_weakness: strength_weakness: 1/ Comparison with State-of-the-art methods:

While UM2N is compared with a few existing methods, the comparison could be broadened to include more recent state-of-the-art approaches. This would provide a clearer context for UM2N's performance and highlight its relative advantages and limitations more comprehensively. Expanding the experimental section to include comparisons with more recent and relevant learning-based mesh movement methods would be valuable. If new experiments are not feasible during the rebuttal, a detailed qualitative comparison discussing potential strengths and weaknesses based on the literature can be provided.

2/ Ablation Studies:

The ablation studies are somewhat limited. A more comprehensive analysis of different components of UM2N, such as the impact of various hyperparameters, the contribution of each network component (Graph Transformer vs. GAT), and the sensitivity to different types of monitor functions, would provide deeper insights into the model's workings. The authors could add more ablation studies focusing on hyperparameter tuning, the role of different network components, and the sensitivity analysis to different monitor functions.


3/ Handling extreme geometries and mesh qualities:

The current paper does not extensively explore how UM2N handles extremely poor-quality initial meshes or highly irregular and extreme geometries. These scenarios are crucial for practical applications, as real-world problems often involve complex and challenging geometrical domains. Poor-quality initial meshes, characterized by elements with highly skewed aspect ratios, large variations in element sizes, or even degenerate elements, can significantly complicate the mesh movement process. For instance, elements with aspect ratios far from unity can lead to numerical instabilities and inaccurate solutions due to poor approximation properties. Large variations in element sizes can cause localized errors to propagate or lead to uneven error distribution across the mesh, while degenerate elements with nearly zero area or volume can invert or fold during movement, leading to invalid meshes.

Similarly, highly irregular geometries, such as those with sharp corners, narrow channels, intricate boundary shapes, or complex topological features like holes and multiple connected components, can test the robustness and adaptability of mesh movement algorithms. Sharp corners and narrow channels require fine mesh resolution to capture the geometric details accurately, while avoiding mesh element inversion or tangling. Intricate boundary shapes and complex topological features increase the difficulty of maintaining mesh quality and conformity to the domain boundaries during adaptation.

A. Aspect Ratio and skewness:

Maintaining aspect ratios close to one is crucial to minimize numerical errors. High aspect ratios can lead to poor interpolation properties, making it challenging to preserve or improve aspect ratios during significant mesh adjustments.

High skewness causes poor approximation and solver instability. Controlling skewness during mesh movement to prevent highly distorted elements is critical.

Suggestion: implement a loss term that penalizes high aspect ratios and skewness during training to encourage well-shaped elements.

B. Element size variation:

Uniformity and adaptivity: 
Balancing element sizes to ensure smooth transitions between refined and coarse regions is complex. Small elements can lead to computational overhead, while large elements might miss critical solution features.

Dynamic adaptation:
Efficiently refining in regions of high gradients and coarsening elsewhere requires dynamic strategies, balancing computational cost and accuracy.

Suggestion: 
Use adaptive mesh refinement algorithms that dynamically adjust element sizes based on local error estimates and monitor function gradients.

C. Mesh tangling:

Inversion and Overlap:

Preventing elements from inverting or overlapping during movement is essential. This requires strategies like regularization terms in the loss function to penalize negative volumes and ensure coherent adaptation.

Local and Global Coherence:
Ensuring local coherence in element displacement to prevent tangling, especially in regions with high gradient monitor functions, is challenging.

Suggestion:
Introduce a regularization term in the loss function to penalize negative volumes and use local displacement constraints to maintain element integrity.

D. Boundary conformance:

Complex boundaries: 
Accurately adapting the mesh to complex geometries with sharp features and varying topologies without introducing distortions requires careful refinement near boundaries.

Boundary preservation: 
Techniques such as boundary snapping and boundary layer refinement are necessary to maintain accurate geometric fidelity.
    
Suggestion:
For instance applying boundary snapping techniques and boundary layer refinement to ensure elements conform accurately to the physical boundaries.


E .Adaptive refinement and coarsening:

Dynamic adaptation:

Adapting the mesh based on evolving solution features, such as moving fronts, involves adding or removing elements dynamically.

Smooth transitions: 

Ensuring smooth transitions between different resolution regions without creating poorly shaped elements is crucial. Hierarchical refinement and mesh smoothing techniques are often employed.



4/ Theoretical guarantees and limitations:

While the paper mentions that there are no theoretical guarantees that mesh tangling will be completely prevented, it would be beneficial to delve deeper into this limitation. Understanding the theoretical bounds of UM2N's performance and the conditions under which it might fail is important for setting realistic expectations.


5/ Real-world application scenarios:

Although the tsunami simulation is an excellent example, including more diverse real-world application scenarios would further demonstrate UM2N's versatility and robustness. For instance, applications in aerodynamics, biomechanics, or climate modeling could be explored.

Suggestion: 

Add more case studies or at least detailed discussions of how UM2N could be applied to other real-world problems, emphasizing its practical benefits and potential challenges in these domains.


6/ Computational efficiency and scalability:

The paper discusses the reduction in computational costs compared to traditional methods but does not provide detailed benchmarks or discussions on the scalability of UM2N for very large meshes or real-time applications.

Suggestion:

Include more detailed benchmarks of computational efficiency and scalability, comparing UM2N's performance on large-scale problems with other methods. 
Discuss strategies for optimizing performance and potential bottlenecks in the current implementation.


7/ Standpoint from Multi-Scale phenomena:

The paper does not address the specific challenges posed by multi-scale phenomena, where different parts of the domain may require vastly different resolutions to capture fine-scale features accurately while maintaining computational efficiency. Multi-scale phenomena often involve a wide range of spatial and temporal scales, making it difficult to balance resolution and computational cost effectively.

Suggestion:

Introduce experiments that specifically target multi-scale phenomena, demonstrating UM2N's ability to adapt the mesh dynamically to capture fine-scale features in high-gradient regions while coarsening in less critical areas. This could involve benchmarks on problems known for their multi-scale nature, such as turbulent flows or geophysical simulations. Additionally, discuss potential enhancements to the model, such as multi-grid techniques or hybrid approaches that combine UM2N with other multi-scale modeling strategies, to better handle these complex scenarios.

8/ Analysis from mesh continuity, local and global deformation viewpoints:

The paper does not thoroughly address how UM2N manages local and global deformations of the mesh, which are essential for accurately capturing complex physical phenomena. Effective mesh adaptation requires the ability to handle fine-scale local deformations to capture detailed features and large-scale global deformations to adapt to overall changes in the domain. Additionally, ensuring mesh continuity during these deformations is crucial to maintain the fidelity and stability of numerical simulations. Inadequate handling of these deformations can lead to poor resolution of critical areas, excessive computational costs, and potential discontinuities that degrade the accuracy of the simulation.

A/ Coupling local and global deformations: 

Balancing local and global deformations is crucial for maintaining overall mesh quality and continuity. This involves ensuring that local refinements do not introduce excessive computational overhead and that global adaptations do not degrade the resolution of critical regions. Achieving this balance is technically challenging and requires sophisticated mesh adaptation algorithms that can handle both fine-scale and large-scale changes seamlessly.

B/ Mesh quality and continuity maintenance:

During both local and global deformations, maintaining high-quality elements (in terms of aspect ratio, skewness, and smoothness) is essential to avoid numerical instability and ensure accurate simulations. Poorly shaped elements can significantly degrade the performance of numerical solvers. Additionally, ensuring continuity in the mesh, where element shapes and sizes transition smoothly, is vital for maintaining numerical stability and accuracy.


9/ Analyze the optima of the learned meshes:

The paper does not address the issue of ensuring that the learned mesh configuration is globally optimal, nor does it consider the potential existence of multiple local optima in the optimization landscape. This oversight can result in suboptimal mesh configurations, which may not fully capture the desired features of the simulation domain or may introduce unnecessary computational overhead. In complex adaptive meshing scenarios, the risk of converging to local optima rather than the global optimum can significantly impact the accuracy and efficiency of the numerical solutions.

The optimization process for adaptive mesh movement often involves a highly non-convex loss landscape with many local minima. A single global optimum represents the best possible configuration of the mesh in terms of accuracy and computational efficiency. However, due to the complex nature of the problem, the optimization algorithm may converge to multiple local optima, each providing a suboptimal solution that fails to maximize the potential benefits of the adaptive meshing process.

The presence of multiple local optima can lead to inconsistent mesh configurations across different runs or simulations. This variability can make it difficult to ensure that the mesh is optimally adapted to the specific features of the simulation domain. Inconsistent convergence can also lead to variability in simulation results, reducing the reliability and robustness of the numerical methods.

Suggestion:

Benchmark the learned mesh against known optimal solutions or high-resolution reference meshes. This benchmarking can help assess how close the learned meshes are to the ideal configuration and identify specific areas where the model falls short.
questions: questions: 1/ How does UM2N handle extremely poor-quality initial meshes and highly irregular geometries? 

Suggestion: 

Conduct additional experiments with initial meshes characterized by highly skewed aspect ratios, large variations in element sizes, and complex boundary geometries. Include quantitative metrics on mesh quality before and after adaptation to highlight the robustness of UM2N.

2/  How does UM2N ensure mesh continuity and high-quality element shapes when dealing with both local and global mesh deformations?

3/ How does UM2N's learned mesh configuration compare to known optimal solutions or high-resolution reference meshes? Are there benchmarks or validation cases included to demonstrate the accuracy and efficiency of the mesh adaptation?

Suggestion: 

Benchmark the learned mesh configurations against known optimal solutions or high-resolution reference meshes. Include comparisons using quantitative metrics such as error norms and gradient capture to highlight the accuracy and efficiency of UM2N in approaching the optimal mesh configuration.

4/ How does UM2N perform in resolving boundary layers and turbulent flows? Could you provide experimental results or benchmarks that validate its effectiveness in these critical fluid dynamics applications?

Suggestion:

Conduct targeted experiments on canonical boundary layer cases (e.g., flow over a flat plate) and turbulent flow benchmarks (e.g., turbulent channel flow). Use quantitative metrics like wall shear stress and turbulence intensity to evaluate performance and compare with high-fidelity simulations or experimental data.

5/  Can you elaborate on the rationale behind selecting Graph Attention Networks (GAT) over other Graph Neural Network (GNN) methods for mesh movement? 
Additionally, have you considered or benchmarked against hierarchical GNNs, Octree-based methods, or point cloud-based methods, such as those utilizing multi-scale architectures like U-Net? 
Exploring these alternatives could provide insights into potential improvements in handling mesh adaptation across different scales and complex geometries.

if the authors address the weaknesses and questions outlined above, I would be happy to increase my score. Doing so will significantly enhance the robustness and applicability of the proposed work.
limitations: limitations: Yes.
paper_summary: paper_summary: This paper introduces a learnable model for mesh movement - which is a method for improving efficiency of a PDE solver by moving mesh nodes, while keeping the topology fixed. The proposed model itself is a two-stage neural architecture comprising a transformer encoder - taking a representation of the input mesh - and a graph attention network as a decoder outputting an updated mesh. The resulting model is independent of specific PDE type and thus can be applied to novel problem types in zero-shot manner. Quantitative evaluation is conducted on multiple synthetic and one realistic (tsunami prediction) tasks.
strength_weakness: strength_weakness: - Although overall the architecture makes sense, there is no ablation study conducted on the individual parts of the architecture.  
- Similarly, there are no baselines to off-the-shelf graph convolutional or transformer methods (any recent method e.g. for semantic mesh segmentation would work here?). 
- The model is only concerned with vertex movement of the model, which means there is no guarantee that there won't be any degraded triangles, which can influence the quality of the dowstream solutions. This could be a fundamental flaw of purely data-driven methods since there is not simple way to enforce constraints at test time. 
- The scale of the dataset seems to be very small for the model to be directly applicable in realistic scenarios - both in terms of number of samples and number of vertices. 
- Model relies on a transformer encoder, which will not scale to meshes with large number of vertices.
questions: questions: - In figure 6, you demonstrate that um2n is better at handling complex boundaries. From what I can see there are no guarantess for your purely data-driven method to produce non-tangled meshes? Do you have an intuition to why this is the case?
- Would this model work for larger scale problems with >1M?
limitations: limitations: Limitations are discussed.
paper_summary: paper_summary: The authors propose a method for general mesh movement. They design a network that takes the original mesh and the monitor values as input to predict the mapping for the adapted mesh. This makes it able to handle any mesh without the need of training PDE-type-specific models.
strength_weakness: strength_weakness: The idea of this paper is interesting to me, and the results, especially those shown in the supplementary video, are pretty nice. I believe the proposed method can have many potential application in mesh formation and the modeling of physical phenomena. I didn't see any obvious weaknesses from this paper. However, I have to admit that I'm not the expert in this field.
questions: questions: 1. The network takes the original mesh and the monitor values $m$ as input. $m$ is the values for the original mesh or the target mesh? If it is the value of the original mesh, then how does the network know the target? If it is the value of the target mesh, then the symbol of $m_\xi$ is a bit misleading, since $\xi$ denotes the original mesh.
2. When using chamfer distance for the optimization of mesh matching, the result is easy to be stuck at the local minimum. Is it the case for the training of the proposed method?
limitations: limitations: In the section of Limitation, the authors mentioned that a low-quality original mesh can lead to poor prediction. A visualization of it will be nice.

Comments: title: Thank you
Thank you
Response to follow-up questions and thank you
Thank you!
Thank you!
reply to Rebuttal
Summary
Follow-up questions
Response to the rebuttal and follow-up
comment: Thanks again for your comments and engaging dialogue. We commit to include the discussion into the revision.
Thanks for the reviewer's acknowledgment of our efforts to address the concerns!
Thank you for the detailed responses. Please include the discussion in the revised paper
Thanks for the detailed response. This addresses my questions/concerns so I am keeping my original score.
We thank for the reviewer's constructive feedback and acknowledgment of our efforts to address the concerns. We are very happy and greatly appreciate that the reviewer improve the score!

Here are our responses to the follow-up questions:

**Response to Q1:** Thanks for the suggestion. We would commit to provide more quantitative metrics for the UM2N for the mesh quality and computational efficiency in the revision.

**Response to Q2:** Thanks for raising this question. For large-scale problems, in addition to applying memory-efficient transformers, the computation efficiency is another issue as the time-complexity of self-attention is O(N^2) where N is the number of vertices of a mesh. 
The main strategy is to apply linear-attention transformers. The linear-attention reduces the time-complexity  from O(N^2)  to O(N). There are many existing works in this direction [1][2] from the natural language processing community as well as in the neural PDE solver community [3][4]. In addition, attention with masks may also help improve the efficiency. Similar intuitions from [5], we hypothesize there is redundant information in a representation of meshes and masking would not severely degrade the performance. These preliminary ideas for handling large-scale problems can be explored in the future work.

[1] Katharopoulos, A. et al, Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention (ICML 2020)

[2]Kitaev et al, Reformer: The Efficient Transformer (ICLR 2020)

[3] Cao et al, Choose a Transformer: Fourier or Galerkin (NeurIPS 2021)

[4] Hao et al, GNOT: A General Neural Operator Transformer for Operator Learning (ICML 2023)

[5] He et al, Masked Autoencoders Are Scalable Vision Learners (CVPR 2022)

**Response to Q3:** This is a great question that touches on important aspects of the paper. One distinction in types of scenarios, is on the one hand steady state problems, or problems where one doesn't attempt to dynamically follow time-dependent features with mesh resolution, which focus on the optimisation of a single mesh and its spatial properties, and unsteady problems where the mesh is adapted in time to focus resolution on time-dependent features or to adjust to moving boundaries. For the second category minimizing the amount of change between subsequent meshes, as you mention, becomes very important. One important practical aspect in this category, which we didn't really get the chance to discuss in the paper, is that of solution transfer between subsequent meshes, e.g. through interpolation which introduces artificial diffusion and which gets exacerbated by "unnecessary" mesh changes. Another approach, the so called Arbitrary Lagrangian Eulerian (ALE) method, does not interpolate the solution but allows the solution to "move with the mesh" which is accommodated by evaluating the PDE in a moving reference frame defined by the mesh velocity derived from the change  between subsequent meshes. This method can reduce both the amount of diffusion and costs associated with interpolation, but this is again limited by the amount of change in each timestep which can be related to a CFL-like condition. Focusing on this aspect of continuity between meshes there can indeed be a trade-off with the spatial, instantaneous mesh quality properties of each mesh individually. We believe this trade off should also be seen in light of the core assumption of mesh movement methods, which is that the topology of the mesh is unchanged. If flow features are followed slowly over time, the twisting of these features may lead to inevitable tangling of the mesh structure, even if large movements are allowed, which may only be resolved through complete remeshing, altering the connectivity of the mesh. The same problem arises with moving-boundary problems, e.g. the flow around rotating structures. For
future directions, we are therefore very much interested in hybrid approaches that combine r and h adaptivity, allowing for pure mesh movement in most timesteps but intervening with topological operation when the quality of the mesh degrades to much.

**Response to Q4:** There are no explicit terms during the training promoting mesh continuity other than our use of the Monge Ampere solution as the "optimal" reference mesh. The minimal change property is in a sense the defining feature of the Monge-Ampere approach, and using it to move meshes in time-dependent problems indeed results in smoothly varying meshes over time provided the changes in the monitor function are also smooth. Although there is no guarantee, we have observed the same smoothness properties and not seen any timestep-to-timestep back-and-forth changes that might be expected when approximating the movement induced by the monitor function.
Dear Reviewer xMNp,

Thanks again for the time and effort you've invested in reviewing our manuscript. We also greatly appreciate your acknowledgment of our efforts to address your concerns. 

Your insights have been instrumental in improving our work, and we are committed to incorporating your suggestions into the revision. We would also greatly appreciate your consideration in possibly raising the original rating if you find our responses satisfactory. If you believe there are any remaining areas where additional clarifications/responses could contribute to such a higher rating, please don't hesitate to inform us. Your guidance is always crucial to improve the quality of our submission.

Thank you very much, 

Authors
Dear Jzyt,

We sincerely thank you for your insightful comments with us! We commit to incorporate the new results in the revised paper.

Best regards,
Authors
Thanks for your clarification for my questions. Please include the new results in the revised paper.
I thank authors for their clear and instructive reply to all the issues I raised. 

Authors added new experiments and showed where their method fails and provides mesh tangling.

Also, they showed non-tangled results for flow-over-cylinder for the MA method. I believe this was not very informative when the proposed method seems to perform fine, while others completely fail (even if this fail is inherent). Authors found where this fail boundary lies and illustrated a more compelling comparison; this definitely improves the overall picture.
The authors have provided a comprehensive and thoughtful rebuttal, effectively addressing the primary concerns and questions raised about the UM2N architecture. They clarified their focus on mesh movement rather than generation and highlighted the challenges of comparing their work with the limited number of learning-based mesh movement methods available. The additional experiments conducted, especially on poor-quality initial meshes and irregular geometries, demonstrate UM2N’s robustness and versatility. Their detailed responses on the integration of Graph Transformer and Graph Attention Network (GAT) components, the role of element volume loss, and handling mesh deformations provide valuable insights into the technical strengths of their approach. However, as the field evolves, ongoing comparisons with new methods, further exploration of hyperparameter sensitivities, and expansion of real-world applications will strengthen the work’s impact and applicability.

Overall, the authors have made a strong case for the effectiveness and innovation of UM2N, particularly through their emphasis on zero-shot generalization and computational efficiency. The clear explanations and additional experiments help illustrate the model's capabilities in diverse scenarios. I appreciate the thoroughness and clarity of the paper, as well as the authors' proactive engagement with the feedback. The improvements and insightful discussions provided in the rebuttal demonstrate the paper's potential to advance simulation technologies through adaptive mesh movement.
1. In future revisions, could you provide more detailed quantitative metrics to highlight UM2N's performance, especially concerning mesh quality and computational efficiency?
2. Could you discuss any specific strategies you are considering for optimizing UM2N’s performance on large-scale problems, beyond memory-efficient transformers?
3. Given the trade-offs between mesh quality, continuity, and minimal movement, are there any specific scenarios where you prioritize one over the others? How do you envision addressing these trade-offs in future work?
4. Regarding the temporal continuity of meshes, how does UM2N ensure smooth transitions between time steps, especially in dynamic simulations? Are there any specific temporal regularization techniques or loss functions applied during training?
Thank you for providing detailed responses to my comments and questions. I appreciate your efforts to address the points raised and for conducting additional experiments to clarify the strengths and limitations of the Universal Mesh Movement Network (UM2N). Here are some additional thoughts.

**Comparison with State-of-the-art methods:**

I understand the challenge in comparing UM2N with a broader set of state-of-the-art methods due to the limited number of learning-based mesh movement approaches. Your focus on the M2N method and the confirmation from the Flow2Mesh paper about the scarcity of learning-based mesh movement methods is noted. However, as the field evolves, it might be beneficial to continually update your comparisons with new emerging methods. Exploring the literature for recent advancements and including these in future versions of the paper would be valuable.

**Ablation studies:**

Thank you for conducting additional ablation studies. The results provided in Table R1 and Figure R2 help elucidate the contributions of each network component. It would be interesting to see a more detailed analysis of how different hyperparameters impact the performance of UM2N, which could be explored in future work.

**Handling poor-quality meshes and irregular geometries:**

Your additional experiments on poor-quality initial meshes and irregular geometries, including the real-world tsunami case and stress tests, effectively demonstrate UM2N's robustness. The clarification that UM2N targets mesh movement rather than mesh generation is helpful. It’s commendable that UM2N performs well even with challenging input meshes. However, it would be beneficial to include detailed quantitative metrics in future revisions to further emphasize UM2N's capabilities.

**Computational efficiency and scalability:**

The scalability experiments and the comparison with the MA method are informative. It’s promising to hear about future work involving memory-efficient transformers for improved inference efficiency. Continued focus on optimizing computational performance and discussing potential bottlenecks will be crucial as UM2N scales to even larger and more complex problems.

**Mesh continuity and quality:**

Your explanation of the trade-offs between mesh quality, continuity, and the minimal movement property is insightful. It provides a clear rationale for UM2N's design choices. Future work could explore strategies for balancing these trade-offs, especially in scenarios requiring significant global redistributions.

**Optimal mesh configurations:**

I appreciate your discussion on the challenges of proving mesh optimality and the use of PDE solution error reduction rate as a metric. The references to high-resolution meshes and your detailed error analysis are compelling. Benchmarking against known optimal solutions or employing more sophisticated error metrics could further strengthen these comparisons in future work.

**Performance in boundary layers and turbulent flows:**

The experiments on flow past parallel plates provide valuable insights into UM2N’s capability to resolve boundary layers. Continued exploration of turbulent flow scenarios, along with detailed performance metrics, would further solidify UM2N’s applicability in critical fluid dynamics applications.