Better Neural PDE Solvers Through Data-Free Mesh Movers
Authors: Peiyan Hu
Yue Wang
Zhi-Ming Ma
Venue: ICLR 2024 poster (2024)
Keywords: neural PDE solvers
adaptive moving mesh
neural operators
Monge-Ampère equation
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: This paper introduces a neural-network-based mesh adapter called Data-free Mesh Mover (DMM), which is trained in a physics-informed data-free way. The DMM can be embedded into the neural PDE solver through proper architectural design, called MM-PDE.

Abstract: Recently, neural networks have been extensively employed to solve partial differential equations (PDEs) in physical system modeling. While major studies focus on learning system evolution on predefined static mesh discretizations, some methods utilize reinforcement learning or supervised learning techniques to create adaptive and dynamic meshes, due to the dynamic nature of these systems. However, these approaches face two primary challenges: (1) the need for expensive optimal mesh data, and (2) the change of the solution space's degree of freedom and topology during mesh refinement. To address these challenges, this paper proposes a neural PDE solver with a neural mesh adapter. To begin with, we introduce a novel data-free neural mesh adaptor, called Data-free Mesh Mover (DMM), with two main innovations. Firstly, it is an operator that maps the solution to adaptive meshes and is trained using the Monge-Ampère equation without optimal mesh data. Secondly, it dynamically changes the mesh by moving existing nodes rather than adding or deleting nodes and edges. Theoretical analysis shows that meshes generated by DMM have the lowest interpolation error bound. Based on DMM, to efficiently and accurately model dynamic systems, we develop a moving mesh based neural PDE solver (MM-PDE) that embeds the moving mesh with a two-branch architecture and a learnable interpolation framework to preserve information within the data. Empirical experiments demonstrate that our method generates suitable meshes and considerably enhances accuracy when modeling widely considered PDE systems. The code can be found at: https://github.com/Peiyannn/MM-PDE.git.

Published as a conference paper at ICLR 2024
BETTER NEURAL PDE SOLVERS THROUGH DATA- FREE MESH MOVERS
Peiyan Hu', Yue Wang"; Zhi-Ming Ma
AAcademy of Mathematics and Systems Science, Chinese Academy of Sciences, 2 Microsoff Research A14Science
hupeiyand 8 8@mailz 1s. ucas ac. en, yuwanggmmicrosoft..com ma zm22mmmtt ac. on
ABSTRACT
Recently, neural networks have been extensively employed solve partial dif- ferential equations (PDEs) in physical system modeling. While major studies fo- cus on learning system evolution on predefined static mesh discretizations, some methods utilize reinforcement learning or supervised learning techniques to cre- ate adaptive and dynamic meshes, due the dynamic nature of these systems. However, these approaches face two primary challenges: (I) the need for ex- pensive optimal mesh data, and (2) the change of the solution space's degree of freedom and topology during mesh refinement. To address these challenges, this paper proposes neural PDE solver with a neural mesh adapter. To begin with, we introduce a novel data-free neural mesh adaptor, called Data-free Mesh Mover (DMM), with two main innovations. Firstly, it is an operator that maps the solution to adaptive meshes and is trained using the Monge-Ampère equa- tion without optimal mesh data. Secondly, it dynamically changes the mesh by moving existing nodes rather than adding deleting nodes and edges. Theoret- ical analysis shows that meshes generated by DMM have the lowest interpola- tion error bound. Based on DMM, efficiently and accurately model dynamic systems, we develop a moving mesh based neural PDE solver (MM-PDE) that embeds the moving mesh with a two-branch architecture and learnable interpo- lation framework to preserve information within the data. Empirical experiments demonstrate that our method generates suitable meshes and considerably enhances accuracy when modeling widely considered PDE systems. The code can be found at: htttps::/ com /occooeeeeiiii yann
git.
INTRODUCTION
The simulation of physical phenomena is a popular research topic in many disciplines, ranging from weather forecasting (Schalkwijk et al., 2015), structural mechanics (Panthi et al.] (007) to turbu- lence modeling (Garnier et al. 2021). Meanwhile, due to the rapid development of deep learning techniques, there are emerging neural network based approaches designed for simulating physical systems (Li et al. 2020 Raissi et al. 2019, Brandstetter et al.) [022] Hu et al., [022]] Gong et al., 2023). Because of the complexity of such systems and the requirement of both high accuracy and low cost, learning of physical phenomena using neural networks is still challenging.
For physical systems, traditional solvers and most neural solvers need to represent the systems' states discretely on meshes. Since the system is often non-uniform, dynamic and multi-scale, uniform and static mesh will waste resources and lose accuracy. On the contrary, an adaptive and dynamic mesh can help allocate attention to different regions efficiently. There have been two meth- ods (Uthman & Askes 2005) including l-addapion that generates adaptive meshes by refining and coarsening cells, and r-adaptation or moving mesh method that relocates the positions of nodes. To accelerate the mesh adaptation process, deep learning techniques have been introduced to these two
categories of methods (Wu et al. 2023 Yang al. 2023 Song et al. 2022).
"Corresponding author.

---

Published as a conference paper at ICLR 2024
Nonetheless, several challenges exist for the existing approaches. The majority of traditional meth- ods are often conseminnnn particularly for dynamic systems with vire-ngrying optimal meshes. Deep learning based approaches, such reinforcement learning for learning h-adaptive meshes, exhibit low efficiency and pose difficulties in training (Wu et al. 2023 Yang et al., 2023). More- over, h-adaptation does not strictly maintain fixed number of nodes, leading to alterations in data structure and topology, complicated loaa-balancing, and abrupt changes in resolution (McRac et al.) 2018). Another deep learning method, based on supervised learning for r-adaptive meshes, relies on optimal mesh data generated by traditional numerical solvers and does not consider benefiting neural networks from moving meshes, which limits its applicability (Song et al.) 2022).
Targeted at these challenges in simulating dynamic systems with adaptive meshes, we propose a moving mesh based neural PDE solver (MM-PDE) whose kernel novel data-free neural mesh adaptor, named Data-free Mesh Mover (DMM). For the DMM, specifically, (1) to keep the degree of freedom of the numerical solution's 's space and avoid sharp resolution changes to promote stability, we let DMM generate r-adaptive meshes (moving meshes) so that the number of nodes and mesh topology are fixed. (2) To train DMM without supervised optimal mesh data accurately, we design physical loss and a corresponding sampling strategy motivated by the concept of monitor function and optimal-transport transport based Monge-Ampère method. Theoretical analysis of the interpolation error bound helps deduce the optimal form of monitor function which is crucial in the physical loss.
After obtaining the DMM, we propose MM-PDE, a graph neural network based PDE solver that relocates nodes according to the DMM. The goal is to learn the evolution while efficiently utilizing information in the data represented using the original mesh resolution. To achieve this, efficient message passing is needed both between two time steps and from the original mesh the moving mesh. For message passing between time steps, we leverage the DMM for efficient node allocation. For message passing from the original the moving mesh, we design a novel architecture with two key components. First, learnable interpolation framework that generates self- aaf--adating interpola- tion weights and incorporates a residual cut network. Second, the model consists of two parallel branches that pass messages on the original and moving meshes respectively, which improves per- formance over original mesh based GNNs.
Our main contributions are threefold: (i) We propose a neural mesh adapter DMM, which generates adaptive moving meshes and is trained in an unsupervised manner. Theoretical analysis indicates that resulting meshes minimize interpolation error. (ii) We design an end-to-end neural solver MM- PDE that adopts moving meshes from DMM to improve performance. To the best of our knowledge, it is the first moving mesh based neural solver. (iii) We conduct extensive experiments and ablation studies. Results provide strong evidence for the effectiveness and necessity of our approach.
RELATED WORKS
Neural solvers There are emerging deep-learning based numerical solvers recent years. There two main categories of methods: the physics-informed method and the data-driven method. The first one is trained with the constraint of physical loss to force the output to minimize residual of PDEs (Raissi al. 2019) or satisfy the variational principle (Yu al., 2018), which means that the specific form of PDES should be known. Conversely, the second type only requires data to learn the models. These neural operators approximate mappings between spaces of functions using neural networks, such as FNO (Li et al. 2020), MP-PDE (Brandstetter et al. 2022) and DeepONet (Lu ct al. 2021). However, these approaches do not take into account the meshes that serve as both carriers of data and the foundation for learning evolution.
Traditional mesh adaptation techniques The two most commonly deployed traditional adaptation techniques are h-adaptation and r-adaptation. h-adaption is also called adaptive mesh refinement, which means that mesh nodes are adaptively added or deleted (Belytschko & Tabbara 1993] [en- ito et al.] 2003). They suffer from certain implementation issues, such as the need for allowance for additional mesh points, and the constantly changing sparsity structure of various matrices used in calculations, which is especially essential in large computations. r-adaptation, or mesh move- ment, includes methods that move the mesh nodes and can avoid issues mentioned before. Some researchers use variational approaches to move the mesh points by solving associated coupled system of PDEs (Winslow) 1966; [Cniceros & Hou [001] Tang & Tang (003), while in recent years, methods based on optimal transport have gained increasing attention (Budd & Williams
2

---

Published as a conference paper at ICLR 2024
2006
Weller et al. 2016; Clare et al. 2022). The primary issue with traditional methods is the consuminn process of generating new meshes for each updated state.
AI & Mesh adaptation Considering the traditional mesh adaption methods quite consuming in time and computation, deep learning techniques have been combined with mesh adaptation in many ways. Many works employ graph neural networks to learn solutions on generated meshes. For example, MeshCraphNets adopt the sizing field methodology and a traditional algorithm MINIDISK generate refined meshes (Pfaff al. 2020). And LAMP takes the reinforcement algorithm to learn the policy of h-adaption, in which case meshes' topology changes unstably (Wu et al., 2023). In addition, some researchers just focus on the generation of mesh using supervised learning (Song et al., (022). They introduce the neural spline model and the graph attention network into their models. For improvement in both the generation and utilization of meshes, there exists considerable room.
PRELIMINARY
In this section, we introduce the definition of moving mesh. A mesh I can be defined the set of all its cells K. To enhance solvers' accuracy and efficiency, it is natural to adjust the resolution of the mesh in different regions. For moving mesh adaptation, this adjustment is repositioning each grid point of uniform mesh to a new location. Thus we can describe the moving mesh I as a coordinate transformation mapping [0, I] > l where T is the terminal time and Ω is the domain. To learn specific moving mesh, we need to approximate the corresponding mapping f.
We then introduce monitor function which controls the density of mesh. The monitor function M (z) is matrix-valued metric function on S2 and p /det (x))())))) is its corresponding mesh density function. The adaptive mesh moved according to M is termed M M-uniform mesh. A moving mesh or coordinate transformation mapping f is M Muuniform if and only if the integrals of p over all its cells are equal. This equidistribution principle can be mathematically described as
J& (((()))(())= K
a N
AKEg.
(1)
where N is the number of the cells of I and a = Sq p(æ)de is constant. Intuitively, the M- uniform mesh has larger mesh density the region with larger value of M.
However, for spaces with more than one dimension, the solution of Eq (1) is not unique, which means that there more than one way to move the mesh nodes according to M. To narrow down, we seek the map closest to the identity that satisfy Eq (.. From optimal transport theories (Huang & Russell 2010). the constraint problem is well posed, has a unique convex solution, and can be transformed into Monge-Ampère (MA) equation: det (Ø)) = [))(V.)) where ø is the potential function of the coordinate transformation function f, i.e. f(x) = V. and (()) = V the Hessian matrix of ó. The equation's ' boundary condition is V. in, which means the coordinate transformation maps each boundary to itself. Taking the 2-D case as an example, we can transform the boundary condition into
Φx n) 0, sb (E.0)
& (1, 1, by (E.1)
0,
To sum up, the target of seeking the M-uniform moving mesh is transformed solving MA equation under boundary condition (Huang & Russell (2100) Budd et al. 2013). To obtain the optimal way to move mesh nodes, namely the optimal monitor function, we need to analyze which M -uunforrm moving mesh corresponds to the lowest interpolation error, which is discussed in the next section.
CHOOSE MONITOR FUNCTION WITH THEORETIC GUARANTEE
In this section, will introduce the main theorem about how decide the optimal monitor function corresponding to the lowest interpolation error. The complete proof can be found in Appendix F
MonngeAAmèrr equation nonlincar second-order PDE of the form det dd f(z, u. Du).
3

---

Published as a conference paper at ICLR 2024
Suppose the interpolated function belongs to the Sobolev space (LP {f L'(2))
vlal & f E [.( where a = (a1..... Ota), aij = a1 +... + ad, and derivatives j = 021 .........f. are taken in weak sense. The error norm belongs to ( and (x( is the space of polynomials of degree less than +++1 on f2. According to traditional mathematical theories, the form of optimal monitor function corresponding to the lowest interpolation error bound is shown in the following proposition. Note, we consider the interpolation error in the Sobolev space rather than specific state u, so we deduce the interpolation error bound instead of a specific interpolation error value which is a commonly considered setting (Huang & Russell 2010 McRae et al., 2018 Hetmaniuk & Knupp 2007).
Proposition 1 H(Hung & Russell 2010) On mesh I every cell K, the optimal monitor function corresponding to the lowest error bound of interpolation from (12) to Px( is of the form
M = (1 + (w))))(K))
Jouil ml
AK E g.
where |-Iwm.r(K) = (Ejolm SK ( is the semi-norm of the Sobolev space WmM,(K), and ()) = (1/ (wm..(K)) is the scaled semi-norm.
From Eq (2). the optimal monitor function is proportional to (u) w so regions where deriva- tives are larger will have larger M and thus higher mesh densities. In other words, the mesh density is positively related to the level of activity. And in Eq (2) there is key intensity parameter that controls mesh movement's 's intensity. When increases, the mesh becomes more uniform. To decide a, since p is numerically calculated in practice and the error of approximating p non-negligible in this case, we consider this error to analyze the interpolation error bound of Mx with different a.
Taking this error into account, we then consider the reasonable choices of a motivated by traditional methods (Huang & Russell 2010)). Our first analysis is to take a + 0, in which case M is dominated by (1) (K)) For the second, a is taken such that a is bounded by a constant while M is invariant under a scaling transformation of u.
Theorem Suppose the error of approximating satisfies ||p عاا -+श(-m
(R) < 6. Also suppose
E (S)), S and the monitor function is chosen as in Proposition 1. Let IT, be the interpolation operator from Wi-P(K) to (K). For notational simplicity, we let
d+n(l-m)
B(K) = Σ | (1) WiPP(K)
(a) Choose a such that a - 0, if there exists a constant B such that ('(()(((())))))))()(()(())))(( 11 2 3 3>0 a.e. in S holds ensure that M is positive definite, then |u |/--lTa ---gu//mmm S CN (--) B(K) (b) Choose
both cases, as the number of cells N increases to infinity, the right-hand side of the inequality has the bound only relative to the semi-none of u as lim B(K)= I Clul
= [2 d+til-m)
B(K), then | TT (m) l
CNN 3 (1
From the theorem, we can
w "' (9)
obtain that in case (a), the error bound is not related to while increases the interpolation error in case (b). Combining the actual condition, because the error of approximating p is not ignorable when the mesh is not dense enough, choose small a to approach the first case. So the ultimate form of monitor function utilized is Eq (2) with small a.
A MOVING MESH BASED NEURAL SOLVER
After theoretic analysis of optimal moving meshes, in this section, will introduce the model architecture, physics loss and sampling strategy of Data-Free Mesh Mover (DMM), innovative ar- chitecture of MM-PDE to maintain original data's information, and the effective training framework.
5.1 DATA-FREE MESH MOVER (DMM)
To approximate the coordinate transformation mapping f W x [0, X + Ω we take neural operators with physics loss and special sampling strategy.
4

---

Published as a conference paper at ICLR 2024
C11
aICDORY
12
Itps
S2
Itps
C1
decoder
fou
(, encoder
(a) DMM
(b) MM-PDE
Model Architecture Summary: (a) Encodes state u and coordinates x)) using two net- works, followed by third network that processes their combined outputs. (b) MM-PDE has two branches: the bottom branch is a graph neural network for original meshes, and the top branch uses DMM-generated moving meshes for evolution learning. Interpolation of the discrete state onto mov- ing meshes is done with Itp), processed by a graph neural network, then re-interpolated to original meshes using Ilp2. Both interpolations involve weights from Ilp and 1pp.. A residual cut network ensures the preservation of previous state information during interpolation.
Figure 1:
Model architecture As the monitor function is related the state u, it is efficient and suitable to adopt neural operator that can generate moving meshes corresponding to different u. We follow and adjust the method proposed in DeepONet (Lu al., (22T) as shown in Figure [(a)] We divide the input into two parts: the state and coordinate (4, r.. The former can be treated as matrix (regular mesh) or graph (irregular mesh, e.g. triangular mesh), thus we encode with a CNN and a GNN respectively. For the coordinate, we encode it with an MLP. Different from adding the product of each dimension two outputs, since addition and first-order product are linear and do not have enough expressiveness, we concatenate two outputs and take another MLP decode it.
Physics loss The next thing is to design the physics loss for the data-free training. According to Section 3] the coordinate transformation mapping is the derivative of the Monge-Ampère equa- tion's s solution. Intuitively, since mesh nodes are relocated on the basis of the uniform mesh, learning nodes' movement relative to the uniform mesh is an easier task. Consequently, we let the output of the model approximate the residual v, which can be described as
V V - x f(z) - x.
(3)
Thus the Monge-Ampère equation Eq and the boundary condition turn to
det (Hb + 1o) = [9 p
(4)
(0 0,
(5)
where Io is the identity matrix. To learn s/. we design the physics loss as the weighted sum of the loss of Monge-Ampér equation Eq (4), the loss of the boundary condition Eq (5) and the loss of d's convex condition which will be discussed below: 1 = Lequation Bloound + Zeoveer: Here lequation
T) Ø||, hound = II x 1/ (+1))l, where and y weights to adjust the scale of different losses. For Leonvez, given that 6 is convex if and only if its Hessian is positive definite, suffices constrain the eigenvalues of ø's Hessian matrix to be nonnegative. In the 2-D case, eigenvalues of Hessian matrix H (Ø)) = (h.j 22x2 are
[2] (V + x))et(HH(()
h1 + h22 (h1 h22)² - det !
2
Because the rightside of MA equation is nonnegative, the constraint can be transformed to hu + h22 2 0, which is the same as leonvex. = min(0, 82, o}² + min 22 2)². According to the definition of de in Eq 3 H()) To H So the final form of looveer is Leoneex min + 02, 1))2 min
When calculating these losses, only the mesh density function P is caleulated using the finite dif- ference method since the input u is discrete, and all other derivatives are obtained using automatic gradient computation in Pytorch (Paszke et al. 2017).
5

---

Published as a conference paper at ICLR 2024
Sampling Because unsuitable sampling strategy will cause difficulty in optimization and prob- lems of getting stuck at trivial solutions (Daw et al. 2023). we design a sampling strategy specific to the Monge-Ampère equation Eq 4 Since the coordinate transformation goes dramatic with the increase of monitor function, the optimal mesh in regions of high monitor function is harder to learn and thus requires more attention. To achieve this, we let the probability of sampling points be proportional to the monitor function value, which makes the density of sampling points positively associated with the monitor function.
5.2 MM-PDE
After implementing DMM, We introduce MM-PDE, as shown in Figure ((b) We focus on efficient message passing which is critical for neural PDE solvers. Efficiency boosted in two ways: improv- ing message passing across time steps on a discrete graph with DMM, and employing a dual-branch, three-sub-netwondd architecture for transferring messages between original and moving meshes. This maintains data integrity in original meshes while integrating moving meshes.
Both two parallel branches serve to convey messages between time steps. The distinction lies in the fact that the message is transmitted through the original mesh and the moving mesh, respectively. This approach ensures that data within original meshes are preserved while the moving meshes are embedded. Consequently, the output of MM-PDE can be characterized as:
M (wx) (wx) ) + 1B2,, J, we),
(6)
where I1 and Lg are two graph neural networks, f is DMM, u, is the discrete representation of the state time on the original uniform mesh, is the interpolation framework. The process of the second term is first mapping states on uniform meshes to moving meshes generated by f. then evolving the system on moving meshes to the next timestep with GNN, and finally mapping states back uniform meshes. Every node is set to pass messages to its nearest & neighbors. Since meshes are moved according to the monitor function positively related to derivatives, information passing in dynamic regions will be promoted.
The interpolation network 7 consists of three networks: two, ltp1 and Ttp2, for adaptive interpola- tion weight generation for transitions between mesh types, and a residual cut network for previous state information transfer. Itpi and Itp₂ generate weights for the nearest & neighbors during these anterpolations:/tpp (na.. 2)) = (Wi1, Wak), i = 1,2, where m,,, 7iik are coor- dinates of x's is nearest k neighbors. The output of interpolation with Itp: on point is then the
weighted sum Σ}=1 wyyu(ni)) Besides, remedy the loss of information, we introduce a residual network to carry the information of previous states, whose output is added to the interpolation result of Itp2-
5.3 TRAINING
The entire training contains two stages: training of DMM and MM-PDE. For the DMM, because of the complexity of solving a series of second order PDEs with the Hessian matrix's 's determinant and the requirement of accuracy for downstream tasks, we specially design the optimization algorithm.
We also notice the high memory consumption and alleviate it. For the second stage, we introduce the pretraining of the interpolation framework to enhance the efficiency of message passing.
In the first stage, to enhance the optimization, after optimization with Adam optimizer, we take a traditional and more accurate optimizer BFGS. However, a key challenge we encounter during op- timization is the issue of memory consumption. This is due to the neural operator requiring training on numerous collocation points across different states, with the entire state also being fed into the model. To address this issue, firstly, input batch of points alongside each state. This approach effectively reduces the number of inputs for every state, consequently lowering the total memory consumption. Secondly, we use BFGS optimizer only optimize the last layer of the decoder, which also achieves great performance. Note, the memory of the A100 we use is insufficient when considering parameters of the last two layers simultaneously.
Next, MM-PDE is trained while parameters of DMM are fixed. To further enhance the accuracy of interpolation, the interpolation network framework 7 is first pretrained states in the training set.

---

Published as a conference paper at ICLR 2024
Table 1: Results of Burgers' equation and flow around cylinder.
Meshes from DMM on flow around
(a) Meshes from DMM the Burgers' equation.
cylinder.
ORIGINAL UNIFORM
MESH
ORIGINAL UNIFORM
MESH
METRIC
FROM
METRIC
FROM
MESH
DMM
MESH
DMM
std
0.0136
0.0094
std
0.1027
0.0469
range
0.7524
0.2061
range
0.1157
0.0733
(d) MM-PDE and baselines
(b) MM-PDE and baselines on the Burgers'
flow around a cylinder.
equation.
MODEL
ERROR
MODEL
ERROR
TIME(s)
MM-PDE
0.0846
MM-PDE
1.04e-05
0.5192
GNN
0.2892
GNN
3.10e-05
0.3078
bigger GNN
0.1548
bigger GNN
1.19e-05
0.3298
CNN
CNN
1.26e-05
0.0027
FNO
FNO
1.18e-05
0.0159
LAMP
0.4040
LAMP
1.6le-05
1.4598
Geo-FNO
0.2844
In this process. we map every state on the original mesh to its corresponding moving mesh and then back to the original without evolution to the next state. The pretraining loss can be described as
pre MSE(I(9) f. u), u),
where S, is the fixed identity mapping. MSF is the mean squared error. Then the training loss is the same as common autogressive methods, which is the error between predicted states and groundtruth.
EXPERIMENTS
In the experiments, we evaluate our method on two datasets. The first a time dependent Burgers' equation, and the second is the flow around a circular cylinder. Our target is to verify that our method can generate and utilize moving meshes reasonably, and is superior in accuracy, Also, to further demonstrate the effectiveness of components in MM-PDE and discuss other possible training patterns of the mesh mover, we do ablation studies to evaluate other variants of MM-PDE.
Baselines We compare MM-PDE with GNN (Brandstetter et al. 2022 bigger GNN, CNN, FNO (Li et al. 2020) and LAMP (Wu et al.) (2023) under mean squared error, where GNN is the same the GNN component in MM-PDE, and bigger GNN is deepened and widened to have close number of parameters to MM-PDE. As LAMP adjusts vertexes' number dynamically, we keep almost fixed make the computation cost the same others.
Evaluation metric To provide quantitative results, we design some evaluation metrics. For DMM, shown in Eq ( the optimal moving mesh is the satisfying the equidistribution principle under optimal monitor function M, which means mesh movement uniformizes cells' volumes under M. Thus we calculate the standard deviation (std) and range of all cells' volumes. As for MM-PDE,
we choose MSE referring to previous works (Brandstetter et al. 2022 Wu et al. 2023).
6.1 BURGERS' EQUATION
We first consider the 2-D Burgers' equation, which has wide applications in fields related to fluids.
Ou. Øt
(u V)u uv 0, (t,z,) [0, Ω.
The DMM has parameter count of 1222738, with training time of 7.52 hours and memory usage of 15766MB. The results are reported in Table Compared with original uniform meshes, meshes from DMM have much lower standard deviation and range, which means the volumes of

---

Published as a conference paper at ICLR 2024
cells under M more even. So DMM moves meshes to satisfy the equidistribution condition, which is equivalent to the optimal moving mesh's definition.
From Table I(b) method shows the lowest error and has comparable inference time to the base model MP- PDE, though longer than CNN and FNO due to its architecture. Compared to GNN, moving meshes notably boost performance. Increasing GNN's depth and width only marginally improves it, indicating our performance gains aren't 't just from more parameters. Our moving mesh method and physics-infone -nnfiss- training outperform LAMP's mesh refinement and RL.
6.2 FLOW AROUND CYLINDER
The next case is the simulation of real physical system, and has more complicated dynamics than the first experiment. We test the models on the flow around a circular cylinder.
The DMM has 2089938 parameters, with a training time of 22.91 hours and memory usage of 73088MB. Due to the irregular shape the original mesh, FNO and CNN are not applicable. We thus compare with Geo-FNO (Li et al. 2022). As shown in Table T(c] because the std and range of cells' volumes after DMM's movement are greatly lower than the original, our proposed method for moving meshes significantly uniformize the volumes of cells under metric M. The visual- izations of moving meshes and rollout results of MM-PDE are shown in Appendix E. and E.2
From Table [(d] and I MM-PDE has the best performance in simulation of a real and complex physical system as well. Compared with GNN and bigger GNN, the embedding of DMM consider- ably improves accuracy and this improvement is not brought by increasing the parameters' number. Also, the mesh mover trained with the Monge-Ampèr equation shows progress over LAMP's mesh refinement by reinforcement learning. Relative MSE results reported in Appendix E.
6.3 3-D GS EQUATION
Finally, we extend DMM and MM-PDE to 3-D scenarios to demonstrate the universality of our method. We consider the 3-D Gray-Scott (GS) reaction-diffusion equation with periodic boundaries The results are reported in Appendix suggesting MM-PDE's scalability.
6.4 ABLATION STUDY
To further discuss advantages of our proposed method, we provide results of 8 ablation studies.
Physics loss optimization Table 2 reports physics loss of DMM on the Burgers' equation and flow around a cylinder. From the tables, physics loss decreases substantially, and is lower after BFGS optimizer's 's optimization.
Table Physics loss for DMM hefore and after optimization using the Adam and BFGS optimizers
(a) Burgers'
equation.
(b) Flow around a cylinder.
LOSS
INIT
ADAM
BFGS
LOSS
INIT
ADAM
BFGS
28.248
0.045
0.026
0.881
0.101
0.095
lequation
28.209
0.044
0.023
equation
0.547
0.095
0.093
hound
4.94e
1.07e-06
2.80e-06
bound
3.34e
5.99e-6
1.54e-6
lconvex
2.47e-7
0.0
0.0
Loonvez
0.0
0.0
0.0
Resolution transformation (Appendix C.1 A DMM's advantage is its ability to generate meshes of varying resolutions during inference. But due to the selected encoder I's inability to handle vary- ing resolutions, this requires interpolation prior to inputting states of different resolutions. We assess bicubic interpolation and the method in Eq [6] with training data at 48 48 and 2521 resolutions. The meshes from the pre-trained DMM illustrated in Figure 2 and associated quantitative results are Table and g The data, regardless of resolution, adheres well to the MA equation.
Mesh tangling and lconvex (Appendix C.2 When the Jacobian determinant of is negative, mov- ing meshes will encounter the issue of mesh tangling. where edges become intertwined. As we use meshes solely to enhance neural solvers, this issue does not pose problem in context. But
8

---

Published as a conference paper at ICLR 2024
explore it for future potential expansions of DMM's applications. Constraining the Jacobian determinant of f to be non-negative is equivalent to requiring that & is convex, which is already constrained by Loonnes. The trend of Looneer during training is illustrated in Figure 3 showing that can reach a very low level early in the training process. Additionally, we find that the generated grid points of test data has cconver O. These demonstrate that this constraint is well met.
Effects of each component of MM-PDE (Appendix C.4) To show the effects of each component of MM-PDE, we delete or replace some parts. We select 'no 9 '9 +9', 'no Residual' and 'uniform mesh' to compare with MM-PDE. As shown in Table | well-defined and suitable moving mesh crucial for maintaining accuracy. And our introduced ways to preserve information, including the branch 91 on original meshes and the residual cut network, are effective.
Roles of three losses within the physies loss (Appendix the three losses within the physics loss by factor of ten for training. Table [ displays the std and range of meshes generated by new DMMs. When the weight of Lequation is decreased, the std and range increase the most, suggesting that the MA equation is not adequately satisfied. The result does not deviate significantly from the original when the weight of leonvez decreased, as leonvez can already reach a low level as stated in Appendix C.2 Figure 4 shows the plotted meshes of decreased Ibound, implying that boundary conditions are not as well satisfied as before.
We respectively reduce weights of
Table 3: Results of different loss weights. The three weights are respectively weights of leguations Abound and loonvez, and (1,1000,1) is the original weight.
METRIC ORIGINAL UNIFORM MESH
(1,1000,1)
(1,100,1)
(1,1000.0.))
(0.1,
std
0.103
0.047
0.048
0.047
0.054
range
0.752
0.206
0.239
0.225
0.265
Relationship between std, range and MM-PDE To further explore the relationship between std and range of moving meshes and MM-PDE's performance, we introduce a weighted combination of the identity map /r and f to replace Specifically, M (wx) = 91 (wt) + I(S,, af + (1 - -a)aa, u.). Table 4 shows the monotonic relation between std, range and MM-PDE.
Table 4: Relationship between std, range and error. Specifically, f of MM-PDE af + (1 a)fi-
std
range
ERROR
0.8
0.0581
0.3432
6.1e4--7
0.6
0.0708
0.4685
8.714e-7
0.4
0.0826
0.5695
9.209e-7
0.2
0.0939
0.6636
1.224c-6
MM-PDE with MGN (Appendix A
We conduct tests on the Burgers' equation using the Mesh Graph Net the GNN component, and additionally introduced the Mesh Graph Net baseline (Plaff et al. 2020). Results in Table 5 shows that our methods still works.
Training ways of DMM (Appendix C.5) As DMM is the core component of MM-PDE, we also plore the influence of training ways of MM-PDE's moving mesh component. We consider learning DMM only when training MM-PDE (1. end2end) and setting trained DMM also learnable during training of MM-PDE (2. mm-tenZend)). Results in Table [0 display that mm-tendZend achieves the lowest error, implying DMM trained with the physics loss nice initial training state.
7 CONCLUSION AND DISCUSSION
In this paper, we introduce the Data-free Mesh Mover (DMM) for learning moving meshes without labeled data, choosing the monitor function through theoretical analysis. We then present MM- PDE, a neural PDE solver using DMM, with moving and original meshes for information transfer and a neural interpolation architecture. We note that the current monitor function, based on the state at time t, may not optimally capture the variation of future states. Future work could consider how to incorporate future state information in DMM's design and training, possibly through a surrogate model predicting the next moment.

---

Published as a conference paper at ICLR 2024
REFERENCES
Ferran Alet, Adarsh K. Jeewajee, Maria Bauza, Alberto Rodriguez, Tomas Lozano-Pere -ernz, and Leslie Pack Kaelbling. Graph element networks: adaptive, structured computation and memory, 2019.
Ted Belytschko and M Tabbara. H-adaptive finite element methods for dynamic problems, with emphasis on localization. International Journal for Numerical Methods in Engineering, 36(24): 4245-4265 1993.
J.J. Benito, F. Ureña, L.. Gavete, and R. Alvarez. An h-adaptive method the generalized finite differences. Computer Methods in Applied Mechanics and Engineering, 192(5):735-75) 759, 2003. ISSN 0045-7825. doi: htt.:/occ...........o..ocoooa/aa
URL https: //www.
Johannes Brandstetter, Daniel Worrall, and Max Welling. Message passing neural pde solvers. arXiv preprint ar arXhv:2202.03377 2022.
Chris J Budd and JF Williams. Parabolic monge-ampeen methods for blow-up problems in several spatial dimensions. Journal of Physics A: Mathematical and General, 39(19):5425, 2006.
Chris J Budd, MJP Cullen, and EJ Walsh. Longe-ampére based moving mesh methods for numeri- cal weather prediction, with applications to the eady problem. Journal of Computational Physics, 236:247-270, 2013.
Hector Ceniceros and Thomas Y Hou. An efficient dynamically adaptive mesh for potentially singular solutions. Journal of Computational Physics, 172(2):609-633) 2001.
Philippe G Ciarlet. The finite element method for elliptic problems. SIAM, 2002.
Mariana CA Clare, Joseph G Wallwork, Stephan C Kramer, Hilary Weller, Colin J Cotter, and Matthew D Piggott. Multi-scale mydho-morphodynamic modelling using mesh movement meth- ods. GEM-International Journal on Geomathematics, 13(1):2, 2022.
Arka Daw, Jie Bu, Sifan Wang, Paris Perdikaris, and Anuj Karpatne. Mitigating propagation failures in
neural networks using retain raaiin-rsamlle-reeesse (r3) sampling, 2023.
Paul Garnier, Jonathan Viquerat, Jean Rabault, Aurélien Larcher, Alexander Kuhnle, and Elie Hachem. A review on deep reinforcement learning for fluid mechanics. Computers & Fluids, 225: 2021.
Shiqi Gong, Peiyan Hu, Qi Meng. Yue Wang. Rongchan Zhu, Bingguang Chen, Zhiming Ma, Hao Ni, and Tie- Liu. Deep latent regularity network for modeling stochastic partial differential equations. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp- 7740-7747, 2023.
Ulrich Hetmaniuk and Patrick Knupp. An An r-adaptive mesh optimization algorithm based on interpo- lation error bounds. Technical report, Sandia National Lab(SNL- Albuquerque, NM (United States), 2007.
Philipp Holl, Vladlen Koltun, and Nils Thuerey. Learning control pdes with differentiable physics, 2020.
Peiyan Hu, Qi Meng, Bingguang Chen, Shiqi Gong, Yue Wang, Wei Chen, Rongchan Zhu, Zhi- Ming Ma, and Tic- Yan Liu. Neural operator with regularity structure for modeling dynamics driven by spdes. arXiv preprint arXiv:2204.00225. 2022.
Weizhang Huang and Robert D Russell. Adaptive moving mesh methods, volume 174. Springer Science & Business Media, 2010.
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, An- drew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. arXiv preprint 100.0:20 2020.
10

---

Published as a conference paper at ICLR 2024
Zongyi Li, Daniel Zhengyu Huang, Burigede Liu, and Anima Anandkumar. Fourier neural operator with learned deformations for pdes on general geometries, 2022.
Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature machine intelligence, 3(3):218-22) 2021.
Andrew TT McRae, Colin J Cotter, and Chris J Budd. Optimal-tanssoot-bassse mesh adaptivity on the plane and sphere using finite elements. SIAM Journal on Scientific Computing, 40(2): A1121-A1148 A144 2018.
S.K. Panthi, N. Ramakrishnan, K.K. Pathak, and J.S. Chouhan. An analysis of springback in sheet metal bending using finite element method (fem). Journal of Materials Processing Tech- nology, 186(1):120-12) 124, 2007. ISSN 0924-0136. doi:
2006. 12.026. URL https://wwet sci
rect. wwt.://ewweeeeeeeeo.eeooeeeeeeee
50924013606U0 666
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
Tobias Pfaff, Meire Fortunato, Alvaro Sanchez-Gonzaler and Peter W Battaglia. Learning mesh- based simulation with graph networks. arXiv preprint arXiv:2010.034009 2020.
Maziar Raissi, Paris Perdikaris, and George Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686-707, 2019.
Pu Ren, Chengping Rao, Yang Liu, Zihan Ma, Qi Wang, Jian-Xun Wang, and Hao Sun. Physr: Physics-informed deep super-resolution for spatiotemporal data. Journal of Compu- tational Physics, 492: 12438, 2023. ISSN 0021-9991. doi: https://doi.org/1010000////
2023.112438. URL https: http://wwwwww..ccce rect. htt///cieennee/tttieee////....
SD021999123005338
Jerôme Schalkwijk, Harmen JJ Jonker, A Pier Siebesma, and Erik Van Meijgaard. Weather forecast- ing using gau-based large-eddy simulations. Bulletin of the American Meteorological Society, 96 (5):715-723, 2015.
M. W. Scroggs, J. S. Dokken, C. N. Richardson, and G. N. Wells. Construction of arbitrary order finite element degree-of of-freedomm maps on polygonal and polyhedral cell meshes. ACM Transac- tions on Mathematical Software, 48(2):18:1 18:1: 1::1:88:111:::: 18:233, 2022. doi: 10. 10.1145/352445.
Wenbin Song, Mingrui Zhang, Joseph G Wallwork, Junpeng Gao, Zheng Tian, Fanglei Sun, Matthew Piggott, Junqing Chen, Zuoqiang Shi, Xiang Chen, al. M2n: mesh movement net- works for pde solvers. Advances in Neural Information Processing Systems, 35:7199 72100 2022.
Huazhong Tang and Tao Tang. Adaptive mesh methods for one-and two-dimensional hyperbolic conservation laws. SIAM Journal on Numerical Analysis, 41(2):487-515 2003.
Zana Uthman and Harm Askes. R-adaptive and h-adaptive mesh optimisation based on eиror assess- ment. 13th ACME conference: Department of Civil and Structural Engineering, University of Sheffield, 2005.
Hilary Weller, Philip Browne, Chris Budd, and Mike Cullen. Mesh adaptation the sphere ing optimal transport and the numerical solution of a monge-ampère type equation. Journal of Computational Physics, 108:102-1253 2016.
Alan M Winslow. Numerical solution of the quasilinear poisson equation in a nonuniform triangle mesh. Journal of computational physics, 1(2):149-172, 1966.
Tailin Wu, Takashi Maruyama, Qingqing Zhao, Gordon Wetzstein, and Jure Leskovec. Learning controllable adaptive simulation for multi-resolution physics. arXiv preprint arXiv:2305.01122 2023.
11

---

Published as a conference paper at ICLR 2024
Jiachen Yang, Tarik Dzanic, Brenden Petersen, Jun Kudo, Ketan Mittal, Vladimir Tomov, Jean- Sylvain Camier, Tuo Zhao, Hongyuan Zha, Tzanio Kolev, et al. Reinforcement learning for adaptive mesh refinement. In International Conference Artificial Intelligence and Statistics, pp- 5997-6014. PMLR, 2023.
Bing Yu et al. The deep ritz method: deep learning- -earning-based numerical algorithm for solving varia- tional problems. Communications in Mathematics and Statistics, 6(1):1-12, 2018.
A EXPERIMENT SETTINGS
We first consider the 2-D Burgers' equation, which has wide applications in fields related fluids.
du Ōt
+ (u V)u vv n =0 0, (1,21) E [0, x Ω2, (t, 0, = ult, 1, x22, utt,z110) = u(t, x1, 1), 00,,71, no (21, 22), x E Ω
(Periodic BC),
where u is the velocity field, 0.1 is the viscosity and uo exp(-100(z; a)² 100((21 (1 3))²+ (x2 (1- ())3)) is the initial condition where a and are uniformly sampled in [0, 1].
The equation is defined on Ω [0, 1] [0, 1] and the length of trajectories is T 29s. To ensure the accuracy of the numerical solution, we solve the equation numerically on the uniform 192 x 192 grid with 30 timesteps using simulation toolkit (Holl al.) (2020), and one timestep corresponds to one second. When training and testing the models, we downsample the solution to 48 x 48. All the models autogressively predict the solution, taking one timestep input.
Table 5: MM-PDE combined with the Mesh Graph Net.
MODEL
ERROR
TIME(s)
MM-PDE(MGN)
2.13e-05
0.1187
MGN
2.46e-05
0,0400
The next case is the simulation of real physical system, and has more complicated dynamics than the first experiment. We test the models on the flow around circular cylinder. More precisely, circular cylinder of radius r 0.04 centered at (0.06, 0.25) replaced in an incompressible and inviscid flow in the [0, 0.5) x [0, 0.5] square. The viscosity coefficient v is set to be 10
We use FEniCS (Scroggs al. 2022), finite element computing toolkit, to generate the velocity field based on the Navier-Stokes equation. The solver rolls out 40 timesteps with di 0. on 2521 1 triangular lattices. Before 0.9s, the flow is under a constant external force ʃ with magnitude randomly sampled in 20 20). Then horizontal velocity after 0.9s are used as training and test sets.
The third case the 3-D Gray-Scott (GS) reaction-diffusico equation with periodic boundaries and generate data using the high-order finite difference (FD) method as referenced in prior literature (Ren ct al. 2023). The data are then downsampled uniformly to a 24 x 24 x 24 grid with dx = 100/24, spanning across 11 time steps with dt 20. As the solution of 3-D GS equation is in the form of (x y, 2) = (uy (z. y, 2), (z(2, y, 2)), we take the first dimension of solutions as training and test data.
B 3-D GS EQUATION
In this section, report the results 3-D GS reaction-diffusion equation in Table 6 From this table, can be observed that our proposed model architecture can be extended to 3-dimensional equations. Furthermore, it outperforms the baselines, demonstrating excellent performance.
12

---

Published as a conference paper at ICLR 2024
Table 6: Results of the 3-D GS equation.
MODEL MM-PDE
MSE
1.852e-04
GNN
2.106e-0k
C ABLATION STUDY
C.I RESOLUTION TRANSFORMATION
In this section, we further demonstrate the effectiveness of method in resolution transformation. One advantage of DMM is that, after training, it can generate meshes of any resolution during the inference stage. The only change needed is that, since the encoder we select encode states cannot handle data of any resolution, when we input states of other resolutions, we first need to interpolate them. We evaluate two kinds of interpolation: bicubic interpolation for the Burgers' equation and the interpolation method in Eq 16 for the flow around a cylinder. The meshes generated by previously trained DMM are reported in Figure 2 The quantitative results obtained are shown Table] 7 and where the training data of DMM is of 48 x 48 and 2521 resolution. It displays that data of no matter lower or higher resolution can still satisfy the MA equation well.
Table 7: Test on data of different resolutions on the Burgers' equation.
Metric
Uniform 24 x 24 Mesh 24 x 24 Mesh from DMM
std 0,3315
0.1596
range 1.6879
0.7055
Metric
Uniform 96 x 96 Mesh
96 96 Mesh from DMM
std 0.0304
0.0136
range 0.3273
0.0858
Table 8: Test on data of different resolutions on the flow around a cylinder.
Metric
Original 2521 Mesh
3286 Mesh from DMM
std
0.0112
0.0077
range 0.1111
0.0665
Metrie
Original 2521 Mesh
1938
Mesh from DMM
std
0.0163
0.0111
range
1488
0.0833
C.2 MESH TANGLING AND Leovvez
Moving meshes can sometimes encounter the issue of mesh tangling, where the edges of different meshes become intertwined. This situation corresponds to cases where the Jacobian determinant negative. Although this issue does not pose a problem in
of the coordinate transformation our context we are using the meshes generated by DMM solely to enhance the performance of neural solvers, we nevertheless explore this issue for future potential expansions of DMM's range of applications.
Constraining the Jacobian determinant f to be non-negative is equivalent to requiring that & is a convex function. This condition is already constrained by leonven in the physics loss of DMM. The trend of Lcovvez during the training process is illustrated in Figure showing that it can reach a very low level early in the training process. Additionally, test is carried out on the generated grid points of test data, resulting in an Looneer of This demonstrates that this constraint well met during the training process.
13

---

Published as a conference paper at ICLR 2024
t= Os
t= t=12s
t=24s
1.0
1.0
1.0
5000
4000
0.5
0.5
0.5
3000
2000
1000
0.0
0.0
0.0
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
t=0s
t=12s
t=24s
1.0
1.0
1.0
10000
8000
6000
0.5
0.5
0.5
4000
2000
0.0
0.0
0.0
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
1.0
600
500
500
500
400
400
0.5
300
0.5
300
300
200
100
100
0.0
0.0
0.0
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
1.0
600
1.0
500
500
500
400
400
300
300
200
200
200
100
0.0
0.0
0.5
1.0
0.0
0.5
1.0
0.6
0.5
1.0
Figure 2: Meshes generated by previously trained DMM with different resolution data. Four lines respectively correspond to 24 24 resolution, 96 96 resolution, 1938 resolution and 3286 resolu- tion.
C.3 ROLES THREE LOSSES WITHIN THE PHYSICS Loss
To investigate the roles of the three losses within the physics loss, we reduce their respective weights by factor of ten for training. Table 3 displays the std and range of meshes generated by DMM after training, where three weights are respectively weights of Ibounds Iconvez and lequations and (1,1000,1) is the original weight. It can be observed that when the weight of Lequation is decreased, the std and range increase the most. This suggests that the Monge-Amper (MA) equation is not adequately satisfied, implying that the volume of cells under M uniformly distributed. When
14

---

Published as a conference paper at ICLR 2024
1e-7
loss convex
25
loss of convexity
20
15
100
200
Figure 3: lconvez of the Burgers' equation during training. The horizontal axis represents the number of training epochs, while the vertical axis depicts the value of the loss.
the weight of Loonver is decreased, the result does deviate significantly from the original, as
leonver can already reach a very low level as stated in Appendix C.2 To observe the effect liounds
Figure 4 shows the plotted meshes. It is evident that when the weight of bounnd is decreased, the boundary conditions are not as well satisfied as before.
t=0s
t= 12s
t=24s
1.0
1.0
1.0
10000
8000
6000
0.5
0.5
0.5
4000
2000
0.0
0.0
0.5
1.0
0.0
0.5
1.0
0.0
t=0s
t=12s
t=24s
1.0
1.0
1.0
10000
8000
6000
0.5
0.5
0.5
4000
2000
0.0
0.0
0.0
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
Figure 4: Moving meshes of DMM with reduced weight of bsound (lop) and lequation (bottom).
C.4 VARIANTS OF MM-PDE
We respectively test the effectiveness of different components of MM-PDE. To test necessity of the two-branch architecture, the first variant is to delete the branch 9, on original meshes (1. no 91), resulting in (ux) 1 u.). The second is to delete the residual cut network (2. no Residual), which aims to prove its ability to maintain information in the interpolation framework. We also replace the branch moving meshes with another branch original meshes without the interpolation framework (3. 3 + 3)) to display the importance of moving meshes, whose output can be formulated M (ww) (u:) B2(u). And show the validity of meshes'
15

---

Published as a conference paper at ICLR 2024
Table 9: Results of variants of MM-PDE.
MODEL
ERROR
MM-PDE
6.63e-07
no I1
1.258e-06
91 + 92
8.852e-07
no Residual
8.467e-07
uniform mesh
1.324e-06
Table 10: Results corresponding different training of DMM
MODEL
HRROR
end2end
6.640e-07
mm+ten2end
3.447e-07
optimality, we replace moving mesh with uniform mesh in MM-PDE (4. uniform mesh). In this case. Al (u:) 9 (ux) ( fr, uc), where is the identity map.
As shown in Table 9 we can conclude that our introduced ways to preserve information, including the branch 31 on original meshes and the residual network, are effective. Besides, well-defined and suitable moving mesh is crucial for maintaining accuracy. The absence of moving meshes results decrease in accuracy, and the use of uniform meshes can bring enhancement.
C.5 TRAINING WAYS DMM
D EXTENDED RESULTS OF THE BURGERS' EQUATION
To let readers see clearly, we first present moving meshes of x 20 resolution generated by the DMM in Figure 5 And then provide a larger image of the mesh the 48 x 48 grid in Figure The DMM used to generate meshes in the main text is the same as the one used here.
1.0
8000
8000
0000
0000
2000
2000
0.0
0.0
0.0
0.5
0.0 0.5
0.0
1.0
0.5
3.0
0.0
0.5
0.0
0.5
1.0
Figure 5: Moving meshes of the 2-D Burgers' equation generated by DMM. The left three figures and the right three ones present moving meshes of x 20 resolution and monitor functions two trajectories at t 0, 12, 24s respectively.
Furthermore, we examine the quality of meshes when the value of the monitor function is extremely large near boundaries to verify the quality of the meshes generated by DMM, as this represents more complex scenario. Given that the Burgers' equation we select has periodic boundaries, we choose some states where the maximum value of the monitor function is relatively close to the boundary to generate meshes. Additionally, we generate new states with maximum values even closer boundaries and obtained the new mesh. These diagrams all displayed in Figure 7 demonstrating that DMM can also handle such situations.
E EXTENDED RESULTS OF FLOW AROUND A CYLINDER
VISUALIZATION OF ROLLOUT RESULTS
To more intuitively present the result of the experiment flow around cylinder, display the relative Mean Squared Error (RMSE) models Table IT, which shows relatively low values. Although the MSE appears large in Table T(d) this is due the overall large norm of the data.
16

---

Published as a conference paper at ICLR 2024
1.0
1.0
1.0
10000
8000
6000
0.5
0.5
0.5
4000
2000
0.0
0.0
0.0
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
1.0
1.0
1.0
10000
8000
6000
0.5
0.5
0.5
4000
2000
0.0
0.0
0.0
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
Figure 6: 48 48 moving meshes of the Burgers' equation generated by DMM. The bars the right present the correspondence between color and the value the monitor function.
1.0
1.0
1.0
0.5
0.5
0.5
0.0
0.0
0.0
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
Figure 7: Large monitor function near boundaries. The figures on the left are states from the original dataset, while the figure on the right represents a newly generated state with large M value closer the boundary.
Moreover, in Figure g we depict the real trajectory alongside the trajectory generated by MM-PDE. It can be observed that the data produced by MM-PDE aligns well with the real trajectory.
E.2 MOVING MESH VISUALIZATION
For flow around cylinder, present additional visualizations of moving meshes generated by DMM in Figure p The pictures show that the triangular cells' density is higher in regions with higher monitor functions, which verifies the equidistribution principle intuitively.
17

---

Published as a conference paper at ICLR 2024
Table 11: RMSE of flow around cylinder.
MODEL
RMSE
MM-PDE
0.0197
GNN
0.0649
bigger GNN
0.0349
CNN
FNO
LAMP
0.0693
Geo-FNO
0.0602
350
330
600
500
660
400
400
480
200
200
200
100
1.0
1.4
4co
500
350
900
0.5
05-
0.5
35-
300
369
200
100
0.1
3.0
6.0
0.0
05
1.5
0.3
0.3
0.0
05
1.0
00
455
0.3
0.5
Figure 8: Ground truth (top) and trajectories generated by MM-PDE (bottom).
F THEORY
In this section, we provide the proof of the proposition and theorem mentioned in Section 4 We first introduce three basic lemmas for preparation of proof (Huang & Russell 2010))
In the mathematical analysis below, every cell is defined as couple (K, Pk), where K is a mesh cell, P a finite-dimensional linear space of functions defined on K that are used to approximate the solution.
Lemma 1.1 (Ciarlet 2002) Let (K, Pk) be a cell associated with the simplicial reference element K. For some integers m, and (: 0 S m < 1 < k 1, and some numbers p, E [o],
w + W (K)
P&(K) PK C wm WK))(K)
where P&(K) is the space of polynomials of degree < Then, for any u W'P((K
Iu - Kk ((k)) S C'I((F() -IIm ||F | -lulwi.p())
(7)
where Hg : (K) + Px denotes the Px-interpolation operator on K, F&k is the Jacobian matrix of the affine mapping between two simplicial cells (K, Px), and (K, Pk), and the la matrix norm.
denotes
Lemma 1.2 For any matrix A E Rddd
= =AAAA)) = ||A112
where || || denotes the Frobenius matrix norm.
Lemma 1.3 An Munnformm mesh satisfies
1 7 [r(((Fk) )" "MxFF) Mrx VK E Tn.
-r((Fk)- MM (Fx) dct((F'k) -1 MM
VK E T..
18

---

Published as a conference paper at ICLR 2024
1.0
600
1.0
400
400
0.5
0.5
300
0.5
200
100
0.0
0.0
0.0
0.0
1.0
1.0
1.0
700
1.0
1.0
800
500
600
500
400
400
400
300
300
300
200
200
100
100
100
0.0
0.0
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
1.0
600
1.0
700
1.0
600
500
500
400
400
0.5
300
0.5
400
0.5
300
200
100
100
100
0.5
0.0
0.0
1.0
Figure 9: Moving meshes of the flow around cylinder generated by DMM. The bars the right present the correspondence between color and the value of the monitor function.
Next, give the proof of the Poposition which indicates the form optimal monitor function M. The monitor function has regularization parameter a. It is often referred to as the intensity parameter since it controls the level of intensity of mesh concentration. As a increases, the mesh becomes more uniform.
Proposition The optimal monitor funetion of the form
Mx = (1 [1+0 a (u)
d=g((-m)
VK E G,
PK (1 (w)
Ti my
YKE..
Proof:
Taking the q-th power on both sides of Eq 7 and summing over all of the cells, we obtain, for u W/. (S2)
Iu II
SC
K [1(F
| (1) (w)
K
where
(1) =
Σ
D p 2u) 12 dx
= -(xx) [ I1D'ull,, dx
K
[
K
de.....
Since
F S |Flll
19

---

Published as a conference paper at ICLR 2024
from Lemma 1.2 we have
| 12 S ir (FW)
[1(Fk)='|P sin (Tx))-
Thus Eq 9 can be rewritten as
m
tr (FF))
{ te (LFR)
Iu- 14 fym. (2)
S C
K
K
4
S SC> Σ |K|
[1 tr
[ (1FA)
x
(a+ (u)wio(k))
2 [7]
x ir ((F))()3)))
S Co Σ
|k| /rr
K
1+ (w)w)o(())
Denote
m
E(9)= - N dmmbn
1 tr (FFk) ((F)))))))))]]
d tr (FF))(F()
K
x
(9)
K d
(I+a (u)wio())
(10)
Then the task for finding the optimal monitor function becomes
minE(
From the arithmetic mean geometric-mentt meat inequality and (ek(F))) [K], we have
y ((F5))(((a)] #
(()
(t-maa
K
a tr
x
(1+a (w)mi())))))
N Mm-mj
|det ((Fz) 77)
[dott |dot ((FR) 11
|
x
K
(1+a +++=((())mo))))))
(1I)
mia
K 1+ ع-mسs
N
K
K| - (1 (1+a-1 -1 (u))))o()) Ji d-g(l-m) m)
N
N
K
dl
dy [+al-mm
dlull m!
| . (1 (1+a=* (4) (w)
N
(12)
N
dg ZTv( my
Σ K
KI (1+a-) (w)wine)))
Since the equality in
holds when
tr ((Tis) ((Fx) (F()) det (F(x)
1 2 (1T4) (T()) = det (T'))((())))
20

---

Published as a conference paper at ICLR 2024
from Lemma 1.3, M is of the form
Mr = 0
for some scalar function 0 = xx- The equidistribution condition implies that
px|K] C1, AK E g
for some constant C1. Noticing that the equality
holds when
K| (1 (1+66 a (u))).((K) गiतil mi C, AK E Th: for some constant C, we can get that
a1+a- (u)
vK Eg
MR (1 a (u)wtr(k))
AK
Then take the error of approximating p into consideration, since it can not be ignored in practice. And we provide the interpolation error bound involving the error of approximating p.
Theorem Choose monitor function Proposition
Suppose the error of approximating p
| د
(9)
--
then the interpolation error of the M-uniform mesh 9 is
(6m))
dto m)
d+gli- 44
Iu IT S CaN
a
(0 2g
where o = =xx |KpK-
Proof: In this case,
Pk pk| = C1,
Suppose <x =llpx -kklle -+q(-m)
(K) (AK E F), then
(MT( = N d+)((-m)
|K| (pK PK + pk)] dlvll
dleel ml
[[K] .pk PR) + Ci]
d+g((-m)
(KK) + Ci) dtal-m)
N
N
K
Because < m S and 0 S q, d, the exponent
d+g(I-m)
> 1. Consequently,
d
S oil mi
diaft ml
+ q(1 m) (KK) 6K + C2) eK
(M)
d+g(t-m)
< N d+n(l-))
1 N
Σ
Gi
K
dtqit-m)
d + q(l m)
m(-m)
+ N
([K) CK C1)
CK
N
d
K
d+o(t-m)
< N
d m)
Σ G
N
dtolll ml
11
(C2Ek) [-p((-m)
N
N
K
dttg(! (agg(-mm) m)
4t-(1-m
Σc
Ci
(+g(l-m))
Σ
(CZER) (+qul-m
K
K
dtall-mi ml
d+all-m)
Σ |K| (1+ a- a
(1) (w)
d+g(i-m)
+ C2
Σ
CK
Je mj
K
21

---

Published as a conference paper at ICLR 2024
Since
d+g(t-m)
Σ &
atali-m)
Σ K
1/ PK pxl
- da
d-
(L/ 10 [] 2 1
dx
||p -pllz
1+
c,
we finally have
(1-m)
d+ ml
d+aa m)
Iu (m) CaN
2g
4q
The theorem shows that the error bound increases as increases. Besides, there is trade-off between a and Ø. To examine how to choose a, we then provide some lemmas.
Lemma 2.1 For any real number t 1>0 0
la + bld v cz (1aa + (b19)
Va, b E I.
la|" co|ble S cxla blt
where
C = 2max((-1.0),
Moreover, if E (0, 1), then holds that
|a|l
|b||| S a bl" Va, ba E R.
Lemma 2.2 Suppose that the assumptions in Lemma 1.1 and Theorem hold,
difff- mm)
d-u(t-m)
1 | dtali-ms dx
Σ
| (1)
d-got-m (K)
(13)
as long as
h max hg as N oc,
where lig the diameter K.
Lemma 2.3 (Poincaré's inequality for a convex domain; un) Let Q Te bounded convex domain with diameter hg. Then, for any 1 S q S < 0c,
| uall u S ha
| Vullzr(()) Vu E W
where Gp.g is constant depending only on p and q which has values bounds follows:
<
Cp. S qt
2 2
1-1
p
(f(((11 ((((1= (z)
F-1
S $2 (f
x))= dx
1
101p=g-1,,.1-
2
1f) 2, 0,
22

---

Published as a conference paper at ICLR 2024
Lemma 2.4 For any real numbers (0, 1] and [1, oo) and any mesh I for !2
||e| - 1010> (K())
S M Mull (PP()) + Cyp (1 + Cyp
(P(
for any v E W.),(s),, where h maxke @ hx and the constants Cyp and Cp.p defined in Lemma 2.1 and Lemma 2.3, respectively.
Lemma 2.5 For any real numbers y (0, 1] and [1, oo) and any mesh I for ?
S Σ
|
|| (P(K) s |o] -
Vue E ( I|] [/]
K
Lemma 2.6 For M-uniform mesh J
hK S N
AK E g,
Vemin
where h = 2 241
and Xmin is the minimum eigenvalue of MK-
Vd-I
Two different choices of a are examined. The first choice is taking O, which means that M is dominated by (w) WL.P(K)- For the second choice, a taken such that a is bounded by a constant while keeping M invariant under scaling transformation of u.
Theorem Suppose that the assumptions in Lemma 1.1 hold and that p. Suppose also that the mesh density function and the monitor function are chosen as in Proposition
(a) Choose such that - 0, if there exists constant / such that
[')u(x)llo> 2 3 > O, a.e. in
holds to ensure that M is positive definite, then for E WL.P((2)
d+all- m dy
11 amm. (5)) I CN
(m-)
Σ K (2)
Zoadd my
(K)
with
dt-el mm)
de
lim
Σ
2+p [ (u) Wii(()))
Clul W w wha-to((-) (0)
N-00
further u. E WL+ then
Iu [luu)wme(())
< CN
m)
|ul af (52)
+ N=
|ul T:g |M (1/w++))(()))
((.p(2))
(b) Choose
d+ m)
1
Jruit Ki
(u)
Ω
then for E WLP(O),
d1 m))
Iu 1kll
I CN
Σ K
K (u) wi)
(1 (1+
m)
with
d4
lim
Σ
K
z000 - ( (1)(k)))
S Clu
de (9)
N=000
w/
23

---

Published as a conference paper at ICLR 2024
holding. If further W then
u [gu CN
(-m) W (00)
a
(Iu)
+ N (u/ (1 +
dbe(l-m
Proof: (a) Since the equidistribution and alignment conditions are invariant under a scaling transfor- mation of Mx, let
24 Mr (+q((-m) M (a + (u)wi>(K)) (+olt-m) I, AK E g,
so the corresponding mesh density function is
du PK a JFall m) PK = (a+ (u)wi.r())
(tgil-m
AK E T.
As d+](i-m) -0, 0,
11p p11
(52) - 0,
(+g(t-m
PK (u) Teqlt-m (K)
By the same argument as Proposition 1, we get
d+u((-m)
dg
|u [m.ul S CN
Σ
K (u) बाvi
(l)
(14)
From Lemma 2.5 and Lemma 2.6.
hx S CCN- Xmiin XmminMMx) 22
< CN--2--3
Σ | (u) d+gll-m)
(
K
CN 3
2 P
m
(15)
which means that this M-uniform mesh satisfies the condition of Lemma 2.2. And from E3 obvious that
d oe mc) 49
lim N0000
Σ K| (u)
d+) (K)
Clul
de
W
(0)
For u. WL)). this limit can be refined from Lemma 2.4 and Eq 15 to
d+g(t-m)
Σ K (u)
24g00-m m) (P)
K
d+g(-m) dg
Σ
K md--il- mi (u 2+g
d+-0--m
|u|| wil du mi + h a m) lul (10)
<
[u] 1 1. (0)
+ N-
[M
wir' (0)
(1
24

---

Published as a conference paper at ICLR 2024
Inserting this into Eq gives
3 11 (5I)
CN "mm² [u|] w ah (1ul (())
N /// Z
|u| [ulwi+1-((23)
(b) a S C' for some constant:
EiKI |K] [ +1aaala (())))))))))))
3 w. mm
Σ |K| 1+ a a=gli-m) (2) (
S c
[+]([-m]
C de (tm
[ + a 2+0lt-m)
K (u)
ml (K))
2
[2]
dg
d-t-ot--m: m
where constant
is defined as Lemma 2.1. From this and Theorem we get
d+ m))
dg
u 11 w (m)) S CN
Σ
K (u) WLP((R)
(1 (1+
s+g(t-m)
From the definition of a, [a)(a) (tit--)
| (1)
[+q(l-) (L)(K)
K| p(d+glimm) m)) |u| (+
(K)
K
Lemma 2.4 and the above equation give
|u| 2
$ (a) काता dg
W 20o0 (0)
SC
|u (+)
+ N 2न m) |u
w 2 mi (0)
Combining these with Theorem we obtain
[u - [Tuu/ymm S CN- ("--m)
+ N (o/ (1 + d+a(l-m)
[tc]
--qqt-m (02)
From the theorem, we can infer that the error bound decreases with the increase of N, and increases with the increase with the increase of 6 in case (b). Intuitively speaking, because the term that con-
d+v((t-))) as a + 0, e does not influence the error
tains e in Theorem I's error bound is CaN bound in case (a). Consequently, taking a + 0 is more suitable when the error of approximating negligglee
e
G MONITOR FUNCTION
G.1 FORM
We decide the index I, m. P.g q following the assumptions Proposition the interpolated function
E WlP(K) and error norm e E Wm.a(K) (K). Noticing that we treat the discrete input u as
piecewise constant function and we take MSE as the error norm, we choose = 1, m = 0, p = q =
2, so the monitor function for 2-D settings is
M (1 + \V)ll,//)'1.
As for a, we choose
Σ Kl (u)
100
to let a be as close to case (a) as possible and make M still invariant under a scaling transformation of The constant 100 is chosen with parameter tuning as shown in Appendix]] []
25

---

Published as a conference paper at ICLR 2024
G.2 CALCULATION
The form of the monitor function needs to take the derivative of the discrete u. For rectangular original mesh, we take the one-order finite difference. For the triangular mesh, first extrapolate u to get continuous function and then get the derivative with automatic gradient computation in
Pytorch (Paszke al. 2017). Adopting the way used in GEN (Alet al. 2019), we extrapolate (w)) as
u(v) = Σ Er(y,aiuf
(16)
over the domain Ω. r(x) = softmar(-Clll - =Vll) is a soft nearest neighbor representation func- tion, where C' is a hyper-parameetr. practice, is chosen as LVN].
H
MODEL ARCHITECTURE
H.1
DMM
Encoder 1 is used to encode state For the Burgers' equation, we take 2-D CNN for the regular rectangular mesh:
I input channel, 8 output channels, 5x5 kernel,
8 input channel, 16 output channels, 5x5 kernel,
16 input channel, 8 output channels, 5x5 kernel,
8 input channel, 5 output channels, 5x5 kernel,
followed by a flatten operation and two linear layers with latent dimensions 1024 and 512. Residual connections used between 2-D convolution layers. And the nonlinear activation function is the tangent function.
For flow around cylinder, the mesh triangular, we take the model architecture follows:
embedding MI.P: a linear layer from dimension 3 to latent dimension - 1-D hatch norm layer tangent activation function - linear layer from latent dimension to latent dimension 4 - I-D batch norm layer,
three message-passing layers with latent features and tangent activation function,
decoding MIPP: linear layers from dimension 4 128 1.
Encoder 2 is used to encode coordinates (t, and taken as MLP with tangent activation function. For the Burgers' equation, the layers' numbers of latent nodes are 32 and 512, while in the second experiment are 16 and 512.
The decoder is the for experiments. We take linear layers with tangent activation function from latent dimension 1024 to 512 and then to 1.
H.2 MM-PDE
MM-PDE's model architecture is the same for the settings. D, and 9g are the same referring to MP-PDE (Brandstetter et al., 2022). There is an MLP encoder, 6 message-passing layers, and an MLP decoder.
The encoder gives every node i a new node embedding, which is computed as n Encoder([u,; [1, 4)),
where uy is the solution's value node and x; is the space coordinate. Then the embedding is inputted into message-passing layers. Every layer has 128 hidden features, and includes a 2-layer edge update MLP EU and 2-layer node update MLP NU. For the m-th step of learned message passing, the computation process of the edge update network from node to node i is
m
EU (n." n.
Uj x.5)
26

---

Published as a conference paper at ICLR 2024
And the computation process of the node update network node is
m. = NU
n
my
(EN()
where M is the neighbors of node i. Here we pick the nearest K nodes of the node as its neighbors No and K is be 35, At last, shallow ID convolutional network is used, which shares weights across spatial locations.
In the framework 1, Itp1 and Itp2 have the same architecture: two linear layers with latent dimen- sions 128 and 64 respectively. The input of them is the concatenation of its position and its nearest K neighbors' positions, where K 30. Then its output is weights W1, u..... uK. and the final interpolation result is the weighted sum of neighbors' values. As for the residual cut network, it is 4-layer MLP with latent dimensions from 2048 to 512 to 2048 then to the original dimension.
I ADDITIONAL EXPERIMENT RESULTS
1.1 HYPERPARAMETER SEARCI
We conduct experiments for selecting optimal key hyperparameters. The results presented in Table 12 from which we choose 3 1000, Kitp = 30, Kmp = 35.
Hyperparameter search for the Burgers' equation. 3 is the weight of Isound in neural moving mesh's 's physics loss. Kitp and Kmp means the number of neighbors for interpolation and message passing used in MM-PDE respectively.
Table 12:
HYPERPARAMETER
ERROR
8 1000, Kitp 30, K mp = 35
6.63e-07
3 100, Kitp = 30, K Kmp = 35
1.02e-06
19 10000, Kitp 30, Kmp 35
1.01e-06
3 1000, Kitp 35, Kmp =35
1.23e-06
3 1000, Kitp 25, Kmp =
6.67c-07
9 1000, Kip 30, Kmp =35
2.51e-05
3 1000, Kip 30, Kmp 35
9.41e-07
I.2
CHOOSE a
As explained before, the form of is
K (u) (1.2()))))
C
To decide the value of hyperparameter C, we try C' = 10, 100, 1000 respectively. From Table L3 we finally choose C 100 whose error lowest.
Table 13: Choose the monitor function.
C
ERROR
10
7.14c-07
100
6.63e-07
1000
1.02c-06
27


Reviews:
[{"review_id": "L5IBkYwCyf", "reviewer": "Reviewer_qbBA", "summary": "The authors have introduced the Data-free Mesh Mover (DMM) as an innovative component for neural PDE solvers. DMM functions as a mesh adaptation solver, facilitating the adjustment of node positions within a uniform mesh. It's referred to as \"data-free\" because it achieves an optimal mesh configuration by solving the Monge-Ampere equation. The DMM is subsequently integrated with the MM-PDE (Message passing neural PDE solvers) to enhance overall accuracy.", "questions": "- My main concern is on the efficiency of DMM. As data-free, for each input solution $u$, training is needed. Then this can be very costly. Can authors compare the time if using the conventional PDE solver for the mesh adaptation? \n- For time-dependent problems, what is the input $u$? At the current time step or the next time step? The singularity of the solution could move. The mesh obtained for $u_k$ may not be suitable for $u_{k+1}$. \n- The error listed in Table 1 shows MM-PDE only improves from around $2 \\times 10^{-6}$ to $6.7 \\times 10^{-7}$ and in Table 2 is from $0.0258$ to $0.0141$ using a bigger GNN. Then a fair comparison is the computational time of MM-PDE and the bigger GNN to achieve the same error. A bigger GNN may need a larger model size, but since no DMM solver is involved, the overall time might be smaller. Again, this is related to Question 1: the efficiency of DMM.\n- Evaluation metric. Why not compute the interpolation error directly instead of an upper bound on the interpolation error?\n- \"Two-order\" should be corrected to \"second order.\"", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The application of adaptive meshes will improve the accuracy.", "weaknesses": "- The efficiency of DMM requires further comprehensive demonstration.\n- The contribution of this work appears to be incremental, as the core neural PDE solver used is essentially the message passing neural PDE solver developed by Brandstetter et al.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QMaPvl0GXP", "reviewer": "Reviewer_W4Uo", "summary": "The authors propose a mesh moving method to improve dynamics learning of graph neural networks. Theoretical understanding of the neural mesh adapter is provided, and empirical results are shown on two problems: the Burgers' equation and flow past cylinder, both in 2D.", "questions": "1. The implications of the ablation study are that having a trainable DMM performs better, but having it trainable also means now it is finetuning on the MSE loss from the solution data, hence the DMM equations may no longer hold. Did you quantify the std and range of cell volumes after this mm+end2end training as well?\n2. How are the DMM loss derivatives computed, using finite differences, or autodifferentiability?\n3. Is there any limitation in terms of extending the framework to 3D problems? Computational complexity?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "A detailed explanation was provided for the framework, with plenty of background information. The method applies well in practice, and the appendix provides enough technical detail on results in the main paper.", "weaknesses": "1. Runtimes should be mentioned for the proposed model, especially when comparing with equivalent methods such as GNN, CNN, FNO and LAMP. How much overhead does DMM require, and what number of extra trainable parameters are we talking about in practice?\n2. When training the DMM separately, how well is the physics loss being minimized? It is important to know to what extent the equations need to be satisfied until we see an improvement in MM-PDE.\n3. A sentence of explanation should be added when training the DMM, where we use a combination of both Adam and BFGS, but only finetune the last layer. Did this show the best performance? Why did Adam alone not suffice. When the high memory consumption is mentioned, what numbers is required by the two example problems shown in this paper?\n4. From a practitioner's perspective, it is unclear why the metric for cell volume matters for the examples, aside from showing that the DMM loss was minimized. Most importantly, we never perform any testing on the adapted/moved mesh, hence its structure can be arbitrary as long as it is conducive to the training of the GNN. It has been shown that uniform meshes as a representation perform worse, but is it the case that reducing the cell volume std and range will also monotonically decrease the test loss of the model? \n5. For the non-uniform grid example in flow past cylinder, CNN and FNO were not considered. Would the non-uniform equivalent of FNO, the Geometry-Aware FNO GeoFNO, be applicable as a baseline? If so, comparing with non-uniform state-of-the-art methods would strengthen the paper as well.  \n6. More details should be added on the experimental results, for example, for the flow past cylinder, is the MSE computed on just the velocity, or also pressures? The values reported in Table 2 seem relatively high then, perhaps visualization of the rollout results could also be added to the appendix.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9qgOUytLqz", "reviewer": "Reviewer_Uu92", "summary": "The authors proposed a mesh mover based PDE solver in this paper. The solver consists of two main components: a mesh mover and a neural PDE solver. The mesh mover is trained unsupervised based on a physical loss motivated by optimal-transport based Monge-Amp`ere mesh movement method. The GNN neural PDE solver is trained supervised given solutions from numerical simulations. The experiments show that the proposed method can outperform existing NN solvers on 2D Burger's equation and flow past cylinder cases.", "questions": "- The mesh tangling is a key challenge for mesh movement methods. It is hard to evaluate the performance of the proposed mesh mover without the evaluations on this potential issue. I would suggest that adding a solid discussion for mesh tangling will improve the soundness of this paper.\n- Regarding to the physical loss, the $loss_{equation}$ part is designed based on the difference between the current mesh and the uniform mesh. In a time-dependent problem, does it indicate the mesh mover always move the mesh nodes starting from the very initial uniform mesh without using the moved mesh in the last timestep? Does this strategy perform better comparing to reuse the previous moved mesh as the starting mesh? \n- An ablation study for contributions of each components in the physical loss ($loss_{equation}$, $loss_{bound}$ and $loss_{convex}$) will help evaluate its effectiveness.\n- Mesh movement methods may struggle with cases where the monitor functions give large values near boundaries, e.g. the peak values of the initial conditions of the burger's equation are located close to simulation domain boundary. It would be more convincing if there are such test cases for evaluating the proposed model. \n- It is mentioned in the paper that the mesh for Burger's is 48x48, however it is only 20x20 shown in Figure 2. Are there any missing results for the 48 x 48 resolution?\n\nMinors:\n- The DMM model architecture is not shown in Figure 2(a)", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "- The idea to design a neural PDE solver with mesh movement method is novel.\n- The Monge-Amp`ere equation motivated physics loss for mesh mover training is intuitive. \n- The use of BFGS for memory intensive training and the sampling method for Monge-Ampere equation are sound.\n- The background part is informative and help understand the proposed method.", "weaknesses": "- One essential challenge for r-adaptive (i.e., mesh movement) based mesh adaptive methods is mesh tangling. The mesh tangling happens when the mesh have elements with negative Jacobian-determinant, or in other words, the edges of mesh nodes come across each other after being moved. Although the optimal-transport based Monge-Amp`ere mesh movement method is designed to alleviate the mesh tangling issue, it is hard to guarantee tangling-free especially for a learned model, which only has a soft physical loss as the constraint. It is unclear how the proposed DMM perform regarding mesh tangling as there are no related evaluations in the paper.\n- The Monge-Amp`ere based physical loss is the key contribution of this paper, however it is not clear how each part of the loss i.e. $loss_{equation}$, $loss_{bound}$ and $loss_{convex}$ contribute to the model performance.\n- The experiments are limited to one fixed mesh resolution (48 x 48 or 2521 triangular lattices) and there are no evaluations on generalization of the proposed model, i.e., trained on a (small) resolution and test on other (potentially larger) resolution. Therefore, it is not clear how the proposed method perform given unseen data / boundary conditions/ physical parameters in equations. \n- The presentation and text of the paper should be improved.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)