Learning Mesh-Based Simulation with Graph Networks
Authors: Tobias Pfaff
Meire Fortunato
Alvaro Sanchez-Gonzalez
Peter Battaglia
Venue: ICLR 2021 Spotlight (2021)
Keywords: graph networks
simulation
mesh
physics

Abstract: Mesh-based simulations are central to modeling complex physical systems in many disciplines across science and engineering. Mesh representations support powerful numerical integration methods and their resolution can be adapted to strike favorable trade-offs between accuracy and efficiency. However, high-dimensional scientific simulations are very expensive to run, and solvers and parameters must often be tuned individually to each system studied.
Here we introduce MeshGraphNets, a framework for learning mesh-based simulations using graph neural networks. Our model can be trained to pass messages on a mesh graph and to adapt the mesh discretization during forward simulation. Our results show it can accurately predict the dynamics of a wide range of physical systems, including aerodynamics, structural mechanics, and cloth. The model's adaptivity supports learning resolution-independent dynamics and can scale to more complex state spaces at test time. Our method is also highly efficient, running 1-2 orders of magnitude faster than the simulation on which it is trained. Our approach broadens the range of problems on which neural network simulators can operate and promises to improve the efficiency of complex, scientific modeling tasks.

Published as a conference paper at ICLR 2021
LEARNING MESH-BASED SIMULATION WITH GRAPH NETWORKS
Tobias Pfaff; Meire Fortunato, Alvaro Sanchez-Gonzalez; Peter W. Battaglia
Deepmind, London, UK
(tpfaff, seeeffmmeizee meirefortunato a vaross, peterbattagl |ttp: com
ABSTRACT
Mesh-based simulations are central to modeling complex physical systems many disciplines across science and engineering. Mesh representations sup- port powerful numerical integration methods and their resolution can be adapted strike favorable trade-off's between accuracy and efficiency. However, high- dimensional scientific simulations are very expensive to run, and solvers and pa- rameters must often be tuned individually to each system studied. Here we intro- duce MESHGRAPHNETS, a framework for learning mesh- based simulations us- ing graph neural networks. Our model can be trained to pass messages graph and to adapt the mesh discretization during forward simulation. Our results show it accurately predict the dynamies of wide range of physical systems, including aerodynamics, structural mechanics, and cloth. The model's adaptivity supports learning resolution-independee independent dynamics and can scale more complex state spaces at test time. Our method is also highly efficient, running 1-2 orders of magnitude faster than the simulation on which it is trained. Our approach broad- ens the range of problems on which neural network simulators can operate and promises to improve the efficiency of complex, scientific modeling tasks.
mesh
INTRODUCTION
State-of-the modeling of complex physical systems, such deforming surfaces and volumes, often employs mesh representations to solve the underlying partial differential equations (PDEs). Mesh-based finite element simulations underpin popular methods structural mechanics [31, 48), aerodynamics [13, 34), electromagnetics [32], geophysics [35, 39], and acoustics [26]. Meshes also support adaptive representations, which enables optimal use of the resource budget by allocating greater resolution to regions of the simulation domain where strong gradients are expected or more accuracy is required, such as the tip of an airfoil in an aerodynamics simulation. Adaptive meshing enables running simulations accuracy and resolution levels impossible with regular discretization schemes [8, 27] (Figure 3b).
Despite their advantages, mesh representations have received relatively little attention in machine learning. While meshes are sometimes used for learned geometry processing [9] and generative models of shapes [15, 29], most work on predicting high-dimensional physical systems focuses grids, owing to the popularity and hardware support for CNN architectures [19]. We introduce a method for predicting dynamics of physical systems, which capitalizes on the advantages of adap- tive mesh representations. Our method works by encoding the simulation state into graph, and performing computations in two separate spaces: the mesh-space, spanned by the simulation mesh, and the Euclidean world-space in which the simulation manifold is embedded (see Figure 3a). By passing messages in mesh-space, we can approximate differential operators that underpin the in- ternal dynamics of most physical systems. Message-passing in world-space can estimate external dynamics, not captured by the mesh-space interactions, such as contact and collision. Unstructured irregular meshes, as opposed to regular grids, support learning dynamics which are independent of resolution, allowing variable resolution and scale at runtime. By learning a map of desired resolution over the mesh (sizing field), together with local remesher, our method can even adaptively change
"equal contribution
Videos of all our experiments be found http:///ii..gggggg...cc///////es

---

Published as a conference paper at ICLR 2021
Iterative rollout
Learned one-step simulator
q
Update
p
M
M1
M
Cloth mesh nodes
V
Encoder
Processor
Decoder
Obstacle
Mesh-space edges
E
F W
World-space edges
Mesh-space messages
e
World-space messages
e
Message
passing x
Decoded accelerations P;
Figure 1: Diagram of MESHGRAPHNETS operating on our SPHEREDYNAMIC domain (video). The model uses Encode- Procdd- Process-Decode architecture trained with one-step supervision, and can be applied iteratively to generate long trajectories at inference time. The encoder transforms the input mesh M into graph, adding extra world-space edges. The processor performs several rounds of message passing along mesh edges and world edges, updating all node and edge embeddings. The decoder extracts the accelcration for each node, which is used update the mesh produce M++1,
the discretization during rollouts, budgeting greater computational resources for important regions of the simulation domain.
Together, our method allows us to learn the dynamics of vastly different physical systems, from cloth simulation over structural mechanics to fluid dynamics directly from data, providing only very general biases such as spatial equivariance. We demonstrate that by using mesh-space computation we can reliably model materials with free prediction models [37]. MESHGRAPHNETS outperform particle- and grid-based baselines, and can generalize to more complex dynamics than those on which it was trained.
state such elastics, which are challenging for mesh-
RELATED WORK
Modelling high-dimensional physics problems with deep learning algorithms has become an area of great research interest in fields such computational fluid dynamics. High resolution simulations are often very slow, and learned models can provide faster predictions, reducing turnaround time for workflows in engineering and science [16, 6, 49, 20, 1]. Short run times also desirable property for fluid simulation in visualization and graphics [46, 41, 47). Learned simulations can be useful for real-world predictions where the physical model, parameters
boundary conditions are not fully known [12]. Conversely, the accuracy of predictions can be increased by including specialized knowledge about the system modelled in the form of loss terms [43, 23], or by physics-informed feature normalization [40].
The methods mentioned above are based on convolutional architectures on regular grids. Although this is by far the most widespread architecture for learning high high-dimensional physical systems, re- cently there has been an increased interest particle-based representations, which are particularly attractive for modelling the dynamics of free-surface liquids and granular materials. Ladicky et al. [22] use random forests to speed up liquid simulations. Various works [24, 42, 37] use graph neural networks (GNNs) [38, 4] to model particle-based granular materials and fluids, well as glassy dynamics [3]. Learned methods can improve certain aspects of classical FEM simulations, e.g. more accurate handling of strongly nonlinear displacements [25] learned elements which di- rectly map between forces and displacements [10]. Finally, dynamics of high dimensional systems can be learned in reduced spaces. Holden et al. [18] performs PCA decomposition cloth data, and
2

---

Published as a conference paper at ICLR 2021
(a) FlagDynamic
(b) DeformingPlate
CylinderFlow
(d) Airfoil
Cantel
Our model can predict dynamics of vastly different physical systems, from structural mechanics over cloth to fluid dynamics. We demonstrate this by simulating (a) flag waving the wind, (b) a deforming plate, (c) flow of water around cylinder obstacle, and (d) the dynamics of air around the cross-section of an aircraft wing (videos). The color map shows the von-Mises stress in (b), and the x-component of the velocity field in (c).(d).
Figure 2:
learns a correction model to improve accuracy of subspace simulation. These models are however very domain-specifie, and the expression range is limited due the of the linear subspace.
There is increased attention in using meshes for learned geometry and shape processing [9, 29, 17]. But despite mesh-based simulations being the tool of choice in mechanical engineering and related disciplines, adaptive mesh representations have not seen much use in machine learning for physics prediction, with few notable exceptions [5, 2]. Belbute-Peres et al. [5] embed differentiable aerodynamics solver in a graph convolution (GCN) [21] prediction pipeline for super-resolution in aerodynamics predictions. Our method has similarities, but without a solver in the loop, which potentially makes it easier to use and adapt to new systems. In Section 5 we show that MESH- GRAPHNETS are better suited for dynamical prediction than GCN-based architectures. Finally, Graph Element Networks [2] uses meshes over 2D grid domains to more efficiently compute pre- dictions and scene representations. Notably they use small planar systems (< 50 nodes), while we show how to scale mesh-based predictions to complex 3D systems with thousands of nodes.
MODEL
We describe the state of the system at time i using simulation mesh M EM) with nodes is associated with a reference mesh-space
V connected by mesh edges E,M Each node coordinate u; which spans the simulation mesh, and additional dynamical quantities qi that we want to model. Eulerian systems (Figure 2c,d) model the evolution of continuous fields such as velocity
fixed mesh, and qi sample these fields at the mesh nodes. In Lagrangian systems, the mesh represents a moving and deforming surface or volume (e.g. Figure 2a.b), and contains an extra world-space coordinate x; describing the dynamic state of the mesh in 3D space, in addition to the fixed mesh-space coordinate (Figure 3a).
3.1 LEARNING FORWARD DYNAMICS
The task is to learn forward model of the dynamic quantities of the mesh at time /+1 given the
current mesh M and (optionally) a history of previous meshes (M'-.,... M/-h). We propose MESHGRAPHNETTS
graph neural network model with an Sncode-Process, Proces--Deoode Decode architecture [4, 37], followed by an integrator. Figure shows a visual scheme of the MESHGRAPHNER archi- tecture. Domain specific information the encoding and integration
be found in Section 4.
Encoder The encoder encodes the current mesh M into a multigraph G = (V, E EW). Mesh nodes become graph nodes V. and mesh edges become bidirectional mesh-edges EM in the graph. This serves to compute the internal dynamics of the mesh. For Lagrangian systems, we add world edges F.W the graph, to enable learning external dynamics such (self-) collision and contact,
3

---

Published as a conference paper at ICLR 2021
[b)
world space
mesh space U
regular mesh
adaptive mesh
Figure 3: Simulation of cloth interacting with a sphere. (a) In red, we highlight two nodes which close in world-space but far in mesh-space, between which world edge may be created. (b) With the same number of nodes, adaptive remeshing enables significantly more accurate simulations than regular mesh with the same number of nodes.
which are non-local in mesh-space.' World-space edges are created by spatial proximity: that is, given fixed-radius rw on the order of the smallest mesh edge lengths. we add a world edge be- tween nodes i and j if x, Xyl < rw, excluding node pairs already connected in the mesh. This encourages using world edges to pass information between nodes that are spatially close, but distant in mesh space (Figure 3a).
Next, we encode features into graph nodes and edges. To achieve spatial equivariance, positional features are provided as relative edge features. We encode the relative displacement vector in mesh space ü,j = u, u, and its norm [uy] into the mesh edges e E E Then, we encode the relative world-space displacement vector Xij and its norm [x.y into both mesh edges eij E E and world M
edges e! E EW, All remaining dynamical features q, well one-hot vector indicating node type, are provided as node features in V;.
Finally, the concatenated features above encoded into latent vector of size 128 at each node and edge, using the encoder MLPs M c , for mesh edges M world edges W and nodes v; respectively. See sections 4 and A.1 for more details input encoding.
Processor
The processor consists of I, identical message passing blocks, which generalize Graph- Net blocks [36] multiple edge sets. Each block contains separate set of network parameters, and applied in sequence to the output the previous block, updating the mesh edge e
e. world edge
W and node v; embeddings e e v'; respectively by
W + (W Vi+ (V. city t ( vovs), (c),,,))
Σ
c
Σ ij
ch 3
(1)
where fM, fW, fV are implemented using MLPs with a residual connection.
Decoder and state updater For predicting the time t+1 state from the time input, the decoder uses an MLP 8V to transform the latent node features after the final processing step into one or more output features P:-
We can interpret the output features p; as (higher-order) derivatives of q, and integrate them using forward-Eule integrator with At 1 to compute the next-step dynamical quantity q+ For first- order systems the output P; is integrated once to update 1+1 = p, + q while for second-order integration happens twice: q [+1 = p: + 2q q' Additional output features p; are also used to make direct predictions of auxiliary quantities such as pressure or stress. For domain-specinf details on decoding, see Section 4. Finally, the output mesh nodes V are updated using q to produce M For some systems, dynamically adapt the mesh after each prediction step; this is explained in the following section.
From here on, any mention of world edges and world coordinates applies only to Lagrangian systems; they omitted for Fulerian systems.
4

---

Published as a conference paper at ICLR 2021
3.2 ADAPTIVE REMESHING
Adaptive remeshing algorithms generally consist of two parts: identifying which regions of the simulation domain need coarse or fine resolution, and adapting the nodes and their connections to this target resolution. Only the first part requires domain knowledge of the type of physical system, which usually comes in the form of heuristics. For instance, in cloth simulation, one common heuris- tic is the refinement of areas with high curvature to ensure smooth bending dynamics (Figure 3b), while in computational fluid dynamics, is common to refine around wall boundaries where high gradients of the velocity field are expected.
In this work we adopt the sizing field methodology [27]. The sizing field tensor S(u) E R specifies the desired local resolution by encoding the maximally allowed oriented, edge lengths in the simulation mesh. An edge Ut is valid and only if u,Js, ui I 1, otherwise is too long, and needs to be split". Given the sizing field, a generic local remeshing algorithm can simply split all invalid edges to refine the mesh, and collapse as many edges as possible, without creating new invalid edges, to coarsen the mesh. We denote this remeshing process as M' = -),,SS).
Learned remeshing To leverage the advantages in efficiency and accuracy of dynamic remeshing. we need to be able to adapt the mesh at test time. Since remeshing requires domain knowledge, we would however need to call the specific remesher used to generate the training data each step during the model rollout, reducing the benefits of learning the model. Instead, we learn model of the sizing field (the only domain-specific part of remeshing) using the same architecture as Sec- tion 3.1 and train decoder output p; to produce a sizing tensor for each node. At test time, for each time step we predict both the next simulation state and the sizing field, and use a generic, domain- independent remesher R to compute the adapted next-step mesh as M = R( ,Št+1). We demonstrate this on triangular meshes, Section A.3 describes the simple generic remesher that we use for this purpose. While the sizing field is agnostic to the mesh type, other mesh types may re- quire different local remeshers; for tetrahedral meshes a method such as Wicke et [45] could be used, while quad meshes can simply be split into triangular meshes.
3.3 MODEL TRAINING
We trained our dynamics model by supervising on the per-node output features p. produced by the decoder using L2 loss between p; and the corresponding ground truth values p.. Similarly, the sizing field model is trained with an L2 loss on the ground truth sizing field. If sizing information is not available the training data, e.g. not exposed by the ground truth simulator, we can still estimate a compatible sizing field from samples of simulator meshes, and use this estimate as labels (details in Section A.3.1).
4 EXPERIMENTAL DOMAINS
We evaluated our method on a variety of systems with different underlying PDEs, including cloth, structural mechanies, incompressible and compressible fluids (Figure 2). Training and test data was produced by different simulator for each domain. The simulation meshes range from regular to highly irregular: the edge lengths of dataset AIRFOIL range between 2- 10 *In 3.5m, and we also simulate meshes which dynamically change resolution over the course of a trajectory. Full details on the datasets can be found in Section A.
Our structural mechanics experiments involve hyper-elastic plate, deformed by kinematic actu- ator, simulated with quasi-static simulator (DEFORMINGPLATEE) Both actuator and plate are part of the Lagrangian tetrahedral mesh and are distinguished by one-hot vector for the corresponding node type n,, We encode the node quantities u,, x,, n, the mesh, and predict the Lagrangian velocity x', which is integrated once to form the next position x! ++1 As second output, the model predicts the von-Mises stress o; at each node.
Our cloth experiments involve flag blowing in the wind (FLAGDYNAMIC) and piece of cloth interacting with a kinematic sphere (SPHEREDYNAMIC) on an adaptive triangular mesh which
TTii formulation allows different maximal edge lengths depending the direction. For e.g. mesh hend around cylinder, it allows specify shorter edge lengths in the bent dimension than along the cylinder.
5

---

Published as a conference paper at ICLR 2021
Ground truth
Prediction
Generalization experiment 20k nodis
Training domain 2k nodes
Figure 4: (a) Rollout of our model versus ground truth on dataset AIRFOIL. Adaptive meshing allows us accurately predict dynamies large and small scales. The grid-based U-Net baseline is capable of making good predictions at large scales, but it cannot resolve the smaller scales, despite using four times more cells than our model (video). (b) inference time, our model can be scaled up to significantly larger and more complex setups than seen during training (video).
changes resolution each time step. The dataset FLAGSIMPLE shares the setup of FLAGDY- NAMIC, but uses a static mesh and ignores collisions. The node type distinguishes cloth and obstacle/boundary nodes, and we encode inputs u, X;, n, above, but since this a fully dynamic second order system, we additionally provide = 1 steps of history, by including the velocity es- as a node feature. The decoder outputs acceleration X; which integrated
timate x = twice.
Our incompressible fluid experiments use the CYLINDERFLLOW dataset, which simulates the flow of water around cylinder on fixed 2D Eulerian mesh. The mesh contains the node quantities u, n,, w,, where w, is a sample the momentum field at the mesh nodes. In all fluid domains, the node type distinguishes fluid nodes, wall nodes and inflow/outflow boundary nodes. The network predicts change in momentum W,, which is integrated once, and direct prediction of the pressure field p.
Our compressible fluid experiments use the AIRFOIL dataset, which simulates the acrodynamics around the cros-secttion of airfoil wing. We model the evolution of momentum' w and density p fields, and hence the 2D Eulerian mesh encodes the quantities u, n, W,, Ps- We treat this as a first order system and predict change in momentum w, and density Pú, well pressure pi.
RESULTS
We tested our MESHGRAPHNETS model four experimental domains (Section 4), and com- pared it to three different baseline models. Our main findings are that MESUGRAPINNTTS are able produce high-quality rollouts all domains, outperforming particle- and grid-based baselines, while being significantly faster than the ground truth simulator, and generalizing to much larger and more complex settings test time.
Videos of rollouts, as well as comparisons, can be found at https://sites. http://sitesgoogllccooo google.com/vieww meshgraphnets. Visually the dynamics remain plausible and faithful the ground truth. Table shows 1-step prediction and rollout errors in all of our datasets, while qualitative and quantitative comparisons are provided in Figure and Figure Even though our model was trained on next-step predictions, model rollouts remain stable for thousands of steps. This video shows a model trained on trajectories of 400 steps rolled out for 40000 steps.
Learned remeshing We trained both dynamics and sizing field model to perform learned dynamic remeshing during rollout on FLAGDYNAMIC and SPHEREDYNAMIC, We compare learned remeshing variants with sizing model learned from labeled sizing data, as in Section 3.2, as well as
In visualizations, we show velocity, caleulated momentum divided by density p.

---

Published as a conference paper at ICLR 2021
(a)
1e1
1e-1
(c)
1e-1
1c-2
Rolbut RMMSERMEE
Cylnnderloww
GCN
Rollout RMSE
(ground truth meshes)
0.50
(MLP)
0.50
Ours
(learned sizing field)
UNet-120
0.25
0.25
(larned soing field from estimated targets)
0.30
0.00
(b)
GNS 1
(d)
RMSE HlggSimple
1.0
Gais-mesh-ppo
Rollout PMSSE RegDynamic
0.75
Rellout RANSE
Reabynairi
0.10
0.5
0.05
1113151720
2
Simulation Rollout Srep
Num Message Passing Steps (1)
History Stre
Figure 5: (a) Our model outperforms GCN and CNN-based baselines. (b) GNS diverges on cloth datasets; providing mesh-space positions (GNS+mesh-pos helps, but still fails on dynamic meshes. (c) Remeshing with learned or estimated sizing fields produces accurate rollouts. (d) Taking suf- ficient message passing steps is crucial for good performance, and limiting history size increases accuracy by preventing overfitting.
from estimated targets, as in Section A.3.I. As a baseline, we ran our forward model on the ground truth mesh sequence. As observed in the video, all learned remeshing variants are able shift the resolution to the new folds as they appear in the cloth, yield equally plausible dynamics, and are on par" in terms of quantitative performance (Figure Sc). Thus, our learned remeshing method provides the benefits of adaptive remeshing, which can be substantive in some domains, without requiring domain-specific remesher the loop.
Computational efficiency Our approach is consistently faster than ground truth solvers by one to two orders of magnitude on all domains (Table 1). We believe this due to our model being able to take much larger timesteps than classical solvers, and avoiding performance bottlenecks. Addi- tionally, classical general-purpont often do not scale well on hardware accelerators, while our model is built from neural network build- ing blocks, highly suitable for hardware acceleration. A more detailed breakdown of performance on e.g. hardware setup available in the appendix (section A.5.1). Our model's strong efficiency advantage means it may be applicable in situations where computing costs are otherwise prohibitive.
solvers on irregular domains, such as those studied in this paper,
Generalization Our MESHGRAPHNE
model generalizes well outside of the training distribu- tion, with respect to underlying system parameters, mesh shapes, and mesh size. This is because the architectural choice of using relative encoding on graphs has shown to be very conducive to
"Note that the comparison ground truth requires interpolating to the ground truth mesh, incurring a small interpolation penalty for learned remeshing models.
RMSE
RMSE
RMSE
Dataset
# nodes (avg.)
# steps
Emodel ms/step
trull ms/step
tGT ms/step
1-step
rollout-50 x10
rollout-all x10
x10
PLAGSIMPLE
1579
400
19
19
4166
1.08 + 0.02
92.6
5.0
139.0 + 2.7
FLAGDYNAMIC
2767
250
43
837
26199
1.57 + 0.02
72.4 4.3
151.1 + 5.3
SPHEREDYNAMIC
1373
500
32
140
1610
0.292 + 0.005
11.5 = 0.9
28.3 + 2.6
DEFORMINGPLAND
1271
400
24
33
2893
0.25
+ 0.05
1.8 0.5
15.1 + 4.0
CYLWNDERFLOW
1885
600
21
23
820
2.34 1 0.12
6.3 = 0.7
40.88 1 7.2
AIRFOI.
5233
600
37
38
11015
314 + 36
582
37
11529 + 1203
Table 1: Left: Inference timings of our model per step on a single GPU, for pure neural network inference (tmodel) and including remeshing and graph recomputation (trai). Our model has significantly lower running cost compared to the ground truth simulation (tGT). more detailed breakdown can be found in the section 1.5.1. Right: Errors of our methods for a single prediction step (I-step), 50-step rollouts, and rollout of the whole trajectory.
7

---

Published as a conference paper at ICLR 2021
generalization [37]. Also, by forcing the network to make predictions very irregulaly--shaped and dynamically changing meshes, we encourage learning essolution-independden physics.
In AIRFOIL, we evaluate the model on steeper angles (-359... ...35" vs -25 in training) and higher inflow speeds (Mach number 0.7...0.9 vs 0.2...0.7 in training). In both cases, the behavior remains plausible (video) and RMSE raises only slightly from 11.5 at training angles and 13.1 for higher inflow speeds. We also trained a model on a FLAGDYNAMIC variant with wind speed and directions varying between trajectories. but constant within each trajectory. At inference time, we can then vary wind speed and direction freely (video). This shows that the local physical laws our models learns can extrapolate to untrained parameter ranges.
12.4 for steeper
We also trained model in the FLAGDYNAMIC domain containing only simple rectangular cloth, and tested its performance on three disconnected fish-shaped flags (video). Both the learned dynam- ies model and the learned remesher generalized to the new shape, and the predicted dynamics were visually similar to the ground truth sequence. In a more extreme version of this experiment, we test that same model on windsock with tassels (Figure 4b, video). Not only has the model never seen a non-flat starting state during training. but the dimensions are also much larger - the mesh aver- ages at 20k nodes, order of magnitude more than seen in training. This result shows the strength of learning resolution and scale-independent models: we do not necessarily need to train on costly high-resso high-resolution simulation data; we may be able to learn to simulate large systems that would be too slow on conventional simulators, by training on smaller examples and scaling up at inference time. A more in-depth analysis on scaling can be found in the appendix A.5.3.
Comparison to mesh-free GNS model We compared our method to the particle-based method GNS [37] on the fixed-mesh dataset FLAGSIMPLE to study the importance of mesh-space embed- ding and message-passing. As in GNS, the encoder builds a graph with fixed radius connectivity (10-20 neighbors per node). and relative world-space position embedded as edge features. As GNS lacks the notion of cloth's 's resting state, eπтоr accumulates dramatically and the simulation becomes unstable, with slight improvements if providing 5 steps of history (Figure 5b).
We also explored hybrid method (GNS+mesh-pos) which adds a mesh-space relative position feature U,y to the GNS edges. This yields rollout errors on par with our method (flattening after 50 steps due to decoherence in both cases), however, it tends to develop artifacts such as entangled triangles, which indicate a lack of reliable understanding of the mesh surface (video). On irregularly spaced meshes (FLAGSIMPLE)) GNS+mesh-pos was not able to produce stable rollouts at all. A fixed connectivity radius will always oversample high-res regions, and undersample low-res regions of the mesh, leading instabilities and high rollout errors (Figure 5b, right). We conclude that both having access to mesh-space positions as well as passing messages along the mesh edges are crucial for making predictions on irregularly spaced meshes.
Conversely, we found that passing message purely in mesh-space, without any world-space edges, also produces substandard results. On FLAGDYNAMIC and SYHEREDYNAMIC we observe in- crease in rollout RMSE of 51% and 92% respectively, as (self-)collision are harder to predict with- out world edges. In the latter case this is particularly easy to see: the obstacle mesh and cloth mesh are not connected, so without world edges, the model cannot compute their interaction at all.
Comparison to GCNs To study the role of the graph network architecture, we tested our model against GCNs [21], which do not compute messages on edges. We adopted the GCN architecture from Belbute-Peree et al. [5] (without the super-resolution component) and trained it in the same setup as in our approach, including e.g. training noise and integration. We replicated results on the aerodynamical steady-state prediction task it was designed for (see Section A.4.2). On the much richer AIRFOII. task, however, GCN was unable to obtain stable rollouts. This is not simply a question of capacity; we created a hybrid (GCN-MIPP) with our model (linear layers replaced by 2- 2-hidden-layer MLPs + LayerNorm; 15 GCN blocks instead of 6), but the rollout quality was still poor (Figure 5a, video). We also ran an ablation of MESHGRAPHNETT without relative encoding in edges, for which absolute positional values used as node features. This version performed much worse than our main model, yielding visual artifacts in the rollouts, and a rollout RMSE of 26.5 in AIRFOIL. This is consistent with our hypothesis that the GCN performs worse due to the lack of relative encoding and message computing, which makes the GCN less likely to learn local physical laws and more prone to overfitting.
8

---

Published as a conference paper at ICLR 2021
Comparison to grid-based methods (CNNs) Arguably the most popular methods for predicting physical systems are grid-based convolutional architectures. It is fundamentally hard to simulate Lagrangian deforming meshes with such methods, but the Fulerian 2D domains CYL.INDER FLOw and AIRFOT,, by interpolating the ROI onto a 128x 128 grid. We implemented the UNet architecture from Thürey et al. [40], and found that on both datasets, MESHGRAPHNNTT outperforms the UNet in terms of RMSE (Figure 5a). While the UNet was able make reasonable predictions region around the wingtip (Figure 4a), even while using four times more cells to span region 16 times smaller than our method (Figure A.I). We observe similar behavior around the obstacle in CYLINDERFL.WW Additionally, as seen in the video, the UNet tends to develop fluctuations during rollout. This indicates that predictions over meshes presents advantages even flat domains.
compare to grid-based methods on
larger scales on AIRFOIL, it undersampled the important wake
Key hyperparameters We tested several architecture variants and found our method is not very sensitive to many choices, such as latent vector width, number of MI.P layers and their sizes. Nonetheless we identified two key parameters which influence performance (Figure 5d). Increasing the number of graph net blocks (message passing steps) generally improves performance, but it in- curs higher computational cost. We found that value of 15 provides good efficiency/accuracy trade-off for all the systems considered. Second, the model performs best given the shortest possible history (h=1 to estimate x in cloth experiments, h=0 otherwise). with any extra history leading to overfitting. This differs from GNS [37], which used h E 2...5 for best performance.
6 CONCLUSION
MESHGRAPHNETS are general-purpose mesh-based method which can accurately and efficiently model a wide range of physical systems, generalizes well, and can be scaled up at inference time. Our method may allow more efficient simulations than traditional simulators, and because it is dif- ferentiable, it may be useful for design optimization or optimal control tasks. Variants tailored to specific physical domains, with physics-based auxiliary loss terms, or energy-conserving integra- tion schemes have the potential to increase the performance further. Finally, learning predictions on meshes opens the door for further work on iisolution-adaptivi adaptive meshing from ground truth data, we could learn a discretization which directly optimizes for prediction accuracy, or even performance on downstream task. This work represents an impor- tant step forward in learnable simulation, and offers key advantages for modeling complex systems in science and engineering.
For example, instead of learning
ACKNOWLEDGMENTS
We would like to thank Danilo Rezende, Jonathan Godwin, Charlie Nash, Oriol Vinyals, Matt Hoff- man, Kimberly Stachenfeld, Jessica Hamrick, Piotr Trochim, Emre Karagozler and our reviewers for valuable discussions, implementation help and feedback on the work and manuscript.
REFERENCES
[1] MS Albergo, G Kanwar, and PE Shanahan. Flow-based generative models for markov chain monte carlo in lattice field theory. Physical Review D, 100(3):034515, 2019.
[2] Ferran Alet, Adarsh Keshav Jeewajee, Maria Bauza Villalonga, Alberto Rodriguez, Tomas Lozano-Perez, and Leslie Kaelbling. Graph element networks: adaptive, structured computa- tion and memory. International Conference on Machine Learning, pp. 212-222, 2019.
[3] V Bapst, T Keck, A trabska-Barwinska C Donner, ED Cubuk, SS Schoenholz, A Obika, AWR Nelson, Back, Hassabis, et al. Unveiling the predictive power of static structure in glassy systems (vol 16, pg 448, 2020). NATURE PHYSICS, 16(6):702-702. 2020.
[4] Peter W Battaglia, Jessica Hamrick, Victor Bapst, Alvaro Sanchez- Sanchez-Gonzalez, Vinicius Zam- baldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, et al. Relational inductive biases, deep learning, and graph networks. arXiv preprint arXiv: 1006.0000002222 2018.

---

Published as a conference paper at ICLR 2021
[5] Filipe de Avila Belbute-Peress Thomas D. Economon, and J. Zico Kolter. Combining differen- tiable PDE solvers and graph neural networks for fluid flow prediction. In Proceedings of the 37th International Conference Machine Learning ICML 2020, 2020.
[6] Saakaar Bhatnagar, Yaser Afshar, Shaowu Pan, Karthik Duraisamy, and Shailendra Kaushik. Prediction of aerodynamic flow fields using convolutional neural networks. Compulational Mechanicx, 64(2):525-544 2019.
[7] Frank Bossen and Paul S Heckbert. A pliant method for anisotropic mesh generation. In 5th Intl. Meshing Roundtable, pp. 63-74. Citeseer, 1996.
[8] Ricardo Branco, FV Antunes, and JD Costa. A review on 3d-fe adaptive remeshing techniques for crack growth modelling. Engineering Fracture Mechanics, 141:170-195, 2015.
[9] Michael M. Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. Geometric deep learning: going beyond euclidean data. CoRR, abs/1611 10011110007 2016.
[10] German Capuano and Julian Rimoli. Smart finite elements: A novel machine learning appli- cation. Computer Methods in Applied Mechanics and Engineering, 345:363-233 1-11 2019.
[11] Comsol. Comsol multiphysics@ 5.4. http://comsol.com, 2020.
[12] Emmanuel de Bezenac, Arthur Pajot, and Patrick Gallinari. Deep learning for physical pro- cesses: Incorporating prior scientific knowledge. Journal of Statistical Mechanics: Theory and Experiment, 2019(12):124009, 2019.
[13] Thomas D Economon, Francisco Palacios. Sean R Copeland, Trent W Lukaczyk, and Juan J Alonso. Su2: An open-source suite for multiphysics simulation and design. Aiaa Journal, 54 (3):828-844 846, 2016.
[14] Exxact.
ANSYS Mechanical GPU Benchmarks.
https://blog.xxxaccoorg wtw.:/aaw..............ea//e//aa
2020.
[15] Thibault Groueix, Matthew Fisher, Vladimir G Kim, Bryan C Russell, and Mathieu Aubry. A papier-måché approach to learning 3d surface generation. In Proceedings of the IEEE confer- ence on computer vision and pattern recognition, pp. 216-224, 2018.
[16] Xiaoxiao Guo, Wei Li, and Francesco lorio. Convolutional neural networks for steady flow approximation. Proceedings of the 22nd ACM SIGKDD international conference on knowl- edge discovery and data mining, pp. 481-490, 2016.
[17] Rana Hanocka, Amir Hertz, Noa Fish. Raja Giryes. Shachar Fleishman, and Daniel Cohen-Or Meshenn: a network with an edge. ACM Transactions on Graphics (TOG), 38(4):1-12, 2019.
[18] Daniel Holden, Bang Chi Duong, Sayantan Datta, and Derek Nowrouzezahrai. Subspace neu- ral physics: Fast data-driven interactive simulation. In Proceedings of the 18th annual ACM SIGGRAPH//uungapphie Symposium Computer Animation, pp. 1-12, 2019.
[19] Sara Hooker. The hardware lottery. arXiv preprint arXiv:2009.06488) 2020.
[20] Gurtej Kanwar, Michael S. Albergo, Denis Boyda, Kyle Cranmer, Daniel C. Hackett, Sébastien Racanière, Danilo Jimenez Rezende, and Phiala E. Shanahan. Equivariant flow-based sampling for lattice gauge theory. Phys. Rev. Lett., 125:121601, Sep 2020. doi: 10. 10.1100/hysReeLLett 125.121601.
[21] Thomas N. Kipf and Max Welling, Semi-supervised classification with graph convolutional networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017.
[22] L'ubor Ladický, SoHyeon Jeong, Barbara Solenthaler, Marc Pollefeys, and Markus Gross. Data-driven fluid simulations using regression forests. ACM Transactions on Graphics (TOG), 34(6):1-9, 2015..
10

---

Published as a conference paper at ICLR 2021
[23] Sangseung Lee and Donghyun You. Data-driven prediction of unsteady flow circular cylinder using deep learning. Journal of Fluid Mechanics, 879:217-254, 2019. doi: 10.1017/ jfm.2019.700.
[24] Yunzhu Li, Jiajun Wu, Russ Tedrake, Joshua B. Tenenbaum, and Antonio Torralba. Learning particle dynamics for manipulating rigid bodies, deformable objects, and fluids. In Interna- tional Conference on Learning Representations. 2019.
[25] Ran Luo, Tianjia Shao, Huamin Wang. Weiwei Xu, Xiang Chen, Kun Zhou, and Yin Yang. Nnwarp: Neural network- based nonlinear deformation. IEEE transactions on visualization and computer graphics, 2018.
[26] Steffen Marburg and Bodo Nolte. Computational acoustics of noise propagation in fluids: finite and boundary element methods, volume 578. Springer, 2008.
[27] Rahul Narain, Armin Samii, and James F. O'Brien. Adaptive anisotropic remeshing for cloth simulation. ACM Trans. Graph., (6)) 2012.
[28] Rahul Narain, Tobias Pfaff, and James F. O' O'Brien, Folding and crumpling adaptive sheets. ACM Trans. Graph., 32(4), 2013.
[29] Charlie Nash, Yaroslav Ganin, SM Eslami, and Peter W Battaglia. Polygen: An autoregressive generative model of 3d meshes. In International Conference on Machine Learning, 2020.
[30] Vidia.
GPU-accelerated ANSYS FLUENT. apa.eeerrateeaapplpicctooonassss fluent, 2019.
https:/wwww.aiiddaaccccccrraraav center/
[31] SK Panthi, N Ramakrishnan, KK Pathak, and JS Chouhan. An analysis of springback in sheet metal bending using finite element method (fem). Journal of Materials Processing Technology, 186(1-3):1 10--124, 2007.
[32] D Pardo, L Demkowicz, C Torres-Verdin, and M Paszynski. A self-adaptive goal goal-oriented hp- finite element method with electromagnetic applications. part ii: Flectrodynamics. Computer methods in applied mechanics and engineering. 196637-40)
2007.
[33] Tobias Pfaff, Rahul Narain, Juan Miguel De Joya, and James FO' O'Brien. Adaptive tearing and cracking of thin sheets. ACM Transactions on Graphics (TOG), 33(4):1-9, 2014.
[34] Ravi Ramamurti and William Sandberg. Simulation of flow about flapping airfoils using finite element incompressible flow solver. AIAA journal, 39(2):253-260, 2001.
[35] Zhengyong Ren and Jingtian Tang. 3d direct current resistivity modeling with unstructured mesh by adaptive finite-element method. Geophysics, 75(1):H7-H17, 2010.
[36] Alvaro Ganzaez- Nicolas Heess, Jost Tobias Springenberg, Josh Merel, Martin Ried- miller, Raia Hadsell, and Peter Battaglia. Graph networks as learnable physics engines for inference and control. In Proceedings of the 35th International Conference Machine Learn- ing, ICML, pp. 4467-4476, 2018
[37] Alvaro Fanchez-Gonzaled Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, and Pe- ter W. Battaglia. Learning to simulate complex physies with graph networks. In Proceedings of the 37th International Conference on Machine Learning ICML 2020, 2020.
[38] Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfar- dini. The graph neural network model. IEEE Transactions on Neural Networks, 20 1-0-00 2008.
[39] Christoph Schwarzbach, Ralph-Uwe Börner, and Klaus Spitzer. Three-dimensional adap- tive higher order finite element simulation for geo-electromannetics-- marine esem example. Geophysical Journal International, 187(1):63-74, 2011.
[40] Nils Thuerey, Konstantin Weißennww Lukas Prantl, and Xiangyu Hu. Deep learning methods for aeynolds-averaged pavier-stoker simulations of airfoil flows. AIAA Journal, 58 58(1):55-36 2020.
11

---

Published as a conference paper at ICLR 2021
[41] Kiwon Um, Xiangyu Hu, and Nils Thuerey. Liquid splash modeling with neural networks. In Computer Graphics Forum, volume 37, pp. 171-182. Wiley Online Library, 2018.
[42] Benjamin Ummenhofer, Lukas Prantl, Nils Thürey, and Vladlen Koltun. Lagrangian fluid simulation with continuous convolutions. In International Conference on Learning Represen- lations, 2020.
[43] Rui Wang, Karthik Kashinath, Mustafa Mustafa, Adrian Albert, and Rose Yu. Towards physics-informed deep learning for turbulent flow prediction. In ACM SIGKDD international conference on knowledge discovery and data mining, 2020.
[44] Emo Welzl. Smallest enclosing disks (balls and ellipsoids). In New results and trends in computer science, pp. 359-370. Springer, 1991.
[45] Martin Wicke, Daniel Ritchie, Bryan M. Klingner, Sebastian Burke, Jonathan R. Shewchuk, and James F. O' O'Brie. Dynamic local remeshing for elastoplastic simulation. ACM Trans. Graph., 29(4), 2010.
[46] Steffen Wiewel, Moritz Becher, and Nils Thuerey. Latent space physics: Towards learning the temporal evolution of fluid flow. In Computer Graphics Forum, pp. 71-82. Wiley Online Library, 2019.
[47] You Xie, Erik Franz, Mengyu Chu, and Nils Thuerey. Tempogan: A temporally coherent, volumetric gan for super-resolution fluid flow. ACM Trans. Graph., 37(4), July 2018.
[48] Abdelaziz Yazid, Nabbou Abdelkader, and Hamouine Abdelmadjid. A state-of-the-art review of the x-fem for computational fracture mechanies. Applied Mathematical Modelling. 33(12): 4269 4282, 2009.
[49] Yao Zhang, Woong Je Sung, and Dimitri N Mavris. Application of convolutional neural net- work to predict airfoil lift coefficient. In 2018 A/AA/ASCE/ATIS/ASC Structures, Structural Dynamics, and Materials Conference, pp. 1903, 2018.
12

---

Published as a conference paper at ICLR 2021
A
APPENDIX
Figure A.I: Many of our datasets have highly irregular meshing, which allows to predict dy- namics at several scales. With only 5k nodes, the dataset AIRFOIL spans a large region around the wing (left: entire simulation domain), while still providing high resolution around the airfoil (mid- dle: ROI for visual comparison and RMSE computation), down to sub-millimeter details around the wing tip (right).
A.1 DATASET DETAILS
Below list details for all of our datasets. "System" describes the underlying PDE: cloth, hyper- elasticity or compressible and incompressible Navier-Stokes flow. We used AreSim [27] for simu- lating the cloth datasets, SU2 [13] for compressible flows, and COMSOL [11] for incompressible flow and hyperelastic simulations. Hyper-elasticity and cloth are simulated using linear elements. Each dataset consists of 1000 training, 100 validation and 100 test trajectories, each containing 250- 600 time steps. Meshing can be either regular, i.e. all edges having similar length, irregular, i.e. edge lengths vary strongly in different regions of the mesh or dynamic, i.e. change at each step of the simulation trajectory. For Lagrangian systems, the world edge radius Tw provided. Our model operates on the simulation time step At listed below. However, for each output time step, the solvers compute several internal time steps (16 for AreSim, 100 for SU2, adaptive for COMSOL). As quasi-static simulation, DEFORMINGPLAYE does not have a time step.
Datasct
System
Solver
Mesh type
Meshing
At
steps
s
FLAGSIMPLE
cloth
ArcSim
triangle 3D
regular
400
0.02
FLAGDYNAMIC
cloth
AreSim
triangle
dynamic
250
0.02
0.05
SPHEREDYNAMIC
cloth
AreSim
triangle 3D
dynamic
500
0.01
0.05
DEFORMINGPLATE
hyper-el.
COMSOL
tetrahedral 3D
inregular
400
0.03
CylinDeRFLow
incompr.
COMSOL
triangle
irregular
600
0.01
AIRFOII
compr. NS
SU2
triangle 2D
irregular
600
0.008
Next, we list input encoding for mesh edges ety. world edges e and nodes v.. as well as the predicted output for each system.
inputs M
inputs e/y'
inputs V;
outputs p
history h
System
Type
e
Cloth
Lagrangian
Ugys
Xijo
Xij
X
X,g
n1; (x{ x
X;
1
Hyper-El.
Lagrangian
Ugjs
U1g
Xijo
X,y
Xiyo Xxq
n2
X,, T.
(
Incomp. NS
Fulerian
Uzj.
Uij
Di; W;
Wi,Ps
(
Compr. NS
Eulerian
n1; W;s Pi
W;, Pis Pi
13

---

Published as a conference paper at ICLR 2021
ground truth pressure
(b) predicted pressure
Figure A.2: Beside output quantities such as position momentum, which are integrated and fed back into the model as an input during rollout, we can also predict auxiliary output quantities, such pressure stress. These quantities can be useful for further analyzing the dynamics of the system. Here, we show a snapshot of auxiliary predictions of the pressure field in CYLINDERFLOW.
All second-derivative output quantities (Ö) are integrated twice, while first derivative outputs (i) are integrated once as described in Section 3.1; all other outputs are direct predictions, and are not integrated. The one-hot node type vector n, allows the model distinguish between normal and kinematic nodes. Normal nodes are simulated, while kinematic either remain fixed in space (such as the nodes which keep the cloth from falling). or follow scripted motion (as the actuator in DEFORMINGPLATE), For scripted kinematic nodes, we additionally provide the next-step world- space velocity x x as input; this allows the model to predict next-step positions which are consistent with the movement of the actuator. In the variant of FLAGDYNAMIC with varying wind speeds (generalization experiment in Section 5), the wind speed vector appended to the node features.
In the dynamically meshed datasets (FLAGDYNAMIC, SPHEREDYANMIC), the mesh changes be- tween steps, and there is no 1:1 correspondence between nodes. In this case, interpolate dynam- ical quantities from previous meshes M M as well as M into the current mesh M using barycentric interpolation in space, in order to provide history and targets for each node.
A.2
ADDITIONAL. MODEL. DETAILS
A.2.1 ARCHITECTURE AND TRAINING
The MLPs of the Encoder M e activated two-hidden-layer MIPs with layer and output size of 128, except for 5 whose output size matches the prediction p,. All MLPs outputs except 8V are normalized by a LayerNorm. All input and target features are normalized to zero-mean, unit variance, using dataset statistics.
the Processor JM, JW.V and Decoder 8V are ReLU-
For training, we only supervise on the next step in sequence; to make our model robust to rollouts of hundreds of steps GPU with the Adam optimizer for 10M training steps, using an exponential learning rate decay from 10 to 10 over 5M steps.
training noise (see Section A.2.2). Models trained single v100
A.2.2 TRAINING NOISE
We used the same training noise strategy in GNS [37] make our model robust rollouts of hundreds of steps. We add random normal noise of zero mean and fixed variance to the most recent value of the corresponding dynamical variable (Section A.2.3). When choosing how much noise to add, we looked at the one-step model error (usually related to the standard deviation of the targets the dataset) and scanned the noise magnitude around that value on a logarithmic scale using two values for each factor of 10. For the exact numbers for each dataset, Table A.2.3.
In the cases where the dataset is modelled as a first-order system (all, except cloth domains); we adjust the targets according to the noise, that the model decoder produces output that after integration would have corrected the noise at the inputs. For example, DEFORMINGPLATE. as- sume the current position of node is x 2, and & 2.1 after adding noise. If the next position xt 3, the target velocity for the decoder ii 1 will be adjusted to ä. 0.9, that after
14

---

Published as a conference paper at ICLR 2021
integration, the model output = matches the next step z effectively correcting for the added noise, i.e. i// i 4. = 3 1
In the second-order domains (cloth), the model decoder outputs acceleration ž; from the input po- sition =f and velocity i x _i (as in GNS). As with other systems, add noise to the position 2i which indirectly results on a noisy derivative á estimate. In this case, due to the strong impossible adjust the targets simultaneously
dependeney between position and velocity, correct for noise in both values. For instance, assume o = 1.4, Φं = 2 = 3, which im- plies : = 0.6, + = and ground truth acceleration ž; = 0.4. After adding 0.1 of noise the inputs are ot. = 2.1 + 2 0.7. At this point, we could use modified acceleration target of i = 0.2, so that after integration, the next velocity is 2 2 F _ - E5+1 3 u! = effectively correcting for the noise added to the position. However, note that in this case the predicted next step velocity _ 0.9 does not match the ground truth J. = Similarly, we chose modified target acceleration of j. = 0.3, the next step velocity E--1 = would match the ground truth, correcting the noise in velocity, but the same would not be 2.
+ _r 0.9, and the next position
true for the next step position a++1 3.1.. Empirically, we treated how to correct the noise for cloth simulation hyperparameter y E [0, 1] which parametrizes a weighted average between the two achieved with
options: 8 = - (1 > Best performance
0.1.
Finally, when the model takes more than one step of history (h > 1) (e.g. in the ablation from Figure 5d on FLAGDYNAMIC), the noise added in random walk manner with a per-step variance such as the variance at the last step matches the target variance (in accordance with GNS [37]).
A.2.3
HYPERPARAMETERS
Dataset
Batch size
Noise scale
FLAGSIMPLE
pos: lc-3
FLAGDYNAMIC
pos: 3e-3
SPHEREDYNAMIC
pos: 1e-3
DEFORMINGPI..
2
pos: 3e-3
CYLINDER FLOWW
2
momentum: 2e-2
AIRFOIL
2
momentum: lel, density: le-2
Table 2: Training noise parameters and batch size.
A.3 A DOMAIN-INVARLAN LOCAL REMESHER FOR TRIANGULAR MESHES
A local remesher [27, 28, 33] changes the mesh by iteratively applying of three fundamental operations: splitring an edge to refine the mesh collapsing an edge to coarsen it, and flipping an edge change orientation and preserve sensible aspect ratio of its elements. Edge splits create a new node whose attributes (position, etc.), ). as well as the associated sizing tensor, are obtained by averaging values of the two nodes forming the split edge. Collapsing removes node from the mesh, while edge flips leave nodes unaffected.
edge split
edge
edge collapse
Given the sizing field tensor S; each node i, define the following conditions for performing edge operations:
An edge connecting node and should be split if it is invalid, i.e. u u.y.u > 1 with the
averaged sizing tensor S;y (S: S2))
15

---

Published as a conference paper at ICLR 2021
An edge should be collapsed, if the collapsing operation does not create any new invalid edges.
An edge should be flipped if the an-isotropic Delaunay criterion [7]
(uzk x U.s u < S (u, x ugi),
SA - 4 + 1s + Si)
is satisfied. This optimizes the directional aspect ratio of the mesh elements.
We can now implement simple local remesher by applying these operations in sequence. First, we split all possible mesh edges refine the mesh (in descending order of the metric u u,yS then flip all edges which should be flipped. Next, we collapse all edges we can collapse (in ascending order of the metric S. Ugy) to coarsen the mesh as much as possible, and finally again flip all possible edges to improve mesh quality.
A.3.1 ESTIMATING SIZING FIELD TARGETS
no sizing field is available to train the sizing model, we can estimate from a sequence of meshes. That is, for two consecutive meshes M MM+1 we want to find the sizing field S that would have induced this transition with a local remesher, i.e. M (t), S). To do this, we assume that the remesher is near-optimal, that is, all resulting edges valid, yet maximum-length under the metric S. For each S; associated with the node i, this can be expressed as:
S = argmax
Σ S.S. s.t.vj E M M u. $ JEN.
(2)
This problem corresponds finding the minimum-aree area, zero-centred ellipse containing the points Uij. and can be solved efficiently using the MINIDISK algorithm [44].
A.4 ADDITIONAL BASELINE DETAILS
A.4.1 BASELINE TRAINING
Baseline architectures were trained within our general training framework, sharing the same nor- malization, noise and state-update strategies. We optimized the training hyperparameters separately in each case.
A.4.2 GCN BASELINE
We re-implemented the base GCN architecture (without the super-resolution component) from Belbute- Parbute-Peresseee et al. [5]. To replicate the results, and ensure correctness of our implementation the baseline, created a dataset AIRFOILSTEADY which matches the dataset studied in their work. It uses the same solver and a similar setup as our dataset AIRFOIL, except that it has a narrower range in AIRFOII.. The biggest difference is that the predic-
of angle of attack (-100...10' tion task studied in their paper is not dynamical simulation as our experiments, but a steady-state prediction task. That instead of unrolling a dynamics model for hundreds of time steps, this task consists of directly predicting the final steady-state momentum, density and pressure fields, given only two scalars (Mach number m, angle of attack a) as well the target mesh positions u, essentially learning a parametrized distribution.
In STEADY the GCN predictions are visually indistinguishable to the ground truth, and qualitatively match the results reported in Belbute- et al. [5] for their "interpolation regime" experiments. We also trained our model in STRADI.STEADY, as a one-step direct prediction model (without an integrator). with encoding like in AIRFOIL (see Section A. but where each node is conditioned on the global Mach number m and angle of attack a), instead of density and momentum. Again, results visually indistinguishable from the ground truth (video), and our model outper- forms GCN in terms of RMSE (ours 0.116 vs GCN 0.159). This is remarkable, as our models' spatial equivariance bias works against this task of directly predicting global field. This speaks of the flexibility of our architecture, and indicates that it can be used for tasks beyond learning local physical laws for which it was designed.
16

---

Published as a conference paper at ICLR 2021
A.4.3 GRID (CNN) BASELINE
We re-implemented the UNet architecture of Thurey et al. [40] to exactly match their open-sourced version of the code. We used batch size of 10. The noise parameters from Section A.2.3 are absolute noise scale on momentum 6c-2 for CYLINDERFLOW, and lel on momentum and 1.5e-2 on density in the AIRFOIL dataset.
A.5 ADDITIONAL. ANALYSIS
A.5.1 PERFORMANCE
In the table below, we show detailed breakdown per-step timings our model run on CPU (8-core workstation) or a single v100 GPU. moddll measures inference time of the graph neural network, while full measures the complete rollout, including remeshing and graph recomputation. The ground truth simulation (tGr) was run on the same 8-core workstation CPU. On our datasets, inference uses between 1-2.5GB memory, including model variables and system overhead.
CPU
CPU
GPU
GPU full ms/step
tGT ms/step
CPU speedup
GPU speedup
Dataset
model ms/step
fulll ms/step
model ms/step
FLAGSTMPLE
186
187
19
19
4166
22.3
214.7
FLACDYNAMIC
534
1593
837
26199
16.4
31.3
SPHEREDYNAMIC
221
402
32
140
1610
4.0
11.5
DEFORMINGPLATE
172
174
24
33
2893
16.6
89.0
CYLINDERFLOW
166
168
820
35.3
AIRFOIL
497
499
37
38
11015
22.1
289.1
The NN bulding blocks used in our model are highly optimized for hardware acceleration. However, our ground truth solvers (AreSim, COMSOL and SU2) do not support GPUs; and more broadly, solvers have varying levels of optimization for different hardware, so we find it hard to provide a 'true" hardware-agnostic performance comparison. We do note a few trends.
In the simulation regime studied in this paper (i.e. general-purpon simulations on complex, irregular domains) classical GPU solvers tend be comparably hard to implement and they do not scale very well, thus many packages do not provide such support. As an example of general-puue purpose solver with partial GPU support, ANSYS shows limited speedups of 2x-4x on GPU, even under optimal conditions [30, 14]. On the other hand, evaluating our model on the same CPU hardware as the ground truth solvers, still achieves speedups between 4x-22x, even in this setting which is sub- optimal for NN models.
In practice, using a single GPU, we see speedups of 11x-290x compared to AreSim, COMSOL and SU2, and users of such simulators with access to GPU could benefit from these speedups.
A.5.2 ERROR METRICS
Rollout RMSE is calculated as the root mean squared error of the position in the Lagrangian systems and of the momentum in the Eulerian systems, taking the mean for all spatial coordinates, all mesh nodes, all steps in each trajectory, and all 100 trajectories in the test dataset. The error bounds in Ta- ble I and the error bars in Figure 5(a-c) indicate standard error the RMSE across 100 trajectories. Error bars in Figure 5(d) correspond to min/median/max performance across 3 seeds.
In FLAGSIMPLE and MAAC we observed decoherence after the first steps (Figure 5b), due to the chaotic nature of cloth simulation. Since the dynamics of these domains are stationary, we use the rollout error in the first 50 steps of the trajectory for the comparison shown in the bar plots, more discerning metric for result quality. However, the reported trends also hold when measured over the whole trajectory.
In AIRFOIL, compute the RMSE in region of interest around the wing (Figure A.1 middle), which corresponds to the region shown in figures and videos. For comparisons with grid- methods, we map the predictions the grid to the ground truth mesh to compute the error.
17

---

Published as a conference paper at ICLR 2021
0.10
0.30
Flagbnamicc
FishFlag
50
INAGE talbus--11
0.25
WineSork
FlagDynamic [Train]
0.08
0.20
xx
0.15
0.06
0.05
2500
5000
7500
10000
12506 15600 17500 nodes
20000
2500
5000 7500 10000 12500 15000 17560 20000
nodes
Figure A.3: A model trained on the regular-sized FLAGDYNAMIC domain was run on variants of FLAGDYNAMIC, WINDSOCK, FISHFLAG with different scale and resolutions. We show the RMSE for 50-step (left) and full-trajectory rollout (right) as a function of the simulation node count.
A.5.3 ADDITIONAL ANALYSIS ON GENERALIZATION AND SCALING
We inference of model trained the FLAGDYNAMIC domain (with learned remeshing), several scaled-up and scaled-down versions of FLAGDYNAMIC, and the generalization experiment WINDSOCK and FISHFLAAG (see Section 5). In Figure A.3, report the error compared to the respective ground-truth simulations.
When evaluating the 50-step RMSE rollout in FLAGDYNAMIC we do not observe systematic trends of the error as function of the simulation size, indicating that the model performs similarly well on larger and smaller systems. The error when generalizing new shapes (WINDSOCK, FISH- FLAG) is slightly higher, but comparable.
The RMSE rollout errоr evaluated the full trajectory shows a stronger correlation with the system size. However, we believe this simply tracks the systematic positional error incurred due to decoher- ence (e-g. small angle perturbation due decoherence incurs higher positional error at the tip of the flag the larger the flag is), and as shown in Figure 5b, decoherence becomes the main source of error after the first steps the simulation in this domain.
18


Reviews:
[{"review_id": "yohwDR4-Z-D", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\n\nThe paper presents a scheme for applying graph networks to surface and volume meshes for simulation of fluid flow and structural analysis.\nTo this end, the method adds problem specific positional features to the mesh nodes and uses a message passing approach to compute the desired outputs.\nFor Lagrangian systems the discretization is augmented by adding additional edges between nodes that fall below a distance threshold.\nThe experiments demonstrate better performance than grid-based or other graph-based approaches on the selected problems.\n\n\nScore:\n\nI am on the fence here. I think the paper has some good results but only scratches some important questions with respect to the mesh.\nFurther, claims like computational efficiency are only weakly supported.\nI hope my questions and concerns in can be addressed.\n\n\nPros:\n\n* The videos are good and show the stability of the method and the generalization extent.\n\n* The network architecture is straightforward and simple. \n  Learning the sizing field is a good addition and a good future direction for adapting the mesh.\n\n* The experiments show well the advantages of adaptive grids versus regular grids and meshless methods.\n\n\nCons:\n\n* The claim that this approach can be used to speed up simulations is not well supported.\n  Table 1 gives the inference time per frame but does not give any information about the hardware used in the experiments.\n  (The appendix mentions a V100 GPU but I could not find information about the CPU which is used for the GT simulation.)\n  The comparison would also be more interesting if GPU solvers would be added to the comparison.\n  As an alternative the graph network can be run on the CPU.\n\n* The appendix contains some information about the remesher for the cloth simulation but apart from that meshing is not studied in the paper despite being crucial for this method.\n  The meshes in the datasets use triangles and tetrahedra but another possible choice can be quads and hexahedra.\n  I would be valuable to know if the proposed method can generalize to this, needs to be trained for this specifically or does not work at all in this case.\n  A similar direction would be to experiment with badly shaped elements that are large but have small area/volume to check if the learning approach is more robust.\n\n* The related work should be extended with more works that share similar goals (e.g. improving runtime).\n  - Capuano and Rimoli, \"Smart finite elements: A novel machine learning application\" Computer Methods in Applied Mechanics and Engineering 2019.\n  - Luo et al., \"NNWarp: Neural Network-Based Nonlinear Deformation\" IEEE Transactions on Visualization and Computer Graphics 2018.\n\n\nQuestions:\n\nThe description of the world-space edges is not clear to me. Since r_w is an absolute threshold, can this lead to crossing edges in extreme situations?\n\nThe \"Computational efficiency\" section mentions that the network can do much larger time steps. \nWhat is the time step used during inference for the network? What is the time step for the classical simulator?\n\nCan you give a bit more information about the elements used in the DeformingPlate dataset? Are these linear elements?\n\nThe flag videos show generalization with respect to rotation about the gravity axis.\nIs the network solution invariant with respect to all 6 DOF?\n\nWhat is the memory footprint of the method? Can problems with millions of nodes be simulated on a recent GPU?\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "rYfXQIaTDlc", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 9, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "- Summary\n\nThis paper presents a graph-network-based architecture for learning to perform mesh-based simulations, which can be run more efficiently than the full, \"ground-truth\" simulations. The experiments demonstrate that the proposed method is able to learn to simulate a wide range of different physical scenarios. Moreover, the presented results also demonstrate an ability to generalize to configurations different from the one seen in training.\n \n\n- Pros\n\nThe paper is clearly written and strikes a good balance of presenting the relevant information in its main body and including the necessary details in the supplementary material. The additional videos provided are helpful in demonstrating the model's capabilities. \n\nThe experiments performed are comprehensive, with comparisons to a wide variety of significant baselines (e.g., other graph neural networks, grid based methods, etc.) and evaluations of the impact of diverse hyperparameters.\n\nMost importantly, the results presented are strong, containing both impressive qualitative demonstrations and good quantitative performance when compared to baselines. \n \nThe ability to operate directly on meshes, in contrast to previous work that operated on structured grids, makes this method more relevant to real-world applications.\n\n\n- Cons\n\nThe proposed method is, conceptually, a straightforward evolution of previously proposed graph-network-based methods for physics simulations, with most notable similarities to Sanchez-Gonzalez et al. (2020) ([33] in-paper reference).\n\nThe ability to generalize to higher-dimensional meshes is only briefly (quantitatively) evaluated (more on this in the comments below).\n\n\n- Reasons for score\n\nOverall, given the \"pros\" described above, notably the strength of the results presented on a wide variety of tasks, I recomend this paper for acceptance. \nAs mentioned above, the method proposed is a modification of a graph-network based method previous employed to simulate particle dynamics. In order to work on meshes, some modifications are proposed. such as adding mesh- and world-space connections to the graph, learning to predict the S matrix to remesh, etc. As such, this works stand on the strength of the practical results it achieves. The results demonstrate an impressive ability of the proposed method to learn to capture both Eulerian and Lagrangian mesh simulations. The wide diversity of tasks for which the model works well is unprecedented. The experimentation is extensive, with comparisons to significant baselines and evaluations of the relevance of different hyperparameter choices.\n\nMoreover, methods that learn to simulate physical processes more efficiently, and applications of graph neural networks are two research directions that have garnered a lot of interest in recent work. As an intersection of these two areas this paper should be of interest to a wide audience in the conference.\n \n\n- Additional comments\n\nOne important drawback of these types of physical simulation methods, which require training on ground truth data, is that training itself requires a large amount of data from the very process we want to simulate. If the learned model only presents a limited ability to generalize, this can severely limit the applicability of the method. This does not seem to be the case here, as the experiments seem to demonstrate an good ability to generalize to unseed scenarios, provided in part by the ability of the graph neural network to learn local/scale-invariant interactions.\nIn this direction, an important capability of the model, suggested in the paper in the \"tassel sub-experiment\" of the FlagDynamic experiment, is the ability to generalize to much larger meshes than the ones seen in training. As mentioned in the \"cons\" section above, however, this ability is only briefly (qualitatively) evaluated. It would be interesting to see a more thorough evaluation of this ability, due to its potential large importance to practical applicability, by allowing the model to be trained on smaller examples and then scaled up at inference time.\nMoreover, a thorough analysis of the error trade-offs when moving from smaller to larger meshes would also be interesting.\n\nIn the section on \"Key hyperparameters\", within the experimental results, it is claimed that \"the model performs best given the shortest possible history ..., with any extra history leading to overfitting. This differs from GNS [33], which used $h \\in 2 \\dots 5$ for best performance.\" Do the authors have any hypothesis as to why this might be the case here, versus the observation in [33]?", "extra_scores": {}, "extra_text": {}}, {"review_id": "T517x8jZW5I", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 10, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a generic strategy to accelerate time integration of physical systems, including elasticity, fluid simulation, and cloth simulation. The results are impressive not only for  the generality, but because this is the first paper that I read that shows a fair speedup combined with stability for such complex physical systems. I wish the authors attached the source code so that I could analyze it more closely to make sure that no shortcuts were taken in the evaluation.\n\nAssuming that the evaluation is correct, this is a seminal paper that will allow to use data-driven methods in scientific computing applications.\n\nComments:\n- the entire discussion about adaptive refinement seems out of place. Would the method also work without and is this an extra feature? Or is it necessary to use this feature to make it work? My understanding is that this is optional, and only helps, in particular for cloth. Can you confirm that this is the case?\n- I do not understand the sentence: “Each output feature is then processed by a forward-Euler integrator with ∆t = 1”. It only becomes clear after reading section 4. Please clarify earlier\n- Why is the model stable for hundreds of steps? Is this due to training with noise? Please do experiment to try to identify which feature makes it stable. Will it eventually break if you let the system run for thousands of frames\n- while the learning baseline are clearly described, I do not understand how the timings are measured for the GT in table 1. Is every row computed with a different software, as listed in A.1? If so, are all of these using a single core or multiple? This should be clarified, especially since the machine learning results are run on a gpu, making the comparison not fair. Could you report also the timing for doing the prediction of the NN using a single CPU core, or alternative run the iterative solvers of the classical methods on the same GPU?\n- Is there a case where you could notice that the integration is not stable? Maybe by reducing the noise in the training or by reducing the training data size or by simulating scenes far from the training data? It would be interesting to know what the limitations of the method are and where there is room for improvement.\n- Could you add a simple nearest neighbor baseline?\n- It would be interesting to add a comparison of running time *for a given accuracy* instead of simply showing what is the error in the table. This could be achieved running many simulations with analytic methods until they match the error of the data-driven model wrt a finely discretized ground truth.\n\nOverall, my impression is that the results look amazingly good: it is the first paper I read that shows that data-driven method can beat standard scicomp approaches on complex scenes. The reason for the performance of  the methods likely lies in the combination of using mesh and space variables, plus making the integration step explicit and enriching the training with controlled noise. I wish the authors attached the source code of  their method and of baselines (and I encourage them to do it after publication), I look forward to experimenting with this method and test it on my applications. \n\nUpdate:\nThe revision further improved the paper and addressed most of my comments. I am still positive and voting for accepting this paper.", "extra_scores": {}, "extra_text": {}}, {"review_id": "Q3-NKyTGlKd", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Contributions\n\nThe authors present a graph convolution architecture suitable for learning physical simulation on meshes.\nThis method supports dynamically re-meshing to locally adapt to the complexity of the simulated physics, and is stable to relatively long rollouts of physical simulations.\n\nStrong points\n\nThe architecture presented is generic and seems suitable for a wide variety of physical tasks. This point is supported by multiple experiments on diverse tasks.\nGetting a second architecture to learn the sizing field is very effective. The authors demonstrate a way to supervise this second network even when the sizing filed is not exposed by the ground truth physical simulator. This way is experimentally and quantitatively demonstrated to be robust. As a consequence, the whole architecture remains generic enough to be applicable to emulation tasks even when the simulator is not providing an explicit sizing tensor.\n\nThe proposed architecture is surprisingly stable in long rollouts, and does not diverge nor accumulates errors from previous steps. As noted in the Appendix, robustness is enforced via a noisy training scheme from previous literature. Thanks to its convolutional architecture and positional encoding, it can be trained on small systems that are inexpensive to simulate and then later be applied to larger, more complex and fine grained problems. This has the potential to enable fast physical emulation without huge training costs.\nAdding one-hot encoding allows to model complex interactions between physical systems, such as an actuator using on an elastic plate (DeformingPlate experiment).\n\nFinally, comparison against existing methods (based on CNN, graph convolutions or particle-based representation) seems favorable to the presented technique.\n\nThe paper is well written. Linking to anonymised videos showing experiment results is well done and provides an enjoyable reading experience.\n\nWeak points\n\nAuthors do not provide ablations studies on the main novelties of their graph convolution architecture. A fully grounded paper would show the quantitative benefits of:\nAdding world edges in addition to existing mesh edges\nAdding labels on edges that are initialised with distances\nDynamically remeshing at each step: what happens if you just do not re-mesh? Fig. 3b provides an intuition for it, but I cannot find any quantitative study of it. More than that: in table 1, FlagSimple (fixed meshing) metrics are comparable or better than FlagDynamic. This seems to suggest re-meshing is not useful in this case.\nThe above 3 points are really what sets this paper apart from the rest of the literature on graph convolution nets. Seeing how they impact results seems mandatory to assess how MeshGraphNet is more than just an existing mesh convolutional network architecture plugged into a physical problem.\n\nAbout benchmarks: it is not clear why the concurrent method GCN was not taken in full. Its super-resolution part was removed for the purpose of this submission, but it seems to be inherently part of it. Removing this component is a-priori an alteration of its architecture, and the reasons for discarding it are not justified.\nMoreover, in comparing to GCN the authors conclude that the key reason for MeshGraphNet to outperform GCN is the positional encoding added to edges, but this claim is not supported experimentally. This could either be done by removing it from MeshGraphNet, or somehow add something similar to GCN.\n\nSince GCN seems good at predicting static values but very quickly diverge for longer rollouts, the two tasks seem quite different. As noted in the Appendix, training noise was added to make the presented method robust to longer rollouts. A fair comparison with GCN would also include this.\n\nGeneralization is mostly demonstrated qualitatively : the difference between the training/testing scenarios are not stressed enough - apart for the impressive extrapolation case from Airfoil. For example, the generalisation on FishFlag remains qualitative, but is not quantitatively assessed, and similarly for the Windsock. “Visual similarity” is not 100% satisfying.\n\nRecommandation\n\nWeak accept: I recommend to accept the paper for the promising results it displays, but would like to see the missing quantitative results I mention above addressed - ablation study and experiments.\n\nQuestions\n\nIn grossly in decreasing order of relevance (first ones = useful to assess the paper / last ones = personal curiosity):\n1 - Why was the super-resolution module of GCN removed?\n2 - Varying wind speeds/directions on the flag experiment: since each time step is computed separately, to what extent does it really demonstrate generalisation to vary wind direction at each tilmestep, knowing that they were each independently seen during training?\n3 - On the Windsock experiment demonstrating generalisation: how much is due to the architecture (convolutions that can by nature be applied to bigger graphs) and how much the network has “learned” the physics?\n4 - In Table 1, what is causing t_full to be strictly larger than t_model for the FlagSimple experiment, where no graph re-meshing or re-computation is happening?\n5 - What is the chosen time step for 1 prediction? Since it is fixed, how was it chosen?\n6 - How are mesh space coordinates u_i’s attributed? Does it matter ?\n7 - Does jointly predicting related physical values help? This is something usually observed in the literature, but did you observe similar results? For example in the CylinderFlow experiment you have one network predicting the change in momentum and the pressure field. Does it degrade RMSE to get 2 networks, one for each of these quantities?\n\nAdditional recommandations\n\nClarity could be improved. I make the following remarks, that did not influence on my rating of the paper:\nSection 3.1 “Encoder” : I would split the encoding part into two, to clearly mark the difference between initial values provided to the network : (A) one is defined by physical quantities that you provide as input to the mesh vertices (you denote it q_i). The other (B) is more of an internal, method-specific pre-processing step and consists in creating world edges, and labelling world and mesh edges with their length. Presenting (A) and (B) in the same paragraph can be confusing, since they really are not on the same level of processing. (A) is the real, physical input, (B) is already a pre-processing step.\nSection 3.1 “Processor” : why denote GraphNet blocks as P_i if it not mentioned anywhere else? Moreover, using _i as an index is confusing since it is elsewhere reserved to graph node indices.\nHistory size h is mentioned in the introduction of 3.1, in 4 (FlagDynamic and SphereDynamic) and in Fig. 5d, but does not fit the formalism presented in 3.1. It should  be presented in a way that accommodates for the possibility of feeding more than just the previous step’s information.\nSection 4: could be improved with regards to what is provided as input to the network. For example, u_i is always provided as input, so why repeat it? As far as I understand it, what matters is what you input as physical properties q_i, and what/how you label n_i. In this section q_i are well defined, but for some experiments I have trouble seeing what you encode into n_i - it is only explicated for DeformingPlate.\nSection 4: all experiments and datasets are described except for FlagSimple. Is this an oversight?\nGlobally, clarifying the formalisation (3.1) and values (4) of the network’s inputs would help.\n\nTypos or minor improvements:\n3.2, top of page 5: “iff” > “if and only if”\nEnd of A.1 page 13: “… is appended TOO the node” > “to the node”", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper develops a methodology for using graph neural networks for mesh-based physics simulation. This extends recent work that focused on grids or particles to mesh-based domains, which are challenging due to irregular (and possibly changing) connectivity. The reviewers had some concerns but recognized that this is an important work that will be of broad interest and may have significant impact.
Decision: Accept (Spotlight)