MAgNet: Mesh Agnostic Neural PDE Solver
Authors: Oussama Boussif
Yoshua Bengio
Loubna Benabbou
Dan Assouline
Venue: NeurIPS 2022 Accept (2022)
Keywords: physical simulations
implicit neural representations
graph neural networks
learned simulators

Abstract: The computational complexity of classical numerical methods for solving Partial Differential Equations (PDE) scales significantly as the resolution increases. As an important example, climate predictions require fine spatio-temporal resolutions to resolve all turbulent scales in the fluid simulations. This makes the task of accurately resolving these scales computationally out of reach even with modern supercomputers. As a result, current numerical modelers solve PDEs on grids that are too coarse (3km to 200km on each side), which hinders the accuracy and usefulness of the predictions. In this paper, we leverage the recent advances in Implicit Neural Representations (INR) to design a novel architecture that predicts the spatially continuous solution of a PDE given a spatial position query. By augmenting coordinate-based architectures with Graph Neural Networks (GNN), we enable zero-shot generalization to new non-uniform meshes and long-term predictions up to 250 frames ahead that are physically consistent. Our Mesh Agnostic Neural PDE Solver (MAgNet) is able to make accurate predictions across a variety of PDE simulation datasets and compares favorably with existing baselines. Moreover, our model generalizes well to different meshes and resolutions up to four times those trained on.

MAgNet: Mesh Agnostic Neural PDE Solver
Oussama Boussif
Dan Assouline Mila Québec Institute DIRO, Université de Montréal d www.alsseeeeemmmmmm.oo quueeeeeee..u.e.eeeeeee...e
Mila Québec AI Institute
DIRO, Université de Montréal
oussama.. boussif fwwfffmmll...mmmml @mila.quebec
Loubna Benabbou Université du Québec à Rimouski
Yoshua Bengio*
Mila Québec AI Institute DIRO, Université de Montréal
Buubna_Benabbou@ugarr ca
yoshua bengeeeiiiiiieeeeeee quebbc
Abstract
The computational complexity of classical numerical methods for solving Partial Differential Equations (PDE) scales significantly as the resolution increases. As an important example, climate predictions require fine spatio-temporal resolutions resolve all turbulent scales in the fluid simulations. This makes the task of accurately resolving these scales computationally out of reach even with modern supercomputers. As a result, current numerical modelers solve PDEs on grids that are too coarse (3km to 200km on each side), which hinders the accuracy and usefulness of the predictions.
this paper, we leverage the recent advances in Implicit Neural Representations (INR) to design novel architecture that predicts the spatially continuous solution of PDE given spatial position query. By augmenting coordinate- based architectures with Graph Neural Networks (GNN), we enable zero-shot generalization to new non-uniform meshes and long-term predictions up 250 frames ahead that are physically consistent. Our Mesh Agnostic Neural PDE Solver (MAgNet) able make accurate predictions across variety of PDE simulation datasets and compares favorably with existing basclines. Morcover, MAgNet generalizes well different meshes and resolutions up to four times those trained or
1 Introduction
Partial Differential Equations (PDEs) describe the continuous evolution of multiple variables, e.g. over time and/or space. They arise everywhere in physics, from quantum mechanics to heat transfer and have several engineering applications in fluid and solid mechanics. However, most PDEs can't be solved analytically, so it is necessary to resort to numerical methods. Since the introduction of computers, many numerical approximations were implemented, and new fields emerged such as Computational Fluid Mechanics (CFD) [Richardson and Lynch 2007). The most famous numerical approximation scheme the Finite Flement Method (FFM) (Courant) 1943 Hrennikoff, 194T). the FEM, the PDE is discretized along with its domain, and the problem is transformed into solving set of matrix equations. However, the computational complexity scales significantly with the resolution. For climate predictions, this number can be quite significant if the desired error is to be reached, which renders its use impractical.
"CIAAR Senior Fellow
"Code and dataset can be found on: httpss///gtthub..cc//////oo/oo/r
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

In this paper, we propose to learn the continuous solutions for spatio-temporal PDEs. Previous methods focused on either generating fixed resolution predictions or generating arbitrary resolution solutions on fixed grid (Li et al., 2021 Wang et al. 2020). PDE models based on Multi-Laye Perceptrons (MI.ss) can generate solutions at any point of the domain (Dissanayake and
1994 Lagaris et al. 1998| Raissi et al. 20] However, without imposing a physics-motivane that constrains the predictions to follow the smoothness bias resulting from the PDE, MLPs become less competitive than CNN-based approaches especially when the PDE solutions have high-frequency
loss
information (Rahaman et al. 2018).
We leverage the recent advances in Implicit Neural Representations (Tancik et al. 2020). CChnn et al., 2020). (Jiang et al. (020)) and propose general purpose model that can not only learn solutions to a PDE with a resolution it was trained on, but it can also perform zero-shot super-resolution on irregular meshes. The added advantage is that we propose a general framework where we can make predictions given any spatial position query for both grid-based architectures like CNNs and graph-based ones able to handle sensors and predictions at arbitrary spatial positions.
Contributions Our main contributions are in the context of machine learning for approximately but efficiently solving PDEs and can be summarized as follows:
We propose a framework that enables grid-based and graph- based architectures to generate continuous-space PDE solutions given a spatial query at any position.
We show experimentally that this approach can generalize to resolutions up to four times those seen during training in zero-shot super-resolution tasks.
2
Related Works
Current solvers can require a lot of computations to generate solutions on a fine spatio-temporal grid. For example, climate predictions typically use General Circulation Models (GCM) to make forecasts that span several decades over the whole planet (Phillips) 1956). These GCMs use PDEs to model the climate in the sumnosphere-ocea--leee solvers are used. However, the quality of predictions is bottlenecked by the grid resolution that is in turn constrained by the available amount of computing power. Deep learning has recently emerged as an alternative these classical solvers in hopes of generating data-driven predictions faster and making approximations that do not just rely on lower resolution grids but also on the statistical regularities that underlie the family of PDEs being considered. Using deep learning also makes possible to combine the information in actual sensor data with the physical assumptions embedded in the classical PDEs. All of this would enable practitioners to increase the actual resolution further for the same computational budget, which in turn improves the quality of the predictions.
system and to solve these PDEs, classical numerical
Machine Learning for PDE solving: Dissanayake and Thiem (1994) published one of the first papers on PDE solving using neural networks. They parameterized the solutions to the Poisson and heat transfer equations using an MLP and studied the evolution of the error with the mesh size. Lagaris et al. (1998) used MLPs for solving PDEs and ordinary differential equations. They wrote the solution as a sum of two components where the first term satisfies boundary conditions and is not learnable, and the second is parameterized with an MLP and trained to satisfy the equations. In Raissi et al. (2017) 7a) the authors also parameterized the solution to a PDE using an MLP that takes coordinates as input. With the help of automatic differentiation, they calculate the PDE residual and use its MSE loss along with an MSE loss on the boundary conditions. In follow-up work, Raissi et al.. (2017h) also learn the parameters of the PDE (e.g. Reynolds number for Navier-Stokes equations).
The recently introduced Neural Operators framework (Kovachki et al. 2021 Li et al. 2020b a) attempts to learn operators between spaces of functions. Li et al. (2021) use 'Fourier Layers" to learn the solution to PDE by framing the problem as learning an operator from the space of initial conditions to the space of the PDE solutions. Their model can learn the solution to PDEs that lie on a uniform grid while maintaining their performance in the zero-shot same spirit, Jiang et al. (2020) developed model based on Implicit Neural Representations called "MeshFrees FresooFeeel where they upsample existing PDE solutions to a higher resolution. They use 3D low-resolution space-time tensors as inputs to a 3DUnet in order to generate feature map. Next, some points are sampled uniformly from the corresponding high-resolution tensors and fed to an MLP called ImNet (Chen and Zhang 2018). They train their model using a PDE residual loss and
resolutioo setting. In the
2

---

Exckkr
Sparta
MSP
Interpobation
Figure 1: We illustrate the Mnicode-Interpolate Forecast" framework of MAgNet. The parent mesh is fed to the encoder to generate the parent mesh embedding. Next, we estimate the values at the spatial queries using the interpolation module that uses features from both the parent mesh points and the parent mesh embedding points closest to these queries. Finally, the parent mesh observations and interpolated values at spatial queries are gathered as nodes forming a new graph using nearest neighbors and the PDE solution is forecast for all nodes (therefore all spatial locations) into the future using the forecasting module.
are able to predict the flow field at any spatio-temporal coordinate. Their approach closest to the one we propose here. The main difference is that we perform super-resolution on the spatial queries and forecast the solution to a PDE instead of only doing super-resolution on the existing sequence.
Brandstetter al. (2022) use the messagg-passing paradigm ((Gilmer et al. 2017), (Watters et al., 2017). (Sanchez-Gonzalez et al., 2020)) to solve 1D PDEs. They are able to beat state-of-thh-aa Fourier Neural Operators (Li et al. 102I and classical WENO5 solvers while introducing the "pushforward trick" that allows them to generate better long-term rollouts. Morcover, they present an added advantage over existing methods since they can learn PDE solutions at any mesh. However, they are not able to generalize to different resolutions, which a crucial capability of our method.
Most machine learning approaches require data from a simulator in order to learn the required PDE solutions and that can be expensive depending on the PDE and the resolution, Wandel et al. (0200) alleviate that requirement by using PDE loss.
Machine Learning for Turbulence Modeling: Recent years have known surge in machine learning-based models for modeling turbulence. Since it is expensive to resolve all relevant scales, some methods were developed that only solve large scales explicitly and separately model sub-grid scales (SGS). Recently, Novati et al.] (22TT) used multi-agent reinforcement learning to learn the dissipation coefficient of the Smagorinsky SGS model (Smagorinsky. (963) using as reward the recovery of the statistical properties of Direct Numerical Simulations (DNS). Rasp et al. (2018) used MLPs to represent sub-grid processes in clouds and replace previous parametrization models in a global general circulation model. In the same fashion, Park and Choi (202I) used MLPs to learn DNS sub-grid scale (SGS) stresses using as input filtered flow variables in a turbulent channel flow. Brenowitz and Bretherton (2018) use MLPs to predicts the apparent sources of heat and moisture using coarse-grained data and use multi-step loss to optimize their model Wang et al.) (2020) used one-layer CNNs to learn the spatial filter in LES methods and the temporal filter in RANS well as the turbulent terms. A UNet (Ronneberger et al. 2015) is then used as a decoder to get the flow velocity. de Bezenac et al. (017) predict future frames by deforming the input sequence according to the advection-diffuuiii equation and apply it to Sea-Surface Temperature forecasting.
(Sanchez-Gonzalee et al. 2018 2020) paradigm along with dilated convolutional networks to capture turbulent dynamics seen high- resolution solutions only by training on low spatial and temporal resolutions. Their approach beats existing neural PDE solvers in addition to the state-of-the-art Athenat+ engine (Stone et al. 2020).
Stachenfeld et al. (202T) use the
We take inspiration of this approach but replace the process module by interpolation module, to allow the model to capture spatial correlations between known points and new query points.
3 Methodology
We present the developed framework that leverages recent advances Implicit Neural Representations
(INR) (Jiang et al. 2020 Sitzmann et al. 2020 Chen et al. 2020 Tancik al. 2020) and draws
3

---

inspiration from mesh-free methods for PDE solving, We first start by giving a mathematical definition of PDE. Next, we showcase the proposed "MAgNet" and derive two variants: grid- based architecture and graph-based one.
3.1 Preliminaries
We define PDE as follows, using D* denote k-th order derivatives:
Definition 3.1. Evans 2010)) Let U denote open subset of R" and k 2 1 an integer. An expression of the form:
C(D²u(x)) D
u(x), (())
Vx EU
is called k-th order system of PDEs, where C Rmnn x Iman" x... X Im T x U - 1
is given and u U + "R, (u' (u",...,"") u") is the unknown function to be characterized.
interested in spatio-temporal PDEs. In this class of PDEs, the domain is U' = +oo x S (time x space) where S C IR" n n 1 and, with D indicating differentiation w.r.t 2t, any such PDE can be formulated as:
In this paper,
tu L(D*u(x)) .... ())),,,)) VI 2 0, Vo E. u u(0, x) g(x) Bu = 0
Ve
Vt 2 0, Vo E as
Where aS is the boundary of S, B is a non-linear operator enforcing boundary conditions on u and 9 S. Rm represents the initial condition constraints for the solution u.
Numerical PDE simulations have enjoyed great body of innovations especially where their use paramount in industrial applications and research. Resh-based methods like the FEM numerically compute the PDE solution on a predefined mesh. However, when there are regions in the PDE domain that present large discontinuities, the mesh needs to be modified and provided with many more points around that region order to obtain acceptable approximations. Mesh-based methods typically solve this problem by re-meshing in what is called Adaptive Mesh Refinement (Berger and Oliger 1984 Berger and Colella 1989). However, this process can be quite expensive, which is why mesh-free methods have become attractive option that goes around these limitations.
3.2 MAgNet: Mesh-Agnostic Neural PDE Solver
3.2.1
"Ennodd-IIneepollteeForeee framework
Let (I1, x2,...... FT E CCN denote a sequence of T frames that represents the ground-truth data coming from PDE simulator or real-world observations. C' denotes the number of physical channels, that is the number of physical variables involved the PDE and N is the number of points in the mesh. These frames are defined the same mesh, that is the mesh does not change in time. We call that mesh the parent mesh and denote its normalized coordinates of dimensionality n by {pi)isss N E [-1, 1]" Let {c; JessissMM E [-1, 1J" denote set of M coordinates representing the spatial queries. The task is to predict the solution for subsequent time steps both at: (i) all coordinates from the parent mesh (p:isse and (ii) coordinates from the spatial queries (c, Jissssa as At test time, the model can be queried at any spatially continuous coordinate within the PDE domain to provide estimate of the PDE solution at those coordinates.
To perform the prediction, we first estimate the PDE solutions at the spatial queries for the first T frames and then use that to forecast the PDE solutions at the subsequent timesteps at the query locations. We do this through three stages (see Figure TT:
1. Encoding: The encoder takes as input the given PDE solution {=th each point of the parent mesh {p: hisgN and generates a state- rnteerepresentatioon of original frames, which can be referred as embeddings, and which we note {=hsigg. This representation will be used in the interpolation step to find the PDE solution at the spatial queries {c: hisis ar- Note that in this encoding step, we can generate one embedding for each frame such that we have T embeddings or summarize all the information in the T frames into one embedding. We will explain the methodology using T embeddings, as is easier to grasp the time dimension in this formulation, but the implementation has been done using a summarized

---

single embedding, mentioned in section 3.2.2 remains the same, i.c. we don't 't change it by upsampling or downsampling it.
We also note that the embedded mesh
2. Interpolation: We follow the same approach as Jiang et al. (220) and Chen et (2020) by performing an interpolation in the feature space. Note that in case we generate one representation that summarizes all T frames into one, then 2 for 1...., T. Let {4 hgAgT denote the timesteps at which the are generated.
For each spatial query C₁ let ((c;)) denote the nearest points in the parent mesh py. We generate an interpolation of the features 2n sn|cc] at coordinates e; and at timestep & as follows:
Ak E (1,T),iE (1,M) 2x[e Emsevios Wygo (2x (P5) su(p,l Pz. tu)
(3)
EpjeA(ca) Wj
Where =x[pj] and na [P]] denote the embedding and input frame at position Pj and time la respectively. Moreover, wj are interpolation weights and are positive and sum to one. Weights are chosen such that points closer to the spatial query have a higher contribution to the interpolated feature than points farther away from the spatial query. The go is MLP. To get the PDE solution [x[c]] at coordinate C₁, we use decoder de which is an MLP here: rx/c,] = do (c)(cc)). In practice, the number of neighbors that we choose is 2n where is the dimensionality of the coordinates.
3. Forecasting: Now that we generated the PDE solution at the spatial queries c, for all the past frames, forecast the PDE solution future time points at both spatial queries and the parent mesh coordinates. Let G denote the Nearest-Neighbors Graph (NNG) that has as nodes all the N locations in the parent mesh (at original coordinates {p, (P.Is all the M query points (at locations {ch with edges that include only the nearest neighbors of each node among the N M others. This corresponds represented by the graph G. Let {4] denote the corresponding new coordinates. We generate the PDE solution for subsequent time steps this graph auto-regressively using decoder Ag as follows:
well as
mesh
1 [4] = xxIck + (fx+1 -
T 1,
(4)
We train MAgNet by using two losses:
Interpolation Loss: This loss makes sure that the interpolated points match the ground-trut truth and is computed as follows:
| [cs] - sx =<(c;]]]
I innterpolation
(5)
Tx M
Where d' [cx] denotes the interpolated values generated by the model the spatial queries. Forecasting Loss: This loss makes sure that the model predictions into the future accurate. If H is the horizon of the predictions, then we can express the loss as follows:
M+N En 1 -aarrc]]=xt+llee xxtT<4]1 ΣN
L Loreeasting
(6)
H (M N)
Where Tx+T[c] denotes the forecasted values generated by the model at the graph G which combines both spatial queries and the parent mesh
The final loss is then expressed as:
L
Lorecasting
Linterpolation-
3.2.2 Implementation Details
In the previous section, we described the general MAgNet framework. In this section, we present how we build the inputs to MAgNet as well as the architectural choices for the encoding, interpolation and forecasting modules and suggest two main architectures: MAgNet[CNN] and MAgNet[GNN]..
5

---

Starting
Sample number points randonly to form the parent mesh. will be the spatial queries
From spatial queries, randornly sample a number of training spatial queries, from
Training spatial queries sequence of frames another.
Figure We illustrate the data pre- preproceessin pipeline. We sample points randomly from the starting mesh to form the parent mesh and the remaining points form the spatial queries. Next, during training, we can sample from the spatial queries and form what we call "training spatial queries". The distinction is that the number of "training spatial queries" can be less than the total number of spatial queries and we investigate the impact of this number in Section 4.3
Data pre-processing: We first consider a mesh that contains N' points (N' 2 N). We randomly sample N points from the mesh to form the parent mesh. During training, M spatial queries are randomly sampled from the N' N remaining points. We tried multiple values of M (that the number of training spatial queries) to assess its impact on the performance of the method within
sensitivity study presented in Section 4.3 The data pre- pre-processing is illustrated in Figure 2
MAgNet : In this architecture, we follow Chen et al. (2020) and adopt the EDSR architecture (Lim al., 2017 as our CNN encoder. We concatenate all frames {z, JeSsT the channel dimension and feed that to our encoder in order to generate single representation 2. For the forecasting module, we use the same GNN in (Sanchez- Gooballz- al.) 2020). A key advantage of this architecture is that it effectively turns existing CNN architectures into mesh-agnostic ones by querying them at any spatially continuous point of the PDE domain at test time.
This model is similar to MAgNei[CNN] except that instead of using a CNN as an encoder, use GNN: the same architecture as the forecasting module but each architecture having its separate set of parameters. This is better suited for encoding frames with irregular meshes. Similarly to MAgNet[CNNJ, we generate single representation that summarizes all the information from the frames (xchess.
MAgNet[GNNJ:
4 Results
In this section, we evaluate MAgNet's performance against the following baselines:
2021 Considered the state-of-the-ee model in neural PDE solving, FNO casts the problem of PDE solving as learning an operator from the space of initial conditions to the space of the solutions. It is able to learn PDE solutions that lie uniform grid and can do zero-shot super resolution.
Fourier Neural Operators (FNO) (Li et al.
Pessage-Passing Neural PDE Solvers (MPNN) (Brandstetter et al. 2022) Graph Neural
Networks have been used to learn physical simulations with great success Gonzalez et al. 2020). Recently, they have been used learn solutions to PDEs (Brandstetter et al. 2022 Sanchez Gonzalez et al. 2020). MPNN-based GNNs coupled with an autoregressive strategy demonstrate a superior performance to FNO and are able to make long rollouts with the help of the "pushforward- trick" that only propagates gradient of the last computed frame.
We evaluate all models on both 1D and 2D simulations (the datasets generated from ID and 2D PDEs are presented in Appendix [A.4p. All training sets the case contain 2048 simulations and test sets contain 128 simulations. For the 2D case, training sets contain 1000 simulations and test sets contain 100 simulations. All models are evaluated using the Mean Absolute Error (MAE) on the rolled out predictions averaged across time and space:

---

280
FNG
0.30
1065
0.25
6.20
0.20
0.2
6:15
0.10
0.05
200
300
(a) Zero-shot super-resolution per- (b) Zero-shot super-resolution per- (c) Zero-shot super-resolution per- formance on E1 test.
formance on E2 test.
formance on E3 test.
Figure 3: We present the models predictive performance on zero-shot super-resolution, with a training spatial resolution of nh = 50 for the regular grids. MAgNet[CNN] outperforms baselines on both E1 and E2 test sets but lags behind FNO on E3. Error bars represent one standard deviation in both plots.
DES Σ Ix [x,[cs] Rx[c (n+ T) N
MAE
Where n, the total number of frames and T the number of frames input to the models.
We train models for 250 epochs with early stopping with a patience of 40 epochs. See Appendix A.6 and A.7 for more implementation details.
We present a short summary of our results:
ID Case: Our model MAgNet is able to robustly generalize to unseen meshes in the regular case compared to the SOTA models FNO and MPNN. The performance is even better in the irregular case. The details of the results are given in the following section 4.1
2D Case:
Our model MAgNet is less competitive than MPNN when it comes to generalizing unseen meshes in the regular case. However, in the irregular case, more competitive especially when trained on sparse meshes. The results regarding the 2D PDEs are presented in section 4.2
4.1
1D Case
For all the subsequent sections, nh. and n'g denote the training and testing set's resolutions respectively. The temporal resolution 250 remains unchanged for all experiments.
4.1.1 General Performance and zero-shot super resolution On Regular Meshes
In this section, compare MAgNet's performance all three datasets. All models trained resolution of n = 50 and the PDE solutions lie on a uniform grid. We test zero-shot super-resolution on n'. E {40, 50, 100, 200}. Results are summarized in Figure and visualizations of the predictions can found in Appendix A.2 MAgNet[CNN] outperforms both baselines on both El and datasets yet is slightly outperformed by FNO on E3 (Figure 3c). Nonetheless, MAgNet[CNN]Js predictive performance stays consistent up to n' 200 while MPNN does not generalize well to resolutions not seen during training.
4.1.2 General Performance and zero-shot super-resolution Irregular Meshes
In this section, we study how MAgNets compare against the other baselines when it comes to making predictions on irregular meshes. In order to do so, we take simulations from the uniform-mest E1 dataset with resolution of 100 do the following:
Let n, E (30, 50, 70). Then, for each simulation in the E1 dataset, randomly sample the same subset of n, points: the mesh remains for each single simulation in the E1 dataset.
The procedure the same for the test set but we instead take the original E1 test set at a starting resolution of 200 and generate four test sets with irregular meshes for n2, {40, 50, 100, 200}. This
7

---

-40
50
-70
Model
100
IN 200
100
200
50
100
200
FNO
0.2264
0.2671
0.2574
0.2501
0.5797
0.3524
0.3541
0.3521
0.2798
012344
0.2533
0.3905
MAgNee[CNNI
0.2081
0.1924
0.2063
0.2150
0,1869
0.3630
0.1599
0,1429
0.2237
1.3654
0.1385
0.1324
MPNN
0.2641
1.1601
0,3451
0.3667
0.5027
0.2521
0.3226
0.324
0.2885
1.1541
0.3403
0.3570
MAgNe1[GNNI]
0.2422
0.2230
6.193M
0.1902
0.2302
0.3639
0.1590
0.1484
0.2400
1.3399
0.139M
0.1070
Table 1: We report the MAE per frame on the E1 dataset. We train all four models on three different resolutions n, {30, 50, 70} and for each training resolution, we evaluate zero-shot super-resolution on irregular meshes for n'z E {40, 50, 100, 200}. We notice that even when we use CNN encoder, MAgNet not only performs better than the existing baselines, but its performance stays consistent across different test resolutions. MAgNet with CNN encoder beats MPNN even when using an encoder not suited for the task, which suggests MAgNet successfully turns existing CNN architectures into mesh-agnostic ones.
different from the test set of the previous section albeit considering the same resolution since this one has irregular meshes. We summarize our findings in Table
MAgNer[GNN] performs better than MAgNet[CNN] on irregular meshes which expected since GNN encoders are better suited for this task. However, surprisingly, even though we use CNN encoder for MAgNet[CNN],, the performance seems be better most cases not only compared to FNO but also MPNN which is graph- based architecture. This effectively shows that MAgNet can be used to turn existing CNN architectures into mesh-agnostic solvers. This is particularly interesting for meteorological applications where one needs to make predictions at the sub-grid level (at a specific coordinate) while only having access to measurements on a grid.
4.2 2D case
In this section, we present results for the 2D PDE simulations. We use the datasets and B2 as our experimental testbeds (see appendix (A.4). We use Narrain to denote the train resolution and neasst for the test resolution. All models are fed a history of T 10 frames and are required to generate rollout of n2 T = 40 frames in the future.
4.2.1 General performance and zero-shot super resolution on regular meshes
In this section, compare MAgNet's performance both B1 and B2 datasets. All models were trained on a resolution of Nerrii = 64 and the PDE solutions lie on a uniform grid. Zero-shot super resolution is tested Niest {32 642, 128 256²). We summarize our findings in Figure 4 We notice that MAgNet[CNN] falls behind when it comes to making good predictions and zero-shot super-resolution both datasets while MAgNet[GNN] and MPNN take the lead. Suprisingly, FNO suffers when comes to generalizing to unseen resolutions. Indeed, leveraging interactions between points after the interpolation module allows MAgNet[GNN] not only to make good predictions but also to generalize to denser and unseen resolutions.
Visualizations of the prediction for sample for the B1 dataset for all test resolutions can be found in Figures
and 13 respectively the appendix.
0.125
0.125
MighertCCii MAJMAtIGNNN)
0.675
0.050
0.052
0.035
0.000
0.000
2567
(a) Results for the zero-shot super-resolution on the B1 (b) Results for the zero-shot super-resolution on the B2 datasct.
datasct.
Figure 4: Results for the zero-shot super-resolution setting for models trained in the regular setting on a resolution of Atrrinn 642. While FNO and MAgNet[CNN] fall behind, MPNN and MAgNet[GNN] take the lead by leveraging the message-passing paradigm of Graph Neural Networks (Brandstetter al. 2022
Gilmer al. 2017)
8

---

4.2.2 General performance and zero-shot super resolution on irregular meshes
In this section, we present results for irregular meshes. We consider two settings:
Uniform: N nodes are randomly and uniformly sampled from regular grid of resolution 32
Condensed: N nodes are randomly sampled in non-uniform way from a regular grid of resolution 32 following the distribution: p(a,y) x exp (-8 ((x - 0.25)² + (y - 0.25)²)
We train MPNN and MAgNe([GNN] on four resolutions N E (64, 128, 256, 512] where again N is the number of nodes in the mesh for both Uniform and Condensed settings. For each training setting, we evaluate both models on regular grid PDE simulations from the B1 dataset. Figure shows the mesh nodes for the Uniform setting and Figure i9 for the Condensed setting.
Our findings are summarized in Figure We notice that our model MAgNei[GNN] is especially more appealing in case we have fewer points to work with. In the Condensed setting which is more realistic than the uniform one, MAgNei[GNN] is even more competitive than MPNN. We note however, that while MAgNet[GNNIT performance is better for small test resolutions, it quickly deteriorates test on higher resolutions which a main limitation our approach the 2D case.
0.200
Train Resolution=64
Train Resolution=122
Train Resolutton=2566
Train Resolution 512
0.175
MAgNe:[GNN]
0.150
0.125
0.100
0.075
0.025
0.000
32
64
128
256
32a
64
128
256
32
64"
128
256
32 64
128 128-2564
256
Test Resolution
(a) Results for the zero-shot super-resolution on the irregular B1 dataset for the Uniform setting.
0.200
Re
Resolution-126
Resolution=256
Resolution
0.175
MAgNet[GNN]
0.150
MPNN
0.125
0.100
0.075
0.050
0.025
0.000
32"
64'
128"256
128'
256
32d
64' 128
2562
32
64" 128' 256
32 64 128º
256
Resolution
(b) Results for the zero-shot super-resolution on the irregular B1 dataset for the Condensed setting.
Figure 5: Results for the zero-shot super-resolution setting for models trained in the irregular setting on resolutions of N E {64, 128, 256, 512}. MAgNet[GNN]] is especially appealing in case have fewer points and case the node distribution non-uniform. However, the performance gap reduces both models see more points during training. We note however, that MAgNes performance deteriorates as we test on higher resolutions compared to MPNN which is a main limitation of our approach in the 2D case.
4.3 Ablation study: Basic Interpolators vs Learned Interpolators
We investigate the contribution the interpolation module to the general predictive performance of MAgNet. We compare the MAgNet[CNN] architecture against three ablated variants:

---

We use K-Nearest-Neigghe interpolation (Qi et al. 2017 on the original frames directly to obtain the interpolated values at the spatial queries.
KNN:
Linear: We use Linear interpolation on the original frames directly to obtain the interpolated
values at the spatial queries.
Cubic: We use Cubic interpolation on the original frames directly to obtain the interpolated
values at the spatial queries.
Everything else is kept the same. The evaluation is done on the E1 dataset with regular meshes and a resolution of nt 50. Performance tested on E1 with regular meshes for test resolutions n', E (40, 50, 100, 200). Results are summarized in Figure and additional ablation studies can be found in the appendix A.1
0.16
Cubic
0.15
KNN
0.14
Ours
2033
0.12
0.11
0:30
75
125
150
200
Test Resolution
Figure 6: We study the impact of having learned interpolator (Ours) compared existing
interpolation schemes. Error bars represent one standard deviation.
5 Limitations and Future Work
In this paper introduced novel framework that call MAgNet for solving PDEs any mesh, possibly irregular. We proposed two variants of the architecture which gave promising results on benchmark datasets. We were effectively able to beat graph-based and grid-based architectures when using the CNN variant of the proposed framework, therefore suggesting a novel way of adapting existing CNN architectures to make predictions method is its very good performance on irregular meshes, including for super-resolution tasks, as be observed for the presented ID and 2D experiments, when compared SOTA methods. Notably, seems to perform best when handed the smallest amount of data to work with (with 64x64 training resolution), even in the condensed non-uniform case. This is very desirable property for real-life data. A limitation of our work however, is the significance of the learned interpolator. Indeed, compared with a simple cubic interpolation, the approach introduced here doesn't 't seem to offer a significant advantage and we leave improvement regarding this point for future work. Another improvement could be seen in the forecasting module. For now, MAgNet forecasts using first-order explicit time- stepping scheme that is known to suffer from instability problems in numerical PDE and ODE solvers. Learned solvers seem to somehow circumvent this limitation even when using large time steps
any mesh. The main added value of the proposed
Gonealezz et al. 2020; Brandstetter et al.) 2022] Stachenfeld et al. (001)) In future work, we wish to explore other time-stepping schemes such as the 4th order Runge- method [Runge| 1895 Kutta 1901) which is commonly used for solving PDEs. Finally, we observed that our MAgNet model appears to perform significantly better 1D than in 2D setting (as shown in section 4.2) While it is retaining its superiority in an irregular mesh setting, it appears little less performant than the MPNN method, even with its GNN variant. Further research on why the regular setting poses issues is thus also reserved for future work.
Acknowledgments and Disclosure of Funding
This work is financially supported by the government of Quebec and Samsung. The authors would like to thank Shruti Mishra, Victor Schmidt, Dianbo Liu, and Ayoub Ajarra for their fruitful discussions and useful insights.
10

---

References
Falcon, W., Borovec, J., Wälchli, A., Eggert, N., Schock, J., Jordan, J., Skafte, N., IrIdXD, Bereznyuk, V., Harris, E., Tullie Murrell, Yu, P., Præsius, S., Addair, T., Zhong, J., Lipin, D., Uchida, S., Shreyas Bapat, Schröter, H., Dayma, B., Karnachev, A., Akshay Kulkarni, Shunta Komatsu, Martin. B, Jean-Ba Baptistt SCHIRATTI, Mary, H., Byrne, D., Cristobal Eyzaguirre. Cinjon, and wwacchiighniing/pttocc---te Bahdanau, D., Cho, K., and Bengio. Y. (2015). Neural machine translation by jointly learning to align and translate. In Bengio, Y. and LeCun, Y., editors, 3rd International Conference Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.
Bar-Sinai, Y., Hoyer, S., Hickey, J., and Brenner, M. P. (2018). Learning data driven discretizations for partial differential equations.
Berger, M. and Colella, P. (1989). Local adaptive mesh refinement for shock hydrodynamics. Journal of Computational Physics, 82(1):64-84.
Berger, M. J. and Oliger, (1984). Adaptive mesh refinement for hyperbolic partial differential equations. Journal of Computational Physics, 53(3):484-512,
Brandstetter, J., Worrall, D., and Welling, M. (2022). Message passing neural pde solvers.
Brenowitz, N. D. and Bretherton, C. S. (2018). Progmostic validation of a neural network unified physics parameterization. Geophysical Research Letters, 45(12):6289-622
Chen, Y., Liu, S., and Wang, (2020). Learning continuous image representation with local implicit image function.
Chen, Z. and Zhang, H. (2018). Learning implicit fields for generative shape modeling.
Courant, R. (1943). Variational methods for the solution problems of equilibrium and vibrations. Bulletin of the American Mathematical Society, 49(1):1-23.
de Bezenac, E., Pajot, A., and Gallinari, P. (2017). Deep learning for physical processes: Incorporating prior scientific knowledge.
Dissanayake, M. W. M. G. and Phan-Thien. N. (1994). Neural-network-bret approximations for solving partial differential equations. Communications in Numerical Methods in Engineering, 10(3):195-201.
Evans, L. (2010). Partial Differential Equations. American Mathematical Society.
Bakhtin, A. (2020).
0.7.6 release.
Gilmer, J., Schoenholz, S. S., Riley, P. F.,. Vinyals, O., and Dahl, E. (2017). Neural message passing for quantum chemistry. Proceedings of the 34th International Conference on Machine Learning Volume 70, ICML' 17, page 1263-1272. JML..
Hochreiter, and Schmidhuber, (1997). Long short-term memory. Neural Computation, 98):17335 1780.
Hrennikoff, A. (1941). Solution of problems of elasticity by the framework method. Journal of Applied Mechanics, 8(4):A169-A175.
Jiang, C. M., Esmaeilzadeh, S., Azizzadenesheli, K., Kashinath, K.,, Mustafa, M., Tchelepi, H. A..
Marcus, P., Prabhat, and Anandkumar, A. (2020). MeshfreeFlooNNet A Physics- Deep Continuous Space- Time Super-Resolutton Framework. IEEE Press.
Kingma, D. and Ba, J. (2014). Adam: method for stochastic optimization.
Kovachki, N., Li, Z., Liu, B., Azizzadenesheli, K., Bhattacharya, K., Stuart, A., and Anandkumar, A. (2021). Neural operator: Learning maps between function spaces.
Kutta, W. (1901). Beitrag zur näherungsweisen Integration totaler Differentialgleichnngen Zeit. Math. Phys., 46:435-53.

---

Lagaris, I. E., Likas, A., and Fotiadis, D. I. (1998). Artificial neural networks for solving ordinary and partial differential equations. IEEE transactions on neural networks, 9 5:987-1000.
Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., and Anandkumar, A. (2020a). Neural operator: Graph kernel network for partial differential equations.
Li, Z., Kovachki, N., Arizzadenesheli, K., Liu, B., Stuart, A., Bhattacharya, K., and Anandkumar, A. (2020b). Multipole graph neural operator for parametric partial differential equations. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H., editors, Advances in Neural Information Processing Systems, volume 33, pages 6755-6766. Curran Associates, Inc.
Li, Z.-Y., Kovachki, N. B., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., and Anand- kumar, A. (2021). Fourier neural operator for parametric partial differential equations. ArXiv, abs/2010.08895.
Lim, B., Son, S., Kim, H., Nah, S., and Lee, K. M. (2017). Enhanced deep residual networks for single image super-resolution.
Novati, G., de Laroussilhe, H. L., and Koumoutsakos, P. (2021). Automating turbulence modelling by multi-agent reinforcement learning. Nature Machine Intelligence, 3(1):87-96.
Park, J. and Choi, H. (2021). Toward neural-network-basee based large eddy simulation: application to turbulent channel flow. Journal of Fluid Mechanics, 914.
Paszke, A., Gross, S.. Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Köpf, A., Yang, E., De Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S. (2019). PyTorch: An Imperative Style, High- Performance Deep Learning Library. Curran Associates Inc., Red Hook, NY, USA.
Phillips, N. A. (1956). The general circulation of the atmosphere: A numerical experiment. Quarterly Journal of the Royal Meteorological Society, 82(352):123-164.
Qi, C. R., Yi, L., Su, H., and Guibas, (2017). Pointnet++:: Deep hierarchical feature learning on point sets in metric space.
Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F. A., Bengio, Y., and Courville, A. (2018). On the spectral bias of neural networks.
Raissi, M., Perdikaris, P., and Karniadakis, G. E. (2017a). Physics informed deep learning (part i): Data-driven solutions of nonlinear partial differential equations.
Raissi, M., Perdikaris, P., and Karniadakis, G. E. (2017b). Physics informed deep learning (part ii): Data-driven discovery of nonlinear partial differential equations.
Rasp, S., Pritchard, M. S., and Gentine, P. (2018). Deep learning to represent sub-grid processes in climate models.
Richardson, L. F. and Lynch, P. (2007). Weather Prediction by Numerical Process. Cambridge University Press.
Ronneberger, O., Fischer, P., and Brox, T. (2015). U-Net: Convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention MICCAI 2015.
Runge, C. (1895). Ueber die numerische auflösung von differeenialgleechungen.. Mathematische Annalen, 46(2):167-178.
Sanchez-Gonzalez, A., Godwin, J., Pfaff, T., Ying. R., Leskovec, J., and Battaglia, W. (2020). Learning to simulate complex physics with graph networks.
Sanchez-Gonzalez, A., Heess, N., Springenberg. J. T., Merel, J., Riedmiller, M., Hadsell, R., and Battaglia, P. (2018). Graph networks as learnable physics engines for inference and control.
Sitzmann, V., Martel, N. P., Bergman, W., Lindell, D. B., and Wetzstein, G. (2020). Implicit neural representations with periodic activation functions.
12

---

Smagorinsky, J. (1963). General circulation experiments with the primitive equations. Monthly Weather Review, 91(3):99-164
Stachenfeld, K., Fielding, D. B., Kochkov, D., Cranmer, M., Pfaff, T., Godwin, J., Cui, C., Ho, S., Battaglia, P., and Sanzhez-Gonzalezz A. (2021). Learned coarse models for efficient turbulence simulation.
Stone, J. M., Tomida, K., White, C. J., and Felker, K. G. (2020). The athena++ adaptive mesh refinement framework: Design and magnetohydrrodynmic solvers. The Astrophysical Journal Supplement Series, 249(1):4.
Tancik, M., Srinivasan, P. P., Mildenhall, B., Fridovich-Keil, S., Raghavan, N., Singhal, U., Ra- mamoorthi, R., Barron, J. T., and Ng, R. (2020). Fourier features let networks learn high frequency functions in low dimensional domains.
Wandel, N., Weinmann, M., and Klein, R. (2020). Learning incompressible fluid dynamics from scratch towards fast, differentiable fluid models that generalize.
Wang, R., Kashinath, K., Mustafa, M., Albert, A., and Yu, R. (2020). Towards physics-informed deep learning for turbulent flow prediction. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery Data Mining, ACM.
Watters, N., Zoran, D., Weber, T., Battaglia, P., Pascanu, R., and Tacchetti, A. (2017). Visual interaction networks: Learning physics simulator from video. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R., editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [No]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main exper- imental results (either in the supplemental material or as URL)? [Yes] Link to the code and dataset is provided in the first footnot.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] Training and implementation details can be found the Appendix (c) Did you report error bars (e.g.. with respect to the random seed after running experi- ments multiple times)? [Yes] We models for 5 random seeds.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes]
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as URL? [Yes]
We provided link to our repository that contains code and datasets in the abstract.
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
13

---

(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsoureing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "sLMmIKYklw9", "reviewer": "Reviewer_yyq7", "summary": "1- Summary\n\nThe authors propose a PDE solver method based on implicit neural representations address the general limitation of generic numerical PDE solvers that assume the the underlying PDE is discretized and solve it with the Finite Element Method. More specifically, the proposed method lifts such assumptions while increasing prediction accuracy and resolution over generic numerical PDE solvers. The proposed method uses a graph neural network to infer the next state of the system in addition to a physics-based inductive bias in its loss term to constraint the neural network behavior by encouraging it to produce more physically-accurate results. The experiments show that the proposed method is able to estimate spatially continuous solution of the underlying PDE up to 250 frames with physically-consistent accuracy.", "questions": "- It is not clear how/when the proposed model fails. Have the authors tested their model's limits?", "limitations": "I am personally hesitant to question societal impacts of any scientific discovery that is not purposefully aiming to promote certain views/directions that directly and objectively harm the society. This is because I believe in the long-term this limits the natural evolution of discovery of the Truth, similar to how protection of the free expression principle leads to democracy. Arguably, any genuine scientific discovery could be eventually used for negative purposes. Therefore, I do not think the reviewers’ subjective opinion should be used in any way as an objective to decide the direction of scientific discoveries.\n\nRegardless, I do not think the proposed method would have any direct negative impact in the society.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "2- Paper strengths\n\n- The proposed PDE solver can generalize to PDEs with higher resolution that PDEs than it was trained on\n- The experiments show that the proposed method performs better or similar than SOTA methods\n\n3- Paper weaknesses\nNote that some of the weaknesses below are not meant to suggest that the authors should address them during the discussion prior in order to change my opinion. This is because I understand that addressing some of the weaknesses might take a long time or might require significant changes to the proposed method. Although addressing any of these might change my judgment about the work but for the time being the authors may prioritize addressing questions.\n\n- The authors start the paper by talking about climate forecasting as a challenging, high-resolution, problem but their model is far from handling anything related to weather forecasting and their experiments also does not include anything for climate forecasting\n\n4- Additional Comments\n\nRelated Work:\n- The related work section is written a bit mechanistically and does not highlight some of the distinctive differences of the prior methods in contrast to the authors’ proposed method.\n\nResults:\n- It would be good if the authors could provide a summary of the results at the beginning of this section.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "neUesiJ5l2Z", "reviewer": "Reviewer_4g6A", "summary": "This paper introduces an architecture for learning mesh-agnostic neural PDE solvers. The architecture consists of encoding, interpolation and forecasting, trained via interpolation loss and forecasting loss. The authors show in a 1D PDE dataset, it outperforms state-of-the-art model MPNN and FNO most of the time in terms of super-resolution on regular and irregular meshes. The authors also investigated how different aspects of the model influences the performance.", "questions": "No questions.", "limitations": "The main limitation is indicated in the above weaknesses part.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\nThe paper tackles an important problem of learning neural PDE solvers which go beyond the resolution it is trained on. The paper is written clearly. The proposed method makes sense.\n\nWeaknesses:\n\nThe most important weakness in this paper is its limited evaluation, which only evaluates in 1D PDEs. Since the paper's main contribution is \"Mesh Agnostic\", we should see experiments where how the method is evaluated in 2D meshes, which is the standard understanding of a mesh. In 1D, a mesh reduces to connected line segments. Compared with 1D, 2D-mesh introduces many new aspects of difficulty, such as more diversity and variations in the shape of the cells, which leads to more complex interpolations, etc. The results in 1D may not generalize to 2D. I would expect having experiments in 2D mesh, both in a regular grid and a irregular mesh, to see how the model compare with SOTA (e.g. using a 2D experiment where FNO or MPNN/MeshGraphNet is evaluated on but evaluated on super-resolution). Also, in 2D where problems are more difficult, the strength of the method compared to baselines may better exhibit.\n\n--\nUpdate\n\nThe authors have addressed my main concerns in the revision, which add experiments in 2D. In 2D experiments, the MPNN still outperforms the proposed method in the Fig. 15 in all cases, which may not show the relative strength of the proposed method. In light of the above, I have updated my rating from 4 to 5.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "B9SpXIiSVtq", "reviewer": "Reviewer_XU9U", "summary": "This paper proposes an algorithm to solve PDEs that's agnostic to the input mesh. Precisely, the proposed method follows the \"encoder-interpolation-forcast\" pipeline. This pipeline will first extract features for each of the sampled mesh points, then these features will be passed into the interpolator to create features for the query points. The query points features will passed through the forcaster to make the final prediction. The experimental results show that MAgNet is capable of performing un par with baselines such as FNO and MPNN. The results also show the proposed interpolation in feature space does lead to improvement. The results also show that the model is able to test on a mesh resolution that's different from the trained mesh resolution.", "questions": "In figure 4(a), is there any intuition why the performance is best at resolution 50? How significant is the the gap between cubic and the proposed method?\n\nIs there any visualization of the input and testing mesh? I think it would be helpful to see some of the visualization on some problem setting to provide better intuition for readers.", "limitations": "The limitation is properly addressed in section 5.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nI think the main strength of the proposed idea is that they did interpolation and interaction in a learned feature space instead of in the physics space directory. This allows the encoder and the interpolator to work together to make a potentially better interpolation schema which is agnostic to the sampling of the mesh vertices. The ablation studies seem to suggest that the learned interpolation is indeed bringing some advantages (per figure 4(a)). I suspect such advantage will be larger (i.e. comparing to Cubic interpolation on the original frames) when it comes to more irregular input meshes. \n\nWeakness:\n1. Is the model actually \"mesh agnostic\"? My understanding of \"mesh-agnostic\" is that if both meshes represent the same geometry, then the method should output the same results. The experiments seem to show that if two meshes are representing the same geometry, then the method will output equally accurate results, which is a weaker guarantee. It's not clear to me that the encoder architecture will provide exactly the same feature if the input mesh is slightly changed.\n\n2. How well is the method robust to irregular mesh? One major concern I have toward this method is that, if the input mesh is sample in a extremely non-uniform way (which seems to be the case for many physics problem as higher numerical precision will be required in one place than the other), this will require the interpolation to work with different descretization density, which might potentially lead to interpolation error. The experiment regarding irregular mesh is in section 4.2. There is very little description about how the irregular mesh is sampled (or visualization). Also to the bet of my understanding, it's only tested on the irregular mesh, but not using irregular mesh as input. I suspect this also seems to be related to the limitation of the ablation study on interpolation schema, where the difference between the cubic interpolation is not much larer than that of the feature space interpolation. More clarification toward this point will be helpful.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept