MeshMask: Physics-Based Simulations with Masked Graph Neural Networks
Authors: Paul Garnier
Vincent Lannelongue
Jonathan Viquerat
Elie Hachem
Venue: ICLR 2025 Poster (2025)
Keywords: graph networks
simulation
mesh
physics
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: We introduce a novel masked pre-training technique for graph neural networks (GNNs) applied to computational fluid dynamics (CFD) problems

Abstract: We introduce a novel masked pre-training technique for graph neural networks (GNNs) applied to computational fluid dynamics (CFD) problems. By randomly masking up to 40\% of input mesh nodes during pre-training, we force the model to learn robust representations of complex fluid dynamics. We pair this masking strategy with an asymmetric encoder-decoder architecture and gated multi-layer perceptrons to further enhance performance. The proposed method achieves state-of-the-art results on seven CFD datasets, including a new challenging dataset of 3D intracranial aneurysm simulations with over 250,000 nodes per mesh. Moreover, it significantly improves model performance and training efficiency across such diverse range of fluid simulation tasks. We demonstrate improvements of up to 60\% in long-term prediction accuracy compared to previous best models, while maintaining similar computational costs. Notably, our approach enables effective pre-training on multiple datasets simultaneously, significantly reducing the time and data required to achieve high performance on new tasks.
Through extensive ablation studies, we provide insights into the optimal masking ratio, architectural choices, and training strategies.

Published as a conference paper at ICLR 2025
MESHMASK: PHYSICS-BASED SIMULATIONS WITH MASKED GRAPH NEURAL NETWORKS
Paul Garnier'
Vincent Lannelongue
Mines Paris PSL University
Mines Paris PSL University
Centre for Material Forming (CEMEF)
Centre for Material Forming (CEMEF) CNRS
CNRS
paul wwt..ggeeeeeeeeeearrii..... psl.eesspeee eu
vincent hww.eeaeeeooooggeeeeeeeeasssissa eu
Jonathan Viquerat
Elie Hachem
Mines Paris PSL University
Mines Paris PSL University
Centre for Material Forming (CEMEF)
Centre for Material Forming (CEMEF) CNRS
CNRS
jonathan, wigperrateminnsspriis.... psl. eul
elie www..hhhemmmmeaarii........ psl. eu
ABSTRACT
We introduce a novel masked pre-training technique for graph neural networks (GNNs) applied to computational fluid dynamics (CFD) problems. By randomly masking up to 40% of input mesh nodes during pre-training, we force the model to learn robust representations of complex fluid dynamics. We pair this masking strategy with an asymmetric encoder-decoder architecture and gated multi-layer perceptrons to further enhance performance. The proposed method achieves state- of-the-art results seven CFD datasets, including new challenging dataset of 3D intracranial aneurysm simulations with over 250,000 nodes per mesh. Morcover, it significantly improves model performance and training efficiency across such diverse range of fluid simulation tasks. We demonstrate improvements of up to 60% in long-term prediction accuracy compared to previous best models, while maintaining similar computational costs. Notably. our approach enables effective pre-training on multiple datasets simultaneously, significantly reducing the time and data required to achieve high performance new tasks. Through extensive ablation studies, we provide insights into the optimal masking ratio, architectural choices, and training strategies.
INTRODUCTION
The quest to efficiently supplement traditional computational fluid dynamics (CFD) Hachem et al. (2010) with machine learning techniques marked by multiple advances in the past decade, progressing from convolutional neural network approaches Tompson et al.] (2016) Thuerey et al. (2018): Chen et al. (2019); Chu & Thuerey (2017) and physics informed neural networks Raissi et al. (2019) to advanced graph neural networks (GNN) methods. Indeed, graph neural networks, and more specifically the message-passing approach Battaglia et al. (2018), have emerged as natural candidates for the processing of mesh-based data, leading to seamless coupling with CFD Pfaff
al. 2021)
This architecture has brought spectacular results for the auto-regressive inference of non-stationary physical phenomena Sanchez-Gonzalez et al.) (2020); Lam et al. (2023). The recent introduction of attention layers and transformer architectures GNNs has brought even more improvements, coupled with more complex training procedure such as diffusion models Price al. (202)) although with limited mesh sizes.
Corresponding author

---

Published as a conference paper at ICLR 2025
Meedaed Gruph
Herooing % the rode fon mech
Oxpa
Graph
2
Ensoder
Wicycle Processor So Mlssage Paesing
Wioycle Processor 15c Message Paeding
Encodar
Decoder
Decoder
Decoder
The proposed Masked Graph Neural Network architecture. During pre-training, we remove a portion of the edges and nodes of the mesh (e.g. 50%) and use this new graph as input for the Encoder. Masked nodes are then replaced by a [MASKED] token before the mesh is passed to the Decoder. After pretraining, only the Encoder finetuned on a specific dataset, without any nodes being removed.
Figure 1:
The refinement of the architectures and training techniques to overcome this limitation is still an active research field, with multi-grid paradigms John (2002) showing promising results] Lino et al.
(2021): Fortunato al. (2022); Yang et al. (2022); TTghibbkksshi al. (2023); Garnier al. (2024). Yet, training times remain a major concern, and the required amount of training data can still represent a prohibitive pre-processing cost.
Recently, pre-training techniques such as the Cloze Task Taylor (155) have been introduced in the domain of natural language processing (NLP) and computer vision that improve performance while significantly reducing training times. Most of the contributions rely on masked auto-encoders, an asymmetric architecture in which an encoder is provided with a random subset of the input, proportion of it being masked (usually around 70% of the data is randomly masked for computer vision and 15% for NLP). The masked tokens are re-introduced on the latent representation, after what a lightweight decoder reconstructs the full output. This pre-training step is usually followed by fine-tuning training on specific tasks. This approach has led to significant improvements in computer vision He et al. (2021) and NLP Devlin et al. (2019): Radford et al. (2018).
Such methods have already been transferred in the context of GNNs. In Hu et al. 2019), the authors consider graph classification datasets, and develop pre-training approach mixing both local and global representations. The authors Hu et al.] (2020) develop a generative framework for the reconstruction of the inputs, leading significant improvements against state of the art supervised results. In Tan et al. (2022), the authors propose the masked graph auto-encoder (MGAE) paradigm that makes of tailored cross-correlation decoder structure. The results show the superiority of this approach for link prediction and node classification tasks. We also underline the contribution of Zhou & Farimani (2024), who successfully apply computer vision approaches to the resolution of one- and two-dimensional PDE problems, including the Burgers equation or the Kuramoto- Kuramoto-Sivashinsky Sivashinsky system.
In the present contribution, we propose architecture variation with a new masking pre-training technique for CFD datasets. We also introduce a new dataset much more complex than previous ones, both in terms of physics and mesh size (up to 250,000 nodes and millions edges). We conduct a comprehensive ablation study on model architecture. Encoder and Decoder asymmetry and on key parameters of our pre-training method. Finally, we train models 7 different datasets, all different terms of physics, mesh size and trajectory length.
Our finding suggests that pre-training technique gives large improvements in term of accuracy (from 15 to 40%), and more importantly, offers a very convenient method to pre-train model on different datasets before being finetuned, making it much easier train on large datasets.
The paper is organized as follows: the AutoEncoder architecture and masking strategy are presented in subsection 2.1 and 2.2 The overall structure of each model is defined in 2.3 Training methodology
2

---

Published as a conference paper at ICLR 2025
Figure 2: Examples from test meshes in 3 different datasets: CYLINDER, BEZIER and 2D- ANEURYSM. We show (left) the masked mesh with hidden nodes in gray, (middle) the prediction from the AutoEncoder architecture, and (right) the ground truth prediction. We replace the masked nodes by the ground truth ones.
and datasets are presented in section 3 We then perform a full ablation study in subsection 4.1 and present the rest of the obtained results section Finally, future perspectives given. The code used this paper will be released after publication.
THEORETICAL FRAMEWORK
We consider mesh as undirected graph G = EV, V {v.)e=IN= is the set of nodes, where each v; represents the attributes of node i. ((ek) Tk. sx)]==i.N= is the set of edges, where each C₁ represents the attributes of edge k, Fkk is the index of the receiver node, and 8y is the index of the sender node.
Each edge feature is made of the relative displacement vector in mesh space u, u, u, and its norm ||u,]]|- |- Each node feature V, (such as the pressure, velocity) also receives a one-hot vector indicating the node type (such as inflow outflow for boundary conditions, obstacles denote where shapes are inside the domain, etc) and global information (viscosity, gravity) creating x; In the case of the ANEURSYM dataset, we also add the nodes positions and the acceleration as inputs. Finally, given the complexity of the blood inflow (see subsection 3.I. we also add the next step inflow velocity as inputs.
2.1 ENCODER-DECOME ARCHITECTURE
The proposed Incoder-Decodee architecture simply stacks 2 GNNs one after the other in an AutoEn- coder fashion. Each of these two global networks is multi-grid GNN built in an Incode-Process- Decode fashion, shown in Figure (selsssscttonn 2.3).
The aim to make the next-step prediction (or the reconstruction) of mesh given a partially visible input. We follow the strategy Hf al. (2021) by having asymetric architecture, meaning Encoder is much larger than our Decoder. This does not increase the training budget since most spent during pre-training, where the Encoder inputs mostly masked.
In practice, we start by pre-training the Encoder and the Decoder on a next-step prediction or reconstruction task with masked input (as detailed in 2.2). We then finetune the Encoder only a next-step prediction task. The Encoder architecture does not change between pretraining and fine-tuning.

---

Published as a conference paper at ICLR 2025
Masked nodes added back the mesh, with the valuo of [MASKED]
Dacorier
Masking
Encoder
Figure 3: The overall AutoEncoder architecture.
Masking
Masknng
Masking
Figure 4: Predictions on the CYLINDER with different Masking Ratio. Even with high ratio (85%), the model is able to generalize well.
2.2 MASKING
At each training step, we randomly sample fraction of the existing nodes a given mesh. We then proceed completely remove them from the mesh, including all edges that connected least one masked node. To ensure that the model could process long range interactions even with high masking ratios and disconnected sub- uub-graphs, we considered computing the K K-hop of each remaining node in the original graph and adding those new longer edges if both nodes were not masked, for different K values.
We refer to this simply as "masking". What we end up with inputs is now much smaller mesh (up to 95% smaller our experiments), thus being much faster to process. We display in Figure examples of different masking ratio.
After being passed the Encoder, we reconstruct the initial mesh by:
Replacing hidden nodes by shared learnable [MASKED] token (following Devlin al. (2019)
Replacing other nodes by their predicted values from the Encoder Rebuilding hidden edges with geometric data
Theoretical justification This architecture is inspired by three factors: the inherent locality of finite element solvers, the information propagation boundaries of message passing, and the scale of error reduction in GNNs. First, since finite element solvers approximate solutions to equations such as the Navier-Stokes equation neighhoring elements, masking element naturally makes much
4

---

Published as a conference paper at ICLR 2025
more difficult task. We believe that by doing so, force model to find other relevant elements to make predictions rather than relying solely on direct neighbors. Second, since a message passing architecture is bounded by the number of steps multiplied by the length of edges, masking large part of the mesh drastically reduces both the available information and how far it can propagate. We thus prevent the model from simply extrapolating from neighboring physics. Third, since GNNs only reduce errors locally, like Gauss-Seide smoothers, we are forcing them to retrieve information from more distant parts of the mesh than they usually do.
Finally, thanks to the unstructured nature of a mesh, uniform random masking allows the model to focus more on fine- mne-grained area of the mesh. The overall process is detailed in Figure 3
2.3 OVERALL ARCHITECTURE
Both our Encoder and Decoder follow an Perrodderroces--DDee architecture, following the work from [Btttalia et al. (018). We encode nodes and edges features with simple Multi Layer Perceptron (MLP) into latent vectors of size pt e₃ MLP(u |uu
vk E, v. MLP(x,) Vr E V
where the MLP is made of 2 hidden layers of size p. ReLU activation and Layer Normalization.
The proposed process block is made of m message passing blocks, each being either Graph Net block from Battaglia et al. (2018) or UpScale and DownScale blocks from Garnier et al.) (2024) to create Multigrid W-cycle models. If not specified, our models are W-cycle with 15 message passing, and latent size of 128 neurons.
RLP
MIF
Usually, the processing is done by an MLP similar the ones seen in the Encoder. Here, we improve on that architecture by replacing them with Gated MI.P (see subsection 2.4). We also considered using Graph Atten- tion Networks (GAT) introduced by Veličković et al. (2018) for pro- cessing, since self-attentional layers
Figure 5: (top) Each edge is updated from its own feature, and the one from the nodes connected to it. (bottom) Each node updated from its features, and from the edges its connected to. (right) Information flow from node perspec- tive after one step of message passing.
have proven to be effective for such applications Garnier et al. (224). The message passing process detailed in F
Finally, we add a Decoder to transform the latent v into the output features: yr = MLP((v,) Vr E V
2.4 GATED MLP
We replace the MLPs in the Processor by Gated MLP (see (see Dauphin et al. (2017)). The Gated MLP takes an input of size p, creates two branches where linear layers are applied with expansion factor of size and apply the GeLU non-linearity Hendrrykss & Gimpell (2023) on one of those branches. We then merge the two branches with element-wise multiplication before applying final linear layer with an output of size p.
While this architecture leads to much higher number of parameters, training and inference time similar as to their full MLP counterpart. We find this approach similar to /Sazeer (2020) and that leads to improvements in the predictions.
TRAINING
3.1
DATASETS
We evaluated our models on different use cases. We detailed below the different datasets (see Table 3.1 and Figure 6) parameters used and the simulation time step Al. Each training set contains 100
5

---

Published as a conference paper at ICLR 2025
Figure 6: Sample of our datasets, in the same order as in Table 3.1
trajectories, and testing set 20 trajectories. Datasets from the COMSOL solver are originally from Pfaff al. (202I). The MULTIPLE BEZIER dataset is from Garnier et al. (2024). The 3D ANEURYSM dataset from Goetz et al. 2024b). Given the considerable leap in complexity (both in terms of mesh size, inputs and physics), more detailed presentation is given below in 3.I.1
#
At
Dataset
Solver
Dimension
nodes
traj
steps
CYLINDER
COMSOL
2k
2D
Fixed Mesh
100
600
0.01
PLATE
COMSOL
1k
3D
Fixed Mesh
100
400
FLAGSIMPLE
AreSim
2k
2D
Fixed Mesh
100
400
0.02
AIRFOIL
SU2
5k
2D
Fixed Mesh
100
600
0.008
BEZIER
Cimlib
30k
2D
Fixed Mesh
100
6000
0.1
AD-URNUR
Cimlib
250k
Fixed Mesh
100
80
0.01
2D-ANHURYSM
Cimlib
10k
2D
Fixed Mesh
100
80
0.01
3.1.1 3D ANEURYSMS
Goetz et al. (2024b) developed 101 semi-idealized geometries derived from patient-specifi intracra- nial aneurysms, segmented from medical imaging data. They conducted CFD simulations of blood flow within these vessels, numerically solving the transient incompressible Navier-Stokes equation over a complete cardiac cycle. For a more detailed explanation of the methodology, refer to Goetz al. [2023 2024a).
Training on this dataset (Figure represents a significant advancement and introduces unprecedented challenges. The transition three-dimensional fluid dynamics substantially increases the complexity of observed flow patterns and expands mesh sizes to over 250,000 nodes and million edges, far surpassing previous studies in this field, which typically didn't exceed 40,000 nodes. Moreover, the inflow conditions follow a pulsatile cardiac cycle, introducing time-dependent variations that require accurate modeling. This pulsatile nature demands more profound understanding of fluid dynamics to simulate the rhythmic fluctuations in blood flow and pressure.
From the 3D-ANBURYSS dataset, we also derived the AD-ANEURYSM dataset by re-meshing slices extracted from the 3D simulations. This dataset allowed us to iterate more quickly while retaining the challenges of varying geometry, pulsatile inflow, and complex flow patterns.
3.2 PARAMETERS
Network Architecture All of the MLPs (except the Gated MLPs) are made of 2 hidden layers of size 128 with ReLU activation functions. Outputs are normalized with a LayerNorm. The Gated MIPs are using hidden dimension of size 128 and expansion factor = 3.
In the case of MultiGrid model, DownScale blocks use a ratio of 0.5. We follow the state-of-the-art and all our models are W-cycle with 15 message passing steps.
Training We trained our models using an T22 loss, with batch size of During pretraining, the loss is only computed on masked nodes, similar to Devlin al. (2019) : He al. (202I).
We start by pre-training our Encoder and Decoder for 500k training steps, using an exponential learning rate decay from 10-4 10 over the last 250k steps. We then finetune the Encoder for
In the of the 3D ANEURYSM, hatch size of with sub-mesh partitioning (see Appendix)

---

Published as a conference paper at ICLR 2025
Inflow rate (mL (g)
ms
14 18
15
Time (8)
Figure 7: (top-left) Overview of the 101 aneurysms. (top-right) Velocity inflow profile imposed at the inlet boundary as parabolic flow. (bottom-left) Presentation of the ey flow in aneurysms. (bottom-right) Example of mesh.
another 500k training steps, with the same strategy for the learning rate. During the pre-training, if not specified, we use a node masking ratio of 40% (roughly equivalent to masking 60 to 70% of the mesh information depending on its geometry).
All models are trained using an Adam optimizer Kingma & Ba 2017). The baseline models are trained for a million training steps to evenly compare to models. Finally, following the same strategy as SSanchez--Gnzzleez et al. (2020); Platf et al. (202T); Garnier et al. (2024), we introduce noise our inputs. More specifically, we add random noise N NQ0, or) to the dynamical variables.
RESULTS
4.1 ABLATION STUDY
We trained our models on very different datasets (results be seen here) and compared it to baseline models as well as statc-of-the-an model. We also pretrained our models on multiple datasets the same time, before evaluating them in 0-shot fashion and with finetuning. The main finding is that masking large portion of the input mesh during pretraining leads to large improvement for long-term autoregressive prediction (see table 2). We are also able to pretrain models on different datasets at the same time, largely improving the time needed to fully train a model. Finally, our models decreases inference time by large margin comparison to classical finite element solver (see Table A.I.II. Even when taking the training time into account, it becomes more interesting to train a GNN and use for inference after roughly 250 simulations of Cylinder, and only 20 3D Aneurysms
Masking Ratio Figure 8 shows how the masking ratio impacts the long-term autoregressive
prediction. We find that the best masking ratio lies between 25% and 40%. This represents masking between 40% and 70% of the overall mesh (nodes and edges). While this is much higher than the usual 155 from Devlin et al. (2019). this is similar to the results obtained by He al. (202T). We find that adding K hhop edges did not have significant impact for masking values up 20%. but improved performance for larger values with K = 2 and
For the 3D Aneurysm, it takes 12 hours to simulate full trajectory while only 80 seconds with GNN that took 10 days of training.
7

---

Published as a conference paper at ICLR 2025
Layer
Depth
1-step all-rollout
MLP
5
6.23
185
Task
1-step all-rollout
ML.
15
3.13
Gated MI.P
5
3.6
71
Reconstruction
3.49
86
Gated MLP
15
2.1
58
Next-step prediction
2.49
52
GAT
15
2.2
58.2
(a) Pre-training task. Training for next-
(b) Processing layers. MLP performs better.
Gated
step prediction leads to much better re- sults.
Masking
all-rollout FLOPS
Decoder Depth 1-step all-rollout
encoder w/ [MASKED]
54.3
2.8x
49
encoder w/out [MASKED]
52
1x
3
46
(c) Encoder Mask token. No mask token makes it faster.
(d) Decoder Depth. Increasing the number of message passing steps only slightly increases per- formances.
Table 1: Ablation study on the CYLINDER Dataset. We tracked one-step RMSE and the RMSE averaged over the entire trajectory. We highlight the settings chosen by model in the following experiments, All results are
Node Ratio
Node+
30
Figure 8: A masking ratio between 25% and 40% leads to the best finetuning results the CYLIN- DERFLOW dataset.
Impact of Masking Pretraining Table shows the impact of masking pretraining a simple MeshGraphNet architecture. Overall, we find that keeping the same model but pretraining it 500k steps with masking ratio of 40% leads to 25% improvement consistently. This shows that novel method allows the model to learn the inherent physics with more depth. The improvement in performances without any change in the underlying architecture extra training time makes it very robust technique to systematically improve the model's performances.
Gated MLP We find that using Gated MLP instead of regular MLP yields much better results. We find similar results for model 3 times smaller using Gated MLP versus a model using MLP. Using attention-based processing layers here did not prove enhance the performance compared to the Gated MLP.
At scale with 15 message passing steps, still find improvements over the usual MeshGraphNet architecture from Pfaff et al. (2021) (see table [bb
Auto-Encoder Architecture The proposed Decoder architecture can be defined independently from the Encoder. We study this in table with different amount message passing steps. Similarly to He et al. (202T), find that Decoder with only message passing step already yields very strong
8

---

Published as a conference paper at ICLR 2025
performance. Overall, select Decoder with message passing steps since the performances on-par with bigger Decoder, but at a much lighter cost training time.
4.2 COMPARISON
Our experiments demonstrate significant improvements in model performance across various ar- chitectures when employing the proposed novel pre-training masking techniques. Regardless of the underlying model, observe performance gains of up to 25% compared to baseline models without masking (Table (2). The most striking results come from our multigrid architecture combined with masking pre-training, which establishes outperforms the previous best model by up to 60%, marking a substantial leap forward in predictive accuracy for fluid dynamics simulations.
Kate-of-thee-ae in the field. This configuration
Notably, these improvements are achieved with the same number training updates as baseline models, underscoring the efficiency of our masking techniques helping the model grasp the intricate physics of fluid dynamics. This suggests that our approach not only enhances performance but also accelerates the learning process, reducing computational costs for future large-scale simulations. comprehensive overview of our results is presented in Table 2 clearly illustrating the performance gains across different model architectures and evaluation metrics.
Interestingly, we observe minimal or no improvements in 1-step RMSE across our experiments. This observation points two key insights: firstly, we may be approaching a performance plateau in terms of pure ststep loss prediction. Secondly, and perhaps more significantly, our models demonstrate an improved ability to mitigate long-term error propagation, a crucial factor extending the predictive horizon of fluid dynamics simulations.
Table 2: All numbers are 10033 DATASET-1 means one-step RMSE, and DATASET-AlI means all- rollout RMSE. MGN results reproduced according to Pfaff al. (202T), BSMS-GNY according to Cao et al. (2023). MultiScaleGNNN according to Lino et al. (2021) and Multigrid results are reproduced according Garnier al. (024)).
ELALE
AIRFOIL 1-RMSE
I-RMSE +
1-RMSE
I-RMSE 1
I-RMSE
1-RMSE
1-RMSE
MGN
1.15
0,07
329
27.7
794
1795
Multigrid
2.8
1.02
0.17
302
24.1
638
749
MultiSaleeGNN
BSMS-GNN
2.83
0.15
314
23.2
632
719
MGN w/ musking
0.09
26.5
718
Ours
2.5
1.04
0.11
310
22.7
645
725
ALL-RMSE
ALL-RMSB 1 ALL-RMSE
ALL-RMSB
ALL-RMSE
ALL-RMSE J
ALL-RMSU |
MGN
7513
Multigrid
56.9
121
8.1
9871
275
7009
11327
MuliiScaleGNN
RSMS-GNN
56.9
16.6
9418
262
6983
10993
MGN w/ masking
46.5
12.2
281
7112
Ours
29
98
4.5
8794
212
6489
8772
Table Every model trained for total of 1M steps. vRAM and Inference Time are computed on the CYLINDER dataset. The large increase of parameters in our method is mostly due to the gatedMLP and the expansion factor e 3.
MODEL
# TRAINING STEPS
# PARAMETERS
VRAM (IN GB)
INFERENCE TIME (MS/STEP)
MGN
IM
2.8M
7
49.3
Multigrid
IM
3.5M
11
55.2
MultiScaleGNN
IM
2.6M
8
53.7
IM
2.1M
7
52.8
MGN w/ masking
500k 500k
2.8M
7
49.3
Ours w/ GatedML.P
500k 500k
9.2M
16
59.1
4.3 TRANSFER LEARNING EXPERIMENTS

---

Published as a conference paper at ICLR 2025
Pretraining multiple datasets We evaluate transfer learning two datasets: CYLINDER and BEZIER. Both datasets share the same inputs and outputs (see /ubsubsection A.I.I). and are generated from the simulation of an incompressible flow past multiple nigid bodies in fixed Reynolds number range, thus sharing physical similarities. We conducted three experiments:
masked pretraining on CYLINDER -> finetuned on BEZIER
masked pretraining on BEZIER - finetuned on CYLINDER
masked pretraining on both CYLINDER and BEZIER before being duplicated and fine- tuned on both datasets
We compare those experiments to both our best model without pretraining and our best model with pretraining. Results available in Table 4
We find that pretraining on similar (although with different physical parameters such as the viscosity the inlet velocity) dataset yields good performances similar to the models without any pretraining (+7.5% on CYLINDER, + 14.55 on BEZIER).
Our pretraining methods allows for efficient pretraining multiple datasets the same time before re-using this model for finetuning phase. model pretrained on both CYLINDER and BEZIER improves its results respectively by 22.7% and 15.6%, while being 33% faster train those models. This highlights two important points:
1. This shows that our pretraining method not only allows to train multiple models faster, but also that it can lead to the pretraining of a larger model on multiple datasets of different physics use cases before being finetuned.
2. We find that pretraining, even multiple datasets at the same time, always leads to better performances terms of all-rollout RMSE (-18% on average) than no pretraining all.
Table 4: All numbers are x10 3 Each row presents on which dataset the model was pretrained. Each column presents on which dataset the model was finetuned and then evaluated.
PRETRAINING DATASET
CYLINDER
BEZIER
ALL-RMSE 4
DIFFERENCE W/ BASELINE
ALL-RMSE
% DIFFERENCE W/ BASELINE
No PRETRAINING
56.9
275
CYLINDER
29
-49.0%
315
+14.5%
BEZIER
61.2
+7.5%
212
-22.9%
CYLINDER+BEZIER
44
-22.7%
229.6
-15.6%
meshes We evaluated the performance of our model on out-of-distribution meshes by training model graphs with given refinement and applying this model directly to much finer graph (from 10k to 250k nodes in our experiments). We find that even with this large gap in refinement, our model is able to generalize well and make accurate predictions it performs only 75% worse than model fully trained on fine meshes, which remains on par with results from MGN for example. These results also mean that we do not necessarily have to produce the entirety of the training on large and costly meshes, but that fraction might be sufficient.
5 CONCLUSION
In this study, show multiple datasets and through transfer learning that similar to NLP and Computer Vision, our simple masking pretraining technique demonstrated significant improvements in model performance and training efficiency.
Our approach achieved state-off sate--oftthe-e of-the-art results on the seven presented CFD datasets, including a new challenging dataset of 3D intracranial aneurysm simulations. Importantly, method allows for efficient pre- trainirinng across multiple datasets simultancously, greatly reducing both the time and data needed attain high performance new tasks.
10

---

Published as a conference paper at ICLR 2025
The authors acknowledge the financial support from ERC grant no 2021 2021-Cor I-CoG- 101045042, CURE. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Research Council. Neither the European Union nor the granting authority can be held responsible for them.
Acknowledgementss
REFERENCES
Peter W. Battaglia, Jessica B. Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalee Vinícius Flores Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, Çaglar Gülçehre, H. Francis Song, Andrew J. Ballard, Justin Gilmer, George E. Dahl, Ashish Vaswani, Kelsey R. Allen, Charles Nash, Victoria Langston, Chris Dyer, Nicolas Heess, Daan Wierstra, Pushmeet Kohli, Matthew M. Botvinick, Oriol Vinyals, Yujia Li, and Razvan Pascanu. Relational inductive biases, deep learning, and graph networks. CoRR, abs/ 806.01261, 2018.
URL http::ttta//iiiiii org/ abs/ 1805.1186600000 01261
Yadi Cao, Menglei Chai, Minchen Li, and Chenfanfu Jiang. Efficient learning of mesh-based physical simulation with bsms-gnn, 2023. URL https://arxx:: wtt.as///rggaa////ea///o 2210. C2573
Junfeng Chen, Jonathan Viquerat, and Elie Hachem. U-net architectures for fast prediction of October 2019, doi: 10.48550/
incompressible laminar flows. arXiv e-prints, art. S.KV.19110135332
Mengyu Chu and Nils Thuerey. Data-driven synthesis of smoke flows with enn-based feature descriptors. ACM Transactions on Graphics, 36(4):1-14, July 2017. ISSN 1557-7368, doi: 10.1145/3022959.307333 10 01..:..1111..311 307334443339393333
dww...oooooooooo owg.
Yann Dauphin, Angela Fan, Michael Auli, and David Grangier. Language modeling with gated convolutional networks, 2017.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding, 2019.
Hugues Digonnet, Luisa Silva, and Thierry Coupez. Cimlib: A fully parallel application for numerical sinulations based on components assembly. AIP Conference Proceedings, 908:269-274, 05 2007. doi: 10. 1063/1 1..11//.777482... .2740823.
Thomas D. Economon, Francisco Palacios, Sean R. Copeland, Trent W. Lukaczyk, and Juan J. Alonso. Su2: An open-source suite for multiphysics simulation and design. AIAA Journal, 54(3): -46. March 2016. ISSN 1533-385X. doi: 0..2514/1.j0550 URL http: //x doi. orgg/00
Meire Fortunato, Tobias Pfaff, Peter Wimsberger, Alexander Pritzel, and Peter Battaglia. MultiScale MeshGraphNets. arXiv e-prints, art. arXiv:2210.00612. October 2022. doi: 10.4 00612.
Paul Garnier, Jonathan Viquerat, and Elie Hachem. Multi-grid graph neural networks with self- attention for computational mechanics. arXiv preprint arXiv:2409. 1199,, 2024.
Aurèle Goetz, Pablo Jeken Rico, Yves Chau, Jacques Sédat, Aurélien Larcher, and Elie Hachem. Proposal for numerical benchmarking of stuid-structure interaction cerebral aneurysms, 2023.
URL htt.httts///xxiiii org/ abs/2308..... 08301
Aurèle Goetz, Pablo Jekon-Rico, Yves Chau, Jacques Sédat, Aurélien Larcher, and Elie Hachem. Analysis of intracranial aneurysm haemodynamics altered by wall movement. Bioengineering, 11(3), 2024a. ISSN 2306 5354. doi: 10. 10.33900iienniinerring 1030269. URL https::///////. //wwl
mdbpi.com/2005-35///11
Aurèle Goetz, Pablo Jeken Rico, Ugo Pelissier, Yves Chau, Jacques Sédat, and Elie Hachem. AnXplore: A comprehensive fluid-structure interaction study of 101 intracranial aneurysms. Frontiers in Bioengineering and Biotechnology, 12. 2024b. ISSN 2296-4185. doi: 10.3389/fbioe. 3389/ fbice
1433811.
2024 4

---

Published as a conference paper at ICLR 2025
E. Hachem, B. Rivaux, T. Kloczko, H. Digonnet, and T. Coupez. Stabilized finite element method for incompressible flows with high reynolds number. Jourmal of Computational Physics, 229(23): 2010. ISSN 0021-9991. doi: https///ooooog/////lllooggoo0000
URL https:
/ 1/www.se sci encedi rect com http:///ooo...oa/a./////////a///
William L. Hamilton, Rex Ying, and Jure Leskovec. Inductive representation learnin learning large graphs,
2018. URL [ttL/ttss//axcat www. aba http:///g/aaaaaaaaaaaaaa 02216
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners, 2021.
Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus), 2023.
Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Perey Liang, Vijay S. Pande, and Jure Leskovec. Strategies for pre- pre-training graph neural networks. CoRR, abs/1195..1226, 2019, URL
http://arxiv.org/abbbb
12265
Ziniu Hu, Yuxiao Dong, Kuansan Wang, Kai-Wei Chang, and Yizhou Sun. GPT-GNN: generative
pre-training of graph neural networks. CoRR, abs/2006. 15437, 2020. URL http:::tt///sttt
org/ hts//bb//ae 2006 15437
Volker John. Higher order finite element methods and multigrid solvers in benchmark problem for the 3d navier-stokes equations. International Journal for Numerical Methods in Fluids, 40(6): 775-798, October 2002. ISSN 1097-0363. doi: 10. 1002///d..77 URL http: /xxx doi org/ 10.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017.
Remi Lam, Alvaro Lanchez-Gonzalee Matthew Willson, Peter Wirnsberger, Meire Fortunato, Ferran Alet, Suman Ravuri, Timo Ewalds, Zach Eaton-Rosen, Weihua Hu, Alexander Merose, Stephan Hoyer, George Holland, Oriol Vinyals, Jacklynn Stott, Alexander Pritzel, Shakir Mohamed, and Peter Battaglia. Graphcast: Learning skillful medium-range global weather forecasting, 2023. URL
httrgg/bb//222/ 2272.12794) 12794
Mario Lino, Chris Cantwell, Anil A. Bharath, and Stathi Fotiadis. Simulating continuum mechanics with multi-scale graph neural networks, 2021.
Comsol multiphysics@. Comsol, 2020. URL http: comsol comm
Rahul Narain, Armin Samii, and James O'Brien. Adaptive anisotropic remeshing for cloth
simulation. ACM Transactions on Gruphics, 31:11117 November 2012. URL ht
//graphi cs www.berreeeeee. www.lll..eede.ee..e..ee www.ee//paapreeaaaaiiaaaaaae SIGGRAPH Asia 2012, Singapore.
Proceedings ACM
David Padua, Amol Ghoting, John A. Gunnels, Mark Squillante, José Meseguer, James H. Cownie, Duncan Roweth, Sarita V. Adve, Hans J. Boehm, Sally A. McKee, Robert W. Wisniewski, George Karypis, Allen D. Malony, Steven Gottlieb, Rolf Riesen, Arthur Maccabe, Gianfranco Bilardi, Andrea Pietracaprina, Arun Kejariwal, Alexandru Nicolau, Christian Lengauer, John L. Gustafson, William Gropp, Jean-Pierre Prost, David Padua, Geoff Lowney, Patrick Amestoy, Alfredo Buttari, Iain Duff, Abdou Guermouche, Jean-Yver L'Excellent, Bora Uçar, Robert H. Halstead, Mario Nemirovsky, Patrick Amestoy, Alfredo Buttari, lain Duff, Abdou Guermouche, Jean-Yves L E'xcellent Bora Uçar, and Scott Pakin. METIS and ParMETTS, pp. 1117-1124 Springer US, 2011. ISBN 9780387097664, doi: 10. 00010077978--0-37--9966----- //dx doi rg/ 1007 978-
URLhttp!
4__00
Tobias Pfaff, Meire Fortunato, Alvaro Sanchez-Gonzalee and Peter W. Battaglia. Learning mesh- based simulation with graph networks, 2021.
Iln Price, Alvaro Sanchez-Gonzalez, Ferran Alet, Tom R. Andersson, Andrew El-Kadi, Dominic Masters, Timo Ewalds, Jacklymn Stott, Shakir Mohamed, Peter Battaglia, Remi Lam, and Matthew Willson. Gencast: based ensemble forecasting for medium-ranne -aante weather, 2024. URL
12

---

Published as a conference paper at ICLR 2025
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Lan- guage models are unsupervised multitask learners. 2018. URL http:: //wc:////mmm/mmm fpksywy
M. Raissi, P. Perdikaris, and G.E. Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686-777 707, 2019. ISSN 0021-9991, doi: https://
2ah.org/00.0016f1..p..000..... 10.045. URthttps:: www. scwencedirect. com/ science/ httpp//ppp1/00009991999100
Alvaro Sanchez-Gonzalez, Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, and Peter W. Battaglia. Learning to simulate complex physics with graph networks, 2020.
Noam Shazeer. Glu variants improve transformer, 2020.
Ali Taghibakhshi, Nicolas Nytko, Tareq Uz Zaman, Scott MacLachlan, Luke Olson, and Matthew West. Mg-gnn: Multigrid graph neural networks for learning multilevel domain decomposition methods, 2023.
Qiaoyu Tan, Ninghao Liu, Xiao Huang, Rui Chen, Soo-Hyun Choi, and Xia Hu. MGAE: masked autoencoders for self-supervised learning on graphs. CoRR, abs/2201.02522 2022. URL https
waaxxiv.orgg/bs//20000 22233 02534
Wilson L Taylor. "cloze procedure": A new tool for measuring readability. Journalism quarterly, 30 (4):415 1953.
Nils Thuerey, Konstantin Weissenow, Harshit Mehrotra, Nischal Mainali, Lukas Prantl, and Xiangyu Hu. Well, how accurate is it? A study of deep learning methods for reyrolds-avrragee navier-stokes simulations. CoRR, abs/1810.081) 2018. URL utthttp:/ arxiv. wwt.:/s/g/aaaaeeeee 08217
Jonathan Tompson, Kristofer Schlachter, Pablo Sprechmann, and Ken Perlin. Accelerating eulerian fluid simulation with convolutional networks. CoRR, abs/1607.03355 00359,, 2016. URL http://
arxiv. wtw.vvo.rg/abs/167.... 03597
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks, 2018.
Zhishuang Yang, Yidao Dong, Xiaogang Deng, and Laiping Zhang. Amgnet: multi-scale graph neural networks for flow field prediction. Connection Science, 34:2500-2519, 10 2022. doi: 1111080095440991.2222111117
Anthony Zhou and Amir Barati Farimani. Masked autoencoders are pde learners, 2024.
A APPENDIX
A.1 DATASETS
A.1.1 DETAILS
We give details below about the inputs and outputs used for each dataset (see A.1.I CYLINDER, PLATE were generated with COMSOL mmlttphhsics@@ (2020) and were introduced by Pfaff et al.
)2021).
FLAGSIMPLE
generated with ArcSim Narain et al. (2012) and was introduced by Pfaff AIRFOIL was generated with SU2 Economon et al. (2016) and was introduced by Pfaff
et al.
2021).
introduced by Garnier et al. (024).. ANEURRY, AD-ANNURYSM were generated with CimLib Digonnet et al.
al.
2021 BEZIER generated with SimLib/iggomin et (2007)
(2007) and were introduced by Goetz al. (2024)).
13

---

Published as a conference paper at ICLR 2025
Figure 9: Overview of our unstructured mesh. (top) Mesh from the Bezier dataset with around 30k nodes. (bottom) Mesh from the 3D-Aneuyssm dataset with around 250k nodes.
Datasct
Inputs
Outputs
History
fynn (ms/step)
fge (ms/step)
CYLINDER
n., Uz, My
U2, By
33
820
PLATE
fin
æ. 2,
36
2893
FLAGSIMPLE
n. R. Mr 2, fin
T; Us
22
4166
AIRFOIL
TI, Mps Ugs P.
Vir Uyy
46
11015
BEZIER
My
Vy
0
57
986
2D- AD-ENUURYSM
Tt, Ux. Uy, Vin
U2,
243
3D-ANRURYSM
n1, Vay Pyy U2, Un
Up, Mys M/
981
540000
In A.I.I n is the node type, fm the force being applied at the current time step to the object and Un the inflow velocity at the current timestep. When history is different than O, first-order derivative the inputs as extra feature. For example, we also add az ag ag to each node from an aneurysm mesh.
A.1.2 NOISE
Following the same strategy as Sanchez-GGozalee et al. (2020), we make our inputs noisy. More specifically, add random noise (100 σr) to the dynamical inputs. Each noise either selected from previous papers, or selected by looking at average one-step error predictions. Noises are presentend in Table A.1.2
14

---

Published as a conference paper at ICLR 2025
Error Distribution Over Time
atddd
HO
Roojatey
2000
2003
1000
1003
-6.25
Errer
Emoor
Ensr
vy
Frogutey
2000
Troquanyy
1000
1003
Erior
Error
Enor
Propanty
1000
-6
-2
Froeantd
1000
3000
2003
13.55
5:00
1:00
Frecunce
Precurray
Eroor
Figure 10: Error distribution on different timesteps in trajectory predicted from the 3D-Aneurysm dataset. The magnitude evolves both with the axis and with the time.
Dataset
Noise
Sub-mesh
CYLINDER
0.02
PLATE
0.003
FLAGSIMPLE
0.001
AIRFOIL
10
BEZIER
2D ANBURYSM
10
3D-A AN-URYSR
U2. Uy: 10, M2: 0.5
100000, 10
A.1.3 ERROR DISTRIBUTION
In the case of 2D-ANEURYSM and 3D-ANEERYSM, since the velocity inflow different depending on the timesteps, the error distribution varies lot (see Figure T0p.
While this did not prove to be an issue 2D, we had more trouble to find an acceptable noise in 3D. One solution that tried but proved to be unsuccessful was dynamic noise depending on the time steps, using the error distribution seen above. This led to similar results to the static noise we end up using.
A.2 SUB-MEST PARTITIONING
Some meshes from the 3D- AD-AANDUS AD-AANBRYSSM dataset were too large to fit in GPU memory and thus necessitated to be split into sub-meshes. The strategy applied was to split one mesh into smaller sub-meshes, and instead of one gradient descent apply multiples ones each of the sub-mesh. Some sub-meshes are presented Figure
Sub-meshes generated using two strategies: random neighbor sampling strategy from Hamilton et al. (2018) using between 50000 and 100000 random edges. The second strategy used the METIS Padua et al. (201T) algorithm to generate between 7 and 15 disjoint sub-meshes.
We conducted an extensive study to compare results on model trained with and without this uub-meshs strategy and found meaningful difference in accuracy.
15

---

Published as a conference paper at ICLR 2025
Figure 11: Different sub-mesh generated at each batch update.
1,000
Railoss IMMSS x1I
800
all-obbott
Rollon TMSSE xX0I
600
10
16
20
30
10
76
00
10
15
20
Number of Netroni
Figure 12: (left) Ablation study of the number of message passing steps. We encounter a plateau starting from 10. (right) Impact of the number of neurons per layer. We encounter plateau starting from n = 128.
A.3 ABLATION STUDY
We also conducted an ablation study the Cylinder dataset regarding the number of message passing steps and the number of neurons per layer. Results can be seen in Figure [2]
A.4 ADDITIONAL RESULTS
Additional Results are presentend in Figures 13 and 14
16

---

Published as a conference paper at ICLR 2025
Figure 13: Uncurated random shapes from the validation cylinder and bezier shapes. (Ieft) masked, (middle) predicted and (right) original.
Figure 14: Uncurated random shapes from the validation aneurysm and bezier shapes. (left) masked, (middle) predicted and (right) original.
17


Reviews:
[{"review_id": "20aRAyrvtv", "reviewer": "Reviewer_thnV", "summary": "The paper proposes a masked pretraining strategy for learning mesh-based simulation. An asymmetric encoder/decoder architecture is proposed for doing masked-node reconstruction. Throughout numerical experiment, the authors showcase the pretraining can improve temporal prediction accuracy over other baselines.", "questions": "* Does the masking algorithm consider irregular connectivity? For example, a sub-graph is detached from other sub-graphs after masking.\n\n* Following point 2 in the Weakness, can authors provide more details in terms of the training time/ number of parameters of different models?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The proposed pretraining consistenly improves GNN’s performance on various physical systems.\n\n* The numerical experiments are extensive and cover relatively challenging problems. Specifically, a large-scale 3D simulation with 250k nodes is studied.", "weaknesses": "* The discussion related to the model architecture and the graph construction after masking is relatively vague. The authors say they used a multi-grid GNN with a W-cycle structure and is a Encoder-Processor-Decoder setup, but it is also stated that in the pretraining section the model for pertaining is an Autoencoder with graph encoder/decoder. Does the model architecture and grid structure change or remain the same in pretraining and fine-tuning? In addition, as the masking nodes are randomly sampled, is the masked mesh still valid after an excessively high masking ratio?\n\n* For the numerical experiments, the authors describes the detail for pre-training and fine-tuning of their proposed model, but it seems that there is not too much information in terms of the training budget of the baseline model (or I have missed them). For example, suppose training a GNN on next-step prediction without masking for 100k steps needs 1e9 flops, and then pretraining it for 100k steps needs another 1e9 flops. Then for the baseline models without pretraining, a fair comparison would be training them using 2e9 flops.\n\n* Following the last point, if the pretraining can be done on a dataset order-of-magnitude larger than almost all downstream tasks (for example, LLMs), then maintaining similar flops between pretraining/no pretraining model indeed becomes meaningless. However, according to the result shown in Table 3, pretraining on a mutli-physics dataset actually performs worse than pretraining on the same physics, and pretraining on a completely different physics is actually harmful to the downstream tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gCaphkFuSe", "reviewer": "Reviewer_i9XS", "summary": "The paper introduces a masked pre-training technique for Graph Neural Networks (GNNs) aimed at solving physics simulations, particularly in computational fluid dynamics. The proposed approach uses an asymmetric encoder-decoder architecture in conjunction with gated multi-layer perceptrons. Extensive experiments are conducted, including independent training on multiple datasets, transfer learning, and multi-dataset pretraining. The results demonstrate that the MeshMask model delivers competitive performance compared to established baselines.", "questions": "1. Given that your input mesh includes masked data, how do you effectively learn long-range interactions on the masked graph during the repeated message passing?\n2. In Sec 4.3,  given that the input and output physics quantities differ across the datasets, the potential for a significant gap in the encoded latent space raises concerns about the legality and effectiveness of the transfer learning approach. How do you ensure that the latent spaces of the two datasets are compatible? Secondly, the length of input and output quantities varies across datasets; how do you exactly fine-tune the model when faced with these differing lengths?\n3. What is the model's performance on out-of-distribution (OOD) mesh resolutions?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The paper introduces a novel approach by implementing masked pre-training for Graph Neural Networks (GNNs) in physics simulations. It is well-structured and easy to follow, with rich qualitative results and visualizations.", "weaknesses": "1. The introduction of gated MLP and encoder-decoder architecture potentially increases the model's computational complexity. There is a lack of detailed discussion on the computational demands (e.g. training/inference time,  training/inference RAMs) compared with baselines, which is critical for evaluating the feasibility of deploying MeshMask in real-time applications or on large-scale datasets.\n2. The compared baselines are limited, authors should consider more GNN-based simulation models such as [1-4]. Furthermore, both GCN and U-Net baselines have only been tested on a few datasets.\n3. The novelty of replacing standard MLPs with gated MLPs in the processor is somewhat limited. The authors should consider a wider variety of network architectures such as attention-based models [1,2,4] for comparison to make their findings more persuasive.\n\n[1] Eagle: Large-scale learning of turbulent fluid dynamics with mesh transformers.\n\n[2] Learning flexible body collision dynamics with hierarchical contact mesh transformer.\n\n[3] Efficient Learning of Mesh-Based Physical Simulation with BSMS-GNN.\n\n[4] Transformer with Implicit Edges for Particle-based Physics Simulation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "T0uwpbDrfM", "reviewer": "Reviewer_gGsd", "summary": "This paper introduces MeshMask, a GNN masking pre-training technique designed to enhance the efficiency and performance of GNNs in solving CFD problems. Drawing inspiration from the success of masked autoencoders in NLP and CV, the authors adapt masking pre-training techniques to GNN models in the CFD domain. MeshMask demonstrates performance improvements across multiple CFD datasets.", "questions": "1. Please discuss and address the concerns raised in the weaknesses.\n\n2.  Why is the transfer learning on CFDs effective? Could the authors provide some high-level discussion about the underlying mechanisms?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The authors developed an appropriate masking pre-training scheme for GNNs in CFD tasks.\n\n2. The method shows notable performance improvements compared to baseline approaches.\n\n3. The paper is well-organized, with clear figures and visualizations that help the reader understand.", "weaknesses": "1. The comparison seems limited to very basic baseline methods~(i.e., MGN, Multigrid). Given the recent attention to GNN-based CFD modeling, more recent baselines could have been included in the comparison.\n\n2. Despite multiple claims about *training efficiency*, there appears to be a lack of comprehensive discussion on efficiency metrics.\n\n3. The experiments lack testing across varying *graph density* scenarios. The effectiveness of the method across different graph densities could have been better demonstrated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)