Conditionally Parameterized, Discretization-Aware Neural Networks for Mesh-Based Modeling of Physical Systems
Authors: Jiayang Xu
Aniruddhe Pradhan
Karthikeyan Duraisamy
Venue: NeurIPS 2021 Poster (2021)
Keywords: mesh-based model
graph neural networks
conditional parameterization
computational physics

Abstract: Simulations of complex physical systems are typically realized by discretizing partial differential equations (PDEs) on unstructured meshes. While neural networks have recently been explored for the surrogate and reduced order modeling of PDE solutions, they often ignore interactions or hierarchical relations between input features, and process them as concatenated mixtures. We generalize the idea of conditional parameterization -- using trainable functions of input parameters to generate the weights of a neural network, and extend them in a flexible way to encode critical information. Inspired by discretized numerical methods, choices of the parameters include physical quantities and mesh topology features. The functional relation between the modeled features and the parameters is built into the network architecture. The method is implemented on different networks and applied to frontier scientific machine learning tasks including the discovery of unmodeled physics, super-resolution of coarse fields, and the simulation of unsteady flows with chemical reactions. The results show that the conditionally-parameterized networks provide superior performance compared to their traditional counterparts. The CP-GNet - an architecture that can be trained on very few data snapshots - is proposed as the first deep learning model capable of standalone prediction of reacting flows on irregular meshes.

Conditioally-Prramettiiee, Diserratiaaeree Neural Networks for Mesh-Based Modeling of Physical Systems
Jiayang Xu davidxu@uiice edu
Aniruddhe Pradhan anipra@umich edu
Karthik Duraisamy kdur@umich edu
Department of Aerospace Engineering, University Michigan, Ann Arbor, MI 48109.
Abstract
Simulations of complex physical systems are typically realized by discretizing partial differential equations (PDEs) on unstructured meshes. While neural net- works have recently been explored for the surrogate and reduced order modeling PDE solutions, they often ignore interactions or hierarchical relations between input features, and process them as concatenated mixtures. We generalize the idea of conditional parameterization - using trainable functions of input param- eters to generate the weights of neural network, and extend them in flexible way to encode critical information. Inspired by discretized numerical methods, choices of the parameters include physical quantities and mesh topology features. The functional relation between the modeled features and the parameters is built different networks
into the network architecture. The method is implemented and applied to frontier scientific machine learning tasks including the discovery of unmodeled physics, super-resole resolution of coarse fields, and the simulation of unsteady flows with chemical reactions. The results show that the conditionally- parameterized networks provide superior performance compared their tradi- tional counterparts. The CP-GNet an architecture that can be trained on very few data snapshots is proposed as the first deep learning model capable of standalone prediction of reacting flows on irregular meshes.
Introduction
Numerical simulations of partial differential equations (PDEs) have become an indispensable tool in the study of complex physical systems. High-resolution simulations are, however, prohibitively expensive or intractable in many practical problems. Machine learning techniques have recently been explored to improve the efficiency and accuracy of traditional numerical methods. Successful applications include nonlinear model order reduction [1, 2, 3], model augmentation [4, 5, 6]. and rupertresolution [7, 8, 9]. Neural networks have also been used to replace traditional PDE-based solvers, and serve as a standalone prediction tool. Popular approaches include rego-seeressive time- series predictions [10, 11, 12, 13, 14], Physics-Informe Neural Networks (PINNs) [15, 16, 17].
Despite promising results on canonical problems, commonly used network architectures such as au- toencoders and CNNs have inherent limitations. An autoencoder generates a fixed mapping between geometric coordinates and the encoded digits. This limits their portability for new geometries and dynamic patterns. A CNN requires interpolation of existing data to structured, Fuclidean space, introducing additional cost and error. Irregular geometry boundaries require constructs such elliptic coordinate transformation [18] and Signed Distance Function (SDF) [1]. Moreover, mod- els often ignore the hicrarchical relations between heterogeneous features, and concatenate them into single input vector, e,g. the common concatenation of the edge and node features in Graph
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Neural Networks (GNNs). The learning of high-order terms remains mostly unguided - even simple quadratic terms are often fitted via number of hidden units brute-force manner.
With focus on mesh-based modeling of physical systems, we use the idea of conditional param- eterization (CP) build the hierarchical relations between different physical quantities well as numerical discretization information into the network architectures. The key contributions of our work are as follows :
1. We demonstrate that a drop-in CP modification can bring significant improvements for various existing models on several tasks essential to the modeling of physical systems.
We propose a conditionally parameterized graph neural network (CP-GNet) which effectively models complex physics such as chemical source terms, irregular mesh discretizations, and different types of boundary conditions.
3. We conduct extensive numerical tests and demonstrate state-of-the-ar performances on problems of different complexities, ranging from the basic viscous Burgers equation to a complex reacting flow.
2 Methodology
Conditional Parametrization: The idea conditional parametrization (CP) to use trainable functions of input parameters to generate the weights of neural network. To demonstrate this, we start from standard dense (fully connected) layer:
h(u: W, b) (Wu+
where u E R"* the input feature vector, Rh is the output hidden state vector, W R X and b E Th are the trainable weights and bias, and a is the activation function. It can be seen that in the evaluation stage, the values of W and b fixed regardless of the inputs. Thus the performance of Eq. (1) is largely limited by the interpolation range of training data.
By introducing parameter vector p R and trainable function f(p): R - R computes the weights W based on p, the conditionally parameterized version of Eq. (1) is given by:
x that
huu: f(p), b) off(p) b).
(2)
An easy way incorporate the formulation into existing neural network models is by making f single-layer MLP, the conditionally parameterized dense (CP-Dense) layer can be represented by:
h p: B, a (a ((W.p) + B)
It should be noted that this would bring change in the dimensions of weights and biases, which become W E ( Xnp, B E R".. When the layer width is kept the same, the total number of trainable parameters increases linearly with the parameter size ny. In applications, p is not limited to an additionally--ntroducee parameter. When simply taking as the parameter for itself, the quadratic terms will be introduced. High-order terms, which are prevalent in physical systems, can be easily modeled using multiple such layers. In Appendix B, we demonstrate how certain discretized PDE terms can be fitted exactly with simple conditionally parameterized layers.
2.1 CP-GNet for mesh-based modeling of physical systems
Graph representation of discretized systems: Consider physical system governed by set PDEs for a time-variant vector of variables qt).. Using the popular finite volume discretization, the com- putational domain is divided into contiguous small cells, indexed by i. The discretized form of equation can be written as:
(1:(1) 1 Σ f qj, Diy) Au s(q.).
(4)
dt Ω JEN())
'The source code is released to facilitate future research at https://github. com/ davidxu www.iiiwjaaayggg//oooo epnets

---

where is the cell-centered value of cell i, 22 is the volume (3D)arrea (2D) of the cell, and (i) the neighborhood set of cells around i. Between neighboring pair of cells i and j. A.j is area (3DDlength (2D) of the shared cell boundary, and n.j = (x; x xj is a vector between the cell center locations X; and xj. In the explicit numerical simulation of Eq. (2.1), solutions are updated by computing the increment of Aqu = q k+1 o between discrete time steps indexed by k, which is determined by two terms. The flux term f computes the exchange of quantity between neighboring cells, which is a complex function involving both the cell values as well as the vector between them, e.g- [19]. The source term computes physics that are local to the cell, such as the reaction of chemical species.
In setting, the discretized system is mapped to graph GVV, E),, defined by nodes V of size [V] = ny connected by edges E C V V of size [ = Re- Each node v; is located at the corresponding cell center X,, and each edge (i.j) corresponds to a shared boundary between the finite volume cells. Denoting the sets of mapped quantities on all nodes and edges of G, Q = fqu, VJ. N (n.j. (i,5) E}. the target is to develop graph neural network operator g that predicts the increment AQ g(Q ((Q*,N). N).
The architecture for the proposed conditionally parameterized graph neural network, CP-GNet, can be written in an encoder- prrooddr-pooesoor--ee -eecode form. A schematic is provided in Fig. 1. For clarity of different variables in the description of the network, we use u, for the latent variables node to distinguish from the physical variables q; and use eij for latent variables on edge (i,j) to distinguish from the vector M.j-
CP-GNet architecture:
Rollout prodiction
Fintto Discrettzation
Processor
ppul 1t"
Input
Noda encoder
Edge encoder
volume
Inpal of
veui By
Source
nuliiicc block
Irporte
Irpaca
Cell-centerec quantity of
section
Encoded node u."
CP Derse
EP Dense
Daran
Flac larm
IN
Processed node feature u²
CF-Dersee
Gunorl with
Edge vector mhy
Oorree IN t Oubutul
Oupur
raigites
CP-Darme
Encoded edge feature e
GRayP
Output
Message passing
Ads
Input feature
Conditional parameter
Output
Figure 1: Schematic of CP-GNet architecture
Encoder: Numerical solution of PDEs (e.g., the compressible Navier-Stokes equations) requires the processing of arbitrarily complex interactions between mesh elements. While large MLP archi- tectures can represent this complexity, the data requirements to reliably train such networks might be large. In contrast, our proposed encoder takes two CP-Dense layers, taking the output from the previous layer both the input and the conditional parameter. Through the encoder, high-order interactions can be easily extracted, allowing a degree of extrapolation by virtue of linearity. The CP-GNet uses separate, but similarly constructed encoders process the input node features q and edge features nij, respectively.
The flux term in Eq. (2.1) can be effectively approximated by CP message-passing (CP-MP) between adjacent nodes on a graph. The source term s, on the other hand, can be modeled by CP-Dense layers. In CP-GNet the processor consists of multiple identical blocks with inde- pendent weights. Residual connections are added between the blocks. As shown in Fig. I, each block includes CP-MP based section and CP-Dense based section to address the two types of terms. Modified from the Edge Conditioned Convolution (ECC) [20), the CP-MP computation is formulated as:
Processor:
W wy= = (ow,(W
+ +B
h, = Σ wiy ((W i5, (ل,,,, JEN())
(5)
3

---

where e is the output from the edge encoder, u, and u, are the latent node features from the previous layer, h; is the nodal latent output, and waj = Aty/22; is the flux weight from Eq. (2.1).
Decoder: It is common in PDE solver to use Jacobian matrix J 0u/Oh transform the variable increments Au = JAh. The decoder in the GP-GNet serves a similar purpose - to convert hidden variables to the output on the physical space. Similar to the encoder, the decoder consists of three conditionally parameterized dense layers. The first two layers can actually be viewed as a dedicated "encoder" that similar to the initial node encoder, taking o as the input, but with independent weights. The purpose of this "encoder" is to extract a final conditional parameter, which used in the third CP-Dense layer in the decoder to determine the weights for the output node feature uz from the processor. The third decoder layer is also the final layer of the model, which outputs o (with proper scaling). Except for the edge encoder and the last two layers in the decoder, all dense, CP-Dense, CP-MP layers are appended with LayerNormalization (LN) layers.
The computational domain of practical problem includes multiple types of boundaries, e.g. the case in Sec. 4.3. In classic PDE solvers, they treated with different boundary conditions, which define explicit formulations to compute relationships of the domain with the external world. However, these conditions and formulations are only defined for the physical quantities, thus cannot be easily transferred for latent variables, especially when multiple message- passing/convolution steps are used. To enable the GP-GNet to model different types of boundaries efficiently, special treatments are necessary. For boundaries with known inputs, such as the inlet and the outlet, their values are directly input to the corresponding nodes at every time step. For the boundaries imposing certain constraints. instead of a given physical value, such as Neumann and symmetry boundaries, ghost edges are introduced. For a cell with a face lying on a boundary, we introduce ghost edge vector Dig. that points from the corresponding node to the center of the boundary face. Ghost edges are processed together with the normal edges in the edge encoder. However, the CP-MP layer in the processor of the CP-GNet is slightly modified. More specifically, the concatenation [u.; uz] in Eq. (5) is replaced with only u, And for each type of boundary, the weights for the CP-MP layer are trained independently, to let the model learn different types of boundary condition for the latent variables. The effectiveness of this treatment is shown in Sec. 4.3 and further discussed in Appendix. A.3.3.
Treatments for boundaries:
3 Related Work
There have been successful attempts towards making networks directly parametric to certain fea- tures, such connectivity patterns [21]. layer embedding [22], mean image features [23]. The Con- ditionally Parameterized Convolution (CondConv) model [23]. makes convolution kernel weights as linear combination of functions of the input features, and achieves efficient expansion of the network capacity. The Hypernetwork [22] uses a single network that takes layer embeddings, e.g.- layer index, to generate the weights for different layers of the main network, and reduced the total number of trainable weights. A popular framework to perform convolution on graphs is the message passing neural network (MPNN) [24], which treats graph convolutions as messages passed between nodes through edges. In this approach, the node features and edge act on intermediate variables, and the output is expressed a linear combination through concatenation. This can fail when the impact of node features rely on the edge features in a non-linear fashion. To address this, Edge Conditioned Convolution [20] (ECC) makes the weights for node features dependent on edge features. After the modification for conditional parametrization, ECC was shown to achieve excellent performance on irregular point cloud data. comparison, our method extends the choice of parameters to physical quantities, hidden inputs themselves, as well as discretization information.
Multiple architectures in the family of GNNs have shown successes in processing irregular, non- Euclidean features. Applications include cloud classification [25, 26), action recognition [27] and control [28], traffic forecasting [29, 30]. quantum chemistry [24]. Attempts on using GNNs in scien- tific computation are relatively limited and are mostly focusing on particle-basee based methods [31, 32]. Recently, pioneering work has demonstrated the potential of using GNNs for mesh-based scientific computation. CFD-GCN [33] coupled a GNN with an existing PDE solver to perform hybrid-fidelit prediction and achieved higher efficiency than traditional high-fidelity solvers. MeshGraphNets [34] extends the mncodder-processor-deee and demonstrated impressive performance on mesh-based simulations for wide range of physical systems. Compared to these approaches, our method with CP models the high-order terms and irreg-
structure from Graph Network-based Simulators (GNS) [32],

---

ular discretizations more effectively. Appendix C compares our method with the MeshGraphNets on flow simulation tasks.
Numerical Tests
We applied conditional parametrization to network architectures for three distinct, but important tasks in scientific computing. The first two tasks are on uniform Euclidean grids, and the discretiza- tion information is directly included in the conditional parameters such as the differential terms and the local Reynolds number. Comparisons between appropriate baseline models and their CP modifications are performed. The third task uses an irregular mesh with complex boundaries and conducted with the CP-GNet model we proposed. The non-CP modification, which takes standard dense and message passing layers with more units, is used as the baseline. Appendix A. provides more details on the studied system and the generation of data; A.2 provides details on network training: A.3 provides additional results and analysis. An additional test for the flow over a cylinder is performed in the comparison against the MeshGraphNets in Appendix C.
4.1 Discovery and solution of coarse-grained models
In many practical problems, high-fidelity simulations are not affordable. Instead, computations are performed using coarse-grained models, e.g. the Large Eddy Simulation [35]. In such models, the small-scale physics are unresolved, and are approximated using additional closure terms in the PDEs, the development of which constitutes important area of research. In fact, even for the seemingly simple (yet richly non-linear) equation presented below, a perfect closure model is un- known. In this work, demonstrate how CP models can be used to develop closure model for the coarse-grained ID viscous Burgers equation that is often used in the study of shock formation, traffic flows, and turbulent interactions, etc. For the unknown spatio-temporal field u(x, t) spatially periodic domain x E [0, L], the original equation is given by:
Ou
Ou
0u u
M
0.
(6)
Ot
dr
Oz
where v is diffusion coefficient and u(x, 0) is random initial condition (See Appendix A.I.I).
When this equation is solved on finely discretized mesh, the dynamics can be regarded as fully resolved. However, if a solution is attempted on coarse mesh with Eq. (6) without any additional treatments, the solution becomes inaccurate and numerically unstable, thus a closure operator C()) is needed. Representing the quantity on the lower resolution mesh by the "closed" equation is:
Oū Øt
Ou dr
84 Ox
C= 0.
In this experiment, two baseline models for C and their CP developments are compared. The first model is 2-layer CNN with a dense layer with ReLU activation, followed by a ID convolution layer. This model assumes the closure term be a function of convection term 00 and the diffusion term 88
dü 0'0 the input. Its CP variant, CP-CNN, replaces Baa' and takes their concatenation q [uz Pz '0z2 the first layer with a CP-Dense layer that takes q as the parameter for its own weights. The second baseline model is reference Data-Driven Parameterization (DDP) model [36]. The model takes C as a function of the filtered variable ū, which is modeled by an 8-layer MLP with swish activation. Similarly, the CP variant, CP-DDP replaces the first layer with CP-Dense layer that takes q as the parameter for the weights for ū. The network architectures are presented in Fig. 2.
Two sets of data are used. The high resolution runs solved with Eq. (6) from two different initial conditions (ICs) on a shared 2048-grid-node mesh. The low resolution solutions are obtained by applying box filter to each step of the high resolution solutions onto 32-grid-node mesh. The ground truth for C is then computed based on the low resolution data. Each set of data consists of 267 time steps, spanning period of 2 s. The first 0.2 of data for one IC is used for training.
Online testing computations are then carried out from the filtered, low resolution ICs using Eq. (7), with C computed based on the online solution at every time step. x-l contours are present in Fig. to compare the evolution u. Spatial profiles are also plotted at few steps to provide more details. Despite small time step (CFL number< 0.5, without any closure term, the computation
5

---

Input 6 (1)
Inout 8
Input (?)
Dense (16)
CP-Dense (16)
Inpur (2)
Inpur (2)
4
Dense (32)
Dense (52)
Dense (18)
CP (10)
Donse (32)
(32)
ConviD Output)
ConviD
Outputi
Lense (1, Outpub
Dense (1, Output)
(a) CNN
(b) CP-CNN
(c) DDP
(d) CP-DDP
Figure 2: Closure modeling network architectures. Solid arrow: input feature; dashed arrow: condi-
tion parameter, numbers: layer width.
Table I: Closure model MAE. 7 Avg.: averaged over all steps for online prediction for ū; à final: for the final step of online prediction; Inf.: Unbounded cases.
Training IC
Testing IC
Avg.
u linal
Avg.
linal
CNN
0.23
0.41
0.16
0.23
CP-CNN
0.15
0.21
0.09
0.13
DDP
Inf.
Inf.
Inf.
Inf.
CP-DDP
0.42
0.89
0.3
0.41
is numerically unstable and the eπrо grows unbounded. The baseline CNN model is able to keep the solution stable within the period studied, and the CP-CNN improves the aceuracy noticeably. The baseline DDP model is only able to postpone the "blow-up" to slightly later. The solution with CP-DDP closure is bounded throughout the period. The improvements are also valid for both the unseen IC. The Mean Absolute Error (MAE) for ū is provided in Table. 1.
0=
0.5
t=1.0
2.0
t=0.1
0.5
1=0
1=2.0
No cloure CNR
No chaure
DCP
DOP
X
X
X
Tu5
No choure
No doars
-88
-58
CP-CKN
ONV
CP-CNN
CMN
XDP
0.
CP 00
D00
08
(5)
(i)
(5)
Training IC
Testing
Figure 3: Closure modeling results. The first t 0.2 for the left case is used for training, marked by the black dashed line in the first contour. The z-t contours show the evolution of The reference DDP model solution grows into infinity, shown white areas in the contour. The gaps between models are more visible in the spatial profiles at time steps marked by the cyan lines.
4.2 Super-resolution of chaotic flows
In this experiment, we perform enrichment of low-resolution snapshots of turbulent flow fields. In an enrichment/super-resool ressuuutttoo process, one inputs low-resolution snapshot the solution, and seeks snapshot with better resolution. One way to achieve different resolutions on a given mesh is to use Discontinuous Galerkin (DG) projection [37]. In this method, the solution within a mesh element is represented by coefficients for a set of polynomial bases, of which the size is determined by the polynomial order P. The final resolution of the solution is jointly determined by P and the element width L. More specifically, wall- wall-parallel snapshots from the solution of turbulent channel flow [38] studied, and the task to recover high-order (P = 3) DG coefficients a E R2 for the z-velocity from lower-order (P 1) ones at E Rd. snapshots are generated in total at different normalized wall-normal heights = E (650, 700, 750, 800, 850), illustrated in Fig. 4. Each snapshot spans an area of X X 2x म, and is projected onto shared set of uniform meshes with different widths L {T/4, 12, 16, x 7/24, π /32), for the two studied polynomial orders P. (1,3). Thus,

---

Table 2: Average and maximum absolute errors in the integral of super-resolved energy spectra.
Training
Testing
Eg Avg.
Eg Max.
Ey Avg.
Ey Max.
Es Avg.
Eg Max.
Ey Avg.
Ey Max.
MLP
0,0145 0.0120
0.0391
0.0272
0.0609
0.0098
0.0364
0.0184 0.0158
0.0675
CP-MLP
0.0328
0.0217
0.0429
0.0081
0.0260
0.0519
for each =_ 12 sets of data, each for one combination of L and P. are provided. Fig. 4 shows a few example contours at different combinations for z 800. The data for 2 E 700, 800 is used for training. It should be noted that the coefficients are computed independently for each mesh element, thus the total number of training points is a few thousand, instead of 24 (which should be multiplied by the number of elements). More details on the data generation are provided in Appendix 1.1.2.
Input [36];
Input/
Input (36)
Input
Dengg
Donse (48)
Dense (48)
Dense (48)
P.
Dense (32)
Dense (32)
P- 1
(9. Outsul)
19, Ouguu
(a) MLP
(b) CP-MLP
1-nie
Super-resolution network architec- tures Solid arrow: input feature; dashed arrow: condition parameter, numbers: layer width.
Figure 5:
Figure 4: Snapshots for super-resolution
In this task, the baseline model is from the compact super-resolution model by Pradhan and Du- raisamy [8]. It takes at as function of two inputs. The first input is concatenation of normalized low-order basis coefficients for and its neighbors (i):
[ = Ka J E (11) Uill)uuu RMS
(8)
where [[- denotes the concatenation of all elements in set, and a" is the mean of the set. In our case, we include all immediate neighbors, including corner ones in N (2), thus [a*]; E R The second input to the model is an indicator 4 = log( Re!-) for the loss of information in the low-order projection process. Ref =
RaMMs is the local Reynolds number. The indicator reflects that the loss is function of the kinetic energy, measured by uRMS, mesh resolution L, and fluid viscosity v. Because ReF can vary by orders of magnitude across elements, log scaling is used. The two inputs first concatenated and then processed in a 4-layer MLP in the baseline model. In contrast, the conditionally parameterized model CP-MLP processes only the first input [a"; in the dense layers. The second dense layer is replaced by CP-Dense layer, where the second input instead taken as a conditional parameter for the weights for the latent output of the first layer. A comparison of the model architectures is provided in Fig. 5.
Results for two sample testing cases, (2+ = 650, L = T/4) and (a+ 750, L = r/8) are shown in Fig. 6. It can be observed that the CP-MIP is able to reconstruct more small scale structures compared with the MLP. The performance can be qualified by the stream-wise and span-wise energy spectra, eg and Cy (see Appendix A.3.2 for definitions). e, for different stream-wise wave numbers ke is shown in Fig. 6. It can be observed that for high-order projection or super-resolution, the high- wave-number spectra are much richer than those for the low-order projection. The CP-MLP plots follow the truth noticeably better than the MLP baseline, which confirms our observation from the contours. Absolute error in the integrals of energy spectra, Es Sk. egdk; and Eg = Se, eydky are computed for the 24 training and 36 testing sets and summarized in Table 2. Both training and testing errors are reduced significantly when CP is applied.
4.3 Simulation of reacting flows in a rocket engine injector
We use highly complex public dataset [39] as a model of combustion processes in a rocket engine injector [40]. The dataset includes solutions 2D finite-volume mesh with 308184
7

---

Input (P=1)
Truth
MLP
CP-MLE MLP
Spectre
Ps1
Truth
650
L=
MLP
CP MLP
750 1.mii
16
Figure 6: Super-resolved flow field and stream-wist energy spectra e, for example test cases (z 650, π / and (zt 750, x x/8). The CP-MLP shows finer details on the edge of elements (adjacent squares), showing a better prediction of high-order coefficients. The observation is proved by richer high h energy spectra the right plot.
unknowns at every time instant. This includes eight variables at each discretized cell: q = [p, u, v, T, YCH4, Yoz. Yh2o, Yco4|", where is the pressure, u and v are the and y velocity com- ponents, 7 is the temperature and {Ycu4, Yoz, Yizo, Ycos), are the mass fractions for the chemical species involved in the combustion process. The injector is outlined in Fig. 7, where the oxidizer (02 diluted in H2O vapor) and fuel (CH4) are injected from two inlets, respectively, into a tube-like combustion chamber in which they mix and react. The products are exhausted through outlet. A probe monitor is placed inside the physics-intensive area, which is also marked in the figure. The strong instabilities in the simulation is triggered by strong 2000 Hz pressure perturbation the outlet. Fig. 7 shows the responses for and 7 at the probe. It should be noted that, although the pressure perturbation the outlet is periodic, the upstream behavior affected by complex coupled physics and is not as periodic, especially for other variables such as T" Fig. 8 shows the graph gen- crated following the method Sec. 2.1, where special nodes and edges, well as irregular local structures, are provided in zoomed-in views. Two groups of ghost edges are used, corresponding to two types of wall boundary conditions in the simulation: no-slip and symmetry, respectively.
No-slip wall
Probe
Outlet
Inlet
Symmetry wall
1.3
(m65)
Figure 7: Injector outline and probed response for p and T. Orange marker: probe location. L/S: long (0.2 s)/short (0.02 s) training period (0.2 s); P: prediction period (0.2 s).
Figure 8: Graph details. Black dots: standard nodes; black lines: standard edges; red dots: in- let/outlet nodes; green/blue lines: two groups of ghost edges (extruded for visualization).
In this experiment, we attempt to predict the future states of q using the CP-GNet introduced in Sec. 2.1. Two CP-GNets of two different depths, with a 5-block and 10-block processor respec- tively, are tested. Both CP-GNets work with an encoded node feature size of 36, and an encoded edge feature size of 4. The baseline model for comparison replaces all CP layers with standard dense layers of 128 units. More specifically, after the replacement, the layers taking node features conditional parameters will retain the original input. The layers originally taking edge features as conditional parameters will take a concatenation of the original inputs and the edge features as the new input. The non-CP model is referred to the GNet. GNets, with 10-block and 15-block processor respectively, are studied.
The simulation results sampled at time interval of 5 x 10 ms are used the ground truth. Tests are conducted on two different lengths training data. The long period consists 400 steps, spanning 0.2 ms, the last 10% of which is used the short training period. Thus, both periods end the same point, and rollout prediction carried out from the end of training for another 0.2 ms. These periods are illustrated in Fig. 7. For simplicity, add the number of processor blocks and L

---

(long) or (short) as suffixes to the model names to distinguish them. For example, "CP-GNet1OL" refers to the CP-GNe with 10 processor blocks trained on the long period. The predictions for representative variables, visualized in Fig. at steps evenly spanned over the prediction period. The probed results are also plotted, which also covers the other models tested. It is notable that small phase shift in the resolved structures can cause a high level deviation the probe measurements, and thus the flow field contours should be viewed broader indicators of the performance.
7, Yси4, from the two deeper models, CP-GNet10L. and GNet1 are
histery
122
D2 OOO
OPON huth
TOL
oNC o pan fcm
time]
Figure 9: Predicted reacting flow. From top to bottom: pressure p. velocity u, temperature T, mass fraction YCHI. CP-GNET maintains a high level of accuracy over multiple scales of mesh resolution and near complex geometry boundaries.
It is seen that the CP-GNET predicts the evolution of the reacting flow accurately hundreds of prediction steps. In comparison, the non-CP model deviates quickly from the ground truth within 100 steps. Even with smaller model (CP-GNet5L. 1.3M parameters), data (CP-GNet1OS), the CP models still show comparable or even better performances compared with the largest baseline 5L, 1.8M parameters). There is no significant difference in the level of error across the predicted field from our model, in spite of the vast changes in mesh density and distortion, whereas the GNets clearly suffer from more errors around the inner corners, where the mesh is the most irregular. This shows that, by combining CP with graph, discretization information can be efficiently processed. The proposed boundary treatment is also proven successful even in such a complex case with multiple types of boundaries (see Appendix A.3.3 for results without ghost edges).
small fraction of training
Summary
This work draws inspiration from discretized numerical methods, and generalizes the idea of ditional parametrization for mesh-based models. Conditionalll-parameteer ibly incorporate physical quantities well numerical discretization information into trainable weights, leading to efficient learning of high-order and unstructured features. Drop-in modifications are demonstrated on different architectures for several important tasks related mesh-based mod- cling of physical systems. Considerable performance improvements are achieved in the numerical tests compared with the traditional counterparts. In the coarse- gaanse-grainngg and super-resolution tasks, small network with simple CP-Dense layer is capable of stabilizing or improving numerical so- lutions.
networks can flex-
of future state prediction of rocket injector, the CP-GNet is shown be capable of predicting the flow with a complex combustion process for few hundred steps on an irregular mesh. Although direct CP modification will cause a linear increase in the number of parameters w.r.t. the chosen parameter, such an increase can be compensated by reducing the size of the latent vectors. Indeed, the CP-GNet more efficient than the non-CP variant with only a fraction of the training data or with more shallow architecture. In the appendix, we compare the CP-GNet with

---

the MeshGraphNet on two flow simulation tasks. Overall, the proposed architecture improves the potential for incorporating physical intuition as well knowledge of numerical discretization.
Acknowledgments
J.X and K.D acknowledge support from the Air Force under the Center of Excellence grant titled Multi- Fideliy Modeling of Rocket Combustor Dynamics. A.P. is supported by NASA under the grant COONSSCIMM014 We thank Alvaro Sanchez-Gonzalez and Peter Battaglia for valuable advice on training noise injection for robust prediction.
References
[1] X. Guo, W. Li, F. lorio, Convolutional neural networks for steady flow approximation, in: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, 2016, pp. 481-490.
[2] K. Lee, K. Carlberg, Model reduction of dynamical systems nonlinear manifolds using deep convolutional autoencoders, arXiv preprint arXiv: 1rXiv:1812.088 1812.8833 (2018).
[3] T. Murata, K. Fukami, K. Fukagata, Nonlinear mode decomposition with convolutional neural networks for fluid dynamics, Journal of Fluid Mechanics 882 (2020).
[4] A. P. Singh, S. Medida, K. Duraisamy, Machine-leaminn--aggmeeee predictive modeling of turbulent separated flows over airfoils, AIAA journal 55 (2017) 2215-2227.
[5] J.-L. Wu, H. Xiao, E. Paterson, turbulence models: A comprehensive framework, Physical Review Fluids 3 (2018) 074602.
machine learning approach for augmenting
[6] K. Duraisamy, Perspectives on machine learning-augmentedd reynolds-averaged and large eddy simulation models of turbulence, Physical Review Fluids 6 (2021) 050504.
[7] K. Fukami, K. Fukugata, K. Taira, Machine-lcarning-ball based spatio-temporal super resolution reconstruction of turbulent flows, Journal of Fluid Mechanics 909 (2021).
[8] A. Pradhan, K. Duraisamy, Variational multi-scale super-resolution A data-driven approach for reconstruction and predictive modeling of unresolved physics, 2021. 2tt.::2111099900
[9] L. Guo, S. Ye, J. Han, H. Zheng, H. Gao, D. Z. Chen, J.-X. Wang, C. Wang, Ssr-vfd: Spa- tial super-resolution for vector field data analysis and visualization, in: 2020 IEEE Pacific Visualization Symposium (Pacific IEEE Computer Society, 2020, pp. 71-80.
[10] J. Xu, K. Duraisamy, Multi-level convolutional autoencoder networks for parametric prediction of spatio-temporal dynamics, Computer Methods in Applied Mechanics and Engineering 372 (2020) 113379.
[11] F. J. Gonzalez, M. Balajewicz, Deep convolutional recurrent autoencoders for learning low- dimensional feature dynamics of fluid systems, arXiv preprint arXiv: 100.10088014 (2018).
[12] R. Maulik, B. Lusch, Balaprakash, Reduced-order modeling advection-dominated systems with recurrent neural networks and convolutional autoencoders, arXiv preprint arXiv:2002.00470 (2020).
[13] A. Mohan, D. Daniel, M. Chertkov, D. Livescu, Compressed convolutional Istm: An efficient deep learning framework to model high fidelity 3d turbulence, arXiv preprint ArXiv:1903.0000 00030 (2019).
[14] R. Maulik, Mohan, B. Lusch, S. Madireddy, P. Balaprakash, Livescu, Time-series learn- ing of latent-ppace dynamics for reduced-order model closure, Physica D: Nonlinear Phenom- ena (2020) 132368.
[15] I. E. Lagaris, A. Likas, D. Fotiadis, Artificial neural networks for solving ordinary and partial differential equations, IEEE transactions neural networks 9 (1998) 987-1000.
10

---

[16] M. Raissi, P. Perdikaris, G. E. Karniadakis, Physics-inforee informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, Journal of Computational Physics 378 (2019) 686-707.
[17] L. Sun, H. Gao, S. Pan, J.-X. Wang, Surrogate modeling for fluid flows based on physics- constrained deep learning without simulation data, Computer Methods in Applied Mechanics and Engineering 361 (2020) 112732.
[18] H. Gao, L. Sun, J.-X. Wang, Phygeonet: Physics-informed geometry-adaptive convolu- tional neural networks for solving parametric pdes irregular domain, arXiv preprint arXiv:2004. 13145 (2020).
[19] E. F. Toro, Godunov methods: Theory and applications, Springer Science & Business Media, 2012.
[20] M. Simonovsky, N. Komodakis, Dynamic edge-conditioned filters in convolutional neural networks on graphs, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 3693-3702
[21] K. O. Stanley, D. B. D' A'mbrrsii,, Gauci, A hypereube-basee encoding for evolving large- scale neural networks, Artificial life 15 (2009) 185-212.
[22] D. Ha, A. Dai, Q. V. Le, Hypernetworks, arXiv preprint 161..099090000 (2016).
[23] B. Yang, G. Bender, Q. V. Le, J. Ngiam, Condconv: Conditionally parameterized convolutions for efficient inference, in: Advances in Neural Information Processing Systems, 2019, pp-
1307-1318.
[24] J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, G. E. Dahl, Neural message passing for quantum chemistry, arXiv preprint arXiv:1704.01212 (2017).
[25] L. Landrieu, M. Simonovsky, Large-scale point cloud semantic segmentation with superpoint graphs, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 4558-4567.
[26] Y. Wang. Sun, Z. Liu, S. Sarma, M. M. Bronstein, J. M. Solomon, Dynamic graph enn for learning on point clouds, Acm Transactions On Graphics (tog) 38 (2019) 1-12.
[27] S. Yan, Y. Xiong, D. Lin, Spatial temporal graph convolutional networks for skeleton-based action recognition, arXiv preprint arXiv: Siti::800107455 (2018).
[28] A. Sanzhez-Gonzalez, N. Heess, J. T. Springenberg. Battaglia, Graph networks as learnable physics engines for inference and control, in: Inter- national Conference Machine Learning, PMLR, 2018, pp. 4470 4479.
J. Merel, M. Riedmiller, R. Hadsell,
[29] B. Yu, H. Yin, Z. Zhu, Spatio-temporal graph convolutional networks: A deep learning frame- work for traffic forecasting, arXiv preprint arXiv:
(2017).
[30] Zhang, X. Shi, J. Xie, H. Ma, I. King, D.-Y. Yeung, Gaan: Gated attention networks for learning on large and spatiotemporal graphs, arXiv preprint arXiv:1 880.07294 (2018).
[31] Y. Li, J. Wu, Tedrake. B. Tenenbaum, A. Torralba, Learning particle dynamics for manip- ulating rigid bodies, deformable objects, and fluids, arXiv preprint arXiv:1810.01566 (2018).
[32] A. Sanchez-Gonzzlez,, J. Godwin, T. Pfaff, R. Ying. J. Leskovec, P. W. Battaglia, Learning simulate complex physics with graph networks, arXiv preprint arXiv:2002.09405 (2020).
[33] F. d. A. Celbute-Peres, T. Economon, Z. Kolter, Combining differentiable pde solvers and graph neural networks for fluid flow prediction, in: International Conference on Machine Learning, PMLR, 2020, pp. 2402-2411.
[34] T. Pfaff, M. Fortunato, Sanchez-GGozaleer P. W. Battaglia, Learning mesh-based simulation with graph networks, arXiv preprint arXiv:2010.03409 (2020).

---

[35] P. Moin, Advances in large eddy simulation methodology for complex flows, International journal of heat and fluid flow 23 (2002) 710-720.
[36] A. Subel, A. Chattopadhyay, Guan, P. Hassanzadeh, Data-driven subgrid-scale modeling of forced burgers turbulence using deep learning with generalization to higher reynolds numbers via transfer learning, Physics of Fluids 33 (2021) 031702.
[37] B. Cockburn, G. E. Karniadakis, C.-W. Shu, Discontinuous Galerkin methods: theory, compu- tation and applications, volume 11, Springer Science & Business Media, 2012.
[38] C. Del Alamo, Jiménez, P. Zandonade, R. D MOSER, Scaling of the energy spectra of turbulent channels, Journal of Fluid Mechanics 500 (2004) 135.
[39] C. Huang, K. Duraisamy, C. L. Merkle, Investigations and improvement of robustness of reduced-order models of reacting flow, AIAA Journal 57 (2019) 5377-5389.
[40] C. Huang, R. Wentland, Duraisamy, C. Merkle, Model reduction for multi-scale transport problems using model-form form preserving least-squares projections with variable transformation, Journal of Computational Physics (2021) 110742.
[41] C. Basdevant, M. Deville, P. Haldenwang, J. Lacroix, J. Ouazzani, R. Peyret, P. Orlandi, A. Pa- tera, Spectral and finite difference solutions of the burgers equation, Computers & fluids 14 (1986) 23-41.
[42] M. E. Harvazinski, C. Huang, V. Sankaran, T. W. Feldman, W. E. Anderson, C. L. Merkle, D. G. Talley, Coupling between hydrodynamics, acousties, and heat release in self-excited unstable combustor, Physics of Fluids 27 (2015) 045102.
[43] S. A. McQuarrie, C. Huang, K. Willcox, Data-driven reduced-order models via regularized operator inference for a single-injector combustion process, arXiv preprint 2Xiv:2008.02862 (2020).
12


Reviews:
[{"review_id": "r4TTdvez6qf", "reviewer": "Reviewer_t6t9", "summary": "The paper presents a new \"conditionally parametrized\" NN layer for learned physics models, and demonstrates higher prediction accuracy on several physics prediction tasks by injecting this new layer into CNN and GNN-based models.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This is an interesting paper; the authors showed results on a range of interesting & complex physics prediction tasks, and the method seems to perform well compared to state-of-the-art baselines. I however wish the paper did a more thorough analysis as to why the model performs better, and provide the relevant ablation experiments.\n\nThe method is first introduced and motivated as a type of hyper network, i.e. predicting the weights using a different network. I'm not sure this is the most useful framing; there is no separation between the network and hyper network, the model is trained end-to-end, and in most cases conditioning p and input u as set to be the same vector.\nLater in the paper it is mentioned at that the CP layer performs well because it explicitly contains products of u, and thus can better model the nonlinear PDE terms. This intuition makes much more sense to me; and I think in spirit this paper is much closer to multiplicative conditioning methods such as e.g. FiLM [Perez et al 2017] than hyper networks. \nIf this is the case, I'd be very interested to see how other, less exotic form of multiplicative layers would perform-- e.g. f(g(u) * h(p)) (with * being element-wise multiplication), forms of FiLM, or simply providing the matrix u'u as an input to the first network layer. This is a bit of a missed opportunity, as it could not only provide more insight in where the performance gains really come from, but also allow to construct a layer which can use wider latents without explosion in parameters (as CP-Dense is O(n^3)).\n\nThe second claimed contribution (\"discretization-aware\") of the method over other MPNN models is explicitly weighting the incoming edge messages with the FVM flux weight w_ij. It should probably be mentioned that this only works with cell-centered data from a FVM simulation, and wouldn't work on the particle/FEM datasets used in related work. More importantly, there needs to be an ablation to determine how much of the performance gain is due to this, vs. the CP-* layers. This should be very easy to do; remove the weighting in eq.5 and instead provide the unnormalized edge vector x_i-x_j (and perhaps edge length) as an edge features as in [32, 34].\n\nI really enjoyed the range of complex physics tasks studied in the results section. However, the comparison often were not performed in an apple-to-apples manner, e.g. replacing a Dense layer (O(n^2) parameters, 1 nonlinearity) with a CP-Dense layer (O(n^3) parameters, 2 nonlinearities). These comparisons really need to be same-capacity, i.e. more & wider Dense layers.  \nThis is particularly obvious in 4.1: it's really not that surprising that a 2-layer CNN won't be very good at capturing nonlinearities, but perhaps a deeper network with matched capacity might.\nThe other observation on results is that the test sets are generally in-domain; and the only generalization test in the appendix showed stability issues. This is slightly concerning, as [32, 34] demonstrate stable very long-term rollouts and strong OOD generalization, so it'd be good to a) show baseline comparison for this generalization test to determine if this behavior is specific to the CP model, and b) investigate the cause of the instabilities.\n\n---\nMinor comments:\n- Line 75: wrong equation reference\n- I assume DDP has a Dense(16), not CP-Dense(16) layer?\n- How exactly does the CP layer in CP-CNN work?\n- Tab:1 is highlighted incorrectly, DDP has lower c_avg/c_max", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "No issues here.", "review_text": "", "extra_scores": {"ethical_concerns": "None.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "pJpmFK0SWWO", "reviewer": "Reviewer_ifyj", "summary": "This paper proposes methods for neural simulation of physical systems on graphs. In particular it proposes the use of hyper-network style layers (CP-dense) where the weights of the layer are determined by regressing some context parameter. These layers are combined in the CP-GNet architecture, which shares a similar encoder / process / decode structure to MeshGraphNets. The proposed layers are shown to yield performance improvements on simulation tasks relative to dense layers.", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper’s main contribution is the use of hypernetwork-style layers (CP-dense) in the context of neural simulations on meshes. CP-dense layers are used primarily to integrate heterogeneous input sources. While I don’t believe this is a particularly novel architectural innovation, its effectiveness in this setting is useful to study. It may be of interest to explore other ways to combine inputs, e.g. gating or attention mechanisms. ‘Multiplicative interactions and where to find them’ (Jayakumar et al, 2019) surveys some of these methods. It would also be of interest to evaluate the architecture on more standard graph processing tasks to see if similar gains can be achieved.\n\nThe tasks look reasonable, and in particular predicting reacting flows may be of interest to the community. \n\nFor the first two tasks, the baseline models with dense layers have fewer parameters than the CP-dense versions, due to the additional parameters introduced in the CP layers. This is problematic given the very small models used, as additional parameters are likely to have a large impact on function approximation capacity. For the DDP models in the first task, the baseline has access to less information than the proposed model. I’d expect the appropriate baseline to concatenate the u and q values in the inputs. For these reasons it’s hard to determine the additional expressive capacity granted by CP layers beyond an addition of model parameters / information.\n\nFor the reacting flows task, the baseline method is a variant of the proposed CP-GNet architecture that replaces the CP layers with standard dense layers. This baseline is described as being similar to MeshGraphNets, and uses larger hidden sizes / a greater number of layers, presumably to make the overall parameter counts similar. It would be helpful to report the overall number of parameters for the models used, as it’s hard work to determine if the counts are actually similar. \n\nThe evaluation would be stronger if the proposed model was compared prior work such as MeshGraphNets itself and other baselines on a common task. Otherwise we can’t really determine which model is more performant in general.\n\n\nOverall, I think the proposed CP-dense layer and the associated architecture CP-GNet are useful contributions to the mesh-based simulation field; more work needs to be done to evaluate the methods against parameter-normalized baselines, or against alternative work on common tasks.\n\nMinor critique: I found some parts of the introduction a bit vague, in particular the lines: \n\n[l28-31] MLPs and autoencoders typically observe the solution over the computational domain as a single entity (for every time instance), and the physical quantities from different parts are processed as entangled vectors through hidden layers\n\n[l34-35] Current models often ignore the interactions or hierarchical relations between input features and process them as concatenated mixtures\n\nI now understand that the authors are contrasting hypernetwork-style mixing of inputs from heterogeneous domains, with simple concatenation methods. But on first read I didn’t get it, so I think this paragraph could do with a bit of a re-work.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have discussed some of the limitations: particularly the parameter count scaling with the dimensionality of context parameters. I think the limitations of the evaluation (mentioned above) haven’t been addressed adequately. I agree with the authors that the potential for negative societal impact of the work is low.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "Y4BhaakaqZU", "reviewer": "Reviewer_hEWt", "summary": "The authors of the paper are adding a  parametrized (by input data) layer to perform a renormalisation step from a small scale PDE to larger scale system. This requires a definition of a closure term that modifies the equation at the larger scale to match the behaviour at the a more accurate, smaller scale. This is shown to significantly reduce the required computing resources, while still retaining high accuracy.  More, the system is defined on graph convolutional network that is applicable to complex mesh structures that are required for proper description of many real world problems.\n\nThe authors introduce a simplified form of the attention mechanism that is missing the value vectors and softmax portions, but gives a possibility to introduce a degree of  non-linearity to the activation, besides the activation function. The expectations is that this will help the network to reach accuracy with fewer computations. This is  validated in the  CFD cases considered in the paper.", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The authors report significant advances for easing the computational load of turbulent flows using AI methods. It is also done in a manner that requires only limited, practical amount of for training as quite a lot of the architecture is directly defined by the nature of the PDE to be solved.\n\nThe text is well, written and understandable, although it requires a bit of domain knowledge on both of the AI side and numerical methods for solving PDEs. All essential parts are explained and appendix provides clarifying information. The authors make a good work in compressing the topic to a concise package. Especially the I like the Figure 1 that describe the used architecture in clear and explicit manner. The related work is well represented, even in terms of comparing performances.\n\nAs their first test case, the authors use the 1-d, Naiver stokes equation with viscosity i.e the Burgers equation. The authors do not mention the Courant condition, and how the viscosity used would be related to the real physical value of the viscosity the system represents. Doubling the spatial size, would require a significant reduction of the temporal step. Also, how the stability of the method is followed on irregular meshes with a given time step size. The authors should, perhaps in the Appendix, consider how their scheme is solving these issues. Just looking at nice conforming solution examples does not bring a certitude required from a PDE solver. One-dimensional Burgers equation has analytically expressed solutions that can be used for verification as well. The example on rocket engine injection, using public high quality numerical solution, shows quite impressive performance of the method. \n\nFrom a point of view of the deep learning, the architecture, and the structure of the new solution is described well, Also the the training data and the training process is clear. \n\nSome interesting points remain unsolved: as far I understand the system trains itself on a solution itself that it will propagate further in time. Hence, one has to start solving the system from a given boundary conditions (and parameters like the viscosity), train the proper parametrised weights for the system (restricted  amount of data needed!) and the the AI solutions brings an efficient prediction of the future values. This does not show a very high degree of generalisation. Could one solve the system with a more extended set of varying conditions, like the changing, for example,  the viscosity (the physics of the system) and test it with viscosity values it has not been trained on before? Of course, the viscosity could be entered in as a conditional parameter.  This would increase the usability of the code, as a more general form of the closure would be already trained and available. I would try to remove the need of a training step in the inference that now seems to be there, at least in the examples shown.\n\n\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "No limitations. The code may have social impacts if the results are used for traffic or crowd modelling, and the models would not be accurate enough for example for escape route planning from burning buildings.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {"ethical_concerns": "The paper domain does not bring up ethical questions. Some use cases in the above mentioned domain may be considered. My personal opinion is that this is technical development that has still some steps to go before \"blind\" adoptions for crowd and traffic control applications. I do not think an ethical review is needed. The ethical aspects arise in those applications when someone starts utilising the results of this work  for those purposes. This is just to remind the authors that ethical consideration may arise from unexpected uses of algorithms. \n\nOf course, this comment, is valid for all PDE solvers that can be used for in the realm of social science.  I think an ethical review is applicable when there is a paper working in those specific areas explicitly. "}}, {"review_id": "P9PTEb8OMvW", "reviewer": "Reviewer_o2jT", "summary": "The paper proposes to incorporate the idea of 'conditional parametrization' (CP) in neural network modeling of PDEs. In experiments covering a closure model, superresolution, and time evolution of a multidimensional system, CP is shown to significantly improve results.\n", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The authors propose that the idea to use a neural network to generate the weights of another neural network is useful in the context of PDE modeling. This concept, referred to as CP in the present work, is not in itself novel -- similar solutions were discussed before in CondConv, hypernetworks or Edge Conditioned Convolution. The specific application to PDEs in general, and PDEs modeled with GNNs in particular, is a contribution of the present work. Prior work is appropriately cited and discussed.\n\nThe main claim of why CP is helpful is that \"higher order terms and irregular discretizations\" can be modeled more efficiently. In the appendix, the authors show that in some simple cases when CP is used and the network is appropriately structured, the exact form of the PDE can be recovered even with very limited training data. CP is then tested on 3 problems of increasing complexity, of which only the last involves non-Euclidean grids and GNNs.\n\nThe 1st problem is closure modeling for the coarse-grained Burgers' equation. The baselines are simple CNN and MLP models, which are then extended with CP. CP is shown to keep the solution bounded for all models and significantly improves precision for the CNN.\n\nThe 2nd problem is super-resolution in turbulent channel flow snapshots, realized with a discontinuous Galerkin projection. The network is trained to predict high-order basis coefficients from low-order basis coefficients for the node and its neighbors, as well as the local Reynolds number, which is concatenated to the input when an MLP is used and provided as a parameter to the CP-MLP. CP-MLP is visually shown to produce more complex flow patterns, richer energy spectra, and lower energy errors.\n\nThe 3rd problem is combustion in a 2d model of rocket engine injector, solved with a 2d FVM on a mesh with ~38k nodes, each containing 8 variables. In this experiment, variants of MeshGraphNets with and without CP are tested, with the CP ones staying significantly closer to the ground truth. No metrics are provided for this experiment, but a visualization of the flow fields together with the measurements from a probe point are sufficient to see the difference.\n\nThe paper is easy to read, introduces core concepts in a sensible order, and keeps the right amount of details in the main text. A statement of contributions could be a helpful addition so that it's clear what exactly is new. For instance, ~2 pages are spent introducing the finite volume formulation and CP-GNet, which can be viewed as a CP modification of MeshGraphNets. Some of the claims could be better explained and motivated -- for instance, in the discussion a claimed advantage of CP is that it improves the \"potential for incorporating physical intuition as well as knowledge of numerical discretization\", but it isn't clear how this was the case in the 3 experiments.\n\nThe reported results are interesting and promising, but there are two downsides to the experimental protocols. First, when CP is applied to a network, it increases the number of trainable parameters and the effective depth. This is however not taken into account at all in the first two examples, and partially taken into account in the last example (but without any careful accounting for either parameters or depth). Arguably, a fair baseline for the CP models should have comparable depth and parameter count, and access to the same input information (e.g. DDP in the closure problem doesn't get 'q' as an input).\n\nThe second downside is the lack of comparison against externally reported baselines. In the Burgers' equation one could compare against e.g. https://doi.org/10.1073/pnas.1814058116. The MeshGraphNets paper covers a number of nontrivial cases -- given the focus on fluid flow in the present work, the airfoil or cylinder flow examples would be natural settings to test before introducing the more complex combustion case.\n\nAll in all, the paper presents interesting results but the reported experimental settings make it harder than necessary to understand the real impact of CP.\n\n----\nScore raised +1 based on the promise to improve baselines in the revised version of the paper.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors do not discuss potential negative impact due to perceived low risk, which seems fine for this type of work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}]

Author rebuttal:

Meta-review: 3 of 4 ratings were "accept". A key concern shared by several reviewers was about comprehensive experimental baselines, and the authors responded by performing requested experiments which supported their results. Because the one reviewer who gave the lowest rating (a 5) suggested this experiment, the authors performed it and it was favorable, and the reviewer didn't respond, I'm inclined to treat the reviewer's rating as a 6.
Decision: Accept (Poster)