P$^2$C$^2$Net: PDE-Preserved Coarse Correction Network for efficient prediction of spatiotemporal dynamics
Authors: Qi Wang
Pu Ren
Hao Zhou
Xin-Yang Liu
Zhiwen Deng
Yi Zhang
Ruizhi Chengze
Hongsheng Liu
Zidong Wang
Jian-Xun Wang
Ji-Rong Wen
Hao Sun
Yang Liu
Venue: NeurIPS 2024 poster (2024)
Keywords: physics-informed learning
coarse model
spatiotemporal dynamics prediction
Primary area: machine_learning_for_physical_sciences
TL;DR: Introduced a PDE-Preserved Coarse Correction Network for efficient prediction of spatiotemporal dynamics.

Abstract: When solving partial differential equations (PDEs), classical numerical methods often require fine mesh grids and small time stepping to meet stability, consistency, and convergence conditions, leading to high computational cost. Recently, machine learning has been increasingly utilized to solve PDE problems, but they often encounter challenges related to interpretability, generalizability, and strong dependency on rich labeled data. Hence, we introduce a new PDE-Preserved Coarse Correction Network (P$^2$C$^2$Net) to efficiently solve spatiotemporal PDE problems on coarse mesh grids in small data regimes. The model consists of two synergistic modules: (1) a trainable PDE block that learns to update the coarse solution (i.e., the system state), based on a high-order numerical scheme with boundary condition encoding, and (2) a neural network block that consistently corrects the solution on the fly. In particular, we propose a learnable symmetric Conv filter, with weights shared over the entire model, to accurately estimate the spatial derivatives of PDE based on the neural-corrected system state. The resulting physics-encoded model is capable of handling limited training data (e.g., 3--5 trajectories) and accelerates the prediction of PDE solutions on coarse spatiotemporal grids while maintaining a high accuracy. P$^2$C$^2$Net achieves consistent state-of-the-art performance with over 50\% gain (e.g., in terms of relative prediction error) across four datasets covering complex reaction-diffusion processes and turbulent flows.

Net: PDE-Preserved Coarse Correction Network for Efficient Prediction of Spatiotemporal Dynamics
Qi Wang', Pu Ren", Hao Zhou', Man- Xin-Yang Liu", Zhiwen Deng", Yi Zhang', Ruizhi Chengze', Hongsheng Liu", Zidong Wang", Jian--un Wang", Ji-Rong Wen', Hao Sun Yang Liu
Gaoling School of Artificial Intelligence, Renmin University of China. Beijing, China DDeparmment of Civil and Environmental Engineering, Northeastern University, Boston, MA, USA
Department of Aerospace and Mechanical Engineering, University of Notre Dame, Notre Dame, IN, USA "Huawei Technologics, Shenzhen, China
Schooo of Engineering Science, University of Chinese Academy of Sciences, Beijing, China
Emails: qi www.aannru.ceeee (Q.W.; haosunrruc. edu wnw..nnnc......ecc..o. (H.S.); liuyang22Qucas ac. cn (Y.I.))
Abstract
When solving partial differential equations (PDEs), classical numerical methods often require fine mesh grids and small time stepping to meet stability, consis- tency, and convergence conditions, leading to high computational cost. Recently, machine learning has been increasingly utilized to solve PDE problems, but they often encounter challenges related to interpretability. generalizability. and strong dependency on rich labeled data. Hence, introduce new PDE- Preserree Coarse Correction Network (P2C2Net) to efficiently solve spatiotemporal PDE problems coarse mesh grids in small data regimes. The model consists of two synergistic modules: (1) a trainable PDE block that learns to update the coarse solution (i.e., the system state), based on a high-order numerical scheme with boundary condition encoding, and (2) a neural network block that consistently corrects the solution on propose learnable symmetric Conv filter, with weights
the fly. In particular, shared over the entire model, to accurately estimate the spatial derivatives of PDE based on the neural-correeted system state. The resulting physics-encoded model is capable of handling limited training data (c.g., 3-5 trajectories) and accelerates the prediction of PDE solutions on coarse spatiotemporal grids while maintaining a high accuracy. P²C*Net achieves consistent sfate-of-the-e art performance with over 50% gain (e-g., in terms of relative prediction error) across four datasets covering complex praction-diffssie processes and turbulent flows.
Introduction
Complex spatiotemporal dynamical systems are pivotal and commonly seen in numerous fields such biology, meteorology, fluid mechanics, etc. The behaviors of these systems are primarily governed by partial differential equations (PDEs), conventionally solved by numerical methods 4 However, direct numerical simulation (DNS) demands in-depth knowledge of the underlying physics, and the efficacy of numerical solutions is intricately linked to the resolution of mesh grids and time steps. High-resolution spatiotemporal grids are essential for accurate and convergent calculations, yet leading to substantial computational costs. For instance, simulating the flow field around large aircraft [5] 6 involves creating over millions of mesh nodes and may consume vast simulation time even on high performance computing. Additionally, any changes in initial and boundary conditions (I/BCs) or design parameters necessitate recalculations, further compounding the complexity.
"Correpoondigg authors
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Recently, tremendous efforts have been placed on machine learning for outa-driven simulation of these systems 17.9, demonstrating promising potential. These methods do not require the a priori knowledge of physics and, meanwhile, help bypass some traditional constraints, e.g., the smallest size of mesh grid and time step to guarantee solution accuracy, stability and convergence [10]. However, they typically face issues of poor interpretability, weak generalizability and strong dependency of rich labeled data. Their performance deteriorate significantly particularly in small data regimes.
Embedding prior physics knowledge into the learning process has demonstrated effective to overcome the aforementioned issues. A brute-force way lies in creating regularizers (e.g., the residual form of PDEs and I/BCs) as "soft" penalty in the loss function, e.g., the family of physics-informet informed neural works (PINNs) [1TI However, such a strategy has limited scalability and generalizability. and the solution accuracy relies largely on a proper selection of loss weight hyperparameters. Embedding physics explicitly into the network architecture, which imposes "hard" constraints such as physics- encoded recurrent convolutional neural network (PeRCNN) [8 19). possesses better generalizability as well as offers better convergence and flexibility for model training without the need of fine-tuning hyperparameters. Nevertheless, existing methods fail to handle coarse mesh grids and suffer from instability issues especially for long-range prediction of dynamics. Hybridizing classical numerical schemes and neural networks, e.g., the learned interpolation (LI) model 220], can enable accelerated simulation on coarse mesh grids with satisfied accuracy. Yet, since the numerical part is non- con-trainable, such models still require rich labeled data to retain accuracy.
To tackle these critical challenges, we introduce the Pe model for efficient prediction of spatiotemporal dynamics on coarse mesh grids in small training data regimes. Specifically, a trainable PDE block (a white box) designed to learn the coarse solution at low resolution, where the temporal marching of system states is handled by fourth-order Runge-Kurt Kutta (RK4) scheme. We also propose a learnable symmetric Conv filter for more accurate estimation of spatial derivatives on coarse grids, as required in PDE block. A neural network (NN) block, which serves as a correction module, is further introduced to correct the coarse solution, restoring information lost due to reduced resolution. We also encode BC into the solution via a padding strategy. Our primary contributions are threefold:
We propose a new physies-encoded correction leamning model (P" to efficiently predict complex spatiotemporal dynamics on coarse mesh grids. The model requires only small set of training data and possesses plausible generalizability.
We introduce a structured Conv filter that preserves symmetry to improve the estimation accuracy of coarse spatial derivatives required in the solution updating process, which makes the PDE block trainable with flexibility of handling coarse grids.
Net achieves consistent state-of- of-the-art performance with at least 50% gain (e.g.. in terms of relative prediction error) across four datasets covering complex faction-diffusion (RD) processes and turbulent flows, simultaneously retaining accuracy and efficiency.
2 Related work
Numerical methods. Conventionally, PDE systems are solved by classical numerical methods such as finite difference M. finite volume (2, finite element R and spectral methods 4.. These methods often require fine mesh grids and reasonable time stepping to meet stability, consistency and convergence conditions. When dealing with large scale simulations or inverse analyses, the computational costs remain remarkably high.
Deep learning methods. Given sufficient labeled training data, deep learning has been recently applied to solve PDE problems. Representative approaches include Conv-based NN models 99 (1). U-Net (22), ResNet (23], graph neural networks [24. 25) and Transformer-based models (26 In
addition, neural operators such as DeepONet 77]. multiwavelet-based model (MWT) [29] Fourier
neural operator (FNO) [8] and their variants 30 have been designed to directly learn mappings
between function spaces, making them particularly well-suited for modeling PDE systems. Diffusion
models have also been employed for prediction of spatiotemporal dynamics B3
Physics-aware learning methods. Recently, physics-aware deep learning has demonstrated great potential in modeling spatiotemporal dynamics under conditions of small training data. This paradigm can be divided into two categories based on the way of embedding PDE information: (1) physics- informed, and (2) physics-encoded. The former formalizes PDE soft constraints including equations

---

and I/BCs via loss regularization on point-wise or mesh-based NNs (e.g. PINN 13 Phy- GeoNet B4, PhyCRNet [35]. PhySR [6],, etc.), while the latter imposes hard constraints via encoding PDE structures (e.g.. equations, I/BCs, law of thermodynamics, symmetry) into NN archi- tectures such as PeRCNN [8 19]. TiGNN [7 and EquNN [38. Other related works include the
PDE-Net models [3) 40 with designed convolution kernels approximating differential operators,
thereby modeling the dynamics of the system.
Hybrid learning methods. The hybrid learning scheme represents novel research direction that has emerged in recent years, which integrates numerical methods with NNs. The resulting solver can leverage variety classical numerical methods such as finite difference (e PPNN
numerical discretization learning (42).. finite volume (e.g.. LI (0) and TSM [43) and spectral methods (e.g.. muchine-learning learning-augmeetter spectral solver [44]). These approaches can operate on coarse grids, enabling faster simulations compared with traditional numerical solvers while retaining accuracy. However, since the numerical part is non-trainable, such models generally require rich labeled data.
3 Methodology
3.1 Problem formulation
Let us consider spatiotemporal dynamical system governed by the general form of PDEs:
tu F(u, u² V u
u, 14)
Øt
where u(x, f) E R denotes the nz-iimensional physical state within spatiotemporal domain Q [0, T]; 0u/00 0t the first-order time derivative; F a linear/nonlinear function; V E IR" the Nabla operator; the PDE parameters (e.g., the Reynolds number Re); the external force (e.g.. f 0 for source-free cases). Besides that, we define the initial condition (IC) as Z(u, u E Ω 0) = 0 and the :..::X x a?) O, where an denotes the boundary of Ω
boundary condition (BC) as B(u Vu....::
Our aim is to develop a learnable coarse model that accelerates the simulation and prediction of spatiotemporal dynamics based on minimal set of sparse data (e.g., low-resolution data down- sampled across space and time). The learned model is expected to achieve high solution accuracy and superior generalizability over various PDE scenarios, including ICs, force terms, and PDE parameters.
3.2 Network architecture
Herein, we introduce the PeC architecture, as shown in Figure taking the simulation of Navier-Stokes (NS) flows as an example. The model is composed of four blocks, namely, the state variable correction block, the learnable PDE block, the Poisson block, and the NN block. Note that the network architecture is flexible and features Poisson block that solves for the pressure term p. which is absent in other cases.
3.2.1
The flow of data
In Figure (a). the network architecture includes two paths: the upper path computes the coarse solution using learnable PDE block, while the lower path is incorporated into the network correct the solution on a coarse grid with a Poisson block and a NN block. The data flow operates as follows: (1) the network accepts U₁ as input and processes it by the PDE block on the upper path, where the PDE block computes the residual of the governing equation. A filter bank, defined as a learnable filter with symmetry constraints, calculates the derivative terms based on the corrected solution (produced by the correction block). These terms are combined into an algebraic equation (a learnable form of FF. This process incorporated into the RK4 integrator for solution update. (2) In the lower path, u, is first corrected by the correction block, and Pı is computed by the Poisson block. Inputs, including solution states {ug: } and their derivative terms, forcing term, and Reynolds number, are fed into the NN block. The output from this block serves as correction for the upper path. (3) The final result u is obtained by combining the outputs from both the upper and lower paths. During the gradient
process, the NN block learns to correct the coarse solution output of the PDE block on the fly, and ensures that their combined results more closely approximate the ground truth solution.
3

---

?((-)
d+
Copy
Conrection
fie
Block
Poisson Biock
NN
12
Up+1
Polsson Block
By
Filter Bank
1
Correction Blook
Poisson
Biock
Filter Bank
8% d%g
Assemble (f Ea)
Filter Bank
Collect
Collect feature torms Mup, Vúi, dúú, Vp, Re
derivativns:
Polsson
Equaton
Block
Poisson Selver
Notation
Re Reynold
42
forcs
pressure field
24m
ocarse velcalt
corrected solution of velocity field fx
Tex fly
Wavenumbers the and yddrectionn
Fast Fourler Transform
learmed Nabia openator
Fast Fourter Transform
parametors
Figure 1: Schematic of Nec for learning Navier-Stoker flows. (a). Overall model architecture. (b), Poisson block. (c), learnable PDE block. (d), NN block. (e), Poisson solver. (f), Symbol notations. (g). Conv filter with symmetric constraint.
3.2.2 RK4 integration scheme
Similar to standard numerical solvers, the goal is to predict the solution at every time step satisfying the underlying PDEs given specific I/BCs. Here, we aim to address the challenge of spatiotemporal dynamics evolution coarse grids (e.g., low resolution). Given the coarse solution at timestep l.. denoted by ug, we expect the model yielding an accurate prediction of u vi.z.,
uu +
[H. (u(x, (2,,
14) + f(T)] dr (2) V (
f/.
where H (the PDE block) denotes learnable form of F. which is discussed in Section (.2.3] x
represents the coarse grid coordinates. We herein employ the RK4 scheme as the integrator for time marching of the dynamics, offering the fourth-order accuracy (O(tt)), where òt = 4441 1g is the coarse time step. More details RK4 can be found Appendix Section B.3
3.2.3 Learnable PDE Block
We assume that the PDE formulation in Eq. is given, e.g., the explicit expression of known. With coarse mesh grids and large time stepping, numerical methods such finite difference (FD) tend to diverge. This issue becomes more pronounced with greater grid coarsening. Hence, we propose learnable PDE block (depicted in Figure [c)), denoted by H. to approximate J on coarse grids, as to enable the coarse simulation simultaneously retaining efficiency and accuracy, expressed as
H (use u
Vug.
Ug :4) < [u uu,
Vúg, i :
(3)
where us denotes the coarse solution at time ty: ún the corresponding neural coarse solution state, obtained by the correction block shown in Figure [ and (c), used for estimation of spatial derivatives, e.g., ú₂ NN (u:)). Here, & represents learnable Nabla operator composed of a Conv filter with symmetric constraint (e.g., an enhanced FD kernel for numerical approximation of spatial derivatives). Through the RK4 integration, for the next time step. Note that the PDE parameters can be set as trainable if unknown. Despite information loss at the low resolution, such learnalee PDE block allows for more accurate derivative estimation on coarse grids and ensures better adherence of the updated coarse solution to underlying PDEs. Clearly, such block adds fully interpretable "white box" to the overall network architecture.
then able predict the coarse solution

---

Correction block: The correction block takes a NN to correct the coarse solution. In particular, we choose FNO 8 the neural correction operator performed on the coarse solution field in such a block. In the Fourier space, FNO decomposes the input data into components with specific frequencies, processes each component separately, and then applies Fourier transform to restore the updated spectral features back to the physical domain. The layer-wise update can be expressed as:
v=( (x) = a (W'v'xx) + (XC())v') (x))
where, vxxx) denotes the 1-th layer latent feature map on the coarse grids x. Note that (x) = P(us). in which P is local transformation function that lifts U2 higher dimensional representation. Here, (z) = iF.. (z) denotes kernel integral transformation of the latent feature map which encompasses Fourier transform, spectral filtering and convolution in the frequency domain R.., and inverse Fourier transform; Φ the network parameters; of-) the GELU activation function; W the weights of a linear transformation. Given an L-layer FNO, the corrected coarse solution reads u, 2 (X)). where 2 is local projection function. Details of the FNO correction block are found in Appendix Section B.
Conv filter with symmetric constraint: Based on FD schemes, spatial derivatives can be ap- proximated by central difference stencils represented by convolution kernels [35] 39H[41]. Such an approach often requires fine mesh grids to ensure accuracy; otherwise coarse grids, there exists solution divergence issue. To this end, we leverage our understanding of FD stencils and propose a Conv filter with symmetric constraints to improve the accuracy of spatial derivative approximation on coarse mesh grids. As shown in Figure Tgg, the filter weights are transformed into 5 x 5 matrix g with 7 learnable parameters (Table SI demonstrates the significant differences in results across various kernel sizes when applied to the Burgers dataset.), which satisfies symmetry, to estimate the first-order derivative along the horizontal direction (e.g., g., the vertical direction takes g²)), v.i.z.
Q1
a1
a7
04
a1
02
05
207
05
-2
g
03
a6
06
03
(5)
02
a5
207
as
a2
01 04
04
a)
Although the number of learnable parameters in the Conv kernel is limited, the coarse derivatives can still be accurately approximated after the model trained (see the ablation study in Section 4.2). The structured filter is designed for Conv operation which satisfies the Order of Sum Rules stated in Lemma [39 (see Appendix Section A for more details).
Lemma I: A 2D filter gmx mm has sum rules of order (41, where E Z2, provided that
Σ
h 9 k2] =0
(6)
for all a = ((1102) E 22 with |a| := a1 1 for all 22, with |a| |e| but # If this holds for all E z² with lal < A except for + à with certain à E 23, and |o| B A. then we say g to have total sum rules of order A 1B.
Corollary 1: The filter we designed in Eq. (5) has the sum rules of order (1, 0). By adjusting the parameters g. e.g., a7 - O, 06 + 8a, + O, satisfies the total sum rules of order 5 (2), and achieves an approximation to the first-order derivative with the fourth-order accuracy. For example, for a smooth function w(x, u) and small perturbation > O, we have (3)::
glki, k2]w(( + ski, + ek2) C @wlzz. v) + +O(E), as 0. (7)
k
The proof of Corollary can be found in Appendix Section A Given that the order of sum rules closely related to the order of vanishing moments in the wavelet theory, Lemma 1 ensures that the filter not only maintains high sensitivity to local changes in the feature but also effectively suppresses irrelevant low-frequency components, thereby enhancing the estimation accuracy of the first derivative 39 Furthermore, Corollary 1 further guarantees that the designed filter can approximate the first-order derivative with up to the fourth-order accuracy by properly adjusting the trainable parameters.
5

---

BC encoding: To ensure that the predicted solution com- plies with the given BCs, while also retaining the shape of the feature map after Conv operations, we employ BC hard encoding method [19]. In particular, we consider periodic BCs in this work and apply padding padding, as shown in Figure 2 on the predicted solutions. Such an encoding technique not only ensures the compliance of the predicted solution with the BCs, but also enhances the solution's 'ss accuracy.
82
84
85
86
57
85
80
90
92
90
94
95
96
97
99
100
12
13
14
15
16
17
16
19
20
3.2.4 Poisson block
81
U2
853
84
805
06
57
88
09
90
95
100
When solving NS equations, there exists the pressure term p in the governing PDEs. However, for incompressible flows, the pressure can be inferred by solving a Poisson equation. We designed this block to achieve this aim, the poisson equation namely, Ap = vu. f), where viu (u, f)
2
5
7
10
15
Filter g
Padding Nodes
Nodes
Figure 2: Periodic BC padding.
-2 (1 "xr V f for 2D problems and the subscripts denote spatial differentiation. Hence, we leverage the spectral method to estimate the pressure quantity (see Appendix Section Section(B.2). as depicted in Figure T(e). which updates p7c based (u,, fz). The resulting Poisson block (Figure Ib) efficiently derives the pressure field from the velocity field and external force on the fly, streamlining computations without the need for labeled training data of pressure.
3.2.5 NN block
is noted that the predictions by the learnable PDE block on coarse grids may encounter instability issue due to егror accumulation over time, especially in scenarios involving long-range rollout prediction, Hence, we propose an additional NN block to consistently correct the coarse solution predicted by the PDE block on the fly, restoring information lost due to reduced resolution. This module can be any trainable NN model, such as FNO [8) DenseCNN [41,, or UNet 45), In particular, we consider FNO the NN block, with more details found in Appendix Section B. However, the NN block is not always necessary. For cases simpler systems, such as the Burgers equation, the learnable PDE block alone can perform well where the NN block can be omitted.
3.2.6 Model generalization
We herein introduce the setup of the model's generalizability over various PDE scenarios, including ICs, force terms, and PDE parameters (e.g., the Reynolds number Re). The time marching in our network design inherently integrates ICs, automatically ensuring generalization over ICs given well trained model. We embed the force term in the learnable PDE Block (naturally in the PDE formulation) shown Figure [cc,, and meanwhile incorporate it as feature map into the NN Block as part of the input (see Figure This enables the joint learning of force feature variations for generalization. In addition, the Reynolds number (Re) is incorporated by creating 2D feature map embedding (e.g.. for the 2D case) by introducing two trainable vectors and b, namely, Reemtb = 1/Re Re (a & b), in both the PDE Block and the the NN Block. Such an approach can correct the error propagation of the diffusion term in the PDE Block caused by coarse grids, and enhance the model's 's ability of generalization across different Re's.
Experiments
We evaluate the performance of Nec against several baseline models across diverse PDE systems, including fluid flows and RD processes. The results have demonstrated the supe- riority of our model in terms of solution accuracy and generalizability thanks to the unique design of embedding PDEs into the network. The source codes and data are found at httpps///giihbb..ccciiteeeleeeee sccppttttee-ciiiooooooo////eeeee
(PyTorch) and hnttps://git com/ intell schellssci--i wwwput/ git (MindSpore).

---

Table I: Summary of datasets and training implementations. Note that " denotes the downsampling process from original resolution (simulation) to the low resolution (training).
Dataset
Numerical Method
Spatial Grid
Time Steps (Temporal Grid)
# of Training Trajectories
# of Testing Trajectories
Rollout Steps
Burgers
FD
100 + 25
400
5
10
20
FN
FD
128 + 642
5500 + 1375
3
10
32
GS
FD
128 - 32
4000 - 1000
3
10
50
NS
FV
2018 -> 641
153600 -> 1800
Entor dizeibutioo
Enor propagation
Ret
PCchee
FNO
Uhet
PORONN
DespONet
0.35
10
20
10-1
10-1
20
PCC Met
PND
FelCC
DoppOtet
0.11
10
Uiel
Modzls
20-1
(16)
Stapphhota
disoibbtion
Eror propagation
Ritt
FND
UNct
PoRC VN
DeepOhet
101
1.08
s0f
NaN
10-
to
0.17
10
10-1
PRO
pichtse
Liiee DoccoMec
10"
PCCWee
Pelici
INTICIV
Models
10
1000
Snapshots
.004
DeepDiive
12"4
4000
1.12
0.42
Ulet
10--
#lo/ei
KCN Deapolee
Modbels
10"2
10
Srepalatt
0.12
cl
1.0
301
ptopagation
Fiwt
PARCAN
0.00
30/4
10m2
PCleee
5.00
INO
Uset
10"
10 0.0
3.5
7.0
Snapchots at
180
Figure An overview of the comparison between our N'C baseline models, including error distributions (left), error propagation curves (middle), and predicted solutions (right). (a)-(d) show the qualitative results on Burgers, GS, FN, and NS equations, respectively. These PDE systems are trained with grid sizes of 25 25, 32x32, 64x64, and 64 accordingly.
4.1 Setup
Datasets. We consider four PDE datasets, including 2D Burgers, FitzHugh-Nagumo (FN), Gray- Scott (GS), and NS equations. Each dataset exhibits intricate spatiotemporal patterns, presenting significant challenges for prediction coarse grids. High-order FD and finite volume (FV) methods are utilized to generate these datasets, We segment the training samples based on different rollout lengths. Additionally, the simulated high-resolutin data E IR XnxHXW is down-sampled across space and time to low-resolution counterparts û R=XXnxH'xXV xW" where n' < n4, H' < H. W" < W. The low-resolution data serves as labels for training. Moreover, for each PDE case, only 3-5 sets of data trajectories are used for training while 10 trajectories are applied to test the model performance. The summary datasets and training configuration shown in Table I More details regarding datasets and their simulations can be found in the Appendix (see Table and Section
7

---

Evaluation metrics. Four metrics are used to assess the model's performance: Root Mean Square Error (RMSE), Mean Absolute Error (MAE), Mean Normalized Absolute Difference (MNAD), and High Correction Time (HCT). The definition of each metric is listed in Appendix Section D
Model training. Given the low-resolution training data, we aim to learn the evolution of spatiotem- poral dynamics on coarse grids with bigger time stepping. This learning task is formulated as an auto-regressive rollout problem, where the models are only constrained by low-resolution data loss. B j==1 MSF St) where Si denotes the Σ
The loss function is defined as [(0) BN
rollout-predicted coarse solution for the j-th sample in the i-th batch, Siy the corresponding label, B and N the number of batches and the batch size, and 0 the trainable parameters. The detailed network parameters and training configurations are provided in Appendix Section
Baseline models. To validate the superiority of the proposed Net model, we conducted comparisons with various baseline models, including FNO [8. UNet (45), PeRCNN [19], DeepONet 77. and Learned Interpolation (LI) [20]. The descriptions and training setup of the baseline models are provided in Appendix Section
Table 2: Quantitative results of our model and baselines. For the
case of Burgers, GS, and FN, our model inferred the test set's 's upper time limits of 1.4 2000 and 10 respectively, as the
4.2 Main results
trajectories dynamics get stabilized. We take these limits in
Figure 3 presents the results of compar- ison between Net and baselines, in- cluding error distribution, error propaga- tion, and predicted trajectories. More- over, Table 2 provides the quantitative model performance results.
HCT to facilitate evaluation metrics calculation.
Case
Model
RMSE
MAE
MNAD
HCT (s)
FNO
0.0980
0.0762
0.062
0.3000
0.3316
0,2942
0.2556
0.0990
DeepONet
0.2522
0.2106
0.1692
0.0020
Burgers
PeRCNN
0.0967
0.1828
0.1875
0.4492
Burgers Equation. Generally, our NeC and baseline models (PeRCNN and FNO) all capture the evolving dy-
Pe (Ours)
0.0064
0.0046
0.0037
1.4000
Promotion (1)
93.4%
94.0%
94.0%
211.7%
FNO
NaN
NaN
NaN
354
UNet
NaN
NaN
NaN
4
namics and provide acceptable results as shown in the right part of Figure a). However, our method shows a notably lower error level compared with base- lines, as emphasized in the error distri- bution and error propagation presented in the left and middle parts of Fig- ure Ba). Moreover, Jablaaasusttantiates this observation, revealing a significant improvement in our model's s performance compared with the bascline models from a minimum of 93,4% to a maximum of 211.7%. We further conducted boundary condition generalization tests, as detailed in Appendix Section G.1
GS
DeepONet
0.3921
0,2670
0,2670
852
PeRCNN
0.1586
0.0977
0.0976
954
PoC (Ours)
0.0135
0.0062
0.0062
2000.0
Promotion (t)
91.5%
93.7%
93.6%
109.6%
FNO
0.8935
0,5447
0.2593
3,5000
UNet
0.1730
0,0988
0.0470
6,5000
DeepONet
0.5474
0.3737
0.1779
0.5128
PeRCNN
0.5703
0,2258
0.1075
5,3750
NeC (Ours)
0.0390
0.0149
0.0071
10.000
Promotion (1)
77.5%
84.9%
84.9%
53.8%
FNO
1.0100
0,7319
0.0887
2.5749
UNet
0.8224
0.5209
0.0627
3,9627
1II PeRCNN
NaN
NaN
NaN
3,5000
NS
1.2654
0.9787
0.1
0.6030
PCC (Ours)
0.3533
0.1993
0.0235
7.1969
Promotion (1)
57.0%
61.7%
62.5%
81.6%
GS Equation. The primary challenge of this dataset lies in its sparsity and the intricate patterns it presents, as depicted in Figure 3) (right). Each baseline model struggles to accurately predict the trajectories, especially the UNet model demonstrating significant divergence. Nevertheless, our method exhibits superior stability and is capable of learning the underlying dynamics. This is further validated by the error analysis presented in Figure b)) where our P'C model outperforms the baselines by one to two orders of magnitude. Table 2 shows comprehensive summary of the performance metrics, with our model achieving an improvement of over 91% across all evaluations for the GS equation.
This RD system is another classic but challenging case due to its two-scale fast-slow evolving patterns. As illustrated in the predicted snapshots of Figure Bjc) the baseline models can capture the global patterns but struggle with the local details. Our method shows superiority in learning the multi-scale features. Moreover, the emror analysis demonstrates that our Pat achieves errors one to two orders of magnitude lower than those of the baseline models, with an improvement ranging from 53.8% to 77.5% compared to the best bascline (see Table 2).
FN Equation.
8

---

NS Equation. We evaluate our method on an NS dataset with a Reynolds number of 1000, benchmark dataset known for its significant challenges. As shown in Figure d), the snapshots produced by the baseline models at / = exhibit incorrect dynamical patterns, especially the LJ model showing divergence. The average test error of our model is at least an order of magnitude lower than those of the baselines. Furthermore, Net consistently outperforms the baselines throughout the er- propagation process. Table 2 further validates the superior performance our model, with improvements across all metrics of at least 57%. Additionally, ine the physical properties of the learned fluid dynamics, such as energy spectra. As illustrated in Figure 4 the energy spectra curve of Net closely aligns with that of the ground truth, demonstrating its effectiveness in capturing high-frequency features.
Piç/Nct
FNO
UNel
exam-
EIAW
PeRCNN Truth
10
2.8
leg
Generalization test. Taking the NS equation as ex- ample, Net is able to generalize to different external forces f and Reynolds numbers Re. Our model is trained with f = sin(4y)
0.lu and Re = 1000, where n, [L, 0]. We conduct the generalization test under six external force scenarios and four distinct Reynolds numbers Re- {200, 500, 800, 2000}. As depicted in Figure 5 Ia),, the error distribution presents stable performance with errors generally below 0.1 across six distinct external forces. Figure (]bb shows the acceptable error propagation with gradual upward trend, which aligns with the long- term rollout outcomes of our model. For the generalization test on Reynolds numbers, Figure[jcc) demonstrates satisfactory and robust error levels for all Reynolds numbers, and smaller errors are observed when the Reynolds numbers are closer to those in the training set, as validated by the error propagation curves shown in Figure d). Overall, our peC model exhibits robust generalization capabilities and stable performance across test samples for external forces, and Reynolds numbers. Further details on the snapshots are provided in Appendix Figure S3
10a
Wavenumber K
Figure 4: Energy spectra.
Robustness to noisy/incomplete data. Using the Burgers equation as an example, we introduced Gaussian noise of varying scales during the training process and observed its minimal impact on the results. The results are presented in Table which indicate that our model is robust to the training data noise and maintains the HCT (high correlation time) metric without reduction.
Moreover, for the Burgers a
Enor distribution
b
Error propagation
case, the time steps of the training data consist of only 28% of the inference steps in the test data. Namely, our training data itself is incomplete, and the miss-
10
10a
10
10a
10-1
10-2
R-RMSE
10
110
10
10-4
10-3
ing data accounts for 72% of the test set. Based on this sparse dataset, fur- ther randomly reduced the training data by 20% sim- ulate data incompleteness (e.g.. randomly deleting data according to the rollout step size to make the trajectory in- complete, where the specific size can be found in Table in the paper). The results of this experiment are shown Table [S4] where values are averaged over 10 test sets.
10-
10-
10-1
0.0
3,5
7.0
Extemal forcings
(5)
Error distribution
d
Error propagation
10a
10a
109
10a
10-1
a-RMSE
10
10--
10-2
Re 200
10-3
Re 500
10-4
Ro 200
10-3
Re 800
Re 500
2000
Re 2000
10-3
10-0.0
3.5
7.0
Reynolds numbers
t (s)
Figure 5: The error distribution and propagation of C for generalization over different external forces (a, b) and Reynolds numbers (c. d).
We can observe that, after making the data sparser, our model performance slightly decreases, which means that our model is capable of handling scenarios with incomplete data.

---

Ablation study. To assess the impact of different components on model performance, we design and conduct series of ablation experiments based on the Burgers equation. The experimental setups include: (1) Model 1, replacing symmetric convolutions with regular convolutions; (2) Model 2, using convolution kernels with finite difference stencils instead of symmetrie convolutions; (3) Model 3, removing the Correction Block; (4) Model 4, substituting RK4 integration with first-order Euler methods; and (5) the full Pet architecture. All experiments are performed using the same training data and model hyperparameters previously defined.
As shown in Table 3 the network performs poorly when the Fourier block is removed, with error levels two orders of magnitude higher com-
Table 3: Results for the ablation study of C
Ablated Model
RMSE
MAE
MNAD
HCT (s)
pared to using the complete P*C² Net architec- ture. This emphasizes the critical role of the physics-encoded variable correction learning method. Configurations using traditional finite
Model 1
0.1450
0.1100
0.0924
0,1073
Model 2
NaN
NaN
NaN
0,2013
Model 3
0.1463
0.1141
0.0977
0,1100
Model
0.0322
0.0241
0.0193
1.1860
Full PaC
0.0064
0.0046
0.0037
1,4000
difference stencils kernels also exhibit simi-
larly poor performance due to the large numerical errors when approximating derivatives caused by coarse grids. Moreover, removing symmetry constraints leads to significant increase in errors, emphasizing the importance of symmetric features for model convergence, especially in scenarios with limited data. Using the Euler scheme for time stepping instead of RK4 results in reduced stability and increased error accumulation, leading to a decrease in performance compared to the complete variable correction learning method
PeC Overall, the results confirm that the and convolution filters satisfying symmetry are indispensable components of the network framework.
5 Conclusion
800
Numerical Solver Pºc²Net
700
600
543
This paper presents physics-encoded variable rection learning method designed to embed prior knowledge on coarse grids for solving nonlinear dy- namic systems. This approach enables the model to focus fitting equations, ensuring excellent general- ization capability and interpretability. It introduces a convolutional filter that follows symmetry constraints, requiring only seven Icarnable parameters to adap- tively compute the derivatives of system state vari- ables corresponding to the flow field on coarse grid. This allows the model to learn spatiotemporal dynamies with limited data. Our model has been trained and tested on four different nonlinear dynamic systems, achieving the SOTA results. Even with minimal data, it can generalize to different initial conditions, external force terms, and PDE parameters. In summary, our model effectively adapts to various nonlinear dynamic systems. Moreover, the trained model shows remarkable speedup for simulation under the same condition of accuracy, shown in Figure 6
1 time
500
400
300
200
11x
30x
203
10x
100
18
20
65
FN
NS
Equations
Figure 6: Computational time for comparison.
Despite demonstrating excellent generalization ability, the model faces two challenges. Firstly, the model based on regular grids with periodic boundaries. limiting its ability to solve problems on irregular grids. We will explore graph structures or coordinate transformations to handle irregular grids and incorporate special boundary treatment methods to adapt to various boundary conditions. Secondly, we expect to expand our research from 2D problems to 3D dynamical systems.
Acknowledgement
The work is supported by the National Natural Science Foundation of China (No. 62276269 and No. 922701 18) 8), the Beijing Natural Science Foundation (No. 1232009), and the Strategic Priority Research Program of the Chinese Academy of Sciences (No. XDB0620103). In addition, H.S and Y.L.. would like to acknowledge the support from the Fundamental Research Funds for the Central Universities (No. 202230265 and No. E2EG2202X2). Q.W. acknowledges the support by the Research Program of the Institute of Interdisciplinary Sciences, Renmin University of China. P.R. would like to disclose that he was involved in this work when he was at Northeastem University, who has not been supported by Huawei Technologies.
Interdsciplinnry--nnove
10

---

Impact statement
The aim of this work is to develop a novel physics-cncodent learning scheme to accelerate predictions and simulations of spatiotemporal dynamical systems. This method can be applied to various fields, including weather forecasting, turbulent flow prediction, and other simulation tasks. Our research is solely intended for scientific purposes and poses no potential ethical risks.
References
[1] John David Anderson and John Wendt. Computational fluid dynamics, volume 206. Springer, 1995.
[2] Fadl Moukalled, Luca Mangani, Marwan Darwish, F Moukalled, L Mangani, and M Darwish. The finite volume method. Springer, 2016.
[3] Olek Zienkiewiez, Robert I. Taylor, and Jian Zhu. The finite element method: its basis and funlamentals. Elsevier, 2005.
[4] George Karniadakis and Spencer J Sherwin. Spectral/hp element methods for computational fluid dynamics. Oxford University Press, USA, 2005.
[5] Nashat Ahmad, Numerical simulation of the aircraft wake vortex flowfield. In 5th AIAA Amossheric and Space Environments Conference, page 2552, 2013.
[6] Konrad A Goe, Oriol Lehmkuhl, George Ilhwan Park, Sanjeeb Bose, and Parviz Moin. Large eddy simulation of aireraft at affordable cost: milestone in computational fluid dynamics. Flow, 1:E14, 2021.
[7] Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature Machine Intelligence, 3(3):218-22) 12-22,, 2021.
[8] Zongyi Li, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. In International Conference on Learning Representations, 2021.
[9] Kimberly Stachenfeld, Drummond B Fielding, Dmitrii Kochkov, Miles Cranmer, Tobias Pfaff, Jonathan Godwin, Can Cui, Shirley Ho, Peter Battaglia, and Alvaro Sanchez-Gonzalez. Learned coarse models for efficient turbulence simulation. International Conference on Learning Representations, 2022.
[10] Joe D Hoffman and Steven Frankel. Numerical methods for engineers and scientists. CRC press, 2018.
[11] Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics-informed informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686-707, 2019.
[12] Maziar Raissi, Alireza Yazdani, and George Em Karniadakis. Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations. Science, (67()) 1026-1000,,, 2020.
[13] Rui Wang, Karthik Kashinath, Mustafa Mustafa, Adrian Albert, and Rose Yu. Towards physics- informed deep learning for turbulent flow prediction. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1457-1466, 2020.
[14] Ruiyang Zhang, Yang Liu, and Hao Sun. Physics- informed multi-lstm networks for meta- modeling of nonlinear structures. Computer Methods Applied Mechanics and Engineering, 369:1 13226, 2020.
[15] Zhao Chen, Yang Liu, and Hao Sun. Physies-informed learning of governing equations from scarce data. Nature Communications, 12(1):6136, 2021.

---

[16] Chengping Rao, Hao Sun, and Yang Liu. Physics-informan deep learning for computational elastodynamics without labeled data. Journal of Engineering Mechanics, 147(8):04021043. 2021.
[17] Kejun Tang, Jiayu Zhai, Xiaoliang Wan, and Chao Yang. Adversarial adaptive sampling: Unify pinn and optimal transport for the approximation of pdes. In International Conference on Learning Representations, 2024.
[18] Chengping Rao, Pu Ren, Yang Liu, and Hao Sun. Discovering nonlinear pdes from scarce data with physics-encoded learning. In International Conference Learning Representations, 2022.
[19] Chengping Rao, Pu Ren, Qi Wang, Oral Buyukozturk, Hao Sun, and Yang Liu. Encoding physics to learn reaction-dins diffussion processes. Nature Machine Intelligence, 5(7):765-779, 2023.
[20] Dmitrii Kochkov, Jamie A Smith, Ayya Alieva, Qing Wang, Michael Brenner, and Stephan Hoyer. Machine learning accelerated computational fluid dynamics. Proceedings of the National Academy of Sciences, 118(21):e2101788118 2021.
-eaming-acceleratee
[21] Yohai Bar-Sinai, Stephan Hoyer, Jason Hickey, and Michael P. Brenner. Learning data-driven discretizations for partial differential equations. Proceedings of the National Academy of Sciences, 1 (11661):2015)))) 4058, 2019.
[22] Jayesh K Gupta and Johannes Brandstetter. Towards multi-spatiotemporal-sccce generalized PDE modeling. Transactions on Machine Learning Research, 2023.
[23] Yiping Lu, Aoxiao Zhong, Quanzheng Li, and Bin Dong. Beyond finite layer neural networks: Bridging deep architectures and numerical differential equations. In International Conference on Machine Learning, pages 3276-32663 3285, 2018.
[24] Alvaro Fanchez-Gonzalez Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, and Peter Battaglia. Learning to simulate complex physics with graph networks. In International Conference on Machine Learning, pages 8459-8468, 2020.
[25] Tobias Pfaff, Meire Fortunato, Alvaro Sanehez-Gonzalez, and Peter Battaglia. Learning mesh- based simulation with graph networks. International Conference on Learning Representations, 2021.
[26] Steeven Janny, Aurélien Beneteau, Madiha Nadri, Julie Digne, Nicolas Thome, and Christian Wolf. Eagle: Large-scale learning of turbulent fluid dynamics with mesh Transformers. In International Conference Learning Representations, 2023.
[27] Zijie Li, Dule Shu, and Amir Barati Farimani. Scalable transformer for pde surrogate modeling. Advances in Neural Information Processing Systems, 36, 2024.
(28) Haixu Wu, Huakun Luo, Haowen Wang, Jianmin Wang, and Mingsheng Long. Transolver: A fast transformer solver for pdes on general geometries. arXiv preprint arXiv:2402.02366. 2024.
[29] Gaurav Gupta, Xiongye Xiao, and Paul Bogdan. Multiwavelet-basee operator learning for differential equations. Advances in Neural Information Processing Systems, 14:24048-24062. 2021.
[30] Alasdair Tran, Alexander Mathews, Lexing Xie, and Cheng Soon Ong. Factorized fourier neural operators. In The Eleventh International Conference on Learning Representations, 2022.
[31] Zongyi Li, Nikola Kovachki, Chris Choy, Boyi Li, Jean Kossaifi, Shourya Otta, Moham- mad Amin Nabian, Maximilian Stadler, Christian Hundt, Kamyar Azizzadenesheli, et al. Geometry-infomme neural operator for large-scale 3D PDEs. Advances in Neural Information Processing Systems, 36, 2024.
[32] Somdatta Goswami, Katiana Kontolati, Michael D Shields, and George Em Karniadakis. Deep transfer operator learning for partial differential equations under conditional shift. Nature Machine Intelligence, 4(12):1155-1166 2022.
12

---

[33] Phillip Lippe, Bas Veeling, Paris Perdikaris, Richard Turner, and Johannes Brandstetter. Pde- refiner; Achieving accurate long rollouts with neural pde solvers, Advances in Neural Informa- lion Processing Systems, 36, 2024.
[34] Han Gao, Luning Sun, and Jin Wang. PhyGeoNet: Physics-informed geometry-adaptive convolutional neural networks for solving parameterized steady-state pdes on irregular domain. Journal of Computational Physics, 428: 110079, 2021.
[35] Pu Ren, Chengping Rao, Yang Liu, Jian-Xun Xun Wang, and Hao Sun. PhyCRNet: Physics-infommn convolutional-recureee network for solving spatiotemporal pdes. Computer Methods in Applied Mechanics and Engineering, 389:1 14399, 2022.
[36] Pu Ren, Chengping Rao, Yang Liu, Zihan Ma, Qi Wang, Jian-Xun Wang, and Hao Sun. Physr: Physics-informed deep super-resolution for spatiotemporal data. Journal of Computational Physics, 492:1 12433, 2023.
[37] Quercus Hernández, Alberto Badías, Francisco Chinesta, and Elías Cueto. Thermodynamics- informed neural networks for physically realistic mixed reality. Computer Methods in Applied Mechanics and Engineering, 407:1 15912, 2023.
[38] Rui Wang, Robin Walters, and Rose Yu. Incorporating symmetry into deep dynamics models for improved generalization. In International Conference Learning Representations, 2021.
[39] Zichao Long, Yiping Lu, Xianzhong Ma, and Bin Dong. Pde-net: Learning pdes from data. In International conference on machine learning, pages 3208-3216, 2018.
[40] Zichao Long, Yiping Lu, and Bin Dong. Pde-net 2.0: Learning pdes from data with a numeric- symbolic hybrid deep network. Journal of Computational Physics, 399:108925, 2019.
[41] Xin- Kin-YangTa Liu, Min Zhu, Lu Lu, Hao Sun, and Jian-Xun Wang. Multi-resolution partial differ- ential equations preserved learning framework for spatiotemporal dynamics. Communications Physics, 7(1):11, 2024.
[42] Jiawei Zhuang, Dmitrii Kochkov, Yohai Bar-Sinai, Michael P Brenner, and Stephan Hoyer. Learned discretizations for passive scalar advection in two-dimensional turbulent flow. Physi- cal Review Fluids, 6(6):044005, 2021.
[43] Zhiqing Sun, Yiming Yang, and Shinjae Yoo. A neural PDE solver with temporal stencil modeling. In International Conference on Machine Learning, pages 33135-33155, 2023.
[44] Gideon Dresdner, Dmitrii Kochkov, Peter Christian Norgaard, Leonardo Zepeda- Jamie Smith, Michael Brenner, and Stephan Hoyer. Learning to correct spectral methods for simulating turbulent flows. Transactions on Machine Learning Research, 2023.
[45] Jayesh K Gupta and Johannes Brandstetter. Towards multi-spaatooteppraalsssee generalized pde modeling. arXiv preprint arXiv:2209.156// 2022.
[46] Johannes Brandstetter, Rianne van den Berg, Max Welling, and Jayesh K Gupta. Clifford Neural Layers for PDE Modeling. In The Eleventh International Conference on Learning Representations, 2022.
[47] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assister intervention-MICE MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pages 234 241. Springer, 2015.
13

---

APPENDIX
A Design of the filter with symmetric constraint
Firstly, inspired by the central finite difference method, we design a filter g with the size of m x m, which meets the symmetric constraint. For example, when 5, g given by
d1
04
d7
04
02
05
0g
Q5
02
g
d3
06
0
Q6
03
(S1)
02
05
ag
0ş
a2
d1
04
(17
04
01
Taking a (1, we have:
Σ
ki h2 g k2] = -407 + 2ag.
(S2)
In order satisfy the sum rules of order re-wit re-written as
(1, 0), we can get ag -207. Then, the filter g can be
a1
04
004
01
02
05
-007
05
02
g
06
(S3)
022
05
207
As
02
017
@1
Corollary 1 Proof:
Consider the filter we designed above with a = (0,1), (0, 3) and
(3, 0). have:
Σ
kikgglki = -4a5 2006
(S4)
k
1 ho [k,, K2] = =116a5 -2a
(S5)
kj
kg
kgkgglhn [k1 kzg -1207
(S6)
lgg
Satisfying the sum rules of order (0, 3) and (3, 0) leads to strictly - -16a, 20g
0, -12a7 = O. By adjusting the trainable parameters, we have ag + 8a; + 0, a7 - 0, and
-4as 20g O. According to Lemmal, the resulting filter g satisfies the total sum rules of order 5 (2]
For any smooth function w, we perform a convolution operation on it with the filter Applying Taylor expansion this process, we obtain:
2
gik,, 12)w(a + kr y + k2ōy)
ki A12==2
Satay' gi+sw
Σ
glkn Kz]
dlj! Or (a, v) + o(18x + 18yl
(S7)
In 2
1-==0
gits
rizdx' "y
Or (z, v) + o(10x)2 + (8ylt)
====0
14

---

Fourier layer I
Fourier layer l/
Fourier layer L
(xx)
W
Figure S1: The architecture of FNO Model
Then, if the filter g has total sum rules of order 5 \ {2}, for a smooth function w(x, y) and small perturbation O, we have:
gikı, k2/w(x Eki,y y + ck2) = Ow(ar,)
Σ
Σ
+ O(E)), as -
(S8)
8
Ōr
If there is no such setting, for de + 8a5 - 0, a7 O, through continuous training, we iteratively optimize the network and adjust the values parameter a1 to a7, such that:
gitiw
Tazdx'dd dy
Ox'0y' (x,y) Vi,j\ {i = 11j1,j- 1,j= = 0}
(S9)
In this case, Eq. (S8) still holds. Hence, by employing a filter g with enhanced symmetry constraints and through the learning process of the network, ensure that fourth-order accuracy is preserved.
B
Implementation Details
B.1 FNO Model
As illustrated in Figure S1 the network structure primarily consists of three key components: P (lift
operation), Q (projection operation), and Fourier layers. Both P and are convolutional operations designed for channel transformation. Every Fourier layer employs the Fast Fourier Transform (FFT) and the inverse Fast Fourier Transform (iFFT) for frequency domain transformations. Additionally, R, represents spectral filtering and convolution in the frequency domain, and W' denotes the local linear transformation specific to the 1-th layer. ar the activation function GeLU.
Our FNO model omits normalization schemes due to their detrimental impact on performance [46]. When used in the Correction Block, we set 2. For the Burgers equation, we configure modes 12, width = 12, and the projection operation from channel 12 to channel 50. Both the FHN and GS equations share the same configuration: modes 12, width 20, with the projection operation from channel 20 to channel 50. The NS equation requires distinct configuration: modes = 25 width 20, and the projection operation from channel 20 channel 128. particular, when it appears as module of Neural Network, the hyperparameters are set differently: L = 5, modes = grid/2 2 2, and width to be the same as modes.
B.2 Poisson Solver
The objective of the Poisson solver is to determine the state quantities of system within two- dimensional spatial domain using the spectral method for a specified Laplacian term. The Laplace equation in two dimensions is represented as:
Ap (u, f).
(S10)
15

---

X
Add
Equatin Residux Block
Wbe1
At
Equation
82
1 bt
3
Add
8t
68
6
Resictul Block
X
Equitian Raskuul Block
$4
had
Figure S2: RK4 integration scheme
where (i, -2 (uy Uy + (FFT) on Eq. (S10). we have
for 2D problems. Applying Fast Fourier Transform
n n p' (u, f),
(S11)
where 7h and My are the wavenumbers along the x and y axes, respectively, assuming 7i + ny * to avoid division by zero. In the frequency domain,
get
66 (u,
p
(S12)
(12 + n
Next, we transform the field from the frequency domain to the spatial using inverse Fast Fourier Transform (iFFT):
1 (u. (n² +
iFFT [p"]
iFFT
(S13)
By applying the above process to (u, f), we can efficiently decouple the pressure field without any labeled data or training process.
B.3 RK4 integration scheme
The general numerical integration method for time marching from u., to U can be written as:
Uk+1
U1g
(x,
(S14)
Among them, Up-1 and ug are solutions at time k and k. As shown in Figure S2 RK4 is high-order integration scheme, which divides the time interval into multiple equally spaced small time steps to approximate the integral. The final updating of the above state change can be written as:
81 H [u, 4x,
81
82 H
u
x S1, t +
2
2
8t
83 H
u
x S2 tx +
(S15)
2
2
84 H
[n
x 83, 4 + 56).
u
U
8t
28 + 23g s1).
where št denotes the step size and $1 S2, S3, $1 represent four intermediate variables (slopes). The global error proportional the step size to the fourth power, i.c. (8t4).
C Baseline models
To evaluate the performance of our proposed method, we compare it against multiple -taee-of-the-art (SOTA) baseline models. The detailed descriptions each model are presented below, and the training configurations are listed in Section
16

---

Fourier Neural Operator (FNO). The FNO [ combines neural networks with Fourier transforms and consists of two major parts. The first part primarily involves performing Fourier transforms on system state quantities, convolving in the frequency domain, and then inversely transforming to extract global features. The second part includes linear transformations through convolutions on the system state quantities to extract local features. Finally, activation functions are applied, and the outputs from both parts are combined to produce the final result.
employs an encoder-decoder structure, where the encoder extracts multi-scale features through multiple downsampling operations and the decoder recovers the original image size through multiple upsampling steps. Skip connections are utilized during the decoding process to fuse feature maps from corresponding layers, preserving both local details and capturing global features.
UNet. UNet 47
PeRCNN. PeRCNN 19 is a physics-encoded learning approach with physical laws embedded into the neural networks. It comprises multiple parallel CNNs, leveraging the simulation of polynomial equations through feature map multiplication. This approach enhances the model's capacity for extrapolation and generalization.
DeepONet. The aim of DeepONet 7 is to use neural networks to learn end-to-end mapping from input data to target output by approximating operators. It includes a trunk net and branch net, which enables effectively capturing intricate functional relationships.
Learned Interpolation (LI), LI 20 is based on finite volume method scheme. It leverages neural networks to replace the traditional numerical approach that uses polynomial interpolation for the velocity tensor product. This network learns how to dynamically adjust the interpolation function based on the characteristies of the local flow field. Therefore, the LI method facilitates the predictions of dynamics on coarser grids.
D Evaluation Metrics
In this paper, we several metrics to evaluate our model, including RMSE, Mean Absolute Error (MAE), Mean Normalized Absolute Difference (MNAD), and High Correction Time (HCT). RMSE measures the average magnitude of the error between predicted and observed values, reflecting the model's accuracy. MAE evaluates the average absolute difference between predicted and observed values, indicating the actual magnitude of the errors. MNAD is a crucial metric for assessing the accuracy of model outputs over time. MNAD calculates the average discrepancy across a series of temporal data points, offering a normalized measure of prediction error relative to the range of the actual data. HCT quantifies the model's i ability to make accurate long-term predictions. These metrics are defined as follows:
1
RMSE
S
S
(S16)
i=1
MAE
1
Σ
S: 5
(S17)
i=1
S. S
MNAD =
(S18)
|
| mmn
max
N
HCT
At (PCCC(S, > 0.8),
(S19)
where
PCC (O.(S.,
(S20)
Sg TS
Here denotes the number of trajectories; S, the ground truth of trajectories; S, the spatiotemporal sequence predicted by the model. "cov" is the covariance function and "o" is the standard deviation of the given sequence. 1(.) is the indicator function that takes a value of 1 when the condition (PCC(S,,S) > 0.8) is true, and 0 otherwise. N is the total number of time steps.
17

---

E Dataset Informations
To facilitate a comprehensive evaluation of our proposed method, we employ datasets generated from various well-known equations that model complex systems. These datasets cover a wide range of applications from fluid dynamics to reaction-diffusion systems, ensuring a robust assessment of model performance.
For faireess of testing datasets, randomly select 10 seeds from the range of [1, 10,000], which are used to generate different Gaussian noise disturbances, which are then applied to the initial velocity field to create varying ICs. We also perform warm-up phase during data generation ensure that the variance and mean of the trajectories are closely aligned, thereby maintaining fairness. More details about dataset generation can be found in Table S2
Burgers. This equation describes the velocity flow a viscous fluid, which combines compressibility and nonlinear effects. It has wide applications in many scientific fields, including weather and climate simulation, the petroleum industry, acoustics, and quantum field theory. The equation is given by:
Ōu
- Vu, Ōt
1 E [0.
(S21)
where u = {u, oj E R² represents the fluid velocities, v is the viscosity coefficient set to 0.002, and V is the Laplacian operator.
To generate the dataset, use the FD method with periodic boundary conditions the spatial domain x E [0, 1]. The data is initially simulated on a 1002 grid and then downsampled to 25 for numerical experiments. The simulation timestep is set to 1 10 3 seconds and the time duration is T= 1.4 s. We use five trajectories for training, each of which contains 400 snapshots. For testing, utilize ten groups of trajectories, each with 1400 snapshots.
GS. The GS equation models the interaction between two chemical species and has been widely used to simulate various chemical reactions under different conditions. It helps to explore dynamic phenomena such as self-organized patterns, chemical waves, reaction-diffusicn patterns, collective behavior, natural pattern formation, cell growth, and other areas in physics. The GS equation is described as:
Ou Øt
Da V
uo + F(1 u),
(S22)
du
D, 2 0 uu² (F K));
Ot
where u and v represent the concentrations of two different chemical substances, D,, and D, are the diffusion coefficients, and F and x denote the growth and death rates of the substances. respectively. In our case, we set D. = 2.0 x 10-5, D = 5.0 x 10-6, 00-6, F = 0.04, and = 0.06. We create the dataset using the FD method on a 128 grid under periodic boundary conditions, with the spatial domain x E [0, 1)2. The simulation timestep is 0.5 and the total simulation duration is 1900 The data is then downsampled to a 322 grid and the timestep coarsened to s to establish the ground truth. We leverage three training trajectories, each containing 1000 snapshots. Additionally, employ ten separate testing sets with diffenent ICs.
FN. This case is widely recognized reaction-diffusion system and is used to simulate the propagation of neural impulses. The governing equations are given by:
du Øt
v M(u), I LE [0
(S23)
where u = {u, b} E I22 are the two interactive components, and is the diffusion coefficient. The reaction source terms M(u) are defined as:
My(u,v) M,(u, v) = 3(u - (v),
u² a,
(S24)
with a 0.01 denoting the external stimulus and B 0.25 as the reaction coefficient.
We generated the dataset using the method on a 1282 grid and timestep of 2.0 x 10-3 s. The spatial domain was set as x E [0, 128]2 with periodic boundary conditions applied. The simulation
18

---

T=0
T=
T= 600
==
T= 1200
DNS 204B
fo
P'C
Vorticity
DNS 2048
coa dy)a
P2c
-10
T=0
T= 300
T= 600
T= 900
T 1200
DNS 2048
2000
pèc
Vorticity
DNS 2048
Re 500
-5
Net
-10
Figure S3: Generalization across multiple Re's and forces.
time is T' = 10 s. This data is subsequently downsampled to 64 with timestep of 8.0 10-3 to serve as the ground truth. For training purposes, we employ three groups of trajectories, each comprising 1375 snapshots. For our testing phase, leverage ten distinct groups of trajectories, each with unique ICs.
NS. The NS equations are fundamental to the study of fluid dynamics, which govern the behavior of fluid motion. In this paper, we consider a 2-dimensional incompressible Kolmogorov flow with periodic boundary conditions in velocity-pressure pressure form, which is written as
du
(u u
V Vp+f, i [0,7], Re
(S25)
where fu, /) RR² denotes the fluid velocity vector, p is the pressure, Re representes the Reynolds number that characterizes the flow regime. The Reynolds number serves as a scaling parameter in the NS equation, resulting in balance between the inertial forces (captured by the advection term (u V)u) and the viscous fortes(captured by the Laplacian term Vu) Thus, for
19

---

Table S1: Impact of different kernel sizes.
Filter
RMSE
MAE
MNAD
HCT
x
0.1274
0.0926
0.0773
0.1947s
x
0.0064
0.0046
0.0037
1.4s
7
NaN
NaN
NaN
0.1673s
Table S2: Dataset generate details
Parameters/ / Case
Burgers
GS
FN
NS
DNS Method
FD
FD
FD
FV
Spatial Domain
[0, lI?
[0, lJ?
[0. 128]2
[0. 2m]²
Calculate Grid
100
128
128
2048
Training Grid
25
322
64
64
Simulation öt (8)
1.00 10-3
2.00
10
5.00 10-1
2.19 10
Warmup (S)
0.1
0
9
40
Training data group
5
3
Testing data group
10
10
10
10
Spatial downsample
16x
16x
4x
1024x
Temporal downsample
1x
4x
4X
32x
low Reynolds numbers (i.e., when Re is small), the viscous forces dominant, and the fluid flow is mostly laminar and smooth. On the other hand, at high Reynolds numbers (i.e., when Re large), the inertial forces dominant, and the fluid flow becomes more turbulent and chaotic.
In this paper, we generate the training data for a Re 1000, using periodic boundary conditions within the spatial domain [0, 2T|².. The data generation employs the FV method on 2048² grid with a simulation timestep 2.19 10-4, s. Subsequently, this data downsampled to 64 with time step of 7.0 x 10-3 to produce the ground truth. Five groups of trajectories, each containing 4800 snapshots, are employed for training. Ten groups of trajectories with diverse ICs are used for each of the tests. Specifically, to further test the model's generalization ability, expand the test set: (1) generating data corresponding to Re values of 200, 500, 800, 1000, and 2000, and (2) creating data with different external forces:
cos(4y) n 0.1u, f2 sin (in44) (4y)n, n 0.4n,
f3 cos (2y n 0.1u,
(S26)
f4 sin (2y) n 0.1u,
f5 cos( (44))) n 0.4u,
fg O
F
Training Details
All experiments were conducted single 80GB Nvidia A100 GPU, using Intel(R) Xeon(R) Platinum 8380 CPU (2.30GHz, 64 cores). We only give some of the changed parameters here, and the other hyperparameters remain the same as the default settings.
p2 Net. The architecture of Net, as shown in Figure 1 utilizes the Adam optimizer with learning set at 10-3 The model is trained with a batch size of 16 over 500 epochs. The rollout timestep settings are detailed in Table We employ the StepI.R scheduler scaling the learning rate by 0.96 every 200 steps.
FNO. The network architecture of FNO remains largely consistent with the original paper [8] with the primary modification being the adaptation the training method for this model to an autoregressive approach. We utilize the Adam optimizer, with learning rate of 1 and
20

---

Table S3: Impact noise on P'C'Net performance
Training
RMSE
MAE
MNAD
HCT (8)
1% noise
0.0092
0.0088
0.0062
1.4
0.5% noise
0.0078
0.0057
0.0047
1,4
w/o Noise noise
0.0064
0.0046
0.0037
1,4
Table S4: Impact of sparser Burgers datasct on Net performance
Training
RMSE
MAE
MNAD
HCT (8)
reduce 20%
0.0073
0.0052
0.0050
1.4
4x000 snapshots (in paper)
0.0064
0.0046
0.0037
1.4
batch size of 20. The training spans 1000 epochs, and the rollout timestep is aligned with that of the Net
UNet. We employ the modern UNet [22] architecture with the default setting, with the rollout timestep consistent with that of Net. The scheduler utilized StepL. with step size 100 and gamma of 0.96. The optimizer employed is Adam, with learning rate of 1 10 and batch size 10. The training is conducted over 1000 epochs.
We adopt the default architecture of DeepONet 77,, with the Adam optimizer. The learning rate set at 10-4, with a decay applied every 5000 steps scaling at 0.9. The training is conducted with batch size of 16 and spans 20000 epochs.
DeepONet.
PeRCNN. We utilize an architecture that is identical to the standard PeRCNN configuration [19] The optimization is performed using Adam, with StepLR scheduler that reduces the learning rate by a factor of 0.96 every 100 steps. The initial learning rate set at 0.01. The training regimen spans 1000 epochs with batch size of 32.
LI. We utilizes its default network architecture and parameter configurations [20]. The optimizer chosen is Adam with 31 0.9 and B3 0.99. The batch size is set to 8, with global gradient norm clipping of 0.01. The learning rate x 10-3, and the weight decay 1 x 10-6
G Additional Results
G.1 Applicability different BCs
To verify the applicability of our model to different BCs, we use the Burgers equation as an example and set the left boundary Dirichlet, the right boundary as Neumann, and the top/bottom boundaries as Periodic. Here, we denote this case as Complex Boundary Conditions (CBC). The rest of the data generation setup (e.g.. ICs, mesh grids) remains the same as used in the paper. We generated 10 CBC test datasets resulting from different random ICs.
We then directly tested the model previously trained based on Periodic BC datasets reported in the paper, meanwhile processing the boundaries using the BC encoding strategy [19] during inference. The quantitative results (average over 10 datasets) presented in Table SS where also list the predicted snapshots at 1.4 s for two random ICs in Figure SS4 We can see that our model is capable of generalizing different BCs.
G.2 Generalization Results
For the NS equation, we also evaluate the generalization capability of Net with respect to multiple Re's 'ss and forces. Please refer to Figure S3 for details.
21

---

Table S5: Generalization of Not over different boundaries
on the Burgers example for 10 trajectories.
BC Type
RMSE
MAE
MNAD
HCT(s)
Complex
0.0202
0.0103
0.0087
1.4
Periodic
0.0064
0.0046
0.0037
1.4
Ref.
22c
Ref.
22c2Net
0.21
0.28
u
-0.17
-0.12
0.54
-0.51
0.36
0.15
V
-0.19
-0.34
-.73
-0.82
Figure $4: PCC generalization over complex BCs (left Dirichlet, right Neumann, top/bottom Periodic) on the Burgers example for two random ICs, Snapshots at 1.4 s.
22

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: In the abstract and introduction, our content covers the challenges be addressed by the paper, contributions, and detailed overview of the applications.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: In the conclusion, we discuss the limitations of this work, including the focus on 2D problems.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
23

---

Answer: [Yes]
Justification: We state the lemma and corollary in the main paper, with detailed derivations provided in the Appendix Section A
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: As detailed in the paper, our model easily reproducible, with key training details provided in the Appendix, and the code can help readers reproduce the results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
24

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://niis.cc)
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nips. //nips.cc/putt wttpcc/pcccppuddeeeeeoeeeppeeeee hwtp://eummmmsiiinn.....
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide detailed descriptions of our experimental methods, comprehensive code for replication, and further information on additional results, proofs, and supplementary content the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We visualize our experimental results with informative figures and tables. These include violin plots for error distribution, error propagation curves, and key metrics like RMSE, MNAD, and MAE.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
25

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We offer sufficient computational resources and comprehensive information to reproduce the experiments, which can be found in Appendix Section
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics htttpp//..aaarrpp..c///ppp//ssss
Answer: [Yes]
Justification: Our research is solely intended for scientific purposes, conforms in every respect with the NeurlPS Code of Ethics, and poses no potential ethical risks.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. if there special consid- eration due to laws or regulations in their jurisdiction..
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
potential revolutionize fields like weather forecasting, turbulent flow prediction, and various other simulation tasks with its ability to accelerate computations. We emphasize that our work is intended for positive societal impact, with detailed discussions provided in the Appendix.
Justification: This method
26

---

Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considcrations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning conrectly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification: Our model addresses scientific computing problems and has been used by other researchers or institutions, such as Microsoft, with relevant data, which does not involve high-risk scenarios
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Our code and data are developed in-house, so they are approved by the creators.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
27

---

The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperssippsiiiie thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-pa se-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We pleased to provide the necessary documentation, data, and code to facilitate the reproduction our results, ensuring convenience for all users.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our research subjects spatiotemporal dynamical systems and does not involve humans human subjects
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our paper is limited to PDE issues, complies with national laws and regulations, and has no potential risks.
Guidelines:
28

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
29


Reviews:
[{"review_id": "TkVVOJPZfk", "reviewer": "Reviewer_xwvp", "summary": "In this paper the authors propose a PDE preserved coarse correction network for efficient prediction of spatio-temporal dynamics. The aim is to develop a learnable coarse model that accelerates simulation and prediction of spatio-temporal dynamics based on down-sampled data. The method mainly consists of 4 blocks, PDE block for computing spatial derivatives on coarse grids, correction block, NN block and Poisson block. The method proposed by the authors operates in tandem with the RK-4 numerical integration scheme and uses spectral methods to estimate the pressure field on the fly to condition the networks for predicting better corrections. The method yields significant accuracy gains over baselines for several use cases and showcases impressive generalization.", "questions": "- It seems like the Poisson block, NN block are dependent on the PDE block. If I consider the sequence of actions then the PDE block needs to be evaluated first. Fig 1 needs to be modified to reflect that.\n- In the PDE block, 3 types of inputs go to the RK-4 scheme, original solution, corrected solution and then the output of the poisson block. It is not clear how these different components are used for time integration? What had to be changed in the PDE block to implement the Euler scheme?\n- How is the Poisson block stable when initially the velocity derivatives calculated by the untrained weights are inaccurate? Wouldn’t this cause severe instability early on in the training? Is there some under-relaxation required? \n- It needs to be clarified that the equation residue block is basically the RK-4 integration.\n- Authors claim that “although the number of learnable parameters are limited, the coarse derivatives can still be accurately approximated after the model is trained (see ablation study in 4.2)”. However this claim cannot be verified from the ablation study that the derivatives computed truly match the derivatives computed at the same location from a finer grid. In order to verify this, the authors will need to compare the gradient computed by their kernel at a certain location on the coarse grid and compare it to the finite-difference gradients at the same location from a fine grid at the same snapshot in time.\n- There is a slight mismatch in the methodology between the baselines and the proposed approach. The baselines are designed to learn the mapping to predict the next solution, whereas the proposed method is designed to learn the correction. The paper should ideally include some baselines that are trained to learn just the correction as your method so that true advantage of the proposed approach is clear? Additionally, most of these baselines perform better when a history of solutions is provided as an input. Were the baselines such as FNO and UNet trained with a time history?\n- Why is a filter of 5x5 enough? Should the size of the filter be dependent on the amount of coarsening? Is the 4th order accuracy of the derivatives verified? How are the gradients calculated at the boundaries for non-periodic BCs?\n- Authors claim that the learnable PDE block is enough in many scenarios, can this be verified through an ablation study? One of the ablation studies should involve removing the NN block to verify that the gradients predicted by the PDE block truly contribute to the accuracy gains achieved by the method.\n- Another ablation study that would be interesting is to sum up the derivative from finite-difference kernel and from the trainable conv symmetric filter in the PDE block and compare it with just the derivative from the trainable conv filter.\n- Ablation study to understand the importance of the Poisson block is also required. Is the Poisson block really required? What happens if the pressure is also predicted as an output of the network? How would those results compare.\n- I don’t agree with the authors claim that the time marching in the network will inherently generalize the predictions to all ICs. Small discretization errors due to coarse grids can accumulate and the solution trajectories can be significantly different. The burden is actually on the network predicting corrections to correct the deviation in the trajectory.\n- How different are the training and testing initial conditions?\n- Need to see correlation plots over time steps for all the use cases to show how much the prediction and ground truth decorrelate over time for the proposed method as well as baselines.\n- Computational cost of the method?", "limitations": "NA", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- Trainable conv filter with symmetric constraints\n- Periodic BC padding embeds physical constraints.\n- Poisson block computes the pressure field on the fly using a spectral method.\n- The integration of ML methods with the numerical time integrators such as RK-4 to develop robust end-to-end solutions\n- Generalization and accuracy gains are very impressive over long time horizons, considering the fact that limited ICs are used for training.", "weaknesses": "- Is the method only applicable for problems with periodic BCs? The effectiveness of the derivative calculation as well as the imposition of the BCs will need to be verified.\n- Require a structured grid for the method to work as subsampling can be very difficult for unstructured methods especially when non-uniform input conditions such as geometry or source terms are involved. Additionally, developing a kernel for spatial derivative calculation might not be trivial, application of BCs and solving for the pressure on the fly will be challenging.\n- Method needs to be explained in a better way. Some of the questions related to the method are outlined below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### Weakness\n> **W1. Different BC encoding and derivative calculations.**\n\nDespite we showcase the eficacy of our model on datasets with Periodic BCs, it is applicable to handle other types of BCs (see **Table R2**, **Table R5** and **Fig. R1** in the *1-page PDF rebuttal file*). **Please see our detailed reply to Q1 in General Reply**. \n\n> **W2. Challenges for unstructured grids.**\n\nGreat remark! We acknowledge the challenges on extending the current pipeline to unstructured meshes. Nevertheless, the overarching framework is extensible, e.g., integrating graph neural networks and differentiable finite element/volume methods to learn on unstructured meshes. We will discuss the challenges in the revised paper.\n\n### Questions\n> **Q1. Re-organization of PDE-block, Poisson block, & NN block in Fig. 1.** \n\nConsidering the sequence of actions, placing the PDE block in Fig. 1.b first, followed by the Poisson block, would be better. We will modify it in the revised paper. \n\n> **Q2. Clarification of three input components for time integration in the PDE block.**\n\nPlease see **our reponse to Q1 in General Reply**.\n\n> **Q3. Stability of Poisson block in early stage of training.** \n\nThe Poisson block mainly relies on a numerical solver without any trainable parameters. We scale the initialized weights of the symmetric filter to ensure that the Laplacian term has a low magnitude. Thanks to the correction block, the solution does not reach a large magnitude, either. We added a loss curve to prove this point (averaged over 5 independent training trails), shown in **Fig. R4** of the *1-page PDF rebuttal file*. The consistent performance across these trials demonstrates that our model does not encounter instability during early training. We also explored relaxing the Poisson block by adding trainanle parameters (e.g., scaling coefficients). However, no improvement is observed. \n\n> **Q4. Equation residue block is the RK-4 integration.**\n\nYou are right -- the PDE block is indeed designed based on the RK4 integrator. However, the PDE block is trainable, consisting of the correction block and a trainable filter bank, which differs from the standard RK4-based finite diference (FD) solver.\n\n> **Q5. Comparing derivatives from kernels on coarse grid with those from FD on fine grid.**\n\nThe derivatives of these two strategies are different. We learn an approximate surrogate of derivatives on the coarse grid, which are different from the ground truth derivative values. Our goal is to minimize the overall PDE residual, instead of making each derivative equal to the corresponding ground truth.\n\n> **Q6. More baselines that learn corrections and take history of solutions as inputs.**\n\nWe further considered 3 correction learning baselines, including learned correction (LC) [PNAS 2021, 118(21):e2101784118], two variants of UNet and FNO by adding the NN Block for correction. The results in **Table R1**, in the *1-page PDF rebuttal file*, showed that our model has the best performance.\n\nWe have also trained UNet and FNO that use 12 history snapshots as inputs. However, their performance didn't improve. Note that LI and TSM in our baselines also use historical information (32 steps) as inputs. These baselines cannot capture the underlying dynamics due to the limited training samples.\n\n> **Q7. Filter size and its dependence on grid size.**\n\nFollowing your suggestion, we conducted experiments on different filter sizes, e.g., 3x3, 5x5, 7x7. As shown in **Table R4** in the *1-page PDF rebuttal file*, the model with the 5x5 filter outperforms the other two (the 3x3 filter has low accuracy while the 7x7 filter has instability issue on coarse grids). Hence, the 5x5 filter is empirically suggested.\n\n> **Q8. Ablation on removing the NN block.**\n\nFor simple systems, removing the NN block does not affect the results. For complex systems, e.g., NS, it causes the long rollout prediction diverged. Please see **our reply to Q3 in General Reply**.\n\n> **Q9. Ablation to replace trainable symmetric filter with FD kernel in the PDE block.**\n\nWe conducted an ablation study on the Burgers dataset, e.g., replacing our trainable conv filter with FD kernel in the PDE block (NN block excluded in the model). However, this led to NaN values in the test results due to large error accumulation. This indicates that FD kernels fails to approximate the derivatives on coarse grids.\n\n> **Q10. Ablation on the Poisson block.**\n\nWe conducted an ablation study on the NS dataset to evaluate the significance of the Poisson block, as shown in **Table R6** in the *1-page PDF rebuttal file*. We can see that the model's performance deteriorates without the Poisson block.\n\n> **Q11. Generalization to all ICs; discretization error accumulation.**\n\nSince PDE is embedded into the network with RK4 integrator, the model can in theory generalize prediction of solution trajectories over different ICs. However, we agree with you that large error accumulation over time might occur given a complex discretized IC on coarse grids. We will clarify this in our revised paper.\n\n> **Q12. Difference between the training and testing ICs?**\n\nWe use random seeds to generate different ICs. In the NS example, different ICs are created by generating random noise for each component of the velocity field and filtering it to create a divergence-free field with desired properties. We randomly selected 10 ICs and drew histograms to show the differences, shown in **Fig. R3** in the *1-page PDF rebuttal file*.\n\n> **Q13. Correlation plots over time steps.**\n\nThe correlation curve over time steps for the NS dataset is shown in **Fig. R2** in the *1-page PDF rebuttal file*. To provide a more comprehensive comparison, we also included DNS results for various grid resolutions in the figure. \n\n> **Q14. Computational cost.**\n\nPlease see **our reply to Q4 in General Reply**.\n\n**Remark:** Thanks for your constructive comments. Your consideration of improving the rating of our paper will be much appreciated!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hoeNGg584O", "reviewer": "Reviewer_WfL9", "summary": "The paper introduces the $P^2C^2Net$, which is designed to solve spatiotemporal partial differential equations (PDEs) using minimal training data. The architecture consists of two main components: a trainable PDE block and a neural network block. The trainable PDE block updates the coarse solution using a high-order numerical scheme with boundary condition encoding. The neural network block corrects the solution.", "questions": "1. What is the purpose of the lower path in the architecture of model shown in Figure 1(a)? The architecture already includes Poisson and correction blocks in the upper path. More explanation is needed to understand the function and necessity of this lower path.\n\n2. How are the convolution filters and the correction block initialized? This information is crucial for understanding the training process and the model's performance. Can the authors also plot the learning curves and other metrics in the training?\n\n3. How are the datasets sampled for each task? The current information is insufficient to determine whether the test datasets are fair. More details on the sampling process and how fairness is ensured are needed.", "limitations": "Yes.", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The model integrates physics knowledge directly into the network architecture, improving interpretability and generalizability, especially with limited data.\n\n2. The model achieves consistent state-of-the-art performance with over 50% gain.", "weaknesses": "1. The data generation process is not well explained. For instance, the training dataset for the Gray Scott model only includes three trajectories. What are the initial conditions for these three training samples and the ten testing samples? Initial conditions can lead to completely different patterns for the steady states. It is hard to believe that the model can generalize well with only three training samples unless it heavily relies on the classical solver. Even in this case, the improvement based on classical solver should relies on extensive data for generalization.\n\n2. What is the difference between $\\tilde{x}$ and $x$. The definitions are not clear. Additionally, why do you use $u$ in equation (1) and use $\\hat{u}$ in equaition (3). Even though some of the functionals are learnable, rigorouse definitions should be given.\n\n3. The comparison between P2C2Net and models like FNO and UNet is not fair because those models are purely data-driven, while P2C2Net incorporates classical solvers. There should be a performance comparison and discussion between P2C2Net and classical solvers. Additionally, the current runtime comparison in Figure 6 lacks detailed settings and hardware configuration.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### Weakness\n> **Q1(a). Generation of ICs of GS model for training and testing data.**\n\nFirst, to create ICs for the GS equation, we define a grid based on the spatiotemporal resolution and initialize the concentrations of chemicals A and B. Second, we set different random seeds to add random noise, changing the values of the chemicals at different positions. Finally, we obtain different ICs. Hence, the training and testing samples are independent. We will clarify this in our revised paper.\n\n> **Q1(b). Concern on generalization with limited training data.**\n\nOur model is designed to adhere to PDEs. While different ICs lead to varied steady-state patterns, these variations are ultimately governed by the underlying PDEs. Our model encodes the PDEs within a learnable PDE block which helps reduce the model’s dependence on data (please see **our reply to Q3 in General Reply**). This module ensures the model to accurately capture the underlying dynamics even on a coarse grid and with **limited data**. Since PDE is embedded into the network with RK4 integrator, the model is able to generalize prediction of solution trajectories over different ICs. \n\nMoreover, we employ different rollout timesteps during the training stage to improve the model's performance. That is, we treat a geneirc frame as an IC, and divide the trajectories into many samples based on the selected rollout timestep, which are then shuffled and grouped into batches for training (see Section 4.1 in our paper). As a result, the trained model is capable of extrapolating the prediction in the time horizon. \n\nFinally, we also demonstrated that our model can generalize to different types of boundaries (as discussed in **our reply to Q1 in General Reply**).\n\n> **Q2. Clarifications on $\\tilde{x}$ and $x$, and $u$ in Eq. (1) and use $\\hat{u}$ in Eq. (3).**\n\nWe downsample the data from the fine grid to the coarse grid, denoted by $x$ and $\\tilde{x}$ respectively, as mentioned in Sections 3.1 and 3.2.1 in our paper. We denote $\\mathbf{u}$ as the ground truth of spatiotemporal dynamics and $\\hat{\\mathbf{u}}$ as the solution after passing through the correction block, resulting in a corrected coarse solution. This process can be represented as $\\hat{\\mathbf{u}}_k=\\texttt{NN} (\\mathbf{u}_k)$, where the correction block is a trainable network defined in Section 3.2.2. Hope this clarifies your question.\n\n> **Q3(a). Fair comparisons between our method and baselines.**\n\nOur baselines include both purely data-driven (UNet and FNO) and physics-aware (PeRCNN and LI) methods. The reason why we consider UNet and FNO as baselines is to test their extreme performance on small datasets. Initially, we also considered DeepONet, but it was removed from the NS case due to its poor performance in the first three experiments. \n\nWe have also added temporal stencil modeling (TSM) [Sun, et al. ICML 2023] and learned correction (LC) [Kochkov, et al. PNAS 2021] as additional baselines, which use a history of solution as input and incorporate physics knowledge. The correlation curve is ploted in **Fig. R2** in the *1-page PDF rebuttal file*. We believe the additional experiments would improve the fairness in the baseline comparisons. \n\n> **Q3(b). Details of settings and configurations of runtine comparisons in Figure 6.**\n\nPlease see **our detailed response to Q4 in General Reply**. We will clarify this in our revised paper.\n\n### Questions\n> **Q1. The purpose and necessity of the lower path in Figure 1(a).**\n\nWe present the role of each module in **our response to Q2 in General Reply** and discuss their importance in **our response to Q3 in General Reply**. Given the potential instability and accumulated errors in the learnable PDE block, especially in the NS example, the lower path is indispensable. It serves to correct the solution generated by the PDE block. Moreover, since the input of the NN block includes pressure ${p}$ to provide more information, the lower path also includes an optinal Poisson block. Adding such a block help improve the model's performance as shown in **Table R6** in the *1-page PDF rebuttal file*.\n\n> **Q2. Initialization of conv filters and the correction block, and learning curves in training.**\n\nWe use random initialization for the symmetric Conv filters and scale the parameters to a low magnitude with an empirical value of 0.001. For the correction module, the Conv filters use the default Kaiming Uniform initialization, while other components use random initialization. We also plot the learning curves (averaged over 5 independent training trails for the NS dataset) as an example, shown in **Fig. R4** in the *1-page PDF rebuttal file*.\n\n> **Q3. Sampling process for datasets and fairness of the testing datasets.**\n\nFor the Burgers, FN, and GS datasets, which utilize the finite difference method as solver, the solution is calculated at mesh grids. To establish a coarse grid ground truth, we apply uniform downsampling [Rao, et al. Nature Machine Intelligence 2023] in both temporal and spatial dimensions. In contrast, the NS dataset, which uses the finite volume method, computes solutions at cell faces in a staggered manner. Therefore, we use staggered average downsampling [Kochkov, et al. PNAS 2021] (shown in **Fig. R5** in the *1-page PDF rebuttal file*) in the spatial dimension and uniform downsampling in the temporal dimension. \n\nFor fairness of testing datasets, we randomly select 10 seeds from the range of [1, 10,000], which are used to generate different Gaussian noise disturbances, which are then applied to the initial velocity field to create varying ICs. We also perform a warm-up phase during data generation to ensure that the variance and mean of the trajectories are closely aligned, thereby maintaining fairness. We wiil clarify this in the revised paper.\n\n**Remark:** Thanks for your constructive comments. Your consideration of improving the rating of our paper will be much appreciated!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "65lBPmn1wK", "reviewer": "Reviewer_aBLf", "summary": "The paper solves the problem of predicting complex spatiotemporal dynamics on coarse mesh grids with only a small set of data. It proposed a learnable symmetric Conv filter to estimate the spatial derivatives on coarse grids and incorporated RK4 for correcting coarse solution at low resolution.", "questions": "In addition to the Weakness, I have the following two questions regarding the implementation:\n\n1, To train all the components, is it an end-to-end training or each block is trained individually?\n\n2,  For the generalization test, did more training was done to test on six external force scenarios and Reynolds numbers?", "limitations": "The authors adequately addressed the limitations.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The problem is well-formulated with real applications. A innovative structured Conv filter is proposed to estimate the spatial derivatives. The effectiveness of the method is demonstrated through extensive and robust experimental results.", "weaknesses": "1, The proposed scheme has a lot of components but the connection among these blocks are not obvious and the necessities of having all these components is not really justified. For example, what is the difference between correction block and the NN block? How is this NN block learned on the fly exactly? \n\n2, Could you please provide some details about the difference between the proposed method and PERCNN except for the poisson block?\n\n3, In comparison with other baseline models, what adaptation is used on example NS? For the proposed model, NS requires an additional NN block.\nI’m not convinced that without specific adaptation of other models to this example is a fair comparison.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for acknowledging the novelty and the effectiveness of our proposed method. We address your concerns as follows.\n\n### **Weakness**\n> **Q1. Clarifications of each block component, their connections and differences.**\n\nThe clarifications of each block component and the connections can be found in **our response to Q2 in General Reply**. \n\nIt is noted that the correction block aims to address the information loss that occurs due to resolution reduction before calculating derivatives, helping the model adapt to the coarse grid. During training, it serves as a scaling factor for the derivative term. Achieving this with only a shallow network (2 layers with modes 12) is feasible. However, since the network is trained on a coarse grid with a high spatial coarsening ratio (up to $32^2=1024$ on NS), the reduced high-resolution values do not fully satisfy the governing equation. To mitigate error accumulation during the rollout prediction, compensating the PDE block is necessary. This requires a deeper network (e.g., 4 to 6 layers of FNO).\n\n> **Q2. Difference between our method and PeRCNN except for the Poisson block.**\n\nThe differences between these two methods are listed as follows.\n\n-PeRCNN utilizes feature map multiplication to build polynomial combinations (with redundant terms) to approximate the underlying governing equation, which has great performance given *fine mesh grids*. In contrast, our proposed model directly incorporates the governing PDE, preserving its complete and precise form, to predict spatiotemporal dynamics on *coarse mesh grids* under a *small and sparse data environemnt*. \n\n-Learning the diffusion term (e.g., $\\Delta \\mathbf{u}$) in the governing PDE is challenging. PeRCNN employs fixed finite difference Conv to compute the diffusion term, leading to significant errors. Our approach introduces a learnable Conv with symmetric constraint to compute the diffusion term, effectively mitigating errors on the coarse grid. \n\n-PeRCNN is limited to testing simple cases (e.g., small domains and no external forces) and struggles with generalization to external forces and Reynolds number. Finally, PeRCNN aims to learn high-resolution dynamics with an initial state generator super-resolving from low-resolution grids. \n\nHence, our model is fundamentally different from PeRCNN.\n\n> **Q3. Adaptations for NS equations and fair comparisons by adding NN blocks for baselines.**\n\nThe NN block is a flexible component of our model that can be included or excluded during training based on demand. Since the PDE block alone is sufficient to achieve the state-of-the-art performance on Burgers, FN, and GS datasets, we chose to remove the NN block in these cases to establish a lighter model. However, for the NS equation, given the complexity of the flow ($Re=1000$), we need the NN block as a supplement to alleviate the error accumulation of long-term prediction and the instability issue on coarse grids.\n\nTo address the fairness comparison issue, we integrated the NNblock into both UNet and FNO models. The results, presented in the **Table R1** in the *1-page PDF rebuttal file*, show that while NNblock enhances the performance of both models, the improvements are still significantly lower compared to our proposed model.\n\n### **Questions**\n> **Q1. End-to-end training or individual training?**\n\nGood question! Our model features end-to-end training, making the training process user-friendly and avoiding separate training of individual modules. This also can lead to better overall performance and coherence in the model's outputs, as the interdependencies between different parts of the model are explicitly accounted for during training. Additionally, the inclusion of prior knowledge facilitates the model convergence as shown in **Fig. R4** in the *1-page PDF rebuttal file*. We will clarify this in our revised paper.\n\n> **Q2. More training for generalization tests, such as external force scenarios and Reynolds numbers.**\n\nNo additional training was performed for the generalization tests. This aspect has been detailed in our paper, specifically in Section 4.2 (Generalization Test). The model was trained solely on five training sets (trajectories) with Reynolds number $Re=1000$ and external force $\\mathbf{f} = \\mathrm{sin}(4y)\\mathbf{n}_{\\mathit{x}} - 0.1\\mathbf{u}$. Namely, once our model is trained, it can generalize to the different ICs, Reynolds numbers, and external force terms as shown in the paper.\n\n\n**Remark:** We sincerely thank you for putting forward constructive comments/suggestions. We hope the above responses are helpful to clarify your questions. We will be happy to hear your feedback and look forward to addressing any additional questions. Your consideration of improving the rating of our paper will be much appreciated!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "z27ISHoDqY", "reviewer": "Reviewer_bvnV", "summary": "The authors present a P^2C^2Net framework for efficiently solving and simulating PDEs, specifically on coarse grids. To overcome the challenges of simulating PDEs on coarse grids, including the difficulty of estimating numeric derivatives and the inaccuracies in the right-hand side of PDEs on coarse grids, the P^2C^2Net architecture comprises the following modules: (1) an RK4 integration scheme to advance in time, (2) a learnable PDE block that approximates the true dynamics on coarse grids, (3) a symmetric convolutional filter that learns numerical derivatives from data, and (4) a neural corrector module. When modeling the Navier-Stokes equations, an optional Poisson block can be included. The proposed network has shown promising results on several classical benchmark problems, including the Burgers' equation and Navier-Stokes equations, compared to standard baselines in the literature.", "questions": "1. What is the training time for the proposed network on all benchmark examples? How efficient is it compared to other baselines?\n2. How robust is the model to noisy or incomplete training data?\n3. How does the proposed network handle different boundary conditions beyond the periodic ones discussed in the paper?", "limitations": "Yes, the authors did address the limitations.", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The authors present the P^2C^2Net framework for the simulation of PDEs on coarse grids, which is data-efficient. Several modules are incorporated, including the learnable symmetric convolutional filters and a neural correction module to overcome the insufficient information from the low-resolution data. Extensive experimental examples performed by the authors have shown that the architecture demonstrates better accuracy than existing baselines, including Fourier Neural Networks and PeRCNN. In addition, the model achieves satisfactory performance on out-of-distribution tests, i.e., the network shows promising results with different Reynolds numbers in the Navier-Stokes equation example.", "weaknesses": "Although the structure of the network is clearly presented in the paper, some issues remain to be addressed. First, the Poisson solver and neural correction module are both Fourier Neural Networks. This raises the concern that the advantage of the model may stem from its larger size compared to other baseline models. Although an ablation study is performed on the importance of different modules, it lacks a direct comparison with other models regarding the sizes of the model's parameters. Secondly, the training efficiency is questionable. From the paper, it appears that the proposed model is much larger than the considered baselines. Overall, the efficiency of the proposed model remains unclear, making it difficult to conclude whether the architecture design or simply the size of the model is responsible for its superior performance compared to other baselines.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for achknowledging the data-efficiency and satisfactory performance on OOD tests of our work. We address your concerns as follows. \n\n### Weakness\n> **Q1(a). Concerns on the larger size of the Poisson solver and neural correction modules compared to baselines.**\n\nThe Poisson solver is essentially a numerical method (e.g., a spectral solver). The basic idea is to convert the original problem into the frequency domain, where the spatial differential operation becomes a multiplication operation via Fourier transform. The Poisson equation is then solved in the frequency domain, and the inverse Fourier transform is then used to obtain the solution in the original spatial domain. This entire process does not involve a Fourier neural network or any learnable parameters. \n\nThe correction block utilizes only two Fourier layers that retain a few number of modes, and is therefore not considered a large-scale neural network. Given the unique network architecture, our model achieves good generalization performance under a small and sparse data environment, highlighting its advantages over existing baseline models. For your reference, we show the number of model parameters in **Table R1** in the *1-page PDF rebuttal file*. \n\n> **Q1(b). Training efficiency of the proposed method.**\n\nPlease see **our response to Q4 in General Reply Q4** for more details.\n\n### Questions\n> **Q1. Training time on all benchmark examples compared to baselines.**\n\nExcellent comment! The training time for different mothods is shown in **Table A** below. It can be observed that the training time of our model is generally acceptable. Note that the training has been performed on a single 80GB Nvidia A100 GPU, as described in Appendix F in the paper.\n\n**Table A.** Training time for different models.\n\n| Training cost | Burgers | FN | GS | NS |\n| -------- | -------- | -------- | -------- |-------- |\n|   P$^2$C$^2$Net (ours)  | 2h     |  0.6h   |  2.5h    | 69h  |\n|  FNO   | 1.5h     |  2.2h    |  2.1h    | 33h  |\n|  UNet   | 8.7h     |  15.3h    |  14.1h    | 178h  |\n|  DeepONet   | 0.7h     |  0.9h    |  1.0h    | -  |\n|  PeRCNN   | 6.1h    | 12.4 h    |  10.5h    | 73h  |\n|  LI   | -     |  -    |  -    | 67h  |\n\n> **Q2. Robustness to noisy or incomplete training data.**\n\nGreat remark! Using the Burgers equation as an example, we introduced Gaussian noise of varying scales during the training process and observed its minimal impact on the results. The results are presented in **Table B** below, which indicate that our model is robust to certain noise and maintains the HCT (high correlation time) metric without reduction.\n\n**Table B.** Impact of noise on P$^2$C$^2$Net performance.\n| Training | RMSE | MAE | MNAD | HCT (s)|\n| -------- | -------- | -------- | -------- |-------- |\n| + 1%   noise     | 0.0092     | 0.0088     | 0.0062     |1.4     |\n| + 0.5% noise    | 0.0078     | 0.0057     | 0.0047     |1.4     |\n| + 0.3% noise    | 0.0072     | 0.0050     | 0.0041     |1.4     |\n| w/o Noise     | 0.0064     | 0.0046     | 0.0037     |1.4 \n\nMoreover, for the Burgers case, the time steps of the training data consist of only 28% of the inference steps in the test data. Namely, our training data itself is incomplete, and the missing data accounts for 72% of the test set. Based on this sparse dataset, we further randomly reduced the training data by 20% to simulate data incompleteness (e.g., randomly deleting data according to the rollout step size to make the trajectory incomplete, where the specific size can be found in Table 1 in the paper). The results of this experiment are shown in **Table C** below, where values are averaged over 10 test sets. Additionally, we conducted the same experiment on the GS dataset, with the results presented in **Table D** below. We can observe that, after making the data sparser, our model performance slightly decreases, which means that our model is capable of handling scenarios with incomplete data.\n\n**Table C.** Impact of sparser Burgers dataset on P$^2$C$^2$Net performance.\n\n| Training data | RMSE | MAE | MNAD | HCT (s)|\n| -------- | -------- | -------- | -------- |-------- |\n| reduce 20%   |  0.0073    | 0.0052    |  0.0050   |1.4     |\n|5x400 snapshots(in paper)| 0.0064  | 0.0046  | 0.0037  | 1.4 |\n\n**Table D.** Impact of sparser GS dataset on P$^2$C$^2$Net performance.\n\n| Training data | RMSE | MAE | MNAD | HCT (s) |\n| -------- | -------- | -------- | -------- |-------- |\n| reduce 20%   |  0.0182    |   0.0091  |  0.0093   | 2000    |\n| 3x1000 snapshots(in paper)   | 0.0135     |  0.0062   |  0.0062   | 2000   |\n\n> **Q3. Applicability to different BCs beyond the periodic BCs.**\n\nDespite we showcase the eficacy of our model on datasets with Periodic BCs, it is applicable to handle other types of BCs, such as Dirichlet and Neumann BCs (see **Table R2**, **Table R5** and **Fig. R1** in the *1-page PDF rebuttal file*). **Please see our detailed response to Q1 in General Reply**. \n\n**Remark:** We sincerely thank you for yopur constructive comments. We hope the above responses are helpful to clarify your questions. We will be happy to hear your feedback and look forward to addressing any additional questions. Your consideration of improving the rating of our paper will be much appreciated!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
## General reply

We deeply appreciate the insightful and constructive comments from the reviewers, which are helpful in improving our paper. We are pleased that all the reviewers recognized the novelty and excellent generalizability of our work. In particular, we thank the reviewers for recognizing the *robustness* (xwvp and aBLf), *interpretability* (WfL9), and *data-efficiency* (bvnV) of our method. 

We have summarized a detailed reply to several **common questions** and addressed other concerns in each individual rebuttal. In addition, we have also listed **six tables** and **five figures** in the *1-page PDF rebuttal file* to support our rebuttal.

> **Q1. Applicability to different BCs beyond periodic BCs.**

Excellent comment! To verify the applicability of our model to different BCs, we use the Burgers equation as an example and set the left boundary as *Dirichlet*, the right boundary as *Neumann*, and the top/bottom boundaries as *Periodic*. Here, we denote this case as Complex Boundary Conditions (CBC). The rest of the data generation setup (e.g., ICs, mesh grids) remains the same as used in the paper. We generated 10 CBC test datasets resulting from different random ICs. 

We then directly tested the model previously trained based on Periodic BC datasets reported in the paper, meanwhile processing the boundaries using the BC encoding strategy as shown in **Table R2** in the *1-page PDF rebuttal file* during inference. The quantitative results (average over 10 datasets) are presented in **Table R5** in the *1-page PDF rebuttal file*, where we also list the predicted snapshots at 1.4 s for two random ICs in **Figure R1**. We can see that our model is capable of generalizing over different BCs. We will add this result in the revised paper.

> **Q2. The flow of data between blocks and their explanation.**

In Figure 1a, the network architecture includes two paths: the upper path computes the coarse solution using a learnable PDE block, while the lower path is incorporated into the network to correct the solution on a coarse grid with a Poisson block and a NN block. The data flow operates as follows. 

-The network accepts $\mathbf{u}_k$ as input and processes it by the PDE block on the upper path, where the PDE block computes the residual of the governing equation. A filter bank, defined as a learnable filter with symmetry constraints, calculates the derivative terms based on the corrected solution (produced by the correction block). These terms are combined into an algebraic equation (a learnable form of $\mathcal{F}$). This process is incorporated into the RK4 integrator for solution update. 

-In the lower path, ${\mathbf{u}_k}$ is first corrected by the correction block, and ${p_k}$ is computed by the Poisson block. Inputs, including solution states $\{\mathbf{u}_k, p_k\}$ and their derivative terms, forcing term, and Reynolds number, are fed into the NN block. The output from this block serves as a correction for the upper path. 

-The final result $\mathbf{u} _{k+1}$ is obtained by combining the outputs from both the upper and lower paths. During the gradient back-propagation process, NN block learns to correct the coarse solution output of the PDE block on the fly, and ensures that their combined results more closely approximate the ground truth solution.

We will include more details in the revised paper to make this procedure more straightforward.

> **Q3. The importance of PDE block, NN block, and Poisson block.**

We found that using a simplified version of the model (without the parallel NN block) can still achieve SOTA performance on relatively simple systems such as the Burgers, FN, and GS datasets. That is said, the learnable PDE block itself alone has satisfactory performance. However, for the NS equation, given the complexity of the flow ($Re=1000$), we need the NN block as a supplement to alleviate the error accumulation of long-term prediction and the instability issue on coarse grids. 

In fact, the learnable PDE block aims to reduce the network's reliance on training data. To achieve better results, both the PDE block and NN block are necessary. We clarify that the Poisson Solver is a numerical solver without learnable parameters. Moreover, we conducted an additional ablation study via removing (1) the PDE block and (2) the Poisson block, respectively. These results are shown in **Table R6** in the *one-page PDF rebuttal file*, and will be added to the revised paper. We can see that both blocks are essential to maintain satisfactory prediction accuracy.

> **Q4. Training and inference time.**

We recorded the training and inference time taken by the model on Burgers, FN, GS, and NS in **Table A** (with the inference time steps of 1400, 1250, 1000, and 1200, respectively). Please note that this result is based on optimized codes showing a better performance compared with that reported in Figure 6 in the paper (this figure will be updated). It is observed that the training time of our model on the first three datasets are relatively short. On the NS dataset, due to the increased model complexity, the training time increases accordingly. Nevertheless, the inference time for all cases remians similar. Note that both training and inference are performed on a single 80GB Nvidia A100 GPU, as described in Appendix F of the paper.

**Table A.** The effiency of P$^2$C$^2$Net on different datasets.

| Time cost | Burgers | FN | GS | NS |
| -------- | -------- | -------- | -------- |-------- |
|  Training   | 2h     |  0.6h   |  2.5h    | 69h  |
|  Infer Cost   | 17s     |  16s    |  15s    | 20s  |

Please do feel free to let us know if you have any further questions. Thank you very much!

Best regards,

The Authors of the Paper

Meta-review: The paper proposes a new correction method for solve PDEs on coarse meshes. The reported numerical results are quite convincing, giving significant improvement. All the reviewers agree that the paper should be accepted, and I agree as well.
Decision: Accept (poster)