Incomplete to complete multiphysics forecasting - a hybrid approach for learning unknown phenomena
Authors: Nilam Nandkishor Tathawadekar
Nguyen Anh Khoa Doan
Camilo Fernando Silva
Nils Thuerey
Venue: Submitted to ICLR 2023 (2023)
Keywords: neural physics simulations
multi-physics systems
reactive flows
differentiable PDE solvers

Abstract: Modeling complex dynamical systems where only partial knowledge of their physical mechanisms is available is a crucial problem across all scientific and engineering disciplines. Purely data-driven approaches, which only make use of an artificial neural network and data, often fail to accurately simulate the evolution of the system dynamics over a sufficiently long time and in a physically consistent manner. Therefore, we propose a hybrid approach that uses a neural network model in combination with an incomplete PDE solver that provides known but incomplete physical information. In this study, we demonstrate that the results obtained from the incomplete PDEs can be efficiently corrected at every time step by the proposed hybrid neural network – PDE solver model, so that the effect of the unknown physics present in the system is correctly accounted for. For validation purposes, the obtained simulations of the hybrid model are successfully compared against results coming from the complete set of PDEs describing the full physics of the considered system. We demonstrate the validity of the proposed approach on a reactive flow, an archetypal multi-physics system that combines fluid mechanics and chemistry, the latter being the physics considered unknown. Experiments are made on planar and Bunsen-type flames at various operating conditions. The hybrid neural network - PDE approach correctly models the flame evolution of the cases under study for significantly long time windows, yields improved generalization, and allows for larger simulation time steps. 

Under review as a conference paper at ICLR 2023
INCOMPLETE TO COMPLETE MULTIPHYSICS FORE- CASTING A HYBRID APPROACH FOR LEARNING UN- KNOWN PHENOMENA
Anonymous authors Paper under double-blind review
ABSTRACT
Modeling complex dynamical systems where only partial knowledge of their physical mechanisms is available a crucial problem across all scientific and engineering disciplines. Purely data-driven approaches, which only make use of an artificial neural network and data, often fail to accurately simulate the evo- lution of the system dynamics over sufficiently long time and in a physically consistent manner. Therefore, we propose a hybrid approach that uses a neu- ral network model in combination with an incomplete PDE solver that provides known, but incomplete physical information. In this study, we demonstrate that the results obtained from the incomplete PDEs can be efficiently corrected at ev- ery time step by the proposed hybrid neural network - PDE solver model, so that the effect of the unknown physics present in the system is correctly accounted for. For validation purposes, the obtained simulations of the hybrid model are successfully compared against results coming from the complete set of PDEs de- scribing the full physics of the considered system. We demonstrate the validity of the proposed approach on a reactive flow, an archetypal multi-physics system that combines fluid mechanics and chemistry, the latter being the physics considered unknown. Experiments are made on planar and Bunsen-type flames at various op- erating conditions. The hybrid neural network PDE approach correctly models the flame evolution of the cases under study for significantly long time windows, yields improved generalization, and allows for larger simulation time steps.
INTRODUCTION
Modeling and forecasting of complex physical systems described by nonlinear partial differential equations (PDEs) are central to various domains with applications ranging from weather forecast- ing (Kalnay 2003), design of airplane wings (Rhie & Chow 1983), to material science (Wheeler) et al. 1992). Typically, chosen set of PDEs are solved iteratively until convergence of the solution. Modeling complex physical dynamics requires a good understanding of the underlying physical phc- nomena. For cases where the complete physics information is missing, deep learning models can be employed to complete the physical description when additional data of the system is available. Deep learning methods have shown promises to account for these unknown components of the sys- tem (Yin et al. (202I). We consider a set of partial differential equations with partially unknown physics represented. The corresponding PDE model for general state é is given by
OΦ = Pi(á. Oo 0² Or Oz :...) + Sa.
(1)
ot
where P; models the known but incomplete PDE description, Sp represents the unknown physics.
Within the scope of this model, the influence of Sg can lead to fundamentally different scenarios. commonly targeted case is when the governing equations of the complete PDE description are computationally too expensive to solve, turbulence modeling in computational fluid dynamics (CFD) being a good example. In CFD, a spacial filtering is performed on the original governing PDEs. This step introduces unclosed terms in the model equations that correspond to unrepresented physies in equation I due to the effects of the filtered seales. This is a widely studied problem, where the use of deep learning models is currently being explored (Lapeyre al. 2019 Kochkov et al. 2021 List

---

Under review as a conference paper at ICLR 2023
et al. 2022 Stachenfeld et al. 2022). In the following, are targeting more challenging problem, where increasing spacial resolution and/or reducing time-scales of the incomplete PDE solver does not lead to converged full solution. Rather, the incomplete and complete PDEs produce drastically different solutions due to the unknown physics. The central learning objective to correct this behavior and retrieve the evolution that would be obtained with the complete PDE description.
Our work expands on the combination incomplete PDE solvers and neural networks (NNs) al. 2021 Takeishi & Kalousis 2021) account for the effects of incomplete physics model. The NN aims to complete the PDE description, where the differences in complete and incomplete PDE solutions are beyond the effects of spacial and temporal scales. We showcase that combining the trained NN model with differentiable solver for the incomplete PDE can accurately reproduce the physical solutions of the complete, multi multi-p PDE solver with stable long-term rollouts.
We demonstrate the capabilities of this approach for an archetypal multi-physics system, namely complex reactive flow which combines fluid mechanics and chemistry, where the latter is considered unknown. Reactive flow modeling has applications in numerous domains such as combustion processes gas turbines (Lieuwen) 2012), climate modeling (Jacobson, 1999, Rol- nick et al.] 2022) and astrophysics simulations (Gamezo et al.) [200)). Resolving the Navier- Stokes equations lies at the core of these problems, where additionally the transport of dif- ferent species of relevance must also be accounted for, together with their production or sumption often following complex reaction mechanisms (Poinsot & Veynante (005). For chem- ically reacting flows, generation consumption of multiple species via some chemical reac- tion are modeled using a net source term. It is well-known fact that the incorporation of detailed chemical kinetic mechanism in reacting flow model can result in stiff sys- tem of governing equations (Wanner & Hairer 1996 Najm et al. 1998 Knio et al. 1999).
We consider the reactive flow simulation to be the complete PDE description, while the non-reactive flow simulation represents the incomplete PDE ba-
Incomplete PDE
2003
sis, where the chemical kinetic mechanisms are col- lected in the unknown physics term of equation [ Figure I shows a visual example of the fundamental differences in system dynamics that can be caused by unknown reaction terms. We showcase the effec- tiveness of our approach for different cases of planar 2D premixed methane-air flames, and the varying transient evolution of Bunsen-type flames. We show that the proposed approach can handle large domains with highly resolved flames, which is closer the practical flame domains used in many industrial ap- plications. Specifically we concentrate on training a NN model to correct the spatio-temporal effects of energy and species transport source terms. We show that in addition to recovering the desired solutions, this approach overcomes inherent problems of temporal stiffness due to the complex reaction mech- anism. Lastly, we demonstrate the applicability of neural network model to control the flow inlet velocities to arrive at desired flame shapes, when combined with a differentiable flow solver.
Complete PDF
0
50
t= 300
Figure The incomplete/non-rractiive (top) and complete/reactive (bottom) PDE solvers we consider can yield fundamentally differ- ent evolutions, shown here for sample temperature field over time.
RELATED WORK
Deep learning methods have been widely used model the solutions of partial differential equations
(Lagaris et al., 1998: Long et al., 2018 Han et al.) 2018 Bar-Sinai et al. 2019) and in particular,
the Navier-Stokes equations (Thuerey et al., 2020 Fukami et al. 2019) These models can be very fast and do not suffer from the time-step stability issues associated with traditional numerical solvers. Nevertheless, as these purely data-driven approaches lack the physical understanding of the system being modeled, they generally fail in generalizing to other operating conditions (Kim al. 2019 Lapeyre et al.] 2019). To leverage the potential of deep learning in physical simulations, it is therefore necessary incorporate some physical information within the deep learning framework.
2

---

Under review as a conference paper at ICLR 2023
Deep learning models enforce physical constraints partially through the loss function SarrSinai et al. 2019 Raissi et al. 2019) or changes in neural network architecture (Greydanus et al. 2019 Lu
et al. 2021). However, these approaches struggle to enforce physical constraints such as boundary
conditions or predict the strong unsteadiness and chaotic nature of flows. Neural operators (Li et al.
22020b] [Bhattacharya al. 2021; Patel al. 202T), specifically, the Fourier neural operator (FNO) of[Li et al. (2020a) introduce an interesting line of work by learning mesh-free, infinite-dimensionad operators with neural networks, but do not necessarily offer advantages for longer term predictions.
Hybrid approaches that combine machine learning techniques with numerical PDE solvers (Wang at have attracted a significant amount of interest due to their capabilities for generalization (Chen et al. 2018). In this context, neural networks are typically used to model or replace part of the conventional PDE solver to improve aspects of the solving process. For example, [Tompson] et al. (2017). Ozbay et al. (2021) and Ajuria Illarrmenddi et al. (2020) proposed a convolutional neural network based approach solve the Poisson equation CFD simulation.
recent years, number of deep learning based models have been introduced to accurately model turbulent flows
(Pahhkk et al. [2200] Stachenteld et al.) 2022 Dresdner et al. 2022). Um et al. (2020) and [elbute-
Peres et al. (2020) showed the advantages of training neural networks with differentiable physics to correct the numerical errors that arise in the discretization of PDEs. [Sirignano et al. (2020) and Kochkov al. (2021) additionally correct for closure error by integrating neural network with differentiable CFD simulator. These approaches demonstrate the capabilities of neural networks to correct errors in fast, under-resolved simulation.
Similar to the goals of our work. Yin et al. (2021) and Takeishi & Kalousis 2021) have introduced frameworks of augmenting incomplete physical dynamics with neural network models. These ap- proaches demonstrate the applicability on ODE/PDE systems, which are weakly nonlinear and the unknown dynamics are linear combinations of the underlying flow fields. We expand on these works to explore the significantly more challenging scenario of reactive flows. These are characterised by a multi-physics system with nonlinear advective terms and strongly nonlinear dynamics, described by exponential source terms that exhibit nonlinear combinations of the flow fields.
3 METHODOLOGG
We consider two different sets of PDEs with their associated numerical solver, which we denote as the incomplete PDE P. and the complete PDE Pe. By evaluating Pe on an input state Φr at time t, we can compute the points of the phase space sequences; Φetai = Pe(())). Without loss of generality, we assume fixed time-step At and denote state Φt-At at next time instance as Φ1+1- The main goal of this study is to learn a correction function that models the effects of the unknown physical system on the incomplete PDE solver to obtain the complete PDE solutions. We train correction model C():0)= : + X' that maps P(())) to di+1, defined over a finite time interval [0, T] and a spacial domain Ω c R22. The learning objective is to find the best possible correction function C'o;; () given the weights 0 and input flow state, 6. The model parameters 0 are estimated from the complete PDE solution trajectories (0o, $1, ΦT). The learned predictions obtained after repeatedly applying the corrector C and invoking Pe are denoted by (do. $1,..,
3.1 PARTIAL DIFFERENTIAL EQUATIONS FOR REACTIVE FLOWS
The physical system we investigate is a laminar premixed methane-air flame with a single step chemistry. It is governed by the following Navier-Stokes equations (Poinsot & Veynante 2005)
Op ppu.
pCpl ØT + Ix. O1
duiT w/ + Ox.
δ (A
OT
01 +
dx:
Oz:
(2)
0pm,
0Tg Øt 0x: dx, Oz:
Opuuu Op
OpYz OpuYro Ot + dr.
+
=
wy + or:
(pDx
dr:
where P. u4 p. T, Y& denote the density, velocity, pressure, temperature, and species mass fractions of species k. respectively. т, DR. À are the strain rate tensor, the diffusion coefficient of species k and mixture thermal conductivity. In addition, Cp denotes the mixture specific heat capacity. de and wy. are the species reaction rate and heat release rate, respectively.
3

---

Under review as a conference paper at ICLR 2023
(A)
+
(B1)
Purely Model
Neural Network
Predictor Block
Model ((d.10)
6
(B2)
de01
Hybrid NN-PFE Hodel
Pc+2
Predictor Block
Incomplete Solver P:(ok)
Neural Natwark COP-C6110
Co. Serae Saaie
Figure 2: (A) Multi-step training framework helps to learn the dynamics of complete PDE solver over longer rollouts. (B1) Details of the input flow state and predictor block used in a purely data- driven approach and (B2) the hybrid NN-PDE approach, where S denotes the concatenation of different fields to obtain the complete flow state Φ next time step.
We consider a one-step chemical system of the type F v/00 - products, where vip and vi the stoichiometric coefficients corresponding by Williams (1955) and Mitani (1980) are used to model the reaction rates Wy for each species. The fuel consumption rate Wp is assumed have the Arrhenius form.
fuel and oxidizer. The simplifications proposed
pr WF
p Wo
F RT
wF = BB
no exp(
The heat release source term w/ and the fuel source term we are linked by wh = - -Wp,, where Q is the heat of reaction per unit mass. Following Poinsot & Veynante (2005), parameters corre- sponding to real-world methane-air flame are chosen as: B1 = 1.0810" uSI; B1 = 0; E,
83600 J/mole; 7F = 1; no 0.5; Q = 50100 kJ/kg; C, = 1450 J/(kgK). Taken together, the system of equations above is challenging scenario even for classical solvers, and due to its practical relevance likewise highly interesting environment for deep learning methods.
3.2 PROBLEM FORMULATION
The incomplete PDE solver solves the set of equations without the source terms and reaction rates Wr and wy. while the complete PDE solver solves the full set from equation 2 The neural network model, denoted by (()(())))) corrects the incomplete/non-reactinn
flow states (Ø) to obtain
complete/reacting states (e) as shown in figure (B2). The neural network is trained to model the effects of the unknown chemistry using parameters given an input flow state, o = [u,, p. T,,,, Yol- As seen from equation 2 and equation 3] the temperature and species
fractions fields strongly coupled which significantly increases the prediction problem complexity. A slight error in one of the fields will quickly propagate into the other fields, making the predictions diverge. In the following, a subscript Co) will denote that the neural network C' generates the field 8, e.g. CT gen- erating the temperature field. Φ.+1 = [ui,++1: P.+1, (P:())))))) Cr, (o.)). Cr. (o:)))) where indicates a corrected state. u1,t1 and P++1 are the time-advanced velocity and pressure field, respectively, predicted using the incomplete PDE solver.
3.3 TRAINING METHODOLOGGY
We employ a hybrid NN-PDE approach that augments a neural network model with a PDE solver (Um al. 2020 Kochkov et al. 202T). In contrast to previous work, we use the incomplete PDE solver basis, and hence the solver does not converge to the desired solutions under refinement, as explained i31 The neural network is integrated and trained in a loop with the incomplete PDE solver using stochastic gradient descent for time steps, shown in figure 2 (A.B2). Here, the paremmparretter of the training process.
number of temporal look-ahead steps, m. is an important Higher provides the network with longer-term feedback at training time through the gradient roll- outs. This gives the model improved feedback on how the time dynamics of the incomplete PDE solver affect the input states, and hence which corrections need to be inferred by the model.
4

---

Under review as a conference paper at ICLR 2023
A central component of the hybrid NN-PDE model is the differentiable solver, which allows to embed the solver for the incomplete PDE system in the training of a neural network. The differen- tiable solver acts additional non trainable layers in the network. Instead, they provide derivatives of the outputs of the simulation with respect to its inputs and parameters. Here, we use the differ- entiable PDE solver from the phiflow framework of Holl et al. (2019) in combination with Tensor- flow to obtain the non-reactive flow solver and reactive flow solver solutions. The marker-and method (Harlow & Welch, 1965) is adopted to represent temperature, pressure, density, species mass fraction hields in a centered grid, and velocities in a staggered grid.
purely data-driven model (PDD) is used as a baseline. employs neural network model to learn the complete flow states Po( given an input flow state o% where Φḥ = [T. y,; Yol, as shown in figure 2 (BI). The new state predicted by the trained neural network model is Φ C(of ()) For the hybrid NN-PDE as well as PDD, the neural network part of the predictor block in figure 2 consists of fully convolutional neural network model. For both, we experimented with both the ResNet (He et al. 2016) and UNet (Ronneberger et al.) 2015) architectures. We found that the ResNet performed the best for the PDD) setting, while for the hybrid NN-PDE approach the UNet performed consistently better. Hence, the following results will use a ResNet for PDD models, and a UNet for the NN-PDE hybrids. Additional details of the neural network architectures are provided in the appendix. The network receives the input states as shown in figure 2 The output of the neural network model is used to obtain the corrected state 0:+1 as specified above. We constrain the mass fraction fields Y& to contain physical values in the range Y E [0, 0.05) and Y, E [o, 1]. All models are trained for 100 epochs with a batch size of 3 and learning rate of 0.0001.
We additionally compare against the Fourier neural operator (FNO) of [i et al. (2020a) as an example of state-of-the-art neural operator method. The new state predicted by the trained model is given by Φ6 C(o)10) where Sه [T, Y,, Yo] and C(o) represents the Fourier neural operator. All three approaches use an L2 based loss that is evaluated for m steps as C(0)= = E Ed=1x,,, ll Po(
4 NUMERICAL EXPERIMENTS
We consider case of planar 2D premixed methane-air air flame propagating in quiescent mixture (Planar-v0) and two cases of the transient evolution of initially planar laminar premixed flame into a Bunsen-type flame under different inlet velocity conditions Bunsen, nonUniform- Bunsen). We obtain the target data by considering the source terms as defined by equation 3 with the parameters mentioned in section 3.1
Planar-v0 For the most basic case, the planar 2D flame model setup, we consider the reacting
Navier-Stokes equations described in section 3 with zero inlet velocity i.e. ur 0, uyy 0. We consider a square domain of size 0.05 m x 0.05 m with 32 x 32 resolution and closed boundary conditions. The simulation is initialized using a steep transition between a premixed methane-air mixture and burnt gases. Our training data consists of 6 simulations of 300 steps created by varying the equivalence ratio E. It represents the stoichiometric mixture (1º) of fuel Y, and oxidizer Y, mass fractions, i.e., E = o y and thus fundamentally influences the dynamics of the chemical reaction. For the training data we use Enain {1.0, 0.9, 0.8, 0.7, 0.6}, while the test dataset contains Frest (0.95, 0.85, 0.75, 0.65).
In contrast to the planar case, the premixed methane-aid air mixture is now fed with a constant inlet velocity. The boundary conditions upstream, at y = 0. are (uz, uy)a. (0, k) where K is a n-dimensional vector with constant amplitude. The target simulation contains heat release rate term, and in this case the initial temperature field evolves into different A-shaped flames at the end of the 300th time step. Training and testing datasets are created by varying the equivalence ratio and inlet velocity amplitude U: Emain = { {1.0, 0.9, 0.8) and Utrain = (0.45, 0.4, 0.3). The test dataset uses E (0.95, 0.85) Uwa f0.43, 0.375, 0.325]. The length and temperature of the flame significantly vary depending on the inlet velocity and equivalence ratio provided. Further details on the boundary conditions and training data visualizations can be found in the appendix.
uniform-Bunne
nonUniform-Bunsen As a third case we consider the transient evolution of a premixed methane- air flame with non-uniform inlet velocity. The boundary conditions upstream, at y 0,
5

---

Under review as a conference paper at ICLR 2023
PND
(33:730
ICE Hybrid MN-zE
Seound Tush
221
2254
2746
2108
1008
158
SCE
0.00
0.04
9.02
306
3.00
3.54
9:00
3.54
335
8.05
1on1
Figure 3: ID cut of the planar flame simulation over 300 steps. The initial state is plotted in red, target state in green. Hybrid NN-PDE approach predicts physically accurate results over longer rollouts with correct flame temperature and relative displacement of flame front across different equivalence ratios.
Table I:
Mean and standard deviation of errors over all time steps of testsets. The Hybrid NN-
PDE approach outperforms all other baselines considered.
Baseline FNO
Baseline NN
Hybrid NN-PDE
Hybrid NN-PDE-dt
Planar-vO
8.27 + 5.51%
6.33 3.05%
1.40 + 0.65%
1.21 0.39%
MAPE
uniform- mniform-Bunsen
15.57 = 8.72%
7.58 + 3.73%
0.72 +: 0.37%
1.11 = 0.47%
12.30 + 7.98%
12.48 + 11.31%
2.04 + 1.39%
2.46 + 1.61%
nonUniform 000
3,23 +: 3.76%
4.14 4.99%
Planar-v0
88316 + 124394
34491 43966
1122 1300
292
MSE
uniform-Bunsen Bunsen
220817 167024
72563 + 56919
721 + 1007
1267 + 1622
nonUniform- mmunnifomm-Buseen22
184860 185694
130219 + 142094
9647 + 13903
17569 + 24392
non Bunsenl
23293 + 37451
26862 + 47639
(Ux: Myya.v=0 x)) where is n-dimensional vector whose elements are each sampled from a uniform distribution from [0.2, 0.65]. We experiment with two different domain sizes, 32 X 32 and 100 x 100 Moonnnifrmm-Bnneen1000
shown in figure 4 The larger domain size used is closer to the practical reactive flow domain used in CFD applications (Jaensch et al. 2017) with highly resolved flame. These inlet velocity conditions generate com- plex flame shapes which increase the difficulty of the prediction problem. We consider simulation sequences with 500 steps as it takes longer time for the flame reach the steady-state solution.
SeUnniforr-Bunnsenn..
RESULTS
We demonstrate the capabilities of the proposed learning approach represent the complete PDE description with the aforementioned cases of increasing difficulty. We also study its ability to gener- alize unseen operating conditions such as equivalence ratios, simultaneous variations inlet velocity and equivalence ratio, and non-uniform inlet velocity profiles. The source code of our work will be published upon acceptance. As baselines, we compare against a purely data-driven approach; a neural network model with exactly the same thek-ahead steps, and the Fourier neural operator of Li et al. (2020a) that likewise includes m look-ahead steps as discussed section
constant
5.1 QUALITATIVE AND QUANTITATIVE ANALYSIS
Table compares the mean absolute percentage errors (MAPE) and mean squared errors (MSE) of temperature field for all the cases discussed in section 4 For Planar-vO, the FNO and PDD approaches yield large errors with MAPE 8.27% and 6.33%, respectively. On the other hand, the hybrid NN-PDE model trained with 32 look-ahead steps reduces the error to 1.4%, and thus performs significantly better than the two baselines. This behavior is visualized in figure 3 with ID transverse cut of the simulation domain over 300 time-steps. The hybrid NN-PDE approach successfully captures the propagation of the flame.
In figure (E), also compare two important physical guantities, the flame temperature and relative displacement of the flame front, across different equivalence ratios. It can be seen that the hybrid NN-PDE model (blue circles) accurately predicts the flame temperature for different test cases (solid

---

Under review as a conference paper at ICLR 2023
Conditon
Hyord
Ground Trush
Dffference
64)
QIT- 50
-11-100 100
101- 10:1 220
200
meuntio
100
restnttomn EUrsen
333 INIT 200
ONIT DC3T 400
140 550
WIT 500
Figure 4: Left Inlet velocity profile used. (a-d) Temperature field prediction by hybrid NN-PDE approach over different steps given the inlet velocity profile. (e) Ground truth data. Right : Differ- ence between ground truth data and hybrid NN-PDE output at last snapshot. Top to bottom: 32 x 32 resolution cases of (A) uniform-Bunsen, (B) nonUniform- Bunnnfrrm-B and (C) 100 x 100 testcase Siaiifoorm-unseen
represents the instantaneous MAPE.
blue line). The relative displacement of the flame front is computed as 12.-2ol/12;--20 where 2, is the position along the flame normal at time t on the 1200 isotherm the ground truth simulation, and ž denotes the predicted position. For all test cases, the hybrid approach accurately predicts the flame displacement over 300 steps, while the other approaches yield significant errors.
For uniform- Bunsen case, amongst the baseline models, PDD performs better with error of 7.58% compared to FNO model which yields an error of 15.57%. A neural network model combined with incomplete PDE solver for 32 look-ahead steps yields significantly lower error of 0.72% as shown in table [ This means, the hybrid approach reduces the errors by a factor of 10 over the baselines considered. Figure ((A) shows the temporal evolution of the hybrid NN-PDE approach predicted over 300 time steps for the muiform-Bunse case. It shows the results for a test case with U 0.4375 and 0.95. It predicts symmetric flame with accurate flame height and achieves very low instantaneous MAPE of 0.73% at t=300, when compared with the ground truth data.
Next, we study complex scenario of non Monnniform-Bunne Bunsen flame with 32 32 resolution. The hy- brid NN-PDE approach outperforms the PDD approach and FNO with an improvement of ~ 80%. The MSE values show this trend even more clearly. The large standard deviations of the MSE num- bers indicate that the predictions made by the baseline approaches contain substantial deviations from target values. Important to highlight is that, despite using the same training data and similar neural network architecture, with same look-ahead steps, the hybrid approach outperforms the PDD approach thanks to its learned collaboration with the incomplete PDE solver. accurately repro- duces the complete PDE behavior. Figure (B) highlights this with visualization of the hybrid NN-PDE model predictions for nonUniform-Bunsent the flame simulation over long roll-outs of 500 steps and achieves the complex flame shape with low, instantaneous MAPE of 2.38%. Finally. we showcase the ability of hybrid approach to predict the temporal evolution of highly resolved flames with the Sniform-Buuseee 100 scenario. Despite the increased complexity of the larger resolution. it achieves a very good overall MAPE of 3.23% over 12 test cases of 500 simulation steps. Figure 4 CC) shows an example of physically accurate predic- tions made by the hybrid NN-PDE model. We omit the evaluation of baselines for high resolution cases as they do not succeed to model the flame dynamics for low resolution cases.
case. The trained model accurately predicts
Effect of training dataset size Figure 5 (A) shows that increasing the number of training sets has little or no effect on the prediction capabilities the PDD models for Unniforo--unneennn case. The hybrid model with 12 training sets achieves MAPE of 12.49 + 4.17%, an improvement of 38% over the PDD model with 32 training sets. This result strengthens the hypothesis that integrating the incomplete PDE solver into the neural network training yields learning signal that fundamentally differs from that produced by training with precomputed data. The purely data-driven models cannot in the presence of large amounts of data.
achieve the same level of accuracy
Generalization to incorrect PDE parameters The proposed hybrid NN-PDE model is capable of completing the PDE description even if the underlying incomplete PDE solver does not have access

---

Under review as a conference paper at ICLR 2023
nonUnifom Bunssen32
nontinifomm Bunsen32
12
16
24
32 Hybrid-12
Me
dt. 9.04
[A] No. of simulations training sot
[BI Data timestep
Figure 5: (A) Effect of increasing training dataset size for PDD approach, over fixed testsets, is compared with equivalent (trained with same look-ahead steps m = 2) hybrid NN-PDE model. (B) Effect of temporal coarsening on PDD models trained with 32 look-ahead steps.
0.30
TE
Figure 6: Left Constant inlet velocity profile used. (a-d) uniform-Bunse case trained and inferred double time-step At, using hybrid NN-PDE approach. It achieves same flame shape at 150 for which the complete PDE solver requires 300 time-steps.
to the correct parameters, referred to as 'incomplete, incorrect PDE". For this test we assume that the values of т, Da, À in equation 2 are unknown and set to incorrect values. For the nonUniform- Bunsen32 case, the hybrid model with the incomplete, incorrect solver achieves an overall MAPE of 2.48 + 1.20 %, slight increase over the MAPE of 2.04 + 1.39 % for the hybrid model with the incomplete, correct PDE. Additional details can be seen in appendix
5.2 RELAXING TEMPORAL STIFFNESS THE PDE SOLVER
Traditionally, the source terms and reaction rate terms involved in modeling the fast chemistry of reacting flows require the use of very small time-steps simulations due to the stiffness of the chemical mechanisms. The incomplete PDE solver used in the hybrid NN-PDE approach, does not contain the source terms and reaction rate terms. Therefore, the time scales associated with the chemical reactions play less important role in maintaining numerical stability, and it becomes possible for the solver to employ larger time steps. To illustrate this advantage, we train the neural network model with time-step that is twice large the largest time-step Ate required for the complete PDE solver to yield a stable result.
We mimic the setup described in section 4 but now the incomplete PDE solver uses time-step of 24 which is too large for the complete PDE solver by itself to converge to solution. Figure b shown as first example, indicates that the results obtained from the models trained with larger time-step at t {25, 50, 100, 150} are in good agreement with the target data for the difficult -unnsnnmBBunenn case. The flame dynamics are predicted accurately while using less simulation steps. For the uniform-B Bunsen case, the hybrid approach takes 8, 23 2 0.011s Is to infer one simulation run, whereas the complete PDE solver requires 15.21 + 0.003s. Similar performance gains are observed across the other cases studied. Note that for the previous cases, the trained model incurs negligible runtime overhead. In these cases, even for costly runs with high resolutions incomplete PDE solver would not converge to the solutions of the complete description.
The last column of table (Hybrid NN-PDE-dt) summarizes the errors the large time-step ap- proach for all four reactive cases considered. Despite effectively doubled computational per- formance, this model achieves similar errors to the hybrid NN-PDE approach. This highlights the capabilities of learned, hybrid PDE solvers, which produce these solutions without the stability problems exhibited by the complete PDE solver, while at the same time being more accurate than pure data-drives predictions.
The PDD approach is not restricted by the time-step of the PDE solver. Therefore, we investigate the performance of the PDD approach the monuniform-Buneent case with larger time-steps as
8

---

Under review as a conference paper at ICLR 2023
Shape
You
50
10
10
Temperature field predictions achieved by the neural network model T:NN from given initial flame shape To to achieve target flame shape T, by controlling flow velocity.
Figure 7:
shown in the figure 5 (B). PDD achieves higher MAPE of 22.44 % for 8At, setup as compared to the MAPE of 12.48 % for Ate setup. The PDD approach does not predict the dynamics accurately for any of the larger time-steps considered. Additional visualizations can be found in appendix
5.3 FLAME SHAPE CONTROL
We lastly demonstrate the potential of our approach with problem from multi-physics system control: we test the joint applicability of a neural network with reactive flow solver to obtain a desired flame shape. We use a predictor-corrector approach (Holl et al.) 2019) with differentiable physics loss function and control sequence refinement for non uninorrn-Bussee case with 32 x 32 resolution. The learning objective to arrive at a target temperature field with the desired flame shape from an arbitrary, given initial configuration by controlling the velocity field. We assume that the temperature and mass-fraction fields are observable and only the velocity field is controllable. Figure 7 (a-e) shows the transition achieved and compares the target flame shape obtained using the neural network model with the target flame shape (T,) for a test case. Figure (f.g) compares the flame shape obtained for additional test cases. The neural network model is able to control the velocity of fuel-air mixture flow to obtain the desired flame shape. To quantify these results, the learned model achieves MAPE of 1.87 + 0.73% over 50 test cases considered. achieves an improvement of 52% compared to a baseline model without predictor network for the long term prediction of temperature fields. This baseline model has MAPE of 3.91 2.19%. This case highlights the capabilities of differentiable multi-physie solvers, and that our full neural network model is able reliably control the complex physics of the reactive flow over long time spans.
6 CONCLUSIONS
These results demonstrate that neural networks can be employed for learning the completion of par- tial PDE solvers in the context of a multi-physics system. The trained neural network model works alongside a partial solver to accurately account for and predict the effects of unknown physics. The ability of the presented hybrid NN-PDE approach to predict the long-tew meg-term temporal evolution of multi-physics systems was demonstrated on the specific case of reacting flow in different configu- rations. When compared with purely data-driven approach and state-of-the-arr FNO approach, the qualitative and quantitative performance of the hybrid NN-PDE model is found to be clearly supe- rior. The incomplete PDE description helps the neural network model recover the target simulation with a significantly improved accuracy. This hybrid NN-PDE model is able to predict the correct evolution of the important features of the multi-physics system (in our case, the flame interface and flame shape) for longer simulation steps in all scenarios and is able to generalize to other (initial and boundary) conditions of the system. This is demonstrated by variations and inlet velocity forcing. We also show that such hybrid approach may have the added benefits of allowing to relax numerical constraints linked the potential stiffness of the unknown physics.
the equivalence ratio
Our work represents a stepping stone for numerous avenues of future work, e.g., the proposed hybrid NN-PDE solver could be further utilized to predict and control the dynamics of other tightly coupled systems (Levy) 1999 Dowell & Hall 2001). Here, the inclusion of transfer learning techniques would pose an interesting avenue for future work. We note that the proposed approach does not seek replace the classical physical simulation approaches. However, it is important step in the direction of harnessing the capabilities of neural network models using the knowledge of partial differential equations for complex multi-physical systems.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Ekhi Ajuria Illarramendi, Antonio Alguacil, Michaäl Bauerheim, Antony Misdariis, Benedicte Cuenot, and Emmanuel Benazera. Towards an hybrid computational strategy based on deep learn- ing for incompressible flows. In AIAA AVIATION 2020 FORUM, pp. 3058, 2020.
Yohai Bar-Sinai, Stephan Hoyer, Jason Hickey, and Michael P Brenner. Learning data-driven dis- cretizations for partial differential equations. Proceedings of the National Academy of Sciences, 116(31):15344-15544 2019.
Filipe de Avila Belbute-Peres, Thomas Economon, and Zico Kolter. Combining differentiable pde solvers and graph neural networks for fluid flow prediction. In International Conference on Ma- chine Learning, pp. 2402-2411. PMLR, 2020.
Kaushik Bhattacharya, Bamdad Hosseini, Nikola B Kovachki, and Andrew M Stuart. Model reduc- tion and neural networks for parametric pdes. The SMAI journal of computational mathematics, 1:721-157, 2021.
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural ordinary dif- ferential equations. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pp. 6572-6583, 2018.
Earl H Dowell and Kenneth C Hall. Modeling of fluid-structure interaction. Annual review of fluid mechanics, 33:445, 2001.
Gideon Dresdner, Dmitrii Kochkov, Peter Norgaard, Leonardo Zepeda-Núñez, Jamie Smith, Michael P Brenner, and Stephan Hoyer. Learning to correct spectral methods for simulating turbulent flows. arXiv preprint urXiv:2207.00556, 2022.
Kai Fukami, Koji Fukagata, and Kunihiko Taira. Super-resolution reconstruction of turbulent flows with machine learning. Journal of Fluid Mechunics, 870:106-120, 2019.
Vadim N Gamezo, Alexei M Khokhlov, Elaine S Oran, Almadena Y Chtchelkanova, and Robert O Rosenberg. Thermonuclear supernovae: Simulations the deflagration stage and their implica- tions. Science, 299(5603):77-81, 2003.
Samuel Greydanus, Misko Dzamba, and Jason Yosinski. Hamiltonian neural networks. Advances in Neural Information Processing Systems, 32:15379-15389, 2019.
Jiequn Han, Arnulf Jentzen, and E Weinan. Solving high-dimensional partial differential equations using deep learning. Proceedings of the National Academy of Sciences, 115(34):8505-852) 2018.
Francis H Harlow and Eddie Welch. Numerical calculation of time-dependent viscous incompress- ible flow of fluid with free surface. The physics of fluids, 8(12):2182 2189, 1965.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Philipp Holl, Vladlen Koltun, and Nils Thuerey. Learning control pdes with differentiable physics. In ICLR, 2019.
Mark Z Jacobson. Fundamentals of atmospheric modeling. Cambridge university press, 1999.
Jaensch, M Merk, EA Gopalakrishnan, S Bomberg, Emmert, RI Sujith, and W Polifke. Hybrid CFD/low-order modeling of nonlinear thermoacoustic oscillations. Proceedings of the Combus- tion Institute, 36(3):3827-388 30333422-33,,110 2017.
Eugenia Kalnay. Atmosphrric modeling, data assimilation and predictability. Cambridge university press, 2003.
Byungsoo Kim. Vinicius C Azevedo, Nils Thuerey, Theodore Kim, Markus Gross, and Barbara Solenthaler. Deep fluids: A generative network for parameterized fluid simulations. In Computer Graphics Forum, pp. 59-70. Wiley Online Library, 2019.
10

---

Under review as a conference paper at ICLR 2023
Omar M Knio, Habib N Najm, and Peter S Wyckoff. A semi-implici numerical scheme for reacting flow: li. stiff, operator-splic formulation. Journal of Computational Physics, 154(2))428-4555 1999.
Dmitrii Kochkov, Jamie A Smith, Ayya Alieva, Qing Wang, Michael P Brenner, and Stephan Hoyer. Machine learning accelerated computational fluid dynamics. Proceedings of the National Academy of Sciences, 1 118(21), 2021.
Isaac E Lagaris, Aristidis Likas, and Dimitrios I Fotiadis. Artificial neural networks for solving ordinary and partial differential equations. IEEE transactions on neural networks, 95):987-11000, 1998.
Corentin J Lapeyre, Antony Misdariis, Nicolas Cazard, Denis Veynante, and Thierry Poinsot. Train- ing convolutional neural networks to estimate turbulent sub-grid scale reaction rates. Combustion and Flame, 203:255-264, 2019.
Salomon Levy. Two-phase flow in complex systems. John Wiley & Sons, 1999.
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, An- drew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010. 2020a.
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, An- drew Stuart, and Anima Anandkumar. Neural operator: Graph kernel network for partial differ- ential equations. arXiv preprint arXiv:2003.08 03485, 2020b.
Tim Lieuwen. Unsteady Combustor Physics. Cambridge University Press, 2012.
Bjöm List, Li-Wei Chen, and Nils Thuerey. Learned turbulence modelling with differentiable fluid solvers, arXiv preprint arXiv:2202.06988 2022.
Zichao Long, Yiping Lu, Xianzhong Ma, and Bin Dong. Pde-net: Learning pdes from data. In International Conference Machine Learning, pp. 3208-3216. PMLR, 2018.
Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via deeponet based the universal approximation theorem of operators. Nature Machine Intelligence. 3(3):218 229, 2021.
Tohru Mitani. Propagation velocities of two-reactant flames. Combustion Science and Technology. 21(3-4):155-17,, 1980.
Habib N Najm, Peter S Wyckoff, and Omar M Knio. A semi-implicit numerical scheme for reacting flow: I. stiff chemistry. Journal of Computational Physics, 143(2):381-403 1998.
Ravi G Patel, Nathaniel A Trask, Mitchell A Wood, and Eric C Cyr. A physics-informed operator gression framework for extracting data-driven continuum models. Computer Methods in Applied Mechanics and Engineering, 373: 13500, 2021.
Jaideep Pathak, Mustafa Mustafa, Karthik Kashinath, Emmanuel Motheau, Thorsten Kurth, and Mareus Day. Using machine learning to augment coarse- -ridd computational fluid dynamics sim- ulations. arXiv preprint arXiv:2010.00072. 2020.
Thierry Poinsot and Denis Veynante. Theoretical and numerical combustion. RT Edwards, Inc., 2005.
Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686-707, 2019.
Chae M Rhie and Wei-Liang Chow. Numerical study of the turbulent flow past an airfoil with trailing edge separation. AIAA journal, 21(11):1525-1532 1983.

---

Under review as a conference paper at ICLR 2023
David Rolnick, Priya L Donti, Lynn H Kaack, Kelly Kochanski, Alexandre Lacoste, Kris Sankaran, Andrew Slavin Ross, Nikola Milojevic-Dupon Dupont, Natasha Jaques, Anna Waldman-Brood Brown, et al. Tackling climate change with machine learning. ACM Computing Surveys (CSUR), 55(2):1-96, 2022.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedi- cal image segmentation. In International Conference on Medical image computing and computer- assisted intervention, pp. 234-241. Springer. 2015.
Justin Sirignano, Jonathan F MacArt, and Jonathan B Freund. Dpm: A deep learning pde augmen- tation method with application to large-eddy simulation. Journal of Computational Physics, 423: 109811, 2020.
Kim Stachenfeld, Drummond Buschman Fielding, Dmitrii Kochkov, Miles Cranmer, Tobias Pfaff, Jonathan Godwin, Can Cui, Shirley Ho, Peter Battaglia, and Alvaro Sanchez-Gonzalez. Learned simulators for turbulence. In International Conference on Learning Representations, 2022. URL htttp:://openzeet
net wwrumorummiioo Fod
Naoya Takeishi and Alexandros Kalousis. Physics-integrattd variational autoencoders for robust and interpretable generative modeling. Advances in Neural Information Processing Systems, 34: 14809-14821, 2021.
Nils Thuerey, Konstantin Weißenow, Lukas Prantl, and Xiangyu Hu. Deep learning methods for reynolds-averaged navier-stokes simulations airfoil flows. AIAA Journal, 58(1):25-36, 2020.
Jonathan Tompson, Kristofer Schlachter, Pablo Sprechmann, and Ken Perlin. Accelerating eulerian fluid simulation with convolutional networks. In International Conference Machine Learning, pp. 3424 3433. PMLR, 2017.
Kiwon Um. Robert Brand, Fei Yun, Philipp Holl, and Nils Thuerey. Solver-in-the-lopp Learning from differentiable physics to interact with iterative pde-solvers. In Neural Information Process- ing Systems (NeurIPS), volume 33, pp. 6111-6122, 2020.
Rui Wang, Karthik Kashinath, Mustafa Mustafa, Adrian Albert, and Rose Yu. Towards physics- informed deep learning for turbulent flow prediction. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1457-1466, 2020.
Gerhard Wanner and Ernst Hairer. Solving ordinary differential equations II, volume 375. Springer
Berlin Heidelberg New York, 1996.
Adam A Wheeler, William J Boettinger, and Geoffrey B McFadden. Phase-field model for isother- mal phase transitions in binary alloys. Physical Review A, 45(10):7424, 1992.
Forman A Williams. Combustion theory, (1985). Cummings Publ. Co, 31, 1985.
Yuan Yin, Vincent LE GUEN, Jérémie DONA, Emmanuel de Bezenac, Ibrahim Ayed, Nicolas THOME, and patrick gallinari. Augmenting physical models with deep networks for complex dynamics forecasting. In International Conference on Learning Representations, 2021.
Ali Girayhan Özbay, Arash Hamzehloo, Sylvain Laizet, Panagiotis Tzirakis, Georgios Rizos, and Bjöm Schuller. Poisson cnn: Convolutional neural networks for the solution of the poisson equa- tion on cartesian mesh. Data- Centric Engineering, 2:e6, 2021. doi: 1110///cc.20211111
12

---

Under review as a conference paper at ICLR 2023
APPENDIX
A NEURAL NETWORK ARCHITECTURE
All results provided in the main paper use a fully convolutional neural network. Different input flow field quantities have different value ranges of the data e.g. T E [400, 2500], Y, e [0, 0.05). There- fore, the input data provided to the neural network model is normalized using mean and standard deviation. The output of neural network model is then transformed back using the same mean and standard deviation. When predicting correction of Y& fields, the output of the neural network is constrained to provide values the range Y, E [0, 0.055,, Y, E [0, 1]. In the hybrid NN-PDE approach, the output of the neural network model is provided to the incomplete PDE solver as an input, as shown figure 2 Each network is trained for 100 epochs with learning rate of 0.0001. An L2 based loss function used defined in section 3.3
We additionally experimented with ResNet and UNet architectures for the neural network part of the purely data driven and hybrid NN-PDE approach discussed in the paper. Table [able2[provide details of the convolutional blocks, layers of the network, and activation function. The UNet32 architecture with 2 layers is used for 32 x 32 cases and the UNet100 architecture with 3 layers is used for 100 x 100 case discussed in the paper. Table shows the comparison between MAPE achieved by ResNet vs. UNet architecture for the purely data-driven and the hybrid NN-PDE approach for different cases considered in the paper. For the hybrid NN-PDE approach with 32 x 32 resolution cases, ResNet and UNet achieve comparable performance with the UNet yielding the lowest error. The main difference arises in the 100 x 100 resolution case. The ResNet fails to converge during training, leading to high test error. On the contrary, the UNet achieves low training and testing errors. The main advantage of UNet comes from transforming high resolution input to low resolution representation and reconstructing it back to the high resolution output. Therefore, we choose the UNet architecture for the hybrid NN- NN-PDE approach. For the purely data-driven approach ResNet better for out of 3 cases. Specifically for the nonUniform- -unnmm-nn case, UNet performs very poorly for the PDD approach. Therefore, report ResNet errors for the PDD approach throughout the paper. Figure shows the details of the predictor block for PDD and hybrid NN-PDE approach.
found to be
Table 2: Details of various neural network architectures used in this paper.
plyper-parameters
ResNet
UNet32
UNet100
Kernel size
Latent size
32
Activation
LeakyReLa MSE
LeakyReLu MSE
ReLu
Loss
MSE
# ResBlooks
UNet layers
3
CNN stack depth
2
2
Base latent size
16
Spacial down-sample by layer
2
2
latent sizes
(32)
(32,64)
# trainable parameters
261,953
136,227
362,371
Table 3: Mean and standard deviation of errors different architectures for purely data-driven and hybrid NN-PDE approach.
PDD ResNet
PDD UNet
Hybrid NN-PDE ResNet
Hybrid NN-PDE
UNel
Planar-v()
6.33 3.05%
7.09 4.40%
1.62 + 0.44%
1.40 0.65%
MAPE
uniform- Bunsen
uniform-Bunsen
7.58
3.73%
2.87 + 1.53%
2.01 1 0.99%
= 0.37%
nonUniform Bunsen32
12.48 11.31%
19.19 + 14.92%
3.25 + 2.35%
2.04 1.39%
non nonUniform- Bartrrm--uuennllnn 100
15.68 + 10.44%
3.23 3.76%
13

---

Under review as a conference paper at ICLR 2023
Table 4: Details of the boundary conditions used for the planar flame and various cases of Bunsen-type flames. The unifrrm-Bunsee case is obtained with constant and nonUniform-
Bunsen case is obtained with Rx discussed in section 4
Planar flame
Bunsen-type flame
field
inlet
left and right wall
field Neumann BC
inlet
left and right wall
T
400 K
800
Neumann BC
By
0
My
no-slip slip
Ux
0
U2
P
101325
Neumann BC
P
101325
Neumann BC
slip
YG
slip
1+ #143.700
W
1+ L143..000 WN2)
2000
160
PAT
5-008
C-00
64
O
Figure 8: Instances of (A): incomplete PDE solver; (B): complete PDE solver for different operating conditions. (C): Snapshot of complete PDE solver 300 for training and testing datasets of uniform- uniform-Bunsen case. (D): Snapshot of complete PDE solver at t 500 for all training and testing datasets of SeLUniffrr-Buunnenni. case.
B DATA GENERATION
To generate input and target data for training, simulate the temperature T, mass fractions Y,, Y. and velocity Ug fields of all flames under study. Table summarizes the boundary condi- tions applied for the planar and Bunsen-type flame conditions are used for the velocity field My to obtain the A-shaped flames.
diseussed in the paper. No-slip boundary
We consider variations in equivalence ratio E and inlet velocity amplitude uy to obtain training and test datasets. Figure g (A) shows the temperature field evolution of the incomplete PDE solver at different time instances for 3 different operating conditions for the uniform-Bunsed case. Figure (B) shows the corresponding target training data for the uniform-Bunsen case. All the simulations are run for 300 steps. It shows the difference between non-reactive and reactive flow solver simu- lations for different operating conditions over 300 time-steps. Figure g (C) shows the last snapshot 300) of training simulations and 6 testing datasets with operating parameters interpolated be- tween the training data parameters for uniform-Bunn Bunsen case. These datasets are obtained by varying equivalence ratio and constant inlet velocity as discussed on the equivalence ratio used and flame height depends on the inlet velocity amplitude. Figure (D) showcases the snapshots training and testing dataset at t = 500 for nonUniform -arrmm--Bnse222 case. Non-uniform variations in inlet velocity profile leads to different complex flame shapes. We use 12 datasets with 500 simulation steps to train the models and test it on test cases shown in figure (D).
section Flame temperature depends
14

---

Under review as a conference paper at ICLR 2023
Trum
Complete POR
2020
bomgdds ine
902
602
Figure 9: Generalization to incorrect PDE parameters (A) visualization of differences in temperature
fields due incorrect parameters in incomplete PDE description. (B) Modified hybrid NN-PDE model, which combines the incomplete, incorrect PDE solver with neural network model is able to recover solutions of complete, correct PDE.
PlanarivO
(8) uniform- Bunsen
nonüniform Bunsen32 (D) nonüniform Bunsen100
20
20
mm2
15
15-
15
15-
mm
10
10
10
10-
5
200
100
300
200
400
Simulation steps
steps
Simulation steps
Simulation steps
Figure 10: Effect of longer look-ahead steps. MAPE of temperature field predictions by hybrid NN-PDE model, over all testsets. Models trained with higher look-ahead steps accurately predict the temporal evolution of dynamics for longer duration across all cases considered.
C ABLATION STUDY
Effect of training dataset size We study the effect of training dataset size on the accuracy of PDD predictions. We train PDD models with different numbers of simulations in the training dataset. Each simulation contains 500 time-steps. Figure 5 (A) shows the MAPE of PDD models trained with {4, 8, 12, 16, 24,32] 32} training datasets, over fixed testset. We compare the performance of these PDD models with an equivalent (trained with same look-ahead steps m = 2) hybrid NN-PDE model, trained with 12 simulation sets. Figure (A) shows increasing the number of training sets has little or no effect on the prediction capabilities of the PDD models for nonUniform- -urorm--Bnnen2222 case. The hybrid model with 12 training sets achieves MAPE of 12.49 + 4.17%, an improvement of 38% over the PDD model with 32 training sets. The purely data-driven models cannot achieve the same level of accuracy even in the presence of large amounts of data.
Generalization to incorrect PDE parameters The proposed hybrid NN-PDE model is capable of completing the PDE description even if the underlying incomplete PDE solver has incorrect parame- ters. We refer to the incomplete solver with incorrect parameters as 'incomplete, incorrect PDE'. We experiment with modified hybrid NN-PDE approach wherein we combine an incomplete, incorrect PDE solver with a neural network model. We assume that the known values of the incomplete PDE parameters equation 2 strain rate tensor (r), the diffusion coefficient of species k (Dx) and mix- ture thermal conductivity (A), are incorrect. Figure (A) shows the difference between temperature field evolution of the incomplete PDE solver with correct parameters ([7, Dr. A] = 10.1, 0.1, 0.1)) and incorrect parameters ([r, Dx. A] [0.05. 0.05, 0.05)) at different time-steps. The hybrid NN- PDE combines this incomplete, incorrect PDE solver with the neural network model to obtain the solutions of the complete PDE solver with correct parameters. Figure 9 (B) compares the flame dynamics predicted by this hybrid NN-PDE model with that of the complete, correct PDE solver. A good match is observed over various test cases. The hybrid model with incomplete, incorrect solver achieves an overall MAPE of 2.48 + 1 1.20 %, compared to the MAPE of 2.04 + 1.39 % for hybrid model with incomplete, correct PDE.
15

---

Under review as a conference paper at ICLR 2023
(A) Planar-vO
(B) uniform-Bunse
HDD Hybrid
MAPE
MAPE
(C) Beatunfform-Bunseenn
10
MAPE
10 11 12
Testset number
Figure 11:
Bar plot of MAPE of temperature field predictions by FNO, PDD and hybrid NN-PDE model time-step Ate, for different testcases.
Effect of longer look-ahead steps We evaluate the effect of varying look-ahead steps the perfor-
mance of the hybrid model. We show the comparison of models trained with m = (2, 4, 8, 16, 32) for different cases in figure TO Models with smaller (2) do not learn accurately correct the fields over long time and quickly diverge from the target simulation. Using larger m improves the quality of prediction drastically
the model Iearns the correction via the gradients over longer simulation steps. For Planar-vO and uniform- Bunsen cases, iterating the NN and PDE solver for 32 time- steps improves the accuracy by 81.7% and 94% respectively compared the m model. For non nonUniform unnnforr-Bunseen32 and nonUniform-Bunn 100, the model performance improves by 84.2% and 70.9% respectively, by using m 32 instead of m = 2 model.
Effect of temporal coarsening Figure 11 and table compare the MAPE of temperature field pre- dictions at time-step Ale, for all three scenarios. As seen from the figure [T] the hybrid NN-PDE approach consistently performs better than both the baselines for all three scenarios considered. Higher standard deviations Table indicate the large differences in the prediction of flame tem- peratures for baseline approaches. The multi-physice systems study provide substantially more difficult environment than regular fluids: the chemical reactions are numerically very stiff, and the resulting dynamic interactions difficult to capture by numerical solvers. To further illustrate the temporal behavior of the simulations, we perform additional experiments with the baseline ap- proaches using twice the time-step size of the complete solver (2At.). As shown in figure I2 for the Planar-v0 and uniform-Bunn Bunsen case, PDD catches up with the hybrid approach whereas the PDD approach completely fails predict the dynamics of the nonUniform- -aaomm--nnee32222 case. We further investigate the non Uniforr-BBuneen32 case with larger time-steps (2 times, 4 times and 8 times) as shown in 5 the PDD model. The PDD approach does predict the dynamics accurately for any of the larger time-steps considered. Figure 13 visualize these results over different time-steps for one of the test cases. The predictions made by FNO, PDD and hybrid NN-PDE model for non nonUniform Bunsen32 case at twice the time-step of 2Atz. are compared with the ground truth solutions coming from the complete PDE solver at time-step of At.. FNO and PDD (upper two rows) fail to recover the cor- rect flame shapes over 250 simulation steps. The hybrid NN-PDE model (second row from below) predicts the flame shape accurately, thus relaxing the temporal stiffness of the complete PDE solver.
shows MAPE of the predictions over all the testsets and simulation steps by
/niform-Buusee322
D ADDITIONAL RESULTS
In this section, present additional visualizations of all reactive flow scenarios considered in sec- tion 4 for different test cases. Additionally, we discuss the results the mass fraction fields we obtained. Table summarizes MSE values of fuel (Yy) and oxidizer (Y,) mass fraction fields for
16

---

Under review as a conference paper at ICLR 2023
(A) Planar-vO
(B) mniform-Bunseer
Hybrid
PDD
MAPE
MAPE
(C) nonuniform-Bunseee
10
MAPE
100
11
12
Testset number
Figure 12: Bar plot of MAPE of temperature field predictions by FNO, PDD and hybrid NN-PDE model time-step 24t for different testcases.
FNO
2000
2000
PDD
1600
Hyerd
1200
50
100
150
200
225
1-
250
1006
Ground
Irush
Comparison between different approaches time-step 2 from top bottom FNO, PDD, hybrid NN-PDE for monnnffrrnBunsse:32 test case. Hybrid approach predictions accurately match with the ground truth data over long roll-outs.
Figure 13:
different baselines and the hybrid NN-PDE approach. We observe similar performance the tem- perature field predictions. The hybrid NN-PDE model consistently outperforms the FNO and PDD approaches for all the scenarios considered. The hybrid NN-PDE model is able to simultaneously correct temperature and mass fraction fields, thus successfully modeling the effects of unknown chemical reactions all the fields of the flow state.
D.1 PLANAR-VO
Figure [4(A) shows the 2D visualization of the temperature field predictions for Planar-vO case. As seen from the ground truth images, methane-air mixture (black color) converts into the burned products (yellow color) due the chemical reaction at the flat flame surface (red color). The dotted, horizontal red line helps to compare the transition the flame interface, i.e. the displacement of the flame front. Due to the chemical reaction, fuel-air mixture is consumed and turns into burned product as the simulation progresses. The FNO approach completely fails to predict the propagation of the flame for the given test case. Its output does show any evolution from the initial temper- ature profile for the given operating condition. The purely data-driven approach fails to capture the flame front displacement correctly, thus leading to inaccurate prediction with large emors. The
17

---

Under review as a conference paper at ICLR 2023
PDD
1500
1200
1500
Hybrid NN-PDE
1200
Ground
t= 1000
200
t= 300
t- 20
50
(B) 172
225
(A)
Figure 14: Temperature field predictions for (A): Planar-v0 flame case with E = 0.95; (B) uniform- Bunsen case with constant inlet velocity = 0.375 and 0.95 for different approaches. Hybrid NN-PDE model predicts physically accurate evolution of the flame cases under study.
P--7-- 12m1 TNO Enerr
6m1619
14331
22.488
f=0114
22.25
26.76
(P 1p-1) P00 Error
2.3005
C.4
2=308
1=13.43
4=4
1-
6=6.57
C- 6.51
PETTa- 7P")
INcr Sterr
(8)
Figure 15: Absolute error between target field (P.(Ti) and the output predicted from top to bottom top by: FNO (f:"0)), purely data-driven approach (fmddd) and hybrid NN-PDE (fthu)) model, for the instances shown in figure 14 The numbers represent instantaneous MAPE.
18

---

Under review as a conference paper at ICLR 2023
Table 5: Mean and standard deviation of squared errors of fuel (Y₁) and oxidizer (Y,) mass frac- tion fields over all time steps of all testsets. The Hybrid NN-PDE approach outperforms all other baselines considered.
Baseline FNO
Baseline NN
Hybrid
Hybrid
NN-PDE
NN-PDE-dt
Planar-vO
1,3c-4 + 7.4c-5
2.9c-5 + 3.8e-5
8.9c-7 + 1.0c-6
1.6e-7 + 1.5c-7
Sinform-Bunsenr
6,4c-4 + 2.4e-4
5. + 4.1e-5
5.6--7 + 7.4e-7
9.8c-7 + 1.3e-6
Yy
MmnUniforn-Buneenn.
2.2c-4 + 1.9c-4
9.6e-5 1.0c-4
8.0e-6 + 1.1e-5
1.4e-5 + 1.9c-5
nonUniform- Bunsen Caliifff---Bueenllt 100
1.8e-5 1 2.8e-5
1.8e-5 2.9e-5
Planar-vO
1.0e-2 + 6.8e-
1.3e-3 + 1.2e-3
3,7e-5 + 2.9e-5
4.1e-5 + 4.4e-5
uniform-Bund Bunneenn
4.7c-1 + 9.6c-2
2. + 2.7e-2
3.6e-5 + 3.0c-5
1.4e-4 + 1.2c-4
Yo
nonUniform- Bunsen32
1.400 + 1.8c0
6.1e-3 7.9c-3
1.6e-4 + 1.6c-4
2.8c-4 + 3.0c-4
nonUniform- BunneeBunnnnnnnne 100
3.3e-4 + 4.2c-4
5.0c-4 + 5.8c-4
hybrid NN-PDE model accurately captures this evolution of planar methane-air flame in quiescent mixture. Figure 15 shows the instantaneous MAPE w.r.t. ground truth data for predictions shown in figure 14 The absolute error shown figure [3]A) exceeds 1 100 for the FNO and purely data- driven approaches as these do not predict the flame temperature correctly. We use the upper limit of 1 1100 K for colorbar to highlight the errors in the hybrid NN-PDE approach more clearly. Large errors in FNO and PDD results stem from their inability to reliably predict the flame temperature and flame front displacement.
D.2 UNIFORM-BE BUNSEN
Figures B) and 15 B) show an example of uniform- Bannsenn case; case with Uy 0.375 and E = 0.95. The constant inlet velocity results in a symmetric, A-shaped flame. Vertical, dotted, red line in figure B) helps to assess the symmetric nature of the flame. The purely data-driven model predicts thicker flame (t 50) or flame with a spurious tip (t 172) or an asymmetric flame (t = 225). Snapshots of the hy NN-PDE model predictions in figure [4] B) show that adapts to this scenario very well and succeeds in obtaining the correct results for long term forecasts of the temperature field. Furthermore, the flame shape and height are also better predicted, as shown in figure 14 B). Very low error levels in figure B)) such as 1.26
300, indicate that the hybrid model recovers the temperature field satisfactorily. Figure [6 compares the performance of the hybrid NN-PDE model for the Y₁ and Y, fields for different test cases. We again observe that the hybrid NN-PDE model predicts all the fields accurately with very low MSE.
D.3 NONUNIFORM- BUNSSEN
Figure4comppr compares the temporal predictions made by PDD PDD and the hybrid NN-PDE model with ground truth data. Compared to Planar-v0 and mniform-Bunsen case, FNO predicts qualitatively better results for this case. Although highly still accurate, it predicts shapes that come closer to the target flame shapes for the later part of the simulation (for 300). This improvement might be due to the larger training dataset used for this scenario compared to the previous two scenarios. However, it fails to predict accurate temporal predictions over the entire simulation. The hybrid NN-PDE approach satisfactorily predicts the accurate flame evolution. Additionally, we showcase the performance of the hybrid NN-PDE approach on two different test cases with complex flame shapes in figure 18 The neural network model along with incomplete PDE solver reconstructs these complex flame shapes in an accurate manner.
Finally we showcase the performance of the hybrid NN-PDE approach different test cases for highly resolved flame case of nonUniform-Bunser 100. Figure 19 compares the predictions made by hybrid NN-PDE over {0, 100, 200, 20,,25,,300 300, 325, 350, 400, 450, 500} simulation steps with the ground truth data. It also shows the absolute difference between them. As the simulation progresses, higher errors observed around the flame front. However, the hybrid NN-PDE approach captures the flame shape very accurately for longer roll-outs of 500 simulation steps. Currently this approach uses training dataset similar MauUnnform-Bunseein Further improvements in accuracy can be
19

---

Under review as a conference paper at ICLR 2023
P Target
0.04
0.03
0.02
V11 NN-POE
0.01
IPaM Fedl Ciror
0.008
0.06
0.004
E 0,0e400
<= 6.8e-08
4.4.0000
s- 1-406
1.2e-00
1.2e-06
1.5e-06
4.7e-06
P.tYc.a Targes
0.8
Fo NN-POI
022
Patk, d. %aul Grar
0.055
1000
= 0,0e+00
= 1.3e-05
6.9--5
0e-05
= 06
1.e-05
1.2e-05
2.7e5005
Figure 16: Comparison between ground truth (or Target) data and the hybrid NN-PDE approach for the Uniform-Bunse case for fuel (top) and oxidizer (bottom) mass fraction fields for different operating conditions. Error field shows the absolute difference between target state and hybrid prediction over (0, 50, 100, 150, 200, 225, 250, 300} steps. indicates the instantaneous MSE.
20

---

Under review as a conference paper at ICLR 2023
PNO
2216
PDD
1806
Hybrn tOL.DD
1200
Ground Truth
t- 25
1-550
t 100
t
150
1- 200
t- 250
1-330
t= 350
1- 400
1- 500
Figure 17: Comparison between different approaches from top bottom FNO, PDD, hybrid NN-PDE for the ground truth data over long-rollouts 500 time steps.
test case. Hybrid approach predictions accurately match with
FATa TarQue
1600
1200
800
Eroor
250
6-6.00
2-0.5
E-1.44
6-130
6-274
--261
6-166
1-1.7
6-1.83
1-153
Fdita) Targue
1200
800
P4770 Fal
250
Figure 18: Additional visualization of reactive flow trajectories predicted by the hybrid approach for different test cases nonUniform 2arrmm--nnse22222 scenario.
achieved by increasing the training dataset size or training the hybrid model with longer look-ahead steps.
21

---

Under review as a conference paper at ICLR 2023
P.tTal Target
1600
NN PNDDEE
1200
Tr
800
IP/77) Ful Errer
500
250
8.0.0
E=054
8-1.2
6-2.06
6-2.29
5-2.38
5-2.87
5.30
3.46
PaT Target
2000
1600
Ts NN INE
800
PPtTa Fal Error
500
250
C=0.82
8=1.7
4.96
Pam Target
2000
NN PDE
1200
800
(P,07) fil Errer
250
E 0.00
E=1.05
6.59
2-2.40
6
E
3.99
6 4.62
6m5.81
6
6.40
6
6.65
Figure 19: Comparison between ground truth data (T.(T.) and hybrid NN-PDE approach predic-
tions (T)) for various test cases complex, high resolution scenario nonUniform B0
22

---

Under review as a conference paper at ICLR 2023
PAT7
Torget
1600
t=0
t
t0000
150
t 200
t 226
t 250
t 300
Fr NN-OE
1200
800
IPo07 fi1 Erfor
500
250
Patra Target
2000
250
f: NN NN-PDE
1200
t
t= 75
100
113
125
t= 150
IP (T. 7 Error
500
250
0.0.00
< 0.58
t.300
0.83
<- 0.98
c=1.17
<= 1:23
c=121
Petra Target
2000
t 100
T 200
t 300
t 350
t 400
I 450
1 500
1200
Fr NN-PDE
800
t
t 100
t 150
t 175
t 200
t 225
t 250
JPa77 Tal Error
500
250
6.0.0
2-1616
F=2.21
Putra Target
2000
1600
450
1200
NN-IDE
800
200
(PtT)) Fd Eiror
500
250
t=00.0
0.93
1.34
3.06
3.90
==9.99
5.02
6.37
Figure 20: Predictions made by hybrid NN-PDE model (T) with an incomplete PDE solver at twice the time-step of 2At,. are compared with the ground truth solutions coming from the complete PDE solver at time-step of At.. We showcase the effectiveness of hybrid approach in relaxing temporal stiffness of the complete PDE solver on reactive flow cases of from top to bottom Planar-vO, uniform-Bunsen, non nonUniform 321ooen3- and nonUniform Seiforr--Bunsen 100 for different test cases.
23


Reviews:
[{"review_id": "QMuwZmQ6Gk", "reviewer": "Reviewer_c79Z", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper argues for the combination of a neural network with an incomplete description of a system (a hybrid model), in their case, an incomplete PDE, rather than a fully data-driven approach that only relies on a neural network. They compare the two approaches on a set of systems described by thermodynamics and fluid dynamics. \n\nThe authors consider a supervised setting where the complete PDE, or solutions for it, are given and used for training the neural networks with an MSE loss. In this setting, their experiments suggest that the hybrid models outperform the data-driven approach by a significant margin. In addition, they argue that hybrid models can lead to faster solving of the PDEs than relying on the complete description when it is available. ", "summary_of_the_review": "Overall I recommend rejection of this paper for the aforementioned reasons. In particular, the lack of novelty of the approach and the limited reliability of the experiments.", "strength_and_weaknesses": "# Strength\nThe idea of using the physical understanding of the studied phenomenon, rather than an entirely data-driven one, makes a lot of sense to me. In addition, the systems studied in this paper seem novel to the hybrid learning literature and constitute solid test scenarios for assessing future developments in hybrid learning. \n\n# Weaknesses\nOverall I find the contribution very limited. Combining a neural network with equations from physics is a long-standing idea in the community. This paper does not introduce anything subtle regarding a good strategy to combine these equations with neural networks. \n\nThe experimental validation of their approach also seems limited in terms of the train/test scenarios considered. In addition, it is unclear how the authors selected the best models and hyperparameters, as no cross-validation or validation set is discussed. This also reduces the solidity of the presented results. For instance, I am pretty sure that at some point, the data-driven approach could catch up with the hybrid approach if the number of train scenarios increases. An analysis of when this (might) happen would have been valuable to motivate the hybrid approach for these problems.\n\nIt is unclear to me whether the parameters of the PDE are also learned at training time. If not, this makes the proposed approach's applicability very limited as, in most cases, if we can observe a system and have little understanding of the physics behind this also means that we do not know the corresponding parameters of the incomplete physical description. I have probably misunderstood something there, but I would encourage the authors to clarify this in the paper. Overall, this paper would benefit from a real-world demonstration of the proposed approach. Indeed, the contribution is not methodological. Thus I would argue that this should contain a more solid empirical evaluation.\n\nAs a remark, I also feel that some confusion is made between the \"model\" and the \"inference\" (in the case of PDEs, the solver). The neural network aims to complete the model, create a more accurate description of the real world, and create a better model. The solver is related to how we make inferences given a model; the terms complete and incomplete solvers do not make sense to me.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "LARm1WVykYr", "reviewer": "Reviewer_95Sg", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper studies the task of learning continuous dynamical systems from data using a hybrid (machine learning + PDE solvers) approach and compares the performance to purely data-driven methods. In particular, authors consider a regularly occurring in practice setting where only a partial knowledge of the governing equations are available. Authors use reactive flow systems (specifically various flame simulations) as the test ground for evaluation and comparison. They find that by incorporating physical priors in the form of a partial PDE solver improves the performance of the learned model and is able to accurately capture the dynamics of the system and works well for test simulation settings. Additional findings include observation that learned hybrid models result in less stiff updates compared to the underlying ground truth solver.", "summary_of_the_review": "I enjoyed reading the paper in general, as the authors address an important question of whether physical priors help/needed to model continuous dynamical systems well. I find the step up in complexity of the simulated system a bonus and in general think this is an important research avenue.\n\nI’m generally leaning towards accepting this paper (score of 8), assuming some flaws and weaknesses can be addressed. (hence the initial score of 6).\n\nMy (1) primary concern is related to the evaluation metric and error analysis. Unless I’ve missed something, it is not clear how significant the mean values of MAPE and MSE are given the standard deviations. Is there a way to present the results in a way that clearly indicates that hybrid approach generally improves over data-driven methods? Maybe using a histogram of errors over different sample trajectories / random seeds would provide a more complete picture of error distribution? The fear is that coarse summary might hide stability problems of the purely data driven approaches which would misrepresent their performance as there are numerous methods to address stability issues.\n\nMy (2) concern is related to the time-step used for data-driven predictions. In general, data-driven approaches often subsample the predictions in time, which result in better accuracy and efficiency. (e.g. see Fig. 3 “Stachenfeld et al., 2022” (reference used in the paper)). Given that the underlying ground truth solver involves stiff dynamics it could turn out that data-driven models are evaluated in a highly suboptimal regime where they are tasked to predict a very slowly varying solution.\n\nFinally, I think the paper would read better if the stakes for “primacy” were omitted. Arguably a number of previous works learned various closure models in a tandem with differentiable solvers and present work just extends it to a more challenging setting, which is important in its own right.\n\nI’d be happy to provide an additional set of minor comments regarding potential improvements wrt references after primary concerns are addressed.\n", "strength_and_weaknesses": "Strengths:\n* Authors consider a dynamical system that is more complex than that commonly studied in learned simulators literature.\n* The considered method is tested against relevant baseline approaches.\n* Paper is clearly written.\n\nWeaknesses:\n* Standard deviation estimates of the quantitative results (e.g. Table 1) are O(1) wrt. relevant mean values. While visualizations support the conclusions based on the trend of the mean MAPE and MSE, I find the results somewhat hard to trust.\n* An important time subsampling hyperparameter has not been considered for data-driven models.\n* AFAICT datasets are not made available to perform independent evaluations/tests.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "HKCxgZ3lHX", "reviewer": "Reviewer_hgQY", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper aims at deploying a deep learning approach in combination with partial differential equations (PDEs) with known yet incomplete physical information. \nFor example, a deep learning model can be employed when the information on the system to be solved is limited but additional data are available. A common benchmark is represented by turbulence modelling in computational fluid dynamics where the full PDEs descriptions are too expensive to be solved computationally. Therefore, the PDEs can be rewritten as a sum of an incomplete PDE description plus another term representing the physics arising from the unknown information (reaction rate terms). Correctly representing this additional term is of crucial importance to solve the task. \n\nContrarily to standard Neural PDE solvers, this method aims at combining physical incomplete PDE information with some additional machine learned correction due to the unknown part of the PDE description. In this work, the latter is represented by the chemical kinetic mechanism of non reactive flow simulation. The complete (reactive) simulation is used as a baseline. \nThe downstream task is to solve a simplified version of the Navier-Stokes equation for a specific system where the chemistry is removed from the analytic description (i.e., is unknown).\n\nA neural network is trained to learn correction fields that would correct the incomplete (non reactive) PDE equations where both source terms and reaction rates have been removed.\nThe work compares three baselines: Ground truth, Pure Data Driven approach, Fourier Neural Operators (state-of-the-art neural operator method).\n\nThe framework is clearly explained in figure 2 where the PDD and the NN-Hybrid approaches are compared in B1 and B2 respectively. \nThe 3 different setups on which the approach was tested are carefully explained.  \n\nA few comments/concerns are in place:\n\n- The following sentence is a bit unclear to me ‘Its objective is to learn to model the effects of the unknown chemistry using the neural network parameters θ given an input flow \n State […]’. What is it meant by ‘[…] using the neural network parameters \\theta […]’?\n- In the last row of table 1 some baselines are missing (i.e., FNO and NN.) I presume this is because they never converge to the target value? It would be helpful to write something about it in the table caption. \n- On page 7 below figure 4, in the second paragraph the authors define the relative displacement. Right afterward, they mention x_t and \\tilde{x}_0. I cannot relate these quantities easily to the definition and this may be overall a bit confusing. I would suggest to keep the definition of relative displacement general \\tilde{x}_300 -> \\tilde{x}_t and they specify that t=300. As it is now the discussion might not sound very linear and straightforward. \n- I would suggest the authors to add additional plot to figure 6 to visualise the performance of other baselines like FNO and NN. From the discussion of table 1 is already clear they perform worse than NN-PDE, but visualising them (as it’s done in figure 12) might be useful.\n- In section 5.3 the author only discuss the full NN method without mentioning the NN-PDE. I am wondering it that is intentional. If so, there’s a reason for that? Are there any limitations that prevent NN-PDE to be applied for this task? \n- I suggest the authors to make the labels explicitly in figure 2 saying what’s the PDD and what’s the NN-hibridy approach for an easy first-sight intuition.\n- The caption of figure 2 does not explain what is S. Though this is done in the main text would be useful to repeat it in the caption below the figure. \n- I’d rename section 2 something like ‘Literature Review’ or ‘Related Work’. To me ‘Background’ sounds more like if you are introducing the physics behind the work.  \n- Missing full stop at the end of equation 3. \n- Few typos/stylistic issues were found throughout the manuscript. I’d recommend the authors to run a spellchecker for making sure to detect them.", "summary_of_the_review": "To incorporate physical prior information when training a neural network to solve a physical problem is known to be a successful approach. \nIn this paper the author propose to learn an incomplete description of a PDE and correct a posteriori using a NN for removing two terms which in general act on different time scales. Doing so, the PDE solver happens to be much more efficient in learning the dynamics of the incomplete system. Contrarily,  when learning the full dynamics of the complete PDE description (pure NN approach) the interplay between different time-scales makes the learning task much more difficult. \n\nThe idea of separating the PDE and learn part of the dynamics in a supervised learning fashion and correct the PDE result a posteriori is indeed a valuable and interesting contribution to the community. \n\nDespite not being a domain expert, I still do see some potential in this work, especially in future developments. \nFor these reasons and all the considerations from above, I would recommend to consider this manuscript for acceptance. ", "strength_and_weaknesses": "The problem tackled by the paper is of great practical relevance. It is common in the physical sciences to face numerical PDEs where the complete description is either very expensive, from a computational perspective, or non accessible. Therefore, when a full analytic description is available, as well as reference, experimental, data, one can combine a numerical PDE solver for an incomplete description  of the full PDE with a correction term which can be learned by a NN upon training on some appropriate data. \n\nOn the other hand, physics-informed ML is a quite well-established avenue of research. It is not at all surprising to me that combining incomplete physical information of an incomplete PDE description with a NN that learns phenomena acting on different time scales enhance the performance. Therefore, while I am not too impressed nor surprised by the results, I do indeed see some value in the method and I thereby recommend consider this manuscript for acceptance.  \n\nAs a further suggestion the author may study some transfer learning properties, i.e., how well the correction NN would perform when being trained on a specific set of initial condition but used in different setups. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "BfKcl2KawU", "reviewer": "Reviewer_NRLv", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper considers the case of incomplete physics for numerically solving Partial Differential Equations. It uses a neural network to complete the missing terms and more accurately forecast the evolution of the dynamical system. The proposed approach is evaluated on different variants of a reactive flow problem governed by Navier-Stokes equations. ", "summary_of_the_review": "This is mostly an application paper that proposes to learn to complete incomplete physics models using neural networks in the context of reactive flow problems. My main concerns with the paper is how it relates to previous work that also learns to complete incomplete physics models as well as the fact that, at least to my understanding, the baselines that are considered are weak. ", "strength_and_weaknesses": "Weaknesses\n\nThe paper claims that it is the first of each kind that learns to complete physics in a PDE setting. Nevertheless learning with incomplete physics models using neural networks to complete the missing part has been studied rather extensively. For example: APHYNITY, Yin et a, Journal of Statistical Mechanics: Theory and Experiment (also cited in beginning of the present submission)  and ICLR2021; Physics integrated VAEs for robust and interpretable generative modelling, Takeishi & Kalousis, NeurIPS, 2021. APHYNITY tackles forecasting tasks, similar to the current submission, for physics systems that are described by ODES/PDES. It models the underlying dynamical system with two additive components, one given by the known physics and a neural network modelling the unknown parts. Physics VAE learns generative models where the VAE decoder also decomposes, to a more general composition than additive, of an incomplete physics component and a neural network. The paper should at least discuss differences from these two works. \n\nWith respect to the experimental evaluation it is not clear to me how the neural network baseline has been trained. It seems that it learns state-state transitions. How is this done? does the model during training sees only consecutive state pairs? or does the model evolve in a longer horizon, i.e. over a sequence of states, and then the model update would have to happen over such sequences, accounting like that for longer temporal dependencies and error compounding. From what I understand the neural network baseline is trained over state state transitions, which, if I am correct, puts it at a clear disadavantage with the hybrid model. More appropriate baselines will explicitly model for long term dependencies, e.g. a solver where there is no physics but just a neural network, neural ODEs, models with recurency. The two papers above provide a number of such baselines. \n\nStrengths\nThere is a detailed evaluation of the model in a complex scenario exploring different extrapolation scenarios. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject