Coupled Multiwavelet Operator Learning for Coupled Differential Equations
Authors: Xiongye Xiao
Defu Cao
Ruochen Yang
Gaurav Gupta
Gengshuo Liu
Chenzhong Yin
Radu Balan
Paul Bogdan
Venue: ICLR 2023 poster (2023)
Keywords: Neural operators
coupled differential equations
multiwavelet transform
partial differential equations

Abstract: Coupled partial differential equations (PDEs) are key tasks in modeling the complex dynamics of many physical processes. Recently, neural operators have shown the ability to solve PDEs by learning the integral kernel directly in Fourier/Wavelet space, so the difficulty of solving the coupled PDEs depends on dealing with the coupled mappings between the functions. Towards this end, we propose a \textit{coupled multiwavelets neural operator} (CMWNO) learning scheme by decoupling the coupled integral kernels during the multiwavelet decomposition and reconstruction procedures in the Wavelet space. The proposed model achieves significantly higher accuracy compared to previous learning-based solvers in solving the coupled PDEs including Gray-Scott (GS) equations and the non-local mean field game (MFG) problem. According to our experimental results, the proposed model exhibits a $2X-4X$ improvement relative $L$2 error compared to the best results from the state-of-the-art models.

Published as a conference paper at ICLR 2023
COUPLED MULTIWAVELET NEURAL OPERATOR LEARN- ING FOR COUPLED PARTIAL DIFFERENTIAL EQUATIONS
Xiongye Xiao
Defu Cao 1
Ruochen Yang'
Gaurav Gupta' Paul Bogdan'
Gengshuo Liu'
Chenzhong Yin'
Radu Balan 2
University of Southern California, Los Angeles, CA 90089, USA University of Maryland, College Park, MD 20742, USA
(xiengyex, defucao, ruocheny, ggaurav, gengshuo, chenzhoy, pbogdar Busc. lemaaadn@ddd...mmm rvba lan umd. edu,
ABSTRACT
Coupled partial differential equations (PDEs) are key tasks in modeling the complex dynamics of many physical processes. Recently, neural operators have shown the ability to solve PDEs by learning the integral kernel directly in Fourier/Wavelee space, so the difficulty for solving the coupled PDEs depends on dealing with the coupled mappings between the functions. Towards this end, we propose coupled multiwavelets neural operator (CMWNO) learning scheme by decoupling the coupled integral kernels during the multiwavelet decomposition and reconstruction procedures in the Wavelet space. The proposed model achieves significantly higher accuracy compared to previous learning-basee solvers in solving the coupled PDEs including Gray-Scott (GS) equations and the non-local mean field game (MFG) problem. According to experimental results, the proposed model exhibits a 2x 4x improvement relative L2 error compared to the best results from the itate-oftthe -hetthr-rt models.
INTRODUCTION
Human perception relies on detecting and processing waves. While our eyes detect waves of electromagnetic radiation, our ears detect waves of compression in the surrounding air. Going beyond waves, from complex dynamics of blood flow to sustain tissue growth and life, to navigating underwater, ground and acrial vehicles at high speeds requires discovering, learning and controlling the partial differential equations (PDEs) governing individual or webs of biological, physical and
chemical phenomena (Lacasse al. 2007 Henriquez 1993 Laval & Leclercq 2013 Ghanavati ett al. 2017 Radmanesh et al., 2020). Within this context, neural operators have been successfully used learn and solve various PDEs. By representing the integral kernel termed as Green's 's function in the Fourier or Wavelet spaces, the fourier neural operator (Li et al. 2020b) and the multiwavelet- based neural operator (Gupta et al. 2021b a)) exhibit significant improvements on solving PDEs compared with previous work. However, when it comes to coupled systems characterized by coupled differential equations such as mean field games (MFGs) (Lasry Lions 2007 Achdou Capuzzo Dolcetta 2010), analysis of coupled cyber-physical systems (Xue & Bogdan (2017). or analysis of the surface currents in the tropical Pacific Ocean Bonjean & Lagerloef (2002). the interactions between the variables/functionss need to be considered to decouple the system. Without the knowledge of underlying PDEs, the complex interactions can be hardly represented in the data-driven model. To build data-driven model that give general representation of the interactions to efficiently solve coupled differential equations, we propose the coupled multiwavelets neural operator (CMWNO).
|
Neural Operators. Neural operators (Li et al. 2020b с Gupta al. 2021b) Bhattacharya et al. 2020: Patel et al.] (022)) focus on learning the mapping between infinte-dimensione spaces of functions. The critical feature for neural operators to model the integral operator namely the Green's 's function through various neural network architectures. The graph neural operators (Li et al. 2020ble) use the graph kernel to model the integral operator inspired by graph neural networks; the
$ Equua Contribution

---

Published as a conference paper at ICLR 2023
Fourier neural operator (Li al. 2020b) uses an iterative architecture to learn the integral operator in Fourier space. The multiwavelet neural operators (Gupta et al. 2021 a) utilize the non non-standard form of the multiwavelets to represent the integral operator through 4 neural networks in the Wavelet space. The neural operators are completely data-driven and resolution independent by learning the mapping between the functions directly, which can achieve the state-of-the-art performance on solving PDEs and initial value problems (IVPs). To deal with coupled PDEs in the coupled system and be data-efficient, we aim to decode the various interaction scenarios inside the neural operators.
Multiwavelet Transform. In contrast to wavelets, multiwavalets (refer to Appendix C use more than one scaling functions which are orthogonal. The multiwavelets exploit the advantages of wavelets, such as (i) the vanishing moments, (ii) the orthogonality, and (iii) the compact support. Along the essence of wavelet transform, series of wavelet bases are introduced with scaled/shifted versions in multi wavelets to construct the basis of the coarsest scale polynomial subspace. The multiwavelet bases have been proved to be successful for representing integral operators as shown in (Alpert ct al. 1993) (the discrete version of multiwavelets) and (Alpert. 1993b). In our proposed model, to develop compactly supported multiwavelets, we use the Legendre polynomials (Appendix D which are non-zero only over finite interval as the basis. The differential (0/ōt) and the integral ) operators can be represented by the first-order multiwavelet coefficients (s and d) of orthogonal bases via decomposition in the Wavelet space.
Mean Field Games (MFGs). As a representative problem for coupled systems in the real world, MFGS gains raising attentions in various areas, including economics (Achdou et al. 2014 2022).
finance (Guéant et al. 2011 Huang et al., 2019) and engineering (De Paola et al. 2019 Gomes al. 2021). etc. Building on statistical mechanics principles and infusing them into the study of strategic decision making, MFGs investigate the dynamics of large population of interacting agents seen as particles in a thermodynamic gas. Simply speaking, MPGS consist of (i) a Fokker-Planck equation (or related PDE) that describes the dynamics of the aggregate distribution of agents, which coupled to (ii) a Hamilton-Jacobi -Belllmn equation (another PDE) prescribing the optimal control of individual (Lasry& & Lions| 2006 2007| [uang et al., 2006 2007). Among different types of MFGs, the class of non-potential MFGs system with mixed couplings is particularly important as it is more reflective of the real world with a continuum of agents in differential game.
Solutions on MFGs. Previous works either only restrict to systems without non-local coupling.
such alternating direction method of multipliers (ADMM) (Benamou & Carlier) 2015 Benamou
iet al. 2017) and primal-dual hybrid gradient (PDHG) algorithm (Briceno-Ariae et al. [019 2018)
or use general purpose numerical methods for solving the MFG problems (Achdou al.) 20 b
Achdou & Capuzzo- Dolccetta 2010), which misses specific information from the target structure. In
addition, the aforementioned works are not parallelizable with linear computational cost under the
coupled MFGs settings. Recently, (Liu & Nurbekyan (020) considers dual variables of nonlocal couplings in Fourier or feature space. Furthermore, (Liu et al. (02T) expands the feature-space space in the -ernel-based representations of machine learning methods and uses expansion coefficients to decouple the mean field interactions. However, both dual variables and expansion coefficients need to bound the interactions of coupled system in a reasonable interval with prior knowledge. In our work, we first introduce the neural operator into coupled MFG fields, which can decouple the various interactions inside the multiwavelet domain.
Novel Contributions.
The main novel contributions of our work are summarized as follows:
For coupled differential equations, we propose a coupled neural operator learning scheme, named CMWNO. To the best of our knowledge, CMWNO is the first neural operator work using pure data-driven method to decouple and then solve coupled differential equations.
Utilizing multiwavelet transform, CMWNO can deal with the interactions between the kernels of coupled differential equations in the Wavelet space. Specifically, we first yield the representation of coupled information during the decomposition process of multiwavelet transform. Then, the decoupled representation can interact separately to help the operators' reconstruction process. In addition, we propose a dice strategy to mimic the information interaction during the training process.
2

---

Published as a conference paper at ICLR 2023
The proposed model successfully learns the interaction between the coupled variables when the couple degree is increasing and thus it could open new directions for studying complex coupled systems via data-driven methods. Experimentally, the proposed CMWNO framework offers the state-of-the-art performance on both Gray-Scotts (GS) equations and non-local MFGs. Specifically, CMWNO outperforms the best baseline (MWT.) by 54.0% on GS equations with various resolutions and outperforms the best baseline (FNO.) by 61.4% % on non-local MFGs with different time steps.
2 COUPLED MULTIWAVELET NEURAL OPERATORS LEARNING
To solve coupled control system characterized by coupled state equations in control theory, popular way is to use the Laplace operator to represent differential and integral operators (Gilbarg et al. 1977). Therefore, the coupled high-order differential equations first-order differential equations in the Laplace space which will reduce the decoupling difficulty. assume that
be transformed into the
Inspired by the use of the Laplace operator and the properties of the multiwavelets, the interactions between kernels can be used to approximate the coupled information by reducing the degree of high-order operators in multiwavelet bases. With this assumption, we are able to build the coupled multiwavelet neural operators (CMWNO) learning scheme, which utilizes decomposition representation from the operator and mimic the interaction via a dice strategy.
2.1 COUPLED DIFFERENTIAL EQUATIONS
To provide a simple example of the coupled kernels, K1 and K2, let us consider a general coupled system with 2 coupled variables u(x, t) and v(2, t) with the given initial conditions un(x) and vo(x). Given A and L/ as two Sobolev spaces 7 with s > 0.p 2, let T denote generic operator such that T 1A-2 U. Without the knowledge of how these two variables are coupled, to solve for u(z r) and v(z, r), we need two operators Ti and T2 such that Tiuo(z) u(æ,T) and T2vo(x) v( T). The coupled kernels termed as Green's function can written as follows:
Tuu((z) =
Ki (I. y, uo(x), uoly). vo(x), volv), K2) (y)
(1)
T2v0(2)= (2, y, un (r), un(y), vo (2), voly) Kn ()) (y)
u( 0) uo(x); v(z, 0) (z)x) E D,
where D = Rd is a bounded domain. The interacted kernels cannot be directly solved without considering the interference from the other kernel, and our idea to simplify the kernels first and deal with the interactions in the multiwavelet domain.
2.2 MULTIWAVELET OPERATOR
To briefly introduce the multiwavelet operator, we explain how the neural networks are used to represent the kernel in this section. The basic concept of multiresolution analysis (MRA) and multiwavelets (Alpert al. 1993 Alpert T993a b are provided the Appendix
Notation For k E Z and n E N. the space of piecewise polynomial functions is defined as: VE {he restriction of ff to the interval 2 "(1 1)) is a polynomial of degree for all 0, 2n - 1, and f vanishes elsewhere). V consists of the orthogonal scaling O..... k 1, and V can be spanned by shifting and scaling these functions as (x) 2n/ (2"xr r 1), where j 0.... (I) are called scaling coefficients marked as Sh. The multiwavelet subspace W is defined as the orthogonal complement of V in V n+1 such that V + W V n+1 V 1 W W consists of the orthogonal wavelet functions ti with i O...., wavelets functions (x) with wavelets coefficients dji-
functions io; with .,
The coefficients of and I 0, 0.....2 2n
Similar V We is composed of the
To represent the functions and learn the mapping in multiwavelet space, the nonstandard form is used represent the integral operator. According to (Beylkin al. 1991 Alpert et al. 20026),
3

---

Published as a conference paper at ICLR 2023
orthogonal projection operator Pk then an single operator T in our coupled system can be represented as:
H V u. and Q 7/.2 - W with Q Pi+I P.
+ T
Σ (A 2=0
B C.),
(2)
where Th POTPe A Q B T C wavelet operator, Therefore, the nonstandard forms of the operator is a collection of triplets {To. (Af, B, C )==0...... For given operator T Tuo(r) (z), the map under wavelet space can be written as:
n PATQ Q is the multi-
Ta
C,d, Ts i= == O..,...,
where, (T (st) di) are the scaling/wavelet coefficients of (z) uo(x) in subspace V In our model, one kernel is approximated using 4 simiple neural networks A, B, C and T such that Ta Aox (d)) + Bog (st), Tj 2 Coo (di), and To Top (8f)
2.3 COUPLED MULTIWAVELETS MODEL.
This section introduces coupled multiwavelets model to provide general solution coupled differential equations. First, we make a mild assumption to decouple two coupled operators given in Section 2.1 To simplify eq. [ without loss of generality,
build operators T, and to approximate u T and v(x, T), where Tμ and T, are decoupled and do not carry any interference from each other. In other words, can write T,,0((1
assume that
u (2, r); (x))(z) o'(x, r), where u'(a u'(x,7 T) and v'(x, T) are the approximations of u( and v(x,7) 7) without coupling. The assumption mild and easy get satisfied in the Wavelet space since the operators
be represented
by the first-ond order multiwavelet coefficients. According to this assumption, we can derive the following relations:
u(z, r) = T,u(2,0) + (To), FE D IE
u(x, T) T, + (T(w),
where e (Tw) quantifies the interference from operator T, to solve u(x, r) and e(T.) represents the measurable interaction from operator T... Therefore, the integral operators can be written as:
Tw)o(z) = =S. (x, vo)) Tooo(z) [[ (z, y) vyy)dd
(5)
the kernels Ny and K. termed as Green's functions can be learned through neural operators, where Ki can be learned using the data of while the kernel K, is learned from v. To model 6, (T.) and (T,),, we transform the operators into multiwavelet coefficients in the Wavelet space and embed through simple linear combination after the decomposition steps.
Based on the concept of multiwavelets (Appendix Section C). here we simply explain the de- composition step and reconstruction step of multiwavelets in coupled system. Since V V + w according to Section 2.2 the bases of V/ can be written as a linear combi- nation of the scaling functions spp-1 and the wavelet functions v;--1. The linear coefficients (H ()), (()) are termed as multiwavelet decomposition filters, transforming represen- tation between subspaces V V W W and V For given function f(x), the scaling/wavelet coefficients ld of scaling/waveler functions sp)/// (b)) are computed as:
(1+1)
2 ((+1)
$)
(z));
-
di
f())
(6)
2-
2-
Using the multiwavelet decomposition filters, the relations between the coefficients on two consecutive levels n and n + are computed as (decomposition step):
sp H1 (0) s H (()st]
(n+1+ - (()) (+t)
(7)
Therefore, starting with the coefficients sy, we repeatedly apply the decomposition step in eq. to compute the scaling/wavelet coefficients on coarser levels. Similarly, the reconstruction step can be represented as:
H sp ((
a+1 2111
H(1 sp
(())
(8)
4

---

Published as a conference paper at ICLR 2023
Decomposition
Reconstruction
T D Tud
06,5
Ub.s
Tu.d
Tu
H.
P
+ Tus
H. G
1,5
Uba
Tus
Tuss
U
T
Pú T
Vo,s
v.d
12441
T
0,5
H.G
H, G
...
Vod
v.3
VO,
T
V
V 7o + Tpp
Figure I: Architecture of CMWNO. Note that there are two coupled operators, Tu and T., in our system, which aligns the number of coupled variables. The network 7 is only applied for the coarsest scale L (0 in this system). The dashed arrows correspond to the auxiliary information from the unused operator without gradient during training process. For interaction between operators. when we update operator Tu, the decomposed ingredients from T, he equipped into the reconstruction module Tu Wavelet domain, vice versa.
Repeatedly applying the reconstruction step, di, ,7. In general, the function can be parameterized as the scaling/wavele coefficients in the Wavelet space after the decomposition steps, and the coefficients can be mapped to the function after reconstruction steps. In our work, to model the interference (Tw) and (T,).. obtain the multiwavelets coefficients of each kernel during the decomposition steps and embed them into the other kernel in the reconstruction step. Note that will elaborate the detailed training strategy of how to mimic interactions inside our system in Section 2.4
compute the coefficients sf from of and
Our idea is to represent the functions and operators in Wavelet space to decouple the system using simple linear combinations. Considering the example in Section 2. according to the eq. 4 and we first build two operators T. and T, such that T,,)((x) u, (x) (ov)(1)
(1r). For the operators Tu and T., we denote their scaling/waveler coefficients in wavelet domain as TV u T and T i Ti ae respectively. For the input uo(x); (z( and the output u, (x); (x(),, we denote their coefficients as Uj o Vo,sta and U sayz V coefficients of T, and T, can be calculated as:
(d)()) According eqs. 3 and 5 the multiwavelet.
Ty = AL, + Bk Ui.st Ti,si = Ckk Uh,
T TUg
Tí 0111 = V,, B Vo, Ti Ch,
To T0,t
Tvo 0,5
where i = 0, 1,... Considering the interference from the other operators, the coefficients of the solutions u, (x) and (x) the Wavelet space can be written as:
U Ty.as + Fú Tea
UR Th + Ph
U
T + 7o u,s1i
Vi 7. T as + Ti
Vi V7,a1 T T + Th a.at
VO
T To. + 7o T
(10)
where
the training process, the inputs of the neural networks [A[,.)) B(,,o]. Cu.,o) } are the multiwavelet coefficients of uo(z); vo(z). and the outputs are the multiwavelet coefficients of Tu: : T., When the neural networks (A,, Bu, Cu, Tu) are trained for T., the neural networks (A,, Ba, T.) output T.,sstay without backpropagation, we use Zh [u,et_ast,at] to mark the coefficients without gradient. Utilizing the orthogonality of the multi- wavelets, the coefficients embedding the information the operators T,; T, can be directly added to Th, Tú in the same Wavelet space V/ then the neural networks with backpropagation can learn the
.n.....n.
5

---

Published as a conference paper at ICLR 2023
information from the other operator. In that way, the complex coupled equations can be solved via reducing the order of the functions and directly approximate decoupled functions at each iteration.
The architecture of the CMWNO is shown in Fig. [ which illustrates the mapping process inside the wavelet space of layer n. The operations inside the wavelet space can be matched by the order of layers in the models, which means the decomposition operations for different resolutions are done independently. After decomposing s" via eq. 7 we can get the transferred information of input where each component will be used to reconstruct the original input at the layer n.
2.4
DICE STRATEGY
Inspired by scheduled sampling (Bengio et al. 2015), which is designed to gently bridge the discrepancy between training and inference sam- ples, we propose rolling the dice to randomly decide the interaction order between each neural operators, which ís named dice strategy. Specifi- cally, we roll the dice for every sample to decide which path to use, which can effectively mitigate the imbalance update problem for each kernel caused by the fixed training order. As illustrated in Fig. 2 when the dice tells the model to use path (upper path), we will update operator T, by equipping the coupled information from the other operator T. first. Note that, T, is learned by previous samples and have not updated yet. Then we use the updated operator Tu to decom- pose the initial state No: which can be used to update T.. Inside the Wavelet space with well-defined basis, where we are able to utilize vary orthogonal information from each initial state jointly. Note that this strategy is scalable to more operators referring to Fig. 6 Supplementary and we left the design of this strategy for future work.
(U k
Dos
Dice
Dec
Dice
Dec
1 DELLY
Figure Dice strategy. For each sample, one only needs to go through a specific path (round diagonal comeer rect- angle). Inside cach path, the order of updating is from left to right, where the darker block indicates the oper- ator want to update and the lighter blocks provide decomposition information from the fixed operator.
EXPERIMENTS
In this section, empirically evaluate the proposed model famous coupled PDEs such as the Gray-Scott (GS) equations and the non-local mean field game (MFG) problem characterized by coupled PDEs. Note that we compare against the state-of-the-art data-driven models which fits for our research goal to build efficient coupled operators for general downstream data-driven applications without sufficient expert knowledge. The experiments show that CMWNO not only achieves the lowest T.2 relative errors when solving coupled PDEs, but also works consistently great under different input conditions. For the data structure, since our datasets are functions, we apply point-wise evaluations on the input and output data. For example, for the function f(x), e D, we discretize the domain as æi,... H. E D, where I, are s-point discretization of the domain. Unless stated otherwise, we train on 1000 samples and test on 200 samples.
Model architecture. In our proposed model, for each operator, the neural networks A, B and C single-layered convolutional neural networks while T uses single linear layer. Our model is extensible and each kernel constructed by 4 neural networks (A, B, C, C3 learning the mapping in wavelet space. The number of the kernels can be chosen based on the number of coupled variables or the number of explicit operators.
Benchmark models. We compare our model with the state-of-the-art neural operators including Fourier neural operator (FNO), Multiwavelet-based neural operator (MWT), and Padé exponential model (Padé), which show the best performance on solving PDEs according to the experiment results in (Li et al., 2020b al Gupta et al.) 2021b ab. For the benchmark neural operator models, since have the coupled functions as input and output (c.g., u and v), we concatenate u and for the models and marked the models FNO.,, MWT,, and Padé.. We also use two single multiwavelet-hasee

---

Published as a conference paper at ICLR 2023
neural operators to learn (z)x);; (2)2) from ey (x();; ur (z(2) independently and mark the model as MWT,.
Similar to the coupling structure of our CMWNO, by creating the multiple kernels learned in Fourier space and applying the dice strategy during the Fourier transform, we build the coupled Fourier neural operator and mark it as CFNO.
Training parameters. The neural operators are trained using Adam optimizer with a learning rate of 0.001 and decay of 0.95 after every 100 steps. The models are trained for total 500 epochs which is the same with training CMWNO for fair comparison. All experiments are done on an Nvidia A100 40GB GPUs.
3.1 GRAY-SCOTT (GS) EQUATIONS
The GS equations are coupled differential equations which model the underlying reaction and diffusion patterns of chemical species. It is also able to generate a wide range of patterns which exist in nature, such bacteria, spirals and coral patterns. Each variable (i.e., and v) diffuses independently with linear growth or decay term, while coupled together by tuv² (Trefethen & Embree 2001 Driscoll al. 2014). For given field u(x; t); v)x,t), the GS equations take the form:
d,((1, () = El Oza (1r () + F(1 u(x, ()) - Au(a, (x,
IE (0, 10), I E (0, 1]
Qu((x = 22 t) (K + FFv( t) + Au(z,t)u (2) re (0, 10), t e (0. 1]
(11)
w(x, 0) uo(x); v(2, 0) vo(x), x e (0. 10)
where 61 1, 10-
6.62 10 10-2,71 F 2 10-2 We the coupling coefficient E (0, 1] to control the degree of coupling of u and v. We aim to learn the operators (i) mapping
the initial condition u(x. 0) to the solution u(x,f 1) with the interference of v(x, t): (ii) mapping
the initial condition v(x, 0) to the solution c(x, 1) considering the interference of u(x, t). The initial conditions are generated in Gaussian random fields (GRF) according to (z)). v0(2))
N/O,, (*-- + 721)-255 with periodic boundary conditions. We also use a different scheme to generate x)(x) by using the smooth random functions (Rand) chebfin package (Driscoll function defined by Fourier series with independent random coefficients; the parameter y specifies the minimal wavelength and here we choose y = 0.5. have two combinations of the initial conditions (i.e., un( (x) and (z)) and we mark them as (U-GRF, V-GRF) and (U-Rand, solve the
et al. 2014) which returns
Therefore, generating the initial conditions by different schemes,
V-GRF) respectively according the generating schemes. Given the initial conditions, equations using fourth-order stiff' time-stepping scheme named as ETDRK4 (Cox & Matthews (0002) with resolution of 20 and sub-sample this data to obtain the datasets with the lower resolutions.
2=256
5=512
s= 1024
Models
U
V
U
V
CMWNO
0.00468
0.00464
0.00492
0.00434
0.00471
0.00450
CFNO
0.01371
0.00654
0.01345
0.00643
0.01421
0.00619
MWT,
0.08075
0.07308
0.08041
0.07382
0.07996
0.07213
MWT, 0.01445
0.00742
0.01408
0.00744
0.01334
0.00779
FNO. 0.01431
0.00812
0.01542
0.00819
0.01
0.00885
Padé
0.01904
0.00964
0.02070
0.01022
0.02233
0.01055
Table 1: Gray-Scott (GS) equation benchmarks for different input resolution s initial condition (U-GRF.V-
GRF). The relative L. emrors are shown for each model. Bolded values are the best results of all the models, and underlined values are the best results of the existing models. Set the same below.
Varying resolution. The results of our experiments on GS equations with different resolutions (i.e.. s = 256, 512, 1024) are shown in [ As shown in the results, all the models exhibit the resolution independence. The model MWT, with concatenated data performs better than model MWT, using two independent single MWT models to train u and v separately, which indicates the information from cz(x); ug(x) benefits the model predicting for u, u+(x;; ey (z).. For solving ur (x); 2, (z). our proposed CMWNO outperforms 2X- 3.X improvements compared with the best benchmark with respect relative L.2 error. CFNO also outperforms FNO., however, compared
7

---

Published as a conference paper at ICLR 2023
10
VIRTS
MOOD
INTV
UNI
Relative ctvor
Relative ernor
10"
10
0.2
0.4
0.6
0.8
1.0
0.2
0.4
0.6
0.8
1.0
A
Figure 4: Comparing the models by varying the coupling cocfficient at the initial condition (U-GRF, V-GRF) with resolution & = 1024.
with the improvement of CMWNO MWT, the improvement of CFNO on FNO is not significant, indicating that decoupling in the Fourier space is not as efficient as decoupling in the multiwavelets domain. The learning curve of the neural operators solving ur (x) resolution 1024 is shown in Fig. 3
Varying coupling coefficient By varying the coupling coefficient À in the GS equa- tions, can get different degree of cou- pling between u and v according to eq. []] The higher value of À means higher de- gree of coupling between u and v. Given the same initial conditions uo(x) and oo(x), the outputs with different 0.2, 0.4, 0.6, 0.8, 1) are shown in Fig. It shows that as À increases, all the mod- els perform worse. For solving (z). compared with at X 1.2 errors at
CMENNO MIVTY Paliy
Relatve eree
(i.e.. A
4
spochs
0.2, the relative of the models in- crease by 22.0 % (CMWNO): 459.6% (MWT)): 105.9% 9% (MWT.): 107.4% (FNO.); 146.7% (Padéæ). In terms of U(r,7), the numbers are 11.6% (CMWNO); 326.8% (MWT,) 34.5% (MWT.): 44.8% (FNO,); 44.3% (Padé,). As we see, the MWT, works the worst since the model cannot learn the interaction between u and v. The models learning coupled operators through concatenated data works better than the single model but still do not perform well on high coupling data. On the contrary, our CMWNO outperforms well consistently with both low high coupling coefficient, which indicates that our architecture is able decouple the coupled kernels.
Figure 3: learning for neural operators.
Relative 7,2 vs epochs
Varying initial conditions In addition to experimenting with both the initial conditions uo (2:) and vo(x) generated in the GRF as marked (U-GRF, V-GRF), we also perform the models on (U-Rand, GRF). The numerical results are shown in Table 3 (see Appendix F). Our CMWNO achieves the lowest relative L2 error on both u and v with 3.X and 2.X improvements respectively. We provide sample of initial conditions in Fig (see Appendix E)), and Fig. 5 shows its predicted outputs from models CMWNO. MWT, and MWT.. It shows that our proposed CMWNO can give a precise prediction in smooth way while MWT, and MWT, can only fit the true curve roughly.
0-2
t=0.4
t=0.6
t=0.8
Models
P
P
P
CMWNO
0.00083
0.00073
0.00154
0.00252
0.00543
0.00467
0.02417
0.00305
CFNO
0.00767
0.00162
0.00890
0.00638
0.02011
0.01029
0.04780
0.00744
MWT
0.00328
0.00646
0.00916
0.02244
0.02245
0.02768
0.06011
0.01622
FNO. 0.00241
0.00278
0.00473
0.00667
0.01329
0.01096
0.04950
0.00818
Padé
0.00213
0.00320
0.00473
0.01307
0.01189
0.02466
0.03676
0.01171
Table 2: The relative error for predicting p(u, (), with 0.2, 0.4, 0.6, and 0.8.
8

---

Published as a conference paper at ICLR 2023
0.3
MWT CMWNO
0.050
0.2
Tree
0.1
kk.t)
0.000
-0.1
40.025
MWT CMWNO
0.050
Thm
-0.3
400
800
1000
Samples
Samples
Figure 5: The output of GS couple equations at the initial condition (U- (U-Rand, V-GRF). (Left) The predicted output of the models to u(z, 1). (Right) The predicted output of the models to v(z, 1).
3.2 MEAN FIELD GAME PROBLEM
For local interactions, directly discretizing interaction terms is economical. However, non-local MFG requires each player in making decisions to take into account the global information rather than local information, which will increase the amount of computation in the process of calculation. other words, we need matrix multiplication on full grid to calculate the interaction terms by evaluating the expressions S., (xx, yly t)dy. this work, we propose more general framework, CMWNO, to model the interactions in the Wavelet space and the results show that our model can be used to deal with the coupled systems. Here we solve the non- non-local MFG which can be characterized as:
Oeplar 1) (o(a,
[0, 1], (0, 1)
Ōep((,, () ) llo 011 S. K(x, v) K(x,v) O) O, a x [0,1)] (0,
(12)
where p(x, f) is the density distribution of the players, and c0x t) is the cost function. In a forward- forward MFG setting (Gomes & Sedjro| (2017). we can obtain the value of p(z, 0) and ş(x 0). We aim to learn the operators: (i) mapping the initial condition p(x; 0) to the solution p(z; ( 7) with
the interference of (x 1): (ii) mapping the initial condition sp(r, 0) to the solution y( = r) con-
sidering the interference of p(x. t). To obtain the datasets, generate p(z, 0); p(z, f 1) by using the random functions in chebfun package with the wavelength parameter 7 = 0.3; 0,1, respectively. The coupled equations are numerically solved by the primal-dual hybrid gradient (PDHG) algorithm (Briceno-AMe -arice et al. 2019 2018) with the resolution s 256. The initial conditions of p(x; 0) and (0 0) are used the imput while the p(x, t) and (2 (t 0.2, 0.4, 0.6, 0.8) taken as the output.
We perform all the models working for coupled datasets mentioned above to solve this MFG coupled PDEs, and the results with different i. are shown in Table Compared to the existing model with the best results, our proposed CMWNO yields 34.2% 67.4% improvements in terms of p and 57.4% ~ 73.7% in terms of & with respect to the relative L2 error. It is worth noting that MWT, performs the worst in most cases which indicates that the interactions between P and yp can not be leamed through single multiwavelet kernel. By interacting two kernels in the Wavelet space after decomposition steps, our proposed CMWNO can better decouple the interactions between p and yp to solve the MFG PDEs.
4 CONCLUSION
In this work, propose a coupled multiwavelets neural operator using multiwavelet discretization of the spatial domain. Solving for coupled equations requires an information entanglement across operators for individual process. We found that combining operators in the projected domain of multiwavelets is effective. Numerical experiments using representative coupled PDEs including Gray-Scott and mean field game problem show that coupling mechanism effectively learns the two processes in comparison with standalone operators.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGEMENT
We are thankful to the anonymous reviewers for providing their valuable feedback which improved our manuscript. We would also like thank Dr. Justinian Rosca and Lisang Ding for their valuable feedback. We gratefully acknowledge the support by the National Science Foundation Career award under Grant No. Syber-Physical Systems/ 145360,,,60, the NSF award under Grant ICF-1837131, MCB-1936775, CNS-193200 the U.S. Army Research Office (ARO), the Defense Advanced Research Projects Agency (DARPA) Young Faculty Award and DARPA Director Award under Grant No. N66001-17-1-4044, an Intel faculty award, the Okawa Foundation award, Northrop Grumman grant, and Google cloud program. part of this work used the Extreme Science and Engineering Discovery Environment (XSEDE), which is supported by National Science Foundation grant number ACI-1548562. R.B. has been supported in part by a NSF award under grant DMS- RMS-2108900 and by the Simons Foundation. The views, opinions, and/or findings contained in this article are those of the authors and should not be interpreted as representing the official views or policies, either expressed or implied by the Defense Advanced Research Projects Agency, the Army Research Office, the Department of Defense, or the National Science Foundation.
10

---

Published as a conference paper at ICLR 2023
REFERENCES
Yves Achdou and Italo Capuzzo-Dolcetta, Mean field games: numerical methods. SIAM Journal on Numerical Analysis, (133::116--116 2010.
Yves Achdou, Guy Barles, Hitoshi Ishii, and Grigorii Lazarevich Litvinov. Hamilton-jacobi equations: approximations, numerical analysis and upplications. 2013a.
Yves Achdou, Fabio Camilli, and Italo Capuzzo-Dolcetta. Mean field games: convergence of finite difference method. SIAM Journal on Numerical Analysis, 510.,,55555) (1))22585-2612, 2013b.
Yves Achdou, Francisco J Buera, Jean-Michel Lasry, Louise-Louis Lions, and Benjamin Moll. Partial differential equation models in macroeconomics. Philosophical Transactions the Royal Society A: Mathematical, Physical and Engineering Sciences, 372(2028)22113337) 2014.
Yves Achdou, Jiequn Han, Jean-Michel Lasry, Pierre-Louis Lions, and Benjamin Moll. Income and wealth distribution in macroeconomics: A continuous-time approach. The review of economic studies, 89(1):45-86, 2022.
B. Alpert, G. Beylkin, Gines, and Vozovoi. Adaptive solution of partial differential equations in multiwavelet bases. Journal of Computational Physics, 182(1):149-199)) 2002a. ISSN 002 19991.
Beylkin Alpert, Gregory Beylkin, David Gines, and Lev Vozovoi. Adaptive solution of partial differential equations in multiwavelet bases. Journal of Computational Physics, 182(1)::4--190, 2002b.
Bradley Alpert, Gregory Beylkin, Ronald Coifmaan and Vladimir Rokhlin. Wavelet-like bases for the fast solution of second-kind integral equations, SIAM journal on Scientific Computing, 14(1): 159, 1993.
Bradley K. Alpert. A class of bases in L for the sparse representation of integral operators. SIAM Journal Mathematical Analysis, 24(1):246-205 1993a. doi: 10.1137/0524016.
Bradley K Alpert. A class of bases in 1'2 for the sparse representation integral operators. SIAM journal on Mathematical Analysis, 24(1):246-261 1993b.
Jean-David Benamou and Guillaume Carlier. Augmented lagrangian methods for transport opti- mization, mean field games and degenerate elliptic equations. Journal of Optimization Theory and Applications, 167(1):1-26, 2015.
Jean-David Benamou, Guillaume Carlier, and Filippo Santambrogio. Variational mean field games. In Active Particles, Volume 1, pp. 141-171. Springer, 2017.
Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. Scheduled sampling for sequence prediction with recurrent neural networks. Advances in neural information processing systems, 28, 2015.
Gregory Beylkin, Ronald Coifman, and Vladimir Rokhlin. Fast wavelet transforms and numerical algorithms i. Communications on pure and applied mathematics, 44(2):141-183, 1991.
Kaushik Bhattacharya, Bamdad Hosseini, Nikola B. Kovachki, and Andrew M. Stuart. Model reduction and neural networks for parametric pdes, 2020.
Fabrice Bonjean and Gary SE Lagerloef. Diagnostic model and analysis of the surface currents in the tropical pacific ocean. Journal of Physical Oceanography, 32(10):2938-2988 2002.
Luis Priceno-Arias Dante Kalise, Ziad Kobeissi, Mathieu Lauriere, Mateos González, and Francisco J Silva. On the implementation of primal-dual algorithm for second order time- dependent mean field games with local couplings. ESAIM: Proceedings and Surveys, 65:330-348, 2019.
Luis M Briceno-Arias Dante Kalise, and Francisco J Silva. Proximal methods for stationary mean field games with local couplings. SIAM Journal on Control and Optimization, 56(2):801-836. 2018.
11

---

Published as a conference paper at ICLR 2023
Defu Cao, Yujing Wang. Juanyong Duan, Ce Zhang, Xia Zhu, Congrui Huang. Yunhai Tong, Bixiong Xu, Jing Bai, Jie Tong, et al. Spectral temporal graph neural network for multivariate time-series forecasting. Advances in neural information processing systems, 33:17766-17778, 2020.
Defu Cao, Jiachen Li, Hengbo Ma, and Masayoshi Tomizuka. Spectral temporal graph neural network for trajectory prediction. 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 1839-1845 IEEE, 2021.
Defu Cao, Yousef EI- El-Laham, Loc Trinh, Svitlana Vyetrenko, and Yan Liu, A synthetic limit order book dataset for benchmarking forecasting algorithms under distributional shift. In NeurlPS 2022 Workshop Distribution Shifts: Connecting Methods and Applications, 2022. URL https
//openreview net ferest/frmmmddecmm
1BBV
Tianping Chen and Hong Chen. Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application dynamical systems. IEEE Transactions Neural Networks, (44):911-911 1995.
Jeffrey M. Connors, Jason S. Howell, and William J. Layton. Partitioned time stepping for a parabolic two domain problem. SIAM Journal on Numerical Analysis, 47(5):3526-3444 2009. doi: 10. 10.1000087740891
Steven M Cox and Paul C Matthews. Exponential time differencing for stiff systems. Journal of Computational Physics, 176(2)430-455, 2002.
Antonio De Paola, Vincenzo Trovato, David Angeli, and Goran Strbac. A mean field game approach for distributed control of thermostatic loads acting in simultancous energy-frequency response markets. IEEE Transactions on Smart Grid, 10(6):5987-5999 2019.
T. A Driscoll, N. Hale, and L. N. Trefethen. Chebfun Guide. Pafnuty Publications, 2014.
Meysam Ghanavati, Animesh Chakravarthy, and Prathyush P Menon. Analysis of automotive cyber- attacks on highways using partial differential equation models. IEEB Transactions on Control of Network Systems, 1(4):1775-1786, 2017.
David Gilbarg, Neil S Trudinger, David Gilbarg, and NS Trudinger. Elliptic partial differential equations of second order, volume 224. Springer, 1977.
Diogo Gomes and Marc Sedjro. One-dimensional, forward-forward mean-field games with congestion. arXiv preprint arXiv: 10w.11003.0022 2017.
Diogo A Gomes et al. A mean-field game approach price formation. Dynamic Games and Applications, 11(1)22-53, 2021.
Somdatta Goswami, Aniruddha Bora, Yue Yu, and George Em Kariiadakis. Physics-informed neural operators, arXiv preprint arXiv:2207.05745 2022.
Olivier Guéant, Jean-Michel Lasry, and Pierre-Louis Lions. Mean field games and applications. In PParis-Prnneeton lectures on mathematical finance 2010, pp. 205-266. Springer, 2011.
Gaurav Gupta, Xiongye Xiao, Radu Balan, and Paul Bogdan. Non-linear operator approximations for initial value problems. In International Conference Learning Representations, 2021a.
Gaurav Gupta, Xiongye Xiao, and Paul Bogdan. Multiwavelet- operator learning for differential equations, 2021b.
Craig Henriquez. Simulating the electrical behavior of cardiac tissue using the bidomain model. Critical reviews in biomedical engineering, 21(1):1-77, 1993.
Minyi Huang, Roland P Malhamé. and Peter E Caines. Large population stochastic dynamic games: closed-loop mekcan-vlason systems and the nash certainty equivalence principle. Communications in Information & Systems, 6(3):221 1-22,, 2006.
Minyi Huang, Peter E. Caines, and Roland P. Malhame. Large-population cost-coupled lgg problems with nonuniform agents: Individual -mss behavior and decentralized e-nash equilibria. IEEE Transactions Automatic Control, 52(9):150-1157 2007. doi: 10. 109/TAC.2007.904455.
12

---

Published as a conference paper at ICLR 2023
Xuancheng Huang. Sebastian Jaimungal, and Mojtaba Nourian. Mean-field game strategies for optimal execution. Applied Mathematical Finance, (662):153-185, 2019.
Nikola Kovachki, Zongyi Li, Burigede Liu, Kamyar Arizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, and A Anima Anandkumar. Neural operator: Learning maps between function spaces. arXiv preprint arXiv:2108.0844
2021.
David Lacasse, André Garon, and Dominique Pelletier. Mechanical hemolysis in blood flow: user- independent predictions with the solution of partial differential equation. Computer Methods in Biomechanics and Biomedical Engineering, 10(1):1-122 2007.
Jean-Michel Lasry and Pierre-Louis Lions. Jeux à champ moyen. i-le cas stationnaire. Comptes Rendus Mathématique, 343(9):61--625, 2006.
Jean-Michel Lasry and Pierre-Louis Lions. Mean field games. Japanese journal of mathematics, (1):229-260, 2007.
Jorge A Laval and Ludovic Leclereq. The hamilton jacobi partial differential equation and the three representations of traffic flow. Transportation Research Part B: Methodological, 52:17 12:17-30, 2013.
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations, 2020a.
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Neural operator: Graph kernel network for partial differential equations, 2020b.
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Andrew Stuart, Kaushik Bhattacharya, and Anima Anandkumar. Multipole graph neural operator for parametric partial differential equations. In Advances in Neural Information Processing Systems, volume 33, pp. 6755 6766, 2020c.
Siting Liu and Levon Nurbekyan. Splitting methods for a class of non-potential mean field games. arXiv preprint arXiv:2007.0009: 2020.
Siting Liu, Matthew Jacobs, Wuchen Li, Levon Nurbekyan, and Stanley J Osher. Computational methods for first-order nonlocal mean field games with applications. SIAM Journal Numerical Analysis, 59(5):2639-2668 2021.
Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature Machine Intelligence, 3(3):218-22) 2021.
Lu Lu, Xuhui Meng, Shengze Cai, Zhiping Mao, Somdatta Goswami, Zhongqiang Zhang, and George Em Karniadakis. A comprehensive and fair comparison of two neural operators (with prac- tical extensions) based on fair data. Computer Methods Applied Mechanics and Engincering. 393: 14777, 2022. 2022.
Chuizheng Meng, Sungyong Seo, Defu Cao, Sam Griesemer, and Yan Liu. When physics meets ma- chine learning: survey of physics-informed machine learning. arXiv preprint arXiv:2203.16797, 2022.
Vinod Nair and Geoffrey E Hinton. Rectified linear units improve restricted boltzmann machines. In Ieml, 2010.
Ravi G. Patel, Nathaniel A. Trask, Mitchell A. Wood, and Eric C. Cyr. A physics- innfores operator regression framework for extracting data-driven continuum models. Computer Methods in Applied Mechanics and Engineering, 13550, 2021. ISSN 0045-7825. doi: https://doi.org/10. 1016/..
cma.2020. 133000
Jaideep Pathak, Shashank Subramanian, Peter Harrington, Sanjeev Raja, Ashesh Chattopadhyay, Morteza Mardani, Thorsten Kurth, David Hall, Zongyi Li, Kamyar Azizzadenesheli, et al. Fourcast- net: A global data-driven nigh-resolution weather model using adaptive fourier neural operators. arXiv preprint arXiv:2202.1 1214, 2022.
13

---

Published as a conference paper at ICLR 2023
Mohammadreza Radmanesh, Manish Kumar, and Donald French. Partial differential equation- trajectory planning for multiple unmanned air vehicles dynamic and uncertain environments. Journal of Dynamic Systems, Mcasurement, and Control, 142(4). 2020.
Kimberly Stachenfeld, Drummond Fielding, Dmitrii Kochkov, Miles Cranmer, Tobias Pfaff, Jonathan Godwin, Can Cui, Shirley Ho, Peter Battaglia, and Alvaro Sanzhez-Gonzalez. Learned coarse models for efficient turbulence simulation. arXiv preprint arXiv:2112.152275 2021.
Chen Tang, Lin Han, Hongwei Ren, Tao Gao, Zhifang Wang, and Ke Tang. The oriented-couple partial differential equations for filtering in wrapped phase patterns. Optics Express, 17(7):5006-5617, 2009.
L. N. Trefethen and K. Embree. The (unfinished) pde coffee table book, 2001. URL https 7/people www.s.tthss..mmm. wwt.e//effeaeeee//eecttt......a.
Gege Wen, Zongyi Li, Kamyar Azizzadenesheli, Anima Anandkumar. and Sally M Benson. U-fno-an an enhanced fourier neural operator-based deep-learning model for multiphase flow. Advances in Water Resources, 163:10418 (0000001100000 2022.
Xiongye Xiao, Hanlong Chen, and Paul Bogdan. Deciphering the generating rules and functionalities of complex networks. Scientific reports, 11(1):22964 2021.
Yuankun Xue and Paul Bogdan. Constructing compact causal mathematical models for complex dynamics. In 2017 ACM/EEEE 8th International Conference on Cyber-Physical Systems (ICCPS), pp. 97-108, April 2017.
Chenzhong Yin, Xiongye Xiao, Valeriu Balaban, Mikhail Kandel, Young Jae Lee, Gabriel Popescu, and Paul Bogdan. Network science characteristics of brain- derived neuronal cultures deciphered from quantitative phase imaging data. Scientific reports, 10(1):15078, 2020.
Yizhou Zhang, Defu Cao, and Yan Liu. Counterfactual neural temporal point process for estimating causal influence of misinformation social media. In Advances in Neural Information Processing Systems, 2022.
14

---

Published as a conference paper at ICLR 2023
121+1
W
1i+1 (U
1)
Rec
Roc
Rec
Puth
Dec
Dec
D
Doc
Dice-1
Rec
Rec
Roc
1/41
U
W
Rec
Rec
Rec
Path mn
(V U
Dec
Dec
Dec
Figure 6: The scalable dice strategy with multiple coupled kernels. For each sample of coupled variables, one only needs to go through specific path (round diagonal corner rectangle). For example, we get dice equal in this from right, where darker block indicates the operator we want to update the lighter blocks provide decomposition information from the fixed operator. Note that the updated operator can help to update other operators at the same stage that occurs after the specific operator. Thus, for n kernels, we have n operators that need to be updated and m paths that can be selected, where A,, and A is the function of all permutations.
that this specific sample will go through Path 1. Inside each path, the order of updating
A RELATED WORK
The operator network can be made up of several neural networks, which aims approximate any function defined as an input in one network and evaluated at the locations specified in the target network (Lu et al., 2022). Chen Chen propose the universal approximation theory of operators for a single layer, which has led to several research works recently. Among them, DeepONet (Lu al. 202T) first applies deep neural network on the universal approximation theorem learn nonlinear operators. After that, Fourier Neural Operator (FNO) formulates the operator regression by parameterizing the integral kernel directly in Fourier space (Li et al., 2020a)). Several works take advantage of FNO's ability to efficiently solve PDEs to design models for applications in practical chaotic systems such as turbulence simulation (Stachenfeld et al. (022), multiphase flow simulation (Wen et al. 2022), and weather forecasting (Pathak et al. 2022) In addition, would like to highlight that neural operators are able to tackle the more fundamental problems in time series analysis (Cao al. 2021 Zhang et al. 2022 Cao al.) (2020) and easily finds many application in transportation, healthcare, manufacturing, finance (Cao et al. 2022). etc. Guppt et al. introduce multiwavelet-based neural operator that compresses the associated operator's kernel using fine-grained wavelets and the same group further proposes non-linear operator approximation for initial value problems. Besides, ahysics-informed neural network and machine learning methods provide new research direction to equip specific physics information into neural operators Goswami al. 2022); Meng et al. (2022). For more information of neural operators, please refer to the new survey paper: (Kovachki et al. 2021). Another related research line of our work is coupled PDEs Tang et al. (2009), which is usually discussed the form MFGs (Benamou & Carlier 2015 2017 Briceno-
Benamou et al.
al. 2019 2018 Liu & Nurbekyan, 2020 Liu et al. 2021 1). In addition, the interaction between terms in complex systems Xue Bogdan (2017); Xiao et al.
(2021); Yin et al. (2020) can be characterized by coupled PDEs. Combining those research lines, this is the first work which proposes a decoupled multiwavelet- based neural operator learning schema
to solve coupled PDE problems.
15

---

Published as a conference paper at ICLR 2023
B REPRODUCIBILITY & CODE AVAILABILITY
Architecture description in detail: The CMWNO model in Figure D presented in the form of a recurrent cell. In the decomposition stage, the input data at each iteration are U.s (z(i-1) (Vo,5++) which
then gets transformed into U. U a (Vo.a: Vo,a) using the filters H, G. At the same iteration, we also obtain the corresponding outputs Ti u.d and Ti u.s (T and T. The same process repeated in the next iteration but now with using Uj.. (V (obtained from the previous step) the input, thus this makes a recurrent chain of operations and is a kind of ladder-down operation. The loop is repeated till we reach the L-th scale (coarsest scale) at which the final operation of T is applied according to eqs [9] The trainable neural networks layer in this stage is composed of (A,, Bu: C.., T.,) and (Ag, Bey Cu, To). We two cascading neural network layers with normal Xavier initialization to handle ID coupled equations in our experiments. Moreover, { Bx. C. are all one-layer CNNs with the ReLU & Hinton (2010) activation functionn followed by linear layer, where CNN's kernel size equals 3, stride equals I, and padding size equals 1. The input channel of CNN equals the feature number and the output channel is set to 128 in all the experiments. single k x k linear layer with &- = 4 suggested by (uupta et al. (2021b). In the reconstruction stage (which is ladder-up), iteratively, the outputs decomposition part 7 add and TW.s (T² and T. are first combined by the dice strategy in section 2.4 and then we use a reconstruction filter H, G. to obtain the finer scales U(i+1) (V:(+1), ), and finally, UH., (V...) is the finest scale of the output.
addition, (T.) is
Our code to the experiments be found at htttps::/githuub.com CMWNO/]
thub. com cowp:/bb.ccmm/ooo.........ocoo
C MULTIWAVELET BASES
C.I MULTIRESOLUTION ANALYSIS
The basic idea MRA is to establish a preliminary basis in a subspace Vo of (I), and then use simple scaling and translation transformations to expand the basis of the subspace Vo into (IR for analysis on multiscales. Multiwavelets further this operation by using a class of orthogonal polynomials (OPs), in our case, we use Legendre polynomials for an efficient representation over finite interval (Alpert al. 2002a).
For k E Z and n E N. the space of piecewise polynomial functions is defined as: V the restriction of f the interval (2-*1, 20-",, + 1)) is a polynomial of degree k, for all 0, 1,... 2 - 1, and f vanishes elsewhere). Therefore, the space Vk has dimension 2n and each subspace V is contained in V shown as v c v c Vk
V.. c c.... Given a basis Po. PI.... - of Vt. the space V is spanned by 2" functions obtained from Po. PI...., Pk-1
of Vg by shifts and scales
()(() = 2n/2 22 1), j= 1,.,,,..,,kk 1, 1 0,1......2 2"
(13)
The functions şo. #1 1h-1 also called scaling functions which can project function to the approach space Vb.
C.2 MULTIWAVELETS
The multiwavelet subspace W defined as the orthogonal complement of V in V n++1+ such that
V W
vh+
v
W
(14)
and W has dimension k.. Therefore, the decomposition be obtained
V
W
W
+ Wk
(15)
To form the orthogonal bases for W n. a class of bases is constructed for (RR) Each basis consists of translates and dilates of finite of functions th ... the shown follows:
(x) 2n 1b 1), 0, 1, ...., 1 0,
2 1.
(16)
16

---

Published as a conference paper at ICLR 2023
where the wavelet functions 111,... ................................ the are piecewise polynomial and orthogonal low-order poly- nomials (vanishing moments):
I. x (2)) = 0, == l 1.
(17)
Here we restrict our attention to the interval [0, 1] R; however, the transformation to any finite interval [p. 4l could be directly obtained by the appropriate translates and dilates.
D LEGENDRE POLYNOMIALS
The Legendre polynomials are defined with respect (w...t)) uniform weight function wz (2) 1 for -1 & x & or wr. (z) = (z) such that
((()))(o)do=
21+1 i= j. i
(18)
For our work, we shift and scale the Legendre polynomials so they are defined over [0, 1] as P2 1), and the corresponding weight function wt. (2x 1). The Legendre polynomials satisfy the following recurrence relationships
(z) = (2i 1)x - (i - 1));;2(2)), (2i + ()))
=
which allows the expression of derivatives as linear combination of lower-degree polynomials itself as follows:
(f(x) = (2i 1) + (2i - P + ....
(19)
where the summation ends either Po(z)) or P1(x), with Po(z) and Pi(x) x.
A set of orthonormal basis of the space of polynomials with degree < d defined over the interval
[0, 1] is obtained using shifted Legendre polynomials such that
Ø = 2i + IP,(2r 1);
w.r.t. weight function w(x) w2(2x 1), such that
(o,, b5), [ (1()),,())dr = =dg-
The basis for Vi are chosen normalized shifted Legendre polynomials of degree upto & w.r.t.
weight function wy (2x 1) = 1z)) from Section For example, the first three bases are
<o(x) 1,
d (z) v3(2x 1),
(20)
do (x) = v5(6x² - 6x + 1), 0E
For deriving set of basis U); of W using GSO, we need to evaluate the integrals efficiently, which could be achieved using the Gaussian quadrature.
17

---

Published as a conference paper at ICLR 2023
1.5
W Mmmm Mn
1.0-
0.5
0.0
0.0
0.5
-0.2
-1.0
0g(
200
400
600
800
1000
200
400
600
800
1000
Samples
Samples
Figure 7: The sample of the initial conditions at (U-Rand, V-GRF).
0.3
sample1 sample2
0.2
61--
0.4
0.2
0.0
-0.2
6.3
sumplc sample2
-0.4
100
200
300
600
300
800
700
100
100
1000
100
200
300
400
550
650
750
900
900
1000
samples
samples
0.81
0.15
sample] sampli2
0.6-
0.4
0.2
0.0
-0.2
-0.0
-0.5
-0.6
samplel sample2
0.70
166
300
410
610
700
500
1000
-0.9
400
550
600
750
800
900
1000
samples
samples
Figure 8: The input/output samples the initial conditions (U-GRF, V-GRF).
E INITIAL STATES SAMPLES
In this section, we use Fig. to illustrate the initial states (u-rand and v-grf) for Gray-scott equations; Fig. to exhibit the different initial states (u-grf and v-grf) and solutions for Gray-scott equations; use Fig. to show the samples of p(x, t) and Φ(z, t) at different time in our non-local MFG case.
F ADDITIONAL RESULTS
F. ADDITIONAL RESULTS FOR GRAY-SCOIT (GS) EQUATIONS
In this section, we provide more results on the L. errors with different initial conditions Table 3
F.2 BELLOUSOV-ZHAABTTISSKY (BZ) EQUATIONS
Adapted from the Belousov
Belousov-ZZabotinge Zhabotinsky dynamic system, the coupled Aaabuuov-ZZhbotinsky (BZ)
equations in 21 describe reaction-diffusiae process with three species Driscoll et al. (201)) For
18

---

Published as a conference paper at ICLR 2023
samplel sample2
133
-0.305
((x--0)
-0.310
0.10
sample] sample2
-0.20
0.7
102
320
103
samples
samples
1.35
sample]
sample?
1.30
-0.152
1.05
0.14
sample! sample?
-0.552
0.85
samples
samples
sample] sample2
0.057
0.96
-0.055
0.059
0.00
0.92
sample sample2
50
150
130
96
100
156
200
250
samples
samples
0.96
sample2
0.04
0.00
0.048
(001-06))
0.95
0.047
0.92
0.044
0.00
0.943
sample sample
0.042
80
s6o
150
150
$5
103
155
samples
samples
1.090
0.124
1.025
samplc2
0.122
1.000
0.95
0.900
0.925
0.168
0.900
0.665
sample1 samplec
0.75
samples
samples
Figure 9: The sample input/output for and Φ. The top two figures are the initial conditions of and & as the input. The other figures are the output of and with different time I.
19

---

Published as a conference paper at ICLR 2023
s=256
s=512
1024
Models
u
V
V
u
CMWNO
0.00937
0.00856
0.00866
0.00726
0.00819
0.00769
MWT
0.09353
0.05203
0.09137
0.05121
0.09381
0.05161
MWT. 0.02997
0.01804
0.02799
0.01832
0.02619
0.01866
FNO. 0,02496
0.01511
0.02433
0.01539
0.02335
0,01516
2adé,
0,04605
0.01449
0.04605
0.01479
0.04747
0.01509
Table 3: Gray-Scott (GS) equation benchmarks for different input resolution at initial condition (U-Rand, V- GRF). The relative L2 errors are shown for each model.
given field u(æ, t); v(x, t); w(x t), the BZ coupled equations take the form:
Ōzu(,, 1) ei + u tut we uuu xe (0, 1), (0,0.2]
Gul(,, 1) e2 (Iz +w - - uc, ),w((, t) 13 w(z + 2, xe 1 (0,0.2]
-
(0, (),1) (0,0.2)
(21)
where 10- = x 10 2 10 Our goal is to learn the operators mapping the initial condition of each variable to the solution with the interference of other variables. The generated by using the smooth random functions (Rand) in chebfin package Driscoll et al. 2014). For the initial conditions u(z, 0); v(z 0); w x, 0), we set the parameters 0.3; 0.2; 0.11 respectively. Given the initial conditions, we solve the equations using the fourth- order stiff time-stepping scheme named as ETDRK4 (Cox & Matthews (002) with a resolution of 210 and sub-sample this data obtain the datasets with the a resolution of 2 The results of the experiments on BZ coupled equations with different resolutions (i.e., 2=256, 1024) are shown in Table
initial conditions
4
s=256
s= 1024
Models
CMWNO
0.02306
0.02727
0.02412
0.01994
0.02505
0.02481
CENO
0.04294
0.05738
0.05856
0.04087
0.06056
0.06148
MWT,
0,28703
0.34383
0.51297
0.27376
0.33410
0.53644
MWT. 0.04654
0.06489
0.07371
0.04723
0.06537
0.07287
FNO, 0.03575
0.09662
0.09302
0.03611
0.09371
0.09119
Padé 0.04361
0.06766
0.08508
0.04479
0.06960
0.08486
Table 4: Behausow-Zhabotinsky (BZ) equation benchmarks for different input resolution The relative L2 shown for each model.
To handle multiple coupled variables, we apply the dice strategy referring to Fig. 6 to mimic the interaction between u; ug w. As we can see the results, our CMWNO still achieves the new state-of-thee-rt.
20

---

Published as a conference paper at ICLR 2023
MWT, 508754
MWTW.
Pade,
FNO,
CMWNO
Parameters
254377
143957
287425
508754
Table 5: Comparison of model's parameters.
G MODEL COMPARISON
Table comparees in detail the data parameters from CMWNO and baselines. Although the number of parameters of our model is about twice many the second best model (FNO), the performance of our model is improved by 57.68% and 61.41%, separately. In solving the coupled PDE problem, model is the optimal choice in terms of performance and power balance. One of our future efforts to improve model efficiency, which we leave as the further work.
H DISCUSSION ON PINN
We note that neural operators and PINN are two recent deep learning approaches that have gathered the tractions the community. The PINN combines the advantages of data-driven machine learning and physical modeling to train a model that automatically satisfies physical constraints with insufficient training data and has comparable generalization performance to predict important physical parameters of the model while ensuring accuracy. One can incorporate the differential form constraints from PDEs into the design of the loss function of the neural network with automatic differentiation techniques in deep neural networks. In addition, PINN cannot be used directly in a complete data- driven scenario without an exact PDE structure, and PDE function is hard to decided in take compromise approach by relying specific PDE (such
wild upplications, However, as (Connors et al.) (2009)) to design its loss function and using it on different input functions, which should be undesirable.
I DEFAULT NOTATION
A scalar (integer real)
T;
Generic operator neural networks
A,B,
A
A set
The set of real numbers
Ki
The kernel in the opeartor
[a, b]
The real interval including a and b
(a, b]
The real interval excluding but including
V
(flf are polynomials of degree < k defined over interval (2-"1,2-"(1 + 1)) for all 1 = 0, 0, 2n 0 elsewhere)
and assumes
w
Orthogonal space to V such that w + Vk V n+1
T"
Coefficients according to operator
V
Coefficients according to input u. and **
Pn
Projection operator such that P,, H Vk
Qn
Projection operator such that Qn 7a.2 W
Coarsest scale of the multiwavelet transform
21


Reviews:
[{"review_id": "S7C5HID7UG", "reviewer": "Reviewer_iKvj", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes using the multiwavelets framework to solve coupled partial differential equations in a data-driven manner. Multiwavelets are a generalization of standard scalar wavelets but instead use more than one scaling function and the multiwavelet transform is not a straightforward separable extension of the wavelet transform but contains interaction coefficients due to this multivariable choice. \n\nAt the other hand, in scenarios of coupled partial differential equations in which more than one differential equation needs to be solved with interacting terms, the greens function kernels are not independent and hence not amenable to be solved using well-known neural operators like PINNs, Fourier Neural Operators (FNO), etc in a straightforward manner. \n\nThe authors combine the two scenarios and propose the coupled multiwavelet neural operator (CMWNO). Experiments are demonstrated for 2 problems - Gray Scott Equations and the Mean Field Game Problem. The results show good improvements over baseline neural network approaches by a significant margin. \n", "summary_of_the_review": "Overall, I think this paper is quite decent in its methodology and evaluation, and contains an interesting combination of multiwavelets and coupled PDEs. Despite that, I find some ambiguity in the proposed framework, especially regarding the architecture design and assumptions leading to Eq 4 and 5. I’ll wait for the rebuttal to make a more informed opinion, but as of now, it is a borderline rating from me. ", "strength_and_weaknesses": "Strengths \n\n- The high-level motivation combining multiwavelets and coupled PDEs is very interesting and original (to the best of my knowledge) \n- The results are significant when compared to standard learning-based baseline methods \n\n\nWeaknesses and Questions\n\n- My main concern is the assumptions leading to equations 4 and 5. I am not fully on board with why the multiwavelet space automatically guarantees decoupling? I presume this assumption is crucial since the construction of the architecture for CMWNO is hinged on such a separation and solved in an iterative fashion (i.e. fix the coupling dependence and solve for the kernel and vice versa). Despite this, what is the need for multiwavelets at this step? and not simple wavelets or the Fourier transform?  (which can also lead to an interesting baseline) \n- Even though the improvement over shown baselines is good, I am not sure if it is a super convincing comparison since most of these approaches do not seem to be designed for coupled PDE’s. I do feel that the concatenation approach to make these methods work for the coupled system is a fair attempt, but I am not too surprised they don’t work well. \n- The paper focuses on coupling with only two variables. How can the proposed framework be extended for a more sophisticated coupling PDE system with many variables? especially the dice strategy? \n- Broadly, it appears that the positioning of this paper appeals more to a core PDE audience (giving them a new tool with neural networks for complex PDEs like Gray Scott equations, etc) rather than a representation learning conference. However, this paper is not entirely out of place for ICLR, but I feel that the background and motivation for coupled PDEs and Multiwavelets can be made more accessible. For example, a visual schematic, or some visualizations depicting the use cases for multiwavelets and coupled PDEs would go a long way to make everything convincingly self-contained. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "HbYoO02mlT1", "reviewer": "Reviewer_k8FT", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The manuscript presents a neural operator approach for solving coupled PDEs, motivated by multiwavelets. The authors demonstrate some improved accuracy on Gray-Scott and nonlocal mean field game problems. The multiwavelet transform is used to represent the coupled information from the coupled PDEs and a dice strategy is employed to mimic the interaction of the information during the training process. ", "summary_of_the_review": "The authors are proposing a strategy for solving coupled PDEs that sounds reasonable, and uses a multiwavelet discretization of the spatial domain. There are some improvements in accuracy for the Gray-Scott and nonlocal mean field game problems. Given the assumption in  (5), I find it difficult to assess how well this approach will work as a general approach for coupled PDEs. \n", "strength_and_weaknesses": "I am not aware of any existing neural operator approaches for coupled PDEs, and the use of the multiwavelet transform is very reasonable in this context. The dice strategy is a nice observation to deal with the imbalance update problem that could occur for each kernel.  The authors compare their approach with many other reasonable neural operators, and some improved accuracy is observed. \n\nThere's an assumption that one can write down decoupled solution operators that solve for each component of the coupled PDEs in (5). I find it hard to assess if this is a mild assumption or not.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "1jgrdK93OBc", "reviewer": "Reviewer_Zj6R", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors proposed a modification of multiwavelet architecture introduced in [1] for the case of coupled PDEs. Comparisons with the Fourier Neural Operator [2], two versions of the original multiwavelet operator [1], the Pade exponential model [3], and Physics-informed neural networks [4] show that the proposed modification improves relative L2 error.\n\n\n[1] - Gupta G, Xiao X, Bogdan P. Multiwavelet-based operator learning for differential equations. Advances in Neural Information Processing Systems. 2021 Dec 6;34:24048-62.\n\n[2] - Li Z, Kovachki N, Azizzadenesheli K, Liu B, Bhattacharya K, Stuart A, Anandkumar A. Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010.08895. 2020 Oct 18.\n\n[3] - Gupta G, Xiao X, Balan R, Bogdan P. Non-linear operator approximations for initial value problems. In: International Conference on Learning Representations 2021 Sep 29.\n\n[4] - Raissi M, Perdikaris P, Karniadakis GE. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics. 2019 Feb 1;378:686-707.", "summary_of_the_review": "Overall, the paper is well-written and organized and mostly well-explained. Provided experiments are of good quality and models chosen for benchmarks include state-of-the-art architectures. The approach given by the authors is empirically justified and illustrated with (vague) theoretical considerations.\n\nThe main concern of the present reviewer is that the paper does not contain significantly novel techniques. It only proposes a minor correction of architecture from [1].\n\n\n[1] - Gupta G, Xiao X, Bogdan P. Multiwavelet-based operator learning for differential equations. Advances in Neural Information Processing Systems. 2021 Dec 6;34:24048-62.", "strength_and_weaknesses": "Strength\n\n1. Main objective (to solve coupled PDEs in the regression setting) is clearly formulated.\n2. Most of the main claims are reasonably substantiated (for exceptions see below).\n3. Most ideas are clearly explained (for exceptions see below).\n\nWeaknesses\n\n1. Architecture is not described in detail.\n\n    In Section 3 (page 6) authors describe neural networks A, B, C, and T used in multiwavelet neural operators as follows\n    ``In our proposed model, for each operator, the neural networks A, B, and C use single-layered convolutional neural networks while T uses a single linear layer.``\n    \n    This description is incomplete. What are the sizes of convolutions, the strides the number of input and output channels (features)? Are there any nonlinear activations and what are those? Are there multiple multiwavelet kernels (cells) involved and how many of them?\n\n2. The proposed model is a minor modification of [1].\n\n    From the description in the main text and the parameter count in Table 6 (supplementary materials), it looks like the proposed architecture is exactly the same as in [1]. The contribution then is a way how the known architecture can be applied to coupled equations.\n\n3. Awkward notation\n\n    In many places, authors use backslash / to refer the reader to distinct cases. This does not look appropriate when mathematical expressions are involved since / is also used to denote division, quotient group, and other mathematical operations.  For a concrete example see page 5, text right before equation 9.\n\n4. The first claimed contribution is unjustified.\n\n    ``For coupled differential equations, we propose a coupled neural operator learning scheme, named CMWNO. To the best of our knowledge, CMWNO is the first work using pure data-driven method to solve coupled differential equations.``\n    \n    It is not clear what the authors mean. Any work that solves the Navier-Stokes equation using machine learning deals with coupled PDEs since velocities are coupled. See [2] for a concrete example. Besides that, there are works on multiphase flows [3], weather forecasts [4], e.t.c. Essentially, there are hundreds if not thousands of papers dealing with coupled PDEs in regression and physics-informed setups.\n\n5. The choice of a physics-informed neural network (PINN) as a benchmark is questionable.\n\n    PINN provides a way to solve PDE given initial data. As the present reviewer understands vanilla PINN takes only coordinates for input and produces the solution. That is, PINN is not a neural operator in contrast with all other considered architectures. How PINN is applied precisely in this context? Is it retrained for each new input function? In the opinion of the present reviewer, PINN adds little to the understanding of how the accuracy of the presented model compares to the state-of-the-art.\n\n[1] - Gupta G, Xiao X, Bogdan P. Multiwavelet-based operator learning for differential equations. Advances in Neural Information Processing Systems. 2021 Dec 6;34:24048-62.\n\n[2] - Stachenfeld K, Fielding DB, Kochkov D, Cranmer M, Pfaff T, Godwin J, Cui C, Ho S, Battaglia P, Sanchez-Gonzalez A. Learned Coarse Models for Efficient Turbulence Simulation. arXiv preprint arXiv:2112.15275. 2021 Dec 31.\n\n[3] - Wen G, Li Z, Azizzadenesheli K, Anandkumar A, Benson SM. U-FNO—An enhanced Fourier neural operator-based deep-learning model for multiphase flow. Advances in Water Resources. 2022 May 1;163:104180.\n\n[4] - Pathak J, Subramanian S, Harrington P, Raja S, Chattopadhyay A, Mardani M, Kurth T, Hall D, Li Z, Azizzadenesheli K, Hassanzadeh P. Fourcastnet: A global data-driven high-resolution weather model using adaptive Fourier neural operators. arXiv preprint arXiv:2202.11214. 2022 Feb 22.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster