A Unified Hard-Constraint Framework for Solving Geometrically Complex PDEs
Authors: Songming Liu
Zhongkai Hao
Chengyang Ying
Hang Su
Jun Zhu
Ze Cheng
Venue: NeurIPS 2022 Accept (2022)
Keywords: PDE
neural network
physics

Abstract: We present a unified hard-constraint framework for solving geometrically complex PDEs with neural networks, where the most commonly used Dirichlet, Neumann, and Robin boundary conditions (BCs) are considered. Specifically, we first introduce the "extra fields'' from the mixed finite element method to reformulate the PDEs so as to equivalently transform the three types of BCs into linear forms. Based on the reformulation, we derive the general solutions of the BCs analytically, which are employed to construct an ansatz that automatically satisfies the BCs. With such a framework, we can train the neural networks without adding extra loss terms and thus efficiently handle geometrically complex PDEs, alleviating the unbalanced competition between the loss terms corresponding to the BCs and PDEs. We theoretically demonstrate that the "extra fields'' can stabilize the training process. Experimental results on real-world geometrically complex PDEs showcase the effectiveness of our method compared with state-of-the-art baselines.

A Unified Hard-Constraint Framework for Solving Geometrically Complex PDEs
Songming Liu', Zhongkai Hao', Chengyang Ying', Hang Su 12 Jun Zhu 1.2 Ze Cheng?
Deett. of Comp. Sci. and Tech., Institute for AI, THBI Lab, BNRist Center, Tsinghua-Bose Bosch Joint ML Center, Tsinghua University Peng Cheng Laboratory; Pazhou Laboratory (Huangpu), Guangzhou, China Bosch Center for Artificial Intelligence csuastt@gmail. com
Abstract
We present a unified hard-constraint framework for solving geometrically complex PDEs with neural networks, where the most commonly used Dirichlet, Neumann, and Robin boundary conditions (BCs) are considered. Specifically, we first in- troduce the "extra fields" from the mixed finite element method to reformulate the PDEs so as to equivalently transform the three types BCs into linear forms. Based on the reformulation, we derive the general solutions of the BCs analytically, which are employed to construct an ansatz that automatically satisfies the BCs. With such a framework, we can train the neural networks without adding extra loss terms and thus efficiently handle geometrically complex PDEs, alleviating the unbalanced competition between the loss terms corresponding to the BCs and PDEs. We theoretically demonstrate that the "extra fields" can stabilize the training process. Experimental results on real-wwe real-world geometrically complex PDEs showcase the effectiveness of our method compared with tatte-of-the-ar baselines.
Introduction
are characterized by partial differential equations (PDEs) with the solution constrained by boundary conditions (BCs) that are derived from the physical system of the problem. Among all types of BCs, Dirichlet, Neumann, and Robin are the most commonly used [33]. Figure[]ivv an illustrative example on these three types of BCs. Furthermore, in practical problems, physical systems (where the geometry of the definition domain is irregular or has complex structures, c.g., a lithium-ion battery (II] heat sink [40), etc), leading become a challenging problem shared by both the scientific and industrial communities.
Many fundamental problems in science and engineering (e.g.. 22 19 28
be very geometrically complex
large number of BCs. How to solve such PDEs has
The field of solving PDEs with neural networks has history of more than 20 years 6 34 36 Such methods are intrinsically mesh-free and therefore can handle nigh-dimensional as well as geometrically complex problems more efficiently compared with traditional mesh-based methods, like the finite element method (FEM). Physical-informed neural network (PINN) (25) one of most influential works, where the neural network is trained in the way of taking the residuals of both the PDEs and the BCs as multiple terms of the loss function. Although there are many wonderful improvements such DPM [3] PINNs still face serious challenges discussed in the paper [15). Some theoretical works [37 38 point out that there exists an unbalanced competition between the terms of PDEs and BCs, limiting the application of PINNs to geometrically complex problems. To address this issue, some researchers
18 have tried to embed BCs into the ansatz. Some of
CCorresponiing author
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

ENVIRONMENT
HEAT
T
THEAT
0T -k är h A(T(2s) Tw.)
(ms)
T(m)) TINAT
-k
(m))
0b
Tx
== Bo
x
22 Robin BC
Dirichlet BC
Neumann BC
Figure 1: An illustration on three types of BCs, We give an example from heat transfer, where T= (x) is the temperature, & is the thermal conductivity, and h is the heat transfer coefficient. (1) The Dirichlet BC specifies the value of the solution at the boundary. Here we assume a constant temperature THEAT the heat source (r 20)). (2) The Neumann BC specifies the value of the derivative at the boundary. We assume that the right wall (x = x22) is adiabatic and force the derivative it describe the heat convection between the heat source and the environment (7xo) at the surface (x = 222)
to be zero. (3) The Robin BC is combination of the first two. And
have considered solving the equivalent variational form of the PDEs. In this way, the neural networks can automatically satisfy the BCs and no longer require adding corresponding loss terms. Nevertheless, these methods are only applicable to specific BCs (e.g., Dirichlet BCs, homogeneous BCs, etc) or geometrically simple PDEs. The key challenge is that the equation forms of the Neumann and Robin BCs have no analytical solutions general and are thus difficult to be embedded into the ansatz.
them 16, 27
follow the pipeline of the Theory of Connections 22 while others 41
In this paper, we propose a unified hard-constraint framework for all the three most commonly used BCs (i.e., Dirichlet, Neumann and Robin BCs). With this framework (see Figure 2 for an illustration), we are able to construct ansatz that automatically satisfies the three types of BCs. Therefore, we can train the model without the losses of these BCs, which alleviates the unbalanced competition and significantly improves the performance of solving geometrically complex PDEs. Specifically,
we first introduce the extra fields from the mixed finite element method 20 This technique substitutes the gradient of a physical quantity with new variables, allowing the BCs to be reformulated as linear equations. Based on this reformulation, we derive a general continuous solution of the BCs with simple form, overcoming the challenge that the original BCs cannot be solved analytically. Using the general solutions obtained, we summarize a paradigm for constructing the mard-constraint ansatz under dime-dependent, multi-boundary, and nigh-dimensional cases. Besides, in Section 4 demonstrate that the technique of extra fields can improve the stability of the training process.
We empirically demonstrate the effectiveness of our method through three parts of experiments. First, we show the poteney our method in solving geometrically complex PDEs through two numerical experiments from real-world physical systems of battery pack and an airfoil. And our framework achieves supreme performance compared with advanced baselines, including the learning rate annealing methods (37]. domain decomposition- based methods [10 233, and existing hard-constraint methods [30 31). Second, we select a digh--iinnsiioaal problem to demonstrate that our framework can be well applied to nigh-dimensional cases. Finally, study the impact of the extra fields as well as some syper-parameters and verify our theoretical results in Scction 4
To sum up, we make the following contributions:
We introduce the extra fields reformulate the PDEs, and theoretically demonstrate that our reformulation can effectively reduce the instability of the training process.
We propose a unified hard-constraint framework for Dirichlet, Neumann, and Robin BCs. alleviating the unbalanced competition between losses in the physics-informed learning.
Our method has superior performance over state-of-the-art baselines on solving geometri- cally complex PDEs, validated by numerical experiments in worldw physical problems.

---

Boundartes s Una/Uia
f
Boundary olals + (aa)(())) Where u. sta) urknow funstion
Approximation
Arsatz which satsfles Boundary Conditions
Exra felds
Reformuinted Boundary Conditions a(o)x He](mw))) p) sle). Tix
Unbalanced competiton Hypothesis space to explore.. with only slightly relaxation
(o (u?)p?))
(u p7))
Deduce
Xweight
Xwiliht
General Solutions
(x").'1)) (0N"
Kenright,
Σ
Proposed Arnadz
whene XON reunaal retoork
(11)
Figure 2: A pipeline of proposed method. In this paper, We consider PDEs with multiple boundary conditions of Dirichlet, Neumann, and Robin. We first introduce the extra fields reformulate the BCs as linear equations whose general solutions are then deduced. Finally, we aggregate the general solutions for each boundary to obtain our final ansatz via Eq. (IT). Since our ansatz automatically satisfies the BCs, the unbanlanced competition is alleviated and the hypothesis space is reduced.
2 Background
2.1 Physics-Informen Neural Networks (PINNs)
We consider the following Laplace's equation as a motivating example,
Au(z1, x2) = 0, (22)) = g(x2);
3i E (0, 10, E [0.
(1a)
3 0, 22 E [0,
(1b)
where Eq. (Ta) gives the form of the PDE, and Eq. (bb is a Dirichlet boundary condition (BC). solution to the above problem is solution to Eq. lab which also satisfies Eq. (lb).
Physics-informed neural networks (PINNs) 25 employ neural network (N(111,
22; 0) to approx- imate the solution, i.c., ú(I1, x2; 0) = NN 12; 0) ss u(21, z22, where 0 denotes the trainable parameters of the network. And we learn the parameters 0 by minimizing the following loss function
N
N
((0) = C= &
Au(a
(6)
2 f22
(
0)
2
1
a(0, (00,2,2200
-
_(=) (2)
NJ
No
where LF is the term restricting â to satisfy the PDE (Eq. Tap) while L8 is the BC (Eq. lb
(a/11 "f11" 2 (2 1 is a set of N, collocation points sampled from [0, 1]2, and {ax (3) (0,
Fg. i=
No
is
set of No collocation points sampled from ø = O.
PINNs have wide range of applications, including heat [5), flow 21 and atmosphere 42 However,
PINNs are struggling with some issues on the performance [15] Previous analysis 37 38) has
demonstrated that the convergence of Ly be significantly faster than that of Lg- As demonstrated in later experiments (see Section 5)), this pathology may lead to nonphysical solutions which does not satisfy the BCs initial conditions (ICs). Moreover, for geometrically complex PDEs where the number of BCs is large, this problem is exacerbated and can seriously affect accuracy, as supported by our experimental results in Table T
2.2 Hard-Constraint Methods
One potential approach overcome this pathology is embed the BCs into the ansatz in way that any instance from the ansatz can automatically satisfy the BCs, as utilized by previous work B 24 35 39 30 18 31]. We note that the loss terms corresponding to the BCs are no longer needed, and thus the above pathology is alleviated. These methods are called hard-constraint methods, and they share a similar formula of the ansatz as
((:;; 0)
u 00 00( + 10 (x))(())))
(3)
3

---

where æ is the coordinate, Ω is the domain of interest, (x) is the general solution at the boundary o2, and 10 (xx) is an extended distance function which satisfies
if x: 02, otherwisc.
po (x)
(4)
In the case of Eq. ① (where x = 1 22), Ω = [0. IJ²), the general solution is exactly g(22), and we can use the following ansatz (which automatically satisfies the BC in Eq. (Ib))
ú(01, 22; 0) g + 21 (2; 0).
(5)
However, it is hard to directly extend this method to more general cases of Robin BCs (see Eq. (7), since we cannot obtain the general solution u (x)) (x) analytically. Existing attempts are either mesh- dependent (9] 43 which are time-consuming for high nigh-dimensional and geometrically complex PDEs, or ad hoc methods for specific (geometrically simple) physical systems (26). It is still lacking unified hard-constraint framework for both geometrically complex PDEs and the most commonly used Dirichlet, Neumann, and Robin BCs.
Methodology
We first introduce the problem setup of geometrically complex PDEs considered in this paper and then reformulate the PDEs via the extra fields, followed by presenting our unified hard-constraint framework for embedding Dirichlet, Neumann and Robin BCs into the ansatz.
3.1 Problem Setup
We consider physical system governed by the following PDEs defined on a geometrically complex domain Q TRd
F(u(())] = O, x x= =x .a) E Ω
where F = (FI...,, FN) includes N PDE operators which map u to a function of æ, u and its derivatives. Here, u(x) (ny (a).... ...., Un (x)) is the solution to the PDEs, which represents physical quantities of interest. For each 2j. 1,..., n, we pose suitable boundary conditions (BCs) as
a. (xxx),; (x)) Vu,) (x(x),
Vi Wi=1, 1......, -,.
where 71...., are subsets of the boundary aQ whose closures are disjoint, a, and by are two functions satisfying that a (ax) + b2 (a=) * 0 holds for Va E 7. and n(x) = (n, (a) ..... na (x) is the (outward facing) unit normal of corresponding
at It is noted that Eq. represents Dirichlet BC if a; = b, = 0, a Neumann BC a; = 0, b, = 1, and a Robin BC otherwise.
For such geometrically complex PDEs, if we directly resort to PINNs (see Section 2.1). there would be a difficult multi-task learning with least (Σ my NN) terms in the loss function. As discussed in the previous analysis (37 38 it will severely affect the convergence of the training due to the unbalanced competition between those loss terms. Hence, in this paper, we will discuss how to embed the BCs into the ansatz, where every instance automatically satisfies the BCs. However, is infeasible to directly follow the pipeline of hand-constraint methods (see Eq. (3) since Eq. ( does not have an general solution of analytical form. Therefore, a new approach is needed address this intractable problem.
3.2 Reformulating PDEs via Extra Fields
In this subsection, we present the general solutions of the BCs, which will be used to construet
the hard-constrain ansatz subsequently. We first introduce the extra fields from the mixed finite
element method 20 to equivalently reformulate the PDEs. Let (xxx) (P (z)........ .......... (x)) (Pja)) =
Vu;,j=1 =
And we substitute them into Eq. (6) and Eq. to obtain the equivalent PDEs,
pi .P.()) = 0.
x E Ω x E UN
(8a)
(x) = Vuj.
Vj -,...,
(8b)

---

where (u(x)), (u(x))Pi(<)) (..).....PPp (x)) is the solution of the new PDEs, = (FI...... - operators after the reformulation. And for j I...., n, we have the corresponding BCs
Fx) are the PDE
a;(2) + ( p. (z)) = (.(m,, x E %6.
Vi = 1,....... my-
(9)
Now, we can see that Eq. 7 has been transformed into linear equations with respect to (uj,P,), which are much easier for us derive general solutions. Hereinafter, we denote (uy, PT) by Pj. and their general solutions at ?/ (i.e., (u]: .P))) by P) Next we will discuss how to find p)
To obtain the general solution of Eq. D.. the first step is to find a basis ((x) of the null space (whose dimension is d). However, we must emphasize that (xx) should be carefully chosen. Since Eq. ( is parameterized by æ, for any æ E %/s (xx) should always be a basis of the null space, that is, its columns cannot degenerate into linearly dependent vectors (otherwise it will not be able to represent all possible solutions). An example of an inadmissible B (xx) ) is given in Appendix A.I.
Generally, for any dimension d E IR, we believe it is non-trivial to find a simple expression for the basis. Instead, we prefer to find (d + 1) vectors the null space, of which are linearly independent (that way, ((x) E IR (dd++)) We now directly present our construction of the general solution while leaving detailed derivation in Appendix A.3.
Pj (a:: 03) = NNY) (x; 030) + (()))
(10)
where ñ (aa, b.n)/////// a g./ Va + + NN R2d - gd+1 is neural network with trainable parameters 07', and B(x) = La+1 n(x)(()) Incidentally, in the case of d 1 or d = 2, we can find simpler expression for B (see Appendix A.2). Besides, we note that all the neural networks are implemented as MLPs in this paper, unless otherwise stated.
3.3 A Unified Hard-Constraint Framework
With the parameterization of a neural network, Eq. (10) can represent any function in n. as long as the function satisfies the BC (see Eq. (9). Since our problem domain contains multiple boundaries, we need to combine the general solutions corresponding to each boundary % to achieve an overall approximation. Hence, we construct our ansatz as follows
mi
(ūj.P,) = (x) main (x; Qmain)
Σ
exp [ (x))) Vj 1, .,....,,.. n (11)
where NNNain Rdd Rd+1 the main neural network with trainable parameters Omain- 1003 176 i 1,..., m, are continuous extended distance functions (see Eq. (4) ), and a; (i 1. ....., m2) are determined by
B.
0:3- min 17% (xx)
(12)
where B. IR. is hyper-parameter of the "hardness" in the spatial domain. In Eq. extended distance functions to "divide" the problem domain into several parts, where P (NN) is its learnable part) is responsible for the approximation on the boundaries while NN is responsible for internal. Furthermore, Eq. (12) ensures that the weight of D. decays to e - at the nearest other boundaries. We provide a theoretical guarantee of the satisfaction of the BCs along with the approximation ability for Eq. (II in Appendix A.4. Besides, if a, b, n or gi are only defined at %. we can extend their definition to Ω U On using interpolation or approximation via neural networks (see Appendix A.5). And it is easy to extend this framework the ppepoolltemporal domain as discussed in Appendix A.6.
utilize
neighbor of yr that 7 does not interfere with the approximation
Finally, we can train our model with the following loss function
N
N
Pn (x)))))))
N
1=1
(13)
N,
1
Σ
(+))
-=
5

---

where á = (u....., . ún), {úg.Po}ブ=1 is defined in Eq. and (x((k) Ny points sampled S. For neatness, we omit the trainable parameters of neural networks here. We note that Eq. 13 measures the discrepancy of both the PDEs (i.e., ......... FN) and the equilibrium equations introduced by the extra fields (i.e., Eq. (8b) at N collocation points.
k=1 is a set of collocation
According to Eq. (T). we have now successfully embedded BCs into the ansatz, and no longer need to take the residuals of BCs as extra terms in the loss function. That is, our ansatz strictly conforms BCs throughout the training process, greatly reducing the possibility of generating nonphysical solutions. Nevertheless, this comes at the cost of introducing (nd) additional equilibrium equations (see Eq. ). But in many physical systems, especially those with complex geometries, the number of BCs (cnt( is far larger than (nd) (e.g.,
3. d = 2, cnt(BCs) = 1260 for classical physical system, a heat exchanger 7 So we may actually reduce the number of loss terms by an order of magnitude or two (Acnt(losses) = nd cnt(BCss) < 0), alleviating the unbalanced competition between loss terms.
Theoretical Analysis
In Section 3,2 .2 we introduce the extra fields and reformulate the PDEs (from Eq. (6) and Eq. 7 to Eq. and Eq. (9). To further analyze the impact of this reformulation, we consider the following abstraction of ID PDEs (Eq. (14a)) and the one after the reformulation (Eq. (14b))
L [u, du d²u
duu
[(x),
I E Ω
(14a)
dx dx
dr
L [u, PI, P2. PP--1] f(x),
I EE
(14b)
where L is an operator, f is a source function, and p;-1 - dp:/ dæ, pi = du/ dæ are the introduced extra fields. From the above equations, we can find that the orders of derivatives in the PDEs are reduced. Intuitively, as the PDEs are included in the loss function (see Eq. (2). lower derivatives
result in less accumulation of prok-prop, and thus stabilize the training process.
To formally explain this mechanism, without loss of generality, we consider simple case (with proper Dirichlet BCs) where L[--.] ] = Au, f(z) = -a² sin ar. And our ansatz is a single-layer neural network of width K, i.e., û = c o(wr + b), p p= c, owwr + b). where c, w. b Cp E RRK a an element-wise activation function (for simplicity, are given in Appendix A.7.1. Next, we focus on the gradients of the loss terms of the PDEs F, since that of the BC stays the same during the reformulation. We state the following theorem.
take a as tanh). More details on the PDEs
Theorem 4.1 (Bounds for the Gradients of the PDE Loss Terms). Let O (c, w. b). Ô
(c, w. b, cp). and CF as well as EF be the loss terms corresponding to the original and trans- formed PDEs, respectively. We have the following bounds
(VoLz) [ O[c]] w² + +a²) (w² |cl |w)| (|w + +1,,/c| w²))
(15a)
(VoL.) | (1lep1 max( (oxc(lc,,ll lcpl)" w/ + d²) (lwl, max(Ic), (pp)) max([ur), 1), max(Ic], lcpl) max[[w], 1), max([w],11
(15b)
where the element-wise multiplication, is all-ones vector, max(.) is the element-wise maximum of vectors, and operations on vectors (1:1, (+)?, etc) are element-wise operations.
Youter
ntop
Ω
Ω
Yairfoil
Youter
Yoell,
Ypipe,
Youter
Vuleet
Youtlet
Youter
Ybottom
(a) A 2D battery pack
(b) An airfoil
Figure 3: An illustration of the geometries of the problems in Section 5.2 and Section 5.3

---

The proof is deferred to Appendix A.7.2. We can find that (VoLx)" has a high-order relationship with the trainable parameters 0. This often makes training unstable, as the gradients (VeLz) may rapidly explode or vanish when 0 are large small. After the reformulation, (VoCz) is associated with the parameters instability. We note that the improvement in stability comes from lower derivatives in the PDEs, and this mechanism does not depend on the specific form of the PDEs. Section 5.5.1
lower order, and some terms are controlled by alleviating this issue the
we empirically
show that this mechanism is general, even for nonlinear PDEs.
5 Experiments
In this section, we will numerically showcase the effectiveness the proposed framework on geometrically complex PDEs, which is also the main focus of this paper. To this end, we test our framework on two real-world problems (sce Section 5.2 and Section from a 2D battery pack and an airfoil, respectively. Besides, to validate that framework is applicable to high-dimensional cases, an additional high-dimension problem is considered in Section 3.4 followed by an ablation study in Section 5.5
We refer to Appendix A.8 for experimental details.
5.1 Experiment Setup
Evaluation In our experiments, we utilize mean absolute error (MAF) and mean absolute per- centage error (MAPE) to measure the discrepancy between the prediction and the ground truth. In Section 5.3] we replace MAPE with weighted mean absolute percentage error (WMAPE) to avoid the "division by zero", since considerable part of the ground truth values are very close zero,
WMAPE E 9
(16)
E
where n is the number testing samples, d is the prediction and y is the ground truth.
Baselines
We consider the following baselines in subsequent experiments.
PINN: original implementation of the PINN [25].
PINN-LA & PINN-LA-2: PINN with learning rate annealing algorithm address the unbalanced competition (37 and a variant we modified for numerical stability.
xPINN& FBPINN: two representative works for solving geometrically complex PDEs via non-overlapping [10] and overlapping (23) domain decomposition.
PFNN & PFNN-2: representative bard-constraint method based on the variational formu- 30 and its latest variant incorporating domain decomposition 3T
lation of PDEs
5.2 Simulation of 2D battery pack (Heat Equation)
To emphasize the capability of the proposed hard-constraint framework (HC) to handle geometrically complex cases, we first consider modeling thermal dynamies of 2D battery pack [32] which is
Table I: Experimental results of the simulation a 2D battery pack
MAE of T
MAPE T
== 0.5
t=
average
[ 0
1 0.5
( 1
average
PINN
0.1283
0.0457
0.0287
0.0539
128.21%
11.65%
4.47%
24.82%
PINN-LA
0.0918
0.0652
0.0621
0.0661
91.72%
19.13%
11.96%
27.06%
PINN-LA-2
0.1062
0.0321
0.0211
0.0402
106.03%
8.94%
4.09%
19.76%
FBPINN
0.0704
0.0293
0.0249
0.0343
70.33%
8.13%
5.87%
14.74%
xPINN
0.2230
0.1295
0.1515
0.1454
222.83%
30.28%
20.25%
54.70%
PENN
0.0000
0.3036
0.4308
0.2758
0.02%
79.64%
84.60%
68.29%
PFNN-2
0.0000
0.3462
0.5474
0.3215
0.02%
66.06%
90.21%
59.62%
HC
0.0000
0.0246
0.0225
0.0221
0.02%
5.38%
3.77%
5.10%

---

Table 2: Experimental results of the simulation of an airfoil
MAE
WMAPE
11
2/2
p
R/1
U2
PINN
0.4682
0.0697
0.3883
0.5924
1.1979
0.3539
PINN-LA
0.4018
0.0595
0.2652
0.5084
1.0225
0.2418
PINN-LA-2
0.5047
0.0659
0.2765
0.6385
1.1325
0.2521
FBPINN
0.4058
0.0563
0.2665
0.5134
0.9676
0.2429
XPINN
0.7188
0.0583
1.1708
0.9095
1.0029
1.0672
HC
0.2689
0.0435
0.2032
0.3402
0.7474
0.1852
governed by the heat equation (see Appendix 1.8.1), where E 92 L [0, 1] are the spatial and temporal coordinates, respectively, T(x t) is the temperature over time. The geometry of the problem Q is shown in Figure 3(a) where Tcell, the cell and Tpipe, the cooling pipe.
The baselines are trained with N,- = 8192 collocation points, N₂ = 512 boundary points and N; 512 initial points (an additional N₂ 512 points on the interfaces between subdomains required for the xPINN), while the HC trained with N₂ = 8192 collocation points. All the models trained for 5000 Adam [4]] iterations (with learning rate scheduler) followed by L-BFGS optimization until convergence and tested with N, = 146,487 points evaluated by the FEM. The testing results are given in Table 1 (where "average" means averaging over [0, 1)). We find that the accuracy of the HC is significantly higher than baselines almost all the time, especially at t This is because the ansatz of the HC always satisfies the BCs and ICs during the training process, prevents the approximations from violating physical constraints at the boundaries. Besides, we also provide an empirical analysis of convergence in Appendix A.9 and results of 5-run parallel experiments (where problems in all three cases are revisited) in Appendix A.10.
5.3 Simulation of an Airfoil (Navier-Stokes Equations)
In the next experiment, we consider the challenging benchmark PDEs in computational fluid dynamics, the 2D stationary incompressible Navier-Stokee equations, in the context of simulating the airflow around real-world airfoil (w1015. dat) from the UICC airfoil coordinates database (an open airfoil database) [29]. Specifically, the airfoil represented by 240 anchor points and the governing equation be found in Appendix A.8.2, where u(x) (u1 (az), u2 uz(x)), p(æ:) are the velocity, and the pressure of the fluid, respectively, and the geometry of the problem is shown Figure (36)
The baselines (PFNN and PFNN-2 are not applicable to Navier-Stoker Equations) are trained with Ny = 10,000 collocation points, N₂ 2048 boundary points (N, 2048 points for the xPINN), while the HC is trained with N₂ = 10,000 collocation points. All the models are trained for 5000 Adam iterations followed by L-BFGS optimization until convergence and tested with N, = 13,651 points evaluated by the FEM. According to the results in Table 2
the HC has more obvious advantages, compared with the previous experiment. This is because complex nonlinear PDEs are more sensitive to the BCs, and the failure at the BCs can cause the approximations to deviate significantly from the true solutions. In addition, we find that the WMAPE on uz is relatively large for all methods, which because the true values of U2 close to zero, amplifying the effect of the noise.
Table 3: Experimental results of the high-dimensional heat equation
MAE of u
MAPE of
t ==0 0
t == 0.5
t=
average
= 0
f == 0.5
t t= 1
average
PINN
0.0219
0.0428
0.1687
0.0582
1.43%
1.70%
1.01%
1.99%
PINN-LA
0.0085
0.0149
0.0727
0.0235
0.55%
0.59%
1.73%
0.78%
PINN-LA-2
0.0122
0.0274
0.1495
0.0466
0.79%
1.08%
3.57%
1.49%
PFNN
0.0000
0.1253
0.3367
0.1425
0.00%
5.02%
8.19%
4.64%
HC
0.0000
0.0029
0.0043
0.0026
0.00 %
0.12%
0.11%
0.10%
8

---

3
11/000
8.00
5
5
50000
Ratio
Ratio
4.00
2.00
1.0
1.00
0.5
2000
4000
6000
8000
10000
2000
4000
6000
8000
10000
Iterations
Iterations
(a) Poisson's equation
(b) Schrödinger equation
Figure 4: The history of MorVar([VeCC)
5.4 High-Dimennsinal Heat Equation
To demonstrate that our framework can generalize to high-dimensional cases, we consider a problem of time-dependent high-dimensional PDEs in 52 x [0, 1] (see Appendix A.8.3), where Q is unit ball in IR_0 and u is the quantity of interest.
Here, we do not consider the baselines of the xPINN, FBPINN, and PFNN-2, because the method of domain decomposition is not suitable for high-dimensional cases. And the other baselines are trained with y 1000 collocation points, N₂ = 100 boundary points and N, 100 initial points, while the HC is trained with N, 1000 collocation points. All the models are trained for 5000 Adam iterations followed by L-BFGS optimization until convergence and tested with N, 10,000 points evaluated by the analytical solution to the PDEs. We refer to Table for the testing results. It can be seen that the HC achieves impressive performance in nigh-dimensional cases. This illustrates the effectiveness of Eq. (10) and the fact that our model can be directly applied to high-dimensional PDEs, just geometrically complex PDEs.
5.5
Ablation Study
In this subsection, are interested in the impact of the "extra fields" and the hyper-parameters of "hardness" (i.e., B3 and Br, see Section 3.3
5.5.1 Extra fields
We test the PINNs (with and without the "extra fields") on the Poisson's equation and the nonlinear Schrödinger equation (see Appendix A.8.4). We vary the network architecture and report the ratio of
the the moving variance (Mov Var) of VoLr to that of [gLF at each iteration (see Figure 4p. In
both linear and nonlinear PDEs, we find that models with the "extra fields" (VV&C=)) achieve smaller gradient oscillations during training, which is consistent with our theoretical results in Section 4 This is because the "extra fields" reduces the orders the derivatives, which in turn reduces accumulation of backpropagation, avoiding vanishing exploding gradients.
5.5.2 Hyper-parameters of Hardness
We repeated the experiment in Section 5.2 with different combinations of the values of 9, and B and all other settings stay the same. Table gives the testing results (where the MAE and MAPE
Table 4: The MAE/ MAPE of I different 8, and Be
3e =
3, = 2
G, = 5
Be = 10
8. 1
0.3492 48. 28%
0.3539
48 38.566
0.3226 / 44.64%
0.2889 40.69%
8. 2
0.2800 40.30%
0.1670/
26. 16%
0.2140 / 31..2%
0.1619 25.20%
8, 5
0.1878/ 28.68%
0.1195 19.68%
0.0542 / 10.35%
0.0221 5.10%
8, 10
0.1896 29.15%
0.1104 18.70%
0.0517 10.56%
0.0329 8.15%

---

are the average results over t [0, 1). In general, the accuracy increases as the values of 3g and St become larger. The reason is that "harder" 3, and B, cause the BCs be better satisfied, which in turn makes the approximations more compliant with physical constraints. However, too large B, and 3e (they are both on the exponential, see Eq. (11) can lead to performance degradation, which may be related to numerical stability.
6 Conclusion
In this paper, we develop unified murd-constraint framework for solving geometrically complex PDEs. With the help of "extra fields", , we reformulate the PDEs and find the general solutions of the Dirichlet, Neumann, and Robin BCs. Based on this derivation, we propose a general formula of the hard-constraint ansatz which is applicable to time-depentn dependenn,,,,, multi-boundary, and nigh-dimensiona cases. Besides, our theoretical analysis demonstrates that the reformulation helpful for training stability. Extensive experiments show that our method can achieve state-of-the-ee art performance real-world geometrically complex as well as high-dimension PDEs, and our theoretical results are universal to general PDEs. One of the limitations is that here we only consider the three most commonly used BCs and future works may include extending the framework to more general BCs.
Acknowledgements
This work was supported by the National Key Research and Development Program of China (2017YFA0700904, 2020AAA0106000, 2020AAA0104304, 2020AAA01063022, NSFC Projects (Nos. 62061 136001, 62076145, 62076147. U19B2034. U181 1461, U19A2081, 61972224), Beijing NSF Project (No. JQ19016), BNRist (BNR2022RC010N) Tsinghua Institute for Guo Qiang, and the High Performance Computing Center, Tsinghua University. J.Z. is also supported by the XPlorer Prize.
References
[1] Leah Bar and Nir Sochen. Unsupervised deep learning algorithm for pde-based forward and inverse problems. arXiv preprint 100.v:1904.05417, 2019.
[2] Cx K Batchelor and GK Batchelor. An introduction to fluid dynamics. Cambridge university press, 2000.
[3] Jens Berg and Kaj Nyström. unified deep artificial neural network approach to partial differential equations in complex geometries. Neurocomputing, 317:28 41, 2018.
[4] Uirich Brink and Erwin Stein. On some mixed finite element methods for incompressible and nearly incompressible finite elasticity. Computational Mechanics, 19(1): 119, 1996.
[5] Shengze Cai, Zhicheng Wang, Sifan Wang, Paris Perdikaris, and George Em Kamiadakis. Physics-informed neural networks for heat transfer problems. Journal of Heat Transfer, 143(6), 2021.
[6] MWMG Dissanayake and Nhan Thien. beural-netwoor--base approximations for solving partial differential equations. communications in Numerical Methods in Engineering, 10(3):195- 201, 1994.
[7] Vadim Dubovsky, Gennady Ziskind, and Ruth Letan. Analytical model of a pem-air heat exchanger. Applied Thermal Engineering, 31(16):3453- 3462, 2011.
[8] Soheil Esmaeilzadeh, Kamyar Azizzadenesheli, Karthik Kashinath, Mustafa Mustafa, Hamdi A Tchelepi, Philip Marcus, Mr Prabhat, Anima Anandkumar, et al. Mcshfreeflownet: a physics- constrained deep continuous space-time super-resolution framework. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1-15. IEEE, 2020.
[9] Han Gao, Luning Sun, and Xian-Xun Wang. Phygeonet: physics-informed geometry- convolutional neural networks for solving parameterized steady-state pdes on irregular domain. Journal of Computational Physics, 428:110079) 2021.
10

---

[10] Ameya D Jagtap and George Em Karniadakis. Extended physics-informed neural networks (xpinns): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations. Communications in Computational Physics, 28(5):2002- 2041, 2020.
[11] Dong Hyup Jeon and Seung Man Baek. Thermal modeling of cylindrical lithium ion battery during discharge cycle. Energy Conversion and Management, 52(8-9):2973-299 2011.
[12] Ehsan Kharazmi, Zhongqiang Zhang, and George Em Karniadakis. Variational physics-infommdd neural networks for solving partial differential equations. arXiv preprint 1WXxv:1912.0087: 2019.
[13] Jungeun Kim, Kookjin Lee, Dongeun Lee, Sheo Yon Jin, and Noscong Park. DPM: A novel training method for physics neural networks in extrapolation. CoRR, abs/2012.02681, 2020.
[14] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR (Poster), 2015.
[15] Aditi Krishnapriyan, Amir Gholami, Shandian Zhe, Robert Kirby, and Michael W. Mahoney. Characterizing possible failure modes in physics-informed neural networks. In A. Beygelzimer, Y. Dauphin, P. Liang. and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021.
[16] Carl Leake and Daniele Mortari. Deep theory of functional connections: A new method for estimating the solutions of partial differential equations. Machine learning and knowledge extraction, 2(1):37-55, 2020.
[17] Yulei Liao and Pinghing Ming. Deep nitsche method: Deep ritz method with essential boundary conditions. arXiv preprint arXiv:1912.01309 2019.
[18] Lu Lu, Raphael Pestourie, Wenjie Yao, Zhicheng Wang, Francese Verdugo, and Steven G Johnson. Physics-informed neural networks with hard constraints for inverse design. SIAM Journal on Scientific Computing, (566)BB105-DD 132, 2021.
[19] Andrew Majda. Introduction to PDEs and Waves for the Atmosphere and Ocean, volume American Mathematical Soc., 2003.
[20] David S Malkus and Thomas JR Hughes. Mixed finite element methods- and selective integration techniques: a unification of concepts. Computer Methods in Applied Mechanics and Engineering. 15(())63-8)) 81, 1978.
[21] Zhiping Mao, Ameya D Jagtap, and George Em Karniadakis. Physics-informed for high-speed flows. Computer Methods in Applied Mechanies and Engineering, 360: 12789, 2020.
informed neural networks
[22] Daniele Mortari. The theory of connections: Connecting points. Mathematics, 5(4):57, 2017.
[23] Ben Moseley, Andrew Markham, and Tarje Nissen-Meyer. Finite basis neural networks (fbpinns): scalable domain decomposition approach for solving differential equations. arXiv preprint arXiv:2107.072 2021.
[24] Guofei Pang, Lu Lu, and George Em Karniadakis. fpinns: networks. SIAM Journal on Scientific Computing, 4144):2603-A2226, 2019.
Fractional physics-infone informed neural
[25] Maziar Raissi, Paris Perdikaris, and George E Karniadakis. dnysics-informed neural networks:
A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686-707, 2019.
[26] Chengping Rao, Hao Sun, and Yang Liu. Physics-informed deep learning for computational elastodynamics without labeled data. Journal of Engineering Mechanics, 040(1004 2021.
11

---

[27] Enrico Schiassi, Roberto Furfaro, Carl Leake, Mario De Florio, Hunter Johnston, and Daniele Mortari. Extreme theory of functional connections: A fast physics-informes neural network method for solving ordinary and partial differential equations. Neurocomputing. 457:334-356, 2021.
[28] William E Schiesser. Computational mathematics in engineering and applied science: ODEs, DAEs, and PDEs. CRC press, 2014.
[29] Michael Selig. The UTCC airfoil coordinates database. https://m-sellg. ae. wtw.ll........l.o............ illinois aa/aaddsccoooo database. Accessed: 2022-05-04.
[30] Hailong Sheng and Chao Yang. Pinn: penalty-free neural network method for solving a class of second-order boundary-value problems on complex geometries. Journal of Computational Physics, 428: 10085, 2021.
[31] Hailong Sheng and Chao Yang. Pfnn-2: A domain decomposed penalty-free neural network method for solving partial differential equations. arXiv preprint WwXiv:2205.00593 2022.
[32] Andrey Smyshlyaev, Miroslav Krstic, Nalin Chaturvedi, Jasim Ahmed, and Aleksandar Kojic. Pde model for thermal dynamics of a large li-ion battery pack. In Proceedings of the 2011 American Control Conference, pages
IEEE, 2011.
[33] Walter A Strauss. Partial differential equations: An introduction. John Wiley & Sons, 2007.
[34] Hongli Sun, Muzhou Hou, Yunlei Yang, Tianle Zhang, Futian Weng, and Feng Han. Solving partial differential equation based on bernstein neural network and extreme learning machine algorithm. Neural Processing Letters, 50(2):1153 1172, 2019.
[35] Luning Sun, Han Gao, Shaowu Pan, and Jian-Xun Wang. Surrogate modeling for fluid flows based on physics-ccnssrrineed deep learning without simulation data. Computer Methods in Applied Mechanics and Engineering, 361:112732. 2020.
[36] Remco van der Meer, Comells Oosterlee, and Anastasia Borovykh. Optimally weighted loss functions for solving pdes with neural networks. Journal of Computational and Applied Mathematics, page 2021.
[37] Sifan Wang, Yujun Teng, and Paris Perdikaris. Understanding and mitigating gradient flow pathologies in physics-ifformed neural networks. SIAM Journal on Scientific Computing, 43(5):A3055-A3081 2021.
[38] Sifan Wang, Xinling Yu, and Paris Perdikaris. When and why pinns fail to train: A neural tangent kernel perspective. Journal of Computational Physics, 449: 10768, 2022.
[39] Yizheng Wang, Jia Sun, Wei Li, Zaiyuan Lu, and Yinghua Liu. Cenn: Conservative en- ergy method based on neural networks with subdomains for solving heterogeneous problems involving complex geometries. arXiv preprint arXiv:21 2021.
[40] Tong Wu, Burak Ozpineci, Madhu Chinthavali, Zhiqiang Wang, Suman Debnath, and Steven Campbell. Design and optimization of 3d printed air-cooled heat sinks based on genetic algorithms. In 20/7 IEEE Transportation Electrification Conference and Expo (ITEC), pages 650-655 655. IEEE, 2017.
[41] Bing Yu et al. The deep ritz method: a deep learning- numerical algorithm for solving variational problems. Communications Mathematics and Statistics, 6(1):1-12, 2018
[42] Jincheng Zhang and Xiaowei Zhao. Spatiotemporal wind field prediction based on physics- informed deep learning and lidar measurements. Applied Energy, 288: 16641, 2021.
[43] Xiaoyu Zhao, Zhiqiang Gong, Yunyang Zhang, Wen Yao, and Xiaoqian Chen. Physics-infonee inffrmedd convolutional neural networks for temperature field prediction of heat source layout without labeled data. arXiv preprint arXiv:2 12482, 2021.
12

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section 6
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See Appendix A.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] See Section Appendix A.3, A.4, and A.7.
(b) Did you include complete proofs of all theoretical results? [Yes] See Appendix A.3,
A.4, and A.7.
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main ex- perimental results (either in the supplemental material as URL)? [Yes] See the supplementary material.
(b) Did you specify all the training details (c.g., data splits, hyperparameters, how they were chosen)? [Yes] See Appendix A.8.
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [Yes] See Appendix A.10.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Appendix A.8.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you cite the creators? [Yes] See Section 5.3
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you'l 're using/curating? [Yes] See Section 5.3
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A] No such information the data.
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
13


Reviews:
[{"review_id": "sdk11lxafPw", "reviewer": "Reviewer_HVPE", "summary": "The submission introduces a new technique to enforce Dirichlet, Neumann, and Robin boundary constraints into the PINNs model, based on the extra fields technique from the mixed finite element method. Extra fields decomposes the solution into a linear combination of a boundary term, which is given to us as a constraint, and an interior term, learned by a neural network. On the boundary, the interpolation is designed so that only the boundary term is accounted for in the solution. As such, this is a handcrafted hard constraint method. The authors test on 3 PDEs: the heat equation in a battery pack, the Navier-Stokes equation over an airfoil, and a 10 dimensional heat equation, comparing against a variety of PINNs-based models. They claim that their method is stabler to train.", "questions": "“Geometrically complex” -  this term is used a lot and yet not defined anywhere. It would be useful to have a definition or at least a description of what this could mean.\n\n23 - It is stated that neural networks have only recently been applied to the solution of PDEs, but the field is in fact older, dating back more than 20 years.\n\nEqn 1a:  is the domain of x2 correct? I think it may be x2 \\in (0,1].\n\n107 - what is meant by “PDE operator”? I think I understand, but a clear definition would be useful. Do you not just mean a differential operator?\n\n133: it would be useful to provide an example where you show how the nullspace of eqn. 9 is computed.\n\nEqn 10 - what architecture is used for NN_i? This is missing from the paper and highly influences the results that can be achieved.\n\nSec 3.3 - Would it be possible to achieve the decomposition of the solution\t into a hard-coded boundary term and a learned interior term without the introduction of the extra fields per se? In eqn 13 you have to add a loss term encouraging p to match \\nabla u. Instead, you could have just set p to be \\nabla u itself and trained this directly. Or is it too difficult to achieve this given that p itself has to live in the nullspace of eqn 9?\n\n182 -  I think you mean width, not length\n\nEqn 16:  where are the weights?\n\nExperiments: it would be useful for the boundary conditions to be explicitly shown in this section, along with the architectures used, how the nullspace basis is solved for and the choice of extended distance function l.\n\nTables: were the results computed over multiple runs? It would have been useful/more significant to have confidence internals/standard deviations shown, so that the significance of each result could be ascertained.\n\nTypos: \n13 - effeteness\n63 - alliviating\n138 - non-trival\n203 - addition -> additional", "limitations": "Limitations of this work are not addressed. It would be useful to write a paragraph explaining the shortcomings of having to solve for the nullspace of the boundary condition equations and other limitations of using PINNs as a baseline.", "rating": 3, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "+ The mathematics and derivations appear to be sound (although a little hard to follow) and it appears from the experimental section that this technique is better than competing methods.\n\n+ The method is grounded in a standard technique from the field of mixed finite element solvers, and so it inherits a lot of the benefits of that method.\n\n+ The submission steps the reader through a lot of needed material and builds up intuition.\n\n- It is hard to gauge the significance of the method, since there are no confidence intervals in the experiment section\n\n- I could not reimplement this submission. The main paper lacks many essential details.\n\n-  In many places, the writing lacks clarity. Terms like geometrically complex or PDE operators are used,  without clear definitions. This should, however, be easy to clear up.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "rBSjICWGkiO", "reviewer": "Reviewer_GzFR", "summary": "Physic-informed neural networks (PINNs) are a new paradigm for injecting the knowledge of governing equations into neural networks. To solve the problem, we need to consider initial conditions and boundary conditions, of which boundary conditions are hard to train for. Therefore, this paper proposes the general solutions to boundary conditions, which are later used to develop their \"extra field\" method. They experiment with two real-world problems.", "questions": "Please check the above weak points and try to answer.", "limitations": "They discuss a limitation at the end of the conclusion section.", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This paper tackles a critical problem of PINNs since training for boundary conditions frequently collides with training for governing equations. It is needed to be balanced between them. Their method solves the problem by designing the general solutions to boundary conditions and proves several forthcoming facts.\n\nHowever, this paper missed several other related works:\n- https://openreview.net/forum?id=a2Gr9gNFD-J\n- https://arxiv.org/abs/2012.02681\n\nI am also not sure whether their method is theoretically correct. In general, we need to double-check whether a constructed ansatz is correct or not. Their high accuracy indirectly proves it. But it seems not sufficient.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "S7CeFUsq1kI", "reviewer": "Reviewer_y3Mx", "summary": "This paper introduces a method for embedding the hard-constraints for boundary conditions (BCs) for physics-informed neural networks (PINNs). It introduces the \"extra fields\" to transform the original PDE into equivalent forms so that the BCs can become linear. This adds equations between the extra fields and the original PDE solutions, but can reduce total number of equations if the number of boundary conditions is high. The paper provides theoretical analysis of bounds for the gradients of the PDE loss terms. In experiments, the paper shows that the proposed method outperforms several baselines, and ablation study shows that it has smaller moving variance than without extra fields.", "questions": "N/A", "limitations": "The paper adequately addresses the limitations.", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\nThe paper address the important problem in the PINN field about the dealing with complex boundary conditions. It is well-motivated, the method makes sense and is novel. The experiment evaluation is sound. The paper is written clearly. It also has good reproducibility (in terms of set up for hyperparameters and providing the code)\n\nWeaknesses:\n\nI don't find apparent weaknesses.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "7LzwSVkmOiQ", "reviewer": "Reviewer_aRWM", "summary": "PINN has shown potential in solving high-dimensional PDE with complex geometries. Related works generally enforce the boundary conditions as a soft constraint via boundary condition loss term, which may lead to instability and hardness in convergence. The proposed work developed a reformulation of PDE to enforce boundary conditions (Dirichlet, Neumann, and Robin BCs) as a hard constraint under the PINN framework by introducing the extra fields. Thorough experiments have been done to demonstrate the proposed approach's effectiveness in terms of accuracy (measured by mean absolute error/mean absolute percentage error).  ", "questions": "1. Why does the proposed approach enforce the boundary condition as a ***hard*** constraint? Does it require that $p=\\nabla u$ is fulfilled, which is only enforced as a soft constraint?\n2. How does eq11 help enforce the boundary condition and what is the motivation for using eq.10 and eq,11 as the general solution, and what's the relationship between those two equations (are the ***NN*** in both equations refer to the same thing? \n3. Could more explanation be given to $g(\\cdot)$ between eq.A16 and eq.A15, and their correspondences to eq.10 and eq,11.\n4. What's the motivation for using extended distance functions here? ", "limitations": "1. It would be more reader-friendly if more connections can be built between the eq 10/11 and the derivation for each experiment shown in the supplementary.   \n2. It's nice to include some qualitative results and discussions on run time/convergence speed, rather than just comparing the final MAE/MAPE.\n\n", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "***originality*** -  to the best of my knowledge, the proposed reformulation that can handle all three boundary conditions is novel in the context of PINN. \n\n***quality*** - authors provide experiments on high dimensional/complex geometry PDEs. The derivation is mostly clear with some confusion listed in the \"clarity\" section and \"questions\" section. \n\n***clarity*** - the motivation is clear. The derivation may be a little hard to follow when eq(10) is proposed and the extended distance function is used.  It's a bit hard to understand by boundary condition is now satisfied as a hard constraint. While numerical results have been presented, it may also be interesting to include some qualitative results to illustrate the simulation results, etc. \n\n***significant*** - PINN is a simple approach to solving PDE, but it still with many challenges. Research works trying to develop and improve the convergence/accuracy of the PINN are of great interest. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}]

Author rebuttal:

Decision: Accept