Fast-PINN for Complex Geometry: Solving PDEs with Boundary Connectivity Loss
Authors: Jian Cheng Wong
Pao-Hsiung Chiu
Chin Chun Ooi
My Ha Dao
Yew-Soon Ong
Venue: Submitted to ICLR 2023 (2023)
Keywords: Physics-informed neural networks
physics-informed loss formulation
multi-layer perceptron
convolutional neural network
fluid dynamics

Abstract: We present a novel loss formulation for efficient learning of complex dynamics from governing physics, typically described by partial differential equations (PDEs), using physics-informed neural networks (PINNs). In our experiments, existing versions of PINNs are seen to learn poorly in many problems, especially for complex geometries, as it becomes increasingly difficult to establish appropriate sampling strategy at the near boundary region. Overly dense sampling can adversely impede training convergence if the local gradient behaviors are too complex to be adequately modelled by PINNs. On the other hand, if the samples are too sparse, PINNs may over-fit the near boundary region, leading to incorrect solution. To prevent such issues, we propose a new Boundary Connectivity (BCXN) loss function which provides local structure approximation at the boundary. Our BCXN-loss can implicitly or explicitly impose such approximations during training, thus facilitating fast physics-informed learning across entire problem domains with order of magnitude fewer training samples. This method shows a few orders of magnitude smaller errors than existing methods in terms of the standard L2-norm metric, while using dramatically fewer training samples and iterations. Our proposed Fast-PINN method does not pose any requirement on the differentiable property of the networks, and we demonstrate its benefits and ease of implementation on both multi-layer perceptron and convolutional neural network versions as commonly used in current physics-informed neural network literature.

Under review as a conference paper at ICLR 2023
BCXN-PINN FOR COMPLEX GEOMETRY: SOLVING PDEs WITH BOUNDARY CONNECTIVITY LOSS
Anonymous authors Paper under double-blind review
ABSTRACT
We present a novel loss formulation for efficient learning of complex dynam- ics from governing physics, typically described by partial differential equations (PDEs), using physics-informed neural networks (PINNs). In our experiments, existing versions of PINNs are seen to learn poorly in many problems, especially for complex geometries, it becomes increasingly difficult to establish appro- priate sampling strategy at the near boundary region. Overly dense sampling can adversely impede training convergence if the local gradient behaviors plex to be adequately modelled by PINNs. On the other hand, if the samples are too sparse, PINNs may over-fit the near boundary region, leading incorrect so- lution. To prevent such issues, we propose new Boundary Connectivity (BCXN) loss function which provides local structure approximation the boundary. Our BCXN-loss can implicitly explicitly impose such approximations during train- ing, thus facilitating fast physics-informed learning across entire problem domains with order of magnitude fewer training samples. This method shows few or- ders of magnitude smaller errors than existing methods in terms of the standard L2-norm metric, while using dramatically fewer training samples and iterations. Our proposed BCXN-PINN method does not pose any requirement on the dif- ferentiable property of the networks, and we demonstrate its benefits and ease of implementation on both multi-layer perceptron and convolutional neural network versions commonly used in current physics-ine -nnssccs-nforree neural network literature.
too com-
INTRODUCTION
Physics-infornen neural networks (PINNs) have emerged promising method for learning the solution of dynamical system from the governing physics (Raissi et al. 2019). PINNs have recently been studied for wide range of physical phenomena and applications across science and engineer- ing domains electromagnetic, fluid dynamics, heat transfer, etc (Karniadakis et al.) 2021| Cuomo eet al.) (022). The distinctive feature of PINNs is the of governing physics law, typically in the form of partial differential equations (PDEs), as the learning objective. This physics-informed learn- ing constrains the PINN from violating the underlying physics all training points sampled from the problem domain.
Existing PINNs evaluate the PDE constraints in their training loss by either automatic differentia- tion (AD) or numerical differentiation (ND)-type method (Wandel et al. 2020). While both methods have their pros and cons, ND-type PDE loss can be flexibly implemented across many different neu- ral network (NN) architectures, including both multi- perceptrons (MLPs) and convolutional neural networks (CNNs), because they do not require the NN to retain differentiability, unlike AD. Recent studies (Gao et al. 2021 Fang 2021 Chiu et al., 2022) have also suggested that ND-loss can more robustly and efficiently produce accurate solutions with fewer training samples, whereas conventional AD-loss proximate high order derivatives using PINN output from neighbouring samples, hence, they can effectively connect sparse samples into piecewise regions via these local approximations, thereby facilitating fast physics-infer infoormdd learning across the entire domain with less dense samples.
prone to failure during training This is because ND-type methods ap-
When dealing with irregular geometries, it becomes increasingly difficult for existing PINNs to perfectly connect training samples in the domain's 's interior to the boundary. Failing to do so can cause undesirable training failure as the PINN starts over-fit at the near boundary region. Since

---

Under review as a conference paper at ICLR 2023
many PDEs of practical interest are boundary-value problems. it is desirable to have the PINNs model the correct boundary behaviors. While adding dense samples to better refine the piecewise local regions near the boundary may improve accuracy, the extent to which sampling needs to be increased empirical, and a denser sampling strategy may adversely impede training convergence.
Hence, propose loss function formulation in this work that helps provide approximation to the local gradient behaviour at the boundary, thereby restoring connectivity between domain bound- ary and near boundary interior samples. This new boundary connectivity (BCXN)-loss function is key to a novel class of BCXN-PINN method which can more efficiently learn the solution to PDEs with fewer training samples, regardless of domain geometry; see example in Fig. [ addition, this method can be jointly implemented with other PINN advances such as in loss balancing, domain decomposition, adaptive sampling and other improved optimization methods (Zeng al. 2022).
Spare exmple
Democamapl
14
Figure 1: PINNs learning the solution of 2D N-S equations in complex geometry (wavy channel flow problem, Re = 100). Our BCXN-PINN method can learn accurate solution with faster speed (50% less training iterations) and fewer (50% less) training samples.
In the rest of this work, we present two versions of this BCXN-loss: i) a soft forcing approach which imposes a linear approximation constraint via an additional loss term, and ii) direct forcing approach which strongly enforces linear constraint during the evaluation of PDE loss near- boundary samples. With the latter approach, there is no longer need to explicitly evaluate the boundary samples as the exact BCs have been implicitly "infused" into the near-boundary samples. Moreover, the direct forcing BCXN-loss can be beneficial to CNN-type architectures which utilize structured grid and lack the ability to model exact domain boundaries irregular geometries.
We present comprehensive experiments to demonstrate i) the flexible implementation of BCXN- PINN method for both MLP and CNN architectures; and ii) the effectiveness of BCXN-PINN for learning multiple complex fluid dynamical systems, spanning forward, inverse and meta-model problems in two-dimensions (2D) and three-dimensions (3D). Compared to conventional PINNs with the AD- and ND-loss, BCXN-PINNs with BCXN-loss are shown be capable of tack- ling challenging PDE problems while using fewer training samples, hence expanding the exciting potential of PINNs for learning complex dynamical evolutions encountered in the real-world.
RELATED WORK
Efficient sampling in PINNs. The theoretical limit of physics-informed loss learning in relation to
training samples has been provided by prior studies (Lu et al. 2021c| Mishra & Molinaro 2022
With the goal of improving the PINN training speed for practical applications (Markidis. 2021
several studies have focused on efficient sampling strategies such as importance sampling, adaptive sampling, and sequential sampling to reduce the amount of training samples being required during
PINN trainings (Anitescu et al. al.| 2019 McClenny & Braga-Neto: 2020 Wight & Zhao 2020 Nabian
et al. 2021 Lu et al.) 202 Lye et al. 2021 Daw et al. 2022 Mattey & Ghosh 2022 Wu et al.)
2023 Domain decomposition and parallelization strategies have also been explored to speed up the
training (Jagtap al. 2020 Jagtap & Karniadakis 2021 Shukla et al. 2021 Dong & Li 2021
cet al. 2019; Kharazmi et al. 2021). ). Our method differs from these works in that we make physics-
informed learning more robust in the sparse sample regime via newly-proposed BCXN-loss.
2

---

Under review as a conference paper at ICLR 2023
CNN architecture and numerical differentiation (ND)-type loss for PINNs. CNN-based formu- lation allows us to design and train larger, more powerful networks, hence it has potential to be
scalable for more complex. large-scale PDE problem (Wandel et al. 2020 Gao et al. 2021 Wandel eet al., [021] RRnade et al.) 2021], Wandel et al., 2022] Ren et al. 2022). However, cumbersome
coordinate transformations may need to be performed to better handle the irregular domain. CNN- architecture PINNs usually utilize ND methods for computing the PDE loss on their input grid, which is much cheaper to compute as compared AD-based loss (Gao et al. (021). Hybrid frame- works that couple both AD- and ND-type loss have also been proposed for MLP architecture PINNS to unify the advantages of both methods (Jagtap et al. 2020, Kharazmi et al. 202T] Mitusch al.)
2021 Fang 2021; Chiu et al.) 2022). Our method further augments the efficiency and applicability of ND-type loss on irregular geometries.
Enforcing BC constraint in PINN loss. It is important for the PINNs to prioritize learning the correct boundary behaviors (Shin et al. 2020 Wang et al. (022). The BCs are usually enforced into the PINN loss as soft constraint using penalty method. Various strategies have been explored in this context to dynamically calibrate the relative important between PDE and BC constraints during
PINN training (Elhamod et al.] 2020; Bischof & Kraus, 2021; Jin et al. 2021
Thanasutives al.
2021 Maddu et al.j 2002) de Wolff et al. [022] van der Meer et al. 2022
Xiang et al. 2022
bypass the loss balancing issue. For example, one can either devise an ansatz. function such that the BCs are exactly satisfied
Huang a Wang al. 2021ap. There also other approaches
by construction or implicitly formulate the BC constraints into the PDE loss (Lagaris et al.) 1998
2000
McFall & Mahan 20099; Berg & Nyström 2018 Nabian & Mcidani 2019 Karumuri et al,
2020
Wang & Zhang 2020). leaving only single loss term from PDE residuals to be optimized.
Another example is the use of augmented Lagrangian method to impose the BC constraints into
PINN loss (Lu et al. 2021b). Our method implicitly incorporates BCs into the PDE loss via a
different strategy.
PRELIMINARY
3.1 GOVERNING PHYSICS INCOMPRESSIBLE NAVIER-STOKES (N-S) EQUATIONS
The present study focuses on learning fluid dynamics with PINNs. Fundamentally, PINNs have been shown to be applicable to many different physical systems. We consider fluid problems where the governing physics are the steady-state, incompressible N-S equations derived from the conservation of mass and momentum:
0 i V) -= -A Vp
(la)
(1b)
In the above PDEs, the primitive variables ū and p are velocity vector and pressure while Reynolds number (Re) represents the ratio of inertial to viscous forces. iī consists of 2 components (u,v) for a 2D case and 3 components (u. w.) for 3D case. While Cartesian coordinates are used in this work, this formulation is extendable to other coordinate systems.
3.2 MLP- AND CNN-ARCHITECTURE PINNS
We use a fully connected DNN architecture (e.g., MLP, CNN) to represent the solution of the dy- namical process = [ū. p)". For MLP architecture, the input x is point coordinate 2D or 3D spatial domain. For CNN architecture, the input X is tensor with fixed shape. comprising the entire (discretized) spatial domain. The accuracy of the PINN outputs U(Ew) given input is de- termined by the network parameters w, which are optimized w.r.t. the PINN loss function during training. The PINN loss function is defined as the composition of a PDE loss component (LpDE) and BC loss component (Lac):
LPINN APDELFDE 7BBCLBC
(2a)
LPDE V- ||2 + (((::))VV)((z.)) -R---Ree Re Ai((:: W
(2b)
LBC = || [[][[[]]] |1 ||2
(2c)
3

---

Under review as a conference paper at ICLR 2023
The PDE loss penalizes deviation from governing N-S equations for the PINN output U(Xw) over the fluid domain x' E Q, whereas the BC loss penalizes deviation from the desired Dirichlet boundary condition (3c(3) which constrains the velocity field at the domain boundary X SsS. The relative weights RPDE and RBc in Eq. (2 a control the trade-off between different loss components.
The PDE and BC loss components are defined over continuous domain, but for practical reasons, we evaluate the discretized PINN loss from a finite set of n samples D = {xx; Vi=1 during training. The -NN-architecture PINNS naturally acquire an input with equidistantly spaced grid where all the samples are coming from. For the MLP-architecture PINNs, it is also convenient to design training samples based on equidistantly spaced grid, then m( < n) samples can be randomly drawn compute the PINN loss during each stochastic gradient descent (SGD) mini-batch training iteration.
Although ideal for PINN to satisfy the PDE constraint on very dense set of samples, i.c., - 0 where AT is the distance between two adjacent sample points, training such PINN may not be computationally feasible even with the state-of-the-art SGD variants. For more complex physics problem, the training on dense samples is naturally more computationally expensive, and is also more likely to become unsuccessful due to adverse impact by some local region with very complex gradient behaviours, e.g., abnormally high gradient (Fuks & Tchelepi 2020 Michoski et al. 2020) Ramabathiran & Ramachandran 2021; Ji et al.,
et al., 2021; Huang et al.). Given the same problem, training PINNs fewer samples means fewer restrictions (the PDE constraint that needs to be satisfied) and an increased likelihood of avoiding problematic gradient points, therehy accelerating training. The relaxation of PDE constraint via reducing the sample density makes learning physically meaningful solution feasible, although it may introduce some approximation error to the solution. Note that there is still an open question as to how to strike the best balance between sample density and solution accuracy. PINN methods that can consistently produce more accurate solutions with fewer training samples are nevertheless hugely advantageous for learning complex dynamical evolutions encountered in the real-world.
3.3 LEARNING PINNS ON FEWER SAMPLES
Conventional PINNs evaluate their PDE constraints with either AD or ND. On simple 1D convection-diffusion equation problem as exemplified in Fig PINNs with both AD- and ND-loss exhibit incorrect solution when sparse samples used for training. Note that this is not the only possible failure mode for PINNs (Krishnapriyan et al.) 202I). The issue is that although the training fast and the convergence is good (training losses are close to machine precision, i.c., < le-10), their final solutions are incorrect. Moreover, their failure patterns are different.
125
loss
PINN let,
Ground truth
BCXN-PINN
180
PDE
12-
in betmeem
1.0
Implist local gnciect st nee bounday fined by BK vahas (via lince apuossaatioo))
033
0.1
BCIN--NN
135
Ground trut
ID FDNN
Mi-lee Z0NN ovar-ii aditiooal panmeter"
04-
653
02-
00
33
4.4
0.5
0.5
10
00
02
0.4
0.6
0.3
1.0
Figure 2: Schematic of different PINNs learning the solution of ID convection-diffusion equation. For PINN with ND- and BCXN-loss, external stencil points (x-marked) are evaluated for computing the PDE constraint at near boundary samples. The ground truth solution is the dashed line.
Failure pattern of AD-loss PINN. AD has become a popular method for evaluation of the PDE constraint in PINN loss because it can exactly compute the derivative terms at any sample location directly from the input and the network weights. However, the AD-loss requires dense samples to guide the correct training. When samples sparse relative the local complexity, highly flexible, over weerpparameterized PINN "over-fits" in between regions, leading to incorrect solution even as the PDE constraint is fulfilled at all sample points (Fig. 2 example..
4

---

Under review as a conference paper at ICLR 2023
Failure pattern of ND-loss PINN. On the contrary, ND-loss more robust to sample density. The fundamental difference is that ND-type methods approximate the derivative terms by using PINN output from neighbouring samples, e.g.. finite difference-type stencil, which connects sparse sam- ples into piecewise continuous regions to facilitate fast training across the entire problem domain. The use of lower order approximation to the derivative terms also provides certain structural bias for PINNs to be more easily trained. For example, abnormally high gradients are regularized so that it easier for PINNs learn the solution, with the accuracy being dependent on the approximation employed. Moreover, ND methods do not pose any requirement on differentiability of the network, and they naturally fit the erhhacchie which utilizes a connected set of grid points as input.
The example in Fig. demonstrates that ND-loss can still be subject to undesired training failure. The reason here is because the near-boundary training samples do not perfectly connect to the do- main boundary through their stencils (as illustrated in Fig. 3) Instead, their stencils fall outside the boundary. Their outputs essentially act as free parameters during optimization for satisfying the PDE constraint at near-boundary samples, and the PINN training may "over-fit" the near-boundary region to more easily jointly satisfy the PDE and BC constraints. This becomes a major issue for ND-loss when dealing with irregular geometries, because it is almost impossible to design sample locations such that their stencils perfectly connect the domain boundary to other training samples.
One potential solution is increase the sampling density near the boundary, i.c., AF - 0, to avoid such over-fitting. This however increases the complexity of the optimization problem and can cause adverse effects training. It is also very challenging PINNs with CNN-architecttue. To resolve this issue without increasing the sample density, we propose BCXN-PINY to impose local structure to govern the near boundary gradient behaviour and restore connectivity between domain boundary and near-boundary samples.
locally refine the sampling density for
BC info
Near bo boundary sample
Internal stanci
External stenci
Boundary point
Mirror point
(a) Stencil points connect to boundary Figure 3: (a)-(b) Schematic of near-boundary training sample (red circle) and its stencil points (square) typically used for evaluating PDE constraint in ND-loxs. BC information is passed to stencils in fluid domain (blue square) (a) successfully in regular domain: and (b) unsuccessfully in an irregular domain due to under-defined stencils (unfilled square) outside the domain. (c) Schematic of related definitions under the present BCXN-PINN framework. BCXN-loss enforees a constraint across 3 points: external stencil point A, boundary point P. and mirror point Q inside fluid domain.
(b) Stencil points fall outside domain
(c) Extrapolation along normal direction
METHOD
4.1 ENFORCING LINEAR CONSTRAINT AT NEAR-BOUNDARY SAMPLES
This section outlines the proposed BCXN-PINN method. Here we denote the external (out-of- domain) stencil of near-boundary sample ES point(s). As per Fig. Br. the boundary condition Tnc and field value 7MI are defined at the boundary point and a chosen mirror point Q inside the domain, while the field value ugs at the ES point A is the value to be determined.
By employing Taylor series expansion on points A and Q relative to point P along AQ with respect to local coordinate n (normal direction of solid surface), the following equations can be derived:
TEs = ūBC + AP dunc
AP
+ +((A))
(3a)
dn
2 dn
dūpc
PQ
2
dugc
ÜMI = uBc PQ
+
+ O(P2')
(3b)
dn
2
dn?
5

---

Under review as a conference paper at ICLR 2023
where AP and PQ are the distances between A and P and between P and Q, and AQ A PQ. By further manipulation of the above equations, the field value ugs at point can be derived as:
ugs
UMI
(ūBC (MI)) x
AQ PQ
O(AQ')
(4)
From the above equation, it can be seen that ügs is linearly constrained by ugc and üMI. While the mirror point Q can be conveniently chosen at the center of stencil, i.e., the nean-boundary sample location, we choose mirror point Q along the normal direction of boundary P (ref. Fig. (3) is
more general and performs better in our experiments. Hence, AP = PQ. and Eq. (4) becomes:
igs 2ûBc ÜMI
A similar process can be applied for the derivation of corresponding linear constraints for Neumann type and Robin boundary conditions, as detailed in Appendix Section A..] In addition. PINN meth- ods generally require routine to sample inside the geometry of interest and at the boundary. As compared to conventional PINNs, BCXN-PINN involves additional steps. We describe the steps to compute the mirror point location (along normal direction) in Appendix Section A.2
4.2 BOUNDARY CONNECTIVITY (BCXN)-LOSS WITH DIRECT FORCING APPROACH
Essentially, the use of a linear constraint can connect under-defined external stencil points to the BCs and the solutions from inside the boundary. We can directly apply Eq. (5) to compute the field value ugs for any external stencil point during the evaluation of PDE constraint on near-boundary samples with ND-type schemes. This direct f approach has an association the direct forcing immersed boundary methods in numerical computing. We can then use the following df- NT-BBXN-loss
BPINN( LPDE(dJ-BCCN)
(6)
where LPDE(AS-BCXK) -BX)) is our BCXN-PINN's PDE loss term whereby the PDE constraint on near- boundary samples is modulated by the direct forcing approach. In such an implementation, the BCs are implicitly "infused" into the training loss through the linear constraint and there is no longer need to explicitly evaluate the BC loss.
4.3 BOUNDARY CONNECTIVITY (BCXN)-LOSS WITH SOFT FORCING APPROACH
The df-BCXN-loss strongly enforces the linear constraint during the evaluation of PDE constraint at near-boundary samples. However, the linear condition described by Eq. (5) does not guarantee the best approximation the local gradient behaviour. The imposition of such over-simplified constraint can be inappropriate in some scenarios, hence slowing convergence or reducing accuracy in certain instances. To alleviate the issuc, we propose relax the linear constraint which may be in conflict with the local gradients used for propagating BCs to the near-boundary samples in BCXN-PINN by introducing an additional loss term, in an approach referred to as forcing.
Let us denote ugs as the field value computed at the i-th external stencil point by Eq. (5, and dgs, (X:w) as PINN output at corresponding external stencil point. We then define BCXN-loss as:
LBCXN
"ES [Es: - WES, (X: NES
for all the ES points i i= I,,,,.,. ,ng. ngs which are required for evaluating the PDE constraint using ND- type schemes. The newly introduced LacxN specifies the relation at near boundary points, it also explicitly "infuses" the BC information into the PDE samples, hence helping to propagate the correct BC information during training. The new sf-BCXN-loss is then defined as:
LPINNGJ-BCXM = APDE + LBCLBC BBCX
(8)
with additional weight, 7BCXN, controlling the relative importance of the loss term LACxN in the loss function. When 7BCXN + O, the LPINN(sf-BCXN) reverts to the conventional ND-PINN loss LPINN-
5 RESULTS
We first study the performance of the proposed BCXN-PINN on diverse 2D and 3D fluid dynam- ics test. cases under forward and inverse problem settings using MI.P architecture (inverse problem

---

Under review as a conference paper at ICLR 2023
Then, we demonstrate its efficacy on CNN architecture for both for- ward and meta- modelling problems. We also demonstrate that BCXN-PINN is broadly applicable to different ND-type training losses, including finite difference-type schemes and coupled-automatte numerical differentiation (CAN) scheme.
results are in Appendix A.12
The BCXN-PINN is compared with baseline PINNS which use ND-loss and AID-lsss based on mean squared error (MSE) of the respective PINN solutions' velocity vector iř relative to ground truth. To facilitate comparison, identical network architectures, initialization distribution, and training settings are employed and summarized in Table and Table 3 in Appendix A.33 For each experiment, we perform and present results based on 10 independent runs with different initialization. The ground truth solutions are obtained by an in-house numerical solver based on the improved divergence- free condition compensated (IDFC) method (Chiu 2018). The 4 fluid dynamics test cases (schematics in Appendix A.4) presented are:
2D semi-circle lid-driven cavity flow, Re- 1000. The fluid flow inside the semi-cirele cavity is driven by a lid velocity Wid = 1, Vid = 0 at the top wall. No-slip condition (u
v 0) is applied the other wall. There a primary eddy near the cavity's center.
2D lid-driven cavity flow, Re- 1000. The fluid flow inside I x unit square cavity is driven by a lid velocity und = 1, Vind = 0 at the top wall. No-slip condition is applied to the other walls. The lid-driven cavity flow has been widely chosen as a benchmark case for many numerical methods due to the complex physics encapsulated within. At Re = 1000, there is a primary eddy near the center of the cavity, with secondary and tertiary eddies at the bottom boith and bottom- regions. This test case has a regular domain, but we chose sample locations such that the near boundary samples have their stencils fall outside the domain to show that the proposed method can be useful for this kind of scenarios.
2D wavy channel flow, Re = 100. The fluid flow passing through a long wavy channel is studied. The inlet profile at left boundary is defined as u(0.y) = - + 3, , r= 0. A non-slip condition is applied to the top and bottom walls, while outlet boundary conditions du = p = 0) are applied to the right boundary.
3D bend tube flow, Re = 100. The fluid flow passing through a 90" bending circular tube is studied. The circular channel diameter (D) for the pipe set as 1, while the mean curvature of the bending curve (R) is set as 3. For this problem, the inlet velocity is specified as a fully developed parabolic profile with bulk velocity equal to unity. The length before the bend curve (L1) is set as 3.5, while the length after the bend curve (L2) is set as 10.
5.1
2D AND 3D FORWARD PROBLEMS WITH MLP-ARCIITECTURE BCXN-PINN
In the forward problem experiment, we train PINN models to learn the solution directly from the governing law using different loss functions, i.e., df-BCXN- and sf-BCXN-loss for BCXN-PINNs, and AD- and ND-loss for baseline PINNs. We perform 10 independent runs with different initial- ization for each PINN model. Their learning outcomes which we observe a noticeable improvement in solution accuracy (>2 orders of magnitude lower i7 MSE) with the BCXN-PINN methods. Fig. g Fig. 10 and Fig. 11 in Appendix A.5 visualize the velocity magnitude || and pressure p contours for the median solutions. Visually, the solutions obtained from both df- BCXN- and sf-BCXN-loss have a very good agreement with the ground truth. The results indicate that our present BCXN-PINN method can successfully learn a good solution with low MSE across a diverse set of problems with irregular domains and complex physics. Both the conventional ND- and AD-loss PINNs fail to produce a reasonable solution, i.e., they cannot properly learn the correct flow with current sampling density and training iteration. In addition, we apply our method to a i) real-world relevant problem of flow past an airfoil and ii) multi-physics coupled heat and flow problem. Similar improvements are observed, and results are presented in We discuss the benefits of this connectivity in Appendix A.8
all 4 test cases are shown in Fig. 4 in
Appendix A.6 and A.7
Experiment with different sampling density. We study the 2D lid-driven cavity flow and 3D bend tube channel flow problems to understand the trade-off between the convergence in accuracy and sampling density. We train the same PINN model on a denser (i.e., 3-4x) set of samples until the accuracy of the ND- and AD-loss PINNs start improve. Note that denser sampling requires larger training iteration and batch size, and the tuning of these training arperpparametes can be very time consuming. Fig. b compare the convergence trends in terms of ū MSE between sparse
7

---

Under review as a conference paper at ICLR 2023
107
32
100
10
10
10'5
13
101
10-1
13"
asacin
alcci afeCAV
(a) semi-cirele lid driven cavity flow, Re-1000
(b) 2D lid driven cavity Blw, Re-1000
2D wavy charmel
(d) 3D bend tube
flow, Re-100
flow, Re-100
Figure 4: Distribution of ū MSE from BCXN-PINNS (df-BCXN- and sf sf-sCXN-loss) and PINNs (ND- and AD-loss) while using MiP-atccitecture for the 4 forward problem test cases.
PINN
BCXN-PINN
10P
10a
10-1
10
915,154
butc
10-1
10-1
mose
2 stmple - 245.096
mose
Famplle- 10.400
tetch size 300
londisia - 2000
10--
Fimple - 40,500
10
10-3
hanh 5030
10ah
10th
1.0 Iteration (million)
1.5
0.5
1.0
1.5
2.0
0.5
0.75
0.25
0.5 0.75
1.0
No. (million)
iteration (million)
No. (million)
(a) 3D bend tube sparse sample
(b) bend tube- dense sample
(c) lid driven cavity- sparse sample
(d) 2D lid driven cavity dense sample
Figure 5: PINNs learning the solution of (a-b) 3D bend tube channel flow and (c-d) 2D lid-driven cavity flow under sparse and dense training scenarios respectively. The convergence trends of MSE are plotted. Bold lines indicate the median convergence path, and the shaded areas indicate the inter-percentile range from their 10th 90th percentiles.
(i.e., the one shown in previous section) and dense sampling training scenarios for the 3D bend tube channel flow problem. BCXN-PINN learns more accurate solutions (>1 order of magnitude lower ū MSE) than the conventional PINNs with faster speed (50% less training iterations), while using fewer (<50%) training samples. A similar trend is observed for the 2D lid-driven cavity flow problem (Fig. 5P) where we observe that BCXN-PINN quickly learns an accurate solution under sparse sample training. Although all PINN models can eventually achieve improved accuracy in this test case, Fig. Sit indicates that it is much harder and slower to learn from a dense set of samples, thereby demonstrating the benefits of BCXN-PINN. Additional details are in Appendix A.9
Hyper-parameter tuning, We use settings as per Table | in Appendix A.3 and do not exhaus- tively search for best set of paper-pprameters for the individual PINN models. Apart from sampling density, we observe that PINN learning is sensitive to NN capacity, batch size and learning rate. An- other important hyper-parameter for PINNs is the relative weights XPDE and ÀBc in the loss function LPINN. Specifically, we can further decompose ÂPDE into 'DIV for the divergence free condition (Eq. (T)) and λMом for the conservation momentum equations (Eq. (Ib)) in the PDE loss. A properly tuned ÂDiv can further improve the solution accuracy of BCXN-PINN by another half order of magnitude in i MSE for the 2D wavy channel flow (upry = 5) and 3D bend tube channel flow (NDIv- 20). We provide more details in Appendix A.11
5.2 CNN-ARCHITECTURE BCXN-PINN: FORWARD AND META-MODELLING PROBLEMS
The BCXN-PINN method does not pose any requirement on the differentiable property of the net- works hence it can be freely implemented in any neural network architecture, including CNN. We demonstrate this by using the df- df-BCXN-loss for learning the solution of 2D cami-circle lid-driven cavity flow problem with U-Net architecture. We show that the df-BCXN-loss can benefit CNN- architecture PINNs by remedying training issues stemming from being unable to exactly resolve complex geometries due to the use of a discretized input. We also test the BCXN-PINN method with df-BCCX-
2D lid-driven cavity flow problem to demonstrate its superior performance. The U-Net architecture and training settings are summarized in Appendix Appendix/A.3 Table We train the forward BCXN-PINN models single discretized 2D input which consists of channels, namely the x-, y-coondinatee and an indicator to differentiate fluid and non-fluid domain.
The performance of BCXN-PINN and baseline PINN (i.e., using ND-loss) on the 2 test cases compared in Fig. 6 Like the MLP experiment, BCXN-PINN has a noticeable improvement in the solution accuracy (2-3 orders of magnitude lower MSE). Visually, the solutions obtained from the
8

---

Under review as a conference paper at ICLR 2023
BCXN-PINN have very good agreement with the ground truth, while the baseline PINN fails to
produce a reasonable solution (in Appendix 13. 19). Additional results for a transient version of 2D semi-circle lid-driven cavity flow problem presented in Appendix a.lngl with details on the impact of Re on model error and training time in Appendix A.16
OCXN
BCAN
OCXU
10-3
10-3
10
10-3
10a4
10-4
100k
(a) 2D semi-cirele lid-drive did-driven cavity flow
(b) 2D lid-driven cavity flow
Figure 6: The convergence trends of ū MSE for PINN models using CNN architecture are plotted.
Meta-PINN model. The -Nr-architecture BCXN-PINN represent the entire spatial domain single input. We can train the model on multiple samples, each representing a different scenario such as changes in geometry or physical behaviour. Such a meta-PINN model can be used predict new scenarios. Hence, we further study the ability of BCXN-PINN to learn the solutions to multiple 2D semi-circle lid-driven cavity flow scenarios concurrently (spanning Re = 50 1000). The model architecture and training setting is given in Appendix A.3 Table 3 The df-BCXN-PINN model is used to predict Re = 300, 500, 600, 700, 900 scenarios which are not used during training. The results summarized in Fig. 7 The present BCXN-PINN method successfully learns good solution with
low 7 MSE (i.c., < le-4) 4) for most the Re scenarios, and achieves good prediction accuracy.
10
Train
Test
10-1
10-2
10-3
10-4
10-5
50
100
200
300
400
500
600
700
800
900
1000
Re
Figure 7: BCXN-PINN learning the solutions of multiple 2D semi-circle lid-driven cavity flow scenarios concurrently, using CNN architecture. The ū MSE between ground truth and BCXN- PINN solution for train and test Re plotted. Inset and y-velocity contours of the ground truth and BCXN-PINN solutions for Re 600 and Re = 900.
CONCLUSION
We present BCXN-PINN method which can more efficiently learn the solution to PDEs for com- plex geometry with fewer training samples. This is accomplished by enforcing a linear constraint during the training implicitly and explicitly via direct forcing and soft forcing approaches. While linear constraints are applied in this work due to truncation of the Taylor series expansion in the derivation,
worth studying if different, higher order approximations can also be utilized to achieve better performance in future work. Nonetheless, our comprehensive experimental studies demonstrate practical advantages of BCXN-PINN on diverse test problems in fluid dynamics, span- ning both forward and inverse problems in 2D and 3D, and meta- meta--odel problem, improving the accuracy of solutions by orders of magnitude even while requiring much less training iterations and samples.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Cosmin Anitescu, Elena Atroshchenko, Naif Alajlan, and Timon Rabezuk. Artificial neural network methods for the solution of second order boundary value problems. Computers, Materials and Continua, 59(1):345-35, 2019.
Jens Berg and Kaj Nyström. A unified deep artificial neural network approach to partial differential equations in complex geometries. Neurocomputiing 317:28 41, 2018.
Rafael Bischof and Michael Kraus. Multi-objective loss balancing for physics- deep learn- ing. arXiv preprint (rXiv221000098// 2021.
Pao-Hsiung Chiu. An improved divergence-frre-condtiio compensated method for solving incom- pressible flows on collocated grids. Computers & Fluids, 162:39-54, 2018.
Pao-Hsiung Chiu and Hee Joo Poh. Development of an improved divergence-free- freeccdiion com- pensated coupled framework to solve flow problems with time-varying geometries. International Journal for Numerical Methods in Fluids, 93(1):44-70, 2021.
Pao Pao-Hsiung Chiu, Jian Cheng Wong, Chinchun Ooi, My Ha Dao, and Yew-Soon Ong. Can-pinn: differentiation
A fast physics-informed neural network based on coupled-atiomatic-numeee method. Computer Methods in Applied Mechanics and Engineering, 395:1 14909, 2022.
Salvatore Cuomo, Vincenzo Schiano Di Cola, Fabio Giampaolo, Gianluigi Rozza, Maizar Raissi, and Francesco Piccialli. Scientifie machine learning through thysics-informet neural networks: Where we are and what's next. arXiv preprint arXiv:2201.05664 2022.
Arka Daw, Jie Bu, Sifan Wang, Paris Perdikaris, and Anuj Karpatne. Rethinking the importance of sampling physics-informer neural networks. arXiv preprint arXiv:2207 02333,,3 2022.
Taco de Wolff, Hugo Carrillo Lincopi, Luis Marti, and Nayat Sanchez-Pi Mopinns: an evolutionary multi-objective approach to physics-infomma neural networks. In Proceedings of the Genetic and Evolutionary Computation Conference Companion, pp. 228-231, 2022.
Suchuan Dong and Zongwei Li. Local extreme learning machines and domain decomposition for solving linear and nonlinear partial differential equations. Computer Methods in Applied Mechan- ics and Engineering, 387:1 14129, 2021.
Mohannad Elhamod, Jie Bu, Christopher Singh, Matthew Redell, Abantika Ghosh, Viktor Podol- skiy, Wei-Cheng Lee, and Anuj Karpatne. Cophy-pgnn: Learning physics-guided neural networks with competing loss functions for solving eigenvalue problems. ACM Transactions on Intelligent Systems and Technology (TIST), 2020.
Zhiwei Fang. A high-efficient hybrid physics-infommed neural networks based on convolutional neural network. IEEE Transactions on Neural Networks and Learning Systems, 2021.
Olga Fuks and Hamdi A Tchelepi. Limitations of physics informed machine learning for nonlinear two-phase transport in porous media. Journal of Machine Learning for Modeling and Computing, 1(1), 2020.
Han Gao, Luning Sun, and -ian-Xun Wang. Phygeonet: Physics-informed geometry addatity- convo- lutional neural networks for solving parameterized steady-state pdes on irregular domain. Journal of Computational Physics, 428:1 10079, 2021.
Xiang Huang, Hongsheng Liu, Beiji Shi, Zidong Wang. Kang Yang. Yang Li, Min Wang, Haotian Chu, Jing Zhou, Fan Yu, et al. universal pinns method for solving partial differential equations with a point source.
Ameya D Jagtap and George E Karniadakis. Extended Ahysics-informed neural networks (xpinns): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations. In AAAI Spring Symposium: MLPS, 2021.
Ameya D Jagtap, Ehsan Kharazmi, and George Em Karniadakis. Conservative chysics-informel neural networks on discrete domains for conservation laws: Applications to forward and inverse problems. Computer Methods in Applied Mechanics and Engineering, 365:1 13028, 2020.
10

---

Under review as a conference paper at ICLR 2023
Weiqi Ji, Weilun Qiu, Zhiyu Shi, Shaowu Pan, and Sili Deng. Stiff-pinn: Physics-informit neural network for stiff chemical kinetics. The Journal of Physical Chemistry A, 125(36):8098-3))) -0000000888800000088000000000808 2021.
Xiaowei Jin, Shengze Cai. Hui Li, and George Em Kamiaaakis. Nsfnets (navier-stokes flow nets): Physics-informed neural networks for the incompressible navier-stokes equations. Journal of Computational Physics, 126:109 2021.
George Em Karniadakis, loannis G Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, and Liu Yang. Physics-informed machine learning. Nature Reviews Physics, 3(6))422-440, 2021.
Sharmila Karumuri, Rohit Tripathy, Ilias Bilionis, and Jitesh Panchal. Simulator-free solution of high-dimensional stochastic elliptic partial differential equations using deep neural networks. Journal of Computational Physics, 104:1091 2020.
Ehsan Kharazmi, Zhongqiang Zhang, and George Em Karniadakis. hp-vpinns: Variational physics- informed neural networks with domain decomposition. Computer Methods in Applied Mechanics and Engineering, 374:1 13547, 2021.
Aditi Krishnapriyan, Amir Gholami, Shandian Zhe, Robert Kirby, and Michael W Mahoney. Char- acterizing possible failure modes in physics-informed neural networks. Advances in Neural In- formation Processing Systems, 1726548-26560) 2021.
Isaac E Lagaris, Aristidis Likas, and Dimitrios I Fotiadis. Artificial neural networks for solving ordinary and partial differential equations. IEEE transactions on neural networks, 95):9877-1000, 1998.
Isaac Lagaris, Aristidis C. Likas, and Dimitris Papageorgiou. Neural-network methods for boundary value problems with irregular boundaries. IEEE Transactions on Neural Networks, 11 (5):1041-1049, 2000.
Ke Li, Kejun Tang, Tianfan Wu, and Qifeng Liao. D3m: A deep domain decomposition method for partial differential equations. IEEE Access, 8:5283-5228 2019.
Lu Lu, Xuhui Meng, Zhiping Mao, and George Em Karniadakis. Deepxde: A deep learning library for solving differential equations. SIAM Review, 63(1):208 228, 2021a.
Lu Lu, Raphael Pestourie, Wenjie Yao, Zhicheng Wang, Francesc Verdugo, and Steven G Johnson. Physics-informed neural networks with hard constraints for inverse design. SIAM Journal on Scientific Computing, 43(6):B1 (05-B1132, 2021b.
Yiping Lu, Haoxuan Chen, Jianfeng Lu, Lexing Ying, and Jose Blanchet. Machine learning for elliptic pdes: fast rate generalization bound, neural scaling law and minimax optimality. arXiv preprint arXiv:21 16.067 2021c.
Didier Lucor, Atul Agrawal, and Anne Sergent. Physics-aware deep neural networks for surrogate modeling of turbulent natural convection. arXiv preprint arXiv:2103.03300 2021.
Kjetil O Lye, Siddhartha Mishra, Deep Ray, and Praveen Chandrashekar. Iterative surrogate model optimization (ismo): an active learning algorithm for pde constrained optimization with deep neural networks. Computer Methods in Applied Mechanics and Engineering, 374:113577 2021.
Suryanarayana Maddu, Dominik Sturm, Christian L Müller, and Ivo Sbalzarini. Inverse dirichlet weighting enables reliable training of physics informed neural networks. Machine Learning: Science and Technology, 311:0110026, 2022.
Stefano Markidis. The old and the new: Can physics-infomne informed deep-learning replace traditional linear solvers? Frontiers big Data, pp. 92, 2021.
Revanth Mattey and Susanta Ghosh. A novel sequential method to train physies informed neural networks for allen cahn and cahn hilliard equations. Computer Methods in Applied Mechanics and Engineering, 390:1 14474, 2022.
11

---

Under review as a conference paper at ICLR 2023
Levi McClenny and Ulisses Braga-Neto. Self-adaptive physics-informet neural networks using soft attention mechanism. arXiv preprint 2xXiv:2009.045444 2020.
Kevin Stanley McFall and James Robert Mahan. Artificial neural network method for solution of boundary value problems with exact satisfaction of arbitrary boundary conditions. IEEE Trans- actions on Neural Networks, 20(8):1221-122) 2009.
Craig Michoski, Miloš Milosavljević, Todd Oliver, and David R Hatch, Solving differential equa- tions using deep neural networks. Neurocomputing, 399:193-211 2020.
Siddhartha Mishra and Roberto Molinaro. Estimates on the generalization error of physics-inforne neural networks for approximating a class of inverse problems for pdes. IMA Journal of Numerical Analysis, 42(2):981-1022, 2022.
Sebastian K Mitusch, Simon W Funke, and Miroslav Kuchta. Hybrid fem-nn models: Combining artificial neural networks with the finite element method. Journal of Computational Physics, 446: 110651, 2021.
Mohammad Amin Nabian and Hadi Meidani. A deep learning solution approach for high- dimensional random differential equations. Probabilistic Engineering Mechanics, 57:14-25, 2019.
Mohammad Amin Nabian, Rini Jasmine Gladstone, and Hadi Meidani. Efficient training of physics- informed neural networks via importance sampling. Computer-Aided Civil and Infrastructure Engineering, 3668):922-977,222 2021.
Stanley Osher and Nikos Paragios. Geometric level set methods in imaging, vision, and graphics. Springer Science & Business Media, 2003.
Tobias Pfaff, Meire Fortunato, Alvaro Sanzaez-Gonzalez, and Peter W Battaglia. Learning mesh- based simulation with graph networks. arXiv preprint arXiv:2010.03409 2020.
Maziar Raissi, Paris Perdikaris, and George Karniadakis. Physics-informi neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686-707, 2019.
Amuthan Ramabathiran and Prabhu Ramachandran. Spinn: Sparse, physics-based, and partially interpretable neural networks for pdes. Journal of Computational Physics, 146:10 2021.
Rishikesh Ranade, Chris Hill, and Jay Pathak. Discretizationnet: A machine-learning based solver for navier-stokes equations using finite volume discretization. Computer Methods in Applied Mechanics and Engineering, 378: 13722, 2021.
Pu Ren, Chengping Rao, Yang Liu, Xun Wang, and Hao Sun. Phycrnet: Physics-informed convolutional-recurren network for solving spatiotemporal pdes. Computer Methods Applied Mechanics and Engineering, 389:1 14399, 2022.
Yeonjong Shin, Jerome Darbon, and George Em Karniadakis, On the convergence of physics in- formed neural networks for linear second-order elliptic and parabolic type pdes. arXiv preprint arXiv:2004.0/886, 2020.
Khemraj Shukla, Ameya D Jagtap, and George Em Karniadakis, Parallel physics-informed neural networks via domain decomposition. Journal of Computational Physics, 447:110683, 2021.
Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein. Im- plicit neural representations with periodic activation functions. Advances in Neural Information Processing Systems, 33:7462-7473. 2020.
Pongpisit Thanasutives. Masayuki Numao, and Ken-ichi Fukui. Adversarial multi-task learning enhanced physics-informed neural networks for solving partial differential equations. 2027 International Joint Conference Neural Networks (IJCNN), pp. 1-9. IEEE, 2021.
Remco van der Meer, Comelis W Oosterlee, and Anastasia Borovykh. Optimally weighted loss functions for solving pdes with neural networks. Journal of Computational and Applied Mathe- matics, 405:1 13887, 2022.
12

---

Under review as a conference paper at ICLR 2023
Nils Wandel, Michael Weinmann, and Reinhard Klein. Learning incompressible fluid dynam- ics from scratch-towan fast, differentiable fluid models that generalize. arXiv preprint wrXiv:2006.08862, 2020.
Nils Wandel, Michael Weinmann, and Reinhard Klein. Teaching the incompressible navier-stoker equations to fast neural surrogate models in three dimensions. Physics of Fluids, 33(4):047117, 2021.
Nils Wandel, Michael Weinmann, Michael Neidlin, and Reinhard Klein. Spline-pinn: Approaching pdes without data using fast, physics-informen Conference on Artificial Intelligence, volume 36, pp. 8529 8538, 2022.
sermite-spline cnns. In Proceedings of the AAAI
Sifan Wang, Yujun Teng, and Paris Perdikaris. Understanding and mitigating gradient flow patholo- gies in physics-informed neural networks. SIAM Journal on Scientific Computing, A3055 A3081, 2021a.
Sifan Wang. Hanwen Wang, and Paris Perdikaris. On the cigenvector bias of fourier feature net- works: From regression to solving multi-scale pdes with physics-ifformed neural networks. Com- puter Methods in Applied Mechanics and Engineering, 384: 13938, 2021b.
Sifan Wang, Shyam Sankaran, and Paris Perdikaris. Respecting causality is all you need for training physics-informed neural networks. arXiv preprint arXiv:2203.07400 2022.
Zhongjian Wang and Zhiwen Zhang. A mesh-free method for interface problems using the deep learning approach. Journal of Computational Physics, 400:108963,, 2020.
Colby L Wight and Jia Zhao. Solving allen-cahn and cahn-hilliard equations using the adaptive physics informed neural networks. arXiv preprint 04X1v22007.04542. 2020.
Jian Cheng Wong, Chinchun Ooi, Abhishek Gupta, and Yew-Soon Ong. Learning in sinusoidal spaces with physics-informed neural networks. IEEE Transactions Artificial Intelligence, 2022.
Chenxi Wu, Min Zhu, Qinyang Tan, Yadhu Kartha, and Lu Lu. A comprehensive study of non- adaptive and residual-based adaptive sampling for physics-infrmmed neural networks. Computer Methods in Applied Mechanics and Engineering, 403:115671, 2023.
Zixue Xiang, Wei Peng, Xu Liu, and Wen Yao. Self-adaptive loss balanced physics-informed neural networks. Neurocomputing, 496: 1134, 2022.
Qi Zeng. Spencer Bryngelson, and Florian Schäfer. Competitive physics informed networks. arXiv preprint 2rX11:2044111144, 2022.
A APPENDIX
A.1 DERIVATION OF LINEAR CONSTRAINT FOR NEUMANN TYPE AND ROBIN TYPE BOUNDARY CONDITION
Based on Eq. the linear constraint for Neumann type boundary condition can be similarly derived utilizing Eqn
b):
dūRC dn
AP
-PQ'
2
Tgs UMl = -Q
+
+(AA0
(9)
2
dn²
or
ügs = uMI +
AQ
dūgC dn
+ O - -PQ²)
(10)
For the homogeneous Neumann boundary condition
aag dn
= 0), Eq.
10
can be further simplified
as
ugs
Umj
(11)
13

---

Under review as a conference paper at ICLR 2023
A similar treatment can be used to derive corresponding linear constraint for problems with Robin type boundary condition:
duBC =c
BBC + B dn
(12)
is then possible to obtain an appropriate expression by substituting the above expression into Eq.
dugc dn
AQ
ugs = UMI + ((c
uMI)
(13)
PQ
In the above, dn
duge is approximated by Eq. 10
We forward demonstrate the ability of the current proposed method handle various combinations of Dirichlet and Neumann type boundary conditions in Appendix A.14
A.2 PROCEDURE TO EVALUATE THE FIELD VALUE UMI AT MIRROR POINT
Note that PINN methods in general require routine to sample from inside the geometry of interest. as well as the boundary. As compared to conventional PINNs, BCXN-PINN involves additional steps. We first describe the steps to compute the mirror point location (along normal direction):
Determine whether a stencil external. In this study, we first construct level set func- tion & of the geometry of interest (Osher & Paragios, 2003), where the shortest distance AP between the stencil point A and boundary point P can be found. From the sign of level set function,
determine whether stencil is external.
Compute mirror point location for external steneil. Once AP determined, we compute the location of mirror point Q inside the fluid domain, with the distance AP = -QQ
In practice given fixed set of training samples, all the external stencil points and their mirror locations can be pre-computed. On the other hand, the field value uæ at mirror points depends on PINN's output and are evaluated inside a training iteration:
MLP-architecture PINN. The üии value can be directly obtained by evaluating (X:w) at the mirror point.
-NN-architecture PINN. The location of mirror point may not coincide with the CNN grid. In such case, the following inverseddistance-ve weighted interpolation function has been utilized to obtain the иы value at mirror point (Chiu & Poh 2021).
X
y-
u(x,5) = Eu(x,y)0'
(14a)
h
h
x x-
x x-x
(14b)
h
/E0
(14c)
h
(33 13-2/r -
(())
5 5--2 2/rl 2 12 4
1< $2
(14d)
|||>
(E,5) is the location of mirror point to be interpolated, while (x,y) is the location of training samples inside the fluid domain. In this study, à is chosen as Ax, which leads to a 2Ax interpolation radius. Once āæ and uBc evaluated, we can compute ugs based on Eq. (5) during training.
14

---

Under review as a conference paper at ICLR 2023
A.3 MLP AND CNN EXPERIMENTAL SETTINGS
Table 1: PINN (MLP) architecture and training settings used for experiments in Section 5.1
Forward Problem
2
3
4
2D semi-circle lid-driven cavity flow, Re = 1000
2D lid-driven cavity flow, Re 1000
2D wavy channel flow, Re 100
3D bend tube channel flow, Re 100
Test case
2D incompressible NS eqns.
2D incompressible NS eqns.
2D incompressible NS eqns.
3D incompressible NS eqns.
Governing physics eqns.
(x,y,z)
MLP architecture for BCXN-PINM
(x.y)- -
(x,y)
(x.y)
128-30-30-38
64-30-30-30- 30-30-30-(O1)
128-30-30-22
128 40 40 40-
[30-30-30-(1 (f),
(30-30-30-(1)
(40-40-40-(2)) (2).
30-30-30-(1),
ND-PINN & AD-PINN
30-30-30-
30-30-30-(())
40-40-40-
30-30-30- W))
30-30-30-(())
30-30-30-(P))
30-30-30-($)
ND scheme for BCXN-PINNs
CAN & finite difference
Finite difference
CAN
CAN
& ND-PINN
Training
sample (equidistantly spaced samples @ fluid domain & boundary)
223,612 + 22,
3,930+300
10, 00,000+4000
14, 14,400 + 800
Batch size
1,000
2,000
1,000
500
Training iterations
300,000
300,000
300,000
1,000,000 000
(I) For the MLP architecture, the numbers in between input and output represent the number of nodes in each hidden layer. For example, (x) 6x)-64-20. (04-00-20 204-20-20-20--22 10-20-20--(i) indicates a single input x, followed by hidden layers with 64, 20, 20 and 20 nodes in each layer, and a single output û.
(II) We incorporate the sinusoidal mapping (Wong et al.) 2022) into the first hidden layer of PINN and initialize its weights by sampling from normal distribution N(0, o²), -= 1. The subsequent hidden layers use "sine" activation, except a "linear" activation function is used in the final (output) layer, and their weights are initialized by He uniform distribution.
(III) Batch size: number of random sample used for evaluation of LPINN(= APDE LPDE + PBC L-c),, 1
APDE [-DDEdJ--CXXV and BBINNG APDE LPDE + NBCXN LBCXN + >BBC LBC). We used default APDE 1, 7BC 1, and ÀBCXN 1, unless otherwise mentioned.
(IV) A training iteration: evaluation of LPINN, LPINNaa- BBX))) or LPINN(sf - BCXN) for back- propagating the weight gradients. We use an initial learning of le-3 and reduce it plateau- ing, until a min. learning rate of 5e-6 is reached.
15

---

Under review as a conference paper at ICLR 2023
Table 2: PINN (MI.P) architecture and training settings used for experiments described in Appendix
Forward Problem
5
7
2D lid-driven cavity flow and heat transfer with a cylinder inside, Re = 100
2D flow past airfoil shape, Re 500
1D convection diffusion problem
Test case
2D incompressible NS eqns. energy equation for temperature
2D incompressible NS eqns.
Governing physics eqns.
20u² Mxx
(x,y)
MLP architecture for BCXN-PINN,
(x,y)
64-30-30-30-
64 14-40-40- 40-40-
(30-30-30-()).
(x,y)
(00440-40(11))
30-30-30-(1),,
14-044444--
ND-PINN AD-PINN
40 40-4))0--())
30-30-30-($),,
40-40-40-(P))
30-30-30-
ND scheme for BCXN-PINNs
CAN & finite difference
CAN
CAN
& ND-PINN
Training
sample
equudisttnnty spaced samples @ fluid domain boundary)
78,690
1,450
2, 184 + 250
30+2
Batch size
500
1,000
15
Training iterations
500,000
100,000
50, 000
(I) For the MLP architecture, the numbers in between input and output represent the number of nodes each hidden layer. For example, 10---42--2-----0 20)(()))))))))))))))))) indicates a single input x, followed by hidden layers with 64, 20, 20 and 20 nodes in each layer, and a single output û.
(II) We incorporate the sinusoidal mapping (Wong et al. 2022) into the first hidden layer of PINN and initialize its weights by sampling from normal distribution (:0 o²), 1. The subsequent hidden layers use "sine" activation, except "linear" activation function is used in the final (output) layer, and their weights are initialized by He uniform distribution.
(III) Batch size: number of random sample used for evaluation of [PINN(= MPDE LPDE + NBC LBC), LPINNUJ- NaCxN LBCxN + >BpC Lac). We used default ApDE = 1, A = 1, and 7RCXN 1, unless otherwise mentioned.
ÂPDE L-DE(dJ-BCXN). and 1PINNG//BCXNN] APDE LPDE +
(IV) A training iteration:
evaluation of LPINN, LPINNaa- or LPNNw(sf- BCXN) for back- propagating the weight gradients. We use an initial learning of 1e-3 and reduce it on plateau- ing, until min. learning rate of 5c-6 is reached.
16

---

Under review as a conference paper at ICLR 2023
Table 3: PINN (CNN) architecture and training settings used in the experiments in Section 5.2
Problem type
Forward
Forward
Meta-modelling
2D semi-cirele
2D semi-circle
2D lid did-nriven cavity
Test case
lid cavity flow, Re- = 1000
lid-driven cavity flow = 1000
flow, Re 1000
Governing physics eqns.
2D incompressible NS eqns.
2D incompressible NS eqns.
2D incompressible NS eqns.
(x(6611123))
(X (X110x104x33)
(x(56x112x4) 8-8-1-16-16
8-8---16-16- 8-8 8-8-1-16-16-4
CNN (U-Net)
32-32 1 64-64-1
32-32- 164-644-------
32-32- 1 -4-664 64 12-32--11 t
architecture for BCXN-PINE & ND-PINN
-, --32-22-1 1
-, 32-32-1 1
16-16- -6
166-16 --6
166 t-6
8 8 8 (0m56x112),
8-8 8 10
8-8 8 (0 (Un.6x(112),
8-8-8 (0,,6xx112)
8-8 8 (0 (0,10000000)
8-8 10088000,,0 (1x)
8-8 8
Ü p.56x112)
8 8- (0 104x
8-8-8 Ü p,56x112)
Kernel size
4 4
6 6
6x6
ND scheme for BCXN-PINNS & ND-PINN
Finite difference
Finite difference
Finite difference
Training sample batch size
1/1
1/1
6/
Training iterations
200,000
200, 000
500,000 000
(I) For the U-net architecture, the numbers in between input and output represent the number of filters in each hidden layer. Each hidden layer consists of convolution operation, batch nor- malization, and nonlinear "sine" activation. Also, represents 2 x 2 max-pooling operation, and f represents a 2 x up-sampling operation. The network weights are initialized by He uniform distribution.
(1I) Batch size: number of sample used for 1 evaluation of LpINN(= NPDE LPDE + ABC Loc) and LPPNNid/-BCMM(-AAM APDE LPDE(ds-BCXN) We used a default NPDE = and NBC = 1, unless otherwise mentioned.
(III) A training iteration: 1 evaluation of LPINN, LPINwiaJ-BCxn or LPINN(sf- (CX)) for backprop- agating the weight gradients. We use an initial learning of 1e-3 and reduce it on plateauing, until a min. learning rate of 5e-6 is reached.
(IV) The semi-circle and regular lid-driven cavity geometries consist of 3,930 and 10,000 interior points which belong to the fluid domain respectively.
17

---

Under review as a conference paper at ICLR 2023
Table 4: PINN (CNN) architecture and training settings used in the experiments described in Ap- pendix
Problem type
Forward
Forward
2D transient
2D Taylor-Couette
semi-circular lid-driven cavity flow, Re- = 500
Test case
flow, Re 1000
2D transient incompressible NS eqns.
Governing physics eqns.
2D incompressible NS eqns.
(X 56x56x3)
(xxx 8-8- 1-8---16-16
8-8 16-1661 16--16-1
CNN (U-Net)
32-32 t -6446466 64-1
-64 641 -, 32-32-2 1
architecture for BCXN-PIN) & ND-PINN
- 32-32-1
-1 -16-16-11 1 +
-16-16-1 1 t-
8 8-8-8-(0 8 (-,,566655))))
8-8 8-8-8- 8-888-(0.566x11 (0u,56x112,,
8-8-8 83-8-(12 (,,,6666))))
8 8-8-8- (U
8 8 8 (0,.66556)
8 8 8 (0 (0,56x1122
Kernel size
4 x 4
4 4x4 4
ND scheme for BCXN-PINM & ND-PINN
Finite difference
Finite difference
Training sample / batch size
1/1
21 / 2
Training iterations
50,000
100,000
(I) For the U-net architecture, the numbers in between input and output represent the number of filters in each hidden layer. Each hidden layer consists of convolution operation, batch nor- malization, and nonlinear "sine" activation. Also, + represents 2 x 2 max-poot max-pooling operation, and f represents 2 x up-sampling operation. The network weights initialized by He uniform distribution.
(II) Batch size: number of sample used for 1 evaluation of LPINN NPDE LPDE + \BC LBC) and LPINNid- (BX)) ÂPDE (oDE(dF-BCXN)) We used default XPDE and ÂBC 1, unless otherwise mentioned.
(III) A training iteration:
evaluation of LPINN, (PINNidJ-BCXN) or LPPNNGSS-BCMM) for backprop- agating the weight gradients. We use initial learning of le-3 and reduce it plateauing, until a min. learning rate of 5e-6 reached.
(IV) The Cayltr-Couette geometry consists of 1,216 interior points in the fluid domain while the transient semi-circle lid-driven cavity problem consists of 3,930 interior spatial points and 21 temporal snapshots (Ar 0.1) respectively.
In addition, we summarize the training results across different problems and model architectures in 2 Summary Tables in Appendix A.18
A.4 PROBLEM SCHEMATIC FOR FLUID DYNAMIC CASE STUDIES
The following figure is schematics of the fluid dynamic case studies in this work, depict the 2D semi-circle lid-driven cavity, 2D lid-driven cavity, 2D wavy channel and 3D bend tube respectively.
18

---

Under review as a conference paper at ICLR 2023
4.- -.V0
Primary vortex
U=V=
0.3
U-v-O
(a)
(b)
Rea
-1
D-1
Cuekt
(d)
Figure 8: Schematic of the investigated problem. (a) 2D semi-circle lid-driven cavity flow problem; (b) 2D lid iddvdriee cavity flow problem; (c) 2D wavy channel flow problem; (d) 3D bend tube flow problem.
A.5 CONTOUR PLOTS FOR FORWARD PROBLEM WITH MLP-ARCHITECTURE BCXN-PIN PINN
The following figures are visualizations of the velocity and pressure contours obtained from the problems described in Section 5.1 Fig. Fig. 10 and Fig. are visualizations of the velocity magnitude || and pressure contours for the median solutions.
A.6 ADDITIONAL FORWARD PROBLEM ON 2D FLOW PAST AIRFOIL SHAPE, Re = 500
To further validate the applicability and efficacy of the present BCXN-loss method for more real- world complex geometries, we also applied the ML.-architectun BCXN-PINN to the modelling of incompressible flow past an airfoil, similar other previous work (Pffff al. 2020).
As shown in Fig. 12 an airfoil with a NACA0012 profile is placed in a modelling domain (x.y) = 33 2 ~ 2) with zero angle attack. uniform inlet velocity, Minter = 1, is specified a 3, and zero pressure outlet boundary condition is specified at xt 5. Typical slip boundary conditions are specified at the side walls. The PINN model details are described Table 3 Briefly, the total training samples for this problem is 79,886, with the sample spacing in the domain corre- sponding to Ar= = Ay = 1/50. In addition, the weight for BCXN-lost is set to be O.I. Contour plots of the obtained velocities for these experiments are shown in Fig. 13
19

---

Under review as a conference paper at ICLR 2025 0
Figure 9; Comparison of velocity magnitude [ and pressure p contour between the ground truth and PINN solutions (median MSE from 10 runs) for the (a) 2D semi-cirel lid-driven cavity flow problem, Re 1000, and (b) 2D lid-driven cavity flow problem, Re 1000.
19
19
24
14
19
24
2
Vdl
0
gound buts
12
4
-03
na
30
000
000
31
80
000
223
900
90
06
221
00
200
96
00
21
-5
22
10
05
33
25
20
60-
10-
700
20
-1a
1
12
-04
-03
8a
000
20
000
210
00
220
200
315
250
200
000
213
35
399
00
255
280
100
30
90
20
32
22
22
-nn
-56
(b)
10
18
the
-02
130
699
100
000
900
000
220
100
019
250
100
100
111
000
110
220
00
0
25
25
200
20
-04
-ss
ON
12
4
-03
08
000
0
190
220
000
319
230
090
200
000
110
199
000
-1.
70
700
10
10
-
22
10.00
-04
-33
-03
no
016
5.0
-33
11
1060
Ane
r50
000
20

---

Under review as a conference paper at ICLR 2023 3
flow problem, Re = 100.
Figure 10: Comparison of ur, v-velocity and pressure contour between the ground truth and PINN solutions (median MSE from 10 (urns)) for the 2D wavy channel
AD loos
ND loss
10
BrAK
AD-ovo
ND
J0
1.2
140
100
000
100
100
1000
+:00
100
AD-loxt DD--oos
ND
BCW
22
15
8:00
8-00
21

---

Under review as a conference paper at ICLR 2023
Figure I:: Comparison of velocity magnitude ||z|| and pressure contour between the ground truth and PINN solutions (median MSE from 10 runs), for the 3D bend tube channel flow problem. Re = 100. Plots are of a 2D cross section at z = 0.0125
gound
bott
1.00
2.00
0:24
8:48
5
di-BCONiioos
Schematic diagram 3D bend tube channel flow
000
0.55
160
-25
100
5225
6000
0.00
0026
0.68
(b) Contour plot of velocity magnitude
(c) Contour plot of Pressure
stackWuss
3FCCKV-tos
field field
0.55
1.50
22:2
3.00
0.00
0.08
3
MO-loss
150
222
375
0000
0:36
045
AC-10S
1.550
+2.5
0.00
F224
22

---

Under review as a conference paper at ICLR 2023
Juioy
duiox dviex 0
0
auley = 0
2
2
Figure 12: Schematic of 2D flow past airfoil shape problem.
gound
0.2
0.0
-0.4
0.16
0.00
0.0
0.5
1.0
1.5
2.0
2.5
3.0
4.0
BCXN-loss
0.4
0.98
0.80
0.0
0.64
0.48
0.32
0.16
0.00
ND-loss
1.12
0.96
0.2
0.80
0.64
-02
0.0
6.
1.0
1.5
2.0
2.5
3.6
4.0
02
0.0
-044
6.16
0.00
0.0
0.5
1.5
2.0
2.5
3.0
4.0
Figure 13: Contour plots of velocity fields obtained from 4 models: (Top) CFD, (2nd from Top)
BCXN-loss, (2nd from Bottom) ND-loss, (Bottom) AD-loss.
23

---

Under review as a conference paper at ICLR 2023
Based on these numerical experiments, similarly notice that the incorporation of BCXN-loss improves the PINN's ability to learn the flow profile around the airfoil by an order of magnitude. Quantitative results are provided in Table 5
Table 5: Model Performance for NACA 0012
Model
MSE
Rel. Error
sf- sf-BCXN-CAN
4.65 x 10
8.10 x 10
CAN
3.32 x 10
1.92 x 10
sf-CCXNND
5.03 x 10-5
6.93 x 10-2
ND
3.54 x 10
1.84 x 10
AD
1.18 x 10
3.17 x 10
A.7 ADDITIONAL FORWARD PROBLEM ON 2D LID-DRIVEN CAVITY FLOW WITH HEAT TRANSFER, Re 100
We further proposed validate the applicability of the proposed method multi-physit example.
In this instance, we applied the archtecceee BCXN-PINN to the modelling of incompressible flow with heat transfer under Boussinesq approximation:
V ū = 0
(15a)
( V) ū = Re Ai vp+ RiOE
(15b)
V) 1
(15c)
in the above, Pr is Prandtl number, Ri is Richardson number, and ē= (0,1).
In the present study, there is 2D lid-driven cavity domain with stationary cylinder of radius 0.2 the centre of the domain, while is set as 0.7 and Ri is set as 1. For the boundary conditions. the bottom wall is set as the hot surface(O = 1) and top wall is set the cold surface (0 = 0). In addition, the side walls and cylinder surface are specified as adiabatic boundary conditions. The PINN model details are described in detail in Table 3 while final results are presented in Table 6
Table 6: Model Performance
d dn
d0/dr. dn =
d
MLP-Model
Variable
MSE
Rel Eir
MSE
Rel Eir
1.31 x 10
3.42 10
4.45 x 10
2.00 10
CC-BCXN-CAN
O
5.06 x 10
3.77 10
8.81 x 10
2.64 10-1
2.32 x 10
4.40 10
6.87 10-4
2.02 10-2
CAN
6.76 x 10
4.27 10
1.07 x 10
2.66 10-1
3.41 x 10
1.85 10
3.89 x 10
2.21 10-1
AD
8.23 x 10 8.77 x 10-1
8.65 x
8.80 10
Based on numerical experiments, the BCXN-loss appears to also yield some benefit multi- physics problems, with consistently lower MSE and relative I.2 errors than the baseline comparisons. However, more thorough experiments need to be conducted to fully validate situations whereby the BCXN-loss may be more beneficial.
A.8 CASE STUDY OF INFORMATION PROPAGATION
Information propagation from the boundary to the inner domain has been suggested to be critical for successful PINN modelling. Hence, investigate the evolution of the PDE residual across multiple training runs for the 2D semi-cirele lid-driven cavity with and without the imposition of the BCXN-loss better understand the impact of enhanced connectivity imposed by the BCXN-loss. Illustrative contour plots of the PDE residuals and the corresponding velocity contours at various training iterations are provided in Fig. and and Fig. 15
24

---

Under review as a conference paper at ICLR 2023
70th coddoal After bur.
13
103
240
Figure 14: Contour plots of (Top) PDE error residual and (Bottom) velocity for ND-PINN.
PhDs roddial ofter
13
102
Figure 15: Contour plots of (Top) PDE error residual and (Bottom) velocity for BCXN-PINN.
As described in the previous sections, the ND-PINN typically fails to converge a good model even after 300,000 training iterations, but this be improved via the addition of a BCXN-loss during training. The velocity contours clearly illustrate the improved convergence. striking difference in the distribution of the PDE residuals across the domain during the early stages of the training. a typical ND-PINN, the PDE residuals rapidly reduce to zero, especially in the parts of the domain far away from the boundary condition. This also corresponds to the ND-PINN velocity profile converging to a wrong solution, and is indicative of the information from the boundary condition being lost within the domain. On the other hand, the PDE residuals within the domain persist for a much longer period when the BCXN-loss is included which is indicative of information from the top boundary being persistently transmitted to the inner domain until the PINN model has converged. This is also consistent with the physical dynamics of the system, we note that the lid-driven cavity flow problem, as the name suggests, is heavily dependent on the top boundary's 's motion to drive the dynamical flow within the entire domain.
addition, there is
A.9 SAMPLING EFFECT ON 2D LID-DRIVEN CAVITY FLOW
To illustrate the impact of having bear-boundary stencil points outside of the domain, we further investigate the impact of two different sets of sampling points: i) Cell-centered (whereby near- boundary points need support points outside the domain) and ii) Node-centered (whereby near- boundary points use support points that are coincident with the domain boundary).
These two sets of sampling points are tested on a lid-driven cavity flow at Re = 1000 with differ- ent training batch sizes and point sampling density for the df-BCXN-CAAI CAN and AD models, and are trained for 300, 000 iterations each. The model MSE and L2 relative errors are collated across independent runs and tabulated in Table 7
In the above table, cells are colored according to the model MSE at the end of 300,000 training iterations. Brown cells indicate scenarios under which no meaningful convergence observed (with MSE > le-2). Yellow, green and blue cells indicate scenarios with increasingly good model performance, with each band indicating order of magnitude improvement in MSE.
25

---

Under review as a conference paper at ICLR 2023
Table 7: Model Performance for Different Sampling Densities, [C] and [N] refer cell-centered
and node-centered sampling point arrangements.
50 X 50
100 x
100
Batch Size
Points
Model
MSH
Rel Error
MSH
Rel Error
df-BCXN-CC B-CN-CCN
2.63 10
7.75 10
3.06 10
2.63 10
[C]
CAN & AD
3.79 x 10-2
9.30 10
3.58 10
8.98 10
500
CAN
4.14 10
9.91 x 10
3.86 10
9.46 10
[N]
AD
4.05 10
9.80 x 10-1
x 10-2
9.66 x 10-1
JJ-BCXN-CAN
2.23 10
7.13 10
3.97
9.47 x 10
[C]
CAN & AD
3.65x100-
9.11 10
3.36x 10-2
8.72 x 10-1
1000
CAN
4.06x10-2
9.82 10
3.56
9.08 10
[N]
AD
4.16x
9.94 10
3.60 10
9.08 10
dj-BCXN-CAN
2.36x 10
7.34 10
7.14 10-
4.02 10
[C]
CAN & AD
3.80 10
9.31 10-1
3.32 10
8.65 x 10-1
2000
CAN
7.41 X 10
1.32 10
3.36
8.82 x 10
[N]
AD
3.84 x 10
9.53 10
3.55 10-2
9.02 x 10
JJ-BCXX-CAN
2.38 x 10
2.31 x 10
[C]
CAN & AD
3.23 x 10
8.54 x 10
4000
CAN
2.94 10
8.24 10
[N]
AD
3.50 10
8.97 10
From this experiment, notice that 4x increase in sampling density (from 50 x 50 to 100 100) can improve model performance, but this also requires the use of a larger batch size, which also add computational cost.
In addition, the node-centered CAN-PINN model has improved performance relative to the cell- centered CAN-PINN version. This further emphasizes hypothesis that accurate modelling of the boundary conditions is an important factor in obtaining good PINN performance. As this is square domain, the use of sampling point distribution that has perfect connectivity with the boundary of the domain naturally leads to improved performance.
Interestingly though, the inclusion of our proposed BCXN-loss on cell-centered model can im- prove model performance to achieve lower error than the properly aligned baseline PINN model. This does suggest that the use of this proposed method can afford flexibility in the choice of sam- pling strategies, whereas conventional sampling strategies may still require sampling that takes into account the ability to maintain connectivity with the boundaries.
A.10 EXPERIMENT WITH BCXN-PINN LOSS VARIANTS
As described in Section 4.2 and Section 4.3 there is no longer a need to explicitly evaluate the BC loss term for the Of-BCCNN-oss as the BCs are implicitly "infused" into the near-boundary domain points. We verify this point by comparing the results from a BCXN-PINN with the inclusion of BC loss term LeC into the df-BCXN-loss:
LPINN(JS (ENNNdJBBCXOS MUNNN(JJBCXXN..pp 7.DE APDE + ABcL
(16)
Based on our observation, the inclusion BC loss term doesn't seem to benefit the BCXN-PINN with df BCXN-loss. It is believed that the imposition of unnecessary BC loss term excessively com- plicates the training given the fact that actual BCs are already implicitly defined and utilized during the computation of df-BCXN-loss porate the loss term from sf-BCXN-loss into Eq. (16):
thereby slowing down convergence. We further incor-
LPINN BCXN) = >uPDE BCXN) + 7BCXNLBBXX + ABCL-C
(17)
to study the consequence of jointly using the df & sf approaches to modulate the BCXN-PINN training loss. Interestingly, the hybrid approach performs worse than the original df- and BC-BC
26

---

Under review as a conference paper at ICLR 2023
This may suggest that the BCXN-PINN training loss should only either df- sf- BC-BC loss, but not both together, potentially because of the confficting base assumptions underlying both derivations.
loss.
A.11 SENSITIVITY TO ÂBсXN FOR FORWARD MODELS
As mentioned in Section 4.3 the BCXN weight λBCXN in sf- Bf-BCXN-loss controls the extent of linear constraint being imposed at near boundary samples. We observe a noticeable improvement in solu- tion quality with the appropriate tuning of 7BCX for the BCXN-PINN in the 2D wavy channel flow QUCCXN = 0.5) and 3D bend tube channel flow (>BCXX = 5) test cases. The joint optimization results of ABCXN and Apry for the 2D wavy channel flow problem ther highlight that an overly stringent imposition of a linear constraint underlying the BCXN-PINN methodology can adversely affect convergence without accompanying increase in the emphasis (NpIv) placed on the divergence free condition.
presented in Fig. These results fur-
10
X-8CXII
sF-BCX),
<I-CCX,, Awexy
+
10-2
mie
10
10
Aow
Distribution of MSE between BCXN-PINN, when using different λpcxN and 7or value in its sf-BCXN-loss, and ground truth solutions for the wavy channel flow problem, Re 100.
Figure 16:
A.12
2D AND 3D INVERSE PROBLEM WITH MLP-ARCHITECTURE BCXN-PINN
We further demonstrate the proposed BCXN-PINNs on inverse problem, where the task is to infer the unknown physical quantities in the governing physies based partially available observation data. In particular, we seek to infer the unknown Re in the incompressible N-S equations (Eq. (.) well the solution over the problem domain, by training PINN model with respect to the following cons-cond physics informed loss functions:
LINV_PIN _PNN = LData + APDE + >BCLBC
(18a)
LINY LINV_PINN(dS JINN(dS- LDaaa >PDE LDDE(A--RXN
(18b)
LINY JINN(sf--CXX)) LData + LPDE + >BxxxLLBXX + LBC
(18c)
In addition the known BCs, these loss functions include additional data loss component:
I
Noobs ii [i;((
(19)
Hlass
to match the PINN output wi( against target i over Robs observations.
Inverse 2D semi-circle lid-driven cavity problem. We assume the availability of limited obser- vations (Nabs = 10 or 20) of velocity i7. The experiment comprises of 10 independent runs, where observations are randomly drawn (i.e. obtained from the simulation). Selected observation sets are displayed in Fig. 17p. Besides these observations, the PINN models are trained on 3,930 equidis- tantly spaced samples for evaluating the PDE loss and 300 BC samples. The inverse modelling results are visualized in Fig. l7p, comprising the distribution of ū MSE and inferred Re for both Nlbs = 10 or nlos = 20 scenarios. With nob = 20, our inverse BCXN -ENN model consistently infers an accurate Re which is always within 10% error from the ground truth (Re 1000) and achieves MSE below 5e-5. The efficacy of df-BBCNN-los drops slightly for the more challenging n = 10 scenario. However, for the sf-BCXN-loss, all inferred Re values are still within 15% of the ground
27

---

Under review as a conference paper at ICLR 2023
truth, and their solutions still achieve ii MSE below 5e-5. Fig. 17p also shows that baseline PINNs using ND- and AD-loss can infer an accurate Re in this inverse problem. This may suggest that, once PINN model is able to conrectly infer the unknown Re, it is much easier to learn the solution with these additional observations.
10
(a) Problem domain with data points
(h) Inverse modelling results
Figure 17: (a) BCXN-PINN is applied to simultancously infer the unknown Re coefficient (ground truth Re 1000) in the incompressible N-S equations and to solve for the solution based on limited observations (x-marked, Robv 10), for inverse 2D semi-circle lid driven cavity problem. (b) The distribution of 7 MSE and inferred Re- obtained from the inverse PINNs Nodv 10, 20. The green dashed lines and shaded areas indicate the ground truth Re and their 10% error bounds.
Inverse 3D bend tube channel flow problem. We assume limited observations of velocity i (Nobs = 50, 100) can be probed near the bending area Fig. 18p. Besides these observations, we train PINN models 19,786 equidistantly spaced PDE samples and 1,632 BC samples, for infer- ring the unknown Re (ground truth Re 100) and predicting the solution near the bending area. We only apply the non-slip BC at the wall boundary since the inlet and outlet conditions are assumed to be unknown. The experiment comprises of 10 independent runs, where observations are randomly drawn (i.e., obtained from the simulation). The inverse modelling results from different PINN mod- els are visualized in Fig. 18p. Again, we observe good performance from all tested PINN models, with their inferred always within 15% error from the ground truth. They can achieve below 2e-4 MSE with Nob: 50, and even better accuracy (i.e., <le-4) with Robs 100. Nonetheless, the experiment demonstrates the effectiveness of BCXN-PINN for solving complex inverse problems from limited random observations, with consistent performance across different sets of random data.
10
mae
Isde
10:4
31SCXX
NO
AD
NO
AD
Rgge50
Agc=100
(a) Problem domain
(b) Inverse modelling results
Figure 18: (a) The problem domain of the inverse 3D bend tube flow problem for inferring the unknown Re coefficient (ground truth Re = 100) in the incompressible N-S equations and solving for the solution based limited observations. (b) The distribution of ū MSE and inferred Re obtained from the inverse PINNs at nots = 50, 100. The green dashed lines and shaded areas indicate the ground truth Re and their 10% error bounds.
Hyper-parameter tuning. We use same MLP architecture, ND scheme, and batch size settings as summarized in Appendix(A.3 for the inverse problem test cases. Each batch contain at most 50 observations for computing the data loss. To ensure sufficiently accurate fit the observations,
28

---

Under review as a conference paper at ICLR 2023
reweight the PINN loss terms (i.e., the weight of data loss to PDE loss is 10 to 1) to give more priority to the data loss. The current inverse problem test cases require less training iterations to converge, compared to the pure physics-informn The max. iteration is set as 200,000 with an initial learning rate of 5e-4 for both test cases. We do not apply additional linear constraints to the near boundary samples if they are part of the observation set.
learning encountered in the forward problem.
A.13 VISUALIZATION OF CNN-ARCHITECTURE RESULTS
[d] gound truth
BCXN
BCAN
0.4
0.72
0.72
0.72
0.60
0.00
0.3
0.60
0.48
0.48
0.48
0.38
0.36
0.2
0.36
0.28
0.24
0.26
0.12
0.12
0.12
0.00
0.00
0.00
gound truth
BCXN
BCXN
1.12
1.12
1.12
0.96
0.96
0.96
0.00
0.48
0.48
0.40
0.32
0.32
0.32
0.2
0.4
0.6
0.8
0.2
0.4
0.6
0.8
0.2
0.4
0.6
0.0
(a) 2D semi-cirele lid-driven cavity flow, Re
1000
I
gound truth
wi BCXN
wio BCXV
0.96
0.84
0.84
-02
-0.2
-0.2
0.72
0.72
0.72
-04
0.60
-0.4
0.60
-0.4
0.60
006
006
5.6
0.24
0.24
0.24
0.12
0.12
0.00
-0.8
-0.6
-0.4
-0.2
-0.8
-0.6
-0.4
-0.2
-0.8
-0.6
-0.4
-02
gound truth
SCKN
1.12
1.12
1.12
0.96
0.96
0.96
0.50
0.80
0.80
0.64
0.64
0.64
0.48
0.48
0.48
-066
0.32
-066
0.32
0.32
0.16
0.16
0.16
-08
-0.8
-0.8
0.00
0.00
0.00
-0.8
-0.4
-0.2
-0.66
-0.8
-0.4
-0.16
-0.8
-04
-0.2
-0.16
(b) 2D lid-driven cavity flow, Re 1000
Figure 19: Comparison of velocity magnitude ||ع||| and pressure contour between the ground truth and PINN solutions (median MSE from 10 runs) for the (a) semi-circle lid-driven cavity flow problem, Re = 1000, and (b) lid-driven cavity flow problem, Re 1000, using CNN architecture.
29

---

Under review as a conference paper at ICLR 2023
A.14 ADDITIONAL FORWARD PROBLEM ON 2D AND NEUMANN TYPE BOUNDARY CONDITION
FLOW WITH DIRICHLET
As a demonstration of the generalizability the current proposed method to different types of boundary conditions based the derivation in Section A.1 further apply the described method to the solution of a 2D Taylor- Couut-e problem.
This problem describes the flow in fluid domain between 2 concentric cylinders. The inner cylinder is of radius R = 0.45 while the outer cylinder has radius R₂ = 0.96.
The analytical solution for the problem is known be:
((x,y) -K(( )2 y
(20)
v(x,y) K(( | -1xx
R2
(21)
12 R
p(x,y) K21
-
R
(22)
2
2,
off R
where angular velocity @ is 1, K
and
R
The numerical experiments performed with Re 1000 for four different pairs of boundary con- dition settings for the outer (inner) cylinder: i) Dirichlet (Dirichlet); ii) Dirichlet (Neumann); iii) Neumann (Dirichlet); and iv) Neumann (Neumann).
For all the boundary setting pairs, the training performed with a CNN-type PINN architecture on 56x56 voxel domain and Ar Ay 2/50. Additional PINN model details described in Table Contour plots of the results are presented in Fig. 20 and model errors are presented in Table
60
431
41
60
05
05
24
45
15
-35
81
Figure 20: Velocity contours of 2D Taylor- flow problem with different boundary condition settings.
Table Model Performance with Different [D]irichlet or [N]eumann Boundary Condition Settings
Outer Inner Cylinder BC Type
Metric
Variable
D/D
N/D
D/N
N/N
2.72 x 10
4.22 x 10
5.22
10
3.92 x 10
MSE
V
2.73 x 10
4.16 x 10
5.22 10
4.12 x 10
0.115
0.143
0.159
0.138
Relative Error
0.115
0.142
0.159
0.142
From the results, it can be seen that the BCXN-loss be flexibly applied to problems with any combination Dirichlet and Neumann type boundary conditions, although the majority of presented examples in this work feature Dirichlet boundary conditions.
30

---

Under review as a conference paper at ICLR 2023
A.15 ADDITIONAL FORWARD PROBLEM ON TRANSIENT 2D SEMI-CIRCLE LID-DRIVEN CAVITY FLOW, Re = 500
In addition to the forward modelling of steady-state 2D semi-circle lid-driven cavity flow, we further evaluate the improvement obtained from application of CNN-architecture BCXN-PINN to the mod- elling of a transient version of this problem. The problem geometry and set-up is consistent with the problem description
Section 5.2 However, the problem's 's initial condition corresponds to that of within the domain from
a fluid domain at rest (at = 0). The PINN model is then used to model the evolution of the flow 1 Additional model settings are provided in Table
The average errors for all time points between 0 and r 2.0 yield MSEs of 3.95e-4 and 9.66e- 3 with and without the inclusion of BCXN-loss respectively. This also corresponds to relative L2 errors of 129 and 0.681 respectively. Hence, the inclusion of BCXN-loss in this transient problem can improve MSE by more than order of magnitude.
Velocity contour plots of the PINN models trained with and without BCXN-loss are provided in Fig. 21 to illustrate the differences in model convergence.
M0 gound trum, t-0.5
BCKV
w/o BCXV
0.86
9.72
0.1
0.72
0.1-
0.72
0.50
0.00
0.2
0.38
0.2
035
0.2
0.56
024
0.6
9.00
0.2
0.4
0.00
6.6
0,00
gound thuth. L-1.0
0.66
0.54
0.6
0.66
0.72
04
0.02
0.72
0.00
0.2
0.2
0.12
21
0.
0.80
0.12
0.1
1.12
9.00
0.2
0.00
02
0.00
17]
quaand butti 1-2.0
BCWV
wib BCXI
Ac
0.66
0.00
0.3
0.3
0.00
0.3
0.36
035
0.5
0.1
6.
0.2
0.2 0.6
0.8
0.2
0.4
0.6
0.5
0.2
0.4
1.8
0.5
Figure 21: Velocity contours of the transient simulation at r= = and / = 2.0 for both the BCXN-PINN and ND-PINN implementation. The velocity contours on the left, middle and right correspond to the results from CFD, BCXN-PINN and ND-PINN respectively. The top, middle and bottom rows are for / = 0.5, = 1.0, and 2.0.
A.16 CNN-ARCHITECTURE BCXN-PINN AND BASELINE PINN PERFORMANCE ON A FORWARD PROBLEM WITH INCREASING COMPLEXITY
For easy reference, we summarize the average MSE and relative L2 error obtained for the 2D semi- circle lid-driven cavity experiments across 5 independent runs. These results indicate the single model errors obtained using the settings described in Table 4 Plots of the velocity MSEs are pro- vided in Fig. 22 We clearly see that the performance difference is most pronounced for more complex boundary value problems. As Re increases, the complexity of the developed flow typically requires more sophisticated algorithms for better capture of the non-linear effects.
31

---

Under review as a conference paper at ICLR 2023
10
w/o BCXN
BCXN
10
msse
10-2
10-3
10-5
200
400
600
800
1000
Figure 22:
Plot of velocity MSEs at different Re for both the df-BCXN- and ND- type CNN- architecture PINN.
Additional quantitative comparisons at select Reynolds numbers are presented Table 9 below for easier reference.
Table 9: Model Performance for the 2D Semi-circular Lid-Driven Cavity Problem
Reynolds Number
Metric
CNN-Model
50
100
200
400
1000
w/ BCXN
2.51 10
2.76 x 10
3.56 10 3
4.58 x 10
1.14 10
MSE
w/o BCXN
2.98 10-4
2.79 x 10 10-4
3.74 10
2.33 x 10
3.96 10 2
Relative
w/ BCXN
0.0258
0.0269
0.0305
0.0335
0.0492
Error
w/o BCXN
0.0823
0.0732
0.0780
0.211
0.855
Importantly, we note that although the current proposed BCXN-PINN method may nominally in- volve more computation, the impact on training time is under 5% in our experiments. The time taken for training a single if-BSCNN-loss PINN about 126-136 minutes while the time taken for training the equivalent ND-PINN with similar model size and optimization parameters is 121-137 minutes. This thus further emphasizes the utility of the current proposed method. A more in-depth description is provided in Appendix Section A.17
A.17 IMPACT OF BCXN-LOSS ON TRAINING TIME
We further compare the time taken for training a model with and without the BCXN-loss on the MI. M..--hcchteccture and CNN-architecture models as applied to the 2D semi-circle lid-driven cavity problem
1000. The average MSE and relative 1.2 error obtained are averaged across 10 independent runs, and the minimum and maximum wall-clock time are recorded. All experiments are run with the standard Tensorflow and Keras implementation on a 20-core workstation with an Intel Xeon Gold 6248 processor and Nvidia RTX 2080 Ti GPU.
Quantitative comparisons for the architettee models are presented in Table 10 All models for identical number of training iterations (300,000) with the same batch size (1,000) and had the same model size to facilitate a fair comparison.
Quantitative comparisons for the CNN-architecture model are also presented in Table 11 In addi- tional, both models were run for an identical number of training iterations (200,000) and had the same model size to facilitate fair comparison.
While not a comprchensive study, we note that the inclusion of the BCXN-loss across all the cases only slightly increases the amount of time required for model training. This the order of less
32

---

Under review as a conference paper at ICLR 2023
Table 10: Model Performance for M..-arrchitecture PINN
Model
MSE
Rel. Error
Time (min)
NI-BCXN-CAN
6.21 x 10
3.55 x 10
55-57
sf-BCXN-CAN
5.65 x 10
3.45 x 10-²
41 422
CAN
2.82 10-
7.62 x 10
40 46
df-BCXN-NN
1.61 x 10
5.95 x 10
23 -32-4
sf BCXN-ND
2.68 x 10
7.54 x 10
22 23
ND
3.91 10
8.56 x 10
20 22
AD (SIREN)
1.74 x 10
5.95 x 10
58
AD (Vanilla)
5.46 10
9.83 10
50
Table 11: Model Performance for a CNN CNN-architecture PINN
Model
MSE
Rel. Error
Time (min)
df-BCXN
1.14x 10
4.92 10
126 136
ND
3.96 x 10-
8.55 10-1
121 137
than for the CNN-type architecture and the ND-MLP-type model. Similarly, the sf-BCXN-CAN had negligible increase in training time relative to the baseline CAN-PINN model.
However, it is worth noting that while the time taken may have increased slightly, the MSE typically decreases more than 1 order of magnitude, suggesting that this trade-off between model training time and performance with the inclusion of BCXN-loss may still be preferred.
A.18 SUMMARY OF BCXN-PINN AND BASELINE PINN PERFORMANCE
For easy reference, we summarize the BCXN-PINN's model performance and compare them to other baseline methods previously published. The average MSE and relative L2 error across multiple independent runs are recorded in Table T2 and Table 13 for both the MlL-arccitecture and CNN- architecture models respectively.
33

---

Under review as a conference paper at ICLR 2023
Table 12: Model Performance Comparisons for MLP-architecture PINNs
Problem
Model
MSE
Rel. Error
df-BCXN- BAN
6.21 x 10
3.55 10
J-BCCN-CAN
5.65 x 10
3.45 x 10
CAN
2.82 x 10-2
7.62 x 10
2D semi-cirele lid-driven cavity, Re 1000
BC-BCXX
1.61 x 10
5,95 x 10
sf-BCXN-ND
2.68 x
7.54 10
ND
3.91 x 10
8.56 10
AD (SIREN)
1.74 10
5.95 10
AD (Vanilla)
5,46 10
9.83 x 10
df-BCXN-CAL
5.10 10-3
3.38 x 10
sf-BCXN-CAN
6.10 x 10
3.72 x 10
2D lid-driven cavity, Re 1000
CAN
4.23 10
9.77 10
AD (SIREN)
3,47 x 10
8.87 10
df- WI-BCNN-CRR CAN
1.34 10
7.28 10
BC-CCXNNAAN
9.00 10
4.09 10
2D wavy channel flow, Re 100
CAN
5.96 x 10
2.55 10
AD (SIREN)
3.52 x 10-2
7.19 x 10
dj-BCXN-ND
8.09 x 10
1.03 x 10
BCXN-ND
8.59 x 10
9.50 10
3D bend tube channel flow, Re 100
ND
2.63 10
6.89 10
AD (SIREN)
2.65 10
6.94 10
S--BCXN-CAN
4.65 10
8.10 10
CAN
3.32 10
1.92 10
2D flow past airfoil, Re 500
sf-BCXN-ND
5.03 x 10
6.93 x 10
ND
3.54 x 10
1.84 x 10
AD (SIREN)
1.18 x 10-3
3.17 10
Implementation for CAN-PINN models based on previous work (Chiu et al.
20022..
(1I) Implementation for SIREN-PINN models based on previous work (Sitzmann et al. 2020
Wang et al. 202Tb; Wong et al. 2022).
Implementation for vanilla PINN models based on previous work (Raissi al. 2019)
Table 13: Model Performance Comparisons for CNN-architecture PINNs
Problem
Model
MSE
Rel. Error
df-BCXN
1.14 10
4.92 x 10
2D semi-circle lid-driven cavity, Re 1000
ND
3.96 10
8.55 x 10
df-BCXN
2.52 10
4.46 x 10
2D lid-driven cavity, Re = 1000
ND
2.38 x 10
1.00 x 10
df-BCXN
3.95 x 10
1.29 x 10
Transient 2D semi-circle lid-driven cavity, Re 500
ND
9.66 10
6.81 x 10
34


Reviews:
[{"review_id": "ltJv7r8OGsa", "reviewer": "Reviewer_Dt9j", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposed a new Boundary Connectivity (BCXN) loss function for PINNs for solving PDEs with complex geometry. BCXN uses a linear interpolation to compute the points outside the domain to address the issue that the derivative stencil points are outside the domain, and then the estimated external points can be enforced directly in the loss or via a penalty loss. Numerical experiments showed that BCXN achieves better accuracy with less training points.", "summary_of_the_review": "The proposed method doesn’t work as well as they claimed. It has many limitations. The numerical experiments are not convincing enough.", "strength_and_weaknesses": "Strengths\n- The proposed method estimates the external points and thus avoid the issue of stencil points falling outside the geomery.\n\nWeaknesses\nComments on the methods:\n- The method only works for Dirichlet BC, but not other BCs, such as Neumann BC, Robin BC, etc.\n- The method uses a linear interpolation to compute external points, which will limit the accuracy.\n- The name “Fast-PINN” is not proper. The focus of the proposed method is complex geometry. It is not a fast version of PINN for any PDE problems with any BCs.\n\nComments on the correctness of the numerical results:\n\n- 1D convection-diffusion PDE has been solved very accuracy in many PINN papers, so it is questionable if the AD and ND results are correct. No details about the setup of Fig 2 are provided.\n- In the 2D cavity flow, it is regular geometry, and all the points can be chosen inside the domain. In this case, the proposed method becomes a standard PINN, so the results of ND and AD don’t seem correct. Also, there are PINN papers with 2D cavity flow problem and PINN works well. The only explanation is that the authors make ND and AD become bad.\n\nOther comments on the results:\n\n- The paper uses MSE error, but other metrics should be provided, such as L2 relative error.\n- The paper only considered steady-state problems, but no time dependent problems are tested.\n\nComments on the comparisons:\n\n- It is unfair to compare AD/ND with BCXN as BCXN is more expensive. The authors should increase the training points of AD/ND such that the computational cost is similar, and then compare the accuracy.\n- There are many other methods, e.g., adaptive sampling methods, as were discussed in Section 2, but no comparison is provided.\n\nOther comments:\n\n- The section 3 is “THEORY”, but there is no theory in the section.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "H--7UYQXiC8", "reviewer": "Reviewer_DAxW", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a novel design of physical loss to find a trade-off between sample density and solution accuracy of PDEs. The formulation of the proposed loss tackles the challenges of approximating the boundary layer by modeling and adding a term in the loss function that takes into consideration the highly complex fluctuations of the gradients near the boundary layer. The work is inspired by a method in numerical computing.\n", "summary_of_the_review": "The contribution of the proposed work will help to make progress in ML&Physics and build efficient PINNs. \nI can increase my score depending on the responses.", "strength_and_weaknesses": "In this work, the proposed method makes the training of PINNs feasible and sufficiently accurate at the boundary layer in a sparse sample regime where existing works fail (or require a highly dense grid at the boundary layer which is not easy to construct and/or is intractable in practice). Moreover, the method shows an extra gain in terms of computational complexity and achieves accurate solutions with drastically fewer iterations in a scarce data regime. \nAt the (near) boundary layer, the fluctuations are highly complex and the geometries are irregular. The proposed design of fast-PINNs loss allows in some way to cope with that. It is challenging because, given the local complexity near the boundary layer, sparse sampling leads to overfitting even though PDE constraint is applied to all the samples.\n\nHowever, l have some considerations:\n\n1/ It is important to underline that the success of PINNs relies on the successful propagation of information from initial and boundary condition points to interior points. This can be observed if we reduce the PDE solving to a local part of the domain. Do you think that your model achieves this successful propagation? how can you quantify that empirically? \n\n2/ In existing PINNs works, the propagation failures are tied to highly imbalanced PDE residual fields.  In other terms, residuals with high values could be observed over very narrow regions. Does the proposed work mitigate/circumvents this failure?  if yes, how does the method succeed to make the balance of PDE residual fields? \n\n3/ Given your method and the impact of a good choice of sampling strategy to make PINNs successful. I am wondering if the breakthrough work on PINNs is more on finding an appropriate sampling strategy near the boundary layer. If the latter is found, standard PINNs loss will work. What do you think?\nA kind of sampling that focuses on points near the boundary layer with high PDE residual at every iteration. In practice, one could imagine sampling more of the points with high residuals. \n\n4/ What do you think of adding the diffusion term in the loss function?\n\n5/  Why did you set manually the different weights of each loss term rather than learning them? \n\n6/ You mentioned that “ It is also very challenging to locally refine the sampling density for PINNs with CNN-architecture”. Recently, lots of efforts have been made to generalize learning to non-Euclidean data, known as Geometric Deep Learning. The latter is suitable to operate on meshes and irregular geometries, namely Graph Neural Networks (GNNs), PointNet.  Are there any reasons for not using GNNs ? Several works on ML&physics rely on these architectures.\n\n7/ Lack of quantitative results. This work could be improved by adding a table of baseline w.r.t existing works on PINNs that proposes different variants of physical loss and w.r.t. different. sampling strategies\n\n8/ It would be interesting to see what the proposed method could bring if it is combined with a given domain decomposition method.\n\n9/ This work could be extended to more complex geometries such as the ones available in the different datasets proposed in the work entitled \"Learning to simulate complex physics with graph networks\". It allows to asses the generalization capabilities of the proposed method. \n\n10/ Do you think that the proposed method could generalize well to high Reynolds (> 10^6)\n\n11/ The figures could be updated with log scales", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "98DJ6AFItQ4", "reviewer": "Reviewer_3A2P", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work proposed a new Boundary Connectivity (BCXN) loss function to improve the accuracy and convergence speed of PINN. Specifically, it developed two variants of BCXN loss, namely: 1) a soft forcing method which imposes a linear approximation constraint via an additional loss term, and ii) a direct forcing approach which strongly enforces a linear constraint during the evaluation of PDE loss at nearboundary samples. Experimental results on multiple datasets demonstrated that the proposed method can significantly improve accuracy while reducing training iterations. ", "summary_of_the_review": "The contributions of this work lie in adding Boundary Connectivity (BCXN) in the loss function, which can improve accuracy and reduce training iterations. In addition, extensive experiments are conducted to validate the effectiveness of the BCXN loss function. The idea is a bit novel and interesting,", "strength_and_weaknesses": "* Strength\n1. The idea of adding Boundary Connectivity (BCXN) is interesting and novel\n2. The proposed loss function can be inserted into any deep neural network architecture\n3. Extensive experimental results show that the introduced loss function can significantly improve accuracy while reducing training iterations.\n\n* Limitations\n1. Compared to existing work on PINN. There exist many studies on PINN, so it would be better to discuss related work and compare the proposed method with some latest baselines, such as [Zeng et al 2022]\n2. There is a typo about AQ = AP+ AQ in the first line on Page 6. \n3. Question:\n    I am curious if the proposed method can solve the prey and predator system which is not stable?\n\nReferences:\n[Zeng et al 2022] Competitive Physics Informed Networks, https://arxiv.org/pdf/2204.11144.pdf", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "7S-stzfRHvn", "reviewer": "Reviewer_KS1E", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper addresses the issue of PINNs overfitting at the boundary of the domain. Since numerical differentiation based methods require points outside the boundary of the domain to work well, the authors propose a simple and efficient solution which constraints the value of the function outside the boundary (external stencil points) in terms of “mirror points” that lay inside the domain. \n\nThis loss can be directly enforced (i.e., the equality can imposed directly for the stencil points outside the boundary) or can be enforced softly by enforcing it as a loss function. \n\nThe authors empirically compare the performance of Fast-PINN with normal PINNs (using numerical and autodiff loss) under 4 fluid dynamics based cases, and show Fash PINNs converge faster and have a lower mse loss.", "summary_of_the_review": "The authors provide a simple loss function take care of the points that lie outside the domain for numerical differentiation type loss for PDE solvers. The solution provided by the authors is simple to implement and shows improvement over normal PINN methods.", "strength_and_weaknesses": "I really enjoyed reading the paper. The loss function mentioned is easy to implement and gets good results when compared to baseline PINNs. The empirical section is clear and easy to read and the authors do a great job at delineating the type overall setup of the PDEs they consider for their experimental section. The PDEs consider mainly differ in the complexity of the boundaries of the domains.\n\nThe improvement introduced by fast-PINNs seems to be quite significant compared to the normal PINNs (both in terms of the rate of convergence and final loss). The authors also point out certain inductive biases that are inherent in CNN based architectures, which is quite intriguing. \n\n\nOne small question for the authors: Is there a reason that the authors on solving focussed fluid dynamics with PINNs, do the gains follow for other PDEs as well (like reaction diffusion etc?)", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject