Original: F AST -PINN FOR C OMPLEX G EOMETRY : S OLVING PDE S WITH B OUNDARY C ONNECTIVITY L OSS
Revised: BCXN-PINN FOR C OMPLEX G EOMETRY : S OLVING PDE S WITH B OUNDARY C ONNECTIVITY L OSS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We present a novel loss formulation for efficient learning of complex dynamics from governing physics, typically described by partial differential equations (PDEs), using physics-informed neural networks (PINNs).
Revised: We present a novel loss formulation for efficient learning of complex dynamics from governing physics, typically described by partial differential equations (PDEs), using physics-informed neural networks (PINNs).

Original: In our experiments, existing versions of PINNs are seen to learn poorly in many problems, especially for complex geometries, as it becomes increasingly difficult to establish appropriate sampling strategy at the near boundary region.
Revised: In our experiments, existing versions of PINNs are seen to learn poorly in many problems, especially for complex geometries, as it becomes increasingly difficult to establish appropriate sampling strategy at the near boundary region.

Original: Overly dense sampling can adversely impede training convergence if the local gradient behaviors are too complex to be adequately modelled by PINNs.
Revised: Overly dense sampling can adversely impede training convergence if the local gradient behaviors are too complex to be adequately modelled by PINNs.

Original: On the other hand, if the samples are too sparse, PINNs may over-fit the near boundary region, leading to incorrect solution.
Revised: On the other hand, if the samples are too sparse, PINNs may over-fit the near boundary region, leading to incorrect solution.

Original: To prevent such issues, we propose a new Boundary Connectivity (BCXN) loss function which provides local structure approximation at the boundary.
Revised: To prevent such issues, we propose a new Boundary Connectivity (BCXN) loss function which provides local structure approximation at the boundary.

Original: Our
Revised: Our

Original: BCXN-loss can implicitly or explicitly impose such approximations during training, thus facilitating fast physics-informed learning across entire problem domains with order of magnitude fewer training samples.
Revised: BCXN-loss can implicitly or explicitly impose such approximations during training, thus facilitating fast physics-informed learning across entire problem domains with order of magnitude fewer training samples.

Original: This method shows a few orders of magnitude smaller errors than existing methods in terms of the standard L2norm metric, while using dramatically fewer training samples and iterations.
Revised: This method shows a few orders of magnitude smaller errors than existing methods in terms of the standard L2-norm metric, while using dramatically fewer training samples and iterations.

Original: Our proposed Fast-PINN method does not pose any requirement on the differentiable property of the networks, and we demonstrate its benefits and ease of implementation on both multi-layer perceptron and convolutional neural network versions as commonly used in current physics-informed neural network literature.
Revised: Our proposed BCXN-PINN method does not pose any requirement on the differentiable property of the networks, and we demonstrate its benefits and ease of implementation on both multi-layer perceptron and convolutional neural network versions as commonly used in current physics-informed neural network literature.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Physics-informed neural networks (PINNs) have emerged as a promising method for learning the solution of dynamical system from the governing physics (Raissi et al., 2019).
Revised: Physics-informed neural networks (PINNs) have emerged as a promising method for learning the solution of dynamical system from the governing physics (Raissi et al., 2019).

Original: PINNs have recently been studied for a wide range of physical phenomena and applications across science and engineering domains — electromagnetic, fluid dynamics, heat transfer, etc (Karniadakis et al., 2021; Cuomo et al., 2022).
Revised: PINNs have recently been studied for a wide range of physical phenomena and applications across science and engineering domains — electromagnetic, fluid dynamics, heat transfer, etc (Karniadakis et al., 2021; Cuomo et al., 2022).

Original: The distinctive feature of PINNs is the use of governing physics law, typically in the form of partial differential equations (PDEs), as the learning objective.
Revised: The distinctive feature of PINNs is the use of governing physics law, typically in the form of partial differential equations (PDEs), as the learning objective.

Original: This physics-informed learning constrains the PINN from violating the underlying physics at all training points sampled from the problem domain.
Revised: This physics-informed learning constrains the PINN from violating the underlying physics at all training points sampled from the problem domain.

Original: Existing PINNs evaluate the PDE constraints in their training loss by either automatic differentiation (AD) or numerical differentiation (ND)-type method (Wandel et al., 2020).
Revised: Existing PINNs evaluate the PDE constraints in their training loss by either automatic differentiation (AD) or numerical differentiation (ND)-type method (Wandel et al., 2020).

Original: While both methods have their pros and cons, ND-type PDE loss can be flexibly implemented across many different neural network  architectures, including both multi-layer perceptrons (MLPs) and convolutional neural networks (CNNs), because they do not require the neural network to retain differentiability, unlike AD.
Revised: While both methods have their pros and cons, ND-type PDE loss can be flexibly implemented across many different neural network (NN) architectures, including both multi-layer perceptrons (MLPs) and convolutional neural networks (CNNs), because they do not require the NN to retain differentiability, unlike AD.

Original: Recent studies (Gao et al., 2021; Fang, 2021; Chiu et al., 2022) have also suggested that ND-loss can more robustly and efficiently produce accurate solutions with fewer training samples, whereas conventional AD-loss are prone to failure during training .
Revised: Recent studies (Gao et al., 2021; Fang, 2021; Chiu et al., 2022) have also suggested that ND-loss can more robustly and efficiently produce accurate solutions with fewer training samples, whereas conventional AD-loss are prone to failure during training .

Original: This is because ND-type methods approximate high order derivatives using PINN output from neighbouring samples , hence, they can effectively connect sparse samples into piecewise regions via these local approximations, thereby facilitating fast physics-informed learning across the entireproblem domain with less sensitivity to the density of samples.
Revised: This is because ND-type methods approximate high order derivatives using PINN output from neighbouring samples , hence, they can effectively connect sparse samples into piecewise regions via these local approximations, thereby facilitating fast physics-informed learning across the entire domain with less dense samples.

Original: When dealing with irregular geometries, it becomes increasingly difficult for existing PINNs to perfectly connect training samples in the domain’s interior to the boundary.
Revised: When dealing with irregular geometries, it becomes increasingly difficult for existing PINNs to perfectly connect training samples in the domain’s interior to the boundary.

Original: We show that failing to do so can cause undesirable training failure as the PINN starts to over-fit at the near boundary region.
Revised: Failing to do so can cause undesirable training failure as the PINN starts to over-fit at the near boundary region.

Original: Since many PDEs of practical interest are boundary-value problems, it is desirable to have the PINNs model the correct boundary behaviors as quickly as possible.
Revised: Since many PDEs of practical interest are boundary-value problems, it is desirable to have the PINNs model the correct boundary behaviors.

Original: While adding dense samples to better refine the piecewise local regions near the boundary mayhelp to improvethe accuracy, the extent to which sampling needs to be increased is empirical, and a denser sampling strategy may adversely impede training convergence.
Revised: While adding dense samples to better refine the piecewise local regions near the boundary may improve accuracy, the extent to which sampling needs to be increased is empirical, and a denser sampling strategy may adversely impede training convergence.

Original: Hence, we propose anew loss function formulation in this work that helps provide an approximation to the local gradient behaviour at the boundary, thereby restoringthe connectivity between domain boundary and near boundary interior samples.
Revised: Hence, we propose a loss function formulation in this work that helps provide an approximation to the local gradient behaviour at the boundary, thereby restoring connectivity between domain boundary and near boundary interior samples.

Original: This new boundary connectivity ( BCXN )-loss function is key to a novel class of Fast-PINN method which can more efficiently learn the solution to 
Revised: This new boundary connectivity ( BCXN )-loss function is key to a novel class of BCXN-PINN method which can more efficiently learn the solution to PDEs with fewer training samples, regardless of domain geometry; see example in Fig.

Original: PDEs with fewer training samples, regardless of geometry of the domain; see example in Fig.
Revised: In addition, this method can be jointly implemented with other PINN advances such as in loss balancing, domain decomposition, adaptive sampling and other improved optimization methods (Zeng et al., 2022).

Original: In the rest of this work, we present two versions of this BCXN-loss : i) a soft forcing approach which imposes a linear approximation constraint via an additional loss term, and ii) a direct forcing approach which strongly enforces a linear constraint during the evaluation of PDE loss at nearboundary samples.
Revised: In the rest of this work, we present two versions of this BCXN-loss : i) a soft forcing approach which imposes a linear approximation constraint via an additional loss term, and ii) a direct forcing approach which strongly enforces a linear constraint during the evaluation of PDE loss at nearboundary samples.

Original: With the latter approach, there is no longer a need to explicitly evaluate the boundary samples as the exact BCs have been implicitly “infused” into the near-boundary samples.
Revised: With the latter approach, there is no longer a need to explicitly evaluate the boundary samples as the exact BCs have been implicitly “infused” into the near-boundary samples.

Original: Moreover, the direct forcing BCXN-loss can be beneficial to CNN-type architectures which utilize a structured grid and lack the ability to model exact domain boundaries for irregular geometries.
Revised: Moreover, the direct forcing BCXN-loss can be beneficial to CNN-type architectures which utilize a structured grid and lack the ability to model exact domain boundaries for irregular geometries.

Original: We present comprehensive experiments to demonstrate i) the flexible implementation of Fast-PINN method for both MLP and CNN architectures; and ii) the effectiveness of Fast-PINN for learning multiple complex fluid dynamical systems, spanning forward, inverse and meta-model problems in two-dimensions (2D) and three-dimensions (3D).
Revised: We present comprehensive experiments to demonstrate i) the flexible implementation of BCXNPINN method for both MLP and CNN architectures; and ii) the effectiveness of BCXN-PINN for learning multiple complex fluid dynamical systems, spanning forward, inverse and meta-model problems in two-dimensions (2D) and three-dimensions (3D).

Original: Compared to conventional PINNs with the ADand ND-loss , our Fast-PINNs with BCXN-loss are shown to be capable of tackling challenging PDE problems while using fewer training samples, hence expanding the exciting potential of PINNs for learning complex dynamical evolutions encountered in the real-world.
Revised: Compared to conventional PINNs with the AD- and ND-loss , our BCXN-PINNs with BCXN-loss are shown to be capable of tackling challenging PDE problems while using fewer training samples, hence expanding the exciting potential of PINNs for learning complex dynamical evolutions encountered in the real-world.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Efficient sampling in PINNs.
Revised: Efficient sampling in PINNs.

Original: The theoretical limit of physics-informed loss learning in relation to training samples has been provided by prior studies (Lu et al., 2021c; Mishra & Molinaro, 2022).
Revised: The theoretical limit of physics-informed loss learning in relation to training samples has been provided by prior studies (Lu et al., 2021c; Mishra & Molinaro, 2022).

Original: With the goal of improving the PINN training speed for practical applications (Markidis, 2021), several studies have focused on efficient sampling strategies such as importance sampling, adaptive sampling, and sequential sampling to reduce the amount of training samples being required during
Revised: With the goal of improving the PINN training speed for practical applications (Markidis, 2021), several studies have focused on efficient sampling strategies such as importance sampling, adaptive sampling, and sequential sampling to reduce the amount of training samples being required during

Original: PINN trainings (Anitescu et al., 2019; McClenny & Braga-Neto, 2020; Wight & Zhao, 2020; Nabian et al., 2021;
Revised: PINN trainings (Anitescu et al., 2019; McClenny & Braga-Neto, 2020; Wight & Zhao, 2020; Nabian et al., 2021;

Original: Lu et al., 2021a; Lye et al., 2021;
Revised: Lu et al., 2021a; Lye et al., 2021;

Original: Daw et al., 2022; Mattey & Ghosh, 2022).
Revised: Daw et al., 2022; Mattey & Ghosh, 2022; Wu et al., 2023).

Original: Domain decomposition and parallelization strategies have also been explored to speed up the training (Jagtap et al., 2020; Jagtap & Karniadakis, 2021; Shukla et al., 2021;
Revised: Domain decomposition and parallelization strategies have also been explored to speed up the training (Jagtap et al., 2020; Jagtap & Karniadakis, 2021; Shukla et al., 2021;

Original: Dong & Li, 2021; Li et al., 2019; Kharazmi et al., 2021).
Revised: Dong & Li, 2021; Li et al., 2019; Kharazmi et al., 2021).

Original: Our method differs from these works in that we make physics-informed learning more robust in the sparse sample regime via a newly-proposed BCXN-loss .
Revised: Our method differs from these works in that we make physicsinformed learning more robust in the sparse sample regime via a newly-proposed BCXN-loss .

Original: CNN architecture and numerical differentiation (ND)-type loss for PINNs.
Revised: CNN architecture and numerical differentiation (ND)-type loss for PINNs.

Original: CNN-based formulation allows us to design and train larger, more powerful networks, hence it has potential to be scalable for more complex, large-scale PDE problem (Wandel et al., 2020;
Revised: CNN-based formulation allows us to design and train larger, more powerful networks, hence it has potential to be scalable for more complex, large-scale PDE problem (Wandel et al., 2020;

Original: Gao et al., 2021;
Revised: Gao et al., 2021;

Original: Wandel et al., 2021; Ranade et al., 2021; Wandel et al., 2022; Ren et al., 2022).
Revised: Wandel et al., 2021; Ranade et al., 2021; Wandel et al., 2022; Ren et al., 2022).

Original: However, cumbersome coordinate transformations may need to be performed to better handle the irregular domain.
Revised: However, cumbersome coordinate transformations may need to be performed to better handle the irregular domain.

Original: CNNarchitecture PINNs usually utilize ND methods for computing the PDE loss on their input grid, which is much cheaper to compute as compared to AD-based loss (Gao et al., 2021).
Revised: CNNarchitecture PINNs usually utilize ND methods for computing the PDE loss on their input grid, which is much cheaper to compute as compared to AD-based loss (Gao et al., 2021).

Original: Hybrid frameworks that couple both AD- and ND-type loss have also been proposed for MLP architecture PINNs to unify the advantages of both methods (Jagtap et al., 2020;
Revised: Hybrid frameworks that couple both AD- and ND-type loss have also been proposed for MLP architecture PINNs to unify the advantages of both methods (Jagtap et al., 2020;

Original: Kharazmi et al., 2021; Mitusch et al., 2021; Fang, 2021; Chiu et al., 2022).
Revised: Kharazmi et al., 2021; Mitusch et al., 2021; Fang, 2021; Chiu et al., 2022).

Original: Our method further augments the efficiency and applicability of ND-type loss on irregular geometries.
Revised: Our method further augments the efficiency and applicability of ND-type loss on irregular geometries.

Original: Enforcing BC constraint in PINN loss.
Revised: Enforcing BC constraint in PINN loss.

Original: It is important for the PINNs to prioritize learning the correct boundary behaviors (Shin et al., 2020).
Revised: It is important for the PINNs to prioritize learning the correct boundary behaviors (Shin et al., 2020; Wang et al., 2022).

Original: The BCs are usually enforced into the PINN loss as a soft constraint using penalty method.
Revised: The BCs are usually enforced into the PINN loss as a soft constraint using penalty method.

Original: Variousweights balancing strategies have been explored in this context to dynamically calibrate the relative important between PDE and BC constraints during
Revised: Various strategies have been explored in this context to dynamically calibrate the relative important between PDE and BC constraints during

Original: PINN training (Elhamod et al., 2020; Bischof & Kraus, 2021; Jin et al., 2021;
Revised: PINN training (Elhamod et al., 2020; Bischof & Kraus, 2021; Jin et al., 2021;

Original: Thanasutives et al., 2021; Maddu et al., 2022; de Wolff et al., 2022; van der Meer et al., 2022; Xiang et al., 2022; Huang et al.).
Revised: Thanasutives et al., 2021; Maddu et al., 2022; de Wolff et al., 2022; van der Meer et al., 2022; Xiang et al., 2022; Huang et al. ; Wang et al., 2021a).

Original: There are also other approaches to bypass the loss balancing issue.
Revised: There are also other approaches to bypass the loss balancing issue.

Original: For example, one can either devise an ansatz function such that the BCs are exactly satisfied by construction or implicitly formulate the BC constraints into the PDE loss (Lagaris et al., 1998; 2000; McFall & Mahan, 2009; Berg & Nystr ¨ om, 2018; Nabian & Meidani, 2019; Karumuri et al., 2020; Wang & Zhang, 2020), leaving only single loss term from PDE residuals to be optimized.
Revised: For example, one can either devise an ansatz function such that the BCs are exactly satisfied by construction or implicitly formulate the BC constraints into the PDE loss (Lagaris et al., 1998; 2000; McFall & Mahan, 2009; Berg & Nystr ¨ om, 2018; Nabian & Meidani, 2019; Karumuri et al., 2020; Wang & Zhang, 2020), leaving only single loss term from PDE residuals to be optimized.

Original: Another example is the use of augmented Lagrangian method to impose the BC constraints into PINN loss (Lu et al., 2021b).
Revised: Another example is the use of augmented Lagrangian method to impose the BC constraints into PINN loss (Lu et al., 2021b).

Original: Our method implicitly incorporates BCs into the PDE loss via a different strategy.
Revised: Our method implicitly incorporates BCs into the PDE loss via a different strategy.

Original:  T HEORY 3. G OVERNING PHYSICS - INCOMPRESSIBLE N AVIER -S TOKES (N-S) EQUATIONS
Revised:  P RELIMINARY 3. G OVERNING PHYSICS - INCOMPRESSIBLE N AVIER -S TOKES (N-S) EQUATIONS

Original: The present study focuses on learning fluid dynamics with PINNs.
Revised: The present study focuses on learning fluid dynamics with PINNs.

Original: Fundamentally, PINNs have been shown to be applicable to many different physical systems.
Revised: Fundamentally, PINNs have been shown to be applicable to many different physical systems.

Original: We consider fluid problems where the governing physics are the steady-state, incompressible N-S equations derived from the conservation of mass and momentum:[Equation] 1b
Revised: We consider fluid problems where the governing physics are the steady-state, incompressible N-S equations derived from the conservation of mass and momentum:

Original: [Equation]
Revised: [Equation]

Original: In the above PDEs, the primitive variables ⃗ u and p are velocity vector and pressure while Reynolds number ( Re ) represents the ratio of inertial to viscous forces.
Revised: In the above PDEs, the primitive variables ⃗ u and p are velocity vector and pressure while Reynolds number ( Re ) represents the ratio of inertial to viscous forces.

Original: ⃗ u consists of 2 components ( u , v ) for a 2D case and 3 components ( u , v , w ) for a 3D case.
Revised: ⃗ u consists of 2 components ( u , v ) for a 2D case and 3 components ( u , v , w ) for a 3D case.

Original: While Cartesian coordinates are used in this work, this formulation is extendable to other coordinate systems.
Revised: While Cartesian coordinates are used in this work, this formulation is extendable to other coordinate systems.

Original: 3. MLP- AND CNN- ARCHITECTURE PINN S
Revised: 3. MLP- AND CNN- ARCHITECTURE PINN S

Original: We use a fully connected DNN architecture (e.g., MLP, CNN) to represent the solution of the dynamical process U =
Revised: We use a fully connected DNN architecture (e.g., MLP, CNN) to represent the solution of the dynamical process U =

Original: [ ⃗ u , p ] T .
Revised: [ ⃗ u , p ] T .

Original: For MLP architecture, the input ⃗ x is a point coordinate in 2D or 3D spatial domain.
Revised: For MLP architecture, the input ⃗ x is a point coordinate in 2D or 3D spatial domain.

Original: For CNN architecture, the input X is a tensor with a fixed shape, comprising the entire (discretized) spatial domain.
Revised: For CNN architecture, the input X is a tensor with a fixed shape, comprising the entire (discretized) spatial domain.

Original: The accuracy of the PINN outputs U ( ⃗ x ; w ) given input ⃗ x is determined by the network parameters w , which are optimized w.r.t.
Revised: The accuracy of the PINN outputs U ( ⃗ x ; w ) given input ⃗ x is determined by the network parameters w , which are optimized w.r.t.

Original: the PINN loss function during training.
Revised: the PINN loss function during training.

Original: The PINN loss function is defined as the composition of a PDE loss component ( L PDE ) and a BC loss component ( L BC ) :
Revised: The PINN loss function is defined as the composition of a PDE loss component ( L PDE ) and a BC loss component ( L BC ) :

Original: [Equation]
Revised: [Equation]

Original: The PDE loss penalizes deviation from governing N-S equations for the PINN output U ( ⃗ x ; w ) over the fluid domain ⃗ x ∈ Ω , whereas the BC loss penalizes deviation from the desired Dirichlet boundary condition ⃗ u BC ( ⃗ x ) which constrains the velocity field at the domain boundary ⃗ x ∈ ∂Ω .
Revised: The PDE loss penalizes deviation from governing N-S equations for the PINN output U ( ⃗ x ; w ) over the fluid domain ⃗ x ∈ Ω , whereas the BC loss penalizes deviation from the desired Dirichlet boundary condition ⃗ u BC ( ⃗ x ) which constrains the velocity field at the domain boundary ⃗ x ∈ ∂Ω .

Original: The relative weights λ PDE and λ BC in Eq.
Revised: The relative weights λ PDE and λ BC in Eq.

Original: (2) a control the trade-off between different loss components.
Revised: (2) a control the trade-off between different loss components.

Original: The PDE and BC loss components are defined over a continuous domain, but for practical reasons, we evaluate the discretized PINN loss from a finite set of n samples D = { ( ⃗ x i ) } ni = 1 during training.
Revised: The PDE and BC loss components are defined over a continuous domain, but for practical reasons, we evaluate the discretized PINN loss from a finite set of n samples D = { ( ⃗ x i ) } ni = 1 during training.

Original: The CNN-architecture PINNs naturally acquire an input with equidistantly spaced grid where all the samples are coming from.
Revised: The CNN-architecture PINNs naturally acquire an input with equidistantly spaced grid where all the samples are coming from.

Original: For the MLP-architecture PINNs, it is also convenient to design training samples based on equidistantly spaced grid, then m ( < n ) samples can be randomly drawn to compute the PINN loss during each stochastic gradient descent (SGD) mini-batch training iteration.
Revised: For the MLP-architecture PINNs, it is also convenient to design training samples based on equidistantly spaced grid, then m ( < n ) samples can be randomly drawn to compute the PINN loss during each stochastic gradient descent (SGD) mini-batch training iteration.

Original: Although it is ideal for a PINN to satisfy the PDE constraint on a very dense set of samples, i.e., ∆ ⃗ x → 0 where ∆ ⃗ x is the distance between two adjacent sample points, training such PINN may not be computationally feasible even with the state-of-the-art SGD variants.
Revised: Although it is ideal for a PINN to satisfy the PDE constraint on a very dense set of samples, i.e., ∆ ⃗ x → 0 where ∆ ⃗ x is the distance between two adjacent sample points, training such PINN may not be computationally feasible even with the state-of-the-art SGD variants.

Original: For a more complex physics problem, the training on dense samples is naturally more computationally expensive, and is also more likely to become unsuccessful due to adverse impact by some local region with very complex gradient behaviours, e.g., abnormally high gradient (Fuks & Tchelepi, 2020; Michoski et al., 2020;
Revised: For a more complex physics problem, the training on dense samples is naturally more computationally expensive, and is also more likely to become unsuccessful due to adverse impact by some local region with very complex gradient behaviours, e.g., abnormally high gradient (Fuks & Tchelepi, 2020; Michoski et al., 2020;

Original: Ramabathiran & Ramachandran, 2021; Ji et al., 2021; Lucor et al., 2021; Huang et al.).
Revised: Ramabathiran & Ramachandran, 2021; Ji et al., 2021; Lucor et al., 2021; Huang et al.).

Original: Given the same problem, training PINNs on fewer samples means fewer restrictions (the PDE constraint that needs to be satisfied) and an increased likelihood of avoiding problematic gradient points, thereby accelerating training.
Revised: Given the same problem, training PINNs on fewer samples means fewer restrictions (the PDE constraint that needs to be satisfied) and an increased likelihood of avoiding problematic gradient points, thereby accelerating training.

Original: The relaxation of PDE constraint via reducing the sample density makes learning physically meaningful solution feasible , although it may introduce some approximation error to the solution.
Revised: The relaxation of PDE constraint via reducing the sample density makes learning physically meaningful solution feasible , although it may introduce some approximation error to the solution.

Original: Note that there is still an open question as to how to strike the best balance between sample density and solution accuracy.
Revised: Note that there is still an open question as to how to strike the best balance between sample density and solution accuracy.

Original: PINN methods that can consistently produce more accurate solutions with fewer training samples are nevertheless hugely advantageous for learning complex dynamical evolutions encountered in the real-world.
Revised: PINN methods that can consistently produce more accurate solutions with fewer training samples are nevertheless hugely advantageous for learning complex dynamical evolutions encountered in the real-world.

Original: 3. L EARNING PINN S ON FEWER SAMPLES
Revised: 3. L EARNING PINN S ON FEWER SAMPLES

Original: Conventional PINNs evaluate their PDE constraints with either AD or ND.
Revised: Conventional PINNs evaluate their PDE constraints with either AD or ND.

Original: On a simple 1D convection-diffusion equation problem as exemplified in Fig.2, PINNs with both AD- and ND-loss exhibit incorrect solution when sparse samples are used for training.
Revised: On a simple 1D convection-diffusion equation problem as exemplified in Fig.2, PINNs with both AD- and ND-loss exhibit incorrect solution when sparse samples are used for training.

Original: Note that this is not the only possible failure mode for PINNs (Krishnapriyan et al., 2021).
Revised: Note that this is not the only possible failure mode for PINNs (Krishnapriyan et al., 2021).

Original: The issue is that although the training is fast and the convergence is good (training losses are close to machine precision, i.e., < 1e-10), their final solutions are incorrect.
Revised: The issue is that although the training is fast and the convergence is good (training losses are close to machine precision, i.e., < 1e-10), their final solutions are incorrect.

Original: Moreover, their failure patterns are different.
Revised: Moreover, their failure patterns are different.

Original: Failure pattern of AD-loss PINN.
Revised: Failure pattern of AD-loss PINN.

Original: AD has become a popular method for evaluation of the PDE constraint in PINN loss because it can exactly compute the derivative terms at any sample location directly from the input and the network weights.
Revised: AD has become a popular method for evaluation of the PDE constraint in PINN loss because it can exactly compute the derivative terms at any sample location directly from the input and the network weights.

Original: However, the AD-loss requires dense samples to guide the correct training.
Revised: However, the AD-loss requires dense samples to guide the correct training.

Original: When samples are sparse relative to the local complexity, a highly flexible, over-parameterized PINN “over-fits” in between regions, leading to incorrect solution even as the PDE constraint is fulfilled at all sample points (Fig. 2 example).
Revised: When samples are sparse relative to the local complexity, a highly flexible, over-parameterized PINN “over-fits” in between regions, leading to incorrect solution even as the PDE constraint is fulfilled at all sample points (Fig. 2 example).

Original: Failure pattern of ND-loss PINN.
Revised: Failure pattern of ND-loss PINN.

Original: On the contrary, ND-loss are more robust to sample density.
Revised: On the contrary, ND-loss are more robust to sample density.

Original: The fundamental difference is that ND-type methods approximate the derivative terms by using PINN output from neighbouring samples , e.g., a finite difference-type stencil, which connects sparse samples into piecewise continuous regions to facilitate fast training across the entire problem domain.
Revised: The fundamental difference is that ND-type methods approximate the derivative terms by using PINN output from neighbouring samples , e.g., a finite difference-type stencil, which connects sparse samples into piecewise continuous regions to facilitate fast training across the entire problem domain.

Original: The use of lower order approximation to the derivative terms also provides certain structural bias for
Revised: The use of lower order approximation to the derivative terms also provides certain structural bias for

Original: PINNs to be more easily trained.
Revised: PINNs to be more easily trained.

Original: For example, abnormally high gradients are regularized so that it is easier for PINNs to learn the solution, with the accuracy being dependent on the approximation employed.
Revised: For example, abnormally high gradients are regularized so that it is easier for PINNs to learn the solution, with the accuracy being dependent on the approximation employed.

Original: Moreover, ND methods do not pose any requirement on differentiability of the network, and they naturally fit the CNN-architecture which utilizes a connected set of grid points as input.
Revised: Moreover, ND methods do not pose any requirement on differentiability of the network, and they naturally fit the CNN-architecture which utilizes a connected set of grid points as input.

Original: The example in Fig.
Revised: The example in Fig.

Original: 2 demonstrates that ND-loss can still be subject to undesired training failure.
Revised: 2 demonstrates that ND-loss can still be subject to undesired training failure.

Original: The reason here is because the near-boundary training samples do not perfectly connect to the domain boundary through their stencils (as illustrated in Fig. 3).
Revised: The reason here is because the near-boundary training samples do not perfectly connect to the domain boundary through their stencils (as illustrated in Fig. 3).

Original: Instead, their stencils fall outside the boundary.
Revised: Instead, their stencils fall outside the boundary.

Original: Their outputs essentially act as free parameters during optimization for satisfying the
Revised: Their outputs essentially act as free parameters during optimization for satisfying the

Original: PDE constraint at near-boundary samples, and the PINN training may “over-fit” the near-boundary region to more easily jointly satisfy the PDE and BC constraints.
Revised: PDE constraint at near-boundary samples, and the PINN training may “over-fit” the near-boundary region to more easily jointly satisfy the PDE and BC constraints.

Original: This becomes a major issue for
Revised: This becomes a major issue for

Original: ND-loss when dealing with irregular geometries , because it is almost impossible to design sample locations such that their stencils perfectly connect the domain boundary to other training samples.
Revised: ND-loss when dealing with irregular geometries , because it is almost impossible to design sample locations such that their stencils perfectly connect the domain boundary to other training samples.

Original: One potential solution is to increase the sampling density near the boundary, i.e., ∆ ⃗ x → 0, to avoid such over-fitting.
Revised: One potential solution is to increase the sampling density near the boundary, i.e., ∆ ⃗ x → 0, to avoid such over-fitting.

Original: This however increases the complexity of the optimization problem and can cause adverse effects on training.
Revised: This however increases the complexity of the optimization problem and can cause adverse effects on training.

Original: It is also very challenging to locally refine the sampling density for
Revised: It is also very challenging to locally refine the sampling density for

Original: PINNs with CNN-architecture.
Revised: PINNs with CNN-architecture.

Original: To resolve this issue without increasing the sample density, we propose Fast-PINN to impose local structure to govern the near boundary gradient behaviour and restore connectivity between domain boundary and near-boundary samples.
Revised: To resolve this issue without increasing the sample density, we propose BCXN-PINN to impose local structure to govern the near boundary gradient behaviour and restore connectivity between domain boundary and near-boundary samples.

Original: [Figure]
Revised: [Figure]

Original: in ND-loss .

Original: BC information is passed to stencils in fluid domain (blue square) (a) successfully in regular domain; and (b) unsuccessfully in an irregular domain due to under-defined stencils (unfilled square) outside the domain.

Original: (c) Schematic of related definitions under the present Fast-PINN framework.

Original: BCXN-loss enforces a constraint across 3 points: external stencil point A, boundary point P, and mirror point Q inside fluid domain.

Original:  M ETHOD 4. E NFORCING LINEAR CONSTRAINT AT NEAR - BOUNDARY SAMPLES
Revised:  M ETHOD 4. E NFORCING LINEAR CONSTRAINT AT NEAR - BOUNDARY SAMPLES

Original: This section outlines the proposed Fast-PINN method.
Revised: This section outlines the proposed BCXN-PINN method.

Original: Here we denote the external (out-of-domain) stencil of a near-boundary sample as ES point(s).
Revised: Here we denote the external (out-ofdomain) stencil of a near-boundary sample as ES point(s).

Original: As per Fig.
Revised: As per Fig.

Original: 3c, the boundary condition ⃗ u BC and field value ⃗ u MI are defined at the boundary point P and a chosen mirror point Q inside the domain, while the field value ⃗ u ES at the ES point A is the value to be determined.
Revised: 3c, the boundary condition ⃗ u BC and field value ⃗ u MI are defined at the boundary point P and a chosen mirror point Q inside the domain, while the field value ⃗ u ES at the ES point A is the value to be determined.

Original: By employing Taylor series expansion on points P and Q relative to point A along AQ with respect to local coordinate ξ , the following equations can be derived:
Revised: By employing Taylor series expansion on points A and Q relative to point P along AQ with respect to local coordinate n (normal direction of solid surface), the following equations can be derived:

Original: [Equation]
Revised: [Equation]

Original: where AP and PQ are the distances between A and Q and between P and Q, and AQ = AP + AQ .
Revised: where AP and PQ are the distances between A and P and between P and Q, and AQ = AP + PQ .

Original: By further manipulation of the above equations, the field value ⃗ u ES at point A can be derived as:
Revised: By further manipulation of the above equations, the field value ⃗ u ES at point A can be derived as:

Original: [Equation]
Revised: [Equation]

Original: From the above equation, it can be seen that ⃗ u ES is linearly constrained by ⃗ u BC and ⃗ u MI .
Revised: From the above equation, it can be seen that ⃗ u ES is linearly constrained by ⃗ u BC and ⃗ u MI .

Original: While the mirror point Q can be conveniently chosen at the center of stencil, i.e., the near-boundary sample location, we choose mirror point Q along the normal direction of boundary P (ref.
Revised: While the mirror point Q can be conveniently chosen at the center of stencil, i.e., the near-boundary sample location, we choose mirror point Q along the normal direction of boundary P (ref.

Original: to Fig.
Revised: to Fig.

Original: 3c) as it is more general and performs better in our experiments.
Revised: 3c) as it is more general and performs better in our experiments.

Original: Hence, AP = PQ , and Eq.
Revised: Hence, AP = PQ , and Eq.

Original: (4) becomes:
Revised: (4) becomes:

Original: [Equation]
Revised: [Equation]

Revised: A similar process can be applied for the derivation of corresponding linear constraints for Neumann type and Robin boundary conditions, as detailed in Appendix Section A.1.

Original:  PINN methods generally require a routine to sample inside the geometry of interest and at the boundary.
Revised: In addition, PINN methods generally require a routine to sample inside the geometry of interest and at the boundary.

Original: As compared to conventional PINNs, Fast-PINN involves additional steps.
Revised: As compared to conventional PINNs, BCXN-PINN involves additional steps.

Original: We describe the steps to compute the mirror point location (along normal direction) inthe Appendix Section A.1.
Revised: We describe the steps to compute the mirror point location (along normal direction) in Appendix Section A.2.

Original: 4. B OUNDARY CONNECTIVITY (BCXN)- LOSS WITH DIRECT FORCING APPROACH
Revised: 4. B OUNDARY CONNECTIVITY (BCXN)- LOSS WITH DIRECT FORCING APPROACH

Original: Essentially, the use of a linear constraint can connect under-defined external stencil points to the
Revised: Essentially, the use of a linear constraint can connect under-defined external stencil points to the

Original: BCs and the solutions from inside the boundary.
Revised: BCs and the solutions from inside the boundary.

Original: We can directly apply Eq.
Revised: We can directly apply Eq.

Original: (5) to compute the field value ⃗ u ES for any external stencil point during the evaluation of PDE constraint on near-boundary samples with ND-type schemes.
Revised: (5) to compute the field value ⃗ u ES for any external stencil point during the evaluation of PDE constraint on near-boundary samples with ND-type schemes.

Original: This direct forcing approach has an association to the direct forcing immersed boundary methods in numerical computing.
Revised: This direct forcing approach has an association to the direct forcing immersed boundary methods in numerical computing.

Original: We can then use the following df-BCXN-loss :
Revised: We can then use the following df-BCXN-loss :

Original: [Equation] 
Revised: [Equation] [Equation]

Original: (6) where L PDE ( df − BCXN ) is our Fast-PINN’s PDE loss term whereby the PDE constraint on nearboundary samples is modulated by the direct forcing approach.
Revised: where L PDE ( df − BCXN ) is our BCXN-PINN’s PDE loss term whereby the PDE constraint on nearboundary samples is modulated by the direct forcing approach.

Original: In such an implementation, the BCs are implicitly “infused” into the training loss through the linear constraint and there is no longer a need to explicitly evaluate the BC loss.
Revised: In such an implementation, the BCs are implicitly “infused” into the training loss through the linear constraint and there is no longer a need to explicitly evaluate the BC loss.

Original: 4. B OUNDARY CONNECTIVITY (BCXN)- LOSS WITH SOFT FORCING APPROACH
Revised: 4. B OUNDARY CONNECTIVITY (BCXN)- LOSS WITH SOFT FORCING APPROACH

Original: The df-BCXN-loss strongly enforces the linear constraint during the evaluation of PDE constraint at near-boundary samples.
Revised: The df-BCXN-loss strongly enforces the linear constraint during the evaluation of PDE constraint at near-boundary samples.

Original: However, the linear condition described by Eq.
Revised: However, the linear condition described by Eq.

Original: (5) does not guarantee the best approximation to the local gradient behaviour.
Revised: (5) does not guarantee the best approximation to the local gradient behaviour.

Original: The imposition of such an over-simplified constraint can be inappropriate in some scenarios, hence slowing convergence or reducing accuracy in certain instances.
Revised: The imposition of such an over-simplified constraint can be inappropriate in some scenarios, hence slowing convergence or reducing accuracy in certain instances.

Original: To alleviate the issue, we propose to relax the linear constraint — which may be in conflict with the local gradients — used for propagating BCs to the near-boundary samples in
Revised: To alleviate the issue, we propose to relax the linear constraint — which may be in conflict with the local gradients — used for propagating BCs to the near-boundary samples in

Original: Fast-PINN by introducing an additional loss term , in an approach referred to as soft forcing .
Revised: BCXN-PINN by introducing an additional loss term , in an approach referred to as soft forcing .

Original: Let us denote ⃗ u ES i as the field value computed at the i-th external stencil point by Eq.
Revised: Let us denote ⃗ u ES i as the field value computed at the i-th external stencil point by Eq.

Original: (5), and ⃗ u ES i
Revised: (5), and ⃗ u ES i

Original: ( ⃗ x ; w ) as PINN output at corresponding external stencil point.
Revised: ( ⃗ x ; w ) as PINN output at corresponding external stencil point.

Original: We then define BCXN-loss as:
Revised: We then define BCXN-loss as:

Original: [Equation]
Revised: [Equation]

Original: (7) for all the ES points i = 1 ,..., n ES which are required for evaluating the PDE constraint using NDtype schemes.
Revised: (7) for all the ES points i = 1 ,..., n ES which are required for evaluating the PDE constraint using NDtype schemes.

Original: The newly introduced L BCXN specifies the relation at near boundary points, it also explicitly “infuses” the BC information into the PDE samples, hence helping to propagate the correct
Revised: The newly introduced L BCXN specifies the relation at near boundary points, it also explicitly “infuses” the BC information into the PDE samples, hence helping to propagate the correct

Original: BC information during training.
Revised: BC information during training.

Original: The new sf-BCXN-loss is then defined as:
Revised: The new sf-BCXN-loss is then defined as:

Original: [Equation]
Revised: [Equation]

Original: (8) with additional weight, λ BCXN , controlling the relative importance of the loss term L BCXN in the loss function.
Revised: (8) with additional weight, λ BCXN , controlling the relative importance of the loss term L BCXN in the loss function.

Original: When λ BCXN → 0, the L PINN ( sf − BCXN ) reverts to the conventional ND -PINN loss L PINN .
Revised: When λ BCXN → 0, the L PINN ( sf − BCXN ) reverts to the conventional ND -PINN loss L PINN .

Original:  R ESULTS
Revised:  R ESULTS

Original: We first study the performance of the proposed Fast-PINN on diverse  are in Appendix A.7).
Revised: We first study the performance of the proposed BCXN-PINN on diverse 2D and 3D fluid dynamics test cases under forward and inverse problem settings using MLP architecture (inverse problem results are in Appendix A.12).

Original: Then, we demonstrate its efficacy on CNN architecture for both forward and meta-modelling problems.
Revised: Then, we demonstrate its efficacy on CNN architecture for both forward and meta-modelling problems.

Original: We also demonstrate that Fast-PINN is broadly applicable to different ND-type training losses, including finite difference-type schemes and coupled-automatic-numerical differentiation (CAN) scheme.
Revised: We also demonstrate that BCXN-PINN is broadly applicable to different ND-type training losses, including finite difference-type schemes and coupled-automaticnumerical differentiation (CAN) scheme.

Original: The Fast-PINN is compared with baseline PINNs which use ND-loss and AD-loss based on mean squared error (MSE) of the respective PINN solutions’ velocity vector ⃗ u relative to ground truth.
Revised: The BCXN-PINN is compared with baseline PINNs which use ND-loss and AD-loss based on mean squared error (MSE) of the respective PINN solutions’ velocity vector ⃗ u relative to ground truth.

Original: To facilitate comparison, identical network architectures, initialization distribution, and training settings are employed and summarized in Table 1 and Table 2 in Appendix A.2.
Revised: To facilitate comparison, identical network architectures, initialization distribution, and training settings are employed and summarized in Table 1 and Table 3 in Appendix A.3.

Original: For each experiment, we perform and present results based on 10 independent runs with different initialization.
Revised: For each experiment, we perform and present results based on 10 independent runs with different initialization.

Original: The ground truth solutions are obtained by an in-house numerical solver based on the improved divergence-free condition compensated (IDFC) method (Chiu, 2018).
Revised: The ground truth solutions are obtained by an in-house numerical solver based on the improved divergence-free condition compensated (IDFC) method (Chiu, 2018).

Original: The 4 fluid dynamics test cases (schematics in Appendix A.3) presented are:
Revised: The 4 fluid dynamics test cases (schematics in Appendix A.4) presented are:

Original: • 2D semi-circle lid-driven cavity flow, Re = 1000 .
Revised: • 2D semi-circle lid-driven cavity flow, Re = 1000 .

Original: The fluid flow inside the semi-circle cavity is driven by a lid velocity u lid = 1 , v lid = 0 at the top wall.
Revised: The fluid flow inside the semi-circle cavity is driven by a lid velocity u lid = 1 , v lid = 0 at the top wall.

Original: No-slip condition ( u = v = 0 ) is applied to the other wall.
Revised: No-slip condition ( u = v = 0 ) is applied to the other wall.

Original: There is a primary eddy near the cavity’s center.
Revised: There is a primary eddy near the cavity’s center.

Original: • 2D lid-driven cavity flow, Re = 1000 .
Revised: • 2D lid-driven cavity flow, Re = 1000 .

Original: The fluid flow inside a 1 × 1 unit square cavity is driven by a lid velocity u lid = 1 , v lid = 0 at the top wall.
Revised: The fluid flow inside a 1 × 1 unit square cavity is driven by a lid velocity u lid = 1 , v lid = 0 at the top wall.

Original: No-slip condition is applied to the other walls.
Revised: No-slip condition is applied to the other walls.

Original: The lid-driven cavity flow has been widely chosen as a benchmark case for many numerical methods due to the complex physics encapsulated within.
Revised: The lid-driven cavity flow has been widely chosen as a benchmark case for many numerical methods due to the complex physics encapsulated within.

Original: At Re = 1000, there is a primary eddy near the center of the cavity, with secondary and tertiary eddies at the bottom-right and bottom-left regions.
Revised: At Re = 1000, there is a primary eddy near the center of the cavity, with secondary and tertiary eddies at the bottom-right and bottom-left regions.

Original: This test case has a regular domain, but we chose sample locations such that the near boundary samples have their stencils fall outside the domain to show that the proposed method can be useful for this kind of scenarios.
Revised: This test case has a regular domain, but we chose sample locations such that the near boundary samples have their stencils fall outside the domain to show that the proposed method can be useful for this kind of scenarios.

Original: • 2D wavy channel flow, Re = 100 .
Revised: • 2D wavy channel flow, Re = 100 .

Original: The fluid flow passing through a long wavy channel is studied.
Revised: The fluid flow passing through a long wavy channel is studied.

Original: The inlet profile at left boundary is defined as u ( 0 , y )
Revised: The inlet profile at left boundary is defined as u ( 0 , y )

Original: = − 3 2 y 2 + 3 2 , v = 0.
Revised: = − 3 2 y 2 + 3 2 , v = 0.

Original: A non-slip condition is applied to the top and bottom walls, while outlet boundary conditions ( ∂ u ∂ x = ∂ v ∂ x = p = 0 ) are applied to the right boundary.
Revised: A non-slip condition is applied to the top and bottom walls, while outlet boundary conditions ( ∂ u ∂ x = ∂ v ∂ x = p = 0 ) are applied to the right boundary.

Original: • 3D bend tube flow, Re = 100 .
Revised: • 3D bend tube flow, Re = 100 .

Original: The fluid flow passing through a 90 o bending circular tube is studied.
Revised: The fluid flow passing through a 90 o bending circular tube is studied.

Original: The circular channel diameter (D) for the pipe is set as 1, while the mean curvature of the bending curve (R) is set as 3.
Revised: The circular channel diameter (D) for the pipe is set as 1, while the mean curvature of the bending curve (R) is set as 3.

Original: For this problem, the inlet velocity is specified as a fully developed parabolic profile with bulk velocity equal to unity.
Revised: For this problem, the inlet velocity is specified as a fully developed parabolic profile with bulk velocity equal to unity.

Original: The length before the bend curve (L1) is set as 3.5, while the length after the bend curve (L2) is set as 10.
Revised: The length before the bend curve (L1) is set as 3.5, while the length after the bend curve (L2) is set as 10.

Original: 5. 2D AND 3D FORWARD PROBLEMS WITH MLP- ARCHITECTURE F AST -PINN
Revised: 5. 2D AND 3D FORWARD PROBLEMS WITH MLP- ARCHITECTURE BCXN-PINN

Original: In the forward problem experiment, we train PINN models to learn the solution directly from the governing law using different loss functions, i.e., df-BCXN- and sf-BCXN-loss for Fast-PINNs, and AD- and ND-loss for baseline PINNs.
Revised: In the forward problem experiment, we train PINN models to learn the solution directly from the governing law using different loss functions, i.e., df-BCXN- and sf-BCXN-loss for BCXN-PINNs, and AD- and ND-loss for baseline PINNs.

Original: We perform 10 independent runs with different initialization for each PINN model.
Revised: We perform 10 independent runs with different initialization for each PINN model.

Original: Their learning outcomes on all 4 test cases are shown in Fig.
Revised: Their learning outcomes on all 4 test cases are shown in Fig.

Original: 4, in which we observe a noticeable improvement in solution accuracy ( > 2 orders of magnitude lower ⃗ u MSE) with the Fast-PINN methods.
Revised: 4, in which we observe a noticeable improvement in solution accuracy ( > 2 orders of magnitude lower ⃗ u MSE) with the BCXN-PINN methods.

Original: Fig.
Revised: Fig.

Original: 12, Fig. 13 and Fig.
Revised: 9, Fig. 10 and Fig.

Original: 14 in Appendix A.4 visualize the velocity magnitude ∥ ⃗ u ∥ and pressure p contours for the median solutions.
Revised: 11 in Appendix A.5 visualize the velocity magnitude ∥ ⃗ u ∥ and pressure p contours for the median solutions.

Original: Visually, the solutions obtained from both df-BCXN- and sf-BCXN-loss have a very good agreement with the ground truth.
Revised: Visually, the solutions obtained from both df-BCXN- and sf-BCXN-loss have a very good agreement with the ground truth.

Original: The results indicate that our present Fast-PINN method can successfully learn a good solution with low MSE across a diverse set of problems with irregular domains and complex physics.
Revised: The results indicate that our present BCXN-PINN method can successfully learn a good solution with low MSE across a diverse set of problems with irregular domains and complex physics.

Original: Both the conventional

Original:  ND- and AD-loss PINNs fail to produce a reasonable solution, i.e., they cannot properly learn the correct flow withthe current sampling density and/or training iteration. 
Revised: Both the conventional ND- and AD-loss PINNs fail to produce a reasonable solution, i.e., they cannot properly learn the correct flow with current sampling density and training iteration. In addition, we apply our method to a i) real-world relevant problem of flow past an airfoil and ii) multi-physics coupled heat and flow problem. Similar improvements are observed, and results are presented in Appendix A.6 and A.7.

Revised: We discuss the benefits of this connectivity in Appendix A.8.

Original: Experiment with different sampling density.
Revised: Experiment with different sampling density.

Original: We study the 2D lid-driven cavity flow and 3D bend tube channel flow problems to understand the trade-off between the convergence in accuracy and sampling density.
Revised: We study the 2D lid-driven cavity flow and 3D bend tube channel flow problems to understand the trade-off between the convergence in accuracy and sampling density.

Original: We train the same PINN model on a denser (i.e., 3-4x) set of samples until the accuracy of the ND- and AD-loss PINNs start to improve.
Revised: We train the same PINN model on a denser (i.e., 3-4x) set of samples until the accuracy of the ND- and AD-loss PINNs start to improve.

Original: Note that denser sampling requires larger training iteration and batch size, and the tuning of these training hyper-parameters can be very time consuming.
Revised: Note that denser sampling requires larger training iteration and batch size, and the tuning of these training hyper-parameters can be very time consuming.

Original: Fig.
Revised: Fig.

Original: 5a-b compare the convergence trends in terms of ⃗ u MSE between sparse (i.e., the one shown in previous section) and dense sampling training scenarios for the 3D bend tube channel flow problem.
Revised: 5a-b compare the convergence trends in terms of ⃗ u MSE between sparse (i.e., the one shown in previous section) and dense sampling training scenarios for the 3D bend tube channel flow problem.

Original: Fast-PINN learns more accurate solutions ( > 1 order of magnitude lower ⃗ u MSE) than the conventional PINNs with faster speed (50% less training iterations), while using fewer[Figure] [Figure] ( < 50%) 
Revised: BCXN-PINN learns more accurate solutions ( > 1 order of magnitude lower ⃗ u MSE) than the conventional PINNs with faster speed (50% less training iterations), while using fewer ( < 50%) training samples.

Original: training samples. A similar trend is observed for the 2D lid-driven cavity flow problem (Fig. 5c), where we observe that Fast-PINN quickly learns an accurate solution under sparse sample training.
Revised: A similar trend is observed for the 2D lid-driven cavity flow problem (Fig. 5c), where we observe that BCXN-PINN quickly learns an accurate solution under sparse sample training.

Original: Although all PINN models can eventually achieve improved accuracy in this test case, Fig.
Revised: Although all PINN models can eventually achieve improved accuracy in this test case, Fig.

Original: 5d indicates that it is much harder and slower to learn from a dense set of samples, thereby demonstrating the benefits of Fast-PINN.
Revised: 5d indicates that it is much harder and slower to learn from a dense set of samples, thereby demonstrating the benefits of BCXN-PINN. Additional details are in Appendix A.9.

Original: Hyper-parameter tuning.
Revised: Hyper-parameter tuning.

Original: We use settings as per Table 1 in Appendix A.2, and do not exhaustively search for best set of hyper-parameters for the individual PINN models.
Revised: We use settings as per Table 1 in Appendix A.3, and do not exhaustively search for best set of hyper-parameters for the individual PINN models.

Original: Apart from sampling density, we observe thatthe PINN learningoutcome is sensitive to the capacity of the network, batch size and learning rate.
Revised: Apart from sampling density, we observe that PINN learning is sensitive to NN capacity, batch size and learning rate.

Original: Another important hyper-parameter for PINNs is the relative weights λ PDE and λ BC in the loss function L PINN .
Revised: Another important hyper-parameter for PINNs is the relative weights λ PDE and λ BC in the loss function L PINN .

Original: Specifically, we can further decompose λ PDE into λ DIV for the divergence free condition (Eq. (1a)) and λ MOM for the conservation of momentum equations (Eq. (1b)) in the PDE loss.
Revised: Specifically, we can further decompose λ PDE into λ DIV for the divergence free condition (Eq. (1a)) and λ MOM for the conservation of momentum equations (Eq. (1b)) in the PDE loss.

Original: A properly tuned λ DIV can further improve the solution accuracy of FastPINN by another half order of magnitude in ⃗ u MSE for the 2D wavy channel flow ( λ DIV = 5) and 3D bend tube channel flow ( λ DIV = 20). 
Revised: A properly tuned λ DIV can further improve the solution accuracy of BCXN-PINN by another half order of magnitude in ⃗ u MSE for the 2D wavy channel flow ( λ DIV = 5) and 3D bend tube channel flow ( λ DIV = 20). We provide more details in Appendix A.11.

Original: 5. CNN- ARCHITECTURE F AST -PINN: FORWARD AND META - MODELLING PROBLEMS
Revised: 5. CNN- ARCHITECTURE BCXN-PINN: FORWARD AND META - MODELLING PROBLEMS

Original: The Fast-PINN method does not pose any requirement on the differentiable property of the networks hence it can be freely implemented in any neural network architecture, including CNN.
Revised: The BCXN-PINN method does not pose any requirement on the differentiable property of the networks hence it can be freely implemented in any neural network architecture, including CNN.

Original: We demonstrate this by using the df-BCXN-loss for learning the solution of a 2D semi-circle lid-driven cavity flow problem with U-Net architecture.
Revised: We demonstrate this by using the df-BCXN-loss for learning the solution of a 2D semi-circle lid-driven cavity flow problem with U-Net architecture.

Original: We show that the df-BCXN-loss can benefit CNNarchitecture PINNs by remedying training issues stemming from being unable to exactly resolve complex geometries due to the use of a discretized input.
Revised: We show that the df-BCXN-loss can benefit CNNarchitecture PINNs by remedying training issues stemming from being unable to exactly resolve complex geometries due to the use of a discretized input.

Original: We also test the Fast-PINN method with df-BCXN-loss on a 2D lid-driven cavity flow problem to demonstrate its superior performance.
Revised: We also test the BCXN-PINN method with df-BCXN-loss on a 2D lid-driven cavity flow problem to demonstrate its superior performance.

Original: The U-Net architecture and training settings are summarized in Appendix A.2 Table 2.
Revised: The U-Net architecture and training settings are summarized in Appendix A.3 Table 3.

Original: We train the forward Fast-PINN models on a single discretized 2D input which consists of 3 channels, namely the x-, y-coordinate, and an indicator to differentiate fluid and non-fluid domain.
Revised: We train the forward BCXN-PINN models on a single discretized 2D input which consists of 3 channels, namely the x-, y-coordinate, and an indicator to differentiate fluid and non-fluid domain.

Original: The performance of Fast-PINN and baseline PINN (i.e., using ND-loss ) on the 2 test cases are compared in Fig.
Revised: The performance of BCXN-PINN and baseline PINN (i.e., using ND-loss ) on the 2 test cases are compared in Fig.

Original: Like the MLP experiment, Fast-PINN has a noticeable improvement in the solution accuracy (2-3 orders of magnitude lower ⃗ u MSE).
Revised: Like the MLP experiment, BCXN-PINN has a noticeable improvement in the solution accuracy (2-3 orders of magnitude lower ⃗ u MSE).

Original: Visually, the solutions obtained from the Fast-PINN have a very good agreement with the ground truth, while the baseline PINN fails to produce a reasonable solution (in Appendix A.8 Fig. 18).
Revised: Visually, the solutions obtained from the BCXN-PINN have a very good agreement with the ground truth, while the baseline PINN fails to produce a reasonable solution (in Appendix A.13 Fig. 19). Additional results for a transient version of 2D semi-circle lid-driven cavity flow problem are presented in Appendix A.15 along with details on the impact of Re on model error and training time in Appendix A.16.

Original: [Figure]
Revised: [Figure]

Original: Meta-PINN model.
Revised: Meta-PINN model.

Original: The CNN-architecture Fast-PINN represent the entire spatial domain as a single input.
Revised: The CNN-architecture BCXN-PINN represent the entire spatial domain as a single input.

Original: We can train the model on multiple samples, each representing a different scenario such as changes in geometry or physical behaviour.
Revised: We can train the model on multiple samples, each representing a different scenario such as changes in geometry or physical behaviour.

Original: Such a meta-PINN model can be used to predict new scenarios.
Revised: Such a meta-PINN model can be used to predict new scenarios.

Original: Hence, we further study the ability of Fast-PINN to learn the solutions to multiple 2D semi-circle lid-driven cavity flow scenarios concurrently (spanning Re = 50 − 1000).
Revised: Hence, we further study the ability of BCXN-PINN to learn the solutions to multiple 2D semi-circle lid-driven cavity flow scenarios concurrently (spanning Re = 50 − 1000).

Original: The model architecture and training setting is given in Appendix A.2 Table 2.
Revised: The model architecture and training setting is given in Appendix A.3 Table 3.

Original: The df-BCXN-PINN model is used to predict Re = 300 , 500 , 600 , 700 , 900 scenarios which are not used during training.
Revised: The df-BCXN-PINN model is used to predict Re = 300 , 500 , 600 , 700 , 900 scenarios which are not used during training.

Original: The results are summarized in Fig.
Revised: The results are summarized in Fig.

Original: The present Fast-PINN method successfully learns a good solution with low ⃗ u MSE (i.e., < 1e-4) for most of the Re scenarios, and achieves good prediction accuracy.  = 900.
Revised: The present BCXN-PINN method successfully learns a good solution with low ⃗ u MSE (i.e., < 1e-4) for most of the Re scenarios, and achieves good prediction accuracy. -velocity contours of the ground truth and BCXN-PINN solutions for Re = 600 and Re = 900.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We present a Fast-PINN method which can more efficiently learn the solution to PDEs for complex geometry with fewer training samples.
Revised: We present a BCXN-PINN method which can more efficiently learn the solution to PDEs for complex geometry with fewer training samples.

Original: This is accomplished by enforcing a linear constraint during the training implicitly and explicitly via direct forcing and soft forcing approaches.
Revised: This is accomplished by enforcing a linear constraint during the training implicitly and explicitly via direct forcing and soft forcing approaches.

Original: While linear constraints are applied in this work due tothe truncation of the Taylor series expansion in the derivation, it is worth studyingin future extensions if different, higher order approximations can also be applied in this framework to achieve better performance.
Revised: While linear constraints are applied in this work due to truncation of the Taylor series expansion in the derivation, it is worth studying if different, higher order approximations can also be utilized to achieve better performance in future work.

Original: Nonetheless, our comprehensive experimental studies demonstratethe practical advantages of the Fast-PINN on diverse test problems in fluid dynamics, spanning both forward and inverse problems in 2D and 3D, and a meta-model problem, improving the accuracy of solutions by orders of magnitude even while requiring much less training iterations andtraining samples. 
Revised: Nonetheless, our comprehensive experimental studies demonstrate practical advantages of BCXN-PINN on diverse test problems in fluid dynamics, spanning both forward and inverse problems in 2D and 3D, and a meta-model problem, improving the accuracy of solutions by orders of magnitude even while requiring much less training iterations and samples. A A PPENDIX A. D ERIVATION OF LINEAR CONSTRAINT FOR N EUMANN TYPE AND R OBIN TYPE BOUNDARY CONDITION Based on Eq. the linear constraint for Neumann type boundary condition can be similarly derived utilizing Eqn 3(a) - 3(b):

Revised: [Equation]

Revised: (9) or

Original: A A PPENDIX A. P ROCEDURE TO EVALUATE THE FIELD VALUE ⃗ u MI AT MIRROR POINT
Revised: [Equation] For the homogeneous Neumann boundary condition ( ∂ ⃗ u BC ∂ n = 0 ) Eq. (10) can be further simplified as

Revised: [Equation]

Revised: A similar treatment can be used to derive a corresponding linear constraint for problems with Robin type boundary condition:

Revised: [Equation]

Revised: It is then possible to obtain an appropriate expression by substituting the above expression into Eq.

Revised: [Equation]

Revised: [Equation]

Revised: In the above, ∂ ⃗ u BC ∂ n is approximated by Eq.

Revised: We forward demonstrate the ability of the current proposed method to handle various combinations of Dirichlet and Neumann type boundary conditions in Appendix A.14.

Original: Note that PINN methods in general require a routine to sample from inside the geometry of interest as well as the boundary.
Revised: Note that PINN methods in general require a routine to sample from inside the geometry of interest as well as the boundary.

Original: As compared to conventional PINNs, Fast-PINN involves additional steps.
Revised: As compared to conventional PINNs, BCXN-PINN involves additional steps.

Original: We first describe the steps to compute the mirror point location (along normal direction):
Revised: We first describe the steps to compute the mirror point location (along normal direction):

Original: • Determine whether a stencil is external.
Revised: • Determine whether a stencil is external.

Original: In this study, we first construct a level set function φ of the geometry of interest (Osher & Paragios, 2003), where the shortest distance AP between the stencil point A and boundary point P can be found.
Revised: In this study, we first construct a level set function φ of the geometry of interest (Osher & Paragios, 2003), where the shortest distance AP between the stencil point A and boundary point P can be found.

Original: From the sign of level set function, we can determine whether a stencil is external.
Revised: From the sign of level set function, we can determine whether a stencil is external.

Original: • Compute mirror point location for external stencil.
Revised: • Compute mirror point location for external stencil.

Original: Once AP is determined, we compute the location of mirror point Q inside the fluid domain, with the distance AP = PQ .
Revised: Once AP is determined, we compute the location of mirror point Q inside the fluid domain, with the distance AP = PQ .

Original: In practice, given a fixed set of training samples, all the external stencil points and their mirror locations can be pre-computed.
Revised: In practice, given a fixed set of training samples, all the external stencil points and their mirror locations can be pre-computed.

Original: On the other hand, the field value ⃗ u MI at mirror points depends on the PINN’s output and are evaluated inside a training iteration:
Revised: On the other hand, the field value ⃗ u MI at mirror points depends on the PINN’s output and are evaluated inside a training iteration:

Original: • MLP-architecture PINN.
Revised: • MLP-architecture PINN.

Original: The ⃗ u MI value can be directly obtained by evaluating ⃗ u MI ( ⃗ x ; w ) at the mirror point.
Revised: The ⃗ u MI value can be directly obtained by evaluating ⃗ u MI ( ⃗ x ; w ) at the mirror point.

Original: • CNN-architecture PINN.
Revised: • CNN-architecture PINN.

Original: The location of mirror point may not coincide with the CNN grid.
Revised: The location of mirror point may not coincide with the CNN grid.

Original: In such case, the following inverse-distance-weighted interpolation function has been utilized to obtain the ⃗ u MI value at mirror point (Chiu & Poh, 2021).
Revised: In such case, the following inverse-distance-weighted interpolation function has been utilized to obtain the ⃗ u MI value at mirror point (Chiu & Poh, 2021).

Original: [Equation]
Revised: [Equation]

Original: ( ˜ x , ˜ y ) is the location of mirror point to be interpolated, while ( x , y ) is the location of training samples inside the fluid domain.
Revised: ( ˜ x , ˜ y ) is the location of mirror point to be interpolated, while ( x , y ) is the location of training samples inside the fluid domain.

Original: In this study, h is chosen as ∆ x , which leads to a 2 ∆ x interpolation radius.
Revised: In this study, h is chosen as ∆ x , which leads to a 2 ∆ x interpolation radius.

Original: Once ⃗ u MI and ⃗ u BC are evaluated, we can compute ⃗ u ES based on Eq.
Revised: Once ⃗ u MI and ⃗ u BC are evaluated, we can compute ⃗ u ES based on Eq.

Original: (5) during training.
Revised: (5) during training.

Original: [Table]

Original: BCXN

Original: [Table]

Original: PINN

Original: [Table]
Revised: [Table]

Original: ND scheme for

Original: BCXN

Original: [Table]

Original: PINNs

Original: [Table]

Original: CAN

Original: [Table]

Original: equidistantly

Original: [Table]

Original: domain &

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table] [Table] [Table] (I) For the MLP architecture, the numbers in between input and output represent the number of nodes in each hidden layer.
Revised: (I) For the MLP architecture, the numbers in between input and output represent the number of nodes in each hidden layer.

Original: For example, ( x ) –64–20–20–20– ( ˆ u ) indicates a single input x , followed by 4 hidden layers with 64, 20, 20 and 20 nodes in each layer, and a single output ˆ u .
Revised: For example, ( x ) –64–20–20–20– ( ˆ u ) indicates a single input x , followed by 4 hidden layers with 64, 20, 20 and 20 nodes in each layer, and a single output ˆ u .

Original: (II)
Revised: (II)

Original: We incorporate the sinusoidal mapping (Wong et al., 2022) into the first hidden layer of PINN and initialize its weights by sampling from a normal distribution N ( 0 , σ 2 ) , σ = 1.
Revised: We incorporate the sinusoidal mapping (Wong et al., 2022) into the first hidden layer of PINN and initialize its weights by sampling from a normal distribution N ( 0 , σ 2 ) , σ = 1.

Original: The subsequent hidden layers use “sine” activation, except a “linear” activation function is used in the final (output) layer, and their weights are initialized by He uniform distribution.
Revised: The subsequent hidden layers use “sine” activation, except a “linear” activation function is used in the final (output) layer, and their weights are initialized by He uniform distribution.

Original: (III) Batch size: number of random sample used for 1 evaluation of L PINN (= λ PDE L PDE + λ BC L BC ) , L PINN ( df − BCXN ) (=
Revised: (III) Batch size: number of random sample used for 1 evaluation of L PINN (= λ PDE L PDE + λ BC L BC ) , L PINN ( df − BCXN ) (=

Original: λ PDE L PDE ( df − BCXN ) ) , and L PINN ( sf − BCXN ) (=
Revised: λ PDE L PDE ( df − BCXN ) ) , and L PINN ( sf − BCXN ) (=

Original: λ PDE L PDE + λ BCXN L BCXN + λ BC L BC ) .
Revised: λ PDE L PDE + λ BCXN L BCXN + λ BC L BC ) .

Original: We used a default λ PDE = 1, λ BC = 1, and λ BCXN = 1, unless otherwise mentioned.
Revised: We used a default λ PDE = 1, λ BC = 1, and λ BCXN = 1, unless otherwise mentioned.

Original: (IV)
Revised: (IV)

Original: A training iteration: 1 evaluation of L PINN , L PINN ( df − BCXN ) or L PINN ( sf − BCXN ) for backpropagating the weight gradients.
Revised: A training iteration: 1 evaluation of L PINN , L PINN ( df − BCXN ) or L PINN ( sf − BCXN ) for backpropagating the weight gradients.

Original: We use an initial learning of 1e-3 and reduce it on plateauing, until a min.
Revised: We use an initial learning of 1e-3 and reduce it on plateauing, until a min.

Original: learning rate of 5e-6 is reached.
Revised: learning rate of 5e-6 is reached.

Original: [Table]
Revised: [Table]

Revised: (I) For the MLP architecture, the numbers in between input and output represent the number of nodes in each hidden layer.

Revised: For example, ( x ) –64–20–20–20– ( ˆ u ) indicates a single input x , followed by 4 hidden layers with 64, 20, 20 and 20 nodes in each layer, and a single output ˆ u .

Original: (I)
Revised: (II)

Revised: We incorporate the sinusoidal mapping (Wong et al., 2022) into the first hidden layer of PINN and initialize its weights by sampling from a normal distribution N ( 0 , σ 2 ) , σ = 1.

Revised: The subsequent hidden layers use “sine” activation, except a “linear” activation function is used in the final (output) layer, and their weights are initialized by He uniform distribution.

Revised: (III) Batch size: number of random sample used for 1 evaluation of L PINN (= λ PDE L PDE + λ BC L BC ) , L PINN ( df − BCXN ) (=

Revised: λ PDE L PDE ( df − BCXN ) ) , and L PINN ( sf − BCXN ) (=

Revised: λ PDE L PDE + λ BCXN L BCXN + λ BC L BC ) .

Revised: We used a default λ PDE = 1, λ BC = 1, and λ BCXN = 1, unless otherwise mentioned.

Revised: (IV)

Revised: A training iteration: 1 evaluation of L PINN , L PINN ( df − BCXN ) or L PINN ( sf − BCXN ) for backpropagating the weight gradients.

Revised: We use an initial learning of 1e-3 and reduce it on plateauing, until a min.

Original:  For the U-net architecture, the numbers in between input and output represent the number of filters in each hidden layer.
Revised: learning rate of 5e-6 is reached. [Table] (I) For the U-net architecture, the numbers in between input and output represent the number of filters in each hidden layer.

Original: Each hidden layer consists of convolution operation, batch normalization, and nonlinear “sine” activation.
Revised: Each hidden layer consists of convolution operation, batch normalization, and nonlinear “sine” activation.

Original: Also, ↓ represents a 2 × 2 max-pooling operation, and ↑ represents a 2 × 2 up-sampling operation.
Revised: Also, ↓ represents a 2 × 2 max-pooling operation, and ↑ represents a 2 × 2 up-sampling operation.

Original: The network weights are initialized by He uniform distribution.
Revised: The network weights are initialized by He uniform distribution.

Original: (II) Batch size: number of sample used for 1 evaluation of L PINN (= λ PDE L PDE + λ BC L BC ) and L PINN ( df − BCXN ) (=
Revised: (II) Batch size: number of sample used for 1 evaluation of L PINN (= λ PDE L PDE + λ BC L BC ) and L PINN ( df − BCXN ) (=

Original: λ PDE L PDE ( df − BCXN ) ) .
Revised: λ PDE L PDE ( df − BCXN ) ) .

Original: We used a default λ PDE = 1 and λ BC = 1, unless otherwise mentioned.
Revised: We used a default λ PDE = 1 and λ BC = 1, unless otherwise mentioned.

Original: (III)
Revised: (III)

Original: A training iteration: 1 evaluation of L PINN , L PINN ( df − BCXN ) or L PINN ( sf − BCXN ) for backprop- agating the weight gradients.
Revised: A training iteration: 1 evaluation of L PINN , L PINN ( df − BCXN ) or L PINN ( sf − BCXN ) for backprop- agating the weight gradients.

Original: We use an initial learning of 1e-3 and reduce it on plateauing, until a min.
Revised: We use an initial learning of 1e-3 and reduce it on plateauing, until a min.

Original: learning rate of 5e-6 is reached.
Revised: learning rate of 5e-6 is reached.

Original: (IV)
Revised: (IV)

Original: The semi-circle and regular lid-driven cavity geometries consist of 3,930 and 10,000 interior points which belong to the fluid domain respectively. 
Revised: The semi-circle and regular lid-driven cavity geometries consist of 3,930 and 10,000 interior points which belong to the fluid domain respectively. [Table] (I)

Revised: For the U-net architecture, the numbers in between input and output represent the number of filters in each hidden layer.

Revised: Each hidden layer consists of convolution operation, batch normalization, and nonlinear “sine” activation.

Original: A. P ROBLEM SCHEMATIC FOR FLUID DYNAMIC CASE STUDIES
Revised: Also, ↓ represents a 2 × 2 max-pooling operation, and ↑ represents a 2 × 2 up-sampling operation. The network weights are initialized by He uniform distribution. (II) Batch size: number of sample used for 1 evaluation of L PINN (= λ PDE L PDE + λ BC L BC ) and L PINN ( df − BCXN ) (= λ PDE L PDE ( df − BCXN ) ) .

Original: The following figures are schematics of  the fluid dynamic case studies studies in this work.
Revised: We used a default λ PDE = 1 and λ BC = 1, unless otherwise mentioned. (III) A training iteration: 1 evaluation of L PINN , L PINN ( df − BCXN ) or L PINN ( sf − BCXN ) for backprop- agating the weight gradients. We use an initial learning of 1e-3 and reduce it on plateauing, until a min.

Original: Fig.
Revised: learning rate of 5e-6 is reached.

Original: Fig.
Revised: (IV)

Original: 9, Fig.
Revised: The Taylor-Couette geometry consists of 1,216 interior points in the fluid domain while the transient semi-circle lid-driven cavity problem consists of 3,930 interior spatial points andtemporal snapshots ( ∆ t = 0 . 1) respectively. In addition, we summarize the training results across different problems and model architectures in 2 Summary Tables in Appendix A.18. A. P ROBLEM SCHEMATIC FOR FLUID DYNAMIC CASE STUDIES

Original: 10 and Fig. 11 depict the 2D semi-circle lid-driven cavity, 2D lid-driven cavity, 2D wavy channel and 3D bend tube respectively. 
Revised: The following figure is schematics of the fluid dynamic case studies in this work, depict the 2D semi-circle lid-driven cavity, 2D lid-driven cavity, 2D wavy channel and 3D bend tube respectively. [Figure]

Original: A. C ONTOUR PLOTS FOR FORWARD PROBLEM WITH MLP- ARCHITECTURE F AST -PINN
Revised: A. C ONTOUR PLOTS FOR FORWARD PROBLEM WITH MLP- ARCHITECTURE BCXN-PINN

Original: The following figures are visualizations of the velocity and pressure contours obtained from the problems described in Section 5.1.
Revised: The following figures are visualizations of the velocity and pressure contours obtained from the problems described in Section 5.1.

Original: Fig.
Revised: Fig.

Original: 12, Fig. 13 and Fig.
Revised: 9, Fig. 10 and Fig.

Original: 14 are visualizations of the velocity magnitude ∥ ⃗ u ∥ and pressure p contours for the median solutions. [Table] [Figure]
Revised: 11 are visualizations of the velocity magnitude ∥ ⃗ u ∥ and pressure p contours for the median solutions. A. A DDITIONAL FORWARD PROBLEM ON 2D FLOW PAST AIRFOIL SHAPE , Re =

Original: [Table]
Revised: To further validate the applicability and efficacy of the present BCXN-loss method for more realworld complex geometries, we also applied the MLP-architecture BCXN-PINN to the modelling of incompressible flow past an airfoil, similar to other previous work (Pfaff et al., 2020).

Original: [Figure]
Revised: As shown in Fig.

Original: [Table]
Revised: 12, an airfoil with a NACA0012 profile is placed in a modelling domain ( x , y ) = ( − 3 ∼ 5 , − 2 ∼ 2 ) with zero angle of attack. A uniform inlet velocity, u inlet = 1, is specified at x = − 3, and a zero pressure outlet boundary condition is specified at x = 5. Typical slip boundary conditions are specified at the side walls.

Original: [Figure] [Table] [Figure] [Table]
Revised: The PINN model details are described in Table 3.

Original: [Figure]
Revised: Briefly, the total training samples for this problem is 79,886, with the sample spacing in the domain corresponding to ∆ x = ∆ y

Original: [Table]
Revised: = 1 / 50.

Original: [Figure]
Revised: In addition, the weight for BCXN-loss is set to be 0.1. Contour plots of the obtained velocities for these experiments are shown in Fig.

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure] 
Revised: [Figure] [Table] [Figure] [Table]

Original:  [Table]
Revised: [Figure] [Table] [Figure] [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Original: [Figure] [Table] 
Revised: [Figure] [Table] [Figure]

Original: [Figure]

Original: [Table][Figure] [Table] [Figure]
Revised: [Table]

Original: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure]

Original: [Table][Figure] [Table] [Figure]
Revised: [Table]

Original: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]
Revised: [Table]

Original: F i gu r e 12 :
Revised: F i gu r e 9 :

Original: C o m p a r i s on o f v e l o c it y m a
Revised: C o m p a r i s on o f v e l o c it y m a

Original: gn
Revised: gn

Original: it ud e ∥ ⃗ u ∥ a nd p r e ss u r e p c on t ou r b e t w ee n t
Revised: it ud e ∥ ⃗ u ∥ a nd p r e ss u r e p c on t ou r b e t w ee n t

Original: h e g r ound t r u t h a nd
Revised: h e g r ound t r u t h a nd

Original: P I
Revised: P I

Original: NN s o l u ti on s ( m e d i a n M
Revised: NN s o l u ti on s ( m e d i a n M

Original: S E fr o m 10 r un s ) , f o r t h e ( a ) 
Revised: S E fr o m 10 r un s ) , f o r t h e ( a ) 2 D

Original: [Table]

Original: D s e m i - c i r c l e li d - d r i v e n ca v it
Revised: s e m i - c i r c l e li d - d r i v e n ca v it

Original: y
Revised: y

Original: fl o w p r ob l e m , R e
Revised: fl o w p r ob l e m , R e

Original: = , a nd ( b ) 2 D li d - d r i v e n ca v it
Revised: = , a nd ( b ) 2 D li d - d r i v e n ca v it

Original: y
Revised: y

Original: fl o w p r ob l e m , R e
Revised: fl o w p r ob l e m , R e

Original: = .
Revised: = .

Original: [Table]

Original:  y[Table] [Table] o
Revised: 1 y

Original: [Table]
Revised: [Table]

Original: t

Original: [Table]
Revised: [Table]

Original: u t h 0 .
Revised: u t h 0 .

Original: [Table]
Revised: [Table]

Original:             p g o un d t r u t h 0 .
Revised:             p g o un d t r u t h 0 .

Original: [Table] [Table] 
Revised: [Table] [Table] [Table]

Original: [Table] 
Revised: [Table] [Table] [Table] [Table]

Revised: f

Original: [Table]
Revised: [Table]

Original: N
Revised: N

Original: [Table]
Revised: [Table]

Original: ss 
Revised: ss 0 .

Original: [Table]
Revised: [Table]

Original: F
Revised: [Table] [Table] [Table] [Table]

Original: [Table]
Revised: [Table]

Original: s t

Original: [Table]
Revised: [Table]

Original: P
Revised: N

Original: I NN ) 0 .

Original: [Table]
Revised: [Table]

Original: [Table]

Original: F i gu r e 13 :
Revised: F i gu r e 10 :

Original: C o m p a r i s on o f u - , v - v e l o c it y a nd p r e ss u r e p c on t ou r b e t w ee n t h e g r ound t r u t h a nd
Revised: C o m p a r i s on o f u - , v - v e l o c it y a nd p r e ss u r e p c on t ou r b e t w ee n t h e g r ound t r u t h a nd

Original: P I
Revised: P I

Original: NN s o l u ti on s ( m e d i a n M
Revised: NN s o l u ti on s ( m e d i a n M

Original: S E fr o m 10 r un s ) , f o r t h e
Revised: S E fr o m 10 r un s ) , f o r t h e

Original: D w a vy c h a nn e l fl  w p r ob l e m , R e = 100 .
Revised: D w a vy c h a nn e l fl o w [Table] r ob l e m , R e = 100 .

Original: [Table]
Revised: [Table]

Original: t
Revised: t

Original: [Table]
Revised: [Table]

Original: f
Revised: f

Original: P r e ss u r e fi
Revised: P r [Table] ss u [Table]

Original: e
Revised: e

Revised: [Table]

Original: l d 
Revised: d F i

Original: F i gu r e 14 :
Revised: [Table] r e 11 :

Original: C o m p a r i s on o f v e l o c it y m a
Revised: C o m p a r i s on o f v e l o c it y m a

Original: gn
Revised: gn

Original: it ud e ∥ ⃗ u ∥ a nd p r e ss u r e p c on t ou r b e t w ee n t
Revised: it ud e ∥ ⃗ u ∥ a nd p r e ss u r e p c on t ou r b e t w ee n t

Original: h e g r ound t r u t h a nd
Revised: h e g r ound t r u t h a nd

Original: P I
Revised: P I

Original: NN s o l u ti on s ( m e d i a n M
Revised: NN s o l u ti on s ( m e d i a n M

Original: S E fr o m 10 r un s ) , f o r t h
Revised: S E fr o m 10 r un s ) , f o r t h

Original: e 3 D
Revised: e 3 D

Original: b
Revised: b

Original: e nd t ub e c h a nn e l fl o w p r ob l e m , R e = 100 .
Revised: e nd t ub e c h a nn e l fl o w p r ob l e m , R e = 100 .

Original: P l o t s a r e o f a 2 D c r o ss - s ec ti on a t z = 0 . 
Revised: P l o t s a r e o f a 2 D c r o ss - s ec ti on a t z = 0 . [Figure]

Original:  A. E XPERIMENT WITH F AST -PINN LOSS VARIANTS
Revised: Based on these numerical experiments, we similarly notice that the incorporation of a BCXN-loss improves the PINN’s ability to learn the flow profile around the airfoil by an order of magnitude. Quantitative results are provided in Table 5. [Table] A. A DDITIONAL FORWARD PROBLEM ON 2D LID - DRIVEN CAVITY FLOW WITH HEAT TRANSFER , Re =

Revised: We further proposed to validate the applicability of the proposed method to a multi-physics example.

Revised: In this instance, we applied the MLP-architecture BCXN-PINN to the modelling of incompressible flow with heat transfer under Boussinesq approximation:

Revised: [Equation]

Revised: in the above, Pr is Prandtl number, Ri is Richardson number, and ⃗ e = ( 0 , 1 ) .

Revised: In the present study, there is a 2D lid-driven cavity domain with a stationary cylinder of radius 0.

Revised: in the centre of the domain, while Pr is set as 0.7 and Ri is set as 1.

Revised: For the boundary conditions, the bottom wall is set as the hot surface( Θ = 1) and top wall is set as the cold surface ( Θ = 0).

Revised: In addition, the side walls and cylinder surface are specified as adiabatic boundary conditions.

Revised: The

Revised: PINN model details are described in detail in Table 3 while final results are presented in Table 6.

Revised: [Table]

Revised: [Equation]

Revised: [Table]

Revised: -BCXN-CAN

Revised: [Table]

Revised: CAN

Revised: [Table]

Revised: Based on our numerical experiments

Revised: [Table]

Revised: the BCXN-loss appears to also yield some benefit on multiphysics problems

Revised: [Table]

Revised: with consistently lower MSE and relative L2 errors than the baseline comparisons.

Revised: However

Revised: [Table]

Revised: more thorough experiments need to be conducted to fully validate situations whereby

Revised: the

Revised: BCXN-loss may be more beneficial.

Revised: A. C ASE STUDY OF INFORMATION PROPAGATION

Revised: Information propagation from the boundary to the inner domain has been suggested to be critical for successful PINN modelling.

Revised: Hence, we investigate the evolution of the PDE residual across multiple training runs for the 2D semi-circle lid-driven cavity with and without the imposition of the BCXN-loss to better understand the impact of enhanced connectivity imposed by the BCXN-loss .

Revised: Illustrative contour plots of the PDE residuals and the corresponding velocity contours at various training iterations are provided in Fig.

Revised: 14 and Fig. 15.

Revised: [Table]

Revised: [Figure]

Revised: [Figure]

Revised: As described in the previous sections, the ND-PINN typically fails to converge to a good model even after 300,000 training iterations, but this can be improved via the addition of a BCXN-loss during training.

Revised: The velocity contours clearly illustrate the improved convergence.

Revised: In addition, there is a striking difference in the distribution of the PDE residuals across the domain during the early stages of the training.

Revised: In a typical ND-PINN, the PDE residuals rapidly reduce to zero, especially in the parts of the domain far away from the boundary condition.

Revised: This also corresponds to the

Revised: ND-PINN velocity profile converging to a wrong solution, and is indicative of the information from the boundary condition being lost within the domain.

Revised: On the other hand, the PDE residuals within the domain persist for a much longer period when the BCXN-loss is included which is indicative of information from the top boundary being persistently transmitted to the inner domain until the

Revised: PINN model has converged.

Revised: This is also consistent with the physical dynamics of the system, as we note that the lid-driven cavity flow problem, as the name suggests, is heavily dependent on the top boundary’s motion to drive the dynamical flow within the entire domain.

Revised: A. S AMPLING EFFECT ON 2D LID - DRIVEN CAVITY FLOW

Revised: To illustrate the impact of having near-boundary stencil points outside of the domain, we further investigate the impact of two different sets of sampling points: i) Cell-centered (whereby nearboundary points need support points outside the domain) and ii) Node-centered (whereby nearboundary points use support points that are coincident with the domain boundary).

Revised: These two sets of sampling points are tested on a 2D lid-driven cavity flow at Re = 1000 with different training batch sizes and point sampling density for the df-BCXN-CAN , CAN and AD models, and are trained for 300 , 000 iterations each.

Revised: The model MSE and L2 relative errors are collated acrossindependent runs and tabulated in Table 7.

Revised: In the above table, cells are colored according to the model MSE at the end of 300 , 000 training iterations.

Revised: Brown cells indicate scenarios under which no meaningful convergence was observed (with MSE > 1e-2).

Revised: Yellow, green and blue cells indicate scenarios with increasingly good model performance, with each band indicating an order of magnitude improvement in MSE.

Revised: [Table]

Revised: From this experiment, we notice that a 4x increase in sampling density (from 50 × 50 to 100 × 100) can improve model performance, but this also requires the use of a larger batch size, which also add computational cost.

Revised: In addition, the node-centered CAN -PINN model has improved performance relative to the cellcentered CAN -PINN version.

Revised: This further emphasizes our hypothesis that accurate modelling of the boundary conditions is an important factor in obtaining good PINN performance.

Revised: As this is a square domain, the use of a sampling point distribution that has perfect connectivity with the boundary of the domain naturally leads to improved performance.

Revised: Interestingly though, the inclusion of our proposed BCXN-loss on a cell-centered model can improve model performance to achieve a lower error than the properly aligned baseline PINN model.

Revised: This does suggest that the use of this proposed method can afford flexibility in the choice of sampling strategies, whereas conventional sampling strategies may still require sampling that takes into account the ability to maintain connectivity with the boundaries.

Original: As described in Section 4.2 and Section 4.3, there is no longer a need to explicitly evaluate the BC loss term for the df-BCXN-loss as the BCs are implicitly “infused” into the near-boundary domain points.
Revised: As described in Section 4.2 and Section 4.3, there is no longer a need to explicitly evaluate the BC loss term for the df-BCXN-loss as the BCs are implicitly “infused” into the near-boundary domain points.

Original: We verify this point by comparing the results from a Fast-PINN with the inclusion of BC loss term L BC into the df-BCXN-loss :
Revised: We verify this point by comparing the results from a BCXN-PINN with the inclusion of BC loss term L BC into the df-BCXN-loss :

Original: [Equation]
Revised: [Equation]

Original: Based on our observation, the inclusion of BC loss term doesn’t seem to benefit the Fast-PINN with df-BCXN-loss .
Revised: Based on our observation, the inclusion of BC loss term doesn’t seem to benefit the BCXN-PINN with df-BCXN-loss .

Original: It is believed that the imposition of unnecessary BC loss term excessively complicates the training — given the fact that actual BCs are already implicitly defined and utilized during the computation of df-BCXN-loss — thereby slowing down convergence.
Revised: It is believed that the imposition of unnecessary BC loss term excessively complicates the training — given the fact that actual BCs are already implicitly defined and utilized during the computation of df-BCXN-loss — thereby slowing down convergence.

Original: We further incorporate the loss term from sf-BCXN-loss into Eq.
Revised: We further incorporate the loss term from sf-BCXN-loss into Eq.

Original: [Equation]
Revised: [Equation]

Original:  to study the consequence of jointly using the df & sf approaches to modulate the Fast-PINN training loss.
Revised: (17) to study the consequence of jointly using the df & sf approaches to modulate the BCXN-PINN training loss.

Original: Interestingly, the hybrid approach performs worse than the original df- and sf-BCXN-loss .
Revised: Interestingly, the hybrid approach performs worse than the original df- and sf-BCXN- loss .

Original: This may suggest that the Fast-PINN training loss should only use either df- or sf-BCXN-loss , but not both together, potentially because of the conflicting base assumptions underlying both derivations.
Revised: This may suggest that the BCXN-PINN training loss should only use either df- or sf-BCXNloss , but not both together, potentially because of the conflicting base assumptions underlying both derivations.

Original: A. S ENSITIVITY TO λ BCXN FOR FORWARD MODELS
Revised: A. S ENSITIVITY TO λ BCXN FOR FORWARD MODELS

Original: As mentioned in Section 4.3, the BCXN weight λ BCXN in sf-BCXN-loss controls the extent of linear constraint being imposed at near boundary samples.
Revised: As mentioned in Section 4.3, the BCXN weight λ BCXN in sf-BCXN-loss controls the extent of linear constraint being imposed at near boundary samples.

Original: We observe a noticeable improvement in solution quality with the appropriate tuning of λ BCXN for the Fast-PINN in the 2D wavy channel flow ( λ BCXN = 0 . 5) and 3D bend tube channel flow ( λ BCXN = 5) test cases.
Revised: We observe a noticeable improvement in solution quality with the appropriate tuning of λ BCXN for the BCXN-PINN in the 2D wavy channel flow ( λ BCXN = 0 . 5) and 3D bend tube channel flow ( λ BCXN = 5) test cases.

Original: The joint optimization results of λ BCXN and λ DIV for the 2D wavy channel flow problem are presented in Fig.
Revised: The joint optimization results of λ BCXN and λ DIV for the 2D wavy channel flow problem are presented in Fig.

Original: These results further highlight that an overly stringent imposition of a linear constraint underlying the Fast-PINN methodology can adversely affect convergence without an accompanying increase in the emphasis ( λ DIV ) placed on the divergence free condition.[Figure] [Equation] [Figure]
Revised: These results further highlight that an overly stringent imposition of a linear constraint underlying the BCXN-PINN methodology can adversely affect convergence without an accompanying increase in the emphasis ( λ DIV ) placed on the divergence free condition.

Original: [Equation]

Original: [Figure]
Revised: [Figure]

Original: A. 2D AND 3D INVERSE PROBLEM WITH MLP- ARCHITECTURE F AST -PINN
Revised: A. 2D AND 3D INVERSE PROBLEM WITH MLP- ARCHITECTURE BCXN-PINN

Original: We further demonstrate the proposed Fast-PINNs on inverse problem, where the task is to infer the unknown physical quantities in the governing physics based on partially available observation data.
Revised: We further demonstrate the proposed BCXN-PINNs on inverse problem, where the task is to infer the unknown physical quantities in the governing physics based on partially available observation data.

Original: In particular, we seek to infer the unknown Re in the incompressible N-S equations (Eq.
Revised: In particular, we seek to infer the unknown Re in the incompressible N-S equations (Eq.

Original: (1)), as well as the solution over the problem domain, by training a PINN model with respect to the following data-constrained physics-informed loss functions:[Equation] 12a) [Equation]
Revised: (1)), as well as the solution over the problem domain, by training a PINN model with respect to the following data-constrained physics-informed loss functions:

Original: 12b

Original: [Equation]
Revised: [Equation]

Original: In addition to the known BCs, these loss functions include additional data loss component:
Revised: In addition to the known BCs, these loss functions include additional data loss component:

Original: [Equation]
Revised: [Equation]

Original: (13) to match the PINN output ⃗ u ( ⃗ x ; w ) against target ⃗ u over n obs observations.
Revised: to match the PINN output ⃗ u ( ⃗ x ; w ) against target ⃗ u over n obs observations.

Original: Inverse 2D semi-circle lid-driven cavity problem.
Revised: Inverse 2D semi-circle lid-driven cavity problem.

Original: We assume the availability of limited observations ( n obs = 10 or 20 ) of velocity ⃗ u .
Revised: We assume the availability of limited observations ( n obs = 10 or 20 ) of velocity ⃗ u .

Original: The experiment comprises of 10 independent runs, where observations are randomly drawn (i.e. obtained from the simulation).
Revised: The experiment comprises of 10 independent runs, where observations are randomly drawn (i.e. obtained from the simulation).

Original: Selected observation sets are displayed in Fig. 16a.
Revised: Selected observation sets are displayed in Fig. 17a.

Original: Besides these observations, the PINN models are trained on 3,930 equidistantly spaced samples for evaluating the PDE loss and 300 BC samples.
Revised: Besides these observations, the PINN models are trained on 3,930 equidistantly spaced samples for evaluating the PDE loss and 300 BC samples.

Original: The inverse modelling results are visualized in Fig.
Revised: The inverse modelling results are visualized in Fig.

Original: 16b, comprising the distribution of ⃗ u MSE and inferred Re − 1 for both n obs = 10 or n obs = 20 scenarios.
Revised: 17b, comprising the distribution of ⃗ u MSE and inferred Re − 1 for both n obs = 10 or n obs = 20 scenarios.

Original: With n obs = 20, our inverse Fast-PINN model consistently infers an accurate Re which is always within 10% error from the ground truth ( Re = 1000 ) and achieves ⃗ u MSE below 5e-5.
Revised: With n obs = 20, our inverse BCXN-PINN model consistently infers an accurate Re which is always within 10% error from the ground truth ( Re = 1000 ) and achieves ⃗ u MSE below 5e-5.

Original: The efficacy of df-BCXN-loss drops slightly for the more challenging n obs =scenario.
Revised: The efficacy of df-BCXN-loss drops slightly for the more challenging n obs =scenario.

Original: However, for the sf-BCXN-loss , all inferred Re values are still within 15% of the ground truth, and their solutions still achieve a ⃗ u MSE below 5e-5.
Revised: However, for the sf-BCXN-loss , all inferred Re values are still within 15% of the ground truth, and their solutions still achieve a ⃗ u MSE below 5e-5.

Original: Fig.
Revised: Fig.

Original: 16b also shows that baseline PINNs using ND- and AD-loss can infer an accurate Re in this inverse problem.
Revised: 17b also shows that baseline PINNs using ND- and AD-loss can infer an accurate Re in this inverse problem.

Original: This may suggest that, once a PINN model is able to correctly infer the unknown Re , it is much easier to learn the solution with these additional observations.
Revised: This may suggest that, once a PINN model is able to correctly infer the unknown Re , it is much easier to learn the solution with these additional observations.

Original: [Figure]
Revised: [Figure]

Original: Inverse 3D bend tube channel flow problem.
Revised: Inverse 3D bend tube channel flow problem.

Original: We assume limited observations of velocity ⃗ u ( n obs = 50 , 100 ) can be probed near the bending area Fig. 17a.
Revised: We assume limited observations of velocity ⃗ u ( n obs = 50 , 100 ) can be probed near the bending area Fig. 18a.

Original: Besides these observations, we train PINN models on 19,786 equidistantly spaced PDE samples and 1,632 BC samples, for inferring the unknown Re (ground truth Re = 100) and predicting the solution near the bending area.
Revised: Besides these observations, we train PINN models on 19,786 equidistantly spaced PDE samples and 1,632 BC samples, for inferring the unknown Re (ground truth Re = 100) and predicting the solution near the bending area.

Original: We only apply the non-slip BC at the wall boundary since the inlet and outlet conditions are assumed to be unknown.
Revised: We only apply the non-slip BC at the wall boundary since the inlet and outlet conditions are assumed to be unknown.

Original: The experiment comprises of 10 independent runs, where observations are randomly drawn (i.e., obtained from the simulation).
Revised: The experiment comprises of 10 independent runs, where observations are randomly drawn (i.e., obtained from the simulation).

Original: The inverse modelling results from different PINN models are visualized in Fig.
Revised: The inverse modelling results from different PINN models are visualized in Fig.

Original: 17b.
Revised: 18b.

Original: Again, we observe good performance from all tested PINN models, with their inferred Re always within 15% error from the ground truth.
Revised: Again, we observe good performance from all tested PINN models, with their inferred Re always within 15% error from the ground truth.

Original: They can achieve below 2e-4 ⃗ u MSE with n obs = 50, and even better accuracy (i.e., < 1e-4) with n obs = 100.
Revised: They can achieve below 2e- ⃗ u MSE with n obs = 50, and even better accuracy (i.e., < 1e-4) with n obs = 100.

Original: Nonetheless, the experiment demonstrates the effectiveness of Fast-PINN for solving complex inverse problems from limited random observations, with consistent performance across different sets of random data. 
Revised: Nonetheless, the experiment demonstrates the effectiveness of BCXN-PINN for solving complex inverse problems from limited random observations, with consistent performance across different sets of random data. [Figure]

Original: Hyper-parameter tuning.
Revised: Hyper-parameter tuning.

Original: We use a same MLP architecture, ND scheme, and batch size settings as summarized in Appendix A.2 Table 1, for the inverse problem test cases.
Revised: We use a same MLP architecture, ND scheme, and batch size settings as summarized in Appendix A.3 Table 1, for the inverse problem test cases.

Original: Each batch contain at most 50 observations for computing the data loss.
Revised: Each batch contain at most 50 observations for computing the data loss.

Original: To ensure sufficiently accurate fit to the observations, we reweight the PINN loss terms (i.e., the weight of data loss to PDE loss is 10 to 1) to give more[Figure] priority to the data loss.
Revised: To ensure sufficiently accurate fit to the observations, we reweight the PINN loss terms (i.e., the weight of data loss to PDE loss is 10 to 1) to give more priority to the data loss.

Original: The current inverse problem test cases require less training iterations to converge, as compared to the pure physics-informed learning encountered in the forward problem.
Revised: The current inverse problem test cases require less training iterations to converge, as compared to the pure physics-informed learning encountered in the forward problem.

Original: The max.
Revised: The max.

Original: iteration is set as 200,000 with an initial learning rate of 5e-4 for both test cases.
Revised: iteration is set as 200,000 with an initial learning rate of 5e-4 for both test cases.

Original: We do not apply additional linear constraints to the near boundary samples if they are part of the observation set
Revised: We do not apply additional linear constraints to the near boundary samples if they are part of the observation set. A. V ISUALIZATION OF CNN- ARCHITECTURE RESULTS

Original: [Figure]
Revised: [Figure]

Original: A. V ISUALIZATION OF CNN- ARCHITECTURE RESULTS
Revised: A. A DDITIONAL FORWARD PROBLEM ON 2D T AYLOR -C OUETTE FLOW WITH D IRICHLET AND N EUMANN TYPE BOUNDARY CONDITION

Revised: As a demonstration of the generalizability of the current proposed method to different types of boundary conditions based on the derivation in Section A.1, we further apply the described method to the solution of a 2D Taylor-Couette problem. This problem describes the flow in a fluid domain between 2 concentric cylinders. The inner cylinder is of radius R 1 = 0 .

Original: [Figure]
Revised: 45 while the outer cylinder has radius R 2 = 0 . The analytical solution for the problem is known to be: [Equation] where angular velocity ω is set as 1,

Revised: K = ω

Revised: [Equation]

Revised: and

Revised: [Equation]

Revised: =

Revised: [Equation]

Revised: The numerical experiments are performed with Re = 1000 for four different pairs of boundary condition settings for the outer (inner) cylinder: i) Dirichlet (Dirichlet); ii) Dirichlet (Neumann); iii)

Revised: Neumann (Dirichlet); and iv) Neumann (Neumann).

Revised: For all the boundary setting pairs, the training is performed with a CNN-type PINN architecture on a 56x56 voxel domain and ∆ x = ∆ y = 2 / 50.

Revised: Additional PINN model details are described in Table 4.

Revised: Contour plots of the results are presented in Fig.

Revised: 20 and model errors are presented in Table 8.

Revised: [Equation]

Revised: [Equation]

Revised: y

Revised: [Equation]

Revised: truth

Revised: [Equation]

Revised: [Figure]

Revised: settings.

Revised: [Table]

Revised: From the results, it can be seen that the BCXN-loss can be flexibly applied to problems with any combination of Dirichlet and Neumann type boundary conditions, although the majority of presented examples in this work feature Dirichlet boundary conditions

Revised: [Table]

Revised: A. A DDITIONAL FORWARD PROBLEM ON TRANSIENT 2D SEMI - CIRCLE LID - DRIVEN CAVITY FLOW , Re =

Revised: In addition to the forward modelling of steady-state 2D semi-circle lid-driven cavity flow, we further evaluate the improvement obtained from application of CNN-architecture BCXN-PINN to the modelling of a transient version of this problem.

Revised: The problem geometry and set-up is consistent with the problem description in Section 5.2.

Revised: However, the problem’s initial condition corresponds to that of a fluid domain at rest (at t = 0).

Revised: The PINN model is then used to model the evolution of the flow within the domain from t = 0 to t = 2.

Revised: Additional model settings are provided in Table 4.

Revised: The average errors for all time points between t = 0 and t = 2 . 0 yield MSEs of 3.95e-4 and 9.66e3 with and without the inclusion of BCXN-loss respectively.

Revised: This also corresponds to relative L errors of 0.129 and 0.681 respectively.

Revised: Hence, the inclusion of BCXN-loss in this transient problem can improve MSE by more than 1 order of magnitude.

Revised: Velocity contour plots of the PINN models trained with and without BCXN-loss are provided in Fig.

Revised: 21 to illustrate the differences in model convergence.

Revised: [Figure]

Revised: for t = 0 . 5, t = 1 . 0, and t = 2 . 0.

Revised: A. CNN- ARCHITECTURE BCXN-PINN AND BASELINE PINN PERFORMANCE ON A FORWARD PROBLEM WITH INCREASING COMPLEXITY

Revised: For easy reference, we summarize the average MSE and relative L2 error obtained for the 2D semicircle lid-driven cavity experiments across 5 independent runs.

Revised: These results indicate the single model errors obtained using the settings described in Table 4.

Revised: Plots of the velocity MSEs are provided in Fig.

Revised: We can clearly see that the performance difference is most pronounced for more complex boundary value problems.

Revised: As Re increases, the complexity of the developed flow typically requires more sophisticated algorithms for better capture of the non-linear effects.

Revised: [Figure]

Revised: Additional quantitative comparisons at select Reynolds numbers are presented in Table 9 below for easier reference.

Revised: [Table]

Revised: Importantly, we note that although the current proposed BCXN-PINN method may nominally involve more computation, the impact on training time is under 5% in our experiments.

Revised: The time taken for training a single df-BCXN-loss PINN is about 126-136 minutes while the time taken for training the equivalent ND- PINN with similar model size and optimization parameters is 121minutes.

Revised: This thus further emphasizes the utility of the current proposed method.

Revised: A more in-depth description is provided in Appendix Section A.

Revised: [Table]

Revised: A. I MPACT OF BCXN - LOSS ON TRAINING TIME

Revised: We further compare the time taken for training a model with and without the BCXN-loss on the

Revised: MLP-architecture and CNN-architecture models as applied to the 2D semi-circle lid-driven cavity problem at Re = 1000.

Revised: The average MSE and relative L2 error obtained are averaged acrossindependent runs, and the minimum and maximum wall-clock time are recorded.

Revised: All experiments are run with the standard Tensorflow and Keras implementation on a 20-core workstation with an

Revised: Intel Xeon Gold 6248 processor and an Nvidia RTX 2080 Ti GPU.

Revised: Quantitative comparisons for the MLP-architecture models are presented in Table 10.

Revised: All models were run for an identical number of training iterations (300,000) with the same batch size (1,000) and had the same model size to facilitate a fair comparison.

Revised: Quantitative comparisons for the CNN-architecture model are also presented in Table 11.

Revised: In additional, both models were run for an identical number of training iterations (200,000) and had the same model size to facilitate a fair comparison.

Revised: While not a comprehensive study, we note that the inclusion of the BCXN-loss across all the cases only slightly increases the amount of time required for model training.

Revised: This was on the order of less

Revised: [Table]

Revised: [Table]

Revised: than 5 % for the CNN-type architecture and the ND-MLP-type model.

Revised: Similarly, the sf-BCXN-CAN had negligible increase in training time relative to the baseline CAN- PINN model.

Revised: However

Revised: [Table]

Revised: it is worth noting that while the time taken may have increased slightly, the MSE typically decreases more than 1 order of magnitude, suggesting that this trade-off between model training time and performance with the inclusion of a BCXN-loss may still be preferred.

Revised: A. S UMMARY OF BCXN-PINN AND BASELINE PINN PERFORMANCE

Revised: For easy reference

Revised: [Table]

Revised: we summarize the BCXN-PINN’s model performance and compare them to other baseline methods previously published.

Revised: The average MSE and relative L2 error across multiple independent runs are recorded in Table 12 and Table

Revised: [Table]

Revised: I) Implementation for CAN- PINN models based on previous work (Chiu et al

Revised: [Table]

Revised: (II) Implementation for SIREN- PINN models based on previous work (Sitzmann et al

Revised: [Table]

Revised: [Table]

Revised: et al.

Revised: [Table]

Revised: 2021b

Revised: [Table]

Revised: et al.,

Revised: [Table]

Revised: [Table]

Revised: (III) Implementation for vanilla PINN models based on previous work (Raissi et al.,

Revised: [Table]

Revised: [Table]