Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper introduces a method for embedding the hard-constraints for boundary conditions (BCs) for physics-informed neural networks (PINNs). It introduces the "extra fields" to transform the original PDE into equivalent forms so that the BCs can become linear. This adds equations between the extra fields and the original PDE solutions, but can reduce total number of equations if the number of boundary conditions is high. The paper provides theoretical analysis of bounds for the gradients of the PDE loss terms. In experiments, the paper shows that the proposed method outperforms several baselines, and ablation study shows that it has smaller moving variance than without extra fields.
strengths_and_weaknesses: Strengths:

The paper address the important problem in the PINN field about the dealing with complex boundary conditions. It is well-motivated, the method makes sense and is novel. The experiment evaluation is sound. The paper is written clearly. It also has good reproducibility (in terms of set up for hyperparameters and providing the code)

Weaknesses:

I don't find apparent weaknesses.
questions: N/A
limitations: The paper adequately addresses the limitations.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: Physic-informed neural networks (PINNs) are a new paradigm for injecting the knowledge of governing equations into neural networks. To solve the problem, we need to consider initial conditions and boundary conditions, of which boundary conditions are hard to train for. Therefore, this paper proposes the general solutions to boundary conditions, which are later used to develop their "extra field" method. They experiment with two real-world problems.
strengths_and_weaknesses: This paper tackles a critical problem of PINNs since training for boundary conditions frequently collides with training for governing equations. It is needed to be balanced between them. Their method solves the problem by designing the general solutions to boundary conditions and proves several forthcoming facts.

However, this paper missed several other related works:
- https://openreview.net/forum?id=a2Gr9gNFD-J
- https://arxiv.org/abs/2012.02681

I am also not sure whether their method is theoretically correct. In general, we need to double-check whether a constructed ansatz is correct or not. Their high accuracy indirectly proves it. But it seems not sufficient.
questions: Please check the above weak points and try to answer.
limitations: They discuss a limitation at the end of the conclusion section.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: PINN has shown potential in solving high-dimensional PDE with complex geometries. Related works generally enforce the boundary conditions as a soft constraint via boundary condition loss term, which may lead to instability and hardness in convergence. The proposed work developed a reformulation of PDE to enforce boundary conditions (Dirichlet, Neumann, and Robin BCs) as a hard constraint under the PINN framework by introducing the extra fields. Thorough experiments have been done to demonstrate the proposed approach's effectiveness in terms of accuracy (measured by mean absolute error/mean absolute percentage error).  
strengths_and_weaknesses: ***originality*** -  to the best of my knowledge, the proposed reformulation that can handle all three boundary conditions is novel in the context of PINN. 

***quality*** - authors provide experiments on high dimensional/complex geometry PDEs. The derivation is mostly clear with some confusion listed in the "clarity" section and "questions" section. 

***clarity*** - the motivation is clear. The derivation may be a little hard to follow when eq(10) is proposed and the extended distance function is used.  It's a bit hard to understand by boundary condition is now satisfied as a hard constraint. While numerical results have been presented, it may also be interesting to include some qualitative results to illustrate the simulation results, etc. 

***significant*** - PINN is a simple approach to solving PDE, but it still with many challenges. Research works trying to develop and improve the convergence/accuracy of the PINN are of great interest. 
questions: 1. Why does the proposed approach enforce the boundary condition as a ***hard*** constraint? Does it require that $p=\nabla u$ is fulfilled, which is only enforced as a soft constraint?
2. How does eq11 help enforce the boundary condition and what is the motivation for using eq.10 and eq,11 as the general solution, and what's the relationship between those two equations (are the ***NN*** in both equations refer to the same thing? 
3. Could more explanation be given to $g(\cdot)$ between eq.A16 and eq.A15, and their correspondences to eq.10 and eq,11.
4. What's the motivation for using extended distance functions here? 
limitations: 1. It would be more reader-friendly if more connections can be built between the eq 10/11 and the derivation for each experiment shown in the supplementary.   
2. It's nice to include some qualitative results and discussions on run time/convergence speed, rather than just comparing the final MAE/MAPE.
review_title: null

final_score: rating: 3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
review_content: summary: The submission introduces a new technique to enforce Dirichlet, Neumann, and Robin boundary constraints into the PINNs model, based on the extra fields technique from the mixed finite element method. Extra fields decomposes the solution into a linear combination of a boundary term, which is given to us as a constraint, and an interior term, learned by a neural network. On the boundary, the interpolation is designed so that only the boundary term is accounted for in the solution. As such, this is a handcrafted hard constraint method. The authors test on 3 PDEs: the heat equation in a battery pack, the Navier-Stokes equation over an airfoil, and a 10 dimensional heat equation, comparing against a variety of PINNs-based models. They claim that their method is stabler to train.
strengths_and_weaknesses: + The mathematics and derivations appear to be sound (although a little hard to follow) and it appears from the experimental section that this technique is better than competing methods.

+ The method is grounded in a standard technique from the field of mixed finite element solvers, and so it inherits a lot of the benefits of that method.

+ The submission steps the reader through a lot of needed material and builds up intuition.

- It is hard to gauge the significance of the method, since there are no confidence intervals in the experiment section

- I could not reimplement this submission. The main paper lacks many essential details.

-  In many places, the writing lacks clarity. Terms like geometrically complex or PDE operators are used,  without clear definitions. This should, however, be easy to clear up.
questions: “Geometrically complex” -  this term is used a lot and yet not defined anywhere. It would be useful to have a definition or at least a description of what this could mean.

23 - It is stated that neural networks have only recently been applied to the solution of PDEs, but the field is in fact older, dating back more than 20 years.

Eqn 1a:  is the domain of x2 correct? I think it may be x2 \in (0,1].

107 - what is meant by “PDE operator”? I think I understand, but a clear definition would be useful. Do you not just mean a differential operator?

133: it would be useful to provide an example where you show how the nullspace of eqn. 9 is computed.

Eqn 10 - what architecture is used for NN_i? This is missing from the paper and highly influences the results that can be achieved.

Sec 3.3 - Would it be possible to achieve the decomposition of the solution	 into a hard-coded boundary term and a learned interior term without the introduction of the extra fields per se? In eqn 13 you have to add a loss term encouraging p to match \nabla u. Instead, you could have just set p to be \nabla u itself and trained this directly. Or is it too difficult to achieve this given that p itself has to live in the nullspace of eqn 9?

182 -  I think you mean width, not length

Eqn 16:  where are the weights?

Experiments: it would be useful for the boundary conditions to be explicitly shown in this section, along with the architectures used, how the nullspace basis is solved for and the choice of extended distance function l.

Tables: were the results computed over multiple runs? It would have been useful/more significant to have confidence internals/standard deviations shown, so that the significance of each result could be ascertained.

Typos: 
13 - effeteness
63 - alliviating
138 - non-trival
203 - addition -> additional
limitations: Limitations of this work are not addressed. It would be useful to write a paragraph explaining the shortcomings of having to solve for the nullspace of the boundary condition equations and other limitations of using PINNs as a baseline.
review_title: null

Meta-review: metareview: The paper considers using neural networks to solve PDEs with complex geometry by incorporating hard constraint into the approximation function class. The method is interesting and useful for practical applications of neural-network based PDE solvers. The authors have adequately addressed the concerns by the referee. The meta-reviewer recommends acceptance of the paper. 
confidence: Certain
Decision: Accept