Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a novel design of physical loss to find a trade-off between sample density and solution accuracy of PDEs. The formulation of the proposed loss tackles the challenges of approximating the boundary layer by modeling and adding a term in the loss function that takes into consideration the highly complex fluctuations of the gradients near the boundary layer. The work is inspired by a method in numerical computing.

strength_and_weaknesses: In this work, the proposed method makes the training of PINNs feasible and sufficiently accurate at the boundary layer in a sparse sample regime where existing works fail (or require a highly dense grid at the boundary layer which is not easy to construct and/or is intractable in practice). Moreover, the method shows an extra gain in terms of computational complexity and achieves accurate solutions with drastically fewer iterations in a scarce data regime. 
At the (near) boundary layer, the fluctuations are highly complex and the geometries are irregular. The proposed design of fast-PINNs loss allows in some way to cope with that. It is challenging because, given the local complexity near the boundary layer, sparse sampling leads to overfitting even though PDE constraint is applied to all the samples.

However, l have some considerations:

1/ It is important to underline that the success of PINNs relies on the successful propagation of information from initial and boundary condition points to interior points. This can be observed if we reduce the PDE solving to a local part of the domain. Do you think that your model achieves this successful propagation? how can you quantify that empirically? 

2/ In existing PINNs works, the propagation failures are tied to highly imbalanced PDE residual fields.  In other terms, residuals with high values could be observed over very narrow regions. Does the proposed work mitigate/circumvents this failure?  if yes, how does the method succeed to make the balance of PDE residual fields? 

3/ Given your method and the impact of a good choice of sampling strategy to make PINNs successful. I am wondering if the breakthrough work on PINNs is more on finding an appropriate sampling strategy near the boundary layer. If the latter is found, standard PINNs loss will work. What do you think?
A kind of sampling that focuses on points near the boundary layer with high PDE residual at every iteration. In practice, one could imagine sampling more of the points with high residuals. 

4/ What do you think of adding the diffusion term in the loss function?

5/  Why did you set manually the different weights of each loss term rather than learning them? 

6/ You mentioned that “ It is also very challenging to locally refine the sampling density for PINNs with CNN-architecture”. Recently, lots of efforts have been made to generalize learning to non-Euclidean data, known as Geometric Deep Learning. The latter is suitable to operate on meshes and irregular geometries, namely Graph Neural Networks (GNNs), PointNet.  Are there any reasons for not using GNNs ? Several works on ML&physics rely on these architectures.

7/ Lack of quantitative results. This work could be improved by adding a table of baseline w.r.t existing works on PINNs that proposes different variants of physical loss and w.r.t. different. sampling strategies

8/ It would be interesting to see what the proposed method could bring if it is combined with a given domain decomposition method.

9/ This work could be extended to more complex geometries such as the ones available in the different datasets proposed in the work entitled "Learning to simulate complex physics with graph networks". It allows to asses the generalization capabilities of the proposed method. 

10/ Do you think that the proposed method could generalize well to high Reynolds (> 10^6)

11/ The figures could be updated with log scales
clarity,_quality,_novelty_and_reproducibility: The claims of the paper are clear and easy to follow.  The whole structure of the paper is good and well-written. 
The work proposes a novel method for training PINNs derived from numerical computing works which is a great idea since these works are sufficiently mature since several decades.
The supplementary part contains all the necessary ingredients to reproduce the experiments. However, it would be better if the code could be made available.
summary_of_the_review: The contribution of the proposed work will help to make progress in ML&Physics and build efficient PINNs. 
I can increase my score depending on the responses.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This work proposed a new Boundary Connectivity (BCXN) loss function to improve the accuracy and convergence speed of PINN. Specifically, it developed two variants of BCXN loss, namely: 1) a soft forcing method which imposes a linear approximation constraint via an additional loss term, and ii) a direct forcing approach which strongly enforces a linear constraint during the evaluation of PDE loss at nearboundary samples. Experimental results on multiple datasets demonstrated that the proposed method can significantly improve accuracy while reducing training iterations. 
strength_and_weaknesses: * Strength
1. The idea of adding Boundary Connectivity (BCXN) is interesting and novel
2. The proposed loss function can be inserted into any deep neural network architecture
3. Extensive experimental results show that the introduced loss function can significantly improve accuracy while reducing training iterations.

* Limitations
1. Compared to existing work on PINN. There exist many studies on PINN, so it would be better to discuss related work and compare the proposed method with some latest baselines, such as [Zeng et al 2022]
2. There is a typo about AQ = AP+ AQ in the first line on Page 6. 
3. Question:
    I am curious if the proposed method can solve the prey and predator system which is not stable?

References:
[Zeng et al 2022] Competitive Physics Informed Networks, https://arxiv.org/pdf/2204.11144.pdf
clarity,_quality,_novelty_and_reproducibility: The paper is well written and organized except for a couple of typos. The idea is a bit novel. It would be better to release the source code.
summary_of_the_review: The contributions of this work lie in adding Boundary Connectivity (BCXN) in the loss function, which can improve accuracy and reduce training iterations. In addition, extensive experiments are conducted to validate the effectiveness of the BCXN loss function. The idea is a bit novel and interesting,
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper addresses the issue of PINNs overfitting at the boundary of the domain. Since numerical differentiation based methods require points outside the boundary of the domain to work well, the authors propose a simple and efficient solution which constraints the value of the function outside the boundary (external stencil points) in terms of “mirror points” that lay inside the domain. 

This loss can be directly enforced (i.e., the equality can imposed directly for the stencil points outside the boundary) or can be enforced softly by enforcing it as a loss function. 

The authors empirically compare the performance of Fast-PINN with normal PINNs (using numerical and autodiff loss) under 4 fluid dynamics based cases, and show Fash PINNs converge faster and have a lower mse loss.
strength_and_weaknesses: I really enjoyed reading the paper. The loss function mentioned is easy to implement and gets good results when compared to baseline PINNs. The empirical section is clear and easy to read and the authors do a great job at delineating the type overall setup of the PDEs they consider for their experimental section. The PDEs consider mainly differ in the complexity of the boundaries of the domains.

The improvement introduced by fast-PINNs seems to be quite significant compared to the normal PINNs (both in terms of the rate of convergence and final loss). The authors also point out certain inductive biases that are inherent in CNN based architectures, which is quite intriguing. 


One small question for the authors: Is there a reason that the authors on solving focussed fluid dynamics with PINNs, do the gains follow for other PDEs as well (like reaction diffusion etc?)
clarity,_quality,_novelty_and_reproducibility: The paper is easy to follow and clearly written. The loss function introduced seems to be novel and achieves good performance. 
summary_of_the_review: The authors provide a simple loss function take care of the points that lie outside the domain for numerical differentiation type loss for PDE solvers. The solution provided by the authors is simple to implement and shows improvement over normal PINN methods.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper proposed a new Boundary Connectivity (BCXN) loss function for PINNs for solving PDEs with complex geometry. BCXN uses a linear interpolation to compute the points outside the domain to address the issue that the derivative stencil points are outside the domain, and then the estimated external points can be enforced directly in the loss or via a penalty loss. Numerical experiments showed that BCXN achieves better accuracy with less training points.
strength_and_weaknesses: Strengths
- The proposed method estimates the external points and thus avoid the issue of stencil points falling outside the geomery.

Weaknesses
Comments on the methods:
- The method only works for Dirichlet BC, but not other BCs, such as Neumann BC, Robin BC, etc.
- The method uses a linear interpolation to compute external points, which will limit the accuracy.
- The name “Fast-PINN” is not proper. The focus of the proposed method is complex geometry. It is not a fast version of PINN for any PDE problems with any BCs.

Comments on the correctness of the numerical results:

- 1D convection-diffusion PDE has been solved very accuracy in many PINN papers, so it is questionable if the AD and ND results are correct. No details about the setup of Fig 2 are provided.
- In the 2D cavity flow, it is regular geometry, and all the points can be chosen inside the domain. In this case, the proposed method becomes a standard PINN, so the results of ND and AD don’t seem correct. Also, there are PINN papers with 2D cavity flow problem and PINN works well. The only explanation is that the authors make ND and AD become bad.

Other comments on the results:

- The paper uses MSE error, but other metrics should be provided, such as L2 relative error.
- The paper only considered steady-state problems, but no time dependent problems are tested.

Comments on the comparisons:

- It is unfair to compare AD/ND with BCXN as BCXN is more expensive. The authors should increase the training points of AD/ND such that the computational cost is similar, and then compare the accuracy.
- There are many other methods, e.g., adaptive sampling methods, as were discussed in Section 2, but no comparison is provided.

Other comments:

- The section 3 is “THEORY”, but there is no theory in the section.
clarity,_quality,_novelty_and_reproducibility: The written is clear.
summary_of_the_review: The proposed method doesn’t work as well as they claimed. It has many limitations. The numerical experiments are not convincing enough.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper proposes a way to handle boundary conditions for training physics-informed neural networks (PINNs) when the required derivatives are computed with numerical differentiation methods and the boundary condition points do not coincide with the collocation points. The proposed idea is simple and the experimental results are encouraging. The reviewers think that the work is quite well executed although the work would benefit from more extensive evaluation of the proposed method, for example, by considering other types of boundary conditions or by using more advanced interpolation techniques instead on the linear one. The reviewers think that Figure 2 is very misleading, it creates an impression that solving this equation is problematic for PINNs although many PINN papers report good results on this equation. We strongly recommend the authors to improve the description of Figure 2 and the accompanying claims. In conclusion, the novelty exists but it is not large, the execution and writing is good but could be improved. Because of the relatively small contribution and the high standards of ICLR, I recommend rejection.
justification_for_why_not_higher_score: relatively minor contribution
justification_for_why_not_lower_score: good experimental results
Decision: Reject