Title: Consistency of Physics-Informed Neural Networks for Second-Order Elliptic Equations
Venue: NeurIPS 2025
Accepted: yes
Keywords: consistency

PINN

kernel gradient flow

second-order elliptic equation
Abstract: The physics-informed neural networks (PINNs) are widely applied in solving differential equations. However, few studies have discussed their consistency. In this paper, we consider the consistency of PINNs when applied to second-order elliptic equations with Dirichlet boundary conditions. We first provide the necessary and sufficient condition for the consistency of the physics-informed kernel gradient flow algorithm, and then as a direct corollary, when the neural network is sufficiently wide, we obtain a necessary and sufficient condition for the consistency of PINNs based on the neural tangent kernel theory. We also estimate the non-asymptotic loss bounds of physics-informed kernel gradient flow and PINN under suitable stronger assumptions. Finally, these results inspires us to construct a notable pathological example where the PINN method is inconsistent.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper consider the consistency of PINNs when applied to second order linear elliptic equations with Dirichlet boundary conditions. The authors establish the sufficient and necessary conditions for the consistency of physics informed kernel gradient flow and sufficiently wide PINN in general cases. The authors also construct a negtive example based on the theoretical results, highlights the risks of directly applying PINNs to specific problems without theoretical assessment of reliability.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nThis paper study the dynamics of the training process and analyzing the convergence of PINNs with large network width.\nThe theoretical results is intresting and promising, show that the PINN solution depends on the true solution's smoothness and regularity.\nThis paper is clearly written, well organized, and a pleasant to read, contribute to the theoretical understanding of PINN community.\n\nWeakness:\n\n1. The results are constrained in seccond order linear elliptic equations with Dirichlet boundary conditions, limiting the theoretical guidence of PINN training for different kinds PDEs with different initial/boundary conditions.\n\n2. The concept of consistency and convergence are not well described for non-mathematicians. For AI conference, I suggest using the concept of convergence directly.\n\n3. The results are based on a continuous gradient flow of network training and neural tangent kernel. Practical gradient descent optimizers  are based on discrete gradient flow. However, this work didn't present results for discrete gradient flow.\n\nQUESTIONS:\n1. Can the results in this paper be extended to nonlinear elliptic equations, or time-dependent PDEs?\n\n2. Can the results be extended to discrete gradient flow? Notice that the convergence of continuous gradient flow and discrete gradient flow have been stated in [1] and related works.\n[1] Du S S, Zhai X, Poczos B, et al. Gradient Descent Provably Optimizes Over-parameterized Neural Networks[C]//International Conference on Learning Representations. 2018.\n\n3. What's the essential difference between this work and previous works? The authors give a short comparison in line 38-57, focusing on the training dynamics of networks, but how it affects the convergence result of PINN compared to previous works, especially [27] and [33] in your paper? A table comparison on the assumption, PDEs, convergence results ,etc, is suggested.\n\n4. What about the influence of initialization of neural networks? The paper suggest the parameters are initialized in a particular way in line 100-104, how about the commonly used Xavier initialization in codes, does it affect the results of this work?\n\n5. I have some doubts about the loss (2) $L$ and (3) $\\hat{L}$. While I think the main results should be about the emprical loss $\\hat{L}$, the continuous loss $L$ is frequently used in this paper. What comes to happen if we replace $L$ by $\\hat{L}$ in the main Theorem 3.7 and Lemma 4.1?\n\nLIMITATIONS:\nThe authors addressed some limitations in the conclusion section.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-16"}

comments: SUMMARY:
This paper establishes a necessary and sufficient condition for Physics-Informed Neural Network (PINN) consistency in solving second-order elliptic PDEs: convergence occurs if and only if the true solution $u^*$ lies in the closure $\bar{\mathcal{H}}$ of the NTK's RKHS under the PINN metric. The theory is proven via kernel gradient flow and NTK analysis, with non-asymptotic bounds derived under stronger assumptions. A pathological counterexample demonstrates PINN failure when the solution is not in $\bar{\mathcal{H}}$.

STRENGTHS AND WEAKNESSES:
**Strengths:**  
- Establishes necessary and sufficient conditions for PINN convergence ($u^* \in \bar{\mathcal{H}}_{NTK}$).  
- Provides non-asymptotic error bounds under stronger assumptions.  
- Pathological counterexample reveals fundamental PINN limitations.  

**Weaknesses:**  
- Verifying $u^* \in \bar{\mathcal{H}}_{NTK}$ is intractable (requires NTK eigenfunctions).  
- The experiments are only on a small scale.
- Theory only applies to "infinitely wide networks" (NTK regime).

QUESTIONS:
1. In Corollary 3.11, is there a typo in $\hat{u}_{\theta(T)}$? I think it should be $\hat{u}_T$.

2. Is it possible to make Lemma 5.6 more quantitative?

LIMITATIONS:
The limitations are clearly discussed in Section 5.1 and Section 6.
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-06-23

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper investigates the consistency (convergence property) requirement of PINN when applying to second order elliptic PDEs. Notably $f$ is not required to be in $\\mathcal{H}$, but rather in the closure of $\\mathcal{H}$ in $H_2(\\Omega)$, which is also the necessary and sufficient condition for consistency. Based on the condition, the authors provide a notable pathological example where $f \\notin \\mathcal{H}$, which makes PINN inconsistent. It is able to deliver a non-asymptotic convergence bound if $f \\in \\mathcal{H}$.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The authors introduce notion of consistency to area of PINN, which is a weaker notion compared to previous works which establishes non-asymptotic bounds. They also derive necessary and sufficient condition, which is novel and interesting.\n- The proofs do not require on standard capacity and source condition, which is standard in previous literature. This is pretty novel in theoretical sense.\n- The negative example is quite intriguing and help understanding the properties of PINN.\n\nWeaknesses:\n- The analysis is very restrictive since it is only conducted on the second order elliptic PDE.\n\nQUESTIONS:\n- In line 47, it is mentioned that \"we provide a different perspective, a new mathematics framework known as NTK ...\", actually using NTK to understand theoretical properties of PINN is not new and has been conducted in many works.\n- It is mentioned that $f$ does not need to be in $\\mathcal{H}$, but only needs to be in its closure w.r.t. $\\mathcal{H}^2(\\Omega)$, known as $\\overline{\\mathcal{H}}$, so what are the elements that are in $\\overline{\\mathcal{H}}$, but not in $\\mathcal{H}$? Are there more negative examples can be introduced in section 5.2?\n- For the positive example, the Poisson equation is provided which satisfies the codiagonalization assumption, does the theoretical analysis in this paper requires this assumption too?\n- Is there any intuition why the theoretical analysis can get rid of the standard capacity and source condition? What aspects make it distinguish from previous work? \n- Is there a regime where the equation is \"consistent\" to solve, but the convergence speed is very low which makes it unsolvable in practice?\n- What happens if we conduct analysis on first-order or higher-order PDEs? Some discussions on the necessity of conducting analysis on second-order elliptic PDEs would be beneficial. Can we bring the similar insights to other PDE problems?\n\nLIMITATIONS:\nSee weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates the consistency of Physics-Informed Neural Networks (PINNs) applied to second-order elliptic partial differential equations (PDEs) with Dirichlet boundary conditions. The authors provide a rigorous theoretical framework built upon the Neural Tangent Kernel (NTK) theory to analyze the training dynamics and convergence of PINNs. The primary contributions of this paper include:\n\n1. The derivation of necessary and sufficient conditions for the consistency of PINNs under the assumption that the PDE solution lies in a certain Hilbert space $\\overline{H}$.\n\n2. A non-asymptotic bound on the loss of PINN estimators under stronger assumptions (i.e., the solution lies in the RKHS of the NTK kernel).\n\n3. The construction of a counter example where the PINN method fails to achieve consistency.\n\n\nWhile the theoretical results are correct and address important aspects of PINNs, they are limited to the analysis of single-hidden-layer networks. Extending these results to deep neural networks (DNNs) is critical for practical relevance, as DNNs are the dominant architecture in real-world PINN applications.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The paper provides a mathematically rigorous framework for analyzing the consistency of PINNs.\n\n2. The use of NTK theory to connect the training dynamics of wide neural networks with kernel methods is a theoretical contribution.\n\n3. The construction of a counter example for the potential failure modes of PINNs and emphasizes the importance of theoretical analysis before applying PINNs to real-world problems.\n\n\nWeaknesses：\n1. Limited Practical Relevance. The analysis is restricted to single-hidden-layer networks, which are rarely used in practice. Modern PINN applications predominantly rely on deep neural networks (DNNs), and the results presented here may not directly extend to such architectures.\n\n2. Strong Assumptions. The results rely on the assumption that the PDE solution lies in a specific Hilbert space $\\overline{H}$ or RKHS, which may not hold in many practical scenarios. These assumptions should be carefully validated or relaxed for broader applicability.\n\nQUESTIONS:\nThe analysis is for two-layer NNs, which are rarely used in practice. Please consider the analysis on deep neural networks.\n\nLIMITATIONS:\nThe results rely on the assumption that the PDE solution lies in a specific Hilbert space $\\overline{H}$ or RKHS, is it possible to extend the results in Sobolev spaces?", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-05"}