Title: Neural Superposition Networks
Venue: NeurIPS 2025
Accepted: no
Keywords: differential equations

physics-informed neural networks

scientific machine learning

differentially constrained architecture

principle of superposition
Abstract: We introduce _Neural Superposition Networks_, a class of physics-constrained neural architectures that exactly satisfy given partial differential equations (PDEs) by construction. In contrast to traditional physics-informed neural networks (PINNs), which enforce PDE constraints via loss regularization, our approach embeds the solution manifold directly into the architecture by expressing the output as a superposition of analytical basis functions that solve the target PDE. This eliminates the need for interior residual loss terms, simplifies training to a single-objective optimization on boundary conditions, and improves numerical stability. 
We show that for linear PDEs—including Laplace, heat, and incompressible flow constraints—this architectural bias leads to provably convergent approximations. Using maximum principles and classical convergence theory, we establish uniform boundary-to-interior convergence guarantees. For nonlinear PDEs such as Burgers’ equation, we demonstrate that partial structural constraints can still be enforced via transformations (e.g., Cole–Hopf), yielding improved inductive bias over standard PINNs. The resulting networks combine the expressiveness of deep learning with the convergence guarantees of Galerkin and spectral methods. Our framework offers a theoretically grounded and computationally efficient alternative to residual-based training for PDE-constrained problems..


Peer reviews:
comments: SUMMARY:
This paper introduces a new class of physics-constrained neural networks to solve PDEs, termed Neural Superposition Networks (NSN). NSN constructs solutions as superpositions of specially designed basis functions that solve the target PDE. NSN achieves comparable accuracy to the existing physics-informed machine learning methods and is supported by theoretical convergence guarantees.

STRENGTHS AND WEAKNESSES:
Strengths: 
- The proposed NSN framework simplifies the training process by reducing a multi-objective formulation to a single objective defined solely on the boundary
- NSN is grounded in well-established theoretical foundations from PDEs and Numerics.

Weaknesses:
- NSN is limited to linear PDEs due to its reliance on the superposition principle. While the authors attempt to address this with an example involving the Burgers equation, such extensions require special treatment, limiting the broader applicability of the approach.
- The main numerical results in Table 1 indicate that NSN underperforms compared to existing methods in several benchmark problems. 
- The experimental evaluation is restricted to relatively simple PDEs in low-dimensional settings, raising concerns about generalizability to more complex settings, including complex domains, high-dimensional problems.

QUESTIONS:
- Can NSNs be extended to general nonlinear PDEs?
- The accuracy of NSN appears to depend heavily on the choice and number of basis functions. How were these basis functions selected in practice? and how were associated hyperparameters tuned? 
- It would be beneficial to include a comparison of computational time and efficiency between NSN and exsiting physics-informed methods.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 2
IMPACT: 1
ORIGINALITY: 2
DATE: 2025-06-24

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focusses on the application of neural networks for solving PDEs. It follows the research thread which aims to hard-wire certain properties into the architectures of the network. The authors of this paper introduce a novel approach \"Neural Superposition Networks\", which avoids the incorporation of residual loss terms, simplifies the training, and also improves robustness. The key novel idea of this approach is to write the solution as a combination --- hence superposition --- of basis functions from PDE theory, which is then trained. Several PDEs are provided to which the approach can be applied to includingLaplace equation, Heat equation, Burger's equation, etc. The theoretical analysis includes a convergence analysis and the classical question about a maximum principle. Numerical experiences for different classes of PDEs show the effectiveness of their approach.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n* This paper introduces a novel, original approach for solving PDEs.\n* The approach is based on an intrguing idea, namely to embed the solution manifold directly in the sense of superpositioning the solution with PDE-consistent basis functions. \n* The theoretical analysis is comprehensive, and typical for numerical analysis of PDEs.\n* The numerical experiments show convincingly the effectiveness of this approach. \n* The authors consider a wide range of PDEs for their analysis, which are some of the classical ones used in applications. \n\nWeaknesses: \n* A study of sensitivity with respect to noise in the sense of robustness of the approach would added to the value of the approach. \n* The theoretical analysis is limited to linear differential operators.\n\nQUESTIONS:\n* What is the main problem when extending the theoretical analysis beyond the linear case? Do you have ideas how also the non-linear situation can be treated? \n* Why does your approach not always exceed the status quo as you correctly mention in the limitations? \n* Are there PDEs which can not be solved with classical approaches, and which you can now solve for the very first time with a feasible solver? This would the make a very strong case.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nAuthors propose Neural Superposition Network (NSN) for solving linear PDEs, where instead of the residual loss in Physics-informed Neural Networks (PINNs), the PDE constraint is strictly imposed by a linear layer that combines the known (or learnable) solution basis functions and symmetries of the PDE. Authors show that this layer satisfies the PDE by design, and the training of NSN is thus reduced to optimizing for boundary conditions only. The proposed method is simple and easy to understand, and addresses the common training challenges with PINNs, but its practical applicability to nonlinear and complex problems is not clear to me.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n- The proposed method clearly addresses the optimization challenges with PINNs (at least for linear PDEs) with a simple and easy-to-understand approach. \n- While NSN is mainly applicable to linear PDEs, the authors further study the nonlinear Burgers' equation and Navier-Stokes through transformations and a hybrid approach, respectively. \n### Weaknesses\n- While authors extend NSNs to some non-linear cases, I doubt their practical applicability for more complex systems. NSNs require a complete knowledge of the physical systems, symmetries, and the basis solutions, which is simply not possible for many non-linear systems. Also, the hybrid approach used for NS is clearly not improving any of the baselines as reported in the paper.\nWhile I appreciate that the authors point out this limitation, I believe neither the experimental nor the theoretical results in this paper are significant enough at this stage.\n- The manuscript skips some of the experimental details that negatively impact the readability and clarity. For instance, the difference between \"Heat 1\" and \"Heat 2\" experiments in Table 1 is only pointed out in the appendix. Also, the baselines are similarly defined in the appendix, while their abbreviations are used in Table 1.\n- The authors mentioned the possibility of using \"learnable\" basis functions instead of analytical ones in NSN. However, it is not clear which experiments, if any, utilize learnable basis functions and how they compare to the analytical ones. An ablation study of these two types of bases would help provide a more complete picture of the method.\n\nQUESTIONS:\n1. Do any of the experiments utilize learnable basis functions? How do the results compare to analytic bases?\n2. It is not clear to me how the auxiliary MLP for Navier-Stokes is trained. Is a regular PINN residual loss used for training?\n3. As NSN is meant to address the training challenges with PINNs, I am wondering what its training curves look like when compared to PINNs. Would it be possible for the authors to provide this information (loss and error training curves)?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a new method for solving PDEs, which hard-encodes the superposition into the neural networks. It aims to address the challenging optimization problems in physics-informed neural networks (PINNs). This paper has considered several linear PDEs and non-linear PDEs as benchmark cases. The empirical results have shown the superiority of the proposed method compared to PINNs and their variants.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n- This paper introduces an interesting idea by embedding superposition into neural networks. Such a hard-encoding method is less explored compared to soft-constraint style PINN methods. \n\n- The theoretical analysis of this paper is well-presented and makes this paper more convincing. \n\n- Although this method should be more applicable to linear PDEs, the authors adapt it to certain nonlinear PDEs with specific transformations.\n\n\n**Weaknesses:**\n\n- My first concern is the applicability of the proposed method to more general PDE settings. This method might be challenging to be applied to many other nonlinear PDEs.\n\n- The current baseline models are kind of limited. They mainly focus on the soft-constrained PINNs. It would be better to also include comparisons with hard-constrained PINN approaches [1,2], which enforce physical constraints more strictly and may offer different performance characteristics. Also, data-driven methods such as Fourier Neural Operators (FNO) [3] could serve as a baseline, given their strong performance in learning dynamics from PDE data.\n\n- The evaluation metrics could be more extensive. The current metric only focuses on RMSE, which might not be sufficient. The relative L2 norm errors and some domain-specific error metrics are also commonly used. The authors may refer to this benchmark paper [4].  \n\n---\n\n**Refs:**\n\n[1] Lu, Lu, et al. \"Physics-informed neural networks with hard constraints for inverse design.\" SIAM Journal on Scientific Computing 43.6 (2021): B1105-B1132.\n\n[2] Cheng, Xiaoran, and Sen Na. \"Physics-Informed Neural Networks with Trust-Region Sequential Quadratic Programming.\" arXiv preprint arXiv:2409.10777 (2024).\n\n[3] Li, Zongyi, et al. \"Fourier neural operator for parametric partial differential equations.\" arXiv preprint arXiv:2010.08895 (2020).\n\n[4] Hao, Zhongkai, et al. \"Pinnacle: A comprehensive benchmark of physics-informed neural networks for solving pdes.\" arXiv preprint arXiv:2306.08827 (2023).\n\nQUESTIONS:\n- How does the computational complexity of the proposed method compare to that of the baseline models? A discussion on training and inference efficiency would help assess the method’s practicality.\n\n- Could the authors discuss the applicability of the proposed method to more complex or noisy real-world systems?\n\n- Does the data resolution (within a regular or irregular mesh) affect the performance of the proposed method, since the PINN method is mesh-free? A discussion on this part would be appreciated.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}