CoPINN: Cognitive Physics-Informed Neural Networks
Authors: Siyuan Duan
Wenyuan Wu
Peng Hu
Zhenwen Ren
Dezhong Peng
Yuan Sun
Venue: ICML 2025 spotlightposter (2025)
Keywords: Physical informed neural network
Self-paced learning
Separable learning.
Primary area: deep_learning->other_representation_learning
TL;DR: This paper introduces a framework called CoPINN, which effectively addresses the Unbalanced Prediction Problem in Physics-Informed Neural Networks for solving Partial Differential Equations.

Abstract: Physics-informed neural networks (PINNs) aim to constrain the outputs and gradients of deep learning models to satisfy specified governing physics equations, which have demonstrated significant potential for solving partial differential equations (PDEs). Although existing PINN methods have achieved pleasing performance, they always treat both easy and hard sample points indiscriminately, especially ones in the physical boundaries. This easily causes the PINN model to fall into undesirable local minima and unstable learning, thereby resulting in an Unbalanced Prediction Problem (UPP). To deal with this daunting problem, we propose a novel framework named Cognitive Physical Informed Neural Network (CoPINN) that imitates the human cognitive learning manner from easy to hard. Specifically, we first employ separable subnetworks to encode independent one-dimensional coordinates and apply an aggregation scheme to generate multi-dimensional predicted physical variables. Then, during the training phase, we dynamically evaluate the difficulty of each sample according to the gradient of the PDE residuals. Finally, we propose a cognitive training scheduler to progressively optimize the entire sampling regions from easy to hard, thereby embracing robustness and generalization against predicting physical boundary regions. Extensive experiments demonstrate that our CoPINN achieves state-of-the-art performance, particularly significantly reducing prediction errors in stubborn regions.

CoPINN:
Cognitive Physics-Informee Neural Networks
Siyuan Duan Wenyuan Wu
Peng Hu
Zhenwen Ren
Dezhong Peng 13
Yuan Sun 13
Abstract
ym-11
y=O
Physics-informen neural networks (PINN) aim to constrain the outputs and gradients of deep learn- ing models to satisfy specified governing physics equations, which have demonstrated significant potential for solving partial differential equations (PDEs). Although existing PINN methods have achieved pleasing performance, they always treat both easy and hard sample points indiscriminately. especially ones in the physical boundaries. This easily causes the PINN model to fall into undesir- able local minima and unstable learning, thereby resulting in Unbalanced Prediction Problem (UPP). To deal with this daunting problem, we propose a novel framework named Cognitive Informed Neural Networks (CoPINN) that imitates the human cognitive learning man- ner from easy to hard. Specifically, we first em- ploy separable subnetworks to encode indepen- dent onc-dimensional coordinates and apply an ag- gregation scheme to generate multi-dimennsinal predicted physical variables. Then, during the training phase, we dynamically evaluate the diffi- culty each sample according to the gradient of the PDE residuals. Finally, we propose a cogni- tive training scheduler to progressively optimize the entire sampling regions from easy to hard, thereby embracing robustness and generalization against predicting physical boundary regions. Ex- tensive experiments demonstrate that our CoPINN achieves state-of-the-ar performance, particularly significantly reducing prediction errors in stub- born regions. The code available at this reposi- tory: https///gghhub.oooom/yynnnn/cc/e
SPINN
(a)
yu-1
y=0
3000
5080
CoPINN (Ours)
(b)
Figure I. The 2D and 3D visualization of the absolute error be- tween the predicted and exact values. The left graph illustrates the absolute of the entire three-dimensional space. The mid- dle graph demonstrates the absolute error of the boundary when y -1, while the right graph displays the absolute error of the cross-section when (SOTA) (Cho et al., 2023) on the Helmholtz equation. These results indicate that SPINN exhibits significantly larger errors near the physical boundary region compared to the middle region, which reveals the Unbalanced Prediction Prohlem (UPP). (b) The abso- lute error of our CoPINN on the Helmholtz equation, which shows that CoPINN maintains consistent small absolute errors both near the physical boundary and in the middle region.
0. (a) The absolute error of SPINN
1. Introduction
As a type of equation in mathematics that describes the re- lationship between functions of multiple variables (Evans, 2022), partial differential equations (PDEs) usually involve partial derivatives of unknown functions and describe the laws of change of natural phenomena in disciplines such as physics, chemistry, and economies. In recent years, solving PDEs has wide range of applications in many fields of science and engineering (Roubíček, 2013; Li et al., 2025), which are crucial for understanding and uncovering the underlying physical laws. Therefore, a large number of tra- ditional numerical methods have been developed to solve PDEs, such as finite element methods (Jagota et al., 2013), finite difference methods (Thomas, 2013), and finite volume methods (Barth et al., 2018). Unfortunately, these methods are resource-intensive and mesh-dependent, thus often re- quiring huge time cost to solve such complex systems, for example, the Navier-Stoker equations (Zou et al., 2024a). Thanks to the powerful nonlinear representation capabilities of deep learning, they are also widely used solve partial
Equull contribution 'College of Computer Science, Sichuan University, Chengdu, China. SSothwwest University of Science and Technology. Mianyang, China. National Key Laboratory of Fundamental Algorithms and Models for Engineering Numerical Simulation, Sichuan University, Chengdu, China. Correspondence to: Yuan Sun <sunyuan_word w 163.com>.
Proceedings of the = International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

CoPINN: Cognitive Physics-Informit Neural Networks
differential equations. As a mainstream solution method, physics-informed neural networks (PINN) (Raissi et al., 2019; Long et al., 2018; Kim et al., 2021; Wright et al., 2022) have become a promising and effective alternative to numerical methods, which aim to embed physical prior knowledge into neural networks to improve flow field pre- diction capability.
to such a gradual learning paradigm, SPL can enhance the generalization ability according to the difficulty differences of these samples. However, although SPL has been explored in numerous areas such as clustering (Zhou et al., 2023; Bai et al., 2024), classification (Yuan et al., 2024; Chen et al., 2024), retrieval (Sun et al., 2024a;b; Pu et al., 2025), and domain generalization (Zhao et al., 2024), it has never been touched in the field of PINN The greatest challenge is how to evaluate the difficulty of samples from the physical boundary and ones from the smooth region.
Compared with traditional data-driven methods, PINN can not only enhance the generalization ability of the model but also improve the prediction accuracy of the model in data-scarce environments. Thus. a large number of PINN methods have been proposed, which mainly focus on the four techniques: i.e, neural network architectures, opti- mization schemes, loss re-weighting, and adaptive sampling. The first category focuses on designing the architecture of the neural network part of PINN, such uncertainty es- timation (Yang et al., 2021; Wu et al., 2025) and solving multi-dimensional PDEs (Cho et al., 2023). The second category, from the optimization perspective, designs the loss function for PINN, such as predicting evolutionary equa- tions (Lii et al., 2024) and region optimization (Wu et al., 2024). The third category aims to construct the loss re- weighting strategy achieve optimization of specific loss terms (Wang et al., 2022) training points (Xiang et al., 2022). The last category dynamically adjusts the distribu- tion of training data and gives priority to dense sampling in arcas with large residuals or drastic solution changes (such as boundary layers, shock waves, etc.), thereby improving the accuracy of the model in key areas and the overall train- ing efficiency (Wu et al., 2023; Daw et al., 2023).
To overcome the above challenge, we propose a novel Cognitive Physical Informed Neural Network (CoPINN) that overcomes the problem of difficult optimization of sam- ples in physical boundary regions. As shown in Figure 2, our CoPINN effectively emulates human cognitive learning, beginning with easier regions and progressively advancing to more challenging ones, thereby endowing the model with generalization in difficult regions. To be specific, we first adopt the separable sub-networks to encode independent one-dimensional coordinates instead of using a single MLP for all multi-dimensional coordinates, thereby reducing the computational complexity of solving PDEs. Then, we utilize an aggregation scheme to obtain the multi-dimensional pre- dicted physical variables. Afterward, in the training process, CoPINN dynamically evaluates the difficulty of predicting each sample by the gradient magnitude of the PDE residuals. Finally, we present a cognitive training scheduler to adap- tively optimize the PINN model from easy to hard, thereby endowing the robustness and generalization against pre- dicting physical boundary regions. In summary, the main contributions of this paper are as follows:
Although these methods have achieved promising perfor- mance, they still face some challenges in solving PDEs. To be specific, almost all of them implicitly assume that all training data are equally important while ignoring the differences between samples. In other words, the existing PINN methods always treat both easy and hard samples equally while ignoring the effect of physical boundaries on the learning difficulty of the data. This easily results in unstable learning and falls into poor local minima, thereby leading to an Unbalanced Prediction Problem (UPP). As shown in Figure 1, for the state-of-the-aa art PINN method (i.e., SPINN (Cho et al., 2023) ). we visualize the abso- lute error between the predicted value and the exact value on the Helmholtz equation. According to their prediction performance, we can observe that the error values in stub- born regions (such as physical boundaries) are much larger than those in the middle smooth regions. This shows that the biggest challenges in solving PDEs are mainly in hard physical boundary regions, rather than easy smooth regions.
We reveal and study an untouched yet pervasive signif- icant problem in PINN, termed the unbalanced predic- tion problem (UPP). Unlike previous PINN methods that always treat both easy and hard samples equally, we propose novel cognitive PINN framework to alle- viate the negative effect of the hard samples in stubborn regions during the learning process.
We propose PINN model with SPL that measures the sample-level difficulty and promotes the neural network to fit samples from easy to hard in solving PDEs. To the best of our knowledge, our CoPINN could be the first work that leverages SPL to enhance the PINN performance in difficult regions.
We conduct series of experiments on multiple widely used PDE equations. These numerical results demon- strate that the proposed CoPINN can consistently out- perform seven sfate-of-the-art PINN methods by a con- siderable margin under different numbers of colloca- tion points.
Inspired by the human cognitive learning process, self-paced learning (SPL) (Jiang et al., 2015) was proposed to train these samples in the training set from easy to hard. Thanks
2

---

CoPINN: Cognitive Physics-Informen Neural Networks
Sample points
Separable learning
Prediction
Original loss
Ground truth
Lie
Total loss
Merge
Lbe
X:
y:
Spde
pdde
2:
Sample points weights
Difficulty measure
Easy
Hard
Cognitive training scheduler
Sample points difficulty 1
85
9,5
85
Sample points weights
Sample points weights
Sample points weights
Epoch
Epoch 25000
Epoch
Figure The frameworks CoPINN. The separable sub-networks are used encode independent one-dimensional coordinates. Then, aggregation scheme is applied to obtain the multi-dinensional predicted physical variables. During the training process, CoPINN dynamically evaluates the difficulty of predicting each sample based on the gradient magnitude the PDE residuals. Finally, cognitive training scheduler is employed to adaptively optimize the model from casy to hard.
2. Method
2.2. Separable Learning
2.1. Notations and Motivation
Traditional PINN usually utilizes an MLP architecture based spatiotemporal coordinates represent the solution func- tion, where the input corresponds to spatiotemporal coor- dinates and the output represents the associated solution quantity. For each training point, computing the PDE resid- ual loss necessitates multiple forward and backward prop- agations. As the number of training points (collocation points) increases, particularly for higher-dimensions or more complex solutions, the computational burden escalates. To address this, follow SPINN (Cho et al., 2023) and adopt separable architecture to handle high-dimensional PDEs efficiently. Specifically, we construct d independent sub-networks (MLPs), each of which takes a single one- dimensional coordinate as input. Each sub-network (() is a vector-valued function that maps the i-th coordinate to an r-dimensional feature representation. Afterward, we aggregate these feature representations obtain the final prediction as follows,
Without loss of generality, we denote the general partial differential equation (PDE) as the following formula, i,e.,
Ui = Nu),, x E u(x, 0) (o()), x Ω
(L)
u(x,t)= = 9(z,,))=(zzt))) t). 82, tET.
where u(x, t) denotes the hidden solution, t and x represent temporal and spatial coordinates respectively, Q represents an open, bounded domain with smooth boundary 02, rep- resents time domain, denotes a differential operator, and g(x, is known boundary condition function. Our goal is to develop PINN method to obtain approximate solutions with reasonable accuracy for Equation (1) through neural network fitting. Although existing PINN methods achieve promising solving performance. all of them overlook the effect of physical boundaries on the learning difficulty of data, which could easily lead to unstable learning and get stuck poor local minimum. Therefore, this inevitably leads to the Unbalanced Prediction Problem (UPP) for solv- ing PDEs, which is shown in Figure 1. To address this, we propose novel Cognitive Physics-Informed Neural Net- works (CoPINN). In this paper, we first introduce separable
Ú( T2 Fa)
ΣI ==
00 (x:)
(2)
where à is the predicted solution function, 2; E R is coordinate of i-th axis, and (10) represents the j-th element learning Section 2.2, followed by description our learning objective in Section 2.3, and conclude with the cognitive learning scheduler in Section 2.4 and Section 2.5.
of f(A), In practice, the input coordinates are provided in batches during both training and inference. To this end, we extend Equation (2) allow the neural network to process
3

---

CoPINN: Cognitive Physics-Informen Neural Networks
mini-batches for efficient training. Specifically, let N input coordinates, i.e., training points, be sampled from each axis. The sampling resolutions for different axes need not be identical. The input coordinates X E IR RNxd form a matrix. Consequently, we could obtain the batchified feature representation F' E R RNxxxdd Further, Equation (2) could be extended as the following formula, i.e.,
Previous PINN methods often implicitly assume that all training data is equally important while ignoring their dif- ferences in learning difficulty. It is unreasonable, which has been proven in Figure 1. To tackle this UPP, we present a CoPINN method to predict the solution of PDE, which imitates human cognitive learning to gradually incorporate more challenging samples from easy to hard during the process of training the PDE loss. Therefore. the learning objective of CoPINN can be formulated as follows:
Ú(X,1,X X..)=
(3)
1=1
min ((a = min (Apii S.Jo u. ". 111 [utor] (z, deddl dxdi
where Ó' E IR.NXXX...... is the discretized solution vector, X represents outer product, F.... E NNxr denotes the i-th frontal slice matrix of vector F, and F. E IN is the j-th column of the matrix F...i- Note that, because each coordinate axis is encoded separately by this scheme, compared with the traditional multi-dimensional coordinate encoding, the memory and time consumption of calculation are greatly reduced.
+ Xic Sa (x, 0) mic(x))l2dr
+ Abc S.lo 118 [ato]] (x, 1) - uso t)ll dxdt
(5)
where on E [0, 1] represents the weight assigned to sam- plc point xs, which dynamically changes as the training proceeds.
2.3. Problem Formulation
2.4. Difficulty Evaluation
In classical PINN (Raissi et al., 2019), deep neural net- work is trained to approximate the solution of PDE. The idea behind PINN is to embed the underlying physics infor- mation (i.c., PDEs) directly into the loss function, alongside traditional data-driven loss terms like initial/boundary condi- tions and data points. Mathematically, the objective function L(ū²) of the vanilla PINN could be represented as
In traditional SPL (Wang et al.. 2021b; Soviany et al., 2022: Shrivastava et al., 2016), difficulty-leve is often based on predefined heuristics, like the training loss certain statisti- cal properties of the data, which might not fully capture the complexities of the solution space in the context of PINN. Utilizing just the training loss (disparity between predictions and ground truth) could not provide a nuanced enough mea- sure for difficult regions of the solution space, especially in PDEs that exhibit phenomena like shock waves, singu- larities, or boundary layers, which exhibit rapid solution changes. These regions are physically important, thus, we (4) should handle them with greater care during training. To this end, we regard the gradient of the PDE residual as a measure of sample difficulty. By measuring the spatial or temporal gradient of the residual, we can identify where these phe- nomena occur. Areas with high gradients are indicative of regions where the solution changes rapidly, meaning the network might need additional effort to approximate the solution. In addition, flatter regions (low gradient) suggest simpler behaviors where the network could focus less on achieving high accuracy. To be specific, we dynamically adjust the difficulty measure based on the current state of the model, thereby enabling SPL be much more flexible and responsive. Instead of relying on static, predefined difficulty metric, the model can continuously adapt the focus during training, emphasizing the more challenging parts of the solution. To be specific, the difficulty of k-th sample in i-th epoch could be expressed as
() = Spde + DicLic + AbcBbe,
///0 / 1W [200] (sr, 1112 do dl,
Cpde
J
Lie S |1 (x, (x,0) 0) - u=c(=)]1? dx,
Lbc [ J |18 [1)0)] (x) - (nn 1)1 dx dt,
where (()) denotes the predicted solution function, rep- resents space coordinates, / represents time coordinates Ω represents an input domain, I represents time domain, N and B denote generic differential operators, and uhe and Uhe are initial, boundary conditions, respectively. X, are balancing factors for each loss term.
In Equation (4), the loss function in PINN often includes terms that ensure the neural network satisfies the initial conditions and boundary conditions of the physical prob- lem being modeled. The network's 's output at every point must satisfy the physics (through the PDE residual) and conform to the known physical constraints (through initial and boundary conditions). Since there are some mutations in the samples of the physical boundaries, this undoubtedly increases the difficulty of learning these stubborn points.
OLi
Dj.
pde
(6)
Ox1
4

---

CoPINN: Cognitive Physics-Informet Neural Networks
2.5. Cognitive Training Scheduler
Once of and vf, are calculated, the weight of the j-th most easy sample in the i-th epoch can be calculated as follows:
As shown in Figure (a), samples from regions with smooth variations in physical quantities are typically easier to handle by current PINN methods. Overemphasizing these samples while neglecting those from more challenging regions can impede the model's ability to accurately fit regions with abrupt changes in physical quantities. Effectively learning from regions with significant variations in physical vari- ables is crucial for accurately solving partial differential equations (PDEs). To address this challenge, after identi- fying where physical phenomena occur, such shocks or singularities, through a difficulty evaluation, we propose a cognitive training scheduler to gradually pay more attention to those challenging regions. Early in training, when the net- work has less capability to fit difficult regions, giving it more weight on easy regions helps the model warm up by learning the basic structure of the solution. As training continues, the network becomes more capable, and the scheduler directs the focus to harder regions. This is essential to prevent the model from overfitting to the simple regions and underfit- ting the complex, critical regions. This dynamic scheduling enables the model to learn progressively, ensuring that it's exposed to both simple and complex regions of the solution, which is vital for accurate and robust generalization.
Dj D D D
5j = (v) -06))))
(10)
where DJ denotes the difficulty measure of j-th most easy sample, D`S represents the difficulty measure of the easiest sample, and Dj represents the difficulty measure of the hardest sample, in i-th epoch. The difficulty value is in the range [o, +oo]. Afterward, we can calculate the weight ry for each sample in the i-th epoch, where j signifies the sample ranked j-th in terms of difficulty measure. could be represented as
o/ -/0 -Te Tee. (1) - - 3,,,
(11)
where 33 is hyperparameter to adjust the weight variety for each sample in each epoch.
Note that 63 can be computed either globally or locally, When computed globally, 6j is computed over the entire dataset at once for each epoch. This approach assigns unique weights to samples based on their difficulty rank across the entire dataset, thereby providing precise weight calculation. However, computing 8j globally can be computationally expensive as it necessitates storing the difficulty measure for each sample until the end of an epoch, limiting its scalability for large datasets. To address this limitation, we suggest computing 8ÿ locally, which means computing it in one batch size.
To this end, we dynamically adjust the weights assigned to easy and difficult samples. As illustrated in Figure 2, during the initial epoch, easier samples (c.g.. those in the middle region) are assigned higher weights compared to more diffi- cult samples (e.g.. those in boundary regions). Throughout the training process, the weight difference between easy and difficult samples gradually diminishes. By the midpoint of training, the weights for both easy and difficult samples equalize. Subsequently, the weights for difficult samples incrementally exceed those for easy samples. To be specific, during the training process, from the first epoch to the final one, the weight assigned to the easiest samples decreases from one to zero. For a training phase with N₂ epochs, we expect the weight of the easiest sample to be 1 in the first epoch and 0 in No epochs. Then, for the easiest sample, the magnitude of the weight modifications in each epoch can be calculated as follows:
2.6. Weight Analysis
To further analyze the proposed cognitive training scheduler, we plot the weight trends for different values of 33 in Fig- ure 3. The results clearly show that the weight of the easiest samples decreases as training progresses and the number of iterations increases. Moreover, from Figure 3 (a), (b), to (c), it can be observed that as 3 decreases, the weight of the more difficult samples gradually increases. If there is only a single loss term, the approach illustrated in Figure 3 (a) is often more efficient. This is because it initially focuses on learning easy samples, progressively incorporating difficult samples as the number of iterations increases, thereby even- tually covering all samples. However, our method includes three loss terms: Cpde. Lie, and Lie- Adopting the self- paced learning strategy shown in Figure 3 (a) may cause the neural network to forget the easy samples learned in the early stages and shift its focus entirely to the difficult samples. To mitigate this issue, we recommend setting 3 to (9) less than 0.5. A detailed parameter analysis of this choice is provided in Section 3.3.
No N.
U
1
N:
where og and vac are the weights assigned to the easiest sample in the first and final epoch, respectively, with v/ = 1 and v... = 0. Therefore, the weights assigned to the easiest and hardest samples in epoch i, denoted as of and of. are computed as follows:
-f = vz - Te (i (i- 1) =1 i-1
(8)
N₂ v = wt T (i 1) = 0 +
i 1
N:
5

---

CoPINN: Cognitive Physics-Informed Neural Networks
Table Full results of the Helmholtz, (2+1)-d Klrion and Diffusion equation. Ne is the numher of collocation points. Due to the increase in training points, some methods are out-of -e-oo-r (O/M). The best and the second-bes best results are highlighted in boldface and underlined. respectively.
Evaluation Metric
RL2
RMSE
Equation
Methods
N
16
322
611
128
256
16a
32
614
128
256
Ref.
ICP'2019
0,9819
0.9757
0.9723
QM
0.3420
0.3198
OM
@PINN
CMAME 2022
0.3852
0.3255
0.4008
O/
0/M
0.1342
0.1133
0.1396
0/M
O/M
AHD-PINN
1ICAF
0.2108
0.1903
0.1871
QM
0/M
0.0734
0.0663
0.0652
Q/M
O/M
Helmholtz
SPINN
Ne02
0.1177
0.0809
0.0592
0.0449
0.0435
0,0410
0.0282
0.0206
0.0156
0.0151
SPINN (m)
NeurIPS'2023
1.1
0.0595
0.0360
0.0300
0.0311
0,0404
0.0307
0.0125
0.0104
0.0108
RoPINN
NeurlPS 2024
0.4059
0.3338
O/M
0/M
0/M
0.1414
0.1162
O/M
0/M
O/M
EPINN
NN
0,3862
0,3502
0.3097
0/M
O/M
0.1345
0.1220
0.1079
O/M
Q/M
CoPINN
Ours
0.0172
0,0050
0,0016
0.0007
0.0006
0.0040
0.0015
0,0004
0.0002
0,0002
92%
96%
98%
98%
90%
93%
97%
98%
PINN
2019
0.0343
0.0281
0.0299
O/M
O/M
0.0218
0.0178
0.0190
O/M
O/M
(2+1ed Seiin-Gordon
gPINN
CMAME 2022
0.0108
0.0025
O/M
O/
O/M
0.0069
0.0016
O/M
O/M
O/M
SPINN
NeurIPS'200 '023
0.0193
0.0060
0.0045
0.0040
0.0039
0.0123
0.0038
0.0029
0.0025
0.0025
SPINN (m)
NeurIPS'208
0.0062
0.0020
0.0013
0.0008
0,0009
0.0039
0.0013
0.0008
0.0005
0.0006
AHD-PINN
20C4
0.0133
0.0082
0.0147
O/M
O/M
0.0084
0.0052
0.0093
O/M
O/M
RoPINN
NeurIPS'200 2024
0.1925
0,1806
0,1740
O/M
O/M
0.1223
0.1147
0.1105
O/M
O/M
FPINN
NN25
0.0331
0.0213
O/M
OM
0/M
0,0210
0.0136
O/M
Q/M
O/M
0.0016
0.00066
0,0004
0.0003
0.0002
0.0010
0.0005
0.0002
0.0002
0.0002
IMP.
74%
70%
69%
63%
78%
74%
62%
62%
60%
67%
PINN
1C19
0,0129
O/M
O/M
O/M
O/M
0.0096
O/M
O/M
O/M
O/M
(3+))dd Klein -oodGGodon
gPINN
CMAME'200 2022
0.0144
O/M
OM
OIM
0.0107
O/M
O/M
SPINN
NeurlPS 2023
0.0151
0.0086
0.0073
O/
O/M
0,0112
0.0064
0.0054
O/M
O/M
SPINN (m)
NeurlPS 2023
0.0078
0.0065
0.0077
O/M
O/M
0.0058
0.0048
0.0057
O/M
Q/M
AHD-PINN
UCAA
0,0109
0,0107
O/M
O/M
O/M
0,0081
0.0080
O/M
O/M
O/M
NeurIPS'2U24
O/M
O/M
0,0100
O/M
O/M
FPINN
NN22025
0.0152
O/M
OIM
O/M
O/M
0.0113
O/M
O/M
O/M
O/M
CoPINN
Ours
0.0041
0.0027
0.0017
O/M
O/M
0.0031
0.0019
0.0012
O/M
O/M
IMP.
47%
58%
78%
47%
60%
79%
JCP'2019
0.0082
0.0081
0.00082
0,00071
0.00070
CMM
O/M
gPINN
CMAME
0.0099
0.0087
0.0090
O/M
O/M
0.00086
0.00076
0.00078
O/M
O/M
SPINN
NeurlPS'2U8 2023
0.0447
0.0115
0.0075
0.0061
0.0061
0.00387
0,00100
0.00065
0.00053
0.00053
Diruusin
SPINN (m))
NeurIPS/2023
0.0390
0.0067
0.0041
0.0036
0.0036
0,00338
0,00058
0.00036
0.00031
0.00031
2024
0.0080
0,00069
RoPINN
NeurlPS 2024
0.0412
0.0308
0.0306
O/M
0/M
0.00357
0.00267
0.00265
O/M
O/M
FPINN
NN'2025
0.0058
0.0048
O/M
O/M
O/M
0.00050
0.00042
O/M
O/M
O/M
CoPINN
Ours
0.0058
0.0038
0.0035
0.0033
0.0033
0.000447
0.00033
0.00030
0,00028
0.00028
21%
8%
8%
21%
17%
10%
10%
Hardest
Helmholtz Equation, (2+1)-d Beein-Gordon Equation, and Flow Mixing Problem) and 4D (i.e., (3+1)-d Klein-Gordon Equation) PDE systems. During training, we perform all experiments on different numbers of collocation points Ne. i.c., 163. 323, 643. 1283, and 2563. Due space limita- tions, the experimental results of 1D Convection Equation and (2+1)-d Flow Mixing Problem are shown in the Ap- pendix C.4 and Appendix C.5.
Handect
Baskest
1999
Eatchs
EpocTs
seice
Esochs
(a) β
(b) 3 0.5
(c) 0.1
Figure 3. Different B values in cognitive training scheduler result a range of changes in weight
Baselines. To verify the effectiveness of our method, we compare the proposed CoPINN with seven state-of- the-art PINN methods, including: PINN (Raissi et al., 2019), gPINN (Yu et al., 2022). SPINN (Cho et al., 2023). SPINN(m) (Cho et al., 2023), AHD-PINN (Dashtbayaz et al., 2024), RoPINN (Wu et al., 2024), and FPINN (Wu et al., 2025).
3. Experiments
3.1. Experimental Setup
Datasets. To show the performance of solving PDEs, we carry out experiments on six popular public datasets, includ- ing 1D Convection Equation, 3D (i.e., Diffusion Equation,
6

---

CoPINN: Cognitive Physics-Informer Neural Networks
Exact solution
Prediction
Absolute
Exact solution
Prediction
Absolute
PINN (0)
(c) FPINN
6.
ANLL (9)
WUBB (
20000
28 8.6
AHD-PINN
SPINN B
(
RoPINN
(b) CaPINN (Ours)
Figure 4. Prediction results of CoPINN and the seven baselines the Helmholtz dataset with N. =22. The exact solution, prediction, absolute error are shown on the left, middle, and right, respectively.
Evaluation Metries. To comprehensively evaluate CoPINN, employ two evaluation metrics: Relative La error (RL2) and Root Mean Square Error (RMSE). To highlight the advantages of our method, we report the per- formance improvement (IMP.).
representing the entire 3D function and completes train- ing.
Under any number of collocation points, CoPINN consistently achieves the best RL2 and RMSE met- ries across all datasets. It is worth noting that on the Helmholtz equation with N = 2563, CoPINN achieves an RL2 of just 0.0006. This represents a 98% improvement compared to the second best method, i.e., SPINN(m). These results validate the effectiveness of easy-to- cognitive training scheduler.
More details of datasets, baselines, evaluation metrics, and the details of implementation summarized in the Appendix B.1, Appendix B.2, Appendix B.3, and Ap- pendix B.4, respectively.
3.2. Comparison with State-of-the-Art Methods
To comprehensively evaluate the performance of our CoPINN, we compare with seven state-of-the-ard baselines. The experimental results on various numbers of collocation points are shown in Table 1, from which we can observe that:
For the Diffusion equation, the RL2 of almost all the comparison methods achieve the order of le-3, This suggests that the Diffusion equation is relatively straightforward to learn. In contrast, on the other three equations (i.e., Helmholtz, (2+ I)-d and (3+1 I)-d Klein- Gordon), the RL2 of most comparison methods typ- ically only reaches the order of 1e-1 or le-2. This discrepancy arises because these three equations fea- ture numerous mutation regions, making them more challenging to learn. Our method learns data from easy to hard through the cognitive training scheduler, which allows the model to gradually optimize, thereby avoiding local optimality and achieving higher IMP for more challenging PDEs. This further illustrates the effectiveness of CoPINN learning hard PDEs.
As the number of collocation points increases, the R.2 and RMSE metrics of most methods gradually de- crease. From these results, we can improve the predic- tion precision by increasing the number of collocation points. In the cases of multiple collocation points (i.e., N_ 1283 and N. = 256%), some comparison meth- ods (such as PINN, gPINN, AHD-PINN, RoPINN, and FPINN) fail to complete training due to out-of of-memory (denoted as O/M). contrast, by leveraging its sep- arable architecture, CoPINN mitigates the burden of

---

CoPINN: Cognitive Physics-Informed Neural Networks
Exact solution
Prediction
Absolute error
Exact solution
Prediction
Absolute error
(a) PINN
(c) FPINN
(b) gPINN
(n)
(c) AINN.PINN
(g) SPINN (m)
(d) RoPINN
(h) CoPINN (Ours)
Figure 5. Prediction results of our proposed method and the state-of-the-an baselines method on the (2+1) )-d Klein- dataset with
N: 32 The exact solution, prediction, and absolute error shown the left, middle, and right, respectively.
E.03E5S
Rulltive AMSE
Table Ablation study on the (2+1)-d Klein-Gordon equation. Evaluation metric is RT2.
RASE
Ne
cossee-
16
32
6
128
256
Methods
COPINN-I
0.0062
0.0020
0.0013
0.0008
0.0009
10
16
10
10t
12:1
32*
10th
10
10
30
CoPINN-2
0.0037
0.0027
0.0015
0.0013
0.0011
CoPINN-3
0.0016
0.0010
0.0008
0.0008
0.0005
(a) Helmholtz
(b) (2+1)-d MeeinGooroon
CoPINN (Ours)
0.0016
0.0006
0.0004
0.0003
0.0002
Figure 6. Performance of CoPINN with varying B on the Helmholtz equation and the (2+1)-d l-d Klein-Gordon equation with N. 32
3.3. Ablation Study
The proposed cognitive training scheduler in CoPINN consists of two components, i.e., the epoch change compo-
results are shown Table We can draw the following ob- servations: (1) Both of these two components are essential, and missing either will result in performance degradation. (2) The epoch change component is more important than the sample change component. More detailed ablation study is provided in Appendix C.I.
nent (T. ()) and the sample change component [3- (3-8)).
To demonstrate the effectiveness of each component, we con- equa- tion with collocation point number N, = 323. be spe- eific, we design the following three variations of CoPINN, i... CoPINN-I, CoPINN-2, and CoPINN-3. Among them, CoPINN-1 represents using the original loss function, i.e., Equation (4). CoPINN-2 represents removing the epoch change component in Equation (11), i.c., the weights are calculated by vy = of - 3 5j. CoPINN-3 represents re- moving the sample change component in Equation (11), ie., the weights are calculated by v/ = mi Te 1). The
duct an ablation study on the (2+1)-d
3.4. Visualisation Analysis
To more comprehensively compare our CoPINN with base- lines, we present the exact solution, prediction results, and absolute errors for each method on the Helmholtz and (2+1)- Elein-Gordon equation with collocation points N, = 323 The visualization results are shown in Figure 4 and Figure
8

---

CoPINN: Cognitive Informeed Neural Networks
PRNN 3.6. Error Analysis
Reletwe La Error
10p
PNW JPAN
REFINN
AHD-HIV
To further observe the performance trend of our CoPINN as the number of iterations increases, we plot the RL2 varia- tion curves for each method across epochs on the Helmholtz and (2+1) 1)dd
Helative
1E-1
AMD
SPRVM
SPANN SPAhim)
30
16-0
COPIN
equations. As shown in Figure 7, we can observe that: (1) Although the RL2 curves exhibit some fluctuations, CoPINN shows an overall downward trend. This shows that CoPINN is correctly optimized. (2) CoPINN outperforms all comparison methods for the ma- jority of the epochs, and as the number of epochs increases, its final performance consistently surpasses that of the oth- ers. This improvement is attributed to CoPINN gradually learning the entire sampling region from easy to difficult, thus overcoming the UPP. Besides, results and analysis on the RMSE metric can be found in Appendix C.3.
10002
23000
2000
42000
50030
10000
20100
30003
40000
50005
Epochs
Epochs
(a) Helmholtz
(b) (2+1)-d Klein-Gordon
Figure 7. Variation curve of RI2 with the increase of epoch under N: = 32
Epoch-500
Epoche600
1C0
Epoch=5000
Epoch=8000
Densily
Epoch-50000
Epochr60000
3.7. Difficulty Analysis
To evaluate the effectiveness of the cognitive training sched- uler of CoPINN, we visualize the difficulty density of sam- ples under different iterations. The results are shown in the Figure 8, from which we can observe that: (1) With the increase in the number of iterations, the overall dif- ficulty gradually decreases with or without the cognitive training scheduler. (2) Compared with without cognitive training scheduler, our method can reduce the difficulty of the samples, thanks to gradually increasing the weight of the difficult samples during the training phase.
3.0
Difficulty
Difiiuuty
(a) Cognitive training
(b) w/o cognitive training
Figure& 8. Difficulty density on different epochs. Experiments are conducted on the 1-d Klein Gordon equation PDE systems with 16 collocation points.
We can observe that: (1) All comparison methods exhibit a significantly poorer fitting performance on the Helmholtz equation than on the -ameesiiial Klein-Gordon equa- tion. This suggests that the Helmholtz equation is more challenging to learn. (2) CoPINN closely aligns with the ex- act solutions for both the more difficult Helmholtz equation and the relatively easier -2mmssaimensoooal Klein-Gordo equation, especially in difficult regions such as physical boundaries, which is attributed to its progressive optimiza- tion of the entire sampling regions from easy to hard. More visualisation and analysis on the Diffusion equation are shown in Appendix C.2.
4. Conclusion
In this paper, we reveal and address previously unex- plored yet pervasive challenge in PINN, referred to as the Unbalanced Prediction Problem (UPP). To tackle this is- sue, we propose novel Cognitive Physics-Informed Neural Networks (CoPINN) method, which imitates the human cognitive learning manner from easy to hard. Specifically, our CoPINN first adopt separable learning to encode each independent one-dimensional coordinate by each separa- ble sub-network, thereby reducing computational complex- ity. Then, an aggregation scheme is applied to produce multi sitti-dimeesional predictions. During the training process, CoPINN dynamically evaluates the difficulty of each sam- ple based on the gradient magnitude of the PDE residuals. Finally, we propose cognitive training scheduler that adap- tively optimizes the PINN model from easy to difficult sam- ples, improving its robustness and generalization, especially in predicting physical boundary regions. To demonstrate its versatility, we evaluate CoPINN on multiple widely used PDEs. Experimental results show that CoPINN consistently outperforms all comparison methods.
3.5. Parametric Analysis
To investigate the impact of different B B-hyperparameter val- ues in Equation (11) on the performance of COPINN, we plot the performance curves of CoPINN with varying / on the Helmholtz equation and the (2+1)-d Klein-Gordon equa- tion with N₂- 323 in Figure 6. On the Helmholtz equation, the performance of CoPINN initially remains stable as 3 in- creases, but then rapidly declines (with a sharp rise in error). For the (2+1)-d Elein-Gordon equation, the performance of CoPINN first improves and then deteriorates as / increases, achieving optimal performance when 3 = 10-2. Therefore, we recommend that the value of be chosen within [10-4,
10-1]
9

---

CoPINN: Cognitive Physics-Informen Neural Networks
Acknowledgements
Jiang, L.,, Meng, D., Zhao, Q. Shan, S., and Hauptmann,
A. Self-paced curriculum learning. In Proceedings of the AAAI Conference Artificial Intelligncee volume 29, 2015.
This work was supported by the Open Research Project of the National Key Laboratory of Fundamental Algorithms and Models for Engineering Numerical Simulation.
Jin, L.. Liu, L.. Wang. X., Shang. M., and Wang, F.-Y. Physical-informee neural network for mpe-based trajec- tory tracking of vehicles with noise considered. IEEE Transactions on Intelligent Vehicles, 2024.
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none of which we feel must be specificallyy highlighted here.
Kharazmi, E.,, Zhang, Z., and Karniadakis, G. E. hp-vpinns: Variational physics-informed neural networks with do- main decomposition. Computer Methods in Applied Me- chanics and Engineering, 374:1 13547, 2021.
References
Kim, J., Lee, K., Lee, D., Jhin, Y., and Park, N. Dpm: novel training method for physics-informed neural net- works in extrapolation. In Proceedings of the AAAI Con- ference on Artificial Intelligence, volume 35, pp. 8146- 8154, 2021.
Bai, S., Ren, X., Zheng, Q., and Zhu, J. Graph-driven deep multi-view clustering with self-paced learning- Knowledge-Baod Based Systems, 296: 11871, 2024.
Barth, T., Herbin, R., and Ohlberger. M. Finite volume methods: foundation and analysis. Encyclopedia of Com- putational Mechanics Second Edition. pp. 1-60, 2018.
Kingma, D. P. and Ba, Adam: A method for stochastic optimization. CoRR, abs/ htss/1412.69800 2014.
Cai, S., Wang, Z., Wang, S., Perdikaris, P., and Kariiadakis, G. E. Physics-informed neural networks for heat trans- fer problems. Journal of Heat Transfer, 066600,,,, 2021.
Krishnapriyan, A., Gholami, A., Zhe, S., Kirby, R., and Mahoney, M. W. Characterizing possible failure modes in physics-informed neural networks. Advances in Neural Information Processing Systems, 34:26548 265600, 2021.
Chen, J., Zhang, R., Wang, J., Hu, C.,, and Mao, Y. Self-paced pairwise representation learning for semi- supervised text classification. In Proceedings of the ACM on Web Conference 2024, pp. 4352-4366 2024.
Li, T., Zou, Y., Zou, S., Chang, X., Zhang, L.., and Deng, X. Learning to solve pdes with finite volume-informed neural networks in a data-free approach. Journal of Com- putational Physics, 530:1 13919, 2025.
Cho, J., Nam, S., Yang, H., Yun, S.-B., Hong, Y., and Park, E. Separable thysics-inffrmme neural networks. Ad- vances in Neural Information Processing Systems, 2023.
Li, Y., Chen, S., Shan, B., and Huang, S.-J. Causality- enhanced discreted physics-informed neural networks for predicting evolutionary equations. In International Joint Conferences on Artificial Intelligence Organization, pp. 4497 2024.
Dashtbayaz, N. H., Farhani, G., Wang, B., and Ling, C. X. Physics-informed neural networks: Minimizing residual loss with wide networks and effective activations. pp. 5853-5861. jjcai.org, 2024.
Long, Z., Lu, Y., Ma, X., and Dong, B. Pde-net: Learning pdes from data. International conference on machine learning. pp. 3208-330-000 02163. PMLR, 2018
Daw, A., Bu, J., Wang, S., Perdikaris, P., and Karpatne, A. Mitigating propagation failures in penysics-iffomen neu- ral networks using retain-rsampll--eeeess (13) sampling- In International Conference on Machine Learning, 2023.
Pu, R., Sun, Y., Qin, Y., Ren, Z., Song, X., Zheng, H., and Peng. D. Robust self-paced hashing for cross- modal retrieval with noisy labels. arXiv preprint arXhv:2501.01699 2025.
Evans, L. C. Partial differential equations, volume 19. American Mathematical Society, 2022.
Jagota, V., Sethi, A. P. S., and Kumar, K. Finite element method: an overview. Walailak Journal of Science and Technology (WJST), 10(1):1-8, 2013.
Raissi, M., Perdikaris, P., and Karniadakis, G. Physics- informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686-707, 2019.
Jagtap, A. D., Kharazmi, E., and Karniadakis, G. E. Con- servative physics-infee -nforred neural networks on discrete domains for conservation laws: Applications to forward and inverse problems. Computer Methods in Applied Mechanics and Engineering, 365:1 13028, 2020.
Roubsček, T. Nonlinear partial differential equations with applications, volume 153. Springer Science & Business Media, 2013.
10

---

CoPINN: Cognitive Physics-Informec Neural Networks
Shrivastava, A., Gupta, A., and Girshick, R. Training region- based object detectors with online hard example mining- In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 761-769, 2016.
Xiang. Z., Peng, W., Liu, X., and Yao, W. Self-adaptive loss balanced physics-informed neural networks. Neurocom- puting, 496: 1--44, 2022.
Yang. L., Meng. X., and Karniadakis, G. E. B-pinns: Bayesian physics-informed neural networks for forward and inverse pde problems with noisy data. Journal of Computational Physics, 425:109913, 2021.
Song, C. and Alkhalifah, T. A. Wavefield reconstruction inversion via physics-informed informed neural networks. IEEE Transactions on Geoscience and Remote Sensing, 60:1- 12, 2021.
Yu, J., Lu, I.., Meng. X., and Karniadakis, G. F. Gradient- enhanced physics-informed neural networks for forward and inverse pde problems. Computer Methods in Applied Mechanics and Engineering. 393: 14823, 2022.
Soviany, Ionescu, R. T., Rota, P., and Sebe, N. Cur- riculum learning: A survey. International Journal of Computer Vision,
2022.
Sun, Y., Dai, J., Ren, 7.,, Chen, Y., Peng, D., and Hu, P. Dual
Yuan, Z., Liu, H., Zhou, H., Zhang, D., Zhang, X., Wang. H., and Xiong, Self-paced unified representation learning for hierarchical multi-label classification. In Proceed- ings of the AAAI Conference Artificial Intelligence, volume 38, pp. 16623-16633 2024.
self-paced cross-modal hashing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 15184-15192, 2024a.
Sun, Y., Qin, Y., Peng, D., Ren, Z., Yang. C., and Hu, P. Dual self-paced hashing for image retrieval. IEEE Transactions Multimedia, 2024b.
Zhao, D., Koh, Y. S., Dobbie, G., Hu, H., and Fournier- Viger, Symmetric self-paced learning for domain gen- eralization. In Proceedings of the AAAI Conference Ar- tificial Intelligence, volume 38, pp. 16961-16969, 2024.
Thomas, J. W. Numerical partial differential equations: finite difference methods, volume 22. Springer Science & Business Media, 2013.
Zhou, P., Sun, B., Liu, X., Du, L., and Li, X. Active clus- tering ensemble with self-paced learning. IEEE Transac- tions Neural Networks and Learning Systems, 2023.
Wang, S., Teng, Y., and Perdikaris, P. Understanding and mitigating gradient flow pathologies in chysics-informen neural networks. SIAM Journal on Scientific Computing,
A3055533355 2021a.
Zou, Y., Li, T., Lu, L., Wang, J., Zou, S., Zhang, L., ceffference-infonene graph network
and Deng, X. for solving steady-state incompressible flows on block- structured grids. Physics of Fluids, 36(10):103008, 10 2024a.
Wang, S., Yu, X., and Perdikaris, P. When and why pinns fail to train: A neural tangent kernel perspective. Journal of Computational Physics, 449:1 10768, 2022.
Wang, X., Chen, Y., and Zhu, W. A survey on curriculum learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, ((9))4555-4576 2021b.
Zou, Z.., Meng, X., and Karniadakis, G. E. Correcting model misspecification in physics-informed neural net- works (pinns). Journal of Computational Physics, 505: 112918, 2024b.
Wright, L. G., Onodera, T., Stein, M. M., Wang, T., Schachter, D. T., Hu, Z., and McMahon, L. Deep physical neural networks trained with backpropagation. Nature, 601(7894):549-5555 2022.
Wu, C., Zhu, M., Tan, Q. Kartha, Y., and Lu, L. A compre- hensive study of non-adaptive and residual-baas based adaptive sampling for physics-infonl informed neural networks. Com- puter Methods in Applied Mechanics and Engineering, 403:115671, 2023.
Wu, H., Luo, H., Ma. Y., Wang. J., and Long, M. Ropinn: Region optimized physics-ifformed neural networks. In Advances in Neural Information Processing Systems, 2024.
Wu, W., Duan, S., Sun, Y., Yu, Y., Liu, D., and Peng, D. Deep fuzzy physics-informed neural networks for for- ward and inverse pde problems. Neural Nenworks, 181: 106750, 2025.

---

CoPINN: Cognitive Physics-Informec Neural Networks
APPENDIX
This document provides additional details, analysis, and experimental results to support the main submission.
In Appendix A, we provide a detailed flow of CoPINN training algorithm.
In Appendix B, provide detailed experimental setup:
Datasets details Appendix B.1
Baselines details in Appendix B.2
Evaluation metrics details in Appendix B.3
Implementation details in Appendix B.4
In Appendix C, we present additional experimental results to highlight the advantages of CoPINN:
Additional ablation study in Appendix C.I
Additional visualisation analysis in Appendix C.2
Additional error analysis in Appendix C.3
Additional experiments on flow-mixing 3D equation in Appendix C.4
Additional experiments Convection equation in Appendix C.5
In Appendix D, we present related work that is closely connected to this study.
In Appendix E, we discuss the limitations of CoPINN.
A. CoPINN Training Algorithm
Algorithm 1 describes the training algorithm Cognitive Physics-Informin Informed Neural Networks (CoPINN). First, the separable sub-networks used encode independent one-dimensional coordinates. Second, aggregation scheme is applied to obtain the dulen-ddimenioonll predicted physical variables. During the training process, CoPINN dynamically evaluates the difficulty of predicting each sample based on the gradient magnitude of the PDE residuals. Finally, cognitive training scheduler is employed to adaptively optimize the model from easy to hard.
Algorithm CoPINN algorithm
Input: maximal epoch number Ne, initial parameters of net-nes of CoPINN (1,0)) 1/ training set D, hyperpa- rameter 3.
Output: optimized network parameters {0,} Compute epoch step size Te by Equation (7). for i N: do
Calculate the output of each sub-network and merge them by Equation (2).
Calculate original PDE loss Lpáe IC loss Lic. and BC loss Lac- Compute the weight assigned to the easiest sample point in epoch i, i.c., u by Equation (8). Compute the weight assigned to the hardest sample point in epoch i, i.e., the by Equation (9).
Compute difficulty D for each sample point x by Equation (6). Compute weights for each sample point according to their difficulty by Equation (11).
Compute weighted loss by Equation (5). Update {0,),-1- end for
B. Detailed Experimental Setup B.1. Dataset Details
Details of the five partial differential equations used in the experiment are as follows.
12

---

CoPINN: Cognitive Physics-Informed Neural Networks
Helmholtz equation. The Helmholtz equation is a second-order partial differential equation that is commonly used to descrihe phenomena such waves and vibrations. It has important applications in physics, engineering. acoustics, electromagnetism, and quantum mechanics. The form of the Helmholtz equation we selected is as follows:
Au + kau q, PES 2 ((x) = 0, x E an,
(12)
(13)
where Q is the spatial domain, Ω [-1, q is the given source term, q = (a1 ) (a²εT) )uu (a3 + k2u, is the manufactured solution, = sin(a1 sin (22 sin ma) where A: = 1, = 4, 02 4, 03 = 3.
Diffusion Equation. Diffusion equations are mathematical models used to describe changes in the diffusion or distribution of matter, energy. or other physical quantities in space. It has a wide range of applications in physics, chemistry, biology, and engineering, such as heat conduction, pollutant diffusion, gas diffusion, and so on. The form of the Diffusion equation we selected is follows:
"y = Vull + udu), TEQ,LET, u(x, 0) (I)( x JEΩ,
(14)
(15)
u(x, t) 0, E 0
(16)
where a is the diffusion coefficient, a 0.05, Ω is the spatial domain, Ω []1, D the temporary domain, r = 0, 1].
Wie is the initial condition. Following SPINN (Cho al., 2023), we use the PDE simulation software FEniCS to obtain the numerical solution the reference, with a resolution of 101x101x101. The initial condition Uie is a superposition of three Gaussian functions, as shown below:
(in = 0.25 (00 (1 0.3)
+ 10
(17)
+ 2033 )²
Klein-Gordon Equation. The Klein-Gordon equation is an equation in quantum field theory that describes scalar fields. an important equation for describing particles in relativistic quantum field theory and is wave equation under the framework of special relativity, especially for describing massless spin particles (such scalar partieles). The solution of
this equation can be used to describe the quantum states of some quantum fields and can be generalized to describe particles with spin (such as electrons and photons). The form of the Klein-Gordon equation we selected is as follows:
Us Au + u² = f. x ZENttr,, u(x, 0) σ1 + I2, E Ω TE Ω uxx t) = ub)()), x E 00,
(18)
(19)
(20)
where 52 is the spatial domain, Ω - 1-1, I' is the temporary domain, I' = [0, 10). u is the manufactured solution, u = (x) + 222 (2t) + 21 sin( f. Use is extracted from manufactured solution u.
(2+1)-d Flow Mixing Problem. Flow mixing a crucial topic in fluid mechanics, typically involving the interaction of multiple fluid components or fluids with varying temperatures and densities during flow. The Flow Mixing equations that govern this process derived from the laws of conservation of mass, momentum, and energy, along with the transport properties of the fluid, such as diffusion, convection, and turbulence. In this paper, we model flow mixing as time-dependent partial differential equation that describes the mixing behavior of two fluids at the interface within two-dimensional environment, as shown in the following form:
I ult, x, v) = - ttanh( S cos(wtt) sin(w))), E [0,4], E
(21)
W Vt
(22)
Ut,max
We + Cu2 + Bug 0,
(23)
13

---

CoPINN: Cognitive Physics-Informec Neural Networks
a(x,y) v)
U2
3/
(24)
Ut,mar
((x,y)
(25)
Ut,maz
0; - sech (r) Lanh(r),
(26)
v x²2 + y
(27)
man 0.385.
(28)
Convection Equation. In this study, we consider a ID Convection equation, which is commonly used to model transport phenomena, described follows:
du
+ du
dr = 0, x [0, 2T).
t [0, 1]
(29)
Ōt
u( 0) h(x)
(30)
O0 t) = u(2x,t) t)
(31)
where & is the convection coefficient and h(x) is the initial condition. Following (Krishnapriyan et al., 2021), we use constant setting of h(x) sin(x) with periodic boundary conditions in all our experiments, while varying the value of g in different case studies.
B.2. Evaluation Baselines
We compare the proposed CoPINN with the following methods:
PINN (Physics-ing (Physics-ifformed informed neural networks) (Raissi et al., 2019) is the first attempt to integrate physical information, namely, the constraints of PDEs, into neural network optimization. This integration allows the model to learn data distribution during training while adhering physical laws, thereby enhancing both its generalization ability and physical consistency.
gPINN sendinet-eehaneed physics-infone informed neural networks) (Yu et al., 2022) leverages gradient information the PDE residual and embeds the gradient into the loss function, thereby improving the accuracy and training efficiency of PINN.
informed neural networks) (Cho et al., 2023) propose a new architecture to operate on a per-axis basis, thereby significantly reducing the number of network propagations in multi-dimensional PDEs. Based SPINN, the author also proposed SPINN with modified MLP (SPINN (m)), which uses modified MLP to replace MLP and achieve better performance.
SPINN (Separable
AHD-PINN (PINN with Higher Derivative the Activation function satisfies bijective) (Dashtbayaz et al., 2024) demonstrates the global minimization of residual loss PINN with sufficient network width, and establishes optimal activation function selection based on bijective derivatives for k-th order operators.
RoPINN (Region optimized physics-ine physics-informed informed neural networks) (Wu et al., 2024) extends the optimization process of PINN from isolated points to their continuous neighborhood regions, which can theoretically decrease the generalization error, especially for hidden high-order constraints of PDEs.
FPINN (Deep fuzzy physics-infore informed neural networks) (Wu et al., 2025) is the first attempt to apply fuzzy neural network solve PDEs, which can handle the ambiguity in data from commercial simulation software.
14

---

CoPINN: Cognitive Physics-Informer Neural Networks
B.3. Evaluation Metries
To comprehensively evaluate our COPINN, we introduce two evaluation metries; Relative L2 error (RL2) and Root Mean Square Error (RMSE). To be specific, for the forward problem of partial differential equations, the Relative L2 error is defined as:
m RL2
1
pred
exace
(32)
m
Meraett
where upred and Waaeet indicate the prediction and reference solutions of the i-th collocation point, m is the number of collocation points, respectively.
The second evaluation metric is the Root Mean Square Error, given by:
m
RMSE
(w U Wexraet )²
(33)
The calculation methods for RL2 and RMSE are referenced in the original PINN paper (Raissi et al., 2019) and the SPINN paper (Cho et al., 2023).
The third evaluation metric is the improvement percentage (IMP.), given by:
M1 ual x 100%,
IMP.
(34)
Us
where u, represents the RL2 RMSE of the second-best method, while up denotes the RL2 RMSE of the best method.
B.4. Implementation Details
The results of all the experiments were averaged from five random seeds. For Helmholtz, (2+1)-d Klein CleinnGordon, (3+1)-d MeeinnGoroon, Diffusion and flow Mixing 3D equations, we set the sampling points to 1003, 100 50 101 and 100 during testing, respectively. Following SPINN (Cho et al., 2023), we divide the dataset into a training set and test set. The training set is used to train the neural network, while the test set is used to evaluate the prediction ability of the model. For all comparative methods, we use the source codes provided by the original authors and apply the parameters recommended in their published papers. For our CoPINN, the network architecture consists of five hidden layers, where each layer has 128 hidden units. We apply the modified MLP introduced in (Wang et al., 2021a) to CoINN. On all datasets, we exploit the Adam optimizer (Kingma & Ba, 2014) train model with large learning rate of 1e-3 for 50,000 epochs. We use the 'tanh' activation function throughout our CoPINN. According to the parameter analysis of our method, the (2+ I-dd Klein-Gordon dataset, we set 8 = 0.01. And on the Helmholtz, Diffusion, and (3+1)-+ d Klei-Gordee datasets, / is set to 0,001. To ensure fair comparison, we set the balance parameter of the loss terms to be equal, i.c., X = 1 in Equation (5), All experiments are implemented in JAX/Flax and trained on single NVIDIA 3090 GPU with 24GB of memory.
C. Additional Experimental Results
C.1. Additional Ablation Study
This section serves supplement to the ablation study in Section 3.3. To demonstrate the effectiveness of each component in the cognitive training scheduler of our CoPINN, we perform ablation study on the (2+1)-d Klein-Gordon equation using the RMSE metric. Specifically, we design the following three variations of CoPINN, i..., 1 CoPINN-2, and CoPINN-3. Specifically, CoPINN-1 uses the original loss function, i.e., Equation (4). CoPINN-2 represents removing the epoch change component in Equation (11), i.e., the weights are calculated by o/ = v 3 8).- CoPINN-3 represents removing the sample change component in Equation (11), i.c., the weights are calculated by ej = of
(i - 1). The results are shown in Table 3 demonstrate that: (1) Both of these two components are essential, and missing either will result in performance degradation. (2) The epoch change component is more important than the sample change component. These observations are the same as leveraging R....
15

---

CoPINN: Cognitive Physics-Informec Neural Networks
Table 3. Ablation study on the (2+1)-d Klein-Gordon equation. Evaluation metric is RMSF.
ce 16a
323
64
128a
256
Methods
CoPINN-1
0.0039
0.0013
0.0008
0.0005
0.0002
CoPINN-2
0.0023
0.0017
0.0010
0.0008
0.0007
CoPINN-3
0.0010
0.0006
0.0005
0.0005
0.0003
CoPINN (Ours)
0.0010
0.0005
0.0002
0.0002
0.0002
C.2. Additional Visualisation Analysis
This section serves a supplement to the visualisation analysis presented in Section 3.4. To more comprehensively compare CoPINN with baselines, present the exact solution, prediction, and absolute error for each method the Diffusion equation in Figure 9. The selected time points are / = 0, 0.5, 1. The prediction results of our CoPINN are highly consistent with the exact solution, further confirming its superior performance.
Exact solution
Prediction
Absolute emror
Exact solution
Prediction
Absolute error
0.0
t= 0.5
0.5
1=1.0
(a) CoPINN
(b) SPINN (m)
Figure 9, Prediction results of CoPINN and SPINN (m) on the Diffusion dataset with N. 256 The left the exact solution, the middle is the predicted value, and the right the absolute value of the error.
C.3. Additional Error Analysis
This section serves supplement to the enror analysis presented in Section 3.6. To further observe the performance trend of our CoPINN as the number of iterations increases, we plot the RMSE metric variation curves for each method across epochs on the Helmholtz and (2+1)-d Klein-Gordow equations. As shown in Figure 10, similar observations can be obtained as the RL, metric: despite fluctuations, RMSE curves for all methods generally decrease over iterations, showing effective optimization, with CoPINN consistently outperforming baselines and achieving superior final performance as epochs increase. These consistent improvement underscores the superiority and stability of our CoPINN.
16

---

CoPINN: Cognitive Physics-Informis Neural Networks
10a
PINN
10a
gPINN
PINN
RoPINN
AHD-PINN
RMSE
RoPINN
RMSE
FPINN
AHD-PINN
FPINN
10"²
SPINNIM)
10-2
CoPINN
SPINN(M)
CoPINN
10-3
10
10000
20000
5000C
10000
20000
40000
Epochs
Epochs
(a) Helmholtz
(b) (2+1)-d Klein-Gordon
Figure 10. Variation curve of RMSE with the increase of epoch under No 32
C.4. Experiments on Flow-mixing 3D Equation
To further validate the performance advantages of the proposed CoPINN, we compare the results of CoPINN, SPINN, and SPINN(m) on a more complex Flow-mixing 3D Equation. As demonstrated by the results in Table 4, CoPINN achieves the smallest error on the Flow-mixing 3D Equation, further confirming its superiority.
Table Full results the Flow-mine Now-mixing 3D equation. No is the number of collocation points. The best and the second- best results are highlighted in boldface and underlined, respectively.
Evaluation Metric
RL2
RMSE
N₂
Methods
163
323
643
1283
2563
163
323
643
1283
2563
Ref.
SPINN
NeurlPS'200 2023
0.1352
0. 1054
0.0861
0.0787
0.0672
0.1087
0.0875
0.0711
0.0575
0.0488
SPINN (m)
NeurIPS'2023
0.0884
0.0263
0.0080
0.0032
0.0023
0.0342
0.0241
0.0063
0.0022
0.0016
CoPINN
Ours
0.0853
0.0230
0.0063
0.0032
0.0022
0.0323
0.0190
0.0060
0.0022
0.0015
IMP.
4%
11%
21%
0%
4%
6%
21%
5%
0%
6%
C.5. Experiments on Convection Equation
To further validate the effectiveness of the proposed difficulty evaluation and cognitive training scheduler on the ID Convection equation, we employ them on the vanilla PINN. In this experiment, we set the number of collocation points 1000, set the 3 in the cognitive training scheduler as 0.1, do not change other default parameters, and average across random runs. The experimental results below show that, in Equation (29), when 30, our method outperforms all the comparison methods, and when 50, our method is also competitive compared to R3 and Casual R3.
Table 5. Relative L2 errors of comparative methods over ID Convection equation.
E
30
50
Methods
RAD (Wu al., 2023)
0.0418
0.6547
R3 (Daw al., 2023)
0.0160
0.5794
Causal R3 (Daw et al., 2023)
0.0073
0.5471
Ours
0.0055
0.5811
17

---

CoPINN: Cognitive Physics-Informin Neural Networks
D. Related Work
Physics-Informed Neural Networks (PINN) (Raissi et al., 2019) is machine learning model that integrates physical laws into the training process of neural networks. Its goal is to solve partial differential equations (PDEs) and other complex scientific problems by combining data-driven learning with known physical principles. Due to its meshless character and low computational cost, PINN has become a promising alternative to traditional numerical methods. Thus, in recent years, large number of PINN methods have been proposed (Song & Alkhalifah, 2021; Cai et al., 2021; Wang et al., 2022; Jin et al., 2024; Zou et al., 2024b). With the fast development of deep learning, the PINN technology has been widely explored from various aspects: novel neural network architectures, novel optimization schemes, loss-weighting, and adaptive sampling. In the following, we review these four aspects of technology.
Neural network architectures attempt to design novel neural network architectures to enhance model capacity. For example, cPINN (Jagtap et al., 2020) constructs a separate neural network in each discrete sub-domain. However, cPINN struggles with solving PDE problems involving noisy data. To address this limitation, BPINN (Yang al., 2021) introduces Bayesian framework to quantify aleatoric uncertainty caused by noisy data. Despite their effectiveness, BPINN incurs significant time overhead. To reduce computational costs while handling noisy data, FPINN (Wu et al., 2025) leverages fuzzy membership functions and fuzzy rules to manage noise and uncertainty. Nevertheless, these methods face challenges in solving high-dimensional PDEs or approximating highly complex solution functions due to their high computational and memory demands. To tackle these issues, SPINN (Cho et al., 2023) adopts a per-axis operation approach, significantly reducing the network propagations required for nulti-dimensional PDBs.
Optimization schemes aim to develop new optimization schemes to cope with the rough loss landscape. For instance, hp-vPINN (Kharazmi et al., 2021) solves PDEs by minimizing residuals through a least-squares approach. gPINN (Yu et al., 2022) incorporates gradient information from PDE residuals into the loss function to enhance precision and training efficiency. However, these methods often violate the temporal causality property, leading to training challenges for evolutionary PDEs. To address this issue, TL-DPINN (Li et al., 2024) adopts implicit time differencing to preserve temporal causality and utilize transfer learning to update PINN sequentially. Despite their advantages. TL-DPINN optimizes models only on scattered points. To achieve accurate solutions across the entire domain, ROPINN (Wu et al., 2024) extends the optimization process to continuous neighborhood regions, thereby improving the performance of PINN.
Loss weighttt aims to develop novel loss we-weighting to balance different losses, thereby achieving more balanced convergence. To be specific, Wang et al. (2022) observed that the weighted combination of competitive multiple loss functions significantly impacts the training of PINN. To address this, they propose IbPINN, which automatically assigns loss weights based on maximum likelihood estimation. Similarly, Xiang et al. (2022) introduces SA-PINN, employing Gaussian Process regression construct a continuous map of self-adaptive weights. This method dynamically increases weights as the corresponding losses rise during training, enhancing model performance.
Adaptive sampling operates as dynamic optimization framework that iteratively adjusts the spatial-temporal distribution of training points during neural network training. This methodology strategically allocates computational resources by continuously refining the selection of collocation points based on real-time error metrics. Through residual-driven refinement mechanisms, the algorithm preferentially concentrates sampling density in regions exhibiting elevated PDE residual errors (indicative of model inaccuracies) or steep solution gradients (marking mathematically challenging zones like boundary layers, shock fronts, or material interfaces) (Wu et al., 2023; Daw et al., 2023). This targeted resource allocation enables the model to resolve physically critical features with increased precision while avoiding computational redundancy in smoother, well-predicted domains, thereby ensuring enhanced solution accuracy and training efficiency compared to static sampling approaches.
However, the above four aspects of technology always treat both easy and hard samples equally while ignoring sample importance and lcarning difficulty. To address this issue, we draw inspiration from human cognitive processes and propose a PINN model with SPL that evaluates sample-level difficulty and guides the neural network to fit samples progressively from easy to hard when solving PDEs.
E. Limitations
Although the proposed CoPINN method demonstrates significant advantages in solving partial differential equations, we acknowledge several limitations that require further investigation. First, when CoPINN increases the weight of difficult samples during each iteration, it may overly focus on these samples. This could result in overfitting to the difficult samples
18

---

CoPINN: Cognitive Physics-Informed Neural Networks
while neglecting other samples, particularly if the difficult samples do adequately represent the overall data distribution. Second, dynamically weighting samples can introduce instability into the training process, particularly when the weights assigned to difficult samples increase significantly. This may cause CoPINN to rely disproportionately on these samples during certain iterations, potentially leading to deviations in gradient updates and more volatile training process.
19


Reviews:
[{"review_id": "cVjvcsBTo7", "reviewer": "Reviewer_doqg", "summary": "The authors look at the PINNs setting, based on training a neural network to confirm to the PDE residual. They employ a method that dynamically samples collocation points according to the gradient of the PDE residual. They use this as a signal to do PINNs training starting with the solution on the collocation points that are easy to learn, and then move to the harder to learn regimes. They demonstrate this on four different PDEs.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewers for their constructive feedback. Below are our responses to the questions raised:\n\n**Q1: Methods And Evaluation Criteria & Questions For Authors**\n\n**R1:** The reference solutions refer to the labels used to compute the relative $L_2$ error and $RMSE$ by comparing them with the model's predicted solutions. Following [1-3], for the Diffusion equation, reference solutions are obtained through the widely-used PDE solver platform FEniCS at a resolution of 101×101×101. For the Helmholtz and Klein-Gordon equations (3D and 4D), reference solutions with 100×100×100 resolution are independently designed and manufactured respectively.\n\n**Q2: Claims And Evidence & Experimental Designs Or Analyses & Essential References Not Discussed & Questions For Authors**\n\n**R2:** According to your suggestion, we will discuss adaptive sampling in the related work section. We acknowledge that existing work in the field of adaptive sampling, such as RAD/RAR-D [4] provides valuable insights relevant to our study. However, CoPINN fundamentally differs from these existing adaptive sampling methods:\n\n1. Existing adaptive sampling methods like RAD/RAR-D [4] primarily focus on the global residual distribution, improving overall accuracy by resampling or adding points to the region near the high-residual points, but they ignore the Unbalanced Prediction Problem (UPP) of PINN. In contrast, the core objective of CoPINN is to resolve the UPP commonly observed in traditional PINNs near physical boundaries. We find that relying solely on absolute residual values (e.g., as in RAD) may overlook the high-gradient nature of boundary regions, leading to local overfitting or underfitting. Therefore, CoPINN introduces a residual gradient-based dynamic difficulty assessment to more accurately identify abrupt changes in boundary regions (e.g., shocks and singularities).\n\n2. Existing adaptive methods (e.g., RAR-D) employ a greedy strategy to incrementally add high-residual points but do not consider the model’s optimization capability at different training stages. To address this, CoPINN incorporates a cognitive training scheduler, which prioritizes simpler regions (low-gradient areas) in the early stages to stabilize training, while progressively increasing the weight of more complex regions (high-gradient areas) in later stages. This prevents the model from prematurely converging to suboptimal local solutions.\n\n**Q3: Experimental Designs Or Analyses**\n\n**R3:** The reviewer notes that prior work [5] addresses similar issues via loss reweighting. However, our CoPINN fundamentally differs in both methodology and scope. [5] focuses on loss-term balancing (e.g., PDE residual vs. boundary loss) without considering spatial/temporal variations in sample difficulty. In contrast, inspired by the human curriculum learning strategy of progressing from easy to hard tasks, CoPINN introduces a cognitive learning approach. It evaluates sample-level difficulty (via PDE residual gradients) and progressively prioritizes harder regions during training. CoPINN is finer-grained, addressing intra-term imbalances (e.g., stubborn points vs. smooth regions).\n\n**Q4: Experimental Designs Or Analyses**\n\n**R4:** There are two key differences between the approach with hard constraints [6-7] and our proposed CoPINN.\n\n1. [6-7] focus on inverse problems (e.g., geometric parameter optimization), with the core objective of simultaneously satisfying PDE constraints and optimization objectives. In contrast, CoPINN targets the Unbalanced Prediction Problem (UPP) in forward PDE solving, aiming to address error caused by varying sample difficulties in PINN training. Their application scenarios are fundamentally distinct.\n\n2. [6-7] employ fixed hard constraints (e.g., modified network architectures) and optimization strategies to ensure strict adherence to physical properties during the prediction process, enhancing accuracy through the incorporation of additional constraints. CoPINN introduces a dynamic cognitive scheduler that evaluates sample difficulty via PDE residual gradients and progressively adjusts training weights from easy to hard samples, constituting a data-driven adaptive learning paradigm. This training strategy, inspired by human cognitive processes, remains unexplored in the PINN domain.\n\n[1] Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.\n\n[2] Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems.\n\n[3] Separable physics-informed neural networks.\n\n[4]A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks\n\n[5]Understanding and mitigating gradient flow pathologies in physics-informed neural networks.\n\n[6]Physics-informed neural networks with hard constraints for inverse design.\n\n[7]Scaling physics-informed hard constraints with mixture-of-experts.", "claims_and_evidence": "The authors claim that the three categories of PINNs methods include designing the architecture of PINNs, changing the loss function, and changing the weights on the loss function for PINNs. The authors claim that their method works the best compared to various other PINNs methods on the four different PDEs. \n\nHowever, they are missing a very important line of work that does adaptive sampling of the collocation points of PINNs, which is similar to what the authors are doing. For example, [1].\n\n[1] Wu et al. A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks. Computer Methods in Applied Mechanics and Engineering, 403, 115671, 2023.", "methods_and_evaluation_criteria": "The method the authors employ is to track gradients, which is a proxy for the difficulty of learning the solution on each collocation point. They start with the model putting more weight on the easier to learn regions of the solution space, and then employ a scheduler that gradually then starts to put more weight on the harder to learn regimes. This is done in a dynamic fashion over the course of training.\n\nAs far as the evaluation, this is done by computing L2 relative error with respect to a reference solution, as well as looking at the RMSE. However, for problems that are not the diffusion problem,  it is not stated exactly what the L2 relative error is compared to: what is the reference solution that the predicted solution is compared to, and how was this data generated?", "theoretical_claims": "There are no theoretical results.", "experimental_designs_or_analyses": "The authors set up 4 PDE problems to solve. They analyze this based on L2 relative error and RMSE. See above comment for questions on how reference solution data is generated and compared to.\n\nAs mentioned earlier, there isn’t a mention or comparison of the many adaptive sampling PINNs papers, such as [1]. It seems like [2] also looks at the Helmholtz problem (different source terms) through changing the loss function, and seems to mitigate this issue through loss reweighting. There are also lines of work that aims to address some of the issues the authors note, such as certain regimes being harder to learn, through imposing hard constraint [3] [4].\n\n[1] Wu et al. A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks. Computer Methods in Applied Mechanics and Engineering, 403, 115671, 2023.\n[2] Wang et al. Understanding and mitigating gradient flow pathologies in physics-informed neural networks. SIAM Journal on Scientific Computing (2021)\n[3] Lu et al. Physics-Informed Neural Networks with Hard Constraints for Inverse Design. CMAME (2021)\n[4] Chalapathi et al. Scaling physics-informed hard constraints with mixture-of-experts. ICLR (2024)", "relation_to_broader_scientific_literature": "This work is part of the vast, and very well-studied, literature on PINNs and ML for solving PDEs. It proposes another method to improve the training of PINNs. Given the amount of literature in this space, there is a lot more that can be done to position this work in the broader context of the field.", "essential_references_not_discussed": "This work does not discuss a broad set of PINNs literature that has focused on “adaptive sampling” techniques where different parts of the domain or different points are weighted differently, and more weight is given to points that have higher PDE residuals. Such as [1]\n\n[1] Wu et al. A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks. Computer Methods in Applied Mechanics and Engineering, 403, 115671, 2023.", "other_strengths_and_weaknesses": "See above. In general the bar for new PINNs methods is high, as this area has been well-studied for a number of years and there are many new methods. This method doesn’t seem to be that different from many other methods, and so any proof-of-concept should also be on harder problems.", "other_comments_or_suggestions": "See above.", "questions_for_authors": "My questions are interspersed above, and some of it is summarized here:\n\n- For problems that are not the diffusion problem,  it is not stated exactly what the L2 relative error is compared to: what is the reference solution that the predicted solution is compared to, and how was this data generated?\n\n- This work is missing any discussion and comparison of a long line of work in doing adaptive sampling, where different parts of the domain are weighted differently (typically where PDE residuals are higher, more weight is put on this). This work is very related to what the authors are proposing. How do such  methods compare?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MUqTzeqwb3", "reviewer": "Reviewer_TArf", "summary": "The paper proposes CoPINN, a Cognitive Physics-Informed Neural Network that addresses the Unbalanced Prediction Problem (UPP) in PINNs. UPP arises from treating easy and hard samples (e.g., boundary vs. smooth regions) equally, leading to unstable training. CoPINN introduces three key components: (1) separable subnetworks for encoding 1D coordinates to reduce computational costs, (2) dynamic difficulty evaluation of samples via PDE residual gradients, and (3) a cognitive training scheduler that progressively focuses on harder samples. Experiments on Helmholtz, Klein-Gordon, and Diffusion equations demonstrate state-of-the-art performance, with significant error reductions. The method also shows scalability to high-dimensional PDEs and robustness in boundary regions.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewers for their constructive feedback. Below are our responses to the questions raised:\n\n**Q1: Questions For Authors(1)**\n\n**R1:** Our proposed CoPINN employs a multi-faceted approach to prevent overfitting to hard samples during training. First, the cognitive training scheduler ensures a controlled and gradual transition of focus from easy to hard samples through the hyperparameter $\\beta$, which is intentionally set to values less than 0.5 to avoid abrupt weight shifts and maintain a balanced emphasis across regions. Second, the physics-informed loss terms, specifically the initial condition loss $\\mathcal L_{ic}$ and boundary condition loss $\\mathcal L_{bc}$, act as implicit regularizers by enforcing physical constraints across the entire domain, thereby anchoring predictions to known conditions even as harder samples are prioritized. Finally, the robust generalization has been experimentally validated, with consistently low errors in boundary and smooth regions (e.g., Table 1, Figures 4, 9, and 10 in the original paper), confirming that the model does not overfit to specific challenging regions while maintaining accuracy across the entire solution space. This combination of gradual weighting, physics-based regularization, and experiment validation ensures stable and generalizable training.\n\n**Q2: Questions For Authors(2)**\n\n**R2:** The separable architecture of our proposed CoPINN inherently supports scalability to higher-dimensional PDEs by encoding each dimension independently through dedicated subnetworks, effectively reducing computational complexity from $O(N^d)$ to $O(dN)$, where $d$ is the number of dimensions and $N$ is the resolution per axis. While our experiments have focused on up to 4D systems (e.g., (3+1)-d Klein-Gordon) due to hardware limitations, the design principles theoretically generalize to higher dimensions. For instance, in 5D/6D scenarios, techniques like tensor decomposition (e.g., CP or Tucker formats) can optimize the aggregation of outer products in Equation 3, further enhancing efficiency. Our future work will explore these optimizations to validate scalability in extreme dimensions while maintaining the model’s accuracy and computational feasibility.\n\n**Q3: Questions For Authors(3)**\n\n**R3:** We agree that clarity in evaluation metrics is critical. The IMP metric in our paper is calculated as:\n\n$IMP=\\frac{|u_s-u_b|}{u_s}\\times 100\\\\%$\n\nwhere $u_s$ is the error of the suboptimal baseline and $u_b$ is the error of CoPINN. Since $u_b<u_s$ in all experiments (as CoPINN outperforms baselines), it is mathematically equivalent to the standard relative improvement formula. To eliminate ambiguity, we will revise the formula in the final version to:\n$$IMP=\\frac{u_s-u_b}{u_s} \\times 100 \\\\%$$\nremoving the absolute value. This aligns with standard practice and ensures no overstatement. For example: In Table 1, a baseline error of $(0.0311)$ SPINN(m) vs. CoPINN’s $(0.0006)$ gives $IMP=\\frac{0.0311-0.0006}{0.0311}\\times 100\\\\%\\approx 98\\\\%$, which correctly reflects the relative error reduction. In addition, we have explicitly reported both relative $L_2$ errors (e.g., Helmholtz's CoPINN: $0.0006$ vs. SPINN(m): $0.0311$), $RMSE$, and $IMP$ in Table 1 of the original paper. The performance is comprehensively demonstrated through two metrics (relative $L2$​ and $RMSE$), with the improvement magnitude illustrated via $IMP$.  We will clarify this in the text.\n\n**Q4: Other Comments Or Suggestions**\n\n**R4:** We have carefully corrected the typos in the paper:\n\n\"Klei-Gordon\" → \"Klein-Gordon\" (Page 14);\n\n\"$u(x,0)=x_1+x_2, x \\in \\Omega, x \\in \\Omega$\"→\"$u(x,0)=x_1+x_2, x \\in \\Omega$\" (Page 12).", "claims_and_evidence": "The claims made in the submission are supported by clear and convincing evidence.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria make sense for the problem at hand.", "theoretical_claims": "I checked the correctness of proofs for theoretical claims. The difficulty measure is heuristic, and the linear weight schedule (Eq. 8–9) is intuitive.", "experimental_designs_or_analyses": "I checked the soundness/validity of the experimental design and analysis. In this paper, a wide range of experiments are carried out on multiple PDEs and dimensions, and the experiments and the analysis of the experiments are sound and valid.", "relation_to_broader_scientific_literature": "The paper builds on PINNs (Raissi et al., 2019) and SPINN (Cho et al., 2023), integrating self-paced learning (Jiang et al., 2015) for difficulty-aware training.", "essential_references_not_discussed": "No relevant works are critical to understanding the main contribution (context) of the paper, but are not currently cited/discussed in the paper.", "other_strengths_and_weaknesses": "Strengths: \n\n1. This paper find the problem of existing PINN through experimental results (equal treatment of samples of different difficulty, resulting in non-optimal performance), and innovatively propose Cognitive PINN, and the experimental results prove the effectiveness of the proposed method.\n\n2. This paper integrates self-paced learning into PINN to effectively solve the unbalanced prediction problem by dynamically prioritizing sample difficulty during training.\n\n3. The proposed method demonstrates scalability to high-dimensional PDEs (e.g., 4D systems), overcoming memory constraints faced by baseline approaches.\n\n4. In this paper, experimental results in different PDEs (Helmholtz, Klein-Gordon, Diffusion) show significant error reduction and robustness in the boundary region.\n\nWeaknesses: \n\n1. In this paper, the proposed cognitive scheduler and difficulty evaluation mechanism are not theoretically proved, so the optimality of this method has not been verified.\n\n2. The proposed methods are sensitive to hyperparameters such as β, which presents tuning challenges, especially in balancing attention between simple and hard samples of different PDEs.\n\n3. Progressive weighting of hard samples risks overfitting to localized boundary phenomena at the expense of global solution accuracy, especially in datasets with small difficulty variations. In the method proposed in this paper, progressive weighting of hard samples may overfit local boundary phenomena at the expense of the accuracy of the global solution, which may be more obvious in datasets with small difficulty variations.", "other_comments_or_suggestions": "This article has some typos, including but not limited to: \"Klei-Gordon\" → \"Klein-Gordon\" (Page 14).", "questions_for_authors": "1. How does CoPINN mitigate overfitting to hard samples in later training stages? In other words, how to ensure that the prediction accuracy of easy samples does not decline in the later training period?\n\n2. Can CoPINN scale to PDEs beyond 4D (e.g., 5D or 6D)? This is crucial for the scalability of the proposed method.\n\n3. Why to use the IMP metric instead of standard relative improvement? This may lead to an overstatement of the experimental results of the proposed method.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6lcQCU0AG4", "reviewer": "Reviewer_wy2g", "summary": "The paper proposes an adaptive sample weighting strategy for physics-informed neural networks. As a measure of difficulty for a sampling point, the magnitude of the (input) gradient of the PDE residual is proposed. The authors suggest to train PINNs via assigning high sample weights to easy samples early on in the training process and then gradually shift towards hard samples, reducing the weight for the easy ones. The authors then compare their proposed method on a number of example equations and benchmark against several baselines. The experimental results suggest improved accuracy compared to the baseline method.\n\n## update after rebuttal\nThe authors were responsive and could clear some of my doubts and misunderstandings. I will raise my score.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We greatly appreciate your valuable comments. Below is our point-by-point response.\n\n**Q1: Essential References Not Discussed & Questions For Authors (1)**\n\n**R1:**\n\n1) We will include the discussion about adaptive sampling in the Introduction and Related Work Sections of the next version. The key to adaptive sampling is to improve the learning efficiency and accuracy of the neural network by dynamically selecting or reallocating sampling points. Specifically, for [1] you mentioned, it balances regions of high and low residuals by dynamically resampling. For [2] you mentioned, it uses resampling to increase the sampling points in high PDE residual regions to avoid so-called propagation failures. We further add a review of other literature: [3] proposes a risk min–max framework to do adaptive sampling to speed up the convergence of the loss and achieve higher accuracy. \n2)  From a technical perspective, adaptive sampling technology uses PDE residuals to adjust sample density. Our CoPINN uses the gradient of PDE residuals to estimate the learning difficulty of samples, thereby optimizing the learning process of the neural network. From a conceptual perspective, adaptive sampling technology increases the density of difficult samples while reducing the density of simple samples, thereby fully learning difficult samples. Unlike them, our CoPINN does not involve changes in sampling and sample density. CoPINN directly evaluates the learning difficulty of samples and imitates the human cognitive process to learn from easy to difficult, thereby improving accuracy. In general, adaptive sampling focuses on adjusting the number of difficult and easy samples, and CoPINN uses the early learning of simple samples to fully mine information from difficult samples in the later stage of learning. They are two different technical routes and there is no conflict.\n3) [2]proposes that the correct solution of PINN should propagate from the boundary to the center, otherwise, it will cause trivial solutions. Therefore, PINN training should focus on the boundary. Our CoPINN doesn‘t ignore boundary conditions and initial conditions during training. Although CoPINN pays less attention to difficult samples (usually at the boundary) according to the gradient of PDE loss in the early stage of training, it does not ignore difficult samples. As learning progresses, CoPINN pays more attention to difficult samples. Therefore, CoPINN doesn't ignore difficult samples in the early stage of learning, which will not cause trivial solutions mentioned by the reviewer and is therefore reasonable.\n\n[1]A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks.\n\n[2]Mitigating Propagation Failures in Physics-informed Neural Networks using Retain-Resample-Release (R3) Sampling.\n\n[3]A Gaussian mixture distribution-based adaptive sampling method for physics-informed neural networks.\n\n**Q2: Questions For Authors (2)**\n\n**R2:** Regarding your question, \"A quantification of the randomness of the training process \", we don‘t understand what it means. Based on your comment, we guess that you are doubtful about the stability of our CoPINN. In our paper, all quantitative experiments are repeated 5 times with different random seeds, and the average values ​​are reported. To demonstrate the stability of CoPINN, below, we report the mean and standard deviation of the relative $L_2$ error on the Helmholtz dataset. The results show that CoPINN achieves both the lowest mean and standard deviation in relative $L_2$ error, and confirm that CoPINN's training process is more robust than competing methods. If we misunderstood, please clarify, and we will provide further details.\n\n|              | $16^3$            | $32^3$            | $64^3$            | $128^3$           | $256^3$           |\n| ------------ | ----------------- | ----------------- | ----------------- | ----------------- | ----------------- |\n| AHD-PINN     | $0.2108\\pm0.0181$ | $0.1903\\pm0.0418$ | $0.1871\\pm0.0372$ | O/M               | O/M               |\n| RoPINN       | $0.4059\\pm0.0353$ | $0.3338\\pm0.0502$ | O/M               | O/M               | O/M               |\n| FPINN        | $0.3862\\pm0.0714$ | $0.3502\\pm0.0676$ | $0.3097\\pm0.0580$ | O/M               | O/M               |\n| SPINN        | $0.1177\\pm0.0451$ | $0.0809\\pm0.0104$ | $0.0592\\pm0.0316$ | $0.0449\\pm0.0337$ | $0.0435\\pm0.0280$ |\n| SPINN(m)     | $0.1161\\pm0.0084$ | $0.0595\\pm0.0113$ | $0.0360\\pm0.0082$ | $0.0300\\pm0.0016$ | $0.0311\\pm0.0100$ |\n| CoPINN(Ours) | $0.0172\\pm0.0052$ | $0.0050\\pm0.0030$ | $0.0016\\pm0.0006$ | $0.0007\\pm0.0002$ | $0.0006\\pm0.0001$ |\n\n**Q3: Other Comments Or Suggestions**\n\n**R3:** Thanks, we will increase the resolution of Figure 1 in the next version.", "claims_and_evidence": "The improved accuracy and training is illustrated by a reasonable set of numerical simulations.", "methods_and_evaluation_criteria": "The set of numerical examples is reasonable and the problems considered are of suitable difficulty to be interested. The comparison to existing methods in the literature is sound. The manuscript would however benefit from a quantification of the randomness (over the network initialization) in the training process. The authors might consider to do this for a subset of the experiments as an evaluation for the full set of experiments is likely unreasonably computationally expensive.", "theoretical_claims": "Not applicable.", "experimental_designs_or_analyses": "The considered PDE problems are reasonable.", "relation_to_broader_scientific_literature": "Some related literature is not discussed: There are a number of works concerning adaptive sampling for PINNs, see for instance [1], [2] and the references therein. Interestingly, these works take a different approach, focusing on hard samples more than on easy samples via adapting the sample distribution based on the PDE residual. This seems somewhat contradictory to the presented findings and requires a thorough discussion. \n\nMoreover, the authors of [1] also present a convincing argument for focusing on high residual/difficult regions first : For many examples — take for instance the convection equation example in [1] — the PDE residual without the boundary conditions possesses trivial minimizers, namely constant functions. Focusing on interior regions first seems therefore not reasonable.\n\n[1] https://arxiv.org/pdf/2207.02338\n[2] https://www.sciencedirect.com/science/article/abs/pii/S0045782522006260", "essential_references_not_discussed": "See above, especially [1].", "other_strengths_and_weaknesses": "Strengths:\n- Convincing numerical results.\n\nWeaknesses:\n- The \"conflict\" with the existing literature should be discussed and ideally resolved. As mentioned above, existing sampling methods focus on hard samples, whereas the present work starts with easy samples.\n- A quantification of the randomness of the training process should be provided, at least for some experiments. Is the training process more brittle than the baseline methods?", "other_comments_or_suggestions": "- The resolution of Figure 1 should be increased.", "questions_for_authors": "See above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3jBhb4js5P", "reviewer": "Reviewer_JLZV", "summary": "The paper presents a novel framework called Cognitive Physical Informed Neural Network (CoPINN) to address the Unbalanced Prediction Problem. CoPINN employs separable subnetworks to encode one-dimensional coordinates, aggregates them to predict multi-dimensional variables, and dynamically evaluates sample difficulty based on PDE residual gradients. It progressively optimizes sampling regions from easy to hard using a cognitive training scheduler, significantly reducing prediction errors in challenging areas.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your detailed comments. We believe the following point-to-point response can address all the concerns:\n\n**Q1: Weaknesses (1) and Questions For Authors(4)**\n\n**R1:** Compared with Helmholtz and (2+1)-d Klein-Gordon datasets, (3+1)-d Klein-Gordon and Diffusion datasets are relatively less challenging. Therefore, almost all the methods achieve good results. In fact, because most of these two datasets are easy, which leads to limited mining of difficult samples by our CoPINN, and the performance advantage is relatively unobvious. In fact, CoPINN yields improvements of $50\\\\%$ (Klein-Gordon in (3+1)-d) and $10\\\\%$ (diffusion equations).\n\n**Q2: Weaknesses (2)**\n\n**R2:** We claim that PINN and FPINN can solve high-dimensional PDEs. However, as the number of training (collocation) points increases, especially for higher-dimensional or more complex problems, their computational burden becomes more pronounced. To solve this problem, we adopt a separable architecture to efficiently handle high-dimensional PDEs. In addition, we also compare with SPINN (which is good at solving high-dimensional PDEs), so the experimental setting is fair.\n\n**Q3: Questions For Authors(1)**\n\n**R3:** Separable learning and Cognitive Training Scheduler are two independent modules. To be specific, Separable Learning mitigates the computational burden in high-dimensional PDE solutions. Conversely, the Cognitive Training Scheduler assesses sample-level difficulty, enabling neural networks to fit samples from easy to hard when solving PDEs.  Cognitive Training Scheduler is plug-and-play and can be used for the vanilla PINN.\n\n**Q4: Questions For Authors(2)**\n\n**R4:** We apologize for the lack of clarity. Following the setup of SPINN, for Helmholtz, (2+1)-d Klein-Gordon, (3+1)-d Klein-Gordon, Diffusion and flow Mixing 3D equations, we set the sampling points to $100^3, 100^3, 50^3, 101^3$, and $100^3$ during testing, respectively. We will include this in the implementation details of the next version.\n\n**Q5: Questions For Authors(3)**\n\n**R5:** Through parameter sensitivity analysis in Figure 5, we find that the performance is better when the $\\beta$ value is $0.01$ to $0.00001$. In fact, if $\\beta$ is fixed to $0.001$, CoPINN achieves SOTA on all datasets.\nTherefore, for other PDEs, such as the N-S equation, we recommend that the $\\beta$ value be selected in $\\\\{0.01, 0.001, 0.0001, 0.00001\\\\}$.\n\n**Q6: Questions For Authors(5)**\n\n**R6:** In Section 3.3 of our original paper, we conduct an ablation study to analyze the effectiveness of each component in the cognitive training scheduler of our CoPINN. CoPINN-1 represents using the original loss function, i.e., removing the Cognitive Training Scheduler. The results in Table 2 show that the performance of CoPINN-1 is lower than our proposed CoPINN.", "claims_and_evidence": "yes, this paper is supported by clear and convincing evidence.", "methods_and_evaluation_criteria": "yes, the proposed method and evaluation criteria make sense.", "theoretical_claims": "Yes, I checked the theoretical part of self-paced learning in this paper.", "experimental_designs_or_analyses": "Yes, I checked the soundness and validity of all experimental designs and analyses in this paper.", "relation_to_broader_scientific_literature": "Although this paper only focuses on the field of physics-informed neural networks (PINNs) and uses self-paced learning (SPL) to improve the overall performance of PINNs, the proposed SPL method has important reference value for other fields, such as classification, clustering, and retrieval.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Strengths:\n\n1.The paper clearly articulates the research motivation with illustrative examples.\n\n2.The paper has sufficient experiments, for example, comparative experiments with seven latest methods on five datasets.\n\n3.The proposed method achieves significant improvements over the SOTA, for example, about 90% improvement on Helmholtz and about 70% improvement on (3+1)-d Klein Gordon Equation.\n\n4.This paper innovatively uses self-paced learning to solve the key unbalanced prediction problem (UPP) in the existing PINNs methods.\n\n5.The proposed method is simple but effective.\n\nWeaknesses:\n\n1.Although the results on the Helmholtz and (2+1)-d Klein-Gordon datasets are significantly better than those on the SOTA method, why is the improvement on the (3+1)-d Klein-Gordon and Diffusion datasets limited? This requires further analysis.\n\n2.The authors compared their method with PINNs and FPINNs, which are not good at solving high-dimensional PDEs, and the experimental setting seems unfair.\n\nSee questions for more.", "other_comments_or_suggestions": "No.", "questions_for_authors": "1.CoPINN consists of Separable learning and Cognitive Training Scheduler. Are the proposed Cognitive Training Scheduler and Separable learning coupled or can they be separated? That is, can the Cognitive Training Scheduler be used for the vanilla PINN?\n\n2.The paper mentions using different numbers of sampling points, i.e. 16^3, 32^3,...,256^3, to train the neural network. Is the same number of sampling points also used for testing? Or is it another number?\n\n3.Hyper parameter $\\beta$ has a great impact on the performance of the algorithm. What value is recommended for solving other equations, such as the NS equation?\n\n4.Although the results on the Helmholtz and (2+1)-d Klein-Gordon datasets are significantly better than those on the SOTA method, why is the improvement on the (3+1)-d Klein-Gordon and Diffusion datasets limited? This requires further analysis.\n\n5.What is the performance removing Cognitive Training Scheduler?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: CoPINN presents a compelling and original contribution to the PINNs landscape. The paper is well-motivated, methodologically sound, and thoroughly evaluated. It introduces self-paced learning in the context of PDE solving, a conceptually novel direction that offers substantial improvements over existing methods, particularly in handling difficult regions like boundaries or singularities.

The rebuttal was professional, detailed, and addressed every reviewer concern effectively, including additional experiments where necessary. The work is highly relevant not only to PINNs, but also to broader efforts in difficulty-aware training and adaptive learning strategies in scientific machine learning.
Decision: Accept (spotlight poster)