AC-PKAN: Attention-Enhanced and Chebyshev Polynomial-Based Physics-Informed Kolmogorov–Arnold Networks
Authors: Hangwei Zhang
Zhimu Huang
Yan Wang
Venue: Accepted by TMLR ()

Abstract: Kolmogorov–Arnold Networks (KANs) have recently shown promise for solving partial differential equations (PDEs). Yet their original formulation is computationally and memory intensive, motivating the introduction of Chebyshev Type-I-based KANs (Chebyshev1KANs). Although Chebyshev1KANs have outperformed the vanilla KANs architecture, our rigorous theoretical analysis reveals that they still suffer from rank collapse, ultimately limiting their expressive capacity. To overcome these limitations, we enhance Chebyshev1KANs by integrating wavelet-activated MLPs with learnable parameters and an internal attention mechanism. We prove that this design preserves a full-rank Jacobian and is capable of approximating solutions to PDEs of arbitrary order. Furthermore, to alleviate the loss instability and imbalance introduced by the Chebyshev polynomial basis, we externally incorporate a Residual Gradient Attention (RGA) mechanism that dynamically re-weights individual loss terms according to their gradient norms and residual magnitudes. By jointly leveraging internal and external attention, we present AC-PKAN, a novel architecture that constitutes an enhancement to weakly supervised Physics-Informed Neural Networks (PINNs) and extends the expressive power of KANs. Experimental results from nine benchmark tasks across three domains show that AC-PKAN consistently outperforms or matches state-of-the-art models such as PINNsFormer, establishing it as a highly effective tool for solving complex real-world engineering problems in zero-data or data-sparse regimes. The code will be made publicly available upon acceptance.

Published in Transactions on Machine Learning Research (01/2026)
AC-PKAN: Attention-Enhann Enhanced and Chebyshev Polynomial- Based Physics-Informed Kolmogorov-Arnold Networks
Hangwei Zhang Zhimu Huang
Yan Wang!*
Institute for AI Industry Research, Tsinghua University
Beihann University
BBejjing Institute of Technology
vangyandair. tsinghua. edu. en, langgelzhanggluua.edi.cc cn, 11202224210bit. edu. cn
Reviewed OpenReview: bttpe://ppaaaeeieeeeeee////rrrtr
Abstract
Kolmogorov- Armold Networks (KANs) have recently shown promise for solving partial differential equations (PDEs). Yet their original formulation is computationally and memory intensive, motivating the introduction of Chebyshev Type-I-based KANs (KAes) Although ChebylKANs have outperformed the vanilla KANs architecture, our rigorons theoretical analysis reveals that they still suffer from rank collapse, ultimately limiting their expressive capacity. To overcome these limitations, we enhance ChebylKANs by integrating wavelet-acttivtted MLPs with learnable parameters and an internal attention mechanism. We prove that this design preserves a full-rank Jacobian and is capable of approximating solutions to PDEs of arbitrary order. Furthermore, to alleviate the loss instability and imbalance introduced by the Chebyshev polynomial basis, externally incorporate Residual Gradient Atiention (RGA) mechanism that dynamically re-weights individual loss terms according to their gradient norms and residual magnitudes. By jointly leveraging internal and external attention, we present AC-PKAN, a novel architecture that constit ules an enhancerent to weakly supervised Physics--nformed Neural Networks (PINNs) and extends the expressive power of KANs. Experimental results from nine benchmark tasks across three domains show that AC-PKAN outperforms matches state-of-the-art models such as PINNsFormer, establishing it as highly effective tool for solving complex real-world engincering problems in zero-data data-sparse regimes. The code is publicly available at com/FoogradioAACPKNN /t/p/////////tthooooooo//ooo/a//
Introduction
Numerical solutions of partial differential equations (PDEs) are essential in science and engincering ZZeenkiewizz
Fornberg. 1998 Brebbia et al. 2012). Physics-informed neural networks (PINNs) Lagaris et al. 1998 Raissi et al. 2019) have emerged as a promising approach in scientific
& Taylor 2005 Liu 2009
machine learning (SciMT.), especially when data are unavailable or scarce. Traditional PINNs typically employ multilayer perceptrons (MLPs) (Cybenko] 1989) due to their ability approximate nonlincar
functions (Hornik et al. 1989 and their success various PDE-solving applications (Yu et al. 2018 Han
et al. 2018
However, PINNs encounter limitations, including difficulties with multi-scale phenomena Kharazmi et al. 2021), the curse of dinensionalityy in high-dimenniooall spaces (Jagtap & Karniadakis 2020), and challenges with nonlinear PDEs (Yuan et al.) 2022). These issues arise from both the complexity of PDEs and limitations in PINN architectures and training methods. To address these challenges, existing methods focus on improving both the internal architecture of PINNs and their external learning stratogies. Internal improvements include novel architectures like Quadratic Residual Networks (Qres) [Bu & Karpatne 2021

---

Published in Transactions on Machine Learning Research (01/2026)
First-Layer Sine (FLS) (Wong et al. 2022). and PINNsformer (Zhao et al. 2023 External strategies are discussed in detail in Section 2 Nevertheless, traditional PINNs based on ML.s still suffer from issues
like lack of interpretability (Crammer 2023), overfitting, vanishing or exploding gradients, and scalability problems (Bachmann et al. 2024p. As an alternative, Kolmogorov-Armold Networks (KANs) (Liu et al.
2024b), inspired by the Kolmogorov-Aiee Arnold representation theorem (Kolmogorov 1961 Braun & Griebel
2009), have been proposed to offer greater accuracy and interpretability. KANs can be viewed as a combination of Kolmogorov networks and MLPs with learnable activation functions (Köppen 2002 Sprecher Draghici
2002 Various KAN variants have emerged by replacing the B-spline functions (SS) 2024 Bozorgasl & Chen
2024
Xu et al. 2024a). Although they still face challenges Yuu et al. 2024). KANs have shown promise
in ad addressing issues like interpretability (Liu et al. 2024a) and catastrophic forgetting (Vaca-Rubio et al,
2024] in learning lasks (Samadi et al. 2024). Recent architectures like KINN (Wang el al.) 2024b) and
DeepOKAN (Abueidda et al. 2024) have applied KANs to PDE solving with promising results.
Despite the potential of KANs, the original KAN suffers from high memory consumption and long training times due to the use of B-spline functions (Shukla et al. 2024). To address these limitations, we propose the Eaattien- and Chebyshev Sarnoommal-Based Plysics-Informed Informed Kolmogoroy-Aroold Arnold Networks (AC-PKAN). Our approach replaces B-spline functions with first-kind Chebyshev polynomials, forming the ChebylKAN layer (SS) 2024), eliminating the need for grid storage and updates. Nevertheless, networks composed solely of stacked ChebyyKKNN layers exhibit pronounced rank diminution (Feng et al., 2022). ). By integrating Cheby IKAN with linear layers and incorporating internal attention mechanisms derived from input features, AC-PKAN addresses these limitations while efficiently modeling complex nonlinear functions and selectively emphasizing distinct aspects of the input features at each layer. Additionally, we introduce an external attention mechanism that adaptively reweights loss terms according to both gradient norms and point-wise residuals, thereby counteracting the large polynomial expansions and gradient maguitudes inherent in ChebyLKAN, mitigating residual imbalance and gradient flow stiffness, and ultimately enhancing training stability and efficiency. To our knowledge, AC-PKAN is the first PINN framework to integrate internal and external attention mechanisms into KAN layers, effectively addressing many issues of original KANs and PINNs. Our key contributions can be summarized as follows:
Rigorous theoretical analysis. We provide the first formal study of Cheby1KAN depth, proving upper bounds on each layer's Jacobian rank and showing that stacked layers suffer an exponential rank-attennation in depth, which establishes the theoretical limits that motivate our design.
Attention-enhanced internal architecture. To overcome rank collapse and the zero-derivative pathology, we introduce AC-PKAN: ChebylKAN layers are interleaved with linear projections, learnable wavelet activations, and lightweight feature-wist attention module, together guarantecing full-rank Jacobians and non-vanishing derivatives of any finite order.
Residual-Gradient Attention (RGA). Externally, we devisc an adaptive roweightttg strategy that couples point-wise residual magniludes with gradient norms. It dynamically balances competing objectives, alleviates gradient stiffness, and accelerates convergence of physics-informed neural networks.
Comprehensive experimental validation. Across three categories of nine benchmark PDE problems and twelve competing models, AC-PKAN attains the best or near-best accuracy in every case, demonstrating superior generalization and robustness to PINN failure modes.
2 Related Works
External Learning Strategies for PINNs. Most advances in physics-informed neural networks improve training from the outside while keeping an MLP-like backbone. Loss rebalancing methods such as PINN-LRA, PINN-NTK and residual-based adaptation Wang et al. 2021 2022b Anagnostopoulos et al.] 2024) adjust PDE, boundary and initial terms with gradient or NTK statistics to correct the mismatch among losses. Sampling based approaches follow the same goal. AAS selects collocation points using adversarial optimal transport (Tang et al. 2023), RoPINN applies regional Monte Carlo sampling (Pan al. 2024 while RAR
2

---

Published in Transactions on Machine Learning Research (01/2026)
and PINNACLE resample high-residual areas and co-optimise all point types (Wu et al. 2023 Lau al. 2024h. Other works modily the objective or the training domain. gPINN adds gradient terms to enforce the PDE (Yu et al. 2022), vPINN uses a variational form (Kharazmi et al. 2019), LAAF and GAAF tune activations during training to accelerate convergence (Jagtap et al. 2020a bb. and FBPINN and hp- VPINN decompose the domain to make multi-scale problems tractable (Moseley et al. 2023 Kharazmi et al. 2021
These methods stabilise PINNs but they still rely on feature spaces that can degenerate when several operators or high order derivatives are imposed. Our approach keeps these mature external pipelines while adding an internal mechanism whose role is to preserve expressive bases during physics-informed optimisation.
KAN and Chebyshev-based Variants. Kolmogorov-Aroll Networks make the activation on edges learnable and can approximate operators with smaller models (Liu et al. 2024b). Follow up designs replace splines with faster or more structured bases, such as FastKAN with RBFs Li 2024), ChebylKAN IKAN and Cheby2KAN with first and second kind polynomials for oscillatory targets (SS) 2024), rKAN and fKAN with rational or fractional Jacobi bases (Aghaci) 2024 2025), and FourierKAN with Fourier modes (Mchrabian
et al. 2024 Surveys place these models in a wider landscape of Kolmogorov-inspieed approximators (Guilhoto strong speed-accuracy choice (SS 2024h. Yet most of these works optimise for approximation, interpretability or inference cost and do not discuss how to keep KANs stable when trained with collocation-based PDE losses. AC-PKAN is positioned at this intersection. retains the approximation benefits of Chebyshev KANs, but augments them with internal feature re- iejinnection, frequency-aware activation and rank-aware gating so that the model remains expressive under the same loss reweighting and sampling strategics used by advanced PINNs.
& Perdikaris 2024b, and preliminary benchmarks still report Chebyshev-base based variants
3 Motivation and Methodology
Preliminaries: Let Q C Rd be an open set with boundary 8n. Consider the PDE:
Diu(a, t)] = f(x,t), (x, t) e Q Blu(,, t)] = t), (x,t) e an,
(1)
where u is the solution, D is a differential operator, and B represents boundary /initial constraints or available data samples. Let à be neural network approximation of u. PINNs minimize the loss:
Nr
No
CPINSS = A Σ
[D(ū(z..tt) - f(x,,, 2.)1 + A Σ [B(((x, - g(x,,
(2)
where (() C Ω are residual points, {( (:)) C 02 are boundary constraints or available data samples, and År, ds balance the loss terms. The goal is to train â to minimize LPINNs using machine learning techniques.
3.1 Chebyshevl- Kolmogorov-Arnole Network Layer
Unlike traditional Kolmogorov-Aroold Networks (KAN) that employ spline coefficients, the First-kind Chebyshev KAN Layer leverages the properties of mesh-free Chebyshev polynomials to enhance both computational efficiency and approximation accuracy (SS 2024 Shukla et al. 2024).
Let x E dd denote the input vector, where dn is the input dimensionality, and let dout be the output dimensionality. Cheby KAN aims lo approximate the mapping x + E Beda using Chebyshev polynomials up to degree N. For I E [-1, 1], = O, 1,......A N, the Chebyshev polynomials of the first kind, (x),,, are defined as:
T, (z) cos (n arccos())))
To ensure the input values fall within the domain |-1, 1], Cheby IKAN applies the hyperbolic tangent function for normalization:
x tanh(x).
(4)
3

---

Published in Transactions on Machine Learning Research (01/2026)
Defining matrix of functions (xx Rdoot where each clement (x,)(;s) depends solely the i-th normalized input component 3y for k = ...,..., dout, i = 1, 1,2, .....,... din:
$ ( = Σ
Ckin (#:)) ==0
(5)
Here, C,ei, are the learnable coefficients. The output vector y Rdou is computed by summing all imput dimensions:
V/:
D (Z:), h 1,2,..... ................................ dout.
(6)
For network comprising multiple Chebyshev KAN layers, the forward computation can be viewed recursive application of this process. Let x; denote the input to the /-th layer, where / = 0,.,..., L- --1 1. After
applying hyperbolic tangent function to obtain & = tanh(x;), the computation procoeds as follows:
p1))2()))
m. (-) 222m)
XI+1
Xi
(7)
A general cheby IKAN network is a composition of layers: given an input vector xo E IR"0, the overall output of the KAN network is:
Cheby1KANg (#L-1 LL-20 o.... $ $o)x.
(8)
In order to prevent gradient vanishing induced by the use of tanh, we apply Layer-Normalization after ChebylKAN Layer.
Compared to the original B-spline-based KANs, Chebyshev polynomials of the first kind in Equation 3
concentrate
spectral energy in high frequencies with frequencies that increase linearly with the polynomial order (Xu et al.
2024b
Xiao et al. 2024), while maintaining global orthogonality over the interval []1
0
n, m n = 0, π
T. (z) 2
m + 0.
This global support and slower decay of high-frequency components outperform locally supported B-splines, which lack global orthogonality and have rapidly diminishing high-frequency capture. Furthermore, Cheby IKAN layers require only coefficient matrix of size (input dim, output dim, degree+1), whereas B-spline-based KANs necessitate storing grids of grid_sizee22 2 spline order+1) and coefficient matrices of size (out features, in features, grid size+ aadine__odery in addition to generating polynomial bases, solving local interpolation systems, and performing recursive updates to achieve high-order interpolation within their support intervals (Liu el al. 2024b). Hence, the Cheby IKAN layer significantly reduces both computational and memory overhead compared to the original B-spline-based KANs, while more effectively capturing high-frequency features. More details can he found at Appendix B
size (in features, grid size+2>
3.2 Rank Diminution in Cheby1KAN Networks
While ChebylKAN layers offer significant advantages, networks composed solely of stacked Cheby1KAN layers,
presented in equation exhibit pronounced rauk diminution (Feng el al. 2022 Consequently. these networks sulfer a
reduced capacity for feature representation, leading to severe information degradation and loss. We present detailed
derivation and proof of this phenomenon below (Roth & Liebig 2024. The complete ma matherntical derivations are provided in Appendix A
4

---

Published in Transactions on Machine Learning Research (01/2026)
Definitions. Consider the 1-th Cheby IKAN layer with imput æt Rd and output uri E R441, The layer mapping is defined by
1H Ca
(10)
where Tn are Chebyshev polynomials and Ciakiin are learnable coeflicients. The Jacobian E 14441 x4 has entries
Ji,K, Σ (anannTa(taann ttah -t,n)
(11)
For an I-layer network, the total Jacobian is
Jeotal JL-1JE-2""
(12)
Theorem 1 (Single Cheb1KAN Layer Rank Constraint). The Jacobian J. satisfies
rank(Ji) S min d( + 1)}-
(13)
Theorem (Nonlinear Normalization Effect).. The normalization tanh(z) in Cheby1KAN layer reduces the numerical rank Rank, (J) of the Jacobian.
Theorem (Exponential Decay in Infinite Depth). When the coefficients CLasi drawn from mutually independent Gaussian distributions, the numerical rank of Jestal decays exponentially to as the depth L. of the Cheby IKAN network increases.
In summary, Cheby IKAN nelworks inherently experience rank diminution due to various factors. Collectively, the bounded rank per Cheby1KAN layer (Theorem 1 the attenuation from tanh(-) (Theorem 2), and the multiplicative rank bound culminate in exponential rank decay (Theorem 3). thereby demonstrating the inherent rank diminution ChebylKAN networks.
Therefore, there is a significant need to improve the internal structure of models based on the Cheby layer, which will be discussed in detail in Section 3.3 Additionally, to address some computational limitations associated with the use Cheby IKAN, we propose an external attention mechanism, which will be elaborated in Section [34] By incorporating both internal and external attention mechanisms,, AC-PKAN model fully leverages the advantages of Chebyshev Type-I polynomials while overcoming their initial drawbacks.
3.3 Internal Model Architecture
resolve the Rank Diminution issue arising from direct stacking of Cheby1KAN layers in network architectares, we propose the AC-PKAN model, featuring an attention-enhanced framework (Wang et al. 2021 2024a designed to mitigate feature space collapse. The architecture synergistically combines linear transformations for input-output dimensional modulation, state-of-the-art activation functions, and residual-augmmnteed ChebylKAt layers. These components are collectively designed to preserve hierarchical feature diversity while capturing high-order nonlinear interactions and multiscale topological dependencies inherent in complex data structures. The algorithmm' details are provided in Algorithm
Linear Upscaling and Downscaling Layers To modulate the dimensionality of the data, the model employs linear transformations both the input and output stages. The linear layer designed achieve a hybridization of KAN and MLP architectures. Tts role both an initial and final projection is inspired by the Spatio-Tempore Mixer linear layer in the PINNsformer model (Zao et al.) 2023). which enhances spatiotemporal aggregation. The input features projected into nigher-dimensional space, and the final network representation a is mapped the output space via:
has = WembX + bembs y W (L + bouts
(14)
where W emb E Re Xdon bemb R Rmoddl W
Wous oou Roou Xdandden, and bout Rout learnable parameters.
5

---

Published in Transactions on Machine Learning Research (01/2026)
Adaptive Activation Function We adopt the stale-of-lhe-art Wavelel activation function in the field of PINNs, us detailed in (Zhao et al. 2023 Inspired by Fourier transforms, it introduces non-linearily and effectively captures periodic patterns:
Wavelet(x) = wi (z) + cos(z),
(15)
where wi and My learnable parameters initialized one. This activation integrates Fourier feature embedding (Wang et al. 2023) and sine activation [Wong et al,, 2022). When applied to encoders U and V, the Wanelet activation preserves the gradient benefits introduced by the triangular activation function while modulating phase and magnitude, This enbancement boosts representational capacity and facilitates adaptive Fourier embedding, thereby more effectively capturing periodic features and mitigating spectral bias.
Attention Mechanism An internal attention mechanism is incorporated by computing two feature representations, U and V, via the Wavelel activation applied linear transformations of the embedded impuls:
U = Wavelet Ou + by), V = Wavelet (y, + bv).
(16)
where 0U By E mondel Xihiiden and bu, by R hhidden are learnable parameters.
Attention Integration the following equations:
The attention mechanism integrates U and V iteratively across Chebyl KKNN layers using
(1) - (111) a =1 0g @
(V+1).
(17)
where ())) U and denotes deeennt-wse multiplication. Here, H E Nxdaidden is the output of the L-th ChebylKAN layer after LayerNormallzation, and N is the number of nodes.
Algorithm Internal AC-PKAN Forward Pass
Data: Input data ChebylKAN layer parameters, Wavelet activation parameters
Initialization: Randomly initialize weights W Wemb, Ov, Oy, Woot and biases bemb; by, by, bout.
I: Input embedding:
bo < WembX + banh
2: Compute representations:
U
Whv by), Vc WW I bv)
3: Initialize attention: ("), U
4: for | = to L do
5: H(() Tayer Cheby 1 K KANL (aaaye(((
6: ag - H()) ) + (1
(8)
7: (i) < (1 ) outa (V+1)
8: end for
9: Output prediction:
y Wouta bout
Approximation Ability Our AC-PKAN's inherent attention mechanism eliminates the need for an additional bias function b(x) required in previous KAN models to maintain non-zero higher-order derivatives (Wang al. 2024b).
This reduces model complexity and parameter count while preserving the ability to seamlessly approximate PDEs of arbitrary finite order. By ensuring non-zero derivatives of any finite order and invoking the Kolmogorov Arnold representation theorem, our model can approximate such PDEs.
Let N be an AC-PKAN model with L layers (L 2 2) and infinite width. Then, the output y = (xx has non-zero derivatives any finite- order with respect to the input
Proposition 1.
Then we prove that the Jacobian matrix the AC-PKAN model is full-rank, thereby rigorously precluding degenerate directions in the input space.
Proposition 2. (x(()
Let N' be an AC-PKAN model with L layers (L 2 2) and infinite width. Then, the Jacobian matric is full rank the input space Rd.
BN. 0zy
mxd
This properly effectively addresses the internal rank diminution issue of Cheby LKAN networks discussed in Section 3. and also ensures stable gradient backpropagation, thereby preventing lele-ddeiiiinc--inddee training failures in AC-PKAN.T The complete mathematical derivations are provided in Appendix A

---

Published in Transactions on Machine Learning Research (01/2026)
Physics- Informed
Lenar
Liyer
Ly (8)
120
- at
RBA
ag
Ly
L(
Lg (
CheLyMAN-UUN
1x
RGA Attention
Figure 1: Architecture of the complete AC-PKAN model. It combines its internal atlention architecture with external attention strategy, yielding weighted loss optimized to obtain the predicted solution.
3.4 Residual-and-Gradiee Based Attention
In the canonical PINN formulation, the loss is split into an unlabeled PDE-residual term L, and labeled term La that enforces boundary/inite constraints and matches available data samples. To improve optimization efficiency and accuracy and to correct the loss imbalance introduced by Chebyshev bases, we propose Residual Gradient Atlention (RGA), adaptive mechauism that rescales each loss lerm according its residual magnitude and its gradient This approach ensures balanced and efficient optimization, particularly addressing challenges with boundary and initial condition losses.
Residual- Baaiduda-BBadd Attention (RBA) Residual-Basel Attention (RBA) dynamically amplisies loss terms with the largest poinl-wise residuals, assigning tensor of weights WRBA to each loss component Li (r, location
(Anagnostopollos et al. 2024
RBA RDA <- (1-1)) wiy max, Cigl W.
(18)
where 17 the RBA learning rate and maxy/Cs.5l normalizes by the maximal residual. Residual- Based Attention (RBA) is lightweight, pointwise weighting scheme that complements the ChebylKAN layer. Chebyl captures strong nonlinear structure and complex distributions but can exhibit slow or unstable convergence. RBA inserts a self-adjusting feedback loop that reweights local training signals according to residual statistics, focusing learning on poorly fitted locations and reducing the influence of noisy or saturated terms. This synergy alleviates numerical optimization difficulties and enhances global convergence efficiency.
Beadient-Related Attention (GRA) Due to the Cheby layer's utilization of high-order Chebyshev polynomials, large coefficients and derivative magnitudes introduced, resulting in increased maximum eigenvalue of the Hessian and exacerbating gradient flow stiffness. Additionally, nonlinear operations such as cos(r) and arccos(r)) create regions of vanishing and exploding gradients, respectively. The heightened nonlinearity from these high- degree polynomials further leads to imbalanced loss gradients, intensifying dynamic stiffness. Therefore, we employ Gradient-Rekan Attention (GRA).
GRA dynamically adjusts weights based on gradient norms of different loss components, promoting balanced training. scalar applied one entire loss term, GRA addresses the imbalance where gradient norms the PDE residual loss significantly exceed those of the data fitting loss (Wang et al.) 2021). which can lead to pathological gradient flow issues (Wang et al. 2022b Fang et al., 20233. Our mechanism smooths weight adjustments, preventing the network from overemphasizing residual loss terms and neglecting other essential physical constraints, thus enhancing convergence and stability.
The GRA weight GRA is computed as:
GRA GTDAX
(19)
Ga
8c. is the maximum gradient norm of the residual loss, G. = p Ep- where G.max = max, 50, gradient norm for C., the number of model parameters, and prevents division by zero.
8C4
is the average
802
To smooth the GRA weights over iterations, we apply exponential moving average:
GRA (1 -B ARRA
(20)
7

---

Published in Transactions on Machine Learning Research (01/2026)
0.-
Prodited sfull
Woskteddgle.t
011
43
06
04
0.5
033
0/7
23
25
53
Figure 2: Visualization of AC-PKAN's predicted values for PDE experiments: (Row 1) 1D-Wave, ID-Reaction, 2D NS Cylinder, ID-Conv.DDiff-Res (Row 2) 2D Lid-driven Cavity, Heterogeneous Problem, Complex Geometry, and 3D Point-Cloute
where B. is the learning rate for the GRA weights, We enforce a minimum value for numerical stability:
A max (AZRA c)
(21)
GRA addresses the aforementioned issues by stabilizing the gradient flow, thereby ensuring efficient and reliable training the network. By combining our AC-PKAN internal archilecture with the external RGA mechanism, we obtain the complete AC-PKAN model. Figure 1 provides detailed illustration of model structure.
Algorithm Implementation of the RGA Mechanism
Data: Model parameters 8, total number of parameters P, learning rate a, hyperparamoters Bw, Initialization: "r, + 0, Ad GRA
+- 1.
I: for each training iteration do Compute gradients:
2:
OC: VoL: + 00
iE (r,d)
3: Update RBA weights for each data point j:
nBA wis
+ (1 m)
RBA +
ie (r.d)
max, Lisl
4: Compute gradient norms:
Cmax max Crlll
G p
Ioll ic (d)
p
5: Update GRA weights:
G
CRAA (1 - -B Brode
ACRA + max(c. SRRA), i (
G.
6: Compute total loss:
CRGA + dr W.
RBA
Cr + Σ
d RBA RBA log log(2 (ARRA) C,
tEld)
T:
Update model parameters:
8 + - aV
8: end for
Combined Attention Mechanism To equilibrate the magnitudes of GRA and RBA weights, we apply logarithmic trausformation to the GRA weights when incorporating them into the loss terms, while retaining their original form during weight updates. This preserves the direct relationship between weights and gradient information,
8

---

Published in Transactions on Machine Learning Research (01/2026)
ID-Wave
1D-Reaction
2D NS Cylinder
ID-Conv.-Dift-FE
2D Lid-driven Cavity
Model
IMAE
+RMSE
IMAE
rRMSE
:MAE
FRMSE
:MAE
TRMSE
IMAE
+RMSE
PINN
0.3182
0.3200
0.9818
0.9810
5.8378
4.0529
0.0711
0.1047
0.6219
0.6182
QRes
0.3507
0.3485
0.9844
0.9849
25.8970
17.9767
0,0722
0.1062
0.5989
0.5674
FLS
0.3810
0.3796
0.9793
0.9773
12.4564
8.6473
0.0707
0.1045
0.6267
0.6267
PINNsFormer
0.2699
0.2825
0.0152
0.0300
0.3843
0.2801
0.0854
0.0927
OoM
OoM
Cheby1KAN
1.1240
1.0866
0.0617
0.1329
3,7107
2.7379
0.0992
0.1644
0.5689
0.5370
Cheby2KAN
1.1239
1,0865
1.0387
1.0256
T2.1708
50. 1039
1.2078
1.2059
6.1457
3.9709
AC-PKAN (Ours)
0.0011
0.0011
0.0375
0.0969
0.2230
0.2182
0.0142
0.6374
KINN
0.3466
0.3456
0.1314
0.2101
4,5306
3.1507
0.0721
0.1058
OoM
OoM
rKAN
217.7560
2593.0750
65.2014
54.8567
N&N
NaN
513.8576
3033.6251
OoM
OoM
FastKAN
0.5312
0.5229
0.5475
0.6030
25.8970
1.4085
0.0876
0.1219
OoM
OoM
FKAN
0.4884
0.4768
0.0804
0. 1033
3.0786
2.1403
0.1186
0.0794
0.7639
0.7386
FourierKAN
1.1356
1.1018
1,15-2
1-1217
9.3295
8.0346
0.91052
0.9708
OoM
OoM
Table I: Combined experimental results across Failure PINN Modes. Results are organized from left to right in the following order: Wave, ID-Reaction, 2D NS Cylinder, ID-Conv.-Diff. -Reac. and 2D Lid-driven Cavity.
Heterogeneous Problem
Complex Geometry
3D Point-Cloud
Model
rMAE
rRMSE
rMAE
rRMSE
rMAE
rRMSE
PINN
0.1662
0.1717
0,9010
0,9289
3.0265
2.1101
QRes
0.1102
0,1140
0.9021
0,9289
3.6661
2.8897
FLS
0.1701
0.1789
0.9021
0,9287
3.1881
2.5629
PINNsFormer
0.1008
0,1610
0.8851
0,8721
OoM
OoM
Cheby1KAN
0.1404
0.2083
0.9026
0.9244
2.4139
1.9646
Cheby2KAN
0.9170
1.0131
4.9177
3.5084
AC-PKAN (Ours)
0.1063
0.1817
0.5452
0.5896
0.3946
0.3403
KINN
0.1599
0.1690
0.9029
0.9261
OoM
OaM
rKAN
24.8319
380.5582
23.5426
215.4764
366.5741
2527. I 1880
FAStKAN
0.1549
0.1624
0.9034
0.9238
OoM
OuM
IKAN
0.9043
0.9303
2.6279
2.2051
FourierKAN
0.4588
0.5154
1.4455
1.5341
0.9314
1.0325
Table 2: Combined experimental results across Complex Engineering Environments. Results are organized from left right in the following order: Heterogencous Problem, Complex Goometry, and 3D Point-Cloud.
ensuring sensitivity to discrepancies between residual and data gradients. The logarithmic transformation mitigates magnitude disparities, preventing imbalances among loss terms. It enables GRA weights to adjust more rapidly when discrepancies are minor and ensures stable updates when discrepancies are substantial. The coefficient X GRA not only attains excessively large values in scale but also exhibits broad range of variation. In the training process, X GRA rapidly increases from zero to very large values, demonstrating a wide dynamic range which is shown in Figure [ in Appendix F The logarithmic transformation significantly constrains this range; without it, the model cannot accommodate drastic changes in X GRA and rigid manual scaling factors further exacerbate the imbalance among loss terms, ullimately causing training failure.
By integrating point-wise RBA with term-wise GRA, the total loss under the RGA mechanism is defined as:
ORGA = A, RBA RBA L, + Aaw
RBA log (Ad (AGRA) La.
(22)
where RBA are the RBA weights, and AGRA are the GRA weights for boundary/hnitio conditions available data samples.
This formulation reweights the residual loss based on its magnitude and adjusts the boundary and initial condition losses according to both their magnitudes and gradient norms, promoting balanced and focused training through a dual attention mechanism.
RGA enhances PINNs by dynamically adjusting loss weights based on residual magnitudes and gradient norms. By integrating RBA and GRA, it balances loss contributions, preventing any single component from dominating the training process. This adaptive reweighting accelerates and stabilizes convergence, focusing on challenging regions with significant and elficient solutions to complex differential equations, performing well our AC-PKAN model and polentially benefiting other PINN variants which is discussed in detail in appendix 4.4
inbalanced gradients. Consequently, RGA provides robust framework for more accurate
9

---

Published in Transactions on Machine Learning Research (01/2026)
Comeergence Speed Comparison
Model
rMAE
rMSE
Loss
0.
Cheby IKAN
0.0179
0.0329
0.0068
MUD
PEARL
Cheby2KAN
0.0189
0.0313
0.0079
CINN
PRAN
MLP
0.0627
0.1250
0.1410
AC-PKAN_
0.0177
0.0311
0.0081
8
KAN
0.0145
0.0278
0.0114
rKAN
0.0458
0.0783
0.1867
IKAN
0.0858
0.1427
0.1722
FastKAN
0.0730
0.1341
0.1399
FourierKAN
0.0211
0.0353
0.0063
33000
35000
20000
30022
(a) Comparison of test rMAE, rMSE, and training Loss
(b) Convergence Comparison of Nine Models
4 Experiments
Goal. Our empirical study highlights three principal strengths of AC-PKAN: (1) its internal architecture delivers powerful symbolic representation and duacion-aaproxiintio capabilities; (2) it significantly improves generalization abilities and mitigates failure modes compared to PINNs and other KAN variants; and (3) it achieves superior performance in complex real-world engineering environments. We evaluate our method on three task suites comprising nine benchmarks and compare it to 12 representative architectures, including PINN, PINNsFormer, KAN, and fKAN. Although operator learning frameworks that rely large volumes of labeled data have recently dominated SciML (Lu
el al.
Calvello et al., 2024). our work remains within the established remit of PINN relinement studies, which locus on unsupervised or weakly supervised settings (Jaglap &
2019 Li el al. 2020a Tripura & Chakraborty 2022
Karniadakis 2020 Yu et al. 2022 Zhang el al. 2025a Hou el al. 2024 Li et al. 2023 Operator-learning methods
typically require dense supervision and therefore address different set of trade-offs compared physics-informed
approaches. Nevertheless, we include a head-to-head comparison of operator-learning and PINN-variant baselines under sparse-data conditions Appendix C.3 The experimental setup was inspired by methodologies in (SS) 2024
Hao et al. 2023 Wang et al., 2024b Zhao et al. 2023 Wang et al. 2023 In all experiments, the best results are highlighted bold Htaics,, and the second-best results bold. For the formal definitions of the evaluation metrics and detailed descriptions of the experimental setup, please refer to appendix D
4.1 Complex Function Fitting
We evaluated the AC-PKAN Simplified model, which retains only the internal architecture, on challenging function interpolation benchmark and compared its performance to a PINN implemented as an MLP, the original KAN, and several KAN variants. Detailed experimental setups and results are provided in Appendices
L and F
As shown Figure 35 the AC-PKAN Simplified model converges more rapidly than MLPs, KAN, and most KAN variants, achieving lower final losses. While Cheby2KAN and FourierKAN demonstrate faster convergence, model produces smoother fitted curves and exhibits greater robustness to noise, effectively preventing overfitting regions with high-freynency variations.Perfommance metrics are presented in Table 3a
4.2 Mitigating Failure Modes in PINNs
We assessed the AC-PKAN model on five complex PDEs known as PINN failure modes the 1D-Wave PDE, 1D- Reaction PDE, 2D Stvier-Stokes Flow around a Cylinder, 1D Canvection-Diffusiin Reaction and 2D Navier-Stokes Lid-driven Cavity Flow (Mojgani et al.) 2022 Daw et al. 2022 Krishnapriyan et al.) 2021) to demonstrate its superior generalization ability compared to other PINN variants. In these cases, optimization often becomes trapped in local minima, leading overly smooth approximations that deviate from true solutions.
Evaluation results are summarized in Table D with detailed PDE formulations and setups in Appendix E Prediction for AC-PKAN are shown in Figure 2 and additional plots including the analysis of loss landscapes are in Appendix F
AC-PKAN significantly outperforms nearly all baselines, achieving the lowest or second-lowesl test errors, thus more elfectively mitigating failure modes than the previous SOTA method, PINN&Former. Other baselines remain stuck in local minima, failing to optimize the loss effectively. These results highlight the advantages of AC-PKAN in generalization and approximation accuracy over conventional PINNs, KANs, and existing variants.
10

---

Published in Transactions on Machine Learning Research (01/2026)
Model
rMAE (RGA)
FRMSE (RGA)
TMAE (No RGA)
FRMSE (No RGA)
PINN
0.0914
0.0924
0.3182
0,3200
PINNsFormner
OoM
OuM
0.2699
0.2825
QRes
0.2201
0,2184
0.3507
0.3185
FLS
0.1610
0.1617
0.3810
0.3796
Cheby1KAN
0.0567
0,0586
1.12400
1,0866
Che
1.0114
1.0048
1.1239
1.0865
AC-PKAN (Ours)
0.0011
0.0011
0.45-19
0,4488
KINN
0.0479
0.0486
U.3466
0.3456
rKAN
NaN
NaN
247.7560
2593.0750
FastKAN
0.1318
0.1376
0.5312
0,5229
RAN
0.2177
0,2149
0.4884
0.4768
FourlerKAN
1.0015
1.0001
1.1356
1.1018
Table 3: Comparison of performance metrics in the 1D-Wave experiment with and without the RGA module applied.
4.3 PDEs in Complex Engineering Environments
We further evaluated AC-PKAN across three challenging scenarios: heterogeneous environments, complex geometric boundury conditions, and three-dimensional spatial point clouds. Literature indicales that PINNs encounter difficulties with heterogeneous problems due to sensitivity to material properties (Aliakbari et al. 2023), siguificant errors near boundary layers (Piao et al. 2024), and convergence issues (Sumanta et al.) 2024). Additionally; original KANs perform poorly with complex geometries (Wang et al. 2024b). The sparsity, irregularity, and high dimensionality of unstructured 3D point cloud data hinder PINNS from effectively capturing spatial features, resulting in suboptimal training performance (Chen et al. 2022 We applied AC-PKAN to solve Poisson equations within these contexts.
Detailed PDE formulations are Appendix and detailed
experinental results are illustrated in Appendix Summarized in Table 2 and partially shown in Figure 2 the results indicate that AC-PKAN consistently achieves the best second- performance. demonstrates superior potential in solving heterogeneous problems without subdomain division and exhibits promising application potential in complex geometric boundary problems where most models fail.
Model
rMAE
rRMSE
AC-PKAN
0.0011
0.0011
AC-PKAN (no GRA)
0.0779
0.0787
AC-PKAN (no RBA)
0.0494
0.0500
AC-PKAN (no RGA)
0.4549
0.4488
AC-PKAN (no Wavelet)
0.0045
0.0046
AC-PKAN (no Eucoder)
0.0599
0.0584
AC-PKAN (no MLPss) 1.0422
1.0246
4.4 Ablation Study
Table 4: Ablation study on the 1D-wave equation, demonstrating the effect of removing each module from AC-PKAN.
Modulc importance. Ablation experiments for the module importance on the WD-Wave equation [ confirm that each module in model is crucial. Removing any module leads to a significant performance decline, especially the MLPs module. These findings suggest that the KAN architecture alone is insufficient for complex tasks, validating our integration of MLPs with the ChebyIKAN layers.
Transfcrability of RGA. Table 3 evaluates our RGA twelve alternative PINN variants. Except for PINNsFormer (out-of-memory due to pseudo-sequent markedly: average rMAE drops by 36% and rRMSE by 34%. Nonetheless, none surpass AC-PKAN, whose coupled architecture and RGA still attain the lowest errors by two orders of magnitude, underscoring both the standalone value of RGA and the holistic superiority of AC-PKAN.
inflation) and rKAN (gradient blow-up), every model benefits
Effect of Logarithmic Transformation in the RGA Module. In this ablation study, investigated the impact of removing the logarithmic transformation in the RGA module across five PDE experimental tasks. To compensate for the absence of the logarithmic scaling, we adjusted the scaling factors to smaller values. Specifically, we employed the original RGA design to pre-train the models for several epochs, during which very large values of X were obtained. We fix the PDE residual scale to and set the data loss scales, including boundary and initial condition terms, to be inversely proportional to the current scale of X GRA, which keeps the different loss contributions comparable magnitudes.
The performance metrics with and without the logarithmic transformation are summarized in Table 5
We observe a significant deterioration in the performance of AC-PKAN when the logarithmic trausformation is removed. This decline is altributed to two main factors: first, X CRA altains excessively large values; second, it exhibits wide range of variation. During the standard training process, the coefficient. X GRA rapidly grows from 0 to a very
11

---

Published in Transactions on Machine Learning Research (01/2026)
Equation
Without Log
With Log
rMAE
rRMSE
rMAE
rRMSE
2D NS Cylinder
532.2411
441.0240
0.2230
0.2182
1D Wave
0.7686
0.7479
0.0011
0.0011
1D Reaction
2.2348
2.2410
0.0375
0.0969
Hctcrogeneous Problem
10.0849
9.6492
0.1063
0.1817
Complex Geometry
164.4283
158.7840
0.5452
0.5896
Table 5: Comparison of performance metrics AC-PKAN with and without the logarithmic transformation in the RGA module.
large value, resulting in broad dynamic range. The logarithmic transformation effectively narrows this range; for instance, in the 1D Wave experiment, the scale of XRRA over epochs ranges from to x 10", whereas In (1(ARA) ranges from 7 to 15 in Picture 0 Removing the logarithmic transformation and attempting to manually adjust scaling factors to match the apparent magnitudes is ineffective. The model cannot adapt to the drastic changes in X GRA, and rigid manual scaling factors exacerbate the imbalance among loss terms, ultimately leading to training failure. By confining the variation range GRA, the logarithmic transformation enables the model to adjust more flexibly and effectively.
The rationale for employing the logarithmic transformation originates from the Bode plot in control engineering, which is semi-logarithmit graph that utilizes logarithmic frequency axis while directly labeling the actual frequency values. This approach not only compresses a wide frequency range but also linearizes the system's 's gain and phase characteristics on logarithmic scale, thereby mitigating imbalances caused by significant differences in data scales.
Integration with Other External Learning Strategies for Enhanced Performance of AC-PKAN.
Integrating AC-PKAN with other external learning strategies, such the Neural Tangent Kernel (NTK) method, resulted in enhanced performance (Table G). This demonstrates the flexibility of AC-PKAN in incorporating various learning schemes, offering practical and customizable solutions for accurate modeling in real-world applications.
Model
rMAE
rRMSE
AC-PKAN + NTK
0.0009
0.0009
PINNs + NTK
0.1397
0.1489
PINNsFormer + NTK
0.0453
0.0484
Table 6: Performance comparison on the ID-wave equation using the NTK method. AC-PKAN combined with NTK achieves superior results across all metrics.
5
Discussion and Limitations
Our current evaluation focuses low to medium dimensional PDE and ODE settings that can isolate the gains from the Chebyshev-based backbone, the internal feature re-injection and the rank-aware gating in a controlled regime. Extending AC-PKAN to gemuinely high dimensional chaotic systems such Lorenz 96 would additionally require techniques for stable time integration, for controlling long-horizon error growth and for conditioning stiff gradients, which outside the scope of this version Karimi & Paul (2010); Wang et al. (022aa) Maiocchi et al. (2024). Full layerwise Jacobian-rank profiling for ultra-deep stacks is infeasible on our 40 GB GPUs because hackprop must retain large activations and Jacobian blocks. We list this scalability limitation and will perform ultra-deep evaluations once
hardware is available. We also rely on a standard AdamW optimiser and have not yet explored KAN-specific optimisation or structured pruning, which we view as promising directions for improving scalability and iulerpretability in future work.
6 Conclusion
We introduced AC-PKAN, novel framework that enhances PINNs by integrating Cheby1KAN with traditional MLPs and augmenting them with internal and external attention mechanisms. This improves the model's ability to capture complex patterns and dependencies, resulting in superior performance on challenging PDE tasks, including previous PINN failure modes and complex physical environments. The RGA mechanisu eubances training stability and convergence by dynamically adjusting loss terms. Experimental results demonstrate that AC-PKAN consistently outperforms or matches saale-of-the-al models like PINNsFormer, confirming its effectiveness in real-world engineering problems.
12

---

Published in Transactions on Machine Learning Research (01/2026)
References
Diab W Abucidda, Panos Pantidis, and Mostafa E Mobasher. Deepokan: Deep operator network based on kolmogorov arnold networks for mechanics problems. arXiv preprint 1914144455.1143, 2024.
Alireza Afzal Aghaei. rkan: Rational kolmogorov-arnold networks. arXiv preprint or 00v:24400. 14495, 2024.
Alireza Afzal Aghaei. fkan: Fractional kolmogorov-amold networks with trainable jacobi basis functions. Neurocomputing, 623:129414, 2025.
Maryam Aliakbari, Mohammadreza Soltany Sudrabadi, Peter Vadasz, and Amirhossein Arzani. Ensemble physics informed neural networks: A framework to improve inverse transport modeling in heterogeneous domains. Physics of Fluids, 35(5), 2023.
Sokratis J Anagnostopoulos, Juan Diego Toscano, Nikolaos Stergiopulos, and George Em Karniadakis. Residual-based attention in physies-informed neural networks. Computer Methods in Applied Mechanics and Engineering, 421: 116805, 2024.
Gregor Bachmann, Sotiris Anagnostidis, and Thomas Hofmann. Scaling mlps: A tale of inductive bias. Aduances in Neural Information Processing Systems, 36, 2024.
Zavareh Bozorgasi and Hao Chen. Wav-kan: Wavelet kolmogorov-aruold networks. arxiv preprind 12Xie:2405.12322 2024.
Jürgen Braun and Michael Griebel. On a constructive proof of kolmogorov's superposition theorem. Constructive approzimation, 30:653-677 2009.
Carlos Alberto Brebbia, José Claudio Faria Telles, and Luiz C Wrobel. Boundary element Lechniques: theory and applications engineering. Springer Science & Business Modia, 2012.
Jie Bu and Anuj Karpatne. Quadratic residual networks: A new class of neural networks for solving forward and inverse problems plysics involving pdes. In Proceedings of the 2021 SIAM International Conference on Data Mining (SDM), pp. 675-683 683. SIAM, 2021.
Edoardo Calvello, Nikola B Kovachki, Matthew E Levine, and Andrew M Stuart. Continuum attention for neural operators. eiviv preprint orXiv:2406.066 06486, 2024.
Linwei Chen, Bowen Fang, Lei Zhao, Yu Zang, Weiquan Liu, Yiping Chen, Cheng Wang, and Jonathan Li. beepurbannowwssale:: A physics informed deep learning framework for high-resolution urban surface temperature estimation via point clouds. International Journal of Applied Earth Observation and Geoinformation, 106:102650, 2022.
Miles Crammer. Interpretable machine learning for science with pysr and symbolicregression jl. arXiv preprial 20Xi:2300500188,, 2023.
George Cybenko. Approximation by superpositions of sigmoidal function. Mathematics of control, signals and systems, 2(4):303-314, 1989.
Hadi Daneshmand, Jonas Kohler, Francis Bach, Thomas Hofmann, and Aurelien Lucchi. Batch normalization provably avoids ranks collapse for randomly initialised deep networks. Advances in Newral Information Processing Systems, 33:18387-18398, 2020.
Arka Daw, Jie Bu, Sifan Wang, Paris Perdikaris, and Amuj Karpatne. Mitigating propagation failures in physics- informed neural networks using retain seaain-rsammll-eeeees (13) sampling, anXiv preprint 0rXiv:2207.02998 2022.
Zhiwei Fang, Sifan Wang, and Paris Perdikaris. Ensemble learning for physics informed neural networks: A gradient boosting approach. arXiv preprint orXiv:232.. 13143, 2023.
Ruili Feng, Kecheng Zheng, Yukun Huang, Deli Zhao, Michael Jordan, and Zheng-Jum Zha. Rank diminishing in deep neural networks. Advances in Neural Information Processing Systems, 08330544330066 2022.
Bengt Fornberg, A practical guide to pscudospectrul methods. Number in Cambridge Monographs on Applied and Computational Mathematics. Cambridge university press, 1998.
13

---

Published in Transactions on Machine Learning Research (01/2026)
Leonardo Ferreira Guilhoto and Paris Perdikaris. Deep learning alternatives of the kolmogorov superposition theorem. arXiv preprinl wrXiv:2410.01990, 2024.
Chunyu Guo, Lucheng Sun, Shilong Li, Zelong Yuan, and Chao Wang. Physics-informet kolmogorov-aroold network with chebyshev polynomials for fluid mechanics. Physics of Fluids, 37(9), 2025.
Jiequn Han, Arnulf Jentzen, and Weinan E. Solving high-dimensiooall partial differential equations using deep learning. Proceedings of the National Academy of Sciences, 155564):850--8510, 2018.
Zhongkai Hao, Jiachen Yao, Chang Su, Hang Su, Ziao Wang, Fanzhi Lu, Zeyu Xia, Yichi Zhang, Songming Liu, Lu Lu, et al. Pinnacle: A comprehensive benchmark of plysics-informed neural networks for solving pdes. ar preprinl arXie:2306.0882 2023.
Kurt Hornik, Maxwell Stinchcombe, and Halbert White. Multilayer feedforward networks are universal approximators. Neurul networks, 2(5):359 366, 1989.
Xianrui Hou, Xingyu Zhou, and Yi Liu. Reconstruction of ship propeller wake field based on self-adaptive loss balanced physics-informed neural networks. Ocean Engineering, 309:1 009:118341, 2024.
Ameya D Jagtap and George Em Karniadakis. Extended physics-informed neural networks (xpinns): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations. Communications in Computational Physics, 28(3), 2020.
Ameya D Jagtap, Kenji Kawaguchi, and George Em Karniadakis. Locally adaptive activation functions with slope recovery for deep and physics-informed neural networks. Proceedings of the Royal Sociely A, 276662239)200003333 2020a.
Ameya D Jagtap, Kenji Kawaguchi, and George Em Karniadakis. Adaptive activation functions accelerate convergence in deep and physics-infore informed neural networks. Journal of Computational Physics, 404:109136, 2020b.
Alireza Karimi and Mark R Paul. Extensive chaos in the lorenz-96 model. Chaos: An interdisciplinnary journal of nonlinear science, 20(4), 2010.
Ehsan Kharazmi, Zhongqiang Zhang, and George Em Karniadakis. Variational physics-informed neural networks for solving partial differential equations. arXiv preprint XIt::192200077, 2019.
Ehsan Kharazmi, Zhongqiang Zhang, and George Em Karniadakis. hp-vpinns: networks with domain decomposition. Computer Methods Applied Mechanics and Engineering, 374:113547, 2021.
Variational physics- -nffomeed neural
Andrei Nikolaevich Kolmogorov. On the representation of continuous functions of several variables by superpositions of continuous functions of smaller number of variables. American Mathematical Society, 1961.
Mario Köppen. On the training of a kolmogorov network. In Artificial Neural Sictworts-IAAA 2002: International Conference Madrid, Spain, August 28 30, 2002 Proceedings 12, pp. 174 179. Springer, 2002.
Aditi Krishnapriyan, Amir Gholami, Shandian Zhe, Robert Kirby, and Michael W Mahoney. Characterizing possible failure modes in physics-informed neural networks. Advonces in neural information processing systems, 34:26548- 26560, 2021.
Isaac F. Tagaris, Aristidis Likas, and Dimitrios I Fotiadis. Artificial neural networks for solving ordinary and partial differential equations. JEEE transactions on neural networks, 955) 1998.
Gregory Kang Ruey Lau, Apivich Hemachandra, See-Kiong Ng, and Bryan Kian Hsiang Low. Pinnacle: Pinn adaptive collocation and experimental points selection. arXiv preprint arXim:2444.07666 2024.
Yao Li, Shengzhu Shi, Zhichang Guo, and Boying Wu. Adversarial training for physics-informet neural networks. arXiv preprint arxiv:2910.11 11789, 2023.
Ziyao Li. Kolmogorov-arnold networks are radial basis function networks. arXio preprint arXio:2405.06722) 2024.
Zongyi Li, Nikola Kovachki, Kamnar Azizadenesheli, Burigede Liu, Kaushik Bhattacharya, Audrew Stuart, and Anima
Anandkumar. Fourier neural operator for parametric partial differential equations. arXio preprint arXio:20100.888 2020a.
14

---

Published in Transactions on Machine Learning Research (01/2026)
Zongyi Li, Nikola Kovachki, Kamnar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuarl, and Anima Anandkumar, Neural operator: Graph kernel network for partial differential equations. arXiv preprind arXiv:2003.00 03485, 2020b.
Gui-Rong Liu. Meshfree methods: moving beyond the finite element method. CRC press, 2009.
Ziming Liu, Pingchuan Ma, Yixuan Wang, Wojciech Matusik, and Max Tegmark. Kan 2.0: Kolmoggorox-arnold networks meet science. arXiv preprint orXio:2408. 10205, 2024a.
Ziming Liu, Yixuan Wang, Sachin Vaidya, Fabian Ruehle, James Halverson, Marin Soljačié, Thomas Y Hou, and Max Tegmark. Kan: Kolmogorov-aruold networks. arXiv preprint urXiv:2404.19756 2024b.
Lu Lu, Pengzhan Jin, and George Em Karniadakis. Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators. arXiv preprint arXiv:1910.00 031999 2019.
Chiara Cecilia Maiocchi, Valerio Lucarini, Andrey Gritsun, and Yuzuru Sato. Heterogeneity of the attractor of the 1orenz'96 model: Lyapunov analysis, unstable periodic orbits, and shadowing properties. Physica D: Nonlinear Phenomena, 457:133970, 2024.
Ali Mehrabian, Parsa Mojarad Adi, Moein Heidari, and liker Hacihaliloglu. Implicit neural representations with fourier kolmogorov-arnold networks. arXiv preprint urXiv:249.00933 2024.
Rambod Mojgani, Maciej Balajewicz, and Pedram Hassanzadeh. Lagrangian pinns: A consorit solution to failure modes of physics-informed informed neural networks. arXiv preprint arXiv:2205. 02900,, 2022.
Ben Moseley, Andrew Markham, and Tarje Nissen-Meyer. Finite basis physics-informed neural networks (fbpinns): scalable domain decomposition approach for solving differential equations. Advances in Computational Mathematics, 49(4):62, 2023.
Farinaz Mostajeran and Salah Faroughi. Scaled-cpikans: Domain scaling chebyshev-based physics-informed kolmogorov-arnold networks. arXiv preprint erXi::201.02700 2025.
Charles M Newman. The distribution of Iyapunov exponents: exact results for random matrices. mathematical physics, 103(1):121 126, 1986.
Communications in
Valery Iustinovich Oseledets. Multiplicative ergodic theorem. Iyapunov characteristic indicators of dynamical systems. Transactions of the Moscow Mathematical Society, 19(0):179-210, 1968.
Renbin Pan, Feng Xiao, and Minyu Shen. ro-pimm: A reduced order physics-informed neural network for solving the macroscopic model of pedestrian flows. Transportation. Research Part C: Emerging Technologies, 163:101658, 2024.
Shiyuan Piao, Hong Gu, Aina Wang, and Pan Qin. A domain-adaptive physics-informed neural network for inverse problems of maxwell's equations in heterogeneous media. IEEE Antennas and Wireless Propagation Letters, 2024.
Maziar Raissi, Paris Perdikaris, and George F. Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686-707, 2019.
Spyros Rigas, Michalis Papachristou, Theofilos Papadopoulos, Fotios Anagnostopoulos, and Georgios A Alexandridis. Adaptive training of grid-dependent physics-informed kolmogorov-arnold networks. IEFE Access, 2024.
Andreas Roth and Thomas Liebig, Rank collapse causes wer-shoothing and over-correlation graph neural networks. In Learning on Graphs Conference, pp. 35-1. PMTR, 2024.
Moein E Samadi, Younes Müller, and Andreas Schuppert. Smooth kolmogorov arnold networks enabling structural knowledge representation. arXiv preprint 111111840..1118 2024.
Khemraj Shukla, Juan Diego Toscano, Zhicheng Wang, Zongren Zou, and George Em Karniadakis. A comprehensive and fair comparison between mlp and kan representations for differential equations and operator networks. arXiv preprint arXiv:2206. 02911, 2024.
David A Sprecher and Sorin Draghici. Space-filling curves and kolmogorov superposition-basee neural networks. Neural Networks, 15(1):57- 67, 2002.
15

---

Published in Transactions on Machine Learning Research (01/2026)
Sidharth SS. Chebyshev polynomial-based kolmogorov-arrold networks: Au elficient architecture for noulinear function approximation. arXiv preprinl arXiw:2405.00 0720000 2024.
Roy Sumanta, Amavarapu Chandrasckhar, Roy Pratanu, and Valiveti Dakshina Murthy. Physics-informed neural networks for heterogeneous poroelastic media. arXiv preprint ar arXiv:2407.0097 2024.
Kejun Tang, Jiayu Zhai, Xiaoliang Wan, and Chao Yang. Adversarial adaptive sampling: Unify pinn and optimal transport for the approximation of pdes. arXio preprint urXin:2305.18700 2023.
Tapas Tripura and Souvik Chakraborly, Wavelet neural operator: a neural operator for parametric partial differential equations. arXiv preprint (xX6:2205.02191, 2022.
Cristian J Vaca-Rubio Luis Blanco, Roberto Pereira, and Márius Caus. Kolmogorov-amool networks (kans) for time series analysis. arXiv preprint (05:200500 2024.
Sifan Wang, Yujun Teng, and Paris Perdikaris. Understanding and mitigating gradient flow pathologies in physics- informed neural networks. SIAM Journal Scientific Computing, (005)--355-38 2021.
Sifan Wang, Shyam Sankaran, and Paris Perdikaris. Respecting causality is all you need for training physics-informed neural networks. arXiv preprint arXim:2208.07404, 2022a.
Sifan Wang, Xinling Yu, and Paris Perdikaris. When and why pinns fail to train: A neural tangent kernel perspective. Journal of Computational Physies, 449:110768, 2022b.
Sifan Wang, Shyam Sankaran, Hanwen Wang, and Paris Perdikaris. An expert's guide to training physics-informed neural networks. arXiv preprint (XXx:2308.08/66 2023.
Sifan Wang, Bowen Tá, Yuhan Chen, and Paris Perdikaris. Piratenets: Physics-informed deep learning with residual adaptive networks. arXiv preprint Xrxiv:8402. 00826, 2024a.
Yizheng Wang, Jia Sun, Jinshuai Bai, Cosmin Anitescu, Mohammad Sadegh Eshaghi, Xiaoying Zhuang, Timon Rabczuk, and Yinghua Lin. Kolmogorov arnold informed neural network: A physics-informed deep learning framework for solving pdes based on kolmogorov arnold networks. ar preprint (xXi:2246.11044 2024b.
Jian Cheng Wong, Chin Chun Ooi, Abhishek Gupta, and Yew-Soon Ong. Learning in sinusoidal spaces with physics-informet neural networks. IEEE Transactions on Artificial Intelligence, 5(3):985-1000, 2022.
Chenxi Wu, Min Zhu, Qinyang Tan, Yadhu Kartha, and Lu Lu. A comprehensive study of non-adaptive and residual-based adaptive sampling for plysics-informed neural networks. Computer Methods Applied Mechanics and Engineering, 403:115671, 2023.
Zipeng Xiao, Siqi Kou, Zhongkai Hao, Bokai Lin, and Zhijie Deng. Amortized fourier neural operators. In The thirty-eighhth Annual Conference on Neural Information Processing Systems, 2024.
Jinfeng Xu, Zheyu Chen, Jinze Li, Shuo Yang, Wei Wang, Xiping Hu, and Edith C-H Ngai. Pourierkan-gef: Fourier kolmogorov-arrold network an effective and efficient feature transformation for graph collaborative filtering, arXiv preprint ar 01034, 2024a.
Zhongshu Xu, Yuan Chen, and Dongbin Xiu. Chebyshev feature neural network for accurate function approximation. arXiv preprint or 0xx::24409119: 2024b.
Jiang Yang, Yuxiang Zhao, and Quanhui Zhu. Elfective rank and the staircase phenomenon:: New insights into neural network training dynamics. arXio preprint arXiv:2412.05144 2024.
Bing Yu et al. The deep ritz method: a deep learning- based numerical algorithm for solving variational problems. Communications in Mathematics and Statistics, 6(1):1-12, 2018.
Jeremy Yu, Lu Lu, Xuhui Meng, and George Em Karniadakis. Gradient-enhancee physics- informed neural networks for forward and inverse pde problems. Computer Methods in Applied Mechanics and Engineering, 393:114823, 2022.
Runpeng Yu, Weihao Yu, and Xinchao Wang. Kan or mlp: A fairer comparison. arXiv preprint arXiv:2407. 166744 2024.
16

---

Published in Transactions on Machine Learning Research (01/2026)
Lei Yuan, Yi-Qing Ni, Xiang-Yur Yun Deng, and Shuo Hao. A-pinm: Auxiliary physics informed neural networks for [orward and inverse problems of nonlinear integro-diflerential equations. Journal of Computational Physics, 462: 111260, 2022.
Rui Zhang, Liang Li, Stéphane Lanteri, Hao Kang, and Jiaqi Li. Bo-sa-pinns: Self-adaptive physics-informed neural networks based on bayesian optimization for automatically designing pde solvers. arXiv preprint 2Xix:250409880 2025a.
Zhaoyang Zhang, Qingwang Wang, Yinxing Zhang, Tao Shen, and Weiyi Zhang. Physics-informed neural networks with hybrid vohmogorov-arnole network and augmented lagrangian function for solving partial differential equations. Scientific Reports, 15(1):10523, 2025b.
Zhiyuan Zhao, Xueying Ding, and B Aditya Prakash, Pinnsformer: A transformer-based framework for physics-informed neural networks. arXiv preprint xxx:22307.11183 2023.
Olek C Zienkiewicz and Robert Taylor. The finite element method Elsevier, 2005.
17

---

Published in Transactions on Machine Learning Research (01/2026)
A Mathematical Proofs
A.1 Proof of Theorem 1
Lemma 1. Let A m and B E _=xx, Then AB E mxxp, and
rank(B)) S minfrrnk(A)) rank(B)),
Wi E 2", let A, matriz of appropriate dimensioon, and rank(A ............ An) I min A1), rank(A2), rank(An))
Proof. Let A E R and B E Rex,. Consider the product AB E R" We aim to show that
rank(AB) & min(rank(AA), rank())))
(23)
First, observe that each column of AB is linear combination of the columns of A. Specifically, if the columns of B are denoted by bi, ba...., by: then the j-th column of AB given by Aby. Consequently, the column space of AB, denoted Co)(AB), satisfies
CAl(A S Col(A).
(24)
By the properties of subspace dimensions, it follows from equation 24 that
rank(AB) = dim(C)l(AB)) & dim(Col(A)) = rank(AA).
(25)
Next, consider the transpose of the product AB:
(AB) = B A
(26)
Applying the same reasoning to B RPxn and A E Raxm we have
rank(B" A I rank(B") rank(B).
(27)
Therefore, from equation 26 and equation
27 it follows that
rank(AB) = rank(B' (BTA TAT)
rank(B).
(28)
Combining equation 25 and equation
28
obtain
rank(AB) S min min(raak(A)) rank(B))
(29)
To generalize this result for any 2, we proceed by induction. Specifically, we aim prove that
rank(ArA2 ... A.) S A.)
rannraa) (ank(A2).....akkk((.
(30)
where each A: is a matrix of appropriate dimensions.
Inductive Hypothesis: Assume that for k.
rank(AiA2 ... Aa) &
(ann) rank(AA))
(31)
Inductive Step: Consider = + 1. We decompose the product
ArAr ... Au41 = (Ar --.-. Ax) Ak+1
(32)
Applying the previously established result equation 29 we obtain
rank(A1A2 ....2....... AR41) & min (rank(A1Aa.- As). rank(Ae))))
(33)
By the inductive hypothesis equation 31 we have
rank(A, ... Ac) S min(rank rank(A))). rank(A)))
(34)
Therefore, substituting equation 34 into equation
33
obtain
Tank(A,45 ... A S min(rank(Ai)) rank) rank(A
(35)
By induction, the inequality equation 30 holds for all 2. This completes the proof.
18

---

Published in Transactions on Machine Learning Research (01/2026)
Theorem 4 (Single CheblKAN Layer Rank Coustraint). The Jacobian Ji salisfies rank( S min(de+1, di 1)).
Proof. Assuming the imput of the 1-th layer is 2 and the output is 30 E BR The Jacobian matrix Ji describes the partial derivatives the output of the 1-th layer of the network with respect to its input:
Oin.j Oiei
J1 =
(36)
dr+1 xd;
The rank of a Jacobian matrix is defined as the maximum linearly independent number of its column row vectors:
rank(Ji) minndim(Col((J)) (im
(37)
which shows that the rank Jacobian is limited by the dimension of its column space and output space.
Each input component Tt.s is expanded through N + Chebyshev polynomial basis functions To, ... Tv. Based
on each input component Tt,; can be expressed as:
31.4
0a Tu (Ti.1)
(38)
k=
where ag the coefficients.
To conclude that N + 1 Chebyshev polynomials Ti (i (x) are linearly independent on interval [-1, 1],
assume the opposite: there exist constants cu: -1.-. CN such that:
Vo E [-,,1], (xTu(z) + +cTT(()) +xxx+xxxxxxx===----
(39)
Since each (x) set of k-degree orthogonal polynomials according to the left side is polynomial of degree at most. N. A non-zero polynomial of degree can have at most roots. However, the equation holds for in 11 which is infinite set of points. Therefore, the polynomial must be the zero polynomial, implying Cu = c1
ON = 0.
Suppose the input to the 1-th layer is ã E Rd, and the output is de E R Each input vector ã; is expanded through Chebyshev polynomial basis functions {Tr)
y as follows:
[Ib( Tw. N+1
(40)
The total expanded dimensionality is d: (N + 1). The output layer is obtained by linearly combining these basis functions:
N Wyack (T.).).
(41)
k=0
where Wj.i.k are learnable parameters. Taking the derivative with respect to the input vector inz
a
Wyick (..).
(42)
0F.
k=
This indicates that the i-th column of the Jacobian (i.e., 05/02:) belongs to the space spanned by (Z.) whose dimension is at most N + 1. The output contribution of each input component be viewed linear combination of + independent basis functions.
The i-th column of J: is the partial derivative vector of the i-th input component ( 05,,, /f der..... 0y...// om..) " Since the derivatives with respect to each input vector
19

---

Published in Transactions on Machine Learning Research (01/2026)
independently span N + 1-dimensional subspace, the dimension of the joint column space of all di columns is al most the sum of the dimensions the subspaces:
dim(Col(Ji) & =
- (N
(43)
The key to this upper bound is that the basis function expansions for different input vectors are independent. Based
Equation 37 although the output dimension d++1 may be much smaller than d: (N 1); the final column space dimension is constrained by the following two factors:
rank(Ji) (Co(((((J))) & min(dr41, di (N 1)}.
(44)
A.2 Proof Theorem 2
Theorem 5 (Nonlinear Normalization Effect). The normalization tanh(x) in ChebylKAN layers reduces the namerical rank Rank (J) of the Jacobian.
Proof. Consider the E-th layer of ChebylKAN network receiving x2 Rd and outputting xe-1 Rdd-1, The forward mapping is
Xe+1 &e (ttth((())))
(45)
where tanh(-) is applied elementwise, and 4, is learnable functional operator using Chebyshev polynomials of the first kind. Indexing each output component by E fl...., de+1 gives
Cenann Ta (tanh(zr.))..
(46)
n=0
where Cr.kin trainable coefficients, and T, [-1,1] defined by Ta(=) cos(n arccos(z)). The Jacobian
Je = @0xet1.0/0m kll
captures the gradient flow. Using dz (z) Un-2(=) for 2 1 (with To(2) 0), where Un-1 are Chebyshev polynomials of the second kind, and the identity
da tanh(x)) = 1- (x),
(47)
define
Te, tanh tane2((ze.))
(18)
Since 0 <. Te,a S each partial derivative becomes
[Jela, EC Ty (lanh((re;)) na.
(49)
Removing 7e, yields an "un-normalized" version
N Σ n=0
[Jelet
Ceniin Th (tan)(
(50)
leading the elementwise relation
[Jele, 7e, [Jelk.
(51)
Hence, in matrix form,
Je Je De:
(52)
20

---

Published in Transactions on Machine Learning Research (01/2026)
where Dj is diagonal with Dx(i.)) 7,, E (0, 1]. By submultiplicatiiity of the spectral and D $1
Jelll Della s ||Jella-
(53)
Since singular values are bounded by the spectral norm,
ai(ee) s ||ellla [Jella.
(54)
each (J( cannot exceed its un-normalized counterpart o, For fixed threshold 0, let
rank, (Jr) == # (Jr) 2 <|[Jella). rank, (J+) == (J)t) 2 < <|| 12)
If at)) 2 [eell, then (J) 2 (J)) 2 || and ||ellla & ||il|2 imply (i( [| Thus
(Ja) S rJa)
(55)
Hence, normalizing via tanh(-) can diminish numerical rank: if many Te, are near (i.e., tanh( 1), fewer singular values of Je remain above [eJel. For ChebylKAN of T. layers, the overall Jacobian from input xo to output xz is
Jusal = J1-1J11-2""
(56)
Repeated multiplication by Dr. whose diagonal entries are small, canses compounded attemnation. As T. grows large, increasing number of coordinates reach saturation, thereby reducing the singular values of Josaal until rank, (Jsotal) becomes strictly lower. This phenomenon, referred to as the Nonlinear Normalization Effect, emerges because tanh(-) shrinks partial derivatives, driving many of the product Jacobian's singular values below e || l| and thus decreasing its numerical rank.
A.3 Proof of Theorem 3
Theorem (Exponential Decay Infinite Depth). When the coefficients Ci,k,,, are drawn from mutually independent Caussian distributions, the numerical rank of Jrstut decays exponentially to as the depth L of the ChebylKAn IKAN network increases.
Proof. Step Random Jacobians in Cheby1KAN and Product Structure.
Recall that the 1-th ChebylKAN layer takes an input x; E R" (after a suitable reshaping or dimension match) and produces X1+1 R" via
N
FLLA ΣΣ Cikiim Trn(tan((zz.)). 1...,
(57)
1 m000
where T., are Chebyshev polynomials of the first kind and Cl are the learnable coefficients.
By differentiating equation 57 w.r.t. each layer's Jacobian Jr E pexx has entries
[1]
8m411
Σ Cl..a.m The (ttth(((2).)) (1 tanh"
(58)
When the coefficients Cr..i,, drawn iidd. from standard Gaussian distribution, the partial derivatives 00141. become random variables with zero mean and finite variance. As the network depth L grows, the total Jacobian can be written as
daey.
Josal Jr J2-1 J1.
(59)
Thus, xL Josall in its linearization around any point.
Step 2: Lyapunov Exponents Random Matrix Products.
Let
o
>
(60)
denote the singular values of Jrotal, Define the Lyapunov exponents by
A = lim
1
log logo(") (L)
i= 1, n.
(61)
1
L
21

---

Published in Transactions on Machine Learning Research (01/2026)
By Oseledec's Multiplicative Ergodic Theorem (Oseledels 1968). these limits exist almost surely for products of i.id. random matrices, In our Cheby IKAN setting, the layers' Jacobians Ji approximate a family of random matrices (Uhe Jacobian entries being determined by ii.d. Gaussian coefficients Ci,,,im), making the product Jeotal amenable the same analysis as in classical random matrix theory.
Step 3: Exact Lyapunov Spectrum for Ginibre- Thpe Ensembles.
When each Jt is sufficiently close (in distribution) to an X n Ginibre matrix with iidd Gaussian entries, the Lyapunov exponents } match those of Ginibre ensembles, given by (Newman 1986
2 1
A 2
i=
(62)
2
where is the digamma function, strictly increasing for pusitive arguments.
Step 4: Normalized Singular Values and Their Ratios.
Define the normalized singular values:
(L)
((1)
[2)] I....,n.
(63)
For large L, taking logarithms yields:
log 0;
(1) logo!! logo! (1)
L (2: A1) + o(L).
(64)
Hence;
lim (3( (T.))) 1/2 = c
(65)
L-00
Since A d for i 2 (because & is strictly increasing and - i < n), have
cc-Ai 1, Vi
(66)
Thus, 07 - 0 exponentially in L for 2 2.
Step 5: Exponential Decay of Numerical Rank in Cheby 1KAN.
The numerical rank Rank, (Jhota) is the number of singular values a0 that at least c Equivalently,
a(
(67)
From equation 66 for 2 2,
(14) exp L (o - -1i) + 0 as I - 00.
(68)
Thus, for any fixed e > O, there exists Lo such that for all L > Lo.
5(4)
vi
(69)
This implies that all singular values except the largest one fall below e ca (1) giving Rank, (Jrota)) = for sufficiently large L. In other words, the numerical rank decays to 1 at an exponential rate with respect to the Cheby IKAN depth L.
Since each layer's Jacobian Jr in Che KAN can be regarded random matrix (due to i.i.d. Gaussian coefficients C,,,i.m) the overall product Joosal inherits the spectral properties of random matrix products. Therefore, the interplay of Chebyshev polynomials and the tanh normalization does not negate the fundamental random matrix behavior; instead, the bounded derivative from tanh can further accelerate the decay the subleading singular values. Hence, L co, the effective degrees of freedom in the Cheby IKAN Jacobian collapse numerically to single direction, confirming the exponential rank diminution.
It important to note that Theorem proved under random--cefficient model with ii.d. Gaussion coeffciennts at initialization, and that trained networks do not necessarily satisfy independence or Gaussionity. For this reason, Theorem should be read learned models. Instead, for truined networks, the more relevant theoretical evidence is given by our deterministic layerwise ronk bounds (Theorems 1 and 2), which are further supported by the empirical measurements reported in the paper.
Remark 1.
motivational sufficient-condition result rather than a universal claim about
22

---

Published in Transactions on Machine Learning Research (01/2026)
A.4 Proof of Proposition 1
Theorem 7. Let an AC-PKAN model with L logers (L 2 2) and infinite width. Then, the output y = ((x) non-zero deriealives of any fauile-order with respect to the impul I.
has
First, we clarify that the term "width" for a KAN denotes functional width, the mumber of Chebyshev basis functions per edge (N+1), not the number of neurons as used for MLPs.
Proof.
Consider the forward propagation process the AC-PKAN. We begin with the initial layer:
ho Womhi + bomb,
(70)
U Wi, sin(hoov + by) + wa, cos(holy bv):
(71)
V wv,1 sin(hodv + bv) + wv,2 cos(haOv bv).
(72)
U.
(73)
For each layer = 1,,,.., L, the computations proceed as follows:
Amn
dout N
H Σ
Ckinn (tanh (ae )))
(74)
i=1
Q = 1 +a
(1)
(75)
a (1 a ) " + a o (V+)),
(76)
W + bous.
(77)
During backward propagation, we derive the derivative of the output with respect imput æ, which approximates the differential operator of the PDEs. Focusing the first-order derivative example:
dy dy (a
dx (1)
Or
Want da
(78)
dr
Expanding
21 dx
(L)
dag (L)
O + (1 a1 -0)
(1)
3
0U dag (L)
@V + 1) + t
(L)
2V
Ox
dr
dr
dr
dr
dah (1.
(V - U +
aq
(L)
.V
OU
OU
dr
dr
de
dr
(LH()) aa (a(1-11
2 (V U
( + a (1-11))
0V
a0
8U
(79)
dx
dr
dr
dz
dr
This establishes recursive relationship for the derivatives. Define:
A (H
(1
(80)
Ox
8x
(
11 + a (1-1)
(81)
for each layer 1,2,....4.
For the base case I 1:
(1())
(!)
(1)
(82)
Ox
dm dout
N
!a
Chan T% (tanh(a.0i) sech (ach)(a (0a
+1
(83)
Or
#=1 n=0
0e (0)
aU
dz
dz
W Wy (wo,) cos(holy bu) wa. sin(hooe bb)) 40,
(84)
23

---

Published in Transactions on Machine Learning Research (01/2026)
Moreover,
D H1 + a (0)
dju dout N
Chimnn (taubi(a!0i) + !
(85)
1=1 R= n=0
For layers 1 > 1, where I E :
din doul
1
ΣΣΣ ChinT (tank(("--1)) sach2(a + m=0
(86)
du
We have established recursive relationship.
Notably, the finst derivalive of the Chebyshev polynomial is given by
d sin (marccos(z))
xx)== = dr Tx(x)=
(87)
VI -I-r²
and higher-order derivatives satisfy
(x) 0 for all k
(88)
Therefore, for any order > the k-th derivative of A is identically zero. Consequently, the k-th derivative of the first part of equation 79 is zero.
However, observe that:
dun dous
100
CRan Tn (tanh(all-t))) + aa
(89)
k=
since the derivatives (1-1) for any finite order are non-zero, the derivatives of B(0 are non-zero. Furthermore, we have:
0V
OU
Womb (fy [wv,1 cos(ha@y + bv) sin + by]]
Ox
-y [wu.1 cos(ha@u by) wi.2 sin bu))),
(90)
and derivatives of any finite order of this term are also non-zero. Additionally, the third component of equation at is non-zero. Bg
79
Define
av g(x)
aU h(x)
f(x) = (1 + (c),
(x), dx
(x), dr
(91)
so that the last two terms of equation 79 can be written as
S(x) = f(x) h(x)) + h(x).
(92)
Suppose, toward a contradiction, that S(x) = every x in the domain. Then
f(=)) h(x) ((x)
Va..
(93)
The functions f.g. depend on disjoint parameter blocks: f on [Cr,im): ). g (0v, wy:))) and & on (0v, wa, w0,)). Requiring the above identity to hold for all æ therefore forces a global functional coupling among these independently tumed parameters, which can only occur perturbation of the parameters breaks this perfect cancellation, implying S(x) * for almost all networks. Hence S(z) possesses non-vanishing derivatives of every finite order.
measure-zero subset of the joint parameter space. Any infinitesimal
Consequently, the k-th derivatives the remaining parts of equation 79 are non-zero, and thus the k-th derivatives of equation 78 are non-zero. Therefore, for any positive integer N, the derivative aM that AC-PRAN approximate PDEs of arbitrarily high order.
a exists and is non-zero, establishing
24

---

Published in Transactions on Machine Learning Research (01/2026)
Remark: The properly of possessing nou-zero derivatives of any finite order with respect to the imput æ specifically addresses enhancements in KAN variants rather than in MLP-based models. The litting capability of KAN models relies polyuomial functions with learnable parameters. To KAN incorporates an additional nonlinear bias function b(z)) In contrast, other KAN variants, such as Cheby 1 KAN, rely solely polynomial bases, which inevitably result zero derivatives when the order of differentiation exceeds the polynomial degree. Therefore, Proposition 1 was introduced to provide a theoretical guarantee for AC-PKAN's ability to solve any PDE, analogous how the universal approximation theorem theoretically establishes the universal fitting capability of neural networks.
derivatives in the output, the original B-spline
A.5 Proof of Proposition
2
Definition For a lineer map a V W, we define the kernel be the set of all elements that are mapped to zero
ker a = for E a(x) = =K
(94)
and the image be the points W which we can reach from V
Ima= (EV)
(95)
We then say that r(a) = dim Im is the rank and n(a) dim ker a is the nullity.
Lemma 2 (the Rank-mullity theorem). For linear map V + W, where V is finite dimensional, we have
r(a) n(()) dim Im a dim ker dim V.
(96)
Proof. Let V. W be vector spaces over field and T defined as in the statement of the theorem with dim V
As Ker C V. is subspace, there exists a basis for it. Suppose dim Ker and let
K == uu. vu C Ker(Z')
(97)
such basis.
We may by the Steinitz exchange lemmna, extend K with k linearly independent vectors wi...., W to form full basis of V.
Let
S (wi,..., wn- CVV Ker(T)
(98)
such that
B= KUS- fui,..., 01, 0i...,, An-k) C
(99)
is a basis for V. From this, we know that
Im T = Span T(B)
( = (wr)) (.,....)))........ = Span T(S).
(100)
We now claim that T(S) is a basis for Tm The above equality already states that T(S) is generating set for Im it remains be shown that it is also linearly independent to conclude that it is basis.
Suppose T(S) is not linearly independent, and let
-ー Σ
aaT((u)) Ow
(101)
for some Oj E F.
25

---

are Published in Transactions on Machine Learning Research (01/2026)
Thus, owing to the linearity of T, it Tollows that
Σ ag = Ow =
Σ ayty
E Ker = Span K C V.
(102)
This is contradiction to B being basis, unless all aj equal to zero. This shows that T(S) is linearly independent, and more specifically that a basis for Im T.
Finally may state that
RTnk(T) Nullity(T) dim Im T dim Ker T + [K] (n-k) - dim V.
(103)
Theorem 8. Let N be an AC-PKAN model with L layers (L 2 2) and infinite width. Then, the Jacobian matrix aN is full rank in the input space R*.
JN(()) =
mxd
Proof. Let the output be y Wouth (t) + bout where Wout E Xdo and da denotes the hidden layer width. Under the infinite-width assumption, d, + cc. The k-th output component corresponds to the k-th row of Wout, denoted as w., i.e.,
wi a (1) + bout,k.
(104)
Its partial derivative with respect to the input æ is:
Oyk
Oa (2)
W4
(105)
Ür
can be decomposed into a nonlinear combination of parameters across layers. Specifically, for any layer the derivative term (a(l) is generated through recursive
Following the recursive relationship in Theorem
(2)
De
Cill operations involving parameters ka,n wu, Ou, 0r, etc.
Consider the partial derivatives
By 0ya and du
# K²). Since:
ou
Oyk (1)
(1)
dyw 8r
Da
We 8x
(106)
dr
dx
if we and wa are linearly independent and the column space of do (L) ox is sufficiently rich, then dys az and
thys
Ox
guaranteed be linearly independent.
Under infinite width, the parameter matrices cC e Rdon xdm ((+1)) (where dm is the intermediate dimension) and the dimension dout of W.n can be independently adjusted, making the parameter space infinite-dimennt dimennsonal Hilbert space, allowing the construction of arbitrarily many linearly independent basis functions.
By the infinite-dimensionall parameter space afforded and wv such that columns of (2,
oo, may construct parameter matrices (C0), wr, dz E R become linearly independent. Specifically, let (vejém be the column Through parameter configuration in hidden layers, ensure:
vectors of (2(2) dz
Ve; E R.
Σ c.. Cv= =0 = 4 = 0, W
(107)
For the output matrix Wont E R mxd,, construct mutually orthogonal row vectors [wa] satislying:
(W&,Ww) Wew -uxr [arr|/wel?? Vk + K
(108)
26

---

Published in Transactions on Machine Learning Research (01/2026)
where òear is the Kronecker delta. The Jacobian become:
Oyk dr
)a.l) Wk
[(w[v:)e
(109)
dx
where { are standard basis vectors in Rd, For distinct K, consider:
Oy
Or
Oye
Or
Σ
(w
Σ vivi
ww
(110)
Since }v.} are linearly independent, Eis V.v. is positive definite. Combining with the orthogonality of {wa), we have:
W
Σ viv! Wk =0 &
(111)
Thus, the Jacobian Su mutually orthogonal and linearly independent. The full rank property follows from the infinite-dimensional orthogonal system.
We proceed by induction the number of layers
Base Case (L= 1): By Equation that
82 there exist parameter choices (wo, (v: and orthogonal weights {Wx) C W such
(0)
Wk'
ôRs Daw///ws/ ||We]]
(112)
dr
1
establishing linear independence of [wX (0 (0)
100
Inductive Hypothesis: Assume do (L-11
has full-rank column space R (allh-1)) Ox
Hi-1 C H. with dim Hr. = oc.
Inductive Step: Let Pa Pay.. be orthogonal projection onto H Through Equations
and 80
we decompose:
(L
C(L)
(L(L)
(L-11)
+o
(113)
dr
Or
Ft
By the parameter freedom ((M), there exists a choice such that:
dim R (Pit., oo and R(Tx) = (0).
(114)
This induces the dimensional extension:
(a(L)) = He-1 R
( (Pit_ITz).
(115)
where denotes orthogonal direct Since dim dim(R(Ptt (R(PR Pt)) oc, the infiiieeddimensional full-rank property propagates to layer L.
By induction, we conclude that: the column space of (a Oz
is infinite-dimensiooaal and full-rank; the row vectors of
W... are mutually orthogoual.
Thus, we have:
Oyk dr
Oyk dx
For + k. Va, IR,
0 =
(116)
27

---

Published in Transactions on Machine Learning Research (01/2026)
Consider the Jacobian matrix Ju (æ) linear mapping Jx( R2 - R". According the rank-mullity theorem, we have:
dim((er(Jxx(z))) rank(J)x())) =d
(117)
Theorem
7 guarantees that rank(Jx(x)) = min(d, m). Thus, the dimension of the kernel space is
dim(ker(Jw<z))) = d min(d,m)
(118)
Specifically, this can be further categorized into two cases:
When 2 d: the Jacobian matrix has full column rank rank(Jy(())) d, resulting in ker(Jx((x) 0.
Ju (x) is injective.
When m < d:the Jacobian matrix has full row rank ronk(Jx(w)) = m, resulting ker(Jv(z)) = m, which means there exist d - m linearly independent non-zero vectors such that Jy(()) = 0.
Let us exclude non-zero mull vectors by contradiction. Assume there exists a non-zero vector v * 0 E Rd such that For any output component Ni; we have
ON. 0a
ON. Ora
ON. Öra
12
(119)
According to Theorem 7 and Equation 116 the only solution is v 0, which contradicts the assumption. Therefore, the null space contains only the zero vector, i.e., (z))))((Jx(z))) =
Suppose that the Jacobian matrix is rank-deficiett, i.e. there exists measure-zero set M IRd with ((M) 0 (where y denotes the Lebesgue measure) such that:
rank (Jx(2=) min(d, m) Yo M.
(120)
This implies that the image of the mapping (ar) is constrained to lower-dimemsional submanifold S C CR", where:
dim(S) S rank (Jx((z)) < min(d,m)
(121)
By Theorem however, all first-order partial derivatives aN of O. Specifically: (1) A direction e Rd {0} . at ON: least one output component Nr such that du # 0; (2) The infinite-width architecture of AC-PKAN ensures that the parameter space is dense in the I,2 function space. Consequently, the image of the output mapping can densely cover any open set in R"
dzy
If there is rank deficiency, then Iv Rd, for Vi, aN dey
0, contradicting the non-degeneracy of the derivatives.
Consequently, except for measure-zero set M, we have:
rank(J((æ)) = min(d, m))
(122)
indicating the Jacobian matrix JN(x)
&N
is full rank in the imput space Bd..
Die,
mxx
A.6 Finite-Widee Width Spectral Analysis of AC-PKAN
In this section, we provide theoretical guarantee for the stability of AC-PKAN in the finite-width regime. While proposition 2 eestablishes full rank in the infinite-width limit, the following theorem quantifies the spectral lower bound of the Jacobian under practical initialization conditions, ensuring that the model avoids rank collapse.
Theorem (Layerwise spectral lower bound under diagonal scaling and near-identity perturbations). Under the per-layer factorization and bounds stated in Assumption A, namely
J = Diagf (I+
min 10 ( 12
A Sn <1,
it holds for every E (I...., L.} that
(2 2 7(1-1).
and for the end-to-end Jacobian Juortl = JAC ... JJC that
mmin (hotal) 2 [1(1 n))
28

---

Published in Transactions on Machine Learning Research (01/2026)
Proof. Let D Diauf (0 and E ===a Ao The smallest singular value admits the variational form
Omin (min(()=
min ||Ar]||. |xl|2=1
(123)
For the diagonal factor, using D Diag(1s} P ....... 1822
dn
|Dxl = T DT DJDD. Dr =
1 P23 2
min 18g
min 1 (o
(124)
1=1
1=1
and hence
mmin (D)
min | |=lla==
2
min 10, 2 7.
(125)
For the near-identity factor, for any ||0||2 = 1,
II + E)a|12 2 ||/||2 |x 2 1 -1
(126)
which yields
Gmin (I + min [I + E)æl12 2 1 |6|| m.
(127)
1=||==1
Combining the two factors, for any |
|D + E)al1a m (D) [(I E)+l2,
(128)
and taking the minimum over the unit sphere gives
mmin D(1 E))
Omin O (1tim|++1 E) 7(1 n).
(129)
so
miin (J 7(1 17).
(130)
For the end-to-end product, using the variational form twice,
Gmin (AB) min
||Ba||2 2 Omin(A) min [|llla=1
Ba omin(() amin (B),
(131)
[|=
and iterating over JAC ... JAC yields
L II J
L.
Tmin (Jroaa))
Oain
m mmin(J (20)
2
II 71 1) = -[>(1- [r(1-1)]² m))
(132)
Theorem 10 (High-probability full rank via width and a generic output layer). Let JN = Wour G(z)) with G(x) E Rd and m rows(Wout) 2 d. For any 8 (0, 1), there exist absolute constants c, C, c1, C2 0 (depending only on the subgoussian class) such that, if
dn 2 C ( d 1ogglog?) log
then with probabilily at least 1 - 8 one has mmin(((x) 2 Vdn - o, Vd es (og(((2))) > 0, hence rank G(x) = d; rank(Wout
moreover, conditioning on G(z), rank (Wou G(r)) = d almost surely for any Wout that is independent of G(z), has continuous distribution, and rank(Wout) d.
Proof. Assume: (a) the rows of of G(x) independent isotropic subgaussian vectors in Rd with ( u)lluz S K for all u 5d-1; (b) Wont is independent of G(z), has continuous distribution, and sank(Wout) 2 d.
Let 7 R xd have i.i.d. hound for the smallest singular value of matrix with independent isotropic subgaussian rows, there exist absolute C, > such that for all t
2J distributed the rows of G(z); then Z 1 G(z). By the non-asymptotic lower tail
IP Tmin (Z) S Vdh c22 - 1 s 2c -ot
(133)
29

---

Published in Transactions on Machine Learning Research (01/2026)
Choose 1 02 (12g(2/0) to gel
Pf Ommml (2) s ddn -dh--KkVvd- 2v log s 8.
(134)
To ensure strict positivity of the right-hand side of the lower bound, require
ddn 2 a c2 log
(135)
It suffices impose
da 2 CK (d log log
(136)
with an absolute large enough so that, using va b 2 (Va+ + vb/ for a,
dd VCK² V d log 2
vC
Kav=
vC \2
x2 2 eik2 oz 108
(137)
V2
Under equation 136
equation
134 yields miin (7) > 0 with probability at least 1 8, hence rank Z d. Since
Z d G(r), we conclude rank ((z) with probability at least 1- 5.
Condition on realization of G(z) with rank G(x) = d. Let U E R xd have orthonormal columns spanning C(G(z)).
Then there exists Rdddd invertible such that G(z) UR. Since is invertible,
rank(Wout
= rank (Wont = =rank[ rank(WoutU) (Wo.U).
(138)
Thus raun = d iff Uaur(Wor = d. The map W + WV is linear from R=xd, to Rmxd The event (rank(WUU) d} is the algebraic set where all d x minors of WT vanish; least one such minor is nonzero polynomial (e.g., take W whose first d rows equal UT), so this set has Lebesgue measure zero. Because Wout has a continuous distribution and independent of G(z),
P(rank(W..nU) < d G(z)) =0
(139)
and therefore G(z)) d almost surely (conditional G(z)). Combining with the high-probability event {rank ((x) = d} completes the proof.
Corollary 1 (Tilde-S width). Choosing 8 = d d' with fixred d' > 0 in Theorem 10 gives the succinct repuirement
dn 2 ((K+d) = SA(T k dlogd),
(140)
under which rank (x) = d holds with probability at least 1 - d'd
Remark 1 (Non-isotropic rows). If the rows of G(z) have coveriance En > O and are subgoussian with perameter K, apply whitening: G(x) = 21/22 Z with Z isotropic subgoussian (up to change in the subgaussian constant). Then
amin (G()) 2 V Jmin (Z),
(141)
02 1og(2/88) and the full-rank conclusion - Theorem 11 (Generic full column rank at fixed input; analytic activations). Let fo - 1h be feedforward network obtained by composing affine maps and coordinatewise real-analytic activations, and fix E Rd. Denote J(z; 0); = fo(x)/ f mxd Assume the hidden width satisfies d,, 2 d and m 2 Then the singular parameter set
c, K22 the same argument yields auin (G(x)) 2 (Em -a Vd follows under the same scaling of dr. (up to constants depending on ).).
Ozig(()) = (0: rank J (x; < d}
is proper real-onalytic (indeed, semianalytic) subset of the porameter space O R. and has Lebesgue measure zero. Consequently, for any initialization drawm from a distribution that is absolutely continuous w.r.t. Lebesgue measure on θ, one has rank J(z; = d almost surely.
Proof. Write the network with L layers
ho = x, fe(x) = Wiht-1 + bb.
ac Wehe-1 be, he:=ve(ae) (1SELL-1),
(142)
30

---

Published in Transactions on Machine Learning Research (01/2026)
where each de R44 -> 12dh acts coordinatewise by a real-analylic scalar nonconstant function (the final layer is afline; if an analytic dr. is also used, the argument below is unchanged by inserting its derivative). The Jacobian at a is
[ax; 0) = WL III W)).
De(2; = Diag (1(((()((((;;0)))
(143)
Each entry of J(x;0) 0) is obtained from (We, be) through finitely many additions, multiplications, and compositions with te and d/- As real-analytic functions are closed under these operations, every entry of [x;; () real-analytic 0. Hence any d x d minor M (0) of (x; is real-tnalytic in 0, and
+ aa)g all d minors M
n
(0: M()) = 0)
(144)
real-analytic (indeed, semianalytic) subset O.
It remains to show that (z) is proper. Choose, for each 1 1 S 1, scalar E R such that (c(ee) * 0 (possible because de is nonconstant real-analytie). Let au d'((ce) Construct follows. For the first layer, set
W1 =
Ta 0
E Rd.
b1
c1 l 0
Wix,
(145)
that (1(2;0) (ci *) and therefore Di 0) Diag(anda, *). For 2 S l L set
Ta 0
E Rdhaxaa
be c
Wehe-1 (x;
(146)
0
which enforces (=(2;)) (crld, *) and hence Dz(x; 0) = Diag(a Las *). By induction,
II t=1
1a
(147)
0
1=11
Choose Wr E mx so that its first columns are linearly independent (possible since mn 2 d), e.g. W, Ia 0] after reordering columns if needed. Then
L-1
J(z; WL [II De 0)
II
Ja
at
Wz
(148)
0
whose x d left block has rank d. Thus rank (w; 0) = d, at least one d x d minor (. is not identically zero e.
Since the zero set of nontrivial real-analytic function has Lebesgue measure zero in R each set {0: M = 0} has measure zero, and the finite union @siu((a) Unr (0: M(0) = 0} has measure zero as well. Therefore Qsing (x) a proper real-analytic (semianalytic) subset of O, and any absolutely continuous initialization lies in O Ozing (æ) with probability
Proposition 3 (Finite input sets). Let X = (ar(1) (.N))....(1))) Rd be finite. If for each (t there exists (it) with rank ((())) = d, then
N Osiie (z (1)
Oxing (X)
1=1
is fonite unson of measure-zero sets and therefore has Lebesgue measure zero. Thus, with probability one under any continuous initialization, the Jacobian has full column rank simultaneously on all points of X.
Remark 2 (Scope and limitatious). The argument existential/ ciistential/generic: generic: certifies that the "bad" parameter set is
zero, bat it does not provide a quantitative lower bound (Jxn(((x;;)) It complemends nonasymplotic concentration bounds (which gield explicit spectrul gaps under width assumptions) by showing that vank-deficient parameters form null set in θ.
31

---

Published in Transactions on Machine Learning Research (01/2026)
B
Explanation for the Efficiency of Chebyshev Type I Polynomials Over B-Splines
Let denote the batch size, Pm the imput dimension, Dout the output dimension, and N the Chebyshev degree. A single forward pass through Cheby1KAN layer performs
(i) tanh(-), clamp/ acos / cos O Pm (N + +));
(ii) einsum contraction: O Di Dow (N + 1)),
yielding overall time complexity of
TChahylKAN O B Din Dont (N 1)).
Iis peak memory usage comprises the coefficient. lensor of size Dw x Dout x (N 1) and the expanded activation tensor of size B x Din x (N + 1), giving
Mcneby1KAN = O(m Dout (N +1) + B (N+1)..
In contrast, B-spline based Kernel Adaptive Network (KAN) with L layers, layer widths (WeYe=o grid size G, and spline order & must, at each layer E, (i) locate each input in knot interval, (ii) evaluate local polynomial bases, and (iii) perform weighted sums. For typical implementations this yields
L-1
TKAN O
We ke
while storing both the grid arrays of size We x (G + k 1) and coefficient arrays of size W441 x We x (G+A), as well as intermediate activations ( E, WeWe+1). Hence
L-1
MKAN E
+ B
Σ We We
Discussion. By replacing piecewise B-splines with globally supported Chebyshev polynomials, ChebylKAN eliminates the need for (i) knot-location logic, (ii) local interpolation routines, and (iii) repeated recursive basis- function updates. All operations reduce to standardized vectorized transforms (lanh, acus, cos) and single rank-3 tensor contraction, which are highly optimized on modern hardware. Cheby1KAN achieves lower asymptotic time complexity and substantially smaller memory footprint than its B spline counterpart while improving the model's ability to capture high-frequency features.
C
Additional Ablation Studies
C.1 Ablation Internal Attention and Wavelet Activation
To clarify which components are responsible for the performance gains of AC-PKAN, we conduct controlled ablation on wave equation benchmark, following recent analyses of Chebyshev- based physics-informed informed KANS and
RGA-style adaptive training for PDEs (Guo et al. 2025 Rigas et al. 2024 Zhang et al. 2025b
Mostajeran & Faroughi
2025). Our ablation consistent with prior work reporting that Chebyshev KAN variants improve expressiveness but
also introduce gradient stiffness and potential rank decay in deeper stacks (Danesmaad et al. 2020 Yang et al. 2024
Ablated variants. We define three progressively enhanced baselines. AC-PKAN(min) augments Cheby with only linear input and output projections, denoted by Wemb and Wout- Wavelet activation, internal feature attention (the U/V gating and the layerwise (0) injection), and the external RGA controller are removed. This model keeps the projection channel that mitigates rank collapse but is the smallest departure from a pure Chebyshev KAN, AC-PKAN(no-attn) further adds the wavelet frequency activation on top of Cheby1KAN and the linear projections, while still disabling the internal feature at atlention and RGA. This variant isolates the contribution of emriching the activation space with multiscale responses. AC-PKAN (sin) restores the internal feature attention and keeps the linear projections, but replaces the wavelet activation with a simpler sine activation. This variant tests whether the layerwise gating the primary factor that improves stability and expressiveness higher effective dimensions.
32

---

Published in Transactions on Machine Learning Research (01/2026)
Table 7: Ablation on the ID wave equation showing the effect of linear projections, wavelet activation, and internal feature attention.
Model
rMAE
rRMSE
AC-PKAN(min)
0.5374
0.5495
AC-PKAN(no-atta))
0.5116
0.5081
AC-PKAN(sin)
0.4478
0.4391
Quantitative results. Table reports relative MAE and relative RMSE on the 1D test set.
Three observations emerge from these results. First, adding linear up and down projections alone not enough to reproduce the overall gain of the full AC-PKAN. Projections widen the channel space and help preserve mondegenerate embedding, but they do not fully address the rank shrinkage induced by Chebyshev expansions. Second, reintroducing the internal feature attention delivers the largest single improvement this axis. This supports design choice that a learnable layerwise gate (4) that mixes the U/V projected features is central for keeping the Jacobian well conditioned, in line with other RGA- AGG-iisppird KANs for PDEs. Third, wavelet activation effect compared to attention. It enriches the frequency content and stabilizes the fit oscillatory targets, which is consistent with reports on Chebyshev KAN domain scaling and hybrid encoder decoder PKANs.
positive but smaller
The ablation confirms that the main driver of stability and scalability in AC-PKAN is the proposed layerwise internal feature attention. Linear projections are necessary to keep wide embedding space, and wavelet activation further improves accuracy oscillatory solutions, but neither of them alone explains the performance level reached when attention present.
C.2 Ablation Chebyshev Polynomial Degree
We further examine how the Chebyshev polynomial degree affects the behavior of AC-PKAN. isolate this factor, we keep the full architecture and all training hyperparameters fixed and sweep the polynomial degree N of the Chebyshev expansion on the 1D wave equation benchmark.
Experimental setup. The backbone the complete AC-PKAN with internal feature re-injection attention and frequencyddomai activation enabled. Only the polynomial degree N is varied. We report relative MAE and relative RMSE on the same held-out test set.
Table 8: Effect Chebyshev polynomial degree on the ID wave equation.
Degree
rMAE
rRMSE
0.0196
0.0200
6
0.0200
0.0205
8
0.0011
0.0011
10
0.0128
0.0131
Observations. The results in Table 3 exhibit clear nommonotonic pattern. Degrees 4 and 6 are underexpressive for this problem, both staying near the 2 10 error level. Increasing the degree to 8 produces sharp drop to the 10-3 regime, which indicates that the model reaches the expressive range where the internal attention and the dremuency-domain activation fully operate. Pushing the degree further to 10 causes the error to rise again toward the 10 scale.
Three conclusions follow from this pattern. First, raising the polynomial order alone is not reliable strategy for improving accuracy. Second, the best performance emerges when the polyuomial degree is matched to the internal feature attention and the Irequency-domain activation, which shows that AC-PKAN benefits from the joint design rather than from a single aggressive expansion. Third, very high polynomial order amplisies gradient magniludes and worsens numerical conditioning, which increases optimization difficulty and offsets the gain in representation power.
33

---

Published in Transactions on Machine Learning Research (01/2026)
AC-PKAN reaches its strongest accuracy when the Chebyshev degree is chosen in range that is expressive bul still well conditioned. The peak result is a product of the selected degree logether with the internal attention and Irequency-aware modules, not of naive parameter scaling.
C.3 Ablation on Operator-Learning Baselines under Sparse Supervision
We add parameter-matched FNO and DeepONet baselines under the same sparse-data protocol Table I and Table across three settings: 1D convection cnneccionddifuusion--- reectio, 2D lid-driven cavity, and 3D point-cloud Poisson. Metrics are reported as rMAE and rRMSE.
Table 9: FNO and DeepONet under the sparse-data protocol used in Table and Table
Setting
Model
rMAE
rRMSE
1D Convection Diffusion Reaction
DeepONet
0.0730
0.1064
1D Convection Diffuusoon Snlfusioo-RRactioon Reaction
FNO
0.0722
0.1061
2D Lid-Driven Cavity
DeepONet
0.6270
0.6242
2D Lid-Driven Cavity
FNO
0.5605
0.5312
3D Poisson (Point-Cloud)
DeepONet
3.4566
2.7978
3D Poisson (Point-Cloud)
FNO
2.4388
2.0426
Observation. Across these sparse-sueervision tests, operator-learning baselines without physics information underperform AC-PKAN. The updated results are included in the revised manuscript.
D
Impact Statement
This work advances Physics-Inlormed Neural Networks (PINNs) by integrating Kolmogorov Armold Networks (KANs) with Chebyshev polynomials and attention mechanisms, improving accuracy, efficiency, and stability in solving complex PDEs. The proposed AC-PKAN framework has broad applications in scientific computing, engineering, and physics, enabling efficient and interpretable machine learning models for fluid dynamics, material science, and biomedical simulations. Ethically, AC-PKAN enhances model reliability and generalizability by enforcing physical consistency, reducing risks of overfitting and spurious predictions. This work contributes to the advancement of physics-informed AI, with potential in digital twins, real-time simulations, and Al-driven scientific discovery.
E
Experiment Setup Details
We utilize the AdamW optimizer with a learning rate x 10 and weight decay of 1 10-4 in experiments. Meanwhile, all experiments were conducted on an NVIDIA A100 GPU with 40GB of memory. And Xavier initialization is applied to all layers. PDE-Solving problems, We present the detailed formula of rMAE and rRMSE as the following:
N Em (ā(an, (n.) u((n: bn)
IMAE
Emeo |u(xx, bn)l
(149)
[ (n) u(xm)
FRUSE
1m=1 |u(In, 12
where N is the number of testing points, à is the neural network approximation, and u is the ground truth. The specific details for each experiment are provided below. For further details, please refer to our experiment code repository to be released.
E.1 Running Time
We present the actual rumning times (bourssminteesssconnis for all eight PDEs experiments in the paper. As shown AC-PKAN demonstrates certain advantages among the KAN model variants, although the running times of all KAN variants are relatively long. This is primarily because the KAN model is relatively new and still its preliminary stages; although it is theoretically innovative, its engineering implementation remains rudimentary and
in Table 10
34

---

Published in Transactions on Machine Learning Research (01/2026)
lacks deeper oplimizations. Moreover, while traditional neural networks benefit from well-established optimizers such as Adam and L-BFGS, optimization schemes specifically tailored for KAN have not yet been Uhoroughly explored. We believe that the performance of AC-PKAN will be further enhanced the overall optimization strategies for KAAN variants improve.
Model
First PDEs
Last
1D-
1D-
2D NS
1D Conv. 2D Lid-
Hetero-
Complex
3D
Wave
Reaction Cylinder Diff. Reac. driven Cavity
geneous
Geometry Point-Cloud
Problem
PINN
00:21:14
00:09:07
00:15:20
00:15:12
00:06:39
00:23:30
00:01:08
00:49:31
PINNsPormer
00:44:21
00:04:09
00:58:54
02:06:37
14:01:55
00:13:31
Qees
01:41:34
00:02:10
00:24:39
00:25:46
00:13:04
00:20:50
00:01:46
01:32:24
01:38:01
00:01:29
00:50:26
FLS
00:13:38
00:01:08
03:04:41
CAN
03:32:10
00:12:08
04:24:59
01:45:37
00:50:45
00:03:21
02:27:27
Cheby2KAN
05:03:18
01:06:54
05:41:42
03:01:40
01:35:40
00:03:27
05:26:42
AC-PKAN
01:13:01
00:15:16
02:21:40
02:01:59
01:13:11
00:01:04
01:34:21
KINN
25:00:20
14:31:42
02:41:49
01:51:44
00:14:07
IKAN
12:44:16 01:21:25
05:19:04
02:06:36
06:21:00
00:16:06
07:33:25
FastKAN
09:35:51
05:51:21
02:04:42
03:37:57
00:17:23
TKAN
08:20:31
00:13:09
03:01:41
01:54:22
00:17:41
00:52:05
00:06:22
04:04:48
FourlerKAN
03:33:46
01:21:50
02:48:50
02:08:08
07:40:43
00:18:26
13:36:48
Table 10: Running times (hh:mm:ss) for all eight PDE experiments. First row: Five simpler PDEs; second row: Three more complex cases.
E.2
Complex Function Fitting Experiment Setup Details
The aim this experiment to evaluate the interpolation capabilities several neural network architectures, including AC-PKAN, Chebyshev-based KAN (ChebyKAN) traditional MLP, and other advanced models. The task involves approximating target noisy piecewise function, defined over three distinct intervals.
Target Function
The target function f(x) is defined piecewise follows:
sin(25 + x + 0.5 (30rr)) + 0.2x
< 0.5,
f(x)
0.5ie + (sin())))) + 0.3r cos() + 0.1e
0.5 < <, 1.5,
In(x=1) (2TT) + 0.2 sin(( + 0.1 In(z+1) In(2)
1.5,
In(3)
with added Gaussian noise e ~ 01..
Dataset
Training Data: 500 points uniformly sampled from the interval [0, with corresponding noisy function values = f(x) +
Testing Data: 1000 points uniformly sampled from the same interval a: [0.2] to assess the models' interpolation performance.
Training Details
Epochs: Each model trained for 30,000 epochs.
35

---

Published in Transactions on Machine Learning Research (01/2026)
Loss Function: The Mean Squared Error (MSE) loss is utilized to compute the discrepancy between predicted and true function values:
MSEE
Σ M 0.)²
N
Weight: Initialization: Xavier initialization is applied to all linear layers.
Model Hyperparameters
The parameter counts for each model are summarized in Table 11
Table 11: Summary of Hyperparameters in Complex Function Fitting Experiment for Various Models
Model
Hyperparameters
Model Parameters
Layer 1: Cheby1KAM Layeeayer
Layer (1, 7, 8)
Cheby1
Layer 2: Cheby (7, 8, 8)
639
Layer 3: Cheby 1KANLayee 8, 1, 8)
Layer 1: Cheby2KANTaayer(I. 7, 8)
Cheby2KAN
Layer 2: Cheby2KANLayerr(T 8, 8)
639
Layer 3: Cheby2KANLaaerr (8, 1, 8)
Layer 1: Linear(in=1, oo0= Activation=Tanh
PINN
Layer 2: (inear(in=16) out=32), Activation=Tanl
609
Layer 3: Linear(in=32,, out=1)
Linear Embedding: (ineer(iin1,, out=4)
Hidden ChebyKAN Layers: x Cheby (ANLaye))
AC-PKAN,
Hidden LN Layers: LayerNorm (fcatuurse)))
751
Output. Layer: Linear(in=6, out=1)
Activations: WaveAct (U and V)
KAN
Layers: 2 X KANLincar (32 neurons, SILU activation)
640
Layer 1: Linear(in=1, out=16), Activaton=-JaobiiKANN
rKAN
Layer 2: Linear(in=16, out=32)), Acivation==PadRRAAN)
626
Layer 3: Linear(in=32,ontt
Layer 1: Linear(in=1, out- 16) Acchattoo-Phettooomlloooeeeeeeee
IKAN
Layer 2: Linear(in=16, out=32), Adtiatior-TTatttoomlaoooVVvnnooo
615
Layer 3: Linear(in=32, out=1)
FastKANLayer 1:
RBF
8 out=32)
Base Linear(in=1, out=32)
FastKAN
658
F 2:
RBF
SpiineLinear(nn2556, out=1)
Base Linear Linear(in=32, (i==32, out=1)
FourierK, ANNeeer NaiveeouuierrKANLayerrr
FourierKA KAN
FourierK A Aayyy
AA
685
AoLavrr 3: NaiveFourierKKNLayerr
E.3
Failure Modes in PINNs Experiment Setup Details
We selected the one-dimensional wave equation (1D-Wave) and one-dimensional reaction equation (1D-Reaction) as representative experimental tasks to investigate failure modes in Physics-Innormnn Informed Neural Networks (PINNs). Below, we provide comprehensive description of the experimental details, including the formulation of partial differential equations (PDEs), data generation processes, model architecture, training regimen, and hyperparameter selection.
1D- Wave PDE.
The WDve equation a hyperbolic PDE that is used to describe the propagation of waves in spatial dimension. It often used in physics and engineering model various phenomena, such as sound
36

---

Published in Transactions on Machine Learning Research (01/2026)
waves, seismic waves, and electromagnetic waves. The system has the formulation with periodic boundary conditions us Tollows:
8'u
0'u de E
3 dx²
E
[0.
0r2
au(z, 0) = 0 0t
(150)
IC: (f: 0) = (ir( sin(3TT))
BC: u(0, = u(1,t)
where 3 is the speed. Here, specifying 8 = 3. The equation has simple analytical solution:
u(x sia( con(x)))os((tt sin(3Tr) cos(2
(151)
1D-Wave PDE Experiment Dataset In 1D-Wave PDE experiment, no dataset were utilized for training. Collocation points generated to facilitate the training and testing of the Plysics-Informed Neural Network (PINN) within the spatial domain x E [0, 1] and the temporal domain t E [o, 1]. A uniform grid was established using 101 equidistant points in both the spatial (x) and temporal (1) dimensions, resulting
total of 101 101 10,201 collocation points. The PINN was trained in data-free, unsupervised manner on this 101 x 101 grid. Boundary points were extracted from the grid enforce Dirichlet boundary conditions, while imitial condition points were identified at t = 0. Upon completion of training, the model was evaluated on the collocation points by comparing the predicted values with the actual values, thereby determining the error.
The one-dimensional reaction problem is a hyperbolic PDE that is commonly used to model chemical reactions. The system has the formulation with periodic boundary conditions follows:
ID-Reaction PDE.
tu Øt
pu(1 - u) = 0, Vr E 2m), E [0,
(152)
[C 0) exp (op((-1-)) (-2(7/4)²) BC: = u(2
where p is the reaction coefficient. Here, we set p= 5. The equation has a simple analytical solution:
(xx) exp(pt)
Uamalytical h(x) exp(pt) + h(x)
(153)
where ((()) is the function of the initial condition.
1D-Rcaction PDE Experiment Datasct In the ID-Reaction PDE experiment, no dataset were utilized for training. Collocation points were generated to facilitate the training and testing of the Physics-Informed Neural Network (PINN) within the spatial domain x E [0, 1] and the temporal domain t E [0, 1]. uniform grid was established using 101 equidistant points in both the spatial (z) and temporal (t) dimensions, resulting in total of 101 x 101 = 10,201 collocation points. The PINN was trained in a data-free, unsupervised ised manner on this 101 x 101 grid. Boundary points were extracted from the grid enforce Dirichlet boundary conditions, while initial condition points were identified at = 0. Upon completion of training, the model was evaluated on the collocation points by comparing the predicted values with the actual values, thereby determining the error.
2D Navier-Stoker Flow around a Cylinder The two-dinenssonall Navier Stokes equations are given by:
Du
Du 11 Dr.
Ou Oy
Op dr
8t Or
8uu Jy
+
+ d
0t
(154)
Ov
Ov
Op
8
Øt
+ Au
Oz
+
v
0y
=
Oy
+
Xa
Oy²
where v.)) and v)) are the I- and y-components of the velocity field, respectively, and p(t, v) is the pressure field. These equations describe the Navier Stokes flow around a cylinder.
We set the parameters A = and A2 = 0.01. Since the system lacks an explicit analytical solution, we utilize the simulated solution provided in Raissi et al. (2019). We focus the prototypical problem of incompressible flow past.
37

---

Published in Transactions on Machine Learning Research (01/2026)
circular cylinder, scenario known exhibit rich dynamic behavior and transitions across different regimes of the Reynolds number, defined as Re = " By assuming a dimensionless free-stream velocity the = 1, a cylinder diameler D and kinematic viscosity by an asymmetric vortex shedding pattern in the cylinder wake, commonly known as the Kármán vortex street. All experimental settings the same in Raissi al. (2019). For more comprehensive details about this problem, please refer to that work.
0.01, the system exhibits periodic steady-state behavior characterized
2D Navier-Stokes Flow around a Cylinder Experiment Datasct For the 2D Navier Stokes Flow around Cylinder Experiment, the dataset used is detailed as follows:
Variable
Dimensions (5000, 2)
Description
X (Spatial Coordinates)
Contains 5,000 spatial points, each with 2 coordinate values (x and y)- Contains 200 time steps, each corresponding to a scalar value.
t (Time Data)
(200, 1)
U' (Velocity Field)
(5000, 2, 200)
Contains 5,000 spatial points, 2 velocity components (u and v), and 200 time steps. The velocity data of each point is a function of time.
P (Pressure Field)
(5000, 200)
Contains pressure data for 5,000 spatial points and 200 time steps.
Table 12: Dataset used in the 2D Navier-Stokes Flow around a Cylinder Experiment
From the total dataset of 1,000,000 data points (N T' = 5,000 200), randomly selected 2,500 samples for training, which include coordinate positions, time steps, and the corresponding velocity and pressure components. The test set consists of all spatial data the 100th time step.
1D Conveciinn-iffiiioo--eeeee Reaction (CDR) equations, which model the evolution of the state variable w under the influence of convective transport, diffusion, and reactive processes. The system formulated with periodic boundary conditions as follows:
Equations. We consider the one-dimensional Convection- DanvectionDifffusionn
du Du
Fu
3
Var E [0, E 10, pull 0,
ōt
Oz
(155)
(x T)² 2( 4J²
IC: u(x, 0) = exp
BC: u(0, = u(2m,()
In this equation, .9 represents the convection coefficient, v is the diffusivity, and is the reaction coefficient. Specifically, we set 3 1, 3, and 5. The reaction term adopts the well-known Fisher's form pu(1 - u). as utilized in Krrihhaapryaan et al. 2021). This formulation captures the combined effects of transport, spreading, and reaction dynamics on the state variable u.
1D Coanection-Diffusinn Reaction Experiment Dataset The datasel for the ID Convection-Diilusiii Reaction experiment comprises three variables: spatial coordinates (x), temporal data (1), and solution values (u). Specifically:
Variable
Dimensions
Description
x (Spatial Coordinates)
(10, 201, I)
Represents spatial points uniformly distributed over the
domain [0,2m]
t
(Time Data)
(10,201, 1)
Denotes temporal data spanning the domain [0, 1 for
solution evolution.
(Solution Values)
(10, 201, 1)
Contains the computed values of the solution function u(x, at corresponding spatial and temporal points.
Table 13: Dataset used in the 1D
Experiment.
Out of the lotal 10,201 data points, the dataset was partitioned into training and lest sets. The training data includes boundary points (where 0 = 2n)) and random sample of 3,000 interior points, which used to compute the loss function during model training. The test data consists of the entire remaining dataset, ensuring compreheusive evaluation of the model's performance.
38

---

Published in Transactions on Machine Learning Research (01/2026)
2D Navier-Stokes Lid-driven Cavity Flow We consider the two-dimensional Navier Stokes (NS) equations for lid-driven cavity flow, which model the incompressible fluid motion within square domain under the influence of moving lid. The system is formulated with periodic boundary conditions follows:
1 n1 Vu Vp Au O,
E [0,
Re
V v-u= vx f2, (0,7)
IC: u(x,0) 0) =0
(156)
BC:
(1x(1 0), xETi
== (0, 0), x x 22
p p= 0, x (0,0)
In this formulation, u= (u, u) represents the velocity field, p is pressure field, and set to Re 100. The domain is Q = [0, 1J?, with the top boundary denoted by Dı where the lid moves with velocity == (4x(1 - z), 0). The left, right, and bottom boundaries are denoted by T2, where a no-slip condition (0,0) is enforced. Additionally, the pressure is anchored at the origin (0,0) by setting = 0.
Reynolds number,
2D Navier-Stokes Lid-driven Cavity Flow Dataset the 2D Navier Stokes Lid-driven Cavity Flow simulation, the dataset is structured follows:
Variable
Dimensions
Description
X (Spatial Coordinates)
(10,201, 2)
Contains 10,201 spatial nodes with (xr, y) coordinates spanning the cavity domain.
U (Velocity Field)
(10,201, 2)
Horizontal (u) and vertical (e) velocity components
at Re = 100, with no-slip boundary conditions and moving lid (y = 1) driving the flow.
P (Pressure Field)
(10,201, 1)
Pressure values normalized with respect to the reference boundary condition.
Table 14: Dataset for 2D Navier-Stoker Lid-driven Cavity Flow at Re = 100
The training set comprises 3,000 randomly sampled spatial points with associated velocity and pressure values, while
the test set evaluates the model on the full dataset of 10,201 nodes. Boundary conditions are explicitly enforced for the moving lid (u 4x(1 z), v and stationary walls (u 0), with the pressure field satisfying the incompressibility constraint.
Epochs: We trained the models until convergence but did not exceed 50,000 epochs.
Reproducibility: To ensure reproducibility of the experimental results, all random number generators seeded with fixed value (seed 0) across NumPy, Python's random module, and PyTorch (both CPU and GPU).
Hyperparameter Selection: The weights used in the external RBA atlention dynamically updated during training using smoothing factor = 0.001 aud 3., = 0.001. Different models employed in our experiments have varying hyperparameter configurations tailored their specific architectures. Table 15 summarizes the hyperparameters and the total number of parameters for each model.
39

---

Published in Transactions on Machine Learning Research (01/2026)
Table 15: Summary of Hyperparameters in PINN Failure Modes Experiment for Various Models
Model
Hyperparamsters
Model Parameters
AC-PKAN
Linear Embedding: 64
460, 101
Hidden ChebyKAN Layers: ChelyyKKANLaer (degroe=8)
Hidden LN Layers: 3 x LayerNorm (128)
Output Layer:
Activations: WavnArt.
Qes
Input Layer: Qites block (2 - 256, Sigmoid)
396,545
Hidden Layers: QRts block
256, Sigmoid)
Output Layer: 256
FastKAN
Layer PastKANLayer (RBF, SplineLinear 8500, Base Linear 8500)
246,518
Layer 2: A (RBF, SplineLinear 68,000 + 1, Base Linear 8500 - 1)
KAN
Layers: 2 KANLinear
neurons, SiLU activation)
270,000
PINNs
Sequential Layers:
527,361 197,361
2 - 512 (Linear, Tanh] 512 512 (Linear, Tanh)
512 - 512 (Linear, Tanh)
512 1 (Linear)
Fourier AN
NaiveFouriorKANT..oo
32, Degreem8
395,073
NaiveloorierKANNI..oo
32 + 128, Degree=8
NsiveFourierKKNNawee 128 128, Degree=8
KaiveFourierKKNNLayer 128 + 32, Degree=8
NsivePourierKKANLayer 32
Degree=
ChebylKAN
Cheby1KANLayer 2 32, Degree=8
406,368
32 + 128, Degree=8
Cheby1KANLayer 128 256, Degree=8
Cheby ThebylKANLayer 256 + Degree=8
ChebylKKNI.n 5: 32 Degren=8
Cheby2KAN
Cheby2KAN 1: 2 - 32, Degree=8
406,368
Cheby2K
128, Degreem8
Cheby2K NLyyerr 3: 128 + 256, Degree=8
Cleby2KAN
Degree=8
Cheby2KAN 5: 32 + Degree=8
IKAN Sequential Layers:
460,813
256 (Linear, (INB(3))
256 + 512 (Linear, (JN)()))
512 512 (Lincar, (JNB(3))
512 + 128 (Linear, (JNB(6))
128 1 (Linear)
:KAN
Sequential Layers:
460,835
2 - 256 (Linear, JacobiKKA((3)
256 512 (Linear, PadeRKAN (2/6)
(Linear, JacobiRK (N
512 + 128 (Lincar, PadelKAAN2/66]
(Linear)
Sequential Layers:
527,361
2 + 512 (Linear, SinAct)
(Linear, Tanh)
512 + 512 (Linear, Tanh)
512 (Linear)
PINNaformer Parameters: d_out=1, d 512den=512, d_model=32,, N=1, heads=2
453,561
This reaches the GPU memory limit, and increasing the number of parameters further would cause an out-of-mmmooy error.
E.4 PDEs in Complex Engineering Environments Setup Details
In this study, investigate the performance of AC-PKAN compared with other models in solving complex PDEs characterized by heterogeneous material properties and intricate geometric domains. Specifically, we focus on two distinct difficult environmental PDE problems: heterogeneous Poisson problem and a Poisson equation defined on domain with complex geometric conditions. The following sections detail the formulation of the PDEs, data generation processes, model architecture, training regimen, hyperparameter selection, and evaluation methodologies employed in our experiments.
We consider a two-dimensional Poisson equation with spatially varying coefficients model heterogeneous material properties. The PDE is defined
Heterogeneous Poisson Problem.
40

---

Published in Transactions on Machine Learning Research (01/2026)
di Au(u)) (x) = 16r
for < ro:
azAu(z) = 16r
for 2 T0: 02
(157)
u(x) _ rd
where r | the distance from the origin, a1 = 15 and 02 are the material coefficients, to 0.5 defines the interface between the two materials, and 0R represents the boundary of the square domain f2 = [-1, 1] The boundary condition a pure Dirichlet condition applied uniformly on four edges of the square.
Heterogeneous Poisson Dataset To train and evaluate the Physics-Informet Neural Networks (PINNs), collocation points were generated within the defined spatial domains, and boundary conditions were appropriately enforced. A uniform grid was established using 100 equidistant points in each spatial dimension, resulting in 101 x 101 = 10,201 internal collocation points for heterogeneous Poisson problem. Boundary points were extracted from the edges of the square domain f = []1, to impose Dirichlet boundary conditions. The PINN was trained in data-free free, unsupervised manner. Upon completion of training, the model was evaluated on the collocation points by comparing the predicted values with the actual values, thereby determining the error.
Complex Geometric Poisson Problem. Additionally, we examine a Poisson equation defined on domain with complex geometry, specifically rectangle with four circular exclusions. The PDE is given by:
Au in Qxee
UR.. Rd.
(158)
where Ore = 0.55, is the rectangular domain and R3 for = 1,2,3, 4 are circular regions defined as:
R1 {(. (2- -v- - s0.1)
Rg (x (v-
Rg = ((x,y)
R1 = {x, (x - - s0.1".
The boundary conditions specified as:
u on OR, Vi 1,2,3,4, Mrra.
(159)
(160)
Complex Geometric Poisson Dataset To train and evaluate the Physics-Informed Neural Networks (PINNs), collocation points were generated within the defined spatial domains, and boundary conditions were appropriately enforced. uniform grid was established using 100 equidistant points in each spatial dimension, resulting in 101 x 101 10,201 internal collocation points for the Complex Geometric Poisson problem. Boundary points are sampled from both the outer boundary 02,.c and the boundaries of the excluded circular regions 0R, for = 1, 2, The PINN was trained in data-free, unsupervised manner. Upon completion of training, the model was evaluated the collocation points by comparing the predicted values with the actual values, thereby determining the error.
3D Point-Cloud Poisson Problem We investigate a three-dimensional Poisson equation defined on a unit cubic domain, [0, 1J3. where the data distribution represented as a point cloud, capturing the complex geometry introduced by excluding four spherical regions. The governing equation is non-horngencous, layered Helmholtz-type partial differential equation given by
=+( Au(() + k(=)Pu(x) sxx) in Ω = (0, Uc. 1=1
(161)
where the spherical exclusion regions C, for i 1,2,3, 4 are defined
c= = ((x,,,)) (a- 0.4)2 (v 003) + (=-0.6) 0.6) 0.22).
(162)
41

---

Published in Transactions on Machine Learning Research (01/2026)
C2 ((,, (2 0.6) (v (= 02-0,)) x {022). (1 C3 = ((1,),2) : (x- 0.2) + (y - 0.8)* + (2 - 0.7)* & 0.1")
(163)
(164)
C1 (1,, 0.6)* (y 0.2)* + (z 0.3)* 0.1
(165)
The material properties exhibit a layered structure at = 0.5, with
md=) = M1 < 0.5,
2 0.5,
k(z) = 1/2 = 10, 2 =2 0.5.
(166)
P2 = 1, 2 0.5,
The source term f(x) incorporates strong nonlinearities, defined
x²
n² 2²
[xx A (i)
x2 + y² 22 + A2 (sin(m11T + sin(mary) + sin(msmz2)],
(167)
where the parameters are set to A 20, A = 100, conditions are imposed on the boundary of the cubic domain, ensuring that
1, m2 = 10, and ms = 5. Homogeneous Neumann boundary
tu
0 on 02,
(168)
On
where 0Ω consists of the six faces of the unit cube.
3D Point-Cloud Poisson Dataset The 3D Point-Cloud Poisson Problem dataset is derived from extensive collection of 65,202 points, each defined by three spatial coordinates (z, y, 2) and an associated scalar solution value u, collectively representing the solution computational feasibility, randomized subset of 10,000 points is selected from the original dalaset for model training and evaluation. This reduced dataset maintains the structural integrity the original data, with spatial coordinates organized in (10,000 x 3) matrix and the solution field in (10,000 x 1) vector. From this subset, a further random selection of 1,000 points constitutes the supervised training set, which includes exact solution values essential for calculating data loss, while the remaining 9,000 points are utilized to enforce physics-informed loss during the training process. This approach ensures computational efficiency while preserving a representative sample the diree-dimensional domain. Subsequently, testing and validation conducted on the entire reduced dataset to assess the model's predictive accuracy across the domain.
Poisson equation within three-dimensione domain. To achieve
Tensor Conversion All collocation and boundary points converted into PyTorch tensors with floating-poit precision and are set to require gradients to facilitate automatic differentiation. The data resides on an NVIDIA A100 GPU with 40GB of memory to expedite computational processes.
Training Regimen: All PDE problems are trained for a total of 50,000 epochs allow sufficient learning iterations. And the RBA atlention mechanism for AC-PKAN is configured with smoothing factors vy = 0.001 and She = 0.001.
Reproducibility: To ensure the reproducibility of our experimental results, all random number generators are seeded with a fixed value (seed 0) across NumPy, Python's random module, and Py'Torch (both CPU and GPU). This deterministic setup guarantees consistent initialization and training trajectories across multiple runs.
Hyperparameter Selection: For the 3D Point-Cloud Poisson Problem, Table 15 provides detailed summary of the hyperparameters and the total number of parameters for each model. Similarly, for the other two problems, Table I6 summarizes the hyperparameters and the total number of parameters for each model.
42

---

Published in Transactions on Machine Learning Research (01/2026)
Table 16: Summary Hyperparameters Complex Engineering Environmental PDEs for Various Models
Model
Hyperparameters
Model Parameters
Linear Embedding: in=2, out==2
ChebyKAN Layers: 4 layers, degree=8
AC-PKAN
LN Layera: layers, featuree=61
152,357
Output Layer: in=64, out=l
Activation: WaveAct
Input Layer: in=2, out= 128
Hidden Layers: 5 QRes blocks, unite=128
Res
166,017
Output Layer: im=128, out=1
Activation: Sigmoid
Layer 2 + 256, Activation=/aul
Layer 256 512, Activation=Taal
PINN
Layer 3: 512 128, Activation Tanh
198,145
Layer
out=1
d hidden=128
PINNsformer
model=8
158,721
N=1
heads=2
Layer 1: 2 - 256, Sitivation=SinAct
Layer 256 256, Activation=Taul
FL.
132,609
Layer 256 256, Activation=Taanl
Layer 4: 256 +
Layer 1: 2 32, Degroe=8
Layer 32 + 128, Degree=8
Cheby1KAN
Layer 128 64, Degree=8
129,888
Layer 4: 64 - 32, Degree=8
Layer 32
Degree-8
Layer
Dagree=8
Layer 2: 32 - 128, Degroem8
Cheby2KAN
Layer 3: 128 + 64, Degree=8
129,888
Layer 64 32, Degree=8
Layer 6: 32 + 1, Degree=8
Layers: 2 KANLinear
KAN"
Neurons: 9000 Activation: SILU
60,000
Layer 2 256, Artvaatio==JccooiRAAAN..
Layer 2: 256 + 256, Activation==PadeRKANPP
rKAN
Layer
256, Activatiou=/accoiikkANN6
165,411
Layer 4: 256 128, Activatloo==PddeRKKAA3..00
Layer 5: 125
FastKAN' FastKANLayer 1: RBF, SplineLinear 16 - 2600, Base Linear 2 - 2600
75,118
FastKANLaye: RBF, SplineLinear 20800 Base Linear 2600
Layer - 256, Activation=(JNBB33
Layer 2: 256
512, Aetivatio==(INB6)
IKAN
Layer 3: 512
512, (ctivation==]NB(
132,618
Layer 512
128, Activation=
Layer 5: 128
Layer 2 32
Layer 2: 32 -
FourierKAN
Layer 64
166,
Layer 4: 64 -
Layer 64
Degree=8
This reaches the GPU memory limit, and increasing the number of parameters further would out-of-mmmmory memory error.
out-of
F Results Details and Visualizations.
Firstly, in the context of the 1D-Wave experiment, we present the logarithm of the GRA weights, log GAA across GRA epochs in Figure 6 Additionally, the progression of ACC over epochs is illustrated in Figure (see below).
In Figure
that the RBA weights for all loss terms eventually converge, indicating mitigation of residual imbalance. In contrast, the GRA weights contimee to increase, suggesting persistent gradient imbalance. The steadily growing GRA weights effectively alleviate the gradient stiffness problem, consistent with findings in (Wang et al.
43

---

Published in Transactions on Machine Learning Research (01/2026)
Pasn Vile andd CO0C
Hear oi. Vigit GU0t
Mepn W8700 Spats
Van nnw
Figure 4: Mean values of GRA and RBA weights over epochs for the ID-Wave experiment. From left to right in the first row: GRA ABC, GRA âic and RBA weights (BC). Second row: RBA weights (IC) and RBA weights (Residual).
Figure 5: Loss landscapes of various models in the 1D-Wave experiment. From left to right in the first row: AC-PKAN, KAN and fKAN. Second row: QRes and Pinnsformer.
44

---

Published in Transactions on Machine Learning Research (01/2026)
Log Epoche
Value Lertabtze 92
P4334 of
10000
30000
3000
4003
30000
20038
20000
30029
4000C
55000
Epec1
Spach
Figure 6: Mean values of GRA weights after logarithmic transformation over epochs for the 1D- 1D-Wave experiment.
2021
The significant magnitude discrepancy between GRA and RBA data justifies using logarithmic function for GRA weights in loss weighting (Figure
Moreover, Figure 5) illustrates the loss landscapes of AC-PKAN, Cheby1KAN, IKAN, QRes, and PINNeFormer. Although Cheby KKAN appears have simpler loss landscape, its steep gradients hinder optimization. PINNsFormer, fKAN, and QRes exhibit more complex, multi-modal surfaces, leading to convergence challenges near the optimal point. In contrast, AC-PKAN shows relatively smoother trajectory, facilitating training stability and efficiency.
45

---

Published in Transactions on Machine Learning Research (01/2026)
Then illustrate the fitling results of nine models for complex functions in Figure] 7 Additionally, we present the plots of ground truth solutions, neural network predictions, and absolute errors for all evaluations conducted in the five PDE-solving experiments. The results for the ID-Reaction, JD-Wave, 2D Navier-Stokes, Heterogeneous Poisson Problem, and Complex Geometric Poisson Problem are displayed in Figures TO and 13
respectively.
Onginnal Data ChebylKAN
Original al Data vs Cheby2KAN
Original Data vs PINN
1.5
1.5
1.5
10
1.0
1.0
03
0.5
00
0.
0.0
-00
-0.3
-0.5
-1.0
-1.0
+2.
0.00
0.5
0.50
075
1.00
125
150
375
2.00
0.00
6.5
0.50
0.75
1.20
155
1.50
1.5
2.00
0.00
025
0.50
0.5
1.00
1.2
1.50
3.55
2.00
Original Data ve AC-PKAN_S
Original Data vs KAN
Original Data IKAN
1.5
1.-
1.0
Ortanna
1,00
1.50
0.25
2.00
0.00
0.50
0.55
1.25
Original
Original
Original Data FourlerKAN
Orinnal Data fuo
Orgnal Dacs
Ongial Dat2 Ywu
MASTOA
130
0/b
200
0/5
L.D
Figure 7: Illustration of 9 Various Models for Complex Function Fitting
46

---

Published in Transactions on Machine Learning Research (01/2026)
Exact u(x.t)
0.2
0.8
0.4
0.6
0.6
0.4
0.8
0.2
x
Ground Truth Solution for the ID-Reaction Equation
Prodctedd sbul
30
13
0.
-022
04
30
10
02-
1.
Phad uteAt
Predictes stxll
Proditted ule.)
02-
00
604
00
Asssiute Errr
Absolate
Absolute Erro
00
22
12
100
Prncitted 4tx/1
1o
Predicted stxil
Predicted ubLD
Predicted sbut)
0A
2382
Abssiuea Enorr
Absolute Enor
Aboclute birt
absokits Eri
110
05-
23
18
(b) From left to right, the first, third, and fifth rows display the predictions of the AC-PKAN, Cheby1KAN, Cheby2KAN, and FastKAN models; PINNs, QRes, rKAN, and IKAN models; and the PINNsformer, FLS, FourierKAN, and KINN models, respectively. The second, fourth, and sixth rows present their corresponding absolute errors.
Figure& 8: Comparison of the ground truth solution for the 1D-Reaction equation with predictions and error maps from various models.
47

---

Published in Transactions on Machine Learning Research (01/2026)
Exart lix)))
00
10
9.2
043
40
0.0
0.5
10
-10
(a) Ground Truth Solution for the WD-Wave Equation
4000
Prodicted
ulkc
1.0
Mub
0.5
638
C--
-05
COC
0.5
0.50
075
1.00
0.0
025
0.0
6.15
1.00
18
650
0.30
0.5
050
075
1.0
Absolute
Absolute Erro
Abcolute
Abnolcte Eiror
0000
0010
0.5
02
1.0
0.5
5.75
020
350
E15
133
325
E50
Predicted (li()
Predicted ubctl
Predicted vlect
3.5
6.5
Predfoed ulu
125030
65
1903
2530
08
1.0
0.50
075
0.0
222
530
0.5
-5530
1.
050
0.58
132
0.0
03
0.0
120003
1.0
06
55000
41
30000
10
Predicted u0c))
Predicted u0471
Predimte (6
Preditted kil.!.
0.4
100
1.3
12
10
trror
Assotutl
3.
1.%
64
34
Q4
33
000
28
ER
025
61
030
3150
(b) From left to right, the first, third, and fifth rows display the predictions of the AC-PKAN, Cheby1KAN, Cheby2KAN, and FastKAN models; the PINNs, QRes, rKAN, and IKAN models; and the PINNsformer, FLS, FourierKAN, and KINN models, respectively. The second, fourth, and sixth present their corresponding absolute errors.
Figure Comparison of the ground truth solution for the 1D-Wave equation with predictions and error maps from various models.
48

---

Published in Transactions on Machine Learning Research (01/2026)
Exact p(c.))
00
00.1
0.2
-03
-0.4
-0.5
-2.5
0.0
2.5
5.0
Ground Truth Solution for the 2D Navier Stokes Cylinder Flow
Proicted gloz
Predicad piael
Predicted
Predicted plck)
64
-02
-25
5.0
4.3
-23
Abbolieee Timr
Acsolutt
5130
0125
E1
235
Predicted poc.t)
Predicted atect
Predicted
81
2.5
75
Abcoluite Error
Error
A-bbciute Error
154
025
0379
-25
pouts
Predidted
Powdicted
Predistes póck)
40
24
Abcolute Errrr
P Absolatee
Absolute Enor
P'Abokite tror
230
58
354
225
30
25
15
00
25
75
25
(b) From left to right, the first, third, and fifth rows display the predictions of the AC-PKAN, Cheby1KAN, Che,y2KAN, and FastKAN models; the PINNs, QRes, and [KAN models; and the PINNsformer, FLS, FourierKAN, and KINN models, respectively. The second, fourth, and sixth rows present their corresponding absolute errors.
Figure 10: Comparison of the ground truth pressure field P of the 2D Navier Vavier-Stokes cylinder flow with predictions and corresponding error maps generated by various models.
49

---

Published in Transactions on Machine Learning Research (01/2026)
(a) Ground Truth Solution for the ID-Conv--Diff-Rees -eacc. Equation
Inndined
Peddond
Pedicacukk
44
66
66
65
Indemacuuo
(b) From left to right, the first, third, and fifth rows display the predictions of the AC-PKAN, Cheby1KAN, Cheby2KAN, and FastKAN models; the PINNs, QRes, rKAN, and fKAN models; and the PINNsformer, FLS, FourierKAN, and KINN models, respectively. The second, fourth, and sixth rows present their corresponding absolute errors.
Figure 11: Comparison the ground truth solution for the DD-Cony-Diff.-Roo Equation with predictions and error maps from various models.
50

---

Published in Transactions on Machine Learning Research (01/2026)
Dast
(a)
Ground Truth Solution for the 2D Lid-driven cavity flow
(b) From left to right, the first, third, and fifth display the predictions of the AC-PKAN, Cheby1KAN and Cheby2KAN; the PINNs, QRes and fKAN models; and the FLS models, respectively. The second, fourth, and sixth rows present their corresponding absolute errors.
Comparison of the ground truth solution for the 2D Lid-driven cavity flow with predictions and error maps from various models.
Figure 12:
51

---

Published in Transactions on Machine Learning Research (01/2026)
(a) Ground Truth Solution for the Heterogeneous Possion equation
(b) From left to right, the first, third, and fifth rows display the predictions of the AC-PKAN, ChebylKAN Cheby2KAN, and FastKAN models; the PINNs, QRes, rKAN, and fKAN models; and the PINNsformer, FLS, FourierKAN, and KINN models, respectively. The second, fourth, and sixth rows present their corresponding absolute errors.
Figure 13: Comparison of the ground truth solution for the Heterogeneous Possion equation problem with predictions and error maps from various models.
52

---

Published in Transactions on Machine Learning Research (01/2026)
(a) Ground Truth Solution for the Complex Geometry Possion equation
PEBOBCL
44
(b) From left to right, the first, third, and fifth rows display the predictions of the AC-PKAN, Cheby1KAN, Cheby2KAN, and FastKAN models; the PINNs, QRes, rKAN, and fKAN models; and the PINNsformer, FLS, FourierKAN, and KINN models, respectively. The second, fourth, and sixth present their corresponding absolute errors.
Figure 14: Comparison of the ground truth solution for the Complex Geometry Possion equation problem with predictions and error maps from various models.
53

---

Published in Transactions on Machine Learning Research (01/2026)
DISTO
(a) Ground Truth Solution for the 3D Point-Cloud Problem
(b) From left to right, the first, third, and fifth rows display the predictions of the AC-PKAN, ChebylKAN and Cheby2KAN models; the PINNs, QRes, rKAN, aud IKAN models; and the FLS and FourierKAN models, respectively. The second, fourth, and sixth rows present their corresponding absolute errors.
Figure 15: Comparison of the ground truth solution for the 3D Point-Cloud Problem with predictions and error maps from various models.
54


Reviews:
[{"review_id": "jKDrcG8oQi", "reviewer": "Reviewer_pTHb", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes AC-PKAN, a novel hybrid neural architecture that combines Chebyshev Type-I based Kolmogorov–Arnold Networks (Cheby1KANs) with attention mechanisms for use in Physics-Informed Neural Networks (PINNs). This work introduces multiple enhancements aimed at addressing major limitations of existing PINNs and KANs in particular, such as residual and border condition losses unbalance, and their risk of rank collapse. To address these limitations, the paper proposes to use a dedicated architecture that interleaves Cheby1KANs layers and linear projection layers, combined with wavelet activations and feature-wise attention to preserve the full rank of the Jacobian Matrix. A Residual Gradient Attention (RGA) is also introduced as a way to automatically weight losses during training.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The claims are supported with both clear theoretical and empirical justifications. Overall, the paper is clear and easy to follow. In particular, the preliminary sections (3.1-3.2) provide a clear understanding of the problem and the limitations of KAN networks. \n\n\nHowever, despite being very well written, there remain a few points that might be dug further:\n\nMost of the rank collapse robustness discussion is based on infinite-width KAN, which might not translate when dealing with finite-width KAN. The improved stability of the approach remains thus empirical.\n\nIn comparison to standard PINNs (MLP‐based) or simpler KAN variants, AC-PKAN is far more complex. It is rather difficult to understand if the improvements are provided by the parameter overhead, a subset of new modules, or fine-grained parameter tuning. The benchmarks chosen are standard but do not fully illustrate the model capacity on complex systems.", "audience_yn": "Yes", "audience_explanation": "This paper is well in phase with current development around machine learning for physical models. Its propositions might bring valuable insights into functional architectures and their use in physics-informed learning.\n\nIn particular, the main contributions: Chebyshev-KANs with internal/external attention as well as the GRA appear original.", "requested_changes": "## Model complexity\n\nMany new modules are incorporated into the presented work to improve Chebychev KANs stability and expressivity. Their contribution is evaluated in a dedicated with/without each module ablation *cf* Tab. 4. The study shows drops each time a component of the method is removed. However, this ablation should be pushed further in order to disentangle module interactions. In its current state, it is rather difficult to understand from the ablations if the different modules are truly contributing as much as they seem to be.\n\nIn comparison to standard PINNs (MLP‐based) or simpler KAN variants, AC-PKAN has more architectural hyper-parameters: Number of ChebyKAN layers, Degree of Chebyshev polynomials, Wavelet activation parameters, Internal attention feature-wise modules (dimensions, heads), External RGA dynamic loss-weighting hyper-parameters (e.g., learning rate for RBA, moving average parameters for GRA). \n\nKAN (especially Chebychev KAN) scales quadratically with the dimension, as high polynomial degrees are needed to represent sharp, local variations, and each feature dimension is related to a given polynomial expansion. Furthermore, each introduced projections add a number of parameters equal to the number of input features times the number of output ones, which might also add a dramatic parameter overhead for high-dimensional ODE.\n\nAdding more ablations exploring the different module combinations, as well as a dedicated model stability towards introduced hyperparameters, would give great insight into the method's capacity. Providing a discussion about parameter overhead depending on the feature state-space dimension is also critical.\n\n## Simplicity of the evaluation\n\n\nFinally, the model was evaluated on nine benchmark problems, including classical low-dimensional PDEs (1D wave equation, 1D Burgers’ equation, 2D Poisson equation,...), function fitting and ODE examples (e.g. Lotka–Volterra-like systems), and a few regression tasks. This evaluation seems to be relatively common across state-of-the-art and shows promising results. However, the dimension of the systems used for evaluation is relatively small (less than 4 input variables). This tends to weaken the methodological contributions aiming at augmenting the model expressivity and limiting rank collapse, and it is difficult to assess if they can actually scale to more complex scenarios. I wonder if the method could be used on high-dimensional state-space ODE *e.g.* Lorenz-96. Discussing this point, even as a limitation or future work, might improve the paper.", "broader_impact_concerns": "N/A", "additional_comments": "* The related work section lacks positioning and is restricted to pointing out existing related methods in the literature. Having a clear highlighting of current challenges and how the proposed methods answer them would greatly improve the paper.\n\n\n* It is not clear how the learnable linear weighting described as an attention mechanism in Section 3.3 corresponds to the concept of attention commonly used in the broader machine learning literature. \n\n* The width of a KAN could be defined as this notion might be ambiguous between node width and functional width.\n\n* The method lacks theoretical or, at least, intuitive justifications. In particular, the rationale behind the architectural adaptation sec. 3.3 could be made explicit.", "extra_scores": {}, "extra_text": {}}, {"review_id": "Qb5ovxXMk6", "reviewer": "Reviewer_LMMU", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes an extended KAN-based approach for solving PDEs. The approach draws on the idea of swapping B-splines in KANs with Chebyshev polynomials. First, theoretical analysis is provided to demonstrate that stacking plain ChebyshevKANs tends to collapse rank. To tackle this limitation, the proposed approach adds an internal attention block around ChebyshevKAN and wavelet layers to preserve a full-rank Jacobian, and additionally incorporates an external attention mechanism (RGA) that adapts loss weights using residual magnitudes and gradient norms. This combined approach,  dubbed AC-PKAN, is shown to mostly outperform prior PINN/KAN variants on nine benchmarks. A proper review of prior work is provided. Extensive ablation studies, including the transferability of the proposed new components, are provided.  Theoretical proofs seem correct to me (though haven't checked thoroughly). The paper is well written and structured. Overall, I find this work interesting, however, there are several concerning aspects which warrant clarification and further consideration.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Most of the claims seem correct with, yet there are several inconsistencies and potential overstatements as detailed in the other parts of my review.", "audience_yn": "Yes", "audience_explanation": "This work would be of interest to the community focusing on PINNs and ML-based approaches to PDEs.", "requested_changes": "Below are my concerns, questions and suggestions:\n\n\n1. The abstract mentions that AC-PKAN consistently outperforms or matches SOTA methods. However, the results in Tables 1 and 2, do not support this claim. The language should be adjusted.\n\n2. On page 9 it is stated that the internal architecture of AC-PKAN is powerful even without RGA, yet in Table 3, AC-PKAN without RGA is far worse than multiple models with RGA. This should be rephrased to avoid overclaiming the standalone strength.\n\n3. The manuscript would benefit from a dedicated \"Discussion and Limitations\" section before the conclusion. While the appendices include a brief discussion (computational complexity) and a limitations note (optimizer/pruning), the scalability of AC-PKAN is not discussed in the main text. I would recommend adding a discussion on the scalability of AC-PKAN and its components (e.g., Chebyshev degree N, layer widths/depth etc. ) as well as outlining directions for future research.\n\n4. Define rMAE/rRMSE precisely.\n\n5. The statement in Theorem 3 seems to hold under i.i.d. Gaussian coefficients. Trained networks do not necessarily satisfy this assumption. There should be an accompanying note explicitly stating this.\n\n6. Further to the above comment, since rank is central, add supporting empirical results illustrating the decay across depth for Cheby1KAN vs AC-PKAN and showing preserved feature diversity.\n\n7. A large fraction of the references in this paper are arXiv preprints or GitHub repositories rather than peer-reviewed papers. While it is commendable that the authors consider recent benchmarks, this heavy reliance on unpublished sources makes the empirical landscape somewhat fluid. This does not invalidate the contribution of this work, however, it weakens the weight of the comparative claims.", "broader_impact_concerns": "N/A", "additional_comments": "Minor comments:\n\n1. On page 4, \"Equation equation 8\" -> \"Equation 8\"\n2. The text contains both \"Chebyshev1kan\" and \"Cheby1KAN\". Standardize.", "extra_scores": {}, "extra_text": {}}, {"review_id": "kX18PHACdL", "reviewer": "Reviewer_Dyvd", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces AC-PKAN, which interleaves Cheby1KAN layers with linear projections, wavelet-based activations, and an internal feature-wise attention mechanism. Together, these preserve full-rank Jacobians and prevent vanishing derivatives, improving expressiveness. Authors evaluates across 9 PDE tasks in three categories (failure modes, heterogeneous/complex geometries, 3D problems) and compares against 12 competing models. Results show AC-PKAN consistently achieves state-of-the-art or near-SOTA performance. \n\nStrengths: \n1. This paper provides formal rank-collapse proofs for Cheby1KANs and rigorous guarantees for AC-PKAN’s rank preservation.\n2. The proposed AC-PKAN smartly combines Chebyshev bases (efficiency) with wavelet activations (periodicity capture) and attention (feature selection).\n\nKey Weaknesses / Limitations:\n1. The paper omits comparisons to operator learners (FNO, DeepONet), assuming they are “orthogonal.” This limits the broader positioning of AC-PKAN.\n2. Proofs rely on infinite-width assumptions, while practical networks may still encounter degraded rank or gradient stability.\n3. Scalability of AC-PKAN to very high-dimensional PDEs isn’t fully tested.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The claims are mostly well-supported by both theoretical and empirical evidence.", "audience_yn": "Yes", "audience_explanation": "This paper will interest audience who're working on using AI methods to solve PDE equations.", "requested_changes": "It is recommended to provide more discussion on the infinite-width assumptions, as they may not strictly hold in practical scenarios. Additionally, including operator-learning baselines, such as FNO and DeepONet, in the comparison would strengthen the claim of achieving \"state-of-the-art\" performance.", "broader_impact_concerns": "n/a", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)