On the Rate of Convergence of Kolmogorov-Arnold Network Regression Estimators

## Abstract

Abstract Kolmogorov–Arnold Networks (KANs) offer a structured and interpretable framework for multivariate function approximation by composing univariate transformations through additive or multiplicative aggregation. This paper establishes theoretical convergence guarantees for KANs when the univariate components are represented by B-splines. We prove that both additive and hybrid additive-multiplicative KANs attain the minimax-optimal convergence rate O ​ ( n − 2 ​ r / ( 2 ​ r + 1 ) ) O(n^{-2r/(2r+1)}) for functions in Sobolev spaces of smoothness r r . We further derive guidelines for selecting the optimal number of knots in the B-splines. The theory is supported by simulation studies that confirm the predicted convergence rates. These results provide a theoretical foundation for using KANs in nonparametric regression and highlight their potential as a structured alternative to existing methods.

## 1 Introduction

The convergence rates of nonparametric regression using neural networks have been extensively studied, building upon a long line of statistical learning theory (Barron, 2002; 1994; Imaizumi & Fukumizu, 2019; Bauer & Kohler, 2019; Schmidt-Hieber, 2020). Early foundational results, such as those of Stone (1982), established the minimax-optimal convergence rate for general nonparametric regression, which remains a benchmark for evaluating modern learning architectures. Subsequent advances in approximation theory demonstrated that fully connected deep neural networks, when endowed with sufficient width or depth, are capable of achieving rates close to these minimax limits. For instance, Kohler & Langer (2021) proved that deep neural networks can attain a rate of order $O((\log n)^{6}n^{-\frac{2r}{2r+K}})$, thereby confirming their statistical efficiency under suitable smoothness conditions. While these results establish that neural networks are competitive with minimax-optimal procedures, their highly entangled representations generally lack explicit structure, limiting interpretability and complicating theoretical analysis.

In contrast, spline-based approximation methods provide a structured and well-understood pathway to minimax-optimality. The convergence behavior of spline estimators, particularly those constructed with B-splines, has been rigorously analyzed in both global and local settings (Speckman, 1985; Nussbaum, 1985; He & Shi, 1994; Li et al., 1995). Under standard smoothness assumptions, spline estimators achieve the optimal rate $O(n^{-\frac{2r}{2r+1}})$, which matches the minimax benchmark $O(n^{-\frac{2r}{2r+d}})$ in one-dimensional settings. Beyond splines, sieve estimators form a general class of nonparametric methods that approximate infinite-dimensional function spaces by growing finite-dimensional bases. Chen (2007) provided a comprehensive analysis of sieve estimation—including splines, wavelets, and polynomial bases—showing that these procedures can achieve minimax-optimal convergence when the basis is chosen adaptively to the function class. Similarly, kernel regression methods, such as the Nadaraya–Watson estimator and local polynomial regression, have long been known to reach minimax-optimal rates under appropriate bandwidth selection (Györfi et al., 2002). Collectively, these results underscore the value of structured nonparametric estimators: not only do they achieve optimal convergence rates, but their theoretical analysis also directly links model structure to approximation properties.

Kolmogorov–Arnold Networks (KANs) (Liu et al., 2024b) arise at the intersection of these two lines of research: like neural networks, they employ layered compositions of functions, but they inherit the structured basis-function philosophy of spline and sieve methods. Their conceptual foundation lies in the Kolmogorov–Arnold representation theorem (Kolmogorov, 1957; Arnol’d, 1957), which asserts that any multivariate continuous function can be represented as a finite sum of continuous univariate functions applied to linear combinations of the input variables. This theorem provides a compositional, low-dimensional lens on high-dimensional learning problems, where complexity is managed through a hierarchy of simple univariate elements (Schmidt-Hieber, 2021). In practice, KANs instantiate this representation by parameterizing univariate components with B-spline expansions, thereby combining the expressive power of neural architectures with the interpretability and approximation guarantees of spline-based methods. The developments extend beyond purely additive structures to include multiplicative and hybrid architectures, improving expressiveness while retaining interpretability (Liu et al., 2024a). From a convergence perspective, this structure suggests that KANs may inherit the minimax-optimal rates known for spline estimators.

Despite their empirical success and conceptual appeal (Vaca-Rubio et al., 2024; Bodner et al., 2024; Kiamari et al., 2024; Koenig et al., 2024; Xu et al., 2024), the theoretical underpinnings of KANs remain underexplored. Universal approximation capabilities have been noted informally, but formal convergence analyses in statistical estimation settings are scarce. Recent work has begun to formalize this intuition. Gao & Tan (2025) studied the optimization and generalization behavior of KANs trained with stochastic gradient descent, providing non-asymptotic rate estimates that highlight how the structured nature of KANs influences their convergence. Unlike classical spline or kernel estimators, whose rates are derived from explicit approximation spaces, KANs introduce an additional layer of complexity through stochastic training dynamics. This makes their convergence properties more closely aligned with neural networks, where optimization and statistical considerations are tightly coupled. At the same time, the B-spline parameterization of univariate components offers a level of control and interpretability absent in generic neural networks. As a result, KANs occupy a distinctive position between classical sieve estimators and modern deep networks: they retain the statistical efficiency and minimax-optimal potential of spline-based methods, while leveraging the compositional flexibility of neural architectures.

We study both additive and hybrid additive–multiplicative KANs, modeling target functions as compositions of univariate spline transformations. We characterize the representable function classes and show that spline-based KAN estimators attain minimax-optimal convergence rates under standard Sobolev smoothness assumptions.

## 2 Nonparametric Regression and B-Splines

In nonparametric regression (Härdle, 1990; Tsybakov, 2008), the goal is to estimate an unknown function
$f:[0,1]^{d}\to\mathbb{R}$ from observations
$\{(X_{i},Y_{i})\}_{i=1}^{n}$ generated according to

$$ $Y_{i}=f(X_{i})+\varepsilon_{i},\quad i=1,\dots,n,$ (1) $$

where $X_{i}=(X_{i1},\dots,X_{id})\in[0,1]^{d}$ are the inputs and
$\varepsilon_{i}$ are i.i.d. noise terms with zero mean and finite variance
(e.g., $\varepsilon_{i}\sim\mathcal{N}(0,\sigma^{2})$). Here, the domain $[0,1]^{d}$ is a standard choice for theoretical analysis without loss of generality (Tsybakov, 2008), since any bounded input domain $\mathcal{X}\in\mathbb{R}^{d}$ can be mapped to $[0,1]^{d}$ via an affine transformation.

Unlike parametric regression, which assumes a predetermined functional form for $f$ (e.g., linear or polynomial), the nonparametric setting imposes minimal structural assumptions, allowing the estimator to adapt to potentially complex and highly nonlinear relationships in the data. The central challenge lies in balancing this flexibility with statistical efficiency, particularly in high dimensions.

A common approach to nonparametric estimation is to approximate $f$ using a finite set of basis functions. Among the many available bases, B-splines are especially popular due to their computational efficiency, local support, and strong approximation properties under smoothness assumptions. In the univariate case, B-splines provide piecewise polynomial representations that achieve minimax-optimal convergence rates for sufficiently smooth $f$, and their tensor-product extensions naturally generalize to multivariate settings.

### 2.1 B-splines in Nonparametric Regression

A univariate B-spline of order $p+1$ (corresponding to polynomial degree $p$) is defined as a linear combination of basis functions
$\{B_{i,p}\}_{i=0}^{K+p}$ associated with a non-decreasing knot sequence
$\{t_{0},t_{1},\dots,t_{K}\}$.
Each basis function $B_{i,p}$ is a piecewise polynomial with continuous differentiability up to order $p-1$ (Schumaker, 2007). These basis functions possess local support, being nonzero only over an interval determined by $p+1$ consecutive knots.
This ensures that modifying one coefficient influences the spline only on a limited subinterval, thereby improving both interpretability and numerical stability.

The B-spline basis functions are constructed iteratively using the Cox–de Boor recursion formula:

$$ $\displaystyle B_{i,0}(x)$ $\displaystyle=\begin{cases}1,&t_{i}\leq x<t_{i+1},\\ 0,&\text{otherwise},\end{cases}$ (2) $\displaystyle B_{i,p}(x)$ $\displaystyle=\frac{x-t_{i}}{t_{i+p}-t_{i}}B_{i,p-1}(x)+\frac{t_{i+p+1}-x}{t_{i+p+1}-t_{i+1}}B_{i+1,p-1}(x),\quad p\geq 1,$ (3) $$

where division by zero is interpreted as zero.

The basis functions form a partition of unity,

$$ $\sum_{i=0}^{K+p}B_{i,p}(x)=1,\quad\forall x\in[0,1].$ (4) $$

Hence, any spline function $f$ of order $p+1$ can be expressed as

$$ $f(x)=\sum_{i=0}^{K+p}c_{i}\,B_{i,p}(x),$ (5) $$

where the coefficients $\{c_{i}\}$ are estimated from data, typically via least squares or penalized regression.

B-splines achieve minimax-optimal convergence rates under standard Sobolev smoothness conditions and extend naturally to higher dimensions via tensor-product constructions (Unser et al., 2002).
In the context of Kolmogorov–Arnold Networks, these properties make B-splines a natural choice for modeling the learnable univariate transformations.

### 2.2 Minimax Optimal Convergence Rate

In nonparametric regression, the performance of an estimator $\hat{f}_{n}$ is often characterized by how quickly it converges to the true regression function $f$ as the sample size $n$ increases.
The *minimax optimal convergence rate* describes the best achievable rate, under worst-case conditions, across all possible estimators for a given function class.

Let $\mathcal{F}$ be a class of functions (e.g., a Sobolev space $\mathcal{H}^{r}([0,1]^{d})$), and let $\hat{f}_{n}$ be an estimator of $f\in\mathcal{F}$.
The minimax risk is defined as (Györfi et al., 2002)

$$ $R_{\mathrm{minimax}}=\inf_{\hat{f}_{n}}\ \sup_{f\in\mathcal{F}}\mathbb{E}\left[\|\hat{f}_{n}-f\|_{L^{2}([0,1]^{d})}^{2}\right],$ (6) $$

where $\|g\|_{L^{2}([0,1]^{d})}^{2}=\int_{[0,1]^{d}}|g(x)|^{2}\,dx$ denotes the squared $L^{2}$-norm over the domain $[0,1]^{d}$.

The minimax optimal rate $\mathcal{R}_{\mathrm{minimax}}(n)$ describes how quickly the risk decays as the sample size
$n$ increases.
For functions in the Sobolev space $\mathcal{H}^{r}([0,1]^{d})$ of smoothness $r>0$, it is well known (Stone, 1982) that

$$ $\inf_{\hat{f}_{n}}\ \sup_{f\in\mathcal{H}^{r}([0,1]^{d})}\mathbb{E}\left[\|\hat{f}_{n}-f\|_{L^{2}}^{2}\right]=O\!\left(n^{-\frac{2r}{2r+d}}\right).$ (7) $$

Here, $n$ denotes the sample size, $r$ the smoothness index of $f$, and $d$ the input dimension. This exponent reflects the trade-off between smoothness $r$ and dimensionality $d$: smoother functions are easier to estimate, while higher dimensions slow convergence.

This expression makes explicit the *curse of dimensionality*: the convergence rate deteriorates rapidly with increasing $d$, even when $f$ is very smooth.
In particular, when $d$ is large, achieving a small estimation error requires exponentially more samples, motivating the design of estimators that can exploit low-dimensional structures in high-dimensional data.

### 2.3 Minimax Optimality of B-splines in Nonparametric Regression

When the target function is assumed to belong to a Sobolev space $\mathcal{H}^{r}([0,1])$ of smoothness order $r>0$, the minimax optimal convergence rate in this one-dimensional setting is known to be
$O\!\left(n^{-\frac{2r}{2r+1}}\right),$
which is attained by a broad class of nonparametric estimators, including kernel methods, local polynomials, and spline-based approaches.

B-spline estimators are a particularly appealing choice for achieving this optimal rate, owing to their locality, numerical stability, and flexibility in representing smooth functions. Classical results Speckman (1985); He & Shi (1994); Li et al. (1995) have established that, when the number of spline basis functions is chosen appropriately (balancing approximation and estimation errors), an estimator

$$ $\hat{f}_{n}(x)=\sum_{i=0}^{K+p}\hat{c}_{i}B_{i,p}(x),$ (8) $$

where $\{B_{i,p}\}_{i=0}^{K+p}$ are B-spline basis functions of degree $p\geq r$, achieves the minimax rate

$$ $\sup_{f\in\mathcal{H}^{r}([0,1])}\mathbb{E}\big[\|\hat{f}_{n}-f\|_{2}^{2}\big]=O\!\left(n^{-\frac{2r}{2r+1}}\right).$ (9) $$

This result is minimax optimal in the sense that no estimator can uniformly outperform this rate over the entire Sobolev ball $\{f\in\mathcal{H}^{r}:\|f\|_{\mathcal{H}^{r}}\leq C\}$. The optimality holds regardless of the specific B-spline degree, provided it is sufficiently high to capture the prescribed smoothness, and hinges on balancing the spline knot density with the available sample size.

**Table 1: Comparison of convergence rates for different nonparametric estimators. Here $r$ denotes smoothness of the target function, $d$ the input dimension, and $n$ the sample size.**
| Method | Convergence Rate | References |
| --- | --- | --- |
| B-splines | $O\!\left(n^{-\tfrac{2r}{2r+1}}\right)$ | Speckman (1985) |
| Sieve estimators | $O\!\left(n^{-\tfrac{2r}{2r+d}}\right)$ | Chen (2007) |
| Kernel regression | $O\!\left(n^{-\tfrac{2r}{2r+d}}\right)$ | Györfi et al. (2002) |
| Deep neural networks (fully connected) | $O\!\left((\log n)^{6}n^{-\tfrac{2r}{2r+K}}\right)$ | Kohler & Langer (2021) |
| Kolmogorov–Arnold Networks (KANs) | Under investigation | Liu et al. (2024b) |

The significance of this property extends to structured neural architectures such as Kolmogorov–Arnold Networks (KANs), where B-spline parameterizations can serve as the univariate building blocks. In such cases, the known optimality of B-splines in one-dimensional regression informs the achievable convergence rates for the multivariate composite model. In the next section, we formalize this connection and prove that KAN estimators inherit the minimax optimal convergence rate $O\!\left(n^{-\frac{2r}{2r+1}}\right)$ when the target function lies in a Sobolev space with smoothness $r$. A summary of representative convergence results across classical and modern nonparametric estimators is provided in Table [1](https://arxiv.org/html/2509.19830v1#S2.T1).

## 3 Model Setup

Let $x=(x_{1},\dots,x_{d})\in[0,1]^{d}$, and consider the problem of approximating a multivariate function $f:[0,1]^{d}\to\mathbb{R}$. A Kolmogorov–Arnold Network (KAN) models $f$ as a finite sum of univariate nonlinearities composed with structured aggregations of input variables:

$$ $f(x)=\sum_{q=1}^{Q}g_{q}\left(T_{q}(x)\right),$ (10) $$

where each $g_{q}:\mathbb{R}\to\mathbb{R}$ is a univariate activation function, and each $T_{q}(x)$ is a structured transformation of the inputs. Specifically, we consider two forms of $T_{q}(x)$:

$$ $T_{q}(x)=\begin{cases}\sum_{j=1}^{d}\psi_{qj}(x_{j}),&\text{(additive node)}\\ \prod_{j=1}^{d}\psi_{qj}(x_{j}),&\text{(multiplicative node)}\end{cases}$ (11) $$

where the functions $\psi_{qj}:[0,1]\to\mathbb{R}$ are univariate spline transformations applied to individual input dimensions. This architecture captures both additive and multiplicative interactions among input features, enabling greater expressiveness while maintaining interpretability.

Each univariate function $\psi_{qj}$ and $g_{q}$ is estimated from data using B-splines. Let $\mathcal{H}^{r}([0,1])$ denote the Sobolev space of univariate functions with $r$ square-integrable derivatives. We assume that each $\psi_{qj}$ and $g_{q}$ belongs to $\mathcal{H}^{r}([0,1])$ for some fixed smoothness index $r\geq 1$.

We define the following function classes associated with KANs:

- •
Additive KAN class $\mathcal{F}_{\mathrm{add}}$: functions of the form
$f(x)=\sum_{q=1}^{Q}g_{q}\left(\sum_{j=1}^{d}\psi_{qj}(x_{j})\right),\quad\text{with }\psi_{qj},g_{q}\in\mathcal{H}^{r}([0,1]).$
(12)
- •
Hybrid KAN class $\mathcal{F}_{\mathrm{hyb}}$: functions where each node may use either an additive or multiplicative composition,
$f(x)=\sum_{q=1}^{Q}g_{q}\left(T_{q}(x)\right),\quad\text{with }T_{q}(x)\in\left\{\sum_{j=1}^{d}\psi_{qj}(x_{j}),\;\prod_{j=1}^{d}\psi_{qj}(x_{j})\right\}.$
(13)

Throughout this paper, we assume that all univariate functions $\psi_{qj}$ and $g_{q}$ are uniformly bounded in sup norm by a constant $M$, and have Lipschitz constants bounded by $L$. These regularity conditions are standard in nonparametric regression and ensure statistical well-posedness.

The KAN model is learned from data $\{(X_{i},Y_{i})\}_{i=1}^{n}$, where $Y_{i}=f(X_{i})+\varepsilon_{i}$, and $\varepsilon_{i}$ are noises with zero mean and finite variance. Estimation of each univariate function is performed via M-type spline regression with B-spline bases of order $m\geq r$, yielding a fully data-driven estimator $\hat{f}_{n}$ within the KAN function class.

This section sets up the framework for our main theoretical results, which establish optimal convergence rates and structural guarantees for KAN estimators under these modeling assumptions.

## 4 Main Results

We now present the theoretical convergence properties of Kolmogorov–Arnold Networks (KANs) under the regression setting introduced in Section [3](https://arxiv.org/html/2509.19830v1#S3). Our goal is to quantify the estimation error of the KAN estimator $\hat{f}_{n}$, which is constructed by fitting the univariate spline units using M-type B-spline estimators from data $\{(X_{i},Y_{i})\}_{i=1}^{n}$, where

$$ $Y_{i}=f(X_{i})+\varepsilon_{i},\quad\mathbb{E}[\varepsilon]=0,\quad\mathbb{E}[\varepsilon^{2}]=\sigma^{2}<\infty.$ (14) $$

We consider two architectures: additive KANs, where each node aggregates input features additively, and hybrid additive–multiplicative KANs, where node-level interactions are either additive or multiplicative. In both settings, the univariate component functions $\psi_{qj}$ and $g_{q}$ are assumed to belong to the Sobolev space $\mathcal{H}^{r}([0,1])$ with smoothness $r\geq 1$, and are uniformly bounded and Lipschitz.

The following results establish convergence rates for $\hat{f}_{n}$ in the $L^{2}$ risk. We show that additive KANs attain the classical minimax rate for univariate nonparametric regression, and that hybrid KANs retain this rate up to a constant overhead.

###### Theorem 1 (Convergence Rate of Additive KAN Estimator) .

Let $f:[0,1]^{d}\to\mathbb{R}$ be a continuous function.
By the Kolmogorov–Arnold representation theorem, there exist continuous univariate functions $g_{q}:[0,1]\to\mathbb{R}$ and $\psi_{qj}:[0,1]\to\mathbb{R}$, $q=1,\dots,Q$, $j=1,\dots,d$, such that $f$ can be expressed in the form

$$ $f(x)=\sum_{q=1}^{Q}g_{q}\!\left(\sum_{j=1}^{d}\psi_{qj}(x_{j})\right).$ (15) $$

Assume further that $g_{q}\in\mathcal{H}^{r}([0,1])$ are Lipschitz continuous and $\psi_{qj}\in\mathcal{H}^{r}([0,1])$ with smoothness index $r\geq 1$.

Suppose we observe $n$ i.i.d. samples $\{(X^{(i)},Y^{(i)})\}_{i=1}^{n}$ from the model

$$ $Y=f(X)+\varepsilon,\quad\mathbb{E}[\varepsilon]=0,\quad\mathbb{E}[\varepsilon^{2}]=\sigma^{2}<\infty.$ (16) $$

Let $\hat{f}_{n}$ denote the additive KAN estimator

$$ $\hat{f}_{n}(x)=\sum_{q=1}^{Q}\hat{g}_{q}\!\left(\sum_{j=1}^{d}\hat{\psi}_{qj}(x_{j})\right),$ (17) $$

where $\hat{g}_{q}$ and $\hat{\psi}_{qj}$ are B-spline estimators of order $m\geq r$ constructed via least squares with backfitting.

Then the mean squared error achieves the minimax-optimal rate:

$$ $\mathbb{E}\!\left[\|\hat{f}_{n}-f\|_{L^{2}([0,1]^{d})}^{2}\right]=O\!\left(n^{-\frac{2r}{2r+1}}\right).$ (18) $$

###### Remark 1 .

This result establishes that additive KANs achieve the minimax optimal convergence rate for functions in the additive Sobolev class with smoothness index $r$. The rate $n^{-\frac{2r}{2r+1}}$ corresponds to the optimal rate for estimating univariate functions in $\mathcal{H}^{r}([0,1])$. Notably, this convergence rate does not depend on the ambient dimensionality $d$, thereby avoiding the curse of dimensionality. This property arises from the compositional structure of KANs, which represent multivariate functions as sums of univariate functions constructed from B-splines.

We now extend the analysis to hybrid KANs, which permit more expressive architectures by allowing either additive or multiplicative compositions within each node.

###### Theorem 2 (Convergence Rate of Hybrid Additive–Multiplicative KAN Estimator) .

Let $f:[0,1]^{d}\to\mathbb{R}$ be a continuous function.
By the Kolmogorov–Arnold representation theorem, $f$ can be expressed in the form

$$ $f(x)=\sum_{q=1}^{Q}g_{q}\!\big(T_{q}(x)\big),$ (19) $$

where each $T_{q}(x)$ is either an *additive term* $\sum_{j=1}^{d}\psi_{qj}(x_{j})$ or a *multiplicative term* $\prod_{j=1}^{d}\psi_{qj}(x_{j})$, with univariate component functions $\psi_{qj}$ and outer functions $g_{q}$ continuous.

Assume further that $g_{q}\in\mathcal{H}^{r}([0,1])$ are Lipschitz continuous and $\psi_{qj}\in\mathcal{H}^{r}([0,1])$ with smoothness index $r\geq 1$, uniformly bounded by $M$ with Lipschitz constant at most $L$.

Suppose we observe $n$ i.i.d. samples $(X_{i},Y_{i})$ from the model

$$ $Y=f(X)+\varepsilon,\quad\mathbb{E}[\varepsilon]=0,\;\;\mathbb{E}[\varepsilon^{2}]=\sigma^{2}<\infty.$ (20) $$

Let $\hat{f}_{n}$ denote the hybrid KAN estimator

$$ $\hat{f}_{n}(x)=\sum_{q=1}^{Q}\hat{g}_{q}\!\big(T_{q}^{(n)}(x)\big),$ (21) $$

where $\hat{g}_{q}$ and $\hat{\psi}_{qj}$ are B-spline estimators of order $m\geq r$ constructed via least squares with backfitting.

Then the mean squared error achieves the minimax-optimal rate:

$$ $\mathbb{E}\!\left[\|\hat{f}_{n}-f\|_{L^{2}([0,1]^{d})}^{2}\right]=O\!\left(M^{2(d-1)}n^{-\frac{2r}{2r+1}}\right)=O\!\left(n^{-\frac{2r}{2r+1}}\right).$ (22) $$

###### Remark 2 .

The hybrid architecture allows for both additive and multiplicative compositions of univariate functions. The multiplicative terms induce a constant overhead of $M^{2(d-1)}$, but the convergence rate in $n$ remains minimax optimal. This overhead arises because estimating multiplicative interactions requires products of univariate spline bases. While this increases the constant factor, the dependence on $n$ is unaffected. For moderate dimensions and bounded smooth components, hybrid KANs remain statistically efficient.

Theorems [1](https://arxiv.org/html/2509.19830v1#Thmtheorem1) and [2](https://arxiv.org/html/2509.19830v1#Thmtheorem2) provide a theoretical explanation for recent empirical findings (Wang et al., 2025; SS et al., 2024; Aghaei, 2024), where KANs achieve near-minimal losses across diverse practical tasks. The following corollary formalizes the minimax optimality of KAN estimators over a broad class of structured functions, encompassing both additive and hybrid compositions.

###### Corollary 1 (Minimax Optimality of KAN Estimators) .

Let $\mathcal{F}_{\mathrm{KAN}}^{r}$ denote the class of continuous functions
$f:[0,1]^{d}\to\mathbb{R}$ that, by the Kolmogorov–Arnold representation theorem,
admit a decomposition of the form

$$ $f(x)=\sum_{q=1}^{Q}g_{q}\!\big(T_{q}(x)\big),$ (23) $$

where each $T_{q}(x)$ is either an additive term $\sum_{j=1}^{d}\psi_{qj}(x_{j})$
or a multiplicative term $\prod_{j=1}^{d}\psi_{qj}(x_{j})$,
and the univariate component functions satisfy
$g_{q}\in\mathcal{H}^{r}([0,1])$ (Lipschitz continuous)
and $\psi_{qj}\in\mathcal{H}^{r}([0,1])$ with smoothness index $r\geq 1$.

Suppose we observe $n$ i.i.d. samples $\{(X^{(i)},Y^{(i)})\}_{i=1}^{n}$
from the model

$$ $Y=f(X)+\varepsilon,\quad\mathbb{E}[\varepsilon]=0,\;\;\mathbb{E}[\varepsilon^{2}]=\sigma^{2}<\infty.$ (24) $$

Then the KAN estimator $\hat{f}_{n}$, constructed using B-splines of order $m\geq r$
with least squares estimation and backfitting for inner functions,
achieves the minimax rate for this class:

$$ $\inf_{\tilde{f}_{n}}\sup_{f\in\mathcal{F}_{\mathrm{KAN}}^{r}}\mathbb{E}\!\left[\|\tilde{f}_{n}-f\|_{L^{2}([0,1]^{d})}^{2}\right]=O\!\left(n^{-\frac{2r}{2r+1}}\right),$ (25) $$

where the infimum is taken over all measurable estimators $\tilde{f}_{n}$.

Finally, we provide a guideline for choosing the number of knots in the spline basis for optimal performance.

###### Corollary 2 (Optimal Knot Number for Univariate Spline Units) .

Let each B-spline unit $\psi_{qj}$ in a KAN approximate a function in the Sobolev space $\mathcal{H}^{r}([0,1])$. Then the optimal number of interior knots per unit that balances the bias–variance trade-off is

$$ $k\asymp n^{1/(2r+1)},$ (26) $$

which yields the minimax optimal convergence rate

$$ $\mathbb{E}\|\hat{\psi}-\psi\|_{L^{2}}^{2}=O\!\left(n^{-\frac{2r}{2r+1}}\right).$ (27) $$

for each univariate spline fit.

###### Remark 3 .

This result follows from classical nonparametric regression theory and provides a principled guideline for selecting the spline resolution in KANs based on the sample size and assumed smoothness of the target function.

These results confirm that spline-based KAN estimators achieve minimax-optimal convergence under standard smoothness assumptions. While additive KANs offer statistical efficiency and simplicity, hybrid architectures expand expressiveness with only a modest complexity overhead. These theoretical guarantees provide a strong foundation for using KANs in structured, interpretable learning tasks.

###### Theorem 1 (Convergence Rate of Additive KAN Estimator) .

###### Remark 1 .

###### Theorem 2 (Convergence Rate of Hybrid Additive–Multiplicative KAN Estimator) .

###### Remark 2 .

###### Corollary 1 (Minimax Optimality of KAN Estimators) .

###### Corollary 2 (Optimal Knot Number for Univariate Spline Units) .

###### Remark 3 .

## 5 Simulation Study

To validate the theoretical results and demonstrate the practical performance of Kolmogorov–Arnold Networks (KANs), we conduct simulation experiments on synthetic datasets generated from known function classes. In particular, we compare additive KANs and hybrid additive–multiplicative KANs against standard multilayer perceptrons (MLPs). The data and code used in this paper are publicly available on GitHub at [https://github.com/liouvill/KAN-Converge](https://github.com/liouvill/KAN-Converge).

### 5.1 Experimental Setup

Synthetic regression data are generated from functions defined on $[0,1]^{d}$. We consider two representative targets.

The first target is a piecewise polynomial function with $d=5$ and exact Sobolev smoothness $r$, given by

$$ $f(x)\;=\;\sin\!\Bigl(\pi\sum_{j=1}^{d}\psi(x_{j})\Bigr),$ (28) $$

where $\psi:[0,1]\to\mathbb{R}$ is defined as

$$ $\psi(t)\;=\;\begin{cases}t^{\,r+1},&0\leq t<\frac{1}{2},\[6.0pt] (1-t)^{\,r+1},&\frac{1}{2}\leq t\leq 1.\end{cases}$ $$

The second target is a periodic function with $d=1$ and exact Sobolev smoothness $r$, constructed as a Fourier series with coefficients

$$ $a_{k}\;=\;\frac{1}{k^{\,r+1/2}\,\log(k+1)},\qquad k\geq 1,$ $$

so that

$$ $f(x)\;=\;\sum_{k=1}^{\infty}a_{k}\sin(2\pi kx).$ $$

To see that $f$ has Sobolev smoothness exactly $r$, recall that for a Fourier series with coefficients $a_{k}$, membership $f\in H^{t}([0,1])$ is equivalent (up to constants) to

$$ $\sum_{k=1}^{\infty}(1+k^{2})^{t}|a_{k}|^{2}<\infty.$ $$

Substituting $a_{k}=1/(k^{r+1/2}\log(k+1))$, for large $k$ the summand behaves like

$$ $(1+k^{2})^{t}|a_{k}|^{2}\sim k^{2t}\cdot k^{-2r-1}\cdot\frac{1}{\log^{2}k}=\frac{k^{2(t-r)-1}}{\log^{2}k}.$ $$

If $t=r$, the summand is $\sim 1/(k\log^{2}k)$, and the series converges by the standard Cauchy condensation test (the condensed summand is $\sim 1/k^{2}$). Hence $f\in H^{r}([0,1])$.

If $t>r$, the exponent $2(t-r)-1>-1$, so the summand behaves like $k^{\alpha}$ with $\alpha\geq-1+\delta$ for some $\delta>0$, and the series diverges. Hence $f\notin H^{t}([0,1])$ for any $t>r$.

Therefore, $f$ belongs to $H^{r}([0,1])$ but not to any $H^{r+\varepsilon}([0,1])$, i.e., its Sobolev smoothness is exactly $r$. This construction provides a controlled setting for evaluating estimators precisely at the smoothness condition.

In the experiments, we use $r=2$, which for the polynomial case yields a piecewise cubic function with continuous first and second derivatives. Training inputs are sampled uniformly as $X_{i}\sim\mathsf{Unif}([0,1]^{d})$, and responses are observed as

$$ $Y_{i}\;=\;f(X_{i})+\varepsilon_{i},\qquad\varepsilon_{i}\sim\mathcal{N}(0,\sigma^{2}),\quad\sigma=0.05.$ (29) $$

Sample sizes are chosen as $n\in[100,\,200,\,400,\,800,\,1600,\,3200,\,6400,\,12800]$.
For each $n$, the number of B-spline grid intervals is set according to the theoretical guideline $k\asymp n^{1/(2r+1)}$. Both additive and hybrid KANs are constructed using cubic ($m=3$) B-splines, trained end-to-end by least squares. Hybrid models allocate half of the hidden units to multiplicative interactions and half to additive terms. For comparison, fully connected multilayer perceptrons (MLPs) with 3 hidden layers of 256 units each and ReLU activations are trained under the same protocol. All models are optimized using the LBFGS algorithm (Liu & Nocedal, 1989).

### 5.2 Results

Figure: Figure 1: Convergence rates of additive KAN, hybrid KAN, and MLP estimators on two synthetic targets. The dashed line indicates the theoretical slope $-4/5$. Left: piecewise polynomial function with exact Sobolev smoothness $r=2$. Right: periodic function with exact Sobolev smoothness $r=2$, constructed from a Fourier series. Mean squared error on a test set is plotted against sample size $n$ on a log–log scale.
Refer to caption: x1.png

Figure [1](https://arxiv.org/html/2509.19830v1#S5.F1) compares the convergence behavior of the three estimators across the two function classes. In both cases, additive and hybrid KANs achieve slopes that closely follow, and in fact are steeper than, the theoretical rate. This is consistent with the theory, since the minimax-optimal rate represents a worst-case upper bound. In favorable instances, such as the structured piecewise polynomial and Fourier series functions considered here, empirical performance can exceed this baseline.

The two KAN variants exhibit nearly indistinguishable performance as sample sizes increase, indicating that the inclusion of multiplicative interactions does not substantially alter convergence in these settings. By contrast, the MLP baseline converges more slowly and requires much larger sample sizes to approach the accuracy attained by KANs.

These results validate the theoretical analysis and confirm that spline-based KANs achieve both statistical optimality and practical efficiency. The convergence pattern highlights the advantage of incorporating structural priors through B-spline representations, which enables KANs to learn efficiently even with moderate data, in contrast to the slower learning dynamics observed in black-box neural networks such as MLPs.

## 6 Conclusion

This paper has presented a theoretical analysis of Kolmogorov–Arnold Networks (KANs) with univariate spline units for nonlinear regression. We established that both additive and hybrid additive–multiplicative KAN architectures achieve minimax-optimal convergence rates under standard smoothness assumptions.

The theoretical guarantees are corroborated by simulation experiments, which show that both additive and hybrid KANs empirically attain the optimal convergence rate, and perform comparably across sample sizes. In contrast, standard neural networks exhibit slower convergence and require larger data to achieve similar accuracy.

Future work will explore scalable algorithmic implementations of KANs for real-world datasets and integration with deep learning architectures for structured end-to-end system dynamics learning and symbolic equation discovery.

## References

- Aghaei (2024)
Alireza Afzal Aghaei.
rkan: Rational kolmogorov-arnold networks.
*arXiv preprint arXiv:2406.14495*, 2024.
- Arnol’d (1957)
Vladimir Igorevich Arnol’d.
On functions of three variables.
In *Doklady Akademii Nauk*, volume 114, pp. 679–681. Russian Academy of Sciences, 1957.
- Barron (1994)
Andrew R Barron.
Approximation and estimation bounds for artificial neural networks.
*Machine learning*, 14(1):115–133, 1994.
- Barron (2002)
Andrew R Barron.
Universal approximation bounds for superpositions of a sigmoidal function.
*IEEE Transactions on Information theory*, 39(3):930–945, 2002.
- Bauer & Kohler (2019)
Benedikt Bauer and Michael Kohler.
On deep learning as a remedy for the curse of dimensionality in nonparametric regression.
*The Annals of Statistics*, 47(4):2261 – 2285, 2019.
doi: 10.1214/18-AOS1747.
URL [https://doi.org/10.1214/18-AOS1747](https://doi.org/10.1214/18-AOS1747).
- Bodner et al. (2024)
Alexander Dylan Bodner, Antonio Santiago Tepsich, Jack Natan Spolski, and Santiago Pourteau.
Convolutional kolmogorov-arnold networks.
*arXiv preprint arXiv:2406.13155*, 2024.
- Chen (2007)
Xiaohong Chen.
Large sample sieve estimation of semi-nonparametric models.
*Handbook of econometrics*, 6:5549–5632, 2007.
- Gao & Tan (2025)
Yihang Gao and Vincent YF Tan.
On the convergence of (stochastic) gradient descent for kolmogorov–arnold networks.
*IEEE Transactions on Information Theory*, 2025.
- Györfi et al. (2002)
László Györfi, Michael Kohler, Adam Krzyżak, and Harro Walk.
*A distribution-free theory of nonparametric regression*.
Springer, 2002.
- Härdle (1990)
Wolfgang Härdle.
*Applied nonparametric regression*.
Number 19. Cambridge university press, 1990.
- He & Shi (1994)
Xuming He and Peide Shi.
Convergence rate of b-spline estimators of nonparametric conditional quantile functions.
*Journaltitle of Nonparametric Statistics*, 3(3-4):299–308, 1994.
- Imaizumi & Fukumizu (2019)
Masaaki Imaizumi and Kenji Fukumizu.
Deep neural networks learn non-smooth functions effectively.
In *The 22nd international conference on artificial intelligence and statistics*, pp. 869–878. PMLR, 2019.
- Kiamari et al. (2024)
Mehrdad Kiamari, Mohammad Kiamari, and Bhaskar Krishnamachari.
Gkan: Graph kolmogorov-arnold networks.
*arXiv preprint arXiv:2406.06470*, 2024.
- Koenig et al. (2024)
Benjamin C Koenig, Suyong Kim, and Sili Deng.
Kan-odes: Kolmogorov–arnold network ordinary differential equations for learning dynamical systems and hidden physics.
*Computer Methods in Applied Mechanics and Engineering*, 432:117397, 2024.
- Kohler & Langer (2021)
Michael Kohler and Sophie Langer.
On the rate of convergence of fully connected deep neural network regression estimates.
*The Annals of Statistics*, 49(4):2231–2249, 2021.
- Kolmogorov (1957)
Andrei Nikolaevich Kolmogorov.
On the representations of continuous functions of many variables by superposition of continuous functions of one variable and addition.
In *Dokl. Akad. Nauk USSR*, volume 114, pp. 953–956, 1957.
- Li et al. (1995)
GY Li, Peide Shi, and Guoying Li.
Global convergence rates of b-spline m-estimators in nonparametric regression.
*Statistica Sinica*, pp. 303–318, 1995.
- Liu & Nocedal (1989)
Dong C Liu and Jorge Nocedal.
On the limited memory bfgs method for large scale optimization.
*Mathematical programming*, 45(1):503–528, 1989.
- Liu et al. (2024a)
Ziming Liu, Pingchuan Ma, Yixuan Wang, Wojciech Matusik, and Max Tegmark.
Kan 2.0: Kolmogorov-arnold networks meet science.
*arXiv preprint arXiv:2408.10205*, 2024a.
- Liu et al. (2024b)
Ziming Liu, Yixuan Wang, Sachin Vaidya, Fabian Ruehle, James Halverson, Marin Soljačić, Thomas Y Hou, and Max Tegmark.
Kan: Kolmogorov-arnold networks.
*arXiv preprint arXiv:2404.19756*, 2024b.
- Nussbaum (1985)
Michael Nussbaum.
Spline smoothing in regression models and asymptotic efficiency in l 2.
*The Annals of Statistics*, pp. 984–997, 1985.
- Schmidt-Hieber (2020)
Johannes Schmidt-Hieber.
Nonparametric regression using deep neural networks with ReLU activation function.
*The Annals of Statistics*, 48(4):1875 – 1897, 2020.
doi: 10.1214/19-AOS1875.
URL [https://doi.org/10.1214/19-AOS1875](https://doi.org/10.1214/19-AOS1875).
- Schmidt-Hieber (2021)
Johannes Schmidt-Hieber.
The kolmogorov–arnold representation theorem revisited.
*Neural networks*, 137:119–126, 2021.
- Schumaker (2007)
Larry Schumaker.
*Spline functions: basic theory*.
Cambridge university press, 2007.
- Speckman (1985)
Paul Speckman.
Spline smoothing and optimal rates of convergence in nonparametric regression models.
*The Annals of Statistics*, pp. 970–983, 1985.
- SS et al. (2024)
Sidharth SS, Keerthana AR, Anas KP, et al.
Chebyshev polynomial-based kolmogorov-arnold networks: An efficient architecture for nonlinear function approximation.
*arXiv preprint arXiv:2405.07200*, 2024.
- Stone (1982)
Charles J Stone.
Optimal global rates of convergence for nonparametric regression.
*The annals of statistics*, pp. 1040–1053, 1982.
- Tsybakov (2008)
Alexandre B Tsybakov.
Nonparametric estimators.
In *Introduction to Nonparametric Estimation*, pp. 1–76. Springer, 2008.
- Unser et al. (2002)
Michael Unser, Akram Aldroubi, and Murray Eden.
B-spline signal processing. i. theory.
*IEEE transactions on signal processing*, 41(2):821–833, 2002.
- Vaca-Rubio et al. (2024)
Cristian J Vaca-Rubio, Luis Blanco, Roberto Pereira, and Màrius Caus.
Kolmogorov-arnold networks (kans) for time series analysis.
*arXiv preprint arXiv:2405.08790*, 2024.
- Wang et al. (2025)
Yizheng Wang, Jia Sun, Jinshuai Bai, Cosmin Anitescu, Mohammad Sadegh Eshaghi, Xiaoying Zhuang, Timon Rabczuk, and Yinghua Liu.
Kolmogorov–arnold-informed neural network: A physics-informed deep learning framework for solving forward and inverse problems based on kolmogorov–arnold networks.
*Computer Methods in Applied Mechanics and Engineering*, 433:117518, 2025.
- Xu et al. (2024)
Kunpeng Xu, Lifei Chen, and Shengrui Wang.
Kolmogorov-arnold networks for time series: Bridging predictive power and interpretability.
*arXiv preprint arXiv:2406.02496*, 2024.

## Appendix A Appendix: Proofs of Theorems and Corollaries

### A.1 Proof of Theorem 1 : Convergence Rate of Additive KAN

###### Theorem 1 (Convergence Rate of Additive KAN Estimator) .

Let $f:[0,1]^{d}\to\mathbb{R}$ be a continuous function.
By the Kolmogorov–Arnold representation theorem, there exist continuous univariate functions $g_{q}:[0,1]\to\mathbb{R}$ and $\psi_{qj}:[0,1]\to\mathbb{R}$, $q=1,\dots,Q$, $j=1,\dots,d$, such that $f$ can be expressed in the form

$$ $f(x)=\sum_{q=1}^{Q}g_{q}\!\left(\sum_{j=1}^{d}\psi_{qj}(x_{j})\right).$ $$

Assume further that $g_{q}\in\mathcal{H}^{r}([0,1])$ are Lipschitz continuous and $\psi_{qj}\in\mathcal{H}^{r}([0,1])$ with smoothness index $r\geq 1$.

Suppose we observe $n$ i.i.d. samples $\{(X^{(i)},Y^{(i)})\}_{i=1}^{n}$ from the model

$$ $Y=f(X)+\varepsilon,\quad\mathbb{E}[\varepsilon]=0,\;\;\mathbb{E}[\varepsilon^{2}]=\sigma^{2}<\infty.$ $$

Let $\hat{f}_{n}$ denote the additive KAN estimator

$$ $\hat{f}_{n}(x)=\sum_{q=1}^{Q}\hat{g}_{q}\!\left(\sum_{j=1}^{d}\hat{\psi}_{qj}(x_{j})\right),$ $$

where $\hat{g}_{q}$ and $\hat{\psi}_{qj}$ are B-spline estimators of order $m\geq r$ constructed via least squares with backfitting.

Then the mean squared error achieves the minimax-optimal rate:

$$ $\mathbb{E}\!\left[\|\hat{f}_{n}-f\|_{L^{2}([0,1]^{d})}^{2}\right]=O\!\left(n^{-\frac{2r}{2r+1}}\right).$ $$

###### Proof.

The strategy of this proof is to first break down the total Mean Squared Error (MSE) of the estimator $\hat{f}_{n}$ into the sum of errors from each of its $Q$ additive components. Then, for each component, we will further decompose the error to isolate the contribution from estimating the outer function ($g_{q}$) and the inner functions ($\psi_{qj}$). By establishing the convergence rate for each of these individual estimators, we can aggregate the bounds to determine the overall convergence rate of $\hat{f}_{n}$.

First, we decompose the total Mean Squared Error.

The total error of the estimator is the $L^{2}$ norm of the difference between the estimated function $\hat{f}_{n}$ and the true function $f$. We begin by expressing this difference and bounding its squared norm.

$$ $\displaystyle\|\hat{f}_{n}-f\|_{L^{2}}^{2}$ $\displaystyle=\left\|\sum_{q=1}^{Q}\hat{g}_{q}\left(\sum_{j=1}^{d}\hat{\psi}_{qj}\right)-\sum_{q=1}^{Q}g_{q}\left(\sum_{j=1}^{d}\psi_{qj}\right)\right\|_{L^{2}}^{2}$ $\displaystyle=\left\|\sum_{q=1}^{Q}\left(\hat{g}_{q}\left(\sum_{j=1}^{d}\hat{\psi}_{qj}\right)-g_{q}\left(\sum_{j=1}^{d}\psi_{qj}\right)\right)\right\|_{L^{2}}^{2}.$ $$

To bound this sum, we apply the Cauchy-Schwarz inequality in the form $(\sum_{i=1}^{N}a_{i})^{2}\leq N\sum_{i=1}^{N}a_{i}^{2}$. In our context, this translates to bounding the norm of a sum of functions: $\|\sum_{q=1}^{Q}F_{q}\|_{L^{2}}^{2}\leq Q\sum_{q=1}^{Q}\|F_{q}\|_{L^{2}}^{2}$. Let $F_{q}=\hat{g}_{q}\left(\sum_{j=1}^{d}\hat{\psi}_{qj}\right)-g_{q}\left(\sum_{j=1}^{d}\psi_{qj}\right)$. This gives us:

$$ $\|\hat{f}_{n}-f\|_{L^{2}}^{2}\leq Q\sum_{q=1}^{Q}\left\|\hat{g}_{q}\left(\sum_{j=1}^{d}\hat{\psi}_{qj}\right)-g_{q}\left(\sum_{j=1}^{d}\psi_{qj}\right)\right\|_{L^{2}}^{2}.$ $$

Now, we take the expectation over the data to get the Mean Squared Error. By the linearity of expectation, we can bring the expectation operator inside the summation:

$$ $\mathbb{E}\left[\|\hat{f}_{n}-f\|_{L^{2}}^{2}\right]\leq Q\sum_{q=1}^{Q}\mathbb{E}\left[\left\|\hat{g}_{q}\left(\sum_{j=1}^{d}\hat{\psi}_{qj}\right)-g_{q}\left(\sum_{j=1}^{d}\psi_{qj}\right)\right\|_{L^{2}}^{2}\right].$ $$

This inequality shows that the total MSE is bounded by the sum of the MSEs of each of the $Q$ additive components. The next step is to determine the convergence rate for each of these individual terms.

Next, we bound the error of a single component.

Let’s focus on the MSE of a single component, indexed by $q$. For notational simplicity, we define the true inner function as $u_{q}(x)=\sum_{j=1}^{d}\psi_{qj}(x_{j})$ and its corresponding estimator as $\hat{u}_{q}(x)=\sum_{j=1}^{d}\hat{\psi}_{qj}(x_{j})$. Our goal is to bound the term $\mathbb{E}\left[\|\hat{g}_{q}(\hat{u}_{q})-g_{q}(u_{q})\|_{L^{2}}^{2}\right]$.

We use a standard technique of adding and subtracting an intermediate term, $g_{q}(\hat{u}_{q})$, inside the norm. This allows us to separate the error into two parts: one from the estimation of the outer function $g_{q}$ and one from the estimation of the inner functions that make up $u_{q}$.

$$ $\displaystyle|\hat{g}_{q}(\hat{u}_{q}(x))-g_{q}(u_{q}(x))|^{2}$ $\displaystyle=|\left(\hat{g}_{q}(\hat{u}_{q}(x))-g_{q}(\hat{u}_{q}(x))\right)+\left(g_{q}(\hat{u}_{q}(x))-g_{q}(u_{q}(x))\right)|^{2}$ $\displaystyle\leq 2|\hat{g}_{q}(\hat{u}_{q}(x))-g_{q}(\hat{u}_{q}(x))|^{2}+2|g_{q}(\hat{u}_{q}(x))-g_{q}(u_{q}(x))|^{2}.$ $$

This step uses the inequality $(a+b)^{2}\leq 2a^{2}+2b^{2}$. By integrating over the domain $[0,1]^{d}$ and taking the expectation, we obtain a bound on the MSE for the $q$-th component:

$$ $\mathbb{E}\left[\|\hat{g}_{q}(\hat{u}_{q})-g_{q}(u_{q})\|_{L^{2}}^{2}\right]\leq 2\mathbb{E}\left[\int_{[0,1]^{d}}|\hat{g}_{q}(\hat{u}_{q}(x))-g_{q}(\hat{u}_{q}(x))|^{2}dx\right]+2\mathbb{E}\left[\|g_{q}(\hat{u}_{q})-g_{q}(u_{q})\|_{L^{2}}^{2}\right].$ (30) $$

Let’s analyze each of the two terms on the right-hand side.

- •
Term 1: Error from estimating the outer function $g_{q}$.
The first term represents the error from estimating $g_{q}$. The established theory for B-spline M-estimators states that for a univariate function $h\in\mathcal{H}^{r}([0,1])$, the Mean Integrated Squared Error (MISE) of its estimator $\hat{h}$ based on $n$ samples achieves an optimal one-dimensional convergence rate. That is, $\mathbb{E}\left[\|\hat{h}-h\|_{L^{2}([0,1])}^{2}\right]=O\!\left(n^{-\frac{2r}{2r+1}}\right)$. By extension, the error in estimating $g_{q}$ is bounded by this rate.
$\mathbb{E}\left[\|\hat{g}_{q}-g_{q}\|_{L^{2}}^{2}\right]=O\!\left(n^{-\frac{2r}{2r+1}}\right).$
- •
Term 2: Propagated error from estimating the inner functions $\psi_{qj}$.
The second term, $\mathbb{E}\left[\|g_{q}(\hat{u}_{q})-g_{q}(u_{q})\|_{L^{2}}^{2}\right]$, captures how errors in estimating the inner functions are propagated through the outer function $g_{q}$. Since $g_{q}$ is assumed to be Lipschitz continuous with constant $L_{q}$, we have $|g_{q}(a)-g_{q}(b)|\leq L_{q}|a-b|$. Applying this to our term pointwise for each $x$:
$|g_{q}(\hat{u}_{q}(x))-g_{q}(u_{q}(x))|^{2}\leq L_{q}^{2}|\hat{u}_{q}(x)-u_{q}(x)|^{2}.$
Integrating over $[0,1]^{d}$ and taking the expectation yields:
$\mathbb{E}\left[\|g_{q}(\hat{u}_{q})-g_{q}(u_{q})\|_{L^{2}}^{2}\right]\leq L_{q}^{2}\mathbb{E}\left[\|\hat{u}_{q}-u_{q}\|_{L^{2}}^{2}\right].$
Now we must bound the MSE of $\hat{u}_{q}$. We again use the Cauchy-Schwarz inequality $(\sum_{j=1}^{d}a_{j})^{2}\leq d\sum_{j=1}^{d}a_{j}^{2}$:
$\displaystyle\|\hat{u}_{q}-u_{q}\|_{L^{2}}^{2}$
$\displaystyle=\left\|\sum_{j=1}^{d}(\hat{\psi}_{qj}-\psi_{qj})\right\|_{L^{2}}^{2}\leq d\sum_{j=1}^{d}\|\hat{\psi}_{qj}-\psi_{qj}\|_{L^{2}}^{2}.$
To justify the convergence rate of each $\mathbb{E}\left[\|\hat{\psi}_{qj}-\psi_{qj}\|_{L^{2}}^{2}\right]$, we consider how the estimator $\hat{\psi}_{qj}$ is obtained. An iterative method like backfitting updates each function by regressing a partial residual onto the corresponding variable. For example, to update $\hat{\psi}_{qj}$, one computes residuals that remove the effect of all other estimated functions. This creates a one-dimensional regression problem of these residuals against the variable $x_{j}$. Since this step is a standard univariate B-spline estimation based on $n$ data points, the theory of nonparametric regression guarantees that the resulting estimator achieves the optimal rate:
$\mathbb{E}\left[\|\hat{\psi}_{qj}-\psi_{qj}\|_{L^{2}}^{2}\right]=O\!\left(n^{-\frac{2r}{2r+1}}\right)\quad\text{for each }j=1,\dots,d.$
Substituting this back, we get:
$\mathbb{E}\left[\|\hat{u}_{q}-u_{q}\|_{L^{2}}^{2}\right]\leq d\sum_{j=1}^{d}\mathbb{E}\left[\|\hat{\psi}_{qj}-\psi_{qj}\|_{L^{2}}^{2}\right]=d\sum_{j=1}^{d}O\!\left(n^{-\frac{2r}{2r+1}}\right).$
Since $d$ is a fixed constant, we absorb the factor into the big-O notation:
$\mathbb{E}\left[\|\hat{u}_{q}-u_{q}\|_{L^{2}}^{2}\right]=O\!\left(n^{-\frac{2r}{2r+1}}\right).$
Therefore, the second term in our main inequality for the component error is also bounded by this rate: $\mathbb{E}\left[\|g_{q}(\hat{u}_{q})-g_{q}(u_{q})\|_{L^{2}}^{2}\right]\leq L_{q}^{2}\cdot O\!\left(n^{-\frac{2r}{2r+1}}\right)=O\!\left(n^{-\frac{2r}{2r+1}}\right)$.

Finally, we aggregate the error bounds.

We substitute the rates for the two terms back into the inequality ([30](https://arxiv.org/html/2509.19830v1#A1.E30)) for the MSE of a single component:

$$ $\mathbb{E}\left[\|\hat{g}_{q}(\hat{u}_{q})-g_{q}(u_{q})\|_{L^{2}}^{2}\right]\leq 2\cdot O\!\left(n^{-\frac{2r}{2r+1}}\right)+2L_{q}^{2}\cdot O\!\left(n^{-\frac{2r}{2r+1}}\right).$ $$

Since $L_{q}$ is a fixed constant, the entire expression simplifies to:

$$ $\mathbb{E}\left[\|\hat{g}_{q}(\hat{u}_{q})-g_{q}(u_{q})\|_{L^{2}}^{2}\right]=O\!\left(n^{-\frac{2r}{2r+1}}\right).$ $$

This demonstrates that each additive component of the KAN estimator converges at the optimal one-dimensional rate. Now we return to our initial bound for the total MSE:

$$ $\mathbb{E}\left[\|\hat{f}_{n}-f\|_{L^{2}}^{2}\right]\leq Q\sum_{q=1}^{Q}\mathbb{E}\left[\|\hat{g}_{q}(\hat{u}_{q})-g_{q}(u_{q})\|_{L^{2}}^{2}\right]=Q\sum_{q=1}^{Q}O\!\left(n^{-\frac{2r}{2r+1}}\right).$ $$

The sum consists of $Q$ terms, so the result is $Q\cdot Q\cdot O\!\left(n^{-\frac{2r}{2r+1}}\right)$. Since the number of additive components $Q$ and the dimension $d$ are fixed constants that do not depend on the sample size $n$, the overall convergence rate is determined by the rate of its constituent parts:

$$ $\mathbb{E}\left[\|\hat{f}_{n}-f\|_{L^{2}}^{2}\right]=O\!\left(n^{-\frac{2r}{2r+1}}\right).$ $$

The significance of this result is that for functions that possess an additive structure, the KAN estimator’s convergence rate depends only on the smoothness $r$ of the univariate component functions, not on the ambient dimension $d$. This demonstrates the estimator’s ability to overcome the curse of dimensionality, a critical advantage for learning in high-dimensional spaces.
∎

###### Theorem 1 (Convergence Rate of Additive KAN Estimator) .

###### Proof.

### A.2 Proof of Theorem 2 : Convergence Rate of Hybrid Additive–Multiplicative KAN

###### Theorem 2 (Convergence Rate of Hybrid Additive–Multiplicative KAN Estimator) .

Let $f:[0,1]^{d}\to\mathbb{R}$ be a continuous function.
By the Kolmogorov–Arnold representation theorem, $f$ can be expressed in the form

$$ $f(x)=\sum_{q=1}^{Q}g_{q}\!\big(T_{q}(x)\big),$ $$

where each $T_{q}(x)$ is either an *additive term* $\sum_{j=1}^{d}\psi_{qj}(x_{j})$ or a *multiplicative term* $\prod_{j=1}^{d}\psi_{qj}(x_{j})$, with univariate component functions $\psi_{qj}$ and outer functions $g_{q}$ continuous.

Assume further that $g_{q}\in\mathcal{H}^{r}([0,1])$ are Lipschitz continuous and $\psi_{qj}\in\mathcal{H}^{r}([0,1])$ with smoothness index $r\geq 1$, uniformly bounded by $M$ with Lipschitz constant at most $L$.

Suppose we observe $n$ i.i.d. samples $(X_{i},Y_{i})$ from the model

$$ $Y=f(X)+\varepsilon,\quad\mathbb{E}[\varepsilon]=0,\;\;\mathbb{E}[\varepsilon^{2}]=\sigma^{2}<\infty.$ $$

Let $\hat{f}_{n}$ denote the hybrid KAN estimator

$$ $\hat{f}_{n}(x)=\sum_{q=1}^{Q}\hat{g}_{q}\!\big(T_{q}^{(n)}(x)\big),$ $$

where $\hat{g}_{q}$ and $\hat{\psi}_{qj}$ are B-spline estimators of order $m\geq r$ constructed via least squares with backfitting.

Then the mean squared error achieves the minimax-optimal rate:

$$ $\mathbb{E}\!\left[\|\hat{f}_{n}-f\|_{L^{2}([0,1]^{d})}^{2}\right]=O\!\left(M^{2(d-1)}n^{-\frac{2r}{2r+1}}\right)=O\!\left(n^{-\frac{2r}{2r+1}}\right).$ $$

###### Proof.

The structure of this proof follows that of Theorem 1, but we must now account for the error propagation through both additive and multiplicative nodes.

First, we decompose the total Mean Squared Error (MSE). As in the purely additive case, we can bound the total MSE by the sum of the MSEs of the individual components:

$$ $\mathbb{E}\left[\|\hat{f}_{n}-f\|_{L^{2}}^{2}\right]\leq Q\sum_{q=1}^{Q}\mathbb{E}\left[\left\|\hat{g}_{q}\left(\hat{T}_{q}\right)-g_{q}\left(T_{q}\right)\right\|_{L^{2}}^{2}\right],$ $$

where $\hat{T}_{q}$ is the estimator for $T_{q}$ constructed from the B-spline estimators $\hat{\psi}_{qj}$.

Next, we bound the error for a single component. We decompose the error for the $q$-th component using the triangle inequality:

$$ $\displaystyle\mathbb{E}\left[\|\hat{g}_{q}(\hat{T}_{q})-g_{q}(T_{q})\|_{L^{2}}^{2}\right]$ $\displaystyle\leq 2\mathbb{E}\left[\|\hat{g}_{q}(\hat{T}_{q})-g_{q}(\hat{T}_{q})\|_{L^{2}}^{2}\right]+2\mathbb{E}\left[\|g_{q}(\hat{T}_{q})-g_{q}(T_{q})\|_{L^{2}}^{2}\right]$ $\displaystyle\leq 2\mathbb{E}\left[\|\hat{g}_{q}-g_{q}\|_{L^{2}}^{2}\right]+2L^{2}\mathbb{E}\left[\|\hat{T}_{q}-T_{q}\|_{L^{2}}^{2}\right].$ $$

The first term is the standard univariate B-spline estimation error, which is $O\!\left(n^{-\frac{2r}{2r+1}}\right)$. The second term depends on the structure of the node $T_{q}$. We analyze this term for both additive and multiplicative cases.

Case 1: $T_{q}$ is an additive node. From the proof of Theorem 1, we have already shown that:

$$ $\mathbb{E}\left[\|\hat{T}_{q}-T_{q}\|_{L^{2}}^{2}\right]=\mathbb{E}\left[\left\|\sum_{j=1}^{d}(\hat{\psi}_{qj}-\psi_{qj})\right\|_{L^{2}}^{2}\right]=O\!\left(n^{-\frac{2r}{2r+1}}\right).$ $$

Case 2: $T_{q}$ is a multiplicative node. Here, $T_{q}(x)=\prod_{j=1}^{d}\psi_{qj}(x_{j})$. To bound the error $|\hat{T}_{q}(x)-T_{q}(x)|$, we use a telescoping sum:

$$ $\displaystyle\left|\prod_{j=1}^{d}\hat{\psi}_{qj}-\prod_{j=1}^{d}\psi_{qj}\right|$ $\displaystyle=\left|\sum_{k=1}^{d}\left(\prod_{j=1}^{k-1}\psi_{qj}\prod_{j=k}^{d}\hat{\psi}_{qj}-\prod_{j=1}^{k}\psi_{qj}\prod_{j=k+1}^{d}\hat{\psi}_{qj}\right)\right|$ $\displaystyle=\left|\sum_{k=1}^{d}\left(\prod_{j=1}^{k-1}\psi_{qj}\right)(\hat{\psi}_{qk}-\psi_{qk})\left(\prod_{j=k+1}^{d}\hat{\psi}_{qj}\right)\right|$ $\displaystyle\leq\sum_{k=1}^{d}|\hat{\psi}_{qk}-\psi_{qk}|\left|\prod_{j=1}^{k-1}\psi_{qj}\right|\left|\prod_{j=k+1}^{d}\hat{\psi}_{qj}\right|.$ $$

Since all functions are bounded by $M$, and the B-spline estimators $\hat{\psi}_{qj}$ also inherit this bound (with high probability), the product terms are bounded by $M^{d-1}$. Thus:

$$ $|\hat{T}_{q}(x)-T_{q}(x)|\leq M^{d-1}\sum_{j=1}^{d}|\hat{\psi}_{qj}(x_{j})-\psi_{qj}(x_{j})|.$ $$

Squaring this expression and applying the Cauchy-Schwarz inequality, we get:

$$ $|\hat{T}_{q}(x)-T_{q}(x)|^{2}\leq M^{2(d-1)}\left(\sum_{j=1}^{d}|\hat{\psi}_{qj}-\psi_{qj}|\right)^{2}\leq dM^{2(d-1)}\sum_{j=1}^{d}|\hat{\psi}_{qj}-\psi_{qj}|^{2}.$ $$

Integrating over $[0,1]^{d}$ and taking the expectation gives the MSE of $\hat{T}_{q}$:

$$ $\displaystyle\mathbb{E}\left[\|\hat{T}_{q}-T_{q}\|_{L^{2}}^{2}\right]$ $\displaystyle\leq dM^{2(d-1)}\sum_{j=1}^{d}\mathbb{E}\left[\|\hat{\psi}_{qj}-\psi_{qj}\|_{L^{2}}^{2}\right]$ $\displaystyle=dM^{2(d-1)}\sum_{j=1}^{d}O\!\left(n^{-\frac{2r}{2r+1}}\right)$ $\displaystyle=O\!\left(d^{2}M^{2(d-1)}n^{-\frac{2r}{2r+1}}\right).$ $$

This shows that the error for a multiplicative node includes a factor of $M^{2(d-1)}$, which depends on the input dimension $d$.

Finally, we aggregate the error bounds. The error for a single component, whether additive or multiplicative, is bounded by the worst case. Therefore, for any node $q$:

$$ $\mathbb{E}\left[\left\|\hat{g}_{q}(\hat{T}_{q})-g_{q}(T_{q})\right\|_{L^{2}}^{2}\right]=O\!\left(M^{2(d-1)}n^{-\frac{2r}{2r+1}}\right).$ $$

Substituting this back into the total MSE bound gives:

$$ $\mathbb{E}\left[\|\hat{f}_{n}-f\|_{L^{2}}^{2}\right]\leq Q\sum_{q=1}^{Q}O\!\left(M^{2(d-1)}n^{-\frac{2r}{2r+1}}\right)=O\!\left(M^{2(d-1)}n^{-\frac{2r}{2r+1}}\right)=O\!\left(n^{-\frac{2r}{2r+1}}\right).$ $$

Since $Q$, $d$, and $M$ are constants with respect to $n$, this completes the proof. The convergence rate maintains the optimal dependency on $n$ but now includes a factor that grows with the dimensionality of the multiplicative interactions.
∎

###### Theorem 2 (Convergence Rate of Hybrid Additive–Multiplicative KAN Estimator) .

###### Proof.

### A.3 Proof of Corollary 1

###### Corollary 1 (Minimax Optimality of KAN Estimators) .

Let $\mathcal{F}_{\mathrm{KAN}}^{r}$ denote the class of continuous functions
$f:[0,1]^{d}\to\mathbb{R}$ that, by the Kolmogorov–Arnold representation theorem,
admit a decomposition of the form

$$ $f(x)=\sum_{q=1}^{Q}g_{q}\!\big(T_{q}(x)\big),$ $$

where each $T_{q}(x)$ is either an additive term $\sum_{j=1}^{d}\psi_{qj}(x_{j})$
or a multiplicative term $\prod_{j=1}^{d}\psi_{qj}(x_{j})$,
and the univariate component functions satisfy
$g_{q}\in\mathcal{H}^{r}([0,1])$ (Lipschitz continuous)
and $\psi_{qj}\in\mathcal{H}^{r}([0,1])$ with smoothness index $r\geq 1$.

Suppose we observe $n$ i.i.d. samples $\{(X^{(i)},Y^{(i)})\}_{i=1}^{n}$
from the model

$$ $Y=f(X)+\varepsilon,\quad\mathbb{E}[\varepsilon]=0,\;\;\mathbb{E}[\varepsilon^{2}]=\sigma^{2}<\infty.$ $$

Then the KAN estimator $\hat{f}_{n}$, constructed using B-splines of order $m\geq r$
with least squares estimation and backfitting for inner functions,
achieves the minimax rate for this class:

$$ $\inf_{\tilde{f}_{n}}\sup_{f\in\mathcal{F}_{\mathrm{KAN}}^{r}}\mathbb{E}\!\left[\|\tilde{f}_{n}-f\|_{L^{2}([0,1]^{d})}^{2}\right]=O\!\left(n^{-\frac{2r}{2r+1}}\right),$ $$

where the infimum is taken over all measurable estimators $\tilde{f}_{n}$.

###### Proof.

The proof of minimax optimality requires establishing both an upper bound and a lower bound for the estimation error over the function class $\mathcal{F}_{\mathrm{KAN}}^{r}$.

First, we establish the upper bound. The minimax risk is defined as $\inf_{\tilde{f}_{n}}\sup_{f\in\mathcal{F}_{\mathrm{KAN}}^{r}}\mathbb{E}\left[\|\tilde{f}_{n}-f\|_{L^{2}}^{2}\right]$. The performance of any specific estimator provides an upper bound on this quantity. From Theorem 2, we know that for any function $f\in\mathcal{F}_{\mathrm{KAN}}^{r}$, the KAN estimator $\hat{f}_{n}$ satisfies:

$$ $\mathbb{E}\left[\|\hat{f}_{n}-f\|_{L^{2}}^{2}\right]=O\!\left(n^{-\frac{2r}{2r+1}}\right).$ $$

The constants hidden in the $O(\cdot)$ notation depend on fixed parameters like $Q,d,L,r$, but not on the specific choice of $f$ within the class $\mathcal{F}_{\mathrm{KAN}}^{r}$ (as long as it satisfies the uniform bounds). Therefore, we can take the supremum over all $f\in\mathcal{F}_{\mathrm{KAN}}^{r}$ to get:

$$ $\sup_{f\in\mathcal{F}_{\mathrm{KAN}}^{r}}\mathbb{E}\left[\|\hat{f}_{n}-f\|_{L^{2}}^{2}\right]=O\!\left(n^{-\frac{2r}{2r+1}}\right).$ $$

This provides the upper bound for the minimax risk.

Next, we establish the lower bound. To find a lower bound for the entire class $\mathcal{F}_{\mathrm{KAN}}^{r}$, we can find a lower bound for any smaller, “harder” subclass contained within it. Consider the simple subclass of univariate functions of the form $f(x)=h(x_{1})$, where $h\in\mathcal{H}^{r}([0,1])$. Any such function can be represented in the KAN architecture, for example, by setting $Q=1$, $T_{1}$ to be an additive node, $g_{1}(u)=u$, $\psi_{11}(x_{1})=h(x_{1})$, and $\psi_{1j}(x_{j})=0$ for $j>1$. Therefore, this class of univariate Sobolev functions is a subset of $\mathcal{F}_{\mathrm{KAN}}^{r}$.

The minimax lower bound for estimating a function from the Sobolev class $\mathcal{H}^{r}([0,1])$ based on $n$ samples is a classical result in nonparametric statistics (e.g., Stone, 1982). This lower bound is known to be:

$$ $\inf_{\tilde{f}_{n}}\sup_{h\in\mathcal{H}^{r}}\mathbb{E}\left[\|\tilde{f}_{n}-h\|_{L^{2}}^{2}\right]=O\!\left(n^{-\frac{2r}{2r+1}}\right).$ $$

Since the class of all KAN-representable functions $\mathcal{F}_{\mathrm{KAN}}^{r}$ contains this univariate Sobolev class, the minimax risk over $\mathcal{F}_{\mathrm{KAN}}^{r}$ must be at least as large as the risk over this subclass. Thus, we have:

$$ $\inf_{\tilde{f}_{n}}\sup_{f\in\mathcal{F}_{\mathrm{KAN}}^{r}}\mathbb{E}\left[\|\tilde{f}_{n}-f\|_{L^{2}}^{2}\right]\geq\inf_{\tilde{f}_{n}}\sup_{h\in\mathcal{H}^{r}}\mathbb{E}\left[\|\tilde{f}_{n}-h\|_{L^{2}}^{2}\right]=O\!\left(n^{-\frac{2r}{2r+1}}\right).$ $$

Finally, we combine the bounds. We have shown an upper bound of $O\!\left(n^{-\frac{2r}{2r+1}}\right)$ using the KAN estimator and a lower bound of $O\!\left(n^{-\frac{2r}{2r+1}}\right)$ by considering a difficult subclass. Together, these results imply that the minimax rate is:

$$ $\inf_{\tilde{f}_{n}}\sup_{f\in\mathcal{F}_{\mathrm{KAN}}^{r}}\mathbb{E}\left[\|\tilde{f}_{n}-f\|_{L^{2}}^{2}\right]=O\!\left(n^{-\frac{2r}{2r+1}}\right).$ $$

This demonstrates that the KAN estimator is minimax optimal in terms of its convergence rate with respect to the sample size $n$.
∎

###### Corollary 1 (Minimax Optimality of KAN Estimators) .

###### Proof.

### A.4 Proof of Corollary 2

###### Corollary 2 (Optimal Knot Number for Univariate Spline Units) .

Let each B-spline unit $\psi_{qj}$ in a KAN approximate a function in the Sobolev space $\mathcal{H}^{r}([0,1])$. Then the optimal number of interior knots per unit that balances the bias–variance trade-off is

$$ $k\asymp n^{1/(2r+1)},$ $$

which yields the minimax optimal convergence rate

$$ $\mathbb{E}\|\hat{\psi}-\psi\|_{L^{2}}^{2}=O\!\left(n^{-\frac{2r}{2r+1}}\right).$ $$

for each univariate spline fit.

###### Proof.

This result follows from analyzing the bias-variance trade-off inherent in nonparametric spline regression. For a single univariate B-spline M-estimator $\hat{\psi}$ of a target function $\psi\in\mathcal{H}^{r}([0,1])$, the Mean Squared Error (MSE) can be decomposed into two primary components: squared bias and variance.

$$ $\mathbb{E}\|\hat{\psi}-\psi\|_{L^{2}}^{2}=\underbrace{\|\mathbb{E}[\hat{\psi}]-\psi\|_{L^{2}}^{2}}_{\text{Squared Bias (Approximation Error)}}+\underbrace{\mathbb{E}\|\hat{\psi}-\mathbb{E}[\hat{\psi}]\|_{L^{2}}^{2}}_{\text{Variance (Estimation Error)}}.$ $$

First, we characterize the error terms. The behavior of these two error components as a function of the number of knots, $k$, and the number of samples, $n$, is well-established in approximation theory and nonparametric statistics.

- •
Squared Bias: The bias arises from the inability of a spline with a finite number of knots to perfectly represent an arbitrary function in $\mathcal{H}^{r}$. The approximation power of a spline increases with the number of knots. The squared approximation error for a function in $\mathcal{H}^{r}$ using a spline with $k$ knots is known to decrease polynomially with $k$. Specifically, the squared bias is bounded as:
$\text{Bias}^{2}=O\!\left(k^{-2r}\right).$
- •
Variance: The variance arises from the statistical uncertainty in estimating the spline coefficients from the $n$ data samples. A larger number of knots means more parameters to estimate, which increases the variance of the estimator. The variance is proportional to the number of parameters ($k+m$, where $m$ is the spline order) and inversely proportional to the sample size $n$. For a fixed order $m$, this gives:
$\text{Variance}=O\!\left(\frac{k}{n}\right).$

Next, we find the optimal balance. The total MSE for the univariate estimator can be written as the sum of these two terms:

$$ $\mathbb{E}\|\hat{\psi}-\psi\|_{L^{2}}^{2}\approx C_{1}k^{-2r}+C_{2}\frac{k}{n},$ $$

for some constants $C_{1}$ and $C_{2}$. To find the number of knots $k$ that minimizes this expression, we seek to balance the two terms. This balance occurs when the terms are of the same order of magnitude:

$$ $k^{-2r}\asymp\frac{k}{n}.$ $$

Solving this relation for $k$ gives:

$$ $\displaystyle n$ $\displaystyle\asymp k^{2r+1}$ $\displaystyle k$ $\displaystyle\asymp n^{1/(2r+1)}.$ $$

This is the optimal scaling law for the number of knots. It dictates that as the sample size $n$ increases, the number of knots should also increase, but at a much slower, sub-linear rate determined by the smoothness $r$ of the function class.

Finally, we determine the resulting convergence rate. By substituting this optimal choice of $k$ back into the MSE expression, we can find the best achievable convergence rate. Both the squared bias and the variance will have the same order:

- •
Bias term: $k^{-2r}\asymp\left(n^{1/(2r+1)}\right)^{-2r}=n^{-\frac{2r}{2r+1}}$.
- •
Variance term: $\frac{k}{n}\asymp\frac{n^{1/(2r+1)}}{n}=n^{\frac{1}{2r+1}-1}=n^{-\frac{2r}{2r+1}}$.

Since both terms decay at the same rate, the overall MSE for the univariate estimator is:

$$ $\mathbb{E}\|\hat{\psi}-\psi\|_{L^{2}}^{2}=O\!\left(n^{-\frac{2r}{2r+1}}\right).$ $$

This rate is known to be the minimax optimal rate for estimating a univariate function with smoothness $r$. As established in Theorems 1 and 2, the overall KAN estimator inherits this rate, confirming that this choice of knot scaling is optimal for the entire model.
∎

###### Corollary 2 (Optimal Knot Number for Univariate Spline Units) .

###### Proof.

## Peer Reviews

confidence: 2
contribution: 3
initial_rating: 6
presentation: 3
questions: See the weakness.
soundness: 3
strengths: 1. This paper provides a complete theoretical treatment of KAN convergence rates, bridging the empirical success of KANs with rigorous nonparametric learning theory. This significantly advances the theoretical foundation of KAN research.

2. By parameterizing univariate KAN components using B-splines, the authors successfully unify KANs with the sieve estimation framework, showing that structured neural architectures can inherit the statistical optimality of spline estimators while maintaining interpretability.

3. The simulation experiments are well aligned with the theoretical analysis, showing empirical slopes near the theoretical exponent
summary: This paper establishes the theoretical convergence rate of Kolmogorov–Arnold Networks (KANs) in nonparametric regression. By representing the univariate components of KANs with B-splines, the authors connect KANs to classical sieve estimation theory. They prove that both additive and hybrid KANs achieve the minimax-optimal convergence rate under Sobolev smoothness assumptions. Experiments on synthetic data confirm that the empirical convergence behavior matches the theoretical rate.
weaknesses: 1. The theoretical results rely on restrictive assumptions, such as all univariate functions belonging to Sobolev spaces with known smoothness and bounded constants, which may limit their applicability to practical, high-dimensional learning settings.

2. The experimental validation focuses solely on smooth synthetic functions under low-noise settings. No results are provided for higher-dimensional, noisy, or real-world datasets, limiting the empirical generality of the conclusions.

3. The paper compares KANs only to standard MLPs. Including baselines such as DeepONet, SIREN, or kernel regression networks would more clearly demonstrate the practical advantages of KANs in terms of interpretability, stability, and efficiency.

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: I may have some misunderstand of the paper but please clarify the questions from Weaknesses.
soundness: 3
strengths: The major contribution of this paper is that for both additive and hybrid KAN estimators, they obtained the minimax-optimal mean-squared error rate under Sobolev smoothness assumptions. This places the models on a solid theoretical foundation within nonparametric statistics.

Moreover, using these results, the paper provides the guidance on the choice of the optimal number of spline knots $k \asymp n^{1/(2r+1)}$.
summary: The paper studies the statistical behavior of Kolmogorov–Arnold Networks (KANs) in the nonparametric regression. KANs are based on the Kolmogorov–Arnold representation theorem, which guarantees that any continuous multivariate function can be expressed as a sum of univariate functions applied to linear combinations of input variables.

The paper focuses on a setting of KANs where each univariate component is modeled using B-spline basis functions. The paper then places this architecture within the framework of classical nonparametric estimation and provides a complete analysis of its convergence properties under standard smoothness assumptions.

The main result shows that, under Sobolev smoothness of order r, both additive and hybrid (additive–multiplicative) KAN models achieve the same minimax-optimal mean-squared convergence rate as traditional spline or kernel estimators. The paper also offers a practical rule for choosing the number of spline knots and validates the theoretical predictions with some numerical experiments.
weaknesses: The main theorems establish the classical minimax rate for Sobolev regression functions. While they considered Kolmogorov–Arnold Networks (KANs) in the nonparametric regression, they only considered additive and hybrid (additive–multiplicative) KAN models, see Equation (11) in the paper. Using B-spline basis functions to obtain minimax optimal regression estimators for Sobolev regression functions is standard and has been studied in many existing papers. Due to this simple form of the model in Equation (11), it is not clear if this is a significant contribution to the statistical and machine learning community at ICLR.

Moreover, the paper [1] studied minimax optimality of a more complex model, the deep composition of anisotropic Besov functions, which seems to cover the setting of this paper, at least, for the theoretical contributions and using B-splines. I think the theoretical contributions of this paper then may not be adequate. Also, in recent regression literature such as [1,2,3], adaptivity to unknown smoothness parameters are usually studied. While this paper provided a suggestion of picking the optimal number of spline knots, it relies on the unknown parameter r.

Overall, I think this paper appears to be a subset of [1], or has a lot of overlaps with [1], making its theoretical contributions unclear. Its contributions seem standard and combinatorial.

[1] Suzuki, Taiji, and Atsushi Nitanda. "Deep learning is adaptive to intrinsic dimensionality of model smoothness in anisotropic Besov space." Advances in Neural Information Processing Systems 34 (2021): 3609-3621.

[2] Shi, Zhaoyang, Krishna Balasubramanian, and Wolfgang Polonik. "Adaptive and non-adaptive minimax rates for weighted Laplacian-Eigenmap based nonparametric regression." International Conference on Artificial Intelligence and Statistics. PMLR, 2024.

[3] Chazal, Frédéric, Ilaria Giulini, and Bertrand Michel. "Data driven estimation of Laplace-Beltrami operator." Advances in Neural Information Processing Systems 29 (2016).

---

confidence: 4
contribution: 2
initial_rating: 2
presentation: 3
questions: Could the authors also please clarify what algorithms are used for training their KANs in the experimental results section? If these algorithms differ from the estimator which is theoretically analyzed, please indicate this.

If these issues about the precise definition of the estimator are addressed, I would be willing to raise my score.
soundness: 2
strengths: Function estimation using KANs is becoming increasingly popular, so the problem addressed is of significant importance. The paper is also well-presented and written.
summary: This paper considers estimating functions from a certain model class using KANs. The model class is closely related to the KAN architecture and essentially consists of functions which can be expressed using KANs with a certain amount of smoothness. The resulting rates are shown to be minimax optimal, and numerical experiments are performed. I think that the paper has quite an interesting potential idea, however, it seems to me that the authors have not described the estimator that they are analyzing. This issue would need to be addressed for the results to make sense.
weaknesses: There are appear to be some significant technical details missing. Specifically, in Theorem 1 what exactly is the estimator $\hat{f}$? You say that $\hat{g}\_q$ and $\hat{\psi}\_{qj}$ are constructed via least squares with back-fitting. What does this mean, exactly? I can't find an explanation elsewhere in the paper. Note that only the composite function $f$ is sampled, you do not have access to the true $g_q$ and $\psi_{qj}$ to do regression on. This point confused me a bit, could the authors please be explicit about what estimator they are studying.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: * Theorems 1 and 2 seemed to assume two-layer KANs. Can this result be extended to deeper KANs?
* In Corollary 2, the optimal number of internal knots is derived. For smooth functions (say $f(x) = {\rm sin}(x)$) when $r\to\infty$, how should one choose the optimal $k$?
* In abstrac, $n$ is used without defnition.
* a nitpick, since $k$ is reserved for the optimal knot number, it's better to use a different subscript letter in $a_k$ (line 402).
soundness: 3
strengths: * The paper is well motivated and nicely written
* The paper presents a theoretical contribution on the convergence rate of KANs
* Theoretical claims are supported by numerical experiments
summary: This paper presents a theoretical framework for analyzing the convergence rate of Kolmogorov-Arnold Networks on Sobolev spaces of smoothness r. The scaling exponent $2r/(2r+1)$ highlights that KANs do not suffer from curse of dimensionality because the scaling exponent for MLPs is $2r/(2r+d)$ where $d$ is the input dimensionality. The authors further derive how to optimally choose grid size k and support their conclusions with numerical experiments. These results provide a theoretical foundation for using KANs in non-parametric regression.
weaknesses: * Generalization of the result to deeper KANs. Theorem 1 and 2 seemed to assume two-layer KANs. Can this result be extended to deeper KANs?
* The optimal number of knots $k$ is not supported by numerical experiments.


Decision: Reject