A Graph Meta-Network for Learning on Kolmogorov–Arnold Networks
Authors: Guy Bar-Shalom
Ami Tavory
Itay Evron
Maya Bechler-Speicher
Ido Guy
Haggai Maron
Venue: ICLR 2026 Poster (2026)
Keywords: Weight Space
Kolmogorov Arnold Networks
Graph Neural Networks
Symmetries
Equivariance
Primary area: learning on graphs and other geometries & topologies
TL;DR: We introduce WS-KAN the first weight-space model for KANs, respecting their permutation symmetries via a graph representation. It achieves strong expressiveness and consistently outperforms baselines.

Abstract: Weight-space models learn directly from the parameters of neural networks, enabling tasks such as predicting their accuracy on new datasets.  Naive methods -- like applying MLPs to flattened parameters -- perform poorly, making the design of better weight-space architectures a central challenge. While prior work leveraged permutation symmetries in standard networks to guide such designs, no analogous analysis or tailored architecture yet exists for Kolmogorov–Arnold Networks (KANs). In this work, we show that KANs share the same permutation symmetries as MLPs, and propose the KAN-graph, a graph representation of their computation. 
Building on this, we develop WS-KAN, the first weight-space architecture that learns on KANs, which naturally accounts for their symmetry. We analyze WS-KAN’s expressive power, showing it can replicate an input KAN’s forward pass - a standard approach for assessing expressiveness in weight-space architectures. We construct a comprehensive ``zoo'' of trained KANs spanning diverse tasks, which we use as benchmarks to empirically evaluate WS-KAN. Across all tasks, WS-KAN consistently outperforms structure-agnostic baselines, often by a substantial margin.

Published as a conference paper at ICLR 2026
A GRAPH META-NETWORK FOR LEARNING ON KOLMOGOROV-ARNOLD NETWORKS
Guy Bar-Shalom"
Ami Tavory
Itay Evron Meta
Maya Bechler-Speicher Meta
Technion, Meta
Meta
Ido Guy
Haggai Maron
Meta, Ben-Gurion University of the Negev
Technion
ABSTRACT
Weight-space models learn directly from the parameters of neural networks, en- abling tasks such as predicting their accuracy on new datasets. Naive methods like applying MLPs to flattened parameters - -eerform poorly, making the design of better weight-space architectures a central challenge. While prior work leveraged permutation symmetries in standard networks to guide such designs, no analo- gous analysis tailored architecture yet exists for Kolmogorov-Amold Networks (KANs). In this work, we show that KANs share the same permutation symme- tries MLPs, and propose the KAN- LuN-graph, graph representation of their compu- tation. Building on this, we develop WS- SS-KAN, AN, the first weight-space architecture that learns KANs, which naturally accounts for their symmetry. We analyze WS-KAN's expressive power, showing it can replicate an input KAN's forward pass a standard approach for assessing expressiveness weight-space architec- tures. We construct a comprehensive "2oo" of trained KANs spanning diverse tasks, which we use as benchmarks empirically evaluate WS-KAN Across all tasks. WS-KAN consistently outperforms structure-agmostic baselines, often by a substantial margin. Our code is available hutps//jittubccom//aSSCCuKKKKK Giraph-Metanetwork
INTRODUCTION
Deep neural networks are now powerful tools for prediction, generation, and beyond. A recent perspective (Navon et al. 2023 Zhang et al.] 2023) views their parameters not merely as weights, but as data - enabling the design of weight-space models, which are networks that operate directly on the parameters of other networks. This shift makes it possible to predict test accuracy (Eilertsen) cct al. 2020, Unterthiner et al. 2020), generate new sets of weights (Erkoç et al. (023), and classify synthesize Implicit Neural Representations (INRs; Mescheder et al. 2019 Sitzmann et al. 2020) all through a single forward pass.
A straightforward way to design weight-space (WS) models is to flatten all weights and biases into a single feature vector for prediction. However, this overlooks neuron permutation symmetries (Hecht-Nielsen] 1990 Brea al., 2019), i.e., parameter transformations that keep the underlying function computed by the neural network unchanged. Thus, such naive models may produce differ- ent predictions for equivalent reorderings. While pioncering approaches used generic architectures to process model parameters (Schürholt et al.) b(e2c(ble). more recent developments incorporate those networks' symmetries. either through weight sharing linear layers (Navon al.) [003]] Zhou et al. 2023) (via geometric deep learning principles; Bronstein et al. 2021). or by treating networks as graphs (Lim et al. 20244) Kalogeropoulos et al.) 2024|]/anp et al. 2023 Kofinas al.,
2024) and applying Graph Neural Networks (GNNs; Scarselli et al. 2008 Kipi 2016, Gilmer et al.
2017 which naturally respect these symmetries.
Previous work on WS learning has only begun to explore the potential of characterizing and ap- plying WS models across different architectures. Initially, research has concentrated on MLPs with
TThis work completed during an internship Meta.

---

Published as a conference paper at ICLR 2026
Layer I-1
Layer
Layer ++1
In a KAN, neuron of layer is computed as xp for some parametric function 0//--
Aba (x
We construct K4N-graph by setting its edge features as:
up
cgg &
pudt
where &ha is a vector representation of &
A 1
Figure 1: Constructing the KAN-graph for given Kolmogorov- Network (KAN).
straightforward extensions to CNNs NNavnn et al. |023 Zhou et al. 2023 Recent developments have expanded to transformer architectures (Tran et al. 2024) and architectures incorporating tensor symmetry structures (Zhou et al., (024). However, designing WS models for diverse architectural paradigms remains an important open challenge that warrants further investigation.
In this work, we introduce the first WS model specifically designed to process emerging class of neural networks: Kolmogorov-Amool Networks (KANs; Liu et al. 2025). Why design weight-space model thal takes KANs as imput? KANs represent fundamentally different neu- ral paradigm- rathhrrrrrtt than employing scalar weight matrices with fixed nonlinear activations, they construct networks from matrices of learnable univariate functions, while preserving the universal approximation properties of conventional neural networks (Kolmogoroy 1954; Braun & Griebel, 2009). This architectural shift yields compelling advantages over standard MLPs, including supe- rior parameter efficiency (Koenig et al. (2225,, accelerated neural scaling (i.e., performance scales faster than expected as model size increases) (Liu et al. 2025), and notably, enhanced interpretability (Barašin et al. 2024). The learnable functions that replace scalar weights can be directly visualized and analyzed, offering unprecedented insight into the network's decision-makinn makinn process.
As KANs gain traction within the deep learning community, we anticipate proliferation of trained KAN models across diverse applications. WS learning will become increasingly valuable for help- ing practitioners understand, compare, and leverage these models effectively. However, developing weight-space models for KANs presents several challenges. The network architecture differs fun- damentally from those previously studied in the weight-space literature, its learnable components are functions rather than simple sealar parameters. Additionally, the structural understanding from symmetry perspectives that have been developed for traditional neural networks remains largely unexplored for KANs.
Our contributions. We begin by showing that KANs also exhibit permutation symmetries - in fact, the same as conventional neural networks. Building on this insight, we introduce the KAN-graph, novel attributed graph with edge features that compactly encodes the structure of given KAN; see Figure On top of this representation, we develop WS- KAN, a GNN-based architecture capa- ble of learning directly over the KAN-graph. We show that simulate the forward pass of the corresponding KAN, validating our approach from a theoretical per- spective, and laying the ground for stronger results such as functional approximation theorems. To validate our approach experimentally, we construct the first "model zoo" of pre-traines KANs across diverse tasks, together with their corresponding KAN-graph representations serving as a benchmark. We show that WS- KAN consistently outperforms both generic baselines (such as MLPs over flat- tened parameters) and more sophisticated architectures, which effectively act as ablation studies and further validate our architectural design from the perspective of the symmetry analysis. Furthermore, because WS-KAN operates directly on graph representations through a GNN-based architecture, can be seamlessly applied KAN architectures of varying sizes, including those not encountered during training. We empirically demonstrate promising results in these settings.
applied KAN-graph can
BACKGROUND AND RELATED WORK
Equivariance and invariance in deep learning. Many learning tasks involve functions that ei- ther remain unchanged (invariant) transform in predictable manner (cquivariant) under specific symmetries of the input. A classic example is the translation invariance of Convolutional Neural Networks (CNN: Krizhevsky al. 2012), where shifting image does not alter its label. While
2

---

Published as a conference paper at ICLR 2026
simple MLPs can, in principle, learn such symmetries from data, this approach is often inefficient. By explicitly incorporating the symmetry into the model architecture [ohen et al. (2018): Maron et 2020); Zaheer al. (2017); RRaanbakkss et al. (2017), , the property becomes inherent fea-
ture of the model rather than something that must be inferred during training. This typically leads to
better generalization and data efficiency (Cohen & Welling 2016 Brehmer al. (024). Building
on these principles, a new class of models has recently emerged, referred to as weight-space models.
weight-space models. The weight-space of a neural network refers to the collection of parameters that fully define its architecture. For a multilayer perceptron (MLP), this is the set of weights and (W1, bi..... ... L. bL). weight-space models are approaches that op- erate directly on this parameter space. While pioneering approaches proposed standard architectures for learning in WS (Schürholt et al., 2021 2022d|a). more recent works (Navon et al., 2023) Zhou et al., 2023) have focused on the symmetries inherent in neural networks FHecht-Nielsse 1990,
biases across all layers,
Brea al. 2019), leading to tailored architectures that explicitly respect these symmetries. Several strategies have emerged: for instance, Navon al. (2023); Zhou et al. (2023) enforce weight sharing in linear layers, while Graph Meta-Networks Kaalgeropoulos et al. [2224] Kofinas et al. 20244] Lim
et al. 2024) leverage graph neural networks to operate directly model's 's computational graph. In particular, Lim et al. (2024) has shown that those neural network symmetries correspond to graph automorphisms of their computational graphs.
Kolmogoroy-Arnold Networks (KANs). KANs are a recently introduced class of neural networks in which edges of the computational graph. rather than carrying simple numerical weights, abstractly represent univariate functions (Liu et al. 2025). Formally, given an input E IRd, an L-layer KAN defines a function f as follows:
di
f(x) =x", where 2
Σ (
x ===0
(I)
where each Φi is matrix univariate functions of size di di-1. That is, every entry a function o' IR + R. Conveniently, and analogous to MLPs, the composition of such layers is defined:
f(x) = (Øt ....o Φt )x,
(2)
where the operator e denotes the application of layer to its input. To clarify the notation, for given layer we define (o' (x) ΣØー (=)-))
Several parameterizations of these ID functions are possible (Bozorgasl & Chen 2024 Zhang al,) 2025). In this work, consistent with the original KAN paper, we adopt a Eespline-based parametriza- tion (Schoenberg 1946), denoted (xx), which represents smooth piecewise polynomial functions over domain. For more detailed review of B-splines, see App. Specifically, we define the ID function (-) composing KANs as follows,
(x) = wg b(x) + W. B(x);
B(x) = B(x) = Σ C,,,(,,,
(3)
where Wh, w, learnable parameters, b(x) silu(x) 1te-x= and Bz(z) pre-defined B- spline basis functions with c, as the learnable coefficients.
3 LEARNING ON KAN PARAMETER SPACES
Overview. We begin by analyzing the structure of KANs' parameters and demonstrate that per- muting hidden neurons does not alter the underlying function, mirroring the behavior observed in MLPs. Inspired by prior work on weight-space models for MLPs that introduced graph representa- tions (Lim et al. 2023. Koñnas al. 2024), our approach can be presented as follows: we repre- sent the input KAN as graph, where nodes correspond to individual neurons and edges represent the connections between them. The learned one-dimensional functions of the KAN are used to define the edge features (details follow). We refer to this construction as the KAN-graph; see Fig- ure Importantly, the permutation symmetries those that leave the underlying KAN function unchanged- correspond to permutations of hidden neurons within the KAN- CAN-graph, which likewise leave the graph itself unchanged. Thus, we employ a graph neural sewwork-basee technique to pro- cess the KAN-graph, leveraging their inherent equivariance node permutations.
3

---

Published as a conference paper at ICLR 2026
In what follows, we (1) demonstrate that the permutation symmetries present in MLPs also hold in KANs: (2) present a method for converting KANs into KAN-graphs; and (3) introduce WS-KAN a GNN-based architecture for processing KAN- KAN-graphs and analyze its expressive power.
3.1 PERMUTATION SYMMETRIES IN KANS
In a seminal work, Hecht-Nielsen (1900) observed that MLPs exhibit permutation symmetries: re- ordering the neurons within any hidden layer leaves the represented function unchanged. In this subsection, we make the observation that the same symmetry also holds for KANs.
A KAN is fully specified by the collection of one-dimensional functions assigned to each in- put-output pair in every layer. For convenience, we denote the functions in an I-layer KAN by [E()) (where [4] == (I,,)...,LJ). Given permutation matrices P1 and Ps, define their action on a matrix of univariate functions & as,
(4) where 01 and 02 are the permutations associated with Pi and Pz, respectively. Intuitively, this corresponds to reordering the rows and columns of Φ according to the given permutations. Below, we formally state the permutation symmetries of KANs.
P (
Proposition 3.1 (KAN symmetries). Let 0 = (o'..... ..) denote the collection of parametric one-dimensional functions composing an L-layer KAN. Consider the group. G == Sá, x Sa, x x Sd the direct product of symmetric groups corresponding to the intermediate dimensions d1,..., ,,,,,, Let g (PI..... ........ Pi-1)) ,1....,PP-1... E G. where each P. the permutation matrix of or E Sa Define the group action 0 = 0' with 0' 0/=(0 =(d" ....,,"') given by.
8² PiT TT Φ" = Pl Φ P1-I. =1 2....... L 1- L-1,
Φ'L = =8''PI---
Then, fo(x)) for (a) for all x.
short proof is given in App. for the case of a single hidden-layer KAN, and the extension to multiple hidden layers follows naturally.
Intuitively, such permutations corre- spond to reordering nodes within hid- den layers. As an example, Fig- ure iilustrarss single-hidden-layer KAN, where the permutation matrix P corresponds
d
Ma
(1,3). In other words, the first and third nodes are mapped one another, while the sec- ond remains unchanged. Importantly, Figure 2: Hidden neuron permutation symmetries in KANs.
(Φ2 6 )x
these permutation symmetries hold independently of the chosen parametrization the ID functions.
In the next subsection, introduce the KAN-graph, graph representation of KANs that (i) com- pactly encodes their structure, and (ii) remains invariant under neuron permutations that do not alter the function computed by the KAN.
3.2 KAN-GRAPH
The main idea behind the definition of the KAN-graph relatively natural: as illustrated in Figure nodes represent the KAN's neurons and edges should represent the univariate functions. In this subsection, we formalize the construction by explicitly defining the nodes, edges, and edge features derived from those univariate functions.
We define the KAN-graph as a directed graph G = (V, F)) where V represents the set of neurons, E represents the set of edges, and E Rexdy, E mgxx are their corresponding features, respectively. Here, n, denote the total number of nodes and edges in the graph, re- spectively, To clarify, in Figure 2iefft), we have π = nodes, m 12 edges, and the corresponding adjacency matrix visualized inset!) !
ola ela
45.
'The adjacency matrix of given KAN-graph is extremely sparse: nonzeros appear only in the first super- diagonal blocks, specifically between blocks and I + for each E [Z.].
4

---

Published as a conference paper at ICLR 2026
Univariate functions edge features in the KAN-graph. To fully define the KAN-graph, must specify how to incorporate the explicit parametrization of [ [e[]]] characterizing the KAN into the KAN-graph. In this work, we focus the parametrization introduced in the original KAN paper (Liu et al. 2025). wherein the learnable univariate functions are based on B-splines; as per Eq.
In this case, the learnable parameters composing those univariate functions be conveniently "collected" into vector &y.g buppp.q. Wssp..' %..gl. Thus, we define, for layer I, input node (or neuron) p. and an output node q. the following edge feature,
e/4 pp.4 = [wpp. W S.p.l.
(5)
which elegantly collects the learnable parameters of the one-dimensional function o%..g
3.3 LEARNING ON THE KAN-graph
To learn on the KAN-graph, we adopt general message-passinng framework motivated by Gilmer et al. (017). as follows (the letters a-d denote the order of execution),
a) of. 4 (L. (2:F)
U/ Σ MMPP! e(:))) 6) o MLP B)
US. Σ MLP!!: ((d)))
eE(EEEE
ee(DEEET
c) ((.)) f ML, By ((,)):
d) v. MLB (u, 0/ ".
Intuitively, node features are updated by aggregating information from both their outgoing and in- coming neighbors. Edge features, in turn, are refined based on the states of their endpoints as well as their own current representation. Each node's representation is then updated by combining its in- trinsic features with the forward- and bockward-aggregae computational graph of KANs is inherently directed, with edges pointing from one layer to the next, explicitly perform bidirectional message passing propagating information not only forward but also reverse as this dual flow was found to enhance performance.
information. We note that although the
Positional encodings (PE) additional edge and node features. We follow prior work (Kofinas et al.) 2024: Lim et al., (024) and augment each node and edge of the KAN- -AN-gapph with positional em- beddings that indicate their position in the computation flow. This breaks potential artificial symmetries that may arise in the KAN- graph. Specifically, all nodes within the same intermediate layer share common positional embedding. By contrast, input and output nodes are assigned distinct embeddings, since permutations of these nodes generally alter the network's function. For edges, we assign a unique identifier to each one, associated with its input and output nodes.
Figure 3: PE.
Figure deppics possible assignment positional encoding (via simple integers) to both nodes and edges for the running example of Fleure Intuitively, permuting neurons within a hidden layer leaves the nntture-augmente graph (i.e., its adjacency matrix) unchanged. In contrast, permuting a node from the first layer with one from the last does alter the graph.
EXPRESSIVE POWER OF WS-KAN
While there are multiple ways to design WS architecture for processing KANs, we adopted one specific approach, which stems from equivariance principles, and models the KAN as graph. How- ever, a question arises: this the right design choice?
In general, imposing group-equivariance constraints might reduce model's expressive power (Maron et al. 2019 Xu et al.) 2019 Morris et al. 2019). the weight-space literature, this ex- pressive power is often analyzed by demonstrating that the architecture can simulate (i.e., approxi- mate) the forward pass of a given input model. For example, Lim et al. (2024); Navon et al. (2023) establish such results in the context of processing standard neural networks. Such approximation results typically serve as an intermediate step toward proving stronger results, such as functional approximation theorems (Navon et al. 2023).
5

---

Published as a conference paper at ICLR 2026
In this section, we show that WS- KAN can simulate forward pass for given input KAN. We begin by showing that there exists a set of weights for singlehiddeen MLP that can approximate any univariate function based on B B-splines (under mild assumptions), per Eq. (3), to arbitrary precision. We then use this result to prove our main result: WS-KAN can simulate the forward pass of the original KAN. All proofs are provided in App.
Lemma 4.1 (MLP
approximation of the univariate functions composing the KAN). Let B denote the family of cardinal B-splines of degree k, defined on a fixed grid G over the domain = |co, ................................ [(+th1))) E C C IRG where the set of admissible coefficients lies in a compact domain C. Consider the function va R + R defined as W. b(x) denotes the silo function, and (xx) denotes the B-spline. Then, for any E > O, and for any compact domain X there exists set of weights for a multilayer perceptron MLP :IR GRG--- - R such that,
(a, b]. and parameterized by coefficients
(xx)) wh b(x) + W, B(x), where wb, W, W C R for some compact
sup=ex MMLP(x, Way Wb, c) - {b(x)] <.
Lemma 4.1 essentially states that, under the assumptions above, there exists MLP that effectively computes the univariate functions composing our input KANs, over any chosen compact domain X. Consequently, we show that under mild conditions, WS-KAN can approximate forward pass of the input KAN model.
Proposition 4.2 (WS-KAN can simulate the forward pass KANs). Let fa be a given KAN ar- chitecture, defined over an input domain [a. b]", where each univariate function is represented by B-spinn from the family B. Let G be its KAN-graph, where the nodes in the first layer are enhanced with the input x E [a, bl". For every 0, there exists WS- WSSKAN such that,
suPaeta.0 WS-KAN(G) fa <E.
Importantly, Proposition holds for any choice of parameters defining the KANs' univariate func- tions (Eq. 3 Below we present the proof idea, while the full proof can be found in App. E
Proof idea. A KAN computation can be viewed as a composition of L. continuous functions (lay- ers). Using Lemma 4.1 we show that WS-KAN can approximate each individual layer to arbitrary precision. Then, by applying standard techniques (c.g., ideas from Lemma 6 of Lim al. 2022) and the message passing mechanism, we construct approximations layer by layer, ensuring that the overall WS-KAN output remains arbitrarily close to that of the original KAN.
EXPERIMENTS
While various "model zoos" (e.g.. Schürbolt et al. 2022eb exist for benchmarking weight-space models in conventional neural networks, no comparable resource has yet been developed for KANs. Nor are there established baselines against which WS-KAN can be evaluated. To address this gap, we take inspiration from tasks and baselines explored in the weight-space literature for con- ventional neural networks and construct several families of model zoos of trained KANs. These zoos are designed to capture both invariant and equivariant tasks and are built from five datasets: MNIST (LeCun et al. 1998), Fashion-MNIS: (K IK-MNIST; et al. 2018). CIFAR 10 (Krzhevssy et al. 2009) and a synthetic dataset that we designed inspired by the one in Navon et al. (2023).
(F-MNIST; Xiao et 2017). Kuzushij-MNISST
We focus on two invariant problems INR classification (Section 5. and accuracy prediction (Sec- tion [5.2 and one equivariant task pruning. where the goal is predict pruning mask directly from the KAN's weights (Section 5.3 We also evaluate WS-KAN's ability to generalize to KAN architectures unseen during training (Section T. For each model in the zoo, we also construct the corresponding KAN-graphe We benchmark WS- KAN against the following baselines.
Standard baselines: (1) MLP: A simple multilayer perceptron applied to a vectorized representation of the KAN's parameters. (2) MLP + Aug.: The same MLP as above, but trained with permuta- tion augmentation, i.c., randomly permuting the input KAN in ways that preserve its underlying function. (3) MIP + Align: Inspired by alignment techniques for MLPs (Ainsworth et al. 2022), where parameters are reordered to maximize model similarity, we extend this idea to KANs. The main challenge is that KAN parameters functions rather than scalars, and therefore alignment

---

Published as a conference paper at ICLR 2026
requires defining distances between functions. Full details are in App. D We consider baseline an MLP applied to aligned KANs. (4) DMC (Eilertsen et al. 2020 convolutional layer applied to the (vectorized) model parameters. Ablation baselines: We introduce two additional basclines to ablate our architectural design choice. (5) DS (DeepSets): A DeepSets (Zaheer et al. 2017) ar- chitecture applied the graph's s edge features. Importantly, while this baseline is invariant KAN permutation symmetries, it is also invariant to many more permutations that do not correspond to these symmetries, and completely neglects the graph topology. (6) SetTrans: Similar (5), but employing transformer architecture (Vaswani et al. 2017) over the set of edges. We note that this baseline is only feasible for relatively small imput KANs, since the attention matrix scales quadrat- ically with the number of neurons Thus, SetTrans is reported only when feasible. Additionally, we ablate key design choices in WS-KAN, including positional encoding and bidirectional message passing: see App. C.4 for details.
In the following sections, we provide additional details on the tasks considered, describe the con- struction of the KAN training datasets, and present our results. For each experiment, report mean performance over three seeds, with error bars for standard deviation. All test results were obtained by optimizing for validation performance. Additional experimental details, including the hyperparameter grid, implementation notes, and extended results, are available in App.
5.1 INR CLASSIFICATION
First, we evaluate WS-KAN on INR (Mescheder et al. 2019; Sitzmann et al. 2020) classification.
What is INR? For given image, an INR learns a mapping from any input coordinate to the corre- sponding grayscale (or RGB) value of that coordi- nate in the image. See inset (top). At the bottom inset, we show example reconstructions produced by KAN-based INRs over CIFAR 10, F-MNIST, MNIST left corresponds to ground truth image, and right corresponds the reconstructed one.
S
Leaming atgonthim of oiclaane
Setup and dataset construction. The setup is il- lustrated inset. For this task, we convert the follow- ing datasets to KAN-based INRs: Sine waves (a syn- thetic dataset), MNIST, F-MNIST, and CIFAR10. Taking MNIST an example, the key idea is that for each image in the dataset, we train an independent KAN-based INR 'reconstruct' Once this zoo of INRs is constructed, we train the weight-space model under study (e.g., WS-KAN) to classify the digit, using as input the parameters of the KAN-based INR (or the KAN-graph when WS- VS-KAN is tested) rather than the raw pixel data. See App. C.I for dataset split and details.
S
Weight-eppse architecture ie our WS-HAN)
cannila
8
cillieent imsgies prodhct original
Results. Table [ reports the accuracy of pre- dicting the class (c.g.. the digit in MNIST) from the weights of the KAN-based INR. WS- KAN outperforms all baselines by large mar- gin. SerTrans ranks second, suggesting that ex- plicitly accounting for symmetries is advanta- geous, albeit suboptimal, as the symmetries it captures are broader than the KAN permutation symmetries. Finally, we observe that "MLP + Align. > MLP Aug. > MLPP", aligning with intuition and validating the effectiveness of our alignment technique. Results over the synthetic dataset are provided in App. C.I.3
Table 1: INR classification accuracy.
Method
MNIST F-MNIST CIFAR-10
MLP
34.110.1
41.3103
16.810.1
MIP + Aug.
62.741.1
63.040.
28.2
MLP + Align.
81.0+0.1
73.6#:0.
30,0+0.2
DMC
73.44:1.
73.1+10
33.0+0.7
DS (Ours)
59.143.3
65.9108
23,24339
SetTrans (Ours)
87.5
80.2:10.1
34.3
WS-KAN (Ours)
94.3
84.6110.6
42,2-088
II practice, found this approach computationally expensive-trainee single epoch could take up one hour some sxperiments-makkee significantly slower than WS-KAN and other baselines. OOr code, including model zoo construction, is available at https://ghthhbboomm///SSn/////n Metanetwork
7

---

Published as a conference paper at ICLR 2026
5.1.1 OUT-OF-DISTRIBUTION GENERALIZATION TO WIDER KANS
Since WS-KAN operates on KAN-grapha via GNN-based architecture, it can naturally be ap- plied to KAN-graphs of varying topologies, e.g. differing in hidden-layer width or number of lay- ers. Hence, here we evaluate WS- AA''s ability to generalize to KAN architectures larger than those seen during training. focusing on the INR classification task over the MNIST and F-MNIST datasets. Our KAN-based INRs from Section 5.1 share the topology [2 + - h + 10), where h denotes the hidden-layer width, with h 32. Here, we take those WS-KAN's trained on the 32 KANs and evaluate on progressively wider KAN architectures with h E (48, 64, 80, 96), none of which were during training.
Results. Table repoort the out-of-distribution (OOD) accuracy of WS-KAN. We observe promis- ing OOD generalization, which, expected, degrades increases and the distribution shift from the training setting (h- = 32) grows.
Table 2: OOD generalization for INR classification. Test accuracies of WS-KAN the INR classification task when trained on KANs with hidden width h = 32 and evaluated on wider, previously unseen architectures (h E f48, 64, 80, 96)) All architectures follow the topology [2 - + h 10). The blue column denotes the in-distribution setting, while
red columns denote out-of- setting.
Dataset
h 32
h = 48
h 64
h 80
h 96
MNIST
94.3 + 0.5
91.4 + 0.5
81.0 3.2
67.0 4.3
57.1 + 6.1
F-MNIST
84.6 + 0.6
84.6 0.6
84.3 + 0.7
83.3 = 0.8
82.2 + 0.7
5.2 ACCURACY PREDICTION
Here, we consider the task of predicting the accuracy of given KAN, based on its parameters.
Setup and dataset construction. We experiment on MNIST, F-MNIST, and K-MNIST. Over each dataset, we train 4000 KAN models with 3000/500/500 train/validation/test split. We observed that dif- ferent KANs trained on these datasets yield similar accuracies. Thus, to make predicting accuracy from parameters more chal- lenging, we introduce label noise: for each KAN, we randomly sample portions of the training data and shuffle their labels. As shown inset, this results a diverse set of trained KANs with
Test accuracy
20
varying test accuracies (results for other datasets provided in
0
Figure 9 in App.[Cr C). The training pipeline is illustrated in Fig-
60 Noisy labels (%)
10 App.
Results. To test how well WS-KAN predicts the test accuracies, we follow Lim et al. (2024) and report metries: Mean Squared Error (MSE) and R-squared (RZ²). The results, summarized in Table 3 exhibit the same trend as in INR classification: WS-KAN consistently achieves the best performance across all datasets. The ordering of baselines remains unchanged, with our ablation
baseline (DS) ranking second, followed by MLP + Alignment as the next most effective approach.
Table 3: Accuracy prediction. Comparison of MSE and R² across datasets.
MSE (lower better; 4) [x 1031
R² (higher is better; 1) [x10²]
Method
MNIST
F-MNIST
K-MNIST
MNIST
F-MNIST
K-MNIST
MLP
14.58++0.02
11.55
6.68:10.16
76.99
80.13:1049 69.59
MLP + Aug-
10.41:10.24
8.86
5.33
83.56.0.38
76.66
84.15
MLP + Align.
5.26
6.32-1.15
3.33:1.10
91.70=0.15
83.3740.41
90.1 10:.11
DMC
7.00++0.06
6.4640.32
3.2740.05
88.95=0009
82,9940.85
90.29+0.14
DS (Ours)
3.29110.12
3.90
2.00.10.11
94.81 10.18
89.73
94.07710.2
WS-KAN (Ours)
3.29.1..1
2.94
1.45.0..8
94.81 1.0.27
92.2710..5
95.6
8

---

Published as a conference paper at ICLR 2026
5.3 PRUNING MASK PREDICTION
In this section, tackle the challenging equivariant task of network pruming for KANs, aiming to discard a subset of weights without significantly degrading performance.
Motivation. Most pruning methods data-driven, requiring large amounts of data to determine which parameters to remove. For instance, activation- approaches rely on recorded activation values, while gradient- methods depend training loss gradients. In contrast, pruning model using only a simple forward pass, as enabled by WS-KAN (demonstrated below), is especially valuable, as it avoids repeated, data-intensive passes.
Setup. Here, we use the same datasets as Section 5.2 Supervision is obtained by applying a data-driven edge-based pruning algorithm for KAN which we denote as Oracle-prune. The core idea behind this algorithm is straightforward: it removes edges whose average activation values, computed from training data, fall below a predefined threshold (set to 0.01 in all experiments). We refer to this pruning method as Oracle-pruning, as it is our supervision. The oracle pruning algorithm outputs a binary mask, where edges marked with are pruned and those marked with are retained. The task of interest is to predict this mask, see Figure [1 in App. C.3 for the training pipeline. Crucially, this task is equivariant: prediction is made for each individual edge of the KAN, rather than single prediction for the entire network. We are interested in evaluating two aspects: (i) how accurately WS- KAN predicts the pruning mask, and (ii) whether using the mask generated by WS-KAN leads to effective downstream pruning performance.
Table 4: Pruning mask prediction.
Accuracy (t, %)
ROC-AUC (1, %)
Method
MNIST
F-MNIST
K-MNIST
MNIST
F-MNIST
K-MNIST
MLP
93.10=c0000
96.65++<0.01
91.39=<0.01
87.1240004
84,92+0038
75.32=0.02
MLP + Aug-
93.29=c0.01
96.65+xc0001
91.39=c0.01
91.36:10.12
86.21+0009
74.89+0.01
MIP + Align.
93.57=0001
96.641<00001
91.52400.1
93.00-10.03
91.66110.10
82.76
DMC
93.07
96.59+=c0.01
91.39=<0.01
84.27:10.66
84.394.0.01
75.06
DS (Ours)
94.34+00.2
96.90=0.09
94.34++0002
95.45-+0.05
95,81+10.04
95.45
WS-KAN (Ours)
97.93:10.19
98.93
97.72.10144
99.54.10.11
99.72::0.02
99.46=0009
Results (i). We evaluate mask prediction as a binary classification task using the metrics ROC-AUC and Accuracy. Results are provided in Table 4 WS-KAN consistently outperforms all baselines across all datasets and evaluation metrics. Again, the DS approach emerges as the second best over- all. Notably, the hierarchy among the MLP variants mirrors our earlier findings, further supporting the effectiveness of our alignment strategy.
Results for downstream pruning performance (ii). To properly evaluate downstream pruning, it is not enough to only report the downstream accuracy achieved by applying a mask. A trivial mask that leaves all weights untouched would naturally yield high accuracy, but would provide no practical benefit. A meaningful mask must therefore strike a balance: it should maintain strong downstream accuracy while also inducing sparsity the model.
We now evaluate how well WS-KAN achieves this trade-off, and present two complementary plots. In Figure we report the downstream accuracy (y-axis) achieved by pruned models across different noise levels in the training labels (x-axis, binned in 20% non-overlapping intervals). In Figure 5b) we show the corresponding fraction of weights retained by each method. Together, these plots illustrate both the accuracy-sparsit trade-off and how each method compares against the oracle- pruning baseline. It is clear that WS-KAN most closely follows the accuracy-sparsin trade-off of the oracle-pruning technique. Moreover, DS consistently ranks second best, whereas all other approaches perform poorly. The only partial exception is MIP + Alignment, which shows some utility but still falls significantly behind both DS and WS-KAN Additional downstream pruning results are available in App. Importantly, we observe that WS- WS-KAN offers a significant timing advantage over Oracle Prune, being up to five orders of magnitude faster (Figure Sc).
*This (off-the-shelf) pruning algorithm is found

---

Published as a conference paper at ICLR 2026
fandon Chaifirr (oun)
000
NJ Agg. Algg
BM:
Oriha
MJ
WS-AN
Deapsoata (oun)
GMC Cradi prane
10"
Weights (%) Kept
Time cocconnds lop scclll
80
6
Test accuracy
60-
40
10-
20
2
0%
20%
6.%
90%
100%
8%
40%
60%
80%
100%
Noisy labels (%)
Noisy labels (%)
MS-
(a) Downstream prune accuracy.
(b) Downstream prune kept weight %.
(c) Prune timing.
Figure 5: Downstream pruning performance across methods over KANs trained on MNIST. We report: (i) Test accuracy: the downstream accuracy of pruned networks, averaged over non- overlapping bins of 20%, to highlight the relative effectiveness of pruning strategies under varying noise levels- Figure [Sa] (ii) Kept weights: the percentage of weights retained after pruning, aver- aged over the same bins as in (i) Figure 5b and (iii) Pruning time (J): the computational cost comparison (log scale) in seconds, between WS-KAN and Oracle prune Figure 5c
low is better.
Discussion. WS-KAN consistently provides the most effective WS technique for learning over KANs. Importantly, our ablation baselines (DS/Se(Trans)) are generally second-best. This under- scores the value of incorporating the structural properties of KANs, since even suboptimal inte- gration (as per DS/Se(Trans) still proves beneficial to some extent. Although our alignment method (see App.[D) does not outperform structure-ate aware approaches. it remains a strong alternative. Across nearly all tasks/datasets/mettic combinations, we consistently observe the ordering: "MLP + Align. MLP + Aug. > MLP". Finally, we note that WS-KAN computationally efficient, with complex- ity scaling linearly in the number of edges of the KAN-graph; see App. C.5 for detailed complexity and runtime analysis.
CONCLUSIONS
We addressed the development of weight-space models for KANs by performing a symmetry anal- ysis that guided the design of our WS-KAN architecture. We also built comprehensive model zoos to support future research and evaluation of weight-space models for KANs.
Several promising future directions remain open. While we have shown that WS-KAN gen- cralize to wider KAN architectures unseen during training (Section 5.I.T), extending this to deeper architectures and broader topology variations remains an exciting avenue. Beyond this, the availabil- ity of weight-space models for both KANs and MLPs naturally invites the study of transformations between the two, enabling us leverage their complementary strengths. Many analytical tools well established for MLPs; by converting a KAN into an equivalent MLP (while preserving its func- tion), these tools can be used to study the original KAN. Conversely, converting an MLP into a KAN allows us to exploit the interpretability of KANs (Barašin et al.) 2024), offering new insights into the MLP. Finally, evaluating WS-KAN's generalization other KANS c.g., CNN-KANs would be an interesting direction.
ACKNOWLEDGMENTS
G.B. is supported by the Jacobs Qualcomm PhD Fellowship. HM is supported by the Israel Science Foundation through personal grant (ISF 264/23) and an equipment grant (ISF 532/23), and by the Career Advancement Chairs in Artificial Intelligence- - Schmidt Futures.
REPRODUCIBILITY STATEMENT
Our code is available at http::/hhhhloooooaaaaaaaanAaaaae tion details required to replicate the results and evaluations are provided in App. C
All implementa-
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Samuel K Ainsworth, Jonathan Hayase, and Siddhartha Srinivasa. Git re-basin: Merging models 04002266,,
modulo permutation symmetries. arXiv preprint arXiv:2209, 04836, 2022.
Irina Barašin, Blaž Bertalanič, Mihael Mohorčič, and Carolina Fortuna. Exploring kolmogorov-
arnold networks for interpretable time series classification. arXiv preprini urXiv:2411.14901
2024.
Lukas Biewald. Experiment tracking with weights and biases, 2020. URL httpps:/wee wandb com/ Software available from wandb.com.
Zavareh Bozorgasl and Hao Chen. Wav-kan: Wavelet kolmogooov networks. arXiv preprint arXiv:2405. /2832, 2024. URL/htpps:/arrxvvoorr////.2
Jürgen Braun and Michael Griebel. On constructive proof of kolmogorov's superposition theorem. Constructive approximation, 30(3):653-675, 2009.
Johanni Brea, Berfin Simsek, Bernd Illing, and Wulfram Gerstner. Weight-space symmetry in deep
networks gives rise to permutation saddles, connected by equal-loss valleys across the loss land- scape. arXiv preprint arXiv:/907.029) 2019.
Johann Brehmer, Sönke Behrends, Pim De Haan, and Taco Cohen. Does equivariance matter at scale? arXiv preprint arXiv:2410.231799 2024.
Michael M Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint arXiv:2 2021.
Tarin Clanuwat, Mikel Bober-Irizar, Asanobu Kitamoto, Alex Lamb, Kazuaki Yamamoto, and David Ha. Deep learning for classical japanese literature, 2018.
Taco Cohen and Max Welling. Group equivariant convolutional networks. In International confer- ence on machine learning. pp. 2990-2999. PMLR, 2016.
Taco S. Cohen, Mario Geiger, Jonas Köhler, and Max Welling. Spherical cons. In 6th International Conference on Learning Representations, ICLR, 2018.
George Cybenko. Approximation by superpositions sigmoidal function. Mathematics of control, signals, and systems, 2(4))303-311 1989.
Carl De Boor. practical guide to splines, volume 27, springer New York, 1978.
Gabriel Eilertsen, Daniel Jönsson, Timo Ropinski. Jonas Unger, and Anders Ynnerman. Classify- ing the classifier: dissecting the weight space of neural networks. In European Conference on Artificial Intelligence (ECAI 2020), volume 325, pp. 1119-1926, 2020.
Ziya Erkoç, Fangchang Ma, Qi Shan, Matthias Nießner, and Angela Dai. Hyperdiffusion: Generat- ing implicit neural fields with weight-space diffusion. In Proceedings of the IEEE/CVF interna- tional conference on computer vision, pp. 14300-14310, 2023.
Matthias Fey and Jan Eric Lenssen. Fast graph representation learning with PyTorch Geometric, In ICLR Workshop Representation Learning Graphs and Manifolds, 2019.
Justin Gilmer, Samuel Schoenholz, Patrick F Riley, Oriol Vinyals, and George Dahl. Neural message passing for quantum chemistry. In International conference on machine learning, pp. 1263-122 1272. PMlr, 2017.
Robert Hecht-Nielsen. On the algebraic structure of feedforward network weight spaces. In Ad- vanced Neural Computers, pp. 129-135. Elsevier, 1990.
Ioannis Kalogeropoulos, Giorgos Bouritsas, and Yannis Panagakis. Scale equivariant graph metanet- works. Advances in neural information processing systems, 37:106800-106840, 2024.
TN Kipf. Semi-supervised classification with graph convolutional networks.
arXiv preprint
arXiv: 1ot..660900977 2016.
11

---

Published as a conference paper at ICLR 2026
Benjamin C Koenig, Suyong Kim, and Sili Deng. Leankan: A parameter-lean dolmogorov-arnol network layer with improved memory efficiency and convergence behavior. arXiv preprint arXiv:2502.17844, 2025.
Miltiadis Kofinas, Boris Knyazev, Yan Zhang, Yunlu Chen, Gertjan Burghouts, Efstratios Gavves, Cees GM Snoek, and David W Zhang. Graph neural networks for learning equivariant represen- tations of neural networks. arXiv preprint arXiv:2403.12145 2024.
Andrej Nikolajewitsch Kolmogorov. On the conservation of conditionally periodic motions for small change in hamilton's 's function [en rus]. In Dokl. Akad. Nauk SSSR, volume 98, pp. 525- 530, 1954.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images, 2009.
Alex Krizhevsky, Ila Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convo- lutional neural networks. Advances in neural information processing systems, 25, 2012.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recogmition. Proceedings of the IEEE, 86(11))2278-2324 1998.
Derek Lim, Joshua Robinson, Lingxiao Zhao, Tess Smidt, Suvrit Sra, Haggai Maron, and Stefanie Jegelka. Sign and basis invariant networks for spectral graph representation learning. arXiv preprint arXiv:2202. 13013, 2022.
Derek Lim, Haggai Maron, Mare T. Law, Jonathan Lorraine, and James Lucas. Graph metanetworks for processing diverse neural architectures, 2023. URL
04501
Derek Lim, Haggai Maron, Mare T. Law, Jonathan Lorraine, and James Lucas. Graph metanetworks for processing diverse neural architectures. In The Twelfth International Conference on Learning Representations, 2024.
Ziming Liu, Yixuan Wang, Sachin Vaidya, Fabian Ruehle, James Halverson, Marin Soljacic, Thomas Y. Hou, and Max Tegmark. KAN: Kolmogorov-arrold networks. The Thirteenth International Conference Learning Representations, 2025.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In 7th International Conference on Learning Representations, ICLR, 2019.
Haggai Maron, Ethan Fetaya, Nimrod Segol, and Yaron Lipman. On the universality of invariant networks. In International conference on machine learning, pp. 4363-4377 PMLR, 2019.
Haggai Maron, Or Litany, Gal Chechik, and Ethan Fetaya. On learning sets of symmetric elements. In International Conference on Machine Learning, pp.
PMLR, 2020.
Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Oc- cupancy networks: Learning 3d reconstruction in function space. In Proceedings the IEEE/CVF conference on computer vision and pattern recognition, pp. 4460-4470, 2019.
Christopher Morris, Martin Ritzert, Matthias Fey, William L. Hamilton, Jan Eric Lenssen, Gaurav Rattan, and Martin Grohe Weisfeiler and leman go neural: Higher-order graph neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pp. 4602-4609, 2019.
Aviv Navon, Aviv Shamsian, Idan Achituve, Ethan Fetaya, Gal Chechik, and Haggai Maron. Equiv- ariant architectures for learning in deep weight spaces, In International Conference on Machine Learning, pp. 25790-25811 PMLR, 2023.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. Advances in neural information processing systems, 32, 2019.
Siamak Ravanbakhsh. Jeff Schneider, and Barnabas Poczos. Equivariance through parameter- sharing. In International conference on machine learning, pp. 2892-2901. PMLR, 2017.
12

---

Published as a conference paper at ICLR 2026
Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model. IEEE transactions on neural networks, 20(1):61-80, 2008
Isaac Schoenberg. Contributions the problem of approximation of equidistant data by analytic functions. part a. on the problem of smoothing or graduation. a first class of analytic approxima- tion formulae. Quarterly of applied mathematics, 4(1):45-99, 1946.
Konstantin Schürholt, Dimche Kostadinov, and Damian Borth. Self-supervised representation learn- ing on neural network weights for model characteristic prediction. Advances in Neural Informa- tion Processing Systems, 34: 14411681-40 11048-116493, 2021.
Konstantin Schürholt, Boris Knyazev, Xavier Giró-i Nieto, and Damian Borth. Hyper-
representations generative models: Sampling unseen neural network weights. arXiv preprint arXiv:2209.14755 2022a.
Konstantin Schürholt, Boris Knyazev, Xavier Giró Nicto, and Damian Borth. Hyper- representations as generative models: Sampling unseen neural network weights, 2022b.
Konstantin Schürholt, Dimche Kostadinov, and Damian Borth. Hyper--eprresenaation::: Self- supervised representation learning on neural network weights for model characteristic prediction, 2022c.
wttp://..wrg/aa///1.... 15288
Konstantin Schürholt, Diyar Taskiran, Boris Knyazev, Xavier Giró-i Nieto, and Damian Borth. Model zoos: A dataset of diverse populations of neural network models. arXiv preprint arXiv:2209.14764, 2022d.
Konstantin Schürholt, Diyar Taskiran, Boris Knyazev, Xavier Giró Nieto, and Damian Borth. Model zoos: A dataset of diverse populations of neural network models, 2022e. URL https:
/ //axxiv.or//bbs /2009 14762999..7444
Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein. Im- plicit neural representations with periodic activation functions. Advances in Neural Information Processing Systems, 33:7462-7473. 2020.
Viet-Hoang Tran, Thieu N Vo, An Nguyen The, Tho Tran Huu, Minh-Khoi Nguyen-Nhat, Thanh Tran, Duy-Tung Pham, and Tan Minh Nguyen. Equivariant neural functional networks for trans- formers. arXiv preprint arXiv:2410.04200 2024.
Thomas Unterthiner, Daniel Keysers, Sylvain Gelly, Olivier Bousquet, and Πyya Tolstikhin. Predict- ing neural network accuracy from weights. arXiv preprint arXiv:2002.11448, 2020.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural informa- tion processing systems, 30, 2017.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmark- 2017.
ing machine learning algorithms. arXiv preprint
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful graph neural networks? In 7th International Conference on Learning Representations, ICLR, 2019.
Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R Salakhutdinov, and Alexander Smola. Deep sets. Advances in neural information processing systems, 30, 2017.
David W Zhang, Miltiadis Kotinas, Yan Zhang, Yunlu Chen, Gertjan J Burghouts, and Cees GM Snoek. Neural networks are graphs! graph neural networks for equivariant processing of neural networks. 2023.
Jusheng Zhang, Yijia Fan, Kaitong Cai, and Keze Wang. Kolmogorov-arood fourier networks. arXiv preprint 0661: 2025.
Allan Zhou, Kaien Yang, Kaylee Burns, Adriano Cardace, Yiding Jiang, Samuel Sokota, J Zico Kolter, and Chelsea Finn. Permutation equivariant neural functionals. Advances in neural infor- mation processing systems, 36:24966-244 24992, 2023.
Allan Zhou, Chelsea Finn, and James Harrison. Universal neural functionals. Advances in neural information processing systems, 37:104754-104775 2024.
13

---

Published as a conference paper at ICLR 2026
A APPENDIX ROADMAP
This appendix gathers the mathematical background, full experimental details, the alignment proce- dure, and proofs that support the main text. See guide below.
App. B (B-splines). Reviews uniform B-splines and clarifies how they parameterize the univariate functions KANs.
App. (Extended Experimental Section). Provides all details to create the model zoos for each task and reproduce all results:
C
App.
INR classification.
App.
Accuracy prediction.
App.
Pruning mask prediction.
App.
C.4
Ablation: positional encoding and bidirectional message passing.
App.
C.5
Complexity and runtime analysis.
App. D (Aligning Kolmogorov-Arnold Networks). Here extend the alignment ideas presented in Ainsworth et al. (2022) to KANs.
App. D.1 (Aligning a model zoo dataset). Here we describe how alignment procedure discussed App. D can be applied an entire dataset of trained KANs.
App.
(Proofs). Collects all formal propositions and proofs.
App. F
(Large Language Model (LLM) Usage). Here we provide details on how we used
LLMs.
B B-SPLINES
B-splines are smooth piecewise polynomial functions over a domain [a, b]. defined using a knot vector T. Specifically, a degree-k B-spline with G grid points is expressed as:
G+k-1
B(x) (c. = Σ Cx,,()),
(6)
Ly 11
8 16
where Ba(x) basis functions and 6; the learnable parameters; the inse illustra- tion for the basis functions corresponding to G = 5 and k = 3. The knot vector T [t_ to fi....,, t....,, is larger than the domain itself, and determines where and how the basis functions are defined. The basis functions are defined recursively using the Cox-de Boor formula (De Boor 1978); the explicit expression provided below. For uniform B-splines, of
which we focus on in this paper, the knots are equally spaced with to = to = b. and spacing AI 1-11.
Cox-de Boor Formula The Cox-de Boor recursion De Boor 1978) defines the B-spline basis functions follows:
[1, if S tith, for k = 0, otherwise,
(x:) = [00
(1) to t+t - to New + 4-*411 (1H) for k' 0.
t4#41 - t+t1
We denote
B,(r) = N,-A(())
In the uniform case, the denominators simplify 1 At.
Thhe figure taken from Liu al. 2025
14

---

Published as a conference paper at ICLR 2026
C EXTENDED EXPERIMENTAL SECTION
The implementation of WS-KAN was carried out using PyTorch (Paszke et al. 2019 and Py- Torch Geometric (Fey & Lenssen 2019). which are distributed under the BSD and MIT licenses, respectively. Hyperparameter optimization was conducted with the Weight and Biases frame- work (Biewald 2020 Below we provide the additional details necessary to reproduce our ex- periments. These include instructions how we constructed the model zoos for each task, how trained both WS-KAN and the baseline models, as well as the hyperparameter grids and other rel- evant configurations. Importantly, for all WS models considered we employed the exact same data split and hyperparameter grid search. For the specific procedure used to align a dataset of trained KANs, please refer to App. D.I
Optimizer and schedulers. For all considered datasets and tasks for WS models considered, we use the AdamW optimizer [oshchilov & Hutter (2019) in combination with a linear learning rate scheduler, incorporating a warm-up phase over the first 100 of training steps.
All KANs we have trained for constructing the various model zoos use the fit function from the PYKAN library o
C. INR CLASSIFICATION: EXTENDED SECTION
C.1.1 MODEL ZOO INR CLASSIFICATION
Constructing the synthetic 2D sine wave INR dataset. We start by sampling frequency vector uniformly random from the range [0.5, 10)2. Each sample defines a 2D sine wave of the form,
((x) siw (x))
where æ is the input and g(x) is the target output. To learn this mapping, we train a KAN-based implicit neural representation (INR) with architecture depth/widtti configuration of 32, 32, 1]. We train 1,000 independent models, using split of 800/100/100 for training, validation, and testing, respectively. Training is performed with batch size of 128, and the learning rate is fixed at 0.01, for 1, 000 epochs. The univariate functions in the KAN is parameterized via B-splines with G = 30 and & = 3. See example of a KAN-based INR for a samples sine function in Figure
Constructing INRs for MNIST, F-MNIST, CIFAR1O. For each dataset, we train a KAN- based implicit neural representation (INR) model that maps pixel coordinates their corresponding intensity values-graysceat grayscale for MNIST and F-MNIST, and RGB for CIFARIO 10. The network ar- chitecture is configured as (2, 32, 1] for MNIST and F-MNIST, and 32, 32, 3] for CIFARIO 10 to account for its three color channels. For all datasets, we adopt batch size of 128 and train for 1,000 epochs using fixed learning rate of 0.01. The univariate functions the KAN is parame- terized via B-splines with G 10 and k- = 3. An illustration of a KAN-based INR is provided in Figure 7
htttps://iithin http:///ggtuub.c/oo/tt. /i ndXi aomi acmm ng/ //pykkn
-25
035
c1
300
Figure 6: Example of a KAN-based INR applied to the synthetic 2D sine wave dataset. The left panel shows the ground truth, and the right panel shows the reconstructed result.
15

---

Published as a conference paper at ICLR 2026
Figure 7: Reconstructions from INR on CIFAR-10 10, Fashion-MNIST, and MNIST. All PSNRs are more than 40.
C.1.2 INR CLASSIHICATION ADDITIONAL DETAILS
Hyperparameters. For all datasets considered (except the sine wave dataset), we trained both WS-KAN and the baseline models randomly sampled subset of 10,000 trained KANs, with an additional 5,000 reserved for validation. The KANs used for testing are those that were trained images. For each considerd WS model, we used layers, hidden
on the original dataset's dimension of 128, batch size of 128, weight decay of 0.01, dropout of 0.2, and 300 epochs. For the learning rate, we performed a search over the set {0.001, 0.00013.
For the sine wave dataset, made the following adjustments: training was conducted for 200 epochs with fixed learning rate of 0.001. Additionally, we experimented with varying training set sizes, using subsets of (100, 200, 500, 800} from the available 800 samples.
C.1.3 RESULTS ON SYNTHETIC DATASET
Synthetic dataset: 2D Sine waves. We constructed a dataset of KAN-based INRs for 2D sine waves see App. C.I.I for more details on this dataset construction. The task is to predict the frequency w a given test KAN-based INR, via its parameters. To evaluate the generalization capabilities this synthetic dataset, training examples (INRs). As shown in Figure g WS-KAN consistently outperforms all baseline methods, even when trained with only a small number of examples.
of the architectures
repeat the experiment while varying the number of
C.2 ACCURACY PREDICTION - EXTENDED SECTION
The weight-space training pipeline for the task of accuracy prediction is illustrated in Figure I0
C.2.1 MODEL ZOO ACCURACY PREDICITON
To construct the model zoos for the task of accuracy prediction, we used MNIST, F-MNIST, and K-MNIST. For each dataset, we trained 4,000 KAN models with 3000/500/500 split for training,
Aug.
DeepSets (Ours)
SetTrans (Ours)
MLP
WS-KAN (Ours)
MLP
(log scals)
10"
10"4
320
400
480
720
800
samples
Figure 8: Results the synthetic 2D sine INR dataset.
16

---

Published as a conference paper at ICLR 2026
Test accuracy (%)
100
100
Test accuracy (%)
100
Test accuracy
60
60
60
40
40
40
20
20g
0
20
40
60
80
100
20
40
60
80
100
0
20
40
60
80
100
Nolsy labels (%6
Noisy labels
Nolsy labels (%6
(a) F-MNIST
(b) K-MNIST
(c) MNIST
Figure 9: KANs test accuracies. Scatter plots of KAN accuracies on the test set (Y-axis) as function of the number of noisy labels (X-axis).
y 93%
Weight-space architecture (e.g., our WS-KAN)
accuracy predictor
y 78%
trained KANs
trained to predict the KAN's accuracy
Figure 10: Weight-space training for accuracy prediction. We train KANs under varying noise levels, then fit a WS model (e.g.. WS-KAN) to predict their accuracy from weights.
validation, and testing. Each KAN model we trained (to classify the images in the dataset at hand) used layers. For MNIST and F-MNIST, the layer dimensions were [784, 32, 32, 10), where 784 = 28 x 28 corresponds to the number of grayscale input pixels. For CIFAR 10, the dimensions were [3072, 32, 32, 10], with 3072 32 32 x accounting for the RGB input pixels. Note that 10 denotes the number of target classes across all datasets.
We observed that KANs trained on these datasets achieved similar accuracy levels. increase the difficulty of predicting accuracy from parameters, introduced label noise. Specifically, for each KAN, we randomly selected portions the training data and permuted their labels. In Figure 9 we report the test accuracies over random sample of 100 test instances.
We trained those KANs for 100 epochs, with a fixed learning rate 0.01 and batch size 128. We again used grid size of G and k for the B-splines that define the univariate functions in the trained KANs.
C.2.2 ACCURACY PREDICTION ADDITIONAL DETAILS
Hyperparameters. The hyperparameters used all experiments, across tasks, datasets, and base- lines, are summarized in Table
Parameter
Values
Embedding size
(128, 32)
Number of epochs
100
Learning rate
{0.001, 0.005, 0.0001)
Dropout
0.2
Number of layers
[4,1] 1]
Batch size
32
Table 5: Hyperparameter configuration used in the experiments for accuracy prediction.
17

---

Published as a conference paper at ICLR 2026
C.3 PRUNING MASK PREDICTION EXTENDED SECTION
The pipeline for training a WS model for the task of predicting the pruning mask is provided in Figure
11
Weight-space architecture (e.g., our WS-KAN)
mask predictor
x
trained KANs trained to predict oracle's pruned mask
Figure 11: Pipeline for training WS model for the task of pruning mask prediction.
C.3.1 MODEL. Zo0 PRUNING
The model zoos used in these experiments build upon those described in App. C.2.1 More specifi- cally, starting from the KANs obtained in App. C.2.1 we applied the Oracle prune algorithm edge-pruning method provided by the PYKAN library" Specifically, all edges whose average activa- tion on the training dataset fell below a fixed threshold were removed. Throughout experiments, used threshold of 0.01.
C.3.2 PRUNING MASK PREDICTION ADDITIONAL DETAILS AND RESULTS
We note that for the test metrics ROC-AUC and Accuracy reported Table we used the full set of 500 test samples. Since Accuracy is threshold-dependders metric, we selected the threshold by maximizing validation accuracy and applied the same threshold during testing.
we used random subset of 100 KANs from the test set, same as the one for Figure This decision stems from the high computational cost of generating such plots, which requires full inference on the entire dataset (c.g., MNIST) for each pruned KAN produced by every WS method.
For the pruning plots for each dataset, as shown in Figures 5a 5b and 13
Hyperparameter. For datasets, we adopted a consistent training setup, closely aligned with the configuration described in App. C.I Specifically, used 4-layer architecture with a hidden dimension 128, a batch size 32, weight decay of 0.01, and dropout 0.2. Training was performed for 15 epochs. For the learning rate, conducted a grid search over {0.001, 0.0001].
Additional results. In Figures and 13 MNIST and K-MNIST, respectively. As shown in Figure [2] WS-KAY achieves the best per- formance, striking the most favorable trade-off between accuracy and parameter sparsity, closely approaching the oracle prune that serves as our supervision. Meanwhile, the other baselines per- form markedly worse. contrast, Figure 13 shows that DS is also highly effective, performing on with WS-KAN.
we present the downstream pruning results on F-
C.4 ABLATION: POSITIONAL ENCODING AND BIDIRECTIONAL MESSAGE PASSING
Here we present ablation study evaluate the importance of both the positional encoding and the bidirectional message passing. Specifically, we report all results in the paper without the positional encoding and the bidirectional message passing.
https:://itthbe
18

---

Published as a conference paper at ICLR 2026
DoopSiets (Ouns)
0000
80
Test accuracy (%)
70
Weights (%) Kept
60
50
40
30
20
10
0%
20%
40%
60%
80%
100%
0%
20%
40%
60%
80%
100%
Noisy labels (%)
Noisy labels (%)
(a) Downstream prune accuracy.
(b) Downstream prune kept weight %.
Figure 12: Downstream pruning performance across methods over F-MNIST. We report: (i)
Test accuracy: the downstream accuracy of pruned networks, averaged over non-overlapping bins of 20%, to highlight the relative effectiveness of pruning strategies under varying noise levels Figure 12a (ii) Kept weights: the percentage of weights retained after pruning, averaged over the same bins as in (i) Figure 12b
Classifarr (Ours)
Orighal
DeepSets
70
Weights (%) Kept
Test accuracy
50
8-
40
30
4-
20
0%
20%
40%
60%
80%
100%
8%
20%
40%
60%
80%
100%
Noisy labels (%)
Noisy labels (%)
(a) Downstream prune accuracy.
(b) Downstream prune kept weight %.
Figure 13: Downstream pruning performance across methods over K-MNIST. report: (i)
Test accuracy: the downstream accuracy of pruned networks, averaged over non-overlapping bins of 20%, to highlight the relative effectiveness of pruning strategies under varying noise levels Figure 13a (ii) Kept weights: the percentage of weights retained after pruning, averaged over the same bins as in (i) Figure 13b
INR Classification Accuracy. The results are provided in Table
Table 6: Ablation study INR classification accuracy.
Models
MNIST
F-MNIST
CIFARI0
WS-KAN- no PE
94.3 + 0.4
46.1 8..5
WS-KAN no BIDIR WS-KAN
93.8 = 0.2
8.14+0.1
10.0 <0.1
94.3
42.2 8.66
Accuracy Prediction.
The results provided in Tables 7 and
Table 7: Ablation study
Accuracy prediction (MSE) [x103]]
Model
MNIST (MSE)
F-MNIST (MSE)
K-MNIST (MSE)
WS- KAS- no PE
2.53 + 0.22
2.44 + 0.50
1.10 + 0.17
WS-KAN- no BIDIR
4.03 0.62
3.64 0.12
1.50 0.08
WS-KAN
3.29 + 0.17
2.94 + 0.13
1.45 - 0.08
19

---

Published as a conference paper at ICLR 2026
Table 8: Ablation study
Accuracy prediction (R²).
Model WS-KAN- PE WS-KAN- no BIDIR WS-KAN
MNIST (Rt*)
F-MNIST (R²)
K-MNIST (R²)
96.00 = 0.35
93.57 = 1.32
96.62
93.64 + 0.98
90.40 + 0.31
95.54 + 0.23
94.81 + 0.27
92.27 + 0.35
95.69 + 0.24
Pruning Mask Prediction.
The results are provided in Tables lndd TO
Table 9: Ablation study Pruning mask prediction accuracy.
Model
MNIST (Accuracy)
F-MNIST (Accuracy)
K-MNIST (Accuracy)
WS-KAN- no PE
97.77 + 0.28
98.68 + 0.04
96.28 + 0.45
WS-KAN- no BIDIR
96.74 + 0.27
98.27 0.04
95.39 + 0.14
WS-KAN
97.93 + 0.19
98.93 = 0.05
97.72 + 0.14
Table 10: Ablation study - Pruning mask prediction AUC.
Model
MNIST (AUC)
F-MNIST (AUC)
K-MNIST (AUC)
WS-KAN no PE
99.51 0.09
99.59 + 0.02
98.85 0.21
WS-KAN- no BIDIR WS-KAN
98.38 + 0.20
98.99 + 0.01
97.11 + 0.37
99.54 0.01
99.72 0.02
99.46 + 0.09
We observe that WS-KAN performs well even without positional encoding. However, on the more challenging equivariant task pruning mask prediction WS-KAN with positional encoding out- performs the version without it 6/6 cases. This indicates that positional encoding plays particularly important role in enhancing WS-KAN's performance on equivariant tasks. We also note that bidircctional message passing crucial: across all tasks and dataset combinations, WS- KAN without bidirectional message passing consistently underperforms the version that includes it.
C.5 COMPLEXITY AND RUNTIME ANALYSIS
We report the running times observed in the INR classification tasks. Table II summarizes the average training and testing time per epoch (in seconds), computed over 10 epochs and presented together with the corresponding standard deviation. The results indicate that our runtimes are con- sistent with those commonly observed for standard GNN models typical graph benchmarks.
Complexity analysis. We assume the input KAN contains ny nodes at layer E {1...., .i....,.. L}, and that all nodes and edges share the same feature dimensionality, which treat constant. The total number of nodes therefore
and the total number of edges due full connectivity between consecutive layers
L-1
ny n441
Referring to our message- passsin mechanism, as presented Section 3.3 we obtain the following:
(a) The function MLP (l:F), is applied once per edge, giving E operations. Aggregating over the incoming neighbors of each node contributes another E operations, since the graph is
(I:F
directed and each edge is counted once. Thus, step (a) costs O(E).
(b) This step mirrors (a), except that the edge directions are transposed. Therefore, also costs (
20

---

Published as a conference paper at ICLR 2026
(c) This step applies MLP independently to each edge, contributing another O opera- tions.
(d) This step is per-node operation, giving a cost of O(N). Because E N, the edge-wise computations dominate the total cost.
Hence, the overall complexity is
O
Table 11: Running times measured (in seconds) for all experiments conducted on the INR classifi- cation task.
Dataset
MNIST
F-MNIST
CIFAR-I0
Train time per epoch (s)
10.12 + 0.06
10.18 + 0.05
10.88 0.10
Test time per epoch (s)
2.84 = 0.12
2.81 + 0.06
3.05 + 0.08
D ALIGNING KOLMOGOROY-ARNOLD NETWORKS
complementary strategy to designing permutation- -qquivvrinnn architectures, to instead keep model class simple and canonize the input. In our context, the core idea would be to map each KAN into a fixed canonical form by permuting its neurons so that two networks differing only by permutations (and thus computing the same function) are aligned to the same representation.
A particularly interesting case arises in the context of MLPs as discussed in Ainsswoott et al. (2022). In that case, O's are collection of scalers, and simple choice of dist to be the Euclidean distance 2| the alignment problem becomes
argmin
A) - (0) argmax vec(@A) ved
We can re-express this terms of the full weights,
argmax a A) P.w(()) )r + (w/"), P.w, P (WL") w 1). x=(P.)
B
(8)
Turning to KANs, the parameters O and θ correspond to collections of univariate functions, rather than scalars. To align such models, must therefore define distance function appropriate for functions. Importantly, there is no single "correct" choice here-different definitions may lead different alignment outcomes.
In this paper, consistent with our formalism in Section 3.2 we associate each such entry in O, with a parameter vector of the form
(O [wh,, Wsy c).
where, for the sake of this example and with a slight abuse of notation, wh, Wa, and are the coef- ficients defining the corresponding i-th ID function. For convenience, we refer to these coefficients as channels, and denote the e-th channel of this vector by (0a): [Wb, W., cle.
With this structure, we define the aligmment objective as the sum of &g distances across all channels:
arg min Σ
(eec((a)) ved π ((g)) ||,
arg max
vec(O.) ved(((
(9)
which parallels Eq. yields
7 but now summed across channels. Expanding this channel-wise objective
Σ
(ofezs)
PIo( + ( Prole P. +... (((A)) o (l:B)) P
argmax x=(P.}
argima((f (A) P. * (B), F (6(1)) Ps x 02 (B) P +... (1)1) 01 P ==(P.)
(B)
21

---

Published as a conference paper at ICLR 2026
In the second line, move the sum channels inside each Frobenius inner product. As result, the channel superscript is omitted: the Frobenius inner product now implicitly sums over absorbing the channel dimension. Importantly, denotes contraction operator that acts only the first two indices, leaving the channel dimension untouched. Concretely, for the middle term we have:
(B) (P2 02
P. P.D....
E(P))y (02 W. (Pi )add v/.
(02
(10)
This formalism provides straightforward extension of the alignment framework of Ainsworth al. (2022) for MLPs to the KAN setting: their algorithm can be directly applied once the * operation is defined as above.
D.I ALIGNING A DATASET OF TRAINED KANS
While the formalism in App. provides a method for aligning a pair of KANs, our experiments re- quire aligning entire dataset of KANs. To achieve this, integrate our aligmment technique with the MergeMany algorithm introduced in Ainsswrrth et al. (2022). The central idea is to iteratively the alignment procedure between model and the average of all other models.
In practice we observed that convergence times could be prohibitively long. To address this, we adopted the following strategy: we fixed the error tolerance 0.001, then randomly sampled 1000 models and applied the MergeMany algorithm to this subset, consistent with Navon et al. (2023). The outcome was new model representing the average of the aligned subset. This model then served as the reference for aligning the remaining training models as well as the test models, ulti- mately yielding fully aligned training and test sets.
Finally, we note that in our formulation, all channels are equally weighted. While this simplifies the alignment, may be suboptimal in certain cases. Alternative weighting schemes could yield improved results. We leave a detailed exploration of such extensions to future work.
E PROOFS
Lemma E.1 (MLP as an approximation of the univariate functions composing the KAN), Let B denote the family of cardinal B-splines of degree k. defined a fixed grid G the domain E C C IRG+ where the set of admissible coefficients lies in compact domain C. Consider the function
(a, b), and parameterized by coefficients с = [co....,CG+t..
R + R defined as (x) wy b(z) + W. B(x), where Wb, E W C E. for some compact set W. b(x) denotes the silo function, and B(x) denotes the B-spline. Then, for any ε > and for any compact domain X there exists set of weights for multilayer perceptron MLP: G-4--3 - R such that,
supzex MLP(Z, Wa Wb, e) xb(z)] <E.
Proof. The claim follows directly from the universal approximation theorem Cybenko (1989).. The input vector to the MLP lies in the compact domain
(x, W. Wh, c) E X' x W W C,
which is compact since finite cartesian products of compact sets are compact. Moreover, the target function (b))) continuous, as a weighted sum of continuous functions- the silo function b(-) and the B-spline (--) are continuous. Therefore, by the universal approximation theorem, there exists an MLP that uniformly approximates &b on this compact domain to arbitrary accuracy > 0. This completes the proof.
Proposition E.2 WS-KAN can simulate the forward pass of KANs). Let fo be given KAN
architectree, defined over an input domain [a, b)",, where each univariate function is represented by B-spline from the family B. Let G be ity KAN-graph, where the nodes in the first layer enhanced with the imput x E [a, 8.. For every & > O, there exists a WS-KAN such that,
sUPæela. [WSKG(/))WWWKKKKGGG) fo(a:)) <.
22

---

Published as a conference paper at ICLR 2026
Proof. We recall that fo(x) = (Ø" o o.... Φ Ja is composition of continuous functions. We denote by v' the nodes in the KAN-graph that correspond to the activations in layer 1, defined a'. We prove via induction, that 1 layers of WS-KAN, a/+1 = ( o... x0") )ar over the nodes v/+1 to arbitrary precision.
uniformly approximate the output
Induction proof. We begin by defining a compact space, S, which encompasses all possible values attained by any neuron within the KAN. Since the input to the KAN lies in (a, BJ (which is compact) and the applied univariate functions are continuous, their outputs form compact sets. With mild abuse of terminology, we define the compact output space to be a compact set that includes the outputs corresponding to all inputs in the unit ball (xx 1) around each input. Each neuron then receives finite sum of elements, each belonging to a compact set, which implies that the neuron's s value also lies within compact set. Moreover, because the number of neurons is finite, the union of all such compact sets is itself contained in a larger compact domain. Thus, there exists a compact space, defined as S, that contains the values of all neurons. Additionally, we denote the maximal width of the network by Nmax
We now proceed with a proof by induction.
Step 1. Initialization. We define the KAN's input over the KAN- KAN-graph as follows,
= 120 O, ep. = 6p. == [wbpppr usppa Sp.gh
so that the node features layer encode the KAN's input æ, while edge features store the spline parameters. Thus, the base case is satisfied.
Step 2. Induction hypothesis. We assume that after layers of WS-KAN message passing we have
||o' a'll < <,
where a' denotes the activations at layer of the KAN, and 6 0 is arbitrarily small approxima- tion eπrоr that we can control. Given >> O, we now show that we are able to obtain e+ such that || |
Step 3. Inductive step.
We recall given WS-KAN update equation:
of MLP
MLP(!:F) (ej (()))
of + MLB
Σ ML ((()))))))))
jee(L,))
e(.)) < MLP (v,, Dy ((,)))
8. - MLP(3) (v,, v. v
The KAN update rule layer 1+ is
a 1+1 = (hng(at))
where 6/ denotes a univariate spline function parameterized by (whing W s..g).
To reproduce this update within WS-KAN, recall that MLP univarivai can be chosen be arbi- trarly close approximation of the univariate function composting the KAN, recall Lemma 4.1 thus choose MLP univaritte such that
MLP univariaro b, - vaaa)l < e/2.Nmax:
where vi stands for the 1D univariate functions with the parameters (b, c, d).
Since univarit (a, e, dl is continuous and defined on compact set, is uniformally continuous. Thus, there exists 8' > 0 such that,
||a a' 8' I (a, b, c, d) MLP univariate (a', c, d) < e/2N max-
Thus, we fix < min(6', 1), and from the induction hypothesis we get that there is a WS-KAN network that guarantee l|ol a'll 8.
23

---

Published as a conference paper at ICLR 2026
It holds that for single neruon, given that ||a d'll < 8,
(ML.uuiivaiass
(a', c,,,,,,) c,
=-))===
MMLPuivvan univariate
(d' b, d) MLP MMLpuuivarialle (a, d) + MLP univauuiarialle (a, b, v/a) 5
[LP univariiae
(a', b,
-MLPunivarille (a, d)] + |ILE univariate (a.b,c,d) xb(a)] s
e/2N max e/2.Nux e/Mmax
We set,
MIP(1:F) (u,, ((s))) = MLP univariate (uyy Whppag " 2p.))
(11)
Thus, it holds that for ||l a < 6, which is our induction assumption, have, ||MLP( (IFF) (v), e(s,s) ( <//Nmax:
We can also assign weight to the following MLP, to output the following precisely, The forward aggregator o.
MLP (2:5)) (a, b) b,
(12)
(13)
ooowward thus becomes
forward
ong (dgll S Se/Nmax.
which approximates the KAN update in this layer.
The backward aggregator, edge update, and auxiliary terms are irrelevant for this construction, so their associated MLPs, namely are the zero function, by assigning zero all weight:
MLP(1:B) MLP(2:B) MLP. 0,
(14)
Finally, we set the weights of MLP such that MIP (3) (a, b) = this ensures that w/t1 vg Thus.
forwardd
|1 11 = Σ (1tt1 op 41122
(e/Nuin Wmnx j²)2 < S//ve/v Seese//Nnne Nmax <
Proposition E.3 (KAN symmetries). Let B (Ø"..... Φ¹) denote the collection of parametric one-dimensional functions composing L-layer KAN. Consider the group. G = Sa, x Ság x Sd, 1. the direct product of symmetric groups corresponding to the intermediate dimensions di....,dl d1-1. Let g .P.....,PA PL-1) G. where each Pt is the permutation matrix of E Sá.. Define the group action
with = =(0". ....,""' given by,
t² P. Φz
Φ" P. P 22
= 6'
Then, fe(x) = for (x) for all x.
Proof. We compute the p-th output component:
fo(x), = (#² o oxx)
= Σ
2 p.g
q. (xx)) = =E S (Ø (o
q.k
9.k
Σ P)
aa (xx) fz(x),
The third equality holds by reindexing the summation using a permutation corresponding to P. Recalling Eq. (4), the fourth equality follows from the identities (62 pPp))p = 0² (.)(()) and (PT 22
24

---

Published as a conference paper at ICLR 2026
F LARGE LANGUAGE MODEL (LLM) USAGE
We used large language models (LLMs) exclusively to support the writing process. Their role was limited to improving clarity in technical explanations, refining grammar and style, and enhancing overall readability. All research contributions-include conclusions are entirely our own. The LLMs served solely as tools improve presentation quality and were not employed to generate research content or influence the substance of our work.
experimental design, data analysis, and
25


Reviews:
[{"review_id": "cMJfjG3a9M", "reviewer": "Reviewer_gQZG", "summary": "This paper introduces applying weight-space symmetric architecture for Kolmogorov–Arnold Networks (KANs), called WS-KAN, which leverages a graph representation of the network’s computation to respect inherent symmetries in the architecture. The authors highlight the equivalence of permutation symmetries in KANs to those in standard neural networks, especially MLPs, and propose the KAN-graph—a directed graph where nodes represent neurons and edges carry univariate functions. WS-KAN uses existing work of graph metanetwork to process these graphs and simulate the forward pass of the original KAN. Empirical validation is conducted using a model zoo of pre-trained KANs across various tasks. The proposed model consistently outperforms structure-agnostic baselines and provides superior performance in INR classification, accuracy prediction, and pruning mask prediction.", "questions": "- What does proposition 3.2 imply and how it is related to the main contributions / arguments in the paper? Here \"the nodes in the first layer are enhanced with the input x\", do you really do that in the implementation?\n\n- Have you tried other weight space symmetry works such as DWSNet and NFN, and do they apply to KANs as well?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The method is the first to specifically address the design of weight-space models for KANs.\n\nThe authors construct a comprehensive \"model zoo\" of KANs and benchmark WS-KAN across various tasks, demonstrating superior performance over baselines. The extensive experiments on multiple datasets such as MNIST, CIFAR10, and F-MNIST validate the proposed method's robustness.", "weaknesses": "The paper basically uses the existing solution of graph metanetwork proposed for MLPs to apply on KAN weight space, which is intuitive, however contribution is limited. Also experiments e.g. INR classification, accuracy prediction, all follow the standard from weight space symmetry works on MLP/CNN. From the technical solution and application tasks I do not see anything new and specific to KANs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "d4ie20iA1m", "reviewer": "Reviewer_1aub", "summary": "This work presents WS-KAN, a meta-network tailored for working with Kolmogorov-Arnold Networks (KANs). The authors introduce a graph representation (KAN-graph) for KANs that accounts for their permutation symmetries, and they employ a GNN structure built on this encoding. They conduct a theoretical examination of the model's expressive capabilities and perform empirical evaluations using a freshly assembled collection of pre-trained KANs across multiple tasks, reliably showing better results than competing methods.", "questions": "See weaknesses.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- This paper introduce the first metanetwork for KANs.\n- The motivation and design process are presented clearly.\n- The KAN model zoo (covering INR classification, accuracy prediction, pruning) provides a valuable benchmark for future WS models on KANs.", "weaknesses": "- The discussion in Section 3.1 on permutation symmetries in the KAN weight space primarily pertains to symmetries in the space of neuron functions and their connections rather than the underlying weight parameters that define those functions. This echoes themes from prior work on weight space symmetries in MLPs [1] and only use permutation symmetries. Could other symmetries extend to KANs?\n- Clarification is needed on the scope of each KAN-graph: How is node features initialized, is it associated to input data? If yes, this suggests the WS-KAN operates beyond pure weight space, potentially incorporating input-specific adaptations, which somewhat contradict to the claim being a weight-space-only method.\n- In Lemma 4.2, please clearly define what is meant by \"the nodes in the first layer are enhanced with the input x\", does this imply concatenation, embedding, or another operation, and how does it affect the overall model and expressivity?\n- The paper focuses on B-spline parameterization for KAN neurons, but it would be beneficial to comment on alternative parameterizations and whether the proposed meta-network generalizes to them.\n- The experiments lack reporting on time and memory complexity. Additionally, how well does the approach scale to generating larger models (eg, deeper and wider KANs)\n- As the WS-KAN is GNN-based, I wonder if the method can be tested for generalization, eg, by training the meta-network on small KANs and evaluating its ability to generate parameters for larger, unseen KAN architectures?\n- Experiments illustrating the importance of positional encoding (if used in the graph representation) would be valuable, perhaps through an ablation study.\n\nReference:\n\n[1] Zhao et al. Symmetry in Neural Network Parameter Spaces. arxiv:2506.13018.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DZt4e2fI9R", "reviewer": "Reviewer_y2se", "summary": "This paper studies “learning in weight space” for Kolmogorov–Arnold Networks (KANs). The authors (i) prove KANs admit the same hidden-neuron permutation symmetries as MLPs, (ii) encode a KAN as a directed “KAN-graph” whose edge features are the parameters of each learned univariate function, and (iii) process that graph with a GNN they call WS-KAN. They further show WS-KAN can simulate the forward pass of an input KAN (a standard expressivity criterion in weight-space works), then build several “model zoos” of trained KANs and evaluate on INR classification, accuracy prediction, and pruning-mask prediction—reporting consistent gains over baselines and large speedups vs an oracle pruning routine that requires data passes.", "questions": "See weaknesses.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. Concrete construction of edge features for KANs’ 1D functions; positional encodings are explained and motivated.\n\n2. A standard expressivity result: WS-KAN can approximate a KAN’s forward pass, with proof sketch and full appendix.", "weaknesses": "1. **Lack of empirical symmetry validation:** The work does not experimentally verify the equivariance or invariance properties that the theory implies.\n\n2. **Limited range of equivariant tasks:** Prior works such as Editing INRs [1] and Learning to Optimize [2] evaluate equivariant models on richer, more practical tasks. Incorporating similar benchmarks would strengthen the empirical relevance.\n\n[1] Zhou et al., Permutation Equivariant Neural Functionals. NeurIPS 2023.\n\n[2] Kofinas et al., Graph Neural Networks for Learning Equivariant Representations of Neural Networks. ICLR 2024.\n\n3. **Unaddressed scalability issues:** The paper omits discussion of computational complexity and runtime behavior. Since the KAN-graph grows quadratically with network connectivity, an analysis of how WS-KAN scales with KAN depth and width is needed.\n\n\n4. **Incomplete design analysis:** Key architectural choices—such as the use of bidirectional message passing and specific positional encodings-are insufficiently justified. Ablation studies would clarify their impact on performance", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Oq8DsNsQp5", "reviewer": "Reviewer_vi4a", "summary": "The paper introduces a graph-based framework for designing meta-networks that operate on the pretrained weights of Kolmogorov–Arnold Networks (KANs).", "questions": "I have a question regarding the permutation symmetry of KANs presented in Section 3.1. While it is clear that the permutation symmetry preserves the network’s functional behavior, I am wondering whether there exist other types of symmetry. For example, in MLPs and CNNs with ReLU activations, there is a well-known scaling symmetry, which has also been leveraged in the design of graph-based metanetworks [1]. A similar analysis for KANs would help clarify whether additional symmetries exist beyond permutation, and would further justify that modeling permutation symmetry alone is sufficient for constructing the proposed graph-based network.\n\n\n[1] Ioannis Kalogeropoulos, Giorgos Bouritsas, and Yannis Panagakis. Scale equivariant graph metanetworks.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is well-written and self-contained. Although I am not very familiar with the recent developments on KANs, I was able to understand the architecture clearly. The authors provide a good demonstration of the symmetry properties of the KAN parameter space, along with a clear explanation of how a KAN can be interpreted as a graph. Their approach closely follows the standard methodology used in the literature for constructing metanetworks for traditional MLPs and CNNs.", "weaknesses": "I did not identify any clear weaknesses. Only a few minor typos were found, such as “silo function” in Lemma 4.1 or “weight-space model” in line 10.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)