Kolmogorov–Arnold Graph Neural Networks

## Abstract

Abstract Graph neural networks (GNNs) excel in learning from network-like data but often
lack interpretability, making their application challenging in domains requiring
transparent decision-making. We propose the Graph Kolmogorov–Arnold Network
(GKAN), a novel GNN model leveraging spline-based activation functions on edges
to enhance both accuracy and interpretability. Our experiments on five benchmark
datasets
demonstrate that GKAN outperforms state-of-the-art GNN models in node
classification, link prediction, and graph classification tasks. In addition to
the improved accuracy, GKAN’s design inherently provides clear insights into the
model’s decision-making process, eliminating the need for post-hoc
explainability techniques. This paper discusses the methodology, performance,
and interpretability of GKAN, highlighting its potential for applications in
domains where interpretability is crucial.

###### Index Terms:

## I Introduction

Neural networks and deep learning have clearly revolutionized countless fields,
encompassing the domains of image, text, audio, and medical data. The basic
building block of neural network models is the multilayer perceptron
(MLP) []. Very recently, a promising alternative to MLPs
was proposed, namely the Kolmogorov–Arnold Network (KAN) [].
MLPs are proved to be universal function
approximators [], whereas KANs are inspired by the
Kolmogorov–Arnold representation theorem [, ],
which states that a smooth multivariate function can be written as the finite
sum of the composition of univariate functions; this has a similar structure to
a two-layer neural network.

The idea of Liu et al. [] is to generalize this two-level
structure to more levels. This extension allows the definition of a novel
framework that proved increased accuracy and interpretability with respect to
standard MLPs. Indeed, lack of interpretability in neural networks is a major
issue, which limits its usage in domains in which interpretable results are
crucial (e.g., medical and financial scenarios). To cope with this flaw,
research focuses on the development of explainability techniques that try to
unveil what was learned by those models in terms of salient
features [, ]. In the domain of neural
networks, models specialized in learning from graph-structure data have emerged.
Graph neural networks (GNNs) [] are able to leverage the
connectivity structure of network-like data, learning representative
topology-aware embeddings. In literature, we find different architectures
addressing different aspects and characteristics of graph data. Graph
convolutional networks (GCNs) [] extend the concept of
convolution to graph structures, aggregating information from each node’s
neighbors using spectral graph convolutions.
GraphSAGE [] is another GNN model that generates node
embeddings by sampling and aggregating information from neighbor nodes. It is
based on an inductive learning procedure that allows embedding generation also
for previously unseen nodes. Graph attention networks
(GATs) [] incorporate attention mechanisms into GNNs,
allowing the model to focus on the most salient parts of a node’s neighborhood.
An additional GNN model is offered by GINs (graph isomorphism
networks) []. Such models are designed to have an
effective discriminative ability, theoretically proven to be as powerful as the
Weisfeiler–Lehman graph isomorphism test []. GNNs
have also been extended to relational structures, with the introduction of
relational graph convolutional networks (RGCNs). They are designed to handle
graphs with multiple types of edges and nodes by incorporating relations
explicitly into the model. Furthermore, variational graph autoencoders
(VGAE) [] aim to encode graph data into a latent
space and then reconstruct the graph from the generated embeddings.

Similarly to classic multilayer perceptron models, GNNs are not immune to the
interpretability curse; to shed light on their predictions, several
explainability methodologies have been developed. Such methods explain GNN
predictions in terms of importance subgraphs built of salient nodes, edges, node
features, connected subgraphs, or a combination of those elements. The
pioneering work in eXplainable Artificial Intelligence (XAI) for GNNs is
GNNExplainer [], which determined explanations by
generating important subgraphs using a mask on the adjacency matrix able to
maximize the mutual information between the prediction and the distribution of
the possible explanation subgraphs. GraphSVX [] in another
method, which determines explanation in terms of important nodes and node
features relying on the theoretical background behind Shapley
values []. It uses a decomposition technique relying on a
surrogate linear model for approximating Shapley values. A methodology targeting
edges as a means for determining explanation subgraphs is
EdgeSHAPer []. It employs Monte Carlo sampling to
approximate Shapley values determining salient edges forming relevant subgraphs
driving predictions. One additional XAI tool is SubgraphX. Also exploiting
Shapley value approximation, it looks for explanations only in terms of
connected subgraphs by using a Monte Carlo Tree Search approach. Given the
underlying mechanisms and the different ideas on which each explainer is based,
rationalizing a model’s predictions may lead to different explanations, given
the intrinsic methodological biases different explainers have, which may lead to
counterintuitive outcomes [].

Motivated by the need for inherently interpretable GNN models, we extend the
Kolmogorov–Arnold representation theorem to GNNs, introducing the Graph
Kolmogorov–Arnold Network (GKAN). Our main contributions are as follows:

- •
Novel GNN Architecture: We propose GKAN, a novel GNN model
that employs spline-based activation functions on graph edges. This design
enhances the model’s flexibility and interpretability while retaining the
efficiency of message-passing mechanisms in GNNs.
- •
Enhanced Interpretability: GKAN provides inherent
interpretability by design, eliminating the need for external
explainability techniques. This feature is crucial for applications in
domains requiring transparent decision-making processes.
- •
Performance Improvement: We demonstrate that GKAN outperforms
state-of-the-art GNN models in node classification, link prediction, and
graph classification tasks on benchmark datasets (Cora, PubMed, CiteSeer,
MUTAG, and PROTEINS).

In the following sections, we detail the GKAN architecture and its components,
present experimental results to validate our approach, and discuss the
interpretability of GKAN compared to existing methods. Our findings suggest that
GKAN represents a significant advancement in the development of accurate and
interpretable GNNs, with potential applications in various fields where
interpretability is fundamental. (^1^11We will make the code available upon publication.)

## II Methodology

This section details the construction
and operation of our proposed GKAN model. We begin by revisiting the key
elements of Kolmogorov-Arnold Networks (KANs), upon which GKAN is built.

### II-A Kolmogorov–Arnold Networks

The core idea of GKAN lies behind KANs. Kolmogorov–Arnold theorem states that
a multivariate continuous function in a bounded domain can be rewritten using a
finite composition of continuous functions on one single variable and the
addition operation. Given $\mathbf{x}$ a vector of dimension $n$, $f$ a function
such that $f:\left[0,1\right]^{n}\rightarrow\mathbb{R}$, it is thus possible to
write:

$$ $f(\mathbf{x})=\sum_{q=1}^{2n+1}\Phi_{q}\left(\sum_{p=1}^{n}\phi_{q,p}(x_{p}) \right),$ (1) $$

where $\phi_{q,p}:\left[0,1\right]\rightarrow\mathbb{R}$ and $\Phi_{q}:\mathbb{R}\rightarrow\mathbb{R}$. Liu et al. [] extended
Equation (1), representing a two-layer KAN with $2n+1$ terms
in the hidden layer, to larger depths and widths, parametrizing each
one-dimensional function as a B-spline curve. This kind of neural network has an
activation function on the edges instead of nodes; the latter simply perform a
summation. From the implementation side, KANs activation functions $\phi(x)$ are
built as a sum of a basis function $b(x)$ and the spline function such that:

$$ $\phi(x)=w_{b}(x)+w_{s}\,\mathrm{spline}(x).$ (2) $$

The original KAN model uses

$$ $b(x)=\mathrm{silu}(x)=\frac{x}{(1+e)},\,\mathrm{spline}(x)=\sum_{i}c_{i}B_{i}( x).$ (3) $$

At initialization, all activation functions are such that $w_{s}=1$ and
$\mathrm{spline}(x)\approx 0$ and $w_{b}$ are initialized using Xavier
initialization. The simplified interpretable KAN is the one at the basis of our
GKAN.

To maximize KANs’ interpretability, some simplifications are needed. The
interpretable KAN is trained with sparsity regularization techniques and
pruning. They use the L1 norm of the activation functions with entropy
regularization to favor sparsity. Once the network has been trained with a
sparsification penalty to reduce the number of activation functions, pruning
occurs at the node level. Reducing the number of active neurons renders
interpretation of the network more straightforward. To prune nodes, two scores
are defined, namely the incoming and the outgoing scores. If both scores are
above a certain threshold ($10^{-2}$ in the default implementation), a node is
considered important and, thus, preserved; otherwise, it is pruned. An
additional adaptation made by Liu et al. consists of symbolification. In the
simplified KAN, if an activation function is suspected to be a symbolic
function, such activation is set to be that symbolic function with shift and
scaling of input and output learned with grid search; for more details we refer
the reader to the original KAN paper [].

Figure: Figure 1: A simplified graphical representation of the GKAN model. The hidden layers are GKAN convolutional layers, in which messages are propagated and then aggregated, the output layer instead is a KAN linear layer. Each neuron in the hidden and output layers has its own set of learnable splines.
Refer to caption: x1.png

### II-B Kolmogorov–Arnold Graph Neural Networks (GKAN)

Building upon the strengths of KANs, we introduce GKAN (Graph Kolmogorov-Arnold
Network), a novel GNN architecture (Figure 1) designed for
processing graph-structured data. GKAN leverages the flexibility, accuracy, and
interpretability of KANs while retaining the efficient message passing
mechanisms of GNNs.

#### II-B 1 GKAN Architecture

GKAN employs learnable spline-based activation functions on the edges of the
graph, allowing for flexible nonlinear transformations of node features based on
their connections. The architecture is composed of multiple layers:

- •
KAN-based convolutional layers: Each layer efficiently
handles the propagation and aggregation of messages between nodes by
applying a KAN-based transformation to the features of each node, taking
into account information from its neighbors.
- •
KAN-based linear layers: A final linear layer performs a
linear transformation on the aggregated node features, producing the final
output of the network (e.g., node labels for classification).

The Xavier uniform initialization, also known as Glorot initialization, is a
widely used method for initializing the weights of neural networks. This
technique aims to maintain the variance of the gradients approximately the same
across all layers, thereby mitigating the vanishing and exploding gradient
problems. The weights are initialized by sampling from a uniform distribution in
the range $[-r,r]$, with $r=\sqrt{\frac{6}{n_{in}+n_{out}}}$, where $n_{in}$
and $n_{out}$ denote the number of input and output units in the weight
tensor, respectively []. As suggested in the KAN
paper, also in GKAN the basis function weights are initialized using the Xavier
initialization, facilitating effective training and optimization.

#### II-B 2 Mathematical Formulation of GKAN

In the next part of this section we will go through the basic mathematical
formulation of the constituent steps of GKAN.

Message Passing and Aggregation: Each node $i$ in the graph has an
initial feature vector $\mathbf{x}_{i}$. For each layer $l$ in the GKAN,
the node representations are updated through message passing and aggregation.
The message from node $j$ to node $i$ at layer $l$ is denoted as $\mathbf{m}_{j\to i}^{(l)}$.

- 1.
Message Passing:
$\mathbf{m}_{j\to i}^{(l)}=\mathrm{spline}(\mathbf{x}_{j}^{(l-1)})$
where $\mathrm{spline}(\cdot)$ denotes the spline-based activation
function applied to the features of node $j$.
- 2.
Aggregation: The aggregated message for node $i$ is
obtained by summing or averaging the messages from its neighbors $\mathcal{N}(i)$:
$\mathbf{a}_{i}^{(l)}=\sum_{j\in\mathcal{N}(i)}\mathbf{m}_{j\to i}^{(l)}$
or
$\mathbf{a}_{i}^{(l)}=\frac{1}{|\mathcal{N}(i)|}\sum_{j\in\mathcal{N}(i)}
\mathbf{m}_{j\to i}^{(l)}$

Node Update: The node representation is then updated using the
aggregated message and a spline-based transformation:

$$ $\mathbf{x}_{i}^{(l)}=\mathrm{spline}(\mathbf{a}_{i}^{(l)})$ $$

Spline-Based Activation Function: The spline-based activation function
$\varphi$ used in GKAN is defined as:

$$ $\varphi(x)=w_{b}b(x)+w_{s}\mathrm{spline}(x)$ $$

where $b(x)$ is a basis function (e.g., SiLU), and $\mathrm{spline}(x)$
is a B-spline function:

$$ $\mathrm{spline}(x)=\sum_{i}c_{i}B_{i}(x)$ $$

with $B_{i}(x)$ being the basis splines.

GKAN Layer: Each GKAN layer combines these steps, resulting in the
following layer-wise update rule for node $i$:

$$ $\mathbf{x}_{i}^{(l)}=\varphi\left(\sum_{j\in\mathcal{N}(i)}\varphi(\mathbf{x}_ {j}^{(l-1)})\right)$ $$

Output Layer: After passing through multiple GKAN layers, the final
node representations are transformed using a KAN-based linear layer for tasks
like classification or regression:

$$ $\mathbf{z}_{i}=\mathrm{KANLinear}(\mathbf{x}_{i}^{(L)})$ $$

where $L$ is the number of layers and $\mathrm{KANLinear}$ applies a
final spline-based transformation.

#### II-B 3 Overall Model

The overall model can be summarized as:

- 1.
Initialize node features $\mathbf{x}_{i}$.
- 2.
For each GKAN layer $l=1,\ldots,L$:
(a)
Compute messages $\mathbf{m}_{j\to i}^{(l)}$ for each edge.
(b)
Aggregate messages $\mathbf{a}_{i}^{(l)}$ for each node.
(c)
Update node features $\mathbf{x}_{i}^{(l)}$ using spline-based
activation functions.
- 3.
Apply a KAN-based linear layer to obtain final node representations.

This section provided a comprehensive overview of the GKAN architecture and its
core components. The next sections will delve into the experimental results and
demonstrate the effectiveness and interpretability of this new approach to GNNs.

## III Experiments

### III-A Datasets

We evaluated GKAN on node classification, graph classification, and link
prediction tasks using the benchmark datasets summarized in
Table I. In the following section, we will discuss the
experimental setup and the results obtained by GKAN in comparison to
state-of-the-art methodologies.
Dataset are available from the Planetoid
(^2^22[https://pytorch-geometric.readthedocs.io/en/stable/generated/torch_geometric.datasets.Planetoid.html](https://pytorch-geometric.readthedocs.io/en/stable/generated/torch_geometric.datasets.Planetoid.html))
and TUDadates
(^3^33[https://pytorch-geometric.readthedocs.io/en/2.5.1/generated/torch_geometric.datasets.TUDataset.html?highlight=tudataset](https://pytorch-geometric.readthedocs.io/en/2.5.1/generated/torch_geometric.datasets.TUDataset.html?highlight=tudataset))
libraries of Pytorch Geometric [].

**TABLE I: Summary of benchmark datasets. Datasets were split in train, validation, and teset sets with proportions of 80%, 10%, and 10%, respectively.**
|  | Node Classification and Link Prediction Datasets |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
| name | #nodes | #edges | #features | #classes |  |
| Cora | 2,708 | 10,556 | 1,433 | 7 |  |
| PubMed | 3,327 | 9,104 | 3,703 | 6 |  |
| CiteSeer | 19,717 | 88,648 | 500 | 3 |  |
|  | Graph Classification Datasets |  |  |  |  |
|  | #graphs | #nodes | #edges | #features | #classes |
| MUTAG | 188 | $\sim$17.9 | $\sim$39.6 | 7 | 2 |
| PROTEINS | 1,113 | $\sim$39.1 | $\sim$145.6 | 3 | 2 |

### III-B GKAN Performances

We compare the performances of our proposed model against established GNN
architectures, namely GCN, GAT, and GraphSAGE. GKAN was able to outperform all
the mentioned GNN models in all tasks with all datasets, with the single
exception of the link prediction with PubMed, in which GCN performed better.

Each method is unique, and the hyperparameters chosen for training are as
crucial as the implementation itself. To ensure a fair comparison, each model
has been trained following the guidelines and hyperparameters provided by their
authors. For training GKAN, we conducted hyperparameter fine-tuning on the Cora
dataset; we then use these hyperparameters to train GKAN also on the other two
citation datasets, PubMed and CiteSeer. This hyperparameter tuning strategy was
also reported in previous work []. Similarly, in the compared
methods, we use their optimal hyperparameters suggested for the Cora dataset to
train, evaluate, and compare on PubMed and CiteSeer.

Hyperparameter fine-tuning was conducted using a grid search over a broad range
of potential values: learning rate $[0.01,0.001,0.005]$, hidden channels
$[8,16,32,64]$, dropout rate $[0,0.3,0.6,0.7]$, number of layers
$[1,2,3,4]$, spline grid size $[2,3,4,8,10,20,30]$, splines degree
$[1,2,4,8,10,15]$, aggregation function $[add,mean]$,
and L2-regularization $[0.0001,0.00001]$.
The optimal hyperparameters that we found for GKAN in the node classification
task are as follows: a dropout rate of 0.7, a learning rate of 0.001, 2 layers,
32 hidden channels, a weight decay of 1e-4, the mean function for
aggregating the messsages, a spline grid size of 3, and a spline degree of 2.
For link prediction, the dropout rate is set to 0.2 and the weight decay to
1e-5. For the graph classification task, the best hyperparameters found (on the
MUTAG dataset) are: dropout rate of 0, learning rate of 0.001, 3 layers, 32
hidden channels, a weight decay of 1e-4, and the add aggregation
function. The results summarized in Table II are
obtained by averaging over 100 runs the test accuray (or AUC-ROC for link
prediction) of the models that achieved the highest validation accuracy during
training over each run.

**TABLE II: Results in terms of accuracy (node and graph classification) and AUC-ROC (link prediction) obtained by GKAN and the compared architectures. Our framework delivers higher accuracy and ROC, being the top-performing architecture.**
| Dataset | GCN | GAT | GraphSAGE | GKAN (Ours) |
| --- | --- | --- | --- | --- |
| Node Classification |  |  |  |  |
| Cora | 76.3 | 78.9 | 73.4 | 81.2 |
| PubMed | 77.4 | 78.2 | 73.9 | 81.0 |
| CiteSeer | 66.7 | 68.2 | 62.3 | 69.4 |
| Link Prediction |  |  |  |  |
| Cora | 79.3 | 81.6 | 80.6 | 83.5 |
| PubMed | 90.6 | 86.3 | 82.4 | 82.3 |
| CiteSeer | 78.7 | 78.1 | 78.3 | 80.3 |
| Graph Classification |  |  |  |  |
| MUTAG | 71.3 | 75.1 | 71.1 | 85.0 |
| PROTEINS | 73.8 | 72.5 | 70.7 | 77.7 |

The models were all trained for up to 300 epochs, utilizing early stopping with a window size of 10.
Training a single epoch of GKAN takes significantly longer compared to other
GNN models (Table III) due to its design and operational
complexity. GKAN employs learnable spline-based activation functions, which
require additional computational resources. This results in more precise and
interpretable transformations but increases computational complexity. The
sophisticated message passing and aggregation process, involving spline
transformations, adds further computational cost. Thus, even though GKAN offers
improved interpretability and accuracy, these benefits come with the trade-off
of increased training time.

**TABLE III: Average time to complete one training epoch on the Cora dataset for each model. These times were measured while training the models on an Apple Silicon M1 processor.**
| Method | Avg time (s) |
| --- | --- |
| GCN | 0.0016 |
| GAT | 0.020 |
| GraphSAGE | 0.065 |
| GKAN (ours) | 2.052 |

### III-C Interpretability

Figure: Figure 2: Interpretability of GKAN at 2 hops from a target node (node with ID 4). The model was trained on the Cora dataset, and in this figure, we are visualizing the weights learned by the splines on the edges to achieve the prediction. It is possible to visualize the direct values that lead to a particular prediction for a number of hops up to the number of layers of the network, as this is the extent to which the messages are aggregated. For visualization purposes, the weights have been normalized.
Refer to caption: x2.png

The added value of GKAN lies not only in its higher accuracy and effectiveness
but also in its inherent interpretability Figure 2. Built upon the simplified and
interpretable KAN, GKAN allows for understanding its predictions without relying
on external explainers, which can sometimes provide biased or approximated
explanations []. As a representative example, we present an
interpretation of a node’s prediction by GKAN in a node classification problem,
and compare it with an explanation generated by GNNExplainer on a GCN prediction
for the same node (Figure 3).
It could be said that in GKAN, the values (which have been normalized
for comparison purposes) truly represent the message of the network,
meaning the values used to arrive at the prediction. In contrast,
in GNNExplainer, they are importance scores calculated post hoc on
a subgraph that maximizes mutual information and do not necessarily
reflect what the network has learned.

Spline-based activation functions, which are central to KAN and consequently
GKAN, are inherently more interpretable compared to traditional neural network
activation functions. Traditional GNNs use fixed, nonlinear activation functions
like ReLU or Sigmoid, which can make it challenging to understand the decision
boundaries or the transformations applied to the input features. These
activations are often considered as “black boxes” because they do not provide
a straightforward way to interpret how input features are transformed and
combined to make predictions.

In contrast, spline functions are defined as piecewise polynomials which can be
easily visualized and understood. The smooth and continuous nature of splines
allows for a clear representation of how inputs are mapped to outputs. By
examining the spline functions, one can see exactly how each input feature
contributes to the final prediction. This transparency makes it possible to
trace the influence of individual features and understand the model’s
decision-making process.

In traditional GNNs, post-hoc explainability techniques like GNNExplainer
attempt to infer the importance of nodes and edges after the model has made its
predictions. These solutions can have methodological bias resulting from the different strategies employed to determine the explanations (e.g., mutual
information maximization, Shapley value estimation, etc.) and may not accurately
represent the underlying decision process of the model. On the other hand,
GKAN’s intrinsic interpretability provides a more reliable and accurate
understanding of the model’s behavior, as it directly reflects the structure and
parameters used during training.

In summary, its inherent interpretability makes GKAN particularly valuable in
applications where understanding the decision-making process is crucial, such as
in biomedical and financial domains.

Figure: (a) Direct interpretation of GKAN by plotting the values of the splines of the last convolutional layer.
Refer to caption: x3.png

## IV Conclusions

In this paper, we propose GKAN, a novel GNN architecture inspired by the
Kolmogorov–Arnold theorem and based on the recently introduced KAN model. The
added value of GKAN is two-fold. First, it is more accurate than established
state-of-the-art GNNs in node and graph classification and link prediction
tasks. Second, thanks to the usage of splines and the simple aggregation
(addition or average) functions, GKAN models are inherently interpretable. As we
showed in our experiments, the message carried by the graph edges represents,
indeed, in a clear and direct way, what the neural network used to deliver its
predictions. GKAN extends the opportunity offered by KAN to render neural
network models interpretable also to GNNs, opening up the possibility of using
GNNs in scenarios in which interpretability is a strong requirement, as in
biomedical and life sciences applications.

### IV-A Limitations and Interpretability

While GKAN exhibits promising results, it faces certain
limitations that requires further investigation:

- •
Memory Usage: Scaling the number of splines or
increasing their degree leads to a significant increase
in memory consumption. This can pose a challenge when
working with large graphs or when higher-degree splines
are required to capture complex relationships.
- •
Lack of Edge Features: The current implementation
of GKAN does not utilize edge features. Many real-world graph
datasets contain valuable information about edges, and
incorporating this information could potentially enhance
model performance.
- •
Interpretability: : We want to make clear that we do not claim
that GKAN is interpretable, but that it is more interpretable
than other models. For instance, if the neural network is deep
(large number of layers) then some of the information is lost.
The added value is that the intepretation is given by the model
itself and not by a surrogate explainability model.

We are actively exploring strategies to address these limitations and improve GKAN’s efficiency, adaptability, and interpretability.

### IV-B Future Work

To further improve and expand the applicability of GKAN, we propose the
following directions for future research:

- •
Framework optimization: Optimize the computational efficiency
of GKAN. This includes reducing memory usage and improving the speed of
training and inference. Techniques such as more efficient spline
implementations, approximation methods, and parallelization strategies
could be explored.
- •
Incorporating edge features: Extend GKAN to handle edge
features. This would involve developing new methods to integrate edge
features into the spline-based message passing and aggregation process,
thereby leveraging additional information that could enhance the model’s
predictive power.
- •
Real-world applications: Apply GKAN to real-life problems
where its increased performance and inherent interpretability can bring
significant improvements. Potential application domains include
biomedical research, where understanding the model’s decision-making
process is crucial, and financial analytics, where interpretability can
aid in regulatory compliance and trust.
- •
Scalability studies: Investigate the scalability of GKAN to
very large graphs. This includes exploring distributed training
approaches and evaluating the model’s performance on large-scale
datasets.
- •
Hybrid models: Explore hybrid models that combine GKAN with
other advanced GNN architectures, such as attention mechanisms or
variational techniques, to further boost performance while maintaining
interpretability.

By addressing these limitations and exploring these future research directions,
GKAN can be further developed into a robust and versatile tool for a wide range
of graph-based machine learning tasks, combining high accuracy with clear
interpretability.

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 5
presentation: 2
questions: 1. "This kind of neural network has an activation function on the edges instead of nodes" What is the edges and nodes here? Are they the same as the edges and nodes in graph?

2. "KANG employs learnable spline-based activation functions on the edges of the graph" Is this similar to graph attention network or graph transformer? What is the difference and what is your advantage?

3. How does KANG perform on heterophilic datasets, especially on those challenging datasets identified in [1]? And how does KANG compare with some SOTA models instead of the baseline models, e.g. [2,3].

4. How does KANG compare with other GNN interpretability methods, e.g. [4]?




[1] The heterophilic graph learning handbook: Benchmarks, models, theoretical analysis, applications and challenges. arXiv preprint arXiv:2407.09618. 2024 Jul 12.

[2] Revisiting heterophily for graph neural networks. Advances in neural information processing systems. 2022 Dec 6;35:1362-75.

[3] Simplifying approach to node classification in graph neural networks[J]. Journal of Computational Science, 2022, 62: 101695.

[4] Miao S, Liu M, Li P. Interpretable and generalizable graph learning via stochastic attention mechanism. InInternational Conference on Machine Learning 2022 Jun 28 (pp. 15524-15543). PMLR.
soundness: 2
strengths: originality: medium
quality: medium
clarity: medium
significance: medium
summary: The paper introduces the Kolmogorov-Arnold Network for Graphs (KANG), a new Graph Neural Network (GNN) model that enhances interpretability and accuracy in tasks like node classification, link prediction, and graph classification. Utilizing spline-based activation functions on edges, KANG not only improves performance on benchmark datasets but also allows for better insight into decision-making processes, eliminating the need for external explainability techniques. This approach is especially beneficial in domains where understanding model decisions is crucial.
weaknesses: 1. It's better to split the second paragraph in the first section for readability.

2. The paper borrows too much content from the original KAN paper and makes incremental modification.

3. In section 2.2.2, where is $m_j^{(l)}$ in the following equations? It seems that this notation is defined but not used. Check out the consistency of the notations.

---

confidence: 5
contribution: 2
initial_rating: 1
presentation: 2
questions: N/A
soundness: 2
strengths: 1. Applying KANs to GNNs is intriguing, and the writing is clear. The core idea of the paper is easy to grasp.

2. The experimental results demonstrate the superior performance of KANs.
summary: This paper introduces Kolmogorov-Arnold Networks (KANs) for enhancing graph neural networks. The core idea is to apply KANs to the aggregation process, replacing linear transformations with KANs.
weaknesses: 1. I don't understand why a widely used benchmark (https://paperswithcode.com/sota/node-classification-on-cora-with-public-split) was not employed for evaluation.

2. The idea lacks novelty.

---

confidence: 4
contribution: 3
initial_rating: 3
presentation: 3
questions: 1. I would encourage the authors to perform a comprehensive evaluation of KANG's interpretability, considering datasets used by existing post-hoc GNN explanation methods, and more baselines such as PGExplainer and EdgeSHAPer.

2. It is interesting to see that KANG outperforms SOTA GNN models (GCN, GAT, GraphSAGE, GIN) in node classification, link prediction, and graph classification tasks. Can you provide more information about the models, such as hidden dimension, the total number of parameters?

3. In general KAN networks are difficult to train. Appendix A.4 shows that KANG scales very well. This is a bit surprising. Can you give more explanation?
soundness: 3
strengths: This is a timely paper to introduce KAN for graph representation learning, with a focus on increasing interpretability of existing GNN models. This makes it different from other papers in integrating KAN and GNNs (see a few examples in the second last paragraph of Section 1). KANG utilises information flow across the graph and different KAN layers to develop two new notions, feature influence and edge importance, for its interpretability. I appreciate their effort to use insightful information from KAN to enhance interpretability. The evaluation in the paper shows that KANG has strong performance in both accuracy and interpretability for node classification, link prediction, and graph classification tasks. Most parts of the paper are clearly written, and some suggestions on how to further improve the paper can be found in the next review section.
summary: This paper presents a novel graph neural network (GNN) model KANG, built on the recently emerged Kolmogorov-Arnold network (KAN). KANG has a KAN-based convolution layer for the propagation and aggregation of messages between nodes, and a KAN-based linear layer to perform a linear transformation on the aggregated node features. The main advantage of KANG is its inherent interpretability (thanks to KAN), making it different from existing post-hoc GNN explainers. For this purpose, KANG computes feature influence (leveraging interaction of the gradients and the spline weights at each KAN hidden layer) and edge importance (combing spline weights and activations of the two node features), and use both of them to interpret the outcomes of KANG. Through extensive experiments on five benchmark datasets, KANG is demonstrated to outperform SOTA GNN models in node classification, link prediction, and graph classification tasks. In particular, feature influence of KANG is analysed on the CORA dataset, and edge importance is also shown to be consistent with the explainability provided by GNNexplainer using the same dataset.
weaknesses: Overall, I like the idea of KANG, but I also notice the following weaknesses.

1. Presentation of the paper can be still improved. I give a few examples.

1.1 Section 2.2: Figure 1 can be improved to better illustrate the main idea of KANG. I cannot fully understand why node x6 is connected to the two dots in the part of KANG convolution.  

1.2 Section 2.2.2: ${\bf m}_j^{(l)}$ is defined, but not used in the following formulation. Should it be $x_j^{(l)}$? The notion $N(i)$ is used but not defined. If it is the set of direct neighbours of node $i$, the aggregation function at the KANG convolution layer does not take node $i$'s own feature into account. 

1.3 Section 2.3.3: I don't fully understand the weights ${\bf W}^{(l)}$ and how to compute mean(${\bf W}^{(l)} \cdot S_{mean}^{(l)}$).

1.4 When reading the paper, I found a few typos: "is it possible", "at layer". 

2. I believe the main contribution of KANG is about interpretability, but the evaluation of KANG's interpretability in this paper is insufficient.

2.1 The evaluation of feature influence and edge importance is only demonstrated on the CORA dataset. 

2.2. KANG is only compared with GNNExplainer. 

3.3 The paper claims that KANG interpretability is consistent with the explainability provided by GNNExplainer. Somehow, I am not fully convinced by this statement. For example, in Figure 2, node 2175 in Figure 2(a) is almost as important as node 2176, but in Figure 2(b), it has a much smaller importance than node 2176. Figure 9 in the Appendix has similar problems: node importances (values and ranks) are quite different for KANG's direct interpretation and GNNExplainer's explanation.

---

confidence: 4
contribution: 2
initial_rating: 3
presentation: 4
questions: 1) What is the synthetic dataset used in the scalability study of Appendix A.4? I didn't see it explained neither in the main paper nor in the appendix. Its description should be included in the paper.

2) Inspired by GNNs, you experimented with `average()`, `sum()` and  `max()` aggregation functions for KANG. Since the Kolmogorov–Arnold representation theorem holds for the `sum()`, the other two aggregation functions do not have the same theoretical guarantees. However, in Table 4, Appendix A2 you actually show that `mean()` is favoured. Is this statistically significant, or is it not an important hyperparameter to tune? Did you investigate what happens to the splines when different aggregations are used? Do you think this result extends to deeper networks and/or traditional KAN?
soundness: 1
strengths: 1) The main strength is the focus on the explainability of the proposed Network. KANs are known to be, by construction, more interpretable than standard Neural Networks [1]. Extending this to both feature and edge influence with virtually no cost is interesting and well-explained.

2) The authors decided to include brief discussion of other KAN-based GNNs at the beginning of the paper even if they are new and still non-peer-reviewed. This paragraph provides helpful context. 

3) The paper is clear and provides the code for reproducibility.
summary: This paper introduces KANG, a Graph Neural Network (GNN) model that incorporates Kolmogorov–Arnold Network Layers into the message-passing framework typical of GNNs. In KANG, messages from node $j$ are computed using spline-based activation functions in each layer. The paper claims that KANG 1) outperforms state-of-the-art GNN models on standard graph tasks (Cora, PubMed, Citeseer, MUTAG, and PROTEINS), and 2) provides interpretable results with negligible computational overhead.
weaknesses: ## Main Weaknesses
---

Having tested and re-run part of the provided code, I have several concerns regarding the empirical evaluation and some significant errors in the code:

1) I ran `comparison.py` to reproduce the Link prediction results for all models (GCN, GATv2, GraphSAGE, GIN, KANG) on the three datasets (Cora, PubMed and CiteSeer). My results differ significantly from those in Table 1. Here are the results I obtained:

| Dataset  | GCN        | GAT        | SAGE       | GIN        | KANG       |
|----------|------------|------------|------------|------------|------------|
| Cora     | 89.5 ± 0.7 | **90.5 ± 0.5** | 85.4 ± 2.4 | 80.8 ± 8.5 | 87.9 ± 0.6 |
| PubMed   | **94.2 ± 0.4** | 90.0 ± 0.2 | 86.4 ± 1.1 | 89.3 ± 0.5 | 85.2 ± 0.3 |
| CiteSeer | 85.3 ± 3.0 | **86.8 ± 2.6** | 82.5 ± 2.8 | 84.2 ± 1.0 | 85.9 ± 0.5 |

These results seem to contradict one of the two main claims of the paper:
"Our experiments [...] demonstrate that KANG outperforms state-of-the-art GNN models in node classification, link prediction, and graph classification tasks." (lines 13-15). I recommend the authors use more seeds (>= 30) and ensure that all runs converge (some results show high standard deviation, indicating potential convergence issues). 

2) In the scalability experiment (lines 357–367), the number of parameters is fixed across networks. This comparison is not fair, as models like GCN tend to require fewer parameters to obtain optimal performances and thus may scale more efficiently. The comparison should be made at fixed performance levels. On my GPU (NVIDIA RTX A5000), KANG's training time was approximately 10 times slower than GCN for Cora, PubMed, and CiteSeer, contradicting the claim that KANG only "slightly" increases computational cost (line 363).

3) For node classification, models are validated on the test set and tested on the validation set. This makes it difficult to compare results with existing literature, where models like GCN and GATv1 might outperform KANG on tasks for Cora and CiteSeer [2], [3].

4) GAT models typically perform better with additional attention heads [3]. The hyperparameter for attention heads should be tuned (e.g., in the set [2, 4, 8]).

5) In the link predictor experiment, average training time per epoch is calculated as ((end_time - start_time) / n_epochs). This calculation is invalid if early stopping is triggered.

## Minor Weaknesses
---

1) KANG introduces three additional hyperparameters (spline grid size, spline degree, and aggregation function). For the hyperparameter search space you considered, this addition makes the hyperparameter search 126 times slower. This fact should be highlighted.

2) The statement "For each run, to achieve unbiased outcomes, we randomly split the datasets, utilizing 80% for training, 10% for validation, and 10% for testing" (lines 355–356) is not accurate for all experiments.

3) The `node_classification.py` script is located only in the `del` folder and throws an error when run both inside and outside the folder. I did not investigate why.

4) There are few typos in the paper:
- In Equation (2) `b()` is missing
- In lines 225-226 change "incluence" with "influence"

---

confidence: 3
contribution: 2
initial_rating: 3
presentation: 2
questions: - The authors should provide clearer motivations to explain why combining KAN and GNNs is necessary.
- The time and space complexity of KANG should also be provided in method part.
- More recent GNN baseline methods should be introduced in the experimental part for fair comparisons, such as rLap [1], UGNN [2], and PANDA [3] for graph classification, as well as GRLC [4], SEK [5] and UGT [6] for node classification.

[1] Kothapalli V. Randomized schur complement views for graph contrastive learning[C] International Conference on Machine Learning. PMLR, 2023: 17580-17614.

[2] Luo X, Zhao Y, Qin Y, et al. Towards semi-supervised universal graph classification[J]. IEEE Transactions on Knowledge and Data Engineering, 2023, 36(1): 416-428.

[3] Choi J, Park S, Wi H, et al. PANDA: Expanded Width-Aware Message Passing Beyond Rewiring[C] Forty-first International Conference on Machine Learning.

[4] Peng L, Mo Y, Xu J, et al. GRLC: Graph representation learning with constraints[J]. IEEE transactions on neural networks and learning systems, 2023.

[5] Yao T, Wang Y, Zhang K, et al. Improving the expressiveness of k-hop message-passing gnns by injecting contextualized substructure information[C] Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2023: 3070-3081.

[6] Lee O J. Transitivity-Preserving Graph Representation Learning for Bridging Local Connectivity and Role-Based Similarity[C] Proceedings of the AAAI Conference on Artificial Intelligence. 2024, 38(11): 12456-12465.
soundness: 2
strengths: - Different from traditional GNN methods, KANG can inherently provide interpretability for GNNs, which is particularly valuable for transparency. Additionally, KANG integrates interpretability directly into its architecture, which removes the need for additional post-hoc explainability tools.
- Moreover, the spline-based activation functions in KANG provide increased flexibility in modeling complex relationships in graph data, which helps capture nonlinear interactions more effectively and contributes to inprove the performance and interpretability of KANG.
summary: In this paper, the authors propose a novel graph neural network (GNN) model named KANG, which combines Kolmogorov–Arnold Networks (KANs) with GNNs and employs spline-based activation functions to enhance the accuracy and interpretability of GNNs. Different from existing methods, KANG can inherently provide interpretability, which is suitable for transparent decision-making. Experiments are conducted on several benchmark datasets, which demonstrates that KANG can outperform existing GNN methods in various graph tasks, such as node classification, link prediction, and graph classification.
weaknesses: - The writing of this paper could be improved. For instance, the introduction is overly long and lacks clarity, which makes it difficult for readers to understand the key motivations and contributions of the work. 
- The motivation of this paper is unclear, which is challenging to understand the specific limitations of existing GNN methods. KANG emphasizes enhanced interpretability and performance, but it does not adequately highlight the practical issues or real-world scenarios.
- The novelty of KANG is limited, as it simply combines Kolmogorov–Arnold Networks (KANs) with GNNs. Although this integration enhances interpretability and performance of GNNs, it does not introduce fundamentally new mechanisms or theoretical advancements beyond those already present in KANs and existing GNN architectures.
- The experimental results are not convincing, and the baseline methods are not strong enough. Although the authors compare KANG with widely used GNN methods like GCN, GAT, GIN, and GraphSAGE, these methods are neither the latest nor the most competitive, which undermines the significance of the reported performance.

Meta-review: This submission presents KANG, a novel Graph Neural Network (GNN) model that integrates Kolmogorov-Arnold Networks with GNNs and utilizes spline-based activation functions. The primary goal of KANG is to enhance the accuracy and interpretability of GNNs, enabling transparent decision-making processes. The authors evaluate their model on several benchmark datasets, highlighting its potential benefits.

Reviewers praised the clarity of the writing in some aspects, but also suggested improvements in presentation and notation. There was a consensus among reviewers that applying Kolmogorov-Arnold Networks to GNNs is an intriguing idea with significant potential, and KANG's inherent interpretability is a notable advantage, as it eliminates the need for external explainability techniques and makes it suitable for high-stakes domains.

However, the reviewers raised several concerns. Specifically, they noted that the computational overhead of KANG is a significant drawback, and the evaluation of its interpretability is insufficient. Additionally, some reviewers felt that the comparison to baseline models was inadequate, and there were technical issues with notation and formulas that need to be addressed.

To strengthen this submission, the authors should prioritize addressing these concerns, particularly by providing more comprehensive evaluations of KANG's interpretability, improving computational efficiency, and enhancing the clarity and rigor of their presentation. With revisions that address these issues, KANG has the potential to make a meaningful contribution to the field of GNNs.

Decision: Reject