The Intelligible and Effective Graph Neural Additive Network
Authors: Maya Bechler-Speicher
Amir Globerson
Ran Gilad-Bachrach
Venue: NeurIPS 2024 poster (2024)
Keywords: Explainable AI
Graph Neural Networks
Graph Learning
Interpretability.
Primary area: interpretability_and_explainability
TL;DR: We present an interpretable by design GNN, that is a novel extension of Generalized Additive Models to graphs.

Abstract: Graph Neural Networks (GNNs) have emerged as the predominant approach for learning over graph-structured data.  However, most GNNs operate as black-box models and require post-hoc explanations, which may not suffice in high-stakes scenarios where transparency is crucial.
In this paper, we present a GNN that is interpretable by design.  Our model,  Graph Neural Additive Network (GNAN), is a novel extension of the interpretable class of Generalized Additive Models,  and can be visualized and fully understood by humans. GNAN is designed to be fully interpretable, offering both global and local explanations at the feature and graph levels through direct visualization of the model. These visualizations describe exactly how the model uses the relationships between the target variable, the features, and the graph. We demonstrate the intelligibility of GNANs in a series of examples on different tasks and datasets. In addition, we show that the accuracy of GNAN is on par with black-box GNNs, making it suitable for critical applications where transparency is essential, alongside high accuracy.

The Intelligible and Effective Graph Neural Additive Networks
Maya Bechler-Speicher Blavatnik School of Computer Science Tel-Aviv University
Amir Globerson
Blavatnik School of Computer Science Tel-Aviv University"
Ran Gilad-Bachras
Department of Bio-Medical Engineering
Edmond Safra Center for Bioinformatics Tel-Aviv University
Abstract
Graph Neural Networks (GNNs) have emerged as the predominant approach for learning over graph-structured data. However, most GNNs operate as black-box models and require post-hoc explanations, which may not suffice in high-staker scenarios where transparency is crucial. In this paper, we present GNN that is interpretable by design. Our model, Graph Neural Additive Network (GNAN), is a novel extension of the interpretable class of Generalized Additive Models, and can be visualized and fully understood by humans. GNAN designed be fully interpretable, offering both global and local explanations at the feature and graph levels through direct visualization of the model. These visualizations describe exactly how the model uses the relationships between the target variable, the features, and the graph. We demonstrate the intelligibility of GNANs in a series of examples
different tasks and datasets. In addition, accuracy of GÑAN is on par with black-box GNNs, making it suitable for critical applications where transparency is essential, alongside high accuracy.
show that the
Introduction
In many domains, ranging from biology to fraud detection, Artificial Intelligence (AI) is applied to data with graph structure. Neural Networks, and specifically Graph Neural Networks (GNNs), have emerged as the predominant approach these applications (see, for example, Zhou et al. [.). While GNNs demonstrate high accuracy, in terms of the correctness of their predictions, they often function as black-box models; thus, their decision-making processes are opaque. Transparency is vital for assessing potential biases safety risks, and is particularly critical in high-stakes areas such as criminal justice, healthcare, and finance, where decisions significantly impact individuals' lives. In such contexts, interpretable models, despite sometimes being less accurate, may preferred over complex black-box models [2]. Furthermore, the transparency of automated decision making processes increasingly becoming legal mandate. While there is ongoing debate over whether the European Union's 's General Data Protection Regulation (GDPR) implies a "right to explanation B 4) the proposed European AI Act explicitly addresses this issue, stating that "To address concerns related to opacity and complexity of certain Al systems and help deployers to fulfill their obligations under this regulation, transparency should be required for high-risk Al systems before they are placed on the market or put into service"
Now also at Google Research
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

In this context, interpretability refers to the ease with which human can understand the reasoning behind model decisions or the general logic of model's operation. It is important to distinguish between interpretability and explainability (2). Interpretability relates to models that are inherently comprehensible by design, while explainability pertains to post-hoe methods that elucidate aspects of black-box models [6]. These explanations often come without correctness guarantees 7 g and may not provide a complete description of the model and its predictions, potentially failing to expose hidden pitfalls [9 [1.
Methods for model explainability or interpretability can be categorized into local and global types. Local methods, such as SHAP [12] and LIME [3], elucidate individual predictions made by model, whereas global methods, such as feature- impoortaccee [14] and partial dependence plots [15). provide holistic insights about the model, i.e., explain the overarching logic of the model decision-making [16]. However, has been noted that local explainability methods may not consistently align with their global counterparts 17 Moreover, local explanations may be inadequate for verifying fairness and
other risks
In this work, we introduce the Graph Neural Additive Networks (GNAN), an interpretable-b--dsiii GNN that offers both transparency and accuracy. GNAN is glass-box model [8] that allows for both local and global interpretability. GNAN extends the family of Generalized Additive Models (GAMs) [9, to accommodate graph data. GAMs are known for their ability to fit complex, nonlinear functions while remaining interpretable and have proven effective across various domains [20] [2] They operate by learning shape functions for each feature and then linearly combining these functions, making it easy to interpret them, as the influence of each feature on the prediction independent of other features and can be visualized through their corresponding shape functions. Similarly, GNAN's interpretability is achieved through an architecture that restricts the use of cross- products features and graphs' topology, thereby reducing its complexity compared to other GNNs. Nonetheless, we demonstrate that GNAN, despite its limited capacity, matches the performance of more expressive GNNs on several real-world datasets. Additionally, GNAN does not rely on iterative local message-passing, avoiding the computational bottlenecks commonly associated with such GNNs [25].
22 23
In Section we showcase through series of examples how users can interpret GNAN and gain precise insights into the connections between the target and the graph, the target and the features, and the interplay between features and graph information. In some cases, an exact description of the model can be visualized through only a few figures. We also demonstrate how the interpretability of GNAN allows users to debug their model, process that can be used for ensuring consistency with prior knowledge and avoiding biases and safety risks. Section we compare the performance of GNAN with other GNN architectures. This comparison underscores that sacrificing performance for intelligibility is not necessary, as the performance GNAN comparable to that of commonly used box GNNs.
The main contributions of this work are:
1. The extension of Generalized Additive Models (GAMs) to graph data.
2. The introduction of a fully interprttable-b-desitt model for graph prediction tasks, demon- strating that its explanations provide both global and local insights, through visualizations of the model itself, and include debugging capabilities.
3. The demonstration that GNAN achieves good performance common real-world graph datasets, despite its limited capacity. This observation supports previous findings that some real-world graph problems are simple and do not require the capacity of other GNNs.
Thus we argue that GNAN is suitable for high-stakes applications due to its interpretability and performance.
2 Related work
Generalized Additive Models Generalized Additive Models (GAMs) are a class of statistical models that build upon generalized linear models by incorporating non-linear functions for each variable while maintaining additivity [19 20 21). Essentially, GAMs model the expected value of the target variable as sum of univariate functions of the features. Formally, in GAMs, prediction

---

for an input x is computed by a (Σ fo (x+)) where ø is predefined activation function, such as the sigmoid" and the f& 's are shape functions learned during the training process. This approach extends generalized linear models, in which predictions are computed by a iΣ WgXg)) where w is learned weight vector.
GAMs are more expressive than generalized linear models while remaining interpretable, as the effect of each predictor is modeled separately. For example, they can capture non-monotone effects of features, which generalized linear models cannot achieve without feature engineering. Traditionally, GAMs utilize splines or other smooth shape functions to model the non-linear relationships between each feature and the target variable. However, other methods, such as trees, have been proposed to fit the shape functions [24). Recently, Agarwal et al. (26) suggested using neural networks to learn the shape functions. This approach combines the representational power of deep learning with the interpretability of additive models.
have emerged as the leading approach for learning over graph data. The fundamental idea behind GNNs is to use neural-networe that combine node features with graph-structure. A commonly used family of GNNs is message- passing GNNs, where the representations of nodes are updated in iterations through neighborhood aggregations. This aggregation is done, for example, through a convolution-like operation or an attention mechanism. 31 32 33
Graph Neural Networks Graph Neural Networks (GNNs) P
23
29
30
Various mewssse--pssssnnee approaches have been explored to disentangle the node features from the graph structure. Such approaches were shown to enhance performance across diverse applications [34] 35 36.. Disentanglement can also reduce overfitting, as popular GNNs which do entangle features and
graph-structure, were shown to have the tendency to overfit non-informative graph information [77
GNAN uses these concepts in order to achieve model that is both high-performing and fully interpretable.
There are different prediction tasks on graphs [8. In graph tasks, the goal is to predict properties of entire graphs. For example, a graph could represent a molecule, and the goal would be to predict its toxicity level. In node tasks, the goal is to predict property of a node (vertex) within graph. An example of node task is predicting whether a user in a social network a human or bot. In link prediction tasks, the goal is to determine whether there is an edge between two nodes of a graph. In this work, we focus on graph tasks and node tasks. Although link prediction tasks are not within the scope of this work, it is possible to view these problems node tasks on the dual line graph [39]
GNNs explanations The inherent complexity of graph-structured data poses unique challenges for explainability. Most approaches for explaining black-box GNNs focus on providing sub-graph or similar structure that can explain a certain example. This is done either as a post-hoc explanation for GNNs 40. 41 42
or by adjusting the data a priory [43 44 45). For example, the method suggested
in Ying et al. 41 identifies both important subgraph structures and node features influencing the GNN's predictions by maximizing the mutual information between the prediction and the distribution
of possible subgraph structures and node features. Yin et al. 4S] suggested a structural pattern learning module that is learning through pre-training. GNAN on the contrary to these methods, does not aim to provide an explanation through a proxy object like a subgraph, nor does it require modification to the data, or the training process. Instead, GNAN a interpretable by design, and its exact description can be visualized through its learned shape functions. In particular, the exact relation between the target, the features, and the graph can be visualized and conveyed to users.
3 Graph Neural Additive Networks
In this section, we introduce the Graph Neural Additive Networks (GNAN). We begin by defining some essential notation. A graph G has set of N vertices, where each vertex is associated with a d-dimensional feature vector. Specifically, x; E IRd' represents the feature vector of the i'th node in G. We define the distance dist (j between node j and node i within the graph G as the number of edges in the shortest path from j to i. This definition implies that the distance from a node to
itself is zero. In cases where no path exists from j to i, we set dist (j co. For enhanced readability, we denote vectors boldface, and an entry of vector x denoted by [x]x- We begin
In the context of Generalized Linear Models (GLMs) a be though of as the inverse of the link function.
3

---

by describing GNAN for applications such as binary classification and regression where the model output is noe-dimensional. At the end of this section, we discuss extensions scenarios such as multi-class classification, where the model output is multi-dimensional.
GNAN generates representation h, E Rd for each node i by learning distance function p(a; R + IR and set of feature shape functions {fx fi: 0x)) R + R. Each of these functions is neural network, and is optimized through back- sack-propagation For brevity, we omit the parameterization 0 and 0% for the remainder of this section. In GNAN, the k'th entry of the representation h, for the
t 'tth node is defined as follows:
b.lk = Σ j=1 (dist,
P dist (i. fk ((x)l))
where #dist; (j. i) represents the number of nodes at distance dist (j. i) from node i. The underlying rationale for this definition is follows: each node's k'th feature transformed by shape function Su. independently from other features. The effect the k'th feature value of node j has on node i's representation is influenced by their distance. Specifically, if dist (j, I. then the cumulative influence of all nodes at distance from node i is captured by p(/((++)). This is achieved by the normalization term 1/mdist (j.e). Here, p is argument 1/(1+1) scales the distance such that a distance of 0 (the self-distance of a node) is mapped to 1, and an infinite distance, which implies no path exists, scaled O. Thus, P spans the interval [0, 1].
The representation of each node is dependent on the entire graph, yet the function p enables weighting the influence from nodes, based their distance. This enables, for example, diminishing the impact of distant nodes, or close neighbors. For each node i, the weighted sum of the transformed feature vectors of all other nodes is computed, with weights assigned according to their distance from i. This weighted sum is computed after the shape functions are applied to the distances and the features of each node.
Given the node representations, both node prediction tasks and graph prediction tasks can be imple- mented. To predict for the i'th node, the computation is as follows:
h
where the entry-wise sum of the representation vector h; is computed and subsequently processed using an activation function such as the sigmoid for classification and the identity for regression. For prediction over the entire graph, the collective node representation is computed via sum-pooling:
Following this, the entry-wise sum of the graph representation is computed and also processed using the activation function:
d
h
Once the model is trained, it can fully described using its univariate functions p and {f
From the definitions provided above, it follows that the entire model can be represented with just few figures, thus providing global interpretability. For local explanations, is feasible to examine the contribution of each feature and each node to the predictions. The following representation convey the influence of each node the k'ts feature:
N
N
hh2 k = 2
Σ fkr (1x)1k) j=1
Σ
#dist, (j. -p + (j.
i=1

---

0.0035
Distance function output
Feature function output
500
0.0030
400
0.0020
300
0.0015
200
0.0005
0.0000
Distance
Feature (atom)
Figure 1: Visualization the distance and feature functions, learned on Mutagenicity. As the features binary, the feature functions evaluated only the value 1. These plots provide an exact description of the functions' signal processing and a global explanation of how the model uses the distances and features.
Here [he]a contains the influence of the k'th feature in node i on the prediction. However, from the definition of [h,]æ we see that it serves as mediator for influences of all other nodes. Therefore, the influence node j on feature the final graph representation can be obtained by:
1
S& ((x)))) #dist, (j,
dist (j,
We can also extract the total contribution of each node i to the prediction, by summing the contribution of the nodes over the features, done in the input to in Equation
N
N
(T+ dist (j,
hi
Σ Su ((x)]a)
#dist (j.
i=
Therefore, the total contribution of node to the prediction is
Σ #dist o (1+ dist (j.
Σ J& ((x)le)
Overall, the model facilitates a detailed understanding of local behavior from multiple perspectives.
The functions and {ff Jd may be implemented using variety of neural network architectures. In our experiments, we employed multi-layer perceptrons (MLPs) with ReLU activations to implement these functions. Nonetheless, other alternatives are viable, such employing learning splines for activations to achieve smoother shape functions [46]. Additionally, it is feasible to develop a separate distance network for each feature to enhance the model's capacity. Specifically, rather than utilizing a single function p, one can train a distinct function Pk for each feature k, which weights the contribution of each feature based its node's 's distance. For graph-level tasks, additional feature networks may be integrated prior to aggregating the graph's representation vector, akin to a readout layer in GNNs. These extensions, along with discussion computation that facilitates efficient GPU utilization, are further elaborated in the Appendix.
tensor representation of the
For multiclass classification involving C classes, we configure the final layers of the feature shape functions [a; 0x) + cxx and the distance function p(a; 0) : > RCx1 to accommodate the required dimensionality. The transformed feature vectors and the distance metrics are combined using an element-wii wise multiplication denoted by @ as follows:
#dist( / (1 dist (j.8)
h
2 Sx((x)]e)
For prediction purposes, the sum operator is applied independently across the dimensions correspond- ing each class, and softmax is employed as the activation function.
5

---

4 Inteligibility
In this section, we demonstrate the intelligibility of GNAN through visualizations. Each GNAN model is characterized by the univariate learned shape functions p and {f&) and can thus be depicted as a set of illustrative figures. Below, we present examples of such figures and explain their utility in generating insights. Our focus in this section is on global interpretability. as local interpretability can utilize analogous ways. We showcase GNAN's application on two datasets, with additional examples detailed in the Appendix.
Our initial examples focus on the task of detecting mutation-causing molecules using the Mutagenicity dataset [47. In this task, molecules are modeled as graphs where nodes correspond to atoms and edges to connections between these atoms. Each atom type is represented by a 14-dimensional one-hot encoding. A GNAN model trained on this dataset is illustrated in Figure T On the left, the function p is presented, demonstrating how distance impacts prediction, with a clear diminishing influence of more distant atoms. On the right, the shape functions for the features are displayed. Given that the features are binary, each shape function manifests only two values: one when the feature is 0 (indicating that the atom is not of the specified type), and another when the feature is 1 (indicating that the atom is of the specified type). Defining b = Σ. fu (0) as the bias term allows us to set Sa (0) 0 for each k thereby enabling the plotting of only Sa (1). The graphical representation reveals that atoms such as Ca, Na, and are predicted to correlate with an increased mutagenicity effect, whereas N and P atoms predicted to be associated with slight protective effect.
It is essential to emphasize that Figure ! displays the entire model comprehensively. This means that combined with the value of the bias term, which is -.66672 in this case, every crucial detail needed to understand and utilize this model for predictions is contained within this single figure. This stands in stark contrast to methods like feature importance, which offer a limited perspective on models. While the figure provides complete information about the model, presenting additional views can sometimes be helpful.
Figure 2 showcases the cross product of the shape functions and the distance function as heatmap. Each cell (k, in the heatmap represents the value p (1/(++)) (1). This figure illustrates the interplay the model has learned between the graph's structure and the attributes of its nodes. As the task involves binary classification, positive values in the heatmap contribute to classifying molecule as mutagenic, whereas negative values indicate mon-mutaeenic properties.
This heatmap illustrates how atoms at specific distances influence the final outcome. For instance, it shows that the model has learned that the presence of a Ca atom (cell (Ca, 0)) or its proximity (cell (Ca, 1)) contributes to mutagenicity. The visualizations can also be used for debugging purposes. This can be crucial, for example. to ensure that the model is free from biases to identify any discrepancies with existing scientific knowledge. If it is already known that Ca atoms actually have a negative effect on mutagenicity, users could identify and correct this misalignment in the model's learning. Additionally, this detailed understanding allows users to select models that not only achieve high accuracy on the given samples but also align with prior knowledge, optimizing both performance and reliability.
Interpreting multiclass prediction tasks poses significant challenges, as noted by Zhang al. [6]. In this context, we showcase the interpretability of GNAN using the PubMed dataset [48] This dataset comprises 19,717 scientific publications related diabetes archived on PubMed and categorized into three distinct classes (type-1 diabetes, type-2 diabetes, and gestational diabetes). The dataset's citation network includes 44,338 links. Each publication, represented TF/IDF weighted word vector derived from a dictionary containing 500 unique words.
node, is characterized by
As there are three classes, we trained the GNAN model such that the output of the distance and feature functions are of dimension three. In this setting it is interesting to compare the three functions, corresponding to the three classes and therefore we draw them on the same figure (6] Figure 3 shows that the model uses only the local neighborhood of each node, and as nodes become more distanced, the information between them is less used. We also observe a difference between the classes; while for type diabetes, the longer the distance, the less their information is used (converges to 0), for type and gestational diabetes, nodes of long-distance have a negative effect.
In Figure 4 we display the feature shape functions for nine selected features, demonstrating GNAN's capability to learn complex, non-monotone functions such as those seen in the 'dict" and 'hepat'

---

1.75
00
140
50 38
1.90
(etom))
1.90
651 90 55 322 55.
551
CC) 330 E00 13
1200
Distanice
Visualization of products of the outputs of the distance function and the feature functions, trained on Mutagenicity. Each cell shows the exact contribution, positive negative, of features at a certain distance to the prediction. Positive values (green) contribute to classifying a molecule as mutagenic, and negative values (red) contribute classifying molecule non-mutagenic.
Figure 2:
Diabetes Type
Distanoe funstion outpot
0.8
Type
Type Disbeles
Gestatioral Diabetes
0.8
0.0
Distance
Figure 3: Visualization of the distance shape function learned on the PubMed dataset. As the output of the function is of dimension three, plot it as three shape functions, for each class. We plot them on the same figure to compare them. The shape functions show that the model uses only the local neighborhood of each node. also shows difference between the classes; while for type diabetes, the longer the distance, the less their information is used (converges to O), for type I and gestational diabetes, nodes of long-distance have negative effect.
features. Observing these shape functions across the three classes simultaneously allows for an understanding of how different feature values utilized by the model to distinguish among the classes. For instance, the shape function for the 'insulin' feature reveals that the absence this word in document (i.e., feature values close to zero) does not significantly indicate the document's 's class. However, as the frequency of 'insulin' increases within the document, its impact on the prediction becomes pronounced, though this effect varies distinctly between type & 2 diabetes and gestational diabetes.
To visualize the contribution of feature value specific distance, employ heatmap for each class, evaluating the products between the outputs of the feature function over the input range ((0, 1J) and the output of the corresponding distance function. Figure exemplifies this visualization technique with the "children' feature. It insightful to observe that the presence of the word 'children' influences the predictions differently across the diabetes types. The model has learned that papers concerning type 1 diabetes seldom mention 'children', nor do related papers. In contrast, the term frequently appears in the context of gestational diabetes.
possible to construct confidence intervals for GAMs using the bootstrap method 49 50 present such example with additional visualization examples in the appendix.
We
7

---

Festure Aunction ouppuc
Foature Runction ourqur
0/15
6.13
Frestor
24
insulin feature value
'chldren' feature value
'dief feature value
Folure Tundion 00/041
600
Fealure fandion
4.8
-41
022
66
64 Thepal teatune valun
'panen Inatura value
'SaX' Inaturn value
Feoture tinetto oolp.t
032
0.
Fosture
1212
03
02
60
65
60
"oaaff feature value
feature value
'lver' feature value
Daneee hape
Type disbates
Type diabetos
Gestational diabates
Figure 4: Visualization of nine features' shape functions, learned over the PubMed dataset.
Typo 1 Diabetes
Type 2 Diabetes
Gestatio nal Diobetes
020
0
0.10
005
IT Distance
Distanos
Distanos
000
-005
-0.10
-3:15
-020
86
45
'children' feature value
'children' feature velue
'children' festure value
Figure Visualization of the products between the outputs 'children' feature function over input range [0, 1] and the outputs of the distance function, learned over the PubMed dataset.
4.1 Local Explanations
Up to this point, we have demonstrated how to visualize GNAN for providing global explanations. These explanations offer a comprehensive visualization of the entire model. Now, we shift our focus to local explanations, i.c.. those relevant to particular examples of interest. To illustrate this, employ the same Mutagenicity model that was visualized globally to explain specific samples within the data. Using Equation 3 we compute the importance of each node and visualize two molecules from the dataset, where the area of each node corresponds to its importance.
Figure 6 presents two such examples. In Figure 6(a) the carbon (red) atoms play the most significant role, and carbon ring (red cycle) is highlighted. In Figure subgraph) is shown to be relatively important for predicting the molecule mutagenic. Both carbon rings and NO, groups are well-known for their mutagenic effects [5T], making them frequently discussed in the literature explainable GNNs 33 41
group of NO, (grey and green

---

(a) A molecule with carbon ring
(b) A molccule with NO₂ subgraph
Figure Local explanations of two molecules from the Mutagenicity datasets, through visualizations of the molecules. The area of each atom corresponds to the node importance according to Equation 3
Empirical evaluation
In this section, we evaluate GNAN on real-world graph and node labeling tasks, including large-scale, long-range, and heterophily datasets We compare GNAN to multiple commonly used black- box GNNs including GraphConv [52), GraphSAGE [30], Graph Isomorphism Network (GIN) 33
the expressive version of the Graph Attention Network (GATv2) (29 53). the Graph Transformer (GTransformer) [54]. We also evaluate the FSGNN model, which disentangles the node features from the graph structure [35. The information on the sypper-parametes tuned for each baseline can be
found in the Appendix. used the following common benchmarks:
Node labeling tasks Cora, Cileseer, PubMed, ogb-arxiv [5 [56] are paper citation networks where the goal is classify papers into one of several topics. The ogb-arxiv dataset is large-scale network. Cornell (57] & Tolokers (58) are heterophilious datasets. Cornell is web-link network with the task of classifying nodes into one five categories. Tolokers dataset based on data from the Toloka crowdsourcing platform. The nodes represent tolokers (workers) who have participated in at least one of 13 selected projects. An edge connects two tolokers if they have worked on the same task. The goal is to predict which tolokers have been banned in one of the projects. Node features are based on the worker's profile information and task performance statistics.
Graph labeling tasks NCII, Proteins, Mutagen & PTC [59] are datasets of chemical com- pounds. In each dataset, the goal is to classify compounds according to some property of interest. Thr μ,a HHHOM (60) datasets are long-range molecular property prediction regression tasks, over the large-scale QM9 molecular dataset.
Additional data information, including the data statistics, can be found in the Appendix.
Protocol For all tasks, used existing splits, protocols, and metrics, as commonly used in the literature for each dataset. The complete protocols for each dataset are given in detail in the Appendix. The metrics we report are: For Cornell, Cora, Citeseer, PubMed, ogb-arxiv, Mutagenicity, PTC, NCI, and Proteins, we report accuracy. For p., a and онома we report MAE, For Tolokers we report ROC-AUC. For the node labeling tasks, used the pre-defined splits in the data and followed the common protocols for each dataset. The results are an average of the test set using or 10 random seeds. For the Proteins and NCII tasks, we followed the splits and the cested-rross-validation protocol from [61]. The final reported result on these datasets is an average of 30 runs (10-folds and 3 random seeds). For NCII and PTC we followed the splits and protocol from and report the average accuracy and std of 10-fold nested cross-validation.
Results The results are presented in Table I GNAN performed the best or second- model in 9 out of the 13 tasks we evaluated. In GNAN, each node gathers information from all others, ensuring complete information flow, while the function modulates influence based on distance.
'The implementation can be found at httpss///gtaaaa.cco////aapllllle
Graph-Neural Neural- Souural-Addiiive wew.tiiveeeeewwrsss..o

---

Table 1: Evaluation of GNAN on node (top) and graph (bottom) tasks. The best and second-best models are marked in cyan and violet colors, respectively. We report accuracy and std for all tasks, except for the Tolokers dataset where we report ROC-AUC and std, and the μ,, a, &HOMO datasets where we report MAE and std.
Model
Cornell
Tolokers
Cora
Citeseer
PubMed
ogb-arxiv
GraphConv
65.9 0.5
83.5 a 0.7
81.3 1.1
75.9
85.9 a 0.5
72.4 + 0.1
GraphSAGE
75.9
82.4
81.4 0.7
76.4 + 0.8
8.44
71.7 + 0.2
GIN
69.0 1.3
8.4
80.0
1..1
8.9
73.8 1.4
GATv2
0.7
83.8
0.9
1.5
8.54
7.1
GTransformer
70.5
039
80.7
7..0
85.3
73.1 #.2
FSGNN
86.0
8..1
83.0 1.3
76.2
8.
72.9 1.7
GNAN
8.87
84.5
81.1 + 1.5
75.8 0.6
86.9 # 1.2
74.1 # 1.5
Model
a
Proteins
Mutagen
PTC
NC11
GraphCony
2.91
4.3
1.46
73.1
64.3 1.7
6.0
76.5 1.2
GraphSAGE
0.2
0.77
1.4
7.50
64.1 0.3
67.1=12.6
76.0
GIN
0.1
4.6
1.42
7.0
69.4 1.2
55.6211.1
80.0 1.4
GATv2
0.1
4.3
1.1
735528
7,9
80.4 59.5
GTransformer
0.3
4.5
0.2
4..9
73.1
3.5
80.5 a 1.1
FSGNN
0.3
4.5
0.3
7..9
66.9 1.5
6.2
79.7 1.1
GNAN
0.1
4.9
1.1
73.2
64.9 1.02
76.9
Consequently, GNAN avoids the computational bottlenecks encountered by some message-passing GNNs 255. Particularly in the long-range tasks a. and (HHMO. GNAN outperformed all other evaluated baselines, aligning with findings by Alon and Yahav (25 that emphasize the benefits of capturing long-range information. While intelligibility sometimes comes at the cost of accuracy, our findings suggest that enhancing intelligibility does necessarily result in significant accuracy loss. It may appear surprising that GNAN, despite its limited capacity, matches the accuracy of more expressive GNNs. However, prior research indicates that even limited-capacity GNNs, can achieve high accuracy on various real-world datasets [62] 61 63), suggesting that some real-world graph problems are simpler than anticipated. Our results corroborate these observations.
capacity GNNs, such as linear
Conclusion
In this work, we introduced the Graph Neural Additive Network (GNAN), a novel extension of the interpretable class of Generalized Additive Models, to accommodate graph data. GNAN is inherently interpretable, and provides both global and local explanations directly from its architecture, eliminating the need for post-hoe interpretations. This direct interpretability enhances the transparency of the model and is particularly useful in high-stakes applications where understanding model decisions is crucial. Furthermore, GNAN demonstrates competitive performance with popular GNNs, showing that intelligibility does not necessarily entail significant degradation in accuracy.
possible enhance GNAN in several ways. To generate smooth shape functions, one could integrate techniques from the recently proposed Kolmogorov-Armol/ Networks [46] or adaptive
activations for graphs [64]. Increasing the capacity of GNAN feasible by learning individual distance functions for each feature. Exploring reduced capacity is also intriguing, particularly in scenarios with many features, where it may be beneficial to employ regularization limit the number of shape functions used. Additionally, applying these techniques to biological network datasets, such as protein interactions, could be a valuable tool to support scientific discoveries. These and other directions are left for future studies.
Acknowledgements
This work was supported by the Tel Aviv University Center for AI and Data Science (TAD) and the Isracli Science Foundation grants 1186/18 and 1437/22.
10

---

References
[1] Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang. Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. Graph neural networks: A review of methods and applications. Al open, 1:57-81, 2020.
[2] Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead, 2019.
[3] Bryce Goodman and Seth Flaxman. European union regulations algorithmic decision decision-making and a "right to explanation". Al magazine, 38(3):50-577 2017.
[4] Andrew Selbst and Julia Powles. "meaningful information" and the right to explanation. In conference on faimess, accountabillity and transparency, pages 48 48. PMLR, 2018.
[5] European Parliament. European parliament legislative resolution of 13 march 2024 the
proposal for a regulation of the curopean parliament and of the council on laying down harmonised rules artificial intelligence (artificial intelligence act). OJ, 2024. URL https://www. http:://rrw..uroprrr...... europar..eeuurpaa europa. eu/ doceo dow.menttTA---22222202 2138222242218000000121 E...EN.po .df
[6] Xuezhou Zhang, Sarah Tan, Paul Koch, Yin Lou, Urszula Chajewska, and Rich Caruana. Axiomatic interpretability for multiclass additive models. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 226-234, 2019.
[7] Blair Bilodeau, Natasha Jaques, Pang Wei Koh, and Been Kim. Impossibility theorems for feature attribution. Proceedings of the National Academy of Sciences, 121())xe3344006120, 2024.
[8] Daniel Vale, Ali EI-Sharif, and Muhammed Ali. Explainable artificial intelligence (xai) post-hoc explainability methods: Risks and limitations in non-discrimination law. Al and Ethics, 2(4): 815-826, 2022.
[9] Rebecca Wexler. When a computer program keeps you in jail. The New York Times, 13:1, 2017.
[10] Michael McGough. How bad is sacramento's 's air, exactly? google results appear odds with reality, some say, Sacramento Bee, 7, 2018.
[11] Marzyeh Ghassemi, Luke Bakden-Rayned and Andrew L Beam. The false hope of current approaches explainable artificial intelligence health care. The Lancet Digital Health, (11):e745-e75) 2021.
[12] Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. Advances neural information processing systems, 30, 2017.
[13] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. why should trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135-1144, 2016.
[14] Sara Hooker, Dumitru Erhan, Pieter-Jan Kindermans, and Been Kim. Evaluating feature importance estimates. arXiv preprint arXiv: 10758, 2, 2018.
[15] Jerome H Friedman. Greedy function approximation: gradient boosting machine. Arnals of statistics, pages 1189-1232, 2001.
[16] Waddah Saeed and Christian Omlin. Explainable ai (xai): A systematic meta-survey of current challenges and future opportunities. Knowledge-Based Based Systems, 263: 10273, 2023.
[17) Gabriel Laberge, Yann Batiste Pequignot, Mario Marchand, and Foutse Khomh. Tackling the xai disagreement problem with regional explanations. In International Conference on Artificial Intelligence and Statistics, pages 2017-2022 PMLR, 2024.
[18] Valentina Franzoni. From black box to glass box: advancing transparency in artificial intelli- gence systems for ethical and trustworthy ai. International Conference on Computational Science and Its Applications, pages 118-130. Springer, 2023.

---

[19] Trevor Hastie and Robert Tibshirani. Generalized Additive Models. Statistical Science, 1 (3):297 310, 1986. doi: 1177013604
URL htttp:://doi.org/111 12te///11.11///111 1214/22///////
[20] Trevor Hastie and Robert Tibshirani. Generalized additive models for medical research. Statisti- cal methods in medical research, 4(3):187-16,, 1995.
[21] Trevor Hastie and Robert Tibshirani. Generalized additive models: Some applications. Journal of the American Statistical Association, 82(398):371-386 1987. doi: 10.10800062244991997, 10478440.
[22] Thomas W Yee and Neil D Mitchell. Generalized additive models in plant ecology. Journal of vegetation science, 2(5):587-602, 1991.
[23] Robert A Rigby and D Mikis Stasinopoulos. Generalized additive models for location, scale and shape. Journal of the Royal Statistical Society Series C: Applied Statistics, 54(3):507-554, 2005.
[24] Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marc Sturm, and Noemie Elhadad. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pages 1721-1730, 2015.
[25] Uri Alon and Eran Yahav. On the bottleneck of graph neural networks and its practical implications, 2021.
[26] Rishabh Agarwal, Levi Melnick, Nicholas Frosst, Xuezhou Zhang, Ben Lengerich, Rich Caruana, and Geoffrey Hinton. Neural additive models: Interpretable machine learning with neural nets, 2021.
[27] Thomas N. Kipf and Max Welling, Semi-supervised classification with graph convolutional networks. In International Conference Learning Representations, 2017. URL https
7 //openreview net/forummid (t-oorummmdd-SSAayyeee
[28] Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. Neural message passing for quantum chemistry, 2017.
[29] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In International Conference on Learning Representations, 2018.
[30] William L. Hamilton, Rex Ying, and Jure Leskovec. Inductive representation learning on large graphs, 2018.
[31] Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie-Ya Liu. Do transformers really perform bad for graph representation?, 2021.
[32] Chloe Wang, Oleksii Tsepa, Jun Ma, and Bo Wang. Graph-mamba: Towards rangrrnge graph sequence modeling with selective state spaces, 2024.
[33] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In International Conference on Learning Representations, 2019. URL hhtpps //openrevies
[34] Aseem Baranwal, Kimon Fountoulakis, and Aukosh Jagannath. Optimality of message-passing architectures for sparse graphs. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL
[35] Sunil Kumar Maurya, Xin Liu, and Tsuyoshi Murata. Improving graph neural networks with simple architecture design, 2021.
[36] Fabrizio Frasca, Emanuele Rossi, Davide Eynard, Ben Chamberlain, Michael Bronstein, and Federico Monti. Sign: Scalable inception graph neural networks, 2020.
12

---

[37] Maya Bechler- Speicher, Ido Amos, Ran Gilad-Bachrache and Amir Globerson. Graph neural networks use graphs when they shouldn't. In Forty-first International Conference on Machine Learning, 2024.
[38] Ines Chami, Sami Abu-El-Haija, Bryan Perozzi, Christopher Ré, and Kevin Murphy, Machine learning on graphs: A model and comprehensive taxonomy. Journal of Machine Learning Research, 23(89):1-64, 2022.
[39] Maya Bechler-Speicher, Amir Globerson, and Ran testing graph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 11022-1 1042,, 2024.
Tree-g: Decision trees con-
[40] Dongsheng Luo, Wei Cheng, Dongkuan Xu, Wenchao Yu, Bo Zong, Haifeng Chen, and Xiang Zhang. Parameterized explainer for graph neural network, 2020.
[41] Rex Ying, Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec. Gnnexplainer: Generating explanations for graph neural networks, 2019.
[42] Kenza Amara, Rex Ying. Zitao Zhang, Zhihao Han, Yinan Shan, Ulrik Brandes, Sebastian Schemm, and Ce Zhang. Graphframex: Towards systematic evaluation of explainability methods for graph neural networks, 2022.
[43] Jun Yin, Chaozhuo Li, Hao Yan, Jianxun Lian, and Senzhang Wang. Train once and explain everywhere: Pre-training interpretable graph neural networks. In seient Conference on Neural Information Processing Systems, 2023. URL httt::///opemerri.v.ooo///ooorr enfx8HM4Rp
[44] Junchi Yu, Tingyang Xu, Yu Rong, Yatao Bian, Junzhou Huang, and Ran He. Graph information bottleneck for subgraph recognition, 2020.
[45] Ying-Xin Wu, Xiang Wang, An Zhang, Xiangnan He, and Tat-Seng Chua. Discovering invariant rationales for graph neural networks, 2022.
[46] Ziming Liu, Yixuan Wang. Sachin Vaidya, Fabian Ruehle, James Halverson, Marin Soljačić, Thomas Y. Hou, and Max Tegmark. Kan: Golmogorov-arroll arnold networks, 2024.
[47] Jeroen Kazius, Ross McGuire, and Roberta Bursi. Derivation and validation of toxicophores for mutagenicity prediction. Journal of medicinal chemistry, 48(1):312-320, 2005.
[48] Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi- Rad. Collective classification in network data. Al magazine, 29(3):93-93, 2008.
[49] David Louis Borchers, Stephen Terrence Buckland, IG Priede, and $ Ahmadi. Improving the precision of the daily egg production method using generalized additive models. Canadian Journal of Fisheries and Aquatic Sciences, 54(12):727-2744, 1997.
[50] Wolfgang Härdle, Sylvie Huet, Enno Mammen, and Stefan Sperlich. Bootstrap inference in semiparametric generalized additive models. Econometric Theory, 20(2):265-3000 3000 2004.
[51] AK Debnath. RL Lopez de Compadre, Debnath, AJ Shusterman, and C Hansch. Structure- activity relationship of mutagenic aromatic and heteroaromatic nitro compounds. correlation with molecular orbital energies and hydrophobicity. Journal of medicinal chemistry, 34(2):
786 797, February 1991. ISSN 0022-2623. doi: 10.1021/jm000 (0021//m0000660444 1060046.
URL. https
/ //doi orgo 100 1021/
[52] Christopher Morris, Martin Ritzert, Matthias Fey, William L. Hamilton, Jan Eric Lenssen, Gaurav Rattan, and Martin Grohe. Weisfeiler and leman go neural: Higher-order graph neural networks, 2021.
[53] Shaked Brody, Uri Alon, and Eran Yahav. How attentive are graph attention networks?, 2022.
[54] Yunsheng Shi, Zhengjie Huang, Shikun Feng, Hui Zhong, Wenjin Wang, and Yu Sun. Masked label prediction: Unified message passing model for semi-supervised classification, 2021.
13

---

[55] Zhilin Yang, William W. Cohen, and Ruslan Salakhutdinov. Revisiting semi supervisd learning with graph embeddings, 2016. URL
08861
[56] Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs, 2020. URL URtptttt::/t///// http:::///axivv..r wttp///ggaaa...oaaaa////// ht0p///0000000077...
[57] Hongbin Pei, Bingzhe Wei, Kevin Chen- Chang, Yu Lei, and Bo Yang. Geom-gen: Geo- metric graph convolutional networks. In International Conference on Learning Representations, 2020.
[58] Daniil Likhobaba, Nikita Pavlichenko, and Dmitry Ustalov. Toloker Graph:
Interaction of
Crowd Annotators, 2023.
[59] Christopher Morris, Nils M. Kriege, Franka Bause, Kristian Kersting, Petra Mutzel, and Marion Neumann. Tudataset: A collection of benchmark datasets for learning with graphs. In ICML, 2020 Workshop on Graph Representation Learning and Beyond (GRL+ 2020), 2020. URL www. grwphhearning......... io
[60] Raghunathan Ramakrishnan, Pavlo Dral, Matthias Rupp, and Anatole von Lilienfeld. Quantum chemistry structures and properties of 134 kilo molecules. Scientific Data, 1, 08 2014. doi: 10. 1103//datta.004.222
[61] Federico Emrica, Marco Podda, Davide Bacciu, and Alessio Micheli. A fair comparison of graph neural networks for graph classification, 2022.
[62] Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. Simplifying graph convolutional networks. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 6861-6877 PMLR, 09-15 Jun 2019. URL https://proceedings. mlr http://nwww......es./eeeee.eseee v97 wu19e html
[63] Chenxiao Yang, Qitian Wu, Jiahua Wang, and Junchi Yan. Graph neural networks are inherently good generalizers: Insights by bridging gnns and mlps. 2023.
[64] Krishna Sri Ipsit Mantri, Xinzhi Wang, Karola-Bibiane Schönlieb, Bruno Ribeiro, Beatrice Bevilacqua, and Moshe Eliasof. Digraf: Diffeomorphic graph-adaptive activation function, 2024. httthtttp///xxxtt htt.aaawwwwa..aaaaaeeeeeee org/abs/2407 020033
[65] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang. Zach Vito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-perfornance deep learning library, 2019.
[66] Matthias Fey and Jan Eric Lenssen. Fast graph representation learning with pytorch geometric, 2019.
[67] Oleg Platonov, Denis Kuznedelev, Michael Diskin, Artem Babenko, and Liudmila Prokhorenkova. A critical look the evaluation of GNNs under heterophily: Are we re- ally making progress? In The Eleventh International Conference on Learning Representations, 2023. URL Ihttps: //tp///ppeeeeeeeeeeeeeeeeeeeeeee http://wwwr....eeeeeeoooroa.aaaa net:/forumii==ttb0ff
[68] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks, 2017.
[69] Wei Jin, Yao Ma, Xiaorui Liu, Xianfeng Tang, Suhang Wang, and Jiliang Tang. Graph structure learning for robust graph neural networks, 2020. URL https:://axxt:t wttp:/iww..rg/bb////////// 10203.
14

---

A Efficient GNAN implementation with tensor products
We formulate GNAN case of classification with C classes, For regression, the exact formulation holds with C = 1. We use the same notation in the main text, only now the output of the feature and distance function is of dimension 1 x C,
For the sake of notation, we assume that every tensor that its last dimension is of size C, is permuted to have the last dimension as its first dimension, without stating it explicitly. This is necessary to achieve valid tensor multiplication.
We denote with M the matrix of the transformed distances that is outputted by applying p. i.e.,
Mi, pl 14dist5.)) ), M E C
We denote with F the matrix of the transformed feature is outputted by applying the corresponding Jt for feature of each node in the graph, i.e., Fa = f& [x()) F E TRXNxx
Both for node and graph tasks, we first computes the matrix M F RRRNxd The rest of the computation then depends on the task.
A.1 Node Tasks
We aggregate the transformed features weighted by the transformed distances. This is done by summing over the rows of M & F:
1
(1) [M F lexaxi];=
pl + dist(j,
Se((x))k)
A.2 Graph Tasks
For graph classification, set p and f to output scalar and aggregate the transformed features over the nodes, i.e., the row of M F, to form a fixed-size vector of size d.
(G) = 1xxIXN & M @ F E Cxixd
Then, can apply another readout NAM 26
(G) = F(O(G)
Such that F is the transformed features using the function {fe) such that fu. - R2xC We can also simply sum over the outputs. In that case, we will set ʃ and m to output a vector of dimension C:
(G) Σ (ii) 1XXIXN M @ F lxxxx1
B Extensions and ablations
In Section we mentioned several possible extensions for GNAN. Here, we discuss them in detail.
Readout layer for graph tasks In graph tasks, after aggregating the node representations, it is pos- sible apply another transformation before aggregating over the entries of the graph representations. There may be many ways to do so, and we did not explore all of them. We did explore an application of another set of feature functions to each feature or the graph representation vector. This approach increases the capacity of the model in the cost of interpretability. This is because the set of addition feature functions should be plotted separately, and the product between the feature function and distance functions does not affect the final output directly but rather through another feature function. Empirically, we observed this approach did not improve performance with respect to the performance reported in Section
15

---

Splines is possible to learn splines for the activations in each feature network to achieve smoother shape functions [46]. We note that the Tolokers example presented Section[]showw that the learned feature shape function is smooth, although we use ReI.U activations. Nonetheless, in other cases, such in the PubMed example presented in the main paper, many of the learned feature functions are step functions. Therefore, it is likely that the model could benefit from spline activations, to smooth its shape functions.
Normalization In GNAN we normalize the weight of nodes of distance / with the number of nodes of distance so that the cumulative weight of nodes of distance will be p(+/(1+1) We examined the effect of removing this normalization. We observed that without normalization, the loss scale drastically larger. Therefore, more epochs are required to the data. As result, for the fixed number of epochs used in experiments (1000), without normalization, the accuracy decreases.
C Additional explanation examples
In the main paper, we presented two examples of explanations over two datasets with different properties, In this section, present additional explanations and examples we could into the main text due to space limitations.
C.1 Confidence Intervals
In main paper, discussed the construction of confidence intervals for Generalized Addi- tive Models (GAMs) using the bootstrap method. Here, we provide a specific example using the Mutagenicity dataset.
We computed 95% confidence intervals by applying the bootstrap method with 200 resamples. This involved resampling the original dataset with replacement 200 times and calculating the statistic of interest for each resample. The resulting bootstrap estimates were sorted, and the 2.5th and 97.5th percentiles were taken the lower and upper bounds of the confidence interval, respectively. Figure presents the distance function with its confidence intervals.
0.00
0.00
Figure 7
C.2 Additional PubMed heatmaps
In the main text, we presented the heatmaps for the 'children' feature. Here we provide additional heatmaps for additional features: the 'fat" feature and the 'young' feature, presented in Figures and
C.3 Tolokers Binary classification with binary and continuous features
The Tolokers dataset is based on data from the Toloka crowdsourcing platform. The nodes represent tolokers (workers) who have participated in least one of 13 selected projects. An edge connects two tolokers if they have worked on the same task. The goal is predict which tolokers have been
16

---

Diabetes
Disbotes
Gestational Diabctes
Distance
Distance
0.75
0.60
0425
0000
-4.025
Taal fesature value
"alf' forature
"latt fomturr
Figure 8: Visualization of the products between the outputs of the 'fat' feature function over the input range [0, 1] and the outputs of the distance function, learned over the PubMed dataset
Type Diabetes
Type 2 Diebeles
Cestalions Diebetes
8.50
005
000
Distance
Dislence
Dislence
0005
03.0
-315
-320
'oonng'
'vongg feature
'young
Figure 9: Visualization of the products between the outputs of the 'young' feature function over the input range [0, and the outputs the distance function, learned over the PubMed dataset
banned in one of the projects. Node features are based on the worker's profile information and task performance statistics. Each node in the graph is associated with nine features. There are continuous features in the range [0, 1] and are binary features. Figure I0 shows the shape functions of the features learned by GNAN. Figure shows the distance shape function learned by GNAN. In Figures 13 and 12 we present the heatmaps of the cross product of the shape functions and the distance function.
Feature function outout
Feature function ourput
Approved rate
Skipped rate
-005
Expired
Emaic stteationn
Midtle adteatio
High education
Engiah profile
English lested
Rejected rate
00
02
64
08
0.8
1.0
Feature Value
Feature
Figure 10: Visualization of the feature functions learned over the Tolokers dataset.
17

---

Distance function output
Banned
0.3
0.1
0.0
-0.1
4
10
Distance
Figure 11: isualization of the distance shape function learned on the Tolokers dataset.
10
Distanee
Distence
05
'Approved rate' feature value
Expired rate' feature value
-10
-15
Distoance
Distonce
-20
-25
-30
"Skipped rale' feature value
"Rejected rate! feature value
Figure 12:
Visualization of the products between the outputs of the continuous feature functions over the input range (0, 1] and the outputs of the distance function, learned over the Tolokers dataset.
D Additional experimental details
All our baselines are implemented using PyTorrc 665 and PyTorcc-Geommtric [66]
D.1 Dataset information
Here we provide additional information about the datasets used in Section 5 The data statistics are given Table
Proteins [59] is dataset of chemical compounds consisting 1113 graphs, respectively. The goal in the first two datasets is predict whether compound enzyme not, and the goal in the last datasets to classify the type of an enzyme among 6 classes.
NCT1 59 is datasets consisting of 41 1110 graphs, representing chemical compounds. Vertices and edges represent atoms and the chemical bonds between them. The graphs divided into two classes
18

---

Britiah wade tapa rghoweadee Mall conadee Bute sheuatas
07
4.05
2,55
0.03
0.00
4,00
...0
0.00
2.00
4000
0.00
0.01
0.20
000
0.1
001
0.01
0..11
9.00
0.30
0.00 0.00
0.00
0.00
0.00
Figure 13: Visualization of products of the outputs of the distance function and the feature functions. trained on Tolokers. Each cell shows the exact contribution, positive or negative, of features at a certain distance to the prediction. Positive values (green) contribute to classifying a toloker as "banned', and negative values (red) contribute to classifying toloker as 'not banned".
according to their ability to suppress inhibit tumor growth.
Mutagenicity [59
is dataset consisting of 4337 chemical compounds of drugs divided into two classes: mutagen and non-mutagen. A mutagen is compound that changes genetic material such as DNA, and increases mutation frequency.
PTC 59 is dataset consisting of 344 chemical compounds divided into two classes according to their carcinogenicity for rats.
Cornell [57 is a heterophilic webpage dataset collected from the computer science department at Cornell University. Nodes represent web pages, and edges are hyperlinks between them. The task is classify the nodes into one of five categories.
Table 2: Statistics of the real-world datasets used in evaluation.
Dataset
# Graphs
Avg # Nodes
Avg # Edges
Node Features
Classes
Proteins 59
1,113
39.06
72.82
NCII (59
4,110
29.87
32.3
37
2
Mutagenicity
4,337
30.32
30.37
7
PTC 159
344
14
14
19
2
QM9
60
130,831
37.3
11
Cora 55
1
2,708
10,556
1,433
Citeseer 55
1
3,327
9,104
3,703
PubMed
55
1
19,717
88,648
500
ogb-arxiv [66
169,343
1,166,243
128
40
Cornell [7]
183
295
1,703
Tolokers [58
11758
519000
10
D.2 Protocols
ogb-arxive The ogb-arxive datasets are large-scale datasets provided in the Open Graph Benchmark (OGB) paper (56) with pre-defined train and test splits and different metrics and protocols for each dataset. As common in the literature when evaluating OGB datasets, we followed its pre-defined
19

---

metric and protocol. The metric used is accuracy. We ran GNAN 10 times and reported the mean accuracy and std over the runs.
Cornell For the Cornell dataset we used the splits and protocol from 57 and report the test accuracy averaged over 10 runs, using the best
found on the validation set.
Tolokers For the Tolokers dataset, we followed the protocol and pre-defined splits from 58 The reported result an average 10-fold nested tross-validttio.
67
30, 29 for the Core, Citeseer and Pubmed datasets Core, Citeseer and PubMed Following 68
we tuned the parameters on the Cora dataset using the pre-defined splits from [68]. For all these datasets we report the test accuracies averaged over 5 runs, using the parameters obtained from the best accuracy on the validation set of Fora.For Cora and Citeseer, we followed the schemes used in 669 where the used graph is the largest connected component, to increase the stability of the evaluated models.
Proteins, NCI We used 10-fold nested cross validation with the splits and protocol of Errica et al. 61.. The final reported result on these datasets an average of 30 runs (10-folds and random seeds).
Mutagenicity, PTC We use the splits and protocols from [39], and use 10-fold nested cross- validation. The final reported test accuracies are averages over the 10 test sets of the outer 10 folds.
D.3 Hyperparameters
All GNNs (excluded GNAN) use ReLU activations with {3.5} 5} layers and 64 hidden channels. They were trained with Adam optimizer over 1000 epochs and early on the validation loss with a patient of 100 steps, eight Decay of le 4, learning rate in (le - 3, le - 4), dropout rate in {0, 0.5), and train batch size of 32.
In GNAN, all the feature and distance networks use ReLU activations with {3, 5) layers and {64, 32} hidden channels. They were trained with Adam optimizer over 1000 epochs weight decay of 0, 5e learning rate in {le 2, le 3), dropout rate in {0, 0.63.
D.4 Compute resources
All experiments NVIDIA GeForce RTX 3090 GPU.
20

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer:[Yes,
Justification: the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: the paper discuss the limitations of the work performed by the authors
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
21

---

Justification: NA
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: the paper fully disclose all the information needed to reproduce the main exper- imental results of the paper to the extent that it affects the main claims and/or conclusions of the paper
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
22

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code permitted.
6. Experimental Setting/eetails
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: the paper report error bars suitably and correctly detined or other appropriate information about the statistical significance of the experiments
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
23

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: the paper provide sufficient information the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics htttpe:/eeerrrp..ccppll//////// ctw.eiieeeeinnciceeeeieenne
Answer: [Yes]
Justification: the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed
Guidelines:

---

The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negatíve societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (c.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
25

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the datasets.paperssiii paperswi thcode. www.coo//aaaaa datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: NA
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
26

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
27


Reviews:
[{"review_id": "yp72g5JQDO", "reviewer": "Reviewer_3kSH", "summary": "The paper proposes an interpretable by design model for graph data. The proposed model GNAN builds upon Generalised additive models and learns node representations as a distance function and feature shape functions explicitly and independently for each function. Interpretability is then offered by means of visualising the distance and feature shape functions.", "questions": "1. How sensitive are explanations to model initialisations?\n2. Could you provide a more concrete example of a local explanation corresponding to a query node in the scenario of node classification?\n3. In the scenario of large feature spaces is it possible to have a quantitative metric to conclude that certain features may not be visually inspected.", "limitations": "yes", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The paper presents a novel (to the best of my knowledge) yet simple idea of extending generalised additive models to achieve interpretability in learning from graph structured data. It provides fresh perspective on the problem.  It not only departs from the common definitions of explanations for graph data but also from the much hyped GNNs. The paper is very well written and easy to follow. If there are limitations to the current method for large graphs and large feature spaces it present a very valid first step which would be picked up the community for further improvements.\n\n**After discussion period**\nI agree with the other reviewers that evaluation can still be improved. To reach the excellence level as my previous score indicated, teh paper would need to demonstrate clearly the claims of the paper (via user studies) for example for model debugging.", "weaknesses": "It would be problematic to apply this method directly to datasets with very large feature spaces and large graphs. Can one come up with a strategy to initially prune these large spaces as a preprocessing step?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are gratified by the reviewer’s appreciation of the novelty and interest of our work and thank the reviewer for the valuable feedback.\n\n\n## Weakness 1: \nThe reviewer claims it could be problematic to apply GNAN to large graphs with many features and asks for methods to prune the space as a pre-processing step. We thank the reviewer for the question. \\\nOne approach is to clip distances greater than a predefined threshold. One can also mask distances using other rules rather than one threshold.\nIf needed, there are also many methods for reducing the graph size in advance [1, 2]. \\\nWe also note that GNAN is implemented in a tensor-multiplication formulation as described in Appendix A, which is therefore optimized when using GPUs. \\\nRegarding the cases where the number of features is large, any feature-selection method, e.g., variance-threshold or univariate feature selection, can be applied as a pre-processing step to reduce the number of features. We also addressed the case of limiting the number of feature visualizations in our answer to Question 3. \n\n[1] On the Ability of Graph Neural Networks to Model Interactions Between Vertices, Razin et al.,ICML 2023. \\\n[2] DropEdge: Towards Deep Graph Convolutional Networks on Node Classification, Rong et al., ICLR 2020.\n\n## Question 1:\nThe reviewer asks how sensitive are the explanations of GNAN to initializations. We thank the reviewer for this important question as it allows us to clarify important aspects of our proposed algorithm. \\ \nIn the literature about explainability, there is a distinction between explaining the model and explaining the data [3]. The explanations in GNAN are explanations of the model. Regardless of the initialization, the explanation will always describe precisely the model. Therefore, the sensitivity of explanations would simply visualize the model’s sensitivity to initializations. \\\nNonetheless, following the reviewer’s question, we visualized the heatmap of the mutagenicity experiment using a GNAN trained with 3 different seeds. The heatmaps are attached in the PDF file allowed in the global comment. The values in the heatmap slightly differ, but the trend is stable.\n\n[3] Chen, H., Janizek, J. D., Lundberg, S., & Lee, S. I. (2020). True to the model or true to the data?. arXiv preprint arXiv:2006.16234.\n\n\n## Question 2:\nThe reviewer asks for a concrete example of a local explanation corresponding to a query node in node classification. We thank the reviewer for the opportunity to address this topic. Given a query graph and node, we highlight other nodes in the graph in a way that corresponds to their contribution to the query node’s prediction based on the learned distance and feature functions.  For example, if we observe from the visualization that neighbors of distance x with feature k contribute towards positive labels, we can highlight such nodes in the given query graph, and the distances are computed with respect to the query node. We will make sure to add such examples to the camera-ready version. \n\n## Question 3:\nThe reviewer asks how to conclude which features to inspect in the case of large feature spaces. We thank the reviewer for this important question. \\\nThis is a fundamental problem in explainability that is common to many methods. Even a linear model or a tree may lack human interpretability in the context of a large feature space. A plausible method to mitigate the problem is to use regularization terms to induce sparsity in the use of features or use feature selection. \\ Another approach is to use a post-processing step of removing the contribution of features for which the shape-function has low variance. \\\n Additionally, in some cases, users have pre-defined features of interest to examine. For example, doctors may be interested in observing the effect of specific genes. \\\nWe will add this important discussion to the camera-ready version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Rrbn44rY48", "reviewer": "Reviewer_5GdN", "summary": "The paper presents the GNAN, a model designed to integrate the interpretability of Generalized Additive Models with GNNs. GNAN aims to address the black-box nature of traditional GNNs by providing explanations through visualization. The model achieves this by learning shape functions for each feature and linearly combining them. Distance functions are used to capture the graph structure influence among nodes. GNAN allows the relationships between the target variable, features, and graph topology to be easily understood. GNAN matches the performance of existing GNNs on several datasets for both node and graph predictions.", "questions": "- Regarding my weakness 1, how is the idea of GNAN different from transformer? I could misunderstood, so maybe the authors can explain more.\n\n- Regarding my weakness 3, How exactly can the method be used for debugging? In line 209 - 215, this point was vaguely discussed as using visualizations to correct misalignment. How can that correction be performed exactly?", "limitations": "N/A", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "- Novelty: the idea of combining the GAM and GNN is novel to me.\n- Interpretability: As a graph ML model, GNAN provides inherent interpretability, allowing users to understand the model's decision-making process without post hoc explanations\n- By avoiding iterative message-passing, GNAN reduces computational bottlenecks and makes parallel computing easier.", "weaknesses": "- Novelty of model design: the current model design is not too novel to me. My understanding is that GNAN is a special type of graph transformer, where the positional encoding (dist in the paper) is explicitly combined with the node features using a heuristic function, also attention is removed. However, the high-level idea of decoupling GNN into node feature and positional feature and encoding them separately is the same as graph transformers\n\n- Empirical performance: this is the biggest weakness of the paper to me. For all the datasets, the GNAN model doesn't seem to be better than the baselines\n\n- Presentation: model debugging is claimed as a major contribution, but from the current writing how exactly that can be done is not clear to me. Also, the \"inteligibility\" in the title and section 4 might be overclaiming. Overall, the model is only for node and graph prediction.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the thoughtful comments and encouraging remarks.\n\n## Weakness 1:\nThe reviewer claims that GNAN is a special type of transformer and asked for clarifications in Question 1. We think there might be some confusion here that we are happy to clarify. We do not see GNAN as an instance of a transformer. \\\nThe fundamental idea in GNAN is that each feature is processed separately by the network using a univariate shape function. The values computed by these shape functions are sum-pooled only in the last layer. This is in contrast to transformers, where each layer may compute functions that mix arbitrary numbers of features. GNANs also restrict the way distance and features interact, which allows for their interpretability. \\\nWhile it is true that both GNANs and transformers allow information propagation between any pair of nodes, they are considerably different, as evidenced by the interpretability of GNANs compared to the black-box nature of transformers.\n\n\n## Weakness 2:\nThe reviewer commented that GNAN does not outperform other baselines. We would like to emphasize that the main goal of GNAN is to provide interpretable model for graph learning. The interpretability requirement restricts the type of models that can be used but surprisingly,  we show that in many settings, there is no tradeoff between explainability and accuracy since GNAN is comparable to the commonly used (non-explainable) models, and in some cases, it is even better. As we mentioned in the paper, e.g., line 284, this is non-trivial as it is usually assumed that interpretability comes at the cost of accuracy. Second, we note that GNAN did outperform the tested baselines on the Tolokers, ogb-arxive,  and alpha, mu, and alpha-homo tasks. \n\n## Weakness 3:\nThe reviewer asks how debugging is done. We thank the reviewer for the question. GAMs, including GNAN, allow for debugging and adjustment of the model in several ways. As mentioned in lines 210-215, because GNAN is transparent and can be fully visualized, this allows for several debugging methods. \\\nOne example given in lines 210-215 suggests using the visualizations of the model for model selection. Specifically, one can select models not only based on accuracy but also based on their alignment with prior knowledge. \\\nThe visualizations of GAMs also enable debugging biases in the model, which can be addressed in different ways [1, 2, 3]. \\\nAdditionally, the ability to visualize exactly what the model has learned and observe whether it \"makes sense\" can help detect code bugs. For instance, we used this property to hunt bugs during the development of the model.\n\n\n\n[1] Interpretability, then what? editing machine learning models to reflect human knowledge and values, Wang et al., 2021. \\\n[2] How interpretable and trustworthy are gams, Chang et a.l, 2021. \\\n[3] Gam changer: Editing generalized additive models with interactive visualization, Wang et al. 2021.\n\n## Question 1:\nThe reviewer asks how GNAN is different from a transformer. This is addressed in Weakness 2\n\n## Question 2:\nThe reviewer asks how the visualization of GNAN can be used for debugging. This is addressed in Weakness 3.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GRTxoSX3eh", "reviewer": "Reviewer_vkt8", "summary": "This paper introduces the Graph Neural Additive Network (GNAN), a novel interpretable graph neural network based on Generalized Additive Models. GNAN is designed to be fully interpretable through visualizations that clearly demonstrate how it uses relationships between the target variable, features, and graph structure. The paper shows that GNAN performs comparably to traditional black-box GNNs, making it suitable for critical applications where both transparency and accuracy are essential.", "questions": "Check the above weaknesses.", "limitations": "NA.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The paper creatively extends Generalized Additive Models to graph neural networks, demonstrating performance comparable to mainstream graph neural networks.\n2. The paper conducts extensive experiments to validate the interpretability of GNAN.", "weaknesses": "1. The formulas and methods section of the paper is somewhat rough and could be further optimized.\n2. GNAN requires calculating the shortest paths and relationships between any two nodes, which could be prohibitively costly for large datasets.\n3. Graphs are complex data types, and merely capturing node-level relationships might not replace previous subgraph-level explanation schemes. The paper lacks further analysis and argumentation on this point.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the thoughtful comments and encouraging remarks.\n\n## Weakness 1:\nThe reviewer suggests improving the presentation of formulas in the methods section. We thank the reviewer for this suggestion, and we use it to improve the camera-ready version.\n\n## Weakness 2:\nThe reviewer mentions that calculating shortest paths between any two nodes can be costly for large graphs. This is indeed true, but this limitation is not unique to GNAN, and remedies for this problem exist.\\\nFirst, note that pre-computing the shortest paths between nodes is a common practice in the GNN literature as a way of improving the expressive power of the model. For example, in the shortest-path GNN [1], the authors apply message-passing over shortest paths in graphs. In [2], the shortest path length is used as a positional encoding for the graph transformer. These approaches inspired us to utilize shortest-path information to incorporate the graph into the GAMs framework.\\\nSecond, in cases where the cost of computing all shortest paths is prohibitive, a natural remedy is to clip distances greater than a threshold D. This can be used in many scenarios as it makes sense that remote nodes should have a small influence on each other.\\\nImportantly, the distance computation is only required once, and the determined neighborhoods can subsequently be reused at no additional cost. Hence, in some cases, this cost can be amortized over multiple runs on the same graph or executed as a preprocessing step that does not affect the online running time of the model.\\\nWe demonstrate in the paper several applications to tasks where graph sizes allow the computation of shortest paths. Additionally, we wish to note that GNAN is the first approach to extend GAMs into graph learning, and we believe it opens up many new directions for future research, including finding more efficient methods.\n\n\n## Weakness 3:\nThe reviewer claims that using only node-level information may not be sufficient to replace subgraph-level explanation schemes and that further analysis is required on that matter. We thank the reviewer for this important note. It is true that there may be cases where subgraph-level information is crucial for accurate predictions. Surprisingly, this was not the case in the many datasets we experimented with. \\\nMoreover, it is important to note that existing subgraph-level explanations are post-hoc explanations used with black-box GNNs. In contrast, GNAN is an inherently interpretable white-box model rather than a post-hoc explanation. As discussed in the paper in detail in lines 31-44 and 114-125, post-hoc explanations over black-box models are not suitable for high-stake applications where transparency is crucial, such as in healthcare and criminal justice. Therefore, while post-hoc subgraph explanations may be useful in some cases, they are not suitable in many domains. We claim that GNAN may serve this unmet need as it provides full transparency for decision-makers in such applications.\n\n\n[1] Shortest Path Networks for Graph Property Prediction, Abboud wt al. LOG 2022. \\\n[2] GRPE: RELATIVE POSITIONAL ENCODING FOR GRAPH TRANSFORMER, Park et al. ICLR 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7AzJBVdx32", "reviewer": "Reviewer_uKSz", "summary": "This paper proposes additive Graph Neural Networks. A combination of interpretable neural additive models and Graph neural networks. Through this combination GNANs are interpretable and similarly to NAMs the feature effects are visualizable.", "questions": "- How do you explain that GNANs, although they have the additivity constraint outperform fully connected networks?\n- Why do you choose different architectures for GNANs and all comparison models? Also different learning rates, weight decay etc? If no hyperparamter tuning is performed, the architectures should at least be identical\n- Why choose MLPs as feature networks when they have been outperformed by newer (and older) architectures?\n- What are the model parameters compared to the benchmarks? Given that NAMs often have a multitude of fully connected networks, I would expect the same here?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper is overall well written\n- The idea is simple yet intuitive", "weaknesses": "- If I am not mistaken the authors are not performing any hyperparamter optimization. Thus, the experimental results, while they average over a lot of folds/seeds are not as conclusive as it might seem\n- The interpretability is not tested. The visualizations could be completely biased. There should at least be an ablation study with simulated feature effects. \n- While the feature effects may be visualizable, what about intelligebility (e.g. [1])\n- There has been quite a lot of research in the NAM area, for e.g. the architecture of the feature networks [2, 3, 4, 5] or distributional approaches [6]. None of these is mentioned and the chosen GNAN architecture with MLPs as feature nets is already a bit outdated.\n\n\n\nMinor:\n- the initial introduction of the GAM could be a bit better. You are missing an intercept, possible feature interactions and while you later define vectors to be bold do not adjust for that in this section. \n- The appendix is very poorly written with a lot of errors, both grammatically and spelling.\n\n[1] Luber, M., Thielmann, A., & Säfken, B. (2023). Structural neural additive models: Enhanced interpretable machine learning. arXiv preprint arXiv:2302.09275.\n\n[2] Chang, C. H., Caruana, R., & Goldenberg, A. (2021). Node-gam: Neural generalized additive model for interpretable deep learning. arXiv preprint arXiv:2106.01613.\n\n[3] Radenovic, F., Dubey, A., & Mahajan, D. (2022). Neural basis models for interpretability. Advances in Neural Information Processing Systems, 35, 8414-8426.\n\n[4] Thielmann, A. F., Reuter, A., Kneib, T., Rügamer, D., & Säfken, B. Interpretable Additive Tabular Transformer Networks. Transactions on Machine Learning Research.\n\n[5] Dubey, A., Radenovic, F., & Mahajan, D. (2022). Scalable interpretability via polynomials. Advances in neural information processing systems, 35, 36748-36761.\n\n[6] Thielmann, A. F., Kruse, R. M., Kneib, T., & Säfken, B. (2024, April). Neural additive models for location scale and shape: A framework for interpretable neural regression beyond the mean. In International Conference on Artificial Intelligence and Statistics (pp. 1783-1791). PMLR.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the thoughtful comments and encouraging remarks.\n\n## Weakness 1:\nThe reviewer mentions that no hyper-parameters were tuned. We did tune the hyper-parameters, and the grid of hyper-parameters is presented in Appendix D3 and referred to in line 252.\n\n## Weakness 2:\nThe reviewer suggests testing the interpretability of GNAN. GAMs are considered explainable models [1,2]. The explanations they provide make the model transparent in the sense that the way the model makes its predictions is humanly comprehensible. However, it is important to distinguish between explaining the model and explaining the underlying data, which often requires causal analysis [3]. \\\nWhile the model may have some inductive biases, the visualization makes the model transparent to users, allowing them to scrutinize and adjust it if needed. Performing a usability study to check the interpretability of the solution is a great idea but is beyond the scope of this study.\n\n\n## Weakness 3:\nThe reviewer asks about the intelligibility of GNAN and refers to a paper that extends NAMs to a stronger neural architecture and proposes methods to extend the intelligibility of NAMs. We thank the reviewer for the suggestion showing that there is great potential for further research in this field. Regarding intelligibility, as GNAN is an instance of GAMs, it is intelligible in the same sense that GAMs are.\n## Weakness 4:\nThe reviewer mentions that MLPs are outdated with respect to the NAMs literature and referred to newer approaches. We thank the reviewer for these references. GNAN is the first extension of GAMs to graphs, and we decided to utilize the commonly used MLPs. The fact that there is potential for improving GNAN even further presents a great future research direction. \\\nWe demonstrated that even with MLPs, GNAN is on par with black-box GNNs while providing full interpretability. We will make sure to refer to these studies and the future research direction in our camera-ready version.\n \n\n## Weakness 5 minors:\nThe reviewer suggests improving the introduction of GNAN. We appreciate this comment and we will improve it in the camera-ready version.\n\n## Weakness 6 minors:\nThe reviewer suggests improving the appendix. We appreciate this comment and we will improve it in the camera-ready version.\n\n\n## Question 1:\nThe reviewer asks how GNAN outperforms fully connected networks despite its limited capacity. We would greatly appreciate it if the reviewer could clarify what they mean by “fully connected networks” in the context of graph networks. \\\nWe compare GNAN to methods that are not restricted in the same way GNAN is, including sparse graph transformers [4]. The empirical evidence reported in this paper shows that GNAN is mostly on par with these methods and sometimes even outperforms them.\n\n## Question 2:\nThe reviewer asks why different hyperparameters are selected for different models if no hyperparameter tuning was performed. We wish to reiterate our response to Weakness 1: we did tune hyperparameters in a standard fashion as described in Appendix D3. \n\n## Question 3:\nThe reviewer suggests replacing MLPs with other newer networks. Please see our response to Weakness 4.\n\n## Question 4:\nThe reviewer asks what the parameters in GNAN are. If the reviewer is referring to hyperparameters, these are listed in Appendix D3.\n\n\n[1]Interpretability, then what? editing machine learning models to reflect human knowledge and values, Chang et al, 2021. \\\n[2]How interpretable and trustworthy are gams, Wang et al, 2021. \\\n[3] True to the model or true to the data? Chen, H., Janizek, J. D., Lundberg, S., & Lee, S. I., 2020. \\\n[4] Masked label prediction: Unified message passing model for semi-supervised classification, Shi et al, IJCAI 2021.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank the reviewers for their important questions and thoughtful feedback.
Attached is a PDF file for reviewer 3kSH.

Meta-review: The paper proposes a novel architecture for graph neural nets, where the influence of nodes y influencing the representation of node x is regulated by normalizing by the number of other nodes y' at that distance and a learnable distance function. Further, predictions are performed via a simple addition operator, which supports interpretability.

Reviews generally agree that the core architecture is at least somewhat novel and enhances interpretability. It is also apparent from the experiments that it can perform roughly on par with existing methods while simplifying the architecture. The topic of the paper - intrinsically interpretable graph neural nets - is also still relatively fresh with most prior work in GNNs focusing on post-hoc explanations.

There are still some concerns, though:
- Using shortest paths for the computation of representations has been present in the literature before. Similarly, additive models are pre-existing. As such, the degree of novelty in the proposed architecture is, hence, somewhat questionable.
- The interpretability is only evaluated on two example cases and no user study has been provided. As such, the interpretability claims are not fully substantiated by evidence.

Overall, the contribution provided by a strongly simplified model seems interesting enough to a broad part of the NeurIPS community that a presentation as poster is warranted.
Decision: Accept (poster)