Score-based Explainability for Graph Representations
Authors: Ehsan Hajiramezanali
Sepideh Maleki
Max W Shen
Kangway V. Chuang
Tommaso Biancalani
Gabriele Scalia
Venue: Accepted by TMLR ()

Abstract: Despite the widespread use of unsupervised Graph Neural Networks (GNNs), their post-hoc explainability remains underexplored. Current graph explanation methods typically focus on explaining a single dimension of the final output. However, unsupervised and self-supervised GNNs produce d-dimensional representation vectors whose individual elements lack clear, disentangled semantic meaning. To tackle this issue, we draw inspiration from the success of score-based graph explainers in supervised GNNs and propose a novel framework, grXAI, for graph representation explainability. grXAI generalizes existing score-based graph explainers to identify the subgraph most responsible for constructing the latent representation of the input graph. This framework can be easily and efficiently implemented as a wrapper around existing methods, enabling the explanation of graph representations through connected subgraphs, which are more human-intelligible. Extensive qualitative and quantitative experiments demonstrate grXAI's strong ability to identify subgraphs that effectively explain learned graph representations across various unsupervised tasks and learning algorithms.

Published in Transactions on Machine Learning Research (10/2024)
Score-based Explainability for Graph Representations
Ehsan Hajiramozanali Genentech
Sepideh Maleki Genentech
malebiiseppidehh@eeee.com
Max W Shen Genentech
shen. Alen.maap@gene.com
Kangway V. Chuang Genentech
chuang.kanguutBggne.ccomm
Tommaso Biancalani Genentech
biancalani. biancalaan..ommaaaaa haap://ww...mmmaasoeeeee...ooom
Gabriele Scalia Genentech
caalea..abbrddeeggeee.com
Reviewed OpenReview:
https: https://openrevieel
openreview. net/ forum? id= K6DKrrp YpJ
Abstract
Despile the widespread use of unsupervised Graph Neural Networks (GNNs), their post-hoc explainability remains underexplored. Current graph explanation methods typically focus on explaining single dimension of the final output. However, unsupervised and self-supervised GNNs produce d-dimensional representation vectors whose individual elements lack clear, disentangled semantic meaning. To tackle this issue, we draw inspiration from the success of score-based graph explainers in supervised GNNs and propose a novel framework, grXAI, for graph representation explainability. grXAI generalizes cxisting score-bascd graph explainers to identify the subgraph most responsible for constructing the latent representation the input graph. This framework can be casily and efficiently implemented wrapper around existing methods, enabling the explanation of graph representations through connected subgraphs, which are more human--ntelligibbe demonstrate grXAl's strong ability to identify subgraphs that effectively explain learned graph representations across various unsupervised tasks and learning algorithms.
Extensive qualitative and quantitative experiments
1 Introduction
Graph neural networks (GNNs) have become increasingly useful in a wide range of applications, such as drug discovery (Lu al. 2024 Hajiramczanali et al.] 2020). large-scale social networks (Hajiramezanali eet al. 2019), and recommender systems (Hasanzadeh al. 2011).. However, due to their complexity and opacity, understanding GNNs can be challenging for human users. To tackle this issue, several tools have been developed to explain supervised GNNs in terms of their predictions. In the supervised sctting, a GNN model learns to map input graphs to labels (f(:) g - y), and explanations shed light on the model's prediction specific label. The interpretability of model-agnostic GNN explanation methods stems from the fact that the particular labe interest, such as a chemical property, is meaningful to humans (Figure left).
The label used the context explainability refers the predicted label for the input graph by the GNN model being explained, which is distinet from the ground truth label used in the of supervised MT. Ying 2019

---

Published in Transactions on Machine Learning Research (10/2024)
d-dimensional voctor based scoring function
eleme wise scoring function
Score
Σ (i 1
Input Graph G
Inpur Graph G
C1 NNO Lilaaoon
(i)
Susgraph
P[Neg(
S Subgrape
SOOWOR (00)
Figure 1: (Left) Post-hoc graph explainability methods typically provide explanations based on a specific label of interest. (Right) Our framework for explaining graph representations provides explanations based on d-dimensional representation vector. The Score refers to range of post-hoc explanation functions proposed in the literature, including Shapley and Saliency.
On the other hand, unsupervised GNNs map input graphs to representations (J(): H) that cannot be easily interpreted using existing supervised explanation methods. Although these methods can be used understand individual elements in the representation space (hy E H), such explanations are not meaningful to humans unless the elements have natural semantic meaning (Lin et al.. of individual elements in the represcntation space is generally unknown. Two possible solutions have been proposed: the first approach enforces semantic meaning in the representations, but it requires concept labels for every training sample, which is often impractical (Koh et al., 2020). The second approach enforces disentangled representations and manually identifies semantically meaningful elements (Paige et al.) 2017). but this approach is not model-agnostic and requires potentially unwanted changes to the modeling and training process (Lin al. 2023).
2023). Unfortunately, the meaning
Another possible approach to solve the aforementioned issue may involve utilizing the existing methods to explain each element of the representation vector independently and then averaging over them (Crabbé & van der Schaar, 2022). However, this solution is not sufficient to provide a global understanding of the latent representation as a whole and does not necessarily generate connected subgraphs as explanations, which are important for human understanding and to improve consistency (Hajiramezanali et al. 2023 Additionally, many existing post-hoc explanation methods, such as gradient- or perturbation- based methods, rely on multiple perturbations or backpropagations of the GNN model to generate explanations for each input graph, making them time-consuming and impractical when applied to each element of high-dimensional represenlations (Chuang et al., 2023). For instance, Shapley-based graph explainers that utilize the GNN architecture to approximate Shapley values are computationally expensive (Xie et al. 2022). This cost escalates lincarly with the dimension of the representation vector, rendering them impractical for explaining unsupervised GNNs. Addressing this issue would require non-trivial extensions.
Our main contributions are: (i) We introduce a general framework, grXAL, that adapts model-agnostic score-based graph explainability methods to explain (representation) vectors. We do this by defining an auxiliary cosine similarity function wrapper around the unsupervised GNN to interpret. (ii) We apply our framework to different types of graph explainability approaches, including SubgraphX, and introduce the first graph explainability method for unsupervised settings that can explain the model using connected subgraphs. (iii) While motivated by the explanation of purely unsupervised graph representation vectors, the proposed approach is beneficial even for supervised multi-class classífications, generating more robust. and accurate explanations for stale-of-the-art results, both qualitatively and quantitatively, on various self-supervised lasks, including graph representation and node embedding.
cases. (iv) Our experiments show that our methods achieve
Learning mcaningful representations of graph-structured data with GNNs is important in many fields, particularly when labeled data are scarce due to costly and time-consuning et al. 2021). To this end, applications of GNNs in a wide range of domains, such as drug discovery and molccular biology (Hasanzadch et al. 2022), have inspired recent unsupervisod and self-supervised strategics to learn from massive corpora of unlabeled structured data (Sun al., 2019 You et al. 2020 Hu et al.] 2019)). The ability to explain unsupervised GNNs can provide valuable insights into the learned representation,
Motivation.
2

---

Published in Transactions on Machine Learning Research (10/2024)
Table 1: Comparison of different graph explainers.
Supervisod
Unsupervisod GNNs
d-dimensional vodor
Connected subgraphs
Explaiuer
GNNs
GNNEsplaine Ving al.
2019
MI-based
PGExplainer [Lio e5 a. 2020
V
TAGE Xic
/
TG & Salwney (5tmoryah
2013
Sundararaja ot al. 2017
Score-based
SubgraphX Yuan ot al
grXAI (ours)
This method is designed to handle probability vectors, such as the outputs of softmax layer, and is not capable of handling embedding vectors learned through unsupervised settings.
help researchers understand and compare graph representation learning methods, well as assist users in effectively monitoring and debugging these models during deployment. Our proposed framework enables score-based explainability in unsupervised settings in a scalable way, making it feasible and effective for real-world applications. Furthermore, our framework offers benefits even in supervised settings, proving particularly helpful when the modol's predictions are uncertain.
2
Related work
With many recent advances in GNNs and their numerous applications across different fields, explainability
methods have become critical for providing insight into their predictions. To this end, many approaches have
been proposed to explain the predictions of supervised GNN models (Baldassarre Azizpour
2019
Ying
et al.
2019
Luo et al.) 2020: Xie et al. 2022 Yuan et al.
2021 Zhang et al.
2022
Ye al.
2024 Huang
al.
2024
Hasanzadeh et al. 2020
These methods can be divided into four main categories Yuan et al. 2020 gradient- perturbation- decomposition-, and surrogate-based methods. In this paper, we mainly focus on gradient- based and perturbation-basee methods. The gradient-based methods are generally fast and easy to implement, but they may not be able to capture more complex relationships in the data (Yuan et al.) 2021 2020 On the other hand, perturbation- based methods can be more computationally expensive, but they generally achieve state-of-the performance in terms of explanation quality (Xie et al. 2022). The common characteristic of most of these methods is that they require labels to specify which element of the GNN's output to explain.
Instead of explaining individual elements in the representation, recent approaches have focused on explaining multi-dimensions representation vectors as whole, which is critical to explaining unsupervised models.
These include TAGE (Xic et al. 2022) in the context of GNNs, and methods such as COCOA (Lin et al. 2023), RELAX |Wickstrøm et al. 2023), Label-Free XAI (Crabbé van der Schaar 2022 and CoRTX
[Chuang et al. 2023) for other domains.
TAGE Xic et al. 2022) successfully addresses unsupervised GNN scttings using a contrastive loss approach. However, it has two main limitations (see Table D First, TAGE is mutual information- based (MI-based) explainability method; however, it has been shown that score-based explainability methods outperform MI-based ones in supervised GNN settings Yuan et al. 2020). Hence, an open problem is how to leverage existing score-based based methods to explain GNNs in unsupervised settings. Second, TAGE focuses on explaining the importance of graph edges, but ignores the subst urcturs of graphs. However, explanations consisting of connected subgraphs have not only been deemed more intuitive to humans in practice (Yuan et al. 2021 2020), but they are also more consistent in supervised GNN settings (Hajiramezanali et al. 2023).
While the explainability of graph representations remains largely unexplored, several methods have been proposed to explain unsupervised models for unstructured data. The RELAX and LFXAI approaches share the goal of identifying features in the sample being explained (i.e., the explicand) that, if removed, would cause the altered representation to diverge from the original representation of the explicand (Lin et al.) 2023). Chuang et al. (2023) introduce the contrastive real-time explanation (CoRTX) framework, which utilizes contrastive learning techniques. Unlike previous feature- based methods, Lin al. (2023) introduce contrastive corpus attribution (COCOA), which allows users to choose corpus and foil samples in order to
3

---

Published in Transactions on Machine Learning Research (10/2024)
identify features that make the explicand's representation similar to the selected corpus, but dissimilar to the foil samples. These methods do not directly generalive explainability methods designed for GNNs, which often take into account the -on-Euclidean nature of the data, arbitrary size, and complex topological structure. Furthermore, the large number of possible subgraphs makes it more difficult to explain GNNs than their counterparts in Convolutional Neural Networks (CNNs). In contrast to these works and in line with Crabbé & van der Schaar (2022), our framework focuses on extending and generalizing existing graph explainability methods (which account for GNN-specific features) the unsupervised representation learning case.
3 Preliminary
Our framework can generalize most of the existing score-based explainability methods to an unsupervised setting. Therefore, we first outline supervised score-based graph explainability and then overview two types as proof-of-concept examples. We will show how our framework can generalize them to explain (unsupervised) graph representations in Section 4 Specifically, we will discuss SubgraphX, which is currently the ofate-of-the-
art perturbation-basee model (Yuan et al.) 2020 Hajiramezanali et al.] 2023). followed by two computationally efficient gradient-based methods (Integrated Gradients and Saliency)- Additional preliminaries less related to the proposed method, including graph self-supervised learning and TAGE, can be found in Appondix B
Notation. Let G = EV,E) denote a graph on nodes V and edges & with the adjacency matrix A and M-dimensional node attributes X RNN where N [... We are given trained GNN model f(),, which is optimized on all graphs (or nodes) in the training set and is then used for prodictions. For graph classification, f()) G + y maps each inpul graph G E G to an output YG J(G) E y. For node
classification, f(.) : G - y maps each input node u E G to an output y. = f(v) E y. Please note that y c dR and d, >>1 1.
3.1 Score-based graph explainability in supervised settings
Given a pre-trained GNN model -(): - y, and assuming {G()) is the set of subgraphs of the input graph G E G, where n is the total number of subgraphs, these methods assign a score to each subgraph GO E G based on its importance to the GNN prediction GG). Here, fG) denotes the j-th element of the GNN output. Intuitively, the score functions aim to identily the most. important subgraphs (G*) that, when removed, decrease the predicted probability of class of interest the most. Therefore, the main component of such models the scoring function Score(f}() G, G(O), which calculates the importance of each possible subgraph with respect to a specific label of interest (Figure 1, left).
A straightforward way to obtain G is to enumerate all possible subgraphs in G' (i.e. (G()) ) calculate their scores using the scoring function, and select the most important one as the explanation. However, this brute-force approach is an intractable combinatorial problem, as the number of potential candidates increases exponentially. Different approaches address this issue various ways. Some methods incorporate search algorithms to efficiently explore the (connected) subgraphs, as seen in SubgraphX, while others calculate the importance of each node separately, similar to the gradient-based methods. The latter approach leads to disconnected subgraphs.
3.2 SubgraphX
Key properties of graphs can often be altributed to important and localized structural information. The goal of SubgraphX (Yuan et al. 2021) is to identify the most important connected subgraph within the input graph that contributes to the GNN's predictions. To achieve this, SubgraphX uses a scoring function to evaluate the importance of different subgraphs based on their interactions with the trained GNN.
Consider the set of connected subgraphs of G as [G where n is the total number of connected subgraphs. SubgraphX explains the GNN prediction y, for the input graph G as:
G = arg max Score Shapley (G()),GG())
(1)
|G00| Nain
4

---

Published in Transactions on Machine Learning Research (10/2024)
where f)) denotes the j-th element of the GNN output, Nuin is upper bound on the size of a subgraph, and the Shapley value (Kuhn & Tucker 1953 has been used as the scoring function. To efficiently explore the space of possible connected subgraphs, SubgraphX employs Monte Carlo Tree Search (MCTS) (Jin et al.) 2020; Silver et al. 2017) to guide the search process. Specifically, it builds a search tree in which the root is associated with the input graph and each of the other nodes corresponds to a connected subgraph. Each odge the search tree denotes that the graph associated with a. child node can be obtained by performing node-pruning from the graph associated with its parent node (Yuan et al. 2021). Please note that Nmin in equation is hyperparameter of MCTS, which controls the size of the important subgraphs.
3.3 Gradient-based methods
Gradient-based methods, including Integrated Gradients (Sundararajan et al. 2017) and Saliency Simonyan et al. 2013), address the problem of attributing the prediction of a black- box, here a GNN, to its imput leatures. Saliency a simple approach for computing input attribution, returning the gradient of the output with respect to the imput features. This approach can be understood as taking a first-order Taylor expansion of the network at the input. and the gradients are simply the coefficients of each feature in the lincar representation of the model (Simonyan et al. 2013). The value of these coefficients can be interpreted the importance of the features (nodes/edgg /dges) in explaining the output of the black-box model. Integrated Gradients (IG) (Sundararajan et al. 2017] computes the integral of the gradients with respect to the inputs along the path from a given vector. Similar to Saliency, an attribution at input x relative to a bascline vector X5 represents the contribution each individual input feature (uodes/edges)) to the model prediction.
Formally, given an input graph G and a GNN classifier f()), these methods rank the nodes e E G based their influence on the predicted label fG).. Specifically, the importance of each node o; to the GNN output can be written as Score(f, G, vi), where {w:}[[ are the nodes of G. We can then explain a GNN prediction based on disconnecled subgraph using the lop-ranked nodes as G = {u.) Here, V. is the total number of important nodes.
4
Method
The grXAI framework is designed to provide explainability for unsupervised GNN models by identifying important subgraphs that can explain the d-dimensional representation vector generated by these models. However, current graph explainability methods have limited capability in handling d-dimensional embeddings, as their score functions are only applicable to a single dimension of the output (Xie al.) 2022). This limitation makes it challenging to determine which models' output(s) interpret in unsupervised settings, as representation dimensions do not generally correspond to any meaningful quantity. To overcome this limitation, we first outline score-based graph explanations for d-dimensional outputs of GNNs in supervised setting, gaining insights that will enable us to extend the framework to unsupervised GNN settings.
4.1 Explaining d-dimensional GNNs with score- based methods
Score-based graph explainability methods aim to explain the predictions of GNN models by computing an importance score Score(f()), G, G()) for cach subgraph G of G. However, as mentioned in Section 3 existing importance score functions are designed to be applied to individual elements of the soflmax output, represented by f.) IR for some 1..... dy, and cannot handle high-dimensional probability vectors. As result, they approximate Sf G, G(O) as Score(f)()), G, G(O), where arg max f+).). That is, the importance score is solely based on the predicted class with the highest probability.
This approach limits the comprehensiveness of the graph explanation. Indeed, it does not account for the inherent uncortainty in the model's predictions, particularly in multi-class problems. Thus, it hinders a complete understanding of the model's behavior. One possible solution to address this issue is to compute the importance scores for each output component separately and then combine them into weighted sum
5

---

Published in Transactions on Machine Learning Research (10/2024)
across all components, using the associated class probabilities as weights, follows:
dy
Scove Naive (f(),, Σ ((G)))))or(a())G...))))
(2)
This formulation considers the contribution of each class and is therefore accurate when class probabilities are more balanced, unlike the standard definition, which is only accurate for low-uncerlainty predictions (i.e., when fG) x 1). However, using equation 2 has important limitations. Primarily, searching for (connected) important subgraphs can be computationally expensive given the high number of output dimensions. Specifi- cally, this approach requires calculating the score function for each input graph dy times (one for each output. dimension), resulting in total of d, n importance scores per input graph.
Additionally, while using equation 2 to find connected subgraphs is theorctically possible-by plugging the naive score function from equation into equation | to examine all subgraphs and select the one with the highest score- score-this approach is intractable due to the additional computational complexity of searching for connected subgraphs. In the case of SubgraphX, to manage this additional complexity, we identify important subgraphs for each output dimension separately using the standard implementation and then merge them. That is, instead of directly computing G arg |ax we identify the important connected subgraphs for each output dimension &s individually using G arg max | Shareih (J& (-), G, G(),, and then construct G* U G where Gÿ is the important. subgraph of k-th element of GNN output.
Σ 2k=1 fG(G) Score(fa()), G(O)
While this approach can reduce the computational load (given that Nuia: can be smaller than Nmin to achieve the same level of sparsity); it may result in a disconnected subgraph even when a connected subgraph is desired, which could limit the effectiveness and comprehensiveness of the graph explanation.
To address the aforementioned issues, we propose leverage the lincarity property of importance scores with respect to the GNN. This property is common among score functions used for graph explainability, including Shapley, Saliency, and IG (Crabbé & van der Schaar 2022 Therefore, propose to rewrite the weighted importance score in equation as follows:
dy Scoresup(f(),G,c = Score > (fe(G) (.0,,GGG))
(3)
where Scoresup refers the scoring function for d-dimensional GNNs output in the supervised settings. The importance score be computed efficiently through the auziliary function ((G()) E fie(G) (a() for all subgraphs (+) E G. This approach avoids the need to compute the importance scores for each outpul component individually, thus significantly reducing the computational complexity. Furthermore, it guarantees explanations consisting of connected subgraphs when applied to a score-based method that returns connected subgraphs.
The approximation in equation g is applicable to most supervisod score-based graph explanation methods that compute linear importance scores, including SubgraphX (Yuan al.] 2021). Rather than interpreting individual elements of the argmax output, normally done in SubgraphX, one equation to explain supervised GNNs based on all dimensions of their soft output. Therefore, can incorporate this approximation into equation by modifying it as follows:
Remark 1.
G
arg max Score Shapley (M(C G,,(()) C,
(4)
|G00 (
Remark 2. Tt is noteworthy that equation can also be easily extended to gradient-based models such Saliency (and IG). In particular, we can calculate the node importance with respect to the GNN output as Score sa (f(), G, u)) (at fa(:),, G, wi), where Score sat is the Saliency value.
6

---

Published in Transactions on Machine Learning Research (10/2024)
4.2 Explanation for graph representations
Notation. In an unsupervised setting, the trained GNN encoder f(() maps graphs (or nodes) to a latent space. For graph representation learning, f(() : + H maps each input graph G E g to the latent representation hag =f E H C Rdd,, where < dx. For node embedding, f(-) : + H maps each imput node e E G to latent vector h = f(u) E H. The goal is to identify important subgraphs that explain the behavior of the GNN and contribute to the latent, representation of the graph or node. Ideally, the importance score should reflect the contribution of subgraph G to the representation he = f(G) or h = f(v). However, in this setting, there is no principled way to select a particular output component fx(() for some & E 1,...,dh, da to be utilized in the current explainability methods (Figure ! right).
Equation proposes a framework that offers the benefit of approximating the importance scores of the entire d-dimensional output probability vector, eliminating the need to select a specific output class, which is a common requirement in many existing ccore-based graph explainability methods. This enables the extension of the same approach to explain graph representations, which is the focus of this paper. However, it is essential to note that while (G) in equation 3 corresponds to the class probability and E (xG) 1 in the supervised setting due to softmax output, the individual dimensions of a latent vector do not necessarily correspond to probabilities.
To address this limitation in our unsupervised setting, we replaced the weighted sum in equation 3 with the cosine similarily, which can be seen a method of normalization and is invariant with respect permutations of the latent dimension: P
Definition 3.2. Given trained GNN encoder f(.) that maps graphs to a latent space H, the importance score is defined as:
Scorerep G, ())) = Score(,, G, G()),
(5)
where Scorerep is referring to the score function for graph representation, vb G - IR is an auxiliary function
defined for all subgraphs GO E G using ((G))= Σde d (((G)(((()))
[( [[Gay where d, is the dimensionality of the latent space H. It is important to note that this importance score can also be modified for node representation learning, where the input graph G would be replaced with single node G, and the subgraph (1 would be a subset of the nodes and edges connected to v.
Extending SubgraphX for graph representation explainability using grX (grSubgraphX). Our framework is applicable to most existing score- based graph explainability methods. In the following, we focus on the generalization of SubgraphX (Yuan al.] (021). The main reasons for our choice are as follows: 1) SubgraphX is currently SOTA for graph explainability (Yuan et al. 2020: Hajiramezanali al.] 2023). 2) its explanations consist of connected subgraphs, and 3) due to its computational complexity, using it for graph representations necessitates non-trivial extension.
In SubgraphX, the effectiveness of both the MCTS process and the selection of explanations rely heavily on the accuracy of the chosen scoring function. Therefore, it is crucial to prociscly approximate the importance of various subgraphs. To achieve this, we propose to modify SubgraphX using the importance score equation 5 for explaining graph representations. Let f(-) : - H. represent a GNN encoder, where G= = E) E G is a given graph with node set V- -VI,...,UNN Let G(s) (D>s), g(()) be the target subgraph with N, nodes, [o) (()),,(())))) is the set of UN.) }
where yr() = (u
[+) ...., ] } is the set of nodes in G(o), and v(o) = V v(+) = (1((0) i
..... ,
- N₂- We define the set of players P as P = (Gl)) /f ....,, (2)) all other nodes not in p(e), and N, = N-
where we consider the entire subgraph ((+) as single player. In the grXAI version of SubgraphX, named
2The latent space is not tied any fixed predetermined labels each axis This means that multiple latent spaces with permuted dimensions can be equivalent one another. Therefore, the set of transformations that preserve the geometry of the latent space, as determined by cosine similarity, is the set of orthogonal transformations.
7

---

Published in Transactions on Machine Learning Research (10/2024)
grSubgraphX, we approximate the Shapley value of the subgraph G(5) as:
Pow (++) = E
|R
PI
R
1)!
-
PPI
RCP
with b - R such that for all G E G, (GG!)) = Sc (1(0))
(6)
where R is a possible coalition set of players, G is connected graph, and Sc is cosine similarity. Algorithm in Appendix G includes the computation steps for grSubgraphX. The proposed approximation in equation 6 is both fair and accurate, as it considers all possible coalitions and adheres to the four fundamental axioms introduced by Lundberg & Lee (2017): efficiency, symmetry, linearity, and the dummy axiom. These desirable axioms guarantee the validity and impartiality of the explanations for graph representations.
grSaliency. Given an input graph G (with N nodes and M-dimensional node attributes X E RNXM), and GNN encoder s(.) G - H. we calculate the Saliency score for each node as follows. First, we calculate the bbb derivative w, = 80g E TR-M by backpropagation, where t is cosine similarity define in equation 6 After that, to derive a single Saliency score for each node, we take the magnitude of the average of w, across all node features: Score sat G, 1;) = |mean(w,)) (wa)l. Finally, we sort the nodes based on their scores and select the nodes with the highest Saliency values as the subgraph. Formally, G = {va(s))==1> V.V where x is a permutation of nodes { J such that Score sai (v v(()) 2 Score G, U+(2)) 2 Score Salresal1ss (ub,,, G, Uz(N)), and V. is the size of desired subgraph.
grIG. Similar to grSaliency, we calculate the TG score for each node individually and select nodes with the highest IG score for the explanation. Specifically, we first calculate w; v fa de de, (ax da. da. Then, we obtain the IG score for each node as Score IG (1b, G, vx) = |mean(w.). Similarly, the identified subgraph is G
(*x(e) V_ === where π is permutation of nodes {v;) ) such that Score ia (1))), G, (a())) 2 Score 1oreec(() G. (2())) 2
2 Score (v), G, (r())).
5
Experiments
To evaluate the effectiveness of the proposed method, we performed experiments on various datasets, graph learning models, and explainability methods. We evaluated our grXAT framework on seven datasets in both unsupervised and supervised settings, covering synthetic, biological, citation network, and text data. The details of the experimental scttings are included in Appendix F.
MUTAG (Debnath et al. 1991 BBBP (Wu et al., 2018), BACE (Wu et al., 2018), and NCII You et al. 2020 are molecular datasets for graph representation learning. Each graph in these datasets represents a molecule, with nodes representing atoms and edges representing bonds. The labels for these datasets correspond to molecular properties and biological activities. BA-Shapes (Yuan et al. 2020 2021 is a synthetic node classification dataset with 4 unique node labels. Each graph includes a base Barabasi-Albert (BA) graph with embedded five-node house-like motifs. The labels for each node are determined by whether it belongs to the base graph or different parts of the motif. The Graph-Twin Twitter (Yuan al.) (020) dataset is a sentiment graph classification dataset with 3 labels. Yuan et al. (2020) convert each tweet sequence into graph, where each node represents word and edges are the relationships between words. Cora (Sen et al. 2008 is citation network for node embedding tasks.
5.1
Evaluation metrics
As the considered real-world datasets do not provide ground truth for explanations, we follow previous studies (Pope et al.] 2019 Xie et al.) 2022 Yuan et al. 2020) and adopt Fidelity and Sparsity scores to quantitatively evaluate the explanations.
Fidelity. This metric is the main available metric for evaluating post-hoc graph explanations. It assesses whether the input subgraphs identified by the explanation method are important (Yuan al., 2020). If so, then removing these subgraphs should result in a significant change in the model's outputs. In the supervisod setting, Fidelity is calculated as the difference in predicted probability between the original predictions and
8

---

Published in Transactions on Machine Learning Research (10/2024)
Frdellyyo
I Pulirnpi plaleryy Inee
grauyyuu and TAGE
Fidelnyes
075.111a77 getl
pidaltyo
Sparaity
Sparsity
Sparsity
Sparsity
Coariity
Figure 2: Quantitative results for various explanation techniques, with a preference for higher Fidelitycs in higher Sparsity levels.
the new predictions after masking out important subgraphs Pope et al. 2019 Yuan et al. 2021). Formally, given a graph G, the softmax output of the predicted class fe(-), and its explanation as the subgraph G", we define G = G \ G as the other graph, i.e. the graph of all possible nodes that are not in G The Fidelity score can be computed as Fidelity(G) = fc(G) - Gc()))
Building on the original definition, we use three types of Fidelity for evaluating explanations for graph representations: (i) Fidelitycs, which measures the negative cosine similarity between the original em- bedding and the new embedding after masking out the important subgraph. Formally, this is calculated as (S.(( ); (ii) Fidelityprob, which evaluates the relative importance of the subgraphs on a downstream task. Specifically, it measures the difference between the predicted probability of the original embedding and the new embedding after masking out the important subgraph. The predicted probability is given by logistic regression model trained on the original emboddings.
Sparsity. Effective explanations should be sparse, which means they should capture the most important subgraphs and ignore the irrelevant ones (Yuan et al. 2020). Post-hoc graph explainability methods, including gTXAI, can control the size of the identified subgraph through various hyperparamcters, and the Sparsity metric measures the proportion of structures identified as important by the explanation method, It is important to note that accounting for Sparsity promotes a fair comparison between different methods. Indeed, larger subgraphs generally improve Fidelity, and therefore explanations with different sizes are not directly comparable. By comparing Fidelity at the same Sparsity, we compare explanations with the same size.
5.2 Results and discussion
Quantitative studies. We evaluate the quality of graph representations in terms of Fidelity and Sparsity scores. Specifically, we assess the Fidelity of both raw embedding vectors (Fidelitycs)) and their downstream tasks (Fidelitypros) by comparing our grXAI-based methods (grSubgraphX, grIG, and grSaliency) with the only available baseline, TAGE (Xie et al. 2022p. Please refer to Appendix B.2 for detailed overview of TAGE. The naive approach (equation 2 is not included due to its significant time cost on real-world datasets. We trained GNN encoders using two different amgs for self-supervised graph representation learning, InfoGraph (Sun et al. 2019) and GraphCL YYou et al. 2020
The performance of the grXAT framework is demonstrated in Figure 2 which shows better Fidelity (higher Fidelitycs) consistently across all levels of Sparsity. This conclusion also supported by Table 2 (a higher pide corresponds to better performance). Notably, our gradient- based extensions, particularly grIG, perform exceptionally well on multiple datascts, despite their lower computational complexity compared to grSubgraphX.
The grSubgraphX approach, the only method that explains graph representation based on connected subgraphs, outperforms TAGE on all datasots except Graph-Twitter. This is likely due to the nature of molecules, where localized functional groups significantly affect global molecular properties. Hence, there is a preference for an inductive bias toward explaining based on connected subgraphs. However, sentiment analysis requires more
9

---

Published in Transactions on Machine Learning Research (10/2024)
Table 2: Fidelityprod scores with controlled Sparsity on a downstream molecular property prediction task based the graph-level embeddings.
SSLGraph
Explainer
MUTAG (1)
Twitter (1)
BBBP (1)
NCI (1)
BACE (1)
gSSubgraphX (ours)
0.44= 0.39
0.014 0.17
0.68 + 0.31
0.13 = 0.38
0.18 + 0.53
GraphCL
grSalieney (ours) grIG (ours)
0.39# 0.68
0.04# 0.26
0.31 0.51
0.09 0.41
0.07 0.52
0.43# 0.73
0.04+ 0.26
0.14 0.49
0.10 0.40
0.07 = 0.48
TAGE (Xie et al. 2022
0.21# 0.33
0.01 0.15
-0.05 + 0.24
0.07 = 0.32
0.06 = 0.58
gSSubgraphX (ours)
0.45= 0.60
0.04 0.23
0.01+ 0.03
0.15# 0.29
0.25 0.30
InfoGraph
grSaliency (ours)
0.381 0.55
0.111 0.33
0.00 0.02
0.131 0.33
0.144 0.11
grIG (ours)
0.45 0.59
0.12: 0.34
0.001 0.03
0.141 0.34
0.101 0.81
TAGE (Xic et al. 2022
0.361 0.52
0.041 0.21
0.011 0.21
0.10 0.30
0.061 0.89
Non-mutagenic
Mutagenic
TAME prthbarratt
TAOE Peonoeonn
Figure 3: (Left) The explainability of graph representations for six molecules from the MUTAG dataset. Carbon, Oxygen, and Nitrogen atoms are highlighted in yellow, red, and blue, respectively. The top row displays the grSubgraphX explanation, which is connected subgraph that is more easily interpretable for molecular graphs. In particular, the grSubgraphX explanation often identifies carbon ring for mutagenic compounds, which is consistent with the prior knowledge of the underlying chemistry. (Right) The explainability of graph representations for five molecules with minor structural changes from the MUTAG dataset. The grSubgraphX method generates highly stable explanations, while TAGE is not as stable, ie., small changes to the molecular graph produce substantially different explanations.
flexible approach to handling disconnected subgraphs. Our gradient-based techniques, grIG and grSaliency, outperform grSubgraphX in such cases since they do not have any constraints providing connected subgraphs. This flexibility allows them to perform betler in scenarios where disconnected subgraphs play a vital role in the task.
In addition to Fidelitycs, we also compare grSubgraphX to TAGE based on the Infidelityes metric, where Infidelity -. (JGG), (G(+))) While Fidelitycs measures how much the representation changes when important subgraphs are masked, Infidelitycs directly measures how much the important subgraphs contribute to the reprosentation. In contrast to Fidelitycs, lower Infidelityes is desired, mcaning that the reprosentation of the identified important subgraph is close to the original graph representation. Figure Al shows the results on this metric. As shown, gTXAT outperforms the baseline in terms of Infidelitycs. particularly in regions of higher sparsity. In most cases, the performance is significantly better across a range of sparsity levels. This is mostly because according to equations ! and based on minimizing the Infidelity metric.
B our model's optimization to find the important subgraph is
Qualitative studies. To determine the effectiveness of explaining by connected subgraphs in real-world molecule datasets, conducted further investigations using the MUTAG dataset, for which deeper understanding of the underlying mechanism connecting structural features to the property is available (Yuan) ct al. 2021). Specifically, it is known that carbon rings tend to be mutagenic (Debnath et al. 1991p. We study whether the explanations provided by different methods can identify the carbon rings characterizing the positive class. Our results, presented in Figure 3 (left), demonstrate that 1) our grSubgraphX successfully and precisely identifies the carbon rings as important subgraphs for the mutagenic class; and 2) grSubgraphX provides more localized and interpretable explanations for both classes. This is a critical factor in understanding molecular properties and aiding scientific decision-making, making grSubgraphX valuable tool for the field.
10

---

Published in Transactions on Machine Learning Research (10/2024)
Stability study. There have been arguments that post-hoe (graph) explainability methods may lack stability (Adebayo et al. 2018), even negligibly slight changes to an instance can lead to significantly different explanations (Hajiramezanali et al., 2023). To test the stability of the grSubgraphX method, we compare it with TAGE using the real-world MUTAG dataset. Results in Figure (right) and Figure A5 indicate that grSubgraphX is able to provide consistent explanations for molecules with minor structural changes, while TAGE produces radically different results in those cases, thus highlighting the stability of our framework. The stability of grSubgraphX is due to its ability to evaluate the importance of subgraphs as a set of interconnected nodes instead of considering the importance of individual edges separately, as is done in TAGE.
Efficiency study. Our framework significantly decreases the computational complexity associated with graph representation explainability from (graphXXAI x dn, to O(graphXAI), where O(graphXAI) is the computational complexity of the original graph explainability method, and da denotes the number of hidden dimensions (o.g., 512 in the case of InfoGraph). To assess the efficiency of our proposed grSubgraphX, we follow the methodology employed in (Yuan et al. 2021 Zhang et. al., 2022) and calculate the average time taken to generate explanations for graphs from the BBBP dataset, with each graph containing average of 24.04 nodes.
Table 3 shows that the naive calculation of SubgraphX
Table 3: Average running time for explaining graph representation of InfoGraph for BBBP.
(equation 2] to explain the embedding of InfoGraph re- quires approximately 11 hours per graph. In contrast, our proposed grSubgraphX method accomplishes the same task in less than 2 minutes per graph. Consequently, uti- lizing the naive approach would take minimum of 916 days to explain a small molecule dataset comprising 2k samples; however, grXAI archives it within 2 days.
Method Naive SubgraphX
grSubgraphX (ours)
TIME
11 hours
106.32 sec
Explanation for node embedding. Although we have focused on graph-level learning tasks, our grXAI framework also applies to node embedding tasks. Figure 4 shows a GNN trained by the GRACE (Zhu et al. 2020) algorithm on the Cora. dataset (Sen et al. 2008), with explanations given by grSubgraphX (top) and TAGE (bottom). Qual- itatively, we observe that grSubgraphX identifies neighbor nodes that are densely connected and have the same class as the target node (i.e., the node we are explaining). In contrast, TAGE ex- planations are more disconnected, scattered, and difficult to interpret. Quantitatively, grSubgraphX achieves better Fidelitycs scores across varying Sparsity values (Figure A2 demonstrating that our framework also improves the performance in node embedding tasks.
Top 5 nodes
Top 20 nodes
TAGE
Figure 4: Explanations for a specific node in the Cora datasct. The plots show part of the Cora graph, high- lighting the target node and its 2-hops neighboring nodes. The target node is shown as a larger green circle, with different colors denoting node labels. The edges connect- ing the most important nodes are bold.
5.2.1 grXAl in supervised settings
Although grXAl has been primarily designed improve performance and efficiency in explaining unsupervised GNNs, natural question is whether it maintains the same performance in supervised settings. In the following, we show that not only grXAl versions of the explainability methods achieve identical performance to their original counterparts with little additional computational overhcad, but they also surpass them in cases with high-uncerlainty predictions. Specifically, we compare our grXAI modificatiins (grIG, grSaliency, and grSubgraphX) to their original counterparts on a variety datasets and tasks, including BBBP, MUTAG, and Twitter datasets for graph classification and BA-shape for node classification. As expected, our grXAI results achieve equivalent performance in this supervised setting (Figure A3 Appendix). This is mainly because, in most cases, the GNNs generate highly confident predictions, ie.. fc.G. (or fc(v,) x 1 in
11

---

Published in Transactions on Machine Learning Research (10/2024)
gruegraph
SubgmphX
PGE
Tweet
Tweet (IG: Negative Tweet PC) Neutral 4811
Tweet (anc)
Tweet (IS: Noura 470.77) Tweat 0o: Posiie peo
Figure 5: grXAT in supervised seltings. (Left)) Explanation results on the BA-Shape dataset. The target node is shown in larger size. Different colors denote node labels. Each graph contains base graph (yellow) and a housc-like five-node motif. grSubgraphX (similar to SubgraphX) preciscly identifies the motifs as the explanations. (Right) Explanation of a GCN classifier for the Graph-Twitter dataset. The blue nodes represent words. Important subgraphs (edges and nodes) are highlighted in yellow. In the first tweet, we removed the words as the tweet was an opinion about specific person.
node classification). Node classification results on BA-Shape dataset are shown in Figure 5 (left), with the important substructures highlighted in bold. We observe that the grSubgraphX identifies motifs that are consistent with the ground truth.
To investigate when grXAl outperforms the original counterparts in supervised settings, we examined specific examples from the Twitter dataset with high uncertainty predictions, i.e., fa(G.) s 0.5. Figure 5 (right) shows IG and grIG explanations for two such examples. grlG simultaneously explains both predictions with two subgraphs that correspond to both labels with a similar probability. Instead, IG explainability changes depending on whether the true class or the predicted class is used. This illustrates that traditional methods based on a single output may not always accurately and reliably explain the GNN, particularly in cases of high uncertainly, whereas our d-dimensional vector-based approach can provide explanations that directly reflect soft distribution over multiple classes.
6 Discussion
In this paper, we introduced grXAI, novel framework to extend score-based graph explainability methods to unsupervised GNN settings. We analyzed our framework and demonstrated its compatibility with many score-based methods, thus highlighting its flexibility. We validated grXAl applied to different self-supervised graph representation learning, self-supervised node embedding, and supervised models across several datasets, both qualitatively and quantitatively. Overall, grXAI leads to accurate and stable explanations which can be computed efficiently and outperform perriously-propoeee method. We also investigated the impact of different augmentation strategics on the explainability of graph reprosentations (Appendix D). This new perspective can help domain experts to identify which augmentation technique is more closely related to the main molecular property of interest, leading to semantically meaningful design choices, and ultimately enhancing the effectiveness of self-supervised graph representation learning methods.
While our framework has demonstrated success in explaining graph representations in an efficient way, it important to acknowledge its limitations. The gAXAT framework identifies subgraphs that increase the similarity to the original graph in the representation space from any direction. However, in practical applications, some of these directions may not be meaningful equally important. Instead, the goal may be to identify subgraphs that increase the similarity to target graph with desired properties, while simultancously scparating them from set of graphs with undesirable propertics in the representation space, such as non-mutagenic molecules. This is similar to how human perception operates (Lin et al. 2023). We believe grX opens the door to several exciting avenues for future research, such as developing example based explainability of graph representation by examining similarity to other graphs' representations (Lin et al. 2023), and extending grXAI to the structure-aware scoring functions, for instance, by adapting GStarX (Zhang et al. 2022].
12

---

Published in Transactions on Machine Learning Research (10/2024)
References
Julius Adebayo, Justin Gilmer, Michael Muelly, Tan Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. Advances in neural information processing systems, 31, 2018.
Federico Baldassarre and Hossein Azizpour. Explainability techniques for graph convolutional networks. In International Conference Machine Learning (ICML) Workshops, 2019 Workshop Learning and Reasoning with Graph-Structured Represenlations, 2019.
Yu-Neng Chuang, Guanchu Wang, Fan Yang, Quan Zhou, Pushkar Tripathi, Xuanting Cai, and Xia Hu. CoRTX: Contrastive framework for real-time explanation. In The Fleventh International Conference on Learning Representations, 2023. URL http:////ppppaaeeeeeaooooo//aaaa
Jonathan Crabbé and Mihaela van der Schaar. Label-free explainability for unsupervised models. In Kamalika Chaudhuri, Stefanic Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (ods.), Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learving Research, pp. 4391 4420. PMLR, 17-23 Jul 2022.
Asim Kumar Debnath, Rosa I. Lopez de Compadre, Gargi Debmath, Alan J Shusterman, and Corwin Hansch. Structure-activity relationship of mutagenic aromatic and heteroaromatic nitro compounds. correlation with molecular orbital energies and hydrophobicity. Journal of medicinal chemistry, 32(2):786 797, 1991.
Jacob Devlin, Ming-Was Wei Chang, Kenton Lec, and Kristina Toutanova. Bert: Pro-training of deep bidirectional transformers for language understanding. urXiv preprinl Xix:18100044800 2018.
Matt Gardner, Joel Grus, Mark Neumann, Oyvind Tafjord, Pradeep Dasigi, Nelson Liu, Matthew Peters, Michael Schmitz, and Luke Zeemoer.. Allennlp: A deep semantic natural language processing platform. arXiv preprint 1XX3.008807764 2018
Ehsan Hajiramezanali, Arman Hasanzadeh, Krishna Narayanan, Nick Duffield, Mingyuan Zhou, and Xiaoning Qian. Variational graph recurrent neural networks. Advances in neural information processing systems, 32, 2019.
Ehsan Hajiramezanali, Arman Hasanzadeh, Nick Duffield, Krishna Narayanan, and Xiaoning Qian. Bayrel: Bayesian relational learning for multi-omics data integration. Advances in Newral Information Processing Systems, 33:19251-1999 19263, 2020.
Ehsan Hajiramezanali, Sepideh Maleki, Alex Tseng, Aicha Ben Taieb, Gabriele Scalia, and Tommaso Biancalani. On the consistency of gnn explainability methods. In XAI in Action: Past, Present, and Future Applications, 2023.
Arman Hasanzadeh, Ehsan Hajiramezanali, Krishna Narayanan, Nick Duffield, Mingyuan Zhou, and Xiaoning Qian. Semi-implieit graph variational auto-encoders. Advances in neural information processing systems, 32, 2019.
Arman Hasanzadeh, Ehsan Hajiramezanali, Shahin Boluki, Mingy Zhou, Nick Duffield, Krishna Narayanan, and Xiaoning Qian. Bayesian graph neural networks with adaptive connection sampling. In International conference machine learning, pp. 4094-4104. PMLR, 2020.
Arman Hasanzadeh, Mohammadreza Armandpour, Ehsan Hajiramezanali, Mingyuan Zhou, Nick Duffield, and Krishna Narayanan. Bayesian graph contrastive learning. arXiv preprint arXiv:2112.075: 2021.
Arman Hasanzadeh, Ehsan Hajiramezanali, Nick Duffield, and Xiaoning Qian. Morel: Multi-omics relational learning. Xix preprint arXiv:2203.088149 2022.
Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec. Strategies for pre-training graph neural networks. Xiv preprint arXie: 1 12265, 2019.
Rundong Huang, Farhad Shirani, and Dongsheng Luo. Factorized explainer for graph neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pp.
2024.
13

---

Published in Transactions on Machine Learning Research (10/2024)
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Multi-objective molecule generation using interpretable substructures. In International conference on machine learning, pp. 4849-4859 PMLR,, 2020.
Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In International Conference on Machine Learning, pp. 5338-533300 5348. PMLR, 2020.
Narine Kokhlikyan, Vivek Miglani, Miguel Martin, Edward Wang, Bilal Alsallakh, Jonathan Reynolds, Alexander Melnikov, Natalia Kliushkina, Carlos Araya, Siqi Yan, and Orion Reblitz-Richardson. Captum: A unified and generic model interpretability library for pytorch, 2020.
Harold William Kuhn and Albert William Tucker. Contributions to the Theory of Games, volume 2. Princcton University Press, 1953.
Chris Lin, Hugh Chen, Chanwoo Kim, and Su-In Lec. Contrastive corpus attribution for explaining representations. In The Fleventh International Conference Learning Represenlations, 2023. URL
https://openrevieww net/foorunnid-eeWNNBBB.
Meng Liu, Youzhi Luo, Limei Wang, Yaochen Xie, Hao Yuan, Shurui Gui, Haiyang Yu, Zhao Xu, Jingtun Zhang. Yi Liu, Keqiang Yan, Haoran Liu, Cong Fu, Bora. M Oztekin, Xuan Zhang, and Shuiwang Ji. DIG: A turnkey library for diving into graph deep learning research. Journal of Machine Learning Research, 22
(240):1-9, 2021. URL http://jmlr http://ww..rrpppppp////////////0
html
Stephen Zhewen Lu, Ziqing Lu, Ehsan Hajiramezanali, Tommaso Biancalani, Yoshua Bengio, Gabriele Scalia, and Michal Koziarski. Cell morphology- souphology-guided small molecule generation with gflownets. In ICML 2024 Workshop on Structured Probabilistic Inference (16) Generative Modeling, 2024.
Scott M Lundberg and Su-In Tee. A unified approach to interpreting model predictions. In Advances in neural information processing systems, pp. 4765 4774, 2017.
Dongsheng Luo, Wei Cheng, Dongkuan Xu, Wenchao Yu, Bo Zong, Haifeng Chen, and Xiang Zhang. Parameterized explainer for graph neural network, 2020. URL https://arxiv..o www.:oorg//bbao htrg/abs/0011.0455 045773
Brooks Paige, Bal-Willem van de Meent, Alban Desmaison, Noah Goodman; Pushmeet Kohli, Frank Wood, Philip Torr, al. Learning disentangled representations with semi-supervised deep generative models. Advances in newral information processing systems, 30, 2017.
Adam Paszke, Sam Gross, Francisco Massa, Adamn Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, perfommrfoe deep learning library. In Advances in Neural Information Processing Systems 32, pp. 8024-8035. Curran Associates, Inc., 2019.
Phillip E Pope, Soheil Kolouri, Mohammad Rostami, Charles E Martin, and Heiko Hoffmann. Explainability methods for graph convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pottern Recognition, pp. 10772-10788 2019.
Prithviraj Sen, Galileo Namala, Mustafa Bilgie, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Collective classification in network data. Al magazine, 29(3):93 93, 2008.
David Silver, Julian Schritwwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. Mastering the game of go without human knowledge. nature, 550(7676)3354 359, 2017.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprinl arXiv:1312.603/, 2013.
Dylan Slack, Anna Hilgard, Sameer Singh, and Himabindu Lakkaraju. Reliable post hoc explanations: Modeling uncertainty in explainability. Advonces neural information processing systems, 34:9391-9400 2021.
14

---

Published in Transactions on Machine Learning Research (10/2024)
Fan- Yun Sun, Jordan Hoffmaann, Vikas Verma, and Jian Tang. Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization. orXie preprint Xiv: 0019990..00000 2019.
Ruoxi Sun. Does gnn pretraining help molecular representation? orXiv preprint ar arXiv:2207.0001 2022.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International conference on machine learning, pp. 3319 3328. PMLR, 2017.
Kristoffer K Wickstrøm, Daniel J Trosten, Sigurd Løkse, Ahcène Boubekki, Karl Øyvind Mikalsen, Michael C Kampffmeyer, and Robert Jenssen. Relax: Representation learning explainability, International Journal of Compuler Vision, pp. 1 27, 2023.
Zhenqin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Ancesh S Pappu, Karl Leswing, and Vijay Pande. Moleculenet: benchmark for molecular machine learning. Chemioal science, 9 (2):513-55 530, 2018.
Yaochen Xie, Sumeet Katariya, Xianfeng Tang, Edward Huang, Nikhil Rao, Karthik Subbian, and Shuiwang Ji. Task-agnostic graph explanations. In Advances in Newral Information Processing Systems, volume 35, pp. 12027 12039. Curran Associales, Inc., 2022.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? International Conference on Learning Representations, 2019. URL https://openreviost htw.pppawrrrro..eeooooooriiiiiie net/forummid
ryGs61A5Km
Ziyuan Ye, Rihan Huang, Qilin Wu, and Quanying Liu. Same: Uncovering gun black box with structure-aware shapley-based multipiece explanations. Advances in Neural Information Processing Systems, 36, 2024.
Zhitao Ying, Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec. Gnnexplainer: Generating explanations for graph neural networks. Advances in neural information processing systems, 32, 2019.
Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph contrastive learning with augmentations. Advances in Newral Information Processing Systems, 33:5812-5822 2-222 2020.
Hao Yuan, Haiyang Yu, Shurui Gui, and Shuiwang Ji. Explainability in graph neural networks: taxonomic survey. IEEE transactions on pattern analysis and machine intelligence, PP, 2020.
Hao Yuan, Haiyang Yu, Jie Wang, Kang Li, and Shuiwang Ji. On explainability of graph neural networks via subgraph explorations. In International Conference on Machine Learning, pp. 12241- PMTR, 2021.
Shichang Zhang, Yozen Liu, Neil Shah, and Yizhou Sun. Gstarx: Explaining graph neural networks with structure-aware cooperative games. In Advances in Neural Information Processing Systems, 2022.
Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Lüu, Shu Wu, and Liang Wang. Deep graph contrastive representation learning. ar preprint arx:v:000 0xX::2006.04111,,, 2020.
15

---

Published in Transactions on Machine Learning Research (10/2024)
A Details of datasets
We evaluated our grXAl framework on seven datasets in both unsupervised and supervised settings, covering synthetic, biological, citation network, and text data. The datasets are summarized as follows:
Graph-Twitter (Yaan et al. 2020) dataset is a sentiment graph classification dataset with labels. Yuan et al. (020) convert each tweet sequence into a graph, where each node represents a word and edges represent the relationships between words. Biaffine parser (Gardner et al. 2018 has been used to extract word dependencies, and pre-trained 12-layer BERT (Devlin et al. 2018 model has been used to extract a 768-dimensional feature vector for each word.
BA-Shape (Yuan et al. 2020) is a synthetic node classification dataset with 4 labels. Each graph contains a base graph (300 nodes) and a house-like five-node motif. The base graph is obtained by the Barabási-Albert (BA) model, which can generate random scale-frec networks with a preferential attachment mochanism. The motif attached to the base graph while random edges are added. Each node labeled based on whether it belongs to the base graph or different spatial locations of the motif.
Molecule dataset. Molecular datasets such MUTAG, BBBP, BACE, and NCT1 are widely used in explanation and graph representation learning tasks (You et al. 2020 Yuan et al. 2021 Sun 2022 Each graph in such datasets corresponds to molecule where nodes represent atoms and edges are the chemical bonds. The labels of molecular graphs are generally determined by the chemical functionalities or properties of the molecules. In particular, in the MUTAG dataset, molecular graphs are labeled based on their mutagenic effects on a bacterium. is known that carbon rings and NO2 chemical groups may lead to mutagenic effects Yuan al. 2021 Zhang ct al. 2022).
Cora Sen et al. 2008 is a citation network for node embedding tasks with seven different classes.
B Additional preliminaries
B.1 Graph self-supervised learning
We applied our explainer to two self-supervised graph-level representation learning models:
GraphCL (You al. 2020). GraphCL is contrastive learning framework for learning unsupervised reprosentations of graph data. GraphCl proposes several graph data augmentations well novel graph contrastive learning framework for GNN pre-training.
InfoGraph (Sun et al.] 2019). Infograph maximizes the mutual information between the graph-level representation and the representations of differont components with different scales (c.g.. nodes, edges, triangles). By doing so, the graph-level representations encode aspects of the data that are shared across different scales of substructures.
We also used GRACE (Zhu et al. 2020 for pro-training GNN encoders in node-level self-supervised settings. GRACE is an unsupervised graph representation learning framework that leverages contrastive objective at the node level. Specifically, it generates two graph views by corruption and learns node representations by maximizing the agreement of node representations in these two views.
B.2 Task-agnostic GNN explainer
TAGE, introduced by Xie al. (002)), decomposes a supervised GNN into an encoder model and a downstream model, designing separate explainers for each component. The embedding explainer part, which is the only available baseline for explaining graph representations, is trained using a self-supervised training framework.
TAGE aims to maximize the mutual information (MI) between two embeddings: one from the input graph G and one from the corresponding important subgraph G* induced by the explainer. It introduces a masking vector p to indicate specific dimensions of the embeddings to maximize the MI. During training, TAGE samples the masking vector p from a multivariate Laplace distribution to exploit sparse gradients, ensuring that only few dimensions of high importance (Xie al.] 2022). This assumes that embeddings from
16

---

Published in Transactions on Machine Learning Research (10/2024)
well-trained GNN encoders are informative with low dimension redundancy. Formally, the learning objective based on the restricted MT is:
max Ep (p ( S(G). p & ((To(p. G)))),
where MI) computes the mutual information between two random vectors, p denotes the random masking vector sampled from certain distribution, To(p, G) identifies the subgraph of high importance, and denotes the element-wise multiplication, which applies masking to the embeddings ʃ(-). Intuitively, given an inpul graph G and the desired embedding dimensions to be explained, the explainer predicts the subgraph whose embedding shares the maximum mutual information with the original embedding on the desired dimensions Xie et al., 2022).
To efficiently compute the mutual information, TAGE uses a contrastive loss such as InfoNCE. Additionally, to restrict the size of the subgraphs provided by the explainer, TAGE includes a size regularization term (Xie et al.
2022 TAGE employs multilayer perceptron (MLP) as the base architecture of the explainer To which predicts the importance score for each edge, leading to disconnected subgraphs.
C Additional experiments
Explanation for graph representations. Figure Al compares grSubgraphX and TAGE based on the Infidelitycs metric, where Infidelity = S. (f(G), )))))) Infidelitycs directly measures how much the important subgraphs contribute to the representation; therefore, lower Infidelitycs means the representation of the identified important subgraph close to the original graph representation. As shown, grXAl outperforms TAGE in terms of Infidelitycs in most cases. This mostly because according to equations and grSubgraphX optimization based on the Infidelity metric.
00 0E
36
Figure A1: Quantitative results for various explanation techniques, with a preference for lower Infidelitycs in higher Sparsity levels.
Explanation for node embedding. As we stated in the main paper, grSubgraphX has a better explanation performance in terms of both Fidelitycs and visualization. For evaluating this, we trained a GNN using GRACE (huu et al. 2020) on the citation network Cora. Then, we compare the performance of grSubgraphX to the baseline (TAGE) using test nodes that have at least 50 nodes in their 2-hop subnetworks. Figure A2 also shows that grSubgraphX outperforms TAGE in terms of Fidelitycs for different Sparsity levels. Please note that due to the higher number of classes in this datasct and the fact that the node embodding has been caleulated as the output of a two-layer GNN, the change in Fidelitycs is lower here compared to other experiments.
XA in supervised settings. We compare our grXAI-based methods, which employ d-dimensional softmax outputs (grIG, grSaliency, and grSubgraphX), with their original counterparts that rely on explaining the predicted class using the BBBP dataset (as shown in Figure A3 Our proposed grXAI framework has identical performance to its original counterparts but is more ellicient, especially for high-confiddence predictions. This is because supervised GNNs are often overconfident in their predicted class, with (G, x 1, and therefore taking into account all the outputs does not significantly affect the performance.
Effect of the wrapper function. In unsupervised settings, the individual dimensions of a latent vector do not nccessarily correspond to probability vectors. To address this, we replaced the weighted sum Equation B with a cosine similarity, which can be seen as a method of normalization that is invariant with respect to latent symmetries. We also experimentally evaluate this selection compared to the dot-product.
17

---

Published in Transactions on Machine Learning Research (10/2024)
CORA (GRACE)
grSubgraphX
-0.970
TAGE
Fidelitycs
0.980
-0.985
-0.990
-0.995
0.70
0.75
0.80
0.85
0.90
0.95
Sparsity
Figure A2: Quantitative performance comparisons between TAGE and grSubgraphX on Cora dataset, with preference for higher Fidelitycs in higher Sparsity levels.
0.4
Fidellyy
0.2
-0.2
Score Function
g:XAi
original
Salency
SubgraphXX
Method
Figure A3: Fidelity comparisons of our proposed grXAl-based methods with their original implementation on supervised task for BBBP dataset.
Our results show that the wrapper function has little effect on the performance of the grSubgraphX. some datasets, cosine similarity yields slightly better average results, as seen in Figure A4 This might be due the fact that the latent vector typically corresponds to the activation functions of the neurons in the GNN (e.g. ReLU sigmoid), which reduce the weight of inactivated components the score function (Crabóé & van der Schaar 2022).
Stability study. As stated in the main paper, there have been arguments that post-hoe (graph) explainability methods may lack stability (llack et al. 2021 Adebayo et al. 2018). as even slight changes to an instance can lead to significantly different explanations. In addition to Figure (right) in the main paper, our results Figure A5 show that grSubgraphX is able to provide very stable explanations for molecules with minor structural differences, while TAGE generates completely different explanations.
Augmentation design
In self-supervised settings, understanding the augmentation strategies can inform the design novel learning techniques that are not solely basod on empirical approaches. For instance, the GraphCI (You et al. 2020) framework introduced various types of graph augmentations, each of which incorporates specific prior knowledge about graph data. Results such as those shown in Figure A6 and Table AT demonstrate that
18

---

Published in Transactions on Machine Learning Research (10/2024)
NC11 (GraphCL)
grSubgraphx (det product)
-0.2
grSubgraphx similarity)
-0.3
Fidelitycs
0.5
-0.6
-0.8
0.9
0.5
Sparsity
Figure A4: Comparison of wrapper function performance on the NC11 dataset for grSubgraphX explainability.
grSubgraphx
Molecule Set |
TAGE
grSubgraphx
Molecule Set II
TAGE
Figure A5: The explainability of graph representations for two sets of molecules with minor structural changes from the MUTAG dataset. The grSubgraphX generates highly stable explanations, while TAGE is not stable, i.c. small changes to the sample produce substantially different explanations.
different angmentations used in GraphCT. training correspond to different explanation subgraphs, and have different Fidelity, meaning that perturbing them will affect the representation learning differently. This insight can aid in understanding the effectiveness of sule-sipe learning on graph data and provide a basis for comparing different pre-training methods beyond their predicted performance.
E
Computational complexity
We further compare the computational complexity of our grXAl-based methods to their traditional coun- terparts on various supervisod node and graph classification datascts. As seen Table A2 our proposed methods have close computational complexity to their traditional counterparts and are significantly faster than averaging all output components. We should also point out that providing connected subgraph
19

---

Published in Transactions on Machine Learning Research (10/2024)
Table A1: Comparison of the Fidelitycs on the explainability of GraphCL using different data augmentations
MUTAG datasct.
Augmentation
grSubgraphX (1)
grIG (t)
grSaliency (1)
dropN
-0.55t 0.83
-0.31 0.14
-0.69 0.13
maskN
-0.42 0.90
-0.201 0.19
-0.46: 0.21
permE
-0.524 0.85
-0.214 0.13
-0.701 0.16
subgraph
-0.50 0.86
-0.16 0.09
-0.621 0.15
random2
-0.61 0.15
-0.22+ 0.18
-0.65 0.14
dropN
maskN
permE
subgraph
random2
Molecule
Molecule II
Figure A6: The explanation for graph representations of two different MUTAG molecules learned by GraphCL. Each column corresponds to a different augmentation technique to train GraphCL. Using different augmentation pushes the model to learn the representation based on different substructures of input molecules, and some of them might be more aligned with prior knowledge.
to explain GNN encoders using grSubgraphX comes at the cost of higher computational complexity. In cases where compulational complexity is bottleneck, suggest. applying gradient-based extensions, i.e. grIG and grSaliency.
Table A2: Comparison of the performance of our proposed methods to their traditional counterparts in terms of average run time (see) for exampling one input graph.
Explainer
BBBP
Twitter
BA-Shar BA-Shape
IG
0.25 0.00
0.262 +: 0.00
IGavg
0.51 0.01
0.786 + 0.03
grIG
0.27 = 0.00
0.264 + 0.00
Saliency
0.0052 + 0.00
0.0052 + 0.00
Saliencyavg
0,0103 = 0.00
0.0156 + 0.00
grSaliency
0.0057 1 0.00
0.0052 1 0.00
SubgraphX
82.41 + 173.64
80.50+ 21.09
0.26 + 0.17
SubgraphXavg
164.82 348.28
241.5# 65.39
1.05 0.68
grSubgraphX
106.32 + 224.09
84.53 + 23.76
0.41 + 0.24
F Experimental settings
We used PyTorch (Paszke al. 2019) Nvidia A100 with 80 GB of memory. For supervised experiments, we used GNN checkpoints from DIG (Liu) et al.
develop our gAXAle framework and conducted experiments on an
2021 as part of Yuan et al. 2020 and did not train any models. We also utilized the DIG package's
20

---

Published in Transactions on Machine Learning Research (10/2024)
implementation of GraphCL, InfoGraph, and GRACE for pre-training GNNs in self-supervised settings Liu et al. 2021). specifically using GIN (Xu et al. 2019) for GraphCT. and InfoGraph, and GCN for GRACE. We followed the hyperparameters outlined in the main papers for these methods, using 3-layer GINs with an embedding dimension of 32 for GraphCL, 4-layer GINs with an embedding dimension of 512 for InfoGraph, and 2-layer GCNs with a node embedding dimension of 128 for GRACE. For grSubgraphX, we modified the original implementation of SubgraphX (Yuan al. (021) and followed the same hyperparameters of the original paper. We have implemented grIG and grSaliency based on their original implementation in the Captum Kokhlikyan et al. 2020 package.
G grSubgraphX algorithm
Algorithm
1 shows how calculate the Shapley score in grXAT framework.
Algorithm The algorithm of grSubgraphX to calculate Shapley score.
Input: L-layer GNN model f(-), input graph G with nodes V = ( uN subgraph G(+) =
with nodes y(+) {u ..... ) Monte-Carlo sampling steps T.
Initialization: Obtain the L-hop neighboring nodes of G(+), denoted as V(L.) (((1 ...., UN, (1.) Then
the of players is P' = (G(+) off.) ....,"N,. "N,.
for - to T do Sampling coalition set R2 from P G(+) Set nodes from V V (Re U G(+)) with zero features and feed to the GNNs f(-) to obtain (R U G(+))
E, (((G))) [[ M(LUGNNN
Σ, L(GI(())
Set nodes from V\ Re with zero features and feed to the GNNs f(() to obtain (R..) = TAGHTR
Then Pit = o UG()) - (Rt).
end for
Return:
= E Ppc
21


Reviews:
[{"review_id": "83OEStIGEo", "reviewer": "Reviewer_XfQg", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: The methods seems to be simple and builds on prior work in a straightforward way. Experimental comparisons show good performance.\n\nWeaknesses: Many parts of the methodology are unclear and explanations are incomplete. Figures and tables are hard to read. Overall presentation requires more work.\n\nOverall, the paper seems to be a good fit for TMLR and can be accepted after major revision needed to make it understandable.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In their paper \"Score-based explainability for graph representations\", the authors develop an 'explainability' method (grXAI) for GNN-based graph representation learning. This means that given a trained GNN and given an input graph, grXAI identifies a small subgraph that is important for the output representation, in the sense that if the subgraph is removed from the graph, then the output would change a lot. The authors compare their method to a single existing competitor and show that grXAI performs much better. \n\nDisclaimer: I know little about explainability research in general, and nothing about explainability methods for graph learning. Therefore I unfortunately cannot judge on the novelty and on how this work relates to the existing literature.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "MAJOR COMMENTS\n\n* For me it was difficult to understand sections 3 and 4. Some parts are explained several times with many repetitions. Some parts are not explained at all. Some things that should be explained in the prior work section (section 3) only appear in the methods section (section 4). Please re-think and re-organize sections 3--4 to make them understandable for somebody not very well familiar with the literature.\n\n  * Section 3.1: \"Shapley value\" -- define, give the formula, explain\n  \n  * Section 3.1: \"SubgraphX employs MCTS\" -- how? explain what it means and how the search is performed\n  \n  * Section 3.2: \"from a given baseline\" -- what does \"baseline\" mean here?\n  \n  * Section 3.2: G* here denotes a subgraph but earlier in Section 3.1 G* meant something else.\n  \n  * Section 3.2: What is \\Nu_*? It's not defined.\n  \n  * Section 4: \"we first review the typical setup\" -- it would be helpful if all review is contained in Section 3\n  \n  * Section 4.1: the entire first paragraph -- shouldn't it appear earlier? You defined Score() here, but you used Score() notation already in Section 3.2.\n  \n  * Eq (2): \"naive\" should be typed in \\mathrm{}\n  \n  * Section 4.1: \"searching for subgraphs can be computationally expensive\" -- very unclear. What searching are you talking about? So far the set of subgraphs G^{(i)} are simply given (that's how I understood Section 3.1), so there is no \"search\".\n  \n  * Section 4.1: \"averaging multiple connected subgraphs ... may not always be a connected subgraph\" -- very unclear. What does \"averaging subgraphs\" even mean? Equation (1) defines a score for each subgraph. If each subgraph G^{i} is connected, then how would a disconnected subgraph appear?? Super confusing.\n  \n  * Section 4.2: first paragraph feels like a repetition\n  \n  * End of page 6: the SubgraphX procedure should have been explained in Section 3!\n  \n  * End of page 6: \"four fundamental axioms\" -- very unclear! What axioms, what do they mean, where did they come from?\n  \n\n* Section 5.2: The methods grSubgraphX, grIG, grSaliency were not fully defined. For grSubgraphX there was at least some explanation in Section 4, even though how exactly the search happens was unclear to me. How grIG and grSaliency work, was not explained.\n\n* Section 5.2: The whole experimental setup is unclear. How do you generate results at different values of sparsity? Is sparsity a hyperparameter of your methods? What do the methods even output? Some score value for each subgraph? For each subgraph in what set? For _some_ set of subgraphs? How do you obtain subgraphs for a given sparsity level?\n\n* Figure 4: what do we see in Figure 4, what graph layout is this? Is it GRACE with 2D network output??\n\n* The only comparison in Figure 1 is TAGE. How does TAGE work? I think it should be properly explained in Section 3. \n\n\nMINOR COMMENTS\n\n* Table 1: very small font, why not make it larger? Also add references (citations) to each row.\n\n* Section 5, first paragraph: \"due to space limitations\" -- there are no space limitations in TMLR.\n\n* Figure 2: font sizes are way too small, the figure is impossible to read in the print-out.\n\n* Table 2: font can be larger.", "broader_impact_concerns": "n/a", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "MwZvGV20mU", "reviewer": "Reviewer_7f3w", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n1. Novel Framework for Unsupervised GNNs.\n\nThe idea of grXAI is quite novel in the field of graph explainability, specifically tailored for unsupervised GNNs. Existing methods are primarily designed for supervised settings, leaving a gap in explainability for unsupervised models. grXAI fills this gap by providing a robust method to interpret d-dimensional representations.\n\n2. Human-Intelligible Explanations.\n\nBy focusing on connected subgraphs, grXAI provides explanations that are more intuitive and understandable to humans. Human-friendly explanations are crucial for practical applications, as they help users comprehend the model's behavior and make informed decisions based on its outputs.\n\n3. Compatibility with Existing Methods.\n\nThe framework can be easily integrated with current score-based graph explainers, enhancing their applicability to unsupervised settings without significant modifications. This compatibility ensures that the framework can be widely adopted and used with minimal additional effort from researchers and practitioners.\n\n**Weaknesses**\n\n1. Computational Complexity.\n\nDespite improvements, the method still involves significant computational resources, particularly for complex models and large datasets. High computational demands can limit the framework's accessibility and scalability, posing challenges for researchers with limited resources.\n\n2. Specificity to Predefined Node and Edge Types.\n\nThe framework requires predefined node and edge types, restricting its flexibility and generalizability to new types of graphs without retraining. This limitation reduces the model's adaptability to different domains or novel graph types, hindering its broader applicability.\n\n3. Lack of Real-world Case Study.\n\nThe paper lacks a detailed case study to showcase the practical effectiveness and real-world impact of the proposed method. A case study would provide concrete evidence of the method's utility in solving real-world problems, enhancing the paper's impact and credibility.\n\n4. Focus on Synthetic Data.\n\nThe reliance on synthetic and controlled datasets may not fully capture the complexity and variability of real-world misuse scenarios, potentially limiting the generalizability of the findings. Real-world data often present unique challenges that synthetic data cannot replicate, making it essential to validate findings in more practical settings.\n\n5. Small Dataset Sizes.\n\nMost of the datasets used in the experiments are relatively small, which may not adequately represent the performance of the method on larger and more complex datasets. Small datasets can limit the ability to generalize the findings, and the method's effectiveness on larger datasets remains uncertain without further testing.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper studies the problem of explaining unsupervised Graph Neural Networks (GNNs), which produce d-dimensional representation vectors without clear semantic meaning. The authors propose a novel framework called grXAI, which generalizes existing score-based graph explainers to identify the subgraph most responsible for constructing the latent representation of the input graph. This framework can be implemented as a wrapper around existing methods, making the explanations more human-intelligible by focusing on connected subgraphs. Extensive qualitative and quantitative experiments demonstrate grXAI’s ability to effectively explain learned graph representations across various unsupervised tasks and learning algorithms.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See the weaknesses above.", "broader_impact_concerns": "No concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "2WJmvP8YUb", "reviewer": "Reviewer_fYbL", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1.\tThe method extends score-based explanations to unsupervised settings, where the output of the model is a d-dimensional vector without clear semantic meanings on each dimension.\n\n2.\tThe method produces connected subgraph, which is better for human interpretation.\n\n3.\tThe presentation is clear and easy to follow.\n\n\nWeaknesses:\n\n1.\tIn Equation (5), using cosine similarity as a measure of similarity between two embedding vectors can be problematic. A high cosine similarity can imply the two embedding vectors are close to each other, but a low (especially negative) cosine similarity does not necessarily mean the two embedding vectors are dissimilar. For example, if the two embedding vectors is only different in their signs (i.e., they are anti-parallel), the cosine similarity is -1, which is the lowest value one can achieve with cosine similarity. However, there is no reason to assert that the two embedding vectors are dissimilar, as one only need to change the sign of weights in subsequent layers to yield the same output.\n\n2.\tIn the paragraph following Equation (6), how can one ensure that $R \\cup G^{(s)}$ is always a connected graph. As shown in Equation (6), $R$ is any set of nodes in $P\\setminus G^{(s)}$, so it is likely that $R \\cup G^{(s)}$ forms a disconnected graph.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a post-hoc explanation method to find the most important connected input subgraph to the hidden embedding vector. The method mainly extends SubgraphX to unsupervised settings and show good results in quantitative and qualitative evaluations.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1.\tIt is better to explain why the function $Shapley(\\cdot, \\cdot, \\cdot)$ in Equation (1) has three inputs, and how these inputs are related to the game-theoretic definition of Shapley values (e.g., which input corresponds to the value function, and which input corresponds to the set the players, etc.).\n\n2.\tCould you explain why a lower $Fidelity_{CS}$ score is considered better? First, it sounds quite counter-intuitive to claim that lower fidelity is better. Second, according to the definition of $Fidelity_{CS}$, it is the “difference between the cosine similarity of the original embedding (with the original embedding itself) and (the cosine similarity of) the new embedding after masking out the important subgraph (with the original embedding)”. Therefore, a lower $Fidelity_{CS}$ score implies that the identified subgraph is not important to the embedding vector, which means an undesirable explanation result. This conflicts with the claim in the paper.\n\n3.\tIn addition to the fidelity $Fidelity(G) = f_c(G) – f_c(G^o)$, could you also run experiments on the *infidelity*, which is analogously defined as $Infidelity(G) = f_c(G) – f_c(G^*)$? In the above definitions, a higher fidelity does not imply a lower infidelity, since the function $f_c$ is not linear. The fidelity metric measures how much the prediction confidence drops when “important” subgraphs are masked, while the infidelity defined above directly measures how much the “important” subgraph contributes to the prediction confidence. In this sense, the latter should not be ignored.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper extended prior work subGraphX into explaining unsupervised graph neural networks. Reviewers generally find the paper provides interesting insights and is solving a new problem, albeit the methodological contribution is minor. After discussion with the reviewers, new experiments are added which improved the comprehensiveness of the paper. 2 reviewers are leaning accept and 1 reviewer is leaning reject, but there is no substantial issues that are not resolved in the final version of the paper other than the minor methodological contribution. Given the TMLR review criteria, the editor decides to accept the paper as it provides new insights to the task of explaining unsupervised graph neural networks.
Decision: Accept (poster)