Task-Agnostic Graph Explanations
Authors: Yaochen Xie
Sumeet Katariya
Xianfeng Tang
Edward W Huang
Nikhil Rao
Karthik Subbian
Shuiwang Ji
Venue: NeurIPS 2022 Accept (2022)
Keywords: Graph neural networks
explainability
task-agnostic

Abstract: Graph Neural Networks (GNNs) have emerged as powerful tools to encode graph-structured data. Due to their broad applications, there is an increasing need to develop tools to explain how GNNs make decisions given graph-structured data. Existing learning-based GNN explanation approaches are task-specific in training and hence suffer from crucial drawbacks. Specifically, they are incapable of producing explanations for a multitask prediction model with a single explainer. They are also unable to provide explanations in cases where the GNN is trained in a self-supervised manner, and the resulting representations are used in future downstream tasks. To address these limitations, we propose a Task-Agnostic GNN Explainer (TAGE) that is independent of downstream models and trained under self-supervision with no knowledge of downstream tasks. TAGE enables the explanation of GNN embedding models with unseen downstream tasks and allows efficient explanation of multitask models. Our extensive experiments show that TAGE can significantly speed up the explanation efficiency by using the same model to explain predictions for multiple downstream tasks while achieving explanation quality as good as or even better than current state-of-the-art GNN explanation approaches. 


gask-Agnostic Graph Explanations
Yaochen Xie Texas A&M University College Station, TX ethanycx@tamu,co edu
Sumeet Katariya Amazon Search Palo Alto, CA katsumee@ammzon..com com
Xianfeng Tang Amazon Search Palo Alto, CA tianto@mmzonnnco com
Edward Huang
Nikhil Rao
Karthik Subbian
Amazon Search
Amazon Search Palo Alto, CA
Amazon Search
Palo Alto, CA
Palo Alto, CA
evhuangdamazon.ccom
com
kaubbiandamazon.co
Shuiwang
Texas A&M University College Station, TX sj 1ritttmm... edu
Abstract
Graph Neural Networks (GNNs) have emerged as powerful tools to encode graph- structured data. Due to their broad applications, there is an increasing need develop tools to explain how GNNs make decisions given graph-structured data. Existing learning-based GNN explanation approaches are task-specific in training and hence suffer from crucial drawbacks. Specifically, they are incapable of producing explanations for a multitask prediction model with single explainer. They are also unable to provide explanations in cases where the GNN is trained in self-supervised manner, and the resulting representations are used in future downstream tasks. To address these limitations, we propose a Task-Agnosse GNN Explainer (Tage) that independent of downstream models and trained under self- supervision with no knowledge of downstream tasks. TAGE enables the explanation of GNN embedding models with unseen downstream tasks and allows efficient explanation of multitask models. Our extensive experiments show that TAGE can significantly speed up the explanation efficiency by using the same model to explain predictions for multiple downstream tasks while achieving explanation quality as good as or even better than current state-of-the-a approaches. Our code is publicly available as part of the DIG library
GNN explanation
Introduction
have achieved remarkable success in learning from real- world graph-structured data due to their unique ability to capture both feature-wise and topological information. Extending their success, GNNs widely applied in various research fields and
Graph neural networks (GNNs) 1|1 25 32
industrial applications including quantum chemistry B
26 drug discovery [16] 28 29), large-scale
social networks 60 (51, and recommender systems |
3
While multiple approaches have been
proposed and studied to improve GNN performance, GNN explainability is an emerging area and
has smaller body of research behind it. Recently, explainability has gained more attention due
This work was performed during an internship at Amazon.com Services LLC.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

End-to and task-specific GNW explainers
Two-stage task-agnostic GNN explanations
apoo- expliiner
Tasiv-paccfic eeplaiier
Task speo flo explainer
Embedding Explainer
Downstream Explaier
Condition vector (dimansiion importance)
Important subgraph
GNN-MIP P
GNN
Millklskk model
Predictions
Enibedding Model
Nedelg embeddinn
Downsheaa model
Need train dirlerent explbiners explain multtask predictioon model. Unable to trais without downstiiaam
Trained indepenndently from downstteam (asks
explsin downsdraam
Figure I: A comparison between typical end-to-end task-specific GNN explainers and the proposed task-agnostic explanation pipeline. To explain multitask model, typical explanation pipelines need to optimize multiple explainers, whereas the two-stage explanation pipeline only learns one embedding explainer that can cooperate with multiple lightweight downstream explainers.
an increasing desire for GNNs with more security, faimess, and reliability. Being able to provide good explanation to a GNN prediction increases model reliability and reduces the risk of incorrect predictions, which is crucial in fields such as molecular biology, chemistry, fraud detection, etc.
Existing methods adapting the explanation methods for convolutional neural networks (CNNs) or specifically designed for GNNs have shown promising explanations multiple types of graph data. A recent survey [37] categorizes existing explanation methods into gradient- based, perturbation, decomposition, and surrogate methods. In particular, perturbation methods involve learning or 36 and, while bearing higher computational costs, generally achieve oate--ff-theaat performance in terms of explanation quality. These methods train post-hoc explanation models on top of the prediction model to be explained. Earlier approaches like GNNExplainer [6]] require training or optimizing an individual explainer for each data instance, i.e., graph node to be explained. In contrast, PGExplainer [15] performs inductive learning, i.e., it only requires a one-time training, and the explainer can be generalized explain all data instances without individual optimization. Compared to other optimization-base improves efficiency terms of time cost without performance loss by learning. Following a similar inductive learning paradigm, more recent work ReFine (27) and GSAT [17] aim to provide multi- grained explanations and jointly learned explanations with GNNs, respectively.
optimization [12 13 15
explanation methods, PGExplainer significantly
However, even state-of-the-ar explanation methods like PGExplainer are still task-speeific at training and hence suffer from two crucial drawbacks. First, current methods are inefficient in explaining multitask prediction for graph-structured data. For example, one may need to predict multiple chemical properties in drug discovery for molecular graph. In particular, ToxCast from MoleculeNet has 167 prediction tasks. In these cases, it is common to apply a single GNN model with multiple output dimensions to make predictions for all tasks. However, one is unable to employ a single explainer to explain the above model, since current explainers are trained specifically to explain one prediction task. As a result, in the case of ToxCast, one must train 167 explainers to explain the GNN model. Second, in industry settings, it is common to train GNN models in a two-stage fashion due to scaling, latency, and label sparsity issues. The first stage trains massive amount of unlabeled data in an unsupervised manner to learn embeddings for nodes or graphs. The second stage trains lightweight models such as multilayer perceptrons (MLPs) using the frozen embeddings as input to predict the downstream tasks. In the first stage, the downstream tasks are usually unknown or undefined, and existing task-specific explainers cannot be applied. Also, there can be tens to hundreds of downstream tasks trained on these GNN embeddings, and training separate explainer for each task is undesirable and downright impossible.
embedding model with
To address the above limitations, we present a new task-agnostic explanation pipeline, where the learned explainer is independent from downstream tasks and can take downstream models input conditions, shown in Figure I Specifically, we decompose a prediction model into GNN embedding model and a downstream model, designing separate explainers for each component. We design the downstream explainers to cooperate with the embedding explainer. The embedding explainer is trained using self-supervised training framework, which we dub Task-Agnostic GNN Explainer (TAGE), with no knowledge of downstream tasks, models, or labels. In contrast to existing explainers, the learning objective for TAGE is computed at the graph or node embeddings without involving task-related predictions. In addition to climinating the need for downstream tasks in

---

Table 1: Comparisons properties of common GNN explainers. Inductivity and task-agnosticism are inapplicable for gradient/rule- based methods as they do not require learning. In the last column, show the number of required explainers for a dataset with N samples and M tasks.
Learning
Inductive
Task Task agnostic
# explainers required
Gradient- & Rule-based
No
GNNExplainer 34
Yes
No
No
M
SubgraphX [36
Yes
No
No
M * N
PGExxplainer 15
Yes
Yes
No
M
Task-agnostic explainers
Yes
Yes
Yes
TAGE, we argue that the self-supervision performed on the embeddings can bring an additional performance boost in terms of the explanation quality compared to existing task-specific baselines such as GNNExplainer and PGExplainer.
We summarize our contributions follows: 1) We introduce the task-agnostic explanation problem and propose a two-stage explanation pipeline involving an embedding explainer and a downstream explainer. This enables the explanation of multiple downstream tasks with single embedding explainer. 2) We propose a self-supervised training framework TAGE, which is based on conditioned contrastive learning to train the embedding explainer. The training of TAGE requires no knowledge of downstream tasks. 3) We perform experiments on real-world datasets and observe that TAGE outperforms existing learning-basse
explanation baselines in terms of explanation quality, universal explanation ability, and the time required for training and inference.
Relations with Prior Work Our work studies a novel explanation problem under the two-stage and multi-task settings. The settings are important in both industrial and academic scenarios but have not been studied by prior work. Whereas existing studies focus on designing optimization approaches 34 Ko and explainer architectures [15] under the typical task-specific setting, our work focuses on an orthogonal problem to enable task-agnostic explanations with the proposed framework including the universal embedding explainer and conditioned learning objectives.
2 Task Task-Agnostic Explanations
2.1 Notations and Learning-based GNN explanation
Our study considers the attributed graph G with node set V and edge set E. We formulate the attributed graph as a tuple of matrices X), where A E {0, V denotes the adjacency matrix and E xdy denotes the feature matrix with feature dimension of dy. We assume that the prediction model F that is to be explained operates on graph-structure data through two components: a GNN-based embedding model and lighter downstream models. Denoting the input space by G. a node-level embedding model & 9 - RIVixd takes graph as input and computes embeddings of dimension d for all nodes in the graph, whereas a graph-level embedding model E, 9 + R xd computes an embedding for the input graph. Subsequently, the downstream model D Rd > R computes predictions for the downstream task based on the embeddings.
Typical GNN explainers consider task-specific GNN- model as a complete unit, i.e., F= DoE. Given graph G and the GNN- based model F be explained, our goal is to identify the subgraph Goub that contributes the most to the final prediction made by F. In other words, we claim that given prediction is made because I' captures crucial information provided by some subgraph Grub- The learning- based (or optimization-basses
GNN explanation employs parametric explainer To associated with the GNN model F to compute the subgraph Grub of the given graph data. Concretely, the explainer To computes the importance score for each node or edge, denoted as w; or Waj. or masks for node attributes denoted as m. It then selects the subgraph Gsub induced by important nodes and edges, iee, whose scores exceed a threshold I, and by masking the unimportant attributes. In our study, we follow [5), focusing on the importance of edges to provide explanations to GNNs. Formally, we have Goub == (V, Esub) Ta(G),, where Esab {(v,, U2): (vi, vy) E, wij t}.
3

---

2.2 Task-Agnostic Explanations
As introduced in Section D all existing learning-based or optimization- bbaedd explanation approaches are task-specific and hence suffer from infeasibility or inefficiency in many real-application scenarios. In particular, they are of limited use when downstream tasks are unknown or undefined, and fail to employ single explainer to explain multitask prediction model.
To enable the explanation of GNNs in two-stage training and multitask scenarios, we introduce a new explanation paradigm called the task-agnostic explanation. The task-agnostic explanation considers whole prediction model as an embedding model followed by any number of downstream models. focuses on explaining the embedding model regardless of the number or the existence of downstream models. In particular, the task- aas--ggnstic explanation trains only one explainer T/(aa)) to explain the embedding model E, which should satisfy the following features. First, given an input graph G, the explainer (( should be able provide different explanations according to specific downstream tasks being studied. Table compares the properties of common GNN explanation methods and the
desired task-agnostic explainers in multitask scenarios. Second, the explainer T (tag) can be trained when only the embedding model is available, e.g., at the first stage of two-stage training paradigm, regardless of the presence of downstream tasks. When downstream tasks and models are unknown,
To (1tag) can still identify which components of the input graph are important for certain embedding dimensions of interest.
The TAGE Framework
Our explanation framework TAGE follows the typical scheme of GNN explanation introduced in the previous section. It provides explanations by identifying important edges in a given graph and removing the edges that lead to significant changes in the final prediction. Specifically, the goal of the TAGE is to predict the importance score for each edge in a given graph. Different from existing methods, the proposed TAGE breaks down typical end-to-end GNN explainers into two components. We now provide general descriptions and detailed formulations to the proposed framework.
3.1 Task-Agnostic Explanation Pipeline
Following the principle of the desired task-agnostic explanations, we introduce the task-agnostie explanation pipeline, where typical explanation procedure is performed two steps. In particular, we decompose the typical end-to-end learning- based GNN explainer into two parts: the embedding explainer To and the downstream explainer Taman. corresponding to the two components in the two-stage training and prediction procedure. We compare the typical explanation pipeline and the two-stage explanation pipeline in Figure I The embedding explainer and downstream explainers can be trained constructed independently from each other. In addition, the embedding explainer can cooperate with any downstream explainers to perform end-to-end explanations on input graphs.
The downstream explainer aims to explain task-specific downstream models. As downstream models are usually lightweight MLPs, we simply adopt gradient-basse explainers for downstream explainers without training. The downstream explainer takes a downstream model and the graph or node embedding vector as inputs and computes the importance score of each dimension on the embedding vector. The importance scores then serve as condition vector input to the embedding explainer. Given the condition vector, the embedding explainer explains the GNN-based embedding model by identifying an important subgraph from the input graph data. In other words, given different condition vectors associated with different downstream tasks or models, the embedding explainer can provide corresponding explanations for the same embedding model. Formally, we denote the x Rd + Rd, which maps input models and embeddings into importance scores m for all embedding dimensions. We denote the embedding explainer associated with the embedding model E by Te Rd x g + G. which maps a given graph into a subgraph of higher importance, conditioned on the embedding dimension importance E Rd
downstream explainer for models from S by Taown
The training procedures the embedding explainer are independent of downstream tasks or down- stream explainers. In particular, the downstream explainer is obtained from the downstream model only, and the training of the embedding explainer only requires the embedding model and the input graphs. As downstream models are usually constructed as stacked fully connected (FC) layers and the

---

P
((P)
Graph-lovsl
Thaining
G
Model
Masked embedding E(G)
Projection
GNN
Rit Concatenation
Edge imporiance
age
Embediina
Node loval Task
Explannrr
[.(p)
Ran Randomiy generated
Select pertatt Edges
cond vector y
E(Teip GJ)
R.j Storget.
E(TalP G))
Masked subgraph embecding
Entowdding for the targel node
Element-oiiee Multipiiatioo
Mutual Informatio Maximization
Figure 2: Overviews of the self-supervised training framework for the embedding model (right) and the architecture of the parametric explainers (left). During training. we generate random condition vectors an input to the embedding explainer and mask the embeddings. The learning objective seeks maximize the mutual information between two embeddings on certain dimensions.
explanation of FC layers has been well studied, our study mainly focuses on the non-trivial training procedure and design of the embedding explainer.
3.2 Training embedding explainer under Self-supervision
straightforward idea of explaining an embedding model with no knowledge of downstream tasks is to employ existing explainers and perform explanation the pretext task, such graph reconstruction [TC or context prediction [6. used during the pre- vre-training of GNNs. However, such explanations cannot be generalized to future downstream tasks there limited dependencies between the pretext task and downstream tasks. Therefore, training an embedding explainer without downstream models or labels is challenging, and it is desirable develop generalizable training approach for the embedding explainer. To this end, we propose a self-supervised learning framework for the embedding explainer.
The learning objective the proposed framework seeks to maximize restricted mutual information (MI) between embeddings. i.e., one of the given graph and of the corresponding subgraph of high importance induced by the explainer, in a conditioned subspace. We introduce a masking vector p E Rd as the condition to indicate specific dimensions of embeddings the MI. During the explanation, we obtain the masking vector from the importance vector computed by any downstream explainer Tdown- As downstream importance vector is available at training, we sample the masking vector p from a multivariate Laplace distribution due to the sparse gradient, i.., only few dimensions of high importance, assuming embeddings from well-trained models are informative with low dimension redundancy. Empirically, the Laplacian assumption holds on observe that gradients follow Laplace distribution, shown in
which to maximize
all datasets we work with Appendix Formally, the learning objective based on the restricted MI is
max Mx E(G), p ((Te(p,)))))
where MI...) computes the mutual information between two random vectors, p denotes the random masking vector sampled from certain distribution, To(p G) computes the subgraph of high impor- tance, and & denotes the element-wise multiplication, which applies masking to the embeddings E()). Figure 2 outlines the training framework and objective. Intuitively, given input graph and the desired embedding dimensions to be explained, the explainer To predicts the subgraph whose embedding shares the maximum mutual information with the original embedding on the desired dimensions.
Practically, the mutual information is intractable and is hence hard to directly compute. A common approach to achieving efficient computation and optimization is to adopt the upper bound estimations of mutual information B1], namely, the -Saanoon Estimator (JSE) [18] and the InfoNCE [ These upper bound estimations are also referred to as contrastive loss and are widely applied in self-supervised representation learning
5
2
for both images and graphs. Adopting these
5

---

estimators, the objectives are efficiently computed as
N log
min
1
210)]] + N -2-NN
Σ it1
log 1 - ((p 2,,0)]|,
N
1
N
expl 8 2p) 2.,0}
mmn
log Σj expl 2T) {n0)]
(3)
for JSE and InfoNCE, respectively, where N denotes the number of samples in mini- denotes Sigmoid function, 2; and 21,0 are embeddings of the original graph G, and its subgraph To(G;) or target nodes of the two graphs. Our objective involves condition vectors masks on the embeddings, which differs from typical contrastive loss used in self-supervised representation learning. We hence call the proposed objective the conditioned contrastive loss.
To restrict the size of subgraphs given by the explainer, we follow previous studies 15 add a size regularization term R, computed as the averaged importance score, to the above objectives. In the case where edge importance scores Wij E [0, 1] are computed, the regularization term is computed as
R(G)= =
Σ (o:
X /mall
Ae [usg log whj (1 miy) log(1 wij)]
where A and Ae are hyper-parameters controlling the size and the entropy of edge importance scores, respectively. We provide detailed descriptions of the objectives in Appendix B
3.3 Explainer Architectures
Embedding explainers. To be consistent with PGExplainer, we adopt the multilayer perceptron (MLP) the base architecture to predict the importance score Way for each edge (u,, Uz) E F,, on top of learned embeddings z; and zj of the two nodes connected by the edge. Edges with scores higher than threshold considered important edges that remain in the selected subgraph. In order for the embedding explainer to cooperate with different downstream explainers and provide diverse explanations for different tasks, it additionally requires a condition vector as input indicating the specific downstream task to be explained. Formally, the graph-level embedding explainer takes the embeddings, 2, and zj. and the condition vector p as inputs and computes the importance score by
Wi MLP ([= 2,] @ (ff())))
(S)
where [s denotes the concatenation along the feature dimension, @ denotes the element-wise multiplication, denotes the activation function, and fg Rd + Rd is linear projection. The node-level embedding explainer takes an additional node embedding as its input, as the explainers expected to predict different scores for the same edge when explaining different target nodes. The formulation of computing the importance score is as follows,
Wij MLP,, (I Zj; Starget) (fx()))).
where fg Rdd + 3d a linear projection, and Starger denotes the embedding of the target node whose prediction is to be explained.
Downstream explainers. We use the following gradient-based explainer compute condition vectors for different downstream models. Formally, given an input embedding z and its prediction probabilities D(=) E [0, 1J among all C' classes, we compute the gradient the maximal probability w.r.t. the input embedding:
a maxesc D(2)(c) E Rixd
8z
where [c] denotes the probability for class To convert the gradient into the condition vector, we further perform normalization and only take positive values reflecting only positive influence to the predicted class probability, i.e., p ReLU(norm(g")

---

BACE
SIDER
HIV
0.7
0.5
0.6
Fidelty
0.4
0.4
0.5
PGExplamne
TAGExp
0.3
0.3
0.3
DeepLIFT
GNNExplaine
0.2
GradCam
0.1
0.1
0.1
0.95
0.85
0.8
0.75
PPI (Task
PPI (Task 1)
PPI (Task
0.5
0.8
0.4
0.5
TAGExpaine
Fidelty
0.3
PGExpliinrr
0.3
0.4
GNNExplsiner
0,2
0.2
DeepLIFT
OrdICAM
0.1
0.2
0.1
0.7
0.6
0.95
0.75
0.65
0.75
Sparsity
Sparsity
Sparsity
Figure 3: Quantitative performance comparisons on six tasks from MoleculeNet (top row) and PPI
(bottom row). The curves are obtained by varying the threshold for selecting important edges.
Experimental Studies
We conduct two groups of quantitative studies evaluating the explanation quality and the universal explanation ability, i.e., training single explainer to explain all downstream tasks, of TAGE. We then compare the efficiency of multiple learning-based GNN explainers in terms of training and explanation time costs. We further provide visualizations to demonstrate the explanation quality as well as the ability to explain GNN models without downstream tasks.
4.1 Datasets
To demonstrate the effectiveness of the proposed TAGE on both node-level and graph-level tasks, we evaluate TAGE three groups of real-world datasets that contain potentially multiple tasks. The datasets are described as follows and their statistics are summarized in Appendix A
MoleculeNet. The MoleculeNet 29 library provides collection of molecular graph datasets for the prediction of different molecule properties. In a molecular graph, each atom in the molecule is considered node, and each bond is considered an edge. The prediction of molecule properties is a graph-level task. We include three graph classification tasks from MoleculeNet to evaluate the explanation of graph-level tasks: HIV, SIDER, and BACE.
Protein-Protein Interaction. The Protein Protein-Protein Interaction (PPI) 39 dataset documents the physical interactions between proteins in 24 different human tissues. In PPI graphs, each protein is considered as a node with motif and immunological features, and there is an edge between two proteins if they interact with each other. Each node in the graphs has 121 binary labels associated with different protein functions. As different protein functions are not exclusive the prediction of each protein function is considered an individual task instead of a multi-class classification. And hence typical approaches require individual explainers for the 121 tasks. We utilize the first five out of 121 tasks to evaluate the explanation of node-level tasks.
each other,
E-commerce Product Network. The E-commerce Product Network (EPN constructed with subsampled, anonymized logs from an e-commerce store, where entities including buyers, products, merchants, and reviews are considered as nodes, and interactions between entities are considered as edges. We subsample the data for the sake of experimental evaluations and the dataset characteristics do not mirror actual production traffic. We study the explanation of the classification of fraudulent entities (nodes), where the predictions for different types of entities are considered individual tasks. We evaluate our framework specifically on classifications of the buyer, merchant, and review nodes.
PProprietary dataset
7

---

4.2 Experiment Seftings and Evaluation Metrics
For each real-world dataset, we evaluate explainers on multiple downstream tasks that share single embedding model. For consistency with industrial use cases, we perform the two-stage training paradigm to obtain GNN models to be explained. In particular, we first use unlabeled graphs train the GNN-based embedding model in an unsupervised fashion. We then freeze the embedding model and use the learned embeddings to train individual downstream models structured as 2-layer MI.Ps. Specifically, for graph-level classification tasks in MoleculeNet, we employ the GNN pretraining strategy context prediction C to train a 5-layer GIN
[22 as the embedding model on ZINC-2M 22 containing 2 million unlabeled molecules. For the node-level classification on PPI, we employ the self-supervised training method GRACE 38 to train a 2-layer GCN | on all 21 graphs from PPI without using labels. For the larger-scale node-level classification on EPN, we use graph autoencoder (GAE) [TO to train the embedding model on sampled subgraphs of EPN. More implementation details are provided in Appendix B
As the involved real-world datasets do not have ground truth for explanations, we follow previous studies (20 37 36
to adopt a fidelity score and a sparsity score to quantitatively evaluate the explanations. Intuitively, the fidelity score measures the level of change in the probability of the predicted class when removing important nodes or edges, whereas the sparsity score measures the relative amount of important nodes or nodes associated with important edges. A formulation of the scores is provided in Appendix B. Note that compared to explanation evaluation with ground truths, the fidelity score is considered more faithful to the model, especially when the model makes incorrect predictions, in which case the explanation ground truths become inconsistent with the evidence of making the wrong predictions. In practice, one needs to trade-off between the fidelity score and the sparsity score by selecting the proper threshold for the importance.
Table 2: Fidelity scores with controlled sparsity on graph-level molecule property prediction tasks. Each column corresponds to explainer model trained on (or without) specific downstream task. Underlines highlight the best explanation quality in terms of fidelity, on the same level of sparsity.
PGExplainer (trained on) HIV
TAGE
Eval on BACE
BACE
BBBP
SIDER
w/o downstream
0.252 20.340
0.007 =0.251
0.026 #0.022
-0.151 #0.330
0.378 #0.293
HIV
-0.001 197
0.473 +0.404
0.013 =0.029
-0.060 =0.356
0.595 #0.321
BBBP
0.001 +0.237
-0.056 20.226
0.182 10.169
-0,252 +0.440
0.193 10.161
SIDER
0.012 10.219
-0.009 #0.212
0.003 20.029
0.444 #0.391
0.521 #0.278
4.3
Quantitative Studies
We conduct two groups of quantitatively experimental comparisons. We first demonstrate the explanation quality of individual tasks in terms of the fidelity score and the sparsity score. We do this by comparing TAGE with multiple baseline methods including non-learning-base based methods GradCAM [20 and DeepLIFT (2I) as well learning-baas basedd methods GNNExplainer [ and PGExplainer 115 We do not include other optimization or bearch-based methods such as Monte- Carlo tree search 9 due to the significant time cost on real-world datasets. Note that to show the effectiveness of universal explanations over different downstream tasks, we only train one embedding explainer for all tasks in a dataset, on top of which a gradient- based downstream explainer is applied to explain multiple downstream tasks. In contrast, for existing learning- based methods, we need to train multiple explainers to explain downstream tasks individually. For all methods, we vary the
Table 3: Fidelity scores with controlled sparsity on the node-level classification dataset PPI. Each column corresponds to an explainer model trained on (or without) specific downstream task. Underlines highlight the best explanation quality in terms of fidelity, on the same level of sparsity.
PGExplainer (trained on)
TAGE
Eval
Task 0
Task
Task 2
Task 3
Task 4
w/o downstream
Task
0.184 #0.3443
-0.005 #0.268
0.033 20.335
0.034
10.310
0.018 #0.194
0.271 +0.385
Task
0.046 #0.447
0.197
#0.380
0.043
10,314
0.008
10.297
0.021 10.183
0.300 #0.415
Task
0.028 +0.434
0.001 +0.283
0,345
+0.458
0.024 +0.320
0.097 +0.320
0.499 40.480
Task
0.075 +0.364
-0.015 +0.219
0.036
#0.317
0.262 20.418
0.040
#0.221
0.289 +0.427
Task
0.035 +0.413
-0.021 +0.238
0.223 +0.438
0.075 +0.374
0.242 #0.373
0.330 #0.442
8

---

Table 4: Fidelity scores with controlled sparsity on the E-commerce product dataset. Each column corresponds to one explainer model trained on different tasks or without downstream tasks. Underlines highlight the best explanation quality in terms of fidelity, on the same level of sparsity.
PGExplainer (trained on)
TAGE
Eval on
Buyers
Sellers
Reviews
w/o downstream
Buyers
0.2009 #0.2233
0.1731 #0.3774
0.1740 #0.4463
0.2713 =0.1834
Sellers
0,5465 #0.4773
0.3246 +0.4026
0.1128 #0.3019
0.6515 #0.3426
Reviews
0.4178 #0.3683
0. #0.3492
0.2310 20.4178
0.5692 #0.4214
Table 5: Comparison of computational time cost among three learning-based GNN explainers on the PPI dataset. The left two columns record time cost breakdown for downstream tasks. The fourth column estimates the total time cost for explaining all 121 tasks of PPI. The last row shows the speedup times compared to GNNExplainer and PGExplainer, respectively.
Time cost
Training (s)
Inference (8)
Total time (T=1) (s)
Est. total for 121 tasks
GNNExplainer
20040.1 1
20040.1
28
PGExplainer
7117.0%T
T
7604.2
10.7 d
TAGE
1405.3
582.7*T
1988.0
0.83 d
Speedup
1
-10.73x
10.1 338x
3 12.9
threshold for selecting important nodes or edges and compare how fidelity scores change over sparsity scores on each task and dataset. The results are shown in Figure 3 In particular, TAGE outperforms other learning-based explainers on BACE, SIDER, and PPI (tasks 0 and 1). For HIV and PPI (lask 2), TAGE more effective higher sparsity levels, i.e., when fewer nodes are considered important and masked.
To justify the necessity of task-agnostic explanation and demonstrate the universal explanation ability of TAGE, we include PGExplainer as our baseline and compare the explanation quality when adopting a single explainer explain multiple downstream tasks. For PGExplainer, we train multiple explainers on different downstream tasks and evaluate each explainer on different downstream tasks. For TAGE, train one explainer without downstream tasks and evaluate on different downstream tasks. Results shown in Table 2 (MoleculeNet), Table 3 (PPI), and Table 4 (EPN) indicate that task-specific explainers fail to generalize to different downstream tasks and hence are unable to provide universal explanations. On the other hand, the task-agnostic explainer, although trained without downstream tasks, can provide explanations with even higher quality for downstream tasks.
GNNExplainer and PGExplainer should generally outperform task-agnostic explainers, as they are specific to data examples tasks. This should especially be true when TAGE and PGExplainer have the same level of parameters. However, we find that TAGE outperforms the learning- based baselines. We believe that the underperformance of baselines is due to the non-injective characteristic of the downstream MLPs, where different embeddings can produce similar downstream prediction results. In other words, similar downstream prediction are not necessarily produced by embeddings that share high mutual information. Due to this characteristic, the learning objective of TAGE computed between embeddings brings stronger supervision than the objective computed between final predictions, as the latter objective does not guarantee consistency between embeddings or between input graphs and subgraphs.
Multitask Explanation Efficiency A major advantage of the task-agnostic explanation is that removes the need for training individual explainers, which consumes the majority of the total time cost explain model on dataset. We hence evaluate the efficiency of TAGE in terms of time cost for explanation and compare it to the two learning based explainer baselines. We record the time cost for the training and inference of different explanation methods on the same dataset and device, shown in Table 5 All results are obtained from running the explanation on the PPI dataset with 121 node classification tasks with a single Nvidia Tesla V100 GPU. Although the inference time cost of TAGE is slightly higher than that of PGExplainer, the results show TAGE costs significantly less time than GNNExplainer and PGExplainer, especially in the multitask cases (T > 1). TAGE allows the explanation of many downstream tasks within a reasonable time duration.

---

(With downstream mask) PGExplainer
downstream tack)
GNNExplainer
TAGE
TAGE-Single Dimension
TAGE-Singa Dimension
0.2883
-D -2.221
0.7011
(D.6287)
(0.6183)
9.69
-0.0145
0.5022
(0.0494)
(0.8153)
0.5647
0.4659
0.6183
(0.0193)
(0-3705)
Figure 4: Visualizations on explanations to the GNN model for the BACE task. The top 10% important edges are highlighted with red shadow. The numbers below molecules are fidelity scores when masking out the top 10% important edges. The right two columns are explanations for two certain embedding dimensions without downstream tasks. Fidelity scores in the right two columns explaining two embedding dimensions are still computed for the BACE task but are just for reference.
4.4 Visualization of Explanations
We visualize the explanations of the three learning-based explanation methods on the BACE task, which aims to predict the inhibitor effect of molecules human f8-scretase (BACE-1) (2)] Additional visualizations on HIV and SIDER are also provided in Appendix E The visualization results are shown in Figure 4 Each molecule visualization shows the top 10% important edges (bonds) predicted by an explainer marked in red, together with the fidelity score on the molecule. The left three columns are explanation results with the BACE downstream task. The right columns explanations by TAGE to two specific graph embedding dimensions, without downstream models. Embedding dimensions with greater values among all are selected in the visualizations. To obtain explanations of certain embedding dimensions, we input the one-hot vectors to the embedding explainer condition vectors. The visualization results indicate that while baseline methods select scattered edges as important, TAGE tends to select edges that form a connected substructure, which is more reasonable when explaining molecule property predictions where a certain functional group is important for the property. While there are no ground -ruth explanations for the BACE, the validity of results produced by TAGE can be evidenced by multiple domain researches 7 Σ. We discuss them in Appendix E In addition, the right three columns indicate that dimensions in the embedding correspond to different substructures and TAGE is able to provide explanations to the dimensions without downstream tasks.
5 Conclusions
Existing task-specific learning- based explainers become inapplicable under real scenarios when downstream tasks or models are unavailable and suffer from inefficiency when explaining real-world graph datasets with multiple downstream tasks. We introduced TAGE, including the task-agnostic GNN explanation pipeline and the self-supervised training framework to train the embedding explainer without knowing downstream tasks or models. Our experiments demonstrate that the TAGE generally achieves higher explanation quality in terms of fidelity and sparsity with a significantly reduced explanation time cost. We discuss potential limitations and their solutions in Appendix G
10

---

References
[1] Wenqi Fan, Yao Ma, Qing Li, Yuan He, Eric Zhao, Jiliang Tang, and Dawei Yin. Graph neural networks for social recommendation. In The World Wide Web Conference, pages 417-426, 2019.
[2] Matthias Fey and Jan E. Lenssen. Fast graph representation learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Gruphs and Manifolds, 2019.
[3] Justin Gilmer, Samuel S Schoenholz, Patrick Riley, Oriol Vinyals. and George E Dahl. Neural message passing for quantum chemistry. In Proceedings of the 34th International Conference on Machine Learning, pages 1263-1272. PMLR, 2017.
[4] Michael Gutmann and Aapo Hyvärinen. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pages 297-304, 2010.
[5] R Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildonn Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. Learning deep representations by mutual information estimation and maximization. In International Conference on Learning Representations, 2019.
[6] Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec. Strategies for pre-training graph neural networks. In International Conference on Learning Representations, 2020.
[7] Dandan Huang, Yonglan Liu, Bozhi Shi, Yueting Li, Guixue Wang, and Guizhao Liang. Comprehensive 3D-QSAR and binding mode of BACE-1 inhibitors using R-group search and molecular docking. Journal of Molecular Graphics and Modelling, 45:65-83, 2013.
[8] Priti Jain and Hemant R. Jadhav. Quantitative structure activity relationship analysis of aminoim- idazoles as bace-i inhibitors. Medicinal Chemistry Research,
2013.
[9] Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Multi-objective molecule generation using interpretable substructures. In Proceedings of the 37th International Conference on Machine Learning, pages 4859, PMLR, 2020.
[10] Thomas N Kipf and Max Welling. Variational graph auto-encoders. arXiv preprint 1oXiv:661.0073000 2016.
[11] Thomas N. Kipf and Max Welling. Semi-supervise classification with graph convolutional networks. In International Conference on Learning Representations, 2017.
[12] Wanyu Lin, Hao Lan, and Baochun Li. Generative causal explanations for graph neural networks. In Proceedings of the 38th International Conference on Machine Learning, 2021.
[13] Wanyu Lin, Hao Lan, Hao Wang, and Baochun Li. Orphiex: model for interpreting graph neural networks. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 13729-13738, 2022.
mousality-inspired latent variable
[14] Meng Liu, Youzhi Luo, Limei Wang, Yaochen Xie, Hao Yuan, Shurui Gui, Haiyang Yu, Zhao Xu, Jingtun Zhang, Yi Liu, Keqiang Yan, Haoran Liu, Cong Fu, Bora M Oztekin, Xuan Zhang, and Shuiwang Ji. DIG: A turnkey library for diving into graph deep learning research. Journal of Machine Learning Research, 22(240):1 2021.
[15] Dongsheng Luo, Wei Cheng, Dongkuan Xu, Wenchao Yu, Bo Zong, Haifeng Chen, and Xiang Zhang. Parameterized explainer for graph neural network. In Advances in Neural Information Processing Systems, pages 19620-19631, 2020.
[16] Youzhi Luo, Keqiang Yan, and Shuiwang Ji. GraphDF: A discrete flow model for molccular graph generation. In Proceedings of the 38th International Conference Machine Learning, pages 7192-7203. PMLR, 2021.
[17] Siqi Miao, Mia Liu, and Pan Li. Interpretable and generalizable graph learning via stochastic attention mechanism. In Proceedings of the 39th International Conference on Machine Learning, pages 15524-15543, PMLR, 2022.
[18] Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-GAN: Training generative neu- ral samplers using variational divergence minimization. Advances Neural Information Processing Systems, pages 271-279, 2016.

---

[19]
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary Vito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems, pages 8024-8035. 2019.
[20] Phillip E Pope, Soheil Kolouri, Mohammad Rostami, Charles E Martin, and Heiko Hoffmann. Explainabilityy methods for graph convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 10772-10781, 2019.
[21] Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. In Proceedings of the 34th International Conference on Machine Learning, pages 3145-3153, 2017.
[22] Teague Sterling and John J Irwin. ZINC 15 - Ligand discovery for everyone. Journal of Chemical Information and Modeling, 55(1 11:2324-23377 2015.
[23] Fan-Yur Yun Sun, Jordan Hoffman, Vikas Verma, and Jian Tang. Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization. In International Conference on Learning Representations, 2019.
[24] Petar Veličković, William Fedus, William L. Hamilton, Pietro Liò, Yoshua Bengio, and Devon Hjelm. Deep graph infomax. In International Conference on Learning Representations, 2019.
[25] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In International Conference on Learning Representations, 2018.
[26] Limei Wang, Yi Liu, Yuchao Lin, Haoran Liu, and Shuiwang Ji. ComENet: Towards complete and efficient message passing for 3D molecular graphs. Advances Neural Information Processing Systems, 2022.
[27] Xiang Wang, Yingxin Wu, An Zhang, Xiangnan He, and Tat-Seng Chua. Towards multi-grained explainability for graph neural networks. In Advances in Neural Information Processing Systems, pages 18446-18458, 2021.
[28] Zhengyang Wang, Meng Liu, Youzhi Luo, Zhao Xu, Yaochen Xie, Limei Wang. Lei Cai, Qi Qi, Zhuoning Yuan, Tianbao Yang, and Shuiwang Ji. Advanced graph and sequence neural networks for molecular property prediction and drug discovery. Bioinformatics, 38(9):2579-2586 2022.
[29] Zhenqin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Aneosh Pappu, Karl Leswing, and Vijay Pande. MoleculeNet: A benchmark for molecular machine learning. Chemical Science, 9():513-530, 2018
[30] Yaochen Xie, Zhao Xu, and Shuiwang Ji. Self-supervised representation learning via latent graph prediction. In Proceedings the 39th International Conference on Machine Learning. pages 24460 24477. PMLR, 2022.
[31] Yaochen Xie, Zhao Xu, Jingtun Zhang, Zhengyang Wang, and Shuiwang Ji. Self-supervised learning of graph neural networks: A unified review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
[32] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In International Conference on Learning Representations, 2019.
[33] Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William I. Hamilton, and Jure Leskovec. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 974-983, 2018
[34] Zhitao Ying, Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec. GNNEX- plainer: Generating explanations for graph neural networks. In Advances in Neural Information Processing Systems, pages 9244-9255, 2019.
[35] Haiyang Yu, Limei Wang, Bokun Wang, Meng Liu, Tianbao Yang, and Shuiwang Ji. GraphFM: Improving large-scale GNN training via feature momentum. In Proceedings of the 39th International Conference Machine Learning, pages 25684-25700 PMLR, 2022.
12

---

[36] Hao Yuan, Haiyang Yu, Jie Wang. Kang Li, and Shuiwang Ji. On explainability of graph neural networks via subgraph explorations. In International Conference on Machine Learning. pages 12241-12252. PMLR, 2021.
[37] Hao Yuan, Haiyang Yu, Shurui Gui, and Shuiwang Ji. Explainability in graph neural networks: A taxonomic survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
[38] Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. Deep graph contrastive representation learning. In ICML Workshop on Graph Representation Learning and Beyond, 2020.
[39] Marinka Zitnik and Jure Leskovec. Predicting multicellular function through multi-layer tissue networks. Bioinformatics, 33(14):1190-i198, 2017.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] In Appendix G.
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] In Appendix B.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [N/A]
(d) Did you include the total amount compute and the type of resources used (e.g.. type of GPUs, internal cluster, or cloud provider)? [Yes] In Section 4.3 Multitask Explanation Efficiency.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
(d) Did you discuss whether and how consent obtained from people whose data you'n 're using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content? [Yes] The EPN dataset anonymized.
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "oYuhaQKJgTe", "reviewer": "Reviewer_7PtK", "summary": "This paper proposed a framework to do graph based explanations. Compared to the existing explainers, the proposed method is able to explain the predictive outcome for any downstream tasks, without re-training the explainer for every new tasks. The core idea is to decompose the explanation into node embedding explanation with subgraph, and downstream classifier explanation with node embeddings. Experiments show that the proposed method is effective at explaining multiple objectives, while also being much faster than alternatives. \n", "questions": "Q1: since directly optimizing the MI might be difficult, can we just (naively) optimize RMSE or some other distance metrics? \n\nQ2: During training it seems the \\Tau_{\\theta} selects the subgraph and feed it into GNN and optimize MI end2end. However the edge selection requires a threshold, which is literally a step function that is non-differentiable. I wonder how this is handled in the paper? \n\nQ3: The subgraph selection is done via the selection of each edge independently. It seems a bit limited in the sense of expressiveness, as sometime maybe a subgraph jointly controls the certain dimension of the embedding. I’d like to learn the authors opinion on this.", "limitations": "Please check the questions in the above section.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Overall the paper is well motivated. The runtime required by existing explainers like GNNExplainer is indeed a bottleneck in applying the model. This paper proposed a simple and timely solution to the explainer community, and from the experiments it turns out to be efficient and effective. I think the contribution regarding this specific problem is valid. \n\nThe paper is overall well written and easy to follow. \n\nThere are some concerns regarding the current draft, mainly about some clarifications and the rationales for the design choices. Please see my questions below. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "nEtyKd2IpJ2", "reviewer": "Reviewer_WNap", "summary": "The paper proposes a novel graph explainer independent of specific downstream tasks. The proposed explainer consists of two parts: embedding explainer and downstream explainers. This task-agnostic attribute of the embedding explainer is implemented by a  self-supervised method via Mutual Information. The extensive quantitative experiments and visualizations demonstrate the merits of the proposed method.", "questions": "1. When it is better to use the proposed method rather than others?", "limitations": "The work has pointed out some limitations. I encourage the authors to add more about its application condition.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The proposed task-agnostic graph explanator has some novelty in this field. The task agnostic explanation may bring new insight from different views compared with task-specific tasks. This character might be very interesting for some directions like drug discovery via graph learning. \n2. The implementation of self-supervised training for the embedding model is interesting. It is also an interesting attempt to apply MI objective.\n3. The experiments show improved computational time cost compared to baselines.\n\nWeaknesses:\n1. The section on related works might need more words to help the readers understand your work. For example, it would be better to add some discussions for [1]. Moreover, the mask-based explanation models are a branch of graph explanation methods. The differences between the proposed and the existing ones may need to be clarified.\n[1]. Siqi Miao, Miaoyuan Liu, Pan Li; Interpretable and Generalizable Graph Learning via Stochastic Attention Mechanism; ICML 2022.\n2. The paper may lack some discussions about its limitations.  As line 293 said, the proposed method may have poorer performance compared with task-specific explainers. When it is better to use the proposed method rather than others?\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "Lze-YhahJ4G", "reviewer": "Reviewer_S3nw", "summary": "This work proposes a method to explain GNNs on node classification and graph classifications without knowing the downstream task in advance. It separates the GNN-based models on these tasks into two parts, i.e., a GNN embedding model and an MLP downstream model. The former takes the input graph and encodes it to node/graph embeddings. The latter takes the output from the former and makes predictions for the downstream tasks. Two explainers are developed regarding these two parts respectively, and the work focuses on the embedding explainer for the first part. The embedding model is pretrained unsupervised. The embedding explainer is trained to extract a subgraph that maximizes the mutual information between the whole graph embedding and the subgraph embedding, without knowing the downstream task. Overall, the proposed method can be more efficient when explaining multiple tasks at once.", "questions": "1. In the paper authors use some pre-training task to initialize GNN. In the experiments (Figure 3), are all the baselines run on the pre-trained model? Or some will fine-tune on downstream tasks? What is the prediction accuracy after fine-tuning? I think this is important as we want to make sure comparing the explanation ability with the same model.\n\n2. Could this approach work for other self-supervised pre-training tasks? Did the authors try any of them? As I think the approach should be a general framework to understand a pre-trained model.\n\n3. As mentioned in \"strength and weakness\", I suggest the authors explain more clearly why and when this task-agnostic model might work, and when it may fail. I doubt there should be a explanation that could be suitable for different tasks. Probably the authors could emphasize their contribution more on how to explain a pre-trained model instead of saying its generalized to different tasks. Probably with a few label and fine-tuning would be more reasonable for such as task-agnostic model.", "limitations": "The authors list the limitation on broad usage setting (cannot do inductive and black-box explanation). But I expect more discussion on whether such a task-agnostic explanation could really be useful for different task, when will it fails (e.g., some task-specific feature could not be captured during pre-training)", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. The proposed explainer runs faster than existing explainers.\n2. The paper is clearly written and easy to follow.\n\nWeaknesses:\n\nThe major weakness to me is the motivation of the work. In what cases will we truly need a task-agnostic explanation? If our goal is to understand why a model makes such a prediction, then how could we hide the prediction in advance when learning an explainer?\nWhat formula 1 says is essentially \"find the important subgraph that gives me a similar encoded embedding as if I encoded the whole graph\". Why would this be reasonable in practice when we are looking into each specific task? Don't we want to give explanations as specific as possible? Personally, I don't think we could assume there is a subgraph that is generally important for all tasks since these tasks are indeed different. (similar to No-Free-Lunch theorem). This is also part of the reason why we have the sparsity metric for evaluation to avoid being too general. Probably the authors could explain or discuss their motivation and where this method is useful more clearly.\n\nI also have some concerns about the setting of the model. Why set the pipeline into two separated parts? As far as I know, the GNN encoder is often used in a pretrain-finetune fashion which has higher accuracy than fixing the encoder for downstream tasks, like the context prediction pretraining on MoleculeNet datasets in your experiments. Why do we want to explain a less accurate model given a more accurate setting is easily available. I understand efficiency is a concern, but may be sacrificing too much accuracy to achieve a task-agnostic explanation isn't worth doing?\n\nThe discussion of related work is not very thorough. Some recent works are missing, e.g., [1] and [2]. Some works like SubgraphX and Gem are cited but not discussed specifically.\n\n[1] Wang, X., Wu, Y., Zhang, A., He, X., & Chua, T. S. (2021). Towards multi-grained explainability for graph neural networks. Advances in Neural Information Processing Systems, 34, 18446-18458.\n\n[2] Lin, W., Lan, H., Wang, H., & Li, B. (2022). Orphicx: A causality-inspired latent variable model for interpreting graph neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 13729-13738).\n\nMinor:\nNotation D mismatch in lines 90 and 145\n ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "G5oD2jgsAn", "reviewer": "Reviewer_61fV", "summary": "The paper sets out to achieve two main objectives - a Graph Neural Network explainer TAGE that is task agnostic, and hence second, can be used in multi-task prediction. The task agnostic claim is because TAGE uses gradient methods for the downstream tasks along with an explainer for GNN embedding which is task agnostic. The gradient methods produce a condition vector during inference. But during training a method using multivariate Laplace distribution is used for generating the condition vector. The overall objective is to maximize the mutual information between the embedding produced from the original graph and the embedding produced from the subgraph given by explainers.\n\nOverall, the approach sounds interesting and useful. But there might be some misgivings about the claims - whether the method is really task agnostic or whether the datasets chosen really have multi-task predictions.", "questions": "1. Was there any ablation done on using the embedding explainer along with PG Explainer for the down stream task? Is the use of the Laplace distribution a problem there?\n2. What other distributions can be used besides multivariate Laplace distribution?\n", "limitations": "The authors mention the limitation of all inductive approaches when the training and inference data are inconsistent.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. The paper makes a lot of practical sense given how GNN models are trained. Explaining at the GNN embedding level, and using it along with a task specific explainer. The use of gradient based methods for the latter seems reasonable. If the cost of explaining multiple tasks is amortized and efficient, this work is practically useful.\n\n2. The paper does a good job of setting out the objectives to the point that there is some redundancy in the earlier sections. Descriptions of the following are well written.\n\na. Maximizing mutual information (MI) between the embeddings as the objective,\nb. how to make this MI objective tractable (JSE, NCE),\nc. using condition vectors from downstream tasks during inference and from the Laplace distribution during training\nd. evaluation for quality and universal explanation ability\n\n3. The visualization and the observation of closeness of chosen edges in the explanation (for MoleculeNet) are very interesting.\n\nWeaknesses:\n\n1. The claim of the method being task agnostic seems a pit provocative though it's largely true.\n2. The definition of multi-tasks in the examples are debatable. Are there really 121 tasks in PPI? Given the limitations in evaluating the quality of the explanations (from an end user perspective), it's hard to judge if the explanations of TAGE are qualitatively better for each of the tasks.\n3. It would be better if the discussion was consistently on one of the datasets (MoleculeNet), while other datasets are described in the appendix.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept