Towards Multi-Grained Explainability for Graph Neural Networks
Authors: Xiang Wang
Yingxin Wu
An Zhang
Xiangnan He
Tat-Seng Chua
Venue: NeurIPS 2021 Poster (2021)
Keywords: Graph Neural Networks
Multi-grained Explainability
Feature Attribution

Abstract: When a graph neural network (GNN) made a prediction, one raises question about explainability: “Which fraction of the input graph is most inﬂuential to the model’s decision?” Producing an answer requires understanding the model’s inner workings in general and emphasizing the insights on the decision for the instance at hand. Nonetheless, most of current approaches focus only on one aspect: (1) local explainability, which explains each instance independently, thus hardly exhibits the class-wise patterns; and (2) global explainability, which systematizes the globally important patterns, but might be trivial in the local context. This dichotomy limits the ﬂexibility and effectiveness of explainers greatly. A performant paradigm towards multi-grained explainability is until-now lacking and thus a focus of our work. In this work, we exploit the pre-training and ﬁne-tuning idea to develop our explainer and generate multi-grained explanations. Speciﬁcally, the pre-training phase accounts for the contrastivity among different classes, so as to highlight the class-wise characteristics from a global view; afterwards, the ﬁne-tuning phase adapts the explanations in the local context. Experiments on both synthetic and real-world datasets show the superiority of our explainer, in terms of AUC on explaining graph classiﬁcation over the leading baselines. Our codes and datasets are available at https://github.com/Wuyxin/ReFine.

Towards Multi-Grained Explainability for Graph Neural Networks
Xiang Wanglit, Ying-Xin Wu', An Zhang', Xiangnan He Tat-Seng Chua \Sea-NExT Joint Lab
National University of Singapore
& University of Science and Technology of China
xiangwangêu. nus. edu, wuyxin@mail. ustc. edu. cn, an_ an_zhang@nus edu. sg xi sianganhe@ggail.. com, dcscts@nus. .edu.
Abstract
When a graph neural network (GNN) made prediction, one raises question about explainability: "Which fraction of the input graph is most influential to the model's decision?" Producing an answer requires understanding the model's inner workings in general and emphasizing the insights the decision for the instance at hand. Nonetheless, most of current approaches focus only on one aspect: (1) local explainability, which explains each instance independently, thus hardly exhibits the class-wise patterns; and (2) global explainability, which systematizes the globally important patterns, but might be trivial in the local context. This dichotomy limits the flexibility and effectiveness of explainers greatly. A performant paradigm towards multi-grained explainability is until-now lacking and thus focus of work. In this work, we exploit the pre-training and fine-tuning idea to develop our explainer and generate multi-grain gratiedg explanations. Specifically, the pre-training phase accounts for the contrastivity among different classes, so as to highlight the class-wise characteristics from global view; afterwards, the fine-tuning phase adapts the explanations in the local context. Experiments on both synthetic and real-world datasets show the superiority of our explainer, in terms of AUC on explaining graph classification over the leading baselines. Our codes and datasets are available at https: // //g/ thub. com/Wuyxin/ /ReFine..
Introduction
While graph neural networks (GNNs) [1, 2] have achieved great success in a variety of applications, they usually come as black-box models. The general problem about GNN explainability [3] is to answer "What knowledge does the model use to arrive at the conclusions in general and the specific decision at hand?". Thoroughly answering this question requires the global understanding of the model's inner workings and the local insights property prediction as an example. The global understanding exhibits the knowledge encoded in the model, such as the distribution of the chemical groups; meanwhile, the local insight identifies certain chemical groups responsible for a given molecule's 's property. Such multi multi-grained grained explainability flexibly and reliably inspects the decision-making process of the GNN [4, 5], which critical to applications safety, fairness, and privacy [6, 7].
specific instance. Take a GNN model for molecular
In the field of GNN explainability [8], explainer models broadly attribute model prediction the input graph, then sample salient subgraph as the explanation for the model prediction. However, most of current explainers focus on either on local [9, 10, 11, 12] or global explainability [13, 7].
thereby suffer from inherent limitations correspondingly:
Xiangnan He the corresponding author.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Prediction: Farm
Before fine-tuning After fine-tuning
1.0
0.99
0.55
0.41
0.21
(a)
[c]
Prediction: Stadium
Probability of being Farm
[b]
Pre-training towards Global Explainability
Fine-Tuning towards Local Explainability
Explanations on Visual Genome dataset generated from ReFine, including the pre- vre-training and tie-ttnnng phases. Right indicates the changes before and after the fine-tuning.
Figure I:
Local explainability aims to customize the explanatory subgraph for each instance individually. However, such local explanations fall short systematizing the prototypical patterns shared within a class or group of instances. Thus, they lack the global understanding of the model's workings [7. 13], which is vital to generalize to other instances being explained.
Global explainability targets at the globally important patterns across multiple instances, which could violate the local fidelity [14] [14]-th the globally important substructure may not be important even appear in the local context, thus might fail to explain a specific instance reliably.
Briefly put, these approaches overlook the that the local and global explainability should be exhibited simultaneously obtain faithful expla- nations. Taking Figure as an example, the global explainability differentiates the explanations for various classes, such as brrestock-backgronn
nature of explainability, while we argue
subgraphs for the farm class, human-sports subgraphs for the stadium class. When zooming in a specific scene graph, the local explainability refines on the farm-wise patterns and specifies (sheep, on, meadow) the final explanation. A paradigm towards such multi grained explainability is until-now lacking, to the best of our knowledge.
Towards multi-grainer explainability, propose novel explainer, ReFine, with pre-trat pre-training and fine-tuning [15, 16] techniques for explaining GNN models. Specifically, pre-training aims to answer "What class-wise knowledge does the GNN leverage make predictions in general?". We combine the contrastive learning [17, 18) into class-wise generative probabilistic models [7], thereby approach coarser-pe -aaaeee-graiedd explanations (i.e. saliency maps of all edges). Going beyond the global view, fine-tuning is to answer "Why the GNN model made the certain prediction for the instance at hand?", where upgrade the coarser-grained explanations the finer-grained explanations (i.e. explanatory subgraphs of salient edges). Through this way, ReFine can faithfully generate multi- grained explanations, and we empirically show its effectiveness as compared to some state-of-the-ar explainers [9, 6, 7, 19]. It is also worth mentioning that, although the general understanding of GNN predictions has been considered in a recent work PGExplainer [7], it is only exploited train generative probabilistic model shared across all the explained instances, rather than dissecting and modeling the class-wiss knowledge explicitly. Overall, our contributions are summarized as:
We investigate the local explainability and global explainability for explaining GNNs and put forward the concept of multi-grained explainability.
We propose pre-training and fine-tuning framework to generate multi-grained explanations, which has both global understanding of model workings and local insights on specific instances.
We achieve state-of-the-ard performance various datasets w.r.f. predictive accuracy on explaining GNNs. Quantitative and qualitative results verify multi-granularity explainability of ReFine.
2 Background & Task Formulation
In this section, we begin with the backgrounds on GNNs and frame the task of generating multi- grained explainability for GNN models.

---

Local Fidelity
Prediction: Cycle
Class-wise Attribution Module
Saliency
Subgraph
Selection Module
Explanatory Subgraph
Pre-training
Fine-tuning
Figure 2: Model construction of proposed ReFine. Left represents the pre-training phase for graph example, which is labeled and predicted as "Cycle", from the BA-3motif dataset. Right demonstrates the fine-tuning process where the saliency map is fine-tuned on the instance to achieve local fidelity.
Graph Neural Networks. We denote the graph data G (V,E) with the node set V and the edge set &. The structural feature of graph be represented by adjacency matrix A {0, 13 | where Aij = indicates an edge starting from node to node j. and Aij = 0 otherwise. The node feature matrix is represented as X E [
Graph neural networks (GNNs) [1, 2] aim to generate powerful representation on graphs in end-to-end fashion. Such representation facilitates the downstream tasks, such as node classification [20, 21], link prediction [22, 23, 24, 25], and graph classification [26]. Without loss of generality,
consider a graph classifier f G IRC which classifies an input graph G E G in C' categories and outputs prediction by c arg max; (G), Typically, consists of three components: (1) learning of node representations, which distills vectorized information from neighboring nodes and updates node representations recursively: (2) learning of graph representation, which aggregates the node representations to establish the representation for the holistic graph; (3) graph classification, which maps the graph representation into the probability distribution of different categories.
Explaining Graph Neural Networks. The explainer model (aka. the explanation method) usually performs two consecutive operations: (1) feature attribution [27, 28), which associates each feature of input G E G with the relevance score for the classifier's prediction; (2) feature selection [29, 6), which extracts salient features based on the relevance scores to construct an explanatory subgraph. The subgraph is regarded as the evidence for the GNN make the prediction.
We follow previous works [6, 7, 10, 19] and focus on the contributions of the structural features (i.e. edges). Our explainer consists of two components: attribution module for edge attribution and selection module H. for edge selection. Specifically, T assigns the adjacency matrix A with a saliency map, i.e.
M T(G.f.c.
where M R| each element of which is the importance score of the edge to the prediction class Such saliency map can further result in an attentive graph Gau A @ M. Then, the selection module H. identifies the edges of explanatory subgraph based on the attentive graph:
S H(Gatt f,c,)).
22)
where S E X constructs the explanatory subgraph Gemp = A @ S, and is the explanation budget [27] that equals to the number of nonzero elements in S.
3 Methodology
Here we present our explainer that purses multi- multi-grained explainability by pre-training and fine-tume -he-ttnnigg. as Figure 2 shows. In the pre-training phase, the attribution module distills the class-wise knowledge, which contrasts the salient structures based on the prediction, answering the question "Why did GNN model assign group of graphs with the same prediction?". In the next phase, the selection module goes beyond the class wise knowledge and fine-tunes the saliency maps on specific instance for answering "Why the GNN model made the certain prediction for the specific graph?".
3

---

3.1 Pre-training Towards Global Explainability
Class-aware Attribution Module. Towards the global explainability of GNN, it is important to specify the class-wise knowledge across the instances with the same prediction. Inspired by the success of generative models [7, 30, 31] in capturing the succinct structures from the graphs, we hire multiple generative probabilistic models [7] our attribution models (short for attributor), i.e. To = (T 1, 1,. , which is parameterized by A. The attributor T(t) is responsible for uncovering the hidden patterns from some graph instances O [g]c arg max; [(G): with the same prediction class c.
Formally, each attributor (() is composed of GNN encoder GNN(1) and MLP decoder MLP(s), whose parameters are shared when explaining graphs in O(s), so to systematize the class-wise pat- terns. Next we introduce the construction of each class-wise attributor, while we omit the superscript for conciseness. Specifically, the encoder GNN embeds each node in 9 with representation and summarize the representations of all nodes as:
Z = GNN X),
(3)
where Z E RR Vixd' encodes the structural feature A and node feature X. On the top of the node representations, we model the graph structure as edge distributions and frame the generation of explanatory subgraphs by sampling from the edge distributions:
P(M II P(MglZ4.22) (K.))E
14.
(4)
where Mij indicates the importance of edge (i.j). Then the MLP encoder takes the concatenation of node representations z, and z; as the inputs and outputs the importance score. To approximate the importance score the discrete distribution and optimize the generator via gradient propagation, we adopt the reparameterization trick [7]. where an independent random variable e ~ Uniform(O, 1) is introduced, As such, the edge probability formulated as:
P(Ma) (z, zy) ((lg T-c + (3))/3), with Og = ((,,((,,,,)):
(5)
where a is the sigmoid function, and 3 denotes the temperature hyperparameter. It is worth empha- sizing that our attributors different from PGExplainer [7], where only one generative probabilistic model is involved. Thus, their attribution results are limited in differentiating the patterns of different classes and systematizing the class-wise knowledge.
Pre-training Class-wise Attribution Module. We devise the following objective function for training the class-wise attributors.
min Li + CLtss
(6)
where y is the trade-off hyperparameter. We start from maximizing the mutual information between the attentive graphs and the target prediction of the graph, which widely-used learning paradigm in the literature [32, 6, 7]. It guides us to find the prediction-relevant explanatory subgraph, which
equals to minimizing the following loss:
Li Ee d'IG log P(Y c'IG clah),,
where and Y are the graph and prediction variables, respectively: G the full graph instance to explain; by sampling e E Uniform(O., 1) and c' E (1..... .1....,C'' C). the class-wise saliency map ((s) can be generated from Equation (4); P(Y = d' G 9) = f(G)² is the output probabilities of the prediction being d' when feeding 9 to the GNN model f; analogously, P(Y = d'IG = =
f(G!o)) audits the output probability when feeding g( A ((
Moreover, we introduce a contrastive learning [33, 34, 18, 35, 36, 37] loss to emphasize differences among the plass-wise patterns the substructure of the full graph that is distant to that of the graphs with a different prediction but close that of the graphs with the same prediction. makes each attributor focus on the unique and discriminative information within the class. Specifically, for the

---

saliency maps Gatti of 91 and Gue of G2, it encourages the agreements between Geed and G when 61 = 02, compared to that when 61 * 02:
Lets Rgg e [1- ((-1)(((=0)) p(elg g(re))
where j is the softplus function [34]; measures the similarity between two subgraphs, which is set as the representation similarity - (gutt Gatt)) = h, h2 where b is the graph representations by feeding Gles) into the encoder GNN(a) and aggregating the node representations. Similar for h2- In addition, following [6], we adopt the Seement-wise entropy and L1 norm on the edge probability. By jointly optimizing these two losses in Equation (6), the class-wise attribution module learns to stratify the discriminative information for different classes and generate the saliency maps with global view of the target GNN. Taking an information aaattionnteeooticaall look at Equation (8), minimizing contrastive learning loss is maximizing lower bound of the mutual information between the latent graph representations of two graphs within the same class.
3.2 Fine-tuning Towards Local Explainability
Having established the saliency map that exhibits the importance of each edge, the standard way is to rank all edges based on their importance scores and simply select the top edges as the explanatory subgraphs. However, we argue that such coarser-grained selection fails consider the dependencies of these selected edges explicitly. Within a high-quality explanatory subgraph, edges are supposed to cooperate with each other, form the coalition, and approach the target prediction better than individuals [38, 39]. Without considering such coalition effect, the quality of the explanatory subgraph is greatly limited. For example, when explaining why the molecule graph is classified as mutagenic [13], two connected vitrogen-oxygen (N-O) bonds form a chemical group NO, and present more discriminative information about the mutagenic property [13]: whereas, two salient but disconnected N-O bonds from different chemical groups are less informative to interpret the mutagenic property.
Clearly, the yoarser-grainet saliency maps insufficient exhibit the coalition effect of edges, thus might be redundant and suboptimal explanations. Hence, we move forward to learn a finer-grained explanatory subgraph. Technically, on the top of the well-trained class-wise attribution module, add the selection module:
g(c) = H(Gle p),,
where p is the number of edges selected in the explanatory subgraph: H. is a sampling (selection) function; s(e) preserves the elements selected by the selection function and sets the other elements as Instead of the hard selection that picks up the edges with the highest probability, H samples edges according to their probabilities. Allowing edges with low probabilities to be sampled can prevent the explainer from collapsing to suboptimal solutions with limited coalition effect.
With the new stochastic adjacency matrix S(e), we are able to extract the subgraph glo) To fine-tune the attribution and selection modules, we resort maximize the mutual information between the explanation candidate g!o)p and the target prediction of the full graph:
L2 E. = e = log P(Y = = -
(10)
By optimizing the loss above, the selection module accounts for the edge coalition within s(e), so as to achieve higher local fidelity. Moreover, as the selection module discards some elements in the stochastic adjacency matrix, it blocks parts of gradient backpropagation and possibly acts dropout function to avoid the overfitting on the instance-level explanations.
4 Experiments
We mainly aim to investigate the following questions:
RQI: How effective is the pre-training phase of ReFine, as compared to that of existing methods?
RQ2: How effective is the fine-tuning phase of ReFine, as compared to that of the pre-training phase?
5

---

4.1 Experimental Settings
Datasets and Target GNNs. We consider four datasets with various target GNNs:
Molecule graph classification. We use the Mutagenicity dataset [40, 41], where 3,7 molecule graphs are classified into two classes based their mutagenic effect on bacterium. The well- trained Graph Isomorphism Network (GIN) [26, 42] has achieved a 100% testing accuracy.
Scene graph classification. Following the previous work [10], we select 4, 443 (images, scene graphs) pairs from Visual Genome [43] to construct the VG-5 dataset. Wherein, the graphs are labeled with five classes: stadium, street, farm, surfing, forest. Each graph contains regions of the objects as the nodes, while edges indicates the relationships between object nodes. The target GNN is an APPNP [44] which achieves 64.3% testing aceuracy.
Handwriting graph classification. We use the MNIST superpixel dataset [45], which converts 70,000 images into the graphs of superpixel adjacency. Every graph is labeled one of ten digit classes. We trained a Spline-based GNN [46] which gains 97.9% accuracy in the testing dataset.
Motif graph classification. We follow prior studies [6. 7] to create synthetic dataset, BA-3motiff, which contains 3,000 graphs. Specifically, we adopt the Barabasi-Albert (BA) graphs as the base, and attach each base with one of three motifs: house, cycle, grid. The trained GNN model, ASAP [47], classifies them according to the type of attached motifs and achieved I 100% testing accuracy.
Baselines. We compare our ReFine with the state-of-the-art explanation methods:
SA [9] directly uses the gradients of the model prediction w.r.f. the adjacency matrix of the input graph as the importance of edges.
GNNExplainer [6] applies the soft masks on the messages carried by edges, where each mask indicates an edge's importance. Note that the masks of graph instances are trained individually.
PGExplainer [7] hires neural network to learn to generate the masks for the input edges. The generative model is trained over multiple explained instances.
PGM-Explainer [19] collects the prediction change the random node perturbations, and then learns a Bayesian network from these perturbation-predicce prediction observations, so as to capture the dependencies among the nodes and the prediction. Here transfer it to model the edge importance.
Optimization. For the parametric explanation methods (GNNExplainer, PGExplainer, PGM- Explainer), we apply grid search tune their own hyperparameters. For our ReFine framework, we use the Adam optimizer and set the learning rate of pre-training and finettuning as 1e-3 and le-4, respectively. All experiments are done on single Tesla V100 SXM2 GPU (32 GB).
Evaluation Metries. It is challenging to quantitatively evaluate the quality of explanations, since the ground-truth truth explanations are usually unavailable. In the literature, there are three widely-used evaluation metries:
Predictive A Accuracy (ACC@p) p) [32, 48, 27]. measures the fidelity of the explanatory subgraphs by feeding it solely into the target model and auditing how well it recovers the target prediction. We report the average ACC@ p over all graphs in the testing sets, and further denote ACC-AUC as the area under the ACC curve over different selection ratios p {0.1, 0.2, 0.9, 1.0). ACC@p and ACC-AUC are suitable for all the datasets.
Recall @ N. As suggested in prior studies [6, 7, 32]. we can create the "ground-truth explanations" for the synthetic dataset. Specifically, for BA-3motif, the motif of each graph can be viewed as the discriminative information coherent in the model knowledge. As such, we frame the evaluation problem as the task of top edge ranking. To be more specific, for an explanatory subgraph, the edges within the motif are positive, while the others are negative. To this end, recall can be adopted the evaluation protocols. More formally, Recall @ N = Eg SlG n // ([ll] where G, is composed of the Nop-N edges and G; is the ground- -tuth explanatory subgraph.
4.2 Quantitative Evaluations
Influence Pre-training (RQ1). To investigate the effectiveness of pre-training, we first compare the performance of the attribution module with the state-of-the-an thet explainers. We denote this variant

---

Table 1: Structure/Training Difference of PGExplainer, ReFine and its ablation models.
Pre-training
Fine-tuning
Class-wiss Attributors
Contrastive Learning
PG-Explainer
RefineCTT
Refine-FT
Refine
Table 2: Comparison of our ReFine and other baseline explainers
Mutagenicity
VG-5
MNIST
BA BA-3motif
ACC-AUC
ACC-AUC
ACC-AUC
ACC-AUC
SA
0.769
0.769
0.559
0.518
0.243
GNNExplainer
0.895+0.010
0.895+00.003
0.535+0.013
0.528+0.008
0.157++0.002
PG-Explainer
0.631=0.0008
0.790
0.504=0.010
0.586=0.004
0.293++.000
PGM-Explainer
0.714++0.007
0.792+0.000
0.615=0.003
0.575=0.002
0.250+0.000
ReFine-CT
0.88810.000
0.891
0.526110007
0.61010.004
0.248
ReFine-FT
0.90510.001
0.906.10.002
0.58710.000
0.61610.003
0.299110.00
ReFine 0.955 100005
0.004
0.636.10.003
0.630
0,304
Relative Impro.
6.7%
2.1%
3.4%
7.5%
3.8%
by ReFine-FT, which disables the fine-tuning phase and simply constructs the explanatory subgraphs based on the saliency scores. Moreover, we build another variant ReFine-CT, which removes the contrastive loss (Equation (8)) from the pre-training phase, to study the effect of the contrastive loss the class-wise knowledge modeling. To be more clear, we present the difference of PGExplainer [7], ReFine and its ablation models in Table 4.2. Table 2 presents the performance comparisons, from which we have several findings:
ReFine-FT outperforms the baseline explainers in most cases. To more specific, achieves significant relative improvements over the strongest baselines w.rl. ACC-AUC by 5.6% and 5.1% Mutagenicity and BA-3motif, respectively. This demonstrates the rationality and effectiveness of the attribution module. We attribute these improvements the class-wise knowledge modeling: (I) By specifying the attributor models for each class, ReFine-FT able to capture the underlying patterns shared across the instances within the same class; and (2) Conducting the contrastive learning between different Mlass-aware attributors makes ReFine-FT better stratify the discriminative information for different classes. The class-wise knowledge endows ReFine- FT with the global view the target model's 's workings.
Although PGExplainer is also equipped with the global view of the target model, its performance is worse than that of ReFine-FT. We ascribe this to the limitations of PGExplainer's global view, which is founded upon all the explained instances, but fails to differentiate the class-wise patterns. This again verifies the rationality and effectiveness of our attribution module.
ReFine-FT outperforms ReFine-CT by large margin, indicating that the contrastive learning plays a critical role in exhibiting the class-wiss wise knowledge. Specifically, it summarizes the patterns across similar instances and focuses on the information pertinent specific classes, while filtering the irrelevant and redundant information out.
Interestingly. in MNIST, the result of ReFine-FT FT is worse than that of PGM-Explainer. One possible reason is that the random perturbations (GGM-Explaing create a collection of broken graphs and offer more comprehensive observation of the graphs. We leave the exploration of Sebgrrah-predicttee relations as future work.
Influence of Fine-tuning (RQ2). To justify the effectiveness of the fine-tuning phase, report the performance ReFine with our selection module Tables and 3, as compared to the performance before fine-tuning. We have the following observations:
7

---

Table 3: Performance under different selection ratios before and after fine-tuning.
Mutagenicity
VG-5
MNIST
AA-3motif
p 0.4
0.6
0.44
0.6
0.44
0.6
0.4
0.6
ReFine-FT
96.8%
94.0%
91.3%
91.4%
41.4%
61.4%
36.0%
65.7%
ReFine
97.8%
96.2%
92.2%
93.4%
71.4%
82.0%
39.0%
72.8%
Improvement
+1.0%
+2.2%
+0.9%
+2.0%
+30.0%
+20.6%
+3.0%
+7.1%
Pre-trained
Fine-tuned
Figure 3: Qualitative Results in MNIST Superpixels dataset. Handwriting graphs are in black, which respectively represent number "O", "2", "8" within each block from left to right. Explanatory graphs in red, where the top 10% edges are highlighted.
Fine-tuning with the selection module improves the explanation performance sustainably, which indicates the effectiveness of our pre-training and fine-tuning paradigm. Specifically, in MNIST, the predictive accuracy of the explanations after fine-tuning improves from 41.4% to 71.4% when the selection rato is 0.4. We attribute these improvements to the local insights on specific instances: (1) Benefiting from the saliency map obtained the pre-training phase, the selection module is able to filter noisy edges out and narrow down to where the target model looks make decisions; (2) Fine-tuning the explanatory subgraphs considers the coalition effect of edges, thus approaches more information to recover the target prediction.
Jointly analyzing Tables 2 and 3, ReFine consistently outperforms all baselines across the four datasets. Advantageous to the local or global explanations, have the global understanding of model workings (i.e. the class-wise knowledge), but also account for the local insights specific instances (i.e. the coalition effect of edges in the local context). It illustrates the superiority of our ReFine paradigm.
multi-grained explanations not only
Overall. the empirical supports justify the significance of fine-tuning well. The contributions of fine-tuning w.t.. the overall improvements over PG-Explainer are 37.1% and 31.8% in MNIST and BA-3motif datasets, respectively. One possible reason that fine-tuning contributes only 3.1% and 6.4% portion of overall improvements in Mutagenicity and VG-5 as compared to PG-Explainer is the existance of rich node features in these two datasets. With the assistance of node features, the global patterns might be well-captured durining pre-training, thus leaving little space for the local patterns improve.
4.3 Qualitative Analysis
We present the qualitative results on MNIST superpixel in Figure 3, where the pre-trained and fine-tuned explanations are the explanatory subgraphs before fine-tuning (i.e. extracted based on the saliency map) and after fine-tuning (i.e. derived from the selection module), respectively.
Influence of Pre-training (RQ1). The pre-trained results (first row) well demonstrate the global patterns, where the explanatory subgraphs for interpreting the digit "0" focus more on the edges between hollows in the middle and the fringe of the number. While interpreting the prediction "5", the explanations identify the edges spread the bend of the number as the most important features. Also, we observe interesting pattern in the results for explaining the prediction "g", where the

---

Prob. 0.999
0.960
1,000
0.978 0,960
Prob. 0.543
0.888
0.598
0.976
0.975
Selection Ratio
Figure 4: Qualitative Results in Mutagenicity dataset. The prediction of the molccule in the first row mutagenic, while the molecule in the second row is predicted non-mutagenic. The selection ratios range from 10% to 50%. Note that some opposite edges are visually coincident.
Table 4: Time costs (in second) of GNNExplainer, PG-Explainer and the fine-tuning phase of Refine.
Mutagenicity
VG-5
MNIST
BA-3motif
GNNExplainer PG- EiEllaainne Refine(Fine-tuning
2.03
1.88
0.637
1.11
0.030
0.035
0.040
0.032
0.821
0.583
0.535
0.423
background edges draw more attention, rather than edges relevant to the digits, revealing the evidence for the target GNN to classify. It also shows the supporting evidence of the difference between the model explanation and the human explanation which focuses more on the digit graphs other than the background graphs. Through the pre-trained examples, the global patterns offer vital model understanding and inspections for the model's decision- process.
Influence of Fine-tuning (RQ2). We now compare the pre-trained and fine-tuned explanations. Clearly, the fine-tuned explanatory graphs make clearer boundaries on the instances. The explanation adapted with the user-defined ratio pays greater attention to details that are only applicable to the specific instances. For example, one can take closer look the explanations in the 4-th column. Without the fine-tuning phase, the explanation may distracted by the edges across the digit and the background, such that these transition edges might be deemed as the most important features while achieve suboptimal predictive accuracies. In contrast, the fine-tuned explanation dispels such misunderstanding, with a higher local accuracy. Similar patterns can be found in other examples.
The qualitative results on Mutagenicity are presented in Figure 4, where each explanation has been fine-tuned on the corresponding ratio. We can see the flexibility on ReFine, which enables the fine-tuning on specific user-defined ratio. With the selection ratio increases, the class probability output by the target GNN is generally stable or further improved. Moreover, the fine-tuning phase focuses more the combination of features, with the constraint of selection ratio, to purse the higher accuracy rather than intercepting on ranking based on the statie edge importance, which only valid under the addictive feature assumption [32].
4.4 Discussions
Efficiency for Generating Explanations. The inference time [7] to explain a new instance by the pre- pre-trained ReFine the same as PGExplainer under the same attributor construction. Different from GNNExplainer which has to retrain the model for each graph, ReFine only needs few fine- tuning steps on the pre-trained model (20 steps on average). Thus, ReFine can gain a boosting performance for explaining graphs while remaining efficient in terms of time complexity. Specifically, we summarize the time costs in the Table 4. Clearly, our ReFine is more efficient than GNNExplainer and is computationally comparable to PG-Explainer.

---

Limitations. Although ReFine can well-encode the class-wise knowledge by learning the parameters of multiple attributors, it can hardly map such knowledge the structure representation as XGNN [13]. This limits the human understanding on the core of input data via a conciseness substructure.
5 Related Work
We consider two classes of related work for GNNs explainability: studies local explainability. which independently explain for each input graph without referring to other knowledge, e.g. training data; studies global explainability, which provide explanations for multiple instances with the guide of the model-level or class-level knowledge. See [49, 8, 50] for more overviews.
Local Explainability. In general, there are two research lines. (1) Non- non-parametric explanation methods [10, 9, 11] use some heuristies as the feature contributions of specific instance, without involving additional trainable models. Gradient-like scores [10, 9, 11] are wisely-used heuristics, which is obtained by backpropagating the model prediction or loss to the input features, such as adjacency matrix [10], along with the model architecture. (2) Parametric explanation methods [6, 19, 51, 52] additionally train a parametrized explainer model to generate the saliency maps explanatory subgraphs for individual instances. The explainer model is typically optimized towards local fidelity [32, 48, 27], which uses the explanations to recover the target predictions. For example, GNNExplainer [6] learns soft masks for instance and applies them the adjaceney matrix. PGM-Explainer [19] trains an Bayesian network upon the pairs of graph perturbations and prediction changes. However, these methods fall short in capturing the prototypical patterns shared within the same groups or classes.
Global Explainability. This direction is less explored compared to the local explainability of GNNs [8]. To provide a global understanding of the model prediction, PGExplainer [7] formulates the generation of multiple explanations based on its collective and inductive property, and designs the attributor as deep neural network whose parameters are shared across the explained instances. XGNN [13] explains GNNs by training a graph generator, which outputs class- bass graph patterns to explain this class. As it is designed to explain the holistic class, making it hardly applicable on an specific instance, e.g., the graph patterns may not even exit on the instance.
6 Conclusion and Future Work
Multi-grained explainability promises to offer a flexible and all-round inspection of deep models' decision-making, which has been less explored in the literature. Motivated by this, proposed a novel generative probabilistic model, ReFine, to approach the multi- aualii-raanulaiity explainability via pre-training and fine-tuning. To exhibit global explanations with the prototypical patterns, the pre-training phase is founded upon the class-aware attribution modules and distills the class-level knowledge by contrastive learning. When given specific instance, the fine-tuning phase further adapts the global explanations in the local context with high fidelity. In the fashion of pre-training and fine-tuning. we can generate explanations with both global patterns and local features. Extensive results in four datasets show that our method indeed improves the quality of explanatory subgraphs.
As future direction, consider the extension of ReFine to fulfill the counterfactual explanation [53], which answers "Why the target GNN model made a certain prediction, rather than another prediction?", to enrich the multi-granularity explainability. Further, multi multi--rained explainability can be exhibited to explore the model robustness and heuristically guide the model construction.
Acknowledgments and Disclosure of Funding
Funding in direct support of this work: the Sea-NExT Joint Lab, Singapore MOE AcRF T2; the National Natural Science Foundation of China (U19A2079, 62121002); the National Key Research and Development Program of China (2020YFB (00YYFB146703))
References
[1] Vijay Prakash Dwivedi, Chaitanya K. Joshi, Thomas Laurent, Yoshua Bengio, and Xavier Bresson. Benchmarking graph neural networks. CoRR, Abs/2003.00982. 2020.
10

---

[2] Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and Philipp S. Yu. A comprehensive survey on graph neural networks. TNNLS, 32(1): (::4))42220 24, 2021.
[3] Shruthi Chari, Daniel M. Gruen, Oshani Seneviratne, and Deborah L. McGuinness. Directions for explainable knowledge-enabled systems. In Knowledge Graphs for eXplainable Artificial Intelligence: Foundations, Applications and Challenges, pages 245-261. 2020.
[4] Christopher J. Anders, Plamen Pasliev, Pan Kessel. Fairwashing explanations with off-manifold detergent. In ICML, pages 314-323, 2020.
Dombrowski, Klaus-Robert Müller, and
[5] Sebastian Lapuschkin, Stephan Wäldchen, Alexander Binder, Grégoire Montavon, Wojeiech Samek, and Klaus-Rober Müller. Unmasking clever hans predictors and assessing what machines really learn. Nature communications, 10(1):1-8, 2019.
[6] Zhitao Ying, Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec. Gnnexplainer: Generating explanations for graph neural networks. In NeurIPS, pages 9240-9251, 2019.
[7] Dongsheng Luo, Wei Cheng, Dongkuan Xu, Wenchao Yu, Bo Zong, Haifeng Chen, and Xiang Zhang. Parameterized explainer for graph neural network. In NeurlPS, 2020.
[8] Hao Yuan, Haiyang Yu, Shurui Gui, and Shuiwang Ji. Explainability in graph neural networks: taxonomic survey. CoRR, abs/2012.155455 2020.
[9] Federico Baldassarre and Hossein Azizpour. Explainability techniques for graph convolutional networks. CoRR, abs/1905. 1306// 2019.
[10] Phillip E. Pope, Soheil Kolouri, Mohammad Rostami, Charles E. Martin, and Heiko Hoffmann. Explainability methods for graph convolutional neural networks. In CVPR, pages 10772-10781, 2019.
[11] Thomas Schnake, Oliver Eberle, Jonas Lederer, Shinichi Nakajima, K. T. Schutt, Klaus-Robert Muller, and Grégoire Montavon. Higher-order explanations of graph neural networks via relevant walks. arXiv, 2020.
[12] Michael Sejr Schlichtkrull, Nicola De Cao, and Ivan Titov. Interpreting graph neural networks for NLP with differentiable edge masking. CoRR, abs/2010.00577, 2020.
[13] Hao Yuan, Jiliang Tang, Xia Hu, and Shuiwang Ji. XGNN: towards model-level explanations of graph neural networks. In Rajesh Gupta, Yan Liu, Jiliang Tang, and B. Aditya Prakash, editors, KDD, pages 430-438, 2020.
[14] Marco Túlio Ribeiro, Sameer Singh, and Carlos Guestrin. "why should trust you?": Explaining the predictions of any classifier. In KDD, pages 1135-1144 2016.
[15] Jeremy Howard and Sebastian Ruder. Universal language model fine-tuning for text classifica- tion. In ACL, pages 328-339, 2018.
[16] Jacob Devlin, Ming-Wa Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.0480) 2018.
[17] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey E. Hinton. A simple framework for contrastive learning of visual representations. In ICML, volume 119, pages 1597-1607, 2020.
[18] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In CVPR, pages 9726-9735, 2020.
[19] Minh N. Vu and My T. Thai. Pgm-explainer: Probabilistic graphical model explanations for graph neural networks. NeurlPS, 2020.
[20] Hongyang Gao and Shuiwang Ji. Graph u-nets. In ICML, volume 97, pages 2083-2092, 2019.

---

[21] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liö, and Yoshua Bengio. Graph attention networks. ICLR, 2018.
[22] William L. Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Newr/PS, pages 1024-1034, 2017.
[23] Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. Neural graph collabo- rative filtering. In SIGIR, pages 165-174, 2019.
[24] Xiang Wang, Xiangnan He, Yixin Cao, Meng Liu, and Tat-Seng Chua. KGAT: knowledge graph attention network for recommendation. In KDD, pages 950-958, 2019.
[25] Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yong-Dong Zhang, and Meng Wang. Lightenn: Simplifying and powering graph convolution network for recommendation. In SIGIR, pages 639 649,-68,240 2020.
[26] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In ICLR, 2019.
[27] Ian Covert, Scott Lundberg, and Su-In Lee. Feature removal is unifying principle for model explanation methods. In NeurIPS, 2020.
[28] Marco Ancona, Enea Ccolini, Cengiz Öztireli, and Markus Gross. Towards better understanding of gradient-based attribution methods for deep neural networks. In ICLR, 2018.
[29] Constantin F Aliferis, Alexander Statnikov, Ioannis Tsamardinos, Subramani Mani, and Xeno- fon D Koutsoukos. Local causal and markov blanket induction for causal discovery and feature selection for classification part Algorithms and empirical evaluation. JMLR, 11(1), 2010.
[30] Peter Hoff, Adrian Raftery, and Mark S Handcock. Latent space approaches social network analysis. Journal of the american Statistical association, 97(460): 1090-00)100019990 2002.
[31] Max Welling Thomas N. Kipf. Variational graph auto-encoders. In NeurlPS Workshops, 2016.
[32] Jianbo Chen, Le Song, Martin J. Wainwright, and Michael I. Jordan. Learning to explain: An infommtiion-thooeee perspective on model interpretation. In JCML, pages 882-891, 2018.
[33] Lukas Faber, Amin K Moghaddam, and Roger Wattenhofer. Contrastive graph neural network explanation. 2021.
[34] R. Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Philip Bachman, Adam Trischler, and Yoshua Bengio. Learning deep representations by mutual information estimation and maximization. In /CLR, 2019.
[35] Fan-Yun Sun, Jordan Hoffmann, Vikas Verma, and Jian Tang. Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization. In ICLR, 2020.
[36] Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph contrastive learning with augmentations. In NeurIPS, 2020.
[37] Jiancan Wu, Xiang Wang. Fuli Feng, Xiangnan He, Liang Chen, Jianxun Lian, and Xing Xie. Self-supervised graph learning for recommendation. In SIGIR, pages 726-735, 2021.
[38] Christopher Frye, Damien de Mijolla, Laurence Cowton, Megan Stanley, and Ilya Feige. Shapley-based explainability on the data manifold. In ICLR, 2021.
[39] H. W. Kuhn and A. W. Tucker. Contributions the theory of games, volume Princeton University Press, 1953.
[40] Jeroen Kazius, Ross McGuire, and Roberta Bursi. Derivation and validation of toxicophores for mutagenicity prediction. Journal of medicinal chemistry, 48(1):312-33) 2005.
[41] Kaspar Riesen and Horst Bunke. Iam graph database repository for graph based pattern recognition and machine learning. In SPR and SSPR, pages 287-297, 2008.
12

---

[42] Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitmik, Percy Liang, Vijay S. Pande, and Jure Leskovec. Strategies for pre-training graph neural networks. ICLR, 2020.
[43] Ranjay Krishma, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A. Shamma, Michael S. Bernstein, and Li Fei-Fei. Visual genome: Connecting language and vision using crowdsourced dense image annotations. IJCV, 123(1):32-73, 2017.
[44] Johannes Klicpera, Aleksandar Bojchevski, and Stephan Günnemann. Predict then propagate: Graph neural networks meet personalized pagerank. In ICLR, 2019.
[45] Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodolà, Jan Svoboda, and Michael M. Bronstein. Geometric deep learning on graphs and manifolds using mixture model cnns. In CVPR, pages 1425-5434, 2017.
[46] Matthias Fey, Jan Eric Lenssen, Frank Weichert, and Heinrich Müller. Splinecnn: Fast geometric deep learning with continuous b-spline kernels. In CVPR, pages 869-877, 2018.
[47] Ekagra Ranjan, Soumya Sanyal, and Partha Talukdar, ASAP: adaptive structure aware pooling for learning hierarchical graph representations. In AAAI, pages 5470-5477, 2020.
[48] Jian Liang, Bing Bai, Yuren Cao, Kun Bai, and Fei Wang. Adversarial infidelity learning for model interpretation. In KDD, pages 286-296, 2020.
[49] Hao Yuan, Haiyang Yu, Shurui Gui, and Shuiwang Ji. Explainability in graph neural networks: taxonomic survey. CoRR, 2020.
[50] Thomas Schnake, Oliver Eberle, Jonas Lederer, Shinichi Nakajima, Kristof T. Schütt, Klaus- Robert Müller, and Grégoire Montavon. XAI for graphs: Explaining graph neural network predictions by identifying relevant walks. CoRR, abs/2006.03589, 2020.
[51] Hao Yuan, Haiyang Yu, Jie Wang, Kang Li, and Shuiwang Ji. On explainability of graph neural networks via subgraph explorations. ArXiv, 2021.
[52] Patrick Schwab and Walter Karlen. Cxplain: Causal explanations for model interpretation under uncertainty. In NewrlPS, pages 10220-10230, 2019.
[53] Raha Moraffah, Mansooreh Karami, Ruocheng Guo, Adrienne Raglin, and Huan Liu. Causal interpretability for machine Seaming-problenee methods and evaluation. SIGKDD Explorations, 22(1): 22(1):18-35 33, 2020.
13


Reviews:
[{"review_id": "Jns_uYtCSA", "reviewer": "Reviewer_gLGL", "summary": "\nThis paper proposed a pretrain-fine tune framework for multi-grained GNN explaination. The experimental results show the proposed method to be effective.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Advantages:\n1.\tExperiments on multiple datasets are conducted to evaluate the method\n2.\tThe proposed method is reasonable and easy-to-understand.\n3.\tThe method is practical and easy for implementation.\n\nDisadvantages:\n1.\tThe proposed method is a combination of existing techniques. Specifically, the pre-training phase seems to be adopting PGExplainers for each class and adding a loss function to constrain the difference between classes. The fine-tuning phase is adopting simulated annealing algorithm to select subgraphs and use the subgraph to further train the model. This limits the technical contribution of this work.\n2.\tThe authors give no theoretical analysis to their model. As the authors only made limited adjustments to existing works, even though the method is effective, more theoretical insights should be shown to provide understanding on how each part improves the performance and give inspirations to the research community. \n3.\tThere are typos in the paper, such as ``Figure 3.1’’ in line 98.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "8"}, "extra_text": {}}, {"review_id": "Fhv4hnHelj2", "reviewer": "Reviewer_AmNU", "summary": "This work proposes an instance-level explanation method called ReFine for graph neural networks. ReFine first generates a probability map over the edges in a similar way as PGExplainer. Then, it uses contrastive loss to encourage the graph embedding of explanations to be class distinctive. Next, the usage of the selection module makes the soft selection of the edges in the explanations.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This work has a clear logic flow and strong motivations for each component in this method. \n\nThe usage of contrastive loss is creative and full of insight. And the ablation study in Table 1 also shows the effectiveness of this loss. \n\nFor the experiments, it provides solid results on both synthetic datasets and real-world datasets with different graph neural networks. \n\nEnhancement: This work only provides the experiments on the graph classification task, while I believe it can also be applied to the node classification task. However, it would be better to also conduct experiments for node classification.\n\nThe major concern is the technical contribution. Compared with PGExplainer, the only difference is the proposed loss functions. Then the technical contribution is somehow limited. \n\nI have a question about this work.\n\n1. Through the experiment in Table 1, finetuning procedure shows its power. But I am still confused about the details of the fine-tuning procedure described in Section 3.2. Could you provide more details and insights about this fine-tuning procedure?\n\nOverall, I think the idea is interesting and this paper is solid while the novelty is somehow limited. Hence, I would recommend a weak acceptance.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "YES", "review_text": "", "extra_scores": {"ethical_concerns": "N/A", "needs_ethics_review": "No", "ethics_review_area": ["I don’t know"], "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "DQ9sC9h9Bg", "reviewer": "Reviewer_6Ffx", "summary": "This paper presents a generative model to explain graph neural networks. The framework contains two steps: pre-training and fine-tuning. The first pre-training step trains edge probabilities in the adjacency matrix to capture class-wise saliency, followed by a contrastive learning by encouraging the similarity between two different adjacency matrices when they have the same class output. The second fine-tuning step employs  stochastic dropout to the edges based on the probabilities learned in the pre-training step while maximizing the mutual information between sampled subgraphs. Experiments on four datasets show consistent improvement over recent competing baselines, including SA, GNNExplainer, PGExplainer, and PGM-Explainer.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths \n* The task being addressed here is of high impact on how to explain learned graph neural networks, and can potentially help the community design better graph architecture in the future when we have deeper understanding of what the graph networks learn.\n* Thorough literature survey and background knowledge. I especially like Sec 4.1 where the authors give a one-sentence summary of each recent competing method. This really helps the readers get up to speed and delve deep into the core experimental results.\n* Strong performance in terms of usual metrics: ACC-AUC and recall. Figure 3 and 4 are also helpful in comprehending what the proposed loss functions capture qualitatively. Sec 4.2 is also well written in a sense that it gives a detailed walk through of the experimental results.\n\nWeaknesses\n* Despite the good metric numbers in tables, I still do not fully capture why the proposed two-stage pipeline can capture the claimed global patterns and local features. It seems like the only justification in the submission is in Figure 3 column 4, where the authors claim the explainer graph does not cross the MNIST digit and background. The authors did not perform the same pre-training / fine-tuning qualitative comparison in Figure 4 on Mutagenicity dataset. Based on the presented material, it is hard to fully believe that the proposed fine-tuning step can really capture the high local accuracy.\n* Based on the formulation in Eq. 9, the proposed fine-tuning for local explainability is essentially a stochastic version of edge dropout based on the learned probability matrix in the pre-training step. Therefore, it is understandable that dropout would give a reasonable amount of performance boost, especially when the training data is relatively small. This is also reflected in Table 1 that most of the performance boost is contributed by the contrastive learning. Again, the overall observation does not tell the story that pre-training captures the global patterns while fine-tuning (edge dropout) captures the local patterns.\n* Why does the fine-tuning (edge dropout) works well on MNIST but not that significant on other datasets?\n* Figure 3 does not speicicy which column represents what digits. It is extremely hard for readers to understand it.\n* Typo: line 55: show it effectiveness -> its, line 86: one extra period in the end of the sentence.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I appreciate the authors discussing the limitation of the proposed method on the mapping between captured knowledge and the structure representation [13]. It would be great for the authors to further address the weaknesses mentioned above, especially on why the fine-tuning stage actually captures local patterns.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "8"}, "extra_text": {}}, {"review_id": "5V3Kv0krIPP", "reviewer": "Reviewer_TzAp", "summary": "The authors improve the graph explainability by two methods. First is introducing the contrastive loss in the embedding space. The second is a random sampling to improve the original greedy top-k selection. The experiments are well done on different benchmarks with numerical results as well as qualitative visualization.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "* From Table 1, it seems the performance improvement are largely from the contrastive loss.  Recently, contrastive loss has been demonstrated to be effective in different applications, including graph learning, which makes the contribution relatively less novel. On the other hand, could you explain more how do you implement the class-wise attributor? Did you just simply train c attributors for each class? if so, is it a fair comparison that you use more parameters? \n\n* Although we see some improvements from Refine-FT to ReFine in Table 1. However, if I understand correctly, PGM is very close to ReFine-CT, except for the finetuning. Comparing PGM and ReFine-CT, we don't see much advantage of having finetuning. Could you comment on this?\n\n* The description from line 167 to line 181 could be improved. How do you sample \"random edges\"? How do you decide \"\\rho\"? What do you mean by decrease exponentially with t?  How sensitive are those parameters? It's known that the the greedy selection is suboptimal in many problem by discarding the dependency. Having so many degree of freedom allows us to find some sweet spot to be better than greedy. Some ablation and robustness study is necessary to make the proposed finetuning steps convincing. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"ethical_concerns": "Yes", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviews for this paper were overall positive, commenting on the high impact, strong performance,  good motivations, and easy to implement method. After the authors helped address some of the reviewers’ concerns (particularly around novelty), all reviewers are recommending acceptance.  

We recommend to take the reviewers' comments and suggestions into account while preparing the camera ready final version of the paper.  Particularly any intuition on why edge dropout helps local explainability would be beneficial.
Decision: Accept (Poster)