C AUSAL S CREENING TO I NTERPRET G RAPH N EURAL N ETWORKS
Anonymous authors
Paper under double-blind review
A BSTRACT
With the growing success of graph neural networks (GNNs), the explainability of GNN is attracting considerable attention.
However, current works on feature attribution, which frame explanation generation as attributing a prediction to the graph features, mostly focus on the statistical interpretability.
They may struggle to distinguish causal and noncausal effects of features, and quantify redundancy among features, thus resulting in unsatisfactory explanations.
In this work, we focus on the causal interpretability in GNNs and propose a method,
Causal Screening , from the perspective of cause-effect.
It incrementally selects a graph feature ( i.e., edge) with large causal attribution, which is formulated as the individual causal effect on the model outcome.
As a model-agnostic tool,
Causal Screening can be used to generate faithful and concise explanations for any GNN model.
Further, by conducting extensive experiments on three graph classiﬁcation datasets, we observe that Causal Screening achieves signiﬁcant improvements over state-of-the-art approaches w.r.t.
two quantitative metrics: predictive accuracy, contrastivity, and safely passes sanity checks.
I NTRODUCTION
Graph neural networks (GNNs) (Gilmer et al., 2017; Hamilton et al., 2017; Velickovic et al., 2018;
Dwivedi et al., 2020) have exhibited impressive performance in a wide range of tasks.
Such a success comes from the powerful representation learning, which incorporates the graph structure with node and edge features in an end-to-end fashion.
With the growing interest in GNNs, the explainability of GNN is attracting considerable attention.
A prevalent technique is to offer post-hoc explanations via feature attribution.
The attributions in GNNs are typically deﬁned as the contributions of input features ( e.g., nodes and edges) to the model’s outcome; thereafter, by selecting the most important features with top attributions, an explanatory subgraph is constructed to answer “Why this GNN model makes such predictions?”.
In this line, current works roughly fall into two categories: (1) decomposing the outcome prediction to graph structures via backpropagating the gradient-like signals (Pope et al., 2019; Baldassarre & Azizpour, 2019); and (2) approximating the decision boundary via structure perturbations (Huang et al., 2020) or structure masking (Ying et al., 2019).
However, these works mostly focus on the statistical interpretability (Pearl, 2018; Moraffah et al., 2020), which could fail to uncover the causation of model predictions reliably.
The key reason is that, they approach the input-outcome relationships from an associational standpoint, without distinguishing between causal and noncausal effects.
Using correlation as causation to interpret feature importance will result in unfaithful explanations, as shown in the running example.
Running Example.
Consider the example in Figure 1, where SA (Baldassarre & Azizpour, 2019) and GNNExplainer (Ying et al., 2019) use gradients and masks as attributions respectively, to explain why the scene type of a scene graph is predicted as Surﬁng by APPNP (Klicpera et al., 2019).
Two limitations in the statistical interpretability are: (1) Confounding association.
The edges with large gradient ( e.g., ( shorts, on, man )) or masks ( e.g., ( man, has, hand )) are highly correlated with the prediction, rather than causing it (Moraffah et al., 2020).
Such confounding associations distort the estimation of the causation ( e.g., ( standing, on, surfboard )); (2) Redundancy.
As the graph structure is highly entangled with GNNs, the gradient-like signals of edges are inﬂuenced, even scaled, by the connected edges.
This makes redundant edges ( e.g., ( man, on ocean ) and ( man, riding, waves )) involved in the top explanations, while forgoing other edges with unique information.
In this work, we focus on the causal interpretability (Pearl, 2018; Moraffah et al., 2020) in GNNs.
From a cause-effect standpoint, we need to answer causality-related questions like “Was it a speciﬁc edge that caused the GNN’s prediction?”.
Technically, explanatory subgraphs of interpretable features should account for two desirable characteristics: (1) Causality association.
We need to identify important features, which may plausibly be causal determinants of the model outcome, away from these associated with the outcome due to confounding.
As Figure 1 shows, ( standing, on, surfboard ) should be ranked at the top, instead of ( shorts, on, man ).
(2) Briefness.
We need brief explanations to avoid redundancy.
For example, ( man, on, surfboard ) is informative but redundant since it can be replaced by the top selection ( standing, on, surfboard ).
Hence, it is important to consider the dependency among edges.
Although causality has been considered in very recent works of other domains (Moraffah et al., 2020), they have yet to be applied in GNNs and fall short in the ability to explain GNNs.
See Section 2 for an exhaustive list and full discussion.
To the best of our knowledge, causal interpretability is yet unexplored in GNNs.
Here we propose a novel method, Causal Screening , to take a cause-effect look at feature attribution.
It takes a graph of interest, along with the prediction made by a trained GNN model, and aims to return an explanatory subgraph that plausibly has the largest causal attribution on the prediction.
Towards this end, Causal
Screening starts from an empty set as the explanatory subgraph, and adopts the screening strategy to incrementally select edges into the subgraph, one edge at a time step.
At its core is to quantify the causal attribution of an edge candidate as its individual causal effect (ICE) (Pearl, 2009), answering “What would happen to the prediction, if we add this edge into the GNN’s input?”.
It coincides with maximizing the information-theoretic measure at each step — conditional mutual information (Ay &
Polani, 2008; Janzing et al., 2013; O’Shaughnessy et al., 2020) between the edge and the prediction, conditioned on the previously selected edges.
Speciﬁcally, at each step, ICE is formulated as the difference between two outcomes, where the edge had received treatment ( i.e., combining it with the previous selection as the GNN input) or control ( i.e., feeding the previous selection alone into the GNN).
Last but not least, we propose an efﬁcient version which considers the cause-effect of edge groups, rather than single edges, in order to speedup the exhaustive search.
We apply Causal
Screening to multiple graph classiﬁcation datasets, generating qualitative results showcasing the effectiveness of our explanation subgraphs, which are more consistent, concise, and faithful to the predictions compared with existing methods.
Contributions of this study can be summarized as:
• We propose a model-agnostic method, Causal Screening, to provide a cause-effect perspective for explaining GNN models, i.e., uncovering causal relationships between graph features and model predictions.
• We conduct extensive experiments on three datasets, showcasing the effectiveness of our method w.r.t.
predictive accuracy, contrastivity, and sanity checks.
R ELATED W ORK
Interpretability in Non-Graph Neural Networks.
We focus preliminarily on feature attribution methods that generate post-hoc explanations for neural networks, especially convolutional neural networks (CNNs).
We roughly categorize current works into two groups: (1) Studies decompose the model prediction to the input features via backpropagating the gradient-like signals.
Early works like Gradient (Simonyan et al., 2014) and Gradient*Input (Shrikumar et al., 2016) directly use gradients w.r.t.
inputs as feature importance.
Some follow-on studies, such as LRP (Bach et al., 2015) and Grad-CAM (Selvaraju et al., 2017), improve gradients by using context within layers, while Integrated Gradient (IG) (Sundararajan et al., 2017) and DeepLIFT (Shrikumar et al., 2017) work on solving the gradient saturation problem; (2) Another line approximates the decision boundary of model via feature perturbations like LIME (Ribeiro et al., 2016) or feature masking like L2X (Chen et al., 2018) and VIBI (Bang et al., 2019).
LIME uses a sparse linear model as a local explainer, while L2X and VIBI learn additional neural networks as explainers to generate feature masks, which maximize the mutual information (or variation) with the masked features and the predictions.
However, these works mostly focus on the statistical interpretability, thus hardly uncovering the causation of model predictions.
Interpretability in Graph Neural Networks.
Interpretability in GNNs is less explored than that in CNNs.
Thus, researchers get inspiration from the methods devised for CNNs and have made some attempts.
For example, SA (Baldassarre & Azizpour, 2019) treats the gradients of GNN’s loss w.r.t.
the adjacency matrix as the attributions of edges.
Pope et al.
employ Grad-CAM family on GNN models.
GraphLIME (Huang et al., 2020) extends LIME to graph-structured data.
More recently, GNNExplainer (Ying et al., 2019) follows L2X and learns structure masking to maximize the mutual information between the subgraphs and GNN predictions.
XGNN (Yuan et al., 2020) uses the graph generation to explore the class-level explanations, rather than the instance-wise.
Despite the success, these methods inherit the limitations of the statistical interpretability.
Causal Interpretability.
Causality has been considered in very recent explanation works, such as structural causal models (Chattopadhyay et al., 2019; Kim & Bastani, 2019), or counterfactual reasoning (Goyal et al., 2019).
For example, based on Granger causality, CXPlain (Schwab &
Karlen, 2019) learns to quantify the causal effect of a single feature by leaving it out.
See Moraffah et al.
(2020) for a survey.
However, these approaches are mostly devised for CNNs and have yet to be applied in GNNs.
Moreover, they possibly fall short in the ability to explain GNNs, due to particular challenges: (1) the relational and discrete data exist in graphs, unlike continuous features in images; and (2) the graph structures are highly entangled with the representation learning of GNNs, such that different graphs likely result in various network architectures and different scales of explanations.
In this work, we take a cause-effect look at the interpretability in GNNs.
M ETHODOLOGY
We ﬁrst frame the explanation generation for GNNs from the causality standpoint.
Afterward, we detail the proposed method towards the causal interpretability by sequentially quantifying the causal attributions of single edges.
Lastly, we offer an efﬁcient version based on clusters.
3. P ROBLEM F ORMULATION
Background of GNNs.
Without loss of generality, let f : G → { 1 , · · · , S } denote a trained
GNN classiﬁer in the graph classiﬁcation task, where G ∈ G is a graph instance involving the node set V and the edge set E , along with node features and optional edge features.
In most GNNs, f is designed as: recursively create neural messages from neighboring nodes (or edges), incorporate these messages to enrich the representation of each ego node, then aggregate all node representations as the holistic graph representation, in order to classify it into S classes.
Task Description.
We deﬁne a graph of interest as G =
{ e | e ∈ E} to highlight the structure features ( i.e., the existence of an edge and its endpoints).
Given its prediction ˆ y = f ( G ) , our goal is to identify the top K causal edges and construct a convincing explanatory subgraph G ∗ K = { e ∗ 1 , · · · , e ∗ K } ⊆ G .
As a result, G ∗ K can be interpreted as the reason causing f to make the prediction ˆ y for the instance G .
In this work, we follow the previous works (Baldassarre & Azizpour, 2019;
Pope et al., 2019) to generate explanations based on structure features, and leave the identiﬁcation of a subset of node features in future work.
Causal Attribution of A Subgraph.
Technically, we obtain the causal explanatory subgraph G ∗ K as maximizing the causal attribution metric A ( · ) , which is formulated as:
[Equation]
Here we frame the metric A ( G K ) as the causal effect (CE) of G K , which is founded on the do ( · ) calculus (Pearl, 2009).
The interventions do ( G = G K ) and do ( G = ∅ ) mean that the GNN input G
𝑮 "𝒚 𝒇 𝑰 𝑮 "𝒚 𝒇 𝑰 ∅ 𝑮 "𝒚 𝒇 𝑰 𝑮 𝑲 𝑮 "𝒚 𝒇 𝑰 𝑮 𝒌"𝟏∗ 𝒆 𝒌 𝑰(𝑮 𝑲 ;"𝒚) 𝑰(∅;"𝒚) 𝑰(𝑮 𝒌#𝟏∗ ∪{𝒆 𝒌 };"𝒚|𝑮 𝒌#𝟏∗ ) 𝑰(𝑮 𝒌#𝟏∗ ;"𝒚|𝑮 𝒌#𝟏∗ ) (b)CausalAttributionofaSubgraph (c)CausalAttributionofanEdge 𝑮 "𝒚 𝒇 𝑰 𝑮 𝒌"𝟏∗ ∅ (a)CausalGraph 𝑮 "𝒚 𝑰 𝒇
GraphInput
GNNPrediction
GNNModel
MutualInformation receives treatment ( i.e., feeding G K into the GNN) and control ( i.e., feeding nothing into the GNN), respectively.
The CE of G K is to measure the difference between potential outcomes under treatment and control:
[Equation]
where I ( do ( G = G K ); ˆ y )
=
I ( G K ; ˆ y ) is the mutual information (Aliferis et al., 2010; Belghazi et al., 2018) between the intervened graph G K and the target prediction ˆ y , to calculate the change in the probability of prediction when the input graph is limited to the subgraph G K ; analogously,
I ( do ( G = ∅ ); ˆ y )
=
I ( ∅ ; ˆ y ) .
As the last term I ( do ( G = ∅ ); ˆ y ) is a constant, the solution of Equationis equivalent to obtain a subgraph G ∗ K with maximum predictivity to recover the target prediction
ˆ y .
Here we summarize the attribution framework in the form of structural causal model (well-known as causal graph) (Pearl, 2009; Pearl et al., 2016; Pearl & Mackenzie, 2018), as Figure 2 (a) shows.
It indicates how variables interact with each others, e.g., ( G , ˆ y )
→ I .
The intervention assigns the GNN input with a certain value, such that the difference caused by the treatment and control measures the causal attribution of a subgraph as shown in Figure 2 (b).
3. C AUSAL S CREENING
Causal Attribution of An Edge.
We further move on to considering the causal attribution of an edge, due to: (1) Highlighting importance of single edges is more preferred than showing a holistic subgraph solely; (2) Directly optimizing Equation 1 is generally NP-hard (Chen et al., 2018; Zhu et al., 2020;
Ying et al., 2019).
Hence we propose a new method, Causal Screening, which offers an alternative solution to Equation 1 via estimating the causal attributions of edges.
Speciﬁcally, it starts from an empty set as the explanatory subgraph, and incrementally selects the edges into the subgraph, one edge at a time.
Thus we revisit the objective function in Equation 1 as follows:
[Equation]
where G ∗ k − 1 = { e ∗ 1 , · · · , e ∗ k − 1 } is the set of the ﬁrst ( k − 1 ) selected edges after ( k − 1 ) steps, and
G ∗ 0 = ∅ at the initial step; and e ∗ k is the edge selected from the O k at step k , where O k = G\G ∗ k −is the pool of edge candidates.
Having established G ∗ k by merging G ∗ k − 1 and e ∗ k , we repeat this procedure until ﬁnding all top K edges as the ﬁnal explanatory subgraph.
The causal attribution A ( e k |G ∗ k − 1 ) is founded upon the individual causal effect (ICE) of e k , which accounts for the causal inﬂuence on the prediction and the redundancy with the previously selected edges.
Given the previous selection G ∗ k − 1 , the intervention do ( G = G ∗ k − 1 ∪ { e k } ) represents that the graph input receives treatment ( i.e., the GNN takes the combinations of e k and the previous selection G ∗ k − 1 ); whereas, do ( G = G ∗ k − 1 ) means that the graph input receives control ( i.e., feeding the previously selected edges alone into the GNN).
As such, the ICE is formulated as:
[Equation]
This coincides with conditional mutual information (Janzing et al., 2013; O’Shaughnessy et al., 2020):
A ( e k |G k − 1 )
=
I ( G k − 1 ∪ { e k } ; ˆ y |G k − 1 ) .
Figure 2 (c) presents the ICE inference.
By calculating the ICE of each edge, we could answer the causality-related questions like “Given the previously selected edges, what is the causal effect of an edge on the target prediction?”.
Following the literatures of feature selection (Kohavi & John, 1997; Guyon & Elisseeff, 2003) and causality (Aliferis et al., 2010; Guyon, 2007; Athey, 2017), we categorize edges into three groups based on the causal relationships with the predictions: (1) Strongly relevant edges contain unique information pertinent to GNN predictions, which are associated with signiﬁcant causal contributions.
(2) Weakly relevant edges are informative but redundant, which can be inferred and replaced by the previously selected edges.
Hence, these edges have varnishing (close-to-zero) causal contributions to the GNN predictions.
(3) Irrelevant edges do not bring any relevant information about predictions, whose causal contributions are zero.
In a nutshell, the strongly relevant edges are more convincing and explainable than the redundant and irrelevant edges.
Hence, the core of causal interpretability is to distinguish causal features ( i.e., strongly relevant edges) from non-causal features ( i.e., redundant edges and irrelevant edges).
We summarize the overall algorithm in Algorithm 1 of Appendix A.1.
3. S CALING TO L ARGE D ATA
However, selecting edges one by one might have some limitations: (1) It hardly measures importance of feature interactions or combinations.
For example, in social network analysis, group-wise patterns of users are more informative than behaviors of a single user; in genetics and genomics, block-like patterns of correlated genes work as functional groups, to determine biological properties.
The computational complexity of Equation 3 is O ((2 |G| − K ) K/ 2) per graph.
This will be a bottleneck of large-scale graphs involving massive edges.
Hence, we provide a cluster-by-cluster screening.
Causal Attribution of A Super-edge.
We ﬁrst apply a clustering method on graph instances to map nodes into clusters.
Here we resort to DiffPool (Ying et al., 2018).
For a given graph G , it outputs a node-cluster matrix X ∈ R |V|×|C| , where the v -th row denotes the soft cluster assignment of node v to C clusters.
By assigning every node to the cluster with the largest conﬁdence, we are able to simplify G as a new super-graph — nodes in the same cluster are combined as a super-node c ∈ C ; as such, intra-cluster edges are grouped as a self-loop of the super-node g = ( c, c ) , while we integrate the parallel inter-cluster edges together as a super-edge g =
( c, c (cid:48) ) across two super-nodes c and c (cid:48) .
Here we treat these two types of super-edges equally, without distinguishing them.
As a result, such super-edges not only reduce the scale of graph dramatically, but also characterize the group-wise patterns of graph ( e.g., user groups in social networks, functional groups in genomics).
We then employ Causal Screening on the simpliﬁed graph, to estimate the causal attributions of all super-edges O = { g } with their connected super-nodes.
At each step t ∈ { 1 , · · · , |O|} , the CE of a super-edge is:
[Equation]
where g t ∈ O t is the candidate from the super-edge set O t =
G\G ∗ t − 1 ; G ∗ t − 1 is the subgraph involving the previously selected super-edges; and the intervention do ( G = G ∗ t − 1 ∪ { g t } ) is to add g t into G ∗ t − 1 — which validates all its component edges — and feed them into the GNN model.
It is worth mentioning that, as one edge belongs to one and the only one super-edge, we could eliminate its inﬂuence on the CE estimations of other super-edges.
Hereafter, we can establish a CE matrix
H ∈ R C × C by ﬁlling the entry at the c -th row and c (cid:48) -th column with the CE of the super-edge g = ( c, c (cid:48) ) , or leaving it as zero if no super-edge presents.
We further distribute the CEs of super-edges to the ICEs of single edges.
However, it is difﬁcult due to the challenges: (1) as the component edges of a super-edge are validated together during the intervention, there lacks the knowledge of edge sequence; (2) beyond the component edges with the connected nodes, one intervention is likely to involve some isolated nodes without any edge, thereby introducing the contribution of these isolated nodes into the CE of the super-edge.
We get inspiration from the link prediction task (Koren, 2008), which exploits the function of two endpoints as a proxy to approximate the ICE of an edge.
Technically, we ﬁrst create the node representations via Z = XH ∈ R |V|×|C| , where the v -th row z v is the representation of node v that describes the contribution distributions over different clusters.
Then, for an edge e = ( v, v (cid:48) ) , we simply apply the inner product on the representations of endpoints v and v (cid:48) to approximate e ’s attribution:
[Equation]
Intuitively, if two endpoints have similar contribution distributions over clusters, the edge is highly likely to be inﬂuential to the information ﬂow along with the graph structure and contribute more to the GNN prediction.
Through this way, we are able to estimate the ICEs of single edges and retrieve the top- K edges as the explanatory subgraph.
This hierarchical fashion essentially reﬂects the modeling of GNNs: single nodes pass information to each others along with the graph structure, enhancing the group-wise patterns; thereafter, the patterns are used to classify the graph types.
Moreover, this approximation allows us to get results signiﬁcantly faster than exact calculations, as its time complexity is O ((2 |O| +1) |O| / 2+ |C| 2 ) , where |O| and |C| are the numbers of super-edges and clusters, respectively.
The algorithm is summarized in Algorithm 2 of Appendix A.1.
3. G ENERALIZATION TO O THER F EATURES
In many real-world scenarios like social networking (Tang et al., 2015) and recommendation systems (Zhang & Chen, 2018), only ID information is usually assigned with nodes or edges, without any pre-existing feature.
Hence, the structure features ( i.e., presence of edges, connections between nodes) attract more attention than other features ( e.g., node features).
We follow the previous studies (Yuan et al., 2020; Pope et al., 2019; Baldassarre & Azizpour, 2019) and build explanations upon the edge importance, which are applicable to general graph data.
Furthermore, as a general idea, Causal Screening can be easily generalized to other feature importance ( e.g., presence of nodes, features of nodes) by simply changing ICEs of edges to ICEs of target features.
E XPERIMENTS
We aim to investigate the research question: How do the explanations of Causal Screening perform?
4. E XPERIMENTAL S ETTINGS
We conduct comparative experiments on several various benchmark datasets over different GNN architectures, performing both quantitative and qualitative analyses to assess explanations.
Datasets with GNNs.
We consider the graph classiﬁcation task on three real-world domains:
• Molecule graph classiﬁcation.
We use the Mutagenicity dataset (Kazius et al., 2005; Riesen & Bunke, 2008), where 4 , 337 molecule graphs are categorized into two different classes based on their mutagenic effect on a bacterium.
The GNN model used is the graph isomorphism network (GIN)
(Xu et al., 2019; Hu et al., 2020).
• Social network classiﬁcation.
REDDIT-MULTI-5K (Yanardag & Vishwanathan, 2015) is used, where 4 , 999 graphs are labeled into ﬁve different classes based on the topics of question/answer communities.
A higher-order graph neural network (k-GNN) (Morris et al., 2019) is trained as the target classiﬁer model.
• Scene graph classiﬁcation.
We follow the previous work (Pope et al., 2019) and extract parts of the data from Visual Genome (Krishna et al., 2017), where each image is paired with a scene graph.
In each scene graph, nodes are objects associated with a region of the image, while edges describe the relationships between two objects.
Here we select 4 ,(images, scene graphs) pairs which are labeled with ﬁve classes: stadium, street, farm, surﬁng, forest.
We then train a APPNP model (Klicpera et al., 2019) on this dataset to classify the scene graphs into these classes.
Following prior studies (Pope et al., 2019; Ying et al., 2019), we partition each dataset into the training, validation, and testing parts with the ratio of 80% : 10% : 10% .
All graph instances are directed.
The GNN models are trained on the training dataset, while we run the explanation methods ﬁve times and report the average classiﬁcation and explanation performance on the testing dataset.
See Table 4 in Appendix A.2.1 for the statistics of datasets and the conﬁgurations of GNN models.
Alternative Baseline Approaches.
We consider the state-of-the-art explanation methods for GNNs:
SA (Baldassarre & Azizpour, 2019), Grad-CAM (Pope et al., 2019), and GNNExplainer (Ying et al., 2019), as our baselines.
Moreover, we transfer and extend IG (Sundararajan et al., 2017), DeepLIFT (Shrikumar et al., 2017), and CXPlain (Schwab & Karlen, 2019) to the graph domains.
Evaluation Metrics.
It is of crucial importance to evaluate the effectiveness of explanation methods.
Although the ground truth of explanations is usually unavailable, previous works have proposed some metrics to quantitatively assess the explanations.
Here we consider the following metrics (see
Appendix A.2.3 for the detailed formulations):
• Accuracy
(Chen et al., 2018; Liang et al., 2020) (ACC) reﬂects the consistency between the predictions based on the whole graph and the explanatory subgraphs.
• Contrastivity
(Pope et al., 2019) (CST) is to capture the intuition that class-discriminative explanations should differ among classes.
• Sanity Check (Adebayo et al., 2018)
(SC) is to test whether the explanations are dependent on the target model.
The attributions on the trained GNN model are compared with that on [Table] an untrained GNN with randomly initialized parameters.
Similar attributions infer that the explanation method is insensitive to properties of the model, thus failing to pass the check. By default
[Table]
we employ the standard Causal Screening on Mutagenicity and Visual Genome
[Table]
while the cluster-based version on REDDIT-MULTI-5K (with |C| = 10 ) due to its larger scale of graphs.
4. E VALUATION OF E XPLANATIONS
Quantitative Analyses.
We begin with the comparison w.r.t.
accuracy, where the ACC curve over different selection ratios is presented in Figure 3 and the area under curve is reported as ACCAUC in Table 1.
We ﬁnd
that: (1) Causal Screening achieves signiﬁcant improvements over the compared approaches.
For example, when only 10% edges are selected in Mutagenicity and REDDIT-MULTI-5K datasets, its ACC scores outperform others’ by a large margin. The edges selected by our method are indeed important and faithful to the GNN models.
Especially, in the Mutagenicity and Visual Genome datasets, our method shows nearly optimal ﬁdelity (ACC-AUC scores are 0 .
995 and 0 .
930 , respectively), which is validated by the GNN models. (3) Having selected the strongly relevant edges, adding more noncausal or redundant edges has only a negligible impact on the classiﬁer accuracy.
This justiﬁes the rationality and effectiveness of our conditional causal attributions.
(4) It veriﬁes the superiority of causal interpretability over to the statistical interpretability derived from other approaches (say, SA family and GNNExplainer).
These methods may attribute importance to the confounding associations, rather than uncovering the causation.
Hence, their explanations are unsatisfactory and less accurate.
Our method performs better than CXPlain, suggesting that screening to combine an edge with the previous selection estimates the causal effects of edges more accurately, compared to leaving a single edge out.
This emphasizes the rationality of considering edge dependency and confounding factors.
In a nutshell, taking advantage of causal attributions, Causal Screening is able to generate faithful explanations to GNN models.
We move on to the contrastivity reported in Table 1, to check whether explanations differ when the predictions of interest are different.
We ﬁnd that Causal Screening achieves the lowest contrastivity scores consistently.
For a graph, the rank correlation for our explanations in different classes is low — that is, the explanations are diverging from each others, such that a clear boundary is easy to identify.
For example, in the Mutagenicity dataset, different chemical groups are selected to explain the mutagenic and non-mutagenic labels.
It evidently shows that our method is able to offer classdiscriminative explanations.
We then focus on the sanity checks, where the results are summarized in Table 1.
We observe that:
(1) Causal Screening achieves low SC scores and passes the checks, indicating that the attributions
[Table]
on the trained and untrained GNNs differ signiﬁcantly.
This sensitivity reveals that our explanations are dependent on the target model.
(2) Grad-CAM achieves higher SC scores than SA substantially.
This is consistent with the ﬁndings in (Adebayo et al., 2018).
Insensitivity to the model parameters makes these methods fail to explain the inner working of GNNs reliably.
This evaluation again supports the rationality of our method.
Qualitative Analyses.
We present visual inspections in Figure 4 and observe that: (1) Our method not only selects more inﬂuential edges that possibly cause the prediction, but also ensures the briefness of edge combinations.
For example, when explaining the prediction of Stadium , the most convincing edge ( team member, in, arena ) is only ranked at the top of explanations by
Causal Screening; moreover, our method treats ( player, in, uniform ) as the second edge, rather than the confounding associations like ( shirt, on, player ) selected by GNNExplainer or the redundant information like ( player, lying down on, ﬂoor ) and ( player, in ﬁeld ) simultaneously captured by
CXPlain.
When explaining the prediction of Street , Causal Screening selects ( city buildings, on, street ), while SA and Grad-CAM fail.
This again veriﬁes the superiority of the causal interpretability to the statistical interpretability.
In Mutagenicity, Causal Screening is able to identify functional groups (which are not precisely deﬁned but evaluated by the domain experts).
For the 1-Nitro-5Pyrene molecule labeled as mutagenic, our method highlights the adjacent carbon rings and the
NO 2 chemical group, which are both responsible for the mutagenic property (Debnath et al., 1991).
Whereas, the baselines fail since they are possibly misled by some noncausal chemical bonds.
As for the prediction of non-mutagenic, our method assigns large causal attributions to chlorine (Cl) and ﬂuorine (F), which is consistent to the ﬁndings that the combination of Cl, bromine (Br), and F usually leads to non-mutagenic predictions (Yuan et al., 2020).
In contrast, Grad-CAM ignores Cl and is distracted by the carbon-hydrogen (C-H) bond, and CXPlain gives high important scores to irrelevant bonds but excludes C-Cl and C-F bonds out.
Time Complexity Analyses.
We also report the actual runtime of explanation methods in Tableon a GPU of NIVIDA Quadro T2000 4G.
In Mutagenicity and Visual Genome where the graphs are
[Table]
of a small scale, the standard Causal Screening generates explanations faster than GNNExplainer; meanwhile, in REDDIT-MULTI-5K where relatively large-scale edges exist in graphs, our clusterbased version has a comparable time cost to GNNExplainer and CXPlain.
4. S TUDY OF C AUSAL S CREENING
Inﬂuence of Cluster Numbers.
Table 3 presents the performance changes of the cluster-based
Causal Screening over the cluster numbers.
The computational cost drops signiﬁcantly when the cluster number changes from 30 to 5 , but all of them are much efﬁcient than the standard version ( i.e., the ﬁrst column).
(2) Surprisingly, the cluster-based version outperforms the standard version by a large margin w.r.t.
ACC-AUC.
We attribute such improvements to the clustering, which not only discovers the group-wise patterns of nodes, but also introduces the interactions among single edges into the causal attributions.
In essence, the cluster-based approximation works as a structural function between the edge and mutual information variables (Pearl et al., 2016) and yields state-ofthe-art results.
This veriﬁes the rationality and effectiveness of our cluster-based approximation.
Showcases of Causal Attributions.
For the top 20% edges selected by Causal Screening, we produce statistics of their ICE values and display their distributions in Figure 5.
In particular, each boxplot shows ﬁve measures of central tendency: minimum, ﬁrst quartile, median, third quartile, and maximum.
We ﬁnd that, the ICE values of causal edges ( i.e., strongly relevant) are much larger than that of non-causal edges ( i.e., redundant or irrelevant).
For example, in Mutagenicity, the causal edge has an ICE value within the range of 1 to 3, while the ICE of the non-causal approaches zero.
In a nutshell, through quantitative evaluations and visual inspections, we show that Causal Screening is able to generate high-quality post-hoc explanations reﬂecting the causation of GNN predictions.
Hence, the explanations as the distilled knowledge can in turn help us inspect the GNN models and improve them.
For instance, by emphasizing the informative structures, we can prevent GNNs from being distracted by redundant or useless information, further improve the model designs.
C ONCLUSION
In this paper, we have explored the causal interpretability of graph neural networks, and proposed a model-agnostic method, Causal Screening, to identify the most inﬂuential edges and generate post-hoc explanations for model predictions.
We make in-depth analyses on causal explanations w.r.t.
quantitatively evaluations and qualitative inspections, and observe that: causal attributions of edges are more reliable than gradient-like signals or feature masks obtained from the statistical perspective.
This work represents an initial attempt to exploit causality in GNN’s explanations.
In the future, we would like to consider more causality concepts, such as counterfactual reasoning to analyze the key features which make predictions different.
Moreover, we will extend our work to class-wise explanations like XGNN (Yuan et al., 2020).
A A PPENDIX A. A LGORITHMS
Here we present the algorithms for Causal Screening and the cluster-based version in Algorithmsand 2, respectively.
Algorithm
Input: G : graph instance, f target GNN predicted conditional entropy
Output: G ∗ K : explanatory subgraph;
G ∗ 0 ← ∅ ; foreach k ∈ { 1 , 2 , · · · , K } do maxA ← 0 ;
O k ← G\G ∗ k − 1 ; foreach e k ∈
O k do
I ( G ∗ k − 1 ; ˆ y ) ← H (ˆ y |G ∗ k − 1 ) ;
I ( G ∗ k − 1 ∪ { e k } ; ˆ y ) ← H (ˆ y |G ∗ k − 1 ∪ { e k } ) ;
A
G
− 1 ) ← I
G ∗ k − 1 ∪ { e k } ; ˆ y ) − I ( G ∗ k − 1 ; ˆ y ) ; if A ( e k |G ∗ k − 1 ) > maxA then e ∗ k ← e k ; maxA ← A ( e k |G ∗ k − 1 ) ; end end
G ∗ k ← G ∗ k − 1 ∪ e ∗
; end
Input
G
Output subgraph
X ← p ( G , C ) is the node-cluster assignment matrix;
O = { g } is the set of super-edges built upon the clusters;
H ∈ R C × C is the attribution matrix, after performing Algorithm 1 over the super-edges;
Z = XH is the attribution embedding matrix;
G ∗ K is the top- K edges with the highest attribution scores based on Equation 6;
A. E XPERIMENTS A.2. D ATASETS WITH GNN S
We summarize the statistics of the datasets in Table 4.
In particular, we use the original Mutagenicity and REDDIT-MULTI-5K datasets 1 .
As for the Visual Genome dataset, we follow the previous work (Pope et al., 2019) to construct the subset from the whole dataset 2 (Krishna et al., 2017).
Different from Pope et al.
(2019) that focuses on two binary classiﬁcation problems, we deﬁne a multi-class classiﬁcation problem: stadium, street, farm, surﬁng, forest.
By querying the whole dataset via a set of keywords, we combine the images returned for a class and discard the images across classes.
We build GNN models for these datasets via the PyTorch Geometric library 3 (Fey & Lenssen, 2019).
For Mutagenicity, we treat the given labels of nodes and edges as their pre-existing features, apply 1layer MLPs to project them into a common space, and then build a 2-layer GIN to perform the graph learning, which follows a average pooling and a 2-layer MLP to do classiﬁcation.
For REDDIT-
[Table]
MULTI-5K, a 3-layer k-GNN is built upon the degree features of nodes, and a average pooling is adopted to obtain the graph representation, which follows a 2-layer MLP to classify.
For Visual
Genome, we ﬁrst apply CNNs on the regional images to generate features for the object nodes, and then use a 2-layer APPNP followed by a 2-layer MLP as the classiﬁer model.
We carefully train these models, and report their best performance in Table 4.
A.2. I MPLEMENTAL D ETAILS
We implement Causal Screening in PyTorch, and will release all datasets and codes upon acceptance.
All the experiments are conducted on the GPU of NIVIDA Quadro
T2000 4G. Without speciﬁcation, we employ the standard Causal Screening ( i.e., Algorithm 1) on Mutagenicity and Visual Genome, while the cluster-based version ( i.e., Algorithm 2) on REDDIT-MULTI-5K (with |C| = 10 ) due to its larger scale of graphs. The empirical results are reported in Table 1. Moreover, we report the comparison between the standard and cluster-based versions, as well as the inﬂuence of cluster numbers in Table 3.
For the baseline SA, we follow the codes released by Baldassarre & Azizpour (2019).
In terms of IG, we create a new graph, which remains all node features but remove all edge features as the graph reference; moreover, the path steps of IG is 20 .
As for DeepLIFT, we further remove node features as the reference in order to solve the saturation problem in graph-structured data.
For
CXPlain and GNNExplainer that involve additional explainer networks, we apply a grid search to tune their hyper-parameters, and get the optimal settings: the learning rate is set as 0 .
01 in CXPlain and GNNExplainer, the regularization term and the weight of mutual information of GNNExplainer are set as 0 . 05 and 0 .
5 , respectively.
A.2. E VALUATION M ETRICS .
We adopt three widely-used metrics to quantitatively evaluate whether the explanations faithfully reﬂect the relationship between the input and model output: accuracy, contrastivity, and sanity check.
Here we do not consider the deletion or removal of edges as the evaluation metric, as suggested by Hooker et al.
Accuracy (Chen et al., 2018; Liang et al., 2020) (ACC).
This reﬂects the consistency between the predictions based on the whole graph and the explanatory subgraphs.
More formally, given a graph, we evaluate whether the GNN model truly uses the explanatory subgraph derived from an explanation method to generate the prediction:
[Equation]
[Equation] where µ is the selection ratio ( e.g., 5% ), K = µ × |G| is the size of explanatory subgraph; I ( · ) is the indicator function to check whether f ( G ) equals to f ( G ∗ K ) .
Contrastivity (Pope et al., 2019) (CST).
This is to capture the intuition that a reasonable explanation method should entail differing explanations with different predicted labels.
Here we formulate it as:
[Equation]
where s means we permute the label ˆ y of G being interpreted; Φ( G , ˆ y ) is the attribution scores of all edges; | ρ ( · ) | is the Spearman rank correlation with the absolute value to measure the invariance between the edge attributions, in response to the label changes.
Sanity Check (Adebayo et al., 2018) (SC).
We use the model randomization test to evaluate whether the explanations are dependent on the target model.
Specially, this test compares the attributions on the trained GNN model f with the attributions on an untrained GNN model ˜ f with randomly initialized parameters.
Here we frame it as the rank correlation between these two attributions:
[Equation]
Similar attributions infer the explanation method is insensitive to properties of the model, thus failing to pass the check.