D ECONFOUNDING TO E XPLANATION E VALUATION IN G RAPH N EURAL N ETWORKS
Anonymous authors
Paper under double-blind review
A BSTRACT
Explainability of graph neural networks (GNNs) aims to answer “Why the GNN made a certain prediction?”, which is crucial to interpret the model prediction.
The feature attribution framework distributes a GNN’s prediction to its input features ( e.g., edges), identifying an inﬂuential subgraph as the explanation.
When evaluating the explanation ( i.e., subgraph importance), a standard way is to audit the model prediction based on the subgraph solely.
However, we argue that a distribution shift exists between the full graph and the subgraph, causing the out-ofdistribution problem.
Furthermore, with an in-depth causal analysis, we ﬁnd the
OOD effect acts as the confounder, which brings spurious associations between the subgraph importance and model prediction, making the evaluation less reliable.
In this work, we propose Deconfounded Subgraph Evaluation (DSE) which assesses the causal effect of an explanatory subgraph on the model prediction.
While the distribution shift is generally intractable, we employ the front-door adjustment and introduce a surrogate variable of the subgraphs.
Speciﬁcally, we devise a generative model to generate the plausible surrogates that conform to the data distribution, thus approaching the unbiased estimation of subgraph importance.
Empirical results demonstrate the effectiveness of DSE in terms of explanation ﬁdelity.
I NTRODUCTION
Explainability of graph neural networks (GNNs) (Hamilton et al., 2017; Dwivedi et al., 2020) is crucial to model understanding and reliability in real-world applications, especially when about fairness and privacy (Ying et al., 2019;
Luo et al., 2020).
It aims to provide insight into how predictor models work, answering “Why the target GNN made a certain prediction?”.
Towards this end, a variety of explainer models are proposed for feature attribution (Selvaraju et al., 2017;
Ying et al., 2019; Luo et al., 2020; Vu & Thai, 2020), which decomposes the predictor’s prediction as contributions ( i.e., importance) of its input features ( e.g., edges, nodes).
While feature attribution assigns the features with importance scores, it redistributes the graph features and creates a new distribution different from that of the original full graphs, from which a subgraph is sampled as the explanation.
Such sampling process is referred to as feature removal (Covert et al., 2020).
Then, to assess the explanatory subgraph, the current evaluation frameworks use the feature removal principle — (1) only feed the subgraph into the target predictor, discarding the other features; (2) measure the importance of the subgraph based on its information amount to recover the model’s prediction.
Such subgraph-prediction correlations uncovered by the removal-based evaluator should offer a faithful inspection of the predictor’s decision-making process and assess the ﬁdelity of the explainers reliably.
However, feature removal brings the out-of-distribution (OOD) problem (Frye et al., 2020; Chang et al., 2019; Lukas Faber, 2021): the distribution shift from full graphs to subgraphs likely violates underlying properties, including node degree distribution (Leskovec et al., 2005) and domain-speciﬁc constraints (Liu et al., 2018) of the full graphs.
For example, graph properties of chemical molecules, such as the valency rules, impose some constraints on syntactically valid molecules (Liu et al., 2018); hence, simply removing some bonds (edges) or atoms (nodes) creates invalid molecular subgraphs that never appear in the training dataset.
Such OOD subgraphs could manipulate the predictor’s
𝑮 : FullGraph 𝑮 𝒔 : Subgraph 𝒀 : PredicateLogits 𝐬 𝓖
House Cycle Crane 0.21 0.70 TargetPredictor
Crane 𝓖 Cycle
House 𝓖 𝒔𝟏 𝑫 : DistributionShift 𝓖 𝒔 𝒔∗ 𝐬
Front-doorAdjustment 𝓖 𝒔𝟐 𝓖 𝒔 (a) Feature Removal to Evaluate
Subgraph G s 𝑮 : Full Graph 𝑮 𝒔 : Subgraph 𝒀 : Predicate Logits 𝐬 𝓖
House Cycle Crane 0.21 0.70 Target Predictor
Crane 𝓖 Cycle
House 𝓖 𝒔 𝑫 : DistributionShift 𝓖 𝒔 𝒔∗ 𝐬
Front-door Adjustment 𝓖 𝒔𝟐 𝓖
[Figure]
outcome arbitrarily (Dai et al., 2018; Zügner et al., 2018), generates erroneous predictions, and limits the reliability of the evaluation process.
Here we demonstrate the OOD effect by a real example in Figure 1a, where the trained ASAP (Ranjan et al., 2020) predictor has classiﬁed the input graph as “House” for its attached motif (see Sectionfor more details).
On the ground-truth explanation G s 1 , the output probability of the “House” class is surprisingly low ( 0 . 21 ).
While for G s 2 with less discriminative information, the outputs probability of the “House” class ( 0 . 70 ) is higher.
Clearly, the removal-based evaluator assigns the OOD subgraphs with unreliable importance scores, which are unfaithful to the predictor’s decision.
The OOD effect has not been explored in evaluating GNN explanations, to the best of our knowledge.
We rigorously investigate it from a causal view (Pearl et al., 2016; Pearl, 2000; Pearl & Mackenzie, 2018).
Figure 1b represents our causal assumption via a structural causal model (SCM) (Pearl et al., 2016; Pearl, 2000), where we target the causal effect of G s on Y .
Nonetheless, as a confounder between G s and Y , distribution shift D opens the spurious path G s ← D → Y .
By “spurious”, we mean that the path lies outside the direct causal path from G s to Y , making G s and Y spuriously correlated and yielding an erroneous effect.
And one can hardly distinguish between the spurious correlation and causative relations (Pearl et al., 2016).
Hence, auditing Y on G s suffers from the
OOD effect and wrongly evaluates the importance of G s .
Motivated by our causal insight, we propose a novel evaluation paradigm, Deconfounded Subgraph
Evaluator (DSE), to faithfully measure the causal effect of explanatory subgraphs on the prediction.
[Figure]
Based on Figure 1b, as the distribution shift D is hardly measurable, we cannot block the backdoor path from G s to Y by the backdoor adjustment.
Thanks to the front-door adjustment (Pearl et al., 2016), we instead consider the SCM in Figure 2, where we introduce the surrogate G ∗ s between G s and Y , which “imagines” what the full graphs is like given the subgraphs.
We obtain the causal effect of G s on Y by identifying the causal effects carried by G s → G ∗ s and G ∗ s → Y , which requires G ∗ s to respect the data distribution.
Hence we design a generative model, Conditional Variational Graph Auto-Encoder (CVGAE), to generate the possible surrogates.
It is worthwhile mentioning that our DSE is explainer-agnostic, which can assist the explanation evaluation reliably and further guide explainers to generate faithful explanations
In a nutshell our contributions are:
• From a causal perspective, we argue that the OOD effect is the confounder that causes spurious correlations between subgraph importance and model prediction.
• We propose a deconfounding paradigm, DSE, which exploits the front-door adjustment to mitigate the out-of-distribution effect and evaluate the explanatory subgraphs unbiasedly.
• We validate the effectiveness of our framework over various explainers, target GNN models, and datasets.
Signiﬁcant boosts are achieved over the conventional feature removal techniques.
Code and datasets are available at: https://anonymous.4open.science/r/DSE-24BC/.
A C AUSAL V IEW OF E XPLANATION E VALUATION
Here we begin with the causality-based view of feature removal in Section 2.1 and present our causal assumption to inspect the OOD effect in Section 2.2.
2. P ROBLEM F ORMULATION
Without loss of generality, we focus on the graph classiﬁcation task: a well-trained GNN predictor f takes the graph variable G as input and predicts the class Y ∈ { 1 , · · · , K } , i.e., Y =
f
( G ) .
Generation of Explanatory Subgraphs.
Post-hoc explainability typically considers the question “Why the GNN predictor f made certain prediction?”.
A prevalent solution is building an explainer model to conduct feature attribution (Ying et al., 2019; Luo et al., 2020; Pope et al., 2019).
It decomposes the prediction into the contributions of the input features, which redistributes the probability of features according to their importance and sample the salient features as an explanatory subgraph G s .
Speciﬁcally, G s can be a structure-wise (Ying et al., 2019;
Luo et al., 2020) or featurewise (Ying et al., 2019) subgraph of G .
In this paper, we focus on the structural features.
That is, for graph G =
( N , E ) with the edge set E and the node set N , the explanatory subgraph G s =
( N s , E s ) consists of a subset of edges E s ⊂ E and their endpoints N s = { u, v | ( u, v )
∈
E s } .
Evaluation of Explanatory Subgraphs.
Insertion-based evaluation by feature removal (Covert et al., 2020; Dabkowski & Gal, 2017) aims to check whether the subgraph is the supporting substructurethat alone allows a conﬁdent classiﬁcation.
We systematize this paradigm as three steps: (1) divide the full graph G into two parts, the subgraph G s and the complement G s ; (2) feed G s into the target
GNN f , while discarding G s ; and (3) obtain the model prediction on G s , to assess its discriminative information to recover the prediction on G .
Brieﬂy, at the core of the evaluator is the subgraphprediction correlation.
However, as discussed in Section 1, the OOD effect is inherent in the removal-based evaluator, hindering the subgraph-prediction correlation from accurately estimating the subgraph importance.
2. S TRUCTURAL C AUSAL M ODEL
To inspect the OOD effect rigorously, we take a causal look at the evaluation process with a Structural
Causal Model (SCM I) in Figure 1b.
We denote the abstract data variables by the nodes, where the directed links represent the causality.
The SCM indicates how the variables interact with each other through the graphical deﬁnition of causation:
• G → G s ← D .
We introduce an abstract distribution shift variable D to sample a subgraph G s from the edge distributions of the full graph G .
• G s → Y ← D .
We denote Y as the prediction variable ( e.g., logits output), which is determined by (1) the direct effect from G s , and (2) the confounding effect caused by D .
In particular, the former causation that led to the result is the focus of this work.
We suggest readers to refer to Appendix A where we offer an elaboration of D .
With our SCM assumption, directly measuring the importance of explanatory subgraphs is distracted by the backdoor path (Pearl, 2000), G s ← D → Y .
This path introduces the confounding associations between G s and Y , which makes G s and Y spuriously correlated, i.e., biases the subgraph-prediction correlations, thus making the evaluator invalid.
How to mitigate the OOD effect and quantify G s ’s genuine causal effect on Y remains largely unexplored in the literature and is the focus of our work.
D ECONFOUNDED E VALUATION OF E XPLANATORY S UBGRAPHS
In this section, we propose a novel deconfounding framework to evaluate the explanatory subgraphs in a trustworthy way.
Speciﬁcally, we ﬁrst leverage the front-door adjustment (Pearl, 2000) to formulate a causal objective in Section 3.1.
We then devise a conditional variational graph auto-encoders (CVGAE) as the effective implementation of our objective in Section 3.2.
3. F RONT - DOOR A DJUSTMENT
To the best of our knowledge, our work is the ﬁrst to adopt the causal theory to solve the OOD problem in the explanation evaluation of GNNs.
To pursue the causal effect of G s on Y , we perform the calculus of the causal intervention P ( Y =
y | do ( G s = G s )) .
Speciﬁcally, the do -calculus (Pearl, 2000; Pearl et al., 2016) is to intervene the subgraph variable G s by cutting off its coming links and assigning it with the certain value G s , making it unaffected from its causal parents G and D .
From inspection of the SCM in Figure 1b, the distribution effect D acts as the confounder between G s and
Y , and opens the backdoor path G s ← D
→ Y .
However, as D is hardly measurable, we can not use the backdoor adjustment (Pearl, 2000; Pearl et al., 2016) to block the backdoor path from G s to
Y .
Hence, the causal effect of G s on Y is not identiﬁable from SCM I.
However, we can go much further by considering SCM II in Figure 2 instead, where a mediating variable G ∗ s is introduced between G s and Y :
• G s → G ∗ s .
G ∗ s is the surrogate variable of G s , which completes G s to make them in the data distribution.
First,
it originates from and contains G s
Speciﬁcally
it imagines how the possible full graphs should be when observing the subgraph G s .
Second G ∗ s should follow the data distribution and respect the inherent knowledge of graph properties, thus no link exists between D and G ∗ s .
• G ∗ s → Y .
This is based on our causal assumption that the causality-related information of G s on Y , i
e., the discriminative information for G s to make prediction, is well-preserved by G ∗ s .
Thus, with the core of G s , G ∗ s is qualiﬁed to serve as the mediator which further results in the model prediction.
With SCM II, we can exploit the front-door adjustment (Pearl, 2000; Pearl et al., 2016) instead to quantify the causal effect of G s on Y .
Speciﬁcally, by summing over possible surrogate graphs G ∗ s of
G ∗ s , we chain two identiﬁable partial effects of G s on G ∗ s and G ∗ s on Y together:
[Equation]
Speciﬁcally, we have P ( G ∗ s | do ( G s = G s ))
=
P ( G ∗ s | G s = G s ) as G s is the only parent of G ∗ s .
And we distinguish the G s in our target expression P
( Y | do ( G s = G s )) between G (cid:48)
s , the latter of which is adjusted to pursue P ( Y | do ( G ∗ s =
G ∗ s )) .
With the data of ( G s , G ∗ s ) pairs, we can obtain
P ( Y | G ∗ s =
G ∗ s , G s = G (cid:48) s ) by feeding the surrogate graph G ∗ s into the GNN predictor, conditional on the subgraph G (cid:48) s ; similarly, we can estimate P ( G s = G (cid:48) s ) statistically; P ( G ∗ s =
G ∗ s | G s = G s ) is the conditional distribution of the surrogate variable, after observing the subgraphs.
As a result, this front-door adjustment yields a consistent estimation of G s ’s effect on Y and avoids the confounding associations from the OOD effect.
3. D EEP G ENERATIVE M ODEL
However, it is non-trivial to instantiate G ∗ s and collect the ( G s , G ∗ s ) pairs.
We get inspiration from the great success of generative models and devise a novel probabilistic model, conditional variational graph auto-encoder (CVGAE), and an adversarial training framework, to generate G ∗ s .
Conditional Generation.
Inspired by previous works (Thomas N. Kipf, 2016; Liu et al., 2018), we model the data distribution via a generative model g θ parameterized by θ .
It is composed of an encoder q ( Z |G , G s ) and a decoder p ( G ∗ s | Z ) .
Speciﬁcally, the encoder q ( Z |G , G s ) embeds each node i in G with a stochastic representation z i , and summarize all node representations in Z : q ( Z |G , G s ) =
N (cid:89) i = q ( z i |G , G s ) , with q ( z i |G , G s ) =
N ( z i | [ µ 1 i , µ 2 i ] , (cid:20) σ 21 i  σ
i (cid:21) ) (2) where z i is sampled from a diagonal normal distribution by mean vector [ µ 1 i , µ 2 i ] and standard deviation vector diag ( σ 2 1 i , σ 2 2 i ) ; µ 1 = f µ ( G ) and log σ 1 = f σ ( G ) denote the matrices of mean
𝓖 𝒔�∗
[Figure] Encoder
AGG
Edge emb.
… … … … 𝓖 𝒔𝟏∗ Sample … 𝝁 𝝈 … … 𝑞 ( 𝒁 | 𝓖 , 𝓖 𝒔 )
Sample 𝑝 ( 𝓔 𝒔∗ | 𝒁 )
𝑷 dse = 𝟎 .
vectors µ 1 i and standard deviation vectors log σ 1 i respectively, which are derived from two GNN models f µ and f σ on the top of the full graph G ; similarly, µ 2 = f µ ( G s ) and log σ 2 = f σ ( G s ) are on the top of the subgraph G s .
Then, the decoder p ( G ∗ s | Z ) generates the valid surrogates:
[Equation]
i
N
[Equation]
j p ( A ij | z i , z j ) , with p ( A ij = 1 |
z i , z j ) =
f A ([ z i , z j ]) , (3) where A ij = 1 indicates the existence of an edge between nodes i and j ; f A is a MLP, which takes the concatenation of node representations
z i and z j as the input and outputs the probability of A ij = 1 .
Leveraging the variational graph auto-encoder, we are able to generate some counterfactual edges that never appear in G and sample G ∗ s from the conditional distribution p ( G ∗ s | Z ) , formally, G ∗ s ∼ p ( G ∗ s | Z ) .
As a result, P ( G ∗ s =
G ∗ s | G s = G s ) in Equation 1 is identiﬁed by p ( G ∗ s | Z ) .
The quality of the generator directly affects the quality of the surrogate graphs, further determines how well the frontdoor adjustment is conducted.
Next, we will detail an adversarial training framework to optimize the generator, which is distinct from the standard training of VAE.
Adversarial Training.
To achieve high-quality generation, we get inspiration from the adversarial training (Goodfellow et al., 2020;
Yue et al., 2021) and devise the following training objective:
[Equation]
where γ , ω are trade-off hyper-parameters.
These losses are carefully designed to assure the generation follows the data distribution.
Next, we will elaborate on each of them.
[Equation]
We ﬁrst minimize the β -VAE loss(Higgins et al., 2017), and the ﬁrst term is the reconstruction loss responsible to predict the probability of edges’ existence; the second term is the KL-divergence between the variational and prior distributions.
Here we resort to the isotropic Gaussian distribution p ( Z ) =
(cid:81)
i p ( z i ) =
(cid:81)
i N ( z i | 0 , I ) as the prior.
β reweighs the KL-divergence, which promises to learn the disentangled factors in Z (Higgins et al., 2017;
Yue et al., 2021; Suter et al., 2019).
Moreover, we highlight the class-discriminative information in Z , by encouraging the agreement between graph representations with the same class compared to that with different classes.
Technically, the contrastive loss is adopted:
[Equation]
[Equation]
where z G is the representation of G that aggregates all node representations Z together; s is the similarity function, which is given by an inner product here; τ is the temperature hyper-parameter;
B
+ is the graph set having the same class to G , while the graphs involved in B − have different classes from G .
Minimizing this loss enables the generator to go beyond the generic knowledge and uncover the class-wise patterns of graph data.
Besides, we introduce a discriminative model d µ to distinguish the generated graphs.
Speciﬁcally, we set it as a probability-conditional GNN (Fey & Lenssen, 2019) parameterized by µ .
It takes a graph as input and outputs a score between 0 to 1 , which indicates the conﬁdence of the graph being realistic.
Hence, given a real graph G with the ground-truth label y , we can use the generator g θ to generate G ∗ s .
Then the discriminator learns to assign G with a large score while labeling G ∗ s with a small score.
To optimize the discriminator, we adopt the Wasserstein GAN (WGAN) (Martin Arjovsky, 2017) loss:
[Equation]
where d ( G ∗ s , y ) is the probability of generating G ∗ s from the generator; λ is the hyper-parameter.
By playing the min-max game between the generator and the discriminator in Equation 4, the generator can create the surrogate graphs from the data distribution plausibly.
Subgraph Evaluation.
With the well-trained generator g ∗ θ whose parameters are ﬁxed, we now approximate the causal effect of G s on Y .
Here we conduct Monte-Carlo simulation based on g ∗ θ to sample a set of plausible surrogate graphs {G ∗ s } from p ( G ∗ s | Z ) .
Having collected the ( G s , G ∗ s ) data, we can arrive the estimation of Equation 1.
E XPERIMENTS
We aim to answer the following research questions:
• Study of Explanation Evaluation.
How effective is our DSE in mitigating the OOD effect and evaluating the explanatory subgraph more reliably?
(Section 4.2) • Study of Generator.
How effective is our CVGAE in generating the surrogates for the explanatory subgraphs and making them conform to the data distribution?
(Section 4.3)
4. E XPERIMENTAL S ETTINGS
Datasets & Target GNNs.
We ﬁrst train various target GNN classiﬁers on the three datasets:
• TR3 is a synthetic dataset involving 3000 graphs, each of which is constructed by connecting a random tree-shape base with one motif (house, cycle, crane).
The motif type is the ground-truth label, while we treat the motifs as the ground-truth explanations following Ying et al.
; Yuan et al.
(2020a).
A Local Extremum GNN (Ranjan et al., 2019) is trained for classiﬁcation.
• MNIST superpixels (MNIST sup ) (Monti et al., 2017) converts the MNIST images into 70, superpixel graphs.
Every graph with 75 nodes is labeled as one of 10 classes.
We train a Splinebased GNN (Fey et al., 2018) as the classiﬁer model.
The subgraphs representing digits can be viewed as human explanations.
[-• Graph-SST
-]Graph-SST2 (Yuan et al., 2020b) is based on text sentiment dataset SST2 (Socher et al., 2013) and converts the text sentences to graphs where nodes represent tokens and edges indicate relations between nodes.
Each graph is labeled by its sentence sentiment.
The node embeddings are initialized by the pre-trained BERT word embeddings (Devlin et al., 2018).
Graph Attention Network (Veliˇckovi´c et al., 2018) is trained as the classiﬁer.
Ground-Truth Explanations.
By “ground-truth”, we follow the prior studies (Ying et al., 2019; Yuan et al., 2020a;
Luo et al., 2020) and treat the subgraphs coherent to the model knowledge ( e.g., the motif subgraphs in TR3) or human knowledge ( e.g., the digit subgraphs in MNIST sup ) as the ground-truth explanations.
Although such ground-truth explanations might not ﬁt the decision-making process of the model exactly, they contain sufﬁcient discriminative information to help justify the explanations.
Note that no ground-truth explanation is available in Graph-SST2.
Explainers .
To explain the decisions made by these GNNs, we adopt several state-of-the-art explainers, including SA (Baldassarre & Azizpour, 2019), Grad-CAM (Selvaraju et al., 2017), GNNExplainer (Ying et al., 2019), CXPlain (Schwab & Karlen, 2019), PGM-Explainer (Vu & Thai, 2020),
Screener (Anonymous, 2021), to generate the explanatory subgraphs.
Speciﬁcally, top- 15% , 20% , 20% of edges on the full graph instance construct the explanatory subgraphs in TR3, MNIST, and
Graph-SST2, respectively.
We refer readers to Appendix D for more experimental details.
4. S TUDY OF E XPLANATION E VALUATION (RQ1)
Deconfounded Evaluation Performance.
For an explanation G s , the conventional removal-based evaluation framework quantiﬁes its importance as the subgraph-prediction correlation, termed
Imp re ( G s ) =
f ( G s ) ; whereas, our DSE framework focuses on the causal effect caused by G s on Y which is computed based on Equation 1, and we denote it as Imp dse ( G s ) for short.
These importance scores broadly aim to reﬂect the discriminative information carried by G s .
Thanks to the ground-truth knowledge available in TR3 and MNIST sup , we are able to get a faithful and principled
[Figure]
[Table]
R s ↑ 0 .0.943 ∗ - − 0 .0.943 ∗ - 0.
0.714 ∗ - metric to measure the discriminative information amount — the precision Prec ( G s , G + s ) between the ground-truth explanation G + s and the explanatory subgraph G s .
This precision metric allows us to perform a fair comparison between Imp re ( G s ) and
Imp dse ( G s ) via:
[Equation]
where ρ is the correlation coefﬁcient between the lists of precision and importance scores.
We present the results in Figure 4 and have some interesting insights:
• Insight 1: Removal-based evaluation hardly reﬂects the importance of explanations.
In most cases, Prec ( G s , G + s ) is negatively correlated with the importance.
This again shows that simply discarding a part of a graph could violate some underlying properties of graphs and mislead the target GNN, which is consistent with the adversarial attack works (Dai et al., 2018; Zügner et al., 2018).
Moreover, the explainers that target high prediction accuracy, such as GNNExplainer, are easily distracted by the OOD effect and thus miss the important subgraphs.
• Insight 2: Deconfounded evaluation quantiﬁes the explanation importance more faithfully.
Substantially, ρ dse greatly improves after the frontdoor adjustments via the surrogate variable.
The most notable case is GNNExplainer in MNIST sup , where ρ dse = 0 .
17 achieves a tremendous increase from ρ dse = − 0 .
Although our DSE alleviates the OOD problem signiﬁcantly, weak positive or negative correlations still exist, which indicates the limitation of the current CVGAE.
We leave the exploration of higher-quality generation in future work.
Revisiting & Reranking Explainers.
Here we investigate the rankings of explainers generated from different evaluation frameworks, and further compute the Spearman rank correlations between these evaluation rankings and the reference rankings of explainers.
Speciﬁcally, for TR3 and MNIST sup with ground-truth explanations, we regard the ranks w.r.t.
precision as the references, while obtaining the reference of Graph-SST2 by a user study 2 .
Such a reference offers the human knowledge for explanations and benchmarks the comparison.
We show the results in Table 1 and conclude:
• Insight 3: DSE presents a more fair and reliable comparison among explainers.
The DSEbased rankings are highly consistent with the references, while the removal-based rankings struggle to pass the check.
In particular, we observe that for TR3, the unrealistic splicing inputs cause a plain ranking w.r.t.
Imp re .
We ﬁnd that various input subgraphs are predicted as cycle class.
That is, the target GNN model is a deterministic gambler with serious OOD subgraphs.
In contrast, DSE outputs a more informative ranking; For MNIST sup , GNNExplainer with the highest precision
[Table] [Table]
is overly underrated by the removal-based evaluation framework [Table] but DSE justiﬁes its position faithfully; For Graph-SST2, although the OOD problem seems to be minor, DSE can still achieve signiﬁcant improvement
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
Case Study.
We present a case study in Graph-SST2 to illustrate how DSE mitigates the potential OOD problem.
See Appendix F for another case study on T R 3.
In Figure 5, G is a graph predicted as “negative" sentiment.
The explanatory subgraph G s emphasizes tokens like “weak” and relations like “n’t → funny”, which is cogent according to human knowledge.
However, its removal-based importance is highly underestimated as 0 .
385 , possibly due to its disconnectivity or sparsity after feature removal.
To mitigate the OOD problem, DSE samples 50 surrogate graphs from the generator, performs the frontdoor adjustment, and justiﬁes the subgraph importance as 0 . 913 , which shows the effectiveness of our DSE framework.
We also observe some limitations of the generator (1) Due to the limited training data, the generators only reﬂect the distribution of the observed graphs, thus making some generations grammatically wrong.
The generations is constrained within the complete graph determined by the node set of the explanatory subgraph, thereby limits the quality of deconfounding.
As we mainly focus on the OOD problem, we will leave the ability of the generator as future work.
4. S TUDY OF G ENERATORS (RQ2)
The generator plays an important role in our DSE framework, which aims to generate the valid surrogates conform to the data distribution.
To evaluate the generator’s quality, we compare it with three baselines: a random generator, a variational graph auto-encoder (VGAE) (Thomas N. Kipf, 2016), and an adversarially regularized variational graph auto-encoder (ARGVA)
(Pan et al., 2018).
We perform the evaluation based on two metrics: (1) Validity.
For the ground-truth explanations G + s that contains all discriminative information of the full graph G , the importance of its surrogate graph G ∗ s should be higher than itself.
The difference between the two importance scores indicates the validity of the generator, thus we deﬁne VAL =
E G
[ Imp ( G ∗ s )
− Imp ( G + s )] .
For Graph-SST2 where the class-wise features are intractable, we leverage the embeddings of training graphs and additionally train a Gaussian Mixture Model (GMM) as our distribution prior.
Then, we compute the average loglikelihood of random subgraphs after in-ﬁlling, thus we have VAL =
E G E G s ∼ Random ( G ) [ GMM ( G ∗ s ) −
GMM ( G s )] .
(2) Fidelity.
Towards a ﬁner-grained assessment w.r.t.
prediction probability of any random subgraphs, we adopt the metric following (Frye et al., 2021): FID =
E G E G s E y |
f y ( G ) − E
G ∗ s [ f y ( G ∗ s )] | 2 .
This measures how well the surrogates cover the target prediction distribution.
Before comparing different generators, we ﬁrst compute the importance or probabilities of the graphs before and after feature removal, which are summarized in Table 2.
When inspecting the Removal’s results without any in-ﬁlls, the OOD problem is severe: in TR3 and MNIST sup , the importance of ground-truth subgraphs only reaches 43 .
8% and 40 .
8% , respectively, which are far away from the target importance of full graphs.
Analogously in Graph-SST2.
For the performance of the generators w.r.t.
the two metrics, we summarize the average results over 5 runs in Table 3:
• The performance of the baselines are poor.
This suggests that they can hardly ﬁt the target conditional distribution.
• CVGAE outperforms other generators consistently across all cases, thus justifying the rationale and effectiveness of our proposed generator and adversarial training paradigm.
For example, in TR3, CVGAE signiﬁcantly increases the VAL scores and mitigates the OOD effect effectively.
Moreover, we conduct ablation studies and sensitivity analysis in Appendix G to better understand the model components and validate the effectiveness of the designed objective.
R ELATED W ORK
Post-hoc Explainability of GNNs.
Inspired by the explainability in computer vision, Baldassarre & Azizpour (2019); Pope et al. ; Schnake et al. (2020) obtain the gradient-like scores of the model’s outcome or loss w.r.t.
the input features.
Another line (Luo et al., 2020;
Ying et al., 2019; Yuan et al., 2020a; Yue Zhang, 2020; Michael Sejr Schlichtkrull, 2021) learns the masks on graph features.
Typically, GNN-Explainer (Ying et al., 2019) applies the instance-wise masks on the messages carried by graph structures, and maximizes the mutual information between the masked graph and the prediction.
Going beyond the instance-wise explanation, PGExplainer (Luo et al., 2020) generates masks for multiple instances inductively.
Recently, researchers adopt the causal explainability (Pearl & Mackenzie, 2018) to uncover the causation of the model predictions.
For instance, CXPlain (Schwab & Karlen, 2019) quantiﬁes a feature’s importance by leaving it out.
PGM-Explainer (Vu & Thai, 2020) performs perturbations on graph structures and builds an Bayesian network upon the perturbation-prediction pairs.
Causal Screening (Screener) (Anonymous, 2021) measures the importance of an edge as its causal effect, conditional on the previously selected structures.
Lately, SubgraphX (Yuan et al., 2021) explores different subgraphs with Monte-Carlo tree search and evaluates subgraphs with the Shapley value (Kuhn & Tucker, 1953).
Counterfactual Generation for the OOD Problem.
The OOD effect of feature removal has been investigated in some other domains.
There are generally two classes of generation (i) Static generation.
For example, Fong & Vedaldi.
(2017); Dabkowski & Gal (2017) adopted blurred input and random colors for the image reference, respectively.
Due to the unnatural in-ﬁlling, the generated images are distributional irrespective and can still introduce confounding bias.
(ii) Adaptive generation: Chang et al.
; Frye et al.
(2021); Agarwal et al.
; Kim et al.
The generators of these methods, like DSE, overcomes the defects aforementioned, which generates data that conforms to the training distribution.
For example, in computer vision, FIDO (Chang et al., 2019) generates imagespeciﬁc explanations that respect the data distribution, answering “Which region, when replaced by plausible alternative values, would maximally change classiﬁer output?”.
For the difference, ﬁrstly, DSE’s formulated importance involves additional adjustment on G s and guarantees the unbiasedness of introducing the surrogate variable G ∗ s , which is commonly discarded by the prior works with in-ﬁllings only.
Speciﬁcally, we offer a comparison with FIDO in Appendix B.
Secondly, the distribution of graph data is more complicated to model than other domains.
And the proposed CVGAE is carefully designed for graph data, where the contrastive loss and the adversarial training framework are shown to be effective for learning the data distribution of graphs.
C ONCLUSION
In this work, we investigate the OOD effect on the explanation evaluation of GNNs.
With a causal view, we uncover the OOD effect — the distribution shift between full graphs and subgraphs, as the confounder between the explanatory subgraphs and the model prediction, making the evaluation less reliable.
To mitigate it, we propose a deconfounding evaluation framework that exploits the front-door adjustment to measure the causal effect of the explanatory subgraphs on the model prediction.
And a deep generative model is devised to achieve the front-door adjustment by generating in-distribution surrogates of the subgraphs.
In-so-doing, we can reliably evaluate the explanatory subgraphs.
As the evaluation for explanations fundamentally guides the objective in GNNs explainability, this work offers in-depth insights into the future interpretability systems.
E THICS S TATEMENT
This work raises concerns about the removal-based evaluation in the explainability literature and proposed Deconfounded Subgraph Evaluator.
For the user study that involves human subjects, we have detailed the fair evaluation procedure for each explanation generated by the explainers in
Appendix E. For real-world applications, we admitted that the modeling of the distribution shift could be a barrier to fulﬁll their evaluation faithfulness.
However, as shown in the paper, improper evaluation under the OOD setting largely biases the inspection of the model’s decision-making process and the quality of explainers.
Therefore, we argue that explainability should exhibit faithful explanation evaluation before auditing deep models’ actual decision-making process.
And a wrongly evaluated explanation might do more signiﬁcant harm than an incorrect prediction, as the former could affect the general adjustment ( e.g., structure construction) and human perspective ( e.g., fairness check) of the model.
R EPRODUCIBILITY S TATEMENT
We have made great efforts to ensure reproducibility in this paper.
Firstly, we make all causal assumptions clear in Section 2.2, Section 3.1 and Appendix A. For datasets, we have released the synthetic dataset, which can be referred to the link in Section 1, while the other two datasets are publicly available.
We also include our code for model construction in the link.
In Appendix D, we have reported the settings of hyper-parameters used in our implementation for model training.
A E LABORATION FOR OOD V ARIABLE
[Figure]
𝒀
[Figure]
Attributor
ModelPrediction 𝑮 𝑮 𝐬 𝑮 𝒀 𝑫
Distribution Shift Collective Representation
[Figure]
𝑮 𝐬 𝒀 𝑾 𝑫 𝒕𝒓 𝑨 𝑮
We offer a more ﬁne-grained introduction of variable D here.
We ﬁrst detail the data-generation process of model training, feature attribution and explanation evaluation of feature removal in Figure 6.
Next, we justify each process:
• Model Training: D tr → W , which represents that the weights of the trained GNN f is a random variable set by the optimizer, e
g., a SGD, from a ﬁnite training dataset starting from random weights.
• Feature Attribution: Taking a view of certain explainers (Baldassarre & Azizpour, 2019; Selvaraju et al., 2017), the logits or probability vector from the last layer of trained GNN is redistributed using weights of the network onto the input and it will highlight the relevant edges.
Thus, the causal relation W → A holds.
Moreover, for parameterized explainers (Luo et al., 2020) which optimizes on the masks of the input features, the attributors (explainers) also results from the training distribution, forming the causal relation D
tr
→ A .
Finally, with input graphs G , we have G → G s ← A where G s is the attribution from A ’s algorithmic functions on the input graphs.
• Explanation Evaluation of Feature Removal: traditional evaluation methods simply evaluate the faithfulness of explanation via model forward, i.e., ˆ Y =
f W ( G s ) and regard the softmax readout on target prediction as the subgraph importance.
For simplicity, we combine variables D tr , W and A collectively as an abstract distribution shift variable D which is unobservable from real data.
Thus, this justiﬁes the existence of D and our proposed SCM I.
B C OMPARISON OF I MPORTANCE E STIMATIONS
In this section, we compare our proposed estimation via front-door adjustment with the estimation in
FIDO (Chang et al., 2019).
We rephrased each estimation as
[Equation]
and
[Equation]
where DSE has alternatively adjusted on G s (represented as G (cid:48) s ).
To make it clear, we consider the underlined part of each equation.
For Equation 9, we have
[Equation]
While for the formulation of Equation 10, we have
[Equation]
In the comparison of these two parts, we can see that Equation 12 is biased under our causal assumption.
Intuitively, each contribution of the importance of G ∗ s on Y should be inversely proportional to the posterior probability, i.e., the probability of G (cid:48) s given the observation G ∗ s .
However, FIDO fails to consider the causal relation between G s → G ∗ s , which biases tha approximation of the genuine causal effect under our causal assumption.
Back to our proposed estimation, as we have collected ( G s , G ∗ s ) -pairs via Monte-Carlo simulation, thus additional adjustment on G s ( G (cid:48) s ) can be achieved via Equation 11.
C
[Equation]
FOR D ELETION - BASED E VALUATION
Based on the idea of deletion-based evaluation, we can instead use the average causal effect (Holland., 1988) (ACE) to look for a smallest deletion graph by conducting two interventions do ( G s = G ) ( i.e., , no feature removal) and do ( G s = G /s ) where G /s denotes the complement of the explanatory graph
G s , meaning that the GNN input receives treatment and control, respectively.
Formally, we have
[Equation]
Then, we can similarly adjust for the individual terms as Equation 1, obtaining the unbiased importance value as the result of deletion-based evaluation.
D E XPERIMENTAL D ETAILS
In this paper, all experiments are done on a single Tesla V100 SXM2 GPU (32 GB).
The well-trained
GNNs used in our experiments achieve high classiﬁcation accuracies of 0.958 in TR3, 0.982 in
MNIST sup , 0.909 in Graph-SST2.
Now We introduce the model construction of the proposed generator.
The encoder used is Crystal
Graph Convolutional Neural Networks (Xie & Grossman, 2018), which contains three Convolutional layers.
The encode dimensions in Tr3, MNIST sup , Graph-SST2 datasets are respectively 256, 64, 256.
For decoder, we adopt two fully connected layers with ReLU as activation layers, where the numbers of neurons are the same with the encode dimensions.
Next, we summarize the pseudocodes for the
Adversarial Training in Algorithm 1.
m =
= 2 ×
−
1 × 10 − λ τ
Require: P r , real graphs’ distribution.
r , masking ratio.
Require: m , batch size.
α , learning rate.
β, γ, λ, ω, τ , hyper-parameters.
1: µ ← µ 0 ; θ ← θ2: while loss in Equation (4) is not converged do 3:
# Discriminator’s training 4:
Sample {G ( i ) } mi =1 ∼ P r a batch from the real graphs.
Randomly generate broken graphs {G ( i ) s } mi =1 from {G ( i ) } mi =1 with masking ratio r .
Embed the nodes through encoder q ( Z |{G ( i ) s , G ( i ) } mi =1 ) 7:
Decode the edge probabilities and sample in-ﬁll graphs { ˆ G ¯ s } mi =1 ∼ p ( ˆ G ¯ s | Z ) 8:
Compute Discriminator’s loss from Equation 7.
Update parameter µ with back-propagation.
10: # Generator’s training 11:
Repeat the operations from line 4 to 7.
Compute Generator’s loss from Equation 4, 5, 6.
Update parameter θ with back-propagation.
14: end while
For other hyper-parameters, we set r = 0 .
3 , γ = 3 in Tr3 dataset.
In MNIST sup and Graph-SST datasets, we set r = 0 .
6 , γ = 1 .
We use Adam (Kingma & Ba, 2014) with weight decay rate 1e-5 for optimization.
The maximum number of epochs is 100.
E D ETAILED U SER S TUDY
The User Study starts by instructions to participants, where they will see a sentence (movie reviews) in each question and its sentiment (Positive of Negative), e.g.,
Sentence: “is more of an ordeal than an amusement”
Sentiment: Negative then several explanations are presented for the answers of “Why the sentiment of this sentence is negative (positive)?”.
The explanations (see Figure 7) are shown in graph form (edges indicate relations between words), and colors of more important features are darker.
Then they were asked to choose the best explanation(s).
A good explanation should be concise, informative, and the rational cause of sentence’s sentiment.
In this case, (B) could be the best explanation since “ordeal” mostly decides the negative sentiment, while (A) only identiﬁes plain words like “more than” and (C) is quite the opposite.
Note that the participants can choose multiple answers and some choices are the same.
Thereafter, 10 questions out of 32 questions in total are presented for each participant and we compute the average scores for the explainers.
F E XTRA C ASE S TUDY
In this section, we further present a case study for T R 3 dataset.
In Figure 8, the OOD probabilities for the ground truth explanatory subgraphs in each row remain the same as the edge selection ratios vary, which are 100%, 0%, 0% respectively.
In contrast, the evaluation results generated from our
DSE have shown strong rationality.
Speciﬁcally, the importance score compute by our DSE increases with the increasing number of selected ground truth edges.
This well validates our DSE framework, where we mitigate the OOD effect by generating the plausible surrogates, making the graphs to be evaluated conforms to the graph distribution in the training data.
In this way, the effect of D → Y could hardly affect our assessment for the explanatory subgraph.
Thereafter, as the explanatory graph becomes more informative and discriminative, it offers more evidence for the GNN to classify it as the target class which we want to explain, yielding faithful evaluation results.
[Figure]
G A BLATION S TUDY & S ENSITIVITY A NALYSIS
We ﬁrst conduct ablation studies to investigate the contribution of the contrastive parameter γ and the penalty parameter λ in CVGAE.
The ablation models are proposed by I. removing the contrastive loss, i.e., setting γ = 0 and II.
removing the penalty term in the Wasserstein GAN (WGAN) (Martin Arjovsky, 2017) loss, i.e., setting λ = 0 .
The performance of the ablation models is reported in Table 4.
We observe that the superiority of CVGAE compared with the ablation model supports our model design by (i) smoothing the model optimization which yields a more performant generator (ii) highlighting the class-discriminative information in the graph embeddings, which implicitly encodes the class information.
[Table]
Also, we conduct sensitivity analysis for CVGAE w.r.t.
the hyper-parameters.
Speciﬁcally, we select λ , the penalty in the WGAN loss ( cf.
Euqation 7) and γ , the strength of the contrastive loss ( cf.
Equation 4).
While we empirically found the performance is relatively indifferent to other parameters in a wide range.
The results are shown in Figure 9.
We observe that the best performance is achieved with λ taking values from 1 to 10 , and γ taking values from 1 to 10 in TR3 dataset and 0 .
1 to 5 in
MNIST sup and Graph-SST2 datasets.
And we found a large λ generally causes an increase in the
FID metric, as it may alleviate the penalty on the reconstruction errors, which further makes a larger difference between f y ( G ) and E [ f y ( G ∗ s )] .
[Figure]