Deconfounding to Explanation Evaluation in Graph Neural Networks
Authors: Yingxin Wu
Xiang Wang
An Zhang
Xia Hu
Fuli Feng
Xiangnan He
Tat-Seng Chua
Venue: ICLR 2022 Submitted (2022)
Keywords: Graph Neural Networks
Explanation Evaluation
Out-of-distribution
front-door Adjustment

Abstract: Explainability of graph neural networks (GNNs) aims to answer ``Why the GNN made a certain prediction?'', which is crucial to interpret the model prediction. The feature attribution framework distributes a GNN's prediction to its input features (e.g., edges), identifying an influential subgraph as the explanation. When evaluating the explanation (i.e., subgraph importance), a standard way is to audit the model prediction based on the subgraph solely. However, we argue that a distribution shift exists between the full graph and the subgraph, causing the out-of-distribution problem. Furthermore, with an in-depth causal analysis, we find the OOD effect acts as the confounder, which brings spurious associations between the subgraph importance and model prediction, making the evaluation less reliable. In this work, we propose Deconfounded Subgraph Evaluation (DSE) which assesses the causal effect of an explanatory subgraph on the model prediction. While the distribution shift is generally intractable, we employ the front-door adjustment and introduce a surrogate variable of the subgraphs. Specifically, we devise a generative model to generate the plausible surrogates that conform to the data distribution, thus approaching the unbiased estimation of subgraph importance. Empirical results demonstrate the effectiveness of DSE in terms of explanation fidelity.

Under review as a conference paper at ICLR 2022
DECONFOUNDING TO EXPLANATION EVALUATION IN GRAPH NEURAL NETWORKS
Anonymous authors Paper under double-blind review
ABSTRACT
Explainability of graph neural networks (GNNs) aims to answer "Why the GNN made a certain prediction?", which is crucial interpret the model prediction. The feature attribution framework distributes a GNN's prediction to its input features (e.g,, edges), identifying an influential subgraph as the explanation. When evaluating the explanation (i.e., subgraph importance). standard way is to audit the model prediction based on the subgraph solely. However, we argue that a distribution shift exists between the full graph and the subgraph, causing the out-of- distribution problem. Furthermore, with an in-depth causal analysis, we find the OOD effect acts the confounder, which brings spurious associations between the subgraph importance and model prediction, making the evaluation less reliable. this work, propose Deconfounded Subgraph Evaluation (DSE) which assesses the causal effect an explanatory subgraph on the model prediction. While the distribution shift is generally intractable, we employ the front-door adjustment and introduce a surrogate variable of the subgraphs. Specifically, we devise a generative model generate the plausible surrogates that conform to the data distribution, thus approaching the unbiased estimation of subgraph importance. Empirical results demonstrate the effectiveness of DSE in terms of explanation fidelity.
INTRODUCTION
Explainability of graph neural networks (GNNs) (Hamilton al., 2017; Dwivedi al., 2020) is crucial to model understanding and reliability in real-world applications, especially when about fairness and privacy (Ying et al., 2019; Luo et al., 2020). It aims to provide insight into how predictor models work, answering "Why the target GNN made certain prediction?". Towards this end, a variety of explainer models are proposed for feature attribution (Selvaraju et al., 2017; Ying et al., 2019; Luo al., 2020; Vu & Thai, 2020), which decomposes the predictor's prediction as contributions (i.e., importance) of its input features (e.g., edges, nodes). While feature attribution assigns the features with importance scores, it redistributes the graph features and creates new distribution different from that of the original full graphs, from which a subgraph is sampled as the explanation. Such sampling process referred as feature removal (Covert et al., 2020).
Then, to assess the explanatory subgraph, the current evaluation frameworks use the feature removal principle (1) (1) only feed the subgraph into the target predictor, discarding the other features; (2) measure the importance of the subgraph based on its information amount to recover the model's prediction. Such
correlations uncovered by the removal-based evaluator should offer faithful inspection of the predictor's 's decision-making process and assess the fidelity of the explainers reliably.
However, feature removal brings the cuttof-ddisributtin (OOD) problem (Frye et al., 2020; Chang et al., 2019; Lukas Faber, 2021): the distribution shift from full graphs to subgraphs likely violates underlying properties, including node degree distribution (Leskovec et al., 2005) and domain-specific constraints (Liu et al., 2018) the full graphs. For example, graph properties of chemical molecules, such the valency rules, impose hence, simply removing some bonds (edges) or atoms (nodes) creates invalid molecular subgraphs that never appear in the training dataset. Such OOD subgraphs could manipulate the predictor's
constraints syntactically valid molecules (Liu et al., 2018);

---

Under review as a conference paper at ICLR 2022
Crate
Huge
Cyrle
D
Distribution
5
Ss1
Ssz
Target Predictor
Gs
Y
Hous
Oyple
Coara
Graph
Gz: Subgraph
V: Predicate Logits
(a) Feature Removal to Evaluate Explanatory Subgraph G.
(h) SCMT
Figure 1: (a) A real example in TR3. The GNN predictor classifies the full graph as 'House". On subgraphs Gs1 and Gs2, the prediction probabilities of being "House" respectively 0.21 and 0.70. (b) The structural causal model represents the causalities among variables; G as the input graph, D the unobserved distribution shift, G2 as the explanatory subgraph, and Y as the model prediction.
outcome arbitrarily (Dai et al., 2018; Zügner et al., 2018), generates erroneous predictions, and limits the reliability of the evaluation process.
Here demonstrate the OOD effect by real example in Figure 1a, where the trained ASAP (Ranjan et al., 2020) predictor has classified the input graph as "House" for its attached motif (see Section for more details). On the ground-truth explanation G.1. the output probability of the "House" class is surprisingly low (0.21). While for G.2 with less discriminative information, the outputs probability of the "House" class (0.70) is higher. Clearly, the removal-based evaluator assigns the OOD subgraphs with unreliable importance scores, which are unfaithful to the predictor's 's decision.
The OOD effect has not been explored in evaluating GNN explanations, the best of our knowledge. We rigorously investigate it from causal view (Pearl et al., 2016; Pearl, 2000; Pearl & Mackenzie, 2018). Figure 1 represents our causal assumption via structural causal model (SCM) (Pearl et al., 2016; Pearl, 2000), where target the causal effect of G2 Y. Nonetheless, as confounder between G, and Y. distribution shift D opens the spurious path G, + D + Y. By "spurious", we mean that the path lies outside the direct causal path from C, to Y, making G, and Y spuriously correlated and yielding an erroneous effect. And one can hardly distinguish between the spurious correlation and causative relations (Pearl et al., 2016). Hence, auditing Y on G, suffers from the OOD effect and wrongly evaluates the importance of G..
Motivated by our causal insight, we propose novel evaluation paradigm, Deconfounded Subgraph Evaluator (DSE), to faithfully measure the causal effect of explanatory subgraphs on the prediction. Based on Figure 1b, the distribution shift D is hardly mea- surable, we cannot block the backdoor path from G, to Y by the backdoor adjustment. Thanks the front-door adjustment (Pearl et al., 2016), we instead consider the SCM in Figure 2, where we introduce the surrogate G between G, and Y, which "imagines" what the full graphs is like given the subgraphs. We ob- tain the causal effect of G. on Y by identifying the causal effects carried by G, G and G + Y, which requires G to respect the data distribution. Hence we design generative model, Con- ditional Variational Graph Auto-Encodas (CVGAE), to generate the possible surrogates. is worthwhile mentioning that our DSE explainer-agnosti which can assist the explanation evaluation reliably and further guide explainers to generate faithful explanations.
D
G
G
G
Front-door Adjustment
Figure 2: SCM with a mediat- ing variable G
In nutshell, our contributions are:
From causal perspective, we argue that the OOD effect is the confounder that causes spurious correlations between subgraph importance and model prediction.
We propose a deconfounding paradigm, DSE, which exploits the front-door adjustment to mitigate the -utto-ddistriuttion effect and evaluate the explanatory subgraphs unbiasedly.
We validate the effectiveness of our framework over various explainers, target GNN models, and datasets. Significant boosts are achieved over the conventional feature removal techniques. Code and datasets are available at: http://annnnnnooo.oo/oeeeeeee///
2

---

Under review as a conference paper at ICLR 2022
2 A CAUSAL VIEW OF EXPLANATION EVALUATION
Here we begin with the causality-based view of feature removal in Section 2.1 and present our causal assumption to inspect the OOD effect in Section 2.2.
2.1 PROBLEM FORMULATION
Without loss of generality, focus the graph classification task: well-trained GNN predictor takes the graph variable G as input and predicts the class Y E (1,..., k,,ie, Le,, Y f(G).
Generation of Explanatory Subgraphs. Post-hoc explainability typically considers the question "Why the GNN predictor f made certain prediction?". A prevalent solution is building an explainer model to conduct feature attribution (Ying et al., 2019; Luo al., 2020; Pope al., 2019). It decomposes the prediction into the contributions of the input features, which redistributes the probability of features according their importance and sample the salient features as an explanatory subgraph G.. Specifically, Go can be structure- wise (Ying et al., 2019; Luo et al.. 2020) or feature- wise (Ying et al., 2019) subgraph of G. In this paper, we focus on the structural features. That is, for graph G = (N, E) with the edge set & and the node set N,, the explanatory subgraph G3 &N,, consists of a subset of edges &, C & and their endpoints N₂ {u, el(uu, v) E.).
Evaluation of Explanatory Subgraphs. Insertion- based evaluation by feature removal (Covert et al., 2020; Dabkowski & Gal, 2017) aims to check whether the subgraph is the supporting substructure that alone allows a confident classification. We systematize this paradigm as three steps: (1) divide the full graph G into two parts, the subgraph 93 and the complement Gz; (2) feed Ge into the target GNN f, while discarding Gz; and (3) obtain the model prediction on G.. assess its discriminative information to recover the prediction on G. Briefly, at the core of the evaluator is the subgraph- prediction correlation. However, discussed in Section 1, the OOD effect is inherent in the nanoval-based evaluator, hindering the subgraph-predictt correlation from accurately estimating the subgraph importance.
2.2 STRUCTURAL CAUSAL MODEL
To inspect the OOD effect rigorously, we take causal look at the evaluation process with Structural Causal Model (SCM I) in Figure 1b. We denote the abstract data variables by the nodes, where the directed links represent the causality. The SCM indicates how the variables interact with each other through the graphical definition of causation:
G - G, < D. We introduce an abstract distribution shift variable D to sample a subgraph G, from the edge distributions of the full graph G.
G2 - by (1) the direct effect from G,, and (2) the confounding effect caused by D. In particular, the former causation that led to the result the focus of this work.
D. We denote Y as the prediction variable (e.g.. logits output), which is determined
We suggest readers to refer to Appendix where we offer an elaboration of D. With our SCM assumption, directly measuring the importance of explanatory subgraphs is distracted by the backdoor path (Pearl, 2000), G2 <- D Y. This path introduces the confounding associations between G, and Y. which makes G. and Y spuriously correlated, i.e., biases the subgraph-prediction correlations, thus making the evaluator invalid. How to mitigate the OOD effect and quantify G, 's genuine causal effect on Y remains largely unexplored in the literature and is the focus of our work.
3 DECONFOUNDED EVALUATION OF EXPLANATORY SUBGRAPHS
In this section, we propose novel deconfounding framework to evaluate the explanatory subgraphs in a trustworthy way. Specifically, we first leverage the front-door adjustment (Pearl, 2000) to formulate a causal objective Section 3.1. We then devise a conditional variational graph auto-encoders (CVGAE) as the effective implementation of our objective in Section 3.2.
We focus insertion-based evaluation here while discuss deletion-hased evaluation in Appendix C.
3

---

Under review as a conference paper at ICLR 2022
3.1 FRONT-DOOR ADJUSTMENT
To the best of our knowledge, our work is the first to adopt the causal theory to solve the OOD problem in the explanation evaluation of GNNs. To pursue the causal effect of G, on Y, we perform the calculus of the causal intervention P(Y = vyddo(G. = G.)). Specifically, the do-calculus (Pearl, 2000; Pearl et al., 2016) is to intervene the subgraph variable G. by cutting off its coming links and assigning it with the certain value G,, making it unaffected from its causal parents G and D. From inspection of the SCM in Figure 1b, the distribution effect D acts as the confounder between G, and Y, and opens the backdoor path G, + D + Y. However, as D is hardly measurable, we can not use the backdoor adjustment (Pearl, 2000; Pearl et al., 2016) to block the backdoor path from G, to Y. Hence, the causal effect of G, on Y is not identifiable from SCM
However, we can go much further by considering SCM in Figure instead, where a mediating variable G3 is introduced between Ge and Y:
G,+G. G. G is the surrogate variable of G,, which completes G, make them in the data distribution. First, it originates from and contains G,. Specifically, it imagines how the possible full graphs should when observing the subgraph G.. Second, G should follow the data distribution and respect the inherent knowledge of graph properties, thus no link exists between D and G..
G: Y. This is based on our causal assumption that the causality-related information of G, on iie,, the discriminative information for G2 to make prediction, is well-preserved by G Thus, with the core of Gs, G is qualified to serve as the mediator which further results in the model prediction.
With SCM II, we can exploit the front-door adjustment (Pearl, 2000; Pearl et al., 2016) instead to quantify the causal effect of G, on Y. Specifically, by summing over possible surrogate graphs 93 of G we chain two identifiable partial effects of G, on G and G. on Y together:
P(Y]do(G. = -.)) = Σ
P(Y]dolG; = -G:) 9 Go, = =.))
9:
P(Y = G2 G. P(G. = P(G; Gildo = -G))
9%
Σ
G(Y = G2 G. g. = P(G; = G. = Gz), (1)
Φ: %
Specifically, we have PoG)ldo(G, g.)) P(G G.) as G. is the only parent of G And we distinguish the G, in our target expression P(Y]do(G: = 9.)) between G. the latter of which adjusted to pursue Po = 9;)). With the data of (G.. pairs, we can obtain P(Y = - G. ()) by feeding the surrogate graph G: into the GNN predictor, conditional on the subgraph G': similarly, we can estimate P(G, = G%) statistically: P(G: = G; = 9.) is the conditional distribution of the surrogate variable, after observing the subgraphs. As result, this front-door adjustment yields consistent estimation of G, 's effect on Y and avoids the confounding associations from the OOD effect.
3.2 DEEP GENERATIVE MODEL.
However, it is non-trive trivirivil to instantiate G; and collect the (Ge, (G pairs. We get inspiration from the great success of generative models and devise novel probabilistic model, conditional variational graph auto-encoder (CVGAF), and an adversarial training framework, to generate G
Conditional Generation. Inspired by previous works (Thomas N. Kipf, 2016; Liu et al., 2018), we model the data distribution via generative model go parameterized by 0. It is composed of an encoder q (ZIG,,,) and a decoder p( Z). Specifically, the encoder g(Z G. G.) embeds each node i in G with a stochastic representation z, and summarize all node representations in Z:
N
q (ZG.9
II (z. (9..), with q(4, G.) = ~((z [116 Uzd.
(2)
where z; is sampled from a diagonal normal distribution by mean vector [M [2: and standard deviation vector diag(a?. o.):: f (r( and log क (o) denote the matrices of mean
4

---

Under review as a conference paper at ICLR 2022
Nodo Encodor
9 ( )
Edge emb.
P(E
Ss
Piee 0.99
Sample
AGG
Sample
y
5m
Figure 3: Model structure of CVGAE. Pase is the average probability of 9 on the target prediction. AGG indicates the representations of the end nodes are aggregated as the edge embeddings.
vectors μa and standard deviation vectors log ou respectively, which are derived from two GNN models fp and fo on the top of the full graph 9; similarly, μ2 (G.) and log fo()) are on the top of the subgraph Gs. Then, the decoder p(); generates the valid surrogates:
p Z
III ((..))...)) with p(A,) 1/z.,2,) = Za((z,,2)))
(3)
where A. indicates the existence of an edge between nodes and j: FA a MLP, which takes the concatenation of node representations z, and 2 as the input and outputs the probability of Aij = 1.
Leveraging the variational graph auto-encoder, we able to generate some counterfactual edges that never appear in G and sample 9 from the conditional distribution p Z), formally, 9; ~ p(G (Z). As a result, P generator directly affects the quality of the surrogate graphs, further determines how well the front- door adjustment is conducted. Next, we will detail an adversarial training framework optimize the generator, which is distinct from the standard training of VAE.
g
G.) in Equation is identified by P() The quality of the
Adversarial Training. To achieve high-quality generation, we get inspiration from the adversarial training (Goodfellow et al., 2020; Yue et al., 2021) and devise the following training objective:
min LVAe + 7CC + max WLD:
where y, w are trade-off hyper-parameters. These losses are carefully designed to assure the generation follows the data distribution. Next, we will elaborate on each of them.
LVvf - E&g (.og....)ooo [)] [DK. (aa..a/iaiZ ())))(
We first minimize the 3-VAE (Higggg al., 2017), and the first term is the reconstruction loss responsible to predict the probability of edges' existence; the second term is the KL-divergence between the variational and prior distributions. Here we resort to the isotropic Gaussian distribution p(Z) = II, p(z,) II, (2 0, I) as the prior. 33 reweighs the KL-divergence, which promises to learn the disentangled factors in (Higgins et al., 2017; Yue et al., 2021; Suter et al., 2019).
Moreover, we highlight the class-discriminative information in Z, by encouraging the agreement between graph representations with the same class compared to that with different classes. Technically, the contrastive loss is adopted:
Ev'e, exp (s(g,, 2g.
Le Egllog Egee exp (s(zg, zg") )/T)
(6)
where zg is the representation of that aggregates all node representations Z together; is the similarity function, which is given by an inner product here; 7 is the temperature syper-parameter B4 the graph set having the same class G. while the graphs involved in B__ have different classes from G. Minimizing this loss enables the generator to go beyond the generic knowledge and uncover the Mass-wise patterns of graph data.
Besides, we introduce discriminative model d,, to distinguish the generated graphs. Specifically, we set a babbabiliiy-condittinn GNN (Fey & Lenssen, 2019) parameterized by μ.. It takes graph as input and outputs score between 0 to 1, which indicates the confidence of the graph being realistic. Hence, given real graph with the ground-truth label y, we can use the generator go to generate G; Then the discriminator learns to assign G with large score while labeling G with small score. To optimize the discriminator, we adopt the Wasserstein GAN (WGAN) (Martin Arjovsky, 2017) loss:
Lo = [E 2) [d(G. v) d(g:.v) v) of 1)lk - 1)31},
(7)
5

---

Under review as a conference paper at ICLR 2022
where d(Gg, 3) is the probability of generating 93 from the generator; is the hyper-parameter By playing the min-mi max game between the generator and the discriminator in Equation 4, the generator can create the surrogate graphs from the data distribution plausibly.
Subgraph Evaluation. With the well-trained generator dá whose parameters are fixed, we now approximate the causal effect of G2 on Y. Here we conduct Monte-Carlo simulation based on ge to sample set of plausible surrogate graphs {9:} from p( Having collected the (? data, we can arrive the estimation of Equation 1.
4 EXPERIMENTS
We aim to answer the following research questions:
Study of Explanation Evaluation. How effective is our DSE in mitigating the OOD effect and evaluating the explanatory subgraph more reliably? (Section 4.2)
Study of Generator. How effective is our CVGAE generating the surrogates for the explanatory subgraphs and making them conform to the data distribution? (Section 4.3)
4.1
EXPERIMENTAL SETTINGS
Datasets & Target GNNs. We first train various target GNN classifiers on the three datasets:
TR3 is synthetic dataset involving 3000 graphs, each of which is constructed by connecting random tree-shape base with one motif (house, cycle, crane). The motif type is the ground-truth label, while we treat the motifs as the cround-truth explanations following Ying et (2019); Yuan et al. (2020a). A Local Extremum GNN (Ranjan et al., 2019) is trained for classification.
MNIST superpixels (MNIST (Monti et al., 2017) converts the MNIST images into 70,000 superpixel graphs. Every graph with 75 nodes is labeled as of 10 classes. We train Spline- based GNN (Fey et al., 2018) as the classifier model. The subgraphs representing digits can be viewed as human explanations.
Graph-SST2 (Yuan et al., 2020b) is based on text sentiment dataset SST2 (Socher et al., 2013) and converts the text sentences to graphs where nodes represent tokens and edges indicate relations between nodes. Each graph is labeled by its sentence sentiment. The node embeddings are initialized by the pre-trained BERT word embeddings (Devlin et al., 2018). Graph Attention Network (Veličković al., 2018) is trained the classifier.
Ground Brouhd-Truth Truth Explanations. By "ground-truth" we follow the prior studies (Ying et al., 2019; Yuan et al., 2020a; Luo et al., 2020) and treat the subgraphs coherent to the model knowledge (e.g.. the motif subgraphs in TR3) or human knowledge (e.g., the digit subgraphs in MNIST sap) as the ground-truth explanations. Although such ground-truth explanations might not fit the decision-making process the model exactly, they contain sufficient discriminative information to help justify the explanations. Note that no ground-truth explanation is available in Graph-SST2.
Explainers. To explain the decisions made by these GNNs, we adopt several state-of-the-art explain- ers, including SA (Baldassarre & Azizpour, 2019), Grad-CAM (Selvaraju et al., 2017), GNNEx- plainer (Ying al., 2019), CXPlain (Schwab & Karlen, 2019), PGM-Explainer (Vu & Thai, 2020), Screener (Anonymous, 2021), to generate the explanatory subgraphs. Specifically, top-15%, 20%, 20% of edges on the full graph instance construct the explanatory subgraphs in TR3, MNIST, and Graph-SST2, respectively. We refer readers to Appendix D for more experimental details.
4.2 STUDY OF EXPLANATION EVALUATION (RQ1)
Deconfounded Evaluation Performance. For an explanation G., the conventional removal-based evaluation framework quantifies its importance as the subgraph-predictton correlation, termed Imp,r (G.) = f(g.): whereas, our DSE framework focuses on the causal effect caused by G, on Y which is computed based on Equation 1, and denote it as Imp (G.) for short. These importance scores broadly aim to reflect the discriminative information carried by G.. Thanks to the ground-truth knowledge available in TR3 and MNIST supp we able to get faithful and principled

---

Under review as a conference paper at ICLR 2022
TR3
0.0
0.13
Avr
0.30
-0.4
0.05
0.00
Ave
-00
SA
CHRtspluteer
cxplain
wow/taploling
5A
Grad-CaM CHNc cumiai
POM-Rxplalnnt Screener
(b) MNIST ou Figure Validation of different frameworks for explanation evaluation.
(a) In TR3
Table 1: Evaluation of explainers under different evaluation frameworks. Rg is Spearman rank correlation function. Best explainers are underlined, Symbol (+) indicates the rank explainers.
TR3
MNIST
Graph-SST2
(16)
Impa.(%)
Prec
Imp..(i)
Impa.((5)
Prec
Imp...%)
Impo.(())
Score
SA Grad-CAM
43,23(4)
86.53 ()
17.60(9)
10.98(9)
32.98(2)
91.93
95.07(1)
4.48
33.07
43.18/4
75.07
16.90(9)
11.51
31.42
91. 91.94
96.21
6,21
GNNExplainer
(1.73(3
56.34(
17.00(4)
12
57.75
89.40(3)
95.20(
4.26(4)
CXPlain
(8.61(9)
34.38/6)
14.30'6)
10.78(P)
11.14(5)
92.40(2)
95.9g(3)
3.9g(5)
PGM-Explainer
33.07
39.58(3)
48.47'3)
22 (2.20(22)
10.77(6)
2.31
89.16(9)
95.45(()
1.68(6)
Screener
40.31
66.49
32,20(()
10.96(4)
19.51
96.04(4)
96.3g(()
6.42
R, 1
0.011
0.943
-0.142
0.943
0.657
0,714
metric measure the discriminative information amount the precision Prec 9 between the ground-truth explanation 9 and the explanatory subgraph G.. This precision metric allows us to perform fair comparison between (Gp) and Impase (G.) via:
Pre (rre 9 (Imp)((s))), Pdee p ((Prec(GSs 9, }}, (G.)))
(8)
where is the correlation coefficient between the lists of precision and importance scores, We present the results Figure 4 and have some interesting insights:
Insight 1: Removal- evaluation hardly reflects the importance of explanations. In most cases, Prec(Gs. 9 is negatively correlated with the importance. This again shows that simply discarding part of graph could violate some underlying properties of graphs and mislead the target GNN, which is consistent with the adversarial attack works (Dai et al., 2018; Zügner et al., 2018). Moreover, the explainers that target high prediction accuracy, such as GNNExplainer, easily distracted by the OOD effect and thus miss the important subgraphs.
Insight 2: Deconfounded evaluation quantifies the explanation importance more faithfully. Substantially, Pase greatly improves after the frontdoor adjustments via the surrogate variable. The most notable case is GNNExplainer in MNIST sup. where Pae = 0.17 achieves a tremendous increase from Pase 1..1 Although our DSE alleviates the OOD problem significantly, weak positive or negative correlations still exist, which indicates the limitation of the current CVGAE We leave the exploration of higher-quality generation in future work.
Revisiting & Reranking Explainers. Here we investigate the rankings explainers generated from different evaluation frameworks, and further compute the Spearman rank correlations between these evaluation rankings and the reference rankings of explainers. Specifically, for TR3 and MNIST,.. with ground-truth explanations, we regard the ranks w.r.f. precision the references, while obtaining the reference of Graph-SST2 by a user study². Such reference offers the human knowledge for explanations and benchmarks the comparison. We show the results Table and conclude:
Insight 3: DSE presents more fair and reliable comparison among explainers. The DSE- based rankings are highly consistent with the references, while the removal-basen rankings struggle to pass the check. In particular, we observe that for TR3, the unrealistic splicing inputs cause plain ranking w.t.t. Imp,-- We find that various input subgraphs are predicted as cycle class. That is, the target GNN model is deterministic gambler with serious OOD subgraphs. In contrast, DSE outputs a more informative ranking; For MNIST sup- GNNExplainer with the highest precision
270 volunteers engaged, where each was asked to answer 10 questions randomly sampled from 32 movie reviews and choose the hest explanations generated by the explainers. See Appendix E for more details.
7

---

Under review as a conference paper at ICLR 2022
Table 2: Importance scores or probabilities of subgraphs before and after feature removal.
TR3
MNIST
Graph-SS12
Imp( C) or GMM(C)
0.958
0.520
0,982
35.3-11.3 11.3
Imp(C2)) ) or GMM(G.)
0.438
0,408
24.0
Table 3: Performances of Generators in terms of Validity and Fidelity.
TR3
MNIST
Gruph-SST2
Imp(GT)
VAL.1
HID
Imp(C
VAL.t
F1D
GMM(G))
VAL.
FI1
Random
0.451
0.013
0.794
0.448
0.040
1.325
38.8
14.8
0.060
VGAE
0.469
0.031
0.754
0.205
-0.203
1.501
37.6
13.6
0.078
ARGVA
0.392
0.061
0.726
0.466
0.058
1.306
31.0
7.0
0.079
CYGAE
0.603
0.165
0.598
0.552
0.144
0.910
45.8
21.8
0.057
is overly underrated by the removal-based evaluation framework, but DSE justifies its position faithfully; For Graph-SST2, although the OOD problem seems to be minor, DSE can still achieve significant improvement.
Case Study. We present a case study in Graph-SST2 to illustrate how DSE mitigates the potential OOD problem. See Appendix F for another case study on TR3. In Figure 5, 9 is graph predicted as "negative" sentiment. The explanatory subgraph G. emphasizes tokens like "weak" and relations like "n't=>funny", which is cogent according to human knowledge. However, its removal-based impor- tance is highly underestimated 0.385, possibly due to its disconnectivity or sparsity after feature removal. To miti- gate the OOD problem, DSE samples 50 surrogate graphs from the generator, performs the frontdoor adjustment, and justifies the subgraph importance as 0.913, which shows the effectiveness of our DSE framework.
0. 3.5
P(M6J 0. 385
Explenation: butle weak
that funny
Sentimmt: Negalive
A DSE
Sumagate Samples
and
mny
Impsi,
-0.061
10
Impp._ -8.527
We also observe some limitations of the generator (1) Due to the limited training data, the generators only reflect the distribution of the observed graphs, thus making some generations grammatically wrong. (2) The generations is constrained within the complete graph determined by the node set of the explanatory subgraph, thereby limits the quality of deconfounding. As mainly focus the OOD problem, will leave the ability of the generator future work.
snerage
Impar
Figure 5: A Case Example.
4.3 STUDY OF GENERATORS (RQ2)
The generator plays an important role in our DSE framework, which aims to generate the valid surrogates conform to the data distribution. To evaluate the generator's quality, we compare it with three baselines: a random generator, a variational graph auto-encoder (VGAE) (Thomas N. Kipf, 2016), and an adversarially regularized variational graph auto-encoder (ARGVA) (Pan et al., 201 8). We perform the evaluation based on two metrics: (1) Validity. For the ground-tree -ruth explanations G that contains all discriminative information of the full graph G, the importance of its surrogate graph 9; should be higher than itself. The difference hetween the two importance scores indicates the validity of the generator, thus define VAL = Egg (g Imp(9 }}- For Graph-SST2 where the class-wise features are intractable, we leverage the embeddings of training graphs and additionally train Gaussian Mixture Model (GMM) as our distribution prior. Then, we compute the average log- likelihood of random subgraphs after in -iliiiing, thus we have VAL = Eg Eg, Randdm (G) (GM(())) GMM(G.)). (2) Fidelity. Towards a finer-grained assessment w.r.t. prediction probability of any random subgraphs, adopt the metric following (Frye et al., 2021): FID E.g E.g, Fyyl,, Eg; (G) (1))) This measures how well the surrogates cover the target prediction distribution.
Before comparing different generators, we first compute the importance or probabilities of the graphs before and after feature removal, which are summarized in Table 2. When inspecting the Removal's results without any in-fills, the OOD problem is severe: in TR3 and MNIST sup- the importance of ground-truth subgraphs only reaches 43.8% and 40.8%, respectivelyy, which are far away from the
8

---

Under review as a conference paper at ICLR 2022
target importance of full graphs. Analogously in Graph-SST2. For the performance of the generators w.E.. the two metrics, we summarize the average results over runs in Table 3:
The performance of the baselines are poor. This suggests that they can hardly fit the target conditional distribution.
CVGAE outperforms other generators consistently across all cases, thus justifying the rationale and effectiveness of our proposed generator and adversarial training paradigm. For example, in TR3, CVGAE significantly increases the VAL scores and mitigates the OOD effect effectively.
Moreover, we conduct ablation studies and sensitivity analysis in Appendix G to better understand the model components and validate the effectiveness of the designed objective.
5 RELATED WORK
Post-hoc Explainability of GNNs. Inspired by the explainability in computer vision, Baldassarre & Azizpour (2019); Pope et al. (2019); Schnake et al. (2020) obtain the gradient-like scores of the model's 's outcome or loss w.r.f. the input features. Another line (Luo et al., 2020; Ying et al., 2019; Yuan al., 2020a; Yue Zhang. 2020; Michael Sejr Schlichtkrull, 2021) learns the masks on graph features. Typically, GNN-Explainer (Ying et al., 2019) applies the instance-wise masks on the messages carried by graph structures, and maximizes the mutual information between the masked graph and the prediction. Going beyond the instance-wise explanation, PGExplainer (Luo et al., 2020) generates masks for multiple instances inductively. Recently, researchers adopt the causal explainability (Pearl & Mackenzie, 2018) to uncover the causation of the model predictions. For instance, CXPlain (Schwab & Karlen, 2019) quantifies a feature's 's importance by leaving it out. PGM-Explainer (Vu & Thai, 2020) performs perturbations on graph structures and builds Bayesian network upon the cnturraatioo-predictti measures the importance of an edge as its causal effect, conditional on the previously selected structures. Lately, SubgraphX (Yuan et al., 2021) explores different subgraphs with Monte-Carke tree search and evaluates subgraphs with the Shapley value (Kuhn & Tucker, 1953).
pairs. Causal Screening (Screener) (Anonymous, 2021)
Counterfactual Generation for the OOD Problem. The OOD effect of feature removal has been investigated in some other domains. There are generally two classes generation (i) Static generation. For example, Fong & Vedaldi. (2017); Dabkowski & Gal (2017) adopted blurred input and random colors for the image reference, respectively. Due to the unnatural in- in-filling, the generated images are distributional irrespective and can still introduce confounding bias. (ii) Adaptive generation: Chang et al. (2019); Frye et al. (2021); Agarwal et al. (2019); Kim et al. (2020). The generators these methods, like DSE, overcomes the defects aforementioned, which generates data that conforms to the training distribution. For example, in computer vision, FIDO (Chang et al., 2019) generates image- specific explanations that respect the data distribution, answering "Which region, when replaced by plausible alternative values, would maximally change classifier output?""
For the difference, firstly, DSE's formulated importance involves additional adjustment G, and guarantees the unbiasedness of introducing the surrogate variable G which is commonly discarded by the prior works with in-fillings only. Specifically, we offer comparison with FIDO in Appendix B. Secondly, the distribution of graph data is more complicated to model than other domains. And the proposed CVGAE carefully designed for graph data, where the contrastive loss and the adversarial training framework are shown to be effective for learning the data distribution of graphs.
6 CONCLUSION
In this work, we investigate the OOD effect on the explanation evaluation of GNNs. With a causal view, we uncover the OOD effect the distribution shift between full graphs and subgraphs, as the confounder between the explanatory subgraphs and the model prediction, making the evaluation less reliable. To mitigate it, we propose a deconfounding evaluation framework that exploits the front-door adjustment to measure the causal effect of the explanatory subgraphs on the model prediction. And deep generative model is devised to achieve the front-door adjustment by generating in nn-distribution reliably evaluate the explanatory subgraphs. As the
surrogates of the subgraphs. In In-so-doing, evaluation for explanations fundamentally guides the objective in GNNs explainability, this work offers in-depth insights into the future interpretability systems.

---

Under review as a conference paper at ICLR 2022
ETHICS STATEMENT
This work raises concerns about the removal-based evaluation in the explainability literature and proposed Deconfounded Subgraph Evaluator. For the user study that involves human subjects, have detailed the fair evaluation procedure for each explanation generated by the explainers in Appendix E. For real-world applications, we admitted that the modeling of the distribution shift could be a barrier to fulfill their evaluation faithfulness. However, as shown in the paper, improper evaluation under the OOD setting largely biases the inspection process and the quality of explainers. Therefore, we argue that explainability should exhibit faithful explanation evaluation before auditing deep models' actual decision- makinn process. And wrongly evaluated explanation might do more significant harm than an incorrect prediction, as the former could affect the general adjustment (e.g. structure construction) and human perspective (e.g. fairness check) of the model.
the model's decision-makinn
REPRODUCIBILITY STATEMENT
We have made great efforts to ensure reproducibility in this paper. Firstly, we make all causal assumptions clear in Section 2.2, Section 3.1 and Appendix A. For datasets, we have released the synthetic dataset, which can be referred to the link in Section 1, while the other two datasets are publicly available. We also include our code for model construction in the link. In Appendix D, we have reported the settings of hyper-parameters used in our implementation for model training.
REFERENCES
Chirag Agarwal, Dan Schonfeld, and Anh Nguyen. Removing input features via a generative model to explain their attributions classifier's decisions. CoRR, 2019.
Anonymous. Causal screening to interpret graph neural networks. Submitted to ICLR. https: httperriieeeeefffoommdddnd/////a
2021.
Federico Baldassarre and Hossein Azizpour. Explainability techniques for graph convolutional networks. CORR, abs//905. 13686, 2019.
Chun-Hao Chang, Elliot Creager, Anna Goldenberg, and David Duvenaud. Explaining image classifiers by counterfactual generation. In ICLR, 2019.
Ian Covert, Scott Lundberg, and Su-In Lee. Feature removal is a unifying principle for model explanation methods. NeurlPS, 2020.
Piotr Dabkowski and Yarin Gal. Real time image saliency for black box classifiers. In NeurIPS, pp. 6967-6976, 2017.
Hanjun Dai, Hui Li, Tian Tian, Xin Huang, Lin Wang, Jun Zhu, and Le Song. Adversarial attack on graph structured data. In ICML. pp. 1123-1132, 2018.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:/810.04880 2018.
Vijay Prakash Dwivedi, Chaitanya K. Joshi, Thomas Laurent, Yoshua Bengio, and Xavier Bresson. Benchmarking graph neural networks. CoRR, abs/2003.00982, 2020.
Matthias Fey and Jan F. Lenssen. Fast graph representation learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds, 2019.
Matthias Fey, Jan Eric Lenssen, Frank Weichert, and Heinrich Müller. Splinecnn: Fast geometric deep learning with continuous b-spline kernels. In CVPR, pp. 869-877, 2018.
R. Fong and A. Vedaldi. Interpretable explanations of black boxes by meaningful perturbation. In ICCV, 2017.
Christopher Frye, Damien de Mijolla, Laurence Cowton. Megan Stanley, and Ilya Feige. Shapley- based explainability the data manifold. CoRR, abs/2006.01172, 2020.
10

---

Under review as a conference paper at ICLR 2022
Christopher Frye, Damien de Mijolla, Laurence Cowton, Megan Stanley, and Ilya Feige. Shapley- based explainability on the data manifold. In ICLR, 2021.
Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. Generative adversarial networks. Commun, ACM, 63(11); 139-144, 2020.
William L. Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In NeurIPS, pp. 1024-1034, 2017.
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. bet:-vae: Learning basic visual concepts with constrained variational framework. In ICLR, 2017.
P.W. Holland. Causal inference, path analysis, and recursive structural equations models. C. Clogg, editor, Sociological Methodology, pages 449-4844 American Sociological Association, Washington, D.C., 1988.
Siwon Kim, Jihun Yi, Eunji Kim, and Sungroh Yoon. Interpretation of NLP models through input marginalization. In EMNLP, pp. 3154-3167, 2020.
Diederik P Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint orXiv:1412.6600 2014.
H. W. Kuhn and A. W. Tucker. Contributions to the theory of games, volume 2. Princeton University Press, 1953.
Jure Leskovec, Jon Kleinberg, and Christos Faloutsos. Graphs over time: densification laws, shrinking diameters and possible explanations. KDD, pp. 177-187, 2005.
Qi Liu, Miltiadis Allamanis, Mare Brockschmidt, and Alexander L. Gaunt. Constrained graph variational autoencoders for molecule design. In NeurIPS, pp. 7806-7815, 2018.
Roger Wattenhofer Lukas Faber, Amin K. Moghaddam. Contrastive graph neural network explanation. In ICLR Workshop on Representation Learning. 2021.
Dongsheng Luo, Wei Cheng, Dongkuan Xu, Wenchao Yu, Bo Zong, Haifeng Chen, and Xiang Zhang. Parameterized explainer for graph neural network. In NeurIPS, 2020.
Léon Bottou Martin Arjovsky, Soumith Chintala. Wasserstein generative adversarial networks. In ICML, 2017.
Ivan Titov Michael Sejr Schlichtkrull, Nicola De Cao. Interpreting graph neural networks for nlp with differentiable edge masking. In ICLR, 2021.
Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodolà, Jan Svoboda, and Michael M. Bronstein. Geometrie deep learning graphs and manifolds using mixture model cnns. In CVPR, pp. 5425-5434, 2017.
Shirui Pan, Ruiqi Hu, Guodong Long, Jing Jiang, Lina Yao, and Chengqi Zhang. Adversarially regularized graph autoencoder for graph embedding. In IJCAI, 2018.
Judea Pearl. Causality: Models, Reasoning, and Inference. 2000.
Judea Pearl and Dana Mackenzie. The book of why: the new science of cause and effect. Basic Books, 2018.
Judea Pearl, Madelyn Glymour, and Nicholas Jewell. Causal inference in statisties: A primer. John Wiley & Sons, 2016.
Phillip E. Pope, Soheil Kolouri, Mohammad Rostami, Charles E. Martin, and Heiko Hoffmann. Explainability methods for graph convolutional neural networks. CVPR, pp. 10772-10781, 2019.
11

---

Under review as a conference paper at ICLR 2022
Ekagra Ranjan, Soumya Sanyal, and Partha Pratim Talukdar. ASAP: Adaptive structure aware pooling for learning hierarchical graph representations. arXiv preprint arXiv:1911.07979. 2019.
Ekagra Ranjan, Soumya Sanyal, and Partha P. Talukdar. ASAP: adaptive structure aware pooling for learning hierarchical graph representations. In AAAI, pp. 5470-5477, 2020.
Thomas Schnake, Oliver Eberle, Jonas Lederer, Shinichi Nakajima, K. T. Schutt, Klaus-Rober Muller, and Grégoire Montavon. Higher-order explanations of graph neural networks via relevant walks. arXiv, 2020.
Patrick Schwab and Walter Karlen. Cxplain: Causal explanations for model interpretation under uncertainty. In NeurlPS, pp. 10220-10230, 2019.
Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localiza- tion. In ICCV, pp. 618-626, 2017.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher Manning, Andrew Y. Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In EMNLP, pp. 1631-1642, 2013.
Raphael Suter, Dorde Miladinovic, Bernhard Schölkopf, and Stefan Bauer. Robustly disentangled causal mechanisms: Validating deep representations for interventional robustness. In ICML, volume 97, pp. 6056-6065, 2019.
Max Welling Thomas N. Kipf. Variational graph auto-encoders. In NeurlPS Workshops, 2016.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. ICLR, 2018. accepted as poster.
Minh N. Vu and My T. Thai. Pgm-explainer: Probabilistic graphical model explanations for graph neural networks. In NeurlPS, 2020.
Tian Xie and Jeffrey C. Grossman. Crystal graph convolutional neural networks for an accurate and interpretable prediction of material properties. Phys. Rev. Lett., 120:145301, Apr 2018.
Zhitao Ying, Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec. Gnnexplainer:
Generating explanations for graph neural networks. NeurlPS, pp. 9240-9251, 2019.
Hao Yuan, Jiliang Tang, Xia Hu, and Shuiwang Ji. XGNN: towards model-level explanations of graph neural networks. In Rajesh Gupta, Yan Liu, Jiliang Tang, and B. Aditya Prakash (eds.), KDD, pp. 430-438, 2020a.
Hao Yuan, Haiyang Yu, Shurui Gui, and Shuiwang Ji. Explainability in graph neural networks: A (axonomic survey. CoRR, 2020b.
Hao Yuan, Haiyang Yu, Jie Wang, Kang Li, and Shuiwang Ji. On explainability of graph neural networks via subgraph explorations. ArXiv, 2021.
Zhongqi Yue, Tan Wang, Hanwang Zhang, Qianru Sun, and Xian-Sheng Hua. Counterfactual zero-shot and open-set visual recognition. In CVPR, 2021.
Arti Ramesh Yue Zhang, David Defazio. Relex: A model-agnostic relational model explainer. arXiv preprint arXiv:2006.00305. 2020.
Daniel Zügner, Amir Akbarnejad, and Stephan Günnemann. Adversarial attacks on neural networks for graph data. In KDD, pp. 2856,2856, 2018.
12

---

Under review as a conference paper at ICLR 2022
A
ELABORATION FOR OOD VARIABLE
Training Oistibuttion
Model Weight
W
Da
W
Collective Representation
Distribution Shift
Actributor
6
G
G
Gg
Input Cruph
Explerattory
Model
Figure Elaboration for variable D in Figure 1b, where the definitions the variables are annotated. The blue, green, red arrows represent the data-generation process of model training, explaining, and removal-bart evaluation, respectively; the dashed arrow --a denotes optional causal relation.
We offer more fine- grei-grridd introduction of variable D here. We first detail the data-generin generation process of model training. feature attribution and explanation evaluation feature removal in Figure Next, we justify each process:
Model Training: Drr + W. which represents that the weights of the trained GNN f is a random variable set by the optimizer, e,g., a SGD, from a finite training dataset starting from random weights.
Taking a view of certain explainers (Baldassarre & Azizpour, 2019; Selvaraju et al., 2017), the logits probability vector from the last layer of trained GNN is redistributed using weights of the network onto the input and it will highlight the relevant edges. Thus, the causal relation W > A holds. Moreover, for parameterized explainers (Luo et al., 2020) which optimizes the masks of the input features, the attributors (explainers) also results from the training distribution, forming the causal relation Dtr + A. Finally, with input graphs G, we have G + G. A where G, is the attribution from A's algorithmic functions on the input graphs.
Feature Attribution:
Explanation Evaluation of Feature Removal: traditional evaluation methods simply evaluate the faithfulness of explanation via model forward, i.e., y fw (G,) and regard the softmax readout on target prediction as the subgraph importance.
For simplicity, we combine variables Dtr; W and A collectively as an abstract distribution shift variable D which is unobservable from real data. Thus, this justifies the existence of D and our proposed SCM
B COMPARISON OF IMPORTANCE ESTIMATIONS
In this section, compare our proposed estimation via front-door adjustment with the estimation in FIDO (Chang et al., 2019). We rephrased each estimation as
Impase (G.) =
Σ P = G. = 9.) G -==-")
5
(9)
P(G: = 9 G, G.) Σ g
P(Y (GG =
and
Imprip((9, Σ
P G G, = 9.) P(Y = =
(10)
9;
13

---

Under review as a conference paper at ICLR 2022
where DSE has alternatively adjusted on G, (represented G,).. To make clear, consider the underlined part of each equation. For Equation 9, we have
P G. 9 G. - (.) P(G. -
P(G, 9
EP G. = G. ). P PG, = G3 G; 93)
P(G, 91 G* 9:)
(II)
PG, = 9.)
P.Y.G, = g. G. = g.)
P = 9 G = 93)
While for the formulation of Equation 10, we have
P( G 9,) Σ P(Y,G, G3 I - 93) 9
(12)
In the comparison these two parts, we can see that Equation is biased under our causal assump- tion. Intuitively, each contribution of the importance of G on Y should be inversely proportional to the posterior probability, i.e., the probability gú given the observation g:, However, FIDO fails to consider the causal relation between G2 G which biases tha approximation of the genuine causal effect under our causal assumption.
Back proposed estimation, as have collected (9,; -paiirs via Monte-Carlo simulation, thus additional adjustment on Gs (9") can be achieved via Equation 11.
C DSE FOR DELETION-BASED EVALUATION
Based the idea of deletion- based evaluation, can instead use the average causal effect (Holland., 1988) (ACE) to look for smallest deletion graph by conducting two interventions do(G. = G) (i.e., feature removal) and do(G. G/a) where 9/s denotes the complement of the explanatory graph G. meaning that the GNN input receives treatment and control, respectively. Formally, we have
Impdce (G, = - do (G, = 9)) - P(Y do (G, = =91s)
(13)
Then, we can similarly adjust for the individual terms as Equation 1, obtaining the unbiased impor- tance value as the result of deletion-based evaluation.
D EXPERIMENTAL DETAILS
In this paper, all experiments are done on a single Tesla V100 SXM2 GPU (32 GB). The well-trained GNNs used in our experiments achieve high classification accuracies of 0.958 in TR3, 0.982 in MNIST sup 0.909 in Graph-SST2.
Now We introduce the model construction of the proposed generator. The encoder used is Crystal Graph Convolutional Neural Networks (Xie & Grossman, 2018), which contains three Convolutional layers. The encode dimensions Tr3, MNIST Graph-SST2 datasets are respectively 256, 64, 256. For decoder. we adopt two fully connected layers with ReLU as activation layers, where the numbers of neurons are the same with the encode dimensions. Next, we summarize the pseudocodes for the Adversarial Training in Algorithm 1.
14

---

Under review as a conference paper at ICLR 2022
Algorithm I Generative Adversarial Training. All experiments in the paper used the default values x 10
m 256, a
10 À 5, = 0.1
Require: Py, real graphs' distribution. r, masking ratio.
Require: m1, batch size. learning rate. 8, W, hyper-parameters.
1: / t- /4;; 4- On
2: while loss in Equation (4) is not converged do
3: # Discriminator's training
4: Sample 1m P, batch from the real graphs.
5: Randomly generate broken graphs (g. ==1 from {g with masking ratio
6:
Embed the nodes through encoder (Z [g.h G pm ===1)
7:
Decode the edge probabilities and sample in-fill graphs {6:) ~ p(C's Z) Compute Discriminator's 's loss from Equation 7.
8:
9:
Update parameter y with back-propag
10:
# Generator's s training
11:
Repeat the operations from line to
12:
Compute Generator's 's loss from Equation 4, 5, 6.
13:
Update parameter with back poeckkopppggtion.
14:
end while
For other hyper-parameters, we set r = 0.3, in Tr3 dataset. In MNIST sup and Graph-SST2 datasets, we set r 0.6, y = 1. We use Adam (Kingma Ba, 2014) with weight decay rate le-5 for optimization. The maximum number of epochs is 100.
E DETAILED USER STUDY
The User Study starts by instructions to participants, where they will see sentence (movie reviews) in each question and its sentiment (Positive of Negative), e.g.,
Sentence: "is more an ordeal than an amusement"
Sentiment: Negative
then several explanations are presented for the answers "Why the sentiment of this sentence negative (positive)?".. The explanations (see Figure 7) are shown in graph form (edges indicate relations between words), and colors of more important features are darker.
Ordeal
an
more
than
amusemen
more
an
of
ordeal
more
Figure 7: Instruction Example for conducting the user study.
Then they were asked to choose the best explanation(s). A good explanation should concise, informative, and the rational cause of sentence's sentiment. In this case, (B) could be the best explanation since "ordeal" mostly decides the negative sentiment, while (A) only identifies plain words like "more than" and (C) is quite the opposite. Note that the participants answers and some choices are the same. Thereafter, 10 questions out of 32 questions total are presented for each participant and compute the average scores for the explainers.
choose multiple
15

---

Under review as a conference paper at ICLR 2022
F EXTRA CASE STUDY
In this section, we further present case study for TR3 dataset. In Figure 8, the OOD probabilities for the ground truth explanatory subgraphs in each row remain the same as the edge selection ratios vary, which are 100%, 0%, 0% respectively. In contrast, the evaluation results generated from our DSE have shown strong rationality. Specifically, the importance score compute by our DSE increases with the increasing number of selected ground truth edges. This well validates our DSE framework, where we mitigate the OOD effect by generating the plausible surrogates, making the graphs to be evaluated conforms to the graph distribution in the training data. In this way, the effect of D - Y could hardly affect our assessment for the explanatory subgraph. Thereafter, as the explanatory graph becomes more informative and discriminative, it offers more evidence for the GNN to classify it as the target class which we want to explain, yielding faithful evaluation results.
Cycle
077
Casa Cooea
House
0.4
6.2
Crane
03
Selection
Figure 8: Three cases in TR3 datasets. Each graph in the left represents the ground truth explanatory subgraphs (red) for explaining a given graph. One of the complement graphs (light blue) generated from CVGAE is also shown with each explanatory subgraph. As the edge selection ratio increases in each row, the importance scores output by our DSE are shown in the right.
G ABLATION STUDY & SENSITIVITY ANALYSIS
We first conduct ablation studies to investigate the contribution of the contrastive parameter and the penalty parameter in CVGAE, The ablation models are proposed by removing the contrastive loss, i.e., setting 7 0 and II. removing the penalty term in the Wasserstein GAN (WGAN) (Martin Arjovsky, 2017) loss, i.e., setting À The performance of the ablation models is reported in Table 4. We observe that the superiority of CVGAE compared with the ablation model supports our model design by (i) smoothing the model optimization which yields a more performant generator (ii) highlighting the class-discimminatie information in the graph embeddings, which implicitly encodes the class information.
Table 4: Ablation study proposed CVGAE.
Ablation Models
TR3
MNIST
Sap
Graph-SST2
VAL
FID4
VAL
FID
VALf
FID|
I. Remove Le
0.068 0.035
0.643
0.038
1.314
21.4
0.065
II. Remove the penalty in Lp
0.739
0.078
1.139
11.3
0.083
CVGAE
0.165*
0.598
0.144
0.910
21.8"
0.057*
Also, we conduct sensitivity analysis for CVGAE w.e.t. the X, the penalty in the WGAN loss (ef. Euqation 7) and the strength of the contrastive loss (cf. Equation 4). While we empirically found the performance relatively indifferent to other parameters in wide range. The results are shown in Figure 9. We observe that the best performance is achieved with À taking values from 1 to 10, and r taking values from to 10 in TR3 dataset and 0.1 to 5 in MNIST sup and Graph-SST2 datasets. And we found a large À generally causes an increase in the FID metric, as may alleviate the penalty on the reconstruction errors, which further makes a larger difference between (9) and (f
Specifically, we select
16

---

Under review as a conference paper at ICLR 2022
Y
y
3
10
D.1
0.1
5
0.167
0.091
0.150
0.306
0.336
0.129
0.111
0.156
0.150
2
4328
4115
10
0.149
0.110
0.118
0.1.9
0.88
0.680
0.697
1.061
0.964
1.270
123
0.061
0.064
0,070
1.086
0.910
3:170
0.086
0.057
0.070
0.971
0.903
1. 090
1.270
10
0.083
639
TR3
MNIST sup
Graph-SST2
Figure 9; The performance of CVGAE using different À and y values.
17


Reviews:
[{"review_id": "tCVMBCKm0aV", "reviewer": "Reviewer_pL1y", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "As I have stated in summary, the paper has caught an interesting problem with current explanation evaluation methods. This aspect is a strength! Another strength is that they cast the problem into a causal framework making the reasoning behind the novel evaluation mechanism reasonably interpretable.\n\nThe main issue with the paper lies in its clarity. It is not challenging to follow the technical details but rather to understand what the authors want to do. For instance, the authors title Section 2.1 as \"Problem Formulation\" but I do not see any problem formulated in that section.\nAlso, the evaluation method requires using a generative model to generate enough samples to be able to apply Equation 1.\n\nFor the above reasons, I would like authors to clearly state:\n1. Whenever a new model for explaining GNNs is developed, should one also generate graphs using CVGAE?\n2. How can one tell the superiority of the novel evaluation method from your paper? Where is exactly that assessed and proved? I tried to grasp it from reading the article, but I've not been able to.\n3. Can you please clearly state what is the impact of the generative model on your evaluation? The two baselines are very weak, in my opinion. One is random, OK we should always include random baselines. The other one, though, is not conditional, which makes only explicit that a conditional generator is better. But I would have been surprised to find out that this was not the case. So, the question is: what are other baselines that would truly show the impact of the generator? Would it be sufficient to use whatever conditional method to generate graphs?\n\n\n====== Minor Concerns ======\n1. with less discriminative information --> Define \"less discriminative information\"\n2. G_s is defined in the caption of Figure 1 but not in the text and when you first use it, is difficult to follow the sentence.\n3. what the full graph like --> what the full graph is like\n4. It harms the removal-based evaluation of the explanatory subgraph --> I've read this sentence over and over again but I've not been able to understand what you actually meant\n5. well-trained GNN predictor --> What \"well-trained\" means?\n6. it is rooted from G_s --> it is rooted on (?) G_s.  (What does actually mean this sentence?)\n7. Equation (2) the product should have i = 1 and not just i\n8. auto-encoder is able to generate --> autoencoder we are able to generate\n9. the formula  that comes after formally does not look correct (or I've not understood it)\n10. Besides, we introduce ... generated graphs --> Cannot understand it\n11. Equation equation 1 (in a couple of places)\n\n\n\nAuthors have addressed most of my concerns, except for the #1, which is still not 100% clear. After the rebuttal, I've decided to raise my score.", "summary_of_the_paper": "The paper sheds an exciting light on the problem of producing a meaningful evaluation of GNN explanation methods (at least a subset of them).\nThe idea is to introduce a deconfounder D to capture the effect of OOD explanations. The authors make an interesting example for a well-known synthetic dataset where the weight of the explanation in the ground truth is lower than a clear non-valid explanation when evaluated using the model to explain.\nThe introduction of the deconfounder D creates a spurious path between the graph variable and the explainer variable. To mitigate this effect, then, they introduce a front-door adjustment to the causal graph.\nThe front door adjustment requires a graph generator and authors use a novel Conditional-VGAE to generate graphs that will also cover the OOD case.\nThe paper finally presents some experiments showing the evaluation method in action.", "summary_of_the_review": "In summary, I like the idea but I believe the paper does not make a good job of conveying the message to the readers.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "WoozYTF_C0", "reviewer": "Reviewer_eQDt", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper presents an interesting new method with a clear focus on debiasing GNN-explainer subgraph importance scores by generating surrogate subgraphs to correct distribution shift problems. The paper is well written and easy to follow, and the core idea sounds very effective and the empirical study using three datasets provides useful demonstrations.\n\nOverall I liked the idea of the paper and found it nice work. Here are a couple of small questions to make sure of the paper's contribution.\n\n- The proposed method is GNN-explainer agnostic. This point is advantageous because we can make importance corrections to any GNN explainers we like. But at the same time, a natural question will be: Is there any chance that we miss important features because the method to generate subgraph patterns didn't consider this OOD problem even if the proposed method can make a correction by posthoc processing. So for example, is it possible to have situations we have no significant important subgraph patterns after OOD-bias correction? Or, either way, any subgraph patterns come from at least one of the data graphs, and so by recovering such data graph within the training distribution, we can approximately resolve the data distribution bias and there are technically no problems??\n\n- The paper focuses on structural features. This will imply that the bias under consideration is primarily distribution shift due to taking subgraphs of data graphs, and we need to recover the original data graph in the training set from a given any subgraph patterns. \nBut relatively small subgraph patterns can occur in multiple instances, and does this generative step actually generate such surrogate graph as intended? \nSay, let G_s be a subgraph pattern we want to calculate the importance score but falls in out of the training distribution. The surrogate graph G_s^* by CVGAE is basically intended to recover the original super graph of G_s in the training dataset, isn't it? It'll be very helpful to make sure this point in some way. \n\n- Just for your interest, and no need to include this in the paper: it might be interesting to consider whether the bias from subgraphs to the data graphs is the main problem. If the generative step can be explicit, we can even have a deterministic mapping from the possible subgraph patterns to the original graphs by graph mining algorithms. Explicit subgraph patterns are intensively investigated in the graph mining field in parallel to GNN-based approaches to the graph classification problem. For example, the book \"Managing and Mining Graph Data (Ed: Charu C. Aggarwal & Haixun Wang) covers a list of relevant papers. So we can directly search explicit subgraph patterns in the data graphs and such approaches are intensively investigated around 10 years ago.  \n  - by direct graph mining such as mining \"discriminative patterns\", \"emerging patterns\", \"contrastive patterns\", (many works such as Llinares-López+, Fast and memory-efficient significant pattern mining via permutation testing, KDD2015)\n  - by LASSO (K. Tsuda, Entire regularization paths for graph data. ICML 2007: 919-926)\n  - by feature-wise boosting (Saigo+, gBoost: a mathematical programming approach to graph classification and regression, Machine Learning, 2009)\n  - by sparse coordinate descent (Takigawa+, Generalized sparse learning of linear models over the complete subgraph feature set. TPAMI 2017)\n  - by decision tree / decision forest (Shirakawa+, Jointly learning relevant subgraph patterns and nonlinear models of their indicators. MLG2018@KDD)  \nBut using these methods, we often see disappointing conclusions that in general, \"smaller subgraph patterns\" are important because smaller subgraphs are more frequently occurring in the data and thus can be contributing in making predictions. \n", "summary_of_the_paper": "This paper presents a novel explainer-agnostic method to adjust the biases of feature importance scores of feature attribution for GNNs. The paper first describes the feature importance scores of the GNN feature attribution framework have biases due to the out-of-distribution (OOD) problem. The subgraph important scores are calculated by inputting a subgraph instead of data graphs, but subgraph patterns can fall into regions outside the distribution of training data graphs. To address this problem, the paper proposed a method to generate surrogate graphs within the data graph distribution by CVGAE to make a front-door adjustment for deconfounding these biases by distribution shift. Experiments using several state-of-the-art GNN explainers shows demonstrated the effectiveness of the proposed framework.\n", "summary_of_the_review": "The paper is well written, easy to follow, and presents a very useful explainer-agostic method to correct the OOD bias of subgraph important scores by GNN explainers. Small questions are made to make sure\n1. Even this explainer-agnostic method can correct the OOD biases, the original GNN explainers didn't take into account those biases. Does this cause any problem?\n2. What the surrogate graphs are like? They are recovered original graphs containing the given subgraphs?\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "S9jjB2yecD", "reviewer": "Reviewer_eWjy", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper proposes a surrogate variable $G_{s}^*$ to denote the out-of-distribution effect and seems find interesting ways to evaluate the causal effects between the subgraph and full graph.\n\n*Strengths*:\n\n1. the out-of-distribution has not been explored before, as the paper claims.\n2. the conditional variational graph autoencoder is well proposed and well trained. \n3. I will the experimental settings, especially about the 3 insights. These results have sufficiently verify the claims and advantages of the paper. \n\n*Weaknesses*\nI did find clear weaknesses. ", "summary_of_the_paper": "This paper has done an excellent work of finding the out-of-distribution between the subgraph and graph as the confounder. Further, this paper proposes a conditional variational graph auto-encoder in assessing the causal effects of subgraph on the prediction. They also introduce a surrogate variable to denote this out-of-distribution effect. Through adversarial training, the effects of the proposed model is correctly verified.  ", "summary_of_the_review": "Based on the innovation, clear model description and solid experimental results, I recommend for accept. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "3c1R2hHSQA", "reviewer": "Reviewer_Tnp6", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\nIn general, the motivation of this paper is very clear. The paper is easy to follow. View GNNs from a causal perspective is not a new idea. However, this paper uses causal theory to investigate the OOD problem in DNNs is new and interesting.\n\nWeaknesses:\n\nMy major concerns are as follows,\n\n1. In Section 3.1, the authors mention 'our work is the first to adopt the causal theory to \\textbf{solve} the OOD problem ...'. However, I think this paper is a causal view on the OOD problem in DNNs instead of solving the OOD problem in DNNs. Especially, at the beginning of Section 4, the authors also highlight that they study the explanation evaluation and the generator, which are verified my opinion. \n\n2. In the adversarial training part, there are several hyper-parameters such as $\\gamma$, $\\omega$, $\\tau$, $\\lambda$, and $\\beta$. However, the authors do not provide any sensitivity analysis about these hyper-parameters. On the other hand, since there are three losses in Eq.(4), the authors should do several Ablation studies to demonstrate the significance of each loss function. \n\n3. In Insight 3 of Section 4.2, the authors mention 'The DSE-based rankings are highly consistent with the references,'. However, it is not clear what is the references and where can we get these references? In specific, how to get the values of Prec column and values of Score column in Table 1.\n\n4. Why the VAL performance of VGAE on MNIST$_{sup}$ is a negative value (-0.203)?\n\n5. Since the work of Causal Screening (Screener) [1] is very close to this paper, the authors should discuss more differences between this paper and paper [1] instead of mentioning it slightly in the Related Work.\n\n[1] Wang, Xiang, et al. \"Causal Screening to Interpret Graph Neural Networks.\" (2020).\n\nMy minor concerns:\n\n1. Some notations are not clear and there are some typos. \na) In Eq. (1), what is the meaning of \\textbf{do}?\nb) Below Eq.(3), 'Equation equation 1' should be 'Equation 1'.\nc) In Figure 3, what is 'AGG'?", "summary_of_the_paper": "In this paper, the authors use a causal view to investigate the OOD effect on the explanation evaluation of GNNs. They find the confounder between the extracted subgraphs and the model prediction, which makes the evaluation less reliable. To solve this problem, the authors proposed a deconfounding evaluation method based on the front-door adjustment from causal discovery. To generate a reliable surrogate subgraph, they proposed a generative model, which contains three losses for training. The experimental results show the effectiveness of the proposed method (DSE).", "summary_of_the_review": "From my perspective, this paper is interesting in some aspects. The causal view on the OOD problem in DNNs is a new idea.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper recieves extensive discussions among SAC, two ACs and PCs. The decision was not made lightly. We hope that you will find the comments below  from two ACs useful for future publication. 
----------------------------------------------------------------
This paper is concerned with the feature attribution framework, which distributes the prediction made by a Graph Neural Network (GNN) to its input features, such as edges or nodes, and identifies an influential subgraph as the explanation. The currently prevailing feature removal strategy, which feeds only the considered subgraph into the target predictor and then measures the importance of the subgraph, will encounter the so-called Out-Of-Distribution (OOD) problem--the new subgraphs may not appear in the training dataset.

This paper proposes to use the causal inference framework to deal with this OOD problem. The proposal seems interesting: it considered the considered subgraph as the cause of the prediction and treats domain shift as a (hidden) common cause for both of them. It proposes to estimate a surrogate graph G_s^* to satisfy the front-door criterion and then estimate the causal effect of the subgraph on the prediction.

While the proposed method seems novel and interesting and the reported empirical results seem encouraging, I have some basic concerns about the proposal.
1. The authors didn't justify why the feature attribution evaluation problem can/should be considered as a causal effect identification problem. In feature attribution evaluation, one is essentially interested in evaluating the prediction given the subgraph. The distribution shift variable, D, contains information that is helpful for this purpose.  Why should one go with the causal effect identification formulation, in which D is made independent from the subgraph and then integrated out? I think this justification is essential.

2. It is not justified why the constructed variable, G_s^*, satisfies the front-door criterion. In Section 3.1, the authors claimed that "G_s^* should follow the data distribution and respect the inherent knowledge of graph properties, thus no link exists between D and G_s^*."  I failed to see why this implies that there is no link from D to G_s^* (or that D and G_s^* are conditionally independent given G_s)--note that D is part of the data distribution. Without this justification, I am not sure whether the application of front-door adjustment is sensible.

Overall, the paper contains interesting ideas and the results look encouraging. It would be highly appreciated if the authors managed to make this work convincing, by properly addressing the issues above.  I hope to see those components in an updated paper.
------------------------------------------------------

This work considers the task of debiasing GNN-explainer subgraph importance scores by generating surrogate subgraphs to correct distribution shift problems. Through a causal model, the authors argue that a model prediction based on the explanatory subgraphs suffers from a distribution shift (e.g., induced subgraph degree distributions are different than those of the full graph). Hence, the associations between the important induced subgraphs and the model prediction may be spurious. In particular, the casting of induced subgraph explanations as a front-door hidden variable should be definitively valuable to the community (but maybe a little too strong of a condition, seems unnecessary for the proposed goal).

Overall the reviewers believe the goal and observations are novel and valuable. Most of the reviewers' concerns were addressed in the rebuttal. As a pure data-driven domain adaptation work, I think the work is good. However, even after discussing with the authors, I am still concerned with the soundness of the causal theory behind the work.

A Conditional Variational Graph Auto-Encoder (CVGAE) are tasked to produce subgraphs that act as a front-door adjustment variable. The argument is that this front-door adjustment solves the challenges of the distribution shift. Front-door adjustment is generally performed over observed variables. If performed with a model, the model is likely mechanistic. CVGAE are extremely flexible models for a front-door adjustment. And while the generated subgraphs are constrained to reproduce a data-driven distribution of induced subgraphs (and that those themselves have constraints), that in itself is not enough to guarantee these generated graphs give a proper front-door adjustment. The work also describes "generate counterfactual edges" without a clear causal model for how these edges are generated or why they are counterfactual. The causal theory needs a lot of work.

The work is very promising and may become a cornerstone contribution to graph explanations. However, as it stands now, the causal theory needs to be more formal (the work offers no proofs of the various claims). I am excited to see the causal theory fully developed in the future.
Decision: Reject