Discovering Invariant Rationales for Graph Neural Networks
Authors: Yingxin Wu
Xiang Wang
An Zhang
Xiangnan He
Tat-Seng Chua
Venue: ICLR 2022 Poster (2022)
Keywords: Interpretability
Graph Neural Networks
Causal Discovery
Invariant Learning

Abstract: Intrinsic interpretability of graph neural networks (GNNs) is to find a small subset of the input graph's features --- rationale --- which guides the model prediction. Unfortunately, the leading rationalization models often rely on data biases, especially shortcut features, to compose rationales and make predictions without probing the critical and causal patterns. Moreover, such data biases easily change outside the training distribution. As a result, these models suffer from a huge drop in interpretability and predictive performance on out-of-distribution data. In this work, we propose a new strategy of discovering invariant rationale (DIR) to construct intrinsically interpretable GNNs. It conducts interventions on the training distribution to create multiple interventional distributions. Then it approaches the causal rationales that are invariant across different distributions while filtering out the spurious patterns that are unstable. Experiments on both synthetic and real-world datasets validate the superiority of our DIR in terms of interpretability and generalization ability on graph classification over the leading baselines. Code and datasets are available at https://github.com/Wuyxin/DIR-GNN.

Published as a conference paper at ICLR 2022
DISCOVERING INVARIANT RATIONALES FOR GRAPH NEURAL NETWORKS
Ying-Xin Wu', Xiang Wang"1, An Zhang*, Xiangnan Hei, Tat-Seng Chua
f University of Science and Technology of China
$ National University of Singapore (wuyxinsh, xiangwang1222 22 gmail com, an_zhangênus. eew.igeunuuuuuu...... edu. sg, ziangnanhe@gmale com, dcsctsênus. edu. .g
ABSTRACT
Intrinsic interpretability of graph neural networks (GNNs) is find small subset of the input graph's 's features rationale - which guides the model prediction. Unfortunately, the leading rationalization models often rely on data biases, es- pecially shortcut features, to compose rationales and make predictions without probing the critical and causal patterns. Moreover, such data biases easily change outside the training distribution. As a result, these models suffer from huge drop in interpretability and predictive performance on out-of-distribution data. In this work, we propose new strategy of discovering invariant rationale (DIR) to con- struct intrinsically interpretable GNNs. conducts interventions on the training distribution to create multiple interventional distributions. Then it approaches the causal rationales that are invariant across different distributions while filtering out the spurious patterns that unstable. Experiments on both synthetic and real- world datasets validate the superiority of our DIR terms of interpretability and generalization ability on graph classification over the leading baselines. Code and datasets are available at wttp:/hhhw...ooooooooi/iii----e
INTRODUCTION
The eye-catching success in graph neural networks (GNNs) (Hamilton et al., 2017; Kipf & Welling, 2017: Dwivedi et al., 2020) provokes the rationalization task, answering "What knowledge drives the model make certain predictions?". The goal of selective rationalization (aka. feature attribu- tion) (Chang et al., 2020; Ying et al., 2019; Luo et al., 2020; Wang et al., 2021c) is to find small subset of the input's graph features rationale - which best guides or explains the model predic- tion. Discovering the rationale in a model helps audit its inner workings and justify its predictions. Moreover, has tremendous impacts on real-world applications, such as finding functional groups to shed light on protein structure prediction (Senior et al., 2020).
Two research lines of rationalization have recently emerged in GNNs. Post-hoe explainability (Ying et al., 2019; Luo et al., 2020; Yuan et al., 2021; Wang et al., 2021c) attributes a model's prediction to the input graph with separate explanation method, while intrinsic interpretabil- ity (Veličković et al., 2018; Gao & Ji, 2019) incorporates a rationaliza- tion module into the model to make transparent predictions. Here focus on intrinsically interpretable GNNs. Among them, graph attention (Veličković al., 2018) and pooling (Lee et al., 2019; Knyazev et al., 2019: Gao & Ji, 2019; Ranjan et al., 2020) operators prevail, which work as computational block of a GNN generate soft or hard masks on the input graph. They cast the learning paradigm of GNN minimizing the empirical risk with the masked subgraphs, which are regarded as ratio- nales to guide the model predictions.
Frequenccy
90%
Tree
Ludder
Bases
of House Motif
Figure 1: Base Distribu- tion House Motif.
Despite the appealing nature, recent studies (Chang et al., 2020; Knyazev et al., 2019) show that the current rationalization methods are prone to
"Corresponding author.

---

Published as a conference paper at ICLR 2022
exploit data biases as shortcuts to make predictions and compose rationales. Typically, shortcuts result from confounding factors, sampling biases, and artifacts in the training data. Considering Figure I, when the most bases of House-motif graphs are Tree, GNN does not need to learn the correct function to reach high accuracy for the motif type. Instead, it is much easier to learn from the statistical shorteuts linking the bases Tree with the most occurring motifs House. Unfortunately, when facing with out-of-distribution (OOD) data, such methods generalize poorly since the shortcuts are changed. Hence, such shortcut-involved rationales hardly reveal the truly critical subgraphs for the predicted labels, being at odds with the true reasoning process that underlies the task of interest (Teney et al., 2020) and human cognition (Alvarez-Melis & Jaakkola, 2017).
Here we ascribe the failure on OOD data to the inability to identify causal patterns, which are stable to distribution shift. Motivated by recent studies on invariant learning (II.) (Arjovsky et al., 2019; Krueger et al., 2021; Chang et al., 2020; Bühlmann, 2018), we premise different distributions elicit different environments of data generating process. We argue that the causal patterns to the labels remain stable across environments, while the relations between the shortcut patterns and the labels vary. Such environment-invenn
patterns are more plausible and qualified as rationales.
Aiming to identify rationales that capture the environment-invariant causal patterns, we formalize learning strategy, Discovering Invariant Rationales (DIR), for intrinsically interpretable GNNs. One major problem how to get multiple environments from standard training set. Differing from the heterogeneous setting (Bühlmann, 2018) of existing II. methods, where environments are observable and attainable, DIR does not assume prophets about environments. It instead generates distribution perturbations by causal intervention - interventional distributions (Tian et al., 2006; Pearl et al., 2016) to instantiate environments and further distinguish the causal and non-causal parts.
Guided by this idea, our DIR strategy consists of four modules: a rationale generator, a distribu- tion intervener, feature encoder, two classifiers. Specifically, the rationale generator learns to split the input graph into causal and non-causal subgraphs, which are respectively encoded by the en- coder into representations. Then, the distribution intervener conducts the causal interventions on the non-causal representations to create perturbed distributions, with which we can infer the invariant causal parts. Then, the two classifiers are respectively built upon the causal and non-causal parts to generate the joint prediction, whose invariant risk is minimized across different distributions. On one synthetic and three real datasets, extensive experiments demonstrate the generalization ability of DIR to surpass current state-of oh-the-art IL methods (Arjovsky et al., 2019; Krueger et al., 2021; Sagawa et al., 2019), and the interpretability of DIR to outperform the attention- and pooling-base based rationalization methods (Veličković et al., 2018; Gao & Ji, 2019). Our main contributions are:
We propose novel invariant learning algorithm, DIR, for inherent interpretable models, improv- ing the generalization ability and is suitable for any deep models.
We offer causality theoretic analysis to guarantee the preeminence of DIR.
We provide the implementation of DIR for graph classification tasks, which consistently achieves excellent performance on three datasets with various generalization types.
2 INVARIANT RATIONALE DISCOVERY
With a causal look at the data- teita--eneratngg process, we formalize the principle of discovering invariant rationales, which guides our discovery strategy. Throughout the paper, upper-cased letters like G denote random variables, while lower-case letters like g denote deterministic value of variables.
2.1 CAUSAL VIEW OF DATA-GENERATING PROCESS
Generating rationales for transparent predictions requires understanding the actual mechanisms of the task of interest. Without loss of generality, we focus on the graph classification task and present causal view of the data-generating process behind this task. Here we formalize the causal view as a Structure Causal Model (SCM) (Pearl et al., 2016; Pearl, 2000) by inspecting on the causali- ties among four variables: input graph G, ground-truth label Y, causal part C, non-causal part S. Figure 2a illustrates the SCM, where each link denotes a causal relationship between two variables.
C + G < S. The input graph G consists of two disjoint parts: the causal part C and the non-causal part S, such as the House motif and the Tree base in Figure 1.
2

---

Published as a conference paper at ICLR 2022
Original Distribution
S
Multiple s-interventional
do(S 0)
G
do(S=
Y
do(s
(a) SCM
(b) Interventional Distributions.
Figure 2: (a) Causal view of data-generating process: (b) Illustration of interventional distributions.
C C. By "causal part", we mean C' is the only endogenous parent to determine the ground- truth label Y. Taking the motif-base example in Figure again, C' is the oracle rationale, which perfectly explains why the graph is labeled as Y.
C' + S. This dashed arrow indicates additional probabilistic dependencies (Pearl, 2000; Pearl et al., 2016) between C and S. We consider three typical relationships here: (1) C' is independent of S. i.e., C S: (2) C' is the direct cause of S, i.e. C - S; and (3) There exists common cause i... C E + S. See Appendix for the corresponding examples.
C' Ce---- S can create spurious correlations between the non-causal part S and the ground-truth S, C is a confounder between and Y. which opens a backdoor path C Y. thus making S and Y spuriously correlated (Pearl al., 2016). We systematize such spurious correlations as Y 1k S. Wherein, we make feature induction assumption on to avoid the confusion of the induced subset of S between C. See Appendix C for the formal assumption. Furthermore, data collected from different environments exhibit various spurious correlations (Teney et al., 2020; Arjovsky et al., 2019), e.g., one mostly picks House motifs with Tree bases as the training data, while another selects House motifs with Wheel bases as the testing data. Hence, such spurious correlations unstable and variant across different distributions.
label Y. Assuming
2.2 TASK FORMALIZATION INVARIANT RATIONALIZAAION
Oracle Rationale. With the causal theory (Pearl et al., 2016; Pearl, 2000), for each variable X in SCM, there exists directed link from each of its parent variables PA(X) to X, and only if the causal mechanism X fx(P (x(AAXX (P((X), (x) persists, where 6x JL. P PA(X) is the exogenous noise of X. For simplicity, we omit the exogenous noise and simplify it as = fx( (P)())) Hence, there exist a function fY C Y in our SCM, where the "oracle rationale" C' satisfies:
= (C),,, SIS
(1)
where YIL S C' indicates that C' shields Y from the influence of S, making the causal relationship C+> Y invariant across different 5.
Rationalization. In general, only the pairs of input G and label Y are observed during training. while neither oracle rationale C' nor oracle structural equation model fy is available. The absence of oracles calls for the study on intrinsic interpretability. We systematize an intrinsically-interreeee
GNN combination of two modules, i.e., h hy he where he G + Ö discovers rationale Č from the observed G, and hy : Ō + Y outputs the prediction Y to approach Y. Distinct from C' and Y which are the variables in the causal mechanisms, Ö and Ý represent the variables in the modeling process to approximate C' and Y. To optimize these modules, most of current intrinsically- interpretable GNNs (Veličković et al., 2018; Lee et al., 2019; Knyazev et al., 2019; Gao & Ji, 2019; Ranjan et al., 2020) adopt the learning strategy of minimizing the empirical risk:
min R(h o hy(G), Y), hg.
where R(.,)) is the risk function, which can be the cross-entropy loss. Neverthcless, this learning strategy relies heavily on the statistical associations between the input features and labels, and can potentially exhibit non-causal rationales.
3

---

Published as a conference paper at ICLR 2022
Invariant Rationalization. We ascribe the limitation to ignoring Y TTL SSS C in Equation which crucial to refine the causal relationship C + Y that is invariant across different S. By introducing this independence, we formalize the task of invariant rationalization as:
min R(h o he(G), Y), s.t. Y $ Ó Ag.
(3)
where 5 = G \ Ö is the complement of Č. This formulation encourages the rationale C seeking the patterns that are stable across different distributions, while discarding the unstable patterns.
2.3 PRINCIPLE & LEARNING STRATEGY OF DIR
Interventional Distribution. However, is difficult to recover the oracle rationale from the joint distribution over the inputs and labels - that is, the causal and non-causal relations are hardly distinguished from each other. We get inspirations from invariant learning (Arjovsky et al., 2019; Krueger et al., 2021; Chang et al., 2020), which constructs different environments to infer the invari- ant features or predictors. To obtain the environments, previous studies mostly partition the training set by prior knowledge (Teney al., 2020) adversarial environment inference (Creager et al., 2021; Wang et al., 2021b). Different from partitioning the training data, we do not assume prophets about environments but introduce the interventional distribution (Tian al., 2006; Pearl al., 2016) instead to model the DIR task. Specifically, on the top of our SCM, we generate s-interventionll dis- tribution by doing intervention do(S s) on S, which removes every link from the parents PA(S) to the variable S and fixes S to the specific value By stratifying different values S = {s} we can obtain multiple s-interventional distributions.
With interventional distributions, we propose the principle of discovering invariant rationale (DIR) to identify rationale Ö whose relationship with the label Y is stable across different distributions.
Definition (DIR Principle) An
model h satisfies the DIR principle if
Y do = +})}, and simultaneously 2. minimizes the variance of various s-interventional risks: Var, ((RM(() Y)do(S =})]])
1. minimizes all s-interventional risks:
where the s-interventional risk is defined over the s-interventional distribution for specific E S.
Guided by the proposed principle, we design the learning strategy of DIR as:
min RDI
()) s))] + AVar, AVar,, (IR(h( =))}},
(4)
where R (N(G), Y do(S = x)) computes the risk under the s-interventional distribution, which we will elaborate in Section 2.4. Var(-) calculates the variance of risks over different s-interventional
distributions; À is hyper-parameter to control the strength of invariant learning.
Justification. We theoretically justify the DIR principle's 's ability discover invariant rationales. Specifically, Theorem shows that the oracle model fy respects the DIR principle. Moreover, we suggest that C can be inferred by making the intrinsically interpretable model conform to the DIR principle under the uniqueness condition (cf. Corollary 1). We leave the detailed proofs in Appendix risks indifferent while pursuing low
C due to the limited space. By making the distribution-relevwa risks, the DIR principle able discover the invariant rationales Ö' the approximation of the oracle rationales C, while encouraging hy approaching the oracle model fy-
2.4 DIR-GUIDED IMPLEMENTATION OF INTRUNSICALLY--ITERRREENNN
GNNs
With the DIR principle and objective, we present how to implement the intrinsically-inerrrraabee GNNs. We summarize the key notations of this section in Appendix A for clarity. Following Equa- tion 2, model h with intrinsic interpretability consists of two modules: = he o hgg where he to extract possible rationale, and he is to make prediction based the rationale. Moreover, to establish the s-interventional distributions, we design an additional module to do the interventions. In nutshell, our framework consists of four components, as Figure 3 shows.
Rationale Generator. It aims to split the input graph instance into two subgraphs: causal part ē and non-causal part s. Specifically, given an input graph instance g (V,E) with the node set V
4

---

Published as a conference paper at ICLR 2022
Distribution Intervener
Y:
R3
No BP
Rationale
Shared Graph Encoder
Shortcut Classifler
y
Generator
Causal Classifier
No
RDIR
Invariant Rationale
Causal Prediction
Figure 3: DIR Implementation GNNs, which includes a rationale generator, distribution inter- vener, an encoder and two classifiers. For the inference, we only use Da as the prediction.
and the edge set E, its adjacency matrix is A E {0, 11IIXMM, where A.j 1 denotes the edge from node node j, and A.j otherwise. The rationale generator first adopts GNN to generate the mask matrix M E R on A, where mask Mij indicates the importance of edge A,y:
Z GNN1 (g); Mi = (Z,
(5)
where (.) the sigmoid function and Z E Vi summarizes the d-dimensional representations of all nodes. The generator then selects the edges with the highest masks construct the rationale 7 and collects ess complement as 8, as follows:
& Top, (M @ A), & = To1 (001-Meas A);
where & and Eg are the edge sets of ē and 8, respectively; (op-(-) selects the top-K edges with K (8), and r is the hyper-parameter (e.g., 40%); is the element-wise product. Having obtained the edge sets, we can distill the nodes appearing in the edges to establish ē and 8.
Distribution Intervener. It targets creating interventional distributions. Formally, it first collects the non-causal part of all the instances into memory bank as S. It next samples memory & $ to conduct the intervention d5 = st), replacing the complement of the critical subgraph éS at hand and constructing an intervened pair (čj; 9,), where i, are indices.
Graph Encoder & Classifiers. Here we represent hy as a combination of graph encoder and two classifiers. Specifically, it employs another GNN encoder on to generate node representations ZE R and then combines them graph representation H; E RDD via glohal pooling operator, e.g., average pooling. Then it uses a classifier & to project the graph representation into probability distribution over class labels da- More formally, the process is as follows:
Z: = GNN2(2), H Pooling(Zaa
ja = t5 (Ha).
Analogously, we can obtain Úa for é via the shared encoder and another classifier T... Da is the pre- diction based merely on the causal part é, while ÿs measures the predictive power of the intervened part S. Inspired by Cadène al. (2019), formulate the joint prediction ý under the intervention do(S = 3) as fa masked by Ds:
ja (())).
where the sigmoid function adjusts the output logits of to compensate for the spurious biases. In Appendix E, present examples of how this operation helps discover the causal part.
Optimization. Having established the prediction ý of an instance y under the intervention do(S = 8), we capable of getting the s-interventional risk similar Equation 4 as follows:
R(((G), (oo( = 8)) = - seteooss=,.co. (= y),
where (g.y) E O is a pair of graph instance and its ground-truth label y from the training set O; 1(-) denotes the loss function on single instance. Moreover, define the loss for &, module
R. [(u:))e0
(10)
Specifically, R. is only backpropagated to the classifier T, and we apart the other components from its backpropagation to avoid interference with representation learning. Thus, this loss promotes the S-only branch to learn spurious biases given the non-causal features only. Overall, we can jointly optimize these components via the DIR objective and shorteut loss, i.e.,
5

---

message passing scheme which enriches Published as a conference paper at ICLR 2022
min d. Rg + min y,0,6. RDIR-
(11)
where 0 and (Øe, o.) are the parameters of the generator, encoder and two classifiers. While in the inference phase, we yield č and de as the causal rationale and the causal prediction of testing graph 9, which exclude the influence of the non-causal part 3.
3 EXPERIMENTS
In this section, we conduct extensive experiments to answer the research questions:
RQI:
How effective is DIR in discovering causal features and improving model generalization? RQ2: What are the learning patterns and insights of DIR training? Especially, how does invariant rationalization help to improve generalization?
3.1
SETTINGS
Datasets. We use one synthetic dataset and three real datasets of graph classification tasks. Different GNNs are used different datasets achieve DIR and early stopping exploited during training- Here we briefly introduce the datasets, while the details of dataset statistics, deployed GNNs, and training process are summarized in Appendix D.
Spurious-Motif is synthetic dataset created by following Ying al. (2019), which involves 18, 000 graphs. Each graph is composed of one base (Tree, Ladder, Wheel denoted by S = 0, 1, respectively) and one motif (Cycle, House, Crane denoted by C 0, 1, 2. respectively). The ground ground-truth truth label Y is determined by C solely. Moreover, we manually construct false relations of different degrees between S and label Y in the training set. Specificallyy, in the training set, we sample each motif from a uniform distribution, while the distribution of its base is determined by datasets
P(S) x T(S = C) + 1-b T(S + C). We manipulate b to create Spurious- of distinct biases. In the testing set, the motifs and bases are randomly attached to each other. Besides, we include graphs with large bases to further magnify the distribution gaps.
MNIST-75sp (Knyazev al., 2019) converts the MNIST images into 70, 000 superpixel graphs with at most 75 nodes each graph. The nodes in the graphs are superpixels, while edges are the spatial distance between the nodes. Every graph is labeled as of 10 classes. Random noises are added to nodes' features in the testing set.
Graph-SST2 (Yuan et al., 2020; Socher al., 2013) Each graph labeled by its sentence senti- ment and consists of nodes representing tokens and edges indicating node relations. Graphs are split into different sets according to their average node degree to create dataset shifts.
Molhiv (OGBG-Molhiv) (Hu et al., 2020; 2021; Wu al., 2017) is molecular property predic- tion dataset consisting of molecule graphs, where nodes are atoms, and edges are chemical bonds. Each graph is labeled according to whether molecule inhibits HIV replication not.
Baselines. We thoroughly compare DIR with Empirical Risk Minimization (ERM) and two classes of basclines:
Interpretable Baselines: Graph Attention (Veličković al., 2018) and graph pooling opera- tions including ASAP (Ranjan et al., 2020), Top-k Pool (Gao & Ji, 2019) and SAG Pool (Lee et al., 2019). We use their generated masks on graph structures as rationales. We also include GSN (Bouritsas et al., 2020), topologically-awer GNNs with interpretable structural features.
Robust/Invariant Learning Baselines: Group DRO (Sagawa al., 2019), IRM (Arjovsky al., 2019), V-REx (Krueger et al., 2021). This class of algorithms improves the robustness and generalization for GNNs, which helps the models better generalize in unseen groups or out-of- distribution datasets. We use random groups or partitions during the model training.
We also include ablation model of DIR, DIR-Var, which sets À 0, i.e., discards the variance term in RDIR. to show the effectiveness of the variance regularization the DIR objective.
Metrics. We use ROC-AUC for Molhiv and ACC for the other three datasets. Moreover, for Spurious-Moti Motif dataset. we use the precision metric to evaluate the coincidence between model rationales and the ground-truth rationales, and validate the interpretability ability quantitatively.

---

Published as a conference paper at ICLR 2022
Performance on the Synthetic Dataset and Real Datasets. Spurious-Motts dataset, we color brown for the results lower than ERM, where b is the indicator of the confounding effect.
Table 1:
Spurious-Motis
MNIST-75sp
Gruph-SST2
Molhiv
Balance
b 0.5
b 0.7
b 0.9
ERM
42.99+1.93
39.69=1.73
38.93+1.74
33.61
12.71
81,4440.9
76,20+1.44
Attention
43.07:2255
39.42:1.50
30.4
33.462204
15.194262
81.57+071
75.84+1.3
ASAP
44.44:1..1
4..25
39.1914.39
31.76.22.9
15.54.1.87
81.57
73.81
Top-k Pool
43.43
41.21
40.27
33.60
14.91
79.78+1135
73.01
SAG Pool
45.23:26.6
43.82226.2
40.45=27.0
33.60=11118
14.31:22444
80.24:11.72
73.26:1084
GSN
43.18:15.65
34.67=1.21
34.03 1.69
32.60
19.03:12.99
12.54
74.53
Group DRO
41.51
39.38
39.32.1221
33.90
15.13
81.29
75.44:12.0
V-REx
42.83+1.59
39.43=2.69
39.08+1.56
34.01
18,92+141
81.76+008
75.62+079
IRM
42.2612.6
41.30=1128
40.1621774
35.122277
18.6221.22
81.01:1.13
74,46+2774
DIR-Va
45,87:261
11.9
42.69
37. 12:21.666
17.74:14.7
81.741089
76.05
DIR
47.03
45.50=2.15
43.36:1.4
39.87:1055
20.36
83,29
77.05
Table 2: Precision@5 @5 on Spurious Spurious-Motif. Motif.
Model
Balance
0.5
b 0.7
b
0.9
Attention
0.183
0.183
0.182=u014
0.134
ASAP
0.187=0000
0.186=uaz7 0.188
0,121 +0,22
Topk Pool
0.215
0,207
0.212=m056
0.148
SAG Pool
0.212.0.003
0,198 10.062
0.201
0.136.10.04
DIR
0.257=00044
0,255
0.247
0.
3.2 MAIN RESULTS (RQ1)
To fairly compare the methods, we train each model under the same training settings as described in Appendix D. The overall results are summarized Table 1, and we have the following observations:
1. DIR has better generalization ability than the baselines. DIR outperforms the baselines con- sistently by a large margin. Specifically, for MNIST-75sp dataset, DIR surpasses ERM by 7.65% and ASAP by 4.82%. Although structure features are shown to be helpful in mitigating feature distribution shift, DIR still performs better than GSN. For Graph-SST2 and Molhiv, DIR achieves the highest performance with low variance. For Spurious-Motif, DIR outstrips IRM averagely by 4.23% and SAG by 3.16% across different degrees of spurious bias. Such improvements strongly validate that DIR can generalize better in various environments.
DIR is consistently effective under different bias degrees, while the baselines easily fail. For interpretable baselines, Attention fails make salient improvements when bias exists, and pool- ing methods also fall through under severe bias. This is empirically in line with our presumption that GNNs easily biased latch on spurious relations or non-causal features and thus gen- eralize poorly in OOD data. For robust/invariant learning baselines, IRM underperforms ERM when b small. This evidence is accordant with the conclusion in Ahuja et al. (2021) that IRM is guaranteed to be close to the desired OOD solutions when confounders exist, while it has no obvious advantage to ERM under covariate shift. Moreover, Group DRO and V-REx follow a similar pattern. In contrast, DIR works well in various scenarios. We credit such reliability the rationales discovery from which the causal features C are potentially extracted, and the relation C' + Y learned by the GNNs is invariant across the distribution changes in the testing set.
Data augmentation by intervention is beneficial while the variance regularization further boosts model performance. Interestingly. the ablation model DIR-Var has already exceeded some of the baselines. We attribute such improvement to data augmentation via interventional distributions. On top of DIR- Rar. DIR improves the model performance by averagely 1.57% in Mpurious-Motif and 2. 2.62% in MNIST-75sp. This suggests that the variance regularization demands stronger invariance condition and is instructive for searching causal features.
4. DIR has better intrinsic interpretability than the baselines. In Table 2, we report intrinsic interpretable models' performance w.r.f. Precision@5. From the consistent improvements over the baselines, we find DIR has an advantage in discovering causal features. And the performance gap between DIR and the baselines becomes more significant when the bias increases.
7

---

Published as a conference paper at ICLR 2022
schivvement
of
grandesr
aequerices
301T4
dialogue
mejestic
epic
astorishing
Training rationale: Positive sentiment.
(b) Training rationale: Negative sentiment.
whhoe
surpetinngt
604
eegrossing
sdnimsdly
Ain
(c) Testing rationale: Positive sentiment.
(d) Testing rationale: Negative sentiment.
Figure 4: Visualization of DIR Rationales. Each graph shows a comment, e,g,, "a majestic achieve- ment, an epic of astonishing grandeur" in (a), where rationales are highlighted by deep colors.
Trting
odaption
stttig
0.000
Panalty
5,300
Training Time
(a) The first two subfigures show the training curves w.r. variance penalty and precision. Spurious-Motif. The last three subfigures present the rationale distributions of the inspection points, which are visualized by t-SNE (van der Maaten, 2008).
Faining Time
Saaving Time
Training Time
Training Time
Trasdeg Time
Trawing Tire
(b) The first three subfigures present the training curves variance penalty and ACC MNIST-75sp, while the last three illustrate curves w.r.. variance penalty and AUC-ROC on Molhiv.
Figure 5: Two-stage Training Dynamics of DIR.
3.3 IN-DEPTH STUDY (RQ2)
We empirically analyze the DIR's properties which hopefully give insights into its mechanisms and can be instructive for the existing training paradigms deep models.
Rationale Visualization. Towards an intuitive understanding of DIR, we first present some cases of the discovered rationale for Graph-SST2 Figure DIR is able to emphasize the tokens that directly result in the sentences' positive negative sentiments, which are reliable and faithful tionales. Specifically, DIR highlights the positive words "majestic achievement" and "astonishing grandeur" in Figure 4a and underscores the negative words "worst dialogue" in Figure 4b as the rationales, which are clearly salient for the positive and negative sentiments, respectively. Further- more, DIR can focus persistently selects surprisingly engrossing and "admittedly middling" in Figures 4c and 4d, respectively. This again validates the effectiveness of DIR: (1) ha is well-learned to distinguish causal and non-causal features under various interventional distributions; and (2) hy- conducts passsage-passing on the highlighted rationales, extracts the graph representations, and finally outputs the predictions with high accuracy. See Appendix F.1 for more examples in Graph-SST2 and Spurious-Motif datasets.
the causal features for OOD testing data. For example, it
Two-stage Training Dynamics. As Figure 5a displays, we find a pattern from the Var-Time curve during training DIR, the variance penalty (i.e., Var, in Equation 4) first increases and then de- creases to almost zero. Moreover, there exists an interesting correlation between the variance penalty and the precision metrics that is, the precision rises dramatically as the penalty increases while growing slowly as the penalty decreases. To probe this learning pattern, we further visualize the rationale distribution in three turning points: (1) the start, (2) the middle, and (3) the end of train- ing. Interestingly, the rationale distribution at the middle point is highly similar to that at the ending point. This illustrates two stages, adaption and fitting, in the patterns. By "adaption", we mean that the exhibition of hg, i.e., learning select salient feature C, is mainly conducted during the initial training stage. Since the penalty value can be seen as the magnitude to violate the invariance con- dition, this stage explores the rationales that satisfy the DIR principle. Correspondingly, hy adapts
8

---

Published as a conference paper at ICLR 2022
quickly with the input of varying rationales generated by hg. By "fitting", we mean that, in the later training process, he only makes small changes, resulting in the substantially unchanged rationales compared to the initial training process, which is learned from the rationale generator to conform to the DIR principle. This could also imply that based on the well-learned rationales, DIR mainly optimizes he to consolidate the functional relation Ö - Y until model convergence.
Moreover, compare the learning patterns of IRM and DIR in Figure 5b, where the penalty term of IRM (the gradient norm penalty in IRMv1 (Arjovsky et al., 2019)) follows a similar pattern to the DIR penalty. Notably, in MNIST-75sp, while IRM consistently outperforms DIR w.r.f. Training ACC, it does not improve and even degrades the performance in the testing dataset due to over- fitting. However, DIR shows the solid resistance for over-fitting, partly thanks to the valid rationales exhibited in the adaption stage. For Molhiv, DIR outperforms IRM as the rationales filter out irrele- vant or spurious structures bootless for classification tasks and are beneficial for generalization.
Sensitivity Analysis. We conduct a sensitivity analysis of model performance w.r.. À in Appendix F.2, which shows that DIR surpasses the best baselines under a relatively large range of A.
4 RELATED WORKS
Inherent Interpretability of GNNs. We summarize two classes of the existing methods build deep interpretable GNNs, (i) Attention (Vaswani et al., 2017; Veličković et al., 2018), which can be broadly interpreted as importance weights on representations Pooling (Lee al., 2019; Knyazev et al., 2019; Gao & Ji, 2019), which selectively performs down-sampling representations. We include it in this category when it involves selection importance. However, the mechanisms to generate the rationales could be epistemic, as they only reflect the probabilistic relations between data and predicted labels (Pearl, 2000), which may not hold true in all data distributions. Thus, the rationales could fail to align with causal features and even degrade model performance due to being "fooled"" by spurious features (Chang et al., 2020).
Invariant Learning. Backed by causal theory, invariant learning assumes the causal relation from the causal factors C' to the response variable Y remains invariant unless we intervene on Y. As the most prevailing formulation, IRM (Arjovsky et al., 2019) extends the invariance assumption from feature level to representation level and finds a data representation & such that Q o & matches for all environments, where Q is the classifier. However, concerns about its feasibility (Rosenfeld et al., 2021: Ahuja et al., 2021) and optimality (Kamath et al., 2021) have been discussed recently. Besides IRM, variance penalization across environments shown to effective for recovering in- variance (Krueger et al., 2021; Xie et al., 2020; Teney et al., 2020). Notably, the existing methods generally require accessing different environments, thus additionally involving environment infer- ence (Creager et al., 2021; Wang et al., 2021b). Similarly motivated as ours, Chang et al. (2020) discover rationales Z by minimizing the performance gap between environment-agnoste predictor f(Z) and environment-awerg aware predictor f(Z, E). In graph domain, Bevilacqua al. (2021) con- struet graph representations from subgraph densities and use attribute symmetry regularization to mitigate the shift of graph size and vertex attribute distributions.
CONCLUSION & FUTURE WORK
In this work, we rigorously study the intrinsic interpretability of Graph Neural Networks from causal perspective. Our concerns are towards the exhibition of shortcut features when generating the rationales. And we proposed an invariant learning algorithm, DIR, to discover the causal features for rationalization. The core of DIR lies in the construction of environments (i.e., interventional distributions) and thus distilling the salient features as rationales that are consistently informative and uniform across these environments. Such rationales serve as the probing towards model mech- anisms and are demonstrated to be effective in generalization. In the experiments, we highlight an adaption-fitting training dynamics for DIR to reveal its learning pattern. In the future, we will build more reliable and expressive interpretable models that are feasible under various assumptions, which potentially calls for high-level interpretability. We recommend interested readers go to the open discussion in Appendix G for the detailed description.

---

Published as a conference paper at ICLR 2022
ACKNOWLEDGMENT
This work was supported by the National Key Research and Development Program of China (2020AAA0106000), the National Natural Science Foundation of China (U19A2079), the Sea- NExT Joint Lab, and Singapore MOE AcRF T2.
ETHICS STATEMENT
In this work, propose novel algorithm for intrinsic interpretable models, where human sub- ject is related. This synthetic dataset made available in the anonymous link (ef. Section 3.1). We believe the exhibition of rationales is beneficial for inspecting and eliminating potential discrimina- tion and fairness issues in deep models for real applications.
REPRODUCIBILITY STATEMENT
We summarize the efforts made to ensure reproducibility in this work. (1) Datasets: We use one synthetic dataset which made available (cf. the anonymous link in Section 3.1), and three public datasets where the processing details are included in Appendix D. (2) Model Training: We pro- vide the procedure of training in Algorithm A and the training details (including hyper-parameter settings) in Appendix D which consistent with our implementation in the code (ef. the anony- mous link in Section 3.1). (3) Thcoretical Results: All assumptions and proofs can be referred to Appendix C.
REFERENCES
Kartik Ahuja, Jun Wang, Amit Dhurandhar, Karthikeyan Shanmugam, and Kush Varshney. Em- pirical or invariant risk minimization? A sample complexity perspective. In ICLR, 2021.
David Alvarez-Melis and Tommi S. Jaakkola. causal framework for explaining the predictions of black-box sequence-to- sequence-to-sequenee models. In EMNLP, pp. 412-421, 2017.
Martín Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization.
CoRR, 0bs//997.0288)) 2019.
Yoshua Bengio, Aaron C. Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell., 2013.
Beatrice Bevilacqua, Yangze Zhou, and Bruno Ribeiro. Size-invariant graph representations for graph classification extrapolations. In ICML,, 2021.
Filippo Maria Bianchi, Daniele Grattarola, Lorenzo Livi, and Cesare Alippi. Graph neural networks with convolutional ARMA filters. CoRR, abs/1901..0 19111139,,33,,.,.. 2019.
Giorgos Bouritsas, Fabrizio Frasca, Stefanos Zafeiriou, and Michael M. Bronstein. Improving graph neural network expressivity via subgraph isomorphism counting. arXiv, 2006.09252, 2020.
Peter Bühlmann. Invariance, causality and robustness. arXiv, 1812.08233, 2018.
Rémi Cadène, Corentin Dancette, Hedi Ben-younes, Matthieu Cord, and Devi Parikh. Rubi: Re- ducing unimodal biases for visual question answering. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' A'Ach--Buc Emily B. Fox, and Roman Garnett (eds.), NeurIPS, 2019.
Kwan Ho Ryan Chan, Yaodong Yu, Chong You, Haozhi Qi, John Wright, and Yi Ma. Redunet: A white-box deep network from the principle of maximizing rate reduction. arXiv, 2105.10444 2021.
Shiyu Chang, Yang Zhang, Mo Yu, and Tommi Jaakkola. Invariant rationalization. ICML, 2020.
10

---

Published as a conference paper at ICLR 2022
Zhengdao Chen, Soledad Villar, Lei Chen, and Joan Bruna. On the equivalence between graph isomorphism testing and function approximation with gnns. In NeurIPS, 2019.
Elliot Creager, Jömn-enrik Jacobsen, and Richard S. Zemel. Environment inference for invariant learning. Marina Meila and Tong Zhang (eds.), ICML, 2021.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint 1rXivv:810.04800 2018.
Vijay Prakash Dwivedi, Chaitanya K. Joshi, Thomas Laurent, Yoshua Bengio, and Xavier Bresson. Benchmarking graph neural networks. CoRR, (bss/20030099,3 2020.
Hongyang Gao and Shuiwang Ji. Graph u-nets. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), ICML, pp. 2083-2092, 2019.
William L. Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large 2017.
graphs. In NewrlPS, pp.
Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning graphs. arXiv preprint arXiv:2005.00687 2020.
Weihua Hu, Matthias Fey, Hongyu Ren, Maho Nakata, Yuxiao Dong, and Jure Leskovec. Ogb-lsc: A large-scale challenge for machine learning on graphs. arXiv preprint arXiv:2103.094: 2021.
Pritish Kamath, Akilesh Tangella, Danica J. Sutherland, and Nathan Srebro. Does invariant risk minimization capture invariance? In Arindam Banerjee and Kenji Fukumizu (eds.), AISTATS, 2021.
Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings. 2015.
Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional net- works. In ICLR, 2017.
Boris Knyazev, Graham W. Taylor, and Mohamed R. Amer. Understanding attention and general- ization in graph neural networks. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' Alché-Buc Buc, Emily B. Fox, and Roman Garnett (eds.), NeurlPS, pp. 4204 4214, 2019.
David Krueger, Ethan Caballero, Jörn-Henrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Rémi Le Priol, and Aaron C. Courville. Out-of-distribttion generalization via risk extrap- olation (rex). In Marina Meila and Tong Zhang (eds.), ICML, pp. 5815-5822 2021.
Solomon Kullback. Information theory and statistics. Courier Corporation, 1997.
Junhyun Lee, Inyeop Lee, and Jaewoo Kang. Self-attention graph pooling. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), ICML, pp. 3734-3743, 2019.
Pan Li, Yanbang Wang, Hongwei Wang, and Jure Leskovec. Distance encoding: Design provably more powerful neural networks for graph representation learning. NeurIPS, 2020.
Dongsheng Luo, Wei Cheng, Dongkuan Xu, Wenchao Yu, Bo Zong Haifeng Chen, and Xiang Zhang. Parameterized explainer for graph neural network. In NeurIPS, 2020.
Haggai Maron, Heli Ben-Hamu, Hadar Serviansky, and Yaron Lipman. Provably powerful graph networks. In NeurIPS. 2019.
Christopher Morris, Martin Ritzert, Matthias Fey, William I. Hamilton, Jan Eric Lenssen, Gaurav Rattan, and Martin Grohe. Weisfeiler and leman go neural: Higher-order graph neural networks. In AAAI, pp. 4602-4609, 2019.
Judea Pearl. Causality: Models, Reasoning, and Inference. 2000.
11

---

Published as a conference paper at ICLR 2022
Judea Pearl, Madelyn Glymour, and Nicholas P Jewell. Causal inference in statistics: A primer. John Wiley & Sons, 2016.
Ekagra Ranjan, Soumya Sanyal, and Partha P. Talukdar. ASAP: adaptive structure aware pooling for learning hierarchical graph representations. In AAAI, pp. 5470-5477, 2020.
Elan Rosenfeld, Pradeep Kumar Ravikumar, and Andrej Risteski. The risks of invariant risk mini- mization. In ICLR, 2021.
Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generaliza- tion. CoRR, 0oss/991.08731,, 2019.
Andrew W. Senior, Richard Evans, John Jumper, James Kirkpatrick, Laurent Sifre, Tim Green, Chongli Qin, Augustin Zidek. Alexander W. R. Nelson, Alex Bridgland, Hugo Penedones, Stig Petersen, Karen Simonyan, Steve Crossan, Pushmeet Kohli, David T. Jones, David Silver, Koray Kavukcuoglu, and Demis Hassabis. Improved protein structure prediction using potentials from deep learning. Nature, 577(7792):706-710 2020.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over sentiment treebank. In EMNLP, pp. 1631-1642, 2013.
Damien Teney, Ehsan Abbasnejad. and Anton van den Hengel. Unshuffling data for improved generalization. arXiv, 2002. 1894, 2020.
Jin Tian, Changsung Kang, and Judea Pearl. characterization of interventional distributions in semi-markovian causal models. In AAAI, pp. 1249,1244, 2006.
G.E. der Maaten, L.J.P: Hinton. Visualizing high-dimensional data using t-sne. Journal of Machine Learning Research 9:2579-2605, 2008.
Tyler J VanderWeele. A three-way decomposition of a total effect into direct, indirect, and interactive effects. Epidemiology (Cambridge, Mass.), 24(2):224, 2013.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (eds.), NeurIPS, 2017.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pictro Liò, and Yoshua Bengio. Graph attention networks. ICLR, 2018. accepted as poster.
Tan Wang, Zhongqi Yue, Jianqiang Huang, Qianru Sun, and Hanwang Zhang. Self-supervised learning disentangled group representation as feature. arXiv, 21 2110.15255, 2021a.
Tan Wang, Chang Zhou, Qianru Sun, and Hanwang Zhang. Causal attention for unbiased visual recognition. arXiv, 2108.08782, 2021b.
Xiang Wang, Yingxin Wu, An Zhang, Xiangnan He, and Tat seng Chua. Towards multi- aalti-grained explainability for graph neural networks. In NeurIPS, 2021c.
Zhenqin Wu, Bharath Ramsundar, Evan N. Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S. Pappu, Karl Leswing, and Vijay S. Pande. Moleculenet: A benchmark for molecular machine learning. arXiv, abs/1703.00564 2017.
Chuanlong Xie, Fei Chen, Yue Liu, and Zhenguo Li. Risk variance penalization: From distributional 2020.
robustness to causality. arXiv,
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? ICLR. OpenReview.net, 2019.
Zhitao Ying. Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec. Gnnexplainer: Generating explanations for graph neural networks. In NeurIPS, pp. 9240-9255 2019.
12

---

Published as a conference paper at ICLR 2022
Hao Yuan, Haiyang Yu, Shurui Gui, and Shuiwang Ji. Explainability in graph neural networks: A taxonomic survey. CoRR, 2020.
Hao Yuan, Haiyang Yu, Jie Wang, Kang Li, and Shuiwang Ji. On explainability of graph neural networks via subgraph explorations. ArXiv, 2021.
13

---

Published as a conference paper at ICLR 2022
A NOTATIONS & ALGORITHM
Key Notations in the Main Paper.
Symbol
Definition
graph instance
c/
ground truth causal confounding subgraph generated rationale or complement of rationale instance variables in the causal graph
č/
C/S
S/S
space of the ground truth or identified spurious features causal or spurious prediction
ia /s
joint prediction
he
rationale generator
Φ1 / 22
causal or spurious classifier
Algorithm Pseudocode for DIR in training interpretable Graph Neural Networks (Batch Version)
Require:
Training data distribution Par (G);; number of classes Q: Stepsize a; hyper-paramette Randomly initialize the parameters of generator hig encoder he (includes GNN2 and Pooling layer), two classifiers $ and 42 which are denoted as y, 0, Φ1, d2 respectively.
2: while not converge do
3: Sample graphs {(o', >²) from PG)(G)
4: Generate each rationale and its complement: (d², +- ho(9") for = I....,..
5: for each 3 do
6: Intervener hs operates do(S 8¹)
7:
Model forward: 0s 0b (ho RIIQ (ūa) $ (ho( (ho( BBQ
8:
# block BP of DIR risk to shortcut branch
Obtain joint prediction 9 [0 ..... 8 where g = úa @ o ( detaach( Compute and record risk R(,,,
9:
Compute and record 3 -inteerventionll risk. end for
11:
12:
Compute RDIR via Eq. 4 and Rg via Eq. 11
13:
Update parameters: dh2 = Φ2 V dz Rgi Φ1 = Φ1 - a RDR
ー a RDIR
VVRRD
14: end while
B INSTANTIATED CAUSAL GRAPHS
We instantiate possible causal graphs in Figure 2a. Specifically, we use the example of Base-Motif' graphs, whose labels are determined by the motif types. We use C' 0, 1,2 to denote cycle, house, crane, respectively; And use 5 = 0, 1,2 to denote ladder, tree, wheels, respectively.
C _L St Base graphs and motif graphs are independently sampled and attached each other. C + S: Type of each motif respects to a given (static) probability distribution. According to the value of C, the probability distribution of its base graph is given by
P(S) {
0.6 if S C 0.2 otherwise
(12)
C: Similar to the example for C S.
S - E - C: Suppose there is a latent variable E takes continuous value from 0 to 1. Then the probability distribution of S and s.t.
S ~~ B(3, F)) C~ B(3, E) where stands for binomial distribution, i.e., for variable X, if ~ B(n, p). then we have
(13)
P(X = k ppp)
p (1 n)
14

---

Published as a conference paper at ICLR 2022
C THEORY
C.I ASSUMPTION
We phrase the SCM in Figure 2a as the following assumption:
Assumption (Invariant Rationalization (IR)) There exists a rationale C' C G, such that the
structural equation model
Y+ fr cy
and the probability relation
SLYYO
hold for every distribution P over P(G,Y) Y), where S denotes the complement of C. Also, denote fy as the oracle structural equation model.
By "oracle", mean that fy the perfect structure equation model, which, when C is available, predicts the response variable with the minimum expected loss over any distribution P. Or formally,
fr == arg min R(f) = arg min min (,
(14)
where is the specifcc loss function and we ignore the exogenous noise cy in fy's input except as otherwise noted.
Next, we argue that the assumption is commonly satisfied. For example, for sentences labeled by sentiment, C' can represent the positive/negative words that cause the sentiment, while $ includes the prepositions and linking words. For molecule graphs labeled by specific properties, C and can represent the functional groups and carbon structures, respectively. Note that IR assumption enables and calls the introduction of interpretability, highlighting salient features and exhibiting human accessible checks. More importantly, it guarantees the model performance under possible feature reduction, i.e., C G.
We also see cases going beyond the IR Assumption. For example, G could be generic function of S and C, instead of simple joint. We use toy example to elaborate this point. Following the Spurious- Motif dataset, we assume each graph has multiple motifs (house, cycle, crane) with only one type and is labeled by the motif type. Thus, the causal feature C' will be the motifs. Let the spurious feature S be "the way we connect the motifs". For example, we can place the house motifs in queue sequence and connect the adjacent motifs, thus forming the graph in a "line" shape. Or cycle order and connect them into ring. We further make such graph structures strongly correlated with the motif types. Thus, individual S and C' may be intractable individually in the feature level. For example, if we separate the cycle-shaped houses into two lines, the spurious pattern could be broken while the part of the causal feature would be lost. In other words, and C' are dependent variables. Thus, they can't be extracted and modeled separately, which goes out of the scope of our work.
we can place the houses
Given that S and C' are separable, we further make the following assumption to avoid the confusion of S and C:
Assumption (Feature Induction) Define power set operation as PP'(.. For data G = S U C and label Y. if S_LL Y | C' holds for any distribution P over P(G,Y), then it implies that for any induced feature S' E (S), we have S' Y IC holds for the distribution P.
This assumption also implies that C could not be induced by when SC Thus, any feature subset C" except for C' would violate the conditional independence condition. For images, this assumption natural for the splicing S doesn't it typically change its semantics. For example, the splicing of land background would still be divided land. While for graphs, here assume the causal subgraph's 's uniqueness among the induced complement graphs.
C.2 PROOFS
Theorem (Necessity) Suppose C. does not exist, then the oracle function fy satisfies the DIR Principle (where C is given) over every distribution Po P(G,Y)
15

---

Published as a conference paper at ICLR 2022
Proof:
We first prove the fact that P(Y y do(S = s)) P(Y y) for distribution P.
Specifically, we use ppa)) to denote the s-interventional distribution.
1+C+5
P(Y = do(S = b)) by definition
-
= (
given C-8
given (L 910)A
given invariance condition
Σ P(Y = = | cC=c))(C=c)
= P(Y = y)
If CI S,
P(Y = y do(S 8))
by definition
pf
S = 5)
given S has no endogenous parent
P(Y y S s)
givenC:
Σ
P c. 5) P(C = c)
P(Y = E
C= = c s
given (Y IL S1018
P(Y =v)
pl==vldo(s = 8))
by delinition
1/ = S|S= ss))
given 5
E//0 -y-l - =
(/,0::/ -
given (Y IL (S.E)(C) and (C_L S
ΣΣ = -== =) = e)
ΣΣ P(Y = y -= c)ppccPCCC-C = e) cc)
ΣΣ P(Y = cE = e) =)
= P(YY= =y)
As P(Y y y do(S== =))= =))= = P(Y = y) holds true for every distribution P. which is invariant w.r.f.
iterative variable S. Moreover, we have P(C = do(S = s)) c) P(C = c). This
16

---

Published as a conference paper at ICLR 2022
indicates that the intervention on S leave the causal structure C. Y untouched. Thus, have
Var ([R(Jv do(s)) E &)) = Var (Faan-/pponcccddl:://00)0111)
Var ([Han)((((()))))))))))
0
Finally, taking the definition of f, we have
fv arg min Eses P(an-ppaaa.ccc//OO...]]
= arg min E.es [((O),
arg min [R([ do(s))]
Hence, JY takes the minimum penalty and satisfies the DIR Principle.
Notably, if S -> C, then Var ((R(fv | do(s)) s E)) may not equal to zero since c ~ (CIS =s). In such case, Jy is not necessarily satisfied DIR Principle. That is, al- though fv still minimizes R( do(S)), we can't be sure whether it reaches the lower bound of Var ([R(Jy do(s)) 8 ()) without knowledge about the specific data distribution. Thus, only consider the cases of C + S, C.L S and C' + S in the following discussion.
Theorem (Uniqueness) Suppose is a strict loss function and there exists one and only one non- trivial subset C, then there exists unique structure equation model fY it satisfies the DIR Principle.
Since fy exists and satisfies the DIR Principle, we only need to prove its uniqueness under the given conditions. Otherwise, suppose we have another structure equation fr fy satifies the DIR Principle. Specifically, there exists a datum (g, y) s.t. ff(c) + fr Thus, we have ((fi-c)).) y) > I(fy (c), Given that Var ({R(fr do(s)) (SES) 2 0 Var (fR(fy) do(s)) E S)), we have RDIR( ) > RDI(ff).
Proof:
In reality, there could be multiple candidates C, egg. C₁ Cj s.l. RDRR( RoIR ( where s(C:) is the structure equation corresponds to C.. Thus, it calls for the selection of C to avoid the learning of suboptimal fv. Inspired by Occam's Razor, we define
C = arg min |C]
(15)
as the preferred rationale, or rationale of parsimony. We argue that rationales are not to extended beyond necessity, which poses simpler hypotheses about causality. As the search of C* is NP- hard (the worst time complexity is exponential), we use fixed size for the learned rationales in our experiments and leave better optimization future work.
Corollary (Necessity and Sufficiency) Suppose I is a strict loss function and there exists one and only one non-trival subset C, then any structure causal model f/y s.t. satisfies the DIR Principle iff. ft = fv.
This is directly obtained from Theorem 2. Thus, under the unique constraint of C, we can approach the oracle fy by optimizing the DIR objective, which maintains the invariant causal relation between the causal feature and the response variable Y. another way, based on the uniqueness of the feasible rationale, the optimization of the DIR Principle the intrinsic interpretable model h (where C' is exhibited inside of hi) pushes the approach to C with rationales č. Then, fy also be approached an invariant predictor based on the learning from Ć.
17

---

Published as a conference paper at ICLR 2022
D SETTING DETAILS
Table 3: Statistics of Graph Classification Datasets.
Spurious Motif
MNIST-75<p freduced)
Graph SST2
OGBG Molhiv
Train
Val
Test
Train
Val
Test
Train
Val
Test
Train
Val
Test
Classes#
3
10
2
2
Graphse
9.000
3,000
6,000
20,000
5,000
10,000
28.327
3,147
12,305
32,901
4.113
4.113
Avg. Na
25.4
26.1
88.7
66.8
67.3
67.0
17.7
17.3
3.45
25.3
27.79
25.3
Avg.
545.9
540.4
33.3
Local Extremum GNN
R-GNNs
ARMA
GIN Virtual nodes (Xu al., 2019, Hu et al., 2021)
Backbone
(Ranjan et al., 2020)
(Morris et al., 2019)
(Bianchi et al., 2019)
Neurona
[4.32,32.32]
[5.32,12,32]
[768,128 128,12
19,300,300,000) global add pool
Global Pool
global mean pool
glohal max pool Noise
global mean pool
Gen. Type
Scale & Correlation Shift
Degree & Scale
Datasets
We summarize dataset statisties in Table 3, and introduce the node/edge features and the preprocessing in each datasets:
Spurious-Motif. We use random node features and constant edge weights in this dataset.
MNIST-75sp. The nodes in the graphs are superpixels, and node features are the concatenation of pixel intensities (RGB channels) and coordinates of their mass centers. Edges are the spatial distance between the superpixel centers, while we filter the edges with a distance less than 0.1 to make the graphs sparser.
Graph-SST2. We use constant edge weight and filter the graphs with edges less than three. We initialize the node features by the pre-trained BERT (Devlin et al., 2018) 8)) word embedding.
OGBG-Molhiv. We use the official released dataset in our experiment.
GNNs. We summarize the backbone GNNs for each dataset in Table 3. The number of neurons in the sequent layers (in forwarding order) is reported. We use ReLU as activation layers and different global pooling layers. In OGBG-Molhiv, we adopt one fully connected layer for the prediction layers while using two fully connected layers for the models in other datasets. For baselines with node pooling/node attention, we add one node pooling/attentio layer the second convolution layer.
Training Optimization & Early Stopping. All experiments done on single Tesla V100 SXM2 GPU (32 GB). During training, we use Adam (Kingma & Ba, 2015) optimizer. The max- imum number of epochs is 400 for all datasets. We use Stochastic Gradient Descent (SGD) for the optimization on Graph-SST2 and OGBG-Molhiv and Gradient Descent (GD) for the other two datasets. Also, we exhibit early stopping to avoid overfitting of the training dataset. Specifically, in MNIST-75sp. Graph-SST2 and OGBG-Molhiv, each model is evaluated on a holdout in-distribution validation dataset after each epoch. While for Spurious-Motis we use an unbiased validation dataset (i.e., without spurious relations compared to the training dataset). If the model's performance the validation dataset is without improvement (i.e., validation accuracy begins to decrease) for five epochs, we stop the training process to prevent increased generalization error.
Hyper-Parameter Settings, We set the causal feature ratio and 0.25, À = 10 10- (r = 0.6, = 10²), (r = 0.8, A = 10 10-3) for MNIST-75sp. Spurious-Motif, Graph-SST2 and OGBG-Molhiv respectively. For other baselines, we adopt grid search for the best parameters using the validation datasets.
= 0.8, = 1)
Model Selection.
We select each model based its performance the corresponding validation dataset. We repeat each experiment at least five times and report the average values and the standard emors in the paper.
18

---

Published as a conference paper at ICLR 2022
E UNIMODAL ADJUSTMENT
We follow Cadène et al. (2019) demonstrate how the shortcut prediction help remove model bias. For clarity, we refer to the model parameters except for 22 as the main branch, i.e., except for the S-only branch.
Given a house-tree graph as the input graph, we suppose the shortcut prediction ÿa of the tree sub- graph leans towards the house class. Then after reweighting ((ss) on ůe. the softmax readout on the house class in the joint prediction will be magnified, which results in smaller loss back- propagated to the main branch and prevents Úa from inductive bias.
In another situation where house-whedd graph is given the input, we similarly suppose the shorteut prediction Ys of the wheel subgraph leans towards other classes except the house, say, the circle class. Then after reweighting o (jss) Da. the softmax readout on the house class in the joint prediction ú will be reduced, which results larger loss back- paok-propagated to the main branch and encourages the model to learn from these examples.
Furthermore, we offer the causal- and information ciriioonnthhrrtccall justifications: (1) From the perspec- tive of causal theory (Pearl, 2000; Pearl et al., 2016), the clement-wise multiplication enforces the spurious prediction to estimate the pure indirect effect (PIE) of the shortcut features, while the causal prediction captures the natural direct effect (NDE) of the causal patterns (VanderWeele, 2013); (2) From the perspective of information theory (Kullback, 1997), the element-wise multiplication makes the causal prediction reflect the conditional mutual information between the causal patterns and ground-truths, conditioning on the complement patterns.
F MORE EXPERIMENTAL RESULTS
F.1 VISUALIZATION
We provide more visualization cases in Graph-SST2 dataset as shown in Figure 6 and Figure 7. The rationales are highlighted in deep colors.
datermineed
hop
actton
derrida
an
fascinating
payyul
undeniably
fellow
(a)
forced
wildly
about
ability
(d)
Figure 6: Visualization of Training Rationales. Each graph represents comment, e.g., "determined to uncover the truth and hopefully inspire action" in (a).
refewent
Inday
(a)
(b)
brr
sharadess
much
raight
(d)
Figure 7: Visualization of Testing Rationales. Each graph represents a comment, e.g., "whimsical and relevant today" in (a).
19

---

Published as a conference paper at ICLR 2022
Table 4: Confidence of the Spurious Predictions. Uniform is the reference indicates the uniform
distributions across the classes.
Spurious-Motif (6=0.9)
MNIST-75sp
GraphSST2
Molhiv
Uniform Spurious Predictions
1.10
2.30
0.693
0.693
0.529
1.93
0.265
0.187
(a) Cycle-Tree
(b) House-Ladder
Crane-Tree
Figure 8: Visualization of Training Rationales Spurious-Moti Dataset. Structures with deeper colors mean higher importance. Nodes of ground truth rationales colored by green.
(a) Cycle-Tree
(b) House-Ladder
(c) Crane-Wheel
Figure 9: Visualization of Testing Rationales in Spurious-Motif Dataset. Structures with deeper
colors mean higher importance. Nodes of ground truth rationales are colored by green.
F.2 SENSITIVITY ANALYSIS
Sparions-Motil
MNIST-75<p
Graph-SST2
OGRG-Molhiv
6.78
ACC
0.38
0.30
0.33
9.76
0.30
117
0:18
10
107
107
1077
9.74
107
18
Figure 10: Sensitivity Hyper-Parameter X. In each chart, dash line represents the performance of the best baseline in the corresponding dataset, and the area between ACC+stt are colored.
We analyze the performance of DIR w.E.t. the hyper-parameter X. As shown in Figure 10, with 0, DIR degrades to optimize the performance in each environment only, without explicitly penalizing the shortcuts' influence on the model predictions. We also see that all testing performances drop sharply if À is too large. Since large weight on the variance term emphasis on the invariance condition while leading to the overlook on the performance loss, could fail to exhibit fp correctly. Notably, such trade-off in the DIR objective is commonly shared among all the datasets.
F.3 STUDY OF THE SPURIOUS CLASSIFIERS
Here we provide more observations about the predictions of the learned spurious classifier, which sheds light on the designed model mechanism. We first look into the confidence predictions and define
v E (iG.:)., H. (Softmax
(16)
20

---

Published as a conference paper at ICLR 2022
Table Performance of the Spurious Classifiers. A indicates the performance gap of the spurious classifiers and the corresponding causal classifiers.
Spurious- Motif (b=0.9)
MNIST-75sp
GraphSST2
Molhiv
Spurious Classifiers
13.23
17.09
81.141135 2.15
51.13
6.44
3.27
25.92
where H is the entropy function, and a lower indicates higher confidence. We report the results for the trained spurious classifiers in Table 4. Thus, results demonstrate the marked tendency of the spurious predictions and validate the design of the S-only branch.
However, we show that spurious classifiers are over-confident and potentially overfit to spurious features, which fails to generalize out-of-distribution. In Table 5, evaluate the spurious classifiers (taking non-causal features as inputs) on the testing sets. We argue that the performance degradation is caused by (i) feature-level problem: it could be theoretically inadequate to infer the label given the non-cat non-causal features, and (ii) paradigm-less problem: minimizing the empirical risk only can hardly exhibit stable relations between the features and labels.
F.4 COMPARISON OF POST-HOC EXPLANATIONS AND INTRINSIC RATIONALES.
Here aim compare the explanations generated by GNNExplainer (Ying al., 2019) and the rationales exhibited by DIR. Specifically, we generate post-hoc explanations from GNNExplainer for Spurious-Mone where we use the models trained under ERM as the models explain. We compute the precision of the explanations in Table 6.
Table Explanatioo/Rattonaale Accuracy in Spurious Motif dataset. The results of DIR is consistent with Table and we repeat them here for better view.
Balance
6=0.5
b=0.7
6=0.9
GNNExplainer
0.249
0.203+00019
0.167=0.039 0.247=0.012
0,066=00007
DIR
0.257+00014
0.255+00016
192
The explanations generated by GNNExplainer reflect the models' inner mechanism, which backs that deep models easily learn from data bias (especially when b is large), being at odds with the true reasoning process that underlies the task. Moreover, even when spurious correlations do not exist, the precisions of rationales generated by DIR still outperform the precisions of the post-hoc explanations, showing the effectiveness of DIR when identifying causal features.
G OPEN DISCUSSIONS
Based on this work, we provide open discussions and future directions for the research community, which are inspired by the insightful comments of the ICLR reviewers.
G.1 EXPRESSIVENESS OF RATIONALE GENERATORS
High expressiveness of the rationale generators could be beneficial for the identification of causal features. Therefore, we have offered additional techniques in our implementation to improve the ex- pressiveness of the graph encoder. Specifically, we incorporate distance encoding measures (Li et al., 2020) like shortest- path distances the extra node features for better structural representation leam- ing. Also, more powerful graph encoders like RingGNN (Chen et al., 2019) and 3WLGNN (Maron al., 2019) can be used the graph encoders distinguish different substructures better.
21

---

Published as a conference paper at ICLR 2022
G.2 GENERALIZATION UNSEEN SPURIOUS PATTERNS
In our implementation, the memory bank only contains the spurious patterns seen in the training set. while it could possibly fail unseen spurious patterns. And we provide discussions and solutions to solve this limitation:
Attribute level perturbation. When the spurious patterns in the testing are different from those in training set only on the attribute level, we can perturb the node/edge attributes of the subgraphs before intervention. And such perturbation is expected to improve the model's 's robustness during inference.
External knowledge base. When the spurious patterns also change on the structure level, for exam- ple, star-shaped unseen base graph appears in the testing set of the Spurious- Motif,, one potential solution is resort to prior knowledge. We can enrich the memory bank with possible spurious patterns, e.g., tree (seen) and star (unseen) base graphs. With the external knowledge base, the model can trained to recognize these possible spurious patterns and be well generalized to testing dataset.
Subgraph matching, In more tricky scenario when the external knowledge base is not available, we can integrate our model with subgraph matching algorithms the inference, For example, we can extract the training rationales into another bank C and use them to query the testing graphs, i.e., checking if similar patterns exist in the testing graphs. The match results may assist the rationale generator in highlighting the causal features and avoiding unseen spurious features.
G.3 HIGHER LEVEL INTERPRETABILITY
The interpretabillity of GNNs in the feature level implicitly demands the separability of graph into causal and non-causal features. At the same time, we see cases going beyond such assumption (cf. Appendix C.I). We believe could resort higher level interpretability. For example,
Interpretability of representations (Wang et al., 2021a; Chan et al., 2021). Instead of highlighting important features for the model decisions, the general goal of representation interpretability an- swers "What's the information encoded by the i-th element of the embedding in the j-th layer?".
Interpretations on top of disentangled variables. Each disentangled latent variable reveals one independent generative factor in data (Bengio et al., 2013). By generating importance score on these variables, we could possibly obtain more semantically rich interpretations than feature- level interpretability.
Wherein, believe there are fewer constraints on the separability of features. Thus, the models equipped with higher level interpretability could be applied to a broader range of data-generating assumptions.
22


Reviews:
[{"review_id": "lY-c1uzPKCu", "reviewer": "Reviewer_qBQC", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\nThe tackled problem of identifying the environment-invariant causal patterns for improving the interpretability of graph neural networks is interesting and important for many graph-structured data representation tasks.\n\nThe proposed DIR model is technically sound with the construction of interventional distributions and discover the salient features from different environments. In addition, the theoretical discussion is provided to analyze the proposed DIR method.\n\nIn the evaluation section, the effectiveness of the new DIR model is analyzed from different aspects: model generalization ability, performance with different bias degrees, as well as the intrinsic interpretability. Furthermore, the identified rationale has been visualized under interventional distributions.\n\nWeaknesses:\n\nIt would be better to summarize the key notations used throughout this paper in a table, for easy reference when reviewer go through the solution details.\n\nThe detailed configurations of the compared baselines could be described in the evaluation so as to provide a more comprehensive performance comparison.", "summary_of_the_paper": "This paper proposes to address the limitation of current interpretable graph neural networks over out-of-distribution data. In this work, the intrinsically interpretable GNN is investigated by identifying the invariant rationales corresponding to environment-invariant causal patterns. The proposed model is evaluated on the graph classification task to demonstrate its effectiveness as compared to state-of-the-arts.", "summary_of_the_review": "The studied problem is important and beneficial for many graph-learning tasks. The proposed method is rationale and technically sound. More details about the experimental settings could be clarified in the evaluation section.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "T9D1-cwGT4A", "reviewer": "Reviewer_derN", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1. DIR is end2end and novel to me.\n2. The DIR principle is intuitive and theoretically justified.\n3. The empirical results look good, namely, DIR consistently excels.\n4. The observations about how the variance changes during the training course are interesting and provide help for understanding other invariant learning methods.\n\nWeaknesses:\n1. What's the motivation behind the element-wise multiplication between causal predictions and spurious predictions? It has not been explained in the paper.\n2. With this multiplication and the various sampled spurious subgraphs, it seems that the spurious classifier should output constant vector 1 consistently, so that the causal predictions would not be modulated a lot, right? It would be better to provide more observations in the experiments about the predictions of the learned spurious classifier.", "summary_of_the_paper": "This paper proposes a novel algorithm DIR that allows the learned GNN model to make predictions based on causal patterns. The whole framework consists of four components where (1) the rationale generator splits the input into causal and spurious parts; (2) the distribution intervener replaces the spurious part; (3) the graph encoder embeds the graph based on the spurious and causal parts, respectively; (4) the shortcut and causal classifiers are applied to the spurious and causal embeddings, respectively. This algorithm is highlighted by its objective which encourages the predictions accurate while invariant to the spurious part. On both synthetic and real-world datasets, DIR outperforms the related baselines consistently. As a novel end2end rationalization method, As a novel end2end rationalization method, the underlying DIR principle is proved to be sufficient and necessary for satisfying the oracle function.", "summary_of_the_review": "This paper is well written. It proposes a novel end2end algorithm DIR towards causality-based graph representation learning. The algorithm is based on an intuitive principle that is theoretically justified by the authors in this paper. Meanwhile, the extensive experiments also confirm the effectiveness of DIR.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "8znCbRieb3", "reviewer": "Reviewer_F9KM", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "*This is an emergency review*\n\nUpdate Nov 20, 2021:\n\nRecommendation: Weak Accept\n\nIncreasing my score to 6 after some discussions with the authors.\n\n***************************************\nInitial Recommendation: Weak Reject\n\nI have tried to reason my recommendation below:\n\nStrengths:\n1. The idea to use a causal graph and the accompanying theory to explain the label associated with the graph is interesting. \n2. The paper is well written and easy to understand.\n\nWeaknesses\n1. Via the top-K approach, it appears like the authors make an implicit assumption about what fraction of edges is important in the graph (a fixed fraction, can't be lower or higher) - in my opinion, this certainly limits the wide applicability of the method. Alternatively, in GNN Explainer [1], they allow up to K edges to explain the label.\n2. The applicability of the method is limited by the graph encoder used (here GNN). 1-WL GNN's are known to be unable to predict links or identify and distinguish certain classes of subgraphs [2][3][4] i.e. GNN's are not well suited for Eq. 5 \n3. The work appears to miss relevant baselines like GNN Explainer[1] / CF-GNN Explainer [5]. Moreover the authors ideally should compare with baselines, which enrich GNN's with structural features [6][7][8] (As they are explainable to a certain extent as well).\n\nMinor:\n1. The separability of a graph into two subgraphs the causal and non causal might not always be possible? (would this need an encoder which is able to accurately capture the discrete topology over graphs of all orders and sizes - if not I can just make a house into a clique and the label would be incorrectly assigned as 1 because the edges required for the house are present) Please elaborate on this. \n2. The set  {s} employed in the test are limited to the ones seen in train.\n3. Moreover, consider a graph with say 20 nodes, and the case where the causal part for instance consists of a house motif and a tree base and the label assigned is 1 or 0 based on the output of House Motif XOR Tree Base - would this be captured by the proposed method (appears like it wont)?\n\n\n\nReferences:\n\n1.Ying, Rex, et al. \"Gnn explainer: A tool for post-hoc explanation of graph neural networks.\" arXiv preprint arXiv:1903.03894 (2019).\n\n2.Srinivasan, Balasubramaniam, and Bruno Ribeiro. \"On the equivalence between positional node embeddings and structural graph representations.\" arXiv preprint arXiv:1910.00452 (2019).\n\n3.Dwivedi, Vijay Prakash, et al. \"Benchmarking graph neural networks.\" arXiv preprint arXiv:2003.00982 (2020).\n\n4.Chen, Zhengdao, et al. \"Can graph neural networks count substructures?.\" arXiv preprint arXiv:2002.04025 (2020).\n\n5.Lucic, Ana, et al. \"CF-GNNExplainer: Counterfactual Explanations for Graph Neural Networks.\" arXiv preprint arXiv:2102.03322 (2021).\n\n6.Bouritsas, Giorgos, et al. \"Improving graph neural network expressivity via subgraph isomorphism counting.\" arXiv preprint arXiv:2006.09252 (2020).\n\n7.Bodnar, Cristian, et al. \"Weisfeiler and lehman go topological: Message passing simplicial networks.\" arXiv preprint arXiv:2103.03212 (2021).\n\n8.Bodnar, Cristian, et al. \"Weisfeiler and lehman go cellular: Cw networks.\" arXiv preprint arXiv:2106.12575 (2021).", "summary_of_the_paper": "The authors propose to improve generalization ability of GNN's for the task of graph classification via a causal theoretic analysis - under the assumption that only a subgraph of the graph is responsible for its label (and everything else are just spurious correlations). To this effect, they use a rationale generator and distribution intervener to test their model on a few synthetic datasets and a real world dataset.", "summary_of_the_review": "The paper is well written and the proposed solution is simple but there are still concerns about the wide applicability of the method, and also lacks comparisons with very related baselines.\n\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The work proposes a method to learn graph representations based on subgraphs that are invariant to spurious subgraphs. The reviewers found the paper easy to read and the theory interesting, well explained and justified. The reviewers seem happy with the existing and new experiments that came during the rebuttal phase. I too found the paper interesting and mostly well-written. 

Besides the corrections done during the rebuttal, in further discussion with the authors, I raised a concern that the work must make additional assumptions about the support of the induced subgraph distributions that were not clearly stated in the paper: The work makes the assumption that there is enough training data such that all spurious induced subgraph patterns $S$ that are smaller than the truly correlated induced subgraph $C$ can be identified as spurious. The authors promised to make this into a clearly demarcated assumption since it a key requirement for the method to work.
Decision: Accept (Poster)