Original: D ISCOVERING I NVARIANT R ATIONALES FOR G RAPH N EURAL N ETWORKS
Revised: D ISCOVERING I NVARIANT R ATIONALES FOR G RAPH N EURAL N ETWORKS

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Intrinsic interpretability of graph neural networks (GNNs) is to ﬁnd a small subset of the input graph’s features — rationale — which guides the model prediction.
Revised: Intrinsic interpretability of graph neural networks (GNNs) is to ﬁnd a small subset of the input graph’s features — rationale — which guides the model prediction.

Original: Unfortunately, the leading rationalization models often rely on data biases, especially shortcut features, to compose rationales and make predictions without probing the critical and causal patterns.
Revised: Unfortunately, the leading rationalization models often rely on data biases, especially shortcut features, to compose rationales and make predictions without probing the critical and causal patterns.

Original: Moreover, such data biases easily change outside the training distribution.
Revised: Moreover, such data biases easily change outside the training distribution.

Original: As a result, these models suffer from a huge drop in interpretability and predictive performance on out-of-distribution data.
Revised: As a result, these models suffer from a huge drop in interpretability and predictive performance on out-of-distribution data.

Original: In this work, we propose a new strategy of discovering invariant rationale (DIR) to construct intrinsically interpretable GNNs.
Revised: In this work, we propose a new strategy of discovering invariant rationale (DIR) to construct intrinsically interpretable GNNs.

Original: It conducts interventions on the training distribution to create multiple interventional distributions.
Revised: It conducts interventions on the training distribution to create multiple interventional distributions.

Original: Then it approaches the causal rationales that are invariant across different distributions while ﬁltering out the spurious patterns that are unstable.
Revised: Then it approaches the causal rationales that are invariant across different distributions while ﬁltering out the spurious patterns that are unstable.

Original: Experiments on both synthetic and realworld datasets validate the superiority of our DIR in terms of interpretability and generalization ability on graph classiﬁcation over the leading baselines.
Revised: Experiments on both synthetic and realworld datasets validate the superiority of our DIR in terms of interpretability and generalization ability on graph classiﬁcation over the leading baselines.

Original: Code and datasets are available at https://anonymous.4open.science/r/DIR/.
Revised: Code and datasets are available at https://github.com/Wuyxin/DIR-GNN.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: The eye-catching success in graph neural networks (GNNs) (Hamilton et al., 2017; Kipf & Welling, 2017; Dwivedi et al., 2020) provokes the rationalization task, which answers “What knowledge drives the model to make certain predictions?”.
Revised: The eye-catching success in graph neural networks (GNNs) (Hamilton et al., 2017; Kipf & Welling, 2017; Dwivedi et al., 2020) provokes the rationalization task, answering “What knowledge drives the model to make certain predictions?”.

Original: The goal of selective rationalization ( aka.
Revised: The goal of selective rationalization ( aka.

Original: feature attribution)
Revised: feature attribution)

Original: (Chang et al., 2020; Ying et al., 2019; Luo et al., 2020) is to ﬁnd a small subset of the input’s graph features — rationale — which best guides or explains the model prediction.
Revised: (Chang et al., 2020; Ying et al., 2019; Luo et al., 2020; Wang et al., 2021c) is to ﬁnd a small subset of the input’s graph features — rationale — which best guides or explains the model prediction.

Original: Discovering the rationale in a model can aid in auditing its inner workings and justifying its predictions.
Revised: Discovering the rationale in a model helps audit its inner workings and justify its predictions.

Original: Moreover, it has tremendous impacts on real-world applications, such as ﬁnding functional groups to shed light on protein structure prediction (Senior et al., 2020).
Revised: Moreover, it has tremendous impacts on real-world applications, such as ﬁnding functional groups to shed light on protein structure prediction (Senior et al., 2020).

Original: [Figure]
Revised: [Figure]

Original: n
Revised: BasesofHouseMotif

Original: [Figure]
Revised: [Figure]

Original: Two research lines of rationalization have recently emerged in GNNs.
Revised: Two research lines of rationalization have recently emerged in GNNs.

Original: The line of post-hoc explainability (Ying et al., 2019;Pope et al., 2019; Luo et al., 2020; Yuan et al., 2021) attributes a model’s prediction to the input graph with a separate explanation method, whilethe line of intrinsic interpretability (Veliˇckovi´c et al., 2018; Gao & Ji, 2019) incorporates a rationalization module into the model to make transparent predictions.
Revised: Post-hoc explainability (Ying et al., 2019; Luo et al., 2020; Yuan et al., 2021; Wang et al., 2021c) attributes a model’s prediction to the input graph with a separate explanation method, while intrinsic interpretability (Veliˇckovi´c et al., 2018; Gao & Ji, 2019) incorporates a rationalization module into the model to make transparent predictions.

Original: Here we focus on intrinsically interpretable GNNs.
Revised: Here we focus on intrinsically interpretable GNNs.

Original: Among them, graph attention (Veliˇckovi´c et al., 2018) and pooling (Lee et al., 2019; Knyazev et al., 2019; Gao & Ji, 2019;
Revised: Among them, graph attention (Veliˇckovi´c et al., 2018) and pooling (Lee et al., 2019; Knyazev et al., 2019; Gao & Ji, 2019;

Original: Ranjan et al., 2020) operators prevail, which work as a computational block of a GNN to generate soft or hard masks on the input graph ( e.g., edges, nodes).
Revised: Ranjan et al., 2020) operators prevail, which work as a computational block of a GNN to generate soft or hard masks on the input graph.

Original: They cast the learning paradigm of GNN as minimizing the prediction risk with the masked subgraphs, which are regarded as rationales to guide the model predictions.
Revised: They cast the learning paradigm of GNN as minimizing the empirical risk with the masked subgraphs, which are regarded as rationales to guide the model predictions.

Original: Despite the appealing nature, recent studies (Chang et al., 2020; Knyazev et al., 2019) show that the current rationalization methods are prone to exploit data biases as shortcuts to make predictions and compose rationales.
Revised: Despite the appealing nature, recent studies (Chang et al., 2020; Knyazev et al., 2019) show that the current rationalization methods are prone to exploit data biases as shortcuts to make predictions and compose rationales.

Original: Typically, shortcuts result from confounding factors, sampling biases, and artifacts in the training data.
Revised: Typically, shortcuts result from confounding factors, sampling biases, and artifacts in the training data.

Original: Considering
Revised: Considering

Original: Figure 1, when the most bases of House -motif graphs are Tree , a GNN does not need to learn the correct behaviors to reach high accuracy for the motif type.
Revised: Figure 1, when the most bases of House -motif graphs are Tree , a GNN does not need to learn the correct function to reach high accuracy for the motif type.

Original: Instead of looking at the motif and assessing its type, it is much easier to learn from the statistical shortcuts linking the bases Tree with the most occurring motifs House .
Revised: Instead, it is much easier to learn from the statistical shortcuts linking the bases Tree with the most occurring motifs House .

Original: Unfortunately, when facing with out-of-distribution (OOD) data, such methods generalize poorly since the shortcuts are changed.
Revised: Unfortunately, when facing with out-of-distribution (OOD) data, such methods generalize poorly since the shortcuts are changed.

Original: Hence, such shortcut-involved rationales hardly reveal the truly critical subgraphs for the predicted labels, being at odds with the true reasoning process that underlies the task of interest (Teney et al., 2020) and human cognition (Alvarez-Melis & Jaakkola, 2017).
Revised: Hence, such shortcut-involved rationales hardly reveal the truly critical subgraphs for the predicted labels, being at odds with the true reasoning process that underlies the task of interest (Teney et al., 2020) and human cognition (Alvarez-Melis & Jaakkola, 2017).

Original: Here we ascribe the failure on OOD data to the inability to identify causal patterns, which are stable to distribution shift.
Revised: Here we ascribe the failure on OOD data to the inability to identify causal patterns, which are stable to distribution shift.

Original: Motivated by recent studies on invariant learning (IL) (Arjovsky et al., 2019;
Revised: Motivated by recent studies on invariant learning (IL) (Arjovsky et al., 2019;

Original: Krueger et al., 2021; Chang et al., 2020; B¨uhlmann, 2018), we premise different distributions elicit different environments of data generating process.
Revised: Krueger et al., 2021; Chang et al., 2020; B¨uhlmann, 2018), we premise different distributions elicit different environments of data generating process.

Original: We argue that the causal patterns to the labels remain stable across environments, while the relations between the shortcut patterns and the labels vary.
Revised: We argue that the causal patterns to the labels remain stable across environments, while the relations between the shortcut patterns and the labels vary.

Original: Such environment-invariant patterns are more plausible and qualiﬁed as rationales.
Revised: Such environment-invariant patterns are more plausible and qualiﬁed as rationales.

Original: Aiming to identify rationales that capture the environment-invariant causal patterns, we formalize a learning strategy, Discovering Invariant Rationales (DIR), for intrinsically interpretable GNNs.
Revised: Aiming to identify rationales that capture the environment-invariant causal patterns, we formalize a learning strategy, Discovering Invariant Rationales (DIR), for intrinsically interpretable GNNs.

Original: One major problem is how to get multiple environments from a standard training set.
Revised: One major problem is how to get multiple environments from a standard training set.

Original: Differing from the heterogeneous setting (B¨uhlmann, 2018) of existing IL methods, where environments are observable and attainable, DIR does not assume prophets about environments.
Revised: Differing from the heterogeneous setting (B ¨ uhlmann, 2018) of existing IL methods, where environments are observable and attainable, DIR does not assume prophets about environments.

Original: It instead generates distribution perturbations by causal intervention — interventional distributions (Tian et al., 2006; Pearl et al., 2016) — to instantiate environments and further distinguish the causal and non-causal parts.
Revised: It instead generates distribution perturbations by causal intervention — interventional distributions (Tian et al., 2006; Pearl et al., 2016) — to instantiate environments and further distinguish the causal and non-causal parts.

Original: Guided by this idea, our DIR strategy consists of four modules: a rationale generator, a distribution intervener, a feature encoder, two classiﬁers.
Revised: Guided by this idea, our DIR strategy consists of four modules: a rationale generator, a distribution intervener, a feature encoder, two classiﬁers.

Original: Speciﬁcally, the rationale generator learns to split the input graph into causal and non-causal subgraphs, which are respectively encoded by the encoder into representations.
Revised: Speciﬁcally, the rationale generator learns to split the input graph into causal and non-causal subgraphs, which are respectively encoded by the encoder into representations.

Original: Then, the distribution intervener conducts the causal interventions on the non-causal representations to create perturbed distributions, with which we can infer the invariant causal parts.
Revised: Then, the distribution intervener conducts the causal interventions on the non-causal representations to create perturbed distributions, with which we can infer the invariant causal parts.

Original: Then, the two classiﬁers are respectively built upon the causal and non-causal parts to generate the joint prediction, whose invariant risk is minimized across different distributions.
Revised: Then, the two classiﬁers are respectively built upon the causal and non-causal parts to generate the joint prediction, whose invariant risk is minimized across different distributions.

Original: On one synthetic and three real datasets, extensive experiments demonstrate the generalization ability of DIR to surpass current state-of-the-art IL methods (Arjovsky et al., 2019; Krueger et al., 2021;
Revised: On one synthetic and three real datasets, extensive experiments demonstrate the generalization ability of DIR to surpass current state-of-the-art IL methods (Arjovsky et al., 2019; Krueger et al., 2021;

Original: Sagawa et al., 2019), and the interpretability of DIR to outperform the attention- and pooling-based rationalization methods (Veliˇckovi´c et al., 2018; Gao & Ji, 2019).
Revised: Sagawa et al., 2019), and the interpretability of DIR to outperform the attention- and pooling-based rationalization methods (Veli ˇ ckovi ´ c et al., 2018; Gao & Ji, 2019).

Original: Our main contributions are:
Revised: Our main contributions are:

Original: • We propose a novel invariant learning algorithm, DIR, for inherent interpretable models, improving the generalization ability and is suitable for any deep models.
Revised: • We propose a novel invariant learning algorithm, DIR, for inherent interpretable models, improving the generalization ability and is suitable for any deep models.

Original: • We offer causality theoretic analysis to guarantee the preeminence of DIR.
Revised: • We offer causality theoretic analysis to guarantee the preeminence of DIR.

Original: • We provide the implementation of DIR for graph classiﬁcation tasks, which consistently achieves excellent performance on three datasets with various generalization types.
Revised: • We provide the implementation of DIR for graph classiﬁcation tasks, which consistently achieves excellent performance on three datasets with various generalization types.

Original:  I NVARIANT R ATIONALE D ISCOVERY
Revised:  I NVARIANT R ATIONALE D ISCOVERY

Original: With a causal look at the data-generating process, we formalize the principle of discovering invariant rationales, which guides our discovery strategy.
Revised: With a causal look at the data-generating process, we formalize the principle of discovering invariant rationales, which guides our discovery strategy.

Original: Throughout the paper, upper-cased letters like G denote random variables, while lower-case letters like g denote deterministic value of variables.
Revised: Throughout the paper, upper-cased letters like G denote random variables, while lower-case letters like g denote deterministic value of variables.

Original: 2. C AUSAL V IEW OF D ATA -G ENERATING P ROCESS
Revised: 2. C AUSAL V IEW OF D ATA -G ENERATING P ROCESS

Original: Generating rationales for transparent predictions requires understanding the actual mechanisms of the task of interest.
Revised: Generating rationales for transparent predictions requires understanding the actual mechanisms of the task of interest.

Original: Without loss of generality, we focus on the graph classiﬁcation task and present a causal view of the data-generating process behind this task.
Revised: Without loss of generality, we focus on the graph classiﬁcation task and present a causal view of the data-generating process behind this task.

Original: Here we formalize the causal view as a Structure Causal Model (SCM) (Pearl et al., 2016; Pearl,2000) by inspecting on the causalities among four variables: input graph G , ground-truth label Y , causal part C , non-causal part S .
Revised: Here we formalize the causal view as a Structure Causal Model (SCM) (Pearl et al., 2016; Pearl, by inspecting on the causalities among four variables: input graph G , ground-truth label Y , causal part C , non-causal part S .

Original: Figure 2a illustrates the SCM, where each link denotes a causal relationship between two variables.
Revised: Figure 2a illustrates the SCM, where each link denotes a causal relationship between two variables.

Original: • C → G ← S .
Revised: • C → G ← S .

Original: The input graph G consists of two disjoint parts: the causal part C and the non-causal part S , such as the House motif and the Tree base in Figure 1.
Revised: The input graph G consists of two disjoint parts: the causal part C and the non-causal part S , such as the House motif and the Tree base in Figure 1.

Original: • C → Y .

Original: By “causal part”, we mean C is the only endogenous parent to determine the groundtruth label Y .

Original: Taking the motif-base example in Figure 1 again, C is the oracle rationale, which perfectly explains why the graph is labeled as Y .

Original: 𝑺 𝒀 𝑮 𝑪 𝑮 𝑺 𝑪 𝒀
Revised: 𝑺 𝒀 𝑮 𝑪 𝑮 𝑺 𝑪 𝒀

Original: House Cycle Crane
Revised: House Cycle Crane

Original: House Cycle Crane …
Revised: House Cycle Crane …

Original: Distributions

Original: House Cycle Crane
Revised: House Cycle Crane

Original: [Equation]
Revised: [Equation]

Original: ∅
Revised: ∅

Original: [Equation]
Revised: [Equation]

Original: 𝑺 𝒀 𝑮 𝑪
Revised: 𝑺 𝒀 𝑮 𝑪

Original: Original

Original: Multiple 𝒔 -Interventional Distributions
Revised: Multiple 𝒔 -Interventional Distributions

Original: [Figure]
Revised: [Figure]

Original: 𝑺 𝒀 𝑮 𝑪 𝑮 𝑺 𝑪 𝒀
Revised: 𝑺 𝒀 𝑮 𝑪 𝑮 𝑺 𝑪 𝒀

Original: House
Revised: House

Original: Cycle
Revised: Cycle

Original: Crane
Revised: Crane

Original: House
Revised: House

Original: Cycle
Revised: Cycle

Original: Crane
Revised: Crane

Original: [Figure]
Revised: [Figure]

Original: House
Revised: House

Original: Cycle
Revised: Cycle

Original: Crane
Revised: Crane

Original: [Figure]
Revised: [Equation]

Original: ∅
Revised: ∅

Original: [Equation]
Revised: [Equation]

Original: [Figure]

Original: [Equation]

Original: [Figure]

Original: [Figure]

Original: 𝑺 𝒀 𝑮 𝑪
Revised: 𝑺 𝒀 𝑮 𝑪

Original: [Figure]
Revised: [Figure]

Revised: • C → Y .

Revised: By “causal part”, we mean C is the only endogenous parent to determine the groundtruth label Y .

Revised: Taking the motif-base example in Figure 1 again, C is the oracle rationale, which perfectly explains why the graph is labeled as Y .

Revised: • C (cid:76)(cid:57)(cid:57)(cid:57)(cid:57)(cid:75) S .

Revised: This dashed arrow indicates additional probabilistic dependencies (Pearl, 2000; Pearl et al., 2016) between C and S .

Revised: We consider three typical relationships here: (1) C is independent of S , i.e., C | =

Revised: S ; (2) C is the direct cause of S ,

Original:  C 
Revised: i e., C → S ; and (3) There exists a common cause E , i e., C ← E → S .

Revised: See Appendix B for the corresponding examples.

Original: C (cid:76)(cid:57)(cid:57)(cid:57)(cid:57)(cid:75)
Revised: C (cid:76)(cid:57)(cid:57)(cid:57)(cid:57)(cid:75)

Original: S can create spurious correlations between the non-causal part S and the ground-truth label Y .
Revised: S can create spurious correlations between the non-causal part S and the ground-truth label Y .

Original: Assuming C → S , C is a confounder between S and Y , which opens a backdoor path
Revised: Assuming C → S , C is a confounder between S and Y , which opens a backdoor path

Original: S ← C → Y , thus making S and Y spuriously correlated (Pearl et al., 2016).
Revised: S ← C → Y , thus making S and Y spuriously correlated (Pearl et al., 2016).

Original: We systematize such spurious correlations as Y (cid:54)⊥⊥ S , where | = denotes probabilistic independence.
Revised: We systematize such spurious correlations as Y (cid:54)⊥⊥ S . Wherein, we make feature induction assumption on S to avoid the confusion of the induced subset of S between C . See Appendix C for the formal assumption.

Original: Furthermore, data collected from different environments exhibit various spurious correlations (Teney et al., 2020; Arjovsky et al., 2019), e.g., one mostly picks House motifs with Tree bases as the training data, while another selects House motifs with Wheel bases as the testing data.
Revised: Furthermore, data collected from different environments exhibit various spurious correlations (Teney et al., 2020; Arjovsky et al., 2019), e.g., one mostly picks House motifs with Tree bases as the training data, while another selects House motifs with Wheel bases as the testing data.

Original: Hence, such spurious correlations are unstable and variant across different distributions.
Revised: Hence, such spurious correlations are unstable and variant across different distributions.

Original: 2. T ASK F ORMALIZATION OF I NVARIANT R ATIONALIZATION
Revised: 2. T ASK F ORMALIZATION OF I NVARIANT R ATIONALIZATION

Original: Oracle Rationale.
Revised: Oracle Rationale.

Original: With the causal theory (Pearl et al., 2016; Pearl, 2000), for each variable X in a
Revised: With the causal theory (Pearl et al., 2016; Pearl, 2000), for each variable X in a

Original: SCM, there exists a directed link from each of its parent variables PA ( X ) to X , if and only if the causal mechanism X = f X
Revised: SCM, there exists a directed link from each of its parent variables PA ( X ) to X , if and only if the causal mechanism X = f X

Original: ( PA ( X ) , (cid:15) X ) persists, where (cid:15) X | = PA ( X ) is the exogenous noise of
Revised: ( PA ( X ) , (cid:15) X ) persists, where (cid:15) X | = PA ( X ) is the exogenous noise of

Original: X .
Revised: X .

Original: For simplicity, we omit the exogenous noise and simplify it as X = f X ( PA ( X )) .
Revised: For simplicity, we omit the exogenous noise and simplify it as X = f X ( PA ( X )) .

Original: Hence, there exist a function f Y : C → Y in our SCM, where the “oracle rationale” C satisﬁes:
Revised: Hence, there exist a function f Y : C → Y in our SCM, where the “oracle rationale” C satisﬁes:

Original: [Equation]
Revised: [Equation]

Original: where Y | =
Revised: where Y | =

Original: S | C indicates that C shields Y from the inﬂuence of S , making the causal relationship
Revised: S | C indicates that C shields Y from the inﬂuence of S , making the causal relationship

Original: C → Y invariant across different S .
Revised: C → Y invariant across different S .

Original: Rationalization.
Revised: Rationalization.

Original: In general, only the pairs of input G and label Y are observed during training, while neither oracle rationale C nor oracle structural equation model f Y is available.
Revised: In general, only the pairs of input G and label Y are observed during training, while neither oracle rationale C nor oracle structural equation model f Y is available.

Original: The absence of oracles calls for the study on intrinsic interpretability.
Revised: The absence of oracles calls for the study on intrinsic interpretability.

Original: Following the rationalization paradigm, we can systematize an intrinsically-interpretable GNNh as a combination of two modules hˆ Y ◦ h ˜ C . That is, h ( G ) = 
Revised: We systematize an intrinsically-interpretable GNN as a combination of two modules, i.e., h = h

Original: h ˆ Y ◦ h ˜ C( G ) , where h ˜ C : G → ˜ C discovers rationale ˜ C from the observed G , and h ˆ Y : ˜ C
Revised: ˆ Y ◦ h ˜ C , where h ˜ C : G → ˜ C discovers rationale ˜ C from the observed G , and h ˆ Y : ˜ C

Original: → ˆ Y outputs the prediction ˆ Y to approach Y .
Revised: → ˆ Y outputs the prediction ˆ Y to approach Y .

Original: Distinct from C and Y which are the variables in the causal mechanisms, ˜ C and ˆ Y represent the variables in the modeling process to approximate C and Y .
Revised: Distinct from C and Y which are the variables in the causal mechanisms, ˜ C and ˆ Y represent the variables in the modeling process to approximate C and Y .

Original: To optimize these modules, most of current intrinsically-interpretable GNNs (Veliˇckovi´c et al., 2018; Lee et al., 2019; Knyazev et al., 2019; Gao & Ji, 2019; Ranjan et al., 2020) adopt the learning strategy of minimizing the empirical risk:
Revised: To optimize these modules, most of current intrinsicallyinterpretable GNNs (Veliˇckovi´c et al., 2018; Lee et al., 2019; Knyazev et al., 2019; Gao & Ji, 2019; Ranjan et al., 2020) adopt the learning strategy of minimizing the empirical risk:

Original: [Equation]
Revised: [Equation]

Original: where R ( · , · ) is the risk function, which can be the cross-entropy loss.
Revised: where R ( · , · ) is the risk function, which can be the cross-entropy loss.

Original: Nevertheless, this learning strategy relies heavily on the statistical associations between the input features and labels, and can potentially exhibit non-causal rationales.
Revised: Nevertheless, this learning strategy relies heavily on the statistical associations between the input features and labels, and can potentially exhibit non-causal rationales.

Original: Invariant Rationalization.
Revised: Invariant Rationalization.

Original: We ascribe the limitation to ignoring Y | =
Revised: We ascribe the limitation to ignoring Y | =

Original: S | C in Equation 1, which is crucial to reﬁne the causal relationship C
Revised: S | C in Equation 1, which is crucial to reﬁne the causal relationship C

Original: → Y that is invariant across different S .
Revised: → Y that is invariant across different S .

Original: By introducing this independence, we formalize the task of invariant rationalization as:
Revised: By introducing this independence, we formalize the task of invariant rationalization as:

Original: [Equation]
Revised: [Equation]

Original: where ˜ S =
Revised: where ˜ S =

Original: G \ ˜ C is the complement of ˜ C .
Revised: G \ ˜ C is the complement of ˜ C .

Original: This formulation encourages the rationale ˜ C seeking the patterns that are stable across different distributions, while discarding the unstable patterns.
Revised: This formulation encourages the rationale ˜ C seeking the patterns that are stable across different distributions, while discarding the unstable patterns.

Original: 2. P RINCIPLE & L EARNING S TRATEGY OF DIR
Revised: 2. P RINCIPLE & L EARNING S TRATEGY OF DIR

Original: Interventional Distribution.
Revised: Interventional Distribution.

Original: However, it is difﬁcult to recover the oracle rationale from the joint distribution over the inputs and labels — that is, the causal and non-causal relations are hardly distinguished from each other.
Revised: However, it is difﬁcult to recover the oracle rationale from the joint distribution over the inputs and labels — that is, the causal and non-causal relations are hardly distinguished from each other.

Original: We get inspirations from invariant learning (Arjovsky et al., 2019;
Revised: We get inspirations from invariant learning (Arjovsky et al., 2019;

Original: Krueger et al., 2021;
Revised: Krueger et al., 2021;

Original: Chang et al., 2020), which constructs different environments to infer the invariant features or predictors.
Revised: Chang et al., 2020), which constructs different environments to infer the invariant features or predictors.

Original: To obtain the environments, previous studies mostly partition the training set by prior knowledge (Teney et al., 2020) or adversarial environment inference (Creager et al., 2021; Wang et al., 2021b).
Revised: To obtain the environments, previous studies mostly partition the training set by prior knowledge (Teney et al., 2020) or adversarial environment inference (Creager et al., 2021; Wang et al., 2021b).

Original: Different from partitioning the training data, we do not assume prophets about environments but introduce the interventional distribution (Tian et al., 2006; Pearl et al., 2016) instead to model the DIR task.
Revised: Different from partitioning the training data, we do not assume prophets about environments but introduce the interventional distribution (Tian et al., 2006; Pearl et al., 2016) instead to model the DIR task.

Original: Speciﬁcally, on the top of our SCM, we generate s -interventional distribution by doing intervention do ( S = s ) on S , which removes every link from the parents PA ( S ) to the variable S and ﬁxes S to the speciﬁc value s .
Revised: Speciﬁcally, on the top of our SCM, we generate s -interventional distribution by doing intervention do ( S = s ) on S , which removes every link from the parents PA ( S ) to the variable S and ﬁxes S to the speciﬁc value s .

Original: By stratifying different values S = { s } , we can obtain multiple s -interventional distributions.
Revised: By stratifying different values S = { s } , we can obtain multiple s -interventional distributions.

Original: With interventional distributions, we propose the principle of discovering invariant rationale (DIR) to identify a rationale ˜ C whose relationship with the label Y is stable across different distributions.
Revised: With interventional distributions, we propose the principle of d iscovering invariant rationale (DIR) to identify a rationale ˜ C whose relationship with the label Y is stable across different distributions.

Original: Deﬁnition 1 (DIR Principle)
Revised: Deﬁnition 1 (DIR Principle)

Original: An intrinsically-interpretable model h satisﬁes the DIR principle if it1.
Revised: An intrinsically-interpretable model h satisﬁes the DIR principle if it

Original: minimizes all s -interventional risks: E s [ R ( h ( G ) , Y | do ( S = s ))] , and simultaneously 2.
Revised: minimizes all s -interventional risks: E s [ R ( h ( G ) , Y | do ( S = s ))] , and simultaneously 2.

Original: minimizes the variance of various s -interventional risks:
Revised: minimizes the variance of various s -interventional risks:

Original: Var s ( {R ( h ( G ) , Y | do ( S = s )) } ) , where the s -interventional risk is deﬁned over the s -interventional distribution for speciﬁc s ∈ S .
Revised: Var s ( {R ( h ( G ) , Y | do ( S = s )) } ) , where the s -interventional risk is deﬁned over the s -interventional distribution for speciﬁc s ∈ S .

Original: Guided by the proposed principle, we design the learning strategy of DIR as:
Revised: Guided by the proposed principle, we design the learning strategy of DIR as:

Original: [Equation]
Revised: [Equation]

Original: where R ( h ( G ) , Y | do ( S = s )) computes the risk under the s -interventional distribution, which we will elaborate in Section 2.4.
Revised: where R ( h ( G ) , Y | do ( S = s )) computes the risk under the s -interventional distribution, which we will elaborate in Section 2.4.

Original: Var ( · ) calculates the variance of risks over different s -interventional distributions; λ is a hyper-parameter to control the strength of invariant learning.
Revised: Var ( · ) calculates the variance of risks over different s -interventional distributions; λ is a hyper-parameter to control the strength of invariant learning.

Original: Justiﬁcation.
Revised: Justiﬁcation.

Original: We theoretically justify the DIR principle’s ability to discover invariant rationales.
Revised: We theoretically justify the DIR principle’s ability to discover invariant rationales.

Original: Speciﬁcally, Theorem 1 shows that the oracle model f Y respects the DIR principle.
Revised: Speciﬁcally, Theorem 1 shows that the oracle model f Y respects the DIR principle.

Original: Moreover, we suggest that C can be inferred by making the intrinsically interpretable model h conform to the DIR principle under the uniqueness condition ( cf.
Revised: Moreover, we suggest that C can be inferred by making the intrinsically interpretable model h conform to the DIR principle under the uniqueness condition ( cf.

Original: Corollary 1).
Revised: Corollary 1).

Original: We leave the detailed proofs in Appendix
Revised: We leave the detailed proofs in Appendix

Original: C due to the limited space.
Revised: C due to the limited space.

Original: By making the distribution-relevant risks indifferent while pursuing low risks, the DIR principle is able to discover the invariant rationales ˜ C as the approximation of the oracle rationales C , while encouraging h ˆ Y approaching the oracle model f Y .
Revised: By making the distribution-relevant risks indifferent while pursuing low risks, the DIR principle is able to discover the invariant rationales ˜ C as the approximation of the oracle rationales C , while encouraging h ˆ Y approaching the oracle model f Y .

Original: 2. DIR-G UIDED I MPLEMENTATION OF I NTRINSICALLY -I NTERPRETABLE GNN S
Revised: 2. DIR-G UIDED I MPLEMENTATION OF I NTRINSICALLY -I NTERPRETABLE GNN S

Original: With the DIR principle and objective, we present how to implement the intrinsically-interpretable
Revised: With the DIR principle and objective, we present how to implement the intrinsically-interpretable

Original: GNNs.
Revised: GNNs.

Original: We summarize the key notations of this section in Appendix A for clarity.
Revised: We summarize the key notations of this section in Appendix A for clarity.

Original: Following Equation 2, a model h with intrinsic interpretability consists of two modules: h =
Revised: Following Equation 2, a model h with intrinsic interpretability consists of two modules: h =

Original: h ˆ Y ◦ h ˜ C , where h ˜ C is to extract a possible rationale, and h ˆ Y is to make prediction based on the rationale.
Revised: h ˆ Y ◦ h ˜ C , where h ˜ C is to extract a possible rationale, and h ˆ Y is to make prediction based on the rationale.

Original: Moreover, to establish the s -interventional distributions, we design an additional module to do the interventions.
Revised: Moreover, to establish the s -interventional distributions, we design an additional module to do the interventions.

Original: In a nutshell, our framework consists of four components, as Figure 3 shows.
Revised: In a nutshell, our framework consists of four components, as Figure 3 shows.

Original: Rationale Generator.
Revised: Rationale Generator.

Original: It aims to split the input graph instance g into two subgraphs:  ˜ c and  ˜ s , which try to reveal the causal and non-causal features, respectively.
Revised: It aims to split the input graph instance g into two subgraphs: causal part ˜ c and non-causal part ˜ s .

Original: Speciﬁcally, given an input graph instance g
Revised: Speciﬁcally, given an input graph instance g

Original: =
Revised: =

Original: ( V , E ) with the node set Vand the edge set E , its adjacency matrix is A ∈ { 0 , 1 } |V|×|V| , 𝒄� 𝒔�
Revised: ( V , E ) with the node set V 𝒄� 𝒔�

Original: RationaleGenerator 𝒉 𝑪 𝑮
Revised: Rationale Generator 𝒉 𝑪 𝑮

Original: Shared Encoder 𝒉 𝜽 𝒁 𝑪 𝒁 𝑺𝒀� 𝑪 …… … 𝒚� 𝒔� 𝒀�
Revised: Shared Encoder 𝒉 𝜽 𝒁 𝑪 𝒁 𝑺 … 𝒚� 𝒔� 𝒀�

Original: Classifier 𝚽 Shortcut Prediction
Revised: Classifier 𝚽 Shortcut Prediction

Original: Causal Prediction
Revised: Causal Prediction

Original: Intervener 𝓡 ��� 𝓡 𝑺� 𝒀
Revised: Intervener 𝓡 ��� 𝓡 𝑺� 𝒀

Original: No BP … ~ ~ ~ ~ ~ ~ ~ ~ 𝒉 𝑰 𝑯 𝑺 ~ 𝑯 𝑺 ~ ~ ~𝒈
Revised: No BP … ~ ~ ~ ~ ~ ~ ~ ~ 𝒉 𝑰 𝑯 𝑺 ~ 𝑯 𝑺 ~ ~ ~

Original: RationaleGenerator
Revised: [Figure]

Original: SharedGraphEncoder 𝒉 𝑪
Revised: Shared Graph [Figure]

Original: ShortcutClassifier
Revised: [Figure]

Original: Causal Classifier 𝒚� 𝒄� 𝒚� 𝒚
Revised: 𝒚� 𝒄� 𝒚� 𝒚

Original: DistributionIntervener
Revised: DistributionIntervener

Original: InvariantRationale
Revised: InvariantRationale

Original: CausalPrediction
Revised: CausalPrediction

Original: No BP where A ij = 1 denotes the edge from node i to node j , and A ij = 0 otherwise.
Revised: No [Figure] and the edge set E , its adjacency matrix is A ∈ { 0 , 1 } |V|×|V| , where A ij = 1 denotes the edge from node i to node j , and A ij = 0 otherwise.

Original: The rationale generator ﬁrst adopts a GNN to generate the mask matrix M ∈ R |V|×|V| on A , where mask M ij indicates the importance of edge A ij : 
Revised: The rationale generator ﬁrst adopts a GNN to generate the mask matrix M ∈ R |V|×|V| on A , where mask M ij indicates the importance of edge A ij : [Equation]

Original: [Equation]
Revised: [Equation]

Original: ) where σ ( · ) is the sigmoid function and Z ∈ R |V|× d summarizes the d -dimensional representations of all nodes.
Revised: where σ ( · ) is the sigmoid function and Z ∈ R |V|× d summarizes the d -dimensional representations of all nodes.

Original: The generator then selects the edges with the highest masks to construct the rationale ˜ c and collects ˜ c ’s complement as ˜ s , as follows:
Revised: The generator then selects the edges with the highest masks to construct the rationale ˜ c and collects ˜ c ’s complement as ˜ s , as follows:

Original: [Equation]
Revised: [Equation]

Original: where E ˜ c and E ˜ s are the edge sets of ˜ c and ˜ s , respectively; Top r ( · ) selects the top- K edges with
Revised: where E ˜ c and E ˜ s are the edge sets of ˜ c and ˜ s , respectively; Top r ( · ) selects the top- K edges with

Original: K = r × |E| , and r is the hyper-parameter
Revised: K = r × |E| , and r is the hyper-parameter

Original: ( e.g., 40% ); (cid:12) is the element-wise product.
Revised: ( e.g., 40% ); (cid:12) is the element-wise product.

Original: Having obtained the edge sets, we can distill the nodes appearing in the edges to establish ˜ c and ˜ s .
Revised: Having obtained the edge sets, we can distill the nodes appearing in the edges to establish ˜ c and ˜ s .

Original: Distribution Intervener.
Revised: Distribution Intervener.

Original: It targets at creating interventional distributions.
Revised: It targets at creating interventional distributions.

Original: Formally, it ﬁrst collects ˜ s of all the instancescomputed previously into a memory bank as ˜ S 
Revised: Formally, it ﬁrst collects the non-causal part of all the instances into a memory bank as ˜ S .

Original: = { ˜ s } .

Original: It next samples a memory ˜ s i ∈ ˜ S to conduct the intervention do ( S = ˜
Revised: It next samples a memory ˜ s i ∈ ˜ S to conduct the intervention do ( S = ˜

Original: s i ) , replacing the complement of the critical subgraph ˜
Revised: s i ) , replacing the complement of the critical subgraph ˜

Original: c j at hand and constructing an intervened pair ( ˜ c j , ˜ s i ), where i, j are indices.
Revised: c j at hand and constructing an intervened pair ( ˜ c j , ˜ s i ), where i, j are indices.

Original: Graph Encoder & Classiﬁers .
Revised: Graph Encoder & Classiﬁers .

Original: Here we represent h ˆ Y as a combination of a graph encoder and two classiﬁers.
Revised: Here we represent h ˆ Y as a combination of a graph encoder and two classiﬁers.

Original: Speciﬁcally, it employs another GNN encoder on ˜ c to generate node representations
Revised: Speciﬁcally, it employs another GNN encoder on ˜ c to generate node representations

Original: Z ˜ c ∈ R |V|× d , and then combines them as graph representation H ˜ c ∈ R D via a global pooling operator ( i.e., average pooling).
Revised: Z ˜ c ∈ R |V|× d , and then combines them as graph representation H ˜ c ∈ R D via a global pooling operator, e.g., average pooling.

Original: Then it uses a classiﬁer Φ 1 to project the graph representation into a probability distribution over class labels ˆ y ˜ c .
Revised: Then it uses a classiﬁer Φ c to project the graph representation into a probability distribution over class labels ˆ y ˜ c .

Original: More formally, the process is as follows:
Revised: More formally, the process is as follows:

Original: [Equation]
Revised: [Equation]

Original: Analogously, we can obtain ˆ y ˜ s for ˜ s via the shared encoder and another classiﬁer Φ 2 .
Revised: Analogously, we can obtain ˆ y ˜ s for ˜ s via the shared encoder and another classiﬁer Φ s .

Original: ˆ y ˜ c is the prediction based merely on the causal part ˜ c , while ˆ y ˜ s measures the predictive power of the intervened part ˜ s .
Revised: ˆ y ˜ c is the prediction based merely on the causal part ˜ c , while ˆ y ˜ s measures the predictive power of the intervened part ˜ s .

Original: Inspired by Cad ` ene et al.
Revised: Inspired by Cad`ene et al.

Original: (2019), we formulate the joint prediction ˆ
Revised: (2019), we formulate the joint prediction ˆ

Original: y
Revised: y

Original: under the intervention do ( S = ˜ s ) as ˆ y ˜ c masked by ˆ y ˜ s :
Revised: under the intervention do ( S = ˜ s ) as ˆ y ˜ c masked by ˆ y ˜ s :

Original: [Equation]
Revised: [Equation]

Original: where the sigmoid function adjusts the output logits of ˜ c to compensate for the spurious biases.
Revised: where the sigmoid function adjusts the output logits of ˜ c to compensate for the spurious biases.

Original: In
Revised: In

Original: Appendix E, we present examples of how this operation helps discover the causal part.
Revised: Appendix E, we present examples of how this operation helps discover the causal part.

Original: Optimization.
Revised: Optimization.

Original: Having established the prediction ˆ y of an instance g under the intervention do ( S = ˜ s ) , we are capable of getting the ˜ s -interventional risk similar as Equation 4 as follows
Revised: Having established the prediction ˆ y of an instance g under the intervention do ( S = ˜ s ) , we are capable of getting the ˜ s -interventional risk similar as Equation 4 as follows:

Original: [Equation]
Revised: [Equation]

Original: where ( g, y ) ∈
Revised: where ( g, y ) ∈

Original: O is a pair of graph instance g and its ground-truth label y from the training set O ; l ( · ) denotes the loss function on a single instance.
Revised: O is a pair of graph instance g and its ground-truth label y from the training set O ; l ( · ) denotes the loss function on a single instance.

Original: Moreover, we deﬁne the loss for Φ 2 module as:
Revised: Moreover, we deﬁne the loss for Φ s module as:

Original: [Equation]
Revised: [Equation]

Original: Speciﬁcally, R ˜ S is only backpropagated to the classiﬁer Φ 2 and we set apart the other components from its backpropagation to avoid interference with representation learning.
Revised: Speciﬁcally, R ˜ S is only backpropagated to the classiﬁer Φ s and we set apart the other components from its backpropagation to avoid interference with representation learning.

Original: Thus, this loss promotes the ˜ S -only branch to learn spurious biases given the non-causal features only.
Revised: Thus, this loss promotes the ˜ S -only branch to learn spurious biases given the non-causal features only.

Original: Overall, we can jointly optimize these components via the DIR objective and shortcut loss, i.e.,
Revised: Overall, we can jointly optimize these components via the DIR objective and shortcut loss, i.e.,

Original: [Equation]
Revised: [Equation]

Original:  where γ, θ and ( φ 1 , φ 2 ) are the parameters of the generator, encoder and two classiﬁers.
Revised: (11) where γ, θ and ( φ c , φ s ) are the parameters of the generator, encoder and two classiﬁers.

Original: While in the inference phase, we yield ˜ c and ˆ y ˜ c as the causal rationale and the causal prediction of a testing graph g , which exclude the inﬂuence of the non-causal part ˜ s .
Revised: While in the inference phase, we yield ˜ c and ˆ y ˜ c as the causal rationale and the causal prediction of a testing graph g , which exclude the inﬂuence of the non-causal part ˜ s .

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: In this section we conduct extensive experiments to answer the research questions:
Revised: In this section, we conduct extensive experiments to answer the research questions:

Original: • RQ1: How effective is DIR in discovering causal features and improving model generalization?
Revised: • RQ1: How effective is DIR in discovering causal features and improving model generalization?

Original: • RQ2: What are the learning patterns and insights of DIR training?
Revised: • RQ2: What are the learning patterns and insights of DIR training?

Original: Especially, how does invariant rationalization help to improve generalization?
Revised: Especially, how does invariant rationalization help to improve generalization?

Original: 3. S ETTINGS
Revised: 3. S ETTINGS

Original: Datasets.
Revised: Datasets.

Original: We use one synthetic dataset and three real datasets of graph classiﬁcation tasks.
Revised: We use one synthetic dataset and three real datasets of graph classiﬁcation tasks.

Original: Different
Revised: Different

Original: GNNs are used in different datasets to achieve DIR and early stopping is exploited during training.
Revised: GNNs are used in different datasets to achieve DIR and early stopping is exploited during training.

Original: Here we brieﬂy introduce the datasets, while the details of dataset statistics, deployed GNNs, and training process are summarized in Appendix D.
Revised: Here we brieﬂy introduce the datasets, while the details of dataset statistics, deployed GNNs, and training process are summarized in Appendix D.

Original: • Spurious-Motif is a synthetic dataset created by following Ying et al.
Revised: • Spurious-Motif is a synthetic dataset created by following Ying et al.

Original: (2019), which involves 18 , 000 graphs.
Revised: (2019), which involves 18 , 000 graphs.

Original: Each graph is composed of one base ( Tree , Ladder , Wheel denoted by S = 0 , 1 ,respectively) and one motif ( Cycle , House , Crane denoted by C = 0 , 1 , 2 , respectively).
Revised: Each graph is composed of one base ( Tree , Ladder , Wheel denoted by S = 0 , 1 ,respectively) and one motif ( Cycle , House , Crane denoted by C = 0 , 1 , 2 , respectively).

Original: The ground-truth label Y is determined by C solely.
Revised: The ground-truth label Y is determined by C solely.

Original: Moreover, we manually construct false relations of different degrees between S and label Y in the training set.
Revised: Moreover, we manually construct false relations of different degrees between S and label Y in the training set.

Original: Speciﬁcally, in the training set, we sample each motif from a uniform distribution, while the distribution of its base is determined by:
Revised: Speciﬁcally, in the training set, we sample each motif from a uniform distribution, while the distribution of its base is determined by P ( S )

Original: [Equation]{+= b × I ( S = C ) + 1 − b 
 ×+}
Revised: {+= b × I ( S = C ) + 1 − b 
 ×+}

Original:  We manipulate bin Equation 12 to create Spurious-Motif datasets of distinct biases.
Revised: I ( S (cid:54) = C ) . We manipulate b to create Spurious-Motif datasets of distinct biases.

Original: In the testing set, the motifs and bases are randomly attached to each other.
Revised: In the testing set, the motifs and bases are randomly attached to each other.

Original: Besides, we include graphs with large bases to further magnify the distribution gaps.
Revised: Besides, we include graphs with large bases to further magnify the distribution gaps.

Original: • MNIST-75sp (Knyazev et al., 2019) converts the MNIST images into 70 , 000 superpixel graphs with at most 75 nodes each graph.
Revised: • MNIST-75sp (Knyazev et al., 2019) converts the MNIST images into 70 , 000 superpixel graphs with at most 75 nodes each graph.

Original: The nodes in the graphs are superpixels, while edges are the spatial distance between the nodes.
Revised: The nodes in the graphs are superpixels, while edges are the spatial distance between the nodes.

Original: Every graph is labeled as one of 10 classes.
Revised: Every graph is labeled as one of 10 classes.

Original: Random noises are added to nodes’ features in the testing set.
Revised: Random noises are added to nodes’ features in the testing set.

Original: • Graph-SST

Revised: • Graph-SST

Original: (Yuan et al., 2020;
Revised: (Yuan et al., 2020;

Original: Socher et al., 2013)
Revised: Socher et al., 2013)

Original: Each graph is labeled by its sentence sentiment and consists of nodes representing tokens and edges indicating node relations.
Revised: Each graph is labeled by its sentence sentiment and consists of nodes representing tokens and edges indicating node relations.

Original: Graphs are split into different sets according to their average node degree to create dataset shifts.
Revised: Graphs are split into different sets according to their average node degree to create dataset shifts.

Original: • Molhiv (OGBG-Molhiv)
Revised: • Molhiv (OGBG-Molhiv)

Original: (Hu et al., 2020; 2021; Wu et al., 2017) is a molecular property prediction dataset consisting of molecule graphs, where nodes are atoms, and edges are chemical bonds.
Revised: (Hu et al., 2020; 2021; Wu et al., 2017) is a molecular property prediction dataset consisting of molecule graphs, where nodes are atoms, and edges are chemical bonds.

Original: Each graph is labeled according to whether a molecule inhibits HIV replication or not.
Revised: Each graph is labeled according to whether a molecule inhibits HIV replication or not.

Original: Baselines.
Revised: Baselines.

Original: We thoroughly compare DIR with Empirical Risk Minimization (ERM) andthe following two classes of baselines:
Revised: We thoroughly compare DIR with Empirical Risk Minimization (ERM) and two classes of baselines:

Original: • Interpretable Baselines: Graph Attention (Veliˇckovi´c et al., 2018) and graph pooling operations including ASAP (Ranjan et al., 2020), Top- k Pool (Gao & Ji, 2019) and SAG Pool (Lee et al., 2019).
Revised: • Interpretable Baselines: Graph Attention (Veliˇckovi´c et al., 2018) and graph pooling operations including ASAP (Ranjan et al., 2020), Top- k Pool (Gao & Ji, 2019) and SAG Pool (Lee et al., 2019).

Original: We use their generated masks on graph structures as rationales.
Revised: We use their generated masks on graph structures as rationales.

Original: We also include GSN (Bouritsas et al., 2020), a topologically-aware message passing scheme which enriches GNNs with interpretable structural features.
Revised: We also include GSN (Bouritsas et al., 2020), a topologically-aware message passing scheme which enriches GNNs with interpretable structural features.

Original: • Robust/Invariant Learning Baselines: Group DRO (Sagawa et al., 2019), IRM (Arjovsky et al., 2019), V-REx (Krueger et al., 2021).
Revised: • Robust/Invariant Learning Baselines: Group DRO (Sagawa et al., 2019), IRM (Arjovsky et al., 2019), V-REx (Krueger et al., 2021).

Original: This class of algorithms improves the robustness and generalization for GNNs, which helps the models better generalize in unseen groups or out-ofdistribution datasets.
Revised: This class of algorithms improves the robustness and generalization for GNNs, which helps the models better generalize in unseen groups or out-ofdistribution datasets.

Original: We use random groups or partitions during the model training.
Revised: We use random groups or partitions during the model training.

Original: We also include an ablation model of DIR, DIR-Var, which sets λ = 0 ( i.e., discards the variance term in R DIR ), to show the effectiveness of the variance regularization in the DIR objective.
Revised: We also include an ablation model of DIR, DIR-Var, which sets λ = 0 , i.e., discards the variance term in R DIR , to show the effectiveness of the variance regularization in the DIR objective.

Original: Metrics.
Revised: Metrics.

Original: We use ROC-AUC for Molhiv and accuracy (ACC) for the other three datasets.
Revised: We use ROC-AUC for Molhiv and ACC for the other three datasets.

Original: Moreover, for Spurious-Motif dataset, we use the precision metric to evaluate the coincidence between model rationales and the ground-truth rationales, so as to validate the interpretability ability quantitatively.
Revised: Moreover, for Spurious-Motif dataset, we use the precision metric to evaluate the coincidence between model rationales and the ground-truth rationales, and validate the interpretability ability quantitatively.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 3. M AIN R ESULTS (RQ1)
Revised: 3. M AIN R ESULTS (RQ1)

Original: To fairly compare the methods
Revised: To fairly compare the methods

Original: [Table]
Revised: [Table]

Original: we train each model under the same training settings as described in
Revised: we train each model under the same training settings as described in

Original: Appendix D.
Revised: Appendix D.

Original: The overall results are summarized Table [Table] and we have the following observations:
Revised: The overall results are summarized Table 1, and we have the following observations:

Original: DIR has better generalization ability than the baselines.
Revised: DIR has better generalization ability than the baselines.

Original: DIR outperforms the baselines consistently by a large margin.
Revised: DIR outperforms the baselines consistently by a large margin.

Original: Speciﬁcally, for MNIST-75sp dataset, DIR surpasses ERM by 7.65% and ASAP by 4.82%.
Revised: Speciﬁcally, for MNIST-75sp dataset, DIR surpasses ERM by 7.65% and ASAP by 4.82%.

Original: Although structure features are shown to be helpful in mitigating feature distribution shift, DIR still performs better than GSN.
Revised: Although structure features are shown to be helpful in mitigating feature distribution shift, DIR still performs better than GSN.

Original: For Graph-SST2 and Molhiv, DIR achieves the highest performance with low variance.
Revised: For Graph-SST2 and Molhiv, DIR achieves the highest performance with low variance.

Original: For Spurious-Motif, DIR outstrips IRM averagely by 4.23% and SAG by 3.16% across different degrees of spurious bias.
Revised: For Spurious-Motif, DIR outstrips IRM averagely by 4.23% and SAG by 3.16% across different degrees of spurious bias.

Original: Such improvements strongly validate that DIR can generalize better in various environments.
Revised: Such improvements strongly validate that DIR can generalize better in various environments.

Original: DIR is consistently effective under different bias degrees, while the baselines easily fail.
Revised: DIR is consistently effective under different bias degrees, while the baselines easily fail.

Original: For interpretable baselines, Attention fails to make salient improvements when bias exists, and pooling methods also fall through under severe bias.
Revised: For interpretable baselines, Attention fails to make salient improvements when bias exists, and pooling methods also fall through under severe bias.

Original: This is empirically in line with our presumption that GNNs are easily biased to latch on spurious relations or non-causal features and thus generalize poorly in OOD data.
Revised: This is empirically in line with our presumption that GNNs are easily biased to latch on spurious relations or non-causal features and thus generalize poorly in OOD data.

Original: For robust/invariant learning baselines, IRM underperforms ERM when b is small.
Revised: For robust/invariant learning baselines, IRM underperforms ERM when b is small.

Original: This evidence is accordant with the conclusion in Ahuja et al.
Revised: This evidence is accordant with the conclusion in Ahuja et al.

Original: (2021) that IRM is guaranteed to be close to the desired OOD solutions when confounders exist, while it has no obvious advantage to ERM under covariate shift.
Revised: (2021) that IRM is guaranteed to be close to the desired OOD solutions when confounders exist, while it has no obvious advantage to ERM under covariate shift.

Original: Moreover, Group DRO and V-REx follow a similar pattern.
Revised: Moreover, Group DRO and V-REx follow a similar pattern.

Original: In contrast, DIR works well in various scenarios.
Revised: In contrast, DIR works well in various scenarios.

Original: We credit such reliability to the rationales discovery from which the causal features C are potentially extracted, and the relation C → Y learned by the GNNs is invariant across the distribution changes in the testing set.
Revised: We credit such reliability to the rationales discovery from which the causal features C are potentially extracted, and the relation C → Y learned by the GNNs is invariant across the distribution changes in the testing set.

Original: Data augmentation by intervention is beneﬁcial while the variance regularization further boosts model performance.
Revised: Data augmentation by intervention is beneﬁcial while the variance regularization further boosts model performance.

Original: Interestingly, the ablation model DIR-Var has already exceeded some of the baselines.
Revised: Interestingly, the ablation model DIR-Var has already exceeded some of the baselines.

Original: We attribute such improvement to data augmentation via interventional distributions.
Revised: We attribute such improvement to data augmentation via interventional distributions.

Original: On top of DIR-Var, DIR improves the model performance by averagely 1 .
Revised: On top of DIR-Var, DIR improves the model performance by averagely 1 .

Original: 57% in Spurious-Motif and 2 .
Revised: 57% in Spurious-Motif and 2 .

Original: 62% in MNIST-75sp.
Revised: 62% in MNIST-75sp.

Original: This suggests that the variance regularization demands a stronger invariance condition and is instructive for searching causal features.
Revised: This suggests that the variance regularization demands a stronger invariance condition and is instructive for searching causal features.

Original: DIR has better intrinsic interpretability than the baselines.
Revised: DIR has better intrinsic interpretability than the baselines.

Original: In Table 2, we report the performance of all intrinsically interpretable models w.r.t.
Revised: In Table 2, we report intrinsic interpretable models’ performance w.r.t.

Original: Precision@5.
Revised: Precision@5.

Original: Clearly, DIR has an advantage in discovering causal features, reﬂected by the consistent improvements over the other baselines.
Revised: From the consistent improvements over the baselines, we ﬁnd DIR has an advantage in discovering causal features.

Original: To highlight, the performance gap between DIR and the baselines becomes more signiﬁcant, with the biasindicator increases. 
Revised: And the performance gap between DIR and the baselines becomes more signiﬁcant when the bias increases. [Figure]

Original: [Figure]
Revised: [Figure]

Original: ③ ① ② adaption fitting adaption fitting ③ ① ② ③ ① ②
Revised: adaption fitting adaption fitting ③ ① ② ③ ① ②

Original: [Figure]
Revised: [Figure]

Original: 3. I N -D EPTH S TUDY (RQ2)
Revised: 3. I N -D EPTH S TUDY (RQ2)

Original: We empirically analyze the DIR’s properties which hopefully give insights into its mechanisms and can be instructive for the existing training paradigms of deep models.
Revised: We empirically analyze the DIR’s properties which hopefully give insights into its mechanisms and can be instructive for the existing training paradigms of deep models.

Original: Rationale Visualization.
Revised: Rationale Visualization.

Original: Towards an intuitive understanding of DIR, we ﬁrst present some cases of the discovered rationale for Graph-SST2 in Figure 4.
Revised: Towards an intuitive understanding of DIR, we ﬁrst present some cases of the discovered rationale for Graph-SST2 in Figure 4.

Original: DIR is able to emphasize the tokens that directly result in the sentences’ positive or negative sentiments, which are reliable and faithful rationales.
Revised: DIR is able to emphasize the tokens that directly result in the sentences’ positive or negative sentiments, which are reliable and faithful rationales.

Original: Speciﬁcally, DIR highlights the positive words “majestic achievement” and “astonishing grandeur” in Figure 4a and underscores the negative words “worst dialogue” in Figure 4b as the rationales, which are clearly salient for the positive and negative sentiments, respectively.
Revised: Speciﬁcally, DIR highlights the positive words “majestic achievement” and “astonishing grandeur” in Figure 4a and underscores the negative words “worst dialogue” in Figure 4b as the rationales, which are clearly salient for the positive and negative sentiments, respectively.

Original: Furthermore, DIR can focus persistently on the causal features for OOD testing data.
Revised: Furthermore, DIR can focus persistently on the causal features for OOD testing data.

Original: For example, it selects surprisingly engrossing and “admittedly middling” in Figures 4c and 4d, respectively.
Revised: For example, it selects surprisingly engrossing and “admittedly middling” in Figures 4c and 4d, respectively.

Original: This again validates the effectiveness of DIR: (1) h ˜ C is well-learned to distinguish causal and non-causal features under various interventional distributions; and (2) h
Revised: This again validates the effectiveness of DIR: (1) h ˜ C is well-learned to distinguish causal and non-causal features under various interventional distributions; and (2) h

Original: ˜ Y conducts message-passing on the highlighted rationales, extracts the graph representations, and ﬁnally outputs the predictions with high accuracy.
Revised: ˜ Y conducts message-passing on the highlighted rationales, extracts the graph representations, and ﬁnally outputs the predictions with high accuracy.

Original: See Appendix F.1 for more examples in Graph-SST2 and Spurious-Motif datasets.
Revised: See Appendix F.1 for more examples in Graph-SST2 and Spurious-Motif datasets.

Original: Two-stage Training Dynamics.
Revised: Two-stage Training Dynamics.

Original: As Figure 5a displays, we ﬁnd a pattern from the Var-Time curve — during training DIR, the variance penalty ( i.e., Var s in Equation 4) ﬁrst increases and then decreases to almost zero.
Revised: As Figure 5a displays, we ﬁnd a pattern from the Var-Time curve — during training DIR, the variance penalty ( i.e., Var s in Equation 4) ﬁrst increases and then decreases to almost zero.

Original: Moreover, there exists an interesting correlation between the variance penalty and the precision metrics — that is, the precision rises dramatically as the penalty increases while growing slowly as the penalty decreases.
Revised: Moreover, there exists an interesting correlation between the variance penalty and the precision metrics — that is, the precision rises dramatically as the penalty increases while growing slowly as the penalty decreases.

Original: To probe this learning pattern, we further visualize the rationale distribution in three turning points: (1) the start, (2) the middle, and (3) the end of training.
Revised: To probe this learning pattern, we further visualize the rationale distribution in three turning points: (1) the start, (2) the middle, and (3) the end of training.

Original: Interestingly, the rationale distribution at the middle point is highly similar to that at the ending point.
Revised: Interestingly, the rationale distribution at the middle point is highly similar to that at the ending point.

Original: This illustrates two stages, adaption and ﬁtting, in the patterns.
Revised: This illustrates two stages, adaption and ﬁtting, in the patterns.

Original: By “adaption”, we mean that the exhibition of h ˜ C , i.e., learning to select salient feature ˜ C , is mainly conducted during the initial training stage.
Revised: By “adaption”, we mean that the exhibition of h ˜ C , i.e., learning to select salient feature ˜ C , is mainly conducted during the initial training stage.

Original: Since the penalty value can be seen as the magnitude to violate the invariance condition, this stage explores the rationales that satisfy the DIR principle.
Revised: Since the penalty value can be seen as the magnitude to violate the invariance condition, this stage explores the rationales that satisfy the DIR principle.

Original: Correspondingly, h ˜ Y adapts quickly with the input of varying rationales generated by h ˜ C .
Revised: Correspondingly, h ˜ Y adapts quickly with the input of varying rationales generated by h ˜ C .

Original: By “ﬁtting”, we mean that, in the later training process, h ˜ C only makes small changes, resulting in the substantially unchanged rationales compared to the initial training process, which is learned from the rationale generator to conform to the DIR principle.
Revised: By “ﬁtting”, we mean that, in the later training process, h ˜ C only makes small changes, resulting in the substantially unchanged rationales compared to the initial training process, which is learned from the rationale generator to conform to the DIR principle.

Original: This could also imply that based on the well-learned rationales, DIR mainly optimizes h ˜ Y to consolidate the functional relation ˜ C
Revised: This could also imply that based on the well-learned rationales, DIR mainly optimizes h ˜ Y to consolidate the functional relation ˜ C

Original: → Y until model convergence.
Revised: → Y until model convergence.

Original: Moreover, we compare the learning patterns of IRM and DIR in Figure 5b, where the penalty term of IRM (the gradient norm penalty in IRMv1 (Arjovsky et al., 2019)) follows a similar pattern to the DIR penalty.
Revised: Moreover, we compare the learning patterns of IRM and DIR in Figure 5b, where the penalty term of IRM (the gradient norm penalty in IRMv1 (Arjovsky et al., 2019)) follows a similar pattern to the DIR penalty.

Original: Notably, in MNIST-75sp, while IRM consistently outperforms DIR w.r.t.
Revised: Notably, in MNIST-75sp, while IRM consistently outperforms DIR w.r.t.

Original: Training
Revised: Training

Original: ACC, it does not improve and even degrades the performance in the testing dataset due to overﬁtting.
Revised: ACC, it does not improve and even degrades the performance in the testing dataset due to overﬁtting.

Original: However, DIR shows the solid resistance for over-ﬁtting, partly thanks to the valid rationales exhibited in the adaption stage.
Revised: However, DIR shows the solid resistance for over-ﬁtting, partly thanks to the valid rationales exhibited in the adaption stage.

Original: For Molhiv, DIR outperforms IRM as the rationales ﬁlter out irrelevant or spurious structures bootless for classiﬁcation tasks and are beneﬁcial for generalization.
Revised: For Molhiv, DIR outperforms IRM as the rationales ﬁlter out irrelevant or spurious structures bootless for classiﬁcation tasks and are beneﬁcial for generalization.

Original: Sensitivity Analysis.
Revised: Sensitivity Analysis.

Original: We conduct a sensitivity analysis of model performance w.r.t.
Revised: We conduct a sensitivity analysis of model performance w.r.t.

Original: λ in Appendix
Revised: λ in Appendix

Original: F.2, which shows that DIR surpasses the best baselines under a relatively large range of λ .
Revised: F.2, which shows that DIR surpasses the best baselines under a relatively large range of λ .

Original:  R ELATED W ORKS
Revised:  R ELATED W ORKS

Original: Inherent Interpretability of GNNs.
Revised: Inherent Interpretability of GNNs.

Original: Rudin (2018) points out that post-hoc explanation methods could be unfaithful to the true model mechanism and gives priority to creating interpretable models.

Original: We summarize two classes of the existing methods to build deep interpretable GNNs, (i) Attention (Vaswani et al., 2017;
Revised: We summarize two classes of the existing methods to build deep interpretable GNNs, (i) Attention (Vaswani et al., 2017;

Original: Veli ˇ ckovi ´ c et al., 2018), which can be broadly interpreted as importance weights on representations, indicating how strongly the features are correlated with the prediction of interest.
Revised: Veli ˇ ckovi ´ c et al., 2018), which can be broadly interpreted as importance weights on representations.(ii)

Original: (ii) Pooling (Lee et al., 2019; Knyazev et al., 2019; Gao & Ji, 2019), which selectively performs down-sampling on representations.
Revised: Pooling (Lee et al., 2019; Knyazev et al., 2019; Gao & Ji, 2019), which selectively performs down-sampling on representations.

Original: We include it in this category when it involves selection importance.
Revised: We include it in this category when it involves selection importance.

Original: However, the mechanisms to generate the rationales could be epistemic, as they only reﬂect the probabilistic relations between data and predicted labels (Pearl, 2000), which may not hold true in all data distributions.
Revised: However, the mechanisms to generate the rationales could be epistemic, as they only reﬂect the probabilistic relations between data and predicted labels (Pearl, 2000), which may not hold true in all data distributions.

Original: Thus, the rationales could fail to align with causal features and even degrade model performance due to being “fooled” by spurious features (Chang et al., 2020).
Revised: Thus, the rationales could fail to align with causal features and even degrade model performance due to being “fooled” by spurious features (Chang et al., 2020).

Original: Invariant Learning.
Revised: Invariant Learning.

Original: Backed by causal theory, invariant learning assumes the causal relation from the causal factors C to the response variable Y remains invariantacross all distributions unless we intervene on Y .
Revised: Backed by causal theory, invariant learning assumes the causal relation from the causal factors C to the response variable Y remains invariant unless we intervene on Y .

Original: As the most prevailing formulation, IRM (Arjovsky et al., 2019) extends the invariance assumption from feature level to representation level and ﬁnds a data representation Ω such that Ω ◦ Φ matches for all environments, where Φ is the optimal predictor.
Revised: As the most prevailing formulation, IRM (Arjovsky et al., 2019) extends the invariance assumption from feature level to representation level and ﬁnds a data representation Φ such that Ω ◦ Φ matches for all environments, where Ω is the classiﬁer.

Original: Ahuja et al.

Original: (2020) further provides an ensemble version of IRM by game theory.

Original: Regardless of IRM’s success, concerns about its feasibility (Rosenfeld et al., 2021; Ahuja et al., 2021) and optimality (Kamath et al., 2021) have been discussed recently.
Revised: However, concerns about its feasibility (Rosenfeld et al., 2021; Ahuja et al., 2021) and optimality (Kamath et al., 2021) have been discussed recently.

Original: Besides IRM, variance penalization across environments is shown to be effective for recovering invariance (Krueger et al., 2021; Xie et al., 2020; Teney et al., 2020).
Revised: Besides IRM, variance penalization across environments is shown to be effective for recovering invariance (Krueger et al., 2021; Xie et al., 2020; Teney et al., 2020).

Original: Notably, the existing methods generally require accessing different environments, thus additionally involving environment inference (Creager et al., 2021; Wang et al., 2021b).
Revised: Notably, the existing methods generally require accessing different environments, thus additionally involving environment inference (Creager et al., 2021; Wang et al., 2021b).

Original: Similarly motivated as ours, Chang et al.
Revised: Similarly motivated as ours, Chang et al.

Original: (2020) discovers rationales Z by minimizing the performance gap between environment-agnostic predictor f ( Z ) and environment-aware predictor f
Revised: (2020) discover rationales Z by minimizing the performance gap between environment-agnostic predictor f ( Z ) and environment-aware predictor f

Original: ( Z, E ) .
Revised: ( Z, E ) .

Revised: In graph domain, Bevilacqua et al.

Revised: (2021) construct graph representations from subgraph densities and use attribute symmetry regularization to mitigate the shift of graph size and vertex attribute distributions.

Original:  C ONCLUSION & F UTURE W ORK
Revised:  C ONCLUSION & F UTURE W ORK

Original: In this work, we rigorously study the intrinsic interpretability of Graph Neural Networks from a causal perspective.
Revised: In this work, we rigorously study the intrinsic interpretability of Graph Neural Networks from a causal perspective.

Original: Our concerns are towards the exhibition of shortcut features when generating the rationales.
Revised: Our concerns are towards the exhibition of shortcut features when generating the rationales.

Original: And we proposed an invariant learning algorithm, DIR, to discover the causal features for rationalization.
Revised: And we proposed an invariant learning algorithm, DIR, to discover the causal features for rationalization.

Original: The core of DIR lies in the construction of environments ( i.e., interventional distributions) and thus distilling the salient features as rationales that are consistently informative and uniform across these environments.
Revised: The core of DIR lies in the construction of environments ( i.e., interventional distributions) and thus distilling the salient features as rationales that are consistently informative and uniform across these environments.

Original: Such rationales serve as the probing towards model mechanisms and are demonstrated to be effective in generalization.
Revised: Such rationales serve as the probing towards model mechanisms and are demonstrated to be effective in generalization.

Original: In the experiments, we highlight an adaptionﬁtting training dynamics for DIR to reveal its learning pattern.
Revised: In the experiments, we highlight an adaption-ﬁtting training dynamics for DIR to reveal its learning pattern.

Original: We would like to establish more reliable and general intrinsic interpretable models that can be faithfully applied to various data types, tasks, and real applications for future works.
Revised: In the future, we will build more reliable and expressive interpretable models that are feasible under various assumptions, which potentially calls for high-level interpretability. We recommend interested readers go to the open discussion in Appendix G for the detailed description. A CKNOWLEDGMENT This work was supported by the National Key Research and Development Program of China (2020AAA0106000), the National Natural Science Foundation of China (U19A2079), the SeaNExT Joint Lab, and Singapore MOE AcRF T2.

Original: E THICS S TATEMENT
Revised: E THICS S TATEMENT

Original: In this work we propose a novel algorithm for intrinsic interpretable models, where no human subject is related.
Revised: In this work we propose a novel algorithm for intrinsic interpretable models, where no human subject is related.

Original: This synthetic dataset is made available in the anonymous link ( cf.
Revised: This synthetic dataset is made available in the anonymous link ( cf.

Original: Section 3.1).
Revised: Section 3.1).

Original: We believe the exhibition of rationales is beneﬁcial for inspecting and eliminating potential discrimination and fairness issues in deep models for real applications.
Revised: We believe the exhibition of rationales is beneﬁcial for inspecting and eliminating potential discrimination and fairness issues in deep models for real applications.

Original: R EPRODUCIBILITY S TATEMENT
Revised: R EPRODUCIBILITY S TATEMENT

Original: We summarize the efforts made to ensure reproducibility in this work.
Revised: We summarize the efforts made to ensure reproducibility in this work.

Original: (1) Datasets: We use one synthetic dataset which is made available ( cf.
Revised: (1) Datasets: We use one synthetic dataset which is made available ( cf.

Original: the anonymous link in Section 3.1), and three public datasets where the processing details are included in Appendix D. (2)
Revised: the anonymous link in Section 3.1) and three public datasets where the processing details are included in Appendix D. (2)

Original: Model Training: We provide the procedure of training in Algorithm A and the training details (including hyper-parameter settings) in Appendix D which are consistent with our implementation in the code ( cf.
Revised: Model Training: We provide the procedure of training in Algorithm A and the training details (including hyper-parameter settings) in Appendix D which are consistent with our implementation in the code ( cf.

Original: the anonymous link in Section 3.1).
Revised: the anonymous link in Section 3.1).

Original: (3) Theoretical Results: All assumptions and proofs can be referred to
Revised: (3) Theoretical Results: All assumptions and proofs can be referred to

Original: Appendix C.
Revised: Appendix C.

Original: A N OTATIONS & A LGORITHM
Revised: A N OTATIONS & A LGORITHM

Original: Key Notations in the Main Paper.
Revised: Key Notations in the Main Paper.

Original: Symbol
Revised: Symbol

Original: Deﬁnition g graph instance c / s ground truth causal or confounding subgraph ˜ c / ˜ s generated rationale or complement of rationale instance
Revised: Deﬁnition g graph instance c / s ground truth causal or confounding subgraph ˜ c / ˜ s generated rationale or complement of rationale instance

Original: C / S variables in the causal graph
Revised: C / S variables in the causal graph

Original: S / ˜ S space of  ground truth or identiﬁed spurious features ˆ y ˜ c
Revised: S / ˜ S space of the ground truth or identiﬁed spurious features ˆ y ˜ c

Original: / ˆ y ˜ s causal or spurious prediction ˆ y joint prediction h
Revised: / ˆ y ˜ s causal or spurious prediction ˆ y joint prediction h

Original: ˜ C rationale generator Φ 1 / Φcausal or spurious classiﬁer
Revised: ˜ C rationale generator Φ 1 / Φcausal or spurious classiﬁer

Original: Require: Training data distribution P tr ( G ) ; number of classes Q ; Stepsize α ; hyper-parameter λ 1: Randomly initialize the parameters of generator h ˜ C , encoder h θ (includes GNN 2 and Pooling layer), two classiﬁers Φ 1 and Φ 2 , which are denoted as γ, θ, φ 1 , φ 2 , respectively.
Revised: Require: Training data distribution P tr ( G ) ; number of classes Q ; Stepsize α ; hyper-parameter λ 1: Randomly initialize the parameters of generator h ˜ C , encoder h θ (includes GNN 2 and Pooling layer), two classiﬁers Φ 1 and Φ 2 , which are denoted as γ, θ, φ 1 , φ 2 , respectively.

Original: 2: while not converge do 3:
Revised: 2: while not converge do 3:

Original: Sample graphs{ ( g i y i ) } Bi =
Revised: Sample graphs

Original: P tr G ) 4: 
Revised: G ) 4: do 6:

Original: Generate each rationale complement 5: for each ˜ s do 6: Intervenerh I operates do( S s i ) 7:
Revised: Intervener operates do

Original: Model forward: ˆ y ˜ s =
Revised: Model forward: ˆ y ˜ s =

Original: Φ 2 ( h θ (˜ s i ))
Revised: Φ 2 ( h θ (˜ s i ))

Original: ∈ R 1 × Q , { ˆ y ˜ c } Bi
Revised: ∈ R 1 × Q , { ˆ y ˜ c } Bi

Original: =1 =
Revised: =1 =

Original: Φ 1 ( h θ ( { ˜ c i } Bi =1 ))
Revised: Φ 1 ( h θ ( { ˜ c i } Bi =1 ))

Original: ∈ R B × Q 8:
Revised: ∈ R B × Q 8:

Original: # block BP of DIR risk to shortcut branch
Revised: # block BP of DIR risk to shortcut branch

Original: Obtain joint prediction ˆ y =
Revised: Obtain joint prediction ˆ y =

Original: [ˆ y 1 , . .
Revised: [ˆ y 1 , . .

Original: , ˆ y B ] , where ˆ y j = ˆ y ˜ c
Revised: , ˆ y B ] , where ˆ y j = ˆ y ˜ c

Original: (cid:12) σ (ˆ y j ˜ s ) .detach
Revised: (cid:12) σ (ˆ y j ˜ s ) .detach

Original: Compute and record risk R (ˆ y ˜ s , y i ) 10:
Revised: Compute and record risk R (ˆ y ˜ s , y i ) 10:

Original: Compute and record ˜ s i -interventional risk.
Revised: Compute and record ˜ s i -interventional risk.

Original: 11: end for 12:
Revised: 11: end for 12:

Original: Compute R DIR via Eq.
Revised: Compute R DIR via Eq.

Original: 4 and R ˜ S via Eq.13:
Revised: 4 and R ˜ S via Eq.13:

Original: Update parameters: φ
Revised: Update parameters: φ

Original: [Equation]
Revised: [Equation]

Original: 14: end while
Revised: 14: end while

Original: B I NSTANTIATED C AUSAL G RAPHS
Revised: B I NSTANTIATED C AUSAL G RAPHS

Original: We instantiate possible causal graphs in Figure 2a.
Revised: We instantiate possible causal graphs in Figure 2a.

Original: Speciﬁcally, we use the example of Base-Motif graphs, whose labels are determined by the motif types.
Revised: Speciﬁcally, we use the example of Base-Motif graphs, whose labels are determined by the motif types.

Original: We use C = 0 , 1 , 2 to denote cycle, house, crane, respectively; And use S = 0 , 1 , 2 to denote ladder, tree, wheels, respectively.
Revised: We use C = 0 , 1 , 2 to denote cycle, house, crane, respectively; And use S = 0 , 1 , 2 to denote ladder, tree, wheels, respectively.

Original: • C | =
Revised: • C | =

Original: S : Base graphs and motif graphs are independently sampled and attached to each other
Revised: S : Base graphs and motif graphs are independently sampled and attached to each other

Original: • C → S : Type of each motif respects to a given (static) probability distribution.
Revised: • C → S : Type of each motif respects to a given (static) probability distribution.

Original: According to the value of C , the probability distribution of its base graph is given by
Revised: According to the value of C , the probability distribution of its base graph is given by

Original: [Equation]
Revised: [Equation]

Original: • S → C : Similar to the example for C → S .
Revised: • S → C : Similar to the example for C → S .

Original: • S ← E → C : Suppose there is a latent variable E takes continuous value from 0 to 1 .
Revised: • S ← E → C : Suppose there is a latent variable E takes continuous value from 0 to 1 .

Original: Then the probability distribution of S and C s.t.
Revised: Then the probability distribution of S and C s.t.

Original: [Equation]
Revised: [Equation]

Original: where B stands for binomial distribution, i.e., for variable X , if X ∼ B ( n, p ) , then we have
Revised: where B stands for binomial distribution, i.e., for variable X , if X ∼ B ( n, p ) , then we have

Original: [Equation]
Revised: [Equation]

Original: C T HEORY C. A SSUMPTION
Revised: C T HEORY C. A SSUMPTION

Original: We phrase the SCM in Figure 2a as the following assumption:
Revised: We phrase the SCM in Figure 2a as the following assumption:

Original: Assumption 1 (Invariant Rationalization (IR))
Revised: Assumption 1 (Invariant Rationalization (IR))

Original: There exists a rationale C ⊆ G , such that the structural equation model
Revised: There exists a rationale C ⊆ G , such that the structural equation model

Original: [Equation]
Revised: [Equation]

Original: and the probability relation
Revised: and the probability relation

Original: [Equation]
Revised: [Equation]

Original: hold for every distribution ˜ P over P ( G, Y ) , where S denotes the complement of C .
Revised: hold for every distribution ˜ P over P ( G, Y ) , where S denotes the complement of C .

Original: Also, we denote f Y as the oracle structural equation model.
Revised: Also, we denote f Y as the oracle structural equation model.

Original: By “oracle”, we mean that f Y is the perfect structure equation model, which, when C is available, predicts the response variable with the minimum expected loss over any distribution ˜ P .
Revised: By “oracle”, we mean that f Y is the perfect structure equation model, which, when C is available, predicts the response variable with the minimum expected loss over any distribution ˜ P .

Original: Or formally,
Revised: Or formally,

Original: [Equation]
Revised: [Equation]

Original: where l is the task-speciﬁc loss function and we ignore the exogenous noise (cid:15) Y in f Y ’s input except as otherwise noted.
Revised: where l is the task-speciﬁc loss function and we ignore the exogenous noise (cid:15) Y in f Y ’s input except as otherwise noted.

Original: Next, we argue that the assumption is commonly satisﬁed.
Revised: Next, we argue that the assumption is commonly satisﬁed.

Original: For example, for sentences labeled by sentiment, C can represent the positive/negative words that cause the sentiment, while S includes the prepositions and linking words.
Revised: For example, for sentences labeled by sentiment, C can represent the positive/negative words that cause the sentiment, while S includes the prepositions and linking words.

Original: For molecule graphs labeled by speciﬁc properties, C and S can represent the functional groups and carbon structures, respectively.
Revised: For molecule graphs labeled by speciﬁc properties, C and S can represent the functional groups and carbon structures, respectively.

Original: Note that IR assumption enables and calls the introduction of interpretability, highlighting salient features and exhibiting human accessible checks.
Revised: Note that IR assumption enables and calls the introduction of interpretability, highlighting salient features and exhibiting human accessible checks.

Original: More importantly, it guarantees the model performance under possible feature reduction, i.e., C ⊂ G .
Revised: More importantly, it guarantees the model performance under possible feature reduction, i.e., C ⊂ G .

Original: We also see cases going beyond the IR Assumption.
Revised: We also see cases going beyond the IR Assumption.

Original: For example, G could be a generic function of S and C , instead of a simple joint.
Revised: For example, G could be a generic function of S and C , instead of a simple joint.

Original: We use a toy example to elaborate this point.
Revised: We use a toy example to elaborate this point.

Original: Following the
Revised: Following the

Original: Spurious-Motif dataset, we assume each graph has multiple motifs (house, cycle, crane) with only one type and is labeled by the motif type.
Revised: Spurious-Motif dataset, we assume each graph has multiple motifs (house, cycle, crane) with only one type and is labeled by the motif type.

Original: Thus, the causal feature C will be the motifs.
Revised: Thus, the causal feature C will be the motifs.

Original: Let the spurious feature S be ”the way we connect the motifs”.
Revised: Let the spurious feature S be ”the way we connect the motifs”.

Original: For example, we can place the house motifs in a queue sequence and connect the adjacent motifs, thus forming the graph in a ”line” shape.
Revised: For example, we can place the house motifs in a queue sequence and connect the adjacent motifs, thus forming the graph in a ”line” shape.

Original: Or we can place the houses in a cycle order and connect them into a ring.
Revised: Or we can place the houses in a cycle order and connect them into a ring.

Original: We further make such graph structures strongly correlated with the motif types.
Revised: We further make such graph structures strongly correlated with the motif types.

Original: Thus, individual S and C may be intractable individually in the feature level.
Revised: Thus, individual S and C may be intractable individually in the feature level.

Original: For example, if we separate the cycle-shaped houses into two lines, the spurious pattern could be broken while the part of the causal feature would be lost.
Revised: For example, if we separate the cycle-shaped houses into two lines, the spurious pattern could be broken while the part of the causal feature would be lost.

Original: In other words, S and C are dependent variables.
Revised: In other words, S and C are dependent variables.

Original: Thus, they can’t be extracted and modeled separately, which goes out of the scope of our work.
Revised: Thus, they can’t be extracted and modeled separately, which goes out of the scope of our work.

Revised: Given that S and C are separable, we further make the following assumption to avoid the confusion of S and C :

Revised: Assumption 2 (Feature Induction) Deﬁne power set operation as P ∗ ( · ) .

Revised: For data G = S ∪ C and label Y , if S |

Revised: = Y | C holds for any distribution ˜ P over P ( G, Y ) , then it implies that for any induced feature S (cid:48)

Revised: ∈

Original: C. P ROOFS
Revised: P ∗ ( S ) , we have S (cid:48) | = Y | C holds for the distribution ˜ P . This assumption also implies that C could not be induced by S when | C | ≤ | S | .

Revised: Thus, any feature subset C (cid:48) except for C would violate the conditional independence condition.

Revised: For images, this assumption is natural for the splicing of S doesn’t typically change its semantics.

Revised: For example, the splicing of land background would still be divided land.

Revised: While for graphs, here we assume the causal subgraph’s uniqueness among the induced complement graphs.

Revised: C. P ROOFS

Original: Theorem 1 (Necessity) Suppose S → C does not exist, then the oracle function f Y satisﬁes
Revised: Theorem 1 (Necessity) Suppose S → C does not exist, then the oracle function f Y satisﬁes

Original: the
Revised: the

Original: DIR Principle (where C is given) over every distribution ˜ P ∈ P ( G, Y ) .
Revised: DIR Principle (where C is given) over every distribution ˜ P ∈ P ( G, Y ) .

Original: Proof: We ﬁrst prove the fact that P ( Y = y | do ( S = s ))
Revised: Proof: We ﬁrst prove the fact that P ( Y = y | do ( S = s ))

Original: =
Revised: =

Original: P ( Y = y ) for distribution ˜ P .
Revised: P ( Y = y ) for distribution ˜ P .

Original: Speciﬁcally, we use P ( s ) I to denote the s-interventional distribution.
Revised: Speciﬁcally, we use P ( s ) I to denote the s-interventional distribution.

Original: • If C → S , [Table] do 
Revised: • If C → S , P ( Y = y | do ( S = s ))

Original: [Table]
Revised: [Table]

Original: • If C | =
Revised: • If C | =

Original: S ,
Revised: S ,

Original: [Table]
Revised: [Table]

Original: • If C ← E → S , P ( Y = y | do ( S = s ))
Revised: Y = y | do ( S = s ))

Original: [Table]
Revised: [Table]

Original: == P ( s ) I ( Y = y
Revised: [Equation] = y

Original: [Equation]
Revised: [Equation]

Original: S = s ) given E → S = = = =
Revised: • If C ← E → S , P

Original: = = == (cid:88)
Revised: [Equation]

Original: e
Revised: do

Original: [Equation]
Revised: [Equation]

Original: [Equation] = 
Revised: S = s )) [Table]

Original: [Equation]
Revised: [Equation]

Original: =
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: = e ) P ( s ) I ( E = e ) = (cid:88) e (cid:88) c P
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Equation] [Equation]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: P
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: s )

Original: I ( C
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: c
Revised: [Table]

Original: [Equation] S [Equation]
Revised: [Equation] [Table]

Original: , E

Original: [Equation]

Original: e

Original: [Equation]
Revised: [Equation]

Original: P ( s ) I ( E = e ) given ( Y | = { S,E }| C ) and ( C | = S | E ) =

Original: = = = =
Revised: [Table]

Original: = = = =
Revised: [Equation]

Original: =

Original: =

Original: =

Original: =

Original: =

Original: =

Original: =

Original: =

Original: =

Original: =

Original: = = = ==
Revised: [Table]

Original: (cid:88) e (cid:88) c

Original: P ( s ) I ( Y = y

Original: | C = c ) P ( s ) I ( C = c | E = e ) P ( s ) I ( E = e ) = (cid:88) e (cid:88) cP
Revised: c

Original: [Equation]
Revised: [Equation]

Original: =
Revised: ==

Original: [Equation]
Revised: [Equation]

Original: c ) [Equation] e ) =[Equation]
Revised: =

Original: e (cid:88) c

Original: [Equation]
Revised: [Equation]

Original: As P ( Y =
Revised: As P ( Y =

Original: y | do ( S = s ))
Revised: y | do ( S = s ))

Original: =
Revised: =

Original: P ( Y = y ) holds true for every distribution ˜ P , which is invariant w.r.t.
Revised: P ( Y = y ) holds true for every distribution ˜ P , which is invariant w.r.t.

Original: iterative variable S .
Revised: iterative variable S .

Original: Moreover, we have P ( C = c | do ( S = s ))
Revised: Moreover, we have P ( C = c | do ( S = s ))

Original: =
Revised: =

Original: P ( s ) I ( C = c )
Revised: P ( s ) I ( C = c )

Original: =
Revised: =

Original: P ( C = c ) .
Revised: P ( C = c ) .

Original: This indicates that the intervention on S leave the causal structure C → Y untouched.
Revised: This indicates that the intervention on S leave the causal structure C → Y untouched.

Original: Thus, we have
Revised: Thus, we have

Original: Var ( {R ( f Y | do ( s )) | s ∈ S } ) =

Original: Var (cid:16)(cid:110) E ( G

Original: [Equation]
Revised: [Equation]

Original: Y ) ∼ [Equation] ( s ) I ( G [Equation]
Revised: Finally, taking the deﬁnition of f

Original: Y )

Original: [Equation]
Revised: [Equation]

Original: C ⊂ G [

Original: [Equation] ( f Y ( C ) , Y )] | s ∈ S (cid:111)(cid:17) [Equation] Finally, taking the deﬁnition of f , we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: Hence, f Y takes the minimum penalty and satisﬁes the DIR Principle.
Revised: Hence, f Y takes the minimum penalty and satisﬁes the DIR Principle.

Original: [Equation]
Revised: [Equation]

Original: Notably, if S → C , then Var ( {R ( f Y | do ( s )) | s ∈ S } ) may not equal to zero since c ∼
Revised: Notably, if S → C , then Var ( {R ( f Y | do ( s )) | s ∈ S } ) may not equal to zero since c ∼

Original: P ( s ) I ( C | S = s ) .
Revised: P ( s ) I ( C | S = s ) .

Original: In such case, f Y is not necessarily satisﬁed to DIR Principle.
Revised: In such case, f Y is not necessarily satisﬁed to DIR Principle.

Original: That is, although f Y still minimizes R ( f | do ( S )) , we can’t be sure whether it reaches the lower bound of Var ( {R ( f Y | do ( s )) | s ∈ S } ) without knowledge about the speciﬁc data distribution.
Revised: That is, although f Y still minimizes R ( f | do ( S )) , we can’t be sure whether it reaches the lower bound of Var ( {R ( f Y | do ( s )) | s ∈ S } ) without knowledge about the speciﬁc data distribution.

Original: Thus, we only consider the cases of C → S , C | =
Revised: Thus, we only consider the cases of C → S , C | =

Original: S and C ← E → S in the following discussion.
Revised: S and C ← E → S in the following discussion.

Original: Theorem 2 (Uniqueness) Suppose l is a strict loss function and there exists one and only one nontrivial subset C , then there exists a unique structure equation model f Y s.t. 
Revised: Theorem 2 (Uniqueness) Suppose l is a strict loss function and there exists one and only one nontrivial subset C , then there exists a unique structure equation model f Y s.t. it satisﬁes the DIR

Original: DIR Principle.
Revised: Principle.

Original: Proof: Since f Y exists and satisﬁes the DIR Principle, we only need to prove its uniqueness under the given conditions.
Revised: Proof: Since f Y exists and satisﬁes the DIR Principle, we only need to prove its uniqueness under the given conditions.

Original: Otherwise, suppose we have another structure equation f (cid:48)
Revised: Otherwise, suppose we have another structure equation f (cid:48)

Original: Y (cid:54) =
Revised: Y (cid:54) =

Original: f Y satiﬁes the DIR Principle.
Revised: f Y satiﬁes the DIR Principle.

Original: Speciﬁcally, there exists a datum ( g, y ) s.t. 
Revised: Speciﬁcally, there exists a datum ( g, y ) s.t. f (cid:48)

Original: [Equation]
Revised: Y ( c ) (cid:54) = f Y ( c ) .

Original: Thus, we have l ( f (cid:48) Y ( c ) , y )
Revised: Thus, we have l ( f (cid:48) Y ( c ) , y )

Original: >
Revised: >

Original: l ( f Y ( c ) , y ) .
Revised: l ( f Y ( c ) , y ) .

Original: Given that Var ( {R ( f (cid:48)
Revised: Given that Var ( {R ( f (cid:48)

Original: Y | do ( s )) | s ∈ S } ) ≥ 0 =
Revised: Y | do ( s )) | s ∈ S } ) ≥ 0 =

Original: Var ( {R ( f Y | do ( s )) | s ∈ S } ) , we have R DIR ( f (cid:48) Y )
Revised: Var ( {R ( f Y | do ( s )) | s ∈ S } ) , we have R DIR ( f (cid:48) Y )

Original: > R DIR ( f Y ) .
Revised: > R DIR ( f Y ) .

Original: (cid:3)
Revised: (cid:3)

Original: In reality, there could be multiple candidates of C , e.g., C i , C j s.t.
Revised: In reality, there could be multiple candidates of C , e.g., C i , C j s.t.

Original: R DIR ( f ( C i ) Y )=
Revised: R DIR ( f ( C i ) Y )

Revised: [Equation]

Original: R DIR ( f ( C j ) Y ) , where f ( C i )
Revised: ) , where f ( C i )

Original: Y is the structure equation corresponds to C i .
Revised: Y is the structure equation corresponds to C i .

Original: Thus, it calls for the selection of C to avoid the learning of suboptimal f Y .
Revised: Thus, it calls for the selection of C to avoid the learning of suboptimal f Y .

Original: Inspired by Occam’s Razor, we deﬁne
Revised: Inspired by Occam’s Razor, we deﬁne

Original: [Equation]
Revised: [Equation]

Original: 16) as the preferred rationale, or rationale of parsimony.
Revised: (15) as the preferred rationale, or rationale of parsimony.

Original: We argue that rationales are not to be extended beyond necessity, which poses simpler hypotheses about causality.
Revised: We argue that rationales are not to be extended beyond necessity, which poses simpler hypotheses about causality.

Original: As the search of C ∗ is NPhard (the worst time complexity is exponential), we use ﬁxed size for the learned rationales in our experiments and leave a better optimization to future work.
Revised: As the search of C ∗ is NPhard (the worst time complexity is exponential), we use ﬁxed size for the learned rationales in our experiments and leave a better optimization to future work.

Original: Corollary 1 (Necessity and Sufﬁciency)
Revised: Corollary 1 (Necessity and Sufﬁciency)

Original: Suppose l is a strict loss function and there exists one and only one non-trival subset C , then any structure causal model f (cid:48)
Revised: Suppose l is a strict loss function and there exists one and only one non-trival subset C , then any structure causal model f (cid:48)

Original: Y s.t.
Revised: Y s.t.

Original:  DIR Principle iff.
Revised: it satisﬁes the DIR Principle iff.

Original: f (cid:48)
Revised: f (cid:48)

Original: Y = f Y .
Revised: Y = f Y .

Original: This is directly obtained from Theorem 2.
Revised: This is directly obtained from Theorem 2.

Original: Thus, under the unique constraint of C , we can approach the oracle f Y by optimizing the DIR objective, which maintains the invariant causal relation between the causal feature and the response variable Y .
Revised: Thus, under the unique constraint of C , we can approach the oracle f Y by optimizing the DIR objective, which maintains the invariant causal relation between the causal feature and the response variable Y .

Original: In another way, based on the uniqueness of the feasible rationale, the optimization of the DIR Principle on the intrinsic interpretable model h (where
Revised: In another way, based on the uniqueness of the feasible rationale, the optimization of the DIR Principle on the intrinsic interpretable model h (where

Original: C is exhibited inside of h ) pushes the approach to C with rationales ˜ C .
Revised: C is exhibited inside of h ) pushes the approach to C with rationales ˜ C .

Original: Then, f Y can also be approached as an invariant predictor based on the learning from ˜ C .
Revised: Then, f Y can also be approached as an invariant predictor based on the learning from ˜ C .

Original: D S ETTING D ETAILS
Revised: D S ETTING D ETAILS

Original: [Table]
Revised: [Table]

Original: Datasets We summarize dataset statistics in Table 3, and introduce the node/edge features and the preprocessing in each datasets:
Revised: Datasets We summarize dataset statistics in Table 3, and introduce the node/edge features and the preprocessing in each datasets:

Original: • Spurious-Motif.
Revised: • Spurious-Motif.

Original: We use random node features and constant edge weights in this dataset.
Revised: We use random node features and constant edge weights in this dataset.

Original: • MNIST-75sp.
Revised: • MNIST-75sp.

Original: The nodes in the graphs are superpixels, and node features are the concatenation of pixel intensities (RGB channels) and coordinates of their mass centers.
Revised: The nodes in the graphs are superpixels, and node features are the concatenation of pixel intensities (RGB channels) and coordinates of their mass centers.

Original: Edges are the spatial distance between the superpixel centers, while we ﬁlter the edges with a distance less than 0.1 to make the graphs sparser.
Revised: Edges are the spatial distance between the superpixel centers, while we ﬁlter the edges with a distance less than 0.1 to make the graphs sparser.

Original: • Graph-SST2.
Revised: • Graph-SST2.

Original: We use constant edge weight and ﬁlter the graphs with edges less than three.
Revised: We use constant edge weight and ﬁlter the graphs with edges less than three.

Original: We initialize the node features by the pre-trained BERT (Devlin et al., 2018) word embedding.
Revised: We initialize the node features by the pre-trained BERT (Devlin et al., 2018) word embedding.

Original: • OGBG-Molhiv.
Revised: • OGBG-Molhiv.

Original: We use the ofﬁcial released dataset in our experiment.
Revised: We use the ofﬁcial released dataset in our experiment.

Original: GNNs.
Revised: GNNs.

Original: We summarize the backbone GNNs for each dataset in Table 3.
Revised: We summarize the backbone GNNs for each dataset in Table 3.

Original: The number of neurons in the sequent layers (in forwarding order) is reported.
Revised: The number of neurons in the sequent layers (in forwarding order) is reported.

Original: We use ReLU as activation layers and different global pooling layers.
Revised: We use ReLU as activation layers and different global pooling layers.

Original: In OGBG-Molhiv, we adopt one fully connected layer for the prediction layers while using two fully connected layers for the models in other datasets.
Revised: In OGBG-Molhiv, we adopt one fully connected layer for the prediction layers while using two fully connected layers for the models in other datasets.

Original: For baselines with node pooling/node attention, we add one node pooling/attention layer in the second convolution layer.
Revised: For baselines with node pooling/node attention, we add one node pooling/attention layer in the second convolution layer.

Original: Training Optimization & Early Stopping.
Revised: Training Optimization & Early Stopping.

Original: All experiments are done on a single Tesla V
Revised: All experiments are done on a single Tesla V

Original: SXM2 GPU (32 GB).
Revised: SXM2 GPU (32 GB).

Original: During training, we use Adam (Kingma & Ba, 2015) optimizer.
Revised: During training, we use Adam (Kingma & Ba, 2015) optimizer.

Original: The maximum number of epochs is 400 for all datasets.
Revised: The maximum number of epochs is 400 for all datasets.

Original: We use Stochastic Gradient Descent (SGD) for the optimization on Graph-SST2 and OGBG-Molhiv and Gradient Descent (GD) for the other two datasets.
Revised: We use Stochastic Gradient Descent (SGD) for the optimization on Graph-SST2 and OGBG-Molhiv and Gradient Descent (GD) for the other two datasets.

Original: Also, we exhibit early stopping to avoid overﬁtting of the training dataset.
Revised: Also, we exhibit early stopping to avoid overﬁtting of the training dataset.

Original: Speciﬁcally, in
Revised: Speciﬁcally, in

Original: MNIST-75sp, Graph-SST2 and OGBG-Molhiv, each model is evaluated on a holdout in-distribution validation dataset after each epoch.
Revised: MNIST-75sp, Graph-SST2 and OGBG-Molhiv, each model is evaluated on a holdout in-distribution validation dataset after each epoch.

Original: While for Spurious-Motif, we use an unbiased validation dataset ( i.e., without spurious relations compared to the training dataset).
Revised: While for Spurious-Motif, we use an unbiased validation dataset ( i.e., without spurious relations compared to the training dataset).

Original: If the model’s performance on the validation dataset is without improvement ( i.e., validation accuracy begins to decrease) for ﬁve epochs, we stop the training process to prevent increased generalization error.
Revised: If the model’s performance on the validation dataset is without improvement ( i.e., validation accuracy begins to decrease) for ﬁve epochs, we stop the training process to prevent increased generalization error.

Original: Hyper-Parameter Settings.
Revised: Hyper-Parameter Settings.

Original: We set the causal feature ratio and λ as ( r = 0 . 8 , λ = 10 − 4 ) , ( r = 0 .
Revised: We set the causal feature ratio and λ as ( r = 0 . 8 , λ = 10 − 4 ) , ( r = 0 .

Original: 25 , λ = 10 − 2 ) , ( r = 0 .
Revised: 25 , λ = 10 − 2 ) , ( r = 0 .

Original: 6 , λ = 10 2 ) ,{+
+} ( r = 0 .
Revised: 6 , λ = {+
+} ( r = 0 .

Original: 8 , λ = 10 −3 ) for MNIST-75sp, Spurious-Motif, Graph-SST2 and OGBG-Molhiv respectively. For other baselines, we adopt grid search for the best parameters using the validation datasets.
Revised: = 10 −

Original: Model Selection.
Revised: Model Selection.

Original: We select each model based on its performance on the corresponding validation dataset.
Revised: We select each model based on its performance on the corresponding validation dataset.

Original: We repeat each experiment at least ﬁve times and report the average values and the standard errors in the paper.
Revised: We repeat each experiment at least ﬁve times and report the average values and the standard errors in the paper.

Original: E U NIMODAL A DJUSTMENT
Revised: E U NIMODAL A DJUSTMENT

Original: We follow Cad`ene et al.
Revised: We follow Cad`ene et al.

Original: (2019) to demonstrate how the shortcut prediction can help to remove model bias.
Revised: (2019) to demonstrate how the shortcut prediction can help to remove model bias.

Original: For clarity, we refer to the model parameters except for Φ 2 as the main branch, i.e., except for the S -only branch.
Revised: For clarity, we refer to the model parameters except for Φ 2 as the main branch, i.e., except for the S -only branch.

Original: Given a house-tree graph as the input graph, we suppose the shortcut prediction ˆ y ˜ s of the tree subgraph leans towards the house class.
Revised: Given a house-tree graph as the input graph, we suppose the shortcut prediction ˆ y ˜ s of the tree subgraph leans towards the house class.

Original: Then after reweighting σ (ˆ y ˜ s ) on ˆ y ˜ c , the softmax readout on the house class in the joint prediction ˆ y will be magniﬁed, which results in a smaller loss backpropagated to the main branch and prevents ˆ y ˜ c from inductive bias.
Revised: Then after reweighting σ (ˆ y ˜ s ) on ˆ y ˜ c , the softmax readout on the house class in the joint prediction ˆ y will be magniﬁed, which results in a smaller loss backpropagated to the main branch and prevents ˆ y ˜ c from inductive bias.

Original: In another situation where a house-wheel graph is given as the input, we similarly suppose the shortcut prediction ˆ y ˜ s of the wheel subgraph leans towards other classes except the house, say, the circle class.
Revised: In another situation where a house-wheel graph is given as the input, we similarly suppose the shortcut prediction ˆ y ˜ s of the wheel subgraph leans towards other classes except the house, say, the circle class.

Original: Then after reweighting σ (ˆ y ˜ s ) on ˆ y ˜ c , the softmax readout on the house class in the joint prediction ˆ y will be reduced, which results in a larger loss back-propagated to the main branch and encourages the model to learn from these examples.
Revised: Then after reweighting σ (ˆ y ˜ s ) on ˆ y ˜ c , the softmax readout on the house class in the joint prediction ˆ y will be reduced, which results in a larger loss back-propagated to the main branch and encourages the model to learn from these examples.

Original: Furthermore, we offer the causal- and information-theoretical justiﬁcations: (1) From the perspective of causal theory (Pearl, 2000; Pearl et al., 2016), the element-wise multiplication enforces the spurious prediction to estimate the pure indirect effect (PIE) of the shortcut features, while the causal prediction captures the natural direct effect (NDE) of the causal patterns (VanderWeele, 2013); (2)
Revised: Furthermore, we offer the causal- and information-theoretical justiﬁcations: (1) From the perspective of causal theory (Pearl, 2000; Pearl et al., 2016), the element-wise multiplication enforces the spurious prediction to estimate the pure indirect effect (PIE) of the shortcut features, while the causal prediction captures the natural direct effect (NDE) of the causal patterns (VanderWeele, 2013); (2)

Original: From the perspective of information theory (Kullback, 1997), the element-wise multiplication makes the causal prediction reﬂect the conditional mutual information between the causal patterns and ground-truths, conditioning on the complement patterns.
Revised: From the perspective of information theory (Kullback, 1997), the element-wise multiplication makes the causal prediction reﬂect the conditional mutual information between the causal patterns and ground-truths, conditioning on the complement patterns.

Original: F M ORE E XPERIMENTAL R ESULTS F. V ISUALIZATION
Revised: F M ORE E XPERIMENTAL R ESULTS F. V ISUALIZATION

Original: We provide more visualization cases in Graph-SST2 dataset as shown in Figure 6 and Figure 7.
Revised: We provide more visualization cases in Graph-SST2 dataset as shown in Figure 6 and Figure 7.

Original: The rationales are highlighted in deep colors.
Revised: The rationales are highlighted in deep colors.

Original:  [Figure]
Revised: [Equation] [Figure] [Figure] [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Original: [Figure] 
Revised: [Figure] [Table] [Figure] [Table]

Original: [Figure]
Revised: [Figure]

Original:  [Figure]
Revised: [Table] [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: We analyze the performance of DIR w.r.t.
Revised: We analyze the performance of DIR w.r.t.

Original: the hyper-parameter λ .
Revised: the hyper-parameter λ .

Original: As shown in Figure 10, with λ → 0 , DIR degrades to optimize the performance in each environment only, without explicitly penalizing the shortcuts’ inﬂuence on the model predictions.
Revised: As shown in Figure 10, with λ → 0 , DIR degrades to optimize the performance in each environment only, without explicitly penalizing the shortcuts’ inﬂuence on the model predictions.

Original: We also see that all testing performances drop sharply if λ is too large.
Revised: We also see that all testing performances drop sharply if λ is too large.

Original: Since a large weight on the variance term emphasis on the invariance condition while leading to the overlook on the performance loss, it could fail to exhibit f ˜ Y correctly.
Revised: Since a large weight on the variance term emphasis on the invariance condition while leading to the overlook on the performance loss, it could fail to exhibit f ˜ Y correctly.

Original: Notably, such a trade-off in the DIR objective is commonly shared among all the datasets.
Revised: Notably, such a trade-off in the DIR objective is commonly shared among all the datasets.

Original: F. S TUDY OF THE S PURIOUS C LASSIFIERS
Revised: F. S TUDY OF THE S PURIOUS C LASSIFIERS

Original: Here we provide more observations about the predictions of the learned spurious classiﬁer, which sheds light on the designed model mechanism.
Revised: Here we provide more observations about the predictions of the learned spurious classiﬁer, which sheds light on the designed model mechanism.

Original: We ﬁrst look into the conﬁdence of predictions and deﬁne
Revised: We ﬁrst look into the conﬁdence of predictions and deﬁne

Original: [Equation]
Revised: [Equation]

Original: [Table]

Original: [Table]
Revised: [Table]

Original: where H is the entropy function, and a lower ν indicates higher conﬁdence.
Revised: where H is the entropy function, and a lower ν indicates higher conﬁdence.

Original: We report the results for the trained spurious classiﬁers in Table 4.
Revised: We report the results for the trained spurious classiﬁers in Table 4.

Original: Thus, the results demonstrate the marked tendency of the spurious predictions and validate the design of the S − only branch.
Revised: Thus, the results demonstrate the marked tendency of the spurious predictions and validate the design of the S − only branch.

Original: However, we show that spurious classiﬁers are over-conﬁdent and potentially overﬁt to spurious features, which fails to generalize out-of-distribution.
Revised: However, we show that spurious classiﬁers are over-conﬁdent and potentially overﬁt to spurious features, which fails to generalize out-of-distribution.

Original: In Table 5, we evaluate the spurious classiﬁers (taking non-causal features as inputs) on the testing sets.
Revised: In Table 5, we evaluate the spurious classiﬁers (taking non-causal features as inputs) on the testing sets.

Original: We argue that the performance degradation is caused by (i) feature-level problem: it could be theoretically inadequate to infer the label given the non-causal features, and (ii) paradigm-level problem: minimizing the empirical risk only can hardly exhibit stable relations between the features and labels.
Revised: We argue that the performance degradation is caused by (i) feature-level problem: it could be theoretically inadequate to infer the label given the non-causal features, and (ii) paradigm-level problem: minimizing the empirical risk only can hardly exhibit stable relations between the features and labels.

Original: F. C OMPARISON OF P OST - HOC E XPLANATIONS AND I NTRINSIC R ATIONALES .
Revised: F. C OMPARISON OF P OST - HOC E XPLANATIONS AND I NTRINSIC R ATIONALES .

Original: Here we aim to compare the explanations generated by GNNExplainer (Ying et al., 2019) and the rationales exhibited by DIR.
Revised: Here we aim to compare the explanations generated by GNNExplainer (Ying et al., 2019) and the rationales exhibited by DIR.

Original: Speciﬁcally, we generate post-hoc explanations from GNNExplainer for Spurious-Motif, where we use the models trained under ERM as the models to explain.
Revised: Speciﬁcally, we generate post-hoc explanations from GNNExplainer for Spurious-Motif, where we use the models trained under ERM as the models to explain.

Original: We compute the precision of the explanations in Table 6.
Revised: We compute the precision of the explanations in Table 6.

Original: [Table]
Revised: [Table]

Original: The explanations generated by GNNExplainer reﬂect the models’ inner mechanism, which backs that deep models easily learn from data bias (especially when b is large), being at odds with the true reasoning process that underlies the task.
Revised: The explanations generated by GNNExplainer reﬂect the models’ inner mechanism, which backs that deep models easily learn from data bias (especially when b is large), being at odds with the true reasoning process that underlies the task.

Original: Moreover, even when spurious correlations do not exist, the precisions of rationales generated by DIR still outperform the precisions of the post-hoc explanations, showing the effectiveness of DIR when identifying causal features.
Revised: Moreover, even when spurious correlations do not exist, the precisions of rationales generated by DIR still outperform the precisions of the post-hoc explanations, showing the effectiveness of DIR when identifying causal features.

Original: G O PEN D ISCUSSIONS
Revised: G O PEN D ISCUSSIONS

Original: Based on this work, we provide open discussions and future directions for the research community, which are inspired by the insightful comments of the  reviewers.
Revised: Based on this work, we provide open discussions and future directions for the research community, which are inspired by the insightful comments of the ICLR reviewers.

Original: G. E XPRESSIVENESS OF R ATIONALE G ENERATORS
Revised: G. E XPRESSIVENESS OF R ATIONALE G ENERATORS

Original: High expressiveness of the rationale generators could be beneﬁcial for the identiﬁcation of causal features.
Revised: High expressiveness of the rationale generators could be beneﬁcial for the identiﬁcation of causal features.

Original: Therefore, we have offered additional techniques in our implementation to improve the expressiveness of the graph encoder.
Revised: Therefore, we have offered additional techniques in our implementation to improve the expressiveness of the graph encoder.

Original: Speciﬁcally, we incorporate distance encoding measures (Li et al., 2020) like shortest-path distances as the extra node features for better structural representation learn- ing.
Revised: Speciﬁcally, we incorporate distance encoding measures (Li et al., 2020) like shortest-path distances as the extra node features for better structural representation learning.

Original: Also, more powerful graph encoders like RingGNN (Chen et al., 2019) and 3WLGNN (Maron et al., 2019) can be used as the graph encoders to distinguish different substructures better.
Revised: Also, more powerful graph encoders like RingGNN (Chen et al., 2019) and 3WLGNN (Maron et al., 2019) can be used as the graph encoders to distinguish different substructures better.

Original: G. G ENERALIZATION TO U NSEEN S PURIOUS P ATTERNS
Revised: G. G ENERALIZATION TO U NSEEN S PURIOUS P ATTERNS

Original: In our implementation, the memory bank only contains the spurious patterns seen in the training set, while it could possibly fail to unseen spurious patterns.
Revised: In our implementation, the memory bank only contains the spurious patterns seen in the training set, while it could possibly fail to unseen spurious patterns.

Original: And we provide discussions and solutions to solve this limitation as follows.
Revised: And we provide discussions and solutions to solve this limitation:

Original: By attribute level perturbation.
Revised: • Attribute level perturbation.

Original: When the spurious patterns in the testing are different from those in training set only on the attribute level (node attributes or edge attributes), we can perturb the node/edge attributes of the subgraphs before intervention.
Revised: When the spurious patterns in the testing are different from those in training set only on the attribute level, we can perturb the node/edge attributes of the subgraphs before intervention.

Original: And such perturbation is expected to improve the model’s robustness during inference.
Revised: And such perturbation is expected to improve the model’s robustness during inference.

Original: By an external knowledge base.
Revised: • External knowledge base.

Original: When the spurious patterns also change on the structure level, for example, a star-shaped unseen base graph appears in the testing set of the Spurious-Motif, one potential solution is to resort to prior knowledge.
Revised: When the spurious patterns also change on the structure level, for example, a star-shaped unseen base graph appears in the testing set of the Spurious-Motif, one potential solution is to resort to prior knowledge.

Original: We can enrich the memory bank with possible spurious patterns, e.g., tree (seen) and star (unseen) base graphs.
Revised: We can enrich the memory bank with possible spurious patterns, e.g., tree (seen) and star (unseen) base graphs.

Original: With the external knowledge base, the model can be trained to recognize these possible spurious patterns and be well generalized to the testing dataset.
Revised: With the external knowledge base, the model can be trained to recognize these possible spurious patterns and be well generalized to the testing dataset.

Original: By subgraph matching.
Revised: • Subgraph matching.

Original: In a more tricky scenario when the external knowledge base is not available, we can integrate our model with subgraph matching algorithms in the inference.
Revised: In a more tricky scenario when the external knowledge base is not available, we can integrate our model with subgraph matching algorithms in the inference.

Original: For example, we can extract the training rationales into another bank ˜ C and use them to query the testing graphs ( i.e., checking if similar patterns exist in the testing graphs).
Revised: For example, we can extract the training rationales into another bank ˜ C and use them to query the testing graphs, i.e., checking if similar patterns exist in the testing graphs.

Original: The match results may assist the rationale generator in highlighting the causal features and avoiding unseen spurious features.
Revised: The match results may assist the rationale generator in highlighting the causal features and avoiding unseen spurious features.

Original: G. H IGHER L EVEL I NTERPRETABILITY
Revised: G. H IGHER L EVEL I NTERPRETABILITY

Original: The interpretability of GNNs in the features level implicitly demandsfor the separability of a graph into causal and non-causal features, while we see cases going beyond such assumption ( cf.
Revised: The interpretability of GNNs in the feature level implicitly demands the separability of a graph into causal and non-causal features. At the same time, we see cases going beyond such assumption ( cf.

Original: Appendix C.1).
Revised: Appendix C.1).

Original: We believe it would be exciting to consider higher-level interpretability, e.g., the interpretability of representations (Wang et al., 2021a; Chan et al.,2021).
Revised: We believe we could resort to higher level interpretability. For example, • Interpretability of representations (Wang et al., 2021a; Chan et al.,

Original: Instead of highlighting important features for the model decisions,we can take a more ﬁne-grained look at the high-dimensional representations.
Revised: Instead of highlighting important features for the model decisions, the general goal of representation interpretability answers

Original: An interesting and general question would be “What’s the information encoded by the i -th element of the embedding in the j -th layer?”. 
Revised: “What’s the information encoded by the i -th element of the embedding in the j -th layer?”. • Interpretations on top of disentangled variables.

Original: Then, we can possibly pursue the subsequent research questions like “How can the interpretations beneﬁt our understanding of models, especially in the real applications.” or “How can the interpretations improve the model’s abilities?”.
Revised: Each disentangled latent variable reveals one independent generative factor in the data (Bengio et al., 2013). By generating importance score on these variables, we could possibly obtain more semantically rich interpretations than featurelevel interpretability. Wherein, we believe there are fewer constraints on the separability of features, thus could be able to apply to a broader range of data-generating assumptions.