Quantifying Distributional Invariance in Causal Subgraph for IRM-Free Graph Generalization
Authors: Yang Qiu
Yixiong Zou
Jun Wang
Wei Liu
Xiangyu Fu
Ruixuan Li
Venue: NeurIPS 2025 poster ()
Keywords: graph neural networks; out-of-distribution
Primary area: general_machine_learning
TL;DR: This paper introduces an IRM-free, norm-guided invariant distribution criterion for causal subgraph discovery in graph OOD generalization.

Abstract: Out-of-distribution generalization under distributional shifts remains a critical challenge for graph neural networks. Existing methods generally adopt the Invariant Risk Minimization (IRM) framework, requiring costly environment annotations or heuristically generated synthetic splits. To circumvent these limitations, in this work, we aim to develop an IRM-free method for capturing causal subgraphs. We first identify that causal subgraphs exhibit substantially smaller distributional variations than non-causal components across diverse environments, which we formalize as the Invariant Distribution Criterion and theoretically prove in this paper. Building on this criterion, we systematically uncover the quantitative relationship between distributional shift and representation norm for identifying the causal subgraph, and investigate its underlying mechanisms in depth. Finally, we propose an IRM-free method by introducing a norm-guided invariant distribution objective for causal subgraph discovery and prediction. Extensive experiments on two widely used benchmarks demonstrate that our method consistently outperforms state-of-the-art methods in graph generalization. Code is available at https://github.com/anders1123/IDG.

Quantifying Distributional Invariance in Causal Subgraph for IRM-Free Graph Generalization
Yang Qiu Yixiong Zou',, Jun Wang"", Wei Liu', Xiangyu Fu!, and Ruixuan Li""
Sccooo of Computer Science and Technology, Huazhong University of Science and Technology. Wuddo Tech
(andeer,, yixionga, xy_fu, rxli)Ohust. edu.
weiliung@gaii..oom com "jaangdimudao. tech
Abstract
generalization under distributional shifts remains critical challenge for graph neural networks. Existing methods generally adopt the Invariant Risk Minimization (IRM) framework, requiring costly environment annotations or heuristically generated synthetic splits. To circumvent these limitations, in this work, we aim to develop an IRM-freee method for capturing causal subgraphs. We first identify that causal subgraphs exhibit substantially smaller distributional variations than non-causal components across diverse environments, which we formalize as the Invariant Distribution Criterion and theoretically prove in this paper. Building on this criterion, we systematically uncover the quantitative relationship between distributional shift and representation norm for identifying the causal subgraph, and investigate underlying mechanisms in depth. Finally, we propose an IRM-free method by introducing norm-guided invariant distribution objective for causal subgraph discovery and prediction. Extensive experiments on two widely used benchmarks demonstrate that method consistently outperforms state-of-the-ar methods graph generalization. Code available at lhtps //githule com/ anders1123/IDG
Introduction
Graph Neural Networks (GNNs) have demonstrated exceptional performance various graph However, they typically assume that training and testing data independent and identically distributed (the i.i.d assumption), a condition that often fails in real-world applica- tions 81. When the testing distribution diverges from the training distribution, model performance can deteriorate substantially. This has motivated the community to investigate out-of oft-o--disttibution generalization in GNNs, aiming make models robust unseen environments.
tasks [17 42 45
To address diitrii-ittiittionn generalization in graphs, most methods are designed to extract the task-relevant causal subgraph/subfeatuurs [5 4
44 47 48). They typically adopt the 18 to train a classifier that maintains predictive consistency across environments, thereby capturing causal features. Despite achieving promising results, IRM requires explicit environment information, such as carefully divided environment data and labels, which is costly to obtain. Alternative approaches for generating synthetic environments via prediction or perturbation also face inherent limitations. This raises challenging research question: can we circumvent IRM and capture the causal subgraph?
6 9 22
32
framework of Invariant Risk Minimization (IRM)
"Corresponding authors. This paper is a collaboration between Intelligent and Distributed Computing Laboratory, Huazhong University of Science and Technology and iWudao Tech.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Env
Env?
Rune danovrt from divens ceninonmeen
Rave desavar frome diverse orrinonmen
Cassal Sabgoggh
Not-cuioul Sudgraph
Cauval Sudgraph
Non con-causal Salgraphh
Figure 1:
Our intuition: causal subgraphs exhibit smaller distributional shifts across environments than non-causal ones. For instance, in Motif, the causal subgraph comprises three typical motifs across environments, while the base graph shows diverse. Similarly, molecular properties rely on small, stable set of substructures while non-causal components vary widely across environments.
Toward this end, in this work, we identify an intuitive phenomenon for detecting causal subgraphs without IRM requirements: the causal subgraph varies far less than the non-causal one across en- vironments. As shown in Figure T in the synthetic Motif dataset [8] the causal subgraph always comprises the three canonical motif structures, while non-causal components vary across environ- ments. Similarly, in chemical molecules, properties often rely on a limited set of substructures (termed fingerprints moieties (38)) that remain stable across environments, whereas the remaining parts vary markedly. Based on this observation, we propose the following hypothesis: causal subgraphs exhibit significantly smaller distributional shifts across environments than non-causal ones, which uncovers an approach for extracting causal subgraphs while bypassing IRM. In this paper, we first establish it as the Invariant Distribution Criterion and offer theoretical justification for it.
However, quantifying distributional shifts of subgraphs is scarcely addressed in graph learning. Drawing on prior work in other fields [16] 30, 23 24 25 26], we demonstrate that, in graph models, distributional shifts lead to a reduction in the activations and representation norms of model outputs similarly. Furthermore,
find the quantitative relationship between distributional shift and samples' representation norms: activations and representation norms systematically decay as distribution shift intensifies, and investigate the underlying mechanisms. This quantitative relationship indicates that representation norms can serve as a proxy for quantifying the extent of distributional shifts in input subgraphs.
Building on these insights, we propose a innovative objective for identifying causal subgraphs: the norm-guided invariant distribution objective, which maximizes subgraph representation norms to constrain and minimize subgraph cross-en enviromeitn shift, and extract causal subgraphs efficiently without IRM. Leveraging this objective, develop novel framework for causal subgraph discovery and prediction. The main contributions are as follows:
We introduce and theoretically validate the Invariant Distribution Criterion, revealing that causal subgraphs exhibit significantly smaller distributional shifts across environments than non-causal subgraphs. This criterion holds naturally under distributional shifts, without necessitating the explicit environmental requirements of IRM.
We systematically uncover the quantitative relationship between distributional shift and representa- tion norm in graph data for identifying causal subgraphs, and investigate underlying mechanisms.
Grounded the criterion and quantitative relationship, we propose an innovative IRM-free method termed IDG (Invariant Distribution Generalization) by introducing norm- norm-guided invariant distribu- tion objective for causal subgraph discovery and prediction.
Extensive experiments on two widely used benchmarks demonstrate that our method surpasses the state the art in graph out-of-distribution generalization.
2 A Closer Investigation into Invariant Distribution
This section is structured as follows: Section 2.1 presents the preliminaries; Sections 2.2 and 2.3 provide theoretical and technical insights of invariant distribution, respectively; and Section 2.4 introduces the final practical paradigm for outto--distiibttioo generalization.

---

2.1 Preliminaries
Notations: Let := (V,E) ) be an undirected graph with nodes and m edges, represented by its adjacency matrix A and node feature matrix X E IRax with feature dimensions. We write G. and G, for the invariant (causal) and spurious subgraphs of G. 9o : + 9 and he 9 + y denote the subgraph extractor and predictor with parameter 0, & respectively, and 7 represents the extracted subgraph Z given by Z = (o) Yz is the predicted label given by Yz label. We denote the training and testing set as D., Jeet,, Dre = JeEEs, Etr + &te.
(Z) while Y is the true
Problem Definition: We focus on OOD generalization in graph classification. Given a collection of graph datasets D = {G* JeEE the objective of OOD generalization on graphs is to learn an optimal GNN model (-) 9 Y with data from training environments Dir {G* JeEE, that effectively generalizes across all (unseen) environments:
(*)) arg min sup R(f ce,,
(I)
eEEull
where R(f = [e Y)] is the risk of predictor f(() in environment e and ((..-) denotes the loss function. Specifically, f(-) = ge o he in subgraph-bast based methods.
Invariant Risk Minimization (IRM) M is learning principle that seeks feature representation on which a single classifier remains simultaneously optimal across multiple training environments, thereby capturing causal features and improving robustness to distribution shifts. In its ideal form, IRM solves the bi-level problem:
min Σ Re (w
s.t. W E araggm min Rd ( o Φ) for all e E
330
where e is feature extractor, w is classifier, and Re the risk environment e.
To enforce classifier optimality across different sub-distributions (the argmii constrait), IRM partitions the training data into explicit environment--peccffc subsets, requiring explicit environment labels which is typically unavailable in practice. More details of IRM are included in Appendix B
48 and is founded on Structural Causal Models (34], which elucidate how latent factors give rise to observable graph properties. We model graph creation via function fgen 2 - g. where 2 R." is the latent space and G the space of graphs. Under this SCM perspective, generation decomposes into three stages-first first producing the causal subgraph G., then the spurious subgraph G., and finally the full observed graph G:
Data Generating Process in work follows prior study 4
(Ge, Ge := SEn G. = -fea(S)) G == ( G.),
(2)
Let C and S be latent codes for causal and spurious factors. The observed graph G splits into causal subgraph G. (from C') and a spurious subgraph G.(from S). C' drives the target Y, while vary across environments. Prior works distinguish Fully Informative Invariant Features(FIIF) when 1 S|C and Partial Informative Invariant (eatures(PilF) when Y S|C (refer to Appendix
2.2 Theoretical Insights: Invariant Distribution Criterion
As illustrated by the intuitive phenomenon in Figure 2, causal subgraphs exhibit significantly smaller distributional shifts across environments than non-causal ones. In this section, we present the theoretical proof of this hypothesis, termed Invariant Distribution Criterion.
We consider input graph G with associated label Y, observed under multiple environments E. Let G. denote the causal subgraph of G that contains the true causal features determining Y, and let G. G \ Ge be the remaining non-causal (spurious or confounded) parts of the graph. As in previous work of graph generalization, we here disregard the effects of noise and other irrelevant factors. We formalize the problem with the following assumptions:
The label Y is generated from G exclusively via Go. In particular, there exists a fixed causal mechanism == f(.)) that is invariant across environments. Equivalently, Y YLe Lel for all environments e, meaning the conditional distribution Pa G.) is the same for every e E E. (In other words, Ge contains all the information needed to determine Y, and this causal relationship does not change with the environment.)
Assumption 1. Causal Mechanism:
3

---

This assumption means that the generation of causal labels is fully determined by the causal subgraph Ge. This assumption derives from the Independent Causal Mechanism (ICM) hypothesis [35] and underpins Graph OOD methods such as CIGA (S and LECI [9] Invariant Risk Minimization (IRM) likewise relies on it by seeking representations that yield the same optimal classifier across environments. For example, if the causal subgraph (e.g., functional groups responsible for key chemical properties) is known, one can infer a molecule's chemical properties regardless of its distribution of origin (environments).
Assumption 2. Environmental Diversity and Interventions: Across different environments, the distribution of the imput graph G may change due to interventions or context changes, but these changes are sparse in the sense of affecting only parts of the data- data-generating process at time. In particular, we assume that environmental changes tend to affect the non-causal parts G, more significantly (or independently) than the causal part Ge-
This assumption originates from Invariant Causal Prediction (ICP) framework and its extensions [3 36 33, this assumption holds that distributional shifts across environments arise from sparse interventions on the non-causal subgraph Gg. while Ge remains stable. This hypothesis has been adopted by Graph OOD methods such as LECI, which impose even stronger assumption that the environment E' is independent of Ge. i.c. E' 1 Ge. For instance, the types and structures of functional groups in a molecule are limited and stable, whereas non-causal parts (e.g., heteroatoms) vary widely across environments.
Assumption Effective Classifier Support: We assume the classifier ho(+) (to be learned) is powerful enough to model the true causal relationship fG.)). The classifier is trained on data from some source environment(s) and achieves low error on those. We further assume that if the test environment provides inputs (Ge, Ga) whose causal subgraph component Ge lies within the support of the training distribution of Go, then the classifier can classify such imputs effectively. In contrast, if the test data's ' Ge lies far outside the training support (an out-of- scenario), no classifier can be expected to perform well without additional extrapolation assumptions.
This assumption is based the classical domain-adaptattoo adaptation bounds presented in (2), this assumption requires overlap between the support of source and target distributions to guarantee generalization. Since classification relies on G., the support of the causal-subgraph distribution in the test domain must lie within that of the training domain. For example, a classifier can correctly and consistently label functional group in the test environment only if that group appeared during training.
we provide theoretical guarantees to address the challenges associated with subgraph discovery. We initiate our analysis with these lemmas if the assumptions hold:
Under the framework of the data generation process outlined in Section B
Lemma 1. Invariant Conditional Distribution: Under Assumption 7 the conditional distribution of the label given the causal subgraph is invariant across environments: for all e, E E, PY G.)= = P. G.). Equivalently, Y depends on G, and not on e or G Moreover, no proper subset of features that excludes part of Ge can enjoy this invariance, and any superset including non-causal parts G2 will generally violate invariance.
Lemma follows directly from the SCM: under both PIIF and FIIF shifts, C' is the unique cause of Y. The formal proof of Lemma T is included in the Appendix C.I
Lemma 2. Domain Adaptation Bound for Representation Shift: For any representation Z = (G) used by classifier h.).), let Re (ho) denote the classification risk (error rate) in environment e. For any two environments e (source) and e! (larget), the difference risk is bounded by(take binary classification as example):
(e S Re + 2 dnan(Pe(2) x
duau is the HAH divergence (distribution discrepancy) between the source and target distributions, and X = ming (h') + R.'()') is the minimal combined error achievable on both domains (accounting for labeling function differences). . In particular, if the labeling function is invariant (as with Z G. by Lemma I then x = 0 (no intrinsic target error beyond distribution shift).
Lemma 3. Support Overlap and Classifier Validity: If the support of the target environment's Z. distribution lies within (or significantly overlaps) the support of the source distribution, then any

---

classifier ho(-) that is consistent on the source support can, in principle, maintain its performance on the target. Conversely, if the target Z distribution produces samples outside the source support (out-of-support region), then no learning algorithm trained only source data can guarantee accurate classification on those novel samples.
Lemma is drawn from the established theory of domain generalization 2 Lemma posits that by extracting the causal subgraph Ge. the model is guaranteed to operate within the support of familiar and stable features (avoiding out-of-support samples caused by novel non-causal feature combinations) and thus preserves generalization under distribution shift. The formal proof of Lemma 2 and Bis is included in the Appendix C.2 and C.3
With these lemmas in hand, we now present the main theoretical statements about the invariant distribution criterion for causal subgraph:
Theorem 1. Causal Subgraph Minimizes Distribution Shift Across Environments
For any two different environments e, e', consider measure AGG == d(P,(G), P. of distribu- tion shift for some divergence d(..-)(e.g H.H distance), for any alternative subgraph G' that not purely the causal subgraph, we have:
AGG <(G)
Equivalently, extracting Ge leads to minimal distribution disparity between environments, whereas any inclusion of non-causal parts or exclusion of causal parts increases distribution shifts.
Theorem 2. Causal Subgraph Ensures Support Coverage and Stable Performance
If the extracted subgraph Z is the real causal Go then in any new environment e', its distribution will remain within the training support (or a reasonable interpolation range). Consequently, a classifier h)(.) trained on Ge in the source environment will retain its performance in e', assuming the causal link remains unchanged. In contrast, using a non-causal subgraph G' may produce out-of-sune -upport subgraph imputs in e', i... out-of cuttoo--dsttribu distriibbution for h-)(), leading to failures and unstable results.
We defer the proofs and accompanying discussion of Theorems I and 2 to the Appendix C.4 and C.5
Theorem and have formally shown that the causal subgraph Ge of input graph G provides an invariant and sufficient representation for -itto--distribution generalization. Theorem confirmed that G. experiences the least distribution shift across environments compared any representation entangled with non-causal parts. Theorem Seeeemm@ess esttablished that using Ge ensures (causal subgraphs distribution) in new environments stay within the classifier's support, leading stable performance. Theorem
provides a theoretically rigorous proof of the proposed Invariant Distribution Criterion, while guaranteeing the generalizability of the causal subgraph.
| and _
2.3 Technical Insights: Relation between Distributional Shift and Representation Norm
While the Invariant Distribution Criterion offers a pathway for circumventing IRM, it specifies a challenge for quantifying distributional shifts for graph learning. In this Section, establish pioneering quantitative connection between distributional shifts and representation norms in GNNs: activations and representation norms systematically decay distribution shift intensifies.
2.3.1
Norm and Activation Reduction when Distribution Shift Occurs
0.2
0.1
(a) Activation/Norm train/test data
(b) Accuracy and perturbation ratio
Figure 2: (a) Average activation values and L2 norms in GNN on training (in-distribution) and test sets -iutrff-distributionn)... (b) Accuracy and norms under varying perturbation ratios ratios-highe ratios indicate greater distributional shifts. Motif-basis, Motif-size, and Graph-SST2 from left to right.
5

---

Traederneto Lyye
Tornforraion Layer
Transortiionn Laper
Tranfomaiion Loge:
(a) GIN Motif-basis
(b) GIN SST2-length
Figure 3: SVD results weight matrices GIN (3 graph convolutional layers, each consists 2 transforms) trained with ERM objective. Singular values are sorted in descending order.
Prior work other research area has shown that distributional shift may manifest in network activations and representation norms [16], but the universality of this phenomenon has never been verified on graph learning.
We begin our investigation on the basic GNN. We train the GNN with the empirical risk minimization (ERM) objective in-distribution data and evaluate it on OOD, As shown in Figure 2 (a), find:
Finding 1: Activations and representation norms diminish under distributional shift.
Furthermore, to quantify how shift severity affects representation norms, we consider varying degrees of structural shift. Since manually curating datasets with precise shift levels is impractical, we adopt an alternative inspired by prior work: emulate structural shifts of differing severity by randomly delete and generate fixed proportion of edges (not change edge count). From Figure (b), we find:
Finding 2: Increasing shift severity leads to progressively lower representation norms and a corresponding drop in predictive accuracy.
Findings 1 and show that the representation norm steadily decreases as the severity of input distribu- tion shifts increases, suggesting that the norm In the following Section, we further examine the underlying mechanism of this quantitative relation.
proxy for quantifying distributional shift.
2.3.2 Why Distributional Shifts are Reflected in Norms
Prior studies indicate that the relationship between input distributions and output activations can be driven by the low-rank property of neural network weight matrices Low-rank refers to the case where a layer's weight matrix can be approximated by a matrix with lower rank, leading the network to concentrate on limited set of directions. However, this property has not yet been confirmed for graph neural networks (GNNs). We find the same behavior in the weight matrices of GNNs, as shown Figure 3 Singular value decomposition results reveals that only small subset of singular values is large, while the remainder are relatively small or near zero, demonstrating that the network attends constrained of directions. More results are included in the Appendix E
Specifically, owing to the low-rank nature of neural networks- where weight matrices in certain inherently represented by lower-rank factors-the weights in networks
layers can be approximated attend only to limited set of principal directions. When inputs within or near the training support is mapped by such a weight matrix, its energy (or norm) is preserved primarily along those directions that are amplified or effectively transmitted, yielding higher activation values. Under substantial distribution shift, however, the dominant components of shifted inputs may fail to project efficiently onto the low-dimensional subspace to which the network weights are "tuned" during training. In other words, these input features misalign with the weight matrix's 's' principal directions, resulting in smaller projections and consequently lower activations- overall, the representation norm decreases. We present more example illustrate how different inputs manifest in norm in Appendix
2.4 Practical Paradigm: Norm-Guides Invariant Distribution Objective
Based on the theoretical and quantitative insights in Section 2.2 and we can draw the following inference: Causal features remain aligned while spurious or misaligned features collapse in distinct environments, yielding higher representation norm for the causal components. Leveraging this insight, we introduce novel paradigm for extracting causally invariant subgraph Z. termed Norm- Guided Invariant Distribution Objective. The objective to maximize the representation norm of the extracted subgraph Z while minimizing the distributional shift across environments, formulated as:

---

Norm-Guided Invariant Distribution Objective:
[[[orm((Hz)) 1(
max 2
(5)
Minimal Distribution Shifl Stable Prediction
S.r. 7 = go (G,Y) DDDDDDeeeaa Desseg: 61
62
Z is the extracted subgraph, Hz the representation given by the preditcor ha and [(; mutual information. Intuitively, the network's ' feature norm can be viewed as a proxy for how much signal from the input is influencing the prediction. The causal subgraph, being the truly predictive part of the input, is the only subset that can consistently supply strong signal across environments. Spurious features might look useful a training set of a definite environment, but when the environment changes, the network effectively "turns down the volume" on those inputs resulting in small-norm features and a prediction that defaults to a constant, since the distribution of spurious features is less stable than that of causal features, according Theorem [ The causal features, by contrast, continue to drive high-magnitude representations and confident predictions in different environments. Therefore, optimizing for maximal feature norm in the classifier naturally leads it rely the causal subgraph. This theoretical result aligns with the idea that causal features are the most stable and predictive presented in Theorem 2 while non-causal correlations get suppressed under shift, manifesting as low-norm, non-in non-informative representations. Note that our method merely requires the dataset include samples from multiple environments (i.c., exhibit environment shift) and does not rely on explicitly partitioning environments as in IRM.
2.5 Conclusion
Based the above theoretical results, experiments, and analyses, we draw the following conclusions: (1) Causal subgraphs exhibit significantly smaller distributional shifts across environments than non- causal ones (Section 2.2 (2) Representation norms, which systematically decay distribution shift intensifies, can serve as a proxy for quantifying distributional shift for identifying causal subgraphs (Section 2.3 (3) Therefore, by maximizing subgraph representation norms to constrain and minimize
cross-enviromment shift, causal subgraphs can be extracted efficiently. (Section 2.4).
3 Methodology
2 - (CMM
maximize 1(..)
Predictor 4
Stable Prediction
Extractor
M
H2 representation
Prediction Yg
Imput 30
graph
Subgraph
masrimize Norm(Hl))
Minimal Distribution Shift
Figure 4: IDG framework. The extractor selects the subgraph Z and feeds it to the predictor. Subgraph distributional shift are minimized by maximizing the representation norm of Z.
In this section, we propose novel method, termed IDG (Invariant Distribution Generalization) for attaining the Norm-Guided Invariant Distribution Objective.
Framework: Based on the basic extract-predicc framework as presented in Figure Noting the imput graph as
(A,X), where A is the adjacency matrix and X represents node attributes. The subgraph extractor ge G + G is tasked with selecting the optimal subgraph Z from the input graph. We employ the Graph Isomorphism Network (GIN) as the backbone of the extractor and predictor. Specifically, the subgraph extractor generates representation for each node:
HG (G) E IR" nxd
(6)
d' is the representation feature dimension. For each edge (i.j), the representation of node and node j are concatenated and transformed to get the importance score of the edge:
My (MLP, ((H,| = (i.j) G
(7)
7

---

- (Ac. x)
where A. TMp G A)
(8)
Here Top, selects the top edges with the highest importance scores, adopted from in previous work [22). is a predefined hyperparameter commonly set to 0.5. The extracted subgraph Z is then fed into the predictor h to make representations and final predictions:
Hz READOUT((h(()) E ER", Yz (L
Optimization: Guided by the Norm-Guided Invariant Distribution Objective in Equation the subgraph extractor identifies the causal subgraph that minimizes distributional shift (i.e., maximizes the representation norm), while the predictor infers the corresponding labels. We then reformulate the optimization objective of Equation i the following practical scheme:
Extractor: Lg = CE(Y2, Y) + X1 [[log(( Hzll2)l + >2 -coop
(10)
Predictor: L. = CE(2, Y) s.t., Z = =ga(G); (G,Y) ~ Der,e2s
The coefficient λ1, A2 is a task-dependent balancing parameter. CE(.,)) -) is the cross-entropy loss.
The -log( Hz (|2) term, by maximizing the representation norm of the selected subgraph, compels the extractor to identify the subgraph exhibiting minimal distributional shift (i.e., the causal subgraph) in accordance with our Invariant Distribution Criterion, thereby resulting in stable predictions.
We also adopt the entropy term from prior work (50 31 to encourage compactness of the extracted subgraphs: Leomp = (1 M) log (1 M) M Mlog M]
During training, since the extractor and the predictor optimize different objectives, we split each epoch into two stages: the extractor's parameters are first frozen while the predictor is updated via Equation I0, then the predictor's parameters are frozen and the extractor is updated. During inference, the subgraph extractor directly extracts the subgraph, and the predictor makes predictions based on it.
Experiments
4.1 Datasets, Metrics and Baselines
We adopt two widely used benchmarks for graph OOD generalization generaizzation-Graatooo g and Dru- gOOD 15] across seven datasets: Motif, CMNIST, HIV, SST2, and Twitter from GraphOOD, and EC50 and IC50 from DrugOOD. These datasets span synthetic, superpixel, molecular, and text graphs. Each dataset contains one or more domains and is divided into domain splits, thereby introducing distribution shifts. ROC-AUC metric used for the binary classification dataset and Accuracy for the others. Refer to the Appendix H for dataset details.
Following [9] we employ GIN for both the extractor and predictor, set (A1, A2) (0.1, 0.01), and retain the original learning-rate and batch-size settings. We compare our method with several com- petitive baselines, including empirical risk minimization (ERM), four traditional out-of-distt (OOD) baselines including IRM [. VREx [19. and Coral [3). and eight graph-specific OOD
baselines including DIR GIL 22 GSAT 322. CIGA [5). LECI (9, iMoLD 51 EQuAD 47
and LIRS 49). Refer to Appendix for details about baselines.
4.2 Comparison with State-of-the-Art methods
Table 1 present comparison with IDG and other OOD methods on the GraphOOD and DrugOOD datasets. From the results, we can conclude that IDG attains state-of-the-art performance on 15 out of 17 datasets and achieves comparable results on the other two, demonstrating its superior generalization ability across different types of datasets and domain shifts. Morcover, some OOD methods even underperform ERM on certain datasets because the complexity of the domain information precludes their simple division into few explict environment splits, further underscoring the advantage IDG.
4.3 Contribution of Norm-Guided Invariant Distribution Objective
To rigorously assess the impact of the proposed Norm-Guided Invariant Distribution Objective (hereafter "our objective""). introduce baseline model obtained by removing the norm term from
We do not impose any restrictions on the node attributes X because nodes with all edges masked will be excluded from the GNN's pessagg-passsing process, thus not influencing the results.
8

---

Table Results on GraphOOD and DrugOOD dataset rounds.
Dataset
Motif
CMNIST
HIV
SST2
Twitter
IC50
BC50
Domain
size
basis
color
scaffold
size
length
longth
scaffold
size
assay
scaffold
size
assay
ERM
53.464001
63.8
27,82 69 55 59. 19 10.52 57.040.2 68, 79 67 67.50 71.6300 64 28..2 65. 10 67 59
IRM
53,68
59.93
29.04 29
10.17 59.94.
m0 57.720m
67. 62.22 61 58.88 71.15 63.860
19
67.77
53.71
66.23mm
19 70.69k.29 s0
22 34.140
68. (60361 53,33 71,28 64 830s0 48.47
72.08
VREX
m4
66.334co
27 17.65.300 so 69. s4 58.49 1o 80.20) 19
56.37mo0 67.320 63.47,94 on; 70,33000 63.63009 59 890 01 69. 20.2280.. 54
DIR
44.83
39,99
26.20v440
68.442230
57.67 81 31.55 56,81
66, 66,33
62 69.84 63,76022
61. 65.811200
53.92;44;
64.23
27.13
69.430.30
49.22(134)
m0 55.40
65,38mm7
63.06 so: m 62.5662.0 61.73 66.84
GSAT
60.765a4
55.136-41
35.62
70.07
60.73
81.490
56.07ams
66.45us5
66.70aam: 70.59 64. 64.25euam 62. 22.651779) 73.82/263)
CIGA
14.42 67.15
32.11
69..401mm
19.55e-50 s0 30.46:2000 57.190.5
69.14
66.92/954 71.860 so 67.33x 39 65.65
69.15
71.43138
73,16220
51.80
71.361.9
(4) 83.44:3.n 57.63004
MMoLDD
25
48.35
72.930.2
62. (6) 82.13:3 56.461126
68 54.44
67.92944 4x; 12s 67.79k65 67.00 0Ma0: 77, 77.481
EQLAD 59.72
18.98.230)
72.24m044 69.19 82. 62.571530
57.470.49
69.27
68. 19.2:4 73.26144) 68. 120440 66. 37
79, 36..66
LIRS 74.98mmn
15.5
49,871240
72.82040
66.64144 82.48ia7v 58.29gum
69.78mm 68 68.32 72.56i0as) 68.17 67. 67.23 79.46.490
IDG
73.23
82.53xa0
55.32am 73.24mat
67.442230)
83 59.76 69.97a33 69.02 20 72.861054) 68.32m..0 68.03
80.54
108
Baseline
DG
Figure 5: From left to right: (1) Training loss vs. epochs in IDG and baseline (2) Testing loss vs.
epochs in IDG and baseline (3) Testing Accuracy vs. epochs in IDG and baseline (4) I-SNE results of IDG representations (5) t-SNE results of baseline representations
Equation TO and both models were trained and evaluated on the ground-truth truth--nnotttedd Motif-basis dataset. From the results in Figure and Table 2 we draw the following conclusions:
Our objective accurately captures distributional shifts. Although the IDG and baseline model exhibit nearly identical loss trajectories on the training set, the testing loss in baseline anomalously increases in the test set (out-of-dine citt-o--ddsrrbuttinn). whereas the IDG's testing loss continues decline in tandem with its training (Figure [] (1-2)). These results indicate that our objective captures distributional shifts more accurately.
Our objective significantly enhances out-of- distribution generalization. In Figure 5)3). adding our objective accelerates convergence and delivers improved performance. Moreover, t-SNE visualizations (Figure 5 (4-5)) reveal that IDG produces more distinct embeddings for out- of-distribution samples than the baseline.
Table 2: Results with ground-truth Motif.
Muni-backs
Motif-ene
pr
pro
Basoline IDG
0.805
0.4004
0,60377
0.5064
0.9944
0 20099
0.3164 0.3418
012400
0,7337
0.4444
0.7381
0.5547
0,9373
0.2967
0.3171
Table 3: Ablation study results.
Metbod
Motif-basis
CMNIST
Twitter
BC50-assay
Enhanced causal subgraph extraction. As shown in Table 2 subgraphs extracted by IDG align more accurately with (edge) ground-truth, demonstrating that Our Objective steers the model capture causal subgraph faithfully.
ERM
$3.46
27.82
$7.04
67. 39
53.25
27.34
w/o LoB
66.58
48.32
57.47
69.02
w/o CComp
75.41
52.53
78.90 73.96
w/o Exarm.
76.12
49.64
58.43
76.64
IDG
77.83
55.32
59.64
80.54
Twitter
IC50-Asay
4.4 Extended Verification
Hyperparameter Sensitivity To assess IDG's sensitivity to hyperparameters, we conducted experiments on Twitter and IC50-assay. The results in Figure 6 (a) demonstrate that IDG is insensitive to the choice of parameters, indicat- ing its stability across different configurations.
0.3
0.003
0.1
0.01
0.02
A
A
Figure 6: Hyperparameter sensitivity on X1, X2
A Alation Study We conduct ablation study to evaluate the impact of different components of IDG. The results are shown in Table 3 In the table, ERM and ERM+LN correspond to GIN trained with ERM and ERM augmented by the invariant distribution objective, respectively. The results show that merely increasing the representation norm hardly improves generalization. w/o LCE, w/o LComp and w/o NNrrm denote training without the cross entropy, compactness constraint and norm for the extractor respectively. The results indicate that all objectives enhance performance, and confirm the effectiveness of our objective.

---

Efficiency Study The time complexity of the IDG method is (mdd) where n, m, and denote the average number of nodes, edges, and
Table 4: Training and inference time (ms).
Method Motif-basis
HIV-scaffold
SST2-length
feature dimensions per graph. Specifically: (1) message passing and node updates per layer in- cur a cost of O(md + +dd).. ). (2) edge scoring and lop-r selection can be completed in (m log m) in the worst case. (3) norm regularization and compactness term require (m) time. In our method, the norm caleulation incurs minimal computa- tional overhead. To further demonstrate its efficiency, we measured training time and inference time. The results are presented in Table 4 Extended results about visualizations, hyperparameter r and
Train Inference
Train Inference Train Inference
GIL
56801
2037
102594
22057
86658
16472
DIR
14415
794
38477
1098
32392
9377
IDG
11262
493
35232
995
26326
7659
efficiency are included in the Appendix K andd]
5 Related Work
Out-of-distribution Generalization in Graph. OOD generalization is a critical challenge in graph learning, where models trained on a specific data distribution often fail to generalize well unseen distributions. IRM [. which seeks to learn causally relevant representations that remain stable across different environments, is widely adopted in graph generalization [46] 51 22 21
44 29
5
These
methods either depend on provided or predicted environment labels or on enviromment-specifie causal assumptions, which limits their practical applicability. Although some methods
49] 47 32
extract causal features using Infomax or the Information Bottleneck principles without necessitating environment labels, they only partially address the causal- -ppurioss distinction. To bridge this gap. we introduce an novel criterion via an entirely distinct paradigm. More discussions are in Appendix G
Low-Rank and Distribution Shift. Previous studies have shown that trained deep neural networks generally exhibit low-rank property [14 12 41.. but this phenomenon has not been rigorously evaluated in graph models. [16] demonstrated that distribution shifts manifest in the network's activations, which they attributed to the low-rank structure of neural weight matrices, However, these phenomena remain unexplored and unverified within graph model, and provides thorough analysis of this phenomenon in this work.
Subgraph-base based Methodology Subgraph- based methods have gained prominence in graph out- of of-generalization and explanation due to their ability to capture local structures and patterns. The extractor architecture employed in this work, which maps node features to edge masks, is also widely used in Graph OOD research such as DIR 44, GIL 222. LECI 9 CIGA (5), and GSAT 32
Its origins trace back to earlier graph explanation methods like PGExplainer (50 31 37.. as well as similar techniques in the text (20 and vision domains 377. The central challenge these methods is establish principled rigorous framework for accurately identifying causal sub-features. such as IRM and its variants adopted in prior works like DIR and GIL. In contrast, our method is completely IRM-free and requires environment information, it identifies causal subgraphs simply by finding solutions that satisfy the minimal shift criterion via norm optimization.
6 Conclusion
In this paper, we propose Invariant Distribution Criterion, which demonstrate causal subgraphs undergo markedly smaller distribution shifts than non-causal ones. By linking representation norms to distribution shift, we derive a practical norm-based objective and instantiate it as IDG. Empirical results on diverse benchmarks show IDG consistently outperforms state-of-the-an OOD baselines.
7 Acknowledgement
This work is supported by the National Key Research and Development Program of China under grant 2024YFC3307900; the National Natural Science Foundation of China under grants 62436003, 62206 102, 62376103 and 62302184 Major Science and Technology Project of Hubei Province under grant 2025BAB011, 2024BA Hubei Science and Technology Talent Service Project under grant 2024DIC078. The computation completed in the HPC Platform of Huazhong University of Science and Technology.
10

---

References
[1] Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant Risk Mini- mization, 2020.
[2] Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jen- nifer Wortman Vaughan. A theory of learning from different domains. Machine Learning, 79(1):151-775, 2010.
[3] Peter Bühlmann. Invariance, Causality and Robustness: 2018 Neyman Lecture. Statistical Science, 3503):004426, 2020.
[4] Yongqiang Chen, Yatao Bian, Kaiwen Zhou, Binghui Xie, Bo Han, and James Cheng. Does Invariant Graph Learning via Environment Augmentation Learn Invariance? Advances in Neural Information Processing Systems, 36:71486-71519 2023.
[5] Yonggiang Chen, Yonggang Zhang, Yatao Bian, Han Yang. M. A. Kaili, Binghui Xie, Tongliang Liu. Bo Han, and James Cheng. Learning Causally Invariant Representations for Out-of- Distribution Generalization Graphs. Advances in Neural Information Processing Systems, 35:22131-22148, 2022.
[6] Shaohua Fan, Xiao Wang. Yanhu Mo, Chuan Shi, and Jian Tang. Debiasing Graph Neural Networks via Learning Disentangled Causal Substructure, 2022.
[7] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario March, and Victor Lempitsky. Networks. Journal of Machine Learning Research, 17(59):1-35. 2016.
Training of Neural
[8] Shurui Gui, Xiner Li, Limei Wang, and Shuiwang Ji. GOOD: A Graph Cit-of--Disributtion Benchmark. Advances in Neural Information Processing Systems, 35:2059-2005 2022.
[9] Shurui Gui, Meng Liu, Xiner Li, Youzhi Luo, and Shuiwang Ji. Joint Learning of Label and Environment Causal Independence for Graph Out-of-Distribution Generalization. Advances in Neural Information Processing Systems, 36:3945-3977 2023.
[10] Kai Guo, Hongzhi Wen, Wei Jin, Yaming Guo, Jiliang Tang, and Yi Chang. Investigating Out-of-Distribution Generalization of GNNs; An Architecture Perspective. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 932-943, Barcelona Spain, 2024. ACM.
[11] Christina Heinze-Deml, Jonas Peters, and Nicolai Meinshausen. Invariant Causal Prediction for Nonlinear Models. Journal of Causal Inference, 6(2), 2018.
[12] Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-Rank Adaptation of Large Language Models. In International Conference Learning Representations, 2021.
[13] Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong. Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open Graph Benchmark: Datasets for Machine Learning on Graphs. Advances Neural Information Processing Systems, volume 33, pages 22118-22133. Curran Associates, Inc., 2020.
[14] Minyoung Huh, Hossein Mobahi, Richard Zhang, Brian Cheung, Pulkit Agrawal, and Phillip Isola. The Low-Rank Simplicity Bias in Deep Networks, 2023.
[15] Yuanfeng Ji, Lu Zhang, Jiaxiang Wu, Bingzhe Wu, Long- Kai Huang, Tingyang Xu, Yu Rong, Lanqing Li, Jie Ren, Ding Xue, Houtim Lai, Shaoyong Xu, Jing Feng. Wei Liu, Ping Luo, Shuigeng Zhou, Junzhou Huang, Peilin Zhao, and Yatao Bian. DrugOOD: Out-of- Diittr--ittiittiion (OOD) Dataset Curator and Benchmark for Al-aided Drug Discovery A Focus on Affinity Prediction Problems with Noise Annotations, 2022.
[16] Katie Kang, Amrith Setlur, Claire Tomlin, and Sergey Levine. Deep Neural Networks Tend To Extrapolate Predictably, 2024.

---

[17] Thomas N. Kipf and Max Welling. Semi-Supervisse Classification with Graph Convolutional Networks. In Sth International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017.
[18] Devin Kreuzer, Dominique Beaini, Will Hamilton, Vincent Létournau, and Prudencio Tossou. Rethinking Graph Transformers with Spectral Attention. In Advances in Neural Information Processing Systems, volume 34, pages 21618-21622 Curran Associates, Inc., 2021.
[19] David Krueger, Ethan Caballero, Joem--enrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. Out-of-Distribution Generalization via Risk Extrapolation (REx). In Proceedings of the 38th International Conference Machine Learning, pages 5815-5826. PMLR, 2021.
[20] Tao Lei, Regina Barzilay, and Tommi Jaakkola. Rationalizing Neural Predictions, 2016.
[21] Haoyang Li, Xin Wang, Zeyang Zhang, Haibo Chen, Ziwei Zhang, and Wenwu Zhu. Disen- Generalization. In Forty-First
tangled Graph Self-super Learning for International Conference on Machine Learning, 2024.
[22] Haoyang Li, Ziwei Zhang, Xin Wang, and Wenwu Zhu. Learning Invariant Graph Represen- tations for Out-of- Distriibbbttn Generalization. Advances in Neural Information Processing Systems, 15:11828-11 11841, 2022.
[23] Shiwei Li, Xiandi Luo, Xing Tang, Haozhao Wang, Hao Chen, Weihong Luo, Yuhua Li, Xiuqiang He, and Ruixuan Li. Beyond zero initialization: Investigating the impact of non- zero initialization on lora fine-tuning dynamics. In Forty-second International Conference on Machine Learning, 2025.
[24] Shiwei Li, Xiandi Luo, Haozhao Wang, Xing Tang, Ziqiang Cui, Dugang Liu, Yuhua Li, Xiuqiang He, and Ruixuan Li. Beyond higher rank: Token-wise input-output projections for efficient low-rank adaptation. In Advances in Neural Information Processing Systems, 2025.
(25) Shiwei Li, Xiandi Luo, Haozhao Wang, Xing Tang, Ziqiang Cui, Dugang Liu, Yuhua Li, Xiugiang He, and Ruixuan Li. Bora: Towards more expressive low-rank adaptation with block diversity. arXiv preprint arXiv:2508.0665 2025.
[26] Shiwei Li, Xiandi Luo, Haozhao Wang, Xing Tang, Shijie Xu, Weihong Luo, Yuhua Li, Xiuqiang He, and Ruixuan Li. The panaceas for improving low-rank decomposition in communication- efficient federated learning. In Forty-second International Conference on Machine Learning. 2025.
[27] Xiner Li, Shurui Gui, Youzhi Luo, and Shuiwang Ji. Graph Structure Extrapolation for Out-of- Distribution Generalization. In Proceedings of the 41st International Conference on Machine Learning, pages 27846-27874, PMLR, 2024.
[28] Yong Lin, Shengyu Zhu, Lu Tan, and Peng Cui. ZIN: Without Environment Partition? Advances Neural Information Processing Systems, 35:24529- 24542, 2022.
When and How to Learn Invariance
[29] Gang Liu, Tong Zhao, Jiaxin Xu, Tengfei Luo, and Meng Jiang. Graph Rationalization with Environment-basse based Augmentations. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1069-1078, 2022.
[30] Wei Liu, Zhiying Deng, Zhongyu Niu, Jun Wang, Haozhao Wang, Zhigang Zeng, and Ruixuan Li. Breaking Free from MMI: A New Frontier in Rationalization by Probing Input Utilization, 2025.
[31] Dongsheng Luo, Wei Cheng, Dongkuan Xu, Wenchao Yu, Bo Zong, Haifeng Chen, and Xiang Zhang. Parameterized Explainer for Graph Neural Network. In Advances in Neural Information Processing Systems, volume 33, pages 19620-196311 Curran Associates, Inc., 2020.
[32] Siqi Miao, Mia Liu, and Pan Li. Interpretable and Generalizable Graph Learning via Stochastic Attention Mechanism. In Proceedings of the 39th International Conference on Machine Learning, pages 15524-15543. PMLR, 2022.
12

---

[33] Judea Pearl. Causality: Models. Reasoning, and Inference. Cambridge University Press, Cambridge, U.K. New York, 2000.
[34] Jonas Peters, Peter Bühlmann, and Nicolai Meinshausen. Causal Inference by using Invariant Prediction: Identification and Confidence Intervals. Journal of the Royal Statistical Society Series B: Statistical Methodology. 78(5):947-1012, 2016.
[35] Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Elements of Causal Inference: Foundations and Learning Algorithms. The MIT Press, 2017.
[36] Niklas Pfister, Peter Bühlmann, and Jonas Peters. Invariant Causal Prediction for Sequential Data. Journal of the American Statistical Association, 1 114(52)) 2019.
[37] Yang Qiu, Wei Liu, Jun Wang, and Ruixuan Li. PAGE: Parametric Generative Explainer for Graph Neural Network, 2024.
[38] Stephanie D. Steichen, Mary Caldorera- Moore, and Nicholas A. Peppas. A review of current nanoparticle and targeting moieties for the delivery of cancer therapeutics. European Journal of Pharmaceutical Sciences, (043):46-427,2220
[39] Baochen Sun and Kate Saenko. Deep CORAL: Correlation Alignment for Deep Domain Adaptation. In Gang Hua and Hervé Jégou, editors, Computer Vision ECCV 2016 Workshops, pages 443-450, Cham, 2016. Springer International Publishing.
[40] Xin Sun, Liang Wang. Qiang Liu, Shu Wu, Zilei Wang, and Liang Wang. DIVE: Subgraph Disagreement for Graph Out-of- Diistribiitiino Generalization, 2024.
[41] Nadav Timor, Gal Vardi, and Ohad Shamir. Implicit Regularization Towards Rank Minimization in ReLU Networks. In Proceedings of The 34th International Conference on Algorithmic Learning Theory, pages 1129-1459, PMLR, 2023.
[42] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph Attention Networks. In International Conference Learning Representations, 2018.
[43] Luzhi Wang, Dongxiao He, He Zhang, Yixin Liu, Wenjie Wang. Shirui Pan, Di Jin, and Tat-Seng Chua. GOODAT: Towards Test-Time Graph Out-of-Distribuns Detection, Proceedings of the 2024.
AAAI Conference on Artificial Intelligence,
[44] Yingxin Wu, Xiang Wang. An Zhang, Xiangnan He, and Tat-Seng Chua. Discovering Invariant Rationales for Graph Neural Networks. In International Conference on Learning Representa- tions, 2021.
[45] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How Powerful are Graph Neural Networks? In International Conference on Learning Representations, 2018.
[46] Nianzu Yang, Kaipeng Zeng, Qitian Wu, Xiaosong Jia, and Junchi Yan. Learning Substruc- ture Invariance for Information Processing Systems, 2022.
Molecular Representations. In Advances in Neural
[47] Tianjun Yao. Yongqiang Chen, Zhenhao Chen, Kai Hu, Zhiqiang Shen, and Kun Zhang. Empowering Graph Invariance Learning with Deep Spurious Infomax. 2024.
[48] Tianjun Yao, Yongqiang Chen, Kai Hu, Tongliang Liu, Kun Zhang, and Zhiqiang Shen. Learning Graph Invariance by Hamessingg Spuriosity. In The Thirteenth International Conference on Learning Representations, 2024.
[49] Tianjun Yao, Yonggiang Chen, Kai Hu, Tongliang Liu, Kun Zhang, and Zhiqiang Shen. Learning Graph Invariance by Harnessing Spuriosity. In The Thirteenth International Conference on Learning Representations, 2024.
[50] Zhitao Ying, Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec. GNNExplainer: Generating Explanations for Graph Neural Networks. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
13

---

[51] Xiang Zhuang, Qiang Zhang, Keyan Ding, Yatao Bian, Xiao Wang, Jingsong Lv, Hongyang Chen, and Huajun Chen, Learning Invariant Molecular Representation in Latent Discrete Space, 2023.
14

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justifiicton right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No] ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
15

---

Justification:
We discuss the limitations of the work in the appendix.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model sell-specifiitiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-texi system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: Full set of assumptions and complete (and correct) proof are provided in the paper and appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We have reported the detailed settings and hyper-parameters. We will release our codes.
Guidelines:
16

---

The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We have reported the detailed settings and hyper-parameters. We will release our codes.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
gublic//uides/Codeee http///eSsaisssssooooololooooooo
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nisscccppblic//guut http:///cc/giiee//eeeuuubiiiiiii
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
17

---

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
They can be found in the paper and appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification:
We have reported the confidence interval in our ablation studies.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula,
call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We have reported the computer resources in the implementation details.
Guidelines:
The answer NA means that the paper does not include experiments.
18

---

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
We conform, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We have discussed the broader impact in the appendix.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our work does not have the risk for misuse.
19

---

Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We have respected the license of assets used in this paper.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of in the package should be provided. For popular datasets, paperssithcode com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re- re-packaged, both the original license and the license of the derived asset (if has changed) should provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do not have new assets in this paper.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
20

---

Answer:
[NA]
Justification:
This paper does not include human subjects information.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: This paper does not include human subjects information.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing. editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification:
The core method development in this research does not involve LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non standadd components.
Please refer to LLM policy (nttps://neuri)) http://euuripss.ccoeeeeeeeeeeeee /00p:////o./o/ooreeee//////////0 for what should or should not be described.
21

---

A Invariant Risk Minimization and Graph Out-of-Distribution Generalization
IRM 1 Invariant Risk Minimization (IRM) is a framework for learning predictors that remain robust under distribution shifts by enforcing that the same classifier is simultaneously optimal across multiple training environments. In its ideal form, IRM solves the bi-level problem
min
(w s.t. w E arg min Re ( o Φ) for all E E 3E0
(11)
where é is a feature extractor, w a classifier, and Re the risk in environment e. Many approaches for out-of-distribution generalization on graphs are based on the IRM framework. In practice, IRMv1 approximates this constraint by adding penalty on the squared norm of the gradient of each environment's 's risk with respect to w, encouraging the learned representation to capture only invariant (i.e., causal) features:
min
Σ (w + +1Σ aEE
V w//== W (
(12)
6, REE
Graph out-of-distribution methods typically build on the frameworks of IRM, which seek to extract the causal subgraph/features and learn an equipredictive classifier across environments in order to capture invariant features, which demands that the dataset be divided into well-defined environments. Depending on the environmental partitioning strategy, these environments fall into three main categories: Approaches such LECI [9] and G-splice (27] depend on environment labels provided in the dataset, but these labels are not always available and incur high annotation costs. Other methods such GIL [22] and OOD-GO GCL (21) use unsupervised clustering to infer environment labels, which may not always align well with real environment distribution. Other approaches such as DIR 44 explicitly create distinct environments by applying causal interventions the dataset. However, designing causal interventions to generate training distributions should require domain expertise or incur additional overhead for different task, and unreasonable designed interventions may fail to remove all spurious features or even damage crucial information (33] Such limitations relying on environment information hinder the deployment of these methods real-world scenarios. Recent work has further shown that recovering real environment information is infeasible without external information 28 In summary, the limitations of these invariant learning methods have prompted explore an alternative approach for uncovering causal subgraphs.
B Graph Data Generation Process
In graph data generation process presented 2 C and S denote latent codes for the causal and spurious factors, respectively. The observed graph G is composed of two latent components: an causal subgraph G, driven by the causal factor C, and spurious subgraph G, driven by the non-causal factor S, regardless of noise. The variable C' causally influences the target Y. whereas S may vary across environments E. Depending on how S interacts with Y conditional on C, prior work typically distinguish two scenarios. i.c., (i) Fully Informative Invariant Features (FIIF) when Y 1 SIC and (ii) Partial Informative Invariant Features (PIIF) when Y // SIC.
In case (i), the invariant factor C' is fully informative (FIIF) to the target label Y and the latent spurious factor provide no further information. In case (ii), the invariant factor C' is only partially informative (PIIF) about Y. spurious factor S can further provide additional information to aid the prediction of Y however, as is directly affected by E, is not stable across different environments. The SCMs for the two scenarios are illustrated in Figure 7 and these two assumptions have been extensively discussed and empirically validated prior work on out-of-distribution graph
tasks 4 5 6
9
22
32
44
47 48) and is founded Structural Causal Models (34).
22

---

E
E
Y
Y
(0) FII SCM
(b) PUP SCM
Figure 7: Structure causal models for graph data generation.
C Proofs for Theoretical Results
C.1 Proof of Lemma 1
Proof. By Assumption 1 Y = S(.)) fixed causal mechanism f(-) that does vary with This means that for any value Jr of G.. P(Y G. = ge) is defined entirely by [((e) and is the same in every environment. More formally, for any measurable subset A of the range of Y and for any 9c. PY A Ge 9e) = 1f(ge) E A, which is evidently independent of e. Hence (Y G.) P(Y G.) for all e. This captures the essence of invariant causal prediction, wherein the correct causal features G. yield predictor that holds across domains.
Now, if we remove any component of Ge, the remaining features would be incomplete causal subgraph, insufficient to fully determine Y. In that case, the conditional (Y Ge) (with Ge G.) would generally depend the part of Ge that is missing. Similarly, if we include any non-causal features from G2 to form an augmented subgraph G. U.,, then PY | Ge U..)) may vary with because G. can carry spurious correlations with Y that differ by environment. By Assumption [ the correlation between G, and Y is not stable: there exists at least two environments e, for which P. G.)) * Pa G.) (since G, has no direct causal link to Y, any association is incidental and can change). Therefore, (Y Ge. G.) would generally differ from PY G.) because conditioning on G, can introduce environment-speccin information. We conclude that only the true causal subgraph Ge (or any superset that does not include spurious features) yields an invariant conditional for Y.
because the relationship between Ge and Y could be confounded by
C.2 Proof of Lemma
Proof. This is standard result from domain adaptation theory 2 We treat each environment as a domain with distribution Pc(2,Y) Y). The HAH-dive -diveegencc between Pa and Pa.(Z) (Z) measures how well classifier can distinguish between source and target representations; twice the supremum difference in probabilities assigned to sets by the two distributions (related to total variation distance restricted to hypothesis class H). The cited bound (with d(De, De denoting this divergence) shows how much the source error can fail to transfer to target. For completeness: one derivation is
be seen as
(e) Rc(h) s 1 (n +>'
(13)
and similarly (h) R_) (h) is bounded by the same quantity, yielding the two-sided inequality mentioned in different form The term X" represents the best possible joint error; if the labeling rule is identical across domains, there exists a hypothesis (namely the Bayes-optimal classifier on that rule) that achieves low error on both, so would small (zero in the ideal case where Bayes error is zero for that representation). Under our Assumption [ the same causal labeling function f(G.) applies in all environments, so for 7 G. one can achieve X by choosing h f. Thus, for the causal subgraph representation,
23

---

1 Red(h") = R1")")) + 2 dHHH((P(GGe) Pc.().),
(14)
2
with h² being the invariant optimal classifier. This quantifies that any degradation in accuracy is due solely to the shift in Ge distribution across and e'. This formal result confirms: an invariant representation (causal G.) minimizes the transferable risk penalty to just the distribution divergence term, whereas a spurious representation incurs an additional irreducible error jump term.
C.3 Proof of Lemma
Proof. This statement reflects basic requirement for generalization under distribution shift. If Supp( Par (Z)) S (uup((P(())) the classifier (-)-)) trained on Pe is at least receiving familiar inputs under Par In the ideal case, if (-) has learned the correct decision rule Pz(Z) (e.g. the true f()) for G.) and the rule remains the same (invariant labeling), will apply equally to P..(Z) as long as those inputs are not qualitatively new. Formally, for any 2 in the target support, since is also in source support, (-) had the opportunity to adjust its decision (or an equivalent z) during training: thus (+)'s prediction at z can be expected to be reliable as in training. If the supports overlap heavily but not completely. we can expect performance to degrade gracefully in proportion to how much probability mass falls the unfamiliar regions.
On the other hand, if Supp(P. (Z)) extends to regions where (Z) has zero (or very low) density, then those z values are effectively never seen during training. A classifier cannot be expected to extrapolate correctly to arbitrarily novel inputs without additional knowledge; in the worst case, an adversarially chosen out-of-support input could be assigned an incorrect label by (+) since (.) has no basis to learn the correct behavior there. In domain adaptation terms, when support does not overlap, the HA H-diveegeecce reaches its maximum (because a hypothesis can perfectly separate source and target supports), yielding a trivial bound Re < R. (h) + 1/2 - + X = (h) + +++ X which means essentially no guarantee of generalization. In summary. overlapping support is a necessary condition for successful transfer, without it, the new domain may contain feature patterns fundamentally outside the model's experience, leading to unpredictable performance.
C.4 Proof of Theorem 1
Proof. We fix two arbitrary environments e (source) and e' (target) and compare the cross- environment divergence A(Z) for different choices of the subgraph Z. There are three typical cases to consider for an alternative subgraph G' that is not equal to Ge:
Case 1: G" includes non-causal parts (G,). In this case, G" can be viewed as =² U where U S G2 is some subset of spurious features (or possibly all of G.. including the trivial case G' = G). Because G, by definition contains the features that are not causally relevant to Y, any correlation between and Y is spurious or enviromment-specifee affect G. significantly; thus the marginal distribution of U (and its correlation with G. or Y) varies across environments. This implies that the joint distribution Pa(G., U) differs from (G.G,, U) to greater extent than P=(G.) differs from Par (G.)).. Intuitively, since Ge is relatively stable but U is highly variable across e and e', including U will amplify the cross-enviromment disparity. Formally, most divergence measures are monotonic under the introduction of additional differing variables; for example, if Pa (G.) Pe (G.) but P. (U) # Pe (U), then the joint divergence satisfies
By Assumption 2 environmental changes
(P. U), (G., U)) Z (U), (U)) > 0. Even if Pe.G) changes slightly across environments, the changes in U (spurious part) are strictly larger (by Assumption D. so A(G. U) will still exceed A(G.).
In particular, consider the -iv-diieegeece as the measure d. If 2 = G. U contains environment- varying spurious components, one can construct a hypothesis in HAH distance that focuses on U effectively distinguish which environment a sample came from. For instance, a classifier h E H. that predicts the environment identity from U will achieve better-than-chant chaance accuracy due to U's distribution shift, implying a large dæaH (P. (z) In contrast, if Z = Ge (with all G, removed), then no classifier can reliably distinguish e' because G. by itself varies minimally the ideal case, (G.) = (G.) if the causal features are entirely invariant. Thus (MAH(P.(G.)) Pe (G.)) will be small (in fact zero if G.'s 's distribution is truly identical across e, e').
24

---

This reasoning formalizes that
A(G,UU) d(P.(GG.U)) P. (c...) (G.)) AG..)
Hence any inclusion of non-causal features U increases the divergence across environments.
Case 2: G' excludes part of the causal subgraph. In this scenario, G' is a strict subset of Ge (or possibly disjoint, but disjoint subgraph would be pure G. which is covered by Case 1). Let Ge ===C0 G' U Cmiss. where Caiss is the portion of the true cause that is left out of G'. Because G' is missing some of the true causal features, it no longer fully determines the label Y. In fact, by Lemma [] Y is not conditionally independent of the environment given G' since G' omits part of G.. the remaining features alone cannot guarantee the invariant relationship with Y. Equivalently, the effective labeling function on G' (i.e. the relationship between G' and Y) varies with the environment. In one environment, Y may depend on G' in one way, whereas in another environment the relationship shifts due to the influence of the missing causal factors Cmiss. This means there is no single classifier on G" that perfectly fits P(Y|G") across both and e' some environment-specifie discrepancy in prediction is unavoidable.
Even if the marginal distributions of G' happen to be similar across environments (for instance, if P.(G.) itself is invariant or if the environment does not directly alter the observed part G'), the fact that Y|G' differs implies a significant distribution shift in the joint distribution of features and labels. To see this, consider the joint divergence (e.g. total variation or KL.) between Pa Y) and Pe.(G" Y). We can decompose it as differences in the conditional label distributions: if there exists any z' in the support of G" for which PY = z') & (Y = z²), the joint distributions will differ. In quantitative terms, one can lower,bound, for example, the total variation distance by the average conditional difference:
TV(P,(G',Y) (e 2
(15)
TV(P, Q) is the total-variation distance between two probability measures P and Q. By Lemmal such a difference is nonzero for G' that excludes part of G. (there is at least some z' for which the label distributions diverge across environments). Therefore, the joint distribution shift is positive. In contrast, for Z Ge. we have P& (Y (Y exactly (labeling function invariant), so no such difference occurs and the joint distributions Pe (Ge, Y) and (e Y) align on the conditional label component (any remaining shift comes only from P(G.) differences, which are small by Assumption 2)
From domain adaptation viewpoint, the omitted causal features lead to an intrinsic labeling mismatch across domains. In the bound of Lemma this manifests as a nonzero X term for Z = G'. In fact, X in inequality 3 represents the minimum combined error on both environments; if no single classifier can simultancously achieve low error on both e and e' because the label mappings differ, then X is bounded away from O. This contributes an effective increase in distribution shift beyond what the feature divergence alone (dнAн) captures. Mcanwhile, for Z = Go, Lemma guarantees the labeling function is identical in and e' (so X = 0), and we are left only with the feature divergence term. Thus, even if d(P'(G)), (G"), (G")) were as low as (Ge), (e.))) on the surface, the true shift relevant to classification is larger for G' due to the distribuutnn change. In summary, excluding part of Ge makes the cross-erve ensiromnito difference strictly worse in terms of maintaining a stable predictor.
Case 3: G' both includes G, and misses part of Ge. In this scenario G' contains some spurious components and is also missing some causal components. By the arguments above, such G' will suffer from both a larger marginal distribution shift (due to the spurious parts varying across e, e') and a label conditional shift (due incomplete causal information), each of which increases the divergence between Pe (G') and Pe (G'). Therefore, this case trivially yields A(G") > A(G.) as well.
Combining the cases, we conclude that any alternative subgraph G' that is not the full causal subgraph incurs a strictly greater distribution discrepancy between environments than Ge does. The causal subgraph G. uniquely achieves the minimal cross-ent enviromeit divergence by exactly capturing the invariant factors and nothing extra.
Moreover, by focusing on Ge, the learning algorithm sees an input distribution that as invariant as possible across environments (Assumption 2 ensures minimal shift in Ge), and the label-generating
25

---

mechanism is completely stable (Assumption I ensures Y depends only on Ge in all environments). Consequently, both the feature distribution shift and the label conditional shift are minimized. Any deviation from Ge either introduces additional feature shift (by including G2) or label shift (by losing part of G.), hence increasing the overall divergence, In formal terms, for any divergence measure d. d(P. Pa (G.)) < d( (C"), Per (G") for all G' * G.. This proves the claim A(G,) < A(G").
Finally, note that by Lemma 3 using G. also ensures the support of the target distribution is covered by the source distribution (no out-of-support surprise in new environments), which means the classifier can confidently generalize without encountering completely novel feature combinations. In contrast, a subgraph G' containing G, might lead to out-of-suppor samples in new environment (since G, can take unprecedented values), which is another manifestation of increased distribution shift and would break the classifier as for Lemma] 3
In conclusion, extracting the true causal subgraph Ge yields the most invariant representation across environments, while minimizing reasonable measurements of distribution shift. Any other choice G' either violates the invariant label relationship or introduces extra environment-dependen variation, thereby increasing the trossennvionmmett divergence. This completes the proof that Ge uniquely minimizes distributional disparity across environments of Theorem
C.5 Proof of Theorem 2
Proof. Under Assumption 3 we require that for robust performance, the test inputs should not be completely novel relative to training. We argue that focusing on G. satisfies this requirement across environments, whereas including G, may violate it. Because G. is tightly related to Y, all environments that share the same task (same Y definition) are likely to exhibit G2 patterns that are necessary produce Y. Even if the marginal distribution (G.) shifts bit (e.g.. some G. patterns become more or less frequent), the set of possible G. values remains linked to the support of Y. Unless the new environment introduces an entirely new causal factor (which would effectively change the task definition and violate Assumption []. Ge in the new environment should fall within the realm of possibilities seen in training (perhaps with different probabilities). For example, if Ge is a subgraph motif that causally triggers a certain label, any environment where that label can occur will contain that motif in those instances; it would not spontaneously create a completely different unseen motif to cause the same label, since Y still comes from fG2)). This intuitive argument is backed by the idea that the causal mechanism f()) is invariant one cannot get new output Y without the appropriate G, input, so new environments cannot generate different valid Ge's for the same Y (they could only omit some or add irrelevant decoration via G.). Therefore, we expect Supp (Pa Par (G.)) C Supp(Pmmin (G.)) (or at least a strong overlap), for any environment e' that does not fundamentally alter the nature of the task. This fulfills the support overlap condition of Lemma 3 for Z Ge. By that lemma, the classifier h-)-) (which without loss of generality take as the optimal invariant predictor f() or an approximation thereof) will perform equally well in environment e' as it did in training. because it is operating familiar ground. The risk in e' can thus remain as low as the risk in training, i.e. performance is stable.
Conversely, if one uses a subgraph G' that includes spurious elements, the new environment might present combinations of G' that were never seen before. For instance, perhaps in training, a certain spurious pattern in G' always coincided with a certain label (making the classifier think it was a useful feature), but in a new environment that pattern might appear with a different label or in a new context. The classifier, having learned a correlation, will mispredict because this input lies outside the training support for the joint (G", Y) distribution (the model never learned the correct response to that scenario). In formal terms, Pe PG"" may put mass on regions of the G' space where Preinn (G') had nearly zero mass (for example, motif with unseen basis before). Thus (Pp (G)) & Supp(P (G"). The violation of support overlap triggers exactly the failure mode highlighted in Lemma 3) the classifier (.).) is asked to extrapolate. If ha(-) is a complex model (e.g. deep network), might still output something for those novel inputs, but there is no guarantee it aligns with the true label - in fact it often will not, as it relies on the wrong features. This leads to performance drops even arbitrarily bad predictions in the new environment.
Thus, using the causal subgraph G. ensures that the classifier is always seeing data within (or very near) the domain it was trained on (since what changes across e is mostly the frequency of G. features,
26

---

not the support itself), guaranteeing stable performance. In contrast, using non-causal subgraph means the classifier is likely eventually step out of distribution, suffering from the classic OOD generalization failure. We conclude the proof for Theorem 2
D Additional Discussion of Empirical Examples on Distribution Shift and Representation Norms
RDC-AUC vs Perturbation
ROC-UU Perturbation
Accuracy and Norm Perturtration
0.70
0.25
63.0
1.8
7.75
7.6
ACCuRTY PNU
7.50
0.60
57.5
Norm
$3.0
0.55
0.30
6.50
47.5
0.0
2.5
5.0
7.5 10.0 12.5 15.0 17.5
The ratio of perturbed edges to the tstal 150
The natio of parturbes adgen the total [%)
The rabo of pe turbieeed enges to the tocal (96)
(a) HIV- scaffold
(b) HIV size
(c) Twitter
Arturacy and 12 Norm Netturbation Ratio
Accurary and Norm Nerturiaation Ratie
30.00
27.5
7 22.5
Accurccy (90
20.0
22.5
6.0
37.5
15.0
17.5
3.0
The radoo of perturted edges the rocal
Tre of perturtbed edges to the toral 0%)
(d) CMNIST baakkr-b
(c) CMNIST color
Figure 8: Structure causal models for graph data generation.
To illustrate that the representation norm decreases with increasing input distribution shift, we plot in Figure 2 (b) how the norm varies when we perturb the model's inputs. Figure g shows additional examples on other datasets. Specifically, following the experimental design described in the paper [16] we first train high-accuracy GNN the single-environment (no shift) dataset and freeze its parameters once converged. We then perturb the input graphs to simulate distribution shifts. Concretely, model varying degrees of structural shift, we randomly insert a given proportion of edges into each input graph while simultancously removing the same number of edges (thus preserving the total counts of nodes and edges so to minimize any effects on the GNN's message- passing and aggregation). As the figures demonstrate, the GNN is highly sensitive to structural shifts: as the shift magnitude grows, the overlap between the perturbed inputs and the low- low-dimensional weight subspace diminishes, causing the representation norm to fall and the model's prediction accuracy to decline. These results show that the norm can be robust indicator of distribution-shie severity.
E Low-Rankness in Graph Neural Networks
In neural networks, low-rank usually refers to the case where a layer's weight matrix can be approxi- mated by matrix with lower rank. This property is widely used in model compression, fast inference, and generalization analysis. A common method to evaluate low-rank is singular value decomposition (SVD). If most singular values are close to zero and only few are large, the matrix is considered approximately low-rank.
We examine the low-rank of graph neural networks using two common models: Graph Convolutional Network (GCN) and Graph Isomorphism Network (GIN). Experiments are conducted on the synthetic dataset GOODMotif and the real-world dataset GOODSST2. Both models use three layers, and each GIN layer includes a two-layer MIP for feaurre transformation. As shown the Figure and the weight matrices in each convolutional layer show clear low-rank patterns for both GCN and GIN.
27

---

Singularity Distribution of lin.wllght
Singularity Value Distriuuiion een.eeight
Comn
Comy
Cone
Conn
Con 2
-0
Oegularly Vatee
10
1.5
Siruulertyy
10
20-
100
150
250
50
200
250
Comvolution Index
Convolution Index
(a) GCN/ Motif-basis
(b) GCN / SST2-length
Figure 9: Singular value decomposition (SVD) results of GCN weights. Both models trained with the empirical risk minimization (ERM) objective. Singular values are sorted descending order. Clear low-rank patterns observed across layers.
F A Toy Example of Low-Rank and Norm
Consider the weight matrix W E RR33 given by:
1
W
1
0
0
0
which has rank rank(W) Now take two input vectors of unit length in different directions:
0000
X1
X2
We compute their images under W, regardless of the bias:
0
0
W X1 =
1
X2
0
0
Consequently, their Euclidean norms satisfy
Wxilla = v 02 + 12 0 V2
|W x2|12 =0
This simple example illustrates that a low-rank weight matrix can produce substantially different output norms for inputs aligned with its row-space versus those not aligned.
G Related Work
diitr--iittiitttonn (OOD) generalization critical challenge in graph machine learning. as models trained on given data distribution often fail to perform well on unseen distributions. Invariant learn- ing, grounded in causal theory [33], is a primary approach to this problem: it seeks learn causally relevant representations that remain stable across different environments. To acquire environment information, some methods leverage dataset-provided environment labels, e.g., IRM ① while others predict environment labels via unsupervised clustering, as in MoleOOD 46), GIL 22 and OOD-GCL 211, which entails prior assumptions about the environment distribution. Approaches such DIR (44, GREA [29] and iMoLD [51] identify invariant features through structure- or feature- level disentanglement and recombination; CIGA (5), EQuAD 471. and LIRS 49 learning to separate invariant from spurious features.
LECI 9
use self-supervised
Beyond invariant learning, alternative strategies have been developed to enhance generalization.
DANN M applies domain domain-generian generalization techniques to tackle OOD issues; GSAT B2 and
GOODGAT [3]
exploit the graph information bottleneck to discover causal subgraphs: G-Splice 27
uses linear extrapolation to broaden dataset distributions; DGAT I0] leverages GAT 42 's attention
mechanism to strengthen GNN generalization; DIVE 40 makes predictions and summaries by selecting different and non-overlapping subgraphs from single input graph respectively.
28

---

H
Datasets
We adopt two widely used benchmarks for graph OOD generalization GraphOOD g and Dru- gOOD [1]] -which together cover synthetic graphs, superpixel graphs, molecular graphs, and textual graphs:
GraphOOD: systematic benchmark tailored to graph OOD problems. We draw on four dataset groups of covarite shift in GraphOOD for graph classification: (1) GOOD-Motif: a synthetic dataset with two domain types- trpes-base-eee base-graph structure and graph size. (2) GOOD-CMNIST: a multi-class, cemi-synthetic dataset obtained by converting Colored MNIST M into superpixel graphs, with different digit-color as domains. (3) GOOD-HIV: a real-world binary classification task predicting whether molecule inhibits HIV replication, with scaffold and size as domains. (4) GOOD-SST2 and GOOD-Twitter sentiment-analy tasks (binary and ternary. respectively) derived by encoding sentences as syntax trees, using sequence length the domain.
DrugOOD, an molecule OOD benchmark for drug discovery, defines three domain splits splits-assay. scaffold, and size-applied two binding-affinity measurements (IC50 and EC50). This yields six binary-classification datasets, each predicting drug-target binding affinity.
As in prior work, partition each dataset by its domain attribute to induce distribution shifts. For example, in the Motif basis-shift setting, the motif types in the test set are entirely disjoint from those in the training and validation sets, thus rigorously assessing model generalization.
We use the ROC-AUC metric for the binary classification dataset and Accuracy for the others. More details on the datasets can be found in the original papers [ [5].
I Baselines Details
We adopt the following methods as baselines for comparison:
General methods:
ERM minimizes the empirical loss on the training set.
IRR ( seeks to find data representations across all environments by penalizing feature distributions that have different optimal classifiers.
Coral (39 encourages feature distributions consistent by penalizing differences in the means and covariances of feature distributions for each domain.
VREx 19 reduces the risk variances of training environments to achieve both covariate robustness and invariant prediction.
Graph-specific OOD methods: DIR 44 discovers the subset of graph as invariant rationale by conducting interventional data augmentation to create multiple distributions.
GIL employs unsupervised clustering infer environmental labels and leverages the invariant principle to identify causal subgraphs.
proposes build interpretable graph learning method through the attention mecha- nism and inject stochasticity into the attention to select label-relevant subgraphs.
GSAT 32
CIGA 5
proposes information-theoortic objective to extract the desired invariant subgraphs
from the lens of causality.
LECI 9 assume the availability of environment labels, and study environment exploitation strategies for graph OOD generalization.
iMoLD [5]] employ environment augmentation techniques to facilitate the learning of invariant graph-level representations.
EQQAD 47 adopts self-supervised learning to learn spuriosu efatures first, followed by learning
invariant features by unlearning spurious features.
29

---

LIRS [49] takes an indirect approach by first learning the spurious features and then removing them from the ERM-leamed features.
Our selected baselines encompass a diverse array of approaches for tackling graph out-of-distribution (OOD) problems, including state-of-the-art and recently proposed methods. Some approaches such as OOD-GCL [21], GOODGAT [3), G-Splice [27] DGAT [[0], et al. are omitted owing to lack of comparable performance results available implementation details. Moreover, the baselines selected already encompass the main research directions of most state-of-the-ai graph OOD methods.
J Visualized Cases
1.0
0.8
0.6
0.4
0.2
Figure 10: Visualized cases of the Motif-OOD dataset with size and basis domain shift. Nodes with dark blue and light blue colors represent the motif nodes and base graph nodes, respectively. The shading the edges indicates the importance score of each edge generated by the subgraph extractor.
We present several visualized cases with size and basis domain shift in Figure 10
These examples demonstrate that our method can effectively extract the causal subgraph (Motif) from the input graph rather than selecting spurious factors (basis graphs). This also highlights the inherent interpretability of our approach.
K Discussion on the Effect of Top Ratio Hyperparameter
Motif-busis
-
SST2
1C50-Seaftold
85
82:74
82.63
83.65
82.76
82.38
82.74
Accuracy
83
83.03
82
83.1
83.17
81.97
81.77
71,51
71.71
72.86
72.15
71.68
72.39
70.43
69.69
0.3
0.6
0.8
Figure 11: Performance three datasets with different
In our approach, the hyperparameter controls the fraction of edges designated as the causal subgraph. However, the real edge ratio of subgraph real-world datasets is typically unknown. As shown in Figure I1 we evaluate model accuracy across three datasets for different values and find that has no significant impact on generalization performance. From these results, we draw two empirical conclusions:
30

---

Avoid overly small r: If r is set too low, the selected subgraph fails to fully cover the causal structure, degrading performance.
Robustness for a moderate r: When r exceeds a minimal threshold, its exact value has little effect. We find that this robustness stems from the entropy-regulariiztton term Leomp in TO which automatically enforces an appropriate level of mask sparsity: some selected edges acquire near-zero mask weights and are thus effectively omitted during prediction.
Accordingly, we recommend r = 0.5 as a reasonable default for training.
Efficiency Study
Experiments in this paper are conducted NVIDIA RTX3090 GPUs. Our method is concise and streamlined: norm computation introduces virtually no additional overhead, and the two-stage parameter updates have negligible impact on efficiency. Moreover, the Table 6ndd training and inference times of IDG and baselines, underscoring the high efficiency of our approach.
reports
Dataset
Training Batch Size
Testing Batch Size
Training Time (s) Inference Time (s)
ERM
64
256
1032
1.3
LECI
64
256
3404
1.6
DIR
64
256
3213
1.7
IDG
64
256
1603
1.6
Table 5: Efficiency study of our method on Motif-basis.
Dataset
Training Batch Size
Testing Batch Size
Training Time (s)
Inference Time (s)
ERM
64
256
1148
0.51
LECI
64
256
3973
1.43
DIR
64
256
3472
1.62
IDG
64
256
1855
1.53
Table Efficiency study of our method on SST2.
M
Broader Impact
Our work aims to enhance the generalization graph neural networks (GNNs) in duitof-distriuution (OOD) scenarios, which is crucial for real-world applications. By focusing causal subgraph extraction, we provide a method that can potentially improve the robustness and reliability of GNNs in various domains, including drug discovery, social network analysís, and biological data interpretation. However, is important to acknowledge that our approach may not be universally applicable to all graph-based tasks or datasets. The evaluations of our approach are mainly across limited graph domains, which may not represent all possible real-world scenarios. The approach can be evaluated various environmental domains be validated in more realistic setting.
N Limitations
As with other graph generalization methods, although our approach improves the model's out- of-distribution performance to some extent, its transferability to other domains remains uncertain. Moreover, for datasets without any ground truth, leveraging the extracted subgraphs to further enhance generalization a direction that has yet to be fully explored.
31


Reviews:
[{"review_id": "1tgZvkpK7h", "reviewer": "Reviewer_rEir", "summary": "The paper proposes a method to extract causal subgraphs to address the problem of Out-of-Distribution (OOD) for graph classification. Their analysis unravels the relationship between distribution shifts and representation norms produced by GNNs: representation norms decay when distribution shifts increase. Thus, these findings inspire the paper’s methodology to extract causal subgraphs by incorporating the L2 representation norm into the loss function.", "questions": "Please refer to Weaknesses.", "limitations": "Please refer to Weaknesses.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strength**: \n* The paper’s organization is clear and easy to read.\n* Propose detailed findings and theoretical analysis to inspire the methodology.\n* Detailed report for experimental settings and reproducibility. \n\n**Weaknesses**:\n* Compared to previous works on extracting causal / invariant subgraphs (DIR [1], GIL [2]), IDG’s extractor architecture shares some similarities with these works’ extractor (node representation $\\to$ masking probabilities $\\to$ selecting top edges). Can you provide theoretical analysis or elaboration on why your proposed optimization framework (incorporating L2 norm) can help yield better performance than DIR [1]’s intervention mechanism or GIL [2]’s optimization framework?\n* Can you provide time complexity analysis for IDG?\n* Theoretical findings for IDG are backed by 3 assumptions listed in Section 2.2. I think it would make the paper more comprehensive if you include a brief explanation for why these 3 assumptions are reasonable and whether these 3 assumptions are also utilized by previous OOD works. \n\n[1] Wu et al., Discovering Invariant Rationales for Graph Neural Networks. ICLR 2022.\n\n[2] Li et al., Learning Invariant Graph Representations for Out-of-Distribution Generalization. NeurIPS 2022.", "was_revised": true, "final_justification": "All of my concerns about the method's novelty and optimization details compared to previous Graph OOD works along with the method's time complexity details are fully addressed, so I decided to keep my rating score.", "rebuttal": "Thank you for your appreciation on our work!\n\n***Weakness***\n\n- **\"Optimization framework comparing to DIR/GIL\"** The extractor architecture employed here, which maps node features to edge masks, is widely used in Graph OOD research. Besides GIL and DIR, it is also adopted by methods such as LECI[1], GALA[2], CIGA[3], DiSC[4], and GSAT[5]. Its origins trace back to earlier graph explanation methods like PGExplainer[6], as well as similar techniques in the text[7] and vision domains[8]. We would like to point out that the key contribution of this work is a **new theoretical framework** that, without relying on invariant learning or any environmental labels, guarantees the extractor identifies causal subgraphs rather than the extracting procedure.  The main challenge of this framework lies in finding an effective optimization approach that jointly trains the extractor and classifier to improve their coordination on extracting causal features and predicting. Compared to DIR and GIL, our method offers the following key distinctions and advantages:\n\n  - 1, Both DIR and GIL build on invariant risk minimization (IRM) [9] by attempting to find environment-invariant predictors, which requires inferring each sample's environment label. DIR induces different distributions via explicit intervention perturbations, while GIL derives environment labels through unsupervised clustering of k-means. Neither approach has theoretical guarantees that the constructed environments match the real data-generating distributions, which is also noted in related work such as GALA[2] and CIGA[3]. Our experiments below further reveal two points:\n\n    - Removing the invariance (variance) objective from DIR and GIL (denoted DIR-0 and GIL-0) can actually improve performance in some settings.\n    - In certain scenarios, both DIR and GIL perform even worse than standard ERM.\n\n    |Dataset|PIIF|FIIF|SP=0.6|SP=0.7|SP=0.8|SP=0.9|\n    |-|-|-|-|-|-|-|\n    |ERM|62.45|37.22|91.67|91.25|90.63|85.92|\n    |IRM|68.34|44.33|91.85|91.25|90.48|85.38|\n    |GIL|59.66|64.66|89.78|80.33|72|51.09|\n    |GIL-0|53.67|37.49|90.78|78.45|79|83.46|\n    |DIR|69.33|42|88.7|87.82|86|84.8|\n    |DIR-0|53.67|36|87.67|89.79|87.8|86.74|\n    |**IDG**|**79**|**83.67**|**92.67**|**92.06**|**91.89**|**91.33**|\n\n    These datasets adhere to the GALA[2], CIGA[3], and LECI[1] settings. PIIF and FIIF are two typical distribution-shift scenarios, and SP denotes the spurious-correlation probability. **These results indicate that the optimization strategies of DIR and GIL are not universally effective.** In contrast, our method is completely **IRM-free and requires no environment information**: it identifies causal subgraphs simply by finding solutions that satisfy the minimal shift criterion via norm optimization.\n\n  - 2, From an optimization perspective, DIR also demands a large memory bank for non-causal features $S$, which hinders its scalability to large datasets, and GIL must collect representations across the entire batch for clustering, both of which incur substantial resource overhead. By comparison, our approach only requires two forward passes and minimal additional computation of computing the norm, demonstrating huge efficiency advantages.\n\n  - We will include a detailed comparison against DIR, GIL, and other related methods in the Related Work and Discussion sections in the paper.\n\n- **\"Time complexity analysis\"** The time complexity of the IDG method is $O(m d + n d^2)$, where $n$, $m$, and $d$ denote the average number of nodes, edges, and feature dimensions per graph. Specifically:\n\n  - Message passing and node updates (per GNN layer) incur a cost of $O(m d + n d^2)$.\n  - Edge scoring and top-r selection can be completed in $O(m \\log m)$ in the worst case.\n  - Norm regularization and compactness term require $O(m)$ time.\n\n  Thus, a single forward–backward pass on one graph remains $O(m d + n d^2)$. In our method, the norm calculation incurs minimal computational overhead (less than that of edge selection). To further demonstrate its efficiency, we measured both training time (per epoch) and inference time on the Motif, HIV, and SST2 datasets. The results are presented in the table below:\n  ||Motif-basis||HIV-scaffold||SST2-length||\n  |-|-|-|-|-|-|-|\n  |Time(ms)|Training(ms)|Inference(ms)|Training(ms)|Inference(ms)|Training(ms)|Inference(ms)|\n  |GIL|56801|2037|102594|22057|86658|16472|\n  |DIR|14415|794|38477|1098|32392|9377|\n  |**IDG**|**11262**|**493**|**35232**|**995**|**26326**|**7659**|\n\n- **\"Explanation of 3 assumption\"** We discuss the rationale of these three core assumptions here:\n\n  - *Assumption 1 (Causal Mechanism)*\nThe generation of causal labels is fully determined by the causal subgraph $G_c$. This assumption derives from the Independent Causal Mechanism (ICM) hypothesis [9] and underpins Graph OOD methods such as CIGA[3] and LECI[1]. Invariant Risk Minimization (IRM) likewise relies on it by seeking representations that yield the same optimal classifier across environments. For example, if the causal subgraph (e.g., functional groups responsible for key chemical properties) is known, one can infer a molecule's chemical properties regardless of its distribution of origin (environments).\n\n  - *Assumption 2 (Environmental Diversity and Interventions)*\nThis assumption originates from Peters et al.'s ICP framework and its extensions [10–13], this assumption holds that distributional shifts across environments arise from sparse interventions on the non-causal subgraph $G_s$, while $G_c$ remains stable. This hypothesis has been adopted by Graph OOD methods such as LECI, which impose an even stronger assumption—that the environment $E$ is independent of $G_c$. i.e. $E \\perp G_c$. For instance, the types and structures of functional groups in a molecule are limited and stable, whereas non-causal parts (e.g., heteroatoms) vary widely across environments.\n\n  - *Assumption 3 (Effective Classifier Support)*\nThis assumption is based on the classical domain-adaptation bounds presented first in [14], this assumption requires overlap between the support of source and target distributions to guarantee generalization. Since classification relies on $G_c$, the support of the causal-subgraph distribution in the test domain must lie within that of the training domain. For example, a classifier can correctly and consistently label a functional group in the test environment only if that group appeared during training.\n\n  - To further support the validity of Assumption 2 in real applications, we conducted two experiments on real world datasets comparing causal and non-causal subgraphs. We devised two distinct experimental setups to validate our assumption under different settings. Distribution shift was measured by the maximum mean discrepancy (MMD) between representations produced by the a same GNN model, using a Gaussian kernel on standardized features with bandwidth set by the median heuristic. We denote the cross‐environment distribution shift of causal and non‐causal subgraph by $\\Delta{G_c}\\approx MMD(G_c^{e1},G_c^{e2}), \\Delta{G_s}\\approx MMD(G_s^{e1},G_s^{e2})$ and compute the ratio of $\\Delta{G_s}/\\Delta{G_c}$. The experimental designs and results are described below:\n\n    - Experiment 1. Four real-world molecular graph datasets with ground-truth causal subgraphs—Mutag, Benzene, Fluoride, and Alkane—were selected from the graph explanation literature. Each dataset was split into two environments by molecular size and by random assignment. The results are presented below:\n     \n    |$\\Delta{G_s}/\\Delta{G_c}$|Mutag||Benzene||Fluoride||Alkane||\n    |-|-|-|-|-|-|-|-|-|\n    |split|size|random|size|random|size|random|size|random|\n    |MMD|3.43|1.56|5.79|1.83|6.67|1.94|6.48|2.03|\n\n    The results indicate that, the non-causal subgraph exhibits a larger shift than the causal subgraph across all environment partitions($\\Delta{G_s}>\\Delta{G_c}$), which aligns with Assumption 2.\n\n    - Experiment 2. We apply the same evaluation to the out-of-distribution (OOD) dataset used in our study. Due to the absence of true ground truth, we treat the subgraphs extracted by our method, GIL, and GSAT as causal subgraphs. The results are presented below:\n\n    |$\\Delta{G_s}/\\Delta{G_c}$| HIV-scaffold | HIV-size | SST2  | Twitter | IC50-size| IC50-scaffold |\n    |-|-|-|-|-|-|-|\n    | IDG| 7.28| 3.915|1.963| 3.983| 1.1784 | 1.5296 |\n    | GIL | 1.969 |1.429| 1.044 | 3.25| 1.154| 1.74|\n    | GSAT  | 5.292| 4.671| 1.868 | 1.244 | 1.128| 1.156 |\n\n    The results indicate that, the extracted non-causal subgraph by different methods still exhibits a larger shift than the causal subgraph($\\Delta{G_s}>\\Delta{G_c}$), which also aligns with Assumption 2.\n  \n  These two experiments, viewed from different perspectives, clarify the validity of Assumption 2. We will include these explanations and experiments in Section 2.2 (Theoretical Insights) of the paper.\n\n  We will include the explanations of 3 assumptions in Section 2.2 in the paper.\n\n[1] Joint Learning of Label and Environment Causal Independence for Graph Out-of-Distribution Generalization\n\n[2] Does Invariant Graph Learning via Environment Augmentation Learn Invariance?\n\n[3] Learning Causally Invariant Representations for Out-of-Distribution Generalization on Graphs\n\n[4] Debiasing graph neural networks via learning disentangled causal substructure\n\n[5] Interpretable and Generalizable Graph Learning via  Stochastic Attention Mechanism\n\n[6] Parameterized Explainer for Graph Neural Network\n\n[7] Rationalizing Neural Predictions\n\n[8] Interpreting Image Classifiers by Generating  Discrete Masks\n\n[9] Elements of Causal Inference: Foundations and Learning Algorithms\n\n[10] Causal inference using invariant prediction: identification and confidence intervals\n\n[11] Invariant Causal Prediction for Nonlinear Models\n\n[12] Invariant Causal Prediction for Sequential Data\n\n[13] Invariance, Causality and Robustness\n\n[14] A theory of learning from different domains"}, {"review_id": "i4q2DilwX6", "reviewer": "Reviewer_q4r2", "summary": "This paper aims to address out-of-distribution generalization in graph neural networks. The main contribution of the paper is the development of an IRM-free method, i.e., the invariant distribution criterion, for capturing causal subgraphs, which are shown to exhibit smaller distributional variations across different environments compared to non-causal components. Specifically, they propose a norm-guided invariant distribution objective to efficiently discover and predict causal subgraphs, demonstrating superior performance over state-of-the-art methods in extensive experiments.", "questions": "Please refer to my summary of weakness.", "limitations": "Yes.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "This paper has the following advantage that I appreciate:\n\n+ The paper introduces a novel method for graph generalization that does not rely on the traditional IRM framework, making it more practical and cost-effective.\n\n+ Extensive experiments on multiple benchmarks seem to show that the proposed method consistently outperforms existing state-of-the-art methods.\n\nHowever, the paper also have the following weakness:\n\n- My main question lies in the practicality of the assumption. The core theory presumes distribution shifts affect only non-causal parts more than causal ones (Assumption 2). In real applications, causal mechanisms can drift too, potentially breaking guarantees.\n\n- Lack in-depth discussion of the limitations. I'm curious that since representation norm can be influenced by multiple other effects, how confidence are we to attribute the norm difference to the distributional shift due to spurious correlations?", "was_revised": true, "final_justification": "I have carefully read the rebuttal provided by the authors and confirm that they have addressed my concerns. Therefore I maintain my positive score.", "rebuttal": "Thank you for your appreciation on our work!\n\n***Weakness***\n\n- **\"Assumption 2, distribution shifts affect only non-causal parts\"** This assumption originates from Peters et al.'s ICP framework and its extensions [1–4], this assumption holds that distributional shifts **across environments** arise from sparse interventions on the non-causal subgraph $G_s$, while $G_c$ remains stable. This hypothesis has been adopted by Graph OOD methods such as LECI[5], which impose an even stronger assumption—that the environment $E$ is independent of $G_c$. i.e. $E \\perp G_c$. For instance, the types and structures of functional groups in a molecule are limited and stable, whereas non-causal parts (e.g., heteroatoms) vary widely across environments.\n\n  - To further support the validity of Assumption 2 in real applications, we conducted two experiments on real world datasets comparing causal and non-causal subgraphs. Quantifying distribution shifts in real-world graphs is challenging, since such data with ground-truth annotation of causal subgraph are rarely available, so we devised two distinct experimental setups to validate our assumption under different settings. Distribution shift was measured by the maximum mean discrepancy (MMD) between representations produced by the a same GNN model, using a Gaussian kernel on standardized features with bandwidth set by the median heuristic. We denote the cross‐environment distribution shift of causal and non‐causal subgraph by $\\Delta{G_c}\\approx MMD(G_c^{e1},G_c^{e2}), \\Delta{G_s}\\approx MMD(G_s^{e1},G_s^{e2})$ and compute the ratio of $\\Delta{G_s}/\\Delta{G_c}$. The experimental designs and results are described below:\n\n    - Experiment 1. Four real-world molecular graph datasets with ground-truth causal subgraphs—Mutag, Benzene, Fluoride, and Alkane—were selected from the graph explanation literature. Each dataset was split into two environments by molecular size and by random assignment. The results are presented below:\n     \n    |$\\Delta{G_s}/\\Delta{G_c}$|Mutag||Benzene||Fluoride||Alkane||\n    |-|-|-|-|-|-|-|-|-|\n    |split|size|random|size|random|size|random|size|random|\n    |MMD|3.43|1.56|5.79|1.83|6.67|1.94|6.48|2.03|\n\n    The results indicate that, the non-causal subgraph exhibits a larger shift than the causal subgraph across all environment partitions($\\Delta{G_s}>\\Delta{G_c}$), which aligns with Assumption 2.\n\n    - Experiment 2. We apply the same evaluation to the out-of-distribution (OOD) dataset used in our study. Due to the absence of true ground truth, we treat the subgraphs extracted by our method, GIL, and GSAT as causal subgraphs. The results are presented below:\n\n    |$\\Delta{G_s}/\\Delta{G_c}$| HIV-scaffold | HIV-size | SST2  | Twitter | IC50-size| IC50-scaffold |\n    |-|-|-|-|-|-|-|\n    | IDG| 7.28| 3.915|1.963| 3.983| 1.1784 | 1.5296 |\n    | GIL | 1.969 |1.429| 1.044 | 3.25| 1.154| 1.74|\n    | GSAT  | 5.292| 4.671| 1.868 | 1.244 | 1.128| 1.156 |\n\n    The results indicate that, the extracted non-causal subgraph by different methods still exhibits a larger shift than the causal subgraph($\\Delta{G_s}>\\Delta{G_c}$), which also aligns with Assumption 2.\n\n- **\"Norm difference to  the distributional shift due to spurious correlations\"** We performe a detailed comparison between IDG and a variant without the norm objective (denoted IDG-0) across varying levels of spurious correlation strength to assess how much the norm objective contributes to mitigating spurious correlations. As shown in the table below, the norm objective's impact on performance grows with increasing spurious correlation strength, with a *Pearson correlation coefficient* of 0.86 between the spurious correlation probability and the performance gain. This finding demonstrates the effectiveness of our method in addressing spurious correlations.\n\n  |       | SP=0.5 | SP=0.6 | SP=0.7 | SP=0.8 | SP=0.9 | SP=0.95|\n  |-------|--------|--------|--------|--------|--------|--------|\n  | IDG   | 93.33  | 92.67  | 92.06  | 91.89  | 91.33  | 83.26  |\n  | IDG-0 | 90.66  | 90.43  | 90.16  | 86.74  | 83.13  | 70.72  |\n  |$\\Delta$| 2.67  | 2.24   | 1.90   | 5.15   | 8.20   | 12.54  |\n\n  SP denotes the spurious-correlation probability. We will incorporate these results and their analysis into the comparative experiments in Section 4.2.\n\n[1] Causal inference using invariant prediction: identification and confidence intervals\n\n[2] Invariant Causal Prediction for Nonlinear Models\n\n[3] Invariant Causal Prediction for Sequential Data\n\n[4] Invariance, Causality and Robustness\n\n[5] Joint Learning of Label and Environment Causal Independence for Graph Out-of-Distribution Generalization"}, {"review_id": "bW5F099ak5", "reviewer": "Reviewer_9Fyu", "summary": "This paper proposes an IRM-free approach for out-of-distribution (OOD) generalization in graph neural networks. The core idea is that causal subgraphs tend to exhibit smaller distributional shifts across environments compared to non-causal parts. The authors formalize this observation as the Invariant Distribution Criterion and provide supporting theoretical analysis. They further establish a quantitative relationship between distribution shift and representation norm, using it to design a norm-guided objective for identifying causal subgraphs without requiring explicit environment labels. Experimental results on multiple OOD graph benchmarks show that the proposed method outperforms various strong baselines.", "questions": "How sensitive is the method to cases where the causal subgraph does not have a significantly larger norm than spurious parts, or when distributional shift affects the causal structure itself? Has the method been tested in partial information or weakly informative environments where spurious correlations are subtle rather than dominant?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "The paper provides a formal theoretical framework that links causal subgraph stability with distribution invariance. It introduces a simple and interpretable mechanism—representation norm—as a proxy for detecting distributional shifts. The method avoids explicit environment annotations, which enhances its practical applicability. The experimental evaluation is conducted on diverse datasets and includes ablation studies and comparisons with many competitive methods.\n\nHowever, the key insight that causal subgraphs are more invariant is derived from intuition and synthetic examples, and its generalizability to real-world datasets is not rigorously validated. The theoretical results depend on strong assumptions that may not hold in practice, and the paper lacks a detailed discussion of these limitations. The idea of leveraging norm as a proxy for distribution shift has appeared in prior work in other domains, and the contribution here appears incremental in the absence of deeper justification for why this mechanism specifically works for graph data. Related work on invariant learning and graph causal reasoning is not thoroughly contrasted.", "was_revised": true, "final_justification": "Most of my concerns are addressed only except for related work discussions.", "rebuttal": "Thank you for the feedback and please find below responses to your comments.\n\n***Weakness***\n\n- **\"Generalizability to real-world\"** Because real-world datasets lack ground-truth annotations for causal subgraphs, validating this hypothesis on them is extremely challenging. Nevertheless, we devise two distinct experimental setups to validate our assumption under different settings:\n\n  - To validate that causal subgraphs are more invariant, we ran two experiments on real-world graphs, comparing causal and non-causal subgraphs under different settings. Given the scarcity of ground-truth causal annotations, we design distinct experimental setups and quantify distribution shift via maximum mean discrepancy (MMD) between GNN representations using a Gaussian kernel on standardized features with bandwidth set by the median heuristic. We denote the cross‐environment distribution shift of causal and non‐causal subgraph by $\\Delta{G_c}\\approx MMD(G_c^{e1},G_c^{e2}), \\Delta{G_s}\\approx MMD(G_s^{e1},G_s^{e2})$ and compute the ratio of $\\Delta{G_s}/\\Delta{G_c}$. Table values above 1 indicate that non‐causal subgraphs undergo larger distribution shifts across environments. The experimental designs and results are described below:\n\n    - Experiment 1. Four real-world molecular graph datasets with ground-truth causal subgraphs—Mutag, Benzene, Fluoride, and Alkane- from the graph explanation literature. Each dataset was split into two environments by molecular size or random. The results are below:\n     \n    |$\\Delta{G_s}/\\Delta{G_c}$|Mutag||Benzene||Fluoride||Alkane||\n    |-|-|-|-|-|-|-|-|-|\n    |split|size|random|size|random|size|random|size|random|\n    |MMD|3.43|1.56|5.79|1.83|6.67|1.94|6.48|2.03|\n\n    The results indicate that, the non-causal subgraph exhibits a larger shift than the causal subgraph across all environment partitions($\\Delta{G_s}>\\Delta{G_c}$), which aligns with Assumption 2.\n\n    - Experiment 2. We apply the same evaluation to the out-of-distribution (OOD) dataset used in our study. Due to the absence of true ground truth, we treat the subgraphs extracted by our method, GIL, and GSAT as causal subgraphs. The results are below:\n\n    |$\\Delta{G_s}/\\Delta{G_c}$|HIV-scaffold|HIV-size|SST2|Twitter|IC50-size|IC50-scaffold|\n    |-|-|-|-|-|-|-|\n    |IDG|7.28|3.915|1.963|3.983|1.1784|1.5296|\n    |GIL|1.969|1.429|1.044|3.25|1.154|1.74|\n    |GSAT|5.292|4.671|1.868|1.244|1.128|1.156|\n\n    The results indicate that, the extracted non-causal subgraph by different methods still exhibits a larger shift than the causal, which also aligns with Assumption 2.\n\n  We will include these explanations and experiments in Section 2.2 (Theoretical Insights) of the paper.\n\n- **\"Limitations about the assumption\"** We discuss the rationale, limitations, and extreme violation cases here:\n\n  - *Assumption 1 (Causal Mechanism)* The generation of causal labels is fully determined by the causal subgraph $G_c$. This assumption derives from the Independent Causal Mechanism (ICM) hypothesis [6] and underpins Graph OOD methods such as CIGA[7] and LECI[5]. Invariant Risk Minimization (IRM) likewise relies on it by seeking representations that yield the same optimal classifier across environments. For instance, knowing the causal subgraph (e.g., functional groups) enables inferring a molecule's properties in different environments.\n\n  - *Assumption 2 (Environmental Diversity and Interventions)* This assumption originates from Peters et al.'s ICP framework and its extensions [1-4], this assumption holds that distributional shifts across environments arise from sparse interventions on the non-causal subgraph $G_s$, while $G_c$ remains stable. This hypothesis has been adopted by Graph OOD methods such as LECI[5], which impose an even stronger assumption—that the environment $E$ is independent of $G_c$. i.e. $E \\perp G_c$. For instance, functional groups remain consistent in type and structure, while non-causal elements (e.g., heteroatoms) vary across environments.\n\n  - *Assumption 3 (Effective Classifier Support)* This assumption is based on the classical domain-adaptation bounds presented first in [8], this assumption requires overlap between the support of source and target distributions to guarantee generalization. Since classification relies on $G_c$, the support of the causal-subgraph distribution in the test domain must lie within that of the training domain. For example, a classifier can only correctly label a functional group at test time seen during training.\n\n  Limitations and Extreme Violations:\n  - *Assumption 1:* If causal labels depend on factors beyond G_c, our model may miss key causal relationships, harming classification. For example, if labels reflect post-reaction metrics rather than intrinsic chemical properties. **This scenario is at odds with existing work[1-9].** Like other Graph OOD and causal inference studies, our study focus solely on input–output mapping.\n  - *Assumption 2:* If shifts involve global feature changes rather than sparse interventions on $G_s$, this assumption fails—for instance, if environments are defined by critical functional groups, but **these cases are extremely rare** and we cannot find supported datasets and evidence from current Graph OOD benchmarks.\n  - *Assumption 3:* If the support of $G_c$ in the test domain does not overlap with that of the training domain—e.g., when test samples include unseen functional groups—no existing method can classify effectively. **This scenario also lies outside the scope of Graph OOD tasks in general.**\n\n  We will include the discussion about explanations and limitations of 3 assumptions in Section 2.2 in the paper.\n\n- **\"Incremental contribution of leveraging norm\"** The work [10] discussed in our paper, to the best of our knowledge, is the only work we have found to be related to us. However, compared to [10], our work differs in three key aspects:\n\n  - *Target application.* While [10] examines whether inputs are from different distributions, our method focuses on training models to correctly classify inputs from different distribution.\n  - *Evaluation domain.* Work [10] is conducted entirely within the vision domain, whereas our study is dedicated to graph data and graph neural networks.\n  - *Theoretical foundation.* The conclusions in [10] are based on analyses of linear networks, while our framework is grounded in causal inference, domain generalization, and graph learning.\n\n  Moreover, our contributions are threefold:\n    - We are the first to empirically establish a quantitative relationship between distribution shift and representation norms in graph data, and to explain its connection to the low-rank property of GNNs. (As recognized by Reviewer Ti4B)\n    - We provide a rigorous theoretical proof that causal subgraph minimizes distribution shift across environments. (As recognized by Reviewer Ti4B, rEir)\n    - We are the first to introduce norms into graph OOD research and to offer a theoretical justification for using norms to identify causal subgraphs. (As recognized by Reviewer q4r2, Ti4B)\n\n- **\"Related work on invariant learning and causal reasoning\"** Due to space constraints, details of existing invariant-learning and causal-reasoning baselines are provided in Appendices G and I. Please refer to our response to reviewer rEir *(Weak1: Optimization framework)* and Ti4B *(Weak2: Compared to environment-inference-based methods)*. and we plan to add a more in-depth discussion of how our method differs from these approaches in the Related Work section in the paper.\n\n***Questions***\n\n- **\"Cases where the causal subgraph does not have a larger norm/distributional shift affects the causal\"** These settings contradict the assumptions and scenarios of both our work and prior Graph OOD studies like in [5,9,7,11]. In our experiments, we did not encounter any cases in which the causal subgraph fails to exhibit a larger norm than the spurious components, nor cases in which distributional shift affected the causal structure. If possible, could you suggest a dataset or benchmark that exhibits these conditions? We would then use the recommended benchmark to present further experimental results.\n\n- **Testing in partial information or weakly informative environments** We conduct experiments in these datasets. The results below demonstrate that our method performs strongly across all scenarios (denoting the variant without the norm objective as IDG-0):\n\n  ||SP=0.1|SP=0.3|SP=0.6|SP=0.7|SP=0.8|SP=0.9|PIIF|FIIF|\n  |-|-|-|-|-|-|-|-|-|\n  |ERM|94.53|93.26|91.67|91.25|90.63|85.92|62.45|37.22|\n  |IRM|93.65|92.87|91.85|91.25|90.48|85.38|68.34|44.33|\n  |GIL|92.14|91.57|89.78|80.33|72.00|51.09|59.66|64.66|\n  |DIR|90.63|90.10|88.70|87.82|86.00|84.80|69.33|42.00|\n  |**IDG**|**95.30**|**94.67**|**92.67**|**92.06**|**91.89**|**91.33**|**79.00**|**83.67**|\n  |IDG-0|94.89|94.29|90.43|90.16|86.74|83.13|66.33|70.66|\n\n  SP denotes the probability of spurious correlation, with lower values indicating weakly informative environments. PIIF and FIIF refer to partially and fully informative scenarios. These datasets adhere to the settings in [5,7,9]. We will incorporate these results and their analysis into the comparative experiments in Section 4.2.\n\n[1]Causal inference using invariant prediction: identification and confidence intervals\n\n[2]Invariant Causal Prediction for Nonlinear Models\n\n[3]Invariant Causal Prediction for Sequential Data\n\n[4]Invariance, Causality and Robustness\n\n[5]Joint Learning of Label and Environment Causal Independence for Graph Out-of-Distribution Generalization\n\n[6]Elements of Causal Inference: Foundations and Learning Algorithms\n\n[7]Learning Causally Invariant Representations for Out-of-Distribution Generalization on Graphs\n\n[8]A theory of learning from different domains\n\n[9]Does Invariant Graph Learning via Environment Augmentation Learn Invariance?\n\n[10] Deep Neural Networks Tend To Extrapolate Predictably\n\n[11] GOOD: A Graph Out-of-Distribution Benchmark"}, {"review_id": "I3CaTS2ODL", "reviewer": "Reviewer_Ti4B", "summary": "This paper proposes a norm-guided invariant objective function for **quantifying distribution shift** in graph learning. The key insight is that representation norm correlates with distributional invariance of subgraphs, and can be used as a proxy signal to guide causal subgraph extraction, without relying on explicit environment labels. The method is theoretically motivated and empirically validated, and the proposed IDG framework is shown to perform well on multiple OOD benchmarks.", "questions": "1. Beyond Edge Perturbation: Can You Handle Real-World Shifts?  In the experiments (Section 2.3.1), the authors simulate OOD shift using edge perturbation. But in real-world datasets like **DrugOOD**, shifts come from **many sources,** like assay types, scaffold structures, dataset size, or even **activity cliffs**, where similar molecules can have very different labels. So I wonder: Does the norm-based quantification also work for these more complex or subtle shifts? Or is it mainly effective for simple structural changes?\n    \n2. About Continuous IRM. The quantitative method reminds me of **continuous IRM** methods that avoid hard environment partitions, like CORAL or DIR, and we can simply apply the soft assignment from multi-distribution (Dirichlet) to learn the continuous environments.     Could you kindly explain how your norm-based approach is similar or different from those in terms of objective and learning behavior?\n    \n3. Could Environment Modeling Help the Method? Even though your method is IRM-free, I don't think it conflicts with using environment info. For example, what would happen if we added known environment labels to your setup? Would the norm objective still behave the same, or would it change depending on how environments are defined? Maybe this could help reveal how sensitive your method is to the granularity of domain shifts.", "limitations": "N/A", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strength:\n\n1. The idea of quantifying OOD shift through representation norm is highly promising. This research direction has important implications for understanding model robustness and moving beyond heuristic environment partitioning in complicated out-of-distribution scenarios, e.g., drug discovery.\n2. The paper connects the norm of representations with distributional shift, both theoretically (via Lemmas and Theorems) and empirically. This dual support makes the argument more convincing.\n3. The writing is clear and easy to follow, and the paper is well-organized.\n\nWeakness:\n1. I had a hard time fully understanding Section 2.3.2. Could the authors please provide a simple example or an intuitive explanation of this part? Especially how the norm relates to empirical risk across domains.\n    \n2. The framework is IRM-free and avoids environment labels, which I appreciate. But the two-step training schedule (first freeze extractor, then predictor, and vice versa) looks similar to methods like environment-inference-based invariant learning methods (EIIL, GOODHSE, and MoleOOD, etc), which also extract latent structure before enforcing invariance.  What is the essential difference between alternating optimization and those used in environment inference-based methods?\n\n\nExtended Questions (Personal Curiosity)\nI’m very interested in this direction, so I also have some follow-up thoughts and questions in the Question part.", "was_revised": true, "final_justification": "Thank the authors for their hard work on the rebuttal. Most of the points are addressed, and I will maintain my score.", "rebuttal": "Thank you for your appreciation on our work!\n\n***Weakness***\n\n- **\"Intuitive explanation of Section 2.3.2\"** To better illustrate the connection between cross-environment/domain risk and the norm, we present the examples below:\n\n  1. Consider the weight matrix $W\\in\\mathbb{R}^{4\\times3}$ given by:\n  $$W =\n  \\begin{pmatrix}\n  (1 & 0 & 0),\n  (0 & 1 & 0),\n  (1 & 0 & 0),\n  (0 & 1 & 0)\n  \\end{pmatrix}$$\n  which has rank $\\mathrm{rank}(W)=2$.\n\n  1. We now pick two 'environmental' inputs $[C1,S1]$ from environment 1 and $[C2,S2]$ from environment 2, each decomposed into a causal part C and a spurious part S. From Theorem 1 that causal subgraph minimizes distribution shift across environments, we suppose $\\mathrm{sim}(S_1,S_2)>\\mathrm{sim}(C_1,C_2)$. Then we define:\n      - Env 1:\n      $$C_1 = \\begin{pmatrix}1,0,0\\end{pmatrix},\\quad\n      S_1 = \\begin{pmatrix}0,1,0\\end{pmatrix}.$$\n      - Env 2:\n      $$C_2 = \\begin{pmatrix}0.9,0.1,0\\end{pmatrix},\\quad\n      S_2 = \\begin{pmatrix}0,0.1,0\\end{pmatrix}.$$\n      Here $\\mathrm{sim}(C_1,C_2) = 0.9$ while $\\mathrm{sim}(S_1,S_2) = 0.1$, which satisfies $\\mathrm{sim}(S_1,S_2)>\\mathrm{sim}(C_1,C_2)$.\n\n  1. Ignoring the bias, we then consider the norm of the causal outputs after the model of weight $W$:\n  $$W\\,C_1\n  = \\begin{pmatrix}1,0,1,0\\end{pmatrix},\\quad W\\,C_2\n  = \\begin{pmatrix}0.9,0.1,0.9,0.1\\end{pmatrix},\\quad$$\n  $$\\|W\\,C_1\\|_2 = \\sqrt2 \\approx 1.4142,$$\n  $$\\|W\\,C_2\\|_2\n  = \\sqrt{0.9^2 + 0.1^2 + 0.9^2 + 0.1^2}\n  = \\sqrt{1.64}\n  \\approx 1.2806.$$\n\n  1. The norm of the non-causal outputs after the model of weight $W$:\n  $$W\\,S_1\n  = \\begin{pmatrix}0,1,0,1\\end{pmatrix},\\quad\n  \\|W\\,S_1\\|_2 = \\sqrt2 \\approx 1.4142,$$\n  $$W\\,S_2\n  = \\begin{pmatrix}0,0.1,0,0.1\\end{pmatrix},\\quad\n  \\|W\\,S_2\\|_2 = \\sqrt{0.1^2 + 0.1^2} = \\sqrt{0.02} \\approx 0.1414.$$\n\n  1. We can get that:\n  $$\\mathbb{E}(\\|W\\,C\\|_2) \\approx 1.34 > \\mathbb{E}(\\|W\\,C\\|_2) \\approx 0.78,$$ justifying the relationship between norm and distribution shift.\n\n  We will refine the description of the relationship in Section 2.3.2 and replace the example in Appendix F with this one to make a better explanation.\n\n- **\"Compared to environment-inference-based invariant learning methods\"** To clarify the distinction between the alternating optimization in our IDG method and environment inference approaches (e.g., EIIL, GOODHSE, MoleOOD), the following points are highlighted:\n\t1. In IDG training, alternating optimization is used mainly to prevent the subgraph extractor and predictor from converging to degenerate solutions or experiencing gradient conflicts when trained jointly under the same objective (classification loss and norm regularization). The extractor and predictor parameters are still updated alternately within each epoch, so they still train in a \"collaborative\" manner rather than being fully \"decoupled\" (for example, training the extractor to convergence before the predictor would be completely unworkable). Moreover, no explicit environment splits or pseudo-environment labels are produced.\n\t2. In environment inference approaches (e.g., EIIL, GOODHSE, MoleOOD), alternating optimization is required because the invariant learning phase demands explicit environment partitions or labels. These methods first infer environment information and then carry out invariant learning or related strategies. This two-stage process—environment inference followed by invariant learning—should be strictlly followed and can be fully decoupled, as in EIIL and MoleOOD.\n\n  We also conduct experiments on DrugOOD to compare performance under alternating versus non-alternating optimization. The results below show that alternating optimization significantly outperforms non-alternating optimization.\n\n  |DrugOOD|IC50Scaffold|IC50Size|IC50Assay|EC50Scaffold|EC50Size|EC50Assay|\n  |-|-|-|-|-|-|-|\n  |ERM|68.79|67.50|71.63|64.98|65.10|67.39|\n  |IRM|67.22|61.58|71.15|63.86|59.19|67.77|\n  |MoleOOD|68.02|66.51|71.38|66.69|65.09|73.25|\n  |GOODHSE|69.43|68.64|73.26|69.15|66.87|80.15|\n  |**IDG**|**69.97**|**69.02**|**73.34**|**69.57**|**68.03**|**80.54**|\n  |**IDG-noalter**|**67.97**|**67.23**|**70.84**|**65.43**|**65.67**|**71.24**|\n\n  Thank you for your kind attention to the training procedure. We will include comparisons with the environment-inference-based invariant learning methods you mentioned—such as GOODHSE and MoleOOD—in both the Related Work and Experimental sections in the future version.\n\n***Questions***  \n\n- **\"Handle Real-World Shifts\"** Indeed, on real-world datasets our method can also mitigate distribution shifts to some extent. The results on the DrugOOD benchmark, as you mentioned, are presented in the table above in the response of weakness 2. More importantly, we observe that our assumption—that causal subgraph distribution shifts are relatively small—still holds across multiple real-world datasets. Because real-world datasets lack ground-truth annotations for causal subgraphs, validating this hypothesis on them is extremely challenging. Nevertheless, we devised two distinct experimental setups to validate our assumption under different settings:\n\n  - To further support the validity of the causal subgraphs being more invariant in real applications, we conducte two experiments on real world datasets comparing causal and non-causal subgraphs. Quantifying distribution shifts in real-world graphs is challenging, since such data with ground-truth annotation of causal subgraph are rarely available, so we devise two distinct experimental setups to validate our assumption under different settings. Distribution shift was measured by the maximum mean discrepancy (MMD) between representations produced by the a same GNN model, using a Gaussian kernel on standardized features with bandwidth set by the median heuristic. We denote the cross‐environment distribution shift of causal and non‐causal subgraph by $\\Delta{G_c}\\approx MMD(G_c^{e1},G_c^{e2}), \\Delta{G_s}\\approx MMD(G_s^{e1},G_s^{e2})$ and compute the ratio of $\\Delta{G_s}/\\Delta{G_c}$. Table values above 1 indicate that non‐causal subgraphs undergo larger distribution shifts across environments. The experimental designs and results are described below:\n\n    - Experiment 1. Four real-world molecular graph datasets with ground-truth causal subgraphs—Mutag, Benzene, Fluoride, and Alkane- from the graph explanation literature. Each dataset was split into two environments by molecular size or random. The results are below:\n     \n    |$\\Delta{G_s}/\\Delta{G_c}$|Mutag||Benzene||Fluoride||Alkane||\n    |-|-|-|-|-|-|-|-|-|\n    |split|size|random|size|random|size|random|size|random|\n    |MMD|3.43|1.56|5.79|1.83|6.67|1.94|6.48|2.03|\n\n    The results indicate that, the non-causal subgraph exhibits a larger shift than the causal subgraph across all environment partitions($\\Delta{G_s}>\\Delta{G_c}$), which aligns with Assumption 2.\n\n    - Experiment 2. We apply the same evaluation to the out-of-distribution (OOD) dataset used in our study. Due to the absence of true ground truth, we treat the subgraphs extracted by our method, GIL, and GSAT as causal subgraphs. The results are below:\n\n    |$\\Delta{G_s}/\\Delta{G_c}$|HIV-scaffold|HIV-size|SST2|Twitter|IC50-size|IC50-scaffold|\n    |-|-|-|-|-|-|-|\n    |IDG|7.28|3.915|1.963|3.983|1.1784|1.5296|\n    |GIL|1.969|1.429|1.044|3.25|1.154|1.74|\n    |GSAT|5.292|4.671|1.868|1.244|1.128|1.156|\n\n    The results indicate that, the extracted non-causal subgraph by different methods still exhibits a larger shift than the causal, which also aligns with Assumption 2.\n  \n  These two experiments, viewed from different perspectives, clarify the validity of Assumption 2—that causal subgraphs typically exhibit smaller distribution shifts across environments.\n\n- **\"Continuous IRM\"** Indeed, our norm can be viewed as a continuous indicator of environments, but unlike continuous IRM, our method does not rely on any **hard or soft partitioning** of environments. It only assumes that the dataset contains data from different environments. Regarding the continuous IRM you mentioned, such as CORAL and DIR align the full-batch feature distributions—either in a latent or explicitly defined environment space—and enforce a single predictor to perform consistently across all environments. In contrast, our approach does not group or softly assign environments, nor does it explicitly align batch-level environmental distributions. Instead, we maximize a norm-based objective to identify at the sample-level causal subgraphs that are least prone to drift, without any hard or soft environment partitioning.\n\n- **\"Environment Modeling Help the Method?\"** Yes our method is fully compatible with IRM. When explicit environment labels are available, shifts in non-causal features across environments can be more evident. Furthermore, we integrat the IRM objective into our framework. As shown in the table below, this integration yields improved performance.\n  | Method   | PIIF  | FIIF  | SP=0.6 | SP=0.7 | SP=0.8 | SP=0.9 |\n  |----------|-------|-------|--------|--------|--------|--------|\n  | ERM| 62.45 | 37.22 | 91.67  | 91.25  | 90.63  | 85.92  |\n  | IRM| 68.34 | 44.33 | 91.85  | 91.25  | 90.48  | 85.38  |\n  | IDG| 79    | 83.67 | 92.67  | 92.06  | 91.89  | 91.33  |\n  | **IDG+ERM**  | **80.52**| **84.63** | **92.67**  | **92.34**  | **92.00**  | **91.89**  |\n\n  In this table, SP denotes the probability of spurious correlation. Lower SP values indicate weakly informative environments. PIIF and FIIF stand for partially informative and fully informative scenarios, respectively. These datasets adhere to the GALA[2], CIGA[3], and LECI[1] settings.\n\n[1] Joint Learning of Label and Environment Causal Independence for Graph Out-of-Distribution Generalization\n\n[2] Does Invariant Graph Learning via Environment Augmentation Learn Invariance?\n\n[3] Learning Causally Invariant Representations for Out-of-Distribution Generalization on Graphs"}]

Author rebuttal:

Meta-review: This paper initially got mixed scores: one accept, one borderline reject and two borderline accept. The authors have submitted a rebuttal, and after considering it, all reviewers expressed satisfaction with the responses and updated their scores to borderline acceptance (score 4/5). The AC concurs with the reviewers that this submission has the advantages of good results, novel idea, and well-written. Therefore, the AC recommends acceptance and encourages the authors to incorporate the clarifications—particularly the more detailed discussion of related work and the additional experiments provided in the rebuttal—into the final version.
Decision: Accept (poster)