Generative Risk Minimization for Out-of-Distribution Generalization on Graphs
Authors: Song Wang
Zhen Tan
Yaochen Zhu
Chuxu Zhang
Jundong Li
Venue: Accepted by TMLR ()

Abstract: Out-of-distribution (OOD) generalization on graphs aims at dealing with scenarios where the test graph distribution differs from the training graph distributions. Compared to i.i.d. data like images, the OOD generalization problem on graph-structured data remains challenging due to the non-i.i.d. property and complex structural information on graphs. Recently, several works on graph OOD generalization have explored extracting invariant subgraphs that share crucial classification information across different distributions. Nevertheless, such a strategy could be suboptimal for entirely capturing the invariant information, as the extraction of discrete structures could potentially lead to the loss of invariant information or the involvement of spurious information. In this paper, we propose an innovative framework, named Generative Risk Minimization (GRM), designed to generate an invariant subgraph for each input graph to be classified, instead of extraction. To address the challenge of optimization in the absence of optimal invariant subgraphs (i.e., ground truths), we derive a tractable form of the proposed GRM objective by introducing a latent causal variable, and its effectiveness is validated by our theoretical analysis. We further conduct extensive experiments across a variety of real-world graph datasets for both node-level and graph-level OOD generalization, and the results demonstrate the superiority of our framework GRM.

Published in Transactions on Machine Learning Research (02/2025)
Generative Risk Minimization for Out-of-Distribution Generalization on Graphs
Song Wang
suShun@virginia.ed
Deportment of Electrical and Computer Engineering University of Virginia
Zhen Tan
slan36 edu
Department Electrical and Computer Engineering University of Virginia
Yaochen Zhu
uqp4 @vviggnna edu
Department of Electrical and Computer Engineering University of Virginia
Chuxu Zhang
chuzz..hangguucom. edu
School of Computing Universily of Connecticul
Jundong Li
Diviniiiavii edu
Department of Electrical and Computer Engineering University of Virginia
Reviewed OpenReview: httpss//pperreeeeessss/////mooor
Abstract
(OOD) generalization graphs aims at dealing with scenarios where the test graph distribution differs from the training graph distributions. Compared to i.i.d. data like images, the OOD generalization problem on graph-structured data remains challenging due to the non-i.i.d. property and complex structural information on graphs. Recently, several works on graph OOD generalization have explored extracting invariant subgraphs that share crucial classification information across different distributions. Nevertheless, such strategy could be suboptimal for entirely capturing the invariant information, the extraction of discrete structures could potentially lead to the loss of invariant information or the involvement of spurious information. In this paper, we propose an innovative framework, named Generative Risk Minimization (GRM), designed to generate an invariant subgraph address the challenge of
for each input graph to be classified, instead extraction. optimization in the absence of optimal invariant subgraphs (i.e., ground truths), we derive tractable form of the proposed GRM objective by introducing latent causal variable, and its effectiveness is validated by theoretical analysis. We further conduct extensive experiments across variety of real-world graph datasets for both node-level and graph-level OOD generalization, and the results demonstrate the superiority of our framework GRM. Our code provided at hhttps///gnt https://github.comm www../oooo//onSS//WWS W-S GRM
1
Introduction
In recent years, it has become increasingly crucial to develop machine learning models that can handle tasks with test data distributions differing from training data, commonly referred to as out-of-distribatiin (OOD) generalization (Mansour al.) 2009 Blanchard et al. 2011 Muandet et al. 2013 Beery et al., 2018 Recht
let al. 2019 Su et al.] 2019). Such disparities, termed distribution shifts, substantially undermine the

---

Published in Transactions on Machine Learning Research (02/2025)
efficacy of the empirical risk minimization (ERM) paradigm, which presumes consistency in data distribution across training and test phases ( Candelllo-Candela et al.) 2008: Lazer et al. 2014 Zhang et al. 2018). While
there exist numerous works (Hu et al. 2018 Krueger et al., 2021 Chang et al. 2020 Sagawa et al. 2020
Koh et al. 2 on OOD generalization for i.i.d. (independent and identically distributed) data (c.g..
images), lew have focused on grapl-structured data, despite the prevalence of distribution shifts in real- world graphs (Fakhraci et al. 2015 Gui ct al. 2022 Yu et al. 2023). For instance, in citation networks (Hu) et al. 2020), the distribution of paper topics (i.e., node labels) may considerably change over time, leading to
differences in graph structures. However, Graph Neural Networks (GNNs) (Kipf & Welling 2017 Hamilton
ct al. 2017 Xu et al.; 2019: Zhou et al. 2020 You et al. 2020), despite being the de facto choice to model
graphs often fall short in addressing the challenge of distribution shifts on OOD graph data (AlBadawy
et al.
2018 Dai & Van Gool 2018 Li et al.
2022a Tan et al. 2022a Zhang et al. 2024).
Existing works for OOD generalization primarily focus on identifying invariant relatäonships across diverse data distributions, generally referred to as encironments (or domains) (Arjovsky et al. 2019 Chang et al., 2020 Abuja et. al.) (020). They typically aim to identify this invariant. relationship, which maps input invariant features to the outputs (e.g., labels), through robust optimization or learning an invariant fea-
ture space /Creager ct al. 2021 Krueger et al.) 2021). Regarding OOD generalization on graphs, existing
works (Chen et al. 2022 Miao et al. 2022; Chen et al. 2023 Tan et al. 2023 Wang et al. 2024) primarily aim to identify an invariant subgraph Ge from a given graph G for predictions. However, such an extraction strategy could be subpar for completely capturing the invariant information, which could be mixed with spurious information in a graph and could not be distinctly separated (Bevilacqua et al. 2021). For exam- ple, due to the complicated interactions (as edges) of atoms (as nodes) in a molecule graph, extracting a node may inevitably incorporate both invariant and spurious information, thus failing to achiove procise invariant subgraph (Gui et al.] 2022). Concretely, the stralegy of extracting (discrete) structures may not extract invariant information on graphs
To deal with this, we propose an innovative framework, named Generative Risk Minimization (GRM), to fully exploit invariant information on graphs. Different from the distinct extraction of invariant subgraphs used in existing works (Chen et al. 2023; 2022), the core idea of GRM to generate the invariant subgraphs in continuous manner. In particular, the generated invariant subgraph preserves the same set of nodes the imput G, while possessing continnous edge weights and node representations. This design allows to flexibly preserve the invariant information without the need for extracting discrete structures, which could potentially lead to loss of invariant information. To ensure that the generated subgraphs contain sufficient invariant information, our proposed GRM framework involves two objectives: (1) generation objective, which aims to generate precise subgraphs with continuous edge weights and node representations, and (2) invariant objective, which ensures the independence of the invariant subgraph and the domains.
Although the above two objectives are straightforward and intuitive, it is challenging directly optimize them, especially when the domain labels are unavailable (Wu et al., 2022b). Therefore, we transform our GRM objective into three correlated losses that could be used for optimization, based on our theoretical analysis. In particular, our GRM framework achieves two attractive properties for OOD generalization. (1) Maximally involving invariant information. We introduce a variational approximation of the latent causal variable Z to both the generation objective and the invariant objective. Our derivation results ensure that the learned latent representation of Z involves minimal loss of invariant information. (2) Minimally involving spurious information. Our GRM framework could directly minimize the mutual information between the invariant subgraph and the domains when combined with our generation objective, based on our theoretical analysis. The derived loss forces the generator to focus less on domain-rclated information and thereby reduces the incorporation of spurious information. In summary, our contributions are as follows:
We develop the Generative Risk Minimization (GRM) framework, a novel approach that aims to generate invariant subgraphs for graph OOD generalization.
We provide a theoretical analysis that ensures the effectiveness of the generated subgraphs and sheds light on the rationales of GRM and its validity in graph OOD generalization tasks,
We evaluate GRM through extensive experiments on various real-world datasets that cover multiple types of distribution shifts. The results validate the superiority of GRM over state-of-the-art baselines.
2

---

Published in Transactions on Machine Learning Research (02/2025)
Influential Nodes
Domain-Specific Representations
D
H
?
Inv. Loss
Sup. Loss
Enc.
G
Z
G
Target
Dec.
Influential
Input Graph
Latent Variable
Invariant Subgraph
Generated
Reg. Loss
The overall framework of GRM. Each input graph G' is processed by the encoder of our generator to learn the latent variable Z. Then we extract the most influential nodes from the domain and learn a domain-specific representation for each node in G. These domain-specific reprosentations will be used in the invariance loss. We further classify the output invariant subgraph with a classifier to obtain the predictions. The regularization loss is calculated for Z and the invariant subgraph.
Figure 2:
2 Preliminaries
In this section, we provide the formulation for our stud- ied graph OOD generalization problem. We start by reprosenting graph (or local subgraph of node in node-level tasks) G EV,E, X), where V and & are the node set and the edge set, respectively. Moreover, X E M is a feature matrix, where the j-th row vee- for (dz-dimensional) represents the altribute of the j-th node. We can define the distribution of a graph and its label from domain D; as (G,Y) P(G, Y Di), where Y TEYis is the label of G. Here is the label space shared across domains. We further denote the training and test domains (i.c., graphs) as Der = (D1, D2..... DID..1 and respectively. Generally, existing works for OOD generalization on graphs pri- marily rely on the Structural Causal Models (SCMs), as shown in Fig. [ to interpret distribution shifts on graphs (Chen et al.] 2022 2023). Specifically, the observed graphs G and labels are affected by the latent causal variable Z and spurious variable S, which decide the underlying invariant subgraph Ge and the spurious subgraph Gs, respectively. As the spurious variable S is related to the domain D, existing works aim to identify the invariant subgraph Ge for the precise prediction of its label Y, without the effect of S. Specifically, the goal is to develop an invariant GNN, represented as M := := fa o g. This model comprises: 1) an extractor G + G. that identifies the invariant subgraph Go, and 2) a classifier ʃ G. + ) that predicts the label Y using the extracted G where G. denotes the space of subgraphs within G. However, the extractor in existing works could only output a discrete invariant subgraph Ge, which is a subset of edges and nodes in the input graph G. As result, the invariant information and spurious information cannot be entirely separated when node edge consists of both types of information.
Z
G
Y
G
D
Figure 1: The SCMs with distribution shift (left) and without distribution shifts (right).
Dre = =i,,
3 Methodology
In this section, we elaborate on our proposed Generative Risk Minimization (GRM) framework, which aims tackle the graph OOD generalization problem by generating invariant subgraphs instead of extraction. In the following, we first derive our proposed GRM objective and then introduce specific designs optimize the objective in generative manner. The overall process of our GRM framework is illustrated in Fig.
3

---

Published in Transactions on Machine Learning Research (02/2025)
3.1 GRM Objective
In our GRM framework, propose to learn classifier f(.) and a generator g(-), such that the generalor g(+) will output an invariant subgraph for each imput graph. Considering a graph input G = (V, E, X), the generator aims to outputs the invariant subgraph Ge = EV,E, for classification.
To ensure that the obtained subgraph is maximally invariant across domains while preserving the causal information, we consider the following learning objective for f and which is adopted in existing works (Chen
ct al. 2022 )023):
max Y(G s.t. G.I.D D, = g(G)
(1)
However, it is difficult to directly optimize this objective. Generally, the optimization objective of the generator maximize the log- likelihood term log P(Go(G). Combining this term, we propose a more feasible objective for Eq.
max E log P(G.GG) D)G
(2)
which is referred to as our proposed GRM objective. Although the GRM objective is straightforward, is intractable due to the lack of ground truth, i.c., Ge, for the generated invariant subgraph G.. Alternatively, based the SCMs distribution shifts as illustrated in Fig. 1 propose model the causal variable Z
as latent variable for graph generation. By introducing the latent causal variable Z, we are able to derive the following theorem that allows for an end-to-end optimization for our objective in Eq. (2).
Theorem 3.1. An evidence lower bound (ELBO) for optimization of the GRM objective, by introducing latent causal variable Z and variational approcimations QZZ) and Q(Go), is as follows:
max E' P(G. G.)) KL( - CI + [log P(ZID, (3)
The proof provided in Appendix A.1
(T((-I-)) denotes the Kullback-Leibler (KI.) divergence. Based on the above objective, we could devise specific losses for optimization of the classifier and generator.
3.2 Generator Implementation
Before we derive the detailed optimization losses basod on Theorem 3.1 we first introduce the implementation of our generator. In particular, we aim to model Q(Z) using the generator 9, which uses any graph G input. However, it remains challenging to model Z with suitable architecture of the generator g in the absence of the ground truth G.. In particular, we propose to leverage the Variational Graph Auto-Encoder (VGAE) (Kipf & Welling 2016 Simonovsky & Komodakis, 2018) for the generation of invariant subgraphs. This is because the optimization objective of VGAF involves a latent variable Z and aligns with the first term of the GRM objective. As such, we propose to implement the generator g(.) as a VGAE.
Following the VGAE architecture, our generator consists of an encoder and decoder. Given graph input G, the encoder maps it into latent space and outputs the latent variable 7 RIV[xd Here d, is the dimension size of Z. Moreover, Z involves [V] latent representations, i.e., Z {21, .,, Z which means we learn a latent representation for each node in G, and thus the number of nodes in Ge equals that in G. For each node vi, where i E 21, [V],, we learn its representation as follows:
2: ~ N diag(o?)), where μi GNN, (V, E)); and loga; GN,,
(4)
To generate node features of the invariant subgraph, ie., X E TR/Vxxd,, we leverage the obtained latent variable Z along with a lincar projection layer fz(:):
X (f ...
fa(zp))), fa
where fz.) = Wxz; + bx
(5)
Here Wx E Rd,xdd xd, is the weight of the projection layer, and bx E Rd= is the bias. Then we further generate edges from the latent variables z; as follows:
E = ( 1.2. .... VB} where = (z,) fc(z,)) and fa(z,) Waz + b
(6)
4

---

Published in Transactions on Machine Learning Research (02/2025)
Here We Rdaxd is the weight of the projection layer, and be E R is the bias. de is the dimension size of fa(z.). ((x)= = 1/(1 exp( -x)) is the sigmoid function. Notably, unlike traditional graph generation tasks (De Cao & Kipf 2018 Jin et al. 2018), here we keep the continuous values of Ey as the edge weight and do not sample discrete edges. This is because we aim to generate precise subgraphs that maximally preserve the invariant information, and sampling discrete edges could potentially incorporate spurious information or cause the loss of invariant information. Through the above steps, we could generate an invariant subgraph G. - (V X), given an input graph G.
3.3 Optimization based on GRM
In this subsection, we introduce the detailed process to optimize our framework based on the GRM objective derived in Theorem 3.1 In particular, we design three different losses for the terms in the derivation result.
Supervision Loss. For the supervision loss, we first consider the term Eloog( (P(og(((GGlG, Z))]- As the ground truth Go is unobserved, the common choice of reconstruction loss in VGAE is unavailable. Therefore, we propose to adopt the label Y of G as a proxy for Ge. based on the intuition that the optimal G, should maximally reflect the information of the label Y. In this manner, we could formalize the supervision loss as follows:
C. c. ---(G) log log)(y)).), where plu]Ge) = fy(GG) and G. = g(G).
P(ylG) DEY
In the above loss, P(y|G.) is obtained by taking G. as input to the classifier f(),, and f,(()) denotes the output class probability regarding class y. Moreover, we set p(y|G) = 1 if y is the label of G, and p(yl G) 0, otherwise. The above supervision loss could be interpreted as a cross-entropy classification loss for Ge.
Regularization Loss. Generally, KL-divergence terms act as regularization in variational generation Kipf
& Welling 2016; Kingma et al.] 2019 Simonovsky & Komodakis 2018). In our derivation of the GRM objective in Theorem 3.1, the two KL-divergence terms (Q Z) P(z G)) and represent the differences between the distributions of Z (given D) and Q(Z), as well as between the dis- tributions of G. (given D and Z) and Q(G.). Notably, the derived result is applicable for any Q(Z) and Q(ā.). Specifically, we first define Q(Z) Gaussian distribution (0, I), where I E Rd xd" is the identity matrix. In this way, we could directly regularize the learned and log of Z, as P(Z G') is also a Gaussian distribution, and thus we could explicitly derive the KL-divergence between it and (V,0 I). For the second KT-divergence term, i.c., (I, this manner, we could obtain:
we first formulate Q(G.) as Q(G.) = Q(X) Q(E). In
= -KL(P(X)) -KL(((10,2)1QQ))))
(8)
Notably, as X is the linear projection of Z, the term (L ())((()) could also use Z for calculating the regularization loss in similar way to (LL(Q()) For another term- -KL(P(Ê)D, (L)(((()ZZIIII(() we could decompose Q(E) into multiple independent Bernoulli distributions as ēGg ~ Bernoull(d)), where E [] is controllable hyper-parameter. In this manner, we could consider the learned edge weight ēij as the parameter in a Bernoulli distribution and compute its KI--divereence with Q(E). In conerete, wo formulate the regularization loss as follows:
dx
[ v +
c, =
Σ 1=1
(0 + m²7) - log
logo; 0i
ΣΣ
(r(a.j, 0) + r(1 - aij. 0)),
(9)
where 0) = a log((a//). The first term is calculated from the KT-divergence of the two Gaussian distributions, which is KL(P(&ID, ())). The second term is calculated from KL-divergence between the two Bernoulli distributions, which is (L(P(E)D, ()).. Particularly, this loss regularizes the learning process of latent variable Z and the generation process of invariant subgraph G., such that the obtained G. is more generalizable to various domains.
Invariance Loss Finally, we consider the thrid term derived in Theorem 3.1 i.c., Elog P,Z|D, Intuitively, this term aims to derive the correct latent variable Z given the generated invariant subgraph G.
5

---

Published in Transactions on Machine Learning Research (02/2025)
and domain D. However, the ground truth of Z is unavailable during. Thus, propose to use the latent variable Z generated from Ge in Eq. (4) the proxy and minimize the discrepancy between Z and another set of latent variable H = {h1, (h,, hyy..,,byy ) learned from the domain D. We refer to H as the domain- specific latent variable. In this manner, optimizing this term could make the Iearned latent variable Z less vulnerable to the effect of domain-specific information, thereby enhancing the invariance of Z. Specifically, we aim to precisely capture the domain information that is maximally related to nodes in V. Due to the diversity of nodes within each domain, the useful domain information can be different for various nodes in G and also distributed across the entire graph (Gui et al. 2022). Therefore, we propose to learn h, by considering nodes that are influential on 0g- Specifically, we construct a subgraph from these influential nodes for U2 and learn h, from this subgraph. To effectively select influential nodes, we consider both the shortest path distance and the number of shortest paths. In practice, we choose the one-hop neighboring node set Nr of in and select nodes that are most influential to NN to maximally capture domain information. Notably, we select the one-hop neighboring node set because according to Theorem in (Huang & Zitnik 2020), the influence of one node on another decreases exponentially as the distance between the two nodes increases. As a result, to determine the nodes from the domain that are most influential for specific node, it is logical to prioritize nodes with the smallest distances. To summarize, we can represent the selected nodes for learning the domain-specific representation h, of node U/ (the i-th node in V) as follows:
vp- = (u(TTs(u,N)) S L", Ps( }, where i =
(10)
Here L' E R and P E R are hyper-parameters that control the number of selected nodes for learning domain representations based on Ls and Ps, respectively. N₁ is the set of one-hop neighboring nodes of 0s and V is the node set of G. In this way, we can learn the domain-specifie representation h, of node es as follows:
h; Mean (XP VP (PP)), where E.- = =[(vas ([(ta)))/uaa ub) us E vPJ.
(11)
Here, X and E are the corresponding node features and edge set of V,P; respectively. h, is achieved by mean-pooling over learned representations of nodes in Vp, learned by the same GNN enocder in Eq. (4)
Then we could achieve the invariance loss for optimizing the term E[log P(Z)D, Ga)] in Theorem 3.1 as follows:
V h. 2111,, where z, (((aa,,, diag(o?)).
1 V
(12)
Here 2; is obtained in the same way as in Eq. 44 As such, the invariance loss could be used to alleviate the domain influence on learned Z, which may involve spurious information.
Optimization. With our derived losses, the overall GRM objective for optimization is formulated as follows:
L =
(G, (G) + BLa(G,
(13)
where a and B are two pyper-parametery to control the weight of C, and La, respectively. In this way, we can effectively optimize our proposed GRM objective to lackle the OOD generalization on graph data.
3.4 Complexity Analysis
In this subsection, we analyze the time complexity of our framework. Particularly, the time complexity of our framework is primarily determined by the GNN encoder and the VGAE generator module, along with the three losses. Therefore, we first break down the complexity by considering the GNN and VGAE separately, then combining their contributions. Note that the time complexity of the GNN encoder is O(DVId + [E]d). For the VGAE complexity, the module (1) encodes each node's representation and (2) reconstructs the node's representation. For cach node, the VAE in VGAE performs operations involving encoding and decoding, which lypically, and thus the time complexity for each node's VAE operation is proportional to d². Thus, for all nodes, the VAE complexity is O(V]d)). Note that this process already involves the time complexity of the regularization loss. For the remaining two losses, the supervision loss and the invariance loss, we compute the time complexity as follows. First, since we are using the cross-entropy loss as the supervision loss, the time complexity is O(V]d). The invariance loss involves computing the Euclidean distance bet
6

---

Published in Transactions on Machine Learning Research (02/2025)
Table 1: Statistics of six out-of-distributiin node classification datasets.
Shift Type
Dataset
# Nodes
# Edges
# Classes
Domains
Metrie
Artificial
Cora
2,703
5,278
10
1/1/8
Accuracy
Transformation
Photo
7,650
119,081
10
1/1/8
Accuracy
Twitch
1,912 9,498
31 31,299-
153, 138
2
1/1/5
ROC-AUC
Transfers
FB-100
769 - 41,536
16,656
1,590,655
2
3/2/3
Accuracy
Temporal
Elliptic
203,769
234,355
2
5/5/9
Fl Score
Evolution
Arxiv
169,343
1,166,243
40
1/1/3
Accuracy
Table 2: The graph OOD generalization results (test accuracy in % for Cora, Photo, FB-100, and ROC-AUC in % for Twitch). The best results are in bold.
Dataset
Cora
Photo
FB-100
Twitch
Method
Min.
Avg.
Min.
Avg.
Min.
Avg.
Min.
Avg.
ERM
65.0
68.210.4
84.411.5
88.611.3
50.520.4
52.810.6
49.7.1.1
52.210.9
DRNN
56.4+1.4
74.8+1.2
76.7+1.5
77.1+1.2
48.0=1.0
51.4+0.7
44.0+0.5
48.1+1.4
MMD
52.4=1.5
75.8+0.6
82.1
84.8+0.6
51.4=0.9
53.3+0.7
42.8+006
49.1
ARM
60.6-1.1
62.9+1.4
58.3+1.1
74.6+0.7
50.7=1.3
54.5+009
43.2+1.5
48.5+1.3
EERM
68.0=0.6
70.5+1.0
90.8+0.5
91.8+0099
50.9=0.4
54.3+114
51.6+00.8
54.1#0.9
LISA
71.
76.7+0.8
90.3+1.2
91.5+1.5
48.8=1.2
54.2+1.0
48.6+1.2
55.8+2.2
IS-GIB
71.311.9
78.611.5
87.210.6
90.210.9
49.611.6
54.61112
51.21119
56.011.2
MARIO
70.8#1.3
76.
88.6+0.8
89.4+1.4
50.3=1.9
53.9+1.4
50.7+2.0
55.1+1.9
GRM
74.211.2
81.2+1.5
91.3
92.7+1.6
52.0#1.33
55.1+1.1
52.5=1.7
56.71110
two embeddings of each node, averaging across the graph. Therefore, the time complexity is O(I(?d). In conclusion, the overall time complexity is calculated
O(VId + [E|d + Mid + Mid + 1)
(14)
By simplifying the above time complexity, we can obtain the final time complexity as
O(Vid +18|d+1y/d)
(15)
4 Experiments
4.1 Experimental Setup
Datasets. In our node-level OOD generalization experiments, evaluate GRM and other state-of-the- art baselines on six real-world datasets that cover different topics and tasks, following EERM (Wu et al. 2022a). We summarize the statistics of these datasets in Table [ Specifically, we use datasets that involve three different types of distribution shifts: (1) " "Artificial Transformation" denotes that synthetic spurious fcatures are added to these datascts; (2) a "Cross-Domate Transfers" means that each domain in the datascts corresponds to a graph distinct from each other; (3) "Temporal Evolution" means that the datasets are dynamic with evolving nature. Each type includes two datasets. More details about these datasets can be found in Appendix B
Baselines. We evaluate our GRM framework in comparison to two sets of baselines. The general OOD generalization methods include ERM, DRNN (Koh et al. 2021). MMD (Li et al.) 2018), and ARM (Zhang et al. 2021). The stato-of-the-art graph OOD generalization methods include EERM (Wu al, 2022a IS-GIB YYagg al.] (2023), MARIO (Zhu et al., 2023), and LiSA (Yu al. 2023). We provide more details and the parameter settings of these baselines in Appendix

---

Published in Transactions on Machine Learning Research (02/2025)
Table 3: The graph OOD generalization results (test accuracy % for Arxiv and Fl score in % for Elliptic). The best results are in bold.
Dataset
Elliptic
Arxiv
Method
T1
T2
T3
Avg.
T1
T2
T3
Avg.
ERM
59.6:1.4
63.5=1.3
61.740.6
61.6+11
47.6
45.5+14
41.411.
44.8:11.4
DRNN
73.2+1.4
71.4=0.7
70.6+0.3
71.8+0.8
46.8+0.5
44.7+1.1
40.5+1.
44.0+1.0
MMD
71.910.7
70.1
69.940.8
70.6+0.8
44.6+1.3
42.410.7
38.941.0
42.040.5
ARM
72.141.5
69.7=0.7
67.9+1.4
69.9
44.9
42.320.6
39.71..88
42.3:11.0
EERM
66.3+0.4
63.8=0.6
55.5+0.6
61.9+1.1
50.3#1.4
48.3+0.4
44.7+1.4
47.8+1.4
LISA
68.810.9
65.6
69.311.0
67.910.8
45.920.6
42.310.5
46.1
44.7:10.6
IS-GIB
71.2+1.11
70.0=1.0
70.4+1.2
70.5+1.1
49.3#0.8
46.6:0.9
50.5+1.3
48.8
MARIO
69.8+1.9
72.8=2.4
71.1+14
71.2+2.0
48.8+2.3
50.1+2.4
49.2+2.4
49.4+2.8
GRM
19.4
85.511.1
89.1
88.0=1.4
52.210.9
52.611.4
56.111.4
53.6
GRM Settings. In this subsection, we introduce the detailed parameter settings in our framework GRM. Specifically, we use the Adam optimizer (Kingma & Ba 2015) for training. The dropout rate is set as 0.3, and the weight decay rate is 0.001. The learning rate set 0.01. Given an input graph, we utilize two 2-layer GCNs (Kipf & Welling, 2017), with hidden dimension size of 128, to learn domain-specific representations and node representations. Then concatenate these two representations of our VAE-based generator. The encoder of the generator is also implemented as 2-layer GCN. The dimension of latent variables (i.e., dz) is set as 128. For the specific values of I,* and p in selecting nodes for learning domain-specific representations, we set them as 3 and 1.5, respectively. For the neighborhood size the computation graph G of node v, i.e, L, we set it as In other words, two-hop neighbors will be included in the computation graph G. We run 5 times for this process and aggregate the classification results. We provide our code in the supplementary materials. During training, one NVIDIA A6000 GPU with 48GB of memory. We adopt the same GNN encoder for all baselines, i.e., 2-layer GCN (Kipf & Welling, 2017) Notably, since DRNN and MMD are not designed for scenarios with only one training domain, we use the interpolated domains generated by EERM as training domains for these two methods.
the input
conduct all experiments
4.2 Comparative Results on Node-Level Tasks
To comprehensively demonstrate the effectiveness of GRM, we evaluate its performance on six node-level datasets with different types of distribution shifts and provide results in Table and Table 3 We report the worst case result (Min.) and average result (Avg.) for the first four datasets since they consist of multiple (larger than three) test domains. The detailed results on each test domain are provided in Appendix D From the results, we summarize the observations as follows:
Across all datasets with various types of distribution shifts, GRM consistently outperforms all other basclines both the worst case (Min.) and average (Avg.) results, which validates the superiority of GRM on graph OOD generalization of node classification tasks.
The performance improvement of GRM over other baselines is substantially larger on Elliptic. This is because it contains a large number of test domains, leading to difficulties in generalizing to various test domains. Nevertheless, our GRM framework optimized with the invariance loss will provide better performance in this situation.
The performance variances of GRM across test domains are lower on Photo and Arxiv compared to other baselines. These datasets presorve greater node degrees (i.c., more complex structures) and a larger class set. Our generative framework can learn more precise invariant subgraphs with the designed regularization loss.
8

---

Published in Transactions on Machine Learning Research (02/2025)
4.3 Effect under Distribution Shifts
In this subsection, we evaluate the effectiveness of GRM under various degrees of distribution shift the Cora. dataset. We introduce artificial distribu- tion shifts on Cora by mixing node features gener- ated from labels and domain IDs (details are pro- vided in Appendix B and we refer to the modified dataset as Cora-Mix. We systematically evaluate our framework and other baselines on Cora-Mix un- der different spurious feature ratios and present the rosults in Fig. The results show that the perfor- mance of all methods drops signilicantly when the bias ratio increases. The performance drop is partic- ularly sharp when the bias ratio increases from to 0.1, indicating that spurious features can adversely affect all models even with a small ratio. Moreover, our proposed framework GRM consistently outperforms other baselines, especially when the bias ratio relatively large (e.g., 0.5 0.9). This demonstrates that GRM can effectively alleviate the adverse impact of spurious information by generating invariant subgraphs for classification.
90-
ERM
ARM
IS-GIB
Test Accuracy (%)
DRNN
BERM
MARIO
70
MMD
LISA
GRM
50
30
10
0.1 0
0.2
0.3
0.4
0.5 0.6 0.7 0.8 0.9 1.0 Bias Ratio
Figure 3: The results of various methods on dataset Cora-Mix with different degrees of distribution shifts.
4.4 Ablation Study
In this subsection, we perform a series of ablations studies to evaluate the efficacy of different compo- nents in our framework GRM. Specifically, we com- pare our proposed framework GRM with three de- generate versions: (1) GRM without the regular- ization loss, denoted as RRM (2) GRM with- out the invariance loss. We denote this variant as GRM,T; (3) GRM without the VGAF-based gener- ator, which mcans remove the stochastic sam- pling of Z during generation, denoted GRM,V. V.
90
GRM
GRMRR
GRMI
Test Result (%)
80
GRMW
70
60
50
From the results presented in Fig. 4 we obtain fol- lowing insights. (1) GRM consistently outperforms its variants with different components removed, in- dicating that each module GRM plays a vital role in handling distribution shifts. (2) Deprecating the invariance loss greatly reduces the performance Twitch and FB-100 with a limited number of do- mains. This result implies that the invariance loss is crucial for datasets with few domains for existing works to learn invariant representations. (3) The performance decreases differently by removing the VGAE module or the regularization loss. Specifically, removing the regularization loss typically leads to a more significant performance drop, as it is more challenging to generate precise invariant subgraphs without regularization. Namely, the potential risk of overfitting is detrimental to performance. (4) From a broader perspective, the invariance loss generally plays more critical role than the other two components, as its removal causes larger performance drop. This is because learning invariant subgraphs is essential for addressing distribu- tion shifts, as it directly impacts the effectiveness of a classifier trained on domain different from the test domain. (5) Moreover, the regularization loss and the VGAE module contribute in complementary ways. The regularization loss prevents the generated graphs from deviating from a specific distribution, while the VGAE module introduces randomness during generation. Both are crucial for maintaining the diversity and robustness of the generated subgraphs. In summary, these components work together to enable GRM achieve robust generalization across diverse datasets and distribution shifts.
40
Cora
Photo EllipticFB-100 FB-100 Twitch Arxiv
Figure 4:
Ablation study of our framework GRM with
different variants evaluated on six real-world datasets.
9

---

Published in Transactions on Machine Learning Research (02/2025)
4.5 Comparative Results on Graph-Level Tasks
Although we focus the node classification task, our method is also applicable to graph classifica- tion, ie., graph-level out-of-distribution generaliza- tion. In the setting for graph-level lasks, the domain information exists in other graphs and thus could not directly calculate the node inffuence. Thus, we still use the nodes in the input graph G to learn domain-specific representations H. Notably, as these nodes will not cover the entire graph G, the learned H will not be trivial, i.c., the same for all nodes in G. For graph-level experiments, We consider four prevalent datasets, namely SP- Motif (Ying et al. 2019), MNIST-75sp (Knyazev et al. 2019), G-SST2 (Graph-SST2) (Socher al.]
Table 4: The OOD graph classification results (ROC- AUC for Molhiv and accuracy in % for other datasets) of various methods on four datasets, with the best re- sults in bold.
Method DIR
SP-Motif
MNIST
G-SST2
Molhiv
39.87
20.36
83.29
77.05
GIL
46.04
21.94
83.44
79.08
CIGA
64.01
25.29
81.02
79.75
GALA
64.54
26.09
83.79
80.53
GRM
65.05
26.53
83.86
81.02
2013 and Molhiv (OGBG-Molhiv) (Hu et al.] 2020 ; with detailed provided in Appendix B.4 For base-
lines, we consider four state-of-the-art methods: DIR (Wu et al. 2022b GIL (Li ct al. 2022b), CIGA (Chen et al. 2022), , and GALA (Chen et al. 2023). From the results presented in Table [ we observe that GRM still exhibits competitive performance on OOD graph classification. Specifically, it achieves the best results over other basclines on all four datasets. The performance improvement is better on the datasct Molhiv with
a larger graph size, thereby providing richer domain knowledge for our GRM to learn invariant information.
5
Related Works
5.1
Out-of-Distribution (OOD) Generalization
OOD Generalization aims to learn a model that can generalize to an unseen test domain, given several different but related training domain(s). Prior invariant methods (Ganin & Lempitsky
2015 Li et al., 2018;
Arjovsky et al. 2019) genrcally focus learning invariant features (Sun et al. 2016 Peng et al.) 2019)
optimizing for the worst-case group performance (Hu et al. | Sagawa et al. 2020). Recent works for OOD
generalization on graphs (Chen et al.) 2022 Li et al.) 2022b Wang et al. 2024) could be typically categorized
into two classes: invariant learning and graph augmentation (Li et al. 2022a). Among invariant learning methods, CIGA (Chen et al., 2022) proposes to extract subgraphs that maximally preserve the invariant intra-class information based on causality. DIR (Wu el al. (202bb) uses a set of graph representations as the invariant rationales to create additional distributions. GIL (Li et al. (0022)) identifies invariant subgraphs via GNN-basod generator. More recently, MARIO (Zhu et al.) 2023) utilizes the Information Bottleneck (IB) principle to learn invariant information. Among augmentation methods, LiSA (Yu el al. 2023) proposes to leverage graph augmentation to obtain more diverse training data for learning invariant information. EERM (Wu et al. 2022a) generates domains by maximizing the loss variance between domains in an adversarial manner, such that the obtained domains could aid in learning invariant representations.
5.2 Graph Generative Models
In recent years, numerous works have been proposed for graph generation Youu et al., 2018 Grover 2019). Specifically, GraphVAE (Simonovsky & Komodakis, 2018) proposes & framework based on
et al.]
VAE
KKngmma & Welling, 2013) to generate graphs by encoding existing graphs. GraphRNN (You et al.
2018
generates graphs through sequence of node and edge formations. Moreover, several methods (Jin
ct al.
2018: Preuer et al.
2018) focus on generating graphs based on specific knowledge. For example.
MoIGAN (De Cao & Kipl 2018) adapts the framework of Generative Adversarial Networks (GANs) (Good-
fellow et al. 2014) to operate directly on graph-structured data with a reinforcement learning objective. Note that although these mothods leverage different information for generating graphs, they are not explic- itly proposed for handling the distribution shift problem on graphs. In contrast, our framework GRM aims to utilize domain information to generate graphs that are suitable for trained classifier.
10

---

Published in Transactions on Machine Learning Research (02/2025)
6 Conclusion
In this paper, propose novel framework, namely Generative Risk Minimization (GRM), generate invariant subgraphs for each imput graph to tackle the OOD generalization problem on graphs. Instead of extracting structures that may cause the loss of invariant information, we propose our GRM objective that incorporates a gencration term and a mutual information term. We derive three types losses to enable the optimization of our GRM objective the absence of ground truths for the invariant subgraphs. The effectiveness of GRM validated by our theoretical analysis and also the extensive experiments across both node-level and graph-level OOD generalization tasks. The results indicate the superiority of GRM over other state-of-the-art baselines.
Acknowledgments
This work is supported in part by the National Science Foundation under grants (IIS-2006444, IIS-2144209, IsS 2223769, CNS-2154962, BCS-2228534, and CMM12411248), the Commonwealth Cyber Initiative Awards under grants (VV-1Q24-011, VV-1Q25-004), and the research gift funding from Netflix and Snap.
References
Kartik Ahuja, Karthikeyan Shanmugam, Kush Varshney, and Amit Dhurandhar. Invariant risk minimization games, In ICML, 2020.
Ehab A AlBadawy, Ashirbani Saha, and Maciej A Mazurowski. Deep learning for segmentation of brain tumors: Impact of cross-institutiona training and testing. Medical Physics, 2018.
Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopoz-Paz. Invariant risk minimization.
arXiv:1907.022: 028993 2019.
Sara Beery, Grant Van Horn, and Pietro Perona. Recognition in terra incognita. In ECCV, 2018.
Beatrice Bevilacqua, Yangze Zhou, and Bruno Ribeiro. Size-invariant graph representations for graph clas- sification extrapolations, In ICML, 2021.
Gilles Blanchard, Gyemin Lee, and Clayton Scott. Generalizing from several related classification tasks to a new unlabeled sample. In NewrIPS, 2011.
Shiyu Chang, Yang Zhang, Mo Yu, and Tommi Jaakkola. Invariant rationalization. In ICML, 2020.
Yongqiang Chen, Yonggang Zhang, Yatao Bian, Han Yang, MA Kaili, Binghui Xie, Tongliang Liu, Bo Han, and James Cheng. Learning cansally invariant representations for out-of-distribution generalization on graphs. NewrIPS, 2022.
Yongqiang Chen, Yatao Bian, Kaiwen Zhou, Binghui Xie, Bo Han, and James Cheng. Does invariant graph learning via environment augmentation learn invariance? Advonces in Newral Information Processing Systems, 36, 2023.
Elliot Creager, förn-Henrte Jacobsen, and Richard Zemel. Environment inference for invariant learning. TCML, 2021.
Dengxin Dai and Lue Van Gool. Dark model adaptation: Semantic image segmentation from daytime to nighttime. In ITSC, 2018.
Nicola De Cao and Thomas Kipf. Molgan: An implicit generative model for small molecular graphs. orXiv:1805.11973 2018.
José Dolz, Pablo Piantanida, and Ismail Ben Ayed. Transductive information maximization for few-shot learning. In NeurIPS, 2020.
11

---

Published in Transactions on Machine Learning Research (02/2025)
Shobeir Fakhraci, James Foulds, Madhusudana Shashanka, and Lise Getoor. Collective spammer detection in evolving multi-relational social networks. In SIGKDD, 2015.
Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In ICML, 2015.
Ian Goodfello,, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua. Bengio. Generative adversarial nets. In NeurIPS, 2014.
Aditya Grover, Aaron Zweig, and Stefano Ermon. Graphite: Iterative generative modeling of graphs. ICML, 2019.
Shurui Gui, Xiner Li, Limei Wang, and Shuiwang Ji. Good: graph out-of-distribution benchmark. In NeurIPS Datasets and Benchmarks Track, 2022.
William L Hamilton, Rex Ying, and Jure Leskovec. Inductive representation learning on large graphs. NewrIPS, 2017.
Weihua Hu, Gang Niu, issei Sato, and Masashi Sugiyama. Does distributionally robust supervised learning give robust classifiers? ICML, 2018.
Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. In NeurIPS, 2020.
Kexin Huang and Marinka Zitnik. Graph meta learning via local subgraphs. In NewrIPS, 2020.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molecular graph generation. In JCML, 2018.
Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot,
Ce Liu, and Dilip Krishnan. Supervised contrastive learning. Advances in Newral Information Processing Systems, 2020.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Proceedings of the 2015 International Conference Learning Representations, 2015.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv:1912.6114, 2013.
Diederik P Kingma, Max Welling, et al. An introduction to variational autoencoders. Foundations and TrendsD in Machine Learning, 2019.
Thomas N Kipf and Max Welling. Variational graph auto-encoders. arXiv:1611.07300 2016.
Thomas N Kipf and Max Welling, Semi-supervised classification with graph convolutional networks. In TCLR, 2017.
Boris Knyazev, Graham W Taylor, and Mohamed Amer. Understanding attention and generalization in graph neural networks. In NewrIPS, 2019.
Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Trena Gao, et al. Wilds: A benchmark of in-the-wild distribution shifts. In ICML, 2021.
David Krueger, Ethan Caballero, Joern-Henrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. Out-of-distribution generalization via risk extrapolation (rex). In TCML, 2021.
David Lazer, Ryan Kennedy, Gary King, and Alessandro Vespignani. The parable of google flu: traps in big data analysis. Science, 2014.
Haoliang Li, Sinno Jialin Pan, Shiqi Wang, and Alex C Kot. Domain generalization with adversarial feature learning. In CVPR, 2018.
12

---

Published in Transactions on Machine Learning Research (02/2025)
Haoyang Li, Xin Wang, Ziwei Zhang, and Wenwu Zhu. Out-of-distribution generalization graphs: A survey. Xiv preprint orXiv:2202. 07987, 2022a.
Haoyang Li, Ziwei Zhang, Xin Wang, and Wenwu Zhu. Learning invariant graph representations for out-of- distribution generalization. In NeurIPS, 2022b.
Yishay Mansour, Mehryar Mohri, and Afshin Rostamizadeh. Domain adaptation: Learning bounds and algorithms. In COLT, 2009.
Hirthik Mathavan, Zhen Tan, Nivedh Mudiam, and Huan Liu. Inductive linear probing for few-shot node classification. In International Conference on Social Computing, Aataviooral-ulluuree Modeling and Pre- diction and Behavior Represenlation in Modeling and Simulation, pp. 774-284. Springer, 2023.
Siqi Miao, Mia Liu, and Pan Li. Interpretable and generalizable graph learning via stochastic attention mechanism. In ICML, 2022.
Krikamol Muandet, David Balduzzi, and Bernhard Schölkopf. Domain generalization via invariant feature representation. In ICML, 2013.
Aaron den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint ar arXic:1807.037// 2018.
Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. In ICCV, 2019.
Kristina Preuer, Philipp Renz, Thomas Unterthiner, Sepp Hochreiter, and Gunter Klambauer. Fréchet chemnet distance: a metric for generative models for molecules in drug discovery. Journal of Chemical Information and Modeling, 2018.
Joaquin Candel,, Masashi Sugiyama, Anton Schwaighofer, and Neil D Lawrence. Dotaset shift in machine learning. Mit Press, 2008.
Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers generalize to imagenet? In ICML, 2019.
Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang, Distributionally robust neural networks. In ICLR, 2020.
Martin Simonovsky and Nikos Komodakis. Graphvae: Towards generation of small graphs using variational autoencoders. In ICANN, 2018.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. EMNLP, 2013.
Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai. One pixel attack for fooling deep neural networks. IEEE TEVC, 2019.
Baochen Sun, Jiashi Feng, and Kate Saenko. Return of frustratingly easy domain adaptation. In AAAI, 2016.
Zhen Tan, Kaize Ding, Ruocheng Guo, and Huan Liu. Graph few-shot, class-incremental learning. In WSDM, 2022a.
Zhen Tan, Kaize Ding, Ruocheng Guo, and Huan Liu. Supervised graph contrastive learning for few-shot node classification. In Joint Europeon Conference on Machine Learning and Knowledge Discovery in Databases, 2022b.
Zhen Tan, Song Wang, Kaize Ding, Jundong Li, and Huan Liu. Transductive linear probing: A novel framework for few-shot node classification. XrX:m:2212 05606, 2022c.
13

---

Published in Transactions on Machine Learning Research (02/2025)
Zhen Tan, Ruocheng Guo, Kaize Ding, and Huan Liu. Virtual node tuning for few-shot node classification. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp- 2177 2188, 2023.
Song Wang, Zhen Tan, Huan Liu, and Jundong Li. Contrastive meta-learning for few-shot. node classification. In SIGKDD, 2023.
Song Wang, Yushun Dong, Binchi Zhang, Zihan Chen, Xingbo Fu, Yinhan He, Cong Shen, Chuxu Zhang, Nitesh Chawla, and Jundong Li. Safety in graph machine learning: Threats and safeguards. arXiv preprinl Xiv:24405. 1 1034, 2024.
Song Wang, Xiaodong Yang, Rashidul Islam, Huiyuan Chen, Minghua Xu, Jundong Li, and Yiwei Cai. Enhancing distribution and label consistency for graph out-of-distributinn generalization. In JCDM, 2024.
Qitian Wu, Hengrui Zhang, Junchi Yan, and David Wipf. Handling distribution shifts on graphs: An invariance perspective. In ICLR, 2022a.
Yingxin Wu, Xiang Wang, An Zhang, Xiangnan He, and Tat-Seng Chua. Discovering invariant rationales for graph neural networks. In ICLR, 2022b.
Zhenqin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S Pappu, Karl Leswing, and Vijay Pande. Moleculenet: a benchmark for molecular machine learning. Chemical Science, 2018.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In ICLR, 2019.
Ling Yang, Jiayi Zheng, Heyuan Wang, Zhongyi Liu, Zhilin Huang, Shenda Hong, Wentao Zhang, and Bin Cui. Individual and structural graph information bottlenecks for out-of-distribution generalization. TKDE, 2023.
Zhitao Ying, Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec. Gnnexplainer: Generating explanations for graph neural networks. In NeurIPS, 2019.
Jiaxuan You, Rex Ying, Xiang Ren, William Hamilton, and Jure Leskovec. Graphrnn: Generating realistic graphs with deep auto-regressive models. In ICML, 2018
Jiaxuan You, Zhitao Ying, and Jure Leskovec. Design space for graph neural networks. In NewrIPS, 2020.
Junchi Yu, Jian Liang, and Ran He. Mind the label shift of augmentation- graph ood generalization. In CVPR, 2023.
Hao Yuan, Haiyang Yu, Shurui Gui, and Shuiwang Ji. Explainability in graph neural networks: laxonomic survey. IEEE TPAMI, 2022.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In International Conference on Learning Representations, 2018.
Kexin Zhang, Shuhan Liu, Song Wang, Weili Shi, Chen Chen, Pan Li, Sheng Li, Jundong Ti, and Kaize Ding. A survey of deep graph learning under distribution shifts: from graph out-of-distribution generalization to adaptation. arXiv preprint ar XrXiv:24 10004/1......... 19265, 2024.
Marvin Zhang, Henrik Marklund, Nikita Dhawan, Abhishek Gupta, Sergey Levine, and Chelsea Finn. Adap- live risk minimization: Learning to adapt to domain shift. NeurIPS, 2021.
Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Ti, and Maosong Sun. Graph neural networks: A review of methods and applications. AT Open, 2020.
Yun Zhu, Haizhou Shi, Zhenshuo Zhang, and Siliang Tang. Mario: Model agnostic recipe for improving ood generalization of graph contrastive learning. 13X0002307..3055 2023.
14

---

Published in Transactions on Machine Learning Research (02/2025)
A Theoretical Analysis
A.1 Theorem 3.1 and Proof
In this section, provide proof for Theorem 3.1
Theorem 3.1. An evidence lower bound (ELBO) for optimization of the GRM objective, by introducing latent causal variable Z and a variational opprozimation Q(G.), is as follows:
max E log P(Gg(,))) KL(Q(Z(G))1(((1G)
(16)
EKKL(PG + E log P(ZID, G.)
Proof. We first present the GRM objective:
max E log P(G.)G) - T(G.:: D).
(17)
We first derive the ELBO for the generation objective, which a standard derivation for the variational auto-enocder (VAE):
log P(G.GG
=log| / P(Ge, Zl))d
-log| log / Q(ZIG) P(G 2|G) dz
G)
(using Jensen's Inequality)
P(G Z|G) dz
Q(IIG) log
G)
= =Eqllog P(G., 7
(18)
Q(Z(G)
(using the property of conditional probabilities)
P(G. G) P(Z|G)
= EEollog
Q(Z[G)
-Eollog P(G. Eqllog Q(ZGG)
P(Z|G)
(using the defimition of Kl-ddivergence)
=Eollog P(Gog - (ILGG
Then we decompose the second term -I(G,; D) of our GRM objective as follows, based on the definition of mutual information:
D) I(G.; AD) I(Gr: D,
(19)
We first decompose the first term:
P(G.ID,Z)
IG. ZD) = E
log
P(G.(D)
log P(Z]D.G.))
(20)
E
P(ZID)
= E
log P(ZID, G.) + H(Z
15

---

Published in Transactions on Machine Learning Research (02/2025)
We consider P(Z deterministic distribution for each D, and thus it could be ignored for optimization.
Then we dereive the lower bound for the second term:
P.
-IGEe (.Z)
GB.D
log
P(G.)
2(
-Booo
log
+ +KL(P(G))IQGC..)
Q(G.)
(21)
P((G.D.Z)
2 G
log
Q(G..)
Finally, could combine the above three derivation results to achieve the final evident lower bound for optimization of the GRM objective:
max E log P( )
(22)
EKKI + [oo P(ZID, Ga)
B Datasets in Experiments
In this section, we provide further details on the six datasets used in our experiments: Cora, Photo, Twi tch, FB-100, Elliptic, and Arxiv. Note that the datasets are originally processed by EERM (Wu et al. 2022a We follow the same dataset setting and splitting keep consistency. Additionally, Sec. 4.2 we report the Min. and Avg. performance on four datasets, and further detailed results of these datasetss are presented in Appendix D
B.1 Artificial Distribution Shifts on Cora and Photo
Cora and Photo are two popular benchmark datasots used for node classification tasks and are also widoly adopted to assess the effectiveness of GNN models. Specifically, these dalasets are of moderate size, con- taining thousands of nodes (2,703 and 7,650, respectively). The data statistics are provided in Table [ In particular, Cora is citation network, where nodes represent papers and edges indicate the citation relation- ship between them. On the other hand, Photo is co-purchasing network, with nodes representing specific goods and edges denoting frequent co-purchases of two goods. In the original dataset, the provided node features exhibit a strong correlation with node labels. Following EERM (Hamilton et al. 2017), in order to assess the model performance for graph OOD generalization under various distributions, we manually introduce distribution shifts into the training and testing data.
More specifically, we construct node labels and spurious domain-sensitive attributes from node features. Given the node features as X1, start by randomly initializing Graph Neural Network (GNN) with X; input and adjacency matrix to generate node labels Y. To obtain the one-hot hot label vectors, perform argmax operation in the output layer. Then, we randomly initialize another GNN with the concatenation of Y and a domain ID as input to generate spurious node features X2. The next step is lo concatenate these two sets of node features, ie., X [X1,X2) to create new node features for training and test data. times for each datasot, resulting ten graphs with different domain IDs.
This process is performed For training and validation, we utilize one graph each, while the classification accuracy reported on the remaining graphs.
16

---

Published in Transactions on Machine Learning Research (02/2025)
B.2 Cross- Domaan Transfers Twitch and FB-100
Cross-domain transfers are common occurrence in scenarios involving distribution shifts graphs. In various real-world situations, multiple observed graphs are available, each belonging to a specific domain. For instance, in social networks, domains can be defined based on where or when the networks are collected. Similarly, in protein networks, distinct species may have their own observed graph dala, such as protein- protein interactions, representing separate domains. The key point is that graph data typically captures relational structures among specific entitics, which often exhibit unique charactcristics for different entity groups. As a result, the dala-generating distributions can vary across these groups, leading to domain shifts.
However, in order to facilitate transfer learning across graphs, it is necessary for the graphs within a dataset to share the same input feature space and output space. To achieve this roquirement, we utilize two publicly available datasets, Twitch and FB-100, which satisfy these conditions.
Specifically, the Twitch dataset consists of seven networks, where nodes and edges represent Twitch users and their mutual friendships, respectively. These networks are collected from different regions, namely DE, ENGB, ES, FR, PTBR, RU, and TW. Although these networks have similar sizes, they exhibit variations in terms of density and maximum node degrees, as presented in Table
The FB-100 dataset comprises 100 snapshots of Facebook friendship networks from 2005. Here each network contains nodes that represent Facebook users from a specific American university. In our experiments, we utilize fourteen networks: John Hopkins, Caltech, Amhorst, Bingham, Duke, Princcton, WashU, Brandeis, Carnegie, Cornell, Yale, Penn, Brown, and Texas. These graphs exhibit siguificant variations in terms of sizes, densities, and degree distributions, indicating that the model capability in handling different graph structures becomes crucial for this dataset.
B.3 Temporal Evolution on Dynamic Graph Data: Elliptic and Arxiv
The distribution shift problem can also occur in temporal graphs that dynamically change over time. The evolution of these graphs can be generally categorized into two types. In the first type, there exist multiple snapshots of the graph, with each snapshot captured at a specific time. As time progresses, a sequence of graph snapshots is generated, which may exhibit variations in terms of node sets and data distributions. For example, financial networks capture the payment flows among transactions within different time intervals and thus result in different domains. In the second type, there exists only one single graph that evolves through the addition or delction of nodes and edges. This type is commonly seen in large-scale real-world graphs, such social networks and citation networks. In these graphs, the distribution of node features, edges, and labels often exhibit strong correlation with time at different scales. For our graph OOD generalization experiments, we utilize two public real-world datascts, namely Elliptic and Arxiv. These datascts are suitable for exploring node classification tasks within the context of evolving temporal graphs.
The Elliptic dataset consists of a series of 49 graph snapshots, where each snapshot represents a network of Bitcoin transactions. Specifically, each node corresponds to transaction and cach edge represents payment flow. Within these transactions, approximately 20% are labeled as either licit or illicit, with the objective being to identify illicit transactions within future networks. In the original dataset, the first six graph snapshots contain highly imbalanced classes, with the number of illicit transactions being less than 10 among thousands of nodes. Consequently, we exclude these snapshots and focus on the 7th to 11th, 12th to 17th, and 17th to 49th snapshots for training, validation, and testing, respectively. Furthermore, due to the low positive label rate observed in cach graph snapshot, we organize the 33 testing graph snapshots into 9 distinct test sets based on their chronological order. The dataset also requires the framework to effectively handle diverse label distributions encountered during the transition from training to testing data.
The Arxiv datasct comprises 169,343 Arxiv CS papers covering 40 subject arcas, along with their citation relationships. The objective is to predict the subject area of a given paper. In (Hu et al. 2020), the papers published before 2017, in 2018, and since 2019 were utilized for training, validation, and testing, respectively, They employed a transductive learning setting (Tan et al., 2022c Dolz et al., 2020 Mathavan et al. 2023). wherein the nodes in the validation and test sets were also present in the training graph. Instead, Arxiv adopts an inductive learning setting, which better reflects real-world scenarios. Here, the nodes in the
17

---

Published in Transactions on Machine Learning Research (02/2025)
validation and test sets are unseen during training, introducing a greater level of novelty. Specifically, the dataset consists of papers published before 2011 for training, papers from 2011 1 to 2014 for validation, and papers after 2014 for testing. Such a splitting strategy introduces a distribution shift between the training and testing data, as specific latent influential factors (such as research topic popularity) in data generation would change over time.
Table 5: Statistics of Twitch dataset.
Domain
Nodes
# Edges
Density
Avg. Degree
Max. Degree
DE
9,498
153,138
0.0033
16
3,475
ENGB
7,126
35,324
0.0013
465
ES
4,648
59,382
0.0054
12
809
FR
6,549
112,666
0.0052
17
1,517
PTBR
1,912
31,299
0.0171
16
455
RU
4,385
37,304
0.0038
575
TW
2,772
63,462
0.0165
22
1,171
B.4 Graph Classification Datasets for coat-o--Disirbbttion Generalization
Here we introduce the four datasets used in our experiments in Sec. 4.5 focusing on graph classification
tasks. In particular, we leverage the following datasets:
SP-Motif (Ying et al. 2019) The SP-Motif dataset, comprising 18,000 synthetic graphs, is constructed
by combining a base graph (denoted by Tree, Ladder, or Wheel, represented as S = 0, 1, respectively) with a motif (Cycle, House, or Cranc, denoted as C 1.2 respectively). The label Y of cach graph exclusively determined by its motif C. In the rucstonttion of the training set, deliberate spurious correlations between the base S and the label Y are introduced. These correlations are quantified by the formula P(S) b x I(S C) + (1 b)/2 x I(S & C), where the motif follows uniform distribution, and the base's 's distribution is contingent on the motif. The parameter b is varied to produce different levels of bias within the Spurious-Man Mottif datasets. The testing set features randomly combined motifs and bases, including graphs with larger bases, to accentuate the distributional disparities. In our experiments, we the value of as 0.9.
MNIST-75sp (Knyazev et al. 2019): This dataset transforms MNIST images into 70,000 graphs, each comprising up to 75 superpixels. These superpixels, which serve graph nodes, are interconnected based on their spatial proximity, forming the graph edges. Each graph is categorized into one of 10 classes. Notably, the testing set is augmented with random noise in the node features to introduce variability.
Graph-SST2 (Socher et al. 2013; Yuan et al. 2022 This dataset includes graphs labeled according to sentence sentiment, with nodes representing tokens and edges reflecting the syntactic relationships between them. The graphs this dataset are partitioned into different subsets based on their average node degree, thereby creating dataset shifts.
Molhiv (OGBG-Molliv) (Wu et al. 2018 Hu et al. 2020 Designed for the task of molecular property prediction, the Molhiv dataset encompasses graphs that represent molecules, where nodes correspond to atoms and edges denote chemical bonds. Each graph is labeled based on its efficacy in inhibiting HIV replication.
C Implementation Details
C.1 Baseline Settings
In this subsection, introduce the detailed settings for baseline methods used in our experiments.
18

---

Published in Transactions on Machine Learning Research (02/2025)
ERM: ERM denotes Empirical Risk Minimization, which conducts learning across domains without designs for distribution shifts. This baseline acts vanilla comparison where specific techniques are employed for OOD generalization. We use the same GNN encoder as our framework and set the learning rate as 0.001.
EERM (Wu et al. 2022a): EERM denotes Explore-to-Extrrpplle Risk Minimization, which minimizes the mean and variance of risks from multiple domains that are simulated by adversarial context genera- tors. For the parameter setting, we follow the choice in the original paper and search the hyper-parameters with grid search on the validation dataset. Specifically, the learning rate for GNN is chosen from {0.0001, 0.0002, 0.001, 0.005, 0.01), the learning rate for graph editers from {0.0001, 0.001, 0.005, 0.01), and the weight. for combination is chosen from {0.2, 0.5, 1.0, 3.0).
ARM Zhang et al. 2021 1: ARM denotes Adaptive Risk Minimization, which adapts the model parame-
ters to various domains based on a small batch of data from the domain. The ARM framework consists of three variants: ARM-CML, ARM-BN, and ARM-LL. In our experiments, we compare the variant of ARM-CML, which significantly outperforms all variants. Since ARM is not explicitly designed for graph data, employ the identical GNN architecture to our framework as the encoder and randomly select nodes from each domain for adaptation. Following the original setting in the paper, we set the learning rate as 0.0001, the weight decay rate as 0.0001, and the support size as 50. Furthermore, for ARM-CML, the number of context channels is set as three.
DRNN (Koh et al., 2:21): DRNN aims to tackle the distribution shift problem by ensuring that the distribution minority receives sufficient training. Following the setting in ARM, we set the learning rate as 0.0001 and the robust step size as 0.01.
MMD Li et al. 2018): MMD aims to maximize the mean discrepancy across domains. For the parameter setting, we set the learning rate as 0.0001 and the gamma value as 1. The support size is set the same as ARM as 50.
IS-GIB (Yang et al. 2023 IS-GIB aims to discard spurious features while learning invariant features from a high-order perspective via preserving class relationships under various distribution shifts. We follow the parameter setting in their code and set the learning rate as 0.01.
MARIO (Zhu et al. 2023): MARIO proposes to simultancously achieve generalizable representations while obtaining invariant representations via adversarial data augmentations, based on graph contrastive
learning strategics (Oord et al. 2018 Khosla et al. 2020 Tan ct al. 2022b Wang et al. 2023 The
learning rale set as 0.001.
LiSA (Yu et al. 2023 LiSA proposes to leverage variational subgraph generators to extract locally predictive patterns that could be used for constructing label-invariant subgraphs. These subgraphs are then used create augmented environments with enhanced diversity.
We adopt the same GNN encoder for baselines, ie., a 2-layer GCN (Kipf Welling, 2017 Since DRNN and MMD are not designed for scenarios with only one training domain, we use the interpolated domains generated by EERM as training domains for these two methods.
C.2 Training Details
During training, we conduct all experiments on one NVIDIA A6000 GPU with 48GB of memory. The package requirements of our experiments listed below.
Python 3.7.10
torch = 1.8.1
numpy = 1.18.5
scipy 1.5.3
19

---

Published in Transactions on Machine Learning Research (02/2025)
networkx = 2.5.1
scikit-learn - 0.24.1
pandas = 1.2.3
C.3
Training Time and Memory Usage
In this subsection, we provide the time/memory of all experiments conducted on one NVIDIA A6000 GPU with 48GB of memory in Table 6
Table 6: Training time and memory usage for different datasets.
Dataset
Total Training Time (s)
Time Per Epoch (s)
Memory Usage (MB)
Cora
1,782.96
3.91
1,329
Photo
3,504.65
24.50
6,255
Twitch
1,156.37
7.93
3,011
FB-100
5,902.70
44.05
30,387
Elliptic
539.43
16.19
4,103
Arxiv
4,355.21
4.50
5,505
From the results, we observe that all training times and memory usages are within controllable range, demonstrating that our method is applicable to wide varicty of sccnarios with different types of distribution shifts. Moreover, the training time per epoch is particularly higher for Elliptic, as graphs in this dalaset contain significantly larger numbers of nodes and edges. Nevertheless, the total training time remains reasonable, indicating that our method is scalable to large datasets.
D Detailed Results
In this section, we provide detailed results for the specific test domains on Cora (8 test domains), Photo (8 test domains), FB-100 (3 test. domains), and Twitch (5 test domains). The results are provided in Table 7
Table and Table 9
E Created Datasets with Different Degrees of Distribution Shifts
In this section, we introduce the details the dataset Cora-Mix used in Sec. 4.3 Specifically, we aim to manually control the degree distribution shifts across different domains. However, the original dataset Cora provided in EERM (Wu al. 2022a) creates distribution shifts via the concatenation of comminn-ennsiive features and label-related features, which means the degree of distribution shifts cannot be casily quantified. Therefore, propose to mix up these two types of features with weight to control the distribution shift degree. In particular, we follow the same strategy of generating these features, except that we changed their dimensions to be equal. In this manner, we can perform mix-up on them with a specific weight, i.e., the bias ratio. We also keep the domain split setting of 1/1/8 for training, validation, and test, respectively.
F Limitations
Despite its superior performance, our framework still possesses several limitations. For example, our GRM framework involves the learning of domain information from other nodes or graphs in the domain. However, in practice, the available graphs each domain may not be sufficient. As a result, the performance of our framework may be impacted. In addition, although our GRM framework is validated in both node-level and graph-level tasks, its performance edge-level tasks is not evaluated.
20

---

Published in Transactions on Machine Learning Research (02/2025)
Table The graph OOD generalization results on Cora.
Method
T1
T2
T3
T4
T5
T6
T7
T8
ERM
67.86
65.03
71.25
66.28
70.34
66.72
70.53
67.58
DRNN
76.62
73.63
83.09
56.35
78.18
78.47
79.84
72.49
MMD
75.30
86.23
82.87
52.44
82.02
77.70
80.61
69.05
ARM
62.22
64.25
62.74
62.27
64.11
62.92
60.64
64.40
EERM
70.09
70.99
72.55
71.13
71.03
68.04
71.29
68.88
LISA
76.05
74.39
81.26
71.08
78.56
79.98
81.27
71.15
IS-GIB
75.67
76.41
84.61
74.07
82.90
80.79
82.74
71.32
MARIO
73.50
71.01
80.59
70.84
78.68
80.90
78.94
74.42
GRM
74.92
84.95
89.15
76.83
79.74
85.04
84.63
74.23
Table 8: The graph OOD generalization results on Photo.
Method
T1
T2
T3
T4
T5
T6
T7
T8
ERM
84.35
89.57
89.39
90.14
87.63
90.08
87.34
90.04
DRNN
77.08
77.28
76.71
76.86
77.33
77.33
77.01
77.18
MMD
83.83
82.09
86.26
85.50
83.90
84.98
86.78
84.80
ARM
82.05
69.62
74.03
77.23
86.18
58.33
69.49
79.59
EERM
92.70
92.18
92.20
90.78
92.14
91.46
91.11
91.80
LISA
92.01
92.14
90.88
90.26
91.62
91.09
92.24
91.71
IS-GIB
89.92
92.47
90.38
90.08
93.21
90.24
87.15
88.33
MARIO
91.18
89.24
89.37
88.34
89.81
88.87
88.57
90.10
GRM
93.37
92.53
91.91
93.86
94.33
91.56
91.30
92.42
Table The graph OOD generalization results on FB-100 and Twitch.
Dataset
FB-100
Twitch
Method
T1
T2
T3
T1
T2
T3
T4
T5
ERM
50.48
54.53
53.23
54.20
55.20
50.41
51.58
49.73
DRNN
49.56
56.76
47.98
50.92
53.33
43.98
45.87
46.47
MMD
51.35
57.00
51.58
53.94
54.13
42.81
48.30
46.30
ARM
50.73
56.64
56.11
52.18
49.49
43.24
50.21
47.13
EERM
50.85
56.73
55.39
37.19
55.17
51.61
52.54
53.79
LISA
59.13
48.83
54.73
63.04
57.94
48.58
54.45
55.13
IS-GIB
49.55
53.25
60.87
61.23
57.08
51.69
54.50
55.36
MARIO
50.33
55.73
55.57
59.45
56.81
53.57
50.67
54.98
GRM
51.95
56.11
57.33
61.45
56.82
60.25
52.52
52.64
21


Reviews:
[{"review_id": "EvVMbiSyYb", "reviewer": "Reviewer_pCdy", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n- SOTA performance on widely-used benchmarks.\n- Works for both node-level and graph-level tasks.\n- Provides theoretical justification.\n\n**Weaknesses**\n- Presentation needs to be improved. For example, there are some typos, and the transition from equation (8) to equation (9) is not very straightforward. Omitting the derivation makes comprehension more difficult.\n- It is not intuitive to understand that the domain-related information is related to the 1-hop neighbors of each node.\n- The artificial distribution shift experiments (Cora-Mix) are simple and may not be general enough. Different types of shifts are less explored.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces Generative Risk Minimization (GRM), a novel framework for handling out-of-distribution (OOD) generalization on graphs. The contributions include:\n- Instead of extracting subgraphs, GRM generates invariant subgraphs in a continuous manner.\n- The framework preserves the same set of nodes as the input graph while having continuous edge weights and node representations.\n- A theoretical analysis that ensures the effectiveness of the generated subgraphs.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- It is not intuitive to understand that the domain-related information pertains to the 1-hop neighbors of each node. Could the author include some visualizations (or other illustrations) to validate this hypothesis?\n- Could the author provide the training time for larger graphs (Elliptic&Arxiv) to better reflect the time complexity?\n- For the ablative studies, could the author provide insights on why certain components work better?", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "zxNzlHT7bf", "reviewer": "Reviewer_EFEr", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strength\n- The proposed method is well-motivated.\n- The paper provides rigorous theoretical analysis that effectively validates the tractable objective functions of the GRM framework.\n- The performance of the proposed model is verified by both node-level and graph-level prediction tasks.\n\n### Weakness\n- While generating invariant subgraphs is an appealing idea, the paper lacks theoretical analysis for concept shift scenarios where $P(Y|X)$ changes between train and test distributions while $P(X)$ remains identical. The effectiveness of the proposed method under such concept shifts remains uncertain. The paper would be more solid if it either provides theoretical guarantees for concept shift scenarios or clearly specifies which type of distribution shift (covariate shift vs concept shift) each dataset exhibits.\n- While the influential node extraction method appears reasonable for transductive learning scenarios, the paper lacks clear explanation for handling inductive learning cases like molecular property prediction, where only the input graph is available without access to other domain graphs. Although the authors mention using nodes within input graph $\\mathcal{G}$ to learn domain-specific representations $H$ for graph-level tasks, exploring the use of virtual nodes could be an interesting direction as they might better capture domain-specific information without relying on the existing node set.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "- The authors propose Generative Risk Minimization (GRM), a framework that addresses OOD generalization on graphs. Unlike previous approaches that extract discrete invariant subgraphs, GRM generates invariant subgraphs with continuous edge weights while preserving the original node-set. The framework employs a Variational Graph Auto-Encoder (VGAE) architecture to generate invariant subgraphs. It is evaluated on six node-level and four graph-level real-world datasets covering various distribution shifts (artificial, cross-domain, temporal), consistently outperforming state-of-the-art baselines.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please see my concerns in the weaknesses section in my review.\n\n[Typos]\n\nOn page 2, third paragraph:\n- 'its' should be 'it'\n- 'unaviable' should be 'unavailable'", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "naCi8V5hsN", "reviewer": "Reviewer_E2za", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths** : The approach of leveraging causal variables to avoid the dependence on ground truth invariant graphs and its combination with the regularization loss seems interesting.\n\n**Weaknesses:** \n\n1. The presentation of the paper needs to be improved as it is currently unclear as to how the variational forms of latent variables are actually computed and used. My confusion arises from inconsistent notations.\n2. Details about the computational aspects of the approach is lacking. In particular, details about the VGAE, the time/memory overheads of training with the GRM approach seem to be lacking.\n3. In Appendix D, the GRM approach does not seem to do well on cross-domain OOD generalization with FB-100 and Twitch datasets. Can the authors explain this discrepancy when comparing the results in Table 2 and Table 7?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a 'Generative Risk Minimization' (GRM) framework to improve OOD generalization on graphs. Especially for node and graph classification tasks. For a given graph, the GRM framework generates an invariant sub-graph and uses it for node/graph classification. The loss function (objective) is a combination of a generation objective and an invariance objective, which does not rely on ground truth invariant graphs.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "**Required major changes**\n1. In equation (4), please clarify what $\\text{GNN}_{\\mu}$ denotes. Is it the mean of node features after passing through the GNN?\n2. Please add details about the GNN/VGAE architecture in the main paper as well. Right now, I could only find it in Appendix C The details about the optimizer used for training is also missing.\n3. The complexity analysis in Section 3.4 does not seem to consider the depth of the GNN and VGAE. Additionally, there are no details regarding the choice of data/hidden state dimensions.\n4. Please add a formal definition of $Q(Z)$. Right now, it is hard to understand what $Q(.)$ is. Also, in Section 3.2, the latent variable Z obtained via the VGAE shares the same notation as the actual Z which the authors aim to approximate. This notation inconsistency is confusing. See also the line above Eq (8).\n5. Throughout the paper, the usage of the term 'continuous values of edges' is misleading. As per Eq (6), it seems like the invariant graph $\\hat{G}_c$ is a complete graph and seems like the authors are referring to this structure as 'continuous values of edges'? Please clarify.\n6. The details about the selection of influential nodes is a bit confusing. As per the Figure 2, it seems like the authors clearly distinguish between the nodes belonging to graph G and the influential nodes. However, no such demarcation was made in  Eq (10). Am I missing something? Also, how does this influential node selection vary for graph and node classification tasks? Please clarify.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers acknowledge that the proposed method advances OOD generalization on graphs by generating continuous-weighted invariant subgraphs, supported by rigorous theoretical analysis. Despite some limitations in training efficiency for large graphs and questions about artificial distribution shifts in experiments, the work represents a substantial contribution to the field with its strong theoretical foundations and practical utility. However, there are still some concerns regarding the experimental settings and ablation studies. Thus, the authors should revise the manuscript based on the reviewers' comments.
Decision: Accept (poster)