A Recipe for Causal Graph Regression: Confounding Effects Revisited
Authors: Yujia Yin
Tianyi Qu
Zihao Wang
Yifan Chen
Venue: ICML 2025 poster (2025)
Keywords: graph
Primary area: general_machine_learning->causality

Abstract: Through recognizing causal subgraphs, causal graph learning (CGL) has risen to be a promising approach for improving the generalizability of graph neural networks under out-of-distribution (OOD) scenarios. However, the empirical successes of CGL techniques are mostly exemplified in classification settings, while regression tasks, a more challenging setting in graph learning, are overlooked. We thus devote this work to tackling causal graph regression (CGR); to this end we reshape the processing of confounding effects in existing CGL studies, which mainly deal with classification. Specifically, we reflect on the predictive power of confounders in graph-level regression, and generalize classification-specific causal intervention techniques to regression through a lens of contrastive learning. Extensive experiments on graph OOD benchmarks validate the efficacy of our proposals for CGR. The model implementation and the code are provided on https://github.com/causal-graph/CGR.

A Recipe for Causal Graph Regression: Confounding Effects Revisited
Yujia Yin Tianyi Qu 22 Zihao Wang" Yifan Chen
Abstract
Through recognizing causal subgraphs, causal graph learning (CGL.) has risen to be promis- ing approach for improving the generalizability of graph neural networks under out-of-distributio (OOD) scenarios. However, the empirical suc- cesses of CGL techniques are mostly exemplified in classification settings, while regression tasks, a more challenging setting in graph learning, are overlooked. We thus devote this work tackling causal graph regression (CGR); to this end we shape the processing of confounding effects in ex- isting CGL studies, which mainly deal with clas- sification. Specifically, we reflect on the predic- tive power of confounders in graph-level regres- sion, and generalize classification-spese intervention techniques to regression through a lens of contrastive learning. Extensive experi- ments graph OOD benchmarks validate the effi- cacy of our proposals for CGR. The model imple- mentation and the code are provided on https: / //githtle com/ causal -aallggraphhhhe CGR.
Full Graph
C: Causal Subgraph
Confounding Subgraph Response
Figure f. Structural causal model (SCM) for graph regression.
The core methodology of causal learning involves the identi- fication and differentiation of causal features from confound- ing ones. As shown in Figure 1, causal features C' are those directly deciding responses Y, whereas confounding fea- tures S (shorthand for "spurious") solely present spurious correlations. Therefore, understanding how causal features (as well as confounding features) and responses interact plays central role in practical designing of causal learning methods. From this perspective, causal graph regression (CGR) warrants specialized handling since the interaction between features and responses therein is significantly differ- ent from classification. Furthermore, regression is in general more challenging task than classification, and techniques working for classification, Perceptron (Rosenblatt, 1958) for example, may not apply regression.
causal
1. Introduction
Causal graph learning (CGL) (Lin et al., 2021) holds par- ticular importance due to its relevance in fields such as drug discovery (Qiao et al., 2024) and climate modeling (Zhao et al., 2024). However, previous CGL studies focus on classi- fication settings. Some of them cannot be directly extended to regression tasks, such as property prediction (Rollins al., 2024), traffic flow forecasting (Li et al., 2021), and credit risk scoring (Ma et al., 2024), because the transition from finite to infinite support makes discrete labels unavail- able. Graphs thus cannot be informatively grouped. A systematical understanding of how CGL techniques should be adapted to graph-level regression is still under-explored.
Specifically in CGL, the identification of causal subgraphs is seemingly transferable since this step, explicitly or im- plicitly, relies on the calculation of mutual information and is compatible with both settings (c.f. Section 3.2). How- ever, the empirical performance of this vanilla adaptation regression tasks dwarfed by empirical risk minimiza- tion (Vapnik, 1991, ERM) w.r.t. least squares loss (see the results in Sections 5.3 and 5.4).
To crack CGR, we revisit the processing of confounding ef- fects, which conceptually constitutes causal graph learning along with causal subgraph identification as shown Fig- ure 1. Existing CGL methods, such as CAL (Sui et al., 2022) and DisC (Fan et al., 2022), are built on a strong assumption that confounding subgraphs contain strictly no predictive power. We reflect on this assumption and speculate it is hardly practical due to the contradiction with real-world observations: in molecular property prediction, for example, molecular weight is noncausal to toxicity while does exhibit strong correlations.
Eqqual contribution 'Hong Kong Baptist University RSF Tech *Zhejiang University 'Hong Kong University of Science and Technology. Correspondence to: Tianyi Qu <quuianyi@@ sf- express.com>, Yifan Chen <yifanc@ waw.uneeuhuu......kkk
Proceedings of the 4nnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

A Recipe for Causal Graph Regression: Confounding Effects Revisited
In this work, develop an enhanced graph information bottleneck (GIB) loss function, which no longer takes the strong assumption. Moreover, some confounding effect processing techniques, such as backdoor adjustment (Sui et al., 2022; 2024) and counterfactual reasoning (Guo et al., 2025), heavily rely on discrete label information and cannot be adapted to regression at all. We follow the principle of those methods and generalize it from class separation to instance discrimination; the discrimination principle aligns with the philosophy of contrastive learning (CL) and CL techniques are therefore leveraged to tackle CGR in our proposal.
tations during the training process.
Causal modeling leverages structural causal models (SCMs) to improve the performance of graph neural net- works (GNNs) on out-of -is--iibbttiitiinnnn (OOD) data. These approaches incorporate various traditional causal inference techniques, such as backdoor adjustment (e.g.. CAL (Sui et al., 2022), CAL+ (Sui et al., 2024)), frontdoor adjustment (c.g.. DSE (Wu et al., 2022c)), instrumental variables (e.g.. RCGRL (Gao et al., 2023)), and counterfactual reasoning (e.g.. DisC (Fan et al., 2022)). By simulating causal inter- ventions through supervised training, these methods aim to achieve OOD generalization. However, they often disregard the predictive potential of confounding features, which hin- ders effective disentanglement. Moreover, the supervised loss functions tailored for classification tasks are not easily adaptable to regression problems, as the inherent complexity of regression introduces additional challenges.
Following the intuition, we develop a new framework for causal graph regression, which spotlights the confounding effects within. In summary, our contributions are as follows:
To the best of our knowledge, are the first to explicitly consider the predictive role of confounding features in graph regression tasks, a critical yet overlooked aspect in graph OOD generalization.
Stable learning aims to ensure consistent performance across environments by reweighting samples or balancing covariate distributions. For example, StablcGNN (Fan et al., 2023) employs a regularizer to reduce the influence of confounding variables. However, such methods often rely on heuristic reweighting strategies, which may not fully disentangle causal from confounding factors.
We introduce a causal intervention approach that generates random graph representations by leveraging contrastive learning loss to enhance causal representation, outperforming label-dependent methods.
Extensive experiments on OOD benchmarks demonstrate that our method significantly improves generalization in graph regression tasks.
In addition to graph-based approaches, traditional machine learning methods have also explored causality in regression tasks. For instance, Pleiss et al. (2019) observed that causal features tend to concentrate in a low-dimensional subspace, whereas non-causal features Similarly, Amini et al. (2020) proposed framework for learning continuous targets by placing an evidence prior on Gaussian likelihood function and training non- non-Bayesian neural network to infer the hyperparameters of the evidence distribution. These methods highlight the potential of lever- aging causal insights for improved regression performance.
2. Related Work
more randomly distributed.
-utoo--distribution (OOD) challenges in graph learning has drawn significant attention, particularly in methods aiming to disentangle causal and confounding factors (Ma, 2024). Existing approaches can be broadly categorized into invari- ant learning (Wu et al., 2022a), causal modeling (Sui et al., 2024), and stable learning (Li et al., 2022).
Invariant learning focuses on identifying features that remain stable across different environments, filtering out spurious correlations in the process. While not explicitly grounded in causal reasoning, prior studies (Wang& & Veitch, 2022; Mitrovic et al., 2020) have highlighted its inherent connection to causality. Methods in invariant learning, such as CIGA (Chen et al., 2022), GSAT (Miao et al., 2022), and GALA (Chen et al., 2024), aim to learn invariant repre- sentations by isolating causal components.
3. Preliminaries and Notations
Along this paper, we denote a graph G as (A.X). Here, A E {0, 1 is the adjacency matrix indicating connec- tivity among n nodes (A.j = if nodes and j are con- nected, otherwise 0); X E RRad is the node feature matrix, where each row X; represents the d-dimensional feature vector of node i. The regression task in graph learning is to learn function f G + y, where y E R denotes the response for the graph G.
However, these approaches are typically designed for clas- (OOD)
sification tasks, limiting their generalization capability in regression settings. Post-hoc methods, such as PGExplainer (Luo et al., 2020) and Reg- Explainer (Zhang et al., 2023), attempt to discover invariant subgraphs after training. However, these methods fail to equip the model with the ability to learn invariant represen-
3.1. Causal Graph Learning
In causal graph learning, a graph G can be split into causal subgraph C' and a confounding subgraph S. This process is non-trivial and our proposed paradigm will hinge on the output of this process. We follow the definition in Sui et al.
2

---

A Recipe for Causal Graph Regression: Confounding Effects Revisited
(2022) and first introduce the construction of the causal subgraph C:
quires additional interventions (Miao et al., 2022; Chen
et 2022).
Formally, the GIB objective is expressed as:
=== (Medge O Made X),
(1)
where the mask matrix Medge [0, _| and the diagonal matrix Mooe (whose diagonal elements are in [0, 1) will filter out the non-causal nodes and edges. The confounding subgraph is then the "complement": S == G C.
-I(C
(3)
where I(O;Y) quantifies the predictive information re- tained by C' (and thus needs to maximize). I [[; G)) serves as a regularizer to exclude irrelevant details from the origi- nal graph; the parameter a controls the trade-off between information preservation and compression.
In our framework, these masks Madge and Mmote are not pre- defined. Instead, they are learnable soft masks, generated by MLPs conditioned on the representations of G. The param- eters of these MLPs are optimized end-to-end as part of the overall model training, enabling the model to autonomously learn how to construct C' and S. Further architectural details are provided in Appendix B and illustrated in Figure 2.
3.3. Causal Intervention in GNNs
We borrow the structural causal model (SCM) diagram in Figure to illustrate the causal intervention techniques. As shown in Figure 1, the graph G decides both the causal subgraph C and the confounding subgraph S, and the former C affects the prediction of response Y. In more detail,
Notably, mutual information plays an essential role in CGL, and we introduce its caleulation, exemplified by the mutual information between the hidden embedding vectors (learned by graph neural network) of the causal subgraphs and the original graphs, as follows:
C G + S: Graph data G encodes both C, which directly impacts Y, and S, which introduces spurious correlations.
C The causal feature C has the potential to predict Y not only directly but also indirectly through its influence along this backdoor path S - - Y.
IC; G) == Ec [Co G)/p(C)),
(2)
where we follow the convention in CGL literature and abuse the notation C, G to represent a random variable following the underlying distribution of embedding pairs Hg.'s 's and He,''. 's. In particular, those hidden embeddings are assumed Gaussian and the joint distribution can thus be well-estimated by sample embedding pairs. We refer read- ers interested to Miao et al. (2022, Appendix A) for more details. Moreover, the conputation/approximattoo of the mutual information terms is a crucial component in causal graph learning, while still under-explored for CGR; we will dissect the computation of our proposed terms in Section 4.2 through deriving the variational bounds.
In causal inference, confounder S incurs spurious correla- tions, preventing the discovery of underlying causality. To address this issue, backdoor adjustment methods focus on the interventional effect P(Y]do(C) and suggest to esti- mate it by stratifying over S and calculating the conditional distribution P(Y|C, S) (Pearl, 2014; Sui et al., 2024).
4. Revisiting Confounding Effects for CGR
In this section, we present causal graph regression paradigm that integrates an enhanced graph information bottleneck (GIB) objective with causal discovery, reshaping the processing of confounding effects in CGL.
3.2. Graph Information Bottleneck
The information bottleneck (Tishby et al., 2000; Tishby & Zaslavsky, 2015, IB) principle aims to balance the trade-off between preserving the information necessary for predic- tion and discarding irrelevant redundancy. Specifically, IB suggests to maximize I (Z;Y) while minimizing I(Z; X) for regular data compression, where Z is the compressed representation, X is the input, and Y is the response.
4.1. Overview
We first provide an overview of how graph inputs are turned into regression outputs. As shown in Figure 2, we follow the framework of Sui et al. (2024) and first encode graph embeddings Hg. 's using a GNN-based encoder. Attention modules are then adopted to generate soft masks for extract- ing causal and confounding subgraphs (c.f. Equation (1)). These subgraphs are processed through two GNN modules (Ge and G.) with shared parameters to extract causal (He,i 's) and confounding (H..'' 's) representations, which are passed through distinct readout layers for regression.
Graph information bottleneck (GIB) (Wu et al., 2020) ex- tends the IB principle graph-structured data, facilitating the identification of subgraphs that are most relevant for predicting graph-level responses. By minimizing the mutual information I (C; G)) between the extracted causal subgraph C' and the original graph G, GIB reduces redundant informa- tion. However, GIB alone does not guarantee the extraction of a purely causal subgraph, as isolating causal effects re-
The optimization features an enhanced graph information bottleneck (GIB) loss LGIB, comprising the causal part Le and the confounding part L.,, to disentangle causal signals
3

---

A Recipe for Causal Graph Regression: Confounding Effects Revisited
Causal subgraahh
Positive Pair
H
gc
Negative Pains
Random Addition
Readout
Lc
H
Contrastive Leaming
GNN Encoder
H
H
Attention
H m
La
Mini-batch graphs
HAA
s.
Readout
gs
H
Confounding subgruphs
Figure 2. Given a mini-batch of graphs, (1) the GNN encoder computes the graph embeddings Hy, and an attention layer generates soft masks extract causal and confounding subgraphs. GNN Ge processes the causal subgraph generates its representation He, and employs readout to predict responses; it is optimized with causal subgraph loss T.c. (3) GNN G.. sharing parameters with Ge. processes the confounding subgraph S. generates H., applies readout prediction; is optimized with confounding subgraph loss L,, (4) For causal intervention, contrastive learning guides the process. Given graph Hy.. the positive sample is mixed graph Hmix.sy from random addition, while any other graph Hy serves as negative sample. causal intervention loss Leı used accordingly.
(c.f. Section 4.2). Also, counterfactual samples (Hmix,y) are generated by randomly injecting confounding repre- sentations into causal ones; unsupervised learning then performed, guided by lerraninve-lerringg.. based causal in- tervention loss Lei (c.f. Section 4.3). More implementation details of the overall framework are deferred Appendix B.
sponse Y. In its current form, the GIB framework overlooks this aspect, causing the model to allocate all Y-relevant information to C' and potentially lose meaningful content.
This limitation leads incomplete causal disentangle- ment, which impacts the generalization of models out- of-distribution (OOD) settings. To overcome this issue, propose an enhanced GIB loss function that takes the predic- tive roles of both C and into consideration. By introduc- ing mutual information terms on S during optimization, we avoid overburdening C with all relevant information, and consequently enable more precise disentanglement.
4.2. Enhanced GIB Objective
CGL adopts the GIB objective extract subgraphs that retain essential predictive information while excluding dundant components (Zhang et al., 2023), which aligns with the disentanglement of causal subgraph and confounding subgraph S' in CGL. Original GIB assumes the confounding subgraph pure noise and cannot predict the response Y (Chen et al., 2022), while as we discussed in Section may still contain information that is predictive of the
Overall, our enhanced GIB objective is defined follows:
-ICCY)
BI(SYY),
(4)
which formally extends the original GIB objective by in- troducing a confounder-related term I(S; Y) to capture the

---

A Recipe for Causal Graph Regression: Confounding Effects Revisited
predictive capacity of S, along with parameter 3. In par- ticular, we intentionally exclude the T(S; G) term because, in the SCM diagram of Figure 1, S primarily introduces shortcut rather than directly encoding causality; overly im- posing structural regularization on S can disrupt disentan- glement and lead to suboptimal separation between C' and S. Notably, the conceptual objective (4) is incomputable in practice. We devote the remainder of this subsection to the practical computation of Equation (4) for CGR.
The computation of H(Y C) is supposed to hinge on the hidden embeddings Ha, 's produced by a GNN Ge (see Section 4.1); we model the conditional distribution p(Y He) as a Gaussian distribution:
p(Y Ho) M())
(1I)
where A(e) and (e)) represent the scalar conditional mean and variance of Y (estimated by networks) given a causal
subgraph representation He. The probability density func- tion for this Gaussian is:
Variational bounds for approximating G). The mu- tual information (C; G) is mathematically defined based
(Y (Y-HH))) H(O)) 20
on the marginal distributin p(C) = Ea p(C)G) Since p(C) is intractable, a variational distribution g(C) introduced and induces an upper bound:
P( Ho) =
2n02 exp
(12)
Substituting Equation (12) into Equation (10), we can fur- ther approximate H(Y C*) through empirical data:
I(C; G) S Ep(c) [pC G)]l[(O)]].
(5)
To efficiently compute the KI. divergence in Equation (5), we follow the literature (Chechik et al., 2003; Kingma et al., 2013) and assume that p(C G) and ((C)) are multivariate Gaussian distributions:
(Y MMM,)) 20
+ (0og(2n0(o..))
(13)
where N represents sample size, Y, is the target response for the i-th sample. and p(e),, and o0), the corresponding mean and variance of Y given He.
p(C G) = =N(us(G)) EG()), ((C)= -V I), (6)
where Pa(G) and (G) are the mean vector and covari- ance matrix estimated by GNNs. To simplify computa- tion and stabilize training, we further assume [G) is an identity matrix, removing the need to learn covariance pa- rameters. This simplification is not only practical but also theoretically justified, as any full-rank covariance can be whitened without loss of generality (Chechik et al., 2003, Appendix A). (p(C G)I)((C) then reduces to:
If a constant conditional variance (i.c., oee a choice adopted for stability and aligning with approaches in (Nix & Weigend, 1994; Yu et al., 2024), then [(C; Y) (or, equivalently, - H(Y| C)) reduces to the least squares loss:
(ee) = 1) is assumed,
1
N
(Yr - P((..))
1
(000(2
+ 2
N
20 (o)
1=1
1
x
Σ
(Y, M()),)
(14)
(r(Ep(G)) + (1x)))+(((()))) [Q))2 d - log det (G)
(7)
which turns to the causal subgraph objective LCP-
2 (C))
Similarly, the mutual information 1(S; can induce the confounding subgraph objective
where d is the dimensionality of C. Further substituting Equation (7) into Equation (5), we obtain an upper bound for G(;
N N -YG-MON)
1
Lsp x
(15)
(C; 1 F
(8)
Empirically, we employ two independent readout layers to compute the causal and confounding subgraph mean M(cj., 's and ((s).s's.
which serves as an easy-to cosympo- proxy for I(C;
Variational bounds for approximating T(C;Y), ((SY).
In summary, our enhanced GIB objective can be decom- posed into two distinct loss components: the causal sub- graph loss Lo(G, Lc(G, confounding subgraph loss Y, = -I( The complete enhanced GIB objective we propose is:
We first recall (C;; Y) mathematically reads:
((; = H(Y) - HY (C),
9
-I(CYY) + aI(C; G') and the
where H() denotes the entropy of Y, representing the overall uncertainty in the target variable. Since !(Y) re- mains constant, maximizing [C;;; Y) reduces to minimizing the conditional entropy HY C), given by:
LGB L. BL.
= =1((C; + (l BI(s))),
H(Y C)= E (lgg(Y [C]-
(10)
and in practice we use - Ler + (Bp())) (I/o -BTsp.
5

---

A Recipe for Causal Graph Regression: Confounding Effects Revisited
4.3. Causal Intervention
where B is the batch size, Hmix.ss is the representation of the mixed graph combining the i-th causal subgraph and the j-th confounding subgraph, and Hg is the representation of the original graph.
To further strengthen causal learning in CGR, we introduce causal intervention loss and reshape the processing of con- founding effects therein. In general, our approach injects randomness at the graph level by randomly pairing con- founding subgraphs with target causal subgraphs from the entire dataset. By generating counterfactual graph represen- tations through the random combination of these subgraphs, we effectively implement causal intervention.
Remark 4.1.
The ultimate loss used in our paradigm is a simple combination of the GIB objective and the causal intervention loss: L = LGIB + XLCI.
5. Experiments
This strategy can be understood as implicit realization of backdoor adjustment (Pearl, 2014) in the representation space. In existing research on graph classification tasks (Fan et al., 2022; Sui et al., 2024), causal intervention is typi- cally modeled by predicting P(YIC, through intervened graphs, adjusting for causal effects by comparing predictive distributions under different confounding conditions. How- ever, in regression tasks, Y is a continuous variable, and directly modeling P(YIC,S) S) becomes significantly more challenging. To overcome this, we follow the spirit of con- trastive learning to get rid of the reliance on explicit labels.
In this section, we evaluate the prediction performance and OOD generalization ability of our method. We comprehen- sively compare our method with existing models to demon- strate the superior generalization ability of our method on regression tasks. We briefly introduce the dataset, baselines, and experimental settings here.
5.1. Datsets
GOOD-ZINC. GOOD-ZINC is a regression task in the GOOD benchmark (Gui et al., 2022), which aims to test the out-of-distribution performance of real-world molecu- lar property regression datasets from the ZINC database (r6mez-Bombarelle et al., 2018). The input is a molecular graph containing up to 38 heavy atoms, and the task is to predict the restricted solubility of the molecule (Jin et al., 2018; Kusner et al., 2017). GOOD-ZINC includes four specific OOD types: Scaffold- Scaffold-Covariate Scaffold Connepp,, Siva-Cova and Size-Concept. Scaffold OOD involves changes in molecular structures, while Size OOD varies graph size. Each can manifest as Covariate Shift (P(X) changes, PXY remains stable) or Concept Shift (spuri- ous correlations in training break in testing).
In more detail, following Sui et al. (2022), we use a random addition method to pair the confounding subgraph with the target causal subgraph, which gives mix:
(16) Comparing the predictions of mixx with the original graph's labels, as shown in Sui et al. (2022), can inadvertently force the mixed graph to discard all confounding effects, thereby nullifying the intended causal disentanglement.
Hmix,jj He,i + Haj-
To mitigate this issue, we suggest learning causal represen- tations through contrastive learning. Specifically, the causal subgraph, when combined with different confounding sub- graphs, consistently produces mixed graph representations that are aligned with the original graph representation. This formulation enables the model to learn causal subgraphs that are invariant across varying confounders, and to avoid the causal subgraphs boiled down to non- con-informative ones.
ReactionOOD-SOOD. In addition to the GOOD bench- mark, we also used three S-OOD datasets in the Reac- tionOOD benchmark (Wang al., 2023), namely Cycload- dition (Stuyver et al., 2023), E2&Sx2 (von Rudorff et al., 2020), and RDB7 (Spiekermann et al., 2022), which are designed to extract information outside the structural dis- tribution during molecular reactions. Cycloaddition and RDB7 have two domains: Total Atom Number (where the total number of atoms in a reaction exceeds the training range) and First Reactant Scaffold (where the first reactant has a new molecular scaffold unseen in training), while S2&Sx2 dataset contains reactions with molecules whose scaffold cannot be properly defined, which prevents the scaffold from being an applicable domain index for this dataset. The definitions of two shifts Covariate and Concept in ReactionOOD are consistent with those in GOOD.
To achieve this, we propose causal intervention loss guided by contrastive learning. Specifically, the method aligns the representation of the original graph with that of its cor- responding random mixture graph, while simultancously ensuring that representations of unrelated graphs remain distinet. In implementation, draw inspiration from the In- foNCE loss (Oord et al., 2018), we treat Hy and Hmix from the same causal subgraph as positive pairs, and H, with representations of other graphs within the batch as negative pairs. Formally, the mixed graph contrastive loss is defined as:
B Σ log
exp(sim (xp(sim(HHgss (gg, Hmix.as)
5.2. Baselines and Setup
Lei- =
B
ΣB 1 exp(sim(HH,,) H
As our framework is general and aims to address distribu- tion shifts, we compare it against several baseline methods.
1=1
(17)
6

---

A Recipe for Causal Graph Regression: Confounding Effects Revisited
Table GOD generalization performance on GOOD ZIND- dataset, with boldface being the best and underline being the runner-up.
SCARFOLD
SIZE
GOOD-ZINC
COVARIATE
CONCEPT
COVARIATE
CONCUPT
0OD
OOD
ID
0OD
ID
OOD
ERM
0.1188+0.000 0030
061660=
0093
0.1 0013
0.1248=0.0018
0.1222+0.006]
0.2331+0.0166
0.1304+0.00 2004+0000000
0.1406=0. 1400=00.002
IRM
0.1258+0.0033
0.2313+0.00
02243
0.1245:0.000
0066
0.1217+0.001:
0.5840:0.000
0.1331+0
0.133820.001 0.445
VREX
0.0978+0.000 16
0.1561#0 0021
0.1928=0.000 0021
0.1271=0.0022
0.1841+0.0009
0.2276+0.000 000+
0.1206#0. 0008
1289=00.003
MIXUP
0.1348*0.0003 0025
0.2157=0. 0009
0.1 0.1192=0. 0026
0.129620.0044
0.1431+0.000 00000000
0.2573+0.000 0042
0.1625#0 0121
0.
0.1660=0 0063
DANN
0.1152+0.00 0021
0.173440 0005
0.1284=0. 0031
0.1289:0 0020
0.1053+0.0081
0.2254+0.01 0144
0,1227=0. 0008
0.127 =0.0039
CORAL
0.1252+6.0004
0.173440. 07334
0.117330. D029
0.1260=0.002
0.1 0.1164+0.0004
0.2243+0.0147
0.1246#0. 0062
0.1270=0 0020
CIGA
0.1568+0, 0034
0.2986+0 0041
0.1926=0. 20
0.2415=0.011
0.1500+0.000
0.6102+0 0148
0.3560=0 01
0.3240=0. 0451
DIR
0.2483+0.0055
0.3650=0. 0032
0.2510=0. 0001
0.261990 0076
0.25 050.0059
0.4224+0.0673
0.4831+0. 0823
0.3630=0.00 0872
GSAT
0,0890+0,00 003
0. 0.144990.o0 0043
0.0928=0. 0029
0.0999=0. 0029
0.0876+0.0032
0.2112+0.00 0033
1002+00 0013
001043=0 0001
OURS
0.0514+0.0061
0.10 1046:0000007
0.065920.0041
0.0518=0.0007
0.0466+0.0034
0.1484+0.0033
0.0577=0.0008
0.058020.0016
Empirical Risk Minimization (ERM) (Vapnik, 1991) serves as a non-OOD baseline for comparison with OOD methods. We consider both Euclidean and graph-based state-of-the- art OOD approaches: (1) Euclidean OOD methods include IRM (Arjovsky et al., 2019), VREx (Krueger et al., 2021), GroupDRO (Sagawa et al., 2019), DANN (Ganin et al., 2016), Coral (Sun & Saenko, 2016), and Mixup (Zhang, 2017); (2) Graph OOD methods include CIGA (Chen et al., 2022), GSAT (Miao et al., 2022), and DIR (Wu et al., 2022b).
error compared to GSAT. Similarly, under the Concept shift,
our approach yields an MAE of 0.0577+0.0000 (ID) and 0.0580#0.000 0004 (OOD), improving upon GSAT by 42.4% and 44.4%, respectively.
In addition to achieving lower MAE values, method exhibits significantly reduced variances compared to other approaches, highlighting its stability under diverse condi- tions. These findings confirm the strong generalization ca- pability of our method across different domains and types of distributional shifts.
For a fair comparison, all methods are implemented with consistent architectures and hyperparameters, ensuring that performance differences arise solely from the method itself. To provide reliable results, each experiment is repeated three times with different random seeds, and we report the mean and standard error of the results. Detailed settings and hy- perparameter configurations are described in Appendix A.4.
5.4. Results of ReactionOOD
Table and Table 3 highlight the robust generalization abil- ity of our method across multiple datasets and evaluation settings, as measured by RMSE. Our method achieves the best OOD performance in 6 out of 10 cases and ranks sec- ond in 2 cases. Notably, in cases where another method outperforms ours, the performance gap is within small margin.
5.3. Results of GOOD
As shown in Table 1, our proposed method achieves SOTA performance on GOOD-ZINC, consistently outperforming all baseline methods across both domains (Scaffold and Size) and under different distribution shifts (Covariate and Concept). Specifically, in terms of Mean Absolute Error (MAE), our method demonstrates significant improvements in both in-distribution (ID) and out-of-distributie (OOD) settings.
For instance, in the Cycloaddition dataset, under the to- tal atom number domain with a concept shift, Our method achieves an OOD RMSE of 5.53 = 0.12, outperforming all baseline methods. While some non-causal baselines (e.g.. Coral in this specific setting, achieving an ID RMSE of 4.10 + 0.05 versus our 4.41 = 0.22) might get better ID perfor- mance by exploiting spurious but predictive features, such approaches can become less reliable under OOD conditions (c.g., Coral's OOD RMSE degrades to 5.74 + 0.04). In con- trast, our method's s focus on identifying and removing these spurious features contributes to its stable and superior OOD performance. Even in other Cycloaddition cases where ours ranks second, such as the same domain with a covariate shift, the OOD RMSE (4.42 = 0.24) is only 0,06 away from the best-perf method (4.36 + 0.15).
For instance, in the Scaffold domain under the Covariate shift, our method achieves an MAE of 0.05 105114000061 (ID) and 0. 104620.0007 (OOD), outperforming GSAT, the next- best method, by 42.2% in ID and 26.3% in OOD perfor- mance. Similarly, under the Concept shift, our method achieves 0.0659+0.004 (ID) and 0,05 115118000000 (OOD), representing improvements of 29.0% and 48.1 %, respec- tively, over GSAT. In the Size domain, our method also achieves remarkable results. Under the Covariate shift, it achieves an MAE of 0.0466+00000 .034 (ID) and 0. 1484=0.0003 (OOD), which translate to 46.8% lower ID error and 29.7% lower OOD
In RDB7, a smaller dataset within the ReactionOOD where causal inference can be more difficult, our method achieves the lowest OOD RMSE (15.73 0.37) under the concept shift. Our method's 's principled focus on true causal features, which leads to better OOD generalization ability and stabil-
7

---

A Recipe for Causal Graph Regression: Confounding Effects Revisited
Table 2. OOD generalization performance on Cycloaddition and RDB7 dataset.
FIRST REACTANT SCAFFOLD
TOTAL ATOM NUMBER
DATASET
METHODS
COVARIATE
CONCEPT
COVARIATE
CONCEPT
ID
ID
OOD
ID
OOD
ID
OOD
ERM
4,38+0 04
4. 4.0000.88
4.79=0. 03
5.60=0. 02
3.77=0.01
4.36+0.15
4.22 04
5.69+0.03
IRM
15.3040.05
21.1620.01
17.55a0.03
18.6420.25
17.53=0.17
17.44 14
23.14a0.02
22.56:0.01
VREX
5.54+0.02
6.69+0.48
5.02=0.05
6. 6.1440.09
4.79+0.03
5.22+0.06
4.92+0.14
6.39=0.04
MIXUP
4.5120.04
5.24+0.83
4.90:0.01
5.9010.00 05
3.90:0.13
4.5340.03
4.1120.09
5.9320.13
DANN
4.42+0.03
4.68=0.12
4.81=0.01
5.75=0.06
3.87=0.05
4.6540.10
4.1820.02
5.6820.10
CYCLOADDITION
CORAL
4.3620.07
4.95=0.30
4.82=0.03
5.72+0.16
4.39:0.59
5.05+0.48
4.1020.05
5.7420.04
CIGA
5.26+0.04
5.67=0.04
5.30=0.29
5.64
4.93=0.05
6.62+1.09
5.03#0.09
6,21
DIR
4.94+0.02
5.31=0.79
5.85=0.20
6.30+0.38
5.52+0.03
6.86+0.05
5.21+0.12
7.09=0.03
GSAT
4.42+0.05
4.63+0.05
4.87=0.01
5 5.69+0.01
3.81#0.01
4.56+0.01
4.12=0.06 04
5.64#0.11
QURS
4.57a0.13
4.22+0.09
4.5320.04
5.37
4.06+0.01
4.4240.24
4.41+0.22
5.5320.12
ERM
10.28+0.05
22. 02.95+0.90
11,38+0,00 08
14.81=0.05
10,86=0.00 86+0.01
7.66+0.55
11.28=0.15
15. 15,79+0.22 24
IRM
$9.87=0.02
76.5 16.5120.46
65.72=0.1 13
63.03#0.15
63.55=0.02
69 00.06600.7
81.14=0.02
46, 16.84400om 42
VREX
16.62+0.1
21.89+0.02
14.62+0.04
18,28+0.09
14.60=0.01
13.84=0.07
34.66+1.56
32.59+3.22
MIXUP
10.7640.00 07
23.4910.09
11.8920.05
15.6420.10
11.13=0.02
10.7840. 17
11.6610.04
17.21 10.28
DANN
10,28+0.015
23.54+0.07
11.28+0.01
14.93+0.05
10.77=0.22
8.29&0.10
11.34+0.05
16,28 15
RDB7
CORAL
10,30=0.1 12
22.1940.63
11.1240.03
14.81=0.06
10.61=0.01
8. 8.0411.14
11.33+0.08
16. 08
CIGA
14.97:0.75
30,0810.84
18.6811.94
1..3
16.4810.69
19.12:1185
20.58:1.54
18.5341.30
DIR
14.34=0.66
26.99:0.49
17.13#1.76
10.18:1.86
14.03=2.06
15.01=0.98
13.52=0.51
16.60=1.99
GSAT
10.52#0.04
23.45+0.11 11
11.26#0.22
14.85+0.12
10. 80.000000
8. 6.66+0.10
11.58+0.03
16. 41
OURS
10.12+0.08
23.1
11.26=0.02
14.94=0.25
10.51=0.08
6.84+0.32
11.46+0.06
15.73=0.37
results, evaluated in terms of accuracy, are reported on the OOD dataset, as shown in Figure 3. The ablation study on Lem aims to examine our hypothesis that confounders pos- sess certain predictive power; thus, this experiment excludes the causal intervention loss LCι Conversely, the ablation study on LCi evaluates whether the contrastive learning- driven causal intervention loss can independently achieve strong OOD performance. Therefore, in this experiment,
Table 3. OOD generalization performance on E2&Sy2 dataset.
COVARIATE
CONCEPT
METHODS
ID
0OD
ID
OOD
ERM
4.45+0.04
5.47+0.27
4.87+0.02
5.04+0.02
IRM
11.61=0.18
1.77
20.95=0.02
17.57=0.03
VREX
4.5820.02
5.4820.13
10.75#1.54
8.7742.31
MIXUP
4.55+0.09
5.55+0.01
4.6920.08
5.1120.01
DANK
4.5120.06
5.3820.04
4.4820.10
5.0420.02
CORAL
4.44+0.11
5.68+0.20
4.54+0.02
4.97+0.07
CIGA
5,05+0.35
6.57+0.52
4.65+0.26
5.39+0.47
we do not incorporate the predictive power of confounding factors.
DIR
5.61+0.26
6.5920.31
6.5620.34
6.2920.11
GSAT
4.55=0.01
5.6920.05
4.55+0.09
5.0420.03
OURS
4.4020.03
4.8320.10
4.5320.12
5.0320.09
Predictive power of confounding subgraphs. The left panel compares minimizing confounding subgraph predic- tion alone versus introducing constraints to model their pre- dictive ability. The results show that ignore the predictive role of confounding subgraphs leads to incomplete disen- tanglement and weaker OOD generalization, demonstrating that accounting for their influence is crucial.
ity. Even though causal methods generally face challenges in smaller datasets (Guo et al., 2020), our approach consis- tently outperforms other listed causal intervention baselines such as CIGA in all RDB7 settings. In the E2&Sx2 dataset, our method delivers the best OOD RMSE (4.83 + 0.10) under the covariate shift and achieves highly competitive results under the concept shift (5.03 0.09).
As noted in OOD-GNN (Tajwar et al., 2021), no method consistently performs best on every dataset due to varying distribution shifts and inductive biases. Our approach, de- signed under more general and weaker assumptions which do not assume that spurious features are non-predictive, aims to tackle a wider range of real-world distribution shifts.
Effectiveness of contrastive learning. The right panel compares using predictions from randomly generated coun- terfactual graphs as causal intervention loss versus our pro- posed contrastive learning loss. The results show that our contrastive learning approach, initially validated in regres- sion tasks, is equally effective in classification tasks, high- lighting its general applicability.
5.5. Effectiveness of OURS in Classification Task
These studies confirm the importance of explicitly modeling confounding subgraphs and the robustness of our contrastive learning loss for OOD generalization. More experimental results are provided in the Appendix A.5.
To validate the generality and effectiveness of our proposed losses, LGIB and LCI. we conduct ablation studies on the GOOD-Motif dataset under the size domain setting. The
8

---

A Recipe for Causal Graph Regression: Confounding Effects Revisited
Confounding Ignored
Supervised Intervention Contrastive Intervention
Confounding Predictive
Test Acc.
Epoch
Epoch
Figure 3. Ablation study on confounder predictive power (left) and causal intervention methods (right) for OOD generalization on GOOD-Motif.
6. Conclusion
References
In this work, we propose recipe for causal graph regres- sion through reshaping the processing of confounding ef- fects in existing CGL classificationsspccff. techniques. In particular, we develop an enhanced graph information bot- tleneck (GIB) loss function which highlights the impact of confounding effects and consequently benefits the recogni- tion of causal subgraphs. Moreover, we revisit the causal intervention technique, which randomly combines causal subgraphs and confounder from the same class (label) to eliminate confounding effects. Adapting this technique to regression requires removal of label information; to this end, we analyze the principle of causal intervention and propose to connect it with unsupervised contrastive learning loss. Ex- perimental results on graph OOD benchmarks demonstrate the effectiveness of proposed techniques in improving the generalizability of graph regression models.
Amini, A., Schwarting, W., Soleimany, A., and Rus, D. Deep evidential regression. Advances neural informa- tion processing systems, 33:14927-14937, 2020.
Arjovsky, M., Bottou, L., Gulrajani, I., and Lopez- Paz, D. Invariant risk minimization. arXiv preprint arXiv: WrXiv:1907002890 2019.
Chechik, G., Globerson, A., Tishby, N., and Weiss, Y. In- formation bottleneck for gaussian variables. Advances in Neural Information Processing Systems, 16, 2003.
Chen, Y., Zhang, Y., Bian, Y., Yang, H., Kaili, M., Xie, B., Liu, T., Han, B., and Cheng, J. Learning causally invari- ant representations for cot-of-disttibution generalization on graphs. Advances in Neural Information Processing Systems, 55:22131-22148 2022.
Acknowledgements
Chen, Y., Bian, Y., Zhou, K., Xie, B., Han, B., and Cheng, J. Does invariant graph learning via environment augmenta- tion learn invariance? Advances in Neural Information Processing Systems, 36, 2024.
We sincerely thank the Area Chair and the anonymous reviewers for their valuable feedback and constructive suggestions, which helped improve this work. The thors acknowledge funding from Research Grants Coun- cil (RGC) under grant 22303424 and GuangDong Ba- sic and Applied Basic Research Foundation under grant 2025A151501025
Fan, S., Wang, X., Mo. Y., Shi, C., and Tang. J. Debiasing graph neural networks via learning disentangled causal substructure. Advances in Neural Information Processing Systems, 35:24934 229444 2022.
Impact Statement
Fan, S.. Wang, X., Shi, C., Cui, P., and Wang, B. Generaliz- ing graph neural networks on out-of-distribution graphs. IEEE Transactions on Pattern Analysis and Machine In- telligence, 2023.
This paper presents work whose goal advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle. H., Laviolette, F., March, M., and Lempitsky, V. Domain- adversarial training of neural networks. Journal of ma- chine learning research, 17(59):1-35, 2016.

---

A Recipe for Causal Graph Regression: Confounding Effects Revisited
Gao, H., J., Qiang, W., Si, L., Xu, B., Zheng, C., and Sun, F. Robust causal graph representation learning against confounding effects. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, volume 37, pp. 7624-7632, 2023.
Luo, D., Cheng, W., Xu, D., Yu, W., Zong, B., Chen, H., and Zhang, X. Parameterized explainer for graph neural network. Advances in neural information processing 2020.
systems,
Ma, F., Li, H., and Iyyas, M. Utilizing reinforcement learn- ing and causal graph networks to address the intricate dynamics in financial risk prediction. International Jour- nal of Information Technologies and Systems Approach (IJITSA), 17(1):1-19, 2024.
Gómez- Banaz- R., Wei, J. N., Duvenaud, D., -errandzz--obbtoo J. M., Sánchez- Lennelii- B., She- berla, D., Aguilera-Iparragurne.. J., Hirzel, T. D., Adams, R. P., and Aspuru-Guzik, A. Automatic chemical de- sign using data-driven continuous representation of molecules, ACS central science, 4(2):268-277 2018.
Ma, J. A survey of out-of-distribution generalization for graph machine learning from causal view. arXiv preprint arXiv:2409.09883 2024.
Gui, S., Li, X., Wang, L., and Ji, S. Good: A graph out-of- distribution benchmark. Advances Neural Information Processing Systems, 35:2059 2-705992077, 2022.
Miao, S., Liu, M., and Li, P. Interpretable and generaliz- able graph learning via stochastic attention mechanism. In International Conference on Machine Learning, pp- 15524-15543. PMLR, 2022.
Guo, R., Cheng, L., Li, J., Hahn, P. R., and Liu, H. A survey of learning causality with data: Problems and methods. ACM Computing Surveys (CSUR), 53(4):1-37, 2020.
Mitrovic, J., McWilliams, B., Walker, J., Buesing, I.., and Blundell, C. Representation learning via invariant causal mechanisms. arXiv preprint arXiv:2010.07922, 2020.
Guo, Z., Wu, Z., Xiao, T., Aggarwal, C., Liu, H., and Wang, S. Counterfactual learning on graphs: A survey, Machine Intelligence Research, 22(1):17-59, 2025.
Nix, D. and Weigend, A. Learning local error bars for nonlinear regression. Advances in neural information processing systems, 7, 1994.
Jin, W., Barzilay, R., and Jaakkola, T. Junction tree vari-
ational autoencoder for molecular graph generation. In International conference machine learning, pp. 2323- 2332. PMLR, 2018
Oord, v. d., Li, Y., and Vinyals, O. Representation learn- ing with contrastive predictive coding. arXiv preprint arXiv:1807.03748 2018.
Kingma, D. P., Welling, M., et al. Auto-encoding variational bayes, 2013.
Pearl, J. Interpretation and identification of causal mediation.
Krueger, D., Caballero, E., Jacobsen, J.-H. Zhang, A., Bi- nas, J., Zhang, D., Le Priol, R., and Courville, A. Out-of- distribution generalization via risk extrapolation (rex). In International conference on machine learning, pp. 5815- 5826. PMLR, 2021.
Psychological methods, 19(4):459, 2014.
Pleiss, G., Souza, A., Kim, J., Li, B., and Weinberger, K. Q. Neural network out-of-distribution detection for regres- sion tasks. 2019.
Kusner, M. J., Paige, B., and Hernández-Lobato, M.
Qiao, G., Wang, G., and Li, Y. Causal enhanced drug- target interaction prediction based graph generation and multi-source information fusion. Bioinformatics, 40 (10):btae570, 2024.
Grammar variational autoencoder. International con- ference on machine learning, pp. 1945-1954, PMLR, 2017.
Li, G., Knoop, V. L., and Van Lint, H. Multistep traffic fore- casting by dynamic graph convolution: Interpretations of real-time spatial correlations. Transportation Research Part C: Emerging Technologies, 128: 103185, 2021.
Rollins, Z. A., Cheng, A. C., and Metwally, E. Molprop: Molecular property prediction with multimodal language and graph fusion. Journal of Cheminformatics, 16(1):56, 2024.
Li, H., Wang, X., Zhang, Z., and Zhu, W. Ood-gnn: Out- of-distribution generalized graph neural network. IEEE Transactions on Knowledge and Data Engineering, 35 (7):7328- -348,,73,2200 2022.
Rosenblatt, F. The perceptron: probabilistic model for information storage and organization in the brain. Psy- chological review, 65(6))386, 1958.
Sagawa, S., Koh, P. W., Hashimoto, T. B., and Liang, P Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case gener- alization. arXiv preprint 19110/// 2019.
Lin, W. Lan, H., and Li, B. Generative causal explanations for graph neural networks. In International Conference on Machine Learning, pp. 6666-6679 PMLR, 2021.
10

---

A Recipe for Causal Graph Regression: Confounding Effects Revisited
Spiekermann, K., Pattanaik, L., and Green, W. H. High accuracy barrier heights, enthalpies, and rate coefficients for chemical reactions. Scientific Data, 9(1):417, 2022.
Wu, T., Ren, H., Li, P., and Leskovec, Graph information bottleneck. Advances in Neural Information Processing Systems, 33:20437-20448, 2020.
Stuyver T., Jorner, K., and Coley, C. W. Reaction profiles for quantum ceimiistry-oopuuttrr [3+ 2] cycloaddition reactions. Scientific Data, 1061) 2023.
Wu, Y.-X,, Wang. X., Zhang. A., He, X., and Chua, T.- S. Discovering invariant rationales for graph neural net- works. arXiv preprint arXiv:2201. 22221.227722 2022b.
Sui, Y., Wang, X., Wu, J., Lin, M., He, X., and Chua, T.-S.
Wu, Y.-X., Wang, X., Zhang, A., Hu, X., Feng, F., He, X., and Chua, T.-S. Deconfounding to explanation evaluation in graph neural networks. arXiv preprint arXiv:2201.088 2022c.
Causal attention for interpretable and generalizable graph classification. Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1696-1705, 2022.
Yu, S., Yu, X., Lokse, S., Jenssen, R., and Principe, J. C. Cauchy-schwarz divergence information bottleneck for regression. ICLR, 2024.
Sui, Y., Mao, W., Wang, S., Wang, X., Wu, J.. He, X., and Chua, T.S. Enhancing duiti--disttibution generalization on graphs via causal attention learning. ACM Transac- tions on Knowledge Discovery from Data, 18(5): 2024.
Zhang, H. mixup: Beyond empirical risk minimization. arXiv preprint arXiv: 2017.
Zhang, J., Chen, Z., Mei, H., Luo, D., and Wei, H. Regexplainer: Generating explanations for graph neural networks in regression task. arXiv preprint arXiv:2307.07840, 2023.
Sun, B. and Saenko, K. Deep coral: Correlation alignment for deep domain adaptation. In Computer Vision-ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8- 10 and 15-16, 2016, Proceedings, Part III 14, pp. 443- 450. Springer, 2016.
Zhao, S., Prapas, I., Karasante, I., Xiong, Z., Papoutsis, I., Camps- Valls, G., and Zhu, X. X. Causal graph neural networks for wildfire danger prediction. arXiv preprint arXiv:2403.08414 2024.
Tajwar, F., Kumar, A., Xie, S. M., and Liang, P. No true state-of-the-art? ood detection methods are inconsistent across datasets. arXiv preprint arXiv:2109.055: 2021.
Tishby, N. and Zaslavsky, N. Deep learning and the infor- mation bottleneck principle. In 2015 ieee information theory workshop (inw), pp. 1-5. IEEE, 2015.
Tishby, N., Pereira, F. C., and Bialek, W. The informa- tion bottleneck method. arXiv preprint physics/0000077, 2000.
Vapnik, V. Principles of risk minimization for learning the- ory. Advances in neural information processing systems, 1991.
von Rudorff, G. F., Heinen, S. N., Bragato, M., and von Lilienfeld, O. A. Thousands of reactants and transition states for competing e2 and s2 reactions. Machine Learn- ing: Science and Technology, 1(4):045026, 2020.
Wang, Z. and Veitch, V. A unified causal view of domain invariant representation learning. 2022.
Wang, Z., Chen, Y., Duan, Y., Li, W., Han, B., Cheng, J., and Tong, H. Towards aut-of-distribution generalizable predictions of chemical kineties properties. arXiv preprint arXiv:2310.03112 2023.
Wu, Q., Zhang, H., Yan, J., and Wipf, D. Handling distribu- tion shifts on graphs: An invariance perspective. arXiv preprint arXiv:2202.22460 2022a.

---

A Recipe for Causal Graph Regression: Confounding Effects Revisited
A. Supplementary Experiments
A.1. GOOD Benchmark
The Graph Dut-Of-Diitributtion (GOOD) benchmark is the most comprehensive and authoritative benchmark for assessing the OOD generalization of graph learning models. includes 11 datasets, covering six graph-level and five node-level tasks, with 51 dataset splits across covariate shift, concept shift, and no shift scenarios. Among them, nine datasets focus on classification (binary and multi-class), one (GOOD-ZINC) regression, and one (GOOD-PCBA) on multi-objective binary classification. GOOD is the first benchmark to incorporate both covariate and concept shifts within the same domain, enabling controlled comparisons. It evaluates 10 state-of-te ofte-ooftthe-art OOD methods, including four tailored for graphs, resulting in 510 dataset-model combinations. As a result, GOOD provides a systematic and rigorous framework for benchmarking OOD generalization in graph learning
A.2. ReactionOOD Benchmark
The ReactionOOD benchmark is a specialized out-of-distribution (OOD) evaluation framework designed to systematically assess the generalization capabilities of machine learning models in predicting the kinetic properties of chemical reactions. introduces three distinct levels of OOD struts--tructurall,, conditional, and mechanistic-and comprises six datasets, all formulated as regression tasks. Structural OOD (S-OOD) examines variations in reactant structures, including shifts based on total atomic count (F2 & SN2) and reactant scaffolds (RDB7, Cycloaddition). Conditional OOD (C-OOD) investigates the effect of environmental conditions kinetic properties, considering shifts in temperature (RMG Lib. T) and combined enperratuue-pressuure settings (RMG Lib. TP). Mechanistic OOD (M-OOD) explores the impact of different reaction mechanisms (RMG Family) on kinetic property predictions.
A.3. GOOD-ZINC Dataset Details
Table 4 presents the number of graphs/nodes in different dataset splits for the GOOD-ZINC dataset. The dataset is analyzed under three types of distribution shifts: covariate, concept, and no shift. Each row represents the number of graphs/nodes in training, in-distribution (ID) validation, ID test, out-of-distribution (OOD) validation, and OOD test sets. The no-shift scenario serves as a baseline with no distributional difference between training and test sets.
Table 4. Details GOOD-ZING dataset.
Dataset
Shift
Train
ID validation
III test
OOD validation
OOD test
covariate
149674
24945
24945
24945
24946
GOOD-ZINC
concept
101867
21828
21828
43539
60393
shift
149673
49891
49891
A.4. Experimental Settings
We use the GOOD-ZINC dataset from the GOOD benchmark and the S-OOD tasks from ReactionOOD, excluding other OOD tasks from ReactionOOD as they are still under maintenance. Our baseline results on ReactionOOD have been acknowledged by the original authors. We use a three-layer GIN as the backbone model, with 300 hidden dimensions, which consistently applied in both OURS and baseline models. The model is trained for 300 epochs, with the learning rate adjusted using the cosine annealing strategy. The initial learning rate is set to 0.001, with minimum value of Ie-8. For the OURS model, all tunable hyperparameters in the loss function L are set to 0.5.
A.5. Ablation Studies
Effectiveness Analysis To evaluate the effectiveness of the proposed loss functions LGIB and Lor in improving the model's OOD generalization ability, conducted series of ablation studies across four ood datasets: ZINC, Cycloaddition, F2SN2, and RDB7. Ours w/o BO serves as the baseline model, where both loss functions are removed, and only the causal subgraph readout layer's 4 loss is used for optimization. Ours w/o GIB ablates LGIB, eliminating the constraint on confounding subgraphs to assess the impact of removing confounder control on generalization. Conversely, Ours w/o CI removes Lei while keeping LGIB, allowing us to examine the contribution of the causal intervention loss to OOD generalization. Ours represents the complete model, incorporating both loss functions for optimization. Notably, ZINC
12

---

A Recipe for Causal Graph Regression: Confounding Effects Revisited
Ours w/o
Ours w/o GIB
Ours
6
Result
2-
ZINC
E2SN2
RDB7
Datasel
Figure 4. The comparison of different components.
evaluated using MAE, while the other datasets adopt RMSE as the evaluation metric. Given that the ZINC results are small (approximately 0.0x), we scale them by a factor of 10 the Figure 4 for better visualization and comparison.
The results reveal several key insights. The full model (green) consistently achieves the lowest RMSE across all datasets, demonstrating the effectiveness jointly applying both the enhanced GIB loss and the CI loss. Removing both components (yellow) leads the worst performance, confirming that both components are essential. Between the two losses, removing CI (orange) generally causes larger degradation than removing GIB (blue), suggesting that CI plays more dominant role. On E2SN2, however, GIB contributes more significantly. These results indicate that GIB and CI provide complementary benefits, and that using both yields the best OOD generalization.
Parameter Sensitivity Analysis In this experiment, we analyzed the sensitivity of loss function hyperparameters under different settings in the Cycloaddition dataset, focusing on two key components of our proposed loss function: the hyperparameter À for the causal intervention term and a, I for the confounding constraint term. The results in Figure
Scaffold Concept
Scatfold Covariate
Size Concept
Size Covariate
6.25
6.0
575
5.50
Result
5.25
45*
5.0
4.75
4.5
4.50
4.0
Parameter Value
Parameter Value
Parameter Value
Figure 5. Parameter sensitivity.
indicate that, there is no clear trend toward getting better worse for For 3, which balances the GIB loss, there is gradual increase in RMSE when it is too large, especially in scaffold-covariate settings, suggesting an optimal range around 0.3-0.6. For A, which controls the causal intervention loss, has the strongest impact. A suitable parameter interval (0.2-0.4) consistently leads to lower RMSE, while overly large or small À causes performance degradation, especially in the size-concept setting. This demonstrates the importance carefully tuning À to achieve effective OOD generalization.
13

---

A Recipe for Causal Graph Regression: Confounding Effects Revisited
B. Framework Details
Given a GNN-based encoder f()) and graph G; = (As, X;), the graph representation is computed as:
g. = f(),,
(18)
Then, to estimate attention scores, inspired by (Sui et al., 2022), we utilize separate MLPs for nodes and edges. The node attention scores, which can seen as the node-level soft mask can be computed as:
Mnode: Muode = (MMLP mode (Hg,)),
(19)
where a denotes the softmax operation applied across attention dimensions. Similarly, edge-level soft masks are determined by concatenating node embeddings from connected edges, followed by an edge-specific MLP:
Medge, Medge = o(MLPede edge (H [row], Hg.iccol Hg.(coll]))
(20)
These soft masks serve as weighting mechanisms, allowing the model to focus on the most relevant nodes and edges while maintaining differentiability.
Next, decompose the initial graph causal and confounding attened-subgraph:
C {Ai Medge X2 ② Mnode
(21)
S; = { {A, & Medge: X2 @ Mnode]
(22)
To encode these subgraphs, C, and S; processed through pair of GNNs with shared parameters, extracting causal and confounding representations He and H,, respextively. Finally, the representations of the two subgraphs are respectively used to obtain the predictions of the regression task through the corresponding readout layers.
C. Variational Bounds for the GIB Objective
The mutual information I(C;G) quantifies the dependency between and G and is defined as:
I(C: G) Ep(c. log PCC G) p(C))
(23)
However, computing the marginal distribution p(C) = =(C)) G)p(G) intractable, to overcome this challenge, we approximate p(C) with variational distribution (C). Substituting (C)) into Eq. (23), reformulate T(C; G) as:
pCC
G) (C)
G(C;G) Excc,
log g(C)
(24)
The KI. divergence term (T.()(C))l((())) is non-negative, providing an upper bound for T(C: G):
I(C; G) - Epta [L [ [))l)
(25)
14


Reviews:
[{"review_id": "IrDJ2cX3iw", "reviewer": "Reviewer_HaWt", "summary": "This paper proposes an improved causal graph regression method by an enhanced graph information bottleneck loss function and a contrastive learning loss from generated counterfactual graphs. Experiments on OOD datasets confirm its generality.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank Reviewer HaWt for the thorough and constructive feedback. We have carefully addressed the raised concerns and suggestions. We kindly ask the reviewer to reconsider their score if the following points resolve the reservations with our work, or to provide more pointers for us to address.\n\n### W1. How to obtain the causal subgraph $C$ from graph $G$, i.e., how to obtain the mask matrices $M_{edge}$ and $M_{node}$.\n \nWe appreciate the reviewer's question. The construction of the causal subgraph is introduced in the main text (Eq. (1)), and the details of generating the mask matrices are in Appendix B (Lines 647–692). \n\nIn the revision, we will briefly summarize the mask generation process in the main text.\n\n### W2. The assumptions in Section 4.2 remain unverified. Are they reasonable?\n\nThank you for raising this point. We go through and clarify each assumption below:\n* Assumption 1 (Gaussian assumption of $p(C \\mid G)$ and $q(C)$):\nWe follow the variational information bottleneck literature [4], modeling both as multivariate Gaussians to enable tractable KL estimation. This is widely adopted in similar works [5,6].\n* Assumption 2 (Covariance $\\Sigma_\\phi(G) = I$):\nSetting the covariance to identity simplifies optimization and is theoretically justified in [4, Appendix A], which shows that any full-rank covariance can be whitened without loss of generality.\n* Assumption 3 (Gaussian posterior):\nFor $P(Y \\mid H_c)=\\mathcal{N}(Y; \\mu_{(c)}, \\sigma^2_{(c)})$, we use a fixed variance for stability, similar to Phase I assumption in section 1 of [7]. This also allows a closed-form expression for mutual information estimation (e.g., [6], Sec.2.1, Proposition 1).\n\nWe will clarify these assumptions more explicitly and emphasize their empirical validity (the strong OOD generalization results in Section 5) in the revision.\n\n### W3. There is no obvious superiority of the OOD performance compared with the ID performance.\n\nWe respectfully clarify that the goal of CGL methods might be misunderstood; we do not aim for OOD performance to surpass ID performance, which is unrealistic. \n\nWe focus our attention on **the robustness under OOD settings**, demonstrating strong generalization ablity of our method. For more analysis of OOD performance, we humbly refer the reviewer to the response to Q2 from Reviewer jru1.\n\n### [Experimental Designs] The effect of the hyperparameters $\\alpha$ and $\\beta$\n![a](https://hackmd.io/_uploads/H1wfF-upyx.png)  \n\nWe thank the reviewer for this suggestion. Some discussion related to this point has already been included in Appendix A.5. To better illustrate, we rerun the experiments and will update the sensitivity analysis for $\\alpha$, $\\beta$, and $\\lambda$. (More details can be find in the attached figure.) \n\nThe results show that the choice of $\\alpha$ has no clear effect, while the choices of $\\beta$ and $\\lambda$ significantly impact OOD performance.\n\n### [Broader Sci. Literature] Causal graph regression (CGR) seems to be a relatively neglected domain, and the authors also do not provide any important applications of CGR\n\nWe appreciate this review and will revise the introduction to better highlight the motivating applications and clarify the contribution as **improving generalization in graph-based prediction tasks through causal mechanisms**.\n\nFurthermore, we beg to clarify\n- CGR is not a niche topic but a core component of the broader field of **causal graph learning** (CGL). As noted in Section 1 (lines 36-46), CGL has emerged as a key paradigm for building robust models. Our work leverages causal interventions to improve OOD generalization in graph regression, which is essential for many real-world scenarios with distributional shifts.\n- We have discussed the application of CGR in Section 1 (lines 48-51). Moreover, CGR is relevant to tasks such as molecular property prediction [1], traffic flow forecasting [2], and credit scoring [3].\n\n---\n[1] Rollins Z A. et al. MolPROP: Molecular Property prediction with multimodal language and graph fusion. J. Cheminform., 2024.  \n[2] Li G. et al. Multistep traffic forecasting by dynamic graph convolution: Interpretations of real-time spatial correlations. Transp. Res. Part C Emerg. Technol., 2021.  \n[3] Ma F. et al. Utilizing Reinforcement Learning and Causal Graph Networks to Address the Intricate Dynamics in Financial Risk Prediction. Int. J. Inf. Technol. Syst. Approach, 2024.  \n[4] Chechik G. et al. Information bottleneck for Gaussian variables. Adv. Neural Inf. Process. Syst., 2003.  \n[5] Kingma D P, Welling M. Auto-encoding variational bayes[EB/OL].(2013-12-20)  \n[6] Yu S. et al. Cauchy-Schwarz Divergence Information Bottleneck for Regression. arXiv, 2024.  \n[7] Nix D. et al. Learning local error bars for nonlinear regression. Adv. Neural Inf. Process. Syst., 1994.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "There are no formal theoretical claims in this paper.", "experimental_designs_or_analyses": "Yes, I checked the experiments. The results verify the effectiveness of the proposed method. The only issue is that there is no discussion on the effect of the hyperparameters $\\alpha$ and $\\beta$, which seem to be important hyperparameters in the method.", "relation_to_broader_scientific_literature": "Causal graph regression seems to be a relatively neglected domain. The authors also do not provide any important applications of the causal graph regression task. Therefore, I think the key contributions of the paper may not have a large influence to the broader scientific literature.", "essential_references_not_discussed": "I do not know any related works that are essential to this paper.", "other_strengths_and_weaknesses": "- The paper is well-written and easy to follow.\n- The experimental results are good.\n\n1. Could you please explain how to obtain the causal subgraph $C$ from a graph $G$, i.e., how to obtain the mask matrices $M_{edge}$ and $M_{node}$? Even though the concrete approach might be discussed in previous work, I think it would be better to have a brief introduction in the background to make the paper more consistent.\n2. There are several assumptions in Section 4.2. However, it remains unknown whether these assumptions are reasonable in practice.\n3. The OOD generality is claimed as a core contribution of the proposed approach. However, there is no obvious superiority of the OOD performance compared with the ID performance.", "other_comments_or_suggestions": "\"5.2. baselines and Setup \" -> \"5.2. Baselines and Setup\"", "questions_for_authors": "See above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OArzynFZNE", "reviewer": "Reviewer_jru1", "summary": "This paper investigates causal graph regression (CGR), extending causal graph learning (CGL) techniques, which have been successful in classification tasks, to the more challenging regression setting. The authors introduce a novel approach that adapts causal intervention techniques to regression through the use of contrastive learning. Their method aims to mitigate confounding effects and improve the generalizability of graph neural networks (GNNs) under out-of-distribution (OOD) scenarios. The main claims of the paper include:\n\n1. Contrastive Learning for Causal Graph Regression – The introduction of contrastive learning as a means of handling confounders in graph-level regression tasks.\n\n2. Generalization of Causal Interventions – An adaptation of classification-specific causal intervention techniques to the regression setting.\n\n3. Extensive Empirical Validation on Graph OOD Benchmarks – Demonstrations of effectiveness of the proposed method through experiments on multiple OOD datasets.\n\n##update after rebuttal", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank Reviewer jru1 for the thoughtful and encouraging feedback. We appreciate your recognition of our method’s motivation, design, and empirical support. Below we address each of your comments in detail. \n\n### Q1. Explain the fundamental difference between graph classification and regression, and why regression is so challenging.\n\nThe core difference lies in the adoptation of causal discovery, as noted in Lines 057–060, where we state that existing methods \"rely on **discrete label information** and cannot be adapted to regression.\" \n\nIn regression, the absence of discrete labels naturally makes confounder separation harder. For example, in molecular property prediction, continuous outcomes (e.g., solubility values) often arise from overlapping substructures, complicating causal identification.\n\n### [Q2 & Claims And Evidence] Performance Gaps on ReactionOOD Datasets\n\nWe appreciate this observation. The relatively weaker performance is attributable to known challenges rather than methodological flaws:\n* Our method stems from **more general and weaker assumptions**, which naturally implies larger function space and higher requirements on sample size:  \nUnlike prior methods such as IRM or DIR, our method do not assume that spurious features are non-predictive. This allows us to model a broader function class under weaker assumptions, making our method applicable to a wider range of real-world distribution shifts. The trade-off is increased optimization difficulty under limited data or high spurious signal, but this reflects a principled design choice, not a methodological weakness.\n* No method dominates across all OOD benchmarks:  \nAs shown in OOD-GNN [1], it is a regular phenomenon in generalization tasks that no approach consistently performs best on every dataset due to varying distribution shifts and inductive biases. Despite this, our method is among the most stable across diverse settings and achieves best results on ZINC (Table 1), the largest and most complex dataset.\n* Spurious correlations affect ID performance:  \nNon-causal baseline methods (e.g., ERM, CORAL, and DANN) often perform better under ID or mild OOD due to exploiting spurious but predictive features. To handle OOD settings, our method instead recognizes those spurious features and removes them for better generalization, which may slightly reduce ID performance but improves robustness.\n    > For example, in Cycloaddition-ID（Total Atom Number, concept), CORAL achieves 4.10 vs. ours 5.74, but under OOD, CORAL drops to 5.74 while ours remains stable at 5.53 (the smaller the better).\n* Causal and contrastive learning require more data:  \nCausal inference is statistically harder than correlation-based methods and generally requires more data to reduce variance [2]. Contrastive learning also benefits from large batch sizes and diverse representations. This explains why our method performs best on ZINC. On smaller datasets (e.g., RDB7), all causal methods degrade; however, our method remains stable and consistently outperforms CIGA (the best method of the listed causal intervention baselines) across all RDB7 settings.\n\n\nWe will modify the corresponding experimental analysis.\n\n\n### [Typo] Correct typo in Section 5.2 title and line 393.\n\nWe appreciate your attention to detail. We will accordingly revise the manuscript to reflect your suggestion.\n\n---\n[1] Tajwar F et al. No true state-of-the-art? OOD detection methods are inconsistent across datasets. arXiv, 2021.  \n[2]  Guo R et al. A survey of learning causality with data: Problems and methods. ACM Comput. Surv., 2020.", "claims_and_evidence": "The paper provides empirical evidence through experiments on graph OOD benchmarks. The use of contrastive learning to reshape causal intervention in regression is an innovative approach. The results indicate that the proposed method performs well on several datasets, supporting the claim that contrastive learning is effective for CGR. However, the method does not perform well on the Cyloadition and RDB7 datasets. The paper lacks sufficient explanation for these weaker results, which raises concerns about the robustness and generalizability of the approach across diverse datasets.", "methods_and_evaluation_criteria": "The proposed method is well-motivated and aligns with the challenges of causal graph regression. The evaluation is conducted on a range of benchmark datasets, which provides a solid empirical foundation", "theoretical_claims": "The variational bound for the GIB objective seems fine to me.", "experimental_designs_or_analyses": "The experiments are well designed. The analysis on the results on the ReactionOOD dataset can be improved.", "relation_to_broader_scientific_literature": "This work builds on existing research in causal graph learning and graph neural networks, expanding causal intervention techniques beyond classification to regression tasks.", "essential_references_not_discussed": "No, all important related works are cited.", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "Typo: Title of Section 5.2, and check line 393 \"... loss L_{CI} Conversely ...\"", "questions_for_authors": "Question 1: explain the fundamental difference between graph classification and regression, and why regression is so challenging.\n\nQuestion 2: hint on why methods such as ERM, CORAL and DANN outperform your method in Table 2 and Table 3.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jBVCiz02JZ", "reviewer": "Reviewer_iAdk", "summary": "This paper addresses the challenge of adapting causal graph learning (CGL) techniques from classification to regression tasks, introducing a framework called causal graph regression (CGR). The authors identify two key innovations: (1) an enhanced graph information bottleneck loss function that, unlike previous approaches, acknowledges the predictive power of confounding features rather than treating them as pure noise, and (2) a contrastive learning approach for causal intervention that can operate without discrete class labels. By combining these techniques, their method generates counterfactual graphs through random combinations of causal and confounding subgraphs, enabling the model to learn invariant representations. Extensive experiments on GOOD-ZINC and ReactionOOD benchmarks demonstrate performance improvements in out-of-distribution settings.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank Reviewer iAdk for the useful feedback. Detailed responses are provided below. We kindly ask the reviewer to reconsider their score if the following clarifications resolve the concerns.\n\n### Q1. Why are gains on GOOD-ZINC larger than on ReactionOOD?\nGOOD-ZINC is large-scale and has diverse OOD testing samples, thus more suitable for causal discovery (which naturally requires more samples than ERM). In contrast, ReactionOOD datasets are smaller and the OOD samples are more specialized. Our method shows larger gains on GOOD-ZINC because our recipe more effectively handles diverse distributions, while baselines struggle with such complexity.\n\nDetails can be seen in the response to [Q2 & Claims And Evidence] from Reviewer jru1.\n\n### W1. Overly narrow scope of causal graph regression\nWe respectfully disagree, and humbly refer the reviewer to our response to [Broader Sci. Literature] from Reviewer HaWt.\n\n### W2. Method motivation\nWe humbly refer the reviewer to the response to [Methods] below.\n\n### [Claims And Evidence] Claim 2 (restated below) lacks support\n> Claim 2: The reason that existing CGL methods don't work well for regression tasks stems from an assumption that confounding subgraphs contain strictly no predictive power.\n\nWe respectfully clarify a misunderstanding regarding Claim 2. In our paper, we never claim that existing CGL methods fail in regression tasks due to the assumption that confounding subgraphs contain strictly no predictive power. \n\nRather, a similar statement serves as part of our **motivation** (instead of claim): prior methods (e.g., CAL [1]) often assume that confounding subgraphs are non-predictive, which does not hold in practice (both in classification and regression settings). We provide empirical evidence supporting this motivation in Section 5.5 (Lines 385–397, Figure 3 left), showing that completely ignoring the predictive signals of confounders can lead to performance degradation.\n\n### [Methods]\n> - The motivation for using contrastive learning (CL) in regression is unclear.\n\nWe respectfully clarify the motivation for using CL. Existing CGL methods (e.g., [1]) proposed to **align the representation** of intervened graphs (constructed by randomly pairing confounding subgraphs with target causal subgraphs [2]) through the **labels of causal subgraphs**, and due to the explicit usage of labels they cannot be directly applied to regression tasks. \n\nFrom this perspective, CL is a perfect fit for causal graph regression, since the InfoNCE loss in CL aims at \"perfect alignment\" [3] without using labels, pulling positive sample representations (intervened graphs with the same causal subgraphs in our case) closer while pushing negative pairs (intervened graphs with distinct causal subgraphs) apart, which thus distinguishes causal effects between factual and counterfactual samples. This design enables models to implicitly capture causal signals without label supervision, consistent with recent advances in causal representation learning [4].\n\n> - Is the modeling of $I(C;Y), I(C;G)$ different from those in existing work.\n\nThe modeling of $I(C;Y), I(C;G)$ in this paper does differ. \n- Prior works on causal graph learning derive GIB-based objectives under **classification settings**.\n- In contrast, we provide a new formulation of the GIB loss under Gaussian assumptions **specific to graph regression**, allowing principled disentanglement of causal and confounding subgraphs under continuous targets.\n\n### [Theoretical Claims] Assumption of identity covariance\nWe humbly refer the reviewer to our response to W2 of Reviewer HaWt.\n\n### [Supp. Material] Ablation studies\n\n![b](https://hackmd.io/_uploads/HkXuO-_ayl.png)\n\nWe take your suggestion and re-perform the ablation studies.\n\nAblation 1 - Effectiveness Analysis: We evaluate the performance of our model variants across four OOD datasets. Each variant removes several proposed modules (the same setting as in Appendix A.5, Lines 597-627). \n\nDue to varying dataset complexities, the contributions of GIB and CI differ across tasks. Nonetheless, the full model consistently achieves the best performance, indicating the two loss functions work synergistically to enhance OOD generalization. (More details can be found in the attached figure.)\n\nAblation 2 - Parameter Sensitivity Analysis: We humbly refer the reviewer to our response to [Experimental Designs] from Reviewer HaWt.\n\n### [Reference]\nThank you for catching this oversight. We will definitely cite the reference when introducing GIB in Sec. 3.2 and acknowledge its influence.\n\n---\n[1] Causal attention for interpretable and generalizable graph classification. KDD, 2022.  \n[2] Debiasing graph neural networks via learning disentangled causal substructure. NeurIPS, 2022.  \n[3] Understanding contrastive representation learning through alignment and uniformity on the hypersphere. ICML, 2022.  \n[4] Robust causal graph representation learning against confounding effects. AAAI, 2023.", "claims_and_evidence": "Claim 1: Existing causal graph learning can not work well for graph regression tasks.\n\nSupport: 1. Citations. 2. Experimental results in Section 5. This claim is supported with evidence. \n\nClaim 2: The reason that existing CGL methods don't work well for regression tasks stems from an assumption that confounding subgraphs contain strictly no predictive power. \n\nSupport: I don't find clear supports for this claim. I personally deduce that this is related to the optimization dynamics related to tasks (classification and regression).", "methods_and_evaluation_criteria": "Methodologically, the paper extends causal graph learning to regression tasks by:\nRethinking how confounding effects should be handled in regression (acknowledging predictive power of confounding features rather than treating them as pure noise)\nDeveloping a contrastive learning approach to replace label-dependent causal intervention techniques that don't transfer well to regression. \nThese two designs make sense, while the motivation for the second approach is not clearly stated. Authors thoroughly introduce the modeling of $I(C;Y)$ and $I(C;G)$. I wonder if there's any difference from those introduced in existing papers. If not, authors should focus on two mechanisms introduced in this paper. \n\nFor evaluation, the authors use appropriate benchmark datasets:\nGOOD-ZINC: A standard graph regression dataset explicitly designed for OOD testing\nReactionOOD: Specialized datasets (Cycloaddition, E2&SN2, RDB7) with regression tasks for chemical reactions\nThe evaluation criteria are reasonable.", "theoretical_claims": "There's no rigorous theoretical claims in this paper. I find some design choices in Section 4 (like selecting identity covariance matrix) are not well supported.", "experimental_designs_or_analyses": "For evaluation, the authors use appropriate benchmark datasets:\nGOOD-ZINC: A standard graph regression dataset explicitly designed for OOD testing\nReactionOOD: Specialized datasets (Cycloaddition, E2&SN2, RDB7) with regression tasks for chemical reactions\nThe evaluation criteria are reasonable.", "relation_to_broader_scientific_literature": "* Out-of-Distribution Generalization for Graphs: This paper discusses why previous CGL-based methods can't work well on regression tasks and provides a remedy.\n* Molecular Property Prediction: Explaining the effectiveness of contrastive learning for molecular data.", "essential_references_not_discussed": "[1] is closely related but not discussed at all. \n\n[1] Wu, Tailin, et al. \"Graph information bottleneck.\" Advances in Neural Information Processing Systems 33 (2020): 20437-20448.", "other_strengths_and_weaknesses": "I think this paper suffers from two limitations. First, the scope is excessively narrow, focusing on a highly specific problem within graph regression. Second, method design lacks clear motivation and support, as the authors fail to adequately justify their methodological choices or connect them to established theoretical principles.", "other_comments_or_suggestions": "Definition in Section 3: You are considering a set of graphs but not a single graph.\n5.2. baselines and Setup -> Baselines", "questions_for_authors": "I notice that the proposed methods achieve much larger gains on GOOD datasets than ReactionOOD datasets. Do you have any ideas on the reason?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The work deals with the problem of regression using a causal graph where the most important subgraph from a given input graph can be learned for the given task at hand using a graph neural network. As claimed by the authors, this task, termed "causal graph learning," is a relatively new problem, and the major focus has been on classification tasks. This work extends the literature towards the regression task and adapt causal intervention techniques to regression through the use of contrastive learning.

The ratings of the paper lean towards the positive, with the issue that no reviewer is willing to champion the paper. The AC read the paper and believes that the paper proposes a solid contribution to a niche area of research and can be an interesting paper to discuss among the causal and GNN community in the conference. I thus recommend acceptance.
Decision: Accept (poster)