Adversarial Causal Augmentation for Graph Covariate Shift
Authors: Yongduo Sui
Xiang Wang
Jiancan Wu
An Zhang
Xiangnan He
Tat-Seng Chua
Venue: Submitted to ICLR 2023 (2023)
Keywords: Graph Data Augmentation
Graph Neural Networks
Covariate Shift
OOD Generalization

Abstract: Out-of-distribution (OOD) generalization on graphs is drawing widespread attention. However, existing efforts mainly focus on the OOD issue of correlation shift. While another type, covariate shift, remains largely unexplored but is the focus of this work. From a data generation view, causal features are stable substructures in data, which play key roles in OOD generalization. While their complementary parts, environments, are unstable features that often lead to various distribution shifts. Correlation shift establishes spurious statistical correlations between environments and labels. In contrast, covariate shift means that there exist unseen environmental features in test data. Existing strategies of graph invariant learning and data augmentation suffer from limited environments or unstable causal features, which greatly limits their generalization ability on covariate shift. In view of that, we propose a novel graph augmentation strategy: Adversarial Causal Augmentation (AdvCA), to alleviate the covariate shift. Specifically, it adversarially augments the data to explore diverse distributions of the environments. Meanwhile, it keeps the causal features invariant across diverse environments. It maintains the environmental diversity while ensuring the invariance of the causal features, thereby effectively alleviating the covariate shift. Extensive experimental results with in-depth analyses demonstrate that AdvCA can outperform 14 baselines on synthetic and real-world datasets with various covariate shifts.

Under review as a conference paper at ICLR 2023
ADVERSARIAL CAUSAL AUGMENTATION FOR GRAPH COVARIATE SHIFT
Anonymous authors
Paper under double-blind review
ABSTRACT
Out-of-distribution (OOD) generalization on graphs is drawing widespread atten- tion. However, existing efforts mainly focus on the OOD issue of correlation shift. While another type, covariate shift, remains largely unexplored but is the focus of this work. From a data generation view, causal features are stable substructures in data, which play key roles in OOD generalization. While their complementary parts, environments, are unstable features that often lead to various distribution shifts. Correlation shift establishes spurious statistical correlations between en- vironments and labels. In contrast, covariate shift means that there exist unseen environmental features in test data. Existing strategies of graph invariant learn- ing and data augmentation suffer from limited environments or unstable causal features, which greatly limits their generalization ability on covariate shift. In view of that, we propose novel graph augmentation strategy: Adversarial Causal Augmentation (AdvCA), to alleviate the covariate shift. Specifically, it adversari- ally augments the data explore diverse distributions of the environments. Mcan- while, it keeps the causal features stable across diverse environments. maintains the environmental diversity while ensuring the invariance of the causal features, thereby effectively alleviating the covariate shift. Extensive experimental results with in-depth analyses demonstrate that AdvCA can outperform 14 baselines on synthetic and real-world datasets with various covariate shifts.
INTRODUCTION
Training samples
Ptrain
Parop
DropEdge samples
Graph learning mostly follows the assumption that training and test data are independently drawn from an identical distribution. Such an assumption is difficult to be satisfied in the wild, due to out-of-distribution (OOD) issues (Shen al., 2021), where the training and test data are from different distributions. Hence, OOD generalization on graphs is attracting widespread attention (Li et al., 2022b). However, existing stud- ies mostly focus on correlation shift, which is just one type of OOD issue (Ye al., 2022; Wiles et al., 2022). While another type, covariate shift, remains largely unexplored but is the focus of our work.
Covariate shiff
Pours
AdoCA simples
samples
Prest
Causal Ieatuce (bouse)
Factronmental fradures (ladtr & tree)
Covariate shift is in stark contrast to correlation shift w.E.t. causal and environmental features of data'.. Specifically, from a data generation view, causal fea- tures? are the substructures of the entire graphs that truly reflect the predictive property of data, while their complementary parts are the environmental features that are noncausal to the predictions. Following prior studies (Arjovsky et al., 2019; Wu et al., 2022b), we assume causal features are stable across distri- butions, in contrast to the environmental features. Correlation shift denotes that environments and labels establish inconsistent statistical correlations in training and test data; whereas, covariate shift
Ptrain and Prest denote the train- ing and test distributions. Parop and Pours represent the distributions of augmented data via DropEdge and AdvCA. AdvCA establishes a smaller covariate shift (+) with test distribution than DropEdge (++).
Figure 1:
A We provide detailed discussions of these two distribution shifts in Appendix C.
We provide formal definition in Assumption 1.

---

Under review as a conference paper at ICLR 2023
means that the environmental features in test data are unseen in training data (Ye et al., 2022; Wiles et al., 2022; Gui et al., 2022). For example, in Figure 1. the environmental features ladder and tree different in training and test data, which forms the covariate shift (++). Taking molecular property predictions as another example, functional groups (e.g., nitrogen dioxide (NO2)) are causal features that determine the predictive property of molecules. While scaffolds (e.g., carbon rings) are irrelevant patterns (Wu et al., 2018), which can be seen as the environments. In practice, we often need to use molecular graphs collected in the past to train models, hoping that the models can predict the properties of molecules with new scaffolds in the future (Hu et al., 2020).
Because of the differences between correlation and covariate shifts, we take close look at the exist- ing efforts on graph generalization. Existing efforts (Li et al., 2022b) mainly fall into the following research lines, each of which has inherent limitations to solve covariate shift.
Invariant graph learning (Wu et al., 2022b; Liu et al., 2022; Sui et al., 2022) gradually becomes a prevalent paradigm for OOD generalization. The main idea is to capture the causal features by minimizing the empirical risks within different environments. Unfortunately, it implicitly makes a prior assumption that all test environments are available during training. This assumption unrealistic owing to the obstacle of training data covering all possible test environments. Learning in limited environments can only alleviate the spurious correlations that are hidden in the training data, but fail to extrapolate test distributions with unseen environments.
Graph data augmentation (Ding et al., 2022; Zhao et al., 2022) perturbs graph features to enrich the distribution seen during training for better generalization. It can be roughly divided into node-level (Kong et al., 2022), edge-level (Rong et al., 2020), and graph-level (Wang et al., 2021; Han al.. 2022) with random (You al., 2020) or adversarial strategies (Suresh et al., 2021). However, they are prone to destroy the causal features, which easily loses control of the perturbed distributions. For example, in Figure I, the random strategy DropEdge (Rong al., 2020) will inevitably perturb the causal features (highlighted by red circles). As such, it fails to alleviate the covariate shift (++), even degenerating the generalization ability.
Scrutinizing the limitations of the aforementioned studies, insufficient environments and unstable causal features largely hinder the ability of these generalization efforts against the covariate shift. Hence, we naturally ask question: "Can the augmented samples simultaneously preserve the di- versity of environmental features and the invariance of causal features?"
Towards this end, we first propose two principles for graph augmentation: environmental diversity and causal invariance. Specifically, environmental diversity encourages the augmentation to extrap- olate unseen environments; meanwhile, causal invariance shortens the distribution gap between the augmented data and test data. To achieve these principles, we design a novel graph augmentation strategy: Adversarial Causal Augmentation (AdvCA). Specifically, network, named adversarial augmenter. It adversarially generates the masks on edges and node fea- tures, which makes OOD exploration for improving the environmental diversity. To maintain the stability of the causal features, we adopt another network, named causal generator. It generates the masks that capture causal features. Finally, we delicately combine these masks and apply them to graph data. As shown in Figure 1, AdvCA only perturbs the environmental features, while keeping the causal parts untorched. Our quantitative experiments also verify that AdvCA can narrow the distribution gap between the augmented data and test data, as illustrated in Figure 1 (++), thereby effectively overcoming the covariate shift issucs. Our contributions can be summarized as:
augment the graphs by
We are exploring one specific type of OOD issue in graph learning: covariate shift, which is of great need but largely unexplored.
Problem:
Method: We design a graph augmentation method, AdvCA, which focuses on covariate shift
issues. It maintains the stability of causal features while ensuring environmental diversity.
Experiment: We conduct extensive experiments on synthetic and real datasets. The experimental results with in-depth analyses demonstrate the effectiveness of AdvCA.
2 PRELIMINARIES
In this section, we first give the formal definitions of causal features, environmental features, and graph covariate shift. Then we present the problem of graph classification under covariate shift.
2

---

Under review as a conference paper at ICLR 2023
2.1 NOTATIONS
We define the uppercase letters (e.g. ., G) as random variables. The lower-case letters (e.g,, g) are samples of variables, and the blackboard bold typefaces (e.g., G) denote the sample spaces. Let g (A, X) E G denote graph, where A and X are its adjacency matrix and node features, respectively. It is assigned with a label y Y with a fixed labeling rule G Y. Let D = {(g., 9:)3 denote a dataset that is divided into a training set Dir {
and a test set Dte = {( 34 ) eeE... &ur and Ete are the index sets of training and test environments, respectively. this work, we focus on graph classification scenario, which aims to train models with Der and infer the labels in Dic-
2.2 DEFINITIONS AND PROBLEM FORMATIONS
Following studies (Arjovsky et al., 2019; Wu et al., 2022b), we assume that the inner mechanism of the labeling rule G + Y usually depends on the causal features, which particular subparts of the entire data. Causal invariance denotes that the relationship between the causal feature and label is invariant across different environments or distributions, which makes OOD generalization possible (Ye et al., 2022). While the complement causal parts, environmental features, are noncausal for predicting the graphs. Now we give a formal definition of these features.
Assumption (Causal & Environmental Feature) Assume input graph G containing two fea- tures Geau- Geny, and they satisfy: Geau U. Genv G. If they obey the following conditions: i) (sufficiency condition) P = P(Y(G); üi) (independence condition) Y IL Ganv Geau, then we define Geau and Geny as the causal feature and environmental feature, respectively.
Sufficiency condition requires that causal features should be sufficient to preserve the critical infor- mation of data G related to the label Y. While the independence indicates that causal feature can shield the label from the influence of the environment feature. It makes causal features establish an invariant relationship with labels across different environments. Hence, distribution shifts are only caused by the environmental features rather than causal features. Recent studies (Ye et al., 2022; Gui et al., 2022) have pointed out that OOD issue can be specifically divided into correlation shift and covariate shift. Since we mainly focus on the latter, we put detailed discussions between them in Appendix C. Now we give formal definition of the covariate shift on graphs.
Definition (Graph Covariate Shift) Let Ptr and Pto denote the probability functions of the train- ing and test distributions. We measure the covariate shift between distributions Ptr and P.o as
GCS(Pu., 1 (Pull)) (g)ldg
(I)
where S = ())(())) (g) = 0}. which covers the features (e.g., environmental featuurs) that do not overlap between the distributions.
(Ce( is always bounded in [0.1]. The issue of graph covariate shift is very common in practice. For example, the chemical properties of molccules are mainly determined by specific functional groups, which can be regarded causal features to predict these properties (Arjovsky et al., 2019; Wu et al., 2022b). While their scaffold structures (Wu et al., 2018), which are often irrelevant to their properties, can be seen environmental features. In practice, train models on past molecular graphs, and hope that the model can predict the properties of future molecules with novel scaffolds (Hu et al., 2020). Hence, this work focuses on the covariate shift issues. Now we give formal definition of this problem as follows.
often need to
Problem (Graph Classification under Covariate Shift) Given the training and test sets with en- vironment sets Eur and Euc, they follow distributions Ptr and Pt.. and they satisfy: GCS(Pur Pue) O. We aim to use the data collected from training environments Enr. and learn a powerful graph classifier f G that performs well in all possible test environments Ero:
= arg min min
sup E*(e(f(g).))1. etE.n
f
where (E( y) is the empirical risk on the environment e, and f(-,-) is the loss function.
Problem states that it is unrealistic for the training set to cover all possible environments in the test set. It means that we have to extrapolate unknown environments by using the limited training environments at hand, which makes this problem more challenging.
3

---

Under review as a conference paper at ICLR 2023
3 METHODOLOGY
In this section, we first propose two principles for graph data augmentation. Guided by these prin- ciples, we design a new graph augmentation strategy that can effectively solve Problem I.
3.1 TWO PRINCIPLES FOR GRAPH AUGMENTATION
Scrutinizing Problem 1, we observe that the covariate shift is mainly caused by the scarcity of training environments. Existing efforts (Liu et al., 2022; Sui et al., 2022; Wu et al., 2022b) make intervention or replacement of the environments to capture causal features. However, these environ- mental features still stem from the training distribution, which may result in limited diversity of the environments. Worse still, if the environments are too scarce, the model will inevitably learn the shortcuts between these environmental features, resulting in suboptimal learning of the causal parts. To this end, we propose the first principle for data augmentation:
Principle (Environmental Diversity) Given a set of graphs {g} with distribution function P. Let T-) denote an augmentation function that augments graphs (T(g)} to distribution function P. Then T() should meet GCS( P, P) 1.
Principle I states that P should keep away from the original distribution P. Hence, the distribu- tion of augmented data tends not to overlap with the original distribution, which encourages the diversity of environmental features. However, from a data generation perspective, causal features are stable and shared across environments (Kaddour et al., 2022), so they are essential features for OOD generalization. Since Principle 1 does not expose any constraint the invariant property of the augmented distribution, we here propose the second principle for augmentation:
Principle (Causal Invariance) Given a set of graphs {g} with a corresponding causal feature set {9eau (Acaus Xeau)} }. Let T-) denote an augmentation function that augments graphs {T())} with corresponding causal feature set (Yeau = (Acau: Xean)]. Then T()) should meet B[ Acan
Acau [+] 0 and Ee[ x
[+]]
- 0, where |1 1p is the Frobenius norm.
Principle 2 emphasizes the invariance of the graph structures and node features in causal parts after data augmentation. As illustrated in Figure/ 1, Principle 1 keeps the distribution of augmented data away from the training distribution; meanwhile, Principle 2 restricts the distribution of augmented data not too far from the test distribution. These two principles complement each other, and further cooperate together alleviate the covariate shift.
3.2 OUT-OF-DISTRIBUTION EXPLORATION
Given a GNN model f(-) with parameters 6, we decompose f 4 h, where h(-) G Rd is graph encoder to yield d-dimensional representations, and ( Rd - Y is a classifier. To comply with Principle 1, we need to do OOD exploration. Inspired by distributionally robust optimization (Sagawa et al., 2020), we consider the following optimization objective:
min sup sup((pp((((().))) D(P,))
where 1' and P denote the original and explored data distributions, respectively. D(--)) is distance metric between two probability distributions. The solution to Equation 3 guarantees the generaliza- tion within a distance p of the distribution P. To better measure the distance between distributions, as suggested by Sinha et al. (2018), we adopt the Wasserstein distance (Arjovsky et al., 2017; Volpi et al., 2018) as the distance metric. The distance metric function can be defined as:
D)P,P) inf Ex[c(T,g)], (eT((PP)
where [(P, P) is the set of all couplings of Pnd P. c(:.)) is the cost function. Studies (Dosovitskiy & Brox, 2016; Volpi et al., 2018) also suggest that the distances in representation space typically correspond to semantic distances. Hence, we define the cost function in the representation space and give the following transportation cost:
c( (I)(7) h(9))1
(5)
4

---

Under review as a conference paper at ICLR 2023
Causal Generator
Keeping Causal F Features Invariant
Augmented Graphs
GNN Classifier
Input
Training time
Keeping Environmental Features Diverse
Adversarial
Augmenter
Figure 2: The overview of Adversarial Causal Augmentation (AdvCA) Framework.
It denotes the "cost" of augmenting the graph g T. We can observe that it is difficult set proper in Equation 3. Instead, we consider the Lagrangian relaxation for fixed penalty coefficient y. Inspired by Sinha et al. (2018), we can reformulate Equation 3 as follows:
min
(6)
where ((f(g)))) supger v) g)) And we define ((f v) as the robust surro- gate loss. we conduct gradient descent on the robust surrogate loss, we will have:
Voo(S()),)) vol(I(T)..)
(7)
where T" arg ( (T( -c( geB
(8)
D" is an augmented view of the original data g. Hence, to achieve OOD exploration, we just need to perform data augmentation via Equation 8 on the original data g.
3.3 ADVERSARIAL CAUSAL AUGMENTATION
Equation 8 endows the ability of OOD exploration data augmentation, which makes the aug- mented data meet Principle 1. In addition, to achieve Principle 2, we also need to implement causal feature learning based on the sufficiency and independence conditions in Assumption 1. Hence, we design a novel graph augmentation strategy: Adversarial Causal Augmentation (AdvCA). The overview of the proposed framework is depicted Figure which mainly consists of two com- ponents: adversarial augmenter and causal generator. Adversarial augmenter achieves OOD ex- ploration through adversarial data augmentation, which encourages the diversity of environmental features; meanwhile, the causal generator keeps causal feature invariant by identifying causal fea- tures from data. Below we elaborate on the implementation details.
Adversarial Augmenter & Causal Generator. We design two networks, adversarial augmenter Te, (.) and causal generator Te, (-), which generate masks for nodes and edges of graphs. They have the same structure and parameterized by 0 and 022 respectively. Given input graph = (A, X) with n nodes, mask generation network first obtains the node representations via a GNN encoder Ti.). To judge the importance of nodes and edges, it adopts two MLP networks MLP (.) and M-P to generate the soft node mask matrix M [gnx1 and edge mask matrix M goo for graph data, respectively. In summary, the mask generation network can be decomposed as:
Z T((g), M o (MMLP((h.), Mc,- = o (MLP 2[2. 2,1));
where Z E Raax is node representation matrix, whose i-th row z; Z[i denotes the representation of node i, and (-) the sigmoid function that maps the mask values M and M [0, 1].
Ad Adversarial Causal Augmentation. To estimate & in Equation 8, we define the adversarial leam- ing objective as:
max (Lady Er, [T([(To (9). 7e ()),
(10)
8,
Then we can augment the graph by Ta, (g) (A o M M llement-wise product. Although adversarially augmented graphs guarantee environmental diversity,
X o M where O is the broadcasted
5

---

Under review as a conference paper at ICLR 2023
it inevitably destroys the causal parts. Therefore, utilize the causal generator To (-) to capture causal features and combine them with diverse environmental features. Following the sufficiency and independence conditions in Assumption
define the causal learning objective as:
min (Cean Er, (LL v) + ((]()) 6,8²
(11)
where -= (A e M o M*) is the augmented graph. adopts the mask combination strategy: M (1* M M May + M and M = (1* M @ M M where Ma and M cau are generated by Tag (+), 1 and 1s are all-one matrices, and if there is no edge between node
i and node j. then we set 1
1j
Now we explain this combination strategy. Taking M as an example, since M cau denotes the cap- tured causal regions via Tg. (-), 1 M represents the complementary parts. which are environ- mental regions. Matv represents the adversarial perturbation, so (1* - @ M ad is equivalent to applying the adversarial perturbation on environmental features, meanwhile, sheltering the causal features. Finally, +Meau indicates that the augmented data should retain the original causal features. Hence, this combination strategy achieves both environmental diversity and causal invariance. In- specting Equation 11, the first term indicates that causal features are enough for predictions, thus satisfying the sufficiency condition. While the second term encourages causal features to make right predictions after perturbing the environments, thereby satisfying the independence condition.
Regularization. For Equation 10, the adversarial optimization tends to remove more nodes and edges, we should also constrain the perturbations. Although Equation 11 satisfies the suf- ficiency and independence conditions, it is necessary to impose constraints on the ratio of the causal features to prevent trivial solutions. Hence, we first define the regularization function r r(M, k, A) = (Eij Mij/k A) + (Ei, [MM
0J/k - A), where k is the total number of ele- ments to be constrained, I {0, 1} is an indicator function. The first term penalizes the average ratio close X, while the second term encourages an uneven distribution. Given a graph with n nodes and m edges, we define the regularization term for adversarial augmentation and causal learning as:
fregı = Ep,, [p,[r(M
(r(M adv. n. Aa) rrM adv. m, Aa)]:
(12)
Crega Ep,, Ep,[[r(M [r(M.... can n, Xe) r(M can m, Ac)1.
(13)
where Ao (0, 1) is the ratio of causal features, we set A = 1 for adversarial learning, which can alleviate excessive perturbations. The detailed algorithm of AdvCA is provided in Appendix A.I.
4 EXPERIMENTS
In this section, we conduct extensive experiments to answer the following Research Questions:
RQI: Compared to existing efforts, how does AdvCA perform under covariate shift?
Can the proposed AdvCA achieve the principles of environmental diversity and causal invariance, thereby effectively alleviating the covariate shift?
RQ2:
RQ3: How do the different components of AdvCA affect performance?
4.1 EXPERIMENTAL SETTINGS
Datasets. We use graph OOD datasets (Gui et al., 2022) and OGB datasets (Hu et al., 2020), which include Motif, CMNIST, Molbbbp and Molhiv. Following Gui et al. (2022), adopt the base, color, size and scaffold data splitting to create various covariate shifts. The details of the datasets, metrics and implementation details of AdvCA are provided in Appendix A.2 and A.3.
Baselines. We adopt 14 baselines, which can be divided into the following three specific categories:
Generalization Algorithms: Empirical Risk Minimization (ERM), IRM (Arjovsky al., 2019), GroupDRO (Sagawa et al., 2020). VREx (Krueger et al., 2021).
Graph Invariant Learning and Generalization: DIR et al., 2022b), CAL (Sui et al., 2022),
GSAT (Miao et al., 2022), OOD-GNN (Li et al., 2022a), StableGNN (Fan et al., 2021).
Graph Data Augmentation: DropEdge (Rong et al., 2020), GREA (Liu et al., 2022), FLAG (Kong et al., 2022), M-Mixup (Wang et al., 2021), G-Mixup (Han al., 2022).

---

Under review as a conference paper at ICLR 2023
Table 1: Performance on synthetic and real-world datasets. Numbers in bold indicate the best performance, while the underlined numbers indicate the second best performance.
Motif
CMNIST
Molbbbp
Molhiv
Method
base
size
color
scaffold
size
scaffold
size
ERM
68.6614.25
51.74,2.88
28.60.187
18.10.
78.29.3.7
69.58.251
59.94.2.37
IRM
10.65.4.17
4374
27.83.2.13
67.22,1.15
77.56x248
1..97-184
59.00.2.92
GroupDRO
68.24
51.9525.86
29.07=3.14
66.47.2.39
79.27=2.4
70.64=2.57
58.98-2.16
VREx
71.47.6.99
52.67a5.54
28.48=287
68.7421.03
78.76x237
70.77=284
58.53.288
DIR
62.07
52.27
33.20a6.17
66.86x2.25
76.40.4.43
68.07.2.9
58.08.2.88
CAL
65.63..4.9
51.18.5.60
27.99.3.24
68.0612.0
79.50.481
67.37
57.95.2.3
GSAT
62.80
83.20
28.17.1.26
66.78.1.45
75.63.388
68.66.1.35
58 38.06.1.99 1.98
OOD-GNN
61.10=7.57
52.61
26.49x2.94
66.72.1.23
79.48
70.46=197
60.60s3.77
StableGNN
57.07
46.93
28.3823.49
66.7421.30
27.47
18.44
56.7 56.71
DropEdge
45.08
45.6 4..63
22.65=2.90
66.49+1.55
78.32
58,53 70.78=1.38
GREA
56.7449.23
54.13, 10:2
29.02x3.26
69.72.1.66
77.34
67.79
60.71
FLAG
61.12.5.99
51.6614.14
32.30.269
67.694236
79,2612.26
68.45-2.0
00.59.2295
M-Mixup
70.08
51.4824.91
26.47=3.45
68.7520.4
78.92+243
68. 88.882263
59.03 59.0333.11
G-Mixup
59.6627.03
22.81
31.85x5.82
67 67.4421.62
78.55
70.01
59.3442248
AdvCA (ours)
73.64.5.15
55.85
36.37.4.44
70.79
81.03...15
71.15.181
61.64.3.77
Improvement
2.17%
1.72%
3.17%
1.07%
t 1.53%
0.37%
0.93%
4.2
MAIN RESULTS (RQ1)
To demonstrate the superiority of AdvCA, we first make comprehensive comparisons with baseline methods. The implementation settings and details of baselines are provided Appendix A.4. All experimental results are summarized in Table 1. We have the following Observations.
Obsl: Most generalization and augmentation methods fail under covariate shift. Generaliza- tion and data augmentation algorithms perform well certain datasets or shifts. VREx achieves 2.81% 1% improvement on Motif (base). For two shifts of Molhiv, data augmentation methods GREA and DropFidge obtain 1. 1.20% and 0.77% improvements. The invariant learning methods DIR and CAL also obtain 4.60% and 1.53% improvements on CMNIST and Molbbbp (size). Unfortunately, none of the methods consistently outperform ERM. For example, GREA and DropEdge perform poorly on Motif (base), ! 1.92% and 123.58%. DIR and CAL also fail on Molhiv. These show that both invariant learning and data augmentation methods have their own weaknesses, which lead to unstable performance when facing complex and diverse covariate shifts from different datasets.
Obs2: AdvCA consistently outperforms all baseline methods. Compared with ERM, AdvCA can obtain significant improvements. For two types of covariate shifts on Motif, AdvCA surpasses ERM by 4.98% and 4.1 1%, respectively. contrast to the large performance variances on differ- ent datasets achieved by baselines, AdaCA consistently obtains the leading performance across the board. For CMNIST, AdvCA achieves a performance improvement of 3.17% compared to the best baseline DIR. For Motif, the performance is improved by 2.17% and 1.72% compared to VREx and GREA. These results illustrate that AdvCA can overcome the shortcomings of invariant learning and data augmentation. Armed with the principles of environmental diversity and causal invariance, AdvCA achieves stable and consistent improvements on different datasets with various covariate shifts. In addition, although we focus covariate shift in this work, also carefully check the performance of AdvCA under correlation shift, and the results are presented Appendix D.1.
4.3 COVARIATE SHIFT AND VISUALIZATIONS (RQ2)
In this section, we conduct quantitative experiments to demonstrate that AdvCA can shorten the distribution gap, as shown in Figure 1. Specifically, we utilize GCS(..)) as the measurement to quantify the degree of covariate shift. The detailed estimation procedure is provided in Appendix B. To make comprehensive comparisons, we select four different types of covariate shift: base, size, color and scaffold, to conduct experiments. We choose three data augmentation baselines, DropE- dge, FLAG and G-Mixup, which augment graphs from different views. The experimental results are shown in Table 2. We calculated covariate shifts between the augmentation distribution Paug with the training P.r or test distribution P... "Aug- Train"rrnn" and "Aug- Test" represent GCS( (Paug- P.r) and GCS Pangg Pta), respectively. From the results in Table 2, have the following observations.

---

Under review as a conference paper at ICLR 2023
Original graphs
Augmented graphs
Figure 3: Visualizations of the augmented graphs via AdvCA.
Table 2: Covariate shift comparisons with different augmentation strategies.
Motif (base)
Motif (size)
CMNIST (color)
Molhhbp (scaffold)
Method
Aug-Train
Aug-Test
Aug-Train
Aug-Test
Aug-Train
Aug-Test
Aug-Train
Aug-Test
Original
0.557
0.52240421
0.490+0:226
0
0.419x0079
DropEdge
0.772a0213
0.515
0.161au271 0.85
a.627
0.539au.20
0.758au.192
0.737
FLAG
0.001=000:
0.533=0016
0,002-0018
0.507=0121
0.003.0002
0.442z0002
0.001
0,413
G-Mixup
0.690.0186
0.472s000
0.81620154
0.24920343
0.408
0.35 1..31
0.551a0.25
2.545
AdvCA
0.369
0.462:00m
0.649_0.43
0.098
0.51620.066
0.307=0.108
0.422.0.00
0.393
Obs3: AdvCA effectively closes the distribution gap with test distribution. "Original" rep- resents the original training distribution without augmentation. We observe that there exist large covariate shifts between the training and test distributions, ranging from 0.419 to 0.557. DropEdge greatly enlarges Aug-Train, i.e., 0.627-0.851. While it fails reduce Aug-Test, e.g., on CMNIST (0.490-0.539) and Molbbbp (0.419-0.737)). FLAG only perturbs the node features, leading to small values in Aug-Train and an inability to reduce Aug-Test. G-Mixup significantly increases Aug-Train by generating OOD samples, while it cannot guarantee a decrease in Aug-Test. Finally, our proposed AdvCA enlarges the gap with training distribution by augmenting the environmen- tal features. Meanwhile, the invariance of causal features significantly reduces Aug- e.g., Motif-base (0.557--0:462), Motif-size (0.522-00.09) and CMNIST (0.490-~0.408)
To verify the environmental diversity and causal invariance of AdvCA, plot the augmented graphs Figure: 3. These augmented graphs are randomly sampled during training. More visualizations depicted in Appendix D.3. The Motif and CMNIST graphs are displayed the first and second rows. Figure 3 (Left) shows the original graphs. For Motif, the green part represents the motif-graph, whose type determines the label. While the yellow part denotes the gaaeeg that contains environ- mental features. For CMNIST, the red subgraph contains causal features while the complementary parts contain environmental features. Figure 3 (Right) displays the augmented samples during train- ing. Nodes with darker colors and edges with wider lines indicate higher soft-mask values. From these visualizations, we have the following observations.
Obs4: AdvCA can achieve both environmental diversity and causal invariance. We can observe that AdvCA only perturbs the environmental features while keeping the causal parts invariant. For Motif dataset, the base-graph is ladder and the motif-graph is house. After augmentation, the nodes and edges of the ladder graph are perturbed. and stable during training. The CMNIST graph also exhibits the same phenomenon. The envi- ronmental features are frequently perturbed, while the causal subgraph that determines label "2" remains invariant and stable during training. These visualizations further demonstrate that AdvCA can simultaneously guarantee environmental diversity and causal invariance.
contrast, the house part remains invariant
4.4 ABLATION STUDY (RQ3)
Adversarial augmentation v.s. Causal learning. They are two vital components that achieve en- vironmental diversity and causal invariance. The results depicted in Figure (Left). "w/o Adv" and "w/o Cau" refer to AdvCA without adversarial augmentation and without causal learning, re- spectively. RDCA stands for variant that replaces the adversarial augmentation in AdvCA with
8

---

Under review as a conference paper at ICLR 2023
Motif (base)
Molbbbp (scafreld)
Nolhvv (sios)
75-
75-
AdvCA Adv
Peformance
Cau
RDCA
65
60
55-
55
55-
Mouif
Molbbbp
Malhiv
0.1
0.2
0.3
0.4
0.7 0.8
0.9
0.01
0.1
0.2
0.5 1.5
5.0
Causal
ratio
Adversarial penalty
Figure 4: (Left): Performance comparisons different components in AdvCA. (Middle): Perfor- mance different causal ratios Xe- (Right): Performance over different penalties y.
random augmentation (i.e., random masks). Compared to AdvCA, utilizing either causal learning or adversarial augmentation alone will degrade the performance. On the one hand, removing adver- sarial perturbations loses the invariance condition in causal learning, leading to suboptimal causal features. On the other hand, using adversarial augmentation alone will destroy the causal features, thereby impairing generalization. RDCA exceeds ERM, but is worse than AdvCA, suggesting that randomness will also encourage diversity, even if is less effective than the adversarial strategy.
Sensitivity Analysis. The causal ratio Åe and penalty coefficient determine the extent of causal features and the strength of adversarial augmentation, respectively. We also study their sensitivities. The experimental results are shown in Figure (Middle) and (Right). Dashed lines denote the performance of ERM. A with 0.3~0.8 performs well on Motif and Molbbbp, while Molhiv is better in 0.1-0.3. It indicates that the causal ratio is a dataset-sensitive hyper-paramess that needs careful tuning. For the penalty coefficient, the appropriate values on the three datasets range from 0.1-1.5.
5 RELATED WORK
Invariant Causal Learning (Lu et al., 2021) exploits causal features for better generalization. IRM (Arjovsky al., 2019) minimizes the empirical risks within different environments. Chang et al. (2020) minimize the performance gap between environment-
and environment-agnosic pre- dictors discover rationales. Motivated by these efforts, DIR (Wu al., 2022b) constructs multiple interventional environments for invariant learning. GREA (Liu et al., 2022) and CAL (Sui et al., 2022) learn causal features by challenging different environments. However, they only focus on correlation shift issues. The limited environments hinder their successes covariate shift.
Graph Data Augmentation (Ding et al., 2022; Zhao al., 2022; Yoo et al., 2022) enlarges the training distribution by perturbing features in graphs. Recent studies (Ding et al., 2021; Wiles et al., 2022) observe that often outperforms other generalization efforts (Arjovsky et al., 2019; Sagawa et al.. 2020). DropEdge (Rong et al., 2020) randomly removes edges, while FLAG (Kong et al., 2022) augments node features with an adversarial strategy. M-Mixup (Wang et al., 2021) interpolates graphs in semantic space. However, studies (Arjovsky et al., 2019; Lu et al., 2021) point that causal features are the key to OOD generalization. These augmentation efforts are prone to perturb the causal features, which easily loses control of the perturbed distributions. Due the space constraints, we put more discussions about OOD generalization in Appendix F.
6 CONCLUSION & LIMITATIONS
In this work, we focus on the graph generalization problem under covariate shift, which is of great need but largely unexplored. We propose novel graph augmentation strategy, AdvCA, which is based on the principle of environmental diversity and causal invariance. Environmental diversity allows the model to explore more novel environments, thereby better generalizing to possible unseen test distributions. Causal invariance closes the distribution gap between the augmented and test data, resulting in better generalization. We make comprehensive comparisons with 14 baselines and conduct in in-depth analyses and visualizations. The experimental results demonstrate that AdvCA can achieve excellent generalization ability under covariate shift. In addition, we also provide more discussions about the limitations of AdvCA and future work in Appendix G.

---

Under review as a conference paper at ICLR 2023
ETHICS STATEMENT
This paper does not involve any human subjects and does not raise any ethical concerns. We propose graph data augmentation method address the OOD issue of covariate shift. We conduct experi- ments on public datasets and validate the effectiveness on graph classification tasks. Our proposed method can be applied practical applications, such as the prediction of molecular properties.
REPRODUCIBILITY STATEMENT
To help researchers reproduce our results, we provide detailed instructions here. For the implemen- tation process of AdvCA, we provide the detailed algorithm in Appendix A.1. For datasets, we use publicly available datasets, and detailed introduction is provided Appendix A.2. For train- ing details, provide training settings of AdvCA and baseline settings in Appendix A.3 and A.4, respectively. Furthermore, we also provide an anonymous code link: https:/ happ:://anonymous. 4open. wwwsecceee/eeeeeec..ccceeeeeeeee /a/de////A//a-aa/ 6BBBF 68BF
REFERENCES
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In ICML, pp. 214-223, 2017.
Martin Arjovsky, Léon Bottou, Ishaan Guirajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv: 02289,, 2019.
Kurt Binder, Dieter Heermann, Lyle Roelofs, A John Mallinckrodt, and Susan McKay. Monte carlo simulation in statistical physics. Computers in Physics, 762):156-157, 1993.
Shiyu Chang, Yang Zhang, Mo Yu, and Tommi Jaakkola. Invariant rationalization. ICML, pp- 1448-1458, 2020.
Yongqiang Chen, Yonggang Zhang, Han Yang. Kaili Ma, Binghui Xie, Tongliang Liu, Bo Han, and James Cheng. Invariance principle meets out-of-distribution generalization on graphs. arXiv preprint arXiv:2202.0544/ 2022.
Kaize Ding, Zhe Xu, Hanghang Tong, and Huan Liu. Data augmentation for deep graph learning: A survey, arXiv preprint 2rXiv:2202.08235 2022.
Mucong Ding, Kezhi Kong, Jiuhai Chen, John Kirchenbauer, Micah Goldblum, David Wipf, Furong Huang, and Tom Goldstein. A closer look at distribution shifts and oit-s--distiotttioo generaliza- tion on graphs. In NeurlPS Workshop Distribution Shifts: Connecting Methods and Applica- fions, 2021.
Alexey Dosovitskiy and Thomas Brox. Generating images with perceptual similarity metrics based on deep networks. NIPS, 2016
Shaohua Fan, Xiao Wang, Chuan Shi, Peng Cui, and Bai Wang. Generalizing graph neural networks graphs. arXiv preprint arXiv:2/11. 2021.
Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665-673 2020.
Shurui Gui, Xiner Li, Limei Wang, and Shuiwang Ji. Good: A graph out-of-distributiin benchmark. arXiv preprint 2r06v:2206.0844 2022.
Xiaotian Han, Zhimeng Jiang, Ninghao Liu, and Xia Hu. G-mixup: Graph data augmentation for graph classification. In ICML., pp. 8230-8248, 2022.
Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. In NeurIPS, 2020.
10

---

Under review as a conference paper at ICLR 2023
Jean Kaddour, Aengus Lynch, Qi Liu, Matt J Kusner, and Ricardo Silva. Causal machine learning: A survey and open problems. arXiv preprint arXiv:2206.15475. 2022.
Kezhi Kong, Guohao Li, Mucong Ding, Zuxuan Wu, Chen Zhu, Bernard Ghanem, Gavin Taylor, and Tom Goldstein. Robust optimization as data augmentation for large-scale graphs. In CVPR, pp. 60-69, 69, 2022.
David Krueger, Ethan Caballero, Joern-Henrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. Out-of-distribution generalization via risk extrapo- lation (rex). ICML, pp. 5815-5826, 2021.
Haoyang Li, Xin Wang, Ziwei Zhang, and Wenwu Zhu. Ood-gnn: Out-of-distribttion generalized graph neural network. TKDE, 2022a.
Haoyang Li, Xin Wang, Ziwei Zhang, and Wenwu Zhu. Out-of-distrribtion generalization on graphs: survey. arXiv preprint Sttv:2202.079887, 2022b.
Gang Liu, Tong Zhao, Jiaxin Xu, Tengfei Luo, and Meng Jiang. Graph rationalization with environment-based augmentations. In SIGKDD, pp. 1069-1078, 2022.
Chaochao Lu, Yuhuai Wu, José Miguel Hermández-Lobato. and Bernhard Schölkopf. Invariant causal representation learning for oitoof-diittibution generalization. In ICLR, 2021.
Siqi Miao, Miaoyuan Liu, and Pan Li. Interpretable and generalizable graph learning via stochastic attention mechanism. In ICML, pp. 15524-15543 2022.
Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodola, Jan Svoboda, and Michael M Bronstein. Geometric deep learning graphs and manifolds using mixture model cnns. In CVPR, pp. 5115-5124, 2017.
Christopher Morris, Nils M Kriege, Franka Bause, Kristian Kersting, Petra Mutzel, and Marion Neumann. Tudataset: collection benchmark datasets for learning with graphs. 2020.
Emanuel Parzen. On estimation of a probability density function and mode. The annals of mathe- matical statisties, 33(3):1055-1076, 1962.
Yu Rong, Wenbing Huang, Tingyang Xu, and Junzhou Huang. convolutional networks on node classification. In ICLR, 2020.
Dropedge: Towards deep graph
Shiori Sagawa, Pang Wei Koh, Tatsunori Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generaliza- tion. In ICLR, 2020.
Zheyan Shen, Jiashuo Liu, Yue He, Xingxuan Zhang, Renzhe Xu, Han Yu, and Peng Cui. Towards out-of-distribution generalization: survey. arXiv preprint arXiv:2108. 1at::2088.3362.. /3644, 2021.
Aman Sinha, Hongseok Namkoong, and John Duchi. Certifying some distributional robustness with principled adversarial training. In ICLR, 2018.
Yongduo Sui, Xiang Wang, Jiancan Wu, Xiangnan He, and Tat-Seng Chua. Causal attention for interpretable and generalizable graph classification. In SIGKDD, pp. 1696-1705, 2022.
Susheel Suresh, Pan Li, Cong Hao, and Jennifer Neville. Adversarial graph augmentation to improve graph contrastive learning. NeurlPS, pp. 15920-15933,, 2021.
Riccardo Volpi. Hongseok Namkoong, Ozan Sener, John C Duchi, Vittorio Murino, and Silvio Savarese. Generalizing unseen domains via adversarial data augmentation. NeurIPS. 2018.
Yiwei Wang, Wei Wang, Yuxuan Liang, Yujun Cai, and Bryan Hooi. Mixup for node and graph classification. In WWW. pp. 3663-3674, 2021.
Olivia Wiles, Sven Gowal, Florian Stimberg, Sylvestre Alvise-Rebutee Ira Ktena, Taylan Cemgil, et al. A fininnd analysis distribution shift. In ICLR, 2022.
11

---

Under review as a conference paper at ICLR 2023
Qitian Wu, Hengrui Zhang, Junchi Yan, and David Wipf. Handling distribution shifts graphs: An invariance perspective. ICLR, 2022a.
Yingxin Wu, Xiang Wang, An Zhang, Xiangnan He, and Tat-Seng Chua. Discovering invariant rationales for graph neural networks. In ICLR, 2022b.
Zhengin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh Pappu, Karl Leswing, and Vijay Pande. Moleculenet: benchmark for molecular machine learn- ing. Chemical science, 9(2):113-53) 2018.
Keyulu Xu, Weihua Hu, Jure Leskovee, and Stefanic Jegelka. How powerful are graph neural networks? In ICLR, 2019.
Nanyang Ye, Kaican Li, Haoyue Bai, Runpeng Yu, Lanqing Hong, Fengwei Zhou, Zhenguo Li, and Jun Zhu. Ood-bench: Quantifying and understanding two dimensions of out-of-dditrribtion generalization. In CVPR, pp. 7947-7958, 2022.
Jaemin Yoo, Sooyeon Shim, and U Kang. Model-agnostic augmentation for accurate graph classifi- cation. In WWW. pp. 1281-1291, 2022.
Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph contrastive learning with augmentations. NeurIPS, 2020.
Junchi Yu, Jian Liang, and Ran He. Finding diverse and predictable subgraphs for graph domain generalization. arXiv preprint arXiv:2206.09345 2022.
Xingxuan Zhang, Peng Cui, Renzhe Xu, Linjun Zhou, Yue He, and Zheyan Shen. Deep stable learning for out-of-distribution generalization. In CVPR, pp. 5372-5382, 2021.
Tong Zhao, Gang Liu. Stephan Günnemann, and Meng Jiang. Graph data augmentation for graph machine learning: A survey, arXiv preprint arXiv:2202.0888 088777 2022.
Yanqiao Zhu, Yuanqi Du, Yinkai Wang, Yichen Xu, Jieyu Zhang, Qiang Liu, and Shu Wu. A survey deep graph generation: Methods and applications. arXiv preprint urXiv:200.066114 2022.
12

---

Under review as a conference paper at ICLR 2023
A IMPLEMENTATION DETAILS
A.1 ALGORITHM
We summarize the detailed implementations of AdvCA in Algorithm Inspired by Suresh et al. (2021), we alternately optimize the adversarial augmenter and causal generator with the backbone model, in lines 13 and 14. We adopt the causal features for predictions in the inference stage.
Algorithm 1: Adversarial Causal Augmentation
Require:
Training set D.. Adversarial augmenter To, (-); Causal generator (-); GNN classifier f()) with parameters 8; Learning rates a, B; Batch size N; Causal ratio Xe; Penalty 7.
1: Randomly initilize 6, 01. 0/
2: while not converge do
3: Sample batch Bir ((gt, Bc) Dtr
4: for each (g., 3/4 Bu do
5: M Adv To, (gi) // adversarial perturbations
6: M M can + To (g:) // regions of causal features
7:
M (1 M cau M adv M // augment edges
8:
M + (1 M o M + M // augment nodes
9:
(A; O M X, o M // augmented graph
end for
11:
Compute Ladv Lregs via Equation 10 and Equation 12
12:
Compute Leau + Leg, via Equation 11 and Equation 13
13:
Update parameters of adversarial augmenter via gradient ascent:
01 + 81 + aV (Lady - Lreg,)
Update parameters of GNN and causal generator via gradient descent:
0 - 0 BV o Cran + ): 02 <- 02 3Vo, fran Lrega)
end while
A.2 DATASETS AND METRICS
Datasets. In this paper, we conduct experiments on graph OOD datasets (Gui al., 2022) and OGB datasets (Hu et al., 2020), which include Motif, CMNIST, Molbbbp and Molhiv. We follow Gui et al. (2022) to create various covariate shifts, according to base, color, size and scaffold splitting, Base, color, size and scaffold are features of the graph data and do not determinc the labels of the data, so they can be regarded as environmental features. The statistics of the datasets are summarized in Table 3. Below give brief introduction each dataset.
Motif: It is a synthetic dataset from Spurious-Motif (Wu et al., 2022b; Sui et al., 2022). As shown in original graphs in Figure 5, each graph is composed of a base mase-graph (wheel, tree, ladder, star, path) and motif (house, cycle, crane). The label only determined by the type of motif. We create covariate shift according to the base-graph type and the graph size (i.e., node number). For base covariate shift, we adopt graphs with wheel, tree, ladder base-graphs for training, star for validation and path for testing. For size covariate shift, we use small-size of graphs for training, while the validation and the test sets include the middle- and the large-size graphs, respectively.
CMNIST: Color MNIST dataset contains graphs transformed from MNIST via superpixel tech- niques (Monti et al., 2017). We define color as the environmental features to create the covariate color digits with 7 different colors, where five of them are adopted for training while the remaining two are used for validation and testing.
shift. Specifically,
Molbbbp & Molhiv: These are molccular datasets collected from MoleculeNet (Wu et al., 2018). We define the scaffold and graph size (i.e., node number) as the environmental features create two types of covariate shifts. For scaffold shift, we follow (Gui et al., 2022) and use scaffold split create training, validation and test sets. For size shift, adopt the large-size of graphs for training and the smaller ones for validation and testing.
Metrics. We adopt classification accuracy as the metric for Motif and CMNIST. As suggested by Hu al. (2020), ROC-AUC for Molhiv and Molbbbp datasets. In addition, we use GCS(P, (P
13

---

Under review as a conference paper at ICLR 2023
Table 3: Statistics of graph classification datasets.
Dataset
Motil
CMNIST
Molbbbp
Molhiv
Covariate shift
basc
size
color
scaffold
size
scaffold
size
Graphit
18000
18000
42000
1631
1633
24682
26169
Train
Avg. node# 17.07
16.93
75.00
22.49
27.02
26.25
27.87
Avg. edgeff
48.89
43.57
1392.76
48.43
58.71
56.68
60.20
Graph# 3000
3000
7000
204
203
4113
2773
Val
Avg. nodeit 15.82
39.22
75.00
33.20
12.06
24.95
15.55
Avg. edge#
33.00
107.03
1393.73
71.84
24.27
54.53
32.77
Graph# 3000
3000
7000
204
203
4108
3961
Test
Avg. node#
14.97
87.18
75.00
27.51
12.26
19.76
12.09
Avg. edge#
31.54
239.65
1393.60
59.75
24.87
40.58
24.87
Class#
3
10
2
2
2
2
Table 4: Hyper-parameter details of AdvCA.
Dataset
Motif
CMNIST
Molbbbp
Molhiv
Covariate shift
base
size
color
scaffold
size
scaffold
size
Backbone (layer-hidden
4-300
4-300
4-300
4-64
4-32
4-128
4-128
Augmenter (layer-hidden)
2-300
2-300
2-300
2-64
2-32
2-128
2-128
Generator (layer-hidden)
2-300
2-300
2-300
2-64
2-32
2-128
2-128
Epoch
100
100
100
100
100
100
100
Optimizer
Adam
Adam
Adam
Adam
Adam
Adam
Adam
Batch size
512
512
512
32
128
32
512
Learning rate
le-3
1e-3
le-3
1e-3
5c-3
1e-3
lc-2
Learning rate B
5e-3
5e-3
5e-3
1c-3
5e-3
le-2
le-2
Causal ratio A₂
0.5
0.5
0.5
0.5
0.5
0.1
0.1
Adversarial penalty
0.2
0.2
0.2
0.5
0.5
0.5
0.5
to measure the covariate shift between distributions P and Q. For all experimental results, perform 10 random runs and report the mean and standard derivations.
A.3 TRAINING SETTINGS
We use the NVIDIA GeForce RTX 3090 (24GB GPU) to conduct our experiments. make fair comparison, we adopt GIN (Xu et al., 2019) as the default architectures to conduct all ex- periments. We tune the haper-parameters in the following ranges: a and 3 (0.01, 0.005, 0.001); A2 {0.1,..., 0.9); y {0.01,0.1 hidden layers E (32, 64, 128, 300). The hyper-parameters of AdvCA are summarized in Table 4.
0.5, 1.0, 1.5, 2.0, 3.0, 5.0); batch size {32, 128, 256, 512);
A.4 BASELINE SETTINGS
For more comprehensive comparison, we selected baselines. In this section, we give detailed introduction to the settings of these methods.
For ERM, IRM (Arjovsky al., 2019). GroupDRO (Sagawa al., 2020), VREx (Krueger al., 2021), and M-Mixup (Wang et al., 2021). we report the results from the study (Gui et al., 2022) by default and reproduce the missing results Molbbbp.
For DIR (Wu al., 2022b), CAL (Sui al., 2022), GSAT (Miao al., 2022), DropEdge (Rong et al., 2020), GREA (Liu et al., 2022), FLAG (Kong et al., 2022) and G-Mixup (Han et al., 2022), they provide source codes for the implementations. We adopt default settings from their source codes and detailed hyper-parameters from their original papers.
For OOD-GNN (Li et al., 2022a) and StableGNN (Fan et al., 2021), their source codes are not publicly available. We reproduce them based the codes of StableNet (Zhang et al., 2021).
14

---

Under review as a conference paper at ICLR 2023
For RDCA in Section 4.4, it is variant that replaces the adversarial augmentation in AdvCA with random augmentation. In our implementation, we use all-one matrices to create the initial node and edges masks. Then randomly set 20% of nonzero elements to zero in these masks. Finally, we apply these masks to the graphs for random data augmentation. The process of causal learning is consistent with AdvCA.
B ESTIMATION OF GRAPH COVARIATE SHIFT
In this section, elaborate the implementation details of estimating the graph covariate shift. Without loss of generality, we start with the example of estimating the graph covariate shift between the training and test distributions. Given training set and test set D.. and Des they follow probability distribution functions Per and Pte. The process of estimating GCS( Ptr, Pto) is summarized in the following two steps:
Firstly, it is intractable to directly estimate the distribution in graph space G. Inspired by Ye et al. (2022), we can obtaín the graph features and estimate the distribution in feature space F. Specifically, given sample, we train binary GNN classifier / to distinguish which distribution it comes from, where f(-) = 2 h, h(-) G + FF is a graph encoder, and Φ(.) :F + (0,1) 1] is binary classifier. Then
adopt the pre-trained GNN encoder h to extract graph features.
Secondly, we prepare the features and estimate the distribution of the data via Kernel Density Es- timation (KDE) (Parzen, 1962). Finally, we adopt the Monte Carlo Integration under importance sampling (Binder et al., 1993) to approximate the integrals in Definition 1.
We summarize these implementations in Algorithm 2. In lines and 5, to avoid the label shift (Ye al., 2022), adopt sample reweighting to ensure the balance of each class.
Algorithm 2: Estimation of Graph Covariate shift
Require: Training dataset Der and test dataset D..: Batch size N; Loss function E; GNN f T o h; Importance sampling size M; Threshold c.
Ensure: Estimated covariate shift GCS( Pte).
1: Initialize parameters of
2: # Train graph classifier
3: while not converge do
4: Sample batch Bir +- ((], v1) N i=1 DDr and relabel all 0
5: Sample a batch Bi + [9), y1) ) = Dwo and relabel all y:
6: B Brr Bie
7:
for each (gs, Jii do
8: Compute loss ((f(g)) ys) and pack-propagate gradients
9: end for
10: Update the parameters of via gradient descent and reset the gradients
11: end while
12: # Prepare the features for the estimation
13: Extract training and test feature sets Für and Fre via encoder h
14: F F Fer F
Scale F zero mean and unit variance
16: w fit by KDE the distribution of o
Split F recover the original partition F Fr
18: Pur, Pie + fit by KDE the distributions of Fl
19: # Estimate the covariate shift
20: Initialize GCS( Ptr,, Pta) 0
21: fort (1, M) do
22: 2 sample from s
23: if P:) < (z) < then
24:
GCS(PL. P..) + GCS(Pu,. Po.) (P) (1)(2))/((22)
25: end if
26: end for
GCS(Pu. Pio) GCS(Ptr; Pro)22M
15

---

Under review as a conference paper at ICLR 2023
C CORRELATION SHIFT & COVARIATE SHIFT
From Assumption 1. we can observe that environmental features easily change outside the train- ing distribution, owing to their noncausal nature. Hence, distribution shifts are only caused by the environmental features. Specifically, we define the joint distribution of training and test data as Pc. (G. and YY. respectively. Since their joint distribution can be rewritten as (G) (G, and Pi,(G,Y) =P..(Y we can find that there exist two main reasons that lead to distribution shift Pr,(G,Y) (Y)
Correlation shift (Y # (V]() If the statistical correlation of environmental features and labels is inconsistent in training and test data, well-fitted model in training data may fail in test data, which is also known as spurious correlation or correlation shift (Ye et al., 2022). Formally, correlation shift describes the conditional distribution Pu (Y|G) Pie (Y]G).
Covariate shift (G) (G) If there exist environmental features in the test distribution that the model has not seen during training, it will also result in performance drop. This unseen distribution shift is well known as covariate shift (Gui al., 2022). means that the environ- mental features in test data are unseen in training data, which leads to (G) # Pin(G). Hence, in Assumption 1, we quantitatively measure the covariate shift between Pir (G) and (G)
D MORE EXPERIMENTAL RESULTS
D.1 RESULTS ON CORRELATION SHIFT
Although this work focuses on the OOD is- sue of covariate shift, for completeness, we also evaluate the performance of AdvCA un- der correlation shift. Following Gui et al. (2022), we choose three graph OOD datasets (i.e., Motif, CMNIST, Molhiv) with three dif- ferent graph features (i.e., base, color, size) create correlation shifts. For baselines, we choose three generalization algorithms (i.e., ERM, IRM (Arjovsky et al., 2019), VREx (Krueger et al., 2021)), three graph general- ization methods (i.e., DIR (Wu et al., 2022b), CAL (Sui et al., 2022), OOD-GNN (Li et al., 2022a)) and three data augmentation methods (i.e., DropEdge (Rong et al., 2020), FLAG (Kong et al., 2022), M-Mixup (Wang et al., 2021)). The experimental results are shown in Table 5.
Table 5: Performance comparisons on synthetic
and real-world datasets with correlation shift.
Method
Motif
CMNIST
Molhiv
ERM
81.44=2.51
42.87x1.7
63.26s1.25
IRM
80.71
42.80
59.90=1.17
VREx
81.56-214
43.31
60.23.160
DIR
82.25
44.87+1.56
64.65+134
CAL
11.94
41.8240.85
62.361142
DOD-GNN
80.22-2.8
39,03+1.24
57.49x1.08
DropEdge
78.97
38.4311.94
54.92
FLAG
80.91=1.01
43.41=1.38
66.44x2.32
M-Mixup
77.63=1.12
40.96.1.21
64.87.1.6
AdvCA (ours)
82.51=281
49.73.1.70
68.11
We can observe that AdvCA can also effectively alleviate the correlation shift. These results demon- strate that AdvCA learns better causal features by encouraging environmental diversity, which can effectively break spurious correlations that are hidden in the training data.
D.2 RESULTS ON COMMONLY USED DATASETS
To demonstrate the effectiveness of the proposed AdvCA, we also conduct experiments on com- monly used TU datasets (Morris et al., 2020), which include MUTAG, NCII, PROTEINS, COL- LAB, IMDB-B, IMDB-M. These real world datasets and have negligible distribution shift. For training settings, we follow CAL (Sui et al., 2022) and adopt GIN (Xu et al., 2019) as our backbone model. The experimental results are shown in Table 6. For the results, method can achieve the best performance over different datasets.
observe that our
16

---

Under review as a conference paper at ICLR 2023
Table 6: Performance comparisons TU datasets.
Method
MUTAG
NCII
PROTEINS
COLLAB
IMDB-B
IMDB-M
ERM
89.42.7.40
81.7
73.21
8..08
73.40.3.78
51.53.2.97
CAL
89.9 =9.91
83.89+1.9
76.92+3.1
82.68+1.25
74.13+5.1
52.60+2.36
DropEdge
86. 19:41
82.35+3.77
74.40=3.10
80,59+2.14
72.34a5.83
51,06+3.04
FLAG
89.45=7.20
82.67+2.12
76.89=3.66
82,48
73.37=494
52.16+
M-Mixup
89. 83-73777
83,89+2.38
76.76=3440
82,90=1.43
74.07=4.76
52.89+281
AdvCA (ours)
90.34.7.75
84.12:2.64
77.92.3.72
82.98
74.23:5.10
53.02.2.76
Original graphs
Augmented graphs
Figure 5: Visualizations of the augmented graphs via AdvCA.
D.3 MORE VISUALIZATIONS
We display more visualizations of the augmented graph via AdvCA in Figure 5. To demonstrate the superiority our method, also visualize the captured causal features by AdvCA and compare with other baselines. The results are displayed in Figure 6. From the results, we can easily observe that our method can find causal parts more accurately than other baseline methods.
E COMPLEXITY ANALYSES
Firstly, we define the average numbers of nodes and edges per graph in the dataset to be n and m, respectively. Let N denote the batch size, la and 4. denote the numbers of layers in the GNN backbone, adversarial augmenter and causal generator, respectively. d, d, and d, are the dimensions of hidden layers in the GNN backbone, adversarial augmenter and causal generator, respectively.
Time complexity. The time complexity of the adversarial learning objective is ONN(mmd (lamd, + 2lmd)). For the causal learning objective, the time complexity is O 2lmd)). For the regularization terms, the time complexity D(2N(nz + m)). For simplicity, we assume L = 4, and m)).
d, de. Hence, the time complexity of forward propagation is O22N(lamd
(land, + 2mmd
17

---

Under review as a conference paper at ICLR 2023
Original graphs
GREA
CAL
DropEdge
AdvCA (Ours)
Figure 6: Visualization comparisons.
Model size. In addition to the GNN backbone model, we also introduce two small networks for adversarial augmentation and causal learning. In our implementations, the parameters of AdvCA are around twice as large those of the original GNN model.
F MORE RELATED WORKS
OOD Generalization (Shen et al., 2021) has been widely explored. Recent studies (Ye al., 2022; Gui et al., 2022; Wiles et al., 2022) point out that OOD falls into two specific categories: correlation shift and covariate shift. Correlation shift denotes that the environmental features and labels establish a statistical correlation that is inconsistent in training and test data. Thus, the models prefer to learn spurious correlations and rely on shortcut features (Geirhos al., 2020) for predictions, resulting in a large performance drop. In contrast, covariate shift indicates that there exist unseen environmental features test data. The limited training environment makes this issue intractable. In recent years, OOD generalization on graphs is drawing widespread attention (Li et al., 2022a;b; Fan et al., 2021; Wu et al., 2022a;b; Miao et al., 2022; Yu al., 2022: Sui et al., 2022; Liu al., 2022: Chen al., 2022). However, these efforts mainly focus on correlation shift. While the issue of graph covariate shift is of great need but largely unexplored.
Comprehensive Comparisons with EERM (Wu al., 2022a). Although EERM share similar goals with us, generating several environments through augmentation, there exist many technical and contribution differences. Firstly, EERM ignores the distinction between correlation shift and covariate shift problems, so it is not specifically designed for covariate shift. Different from them, distinguish these two shifts in detail and design novel framework specifically for covariate shift. Secondly, EERM does not model causal and environmental features, which results in the in- ability explicitly distinguish them. In contrast, explicitly model the environmental and causal features. Hence, we can effectively identify causal and environmental features and explicitly sepa- rate them from data. Thirdly, we also design a metric, GCS( P), which can effectively measure the diversity of the environmental features for our augmented data. And we directly encourage the environmental diversity the augmented samples by maximizing GCS(P, P). However, EERM does not provide any evaluation metric for environmental diversity. To encourage the diversity, they "blindly" maximize the variance of the empirical risk K environments. Finally, for generalization scope, EERM is based on the IRM (Arjovsky et al., 2019) by minimizing the empirical risk in K en- vironments. In contrast, inspired by DRO (Sagawa et al., 2020), we can guarantee the generalization within the robust radius p. We summarize the above detailed discussions in Table 7.
G LIMITATION & FUTURE WORK
Although AdvCA outperforms numerous baselines and achieve outstanding performance under various covariate shifts, we also prudently introspect the following limitations of our method. And leave the improvements of these limitations future work.
18

---

Under review as a conference paper at ICLR 2023
Table 7: Comparisons with EERM.
EERM
Our AdvCA
Is it specifically designed for covariate shift?
Scope
Can euvironmenttal/casaal
Separability
features be separated?
Can environmental features
be identified explicilly?
How to model
Mask model Ta (-)
Environmental Diversity
environmental features?
Metric for environmental diversity
GCS(P,P)
Generation principle
"Blindly" maximize V V.[H(e)]
Maximize GCS(P,P)
environmental features
Can causal Teatures
be identified explicitly?
Causal Invariance
How model
Mask model (
causal features?
Learning principles for
ming W.[[(e)]
Sufficieneylndeeeendenee
causal features
Theoretical basis
IRM
DRO Robust radius p D)P,P)
Generalization
Generalization scope
environments
AdvCA performs OOD exploration through an adversarial data augmentation strategy to achieve environmental diversity. However, it only perturbs the existing graph data in given training set, such as perturbing original graph node features or graph structures. Hence, it is possible that there still exist some overlaps between the augmented distribution and training distribution, Principle cannot be thoroughly achieved. In future work, we will attempt to design more advanced data augmentation methods, such graph generation-based strategies (Zhu et al., 2022), generate more unseen and novel graph data, for pursuing Principle 1.
For model training, we adopt adversarial training and causal learning to alternately optimize the adversarial augmenter, causal generator and backbone GNN. This training strategy may make the training process unstable, so the performance AdvCA may experience large variance. In addition, these networks also involve additional parameters. Optimizing these parameters separately will also increase the time complexity, as shown in Appendix E. Hence, in future work, we will explore how utilize more advanced optimization methods and lightweight models to achieve the principles of environmental diversity and causal invariance.
19


Reviews:
[{"review_id": "MwZOfzEFKnG", "reviewer": "Reviewer_gnLo", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper addresses the problem of covariate shift on graphs. This framework is compared with 14 baselines on both synthetic and real-world data. ", "summary_of_the_review": "This paper introduces a framework that addresses covariate shift. They provide baselines to compare their framework to. However, some of the baselines are not exactly designed for their task. There are some false claims in the paper that should be justified. ", "strength_and_weaknesses": "Strength:\n- This paper addresses a rather important and interesting problem.  \n- Many baselines are provided for comparison. \n-----------------------------------------------------------\nWeakness:\n- This paper is poorly written and can be very hard to follow at times. \n- This paper seems to be very incremental. \n- Seems like the problem is not well-justified. For instance, why is covariate shift different from correlation shift in terms of causal features and environment? \n- This paper has many false claims, for example, causal features are NOT invariant across domains.\n- While causal features in images are easy to distinguish, how would one define them on graphs? Better examples and justifications are needed. \n- Contributions are not justified. Are you addressing the flaws of the existing frameworks? If that is the case, these frameworks are not designed for the specific task defined in this paper. The same goes for the experiments. \n- Definition 1 seems not to make sense. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "KTK3LJzXWr", "reviewer": "Reviewer_Jykb", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper introduces a method of dealing with covariate shift in graph learning. To this end, the authors define \"causal\" and \"environmental\" features (i.e. features that influence label prediction and ones that are conditionally independent of the labels given the former). Subsequently, desiderata of \"environmental diversity\" and \"causal invariance\" are introduced and an adversarial learning scheme is devised that leads to learning graphs that keep their \"causal features\" which should translate to superior out-of-distribution generalization. This is then confirmed in a series of experiments on benchmark data that compare the proposed approach to competing methods.\n", "summary_of_the_review": "This paper describes a new method of graph classifier learning by adversarially generating distributions that preserve features that have predictive power towards the labels. The authors provide extensive experiments.", "strength_and_weaknesses": "STRENGTHS\n\nThis paper tackles an important problem of robust learning of graph classifiers.\n\nStarting with the definitions in Section 2, the paper is easy to follow.\n\nThe experiments are extensive and compare the approach to a number of baselines.\n\n\nWEAKNESSES\n\nBeing unfamiliar with graph classifier learning, I found it difficult to develop an intuition as to what \"environmental\" and \"causal\" features are when reading the introduction and Figure 1. It only became clear when definitions were provided.\n\nHaving said that, I still think it is a bit of a misnomer to call the features \"causal\", it appears to me that they are simply the relevant ones when it comes to label prediction, the \"environmental\" ones being simply conditionally independent (and thus irrelevant for classification).\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "3SV4Latr33", "reviewer": "Reviewer_rgVp", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper focus on a specific type of OOD issue, covariate shift. The authors propose Adversarial Causal Augmentation (AdvCA) to alleviate the covariate shift. It adversarially augments the data to explore diverse distributions of the environments. Meanwhile, it keeps the causal features invariant across diverse environments. Extensive experimental results on synthetic and real-world datasets prove the effectiveness of AdvCA.", "summary_of_the_review": "The paper focuses on the graph generalization problem under covariate shift, which is of great need but largely unexplored. However, there are some minor issues to be addressed.", "strength_and_weaknesses": "Strengths:\n1. The paper focuses on the graph generalization problem under covariate shift, which is of great need but largely unexplored.\n2. Comprehensive experiments with 14 baselines are conducted to demonstrate that AdvCA can achieve better generalization ability under covariate shift.\n\nWeaknesses:\n1. Experiments on more commonly used datasets like MUTAG and NCI1 are suggested to better demonstrate the effectiveness. \n2. The example “ladder” and “tree” are absent in Figure 1, but are mentioned in paragraph 2, which make readers confused.\n3. In Definition 1, since the prediction is independent of the environment features, you can somehow learn to distinguish the causal feature and environment features. Why don’t you only keep the causal feature to make a prediction?\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "1jb3FEcBlo", "reviewer": "Reviewer_5fRb", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a new method for out-of-distribution (OOD) generalization on graphs. The authors claim that the existing works mainly focus on the OOD issue of correlation shift, while another type, covariate shift, remains largely unexplored. Then, a graph augmentation strategy called AdvCA is proposed to handle the covariate shift problem with an adversarial augmenter and a causal generator. Experimental results show that the proposed AdvCA can outperform several baselines.", "summary_of_the_review": "In summary, this paper focuses on an important problem and proposes an effective method to tackle this problem, but there are concerns regarding the technical novelty, clarity, and quality.", "strength_and_weaknesses": "Pros:   \n1. The OOD generalization problem of graph machine learning is an important and trending research topic. I appreciate the authors' efforts in formalizing the problem and proposing necessary assumptions for graph covariate shifts.  \n2. The authors propose a causal framework to formulate the problem and guide the graph augmentation.  \n3.  Both synthetic and real datasets are used to show the empirical effectiveness of the proposed approach.   \n\nCons:  \nHowever, I have some concerns:  \n1. The main concern is regarding technical novelty. I think the problem formulation is similar to previous works, e.g., EERM (Wu et al., ICLR 2022a), so the formulation should not be considered a particular contribution. The authors claim that the existing graph generalization efforts cannot guarantee both environmental diversity and causal invariance, but EERM has explored these key concepts and its formulation for ego-graphs can be straightforwardly extended to the graph-level tasks.   \n2. The proposed method is somehow incremental and thus the technical contribution of the paper is limited. In particular:  \n(1) The idea of adversarial data augmentation is similar to EERM. More comparisons and discussions regarding related works are expected to further clarify the novelty of the proposed method.   \n(2) The authors claim that the existing works cannot guarantee environmental diversity while the proposed AdvCA can. However, why AdvCA can increase diversity than augmentation-based baseline methods such as GREA is not clearly discussed.   \n3. The experiments are not convincing enough. The experimental results of baselines seem to be inconsistent with the results in the original papers. The details regarding the hyper-parameters of baselines are not clearly present, which raises potential concerns in fair comparisons. The visualizations should also consider some baselines to verify whether AdvCA finds causal parts more accurately than baselines.  \n4. I would also suggest the authors proofread Figure 2 since the plotted environmental features are identical and not diverse.  \n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject