SlowGuess/NeurIPS_2023_OCR huggingface dataset:
where  $\widetilde{g} = (\mathbf{A} \odot \widetilde{\mathbf{M}}^a, \mathbf{X} \odot \widetilde{\mathbf{M}}^x)$  is the augmented graph. It adopts the mask combination strategy:  $\widetilde{\mathbf{M}}^a = (\mathbf{1}^a - \mathbf{M}_{\mathrm{sta}}^a) \odot \mathbf{M}_{\mathrm{adv}}^a + \mathbf{M}_{\mathrm{sta}}^a$  and  $\widetilde{\mathbf{M}}^x = (\mathbf{1}^x - \mathbf{M}_{\mathrm{sta}}^x) \odot \mathbf{M}_{\mathrm{adv}}^x + \mathbf{M}_{\mathrm{sta}}^x$ , where  $\mathbf{M}_{\mathrm{sta}}^a$  and  $\mathbf{M}_{\mathrm{sta}}^x$  are generated by  $T_{\theta_2}(\cdot)$ ,  $\mathbf{1}^a$  and  $\mathbf{1}^x$  are all-one matrices, and if there is no edge between node  $i$  and node  $j$ , then we set  $\mathbf{1}_{ij}^a$  to 0. Now we explain this combination strategy. Taking  $\widetilde{\mathbf{M}}^x$  as an example, since  $\mathbf{M}_{\mathrm{sta}}^x$  denotes the captured stable regions via  $T_{\theta_2}(\cdot)$ ,  $\mathbf{1}^x - \mathbf{M}_{\mathrm{sta}}^x$  represents the complementary parts, which are environmental regions.  $\mathbf{M}_{\mathrm{adv}}^x$  represents the adversarial perturbation, so  $(\mathbf{1}^x - \mathbf{M}_{\mathrm{sta}}^x) \odot \mathbf{M}_{\mathrm{adv}}^x$  is equivalent to applying the adversarial perturbation on environmental features, meanwhile, sheltering the stable features. Finally,  $+\mathbf{M}_{\mathrm{sta}}$  signifies that the augmented data should preserve the original stable features. Consequently, it satisfies both principles. Upon analysis of Equation (8), the first term implies that the stable features are sufficient for making right predictions. The second term promotes right and invariant predictions under generated environments utilizing stable features.
Regularization. For Equation (7), the adversarial optimization tends to remove more nodes and edges, so we should also constrain the perturbations. Although Equation (8) satisfies the sufficiency and invariance conditions, it is necessary to impose constraints on the ratio of the stable features to prevent trivial solutions. Hence, we first define the regularization function  $r(\mathbf{M}, k, \lambda) = (\sum_{ij} \mathbf{M}_{ij} / k - \lambda) + (\sum_{ij} \mathbb{I}[\mathbf{M}_{ij} > 0] / k - \lambda)$ , where  $k$  is the total number of elements to be constrained,  $\mathbb{I} \in \{0,1\}$  is an indicator function. The first term penalizes the average ratio close to  $\lambda$ , while the second term encourages an uneven distribution. Given a graph with  $n$  nodes and  $m$  edges, we define the regularization term for adversarial augmentation and stable feature learning as:
$$
\mathcal {L} _ {\operatorname {r e g} _ {1}} = \mathbb {E} _ {P _ {\mathrm {t r}}} \left[ r \left(\mathbf {M} _ {\mathrm {a d v}} ^ {x}, n, \lambda_ {a}\right) + r \left(\mathbf {M} _ {\mathrm {a d v}} ^ {a}, m, \lambda_ {a}\right) \right], \tag {9}
$$
$$
\mathcal {L} _ {\mathrm {r e g} _ {2}} = \mathbb {E} _ {P _ {\mathrm {t r}}} \left[ r \left(\mathbf {M} _ {\mathrm {s t a}} ^ {x}, n, \lambda_ {s}\right) + r \left(\mathbf {M} _ {\mathrm {s t a}} ^ {a}, m, \lambda_ {s}\right) \right], \tag {10}
$$
where  $\lambda_s \in (0,1)$  is the ratio of stable features, we usually set  $\lambda_a = 1$  for adversarial learning, which can alleviate excessive perturbations. The algorithm is provided in Appendix D.1.
4 Theoretical Discussions
In this section, we engage in theoretical discussions to elucidate our learning objective and its connections with the covariate shift. We first explore the relationship between our optimization objective and the discrepancy principle. Recalling the optimization objective of Equation 3, we aspire to identify a distribution  $\widetilde{P}$  that can manifest within a Wasserstein ball [36], which is centered on distribution  $P$ , with distance  $\rho$  serving as the radius. Under appropriate conditions, we find that our learning optimization objective can establish a close connection with OOD exploration.
Proposition 4.1 Consider a probability distribution  $P$  defined over a measurable space  $(\Omega, \mathcal{F})$ , where  $\Omega$  denotes the sample space and  $\mathcal{F}$  is a  $\sigma$ -algebra on  $\Omega$ . We construct two Wasserstein balls with  $P$  at their center and radii  $\rho_1$  and  $\rho_2$  respectively. Utilizing Equation 3, we generate two distinct distributions,  $\widetilde{P}_1$  and  $\widetilde{P}_2$ , within the space  $(\Omega, \mathcal{F})$ . If (i)  $P$  is an isotropic distribution; (ii)  $\exists x_1, x_2 \in \Omega$  such that  $P(x) = \widetilde{P}_1(x - x_1) = \widetilde{P}_2(x - x_2)$ ; (iii)  $\rho_1 < \rho_2$ , then we have  $\mathrm{GCS}(P, \widetilde{P}_1) \leq \mathrm{GCS}(P, \widetilde{P}_2)$ .
This suggests that by appropriately increasing the robustness radius in AIA, we can effectively amplify the covariate shift between the training and generated distributions. This in turn underscores the reliability of our discrepancy principle, to a certain degree. Comprehensive proofs and detailed discussions supporting these conclusions can be found in Appendix B.
Table 1: Performance on synthetic and real-world datasets. Numbers in bold indicate the best performance, while the underlined numbers indicate the second best performance.
Type	Method	Motif	CMNIST	Molbbbp	Molhiv
base	size	color	scaffold	size	scaffold	size
General Generalization	ERM	68.66±4.25	51.74±2.88	28.60±1.87	68.10±1.68	78.29±3.76	69.58±2.51	59.94±2.37
IRM	70.65±4.17	51.41±3.78	27.83±2.13	67.22±1.15	77.56±2.48	67.97±1.84	59.00±2.92
GroupDRO	68.24±8.92	51.95±5.86	29.07±3.14	66.47±2.39	79.27±2.43	70.64±2.57	58.98±2.16
VReX	71.47±6.69	52.67±5.54	28.48±2.87	68.74±1.03	78.76±2.37	70.77±2.84	58.53±2.88
Graph Generalization	DIR	62.07±8.75	52.27±4.56	33.20±6.17	66.86±2.25	76.40±4.43	68.07±2.29	58.08±2.31
CAL	65.63±4.29	51.18±5.60	27.99±3.24	68.06±2.60	79.50±4.81	67.37±3.61	57.95±2.24
GSAT	62.80±11.41	53.20±8.35	28.17±1.26	66.78±1.45	75.63±3.83	68.66±1.35	58.06±1.98
OOD-GNN	61.10±7.87	52.61±4.67	26.49±2.94	66.72±1.23	79.48±4.19	70.46±1.97	60.60±3.77
StableGNN	57.07±14.10	46.93±8.85	28.38±3.49	66.74±1.30	77.47±4.69	68.44±1.33	56.71±2.79
CIGA	66.43±11.31	49.14±8.34	32.22±2.67	64.92±2.09	65.98±3.31	69.40±2.39	59.55±2.56
DisC	51.08±3.08	50.39±1.15	24.99±1.78	67.12±2.11	56.59±10.09	68.07±1.75	58.76±0.91
Graph Augmentation	DropEdge	45.08±4.46	45.63±4.61	22.65±2.90	66.49±1.55	78.32±3.44	70.78±1.38	58.53±1.26
GREA	56.74±9.23	54.13±10.02	29.02±3.26	69.72±1.66	77.34±3.52	67.79±2.56	60.71±2.20
FLAG	61.12±5.39	51.66±4.14	32.30±2.69	67.69±2.36	79.26±2.26	68.45±2.30	60.59±2.95
M-Mixup	70.08±3.82	51.48±4.91	26.47±3.45	68.75±0.34	78.92±2.43	68.88±2.63	59.03±3.11
G-Mixup	59.66±7.03	52.81±6.73	31.85±5.82	67.44±1.62	78.55±4.16	70.01±2.52	59.34±2.43
AIA (ours)	73.64±5.15	55.85±7.98	36.37±4.44	70.79±1.53	81.03±5.15	71.15±1.81	61.64±3.37
In this section, we conduct extensive experiments to answer the following Research Questions:
- RQ1: Compared to existing efforts, how does AIA perform under covariate shift?
- RQ2: Can the proposed AIA achieve the principles of environmental feature discrepancy and stable feature consistency, thereby alleviating the graph covariate shift?
- RQ3: How do the different components and hyperparameters of AIA affect the performance?
Datasets. We use graph OOD datasets [2] and OGB datasets [20], which include Motif, CMNIST, Molbbbp, and Molhiv. Following [2], we adopt the base, color, size, and scaffold data splitting to create various covariate shifts. The details of the datasets, metrics, implementations, and other settings are provided in Appendix D.2 and D.4. More experiments are provided in Appendix E.
Baselines. We adopt 16 baselines, which can be divided into the following three specific categories:
- General Generalization Algorithms: ERM, IRM [14], GroupDRO [31], VREx [37].
- Graph Generalization Algorithms: DIR [17], CAL [5], GSAT [38], OOD-GNN [39], StableGNN [40], CIGA [41], DisC [21].
- Graph Augmentation: DropEdge [25], GREA [18], FLAG [24], M-Mixup [26],  $\mathcal{G}$ -Mixup [7].
We first make comparisons with various baselines in Table 1, and have the following observations:
Most generalization and augmentation methods fail under covariate shift. VReX achieves a  $2.81\%$  improvement on Motif (base). For two shifts of Molhiv, data augmentation methods GREA and DropEdge obtain  $1.20\%$  and  $0.77\%$  improvements. The invariant learning methods, i.e., DIR and CAL also obtain  $4.60\%$  and  $1.53\%$  improvements on CMNIST and Molbbbp (size). Unfortunately, none of the methods consistently outperform ERM. For example, GREA and DropEdge perform poorly on Motif (base),  $\downarrow 11.92\%$  and  $\downarrow 23.58\%$ . DIR and CAL also fail on Molhiv. These show that both invariant learning and data augmentation methods have their own weaknesses, which lead to unstable performance when facing complex and diverse covariate shifts from different datasets.
AIA consistently outperforms most baseline methods. Compared with ERM, AIA can obtain significant improvements. For two types of covariate shifts on Motif, AIA surpasses ERM by  $4.98\%$  and  $4.11\%$ , respectively. In contrast to the large performance variances on different datasets achieved by
Figure 3: Visualizations of the augmented graphs via AIA.
Table 2: Covariate shift comparisons with different augmentation strategies.
Method	Motif (base)	Motif (size)	CMNIST (color)	Molbbbp (scaffold)
Aug-Train	Aug-Test	Aug-Train	Aug-Test	Aug-Train	Aug-Test	Aug-Train	Aug-Test
Original	0	0.557±0.141	0	0.522±0.421	0	0.490±0.226	0	0.419±0.079
DropEdge	0.772±0.213	0.515±0.033	0.851±0.138	0.161±0.271	0.627±0.186	0.539±0.260	0.758±0.192	0.737±0.211
FLAG	0.001±0.001	0.533±0.016	0.002±0.018	0.507±0.121	0.003±0.002	0.442±0.062	0.001±0.001	0.413±0.088
G-Mixup	0.690±0.186	0.472±0.043	0.816±0.154	0.299±0.343	0.408±0.228	0.351±0.318	0.551±0.258	0.545±0.231
AIA (ours)	0.369±0.169	0.462±0.063	0.649±0.143	0.098±0.070	0.516±0.106	0.307±0.108	0.422±0.049	0.393±0.028
bases, AIA consistently obtains the leading performance across the board. For CMNIST, AIA achieves a performance improvement of  $3.17\%$  compared to the best baseline DIR. For Motif, the performance is improved by  $2.17\%$  and  $1.72\%$  compared to VREx and GREA. These results illustrate that AIA can overcome the shortcomings of invariant learning and data augmentation. Armed with the principles of environmental feature diversity and stable feature invariance, AIA achieves stable and consistent improvements on different datasets with various covariate shifts. In addition, although we focus on covariate shift in this work, we also carefully check the performance of AIA under correlation shift, and the results are presented in Appendix E.
5.3 In-depth Analyses (RQ2)
In this section, we conduct qualitative and quantitative experiments to support our two principles. Firstly, we utilize  $\mathrm{GCS}(\cdot ,\cdot)$  as the measurement to quantify the degree of covariate shift. The detailed estimation procedure is provided in Appendix C. We select four different domains, i.e., base, size, color and scaffold, to create covariate shifts. The experimental results are shown in Table 2. We calculated covariate shifts between the augmentation distribution  $P_{\mathrm{aug}}$  with the training  $P_{\mathrm{tr}}$  or test distribution  $P_{\mathrm{te}}$ . "Aug-Train" and "Aug-Test" represent  $\mathrm{GCS}(P_{\mathrm{aug}},P_{\mathrm{tr}})$  and  $\mathrm{GCS}(P_{\mathrm{aug}},P_{\mathrm{te}})$ , respectively. From the results in Table 2, we make these observations.
Discrepancy Principle. The term "Original" denotes the training distribution prior to augmentation. It's observed that substantial covariate shifts exist between the training and test distributions, ranging from 0.419 to 0.557. The DropEdge technique notably expands Aug-Train, with a range of 0.627 to 0.851, while concurrently increasing Aug-Test, as evidenced by CMNIST (0.490 to 0.539) and Molbbbp (0.419 to 0.737). However, a distribution that deviates excessively from the test distribution may not effectively address the issue of covariate shift. FLAG, which perturbs only the node features, yields minor values in both Aug-Train and Aug-Test. G-Mixup notably augments Aug-Train by generating OOD samples, but doesn't necessarily limit Aug-Test. Finally, AIA extends the disparity with the training distribution by augmenting environmental features, signifying that AIA can adeptly implement the principle of environmental feature discrepancy. Simultaneously, the imposed consistency constraint on stable features restricts the generated distribution from straying too far from the test distribution, as observed in Motif-base (0.557 to 0.462), Motif-size (0.522 to 0.098), and CMNIST (0.490 to 0.307).
Augmentation Diversity. We further delve into the diversity of data augmentation. The concept of augmentation diversity stems from the intuition that augmentations with greater degrees of freedom yield better performance [42]. Accordingly, we propose conditional entropy to
Table 3: Augmentation Diversity.
Method	Full Graph	Env. Feature	Sta. Feature
DropEdge	0.999±0.065	0.933±0.029	0.971±0.067
AIA (ours)	0.561±0.223	0.508±0.136	0.259±0.106
Figure 4: (Left): Different components in AIA. (Middle): Different ratios  $\lambda_{s}$  of stable features. (Right): Different penalties  $\gamma$ . Dashed lines denote the ERM.
measure the diversity of generated data:  $H(\tilde{G} | G) = -\mathbb{E}_G[\sum_{\tilde{g}} p(\tilde{g} | G) \log (p(\tilde{g} | G))]$ , where  $\tilde{G}$  and  $G$  represent the generated graph and original graph, respectively. To substantiate our ability to manage perturbed environmental features while preserving stable features, we examine diversity at the feature level, i.e., stable and environmental features. We employ the Motif dataset for validation due to its inclusion of ground-truth labels for stable and environmental features. The results in Table 3 reveal that our approach can guarantee the diversity of environmental features while constraining the variation of stable features.
To substantiate two principles of AIA, we present a selection of augmented graphs in Figure 3. These graphs are randomly sampled during the training phase. In the original Motif, the stable feature, is represented by the green portion and its type determines the label, while the yellow portion signifies the base-graph, or the environmental feature. Figure 3 (Right) exhibits the augmented samples generated during training. Nodes depicted in darker colors and edges with broader lines indicate higher soft-mask values. These results lead us to several noteworthy observations.
Visualization Analyses. AIA primarily perturbs the environmental features, while leaving the stable components undisturbed. In the Motif dataset, the base-graph represents a ladder and the motif-graph signifies a house. Following augmentation, the nodes and edges of the ladder graph undergo perturbations. However, the house component remains consistently stable throughout the training process. It shows that AIA successfully adheres to the proposed two principles, thus providing empirical support for our claims. Furthermore, under covariate shift, we also depict the stable features identified by AIA in comparison to other baseline methods (refer to Appendix E.4). It further underscores the limitations of alternative methods and highlights the superior performance of AIA.
5.4 Ablation Study (RQ3)
Adversarial Augmentation & Stable Feature Learning. As illustrated in Figure 4 (Left), "w/o Adv" and "w/o Sta" denote AIA without adversarial augmentation and without stable feature learning, respectively. RDIA is a variant that replaces adversarial augmentation in AIA with random augmentation (i.e., random masks). The performance degrades when either component is used independently, compared to their combined application in AIA. The removal of adversarial perturbations results in a loss of the invariance condition inherent in stable feature learning [17, 29], leading to suboptimal outcomes. Conversely, the sole use of adversarial augmentation disrupts the stable features, thereby diminishing the performance. RDIA surpasses ERM, yet falls short of AIA, indicating that although randomness can foster discrepancy, it is less effective than the adversarial strategy.
Sensitivity Analysis. We conduct experiments to explore the sensitivities of ratio  $\lambda_{s}$  and penalty coefficient  $\gamma$ . The results are displayed in Figure 4 (Middle) and (Right).  $\lambda_{s}$  with  $0.3\sim 0.8$  performs well on Motif and Molbbb, while Molhiv is better in  $0.1\sim 0.3$ . It indicates that  $\lambda_{s}$  is a dataset-sensitive hyper-parameter that needs careful tuning. For  $\gamma$ , the appropriate values range from  $0.1\sim 1.5$ .
Graph Data Augmentation [22, 23, 43] enlarges the training distribution by perturbing features in graphs. Recent studies [44, 13] observe that it often outperforms other generalization efforts [14, 31]. DropEdge [25] randomly removes edges, while FLAG [24] augments node features with an adversarial strategy. M-Mixup [26] interpolates graphs in semantic space. However, studies [14, 45]
point out that stable features are the key to OOD generalization. These augmentation efforts are prone to perturb the stable features, which easily lose control of the augmented data distribution.
Invariant Graph Learning has been widely adopted by recent works as a paradigm for handling distribution shifts on graphs. The pioneering works [3, 17] leverage the causal invariance principle to model the invariant predictive patterns in data for the OOD generalization purpose. With the similar spirit, GREA [18] and CAL [5] aim to learn stable features by considering different environments. Some other works also utilize invariant learning to develop generalizable models and algorithms for improving the generalization w.r.t. molecular graphs [8] and recommender systems [9].
Out-of-Distribution Learning on Graphs has aroused wide research interest in the graph learning community. One line of research is centered around the goal of improving the OOD generalization capabilities of models when encountered with test data from new unseen distributions [46, 3, 47, 39, 40, 17, 38, 48, 5]. Another line of research, differently, aims to identify the OOD data in the testing set and improve the reliability of models against OOD data for which the model should reject for prediction [49, 50, 51]. The latter task is called Out-of-Distribution Detection in the literature and serves as another under-explored area that has different technical aspect from the present work. Due to space constraints, we put more discussions of other related studies in Appendix G.
In this study, we address the pervasive yet largely unexplored issue of covariate shift in graph learning. We introduce a novel graph augmentation method, AIA, grounded in two principles: environmental feature discrepancy and stable feature consistency. The discrepancy principle enables the model to explore new environments, thereby facilitating better generalization to potentially unseen environments. Meanwhile, the consistency principle maintains the integrity of stable features. We conduct extensive comparisons with various baseline models and perform thorough analyses.
8 Limitations and Broader Impacts
This paper presents a graph augmentation method, AIA, designed to bolster the academic community's application of data augmentation methodologies. We do not foresee any immediate, direct, or adverse societal implications resulting from our study's findings. We also present additional discussions regarding AIA's limitations and potential future work in Appendix H.
This research is supported by the National Natural Science Foundation of China (9227010114, U19A2079, 62302321) and the University Synergy Innovation Program of Anhui Province (GXXT-2022-040). This work is also sponsored by Ant Group through CCF-Ant Research Fund and CCF-AFSG Research Fund.
[1] Zheyan Shen, Jiashuo Liu, Yue He, Xingxuan Zhang, Renzhe Xu, Han Yu, and Peng Cui. Towards out-of-distribution generalization: A survey. arXiv preprint arXiv:2108.13624, 2021.
[2] Shurui Gui, Xiner Li, Limei Wang, and Shuiwang Ji. Good: A graph out-of-distribution benchmark. arXiv preprint arXiv:2206.08452, 2022.
[3] Qitian Wu, Hengrui Zhang, Junchi Yan, and David Wipf. Handling distribution shifts on graphs: An invariance perspective. In ICLR, 2022.
[4] Haoyang Li, Xin Wang, Ziwei Zhang, and Wenwu Zhu. Out-of-distribution generalization on graphs: A survey. arXiv preprint arXiv:2202.07987, 2022.
[5] Yongduo Sui, Xiang Wang, Jiancan Wu, Min Lin, Xiangnan He, and Tat-Seng Chua. Causal attention for interpretable and generalizable graph classification. In KDD, 2022.
[85] Yanfang Wang, Yongduo Sui, Xiang Wang, Zhenguang Liu, and Xiangnan He. Exploring lottery ticket hypothesis in media recommender systems. International Journal of Intelligent Systems, 37(5):3006-3024, 2022.
[86] Yuan Gao, Xiang Wang, Xiangnan He, Zhenguang Liu, Huamin Feng, and Yongdong Zhang. Addressing heterophily in graph anomaly detection: A perspective of graph spectrum. In WWW, pages 1528-1538, 2023.
[87] Yuan Gao, Xiang Wang, Xiangnan He, Huamin Feng, and Yong-Dong Zhang. Rumor detection with self-supervised learning on texts and social graph. Frontiers Comput. Sci., 17(4):174611, 2023.
[88] Yanqiao Zhu, Yuanqi Du, Yinkai Wang, Yichen Xu, Jieyu Zhang, Qiang Liu, and Shu Wu. A survey on deep graph generation: Methods and applications. arXiv preprint arXiv:2203.06714, 2022.
A Correlation Shift and Covariate Shift
Following graph data generation process [17, 8, 3, 29], we can observe that environmental features easily change outside the training distribution, owing to their unstable nature. Hence, distribution shifts are mainly caused by the environmental features [12, 13, 2]. Specifically, we define the joint distribution of training and test data as  $P_{\mathrm{tr}}(G,Y)$  and  $P_{\mathrm{te}}(G,Y)$ , respectively. Since their joint distribution can be rewritten as  $P_{\mathrm{tr}}(G,Y) = P_{\mathrm{tr}}(Y|G)P_{\mathrm{tr}}(G)$  and  $P_{\mathrm{te}}(G,Y) = P_{\mathrm{te}}(Y|G)P_{\mathrm{te}}(G)$ , we can find that there exist two main reasons that lead to the distribution shift  $P_{\mathrm{tr}}(G,Y) \neq P_{\mathrm{te}}(G,Y)$ . We give intuitive examples in Figure 5 and formal definitions of these two distribution shifts.
- Correlation shift  $P_{\mathrm{tr}}(Y|G) \neq P_{\mathrm{te}}(Y|G), P_{\mathrm{tr}}(G) = P_{\mathrm{te}}(G)$ . If the statistical correlation of environmental features and labels is inconsistent in training and test data, a well-fitted model in training data may fail in test data, which is also known as spurious correlation [14], correlation shift [12] or concept shift [2]. Formally, correlation shift describes the conditional distribution  $P_{\mathrm{tr}}(Y|G) \neq P_{\mathrm{te}}(Y|G)$ .
- Covariate shift  $P_{\mathrm{tr}}(G) \neq P_{\mathrm{te}}(G), P_{\mathrm{tr}}(Y|G) = P_{\mathrm{te}}(Y|G)$ . If there exist environmental features in the test distribution that the model has not seen during training, it will also result in a large performance drop. This unseen distribution shift is well known as covariate shift [2] or diversity shift [12]. It means that the environmental features in test data are unseen in training data, which leads to  $P_{\mathrm{tr}}(G) \neq P_{\mathrm{te}}(G)$ . In Definition 2.1, we also quantitatively measure the covariate shift between  $P_{\mathrm{tr}}(G)$  and  $P_{\mathrm{te}}(G)$ .
It is worth noting that within the computer vision domain, the general covariate shift is frequently synonymous with sample selection bias [52, 53, 54, 55, 56]. Various factors contribute to covariate shift, such as heterogeneous category distribution or domain-specific variances. In the context of our investigation into graph-based models, we adhere to the assumptions outlined in previous literature [2, 12], which mainly ignore the influence of label shifts. And we posit that covariate shifts are principally attributed to the environmental features. The exploration of more comprehensive scenarios involving covariate shifts will be undertaken in our future work.
Figure 5: Intuitive examples of correlation shift and covariate shift