Deciphering Spatio-Temporal Graph Forecasting: A Causal Lens and Treatment
Authors: Yutong Xia
Yuxuan Liang
Haomin Wen
Xu Liu
Kun Wang
Zhengyang Zhou
Roger Zimmermann
Venue: NeurIPS 2023 poster (2023)
Keywords: Spatio-temporal forecasting

Abstract: Spatio-Temporal Graph (STG) forecasting is a fundamental task in many real-world applications. Spatio-Temporal Graph Neural Networks have emerged as the most popular method for STG forecasting, but they often struggle with temporal out-of-distribution (OoD) issues and dynamic spatial causation. In this paper, we propose a novel framework called CaST to tackle these two challenges via causal treatments. Concretely, leveraging a causal lens, we first build a structural causal model to decipher the data generation process of STGs. To handle the temporal OoD issue, we employ the back-door adjustment by a novel disentanglement block to separate the temporal environments from input data. Moreover, we utilize the front-door adjustment and adopt edge-level convolution to model the ripple effect of causation. Experiments results on three real-world datasets demonstrate the effectiveness of CaST, which consistently outperforms existing methods with good interpretability. Our source code is available at https://github.com/yutong-xia/CaST.

Deciphering Spatio-Temporal Graph Forecasting: A Causal Lens and Treatment
Yutong Xia', Yuxuan Liang"; Haomin Wen Xu Liu', Kun Wang', Zhengyang Zhou', Roger Zimmermann'
'National University of Singapore
The Hong Kong University of Science and Technology (Guangzhou)
3 BBijiin Jiaotong University UUniversity of Science and Technology of China
(yutong. x, yuxl mikklingg aigg) @out look. com; { [1 luxu, rogerz comp nus edu.sq wenhaomin@b edu. en; wk520529 ustc. edu, en; 22y0929 edu.
Abstract
Spatio-Temport Graph (STG) forecasting is fundamental task in many real-world applications. Opatio-Temport Graph Neural Networks have emerged as the most popular method for STG forecasting, but they often struggle with temporal out-of- distribution (OoD) issues and dynamic spatial causation. In this paper, we propose novel framework called CaST tackle these two challenges via causal treatments. Concretely, leveraging causal lens, we first build a structural causal model to decipher the data generation process of STGs. To handle the temporal OoD issue, we employ the back-door adjustment by novel disentanglement block to separate the temporal environments from input data. Moreover, we utilize the front-door adjustment and adopt edge-level convolution to model the ripple effect of causation. Experiments results on three real-world datasets demonstrate the effectiveness CaST, which consistently outperforms existing methods with good interpretability. Our source code is available at https://gthub..oo//uuugg xia/CaSTTT
Introduction
Individuals enter world with intrinsic structure, where components interact with one another across Graph (STG) has been
space and time, leading to a spatio-temporal composition. pivotal for incorporating this structural information into the formulation of real- woaldwrlld issues. Within
the realm of smart cities 660. STG forecasting (e.g., traffic prediction [88, 49 17 52 and air quality 4 27 has become instrumental in informed decision-making and sustainability. With recent advances in deep learning, ppatio-Temporal Graph Neural Networks (STGNNs) (50) [0] have become the leading approach for STG forecasting. They primarily use Graph Neural Networks (GNN) 23 to capture spatial correlations among nodes, and adopt Temporal Convolutional Networks (TCN) Recurrent Neural Networks (RNN) [3] to learn temporal dependencies.
forecasting 28
However, STG data is subject to temporal dynamics and may exhibit various data generation distribu- tions over time, also known as temporal out- cuttof-diitributioo (OoD) issues temporal distribution shift (6) 67). As depicted in Figure Th, the training data (periods A and B) and test data derive from
different distributions, namely Pa (x) + Pu(::) 7 Prest(r)). Most prior studies [26] 61 56 15 30
have overlooked this essential issue, which potentially results in suboptimal performance of STGNNs that are trained specific time period to accurately predict future unseen data.
Meanwhile, dynamic spatial causation is another essential nature of STG data that must be addressed for effective and unbiased representation leamingg in STG forecasting. While the majority of STGNNs
Yuxuan Liang is the corresponding author of this paper. Email: yuxliang@outook..com
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

rely on a distance- based adjacency matrix to perform message passing in the spatial domain [6] 26 I.] they lack adaptability to dynamic changes in the relationships between nodes. This matrix also sometimes inaccurate, as two closely located nodes may not necessarily have causal relationships, e.g., nodes belonging to different traffic streams. As an alternative solution, the attention mechanism [14 (44 | calculates the dynamic spatial correlations between nodes adaptively based on their input features. However, they still fall short of capturing the ripple effects of causal relations. Similar to how node signals can propagate information across graphs over time, causal relations (perceived as edge signals) can also exhibit this effect. For example in Figure D.p. when an accident occurs between node A and B at 1, it directly reduces the causal relation 1. At t 2, this effect propagates to other relations, such as weakening relation 2 and strengthening relation 3. This happens because the accident decreases the proportion of traffic flow from node B observed by node D, thus simultaneously increasing the proportion of traffic flow from node C observed by D.
In this paper, our goal is to concurrently tackle the temporal OoD issue and dynamic spatial causation via causal treatments (36). Primarily, we present a Structural Causal Model (SCM) to gain a deeper understanding of the data gener- ation process of STG data. Based on SCM, we subsequently propose to 1) utilize back-door ad- justment to enhance the generalization capability for unseen data; 2) apply front door adjustment along with an edge-level convolution operator to effectively capture the dynamic causation between nodes. Our contributions is outlined follows:
test
time
(a) Temporal Distribution Shift (b) Ripple Effect of Causations
Figure 1: (a) Illustration of temporal OoD, (b)
Spatial causal relationship in the traffic system.
A causal lens and treatment for STG data. We propose a causal perspective to decipher the underlying mechanisms governing the data generation process of STG-structured data. Building upon the causal treatment, we devise novel framework termed Causal Spatio-Tempal neural networks (CaST) for more accurate and interpretable STG forecasting.
Back-door adjuttent for handling temporal OoD. We articulate that the temporal OoD arises from unobserved factors, referred as temporal environments. Applying the back-door adjustment, we design a disentanglement block to separate the invariant part (we call it entity) and environments from input data. These environments are further discretized by vector quantization 46 which in- corporates a learnable environment codebook. By assigning different weights to these environment vectors, our model can effectively generalize on OoD data from unseen environments.
Front-door adjustment for capturing dynamic spatial causation. Adopting a distanced-based adjacency matrix capture spatial information around node (referred to spatial context) could include spurious causation. However, stratifying this context is computationally intensive, making back-door adjustments impractical for spatial confounding. We thus utilize the front-door adjustment and introduce a surrogate to mimic node information filtered based on the actual causal part in the spatial context. To better model the ripple effect of causation, we propose novel de de-confounding module to generate surrogate representations via causal edge-level convolution.
Empirical evidence. We conduct extensive experiments three real-world datasets to validate the effectiveness and practicality of our model. The empirical results demonstrate that CaST not only outperforms existing methods consistently on these datasets, but can also be easily interpreted.
2 Related work
Spatio-Temporal Graph Forecasting. Recently, STG forecasting has gareered considerable atten- tion, with numerous studies focusing on diverse aspects of this domain. Based on the foundation of GNNs (23). STGNNs (50 18 38 20 have been developed to learn the spatio-temporal dependencies in STG data. By incorporating temporal components, such as TCN [2) or RNN [13), STGNNs are capable of modeling both spatial correlations and temporal dependencies in STG data. Pioneering examples include DCRNN 26). STGCN [6], and ST-MGCN [II] Following these studies, Graph
WaveNet [56) and AGCRN I leverage an adaptive adjacency matrix to improve the predictive performance. ASTGCN (44 and GMAN 64] utilize attention mechanisms to learn dynamic spatio- temporal dependencies within STG data. STGODE (8] and STGNCDE (5) capture the continuous spatial-temporal dynamics by using neural ordinary differential equations. ST-MetaNet [34] 35 and
AutoSTG 33 exploit meta learning and AutoMI. for learning STGs, respectively. However, none of these approaches can simultancously address the temporal OoD issue and dynamic spatial causation.

---

Causal Inference. Causal inference [6] 12] seeks to investigate causal relationships between variables, ensuring stable and robust learning and inference. Integrating deep learning techniques with causal inference has shown great promise in recent years, especially in computer vision [62 51 29
natural language processing [37 44 and recommender system [65] 10. However, in the field of STG forecasting, the application of causal inference is still in its infancy. Related methods like graph-based
causal models are typically designed for graph/node classification tasks [43 66] and link prediction [25] For sequential data, causal inference is commonly used to address the temporal OoD issue by learning disentangled seasonal-trend representations (33 or environment-speene
representations (57]. When adapting to STGs, these methods face hurdles, graph based models cannot tackle temporal OoD issues, while sequence- based models fail to accommodate spatial dependencies. In this study, we investigate the STG data generation process through causal lens and employ causal techniques to mitigate confounding effects in both the temporal and spatial domains for STG forecasting.
3 Causal Interpretation for STG Data Generation
Problem Statement. We denote X E R as the signals of N nodes at time step f, where each node has D features. Given the historical signals from the previous T steps, we aim to learn function F.) that forecasts signals over the next S steps: (X(+-T):i = [
B
B
(a)
(b)
(c)
Figure 2: SCMs of (a) STG generation under real-
where (t- E T ((++))((+))) E X and D' is the output dimension. For
world scenarios; (b) back-door adjustment for F;;
(c) front-door adjustment for C.
conciseness, we refer to ((-T)): as X and ((++))((++5) as Y in the rest of the paper.
A Causal Look on STG. From causal standpoint, we construct a Structural Causal Model (SCM) (36] (see Figure 2h illustrate the causal relationships among four variables: temporal environment E, spatial context C, historical node signals X, and future signals Y. Arrows from one variable to another signify causal-effect relationships. For simplicity, we assume E and C are mutually independent. Based on the above definitions, the causal relationships in Figure 2 can be denoted as P(X, Y|E, = PX C)PY|X, F,, C.). We detail these causal-effect relationships below:
X - F. - Y. The temporal OoD is an inherent property of STG data, e.g., X and Y, where P(X*) + P(X(+At) different time steps and ++ At. This phenomenon arise due changes in external variables over time, which we refer to as temporal environments E' [57]. For example, external factors such as weather and events can significantly affect traffic flow observations.
x Y. The historical and future data X and Y are intrinsically affected by the encompass- ing spatial context C' surrounding node. This influence, however, can comprise both spurious and genuine causal components. The spurious aspects may encompass nodes that exhibit either spatial distance or semantic similarity yet lack causal connections, elucidated in Section
X ++ Y. This relation our primary goal established by the prediction model Y = F(X), which takes historical data X as input and produces predictions for future node signals Y.
Confounders and Causal Treatments. Upon examining SCM, we observe two door paths between X and Y, i.e., X + E + Y and X < -C Y, where the temporal environment E and spatial context C' act as confounding factors. This implies that some aspects of X, which are indicative of Y, are strongly impacted by E and C. To mitigate the negative effect of the two confounders, we leverage the causal tools 12 (36 and do-calculus on variable X to estimate P(Y (X). where do(-) denotes the do-calculus. For the temporal OoD, we employ popular de-confounding method called
back-door adjustment [57] 43 to block the back-door path from F. to X (the red dashed arrow in Figure 2p), so as to effectively remove E's confounding effect. This necessitates implicit environment stratification (see Eq. [ Spatial confounding, however, cannot be addressed by spatial context stratification, due the computational burden engendered by the multitude of nodes, each exhibiting a unique contextual profile. Fortunately. the front-door adjustment allows us to introduce mediating variable X between X and Y to mimic a more accurate representation excluded the spurious parts in C (the red node in Figure 2 [54]. Note that we do not use the front-door adjustment to E' because this method mandates that the mediating variable only affected by the cause variable and not by other confounding factors. While for temporal OoD scenarios, unseen future environments can be affected by time-vane varying factors, thus influencing the mediating variable. These two approaches effectively de-confound E and C's confounding effects, explained as follows.
3

---

6.6
9
Temporal Environment
Data How
Loss Fundion
Backbone Ehooder
Codebook Loot
Env Disentangles
HHH
Predictor
Env. Feat H
A
H
HL Dezonfounder
Classifier
Node signal X
Ent. Feat
Output Y Spre
Spatial Context
Surrogate
Ground Truth Y
Boundary Operator
Entity
Embedding
Edge signal
Figure 3: The pipeline of CaST. Env: Environment. Ent: Entity, Feat: Feature.
Back-door Adjustment for To forecast future time series Y based on historical data X, imperative to address the confounding effect exerted by the temporal confounder E. To achieve this, we initially envisage a streamlined SCM, where F constitutes the sole parents of X (temporarily disregarding C) and employ back-door adjustment (36) to estimate P(Y P(Y]do(XX) by stratifying E
into discrete components E
[e])
P(Y P(Y)do(X))
ー - ei)
(I)
where the prior probability distribution of the environment confounder P(E) is independent of X and Y, allowing us to approximate the optimal scenario by enumerating 6;-
Adjustment for C. Once we have dealt with E, our next step is to de-confound the effect of spurious spatial context by using the front-door adjustment [12]. the SCM depicted in Figure 2b. instrumental variable X is introduced between X and Y to mimic the node
Front-door
representation conditioned on their real causal relationships with other nodes. We then estimate the causal effect of X on Y follows:
P(Y]do(X)) = P,Y = x",
P(Y
2') = 2') x² xx)
By observing (X) pairs, we can estimate P(Y .X (54] This front-door adjustment provides
reliable estimation of the impact of X on while circumventing the confounding associations caused by C. We put the derivations of Eq. and Eq. 2 in Appendix A
4 Model Instantiations
We implement the above causal treatments by proposing a Causal Tempoor neural network (CaST), depicted in Figure Our method takes historical observations X inputs to predict future signals Y. We will elaborate on the pipeline and each core component in the following parts.
Back-door Adjustment (see the top half of Figure 3 In order to attain Eq. I two steps need be taken: (1) separating the environment feature from the input data, and (2) discretizing the environments. To accomplish this, we introduce an Environment Disentangler block, and learnable Environment Codebook obtain the desired stratification of environments.
Front-door Adjustment (see the bottom half of Figure 3). Obtaining X* and collecting the (X, X*) pairs to instantiate Eq. is non-trivial task that involves each spatial context, which can be computationally expensive, especially for large graphs, and (2) quantifying the causal effect of Hodge-Laplacial (HL) Deconfounder, which is a neural topologically- -bssed de-confounding module, capture the dynamic causal relations of nodes as well as position embeddings to learn the nodes' global location information. With these two techniques, we can approximate the surrogate X".
main obstacles: (1) enumerating
X To tackle these challenges, we introduce block called

---

2'22' Conv
EnvEncoder
H
f
2'x Corv
Aughooll
X
Encoder
Hed
H.
P-C
H
2'x2' Conv
Linear
1
Free Feot
EntEncoder
Edge Graph
FFT
Linear
IFFT
Foot
H
Attn
Solf
Linear
Layer Norm
P
A (44)
(a) Env Disentangler
(b) HL Deconfounder
(c) Embedding Space
linear projection. Attn: attention. FFT: Fast Fourier Transform. iFFT: inverse FFT. (b) The overview of HL Decon- founder. GCN: graph convolution. (c) The embedding space for the Environment Codebook. The output He is projected onto the closest vector E3 and the gradient V pushes He to change.
Figure 4: (a) The structure of Eny Disentangler. AvgPool:
average pooling. Linear:
4.1 Temporal Environment Disentanglement
the input signals X are first mapped to latent space H RTxNxF by a Backbone Encoder before entering the Env Disentangler, where F means the hidden dimension. Then, this block separates H into the environmental feature He R and entity feature Hj E RNF analogous to background and foreground objects in the computer vision field [5]] Specifically, captures environmental and entity information using two distinct components (see Figure 4h), including 1) EnvEncoder, which consists of a series of ID convolutions, average pooling, and linear projection; 2) EntEncoder, which extracts features from both time and frequency domains via Fast Fourier Transform and self-attention mechanism, respectively. The intuition of the block design EnvEncoder and EntEncoder are discussed in Appendix B After disentangled, for He- we compare it to an Environment Codebook and select the closest vector as the final representation H. NxF. The handling of H; will be explained in Section 4.2
Overview. As shown in Figure 3
Environment Codebook. To stratify the environment E in Eq. [ draw inspiration from 46 and develop a trainable environment codebook e = {er, , ex}. which defines latent embedding space E RKFF Here, K signifies the discrete space size (i.e., the total number of environments), and denotes the dimension of each latent vector e;. As depicted in Figure 3 after acquiring the environment representation He, we use nearest neighbor look-up method in the shared embedding space to identify the closest latent vector for each node's environment representation. Given the environment feature of the i-th node He (i) E R.. this process calculated in the posterior categorical distribution (Zij = k ke (i)) as follows:
{
for = arg min, [1H -ell2, jE (1,,,,,... K), otherwise.
q (2ig = He (1)
(3)
Once obtaining the latent variable E RNN we derive the final environment representation A. E NxF by replacing each row in H. with its corresponding closest discrete vector in e. Note that this categorical probability in Eq. iss only used during the training process, whereas soft probability used during testing to enable generalization to unseen environments. This soft probability signifies the likelihood of environment representation for each node belonging to each environment, denoted (i) == {(zi)) H.(())e where q(zig) (i)) ranges from 0 to 1.
More discussion on how achieve OoD generalization is provided in Appendix G
Representation Disentanglement. We expect the environment and entity representations to be statistically independent, where entity representations carry minimal information (MI) about the environment. To achieve this, we employ an optimization objective inspired by Mutual Information Neural Estimation 3 MI measures the information shared between H. and H;, which is calculated using the of marginal distributions P(He)P(HH.)
(KL) divergence between the joint probability P(He; H4) and the product
I(He, H.) [ H.)IP(H.))(H))),
By minimizing the mutual information, the overlap between H. and H; decreases. When it approaches zero, each representation ensured to possess only self-contained information. This approach transforms disentanglement into optimization issue, which will be introduced in Section 4.3
5

---

4.2 Spatial Context Filtering
Overview, Until now, we finished separating environment-entig and stratifying the environment E. We then shift our focus to the entity H, Our goal is to derive a surrogate H; (i.e., the latent variable of X* in Figure 2 that emulates a node representation containing only information propagated based on genuine causation within their spatial context. As emphasized in Section [I to account for the ripple effects of causal relationships to accurately learn the surrogate. Thus the challenge is how can we effectively model the ripple effect of dynamic causal relationships? Since nodes' causal relations can naturally be regarded as edge features, an intuitive solution for this challenge is to execute convolution operations on edges. Inspired by [6], we build higher-order graph over edges and use an edge-level spectral filter, iee, the GoodgeLLaplacian operator, to represent the propagation of causal relations. This forms the core of HL Deconfounder block (see Figure [b).
it is essential
Moreover, the locational information of nodes incorporates more global spatial perspective, which can be seamlessly implemented using position embedding [3.. We showcase the effects of it in Appendix F Ultimately. the HL Deconfounder block ingests several inputs: the entity variable H4, the edge signal Xed E R the boundary operator a (24), and the position embedding /' E IR x2,,,,, where M and F' mean the number of nodes and the dimension of their features in the built higher- order graph, respectively, and D, denote the embedding dimension. These inputs are processed to yield H, E RNN We then provide an exhaustive breakdown of this process, complemented with associated formulations.
Edge Graph Construction. We first construct a higher-order graph over edges by employing the boundary operator (24), which is mathematical tool used in graph topology to connect different graph elements. Specifically, the first-order boundary operator 0 maps pairs of nodes to edges, while the second-order boundary operator 0₂ maps pairs of edges to triangles. Here use 0 and 0₂ to establish a higher-order graph on edges, which facilitates subsequent convolution operations.
Hodge-L.pllaiaa Operator & Approximation. With the edge graph, we can perform edge con- volution to filter edge signals that contain genuine causation for node. The Hodge-Laplaacin (HL) operator [16 339 is spectral operator defined on the boundary operator. The first-order HI. operator is defined as: L 020g + 0, a. Solving the eigensystem Leb' = V produces orthonormal bases (4)0 v. 1/2 ........ J. The HL spectral filter /t with spectrum I((A) is defined as: ((.,)) Ex=o h(V ybl (-)ob (.). To approximate l(A),, we follow [16) and expand it as a series of Laguerre polynomials T, with learnable coefficients 0,:
h(A) Σ (AT,((),
(5)
where Tn can be computed via the recurrence relation ( (24+1-A)-Wli--I) with initial
u+1 states (A) = 1 and Ti = 1 x. More details of these operators can be found in Appendix C
Causation Filtering& & Surrogate Variable. Shown in Figure the edge signal Xed is mapped into the latent space by the edge encoder to obtain Hed E R M Next, we acquire a new causal relations representation Hea E RMx' by spectral filtering over Hed: Hes = h x Hed = Σ O,T is Hea- We then use a linear transformer to calculate the causal strength Acau E R N. XKx and derive Hsr E X using Graph Convolutional Networks (GCN), where nodes' information is filtered by their genuine causation. N₂ and K2 are the numbers of causal relations (i.e., edges) and the GCN block's depth. Meanwhile, a position embedding P is used to generate A E NxF via a linear transform. We then obtain the surrogate Å, = HHr + H.g- Ultimately, we concatenate it with H. (obtained in Section 4.1) 1 to form the predictor's 's input A and obtain the final prediction Ý.
4.3 Optimization
Environment Codebook. We train the codebook following [46) During forward computation, the nearest embedding He is concatenated with the entity representation H, and fed to the decoder (i.c.. predictor). In the backward pass, the gradient v.C (see Figure 4p) is passed unaltered to the EnvEncoder within the Env Disentangler block. These gradients convey valuable information for adjusting the EnvEncoder's output minimize the loss. The loss function has two components, namely the prediction loss Lpre and the codebook loss Lood:
Lpee PY H.:),
Lood [|sg[Ht]] ell2 + a aalHte =g[e]]1
(6)

---

where a is balancing hyperparameter and sg[-] denotes the stopgradient operator, acting in dual capacity - as an identity operator during forward computation, and has zero partial derivatives during the backward pass. As a result, it prevents its input from being updated. The predictor is optimized solely by Cpre. whereas the EnvEncoder optimized by both Epre and the second term of Lend- The first term of food optimizes the codebook.
Mutual Information Regularization. To minimize I(H,, H.), we use a classifier to predict a in Eq. based on Hi denoted as E. The objective is to thwart the classifier to discern the true labels, or in other terms, to ensure that the classifier can not determine the true corresponding environment based on the information provided by H. To achieve this, we introduce the MI loss fime that minimize the cross-entropy between z and & to encourage å to move away from the true labels and towards uniform distribution:
Lmi = I(H, = =Ex K
log
where = and g means labels belonging to es. The overall loss function is obtained by combining these three losses: L = Lpre + Lood + 3.mms where /3 regulates the trade-off of the MI loss.
5 Experiments
Datasets & Baselines. We conduct experiments using three real-world datasets (PEMS08 42 AIR- BJ [59], and AIR-GZ (59) from two distinct domains to evaluate our proposed method. PEMS08 contains the traffic flow data in San Bernardino from Jul. to Aug. in 2016, with 170 detectors on 8 roads with a time interval of 5 minutes. AIR-BJ and AIR-GZ contain one-year PM2.s readings collected from air quality monitoring stations in Beijing and Guangzhou, respectively. Our task is to predict the next 24 steps based on the past 24 steps. For comparison, we select two classical methods STGNNs for STG forecasting, including DCRNN AGCRND GMSDR 30 and STGNCDE (5]
(HA (63] and VAR 45) and seven state-of-the--e
26 STGCN (1 ASTGCN[14) MTGNN 5S
More details about the datasets and baselines can be found in Appendix D and respectively.
Implementation Details, We implement CaST and baselines with PyTorch 1.10.2 on server with NVIDIA RTX A6000. We use the TCN [2] the backbone encoder and 3-layer MLP as the predictor and the classifier. Our model is trained using Adam optimizer
[22) with a learning rate of 0.001 and batch size of 64. For the hidden dimension F, we conduct grid search over (8, 16, 32, 64). For the number of layers in each convolutional block, we test it from to 3. The codebook size K is searched over (5, 10, 20). See the final setting of our model on each dataset in Appendix D
5.1 Comparison to State-Of-Tie Ofe-rr-Artt Methods
We evaluate CaST and baselines in terms of Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE), where lower metrics indicate better performance. Each method is executed five times, and we report the mean and standard deviation of both metrics for each model in Table [] From this table, we have three key findings: 1) CaST clearly outperforms all competing baselines over the three datasets, whereas the second- best performing model is not consistent across all cases. This reveals that CaST demonstrates a more stable and reliable accuracy across various datasets, highlighting its versatility and adaptability to various domains. 2) STGNN-based models largely surpass conventional methods, i.e., HA and VAR, by virtue of their superior model capacity. 3) While baseline models such AGCRN and MTGNN can achieve runner-up performance in certain cases, they exhibit
Table 1: 5-run error comparison. The bold/underlined font means the best/the second-best result.
PEMS08 (24 +24)
AIR-BJ (24 +24)
AIR-GZ (24 -24)
Model
MAE
RMSE
MAE
RMSE
MAE
RMSE
HA 2017)
58.83
81.96
32.12
43.95
19.56
25.77
V9R11991]
37.04
53.08
29.79
42.04
14.97
20.61
DCRNN22017
22.10
33.96
2.37
35.84 = 0.56
12.99
0.26
18.27 + 0.41
STGCN 2018)
18.60
0.15
23.21
36.30
0.58
12.69
0.04
17.66 0.09
ASTGCN(2019
0.48
0.55
2.22
3.11
18.02 02.51
MTGNN 2020
28.85 0.10
24.35
38.97 + 1.81
17.99 0.11
AGCRN 2020
1.10
2.15
0.29
35.66
1..74
17.49
GMSDR 2022
0.63
1.01
0.52
0.44
1334
19.04
STGNCDE(2022
17.55
27.28
24.35
35.91 0.48
13.70
1.00
CaST (ours)
1..44
26.61
22.90 2 0.09
0.11
12.36
17.25
7

---

35
AIR_B]
South Wind
PEMSOB
31.5
40
Causall Strength
25
22:9
175
S1
10
S14
CaST
w/sFddge
w/nFnt
og
13t
19
15
(a) Effects of Model Components
(b) Dynamic Causal Relations
(c) Geographical Distribution
Figure 5: (a) Effects of each core component on MAE, w/o: without. (b) Visualization of the dynamic
causal relationships between nodes on AIR-BJ. (c) Distribution of air quality stations.
larger standard deviation compared to CaST. This demonstrates that CaST not only offers superior predictive accuracy but also showcases robustness and generalization capabilities. The evaluation of our proposed model confirms that incorporating causal tools not only enhances interpretability but also improves predictive accuracy and generalization performance across different scenarios. We also present the assessment of model performance on various future time steps Appendix F
5.2 Ablation Study & Interpretation Analysis
Effects of Core Components. To examine the effectiveness of each core component in our proposed model. we conducted an ablation study based on the following variants for comparison: a) w/o Env, which excludes environment features for prediction. b) w/o Ent, which omits entity features for prediction. <) w/o Edge, which does not utilize the causal score to guide the spatial message passing. The MAE results for two datasets, AIR-BI BJ and PEMS08, are displayed in Figure3j. We observe that all components contribute to the model's performance. In addition, removing the entity component harms performance more than omitting the environment component, confirming our model's capacity distinguish between the two. Moreover, removing the edge filtering affects PEMS08 more than AIR-BJ. This can be attributed to the unique properties of the down HL operator and the datasets, aligning better with the incompressible traffic flow of PEMS08 for causal capture, opposed to A more detailed discussion can be found in Appendix G
AIR-BJ's non-divergene con-dvergeene--fee flow 44 4.
Effects of Edge Convolution. Our model utilizes a spectral filter on edges in the spa- tial de-confounding block, which enables it to capture the ripple effects of dynamic causal relationships. To validate the superi- ority of our edge convolution module over existing spatial learning methods, we conduct an ablation study on the AIR-BJ dataset by comparing the performance of CaST against two variants: CaST-ADP which replaces the edge convolution with self-adaptive adjacency matrix [56), and CaST-GAT which employs the graph attention mechanism to obtain the causal score (47). The results presented in Table 2 indicate that our edge convolution can adeptly discern the causal strengths between nodes, thereby resulting enhanced performance.
Table 2: Variant results on MAE over AIR-BJ. steps.
Variant
Overall
1-8s
9-168
17-24s
CaST-ADP
24.28
16.42
26.06
30.36
CaST-GAT
23.77
14.76
25.75
30.80
CaST
22.90
13.79
24.86
30.05
Visualization of Dynamic Spatial Causation. To show the power of edge convolutions, we depict the trend of learned causal relations among four air quality stations in Beijing. Note that do not incorporate any external features (e.g., weather and wind) as input. Considering the fact that the dispersion of air quality is strongly associated with wind direction, we select one-week period in Nov. 2019, during which a south wind occurred on 1 1thh Nov. The varying causal relations among the selected stations and their geolocations are displayed in Figures 55 and 5c. From them, three key observations can be obtained. Obsl: The causal relationships S₁ + S1a and S₁ - S₁ exhibit similar patterns when there is a south wind, as S₁ and S1 are both located south of S₁, which is reasonable to suggest that the causal relationship would be stronger in this case. Obs2: S1 + S₁ shows an opposite changing direction compared the former two relationships, as south winds carry PM,.s southward, resulting in the causal strength from S₃ to eastern stations like Sz. Obs3: S4 - S1 and S + S4 display distinct patterns of change. While the causal strength from S4 to S₁ increases with the occurrence of a south wind, that from S1 to S4 shows no significant change. The reason is that south wind carries PM2.5 towards the south, increasing its concentration in Si and strengthening the causal relationship from Sa to Sj. However, the wind's effect is weaker in the opposite direction, leading to no significant changes in the causal strength from S, to S4. These findings underscore the capability of CaST to effeettiveyy capture the dynamic causal relationships among nodes.
8

---

Frequency (Kile)
20
Frequency
-0
Epoche0
Epoch=1
Epoch=4
(b) Fraquency of Environments
Temperature
Low Temp.
High Press:
1006
1005
15
1004
Epoch-8
Epech-10
Epoch-16
62 C1 PyP
(a) Environment Codebook
(c) External Factors of Envirouments
Figure 7: Case study AIR-BJ (better viewed in color). (a) Environment codebook different training epochs by PCA. (b) Distribution of environments for the station S31 in the train and test sets.
Env: Environment. (c) The mean of temperature (Temp) and pressure (Press) in each environment.
K=1
K
K= 20
24.5
Analysis on Environmental Codebook. We employ the environmental codebook as latent representation to address the temporal OoD issue. In Figure 7h, we visualize the environmental codebook vectors using PCA dimensional reduction with K = 10 on AIR-BJ different training epochs. Each color corresponds to particular environment's 's embedding. Remark- ably, starting from the same initialized positions, the environment embeddings gradually diverge and move in distinct directions. Next, we assess the influence of several critical hyperparameters in CaST, including the hidden dimension and the environment codebook size K. We examine how CaST performs on AIR-BJ while varying these hyperparameters and display the results 6 The empirical results indicate that model's accuracy exhibits lower sensitivity to the hidden size F. When the hidden size is small (i.e., F < 32), the performance relatively unaffected by the choice of K. Consequently, the model less reliant on the granularity of the environment representation, provided by K, when the hidden size is small.
37
24.0
RMSE
23.5
36
23.0
35
22.5
34
F
Figure 6: Effects of and on AIR-BJ.
Interpretation of Temporal Environments. To further investigate what the codebook has learned, we visualize the environment types distribution of station S31 in the training set (Jan. Aug. 2019) and testing set (Nov. Dec. 2019) in Figure 7p. Note that the number of environments in the test set computed as sum probabilities since we use the likelihood for each environment to generate unseen environments during the test phase. A substantial discrepancy in environmental distribution between the two sets is observed, with a significant increase in the occurrence frequency of ey the testing set compared to the training set, indicating the existence of temporal distribution shift. As previously discussed, temporal environments are associated with related external factors that change over time. We then collect metcorological features in Beijing for 2019 and calculate the mean of two related variables, i.e., temperature and pressure, for each environment (see Figure bb. We find that is characterized by low temperature and high pressure, which is consistent with the fact that Nov. Dec. typically exhibit lower temperatures and higher pressures compared to Jan. Oct. The distribution of these variables in accordingly periods is presented in Appendix This highlights model's 's capability to learn informative environment representations.
6 Conclusion
In this paper, we present the first attempt to jointly address the challenges of tackling the temporal OoD issue and modeling dynamic spatial causation the STG forecasting task from causal perspective. Building upon a structural causal model, we present a causal network termed CaST that performs the back-door adjustment and front-door adjustment to resolve two challenges, respectively. Extensive experiments over three datasets can verify the effectiveness, generalizability, and interpretability of our model. We provide more discussions Appendix G including the limitations of our model, potential future directions, and its social impacts.
neural

---

Acknowledgments and Disclosure of Funding
This research is supported by Singapore Ministry of Education Academic Research Fund Tier under MOE's official grant number T2EP20221-0022 This work is also supported by Guangzhou Municiple Science and Technology Project 2023A03J0011. We sincerely thank all reviewers for their insightful and constructive comments in improving this paper.
References
[1] Lei Bai, Lina Yao, Can Li, Xianzhi Wang, and Can Wang. 2020. Adaptive graph convolutional recurrent network for traffic forecasting. Advances neural information processing systems 33 (2020), 17804-17815.
[2] Shaojie Bai, Zico Kolter, and Vladlen Koltun. 2018. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01221 (2018).
[3] Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and Devon Hjelm. 2018. Mutual information neural estimation. In Interna- tional conference on machine learning. PMLR, 531-540.
[4] Yu-Chia Chen, Marina Meilä, and Ioannis G Kevrekidis. 2021. Helmholtzian Eigenmap: Topological feature discovery & edge flow learning from point cloud data. arXiv preprint arXiv:2103.07622 (2021).
[5] Jeongwhan Choi, Hwangyong Choi, Jechyun Hwang, and Noseong Park. 2022. Graph neural controlled differential equations for traffic forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 6367-6374.
[6] Yuntao Du, Jindong Wang, Wenjie Feng, Sinno Pan, Tao Qin, Renjun Xu, and Chongjun Wang. 2021. Adarnn: Adaptive learning and forecasting of time series. In Proceedings of the 30th ACM international conference on information knowledge management. 401-411.
[7] Herbert Edelsbrunner, David Letscher, and Afra Zomorodian. 2000. Topological persistence and simplification. In Proceedings s1st annual symposium on foundations of computer science. IEEE, 454-463.
[8] Zheng Fang, Qingqing Long, Guojie Song, and Kunqing Xie. 2021. Spatial-temporal graph ode networks for traffic flow forecasting. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining. 364-373.
[9] Jie Feng, Yong Li, Zeyu Yang, Qiang Qiu, and Depeng Jin. 2020. Predicting human mobility with semantic motivation via multi-task attentional recurrent networks. IEEE Transactions Knowledge and Data Engineering 34, 5 (2020), 2360-2374.
[10] Chen Gao, Yu Zheng, Wenjie Wang, Fuli Feng, Xiangnan He, and Yong Li. 2022. Causal infer- ence in recommender systems: A survey and future directions. arXiv preprint arXiv:2208.1233 12399 (2022).
[11] Xu Geng, Yaguang Li, Leye Wang. Lingyu Zhang, Qiang Yang, Jieping Ye, and Yan Liu. 2019. Spatiotemporal multi-graph convolution network for ride-hailing demand forecasting. In AAAI, Vol. 33. 3656-3663.
[12] Madelyn Glymour, Judea Pearl, and Nicholas Jewell. 2016. Causal inference in statisties: primer. John Wiley & Sons.
[13] Alex Graves. 2013. Generating sequences with recurrent neural networks. arXiv preprint 1308.01308.0855 (2013).
[14] Shengnan Guo, Youfang Lin, Ning Feng, Chao Song, and Huaiyu Wan. 2019, Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. In Proceedings of the AAAI conference artificial intelligence, Vol. 33. 922-929.
10

---

[15] Liangzhe Han, Bowen Du, Leilei Sun, Yanjie Fu, Yisheng Lv. and Hui Xiong, 2021. Dynamic and multi-faceted spatio-temporal deep learning for traffic speed forecasting. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining. 547-555.
[16] Jinghan Huang, Moo K Chung, and Anqi Qiu. 2023, Heterogeneous Graph Convolutional Neural Network via (2023).
for Brain Functional Data. arXiv preprint arXiv:2302.00932
[17] Jiahao Ji, Jingyuan Wang, Zhe Jiang, Jiawei Jiang, and Hu Zhang. 2022. STDEN: Towards physics-guided neural networks for traffic flow prediction. AAAI, Vol. 36. 4048-4056.
[18) Renhe Jiang, Du Yin, Zhaonan Wang. Yizhuo Wang. Jiewen Deng, Hangchen Liu, Zekun Cai, Jinliang Deng, Xuan Song, and Ryosuke Shibasaki. 2021. DI-traff: Survey and benchmark of deep learning models for urban traffic prediction. In Proceedings of the 30th ACM international conference on information & knowledge management. 4515-4525.
[19] Xiaodong Jiang, Pengsheng Ji, and Sheng Li. 2019. CensNet: Convolution with Edge-Node Switching in Graph Neural Networks.. In IJCAI. 2656-2662.
[20] Guangyin Jin, Yuxuan Liang, Yuchen Fang, Jincai Huang, Junbo Zhang, and Yu Zheng. 2023. Graph Neural Networks for Predictive Learning in Urban Computing: A Survey. arXiv preprint arXiv:2303. (2023).
[21] Jachyeong Jo, Jinheon Back, Seul Lee, Dongki Kim, Minki Kang, and Sung Ju Hwang. 2021. Edge representation learning with hypergraphs. Advances in Neural Information Processing Systems 34 (2021), 7534-7546.
[22] Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint 1rXiv:1412.6660 (2014).
[23] Thomas Kipf and Max Welling. [n.d.], d.j. Semi-Supervised Classification with Graph Convolu- tional Networks. In International Conference on Learning Representations.
[24] Hyekyoung Lee, Moo K Chung, Hyejin Kang, and Dong Soo Lee. 2014. Hole detection in metabolic connectivity of Alzheimer's disease using k- Laplacian. In MICCAI Springer, 297-304.
[25] Haoyang Li, Xin Wang, Ziwei Zhang, and Wenwu Zhu. 2022. Ood-gnn: Oft-of-distributio generalized graph neural network. IEEE Transactions on Knowledge and Data Engineering (2022).
[26] Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. 2017. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. arXiv preprint 1707.0770 (2017).
[27] Yuxuan Liang, Songyu Ke, Junbo Zhang, Xiuwen Yi, and Yu Zheng. 2018. Geoman: Multi-level attention networks for geo-sensory time series prediction.. In IJCAI, Vol. 2018. 3428-3434.
[28) Yuxuan Liang, Yutong Xia, Songyu Ke, Yiwei Wang, Qingsong Wen, Junbo Zhang, Yu Zheng, and Roger Zimmermann. 2022. AirFormer: Predicting Nationwide Air Quality in China with Transformers. arXiv preprint 1rXiv:2211.15997 (2022).
[29] Xiangru Lin, Yuyang Chen, Guanbin Li, and Yizhou Yu. 2022. A causal inference look at unsupervised video anomaly detection. In Proceedings of the AAAI Conference Artificial Intelligence, Vol. 36. 1620-1629.
[30] Dachuan Liu, Jin Wang, Shuo Shang, and Peng Han. 2022. Msdr: Multi-step dependency relation networks for spatial temporal forecasting. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 1042-1050.
[31] Xu Liu, Yuxuan Liang, Chao Huang, Hengchang Hu, Yushi Cao, Bryan Hooi, and Roger Zimmermann. 2023, Do We Really Need Graph Neural Networks for Traffic Forecasting? arXiv preprint arXiv:2301. 12603 (2023).
[32] Meinard Müller. 2007. Dynamic time warping. Information retrieval for music and motion (2007), 69-84.
11

---

[33] Zheyi Pan, Songyu Ke, Xiaodu Yang, Yuxuan Liang, Yong Yu, Junbo Zhang, and Yu Zheng. 2021. AutoSTG: Neural Architecture Search for Predictions of Proceedings of the Web Conference 2021. 1846-1855.
Graph. In
[34] Zheyi Pan, Yuxuan Liang, Weifeng Wang, Yong Yu, Yu Zheng, and Junbo Zhang. 2019. Urban traffic prediction from temporll data using deep meta learning. In Proceedings of the 25th ACM SIGKDD international conference knowledge discovery & data mining. 1720-1730.
[35] Zheyi Pan, Wentao Zhang, Yuxuan Liang, Weinan Zhang, Yong Yu, Junbo Zhang, and Yu Zheng. 2020. Spatio-temporal meta learning for urban traffic prediction. IEEE Transactions on Knowledge and Data Engineering 34, 3 (2020), 1462-1476.
[36] Judea Pearl et al. 2000. Models, reasoning and inference. Cambridge, UK: CanbridgeUniversi ryPress 19, 2 (2000).
[37] Margaret E Roberts, Brandon M Stewart, and Richard A Nielsen. 2020. Adjusting for confound- ing with text matching. American Journal of Political Science 64, (2020), 887-903.
[38] Flora Salim and Usman Haque. 2015. Urban computing in the wild: A survey on large scale participation and citizen engagement with ubiquitous computing, cyber physical systems, and Internet of Things. International Journal of Human- Commant Studies 81 (2015), 31-48.
[39] Michael T Schaub. Austin R Benson, Paul Horn, Gabor Lippner. and Ali Jadbabaie, 2020. Random walks on simplicial complexes and the normalized Hodge I-Laplacian. SIAM Rev. 62, 2 (2020), 353-391.
[40] Michael T Schaub and Santiago Segarra. 2018. Flow smoothing and denoising: Graph signal processing in the edge-space. In 2018 IEEE Global Conference on Signal and Information Processing (GlobulSIP). IEEE, 735-739.
[41] David Shuman, Sunil K Narang, Pascal Frossard, Antonio Ortega, and Pierre Vandergheynst. 2013. The emerging field of signal processing on graphs: Extending high nigh-dimensional data analysis networks and other irregular domains. IEEE signal processing magazine 30, (2013), 83-98.
[42] Chao Song, Youfang Lin, Shengnan Guo, and Huaiyu Wan. 2020. Spatial-Temporal Syn- chronous Graph Convolutional Networks: A New Framework for Data Forecasting. AAAI 34, 1 (2020). 914-921.
Network
[43] Yongduo Sui, Xiang Wang, Jiancan Wu, Min Lin, Xiangnan He, and Tat-Seng Chua. 2022. Causal attention for interpretable and generalizable graph classification. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 1696-1705.
[44] Bing Tian, Yixin Cao, Yong Zhang, and Chunxiao Xing. 2022. Debiasing nlu models via causal intervention and counterfactual reasoning. In AAAI, Vol. 36. 11376-11384.
[45] Hiroyuki Toda. 1991. Vector autoregression and causality. Yale University.
[46] Aaron Van Den Oord, Oriol Vinyals, et al. 2017. Neural discrete representation learning. Advances neural information processing systems 30 (2017).
[47] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua (2017).
Bengio. 2017. Graph attention networks. arXiv preprint
[48] Chunyang Wang, Yanmin Zhu, Tianzi Zang, Haobing Liu, and Jiadi Yu. 2021. Modeling inter-station relationships with attentive temporal graph convolutional network for air quality prediction. In W.DMM. 616-634.
[49] Jingyuan Wang, Jiawei Jiang, Wenjun Jiang, Chao Li, and Wayne Xin Zhao. 2021. Libeity: An open library for traffic prediction. In Proceedings of the 29th International Conference on Advances Geographic Information Systems. 145-148.
[50] Senzhang Wang, Jiannong Cao, and Philip Yu. 2020. Deep learning for spatio-temporal data mining: A survey. IEEE transactions on knowledge and data engineering (2020).
12

---

[51] Tan Wang, Jianqiang Huang. Hanwang Zhang, and Qianru Sun. 2020. Visual commonsense r-cnn. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10760-10770.
[52] Haomin Wen, Youfang Lin, Yutong Xia, Huaiyu Wan, Roger Zimmermann, and Yuxuan Liang. 2023, Diffstg: Probabilistic spatio-temporal graph forecasting with denoising diffusion models. arXiv preprint arXiv:2301. 13629 (2023).
[53] Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, and Steven Hoi. 2022. CoST: Con- trastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecasting. In International Conference on Learning Representations.
[54] Ying-Xinn Bing-Xin oe Wu, Xiang Wang, An Zhang, Xia Hu, Fuli Feng, Xiangnan He, and Tat-Seng Chua. 2022. Deconfounding to explanation evaluation in graph neural networks. arXiv preprint arXiv:2201.088 (2022).
[55] Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chengqi Zhang 2020. Connecting the dots: Multivariate time series forecasting with graph neural networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining- 753-763.
[56] Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, and Chengqi Zhang. 2019. Graph WaveNet for Deep Spatial-Tempont Graph Modeling. In I/CAI. 1907-1913.
[57] Chenxiao Yang, Qitian Wu, Qingsong Wen, Zhiqiang Zhou, Liang Sun, and Junchi Yan. 2022. Towards diittriiittiittoo sequential event prediction: causal treatment. arXiv preprint arXiv:2210. 13005 (2022).
[58] Huaxiu Yao, Xianfeng Tang, Hua Wei, Guanjie Zheng, and Zhenhui Li. 2019. Revisiting spatial-temporal similarity: A deep learning framework for traffic prediction. In AAAI, Vol. 33. 5668-5675.
[59] Xiuwen Yi, Junbo Zhang, Zhaoyuan Wang, Tianrui Li, and Yu Zheng. 2018. Deep distributed fusion network for air quality prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Duta Mining. 965-973.
[60] Chuan Tao Yin, Zhang Xiong, Hui Chen, Yuan Wang. Daven Cooper, and Bertrand David. 2015. A literature survey on smart cities. Sci. China Inf. Sci. 58, 10 (2015), 1-18.
[61] Bing Yu, Haoteng Yin, and Zhanxing Zhu. 2018. Spatio-temporal Graph Convolutional Net- works: A Deep Learning Framework for Traffic Forecasting. IJCAI.
[62] Dong Zhang, Hanwang Zhang, Jinhui Tang, Kian-Sheng Hua, and Qianru Sun. 2020. Causal intervention for weakly-supervised semantic segmentation. Advances in Neural Information Processing Systems 33 (2020), 655-666.
[63] Junbo Zhang, Yu Zheng, and Dekang Qi. 2017. Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Prediction. In AAAI. 1655-1661.
[64] Chuanpan Zheng, Xiaoliang Fan, Cheng Wang, and Jianzhong Qi. 2020. Gman: A graph multi- attention network for traffic prediction. Proceedings of the AAAI conference on artificial intelligence, Vol. 34. 1241.
[65] Yu Zheng, Chen Gao, Xiang Li, Xiangnan He, Yong Li, and Depeng Jin. 2021. Disentangling user interest and conformity for recommendation with causal embedding. In Proceedings of the Web Conference 2021. 2980-2991.
[66] Yangze Zhou, Gitta Kutyniok, and Bruno Ribeiro. [n. d.J. OOD Link Prediction Generaliza- tion Capabilities of Message-Passing GNNs in Larger Test Graphs. In Advances in Neural Information Processing Systems.
[67] Zhengyang Zhou, Qihe Huang, Kuo Yang, Kun Wang, Xu Wang. Yudong Zhang, Yuxuan Liang. and Yang Wang. 2023. Maintaining the Status Quo: Capturing Invariant Relations for OOD Spatiotemporal Learning. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (Long Beach, CA, USA) (KDD '23). Association for Computing Machinery, New York, NY, USA, 3603-3614.
13

---

A Derivation of Back-door and Front-door Adjustment
We first introduce three basic rules of do-caleulus do(-), then we present the derivation of back-door and front-door adjustment for the proposed SCM in Figure 2 based on these rules [36 12).
Rules of Do-calculus. Let G be directed acyclic graph with three nodes: X, Y. and Z. Denote the interventional graph as G)) which is identical to G except for the removal of all arrows leading to X from its parents. Denote the nallified graph Gnull()), where all arrows from X have been removed. For better understanding, we illustrate G, Gao(X) and Gaat(x) in Figure
Z
Z
Z
X
X
a
G
(b)
Gánnn
(c) G
Figure 8: Illustration of (a) directed acyclic graph, (b) interventional graph, and (c) nullified graph.
Based on these definitions, we can introduce the following three rules of do-calculus 12
Rule 1: Insertion/dellttinn of observations
Ply]do(2), 2) Plyldo(z)), if 1 2
Rule Action/obserrvation exchange
P(y)do(z), P(uldo(x),)) 2), if (y L =lar)
Rule 3: Insertion/deletion of actions
P(y]do(x), do(=)) = P(yldo(x)) if 22)1) 2(zt)aa(s).ddte))
where (y 2|z)G means that that y and z are independent of each other given x G. For example, Rule / asserts that variables can be removed from the conditioning set if the intervention variables d-separate y from z in the intervention graph Gdo(o)- Note that Figure is an illustration of the definitions of G and its interventional and nullified counterparts, not mandatory for them to comply with these rules.
After introducing the definition, we shift our focus back to Figure 2 and apply the back-door and front-door adjustment on E and C, respectively.
Back-door Adjustment. We initially envisage streamlined directed acyclic graph, where the enviromment E constitutes the sole parents of X (temporarily disregarding C?). By applying the rules of do-calculus, we can derive the back-door criterion as follows:
P(Y)do(X))= = E. E = eldo(X)) (Bayes Rule)
Σ. (
(Rule 3)
E.
=))
(Rule 2)
By stratifying E. into discrete components F. (e: | |8
express the following:
P(Y|do(X)) = Σ E = e;)
(8)
Front-door Adjustment. After de-confounding E, we have Gnall(E)- Then, we turn our attention to the spatial confounder C via front-door adjustment as follows:
P(Y do (x)) Σ, x PoY) ==")= =""))(("==='dd0"" x
(Bayes Rule)
Σ E P(Y|X" x", x'X x') = x x=)do(X)) Σ Σ, =xX==>1X)) = xx, x x')
(Bayes Rule)
(Rule 2)
14

---

B More Details of Env Disentangler
We devise the Env Disentagneler to capture the environmental and entity representations, as depicted in Figure 4a. The Env Disentangler comprises two distinct components: the EnvEncoder which is responsible for capturing environmental information, and the EntEncoder, which focuses on extracting entity features. In the rest of this section, we will provide detailed explanation of these components and their respective roles our model.
EnvEncoder. We design the EnvEncoder (illustrated by the yellow block) to extract environment features. As we have previously discussed in Section [ environmental factors can subtle and often exhibit long-term changes, such as seasonal variations policy shifts, our aim is to capture more global information from an input time series to better represent these underlying environmental influences. The EnvEncoder is comprised of mixture of ID convolutions with kernel size of 2', E {0, .,..,, ).). where S₁ a hyperparameter. These convolutions are followed by an average pooling layer and linear projection to obtain the environment representation He.
EntEncoder. We devise the EntEncoder (illustrated by the blue block) to extract entity information. As entities often contain more local and periodic information, we learn their representations from two aspects: the frequency domain (upper line) and the time domain (lower line). the EntEneoder's upper line, we first perform Fast Fourier Transform (FFT) operation feed it into a linear layer to capture the frequency information. Subsequently, we apply the inverse Fast Fourier Transform (iFFT) to return it to the time domain. In the lower line of the EntEncoder, we utilize the self-attention mechanism to capture the intricate relationships between different time steps, thereby focusing crucial aspects of the time domain information. This is followed by layer normalization. By combining the frequency and time domain information, and feeding it into a linear layer, obtain the entity features H,.
the input data and then
C More Details of HL Deconfounder
The HL Deconfounder is developed (see Figure 4b) learn surrogate that mimies the node information filtered by the dynamic causal score. To achieve an accurate representation learning for this surrogate, we use an edge-level convolution model the ripple effect of the causal relationships, which is mathematically based on the boundary operator [24] and Hodge-Laplacian operator [16]. We provide details on these two techniques in the following part.
Boundary operator. Graphs consist of nodes and edges, which can be mathematically represented O-dimensional and 1-dimensional simplices, respectively 77]. The topology of graph can be described by a boundary operator Ōş [24), where θ encodes the connections between two O)- dimensional simplices (nodes, denoted σn) to form 1-dimensional simplex (edge, denoted a The second order boundary operator 02 represents the connections between 1-dimensional simplices (edges) form 2-dimensional simplices (triangles, denoted as σ2).. The boundary operator θş can be formulated in the matrix form as follows:
if of is positively oriented w.r.t. of if of is negatively oriented w.r.t. o otherwise
{
18nlis =
where ofe, of E Cki Cs fot................ ... oi) denotes the chain complex and q is the number of simplices. For better understanding, we illustrate an example of the first-order boundary operator 0 in Figure 9
(2,
(2.
[5,
0
0
-1
-1
-1
0
D
-1
Graph
Boundary Operator dz
Figure 9: Example of the first boundary operator 0
15

---

Hodge-Laplacian operator. The Hodge-Laplacian (HL) operator [6] is proposed to achieve node- level and edge-level representation learning based on the features of neighboring nodes and edges. The boundary operators 0r play a key role in achieving this goal and can be incorporated into the k-th Vodge-Lapplacian operator, defined as:
(9) where when k = 0, Lu 0 operates over nodes and is equivalent to the Graph Laplacian. When k 1, the HL operator L1 0202 + 0 01 operates over edges. In this study, the L1 to achieve the edge-level convolution.
L* = 0x+10 + + 0k.
D Dataset and Experiment Settings
Datasets, Our experimental design involved selecting three real-world datasets from two distinct domains. The first dataset, PEMS08 42]. contains traffic flow data that was collected by sensors deployed on the road network. Traffic flow data is often considered to be a complex and challenging type of spatio-temporal data influenced by numerous factors, such as weather, time of day, and road conditions. The PEMS08 dataset is thus an ideal choice for our study as it allows us to examine the effectiveness of our proposed method in the context of real-world traffic flow analysis scenario. The remaining two datasets, AIR-BJ and AIR-GZ [59] contain one-year PM2.s readings obtained from air quality monitoring stations located in Beijing and Guangzhou, respectively. The issue of air pollution is a major concern in many cities around the world, and the AIR-BJ and AIR-GZ datasets offered us the opportunity to investigate the application of our proposed method to spatio- temporal data associated with environmental monitoring. To provide a comprehensive overview of the characteristics of the datasets used the experiments, we present their statistics in Table
Table 3: Statistics of datasets.
Datasct
#Nodes
#Edges
Data Type
Time interval
Date Range
#Samples Talin:Val.Tss
AIR-BI
34
82
PM2.5
I hour hour
Jan.1, 2019 Dec. 31, 2019
8,760
4:1:1
AIR-GZ
41
77
PM2.
Jan.1, 2017 Dec. 31, 2017
8,760
4:1:1
PEMS08
303
Traffic flow minutes
Jul. 1,2016 Aug. 2016
17.856
8:1:1
Edge Features. Edge features are built for causality extraction. For each edge, create three types of features: (1) Euclidean-bart based similarity Wyy, (2) Pearson correlations Pty. and (3) Time-delayed Dynamic Time Warping (DTW) Rig- The first two static values, while the third one is dynamic variable that changes over time.
We compute the Fuclidean distances between stations and construct a weighted adjacency matrix using thresholded Gaussian kernel [4] follows:
-- ( if dist (v,, 0.) S
cxp
Wi
0,
otherwise
where Wij E [0, 1] denotes the edge weight between node es and node Uji dist (1i, vz) represents the distance between nodes v; and 0j; o is used as a standard deviation of distances and & is a threshold to exclude nodes that are considered to be very distant.
The Pearson correlation coefficient measures the linear relationship between two signals generated by i-th node and j-th node, represented mathematically as:
cov (X,,
Paj ox,0x,
which ranges from -1 to 1, where I indicates a perfect positive linear relationship, 0 indicates no linear relationship, and indicates perfect negative linear relationship. cov(s,) represents the covariance between two random variables.
Finally, we create time-delay DTW Rsy(r) with delay window size 7 between the source node's signal X and the target node's 's 7 lag series X 32]:
RG (7) DTW (X/: X
RTy(T) (R), (,) R. (r)
where = T/T,, refers the length of the input series and (1,2,.... N). this work, we set 6 and T' = 24, resulting in 4 dimensions of R,j for each edge.
16

---

Computation of HL Operator. The computation of the first-order HL operator involves the calculation of 22 which characterizes the interaction of edges and triangles. However, for the purpose of STG forecasting in this work, we simplify the computation by disregarding triangles and thus use the down HI. operator (i.e., setting 0₂ zero) [39] While in other applications where triangles or higher-order elements have explicit meanings, such as the Benzene Rings of Protein Structures Molecular Structure Modeling, O₂ should be carefully defined.
Evaluation Metrics We leverage Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) to evaluate the performance of models. Let Y; be the label of each entry of Y, and Y, denote the corresponding prediction result, two metrics are calculated as follows:
Σ MI Yi 8./.
MAE(Y,Y) 8)
(10)
M
RMSE(Y, 8)
Y; (x)-
(11)
=1
Final Settings of CaST. We introduce the best hyperparameter configurations for each dataset as follows.
For the PEMS08 dataset, the number of hidden units set at 64, the codebook size K is 20, and the number of layers in all convolutional blocks is 2. The position embedding size is set to 5, and the balancing coefficient for the mutual information loss, a, is set to 0.5.
For the AIR-B BJ dataset, we configure the model with the following settings: the number of hidden units is set to 64, the codebook size K is 10, and the number of layers for all convolutional blocks is 3. The position embedding size set to 10, and the balancing coefficient for the mutual information loss, a, is set to 1.
For the AIR GZ dataset, we use these configurations: the number of hidden units is set at 32, the codebook size K equals 3, and the number layers in all convolutional blocks is 3. The position embedding size is set to 5, and the balancing coefficient for the mutual information loss, is set to 0.5.
E Details of Baselines
Description & Settings. We opted to include selection of widely-used traditional methods and popular cutting-edge methods for comparative evaluation. We describe these baselines and outline the setting used in our experiments follows. The number of parameters of each model is illustrated in Table
HA 633 forecasts future time series values by calculating the average of historical readings for the corresponding time periods.
VAR 45 is popular time series forecasting method that models the interdependence between multiple time series variables.
DCRNN 26 a type of neural network that uses diffusion convolution and sequence-to-ssqqeeec architecture to learn spatial dependencies and temporal relations. We utilize the source code provided GitHut? For each of the three datasets, we configure the model with DCRNN layers and 64 hidden units. The receptive field in diffusion convolution is set to 3.
STGCN 61 is a spatial-temporal graph convolution network that combines spectral graph con- volution with ID convolution capture correlations between space and time. We employ the implementation shared by the authors on GitH For each of the three datasets, we configure the channels of the three layers the ST-Conv block to 64. We set the graph convolution kernel size K and temporal convolution kernel size K, both to 3.
ASTGCN 14 leverages an attention- based mechanism and a spatial-temporal convolution system dynamically capture spatial-temporal correlations and model various temporal properties of
thtpps//gtthu..coo///iit http://gghhbbcoommllyyyauagccncn Bungs//ggttaaccom///PLUDDSSCCCC
17

---

traffic flows, thereby providing superior traffic flow forecasting, We use the code released by the autho to implement this model. We the number of blocks 2 and the hidden size 64.
MTGNN [35] models the temporal dynamics of graph-structurd data by aggregating information from spatially neighboring nodes and past time steps using a message-passing framework. We employ the official MTGNN implementation For all three datasets, configure each MTGNN block with a graph convolution module and temporal convolution module. The hidden dimension for each node in these modules is set to 40. Additionally, set the convolution channel to 32 and the graph convolution depth to 2.
AGCRN ① uses Node Adaptive Parameter Learning and Data Adaptive Graph Generation modules to automatically infer inter-dependencies in traffic series and capture node-specific patterns. We use the code offered in Githut to run the experiments of this model. We set the order of Chebyshev polynomials to 2 and the hidden size to 32.
GMSDR (30] improves upon RNNs by incorporating the hidden states of multiple historical time steps as input each time unit. We use the available GMSDR code for our experiments. Across all three datasets, we configure the model with 2 RNN layers and 64 hidden units.
STGNCDE [5] is a spatio-temporal graph neural controlled differential equation model that uses neural control differential equations process spatial and sequential data. We employ the provided STGNCDE code for our experiments. Across all three datasets, we configure the model with node embedding size of 10 and hidden vector dimensionality of 32.
Table 4: The number of parameters of models. The magnitude Kilo.
Model
AIR-BJ
AIR-GZ
PEMS08
DCRNN
94
94
94
STGCN
163
163
163
ASTGNC
90
92
MTGNN
155
156
166
AGCRN
189
189
189
GMSDR
229
246
553
STGNCDE
372
372
372
CaST (ours)
350
212
391
F More Experiments Results
Experiments Results for Various Steps. We performed five runs for each model, and Table presents the mean and standard deviation of the results for different future time steps the PEMSO8 dataset. We observe that CaST outperforms the majority of compared models.
Effects of Position Embedding. As delineated in Section 4 we incorporate position embedding P alongside the edge signal to learn global location information. We execute our proposed model five times. both with and without P. for 4,000 iterations and present the loss curves for the training and evaluation sets in Figure I0 Losses are documented at 100-iteration intervals. Solid lines signify the mean loss across five runs, while the filled area indicates the standard deviation. For enhanced visualization, we employ a moving average method with a window size of 5 to smoothen the curves. Our findings suggest that models lacking position embedding may be susceptible overfitting, as evidenced by the decreasing losses on the training set beyond 3,000 iterations, while losses on the evaluation set increase.
Effeets of Loss Balance. We investigated the influence of different weightings for terms a and B in our loss function on prediction performance, shown in Figure [1] The color intensity represents the value the metrics, with lighter colors indicating better performances in MAE or RMSE. From the results, the combination of a 0.5 and 3 = 1.5 yielded the lowest MAE and RMSE, indicating this
httt:://gihub.com/wwanujjjj
"Bunss//jthhbbccom///iiBB//CCCC hann:://gghha..ccmm//tttt/////n
18

---

Table 5: 5-run results on PEMS08 for different time steps prediction. The bold/underlined font means the best/the second best result.
8 steps
9 16 steps
17 - 24 steps
Model
MAE
RMSE
MAE
RMSE
MAE
RMSE
HA
58.60
81.60)
58,85
82.02
59.04
82.27
VAR
21.07
32.05
37.10
53.85
52.94
74.58
DCRNN
16,48
25.86 0.17
22.18
34.32 +..22.0.55 0.59
27.63 0.82
42.00 + 1.07
STGCN
0.00
23.47
0.06
0.16
22.07
33,45 : 0.24
ASTGCN
25.24 06.17
0.47
0.60
2..43
36.53
MTGNN
0..05
23.55
0..1
28.98
21.47
34.18
AGCRN
23.68 10.1
1712
2.11
18.74
0.16
GMSDR
00.58
27.67 0.80
18,50
28.40
18.49 + 0.71
29.00 1.13
STGNCDE
0.30
24.29 0.31
17.66 + 0.30
77.46
19.27 + 0.34
30.15 0.48
CaST (ours)
14.39 0.04
22.83
16.47 0.08
26.80 0.22
18.47 2 0.20
30.28 0.17
17.6
w
Loss
17.2
19-
16.8
16.6
16.4
16.2
Iteration
Iteration
Figure 10: Training losses and evaluation losses on AIR-B.. w: with. w/o: without.
balance might be optimal for minimizing the average magnitude and reducing the impact of larger errors. However, it's noteworthy that the trends in performance improvement are not consistent, implying that the relationship between the terms in the loss function and the prediction performance could be complex and not straightforward.
5
35.41
35.4
22.93
22.96
22.81
22.95
35.23
35.35
34.91
35.33
35.2
22.99
33.0
1.5
22.96
22.96
22.96
22.92
1.5
35.39
35.27
35.33
35.28
22.8
34.8
0.5
1.0
1.5
0.2
0.5
1.0
1.5
Figure 11: Heatmap of MAE and RMSE values for varying loss function weights on AIR-BJ dataset.
External Factors Distribution. The distribu- tion of temperature and pressure in Beijing dur-
Probability
0.075
Probaabilty
0.15
Train
ing the training period (January to August 2019) and the testing period (November to Decem- ber 2019) shown in Figure 12 The plot re- veals that the testing period has lower tempera- ture and higher pressure than the training period, which is consistent with the observation in the case study of temporal environments visualization in Section 5.2 that the occurrence frequency of specific temporal environment is higher in the testing set than in the training set. Note that these external factors are just used do the interpretation analysis. When training the model, we did not these external factors.
0.10
0.050
0.025
0.000
20
40
360
1000
1020
1040
Temperature
Pressure
Figure 12: Distribution of external features.
19

---

G More Discussions
Generalization the Unseen Environment. We utilize the environmental codebook as latent representation to capture information for each temporal environment type, whose goal is to represent unique and informative environments in the real world. During training, each environment is assigned an embedding from the codebook, while during testing, soft probability is assigned to indicate the likelihood of the example belonging to each environment. This approach enables the generation of embeddings for unseen environments. For instance, if we have embeddings for high temperature and low pressure (i.e., 61) and low temperature and high pressure (i.e., e2), and we encounter a moderate environment with middle temperature and pressure, we can use different coefficients to generate representative embedding \1e1 Xgez for this unseen environment.
Divergence-free Property of Datasets. In our ablation study on the effects of core components (see Figure 5) the impact of removing the edge filtering module significantly affects performance on the PEMS08 dataset, less so for AIR-H BJ. This can be attributed to the unique properties of the down HI operator and the datasets. In the case of incompressible traffic flow, which is naturally divergence-free [40, 4), our HL deconfounder is able to capture significant causal information via the edge signals. This indeed might explain the significant drop in performance when edge signals are removed in the ablation study on the PEMS08 dataset. Conversely, for data types where the flow is not divergence- like the PM2.5 of the AIR-BJ dataset, the HL deconfounder may not capture as much information, leading to less significant performance drops when edge signals are removed.
Complexity Analysis. We omit hidden dimensionality in the following analysis for simplicity. The complexity of the spatial module is O(M²) induced by the edge convolution operation (i.e., HI. Deconfounder) (11, with M denoting the number of nodes; the complexity of the temporal module (i.e., Env Disentangler) is O where T represents the historical signal length. Generally, we have T & M, leading to (M2) additional overheads over the original Backbone Encoder.
Limitations and Future Directions. One limitation of our proposed framework is that it requires the stratification of temporal environments, which can potentially be infinite. This could pose a challenge in real-world applications where the environmental contexts may be continuously changing and difficult to categorize into finite set of environments. In the future, we may explore the possibility of modeling continuous environments. Besides, while we have underscored the efficacy of the edge convolution CaST in discerning the causative relations between nodes, its computational speed lags relative to alternative methods, such as the adaptive adjacency matrix (56). due to the incorporation of edge graph construction, thereby increasing computational demands. However, given it can be done in the preprocessing stage in one shot, it won't 't produce extra computational costs in the training phase. Additionally, although we have demonstrated the effectiveness of CaST in various real- world applications, further investigation is required to evaluate its performance in more diverse and complex scenarios.
Comparison with Existing Works. Temporal OoD: CaseQ [57] and our proposed CaST both segment the temporal environment based on the back-door adjustment achieve generalization to a new environment. While CaseQ uses shared inference units to associate different types of contexts, CaST stores seen environments in codebook and approximates unseen environments by the combination of vectors from the codebook. Additionally, CaseQ is not suitable for STG forecasting problems as is designed for future event prediction and does not consider the spatial context. Spatial Causal Sirength: The primary challenge in the spatial aspect of this work is how to effectively model the ripple effect of dynamic spatial causal relationships, which is essential for improving node representations for accurate prediction. Although Graph attention [47 and the self- adaptive adjacency matrix proposed by [56] are two alternative methods for learning causal strength, they cannot account for the ripple effect of causation, which involves operating convolution edges. Specifically, we build a higher-ord -rdher-order graph over the edge based on the boundary operator and then use the Hodge-Laplacian operator [16] to filter the edge signal. In Section 5.2 we demonstrate the advantages and effectiveness of incorporating convolution on edge signals to model this effect of causal relationships, and we also provide case study with visualizations. While there alternative methods for edge convolution, e.g., the switch of nodes and edges [19] 21] these approaches may face substantial computational burden, particularly when dealing with non-sparse graphs (16).
Social Impacts. By incorporating causal inference, our proposed CaST framework possesses the substantial potential to effect meaningful change across a variety of domains, particularly within the
20

---

realm of smart cities. The interpretability offered by CaST promotes a deeper comprehension of complex spatio-temporal systems and their underlying causal dynamics. This integration of causal analysis enables stakeholders to make enlightened decisions, ultimately improving individual and communal quality of life and fostering more resilient, sustainable urban environments. For example, in the context of environmental health, CaST can provide more accurate forecasting of critical air quality indices, like PM2.5 concentrations. As air pollution continues to pose significant economic, en- vironmental, and health challenges, the precision of such predictions becomes increasingly paramount. This enhanced accuracy facilitates more effective human health protection measures and informs mikiin, promoting better air quality management and mitigation strategies.
21


Reviews:
[{"review_id": "5bVbcAzqSh", "reviewer": "Reviewer_o65E", "summary": "This paper studies the problem of Spatio-Temporal Graph forecasting under the lens of causal treatments. The authors proposed a framework consisting of two major components to mitigate the commonly seen limitations for spatial-temporal GNNs, i.e., 1) the backdoor environment disentanglement block, which models the temporal environmental changes, and 2) the Hodge Laplacian deconfounder capturing the spatial context. The proposed framework is applied in the air and traffic flow datasets to study spatial-temporal interactions. ", "questions": "1. Related to the weakness #1, have you tried/considered constructing a $\\partial_2$ from e.g., a clique-complex (filling all triangles in the edges), and see if there is performance gain by using the full up and down Hodge Laplacian?\n1. In L275, is there any specific reason to regularize only the scaling $\\beta$ of $\\mathcal L_{mi}$ rather than $\\mathcal L_{cod}$? I understand that there is a regularization parameter $\\alpha$ in Eq. (6), but how do you make sure the first term of $\\mathcal L_{cod}$ is comparable with the log-probability ($\\mathcal L_{pre}$)?\n1. In L745-750 of Section G, the authors mentioned the edge filter being computationally intensive. If there is space, I would like to see an empirical comparison on the runtime.\n1. How should I correctly understand the removal of the Environment codebook in the ablation study (Figure 5a)? Specifically, it seems like the gain for adding Env feature is not huge, does it mean that the environment disentangler is not disentangle the entity from environment well enough (so that some “environmental features” are still presented in the entity features)?\n", "limitations": "The authors have discussed extensively the limitations as well as the social impact in the Appendix, therefore, I do not have any other aspects to add. While I appreciate the detailed discussions in Section G, I believe that it will improve the paper even more if the authors can at least have a brief overview (3-5 sentences) of the limitation/social impact in the main text. \n", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. [Originality/Significance] The framework is proposed to deal with the limitation of the current STGNN models, i.e., the out of distribution issue and dynamic spatial causation. This paper’s originality comes from a novel combination of causal back-door treatment for temporal components, and a Hodge Laplacian decoupling layer for the spatial contexts. \n1. [Quality] The overall experiments presented in this paper for supporting the proposed CaST pipeline are quite nice. It contains different aspects such as ablation study on the core components of the model, which can provide deep insight into the model. \n1. [Clarity] The paper is well-written, with detailed descriptions starting from the background, STG data generation causal graph, causal treatment formulation, to each building block in the CaST pipeline. \n", "weaknesses": "1. From Appendix D (L646), it looks like the Hodge Laplacian used here is the down Hodge Laplacian $\\partial_1^\\top\\partial_1$, rather than the full Laplacian (because $\\partial_2$ is set to zero here). I would suggest mentioning this in the main text, because this is a big assumption to make. Additionally, there is some existing work using Hodge Laplacian on graphs (e.g., [A]), I would suggest citing this paper for completeness. \n1. Related to #1, after applying the edge filter you created, the filtered edge signal will be divergence free [B,C] (due to the low frequency/null space of the down Laplacian being the curl or harmonic flows). This implies that any gradient edge signal (i.e., a flow $x\\in\\mathbb R^{|E|}$ that can be expressed as $x = \\partial_1 y$ for some node function $y \\in \\mathbb R^{|V|}$) will very likely be filtered out after the convolutional layer, suggesting that the HL deconfounder can learn more information when the flow is divergence free (compared with a curl-free flow). Given that the traffic flow is incompressible (thus divergence free) while air (PM2.5) flow is not, it might also suggest why we see huge drop in performance for the ablation study (Figure 5a) when removing edge signal on the PEM508 (traffic flow) dataset, compared with the AIR_BJ (air flow). I would suggest discussing this assumption/limitation in more detail in the main text. \n\n---\n[A] Schaub, Michael T., Austin R. Benson, Paul Horn, Gabor Lippner, and Ali Jadbabaie. “Random Walks on Simplicial Complexes and the Normalized Hodge 1-Laplacian.” SIAM Review 62, no. 2 (2020): 353–91.\n\n[B] Chen, Yu-Chia, Marina Meilă, and Ioannis G. Kevrekidis. “Helmholtzian Eigenmap: Topological Feature Discovery & Edge Flow Learning from Point Cloud Data.” ArXiv:2103.07626 [Stat.ML], March 13, 2021. https://arxiv.org/abs/2103.07626v1.\n\n[C] Schaub, M. T., and S. Segarra. “Flow Smoothing And Denoising: Graph Signal Processing In The Edge-Space.” In 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP), 735–39, 2018. https://doi.org/10.1109/GlobalSIP.2018.8646701.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to sincerely thank you for the time and effort put into reviewing our submission. Your feedback is invaluable and helps enhance the quality of our paper. We appreciate your acknowledgment of the originality, quality, and clarity of our work. Below, we respond to your comments point-by-point.\n\n**[Weaknesses]**\n\n**wrt the down HL and related work.** We sincerely thank you for the suggestions. We have updated the main text to include this assumption, ensuring clarity for readers. As for the existing work [A], we did cite it for completeness in our revision. Thanks.\n\n**wrt the divergence-free property.** Thank you for your insightful comment. By investigating some related literature in traffic flow theory, we agree that the traffic dataset is incompressible and divergence-free. In this case, message passing is predominantly driven by causal intensity. Conversely, PM$_{2.5}$ in the air quality datasets is compressible [1]. This introduces two challenges: firstly, discerning the real causal score becomes difficult due to potential influences from multiple stations, unlike the explicit relationships between road pairs (i.e., up/downstream influences) within traffic data. Secondly, the vast distances between stations may weaken spatial correlations. We are delighted to add the above discussion to our revision, which considerably strengthens the theory of our model.\n\n[1] Hydrodynamic analysis of compliant foil bearings with compressible air flow[J]. J. Trib., 2004, 126(3): 542-546.\n\n**[Questions]**\n\n- **Q.** Have you tried constructing a $\\partial_2$?  \n**A.** Thank you for raising the insightful question. We’ve opted against constructing a $\\partial_2$ for three primary reasons: (1) our emphasis on primary interactions through causation ripple effects on edges; (2) the lack of explicit meanings for triangles in our task; and (3) reduced computational load. However, considering applications like molecular structure modeling where triangles, such as Benzene Rings, hold significance, a well-defined $\\partial_2$ is essential.  In light of your suggestion, we believe it would be valuable to explore the impact of constructing $\\partial_2$  on our model's performance in future work. \n\n- **Q.** Why is only scaling $\\mathcal{L}_{mi}$?\n\n  **A.** Thank you for your inquiry. We chose not to scale $\\mathcal{L}_{cod}$ because its magnitude aligns closely with that of the first term, following the work [2].\n\n- **Q.** The computational intensity and runtime comparison?\n\n  **A.** We appreciate your insightful comment and apologize for any confusion caused due to the lack of clarification in our paper. Indeed, the computational intensity stems from constructing the higher-order (edge) graph from the original dataset, which can be done in the preprocessing stage in one shot. In other words, it won't produce extra computational costs to the training phase. Thus, its impact on the overall training efficiency is negligible. We have followed your advice to clarify this part in our revision.\n\n- **Q.** Does the slight gain from the Env feature suggest issues with the temporal disentangler?\n\n  **A.** Thank you for your insightful question. In designing the temporal disentangler, our intent is to distinctly identify the entity feature, capturing nuances of time series dynamics like periodicity, and the environment feature, aimed at reflecting global trends. As we had foreseen, the removal of the entity causes a significant decrease in performance, while the removal of the environment yields a subtler drop. This aligns precisely with the outcomes presented in our ablation study (see Figure 5a).\n\n[2] Neural discrete representation learning. NeurIPS 2017.\n\n**[Limitations]**\n\n**wrt a brief overview of the limitations/social impact in the main text.** We appreciate your suggestion and have added a brief overview of them correspondingly in our revision. \n\nOnce again, thank you for your constructive feedback. We hope our revisions and clarifications address your concerns.\n"}, {"review_id": "FS7rZz7ir6", "reviewer": "Reviewer_u4Nz", "summary": "The paper introduces a model for out-of-distribution prediction in spatio-temporal data. The confounding effects are decoupled in spatial and temporal contexts, and treated with frontdoor and backdoor adjustments, respectively. Empirical evidence shows improved performance.", "questions": "### Questions:\n- Line 282: Most of the literature considered a different number of steps (see, eg, DCRNN and MTGNN). Although different choices are valid as well, considering the same setting provides a means to assess reproducibility and allows a broader comparison of the methods. Is there a particular reason for not considering these settings?\n\n### Suggestions:\n- The top and bottom halves of Fig. 3 referred to in Sec. 4 are not easily identified in the figure. I suggest highlighting them, e.g., with boxes or shaded areas.\n- Line 211: This is guaranteed only if the KL divergence is 0. I suggest rephrasing it.\n- The causal strength constitutes an important element. I suggest providing a proper definition and discussion about it.\n- Other methods in the literature are generally winning over the methods considered in the paper (see e.g. [https://arxiv.org/abs/2005.11650](https://arxiv.org/abs/2005.11650) and the reference therein). I suggest considering some of them, like Graph WaveNet [48] which is also a quite established baseline from 2019. \n\n### Typos\n- Line 162: Eq. 8.\n\n\n### After rebuttal\nI have read the author's rebuttal which has addressed all the raised points.", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The tackled problem is extremely relevant.\n- To my knowledge, the proposed model consisting of a combination of edge-level filtering, back-door, and front-door adjustment is novel and sound.\n- The achieved performance is remarkable.", "weaknesses": "- It's unclear to me to what extent the considered assumption of decoupled spatial and temporal environments is reasonable. For instance, the weather (mentioned in line 124) is related to both time and space.\n- The experimental setup is not ideal in my opinion.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed review of our submission. We appreciate the insights and constructive feedback you have provided. We are grateful for your acknowledgment of the relevance, novelty, and performance of our work. Below, we address your comments in a point-by-point manner.\n\n**[Weaknesses]**\n\n**wrt assumption of decoupled spatial and temporal environments.** Thank you for raising a valid point. Our assumption to treat space and time separately aligns with numerous mainstream ST models, such as GraphWaveNet [1] and STGCN [2]. This treatment optimizes computations and reduces memory consumption, making it practical for real-world applications. It's noteworthy that the performance does not be compromised, as seen with STSGCN [3], where space and time are integrated. Based on these efficiencies and results, we adopted this assumption for our model.\n\n[1] GraphWaveNet for Deep Spatial-Temporal Graph Modeling. IJCAI 2019. (citation: 1,100+)\n\n[2] Spatio-temporal Graph Convolutional Net works: A Deep Learning Framework for Traffic Forecasting. IJCAI 2018. (citation: 2,500+)\n\n[3] Spatial-temporal synchronous graph convolutional networks: A new framework for spatial-temporal network data forecasting. AAAI 2020.\n\n\n\n**wrt experimental setup.**  In the context of air quality prediction, forecasting 12 steps (namely 12 hours) is not practical since citizens usually plan their journey one day ahead. Therefore, we adopted a popular setting from air quality forecasting literature [4], i.e., using the previous 24 steps to predict the next 24 steps. We maintained this setting for PEMS08 for consistency. Furthermore, such longer-term forecasting is more challenging, which poses new hurdles to the prediction model.\n\n[4] Airformer: Predicting nationwide air quality in china with transformers. AAAI 2023.\n\n**[Questions]**\n\n- **Q.** Reason for experiment setting?\n\n  **A.** Please kindly see our response to the Weaknesses Section.\n\n\n**[Suggestion]**\n\n**wrt Figure 3 enhancements.** Thanks for your suggestion. Actually, one of the co-authors also raised this issue before paper submission. We have tried to improve the figure, but we haven't managed to revise it beautifully and clearly yet. We will try our best to improve this figure in the revision. Thank you.\n\n\n\n**wrt KL divergence clarification.** Thank you for pointing out this problem. We've revised the text to: \"By minimizing the mutual information, the information overlapping between  $H_e$ and  $H_e$ decreases. When this value approaches zero, each representation is ensured to possess only self-contained information.\"\n\n**wrt definition on causal strength.** Thank you for your suggestions. We’ve defined ‘causal strength’ as the magnitude of the causal effect between a cause and its outcome. It measures how alterations in one variable directly impact another. A stronger causal strength shows a clear effect, while a weaker one suggests a less obvious relationship. We've included this definition in our revised manuscript.\n\n**wrt considering other methods.** Thank you for your suggestion. Due to the limited time, we've assessed Graph WaveNet [1] on two datasets (i.e., PEMS05 and AIR-BJ) for a comprehensive comparison, as shown in the table below.\n\n| Model         | PEMS08 (MAE)      | PEMS08 (RMSE)    | AIR-BJ (MAE)     | AIR-BJ (RMSE)    |\n|---------------|-----------------|----------------|----------------|----------------|\n| GraphWaveNet  | 16.94 ± 0.43    | 26.70 ± 0.68   | 23.48 ± 0.43   | 36.21 ± 0.69   |\n| CaST (ours)   | **16.44** ± 0.10| **26.61** ± 0.15| **22.90** ± 0.09| **34.84** ± 0.11|\n\n\n**wrt Typo in Line 162.** Thank you for kindly bringing this to our attention. We have rectified this typo in our revision. \n\nThank you again for your detailed review and insightful feedback. We have meticulously addressed and incorporated your suggestions into our revised manuscript. We believe these are not essentially technical issues. Considering the other three reviewers agree that our paper has good merits such as satisfactory novelty and comprehensive evaluation, we sincerely hope that you could reconsider our score. Thank you so much!"}, {"review_id": "iNkSZln4rW", "reviewer": "Reviewer_kwLE", "summary": "The paper presents a novel framework, CaST, designed to address the challenges of temporal Out-Of-Distribution (OOD) issues and modeling the underlying dynamic spatial causation in Spatio-Temporal Graph (STG) forecasting. The authors construct a Structural Causal Model (SCM) to uncover the causal mechanisms of STG data, inspiring them to employ back-door and front-door adjustments to mitigate the confounding bias induced by temporal environment and spatial context, respectively. Moreover, they utilize the Hodge-Laplacian (HL) operator for edge-level convolution to capture the ripple effect of causation along space and time. Empirical results from experiments on three real-world datasets demonstrate that CaST outperforms existing methods while maintaining interpretability.\n", "questions": "1. Why do the authors assume that temporal environment $E$ and spatial context $C$ are independent? For example, the states of neighboring nodes of node i will also be influenced by the change of $E$.\n2. How do the proposed modules output statistical estimands that are required to calculate $p(Y|do(X))$ in Eqn. (1) and (2)?\n3. Why do the authors focus on learning $p(Y|do(X))$ rather than $p(Y|do(X), E, C)?$ Although $E$ and $C$ are confounders, they can also provide additional information for accurate prediction.\n4. In ablation studies, why **w/o Edge**, i.e., replacing 'GCN with causal scores' with GCN, causes such a performance drop in PEMS08? The STGCN also adopts GCN for spatial message passing, but it achieves MAE 18.60 in Table 1.", "limitations": "see weakness", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The authors provide an in-depth analysis of the STG forecasting problem, revealing the underlying generation mechanisms of STG data and identifying the key sources of spatio-temporal distribution shift through the lens of causality.\n2. The authors offer an interpretation of the specialties of temporal and spatial confounders (i.e., temporal environment and spatial contexts), making the application of backdoor and frontdoor adjustments in CaST targeted and well-motivated.\n3. Extensive experiments validate the model's superiority, including comparisons against state-of-the-art STGNNs, ablation studies, hyperparameter sensitivity analysis, and case studies with detailed visualizations of the model's ability to capture spatial causality and identify temporal environments.\n4. The figures provide readers with an intuitive understanding of the main problem, key ideas, and compelling experimental results.", "weaknesses": "1. The concept of temporal disentanglement is not entirely novel. Although the specific designs of disentanglement headers for environment feature and entity feature may have some novelty, as shown in Figure 4(a), the authors did not adequately clarify this in the main content. \n\n2. For the backdoor adjustment and corresponding temporal disentanglement module: a) The paper lacks a clear explanation regarding the connection between estimating $p(Y|do(X))$ via backdoor adjustment and temporal disentanglement, raising doubts about the effectiveness of the temporal disentanglement module. b) The proposed mutual information regularization cannot ensure that no causal features are leaked into the disentangled environment features $H_e$, possibly leading to the loss of essential causal information.\n\n3. For the frontdoor adjustment and corresponding spatial context filtering module: a) The interpretation of the mediator variable $\\hat{H}_i$ is confusing. The authors believe that $\\hat{H}_i$ should be \"a node representation containing only information propagated based on genuine causation within their spatial context\". However, the spatial context $C$ is a confounder, and a mediator containing information from the confounder cannot satisfy the frontdoor criterion due to a backdoor path from the mediator to the label $Y,$ i.e., $\\hat{H}_i \\leftarrow C \\rightarrow Y.$ b) The paper lacks a sufficient explanation regarding the connection between estimating $p(Y|do(X))$ via frontdoor adjustment and spatial context filtering, raising doubts about the effectiveness of the spatial context filtering. c) The reviewer fails to find an explanation for why using the HL operator on edge-graph can learn causation while other GNNs like GAT cannot.\n\n4. The reference [1] addressed the spatio-temporal distribution shift in dynamic graphs. Since the spatio-temporal graph is a special type of dynamic graph, the authors should compare CaST with the approach described in [1].\n\n5. The ablation studies are insufficient. Validating the fact that **w/o Env** and **w/o Ent** can cause performance degradation is unnecessary, as it is a natural result of eliminating information predictive of the label. Additionally, the ablation studies on loss functions corresponding to the two disentanglers are missing.\n\n6. The visualization of dynamic spatial causal relationships in section 5.2 is insufficient since the authors do not check whether other SOTA STGNNs can also capture these relationships.\n\n7. The model's sensitivity to the hyperparameters $\\alpha$ and $\\beta$, which control the importance of different terms in the final loss function, should be discussed.\n\n[1] Zeyang Zhang, et al. Dynamic Graph Neural Networks Under Spatio-Temporal Distribution Shift. In NIPS 2022.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thorough review and constructive feedback on our paper. We appreciate the time you've taken to provide valuable insights. Please find our responses below.\n\n**[Weaknesses]**\n\n**wrt the novelty of temporal disentanglement.** Thank you for your feedback. We acknowledge that the temporal disentanglement concept has been proposed by previous efforts. And we have highlighted the novelty of implementing the temporal disentangler module in the revision.\n\n**wrt backdoor adjustment and temporal disentanglement.**\n\n- a) We appreciate your attention to this aspect of our work. The relationship between estimating $P(Y|do(X))$ via backdoor adjustment and temporal disentanglement is detailed in Section 4 (please refer to Lines 167-170). \n- b) Regarding your concern about the potential loss of essential causal information, our ablation study in Section 5.2 can address this issue. By testing removing environment or entity features for prediction, our results show that both components offer valuable predictive information, and the entity contains more essential information that aids forecasting. This aligns with our aim for temporal disentangler design: we expect the entity contains more local and vital information like periodicity while the environment contains more global information like the trendy.\n\n\n**wrt frontdoor adjustment and spatial context filtering.**\n\n- a) Thank you for bringing up this insightful query. In our model, $\\hat{H}_i$ acts as an estimate for $X^*$, derived from the general causal relationship, while  $X^*$ itself is a surrogate that mimics the deconfouned  $X$. As depicted in Figure 2c, when applying frontdoor adjustment by introducing  $X^*$, it is a cause of $Y$. Consequently, all backdoor paths from  $X^*$  to $Y$ can be blocked (i.e., there are no unmeasured confounders, e.g., $C$,  between  $X^*$ and $Y$). As a result, the relationships  $X^*  \\leftarrow C \\rightarrow Y $ and $\\hat{H}_i\\leftarrow C \\rightarrow Y $ don't exist.\n- b) The relationship between estimating $P(Y|do(X))$ via frontdoor adjustment and spatial context filtering is detailed in Section 4 (please refer to Lines 171-177). \n-  c) The HL operator was carefully chosen to address the ripple effect of causation, not merely the causation itself.  This choice aligns with the second challenge we aim to tackle in our work, as outlined in the Introduction, Lines 40-46, and Figure 1b. The effectiveness of the HL operator lies in its ability to perform convolution on the graph's edges rather than nodes. The ablation study in Table 2 demonstrates this.\n\n\n**wrt a related model for discussion.** Thank you for bringing the related work DIDA to our attention. It indeed offers significant insights. While both CaST and the DIDA address spatio-temporal distribution shifts, they focus on different tasks: CaST on STG forecasting, and DIDA on link prediction. The distinct nature of these tasks makes direct comparison less meaningful. Nevertheless, we recognize the significance of DIDA and will incorporate this discussion in Section 2.\n\n**wrt hyperparameters sensitivity in the loss function.** Thanks for commenting on this. We've provided additional experiments on the AIR-BJ dataset (see the table below). We find that adjusting the $\\alpha$ and $\\beta$ weights in the loss function greatly influences the model's performance. We reach a trade-off at $\\alpha=0.5$, $\\beta=1.5$, and $\\alpha=1$, $\\beta=1$ to yield the lowest MAE and RMSE, suggesting an optimal trade-off.\n\n|   $\\alpha$-$\\beta$   |  MAE  |  RMSE |\n|:-------:|:-----:|:-----:|\n| 0.5-0.2 | 23.04 | 35.51 |\n| 0.5-0.5 | 23.06 | 35.41 |\n| 0.5-1.0 | 23.06 | 35.52 |\n| 0.5-1.5 | 22.80 | 34.79 |\n| 1.0-0.2 | 22.93 | 35.23 |\n| 1.0-0.5 | 22.96 | 35.35 |\n| 1.0-1.0 | 22.81 | 34.91 |\n| 1.0-1.5 | 22.95 | 35.33 |\n| 1.5-0.2 | 22.96 | 35.39 |\n| 1.5-0.5 | 22.96 | 35.27 |\n| 1.5-1.0 | 22.96 | 35.33 |\n| 1.5-1.5 | 22.92 | 35.28 |\n\n**wrt visualization of spatial causal relationships.** Thank you for pointing that out. Due to time and space constraints, we'll certainly include such comparisons in the revision.\n\n**[Questions]**\n\n- **Q.** Why do the authors assume the independence of temporal and spatial factors.\n\n  **A.** Thank you for raising this valuable question. Please kindly see our response to Reviewer u4Nz’s Weaknesses Section.\n\n- **Q.** How modules output statistical estimates.\n\n  **A.** Thank you for your question. As outlined in Section 4, our framework indeed implements both Eq.1 and Eq.2. For the former, we employ the Environment Disentangler to isolate environmental features from the input data, and an Environment Codebook to categorize these environments, fulfilling the requirements of Eq.1. As for Eq.2, we use the HL Deconfounder to discern the causal relationships between nodes, assisting in measuring the causal influence of $X$ on $X^*$, thereby meeting the essential criteria of Eq.2.\n\n- **Q.** In ablation studies, why w/o Edge causes such a performance drop in PEMS08?\n\n  **A.** Referencing Line 311, 'w/o Edge' excludes the use of causal scores to guide spatial message passing rather than replacing the process with GCN. This omission naturally leads to a substantial performance drop. While we indeed tested various methods for learning the causal score for spatial message passing (Table 2), where results are only marginally worse than using edge convolution. \n\nThank you again for your valuable feedback. We have revised our manuscript based on your feedback."}, {"review_id": "UxrIPUzXDt", "reviewer": "Reviewer_ya77", "summary": "The paper introduces CaST, a novel framework designed to address challenges in Spatio-Temporal Graph (STG) forecasting. CaST tackles issues related to temporal out-of-distribution and dynamic spatial causation by leveraging a causal lens and employing techniques such as back-door adjustment and front-door adjustment. Experimental results on real-world datasets demonstrate the effectiveness and practicality of CaST, outperforming existing methods while providing good interpretability.", "questions": "Is the causal structure learned through a data driven approach or based on heuristic reasonings?\n", "limitations": "I am debating if this is a limitation or not. Obviously the proposed method is composed of a lot of sophisticated design components. But maybe just considering causal structure in the time series data can be more compelling and easier to be generalizable. ", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "Novel design of CaST.\nThorough quantitative and qualitative analysis of the experiment results. \n\n", "weaknesses": "There are too many design decisions that are made for CaST, making it hard to validate each individual component's contribution to the model performance. \nMissing complexity analysis.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful for your thorough evaluation of our paper and the feedback you provided. Thank you for recognizing the novelty in the design of our framework and the comprehensiveness of our analysis.\n\n**[Weaknesses]**\n\n**wrt complex design decisions.** Thank you for your feedback. In fact, our main contribution is to approach STG data from a causal perspective and use causal tools to address challenges in STG forecasting, as outlined in our first contribution in Introduction and detailed in Section 3. While the framework may appear relatively intricate, it serves as a vehicle to implement our causal-focused approach, which we believe offers distinctive advantages and presents the core of our contribution. For the validation of these modules in CaST, we have carefully designed a comprehensive ablation study in Section 5 to test the impact of each component on overall performance. For instance, by removing the environment and entity features, we assess the effectiveness of our temporal disentangler in distinguishing these two features.\n\n**wrt complexity analysis.** We apologize for not including this crucial part in our main text. We omit hidden dimensionality in the following analysis for simplicity. The complexity of the spatial module is $ O(M^2)$ induced by the edge convolution operation (i.e., HL Deconfounder) [1], with $M$ denoting the number of nodes; the complexity of the temporal module (i.e., Env Disentangler) is $O(T^2)$, where $T$ represents the historical signal length. Generally, we have $T \\ll M$, leading to $O(M^2)$ additional overheads over the original Backbone Encoder. Hope a revision with the above analysis is still considered. Thank you.\n\n[1] Edge representation learning with hypergraphs. NeurIPS 2021.\n\n**[Questions]** \n\n- **Q.** Is the causal structure data-driven or heuristic?\n\n  **A.** Thank you for raising this question. For the CaST framework, the modules’ weights derive from the edge convolution operation, which can be trained through backpropagation. This indicates that our Cast is a data-driven model. As for the SCM, after an extensive literature review, we introduced it specifically for STG data in our scenario, which is also the main contribution of our work.\n\n**[Limitations]**\n\n**wrt sophistication of design components.** Thank you for your thoughtful feedback. As responded in the Weaknesses section, our primary contribution lies in introducing a causal perspective and the novel deep learning framework is used for its implementation. Although CaST comprises multiple components, we believe that each plays a vital role in addressing the challenges we've identified. Beyond the temporal causal effect, it is equally important to model the spatial one considering the highly related spatial correlation in STG data. Thus, to address both the temporal OoD and dynamic spatial causation in STG forecasting, every component is essential.\n\nAgain, thank you for your feedback. We have refined our work based on your insights.\n"}]

Author rebuttal:
Dear Reviewers,

We would like to express our sincere gratitude to all the reviewers for their thorough evaluation and constructive feedback on our manuscript. Your insights have been invaluable in enhancing the quality and clarity of our work. We have made several revisions accordingly to address the concerns raised:

- **Enhanced Clarifications**: We have enhanced our manuscript with more in-depth explanations where necessary, such as an explanation of ablation studies via a divergence-free perspective and a detailed complexity analysis.

- **Additional Experiments**: Based on the feedback, we have conducted additional experiments to further validate the effectiveness of our proposed model. These include an ablation analysis focusing on hyperparameters within the loss function and a comparison with Graph WaveNet [1].

- **Clearer Definitions**: We have added clear definitions for terms like 'causal strength' to ensure readers have a comprehensive understanding of our method.

- **Expanded Literature Discussion**: We have expanded our discussion on related works, like [2, 3]. This helps in positioning our work in the broader context of the field.

We believe that these revisions have significantly improved our manuscript. We hope that our responses and the changes made address the concerns of the reviewers adequately.

Once again, thank you for your time and effort in reviewing our work. We look forward to your continued feedback.

Best regards,

Authors

**Reference**

[1] GraphWaveNet for Deep Spatial-Temporal Graph Modeling. IJCAI 2019. 

[2] Random Walks on Simplicial Complexes and the Normalized Hodge 1-Laplacian. SIAM Review 62, no. 2 (2020): 353–91.

[3] Dynamic Graph Neural Networks Under Spatio-Temporal Distribution Shift. In NIPS 2022.

Meta-review: This paper provides a solid technical contribution to the problem of forecasting on spatio-temporal graphs. Many of the concerns raised by the reviewers were addressed through discussions with the authors.
Decision: Accept (poster)