Robust Spatio-Temporal Centralized Interaction for OOD Learning
Authors: Jiaming Ma
Binwu Wang
Pengkun Wang
Zhengyang Zhou
Xu Wang
Yang Wang
Venue: ICML 2025 poster (2025)
Keywords: Spatiotemporal learning
out-of-distribution
spatiotemporal prediction
Primary area: applications->time_series

Abstract: Recently, spatiotemporal graph convolutional networks have achieved dominant performance in spatiotemporal prediction tasks. However, most models relying on node-to-node messaging interaction exhibit sensitivity to spatiotemporal shifts, encountering out-of-distribution (OOD) challenges. To address these issues, we introduce \textbf{\underline{S}}patio-\textbf{\underline{T}}emporal \textbf{\underline{O}}OD \textbf{\underline{P}}rocessor (STOP), which employs a centralized messaging mechanism along with a message perturbation mechanism to facilitate robust spatiotemporal interactions. Specifically, the centralized messaging mechanism integrates Context-Aware Units for coarse-grained spatiotemporal feature interactions with nodes, effectively blocking traditional node-to-node messages. We also implement a message perturbation mechanism to disrupt this messaging process, compelling the model to extract generalizable contextual features from generated variant environments. Finally, we customize a spatiotemporal distributionally robust optimization approach that exposes the model to challenging environments, thereby further enhancing its generalization capabilities. Compared with 14 baselines across six datasets, STOP achieves up to \textbf{17.01\%} improvement in generalization performance and \textbf{18.44\%} improvement in inductive learning performance. The code is available at https://github.com/PoorOtterBob/STOP.

Robust Centralized Interaction for OOD Learning
Jiaming Ma
Binwu Wang" 12
Pengkun Wang" 12
Zhengyang Zhou 12 Xu Wang 12 Yang Wang
"
Abstract
300
0OD
S500D
ITD
Recently, spatio-temporal graph convolutional networks have achieved dominant performance in spatio-temporal prediction tasks. However, most models relying on node-to-node messaging in- teraction exhibit sensitivity spatio-temporal shifts, encountering out-of-distribttion (OOD) challenges. To address these issues, we intro- duce Spatio-Temporal OOD) Processor (STOP), which employs centralized messaging mecha- nism along with message perturbation mech- anism to facilitate robust spatio-temporal inter- actions. Specifically, the centralized messaging mechanism integrates Context-Aware Units for coarse graiiegr spatio-temporal feature interac- tions with nodes, effectively blocking traditional node-to-node messages. We also implement a message perturbation mechanism to disrupt this messaging process, compelling the model to ex- tract generalizable contextual features from gener- ated variant enviromments. Finally, we customize spatio-temporal distributionally robust optimiza- tion approach that exposes the model to challeng- ing environments, thereby further enhancing its generalization capabilities. Compared with 14 baselines across six datasets, STOP achieves up 17.01 % improvement in generalization per- formance and 18.44% % improvement in inductive learning performance. The code is available http://ggttabbcom/oooootttBBBBBt
MAE
26.0
+25.1%
-24.1%
-3.2%
124.5%
24.0-
-23.5%
-2.1%
+21,0%
20,0
STOCN
GWNee
STNomm
STAEformer D'SIGNN
BigST
Changed nodes
New modes
Removed nodes
Inaccurate modes
Figure 1. Subfigure (a) (upper half) illustrates the predictive perfor- mance of STGNNs in various scenarios and subfigure (b) (lower half) demonstrates the sensitivity of the node-to-node messaging mechanism to spatio-temporal shift.
sis (Xia et al., 2024; Liang er al., 2023; Miao et al., 2024; Zhang et al., 2023; Liu et al., 2024a). Currently, spatio- temporal graph convolutional networks (STGNNs) have emerged the primary tools in this field. These models generalize spatio-temporal graph dependencies and employ node-to-node messaging mechanism (such as GCN or Trans- former) for feature interaction. Finally, the generated unique representation is fed into a decoder to generate prediction.
However, the encouraging success of STGNNs predicated on the independent and identically distributed (IID) assump- tion. In reality, the distributional characteristies (such as mean and variance) graph structures of spatio-temporal data evolve over time, presenting out-of-distribution (OOD) generalization challenges for STGNNs.
1. Introduction
Spatio-temporal prediction is critical task in urban comput- ing with positive effects for various applications including traffic management, energy control, and atmospheric analy-
With the LargeST-SD dataset (Liu et al., 2023b) as an ex- ample, we report the performance advanced STGNNs in both IID and OOD scenarios, as shown Figure (a). The results indicate that their performance rapidly deteriorates when facing spatio-temporal OOD challenges, particularly in structural shift (S-OOD) scenario. Embarrassingly, the models' reliance on node-to-node messaging appears to hin- der their effectiveness. Ablation experiments (as shown in Table 2) further support this: for some advanced STGNNs, variants without node-to-node node messaging mechanism per-
University of Science and Technology of China (USTC), Hefei, China Suzhou Institute for Advanced Research, USTC, Suzhou, China. First author mail: JiamingMa @ mail.ustc.edu.cm Corre- spondence Binwu Wang and Yang Wang as corresponding authors <wbw sustcedd..cn and angyan@ustt www....d..c.........
Proceedings of the nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Robust Spatio-Tempordl Centralized Interaction for OOD Learning
form embarrassingly better. This is because the knowledge learned acquired through this mechanism is coupled with the features of the training graph, and this knowledge is diffi- cult to generalize to unseen graphs during testing (Hamilton et al., 7). As illustrated in Figure (b), when node features change, STGNNs struggle to accurately represent these nodes. Furthermore, when certain nodes disappear from the graph, their neighbors are significantly impacted, as they can no longer aggregate information along the trained paths. This propagation of errors through the node-to-node messaging mechanism adversely affects the accuracy of the entire graph representation. On the other hand, generating accurate representations for new nodes, i.e., inductive leamn ing, also poses a significant challenge for STGNNs (Zheng al., 2023; 2024; Wang et al., 2024b).
2. Preliminaries
We use a graph 9 - (V, A) to represent spatio-temporal data, where V means the node set with N nodes and A N is the weighted adjacency matrix of the graph G. We use X2 E RN to represent the observed graph signal at time step I, where e indicates the number of feature channels.
Training environment e" contains two characteristic ele- ments: a training graph G* = (V", A and training data (*"))") With this training environment, spatio-temporal OOD learning aims to learn a robust function f. which can accurately predict values after Tp time steps given ob- served data of past T time steps X = X2..... Xr] E RT''Nxx xe and the graph sampled from any environment e e~ E, where may have different two characteristic ele- ments with training environment e",
In this paper, Processor (STOP). Specifically, STOP's core contribution is spatio-temporal centralized interaction strategy: first, we design a centralized message passing mechanism configured with Context-Aware Units (ConAU) to learn generalizable contextual features. The interaction is realized through message exchange between ConAU and nodes blocking node-to-node messages. Furthermore, we employ a Gener- alized Perturbation Unit (GenPU) to randomly perturb the message interaction process, effectively promoting diverse training environments through the introduction of uncer- tainty, preventing the model from coupling with a single training environment. Furthermore, we customize a spatio- temporal Distributionally Robust Optimization (DRO) ob- jective for GenPU to help the model learn robust knowl- edge from challenging environments. STOP can effectively generalize this knowledge to new nodes to generate good representations. Finally, STOP synthesizes two prediction components from temporal and spatial dimensions to gener- ate the final prediction, which can enhance model robustness across comprehensive OOD scenarios.
propose Spatio-Tenne Temporal QOD
argmin, sup E x)p(x (f ()) (I) 3EE
where Y = [Xr-1, the ground-truth value.
Xr+T,] E Rxxxxx is
3. Methdology
STOP employs channel mixing module to model temporal and spatial dynamics, and after utilizing centralized mes- saging and message perturbation mechanisms for feature interaction, the generated temporal and spatial prediction components jointly determine the final prediction. The de- tails of STOP are illustrated in Figure 2 and Algorithm
3.1. Temporal Prediction Component
Temporal decomposition. In time series analysis, re- searchers (Cleveland et al., 1990; Wu et al., 2021: Zeng et al., 2023) often decompose time series into components at various time scales. Long-term patterns, such as seasonal or periodic trends, are relatively stable, while short-term patterns, like hourly traffic fluctuations, are unstable. Intu- itively, when the spatio-temporal distribution of the node changes over time, long-term patterns may remain robust (Wang al., 2024b). Hence, we employ temporal decom- position technique to model temporal dynamics different scales. Specifically, we use the padding moving average ker- nel AvgPool (:: with kernel size to decouple the input X E gg'xNxx xe into long-term patterns X, and short-term patterns X,:
Our contributions be four-folds: We introduce Spatio-Temporte OOD Processor (STOP), incorporating robust centralized messaging mechanism and a message perturbation mechanism. constrains nodes to interact exclusively with Context Aware Units (ConAU) for feature interaction, thereby enhancing the resilience of model to spatio-temporal shifts. ① The mes- sage perturbation mechanism, equipped with Generalized Perturbation Units (GenPU), disrupts node interactions with ConAU and includes specialized spatio-temporal distribu- tionally robust optimization (DRO) for GenPU, facilitating the model's s acquisition of causal knowledge across diverse environments, 4 We evaluate STOP's effectiveness against 14 baselines across six datasets, where it can achieve up to 17.01 % improvement in generalization performance and up to 18.44% improvement in inductive learning performance.
The centralized messaging
X = AvgPool (X:E) E TxNxe
(2)
x, x X X, T
where we employ padding operation AvgPool in (::)) along temporal dimension, ensuring a consist time length. Subse- quently, two distinct MLP (.) RR. Txxe - T Nxxxx are
2

---

Robust Spatio-Temporal Centralized Interaction for OOD Learning
leveraged to model the temporal interdependencies within these kinds of patterns. Finally, the outputs are mixed to yield the data representation,
et al., 2023; Han al., 2024b), which limits their general- ization capability on unknown graph structures. To address these limitations, we propose novel centralized message passing mechanism where each graph node interacts specifi- cally with an established context-aware unit through a novel low-rank attention mechanism.
H = 1LP1 (X)) + MLP2 (X,) E TxNxdo
(4)
-empool Embedding. To comprehensively cap- ture nentiitl dynamics, we employ embedding tech- niques to encode various prior information that is inddpen- dent of spatio-temports data, which facilitates the model's ability to capture generalizable spatio-tempors knowledge. Specifically, we utilize a timestamp-off day embedding E, E IR Nexd to capture the periodic dependencies of each temporal step and day-of-week embedding E, Na to model the periodic patterns at daily intervals, where N2 = 7 is the number of days in one week and N, indicates the number of sampling points in a day. For example, for some PeMS datasets, the data sampling frequency of traffic flow is five minutes, so N₂ is set 60 x 24/5 288. de is the dimension of each embedding.
Context Aware Units. We first set K context aware units (ConAU), where K is a hyperparameter and K < N. ConAUs are used to perceive generalizable contextual fea- tures from nodes, which nodes extract to achieve interac- tions. Specifically, we adopt a learnable feature vector e E RR4, for each ConAU, where de indicates the num- ber of feature channels. Thus, we can get a series of context feature vectors C = [ci, c2,..., cx] E Kxd, Next, we propose multi- low-rank attention method to achieve the interaction between nodes and ConAUs.
This mechanism can be summarized in two processes: aggregating node fea- tures to extract contextual features and diffusing contextual features for feature interaction between nodes. takes Zr E NN xd. and C as input. Inspired by the multi-head mechanism (Vaswani et al., 2017), we utilize distinct lin- ear layers project Query, Key, and Value separately into d, d,/ / dimensions with h heads. Specifically, for the i- th head where i = f1.2..... h), the calculation of low-rank attention is as follows:
Multi-head Low-rank Attention.
In addition, we further use the positional embedding P followed by Transformer (Vaswani et al., 2017) to encode the position of each data point in X. Finally, we integrate temporal prior embedding and data positional embedding to generate the output Z₁ denoted as the input representation:
Z = Concat (Ho + P, E,, Ea) E TRT
(5)
Channel Mixing Module. To capture temporal dynamics, we first mix-up the channel and temporal dimensions of the output Z, into shape N x de, where d, T * (dg + 2dp) Subsequently, we use L-layer MLP for hybrid modeling. Given the input of (-th MLP layer with residual connection technology z("), where Z1(0) Zr the forward process of 1-th MLP layer is as follows:
Z (Q, V)
(8)
softmax (aQK) T) x softmax (aKQ T) V,
(9)
Diffusiin
Aggregation
where Q Z, w(O E Rxdx K CJ E Rxxd, (i)
and V ZrJ E Nxd. Here a is scaling factor and equals to 1/vdh. w(") E Rddxdd is a learnable pa- rameter matrix, and J E [o, 10, 1jdidda is a column subma- trix of do-order identity matrix Is, E [0. 1] xd: which contains all rows and the columns (dn, (i +)++1] 1) to
z((+2) = GELU w() wh + 2 E Nxd, (6)
where l E {0, 1, L- 1} 1) and GELU ()) (Hendrycks & Gimpel, 2016) an activation function. w Rddxxdd and w! E dide are learnable parameters. After L MLP layers, we get the temporal representation denoted as Z zi/2) E Nxd, Finally, we use linear transforma- tion decoder to generate temporal prediction component Y, follows,
(dn * i) of I4, J is used to project the feature sub- space corresponding to the i-th head. The computed at- tention matrix is low-rank with high efficiency, which is explained in Appendix C.I. Finally, we splice outputs of multiple heads to generate representation for nodes:
Z. = Concat (z.",, zz!? "
z(N
E ER Nxd,
= Y, WW + b, E TR. (TTTt)))
where W. E IR (Tese) and b, E To are learnable parameters.
This attention comprises both aggregation and diffusion processes, as shown in the right half of Figure 2. The ag- gregation process, denoted by KQ node features for updating context features. Conversely, the diffusion process, denoted by QK E NxK, disperses the context features to individual nodes to facilitate feature interaction and node representation generation.
RKXN extracts
3.2. Robust Spatio-temporal Centralized Interaction
3.2.1. CENTRALIZED MESSAGING MECHANISM
STGNNs, relying on node-to-node message interaction mechanisms, are sensitive to structural shifts (Finkelshtein
Theory (Low-rank Attention). Our attention matrix has
3

---

Robust Centralized Interaction for OOD Learning
Projection MLP
ConAU A GenPU
Aggregation Diffusion
Generalized Perturbation Units
ConAU
GenPU
Changed
Ys
Removed node
GenPU
GcnPU
GenPU
Update parameters
Low-rank
New node
ConAU
Attention
X-
STOP Model
Auoken
L2
L
Prompt
Projection MLLP
+ Max
Y
Distributionally Robust Optimization
Nodes
Decomposition
Spariotemporal data
Centralized messaging mechanism
Optimization process
Figure 2. Overall architecture (left) and centralized messaging mechanism for robust spatio-temporal interaction (right).
low-rank property, resulting in high computational effi ciency of O (KNd,,, which is significantly better than the complexity of vanilla self-attention mechanism O dN2 Detailed description can be found in Appendix Section C.1.
attention mechanism control the aggregation process:
A. V;
(11)
softmax (aQKT) softmax (aKQ + +G.)
V.
3.2.2. MESSAGE PERTURBATION MECHANISM
Perturbation operation
From the perspective of ConAU, random perturbations dur- ing the aggregation process introduce environmental uncer- tainty variations, thereby forcing the model to learn gen- eralizable contextual features from these varying environ- ments. In the training phase, we leverage M GenPU in parallel conduct the perturbation operation. Accordingly, according to Equation 20, the model will individually gen- erate predictions for these M environments, represented as { (Y, 1, Y YM). which will be explained below.
We introduce the Generalized Perturbation Units (GenPU) to perturb the interaction process of centralized messaging improving generalization of the model unknown envi- ronments. Additionally, specifically design Distribu- tionally Robust Optimization (DRO) (Duchi & Namkoong, 2019) objective to optimize models and GenPU.
Generalized Perturbation Units (GenPU), To acquire ro- bust contextual features, our strategy involves disrupting the aggregation process of the centralized messaging mech- anism, which is responsible for updating context features. This approach enables us circumvent the significant com- putational overhead associated with directly perturbing the data. Specifically, we create M learnable perturbation vec- tor in the training process, denoted G = (gI.92..... 92....,99 9..M) ). where g: E R with i E 21, MM means i-th GenPU. Then, we use softmax operation to normalize gi E V get the corresponding masking probability vector g softmax (g.) E (0, 1)a Subsequently, we create a multi- nomial distribution M (gl; s). Based on this distribution, we sample a masking indices ~ (g(:s) E {0, 1 where s E (0, .N) indicates the number of sample hits (i.c. the number of values equal to in g.). Finally, we create K replicas of ğ. corresponding K ConAU. As a result, we can obtain mask matrix with log operation follows:
Spatio-temporal Distributionally Robust Optimization. With M predictions generated from different environments, our propose spatio-temporal DRO, which does not require optimizing all M branches sequentially; instead, it selects the branch with the highest loss for gradient descent, as shown in the right half of Figure 2. This approach indicates that the model performs worst in that particular environment, thereby enhancing training efficiency and encouraging the model to learn purely invariant knowledge. We designate the GenPU responsible for generating this environment as g. The specific optimization objective is defined follows:
min sup E ((x,Y) (4 (5 (X), Y;g)), f gERN
(12)
s.t. - = NO,
(13)
where ||. llo stands for zero norm. GenPUs participate the learning process by influencing the sampling distribution of the mask matrix, which is essentially non-differentiable, rather than participating in the backpropagation process as part of the parameters. Thus, we optimize the model param- eters and GenPUs alternately, as shown in Algorithm 2.
G = log (lg. g. ..........)... g.l) E 03 KNN_
(10)
The logarithmic operation is facilitate the masking op- eration during the exponential activation in the subsequent softmax normalization. If G [m, n] = O, the message be- tween m-th node and n-th ConAU is not be affected. If G. [, n| = -0, the message between m-th node and n-th ConAU is masked. Then we integrate G; into low-rank
In fact, our proposed objective belongs to the DRO paradigm, which theoretically has superior generalization

---

Robust Spatio-Temporal Centralized Interaction for OOD Learning
compared the Empirical Risk Minimization (ERM) paradigm followed by most spatio-temporal models. ERM optimizes the model using only single training environment. The details are provided in Appendix Section D.
the temporal representation Z., denoted as Z.O = Z, Z, Next, we utilize a channel mixing module with L-layer MLP to capture spatial -ighesiiennion features, with the final output denoted as the spatial representation Zs = - The forward process of the 1-th MLP layer is as follows:
Theory 2. Within a predefined uncertainty set of environ- ments E, DRO substitutes the expected risk under a single distribution with the worst-case expected risk. DRO yields tighter upper bounds compared ERM:
Z GELU Zoow w!O + - E Nxd,
(18)
where W (1) Rd xdd, and 4 E Rdd, xd are learnable parameters. Finally, same as the temporal part, we also use a linear layer to decode the spatial representation Z, to produce a prediction from the spatial component:
sup {E (x () ():e)) ( (X), Y)]
cEE' [ ()x (X), Y) 6,
(14)
where 22 Vx-xx.e) (f (X), Y) and the distance D (...) between any E & and training envi- ronment e" is less than or equal to p. and "sup" means the supremum. This reveals that DRO allows the model to adapt more robustly to various environments than ERM.
Y, = Zz + b, (x(TTT)))
(19)
where W. E (dx((Tpe)) and b, E TT are learnable parameters.
3.3. Final Prediction
Robustness Analysis. Centralized message mechanism re- stricts interactions operate between nodes and ConAUs, avoiding the complexities of node-to-node interactions. Structural shift does not significantly disrupt the message passing paths between nodes and ConAUs. Furthermore, GenPU introduces uncertain perturbations during this spatio- temporal interaction process, generating diverse training environments. This strategy prevents the model from over-relying on a single training environment. Our spatio- temporal DRO forces the model to interact with the most challenging instances in the generated environments, which can further enhance the model's 's robustness. Moreover, the learned context features can flexibly extend to newly added nodes, improving its inductive learning capability.
We sum the predictions from the spatial and temporal di- mensions to get finial prediction Y as follow,
Y Y. E ERRNTTX
(20)
4. Experiments
In this section, we conduct comprehensive evaluation of the proposed model. We will answer the following poten- tial questions in the following subsections by-eby-oee Q.1. What is the generalization performance of STOP in spatio- temporal OOD scenarios? Q.2. Is the proposed centralized messaging mechanism effective? Q.3. How sensitive is the model to hyperparameters M and K? Q.4. Is each compo- nent of the model valid for OOD capabilities? Q.5. How efficient is the model? Q.6. What is the inductive learning ability of STOP for new nodes?
3.2.3. SPATIAL PREDICTION COMPONENT
After feature interaction, we extract shared contextual fea- tures from ConAUs for each node. To further enhance the node representation, individual nodes. This refinement involves subtracting the contextual features from the temporal representation to iso- late the personalized feature representation of each node, denoted as Zp:
In Appendix, we analyze the effectiveness of STOP in vari- ous OOD scenarios (temporal OOD or structural OOD in Section E.5, rapid expansion OOD in Section E.6), and provide visualization examples in Section E.
refine the personalized features of
Table I. Summary of the used datasets.
Zp Z, Z. E R. Nxde
(15)
Dataset
Nodes
Edges
Years
LargeST-SD
716
17,319
2017 2021
Subsequently, we concatenate the decoupled context fea- tures Ze and personalized features Zp. and then linearly map them back to the initial representation.
LargeST-GBA
2,352
61,246
2017 ~ 2021
LargeST-GLA
3,834
201,363
2017 ~ 2021
LargeST-CA
8,600
525,888
2017 ~ 2021
PEMSD3-Stream
655
1,577
2011 ~ 2017
Z GELU (Concat (Zp, Ze) W1) We e Nxd, (16)
KnowAir
184
3,796
2015 2018
Z = LayerNorm (Z,- + Zr) E I Nxd.
(17)
4.1. Experiment Setting
where W, E 4d, and W₂ E dddtxd are learnable parameters. We then decouple spatial components by calcu- lating the difference between the input representation Z, and
Setting. We set both the input and prediction windows
to 12 in traffic prediction and 24 atmospheric prediction.
5

---

Robust Spatio-Tempordl Centralized Interaction for OOD Learning
Temporal decomposition kernel size & is equal to 3 in traffic datasets and 7 in KnowAir. The number of ConAU K is set to {8. 24, 32, 64, 8, 4] and the number of GenPU M is equal to {3, 3, 3, 3, 2, 4} in six datasets in Table 1. The dimensions of embeddings are set to 64, We use 8 heads in multi-head low-rank attention. We implement all models using P PyTorch framework of Python 3.8.3 and leveraging the Nvidia A A100-PCIE-4OGB as support, MAE, RMSE, and MAPE are used as metries for comparison.
datasets can be found in Appendix E.3, and detailed year- specific reports can be found in Appendix E.9. The com- parison with spatio-temporal continuous learning models is shown in Table 11.
GCN-based models like STGCN and GWNet underperform in OOD settings due to their reliance on the global messag- ing mechanism of GCN, rendering them highly sensitive to spatio-temporal shifts. Transformer-basee models such as STAEformer and DºSTGNN exhibit improved predictive ac- curacy by leveraging self-attention mechanisms to aggregate global node features, effectively addressing spatio-temporal shift errors. Despite these advancements, STGNNs still face challenges in generalizing weights for unseen graph struc- tures. On the other hand, spatio-temporal OOD learning baselines like STONE introduce diverse training environ- ments utilizing perturbation gennrratd semantic relations to learn invariant causal knowledge, resulting in enhanced performance. STOP demonstrates significant improvements across various metrics, with maximum enhancement of 17.01%. This improvement can be attributed to its robust centralized messaging mechanism, which facilitates effec- tive spatial feature interaction.
@ Datasets & Baselines. We conduct comprehensive eval- uation of our model on six -emporal datasets spanning multiple years across two domains. These datasets include LargeST (Liu et al., 2024b) and PEMSD3- Streem (Chen et al., 2021) in the traffic domain, and KnowAir (Wang et al., 2020) in the atmospheric domain. The dataset summary is presented in Table 1. Our comparison involves advanced spatio-temporal prediction, spatio-temporal OOD learn- ing, and spatio-temporal continual learning model. ! Spatio-temporal predictionn models include STGCN (Yu et al., 2017), GWNet (Wu et al.,, 2019), STNorm (Deng et al., 2021), STID (Shao et al., 2022a), STAFformer (Liu al., 2023a), STNN (Yang et al., 2021), DºSTGNN (Shao al., 2022b), BigST (Han et al., 2024a), and RPMixer (Yeh et al., 2024). @ Spatio-temporal OOD learning models include CaST (Xia et al., 2024) and STONE (Wang et al., 2024a). @ Spatio-temporal continual learning models include Traf- fieStream (Chen et al., 2021), PEMCP (Wang et al., 2023b), and TFMoE (Lee & Park, 2024). Some models require the removal of son-essential components (such as node embed- ding in STID or adaptive graph learning method in GWNet) to adapt them to the ST-OOD setting, as the parameters of them are intertwined with the scale of the graph structure, as elaborated in Appendix E.
4.3. Node-to-node Interaction vs. Ours (Q.2)
We used two representative backbone models, STGCN and STAEformer, to compare the effectiveness of different mes- sage mechanisms. The former utilizes graph convolution for inter-node interaction, while the latter employs self-attention mechanisms for node-to-node interaction. We removed their node-to-node message mechanisms and labeled these vari- ants as "-graph". Additionally, we replaced their inter-node interactions with our spatial interaction mechanism, denot- ing these variants as "+Ours". Using the SD and KnowAir datasets with OOD settings, the performance results are shown in Table 2. We can find that variants without node- to-node message mechanisms performed better than the original models, indicating that the initial mechanism lim- ited the models' generalization performance. Furthermore, these models achieve generalization performance improve- ments after integrating our centralized messaging mecha- nism. Therefore, this further validates the effectiveness and necessity of our central interaction mechanism.
@ ST-OOD Datasets. For the evaluation of temporal shift, we train the models using data from the first year and test them on each subsequent year. The training set comprises the first 60% of data from the initial year dataset, while the following 20% of data is used as the validation set. In each subsequent year, the last 20% of data is designated as the test set. This setup aims to accentuate the temporal distribution difference between the test and training sets, while maintaining a ratio of approximately 6:2:2 for the training, validation, and test sets. Regarding structural shift evaluation, we select a subset of nodes for training and validation. In the test set, decrease the number of nodes by 10% and introduce 30% new nodes to simulate shifts in the graph structure and scale. More detailed settings can be found in Appendix E.2.
Table 2. Comparison of two interaction mechanisms.
SD
KnowAir
Variant
MAE
RMSE
MAPE
MAE
RMSE
MAPE
STGCN
25.72
40.03
18.21
29.49
40.93
63.85
4.2. OOD Performance Comparison((...
STGCN -grahh
25.45
39.62
17.98
26.18
38.03
55.75
STGCN Ours
24.87
38.98
17.65
25.44
37.42
52.80
As shown in Table 3, we report the average values across all years of test sets on four datasets. Experiments on large
Fforerr
26.20
41.18
18.39
27.25
38.93
STAElormer graph
25.80
40.84
17.45
25.82
37.28
55.65
STAFFormer + Ours
24.65
38.46
17.30
25.46
37.25
55.04
6

---

Robust Spatio-Temporal Centralized Interaction for OOD Learning
Table OOD performance comparisons four datasets, The unit of MAPE is percent (%). We hold the best-performing model results in red and underline the sub-optimal model results in blue.
Method
Ours
STONE
CaST
RPMixer
BigST
D'STGNN
STNN
STAFformer
STID
STNorm
GWNet
STGCN
MAE
17.71
18.44
21.35
24.92
18.56
18.70
36.46
18.70
19.68
18.82
20.15
18.68
3
RMSE
28.45
29.55
33.28
39.88
29.93
29.31
56.84
28.97
29.56
30.06
31.34
29.61
MAPE
11.73
12.32
16.04
15.63
12.18
13.04
26.91
12.62
13.18
12.82
14.44
12.92
MAE
25.10
29.28
42.37
25.66
25.13
25.87
26.00
25.25
RMSE
37.71
39.66
45.24
66.45
40.61
38.77
57.59
40.73
40.86
41.20
39.48
MAPE
15.99
17.56
21.49
26.15
18.03
17.46
27.15
17.59
18.03
18.03
21.17
17.34
MAE
32,59
37.12
42.40
77.31
37.89
36,35
41.69
37.17
38,30
38,08
39.75
36.15
RMSE
51.82
54,60
64.05
115.62
58.74
53.60
64,99
57.81
$9.40
59.24
61.08
55.74
MAPE
22.89
25.90
31.73
49.48
27.12
25.98
31.32
27.07
26.90
27.89
31,46
26.41
MAE
18.33
20.19
21.85
24.79
19.92
19.10
40.61
20.91
19.09
20.86
20.65
21.49
3
RMSE
29.70
33.65
34.32
39.59
32,33
32.64
60.07
33.59
31.40
32.92
32.21
33.57
MAPE
13.64
15.10
18.61
17.06
14.75
14.29
33.77
14.93
14.36
16.00
15,70
14.79
MAE
24.75
25.84
29.70
40.77
28.64
40,50
28.61
26.90
31.24
30.05
GBA
RMSE
38.48
41.96
45.16
62.24
43.93
41.72
42.15
46.69
44.97
MAPE
20.48
21.24
25.77
29.48
22.25
21.26
33.68
21.79
25.57
22.74
24
MAE
34,93
39.56
42,60
72,51
42.87
36.26
44,62
41.68
39,36
45.73
39.61
43.29
RMSE
53.10
56.18
63.33
104.93
63.06
56.23
65.61
62.28
59.60
65.62
58.33
62.34
MAPE
31.09
32.18
36.88
56.28
34.52
32.23
38.28
34.99
33.43
41.02
33.67
35.23
MAE
11.39
13.27
15.43
14.68
12.79
12.89
17.04
12.81
12,96
13.03
12.97
13.39
3
RMSE
19.48
21.48
24.53
23.73
20.79
21.14
28.47
21.02
20.95
21.07
21.11
21.60
PEMSD3 Stream
MAPE
15.45
17.06
32.15
18.02
17.30
16.58
23.63
16.48
16.66
20.44
16.41
16.71
MAE
12.47
14.30
17.13
17.41
14.05
14.08
17.26
14.14
14.18
14.51
14.14
14.63
RMSE
21.62
23.68
27.63
28.61
23.07
29.27
23.38
23.19
23.67
23.31
23.82
MAPE
16.02
18.23
33.77
20.90
25.63
18.52
22.43
18.33
MAE
14.36
16.28
20.96
24.00
16.65
16.55
18.19
16.71
16.56
17.04
16,37
17.25
12
RMSE
24.95
28.41
33.82
39.64
27.46
27.44
30.14
27.92
27.31
27.94
27.10
28.20
MAPE
18.66
20.94
39.07
27.84
23.59
20.12
30,81
20.95
21.25
25.30
20.29
21.30
MAE
24.37
25.68
26.20
30.56
26.89
26.43
27.85
26.
26.49
28.46
27.84
27.92
RMSE
36.56
37.59
38.42
45.34
39.16
37.91
39.07
37.82
38.90
41.47
40.25
39.47
MAPE
$1.94
52.41
59.53
69. 69.06
57.45
58.39
65.74
52,90
57.84
65.26
52.42
58.32
Know-Air
MAE
27.03
28. 96
29.49
38.45
29.77
30.06
30.48
29.45
30.85
30,86
31.11
31.63
12
RMSE
40.29
42.64
41.98
55.26
41.75
42.52
42.67
41.71
44.59
43.87
43.65
43.71
MAPE
54.45
71.99
70.15
87.60
68.39
67.10
71.05
61.64
68.44
71.83
61.51
69.83
MAE
28.70
30.56
31.63
42.67
31,57
30.94
31.48
30.96
32,78
32.52
32.99
34.68
RMSE
42.39
45.48
45.21
61.30
44.52
46.21
44.72
43,48
46.67
44.80
44.14
47.19
MAPE
57.96
75.11
75.36
94.76
76.76
69.84
74.14
65.31
74.02
81.32
70.84
80.49
4.4. Hyperparameter Sensitivity Analysis (Q.3)
creases the learning difficulty of the model to extract causal knowledge.
We analyze the sensitivity of the numer of ConAU and GenPU on the SD (upper) and KnowAir (lower) datasets on the Figure 3. Hyperparameter experiments on other datasets are provided in Appendix Section 1. When the number of ConAU K is set to in SD dataset and in KnowAir dataset. When K exceeds this value, the model creates too many ConAU, making it unable to focus extracting in- variant contextual features, thus introducing noise. When K less than this value, too few perception units fail to learn sufficient invariant knowledge, leading to a decrease in the model's generalization performance. The number of GenPU M is set in SD dataset and in KnowAir dataset. A smaller M may not provide sufficient training environment diversity, resulting in performance degradation. On the other hand, an excessive number of GenPU does not necessarily improve performance. Too large M means that the generated environment too complex, which in-
4.5. Ablation Study (Q.4)
In this section, we evaluate the effectiveness of each compo- nent on SD and KnowAir datasets. "w/o decom" removes the time decomposition module, "w/o prompt" eliminates the spatio-temporal prompting method, "w/o Y," uses only spatial prediction the final prediction. "w/o LA" means that
vamilla self-attention mechanism to replace the low-rank attention module.
As illustrated in Figure 4, the results show that each com- ponent of STOP helps to improve the OOD generaliza- tion. "w/o Y achieves poor prediction performance, which proves that the proposed collaborative component effec- tive for OOD. "w/o ConAU" removes ConAU and achieves high errors, proving that spatio-temporal interaction is also

---

Robust Centralized Interaction for OOD Learning
MAT & RASS
0-1
MALE RASE MAPNNN
170
MAEE
MAPEESO
61.4-
KYR2
MAPE (SM
MAPEAM
MMAE RAMSSE
(10
32 64 128 212
ConAU
GenPU M
W.Y.
MO
BRO
MEAE &: RMSE
TALLE RNSE
MAPRIN
MAPRC50
RMNE
373-
20.4
MAA
MAPE
MAPE(N)
ROOSE
290
185
66000
proupt
4014
We CRAd
GeaPd) BRO
# ConAU K
GenPU
Figure 4. Ablation experiments on SD (upper) and KnowAir (lower) datasets.
Figure 3. Hyperparameter Experiments for K and M.
necessary in OOD scenarios. "w/o GenPU" has higher pre- diction errors because GenPU can help the model extract causal knowledge and enhance model robustness. Ablation experiments demonstrate the necessity and effectiveness of each module of our model, while we further perform com- prehensive ablation experiment including double module ablation in Appendix Section E010.
vanced baselines for new nodes on SD and GBA datasets in Table 4 and CA and GLA datasets Table 7. Specifically, Transformer-basee models, like DºSTGNN, are good gen- eralizing because they can create accurate representations for new nodes. GCN-based models are poor at generalizing because they rely heavily on the original graph structure and can not generate accurate representations for new nodes. Transformer-bant models struggle because they can not create robust weights for new nodes. However, the STONE framework uses a novel embedding method to generate good initial representations for nodes. Our model stands out by using centralized messaging mechanism to access con- textual features and enhance representations, making it excel in extending performance to new nodes.
4.6. Efficiency Study (Q.5)
The training time of per epoch is illustrated in Figure 5, we can see that STOP demonstrates remarkable effectiveness and efficiency the SD dataset. Since model primarily uses lightweight MLP layers to model temporal and spa- tial dynamics. Compared to the SOTA model D²SNGNN which a Transformer-based model with RNNs for tem- poral dynamics modeling and GNNs for spatial dynamics modeling, our model has improved the efficiency by about 20 times (60.57 s/epoch vs 1220.79 s/epoch). Because its transformer-base based architecture introduces quadratic de- pendency the number of spatial nodes, leading to high computational complexity. Furthermore, its recursive tem- poral modeling mechanism requires storing hidden repre- sentations at each time step, which significantly increases both memory consumption and inference latency. Conse- quently, such sequential approaches encounter substantial scalability barriers when applied large-scale air quality forecasting tasks. In contrast, model demonstrates supe- rior computational efficiency. This is attributed to our novel spatiotemporal interaction module, which captures complex dynamics with near-linear time complexity. As a result, our approach enables highly scalable deployment.
CaST
80,000 MB
29
45,000 MB
Average MAE
28
GWNet
1.5 0000 MB
3,000 MB
27
Memory-Usage
STNorm
BigST
STAEformer
26
STGCN
STONE
SSGGNN
25
24
STOP (ours)
Training speed (s/epoch)
23
0
200
400
600
800
1000
1200
Figure 5, Efficiency study of STOP SD datasct.
5. Related Work
4.7. Inductive Learning Performance of STOP (Q.6)
5.1. Spatio-temporal Prediction
In OOD scenarios, the model's s ability to represent unseen nodes is a challenge, which mainly involves the model's inductive learning. We report the performance of some ad-
As a crucial task intelligent transportation systems, cur- rent SOTA spatio-temporal prediction models primarily rely
8

---

Robust Spatio-Temporal Centralized Interaction for OOD Learning
stance, RevIN (Kim et al., 2021) employs symmetric structure eliminate and reconstruct distribution informa- tion based on the input window's 's statistics. AdaRNN (Du et al., 2021) categorizes historical time sequences into dif- ferent classes and dynamically matches input data to these
Table 4. Inductive learning performance comparisons.
GBA
Model
MAE
RMSE
MAPE
MAE
RMSE
MАPE
STONE
25.06
39.15
18.12
26.28
42.02
24.12
CaST
28.83
44.10
22.93
29.62
44.97
25.56
classes for contextual information identification. Addition- ally, a reversible instance normalization technique, proposed by (Kim al., 2021), aims to mitigate temporal distribution shift issues. Non-stationary Transformers (Liu al., 2022) introduce commalization-éenommaizzzzz
BigST
25.22
39.22
18.02
28.75
43,80
24.32
DESTGNN
25.85
39,33
20.07
26.36
42.15
25.96
STAEformer
25,28
39.42
18.41
28.61
44.87
27.61
STID
25.40
39.53
18.66
26.83
43.94
24.42
GWNett
29.01
44.18
22.82
29.15
43.60
28.00
technique to sta- bilize time series data, mainly for transformer-base based mod- cls. SAF (Arik et al., 2022) suggests test-time adaptation through a self-supervised objective to enhance adaptation against distribution shifts. DIVERSIFY (Lu et al., 2024) aims to leverage subdomains within dataset mitigate challenges arising from non-stationary generalized repre- sentation learning. However, these models often overlook the modeling of spatial dependencies. Spatial modeling is crucial in the field of spatio-temporal prediction, as it can examine the states of neighboring nodes to enhance predic- tion performance, given the strong correlations that often exist among neighboring nodes (Jin et al., 2023; Shao et al., 2023).
Ours
22.74
36,09
16.82
24.76
38.52
19.67
on STGNNs (Wang et al., 2024b;;; Zhang et al., 2025; 2024; Huang et al., 2023). However, they assume similarity be- tween training and testing environments. To learn from evolving spatio-temporal graphs, some representative con- tinual learning methods (Chen et al., 2021; Wang et al., 2023a) sequentially fine-tune models using data subsets with new distributions to adapt to spatio-temporal changes. Unfortunately, these models' effectiveness is only proven in environments similar to the training set, leading to chal- lenges when encountering OOD scenarios. To overcome OOD challenges, researchers initially addressed temporal shift challenges, with models focusing on extracting invari- ant spatio-temporal knowledge from training data. However, they largely overlooked the evolution of spatioltemporal graph structures- which poses challenges for their relied- upon node-to-node interaction mechanisms. In this paper, we propose a novel message passing mechanism to over- come these challenges.
6. Conclusion
In this paper, we propose a Spatio-temporal Out-of- distribution Processor (STOP), which combines spatial inter- action mechanisms and message perturbation mechanisms to enhance adaptation spatio-temporal variations. We employ a centralized message passing mechanism and mes- sage shape perturbation mechanism to replace traditional point-to-po to-t-to- message interaction strategies. STOP can learn generalizable knowledge from diverse training envi- ronments. Evaluation on extensive datasets across various OOD scenarios demonstrates the model's robust generaliza- tion, inductive learning capabilities, and efficiency.
5.2. Spatio-temporal OOD Learning
Inspired by advances in time series shift learning (Liu et al., 2022) discussed in Appendix A, researchers have specifi- cally designed spatio-temporal OOD learning models. For example, CauSTG (Zhou et al., 2023a) introduces causal framework that transfers global invariant spatio-temporal relationships to OOD scenarios. CaST (Xia et al., 2023) employs a structural causal model to elucidate the data gen- eration process of spatio-temporal graphs. STONE ( (Wang et al., 2024a) proposes causal graph structure learn ro- bust spatio-temporal semantic relationship. STEVE (Hu et al., 2023) encodes traffic data into two disentangled rep- resentations and utilizes spatio-temporal environments as self-supervised signals. In this paper, we reformulate their message-passing mechanism, addressing the OOD chal- lenge from a novel perspective.
Impact Statement
This paper presents research aimed at advancing the field of Machine Learning. While there are many potential societal consequences of our work, we believe that none require specific emphasis this context.
A Acknowledgment
This paper is partially supported by the National Natural Science Foundation of China (No.12227901) The AI- driven experiments, simulations and model training were performed on the robotic Al-Scientist platform of Chinese Academy of Sciences.
5.3. Temporal Shift in Time Series
Various models have been developed in the time series do- main to address temporal shifts in time series data, par- ticularly focusing on OOD learning challenges. For in-
9

---

Robust Spatio-Temporal Centralized Interaction for OOD Learning
References
Han, L.., Chen, X.-Y., Ye, H.-J., and Zhan, D.-C. Softs: Efficient multivariate time series forecasting with series- core fusion. arXiv preprint arXiv:2404. 14197, 2024b.
Arik, S. O.. Yoder, N. C., and Pfister, T. Self-adaptive forecasting for improved deep learning non-stationary time-series. arXiv preprint arXiv:2202.02400) 2022.
Hendrycks, D. and Gimpel, K. Gaussian enror linear units (gelus). arXiv preprint 1xXv:1600.004155 2016.
Arjovsky, M., Bottou, L., Gulrajani, I., and Lopez- Paz, D. Invariant risk minimization. arXiv preprint AXXiv:907.020600 2019.
Hu, J., Liang, Y., Fan, Z., Chen, H., Zheng, Y., and Zim- mermann, R. Graph neural processes for spatio-temporal extrapolation. arXiv preprint arXiv:2305.18719 2023.
Bai, L., Yao, L., Li, C., Wang, X., and Wang, C. Adaptive graph convolutional recurrent network for traffic forecast- ing. Advances in neural information processing systems, 33:17804-17815, 2020.
Huang. Q.. Shen, L.,, Zhang, R., Ding, S., Wang, B., Zhou, Z., and Wang, Y. Crossgnn: Confronting noisy multi- variate time series via cross interaction refinement. Ad- vances in Neural Information Processing Systems, 36: 46885-46902, 2023.
Chen, W. and Liang, Y. Expand and compress: Explor- ing tuning principles for continual spatio-temporal graph forecasting. arXiv preprint arXiv:2410.1259. 2024.
Jin, M., Koh, H. Y., Wen, Q., Zambon, D., Alippi, C., Webb, G. I., King, I., and Pan, S. A survey on graph neural networks for time series: Forecasting, classifica- tion, imputation, and anomaly detection. arXiv preprint arXiv:2307.03759, 2023.
Chen, X., Wang, J., and Xie, K. Trafficstream: A stream- ing traffic flow forecasting framework based on graph neural networks and continual learning. arXiv preprint arXiv:2106.06275. 2021.
Cleveland, R. B., Cleveland, W. S., McRae, J. E., Terpen- ning, I., et al. Stl: A seasonal-trend decomposition. J. off. Stat, 6(1):3-73, 1990.
Kim, T., Kim, J., Tae, Y., Park, C., Choi, J.-H., and Choo, J. Reversible instance normalization for accurate time-series forecasting against distribution shift. In International
Deng, J., Chen, X., Jiang, R., Song, X., and Tsang, I. W. St-norm: Spatial and temporal normalization for multi-
Conference on Learning Representations, 2021.
variate time series forecasting. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, pp. 269-278, 2021.
Lee, S. and Park, C. Continual traffic forecasting via mixture of experts. arXiv preprint arXiv:2406.03114. 2024.
Levy, D., Carmon, Y., Duchi, J. C.,, and Sidford, A. Large- scale methods for distributionally robust optimization. Advances in Neural Information Processing Systems, 33: 8847-8860, 2020.
Du, Y., Wang, J., Feng, W., Pan, S., Qin, T., Xu, R., and Wang, C. Adarnn: Adaptive learning and forecasting of time series. In Proceedings of the 30th ACM international conference on information & knowledge management, pp-
402-411, 2021.
Li, F., Feng. J., Yan, H., Jin, G., Yang, F., Sun, F., Jin, D., and Li, Y. Dynamic graph convolutional recurrent network for traffic prediction: Benchmark and solution. ACM Transactions on Knowledge Discovery from Data, 17(1):1-21, 2023.
Duchi, J. and Namkoong, H. Variance-based regularization with convex objectives. Journal of Machine Learning Research, 20(68):1-55 2019.
Finkelshtein, B., Huang, X., Bronstein, M., and Ceylan, I. Cooperative graph neural networks. arXiv preprint arXiv:23 2023.
Li, Y., Yu, R., Shahabi, C., and Liu, Y. Diffusion con- volutional recurrent neural network: Data-driven traffic forecasting, arXiv preprint 1rXiv:1707.01926, 2017.
2017.
Greub, W. H. Linear algebra, volume 23. Springer Science & Business Media, 2012.
Liang. Y., Xia, Y., Ke, S., Wang, Y., Wen, Q.. Zhang, J.. Zheng, Y., and Zimmermann. R. Airformer: Predicting nationwide air quality in china with transformers. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 37, pp. 13329-14337, 2023.
Hamilton, W., Ying, Z., and Leskovec, J. Inductive repre- sentation learning large graphs. Advances in neural information processing systems, 30, 2017.
Han, J., Zhang, W., Liu, H., Tao, T., Tan, N., and Xiong, H. Bigst: Linear complexity spatio-temporal graph neural network for traffic forecasting on large-scale road net- works. Proceedings of the VLDB Endowment, 17(5): 1081-1090, 2024a.
Liu, C., Yang, S., Xu, Q. Li, Z., Long, C., Li, Z., and Zhao, R. Spatial-temporal large language model for traffic prediction. 25th IEEE International Conference on Mobile Data Management, pp. 31 31-40, 2024a.
10

---

Robust Spatio-Temporal Centralized Interaction for OOD Learning
Liu, C.,, Hettige, K. H., Xu, Q. Long, C., Xiang, S., Cong, G., Li, Z., and Zhao, R. St-llm+: Graph enhanced spatio- temporal large language models for traffic prediction. IEEE Transactions on Knowledge and Data Engineering, 2025a.
Miao, H., Zhao, Y., Guo, C., Yang, B., Zheng, K., and Jensen, C. S. Spatio-temporal prediction on streaming data: A unified federated continuous learning framework. IEEE Transactions on Knowledge and Data Engineering, 2025.
Liu, C., Xu, Q., Miao, H., Yang, S., Zhang, L., Long, C., Li, Z., and Zhao, R. Timecma: Towards llm-empowered multivariate time series forecasting via cross- modslmyy alignment. In Proceedings of the AAAI Conference on Ar- rificial Intelligence, volume 39, pp. 18780-18786 2025b.
Namkoong, H. and Duchi, J. C. Stochastic gradient methods for distributionally robust optimization with f- divergences. Advances in neural information processing systems, 29, 2016.
Pan, Z., Liang, Y., Wang, W., Yu, Y., Zheng. Y., and Zhang, J. Urban traffic prediction from spatio-temporal data using deep meta learning. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 1720-1730, 2019.
Liu, C., Zhou, S., Xu, Q. Miao, H., Long, C., Li, Z., and Zhao, R. Towards cross-modality modeling for time series analytics: A survey in the Ilm era. arXiv preprint arXiv:2505.02588 2025c.
Qiao, F. and Peng, X. Topology aware robust optimiza- tion for out-of-distribution generalization. arXiv preprint arXiv:2307.139443 2023.
Liu, H., Dong, Z., Jiang, R., Deng, J., Deng. J., Chen, Q. and Song, X. Spatio-temporal adaptive embedding makes vanilla transformer sota for traffic forecasting. In Proceedings of the 32nd ACM international conference on information and knowledge management, pp. 4125- 4129, 2023a.
Shaficezadch Abadch, S., Nguyen, V. A., Kuhn, D., and Mohajerin Esfahani, P. M. Wasserstein distributionally robust kalman filtering. Advances in Neural Information Processing Systems, 31, 2018.
Liu, X., Xia, Y., Liang, Y., Hu, J.. Wang, Y., Bai, L., Huang, C., Liu, Z.,, Hooi, B., and Zimmermann, R. Largest: A benchmark dataset for large-scale traffic forecasting. arXiv preprint arXiv:2306.0m 230600890,, 2023b.
Shao, Z., Zhang. Z., Wang, F., Wei. W., and Xu, Spatial- temporal identity: A simple yet effective baseline for multivariate time series forecasting. In Proceedings of the s ACM International Conference on Information & Knowledge Management, pp. 4454-4458, 2022a.
Liu, X., Xia, Y. Liang, Y., Hu, J., Wang, Y., Bai, I.., Huang, C., Liu, Z., Hooi, B., and Zimmermann, R. Largest: A benchmark dataset for large-scale traffic forecasting. Advances in Neural Information Processing Systems, 36, 2024b.
Shao, Z.,, Zhang, Z.,, Wei, W., Wang, F., Xu, Y., Cao, X., and Jensen, C. S. Decoupled dynamic spatial-temporal graph neural network for traffic forecasting. Proc. VLDB Endow, (5(11):2733-2746 2022b.
Liu, Y., Wu, H., Wang, J., and Long, M. Non-stationary transformers: Exploring the stationarity in time series forecasting. Advances in Neural Information Processing Systems, 35:9881-9988 1993, 2022.
Shao, 7., Wang, F., Xu, Y., Wei, W., Yu, C., Zhang, Z., Yao, D., Jin, G., Cao, X., Cong. G., al. Exploring progress in multivariate time series forecasting: Comprehensive benchmarking and heterogeneity analysis. arXiv preprint arXiv:2310.061/ 19, 2023.
Lu, W., Wang, J., Sun, X., Chen, Y., Ji, X., Yang, Q., and Xie, X. Diversify: A general framework for time series out-of distiiibittitoion detection and generalization. IEEE Transactions on Pattern Analysis and Machine Intelli- gence, 2024.
Staib, M. and Jegelka, S. Distributionally robust optimiza- tion and generalization in kernel methods. Advances in Neural Information Processing Systems, 32, 2019.
Vapnik, V. Statistical learning theory. John Wiley & Sons google schola, 2:831-842, 1998.
Ma, J. Wang, G., Huang, S., Yang, K., Wang, B., Wang, P., and Wang, Y. Spatiotemporal causal decou- pling model for air quality forecasting. arXiv preprint 2XXi:25505.20119 2025.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, I.., Gomez, A. N., Kaiser, t.., and Polosukhin, At- tention is all you need. Advances neural information processing systems, 30, 2017.
Miao, H., Zhao, Y., Guo, C., Yang, B., Kai, Z., Huang, F., Xie, J., and Jensen, C. S. A unified replay-based contin- uous learning framework for spatio-tempore prediction on streaming data. In ICDE, 2024.
Wang, B., Zhang, Y., Shi, J., Wang, P., Wang, X., Bai, L., and Wang, Y. Knowledge expansion and consolidation for continual traffic prediction with expanding graphs.

---

Robust Tpatpo-Temppoal Centralized Interaction for OOD Learning
IEEE Transactions on Intelligent Transportation Systems, 2023a.
Yang, S., Liu, J., and Zhao, K. Space meets time: Local spacetime neural network for traffic flow forecasting. In 2021 IEEE International Conference Data Mining (ICDM), pp. 817-826. IEEE, 2021.
Wang. B., Zhang, Y., Wang. X., Wang, P., Zhou, Z., Bai, L., and Wang, Y. Pattern expansion and consolidation on evolving graphs for continual traffic prediction. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 2223-2232, 2023b.
Yeh, C.-C. M., Fan, Y., Dai, X., Saini, U. S., Lai, V., Aboagye, O., Wang, J., Chen, H., Zheng, Y., Zhuang, Z., et al. Rpmixer: Shaking up time series forecasting with random projections for large spatial-temporal data. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 3919-3930, 2024.
Wang, B., Ma, J., Wang, P., Wang, X., Zhang, Y., Zhou, Z., and Wang, Y. Stone: A spatio-temporal ood learn- ing framework kills both spatial and temporal shifts. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 2948-2959, 2024a.
Yu, B., Yin, H., and Zhu, Z. Spatio-temporal graph convo- lutional networks: deep learning framework for traffic forecasting. arXiv preprint arXiv: 7097779944775 2017.
Wang, B., Wang, P., Zhang, Y., Wang, X., Zhou, Z., Bai, L.., and Wang, Y. Towards dynamic spatial-temporal graph learning: A decoupled perspective. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 9089-9097 2024b.
Zeng, A., Chen, M., Zhang, L., and Xu, Q. Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence, volume 37, pp. 11121-11128, 2023.
Zhang, G., Yi, Yuan, J., Li, Y., and Jin, D. Das: Efficient street view image sampling for urban prediction. ACM Transuctions on Intelligent Systems and Technology, 14 (2):1-20, 2023.
Wang, B., Wang, P., Zhang, Y., Wang, X., Zhou, Z., and Wang, Y. Condition-guider urban traffic co-prediction with multiple sparse surveillance data. IEEE Transactions on Vehicular Technology, 2024c.
Zhang. J., Zheng, Y., Qi, D., Li, R., and Yi, X. Dnn-based prediction model for spatio-tempon data. In Proceedings of the 24th ACM SIGSPATIAL international conference advances in geographic information systems, pp. 1-4, 2016.
Wang, S., Li, Y., Zhang, J., Meng, Q., Meng, L., and Gao, Pm2. 5-gnn: A domain knowledge enhanced graph neural network for pm2. 5 forecasting. In Proceedings of the 28th international conference on advances in geographic information systems, pp. 163-166, 2020.
Zhang, Y., Wang, X., Wang, P., Wang, B., Zhou, Z., and Wang, Y. Modeling spatio-temporal mobility across data silos via personalized federated learning. IEEE Transac- tions on Mobile Computing, 2024.
Wang, X., Wang, P., Wang, B., Zhang, Y., Zhou, Z., Bai, L., and Wang. Y. Latent gaussian processes based graph learning for urban traffic prediction. IEEE Transactions Vehicular Technology, 2023c.
Zhang, Y., Wang, X., Yu, X., Sun, Z., Wang, K., and Wang, Y. Drawing informative gradients from sources: A one-stage transfer learning framework for cross-city spa- tiotemporal forecasting. In Proceedings of the AAAI Con- ference on Artificial Intelligence, volume 39, pp. 1147 1 1155, 2025.
Wu, H., Xu, J., Wang, J., and Long, M. Autoformer: Decom- position transformers with auto-correlation for long-term series forecasting. Advances in neural information pro- cessing systems, 34:22419 224300, 2021.
Wu, Z., Pan, S., Long, G., Jiang, J., and Zhang, C. Graph wavenet for deep spatial-temport graph modeling. arXiv preprint arXiv:1906.1.. 00121, 2019.
Zheng, C., Fan, X., Wang, C., Qi, J., Chen, C., and Chen, L. Increase: Inductive graph representation learning for spatio-temporal kriging. In Proceedings of the ACM Web Conference 2023, pp. 673,683, 2023.
Xia, Y., Liang. Y., Wen, H., Liu, X., Wang, K., Zhou, Z., and Zimmermann, R. Deciphering spatio-temporal graph forecasting: A causal lens and treatment. arXiv preprint arXiv:2309.1333 13378, 2023.
Zheng, J., Saxena, D., Cao, J., Yang, H., and Ruan, P. Inductive spatial temporal prediction under data drift with informative graph neural network. arXiv preprint 20X 2024.
Xia, Y. Liang, Y., Wen, H., Liu, X., Wang, K., Zhou, Z., and Zimmermann, R. Deciphering spatio-temporal graph forecasting: A causal lens and treatment. Advances in Neural Information Processing Systems, 36, 2024.
Zhou, Z., Huang, Q., Yang, K., Wang. K., Wang. X., Zhang, Y., Liang, Y., and Wang, Y. Maintaining the status quo:
12

---

Robust Spatio-Temporal Centralized Interaction for OOD Learning
Capturing invariant relations for ood spatiotemporal learn- ing. In Proceedings of the 29th ACM SIGKDD Con- ference Knowledge Discovery and Data Mining, pp. 3603-3614, 2023a.
Zhou, Z., Yang, K., Liang, Y., Wang, B., Chen, H., and Wang, Y. Predicting collective human mobility via coun- tering spatiotemporal heterogeneity. IEEE Transactions on Mobile Computing, 2023b.
Zhou, Z., Yang, K., Sun, W., Wang, B., Zhou, M., Zong, Y., and Wang, Y. Towards learning in grey spatiotemporal systems: A prophet to non-consecutive spatiotemporal dynamics, In Proceedings of the 2023 SIAM International Conference on Data Mining (SDM), pp. 190-198. SIAM, 2023c.
13

---

Robust Spatio-Temporal Centralized Interaction for OOD Learning
A. Related Work
A.1. Spatio-temporal Prediction
Current popular spatio-temporal prediction models are predominantly based on spatio-temporal graph neural networks (STGNNs) (Zhang et al., 2016; Wang et al., 2024b; Zhou et al., 2023c; Liu al., 2025a; Pan et al., 2019; Wang et al., 2023c; Zhou et al., 2023b; Ma et al., 2025). These models focus on developing advanced variants to accurately characterize spatio- temporal data, typically combining GCNs with sequential models to learn complex dynamics. GWNet (Wu et al., 2019), STGCN (Yu et al., 2017) leverage GNNs with TCN to model spatial and temporal dynamics, respectively. D`STGNN (Shao et al., 2022b), DGCRN (Li et al., 2023) and DCRNN (Li et al., 2017) integrate diffusion graph convolutional networks with RNN to effectively capture temporal patterns. Meanwhile, STAFformer (Liu et al., 2023a) and STNN (Yang et al., 2021) utilize Transformer to model long-term temporal dependencies. AGCRN (Bai et al., 2020) directly treats the graph topology as a trainable parameter of the model and goes through the training process to find the optimal graph topology. In addition, some spatio-temporal models choose other ways of spatial interaction, such as STNorm (Deng et al., 2021) which adjusts the data distribution of all nodes by normalization to accomplish spatial information influence among nodes, and STID (Shao et al., 2022a) captures and distinguishes spatial patterns of each node in data-driven manner by assigning learnable node embeddings. However, these kind of spatial modeling on nodes has no capability of out-o--distributiin learning since these parameters in models are coupled with the spatial scale. Furthermore, some continual learning approaches (Chen et al., 1021) sequentially fine-ture tinettne models using data subsets with new distributions to adapt to spatio-temporal changes. Unfortunately, the effectiveness of these models can only be demonstrated environments similar to the training set, leading to challenges when encountering OOD scenarios.
A.2. Continual Learning with Spatio-temporal Shift
Several studies (Chen et al., 2021; Miao et al., 2024; Wang et al., 2023a; Miao et al., 2025; Chen & Liang, 2024) have proposed continual learning strategies to tackle spatio-temporal graph prediction in scenarios with spatio-temporal shifts, When the spatio-temporal data distribution undergoes changes, these models engage in fine-tuning using subset of new data to adjust the updated data distribution. For instance, TrafficStream (Chen et al., 2021) recommends utilizing subsets of newly added nodes and significant temporal pattern data changes for fine-tuning the model. PECPM (Wang et al., 2023b) identifies conflicting nodes to enhance the fine-tuning process, focusing on nodes that have experienced substantial changes. DLF (Wang et al., 2024b) introduces a streaming training strategy to continuously fine-tune the model to adapt to the dynamic nature of spatio-temporal data. TFMoE (Lee & Park, 2024) partitions traffic flow into multiple homogeneous groups and assigns an expert model responsible for each group, enabling each expert model to specialize in learning and adapting to specific patterns. However, these models often compromise performance improve learning efficiency, resulting in lower performance compared to traditional spatio temporal models. Primarily, these models train and fine-tune sufficient amount of new distribution data (approximately 21 days in one month) and test on the new data distribution, thereby adhering to the IID assumption and encountering difficulties in OOD learning.
B. Algorithm & Optimization
We have provided the pseudocode of the algorithm in Algorithm 1, where we can observe that STOP makes final predictions based on the temporal component and spatial component. This includes perturbation process to extract robust knowledge. This perturbation process only occurs in the training phase and we no longer use it in the test phase. We also provide the optimization flow of GenPU and model parameters in Algorithm 2. As shown, we interleaved the optimization of GenPU and model parameters.
C. Theoretical Explanation and Analysis
C.1. Low-rank Attention
In the centralized messaging mechanism, We first define low-rank attention as follows:
z ( =A V)= = sofumax (QQKT softmax (aKQ V,
(21)
Diffusion
Aggregation
where Q W eR
K =
k=C( E Nxds, V = _ E R Nxdh
(i)
(1)
(22)
14

---

Robust Spatio-Temporal Centralized Interaction for OOD Learning
Algorithm 1 STOP for spatio-temporal prediction
Input: Historical data X E R'''XXx,
Output: Future prediction Y R2EXNXC.
Data encode;
Hu - X in Eq.2~ 4
D Temporal decomposition
Zi t Ho, E.E
E in Eq.5
D. Input representation
Temporal modeling and prediction;
Zr Z in 6
Temporal representation learning
Y, + Zr in Eq.
Temporal prediction component
Spatial modeling and prediction;
if test phase then
Ze - -I, C in ~~
b ConAU
end
if training phase then
Z, < ZT, C, in Eq. ~ 9, 10 11
ConAU & GenPU
Z, + Ze: Zr in Eq. 15 18
Spatial representation learning
Y, + Z, in Eq. 19
& Spatial prediction component
# Final prediction;
Ý - Y, + Y, in Eq. 20
b. Final prediction
end
Algorithm 2 Optimization process of STOP
Input: Historical data X E xx,, GenPU G {gi, .2.... .M) S IRN, sample hits (0, N), future label Y E Nxxx loss function L, initialized parameters è of STOP function f, patience P, learning rates and 3. Well-trained parameters O* of STOP.
Output:
while maximum epochs not reached or not converged do for patience = 1.,,...,P do for .., M do
g softmax (gj)
9, < sampling from multinomial distribution M (g):
G; + 9j in Eq.
Generalized Perturbation Units
C (()(X) Y;
end for
L
+- max {L1, L2..... Las)
+ θ V gL
b Update the parameters of STOP
end for
- argmax { [1, Lo...., Las}
gi - gi (11 9:) g.
log xxll: L
b Update GenPU
end while
Let S. = softmax (aKQT) T E TR
KxN be the aggregation component of the attention score, and Sa = softmax (aQK T) E N x x be the diffusion component of the attention score, hence the attention score matrix NxN can be expressed as
S = Sa x S. E NNN
(23)
And the rank of S is satisfied,
rank (S) = rank (S,, x S.) S min (rank (Sz). rank (S.)) S K N.
(24)
The final inequality is consequence of the fact that the maximum rank of matrix is no more than the minimum of the ranks of its rows and columns (Greub, 2012). The rank of S, up to K, is much lower than its size N, iee, the number of rows and columns, hence the attention score matrix of our attention mechanism is a low-rank matrix. This constitutes the basis for the low ranking observed in our low-rank attention mechanism.
15

---

Robust Spatio-Temporal Centralized Interaction for OOD Learning
The low-rank characteristic in the centralized messaging mechanism offers one key advantage: it exhibits linear complexity compared to the self-attention mechanism, allowing for larger spatio-temporal efficiency.
C.I.1. EFFICIENCY ANALYSIS
The low-rank attention fiunction in Equation 21 can be rewritten as follows,
AQ, V) SV (SxS,) V S2 x (S.
(25)
Consequently, in contrast to the unlike vanilla self-attention mechanism (Vaswani et al., 2017), which necessitates the pre-computation of the attention score matrix with complexity O (NVdd),, we have the option of computing S. V e Kxd,, initially with complexity O (K Nd2) and subsequently determining Sy x (S E RNdx, with same complexity O Nd,,), resulting in the efficient computation of low-rank attention with linear time complexity O (N) by K < N. As shown, reduce the computational complexity from quadratic nearly linear. This enables method to effectively process graph data with large number of nodes without requiring excessive GenPU memory resources. See Figure 9 and Figure 6 for experimental analysis.
D. Distributionally Robust Optimization
We theoretically analyzed STOP's generalization performance. Since STOP's optimization objective belongs to the distributionally robust optimization class (Duchi & Namkoong, 2019), which exhibits good generalization properties. Note that distributionally robust optimization class is general term for optimization objectives that satisfy specific conditions- our contribution lies in how to implement optimization strategies that meet these conditions in the spatio-temporal OOD problem. First, we will introduce what constitutes a distributionally robust optimization class and the necessary conditions for membership, then analyze its beneficial properties, and finally extend these concepts to STOP.
D.I. What is DRO?
Distributionally Robust Optimization (DRO) (Duchi & Namkoong, 2019) refers to class of loss functions that aim to optimize by considering the worst-case scenario within a certain range of all possible distributions of the data. In practical terms, an optimization object that takes the following form with respect to the training environment e can be categorized under DRO (Duchi & Namkoong, 2019; Staib & Jegelka, 2019; Levy et al., 2020).
argmin, sup (E (Bxx..))-(((xX))))))) 33E
[C (X), Y)]],
(26)
where f is the function we optimized, usually a deep neural network with Iearnable parameters. (+.)) is the distribution distance metric (Namkoong & Duchi, 2016; Shafieezadeh Abadeh al., 2018), which is used to calculate the distance between distributions. is hyperparamer to limit the extent to which the distribution is explored. In fact, this merely indicates an exploration constraint, limiting the explored environments from being completely different from the training environment.
Mark. If an optimization satisfies: (I) modeling of different environments, (2) applying constraints, and (3) emphasizing the most challenging environments, then this optimization belongs to DRO and possesses the following beneficial properties.
D.2. Robust Properties of DRO
Recall that in the preliminary, the task of spatio-temporal OOD learning aims to learn a robust function f. which can accurately predict values after Tp time steps given observed data of past T time steps X and the graph sampled from any environment e E, where e may have different spatio-temporal distributions with training environment e*,
argmin, y sup E (x, x)>p(Xx))1)) [4 (ʃ (X)
(27)
eEE
In more intuitive sense, Equation. is designed to find function that reduces the loss associated with the most challenging scenario across all possible distributions e ~ E. This task is particularly challenging because we lack access to data from any unfamiliar distributions outside of the training set (Qiao & Peng, 2023). Although traditional Empirical Risk Minimisation (Vapnik, 1998),
argmin, E (xxxY)xx(xx)1e (X)),,, Y)),
(28)
16

---

Robust Spatio-Temporal Centralized Interaction for OOD Learning
which optimises solely based the raw training environment e", performs well under the IID assumption, it is possible guarantee its performance in the presence of distributional drifts (Arjovsky al., 2019). For all possible & and function f. with high probability in mathematics, the following property holds,
E (xx.Y)=xx(xeee))leee..... (1 Y)J
(V
M No
Va (X),,,, Y)
(29)
S E(xx,))-p(xx () +
where N₂- is the number data point in traning environment. Therefore, due to the presence subsequent variance terms, optimizing ERM alone cannot guarantee performance improvement other environments e' E ع {e"). Compared to the IID-only condition of the ERM, distributionally robust optimization explores a certain range of challenging training data distributions, mathematically, distributionally robust optimization is equivalent to adding variance regularization the standard ERM (Duchi & Namkoong, 2019),
sup [ (xxx [C (X), Y)1} <Eع
(30)
= [C (X) Y) + 2p Var -xx (f(((X)) Y)] + (f),
where e (f)= 2 and is (1/NN-) uniformly about f. Therefore, we do not consider the subsequent asymptotic terms e(f), the above formula is equivalent the following inequality,
sup
(......-.t....)))eeeeer) (c (X), cE8
(31)
2 x) [C( Y)] 20 xx (f Y)].
DRO explores a certain range of training data distributions and tries to optimise on data distributions that may match the distribution of the test set, providing ideas for solving the OOD problem. Therefore, DRO mathematically provides more rigorous constraints than using empirical loss functions alone in OOD environments, preventing the model from over-relying training data. This enables the model to flexibly adapt to different environments, improving its generalization performance in unknown environments.
D.3. Does STOP have properties of DRO?
We will demonstrate that our optimization objective of STOP belongs to DRO, inheriting its good properties. Our optimization objective is as follows:
min sup (xxx)))(xx11))) (F Y:g)). s.t. / = NO. BERN
(32)
Next, we demonstrate according to Mark that our proposed optimization strategy satisfies the necessary conditions for DRO, thus inheriting its beneficial properties.
Diverse environments. STOP creates diverse training environment by adding perturbation process through a message perturbation mechanism.
Applying constraints, Our perturbation process follows polynomial distribution sampling, and we strictly control the perturbation ratio, which imposes constraints on the generated environments.
Exploring challenging environments: We emphasize selecting environments with the largest gradients during training for optimization, encouraging the model to be exposed challenging environments.
In summary, optimization strategy belongs DRO and thus inherits its good generalization property.
E. Experiments
E. Baseline Details
In experiments, we compare a lot of spatio-temporal prediction models with spatio-temporal OOD models. However, the original versions of many of these models are not compatible with the OOD setting. Consequently, we had to remove certain
17

---

Robust Spatio-Temporal Centralized Interaction for OOD Learning
non-essential code related graph structures, particularly node embedding techniques and adaptive graph structure learning techniques.
1 Node embedding technology. The researchers set node embedding vector E E I Nxd, to capture node patterns adaptively, which are coupled with the size N of the graph structure. Therefore, when the model is trained, it cannot be run directly into the test environment with ST-OOD. STID, STAEformer, and BigST use this technology.
daptive graph learning. This method generally use two noode embedding vectors Ei E NRd and E2 E RNx,, and they multiply these two node embedding matrices, A, E E RNN,, to generate an adaptive adjacency matrix A. for learning the adjacency matrix, which is then used for GCN. GWNet, DºSTGNN, and CaST adopt this method.
E.2. Experimental dataset details
In this paper. we utilized six datasets to evaluate the effectiveness of the models in OOD scenarios, primarily from the domains of transportation and atmosphere. These datasets often span multiple years. Among them, LargeST (Liu et al., 2024b) collected five years of data from 8600 records, sampled at frequency of five minutes. PEMSD3-Stream (Chen et al., 2021) is a naturally streaming traffic dataset, recording data from July each year from 2011 to 2017, where the traffic structure expands year by year, naturally representing apatio-tempore shifts, Knowair (Wang et al., 2020) collected 18 atmospheric features sampled at an hourly frequency. We followed the following rules to create spatio-tempon OOD datasets.
We used the first 60% of data from the first year as the training set, followed by 20% of data for the validation set. We used the last 20% of data from subsequent years for the test set. This longer time interval ensures changes in temporal distribution characteristics.
Temporal shift:
@ Structural shift: Apart from the PEMSD3-Strcam dataset, we selected a subset of nodes for training and validation, approximately 75% of the total number, in the test set, we randomly masked 10% of nodes to simulate node disappearance and added 30% of nodes as new nodes. This is because for spatio-temporal systems, cities or detection systems generally tend to expand. Since PEMSD3-SSream a natural streaming data set, we use it directly.
Table The details used datasets.
Training set
Test set
Graph
Structural shift
Dataset
Time range
Temporal shift
Nodes
New nodes
Removed Nodes
LTrgeST-SD
First 60% data 2017
550
Last 20% data in 2018-2021
165
LargeST-GBA
First 60% data in 2017
1809
Last 20% data in 2018-2021
542
180
LargeST-GILA
First 60% data in 2017
2949
Last 20% data in 2018-2021
884
294
LargeST-CA
First 60% data in 2017
6615
Last 20% data in 2018-2021
1984
661
KnowAir
First 60% data in 2011
141
Last 20% data in 2012-2017
42
14
PHMSDD3-Stream
First 60% data 2015
655
Last 20% data in 2016-2021
(60, 131, 167, 179, 195, 216)
E.3. OOD Performance Comparison on Large Datasets
As the largest collection of spatio-temporal data available in open source today, CA represents an invaluable test case for the OOD capability of the model. The performance of STOP and the baseline is evaluated on large-scale and large-scale spatio-temporal datasets, respectively, under identical conditions.
Based on the same partitioning strategy as described in Section 4.1, we divide the LargeST dataset into the two largest subdatasets, GLA and CA. Due to the parameter complexity of Transformer- baselines such as STAEformer, STNN, DºSTGNN, and STONE, which scales at least quadratically with the number of nodes, deploying these models on GLA and CA datasets is not feasible.
As shown in Table 6, STOP consistently outperforms the baselines on both the large-scale spatio-temporal OOD dataset in terms of overall performance and performance on newly added nodes, with improvements of up to 14.01% 1.0... On large-scale spatio-temporal datasets, the performance of baselines based on global message passing mechanisms declines significantly due to the introduction of more new nodes. STID, which does not involve node interactions, achieves the second-best
18

---

Robust Spatio-Temporal Centralized Interaction for OOD Learning
performance among the baselines. In contrast, STOP benefits from ConAU by decomposing large-scale spatio-temporal scenes into stable spatio-temporal subenvinonments, leading to the best performance while ensuring node interactions. This highlights STOP's remarkable OOD capabiliities even in large-scale scenarios.
Table OOD performance comparisons GLA and CA datasets. The absence of baselines indicates that the models incur out-of-memory issues.
Method
Ours
CaST
RPMixer
BigST
STID
STNorm
STGCN GWNet
MAE
19.13
23.36
25.89
20.32
19.87
21.05
21.17
20.51
3
RMSE
30.33
35.53
41.10
32.56
31.78
33.03
32.96
32.24
MAPE
11.93
21.44
14.90
12.93
12.03
13.34
13.87
12.81
MAE
26.29
31.43
43.33
28.83
28.30
30.70
29.91
29.13
GLA
6
RMSE
40.66
47.49
66.65
44.69
43.92
46.35
45.47
44.50
MAPE
17.60
27.75
26.18
18.49
17.72
20.57
19.90
19.36
MAE
36.87
43.48
77.32
42.12
41.38
46.13
41.81
43.92
12
RMSE
55.96
65.08
114.02
62.99
62.69
66.98
62.08
64.34
MAPE
27.07
36.46
53.23
30.33
27.90
34.63
28.21
31.14
MAE
17.47
21.87
23.72
18.77
18.35
19.10
19.01
19.23
3
RMSE
28.24
34.44
38.43
30.77
30.01
30.86
30.30
30.89
MAPE
12.69
17.79
16.02
13.60
12.92
15.38
13.62
13.68
MAE
23.70
29.13
39.52
26.80
26.06
27.63
26.64
27.30
3
6
RMSE
37.17
45.30
61.88
42.34
41.33
43.10
41.32
42.51
MAPE
18.39
23.63
27.42
19.98
19.33
23.24
19.56
20.23
MAE
32,86
41.26
70.64
39.59
38.23
40.77
37.63
40.64
12
RMSE
50.28
62.85
105.36
60.24
59.16
61.20
57.07
61.01
MAPE
27.65
34.71
53.26
32.00
29.77
35.50
30.31
31.93
Table Inductive learning preformance GLA and CA datasets of new nodes. The absence of baselines indicates that the models incur
out-of-memory issues.
Method
Ours
CaST
RPMixer
BigST
STID
STNorm
STGCN GWNet
MAE
18.99
23.09
25.65
20.17
19.71
20.92
21.35
20.36
3
RMSE
30.13
35.16
40.89
32.36
31.55
32.84
33.30
32.05
MAPE
11.94
21.32
14.86
12.91
12.03
13.26
14.01
12.81
MAE
26.17
31.15
42.95
28.66
28.14
30.56
30.46
29.00
GLA
6
RMSE
40.57
47.16
66.35
44.51
43.76
46.24
46.51
44.39
MAPE
17.64
27.62
26.09
18.47
17.73
20.47
20.25
19.38
MAE
36.78
43.12
76.60
41.84
41.13
45.87
42,97
43.70
12
RMSE
55.89
64.65
113.41
62.56
62.36
66.70
63.92
64.04
MAPE
27.17
36.35
53.12
30.24
27.87
34.35
28.98
31.14
MAE
17.48
21.86
23.73
18.76
18.35
19.10
19.38
19.23
3
RMSE
28.39
34.50
38.59
30.86
30.14.
30.98
30.87
30.96
MAPE
12.87
18.46
16.15
13.85
13.12
16.06
15.62
13.97
MAE
23.71
29.11
39.50
26.79
26.05
27.65
27.47
27.30
3
6
RMSE
37.29
45.31
62.03
42.38
41.40
43.20
42.50
42.53
MAPE
18.73
24.37
27.65
20.34
19.70
24.45
22.76
20.62
MAE
32.83
41.22
70.53
39.53
38.18
40.75
39.27
40.61
12
RMSE
50.30
62.78
105.38
60.14
59.13
61.20
59.43
60.94
MAPE
28.24
35.75
53.61
32.69
30.44
37.24
35.64
32.53
E.4. Inductive Learning Comparison on Large Datasets
To evaluate the inductive learning capabilities each model, we further report the performance of added nodes in Table We can see that GCN-based models have overall poor inductive capabilities. While they can rely on message passing
19

---

Robust Spatio-Temporal Centralized Interaction for OOD Learning
mechanisms to generalize learned information unseen nodes, the spatially confused interactions cannot guarantee accurate descriptions of added nodes, leading to subpar performance. In this regard, STID achieves better predictive results because it assumes nodes are independent, allowing the model to learn time-related knowledge that is unrelated to nodes, which can generalize to added nodes and avoid error accumulation. Our model demonstrates strong inductive learning capabilities on large-scale graphs, as added nodes can access shared context features to obtain good representations.
E.5. Performance in S-OOD and T-OOD Scenarios
With LargeST-SD dataset, we investigate the performance of models in T-OOD and S-OOD scenarios. Used two datasets are simplified versions of ST-OOD. For S-OOD, we use the last 20% of the 2017 data as the test set with the graph structure unchanged. For T-OOD, we maintain the graph structure consistent between the training and testing environments, aligning the data selection with ST-OOD. The experimental results are shown Table 8, and we can observe that STGNNs exhibit poor performance in the S-OOD scenario, mainly due to the sensitivity of the node-to-node interaction method structural shifts. The poor performance of STNN can be attributed to its use of Transformer, which lacks robustness against noise introduced by temporal and spatial shifts, Our model has achieved competitive performance in both T-OOD and S-OOD scenarios.
Table 8. Comparison in T-OOD and S-OOD scenarios.
Method
Ours
STONE D²STGNN STNN STGCN GWNet
MAE 23.21
25.00
26.56
35.06
29.74
26.79
S-OOD RMSE
36.95
39.12
42.77
55.12
44.45
41.47
MАPE
14.45
16.72
19.80
23.42
21.79
18.16
MAE 22.91
25.41
24.23
36.14
25.73
23.38
T-OOD RMSE
37.17
37.56
39.04
56.26
40.07
37.63
MAPE 15.35
16.38
17.37
26.46
17.68
16.58
E.6. Performance on Rapid Evoluting Spatio-temporal Dynamical System
In the main experiment, the proportion of added nodes is relatively small (only 30%), which may not cover rapidly developing urban scenarios. We further create a challenging scenario where we train on 30% nodes from the year 2017 and test on the remaining 70% of nodes from subsequent years. Details of the experimental dataset are provided in Table 9.
Table 9. Rapidly growth OOD setting on SD dataset.
Training set
Test set
Time range
Graph (Nodes)
Temporal shift Last 20% data in 2018-2021
Strucal shift
Firtst 60% data in 2017
214
500 new nodes & O removed nodes
We observe that for baseline models based on Transformer and GCN, such as D²STGNN and GWNet, the rapid and large influx of new nodes significantly disrupts the model's learning of message passing mechanisms, leading to a decrease in performance for models relying on such global message passing mechanisms. Models like BigST based on linear attention mechanisms and STONE based on relaxed mapping perform better than the former in out-of-distribttion (OOD) scenarios with rapid growth. On the other hand, STID, based on node independence, shows limitations in generalizing features to new nodes when faced with large number of additional nodes. In contrast, STOP benefits from its innovative ConAU and GenPU-oriented low-order attention mechanism, capturing flexible adaptations to changes in the overall spatio-temporal environment through sub-environments, showing the highest relative improvement rate at 16.35% and demonstrating robustness in scenarios with rapid node growth.
E.7. Compare Continuous Learning Method
We compared STOP with several continual learning methods on duttof-distribution (OOD) tasks. Taking the PEMSD3- Stream dataset as an illustration, when encountering spatio-temporal shifts, these models require fine-tuning using 21 2dy data from the new distribution. To ensure a fair comparison, we aligned the OOD task settings by conducting tests directly in the subsequent years following the initial year of training. This training methodology denoted as Shatic-SMMode in
20

---

Robust Spatio-Temporal Centralized Interaction for OOD Learning
Table 10. OOD performance with rapidly growth SD dataset.
Method
Ours
STONE
CaST
RPMixer
BigST
D'STGNN
STNN
STAEfommer
STID
STNorm
GWNet
STGCN
MAE
18.04
18.61
21.47
25.20
18.85
20.98
42.24
18.99
18.78
19.14
22.62
20.61
RMSE
29.17
29.82
33.75
40.13
30.36
33.46
65.12
30.29
30.17
30.84
34.70
32.56
MAPE
12.32
13.74
15.92
15.64
14.50
33.68
14.87
12.91
15.79
17.83
15.15
nodes
MAE
23.64
25.03
28.80
42.69
26.32
30.83
42.67
26.68
26.52
26,60
32.67
28.28
RMSE
38.10
39.92
44.71
66.85
41.79
47.76
65,66
41.82
41.93
42.20
49.31
44.40
MAPE
16.49
18.89
20.73
26.13
17.79
21.61
34.18
22.79
18.89
22.20
25.61
20.54
MAR
32.29
38.97
41.95
77.90
38.60
45.12
46.48
38.76
39.44
38.59
48.05
40.62
12
RMSE
51.74
55.20
63.40
116.56
60.11
68.19
71.04
59.68
60.79
60.35
71.78
63.28
MАPE
22.95
26.80
31,80
49.35
26.72.
31.83
36.88
33.16
30.24
35.07
40.56
29.35
MAE
18.28
18.84
21.53
25.24
18.97
21.26
44.82
19.16
18.92
19.37
22.97
20.86
RMSE
29.47
30.11
33.67
30.34
33.81
68.42
30.43
30.22
31.06
35.27
32.98
МАРЕ
12.50
14.07
16.20
14.98
35.29
15.21
16.38
15.54
New modee
MAE
24.02
25.39
28.94
42.76
26.55
31.34
45,26
27.00
26.79
27.02
33.27
28.71
RMSE
38.57
40.39
44.74
66.81
41.94
48.51
68.97
42.19
42.17
42.74
50.15
45.06
MAPE
16.77
19.40
21.04
26.13
18.02
22.34
35.86
23.34
19.22
23.22
26.58
21.08
MAE
32.81
39.40
42.19
77.95
38.86
45.73
48.99
39.22
39.78
39.24
48.98
41.26
12
RMSE
52.31
55.68
63.55
116.44
60.25
68.94
74.23
60.24
61.08
61.32
72.91
63.97
MAPE
23.33
27.39
32.22
49.28
26.88
32.76
38.59
33.87
30.86
37.24
41.97
30.12
TrafficStream (Chen al., 2021). SSrSTG-Staatie in PEMCP (Wang al., 2023b), and "Static-TFMoE" in TFMoE (Lee Park, 2024). We directly extracted their experimental results from the SEMSSD3-Stteam dataset. For intuitive comparison, we have added the predicted performance of STGCN.
As depicted in Table 11, the performance of continual learning strategies is notably inferior to traditional prediction models because they trade performance for accelerated training processes. And our model significantly surpasses existing continual learning models in OOD tasks.
noteworthy that in this experiment, the performance indicated by STOP is slightly superior that in the primary experiment because the results amalgamate the performance of testing data in the first year, which was omitted in the primary experiment to emphasize the disparities in data distribution between the test and training sets as much possible.
Table I.. Compared with spatio-temporal continuous learning methods on PEMSD3-Stream dataset.
15min
30min
60min
Model
MAE
RMSE
MAPE
MAE
RMSE
MAPE
MAE
RMSE
MAPE
PECMP
13.37
21.10
28.35
14.78
23.54
30.88
16.32
27.20
34.28
TrafficStream
13.98
21.88
29.36
15.12
23.98
31.67
17.46
28.01
36.44
TFMoE
12.95
21.18
18.97
14.51
23.90
19.62
18.07
29.87
24.92
STGCN 13.27
21.03
16.64
14.47
23.64
18.03
17.05
27.95
21.04
Ours
11.37
19.16
15.38
12.41
21.18
15.92
14.24
24.39
18.51
E.8. Efficiency Study KnowAir Dataset
The training time per epoch depicted in Figure showcasing the remarkable effectiveness and efficiency of STOP on the KnowAir dataset. Transformer-based models like STNN, STARformer, and DTGNNNN exhibit substantial computational time and high memory usage due to their utilization of self-attention mechanisms to calculate dependencies between node pairs, resulting in time and space complexity that scales quadratically with the number of nodes. Similarly, GCN-based models rely on GCN mechanisms for spatial feature interactions, leading to time complexity that is also quadratic with the number of nodes. In contrast, our model, with a complexity linear with the number of nodes, significantly reduces the computational complexity.

---

Robust Temporae Centralized Interaction for OOD Learning
29
STNorm
STNN
28
STGCN
Ne
000
5,000
Average MAE
27
STD
B
Memory- (MB)
STAEformer
26
STONE
CaST
DºSTGNN
25
STOP (ours)
24
10
15
20
182
183
184
Training speed (s/epoch)
Figure 6. Visual case and efficiency study of STOP on Know datasct.
E.9. Detailed Performance Analysis of OOD in Each Year
In the main experiment, we reported the average OOD performance over multiple years. To provide more detailed comparison, present the performance changes of each model in LargeST OOD datasets for each year. As shown in Table 12 to 14, the results demonstrate that in fine-grained performance analysis, our model remains highly effective.
E.10. Ablation Experiment
We conduct thorough ablation experiments evaluate the effectiveness of each component. The variants created shown in Table 15 and the experiments shown in Table 16.
For the time module, we found that time decomposition and prompting provided the model with better capabilities to capture the temporal patterns from the sequence perspective, while the introduction of Y, to make predictions from multiple components enhanced the model's robustness.
Regarding the C&S messaging mechanism, the "w/o ConAU" variant, which removes the spatial interaction module, resulted in significant increase in error, indicating that the spatial interaction is still necessary in OOD scenarios. The "w/o LA" variant, which removes the low-rank attention mechanism in the C&S spatial interaction module, performed poorly in prediction, as the traditional node-to-node messaging mechanism is less robust to ppatio-temporal shifts. The "w/o LA + DRO" variant performed better than the "w/o LA + RandomDrop" variant, demonstrating that the proposed message perturbation mechanism is more effective than directly perturbing the dataset to generate diverse training environments in helping the model extract robust representations.
The "w/o DRO" variant exhibited larger prediction error, suggesting that the inability to effectively optimize the deployed GenPU mask matrix increased the complexity of the model's learning process. The "w/o (GenPU&DRO)" variant also showed considerable increase in error, further highlighting the crucial importance of the proposed message perturbation mechanism in enhancing the model's robustness, as it allows the model to learn resilient representations from the perturbed environments.
These ablation studies can demonstrate the positive impact each designed component on enhancing the overall performance scenarios.
of the model in
22

---

Robust Spatio- Tempora Centralized Interaction for OOD Learning
Table 12. OOD performance on LargeST-2018 dataset
Method
Ours
CaST
RPMixer
BigST
STID
STNorm
GWNet
STGCN
STONE
DºSTGNN
STNN
STAEformer
MAE
17.80
22.07
26.22
19.13
18.65
19.48
19.84
18.68
18.83
18.38
35.32
19.01
RMSE
28.23
34.38
41.83
30.76
29.87
30.75
30.62
29.29
29.97
28,51
55.12
30,25
MAPE
10.76
14.68
15.28
11.47
12.25
11.96
12.50
11.56
11.24
11.47
23.38
11.58
MAE
23.40
30.03
44.55
26.41
26.01
26.76
26.97
24.53
26.38
23.98
35.77
26.21
RMSE
37.13
46.41
69.75
41.64
40.92
41.36
38.01
38.53
37.72
55.73
41.04
MAPE
14.52
25.81
16.28
16.57
17.98
15.22
15.03
23.72
MAE
32.06
43.42
80.17
38.86
38.31
38.93
37.66
34.78
38.77
32,97
41.51
38.25
12
RMSE
$1.45
66.43
120.41
60.05
59.78
60.02
58.90
53.80
51.94
51.27
64.44
59.22
MAPE
20.59
29.10
48.67
24.68
24.39
25.31
26.42
22.96
22.15
21.70
28.11
24.79
MAE
19.87
24.51
28.16
22.44
21.70
22.33
21.96
22.61
20.58
20.48
41.67
23.27
RMSE
32.71
38.63
45.26
36.86
35.55
36.07
34.62
36.12
36.49
35.06
62.49
37.74
MAPE
15.74
23.37
21.21
18:03
17.34
18.08
18.71
17.26
16.65
16.67
39.87
18.17
MAE
25.44
32.60 60
45.23
30.73
29.52
30.96
28.76
30,21
26.15
26.05
41.57
31.18
RMSE
40.59
69.42
47.93
46.23
47.50
43.56
46.13
42.86
42.49
62.41
48.28
MAPE
23.19
32.35
37.04
27.13
27.55
27.63
26.81
26.59
24.30
24.63
39.73
27.59
MAE
33.94
45.49
77.43
43.71
41.78
43.38
38.42
41.67
36.96
34,94
46.13
44.53
12
RMSE
53.45
67.92
113.45
65.75
63 68
64.22
57.12
61.36
56.02
54.96
68.65
66.98
MAPE
33,85
45.54
69.67
41.11
42.03
43.41
38.80
40.22
37.77
37.31
46.05
43,32
MAE
19.70
24.78
28.12
21.81
21.03
22.27
21.52
21.73
RMSE
31.31
37.70
44.37
34.75
33.54
34.76
33.47
33,99
MAPE
11.25
19.65
14.84
12.42
11.58
12.62
12.69
11.91
MAE
26.38
33.17
47.11
30.71
29.39
31.64
29.57
30.22
GLA
RMSE
41.01
50.28
72.19
47.55
45.86
47.81
45.05
46.24
MAPE
16.20
25.56
26.16
17.46
16.73
18.70
17.76
17.50
MAE
36.15
45.37
83.11
44.34
42.28
45.84
40.51
44.17
12
RMSE
55.61
68.36
121.95
66.37
64.77
67.03
60.64
65,32
MАPE
24.33
33.70
53,19
27.87
25.94
29.77
24.75
26.93
18.66
26.15
20.59
20.07
19.96
20.84
Out of Memory
RMSE
30.26
37.58
42.44
33.80
32.72
33.06
31.84
33.53
MAPE
13.24
18.53
17.31
14.28
13.71
14,50
14.02
14.24
MAE
24.67
31.61
43.27
28.95
28.05 00
28.61
27.00
29.04
12
RMSE
39.12
49.24
67.85
45.85
44.60
45.05
41.99
45.49
MAPE
18.98
29.84
20.89
20.56
21.59
19.74
20.89
MAE
33,54
44.03
75.81
41.51
40,52
40.85
37.02
42.04
RMSE
52.42
67.48
113.33
63.71
63.08
62.43
56.46
63.80
MAPE
27.78
36.25
57.70
32.88
31.63
32.32
30.10
32.09
23

---

Robust Spatio- Tempora Centralized Interaction for OOD Learning
Table 13. OOD performance on LargeST-22019 dataset
Method
Ours
CaST
RPMixer
BigST
STID
STNorm
GWNet
STGCN
STONE
DºSTGNN
STNN
STAEformer
MAE
18.42
22.51
26.61
19.71
19.51
19.56
21.12
19.63
19.42
19.54
37.94
19.52
RMSE
29.68
35.36
42.80
32.18
31.39
31.25
33.00
31.15
30.88
30.68
59.76
31,62
MAPE
11.73
16.02
16.03
12.27
13.18
12.65
14.57
12.99
12.76
12,91
28.17
12.58
MAE
24.16
30.58
44.89
26.85
26.88
26.32
28.93
26.05
25.86
25.76
38.28
26.58
RMSE
38.73
47.64
70.91
43.07
42.76
41.85
44.61
40.84
40.54
39.74
60.37
42.38
MAPE
15.81
26.91
17.42
17.95
20.96
17.25
17.12
28.40
MAE
32,78
43.96
80.12
39.17
39.31
38.24
40.49
37.07
36.89
35,59
43.02
38.22
RMSE
52.84
67.24
120.77
61.26
61.64
60.13
62.82
57.94
54.16.
54.24
67.88
60.11
MAPE
22.22
31.30
50.17
26.46
26.42
26.47
30.52
26.08
26.00
25.06
32.33
26.42
MAE
19.95
23.90
27.07
21.52
21.01
21.98
21.95
22.43
22.26
21.46
41.80
22.59
RMSE
32.18
37.26
42.81
34.69
34.16
34.68
33.94
35.05
34.94
35.45
62.20
35,99
MAPE
15.45
21.66
19.51
16.54
15.95
17.48
17.87
16.16
16.04
16.84
38.21
16.74
MAE
26.30
31.96
43.87
29.93
29.05
31.42
29.34
30.51
28.25
27.84
41.72
30.51
RMSE
40.91
48.34
66.45
45.96
45.13
43.72
45.62
42,30
43.83
62.10
46.78
MAPE
23.51
30.05
34,24
24.85
25.50
25.63
25.25
25.05
24.59
38.23
25.25
MAE
36.07
44.88
76.42
43.52
41.75
44,80
39.71
43.12
36.93
37.50
45.94
43.56
RMSE
54.99
67.05
110.42
65.05
63.30
64.97
58.44
62.39
56.18
57.40
67.93
65.74
MAPE
34.75
42.42
65.49
37.91
38.94
43,22
37.05
38.59
35.69
36.41
43,50
38.83
MAE
19.69
24.47
27.31
21.23
20.76
21.51
21.69
21.10
RMSE
30.93
37.04
43.19
33.92
32.95
33.70
33.58
33.12
MAPE
11.74
21.06
15.06
12.69
12.01
12.88
13.66
12.35
MAE
26.68
32.83
45.88
29.83
29.44
30.68
30.29
29.50
GLA
RMSE
41.06
70.56
46.45
45.54
46.59
45.96
45.29
MAPE
17.13
27.37
26.73
17.97
19.35
18.42
MAE
36.79
45.09
81,30
42,90
42.73
44,89
41.81
43.33
12
RMSE
56.03
67.71
119.66
64.97
64.92
66.28
62,28
64.54
MAPE
25.92
36.07
55.21
28.69
27.67
31.03
27.11
28.64
18.50
25.21
19.90
19.51
19.81
20.19
Out of Memory
6
RMSE
29.65
40.72
32.59
31.74
32.19
31.40
32.40
MAPE
13.34
18.47
16.80
14.01
13.44
15.19
14.04
13.94
MAE
24.94
30.93
41.86
28.08
27.58
28.22
27.27
28.23
12
RMSE
38.97
48.06
65.53
44.55
43.67
44.30
42.14
44.22
MAPE
19.55
29.04
20.55
20.30
22.95
20.04
20.56
MAE
34.31
43.25
73.86
40.68
40.08
40.80
37.76.
41.13
24
RMSE
52.57
66.25
110.23
62.63
62.31
62.15
57.27
62.72
MAPE
29.07
36.29
56.63
32.69
31.37
34.17
30.78
31.72
24

---

Robust Spatio-Temporal Centralized Interaction for OOD Learning
Table 14. OOD performance on LargeST-2021 dataset
Method
Ours
CaST
RPMixer
BigST
STID
STNomm
GWNet
STGCN
STONE
DºSTGNN
STNN
STABlormer
MAE
18.24
21.42
25.11
19.24
18.92
19.29
20.97
19.33
18.61
19.54
38.32
19.69
RMSE
29.23
33.16
39.71
30.70
30.10
30.74
32.61
30.60
30.01
30.73
58.72
27.31
MAPE
12.02
15.99
15.64
12.27
13.33
12.80
15.29
13.21
12.95
13.81
27.23
12.91
MAE
24.22
29.31
42.46
26.47
26.50
26.78
29.11
26.33
26.09
26.55
27.21
RMSE
38.64
44.84
65.77
41.67
41.67
42.10
44.43
41.13
40,94
41.07
59.25
42.59
MAPE
16.31
21.48
26.12
18.57
18.41
17.96
21.94
17.82
17.58
18.35
27.34
18.10
MAE
33.06
42.01
77.24
38.48
38.91
38.60
40.88
37.16
37.77
37.26
42.59
38.66
12
RMSE
52.31
62.75
114.49
59.15
60.06
59.64
62.34
$7.35
56.07
56.52
65.51
59.78
MAPE
22.98
31.50
49.02
26.74
27.21
27.48
31.56
26.67
26.02
26.77
31,06
27.48
MAE
17.44
20.37
23.25
18.68
17.64
20.12
19.76
21.12
18.23
18.32
39.34
19.63
RMSE
28.33
32.32
37.24
30.49
29.58
31,67
30.91
32.83
29.56
31.67
$7.83
31.84
MAPE
11.46
13.84
13.74
11.88
10.73
13.67
12.54
12.59
10.72
12.06
26.84
12.03
MAE
24.12
27.81
38.87
27.59
25.21
31.23
27.70
30.22
25.87
25.42
39.18
26.96
GBA
RMSE
37.13
42.60
59.14
42.14
39.93
46.19
41.54
44.89
39.06
41.14
57.64
41.73
MAPE
17.16
19.18
23,40
17.63
16.38
22.01
18.15
18.99
17.21
17.75
26.75
17.39
MAE
35,00
39.89
70.24
41.91
37.03
45.97
39.12
43.87
36.03
36.09
42.72
39.04
12
RMSE
52.48
59.20
100.88
60.59
56.12
64,95
57.59
62.31
58.97
55.86
62.52
58.17
MАPE
26.56
27.65
44.32
27.42
26.84
34.75
26.88
29.04
26.67
26.96
29.43
26.71
MAE
18.86
22.84
24.75
19.69
20.31
20.59
20.98
19.98
RMSE
30.05
34.85
39.70
31.81
32.13
32.59
32.77
31.76
MAPE
11.99
21.13
14,56
12.68
12.89
13.15
13.95
12.59
26.22
30.69
41.37
27.88
28.59
30.16
29.73
28.42
GLA
RMSE
40.56
63,92
43.35
45.63
45.22
43.60
MAPE
18.04
27.41
25,56
18.23
18.60
20.36
20.05
19.14
MAE
37.19
42.11
74.06
40.33
41.17
45.21
41.30
43.05
12
RMSE
56.17
62.84
110.23
60.29
61,56
65.44
61.33
62.86
MAPE
28.09
35.91
51.89
29.45
28.61
34.08
27.84
30.83
MAE
16.89
21.15
22.68
17.93
17.58
18.46
18.47
18.46
of Memory
RMSE
27.45
33.42
36.75
29.54
28. 99
30.00
29.52
29.84
MAPE
11.87
16.48
14.87
12.56
12.02
15.25
12.76
12.60
MAE
23.08
28.08
37,95
25.66
25.06
26.82
26.22
26.20
12
RMSE
36.07
59.24
40.57
40.67
40.85
17.21
21.83
25.41
18.40
17.88
18.39
18.58
MAE
32.27
39.63
68.35
37.99
36.71
39.46
37.29
39,01
24
RMSE
49.11
60.16
101.84
57.47
56.62
58.96
56.56
58.40
MAPE
26.04
31.94
49.28
29.39
27.37
34.64
28.40
29.37
Table 15. Variants and their definitions in ablation experiment.
Variant
Definition
w/o decom w/o prompt w/o (decom & prompt) w/o Y, w/o Y.
Remove the decoupling mechanism
Remove the temporal prompt learning
Remove the decoupling mechanism and temporal prompt learning
Remove the temporal prediction component
Remove the spatio-temporal prediction component
w/o ConAU w/o LA w/o LA + GenPU
Completely remove the spatial centralized messaging mechanism
Use naive self-attention mechanism to replace Low-rank attention Add GenPU term with the variant w/o LA
w/o LA GenPU +DRO
Add GenPU and spatio-temporal DRO with the variant w/o LA Randomly mask 20% training nodes and then train variant w/o LA
w/o LA RandomDrop
w/o DRO
Remove spatio-temporal DRO
w/o (GenPU) w/o (GenPU&DRO) + RandomDrop
Remove spatio-temporal DRO and GenPU
Remove spatio-temporal DRO and GenPU and randomly mask 20% training nodes to simulate temporal and spatial shifts
25

---

Robust Spatio-Temporal Centralized Interaction for OOD Learning
Table 16. Ablation experiments SD and KnowAir datasets.
SD
KnowAir
Variant
MAE
RMSE
MAPE
MAE
RMSE
MAPE
Ours
23.79
37.94
16.24
24.78
36.77
51.02
w/o decom
24.09
38.49
17.53
25.10
37.10
54.16
w/o prompt
24.67
39.83
18.20
25.27
36.78
51.42
w/o decom & prompt w/o Y, w/o Y. w/o ConAU w/o LA w/o LA GenPU
25.23
40.46
19.01
25.83
37.25
54.33
23.87
38.02
16.86
25.70
36.99
53.10
26.25
41.25
18.76
27.04
39.21
63.68
26.06
41.47
17.56
26.88
38.22
58.23
26.14
41.86
18.26
25.62
37.10
53.12
26.29
42.15
18.71
25.61
36.86
55.81
w/o LA GenPU DRO
26.11
41.73
17.58
25.10
36.91
54.73
w/o LA + RandomDrop
27.41
43.11
18.32
25.77
37.16
59.09
w/o DRO w/o (GenPU&DRO) w/o (GenPU&DRO) + RandomDrop
24.08
38.17
17.06
24.93
37.24
54.86
24.52
38.65
18.13
25.26
36.98
55.12
24.77
38.90
18.48
25.45
36.87
55.90
E.11. Hyperparameter Sensitivity Experiments
In addition to the hyperparameter experiment in Section4.4 of the main body, we additionally deployed conduct experiments on four datasets- SD, GBA, GLA, and CA-to analyze the sensitivity of two hyperparameters, the number of ConAU K and the number of GenPU M. The numebr nodes for training in these six datasets range from 141 to 6615 nodes. The results on six datasets are shown in Figure and Figure 8.
The number of ConAU K. ConAU the coarsening unit up interact with the node. Thus, the number of ConAU K' is closely related to the spatial scale. Based observations, we find that setting K to approximately 1% of the spatial scale is good choice. A larger number of ConAU can hinder the model's ability to focus capturing generalizable contextual features.
The number of GenPU M. The hyperparameter M represents the number of GenPU, which are used to modulate the interaction process between nodes and ConAU. Each GenPU corresponds to a different training environment. observed that the number of GenPU M is universally effective when set between 2 and When M is set smaller value, overly complex training environment limited training environments may not provide sufficient diversity for the model to extract invariant knowledge. Interestingly, this hyperparameter is insensitive spatial scale.
have
disrupt learning stability. Conversely, if there are too few GenPU, the
We further analyze the sensitivity of this hyperparameter to the temporal span of the dataset. Long-range SD, GBA, GLA, and CA datasets contain full year of training data, and TrafficStream is a short-range dataset containing one month data for training. And we can see that M not highly correlated with the time span of the data.
Summary. Based on the above analysis, we recommend setting the initial values K to 1% of the number of training nodes and the initial values of M between 2 and 4 for hyperparameter tuning in out-of-distributin (OOD) scenarios.
E. Effectiveness Evaluation of Collaborative Structure
STOP's predictions arise from the superposition of spatial and temporal components, which helps improve the model's robustness across various OOD scenarios. Using the SD dataset example, we employ separate scenarios: S-OOD and T-OOD. We use Shapley Values to analyze the contribution levels of Y, and Y, components in each scenario. The normalized results are shown in the following table: We observe that when structural shifts occur (S-OOD), the contribution of the temporal prediction component increases, due to the decreased accuracy of spatial correlation features. The opposite occurs in the T-OOD scenario. Through this collaborative architecture, our STOP can flexibly adapt to various OOD scenarios.
26

---

Robust Tpatoo-Temporal Centralized Interaction for OOD Learning
SD
GBA
GLA
17.2
22.0
44.5-
RMSE
MAE & RMSS
17.0
MAE RMSE
21.5
MAE RMSE
16.8
MAPE(SS
MAPE(%))
MAPEC
16.6
36,4
16.2
20.0
23.7
16.
240
19.5
26.3-
32
32
CA
PEMS3-Strcan
KnowAir
40.6
RRSS
MAPE(%)
2133
RMSE MAPE
18.7
MAR
RMSE
MAPE(S)
MAB & RMSE
MAE RMSE
MAE & RMSE
MAPE(%)
MAPE(%)
MAPE()))
27.2
348-
Figure 7, Sensitivity experiments of ConAU,
SD
GBA
GLA
MAR
IRMSE
MAPE(%)
MAE
RMSE
MAPE())
MAE
RMSE
MAPE(55)
MAE & RMMSE
38.4-
MAE & RMSE
406
21.5
MAE RMSE
42.9-
AAPEE
MA
MAPE(%) MAPE(%
38.0
16.6
21.0
41.2-
24.7-
27
20
29.6-
14.2
2188
28.0
16.2
242
26.5
2
4
CA
PEMS PEMS-Streaar Stream
KnowAir
RRMSS
MAPE(%)
MAPE(%)
RMSE
MAPE(%)
MAE & RMSSE
39.1
MAPE(%)
MAE & RMSE
2238
180
M)
MAE & RMSE
37.3-
17.6
21.6
17
16/7
24.9
13.34
25.2-
23.8
123
24.7
5
2
4
Figure Sensitivity experiments of GenPU.
E.13. Embedding visualization
Using LargeST-SD dataset as example, we visualize the temporal prompt embedding E in Figure 9 (a), Personalized features Zp. and contextual features Z. in Figure (b). We can see that temporal embeddings unveil essential periodic patterns for OOD scenarios. Both node personalized and context features exhibit strong discriminative capabilities. Context features capture shared node patterns, ensuring resilience to individual node variations. Meanwhile, personalized features enhance the model's ability tailor predictions for each node effectively.
F. Discussion
In this section, we discuss the limitations our work as our future work:
Validating the Broad Impact STOP. The spatial interaction module integrated within the STOP framework inherently generic, suggesting its potential for broader applicability. In upcoming research, we will propose replacing the graph convolutional networks utilized by other spatio-temporal backbones with the spatial interaction module validate its effectiveness across various contexts. This initiative will help us better understand the potential value and applicability of the STOP module in wide range of application domains.
Exploring a Wider Range of OOD Scenarios. Current OOD problems are typically defined within the confines of
27

---

Robust Spatio-Temporal Centralized Interaction for OOD Learning
Table 17. Contribution of prediction components in different scenarios
SD
KnowAir
Commpent
S-OOD
T-OOD
S-OOD
T-ODI
Y 64.04
41.95
72.62
36.23
Y
35.96
58.05
27.38
63.77
8
Mon
Fri
Tue
Sat
UOISIN 600 60002200000
Wed
Sun
Thu
5
Personalized feature
10
Contextual feature
Boundary
-10
10
15
-10
10
(a) Temporal prompt pool
(b) Personal feature and context feature
Figure Visual case efficiency study STOP on LargeST-SD dataset.
data exhibits diverse modalities and varied tasks. We believe that an improved spatio-temporal OOD handler should be capable of addressing challenges such cross-task and cross-modal processing, areas that have not been thoroughly explored in the spatio-temporal domain.
single-modes data and single tasks. However,
Integrating Large Language Models for zero-shot learning. In OOD settings, accurately predicting new nodes poses a significant challenge, as these nodes have not been encountered by the model during training sertinn-commnllyy referred to as the zero-shot challenge. Large language models excel in this context, as their representational capabilities, developed from extensive training on massive datasets, can enhance model's zero-shot learning ability (Liu al., 2025agc;b), While this has been successfully demonstrated in the time series community, it remains relatively unexplored within the apatio-tempore domain. In future work, plan to integrate large language models into the STOP framework to further enhance its scalability for predicting new nodes.
28


Reviews:
[{"review_id": "trHqiPLQQ0", "reviewer": "Reviewer_tJb2", "summary": "The paper introduces a spatiotemporal interaction mechanism called STOP. STOP includes centralized message passing mechanisms with message perturbation and DRO to enhance stability for spatiotemporal sifts. Through extensive experiments, STOP's robustness to spatiotemporal shifts is effectively demonstrated.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your appreciation and comments on the paper. Your comments are crucial to improving the quality of the manuscript.\n\n> **W1 & Q2. Efficiency Analysis** \n\nWe sincerely apologize for any misunderstanding caused. Below, we report the efficiency comparison between the proposed method and several advanced models on the CA dataset. The efficiency metrics include memory usage, training time per epoch, total training time, and the amount of memory used. We can find that our model achieves competitive performance while maintaining high efficiency\n\n| CA     | MAE | Train (s/epoch) | Inference time (s)   | Total (h) | Memory (MB)  |\n|:--------:|:--------:|:-----------------:|:-------:|:-----------:|:---------:|\n| STGCN  | 40.64  | 951.2| 219.54 | 23.80| 26,395  |\n| GWNet  | 37.63  | 2279.6| 332.92 | 69.66| 34,103  |\n| STNorm | 40.77  | 360.8| 69.40  | 11.13| 29,275  |\n| BigST  | 39.59  | 328.4| 82.91  | 10.78| 16,283  |\n| CaST   | 41.26  | 577.3| 153.72 | 17.09| 19,515  |\n| Ours   | 32.86 | 290.2| 70.58 | 7.13| 15,243  |\n\n> **W2. OOD Setting**\n\nThis issue is addressed in Appendix E.1. For models such as GWNet and AGCRN, they employ an adaptive graph learning enhancement technique, which generates meaningful node representations for each node but also causes the parameter scale to become coupled with the size of the graph structure. To address this, we have removed this technique.\n\n> **Q1. Spatio-temporal prompt**\n\nThis technique uses embeddings to encode the day-of-week and timestamp-of-day information. These prior knowledge capture the periodic patterns in spatiotemporal data, which are relatively stable and help improve the model's generalization ability against spatiotemporal shifts.\n\n> **Q3. Traditional GCN**\n\nWhether GCN is necessary in the field of spatiotemporal prediction has long been a topic of debate. However, many studies in the spatiotemporal domain have consistently shown that GCN-based models outperform those that only consider temporal dependencies, as they introduce an inductive bias between nodes. In spatiotemporal OOD tasks, the dense message-passing mechanism of GCNs demonstrates sensitivity to spatiotemporal shifts. Therefore, we focus on enhancing the robustness of spatiotemporal interactions and propose a centralized interaction mechanism as an alternative solution.\n\n> **Q4. Message Interference Mechanism**\n\nIn lines 188 to 197, we describe the random sampling process for the stochastic masking matrix. Specifically,  we first create $M$ learnable probability vectors and normalize it, with the result denoted as {$ p_1^\\prime, p_2^\\prime,\\dots, p_M^\\prime$}, where ${p}_i^\\prime \\in \\mathbb{R}^N$ with $i \\in$ {$1, 2, \\cdots, M$} means $i$-th probability vector. Then, for $i$-th probability vector, we can establish a binomial distribution, which is denoted as $\\mathcal{M}\\left({p}_i^\\prime;s\\right)$. Using this distribution, we can sample a masking indices $\\widetilde{g}_i \\sim \\mathcal{M} \\left({p}_i^\\prime;s \\right) \\in$ {0,1}$^N$, where $s\\in\\left(0,N\\right)$ indicates the number of sample hits (i.e. the number of values equal to 1 in $\\widetilde{\\boldsymbol{g}}_i$). Finally, we create $K$ replicas of $\\widetilde{\\boldsymbol{g}}_i$. As a result, we can obtain a mask matrix with log operation, and the output is denoted as $\\mathbf{G}_i=\\log\\left(\\left[\\widetilde{\\boldsymbol{g}}_i,\\widetilde{\\boldsymbol{g}}_i,\\dots,\\widetilde{\\boldsymbol{g}}_i\\right]\\right)\\in\\lbrace-\\infty,0\\rbrace^{K\\times N}$ and $\\mathbf{G}_i$ is used to interfere with the message process. Similarly, we ultimately perform multiple rounds of perturbations on the message-passing mechanism using $M$ interference matrices. To avoid further confusion, we will refine this section to improve clarity and ensure a more precise presentation.", "claims_and_evidence": "The author identified the limitations of existing methods through experimental observations and used this as motivation for the paper.", "methods_and_evaluation_criteria": "The three strategies proposed by the authors to enhance robustness to spatiotemporal shifts are reasonable. The authors have also discussed the roles of each strategy in the method section.", "theoretical_claims": "The formulas in the paper are correct and align with the descriptions provided. The use of DRO theory in the paper can explain its advancements.", "experimental_designs_or_analyses": "The authors compared the performance of STOP in various OOD scenarios, including rich baselines and datasets, encompassing multiple common metrics in spatiotemporal learning. They also conducted ablation studies, hyperparameter experiments, and other supplementary experiments to demonstrate the model's effectiveness.", "relation_to_broader_scientific_literature": "The centralized interaction mechanism proposed by the author can broaden the architectural perspectives of existing models in the field.", "essential_references_not_discussed": "Related work has been compared or discussed.", "other_strengths_and_weaknesses": "Strengths of the paper：\n\n-\tThe centralized node interaction mechanism introduced in this paper is innovative and shows promise as a robust learner for spatial features.\n\n-\tThe comprehensive OOD benchmark evaluation framework utilized in the experiments is praiseworthy, as it covers multiple datasets, rich baselines, and comparisons across different OOD scenarios.\n\n-\tThe paper's assertion that the traditional node-to-node messaging mechanism is fragile is a new finding.\n\nWeaknesses of the paper：\n-\tEfficiency comparisons are available on two datasets in paper, and there is a lack of presentation of efficiency comparisons for large-scale dataset.\n\n-\tMore discussion on baseline experiment details and . As far as I know, some models have parameters coupled with graph size, such as GWNet and AGCRN. How authors handle adaptation to OOD settings?", "other_comments_or_suggestions": "For efficiency comparison experiments, it is recommended to present the data more intuitively using tables.", "questions_for_authors": "-\tWhat is the role of the spatio-temporal prompt in STOP for spatiotemporal shift?\n-\tHow efficient is STOP on the CA dataset?\n-\tIf traditional GCN is limited in OOD scenarios, then there is a natural question: Is GCN really necessary?\n-\tHow is the masking matrix of the message interference mechanism sampled from the distribution?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "R2wp3UQqeK", "reviewer": "Reviewer_riin", "summary": "The spatiotemporal messaging mechanism utilized in STGCN exhibits inherent sensitivity to spatiotemporal variations. To address these limitations, the authors introduce a centralized messaging architecture integrated with a message perturbation mechanism and DRO optimization. Extensive experimental evaluations were conducted across six benchmark datasets, encompassing comparative analyses with 14 baseline methods. The results indicate that the proposed model achieves performance improvements of up to 17.01% compared to existing approaches.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We deeply appreciate your valuable time and effort, which are crucial for improving the quality of our manuscript.\n\n> **W1. General Graph OOD Learning Discussion** \n\nIn the field of graph representation learning, researchers have focused on modifying model architectures to enhance the representation of invariant knowledge, thereby improving generalization performance for OOD problems [1-2]. For instance, GAUG [3] improves downstream training and inference by incorporating an edge prediction module to modify the input graph. DisenGCN [4] emphasizes learning disentangled representations by separating distinct and informative factors from graph data and assigning them to different parts of the decomposed vector representations. OOD-GNN [5] introduces a nonlinear graph representation decorrelation method, leveraging random Fourier features to eliminate statistical dependencies between causal and non-causal graph representations generated by the graph encoder. However, these models fall short in capturing the complex heterogeneous spatiotemporal correlations inherent in spatiotemporal data, resulting in suboptimal performance for spatiotemporal OOD tasks. In this paper, we propose a novel spatiotemporal interaction module to enhance the robustness of models against spatiotemporal shifts. We will add this discussion to the next version.\n\n> **W2. More challenging conditions**\n\nIn Section E.6 of the appendix, we discuss performance comparisons in scenarios where the graph topology grows rapidly. To further evaluate robustness under extreme conditions, we use the SD dataset as an example, where the size of the test graph topology is nine times larger than that of the training graph. The performance results are shown in the figure below.\n||  | ||\n|------------|-------|------------|-------|\n|| MAE| RMSE |MAPE|\n| STGCN| 36.45 | 55.89| 28.01 |\n| GWNet| 37.22 | 56.62| 33.12 |\n| STNorm| 29.79 | 47.71| 24.51 |\n| BigST| 29.72 | 48.55| 21.09 |\n| D $^2$ STGNN | 37.46 |56.37| 29.63 |\n| STAEformer| 28.94 |45.84| 29.40 |\n| CaST| 30.20|47.09| 22.66 |\n| STONE| 36.19|56.71| 30.68 |\n| Ours| **26.73** |**42.35**|**19.38**|\n\n> **W3. Comprehensive Efficiency Analysis**\n\nWe sincerely apologize for any lack of clarity in our explanation. We report comprehensive efficiency metrics for several SOTA models on the SD dataset, including simultaneous training time, total training time, inference time, memory usage under optimal performance conditions, and average MAE performance. We find that our proposed method achieves competitive efficiency compared to the SOTA model D $^2$ STGNN. This is attributed to our model's reliance on a lightweight MLP architecture.\n\n| | Average MAE | Train (s/epoch) | Inference (s) | Total (h) | Memory (MB) |\n|:-:|:-:|:-:|:-:|:-:|:-:|\n| STGCN| 25.79| 82.6|7.4| 2.50| 3,783|\n| GWNet| 28.21| 125.3| 14.3| 3.53| 6,871|\n| STNorm | 26.51| 78.2| 7.0 | 2.36| 2,755|\n| BigST| 26.26| 68.3| 7.3| 2.10| 2,791|\n| CaST| 29.84| 82.1| 6.2| 1.52| 3,255|\n| STAEformer | 26.20| 443.7| 36.8 | 12.96| 32,667|\n| D $^2$ STGNN | 25.43| 1216.4| 58.9| 33.65| 50,171|\n| STONE | 25.50 | 199.8 | 16.6 | 5.21| 12,349|\n| Ours | 23.67  | 59.2 | 6.8| 1.54| 3,652|\n\n> **Presentation Problem**\n\nWe sincerely appreciate your valuable suggestions. In the new version, we will optimize the structure of the paper and correct typographical errors to further improve the overall quality and readability of the manuscript.\n\n> **Q1. MLP- vs Transformer- based Models**\n\nSTID outperforms D $^2$ STGNN in some experimental results, potentially because STID leverages various embedding techniques to capture prior information, which is beneficial for enhancing the model's generalization capability. On the other hand, D2STGNN has a more complex parameter structure, making it prone to overfitting in the training environment. This overfitting can lead to a decline in its ability to generalize to unseen scenarios.\n\n> **Q2. Temporal shifts**\n\nWe tackle temporal shifts using two key techniques: (1) Temporal Embedding Technology : By encoding day-of-week and timestamp-of-day information, this approach captures multi-level periodic patterns in spatiotemporal data, strengthening the model's ability to represent stable temporal structures. (2) Decoupling Mechanism : We employ time series decomposition to model seasonal and trend components separately. The stable characteristics of periodic and seasonal patterns enhance the model's robustness against temporal shifts. \n\nReference:\n\n[1] Park, Hyeonjin, et al. \"Metropolis-hastings data augmentation for graph neural networks.\" NeurIPS 2021.\n\n[2] Wu, Ying-Xin, et al. \"Discovering invariant rationales for graph neural networks.\" ICLR 2022.\n\n[3] Zhao, Tong, et al. \"Data augmentation for graph neural networks.\" AAAI 2021.\n\n[4] Ma, Jianxin, et al. \"Disentangled graph convolutional networks.\" ICML 2019.\n\n[5] Li, Haoyang, et al. \"Ood-gnn: Out-of-distribution generalized graph neural network.\" TKDE 2022.", "claims_and_evidence": "Yes, the authors focus on the sensitivity of the messaging mechanisms in existing STGNNs to spatio-temporal shifts. They substantiate this by conducting comparative experiments and ablation study involving multiple SOTA STGNNs across diverse OOD scenarios.", "methods_and_evaluation_criteria": "Yes, in response to the sensitivity of existing spatiotemporal node-to-node messaging mechanism of STNNs, this study proposes a novel spatiotemporal interaction mechanism that integrates message perturbation and Distributionally Robust Optimization strategies. These strategies are tailored to address specific challenges, and the design motivation is both rational and logically consistent.", "theoretical_claims": "Yes, the application of Distributionally Robust Optimization theory in this study theoretically demonstrates that the proposed model exhibits superior generalization capabilities compared to conventional models.", "experimental_designs_or_analyses": "The experimental section demonstrates rigorous methodology and thorough validation procedures. The study presents a systematic performance evaluation comparing 14 baseline models across six distinct datasets. A comprehensive set of ablation studies has been conducted to quantitatively assess the contribution of individual components. Furthermore, the authors provide detailed analyses including component-wise ablation studies, computational efficiency assessments, and extensive robustness evaluations.\n\nTo further strengthen the study, it would be valuable to consider additional experimental investigations, particularly: (1) comparative analyses under more challenging operational scenarios, and (2) detailed assessments of memory efficiency and resource utilization. These supplementary evaluations could provide deeper insights into the model's practical applicability and scalability.", "relation_to_broader_scientific_literature": "While previous research has empirically demonstrated the sensitivity of spatiotemporal prediction models in OOD scenarios, the authors have made a novel contribution by identifying and analyzing the fundamental source of this vulnerability stemming from the spatiotemporal messaging mechanism they adhere to.", "essential_references_not_discussed": "I believe that much of the work related to the focus of the authors has already been discussed in this paper, primarily in the appendix. I suggest further consolidating the existing progress in the main body of the text. Additionally, I still encourage the authors to discuss general OOD learning research.", "other_strengths_and_weaknesses": "Strengths:\n\nThe authors have identified a critical limitation in existing spatiotemporal messaging mechanisms, specifically their sensitivity to out-of-distribution (OOD) scenarios, which constitutes a significant and well-justified research motivation.\n\nThe technical contributions present a novel centralized interaction framework that incorporates message perturbation mechanisms and distributionally robust optimization objectives. The provided theoretical analysis substantiates the proposed model's advantages and robustness properties.\n\nThe experimental evaluation demonstrates comprehensive methodology, including extensive comparative analyses across multiple datasets, systematic ablation studies, and detailed efficiency assessments. These empirical results provide substantial evidence supporting the model's performance improvements.\n\nWeaknesses and Recommendations:\n\nThe paper would benefit from a broader discussion on general graph OOD learning algorithms, which could provide valuable context and highlight the proposed method's position within the broader research landscape.\n\nThe current experimental setup limits dynamic nodes to 30% of the total nodes. To further validate the model's robustness, it is recommended to conduct additional evaluations under more challenging conditions, such as scenarios where 70% of nodes exhibit dynamic behavior.\n\nWhile computational efficiency has been partially addressed, a more comprehensive evaluation should include memory usage metrics and total training time. These additional metrics would provide a more complete assessment of the model's practical applicability and scalability.", "other_comments_or_suggestions": "1.\t' within Appendix A' in 639 line seems redundant.\n2.\tSome abbreviations of proper nouns appear repeatedly.\n3.\tIn Table 20, D2STGNN should be written D$^2$STGNN.", "questions_for_authors": "Please refer to weaknesses 1-3\nQ.1 In Table 3, why does MLP-based STID achieve better prediction performance than Transformer-based D$^2$STGNN?\nQ.2 Does STOP include a unique design to address temporal shifts?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "W81caAHVYz", "reviewer": "Reviewer_objW", "summary": "The authors introduce a novel spatio-temporal interaction mechanism called STOP, which is tailored to enhance the sensitivity of the conventional node-to-node messaging mechanism favored by existing models to spatiotemporal shift. Key elements of STOP include the centralized message mechanism, the message perturbation mechanism, and an optimization objective rooted in distributed robust optimization. Through a series of comprehensive experiments, the results effectively showcase the efficacy and competitiveness of STOP.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you sincerely for your appreciation and thoughtful comments regarding the paper. Your feedback is invaluable in enhancing the quality of the manuscript.\n\n> **W1. Visualization Cases**\n\nDue to the rebuttal policy restrictions of ICML, we were only able to include some prediction visualization cases in an anonymous link (https://anonymous.4open.science/r/_reb/objW.pdf). Meanwhile, in the submitted manuscript, Figure 9 visualizes the time embeddings, personalized embeddings, and contextual embeddings used in our model, providing evidence for the effectiveness of the proposed embedding techniques. \n\n> **W2. Zero-shot Performance**\n\nIn fact, the zero-shot performance includes the results for newly added nodes, as shown in Table 4 of our submitted manuscript. These nodes had no observed data during training and were excluded from the training process. Despite this, our model demonstrates impressive zero-shot performance. This is largely due to its ability to extract shared contextual features from spatiotemporal data, which can be leveraged by newly added nodes to achieve effective representations. For convenience, we have included the zero-shot performance results for SD below.\n\n|||||\n|:-:|:-:|:-:|:-:|\n||MAE|RMSE|MAPE|\n|GWNet|29.01|44.18|22.82|\n|STID|25.40|39.53|18.66|\n|BigST|25.22|39.22|18.02|\n|D $^2$ STGNN|25.85|39.33|20.07|\n|STAEformer|25.28|39.42|18.41|\n|CaST|28.83|44.10|22.93|\n|STONE|25.06|39.15|18.12|\n|Ours|22.74|36.09|16.82|\n\n> **W3. Few-shot Performance**\n\nTo address your question, we use the SD dataset as an example. Following the experiment setup in the paper, when the testing environment changes, we fine-tune the model using the first week of the test set to evaluate its few-shot performance. The results are shown in the table below. We find that with minimal fine-tuning data, our model quickly adapts to new spatiotemporal patterns, demonstrating strong generalization. This is largely due to the lightweight architecture and effective use of contextual features.\n\n|||||\n|:-:|:-:|:-:|:-:|\n| |MAE|RMSE|MAPE|\n|STGCN|25.38| 38.77|20.61|\n|GWNet|23.15 | 35.53|18.11|\n|STID|25.47 |39.65|18.87|\n|BigST|25.55| 39.61|18.81|\n|D $^2$ STGNN| 22.09|34.82|16.87|\n|STAEformer| 26.22|40.34|19.15|\n|CaST| 28.40| 43.66|20.70 |\n|STONE| 23.08| 34.88| 17.55|\n|Ours| 20.34| 32.64| 14.62|\n\n> **W4. Related Work**\n\nIn Section A.3, we summarize recent advances in continual learning. While these methods address dynamic spatiotemporal data, they typically require extensive new data for fine-tuning, implicitly assuming independent and identically distributed (i.i.d.) data.\n\nIn the updated paper, we discuss progress in out-of-distribution (OOD) learning. Traditional machine learning assumes training and testing data share the same distribution, but real-world applications often face distribution shifts, leading to significant performance drops post-deployment [1]. This has driven growing interest in OOD learning, with methods categorized into unsupervised representation learning, supervised model learning, and optimization-based approaches [2-3]. These techniques leverage causal and invariant learning to extract generalizable knowledge from latent test distributions.\n\nRecently, OOD learning in graph learning has gained attention. For example, DisenGCN [4] disentangles informative factors in graph data, assigning them to distinct parts of vector representations. However, these methods struggle with spatiotemporal OOD problems due to their inability to capture complex, heterogeneous spatiotemporal correlations.\n\nOur key contribution is a novel spatiotemporal interaction mechanism that integrates centralized message-passing, message perturbation, and a distributionally robust optimization (DRO) objective, addressing these limitations effectively.\n\nReference:\n\n[1] Yang, Jingkang, et al. \"Generalized out-of-distribution detection: A survey.\" ICCV 2024. \n\n[2] Liu, Jiashuo, et al. \"Towards out-of-distribution generalization: A survey.\" arXiv preprint arXiv:2108.13624 (2021).\n\n[3] Kaddour, Jean, et al. \"Causal machine learning: A survey and open problems.\" arXiv preprint arXiv:2206.15475 (2022).\n\n[4] Ma, Jianxin, et al. \"Disentangled graph convolutional networks.\" ICML 2019.\n\n> **Q1. Potential Improvement**\n\nIn Section F, we discuss potential improvements to the model, including enhancing its performance by integrating large language models and implementing more advanced perturbation mechanisms, among others.\n\n> **Q2. GCN**\n\nWhether GCNs are essential for spatiotemporal forecasting remains debated. However, research consistently shows that GCN-based models outperform those relying solely on temporal dependencies, owing to their ability to introduce inductive biases among nodes. In general OOD scenarios, the dense message-passing mechanism of GCNs exhibits sensitivity to distributional shifts. To address this, we propose a centralized interaction mechanism as an alternative, enhancing the robustness of spatiotemporal interactions.", "claims_and_evidence": "The authors have elucidated a strong motivation: existing spatio-temporal interaction mechanisms serve as on trigger for the sensitivity of current models to spatio-temporal changes. They have substantiated this motivation through a series of experiments, accompanied by relevant citations.", "methods_and_evaluation_criteria": "The proposed framework integrates three synergistic technical innovations—centralized messaging,  message perturbation, and distributionally robust optimization—which collectively provide a comprehensive solution to the spatiotemporal shift challenge.", "theoretical_claims": "This paper leverages the Distributed Robust Optimization theory to underpin the sophistication of the model. This framework is well-established.", "experimental_designs_or_analyses": "I reviewed the experiments in the paper.  It includes comparison experiments, double/multiple ablation studies, hyperparameter experiments, efficiency experiments, etc., thoroughly evaluated the proposed method.\n\n(1). Authors are expected to add predictive visualization cases to intuitively assess the model's predictive performance.\n\n(2). Add more discussion on the zero-shot performance of the model.\n\n(3). If we allow fine-tuning using a few data with new features, the authors are requested to compare the model's few-shot learning capabilities.", "relation_to_broader_scientific_literature": "The spatiotemporal OOD task that the authors focus on represents a new and emerging research direction in the field of spatiotemporal prediction. Their critical insights into existing message-passing mechanisms are fresh perspectives in this field.", "essential_references_not_discussed": "The efforts made by the authors to cover a comprehensive range of related work are commendable. However, to strengthen the literature review, I suggest two improvements: (1) elaborate on the advancements and novelty of the proposed method, and (2) discuss advancements in OOD research in other domains [1- 2].\n\nRef:\n[1] Yang J, Zhou K, Li Y, et al. Generalized out-of-distribution detection: A survey[J]. International Journal of Computer Vision, 2024, 132(12): 5635-5662.\n[2] Liu J, Shen Z, He Y, et al. Towards out-of-distribution generalization: A survey[J]. arXiv preprint arXiv:2108.13624, 2021.", "other_strengths_and_weaknesses": "Overall, the paper demonstrates high quality: it presents a convincing motivation, introduces an innovative framework, maintains a well-structured presentation, and conducts very comprehensive experimental evaluations.\n\nHowever, I have several concerns:\n\n1.\tExpand the discussion on continual learning methods and general OOD techniques.\n\n2.\tInclude an evaluation of the model's effectiveness in few-shotsettings.\n\n3.\tProvide visualized prediction cases to illustrate the model's effectiveness.", "other_comments_or_suggestions": "None.", "questions_for_authors": "What potential improvements can be made to the STOP framework? \n\nIs the use of GCN essential for spatiotemporal prediction?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IBzAEPQik8", "reviewer": "Reviewer_5Pwy", "summary": "This authors introduce a Spatio-Temporal OOD Processor, a framework for out-of-distribution learning in spatiotemporal graph convolutional networks. As stated, the traditional methods rely on node to node message interactions, which degrade performance in OOD settings. The proposed method addresses such limitations by using a centralized messaging mechanism with Context-Aware Units and Generalized Perturbation Units -- replacing direct node interactions. A spatiotemporal optimization strategy is used to expose models to diverse training environments and this argued to improve generalization. They evaluate on six datasets.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your review. We provide figures and tables at an anonymous link ( https://anonymous.4open.science/r/reb_/5Pwy.pdf ). **The \"#\" before a title means that complete tables in the section are available in the anonymous link.**\n\n> \\# Meta-learning\n\nWe introduce meta-learning based spatiotemporal models, and results on SD dataset show that our model outperforms these models. \n\n|Model|Ours|MegaCRN|MGT|\n|:-:|:-:|:-:|:-:|\n|MAE|**23.67**|25.47|26.72|\n\n> \\# IID\n\nUsing SD dataset as an example, the table below reports the average MAE. This demonstrates that our model still achieves competitive performance under the IID setting.\n\n|Model|Our|STONE|CaST|D $^2$ STGNN|STGCN|\n|:-:|:-:|:-:|:-:|:-:|:-:|\n|MAE|**23.59**|24.98|30.17|24.46|25.44|\n\n> Computational Complexity\n\nAs demonstrated in Theorem 1, our centralized messaging has higher efficiency. While the computational complexity of node-to-node interaction is O( $N^2$ d), our method reduces it to O(PNd), where P is much smaller than N. For example, P can be as small as 8 for a graph with 716 nodes.\n\n> Large Spatiotemporal Dataset\n\nAs noted in line 309, in Section E.3, we evaluate the performance on large datasets. Below, we summarize key results from the CA dataset for easy reference. These results confirm the strong effectiveness of our model for large-scale data. \n\n|Model|Ours|CaST|BigST|GWNet|STGCN|\n|:-:|:-:|:-:|:-:|:-:|:-:|\n|MAE|**32.86**|41.26|39.59|37.63|40.64|\n\n> \\# Non-spatiotemporal Domain\n\nWe regret that, due to significant conceptual and methodological differences between the spatiotemporal domain we focus on and the reinforcement learning and sequential decision-making areas of interest to you, we cannot explore these unrelated areas within the limited time available.\n\nTo address your concerns, we extended our evaluation to discrete graph OOD learning. Following the setup in [1], the table below on Collab-0.8 dataset shows our method performs well in non-spatiotemporal domains. \"w/o DRO\" and \"w/o Per\" indicate the removal of the DRO and message perturbation mechanisms, respectively, underscoring their positive effects in non-spatiotemporal setting.\n\n|Method|Ours|w/o DRO|w/o Per|DIDA|DySAT|VREx|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|AUC|**77.66**|75.62|73.45|72.59|62.19|62.21|\n\n[1] Zhang Z, Wang X, et al. Out-of-distribution generalized dynamic graph neural network with disentangled intervention and invariance promotion. arXiv preprint.\n\n> Different Types of Shifts\n\nIn the third paragraph of the introduction and in Figure 1, we analyze the effects of different shifts. Temporal shifts lead to inaccurate node descriptions, with representation errors propagating through node-to-node messaging and impacting other nodes. spatial shifts disrupt established message passing paths, preventing the model from following trained pathways and degrading its graph representation performance. **Appendix section E.5 evaluates our method’s ability to address these shifts**. To address your concern, we will refine the third paragraph to better emphasize the distinct impacts of each shift.\n\n> Related Work\n\nSorry for any confusion. Due to space constraints, **we discuss related works to spatiotemporal OOD learning in detail in Appendix Section A**. In the new version, ICML permits an additional page, which will allow us to incorporate this discussion.\n\n> \\# Long-term temporal dependencies\n\nFollowing the setup in long-term time series analysis, we use the Knowair dataset as an example, employing 96 time steps to predict 720 time steps. The averaged results below demonstrate that our proposed model effectively handles long-term temporal dependencies.\n\n|Model|Ours|CaST|BigST|GWNet|STGCN|\n|:-:|:-:|:-:|:-:|:-:|:-:|\n|MAE|**24.89**|26.41|27.76|26.35|28.77|\n\n> Decision-making Process\n\n**Although model interpretability—an active research area—is not the primary focus of this paper, which emphasizes spatiotemporal OOD generalization**—we analyze the prediction decision-making process in Section E.14 using Shapley values, a standard tool for evaluating component contributions. Our model integrates temporal and spatial components to generate the final output. Under spatial shifts, the temporal component dominates, while under temporal shifts, the spatial component takes precedence. Section E.15 further improves interpretability through embedding visualization. In future, we will explore enhancing interpretability by incorporating insights from sequential decision-making or reinforcement learning.\n\n> Potential Applications\n\nSpatiotemporal prediction, to which our method belongs, plays an active role in various downstream applications. For example, traffic forecasting can help urban planners optimize city designs, while meteorology forecasting can guide the public in understanding future climate and environmental conditions.\n\n> Presentation\n\nWe will use the full terms instead of acronyms, except for the name of the model STOP. Moreover, we will remove symbols following formulas.", "claims_and_evidence": "Node-to-node messaging in STGNNs is sensitive to spatiotemporal shifts and this reduces generalization: Empirical results show that removing direct node-to-node interactions improves robustness in OOD scenarios\n\nThe centralized messaging mechanism improves model generalization and robustness: The proposed method higher performance over 14 baselines across 6 datasets.\n\nMessage perturbation via GenPU forces models to learn invariant spatiotemporal knowledge: Models trained with GenPU outperform those trained without perturbations in both generalization and inductive learning.\n\nThe DRO-based training strategy improves adaptability to unseen environments: The proposed method outperforms ERM methods under spatiotemporal shifts.\n\nThe claims and evidences appear to be clear and convincing.", "methods_and_evaluation_criteria": "Evaluated on six spatiotemporal datasets and compared against 14 baselines. The evaluation metrics are MAE, RMSE, MAPE. And the experimental settings include multiple OOD settings, including temporal shifts, structural shifts, and rapid expansion scenarios. All these design choices appear to be legit.", "theoretical_claims": "The centralized messaging mechanism reduces complexity to O(KNdh) -- claimed to be more computationally efficient than traditional self-attention methods.\n\nThe proposed method's optimization function follows a distributionally robust paradigm and this is claimed to lead to better generalization compared to traditional empirical risk minimization.", "experimental_designs_or_analyses": "Experimental setup and results appear to be well-designed and executed. Ablation studies fairly confirm that replacing node-to-node messaging with ConAUbased centralized messaging improves robustness. Also, removing GenPU results in degraded performance. This tells about its role in improving feature extraction. The proposed method outperforms conventional methods by up to 17.01% in generalization and 18.44% in inductive learning. The proposed method appears to be efficient in large-scale datasets. \n\nThis work could also benefit from a discussion on the trade-offs between computational efficiency and model interpretability. Also comparison with meta-learning approaches for generalization could add further depth.\n\nSome question marks down here:\n\n- How does the proposed method perform in traditional IID settings compared to its OOD benefits?\n\n- Could you speak to the computational trade-offs of using centralized messaging versus node-to-node interactions?\n\n- How well does the proposed method scale in real-time applications with large spatiotemporal datasets?\n\n- Could the proposed method be applied to non-spatiotemporal domains such as sequential decision-making or reinforcement learning? Would the proposed modules be of particular use for such use cases?", "relation_to_broader_scientific_literature": "This work builds on prior research in spatiotemporal learning, graph neural networks, and OOD generalization. However, some inconsistencies exist: 1- the study assumes that centralized messaging resolves OOD sensitivity, but it does not thoroughly analyze how different types of spatiotemporal shifts impact performance. 2- the role of message perturbation and DRO in non-spatiotemporal settings is not explored and this effectively limits applicability.", "essential_references_not_discussed": "Related Work, Section 5, does not appear to be comprehensive enough. Although recent works have been fairly introduces, the prior work leading to the SOTA could be better described", "other_strengths_and_weaknesses": "- The paper appears to be well-written in terms of language. However, reading is made difficult by the frequent introduction of acronyms. Keeping track of them creates a significant cognitive load, making it harder to follow the content. Simplifying this would greatly enhance readability.", "other_comments_or_suggestions": "Re context:\n\n- Investigating the impact of the proposed method on long-term temporal dependencies.\n\n- Analyzing the proposed methods interpretability and decision-making process for better model transparency.\n\n- Briefly discussing potential applications in climate modeling, urban planning, and environmental monitoring.\n\nRe writing:\n\n- I do not think a full stop or comma is needed after formulations. This could be revised.", "questions_for_authors": "No questions apart from those raised in my comments in other sections.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper presents a well-motivated and technically sound method for spatiotemporal OOD learning. The proposed framework addresses a fundamental limitation in existing STGNNs by replacing node-to-node messaging with centralized interactions enhanced by message perturbation and robust optimization. The authors provide extensive evidence of performance gains, theoretical justification, and real-world efficiency while showing strong rebuttal responsiveness. While minor limitations remain like presentation clarity and broader framing, the overall contribution is novel and experimental validated. 

In the rebuttal period, the authors actively engaged with reviewers, offering detailed clarifications and new results—such as meta-learning comparisons, long-term forecasting, visualization cases, and cross-domain validation—which directly addressed reviewer concerns and led to multiple score increases.

The reviewers largely converge toward acceptance, and the AC recommend acceptance as well.
Decision: Accept (poster)