A General Spatio-Temporal Backbone with Scalable Contextual Pattern Bank for Urban Continual Forecasting
Authors: Aoyu Liu
Yaying Zhang
Venue: ICLR 2026 Poster (2026)
Keywords: general backbone
contextual pattern bank
continual spatio-temporal forecasting
Primary area: learning on time series and dynamical systems

Abstract: With the rapid growth of spatio-temporal data fueled by IoT deployments and urban infrastructure expansion, accurate and efficient continual forecasting has become a critical challenge. Most existing Spatio-Temporal Graph Neural Networks rely on static graph structures and offline training, rendering them inadequate for real-world streaming scenarios characterized by graph expansion and distribution shifts. Although Continual Spatio-Temporal Forecasting methods have been proposed to tackle these issues, they often adopt backbones with limited modeling capacity and lack effective mechanisms to balance stability and adaptability. To overcome these limitations, we propose STBP, a novel framework that integrates a general spatio-temporal backbone with a scalable contextual pattern bank. The backbone extracts stable representations in the frequency domain and captures dynamic spatial correlations through lightweight linear graph attention. To support continual adaptation and mitigate catastrophic forgetting, the contextual pattern bank is updated incrementally via parameter expansion, enabling the capture of evolving node-level heterogeneity and relevance. During incremental training, the backbone remains fixed to preserve general knowledge, while the pattern bank adapts to new scenarios and distributions. Extensive experiments demonstrate that STBP outperforms state-of-the-art baselines in both forecasting accuracy and scalability, validating its effectiveness for continual spatio-temporal forecasting. Code is available at https://github.com/Aoyu-Liu/STBP.

Published as a conference paper at ICLR 2026
A GENERAL SPATIO-TEMPORAL BACKBONE WITH SCALABLE CONTEXTUAL PATTERN BANK FOR URBAN
CONTINUAL FORECASTING
Aoyu Liu, Yaying Zhang"
Tongji University
fliuaoyu, yaying. zhang]@tongjii.dde
ABSTRACT
With the rapid growth of apatio-tempont data fueled by IoT deployments and urban infrastructure expansion, accurate and efficient continual forecasting has become critical challenge. Most existing Spatio-Temporal Graph Neural Networks rely on static graph structures and offline training, rendering them inadequate for real- world streaming scenarios characterized by graph expansion and distribution shifts. Although Continual
Forecasting methods have been proposed to tackle these issues, they often adopt backbones with limited modeling capacity and lack effective mechanisms to balance stability and adaptability. To overcome these limitations, we propose STBP, a novel framework that integrates general spatio-temporal backbone with scalable contextual pattern bank. The backbone extracts stable representations spatial correlations through lightweight linear graph attention. To support continual adaptation and mitigate catastrophic forgetting, the contextual pattern bank is updated incrementally via parameter expansion, enabling the capture of evolving node-level heterogeneity and relevance. During incremental training, the backbone remains fixed preserve general knowledge. while the pattern bank adapts new scenarios and distributions. Extensive experiments demonstrate that STBP outperforms state-of- the-the-rrt baselines in both forecasting accuracy and scalability, validating its effectiveness for continual spatio-temporal forecasting. Code is available http:///gthhb..omm//oylLLLuuu/
the frequency domain and captures dynamic
INTRODUCTION
With the rapid development of urban ToT sensing systems, spatio-temporal data such as traffic flow (Shao et al. (2226) and air quality (Tian et al. 2025) observations continue to surge (Kumar al. 2022) Hu et al. [223) Fang al. 2026). Conducting efficient and accurate forecasting on data streams has become a core task in the development of smart cities (Jin et al.) 2024 Yang et al. 2025). Unlike traditional offline learning based on static assumptions, real-world urban environments are in state of continuous wvolutiondynnani changes in urban structure and behavioral patterns constantly drive the evolution of graph structures and data distributions.
Tentool Graph Neural Networks (STGNNs) (Kong et al. 2024: Gao et al. 2024 Liu & Zhang 2025) have been widely used to model complex spatio-temporal dependencies. However, most existing models still adhere to the paradigm of "fixed topology offline training": the graph structure is predefined and fixed during the training phase, and the model is deployed directly after training. Yet, shown in Figure [ this static assumption becomes difficult to sustain when the node set continuously expands or the connectivity dynamically reconstructs over time. If one relies solely on structural modifications and continuous fine-tuning handle node increments, model performance often degrades significantly. Therefore, Continual Spatio-Temper Forecasting (CSTF) (Miao et al.) 2024b Chen Liang 2025 Ma al., (025b) has garnered increasing attention. Its goal is to achieve incremental learning and efficient inference on new data without repeatedly relying on retraining with historical data. As shown in Figure T] typical CSTF approaches employ a general spatio-temporal
"Coreeponding author.

---

Published as a conference paper at ICLR 2026
backbone integrated with strategies such reg- ularization, replay, or dynamic architectures to adapt to graph structural expansion and mitigate catastrophic forgetting.
T
Static Assumptions
STGNNs
However, two key issues in existing CSTF meth- ods have not yet been adequately addressed. First, the general backbone adopted by most current methods is relatively simple (e.g., stacks of graph and temporal convolutions). making it difficult effectively handle incremental scenarios character- ized by dynamically changing spatio-temporal cor- relations and long-term distribution drift. Forcibly adapting existing STGNNs for continual learn- ing often leads to performance degradation (Shao) ct al. 2024; Ma et al. 2025a). Second, continual
Bridges the Gup
Effective Continual Forecasting
ا يоаоо Dynamic
History Dota
NMWY
Hrifi
Regnlarization
Roplay of
Weak ST Madeling
CSTF Methods
Figure I: Limitations of existing studies.
optimization strategies based dynamic structural expansion often weakly coupled with the backbone-suee such as direct parameter expansion or prompt concatenation- makinn it challenging to achieve good balance among model stability, adaptability, and interpretability. Based on the above issues, we argue that an ideal CSTF framework should simultaneously address the following four key challenges: 1 handling distributional drift: @ modeling dynamic spatio temporal correla- tions; ① alleviating catastrophic fornstrpphcforgettting and @ designing an incremental strategy that efficiently collaborates with the backbone.
To this end, we bridge the gap between STGNNs and continual learning by introducing general- purpose peatio-tempora backbone with scalable contextual pattern bank (STBP). Specifically, the backbone in STBP leverages frequency- -omenn modules to extract stable spatio-temporal components, mitigating distributional drift. Simultaneously, lightweight, scene-agnostic linear graph attention mechanism is introduced to model dynamic spatial correlations with low computational overhead. To mitigate catastrophic forgetting and support continuous graph structure expansion, we design contextual pattern bank composed of trainable parameters. It incrementally updates knowledge via parameter expansion and interacts with the backbone through gating and attention mechanisms, thereby uncovering node relevance and heterogeneity, and gradually adapting to scenario expansion at low cost. Within this framework, the backbone is responsible for modeling general and stable patterns, while the contextual pattern bank captures node-related heterogeneous contexts, working collaboratively to adapt to continuously evolving environments.
Our main contributions summarized follows: 1 We propose efficient and general back- bone tailored for continual forecasting tasks, capable of modeling dynamic spatial correlations and mitigating distribution shift; @ We design prompt-based guidance mechanism using contextual pattern bank, supporting dynamic model adaptation and alleviating catastrophic forgetting; O Exten- sive experiments on multiple real-worlh datasets demonstrate that ST3P significantly outperforms state-of-the-art baselines in terms of forecasting accuracy, adaptability, and scalability.
2 RELATED WORK
Spatio-Temporal Forecasting, Early studies in spatio-temporal forecasting, including methods like STGCN (Yu et al. 2018) and DCRNN et 20188,, primarily focused on combining basic temporal and spatial elements for prediction tasks. These models typically depended on predefined geographic adjacency matrices, which limited their ability to capture the evolving nature of spatial correlations. In contrast, later advancements, such as GWNet (Wu al. 2019), DGCRN (Li al., 2023), and MegaCRN (Jiang et al. 2023b), addressed this limitation by incorporating adaptive adjacency matrices or learning spatial correlations directly from the data. This shift led to notable improvement in forecasting accuracy. More recently, models like STID (Shao et al.) (222a)). STAEformer (Liu ctt al. 2023a),, and HimNet (Dong et al., (024) have emphasized the significance of distinguishing spatial patterns further enhance forecasting performance. These methods incorporate trainable components, including spatial embeddings, parameter pools, and contextual pattern bank, to more accurately capture spatial variations, boosting both prediction precision and model adaptability.
Continual Temporal Forecasting, TrafficStream (Chen et al. 202T), one of the pioneering frameworks in CSTF, integrates spatio-temporal modeling with continual learning by employing
2

---

Published as a conference paper at ICLR 2026
historical data replay and parameter smoothing to manage long-term streaming traffic data and achieve accurate traffic flow prediction. Building on this line of work, STKEC (Wang et al. 2023a) proposes an influence-based knowledge expansion strategy together with memory-auggentte knowledge consolidation mechanism, which better supports the scaling of transportation networks while alleviating catastrophic forgetting. PECPM (Wang al. (023b) leverages pattern matching to dynamically maintain a traffic pattern bank, enabling efficient, historical-data-free continual learning with improved accuracy. STRAP (Zhang al., 2025) adopts retrieval-augmenter learning, constructing multi- dinensooaannii pattern libraries and using plug-and and-play prompting to fuse retrieved patterns, thereby enhancing out-of distribution (OOD) generalization and mitigating catastrophic forgetting. EAC (Chen & Liang] 2025) introduces prompt tuning via dynamic prompt pool that expands and compresses over time, balancing adaptation to new nodes with knowledge preservation in a parameter-efficien manner. Additionally, UFCL. (Miao et al. 2025) leverages federated learning to protect data privacy and employs a global replay buffer of synthetic spatio-temporal data, addressing the challenges of distributed streaming environments.
3 PRELIMINARY
Graph). We define streaming spatio-temporal graph as a sequence of evolving graphs = {G=]T=> where each graph G. Li, A,) represents the graph at incremental period т. Here, V- denotes the node set, E, the edge set, and adjacency matrix A, E IR N,N,N, connections between nodes. The number of nodes at period is denoted by N, = |V+l. The graph evolves incrementally as G- GT-1 + AG,, where AG, captures structural or feature modifications between periods.
Definition 1 (Streaming
Ei,
Definition 2 (Continual Spatio-Temporal Forecasting). Continual spatio-temporal forecasting aims to develop an optimal predictive model at each stage based on dynamic, streaming spatio-temporal graph data. At each incremental period т, given the current graph G, and historical observations X, E R N. xT, the goal is to predict future signals Y, E R N. as follows:
Y, So(G,, X+),
(1)
where Th, is the length of the historical observation window, and Ty is the forecasting horizon. The model So is parameterized by 0, and continually updated by minimizing:
0-
arg min E(G..x-,Y....... [C (fo X,), Y+)l.
(2)
where C(..)) is loss function, and D, denotes the data distribution at period т.
METHODOLOGY
4.1 OVERVIEW OF STBP
The workflow and architecture of STBP are shown in Figure 2 It consists of two core components: general spatio-temporal backbone and a contextual pattern bank. The backbone, comprising temporal and spatial modules with prediction layer, captures spatio-temporal correlations in evolving networks. The contextual pattern bank, made of trainable parameters, is dynamically expanded and fine-tuned as data evolves. While the backbone captures general, stable patterns, the contextual pattern bank adapts to environmental changes, focusing on context-specific patterns. Guided by prompts, both components collaborate to form an efficient and robust continual learning system.
In terms of workflow, streaming spatio-temporal data is sequentially fed into the STBP. During the initial incremental training phase, the backbone and contextual pattern bank are jointly trained to capture spatio-temporal correlations from current data. In later stages, the backbone is frozen (denoted by a snowflake) retain knowledge learned from historical data, while the contextual pattern bank is updated (denoted by a flame) through expansion and fine-tuning. These updates serve as prompts, guiding the frozen backbone to adapt to new data distributions. This continual learning process, driven by the interplay between backbone and contextual pattern bank, enables the model to progressively enhance its representation power and adaptability while preserving core functionality. For detailed workflow steps, refer to Algorithm in Appendix A.3.2
3

---

Published as a conference paper at ICLR 2026
workflow of
Contextual Pattern Bank Prompt-Based Guidance
T
PHd P!
P
FFW
Imput
Expand
Innut
G
Expund
Input
G¹
DONDOR:0:
P
PP.00
ST-Backbone
ST Backbone
ST Backbone
Chunk T 01111
DLGA
update
Di updato
As Prompt
H.
Trequency-oominn Network
Duul-Stream Linear Graph Attention
Contextual Pattern Rank Exponsion
H
F
Linear
ON ONN
H.
Frequency Embedding
H
Linear
Key
Patiom Bank P
Relevance
FFT
Linear
IFFT
Volue
Induce
H
Linear
Prompt
Guidance
Helerggeeetyy
H
DLGA Module
H
Feod Forward
H
Fre Net
Jamtt
Net
Input X
Spatio Temporal Backbone
Prediction
Figure The overall workflow and architecture of STBP.
4.2
CONTEXTUAL PATTERN BANK
Recent studies (Shao et al. 2022a Dong et al. 2024
0.05
Claster
Chen & Liang 2025 have shown that incor- porating node-specific trainable parameters into STGNNs significantly enhance forecasting performance. Following this insight, we propose expandable contextual pattern bank
Choeer
Flow
0.00
005
-00
200
Dim
Timze Step
P, IR N- composed of trainable parameters, consolidate historical spatio-temporal patterns and generalize to new ones, thereby mitigating catastrophic forgetting and continuously adapting to new incremental scenarios, where d denotes the feature dimension.
Cluster
100
Cluster
400
200
100
200
4000
Stop
Timmo
Figure Contextual pattern bank visualization.
We posit that the model can utilize P- to effec- tively distinguish both the relevance and heterogeneity of nodes, enabling more nuanced under- standing of the underlying data structures. Here, relevance refers to shared behavioral patterns among nodes-such as similar trends or periodic fluctuations-while heterogeneity captures differ- ences arising from distinct node functions external factors such geography, policy, events. To validate this hypothesis, we conduct a SSNE-based analysis on P, trained on spatio-tempont datasets (see Figure 3) which reveals meaningful clustering patterns. Each cluster exhibits distinet characteristics, corresponding to heterogeneity, while nodes within the same cluster display similar temporal dynamics, reflecting relevance.
As shown in Figure 2
given a streaming spatio-temporal input X, E N.X% the backbone model Me. and contextual pattern bank P, E N the incremental learning process formulated as:
Mo(x
At the initial training stage (r 1), both the backbone and contextual pattern bank are jointly trained (denoted with flame). For subsequent stages (r > 1). the backbone is frozen (denoted with snowflake), and only the contextual pattern bank is updated through expansion:
P' P AP-
4

---

Published as a conference paper at ICLR 2026
where AP, E R-N represents newly introduced parameters for the current incremental period. Only the expanded contextual pattern bank P E R. xd is fine-tuned during training. Notably, even without explicit clustering constraints, the contextual pattern bank autonomously distinguishes heterogeneous and relevant nodes through data-driven parameter learning and prompt- based interactions with the backbone, driven by the prediction task. This strategy ensures that the backbone retains previously acquired knowledge, while the contextual pattern bank continually adapts to evolving distributions, incrementally expands represent an increasingly diverse set of environmental patterss, thereby avoiding the inadequacy exhibited by fixed models in novel scenarios.
Distinct from existing work (Wang et al., 2023a; Chen & Liang, 2025 Wang et al. 2023b), we introduce a Prompt-Based Based Guidance (Peebles & Xie [023] Zhang et al. 2023) mechanism to enhance Py's capacity to model both node-level relevance and heterogeneity. Specifically, the contextual pattern bank comprises three groups of trainable parameters:
p( E RN,xd for i E 1,2. As illustrated in Figure 2 these components interact with the backbone's 's hidden representation H, via the following prompt-based gating function:
H P(D ho (1 (!)),
(5)
where hi denotes an arbitrary submodule within the backbone. This gating mechanism enables adaptive modeling of node heterogeneity. Additionally, p acts as a key embedding in the attention module, guiding the backbone to generalize correlation-aware information under task constraints. Im- portantly, since the contextual pattern bank encodes high-level abstractions rather than raw historical data, method supports knowledge retention without revisiting prior data dffe-offering advantages in privacy protection and storage efficiency.
4.3 GENERAL TEETTO--TMPOALL BACKBONE
While the contextual pattern bank mitigates catastrophic forgetting in continual learning, lacks the ability to model dynamic spatio-temporal correlations and handle distributional drift. To address this, we design a general spatio-temporal backbone aimed at handling distributional drift, spatio- temporal correlation modeling. and graph scalability during continual learning. The term general implics that the backbone is independent of the number of nodes and does not rely on any predefined adjacency matrix, making adaptable arbitrary spatio-temporal data structures.
As shown in Figure 2 the backbone operates as follows: the input spatio-tempone data first passes through frequency-domain network (FreNet), which maps it into high- sig--dimensional temporal representations and extracts stable components via frequency domain analysis. A dual-stream linear graph attention (DLGA) module then captures dynamic spatial correlations, followed by feedforward layer with multilayer perceptron for enhanced nonlinear expressivity. Finally, the features are reconstructed to their original shape by another FreNet and passed through a prediction layer. We detail the FreNet and DLGA modules below.
Frequency-Domaite Network. Spatio-temporal data in evolving environments often suffer from distributional drift (Wang et al., 2024 Ji et al. 2025 Zhou et al., 2023). Although the contex- tual pattern bank helps retain stable knowledge, we further address this issue through a dedicated frequency-domain analysis (Xia et al. 2023). FreNet is designed to capture temporal correla- tions while emphasizing stable components in the data, such as periodicity and trends, which are more resilient to distributional changes (Liu & Zhang] (025). Specifically, STBP employs two FreNets-on one at the beginning and one at the end of the backbone (Figure 2 The first maps input data X, XTN through linear layer into high-dimensional representation H, N. which is then transformed to the frequency domain using a Fast Fourier Transform (FFT). A learnable requeencydomaale embedding F, E C( +1) adaptively highlights stable features. This process is formalized as:
H IFFT(FFT(H,) ② F-),
(6)
where H/ E IR x- is further processed by linear layer. The resulting representation H then interacts with the contextual pattern bank component p.") via gating-based prompt guidance (Eq. 5) to produce H; E N-xd which serves as input to the subsequent DLCA module. The second FreNet performs an inverse operation, restoring the feature shape to I xT), Compared to traditional temporal modules like RNNs (Li et al. 2018
Bai Bai et al. 20200) or TCNs (Zheng et al.) 2023 Fang et al. 2023), FreNet offers higher computational efficiency and enhanced ability extract
5

---

Published as a conference paper at ICLR 2026
stable low-frequency components (e.g., periodicity and trends) while suppressing feghufreuuency noise, thereby obtaining more robust temporal representations that are resilient to distributional drift across periods and scenarios.
Dual-Stream Linear Graph Attention. After obtaining stable components, it remains essential to capture complex spatial interactions and lime-varying node correlations. An effective spatial module must adaptively learn node correlations in a data-driven manner, maintain computational efficiency, and scale growing graphs. Graph attention mechanisms (Veličković al. (0088) have emerged as promising solutions, enabling dynamic correlation modeling without relying on fixed adjacency matrices. However, conventional graph attention (Zheng al. 2020; Jiang et al. 2023a Liu et al. 2023a) incurs O(N²) complexity, limiting its scalability. To overcome this, we propose DIGA (Figure 2). which improves efficiency using random feature mapping- -based linear attention mechanism (Katharopoulos et al., 2020). Moreover, DLGA introduces dual-stream structure by
incorporating the contextual pattern bank p!!) E Nx as an additional key. This enables the model to assess the relationship between evolving imput patterns and stored knowledge. Formally:
Q = WgH K = W& V == W.
H Attention IaniiiionnnQA QQ K, V (.P(
(8)
Softmax (oltmax(QK'l T + Q(P(2))
Attention (Q, K. V. P(?)
-QQ))
(9)
= =Q(Q) (o(K) V d(P!)))
Here, Wg. Wx, and W. are trainable projection matrices. H; and H R. xd denote the input and the spatially enriched representation passed to the feedforward layer of the DLGA module, respec- tively. The function (b-) denotes a random feature mapping, with Softmax used for approximation in our implementation. For further details on the approximation derivation, see Appendix I Notably, the linear attention approximation does not explicitly construct an adjacency matrix. Instead, implicitly models dynamic correlations by reordering operations in the attention computation. DLGA reduces computational complexity from quadratic to linear, while preserving dynamic spatial modeling and seamlessly integrating prompt- bosett-baeed knowledge from the contextual pattern bank.
5
EXPERIMENT
5.1 EXPERIMENTAL SETTINGS
Datasets. We evaluate our model on three real-world streaming spatio-temporal datasets from the traffic and metcorology domains. The traffic datasets, PEMS--Steam (Chen et al. 2001 and CA-Stream (Liu et al. 2023b), consist of traffic flow measurements provided by the California Department of Transportation (CalTrans), with a sampling interval of 5 minutes. The meteorological dataset, AIR AIR-Stream (Chen & Liang, 2025). is derived from urban air quality platform of the Chinese Environmental Monitoring Center, with hourly sampling intervals. To ensure fair evaluation, all datasets are split into training, validation, and test sets using fixed ratio of 6:2:2. For each prediction task, the model is trained to forecast the next 12 time steps based on the previous 12 observations. Detailed dataset statistics are provided in Appendix A.4.1
Baselines and Metries. We select representative models from two categories as baselines: & Conventional spatio teporal forecasting models, including lightweight spatio- temporal architectures such as GWNet (Wu et al. 2019). STID (Shao et al., 2022a), and iTrans- former (Liu et al.) 2024b). These models are adapted specifically for incremental training in our experiments. Cont Continual spatio- spatio-temporal temporal forecast i ing model 1s, including Traf- fieStream, STKEC (Wang et al. 2023a), PECPM (Wang et al. (203b)), STRAP (Zhang et al. 2025). and EAC CCen & Liang 2025). The performance of all models is evaluated using the following metrics: Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and Mcan Absolute Percentage Error (MAPE). More details on this are included in Appendix A.4.2
5.2
MAIN RESULTS
The main experimental results are summarized in Table I which reports the metrics averaged over all incremental periods. We also present the results at specific forecasting horizons (3, 6, and 12 time steps ahead), together with the overall average across horizons. STGNNs, including GWNet and STID,

---

Published as a conference paper at ICLR 2026
Table 1: Main experimental results. Bold: best, underline: second best.
Dafaset
Horrio
GWNet
STID
ITransformer Trransoomerr/aafiisttvvaa STKEE PECPM STRAP
EAC
STBP
3
19.64.0.1:24
224.344.224
17.6320,75
14.232000
14.29 12 12.2000 12 141.330
13.88
11.6220.09
MAE
6
19.08=0.1225.46--
20.82.0.76
16.43
16.44
16.35 12 12..3440..
15.40
12.26.0.10
12
00.63=..0:22.44222
28, 28.3320,86
21.7840.77
21.6840 11 2121.1140
22215222.10
18,90
29
13.47
Avg.
20.7=a...26.07..b
21. 21.00
16.95+000
16.96 16.86 12 16.88
15.01+0.2
=2.01
3
32.20
62.20=a.11300 300.0_700000..1
28, 2011.11
23.0020.00
23.0 23.0% 23.077 1s 23.0060
22.26
19.20
PEMS-Stream RMSE
32. 34. 10 ...19
21.15
26.87
26.93 26. 262266766o 26 26.7
21,99
20. 20.5
12
33. 73-7030 46..04666.00 46.20-0.43
45. 1.22
35.29+0.11
35. 113.0.134.777
34..0
30.56
22.67
20.13
Avg.
32 59.0.
.0.21
34.88 11.17
27.52400m
27.56gu 377.7730 2n 27.35
25.30
20.52.u.11
3
27 17 2o
79-22s
32. 4G
18 18.342007
18.5 18.5
18, 18.1994 19 ou 18.011 10. 18 18.35
25.00
MAPE
27 22 10.5539.777
20.771071
20.64
sosss2 2=20.79
20.11
10:30
15.55
(%)
12
29 38....1 47
27.99
641
14.31:4.m
27.8 27.88
27.05 s2 33 2228 20
21.30+v..7
16.75
Avg.
27.79
2..0
39.
21.68110:1
21.50 s221. 73 15 12222.17111
10.46
20.12
10.41
15.65_0.2
3
23 23.440=0.0 so27
20. -0.25 16.0.00
17.82+0
17. 17.69
17 17:93
20
17.08
15.01
MAE
12
24.78=0.0 a:31.0
23.31g0 62
-0.44
0..9
34. 10.00500.00
24.
24.37=0..0
20.00
20.38.0017
26.92+0.3
20.41
27.05 27.6.69 1031 10
ox20.30g0
20.08
24.88
19.68 sx
17.19.0.09
15.78
17.
Avg.
23.73 m:2 0..95
35.34_...05
21.09220.
21.09 1221. 1221.04g0 n
20
20,20
15.77.00.0
3
35 87
31. 58=0000
28.01
28.02+0
1o.oo.22 40 1034. a0 27. 27.46.2...
24.37
CA-Stream
RMSF
12
37 37.5771.1110 19.18.0.55
35. 10.6880..8 as
-0.14
51. 10.10
16
41.50+0..4
22
32.43 o631.944 1..40 o0 27.077 30.6440.23 sa
42.08 2141 1444 an 40.7441.90
37.71+1
20.71
28.08.20.14
Avg.
00.200...00 36
00 12
20.01
33.24 1232. 17 03300000..0
25.70.00.
3
211. 21.61ao. 20 29 2...
21. 76
17.0510.4
16.00 1917 13 19.11 18.26
MAPE
24 20.44 3030000.000000
26. 70
19.22+20.30
18.9% 1r 19.74 20.442200
19.45+1.0
10
14.8
(%)
12
25. 71 36 88 1.29
39. 81 81....s
25.17
21.99 2925 91 0021 97 a0... 21.52
26.20
Avg.
24.70 31.3733 73.0.
24.0020
19.98 30
19:61 to 20:40 21. 21. 15
20.
24.94
3
28. 48 422.85
22.37 1o.75
20.7310 1n
20.95 17 a5 a0 20.11
20.00
MAB
6
29.79
20 s333.
13.115...22
26. 26.22220.4 4s
25.6420
25.54
oe 25.5544 1226. 126.12222 12 25,20 20
24.70.0.30
12
$1.30 5233.88
0.25
29.
29.0110m
28.91 1228.95 (88899 to 29. 10:31
28.57
28.28
Avg.
29 10 33.23
0.22
25. 0....
24.58
21 21.00 25.162221112 36 21.21
23
23.6-1.u.2
3
444.88 011.24.....22..0..
34. 34.98_1.11
32.80
33.13g0 2333.07 sa33.7220
32.19
33.115
AIR-Stream
RMSE
6
48.22-11 =0.83
40. 7.755
40.41
40.38
4.40.444. 40 13
39..633..........
30.81
12
48.34 5555.55_......
45. 45.70_...5
45.54.0 47
45.53go
2745.6 55.61g 276.07
44.65
44.97
Avg.
16.01 is.51.1777 0.33
39.67 10.91
18
38,70
2638.76 76 i (39.37 10.35
37.88
(o
37.76
3
28.0222..o043.222
28, 23.64_1.0s
26.332003
26.24go so 25.79
26.80 26.08
20.64
MAPE
6
39.98 44....044.112-0... 12-0277.
34.9 24.9
33.33+0
33. 10
2x 97 1833 30 19 22.88
30
20.6622..2
12
22.37=1.144.00---
40.700...0
39.27
39.0240
1a 0838.87 47 au
38.85
36.23..0.50
Aug.
39 89.87 a1..6. 44.16_..00
34. 11.15
32.29
32.12 31 82 1n322 37 to os 31.77 50
29.70
rely on static graph assumptions and
Table 2: Comparison of few-shot forecasting performance.
are not designed for continual learn-
ing. Following prior work (Chen) & Liang] 2025). we therefore re- train the backbone from scratch at each incremental stage using only data from the current period. In contrast, i'Transfrmer is scenario- agnostic, so we adopt an online training regime: at each stage it is trained on the complete node set of the current spatio-temporal graph, initialized from the previous period's weights, enabling end-to-end fine-tuning. More detailed experimental results are provided in Appendix
Model
PEMS-Stream 10%
CA-Stream 10%
MAE
RMSE
MAPE (%)
MAE
RMSE
MAPE (%)
GWNet
30.15 on 45.30 a5
33..73 50.80) 43 36.32 36.32_n..
STID
33.421290 50.63
63.96.12..0
37.0910.a
55.101a
38.18
ilransformer
20,99 19
332.67.0.25
49.11
39.01
10
28.39_0.5
TraffieStream
17.20 27 27.49=0.17
21.28 27.63 49
33. 33.25 22
20.45..0.45
STKEC
17.75 12 28.23
27.80 as
21 21.20.20.13
o8 33.20
20.233=0.0
PECPM
17.0510.0 27.20
29.08
21.18..0.15
33.33
21.25_.0.0
STRAP
17.68 10 27.98
31.67
26.34_20.9
39.39
on
21.34_..45
EAC
16.191ncs 25.57 10.00
21.02
20.94
32,15
21.37
STBP
13.58 22 24
18 17.89
17.11
27.11 17. 20.60
Results of conventional methods. As shown in Table I STGNNs trained from scratch achieve only poor performance on all datasets. Although these methods work well under static assumptions, they fail to exploit past spatio-temporal knowledge, resulting in unsatisfactory performance. In contrast, iTransformer performs better by leveraging historical spatio-temporal information through online training. but it still suffers from catastrophic forgetting and is therefore not an ideal solution.
Results of CSTF methods. The best-performing models are those that explicitly mitigate catas- trophic forgetting, including CSTF methods such as PECPM, STRAP, and EAC. Compared with fall-parameter fine-tuning strategies (e.g.. PECPM, STKEC, TrafficStream), lightweight prompt- based adaptation frozen backbone (e.g., EAC, STRAP, STBP) yields higher average accuracy, highlighting the benefits of dynamically tuning only a small set of parameters. Nevertheless, STRAP performs notably poorly on CA-Stream, indicating that retrieval-based pattern matching struggles
7

---

Published as a conference paper at ICLR 2026
Retrain
Online
wib Hockbone
PEMS-Stream
CA-Stream
AIR -tr-Stream
25
30-
25
20
20
30
15
MAB
RMSE
MAPE
MAB
RMSE
MAPE
MAB
RMSE
MAPE
Figure 4: Results of ablation experiments.
PEMS--tteam
CA-Stream
AIR- -tt-Strem
22.5
18.0
MAE RMSE
MAE
13.6
RMSE
RMSE
MAE
RMSE
27.0
RMSE
MAE
26.0
40.0
RMSE
21.0
16.5
12.8
25.5
37.5
24.0
12.0
19.5
15.0
24.0
64
128
192
256
64
128
192
256
64
128
192
256
Channels
Channels
Channels
Figure 5: Results of parameter experiments.
in extreme incremental scenarios with rapid, large-scale topology expansion. Overall, our proposed STBP outperforms all competing models. Compared with the best baseline, STBP reduces the average MAE by 21.44% 21.93% and 2.35% on the PEMSSStream, CA-Stream, and AIR-Stream datasets, respectively. This gain stems from the bridge establishes between STGNNs and CSTF methods: the carefully designed general spatio-temporal backbone and contextual pattern bank jointly capture dynamic spatio-temporal correlations, thereby mitigating catastrophic forgetting and alleviating distributional drift.
Results of few-shot forecasting task. To further evaluate the robustness of the proposed model under low-resource scenarios, we construct few-shot training setting and compare it against existing baselines. Specifically, we simulate few-shot setting in which the sample size of the first incremental period is kept unchanged, while the training set size for subsequent periods is reduced to only 10% of the original. The test set size remains fixed throughout. As shown in Table 2 STBP consistently outperforms all other methods, highlighting its strong ability to extract meaningful patterns from limited data. CSTF baselines are more resilient to low-resource conditions than conventional STGNNs (e.g., GWNet, STID). This demonstrates that when data is extremely scarce, conventional models struggle to capture stable spatio-temporal patterns, whereas CSTF methods can leverage knowledge accumulated from historical stages to adapt more quickly new nodes. The continual learning mechanism effectively mitigates catastrophic forgetting, allowing the model to continuously utilize previously learned general features during incremental learning.
5.3 ABLATION STUDY & PARAMETER SENSITIVITY ANALYSIS
Ablation Study Settings. To validate the core contributions of ST3P, we design the following variants for ablation experiments: O Retrain: The contextual pattern bank is removed. Similar to GWNet and STID, a new backbone is trained for each incremental period using the spatio-temporal graph data of that period, with the corresponding model predicting the results for the current test set. @ Online: The contextual pattern bank is removed. Similar to i'Transfrmer, the model is trained on the complete node data the current ppatio-tempore graph and initialized with the model from the previous period, allowing for adjustments across the entire model. @ w/o Backbone: The contextual pattern bank retained, but the spatio-temporal backbone is replaced with the ones used in StaffiS STKEC, and EAC-i.... replacing FreNet and DLGA with CNN and GCN, @ w/o DLGA: The DLGA module the spatio-temporal backbone is ablated. EAC: We also include EAC, which follows a similar approach, for comparison in the ablation study.
Ablation findings. As shown in Figure 4 the ablation results demonstrate that parameter expansion in the contextual pattern bank, together with spatio-temporal pattern distinction and prompt guidance, is essential for alleviating catastrophic forgetting in continual learning. The performance of the Retrain and Online variants supports this conclusion. Notably, even without the contextual pattern
8

---

Published as a conference paper at ICLR 2026
Initial Pattem Bank
Potta Bunk (2011)
Patter Bank (2017)
Dim
Dim
Dim
-20
-20
evolution.
25
65. Nodes
N:7 Nodex
-20
20
-20
20
Dim
Dim
Dim
Flow
400
Rand
hode
Flow
150-
100
Chastor
Ne tai
100
-lustele
host enc
50
200
50-
extiemd
0-
200
400
600
200
400
600
200
600
Time
Time Step
Step
Cluster
Cluster
Clustee
300
2011
Flow
200-
2011
Nolk
Plore
100
2017
extend
400
2000
200
600
Time Stop
Time Stop
Step
Figure 6: Case study on PEMS-Stream.
80
PEMS Stream Node 2
Label
EAC
400
CA- Streum Node 11500
Label EAC STRP
100
AIR Streum Npde 900
Label
EAC
Traffic Flow
STAF MAT 13.40
Traffic Flow
60
200
FAF MAB 75:0
75
Traffic
200
50-
20
100
25-
100
200
100
300
100
200
300
Time Step
Time Step
Time Step
Figure 7: Visualization of real forecasting results.
bank on traffic datasets, the spatio-temporal backbone alone attains performance comparable to EAC under online training, highlighting the critical role of real-time dynamic correlation modeling and temporal distribution-drift mitigation in adapting to new incremental tasks. The performance drop observed in the w/o Backbone variant further confirms the indispensability of the general backbone and highlights the portability and adaptability of the pattern bank across different backbonn architectures. Moreover, removing the DLGA module leads to significant performance degradation, validating its role in capturing dynamic spatial correlations and integrating prompt-based knowledge. The F FreNet module also makes a notable contribution by improving computational efficiency and enhancing the extraction of stable temporal components.
Parameter Sensitivity Analysis. Additionally, we perform a sensitivity analysis on the adjustable hyperparameter d in STBP. In STBP, d represents the feature dimension for each module's 's feature mapping, as well as the feature dimension of parameters in the contextual pattern bank. The analysis results are shown in Figure 5 Increasing d enhances the model's overall parameter count and improves its expressive power. However, the performance gains from increasing d do not grow indefinitely; after reaching a certain threshold, the performance gain stabilizes. Further increases in d not only fail to improve performance but may also lead to negative effects, causing parameter redundancy, More parameter sensitivity analysis can be found in Appendix A.4.5
App4
5.4
CASE STUDY
To illustrate the distinction and expandability the contextual pattern bank in STBP, we apply t-SNE to reduce the dimensionality of P, Xd on the
dataset. As shown in Figure 6 each point represents a graph node. Initially untrained, the pattern bank shows a chaotic distribution. After incremental training, clear clusters emerge. Nodes within the same cluster exhibit similar periodic and trend patterns in their traffic data, while those in different clusters (e.g., Clusters 1-3) show distinct behaviors. New nodes from later stages (e.g., Nodes 693, 809, 834 in 2017) are
9

---

Published as a conference paper at ICLR 2026
GWNc
STID
Tharafemner
TrafiiSecam
STKEE
EAC
PECPM
STRAP
STEP
PEMS Stream
CA CA-etream
Toy
GPU Monoty (GiB)
30-
STBP O(N)
SIBP OIN)
MAB
MAR
20-
Memiryy Footpein
Memort Poceprint
10-
16
LOGIB
3.BGBB
30
2000
4000
6000
8000
10000
Average Training Time (siperiod)
Average Training Time (s/period)
Nodes
Figure 8: Efficiency comparison. STBP O(N2) denotes the version without linear attention, and Retrain refers to removing the contextual pattern bank.
correctly grouped into existing clusters, demonstrating that the pattern bank effectively distinguishes and generalizes -patio-temporal patterns through parameter fine-tuning, enabling continual adaptation.
In addition, we conduct an intuitive comparison of the forecasting performance between ST3P and EAC in real-world application scenarios. As shown in Figure 7 we select representative nodes from three datasets for visualization. Compared to EAC, STEP more accurately captures dynamic trends, and its predictions demonstrate higher practical relevance in real-world continual learning environments. Additional case studies on other datasets can be found in Appendix A.4.6
5.5 EFFICIENCY STUDY
An effective CSTF method must balance scalability, computational cost, and performance. We evaluate the efficiency of STBP against baselines under the same settings, As shown in Figure the average computational cost per period on PEMS-Stream and AIR-Stream is reported, with scatter size indicating GPU memory usage. We further analyze the impact linear attention, full attention, and removal of the contextual pattern bank using toy dataset. Results indicate that non-continual methods- as GWNet, STID, and rrransformer-rrquiree global parameter adjustments each phase, impairing efficiency. i'Transformer, in particular, incurs high memory overhead due to quadratic attention complexity. Even lightweight non-continual models exhibit limited efficiency incremental training.
In contrast, CSTF methods such as EAC, TrafficStream, and STKE achieve higher efficiency through lightweight backbones and localized parameter tuming. While PECPM and STRAP maintain low memory usage, their training speeds remain modest. Despite its more complex backbone, STBP incurs only minimal overhead compared to models like EAC, thanks to optimizations including frequency- domain processing and linear attention. This enables STEP to deliver substantial performance gains with negligible cost increase. Results the toy dataset confirm that linear attention reduces computational load effectively. As node count grows, the contextual pattern bank introduces only linear additional cost through its lightweight interaction with the backbone, avoiding exponential overhead. Furthermore, on CA-Stream, STBP maintains state-of-the the-ttt-rrt hrrt performance even under drastic graph expansion, demonstrating strong scalability.
CONCLUSION
In this work, we propose STBP, a novel framework for continual spatio-tempora forecasting, By combining a general-purpose backbone with a scalable contextual pattern bank, STBP efficiently mitigates catastrophic forgetting while capturing dynamic spatio-temporal correlations. It adapts to evolving urban data without retraining from scratch, making it suitable for real-time applications. Validated on multiple datasets, STBP demonstrates strong continual learning capabilities. Neverthe- less, STBP currently supports continual learning in single-task setting. In the future, we plan to extend its application to cross-domant continual spatio-temporal forecasting, which will be a crucial step towards developing a foundational spatio-temporal model.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was partly supported by the National Key Research and Development Program of China under Grant 2022YFB4501700 the National Natural Science Foundation of China under Grant 72342026, and Fundamental Research Funds for the Central Universities under Grant 2024-6-DD-02.
REFERENCES
Lei Bai, Lina Yao, Can Li, Xianzhi Wang, and Can Wang. Adaptive graph convolutional recurrent network for traffic forecasting. volume 33, pp. 17804-1788 17815, 2020.
Lucas Caccia, Eugene Belilovsky, Massimo Caccia, and Joelle Pineau. Online learned continual compression with adaptive quantization modules. In International conference on machine learning,
pp. 1240-1250. PMLR, 2020.
Chao Chen, Karl Petty, Alexander Skabardonis, Pravin Varaiya, and Zhanfeng Jia. Freeway perfor- mance measurement system: mining loop detector data. Transportation research record, 1748(1): 96-102, 2001.
Wei Chen and Yuxuan Liang. Expand and compress: Exploring tuning principles for continual spatio-temporal graph forecasting. In The Thirteenth International Conference on Learning Representations, 2025.
Xu Chen, Junshan Wang, and Kunqing Xie. Trafficstream: A streaming traffic flow forecasting framework based on graph neural networks and continual learning. In Zhi-Hua Zhou (cd.), Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, pp. 3620-3626, 8 2021. doi: 10.24963fijcai222214488 StLhhttp:: http:/....rr/.....0/ ijcai. 2021/498
Zheng Dong, Renhe Jiang, Haotian Gao, Hangchen Liu, Jinliang Deng, Qingsong Wen, and Xuan Song. Heterogeneity-infoonn -nnformmmd meta-parameter learning for spatiotemporal time series forecasting. In Proceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining. pp. 631-641 2024.
Yuchen Fang, Yanjun Qin, Haiyong Luo, Fang Zhao, and Kai Zheng. Stwave+: multi-scale efficient spectral graph attention network with long-term trends for disentangled traffic flow forecasting. IEEE Transactions on Knowledge and Data Engineering, 2023.
Yuchen Fang, Hao Miao, Yuxuan Liang, Liwei Deng, Yue Cui, Ximu Zeng, Yuyang Xia, Yan Zhao, Torben Bach Pedersen, Christian S. Jensen, Xiaofang Zhou, and Kai Zheng. Unraveling Spatio-Temport Foundation Models Via the Pipeline Lens: A Comprehensive Review IEEE Transactions on Knowledge & Data Engineering, (01):1 January 2026. ISSN 1558-2191. doi: 10..100/TKD.2220....... 366613666...6666....
10 19.1109//1 TKDE 202...6516....... 3651536
Haotian Gao, Renhe Jiang. Zheng Dong, Jinliang Deng, Yuxin Ma, and Xuan Song. Spatial-
temporal-decoupled masked pre-training for spatiotemporal forecasting. In Kate Larson (ed.), Proceedings of the Thirty-Third International Joint Conference Artificial Intelligence, IJCAI-24, pp. 3998-4006. International Joint Conferences on Artificial Intelligence Organization, 2024. doi: 2024/ Main
(0.249663/jcaa.22244444 URL.e.RLLLLEEE BtL:ttts: UtL_ptppse///e....rr/00...225252 Track.
Danlei Hu, Lu Chen, Hanxi Fang, Ziquan Fang. Tianyi Li, and Yunjun Gao. Spatio-temporal trajectory similarity measures: A comprehensive survey and quantitative study. IEEE Transactions on Knowledge and Data Engineering, 36(5)22191-221,, 2023.
Jiahao Ji, Wentao Zhang, Jingyuan Wang, and Chao Huang. Seeing the unseen: Learning basis confounder representations for robust traffic prediction. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. pp. 577-588, 2025.
Jiawei Jiang, Chengkai Han, Wayne Xin Zhao, and Jingyuan Wang. Pdformer: Propagation delay- aware dynamic long- long-range transformer for traffic flow prediction. In Proceedings of the AAA! Conference on Artificial Intelligence, 2023a.
11

---

Published as a conference paper at ICLR 2026
Renhe Jiang, Zhaonan Wang, Jiawei Yong. Puneet Jeph, Quanjun Chen, Yasumasa Kobayashi, Xuan Song, Shintaro Fukushima, and Toyotaro Suzumura. Spatio-tempora meta- graph learning for traffic forecasting. In Proceedings of the AAAI Conference Artificial Intelligence, volume 37, pp. 8078-8086, 2023b.
Ming Jin, Huan Yee Koh, Qingsong Wen, Daniele Zambon, Cesare Alippi, Geoffrey Webb, Irwin King, and Shirui Pan. A survey graph neural networks for time series: Forecasting, classification, imputation, and anomaly detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, and François Fleuret. Transformers are rnns: Fast autoregressive transformers with linear attention. In International Conference on Machine Learning (ICML), pp. 5156-5165. PMLR, 2020.
Weiyang Kong, Ziyu Guo, and Yubao Liu. Spatio-temporal pivotal graph neural networks for traffic flow forecasting. Proceedings of the AAAI Conference on Artificial Intelligence, 38(8):8627-8635 Mar. 2024. doi: 10.1609/aaai, v3818 28707.
wwt.oooggg/i/... ind.:////.........o..........x.. php/
Rahul Kumar, Manish Bhanu, João Hendes-Moreing and Joydeep Chandra. Spatio-temporal predic- tive modeling techniques for different domains: a survey. ACM Computing Surveys, 57(2): 42, 2024.
Sanghyun Lee and Chanyoung Park. Continual traffic forecasting via mixture of experts. arXiv preprint arXiv:2406..0 03 03/0031444 /40, 2024.
Fuxian Li, Jie Feng, Huan Yan, Guangyin Jin, Fan Yang. Funing Sun, Depeng Jin, and Yong Li. Dynamic graph convolutional recurrent network for traffic prediction: Benchmark and solution. ACM Transactions on Knowledge Discovery from Data, 17(1): 2023.
Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Diffusson convolutional recurrent neural network: Data-driven traffic forecasting. In International Conference on Learning Representations (ICLR), 2018.
Aoyu Liu and Yaying Zhang. An efficient spatial-temporal transformer with temporal aggregation and spatial memory for traffic forecasting. Expert Systems with Applications, 250:123884 2024a.
Aoyu Liu and Yaying Zhang. Spatial-temporal dynamic graph convolutional network with interactive learning for traffic forecasting. IEEE Transactions on Intelligent Transportation Systems, 2024b.
Aoyu Liu and Yaying Zhang. Crossst: An efficient pre-traine pre-training framework for cross-district pattern generalization in urban spatio-temporal forecasting. In 41th IEEE International Conference on Data Engineering, 2025.
Chenxi Liu, Sun Yang, Qianxiong Xu, Zhishuai Li, Cheng Long, Ziyue Li, and Rui Zhao. Spatial- temporal large language model for traffic prediction. In 25th IEEE International Conference Mobile Data Management (MDM), pp. 31-40, 2024a.
Chenxi Liu, Kethmi Hirushini Hettige, Qianxiong Xu, Cheng Long, Shili Xiang, Gao Cong, Ziyue Li, and Rui Zhao. ST-LLM+: Graph enhanced spatio-temports large language models for traffic prediction. IEEE Transactions on Knowledge and Data Engineering, 37(8):484-4485 2025a.
Chenxi Liu, Qianxiong Xu, Hao Miao, Sun Yang, Lingzheng Zhang, Cheng Long, Ziyue Li, and Rui Zhao. TimeCMA: Towards Immemmpoeeeed multivariate time series forecasting via cross-modality alignment. In AAAI, volume 39, pp. 18780-18788, 2025b.
Hangchen Liu, Zheng Dong, Renhe Jiang, Jiewen Deng, Q Chen, and X Song. Stacformer: Spatio- temporal adaptive embedding makes vanilla transformers sota for traffic forecasting. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management (CIKM), pp. 21-25, 2023a.
Xu Liu, Yutong Xia, Yuxuan Liang, Junfeng Hu, Yiwei Wang, Lei Bai, Chao Huang, Zhenguang Liu, Bryan Hooi, and Roger Zimmermann. Largest: A benchmark dataset for large-scale traffic forecasting. In Advances in Neural Information Processing Systems, 2023b.
12

---

Published as a conference paper at ICLR 2026
Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer. Inverted transformers are effective for time series forecasting. In International Conference Learning Representations (ICLR), 2024b.
Jiaming Ma, Bingwu Wang. Pengkun Wang, Zhengyang Zhou, Xu Wang, and Yang Wang. Robust spatio-temporal centralized interaction for ood learning. In Proceedings of the Forty-Second International Conference Machine Learning (ICML), 2025a.
Minbo Ma, Kai Tang, Huan Li, Fei Teng, Dalin Zhang, and Tianrui Li. Beyond fixed variables: Exxanding-variate time series forecasting via flat scheme and spatio-temporal focal learning. In Proceedings of the s1 ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2. pp. 2054-2065. 2025b.
Hao Miao, Ziqiao Liu, Yan Zhao, Chenjuan Guo, Bin Yang, Kai Zheng, and Christian Jensen, Less is more: Efficient time series dataset condensation via two-fold modal matching. Proceedings of the VLDB Endowment, 18(2):226-23) 238, 2024a.
Hao Miao, Yan Zhao, Chenjuan Guo, Bin Yang, Kai Zheng, Feiteng Huang, Jiandong Xie, and Christian S Jensen. A unified replay-based continuous learning framework for spatio-temporal prediction on streaming data. In 2024 IEFE 40th International Conference on Data Engineering (ICDE), pp. 1050-1062. IEEE, 2024b.
Hao Miao, Yan Zhao, Chenjuan Guo, Bin Yang, Kai Zheng, and Christian S Jensen. Spatio- temporal prediction on streaming data: A unified federated continuous learning framework. IEEE Transactions Knowledge and Data Engineering, 2025.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 4195 4205, 2023.
Zezhi Shao, Zhao Zhang, Fei Wang. Wei Wei, and Yongjun Xu. Spatial-temporal identity: A simple yet effective baseline for multivariate time series forecasting. In Proceedings of the 31st ACM international conference on information knowledge management, pp. 4454 4458, 2022a.
Zezhi Shao, Zhao Zhang, Wei Wei, Fei Wang, Yongjun Xu, Xin Cao, and Christian S Jensen. Decoupled dynamic spatial-temporal graph neural network for traffic forecasting. arXiv preprint Stt::22006091122 2022b.
Zezhi Shao, Fei Wang, Yongjun Xu, Wei Wei, Chengqing Yu, Zhao Zhang, Di Yao, Tao Sun, Guangyin Jin, Xin Cao, et al. Exploring progress in multivariate time series forecasting: Comprehensive benchmarking and heterogeneity analysis. IEEE Transactions on Knowledge and Data Engineering, 2024.
Chao Song, Youfang Lin, Shengnan Guo, and Huaiyu Wan. Spatial-temporal synchronous graph convolutional networks: A new framework for spatial-temporal network data forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 914-921, 2020.
Jindong Tian, Yuxuan Liang, Ronghui Xu, Peng Chen, Chenjuan Guo, Aoying Zhou, Lujia Pan, Zhongwen Rao, and Bin Yang. Air quality prediction with physics-guided dual neural odes in open systems. In The Thirteenth International Conference on Learning Representations, 2025.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pictro Liò, and Yoshua Bengio. Graph attention networks. In International Conference Learning Representations (ICLR), 2018
Binwu Wang, Yudong Zhang, Jiahao Shi, Pengkun Wang, Xu Wang, Lei Bai, and Yang Wang. Knowledge expansion and consolidation for continual traffic prediction with expanding graphs. IEEE Transactions on Intelligent Transportation Systems, (117)))190-700) 2023a.
Binwu Wang, Yudong Zhang, Xu Wang, Pengkun Wang, Zhengyang Zhou, Lei Bai, and Yang Wang. Pattern expansion and consolidation evolving graphs for continual traffic prediction. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 23, pp. 2223 2232. New York, NY, USA, 2023b. Association for Computing Machinery. doi: 10. 10.114553580305.35533 35599444.
ISBN 9798400701030.
ht
355 0580300.3555 359945355999
13

---

Published as a conference paper at ICLR 2026
Binwu Wang. Jiaming Ma, Pengkun Wang, Xu Wang, Yudong Zhang, Zhengyang Zhou, and Yang Wang. Stone: A spatio-temporal ood learning framework kills both spatial and temporal shifts. In Proceedings of the 30th ACM SIGKDD Conference Knowledge Discovery and Data Mining, pp. 2948-2959, 2024.
Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, and Chengqi Zhang. Graph wavenet for deep spatial-temporal graph modeling. In Proceedings of the 28th International Joint Conference on Artificial Intelligence (ICAAI), pp. 1907-1913, 2019.
Mingyuan Xia, Chunxu Zhang, Zijian Zhang, Hao Miao, Qidong Liu, Yuanshao Zhu, and Bo Yang. Timeemb: lightweight static-dynamic disentanglement framework for time series forecasting. In NeurIPS, 2025.
Yutong Xia, Yuxuan Liang, Haomin Wen, Xu Liu, Kun Wang, Zhengyang Zhou, and Roger Zimmer- mann. Deciphering spatio-temporal graph forecasting: A causal lens and treatment. Advances in Neural Information Processing Systems, 36:37068-37088 2023.
Bin Yang, Yuxuan Liang, Chenjuan Guo, and Christian Jensen. Data driven decision making with time series and spatio-temporal data. In 2025 IEEE s1st International Conference on Data Engineering (ICDE), pp. 4517-4522. IEEE Computer Society, 2025.
Bing Yu, Haoteng Yin, and Zhanxing Zhu. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. In Proceedings of the 33rd International Joint Conference on Artificial Intelligence (IJCAI), 2018.
Haoyu Zhang, Wentao Zhang, Hao Miao, Xinke Jiang, Yuchen Fang, and Yifan Zhang. Strap: Spatio-temppral pattern retrieval for out-of-distribution generalization. The Thirty-Ninnn Aiirty-inttttn Annual Conference on Neural Information Processing Systems, 2025.
Zijian Zhang, Xiangyu Zhao, Qidong Liu, Chunxu Zhang, Qian Ma, Wanyu Wang, Hongwei Zhao, Yiqi Wang, and Zitao Liu. Promptst: Prompt-cnhanced spatio-temporal multi-attribute prediction. In Proceedings of the 32nd ACM International Conference Information and Knowledge Management, pp. 3195-3205 2023.
Chuanpan Zheng, Xiaoliang Fan, Cheng Wang, and Jianzhong Qi. Gman: A graph multi-attention network for traffic prediction. In Proceedings of the AAAI Conference on Artificial Intelligence. volume 34, pp. 1234-1241, 2020.
Chuanpan Zheng, Xiaoliang Fan, Shirui Pan, Haibing Jin, Zhaopeng Peng, Zonghan Wu, Cheng Wang, and Yu Philip. Spatio-temporal joint graph convolutional networks for traffic forecasting. IEEE Transactions Knowledge and Data Engineering, 2023.
Zhengyang Zhou, Qihe Huang, Kuo Yang, Kun Wang, Xu Wang. Yudong Zhang, Yuxuan Liang, and Yang Wang- Maintaining the status quo: Capturing invariant relations for ood spatiotemporal learning. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 23, pp. 3603-3614, New York, NY, USA, 2023. Association for Computing Machinery, ISBN 9798400701030.
doi: 00.1145535803055355950 URL https: htttp:://ooioo: /doi.org/
10. 70.1145/3550305..2 359942
14

---

Published as a conference paper at ICLR 2026
A
APPENDIX
A.1 NOTATIONS
Table summarizes the notations frequently used throughout this manuscript.
Table 3: The notations that are commonly used in the manuscript.
Notation
Definition
G
[G
Streaming spatio-temporal graph Inputs for the period
X,
Y, P- P', H, Mg F,
Prediction for the r period contextual pattern bank for the period Expanded contextual pattern bank Hidden representation for the period Spatio-Temporal backbone Frequency domain embedding
H. W
Representation after frequency-domen processing
Trainable parameter weights
W
Trainable parameter weights
W
Trainable parameter weights
o(-) H.
Random mapping function Input of the DLGA module Output of the DLGA module
H:¹
A.2 RELATED WORK DETAILS
A.2.1 SPATIO-TEMPORAL FORECASTING
Spatio-temporal forecasting aims to support decision-makee makinn critical domains such as intelligent transportation and smart cities by uncovering dynamic correlations embedded in spatio-temporal data.
These data typically exhibit strong spatial-temporal correlations and pronounced heterogeneity. In recent years, deep learning-based STGNNs have emerged effective tools for such forecasting tasks. STGNNs generally employ temporal modules (e.g.. recurrent neural networks (RNNs) (Li et al. 2018 Jiang al. [223bb] Shaoo et al. (022b) and convolutional neural networks (CNNs)) (Yu et al.
2018 Liu & Zhang 2024b a) to capture temporal correlations, while leveraging spatial modules
(e.g., graph neural networks (GNNs)) /Veličković et al. 2018 Song et al. 2020) model spatial relationships.
Early STGNNs, such as STGCN (Yu al. 2018) and DCRNN (Li al. 2018). combined basic temporal and spatial components for forecasting tasks, often relying on predefined geographic adjacency matrices. However, these static assumptions hinder their ability to model dynamically changing spatial correlations in data-driven manner. Subsequent -uch as GWNet (Wu et al. 2019), DGCRN (Li et al. 2023), and MegaCRN (Jiang et al. introdued adaptive adjacency matrices or learned spatial correlations directly from data, significantly improving prediction accuracy. More recent advances, including STID (Shao et al. 2022a), STAFformer (Liu et al., (023a), and HimNet (Dong et al. 2024), have highlighted the importance of spatial pattern distinction enhancing forecasting performance. These models incorporate trainable mechanisms such as spatial embeddings, parameter pools, and contextual pattern banks to distinguish spatial patterns more precisely, thereby improving both accuracy and adaptability.
Despite these advancements, most existing STGNNs are built static assumptions and are not designed to operate in dynamic, continually evolving opatio-temporal environments- -imiitinn their applicability in continual learning scenarios.
15

---

Published as a conference paper at ICLR 2026
A.2.2 CONTINUAL
LEARNING
Early research in continual learning primarily focused on computer vision (Lee & Park 2024; Miao
al. 2024a) and natural language processing (Caccia et al.) 2020 Xia et al. 2025 With the rapid development of IoT and intelligent transportation systems, attention has increasingly shifted
toward CSTF (Chen et al., 202I; Wang al., 2023a Chen & Liang 2025 Wang et al. 2023b] Miao et al. 2025), which addresses the challenges of dynamically evolving and expanding spatio- temporal data. CSTF aims to enable models to continuously learn and adapt to new patterns and knowledge in changing environments, while minimizing forgetting of previously acquired information or performance degradation.
One of the earliest frameworks in this domain. TrafficStream (Chen et al. 2021), pioneered the integration of spatio-temporal modeling with continual learning. It employed strategies such as historical data replay and parameter smoothing to handle long-term streaming traffic data, achieving accurate traffic flow forecasting. Subsequently, the STKEC (Wang et al., 2023a) introduced an influence-bassee based knowledge expansion strategy and mechanism to better accommodate the growth of transportation networks while mitigating catastrophic forgetting. The PECPM (Wang al. 2023b) framework employs pattern matching mechanism to maintain and dynamically update bank of representative traffic patterns from evolving road networks, enabling efficient continual learning without historical data and improving both prediction accuracy and training efficiency. Meanwhile, STRAP (Zhang et al., 2025) introduces a aerrival-augmenteed approach that builds munti-dimeenional pattern libraries. During inference, it retrieves and fuses relevant historical patterns with current inputs via a plug-and-plan prompting mechanism, effectively boosting generalization in OOD scenarios while mitigating catastrophic forgetting.
knowledge consolidation
The EAC (Chen & Liang 2025 further advanced the field by incorporating prompt tuning, enabling CSTF with a small number of trainable parameters. Its dynamic prompt pool, which supports both "expansion" and "compression," enhances adaptability to new nodes while preserving historical knowledge, improving both generalization and computational efficiency. et al. 2025) leveraged federated learning to preserve data privacy and introduced global replay buffer for synthetic spatio centooal data, addressing the challenges of distributed streaming environ- ments. Despite these advancements, most existing methods primarily focus on alleviating knowledge forgetting, while overlooking the critical role of the spatio-temporal backbone in continual learning scenarios.
addition, the UFCL (Miao)
prompt pool that may lead to historical information loss during compression, our contextual pattern bank adopts pure parametric incremental expansion without compression, more completely preserving historical knowledge. Additionally, while EAC's prompt interaction relatively simple (c.g.. feature addition), our pattern bank employs structured multi-component design (P _P!), p(?) that jointly models node relevance and heterogeneity through gating and attention mechanisms, enabling more comprehensive spatio-temporal representation learning.
Advancements Beyond Existing Prompt Methods. Unlike EAC's
A.3 FURTHER METHODS DETAILS
A.3.1 APPROXIMATION DERIVATION OF EQ. 9
An approximate derivation of the attention mechanism in the dual-stream linear graph attention is presented below:
exp ( ko) Uo N
exp (q P..) Vy
Attention (qu, k2 Vo. po)
N
w= exp (q, kw @ (qw) ① (kx) Ve
w= exp ( Pw.) ò (qu)) T Φ (p))) Vv
Σ @ (qu) & (km)) (qu) T Eni (kk) V.
-= Φ (qv)) T Φ (pw) (qu)) IN o (p. D Φ (qw)
(10)
(qw)² EM @ (km))
E &e (P.))
Term 1: Representation--oun aggrogation
Term 2: Prompt-based aggrogation
16

---

Published as a conference paper at ICLR 2026
Algorithm The workflow of STBP for continual spatio-temporal forecasting
Require: Spatio-temporal backbone MM, contextual pattern bank (P1, P....., P, J. streaming
train data {X1, , ).
Ensure: Optimized backbone Ma. and contextual pattern bank (Pi..... P.)
Initialize:.Mp <- {}, P1
for each period i in {1, 2, 3, . do
if = then
> Initial training phase Construct the initial contextual pattern bank P1
Optimize backbone and contextual pattern bank with initial data X₁;
(Ma Pf) t- argming Mo 1X1, P1)
else
Streaming learning phase
Expand contextual pattern bank P:: P; P, AP;
Inherit parameters: (Ma, P,) + (Ma,,P.11.
Freeze backbone parameters Mg: 0 <- freeze(0)
Fine-tune P, with backbone Ma on X₁:
P argming Mo(X (X, P;)
Table Overview continual spatio-temporal forecasting datasets.
Dataset
Domain
Time Range
Period
Node Expansion
Frequency
655 -> 715 786
PEMS-Stream
Traffic
07/10/2011 0110/201100900//00171
7
+ 822 -> 834 + 850
5 min
+ 871
480 - 691 - 1175
CA-Stream
Traffic
01/01/2019-00/302201
4
5 min
+ 1698
1087 - 1154
AIR-Stream
Air Quality 01/01/2016 12/31/2019
4
hour
-> 1193 - 1202
where Qu is the query tensor of node u; k, and are the key and value tensors of node respectively; and p. represents the prompt information for node v.
A.3.2 ALGORITHM WORKFLOW
The overall workflow of STBP for continual spatio-temporal forecasting presented in a more intuitive manner in Algorithm
A.4
ADDITIONAL EXPERIMENT DETAILS
A.4.1 DATASET DETAILS
Table and Table 5 jointly summarize the characteristics the three continual spatio-temporal datasets used in this study: PEMS-Stream, CA-Stream, and AIR-Stream. These datasets differ in domain (traffic vs. air quality D. temporal span, and topological evolution, collectively covering broad spectrum of real-world non-stationary scenarios suitable for evaluating continual learning models. PFMsS-Stream contains highway traffic sensor readings collected across California from July 2011 to September 2017. spans seven periods with gradual increase the number of sensor modes-from 655 to 871 371-resultig in +33% relative growth. This dataset simulates realistic, long-term infrastructure expansion and serves as benchmark for evaluating model adaptability under progressive and stable topological changes. CA-Stream, also in the traffic domain, covers much
'httpss//dot.c..ge// wttp://www......taott.o.ooonnnaa hhtps://ar..cemmm.. en: cn:1 18007/
17

---

Published as a conference paper at ICLR 2026
Table 5: Topological dynamics and evaluation purposes of the datasets.
Dataset
Topology Change
ANodes
Relative Change
Primary Purpose
FEMSS-treamt
Gradual expansion
216
+33%
Realistic progressive growth
CA-Sri CA-Stream
Explosive expansion
1,218
+254%
Extreme incremental stress test
AIR- -tr-Str
Stable expansion
115
+10%
Cross-domane validation
shorter period (January to April 2019) but features sharp and sudden node expansion-frowe 480 to 1,698 corceponnnn to +254% relative increase. This explosive growth intro- duces significant distributional shifts, making CA-Stream a challenging testbed for assessing model robustness under rapidly evolving conditions. AIR-Stream focuses on urban air quality and envi- ronmental measurements from 2016 to 2019. It exhibits modest but steady node growth- 1,087 1,202 (+10%) and represents relatively stable expansion setting. Its distinct domain and smoother structural changes make it particularly suitable for evaluating cross-domain generalization and robustness to gradual environmental variation.
Table 6: Distribution shift analysis based on MMD tests.
PEMS-Stream
AIR-Stream MMD
CA-Stream MMD
Type
MMD
p
p
p
Original Node
0.0939
0.008
0.3324
0.001
0.0792
0.1119 0.0010
Added Node
0.2958
0.001
0.2679
0.001
0.3361
To further assess con-stationnaity. we conduct Maximum Mean Discrepancy (MMD) tests across different periods, separately evaluating original nodes (present from the beginning) and added nodes (introduced during expansion), shown in Table 6 A distribution shift is considered significant when MMD > 0.1 or p < 0.05, Across all datasets, added nodes consistently exhibit stronger distributional shifts, reflecting the spatial disruptions caused by topological expansion. For instance, CA- A--Steem shows a substamtial shift for added nodes (MMD 0.3361, = 0.0010), consistent with its rapid growth. Interestingly, AIR-Stream records the highest MMD among original nodes (0.3324, p = 0.001), despite minimal structural innngaiindi notable temporal drift in environmental data. This highlights AIR-Stream's 's importance for evaluating robustness to evolving distributions even under stable topology. By contrast, PEMS-Stream shows only moderate drift among original nodes (MMD 0.0939), aligning with its smoother expansion. CA-Stream presents weaker drift in original nodes (MMD- 0.0792, p 0. 0.1119), likely due to its limited temporal span. These results underscore the dual challenge in continual apatio-temporal learning: managing both spatial shifts induced by node expansion and temporal von-staaionarity inherent to dynamic environments, with their nature and intensity varying across domains.
A.4.2 BASELINES AND METRICS DETAILS
In this paper, we provide detailed comparison with two categories of representative models:
Conventional Seaaio-Temporra Forecasting Models. GWNet (Wu et al. 2019): A STGNN
model based on adaptive adjacency matrix that can adaptively capture latent spatial dependencies. This model combines graph convolutional networks and temporal convolutions to effectively capture spatio-temporal correlations in the data. @ STID (Shao al. 2022a): An efficient multilayer perceptron model that solves the problem of sample non-separability using trainable embeddings, showing outstanding performance in spatio-temporal forecasting tasks. @ iTransformer (Liu et al.. 2024b): A time-series model that does not rely on a static graph structure. By modeling the interactions between variables, captures temporal features and is effectively applied to multivariate time series forecasting tasks.
Forecasting Models. These models are designed to handle time- varying data and are suitable for continual training tasks. Like STBP, they belong to the category of continual learning models. We selected the following three representative models for comparison: 0 TrafficStream (Chen et al. 2021): The first model for CSTF, it employs traffic pattern fusion approach, historical data replay, and parameter smoothing strategies to efficiently integrate and learn new spatio-tenne temporal patterns in the continuously expanding and evolving traffic network. O STKEC (Wang et al., 2023a): A traffic forecasting model based on the continual learning paradigm. Through an influence-ban -aased knowledge expansion strategy and augmeeea knowledge
Continual
18

---

Published as a conference paper at ICLR 2026
Table 7: Comparison of prediction performance for each incremental period on PEMS-Streamt Bold: best, underline: second best.
StrM-St Period
Model
Metric
2011
2012
2013
2014
2015
2016
2017
Avg.
MAE RMSE
25.90
20.51g2.11
g8.59
16.17
19.37
17. 17.58_0.8
20.95
19.8 29.87
GWNet
44.79
32.43=2.00
32
29.32+1.47
20.37
31.37+2.11
29.89=1.14
34.9621.12
32.50
MAPE (%)
29.7941.61
29.18
28.89
25.45
29.61
24.79=1.44
26.86
27.79 27.79.0.76
MAE
32.68
26.10
25.36
22.78a1.25
23.37a1.38
24.28
27.98
26. 20.07
STID
RMSB
+3.22 41.02211.4
39.500
35.67=2..0
87.15+2.31
1..94_...78
44.20
=1.67=0.21
MAPE (%)
44.33
38.68g1.18
41.40
39.50
42.06ga..0
39.65_6.a6
42.06g4.37
41.09=2249
MAE
25.44+3.24 20.90
20.37
20.5840.85
20.233g.71
20. 20.83.20.80
22.83
21.00.2.77
iTransformer
RMSE
39.734+.11
33 13.3221.25
32.301+1000
33.25=1.35
33.0421044
15.70=1.22
36.69=1.64
34.88 34.18:1.17
MAPE
35.71
38
g4.61
37.35
42.89
40.492.6.4
40.83
41.35
39. 23.83
MAE
18
16.81 a6
16.1020.12
16.622011
16.39
16. 15
18.04.0.00
16.95
TrafficStream
RMSE
26.72 27.75+0.28
25.6140.13
26.96
26.9640.06
27.29
28.93
29.31+0.10
27. 27.52:00.05
MAPE (%)
21 21.35
21.1420.46
21 21.33440.55
22.61
#1.31
21.36
20.84=0.86
2..99
21. 21.66:0.54
MAE
18.0 40.00
16.26
18
16.38+oo115
16.31
18. 18.41g0.35
16. 20.00
STKEE
RMSB
27.47
26
26.85
25.74
27.17 20.47
27.
28.70
30.03.0..8
MAPE (%)
21.00
21.42
20.5420.47
21.53
21.71
20.38
23.87
21.50.0.02
MAE
18. 13
10.91
16.03
16.27
16.09
16.212o.s
18.05g0.30
16. 2...12
PECPM
RMSE
28. 20.00
26. 04.944
25.4840.48
26.19:0.25
26.71=0.28
28. 25.50...2
29.31
29.31=0.6a
27.33 20.37
МАPE (%)
21.57
21.1820.53
20.71
22.8221.61
22. 22.12241.68
20.99.0.a7
22.7da1.88
21.73320.45
MAE
18.18
17.40
16.07+0.11
16.30
16.04:.0.06
16. 16.118
18.0220.12
16.88.00.1
STRAP
RMSE
27.72 27.7240.04
27.60
20.60
25.4640.16
26.49
26.533g..07
28. 28.34=2.11
29.30
27 27.35-20.13
MAPE (%)
22.66
21.06
22.58
22.87
21.37
21.90
22.54g1.14
22.17
MAE
18. 12
15.41
14.67
15.09
14.96.0.15
14.78...21
16.67.0019
25.67
EAC
RMSE
27.72+0.70
21.23
23.08+0.32
24.20
24.58+0.26
26. 26.000
26
26.9640.30
25 20.30:0.220
MАPE (%)
19.6240.16
19.90
20.40).0.G3
20.01 20.32
20.7240.27
19.98-0.70
22.33++.19
20. 22.044..44
MAE
14.29 2005
12. 22.11
11.60
20.0
11.71
11. 20.0s
11. 3. g0.00
13.422011
12 200.07
STEP
RMSB
22.00.0.07
19.47
18.58
19.54
19.55-0.07
21.91
22.56
20.5220.11
MAPE (%)
16.34 10.12
15. 15.26
15.38
15.51
15.47
10.80
16.76.0.20
15.65+0.21
consolidation mechanism, STKEC helps the model effectively integrate new spatio-temporal traffic patterns in an ever-expanding road network while retaining previously learned patterns. 2 PECPM (Wang et al., 2023b): A continual spatio-temporal forecasting model for evolving traffic networks, relying pattern-matching pattern bank to store representative patterns without full historical data. It fine-tunes with new/conflict nodes for knowledge expansion and uses preservation/tracabbiii retrieval-augmeened framework for OOD generalization, building a multi-dimensiooall key-value pattern library with current data in inference, achieving SOTA without task-specific fine-tuning. O EAC (Chen & Liing 2025): A CSTF based on prompt tuning. By integrating base STGNN with continual prompt pool, it efficiently addresses incremental learning and catastrophic forgetting in streaming data using lightweight trainable parameters.
mechanisms to avoid forgetting. O STRAP (Zhang al. 2025): A
during training. It retrieves similar patterns to fuse
The Excluded Models. Some baselines that might be considered relevant for comparison were excluded, and we provide explanations for their exclusion below. O STAEformer (Liu al., 2023a): a widely recognized baseline, was not included in our comparison due to coneeonvereencc observed when applying the same experimental setting as used for GWNet and STID on the selected three datasets. To ensure fair and unambiguous evaluation, we excluded it from the results and have provided the corresponding training logs in the code repository. UFCL (Miao et al.) 2025): The CSTF method UFCL. is not included in the comparison due to differences in experimental settings, which prevent a fair evaluation.
Metries Details. Additionally, the performance metrics used in the experiments to evaluate the model, namely MAE, RMSE, and MAPE, are defined as follows:
11 MAE-
1
Σ
(11)
7
1=1
19

---

Published as a conference paper at ICLR 2026
Table 8: Comparison of prediction performance for each incremental period on CA-Stream and Bold: best, underline: second best.
Model
Metric
CA-Stream
AIR-Stream Period
Jan-19
Feb-19
Mar-19
Apr-19
Avg.
2016
2017
2018
2019
Avg.
MAE
26
26
20..0
20.98.0.45
20.8220.2
23.73
35. 36.54
29.04
26.87=1.57
26, 26.20
29. 29.06-1001
GWNet
RMSE
11.01
39
79..07
32.
31
20.00
56
45.28
23.63:2552
39 29.29
46.01
1.44
MAPE (9)
23.13
23.72.1.0
22.00
26.79
11.77
34.94100
11.09:1.80
11.68
39.87
11.8
MAR
33.832031
20.80+0.25
26.06-0.46
27 27.5020.75
+0.20
40.3820.50
34.0220022
28-36-0045
29.28400.1
33. 20223-00:
STID
RMSE
15. 10.35
39.84
39.84.e.::
10.70.uust
14.12.0.07
61.10aa..1
52.1221.11
13.90 26.78.0.7
13,
51.72.mmm
MAPE (9)
31.21gam
33.63
31.11gaa
30.95g1195
31.73.0
44.78g0.79
42.04g000
45.92,0040
43.87
44.16.000
MAE
30.00
25.00
=0.27=0.0s
22. 11.0.00
25.34
22.00
27.07=0.5s
20.74
22.284
ITransformer
RMSE
45.21,na5 10.76.0.35
35.891...0 1.310
33.90
38.94
10.92 40.22
34.16_0.30
33:37 Lo.65
39.07 10:1
MAPE (%6
31.23g000
29.3
29.31.10.0
28.31
24.500g0
288
34
38.06.9..0 322225gg
33.64.0.4
322.66g:.00
34.152076
MAE
23
21.87 79
19.15
1s
21.09 09
30.
30.09
26,45
20.34.0.30
21.48
20.5
TrafficStream
RMSE
36.46g0.55
34
34.55-20.34
31.07
28.64-022
28
33.00 +0.5s
48.
48.0441.00
30.244004:
34.22
34.2220..5
322
38.582055
MAPE (%)
21.34g1m3
20.08
19.63
18.88
19.06.0...
31.341,0.3 31.34.0m:
33.17.x.08
30.30
32.29anu
MAE
2t
29 20.35
19.79
19:09
21.00 to
10.12.0.50
26.7420
20.54.0.27
21.33
24.63.0.1
STKRC
RMSE
36 1620.z
35.
78-2..2
331
29
29.71
355.24
25.224
48.4820.51 39.57.0.07
34.33320.70
352
38, 28.20
MAPE (%)
20.12.0.4
20.01
19.50.0.7
1854
19:61
33.22 11.87
32.71.0..
30.6910.s m2m
MAE
24.20
21.40:20.1
19.34
10
19:06
10.86.21.5
26.0220.0
20.43..0.0
21. 21.1550.45
24.742025
PECPM
RMSE
37.05en-2
34. 11
30.54.0:25
29.39
77.777
49.63 14.29....0
10.63:2.39
38.9220.0
34.66-20.05
32, 32.28
39.00=n.a
MAPE: (%)
23
10.07
19.77.100
19.07
10.4
31.84.0..
101.221005
31.03
MAE
10.98
23.87
Lo.41
21.50.0.4
26,25 uo
32.14
26,36
20.22 j0.
21, 91.91 10:00
25.16 10az
STRAP
RMSE
44
38. 23.96620.00
35.98
36.412041
30.06
51.46:9.50
38.9920.
34.07
32.97
39.37
MAPE:(%) (%)
21.712007
22
22.0220.55
20.44
20.43.20.4
21.10
34.20020.0
31.6220094
32.30
31.33
32.37 2022
MAE
22.70.na>
10.78
18.77
10.00
18.3 18.35.1..7
20.20 1o
30.36
25.74
19.74.0.3
21.02 10.00
24,21 10:43
EAC
RMSE
34.85=1119
12
70770...00
20.04
28
12
31.1840
48.1331.71
48.
38.22
32
82.85.0.511
$1.89
20.2
37.85
MAPE (%)
20.56g1.11
20.72+1.24
20.26-1.14
19. 18.14.0.00
20.17a..2
13.4320..2
30.78g0.2
32.02:1.31
30 30.8626.51
31, 31.77
MAE
17.38_0.37 15.80 10 14 n0 14.73 so 15.77 25
30.95
24.32_
18 18.8236.a oo 20.47 20 23. 33.64...20
STEP
RMSE
29.56 29.56-0.84 26. 54440.14
23. 23.58-0.30 23. 13.20.3
25. 70
40:3440.70
37.66--015
32 1440.94
31.9926054
37 70
MAPE (9)
15. 15.47.u.12 11.80
14.30ga.12 15: 15.11 14.94ggum
$1.88 g9
27.98_.u2s
20,222 42 28.722 29. 20.35
1
RMSE
Σ
(y
(12)
72
MAPF
1
Σ
Vi
01
100%
(13)
Y/
(=1
where n represents the number of observed samples, y/ denotes the i-th real sample, and ú, is the corresponding predicted value.
A.4.3 IMPLEMENTATION DETAILS
All experiments are conducted on machine with an NVIDIA Tesla V100 GPU and 32 GB of memory. The Adam optimizer, with an initial learning rate of 0.01, is used to optimize the training process. The batch size is set to 64, the number of training epochs is set to 200, and an early stopping mechanism is implemented to ensure efficient convergence. The reported results for all baselines are the average of five repeated runs.
A.4.4 EXPERIMENT RESULTS
Tables 7 aad 8
report detailed results for each incremental period, where the metrics of a given period are averaged over 12 forecasting steps. Figure provides a visual summary of the same results to better illustrate the performance trends across periods. Overall, ST3P consistently achieves strong performance throughout the entire continual spatio-temporal forecasting process, including both the aggregated performance across all periods and the period-wise results. This advantage largely attributed to the well-designed spatio-temporal backbone and the contextual pattern bank, which together support effective knowledge reuse and adaptation under evolving spatio-temporal patterns.
A.4.5 PARAMETER SENSITIVITY ANALYSIS
Beyond the feature dimension d, we further investigated the sensitivity of two key architectural hyperparameters: the number of DLGA layers and attention heads. As shown in Figure I0] increasing either parameter yields marginal gains at best, and in some cases, even leads to slight performance
20

---

Published as a conference paper at ICLR 2026
PECPM
EAC
PEMS-Suram MAB
Allt Sinam Aog. MAB
Straam MAl
12.5
340
27.5
200
17.5
2012
2014
2016
2417
2013
2009
1ib
PEMS-Sucum
Sucan AVERMSE
RMSE
37.5
35.0
30.0
21.5
55-
32-
235
2015
PEMS MAPE
ACR-Surem MAPE
Avg.APE
72
24
14
2015
Figure 9: Visualization of period-wist wise forecasting performance across incremental periods.
PEMS-Stream
CA-Strcam
AIR-Stream
13.5
MAE RMSE
21.0
MAE
25.5
MAE
MAB
13.0
RMSE
MAB
RMSE
25.5
RMSE
MAE
RMSE
RMSE
16.0
12.5
20.0
24.0
12.0
15.0
24.0
36.0
19.0
Layers
Layers
Layers AIR-Stream
PEMS-Stream
CA-Stream
13.5
MAP
17.0
MAF
MAE
MAB
13.0
RMSE
21.0
RMSE
MAB
RMSE
25.5
RMSS
MAE
26.0
RMSH m
40.0
RMSE
16.0
12.5
20.0
37.5
24.0
12.0
15.0
24.0
Heads
Heads
Heads
Figure 10: Additional Results parameter experiments.
degradation. Overall, apart from the feature dimension, model performance remains relatively insensitive to these hyperparameter variations.
21

---

Published as a conference paper at ICLR 2026
Patkern Bank (0)
Putten Bank (3)
Dmn
Dim
Dim
evolution
Nodes
-50
669 Novee
Dim
600
Random
mode
Hakk
500
Clusterr
Jan
Nede 171
Cluster
Mede
Floo:
400
Flow
200
Flow
1971
200
100
extend
100
0-
200
400
600
200
400
600
200
4000
600
Time Step
Time Step
Time Step
Cluster
600
Cluster
Jan
Jan
Apr
Flow
Flow
400
Flowe
200
200-
extend
-0.05
0
0
200
400
600
200
400
500
Time Step
Time Step
Time Step
Figure 11: Case Study on CA-Stream.
Inicial Patton Bank
Pattem Bank (2016)
Pattera Bank (2019)
1202. Nedes
Dim
2
Dim
-25
evolution
1087 Nades
-50
-50
-00
Din
Din
Diim
Randow modes
Clustoor 7 2016
Cluster
155
Valti
400-
Vilue
400
2019
1015
200
200
extend
200
250
Time Step
Time Step
Time Stop
300
Medl 574
Clwster 3.
Node
Clasmer
Mode 121
Clisstor 2.
2016
2019
Value
200-
Valuee
Node
200
Mode
100
100
0-
50
100
150
200
250
50
1.00
150
200
250
1.00
200
250
Time Stup
Time Step
Time Stop
Figure 12: Case Study AIR-Stream.
A.4.6 ADDITIONAL CASE STUDY
To maintain consistency with the case study on PEMS-Stream, we also conduct case studies on the CA-Stream and AIR-Stream datasets to further validate the expansion and distinction capabilities of the contextual pattern bank in STBP. The experimental results for CA-Stream are shown in Figure II Even in the more challenging task of node increment, STBP's 's contextual pattern bank patterns, incorporating new patterns introduced by newly added nodes into the existing pattern clusters.
effectively distinguishes and consolidates different
Figure 12 presents the results on AIR-Stream. Compared to traffic flow data, the spatio-temporal pat- terns this dataset exhibit more complex periodic and trend changes. Nonetheless, STBP continues to accurately differentiate and consolidate diverse patterns, indicating that its contextual pattern bank has adaptive inductive capabilities for various types of spatio-temporal patterns, independent of the specific dataset type. This mechanism enables STBP to exhibit greater flexibility and adaptability in CSTF tasks.
In addition, Figure 13 provides an intuitive comparison of the predictive performance of STBP and the second-best model. EAC, in real-world application scenarios. These representative cases further substantiate the superior practical utility of STRP in realistic continual learning settings.
22

---

Published as a conference paper at ICLR 2026
PEMS-Stream Node 265
Label BAC
200
CA-Stream Node 171
Label
100
Stream Node 800
Label
100
BAC
BAC
Triffic Flow
Traffic Flow
150
Traffic Flow
STAP MAR: 1469
75
BAG MAR 16.31
100
STaa MAE 10.30
25
50
EAG MAD: 14
0
100
200
300
100
200
300
100
200
300
Time Step
Time Step
Time Step
400
PEMS--tream: Node 449
Label
CA-Stream Node 1512
Label EAC STIP
LIR-Stream Node 600
Label
STIeP MAVE 2.80
Traffic Flow
3000
STOP
Traffic Flow
200
Traffie Flow
EAC XAE 922
200
STAP MARI
STRP MARI 12
100-
100
EAC MAE 23.17
DAC MAR 18:57
100
200
200
300
Time Step
Time Step
Time Step
80
PEMS Stream Node 834
EAC
Lanel
800
CA Stream Node 66
Label STAP
100
AIR Str-Stremm Node 200
Label
Traffic Flow
60-
Traffic Flow
600
Traffic Flow
75
40
400
ST MAP: 47.80
50-
20-
200
EAC MAE: F1.55
25
01
01
DAC MAR: 16:63
100
200
100
200
300
Time Step
Time Step
Time Step
Figure 13: Additional visualization of real forecasting results.
STID
Tlmanffrrmer
Traffcötrram
STKBC
EAC
STRAP
AIR-Stream
Train
Infer
20
STBP OIN)
STBP O(N)
15
STBP OINP
STBP O(N
MAE
30
Time (s)
Retrain
Time (6)
Retrain
Memood Uootgem
24
LOGIB
2.0GBB
15
30
45
2000
4000
6000
8000
10000
2000
4000
10000
Average Training Time (s/period)
Number of Nodes
Number Nodes
Figure 14: Additional efficiency comparison. STBP O(N2) denotes the version without linear attention, and Retrain refers to removing the contextual pattern bank.
A.4.7 EFFICIENCY STUDY
Figure provides additional experiments assessing the efficiency and scalability of STBP, Overall, these results confirm that STBP achieves favorable scalability attention design and modular contextual pattern bank structure enable to handle large-scale spatio- temporal graphs in continual learning settings.
and efficiency, and that its linear-
A.5 LIMITATION
Despite STBP's 's strong performance on benchmark datasets, its limitations in cross-domain gen- eralization warrant further investigation. Current continual learning approaches, including ours, typically assume incremental tasks originate from similar domains-an idealization that diverges from real-world dynamic and heterogeneous environments. In practice, cross-domain distribution shift introduce dual challenges: feature space misalignment and exacerbated catastrophic forget- ting. While STBP's 's architecture exhibits inherent adaptability-wwe with DLGA dynamically capturing topological variations and FreNet extracting domain -nvaariant frequency patterns- iits robustness remains unverified under significant structural divergence between source and target domains. Future work should therefore validate the framework's efficacy in such complex cross-domain scenarios.
23

---

Published as a conference paper at ICLR 2026
A.6 BROADER IMPACT
STBP, with its carefully designed general spatio-temporal backbone and contextual pattern bank expansion mechanism tailored for dynamic scenario changes, effectively achieves continual spatio- temporal forecasting. This approach demonstrates that the spatio-temporal backbone can serve as stable infrastructure, consistently retaining the ability to model general spatio-temporal dependencies. When facing new or evolving scenarios, there is no need to retrain the backbone. Instead, by introducing scalable parameters relevant the current scenario, the model can rapidly adapt to new tasks, Building on this concept, we aim to advance the development of spatio-temporal foundational model with enhanced cross-domain generalization, while concurrently exploring the potential of Large Language Models (LLMs) in spatio-temporal and time-series forecasting tasks (Liu et al. 20256 2024a 2025a).
This involves two key directions: introducing explicit domain adaptation mechanisms to better distinguish between domain-speen speciifc and shared features, and @ exploring cross-domain shared contextual pattern banks to enhance adaptability while maintaining efficiency. This approach involves continuously training a unified backbone model with spatio-tempon data from multiple heteroge- neous domains, thereby enhancing its spatio-temporal representational capacity. As data from various domains are continuously integrated and trained, the spatio-temporal foundational model will evolve, enabling efficient generalization and adaptation entirely new scenarios tasks by incorporating only small number of additional parameters. Such a model holds the potential to benefit society by improving intelligent transportation through more accurate traffic forecasting and supporting climate resilience via advanced environmental modeling.
A.7 LLM USAGE
In accordance with the ICLR 2026 policy on large language model (LLM) usage, we disclose that used an LLM (ChatGPT) solely for the purpose of improving the grammar, clarity, and fluency of the manuscript. The content, structure, technical contributions, experiments, analysis, and all scientific writing
entirely conceived, drafted, and validated by the human authors. The LLM was not involved in research ideation, experimental design, data analysis, or any aspect of the scientific content creation. All outputs generated by the LLM were reviewed and edited by the authors ensure accuracy and correctness. We confirm that no hidden prompts. prompt injections, or LLM-generated falsehoods were introduced the manuscript, and all use of LLMs complies with the ICLR Code of Ethics.
24


Reviews:
[{"review_id": "rpKtiqq2Rg", "reviewer": "Reviewer_j7eh", "summary": "This paper proposes the STBP framework for urban continual spatio-temporal forecasting. The method combines a general spatio-temporal backbone with a scalable contextual pattern bank, adapting to dynamically evolving spatio-temporal data through incremental pattern bank expansion while freezing the backbone network. This design effectively balances catastrophic forgetting mitigation, dynamic correlation modeling, and computational efficiency.", "questions": "Please refer to Weaknesses", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The task of continual spatio-temporal forecasting is interesting and has significant practical value for real-world applications.\n\n2. The proposed method demonstrates substantial performance advantages over the compared baselines across multiple datasets.", "weaknesses": "1. While the application scenario is interesting, the proposed method lacks significant insight. The approach of freezing the backbone network while expanding a dynamic pattern bank to handle incremental node expansion appears to be a straightforward solution, and similar strategies have been proposed in prior work (e.g., EAC uses prompt pool expansion).\n\n2. Insufficient baseline comparisons: Only six baselines are compared. More detailed experimental settings explaining how conventional spatio-temporal forecasting models (GWNet, STID, iTransformer) are adapted for this incremental scenario would help readers better understand the main results. \n\n3. The paper does not address a critical question: how much training data do existing baseline models require on new nodes to achieve acceptable performance? If baselines can reach satisfactory accuracy with only a short period of data (e.g., one week), the practical utility of the proposed continual learning approach would be significantly diminished. Conversely, if long-term data is necessary, this perspective could strengthen the motivation and method presentation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "keouNt9XtC", "reviewer": "Reviewer_3Woj", "summary": "This paper addresses the challenge of modeling dynamic and evolving spatio-temporal data in urban environments and proposes STBP, a continual forecasting framework. The method combines a general spatio-temporal backbone with a scalable contextual pattern bank, aiming to handle distributional drift and topological evolution without retraining the entire model. Extensive experiments conducted on three real-world streaming datasets demonstrate that STBP outperforms existing methods in terms of predictive accuracy, scalability, and resistance to catastrophic forgetting.", "questions": "1. During parameter expansion of the contextual pattern bank, have redundancy or performance degradation issues been observed?\n\n2. The use of frequency-domain modeling is claimed to help mitigate distributional drift. From a theoretical perspective, what are the advantages of this approach?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- Originality: The paper introduces a novel combination of frequency-domain modeling, lightweight linear graph attention, and a scalable prompt-based contextual parameter expansion strategy. This bridges the gap between spatio-temporal modeling and continual learning, making a meaningful contribution to the continual spatio-temporal forecasting field.\n\n- Quality: The model architecture is well-structured with clear theoretical underpinnings. Each module is logically integrated to form a coherent pipeline. Specific designs address key challenges in CSTF, such as distributional drift, topological changes, and catastrophic forgetting.\n\n- Clarity: The experimental setup is comprehensive, covering multiple representative real-world datasets. The appendix provides additional implementation details, which enhance reproducibility and interpretability.\n\n- Significance: The proposed framework has practical value and potential applicability in domains such as traffic forecasting and meteorological analysis, where long-term adaptation to evolving topology and data distributions is essential.", "weaknesses": "1. Although the paper briefly acknowledges the challenge of cross-domain generalization, the main text lacks a detailed analysis of how STBP handles domain shifts, especially when there are significant structural differences between source and target tasks.\n\n2. The current design tightly couples the contextual pattern bank with the backbone, which may limit the modularity of the backbone and restrict its transferability to other domains or tasks.\n\n3. There is a typo in the title of Figure 5: \"PESM-Stream\" should be corrected to \"PEMS-Stream\".", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2FzNjBwtPw", "reviewer": "Reviewer_qSGs", "summary": "The paper proposes STBP, a framework that aims to unify STGNNs with continual learning. By combining a general spatio-temporal backbone and a scalable contextual pattern bank, the method addresses challenges of node expansion and concept drift through prompt-guided adaptation. Experiments on three public benchmark datasets show that STBP achieves competitive accuracy while maintaining efficiency and scalability.", "questions": "1.  Figure 3 shows well-separated clusters in the contextual pattern bank $ P_{\\tau} $, yet the method does not include explicit constraints to enforce such separation. Could the authors clarify what drives this differentiation?  \n2.  Compared to similar prompt-tuning approaches such as EAC, what are the specific advantages and contributions of the contextual pattern bank?  \n3. The output of FreNet is $ H_{\\tau}^f $. How is it transformed so that the input of DLGA is $ H_{\\tau}^s $? The author did not mention it in the paper.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "1. Provides a general formulation for streaming spatio-temporal graphs that can be extended beyond traffic applications, with clear practical relevance to smart cities.  \n2.  Conducts comprehensive experiments with diverse presentation formats, supporting reproducibility and interpretability.  \n3.   Well-organized and clearly written.", "weaknesses": "1. The proposed method integrates multiple components, e.g., frequency-domain network, dual-stream linear graph attention, prompt-based continual learning, but each component is relatively straightforward and has been explored in related domains. The overall algorithmic novelty is limited.  \n2.  While the authors propose a model-agnostic continual learning strategy, they do not demonstrate its generality across different backbone architectures, leaving its broader effectiveness unverified.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)