How Different from the Past? Spatio-Temporal Time Series Forecasting with Self-Supervised Deviation Learning
Authors: Haotian Gao
Zheng Dong
Jiawei Yong
Shintaro Fukushima
Kenjiro Taura
Renhe Jiang
Venue: NeurIPS 2025 poster ()
Keywords: Spatio-Temporal Forecasting; Deviation Learning;
Primary area: deep_learning
TL;DR: We propose ST-SSDL, a spatio-temporal forecasting framework by modeling dynamic deviations with self-supervised prototype-based latent structuring.

Abstract: Spatio-temporal forecasting is essential for real-world applications such as traffic management and urban computing. Although recent methods have shown improved accuracy, they often fail to account for dynamic deviations between current inputs and historical patterns. These deviations contain critical signals that can significantly affect model performance. To fill this gap, we propose $\textbf{ST-SSDL}$, a $\underline{S}$patio-$\underline{T}$emporal time series forecasting framework that incorporates a $\underline{S}$elf-$\underline{S}$upervised $\underline{D}$eviation $\underline{L}$earning scheme to capture and utilize such deviations. ST-SSDL anchors each input to its historical average and discretizes the latent space using learnable prototypes that represent typical spatio-temporal patterns. Two auxiliary objectives are proposed to refine this structure: a contrastive loss that enhances inter-prototype discriminability and a deviation loss that regularizes the distance consistency between input representations and corresponding prototypes to quantify deviation. Optimized jointly with the forecasting objective, these components guide the model to organize its hidden space and improve generalization across diverse input conditions. Experiments on six benchmark datasets show that ST-SSDL consistently outperforms state-of-the-art baselines across multiple metrics. Visualizations further demonstrate its ability to adaptively respond to varying levels of deviation in complex spatio-temporal scenarios. Our code and datasets are available at https://github.com/Jimmy-7664/ST-SSDL.

How Different from the Past? Spatio-Temporal Time Series Forecasting with Self-Supervised Deviation Learning
Haotian Gao'; Zheng Dong': Jiawei Yong²
Shintaro Fukushima', Kenjiro Taura', Renhe Jiang"
'The University of Tokyo. Toyyot Motor Corporation
{gaoht6, zhenglongg00/oottloo.. com {jiawei_yong, s_ wwk.uuusimaaaaan toyota..oootttoooooooooooo co. tauβeidos.ice iokkoo. ac. jp. jiangrhdcsis ac.
Abstract
Spatio-temporal forecasting is essential for real-world applications such as traf- fic management and urban computing. Although recent methods have shown improved accuracy, they often fail to account for dynamic deviations between current inputs and historical patterns. These deviations contain critical signals that can significantly affect model performance. To fill this gap, we propose ST- SSDL., TpatoolTemporal time series forecasting framework that incorporates Self-Supervised Deviation Learning scheme to capture and utilize such deviations. ST-SSDL anchors each input to its historical average and discretizes the latent patterns.
space using learnable prototypes that represent typical spatio-tempon Two auxiliary objectives proposed refine this structure: contrastive loss that enhances inter-protte prototype discriminability and deviation loss that regularizes the distance consistency between input representations and corresponding prototypes to quantify deviation. Optimized jointly with the forecasting objective, these com- ponents guide the model to organize hidden space and improve generalization across diverse input conditions. Experiments on six benchmark datasets show that ST-SSDI. consistently outperforms state-of-the-art baselines across multiple metrics. Visualizations further demonstrate its ability to adaptively respond varying levels deviation in complex spatio-temporal scenarios. Our code and datasets are available at https: / thub. com/ Jimmy- 7664/77-6666575555
Introduction
Accurately forecasting the spatio-temporal time series generated from various sensors is essential for wide range of applications like traffic flow regulation, energy demand estimation, and climate impact analysis. Due to the complex dependencies across time and space, considerable efforts have been made to learn rich spatio-temporal patterns [7, 14, 44, 49, 63, 77, 78, 83, 24, 21, 58, 56, 57, 19, 20, 52, 34, 16, 74, 75, 11, 15, 12].
However, a crucial aspect still remains overlooked by current models: the deviation between current observations and their historical states. In real-world transportation systems, the present time series frequently differ from historical states due policy interventions, special events, or external incidents. These deviations can provide valuable insights for forecasting, they often signal changes that impact future behaviors. Although some recent efforts [40, 42, 24, 54] attempt incorporate
Equal contribution. Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

historical context by using fixed temporal offsets, such as observations from the same time on previous days or weeks, these methods struggle to capture the dynamic deviations, limiting their ability to respond effectively when current patterns diverge from history. As shown in Figure (a)), the degree of deviation dynamically varies with spatio-temporal contexts. For example, traffic sensor in downtown often shows high variance, as indicated by the green lines, while a rural road can remain relatively stable, as shown by the blue pairs. Therefore, modeling dynamic deviations remains a challenge in spatio-temporal forecasting. A straightforward approach to do this is to compute the distance between the current input and its historical average, flagging deviations when this distance exceeds threshold [69, 79]. However, this strategy treats deviation as binary event, whereas in reality, it evolves continuously. It is difficult to precisely quantify how the current input differs from the past, especially when encoded into nigh-dimensional latent space. As visualized in Figure 1(b), while the deviations of the green pairs Dj and D2 appear evident in the physical space (i.e., D1 R 40,
D2 ~ 20), it is uncertain how much the corresponding distance Dj and D2 should be in the latent space. These observations highlight our key challenge: how quantify dynamic spatio-temporal deviations in continuous latent space and leverage them to enhance spatio-tempont forecasting.
To address the above challenge, we introduce our core idea: relative distance consistency. Intu- itively, the current-history pairs that close (far) in the physical space should remain close (far) in latent space. i.e., D1 D2 + D2 > Dz. Building upon this idea, we propose Self- supervieee De- viation Learning (SSDL) that enables the spatio- temporal model to capture spatio-temporal devi- ations in fully self-supervised manner without any prior knowledge. Specifically, we first utilize the historical average of each current input as a self-supervised anchor. Then, we discretize the continuous latent space by using a set of learnable prototypes, where a contrastive loss is employed to guide the learning and discretization process. Next, we map each input and its anchor to their nearest, prototypes via gross-attettion, treating each prototype as the proxy of the latent representation. The distance between the proxy prototypes of the current input and its historical anchor then serves as an approximation of the latent-space deviation. Finally, we propose a self self-supervised deviation loss to minimize the discrepancy between the physical-space distance 1) and its intent-space counterpart Đ, thereby enforcing the principle of relative distance consistency, i.e., D + 11. D + D 4. Our contributions are summarized as follows:
(a) Vasher
Curral
Hidory
(b)
Space Quesni Ottoon
B
Dz
40
20
Tims
Figure I: (a) Deviations between current and his- torical states vary with spatio-temporal context. (b) Such deviations in latent space are hard to quantify, so we leverage relative distance con- sistency: surrent-history pairs that are close (far) in physical space should remain close (far) la- tent space, i.e., D₁ > D2 + D₃ > Dz
We propose SSDL, the first self self-super method designed to learn deviations in spatio- temporal time series. It is implemented with two self-supervised objectives: a contrastive loss to discretize the continuous latent space and deviation loss to enforce relative distance consistency between the physical and latent spaces.
We propose ST-SSDI, a novel spatio-temporal forecasting framework that incorporates self-supervised deviation learning enhance spatio-temporal forecasting performance, improving adaptability to various deviation levels.
We validate ST-SSDL through experiments on six benchmark datasets, where it achieves state-of-the-art performance. Comprehensive ablation studies and visualized case analyses further confirm that the model adapts its latent space based on deviation levels.
2 Related Work
2.1
Forecasting
Traditional statistical methods [10, 26, 55, 68] primarily focus on capturing temporal dependencies. With the rise of deep learning, numerous spatio-temporal forecasting models have been proposed to better capture spatio-temporal relations. Early work emphasizes graph neural networks (GNNs), which naturally align with sensor network structures [44, 77, 83, 24, 67, 2, 65, 35, 17, 3, 18].

---

Recently, inspired by the success of Transformer [71]. line of attention- based models [33, 49, 36, 38, 81. 76] has shown strong performance in spatio-temporal forecasting. To address the quadratic time complexity of attention mechanism, Mamba [23] introduces a linear-time sequence modeling framework, which has drawn series of Mamba-base models [29, 66, 70, 43, 39] for spatio-temporal tasks. Beyond these complex designs, several studies [63, 73, 72, 86] have explored the lightweight MLP-based models modeling spatio-temporal data. However, these existing approaches overlook the deviations between current and historical patterns under dynamic spatio-temporal conditions.
2.2 Self-Supervised Learning
Self-supervised learning (SSL) has become powerful paradigm across vision, language, and beyond. In computer vision, contrastive methods [6, 27, 22], alongside masked modeling approaches [28, 80), achieve strong performance without labels. In NLP, self-supervised pretraining strategies [37, 51, 9] also have demonstrated their effectiveness in language modeling tasks. These advances also extend to speech [30, 1, 45], multimodal [82, 25], and time series domains [50, 61]. Recent studies have explored the potential of self-supervised learning in spatio-temports data. Specifically, several methods based on self-supervised masked modeling have shown promising results [64, 21, 48, 47, 32, 84). In addition, self-supervised contrastive learning frameworks [85, 13, 59, 31] have also been developed, demonstrating their effectiveness in learning robust spatio-tempore representations. However, these methods fail explicitly model the dynamic deviation in spatio-temporal data. Our method fills this gap through self-supervised deviation learning approach that enhances robustness without relying on additional supervision.
3 Problem Definition
Given N spatially distributed sensors, at every timestep t, each sensor has C-dimensional observa- tion X E RC, For an input tensor X E RTxNxC = =x,T++1a that contains the most recent T steps, the forecasting task is predict the next 7" steps for all nodes and channels:
R Fo(X-TT11a).
Fg TxNXC R XXxC
where Fe is a learnable spatio-temporal model parameterized by 9. In our study, C is equal to I in the used benchmarks for ppatio-temporal time series forecasting [44, 67, 83).
4 Methodology
4.1
Self-Supervised Deviation Learning
Real-world spatio-temporal data often exhibit informative deviations from historical patterns which are critical for forecasting but typically overlooked by existing methods. Moreover, such deviations not categorical events but emerge continuous, context-dependent variations across time and space, making them difficult to capture using binary method like threshold. To tackle these challenges propose Self- Supervisdd Deviation Learning (SSDL).
of modeling spatio-temporal deviations, anchors current inputs to historical averages and discretizes their latent differences via learnable prototypes. Two auxiliary self ceff-supervised objectives further guide the model to organize latent space and quantify deviations, improving adaptability across diverse input conditions.
History as Seff-Supervised Anchor. A core challenge in modeling deviation lies in the absence of principled reference: without a contextual baseline, dynamic deviations become hard to define and quantify. To address this, we introduce historical averages as anchors as illustrated in Figure 2(a). These anchors summarize recurring modeling. Concretely, the full training sequence X train E I x is partitioned into S non- overlapping weekly segments based on the periodicity of spatio-temporal data. They each contain T timesteps, resulting in tensor X SXT ==SxT"xxxx" "xxx"" The historical anchor X E RR? xNxC is then computed by averaging aligned timesteps across all weeks as X 1 Σ X For the current input X E T'XXCC we retrieve its timestamp-lligned historical anchor X E R'xXNC from X= Both sequences are then encoded via shared function feno(.), (.), producing latent representations H
patterns and serve as references for deviation
ş
3

---

Deviations from Historical Anchor
(b) Self-Supervised Deviation Learning
ST-SSDL. Model Architecture
Vilue
Caerant
Diventiad Latent Space
Inper
TimesAiigaal
Anchor
Historical
Con
X Prittppe
Curient Quary
Anchee Unery
Encoder
Series #1
Self- Self-Supervised Deviation Learning
Dey
Series
Decnder
Eran
Time
Truth
MMA
Figure 2:
The overview of our proposed framework ST-SSDL: Centio-Temporal Time Series Fore-
casting with Self- Self-SSuer Deviation Learning.
and H Vxh with hidden dimension h:
H f (X)) Ha = f (X"
(2)
This anchoring process equips the model with a context-aware pivot, offering a structured reference against which deviations be systematically quantified. By serving intrinsic supervision signals, these anchors guide the model to learn deviation-aware representations without external labels. As a result, grounding hidden representations in historical context supports the formation of structured latent space, forming a concrete basis for self-supervised modeling of dynamic deviations.
Sele-Supervised Space Discretization. Deviations in spatio-temporal data exhibit diverse and complex dynamics, making them difficult to measure directly within the continuous latent space. To mitigate this, introduce set of M learnable prototypes ....... ..P... dimension d that serve as representative prior works [5, 46, 53] mainly focus learning such prototypes to capture typical behaviors, additionally leverage them to discretize the latent space. This enables structured comparison between current and historical representations. The latent space discretized by prototypes provides foundation for quantifying deviation. Moreover, we establish interaction between inputs and the prototypes via query-prototype attention mechanism. For query Q Rd projected from input hidden representation, the attention scores over the prototypes are computed as follows:
E Raxx with hidden patterns as shown in Figure 2(b). While
exp ((Q-PT) PTT//V]
ΣE exp ((Q PJ) /va)
where a; denotes the attention weight assigned the i-th prototype P,. Higher attention scores indicate stronger affinity between the query and the corresponding prototype, reflecting the underlying similarity in spatio-temporal characteristics.
Based on the computed attention scores, we sort the prototypes in descending order for each query. For current input hidden representation H* the query Q' is obtained by applying linear projection to calculate the attention scores. The resulting ranking reveals how the input matches with the prototype structure. This prototype-basee based discretization transforms unstructured deviations into interpretable patterns, facilitating accurate measurement and comparison under diverse spatio-temporal dynamics.
To guide the prototype organization, define contrastive objective that promotes discriminability among prototypes. Specifically, we select the most relevant and second-mosy relevant prototypes as the positive sample P and negative sample A These selections serve supervision targets in the contrastive learning by defining which prototypes should drawn closer or pushed apart. variant of the triplet loss [60] is adopted as:
Lcon max (15(Q))
- p [V(Q) -V +5,0)
where 5 is positive margin and V denotes the stop-gradient operation applied to the query. This contrastive formulation encourages each prototype to pull closer its most semantically aligned query while pushing away less relevant one, resulting in inter-prototype separability. Morcover, this

---

mechanism partitions the latent space into multiple prototype-centered regions, which can be viewed like discrete bins as illustrated in Figure 2(b). As a consequence, the latent space becomes structurally organized, with distinct clusters of queries forming around their associated prototypes. Such design enhances the model's ability to distinguish inputs based on their alignment with prototypes.
Self-Supervised Deviation Quantification. With the prototype-based discretization offering a structured view of spatio-temporal patterns, we can design an explicit training signal to enforce relative distance consistency. To this end, we propose a self-supervised learning strategy that leverages prototype rankings from both current representation H and historical representation H to guide latent organization via deviation loss.
The deviation loss promotes relational consistency between input representations and prototypes in manner. Specifically, for the historical input, the query Q" is obtained by
a cesstace-oo-distanne linearly projecting its hidden representation H7. The corresponding positive prototype Po is then selected using the same ranking procedure as applied to Q.. Lastly, we compute the I, distance between Q' and Q", and enforce the distance between their corresponding positive prototypes p and pa to match it:
Dev -1100- -Q'1) - - P" 11 |b|]]
(5)
By stopping the gradient, VIIQ" - 11) can be taken as an approximate of the physical-space distance D) for the current input X and its historical anchor X With the nearest prototypes as proxies, ||P - p can represent their latent-space distance D. As result, Lev ensures relative distance consistency by minimizing || Dl11 thus inputs with similar semantics are routed to the same prototype or neighboring prototypes, while those with distinct patterns are mapped further apart. Meanwhile, the V operator in Loon and Dev also prevents the model from entering lazy mode where all representations become overly similar and map to the same prototype.
Together, these objectives refine the prototypes for deviation- learning. The contrastive loss shapes inter-prototype separation, while the deviation loss promotes deviation consistency between input representations and corresponding prototypes in latent space. This joint formulation enables the model to quantify dynamic deviations in a fully self-supervised manner.
4.2 Enhancing Opatio-Tempore Forecasting with Self-Supervised Deviation Learning
Current spatio-temporaa forecasting models often rely solely mapping past observations to future targets, overlooking potential deviations between the current input and its historical context. Such dynamic deviations offer significant information that can improve predictive accuracy. As demon- strated in Figure 2(c), building upon the SSDL method introduced above, we propose ST-SSDI, a spatio-temporal forecasting framework that incorporates it core component.
Recent advances in spatio-temporal modeling [44, 2, 41, 62, 35, 17] have demonstrated the effective- ness of injecting graph convolutional operations into recurrent neural architectures. These designs culminate in Graph Convolution Recurrent Units (GCRUs), which whichjointly jointly capture spatial dependencies encoded by the graph topology and temporal dynamics intrinsic to time series. Therefore, ST-SSDL follows this paradigm through a GCRU-based architecture to effectively capture apatio-temporal dependencies. In detail, GCRU are formally defined as follows:
GCRU as the encoding function fone in Equation (2). The details of
Ft ([X, H *g O, + b,)
04 ([x, H kg On, + bw)
(6)
C, tanh ([X, |r, @ H.-1l 11 tg O + ba)
He = 4e @ H + (1 wx) o ct
where X, E RV and H, E Nxh represents input and output at timestep t. Tt, Ue, and c are the reset gate, update gate, and candidate state, respectively. O and denote element-wise multiplication and feature concatenation. The operator kę applies graph convolution over graph G defined as:
Z xg θ
K Ā ZW₁
where Z R.h the input of graph convolution operation. À E NxN denotes the topology of graph G and W3 IR Kxhx is Chebyshev polynomial weights to order K.
5

---

The proposed model follows an encoder- architecture built by stacking multiple GCRU cells. Each input consists current observation sequence X and its aligned historical anchor X", both enriched with input embeddings, node embeddings and time-of-day embeddings [77, 63, 49] to better capture node-specific semantics and periodic temporal patterns. As described in Equation (2), both sequences are processed in parallel through the encoder. Specifically, the input is passed through the GCRU encoder to get the hidden state at the last timestep 1ff H₁, Similarly, we can get 11" from X Each of these representations is then refined through query-protogy
attention in Equation (3), weighted sum of all prototypes: V = E o.,
where the enhanced representation is computed This process yields augmented representations V and Va for H and H respectively. We then concatenate these four components and pass them through linear projection to generate the adaptive adjacency matrix Á, enabling the decoder adaptively model spatial interactions conditioned on the encoded spatio-temporal context. This process can be formulated as:
H' = W [ V H V*] +
(8)
A = Softmax(ReLL(FF H)
where W and b are learnable parameters of linear projection. Finally, a GCRU decoder takes the adaptive graph and the hidden states generated by encoder make the future prediction sequence.
The primary objective of ST-SSDL is to achieve accurate spatio forecasting, supervised by the Mean Absolute Error (MAE) between predictions and ground truth values. To enhance deviation- aware modeling, the model integrates the contrastive loss LCo and deviation loss Dev introduced in Section 4. together with the MAE loss. Finally, the overall training objective is defined as:
C = LAAE + ACon LCon + ADe
LDvv
where ACon and \Dev are hyper-parameter that control the contributions of the contrastive and deviation terms. This joint loss encourages accurate forecasting while adaptively capturing spatio- temporal deviations to improve robustness across diverse scenarios.
Complexity Analysis. We analyze the complexity of ST-SSDL by examining its two main com- ponents: the SSDL method and the GCRU backbone. In SSDL, query wneey-prototype attention over M prototypes yields complexity of O(N For the GCRU backbone, each layer applies K-order Chebyshev graph convolution and recurrent update, resulting in O(K N4?) per timestep. With L GCRU layers and total sequence length T + T', the overall cost of GCRU backbone is OLLKNN?( + T")). Thus, the total complexity of ST-SSDL is ONN + L. 2 + T')).
5
Experiment
5.1 Experimental Setup
Datasets. We evaluate our ST-SSDL on 6 widely used spatio-temporal forecasting benchmarks: METRLA. PEMSBAY, PEMSD7(M) [44, 83] with traffic speed data, and PEMS04, PEMS07, PEMS08 [67] with traffic flow data. All datasets have temporal resolution of 5 minutes per timestep (12 steps per hour). More statistics for these benchmarks are provided in Table 1. Following prior work, METRLA and PEMSBAY adopt a 70%: 10%:20% split [77] for training, validation, and testing, respectively, while PEMSD7(M), PEMS04, PEMS07, and PEMS08 use 60%:20%%00% [24, 49].
Settings. We conducted a hyper- parameter search for each bench- mark to ensure optimal model per- formance. The architecture con- sists of a I-layer encoder and 1- layer decoder (i.e., L=2), with hid- den dimensions h set to 128, 64, or 32, depending on the dataset. We use M=20 prototypes with di- mension d=64. Detailed configu- rations are provided in our public code. The input and prediction horizons are both set to hour (T=T'=12 timesteps). For optimization, we employ the Adam optimizer with an initial learning rate of 0.001. Model evaluation is based on three key metrics: Mean Absolute Error (MAE), Root Mean
Table 1: Summary of our used spatio-temporal benchmarks.
Dataset
#Sensors (N)
#Timesteps
Time Range
METRLA PEMSBAY PEMSD7(M)
207
34,272
03/2012 - 06/2012
325
52,116
01/2017 -0620017
228
12,672
05/2012- -06/2012
PEMS04
307
16,992
01/2018 02/2018
PEMS07
883
28,224
05/2017 - 08/2017
PEMS08
170
17,856
07/2016 08/2016

---

Table 2: Performance on METRLA, PEMSBAY, (EMSD7(M), and PEMS04/077008 benchmarks.
Dutanct
Manicc
H0
GBC
STOCN
GWNe DCRNN
MTTCNN
AGCRN
CTTS
STNomm
STID
ST.RAA
MeraCRN
STON
ST-SSDL
15 mio
Siep
RMSE
14.21
6.00
5.29
5.16
5.16
2.24
5.20
5.57
5.62
3.80
5.84
5/12
MAPE
16.72% 75725
8.14%
7.10A
5,803
6.99%
0.8915
7
7.125
7.40%
7.755
T.015
6.08%
7.7415
6.62%
METRLA
MAR
5.50
5.7
5.15
1.12
3/8
3,05
1.20
N.14
3.18
X.19
3.29
3.21
2.6
RMSE
MAPI.
16.72%
10.7116
8.42%6
8.47%:
K.105
9/002
8.0255
8.47%
9.395
9.2256
8.18%
9.3y6
7.92%
Step
RMSE
MAR
14.21
5.00
4.88
9.75
7.43
5.60
7,47
1.54
3.51
7.38
7.21
1.47
7.45
3.59
7.44
3.90
7.51
3.57
7.55
3.85
7.99
3.68
7.31
3:46
7.80
3.57
8.37
7.17
16.71%
10.328
Step 15 min
RMSE
MAE
7.03
3.05
3.15
2.85
1.36
2.76
2.73
2.50
1.33
2.85
2.02
1.37
2.52
1.33
2.79
2.88
1.37
2.71
2.56
1.36
2.45
1.26
MAPE
6.851
3.01%
2,809
2.7346
2.71%
2.81%6
2.91%
2.856
2.70%
2.75%
2,9046
2.67%
2.91%5
2.60%
PEMSRAY
MAE
1.70
165
1.72
1.65
1.90
1.57
RMSE
7.00
384
375
3.73
3.82
3.86
3.7
3.51
3.94
3.59
min
MAPE
6.6466
4.45%
3.79%
3.715
3.73%
3,7516
3.51%
3889
3.6616
3.75%
3.8116
3,59%
5.819
3,89%
Step 12
MAE
3.05
2.70
2.12
1.97
1.99
1.95
1.94
2.06
1.22
1.91
2.00
1.39
1.2
L86
RMSE
4.63
4.34
MAPE
63216
6:229
4.72%
4.71%
4,6216
4.355
1.40%5
4.54%
4.35%
Step
MAR
5.01
2:31
2.19
2.24
2.50
2.14
2.20
2.17
2
FEXS077:0
15 min
MAPE
RMSE
13.11
9.56
5.405
4.43
5.199
6.14
5.18%
4.31
5.00/
4.03
5,095
4.10
5,163
4.17
4.27
5.11%
4.00
5
4,00
4.17
4.75%
3.88
4.17
4.74%
333
MAF
2776
2.76
2.7
Stop
30 min
MAPK
RMSH
13.115
9.58
6.43
6.89%
5.50
7.54%
0,37
6.9%
5:43
6.77%
5.62
7.765
5.69
5.90
6.88%
5.55
6.815
5,39
6.9415
5.50
6.59%
5:34
7.108
5.01
6.41%
5.18
MAE
5.02
4.62
3.30
4.31
3:35
3.33
3
4.28
3.22
5.19
3.50
3.27
3.26
3J16
9.50
0.36
MAPF
12.32%
12.21%
8.543
11:109
3.34%
559
9,029
11.42%
8.48%
8.405
3.50%
7.96%
DEMSOH
MAE
42.35
25.55
19:57
19.63
18:53
19.17
19.35
20,96
18.96
15.38
19.06
18.09
18,40
18.08
Averige
RMSE
61.56
39.71
31.38
31,26
29-92
31,70
31.25
32.95
30.98
29:95
31.02
30.54
30,22
29.64
MAPE
29.92%
13.59%
14.56%
12.69%
12.52%
12.50%
12.36%
MAE
09.24
21.74
21.16
23.15
20:50
20.74
20,05
Anzrage
RMSH
21.34
42.78
35.27
59.14
35:43
54.06
34.00
35. 10
54.66
$2.79
34.05
32.71
34.77
32.81
MAPF
23:788
11.58%
9.34/
9.02%
5.61%
9.00%
8.74%
9.3854
8.75%
8.90%
8.77%
5.59%
10.98%
8.11%
PEMSIO
MAE
34,56
19.36
15.08
15.22
14.80
15.18
15.32
16.49
15.41
14.21
15.41
14:33
14.25
13,86
RMSE
$0.45
34.17
24.77
24.82
24.39
23.10
MAPE
21.63%
12.43%
10,60%%
10.21%
9.21%
10.30%
10.03%
10.54%
9.76%
9.27%
9.94%
10.67%
9.17%
Square Error (RMSE), and Mean Absolute Percentage Error (MAPE). All following experiments conducted on NVIDIA RTX 5000 Ada GPUs.
Baselines. We evaluate the performance of our model against comprehensive set of widely used baselines: statistical method HI [10], univariate time series model GRU [7], and spatio-temporal mod- els including STGCN [83], DCRNN [44], Graph WaveNet [77), AGCRN [2), GTS [62], STNorm [14].
STID [63], ST-WA [8], MegaCRN [35], and STDN [4].
5.2 Performance Evaluation
The comparison results for spatio-temporal forecasting performance are presented in Table 2. In line with prior research, we report the performance at 3. 6, and 12 timesteps for the METRLA, PEMSBAY, and PEMSD7(M) datasets, and the average performance across all 12 predicted timesteps for the PEMS04, PEMS07, and PEMSO8 datasets. As a result, our proposed model, ST-SSDL, consistently achieves sate-oof-thearr performance across all evaluation metries and datasets with a simple GCRU backhone. This superior performance underscores the effectiveness of our Self-Supervised Deviation Learning approach, which improves the sensitivity and adaptability to varying levels of deviation in complex spatio-temporal scenarios, leading to robust forecasting.
5.3 Ablation Study
In this section, we perform detailed ablation study to systematically assess the contributions of the critical components of ST-SSDL. To this end, we construct several model variants: (1) w/o CCon: removes the contrastive loss, which is designed
ensure the distinction of prototypes. (2) w/o D removes the deviation loss to regularize the consistency between continuous input space and discretized hidden space. (3) w/o Lcon. Dev: removes both the contrastive and deviation losses. (4) w/o SSDL: removes the whole SSDL strategy, downgrading the model to simple GCRU without any historical information as input. The results in Table 3 show that both Lcon and LDvv are necessary for SSDL. Furthermore, removing both of them leads to much worse performance, while removing the whole SSDL results the poorest performance as we expected. Besides, we create a naive version of SSDL by removing latent space discretization with prototypes. Accordingly, LCop is removed and
7

---

the deviation loss is downgraded to Dev = cos ( cos(H", H")), where the distances are measured by cosine similarity eliminate the impact of scale. The results confirm that it is hard to learn deviations in the continuous latent space without discretization. All these validate that ST-SSDL complete and indivisible achieve superior spatio-temporal forecasting performance.
Table 3: 12 steps average prediction RMSE of the ablations.
Model
METRLA
PEMSBAY
PEMSD7(M)
PEMS04
PEMS07
PEMS08
w/o LCon
6.16
3.65
5.18
29.81
33.16
23.31
w/o LDv
3.62
5.14
29.84
32.68
24.09
wio Lcon w/o Liv
6.06
3.67
5.16
29.91
32.81
24.13
w/o SSDL
6.09
3.54
5.22
30.01
32,90
24.96
Naive SSDL
6.07
3.52
5.23
29.90
33.59
23.62
ST-SSDL
6.02
3.51
5.11
29.64
32.51
23.10
5.4
Efficiency Study
We compare the computational efficiency of ST-SSDL with 5 typical baseline models representing different architectures, including GCN, GCRU, and Transformer-based designs. Table 4 reports the number of parameters, training time (per epoch), and inference time on the METRLA, PEMS07, and PEMS08 datasets. For consistency and fairess, we set batch size B=16 normalize the results. As shown in the table, ST-SSDL has the fewest number of parameters. For example, on PEMS08, it has 100K parameters, which about 66% of the second-lightest model AGCRN (150K), and only 1.7% of STDN (5876K). However, its runtime efficiency is constrained by the iterative GCRU backbone, which shows trade-off.
Table Efficiency comparison on METRLA, PEMS07, and PEMS08 datasets.
METRLA (N 207)
PEMS07 (N
883)
PEMS08 (N = 170)
Model
#Params
Train
Infer
#Params
Train
Infer
#Params
Train
Infer
MTGNN
405K
62.95s
1.24s
1368K
212.79s
5.13s
353K
28. 28.48s
0.51s
AGCRN
752K
87.15s
2.68s
755K
342.73s
9.89s
150K
38.44s
1.18s
STNorm
224K
64.80s
0.88s
570K
279.37s
4.68s
205K
28.06s
0.58s
ST-WA
375K
113.90s
7.41s
786K
678.93s
60.99s
353K
47.88s
3.80s
STDN
5971K
459.43s
10.39s
4480K
693.32s
32.56s
5876K
200.60s
5.71s
ST-SSDL
498K
71.84s
8.65s
379K
870.12s
217.80s
100K
65.81s
7.97s
5.5
Hyperparameter Study
In this section, we analyze the impacts of two critical hyper--aramettrs: the number of prototypes M and their dimension d. Figure 3 shows the average RMSE of predictions on METRLA and PEMSBAY datasets when varying M from 5 to 25 and d from 16 to 80. The figure reveals that our model remains fairly stable across these hyper-paramens generally gives good results. However, a very small M or d is insufficient to discretize the deviations ppatio-tempore patterns, while large values can introduce overfitting.
settings. Using 20 prototypes with a dimension of 64
6.16
3.75
RMSE
6.12
RMSE
3.70
6.10
3.65
3.60
6.06
3.55
6.04
5
10
15
20
25
16
32
18
61
50
3.50
5
10
15
20
25
16
32
18
61
30
M
M
(a) METRLA
(b) PEMSBAY
Figure 3: 12 steps average RMSE w.r.t. #prototypes M and dimension d.

---

Low Deviation
(b) Medium Deviation
(c) High Deviation
141
00
90
Tailiic Spoed
Tralfic Speed
-0
Tmfficc
-0
40
23
20
08:25
09:05
09:45
15:50
History Anchor
Ground Truth (Past Steps and Future Steps)
Prediction
Charces Quary
Currert Quayy
Canes Query Anchtr Query Bosetyes
Asshor Query
Ander Quary
Prosaggss
Parotegi
Figure 4: Visualization of predictions and association under various deviation levels.
5.6 Case Study
Performance across Deviation Levels. To assess how ST-SSDL responds to varying levels of deviation, we visualize its forecasting behavior alongside the latent query-prototype association under low, medium, and high deviation scenarios in Figure 4. The gray shaded area on the left represents the past inputs, while the blue shaded part on the right denotes the future values. In the low deviation scenario, where the current sequence remains closely aligned with its historical anchor, ST-SSDL accurately follows the ground truth. The corresponding queries Q and Q" are assigned to the same prototype, forming a compact structure in latent space. This behavior reflects the model's ability to preserve semantic consistency when deviation is minimal. In the medium deviation case, ST-SSDL routes Q' and Q" to different yet nearby prototypes. Although the current input deviates from its historical anchor, the prediction closely follows the ground truth. The proximity between their assigned prototypes indicates that the model preserves latent relational consistency. Under high deviation, the current input diverges substantially from the historical anchor. Despite this shift, the prediction remains aligned with the ground truth. ST-SSDL. adapts by mapping Q' and Q" to well-separated prototypes, effectively capturing the altered spatio-temporal context in the latent space. These visualizations confirm that our method adaptively adjusts prototype associations and prediction outputs in accordance with the degree of deviation, validating its effectiveness in modeling dynamic deviation across spatio-temporal data.
Prototype and Query in Latent Space. Beyond in- dividual cases, we examine the global organization along with the query associations of prototypes to further validate the latent space discretization by our prototypes. Figure illustrates PCA projection of prototypes, with four hundred sampled queries positioned in the same projected space based on their relative distances and colored by their assigned positive prototypes. For clarity, we visualize the seven most frequently selected prototypes. The visu- alization demonstrates that queries are grouped into compact cluster centered around their assigned pro- totypes, with clear separation between clusters. This visualization suggests that the latent space has been effectively discretized into semantically coherent re- gions. Moreover, such arrangement also reflects the
Queries
Protatypes
18
Figure 5: Visualization of prototype and query distribution in latent space.

---

2
hicoppe ProMose Preabps
8
Traftic Spored
Traffic Speed
8
Piltyp Po0typp
ProM
Prokeape 9
Procotyp 16
ProMip
Proleape
Prototipo 18
10
Time Step
Time Step
(a) Rapidly decreasing (rush hour or traffic accident).
(b) Flat or small fluctuation (stable traffic condition).
Proobpu
Prccappe
de Prcabpss
Traffic Speed
Traffic Speed
Protobos
Posthbo
Time Step
Time Step
(c) Gradually decreasing (progressive slowdown).
(d) Increasing (post-rush recovery).
Figure 6: Prototype patterns recovered in the physical space on METRLA dataset. The shaded area indicates the standard deviation.
model's ability to capture diverse spatio-temporal patterns and organize them into representative prototypes, thereby supporting robust deviation modeling across varying input conditions.
Prototype in Physical Space. While Figure visualizes the learned prototypes in latent space, we can also compute each prototype in physical space by collecting input sequences assigned it, i.e., for prototype Pk. we collect input X whose query Q is assigned to Ps. Then we compute the pointwise average of these sequences to derive each prototype Pt. Figure 6 visualizes these patterns on METRLA and groups similar patterns together. Figure 6a exhibits rapidly decreasing pattern, suggesting underlying peak-hour dynamics or incident-induced speed drops. Figure 6b displays flat patterns with small fluctuations. The abundance of prototypes in this category reflects stable operating states across diverse road segments. Figure 6c shows gradually decreasing modes with smooth decline over 12 steps, consistent with progressive slowdowns in traffic speed. Figure 6d presents increasing paradigms aligned with post-peak recovery where speed returns toward free-flow conditions. These physical space diagrams summarize how the learned prototypes reveal recurrent traffic patterns in the real world.
Conclusion
This paper introduces ST-SSDL, self-supervised framework that enhances spatio- apatio-temporal forecast- ing by explicitly modeling latent deviations. By using historical averages as context-aware anchors and discretizing the deviation space with learnable prototypes, ST-SSDL offers structured approach to deviation-aware representation learning. The latent space is jointly optimized by contrastive and deviation losses, while avoiding reliance on explicit labels. Experiments on benchmarks confirm its superior performance, and visualizations illustrate its ability to adapt under varying levels of deviation. Future directions include extending this framework hierarchical prototype structures further assess their adaptability and robustness.
7
Acknowledgement
This work was supported by JST CREST Grant Number JPMJCR21M2, JSPS KAKENHI Grant Number JP24K02996, JST BOOST Grant Number JPMJBS2418, and Toyota Motor Corporation.
10

---

References
[1] Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. wav2vec 2.0: A framework for self-supervised learning of speech representations, Advances in neural information processing systems, 33: 33:12449-12461 2020.
[2] Lei Bai, Lina Yao, Can Li, Xianzhi Wang, and Can Wang. Adaptive graph convolutional recurrent network for traffic forecasting. Advances in neural information processing systems, 33:17804-78815 2020.
[3] Defu Cao. Yujing Wang, Juanyong Duan, Ce Zhang, Xia Zhu, Congrui Huang, Yunhai Tong, Bixiong Xu, Jing Bai, Jie Tong, et al. Spectral temporal graph neural network for multivariate time-series forecasting. Advances in neural information processing systems, 33:17766-17778. 2020.
[4] Lingxiao Cao, Bin Wang, Guiyuan Jiang, Yanwei Yu, and Junyu Dong. Spatiotemporal-awara trend-seasonality decomposition network for traffic flow forecasting. Proceedings of the AAAI Conference on Artificial Intelligence, 39(1)): I): 1463-114777 Apr. 2025.
[5] Jialin Chen, Jan Eric Lenssen, Aosong Feng, Weihua Hu, Matthias Fey, Leandros Tassiulas, Jure Leskovec, and Rex Ying. From similarity to superiority: Channel clustering for time series forecasting. Advances in Neural Information Processing Systems, 37:1 17.133355-136665 2024.
[6] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference machine learning, pages 1597-1607. PmLR, 2020.
[7] Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv: 1:X1:::412.3555, 2014.
[8] Razvan-Gabin Gabriel Cirstea, Bin Yang, Chenjuan Guo, Tung Kieu, and Shirui Pan. Towards spatio- temporal aware traffic time series forecasting. In 2022 IEEE 38th International Conference on Data Engineering (ICDE), pages 2900-2913. IEEE, 2022.
[9] Kevin Clark; Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. ELECTRA: Pre-training text encoders as discriminators rather than generators. In ICLR, 2020.
[10] Yue Cui, Jiandong Xie, and Kai Zheng. Historical inertia: A neglected but powerful baseline for long sequence time-series forecasting. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pages 2965 2969, 2021.
[11] Jiewen Deng, Jinliang Deng, Renhe Jiang, and Xuan Song. Learning gaussian mixture represen- tations for tensor time series forecasting. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI-23, pages 2077-2085 2023.
[12] Jiewen Deng. Jinliang Deng, Du Yin, Renhe Jiang, and Xuan Song. Tts-norm: Forecasting tensor time series via multi-way normalization. ACM Transactions on Knowledge Discovery from Data, pages 1-25, 2023.
[13] Jiewen Deng, Renhe Jiang, Jiaqi Zhang, and Xuan Song. Multi-modality spatio-temporal forecasting via self-supervised learning. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, pages 2018-2026. 2024.
[14] Jinliang Deng, Xiusi Chen, Renhe Jiang, Xuan Song, and Ivor W Tsang. St-norm: Spatial and temporal normalization for multi-variate time series forecasting. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, pages 269-278. 2021.
[15] Jinliang Deng, Xiusi Chen, Renhe Jiang, Xuan Song, and Ivor W Tsang. A muultiview multi-task learning framework for multi-variate time series forecasting. IEEE Transactions on Knowledge and Data Engineering, pages 7665-7680. 2022.
[16] Jinliang Deng, Xiusi Chen, Renhe Jiang, Du Yin, Yi Yang. Xuan Song, and Ivor W Tsang. Disentangling structured components: Towards adaptive, interpretable and scalable time series forecasting. IEEE Transactions on Knowledge and Data Engineering, pages 3783-3800, 2024.
11

---

[17] Zheng Dong, Renhe Jiang, Haotian Gao. Hangchen Liu, Jinliang Deng, Qingsong Wen, and Xuan Song. beerrogeneity-nffommn meta-parameter learning for spatiotemporal time series forecasting. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, page 631-641, 2024.
[18] Jin Fan, Wenehao Weng, Hao Tian, Huifeng Wu, Fu Zhu, and Jia Wu. Rgdan; A random graph diffusion attention network for traffic prediction. Neural networks, 172: 172:106093, 2024.
[19] Yuchen Fang, Yuxuan Liang, Bo Hui, Zezhi Shao, Liwei Deng, Xu Liu, Xinke Jiang, and Kai Zheng, Efficient large-scale traffic forecasting with transformers: A spatial data management perspective. Proceedings of the 31 ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.I, page 307-317, 2025.
[20] Siyuan Feng, Shuqing Wei, Junbo Zhang, Yexin Li, Jintao Ke, Gaode Chen, Yu Zheng, and Hai Yang. A macro- micro ppatio-temporal neural network for traffic prediction. Transportation Research Part C: Emerging Technologies, page 104331, 2023.
[21] Haotian Gao, Renhe Jiang, Zheng Dong, Jinliang Deng, Yuxin Ma, and Xuan Song. Spatial- cemporal-decoupee masked pre- training for spatiotemporal forecasting. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, pages 3998 0998-4006,44 2024.
[22] Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, al. Bootstrap your own latent-a new approach to self- self-supervised learning, Advances in neural information processing systems, 33:21271-21288 2020.
[23] Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. In First Conference on Language Modeling, 2024.
[24] Shengnan Guo, Youfang Lin, Ning Feng, Chao Song. and Huaiyu Wan. Attention based spatial- temporal graph convolutional networks for traffic flow forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 922-929, 2019.
[25] Marah Halawa, Florian Blume, Pia Bideau, Martin Maier, Rasha Abdel Rahman, and Olaf Hellwich. Multi-task multi-modal self-supervised learning for facial expression recognition. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 4604-4614, 4. IEEE, 2024.
[26] James D Hamilton and Raul Susmel. Autoregressive conditional heteroskedasticity and changes in regime. Journal of econometrics, 64(1-2):307-333, 1994.
[27] Kaiming Hc, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 9729-9738, 2020.
[28] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000-16009, 2022.
[29] Sicheng He, Junzhong Ji. and Minglong Lei. Decomposed spatio-temporal mamba for long- term traffic prediction. Proceedings of the AAAI Conference on Artificial Intelligence, 39(11): 11772-11780, Apr. 2025.
[30] Shujie Hu, Long Zhou, Shujie Liu, Sanyuan Chen, Lingwei Meng, Hongkun Hao, Jing Pan, Xunying Liu, Jinyu Li, Sunit Sivasankaran, al. Wavllm: Towards robust and adaptive speech large language model. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 4552 4572, 2024.
[31] Jiahao Ji, Jingyuan Wang, Chao Huang, Junjie Wu, Boren Xu, Zhenhe Wa, Junbo Zhang, and Yu Zheng. Spatio-temporal self- supe-vipe learning for traffic flow prediction. Proceedings of the AAAI Conference Artificial Intelligence, volume 37, pages 4356-4364, 2023.
12

---

[32] Junzhong Ji, Fan Yu, and Minglong Lei. Self-supervised spatiotemporal graph neural networks with dii-iillatiiinn for traffic prediction. IEEE Transactions on Intelligent Transportation Systems, 24(2):1500-159) 2022.
[33] Jiawei Jiang, Chengkai Han, Wayne Xin Zhao, and Jingyuan Wang. Pdformer: Propagation delay-aware dynamic long-range transformer for traffic flow prediction. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 4365-4373, 2023.
[34] Renhe Jiang. Du Yin, Zhaonan Wang, Yizhuo Wang, Jiewen Deng, Hangchen Liu, Zekun Cai, Jinliang Deng, Xuan Song, and Ryosuke Shibasaki. DI-traff:: Survey and benchmark of deep learning models for urban traffic prediction. In Proceedings of the 30th ACM international conference on information knowledge management, pages 4515 4525, 2021.
[35] Renhe Jiang, Zhaonan Wang. Jiawei Yong, Puneet Jeph, Quanjun Chen, Yasumasa Kobayashi, Xuan Song. Shintaro Fukushima, and Toyotaro Suzumura. teapooltemporal meta-graph learning for traffic forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 8088-8086, 2023.
[36] Di Jin, Jiayi Shi, Rui Wang, Yawen Li, Yuxiao Huang, and Yu-Bin Yang. Trafformer: Unify time and space in traffic prediction. Proceedings of the AAAI Conference on Artificial Intelligence, 37(7):81 14-8122, Jun. 2023.
[37] Jacob Devlin Ming-We King-Wei Chang Kenton and Lee Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLL,, pages 4171-4186, 2019.
[38] Hyunwook Lee and Sungahn Ko. Testam: time-enhanced spatio-temporal attention model with mixture of experts. In The Twelfth International Conference on Learning Representations, 2024.
[39] Dongyuan Li, Shiyin Tan, Ying Zhang, Ming Jin, Shirui Pan, Manabu Okumura, and Renhe Jiang. Dyg-mambat Continuous state space modeling on dynamic graphs. arXiv preprint arXiv:2408.000 06966, 2024.
[40] Duo Li and Joan Lasenby. Spatiotemporal attention creentionbbased graph convolution network for segment-level traffic prediction. IEEE Transactions Intelligent Transportation Systems, 23 (7)88337-834,, 2021.
[41] Fuxian Li, Jie Feng, Huan Yan, Guangyin Jin, Fan Yang, Funing Sun, Depeng Jin, and Yong Li. Dynamic graph convolutional recurrent network for traffic prediction: Benchmark and solution. ACM Transactions Knowledge Discovery from Data, 17(1): 1-21, 2023.
[42] Fuxian Li, Huan Yan, Hongjie Sui, Deng Wang, Fan Zuo, Yue Liu, Yong Li, and Depeng Jin. Periodic shift and event-aware spatio-temporal graph convolutional network for traffic congestion prediction. In Proceedings of the 31st ACM International Conference on Advances in Geographic Information Systems, pages 1-10, 2023.
[43] Lincan Li, Hanchen Wang, Wenjie Zhang, and Adelle Coster. Stg- Stg-mamba: Opatial-tempores graph learning via selective state space model. arXiv preprint 2rXiv:2403.124188 2024.
[44] Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Diffusion convolutional recurrent neural net- work: Data-driven traffic forecasting. In International Conference on Learning Representations, 2018.
[45] Yinghao Aaron Li, Cong Han, Vinay Raghavan, Gavin Mischler, and Nima Mesgarani. Styletts Towards human-level text-to-speech through style diffusion and adversarial training with large speech language models. Advances in Neural Information P Processing Systems, 36: 19594-19621, 2023.
[46] Yuxin Li, Wenchao Chen, Bo Chen, Dongsheng Wang, Long Tian, and Mingyuan Zhou. Prototype-oriented unsupervised anomaly detection for multivariate time series. In International Conference Machine Learning, pages 19407-19424. PMLR, 2023.
13

---

[47] Zetao Li, Zheng Hu, Peng Han, Yu Gu, and Shimin Cai. Ssl-stmformer self supe--su learning spatio-temporal entanglement transformer for traffic flow prediction. Proceedings of the AAAI Conference Artificial Intelligence, 39(11): (211):21100-21380 Apr. 2025.
[48] Gang Liu, Silu He, Xing Han, Qinyao Luo, Ronghua Du, Xinsha Fu, and Ling Zhao. Self- supervised spatiotemporal masking strategy- -based models for traffic flow forecasting. Symmetry, 15(11); 2023.
[49] Hangehen Liu, Zheng Dong, Renhe Jiang, Jiewen Deng, Jinliang Deng, Quanjun Chen, and Xuan Song. Spatio-temporal adaptive embedding makes vanilla transformer sota for traffic forecasting. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pages 4125 4129, 2023.
[50] Jiexi Liu and Songcan Chen. Timesurl: Self-supervised contrastive learning for universal time series representation learning. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages
2024.
[51] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint 1rXiv: 2019.
[52] Tengfei Lyu, Weijia Zhang, Jinliang Deng, and Hao Liu. Autostf: Decoupled neural architecture search for cost-effective automated spatio-temporal forecasting. In Proceedings of the 31 ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.I, page 985-996, 2025.
[53] Meike Nauta, Ron Van Bree, and Christin Seifert. Neural prototype trees for interpretable fine-gras the-grained image recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14933-14943, 2021.
[54] Lugang Nie, Benxiong Huang, and Lai Tu. Traffic prediction by integrating multi-cycle features and spatio-temporal correlations. In 2024 IEEE Cyber Science and Technology Congress (CyberSeeifech) pages 1-8. IEEE, 2024.
[55] Bei Pan, Ugur Demiryurek, and Cyrus Shahabi. Utilizing real-world transportation data for accurate traffic prediction. In 2012 ieee 12th international conference on data mining, pages 595-604. IEEE, 2012.
[56] Zheyi Pan, Yuxuan Liang, Weifeng Wang, Yong Yu, Yu Zheng, and Junbo Zhang. Urban traffic prediction from spatio-temporal data using deep meta learning. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1730- 2019.
[57] Zheyi Pan, Songyu Ke, Xiaodu Yang, Yuxuan Liang, Yong Yu, Junbo Zhang, and Yu Zheng. Autostg: Neural architecture search for predictions of spatio-temporal graph. In Proceedings of the Web Conference 2021, pages 1846-1855, 2021.
[58] Hao Peng. Hongfei Wang, Bowen Du, Md Zakirul Alam Bhuiyan, Hongyuan Ma, Jianwei Liu, Lihong Wang, Zeyu Yang. Linfeng Du, Senzhang Wang, et al. Spatial temporal incidence dynamic graph neural networks for traffic flow forecasting. Information Sciences, 521:277-290. 2020.
[59] Hao Qu, Yongshun Gong, Meng Chen, Junbo Zhang, Yu Zheng, and Yilong Yin. Forecasting fine-grained urban flows via spatio-temporal contrastive self-supervision IEEE Transactions on Knowledge and Data Engineering, 35(8):8008-800 2022.
[60] Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 815-823, 2015.
[61] Zineb Senane, Lele Cao, Valentin Leonhard Buchner, Yusuke Tashiro, Lei You, Pawel Andrzej Herman, Mats Nordahl, Ruibo Tu, and Vilhelm von Ehrenheim. Self-supervised learning of time series representation via diffusion process and impuuaaion-ineerpoaaon---fee In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD '24, page 2560-2571, 2024.
mask.
14

---

[62] Chao Shang, Jie Chen, and Jinbo Bi. Discrete graph structure learning for forecasting multiple time series. In International Conference on Learning Representations, 2021.
[63] Zezhi Shao, Zhao Zhang, Fei Wang, Wei Wei, and Yongjun Xu. Spatial-temport identity: A simple yet effective baseline for multivariate time series forecasting. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pages 4454-4458, 2022.
[64] Zezhi Shao, Zhao Zhang, Fei Wang, and Yongjun Xu. Pre-training enhanced spatial-tempora graph neural network for multivariate time series forecasting. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1567-1577, 2022.
[65] Zezhi Shao, Zhao Zhang, Wei Wei, Fei Wang, Yongjun Xu, Xin Cao, and Christian S Jensen. Decoupled dynamic spatial-temporal graph neural network for traffic forecasting. Proceedings 2022.
of the VLDB Endowment, 15(11):2733-2
[66] Zhiqi Shao, Ze Wang. Xusheng Yao, Michael G.H. Bell, and Junbin Gao. St-mambasyne: Complement the power of mamba and transformer fusion for less computational cost in spa- tial-temporal traffic forecasting. Information Fusion, 1 117:102872, 2025.
[67] Chao Song, Youfang Lin, Shengnan Guo, and Huaiyu Wan. -patial-temporal synchronous network data forecasting.
graph convolutional networks: A new framework for In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 914-921, 2020.
[68] James H Stock and Mark W Watson. Vector autoregressions. Journal of Economic perspectives, 105, 2001.
[69] Yanshen Sun, Kaiqun Fu, and Chang-Tien Lu. Roadformer: Road-anchored adversarial dy- namic graph transformer for unlimited-range traffic incident impact prediction. In 2023 IEEE International Conference Big Data (BigData), pages 995-904. IEEE, 2023.
[70] Yulin Tian, Weiyi Wang, Shuqing Liu, Lu Wei, Yuchen Zhang, Tao Zhu, and Xuedong Yan. Mambatraffic: Contrastive traffic flow prediction with spatial-temporal state space models. IEEE Intelligent Transportation Systems Magazine, 2025.
[71] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Fukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[72] Dongjing Wang, Gangming Guo, Tianpei Ouyang, Dongjin Yu, Haiping Zhang, Bao Li, Rong Jiang, Guandong Xu, and Shuiguang Deng. A lightweight spatio-temporal neural network with sampling-buus based time series decomposition for traffic forecasting. IEEE Transactions on Intelligent Transportation Systems, 2025.
[73] Zepu Wang, Yugi Nie, Peng Sun, Nam H Nguyen, John Mulvey, and H Vincent Poor. St-mlp: cascaded spatio-temporal linear framework with channel-indeeenddee strategy for traffic forecasting, arXiv preprint arXiv:2308.077 077499968.0...... 2023.
[74] Zhaonan Wang, Renhe Jiang, Hao Xue, Flora D Salim, Xuan Song, and Ryosuke Shibasaki. Event-aware multimodal mobility nowcasting. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 4228 4236, 2022.
[75] Zhaonan Wang. Renhe Jiang. Hao Xue, Flora D Salim, Xuan Song, Ryosuke Shibasaki, Wei Hu, and Shaowen Wang. Learning spatio-temporal dynamics on mobility networks for adaptation to open- open-world events. Artificial Intelligence, page 104120, 2024.
[76] Xian Wu, Chao Huang, Chuxu Zhang, and Nitesh V Chawla. Hierarchically structured trans- former networks for fine-grained spatial event forecasting. In Proceedings of the web conference 2020.
2020, pages
[77] Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, and Chengqi Zhang. Graph wavenet for deep spatial-tempon graph modeling. In Proceedings of the 28th International Joint Conference Artificial Intelligence, pages 1907-1913, 2019.
15

---

[78] Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chengqi Zhang. Connecting the dots: Multivariate time series forecasting with graph neural networks. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pages 753-763, 2020.
[79] Qinge Xie, Tiancheng Guo, Yang Chen, Yu Xiao, Xin Wang, and Ben Y Zhao. Deep graph convolutional networks for incident-driven traffic speed prediction. In Proceedings of the 29th ACM international conference on information & knowledge management, pages 1665-1674, 2020.
[80] Zhenda Xie, Zheng Zhang, Yue Cao, Yutong Lin, Jianmin Bao, Zhuliang Yao, Qi Dai, and Han Hu. Simmim: A simple framework for masked image modeling. In Proceedings of the IEEF/CVE conference computer vision and patlern recognition, pages 9653-9663, 2022.
[81] Mingxing Xu, Wenrui Dai, Chummiao Liu, Xing Gao, Weiyao Lin, Guo-Jun Qi, and Hongkai Xiong. Spatial-tempore transformer networks for traffic flow forecasting. arXiv preprint arXiv:2001. 0,2908, 2020.
[82] Yiwen Ye, Yutong Xie, Jianpeng Zhang, Ziyang Chen, Qi Wu, and Yong Xia. Continual self-supervised learning: Towards universal multi-modal medical data representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11114-11 11124, 2024.
[83] Bing Yu, Haoteng Yin, and Zhanxing Zhu. Spatio-temporal graph convolutional networks: deep learning framework for traffic forecasting. In Proceedings of the 27th International Joint Conference on Artificial Intelligene, pages 3634-3640, 2018
[84] Yuan Yuan, Jingtao Ding, Jie Feng, Depeng Jin, and Yong Li. A universal pre-training and prompting framework for general urban spatio-temporal prediction. IEEE Transactions on Knowledge and Data Engineering, 2025.
[85] Qianru Zhang, Chao Huang, Lianghao Xia, Zheng Wang, Zhonghang Li, and Siuming Yiu. Au- tomated spatio-temporal graph contrastive learning. In Proceedings of the ACM Web Conference 2023, pages 295-305, 2023.
[86] Tongtong Zhang, Zhiyong Cui, Bingzhang Wang, Yilong Ren, Haiyang Yu, Pan Deng, and Yinhai Wang. Premixer: Mip-based pre-training enhanced mip-mixers for large-scale traffic forecasting. arXiv preprint 2rX.v:2412. 13607, 2024.
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction clearly describe the main contributions and scope of the paper, which focus on modeling deviation spatio-temporal forecasting task. The proposed ST-SSDL method aims to better handle the differences between current and historical patterns, challenge that is often ignored in previous work. The main ideas include using historical averages anchors, introducing learnable prototypes to represent common patterns, and applying two self-supervised losses to guide the learning deviation-awant representations. These ideas are explained in detail in the method section and supported by experiments on six real-world datasets, where ST-SSDL shows strong performance. Overall, the abstract and introduction give accurate and clear overview of what the paper achieves.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We have included the limitations in appendix.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best
17

---

judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification: This paper does not present any theoretical results claims; therefore, assumptions or formal proofs are necessary.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide the training and evaluation settings along with model hyper- parameters in our paper. We also ensure that the code and datasets are accessible.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
18

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: This paper provides code base sttps://ght giut........ com/ 1iimy-76660 ST SSDL in the abstract including the models and datasets used, where the ReadME file contains enough information to ensure reproducibility.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips cc/
jublic/guide /tggddee/ooeeuuuiiiee CoodeSubmissioooolicco for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/ //nips wwtppbccc//acccccccccccccccc wwwtiic/gg/e////eeee /ttgg/ddd///oeuummiiiiieee sondesuubisssioooolii
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permirted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: These details have been described in the "Experiment" sections and are sufficient to understand our results.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification:
We provide the results of multiple runs on OpenReview.
Guidelines:
19

---

The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The type and number of GPUs used in the experiments, as well as the compu- tation time, included the "Experimental Setup" and "Efficiency Study" sections.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: / /nt.risr.r..p//
Answer: [Yes]
Justification: Our research fully follows to the NeurIPS Code of Ethics. We use publicly available datasets that do not involve any ethical concerns.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
20

---

Justification:
We have included the discussion of broader impacts in appendix.
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning conrectly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Not applicable- our research does not involve models data with high risk misuse.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Yes, all external assets used in work are properly credited, and we have adhered to their respective licenses and terms use as specified by the original creators.
Guidelines:
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
21

---

The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors are encouraged reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: No new assets are introduced in this paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
This research does not involve crowdsourcing or human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
This research does not involve crowdsourcing or human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
22

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core methodological development in this research does not incorporate LLMs in any significant, novel, unconventional manner.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https: / //nerrurps.... cc/Conferrencss/2255 for what should or should not be described.
23

---

Symbol A Appendix
A.1 Broader Impact
ST-SSDL. has the potential to improve decision-making in domains such as traffic management, urban planning, and environmental monitoring, where accurate predictions
lead to reduced congestion, better resource allocation, and improved public safety. By introducing a self-supervised mechanism for modeling deviations, ST-SSDL further reduces reliance on labeled data and can adapt to changing conditions more robustly. However, like other data-driven forecasting systems, ST-SSDL relies on historical sensor data, which may reflect biases in urban infrastructure deployment. such biases are not addressed, models may perform unevenly across regions, potentially reinforcing existing disparities.
A.2 Summary of Notation
For reference, Table 5 summarizes the key notations and their descriptions in this paper.
Table 5: Notation Table
Description
Number of nodes
C h
Number of input channels
Dimension of latent representations H
d
Dimension of query Q and prototypes P
T,
Length of input and output sequences, respectively Input sequence at current timestep Timestamp-aligned historical anchor sequence Segmented training data by week Weekly historical anchor (averaged) Encoded latent states of current and historical input Query vectors derived from H and H Learnable prototype vectors in latent space Attention score between query and i-th prototype Attention-weighted representation from prototype pool Positive and Negative prototypes for Q Positive and Negative prototypes for Q"
X E RTXNXC
X RTXNXC
XW E RSSTT XNxC
X E RT xNxC
H" ERNKA
0,Qº Rd
(PI,..., PM} RRxx
V
Pe, Ne
Pa, .N
Ā NNN
Adaptive graph structure computed from latent states
H'
Concatenated augmented hidden representation
Te, Ut
Reset and update gates GCRU Candidate hidden state in GCRU
C
MMAE
Mean A Absolute Error loss for prediction accuracy Contrastive loss and deviation loss
LCous Lie
ACon ADev
Loss weighting hyperparameters
L
Number of GCRU layers
A.3 Detailed Dataset Description
Detailed description of the six benchmark datasets used in the experiments are provided Table The METRLA and PEMSBAY datasets provide traffic speed data collected from Los Angeles and the Bay Area, respectively. The PEMSD7(M), PEMS04, PEMS07, and PEMS08 datasets consist of traffic speed and flow records sourced from California's Performance Measurement System (PEMS)². managed by the California Department of Transportation and provides data from 2001 to the present, collected from approximately 40,000 individual detectors deployed across California's freeway network in major metropolitan areas. These datasets offer temporal resolution of minutes, aggregated from original 30-second raw measurements and distributed in esv format.
brief introduction to PEMS:
"https:///pee dot. ca. gov/
24

---

Table 6: Detailed dataset descriptions.
Dataset
#Sensors (N)
#Timesteps
Time Range
Frequency
Information
METRLA
207
34,272
03/2012 06/2012
5min
Traffic Speed
PEMSBAY
325
52,116
01/2017
1120017-0//001 06/2017
5min
Traffic Speed
PEMSD7(M)
228
12,672
05/2012 09/2012-06/2012
5min
Traffic Speed
PEMS04
307
16,992
01/2018 02/2018
5min
Traffic Flow
PEMS07
883
28,224
05/2017 08/2017
5min
Traffic Flow
PEMS08
170
17.856
07/2016 08/2016
5min
Traffic Flow
A.4
Complete Pseudocode of ST-SSDL
Algorithm 1 Oppaio-Tempont Time Series Forecasting with Self-Supe Deviation Learning
Require:
Current Input X TxxxNx;; corresponding historical anchor X RT imput length T; output length T"; number of variable N; number of prototypes M; dimension of latent representation /;; hidden dimension of query and prototypes d;
Ensure: Forecasted sequence X=, loss L
1: // Encode input and anchor
2: H" + GGRU (X")
H E E D. H E
3: H -GGRRUS (X*)
4: // Project into query space
5: Q + Liiner(H''
D O E Raxd
6: Q* - Linear )
D Q" E RNd
7: // Compute query-prototype attention score
" + softmax
/va)
b RRM a CRM
9:
softmax P/VV
10: Retrieve top-2 prototypes
11: p , < Top2(a"))
12: p .N' Top2(a")
13: // Compute contrastive loss
14: Lco. + -max(IV(Q")) - P 1
- -V 6,00)
15: // Compute deviation loss
dg + -Q-- |
17: d, + P
Lie +- (dv dolla
19: Decode with adaptive adjacency
20: V- + P. va + P
b V Exd E RNd H Exa AĀERNN > X E xxxxx
21: H' W v I +b
22: À + Softmax Softmax(ReLU((T H)
23: X RGR ((H", VJ, Á)
24: // Compute final loss
25: [MAE NT
EN Er X X
26: L LMAE Acoa AneeCnes
27: Return forecasting result
28: return X
A.5 Detailed Explanation of Lazy Mode
The lazy mode refers to a collapse phenomenon in SSDL, where all query representations become overly similar and are mapped to the same prototype. This undermines the discriminative power of the model and limits its ability to capture diverse deviation patterns. In ST-SSDL, this issue arises when the gradient from self-supervised losses directly flows through both the query and prototype representations, allowing the model to minimize objectives trivially without learning meaningful structure.
To address this, we apply the stop-gradient operator V(.) to the query terms in both the contrastive loss LCon and the deviation loss LDv. In the contrastive loss:
25

---

Ccon max (1ō(09)
1V(1)) + 6.0).
and in the deviation loss:
Dev -110- - PP p
the operator freezes the query during optimization, forcing the prototypes adapt around the fixed distribution of query representations. This prevents the model from collapsing into trivial solutions (e.g., assigning all queries a single prototype), which would otherwise minimize both loss terms without learning meaningful deviations.
By isolating the optimization prototypes only, this strategy ensures that the latent space remains diverse and structured. It enables the prototype space to reflect semantic spatio-temporal patterns and maintain sufficient granularity for deviation quantification. Without this design, the model tends to fall into a shortcut solution, hence entering the lazy mode and severely degrading performance.
Besides theoretical analysis, we also empirically verify this phenomenon: removing the stop-gradient operation leads to single prototype to be chosen by all queries.
A.6 Complete Efficiency Evaluation
We report the computational efficiency of ST-SSDL and all bascline models the six datasets in Table 7, including the number of parameters, training time per epoch, and inference time. All of them are tested on an Intel(R) Xeon(R) Silver 4314 CPU @ 2.40GHz, 320G RAM computing server, equipped with NVIDIA RTX 5000 Ada Generation graphics cards. Moreover, we also provide a detailed descriptions of each baseline method as follows:
GRU [7]: Gated Recurrent Unit, a type of recurrent neural network (RNN) that uses gating mechanisms (an update gate and a reset gate) to manage information flow making effective for sequence modeling tasks.
STGCN [83]: spatio-temporal graph convolutional network, a deep learning model for
traffic forecasting on spatio-temporal graphs based on graph convolution operation.
DCRNN [44]: Diffusion convolutional recurrent neural network, neural network for traffic forecasting using diffusion convolution and RNN.
GWNet [77]: GWNet neural network, a CaN-based deep learning model for traffic fore- casting using graph convolution layer and wavenet architecture.
MTGNN [78]: Multivariate Time Series Graph Neural Network, a general framework for multivariate time series forecasting that automatically learns uni-directed relations among variables through graph learning module, and captures spatial and temporal dependencies using novel mix-hop propagation and dilated inception layers, respectively.
AGCRN [2]: Adaptive graph convolutional recurrent network for traffic forecasting, learning node-specific patterns through node adaptive parameter learning module and data adaptive graph generation module.
GTS [62]: model for multivariate time series forecasting that learns the underlying graph structure simultaneously with Graph Neural Network (GNN) when this structure is not explicitly known. achieves this by framing the problem as learning a probabilistic graph model, optimizing mean performance over graph distribution that is parameterized by neural network, enabling differentiable sampling of discrete graphs.
STNorm [14]: Two normalization modules (temporal normalization and spatial normaliza- tion) are proposed to refine the high-frequency and local components of the raw data to help the model distinguish between time and space, respectively.
STID [63]: STID tackles the issue of indistinguishable samples in traffic prediction by simply attaching spatial and temporal Identity information to the data. Using basic Multi- Layer Perceptrons (MIPss) with this approach, STID demonstrates that clearly identifying samples in space and time is crucial.
26

---

aware traffic time series forecasting model, turning spatio- temporal agnostic models into spatio-temporal aware models with encoding time series from different locations into stochastic variables.
ST-WA [8]:
STDN [4]: STDN tackles complex traffic prediction by first building a dynamic graph and using spatio-temporal embeddings. Its core innovation is module that decomposes traffic data into trend-cyclical and seasonal components for each node. An encoder-decoder then uses these components for prediction, achieving superior performance.
Table 7: Efficiency comparison on all 6 datasets.
METRLA (N 207)
PEMSBAY (N
325)
PEMSI7(M) (N
228)
Model
#Params
Train
Infer
#Params
Train
Infer
#Params
Train
Infer
GRU
126K
71.96s
3.21s
126K
172.32s
7.62s
126K
27.96s
1.30s
STGCN
246K
81.39s
2.633
306K
210.90s
6.35s
257K
33.86s
0.97s
DCRNN
372K
529.94s
15.48s
372K
1071.18s
33.03s
372K
854.15s
30.72s
GWNet
309K
101.11s
2.22s
312K
260.98s
5.82s
310K
37.47s
0.88s
MTGNN
405K
62.95s
1.24s
573K
138.64s
2.80s
435K
21.18s
0.47s
AGCRN
752K
87.15s
2.68s
753K
197.93s
5,38s
752K
31.678
1.02s
GTS
38377K
190.48s
4.86s
58363K
546.96s
14.91s
12158K
54.12s
1.70s
STNorm
224K
64.80s
0.88s
284K
167.36s
2.31s
235K
23.37s
0.34s
STID
118K
12.06s
0.75s
122K
19.25s
0.62s
118K
4.71s
0.21s
ST-WA
375K
113.90s
7.41s
447K
284.30s
19.25s
388K
41.21s
2.78s
STDN
5971K
459.43s
10.39s
6273K
1479.01s
25.91s
6025K
170.05s
4.37s
ST-SSDL
498K
71.84s
8.65s
498K
199.60s
32.40s
181K
49.66s
6.64s
PEMS04 (N
307)
PEMS07 (N 883)
PEMSO8 (N 170)
Model
#Params
Train
Infer
#Params
Train
Infer
#Params
Train
Infer
GRU
126K
50.87s
2.30s
126K
249.13s
11.05s
126K
31.23s
1.36s
STGCN
297K
63.95s
1.75s
592K
369.55s
7.67s
227K
32.73s
1.22s
DCRNN
372K
269.71s
9.06s
372K
1330.26s
48.46s
372K
211.56s
6.61s
GWNet
311K
72.94s
1.79s
323K
445.99s
11.26s
309K
37.90s
0.92s
MTGNN
548K
38,84s
0.91s
1368K
212.79s
5.13s
353K
28.48s
0.51s
AGCRN
749K
58.13s
1.69s
755K
342.73s
9,89s
150K
38.44s
1.18s
GTS
16305K
100.87s
3.54s
27088K
1270.23s
62.82s
17136K
69.07s
1.79s
STNorm
275K
44.95s
0.74s
S70K
279.37s
4.68s
205K
28.06s
0.58s
STID
121K
5.74s
0.26s
140K
12.32s
0,64s
117K
5.42s
0.24s
ST-WA
436K
76.69s
5.70s
786K
678.93s
60,99s
353K
47,88s
3,80s
STDN
6227K
312.12s
7.98s
4480K
693.32s
32.56s
5876K
200.60s
5.71s
ST-SSDL
182K
83.48s
12.08s
379K
870.12s
217.80s
100K
65.81s
7.97s
A.7
Supplementary Case Study
To further understand the behavior different models under various real-world conditions, we conduct comprehensive visual comparison of forecasting results across six models: DCRNN, STGCN, AGCRN, MTGNN, STDN, and our proposed ST-SSDL on METRLA dataset. Figure 7, and 10 presents four representative cases: low deviation, medium deviation, high deviation, and partially missing values. Each case highlights specific challenges in spatio-temporal forecasting and demonstrates how different models respond under these conditions.
Cow-Devittion Scenario. In this case, the current input remains highly consistent with its historical average. Most models are able to make accurate forecasts under this setting. However, observe that AGCRN and STDN exhibit slight discrepancies in the prediction trajectory, possibly due to their reliance static graph structures insufficient temporal sensitivity. In contrast, ST-SSDL maintains precise alignment with the ground truth, demonstrating that performs competitively even in standard settings.
Medium- Deeiit--i Scenario. When moderate deviation occurs, certain models begin to show sensitivity to the changing input dynamics. DCRNN, STGCN, AGCRN, and MTGNN all produce predictions that visibly deviate from the ground truth, reflecting a struggle to generalize under moderate shifts. ST-SSDL, by contrast, adapts effectively to this condition and provides accurate
27

---

100
AGCRN (Nods
90
HETSRNA JRCIC Bedder
1400 Toal MCM Pedelee
80
Theltic Spreed
60
50
40
30
20
10
100
NTTGN INode 22)
STEN (Node 221
ST-55TL Ihode 221
90
Oreil ai
Ornad
Oue
80
Traffic Speed
70
60
50
40
30
20
03-28 12:40
03-28 13:30
03-28 14:00
03-28 12:40
03-28 13:20
03-28 14:00
03-28 12:40
03-28 13:20
03-28 14:00
Figure 7: Prediction comparisons under low deviation.
100
DCSTNN (Mode 9)
STECK NNode 91
AGGENN (odde
90
Hittr Vual
00
Pediii
Treffic Speed
70
60
50
30
20
10
MTGNNN (Node 91
STON thoce
ST-55OL (Node 91
90
Traffic Speed
70
50
40
30
20
10
0
03 14 11:25
03:14 12:05
03 14
12:45
03:14 11:25
03-14 12:03
14 12:45
03:14 11:25
03-14
03 12:45
Figure 8: Prediction comparisons under medium deviation.
predictions. This supports the effectiveness of our self-supervised deviation modeling strategy, which guides the model to dynamically adjust its representation based contextual shifts.
High-Deviation Scenario. In this more challenging case, a significant deviation is observed between current input and historical trends. All baseline models fail to capture this change and generate forecasts that diverge considerably from the actual values. ST-SSDL is the only model that main- tains predictive accuracy, closely tracking the ground truth despite the abrupt change. This result underscores the necessity of modeling deviation in spatio-temporaa forecasting and validates the core motivation of our proposed framework.
Missing- Value Scenario. Real-world data often contain missing or incomplete observations. In this setting, models such as DCRNN and STGCN suffer performance degradation due to the missing input points. ST-SSDL, however, continues to produce reliable predictions. This robustness may stem from its ability to interpret missing data as a form of deviation from historical norms, allowing it to handle imperfect inputs more effectively. This case highlights an additional advantage of deviation modeling: improved resilience to data corruption.
28

---

100
90
HECSRAA Bedden
1820 Tou MCCM Prddedes
80
Preditie
Theltic Spreed
60
50
40
30
20
10
100
MTTONN Mlode 163)
STEN (Wode 1631
ST-S5DL (Node 163)
90
Owee 3
Oroand
80
Traffic Speed
70
50
20
03-03 21:20
03-03 22:00
03-03 22:40
03-03 21:20
03-03 22:00
03-03 22:40
03:03 21:20
03-03 22:00
03-03 22:40
Figure 9: Prediction comparisons under high deviation.
100
OCRAE (Nodc 5)
STOCK INode 81
AGCIN (Wod:
Tryth Proditton
80
Ttelitc Speed
70
60
50
30
20
10
MTTCNN (Node
STON
ST-SSOL (Node
90
Oroad Brth
Soll
80
Trattic Speed
70
60
50
40
30
20
10
06:20
07:00
07-40
06:30
07:00
01:40
06:30
07:00
07:40
05
Figure 10: Prediction comparisons under partially missing values.
These visual analyses reinforce the motivation and effectiveness of our self-supervised deviation learning framework. ST-SSDL not only excels under large deviations but also maintains robustness in both common and imperfect scenarios.
29


Reviews:
[{"review_id": "W8EuZXoF2B", "reviewer": "Reviewer_hTWY", "summary": "The paper introduces a method to consider past events through a latent space that is then integrated into the learning process. The authors present experimental data that they claim supports their insight.", "questions": "into  S non-overlapping weekly ->why weekly? Shouldn´t this be application dependent?\n\n. Experiment statistical significance-.> I do not understand tthat reply, can´t you use a fixed number of seeds?\n\n- with dimensions h set to 128, 64,\n242 or 32, depending on the dataset.\"\ndo you have a rule for this choice?\n\nfigure 5: can you move back from the latent space to the original data?", "limitations": "...", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\nthe idea that you can take advantage of re-occurring irregularities makes good sense. For example, a car mightperform differently in a highway or in busy city  street.\n\\\\the evaluation seems to support these claims\n\nWeaknesses\nThe authors present the work in a very abstract way. An example would have make things  clear, and would motivate the work.\nFig 1: I couldn;t quite understand.\nThe results are extensive but for a very specific type of domains", "was_revised": false, "final_justification": "", "rebuttal": "# Response to Reviewer hTWY:\n\nWe would first like to thank you for the positive comments and valuable feedback. We respond to your comments and questions below.\n\n## **Weakness**\n\n> Q1: Let's start from the first weakness. The authors present the work in a very abstract way. An example would have make things clear, and would motivate the work. Fig 1: I couldn;t quite understand.\n\n**A1:** The entire motivation behind Figure 1 is as follows:\n\n(1) **History as anchor**. Historical Average [1,2,3] is always a simple yet effective approach for canonical tasks of time series forecasting. This motivates us to utilize historical average as an anchor (a reference) for each input to boost the forecasting performance. $D_1$ and $D_2$ in Figure 1(a) denotes the deviation (distance) between the current input and its anchor in physical space. The \"?\" in Figure 1(b) represents the corresponding deviation (distance) in latent space. Actually, they should be clearly detoned as $\\widetilde D_1$ and $\\widetilde D_2$.\n\n(2) **How to learn/quantify $\\widetilde D_1$ and $\\widetilde D_2$?** $D_1$ and $D_2$ are quite easy-to-calculate in physical space. In Figure 1(a), $D_1$ is approximately equal to 40 and $D_2$ is approximately equal to 20. However, it is never easy to accurately quantify $\\widetilde D_1$ $\\widetilde D_2$ in continuous latent space. The two “?” marks also conveys the core challenge of our study: how much should $\\widetilde D_1$ $\\widetilde D_2$ be equal to? i.e., $\\widetilde D_1$ = ? and $\\widetilde D_2$ = ?.\n\nTo address the challenge above, we come up with our core idea: **relative distance consistency**. **The current-history pairs that are close (far) in the physical space should remain close (far) in latent space**. i.e. $D_1$ > $D_2$ $\\Rightarrow$ $\\widetilde D_1$ > $\\widetilde D_2$. \n\nTo implement this core idea, first, we propose to **discretize** the continuous latent space with a set of learnable **prototypes**, where a contrastive loss $L_{Con}$ (Eq. (4)) is utilized to guide the learning (i.e., discretizing) process. \n\nNext, we map the latent representation to its nearest prototype through cross attention, so as to take each prototype as the proxy of the latent representation. $P^t$ and $P^a$ in Eq. (5) are the proxy prototypes for the current input $X^t$ and its historical anchor $X^a$. Then their distance $| P^t - P^a |$ can **approximately represent** $\\widetilde D$. By stopping the gradient of $Q^t$ and $Q^a$ ($\\widetilde{\\nabla}$), $|\\widetilde{\\nabla}(Q^t-Q^a)|$ can be taken as an approximate of physical-space distance $D$. \t\n\nFinally, the self-supervised deviation loss $L_{Dev}$ (Eq. (5)) is proposed to minimize $D- \\widetilde D$ to implement the **relative distance consistency**, i.e., $D\\uparrow \\to\\widetilde D\\uparrow$，$D\\downarrow\\to\\widetilde D\\downarrow$. Please also kindly refer to Figure 2 for the whole implementation.\n\nAs a successful learning result, we kindly ask you to refer to **Figure 4** in **Case Study (Sec. 5.6)**:\n\n- **Low deviation**  – current and historical curves nearly overlap; their latent representations are also close and select the *same* prototype.\n\n- **Medium deviation** – a medium gap in the physical space corresponding to a moderate latent distance; the two queries are assigned to two *nearby* prototypes.\n\n- **High deviation** – a large physical gap leads to widely separated latent representations and two *distant* prototypes.\n\nIn the final version, we will make this logical flow clearer by revising the explanation of Figure 1 and explicitly stating the principle of relative distance consistency in the Introduction. We hope this could resolve the concerns and making our motivation clearer.\n\n> Q2: The results are extensive but for a very specific type of domains.\n\nWe thank the reviewer for this insightful suggestion. We agree that evaluating our proposed ST-SSDL on a more diverse range of spatio-temporal domains would provide a broader perspective on its generalization capabilities.\n\nOur primary focus on transportation stems from its status as a standard and challenging domain in spatio-temporal forecasting. The datasets we used are widely recognized benchmarks for validating many classical models, including **DCRNN (ICLR'18, 4900+ citations)** [4], **Graph WaveNet (IJCAI'19, 2900+ citations)** [5], **AGCRN (NeurIPS'20, 1700+ citations)** [6], **MTGNN (KDD'20, 2000+ citations)** [7], and **GTS (ICLR'21, 400+ citations)** [8].\n\nGiven the rigorous evaluation on widely-adopted benchmarks, where our method demonstrates superior performance, we believe it can be readily extended to other spatio-temporal domains such as air quality prediction [9] and weather forecasting [10]. We will try to extend our method to more spatio-temporal domains in the future.\n\n## Question\n\n> Q3: into S non-overlapping weekly ->why weekly? Shouldn´t this be application dependent?\n\n**A3:** Thank you for highlighting this. Yes, your suggestion is absolutely correct, this partition length is **application-dependent**. We adopted weekly partitions due to the clear and widely recognized weekly periodicity in transportation data. As stated in **Sec. 4.1**, *\"These anchors summarize recurring spatio-temporal patterns and serve as references for deviation modeling\".* For domains with different temporal patterns, it could be adjusted accordingly. Importantly, ST-SSDL's core design does not inherently depend on weekly periodicity, making such adjustments feasible. We will clearly note this point in Sec. 4.1 in the final version.\n\n> Q4:  Experiment statistical significance-.> I do not understand tthat reply, can´t you use a fixed number of seeds?\n\n**A4:** Thank you for pointing this out. We have now conducted additional experiments with multiple random seeds (e.g., 5 runs per setting) on all six datasets. The results, including **mean and standard deviation**, demonstrate that ST-SSDL's performance gains over baseline models remain consistent and statistically significant. We will report these new results in the final version.\n\n| Dataset | Horizon | MAE | RMSE | MAPE (%) |\n| :--- | :--- | :---: | :---: | :---: |\n| **METRLA** | 15 min (Step 3) |2.62±0.02|5.06±0.04|6.68±0.06 |\n| | 30 min (Step 6) | 2.98±0.02 |6.08±0.04|8.04±0.09|\n| | 60 min (Step 12)| 3.38±0.02 |7.18±0.03|9.71±0.11|\n| **PEMSBAY** | 15 min (Step 3) |1.28±0.02|2.69±0.06|2.66±0.06 |\n| | 30 min (Step 6) | 1.59±0.03 |3.65±0.07|3.57±0.09|\n| | 60 min (Step 12)| 1.88±0.03 |4.39±0.06|4.41±0.08|\n| **PEMSD7(M)**| 15 min (Step 3) |2.04±0.01|3.87±0.03|4.81±0.05 |\n| | 30 min (Step 6) |2.59±0.01|5.22±0.03|6.50±0.07|\n| | 60 min (Step 12)|3.09±0.01|6.39±0.02|8.06±0.06|\n| **PEMS04** |Average|18.15±0.07|29.74±0.08|12.38±0.10|\n| **PEMS07** |Average|19.23±0.06|32.67±0.12|8.11±0.08|\n| **PEMS08** |Average|13.89±0.09|23.17±0.12|9.15±0.11|\n\n\n> Q5: with dimensions h set to 128, 64, 242 or 32, depending on the dataset.\" do you have a rule for this choice?\n\n**A5:** Actually, we do not follow a rigid rule. Instead, we select *h* to roughly based on dataset size and the hidden dimensions used by competing methods.\n\n> Q6: figure 5: can you move back from the latent space to the original data?\n\n**A6:** We thank the reviewer for this meaningful suggestion. While Figure 5 visualizes clusters in latent space that each prototype surrounded by its assigned queries, we can also compute the prototype's pattern in the input space by following method:\n\n1. Collect all input sequences assigned to $P_k$ (i.e., those $X_i$ whose query $Q_i$ assigned to $P_k$).  \n2. Compute the average of these sequences to form the prototype’s representative pattern curve.\n\nFormally, it can be calculated as following equation:\n$$\n\\text{Pattern}\\_k = \\frac{1}{|\\{i: Q\\_i \\in P\\_k\\}|}\\sum_{Q\\_i \\in P\\_k} X\\_i,\n$$\nwhere ($X_i$) denotes the input sequence whose query $(Q_i)$ assigned to prototype \\($P_k$\\).\n\nWe then visualize the traffic speed patterns of prototypes and observe several distinct patterns on METR-LA dataset. Since NeurIPS does not allow image updates in rebuttal phase, we would like to describe some of the phenomena we discovered: \n\n- **Prototype 7, 12, 14**: Rapidly decreasing curves $\\Rightarrow$ sudden‐drop events (i.e. traffic accident).\n- **Prototype 0, 2, 3, 4, 9, 13, 16, 18**: Flat curves at different levels $\\Rightarrow$ stable traffic conditions with small fluctuation.\n- **Prototype 10, 15, 17**: Gradually decreasing curves $\\Rightarrow$ morning or evening rush hour.  \n- **Prototype 1, 5, 6,**: Increasing curves $\\Rightarrow$ after morning or evening rush hour.\n\nMoreover, these representative patterns also **align well with the visualization of Figure 4**, further demonstrating that these prototypes capture meaningful spatio-temporal patterns. We commit to including this meaningful visualization and analysis in the final version to strengthen interpretability.\n\n**Reference**:\n\n[1] Dl-traff: Survey and benchmark of deep learning models for urban traffic prediction, CIKM, 2021.\n\n[2] Pre-training enhanced spatial-temporal graph neural network for multivariate time series forecasting, SIGKDD, 2022.\n\n[3] Z-GCNETs: Time zigzags at graph convolutional networks for time series forecasting, ICML, 2021\n\n[4] Diffusion convolutional recurrent neural network: Data-driven traffic forecasting, ICLR, 2018.\n\n[5] Graph WaveNet for Deep Spatial-Temporal Graph Modeling, IJCAI, 2019.\n\n[6] Adaptive graph convolutional recurrent network for traffic forecasting, NeurIPS, 2020.\n\n[7] Connecting the dots: Multivariate time series forecasting with graph neural networks, SIGKDD, 2020.\n\n[8] Discrete Graph Structure Learning for Forecasting Multiple Time Series, ICLR, 2021.\n\n[9] A neural attention model for urban air quality inference: Learning the weights of monitoring stations, AAAI, 2018.\n\n[10] Weather forecasting using ensemble of spatial-temporal attention network and multi-layer perceptron. *Asia-Pacific Journal of Atmospheric Sciences*, *57*(3), 2020."}, {"review_id": "epub94z9BZ", "reviewer": "Reviewer_DAZY", "summary": "This paper proposes ST-SSDL, a framework for forecasting spatio-temporal time series by explicitly modeling how current patterns deviate from historical trends. ST-SSDL uses a self-supervised approach: it compares current input data to its historical average, then represents both in a structured latent space using learnable prototypes that capture typical patterns. Two additional training objectives help the model quantify and organize these deviations without needing labeled data. Experiments on six benchmark datasets show that ST-SSDL consistently outperforms existing methods in prediction accuracy and adapts well to changing conditions.", "questions": "1. The paper assumes historical averages are reliable anchors, but real-world data often contains outliers or non-stationary trends (e.g., sudden traffic disruptions). Could ST-SSDL integrate robust statistics or outlier detection to improve deviation modeling in such cases?\n2. The experiments focus on traffic data with stable periodic patterns. How might ST-SSDL need to be modified for domains with irregular or aperiodic deviations (e.g., disease spread or social media activity)?\n3. Have the authors analyzed whether learned prototypes correspond to interpretable or human-understandable patterns?", "limitations": "yes", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The paper is well-written and methodologically rigorous, introducing a novel self-supervised framework (ST-SSDL) that effectively models deviations in spatio-temporal data. It demonstrates strong empirical results across six benchmarks, supported by thorough ablation studies and clear visualizations, while maintaining reproducibility through open-sourced code and detailed experimental setups.\nWhile technically sound, the paper has several limitations: (1) its reliance on historical averages as anchors isn't tested against noisy or non-stationary data (e.g., sudden traffic disruptions); (2) the 20 prototypes, while empirically chosen, lack interpretability as real-world patterns; (3) evaluation is confined to periodic traffic data, leaving generalization to irregular domains (e.g., epidemiology) unverified;", "was_revised": true, "final_justification": "The rebuttal effectively addressed my main concerns, including the robustness of the anchoring mechanism, applicability to irregular spatio-temporal domains, and interpretability of learned prototypes. I found the additional explanations and planned improvements to be reasonable and constructive. While some limitations remain, such as reliance on traffic data and potential generalizability issues, the paper is technically solid, presents a novel self-supervised approach, and demonstrates consistent empirical improvements. Therefore, I am keeping my score of 5.", "rebuttal": "# Response to Reviewer DAZY\n\nWe are very grateful for your positive recognition and would like to thank you for the thoughtful comments. We respond to your feedback below.\n\n## **Questions**\n\n> Q1:  The paper assumes historical averages are reliable anchors, but real-world data often contains outliers or non-stationary trends (e.g., sudden traffic disruptions). Could ST-SSDL integrate robust statistics or outlier detection to improve deviation modeling in such cases?\n\n**A1:** We appreciate the reviewer’s concern. As described in Section 4.1, our anchor is computed as the average over **the whole** training set. This global average inherently **smooths out** occasional outliers: any single disruption (e.g., an accident-induced outlier) contributes negligibly when aggregated with the full history. \n\nNote that, the anchoring strategy is a **flexible** component of our framework. For our task, we consider historical average as a simple yet effective strategy. Of course, robust statistics or outlier detection can be incorporated for better performance. We will add this in future work especially for more noisy applications.\n\n> Q2: The experiments focus on traffic data with stable periodic patterns. How might ST-SSDL need to be modified for domains with irregular or aperiodic deviations (e.g., disease spread or social media activity)?\n\n**A2:** Thank you for this insightful point. We would like to respond in two aspects:\n\nFor aperiodic data, actually, our anchoring mechanism is inspired by $HA$ (Historical Average) [1,2,3] ,which is always an important baseline almost for every application in time series forecasting domain. For most spatio-temporal domains like transportation, climate, and energy, they are with clear periodic cycles (**daily, weekly, monthly**). Here we utilize historical weekly pattern as a **simple yet effective** anchoring strategy. As you suggested, we will try to incorporate more robust anchoring strategies to domains with aperiodic data in the future. \n\nFor irregular deviations, while many spatio-temporal domains like transportation and climate exhibit clear periodic cycles, they also contains significant irregular deviations caused by **accidents and other sudden events (i.e. traffic control)**, which presents a major challenge for forecasting. By explicitly modeling the deviation between current inputs and historical patterns, our model can adapt to irregular disruptions. As we demonstrate in **Figure 4(c)** in **Sec. 5.6 Case Study**, ST-SSDL effectively handles high-deviation scenarios by mapping the current input and its historical anchor to well-separated prototypes in the latent space, thus capturing the potential irregular event.\n\nWe will include such discussion in the final manuscript.\n\n> Q3: Have the authors analyzed whether learned prototypes correspond to interpretable or human-understandable patterns?\n\n**A3:** We thank the reviewer for this meaningful suggestion. While Figure 5 visualizes clusters in latent space that each prototype surrounded by its assigned queries, we can also compute the prototype's pattern in the input space by following method:\n\n1. Collect all input sequences assigned to $P_k$ (i.e., those $X_i$ whose query $Q_i$ assigned to $P_k$).  \n2. Compute the average of these sequences to form the prototype’s representative curve.\n\nFormally, it can be calculated as following equation:\n$$\n\\text{Pattern}\\_k = \\frac{1}{|\\{i: Q\\_i \\in P\\_k\\}|}\\sum_{Q\\_i \\in P\\_k} X\\_i,\n$$\nwhere ($X_i$) denotes the input sequence whose query $(Q_i)$  assigned to prototype \\($P_k$\\).\n\nWe then visualize the traffic speed patterns of prototypes and observe several distinct patterns on METR-LA dataset. Since NeurIPS does not allow image updates in rebuttal phase, we would like to describe some of the phenomena we discovered: \t\n\n- **Prototype 7, 12, 14**: Rapidly decreasing curves $\\Rightarrow$ sudden‐drop events (i.e. traffic accident).\n- **Prototype 0, 2, 3, 4, 9, 13, 16, 18**: Flat curves at different levels $\\Rightarrow$ stable traffic conditions with small fluctuation.\n- **Prototype 10, 15, 17**: Gradually decreasing curves $\\Rightarrow$ morning or evening rush hour.  \n- **Prototype 1, 5, 6,**: Increasing curves $\\Rightarrow$ after morning or evening rush hour.\n\nMoreover, these representative patterns of each prototype also **align well with the visualization of Figure 4**, further demonstrating that these prototypes capture meaningful spatio-temporal patterns. We commit to including this meaningful visualization and analysis in the final version to strengthen interpretability.\n\n**Reference**:\n\n[1] Dl-traff: Survey and benchmark of deep learning models for urban traffic prediction, CIKM, 2021.\n\n[2] Pre-training enhanced spatial-temporal graph neural network for multivariate time series forecasting, SIGKDD, 2022.\n\n[3] Z-GCNETs: Time zigzags at graph convolutional networks for time series forecasting, ICML, 2021."}, {"review_id": "j7EsqWAEPp", "reviewer": "Reviewer_TcCg", "summary": "The paper introduces ST-SSDL, a spatio-temporal forecasting framework leveraging Self-Supervised Deviation Learning (SSDL) to capture dynamic deviations between current inputs and historical patterns. ST-SSDL anchors inputs to historical averages, discretizes the latent space with learnable prototypes, and optimizes using contrastive and deviation losses. Experiments on six benchmark datasets show it outperforms existing methods.", "questions": "1.\tSee Weakeness\n2.\tIn Equation (4) and Equation (5), one loss uses MSE while the other uses MAE. What is the rationale behind this choice?", "limitations": "See above", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1.\tThe motivation is clearly articulated: The paper identifies a critical gap in existing spatio-temporal models—ignoring dynamic deviations between current inputs and historical patterns.\n\n2.\tThe proposed ST-SSDL mechanism, including historical anchoring and prototype-based discretization, provides strong interpretability. Visualizations (Fig. 4-5) demonstrate adaptive prototype assignments under varying deviation levels, enhancing model transparency.\n\nWeaknesses:\n\n1.\tThe baselines used for comparison are outdated. While the paper references more recent models, these are not included as baselines, e.g., the meta-learning framework HIMeta [13] from KDD 2024.\n\n2.\tThe Query-Prototype mechanism closely resembles VQ-VAE’s codebook design, with attention-based soft assignment (Equation (3)) replacing hard quantization. The paper lacks a thorough discussion on how this differs from or improves upon VQ-VAE.\n\n3.\tThe use of averaging to generate historical anchors may be overly simplistic. The paper does not explore or compare alternative dynamic anchoring strategies\n\n4 . What is the difference between \"dynamic deviations\" and \"distribution shift\"? Can you cite literature to explain \"dynamic deviations\"?", "was_revised": true, "final_justification": "Thank you for your detailed response, which has addressed many of my concerns. However, I still think that this method shares similarities with certain VQ-VAE variants. While the application may differ, as you noted, the core approach remains quite similar. Consequently, the method's novelty appears limited. Nonetheless, I have revised my score to 4.", "rebuttal": "# Response to Reviewer TcCg\n\nWe thank you for your valuable feedback and provide our responses below.\n\n## **Weakness**\n\n> Q1: The baselines used for comparison are outdated. While the paper references more recent models, these are not included as baselines, e.g., the meta-learning framework HimNet[13] from KDD 2024.\n\n**A1:** Thank you for this suggestion. While STDN (AAAI'25) is our most recent baseline, we have now added HimNet (KDD'24) [1] and MegaCRN (AAAI'23) [2] to our comparisons. To ensure a direct and fair comparison, we ran the official code for these models locally and report their performance. For ST-SSDL, we report the average of multiple runs with different random seeds.\n\n* *Performance Comparison with MegaCRN and HimNet*\n\n| Model   | Metric  | METRLA (3/6/12)              | PEMSBAY (3/6/12)             | PEMSD7(M) (3/6/12)          | PEMS04 (Avg)         | PEMS07 (Avg)         | PEMS08 (Avg)         |\n|--------------|---------------|------------------------------|------------------------------|-----------------------------|----------------------|----------------------|----------------------|\n| **MegaCRN**   | **MAE** | 2.62 / 3.01 / 3.48           | 1.28 / 1.60 / 1.89           | 2.05 / 2.67 / 3.27          | 18.69               | 19.70               | 14.83               |\n|   | **RMSE** | 5.04 / 6.13 / 7.31           | 2.71 / 3.68 / 4.45           | 3.88 / 5.34 / 6.68          | 30.54               | 32.72               | 23.92               |\n|             | **MAPE** | 6.68% / 8.18% / 9.98%        | 2.67% / 3.59% / 4.48%        | **4.75%** / 6.59% / 8.50%   | 12.71%              | 8.29%               | 9.64%               |\n| **HimNet**    | **MAE** | **2.61** / **2.96** / 3.38   | **1.27** / **1.58** / **1.86** | 2.06 / 2.64 / 3.19          | 18.24               | 19.30               | **13.56**           |\n|    | **RMSE** | **5.04** / 6.09 / 7.26       | 2.69 / **3.63** / **4.37**   | 3.94 / 5.32 / 6.57          | 30.12               | 32.77               | 23.36               |\n|             | **MAPE** | 6.71% / 8.15% / 9.97%        | 2.67% / 3.58% / 4.42%        | 4.87% / 6.64% / 8.30%       | **12.13%**          | **8.05%**           | **9.02%**           |\n| **ST-SSDL (Mean)** | **MAE** | 2.62 / 2.98 / **3.38**       | 1.28 / 1.59 / 1.88           | **2.04** / **2.59** / **3.09** | **18.15**          | **19.23**           | 13.89               |\n|      | **RMSE** | 5.06 / **6.08** / **7.18**   | **2.69** / 3.65 / 4.39       | **3.87** / **5.22** / **6.39** | **29.74**          | **32.67**           | **23.17**           |\n|      | **MAPE** | **6.68%** / **8.04%** / **9.71%** | **2.66%** / **4.39** / **4.41%** | 4.81% / **6.50%** / **8.06%** | 12.38%              | 8.11%               | 9.15%               |\n\n\n* *Efficiency Comparison with HimNet*\n\n|| METRLA (#Para/Train/Infer) | PEMS07 (#Para/Train/Infer) | PEMS08 (#Para/Train/Infer) |\n| ------- | -------------------------- | -------------------------- | -------------------------- |\n| HimNet  | 1251K / 157.50s /17.29s| 1262K / 1354.37s / 201.77s | 2070K / 101.01s / 11.41s|\n| ST-SSDL | 498K / 71.84s / 8.65s| 379K / 870.12s / 217.80s| 100K /65.81s / 7.97s|\n\nOur proposed ST-SSDL outperforms MegaCRN on all six datasets and reaches a better or comparable performance against HimNet. Besides performance, ST-SSDL only uses 5% to 40 % parameters and runs 45% to 65% training time compared to HimNet. \n\n> Q2: The Query-Prototype mechanism closely resembles VQ-VAE’s codebook design, with attention-based soft assignment (Equation (3)) replacing hard quantization. The paper lacks a thorough discussion on how this differs from or improves upon VQ-VAE.\n\n**A2:** We sincerely thank the reviewer for this  insightful comment. While there is a visual resemblance in using a set of learnable vectors, we would like to clarify two **fundamental differences** with VQ-VAE [3] as follows:\n\n1. Goal: \n   Our first fundamental difference lies in the core **goal**. Our method is purposed to **quantify the deviation** between current input and its historical anchor. In this context, the prototypes act as a structured reference grid to measure these dynamic changes. This directly contrasts with VQ-VAE, whose primary goal is **data compression and generation**. It aims to represent an input via a single discrete index from its codebook for efficient, generative purposes, a goal entirely different from ours.\n2. Assignment Mechanism: \nTo measure the dynamic deviations, our model requires a rich representation. Our **soft, cross attention-based** approach in Eq. (3) provides a **smooth and differentiable** landscape, enabling stable and effective optimization of our deviation-aware objectives. Conversely, VQ-VAE uses **hard quantization**, selecting only the single closest codebook vector. While effective for compression, this hard assignment mechanism is **non-differentiable** and can be unstable in training process. \n\nWe appreciate the reviewer for prompting this comparison and will add a discussion to the revised manuscript to clarify these fundamental distinctions.\n> Q3: The use of averaging to generate historical anchors may be overly simplistic. The paper does not explore or compare alternative dynamic anchoring strategies.\n\n**A3:** Thank you for this insightful point. Actually, our anchoring mechanism is inspired by $HA$ (Historical Average) [4,5,6] ,which is always an important baseline almost for every application in time series forecasting domain. Especially for most spatio-temporal domains (data has clear periodic cycles), we consider historical weekly pattern is a simple yet effective anchoring strategy. Of course, we agree that there should be different anchoring strategies such as daily/weekly/monthly average or even moving average according to applications. Our core contribution, the deviation learning, should be recognized regardless of the anchoring strategy employed. We will add such kind of discussion in Sec.4.1 in the future version.\n\n> Q4: What is the difference between \"dynamic deviations\" and \"distribution shift\"? Can you cite literature to explain \"dynamic deviations\"?\n\n**A4:** We thank the reviewer for this comment. We would like to clarify the distinction between \"dynamic deviations\"and the well-known problem of \"distribution shift\".\n**Dynamic Deviation** refers to the changing difference **between a current observation and its corresponding historical average (anchor)**. The term \"dynamic\" emphasizes that the magnitude of this deviation **varies across different nodes and time points**, as we illustrate in the Introduction section. Our work proposes a method to explicitly model this varying signal to improve forecasting accuracy.\n\n**Distribution Shift**, in contrast, is a classical problem concerning a change in the underlying data distribution $P(X,Y)$ **between the training and testing sets**. This is a well-established challenge in machine learning, where a model trained on one statistical distribution performs poorly when evaluated on another.\n\nRegarding the request for literature on \"dynamic deviations,\" we respectfully clarify that, to the best of our knowledge, our work is the **first to propose deviation learning based on this concept** in spatio-temporal forecasting domain. Therefore, the idea as presented is a novel contribution of our paper, aiming to establish it as a critical, yet previously overlooked, aspect of spatio-temporal modeling. We will include this discussion in the final manuscript.\n\n## **Questions**\n\n> Q5: In Equation (4) and Equation (5), one loss uses MSE while the other uses MAE. What is the rationale behind this choice?\n\n**A5:** We thank the reviewer for this question, as our loss functions are deliberately chosen for their distinct objectives. We would like to clarify that Eq. (4), $L_{Con}$ we adopt a **variant of the Triplet Loss** [7] (squared L2 norm is typically used),  as noted in the **line 155** of our manuscript. We use squared L2 norm following its original equation. Conversely, for $L_{Dev}$ , we choose the Mean Absolute Error (MAE) for its robustness. Since this \"distance-of-distances\" manner can encounter large discrepancies in cases of high deviation, using an MSE would square these differences and create unstable gradients. The MAE loss provides a more stable training signal, preventing the model from being overly perturbed by extreme cases. \n\n> Q6: Response to your low score at significance.\n\n**A6**: At last, we would like to emphasis the core contribution of our work. We believe the core contribution of our work is significant as it is the first to formally propose and address the critical, yet previously overlooked, problem of dynamic deviation in spatio-temporal forecasting. By proposing a novel self-supervised deviation learning method that explicitly models these deviations, we provide a new perspective for the field and demonstrate its effectiveness by achieving state-of-the-art performance on six widely used benchmarks.\n\n**Reference**:\n\n[1] Heterogeneity-informed meta-parameter learning for spatiotemporal time series forecasting, SIGKDD, 2024.\n\n[2] Spatio-temporal meta-graph learning for traffic forecasting, AAAI, 2023.\n\n[3] Neural discrete representation learning, NeurIPS, 2017.\n\n[4] Pre-training enhanced spatial-temporal graph neural network for multivariate time series forecasting, SIGKDD, 2022.\n\n[5] Dl-traff: Survey and benchmark of deep learning models for urban traffic prediction, CIKM, 2021.\n\n[6] Z-GCNETs: Time zigzags at graph convolutional networks for time series forecasting, ICML, 2021.\n\n[7] Facenet: A unified embedding for face recognition and clustering, CVPR, 2015."}, {"review_id": "xoBHpkX6me", "reviewer": "Reviewer_CVvy", "summary": "This submission introduces a spatio-temporal time series forecasting framework, ST-SSDL, which is based on a GCRU-based encoder-decoder and adopts Self-Supervised Deviation Learning (SSDL) to explicitly model dynamic deviations between current observations and historical trends using a self-supervised mechanism. The authors evaluated the proposed method on six traffic datasets. Quantitative benchmarks, ablation studies, efficiency metrics, and qualitative visualizations demonstrate the effectiveness of ST-SSDL.", "questions": "1. Since the paper focuses on spatio-temporal time series forecasting, which component in ST-SSDL specifically addresses spatial analysis? \n\n2. How the anchors were computed in Section 4.1? \n\n3. Have the authors considered applying ST-SSDL to other spatio-temporal domains such as weather, electricity, or medical time series? \n\n4. Can you provide further semantic interpretation of the prototypes? Do they correspond to interpretable traffic states (e.g., rush hour, low traffic)?", "limitations": "The authors did not explicitly address the broader limitations or potential societal impacts. Some brief discussions on: i) Applicability to high-risk domains; and ii) Limitations in the anchoring mechanism; could be added.", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\ni) The paper focuses on explicit deviation modeling in spatio-temporal forecasting, and proposes a principled self-supervised approach to quantify and leverage deviations; \nii) The use of prototypes in latent space adds a structured, interpretable layer to the self-supervised learning pipeline; \niii) The proposed model is benchmarked on six well-known datasets and compared against a wide range of SOTA baselines. Ablation and efficiency studies are presented clearly; \niv) Prototype-based modeling provides interpretability in the latent space, and the visualizations (Figures 4 and 5) effectively demonstrate how the model adjusts to varying deviation levels; \nv) The authors stated to release code and data, which supports reproducibility.\n\nWeaknesses\ni) The focus of the paper is the introduction of SSDL, which could be applicable in many dynamic prediction tasks. It is not clear how it may perform on different architectures besides GCRU; \nii) It is not clear how spatial relationships are captured or leveraged in ST-SSDL; \niii) The experiments are all based on traffic datasets. It is unclear how generalizable the proposed method is to other spatio-temporal domains (e.g., climate, mobility, energy);\niv) Although the model is lightweight in model parameters, inference latency is relatively high due to the GCRU backbone. This limits deployment in real-time applications; \nv) There seem to be inconsistent notations: notation inconsistency: The authors used $X^t$ (e.g., first appeared in line 103 and throughout Section 4) to denote the recent input sequence $X_{\\tau−T+1:\\tau}$, but this superscript notation was never clearly defined and may be confused with a time index. Since $\\tau$ was already used to indicate the current time, a more consistent and intuitive notation might be $X^\\tau$ or simply $X_{t-T+1:t}$? \nvi) In Section 5.6, Figure 4 was not really referred or discussed in text.", "was_revised": true, "final_justification": "As reviewer TcCg commented, the methodological contribution in this submission appears limited, in addition to ambiguous presentation. After reading all the reviews and discussions, I still have lingering concerns on the novelty of SSDL, which appears incremental and similar to previous prototype-based time series methods, learning prototypical representations or states to predict future states. I could not find any component of SSDL specifically designed to model spatial dependency, which might need to be considered based on the writing of the current abstract and introduction section. I will keep my current score for now.", "rebuttal": "# Response to Reviewer CVvy\n\nThank you for your detailed comments! Those reviews are really helpful to improve our manuscripts. Below are detailed responses to questions.\n\n## **Weakness**\n\n> Q1: The focus of the paper is the introduction of SSDL, which could be applicable in many dynamic prediction tasks. It is not clear how it may perform on different architectures besides GCRU;\n\n**A1**: We thank the reviewer for this insightful question. To verify its applicability beyond the GCRU architecture, we conduct experiments by creating two additional variants based on MLP and vanilla Transformer backbones. \n\n||Metric (3/6/12 or Avg. )|MLP|MLP+SSDL|Transformer|Trans+SSDL| GCRU|Ours (GCRU+SSDL)|\n| ------ | ------------------------ | ------------------- | ------------------ | ------------------- | ---------- | ------------------ | ----------------- |\n|METRLA|MAE(3/6/12)| 3.11/3.81/4.93|**2.81/3.18/3.56**| 3.00/3.60/4.48|**2.68/3.02/3.40**|2.83/3.25/3.81|**2.62/2.98/3.38**|\n||RMSE (3/6/12)|6.12/7.69/9.71|**5.56/6.66/7.62**| 5.93/7.26/9.04|**5.18/6.13/7.14**|5.38/6.46/7.74|**5.06/6.08/7.18**|\n||MAPE (3/6/12)|8.44%/10.90%/14.95%|**7.64%/9.25%/10.81%**| 8.01%/10.24%/13.68% |**6.99%/8.32%/10.00%**|7.40%/8.87%/10.77% | **6.68%/8.04%/9.71%** |\n|PEMS04|MAE (Avg.)|26.03|**18.30**| 23.46| **18.26**|22.71|**18.15**|\n||RMSE (Avg.)|40.15|**29.87**| 36.92|**30.42**|35.07|**29.74**|\n||MAPE (Avg.)|17.88%|**12.35%**| 15.46%|**12.34%**|15.57%|**12.38%**|\n\nOur results show that integrating the SSDL module brings significant performance improvements to all architectures. This demonstrates SSDL's ability to effectively enhance diverse architectures for spatiotemporal forecasting.\n> Q2: It is not clear how spatial relationships are captured or leveraged in ST-SSDL; \n\n**A2**: We thank the reviewer for this question and would like to clarify this process. Spatial relationships in ST-SSDL are explicitly captured and leveraged through its **Graph Convolution Recurrent Unit (GCRU) backbone**, as detailed in Section 4.2 of our paper. **GCRU is a spatio-temporal encoder.** The core of this mechanism is the **graph convolution operation** together with gating mechanisms which is formally defined in Eq. (6) and Eq. (7). This mechanism enables the model to aggregate information from neighboring nodes at each time step, effectively modeling spatial dependencies based on the graph topology.\n\n> Q3: The experiments are all based on traffic datasets. It is unclear how generalizable the proposed method is to other spatio-temporal domains (e.g., climate, mobility, energy)\n\n**A3**: We thank the reviewer for this insightful suggestion. We agree that evaluating our proposed ST-SSDL on a more diverse range of spatio-temporal domains would provide a broader perspective on its generalization capabilities.\n\nOur primary focus on transportation stems from its status as a standard and challenging domain in spatio-temporal forecasting. The datasets we used are widely recognized benchmarks for validating many classical models, including **DCRNN (ICLR'18, 4900+ citations)** [1], **Graph WaveNet (IJCAI'19, 2900+ citations)** [2], **AGCRN (NeurIPS'20, 1700+ citations)** [3], **MTGNN (KDD'20, 2000+ citations)** [4], and **GTS (ICLR'21, 400+ citations)** [5].\n\nGiven the rigorous evaluation on widely-adopted benchmarks, where our method demonstrates superior performance, we believe it can be readily extended to other spatio-temporal domains such as air quality prediction [6] and weather forecasting [7]. We will try to extend our method to more spatio-temporal domains in the future.\n\n> Q4: Although the model is lightweight in model parameters, inference latency is relatively high due to the GCRU backbone. This limits deployment in real-time applications\n\n**A4**: We thank the reviewer for highlighting this trade-off. We agree and **have clearly discussed** this limitation in the **Sec.5.4 Efficiency Study and Limitation section**. \"The model’s inference speed is slightly affected by the GCRU architecture, though this overhead remains acceptable in most practical settings.\" While the inference latency is a little higher than some non-recurrent baselines due to the iterative GCRU architecture, we believe it is acceptable in most cases. We honestly present such trade-offs, yet believe our work is a valuable contribution in terms of state-of-the-art performance and parameter efficiency.\n\n> Q5: There seem to be inconsistent notations: notation inconsistency: The authors used (e.g., $X^t$ first appeared in line 103 and throughout Section 4) to denote the recent input sequence $X_{\\tau-T+1:{\\tau}^{'}}$, but this superscript notation was never clearly defined and may be confused with a time index. Since was already used to indicate the current time, a more consistent and intuitive notation might be $X^{\\tau}$ or simply $X_{t-T+1:t}$?\n\n**A5**: We thank the reviewer for pointing out this potential notational ambiguity. Our intention for using the superscript 't' (e.g., in $X^t$, $H^t$, $Q^t$) is to create a consistent notation throughout the paper for all variables related to the **current** input, as opposed to the historical anchor ($X^a$, $H^a$, $Q^a$). Because $t$ is already refer to **current**, we use $\\tau$ in subscript to denote the time index in Eq. (1) and Eq. (6). \n\nFor clarity, actually we have included a comprehensive notation table in the appendix to clarify this convention. We acknowledge that defining this more explicitly in the main body of the paper would improve clarity, and we will revise the text accordingly in the final version to prevent any confusion with time index.\n\n> Q6: In Section 5.6, Figure 4 was not really referred or discussed in text.\n\n**A6**: We thank the reviewer for this comment. We acknowledge this mistake. We will revise line **304** from \"high deviation scenarios.\" to \"high deviation scenarios **as Figure 4**.\" \n\n## **Questions**\n\n> Q7: How the anchors were computed in Section 4.1?\n\n**A7**: We would like to clarify the procedure for computing the historical anchors. As described in Section \"**History as Self-Supervised Anchor**\" in lines 120 to 125, the historical anchors are computed from the full training sequence. Specifically, the entire training set is partitioned into non-overlapping weekly segments. The historical anchor is then calculated by averaging the values at aligned time steps across all of these weekly segments. This process ensures that for any given input sequence, we can retrieve a timestamp-aligned historical average that serves as a robust, contextual baseline for deviation modeling. \n\nNote that the weekly anchoring strategy can be replaced by different ones such as daily/weekly/monthly average or even moving average according to applications.\n\n> Q8: Can you provide further semantic interpretation of the prototypes? Do they correspond to interpretable traffic states (e.g., rush hour, low traffic)?\n\n**A8**: We thank the reviewer for this meaningful suggestion. While Figure 5 visualizes clusters in latent space that each prototype surrounded by its assigned queries, we can also compute the prototype's pattern in the input space by following method:\n\n1. Collect all input sequences assigned to $P_k$ (i.e., those $X_i$ whose query $Q_i$ assigned to $P_k$).  \n2. Compute the average of these sequences to form the prototype’s representative curve.\n\nFormally, it can be calculated as following equation:\n$$\n\\text{Pattern}\\_k = \\frac{1}{|\\{i: Q\\_i \\in P\\_k\\}|}\\sum_{Q\\_i \\in P\\_k} X\\_i,\n$$\nwhere ($X_i$) denotes the input sequence whose query $(Q_i)$  assigned to prototype \\($P_k$\\).\n\nWe then visualize the traffic speed patterns of prototypes and observe several distinct patterns on METR-LA dataset. Since NeurIPS does not allow image updates in rebuttal phase, we would like to describe some of the phenomena we discovered: \t\n\n- **Prototype 7, 12, 14**: Rapidly decreasing curves $\\Rightarrow$ sudden‐drop events (i.e. traffic accident).\n- **Prototype 0, 2, 3, 4, 9, 13, 16, 18**: Flat curves at different levels $\\Rightarrow$ stable traffic conditions with small fluctuation.\n- **Prototype 10, 15, 17**: Gradually decreasing curves $\\Rightarrow$ morning or evening rush hour.  \n- **Prototype 1, 5, 6,**: Increasing curves $\\Rightarrow$ after morning or evening rush hour.\n\nMoreover, these representative patterns of each prototype also **align well with the visualization of Figure 4**, further demonstrating that these prototypes capture meaningful spatio-temporal patterns. We commit to including this meaningful visualization and analysis in the final version to strengthen interpretability.\n\n> Q9: Response to your low score at significance .\n\n**A9**: At last, we would like to emphasis the core contribution of our work. We believe the core contribution of our work is significant as it is the first to formally propose and address the critical, yet previously overlooked, problem of dynamic deviation in spatio-temporal forecasting. By proposing a novel self-supervised deviation learning method that explicitly models these deviations, we provide a new perspective for the field and demonstrate its effectiveness by achieving state-of-the-art performance on six widely used benchmarks.\n\n**Reference**:\n\n[1] Diffusion convolutional recurrent neural network: Data-driven traffic forecasting, ICLR, 2018.\n\n[2] Graph WaveNet for Deep Spatial-Temporal Graph Modeling, IJCAI, 2019.\n\n[3] Adaptive graph convolutional recurrent network for traffic forecasting, NeurIPS, 2020.\n\n[4] Connecting the dots: Multivariate time series forecasting with graph neural networks, SIGKDD, 2020.\n\n[5] Discrete Graph Structure Learning for Forecasting Multiple Time Series, ICLR, 2021.\n\n[6] A neural attention model for urban air quality inference: Learning the weights of monitoring stations, AAAI, 2018.\n\n[7] Weather forecasting using ensemble of spatial-temporal attention network and multi-layer perceptron. *Asia-Pacific Journal of Atmospheric Sciences*, *57*(3), 2020."}]

Author rebuttal:

Meta-review: This paper proposes ST-SSDL, a spatio-temporal forecasting framework that models dynamic deviations between current observations and historical anchors using prototype-based self-supervised learning. The method introduces novel deviation and contrastive losses to enforce relative distance consistency in latent space, achieving strong and consistent improvements across six traffic benchmarks with thorough ablations, interpretability analyses, and new baselines added during rebuttal. Reviewer scores (5, 5, 4, 3) reflected both strong enthusiasm for the contribution and lingering concerns about incremental novelty and evaluation scope. While one reviewer remained unconvinced about conceptual distinctiveness, others were persuaded by the rebuttal and emphasized the value of the deviation modeling perspective and the empirical rigor. On balance, this work represents a solid technical contribution with meaningful empirical impact.
Decision: Accept (poster)