Learning to Factorize Spatio-Temporal Foundation Models
Authors: Siru Zhong
Junjie Qiu
Yangyu Wu
Xingchen Zou
Zhongwen Rao
Bin Yang
Chenjuan Guo
Hao Xu
Yuxuan Liang
Venue: NeurIPS 2025 spotlight ()
Keywords: Spatio-Temporal Forecasting
Foundation Models
Primary area: applications
TL;DR: We propose FactoST, a two-stage spatio-temporal foundation model that factorizes universal temporal pretraining from spatio-temporal adaptation, enabling efficient and generalizable forecasting across domains.

Abstract: Spatio-Temporal Foundation Models (STFMs) promise zero/few-shot generalization across various datasets, yet joint spatio-temporal pretraining is computationally prohibitive and struggles with domain-specific spatial correlations. To this end, we introduce FactoST, a factorized STFM that decouples universal temporal pretraining from spatio-temporal adaptation. The first stage pretrains a space-agnostic backbone with multi-frequency reconstruction and domain-aware prompting, capturing cross-domain temporal regularities at low computational cost. The second stage freezes or further fine-tunes the backbone and attaches an adapter that fuses spatial metadata, sparsifies interactions, and aligns domains with continual memory replay. Extensive forecasting experiments reveal that, in few-shot setting, FactoST reduces MAE by up to 46.4% versus UniST, uses 46.2% fewer parameters, and achieves 68% faster inference than OpenCity, while remaining competitive with expert models. We believe this factorized view offers a practical and scalable path toward truly universal STFMs. The code will be released upon notification.

Learning to Factorize Foundation Models
Siru Zhong', Junjie Qiu', Yangyu Wu', Xingchen Zou', Zhongwen Rao", Bin Yang', Chenjuan Guo?, Hao Xu Yuxuan Liang':
'hee Hong Kong University of Science and Technology (Guangzhou), Huawei 2012 Laboratories, East China Normal University (szhong691, ywugg99,,yuu444, x2oo4488) xzou428) (jayjunjieqiu, p iateeaaniinngggee innnmanliannnhhhuutttttt .z.ggzzeuuuuu edu.
hkust maast-gg.... -gw.tt....uuu.... cn,
fraozhongven, xuaahonngeen,xuhao77
xuhao77 7hhaawwei com, (byang, (byang.cjguo] cjguo) ecnu. edu. cn
Abstract
Spatio-Tempon (ST) Foundation Models (STFMs) promise cross-dataset gen- eralization, yet joint ST pretraining is computationally costly and struggles with Nomain-speciffe spatial correlations. To address this, we propose FactoST, factor- ized STFM that decouples universal temporal pretraining from ST adaptation. The first stage trains aggnosic backbone via multi-task learning to capture multi- frequency, cross-domain temporal patterns at low cost. The second stage attaches an lightweight adapter that rapidly adapts the backbone to specific ST domains via metadata fusion, interaction pruning, domain alignment, and memory replay. Extensive forecasting experiments show that few-shot settings, FactoST reduces MAE by up 46.4% versus UniST, uses 46.2% fewer parameters, achieves 68% faster inference than OpenCity, and remains competitive with expert models. This factorized view offers practical, scalable path toward truly universal STFMs.
Introduction
Spatio-temporal (ST) data capture how signals evolve over time across complex spatial struc- tures-such as traffic speeds on road networks, air-pollution levels from citywide sensors, or electric- ity loads substations. Modeling ST data is fundamental to forecasting and decision support across
science, engineering, and society, enabling proactive anticipation and intervention [77 [3 58 73
In deep learning practice, workhorse for modeling such data [58]
Graph Neural Networks (STGNNs) the de facto 39 47 27 28), as depicted in Figure Ta). Given the intricate
nature of jointly learning spatial and temporal dependencies, early attempts decompose the learning problem into two complementary components: (i) a recurrent [32 6 46), convolutional [68] 64 30
that extracts temporal dependencies from each location's history, and (ii) Graph Neural Network (GNN) 29 that propagates information along edges to
attention-based module 34
72
18
capture spatial correlations among locations B3 68 64 57 41]. This design yields strong inductive bias, parameter efficiency, and state-of-the-arr performance on a wide range of benchmarks.
Inspired by the transformative impact of Foundation Models (FMs) in language 44 and vision [ researchers have recently begun to explore STFMs [36] 17 14. The core idea is simple Pretrain single model on diverse ST corpora (e.g.. climate, traffic, energy) and adapt it unseen datasets in a zero-shol or few-shot fashion, as shown in Figure [b). Such cross-domain pretraining equips STFMs with broad cross-dataset spatio-temporal knowledge and generalization beyond single-dataset scopes, often outperforming task-specific STGNNs when labeled data is scarce [40 69 [3 77
CCoresponding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

20-2
XX
Univenaal Temporal Pretraining
Sputis Temporal Adiptiation (hrage
(Stege 1)
Model
Model B Swew
C
Universal Madel
Warker
Triffe
ST Meraias Foster
Seatel
Nusse
87 Tokons
Spakol Filanna f
Ieegonal
Propoaa
Tuppaal
Telemine
Harachic
Froquancy Augrantzion
Costinad Mersary Heplay
Mult-Derza Peampting
Energy
Weather
Trotfl
Energy
Weather
Traflc
MaliTaak Pestnanen
Teaeely
(4) Tomppell Graph Networks
(b) Spatio-Temparal Foundistin Models
(c) Our Fackorizal ST Foundation Model (FactoST)
Figure I: Evolution of ST modeling: (a) Traditional coupled STGNN design; (b) Joint ST pretraining in STEMs with tokens from different space and time; (c) FactoST's factorized paradigm.
Nevertheless, training an STFM at scale presents two pronounced obstacles. First, spatial correlations differ dramatically across domains. For instance, the adjacency structure of a power grid differs greatly from urban road topology networks, making it difficult for a monolithic model to internalize all possible patterns; neighbouring air-quality stations in Beijing exhibit short-range diffusion dynamics, whereas tele-connection effects dominate climate indices across the Pacific Ocean. Second, existing STFMs [69, 33
mostly rely on the paradigm of jointly learning spatial and temporal dependencies for hundreds, thousands, or even millions of locations, which is computationally expensive; memory and time footprints grow quadratically with sequence length or graph size in these architectures.
In this paper, we address these challenges by factorizing STFM learning into two lightweight stages and introduce FactoST, new paradigm that decouples universal temporal learning from domain- specific ST adaptation. Generally, temporal patterns (such as seasonality, trends) share a common 1-D structure across domains, learnable once; spatial correlations hinge domain-specific graphs with sizes and semantics, often requiring tailored reasoning. FactoST exploits this asymmetry: it first distils the simpler temporal dynamics across domains, then attaches a compact adapter that injects the richer, domain-specijn spatial knowledge. Conceptually, FactoST can be seen as an "STGNN" in the era of FMs, reinstating spatio-temporal factorization scale (see Figure aa)+
Universal Temporal Pretraining (UTP): The first stage aims to learn general temporal knowledge (e.g., periodicity) across diverse domains. To achieve this, pretrain a purely temporal backbone on large-scale, cross-domain ST data, deliberately omitting any spatial modules. Multi fuutt--fr augmentation is utilized to encourage the model align multi-frequent information across scales, while dowain-aware prompts guide it in encoding task-specific context without explicit spatial graphs. Overall, this stage is graph-agnostic, lightweight, and highly scalable with strong generalizability.
Spatio-Temporal Adaptation (STA): For a target dataset, the second stage freezes or fine-tunes the UTP backbone and attaches a compact adapter that injects spatial awareness and domain specificity in one streamlined pass. The adapter first enriches temporal features with learnable spatio-temporal identifiers, making each token location- and time-aware. It then adaptively modulates these features how strongly each token aligns with its static
by computing three low-rank affinities affiniies--quantifyee spatial embedding, temporal calendar embedding, and lagged historical context- and fusing them into dynamic per-token weights. Finally, hierarchical soft prompts align representations between the pretraining and target domains at both layer and token levels, while small replay buffer periodically resurfaces earlier sequences to stabilize training and prevent catastrophic forgetting. By seamlessly weaving these components into single stage, STA endows the universal temporal features learned in UTP with just enough ST reasoning to excel in new tasks, achieving this with minimal computation and memory overhead. We summarize our technical contributions as follows:
We propose a two-stage factorized paradigm for STFMs that decouples Universal Temporal Pre- training (UTP) from
Adaptation (STA), enabling efficient learning and adaptation while preserving strong temporal capabilities without the need for costly joint ST pretraining.
We introduce key innovations in both stages: (1) In UTP, we leverage mulli-frequency augmentation, mulii-domain prompting, and multi-task pretraining to learn universal temporal patterns; and (2) In STA, we introduce ST metadata fusion (STMF) for spatial-awarr feature alignment; ST filtering (STF) for sparse interaction modeling: hierarchical domain alignment (HDA) to bridge domain gaps; and continual memory replay (CMR) to mitigate knowledge forgetting.
Extensive experiments show that FactoST outperforms existing STFMs by up to 46.4% in MAE under fho-shot settings, while reducing parameter count by 46.2% and inference latency by 68%. It remains competitive with domain-specific expert models, even without architecture customization.

---

2 Preliminary
2.1 Formulation
Definition of Opatio-Temporal Data. We define Spatio- Peatio-Temporal (ST) data D = (X, A.) as a sequence of multivariate observations recorded regularly at fixed set of spatial locations. Formally, let V = {VI,.... UN} denote N nodes, c.g., traffic sensors, grid cells, and weather stations. Each node provides a D-dimensional feature vector at every time step over a horizon of length L, forming a tensor X E 12XxLD, Spatial interactions are represented by an (often sparse) adjacency matrix A EER- NNN whose entries encode physical distance, functional similarity, or learned affinity. Many real-world datasets also carry node metadata M = {m,}) J.N. such as aco-coordinates or land-use types that supply auxiliary spatial context. This formulation also subsumes several commonly used representations, including multivariate time series 779 80 12
56
and ST raster data 77 69 4 19.
Goal of Spatio-Tempora Foundation Models (STFM). Most existing STFMs centre on ST fore- casting, as accurate future prediction is the cornerstone for a majority of ST applications, from traffic management to weather early-warning systems [36,69 69 17. STFMs therefore seeks to learn a representation function (--) that converts a large, cross-domain corpus of ST datasets into a peneral-purpose representation H (D Daa), where n. is the number of ST datasets. The key requirements for such a representation are: i) Versatility: It should support a broad spectrum of downstream forecasting tasks, including both short-term and long- long-term forecasting across diverse domains. ii) Efficiency: Adapting to a new task or domain must involve only a lightweight prediction head and minimal fine-tuning, while still matching surpassing fully retrained, task-specific models.
2.2 Related Work
Spatio-Temporal Graph Neural Networks (STGNNs). STGNNs are the de-facto backbone for learning representations from complex ST data, powering tasks that range from ST forecasting
and anomaly detection to classification and imputation (52 27 28
15 2]. Early STGNNs often
factorize the learning problem into two complementary components: (i) temporal modules (e.g..
RNNs B3 24). TCNs (64] 65)) to extract sequential patterns at individual nodes, and (ii) spatial modules (e.g.. GCNs (29 78 GATs (1)) to propagate information across graph edges. This factorized design, implemented either in stacked form [68] 76 or as tightly coupled pipelines (32
has proven effective across diverse domains such as traffic forecasting 6 26 energy industry 1 53
and environmental applications 35 11. Building on these foundations, recent work explores self-
supervised objectives (e.g., contrastive or generative pretext tasks) to extract Nomain-agnosioe ST features without dense labels [38 25]. Transformer-style STGNNs further extend receptive fields
with self-attention mechanism while retaining the ST factorization [66] 37). Despite these advances, most existing models are still trained from scratch for each dataset, which limits their cross-domain reuse and falls short of "training once, adapt everywhere".
Spatio-Tempore Foundation Models. Recent efforts have explored STFMs that learn universal representations through cross-domain pretraining 36 17 Notable examples include UNIST [69] and OPENCITY 33 which apply rransformer-basee architectures to large-scale traffic data. As
shown in Figure I b). UNIST tokenizes ST data into a sequence of ST tokens for Transformer-base learning and prompt- boase adaptation. OPENCITY integrates Transformers with GNNs for flexible graph modeling, yet its tightly coupled architecture demands domain-specific pre- pre-processing (e.g.. road networks) and is prone to overfitting to particular spatial dependencies. Both models rely on expensive joint ST pretraining, leading to suboptimal performance and high computational cost. In contrast, time series foundation models like TIMESFM (TO) and CHRONOS [3] achieve strong cross-domain generalization through purely temporal pretraining, but they lack any spatial awareness.
Our factorized framework bridges this gap: it first learns universal temporal patterns in scalable manner, then injects lightweight spatial adapters for rapid ST adaptation, achieving both versatility and efficiency without the heavy cost of joint pertaining on both spatial and temporal dimensions.
Table 1: Qualitative comparison of STGNNs, STFMs, and our factorized STFMs (FactoST).
Aspect
STGNNs
Existing STEMs
FactoST (ours)
Training strategy
Train from scratch on each dreaset
Joist ST pretrinnnn on massive corpor Transfomee on (M
Factorized: UTP STA
Temporal modeling
RNN TUN selr-mmettoon GCN CAT ried graph
tokeus
Freg. aug, dostain prompts Transforee '
Spatial modeling
Tranformer GNN grid tnkenization
Plagzsbble lightweight adspeers dynamic elge prining
Compatational cost
Muderate (pes-oomiin repeat)
Vary high Goint ST peatrininng
Low--to-madenti: (no spetil oost small slaptes in STA)
Cross-domain reuse
Mininal (bound to one graph)
Puriia (lied to graph modeligg)
High (gmph--ggostic backbous)
3

---

Multi-Frqueacc Augmentation
Spatin Tempural Fasisn (STMF)
Mediiaal d NAW
Prompttng
Spotto- Temporal Filtering (STF)
Jan Wo D
Dieereo Promge
Multi-Tae Pretrulaing
Hirrarticcll Domain Alignment (HIDA)
Feei Foae Notuo
Milielied Ancuic
Continual Memory Replay (CMM)
FEE Tuaniime Desier
Conalero Tolem
Prodetmiios
Tranfommet Sronder
Stage 1: Universal Temporal Pretraining (UTP)
Stage 2: Spatlo- Temporal Adaptation (STA)
Figure 2: Overview of FactoST.
3
Methodology
Figure presents the framework of FactoST for factorized STFM, consisting of two stages:
Given cross-domain ST data E RxXL with N nodes, L time steps, and D-dimension, we -eneral-purpose purpose temporal backbone T RL.D + RFXD independently to each node's sequence :.: to predict the future horizon F. This stage integrates multi-frequency augmentation, domain prompting, and multi-task learn universal temporal patterns.
apply a compact general
A lightweight adapter S is designed to rapidly adapt T to specific ST domains. For downstream input Xin R Lx we reuse the pretrained backbone to extract node-wist temporal features z IR N Lxd where d denotes the model's hidden dimension. S parameterized by & with || < ||T||- then refines these features by injecting ST metadata m. pruning redundant ST interactions, aligning domain gaps, and performing strategic sample mixing for fine-tuning. The final output Your = S(z; m) E IR Nx yields forecasts for the F-step future horizon.
3.1 Stage I: Universal Temporal Pretraining (UTP)
To distill transferable temporal dynamics across heterogeneous, ST domains, we pretrain a spatially agnostic temporal backbone on node-wise time series using a Transformer encoder-decoder architec- ture. This stage deliberately omits any spatial graph structure, enabling scalable and domain-agnostic learning of universal temporal patterns such as periodicity, trends, and multi-scale fluctuations.
Multi-Frequena Augmentation. Temporal patterns often exhibit both long-term trends and short- term fluctuations. Drawing on the ideas from [63 g 660. we adopt a frequency isolation strategy to generate diverse temporal views that emphasize distinct spectral components. Given raw input sequence x E LXD we first apply the Fast Fourier Transform (FFT) to obtain its spectral representation Xf CF, where F [T/2] 1. We then stochastically isolate either low- or high-frequency bands by sampling Km random cutoff thresholds {Ti)K< Km with т; ~ (0, F), and binary selectors { K., with μ; ~ {0, 13. For each pair (T.. we retain only the frequency components below if j4;
above Tr if μ4 1, effectively creating spectrally filtered version of the signal. The filtered spectra are transformed back to the time domain via inverse FFT, yielding Km augmented views {xm T==l
(i) Km L. These views-alow with the original input- independently patched into non non-ovcrlap segments length L' and projected into d-dimensional tokens. The resulting token sequences form a multi-view temporal tensor Xaug E IR Km x N x xd where N' L/L', serving as input the Transformer encoder. This design encourages the model to learn representations that are consistent across complementary frequency perspectives.
Multi-Domain Prompting. Drawing inspiration from codebooks in vision 43
we propose soft domain prompting mechanism to encode cross- domain contextual cues. Specifically, we construct learnable codebook P (pI.....P,,) E K,d, where each vector represents a prototypical

---

temporal context from specific domain. Given an input x, we first extract compact embedding x, Rd via global pooling and a linear projection. We then compute its similarity with each prompt vector Pj P using negative squared Euclidean distance. Subsequently, we apply softmax normalization to derive attention weights, and obtain the final Nomain-specinf prompt Xp E IR via weighted combination, thereby fusing reusable knowledge across multiple domains. Finally, xp is expanded into Np tokens and concatenated with the patched input to obtain Xctx E (N,
Kp
exp(s))
8j = -- --pl2 aj
j= Kp.
(1)
k=1 exp(sk)
where sy is the similarity score, aj is the attention weight, and xp is the domain context prompt.
Multi-Task Pretraining. To simultaneously capture universal temporal structures and enable effective cross-domain transfer, we jointly optimize two complementary objectives: self-supervised task that enforces consistency across multi-frequency views the input, and a supervised forecasting task that guides the model learn predictive, domain-aware representations:
self-supervised spectral consistency: The model reconstructs the original time series from the multi-view augmented input Xaug, ensuring learned representations preserve coherent information across complementary frequency bands. Specifically, Xaug, encoded by Transformer to capture deep temporal interactions, then decoded via a Transformer decoder and a linear head to regenerate the original sequence, optimized with mean squared error (MSE) loss.
Espec
|| SpecHead (aeeeeeadDDeddeerr (anoodde(EEnoder((xx..e )))
(2)
forervissi.or forecasting with prompt alignment: The model forecasts future values from the domain- prompted input Xetx to evaluate representation quality. Xetx passes through the shared Transformer encoder-decoder, but the decoder output is detached before the prediction wead-using the forecast- ing loss as a -onbbackproppgatedd supervisory signal. This ensures only the restoration task updates shared parameters, preventing negative transfer from task-specific biases.
Supervised
Cpodd
||y PredHead (detach
(Dccodd(((ncoooe(((==))))))) (rrcleaaadddccchhooooddeeeeCCee ||x, xpl12.
(3)
where y is the ground-truth. The first term minimizes supervised forecasting error, while the second enforces prompt consistency between the input embedding x, and its soft domain prompt Xp. inspired by the codebook alignment objective in VQ-VAE [54) to stabilize training and enhance representation fidelity. The pretraining objective combines both losses: L = Sypre + fpeed During subsequent ST fine-tuning, domain prompts are frozen, and only the Transformer layers and prediction head are updated using Lreed, enabling efficient adaptation while preserving pretrained knowledge.
3.2 Stage II: Spatio-Temport Adaptation
To adapt the pretrained temporal backhone to ST scenarios, we introduce four lightweight modules that incur minimal parameter overhead while effectively capturing ST dependencies.
Spatio-Temporal Metadata Fusion (STMF). This module injects ST context into the tempo- ral backbone via learnable identifiers. Given ST input E RNXLXD, we first get tem- poral representations He E Nxx'xd xd via patch embedding layer, then we define: (1) node- specific spatial embeddings E,, E NDD:; and (2) calendar-aware temporal embedding bank [Ee)ees, where each calendar type (e.g., minute-of-houe month-of-year) has embedding table E. E I K. with K. the number of discrete bins for type e (e.g., 60/24/7/31/12 (hour-of-day, dfy-kf-eeek, day-of- month, month-of-year); minutely: add minute-of-hhor).
hour-of-day, day-of-week, day-of-month,
The active set S is chosen by sampling frequency (e.g., hourly:
For each note-patch pair (i, 7), we map the start timestamp to calendar bins via (b((T) E [1,...,,, for each S, and form an ST identifier by concatenating the node embedding with the selected
calendar embeddings as hat (i, 7) = W, E; 1
E. (e(r) + b,, where || denotes concatenation.
The identifiers' dimension are projected and expanded to obtain H.t E NNN'N aligning with H,- The encoded input representation and ST identifiers are then fused together by residual addition: H H, + Hg. This enables integration of ST context without retraining the temporal model.
5

---

Spatio-Temport Temporal Filtering (STF). While STMF uses static ST identifiers, STF adapts to scenario- dependent cue relevance-es e.g. local spatial context for incidents vs. global temporal patterns for rush hours- by dynamically reweighting spatial and temporal interactions via three learnable affinities. From Hst E ERN*N' xd we extract spatial (E,) and temporal (E₃) embeddings and compute:
Spatial Affinity (S,): Mcasures the compatibility between Hg: and its spatial component E,, via dot-product: S = (Hat: E,) E RN*,", where higher values indicate stronger spatial relevance for each (node, patch) pair. This avoids rigid reliance on fixed spatial identifiers.
Temporal Affinity (S,): Quantifies alignment between Hst and its temporal component E,; S, (Ha.. E,) E R N .capturing dominant temporal patterns while filtering redundant noise.
Time-Lagged Affinity (Sa): Models asynchronous causal effects (e. (e.g..upstream nodes influenc- ing downstream nodes with delay 6). For lags ò 1,..., A, it aggregates historical neighbor -
E R.N'N', where (6) learnable lag weights. Higher values reflect stronger delayed relevance.
states st
(1-6) and computes: Sa Σ 25=1 (6) (H Agg,
To improve scalability, all affinity computations can be performed in a low-rank space. Specifically, we project both operands into I' (r < d) via shared or separate learnable matrices, e.g., for spatial affinity: S, H w E, w( where w w(() RR Analogous projections apply to S, and Sa. This reduces complexity from (dd) to (r)) per inner product while preserving semantic expressiveness. Top-K sparsification may further prune weak interactions.
The three affinities are stacked as S = S,, Sa] E RNxN'*3 projected to dimension via Wan E Ra3dd and normalized with a softmax (temperature Tan) to yield dynamic weights:
Finally, the refined output is obtained by aggregating the three affinity scores into S = [S., S, Sa] E R xLx3 projecting to dimension D via learnable matrix W att E Raxd, normalizing with softmax (temperature Tau) to get dynamic weights W, then modulating Hat with W and applying LayerNorm:
SWatt att
W softmax
H LayerNorm (Hat W) R
Talt
This design enables adaptive integration of ST context without retraining the temporal backbone, effectively balancing spatial and temporal semantics while suppressing irrelevant cues.
Hierarchical Domain Alignment (HDA). To bridge the discrepancy across domains and facilitate effective transfer of domain adaptation knowledge, we propose a hierarchical alignment module using the pretrained domain prompts p E IR KD which operates at two levels:
1. Layer-level alignment: For input embedding x, R.D. retrieve its k nearest prompts in p based on negative Euclidean distance and compute a soft domain prototype via averaging:
K((x.)
Topk (- - P2(12), P je[1,K,]
Σ P5 E
jEK(X.)
Token-level alignment: To capture sptaset- patterss beyond the pretrained prompt knowl- edge, we introduce a low-rank adaptation matrix A uv where u E R Nr and v E IR.. The domain-aware adjustment is then computed as X, = (Ix,PT) & A NR,,d Where 1,, is column vector of ones and @ denotes element-wise multiplication. The final representation fuses the layer-level prototype and token-level refinement for enhanced cross-domain generalization.
Continual Memory Replay (CMR). To mitigate knowledge forgetting during few-shot adaptation, we implement dynamic data mixing, combining current data and historical data. First, we establish memory buffer, given training sequences (X, // of length T, we partition the dataset into:
M (X+) (memory buffer), C (X,)
(current stream),
with Tm memory ssiee TJ (default: 0.2), where the memory buffer M preserves critical temporal patterns from initial learning to ensure stability under domain shift.
Each mini-batch B is constructed by strategically mixing samples from both current stream and memory buffer: = (ee,, U (X,), [j where I. and Im denote shuffled indices of and M, respectively, and R C I. has size r [B]]. with r = 0.3 by default. This mechanism effectively preserves implicit historical knowledge during domain adaptation.

---

4 Experiments
In our experiments, we aim to address the following research questions (RQ):
Can FactoST outperform prior approaches (including STGNNs, STFMs and other existing models) under few-shot and zero-shot scenaries? = Sec. 4.1 & Sec. 4.2.
RQ1:
RQ2: Which model component is critical to the final performance? + Sec. 4.3.1.
RQ3: How the data and computation efficiency of FactoST? = Sec. 4.3.2 & Sec. 4.3.3.
RQ4: Can we provide interpretability of the domain adaptation process in FactoST? + Sec. 4.3.4.
RQ5: the STA module architecture-agnosiic or limited to GNN-based backbones? = Sec. 4.3.5.
Datasets. We pretrain the temporal backbone on diverse ST datasets using Monash [16]. covering six domains (energy, nature, health, transport, web, economics) with 130M observations across multiple spatial nodes and sampling frequencies from 4 seconds to daily. During pretraining, we extract and process univariate time series per node independently to prevent data leakage. For evaluation, we use eight established ST benchmarks-traale -trfffc flow (PEMS03/04/07/00), speed (PEMS-BAY BAY, METR-LA), energy (Electricity), temperature (FTTh2), and climate (Weather--whin vary widely in spatial scale (21-883 nodes), temporal resolution (5 min-1 h), and sequence length (17k-52k steps), enabling a comprchensive assessment of cross-domain and multi-scale generalization (see A.I.1 for details).
Baselines. We compare FactoST with 12 competitive models across four categories: 1) STFMs: OpenCity B3 UniST [69] 2) TSFMs: TimesFM [10], Moirai 62]; 3) ST expert models: BigST 20
STAEformer [37], STID 48, D2STGNN 49]; 4) Time series expert models: TimeMixer [59].
PStchTST 42), DLinear [71]. Informer [79]. Consistent with previous works (69], we adopted Mean
Absolute Error (MAE) and Root Mean Square Error (RMSB) as evaluation metrics. More in A.
4.1 Few-shot Prediction
Setting. We evaluate few-shot adaptation using only 10% of labeled training data under two forecast- ing horizons: short-term (12 - 12) and long-term (96 - 96), following standard protocols [49 48)
Results. As shown in Tables 2 and 3 FactoST consistently outperforms all baselines across both short- and long-term horizons under few-shot adaptation. In the short-term setting (12 - 12).
FactoST improves MAF over STFMs-OpenCity and UniST-by 31.4% and 47.2%, respectively, where UniST suffers from its rigid grid-based design and OpenCity incurs high graph-learning overhead. Against TSFMs (TimesFM, Moirai), we observe average gains of 18.8%, demonstrating that our lightweight ST adaptation effectively enriches universal temporal representations. Notably, FactoST remains competitive with specialized ST expert models despite avoiding domain-specific
Table 2: Few-shot short-term forecasting (12-122 results on 10% training data across multiple ST datasets. Lower values indicate better performance. Red: the best. Blue: the second best.
Foundation Model
Expert Model
Method Type
Spario Temporyi
Tune Seriec
Spatio Temporal
Time Serter
Method
F OpenCity
UniST
TimesFM Moiral
BigST STAEformer STID D2STGNN
TimeMixer
PutchTST DLiieerr Informer
PEMS-US
RMSE
MAE
17.54
28.80
17.90
40.39
21.99
35.31
32.38
21.40
28.45
47.67
30.79
22.93
29.21
18.55
21.41
33.57
21.97
35.59
21.94
23.24
PEMS-04
RMSE
MAE
37.44
24.75
40.41
50.07
42.76
43.15
33.73
54.00
23,97
36.55
68.46
24.86
38.43
27:37
42.16
44.13
25.11
44.57
45.59
29.81
PEMIS-7
RMSE
MAE
41.92
26.48
48,43
63.47
$4.86
40.77
50.20
32.61
35.69
51.36
25.72
39.72
33.50
31.43
31.46
46.72
25.51
39.81
30.31
46.36
31.19
48.91
31,89
49.65
37.55
62.55
PEMS-08
RMSE
MAF
29.59
18.94
32.16
48.47
46.74
35.70
33.87
22.06
53.05
38.01
29.96
19.40
51.05
36.15
34.09
23.17
30.51
19:55
34.09
22.05
35.64
22.42
36.35
23.10
51.53
31.69
PRMS-Ray
RMSG
MAE
1.66
4.51
2.77
6.08
8.28
5.14
2.25
5.49
2.36
5.49
4.26
1.91
4.62
2.01
4.57
2,00
4.72
1.99
4.93
2.11
5.23
2.15
2.21
5:20
2.66
6.23
METR-LA
RMSE
MAF
12.75
4.95
4.61
4.00
4.23
4.34
ETTA2
RMSE
MAE
0.424
0.513
0.710
0.545
0.410
0.135
0.307
1.214
1.208
1.673
1.234
0.966
1.433
128
0.721
1.211
2.946
1,385
2.125
2.898
Electricity
MAE
0.374
0:412
0.565
0.529
0.837
0.638
0.858
0.575
0.686
0.767
0.840
1.282
1.598
RMSE
0.545
1.700
3.276
0.00
1.006
4.545
8,289
1.085
4.535
4.324
5.097
8.837
15.649
Weather
RMSE
MAE
0
0.376
0.414
0.660
1,380
0.309
0.32
0.138
0.433
0.184
0.375
0,951
0.575
1.085
0.920
0.330
0.587
1.269
0.967
0.311
1.074
0.296
0 1.583
1.046
0.958
1.783
Count
1st
9
0

---

Table 3: Few-shot long-term forecasting (96 + 96) results on 10% training data across multiple spatio-temporal datasets. Lower values indicate better performance. Red: the best, Blue: the second
Foundation Model
Expert Model
Method Type
Spatio Temporai
Sertes TimesEM Moiral
Spatlo Truporai
Series
Method
FactoST
OpenCity
BigST
STAEformer
D2STGNN
TimeMixer
PatchTST
DLineur Informer
PEMS-03
RMSE
MAR
28.57
46.78
14.21
54.82
94.00
67.70
50.77
51.40
70.47
75.56
$1.87
115.67
77,42
65.35
GDM
GDM
71.52
47.86
100.33
61:23
113.63
60.41
27
PEMS-04
RMSE
MAE
64.89
42.04
112.20
67,24
112.11
85.14
64.43
93.44
116.26
$1.30
80.25
52.37
91,95
64.12
111.12
78.13
QOM
ODM
58,44
86.67
184.00
70.31
125 44
85,61
54.26
33.42
PEMS-07
RMSE
MAR
72.47
45.60
18.36
50.70
101.2 20
134.98
208.36
157.10
200.30
134.46
82.12
54.92
91.06
61,45
106.95
71.32
00M
ODM
100.45
67.75
118.51
80.00
106 68
147.43
$2.82
$1.76
PEMS-08
RMSC
MAR
56.15
35.69
$2.07
49.47
96.15
73.81
125.27
89.53
97.89
68.73
$6.76
58.68
96,14
68.45
100.15
75.87
ODM
00M
65.53
45,10
87.33
57.31
109.15
76.77
44.25
65.43
PEMS
RMSE
MAG
2.96
12.38
7.40
5.17
5.18
5,78
2.93
6.20
3.28
3.10
00M
4,11
4.32
9.22
4.62
3.27
METR-LA
RMSE
13.07
13.62
19.56
22.56
14:13
22.39
12.17
12:22
11.56
11.91
13.35
14.17
7.20
13,42
12.29
ETTB2
RMSE
MAE
0.358
0.561
0.751
1.040
0.622
0.541
0.365
0.465
0.325
1.164
1.797
1.295
1,918
1.066
1.751
00M
1,015
1.666
0.943
1.609
1,009
1.781
2.968
3,783
Electricity
RMSE
MAE
0,409
0.265
0.305
1.240
0.494
2.512
0.465
0.305
0.484
0.312
0.481
2,843
0.735
6.562
0.440
2.755
G0M
00M
2.835
0.59
0.442
2,747
0.558
3.262
16.536
1.693
Weather
RMSE
MAE
0.226
0.436
3.730
0.653
0.491
0.348
0.484
0.270
0.465
0.262
0.892
1.522
1,171
1.804
0.740
1.446
00M
OOM
0.720
1.401
1,409
0.708
0,731
1.424
3.403
2.249
Count
2nd
Ist
12
0
2
2
0
spatial modules like graph networks, and substantially outperforms time series experts that ignore spatial structure-especiallyy on spatially correlated tasks. The advantage further widens in the long-term setting (96 - 96): FactoST achieves average MAE reductions of 40.5% over STFMs. 33.8% over TSFMs, and remarkably 44.1% 1% over ST experts, even as models like D2STGNN fail due out-of- out-of-memory errors under long-range forecasting. This underscores the scalability and efficiency of our factorized design: by decoupling universal temporal pretraining from plug-and-play ST adaptation, FactoST captures long-range dependencies without end-to-end joint ST pretraining or heavy spatial inductive biases-mall it particularly effective in low-data, long-horizon scenarios.
Table 4: Zero-shot performance comparison of foundation models on short-term (12 - 12) and long-term (96 - 96) forecasting across ST datasets. indicates the dataset seen in pretraining; results marked with are excluded when determining Red (best) and Blue (second-best).
FactoST
OpenCity
UniST
TimedFM
Moirai
Dataset
Horizon
MAE
RMSE
MAE
RMSE
MAE
RMSE
MAE
RMSE
MAE
RMSE
PEMS-Bay
Short
2.02
4.59
3,239
1.97
4.69
Long
6.12
10.47
6.92
11.79*
9:29
12.72
14.16
24.01
6.51*
11.70
METR-LA
Short
4.85
10.12
4.30
8,37"
24.33
29.31
14.99
5.55
13.79
Long
12.68
18.82
10.85*
17.60*
25.88
30.19
14.16
24.01
12.87
22.19
PEMS-03
Sbort
30.12
46.92
30.37
47.49
101.87
129.94
29.71
49.21
28.23*
46.36"
Long
113.62
144.80
125.18
159.23
102.87
137.90
7485*
110,78*
PEMS 04
Short
38.65
56.59
39. 30.
58.50
67.91
87.63
35.00
53.27
34.65*
52.13*
Long
143.11
175.99
153.18*
188.95
115.76
153.58
127.39
171.13
105,95*
141,10*
PEMS-07
Long
Short
156.07
190.59
66.68
172.20
45.18
211 1:225
65.15
104.93
144.67
133.65
184.62
151.9 .99
42.10
203, 18
64.59
125.91
125.
911
169.43
50.25*
PEMS-08
Short
32.14
47.27
32.45*
48.
73.46
93.14
29.68
45.18
38.23*
53, 12
Long
121.43
151.59
128.48*
161.75*
104.77
136.21
92.72
126.10
119.10*
15188*
Count
1st
4
5
0
3
0
2nd
4.2 Zero-shot Prediction
Setting. We evaluate zero-shot forecasting for both short- and long-term scenarios without fine-tuning.
Results. As shown in Table explicit joint ST pretraining does not improve zero-shot generalization. In fact, models without dedicated spatial modeling-suse as FactoST (UTP without STA) and outperform specialized STFMs like OpenCity and UniST. This confirms our core insight: spatial structures are highly domain-specijj and hinder transfer when baked into pretraining. UniST's iss unstable performance cifoommnncc-rasooablle long-term but catastrophic on short-term
8

---

tasks and its need for retraining under topology changes further expose the rigidity of fixed spatial priors. Even in-domain, OpenCity and Moirai (a multivariate TSFM) underperform, underscoring that strong zero-shot capability stems from temporal, not spatial, modeling. Remarkably, FactoST achieves the most top-2 rankings (4 first, second) among all foundation models on unseen domains, despite using only 13M pretraining points-orded of magnitude fewer than Moirai (27B) or TimesFM (100B) (Table 6 Nonetheless, errors remain high in long-term scenarios, highlighting the intrinsic challenge of zero-shot ST prediction. To address this, we apply Test-Time Computing [21 which reduces zero-shot MAE and RMSE by 7.72% and 8.79% on average; see Appendix A.4.4 for details.
4.3 Model Analysis
Ablation Studies. We evaluate each component's contribution via ablation on PEMS- 03 short-term forecasting (Figure 3). Removing CMR causes the largest drop (135. 15% MAE, 132.85% RMSE), underscoring its role in preserving knowledge during few-shot adaptation. Disabling HDA or STMF degrades MAE by 22% confirming their importance for domain alignment and metadata fusion. Among STF variants, omitting the spatial affinity matrix yiclds the sharpest decline (+29.01% % MAE, 133.70% RMSE), highlighting its efficacy in dynamic interaction modeling; temporal and time-delay matrices also contribute, albeit more moderately. We also find that the spectral consistency loss Lapee in pretraining provides nontrivial gains, more details are in Appendix A.4.3
Scaling Analysis. We evaluate FactoST on PEMS- across downstream fine-tuning proportions from zero-shot to full-shot for both short- and long-term forecasting (Figure 4). Remarkably, FactoST achieves rapid performance gains with minimal data: in the short-term setting, MAE drops from 25.96 (1 % data) to 17.54 (10% data) alread approaching full-shot performance (16.59). In the long-term setting, MAE plummets from 123.57 (zero-shot) to 28.57 with just 10% of the training data, and further improves to 25.85 under full supervision. This sharp improvement highlights FactoST 's exceptional data efficiency and fast adaptation capability. We also analyze scaling with respect to model size and pretraining data volume, observing diminishing returns beyond moderate capacity but consistent gains with more pretraining data; more details are in Appendix A.4. and A.4.2
MAE Compurkon by Variant
RMSE Compurison Varlunt
Short term Forecusting
Long term Forecasting
MAT
MAB
KMN
15552
MAR
RMIE
Performance
Performane
-400%
35,96
18122
30,10
24,5
26,50
Pt
91,4
1.
18225
22.01
4416
Fdo
77.
m
STF
Fail
eas so/m
57.77. m7
wis STF.
15:8
18
12
57 25.7
0219
190
Figure 3: Ablation studies of various components.
Figure 4: Data scaling analysis.
Efficiency Analysis. As shown in Figure 5 FactoST achieves a strong MAE of 17.86 on PEMS- 03 under the 10% few-shot setting with only 1.3M parameters and 8.1s inference time outperforming nearly all baselines in both accuracy and efficiency. In contrast, joint ST models like OpenCity (1.67M params, 25.3s) incur high computational overhead due to end-to-end graph learning. while large-scale TSFMs such as Moirai (91. .4MM params, 22.2s) ignore spatial structure entirely. Even though models like D2STGNN attain slightly better accuracy, they suffer from severe latency (54.5s). By decoupling universal temporal pretraining from lightweight, plug-and aad-payy spatial adaptation. FactoST achieves an exceptional cecuracy-efficiene trade-off, making it highly suitable for real-world deployment.
Domain Adaptation Analysis. Figure 6 visualizes ST token embeddings before and after Hier- archical Domain Adaptation (HDA) via t-SNE: original tokens (blue), adapted tokens (orange), and learned domain prompts (cireled clusters). The adapted embeddings shift clearly toward their corresponding prompt clusters, demonstrating that HDA effectively steers generic temporal repre- sentations toward -omain-specific semanties. Crucially, the global structure of the embedding space is
that aduptation is targeted and non-destructive. This provides empirical evidence that FactoST successfully bridges universal knowledge from pretraining with task- -seecciiii spatio-temporal context, enabling effective cross- cross-domain transfer without catastrophic forgetting.

---

Architecture Generality of STA. The STA module is architecture operating solely on feature embeddings without relying on GNN-specific inductive biases. To verify this, we integrate STA into PatchTST-a non-GNN, Transformer- based time series model-an and observe consistent few-shot improvements across all datasets (Figure 7). This confirms STA's ability to inject ST context into diverse backbones. FactoST remains superior, benefiting from large-scale pretraining; this highlights that temporal knowledge learned from diverse domains generalizes effectively, and when combined with lightweight spatial adaptation, enables strong cross-domain performance.
E Efficieney Comparison: MAE vs. Time vs. Params
Token Distribution in Domsin Adaptation Process
Do
Powt-HDA Token
2.42M
UIIST
20IJINI 1478.1x
TlmesPMM
opeciff
MOIKAL P1 91A31 23.25
MAE
STID
E.AM 18U9
Boreind
1.IMMIL 1.33
Paach (LAN) 7.8s
12.58M 1.0%
Intermer
167M 25.3x
OpenCity
13JIMI 5.7 1.11
FR
DISTGNN 9AM 54.5s
I2
-10
20
190
Inference 1ime (3)
+SNE Dimension
Figure 5: Efficiency comparison with baselines.
Figure 6: Domain adaptation visualization.
PEMS-03
PEMS-04
PEMS-07
PEMS-08
125
XAE RMSE
100
MAE RMSE
MAL KNISE
80
XAE RMSE
100-
75
55
75-7
75
2U12
20VA
56
55.15
50
45.35
FacteST
PaiCTTST
FadaST
PuchTTS
PasoSS
Pach1NT
FaceSTT
ParchTST
PachiST
Pawt
ETTh2
Electricity
Weather
MAE
MAE
1.5
RAASE
7222
RMSEE
0.541
3.56
0,5
8334
Q-900
414
0.2
ToceeST
STA
PStcTTST
FoctsNT
PaccTTST STA
FacuST
STA
PachTSS
FacteST
STA
PaccTTS
PatchTST
PatchTST
Figure 7: Few-shot long-term forecasting comparison of FactoST, PatchTST, and PatchTST with STA. Results confirm that STA is architecture-agnoss and consistently improves non-GNN backbones.
Conclusion and Future Work
We introduce FactoST, two-stage spatio-temporal foundation model (STFM) that decouples uni- versal temporal pretraining from lightweight spatio-temporal adaptation. This factorized design avoids the computational cost and poor generalization of joint ST pretraining in existing STFMs. Empirically, FactoST outperforms current STFMs in few-shot settings settings-reducing reducing MAE by up to 46.4% over UniST-whil using 46.2% fewer parameters and achieving 68% faster inference than OpenCity. Notably, it matches or surpasses domain- speciffc models without architectural customiza- tion, demonstrating the power of factorized pretraining as scalable path toward universal STFMs.
We identify two key directions for future work. First, spatial generalization remains a bottleneck: limited zero-shot performance across that rigid spatial priors impede cross-domain transfer, calling for more adaptive, semantics-aware representations. Second, the fine-tuning protocol can be improved: uniform parameter updates underutilize pretrained knowledge; parameter-efficient strategies (e.g., prompt tuning or selective retraining) could enhance transfer efficiency and mitigate catastrophic forgetting.
including our temporal-onl backbone- backbone-suggestt -uggesss
10

---

Acknowledgments and Disclosure of Funding
This work mainly supported by Huawei (Grant No. TC20041023007);; the National Natural Science Foundation of China (Grant No. 62402414); the Guangdong Basic and Applied Basic Research Foundation (Grant No. 2025 1025A151501 1994); the Guangzhou Municipal Science and Technology Project (Grant No. 2023A03J001 I); and the Guangzhou Industrial Information and Intelligent Key Laboratory Project (Grant No. 2024A03J0628).
References
[1] Xwégnon Ghislain Agoua, Robin Girard, and George Kariniotakis. Short-term spatio-temporal forecasting of photovoltaic power production. IEEE Transactions on Sustainable Energy. 9(2): 538-546, 2017.
[2] Md Mahbub Alam, Luis Torgo, and Albert Bifet. A survey on spatio-temporal data analytics systems. ACM Computing Surveys, 54(10s):1-38, 2022.
[3] Abdul Fatir Ansari, Lorenzo Stella, Caner Turkmen, Xiyuan Zhang, Pedro Mercado, Huibin Shen, Oleksandr Shchur, Syama Sundar Rangapuram, Sebastian Pineda Arango, Shubham Kapoor, al. Chronos: Learning the language of time series. arXiv preprint arX1v:2403.07815. 2024.
[4] Gowtham Atluri, Anuj Karpatne, and Vipin Kumar. Spatio-temporal data mining: A survey of problems and methods. ACM Computing Surveys (CSUR), 51(4):1-41, 2018.
[5] Muhammad Awais, Muzammal Nascer, Salman Khan, Rao Muhammad Anwer, Hisham Cholakkal, Mubarak Shah, Ming-Hsuan Yang, and Fahad Shahbaz Khan. Foundation models defining
in vision: survey and outlook. IEEE Transactions Pattern Analysis and Machine Intelligence, 2025.
[6] Lei Bai, Lina Yao, Can Li, Xianzhi Wang, and Can Wang. Adaptive graph convolutional recurrent network for traffic forecasting. Advances in neural information processing systems, 33:17804-17815, 2020.
[7] Abdelhakim Benechehab, Vasilii Feofanov, Giuseppe Paolo, Albert Thomas, Maurizio Filippone, and Balázs Kégl. Adapts: Adapting univariate foundation models to probabilistic multivariate time series forecasting. arXiv preprint arXiv:2502. 10235, 2025.
[8] Muxi Chen, Zhijian Xu, Ailing Zeng, and Qiang Xu. Fraug: Frequency domain augmentation for time series forecasting, arXiv preprint arXiv:2302.09992 2023.
[9] Wei Chen and Yuxuan Liang. Learning with calibration: Exploring test-time computing of spatio-temporal forecasting. arXiv preprint arXiv:2506.00633, 2025.
[10] Abhimanyu Das, Weihao Kong, Rajat Sen, and Yichen Zhou. A decoder-only foundation model for time-series forecasting. In Forty-first International Conference on Machine Learning, 2024.
[11] Mikhail Davidson and Deshendran Moodley. St-gnns for weather prediction in south africa. In Southern African Conference for Artificial Intelligence Research, pages 93-107. Springer, 2022.
[12] Shengdong Du, Tianrui Li, Yan Yang, and Shi-Jinn Horng. Multivariate time series forecasting via batedd encoder-decoder framework. Neurocomputing, 388:269-279, 2020.
[13] James H Faghmous and Vipin Kumar. Spatio-temporal data mining for climate data: Ad- vances, challenges, and opportunities. Data mining and knowledge discovery for big data: Methodologies, challenge and opportunities, pages 83-1 16, 2014.
[14] Yuchen Fang. Hao Miao, Yuxuan Liang, Liwei Deng, Yue Cui, Ximu Zeng. Yuyang Xia, Yan Zhao, Torben Bach Pedersen, Christian S Jensen, et al. Unraveling spatio-temporal foundation models via the pipeline lens: A comprehensive review. arXiv preprint arXiv:2506.01 00066003366 364,, 2025.
[15] Nan Gao, Hao Xue, Wei Shao, Sichen Zhao, Kyle Kai Qin, Arian Prabowo, Mohammad Saiedur Rahaman, and Flora Salim. Generative adversarial networks for spatio-temporal data: survey, ACM Transactions Intelligent Systems and Technology (TIST), 13(2):1-25, 2022.
11

---

[16] Rakshitha Godahewa, Christoph Bergmeir, Geoffrey I Webb, Rob J Hyndman, and Pablo Montero-Manso, Monash time series forecasting archive. arXiv preprint urXiv:2105.06643 2021.
[17] Adam Goodge, Wee Siong Ng, Bryan Hooi, and See Kiong Ng. Spatio-temporal foundation models: Vision, challenges, and opportunities. arXiv preprint arXiv:2501.099 09005,, 2025.
[18] Shengnan Guo, Youfang Lin, Ning Feng, Chao Song, and Huaiyu Wan. Attention based spatial- temporal graph convolutional networks for traffic flow forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 922-929, 2019.
[19) Shengnan Guo, Youfang Lin, Shijie Li, Zhaoming Chen, and Huaiyu Wan. Deep spatial- temporal 3d convolutional neural networks for traffic data forecasting. IEEE Transactions on Intelligent Transportation Systems, (0(10:3913333)0
2019.
[20] Jindong Han, Weijia Zhang, Hao Liu, Tao Tao, Naiqiang Tan, and Hui Xiong. Bigst: Linear complexity spatio-temporal graph neural network for traffic forecasting on large-scale road networks. Proceedings of the VLDB Endowment, 17(5):0811-1090 2024.
[21] Junlin He, Tong Nie, and Wei Ma. Geolocation representation from large language models are generic enhancers for spatio-temporal learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 7094-17104, 2025.
[22] Fei Huang, Jian-Rong Lv, Guo-Long Li, and Yang Yue. Spok: tokenizing geographic space for enhanced spatial reasoning in geoai. International Journal of Geographical Information Science, pages 1-41, 2025.
[23] Jincai Huang, Yongjun Xu, Qi Wang, Qi Cheems Wang. Xingxing Liang, Fei Wang, Zhao Zhang, Wei Wei, Boxuan Zhang, Libo Huang, et al. Foundation models and intelligent decision -akinng: Progress, challenges, and perspectives. The Innovation, 2025.
[24] Xiaohui Huang, Yuan Jiang, and Jie Tang. Mapredmn: multi-attention predictive mn for traffic flow prediction by dynamic spatio-temporal data fusion. Applied Intelligence, 53(16): 19372-19383, 2023.
[25] Jiahao Ji, Jingyuan Wang, Chao Huang, Junjie Wu, Boren Xu, Zhenhe Wu, Junbo Zhang, and Yu Zheng. Spatio-temporal self-supervised learning for traffic flow prediction. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 4356-4364, 2023.
[26] Weiwei Jiang and Jiayun Luo. Graph neural network for traffic forecasting: survey. Expert systems with applications, 207:117921, 2022.
[27] Guangyin Jin, Yuxuan Liang, Yuchen Fang, Zezhi Shao, Jincai Huang. Junbo Zhang, and Yu Zheng. Spatio-temporal graph neural networks for predictive learning in urban computing:
A survey. IEFE Transactions Knowledge and Data Engineering, 2023.
[28] Ming Jin, Huan Yee Koh, Qingsong Wen, Daniele Zambon, Cesare Alippi, Geoffrey I Webb. Irwin King, and Shirui Pan. A survey on graph neural networks for time series: Forecasting. classification, imputation, and anomaly detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
[29] Thomas N Kipf and Max Welling, Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv: 160.00900 2016.
[30] Xiangyuan Kong, Weiwei Xing, Xiang Wei, Peng Bao, Jian Zhang, and Wei Lu. Stgat: Spatial- temporal graph attention networks for traffic flow forecasting. IEEE Access, 8:134363-134372 2020.
[31] Zhenyu Lei, Yushun Dong, Jundong Li, and Chen Chen. St-fit: Inductive spatial-tempon forecasting with limited training data. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 12031-12039 2025.
[32] Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting, arXiv preprint arXiv:1707.01926 2017.
12

---

[33] Zhonghang Li, Long Xia, Lei Shi. Yong Xu, Dawei Yin, and Chao Huang. Opencity: Open spatio-temporal foundation models for traffic prediction. arXiv preprint arXiv:2408.10229, 2024.
[34] Yuxuan Liang, Songyu Ke, Junbo Zhang, Xiuwen Yi, and Yu Zheng. Geoman: Multi-level attention networks for geo-sensory time series prediction. In IJCAI, volume 2018, pages 2428-443,, 2018.
[35] Yuxuan Liang. Yutong Xia, Songyu Ke, Yiwei Wang. Qingsong Wen, Junbo Zhang, Yu Zheng, and Roger Zimmermann. Airformer: Predicting nationwide air quality in china with trans- formers. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 14329-14337, 2023.
[36] Yuxuan Liang, Haomin Wen, Yutong Xia, Ming Jin, Bin Yang, Flora Salim, Qingsong Wen, Shirui Pan, and Gao Cong. Foundation models for spatio-temporal data science: A tutorial and survey, arXiv preprint (rXXiv:5033/35000 2025.
[37] Hangchen Liu, Zheng Dong, Renhe Jiang. Jiewen Deng, Jinliang Deng, Quanjun Chen, and Xuan Song. Spatio-temporal adaptive embedding makes vanilla transformer sota for traffic forecasting. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pages 4125-4122 2023.
[38] Xu Liu, Yuxuan Liang, Chao Huang. Yu Zheng, Bryan Hooi, and Roger Zimmermann. When do contrastive learning signals help 30th international conference on advances in geographic information systems, pages 1-12, 2022.
graph forecasting? In Proceedings of the
[39] Xu Liu, Yuxuan Liang, Chao Huang, Hengchang Hu, Yushi Cao, Bryan Hooi, and Roger Zimmermann. Do we really need graph neural networks for traffic forecasting? arXiv preprint arXiv:2301.1260: 2023.
[40] Xu Liu, Junfeng Hu, Yuan Li. Shizhe Diao, Yuxuan Liang, Bryan Hooi, and Roger Zimmermann. Unitime: A
unified model for cross cross- domain time series forecasting, In Proceedings of the ACM on Web Conference 2024, pages 4095 1006,4406, 2024.
[41] Minbo Ma, Peng Xie, Fei Teng, Bin Wang, Shenggong Ji, Junbo Zhang, and Tianrui Li. Hist- gnn: Hierarchical apatio-temponn graph neural network for weather forecasting. Information Sciences, 648: 19580, 2023.
[42] Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In the Eleventh International Conference on Learning Representations (ICLR), 2023.
[43] Aaron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017.
[44] OpenAI. Gpt-4v(ision) system card. 2023.
[45] Zhongzheng Qiao, Chenghao Liu, Yiming Zhang, Ming Jin, Quang Pham, Qingsong Wen, PN Suganthan, Xudong Jiang, and Savitha Ramasamy. Multi-scale finetuning for encoder-based time series foundation models. arXiv preprint arXiv:2506. 14087, 2025.
[46] Chen Qiu, Yanyan Zhang, Zhiyong Feng, Ping Zhang, and Shuguang Cui. Spatio-temporal wireless traffic prediction with recurrent neural network. IEEE Wireless Communications Letters, 7(4):554-55) 557, 2018.
[47] Zahraa Al Sahili and Mariette Awad. Spatio-temporal graph neural networks: A survey. arXiv preprint arXiv:2301. 100669, 2023.
[48] Zezhi Shao, Zhao Zhang, Fei Wang, Wei Wei, and Yongjun Xu. Spatial-temporal identity: A simple yet effective baseline for multivariate time series forecasting. In Proceedings of the 31st ACM International Conference on Information Knowledge Management, pages 4454 4458. 2022.
13

---

[49] Zezhi Shao, Zhao Zhang, Wei Wei, Fei Wang, Yongjun Xu, Xin Cao, and Christian S Jensen. Decoupled dynamic pratial-tempora graph neural network for traffic forecasting. Proceedings of the VLDB Endowment, 15(1) ((11))273322111 2746, 2022.
[50] Zezhi Shao, Fei Wang, Yongjun Xu, Wei Wei, Chengqing Yu, Zhao Zhang, Di Yao, Tao Sun, Guangyin Jin, Xin Cao, et al. Exploring progress in multivariate time series forecasting: Comprehensive benchmarking and heterogeneity analysis. IEEE Transactions on Knowledge and Data Engineering, 2024.
[51] Junho Song, Jiwon Son, Dong-hyuk Seo, Kyungsik Han, Namhyuk Kim, and Sang- Mooo- Kim. St-gat: A spatio-temporal graph attention network for accurate traffic speed prediction. In Pro- ceedings of the 3 ACM international conference on information & knowledge management, pages 4500-4504 4504, 2022.
[52] Feiyan Sun, Wenning Hao, Ao Zou, and Qianyan Shen. A survey on spatio-temporal series prediction with deep learning: taxonomy, applications, and future directions. Neural Computing and Applications, 36(17):9919-99)) 1943993, 2024.
[53] Akin Tascikaraoglu. Evaluation of spatio-temporal forecasting methods in various smart city applications. Renewable and Sustainable Energy Reviews, 02:424-435, 2018.
[54] Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning, Advances in neural information processing systems, 30, 2017.
[55] Hongjun Wang, Jiyuan Chen, Lingyu Zhang, Renhe Jiang, and Xuan Song. Unveiling the inflexibility of adaptive embedding in traffic forecasting. arXiv preprint arXiv:2411.11448, 2024.
[56] Jun Wang, Wenjie Du, Yiyuan Yang, Linglong Qian, Wei Cao, Keli Zhang, Wenjia Wang, Yuxuan Liang, and Qingsong Wen. Deep learning for multivariate time series imputation: A survey, arXiv preprint arXiv:2402.0444 04059, 2024.
[57] Qiongyan Wang, Yutong Xia, Siru ZHong, Weichuang Li, Yuankai Wu, Shifen Cheng, Junbo Zhang, Yu Zheng, and Yuxuan Liang. Airradar: Inferring nationwide air quality in china with deep neural networks. arXiv preprint arXiv:2501.1314) 2025.
[58] Senzhang Wang, Jiannong Cao, and S Yu Philip. Deep learning for spatio-temporal data mining: A survey. IEEE transactions on knowledge and data engineering, 34(8):3681-377 2020.
[59] Shiyu Wang, Haixu Wu, Xiaoming Shi, Tengge Hu, Huakun Luo, Lintao Ma, James Y Zhang, and Jun Zhou. Timemixer: Decomposable multiscale mixing for time series forecasting. arXiv preprint arXiv:2405. 14616, 2024.
[60] Yihang Wang. Yuying Qiu, Peng Chen, Kai Zhao, Yang Shu. Zhongwen Rao, Lujia Pan, Bin Yang, and Chenjuan Guo. Towards a general time series forecasting model with unified representation and adaptive transfer. In Forty-second International Conference on Machine Learning.
[61] Yuxuan Wang, Haixu Wu, Jiaxiang Dong, Guo Qin, Haoran Zhang, Yong Liu, Yunzhong Qiu, Jianmin Wang, and Mingsheng Long. Timexer: Empowering transformers for time series forecasting with exogenous variables. Advances in Neural Information Processing Systems, 37: 2024.
[62] Gerald Woo, Chenghao Liu, Akshat Kumar, Caiming Xiong, Silvio Savarese, and Doyen Sahoo. Unified training of universal time series forecasting transformers. arXiv preprint arXiv:2402.02592, 2024.
[63] Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Times- net: Temporal 2d-variation modeling for general time series analysis. arXiv preprint arXiv:2210.02186. 2022.
[64] Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, and Chengqi Zhang. Graph wavenet for deep patiaaltemporal graph modeling. arXiv preprint 20001,,,0000012,20000
14

---

[65] Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chengqi Zhang. Connecting the dots: Multivariate time series forecasting with graph neural networks. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pages 753-763, 2020.
[66] Mingxing Xu, Wenrui Dai, Chunmiao Liu, Xing Gao, Weiyao Lin, Guo-Jun Qi, and Hongkai Xiong. Spatial-tempon transformer networks for traffic flow forecasting. arXiv preprint arXiv:2001.02908. 2020.
[67] Zhijian Xu, Hao Wang, and Qiang Xu. Intervention-awaree forecasting: Breaking historical limits from system perspective. arXiv preprint arXiv:2405.1352,, 2024.
[68] Bing Yu, Haoteng Yin, and Zhanxing Zhu. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. arXiv preprint
04875, 2017.
[69] Yuan Yuan, Jingtao Ding, Jie Feng, Depeng Jin, and Yong Li. Unist: A prompt-empowered universal model for urban spatio-temporal prediction. In Proceedings of the 30th ACM SIGKDD Conference Knowledge Discovery and Data Mining, pages 4095-4106, 2024.
[70] Yuan Yuan, Chonghua Han, Jingtao Ding, Depeng Jin, and Yong Li. Urbandit: A foundation model for open-world urban spatio-temporal learning. arXiv preprint arXiv:2411.12164. 2024.
[71] Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 11121-11128, 2023.
[72] Chenhan Zhang, JQ James, and Yi Liu. Spatial-temporal graph attention networks: A deep learning approach for traffic forecasting. leee Access, 7: 7:166246-166256 2019.
[73] Huaiwu Zhang, Yutong Xia, Siru Zhong, Kun Wang. Zekun Tong, Qingsong Wen, Roger Zimmermann, and Yuxuan Liang. Predicting carpark availability in singapore with cross- domain data: A new dataset and a data-driven approach. In Proceedings of the Thirty-Third International Joint Conference Artificial Intelligence, pages 7554-7566 2024.
[74] Junbo Zhang, Yu Zheng, and Dekang Qi. Deep spatio-temporal residual networks for citywide crowd flows prediction. In Thirty-first AAAI conference on artificial intelligence, 2017.
[75] Zhenwei Zhang, Jiawen Zhang, Shun Zheng, Yuantao Gu, and Jiang Bian. Does cross-domain pre-training truly help time-series foundation models? In ICLR 2025 Workshop on Foundation Models in the Wild.
[76] Ling Zhao, Yujiao Song, Chao Zhang, Yu Liu, Pu Wang. Tao Lin, Min Deng, and Haifeng Li. T-gen: A temporal graph convolutional network for traffic prediction. IEEE Transactions Intelligent Transportation Systems, 21 21):3384483388
2019.
[77] Yu Zheng, Licia Capra, Ouri Wolfson, and Hai Yang. Urban computing: concepts, methodolo- gies, and applications. ACM Transactions on Intelligent Systems and Technology (TIST), 5(3): 1-55, 2014.
[78] Chongyang Zhong, Lei Hu, Zihao Zhang, Yongjing Ye, and Shihong Xia. Spatio-temporal gating-adjacency gen for human motion prediction. In Proceedings of the IEEF/CVE Conference on Computer Vision and Pattern Recognition, pages 6447-6456, 2022.
[79] Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 11105-11115, 2021.
[80] Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang. Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International conference on machine learning, pages 27268-27286. PMLR, 2022.
[81] Zhengyang Zhou, Qihe Huang, Binwu Wang, Jianpeng Hou, Kuo Yang, Yuxuan Liang, Yu Zheng, and Yang Wang. Coms2t: A complementary spatiotemporal learning system for data- adaptive model evolution. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes].
Justification: The paper clearly states the contributions of FactoST, a novel spatio-temporal foundation model framework that decouples universal temporal learning from task task-specific spatial adaptation. These claims are backed by theoretical motivation (e.g., multi pre-training and fine-tuning strategies) and extensive experiments across diverse domains (traffic, energy, weather), demonstrating strong few-shot performance and computational efficiency.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes].
Justification:
In Section 5 we discuss the limitations of this work.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
16

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes].
Justification: All the proofs in this paper can be found in the method section and are verified in the experimental part.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes].
Justification: We will provide detailed descriptions of the methods and experimental setup in the supplementary material. Additionally, we will release our code to ensure faithful reproduction of the main results upon the paper's acceptance.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to model. In general. releasing code and data often one good way to accomplish this, but reproducibilityy can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the partieular way they provide for reproducibiliity In the case of closed-source models, it may be that access to the model is limited in
17

---

some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes].
Justification: We will release our code to ensure faithful reproduction of the main results upon the paper's acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. ec/
public/guidess htapa////uidess/oodebbbbissssss
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
//nips. //nips.c/ppie hetpol/c/ggiidee/ooooooo
wwwiensssioniiii ssionPolicy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/eetails
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes].
We have provided detailed experimental settings in Appendix A.1 to facilitate reprodueibilityy
Justification:
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No].
Justification: spatio-temporal forecasting tasks, common not to provide error bars but instead directly calculate the average by conducting multiple experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
18

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes].
Justification:
We provide detailed experimental setup and complexity analysis.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics htttps://eurrps... http:////aaaappppooooooocchaaaaa
Answer: [Yes].
Justification:
We follow the NeurIPS Code of Ethics in this paper.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes].
Justification:
We provide discussion of the broad implications in Appendix
Guidelines:
19

---

The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymentt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA].
Justification: The datasets chosen in this paper are commonly used benchmark datasets for forecasting tasks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes].
Justification: Yes, we have.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
20

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
If assets are released, the license, copyright information, and terms use in the datasets.paperssiii paperswi thcode. www.com//aaaa datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes].
Justification:
This paper follows CC 4.0, and the code is in an anonymized URL.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA].
Justification:
This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA].
Justification:
This paper does involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
21

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [No]
Justification: Large language models were used only for minor tasks such as grammar checking and formatting improvements.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy auttep://eerrrppcccc///oeeeeeeer / for what should or should not be described.
22

---

A Technical Appendices
A.1
Implementation Details
This section provides a comprehensive overview of the implementation setup, including datasets, evaluation metries, hyperparameters, implementation details, and training configurations.
A.1.1 Datasets
Pretraining Datasets: As shown in Table we use cross-domain large-scale ST datasets, including energy, nature, transportation, and web, to frain our temporal backbone. From these datasets, we extract mode-wise temporal sequences across different locations over time. These sequences span multiple frequencies (from seconds to daily) and exhibit various temporal patterns, ensuring that the learned representations are robust and transferable across different forecasting tasks. We also add data volume comparison in Table 6 where FactoST is much lower than other foundation models.
Table 5: List of pretraining spatio-temporal datasets.
Dataset
Domain
Frequency
Time Points
Source
Aus. Electricity Demand
Energy
Half Hourly
1155264
Monash 16
Wind
Energy
4 Seconds
7397147
Monash I6
Wind Farms
Energy
Minutely
172178060
Monash [6
Solar
Energy
Minutes
7200720
Monash 16
Solar Power
Energy
4 Seconds
7397222
Monash [6
London Smart Meters
Energy
Half Hourly
166327216
Monash 16
Temperature Rain
Nature
Daily
23252200
Monash 16
Saugeen River Flow
Nature
Daily
23741
Monash [6
Sunspot
Nature
Daily
73924
Monash 16
Weather
Nature
Daily
43032000
Monash
KDD Cup 2018
Nature
Daily
2942366
Monash [6
US Births
Nature
Daily
7305
Monash 16
Pedestrian Countss
Transport
Hourly
3132346
Monash
Web Traffic
Web
Daily
116485589
Monash [6
Bitcoin
Economic
Daily
75364
Monash [6
Table 6: Pretraining corpora and scale of STFMs and TSFMs used in this study.
Model
Pretraining Corpus
Scale
FactoST
Monash (5 domains, 155 datasets)
13M time points
OpenCity
21 heterogeneous traffic datasets (10,1 regions)
151.1M observations
UniST
21 multi-source grid datasets
TimesFM
Large-scale real-world and synthetic series
100B time points
Moirai
LOTSA dataset
27B time points
Evaluation Datasets: For downstream ST forecasting, we select real-world benchmarks covering traffic flow, speed, electricity consumption, and meteorological data, as detailed in Table These datasets exhibit substantial heterogeneity in spatial granularity (e.g.. city regions vs. sensors), temporal resolution (e.g., 5-minute vs. hourly intervals), and prediction targets (e.g., speed vs. volume), posing challenging testbed for cross-task generalization. This diversity enables rigorous evaluation of FactoST 's adaptability under both short-term and long-term forecasting settings.
23

---

Table 7: List of evaluation spatio-temporal datasets.
Dataset
Category
Features
Sample rate
Time span (Y/M/D)
Time Points
PEMS03
Traffic flow
358
5 minutes
2018/09/01
2018/11/30
26208
PEMS04
Traflic flow
307
5 minutes
2018/01/01
2018/02/28
16992
PEMS07
Traffic flow
5 minutes
2/05/01001
1/08/33
28224
PEMS08
Traftic flow
170
5 minutes
2016/07/01
2016/08/31
17856
PEMS-BAY
Traffic speod
325
5 minutes
2017/01/01
2017/06/30
52116
METR-LA
Traffic speed
minutes
201 2/03/00011
2012/06/27
34272
ETTh2
Transformer temperature
hour
201 6/06/011 2018/06/26
14400
Electricity
Electricity consumption
321
hour 2012/01/01
- 2014/122/1
26304
Weather
Meteorological data
minutes
2020/01/01 2011//11011
52696
A.1.2 Model Architecture and Hyperparameters
We implement FactoST using PyTorch, and all experiments are conducted on high-performance GPU servers. The architecture consists of three encoder layers and three decoder layers, with 16 attention heads and latent dimension d 128. Input sequences are processed using a patching mechanism with patch size of 12, and the dropout rate is set to 0.2 to prevent overfitting. The feed-forward network within each Transformer layer has hidden dimension of 512.
Pretraining. During pretraining, we use the Adam optimizer with an initial learning rate of x 10-1, and apply StepL. decay the learning rate by fixed factor every few epochs, improving convergence. The model is equipped with N, domain prompt learning vectors, each of dimension 128, and in supervised prediction tasks, both the input length and target forecasting horizon fixed at 96 (The length can be set to any value, which is the maximum supported step length, here we 96 for downstream comparison). For spectral consistency modeling, the number of augmented patches is set to Kę = 4. Pretraining is performed with a large batch size of 16,384 to ensure stable optimization.
Fine-tuning. During fine-tuning, we adopt a learning rate of x 10 The lookback window is set to 12 (short-term) or 96 (long-term), with matching prediction horizons. The number of domain prompt tokens (N, 3) and patching configuration remain unchanged from pretraining. A top-k selection (k: 3) applied during domain prompt matching enhance generalization. Additional configuration includes memory replacement ratio of 0.3; memory size of 0.2 relative to total capacity; spatio-temporal identifier embedding dimension of 32; maximum delay step A 3.
A.2 Evaluation Metrics
We use commonly used regression metrics, Mcan Absolute Error (MAE) and Root Mean Squared Error (RMSE), measure the prediction performance. Suppose Y = Y1,..., Ya ground truth for real spatio-temporal data, Y Y ..., YN are the predicted values by the model, and N is the number of total testing samples, These two metrics can be formulated as follows:
1 N N
N Σ Y1
RMSE(Y, 8)
Σ
Y MAE v)
= N
Y
(6)
A.3 Baselines and Implementation
All baseline models are evaluated within unified framework to ensure fair comparisons. models are assessed using standardized metries (MAE and RMSE). Hyperparameters are either set to default values reported in the original papers or tuned via grid search on the validation set. Below, we detail the implementation strategics for each category of baselines.
24

---

A.3.1
Time Series (TS) Expert Models
The following TS expent models are implemented using the BasicTS framework Ihttps://githee com Satm://estaallcogggea//Ba/ssiii to ensure consistency in preprocessing, training, and evaluation:
TimeMixer 59
MLP-based model with desoppecomposable mixing and future multi-pre predictor mixing, enabling multiscale information fusion from both microscopic and macroscopic views.
PatchTST 42
Transformer-based model that treats time series as a sequence of patches, enabling effective long-term forecasting by capturing local and global patterns.
DLinear 71:
simple yet effective linear model that decomposes time series into trend and resid- ual components, followed by independent modeling of each component for accurate forecasting.
Informer
An efficient Transformer variant with self-attention compression and generative decoder design, tailored for long sequence time series forecasting.
79
A.3.2
(ST) Expert Models
The following ST expert models are also integrated into the BasicTS framework, Graph structures follow original designs, utilizing distance-based learned adjacency matrices where applicable.
Proposes linear STGNN, first extracts long sequence input into low representation, then uses global GCN to capture spatial features, effective for large sensor node scenarios.
BigST [0
STAFformer [37] Utilizes spatial-temporal adaptive embeddings enhance the representation learning capability of Transformers for traffic forecasting tasks.
STID 48 Introduces spatial-temporal identity vectors into the Transformer architecture capture node-specific temporal dynamics and spatial dependencies.
Decouples spatial and temporal dependencies using separate graph convolution and recurrent modules for improved modeling of complex spatio-temporal interactions.
D2STGNN 49)
A.3.3 Time Series Foundation Models (TSFMs)
For TSFMs, we adapt the official implementations align with our benchmarking protocol:
T 10 A large-scale pretrained decoder-only time series foundation model developed by Google Research, capable of high-accuracy univariate forecasting across diverse domains and frequencies. The implementation obtained from the official repository (https://gitte comm/google-reserrree ttt/ttimmssmm.m and we fine-tune them using context lengths and forecast horizons consistent with our experimental setup. The checkpoint of the model we use comes from https://huggingfaae. co/ hotg///oolleeeoooo beroooggee//mmsmmmmmooo 1. 0-200m
large-scale pretrained encoder-only time series foundation model developed by Salesforce AI Research, designed to deliver universal forecasting capabilities across diverse domains, frequencies, and variable types. The implementation is obtained from the official repository https: http://.w...aaleeeeeeeeeeeeraaaa iitt The checkpoint of the model we use comes from http:///w/////gggggooo.o.oooo/ea 1-------------------- Rase
Moirai 62]:
A.3.4
Foundation Models (STFMs)
We evaluate two recent STFMs:
UniST 69 A universal STFM empowered by prompt learning, pretrained on multiple ur- ban scenarios to achieve strong generalization. Official codebase (https://github com/ tsinghua-
ialllla--iib-- fib- lebb-lab/ie lab/UniST supports fixed horizon configurations only 6-step prediction. We retrain on 13 datasets from the original release to support 12 and 96-step forecasting scenarios.
OpenCity 33 A versatile STFM that supports zero-shot and few-shot forecasting across diverse city-level applications. Integrated into our pipeline using the checkpoint spencity-puus .thh, with adapter layers introduced to enable efficient few-shot adaptation to new datasets.
25

---

A.4 More Results
A.4.1 Model Size Analysis
As shown in Figure 9 we investigate the effect of model capacity by varying the number of Trans- former layers in FactoST's temporal backbone (3.0M - 4.3M parameters) on ETTh2 long-term forecasting. Zero-shot performance improves steadily from 1 to 3 layers and then platcaus, indicating diminishing returns from deeper architectures. In the 10p few-shot setting, performance peaks at layers and slightly degrades with 5-7 layers, with training logs revealing signs of vveriiting--ikee due to increased depth without proportional increases in regularization hidden dimensionality. These results suggest that moderate model capacity is optimal under data-limited conditions, aligning with the principle of Occam's 's razor in transfer learning.
A.4.2 Pretraining Data Scalability
Figure shows the impact of pretraining corpus size on generalization. We train FactoST on 20% to 100% of Monash dataset and evaluate on ET long-temn forecasting. Performance improves monotonically with more pretraining data, confirming that FactoST effectively leverages larger and more diverse temporal corpora. Notably, even at 100% data (13M points), FactoST remains far below the scale of leading foundation models (e.g., TimesFM: 100B), suggesting substantial headroom for improvement with access to richer and high quality pretraining sources.
Volume
Volume
Model Size
Model
1975
8.5
1ong
MAA
490
0.307
0,509
0.509
0.50
3933
0.744
0.390
0.366
5.5
0,555
0,50
0.335
1.705
0.566
0.311
0.30
0.389
0.500
1.555
6355
1900
10400
sman
74
12.00
10.00
10/00
704
Figure 8: Pretraning data scaling analysis.
Figure 9: Model size analysis.
A.4.3 Pretraining Objective Ablation
Our pretraining objective combines future prediction loss and spectral consistency loss with equal weighting: Cpretrain L + Lype-. While Cpeed captures temporal dynamics through supervised fore- casting, Lypee enables sulf-vsuee modeling of multi multi-frequency patterns via spectral consistency learning across frequency-isolatted views. As shown in Table removing Cypec degrades zero-shot MAE/RMSE by 3.54% but only slightly affects few-shot performance (10.78%%00877%) This indicates that multi-frequency spectral consistency provides complementary, non-redundant signals that are especially crucial when no target-domain supervision is available. In zero-shot transfer, fosee endows the model with robust spectral inductive biases that generalize across domains. During few-shot fine-tuning, limited labels allow the model to partially recover domain-speciff patterns, reducing reliance on Lypec- The modest few-shot gain likely reflects the moderate domain shift in our benchmarks; expect Lopee to yield larger benefits under stronger distributional shifts (e.g.. cross-city or moos--modality transfer).
Table 8: Ablation study on the pretraining objective (Copec) using ETTh2 long-term forecasting. Removing the spectral consistency loss degrades both zero-shot and few-shot performance.
Model Variants
Zero-shot (MAE/ RMSH)
Few-shot (MAE/ / RMSE)
FactoST
0.395 / 0.588
0.383 / 00.578
FactoST w/o Exp
0 / 0.619
0.386 0.583
(%) MAE/ RMSE
+3.54% / +5.27%
+0.78% +0.87%
26

---

Table 9: Impact of Time Computing (TTC) long-tere tergtterm forecasting performance. FactoST with TTC consistently improves over the base model across all datasets.
Datasel
FactoST
FactoST w/ TTC
À (%)
MAE
RMSE
MAE
RMSE
MAE
RMSE
PEMS -3
123.60
159.38
113.62
144.80
-8.07%
-9.15%
PEMS
152.27
189.35
143.11
-7.05%
PEMS-07
172.99 99
214.00
156.07
190.59
-9.78%
-10.94%
PEMS
129.57
163.62
121.43
-7.35%
PEMS- Bay
7.10
12.01
6.12
10.47
-13.80%
-12.82%
Average
117.11
147.67
108.07
-8.79%
A.4.4 Zero-Shot Enhancement
To address the inherent limitations of zero-shot forecasting, we integrate Test-Time Computing (TTC) [9] into FactoST lightweight online adaptation mechanism that refines predictions during inference without retraining. TTC maintains a FIFO memory queue of recent inputs, predictions, and (pseudo) labels, constructs a frequency-doman and updates the calibrator using only historical predictions to avoid temporal leakage. As shown in Table TTC consistently improves long-term zero-shot performance across datasets, reducing MAE and RMSE by 7.72% and 8.79% on average. Future work includes: (1) spatial meta-learning to capture universal topological priors (e.g., distance decay) 22 ; (2) semantic-enhanced spatial
calibrator using FFT-based amplitude/phase offsets,
embeddings via external knowledge (e.g.. LLM-ddrived geolocation representations) [2]]; and (3)
cross-domain latent alignment bridge spatial representation gaps [55].
A.4.5 Temporal Feature Granularity Analysis
FactoST supports flexible multi-scale periodicity modeling through its spatio-temporal metadata fusion (STMF) module. While our main experiments use time_of. lim_____dayy__ _ay (24) and day_of_ _eee (7) identifiers identifies--refecting the dominant daily/weekly cycles in high-frequency traffic datasets like PEMS- 03-the architecture readily accommodates longer-term patterns (e.g., monthly, yearly) by simply extending the temporal feature set, without any architectural changes. To validate this flexibility and assess the impact of temporal granularity, we replace daily/weekly features with month_of __yer (12) in few-shot setting on PEMS-03. As shown in Table TO performance degrades significantly: short- term MAE increases by 12.94% and long-term MAE by 15.20%. This confirms that temporal feature design must align with the intrinsic frequency of the data principle our framework inherently supports through aong-and-lay metadata injection.
Table 10:
Impact of temporal feature granularity on PEMS-03 few-shot forecasting. Using coarse- grained monthly features harms performance due to mismatch with high-frequency data dynamics.
Temporal Features
Short-lerm (MAE / RMSE)
Long-term (MAE RMSE)
time day & day_of_veek nonth, of year
17.54/ 28.10
28.57 46.78
19.81 / 31.37
32,91 / 52.01
(S) MAE/ RMSE
+12.94% 1+16.64%
+15.20% 111.18%
B
Broader Impacts
Our work introduces a factorized framework for spatio-temporal foundation models that enhances efficiency, generalization, and cross-domaie adaptability. By decoupling universal temporal pretrain- ing from lightweight spatio-temporal adaptation, our approach significantly reduces computational cost and enables rapid few-shot deployment -making it well-suited for real-world applications with limited labeled data or constrained resources. The proposed method has the potential to benefit high- impact domains such as urban planning, traffic optimization, climate modeling, energy forecasting, and public health surveillance- -anenceee---eaaae---ee where accurate, scalable, and transferable spatio-temporal prediction is crucial. Furthermore, the modular design promotes sustainable Al development by minimizing redundant large-scale pretraining and reducing overall energy consumption.
27

---

As this work primarily focuses on scientific research and technical innovation in -emporal modeling, it does not present clear negative societal impacts. Instead, contributes to the development of more accessible, efficient, and environmentally responsible foundation models for real world urban dynamics with both spatio and temporal characteristics.
B.1 Limitations
Our work focuses on separating temporal pretraining from spatial adaptation to enable efficient and generalizable spatio-tempore modeling. While this factorized design offers strong empirical performance and flexibilityy, several limitations point to important directions for future research:
Spatial modeling impedes zero-shot generalization. Our results reveal key insight: explicit spatial podeling--sspccillly when baked into pretraining- cross-domain transfer because spatial structures (e.g., graph topology, sensor layout) are highly domain-specific. In contrast, temporal-only pretraining (as in FactoST UTP stage or TimesFM) achieves superior zero-shot performance, confirming that universal temporal patterns, not spatial priors, drive generalization. This explains why specialized STEMs (e.g.. UniST, OpenCity) underperform or even fail catas- trophically on unseen domains. While our factorized design avoids this pitfall by deferring spatial adaptation to the lightweight adapter, zero-shot spatio-temporal forecasting remains inherently particcllary for long hiii/ons--iighiightingg the need for complementary techniques (e.g., test-time computing
(9) to further bridge the domain gap.
The current framework assumes fixed node sets and static spatial topologies (c.g., traffic sensors or weather stations). It is not designed to handle scenarios where spatial units are added, removed, or reconfigured over time (c.g., mobile sensors, evolving infrastructure, or ad-hoc networks). Extending FactoST to support zero-shot spatial generalization such as generating embeddings for unseen nodes or adapting to changing graph structures [31 81
Challenges with dynamic and open-world spatial structures.
remains an open challenge.
Dependency on pretraining corpus diversity. Although our adapter-based design reduces compu- tational overhead, the universal temporal backbone still relies on the breadth and representativeness of the pretraining data 775 755. In domains with strong physical laws or complex spatial couplings (e.g., power grids), the model may lack necessary inductive biases, limiting transferability.
Limited integration of exogenous variables. Our current framework does not explicitly model external factors such as weather, events, or policy interventions -critical covariates in many real-world forecasting tasks [6]. Developing mechanisms to incorporate and adapt such exogenous signals in a few-shot manner an important direction for enhancing practical utility.
Static adapter composition. The adapter S currently applies a fixed set of modules (STMF, STE, HDA, CMR) regardless of input characteristics. A more intelligent system could enable adaptive model composition: by analyzing temporal stability, spatial heterogeneity, or domain shift, it could dynamically choose between full apatio-temports modeling, temporal-onjj inference, specialized lightweight modules--mproowe both robustness and efficiency [50]
Suboptimal fine-tuning protocols. Current adaptation uses uniform gradient updates on the adapter, which may underutilize pretrained knowledge and risk catastrophic forgetting. Parameter- efficient strategies such as prompt tuning, selective layer retraining, or regularization-wwan updates-coule better preserve temporal priors while enabling efficient spatial adaptation 45
These limitations point to several promising directions for future work: (1) rethinking spatial modeling avoid domain-specific biases in pretraining. (2) enabling adaptation to dynamic or unseen spatial configurations, (3) enriching temporal foundations with exogenous context, (4) leveraging more efficient and stable fine-tuning strategies, and (5) developing hybrid inference mechanisms (e.g., test-time adaptation) to bridge the zero-shot performance gap. Addressing these challenges will be key to building truly robust, scalable, and practical spatio-temporal foundation models.
B.2 Pseudocode of FactoST
For reproducibility, we present the detailed pseudocode of FactoST in Algorithm which concisely summarizes the two-stage learning paradigm: universal temporal pretraining (UTP) followed by lightweight apatio-tempone adaptation (STA).
28

---

Algorithm 1 FACTOST: Factorized Temaiol Foundation Model
Require: Cross-domain ST datasets D { Y m()) where X E NRLLD y
R-XxPxD, and m(f) denotes ST metadata.
1: // Stage I: Universal Temporal Pretraining (UTP)
2: Initialize temporal backbone T' and domain prompts p E Rx
3: for each node-wise sequence x E LxL sampled from D do
4: // Mullii-frqquency augmentation
5: X < (x)
6: for 1 to Km do
7: Sample T; Uniform(0, [L/2] + 1), 14; Bernoulli(p)
Mask (2) x : = if 14 =
8:
xy|:7| T =0 pf =1
9:
10:
end for
Xaug < [x, [xm ]=]] apply patching -
12:
// Multi-domain prompting
x, < Linear Linear(Patch(x) (x)) E N xd
14: Compute attention: 8j = ||xe- prll2, a, = softmax(s;)
15: X + E a,P,; form Xetx = (Patch(x),
16: I/ Multi-task pretraining
17:
Licoon f |x - ReconHead (Tone (Xaug) (xaug)))l12
18:
y Preddeead ( (Tene (xtx))
19: Lyred < |ly 312 + Xx
xpl2 1
20: Update T and p with L Lrecon + Lapred
21: end for
22: // Stage II: Pentio-Temmpral Adaptation (STA)
23: Initialize lightweight adapter S with modules: STMF, STE, HDA, CMR
24: Freeze T; initialize memory buffer M <-
25: for each mini-batch {(X, Y m)) C D do
// Temporal feature extraction
z T(X) R.V x xd [via patching and encoder]
28:
// STMF: inject ST metadata
Hat Proj[[[E [EE (D2 (t(())
) E NxN'xd
30:
Hused + z Hat
31:
I/ STF: adaptive filtering
Compute low-rank affinities: S, = HH W/ E, w("), " S. -HH W E, (
32:
S. + Σ (6) (used Agg, (H())))))
W softmax(SS,,, St, W Tau)
35:
Hrefined LLayeNNrmm (Hrssd W)
36:
I/ HDA: hierarchical alignment
37:
K. Topk, (-1|x, Py 12))
). P&
E
38:
) Haligned f Hrefined + (r)j(X,)
A + uv X, - (1N, P JOA
39:
40
// CMR: continual replay
41:
Update memory buffer M (c.g., FIFO or reservoir sampling)
42:
Sample replay batch B, C M, mix with current batch B. x- 13 J 15,
43:
44:
I/ Adapter update
45:
Yom + S(Haligged; m)
46:
Update with Y -Yuu on B
47: end for
48: return Final model: F(X; = ((T()) m)
29


Reviews:
[{"review_id": "gKbWwvje3a", "reviewer": "Reviewer_rPHb", "summary": "This paper proposes a spatio-temporal foundation model, with its training process primarily divided into two stages: universal temporal pretraining and spatio-temporal adaptation, addressing challenges faced by existing spatio-temporal foundation models such as high computational costs.", "questions": "1. How does the proposed method effectively handle inputs with different temporal scales, such as 12 and 96 time steps?\n\n2. Does the spatio-temporal adaptation module participate in the pre-training process, or is it only involved in few-shot learning?\n\n3. During pre-training, are all models (including foundation models and expert models) (re)trained using the same pre-training dataset? Or do the foundation models require only few-shot learning?", "limitations": "yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n1. Compared to existing spatiotemporal models, the proposed method maintains excellent prediction performance while ensuring high computational efficiency.\n\n2. The proposed foundation model combines Universal Temporal Pretraining and novel Spatio-Temporal Adaptation components, enabling effective adaptation to few-shot tasks and making it applicable to a wide range of real-world scenarios.\n\n3. The experiments are comprehensive, validating the model's effectiveness through various aspects including few-shot learning, zero-shot learning, scaling laws, efficiency experiment and ablation studies.\n\n**Weaknesses**\n1. The analysis of the model's zero-shot experiments is limited, with the model's zero-shot performance being comparable to some existing foundation models. What are the potential reasons for this?\n\n2. In few-shot learning experiments, almost all test dataset task types are included in the pre-training set, which limits the validation of the model's cross-task performance.\n\n3. The experimental configuration details are insufficient. For example, specific configurations for both foundation and non-foundation models during the pre-training phase are not clearly described, and it's unclear how the proposed model adapts to predictions with different time steps.", "was_revised": true, "final_justification": "The concerns raised have been sufficiently addressed, including clarification on how the model handles inputs of varying lengths, as well as detailed explanations of the zero-shot and few-shot experimental settings and results. I consider this to be a high-quality paper, and therefore I will maintain my original score.", "rebuttal": "Thank you for recognizing the effectiveness, novel design, and rigorous evaluation of our work. Below we endeavor to address your questions.\n\n> **Q1: How does the proposed method effectively handle inputs with different temporal scales, such as 12 and 96 time steps?**\n> \n\n**A1:** FactoST employs a Transformer encoder-decoder architecture. During the UTP stage, it utilizes both MLP prediction heads and linear reconstruction heads to support multi-horizon forecasting (e.g., 12/96 steps), enabling robust temporal modeling. This flexibility is achieved through patch-based tokenization, where input sequences are divided into non-overlapping patches, allowing adaptive processing of both short-term and long-term inputs. In the subsequent STA stage, only the encoder and prediction heads are fine-tuned. **Horizon-specific prediction heads** are initialized based on the target sequence length, allowing the model to effectively manage varying temporal scales.\n\n> **Q2: Does the spatio-temporal adaptation module participate in the pretraining process, or is it only involved in few-shot learning?**\n> \n\n**A2:** The STA module participates **only** in few-shot fine-tuning and is **explicitly excluded from pretraining**. During UTP, FactoST learns universal temporal patterns without any spatial modeling. As detailed in Section 3.2, spatial information is injected later via lightweight adapters in the STA stage, preserving the pretrained temporal capabilities. This factorized learning approach is further explained in Q1/A1 of Reviewer UUgZ.\n\n> **Q3: During pretraining, are all models (including foundation models and expert models) (re)trained using the same pretraining dataset? Or do the foundation models require only few-shot learning?**\n> \n\n**A3:** No, the models are not retrained on the same pretraining dataset.\n\n- **Foundation Models (e.g., FactoST, UniST, Moirai):** These are pretrained on their respective, distinct cross-domain corpora (e.g., FactoST's UTP uses Monash; UniST uses grid data; TimesFM uses large-scale time series data). They are then evaluated via few-shot adaptation on downstream datasets.\n- **Expert Models (e.g., STID, PatchTST):** These are typically trained from scratch on individual target datasets, without any cross-domain pretraining.\n\nTherefore, foundation models undergo extensive pretraining on their specific corpora and are subsequently adapted (not retrained from scratch) via few-shot learning on target tasks.\n\n> **Q4 (Weakness): The analysis of the model's zero-shot experiments is limited, with the model's zero-shot performance being comparable to some existing foundation models. What are the potential reasons for this?**\n> \n\n**A4:** The zero-shot performance of STFMs is fundamentally limited by **domain-specific spatial heterogeneity**. Notably, FactoST (i.e., UTP without STA) and TimesFM, which **lack explicit spatial learning**, already **outperform** STFMs like OpenCity and UniST in zero-shot scenarios. This validates our core motivation: joint spatio-temporal modeling during pretraining can **hinder generalization** across domains with differing spatial structures.\n\nFurthermore, the pretraining corpora vary significantly in **domain** and **scale**:\n\n| **Model** | **Pretraining Corpus** | **Statistics** |\n| --- | --- | --- |\n| FactoST | Monash (5 domains, 15 datasets) | 13M time points |\n| OpenCity | 21 heterogeneous traffic datasets (10,110 regions) | 151.1M observations |\n| UniST | 21 multi-source grid datasets | - |\n| TimesFM | Large-scale real-world and synthetic time series data | 100B time points |\n| Moirai | LOTSA dataset | 27B time points |\n\nDespite its **significantly smaller scale**, FactoST's zero-shot performance remains **competitive**, demonstrating the effectiveness of our UTP's multi-frequency augmentation, multi-domain prompting, and multi-task pretraining.\n\nWe also acknowledge that there is significant room for improvement in zero-shot performance. To this end, we have explored future directions, such as **Test-Time Adaptation (TTA)**, which can further boost zero-shot results. For a detailed discussion on these potential extensions, please see **Q2/A2 of Reviewer jNeH**.\n\n> **Q5 (Weakness): In few-shot learning experiments, almost all test dataset task types are included in the pretraining set, which limits the validation of the model's cross-task performance.**\n> \n\n**A5:** Although the pretraining and few-shot datasets share overlapping semantic domains, they differ substantially in **spatial granularity** (e.g., regions vs. sensors), **temporal resolution** (e.g., hours vs. minutes), and **task objectives** (e.g., pedestrian count vs. traffic speed). This significant difference in data characteristics demonstrates the model's true cross-task generalization ability, even if the high-level semantics overlap. Detailed dataset statistics are provided in Appendix A.1.1.\n\nFurthermore, UTP's multi-frequency augmentation **explicitly enhances** temporal pattern diversity, improving cross-task transferability. The factorized design (UTP for universal patterns, STA for domain-specific adaptation) **inherently** improves cross-task adaptability compared to joint pretraining approaches.\n\n> **Q6 (Weakness): The experimental configuration details are insufficient. For example, specific configurations for both foundation and non-foundation models during the pretraining phase are not clearly described.**\n> \n\n**A6:** Comprehensive experimental configurations for both the UTP and STA stages are detailed in **Appendix A.1.2**, including model architectures, training procedures, and parameter settings (e.g., 3 encoder/decoder layers, 16 attention heads, latent dimension 128). For baseline models, hyperparameters followed the original papers' defaults or were tuned via grid search on validation sets (Appendix A.3). All baselines were evaluated within a **unified** framework using **identical** data splits to ensure **fair comparison**.\n\n---\n\nThank you again for your constructive feedback. We will incorporate all the suggested clarifications and improvements into the revised manuscript."}, {"review_id": "niX5MPNMVl", "reviewer": "Reviewer_cLs2", "summary": "The paper focuses on developing a spatio-temporal foundation model (STFM) that builds a unified model to capture spatial correlations across domains (e.g., power grids and urban road networks) while jointly learning spatial and temporal dependencies in a computationally efficient manner. In particular, the paper proposes a two stage paradigm for STFM, FactoST that decouples universal temporal pretraining (UTP) from spatio-temporal adaptation (STA).\n\nUTP aims to learn universal temporal patterns across diverse domains through multi-frequency augmentation (encouraging the model to align multi-frequency information across scales), multi-domain prompting (guiding the model with task-specific context using explicit spatial graphs), and multi-task pretraining (combining self-supervised reconstruction and supervised forecasting objectives). ​​STA finetunes the UTP backbone and adds four lightweight, compact adapters to effectively capture spatio-temporal dependencies. ST metadata fusion is used for spatial-aware feature alignment, ST filtering for sparse interaction modeling, hierarchical domain alignment to bridge domain gaps, and continual memory replay to mitigate knowledge forgetting.\n\nExtensive evaluations in a few-shot setting, including comparative and ablation studies, demonstrate the effectiveness of the proposed method compared to domain-specific expert models.", "questions": "Please refer to the points listed in the Weaknesses.", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n1. The paper identifies two key challenges in developing a spatio-temporal foundation model (STFM). First, spatial correlations vary across domains and second, jointly learning spatial and temporal dependencies is computationally expensive. To address these issues, the paper proposes a novel method, FactoST, which decouples universal temporal pretraining from spatio-temporal adaptation. This enables efficient learning and adaptation while preserving strong temporal capabilities without requiring full spatio-temporal pretraining.\n\n2. The paper is technically sound, well-motivated, and clearly addresses the research questions. It is supported by comprehensive experimental results, ablation studies, scaling analysis, and domain adaptation analysis. Extensive experiments on both short-term and long-term forecasting under few-shot settings demonstrate the effectiveness of FactoST compared to other foundational and expert models. It achieves lower Mean Absolute Error and Root Mean Squared Error with fewer parameters and shorter inference times than other state-of-the-art methods.\n\nWeaknesses\n\n1. What is the contribution of L_pred and L_recon during pretraining? Are there any weights associated with these losses in equation 3 and equation 4?\n2. Zero-shot evaluation is a crucial aspect of assessing foundational models, but FactoST has not been evaluated in this setting.\n3. The batch size of 16,384 was selected for pretraining to ensure stable optimization. However, what does \"stability\" specifically refer to in this context? Given the large and computationally intensive nature of this batch size, was the impact of different batch sizes on pretraining and fine-tuning performance explored or evaluated?\n4. Abbreviation MAE used (line 11) before its full form Mean Absolute Error (line 547).\n5. On line 262 Mean Square Error (MSE) is used instead of MAE.", "was_revised": true, "final_justification": "Most of my concerns regarding the contribution of ```L_pred``` and ```L_recon``` during pre-training, clarification on zero-shot evaluation and stability claims have been satisfactorily resolved. Therefore, I will raise my current score and support its acceptance.", "rebuttal": "Thank you for recognizing the good motivation, clear design, comprehensive evaluation, and unique innovation of our paper. Below are our responses to your major questions.\n\n> **Q1 (Weakness): What is the contribution of L_pred and L_recon during pretraining? Are there any weights associated with these losses in Equation 3 and Equation 4?**\n> \n\n**A1:** `L_pred` supervises future sequence prediction, capturing both short-term dynamics and long-term trends. `L_recon` enhances multi-frequency pattern modeling through masked frequency reconstruction. The two losses are **complementary**, so we adopt **equal-weighted** joint optimization (`L_pretrain = L_pred + L_recon`) without separate weighting.\n\nTo validate their effectiveness, we conducted an ablation study on zero-shot and few-shot long-term forecasting for ETTh2. Results show that removing `L_recon` **degrades** performance: zero-shot MAE/RMSE increase by 3.54% and 5.27%, respectively, while few-shot MAE/RMSE increase by 0.78% and 0.87%. This **highlights** the importance of **self-supervised multi-frequency reconstruction** for learning robust temporal representations.\n\n| **Model Variants** | **Zero-shot (MAE / RMSE)** | **Few-shot (MAE / RMSE)** |\n| --- | --- | --- |\n| FactoST | 0.395 / 0.588 | 0.383 / 0.578 |\n| FactoST w/o L_recon | 0.409 / 0.619 | 0.386 / 0.583 |\n| Δ (%) MAE / RMSE | +3.54% / +5.27% | +0.78% / +0.87% |\n\n> **Q2 (Weakness): Zero-shot evaluation is a crucial aspect of assessing foundational models, but FactoST has not been evaluated in this setting.**\n> \n\n**A2:** Zero-shot evaluation is indeed crucial and **has been performed**. Comprehensive results are presented in **Appendix A.4 (Table 6)**, covering both short- and long-term forecasting. We further discuss the **limitation** of the zero-shot capability in Q4/A4 of Reviewer rPHb and propose some **future directions** in Q2/A2 of Reviewer jNeH. \n\n> **Q3 (Weakness): The batch size of 16,384 was selected for pretraining to ensure stable optimization. However, what does \"stability\" specifically refer to in this context? Given the large and computationally intensive nature of this batch size, was the impact of different batch sizes on performance explored?**\n> \n\n**A3:** \"Stability\" refers to a balance between **training speed** and the **observability** of the optimization process. We tested batch sizes of 8,192, 16,384, and 24,576. All converged to similar loss values with **no significant impact** on final downstream performance. We chose 16,384 because: 8,192 resulted in too many iterations (slow convergence), while 24,576 resulted in too few iterations, making it difficult to monitor loss changes. Thus, 16,384 provided the optimal balance for managing the training process.\n\n> **Q4 (Weakness - Typo/Misc): Abbreviation MAE is used (line 11) before its full form \"Mean Absolute Error\" (line 547). On line 262, \"Mean Square Error (MSE)\" is used instead of MAE.**\n> \n\n**A4:** **Corrections have been made in the revised manuscript:** \"Mean Absolute Error (MAE)\" is now defined at its first appearance (line 11), and the typo on line 262 (MSE corrected to MAE) has been fixed. All technical terms are now consistently defined upon first use. \n\n---\n\nThank you for your meticulous suggestions. We will address all your comments, including the experimental details and typographical corrections, in the revised manuscript."}, {"review_id": "fs6YNGUmEv", "reviewer": "Reviewer_jNeH", "summary": "FactoST proposes a two-stage factorized learning framework for Spatio-Temporal Foundation Models (STFMs), decoupling universal temporal pretraining (UTP) from domain-specific spatio-temporal adaptation (STA). The first stage learns a spatially-agnostic temporal backbone via multi-frequency reconstruction and domain-aware prompting. The second stage introduces a lightweight adapter that injects spatial awareness through metadata fusion, sparse interaction modeling, hierarchical domain alignment, and continual memory replay. Experimental results show that FactoST outperforms existing STFMs in both accuracy and efficiency across multiple domains.", "questions": "1. The UTP dataset spans six major domains, but data quantity and quality may vary significantly across them (e.g., energy vs. health). Have the authors investigated how such imbalances affect generalization performance?\n2. The paper notes the limited zero-shot capabilities of current STFMs. While FactoST focuses on few-shot adaptation, can the authors suggest directions for improving zero-shot transfer?", "limitations": "The paper briefly mentions limitations related to dependency on the pre-training corpus and handling evolving spatial structures. A more thorough discussion on how the framework might be extended to address these challenges would strengthen the work and guide future research.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "**Strengths:**\n\n- As the first work to explicitly decouple temporal and spatial learning in STFMs, this approach introduces a conceptually clear and potentially influential paradigm for future research.\n- The method strikes a good balance between building a general-purpose temporal backbone and enabling efficient, domain-specific adaptation — making it well-suited for complex real-world spatio-temporal scenes.\n- Experimental results are consistently strong across diverse domains and forecasting tasks. Ablation studies provide clear insight into the role of key components.\n- The paper is clearly written, with effective visualizations that aid in conveying the model architecture and workflow.\n\n**Weaknesses:**\n\n- The impact of domain imbalance or data quality variations in the UTP phase is not analyzed, which could affect the robustness of the learned representations.\n- While the proposed adapter improves efficiency, the potential gains from more complex spatial modeling are not fully explored.", "was_revised": true, "final_justification": "The author's rebuttal addressed all of my concerns. I will maintain my current positive score and support its acceptance.", "rebuttal": "Thank you for recognizing our work. Below, we hope to resolve some of your doubts.\n\n> **Q1: The UTP dataset spans six major domains, but data quantity and quality may vary significantly across them (e.g., energy vs. health). Have the authors investigated how such imbalances affect generalization performance?**\n> \n\n**A1:** FactoST addresses potential domain imbalance through **dual** strategies:\n\n- **Data-level:** For domains with limited data, multi-frequency augmentation is applied to synthetically **expand temporal patterns**. Consistent preprocessing (outlier removal, gap handling) is applied across all data to ensure quality consistency.\n- **Model-level:** A multi-domain prompting mechanism (Equation 2) dynamically adjusts the influence of different domains using **adaptive** weights (α_j). Ablation experiments (Figure 3) and visualizations (Figure 6) demonstrate its **effectiveness** in learning domain-specific knowledge, **mitigating** the impact of imbalances and improving generalization.\n\n> **Q2: The paper notes the limited zero-shot capabilities of current STFMs. While FactoST focuses on few-shot adaptation, can the authors suggest directions for improving zero-shot transfer?**\n> \n\n**A2:** To explore directions for improving zero-shot performance, we integrated the **Test-Time Computing (TTC)** method proposed in [1] into our framework. This approach performs **online adaptive optimization** during the **testing** phase to correct model predictions. It works as follows:\n\n1. It maintains a FIFO memory queue to cache gradually predicted inputs, predictions, and ground-truth labels during testing;\n2. A calibrator is constructed to transform queue-based historical predictions into the frequency domain via FFT, using group-learned amplitude/phase offsets to dynamically correct non-stationary temporal deviations;\n3. Loss is calculated solely from queue-sampled historical predictions to update calibrator parameters, strictly preserving temporal consistency to prevent information leakage.\n\nThe integration of TTC **effectively** boosts long-term zero-shot performance, as shown in the results below. On average, it **reduces** MAE and RMSE by 7.72% and 8.79%, respectively, demonstrating that online adaptation is a viable direction.\n\n| Dataset | FactoST (MAE/RMSE) | FactoST w/ TTA (MAE/RMSE) | Δ (%) MAE / RMSE |\n| --- | --- | --- | --- |\n| PEMS-03 | 123.60 / 159.38 | 113.62 / 144.80 | -8.07% / -9.15% |\n| PEMS-04 | 152.27 / 189.35 | 143.11 / 175.99 | -6.01% / -7.05% |\n| PEMS-07 | 172.99 / 214.00 | 156.07 / 190.59 | -9.78% / -10.94% |\n| PEMS-08 | 129.57 / 163.62 | 121.43 / 151.59 | -6.28% / -7.35% |\n| PEMS-Bay | 7.10 / 12.01 | 6.12 / 10.47 | -13.80% / -12.82% |\n| Average | 117.11 / 147.67 | 108.07 / 134.69 | -7.72% / -8.79% |\n\nFuture directions for zero-shot improvement include: (1) spatial meta-learning to encode universal topological features like distance decay [2]; (2) semantic-enhanced spatial priors using external knowledge [3]; (3) cross-domain latent space alignment to bridge spatial representation gaps [4].\n\n[1] Learning with Calibration: Exploring Test-Time Computing of Spatio-Temporal Forecasting, arxiv 2025\n\n[2] SPOK: tokenizing geographic space for enhanced spatial reasoning in GeoAI, IJGIS 2025\n\n[3] Geolocation Representation from Large Language Models are Generic Enhancers for Spatio-Temporal Learning, AAAI 2025\n\n[4] Unveiling the Inflexibility of Adaptive Embedding in Traffic Forecasting, Arxiv 2024\n\n> **Q3 (Weakness): While the proposed adapter improves efficiency, the potential gains from more complex spatial modeling are not fully explored.**\n> \n\n**A3:** STA employs **feature-level** dependency modeling: it dynamically identifies key correlations and suppresses noise through interactions between input features and learnable identifiers, while reducing computational complexity via low-rank projection and sparse strategies. \n\nIts advantages include: simultaneously capturing spatial proximity (S_s), temporal coherence (S_t), and delayed causalities (S_d), breaking the limitation of traditional models that only model single or two dimensions; dynamically adjusting the contribution weights of spatial, temporal, and delayed relationships through learnable parameters to adapt to different data characteristics.\n\nComplex spatial modeling remains underexplored, as more sophisticated graph modeling mechanisms (e.g., D2STGNN) incur **extremely high** resource consumption (OOM) in long-term prediction (Table 3), **restricting practical usability**. As shown in our added experiment (**Q2/A2 of Reviewer UUgZ Q2**), the current STA design is already **effective** and **architecture-agnostic**, offering a good **balance**. Future work will explore more lightweight methods for modeling high-order spatial dependencies. \n\n> **Q4 (Limitation Discussion): The paper briefly mentions limitations related to dependency on the pretraining corpus and handling evolving spatial structures. A more thorough discussion would strengthen the work.**\n> \n\n**A4:** We appreciate this valuable feedback. Addressing the limitations related to pretraining corpus dependency and evolving spatial structures is crucial for future work. Our current framework offers several avenues for extension:\n\n1. **Integration of Exogenous Variables**: Enhancing FactoST to effectively integrate and adapt to external factors (e.g., weather, events) in a few-shot manner is an important direction, as this capability remains underexplored in STFMs.\n2. **Handling Dynamic Spatial Nodes:** Future research could focus on making FactoST robust to real-time changes in the number and identity of spatial nodes (e.g., sensors added/removed), ensuring generalization to new environments while preserving pretrained knowledge.\n3. **Adaptive Model Composition**: Leveraging the factorized design, FactoST could be extended to adaptively select components based on data characteristics. By evaluating temporal stability and spatial distinctiveness, the system could dynamically choose between full spatio-temporal modeling, temporal-only modeling, or specialized modules for highly dynamic scenarios, enhancing robustness.\n\n---\n\nWe sincerely appreciate your thoughtful questions and valuable suggestions on future directions. We will expand the discussion on zero-shot learning and model limitations as recommended."}, {"review_id": "oubU3WhWaW", "reviewer": "Reviewer_UUgZ", "summary": "The paper proposes a two-stage factorized paradigm that decouples Universal Temporal Pretraining(UTP) from Spatio-Temporal Adaptation(STA). The first stage UTP learns general, domain-invariant temporal patterns through a graph-agnostic and scalable temporal backbone, enhanced by multi-frequency augmentation and domain-aware prompts. In the second stage, a compact adapter is introduced—either with the UTP backbone frozen or fine-tuned—to inject spatial awareness and domain specificity through metadata-aware token encoding, sparse low-rank dependency modeling, hierarchical prompt alignment, and replay-based training stabilization. This decoupling enables efficient learning and adaptation while preserving strong temporal capabilities without the need for costly joint spatio-temporal pretraining.", "questions": "1. Could the authors explicitly clarify that the specific research focus of this work is spatio-temporal forecasting? Moreover, the use of the term \"factorized\" is a bit unclear. It’s not obvious how the proposed two-stage framework reflects the idea of factorization. The pre-training and fine-tuning setup is a commonly used paradigm, and without a more formal or concrete explanation of how factorization is involved, the term might feel overstated.\n\n2. Is the second-stage SAT method exclusively applicable to GNN-based backbones? Has it been adapted to other types of architectures, such as the more commonly used Encoder-Decoder structures[1] in spatio-temporal tasks? If the method is only applicable to GNNs, I am concerned that the generalizability of this work may be limited.\n\n3. Please include more expert models [2] for a more comprehensive evaluation. Please explain why the proposed method does not achieve state-of-the-art performance on certain datasets. Is the dataset configuration used in the paper aligned with widely accepted evaluation protocols? Could the authors discuss the impact of the number of pre-training datasets on downstream performance, and provide the criteria used for dividing the pre-training and fine-tuning datasets?\n\n4. Could the authors explain why only daily and weekly periodicity are considered, while quarterly and yearly periodicities are not taken into account?\n\n5. Generally, smaller models tend to have faster inference speeds; so why does FactoST, despite having only a fraction of the parameters of Informer and FEDformer, exhibit slower inference speed than these models?\n\n\nReferences:\n\n[1] Tan, C., Li, S., Gao, Z., Guan, W., Wang, Z., Liu, Z., Wu, L. and Li, S.Z., 2023. Openstl: A comprehensive benchmark of spatio-temporal predictive learning. Advances in Neural Information Processing Systems, 36, pp.69819-69831.\n\n[2] Wang, Y., Wu, H., Dong, J., Liu, Y., Long, M. and Wang, J., 2024. Deep time series models: A comprehensive survey and benchmark. arXiv preprint arXiv:2407.13278.", "limitations": "The authors mention the limitations of their approach in the zero-shot setting; however, they do not address the fact that their validation is restricted solely to spatio-temporal forecasting tasks, nor that fine-tuning is conducted exclusively on the GNN backbone.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths: \n1. The paper proposes a novel foundational pre-training and fine-tuning paradigm for time series forecasting. It conducts pre-training on datasets from diverse domains and validates its effectiveness across a wide range of time series forecasting tasks.\n\n2. The proposed model, FactoST, achieves significantly improved performance compared to existing Spatio-Temporal Foundation Models (STFMs). For instance, FactoST outperforms previous STFM approaches by 46.4% in Mean Absolute Error (MAE), highlighting its potential in enhancing model accuracy and robustness.\n\n3. I believe this work is somewhat meaningful and would benefit from further clarification and discussion of its scope, generalizability, and technical novelty, particularly in its attempt to unify pre-training strategies across heterogeneous time series datasets.\n\n\nWeaknesses:\n1. I feel that the scope of the paper is somewhat too broad, and the authors should better define a more specific research focus. Since the paper mentions spatio-temporal modeling, it would be reasonable to expect coverage of various tasks, such as spatio-temporal classification, not just forecasting. However, the current experiments are limited to forecasting tasks. It would be helpful if the authors could clearly state that the focus of this work is on spatio-temporal forecasting. Moreover, the use of the term \"factorized\" is a bit unclear. It’s not obvious how the proposed two-stage framework reflects the idea of factorization. The pre-training and fine-tuning setup is a commonly used paradigm, and without a more formal or concrete explanation of how factorization is involved, the term might feel overstated.\n\n2. Is the second-stage SAT method exclusively applicable to GNN-based backbones? Has it been adapted to other types of architectures, such as the more commonly used Encoder-Decoder structures[1] in spatio-temporal tasks? If the method is only applicable to GNNs, I am concerned that the generalizability of this work may be limited.\n\n3. There are several issues in the experimental section. First, the baseline comparisons are insufficient; it is recommended to include more expert models [2] for a more comprehensive evaluation. Second, for datasets where the proposed method does not achieve state-of-the-art performance, no reasonable analysis is provided. Regarding the experimental setup, the criteria for dividing the pre-training and fine-tuning datasets are unclear. Furthermore, the impact of the number of pre-training datasets on performance is not discussed. Is the dataset configuration used in the paper aligned with widely accepted evaluation protocols? If so, the authors should clarify this and provide justification.\n\n4. In Section 3.2, Stage II: Spatio-Temporal Adaptation, the Spatio-Temporal Metadata Fusion (STMF) approach, where periodicity is modeled only for daily and weekly cycles, is unreasonable. Daily and weekly periodicity typically captures short-term fluctuations and repeating patterns. However, in real-world applications, many data are also influenced by long-term trends (such as inter-annual variations) and multiple periodicities (e.g., quarterly, yearly). For example, certain industries (such as tourism, electricity consumption, etc.) may exhibit seasonal changes or be impacted by economic activities at different times of the year. Can these factors truly be captured solely by modeling daily and weekly periodicity?\n\n5. Generally, the smaller the model's parameter size, the faster the inference speed. However, in Figure 5, FactoST has only a fraction of the parameters of Informer and FEDformer, yet its inference speed is still slower than theirs. The paper does not provide a reasonable explanation for this phenomenon.\n\n\nReferences:\n\n[1] Tan, C., Li, S., Gao, Z., Guan, W., Wang, Z., Liu, Z., Wu, L. and Li, S.Z., 2023. Openstl: A comprehensive benchmark of spatio-temporal predictive learning. Advances in Neural Information Processing Systems, 36, pp.69819-69831.\n\n[2] Wang, Y., Wu, H., Dong, J., Liu, Y., Long, M. and Wang, J., 2024. Deep time series models: A comprehensive survey and benchmark. arXiv preprint arXiv:2407.13278.", "was_revised": true, "final_justification": "The paper proposes a two-stace factorized paradiom that decouples universal temporal pretraining from spatio-temporal adaptations. This decouping enables eficient learning and adaptation while presering strong temporal capabilities without the need for costly joint spatio-temporal pretraining. The authors' response has adequately addressed my concerns, and I support the acceptance of the paper.", "rebuttal": "Thank you for your insightful questions and suggestions. We have revised and expanded our explanations to address your concerns.\n\n> **Q1 (Weakness - Scope/Clarity): Could the authors explicitly clarify that the specific research focus of this work is forecasting? Moreover, the term \"factorized\" is unclear—how does the two-stage framework reflect factorization?**\n> \n\n**A1:** Yes, FactoST is specifically designed for spatio-temporal forecasting. The term **factorized** refers to our core innovation: unlike existing STFMs that perform **joint** spatio-temporal **pretraining**, FactoST **decouples** learning into two sequential stages—first extracting general temporal patterns (UTP), then injecting domain-specific spatial context (STA). This separation explicitly factorizes temporal and spatial learning, in contrast to standard pretraining/fine-tuning where both dimensions are entangled from the start.\n\nAs shown in Table 1, this factorization offers three key advantages: (1) UTP eliminates spatial parameters, **reducing** pretraining **cost**; (2) STA adapters constitute only **a small fraction** of total parameters, enabling **fast** adaptation; (3) the framework maintains cross-domain **versatility** while achieving superior few-shot performance. Figure 1(c) illustrates how this design **avoids** the spatial generalization bottleneck of joint learning approaches. \n\n> **Q2 (Weakness - Generalizability): Is the STA method limited to GNN-based backbones? Has it been tested on other types of architectures, such as the more common encoder-decoder structures? If restricted to GNNs, the generalizability may be limited.**\n> \n\n**A2:** No, **STA is architecture-agnostic** and does **not require** GNN backbones. It operates at the feature level, using ST metadata fusion, dynamic ST filtering, hierarchical domain alignment, and continual memory replay to offer a **lightweight** spatio-temporal modeling solution. This design allows it to work with any temporal backbone architecture, as it processes feature embeddings **regardless** of the source model.\n\nTo validate this, we integrated STA into **PatchTST**, a non-GNN, time-series-focused model, and evaluated its few-shot performance on long-term tasks. The results (below) show that **PatchTST w/ STA** outperforms vanilla PatchTST across all datasets, confirming STA’s ability to generalize beyond GNNs. FactoST itself still achieves superior results, which we attribute to its pretrained temporal backbone’s universal pattern-learning capabilities. This demonstrates STA’s broad compatibility and strengthens the generalizability of our framework.\n\n| **Dataset** | **FactoST (MAE/RMSE)** | **PatchTST w/ STA (MAE/RMSE)** | **PatchTST (MAE/RMSE)** |\n| --- | --- | --- | --- |\n| PEMS-03 | 28.57 / 46.78 | 31.48 / 49.67 | 61.22 / 100.33 |\n| PEMS-04 | 42.04 / 64.89 | 43.69 / 65.17 | 70.71 / 104.00 |\n| PEMS-07 | 45.60 / 72.47 | 51.75 / 80.12 | 80.09 / 118.54 |\n| PEMS-08 | 35.69 / 56.15 | 44.00 / 66.85 | 57.31 / 87.33 |\n| PEMS-Bay | 2.96 / 6.21 | 3.57 / 7.72 | 4.32 / 9.22 |\n| ETTh2 | 0.358 / 0.561 | 0.362 / 0.569 | 0.943 / 1.609 |\n| Electricity | 0.265 / 0.409 | 0.279 / 0.414 | 0.442 / 2.747 |\n| Weather | 0.226 / 0.426 | 0.223 / 0.419 | 0.708 / 1.409 |\n\n> **Q3 (Weakness - Experiments/Baselines): Please include more expert models for a comprehensive evaluation. Why doesn’t the method achieve SOTA on all datasets? Are the dataset protocols standard? How does pretraining data volume affect downstream performance?**\n> \n\n**A3:**\n\n- **Additional Baselines:** As suggested, we include two recent baselines (TimeMixer, BigST) for a more comprehensive comparison. Results (below) show FactoST **tops** seven of the eight datasets, demonstrating its **competitiveness**. This aligns with our core finding that the factorized UTP+STA design outperforms both specialized and foundation models in most scenarios.\n    \n    \n    | **Dataset** | **FactoST (MAE/RMSE)** | **BigST (MAE/RMSE)** | **TimeMixer (MAE/RMSE)** |\n    | --- | --- | --- | --- |\n    | PEMS-03 | 28.57 / 46.78 | 51.87 / 75.56 | 47.86 / 71.52 |\n    | PEMS-04 | 42.04 / 64.89 | 52.37 / 80.23 | 58.44 / 86.67 |\n    | PEMS-07 | 45.60 / 72.47 | 54.92 / 82.12 | 67.75 / 100.45 |\n    | PEMS-08 | 35.69 / 56.15 | 58.68 / 86.76 | 45.10 / 65.53 |\n    | PEMS-Bay | 2.96 / 6.21 | 2.93 / 6.20 | 4.11 / 8.94 |\n    | ETTh2 | 0.358 / 0.561 | 1.164 / 1.797 | 1.013 / 1.646 |\n    | Electricity | 0.265 / 0.409 | 0.481 / 2.843 | 0.459 / 2.833 |\n    | Weather | 0.087 / 0.276 | 0.892 / 1.522 | 0.720 / 1.401 |\n    \n    We will cite the references you provided ([1] OpenSTL by Tan et al., 2023; [2] Deep Time Series Models by Wang et al., 2024) in the revised manuscript and incorporate their insights into our discussion of related work and future directions.\n    \n- **Performance Gap Analysis:** The minor performance gaps on specific datasets highlight important trade-offs in model design. On the **PEMS-07** (short-term), FactoST slightly underperforms D2STGNN, which explicitly decouples traffic flow into diffusion and inherent signals—a fine-grained modeling approach highly effective for its dense topology. However, this comes at high computational cost, causing OOM failures in long-term forecasting and limiting practicality. On datasets like **ETTh2** that lack spatial structure, Moirai performs better as it is is specifically optimized for general time series without spatial priors, while FactoST is designed for spatio-temporal settings. Nevertheless, FactoST uses significantly fewer parameters (1.4% of Moirai’s), runs faster, and still outperforms other STFMs such as UniST and OpenCity. Future work will explore lightweight spatial modeling for complex scenarios, with further discussions in Q2/A2 and Q4/A4 of Reviewer jNeH.\n- **Dataset Protocol:** We follow strict domain separation. UTP pretraining uses the Monash dataset collection; evaluation uses standard benchmarks (PEMS, etc.) with the widely adopted 6:2:2 train/validation/test split. For few-shot settings, we sample 10% of training data, resulting in a 0.6:2:2 effective split.\n- **Pretraining Data Impact:** To evaluate scalability, we conducted an ablation study on ETTh2 (long-term forecasting), varying the UTP pretraining data volume from 20% to 100%. Results show that performance improves **monotonically** with more data, confirming FactoST benefits from larger pretraining corpora. Notably, our current dataset (13M points) is much smaller than those used by existing foundation models (see Q4/A4 of Reviewer rPHb), indicating potential for further improvement with more high-quality data.\n    \n    \n    | **Data Volume** | **Zero-shot (MAE/RMSE)** | **Few-shot (MAE/RMSE)** | **Δ (%) Zero-shot vs. Full** | **Δ (%) Few-shot vs. Full** |\n    | --- | --- | --- | --- | --- |\n    | 20% | 0.405 / 0.598 | 0.394 / 0.592 | +2.53% / +1.70% | +2.34% / +1.72% |\n    | 40% | 0.402 / 0.596 | 0.393 / 0.591 | +1.77% / +1.36% | +1.82% / +1.55% |\n    | 60% | 0.400 / 0.594 | 0.391 / 0.589 | +1.27% / +1.02% | +1.30% / +1.21% |\n    | 80% | 0.397 / 0.589 | 0.389 / 0.586 | +0.51% / +0.17% | +0.78% / +0.69% |\n    | Full | 0.395 / 0.588 | 0.385 / 0.580 | - / - | - / - |\n\n> **Q4 (Weakness - Model Design): Could the authors explain why only daily and weekly periodicity are considered, while quarterly and yearly periodicities are not taken into account?**\n> \n\n**A4:** FactoST supports **flexible multi-scale** periodicity modeling. The current implementation focuses on daily/weekly cycles because they are dominant in the high-frequency traffic and energy datasets used for evaluation (e.g., 5-minute traffic data in PEMS-03). However, the architecture is designed to be general and can incorporate longer-term patterns (like monthly or annual cycles) by utilizing timestamp features. The STMF module can encode longer cycles by expanding time-of-day/day-of-week embeddings to include month-of-year/yearly features, with no architectural changes. We will **clarify this generality** in the revised manuscript.\n\nTo validate this flexibility, we conducted a few-shot experiment on PEMS-03 using \"month-of-year\" features instead of daily/weekly cycles. As shown below, performance **degrades significantly** (short-term MAE ↑12.94%, long-term MAE ↑15.20%), due to a **mismatch** between the dataset’s high temporal frequency and the long-term nature of monthly features. This outcome confirms that temporal feature selection must align with the data's inherent frequency—a design principle our framework **inherently supports**. \n\n| **Temporal features** | **short-term (MAE/RMSE)** | **long-term (MAE/RMSE)** |\n| --- | --- | --- |\n| time_of_day:24,day_of_week:7 | 17.54 / 28.10 | 28.57 / 46.78 |\n| month_of_year:12 | 19.81 / 31.37 | 32.91 / 52.01 |\n| Δ (%) MAE/RMSE | +12.94% / +11.64% | +15.20% / +11.18% |\n\n> **Q5 (Weakness - Efficiency): Smaller models usually have faster inference—why is FactoST slower than larger models like Informer and FEDformer?**\n> \n\n**A5:** While FactoST has fewer parameters than models like Informer/FEDformer, its inference is only slightly slower due to the necessary computations in the STA module—spatio-temporal metadata fusion, sparse interaction modeling via filtering, and hierarchical domain alignment—which adapt the universal temporal backbone to specific spatial contexts. This modest cost is justified by the substantial performance **gains** and the fact that FactoST remains significantly **faster** than joint STFMs (OpenCity, UniST). The trade-off strongly favors FactoST, as it achieves the **best** overall performance-efficiency **balance** among the evaluated models (Figure 5). \n\n---\n\nWe are deeply grateful for your comprehensive feedback, which has greatly helped us strengthen the paper's clarity and rigor. We will carefully revise the manuscript to include the additional experiments and clarifications you suggested."}]

Author rebuttal:

Meta-review: This paper introduces FactoST, a novel two-stage foundation model for spatio-temporal forecasting. The core contribution is a "factorized" paradigm that decouples universal temporal pre-training from spatio-temporal adaptation. This approach amis to circumvent the high computational costs and spatial generalization challenges inherent in existing models.

All reviewers support the acceptance of the paper, recognizing it as a strong contribution.

Strengths:
* novelty and impact: the central idea of decoupling temporal and spatial learning is conceptually clear, novel, and potentially influential for future research in spatio-temporal foundation models (jNeH, UUgz, cLs2).
* strong empirical performance: the results significantly outperform prior models and remain competitive with expert models in few-shot scenarios (rPHb, cLs2)
* comprehensive evaluation: the paper is supported by extensive experiments, including detailed ablation, scaling analysis, domain adaptation visualization (rPHb, cLs2)

Weaknesses:
Initial reviews raised several points for clarification. these include dthe scorep of the term "factorized" (UUgZ), the need for more baselines and zero-shot evaluation (cLs2, UUgZ), the generalizability of the adaptation module, details on handling different data scales and potential imbalances (rPHb, jNeH). 

The authors provided a comprehensive and convincing rebuttal that successfully addressed all major concerns. 
Hence, I recommend an acceptance for this paper.
Decision: Accept (spotlight)