Title: Learning to Factorize Spatio-Temporal Foundation Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: Spatio-Temporal Forecasting

Foundation Models
Abstract: Spatio-Temporal Foundation Models (STFMs) promise zero/few-shot generalization across various datasets, yet joint spatio-temporal pretraining is computationally prohibitive and struggles with domain-specific spatial correlations. To this end, we introduce FactoST, a factorized STFM that decouples universal temporal pretraining from spatio-temporal adaptation. The first stage pretrains a space-agnostic backbone with multi-frequency reconstruction and domain-aware prompting, capturing cross-domain temporal regularities at low computational cost. The second stage freezes or further fine-tunes the backbone and attaches an adapter that fuses spatial metadata, sparsifies interactions, and aligns domains with continual memory replay. Extensive forecasting experiments reveal that, in few-shot setting, FactoST reduces MAE by up to 46.4% versus UniST, uses 46.2% fewer parameters, and achieves 68% faster inference than OpenCity, while remaining competitive with expert models. We believe this factorized view offers a practical and scalable path toward truly universal STFMs. The code will be released upon notification.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a two-stage factorized paradigm that decouples Universal Temporal Pretraining(UTP) from Spatio-Temporal Adaptation(STA). The first stage UTP learns general, domain-invariant temporal patterns through a graph-agnostic and scalable temporal backbone, enhanced by multi-frequency augmentation and domain-aware prompts. In the second stage, a compact adapter is introduced—either with the UTP backbone frozen or fine-tuned—to inject spatial awareness and domain specificity through metadata-aware token encoding, sparse low-rank dependency modeling, hierarchical prompt alignment, and replay-based training stabilization. This decoupling enables efficient learning and adaptation while preserving strong temporal capabilities without the need for costly joint spatio-temporal pretraining.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n1. The paper proposes a novel foundational pre-training and fine-tuning paradigm for time series forecasting. It conducts pre-training on datasets from diverse domains and validates its effectiveness across a wide range of time series forecasting tasks.\n\n2. The proposed model, FactoST, achieves significantly improved performance compared to existing Spatio-Temporal Foundation Models (STFMs). For instance, FactoST outperforms previous STFM approaches by 46.4% in Mean Absolute Error (MAE), highlighting its potential in enhancing model accuracy and robustness.\n\n3. I believe this work is somewhat meaningful and would benefit from further clarification and discussion of its scope, generalizability, and technical novelty, particularly in its attempt to unify pre-training strategies across heterogeneous time series datasets.\n\n\nWeaknesses:\n1. I feel that the scope of the paper is somewhat too broad, and the authors should better define a more specific research focus. Since the paper mentions spatio-temporal modeling, it would be reasonable to expect coverage of various tasks, such as spatio-temporal classification, not just forecasting. However, the current experiments are limited to forecasting tasks. It would be helpful if the authors could clearly state that the focus of this work is on spatio-temporal forecasting. Moreover, the use of the term \"factorized\" is a bit unclear. It’s not obvious how the proposed two-stage framework reflects the idea of factorization. The pre-training and fine-tuning setup is a commonly used paradigm, and without a more formal or concrete explanation of how factorization is involved, the term might feel overstated.\n\n2. Is the second-stage SAT method exclusively applicable to GNN-based backbones? Has it been adapted to other types of architectures, such as the more commonly used Encoder-Decoder structures[1] in spatio-temporal tasks? If the method is only applicable to GNNs, I am concerned that the generalizability of this work may be limited.\n\n3. There are several issues in the experimental section. First, the baseline comparisons are insufficient; it is recommended to include more expert models [2] for a more comprehensive evaluation. Second, for datasets where the proposed method does not achieve state-of-the-art performance, no reasonable analysis is provided. Regarding the experimental setup, the criteria for dividing the pre-training and fine-tuning datasets are unclear. Furthermore, the impact of the number of pre-training datasets on performance is not discussed. Is the dataset configuration used in the paper aligned with widely accepted evaluation protocols? If so, the authors should clarify this and provide justification.\n\n4. In Section 3.2, Stage II: Spatio-Temporal Adaptation, the Spatio-Temporal Metadata Fusion (STMF) approach, where periodicity is modeled only for daily and weekly cycles, is unreasonable. Daily and weekly periodicity typically captures short-term fluctuations and repeating patterns. However, in real-world applications, many data are also influenced by long-term trends (such as inter-annual variations) and multiple periodicities (e.g., quarterly, yearly). For example, certain industries (such as tourism, electricity consumption, etc.) may exhibit seasonal changes or be impacted by economic activities at different times of the year. Can these factors truly be captured solely by modeling daily and weekly periodicity?\n\n5. Generally, the smaller the model's parameter size, the faster the inference speed. However, in Figure 5, FactoST has only a fraction of the parameters of Informer and FEDformer, yet its inference speed is still slower than theirs. The paper does not provide a reasonable explanation for this phenomenon.\n\n\nReferences:\n\n[1] Tan, C., Li, S., Gao, Z., Guan, W., Wang, Z., Liu, Z., Wu, L. and Li, S.Z., 2023. Openstl: A comprehensive benchmark of spatio-temporal predictive learning. Advances in Neural Information Processing Systems, 36, pp.69819-69831.\n\n[2] Wang, Y., Wu, H., Dong, J., Liu, Y., Long, M. and Wang, J., 2024. Deep time series models: A comprehensive survey and benchmark. arXiv preprint arXiv:2407.13278.\n\nQUESTIONS:\n1. Could the authors explicitly clarify that the specific research focus of this work is spatio-temporal forecasting? Moreover, the use of the term \"factorized\" is a bit unclear. It’s not obvious how the proposed two-stage framework reflects the idea of factorization. The pre-training and fine-tuning setup is a commonly used paradigm, and without a more formal or concrete explanation of how factorization is involved, the term might feel overstated.\n\n2. Is the second-stage SAT method exclusively applicable to GNN-based backbones? Has it been adapted to other types of architectures, such as the more commonly used Encoder-Decoder structures[1] in spatio-temporal tasks? If the method is only applicable to GNNs, I am concerned that the generalizability of this work may be limited.\n\n3. Please include more expert models [2] for a more comprehensive evaluation. Please explain why the proposed method does not achieve state-of-the-art performance on certain datasets. Is the dataset configuration used in the paper aligned with widely accepted evaluation protocols? Could the authors discuss the impact of the number of pre-training datasets on downstream performance, and provide the criteria used for dividing the pre-training and fine-tuning datasets?\n\n4. Could the authors explain why only daily and weekly periodicity are considered, while quarterly and yearly periodicities are not taken into account?\n\n5. Generally, smaller models tend to have faster inference speeds; so why does FactoST, despite having only a fraction of the parameters of Informer and FEDformer, exhibit slower inference speed than these models?\n\n\nReferences:\n\n[1] Tan, C., Li, S., Gao, Z., Guan, W., Wang, Z., Liu, Z., Wu, L. and Li, S.Z., 2023. Openstl: A comprehensive benchmark of spatio-temporal predictive learning. Advances in Neural Information Processing Systems, 36, pp.69819-69831.\n\n[2] Wang, Y., Wu, H., Dong, J., Liu, Y., Long, M. and Wang, J., 2024. Deep time series models: A comprehensive survey and benchmark. arXiv preprint arXiv:2407.13278.\n\nLIMITATIONS:\nThe authors mention the limitations of their approach in the zero-shot setting; however, they do not address the fact that their validation is restricted solely to spatio-temporal forecasting tasks, nor that fine-tuning is conducted exclusively on the GNN backbone.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-18"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nFactoST proposes a two-stage factorized learning framework for Spatio-Temporal Foundation Models (STFMs), decoupling universal temporal pretraining (UTP) from domain-specific spatio-temporal adaptation (STA). The first stage learns a spatially-agnostic temporal backbone via multi-frequency reconstruction and domain-aware prompting. The second stage introduces a lightweight adapter that injects spatial awareness through metadata fusion, sparse interaction modeling, hierarchical domain alignment, and continual memory replay. Experimental results show that FactoST outperforms existing STFMs in both accuracy and efficiency across multiple domains.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n- As the first work to explicitly decouple temporal and spatial learning in STFMs, this approach introduces a conceptually clear and potentially influential paradigm for future research.\n- The method strikes a good balance between building a general-purpose temporal backbone and enabling efficient, domain-specific adaptation — making it well-suited for complex real-world spatio-temporal scenes.\n- Experimental results are consistently strong across diverse domains and forecasting tasks. Ablation studies provide clear insight into the role of key components.\n- The paper is clearly written, with effective visualizations that aid in conveying the model architecture and workflow.\n\n**Weaknesses:**\n\n- The impact of domain imbalance or data quality variations in the UTP phase is not analyzed, which could affect the robustness of the learned representations.\n- While the proposed adapter improves efficiency, the potential gains from more complex spatial modeling are not fully explored.\n\nQUESTIONS:\n1. The UTP dataset spans six major domains, but data quantity and quality may vary significantly across them (e.g., energy vs. health). Have the authors investigated how such imbalances affect generalization performance?\n2. The paper notes the limited zero-shot capabilities of current STFMs. While FactoST focuses on few-shot adaptation, can the authors suggest directions for improving zero-shot transfer?\n\nLIMITATIONS:\nThe paper briefly mentions limitations related to dependency on the pre-training corpus and handling evolving spatial structures. A more thorough discussion on how the framework might be extended to address these challenges would strengthen the work and guide future research.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper focuses on developing a spatio-temporal foundation model (STFM) that builds a unified model to capture spatial correlations across domains (e.g., power grids and urban road networks) while jointly learning spatial and temporal dependencies in a computationally efficient manner. In particular, the paper proposes a two stage paradigm for STFM, FactoST that decouples universal temporal pretraining (UTP) from spatio-temporal adaptation (STA).\n\nUTP aims to learn universal temporal patterns across diverse domains through multi-frequency augmentation (encouraging the model to align multi-frequency information across scales), multi-domain prompting (guiding the model with task-specific context using explicit spatial graphs), and multi-task pretraining (combining self-supervised reconstruction and supervised forecasting objectives). ​​STA finetunes the UTP backbone and adds four lightweight, compact adapters to effectively capture spatio-temporal dependencies. ST metadata fusion is used for spatial-aware feature alignment, ST filtering for sparse interaction modeling, hierarchical domain alignment to bridge domain gaps, and continual memory replay to mitigate knowledge forgetting.\n\nExtensive evaluations in a few-shot setting, including comparative and ablation studies, demonstrate the effectiveness of the proposed method compared to domain-specific expert models.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1. The paper identifies two key challenges in developing a spatio-temporal foundation model (STFM). First, spatial correlations vary across domains and second, jointly learning spatial and temporal dependencies is computationally expensive. To address these issues, the paper proposes a novel method, FactoST, which decouples universal temporal pretraining from spatio-temporal adaptation. This enables efficient learning and adaptation while preserving strong temporal capabilities without requiring full spatio-temporal pretraining.\n\n2. The paper is technically sound, well-motivated, and clearly addresses the research questions. It is supported by comprehensive experimental results, ablation studies, scaling analysis, and domain adaptation analysis. Extensive experiments on both short-term and long-term forecasting under few-shot settings demonstrate the effectiveness of FactoST compared to other foundational and expert models. It achieves lower Mean Absolute Error and Root Mean Squared Error with fewer parameters and shorter inference times than other state-of-the-art methods.\n\nWeaknesses\n\n1. What is the contribution of L_pred and L_recon during pretraining? Are there any weights associated with these losses in equation 3 and equation 4?\n2. Zero-shot evaluation is a crucial aspect of assessing foundational models, but FactoST has not been evaluated in this setting.\n3. The batch size of 16,384 was selected for pretraining to ensure stable optimization. However, what does \"stability\" specifically refer to in this context? Given the large and computationally intensive nature of this batch size, was the impact of different batch sizes on pretraining and fine-tuning performance explored or evaluated?\n4. Abbreviation MAE used (line 11) before its full form Mean Absolute Error (line 547).\n5. On line 262 Mean Square Error (MSE) is used instead of MAE.\n\nQUESTIONS:\nPlease refer to the points listed in the Weaknesses.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a spatio-temporal foundation model, with its training process primarily divided into two stages: universal temporal pretraining and spatio-temporal adaptation, addressing challenges faced by existing spatio-temporal foundation models such as high computational costs.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n1. Compared to existing spatiotemporal models, the proposed method maintains excellent prediction performance while ensuring high computational efficiency.\n\n2. The proposed foundation model combines Universal Temporal Pretraining and novel Spatio-Temporal Adaptation components, enabling effective adaptation to few-shot tasks and making it applicable to a wide range of real-world scenarios.\n\n3. The experiments are comprehensive, validating the model's effectiveness through various aspects including few-shot learning, zero-shot learning, scaling laws, efficiency experiment and ablation studies.\n\n**Weaknesses**\n1. The analysis of the model's zero-shot experiments is limited, with the model's zero-shot performance being comparable to some existing foundation models. What are the potential reasons for this?\n\n2. In few-shot learning experiments, almost all test dataset task types are included in the pre-training set, which limits the validation of the model's cross-task performance.\n\n3. The experimental configuration details are insufficient. For example, specific configurations for both foundation and non-foundation models during the pre-training phase are not clearly described, and it's unclear how the proposed model adapts to predictions with different time steps.\n\nQUESTIONS:\n1. How does the proposed method effectively handle inputs with different temporal scales, such as 12 and 96 time steps?\n\n2. Does the spatio-temporal adaptation module participate in the pre-training process, or is it only involved in few-shot learning?\n\n3. During pre-training, are all models (including foundation models and expert models) (re)trained using the same pre-training dataset? Or do the foundation models require only few-shot learning?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-02"}