Title: On the Integration of Spatial-Temporal Knowledge: A Lightweight Approach to Atmospheric Time Series Forecasting
Venue: NeurIPS 2025
Accepted: yes
Keywords: Atmospheric Time Series Forecasting; Spatial-Temporal Position Embedding; Spatial-Temporal Correlations
Abstract: Transformers have gained attention in atmospheric time series forecasting (ATSF) for their ability to capture global spatial-temporal correlations. However, their complex architectures lead to excessive parameter counts and extended training times, limiting their scalability to large-scale forecasting. In this paper, we revisit ATSF from a theoretical perspective of atmospheric dynamics and uncover a key insight: spatial-temporal position embedding (STPE) can inherently model spatial-temporal correlations even without attention mechanisms. Its effectiveness arises from integrating geographical coordinates and temporal features, which are intrinsically linked to atmospheric dynamics. Based on this, we propose **STELLA**, a **S**patial-**T**emporal knowledge **E**mbedded **L**ightweight mode**L** for ASTF, utilizing only STPE and an MLP architecture in place of Transformer layers. With 10k parameters and one hour of training, STELLA achieves superior performance on five datasets compared to other advanced methods. The paper emphasizes the effectiveness of spatial-temporal knowledge integration over complex architectures, providing novel insights for ATSF.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a novel approach called STELLA (Spatial-Temporal knowledge Embedded Lightweight modeL for Atmospheric Time Series Forecasting), which challenges the prevailing assumption that complex architectures like Transformers are necessary for atmospheric time series forecasting (ATSF). The authors argue that spatial-temporal position embedding (STPE) can inherently model spatial-temporal correlations even without attention mechanisms, by integrating geographical coordinates and temporal features intrinsically linked to atmospheric dynamics. STELLA utilizes STPE and an MLP architecture instead of Transformer layers, achieving superior performance with only 10k parameters and one hour of training across five datasets compared to 16 advanced methods. The paper provides theoretical proof of STPE's effectiveness and emphasizes the importance of integrating spatial-temporal knowledge over complex model architectures.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n   （1）Efficiency: STELLA demonstrates significant advantages in efficiency metrics, including parameter counts, training time, and memory usage, making it suitable for environments with limited computational resources.\n   （2）Theoretical Support: The authors provide theoretical analysis and proofs to support the effectiveness and efficiency of STELLA, strengthening the credibility of their claims.\nWeaknesses:\n   （1）The innovation of this work is insufficient. On the one hand, a great many works on lightweight Transformer schemes have already been proposed. For example, PatchTST simplifies the Transformer by introducing linear transformations, which cuts down the number of parameters and computational complexity while delivering promising performance in time series tasks. In this context, the approach of STELLA, which replaces Transformer layers with an MLP, is not sufficiently novel despite its simplicity. The authors fail to effectively highlight the unique merits of their method in contrast to other lightweight Transformer models and provide an inadequate comparison to clarify its innovative contributions. On the other hand, a significant body of research has already been carried out on the integration of physical knowledge into models. For instance, physics - guided neural networks utilize physical laws as regularizers or constraints during model training to improve the prediction of physical systems.\n   （2）The model's performance improvement is rather insignificant.\n\nQUESTIONS:\n（1）The authors should provide a detailed comparison of their method with existing lightweight Transformer models and physics-integrated approaches. Highlight the unique advantages and novel contributions of STELLA in contrast to these prior works. Clarify the specific aspects in which STELLA advances the state of the art.\n  (2) The authors should conduct a thorough analysis to identify the reasons for the limited performance gain.\n  (3) The authors emphasize the significance of spatial - temporal correlation modeling. Existing approaches for spatial - temporal modeling typically fall into two paradigms: asynchronous spatial - temporal modeling (e.g., STTN, CLCRN) and synchronous spatial - temporal modeling (e.g., FourierGNN). The authors seem to adopt a synchronous modeling strategy. However, the paper falls short in elaborating on the details of the spatial - temporal modeling approach.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes STELLA and reveals that spatial-temporal knowlegde plays a pivotal role in atmospheric time series forecasting. The proposed method employs STPE and a simple MLP in place of Transformer layers, achieving SOTA performance with high efficiency (~10k parameters and a hour of training).\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nS1. The paper presents a novel approach to atmospheric time series forecasting by leveraging STPE with a simple MLP. The theoretical foundation is well-explained and the experimental results demonstrate STELLA's efficacy.\n\nS2. The paper is well-organized and clearly written.\n\nS3. The proposed model can significantly improve the efficiency and scalability to large scale ATSF tasks. The STPE module can also enhance the performance of existing models.\n\nS4. The paper uncovers the superiority of the spatial-temporal knowledge over complex architecture, which challenges the prevailing convention that atmospheric time series forecasting necessitates intricate designs.\n\nWeaknesses:\n\nW1. While the paper presents an interesting finding, the technical contribution is somewhat constrained due to the adoption of well-established components (positional embedding and MLP architectures).\n\nW2. The paper lacks an in-depth discussion on whether integrating STPE with complex architectures like Transformers and STGNNs could yield superior performance. if such combinations prove ineffective compared to MLPs, the underlying theoretical limitations should be analyzed.\n\nQUESTIONS:\nQ1. In the generalization experiments, STPE are only applied to Transformer-based models. Does it maintain effectiveness when applied to other architectures such as linear models and other MLP-based models?\n\nQ2. The theoretical analysis appears derived from PDEs of atmospheric motion. Have the authors considered implementing Neural ODEs as an alternative framework?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes STELLA, a lightweight atmospheric time series forecasting model that integrates spatial-temporal position embedding (STPE) into a simple MLP architecture. Theoretical analysis reveals that STPE can inherently capture spatial-temporal correlations, even without attention mechanisms. Extensive experiments on five real-world datasets demonstrate that STELLA achieves competitive performance with significantly fewer parameters and lower computational costs.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. This work bridges a critical gap in atmospheric time series forecasting through its incorporation of spatial coordinates and temporal information that prior works have consistently overlooked.\n2. This paper proposes a simple yet effective baseline model that outperforms existing models with high efficiency.\n3. The spatial-temporal position embedding (STPE) module is model-agnostic and can be integrated into other models to enhance their performance.\n4. This paper provides extensive experiments and rigorous theoretical analysis to validate the effectiveness of STELLA.\n\nWeaknesses:\n\n1. This work focuses on spatial correlations among stations but does not explicitly account for correlations among atmospheric variables, which might further improve predictive performance.\n2. While the experimental comparison covers substantial baselines, the evaluation would be strengthened by including comparisons with recent advanced time series forecasting models such as DUET [1] and FreDF [2].\n3. The paper would benefit from a clearer justification for using MLP as the backbone, particularly discussing its advantages over other architectures for this specific application.\n\n[1] Qiu X, et al. Duet: Dual clustering enhanced multivariate time series forecasting. KDD 2025.\n[2] Wang H, et al. FreDF: Learning to forecast in frequency domain. ICLR 2025.\n\nQUESTIONS:\n1. Have the authors explored alternative approaches for incorporating spatial-temporal knowledge besides STPE?\n2. What motivated the selection of MLP as the backbone, given its known limitations in capturing complex correlation?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
This paper presents STELLA, a spatial-temporal knowledge embedded lightweight model for atmospheric time series forecasting (ATSF). STELLA utilizes spatial-temporal position embedding with a simple MLP-based architecture. The paper provides theoretical foundations for its effectiveness and computational efficiency, with empirical validation across five real-world datasets demonstrating that STELLA achieves state-of-the-art performance while maintaining high efficiency. The approach to spatial-temporal knowledge integration offers novel insights for ATSF tasks.

STRENGTHS AND WEAKNESSES:
Strengths:

1. The authors introduce a novel method, STELLA, for ATSF. The model presents state-of-the-art performance on five different weather datasets.
2. STELLA has significantly fewer parameters, shorter training times, and reduced memory usage, enhancing the scalability for large-scale forecasting.
3. The authors provide theoretical guarantees for the model's predictive performance from the perspective of atmospheric motion.

Weaknesses:

1. For efficiency analysis, what is the specific advantages of N-independent parameters compared to other models with O(N) parameters (which is deemed acceptable in my perspective)?
2. The proposed method only models the position of stations while neglects the relative spatial relationships (e.g., adjacency between stations), which show promise for boosting performance and have been explored in spatial-temporal data mining tasks.
3. The paper does not address whether alternative efficient architectures (e.g., Mamba) could attain similar results.

QUESTIONS:
Please refer to the proposed weakness above.

LIMITATIONS:
Please refer to above reviews.
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-07-05