On the Integration of Spatial-Temporal Knowledge: A Lightweight Approach to Atmospheric Time Series Forecasting
Authors: Yisong Fu
Fei Wang
Zezhi Shao
Boyu Diao
Lin Wu
Zhulin An
Chengqing Yu
Yujie Li
Yongjun Xu
Venue: NeurIPS 2025 poster ()
Keywords: Atmospheric Time Series Forecasting; Spatial-Temporal Position Embedding; Spatial-Temporal Correlations
Primary area: machine_learning_for_sciences

Abstract: Transformers have gained attention in atmospheric time series forecasting (ATSF) for their ability to capture global spatial-temporal correlations. However, their complex architectures lead to excessive parameter counts and extended training times, limiting their scalability to large-scale forecasting. In this paper, we revisit ATSF from a theoretical perspective of atmospheric dynamics and uncover a key insight: spatial-temporal position embedding (STPE) can inherently model spatial-temporal correlations even without attention mechanisms. Its effectiveness arises from integrating geographical coordinates and temporal features, which are intrinsically linked to atmospheric dynamics. Based on this, we propose **STELLA**, a **S**patial-**T**emporal knowledge **E**mbedded **L**ightweight mode**L** for ASTF, utilizing only STPE and an MLP architecture in place of Transformer layers. With 10k parameters and one hour of training, STELLA achieves superior performance on five datasets compared to other advanced methods. The paper emphasizes the effectiveness of spatial-temporal knowledge integration over complex architectures, providing novel insights for ATSF.

On the Integration of Spatial-Temporal Knowledge: A Lightweight Approach to Atmospheric Time Series Forecasting
Fei Wangl.3, 1:2+, Zezhi Shao', Boyu Diao!.2, Lin Wu Zhulin An! Chengqing Yu?.2, Yujie Li Yongjun 2.2 Sttaee Key Laboratory of Al Safety, Institute of Computing Technology, Chinese Academy of Sciences UUniversity of Chinese Academy of Sciences (fuyisong24s, wangfei, shaozezhi, diaoboyu, wulin]@ict. ac. en, fanzhulin, yuchengqing22b, liyujie23s, xyj}@ict.ace ac.
Yisong F
Abstract
Transformers have gained attention in atmospheric time series forecasting (ATSF) for their ability capture global spatial-tempora correlations. However, their complex architectures lead to excessive parameter counts and extended training times, limiting their scalability to large-scale forecasting. In this paper, we revisit ATSF from theoretical perspective of atmospheric dynamics and uncover key insight: spatial-tempora position embedding (STPE) can inherently model spatial- temporal correlations even without attention mechanisms. Its effectiveness arises from the integration of geographical coordinates and temporal features, which are intrinsically linked to atmospheric dynamics. Based this, propose STELLA, a
knowledge Embedded Lightweight model for ASTF, utilizing only STPE and an MLP architecture in place of Transformer layers. With 10k parameters and one hour of training. STELLA achieves superior performance on five datasets compared other advanced methods. The paper emphasizes the effectiveness of spatial-temporal knowledge integration over complex architectures, providing novel insights for ATSF. The code is available https: httts:///tt thub. com/ htt/GestaatCooTeaa /STELA..
Introduction
Atmospheric time series forecasting (ATSF), such as weather forecasting and air quality prediction, is of great significance in wide variety of domains such as agriculture, energy, and economics. In recent decades, automatic weather stations have grown exponentially, becoming cornerstone of modern meteorology [34]. These stations are cost-effective for applications [3, 36] and are ideally positioned provide large volume of data advance deep learning (DL) approaches in ATSF [27].
However, the application of DL in ATSF faces two main challenges: (1) the observations of worldwide stations exhibit intricate spatial-temporaa correlations [40], necessitating models with advanced mining capabilities to ensure accurate forecasting [52]. (2) The requirement for fine-grained and large-scale forecasting [25, 39] calls for highly efficient and scalable models.
These two challenges are often trade-offs in prior studies, as shown in Figure 1. Transformer and its variants, which have gained significant popularity in ATSF, utilize sophisticated architectures to capture global spatial-temporal correlations. For example, AirFormer [16] introduces dartboard atten- tion for air quality prediction, and MRIformer [47] utilizes roltiirresoltion attention to predict wind
"Correpooniigg authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

speed. However, these complex designs come with significant costs, including hundreds of millions of parameters and extended training times, which limit their scalability for large-scale forecasting and hinder their applicability, especially with limited computational resources [6]. Moreover, despite the increased complexity, the performance gains are quite limited and do not justify the trade-off for practical utility. This motivates us to rethink the bottleneck of ATSF.
To this end, we delve deeper into the physical principles of atmospheric dynamics. In the at- mospheric system, the evolution of atmospheric variable y can be described by partial differen- tial equation (PDE):
Informer (2023)
FEDformer 120221
37
PetchTST 12023)
Osfomer 12023)
55
Alrfrmer (2023)
dv
2:10-
iTarsformer 12024)
Cor fommer (2023)
d1 - f 0,,
(1)
Mérformer (2024)
RMSE
2.45
STELLA 10k (Ours)
We demonstrate it theoretically in $3.5. The equation indicates that v/ is directly influenced by time t and geographical coordinates X, ó, However, most previous work has overlooked this spatial-temporal knowledge, simplistically treating f as a function of historical values while focusing fitting it in more complex forms. This misguided direction has led to the key bot- tleneck of ATSF.
S
11727
F4
2656
1.50
22"
101
Epoch Time (s)
comparison on the GlobalWind dataset. A performance-effccinge trade-off can be observed in Transformers, while STELLA leads in both aspects. The area of the plot represents the parameter count of the model.
Figure 1: Performance-
In this paper, we highlight the significance of spatial-tempore knowledge and introduce spatial-temppral position embedding (STPE) to integrate geographical coordinates and tempo- ral features from Eq.(1). Although position embeddings are widely considered as an adjunct to conpuutttionnnvvriinnt attention mechanisms, demonstrate that STPE can inherently model spatial-temporal correlations even in the absence of attention mechanisms, offering a 'free lunch' in balancing the performaacc-effccinney trade-off.
Consequently, we propose STELLA, a Spatial-Temporty knowledge Embedded Lightweight modeL for ASTF. STELLA utilizes STPE and replaces the Transformer layers with simple MLP. Figure 1 shows STELLA's lead in both performance and efficiency. With only 10k parameters and one hour of training, STELLA achieves competitive performance against 17 baselines. Furthermore, is noteworthy that the computational complexity of STELLA grows linearly with the increase of the number of stations N and the parameter count is independent of N. Therefore, STELLA can efficiently scale to the data with a larger N, facilitating large-scale forecasting. STELLA's leading-edge performance and efficiency challenge the prevailing assumptions that ATSF necessitates complex architecture (e.g., Transformers, STGNNs), offering balanced solution for ATSF. Our contributions be summarized as follows.
We innovatively highlight the significance of STPE in Transformer-based ATSF models. Even without attention mechanisms, it can explicitly capture spatial-temporal correlations by integrating spatial-temporal knowledge into the model. We theoretically prove its effectiveness from the perspective of atmospheric dynamics. Furthermore, STPE can also applied other models to improve performance ($4.5).
We propose STELLA, utilizing the STPE and replacing the Transformer layers with simple MLP. To the best of our knowledge, it is the first lightweight model designed for ATSE, challenging the prevailing assumption that ATSF necessitates complex architecture
STELLA offers a performance-efficien balanced solution for ATSF. Extensive experiments across five datasets demonstrate the superior performance of STELLA over 17 baselines.
2 Related Work
2.1 DL in Atmospheric Time Series Forecasting
Atmospheric time series forecasting (ATSF), including applications such as weather forecasting and air quality prediction, involves predicting key atmospheric variables (c.g., temperature) collected from

---

Y
Absolute Position Embedding
Output Layer
(b) Comparison with
Transformer Encoder
Embed
R
Time Features
Embed
X
FFN
2000-12-51
FFN
Zan
Emibed
10.1"
Sel-Artention
Coordiaates
Normolization
X
Figure 2: Architecture of STELLA.
weather stations over time. Spatial-temporal graph neural networks (STGNNs) prove effective in modeling spatial-temporal patterns across global stations [17, 22, 32, 1, 41, 28, 44], but most models are limited to short-term forecasting due to their high computational complexity, which restricts their scalability. Recently, Transformer-based models have gained popularity for their ability capture correlations. For example, AirFormer [16] introduces dartboard attention to model spatial correlations in air quality prediction, while Corrformer [40] uses multi-correlation mechanism as substitute for attention in weather forecasting. MGSFformer [46] and MRIformer [47] employ attention capture multi-resolution correlations through downsampling. aiming to forecast wind speed and air quality. However, the complexity of these models introduces significant computational costs, limiting their practical applicability and scalability for large-scale forecasting.
global
2.2 Studies of the effectiveness of Transformers
The effectiveness of Transformers has been thoroughly discussed in the fields of computer vision (CV) [48, 19] and natural language processing (NLP) [4]. In time series forecasting (TSF), LSTF-Lineer
[49] pioneered the exploration and outperformed a variety of Transformer-basse methods with a linear model. Shao al. [30] posited that Transformer-basee models face an over-fitting problem on specific datasets. Some recent works further questioned the necessity of attention in Transformers for TSF and replaced the attention with other modules. For example, MTS-Mixers [14] attempt to use random matrices and factorized MLPs instead of attention for information aggregation. MEAformer [10] replaces conventional attention with linear- complexxt mixing module. Additionally, SOFTS [7] employs STAR module as substitute for attention, which aggregates all series into global core representation. These studies consider PE as supplementary to attention mechanisms and consequently remove it along with attention, yet none have recognized the importance of PE.
3 Methodology
3.1 Problem Definition
Atmospheric Time Series Forecasting. We consider N stations and each station collects C time series of atmospheric variables (e.g., temperature). Then the observed data time can be denoted as X, E IR NxC. The 3D geographical coordinates of stations are organized as matrix Σ E R N which naturally accessible in mastonn-based forecasting. Given the historical time series of all stations from the past Ti, time steps and optional spatial and temporal information, we aim to learn function Fo(.) to forecast the values of future Ty time steps
YOSTT, Fo (X, E,(),
where X, Trir BRTXNNCC is the historical data, and YEATT, RTyxNXC is the future data.
3.2 Overview of STELLA
As shown in Figure 2, STELLA consists of an STPE module (Figure 2a) and an encoder that retains only the Feed-Forward Network (FFN) and discards all other components (Figure 2b). This
3

---

design significantly enhances computational and memory efficiency while maintaining competitive performance relative more complex architectures. Our choice pair STPE with simple MLP architecture was driven by a key motivation: to isolate and demonstrate the standalone power of STPE in spatial-temporal modeling. This minimalistic yet effective design not only reduces computational overhead but also offers clear interpretability regarding the contribution of spatial-temporal knowledge in ATSF.
3.3 Spatial-Tempose Position Embedding
Position embedding encodes the positional information of tokens in sequence [38] and is widely regarded as an auxiliary component for permutation-invariaan attention mechanisms. However, introduce spatial-temporal position embedding (STPE) integrate geographical coordinates and temporal features into the model, demonstrating that it can inherently capture spatial-temporal correlations by embedding additional spatial-tempore knowledge. Specifically, STPE consists of two components: spatial embedding and temporal embedding.
Spatial Embedding. The spatial embedding provides the geographical coordinates of stations to the model, which can explicitly model spatial correlations among worldwide stations. Specifically, we encode the geographical coordinates of the station into latent space. First, to account for the differing ranges of coordinate values, we perform normalization on each coordinate independently. Then, we utilize a feed-forward network (FFN). Therefore, the spatial embedding SE* E Rd can be denoted as:
SEÉ FFN(E') W2ReLU (W1E* + b1) b2.
(3)
where ' C R represents the normalized coordinates of station i.
Temporal Embedding. Temporal embedding provides real-workd temporal knowledge to the model. We utilize three learnable embedding matrices E 1R2 D E IR3ixd and M E R R212xd to save the temporal embeddings of all time steps [31]. They represent the patterns of weather in three scales T denotes hours in day, D denotes days in month and M denotes the months in year), contributing to modeling the multi-scale temporal correlations of atmospheric states. We add them together to obtain temporal embedding TE;:
TE, T, + D. + M,
3.4 MLP Backhone
Input Layer. Let X E Th be the historical time series of station i and variable j. X is mapped by the input embedding layer to HiJ e Rd in latent space, then added to the spatial and temporal embedding to obtain E,y
Hi J = Linear (X*). E H + SE TE,
(5)
Encoder. We utilize I-llaeer MIP encoder to learn the representation Z from the embedded data E Let (Z E and the I-th MLP layer with residual connection can be denoted as:
FFN' ((Z))) (Z
6
Output Layer
We employ linear layer map the representation Z e pdxNXC to the specified dimension, generating the prediction Y e xNxx
3.5 Theoretical Analysis
In this part, provide theoretical analysis of STELLA, focusing its effectiveness and efficiency.

---

(a) Context Aggregaitn
(b) Spatial-Temporal Knowledge Integration
Atcention
Overlook 2, 4, 2,T
Spacially or Semantically Proximate
MonthiYear Embed:
Q K V
Q
Spring
Summer
Station
Autumn
Winter
learn
Hour/Day
Embed
w/o Knowledge Integration
Day
Night
Spacial Embeddings
v = AV + F(VI)
Vf Avn + F(Vn, 2, z)
The key distinction that enhances the effectiveness of STELLA compared to prior ap- proaches. (a) Most prior methods primarily rely the context aggregation of historical observations (vanilla attention as an example). (b) contrast, STELLA makes predictions guided by spatial and temporal knowledge. For elarity, history data is denoted as vn, future data Vy, and the day-in-month embedding is omitted.
Figure 3:
The effectiveness of STELLA lies in the fact that STPE integrates geographical coordinates and real-world temporal knowledge into the model, which are intrinsically linked atmospheric dynamics. the following, theoretically demonstrate this relationship.
Effectiveness of STELLA.
Theorem 1. Let (X, d, 2} be the longitude, latitude, and altitude of weather station and v be meteorological variable collected by the station, then the time evolution of v is a function of v, time and coordinate X, 0
dv Ōt
= f(,,A,,,2,tt)
Proof. We provide proof with zonal wind speed as example. See Appendix A. for the full proof. According to the fundamental equations of atmospheric dynamics, the wind speed V satisfies:
dV 1 2 V
(8)
where p is the pressure, is the air density, and other terms constants. We transform the equation into spherical coordinates and apply the thin-layer approximation. The zonal wind speed can be expressed as:
du 1 Op
+ fo + ue tan Φ
Fx,
(9)
dt
p cos Φ0\
where d side of the equation spatially independent by rearranging terms:
+ a nos pox
w oe a is the Earth's 's radius. It is possible to render the left
aBs
du Øt
Ou cos ΦDλ
Fu ado
Ou 1 Op
P pacos dūA fo tan
+ Fx.
(10)
02
a
Therefore, we have
du
01 (,,
(11)
5

---

Considering using historical data spanning Th steps to predict future Ty steps, we can derive the following corollary:
Corollary 2.
-T,-- X, 2, where UT-TA+1: is the historical data, U++1++4Y is the fature data, and || =.
DT+IotT,- AV--TH
(12)
The detailed proof is provided in Appendix A.2. According to (12, we can employ neural network Fg to approximate F X, Φ; T). However, most previous models, including Transformers and STGNNs, have overlooked the critical spatial-temporal factors X, Ø. 2, т, instead treating F simplistically as function of historical values, as shown Figure 3 (a). These studies introduce increasingly complex context aggregation methods in an attempt to better fit historical data. However, blindly guessing spatial-temporal correlations by solely aggregating historical context may lead to overfitting, which becomes the key bottleneck for these models.
In comparison, STELLA can explicitly model F with X, 2, introduced by STPE, as shown in Figure 3 (b). STELLA aims to learn similar STPEs for spatially or semantically proximate stations and time steps, thereby capturing spatial-temporal correlations. Specifically, the spatial embedding learns the climatic features of different locations, while the temporal embedding captures the periodicity and seasonality of atmospheric states. We demonstrate this through visualization in $4.6. Therefore, STPE serves as an effective method for modeling heaporlltempora correlations.
Efficiency of STELLA. We theoretically analyze the efficiency of STELLA from the perspectives of parameter volume and computational complexity.
Theorem 3. The total number of parameters required STELLA is
2(d + 1)dL + (Th, + TT + 2)d+ (d 72)d.
(13)
MLP
STPE
The proof is provided in Appendix A.3. According to Theorem 3, the parameter count of STELLA is independent of the number of stations N. Therefore, key advantage of STELLA is that its deploy- ment overhead remains static regardless of how many stations it serves, making it particularly suitable for resource-conssrriied environments. In addition, the computational complexity of STELLA grows linearly with N. In contrast, context aggregation causes quadratic complexity and O(N) parameters, which are unaffordable with large-scale stations. Therefore, STELLA efficiently models spatial correlations with N Wiindependent parameters and linear complexity, ensuring optimal scalability for large-scale data.
4 Experiments
4.1 Experimental Setup
Datasets. We conduct experiments on five real-world datasets: (1) GlobalWind and GlobalTemp [40] comprise the hourly averaged wind speed and temperature of 3,850 stations global scale, spanning two years. Following the prior work [40], we use the past 48 hours to predict the next 24 hours for short-term weather forecasting. (2) ChinaWind and ChinaTemm comprise the daily averaged wind speed and temperature of 396 stations in China, spanning 10 years. We use the past 60 days predict the next 30 days, addressing long-term weather forecasting. (3) China-PM2.5 comprises the hourly averaged wind speed of 1,3 stations in China, spanning five years. We use the past 72 hours predict the next 72 hours for mid-term air quality prediction. See Appendix C. for more details of the datasets.
Baselines. We compare our STELLA with the following five categories of baselines. (1) Classic
methods: HI [5], ARIMA [33]. (2) STGNNs: AGCRN [1], MTGNN [41], GTS [28]. (3) Transformer-
bused TSF methods: Informer [53], FEDformer [54], DSformer [45], PStchTST [23], iTransformer
[20]. DUET [26]. (3) lightweight TSF methods: N-BEATS [24], DLinear [49], FITS [43]. (4) ATSF specialized Transformers: AirFormer [16], Corrformer [40], MRIformer [47]. See Appendix C.2 for
detailed introduction to the baselines.

---

Table I: Weather forecasting results on 5 datasets. The best results are in bold and the second best results are underlined. Dashes denote the out-of-memory (OOM) errors.
GlobalWind
GlobalTemp
China
ChinaTemm
ChinaPM2.5
Methods
RMSE
MAE
RMSE
MAE
RMSE
MAE
RMSE
MAE
RMSE
MAE
Claasic Methods
HI ARIMA
2,697 2.116
1.831
3.859
2.575
9.851
6.751
7,630
5.834
37.11 1
20.29
1.539
4.575
3.267
7.947
5.795
5.396
4.026
38.66
20.40
STGMNs
AGCRN MTGNN
7.458
5,061
4.336
2,999
7.294
5.055
4.221
3.168
28.21
16.12
GTS
7.312
4.997
4.298
3.082
Transformer- -aased TSF Methods
Informer
2.172
1.496
5.770
4.415
7.832
5.279
4.477
3.045
FEDformer
2.159
1.471
3.324
2.405
7.334
5.051
4.665
3.455
28.75
16.14
DSformer
2,007
1.347
3.089
2.057
7.311
5.000
4,919
3.605
27.68
14.94
PatchTST
1.973
1.332
3.130
2.062
7.295
5,033
4,909
3,600
26.87
14.37
iTranfformr
1.969
1.314
2,950
1.883
7.248
4,995
4,292
3.193
26.82
14.40
DUET
1.318
3.072
2.042
7.278
5.029
4,480
3.048
14.41
MLP-Based
Methods
N-BEATS
2.031
1.390
3,034
2.117
7.297
4.998
4.791
3,486
26.48
14.44
DLincar
2.005
1.350
3.149
2.072
7.309
5.031
4.990
3.659
27 70
14.76
FITS
2.021
1.354
3.150
2.072
7.284
5.039
5.283
3.823
27.73
14.96
ATSF Specializon Methods
AirFormer
1.952
1.314
5.594
4.127
7.909
5.377
4.772
3.561
29.63
15.55
Corformer
1.972
1.304
2.777
1.888
7.224
4,950
4.728
3.377
28.07
15.28
MRIformer
1.976
1.318
3.085
1.999
7.264
4.993
5.132
3.678
26.99
14.39
1.294
3,041
7,118
4.876
4.167
3.003
26.32
STELLA-1OK
0.001
0.002
10.002
10.000
10.003
+ 0.001
10.002
+0.003
+:0.003
100002
1.919
1,284
2.724
1.858
7.104
4,869
4,112
2.975
26.15
13.85
STELLAgpe
1 0.002
0.002
10.003
1.00022
10.010
+: 0.000
10.003
1000005
110.004
100001
Implementation Details. We develop STELLA-10K which has approximately 10k parameters. The number of MLP layers is and the hidden dimension 32. To explore the full performance of the model, we also conducted hyperparameter research on different datasets to find the optimal configuration, denoted as STELLAppt- Detailed configurations are provided in Appendix C.4. We adopt the Adam optimizer [11] to train our model and the learning rate is set to 5e-4. We trained all baselines with MAE (Mean Absolute Error) loss [30, 15], and the results all baselines are obtained using the best hyperparameters through hyperparameter search. We evaluate the performance of all baselines using two commonly used metrics: MAE and RMSE (Root Mean Square Error). All models are implemented with PyTorch 2 and trained NVIDIA GeForce RTX 4090 24GB GPU and an Intel Xeon Gold 6330 CPU.
4.2
Main Results
Table I presents the results of the performance comparison between STELLA and other baselines on all datasets. The results of STELLA are averaged over runs, with the standard deviation included. It is evident that the performance of lightweight TSF methods, such DLinear and N-BEATS, is not satisfactory. This indicates that lightweight TSF models, which fail to capture spatial-temports correlations, are inadequate for large-scale prediction tasks. STGNNs suffer from 0OM errors due to their high computational complexity, while the performance is also suboptimal. Similarly, despite the complex architectures of Transformer-baat models, most of them exhibit limited performance.
In contrast, STELLA-IOK achieves competitive performance with a simple MLP architecture and only 10k parameters, while STELLAapt consistently outperforms all other baselines on five datasets and three ATSF tasks. This suggests that integrating spatial-tempon hance model performance, proving to be more effective than the complex architectures of Transformer- based models. In addition, we provide a comparative analysis between STELLA and numerical weather prediction (NWP) methods in Appendix D.
information can significantly en-

---

4.3 Efficiency Analysis
Figure illustrates the performance-effiiienge comparison with Transformers. Here we further compare STELLA and other baselines, evalu- ating parameter counts, epoch time, and GPU memory usage [18]. Experiments are conducted on the most challenging GlodalWind dataset. As shown in Table 2, STELLA surpasses other DL methods in terms of all three efficiency met- rics. When compared to the ATSF specialized methods, STELLA demonstrates an order-of- magnitude improvement across three effi- ciency metrics, being about 10x to 10,000 smaller, 100x to 300x faster, and 50x memory- efficient, respectively. Additionally, due to its compact parameter size and simple computa- tions, STELLA can be efficiently trained in CPU environment. requires only 141 seconds to train STELLA-1OK for an epoch, making it well-suited for environments with limited computational resources. See Appendix D.2 for detailed efficiency experiments under limited resources.
Table Efficiency metrics of STELLA and other
Transformer-baser methods on GlobalWind
METHODS
PARAMS
EPOCH TIME
MAX MEM.
Informer
23.94M
37s
1.39GB
FEDformer
31.07M
163GB
DStormer
85.99M
250s
13.6GB
PatchTST
424.1K
559s
19,22GB
irrnnformmr
4.55M
785s
16.61GB
AirForerr
148.7K
2986s
14.01GB
Corformer
148.7M
11739k
18.41GB
MRIformer
11.66M
3431s
12.69GB
STELLA-1OR
9.98K
792MB
CPU)
4.4 Ablation Study
Effects of STPE. STPE is the key component of STELLA. To study its effects, we first conduct experiments models with the spatial embedding and temporal embedding removed separately. Table reveals that removing either embedding component leads to a decrease in MSE. This indicates that both spatial and temporal embeddings contribute positively model performance. Next, compare relative position embedding (RPE) with STPE. Specifically, RPE embeds the indices of stations instead of the absolute geographical coordinates. Since project the temporal dimension into the hidden space, temporal RPE is unnecessary. The results are presented in Table 3. Although RPE introduces Nd parameters, significantly increasing the model size, its performance still falls short of that of STPE, further validating its effectiveness of STPE.
Table 3: Ablation results of STELLA on five datasets.
GlobalWind
GlobalTemp
ChinaWind
ChinaTemp
ChinaPM2.5
Methods
RMSE
MAE
RMSE
MAE
RMSE
MAE
RMSE
MAE
RMSE
MAE
w/o Spatial Embedding
1.962
1.329
2.811
1.921
7.199
4.953
4.798
3,506
26.32
13.95
w/o Temporal Embedding
1.963
1.327
2,812
1.918
7.209
4.952
4.357
3.163
26.63
14.08
RPE
1.960
1.341
2.900
1.984
7.197
4.925
4.555
3.340
26.50
13.96
STELLA
1.919
1.284
2.724
1.858
7.104
4.869
4.112
2.975
26.15
13.85
1.305
1.935
1.300
1.941
RMSE MAE
RMSE
1.930
1.295
1.938
1.300
RMSE
MAE
RMSE
1.925
1.290
MAE
1.935
1.295
1.920
1.285
1.932
1.290
1.915
1.280
32
64
128
256 512 10242048
Number Layers
Number Layers
a
(b)
Figure 4: Results of hyperparameters analysis on GlobalWind dataset. (a) Effects of the number of
layers (d = 64). (b) Effects of the hidden dimension (L 2).
Hyperparameter Study. We investigate the effects of two important hyperparameters: the number of layers L in the MLP and the hidden dimension d. As illustrated in Figure 4 (a), STELLA achieves

---

Table 4: Improvements obtained by the adoption of STPE.
Datasets
GlobalWind
GlooaalTemp
ChinaWind
ChinaTemp
Metric
RMSE
MAE
RMSE
MAE
RMSE
MAE
RMSE
MAE
Original
1.973
1.332
3.130
2.062
7.295
5.033
4.909
3.600
PatchTST
+STPE
1.947
1.307
3.077
1.994
7.196
4.978
4.490
3.303
Original 2.007
1.347
3.089
2.062
7.311
5.000
4.919
3.605
DSformer
+ STPE
1.982
1.325
3.044
2.020
7.243
4,950
4.762
3,416
Original 1.969
1.314
2.950
1.883
7.248
4,995
4.292
3.193
iTransformer
+STPE
1.917
1.279
2.855
1.804
7.104
4.914
4.188
3.600
Latent Space Distribution
Geographical Distribution
Spacially Proximate Semrantically Proximate
40
20
25
0
20
-5
-40
50
-75
-150
-100
(a)
(b)
100
100
-50
200
-50
-25
0
25
50
-200
-100
100
200
-200
-100
100
(c)
(d)
Figure 5: Visualization of STPE, (a) Spatial embedding in the 2D latent space. (b) Geographical distribution of stations. The model learns spatial and semantic similarity through spatial embedding. (c) Hour-in-day embedding T. (d) Day-in-month embedding D. (e) Month-in-year embedding M.
the best performance when L = whereas increase in L beyond results in over-fitting and consequent decline in model performance. Figure 4 (b) shows that the metrics decrease as the hidden dimension increases and begin to converge when d exceeds 1024. However, STELLA-104 (d 32) already outperforms other Transformer-hased models. This further substantiates that STPE is more effective than the complex architectures of Transformer-basee models.
4.5 Generalization of
Position Embedding
In this section, we further evaluate the effeets of STPE by applying to Transformer-base models, with the results reported in Table 4. Only Transformers that independently embed each channel are compatible with STPE (Informer, FEDformer, etc., are incompatible). as our approach generates spatial embeddings for each station individually. The result shows that STPE significantly enhance the performance of Transformers, enabling them to achieve satisfactory results. In particular, iTransformer achieves state-of-the-art performance Global after the application of STPE.
4.6 Visualization of PE
In this section, we visualize the STPE further study effectiveness. Due to the high dimensionality of the embeddings, employ t-SNE [37] to visualize them on 2D planes.

---

Visualization of Spatial Embedding. Figure 5 (a) indicates that spatial embeddings tend to cluster. The model attempts to learn similar embeddings for spatially or semantically proximate stations. resulting in clustered structure. To demonstrate this, we have marked two clusters with orange and purple and examined the geographical distribution of the stations within each cluster. As shown in Figure 5 (b), the orange cluster is densely distributed in Europe in the geographical space, indicating that the model has learned spatial similarity. Meanwhile, stations in the purple cluster are distributed across South America and Africa, with all distribution areas characterized by tropical or subtropical climates, suggesting that the model has captured semantic similarity.
Visualization of Temporal Embedding, Figure 5 (c-e) shows the temporal embeddings with colors representing the temporal order. The hour-in-day and month-in-year embeddings form ring-like structures in temporal order, revealing the distinct daily and annual periodicities of weather, which is consistent with humans' common understanding.
4.7 Case Study
Ground Truth
Forecasting Results
To comprehensively illustrate STELLA's capability to capture the complex spatial- temporal correlations among large-scale stations, present visualization of the forecasting results of the Global- Wind dataset from multi-station per- spective. Additional illustrative show- cases are available Appendix Krig- ing [13] is employed to interpolate dis- crete points into a continuous surface, en- hancing the visual clarity of spatial vari- ations. As shown Figure 6, the pre- dicted results (right column) are closely aligned with the ground-truth values (left column) across all displayed time steps. This high level of consistency confirms that STELLA effectively captures spatial- temporal patterns in global weather data and delivers accurate predictions.
50
-100
100
-50
-50
100
-50
-50
100
100
-50
-100
100
100
100
Figure 6: Forecasting results of averaged wind speed with a 6-hour interval and (i.e., 64 x 32) resolution.
5 Conclusion
This work innovatively highlights the significance STPE in Transformers for ATSE Even without attention mechanisms, STPE explicitly captures spatial-temporal correlations by integrating geograph- ical coordinates and temporal features, which are inherently linked to atmospheric dynamics. We then present STELLA, lightweight and effective model for ATSF. We leverage STPE and replace Transformer layers with a simple MLP. STELLA can achieve satisfactory performance across five weather datasets. The paper posits that the incorporation of ppatial-empoon knowledge is more effective than intricate model architectures, illuminating novel insights for ATSE.
Limitations and Future Work. STELLA is limited by its channel-indepeeddnt modeling and the restricted expressive power of MLPs, and may produce overly smoothed forecasts for extreme events, which is a common issue with MSE/MAE-trained models. Future work will address these challenges by incorporating covariates and physical constraints. See Appendix for more details.
Acknowledgement
This work is supported by the NSFC underGrant Nos. Nos.62372430 and 62502505, the Youth Innovation Promotion Association CAS No. 12, the Postdoctoral Fellowship Program of CPSF under Grant Number #7C20251078, the China Postdoctoral Science Foundation No. No.2025M77154 and HUA Innovation fundings. We sincerely thank all the anonymous reviewers who gerenously contributed their time and efforts.
10

---

References
[1] Lei Bai, Lina Yao, Can Li, Xianzhi Wang. and Can Wang. Adaptive graph convolutional recurrent network for traflic forecasting. Advances in neural information processing systems, 11917884--7815,, 2020.
[2] Peter Bauer, Alan Thorpe, and Gilbert Brunet. The quiet revolution of numerical weather prediction. Nature, 525(7567):47-555 2015.
[3] Gabriel F.L.R. Bernardes, Rogério Ishibashi, André A.S. Ivo, Valério Rosset, and Bruno Y.L. Kimura. Prototyping low-cost automatic weather stations for natural disaster monitoring. Digital Communications and Networks, 9(4:941-955, 2023.
[4] Yuchen Bian, Jiaji Huang, Xingyu Cai, Jiahong Yuan, and Kenneth Church. On attention redundancy: A comprehensive study. In Proceedings the 2021 conference of the north american chapter of the association for computational linguistics: human language technologies, pages 930-945. 2021.
[5] Yue Cui, Jiandong Xic, and Kai Zheng. Historical incrtia: neglected but powerful baseline for long sequence time-series forecasting. Proceedings of the 30th ACM international conference on information & knowledge management, pages 2965-2969, 2021.
[6] Jinliang Deng. Xuan Song, Ivor W Tsang. and Hui Xiong. Parsimony or capability? decomposition delivers both in long-term time series forecasting, arXiv preprint arXiv:2401.1192) 2024.
[7] Lu Han, Xu- Yang Chen, Han-Jia Yc, and De-Chuan Zhan, Softs: Efficient multivariate time series forecasting with series-core fusion. Advances in Neural Information Processing Systems, 37:64145-66 451155,,, 2024.
[8] Kethmi Hirushini Hettige, Jiahao Ji, Shili Xiang, Cheng Long, Gao Cong, and Jingyuan Wang. Airphynet: Harnessing physics-guided neural networks for air quality prediction. In The Twelfth International Conference on Learning Representations, 2024.
[9] Jincai Huang, Yongjun Xu, Qi Wang, Qi Cheems Wang, Xingxing Liang. Fei Wang. Zhao Zhang, Wei Wei, Boxuan Zhang. Libo Huang, et al. Foundation models and intelligent decision making: Progress, challenges, and perspectives. The Innovation, 2025.
decision-making:
[10] Siyuan Huang, Yepeng Liu, Haoyi Cui, Fan Zhang, Jinjiang Li, Xiaofeng Zhang. Mingli Zhang, and Caiming Zhang. Meaformer: An all-mip transformer with temporal external attention for long-term time series forecasting. Information Sciences, 669:12060S, 2024.
[11] Diederik P Kingma and Jimmy Ba. Adam: method stochastic optimization. arXiv preprint WMXiv:142..6988 2014.
[12] Wenyuan Li, Zili Liu, Keyan Chen, Hao Chen, Shunlin Liang, Zhengxia Zou, and Zhenwei Shi. Deep- physinet: Bridging deep learning and atmospheric physics for accurate and continuous weather modeling, 2024.
[13] Yujie Li, Zezhi Shao, Chengqing Yu. Tangwen Qian, Zhao Zhang, Yifan Du Shaoming He. Fei Wang. and Yongjun Xu. Sta-gann: A valid and generalizable spatio-temporal kriging approach. arXiv preprint arXiv:2508. 16161, 2025.
[14] Zhe Li, Zhongwen Rao, Lujia Pan, and Zenglin Xu. Mts-mixers: Multivariate time series forecasting via factorized temporal and channel mixing. arXiv preprint arXiv:2302.0495 045000 2023.
[15] Yubo Liagg, Zezhi Shao, Fei Wang, Zhao Zhang, Tao Sun, and Yongjun Xu. Basicts: An open source fair multivariate time series prediction benchmark. In International symposium on benchmarking. measuring and optimization, pages 801 Springer, 2022.
[16] Yuxuan Liang, Yutong Xia, Songyu Ke, Yiwei Wang, Qingsong Wen, Junbo Zhang, Yu Zheng, and Roger Zimmermann. Airformer: Predicting nationwide air quality in china with transformers. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 14329 14337, 2023.
[17] Haitao Lin, Zhangyang Gao, Yongjic Xu, Lirong Wu, Ling Li, and Stan L.i. Conditional local convolution for spatio-temporal meteorological forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pages 7470-7478, 2022.
[18] Shengsheng Lin, Weiwei Lin, Wentai Wu, Haojun Chen, and Junjie Yang. Sparsetsf: Modeling long-term time series forecasting with" Ik* parameters. In first International Conference on Machine Learning. 2024.
11

---

[19] Sihao Lin, Pumeng Lyu, Dongrui Liu. Tao Tang, Xiaodan Liang, Andy Song. and Xiaojun Chang. Mip can be good transformer learner. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 19489-19498, 2024.
[20] Yong Liu, Tengge Hu, Haoran Zhang. Haixu Wu, Shiyu Wang. Lintao Ma, and Mingsheng Long. itrans- former: Inverted transformers are effective for time series forecasting. In The Twelfth International Conference on Learning Representations. 2024.
[21] Gurii Marchuk. Numerical methods in weather prediction. Elsevier, 2012.
[22] Qingjian Ni, Yuhui Wang, and Yifei Fang. Ge-stdgn: novel spatio-tent temporal weather prediction model based on graph evolution. Applied Intelligence, 52(7)77638 7652, 2022.
[23] Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In The Eleventh International Conference on Learning Representations, 2023.
[24] Boris N Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. N-beats: Neural basis expansion analysis for interpretable time series forecasting. In International Conference Learning Representations, 2020.
[25] Tim Palmer. Climate forecasting: Build high-resolution global climate models. Nature, 513(75277 2014.
[26] Xiangfei Qiu, Xingjian Wu, Yan Lin, Chenjuan Guo, Jilin Hu, and Bin Yang. Duet: Dual clustering enhanced multivariate time series forecasting. In Proceedings of the 31 ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. pages 1185-1196, 2025.
[27] Martin G Schultz, Clara Betancourt, Bing Gong, Felix Kleinert, Michael Langguth, Lukas Hubert Leufen, Amirpasha Mozaffari, and Scarlet Stadtler. Can deep learning heat numerical weather prediction? Philo- sophical Transactions of the Royal Society A, 379 (1922199):2000099, 2021.
[28] Chao Shang, Jie Chen, and Jinbo Bi. Discrete graph structure learning for forecasting multiple time series. In International Conference on Learning Representations, 2021.
[29] Zezhi Shao, Yujie Li, Fei Wang. Chengqing Yu, Yisong Fu, Tangwen Qian, Bin Xu, Boyu Diao, Yongjun Xu, and Xueqi Cheng. Blast: Balanced sampling time series corpus for universal forecasting models. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2, KDD 25,, page 2502-2513, New York, NY, USA, 2025. Association for Computing Machinery.
[30] Zezhi Shao, Fei Wang. Yongjun Xu, Wei Wei, Chengqing Yu, Zhao Zhang, Di Yao, Tao Sun, Guangyin Jin, Xin Cao, et al. Exploring progress in multivariate time series forecasting: Comprehensive benchmarking and heterogeneity analysis. IEEE Transactions Knowledge and Data Engineering. 2024.
[31] Zezhi Shao, Zhao Zhang, Fei Wang. Wei Wei. Yongjun Xu. Spatial-temporal identity: simple yet effective baseline for multivariate time series forecasting. In Proceedings of the 3 ACM International Conference on Information & Knowledge Management, pages 4454-4458, 2022.
[32] Zezhi Shao, Zhao Zhang, Fei Wang, and Yongjun Xu. Pre-training enhanced spatial-temm temporal graph neural network for multivariate time series forecasting. In Proceedings the 28th ACM SIGKDD conference on knowledge discovery and data mining, pages 1567-1577 1577, 2022.
[33] Robert H Shumway, David Stoffer, and David S Stoffer. Time series analysis and its applications. volume 3. Springer. 2000.
[34] Dipak V Sose and Ajij D Sayyad. Weather monitoring station: a review. Int. Journal of Engineering Research and Application, 6(6):55-60, 2016.
[35] Yanru Sun, Zongxia Xie, Dongyue Chen, Emadeldeen Eldele, and Qinghua Hu. Hierarchical classification auxiliary network for time series forecasting. arXiv preprint arXiv:2405. 18975, 2024.
[36] Sonam Tenzin, Satetha Siyang, Theerapat Pobkrut, and Tecrakiat Kerdcharoen. Low cost weather station for climate-smart agriculture. In 2017 9th international conference on knowledge and smart technology (KST), pages 172-177. IEEE, 2017.
[37] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne, Journal of machine learning research, 9(1)), 2008.
12

---

[38] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances neural information processing systems, 30, 2017.
[39] Fei Wang. Di Yao. Yong Li. Tao Sun, and Zhao Zhang. Ai-enhanced spatial-temports data-mining technology: New chance for next-generation urban computing. The Innovation, 4(2). 2023.
[40] Haixu Wu, Hang Zhou, Mingsheng Long, and Jianmin Wang. Interpretable weather forecasting for worldwide stations with a unified deep model. Nature Machine Intelligence, 5(6):602 611, 2023.
[41] Zonghan Wu, Shirui Pan. Guodong Long, Jing Jiang. Xiaojun Chang. and Chenggi Zhang. Connecting the dots: Multivariate time series forecasting with graph neural networks. Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pages 753-763, 2020.
[42] Yongjun Xu, Xin Liu, Xin Cao, Changping Huang, Enke Liu, Sen Qian, Xingchen Liu, Yanjun Wu, Fengliang Dong. Cheng- Wei Qiu, et al. Artificial intelligence: A powerful paradigm for scientific research. The Innovation, 2(4). 2021.
[43] Zhijian Xu, Ailing Zeng, and Qiang Xu. Fits: Modeling time series with 10k parameters. In The Twelfth International Conference on Learning Representations, 2024.
[44] Chengqing Yu, Fei Wang, Zezhi Shao, Tangwen Qian, Zhao Zhang, Wei Wei, Zhulin An, Qi Wang, and Yongjun Xu. Ginart: A robust end-to-end framework for multivariate time series forecasting with missing values. IEEE Transactions on Knowledge and Data Engineering, 2025.
[45] Chengging Yu, Fei Wang, Zezhi Shao, Tao Sun. Lin Wu, and Yongjun Xu. Dsformer: A double sampling transformer for multivariate time series long- term prediction. In Proceedings of the 32nd ACM international conference on information and knowledge management, pages 3062-3072. 2023.
[46] Chengqing Yu, Fei Wang, Yilun Wang, Zezhi Shao, Tao Sun, Di Yao, and Yongjun Xu. Mgsfformer: A culti-graaularityy spatiotemporal fusion transformer for air quality prediction. Information Fusion, page 102607, 2024.
[47] Chengging Yu. Guangxi Yan, Chengming Yu. Xinwei Liu, and Xiwei Mi. Mriformer: A multi-resolution interactive transformer for wind speed multi-step prediction. Information Sciences, 661:120150, 2024.
[48] Weihao Yu, Mi Luo, Pan Zhou, Chenyang Si, Yichen Zhou, Xinchao Wang, Jiashi Feng, and Shuicheng Yan. Metaformer is actually what you need for vision. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10819-10829, 2022.
[49] Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 11121-11122 2023.
[50] Chi Zhang, Huaping Zhong, Kuan Zhang, Chengliang Chai, Rui Wang, Xinlin Zhuang, Tianyi Bai, Qiu Jiantao, Lei Cao, Ju Fan, et al. Harnessing diversity for important data selection in pretraining large language models. In The Thirteenth International Conference on Learning Representations, 2025.
[51]
Shihao Zhang, Linlin Yang. Michael Bi Mi, Xiaoxu Zheng, and Angela Yao. Improving deep regression with ordinal entropy. In The Eleventh International Conference Learning Representations, 2023.
[52] Tianjic Zhao, Sheng Wang, Chaojun Ouyang, Min Chen, Chenying Liu, Jin Zhang, Long Yu, Fei Wang. Yong Xie, Jun Li, Fang Wang, Sabine Grunwald, Bryan M. Wong, Fan Zhang, Zhen Qian, Yongjun Xu, Chengqing Yu, Wei Han, Tao Sun, Zezhi Shao, Tangwen Qian, et al. Artificial intelligence for geoscience: Progress, challenges, and perspectives. The Innovation, 5:100691, 2024.
[53] Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-serics forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 1106-111155 2021.
[54] Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang. Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International conference on machine learning, pages 27268-27286 PMIRR 2022.
13

---

A Theoretical Proofs
A.1
Full Proof of Theorem
Proof. The fundamental equations of atmospheric motion [21] are
dV 1 Vv
2Ω x V V
dt
dp
pV V =,
dt
C df
1 dp
(14)
p dt Q
dt
0q +
Vq = Sq:
01
p == RRT,
where V is the velocity, is the air density, is the pressure, T is the temperature, g is the specific humidity, and others are physical constants.
Expand the equations into scalar form and transform them into spherical coordinates, yielding the following:
du
Op
uv tan Φ + Fx,
+ fu+
dt
pr cos Φbd
r
du
1 Öp
u² tan ④
fu
F
df
prôp
dw
10p
dt p dr 9 Fr, dp pl
Ou
Ov +
Ow
2w
(15)
tan Φ),
dt cos Φ 0x
rā
dr
r
d7 Q dp
dt C pCp dt'
dg Sp
di pRT,
where u zonal velocity, v meridional velocity, and w vertical velocity. The expansion of d
de
d 0 + u 0i r cos
v.
+ w
(16)
di
ΦOA
r
Dr
Radial distance can be further denoted as r Since a constant and a >> 2, we have 3r 2z and we can approximate with a. Then we render the left side of the equation spatial independent by rearranging terms:
where a is Earth's radius and is altitude.
14

---

Ou
iu
du
du
1
Op
uu tan Φ
+ +.u+
+ Fx:
0t
0x
06
02
p
ΦOA
00
Ou
v 0u
du
Öp
42 lan Φ
fu
+ Fo.
0t
Φ 0x
06
0z
p
ado
Ow
Ow
0w
dw
0p
Fr.
0t
a cos
Φ 0x
0z
p
0p
Op
v 0p
0p
du
du
dw
2w
p
tan の),
10
Φ 0X
04
0z
Φ OX
rō
dr
0T
u øT
v OT
OT
0p
u
Öp
0p
Op
Q
+
+
+
01
cos
Ø YX
06
dz
PCp
01
COS
0X
06
0z
Cp
0q
0q
v Ōq
0q
w
Sq
0t
cos 6 0A
a dΦ
Oz
p pRT.
(17)
Therefore, for each atmospheric variable v, we have
dv = flv,t,,,0,8)
(18)
0t
A.2 Proof of Corollary 2
Proof. According to Theorem 1 we can integrate both sides of the equation with respect to I, from time step 7; step Ty:
Vr, = Ur, S I
(19)
where At is the interval between time steps.
Before proceeding with the mathematical proof, we first illustrate with Figure A directed edge from to T in the figure denotes the evolution constrained by Eq.(1 Figure 7 (a) shows the mechanism that the state of the atmosphere evolves step by step, which we need to adopt an autoregressive neural network (e.g. RNN) to approximate it. Through simple topological transformation, we can obtain the mechanism shown (b), where the unobserved states are calculated by every historical observation. Therefore, we can adopt neural network to predict all unobserved states in parallel and constitutes more robust approach as it fully leverages historical observations.
We provide detailed mathematical proof in the following. For brevity, JT F(A, o, 2, t) is denoted as I/_ For the unobserved data step + (k- 2, ;T),, it can be represented by:
Vi++ V, I1
MT+k = - + 17+k
(20)
Vr+b = Vy-I++1 + +
We can take its linear combination as follows:
Mr-k as.1 (Vy + x² + an.2 (Mr-1
1 + aR,T (Mr-T,+1 1Ith +1)
(21)
= 0 (v (v--t++1: + 7 1111:
+ T
where +1:7 is ( + By repeating this procedure from #/41 Mr+Ty, we have
1T+ and Or E Ts satisfies ||or =
15

---

tA
Observed
Unobserved
Figure 7: Mechanism of the evolution of the atmosphere state. (a) The atmosphere state evolves step by step through the black edges. (b) Predict the unobserved states (a) and (b) topologically equivalent.
parallel through the red edges.
U++1 = 01 (U--T,,1::- + 17 +1: +1:
V++2 02 (V--T)+ I=+2 +1:7
(22)
TTT UT+T, aT, (Ur-Tu+lor +1:7
Therefore, we have
UT+1:7tTy = -U,- F(V, -err-TT++iae -T, A, ti 0, 2,7),
(23)
where A (an1 ", , (,,,) satsifies || and F is the combination of I.
A.3
Proof of Theorem
Proof. The data embedding layer maps the input data into the latent space with dimension d, thereby introducing (T,+1)d parameters. Analogously, the regression layer introduces (Ty+1)d parameters. The parameter count of a L-layer MLP with residual connection is 2Ld(d + 1).
For the STPE module, the spatial embedding costs + 1)d d(d 1) parameters, and the temporal embedding costs (24 + 31 + 12)d parameters.
Therefore, the total number of parameters required for STELLA is 2(d + 1)dL + (Th + + + 2)d (d + 72)d.
B Overall Workflow of STELLA
The overall workflow of STELLA is provided in Algorithm 1.
C Experimental Details
C.1 Dataset Description
To evaluate the comprehensive performance of the proposed model, we conduct experiments on five ATSF datasets with different temporal resolutions and spatial coverages including:
GlobalWind and GlobalTemp [40] are collected from the National Centers for Environ- mental Information (NCEI)². These datasets contain the hourly averaged wind speed and temperature of 3,850 stations around the world, spanning two years from January 2019 to
houryycccss
16

---

Algorithm Overall workflow of STELLA.
1: INPUT: historical data X R2AXNXC geographical coordinates E R2x3, first time step
2: OUTPUT: forecasting result Y X
3: X X. trransoos (1, -1) / X P RONNT, //
4: H = Linear (X) / Input layer, H E BC
5: SE FFN (Σ) / "E E
EEBERNAA RNxd %/
6: SE = SE.repeat(C,1,,
/a & RCxNNx #/
7: hour, day, mon (arneeffature()
1* Obtain hour, month and day from /
8: T T /hourrrreppate N,
9: D D (ddayt.repea((CC C, N, 1)
10: M = M[mon] .rwpeat M[mon]]rrpeat( 1,
11: Zu = H SE + + +D++++++ D M
1* Zo E #C
10 MLP encoder */
12: for in {0, 1,... , 1) do
Z++1 FFN, (Z:) + Zt
14: end for
Y Linear (Zt) /* Regression layer, Y RCNNxT */
16: Y = Y .transpose (1,-1) 1)
return Y
December 2020. Please note that these datasets are rescaled (multiplied ten times) from the raw datasets.
China and ChinaTemm Temp are also collected from NCEI These datasets contain the daily averaged wind speed and temperature 396 stations in China (382 stations for Temp_CN CN due to missing values), spanning 10 years from January 2013 to 31 December 2022.
ChinaPM2.5 is collected from CNEMC It contains the hourly averaged wind speed of 1,316 stations in China, spanning years from January 2020 to 31 December 2024. The original dataset only provides the latitudes and longitudes of stations and we obtain the elevations of stations through Open-Elevation'.
The statistics of the datasets are shown in Table and the station distributions are shown in Figure 8.
Table 5: Statistics of datasets.
DATASET
COVERAGE
STATION NUM SAMPLE RATE
TIME SPAN
LENGTH
GlobalWind
Global
3,850
hour
2 years
17,544
GlobalTemp
Global
3,850
I hour
2 years
17,544
ChinaWind
National
396
day
10 years
3,652
China'Temp
National
382
day hour
10 years
3,652
ChinaPM2.5
National
1,316
5 years
43,539
C.2
Introduction to Baselines
HI [5], short for historical inertia, is simple baseline that adopts the most recent historical data as the prediction results.
ARIMA [33], short for autoregressive integrated moving average, a statistical forecasting method that uses the combination of historical values predict future values.
AGCRN [1] is STGNN that integrates adaptive graph convolution and recurrent networks to dynamically capture spatiotemporal dependencies in multivariate time series.
'hups://air.cnemm..
17

---

Sett tane
Ihaland
(b)
(c)
Figure 8: Distributions of the stations. (a) GlobalWind and GlobalTemp. (b) China Wind and
ChinaTemp. (c) ChinaPM2.5.
MTGNN [41] is GNN-based model for multivariate time series forecasting. It can automatically learn the hidden spatial dependencies among variables.
GTS [28] is a STGNN that learns the structure simultaneously with the GNN when the graph is unknown.
Informer [53] is Transformer for time series forecasting (TSF) with sparse self-attention mechanism.
FEDformer [54] is a eeqquency-enhancee Transformer combined with seasonal-tres -trendd de- composition to capture the overall trend of time series.
DSformer [45] utilizes double sampling blocks model both local and global information.
PatchTST [23] divides the input time series into patches, which serve as input tokens of Transformer.
iTransformer [20] is a Transformer for TSF that simply applies the attention and Feed- Forward Network (FFNN on the inverted dimensions, thereby enhancing the Transformer's capability to capture multivariate correlations.
DUET [26] is dual clustering enhanced model for TSF that clusters temporal and channel dimensions to learn the complex correlations of time series.
DLinear [49] a lightweight baseline for TSF, which consists of a linear model and a time series decomposition module.
N-BEATS [24] utilizes backward and forward residual links and very deep stack of fully-connected layers.
FITS [43] is lightweight baseline for TSF that employs complex-valus linear layer to learn amplitude scaling and phase shifts, enabling interpolation the complex frequency domain.
AirFormer [16] employs Bartboard-like mapping and local windows to restrict attention to focusing solely on local information.
Corrformer [40] utilizes a decomposition framework and replaces attention mechanisms with more efficient multi-correlation mechanism.
MRIformer [47] employs hierarchical structure, stacking attention layers capture correlations from multi-resolution data obtained by downsampling.
C.3 Evaluation Metries
The evaluation metrics we used in the paper are defined as follows.
Mean Absolute Error (MAE)
C
T
MAE(Ý, Y) N.C.
Y
YY
(24)
18

---

Root Mean Square Error (RMSE)
C
RMSE Y, Y)
Yiu
Yi
2
(25)
N.C.T. T
C.4 Optimal Settings
For reproducibility purposes, we provide the optimal hyperparameters of STELLA on the five datasets, illustrated in Figure 6. All experiments were performed NVIDIA GeForce RTX 4090 24GB GPU.
Table 6: The optimal settings of STELLA.
Scttings
GlobalWind
GlobalTemp
ChinaWind
ChinaTemp
ChinaPM2.5
Hidden Dimension
1024
2048
64
32
256
Layers of MLP
2
3
2
Network Architecture
Activation Function
ReLU
ReLU
ReLU
ReLU
ReLU
Dropout
0.2
0.2
0.2
Batch
32
Epoch Optimizer
100
100
100
100
Adam
Adam
Adam
Addm
Adam
Optimization
Learning Rate
5e-4
Se-4
Se-4
Se-4
5e-4
Weight Decay
5e-4
5e-4
LR Scheduler
MultistepLRR
MultistepLR [1,50]
MultistepLR [1,50]
MultistepLR [50]
MultistepLR
Milestone
[1,50]
[1,25,50]
-?
0.5
0.5
0.5
D
Additional Experimental Results
D.1
Comparison with Numerical Methods
In this section, we compare our model with the numerical weather prediction (NWP) methods in short-term global weather forecasting tasks. Conventional NWP methods use PDEs to describe the atmospheric state transitions across grid points and solve them through numerical simulations. Currently, the ERA5 from the European Centre for Medium Mediuum-aange Weather Forecasts (ECMWF) and the Global Forecast System (GFS) from NOAA are the most advanced global forecasting models. ERA5 provides gridded global forecasts at 0.5% resolution while GFS at 0.25 resolution.
To make the comparison practical, we utilize bilinear interpolation with height correction to obtain the results for scattered stations, which is aligned with the convention in weather forecasting [2, 40]. The results are shown Table
Both ERA5 and GFS fail to provide accurate predictions, which indicates that grid-based NWP methods inadequate for fine-grained station-based predictions. In contrast, STELLA rately forecast the global weather for worldwide stations, significantly outperforming the numerical methods.
Table 7: Forecasting results from NWP methods and our model on global weather datasets.
GlobalWind
GlobalTemp
Methods
MSE
MAE
MSE
MAE
ERA5 (0.5%)
2.606
1.847
5.298
3.270
GFS (0.25%)
3.161
2.340
3.864
2.287
STELLA
1.919
1.284
2.724
1.858
19

---

D.2 Efficiency Analysis under Limited Computational Resources
In many application scenarios, training DL model from scratch is necessary, even when compu- tational resources are limited. However, complex model architectures come with significant costs, including hundreds of millions of parameters and extended training times, which hinder their applica- bility. In the era of large models [9, 29, 50, 42], this phenomenon is particularly evident. STELLA comes with an efficient solution for this. In this section, we conduct a further efficiency analysis under limited computational resources. To simulate scenarios with limited computational resources, we utilize an Intel Xeon Gold 6330 CPU to train and test models. The top five performing models on the GlobalWind dataset selected for to experiment. Table presents the training time and inference time of different models on CPU.
The following observations can be made: (1) Except for STELLA, the other top five models are all based on Transformer architecture. (2) The training times for these models on CPU are impractical. For instance, training Corrformer for 50 epochs would take approximately 10 months. In contrast, STELLA can be efficiently trained in a CPU environment, requiring only about hours to train STELLA-10k for 50 epochs. Furthermore, due to its compact parameter size and straightforward computations, STELLA is highly suitable for deployment on edge devices for inference tasks.
Table 8: The training time and inference time the top five models on CPU.
METHODS
PERFORMANCE RANKING
TRAINING TIME / EPOCH
INFERENCE TIME / SAMPLE
PatchTST
5
2.25h
0.10s
Corfformer
4
14th
3.90s
iTrrasformer
3
17.5h
0.70s
AirFormer
2
56. Ih
1.77s
STELLA-IOk
1
141s
2ms
E
Discussion
E.1
Comparison between STELLA and Other Lightweight Methods
In the main text, we provide a comprehensive comparison between STELLA and -ransformer-based models. Here, we present a further discussion about the distinctions between STELLA and other lightweight models in terms of both performance and efficiency. We conduct additional experiments on the GlobalWind dataset and compare STELLA three lightweight TSF models, N-BEATS [24], DLinear [49]. FITS [43]. Table 9 presents the performance-efficiene comparison results, from which
we derive the following conclusions:
Table 9: The training time and inference time of the top five models CPU.
METHOMS
PERFORMANCE RANKING
PARAMS
EPOCH TIME
MAX MEM.
N-BEATS
10
121,78k
37s
2.00GB
DLinear
7
2.35k
27s
1.10GB
FITS
1.80k
29s
1.27GB
STELLA-10k
9.98k
30s
792MB
Performance. In terms of performance, STELLA achieves the top ranking among the 17 baselines (15 of which are DL-based), demonstrating substantial lead over other lightweight models. This superiority stems from the fact that other lightweight models fail in modeling spatial correlations. Taking DLinear as an example, its use of single linear layer to predict data across all sites inherently unsuitable for multi-station forecasting scenarios.
Efficiency, analyze efficiency using three metrics: parameter counts, training time per epoch, and maximum GPU memory usage. FITS and DLinear outperform STELLA in terms parameter counts. However, STELLA achieves comparable training speed to other lightweight models, while its GPU memory usage is even lower. This is because other
20

---

lightweight models incorporate additional operations to balance performance. For instance, DLinear introduces convolutional modules, while FITS employs complex frequency-doman interpolation. Despite having linear layers their backbone, these models are less ef- ficient than anticipated. Overall, STELLA maintains high level of efficiency without compromising performance.
E.2 Comparison between STELLA and PINNs
While PINNs
informed neural networks) often incorporate physical constraints regular- ization, our approach is different in mechanism and purpose. Instead of using physical equations loss terms auxiliary supervision in ATSF tasks [12, 8] draw theoretical motivation from the governing PDFs of atmospheric dynamics to construct input representations (STPE) that inject geographical and temporal priors directly into the model. To the best of our knowledge, this is the first work to prove, both theoretically and empirically, that such a representation alone, even coupled with simple MLP, can surpass many state-of-the-art ATSF models in both accuracy and efficiency.
E.3
Can STPE be Applied to Linear Model?
As shown in $4.5, STPE can applied to Transformer-basee models to enhance their performance, which naturally raises a question: Can STPE also be upplied to more lightweight methods (e.g., linear model)?
To address this, we conducted extensive experiments and found that the improvement was relatively marginal, especially global weather forecasting tasks (GlobalWind and GlobalTemp datasets), whereas more substantial gains were observed in national-scale tasks (China Wind and ChinaTemp datasets), as shown in Table 10. According to Theorem 1, the evolution of atmospheric states is nonlinear, thus, the fitting capability is the core limitation for linear models.
Table 10: Improvements obtained by the adoption of STPE.
Datasets
GiobalWind
GlobalTemp
ChinaWind
ChinaTemm
Metric
RMSE
MAE
RMSE
MAE
RMSE
MAE
RMSE
MAE
Original +STPE
2.005
1.350
3.149
2.072
7.309
5.031
4.990
3.659
DLinear
2.002
1.348
3.138
2.078
7.250
5.002
4.881
3.596
F
Case Study
In the main text, we present multi-station collaborative prediction results to provide an intuitive un- derstanding STELLA's ability to capture spatial correlations and perform collaborative predictions. Here, to enable a clear comparison among different models, we provide supplementary prediction cases from individual stations. We select three representative datasets for each ATSF task, and the results are given by the following advanced models: STELLA, i'Transformer [20]. Corrformer [40], AirFormer [16], PStchTST [23], DLinear [49]. Figure 9-11 indicates that STELLA provides the most accurate prediction and demonstrates superior performance among the models.
G Limitations and Future Work
Multivariate Correlations. Atmospheric variables are tightly coupled (Eq.14); however, in our modeling, we decoupled these variables, overlooking the multivariate correlations and training sepa- rate models for each atmospheric variable. This was primarily driven by performance considerations and was consistent with previous work [40]. Accounting for multivariate correlations and jointly forecasting multiple atmospheric variables might introduce learning challenges, potentially leading to degraded prediction performance. Since the number of atmospheric variables to be predicted usually limited, independently forecasting each variable a cost-effective approach relative to performance improvement, especially when using the lightweight STELLA model. Nonetheless, incorporating other meteorological variables as covariates may enhance prediction performance, which we leave future work.
21

---

STELLA
ITransformer
Airforee
W
mmwm
MMN
MMA
LANN
6-
6-
70
Corrformer
PatchTST
DLinear
10
ground
10-
ground
gro and
wmw M
W
MMA
LAN
4-
Figure 9: Visualization of the prediction results on the GlobalWind dataset.
STELLA
ITransformer
AirFormer
140
340
140
120
120
120
100
100
100
50
60
40
20-
01
01
Corrformer
PatchTST
DLinear
140
ground prediction
140
ground prediction
140
ground buth
120
120
120
100
100
100
80-
60-
60-
40
20
20
0-
120
Figure 10: Visualization of the prediction results on the ChinaPM2.5 dataset.
22

---

STELLA
ITransformer
AirFormer
20-
ground
ground
pridiiction
15-
15-
15.
Corrformer
PatchTST
DLinear
20-
ground truth predicioo
20-
ground truth predicton
20-
ground truth prediction
10-
50
80
20
40
Figure 11: Visualization of the prediction results on the ChinaTemp dataset.
Extreme Weather Forecasting, When forecasting target time series with violent fluctuations, the model may provide over-smooth predictions, leading an inability accurately forecast extreme weather events in practical applications. This is common issue for DL models due to the use of MSE/MAE loss. A possible explanation is that MSE loss compresses the feature representations into constrained space, limiting the model's ability to capture high-entropy features, especially those with significant variability [51]. Incorporating cross-entropy as a classification loss into the loss function may help address this issue [35]. Additionally, although considered the physical principles of atmospheric dynamics, we did not directly incorporate physical constraints into the model's predictions. Doing may help mitigate the issue of over-smoothing and also improve the interpretability of the predictions. We leave this as future work.
Fitting Capacity of MLP Backbone. We recognize that MLPs, due their shallow and simple architecture, may face limitations in fitting capacity. We believe this constitutes the primary bottleneck limiting STELLA's further performance improvement. The motivation of this paper is to demonstrate that even minimalist architectures, when integrated with spatial-temporal knowledge, can achieve SOTA performance while offering superior efficiency. As shown in Table 4, introducing STPE into I'Trasformer leads to a new SOTA on GlobalWind and other datasets, demonstrating that STPE is only effective on MLPs, but also generalizable more expressive architectures. We leave the exploration of alternative architectures for future research.
H Broader Impact
This paper proposes STELLA, a lightweight approach for atmospheric time series forecasting. STELLA demonstrates high training efficiency, which helps reduce energy consumption and benefits domains such as agriculture and the economy. However, the deep learning-based approach may lack interpretability in forecasting results, and regional biases in training data may disadvantage certain populations.
23

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly state the main contributions and scope of the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The paper discusses the limitations Appendix.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]

---

Justification:
The paper provides the assumptions and proofs in the Appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification:
The paper provides detailed descriptions of the datasets, experimental setups, implementation details, and hyperparameters, which are sufficient to reproduce the main results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
25

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The paper provides open access to the data and code in the supplemental material with sufficient instructions to reproduce the main experimental results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cr
ooooolicoyyl for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/ /nips. wctppclicc///cccccccccccccc.cllt /gwui/gss//////ee /Cae//ooeubbiiic commissiinnl ssionPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
The paper thoroughly describes the training and test details.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
The paper provides the standard deviations of the main experimental results and clear explanation of the statistical methods used to compute metrics, ensuring the robustness of the reported findings.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
26

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The paper provides sufficient information on the computer resources.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics /ttpa://neurrie http://weurrpp..cppiiiiiiccc hea.bliieeehhssscddeleeeeeeee SeGGuuddelinnss
Answer:
[Yes]
Justification:
The research aligns with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
The paper discusses the broader impacts in the appendix.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
27

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
The paper credits the owners of any used assets.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperswi twwode.hco...... com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
28

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The paper introduces new assets that are well documented alongside the assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/cooe/mmodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The research does not involve human subjects, so IRB approvals or equivalent review not required.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
29

---

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
The core method in this research does involve LLMs any important, original, or non-standard components.
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to LLM policy (https: //tp:r/inn.ppeeeeeeee cc//onferences///0000 /LLM) for what should or should not be described.
30


Reviews:
[{"review_id": "rPRhq22B6F", "reviewer": "Reviewer_T6sb", "summary": "This paper presents STELLA, a spatial-temporal knowledge embedded lightweight model for atmospheric time series forecasting (ATSF). STELLA utilizes spatial-temporal position embedding with a simple MLP-based architecture. The paper provides theoretical foundations for its effectiveness and computational efficiency, with empirical validation across five real-world datasets demonstrating that STELLA achieves state-of-the-art performance while maintaining high efficiency. The approach to spatial-temporal knowledge integration offers novel insights for ATSF tasks.", "questions": "Please refer to the proposed weakness above.", "limitations": "Please refer to above reviews.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The authors introduce a novel method, STELLA, for ATSF. The model presents state-of-the-art performance on five different weather datasets.\n2. STELLA has significantly fewer parameters, shorter training times, and reduced memory usage, enhancing the scalability for large-scale forecasting.\n3. The authors provide theoretical guarantees for the model's predictive performance from the perspective of atmospheric motion.\n\nWeaknesses:\n\n1. For efficiency analysis, what is the specific advantages of N-independent parameters compared to other models with O(N) parameters (which is deemed acceptable in my perspective)?\n2. The proposed method only models the position of stations while neglects the relative spatial relationships (e.g., adjacency between stations), which show promise for boosting performance and have been explored in spatial-temporal data mining tasks.\n3. The paper does not address whether alternative efficient architectures (e.g., Mamba) could attain similar results.", "was_revised": true, "final_justification": "The author solved my main conern by providing additional experimental results. So I recommend to accpet this paper.", "rebuttal": "*We sincerely thank Reviewer T6sb for their insightful comments. The following addresses their concerns and provides answers to their questions.*\n\n---\n\n1. **Advantages of N-independent parameters**\n\n   The advantages of STELLA's N-independent parameters include: (1) higher computational efficiency and faster inference speed in large-scale forecasting scenarios, and (2) constant memory footprint during deployment that remains unaffected by the number of stations, making it particularly suitable for resource-constrained environments and edge device deployment.\n\n2. **Absolute vs. relative spatial modeling**\n\n   Thank you for raising this issue. According to Theorem 1, atmospheric variables depend on both their spatial coordinates and temporal information. Therefore, we propose that explicitly incorporating absolute spatial positions is a more effective approach for spatial modeling. In contrast, modeling adjacency relationships represents an implicit spatial modeling strategy, as the model remains unaware of actual coordinates. Furthermore, as shown in Fig. 5(b), the spatial embedding can inherently learn adjacency relationships during training.\n\n3. **Other efficient models**\n\n   Thank you for your suggestion. In Table 1, we have compared three efficient architectures (DLinear, NBeats, and FITS). To strengthen our claim, we additionally compare 3 efficient TSF models: S-Mamba [1], SOFTS [2], and SparseTSF [3]. As shown in the table below, STELLA consistently achieves the SOTA performance, demonstrating the effectiveness of STELLA.\n\n   |            | Metric | GlobalWind | GlobalTemp | ChinaWind | ChinaTemp | ChinaPM2.5 |\n   | :--------- | :----- | :--------- | :--------- | :-------- | :-------- | :--------- |\n   | S-Mamba    | RMSE   | 1.958      | 3.001      | 7.301     | 5.012     | 26.74      |\n   |            | MAE    | 1.302      | 1.910      | 5.026     | 3.587     | 14.23      |\n   | SOFTS      | RMSE   | 1.985      | 3.057      | 7.253     | 4.962     | 26.75      |\n   |            | MAE    | 1.325      | 1.951      | 4.993     | 3.529     | 14.34      |\n   | SpaseTSF   | RMSE   | 2.117      | 3.434      | 7.230     | 5.292     | 28.71      |\n   |            | MAE    | 1.442      | 2.337      | 5.050     | 3.840     | 15.69      |\n   | STELLA-10k | RMSE   | 1.933      | 3.041      | 7.118     | 4.167     | 26.32      |\n   |            | MAE    | 1.294      | 2.009      | 4.876     | 3.003     | 14.08      |\n   | STELLA_opt | RMSE   | **1.919**  | **2.724**  | **7.104** | **4.112** | **26.15**  |\n   |            | MAE    | **1.284**  | **1.858**  | **4.869** | **2.975** | **13.85**  |\n\n---\n\n**Reference:**\n\n[1] Wang, Zihan, et al. \"Is mamba effective for time series forecasting?.\" *Neurocomputing* 619 (2025): 129178.\n\n[2] Han, Lu, et al. \"Softs: Efficient multivariate time series forecasting with series-core fusion.\" *Advances in Neural Information Processing Systems* 37 (2024): 64145-64175.\n\n[3] Lin, Shengsheng, et al. \"SparseTSF: modeling long-term time series forecasting with 1k parameters.\" *Proceedings of the 41st International Conference on Machine Learning*. 2024.\n\n---\n\n*Thank you for your time and efforts. We hope this has addressed your concerns and answered your questions. Please don’t hesitate to reach out if you have any further questions.*"}, {"review_id": "rpKxGNNXVd", "reviewer": "Reviewer_kVQ6", "summary": "This paper proposes STELLA, a lightweight atmospheric time series forecasting model that integrates spatial-temporal position embedding (STPE) into a simple MLP architecture. Theoretical analysis reveals that STPE can inherently capture spatial-temporal correlations, even without attention mechanisms. Extensive experiments on five real-world datasets demonstrate that STELLA achieves competitive performance with significantly fewer parameters and lower computational costs.", "questions": "1. Have the authors explored alternative approaches for incorporating spatial-temporal knowledge besides STPE?\n2. What motivated the selection of MLP as the backbone, given its known limitations in capturing complex correlation?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. This work bridges a critical gap in atmospheric time series forecasting through its incorporation of spatial coordinates and temporal information that prior works have consistently overlooked.\n2. This paper proposes a simple yet effective baseline model that outperforms existing models with high efficiency.\n3. The spatial-temporal position embedding (STPE) module is model-agnostic and can be integrated into other models to enhance their performance.\n4. This paper provides extensive experiments and rigorous theoretical analysis to validate the effectiveness of STELLA.\n\nWeaknesses:\n\n1. This work focuses on spatial correlations among stations but does not explicitly account for correlations among atmospheric variables, which might further improve predictive performance.\n2. While the experimental comparison covers substantial baselines, the evaluation would be strengthened by including comparisons with recent advanced time series forecasting models such as DUET [1] and FreDF [2].\n3. The paper would benefit from a clearer justification for using MLP as the backbone, particularly discussing its advantages over other architectures for this specific application.\n\n[1] Qiu X, et al. Duet: Dual clustering enhanced multivariate time series forecasting. KDD 2025.\n[2] Wang H, et al. FreDF: Learning to forecast in frequency domain. ICLR 2025.", "was_revised": true, "final_justification": "In the rebuttal, the authors addressed all the weaknesses proposed in the review. \nNow that I do not have remaining concerns,  I decide to keep the original rating after the rebuttal.", "rebuttal": "*We sincerely thank Reviewer kvQ6 for their insightful comments. The following addresses their concerns and provides answers to their questions.*\n\n---\n\n1. **Correlations of atmospheric variables**\n\n   Thank you for raising this important point. Actually, Eq.(13) can be expressed as\n   $$\n   \\nu_f=\\mathbf A\\nu_h+\\mathbf F\\left(\\nu_h, \\mu_h(\\lambda,\\phi,z,\\tau),\\lambda,\\phi,z,\\tau\\right),\n   $$\n   where $\\mu_h$ is the historical data of specific covariates correlated to $\\nu$. Since $\\mu$ itself is a function of $\\lambda,\\phi,z,\\tau$, it can be treated as a hidden variable and Eq.(13) still holds. The reason for this approach is twofold: (1) Existing datasets such as GlobalWind and GlobalTemp do not include covariates. (2) Most of the previous works do not support simultaneous modeling of correlations between stations and between atmospheric variables.\n\n   The aforementioned reasons make it challenging for us to incorporate covariates into the task for evaluation. However, STELLA can incorporate the covariates. The most straightforward approach is to embed the covariates into the hidden space and add them with STPE to the embedded inputs.\n\n2. **Advanced baselines.**\n\n   Thank you for your kind suggestions. We provide the comparison results with the advanced baselines you mentioned (Since FreDF is a training objective, we follow the original paper's configuration by using iTransformer as the backbone model). As shown in the table below, STELLA still achieves state-of-the-art performance compared to these models.\n\n   |   Model    | Metric | GlobalWind | GlobalTemp | ChinaWind | ChinaTemp | ChinaPM2.5 |\n   | :--------: | :----: | :--------: | :--------: | :-------: | :-------: | :--------: |\n   |    DUET    |  RMSE  |     -      |     -      |   7.273   |   4.480   |   26.61    |\n   |            |  MAE   |     -      |     -      |   5.041   |   3.048   |   14.41    |\n   |   FreDF    |  RMSE  |   1.971    |   2.922    |   7.224   |   4.297   |   26.93    |\n   |            |  MAE   |   1.319    |   1.861    |   5.003   |   3.191   |   14.45    |\n   | STELLA-10k |  RMSE  |   1.933    |   3.041    |   7.118   |   4.167   |   26.32    |\n   |            |  MAE   |   1.294    |   2.009    |   4.876   |   3.003   |   14.08    |\n   | STELLA-opt |  RMSE  | **1.919**  | **2.724**  | **7.104** | **4.112** | **26.15**  |\n   |            |  MAE   | **1.284**  | **1.858**  | **4.869** | **2.975** | **13.85**  |\n\n   *'-' denotes out-of-memory (OOM) errors.*\n\n3. **Motivation of selecting MLP as backbone.**\n\n   Our key insight is that integrating spatial-temporal knowledge into ATSF benefits more than employing complex architectures. This motivates our MLP backbone choice, demonstrating that even minimalist architectures, when integrated with spatial-temporal knowledge, can achieve SOTA performance while offering superior efficiency. Alternative backbones may offer performance gains, and we hope that our work will inspire more architectures that effectively integrate spatial-temporal knowledge.\n\n4. **Alternative approaches for spatial-temporal knowledge incorporation.**\n\n   In experiments, we explored various strategies to incorporate spatial-temporal knowledge. For example, for spatial embedding, we investigated alternative methods to encode coordinates (e.g., discretization + embedding); for temporal embedding, we experimented with encoding different time features (e.g., day-in-year). Our current choices represent the empirically optimal solutions.\n\n---\n\n*Thank you for your time and efforts. We hope this has addressed your concerns and answered your questions. Please don’t hesitate to reach out if you have further questions or need more information.*"}, {"review_id": "dYoA31FmB1", "reviewer": "Reviewer_GjQt", "summary": "The paper proposes STELLA and reveals that spatial-temporal knowlegde plays a pivotal role in atmospheric time series forecasting. The proposed method employs STPE and a simple MLP in place of Transformer layers, achieving SOTA performance with high efficiency (~10k parameters and a hour of training).", "questions": "Q1. In the generalization experiments, STPE are only applied to Transformer-based models. Does it maintain effectiveness when applied to other architectures such as linear models and other MLP-based models?\n\nQ2. The theoretical analysis appears derived from PDEs of atmospheric motion. Have the authors considered implementing Neural ODEs as an alternative framework?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\nS1. The paper presents a novel approach to atmospheric time series forecasting by leveraging STPE with a simple MLP. The theoretical foundation is well-explained and the experimental results demonstrate STELLA's efficacy.\n\nS2. The paper is well-organized and clearly written.\n\nS3. The proposed model can significantly improve the efficiency and scalability to large scale ATSF tasks. The STPE module can also enhance the performance of existing models.\n\nS4. The paper uncovers the superiority of the spatial-temporal knowledge over complex architecture, which challenges the prevailing convention that atmospheric time series forecasting necessitates intricate designs.\n\nWeaknesses:\n\nW1. While the paper presents an interesting finding, the technical contribution is somewhat constrained due to the adoption of well-established components (positional embedding and MLP architectures).\n\nW2. The paper lacks an in-depth discussion on whether integrating STPE with complex architectures like Transformers and STGNNs could yield superior performance. if such combinations prove ineffective compared to MLPs, the underlying theoretical limitations should be analyzed.", "was_revised": true, "final_justification": "The authors' response have addressed my concerns on technical contribution description and generalization. And after reading other reviewers' comments, I would like to recommend the acceptance of this paper.", "rebuttal": "*We sincerely thank Reviewer GjQt for their insightful comments. The following addresses their concerns and provides answers to their questions.*\n\n---\n\n1. **Contribution of our work**\n\n   The contribution of our work lies in:\n\n   (1) **To the best of our knowledge, STELLA is the first dedicated lightweight model for ATSF. Our core innovation is uncovering the superiority of spatial-temporal knowledge over complex architecture.** While MLPs have been explored in TSF, existing methods primarily focus on historical data fitting and fail to effectively capture spatial-temporal correlations.\n\n   (2) **Theoretical contribution**: Beyond empirical results, we provide *theoretical grounding* for STPE’s effectiveness from the perspective of atmospheric dynamics (Section 3.6), which is absent in prior studies.\n\n   (3) **Practical impact**: STELLA demonstrates significant superiority in both performance and efficiency compared to other methods, making it a highly valuable solution for practical application, especially in scenarios with limited computational resources.\n\n2. **Generalization of STPE to other models**\n\n   We notice that W2 and Q1 both concern the generalization of STPE to other models, and we address them together here.\n\n   As shown in Table 4, STPE can be effectively applied to other models to enhance their performance. Notably, when integrated with iTransformer, STPE achieves new SOTA performance on GlobalWind and other datasets. However, not all complex backbones achieve ideal results when combined with STPE, as overly complex models inherently carry overfitting risks.\n\n   STPE can also enhance the performance of other models except for Transformers. The effectiveness of STELLA demonstrates that STPE is applicable to simple *MLP architectures*. Additionally, we supplement the table below with STPE’s performance improvements on DLinear, confirming its validity for *linear models*. However, due to the inherent limitations of linear models’ fitting capacity, the performance gains are limited in complex global forecasting tasks (GlobalWind and GlobalTemp), whereas they are more significant in national tasks (ChinaWind and ChinaTemp).\n\n   |         |           | GlobalWind |           | GlobalTemp |           | ChinaWind |           | ChinaTemp |           |\n   | :------ | :-------- | :--------- | :-------- | :--------- | :-------- | :-------- | :-------- | :-------- | --------- |\n   |         |           | RMSE       | MAE       | RMSE       | MAE       | RMSE      | MAE       | RMSE      | MAE       |\n   | DLinear | w/o       | 2.005      | 1.350     | 3.149      | **2.072** | 7.309     | 5.031     | 4.990     | 3.659     |\n   |         | **+STPE** | **2.002**  | **1.348** | **3.138**  | 2.078     | **7.250** | **5.002** | **4.881** | **3.596** |\n\n3. **Have the authors considered Neural ODEs as an alternative framework?**\n\n   Thank you for this valuable comment. Neural ODEs are indeed feasible choices, as numerous studies have already explored [1,2].  However, the key insight of our work is that integrating spatial-temporal knowledge into ATSF benefits more than employing complex architectures. This motivates us to select a sufficiently simple framework and demonstrate that it can achieve SOTA performance when integrated with spatial-temporal knowledge. In future work, we will explore more alternative frameworks, including Neural ODEs.\n\n   [1] Verma, Yogesh, Markus Heinonen, and Vikas Garg. \"ClimODE: Climate and Weather Forecasting with Physics-informed Neural ODEs.\" *International Conference on Learning Representations*, 2024.\n\n   [2] Tian, Jindong, et al. \"Air Quality Prediction with Physics-Guided Dual Neural ODEs in Open Systems.\" *International Conference on Learning Representations*, 2025.\n\n---\n\n*Thank you for your time and efforts. We hope this has addressed your concerns and answered your questions. Please don’t hesitate to reach out if you have further questions or need more information.*"}, {"review_id": "ggLgLs0WwT", "reviewer": "Reviewer_kupb", "summary": "This paper presents a novel approach called STELLA (Spatial-Temporal knowledge Embedded Lightweight modeL for Atmospheric Time Series Forecasting), which challenges the prevailing assumption that complex architectures like Transformers are necessary for atmospheric time series forecasting (ATSF). The authors argue that spatial-temporal position embedding (STPE) can inherently model spatial-temporal correlations even without attention mechanisms, by integrating geographical coordinates and temporal features intrinsically linked to atmospheric dynamics. STELLA utilizes STPE and an MLP architecture instead of Transformer layers, achieving superior performance with only 10k parameters and one hour of training across five datasets compared to 16 advanced methods. The paper provides theoretical proof of STPE's effectiveness and emphasizes the importance of integrating spatial-temporal knowledge over complex model architectures.", "questions": "（1）The authors should provide a detailed comparison of their method with existing lightweight Transformer models and physics-integrated approaches. Highlight the unique advantages and novel contributions of STELLA in contrast to these prior works. Clarify the specific aspects in which STELLA advances the state of the art.\n  (2) The authors should conduct a thorough analysis to identify the reasons for the limited performance gain.\n  (3) The authors emphasize the significance of spatial - temporal correlation modeling. Existing approaches for spatial - temporal modeling typically fall into two paradigms: asynchronous spatial - temporal modeling (e.g., STTN, CLCRN) and synchronous spatial - temporal modeling (e.g., FourierGNN). The authors seem to adopt a synchronous modeling strategy. However, the paper falls short in elaborating on the details of the spatial - temporal modeling approach.", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n   （1）Efficiency: STELLA demonstrates significant advantages in efficiency metrics, including parameter counts, training time, and memory usage, making it suitable for environments with limited computational resources.\n   （2）Theoretical Support: The authors provide theoretical analysis and proofs to support the effectiveness and efficiency of STELLA, strengthening the credibility of their claims.\nWeaknesses:\n   （1）The innovation of this work is insufficient. On the one hand, a great many works on lightweight Transformer schemes have already been proposed. For example, PatchTST simplifies the Transformer by introducing linear transformations, which cuts down the number of parameters and computational complexity while delivering promising performance in time series tasks. In this context, the approach of STELLA, which replaces Transformer layers with an MLP, is not sufficiently novel despite its simplicity. The authors fail to effectively highlight the unique merits of their method in contrast to other lightweight Transformer models and provide an inadequate comparison to clarify its innovative contributions. On the other hand, a significant body of research has already been carried out on the integration of physical knowledge into models. For instance, physics - guided neural networks utilize physical laws as regularizers or constraints during model training to improve the prediction of physical systems.\n   （2）The model's performance improvement is rather insignificant.", "was_revised": true, "final_justification": "I thank the authors for their rebuttal, which addressed several of my concerns, and I maintain my rating of 4 (borderline accept).", "rebuttal": "*We sincerely thank Reviewer kupb for their insightful comments. The following addresses their concerns and provides answers to their questions.*\n\n---\n\n1. **The innovation of our work (Weakness 1 and Question 1)**\n\n   Thank you for this valuable feedback regarding the novelty.\n\n   **(1) While we acknowledge the existence of numerous lightweight Transformers (e.g., PatchTST, iTransformer), our contribution is fundamentally distinct.** These methods typically reduce the number of tokens in attention to decrease complexity, but they continue to rely on the context aggregation over historical data. In contrast, STELLA challenges this paradigm by demonstrating that spatial-temporal correlations in ATSF can be effectively modeled *without* any attention mechanisms, purely through spatial-temporal knowledge integration via our STPE. This reframing of spatial-temporal modeling shifts the focus from complex architecture design to principled knowledge integration, which is a key novelty of our work.\n\n   **(2) While physics-guided neural networks often incorporate physical constraints or regularization, our approach is different in mechanism and purpose.** Instead of using physical equations as loss terms or auxiliary supervision [1, 2], we draw theoretical motivation from the governing PDEs of atmospheric dynamics to construct input representations (STPE) that inject geographical and temporal priors directly into the model. To the best of our knowledge, this is the first work to prove, both theoretically and empirically, that such a representation alone, even coupled with a simple MLP, can surpass many state-of-the-art ATSF models in both accuracy and efficiency.\n\n   We believe that the discussion on the distinction between lightweight Transformers and physics-guided models would further strengthen our work, and we will incorporate this into the revised version.\n\n   [1] Li, Wenyuan, et al. \"Deepphysinet: Bridging deep learning and atmospheric physics for accurate and continuous weather modeling.\" *arXiv preprint arXiv:2401.04125* (2024).\n\n   [2] Hettige, Kethmi Hirushini, et al. \"AirPhyNet: Harnessing Physics-Guided Neural Networks for Air Quality Prediction.\" *The Twelfth International Conference on Learning Representations*, 2024.\n\n2. **STELLA's performance improvement (Weakness 2 and Question 2)**\n\n   We appreciate the reviewer’s concern and respectfully offer further clarification.\n\n   **(1) The performance improvements are meaningful given the complexity of ATSF tasks.** Large-scale ATSF is a particularly challenging task due to its high dimensional, non-stationary, and chaotic nature. In this context, STELLA demonstrates more significant performance gains than the performance differences between baselines. For example, on the ChinaPM2.5 dataset (Table 1), the MAE improves from 14.39 (MRIformer, 3rd best) and 14.37 (PatchTST, 2nd best) to **13.85 (STELLA_opt, 3.6%)**, highlighting that STELLA’s improvement is not marginal, but rather statistically significant given the complexity of the task.\n\n   **(2) More importantly, STELLA breaks the performance-efficiency trade-off.** A key contribution of STELLA is its ability to outperform or match SOTA models using only **~10k parameters**, orders of magnitude fewer than existing Transformers. As detailed in Table 2, STELLA is **10× to 10,000×** smaller, **100× to 300×** faster, and **~50×** more memory-efficient. Furthermore, STELLA can be easily trained on CPU environments (Table 8), while others require hours on GPUs.\n\n   **(3) We fully recognize that MLPs, due to their shallow and simple architecture, may face limitations in fitting capacity.** We believe this constitutes the primary bottleneck limiting STELLA's further performance improvement. However, our deliberate choice to pair STPE with a simple MLP was driven by a key motivation: to isolate and demonstrate the standalone power of STPE in spatial-temporal modeling. As shown in Table 4, introducing STPE into iTransformer leads to a new SOTA on GlobalWind and other datasets, demonstrating that STPE is not only effective on MLPs, but also **generalizable to more expressive architectures**. We believe our work will inspire future research to develop models with even better performance.\n\n3. **Details of the spatial-temporal modeling approach**\n\n   Thank you for your valuable suggestion. In this paper, we categorize spatial-temporal modeling approaches into two types: (a) relative spatial-temporal modeling based on **context aggregation**, and (b) absolute spatial-temporal modeling based on **STPE**, to highlight the uniqueness of our work. As shown in Fig. 3, most existing methods rely on context aggregation (e.g., GCN modules in STGNNs or attention modules in Transformers) for spatial-temporal modeling, while this paper demonstrates that absolute spatial-temporal modeling, achieved by incorporating spatial-temporal knowledge, is a more effective and efficient approach.\n\n   Your suggestion regarding the classification of synchronous vs. asynchronous modeling is also highly insightful and will help further improve the clarity of our work. Compared to conventional synchronous approaches, asynchronous spatial-temporal modeling accounts for correlations between nodes across different timestamps. In the revised version, we will extend §2 to explicitly incorporate this taxonomy and discuss related works in both categories (e.g., STTN, CLCRN, FourierGNN).\n\nWe appreciate the reviewer’s suggestion, as it will help improve the clarity and positioning of our method in the broader literature.\n\n---\n\n*Thank you for your time and efforts. We hope this has addressed your concerns and answered your questions. Please don’t hesitate to reach out if you have any further questions.*"}]

Author rebuttal:

Meta-review: The paper presents a spatiotemporal deep learning method for atmospheric time series forecasting. The method uses spatiotemporal positions embedding and MLPs (instead of transformer layers), improving upon established methods for atmospheric forecasting.

The reviewers praised the efficiency of the method, its theoretical guarantees and its high performance on its intended application domain. Questions arose concerning alternative efficient models (S-Mamba, SOFTS, SparseTSF), generalizability and comparisons against transformers. The authors have answered these questions, and the reviewers confirmed that their concerns were addressed for the most part. One of the reviewers (Reviewer kupb) had the lingering concern that the model could be more innovative, however the reviewer did acknowledge the considerable performance improvement and is mildly in favor of accepting the paper.

Overall, this paper represents a solid contribution to its application area (atmospheric time series forecasting) with an innovative approach and theoretical insights that might be useful in other domains as well.
Decision: Accept (poster)