S$^2$Transformer: Scalable Structured Transformers for Global Station Weather Forecasting
Authors: Hongyi Chen
Xiucheng Li
Xinyang Chen
Yun Cheng
Jing Li
Kehai Chen
Liqiang Nie
Venue: Accepted by TMLR ()

Abstract: Global Station Weather Forecasting (GSWF) is a key meteorological research area, critical to energy, aviation, and agriculture. Existing time series forecasting methods often ignore or unidirectionally model spatial correlation when conducting large-scale global station forecasting. This contradicts the intrinsic nature underlying observations of the global weather system, limiting forecast performance. To address this, we propose a novel Spatial Structured Attention Block in this paper. It partitions the spatial graph into a set of subgraphs and instantiates Intra-subgraph Attention to learn local spatial correlation within each subgraph, and aggregates nodes into subgraph representations for message passing among the subgraphs via Inter-subgraph Attention---considering both spatial proximity and global correlation. Building on this block, we develop a multiscale spatiotemporal forecasting model S$^2$Transformer by progressively expanding subgraph scales. The resulting model is both scalable and able to produce structured spatial correlation, and meanwhile, it is easy to implement. The experimental results show that it can achieve performance improvements up to 16.8% over time series forecasting baselines at low running costs.

Yun Cheng Published in Transactions on Machine Learning Research (03/2026)
S² 2 Transformer: Scalable Structured Transformers for Global Station Weather Forecasting
Hongyi Chen
chenhonggi@stu.hi hit edu.co
Harbin Institute of Technology, Shenzhen, China
Xiucheng Li Harbin Institute of Technology, Shenchen, China
liziucheng@hit.ede edu.cn
Xinyang Chen
chenzingonn@hiit.tu..o edu.cn
Harbin Institute of Technology, Shenzhen, China
wwn.cheng@sdsc.ctfn wtw.ds.c.ethhccc...
Swiss Data Science Center, Zurich, Switserland
Jing Li
li.jing@hil.c edu.cn
Harbin Institute of Technology, Shenchen, China
Kehai Chen
chenkehai@hit.edu..
Harbin Institute of Technology, Shenzhen, China
Liqiang Nie
mieligiong@gmaikl.cm
Harbin Institute of Technology, Shenzken, China
Reviewed on OpenReview: https: // openreview. net/ forum? id= AL2Vnkno5n
Abstract
Global Station Weather Forecasting (GSWF) is key meteorological research area, critical to energy, aviation, and agriculture. Existing time series forecasting methods often ignore or unidirectionally model spatial correlation when conducting large-scale global station fore- casting. This contradicts the intrinsic nature underlying observations of the global weather system, limiting forecast. performance. To address this, we propose a novel Spatial Struc- tured Attention Block in this paper. It partitions the spatial graph into a of subgraphs and instantiates Intra-subgraph Atlention learn local spatial correlation within each sub- graph, and aggregates nodes into subgraph representations for message passing among the subgraphs via Inter-subgraph Attention-coosiderring considering both spatial proximity and global cor- relation. Building on this block, we develop a multiscale spatiolemporal forecasting model STransformerr by progressively expanding subgraph scales. The resulting model both scalable and able to produce structured spatial correlation, and meanwhile, it easy to implement. The experimental results show that can achieve performance improvements up to 16.8% over time series forecasting baselines at low running costs. The code is available at /tttps://githin
1 Introduction
Global Station Weather Forecasting (GSWF) is vital to modern society, with significant. implications for various sectors, including energy (Dehalwar et al. 2016), aviation (Gultepe et al. 2019), and agriculture
Ukhurebor et al. 2022). Unlike regular, image-like data structures such as the Earth Reanalysis 5, weather
*Corresooningg author.

---

Published in Transactions on Machine Learning Research (03/2026)
station data comprises precise, fine-graines meteorological observations distributed across irregular spatial locations. Compared to radars and satellites, weather stations offer greater flexibility in acquiring scattered data and involve lower deployment costs (Wu et al.) (023). Currently, physics-based Numerical Weather Prediction (NWP) models are the most widely used and reputedly the most accurate for GSWF. Howover, they are inherently computationally intensive, requiring vast resources during execution. Recently, Time Series Forecasting (TSF) methods, which leverage historical weather observations to predict future con- ditions, have demonstrated superior performance on small-scale weather station datasets and provided a more cost-effective alternative to traditional physics-based prediction models (Han et al. 2024ch. However, most of these methods treat GSWF as a multivariate time series forecasting task, trained and tested solely on single-station datasets. They overlook spatial correlation, contradict the intrinsic nature of the global weather system underlying observations, and thus limit forecast performance.
A more appropriate approach to treat GSWF as a spatiotemporal series forecasting task. Spatiotemporal data a particular type of multivariate time scries data in which cach variate is equipped with a spatial coordinate in a metric space. Aside from the general topology shared by multivariate time series, spatiotem- poral data also possess the spatial proximity induced by the spatial coordinates, which serves prior and has a prominent impact on the topology generation. This is summarized by Tobler's first law of coography-everythiing everything is related to everything else, but near things are more related than distant things (Miller 2004). The spatial proximity can be described by a spatial graph, in which each variate is a node, and an edge is drawn between two nodes their spatial distance is less than a predefined threshold. In the spirit of Tobler's first law, Airphynet (Hettige et al.) (024) and Air- Air-DualODE (Tian et al. 2025) encode the spatial prior into the model by rumning graph convolution on the spatial graph make the prediction. Despite the achievement, these methods are mostly trained and tested on metcorological station datascts with localized regions, which limits their applicability in global scenarios. More importantly, it makes dis- tant node pairs very difficult to exchange information due to the limitations (such as over-smoothing and over-squashing) of the message-passing paradigm (Ma et al. 2023). However, Tobler's second law of geog- raphy points out that the phenomenon external to a geographic area of interest affects what goes on inside (Tobler) 2004), and many works (Cirstea el al. 2022 Guo el al. 2022) have also confirmed that the distant nodes could also manifest strong correlation. To sidestep this, Corrformer (Wu et al. 2023) organizes global stations into a hicrarchical tree structure, with stations under each intermediate node undergoing soquential spatial correlation modeling. Despite achieving linear complexily in modeling global spatial correlation, its unidirectional sequential approach struggles to accommodate the prevalent bidirectional or multidirectional spatial correlation in reality. Inspired by Transformer's 's milestones in NLP and computer vision, many spa- tiotemporal forecasting proposals (Bai et al.) 2020 Wu et al., 2020 Shang et al. 2021 simply discard the spatial graph and learn spatial correlation end-to-end via an attention mechanism. The attention mecha- nism enables instant communication between any two nodes, effectively resolving issues with distant message passing and demonstrating significant potential in GSWF. However, such -aaettion-based methods have two drawbacks: 1) The spatial correlation learning is unstructured (lacking structural information embedded in spatial graph) and contains numerous trivial nonzero entries (noise), the accumulated noise is likely to impair the forecasting performance when station nunber N is large (Appendix C for a formal definition); 2) The computational and memory complexity of generating pairwise spatial correlation both reach O(N)), leading to enormous computation costs for large-scale stations.
strong
To address the two limitations, we propose a novel Spatial Structured Attention Block that respects the first law of geography while also permitting long-distance message passing in this paper. The high-level idea is to partition the spatial graph into a set of subgraphs and instantiate self-attention to learn local spatial correla- tion within each subgraph Iinrra-uuggrph Attention). To capture the global spatial correlation, we further aggregate the nodes to produce subgraph representations and exchange information among the subgraphs via self-attention again (Inter-subgraph Attention). Since the entire block is differentiable, the message passed from distant nodes can be backpropagated through Inter-subgraph Attention to their correlated nodes in a parsimonious manner. To further enable the perception of spatial structure, we encode the shortest path distance between any two nodes as bias term in the spatial attention mechanism. Moreover, we stack the proposed Spatial Structured Attention blocks with residual and gradually increase the subgraph scales to develop our eventual GSWF model TTTransformrr. Such a design brings the following two appealing features: 1) our proposed method adopts the spatial graph to facilitate the perception and exchange of local
2

---

Published in Transactions on Machine Learning Research (03/2026)
spatial information, and thus it can yield sparse structure and reduce both the computational and memory burdens; 2) it permits message passing wetween distant node pairs in a parsimonious way, and thus it is capable of capturing global spatial correlation without incurring extra noise, To summarize:
We propose novel Spatial Structured Attention Block for GSWF, which not only perceives spatial structure but also considers both spatial proximity and global correlation.
Building on the proposed block, we develop multiscale GSWF model Trannsforrer by gradually increasing the subgraph scales. The resulting model is scalable and can produce structured spatial correlation.
Our proposed method is effective yet easy to implement. We evaluate its efficacy and efficiency on global station heather datasets from medium to large sizes. It can achieve performance improvements up to 16.8% over time series forecasting baselines while maintaining low running costs.
2 Related Work
2.1
Spatiotemporal Series Forecasting
Spatiotemporal series forocasting, a subfield of multivariate time scries analysis, has boen explored for decades. Common methods for capturing temporal dependencies include recurrent neural networks (RNNs)
[Zhao et al. 2017 Lai et al. 2018), convolutional neural networks (CNNs) (Bai et al.
2018 Wu et al. 2022
and Transformer- bassddd models (Vaswani 2017 Wu et al. 2021 Zhou et al. 2022 Nie et al. 2023). Addition-
ally, lulti-layer perceptrons (MLPs) have been applied for time series forecasting (Zeng et al. 2023 Challu
eet al. 2023 Wang et al., 2024b), showing that even simple models can effectively extract strong lemporal periodic patterns. Beyond temporal dependencies, spatial correlation is equally critical in spatiotemporal forecasting. The advancement of graph neural networks (GNNs) offers an effeetive way to model unstruc- tured spatial adjacency correlation. In the spirit of Tobler's first law of geography, DCRNN (Li et al. 2017) and TGCN (Zhao et al. 2019) leverage spatial graph based real-world distance and propose to fuse the local spatial information via graph convolution operation. However, it makes the distant node pairs very hard to exchange information due to the limitations of the pesssge-ppsssigg paradigm (Ma et al. 2023
violating Tobler's second law of geography. Subsequently, adaptive GNN-based methods have been proposed to solve this problem. AGCRN (Bai et al.) 2020) and MTGNN (Wu et al. 2020) learn a representation for each series and then generate the correlation graph via pairwise node interactions. GTS (Shang et al. 2021
and STEP (Shao al., 2022b) directly learn discrete graph based on historical time series. Benefiting from naturally constructing a fully connected graph with learnable edge weights, the self-attention mechanism is also commonly adopted method for capturing global and dynamic spatial correlation (Jiang et al. 2023
Liu et al. 2024b Wang et al. 2024cb. Nevertheless, the learned spatial correlation matrix in such methods is unstructured (lacking structural information embedded in the spatial graph) and contains a large fraction of trivial nonzero entries (noisc). The accumulated noise is likely to impair the forccasting performance when is large. Moreover, they require O(N2) computational complexity, impeding their application in large-scale datasets. Several researchers have developed scalable spatiotemporal forecasting methods to accommodate larger datasets. Detailed related work on this aspect is provided in Appendix A
2.2 Data-driven Numerical Weather Prediction
In recent years, data-driven Numerical Weather Prediction (NWP) models based on machine learning have developed rapidly. Models including Kangu-Weather (Bi et al. 2023 GraphCast (Lam et al. 2023 and Aurora (Bodnar et al. 2025) have demonstrated the ability to surpass conventional physics- based NWP models in terms of forccast accuracy and operational effectiveness. However, as they operate on grid spaces, they may not be optimal for global station weather forecasting. A direct method is to treat GSWF as an independent time series forecasting task and predict meteorological factors for each station individually [Karevan & Suykens) 20200 Hewage et al. 2020 Wu et al.) (201)). However, global weather constitutes an integrated system with multi-scale interactions. These methods overlook spatial correlation, contradict the intrinsic nature of the global weather system underlying observations, and thus limit forecast. performance.
3

---

Published in Transactions on Machine Learning Research (03/2026)
To address this issue, Airphynet (Hettige et al. 2024 and Air-DualODE (Tian et al. 2025) encode the prior into the model by running graph convolution on the pre-defindd spatial graph. Nevertheless, these methods are mostly trained and tested on meteorological station datasets with localized regions. Learning from localized regional data often fails to capture broader spatial patterns. Furthermore, models overfitted to specific regions lend to lack generalization capability, limiting their applicability in real-world scenar- ios. To accommodate global station weather data, Corrformer (Wu et al., (023) organizes global stations into a hierarchical tree structure, with stations under each intermediate node undergoing sequential spatial correlation modeling. Despite achieving linear complexity in modeling global spatial correlation, its unidirec- tional sequential approach struggles to accommodate the prevalent bidirectional or multidirectional spatial correlation in reality.
3 Preliminaries
Global station weather data refors to multivariate time scries data in which cach scries associated with a station's spatial coordinates. Specifically, we define the spatial coordinates of all stations as R Φ RN and the multivariate time series X E I NxTXC records C-dimensional meteorological variables collected by N weather stations over T time steps. With given threshold e, the spatial coordinates of these stations induce spatial graph G = (V, E), where each node corresponds to a weather station (i.e., [V] = NN and two nodes are connected by an edge e E E if their spatial distance is smaller than e. Besides, we use A E NNN to represent the adjacent matrix of G. Given the past T steps historical observations X1-T+1: along with spatial coordinates X, & and the spatial adjacont matrix A, the goal of global station weather forccasting is to predict the future F steps of meteorological variables X(+1++F,
X++1++ = FX 6.
(1)
where Fo(-) denotes the data-driven weather forecasting model parameterized by 0.
4 Methodology
The framework of our proposed forecasting model, dubbed Scalable Structured Transformer (ransfoorrmrr) is shown on the left of Figure We first perform spatial graph partitioning for global meteorological stations and calculate their spherical harmonics to extract spatial structure information (Section 1.1. Following the setting of previous work (Liu et al. 2024b; Fang et al.] 2025), we reshape and transform the historical observation Xt-T+1: E IR Xx into the input embeddings Xemb E NXD with a linear map, where D is the embedding dimension. Next, the embedding and graph partition are fed into the Spatial Structured Attention Block (as described in the right of Figure [ and Section 4.2) to fuse the information and form the contextual representations, which will be used to make the forecasting. Building on the proposed blocks, we further develop multi-scale attention mechanism to enhance representation learning by stacking multiple Spatial Structured Attention blocks with various spatial scales (Section 4.3)
4.1 Spatial Information Preprocess
In the preprocessing stage, we focus on extracting spatial structure information from input metadata, includ- ing spatial graph partitioning and spherical harmonics calculation. Notably, this stage does not introduce scalability bottlenecks. Compared to the training duration, the time spent on preprocessing is negligible. Detailed efficiency analysis is provided Section 5.3
Graph Partition. Our key observation is that global meteorological observations often exhibit stronger correlation within local areas, which is also supported by Tobler's first law of geography. This motivates us to partition the spatial graph G' into a set of subgraphs g = {G.) where G; = (V, E.}. represents a subgraph of G, satislying UG, G and nG Ø. Then, we can fuse the information locally in each subgraph with self-attention. For graph partitioning algorithms, we opt for the METIS algorithm Karypis
& Kumar 1998) given its efficiency and balanced subgraph outputs.
Location Embedding. In addition, location information serves as valuable metadata in numerous geospa- tial applications, including GSWF. While sine-cosine embedding methods have proven effective in trans-
4

---

Figure Left: The framework our proposed Scalable Structured Transformer. Right: The structure of Published in Transactions on Machine Learning Research (03/2026)
Scabable Structured Transformer
Output: F Rexxxe
Multiscale Spatial Structured Atteation
Prepracess Stage
Projection
SSABlock
Reshape
(G.)
g = (G.)
Fuse
SSABlock
SA Atap
Feed- forward
SSABlock
Inter- subgraph
SAA bies
Attention
Q
Patch Pool
SSABlock
Feed- forward
Posittio Encoding
Intra- subgraph Attention
SA Bias
Data Embedding
Graph Partition
SH Calculation
Input:
Reshape (G.)
A R
NXN
X A
r R
TXNXO
the proposed Spatial Structured Attention Block.
formers, they assume a reclangular domain for longitude and latitude coordinates, which fails to capture the Earth's spherical geometry accurately. Inspired by Geographic Location Encoder (Rubwurm et al.) 2024). employ spherical harmonic basis functions positional embeddings, which well-defined globally (in- cluding the poles) and enable better discrimination of weather stations distributed across the globe. These spherical harmonics are precomputed from coordinates, with their weights learned directly. Specifically, we define all station location embedding as:
SH(A, x) = o I
l oell Y (An,
(2)
Xemb XSH() +))
(3)
where indicates the stack operator and || indicates the concatenation operator. wi" is learnable weight shared across all stations and Y is an orthogonal spherical harmonic basis function with increasingly higher- frequency degroes and orders m. In practice, we choose a maximum number L instead of co in Eq. 2 A
detailed introduction to spherical harmonics is provided in Appendix B
4.2 Spatial Structured Attention Block
In this section, we introduce details of the Spatial Structured Attention (SSA) Block, guided by Tobler's first and second laws of geography. Firstly, reshape the input Xemb from I NxD to MxXD according to the graph partition result G, where P is the number of subgraphs, and M is the number of nodes in the largest subgraph. If the number of nodes in the subgraph is less than M, we pad it with zeros and mask it in subsequent altention operations.
Intra-subgraph Attention. In light of Tobler's first law of geography Miller 2004 we first employ intra-subgraph attention to model the local spatial correlation. Formally, let X be the input of the block (e.g., Xemb) and X, E MD denote the embedding in the p-th subgraph, the representations within the
5

---

Published in Transactions on Machine Learning Research (03/2026)
subgraph are updated as follows.
0p softmax
(4)
vd
y, FFN P))
(5)
Pp. K. V = (WQ. WK. Wy
(6)
where Wo. WK and Wv are learnable parameters that transform X, into different semantic spaces, ap E R MMM M is the intra-subgraph attention map that captures the local spatial correlation the p-th subgraph, and Y, is the learned contextual representations that will be fed to the subsequent inter-subgraph allention.
suber-subgraph Attention. Tobler's second law of geography (Tobler) 2004) states that the phenomenon external to geographic area of interest affects what goes inside. However, mentioned in Section []] directly using the attention mechanism will lead to quadratic complexity O(N2) and incur extra noise that impairs the forccasting performance. Thus, we innovatively learn the attention between different subgraphs to approximate the global attention mechanism. We first apply mean pooling to Y, to obtain the subgraph representation Sp E RDD and then stack them to produce S A (2; E PRPx Then, employ the inter-subgraph attention to exchange information across subgraphs as follows.
Q'K"T
a' softmax
(7)
Vd
S' FFN (a'V")
(8)
Q' K' V = (WQ', WK. W
(9)
where Wo. Wk, and Wy are learnable parameters. a' E PxP captures the global spatial correlation among subgraphs, which allows us to approximate the global attention mechanism by assigning the same attention value apg to any pair of nodes from the subgraphs p and q. This can be considered as a sort of implicil regulorization that encourages the model to focus on mining groups of nodes that share similar patterns while ignoring unrelated noise. In other words, we only permit the distant node pairs exchange information in parsimonious manner, and only valuable information be transmitted across groups.
In the end, to take both local and global spatial information into account, expand the shape of S' in Eq. from Rfx gDxD to Pxxxx which is concatenated with the local representation and then transformed by a lincar map to produce the output:
X' = ('=W(Y\ss')
(10)
where || indicates the concatenation operator and W RDDD the parameter of linear map layer.
Spatial Attention Bias. Self-attention serves the core computational module in our proposed block, however, it is oblivious to the local graph structures due to its permutation invariant, property. To encode the structural information into the attention mechanism, encode the shortest path distance between any two nodes as bias in spatial attention inspired by Graphormer (Ying et al. 2021). Specifically, we procompute the shortest path distance (SPD) between two stations. For unconnected stations, the SPD is set to a special value of -1. We caleulate the spatial attention bias matrix SA blas by element-wise embedding the SPD matrix with a learnable scalar. We then revise Eq. g as follows:
Q Vd
Op = softmax
+ -SA
bias
(11)
SAB bins
(SPD (A,))
(12)
where bias M is the intra-subgraph attention bias matrix in the p-th subgraph and a is an element- wise learnable scalar shared across all blocks. We can modify Eq. 7 in the same manner.

---

Published in Transactions on Machine Learning Research (03/2026)
Table Dataset statistics.
Dataset
Frequency
Time Span
Stations
Variables Name
WEATHER-5K
1 hour
2014-2023
5672
Temp, Dewpoint, Wind Rate, Wind Direc, Sea Level
NCEI Global
1 hour
2019-2020
3850
Wind, Temp
4.3 Multiscale Spatial Structured Architecture
In practice, the spatial correlation often presents multiscale structures due to the multiscale property of underlying physical dynamics. To capture the intrinsic multiscale property, we develop multiscale spatial balance architecture by stacking L SSA blocks by gradually increasing the subgraph scales. Specifically, we perform graph partition with various subgraph scales to produce L sets, G1. Nt.92...,,9 9r such that [g,1 = 1.-11/2. L is set to 2 by default, and we empirically find that it performs quite well in practice. Such a design also brings two additional benefits: 1) it progressively expands the receptive field of the node attention mechanism, which aligns with the spatial diffusion process of the global weather system's 's physical dynamics; 2) it offers the chance for spatially closed nodes at the boundary of two subgraphs to exchange information in the high-level block.
Forecasting. We reshape the output of L-th block ( from IR PxXxD to NR according to the graph partition result and produce the multi-step prediction Xiotoo through a linear projection. The model is optimized by minimizing the mean absolute error:
EA'
1o+E- 21= D |inge - Insool
L(Xio:to+P Xlosto+)
(13)
NxFxC
Complexity Analysis. Our method achieves significant computational efficiency improvements by focusing on Iutra-subgraph and Inter-subgraph Attention. Intra-subgraph Attention (for subgraphs with N nodes) has complexity )
Ox D), and Inter-subgraph Attention (for cross-subgraph correlation) Minimizing this (via derivative calculation)
has D), leading to an overall complexity of gives O(2N4/3 when P = N2/33 with better scalability than quadratic-complextty methods. Our model's efficiency can be further enhanced by linear attention; though Corrformer (specifically designed for GSWF tasks) has a better theoretical complexity of O(NT log DD), it performs poorly in practice. This due operations (for inferred node order), which disrupt data layout,
to its temporal increase memory latency (hindering hardware parallelism), and raise memory occupancy (via intermediate data storage). Detailed efficiency analysis is provided in Section 5.3
5 Experiments
In this section, we evaluate our approach using two benchmark datasets (Section 5.2 and Appendix E Sec- tion 5.3 presents the efficiency analysis, and Section 5.4 describes the ablation study. A more comprehensive analysis of the graph partitioning strategy is provided in Appendix F The sensitivity of hyperparameters is detailed in Section 5.5 and Appendix G To gain a more profound understanding of our model, we also
conducted visualizations, which are included in Scction 5.6 and Appendix H
5.1 Experimental Setup
We evaluate the performance and efficiency of the proposed method on two global station weather forecasting benchmarks. The first benchmark is the WEATHER-5K dataset (Han et al. 2024c). which includes crucial weather elements collected from 5672 global weather stations over ten years. The socond benchmark is the NCEI Global dataset (Wu et al.] 2023), which contains the hourly averagod wind speed and hourly temperature of 3, 3,850 stations worldwide from 2019 to 2020. Dataset statistics are presented in Table 1 and further particulars available in Appendix []
Datasets.

---

Published in Transactions on Machine Learning Research (03/2026)
Table 2: Global station weather forecasting performance comparison. The best results are highlighted in bold, while the sccond-best. results underlined.
Dataset
WEATHER-5K
NCEI Global
Variable
Temperature
Dewpoint
Wind Rate
Wind Direc.
Sea Level
Wind
Temp
Metric
MAE
MSE
MAE
MSE
MAE
MSD
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
NWP Model
1.76
7.39
1.85
7.94
1.48
4.53
63.8
7158.3
0.86
2.68
1.59
5.00
1.91
13.45
Informer
1.88
7.51
1.94
8.30
1.30
3.62
60.7
6906.9
2.01
10.56
1.58
4.93
4.42
33.29
Autoformer
1.93
8.64
2.06
9.57
1.42
3.97
66.5
7710.0
2.26
12.78
1.47
4.69
2.25
10.14
Pyraformer
1.75
6.92
1.83
7.88
1.30
3.58
61.8
6930.2
1.90
9.72
1.51
4.61
3.67
23.33
STID
1.78
7.09
1.83
7.88
1.28
3.53
60.9
6722.2
1.87
9.42
1.34
3.83
1.99
8,46
MTGNN
1.84
7.36
1.89
8.18
1.30
3.59
62.1
6854.5
1.91
9.64
1.37
3.90
2.07
8.51
Corrformer
1.99
8.21
2.09
9.47
1.38
7832.3
2.19
12.30
1.30
3.89
1.89
7.71
iTransformer
1.64
5.94
1.67
6.57
1.24
3.31
58.6
6570.6
1,47
5.53
1.33
3.87
1.90
7.50
RPMixer
1.77
6.60
1.83
7.45
1.28
3.52
60.3
6607.1
1.65
6.45
1.43
4.02
2.47
11.08
PatchSTG
1.65
5.94
1.68
6.58
1.20
3.15
57.2
62544.8
1.41
4.92
1.36
3.89
2.21
9.59
STTransformer
1.47
4.99
1.52
5.67
1.17
3.09
55.5
6135.7
1.26
4.08
1.30
3.61
1.87
7.50
Baselines, We compare our method with the following baselines: (1) Physics- based NWP model: ECMWF-
HRES EC) for WEATHER-5K datasot and ERA5 (rcanalysis, 0.25%) (Hersbach al. 2020) for NCEI Global
dataset; (2) Pure time dependencies modeling methods: Informer Zhoou et al. 2021), Autoformer (Wu al.
2021), Pyraformer (Liu et al. 2022), STID (Shao et al. 2022a
Spatial correlation modeling methods:
MTGNN (Wu et al. 2020 Corrformer (Wu et al. 2023 iTransformer ÇLiu et al. 2024b); (4) Scalable
spatial correlation modeling methods: RPMixer Yeh et al. (2024), PatchSTG (Fang et al. 2025). Notably,
Pyraformer and Corrformer are the Best, Time Series Forecasting methods reported on the WEATHER-5K
and NCEI Global datasets. More details baselines are provided in Appendix D
Evaluation Metrics. We conduct a comprehensive comparison using various evaluation criteria from the performance and efficiency perspectives. We evaluated performance using the mean absolute error (MAF) and mean square error (MSE). We consider efficiency by measuring both the training wall-clock time and maximum memory usage during training.
Implementation details. Given our focus on short-term global station weather forecasting, we predict one day into the future using the past two days data, where the input length 48 (hours) and the predicted length is (hours). The key parameter settings are detailed in Appendix D All experiments this study are implemented using PyTorch Paszke al. (2019) and conducted on an NVIDIA RTX 4090 GPU with 24GB memory. We run each experiment three times and report the average results.
5.2 Performance Comparison
Table 2 presents the average forecast performance in 24 hours of all methods with input length of 48 hours. Notably, following previous studies, the models on the WEATHER-5K dataset adopt unified architecture to predict all variables at once, while those on the NCEI Global dataset are trained and tested separately for each variable. The best results are highlighted in bold and the second-best in underlined. To ensure fair comparison, all baselines are implemented with their official oofigurations. The experimental conclusions are as follows:
First, although current physics- based NWP models are regarded as the most accurate weather forecasting models, time series forecasting methods have demonstraaed comparable performance in short-term forecast-
8

---

Published in Transactions on Machine Learning Research (03/2026)
Table 3: Global station weather forecasting efficiency comparison. BS: batch size used in model training.
Mem: max memory used during training (in gigabytes). Time: total training time (in hours). The bost results are highlighted in bold.
Dataset
WEATHER-55
NCEI Global
Metric
BS
Mem
Time
BS
Mem
Time
MTGNN
64
179.3
12.7
64
64.3
0.6
Corrformer
8
143.6
29.4
143.2
9.0
iTransformer
16
151.2
7.5
24
159.5
0.3
S Transformer
64
52.0
5.0
64
43.3
0.2
ing tasks. Second, among TSF methods, the suboptimal performance of Informer, Autoformer, Pyraformer, and STID highlights the critical role of spatial correlation modeling. In contrast, spatial correlation model- ing methods (MTGNN, I'Transfommer and Corrformer) perform more effectively on the medium-scale NCEI Global dataset, underscoring the significance of this modeling paradigm. Our method outperforms all baso- lines, delivering to a 5.7% performance improvement on the Wind variable of the NCEI Global datasel, attributed to its structured dynamic spatial correlation modeling. Furthermore, the limitations of the afore- mentioned models, such as the quadratic complexity of MTGNN and i'rransformer, and the large parameter scale of Corrformer due to its complex structure, restrict their application to the large-scale WEATHER-5K dataset, further emphasizing the advantages of scalable spatial correlation modeling approaches. However, methods that simplify spatial receptive fields (PatchSTG) and mix channels in the spatial dimension (RP- Mixer) both suffer from information loss, caused by insensitivity to spatial structural information or spatial downsampling. In contrast, our method achieves state-of-the-art performance across most variables on the WEATHER-5K dataset, with an improvement of up to 16.8% on the Temperature variable. This superior performance stemss from the structured spatial correlation constraints of our method: accurate perception and capture of local spatial correlations and restricted global spatial correlations, which preserve key infor- mation and filter out noise. We further provide the results and analysis of long-term global station weather forecasting, along with comparison with physics-informed models, in Appendix E
5.3 Efficiency Study
First, our proposed model consists of two stages; preprocessing and training. Verified by multiple exper- iments, the preprocessing time for both datasets is stably within 2 minutes. Compared with the training duration, the preprocessing time negligible. To further evaluate model efficiency, we compare our method with existing spatial correlation modeling methods in three key metrics: batch size, maximum memory usage during training, and total training time consumption. The models are trained with multi-GPU parallel acceleration using torch. www.nnnDaataollilll As shown in Table 3 the key observations are as follows.
Existing spatial correlation modeling models (MTGNN, (Transformer) exhibit rapid growth in computational and/or memory consumption as the number of nodes increases, primarily due to their quadratic complex- ity. Although Corrformer only models unidirectional spatial correlations, its complex Encoder-Decoder architecture (different from the aforementioned Encoder-Only models) leads a large memory footprint, which further results in smaller batch size and longer training time. In contrast, the proposed model in this paper confines fine-gre fine-graindd spatial correlation modeling within local subgraphs, enabling efficient long- distance information exchange via inter-subgraph spatial correlation modeling. This design not only filters out noise to improve performance but also effectively reduces memory consumption. Across the two datascts, the elliciency ranking of all models remains consistent: TTannsforeer > iTransformer > MTGNN > Cor- rformer. Particularly, compared with Corrformer (specifically designed for GSWF tasks), Trrransformer roduces memory consumption by approximately 64% and improves inference speed by about 83% on the WEATHER-5K dataset; on the NCEI Global dataset, it achieves memory reduction of around 70% and inference speedup of roughly 98%.
9

---

Published in Transactions on Machine Learning Research (03/2026)
Table Ablation study on the WEATHER-5K dataset. The best results are highlighted in bold.
Variable
Temperature
Dewpoint.
Wind Rate
Wind Direc.
Sea Level
Metric
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
w/o. Metis
1.64
5.99
1.71
6.83
1.25
3.40
59.25
6607.08
1.72
7.88
w/o. SH
1.49
5.08
1.54
5.75
1.18
3.09
56.02
6206.36
1.30
4.37
w/o. Intra-Att
1.65
6.03
1.70
6.77
1.26
3.44
59.74
6639.95
1.53
6.02
w/o. Inter-Att
1.57
5,63
1.62
6.34
1.23
3.30
58.44
6493.64
1.52
6.30
w/o. SA
1.48
5.03
1.54
5.72
1.18
3.06
55.75
6171.15
1.26
4.13
w/o. MA
1.59
5.12
1.57
5.81
1.19
3.10
55.86
6188.84 84
1.28
4.18
S Srrnnsfforerr
1.47
4.99
1.52
5.67
1.17
3.09
55.50
6135.69
1.26
4.08
1.312
3.650
7,800
1,310
3.640
MAE
7.700
1.308
3.630
1.890
7.600
1.306
MSE
1.875
MSE
3.620
(a) Blocks on Wind
(b) Blocks on Temp
7.950
-D- MSE
MAE
3.675
1.540
3.663
7.800
MAE
1.314
MSE
3.650
1.311
3.638
1.308
7.500
512
1024
256
768
1024
(c) Dimension Wind
(d) Dimension Temp
Figure 2: Parameter sensitivity analysis
5.4 Ablation Study
We perform an ablation study on the WEATHER-5K dataset to validate the effectiveness of the proposed modules. Specifically, we consider the following variants of our proposed model: (1) w/o. Metis: The
METIS graph partitioning method substituted with random partitioning. (2) w/o. SH: The spherical
harmonic position encoding the preproccssing stage is removod. (3) w/o. Intra-Att: The intra-subgraph attention is removed, and only the spatial correlation between subgraphs is captured. (4) w/o. Inter-Att: The inter-subgraph attention is removed, and only the local spatial correlation is captured. (5) w/o. SA: The spatial attention bias matrix in both inter-subgraph and intra-subgraph attention is removed. (6) w/o. MA: The hierarchical multi-scale architecture is superseded by a sequence of single-scale blocks. As shown in Table 4 the key observations are as follows.
First, the performance drop of w/o. Metis confirms the importance of graph partitioning algorithms. METIS, hicrarchical partitioning algorithm, ensures subgraph balance with only the partition count parameter. Second, w/o. Intra-Att. shows the most severe performance decline, indicating that local
10

---

Published in Transactions on Machine Learning Research (03/2026)
0.12
0.00
Subgraph 21 SrTransformer
Subgraph 21 -Transformer
Max: 0.7589, Min: 0.0000
Max: 0.1292, Min: 0.0000
0.60
0.10
0.05
0.30
0.05
0.25
Subgraph 0.0000
Subgraph
0.1209,
Figure 3: Visualization of intra-subgraph attention matrices for TTrrnsforrer and iTransformer.
spatial correlation is critical for accurate global station weather forecasting, consistent with Tobler's first law of goography. Mcanwhile, w/o. Inter-Att exhibits the significant drop, suggesting global spatial correlation also aids precise forecasting, aligning with Tobler's second law of geography. Finally, the performance of w/o. SH, w/o. SA and w/o. MA validate the effectiveness of the proposed spherical harmonic position encoding, spatial attention bias, and structured scale progression in capturing spatial correlation.
5.5 Parameter Sensitivity Analysis
We evaluate the sensitivity of hyperparameters (including the number of blocks L and embedding dimen- sion D) on the NCEI Global dataset. Figure shows that model performance improves with increasing L, achieving the best results at J, = 2 for the Global Temperature dataset and = for the Global Wind dataset. Further increasing L leads to a degradation
performance. Thus, we select L 2 for model efficiency. Similarly, optimal performance achieved with an embedding dimension of D 768, with no further performance gain from increasing D. This indicates that a small model suffices learn spatiotem- poral knowledge in global station weather forecasting. The sensitivity analysis of look-back window length T and the number of subgraphs P is presented in Appendix G
5.6 Visualization
To gain deeper insights into our proposed model, we conducted supplementary visualization experiments, focusing comparative analysis of the intra-subgraph attention matrix of our model (S*Transformer) and the attention matrix of the i'Tranformer model. Specifically, we selected the intra-subgraph attention matrices corresponding to two subgraphs from the test data. For the purpose of comparison, we extracted the relevant. subgraph attention matrix fragments from iTransformer's global attention matrix, with the extraction process strictly guided by the node indices the two aforementioned subgraphs.
As illustrated in Figure 3 attention values in its corresponding subgraph regions seatt and small (with maximum of approxi-
owing the global receptive field of iTranforrmrr' attention mechanism, the
11

---

Published in Transactions on Machine Learning Research (03/2026)
mately 0.12). When the number of such scattered trivial values is large, they tend to introduce trivial noise and impair model performance. In contrast, our model proactively restricts the receptive field to confine attention within the subgraph- an approach that not only aligus with relevant geographical laws but also yiclds more concentrated and larger attention values (with a broader distribution attention weights). This confirms that. our model effectively avoids noise introduction during local modeling. The quantitative discrepancy between this structured spatial correlation and iTrrasffrmer's unstructured attention serves as direct evidence of noise reduction, validating the rationality of our model design. We provide additional visualization analysis results in Appendix H
6 Conclusion and Limitation
In this paper, we propose a novel Spatial Structured Attention Block that not only perceives spatial struc- ture but also considers both spatial proximity and global correlation. Specifically, we partition the spatial graph into a set of subgraphs and utilize the Intra-subgraph and Inter-subgraph Attention to learn local and global spatial correlation. Building on the proposed block, we develop a multiscale model Trannsoormer by progressively increasing the subgraph scales, The resulting model is scalable and can produce struc- tured spatial correlation. Performance comparison and efficiency analysis validate the superiority of our method in medium and large-scale global station weather forecasting scenarios. Ablation studies confirm the effectiveness of the model designs, and we further explore the hyperparameters in model construction.
However, we noticed that our model still lags behind the numerical weather prediction model for longer lead times. This is mainly because an increase in forecast duration amplifies the non-linear dynamics of the atmospheric system, and pure data-driven methods struggle to fully capture their evolutionary laws. In the future, we would like explore the nascent regime of data-driven and physics-informed paradigms to enhance the model's ability to predict long-term weather processes.
Limitation.
Broader Impact Statement
In this paper, we propose a novel scalable spatiotemporal series forccasting model that captures structured spatial correlation guided by Tobler's laws of geography to enhance global station weather forecasting while maintaining low running costs. By mitigating resource barriers, this framework facilitates the responsible deployment of climate and disaster forecasting in underserved regions. Ultimately, our work democratizes access to reliable predictive tools, fostering global resilience and data-driven disaster management.
Acknowledgments
This work is supported by Guangdong Basic and Applied Basic Research Foundation under Grant No. 2025A1515012932 Shenzhen College Stability Support Plan under Grant No. CXWD200200112233300 Shenzhen Science and Technology Program No. JCY:992212202123500000 2DSY$2023362000120000 and KQTDD22247791102554666
References
Eemw/-hree.
URL https://sww.ecmet http:///m/forrrressss///oottttta -ud-orpor
changes- -at----ewwffmmeeeeee Accessed: 2024-10-02.
Lei Bai, Lina Yao, Can Li, Xianzhi Wang, and Can Wang. Adaptive graph convolutional recurrent network for traffic forecasting. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
Shaojie Bai, J Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint Xxie:180300121 2018
Kaifeng Bi, Lingxi Xie, Hengheng Zhang, Xin Chen, Xiaotao Gu, and Qi Tian. Accurate medium-range global weather forecasting with 3d neural networks. Nature, 2023.
Cristian Bodnar, Wessel P. Bruinsma, Ana Lucic, Megan Stanley, Anna Allen, Johannes Brandstetter,
Patrick Garvan, Maik Riechert, Jonathan A. Weyn, Haiyu Dong, Jayesh K. Gupta, Kit. Thambiratnam.
12

---

Published in Transactions on Machine Learning Research (03/2026)
Alexander T. Archibald, Chun-Chich Wu, Elizabeth Heider, Max Welling, Richard E. Turner, and Paris Perdikaris. A foundation model for the earth system. Nalure, 2025.
Cristian Challu, Kin G Olivares, Boris Oreshkin, Federico Garza Ramirez, Max Mergenthaler Canseco, and Artur Dubrawski. Nhits: Neural hierarchical interpolation for time series forecasting. In Association for the Advancement of Artificial Intelligence (AAAT), 2023.
Xiaodan Chen, Xiucheng Li, Xinyang Chen, and Zhijun Li. Structured matrix basis for multivariate time series forecasting with interpretable dynamics. In Advances in Neural Information Processing Systems (NeurIPS): 2024.
Andrea Cini, Tvan Marisca, Filippo Maria Bianchi, and Cesare Alippi. Scalable spatiotemporal graph neural networks. In Association for the Advancement of Artificial Intelligence (AAAI), 2023.
Rabvan-Gabriel Cirstea, Bin Yang, Chenjuan Guo, Tung Kieu, and Shirui Pan. Towards spatio-temporal aware traffic time series forecasting. In IEEE International Conference on Data Engineering (ICDE), 2022.
Vasudev Dehalwar, Akhtar Kalam, Mohan Lal Kolhe, and Aladin Zayegh. Electricity load forecasting for urban area using weather forecast information. In TEFE International Conference on Power and Renewable Energy (ICPRE), 2016.
Yuchen Fang, Yuxuan Liang, Bo Hui, Zezhi Shao, Liwei Deng, Xu Liu, Xinke Jiang, and Kai Zheng. Efficient large-scale traffic forecasting with transformers: A spatial data management perspective. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (SIGKDD), 2025.
Ismail Gultepe, R Sharman, Paul D Williams, Binbin Zhou, G Ellrod, P Minnis, S Trier, S Griffin, Seong S Yum, B Gharabaghi, et al. A review of high impact weather for aviation meteorology. Pure and applied geophysics, 2019.
Shengnan Guo, Youfang Lin, Huaiyu Wan, Xiucheng Li, and Gao Cong. Learning dynamics and hetero- geneity of spatial-temporal graph data for traffic forecasting, 1EEE Transactions on Knowledge and Dala Engineering, 2022.
Jindong Han, Weijia Zhang, Hao Liu, Tao Tao, Naiqiang Tan, and Hui Xiong. Bigst: Linear complexity spatio-temporal graph neural network for traffic forecasting on large-scale road networks. In International Conference on Very Large Data Bases (VLDB), 2024a.
Lu Han, Xu- Yang Chen, Han-Jia Ye, and De-Chuan Zhan. Softs: Efficient multivariate time series forecasting with series-core fusion. In Advances in Neural Information Processing Systems (NeurIPS), 2024b.
Tao Han, Song Guo, Zhenghao Chen, Wanghan Xu, and Lei Bai. How far are today's time-series models from real-world weather forecasting applications? are preprint arXiv:2406. 14399, 2024c.
Hans Hersbach, Bill Bell, Paul Berrisford, Shoji Hirahara, Andrés Horányi, Joaquín Muñoz-Sabater, Julien Nicolas, Carole Peubey, Raluca Radu, Dinand Schepers, et al. The era5 global reanalysis. Quarterly journal of the royal meteorological society, 2020.
Kethmi Hirushini Hettige, Jiahao Ji, Shili Xiang, Cheng Long, Gao Cong, and Jingyuan Wang. Airphynet: Harnessing physics-guider guided neural networks for air quality prediction. In International Conference on Learning Represendations (ICLR), 2024.
Pradeep Hewage, Ardhendu Behera, Marcello Trovati, Ella Pereira, Morteza Ghahremani, Francesco Palmieri, and Yonghuai Liu. Temporal convolutional neural (ten) network for an effective weather fore- casting using time-scries data from the local weather station. Soft Computing, 2020.
Jiawei Jiang, Chengkai Han, Wayne Xin Zhao, and Jingyuan Wang. Pdformer: Propagation dclay-aware dynamic long-range transformer for traffic flow prediction. In Association for the Advancement of Artificial Intelligence (AAAT), 2023.
13

---

Published in Transactions on Machine Learning Research (03/2026)
Yue Jiang, Xiucheng Li, Yile Chen, Shuai Liu, Weilong Kong, Antonis F Lentzakis, and Gao Cong. Sagdin: A scalable adaptive graph diffusion forecasting network for multivariate time series forecasting. In IEEE International Conference on Data Engineering (ICDE), 2024.
Zahra Karevan and Johan A. K. Suykens. Transductive LSTM for time-series prediction: An application to weather forecasting. Newral Nelworks, 2020.
George Karypis and Vipin Kumar. A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM Journal on Scientific Computing, 1998.
SM Klosko and CA Wagner. Spherical harmonic representation of the gravity field from dynamic satellite data. Planetary and Space Science, 1982.
Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long-and short-term temporal patterns with deep neural networks. In International ACM SIGIR conference on research in information retrieval (SIGIR), 2018.
Remi Lam, Alvaro Gonzalez, Matthew Willson, Peter Wirnsberger, Meire Fortunato, Ferran Alet, Suman Ravuri, Timo Ewalds, Zach Eaton-Roscn, Weihua Hu, et al. Learning skillful medium-range global weather forecasting. Science, 2023.
Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. arXiv preprint orXiv: 107. 00111077011100 019266 2017.
Shizhan Lin, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X. Liu, and Schahram Dustdar. Pyraformer: Low-complexity pyramidal attention for long-range time series modcling and forecasting. In International Conference Leurning Representations (ICLR), 2022.
Xu Liu, Yuxuan Liang, Chao Huang, Hengchang Hu, Yushi Cao, Bryan Hooi, and Roger Zimmermann. Rein- venting node-centric traffic forecasting for improved accuracy and efficiency. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases (ECML-PKDD), 2024a.
Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long, itransformer: Inverted transformers are effective for time series forecasting. In International Conference on Learning Representations (ICLR), 2024b.
Liheng Ma, Chen Lin, Derek Lim, Adriana Romero-Soriano, Puneet K. Dokania, Mark Coates, Philip H. S. Torr, and Ser-Nam Lim. Graph inductive biases in transformers without message passing. In International Conference on Machine Learning (ICML), 2023.
Tanwi Mallick, Prasanna Balaprakash, Eric Rask, and Jane Macfarlane. Graph- Graph--arriitionnnng-bess diffusion convolutional recurrent neural network for largo-scale traffic forecasting. Transportation Research Record, 2020.
Harvey J Miller. Tobler's first law and spatial analysis. Annals of the American Association of Geogruphers, 2004.
Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series worth 64 words: Long-term forecasting with transformers. In International Conference on Learning Representa- tions (ICLR), 2023.
Roland Pail, Sean Bruinsma, Federica Migliaccio, Christoph Förste, Helmut Goiginger, Wolf-Dieter Schuh, Eduard Höck, Mirko Reguzzoni, Jan Martin Brockmann, Oleg Abrikosov, et al. First goce gravity field models derived by three different approaches. Journal of Geodesy, 2011.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killcen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems (NeurIPS). 2019.
14

---

Published in Transactions on Machine Learning Research (03/2026)
Mare Rußwurm, Konstantin Klemmer, Esther Rolf, Robin Zhinden, and Devis Tuia. Geographic location encoding with spherical harmonies and sinusoidal representation networks. In International Conference on Learning Representations (ICLR), 2024.
Chao Shang, Jie Chen, and Jinbo Bi. Discrete graph structure learning for forecasting multiple time series. In International Conference on Learning Represenlations (ICLR), 2021.
Zezhi Shao, Zhao Zhang, Fei Wang, Wei Wei, and Yongjun Xu. Spatial-temporal identity: A simple yet effective baseline for multivariate time series forecasting. In ACM International Conference on Information & Knowledge Management (CTKM), 2022a.
Zezhi Shao, Zhao Zhang, Fei Wang, and Yongjun Xu. Pre-training enhanced spatial-temporal graph neural network for multivariate time series forecasting, In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (SIGKDD), 2022b.
Erwan Thébault, Gauthier Hulot, Benoit Langlais, and Pierre Vigneron. A spherical harmonic model of carth's 's lithospheric magnetic field up to degree 1050. Geophysical Research Letters, 2021.
Jindong Tian, Yuxuan Liang, Ronghui Xu, Peng Chen, Chenjuan Guo, Aoying Zhou, Lujia Pan, Zhongwen Rao, and Bin Yang. Air quality prediction with physics-guided dual neural odes in open systems. In International Conference on Learning Representations (TCL.R, 2025.
Waldo Tobler. On the first law of geography: A reply. Annals of the American Association of Geogrophers 2004.
Kingsley Eghonghon Ukhurebor, Charles Oluwaseun Adetunji, Olaniyan T Olugbemi, W Nwankwo, Aki- nola Samson Olayinka, C Umezurukee, and Daniel Ingo Heff. Precision agricullure: Weather forecasting for future farming. In Ai, edge and iot-based smart agriculture. Elsevier, 2022.
A Vaswani. Attention all you need. In Advances in Neural Information Processing Systems (NewrIPS), 2017.
Binwu Wang, Pengkun Wang, Zhengyang Zhou, Zhe Zhao, Wei Xu, and Yang Wang. Make bricks with little straw: Large-scale spatio-temporal graph learning with restricted gpu-memory capacity. In International Joint Conference on Artificial Intelligence (IJCAT), 2024a.
Shiyu Wang, Haixu Wu, Xiaoming Shi, Tengge Hu, Huakun Luo, Lintao Ma, James Y Zhang, and Jun Zhou. Timemixer: Decomposable multiscale mixing for time scries forecasting. In International Conference on Learning Represendations (ICLR), 2024b.
Xue Wang, Tian Zhou, Qingsong Wen, Jinyang Gao, Bolin Ding, and Rong Jin. Card: Channel aligned robust blend transformer for time series forecasting. In International Conference Learning Representations (ICLR), 2024c.
Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. In Advonces in Neural Information Processing Systems (NeurIPS), 2021.
Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modcling for general time scries analysis. In International Conference Learning Represen- tations (ICLR), 2022.
Haixu Wu, Hang Zhou, Mingsheng Long, and Jianmin Wang. Interpretable weather forecasting for worldwide stations with a unified deep model. Nature Machine Intelligence, 2023.
Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chengqi Zhang. Connecting the dots: Multivariate time series forecasting with graph neural networks. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (SIGKDD), 2020.
15

---

Published in Transactions on Machine Learning Research (03/2026)
Chin-Chia Michael Yeh, Yujie Fan, Xin Dai, Uday Singh Saini, Vivian Lai, Prince Osci Aboagye, Jumpeng Wang, Huiyuan Chen, Yan Zheng, Zhonglang Zhuang, et al. Rpmixer: Shaking up time series forecasting with random projections for large spatial-temporal data. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining (SIGKDD), 2024.
Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di Hc, Yanming Shen, and Tic- Yan Liu. Do transformers really perform badly for graph representation? In Advances in Newral Information Processing Systems (NeurIPS), 2021.
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Association for the Advancement of Artificial Intelligence (AAAT), 2023.
Ling Zhao, Yujiao Song, Chao Zhang, Yu Liu, Pu Wang, Tao Lin, Min Deng, and Haifeng Li. T-gen: tem- poral graph convolutional network for traffic prediction. IEEE Transactions on Intelligent Transportation Systems, 2019.
Zheng Zhao, Weihai Chen, Xingming Wu, Peter CY Chen, and Jingmeng Liu. Lstm network: a deep learning approach for short-term traffic forecast. IET Intelligent Transport Systems, 2017.
Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang, Informer: Beyond efficient transformer for long sequence time-series forecasting. In Association for the Advancement of Artificial Intelligence (AAAI), 2021.
Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting, In International Conference Machine Learning (ICML), 2022.
16

---

Published in Transactions on Machine Learning Research (03/2026)
A Scalable Spatiotemporal Series Forecasting
Several rescarchers have developed scalable spatiotemporal forccasting methods to accommodate larger datasets. Model-agnostic approaches use graph partitioning to decompose large spatial graphs into smaller subgraphs, with experiments conducted via independent training (Mallick et al. 2020) or continual learning (Wang et al. 2024a). In contrast, existing scalable models fall into four paradigms: precomputing spa- tial corrclation, linearizing spatial corrclation computation, mixing channels across spatial dimensions, and
simplifying nodes' spatial receplive fields. SGP (Cini et al.] 2023 and SimST (Liu et. al., 2024a) precom- pute graph convolutions and decouple spatial correlation modeling from training, but their fixed imput-space representations may reduce effectiveness. BigST (Han et al. linearized spatial convolutions to lower complexity, yet low-rank approximations prevent them from cap- turing structured spatial correlation. The channel mixing approach (Yeh et al. 2024 Han et al.) 2024b) enhances scalability by aggregating and routing messages across dimensions, avoiding quadratic complexity but suffering from information dilution during aggregation, leading suboptimal practical performance. Methods simplifying spatial receptive fields, such as SAGDEN (Jiang et al. (2224), use significant neighbor sampling to model spatial correlation but fail to preserve local structural information. Similarly, PatchSTG Taang et al., 2025 partitions traflic nodes into non-overlapping KDTree-based patches, using depth allen- tion for local correlation and breadth attention across same-index patches for global aggregation. However, lacking inherent integration of spatial prior knowledge (e.g., topological connections, geographic proximity), it overly relies on training patterns and struggles to capture critical geographic dependencies.
2024 adopt 2024a) and Sumba (Chen et al.
B Spherical Harmonics
Spherical harmonics have been widely used in Earth science (Klosko & Wagner 1982 Pail et al. 2011
Thébault et al.] 2021). Any function f(A, o) defined on a sphere can be expressed as weighted sum of orthogonal spherical harmonic basis functions Y with increasing frequency, characterizod by degrees and orders
1() = Σ
w/ ( (A,
(14)
1=0 m=-1
Here, w/"h are the weights. Each spherical harmonic Y defined as:
21 + (1- |ml)
(),
PMT (cos e ins
(15)
4x (1+ mm|)!
where (x) denotes the associated Legendre polynomials, given by:
dm
Pl" (x) (1 x²) dxm P((x)
(16)
These involve derivatives of Legendre polynomials (z),, which are defined as:
(((x) 2 dal (x - 1)'
(17)
For associated Legendre polynomials of nogative order (m < 0), the symmetry relation can be used.
P. (x) = (-1) (1- m)!
pm (x)
(18)
(1 m))!
In practice, we use the real form of Eq. 15
1-1)mh ml (cos X) sin(Imlp)
if
y (A, Φ)
Pmm (cos A),
if m = 0, if >
(19)
(01))//2Pp) (cos )) cos(mo),
17

---

Published in Transactions on Machine Learning Research (03/2026)
(a) WEATHER-SK
(b) NCEI Global
Figure 4: Global metcorological station distribution
where Pit(ccos X) denotes the normalized associated Legendre polynomial, defined as:
27 + (1 |ml)!
Pl" A)
V
PIr A)
(20)
4m (1+ mm])
C Structured Spatial Correlation
In spatiotemporal forecasting tasks, structured spatial correlation refers to learned dependency pattern that aligns with the inherent physical topology of the network rather than being collection of arbitrary, dense, or noisy associations. To provide precise characterization, we define spatial correlation matrix e E NNNN (where Øi represents the learned correlation strength between nodes is and vy) as structured if it satisfics the following two constraints:
Geographic Proximity Constraint: For any station Us the cumulative correlation strength of its geographic e-neighborhood Ne(v;) accounts for at least a threshold proportion т of its total correlation strength:
EnjeN, (w:) Oij
2T
(21)
ΣnSev Oi
Sparsity Constraint: The sparsity of θ is bounded by threshold p to eliminate trivial noise:
Sparsity(O) (e) ((i,)) O > 0}
p
(22)
N2
D Implementation Details
D.1 Datasets
All datasets used in our study are open-source or freely available for research purposes.
WEATHER-5K. The WEATHER-5K dataset, derived from global near-surface in-situ observations via the public Integrated Surface Database (ISD), includes data from 5,672 high-quality weather stations (2014-2023) covering variables like wind speed, direction, and temperature. It undergoes rigorous quality control (data interpretation, temporal alignment, completeness filtering, outlier detection), with remaining missing data interpolated using ERA5, followed by standardization and extreme value percentile calculation. Featuring uneven station distribution, it better aligns with actual observations than simulated ERA5 data, addressing limitations of small-scale existing time-series meteorological datasets for relevant research. The dataset. can accessed at https:://ithhu: www.taohannaaccccccccoooooaaaaaa 00w.aaaaaan000o/o.oo/ooaa 0200/ 0200/WEATHER WEATHER- 5K
NCEI Global. The NCEI Global dataset, sourced from the National Centers for Environmental Infor- mation, comprises hourly averaged wind speed and temperature records from 3,850 stations across the
18

---

Published in Transactions on Machine Learning Research (03/2026)
globe, covering the period from January 1, 2019, to December 31, 2020. It is divided into two subsets: 'global wind" and "global temp". The dataset is publicly accessible via the National Oceanic and Atmo- spheric Administration (NOAA) at httpp://www.nnw www.:ncei.nnn noaa. /wwp//ata/ggobalaooooooteeeeeeee lize the available processed versions in this study, which are available the Corrformer GitHub repository: https://github. wwwctoo//aaa/e tttm://oroo//ommrrrrrrr
We uti-
We visualize the station distributions of both datasets in Figure which shows that the stations effectively cover diverse weather patterns across varying geographical scales and station densities. Following previous research, chronologically split the WEATHER-5K dataset into training, validation, and test sets at ratio of 0.8, and the NCEI Global dataset at 0.7/0.1/0.2. Input data were normalized using the Z-score for model training.
D.2 Baselines
We compare the proposed approach with physics-based NWP models and the following advanced time series forecasting baselines:
Informer: It utilizes ProbSparse self-attention reduce computational complexity, enabling efficient long-sequence forccasting by focusing dominant temporal patterns.
Autoformer: It employs a decompesttion--teeniion architecture to model trend and seasonal compo- nents, leveraging an auto-correlation mechanism to capture long-range dependencies without explicit alignment.
Pyraformer: It uses hierarchical pyramid graph to capture multi-scale temporal dependencies with linear complexity, enabling efficient long-range forecasting.
STID: It combines spatial and temporal identity embeddings with multi-layer perceptrons address sample indistinguishability in spatiotemporal dimensions.
MTGNN: It builds an adaptive static global directed graph using learnable node embedding and aggregates information along spatial dimensions through mix-hop propagation.
Corrformer: It integrates multi-correlation mechanisms (spatial cross-correlation and temporal auto- correlation) in learnable tree structure to model complex spatiotemporal dependencies for large- scale global station weather forecasting.
iTransformer: embeds the whole time series into a spatial token and captures dynamic global spatial correlation using the self-attention mechanism.
RPMixer: It employs MLPs to model temporal dependency and integrates random projection layers to capture spatial correlation.
PatchSTG: It uses irregular spatial patching via KDTree and dual attention (depth and breadth) to capture local and global spatial correlations in large-scale spatiotemporal networks.
We obtain the code of baselines directly from their corresponding GitHub repositories. For the model- and relattd configurations, we follow the recommended settings provided in their code.
D.3 Hyperparameters Setting
To better reproduce our model, we summarize all the default hyperparamcters as follows. The dimensions of the input embedding and hidden embedding dimension D are set to 768. The number of blocks is set to 2. The initial number of subgraphs P is set based on hyperparameter tuning results. Specifically, 64 for the WATHER-5K and NCEI Global Wind dataset, and 16 for the NCEI Global Temp dataset. In the calculation of spherical harmonic basis functions, we the maximum order of Legendre polynomials
3. The epsilon is set by following the suggestion of DCRNN Li et al. (017).
19

---

Published in Transactions on Machine Learning Research (03/2026)
Table 5: Long term global station weather forecasting performance comparison. The best results are high- lighted in bold, while the second-best results are underlined.
Lead
Temperature
Dewpoint
Wind Rate
Wind Direc.
Sea Level
Baselines
Time
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
24
1.76
7.39
1.85
7.94
1.48
4.53
63.8
7158.3
0.86
2.68
72
1.87
8.01
1.94
8.48
1.52
4.76
72.4
8215.6
1.06
3.31
NWP Model
120
1.99
8.79
2.14
10.87
1.58
5.11
75.4
8647.7
1.38
5.15
168
2.15
10.06
2.32
12.56
1.66
5.59
78.3
8945.7
1.87
9.52
24
1.75
6.92
1.83
7.88
1.30
3.58
61.8
6930.2
1.90
9.72
72
2.47
13.03
2.67
13.39
1.52
4.97
72.0
8222.4
3.76
33.67
Pyraformer
120
2.77
16.04
3.00
18.95
1.59
5.37
75.1
8610.7
4.43
43.91
168
2.95
17.95
3.20
21.06
1.61
5.56
76.4
8773.5
1.77
49.97
24
1.47
4.99
1.52
5.67
1.17
3.09
55.5
6135.7
1.26
4.08
72
2.20
10.32
2.36
12.33
1.46
4.62
68.9
7934.7
27.54
Trannsformer
120
2.65
14.54
2.87
17.48
1.54
5.13
73.7
8601.2
4.42
44.56
168
2.87
16.88
3.12
20.20
1.59
5.42
75.7
8882.5
4.92
53.55
E More Performance Comparison
E.1
Long Term Forecasting Result
As shown in Table with the increase of forecasting lead time, the error of time series forecasting methods gradually increases: except for wind speed and wind direction, their performance on almost all variables is inferior to that of physics-based numerical weather prediction models. In contrast, NWP models produce more stable predictions partly because they are typically trained on a larger scale and more abundant data, allowing them to deliver robust global almospheric forecasts; partly because an increase in forecast duration amplifies the nonlinear dynamics of the atmospheric system, which pure data-driven methods struggle fully capture. In the future, intend to explore the emerging paradigm that integrates data-driven and physics-informed approaches to enhance the model's capability of predicting long-term weather processes.
E.2 Comparison with informed Models
To ensure a fair and comprehensive evaluation, we conduct additional experiments comparing $ Transformeer with two statc-of-the-art physics-informed basclines, AirPhyNet (Hettige et al. 2024) and Air-DualODE (Tian et al. 2025), on the WEATHER-5K dataset. These models specifically engineered to integrate physical constraints differential equations into neural architectures for environmental forecasting. As summarized in Table 0 the key observations are as follows:
Trannsforeer achieves the lowest MAE and MSE across all core meteorological variables compared to the two physics-infors informed baselines. Notably, physics-infore informedd models like Air-DulOODE rely on intricate neural ODE modules for enforcing physical constraints, leading higher memory usage and slower inference. In stark contrast, Trrrassformrr maintains lightweight, highly scalable architecture while outperforming these counterparts, underscoring its unique competitive edge.
20

---

Published in Transactions on Machine Learning Research (03/2026)
Table 6: Performance comparison with physics- models on the WEATHER-5K, dataset. The best rosults are highlighted in bold.
Variable
Temperature
Dewpoint
Wind Rate
Wind Direc.
Sea. Level
Metric
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
AirPhyNet
3.71
25.99
2.49
13.75
1.50
4.70
67.87
7660.30
2.48
14.62
Air-DuaIODE
3.20
19.35
2.12
10.27
1.49
4.56
67.26
7640.32
2.46
14.04
Trannfforerr
1.47
4.99
1.52
5.67
1.17
3.09
55.50
6135.69
1.26
4.08
Table 7: Sensitivity analysis of subgraph gramularity on the WEATHER-5K dataset. Performance metrics are reported for all nodes and boundary nodes (defined under the default 64-subgraph partition). BS and Mem denote the training batch size and maximum GPU memory usage (GB), respectively,
Subgraphs
All MAE
All MSE
Boundary MAF
Boundary MSE
BS
Mem
16
12.13
1224.86
12.22
1264.10
64
61.5
64
12.19
1230.70
12.28
1270.54
64
52.0
128
12.25
1240.42
12.42
1291.76
64
46.2
F
More Analysis of Graph Partitioning Strategy
To further investigate the robustness and parameter sensitivity of our graph partitioning strategy, this section provides an in-depth discussion and empirical analysis regarding the selection of subgraph granularity and the influence of partitioning stochasticity.
Impact the Number of Subgraphs. Our default configuration of 64 subgraphs is selected to bal- ance predictive performance with computational efficiency optimally. Utilizing fewer subgraphs (c.g., 16 or 32) ensures that most adjacent nodes reside within the same partition, allowing the model to capture comprchensive local correlation patterns and enhance accuracy. However, this approach incurs significant computational redundancy, excessively large subgraphs necessitate calculations across overly extended spatial scopes. Conversely, increasing the subgraph count (e.g., 96) reduces the nodes per partition but sharply increases the number of boundary stations. For these stations, spatial correlations with neighbors in different subgraphs are truncated, leading to elevated prediction errors.
Targeted experiments, as shown in the Table 7 further analyze the impact of subgraph granularity on efficiency and forecasting performance. As the number of subgraphs increases, MAE and MSE for both all nodes and boundary nodes exhibit clear upward trend, confirming that finer partitioning exacerbates spatial consistency loss. While 16 subgraphs yield the lowest MAE, they require 22% more memory. The 64-subgraph setting strikes a near-optimal balance, reducing memory consumption by 15.4% while sacrificing only 0.5% in all-node MAE compared to the 16-subgraph version.
Reducing the number of subgraphs to one equivalent to no partitioning does not represent a performance upper bound. In this configuration, the model reduces to global attention architocture that lacks geo- graphic inductive bias. This forces the model to capture all pairwise correlations, which introduces spurious noise from distant nodes and impairs generalization. Furthermore, the quadratic complexity of full-graph at- tention becomes computationally prohibitive. Thus, proper partitioning acts as necessary regularizer that suppresses spatial noise while ensuring scalability. While our current fixem-earameter approach is effective, exploring adaptive subgraph selection remains a promising direction for future work.
Sensitivity Partition Randomness. Since the METIS algorithm employs randomized soods for initial label assignment and perturbation optimization, we evaluate the model's sensitivity to these variations. We conducted experiments using three distinct random seeds under the same partitioning constraint (P 64)
21

---

Published in Transactions on Machine Learning Research (03/2026)
Table 8: Model Transfoorerr performance across different METIS partitioning seeds on the NCEI Global
dataset.
NCEI Global
Wind
Temp
Metric
MAE
MSE
MAE
MSE
1e 15
1.304
3.616
1.884
7.524
S2
1.307
3.631
1.880
7.516
Seed-2025
1.305
3.620
1.874
7.509
3,750
1.950
8.000
MAE
1.330
MSE
-0- MSE
7.800
MAE
1.320
MIEE
ASE
1.890
7.600
3.650
1.310
1.860
7.400
1.300
3.600
24
46
48
96
46
48
96
(a) Lookback Length on Wind
(b) Lookback Length on Temp
1.920
1.314
MAE MSE
-O-
MAE
7.735
3.660
1.900
7.650
MAE
3.645
MSS St
1.880
1.308
3.630
7.565
1.305
3.615
1.860
77480
32
32
64
(c) Subgraph Number on Wind
(d) Subgraph Number on Temp
Figure 5: More Parameter sensitivity analysis
on the NCEI Global dataset. As summarized in Table X the performance variations across different seeds are negligible. This consistency confirms that the Trannsforrer highly robust the stochastic nature of the graph partitioning process.
G More Parameters Sensitivity Analysis
We evaluate the impact of lookback window length T and initial subgraph number /' on model performance on the NCEI Global dataset. As shown in Figure 5 increasing T improves performance for both datasets. However, longer input sequences cause rapid surges in computational and memory costs for spatiotemporal GNNs, leading most existing models to rely on short-term historical windows, severely limiting their perfor- mance (Han al. 2024a). In contrast, our model achieves spatial scalability while accommodating larger lookback windows to boost performance. Next, (ranging from 16, 32, 64 to 96). A larger P means fewer nodes per subgraph, enabling of local spatial correlations; conversely, smaller P increases nodes per subgraph, expanding the local spatial scope but raising memory and computational costs. Experimental results show that despite the same number of stations, the performance trends and optimal P values vary across datasets with different variables. This is attributed to complex interactions between local and global influences, indicating limitations treating P merely as a hyperparameter without adjusting to balance local and global effects. Exploring adaptive selection of P will be part of our future work.
evaluate the impact of the initial subgraph number P
precise modeling
22

---

Published in Transactions on Machine Learning Research (03/2026)
0.6
0.6
0.4
0.4
Intra subgraph Attention of Block
Intra subgraph Attention of Block
0.8
0.8
0.6
0.6
-0.0
00.0
Inter-subgraph Allention of Block
Inter-subgraph Attention of Block
Figure 6: Visualization of intra-subgraph and inter-subgraph attention matrices for Transsformer
H More Visualization
Attention matrix. We further visualize the intra-subgraph and inter-subgraph attention matrices of S Transformerr. As shown in Figure 6 the two-layer attention mechanism effectively scparates local and global information (i.e., structured spatial correlation modeling): in local modeling, the model filters out global noise (the attention matrix has non-zero values only locally, showing a sparse high-rank pattern); while global modeling is achieved through subgraph aggregation and learning attention between subgraphs (the attention matrix exhibits high weights and a sparse low-rank pattern). We propose that this low-rank pattern be interpreted the discovery of key hubs in large-scale spatiotemporal networks.
Global Station Forecasting result. As shown in Figure 7 we plot the prediction errors of different models for the temperature variable in the WEATHER-5K Dataset from a global perspective (brighter colors indicate larger prediction errors at the corresponding stations). can be observed that all models show an upward trend prediction errors over time; among them, S*Transformer achieves higher accuracy in temperature prediction for high-latitude stations, while baseline models exhibit significantly larger prediction errors these stations. This advantage originates from two designs the proposed model: first, spherical harmonic positional encoding endows it with clear definition worldwide (including polar regions), enabling better distinction of meteorological stations distributed globally; second, the attention mechanism with progressively expanded receptive fields conforms to physical diffusion laws.
Single Station Forccasting result. As shown in Figure 8 and Figure g we plot the predictions of different models for the temperature variable the NCEI Global Dataset from a single-station perspective. It can be observed that. Tr outperforms other baseline models the modeling of seasonal, peak, and stationary sequence values. This is attributed the model's ability effectively capture local spatial correlations and leverage changes in neighboring nodes to achieve more accurate forecasting.
23

---

Published in Transactions on Machine Learning Research (03/2026)
tours
hours
iTTranffrmmer
(b) Traasfoorerrr
Figure Comparison global station forecasting errors.
24

---

Published in Transactions on Machine Learning Research (03/2026)
Soundhahh
Groandhich
40
(a) STID
(b) RPMixer
Goundh Jh Prdddiin
Sronddruhh Pedddtion
Time (hour)
Time (Hour)
(c) PatchSTG
(d) Trannsformer
Figure Comparison of single station forecasting results (seasonal sequences).
Grandd 2th
Sevandiiuhh
Tumoe-atrrs
(a) STID
(b) RPMixer
Geoundrruth Prodittion
Priddtion
12
Tunpe 200r
112
Time Blour
Time (Hour)
(c) PatchSTG
(d) Trransoormer
Figure 9: Comparison of single station lorecasting results (peak sequences).
25


Reviews:
[{"review_id": "byw5Vse2iP", "reviewer": "Reviewer_U1EA", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes $S^2$Transformer, a scalable spatiotemporal forecasting model for global station weather prediction. Its key contribution is the introduction of a Spatial Structured Attention Block, which partitions the global spatial graph into subgraphs to jointly model local (intra-subgraph) and global (inter-subgraph) correlations through attention mechanisms. This design reduces noise and quadratic computational cost, enabling efficient long-range spatial dependency modeling. Experiments on medium- and large-scale weather station datasets show notable performance and efficiency gains over existing baselines.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The paper presents quantitative results on two public datasets, demonstrating that $S^2$Transformer achieves significant advantages in MAE/MSE metrics, memory consumption, and inference speed. It also conducts ablation studies to validate the effectiveness of key components such as subgraph partitioning and spatial attention bias, with results consistent with the model’s design objectives.", "audience_yn": "Yes", "audience_explanation": "Yes. The paper’s findings would interest part of TMLR’s audience, especially researchers in spatiotemporal forecasting, graph-based transformers, and AI for earth system modeling. While not groundbreaking, its structured attention design and scalable architecture address a relevant and timely problem in large-scale weather prediction.", "requested_changes": "1. At line 318, the model’s core mechanism (Intra- and Inter-subgraph Attention) relies entirely on the quality of the subgraph partitioning. However, the authors only compare two variants (removing METIS and using random partitioning), without discussing the impact of partitioning strategy, robustness, or potential bias.\n2. Table 3 includes only general spatiotemporal baselines such as MTGNN, iTransformer, and Corrformer, but omits hybrid physics-driven or physics-informed models, which are essential for a fair and comprehensive comparison.\n3. The paper does not provide a formal mathematical definition of “structured spatial correlation.” I recommend that the authors clarify this concept, either by including a precise formulation in the appendix or by adding comparative experiments to better illustrate what “structured” represents in this context.", "broader_impact_concerns": "No major ethical concerns identified. The model focuses on weather forecasting and does not involve sensitive or private data. A short Broader Impact Statement discussing responsible deployment in climate and disaster prediction would be sufficient.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "tl4NDv2rLf", "reviewer": "Reviewer_mVgX", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a novel transformer-based framework, S²Transformer, for global station weather forecasting. The approach introduces a graph partition mechanism and a multi-scale spatial structure, enabling the model to capture long-range global correlations while preserving scalability. Experimental results demonstrate both the effectiveness and computational efficiency of the proposed method.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The authors make three primary claims:\n• Global correlations are essential for global station weather forecasting (GSWF) tasks;\n• The proposed S²Transformer is more scalable to large graphs compared to existing transformer-based architectures; and\n• S²Transformer achieves performance that is either superior to or on par with other state-of-the-art GSWF methods.\n\nThese claims are well-supported by the presented experimental results, which include comprehensive evaluations demonstrating both scalability and predictive accuracy across multiple datasets.", "audience_yn": "Yes", "audience_explanation": "The paper uses a new transformers framework to deal with GSFW, which could be considered as the application of graph representation learning.", "requested_changes": "• The authors should provide a more detailed discussion regarding the choice of the number of subgraphs in the initial partition. Currently, only the changes in MAE and MSE for two tasks are reported under different subgraph counts. It would be beneficial to include further analysis on how varying the number of subgraphs affects model performance and spatial consistency, particularly for boundary stations located between subgraphs.\n\n• Since both the authors’ claims and experimental results emphasize the importance of the graph partition strategy to overall performance, it would strengthen the paper to include additional analysis on the sensitivity of the model to partition randomness. For example, do different random seeds used by METIS (under the same partitioning constraints) lead to noticeable performance variations?\n\n• As GSWF can be formulated as a node-level forecasting task, and the main contribution of S²Transformer lies in its scalability, it would be informative to compare it against existing linear transformer architectures, such as NodeFormer [1], which can handle graphs with over 100K nodes. Including such comparisons would help better position S²Transformer within the landscape of efficient transformer-based models.\n\nReference:\n[1] Wu Q, Zhao W, Li Z, Wipf DP, Yan J. Nodeformer: A scalable graph structure learning transformer for node classification. Advances in Neural Information Processing Systems. 2022 Dec 6;35:27387-401.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "LBxXGj4ir6", "reviewer": "Reviewer_ztAG", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper modifies the Transformer architecture to improve global station weather forecasting, taking into account spatial relationships between different stations. The authors first construct a spatial graph where two stations are connected if they are close enough. After partitioning the graph, they introduce a spatial structured attention (SSA) block, which first applies attention locally (intra-subgraph), then globally (inter-subgraph). The authors further adapt the positional encodings and add a spatial attention bias. They also propose a multiscale architecture across layers, moving from finer-grained to coarser subgraphs. The proposed approach is evaluated on 2 short-term weather forecasting datasets (WEATHER-5K and NCEI Global), often outperforming existing baselines.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Mostly yes. The proposed approach is compared to a variety of reasonable baselines and often outperforms them. One limitation of the evaluation is that the time frame is fixed (48-hour history, 24-hour prediction).\n\nThe authors include a complexity analysis of their method, as well as memory usage and training time statistics.\n\nThe authors conduct an ablation study for most of the components of their proposed architecture, although additional evidence to demonstrate the impact of using different scales across layers (section 4.3) would be welcome.", "audience_yn": "Yes", "audience_explanation": "The paper proposes changes to the Transformer architecture to better capture spatial relationships, which should be interesting to some of the TMLR audience.", "requested_changes": "**Critical**\n\n- Add an ablation study for graph partition size and the impact of using different scales across layers.\n\n**Would strengthen work**\n\n- Could the approach be extended to take into account microclimates, where nearby stations might have fairly different weather?\n\n- On line 318/319, is the positional encoding completely removed or just simplified?\n\n- In addition to average results, can you include the standard deviation?\n\n- On line 271/272, did you try the opposite approach (single vs joint predictions) on the 2 datasets?\n\n- On line 336, you mention that increasing the number of blocks leads to worse results. Could this be mitigated with better regularization?\n\n- The input and output linear transformations appear to be potential bottlenecks to handle longer sequences. Could you discuss how you could extend the approach to better support longer timeframes?", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)