FSTLLM: Spatio-Temporal LLM for Few Shot Time Series Forecasting
Authors: YUE JIANG
Yile Chen
Xiucheng Li
Qin Chao
SHUAI LIU
Gao Cong
Venue: ICML 2025 poster (2025)
Keywords: Few shot
time series forecasting
multivariate time series
Large Language Models
STGNN
Primary area: applications->time_series
TL;DR: We propose a framework named Few-shot Spatio-Temporal Large Language Models (FSTLLM), aimed at enhancing model robustness and predictive performance in scenarios with limited training data.

Abstract: Time series forecasting fundamentally relies on accurately modeling complex interdependencies and shared patterns within time series data. Recent advancements, such as Spatio-Temporal Graph Neural Networks (STGNNs) and Time Series Foundation Models (TSFMs), have demonstrated promising results by effectively capturing intricate spatial and temporal dependencies across diverse real-world datasets. However, these models typically require large volumes of training data and often struggle in data-scarce scenarios. To address this limitation, we propose a framework named Few-shot Spatio-Temporal Large Language Models (FSTLLM), aimed at enhancing model robustness and predictive performance in few-shot settings. FSTLLM leverages the contextual knowledge embedded in Large Language Models (LLMs) to provide reasonable and accurate predictions. In addition, it supports the seamless integration of existing  forecasting models to further boost their predicative capabilities. Experimental results on real-world datasets demonstrate the adaptability and consistently superior performance of FSTLLM over major baseline models by a significant margin. Our code is available at: https://github.com/JIANGYUE61610306/FSTLLM.

FSTLLM: Spatio-Tempora LLM for Few Shot Time Series Forecasting
Yue Jiang 12
Yile Chen
Xiucheng Li
Qin Chao
Shuai Liu' Gao Cong'
Abstract
namic evolution of data over time, and spatial correlation (or channel correlation), which describes the interdependencies between different time series. These correlations typically emerge from mutual influences between time series, such as those driven by physical proximity (Li et al., 2017) or functional similarity (Bai et al., 2020; Wu et al., 2020; Chen et al., 2024). In practice, spatial correlations in multivariate series are not explicit, and effectively modeling such latent correlations is critical for more accurate predictions (Shao et al., 2022b; Zheng et al., 2020; Han et al., 2021). Tradi- tional approaches. including statistical models like ARIMA (Williams & Hoel, 2003) and VAR (Zivot & Wang, 2006), as well as machine learning methods such as SVR (Drucker et al., 1996) and Gaussian Processes (Xie et al., 2010), fo- cus on modeling short-term temporal dynamies. Recent studies, particularly Transformer-based frameworks (Zhou et al., 2021; Wu et al., 2021; Zhang & Yan, 2023; Woo et al., 2022; Cirstea et al., 2022), have significantly enhanced fore- casting performance by effectively modeling the long-term temporal dependencies. Besides, emerging approaches for Time Series Foundation Models (TSFMs), such as GPT4TS (Zhou et al., 2023) and Time-LLM (Jin et al., 2024), ex- plore the adaptation of Large Language Models (LLMs) to time series forecasting through partial fine-tuning strate- gies. In parallel, Spatio-TTempone Graph Neural Networks (STGNNs) have been developed for scenarios where each channel of a multivariate time series is associated with spa- tial attributes that exhibit mutual influence (Guo et al., 2019; 2022; Shao al., 2022a; Yu et al., 2017: Song et al., 2020; Shao al., 2022b; Shang al., 2021; Miao et al., 2024b; Wang al., 2022). These models are designed to jointly model spatial and temporal dependencies, enabling more holistic modeling for correlated time series data.
Time series forecasting fundamentally relies on accurately modeling complex interdependencies and shared patterns within time series data. Re- cent advancements, such as Ppatio-Temporal Graph Neural Networks (STGNNs) and Time Se- ries Foundation Models (TSFMs), have demon- strated promising results by effectively captur- ing intricate spatial and temporal dependencies across diverse real-world datasets. However, these models typically require large volumes of train- ing data and often struggle in data-scarce sce- narios. To address this limitation, we propose framework named Few-shot Opatio-Temporae Large Language Models (FSTLLLM), aimed at enhancing model robustness and predictive per- formance in few-shot settings. FSTLLM lever- ages the contextual knowledge embedded Large Language Models (LL.Ms) to provide reasonable and accurate predictions. In addition, it sup- ports the seamless integration of existing fore- casting models to further boost their predicative capabilities. Experimental results on real-world datasets demonstrate the adaptability and consis- tently superior performance of FSTLLM over ma- jor baseline models by a significant margin. Our code is available at: https: /ggttgh.t..uu... thub. com/ DIANGYUE616100066/55111
1. Introduction
Multivariate time series forecasting methodologies are sential for assisting human experts in decision-making re- source allocation, and scheduling across various industries. Accurate forecasting requires precise modeling on two di- mensions: temporal dependency, which captures the dy-
Despite these advancements, several limitations persist. First, these models typically require substantial amounts of training data to effectively learn complex spatio-temporal correlations inherent in time series. However, collecting such large-scale data can be time-consuming and resource- intensive in practice, potentially requiring months to accu- mulate sufficient quantities for training. This constraint poses a challenge in many real-world scenarios, where timely and sufficient data may not be readily available, thereby limiting the effectiveness of these data-hungry mod- els. Second, both STGNNs and TSFMs normalize time
'College of Computing and Data Science, Nanyang Techno- logical University, Singapore DAMMO Academy. Alibaba group, Singapore "School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen), China. Correspondence to: Yile Chen <yilc001 @ c.ntu.ceu..gg>.
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

FSTLLM: Spatio-Temporal LLM for Few Shot Time Series Forecasting
series data into numerical vectors and update model pa- rameters without integrating the contextual knowledge of real-world factors, such as geographical information, urban contexts, and human behavioral patterns. Solely relying on numerical inputs limits the models' ability to capture these external factors, which are often crucial for improving fore- casting accuracy across diverse domains. contrast, LLMs demonstrate strong capabiliities in common sense reason- ing (Zhao et al., 2023), making them particularly effective for integrating domain-specific and contextual knowledge via in-context learning (Min et al., 2022) and fine-tuning. Furthermore, LLMs exhibit robust performance in few-shot and zero-shot learning settings (Brown et al., 2020; Kojima et al., 2022), which are highly advantageous data-scarce forecasting scenarios. These two characteristics motivate the exploration of LLMs for few-shot time series forecast- ing. While some recent LLM-based time series forecasting methods (Jin et al., 2024; Zhou et al., 2023; Chang et al., 2023; Pan et al., 2024) have made initial strides toward this direction, they reply on fine-tuning LLMs using purely nu- merical inputs (Zhou et al., 2023; Chang et al., 2023; Liu et al., 2024a), or concatenate general task-specific instruc- tions as prompts (Jin al., 2024). However, these strategies fall short of fully leveraging the contextual understanding of real-world factors inherent in different time series data. Additionally, some studies have reported that replacing the LLM component from these methods with a sample layer does not degrade performance (Tan et al., 2024).
and context-aware prompt design, allowing the model to incorporate reasoning about spatial and temporal contexts and the initial predictions from STGNN backbones for few- shot settings. Furthermore, FSTLLM offers flexible and extensible solution that allows existing state-of-the-art meth- ods to be seamlessly integrated, resulting in performance improvements over their original implementations.
Our contributions are summarized as follows:
LLM Enhanced Graph Construction: We propose a LLMM-Enancce Graph Construction module that uti- lizes LLMS to enhance graph construction by embed- ding contextual information of spatial nodes for time serie channels.
Domain Knowledge Injection: We design Domain Knowledge Injection module by fine-tuning an LLM to integrate domain-relevant knowledge with carefully crafted prompts. This design enables the model to account for human-like considerations of spatial and temporal characteristics, thereby delivering effective performance in forecasting.
Few-Shot Learning Integration: We demonstrate that FSTLLM can augment various time series forecasting models, enhancing their performance in few-shot set- tings without updating their parameters.
Comprehensive Evaluation: Experiments validate that FSTLLM outperforms baseline models on two real-world datasets under a limited data scenario by a large margin.
To address the limitations of existing approaches in data- scarce settings, we propose Few-shot Spatio-Temporal Learning with Large Language Models (FSTLLM), novel and flexible framework designed to enhance forecasting per- formance by integrating diverse STGNN backbones with the contextual reasoning capabilities of L.Mss. FSTLLMM leverages the rich contextual knowledge embedded in LLMs to improve the modeling of both spatial and temporal de- pendencies, thus substantially reducing reliance on large volumes of training data. Specifically, it treats each time series channel as node and employs an LLM-EEnanced Graph Construction Module that generates node-specific contextual embeddings from textual contents. These embed- dings are then used to construct an adjacency matrix that captures semantically meaningful spatial correlations, which are subsequently used as input to an STGNN backbone for spatio-temporal modeling. To further refine the predictions produced by the STGNN, FSTLLM incorporates Domain Knowledge Injection Module that fine-tunes the LLM us- ing carefully designed prompts. These prompts integrate comain-speciffe knowledge, node-level spatio-temporal con- text, and predietive cues derived from the STGNN backbone, enabling the model reason over both structured and con- textual information. Compared to prior LLM--bseed fore- casting methods, FSTLLM introduces a more expressive
2. Related Work
2.1. Classical Neural Network-based Methods
Neural network-based approaches have been extensively applied to tackle time series forecasting tasks. Among them, LSTM (Hochreiter & Schmidhuber, 1997) and GRU (Chung et al., 2014) are two notable variants of Recurrent Neural Networks (RNNs) that are commonly used in time series forecasting (Bai et al., 2020; Shang et al., 2021; Shi et al., 2015). Recently, Transformer- based models (Zhou et al.,
2021; Cirstea et al., 2022; Wu et al.. 2021; Liu et al., 2022; Zhou et al., 2022; Zhang & Yan, 2023; Miao et al., 2024a; 2025) have shown demonstrated considerable promise in long-term time series forecasting, largely due to their capa- bility to model long-range and pairwise temporal dependen- cies. For instance, Informer (Zhou et al., 2021) introduces sparse self-attention mechanism, optimizing computational efficiency by utilizing Kullback-Leibler divergence for at- tention sparsity estimation. PatchTST (Nie et al., 2023) divides input sequences into patches as basic units, which are processed through a Transformer backbone to enhance
2

---

FSTLLM: Spatio-Temporal LLM for Few Shot Time Series Forecasting
forecasting accuracy. In parallel, linear models such as DLinear (Liu al., 2024c) have shown competitive perfor- mance by employing decomposition strategy to separately model trend and residual components, offering lightweight alternative to attention-based architectures. Despite these advances, both Transformer-basee and linear models often neglect the spatial dependencies inherent in multivariate time series, leading to limited effectiveness in scenarios where spatial correlations among channels play critical role.
tasks with only numerical temporal inputs, showing mod- erate performance across range of time series analytical tasks. LLM4TS (Chang et al., 2023) introduces two-stage fine-tuning approach for time series forecasting: the first stage involves task-specific pre-training to align the LLM with the structural characteristics of time series data, fol- lowed by a fine-tuning stage that leverages the model's fore- casting capabilities. Time-LLM (Jin al., 2024) presents approach that reprograms numerical sequences of time series into contextual embedding, and employs the Prompt- as- As-Prefix method to generate predictions with L.L back- bone. AutoTimes (Liu et al., 2024c) repurposes decoder- only LLMs for autoregressive time series forecasting by mapping time series inputs into the embedding space of language tokens. It enables the derivation of variable-length future predictions without updating the LLM weights. Time- MoE (Shi et al., 2025) introduces scalable architecture based on a sparse mixture-of-expert paradigm, trained on the expansive datasets. With large-scale parameters, Time- MoE achieves strong predictive performance but exhibits diminished effectiveness in few-shot scenarios due to lim- itations in its expert routing mechanism. News2Forecast (Wang al., 2024) enhances the performance by integrat- agents using
2.2. Spatio-Temport Graph Neural Networks
Opatio-Temport Graph Neural Networks (STGNNs) have become as a fundamental framework for modeling spatio- temporal correlations in multivariate time series, particularly when the time series data exhibits explicit spatial structure. These models typically employ GNNs to capture spatial correlations via message passing across different time series channels (nodes), while temporal dependencies are mod- eled using established sequential models (e.g., RNNs and Transformers). By jointly learning spatial and temporal correlations, STGNNs have achieved strong performance across forecasting benchmarks in spatio-temporal data, such as in traffic flow and crime prediction. Early STGNN mod- els such as DCRNN (Li et., al., 2017) and T-GCN (Zhao et al., 2020) combine GRUs for temporal modeling with GNNs that operate on predefined graphs derived from prior knowl- edge, such as physical or geographical proximity. However, the use of static graphs can result in suboptimal performance when these predefined relationships fail to accurately reflect the true underlying spatial relationships among nodes. To address this limitation, recent models, such as AGCRN (Bai et al., 2020) and MTGNN (Wu et al., 2020), have incorpo- rated the adaptive graph learning module into the STGNN framework. These models learn node embeddings, and dynamiclly compute the inner product of embeddings to construct adjacency matrices that represents spatial corre- lations for dependency modeling. Further advancements, including GTS (Shang et al., 2021) and STEP (Shao et al., 2022a), incorporate explicit graph learning mechanisms to capture complex, non-linear spatial dependencies using dedicated graph construction components. However, the performance of these models often come with increased data requirements and computational complexity, making them less effective in scenarios with limited training data.
ing social event signals through reflection and reasoning. It fine-tunes a pre-trained LLM to align textual and numerical data, thereby improving fore- casting accuracy. TimeKI (Liu et al., 2025) combines cal- ibrated LLMs with privileged knowledge distillation, uti- lizing a cross-modality teacher trained on both historical inputs and future ground-truth prompts, and a lightweight student model that is accessible only historical inputs.
Despite their promising contributions, these methods dis- regard the spatial correlations among the channels of time series, which are critical for accurate multivariate forecast- ing in veal-world spatio-temporal contexts. Besides, the in- ternal reasoning capabilities of LLMs remain underutilized. Most existing approaches either fine-tune the LLM back- bone directly on normalized temporal inputs or augment these inputs with basic task-specific prompts, thus signif- icantly restricting both the reasoning capacity and overall predictive performance of the models.
3. Method
FSTLLM is an LLM-enhanced framework designed for few-shot time series forecasting. It advances forecasting performance across both spatial and temporal dimensions by leveraging the rich real-world knowledge encoded in LLMs. FSTLLM consists of three key components: LLM- Enhanced Graph Construction module, STGNN backbone module, and Domain Knowledge Injection module. The overall framework of FSTLLM is illustrated in Figure 1.
2.3. Large Language Models
Recent advancements (Jin et al., 2024; Zhou et al., 2023;
Chang et al., 2023; Pan et al., 2024; Liu et al., 2024a;b) in large language models (LLMMs) have demonstrated promis- ing potential in extending their applications to time series analysis, potentially in few-shot time series forecasting.
GPT4TS (Zhou et al., 2023) adapts GPT2 for time series
First, Enhancedd Graph Construction module leverages
3

---

FSTLLM: Spatio-Temporal LLM for Few Shot Time Series Forecasting
Adjacancy
Reshape
G-Entmax Layer
(N.NN)
dn
dout
Numerical Prediction Tokens C
Graph Attention Layer
Recurrent Graph Convolution, A
Convelution, A
H
Mass
Input Prompt
Insert
E (N,d)
Fine fune
Mi-kez Recurrent Graph Convolution,
HcoY-2 Recurrent Convolution, A
P-h Tokenizer +
Projection Layer
Mg-tes Recurrent Graph Convolution, A
Hewr-1
Recurrent Graph Convol ution, A
Task Instruction
Historical Input
Input Time
Output Time
Node Description
Node Pattern
Encoder
Decoder
Prediction
Future Token
Figure 1. The overall architecture the proposed FSTILMM framework. The L.LM Enhanced Graph Construction Block on the left, the STGNN module is in the middle, and the Domain Knowledge Injection module is the right.
the contextual information encoded by LLMs, such as entity descriptions and user reviews, to construct an adjacency matrix A representing the spatial correlations. By incorpo- rating semantically rich knowledge inferred by the I.LM, the resulting adjacency matrix captures meaningful relation- ships between nodes that go beyond purely data-driven prox- imity. This context-aware graph construction is particularly advantageous in data-scarce scenarios, where conventional STGNNs may produce unstable or unreliable node embed- dings due to insufficient training signals. Next, the generated adjacency matrix is fed into a STGNN model, which per- forms joint spatio-temporal modeling and outputs numerical prediction tokens. While this module is compatible with range of existing STGNN architectures, its predictive accu- racy may remain less effective with limited data availability in few-shot settings. To enhance the performance, relying on the reasoning capabilities of LLMs, Domain Knowledge Injection module is proposed to fine-tune an LLM to us- ing carefully curated, supervised dataset composed of prompts enriched with domain-specific knowledge cali- brate the numerical tokens predicted from STGNN. Unlike prior methods such as Time-LLM, which adopt generic task descriptions as prompts, our approach integrates node- specific descriptions, temporal pattern insights, and domain expertise into the prompt design, leading to more human- like contextual reasoning for LLMs. By seamlessly inte- grating these three modules, FSTLLM achieves robust and accurate few-shot time series forecasting, while advancing the capacity for spatio-temporal reasoning with real-world contextual awareness.
vant to each time series node. Effectively leveraging node- specific content is critical for capturing the intricate spatio- temporal dependencies required for accurate forecasting. To this end, we employ a pre-trained L.LM to encode node- specific contextual documents and generate semantically rich embeddings tailored to each node.
For example, in the case of a parking lot occupancy predic- tion task, node-specific documents include features such as parking rates, location details, maximum capacity, and user reviews. These textual inputs are processed using a LLaMA- 2-7B model (Touvron et al., 2023), and the final-layer hid- den state representations from the model extracted the node-specific initial embeddings, denoted as Hp R Nx where N represents the total number of nodes in the graph (e.g., the number of parking lots), and D corresponds to the LLM dimension. To transform these representations into form suitable for subsequent graph-based processing, we apply Feed Forward Network (FFN,) to project HD into lower dimension of d. The resulting matrix, denoted by E Nxd contains refined node embeddings, where each row i-th row E, E gd corresponds to the embedding of the i-th node. The embeddings E serve as inputs to the graph attention layer, which formally defined as:
E, (repeat N). E)
E R Nx2d
(1)
Y, = (F,
E Nx2
(2)
Y stack (Yı, Yz...., YxJ) E N
The operation repeat() duplicates the embedding E, across N rows, which is then concatenated with all node em- beddings in using the @ operator, yielding a matrix E, E Nx2d Each row of E, represents a pairwise combi- nation between node and candidate node. This matrix
3.1. LLM Enhanced Graph Construction Module
In this study, we collect and extract set of textual doc- uments containing real-world contextual information rele-

---

FSTLLM: Spatio-Temporal LLM for Few Shot Time Series Forecasting
serves input the Feed Forward Network (FFN22) which produces Y; RR-x2, Each row of Y, denotes the pair- wise relationship indicating the strength and the non-existent correlation between the node pair respectively.
transformation parameterized by the weight matrix W as shown in Equation 7.
X H.W W.
(7)
By stacking the outputs Y, for all nodes, we construct a matrix Y E R N2*x2 x2, where each row encodes the pair- wise spatial correlation between nodes. To normalize these scores and regulate the sparsity of the resulting attention distribution, we apply the a-Entmax function (Peters et al., 2019; Jiang et al., 2024), which generalizes softmax and enables sparse probability assignments. Subsequently, we use a another ford-forward layer FFN E I to produce the final adjacency matrix A E R. for spatio-temporal modeling. The process is formally defined as:
All learnable parameters are updated based on the prediction loss computed over the training datasets. For this purpose, we employ the Mean Absolute Error (MAE) as the loss:
lo+
2(X, θ
X
X
(8)
NT
We aggregate all prediction results of X across all horizons and nodes from the training/testing dataset, represented as C R. Mx NxT, The tensor is then reshaped into numerical prediction tokens as C E IR NxMxx where M denotes the number of training/testing samples, N represents the num- ber of nodes, and T corresponds to the prediction horizons. These numerical tokens are subsequently utilized in the Do- main Knowledge Injection module to fine-tune the LLM, enriching their predictive capabilities.
A = (a- YY)
(4)
3.2. STGNN Module
We integrate the obtained adjacency matrix A into STGNN backbone to derive numerical prediction tokens, which are subsequently used in the prompt to fine-tune the LLM. The framework is flexible and supports the integra- tion of any standard STGNN framework. In this work, we incorporate graph diffusion convolution into the classical GRU architecture described in GTS (Shang et al., 2021). Specifically, the standard matrix multiplication in the GRU is strategically replaced with graph diffusion convolution operation, as defined below:
3.3. Domain Knowledge Injection Module
The Domain Knowledge Injection module is designed to enhance the model's ability to perform human-like reason- ing over spatial and temporal contexts, thereby enabling superior few-shot forecasting performance. This achieved by fine-tuning LLM using carefully crafted prompts that integrate domain-specifir knowledge along with the numer- ical prediction tokens C. Specifically, we employ the su- pervised fine-tuning (SFT) on the LLaMA-2-78 (Touvron et al., 2023) model using QLoRA (Xu et al., 2024) to reduce the computational cost. Each prompt in the training dataset consists of six input components, as illustrated in Figure 1, with an detailed example provided in the Appendix A. The prompt is defined follows:
R2 a (W, *A @ (X,, -1) + by)
Z, o(W, *A $ [ H(-1) + bx)
(5)
H. tanh W A A + X R² @ H1-1) b,))
H. Z. O H + (1 - Z.) + H.
where X, and H. represent the input and hidden state at time step t, respectively. R, and Z, denote the reset gate and update gate at time step t. The sigmoid activation function is denoted by σ. Wr, W., and WA are set of learnable model parameters used in the graph diffusion convolution operation. The graph convolution operation in Equation is defined as follows:
Task Instruction: Detailed instructions describing the specific forecasting task, including the domain of the time series, time-frequency, historical steps, and forecasting steps.
Node-specific descriptions are con- structed by feeding the j-th node's documents and user reviews into the LLMs, which summarize the relevant details for the j-th node.
Node Description:
S-1
W Σ w. WD
(6)
The j-th node's raw and limited train- ing data is fed into the LLMs, which summarize pat- terns such as daily and weekly trends, peak and off- peak periods, and corresponding values.
Node Pattern:
where W- (W.) == represents the set of learnable pa- rameters, D E NNN is the degree matrix, and S denotes the number of steps in the graph diffusion process. X rep- resents the input to the graph diffusion convolution. The prediction X, at time step is obtained through a linear
Historical Input: X E R represents the input time series for the j-th node used by the STGNN backbone.
5

---

FSTLLM: Spatio-Temporal LLM for Few Shot Time Series Forecasting
Table 1. Performance comparison on Nottingham datasct.
15 mins
30 mins
45 mins
60 mins
MAE
RMSE
MАPE
MAE
RMSE MAРE
MAE
RMSE
МAРE
MAE
RMSE
MAPE
ARIMA
12.98
41.73
8.69%
21.16
63.63
14.98%
24.71
77.56
18.91%
28.49
82.47
23.60%
VAR
11.65
41.44
8.30%
27.70
67.06
16.88%
30.06
77.17
31.52%
32.19
91.85
49.00%
DCRNN
7.93
44.64
5.65%
13.21 61.94
9.76%
18.37
76.80
13.53%
24.10
89.67
20.68%
Graph Net
8.03
43.23
6.94%
13.07 61.60
10.30%
18.02
76.18
14.57%
23.05
88.71
19.89%
STSGCN
10.45
51.94
8.25%
15.28
67.03
12.65%
19.78
78.21
18.37%
24.12
87.59
23.78%
GMAN
42.09
102.41
29.01%
43.87
106.62
30.86%
45.53
110.03
33.11%
47.13
113.23
35.74%
GTS
7.89
42.79
5.54%
12.72
60.75
9.50%
17.80
75.08
13.51%
23.06
87.31
20.40%
PatchTST
12.54
48.42
12.93%
18.78
66.65
19.08%
25.03
81.57
24.06%
31.25
94.62
29.69%
iTransformer
7.91
44.03
6.83%
13.12
63.43
10.33%
19.10
77.79
14.77%
26.09
91.80
22.66%
DLinear
7.98
46.28
6.80%
14.60
62.08
12.58%
21.10
77.02
16.97%
28.17
89.96
23.61%
GPT4TS
8.02
44.09
7.06%
13.16 62.51
10.79%
18.61
77.23
15.78%
24.24
90.00
21.55%
TimeLLM
10.55
47.49
8.45%
15.83
64.77
12.09%
22.06
79.48
16.99%
28.57
92.91
25.57%
FSTLLM
7.83
47.93
6.59%
11,53
60.11
9.47%
15.31
71.59
12.31 %
19.09
80.90
17.76%
mins
90 mins
mins RMSE
120 mins
MAE
RMSE
MAPR
MAE RMSE
MAPE
MAE
MAPE
MAE
RMSE
MAPR
ARIMA
33,95 91.98 30.67%
42.19 100.91 37.42%
51.34
123.13
53.74%
58.51
133.27
77.07%
VAR
37.72 91.36 39.51%
43.48 133.10 79.16%
48.73
143.58
64.04%
54.70
164.36
71.75%
DCRNN
30.02 101.37 26.64%
36.01 112.06 34.30%
41.99
122.06
41.96%
47.88
131.27
50.62%
GraphWaveNo
28.52 100.33 24.16%
34.12 110.26 33.22%
39.59
120.34
37.51%
45.16
128.57
47.50%
STSGCN
28.49 95.53 30.55%
32.69 102.47 37.57% %
36.42
108.72
44.28%
39.87
114.03
52.04%
GMAN
48.65 116.01 38.06%
50.13 118.41 40.64%
$1.79
121.29
43.55%
53.54
124.20
46.00%
GTS
28.44 98.38 25.78%
33,87
108.43 33.17%
39.26
117.75
40.52%
44.45
126.22
49.03%
PatchTST
37.43 106.52 36.69%
43.54 117.45 44.59%
49.52
127.56
53.46%
55.37
136.90
63.83%
iTransformer
31.39
102.89 28.82%
37.43
113.68 36.24%
43.32
123.99
43.25%
49.28
132.99
53.94%
DLinear
34.87 101.61 31.10%
42.09
112.45 37.87%
49.20
122.25
44.07%
56.17
131.31
52.37%
GPT4TS
30.20
101 .88 29.05%
36.21
112.42 36.88%
42.20
122.36
42.54%
48.11
131.58
52.24%
TimeLLMM
34.50
104.53 30.78%
40.18
115.20 38.08%
46.47
125.55
46.66%
53.58
136.11
58.96%
FSTLLM
23.18
89.67 21.80
26.65
95.82
26.51% %
30.43
102.88
32.38%
33,89
109.54
39.09%
The numerical predic- tion Cj R. represents the possible future time series values predicted by the STGNN backbone. Specifi- cally, the prediction token Cj represents the computed prediction token for the j-th node from the i-th train- ing/testing sample, where T denotes the prediction horizon.
Numerical Prediction Token:
followed by a comprehensive comparison of FSTLLM's performance against baseline methods to highlight its com- petitive advantages. To further substantiate the efficacy of FSTLLM, we conduct targeted experiments focusing on its core components, demonstrating their pivotal role in enhancing time series forecasting accuracy. Additionally, we perform experiments to demonstrate that our FSTLLM method can be adapted other baseline methods to enhance their forecasting performance.
Future Token: Xj Xi+T E IR' serves as the ground truth token corresponding to the future time steps for the j-th node during fine-tuning. At the evaluation stage, the future token is omitted.
4.1. Experimental Setup
time Datasets. We conduct experiments on two series datasets, with their detailed statistics provided in the Table 2. The Nottingham dataset contains parking lot avail- ability data from 19 car parks in Nottingham, recorded at 15-minute intervals. We crawled this dataset from the offi- cial TramLink Nottingham website The ECL dataset is a subset of the Electricity dataset (Li et al., 2019), compris- ing hourly electricity consumption (measured in kilowatt- hours) from 19 clients. Both datasets are partitioned into training, validation, and testing sets with a split ratio of 70%/10%//00% To create a few-shot experimental setup, the most recent week of data within the training set is set
During inference, the fine-tuned FSTLLM outputs context- aware predictions by integrating real-world contextual in- formation, node-specific analyses (detailed in Appendix D). historical input time series, and numerical predictions derived from the STGNN backbone. FSTLLM jointly con- siders domain knowledge, temporal dynamics, and spatial correlations for accurate few-shot time series forecasting.
4. Experiments
In this section, we evaluate the effectiveness of the proposed FSTLLM method using two real-world datasets. We be- gin with a detailed description of the experimental setup,
'ntps://ww.thettam..eette www.hhhwwww....eataaaeaaaaaa
6

---

FSTLLM: Spatio-Temporal LLM for Few Shot Time Series Forecasting
Table 2. Statistics of our datasets
Datasets
Time Frequency
Time range
Total Time Points
Nottingham
mins
26 Oct 2016 16 Feb 2017
10828
ECL
hour
Jul 2016 2 Jul 2019
26304
aside
former. While attention-based models like iTTranfformer demonstrate moderate performance extended time hori- zons due to their ability to capture distant temporal depen- dencies, they fall short of FSTLLM on short and mid-range horizons, primarily due to their reliance on large datasets and vulnerability to data sparsity. These results underscore the effectiveness of FSTLLM in accurately capturing both spatial and temporal correlations inherent in the data while maintaining robustness across various forecasting windows.
Baselines. We have selected 12 time-series forecasting methods as comparison, including ARIMA (Williams & Hoel, 2003), VAR (Zivot & Wang, 2006), GTS (Shang et al., 2021). GraphWaveNet (Wu et al., 2019). STSGCN (Song et al., 2020), GMAN (Zheng et al., 2020), DCRNN (Li et al., 2017), DLinear (Liu et al., 2024c), PatchTST (Nie et al., 2023), GPT4TS (Zhou et al., 2023), iTransformer (Liu et al., 2024d), and TimeLLM (Jin et al., 2024). We also report other classical baselines such AGCRN (Bai et al., 2020) and MTGNN (Wu et al., 2020) that fail to converge due to lack of training data in the Appendix I.
Appendix F presents the detailed comparison of average MAE, RMSE, and MAPE across all forecasting horizons on both datasets, further validating the superiority of the proposed FSTLLMM model over existing baseline methods.
Metries. To evaluate the performance of different models, we adopt three widely-used metrics for multivariate time series forecasting, including Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and Mean Absolute Percentage Error (MAPE). A lower score indicates better performance.
The results reveal that FSTLLM achieves the lowest average errors across all three metrics, significantly outperforming all baselines. Overall, FSTLLM's superior performance positions it as a highly reliable solution for real-world time- series forecasting tasks, particularly in data-constrained en- vironments.
4.2. Performance of FSTLLM
Reasoning. FSTLLM exhibits superior reasoning abil- ity by integrating real-world constraints and contextual in- formation into its predictions. Unlike black-box models, which often provide numerical outputs without explana- tion, FSTLLM generates predictions that are both inter- pretable and grounded in domain knowledge. As shown in Appendix B, when forecasting parking lot availability, FSTLLM adjusts its prediction tokens to align with real- world constraints. Specifically, in a weekday scenario where parking demand decreases after a peak period (10 AM to PM), FSTLLM adjusts predictions to reflect an expected rise in availability while adhering to the lot's so's capacity limit at the maximum capacity of 512 spaces. This adjustment reflects the model's capacity to incorporate real-world limits into its reasoning, ensuring predictions remain realistic and actionable for users.
Table 1 presents the forecasting performance across multiple horizons on the Nottingham dataset. The best result for each evaluation is highlighted in bold. The results demonstrate that the proposed FSTLLM model consistently outperforms other baseline methods across most scenarios. Specifically, FSTLLM achieves the best performance on 22 out of 24 evaluations, significantly surpassing state-of-the-an models such as GTS and VAR. Furthermore, FSTLLM exhibits con- sistently lower error rates and greater robustness across both short and long forecasting horizons, achieving a MAPE re- duction of approximately 30% compared to other baselines. By efficiently modeling the spatio-temporal inherent in mul- tivariate time series, FSTLLM surpasses both traditional and deep learning- baselines, solidifying its position as a robust solution for few-shot time-series forecasting tasks.
Table 3 presents the forecasting performance across multi- ple horizons on the ECL dataset. The results demonstrate that the proposed FSTLLM model consistently outperforms other baseline methods across the majority of time hori- zons. Specificallyy, FSTLLM achieves the best performance in 25 out of 36 evaluations and the best two performances in 32 out of 36 evaluations, highlighting its robustness and superior accuracy compared to other sute-of-the-an art meth- ods. Notably, FSTLLM exhibits significant improvements in MAPE, achieving relative error reduction of over 50% compared to baseline methods such as GPT4TS and i'Trans-
By incorporating these human-like considerations and domain-specific knowledge, FSTLLM not only improves the accuracy of predictions but also enhances their utility to users. Users gain insights into the reasoning behind the predictions, allowing them to trust and act upon the results more confidently. This capability positions FSTLLM as a critical advancement over traditional deep learning methods, which lack this interpretability and reasoning depth.
7

---

FSTLLM: Spatio-Temporte LLM for Few Shot Time Series Forecasting
Table 3. Performance comparison on ECT. dataset.
hour
2 hours
hours
4 hours
MAE
RMSE
MAPE
MAE
RMSE
MAPE
MAE
RMSE
MAРE
MAE
RMSE
MAPE
ARIMA
2042
3774
17.64%
2959
5407
19.98%
4114
7044
32.78%
4104
6907
33.50%
VAR
1787
3682
15.40%
2512
5206
19.71%
2800
5271
26.26%
3568
8206
30.21%
DCRNN
2155
4694
19.05%
2448
4839
23.34%
2772
5110
27.92%
2947
5223
3161%
GraphWaveNe
1497
3560
11.78%
2274
4659
17.91%
2573
5120
22.22%
2890
5575
26.81%
STSGCN
3124
6719
22.15%
3092
6385
20.65%
3148
6328
21.07%
3007
6013
21.44%
GMAN
6181 10187 59.69%
6249
10332
60.52%
6313
10460
61.26%
6364
10576
61.80%
GTS
3517 6554
26.00%
3676
6716
26.43%
3827
6913
26.91%
3915
6969
27.59%
PatchTST
4789
8206 58.92%
5526
9370
72.17%
5955
10127
79.00%
6204
10761
85.70%
iTransformer
1764
2947 21.76%
2289
4043
28.49%
2874
5073
36.86%
3399
5840
44.53%
DLinear
2701
4409
28.87%
3769
6023
38.86%
4029
6533
44.50%
4854
7790
54.45%
GPT4TS
1672
3527
15.41%
2384
4759
24.34%
3009
5626
33.03%
3621
6598
42.88%
TimeLLM
4950
7640
61.74%
5319
8308
69.56%
5689
8722
75.36%
5906
9072
79.59%
FSTLLM
1663
3503
13.43%
2264
4607
16.37%
2439
4938
16.65 %
2520
5098
17.10%
hours
6 hours
hours RMSE
hours RMSE
MAE
RMSF.
MAPE
MAE
RMSE
MAPE
MAE
MAPR
MAE
MAPE
ARIMA
4014
5912 31.62%
4122
6858
40.10%
4669
7054
43.48%
5102
7773
48.64%
VAR
3664 8582 35.44%
4066
9677
40.96%
4264
9812
47.27%
S291
11933
63.51%
DCRNN
3079 5298
33.82%
3178
5369
35.24%
3251
5446
36.14%
3301
5537
36.61%
Net
3153 5815 31.11%
3274
5844
35.06%
3364
5868
36.88%
3340
5806
37.99%
STSGCN
3059 6462
22.04%
3061
6235
23.00%
3331
7086
28.40%
3041
6402
27.76%
GMAN
6412 10665 62.39%
6438
10721
62.53%
6444
10753
62.39%
6438
10761
62.25%
GTS
3936 6938 27.70%
3895
6838
27.50%
3798
6694
27.22%
3677
6561
26.75%
PatchTST
6564 11094 89.05%
6283
10977
87.91%
5970
10491
81.81%
5617
9721
74.23%
iTransformer
3656 6333 49.80%
3867
6536
53.33%
3687
6308
50.24%
3631
6246
48.18%
DLinear
5978
9081 62.23%
5242
8196
61.10%
5259
8138
61.33%
4989
7739
57.50%
GPT4TS
4133
7277 53.54%
4420
7514
60.10%
4445
7437
60.31%
4187
7099
55.27%
TimeLLM
6112
9308 82.43%
6121
9358
83.02%
6118
9393
83.32%
6071
9439
81.93%
FSTLLM
2584
5201
17.56%
2648
5279
18.17% %
2688
5333
18.33 %
2760
5412
18.81 %
9 hours
10 hours RMSE
11 hours
12 hours RMSE
MAE
RMSE
MAPE
MAE
MAPE
MAE
RMSE
MAPE
MAE
MAPE
ARIMA
4730 7792 46.96%
5322
8499
51.73%
5130
8304
47.66%
5352
8455
48.49%
VAR
5500 12319 67.96%
5287
12864
65.76%
5424
13536
64.49%
6397
15368
58.73%
DCRNN
3293 5618 36.24%
3175
5544
34.27%
2996
5410
31.12%
2796
5314
27.59%
GraphWaveNe
3267
5816 36.24%
3090
5663
33.90%
2895
5448
30.51%
2652
5260
26.75%
STSGCN
2966
5639 24.05%
3136
7345
29.23%
3099
7346
28.58%
3205
7784
30.55%
GMAN
6421 10742 61.58%
6381
10681
60.66%
6327
10607
59.67%
6260
10499
58.79%
GTS
3567 6484 26.07%
3485
6476
25.28%
3425
6528
24.53%
3361
6606
23.80%
PatchTST
5046 8623 63.58%
4283
7351
51.63%
3501
5953
40.11%
2757
4537
28.15%
iTransformer
3130 5362 37.98%
2890
4750
34.24%
2988
4360
29.50%
1979
3562
19.79%
DLinear
7298 11398 62.10%
3947
6335
43.09%
4830
7488
40.11%
2803
4770
28.93%
GPT4TS
3692 6365 44.80%
3107
5802
34.54%
2461
4578
25.08%
1839
3597
17.00 %
TimeL.LM
5917
9060 78.78%
5751
8727
73.46%
5397
8204
67.24%
4976
7613
60.40%
FSTLLM
2822 5519 19.44%
2860
5460
19.80%
2894
5533
20.04% %
2901
5478
20.37%
4.3. Ablation Study
The performance of these variants is evaluated on the Not- tingham dataset, and the results are summarized in Table 4. As shown, the removal of each component results in a no- ticeable decline in performance across all metrics. Among the variants, FSTLL Nolnjecctio exhibits the most signifi- cant performance degradation, underscoring the importance of the Domain Knowledge Injection module. This result highlights the effectiveness of integrating real-world knowl- edge and human-like considerations into the forecasting framework, particularly in scenarios with limited training data.
To further validate the contributions of the structured organi- zation in FSTLLM, we conduct an ablation study by sequen- tially removing different modules of the framework. For this purpose, we define the following variants of FSTLLM:
(1) FSTLLLM-NNonjjectoo:: This variant removes the Do- main Knowledge Injection module entirely.
(1) PSTLLM-NoLLM: This variant replaces the LLM- Enhanced Graph Construction module with commonly used cosine similarity- graph learning operation, and removes the Domain Knowledge Injection module.
8

---

FSTLLM: Spatio-Temporal LLM for Few Shot Time Series Forecasting
Table Ablation Study of FSTLLM on Nottingham dataset.
Table 5. Few-Shot Learning Integration Performance on Notting- ham dataset.
MAE
МАРE
RMSE
NSTLeM--Nonjjcction M
25.1
23.9%
94.5
MAE
MAPE
RMSE
27.1
25.1%
98.0
GPT4TS
27.6
27.0%
92.7
FSTLLM
21.0
20.7%
82.3
STT4T-FFSTLMM
21.8
21.8%
84.3
iTransformer
28.4
26.8%
96.7
(Transformer-FSIIn FSTLLM
22.3
21.7%
86.4
4.4. Few-Shot Learning Integration Study
and human-like reasoning, offering a robust and adaptable solution for real-world applications.
In this section, we demonstrate that FSTLLM can augment existing state-of-the-arr art forecasting models, enhancing their performance in few-shot experimental setups without cessitating extensive modifications to their architectures and tremendous hyperparameter tuning. To evaluate this, we replace the numerical prediction tokens generated by the STGNN backhone with those produced by alternative transformer-based methods and assess the resulting perfor- mance improvements on the Nottingham dataset. To assess the plug-and-play potential of our LLM-enhanced frame- work, we substitute the numerical prediction tokens gen- erated by the STGNN backbone with those from various transformer-baseee forecasting models. Specifically, we re- move both the L.LM-enhancdd graph construction module and the STGNN backbone, and replace them with external transformer- based methods without modifying or retraining those models. This allows us to evaluate the generality of our integration strategy.
Acknowledgement
This study is supported by Singapore Telecommunication Limited (Singtel), through Singtel Cognitive and Artificial Intelligence Lab for Enterprises (SCALE@NTU). Xiucheng Li supported by the National Natural Science Foundation of China under Grant No. 62206074, Guangdong Basic and Applied Basic Research Foundation under Grant No. 2025A15 1001111122000 5012932, Shenzhen Science and Technology Pro- gram (XCY120041202113533300
Impact Statement
This work introduces FSTLLM, novel framework that en- hances time series forecasting by integrating large language models with spatio-temporal deep learning methods. By leveraging domain knowledge and real-world constraints, FSTLLM provides interpretable and accurate predictions, enabling better decision- making and resource allocation in scenarios with limited training data. There is no obvious negative impact on the community.
The results, summarized in Table 5, highlight significant per- formance gains for both GPT4TS and iTransffrmer models when integrated into the FSTLLM framework. Notably, the improvements are consistent across various metrics, under- scoring the framework's ' ability to seamlessly adapt to differ- ent baseline models. These findings suggest that FSTLLM not only preserves its robust reasoning and forecasting per- formance but also amplifies the strengths of diverse fore- casting architectures, making it a versatile and impactful enhancement for existing methods.
References
Bai, L.,, Yao, L., Li, C.,, Wang, X., and Wang. C. Adaptive graph convolutional recurrent network for traffic forecast- ing. Advances in neural information processing systems (NeurIPS), 2020.
5. Conclusion
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert- Voss,, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu,
In this work, presented FSTLLM, a novel framework that leverages LLMs to enhance few-shot time series fore- casting by integrating domain knowledge and real-world constraints. The framework's 's LLM-Enhanced Graph Con- struction and Domain Knowledge Injection modules enable interpretable, accurate predictions by capturing contextual information and fine-tuning with numerical prediction to- kens. Experimental results and ablation studies demon- strate FSTLLM's 's effectiveness in improving performance across various metrics, even with limited training data. Ad- ditionally, its adaptability to sfate-of-the-art models, such as GPT4TS and ITransformer, highlights its versatility without requiring significant architectural modifications. FSTLLM bridges the gap between deep learning- based forecasting
J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C.,, McCandlish, S., Radford, A., Sutskever, I., and Amodei, D. Language models are few-shot learners. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
Chang, C.,, Peng, W., and Chen, T. LLM4TS: two-stage fine-tuning for time-series forecasting with pre-trained 1lms. CoRR, abs/2308 0084699 2023.
Chen, X., Li, X., Chen, X., and Li, Z. Structured matrix
9

---

FSTLLM: Spatio-Temporal LLM for Few Shot Time Series Forecasting
basis for multivariate time series forecasting with inter- pretable dynamics. In Neural Information Processing Systems (NeurIPS), 2024.
Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., and Iwa- sawa, Y. Large language models zero-shot reasoners. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
Chung, J., Gülgehree ç., Cho, K., and Bengio, Y. Empirical evaluation of gated recurrent neural networks on sequence modeling. CoRR, abs/ 141.3555, 2014.
Li, S., Jin, X., Xuan, Y., Zhou, X., Chen, W., Wang. Y., and Yan, X. Enhancing the locality and breaking the mem- ory bottleneck of transformer on time series forecasting. Advances in Neural Information Processing Systems 2019.
Cirstea, R., Kieu, T., Guo, C., Yang. B., and Pan, S. J. Enhancenet: Plugin neural networks for enhancing corre- lated time series forecasting. In International Conference on Data Engineering (ICDE), 2021.
(NeurIPS), pp.
Li, Y., Yu, R., Shahabi, C., and Liu, Y. Graph convolutional recurrent neural network: Data-driven traffic forecasting. CoRR, abs/17777.0126, 2017.
Cirstea, R., Guo, C., Yang. B., Kieu, T., Dong, X., and Pan, S. Triformer: Triangular, variable-specific attentions for long sequence multivariate time series forecasting. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), 2022.
Liu, C., Xu, Q., Miao, H., Yang, S., Zhang, L., Long, C., Li, Z., and Zhao, R. Timeema: Towards lmm-empowered time series forecasting via cross-modality alignment. arXiv preprint arXiv:2406.01638, 2024a.
Drucker, H., Burges, J. C., Kaufman, L.., Smola, A. J., and Vapnik, V. Support vector regression machines. In Advances in Neural Information Processing Systems (NeurIPS), 1996.
Liu, C., Miao, H., Xu, Q. Zhou, S., Long, C., Zhao, Y., Li, Z., and Zhao, R. Efficient multivariate time series fore- casting via calibrated language models with privileged knowledge distillation. In ICDE, 2025.
Guo, S., Lin, Y., Feng, N., Song, C., and Wan, H. Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. In The Conference on Artificial Intelligence (AAAI), 2019.
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., and Neubig, G. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Comput. Surv., 55(9):155:1119::3, 2023. doi: 10. 145/3560815
Guo, S., Lin, Y., Wan, H., Li, X., and Cong, G. Learning dy- namics and heterogeneity of spatial-temporal graph data for traffic forecasting. IEEE Transactions Knowledge and Data Engineering (TKDE), 2022.
Liu, P., Guo, H.,, Dai, T., Li, N., Bao, J., Ren, X., Jiang, Y., and Xia, S.-T. Calf: Aligning ilms for time series forecasting via cross-modal fine-tuning, arXiv preprint 24Xi::2403077300 2024b.
Han, L., Du, B., Sun, L., Fu, Y., Lv, Y., and Xiong, H. Dynamic and multi-faceted spatio-temporal deep learning for traffic speed forecasting. In Zhu, F., Ooi, B. C., and Miao, C. (eds.). Conference on Knowledge Discovery and Data Mining (KDD), pp. 547-555, ACM, 2021.
Liu, S., Yu, H., Liao, C., J., Lin, W., Liu, A. X., and Dust- dar, S. Pyraformer: Low-complexity pyramidal attention for long- Long-range time series modeling and forecasting. In International Conference on Learning Representations (ICLR), 2022.
Hochreiter, S. and Schmidhuber, J. Long short-term memory. Neural Comput., 98): 1780, 1997. ISSN 0899- 7667.
Liu, Y., Hu, T., Zhang, H., Wu, H., Wang, S., Ma, L., and Long, M. itransformer: Inverted transformers are effective for time series forecasting. In International Conference on Learning Representations (ICLR), 2024c.
Jiang, Y., Li, X., Chen, Y., Liu, S., Kong, W., Lentzakis, A. F., and Cong. G. SAGDFN: A scalable adaptive graph diffusion forecasting network for multivariate time se- ries forecasting. In International Conference on Data Engineering (ICDE), pp. 1255-1268. IEEE, 2024. doi: 10.1 16 46.2024.00101.
Liu, Y., Hu, T., Zhang, H., Wu, H., Wang, S., Ma, L., and Long, M. itransformer: Inverted transformers effective for time series forecasting. In International Conference on Learning Representations (ICLR), 2024d.
Jin, M., Wang. S., Ma, L., Chu, Z., Zhang, J. Y., Shi, X., Chen, P., Liang, Y., Li, Y., Pan, S., and Wen, Q. Time-llm: Time series forecasting by reprogramming large language models. In International Conference on Learning Repre- sentations (ICLR), 2024.
Liu, Y., Qin, G., Huang, X., Wang, J., and Long, M. Au- totimes: Autoregressive time series forecasters via large language models. In Advances in Neural Information Processing Systems (NeurIPS), 2024e.
10

---

FSTLLM: Spatio-Temporal LLM for Few Shot Time Series Forecasting
Martins, A. and Astudillo, R. From softmax to sparsemax: A sparse model of attention and multi-label classification. In International conference machine learning, pp. 1614- 1623. PMLR, 2016.
Shi, X., Chen, Z., Wang, H., Yeung, D.-Y., Wong, W.-k., and Woo, W.-c Convolutional Istm network: A machine learn- ing approach for precipitation nowcasting. In Advances in neural information processing systems (NeurIPS), PP. 802-810, 2015.
Miao, H., Liu, Z., Zhao, Y., Guo. C., Yang, B., Zheng, K., and Jensen, C. S. Less is more: Efficient time se- ries dataset condensation via two-fold modal matching. PVLDB, 18(2):226-22 238, 2024a.
Shi, X., Wang, S., Nie, Y., Li, D., Ye, Z., Wen, Q. and Jin, M. Time-moe: Billion-scale time series founda- tion models with mixture of experts. In International Conference on Learning Representation (ICLR). Open- Review.net, 2025.
Miao, H., Wang, S., Zhang, M., Guo, D., Sun, F., and Yang, F. Deep multi-view channel-wise spatio-temporal net- work for traffic flow prediction. CoRR, abs/2404.15034, 2024b, doi: 10.48550/ARXIV2244..1.
Song, C.,, Lin, Y., Guo, S.,, and Wan, H. Spatial-temporaa synchronous graph convolutional networks: A new frame- work for The Conference on Artificial Intelligence (AAAI), pp. 914- 921, 2020.
network data forecasting. In
Miao, H., Xu, R., Zhao, Y., Wang, S., Wang, J., Yu, P. S., and Jensen, C. A parameterr-efficient federated frame- work for streaming time series anomaly detection via lightweight adaptation. TMC, 2025.
Tan, M., Merrill, M. A., Gupta, V., Althoff, T., and Hartvigsen, Are language models actually useful for time series forecasting? CoRR, abs/2406. 16964, 2024. doi: 1044550/ARXIV.2406..O. 16964.
Min, S., Lyu, X., Holtzman, A., Artetxe, M., Lewis, M., Hajishirzi, H., and Zettlemoyer. L. Rethinking the role of demonstrations: What makes in-context learning work? In Empirical Methods in Natural Language Processing (EMNLP), pp. 1 1064, 2022. doi: 10.18653/V1/ M12.EENLPP-MAIN...
Tezekbayev, M., Nikoulina, V., Gallé, M., and Assylbekov, 7. Speeding up entmax. CoRR, abs/21 11 006832, 2021.
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaci, Y.. Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., Bikel, D., Blecher, L.., Ferrer, C. C.,, Chen, M., Cucurull, G., Esiobu, D., Fernandes, J., Fu, J., Fu, W., Fuller, B.. Gao, C., Goswami, V., Goyal. N., Hartshorn, A., Hosseini, S., Hou, R., Inan, H., Kardas, M., Kerkez, V., Khabsa, M., Kloumann, I., Korenev, A., Koura, P. S., Lachaux, M.-A., Lavril, T., Lee, J., Liskovich, D., Lu, Y., Mao, Y., Martinet, X., Mihaylov, T., Mishra, P., Molybog, I., Nie, Y., Poulton, A., Reizenstein, J., Rungta, R., Saladi, K., Schelten, A., Silva, R., Smith, E. M., Subramanian, R., Tan, X. F.,, Tang. B., Taylor, R., Williams, A., Kuan, J. X., Xu, P., Yan, Z., Zarov, I., Zhang, Y., Fan, A., Kambadur, M., Narang, S., Rodriguez, A., Stojnic, R., Edunov, S., and Scialom, T. Ilama 2: Open foundation and fine-tuned chat models, 2023.
Nie, Y., Nguyen, N. H., Sinthong, P., and Kalagnanam, A time series is worth 64 words: Long-ter term forecasting with transformers. In International Conference on Learning Representations (ICLR), 2023.
Pan, Z., Jiang, Y., Garg, S., Schneider, A., Nevmyvaka, Y., and Song, D. S2IP-LLM: semantic space informed prompt learning with LLM for time series forecasting. In International Conference on Machine Learning (ICML). 2024.
Peters, B., Niculae, V., and Martins, A. F. T. Sparse sequence-to-sequeece models. CoRR, abs/ Abs//995.05702, 2019.
Shang, C., Chen, J., and Bi, J. Discrete graph structure learning for forecasting multiple time series. In Interna- tional Conference Learning Representations (ICLR), 2021.
Wang, S., Zhang, M., Miao, H., Peng. Z., and Yu, P. S. Mul- tivariate correlation-aware spatio-temporal graph convo- lutional networks for multi-scale traffic prediction. ACM Trans. Intell. Syst. Technol., 13(3):38:1338:22 2022. doi: 10. 10.1145/3469087.
Shao, Z., Zhang, Z., Wang, F., and Xu, Y. Pre-training enhanced spatial-temporal graph neural network for mul- tivariate time series forecasting. In Proceedings of the Conference on Knowledge Discovery and Data Mining (KDD), pp. 1567-1577, 2022a.
Wang, X., Feng, M., Qiu, J., Gu, J., and Zhao, From news to forecast: Integrating event analysis in Ilm-based time series forecasting with reflection. In Advances in Neural Information Processing Systems (NeurIPS), 2024.
Shao, Z., Zhang, Z., Wei, W., Wang, F., Xu, Y., Cao, X., and Jensen, C. S. Decoupled dynamic spatial-temporal graph neural network for traffic forecasting. Proc. VLDB Endow. 15(1 15(11):2733-273 2022b.
Williams, B. and Hoel, L. Modeling and forecasting vehicu- lar traffic flow as a seasonal arima process: Theoretical basis and empirical results. Journal of Transportation Engineering. 129:664 672, 2003.

---

FSTLLM: Spatio-Temporal LLM for Few Shot Time Series Forecasting
Woo, G., Liu, C., Sahoo, D., Kumar, A., and Hoi, S. C. H. Etsformer: Exponential smoothing transformers for time- series forecasting. CoRR, abs/2202.01381, 2022.
long sequence time-series forecasting. In The Conference Artificial Intelligence (AAAI), pp. 11106-11115, 2021.
Zhou, T., Ma, Z., Wen, Q. Wang, X., Sun, L., and Jin, Fedformer: Frequency enhanced decomposed trans- former for long-term series forecasting. In International Conference on Machine Learning (ICML), 2022.
Wu, H., Xu, J., Wang, J., and Long, M. Autoformer: Decom- position transformers with auto-correlation for long-term series forecasting. In Advances in Neural Information Processing Systems (NeurIPS)), 2021.
Zhou, T., Niu, P., Wang, X., Sun, L., and Jin, R. One fits all: Power general time series analysis by pretrained L.M. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
Wu, Z.,, Pan, S., Long, G., Jiang, J., and Zhang, C. Graph wavenet for deep spatial-temporal graph modeling. In International Joint Conference on Artificial Intelligncce (IJCAL. pp. 1907-1913, 2019.
Zivot, E. and Wang. J. Vector autoregressive models for multivariate time series. In Modeling Financial Time Series with S-PLUS@, pp. 385-429. 2006.
Wu, Z.,, Pan, S., Long, G., Jiang, J., Chang, X., and Zhang, C. Connecting the dots: Multivariate time series fore- casting with graph neural networks. In Proceedings of theSIGKDD International Conference on Knowledge Dis- covery & Data Mining (KDD), 2020.
Xie, Y., Zhao, K., Sun, Y., and Chen, D. Gaussian processes for short-term traffic volume forecasting. Transportation Research Record, 2165(1):69-77)) 2010.
Xu, Y., Xie, L., Gu, X., Chen, X., Chang, H., Zhang, H., Chen, Z., Zhang, X., and Tian, Q. Qa-lora: Quantization- aware low-rank adaptation of large language models. In International Conference Learning Representations (ICLR), 2024.
Yu, B., Yin, H., and Zhu, Z. Spatio-temporal graph convo- lutional neural network: A deep learning framework for traffic forecasting. CoRR, 0tsp/1709.48777 2017.
Zhang. Y. and Yan, J. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. International Conference on Learning Representations (ICLR), 2023.
Zhao, L., Song, Y., Zhang, C., Liu, Y., Wang, P., Lin, T., Deng, M., and Li, H. T-gen: temporal graph convolu- tional network for traffic prediction. IEEE Transactions on Intelligent Transportation Systems (TITS), 21())3848- 3858, 2020.
Zhao, Z., Lee, W. S., and Hsu, D. Large language models as commonsense knowledge for large-scale task planning. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information
Processing Systems (NeurIPS), 2023.
Zheng, C., Fan, X., Wang, C., and Qi, GMAN: graph multi-attention network for traffic prediction. The Conference on Artificial Intelligence (AAAI), pp. 1234- 1241, 2020.
Zhou, H., Zhang. S., Peng, J., Zhang, S., Li, J., Xiong, H., and Zhang, W. Informer: Beyond efficient transformer for
12

---

FSTLLM: Spatio-Temporal LLM for Few Shot Time Series Forecasting
A. Supervised Fine-tuning Prompt of FSTLLM.
In this section, we present a detailed prompt example used in supervised fine-tuning FSTLLM on Nottingham dataset as in Figure 2.
FSTLLM Fine-tuning Prompt
<SH(NNTT] Role: You are an Al agent responsible for prodicting the availability of parking lots, Objective: Your task to foreaast the number of availble parking lots for the next 2 hours, To this, you mnlyye data
next hours other additional information that could afTect the parking lots availabilit prediction values. Input Data: (1) Car park description St Marks Place NCP Car Park A Afordabbe and A Accessible Parking in Newark. The St. M Marks Place NCP Car Park, located on Lomband Street (NG24 IXT) Newark and Shewwood, offers spacious affordable multi story parking facility with 512 spaces, making it convenient choice for vistarr to the area. With beight iesriction of 1.96 meters and dedicated disahled parking speces, this car park caters to a variety of vehicles and accessciilit needs. Open 2477 St Marks hour say. Eady hird discounts are available for those ariving before 9:00 am, making it an economical option for all-day parking. Payment options include tickated parking, card payments, phone payments, and the NCP A up (Location ID 32089), ensurin convenient experience for all commitment to prvviiig a secure inviromennt, some users have noted that can he tight to nevigste even for smaller cars, due to the high concrete sides, Desgite this, St Marks Place NCP Car Park remains practica and accessible choice for these looking for convenient parking the Newark area. (2) Prediction borizen: you are given hours histocical input from 2017-01-24 14:11:14 to 2017-01-24 16:16:4 on Tuesday, and you are supposed to prodect future parking lots from 201 7-01-22 16:31:14 to 2017-01-2 24 18:16:14, (3) The Natural pattems of parking avaiabiltty (eg, peok and off peak periods throughout the day) are as follow: Based on the grven parking lot records and detailed observation of the evai available parking spaces over the week, bere is a more spectfic breakdown of when peaks (maximum availabliit) and dips (mimimmum evailability) occur: On Weekdays (Monday to Friday): Peak av ailshillyy From PM ln 6 AM traching the maximum capacily of 512 spaces) This reflects the time when parking demand is lowest, typically during the late night corly momning hours. Minimum awailability: Generlly ocnrs between 10 AM to PM, with availability drogping the busiest, likely due to work or daily aies On Weekmnc (Selunday and Sunday): ) Pesk av availabilty. Similar to weekdays, from a PM to 6 RAM (mainlaining the full capacily of with avalability dropping
post hours of parking records, along with simulation hased prodictions
provides flexible and budget-
pricing, with rales starting E199 hou: E3.90 for hours, and just 28,
full 24-
While the car park has comed Parking Safety Award for
spaces.
indicates that these
spaces), indicating lower demand duiing tasse hours, Minimum av evalability Uccus between moderate to high parking usage during midday hours, In summary, the parking lot consistently reabees full capacity during the late e evaning and earty morning hours, with the bighest usage occuring during the day, esperially from mid momiin mid afberoon
as low as 248-37 spaces, showing
both woekdays and weekmnss. (4) Historical Recorts: Parking data for the past 2 hours [326. 339. 344, 348, 359. 369, 370. 382.) (5) Simulation predictions: Forecasts for parking watlabilly for the next 2 hours based on smmlation models (389 95435 407. 14072 426. 15? 443,97525 464 05444 484,45245 5884 524, 72836) Consider all provided informaton (hisionical records, smmulaiinn predictions and additonal factors) predict parking lol availability for the next 2 hours with accuacy. Please melyze the data and provide only your finel numerical predictions. DO NOT include any additional contect in your answer. Striclly eaclose your answer inside square brackets lI and provide exactly numetical velues. 
 [399, 425, 425. 451. 475, 499. 501. 512.].45>>
Thee inturtionn
Node Decntation
Rode Pattern
Notorcal Irpu:
Prodiction Tokon
Future Toker
Figure 2. A sample of detailed FSTLLM supervised fine-tuning prompt.
B. Evaluation Prompt of FSTLLM.
In this section, we present detailed evaluation prompt example used by FSTLLM to provide predictions Nottingham dataset as in Figure 3.
C. Full Prompt of FSTLLM on ECL dataset.
In this section, we present the full prompt example used by FSTLLM on ECL dataset as in Figure 4.
D. Node Description and Node Pattern.
In this work, we utilized ChatGPT-40 to assist in node description and node pattern generation. Specifically, we follow below template to generate node description and node pattern using ChatGPT-40.
For Node Description Prompt:
"You are given a feature description from [carpark description link] and user reviews from [Google review link]. Please synthesize an inductive description of the carpark using content from both sources."
For Node Pattern Analysis Prompt:
"I will provide one week of parking lot records for a carpark in England, recorded every 15 minutes starting from 12:01 PM on 20-6-10-26. Please describe the temporal usage patterns of the carpark, including variations throughout the day and across weekdays versus weekends. Indicate observed peak and low-demand periods, along with any consistent behavioral trends in parking availability. The records are: [extracted training data]."
13

---

FSTLLM: Spatio-Temporal LLM for Few Shot Time Series Forecasting
FSTLLM Evaluation Example
<SS[NNST] Role: You are AT agent responsible for prodicting the availability of parking lots. Objective: Your task fomrast the number of available parking lots for the next 2 hours. To do this, you will anayze data Irom: the past hous of parking records, along with simulatoon--iso predictions for the nexl 2 hours and other additiocal information that could affect the parking lots availability prediction values, Ingut Data: (1) Car park descritiion St Marks Place NCP Car Park A ffortabbl and Accessibbe Parking Newark. The St Marks Place NCP Car Park, located on Lombard Steee, (NG24 IXT) in Newaik and Shewwoo, offers spacious and affondable mulli-story parhing facilly with 512 spaces, making convenient choice for visttors the area. With beight restrctioon of 1.98 meters and dedicated disabled parking spaces, this car park caters to variety of vehicles and access:bility needs, Open 24/7, St Marks Place provides flexible and budget-
pricing, with rotas starting from E1 61005 for hour, E3. for 2 hour, and just 16, a full 24- hour sty, Farty hird discounts are wailabbe for those arivingg before 9:00 am, making it an aconomical option for all day porking. Payment options include ticketed parking, card peymentss, phone pay peymeuss,
the NCP App (Locatio: ID 32089), essuring convenient exgesience for users. While the car park has eamed Parking Salety Award for commitment to groviding a secuze enviroemett, some usets have aoted that it can be Light to navigate, evea for smsiller cars, due to the high coucrete sides. Desgule tais. St Marks conv convenient parking the Newars area (2) Prediction bortzon: you are grven hours historical inpt from 2017-01-24 24 14:31:14 to 201 7-01-24 66:16:14 14 on Tuesday, and you are supposed to prodect future parking lots from 201 7-01 24 16:31: 14 to 2017-01 77-01-24 18:16:1 (3) The Natural pallems of perking wailsbilliy (eg. peak and off pesk periods throughout the day)
Place NCP Car Park remains practical and accessible choice for those looking
follow: B Based on the given parking lot records and detailed observation of the available parking spaces over the week, here a more specific braakdwwn of when peaks (maximum availabllity) and dips (minimum availability) occur On Worksaps (Morday to Friday': Peak availebility From PM Io AM (reahing the maximum capacily of 512 spaces). This reflects the time when parking demand lowest typically during the late night and early merning hours. Minimum evailablitt: Generally occus between 10 AM to 3 PM, with availability drogping to low 223-33 374 spaces. This indicates that these hous are the bustest, likety due to work daily activities. On Weekunds (Satunday and Sunday): Peak availability: Similar to weekdays, from 8 PM to 6 AM (mainaining the full capactty of 512 spaces), catinggin lower demand during these hours. Minimum avalabilit:: Occurs between 11 AM to PM, with arailability dropping to as low as 248-37 spaces, showiigg moderate to high porking usage during midday hours, In summary, the parking lot consistently neches full capucity during the late evening and enrly moming hours, with the highest usage occurring duting the day, especially from mid moming to eemeonn across both weekdays and weekerds.. (4) Historical Recorts: Parking data for the past 2 hours [326. 339, 344, 34R. 350, 369, 370. 302.) (5) Simulation predictions: Foreaatts for perking wes labiliy for the next 2 hours based on simulation models (380 95435 407. 14072 426. 157 #43,97525 464 05444 45445 507 5884 524, 72856) Consider provided information (histoncal records, smulation prediction, and addinonal factors) product parking availabiiity for the next 2 hours with ancuracy Please malyye the data and provide only your finel numesical predictions. DO NOT und any additional content in your answer Strictly enclose your answer inside square brackets [] and provide exartly 8 numerical valoss. 
 [398. 408. 433. 451. 466, 494. 512, 512 oossimorrf prddictions for the parking lot availability over the noxt two hours, 1 relied on a combination of the follow ing factors: Historical Recorts The provided data showed an incressing ueed over the past two hours with values starting at 326 and ending al 382, indicating gradusl tise in the number of available parking spaces. 2. Simulation Predictions The simulation-base predicions showed a contimauus increase in availabiiity poojectiig values that ranged from 389 524, These values suggest an upward trend over the next two hours. 3. Peak and Off-Peal Patterns: The prediction bottzon (trom 16:31 to 18:16) falls just afte the bustest pertod (10 AM to PM] on Weelday, meaning parking availability is likely to rise as the day progesses Based on the natural patterms, parking avaiabilii tends incmase after PM and neches its pook by PM, 4. Capacity Constmints: The parkingg lot has maximum cepacity of 512 speces, which acts as an upper bound for the prediction. This suggests that values neating or above 512 should be adjusted to reflect the full occupancy of the car park. Basad on these consideration, the cted values were affusted align with the obsmvved potterns and capacity lime, mealting in a final prodiction of: [396, 408, 433, 451, 466, 494, 512, 512]. This reflects the expected gradual incrsase in av availshility with the final values capped at the lot's is maximum capacily.
genarata
Thst intructioon
Nods Decatstion
Node Pattom
Inpu:
Prediction
FST Answer
Figure 3. A sample of detailed FSTLLLM evaluation prompt.
E. Implementation Details.
FSTLLM is fine-tuned on the training examples for 2 epochs on a Linux workstation with an Intel(R) Core(TM) 17-13700K CPU @ 5.40GHz and a NVIDIA A6000 GPU. We used 4-bit quantization (Liu et al., 2023) to obtain more compact model representation, and low rank adaptation (LoRA) (Xu et al., 2024) to reduce the numher of trainable parameters and decrease the GPU memory requirements. We set LoRA attention dimension to be 64 and an initial learning rate to be 2e-4 with Adam optimizer. We set a to be 2.0 in the a- e-Ennaax function and the depth of the graph diffusion convolution S is set to 3. The hidden size of GRUs is set 64. For all the baselines, we use their original implementations with minimum modifications required to on our datasets. We set the sequence length and prediction horizon to 12 for the ECL dataset, it exhibits notable daily and weekly patterns similar to traffic datasets such as METR-LA and PEMS-BAY, where a sequence length of 12 is commonly used. For the Nottingham dataset, since the average commuting time is less than 90 minutes?, we set the sequence length and prediction horizon to 8 (equivalent to 120 minutes) to support parking lot predictions for users departing on their commutes.
Due to hardware limitations- usinn only a single GPU for inference we could not employ data parallel methods to expedite inference, which are commonly used in other baseline methods such as (Jin et al., 2024; Zhou al., 2023). Consequently, we extracted 19 users from the original 320 users in the electricity dataset to demonstrate the performance of FSTLLM and the baseline methods. However, we emphasize that FSTLLMM is fully trainable on the entire electricity dataset with 320 nodes. The graph size of 320 nodes is manageable by the STGNN backbone (Shang et al., 2021; Shao et al., 2022a), and the Domain Knowledge Injection module fine-tunes LLLMA-2-77 in a node by node manner. Thus, the node count does not impact the computational cost during fine- tuning. We plan to evaluate FSTLLM and the baseline methods on the full electricity dataset in future work when we have access to multiple GPUs.
14

---

FSTLLM: Spatio-Tempore LLM for Few Shot Time Series Forecasting
ESTLLM Fine-tuning Prompt_ ECL
SSIINST] SsP|INST] Rol: You
agent responsible predicting the user electricity comsumption. Objective: Your to forecsst next 12 electricity consumption hased on client past 12 hours historical records. To do this, you will analyse data from the past 12 hours records, along with simulatioe-hsse predictions for the next 12 other additional information that could affect electricity predictioon valuss. Input Data: (1) Client description: The ekericity the ECL dataset represents electricity consumption (Kwh) of 19 clients. This contains the electricity consumption of clients in certain region of 2 years. There are 20 clients recording every I hour intervals of electricity consumption Each client has his usage pattern and habbit, therefore, their behavior different and should be consider seperately. You now given the data from client no. 2, and you will be focusing on proticting future values for client no. 2.. (2) Prediction horizon: you are given 12 hour historical inpul from 2019-02-15 11:00:00 to 2019-02- 15 22:00:00 on Fiiday, and you are supposed to predict future eleciricity comsumption from 2019-02-15 23:00:00 2019-02-16 10:00:00. (3) The Natoral pallems of electricity consumption of this client (egg, pesk and off-penk periods thrughout the day) are follow: This client electriitty consumption over week displays a clear pattem with higher usage during daytime hours and lower consumption night, suggesting peak activity aligned with business hours. The highest consumption occurs on Thursday at 2 PM, reaching 167.0, likely indicating peak operational scrivity, while lowest consumption recorded Sunday at AM, at 50,043.0, refleating minimal or no activity during hours. Weekdays generally show higher daytime usage. with consumption often supassing 80,000.0 between late morning and early evening. which contrats with the weekends. where usage is genemaly lower, rurely excoeding 75,000.0. This pattern suggests a drop openational demand over the weekends, with energy
hourly
night
substantially tapering off ovanightt and then gradually rising loward midday each day.. (4) Historical Records: dkcctricity consumption for the past 12 hour [ (95501. 107127. 121709, 132627. 126668 119752. hour based sirulltion models 178272.84 69957.67 56337.67 38943.453 28450.633 27399.63 629 27993.75 31317.69 46747. 15 62374.055 71993.08 79677.27 J. Consider all provided information (historical records, simulation predictions, and additional factors) predict electricity consumption for the next hour with accuracy. Please analyze the data and provide only your final numerical predictions. DO NOT include any additional content in your answer. Strictly undosse your answer inside square brackets [] and provide exactly 12 numetical valaes. (JNTT] (91541. $8668.
120418. 124918. 128501. 126043, 118501, 100376.) (5) Simolation predictions: Forecasis for electricity consumption for the
79793, 59167, 54418, 54710. 53251, 51585, 61918. 69502, 72543 85376.
FSTLLM Evaluation Prompt ECL
(PP[NNST] SEP
 Role: You are an Al sgent reponsible for predicting the user electricity consumption, Objective: Your lask is to forccast the nexi 12 hour clactricity consurpption based client post 12 hours historical records, To this, you will analyse data from the past 12 hours
along with simulation-hssod predictions for the 12 hours and other additional information that could affect the electriityy prediction values. Inpur Data: (l) Client description: The eletricity in the ECL dotnset represents the hourly electricity consumpion (Kwh) of clients. This contains the electrictty comsumption of clients in certann region of years. There 20 clients recording every I hour intervals of clectricity consumption. Each elient has his own usage pallern and habbit, therefore, their bchavior are differenl and should be consider seperalely. You are now given the data from client no. 2, and you will be focusing on predicting future values for client no. 2. (2) Prediction horizon: you given 12 hour historical input from 201 9912--2-15 1 11:00 to 209-412- you are supposed to predict funure electriityy consumption from 2019-02-15 23:00:00 to 2019-02-16 10:00:00. (3) The Natuaal patterns of electricity
22:00:00 on Friday, consuription of this elient (egg, penk and off-peak periods throughout the day) are as follow: This client electricity consumption over the week displays clear puttern with higher business hours. The highest consumption occurs on Thansdy
PM, reaching 19.167.0, likely indicating peak operational activity, while the lowest consumption is recorded on Sunday at 3 AM, at 30,043.0, reflecting minimal or no activity during late night hours. Weekdays generally show ligher daytime usage, with consumption eflen surpassing $0,000.0 beween late morning and carly evening. which contrasis
usage during daytime hours and lower consumption night, suggesting peak activity aligned
the woekends. where usage is geverally lower, rarely exceoding 75.000.0 This pollern suggests a drop in operitional demand over the weekends, with energy needs substantially ly tapering off overnight and then gradually rising toward midday each day.. Historical Records: elecrricity consumption for past 12 hour 89501. 103127. 121709. 132627. 126668. 1 119752. 120418. 124918. 128501. 126043. 118501. 100376.). (5) Simulation predictions: Forecasts for electricity consumption for the next 12 hour based on simulation models [78272.84 69957 67 56337 67 38943 453 28450.633 27399 629 27993.75 75 31317.69 46747.15 623740055 71993.00 08 79677.27] Consider all provided information (historical records, simulation predictions, and additiona factors) to prodict cloctricity consurmption for the ned hour with accurncy Plense analywe the data and provide only your findl numerical predictions DO NOT include any additional content in your answer. Striclly enclose your anewer inside squire brackess II and provide exactly 12 numerical values, [JNST]] [74668, 67709.
50085, 36792, 36544, 36168. 35417. 48835. 63793, 74835, 88167.99 82334. J.1 hope this answers your question. Please lel me know if you have any other questions or need further assistance.
Figure 4. A sample of detailed FSTLLLM fine-tuning and evaluation prompt on ECI. dataset.
Nominghand PSTLLM
ARIMA
VAR
DCRNN
Gragh
STSGEN
GMAN
GTS
PuchTST
Trandfomss
DLincar
GPT4TS
T.maLLM
MAE
21.0
$4.2/98 6%
39
27.453.5%
26 2/
15 9%
47.8/56 1%
25.9/19.15
34.2/38 6%
28 1%
8138/33
31.5/33.39 27.623395
RMSE
82.3
39.3/7.9%
101.2/18.7%
91.2998%
88.28 7%
114.0/7.80 8% 37,144.1
89.6/8.1%
97.5/15.8%
92.11144 96 46.99
95 1.
MAPE/N)
33 33.1/37.49
45.053.99
28.9/33
A/7
24.7/16.0%
35,5
6%
28 24.2/66.4
27.
27.0233.25
29 733020
ECI.
ESTLIM
ARIMA
VAR
DXRRNN
Grrggh WawaNe
STSGEN
GMAN
GTS
PachTTT
Transformo
TH Ancor
GPT4TS
Timell 1M
MAE
2291
3H1733.49 49
2666/20.0
255007 1.
28550 39
3108 P
630483888 8%
3774039
3808/00
2954
1266 1b
2964722.877
RMSH
5809/19.66 99
618924.6%
50.3772%
15
66452292
10464155
35
6818831
2012
511388 7%
676331.00 100
5553/16.0%
561045 8%
MAPE/N)
16.2
25.4 27.
27.2/40.9%
28.943.9%
94.9944.%
26.9/39.80 8%
76.977.99
97.9157.%
33.8/52.1% 45.8/64.6%
196
73.7778.0%
Table 6. Average MAE, RMSE, and MAPE for different methods, along with the percentage improvement of FSTLLM over each method (shown as Avg/Improvement %).
15

---

FSTLLM: Spatio-Temporal LLM for Few Shot Time Series Forecasting
F. Average MAE, RMSE, and MAPE Comparison.
G. Data efficiency comparison.
To evaluate data efficiency, we compare FSTLLM trained on just 3 days of data with representative subset of baselines trained on both 3 days of data and 30 days of data, using the Nottingham dataset. The results are summarized in Table 7. These results demonstrate that FSTLLM consistently outperforms strong baselines under limited data scenarios. This further validates the robustness and adaptability of our framework in few-shot settings. Furthermore, the results show that FSTLLM with only 3 days of training data outperforms all baselines trained with 10x more data, highlighting the strong data efficiency of our approach.
Table 7. Performance Comparison of on Different Training Days (3 Days and 30 Days)
Method
MAE
RMSE MAPE (% )
FSTLLM (3 days)
22.84
83.68
22.33
GTS (3 days)
29.57
87.44
26.04
GPT4TS (3 days)
33.24
93.92
26.95
PatchTST (3 days)
34.65
97.45
32.38
DLinear (3 days)
37.52
95.87
30.62
GTS (30 days)
23.54
90.88
23.02
GPT4TS (30 days)
27.50
86.88
25.70
PatchTST (30 days)
33.07
95.20
30.13
DLinear (30 days)
32.82
92.64
26.63
H. The a-Entmax function.
In multivariate time series forecasting, the spatial correlations normalized using the Softmax function (Bai et al., 2020; Wu et al., 2020; Cirstea al., 2021) often include significant proportion of low-weight entries due to the nature of the Softmax function. Such low-weight entries generally imply little to no similarity in trend or seasonality. Applying graph convolution directly to these low-weight entries can lead to inaccurate message passing and dilute the focus on the node of interest, thereby diminishing the effectiveness of graph convolution. To address this issue, the a-Entmax (Tezekbayev et al., 2021) is used, which incorporates a tunable hyperparameter a. This approach mitigates the influence of distant nodes while amplifying the impact of closer nodes. The o-Entmax function provides greater control over the normalized attention scores Z, which represent spatial correlations. The Q-Entmax defined as follows, where [x] max 2, 0:
a-Fntmax(z) = [(x 1)z r1]1/a 1/a-1
(9)
Here, z E Rd, and r IRd + RR is derived as:
Σ El(a I(a I(a- 1)2g -z( =1
(10)
The Q-Entmax function generalizes the Softmax (with = 1.0) and Sparsemax (Martins & Astudillo, 2016) (with = 2.0). offering enhanced flexibility compared to the Softmax function. In our experiments, we employ an a value of 2.0 to suppress the information flow from the noise nodes.
I. Full Forecasting Performance Comparison.
We present the full forecasting performance across multiple horizons the Nottingham and the ECL datasets in this section, including AGCRN and MTGNN. The best result for each evaluation is highlighted in bold, and the second-ber best result for each evaluation is underlined.
hnttps://maps: wttp.a/t......../astttttttttteee staeeerrrttttttttteee
htt.:/ppppttttttttiiiooooooeeeee html
16

---

FSTLLM: Spatio-Temports LLM for Few Shot Time Series Forecasting
Table 8. Performance comparison on Nottingham datasct.
15 mins RMSE
30 mins
45 mins
60 mins
MAE
MAPE
MAE
RMSE
МАРE
MAB
RMSE
MAPE
MAE
RMSE
MAPE
ARIMA
12.98
41.73
8.69%
21.16
63.63
14.98%
24.71
77.56
18.91%
28.49
82.47
23.60%
VAR
11.65
41.44
8.30%
27.70
67.06
16.88%
30.06
77.17
31.52%
32.19
91.85
49.00%
DCRNN
7.93
44.64
5.65%
13.21
61.94
9.76%
18.37
76.80
13.53%
24.10
89.67
20.68%
Graph Net
8.03
43.23
6.94%
13.07
61.60
10.30%
18.02
76.18
14.57%
23.05
88.71
19.89%
AGCRN
153.09
282.06
32.03%
151.12
281.27
30.72%
149.72
280.77
30.0%
149.37
280.89
30.88%
MTGNN
7.94
42.92
6.76%
234.26
314.58
281.20%
234.25
314.59
281.09%
234.25
314.62
281.00%
STSGCN
10.45
51.94
8.25%
15.28
67.03
12.65%
19.78
78.21
18.37%
24.12
87.59
23.78%
GMAN
42.09
102.41
29.01%
43.87
106.62
30.86%
45.53
110.03
33.11%
47.13
113.23
35.74%
GTS
42.79
5.54%
12.72
60.75
9.50%
17.80
75.08
13.51%
23.06
87.31
20.40%
PatchTST
12.54
48.42
12.93%
18.78
66.65
19.08%
25.03
81.57
24.06%
31.25
94.62
29.69%
iTranffommer
7.91
44.03
6.83%
13.12
63.43
10.33%
19.10
77.79
14.77%
26.09
91.80
22.66%
DLinear
7.98
46.28
6.80%
14.60
62.08
12.58%
21.10
77.02
16.97%
28.17
89.96
23.61%
GPT4TS
8.02
44.09
7.06%
13.16
62.51
10.79%
18.61
77.23
15.78%
24.24
90.00
21.55%
TimeLLM
10.55
47.49
8.45%
15.83
64.77
12.09%
22.06
79.48
16.99%
28.57
92.91
25.57%
FSTLLM
7.83
47.93
6.59%
11.53
60.11
9.47% %
15.31
71.59
12.31 %
19.09
80.90
17.76% %
75 mins RMSE
90 mins
105 mins
120 mins
MAPE
MAE RMSE
МAPE
MAE
RMSE
МАPE
MAE
RMSE
MAPE
ARIMA
33.95
91.98 30.67%
42.19
100.91
37.42%
51.34
123.13
53.74%
58.51
133.27
77.07%
VAR
37.72
91.36
39.51%
43.48
133.10
79.16%
48.73
143.58
64.04%
54.70
164.36
71.75%
DCRNN
30.02
101.37
26.64%
36.01
112.06
34.30%
41.99
122.06
41.96%
47.88
131.27
50.62%
Graph Nev
28.52
100.33 24.16%
34.12
110.26
33.22%
39.59
120.34
37.51%
45.16
128.57
47.50%
AGCRN
150.31
281.51 32.29%
152.16
282.45
35.29%
154.89
283.94
38.89%
157.49
285.15
43.43%
MTGNN
234.24
314.64 280.92%
234.24
314.66
280.85%
234.25
314.69
280.79%
234.26
314.72
280.75%
STSGCN
28.49
95.53
30.55%
32.69
102.47
37.57%
36.42
108.72
44.28%
39.87
114.03
52.04%
GMAN
48.65
116.01
38.06%
50.13
118.41 40.64% %
51.79
121.29
43.55%
53.54
124.20
46.00%
GTS
98.38
25.78%
33.87
108.43
33.17%
39.26
117.75
40.52%
44.45
126.22
49.03%
PatchTST
37.43
106.52
36.69%
43.54
117.45
44.59%
49.52
127.56
53.46%
55.37
136.90
63.83%
Transformer
31.39
102.89
28.82%
37.43
113.68
36.24%
43.32
123.99
43.25%
49.28
132.99
53.94%
DLinear
34.87
101.61
31.10%
42.09
112.45
37.87%
49.20
122.25
44.07%
56.17
131.31
52.37%
GPT4TS
30.20
101.68
29.05%
36.21
112.42
36.88%
42.20
122.36
42.54%
48.11
131.58
52.24%
TimeLLM
34.50
104.53
30.78%
40.18
115.20
38. 08.0
46.47
125.55
46.66%
53.58
136.11
58.96%
FSTLLM
23.18
89.67
21.80 %
26.65
95.82
26.51% %
30.43
102.88
32.38 %
33,89
109.54
39.09% %
17

---

FSTLLM: Spatio-Tempont LLM for Few Shot Time Series Forecasting
Table 9. Performance comparison on ECL dataset.
hour RMSE
hours
hours
4 hours
MAE
MAPE
MAE
RMSE
MAPE
MAE
RMSE
MAPE
MAE
RMSE
MAPE
ARIMA
2042
3774
17.64%
2959
5407
19.98%
4114
7044
32.78%
4104
6907
33.50%
VAR
1787
3682
15.40%
2512
5206
19.71%
2800
5271
26.26%
3568
8206
30.21%
DCRNN
2155
4694
19.05%
2448
4839
23.34%
2772
5110
27.92%
2947
5223
31.61%
GraphWaveNe
1497
3560
11.78%
2274
4659
17.91% %
2573
5120
22.22%
2890
5575
26.81%
AGCRN
16523
25513
98.80%
16524
25514
98.80%
16524
25514
98.79%
16524
25514
98.80%
MTGNN
1705
3578
15.23%
11527
19531
141.79%
11527
19531
141.79%
11527
19532
141.79%
STSGCN
3124
6719
22.15%
3092
6385
20.65%
3148
6328
21.07%
3007
6013
21.44%
GMAN
6181 10187
59.69%
6249
10332
60.52%
6313
10460
61.26%
6364
10576
61.80%
GTS
3517
6554
26.00%
3676
6716
26.43%
3827
6913
26.91%
3915
6969
27.59%
PatchTST
4789
8206
58.92%
5526
9370
72.17%
5955
10127
79.00%
6204
10761
85.70%
Transforrmer
1764
2947
21.76%
2289
4043
28.49%
2874
5073
36.86%
3399
5840
44.53%
DLinear
2701
4409
28.87%
3769
6023
38.86%
4029
6533
44.50%
4854
7790
54.45%
GPT4TS
1672
3527
15.41
2384
4759
24.34%
3009
5626
33.03%
3621
65588
42.88%
TimeLLM
4950
7640
61.74%
5319
8308
69.56%
5689
8722
75.36%
5906
9072
79.59%
FSTLLM
1663
3503
13.43%
2264
46077
16.37% %
2439
4938
16.65%
2520
5098
17.10%
hours
6 hours
hours
8 hours RMSE
MAE
RMSE MAPE
MAE
RMSE
MAPE
MAE
RMSE
MAPE
MAE
MAPE
ARIMA
4014
5912
31.62%
4122
6858
40.10%
4669
7054
43.48%
5102
7773
48.64%
VAR
3664
8582
35.44%
4066
9677
40.96%
4264
9812
47.27%
5291
11933
63.51%
DCRNN
3079
5298
33.82%
3178
5369
35.24%
3251
5446
36.14%
3301
5537
36.61%
GraphWaveNet
3153
5815
31.11%
3274
5844
35.06%
3364
5868
36.88%
3340
5806
37.99%
AGCRN
16525
25515
98.81%
16524
25514
98.81 %
16524
25514
98.79%
16523
25513
98.80%
MTGNN
11527 19532
141.79%
11527
19532
141.79%
11527
19532
141.79%
11527
19532
141.79%
STSGCN
3059 6462
22.04%
3061
6235
23.00%
3331
7086
28.40%
3041
6402
27.76%
GMAN
6412 10665
62.39%
6438
10721
62.53%
6444
10753
62.39%
6438
10761
62.25%
GTS
3936 6938
27.70%
3895
6838
27.50%
3798
6694
27.22%
3677
6561
26.75%
PatchTST
6564
11094
89.05%
6283
10977
87.91 %
5970
10491
81.81%
5617
9721
74.23%
Transformer
3656
6333
49.80%
3867
6536
53.33%
3687
6308
50.24%
3631
6246
48.18%
DLincar
5978
9081
62.23%
5242
8196
61.10%
5259
8138
61.33%
4989
7739
57.50%
GPT4TS
4133
7277
53.54%
4420
7514
60.10%
4445
7437
60.31%
4187
7099
55.27%
TimeLLM
6112
9308
82.43%
6121
9358
83.02%
6118
9393
83.32%
6071
9439
81.93%
FSTLLM
2584
5201
17.56%
2648
5279
18.17% %
2688
5333
18.33%
2760
5412
18.81% %
9 hours
10 hours RMSE
11 hours RMSE
12 hours RMSE
MAE
RMSE
MAPE
MAE
MAPE
MAE
MAPE
MAE
MAPE
ARIMA
4730
7792
46.96%
5322
8499
51.73%
5130
8304
47.66%
5352
8455
48.49%
VAR
5500
12319
67.96%
5287
12864
65.76%
5424
13536
64.49%
6397
15368
58.73%
DCRNN
3293
5618
36.24%
3175
5544
34.27%
2996
5410
31.12%
2796
5314
27.59%
GraphWaveNed
3267
5816
36.24%
3090
5663
33.90%
2895
5448
30.51%
2652
5260
26.75%
AGCRN
16522
25512
98.81%
16519
25510
98.80%
16518
25508
98.79%
16516
25506
98.80%
MTGNN
11527 19531
141.81%
11527
19530
141.84%
11527
19530
141.88%
11527
19529
141.91%
STSGCN
2966 5639
24.05%
3136
7345
29.23%
3099
7346
28.58%
3205
7784
30.55%
GMAN
6421 10742
61.58%
6381
10681
60.66%
6327
10607
59.67%
6260
10499
58.79%
GTS
3567 6484
26.07%
3485
6476
25.28%
3425
6528
24.53%
3361
6606
23.80%
PatchTST
5046
8623
63.58%
4283
7351
51.63%
3501
5953
40.11%
2757
4537
28.15%
Traanforrmr
3130
5362
37.98%
2890
4750
34.24%
2988
4360
29.50%
1979
3562
19.79%
DLinear
7298 11398
62.10%
3947
6335
43.09%
4830
7488
40.11%
2803
4770
28.93%
GPT4TS
3692
6365
44.80%
3107
5802
34.54%
2461
4578
25.08%
1839
3597
17.00 %
TimeLLM
5917
9060
78.78%
5751
8727
73.46%
5397
8204
67.24%
4976
7613
60.40%
FSTLLM
2822
5519
19.44%
2860
5460
19.80 %
2894
5533
20.04% %
2901
5478
20.37%
18


Reviews:
[{"review_id": "0ZDKIO5cS5", "reviewer": "Reviewer_jV37", "summary": "This paper proposes a time-series prediction framework that leverages the prior knowledge of LLMs. Based on Author’s introduction about the framework, the framework can be flexibly applied to any advanced time-series prediction model (such as the STGNNs mentioned in the related works). The experiments were conducted on two datasets—one public and the other self-collected (and will be made public later). The results show that the framework is effective in boosting the state-of-the-art. Overall, this is a novel and effective framework. Although it comes with a certain level of computational cost, it is tolerable, as it only LORA fine-tuning a LLM on a single A6000 GPU. The weaknesses are discussed in the following sections.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer,\n\nWe sincerely thank you for your valuable feedback. We address your comments in detail below and will revise the manuscript accordingly:\n\n**Q1. Citation of In-Context Learning**\n\nWe appreciate your suggestion regarding the citation of in-context learning. We will revise the manuscript by incorporating appropriate references in lines 57–60, as follows:\n\n> “In contrast, Large Language Models (LLMs) demonstrate strong capabilities in common sense reasoning (Zhao et al., 2023), making them particularly effective for integrating domain-specific and contextual knowledge via in-context learning (Min et al., 2022), in addition to fine-tuning. Furthermore, LLMs exhibit robust performance in few-shot and zero-shot learning scenarios, which are highly relevant for data-scarce forecasting tasks.”\n\n**Q2. Clarification of Alpha-Entmax Value**\n\nThank you for pointing out the need for clarity regarding the α value in the alpha-entmax transformation. As noted in **Appendix D (Implementation Details)**, we employ an α value of 2.0 in our design. To enhance visibility, we will explicitly state this choice again in **Appendix F (The Alpha-Entmax Function)**, along with a brief justification for its selection.\n\n**Q3. Node Description and Pattern Analysis Methodology**\n\nWe appreciate your interest in the generation of node descriptions and pattern analysis. We utilized ChatGPT-4o to assist in these tasks, and we will provide the prompt templates used in the Appendix to ensure reproducibility. Specifically:\n\n* Node Description Prompt:\n\n>“You are given a feature description from [carpark description link] and user reviews from [Google review link]. Please synthesize an inductive description of the carpark using content from both sources.”\n\n* Node Pattern Analysis Prompt:\n\n>“I will provide one week of parking lot records for a carpark in England, recorded every 15 minutes starting from 12:01 PM on 2016-10-26. Please describe the temporal usage patterns of the carpark, including variations throughout the day and across weekdays versus weekends. Indicate observed peak and low-demand periods, along with any consistent behavioral trends in parking availability. The records are: [extracted training data].”\n\nWe will incorporate both prompts into the appendix to ensure methodological transparency.\n\nPlease let us know if further clarification is needed. Thank you once again for your thoughtful and constructive comments.", "claims_and_evidence": "The main claim of the paper is that LLMs contain reasoning ability and prior knowledge about locations, which can help with time-series modelling. This claim is based on the widely accepted understanding that LLMs aim to model the world and encode a vast number of everyday concepts and relationships in their hidden states. The authors extract features based on this knowledge. In the experiments, they report the effectiveness of the overall framework, along with ablation studies for each module. The evidence is convincing.", "methods_and_evaluation_criteria": "My simplified understanding of the method is: LLM knowledge is introduced before the input of STGNN, and in-context learning is applied on the output of STGNN, and STGNN serves as the backbone of the framework to achieve robust forecasting performance and reasoning to end users.\n\nThe evaluation is primarily based on time-series prediction accuracy compared to previous representative methods. The use of MAE, RMSE, and MAPE are standard, and there is nothing additional to comment on.", "theoretical_claims": "This paper does not focus on theoretical proofs.", "experimental_designs_or_analyses": "The authors use one public dataset and one self-collected dataset, with the latter also submitted for review (Nottingham.h5).\n\nThere are no obvious issues; the settings (such as evaluate a 15/45/60-mins window) are commonly used in the literature.", "relation_to_broader_scientific_literature": "The paper primarily cites relevant works in time-series forecasting and large language models, such as TimeLLM and GPT4TS.\n\nThe baselines are up-to-date.", "essential_references_not_discussed": "The paper lacks sufficient citations related to LLM in-context learning.\nIn-context learning in LLMs is likely beneficial to the model, as seen from their instruction prompt design. The method requires some preliminary predictions incorporated into the prompts, allowing the LLM to refine and improve them further.\n\nI suggest they cite the following paper to help readers better understand and utilize in-context learning:\n\nMin, S., Lyu, X., Holtzman, A., Artetxe, M., Lewis, M., Hajishirzi, H., & Zettlemoyer, L. (2022). Rethinking the role of demonstrations: What makes in-context learning work?. arXiv preprint arXiv:2202.12837.", "other_strengths_and_weaknesses": "Strengths: Already discussed in the previous sections. The notation is consistent, the experimental results are useful, and the proposed method is sufficiently novel.\n\nWeaknesses: \n- The description of Alpha-max is unclear. A value of 1 corresponds to standard SoftMax, while 2 represents a more concentrated distribution.  Is the chosen value <1, 1- 2 or >2?\n- The authors need to explicitly describe the role and numerical value of this hyperparameter, as it appears to be important.\n- The ways to prepare the node description and node pattern analysis are unclear. The authors need to explain more on this preparation, whether they are directly collected from website or specific data engineering operations needed.", "other_comments_or_suggestions": "Typo corrections: \n\nIn the abstract: “methodologies is” -> “methodologies are”\n\nThe authors should proofread the entire paper again.", "questions_for_authors": "Please clarify the issue regarding the Alpha-max parameter and node description preparation mentioned above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ShvLwKhadr", "reviewer": "Reviewer_vnyj", "summary": "Considering the heavy time cost to collect data for training deep learning time series forecasting models, this work focuses on enhancing forecasting performance with limited training data. This paper propose a graph construction module to ensure stable graph construction used in Spatio-Temporal Graph Neural Networks (STGNNs). Furthermore, it proposed an LLM fine-tuning methodology to enhance forecasting performance with LLM’s embedded knowledge and common-sense reasoning ability. Finally, experiments on two real-world datasets demonstrate this method’s solid and robust performance.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer\n\nWe sincerely thank the reviewer for the constructive feedback. We address each of the comments below and will update the manuscript accordingly.\n\n**Q1: Discussion of AutoTimes**\n\nThank you for the suggestion. We will add the following discussion to Section 2.3 (Related Work):\n\n>\"AutoTimes (Liu et al., 2024) repurposes decoder-only large language models (LLMs) for autoregressive time series forecasting by mapping time series inputs into the embedding space of language tokens. This method leverages the sequential modeling strength of LLMs to generate variable-length future predictions without updating the LLM weights.\"\n\nWe acknowledge its relevance and will highlight the distinction between our fine-tuning-based approach and AutoTimes’ frozen-weight generation scheme.\n\n**Q2: 3-Day Few-Shot Forecasting Experiment on Nottingham Dataset**\n\nDue to time constraints, we performed a 3-day training experiment using the Nottingham dataset and a representative subset of baselines. The results are presented below:\n\n| Method     | MAE   | RMSE  | MAPE (%) |\n|------------|-------|--------|-----------|\n| FSTLLM     | 22.84 | 83.68  | 22.33     |\n| GTS        | 29.57 | 87.44  | 26.04     |\n| GPT4TS     | 33.24 | 93.92  | 26.95     |\n| PatchTST   | 34.65 | 97.45  | 32.38     |\n| DLinear    | 37.52 | 95.87  | 30.62     |\n\nThese results demonstrate that FSTLLM consistently outperforms strong baselines under limited data scenarios. This further validates the robustness and adaptability of our framework in few-shot settings.\n\n**Q3: Clarification on Integration Experiment Setup**\n\nThank you for pointing this out. We will revise lines 407–410 in Section 4.4 (Few-Shot Learning Integration Study) to provide a clearer explanation:\n\n>“To assess the plug-and-play potential of our LLM-enhanced graph construction, we substitute the numerical prediction tokens generated by the STGNN backbone with those from various transformer-based forecasting models. Specifically, we remove both the LLM-enhanced graph construction module and the STGNN backbone, and replace them with external transformer-based methods without modifying or retraining those models. This allows us to evaluate the generality of our integration strategy.”\n\nPlease let us know if any further clarification is needed. We appreciate your detailed review and helpful suggestions.", "claims_and_evidence": "This paper is application-driven, and claims are supported by experimental results. \n- Claim 1: The Proposed FSTLLM achieves enhanced few-shot time series forecasting performance. Evidence 1: Supported by experimental results in Table 1 and Table 2. \n- Claim 2: The LLM Enhanced Graph Construction module is able to enhance graph construction by embedding contextual information of spatial nodes. Evidence 2: Supported by experimental results in Table 3 from ablation study. \n- Claim 3: The Domain Knowledge Injection module enables humanlike consideration and few-shot forecasting performance enhancement. Evidence 3: Supported by experimental results in Table 3 from ablation study and case study visualization in reasoning subsection, section 4.2 Performance of LLM. \n- Claim 4: FSTLLM can augment various time series forecasting models, enhancing their performance in few-shot set- tings without updating their parameters. Evidence 4: Supported by experimental results in Table 4 from section 4.4 Few-Shot Learning Integration Study.", "methods_and_evaluation_criteria": "There are detailed running examples and complexity comparison covering 12 baselines. 2 datasets. 3 metrics. The selection of baselines aligns with domain of time series forecasting. The evaluation criteria, MAE, RMSE, and MAPE are standard evaluation criteria for time series forecasting tasks.", "theoretical_claims": "The equations are correct and align with the code submitted. Proposed prompt structure align with the real use case shown in Appendix.", "experimental_designs_or_analyses": "The design of the experiments is clear and comprehensive and support FSTLLM’s claims. Details as below. One experimental setting can be discussed to enhance the quality of this work: though using 7 days of data to simulate limited data situations is commonly used, some works also include 3 days of data for comparison as well, such as used in TransGTR (KDD 2023) and CrossTReS (KDD 2022).", "relation_to_broader_scientific_literature": "The paper contributes to the growing body of research on improving time series forecasting in data-scarce environments compared to methods requiring massive training data such as STGNNs (GraphWaveNet, GTS, STSGCN), Transformer-based methods (PatchTST, iTransformer), and LLM methods (Time-LLM, GPT4TS).", "essential_references_not_discussed": "As far as I know, this work includes the majority of research works related to few-shot time series forecasting from STGNNs, Transformer-based methods, and LLM-based methods. One paper omitted in this work could be ‘AutoTimes: Autoregressive Time Series Forecasters via Large Language Models’ published in NIPPS 2024, which adapts LLM for time series forecasting tasks.", "other_strengths_and_weaknesses": "Strength.\n+ This paper is well written and clear. \n+ Extensive experiments, including abundant baselines, real-world datasets, and comprehensive exploration of model performance have been conducted to show the effectiveness of FSTLLM.\n\nWeakness.\n- Discussing on experiment setting of 3 days data for comparison as well such as used in TransGTR (KDD 2023) and CrossTReS (KDD 2022). \n- Explanation more on integration with existing methods, for instance, which exact components are removed and replaced by the existing methods. Does existing methods need to be re-train in order to suit FSTLLM’s framework.", "other_comments_or_suggestions": "line 80: \"that can commonly used in time series forecasting\" → \"that are commonly used in time series forecasting\" \nline 163: \"After this, due to the limited training data, the numerical prediction tokens is still suboptimal in capturing temporal dynamics.\" \"is\" → \"are\"", "questions_for_authors": "In general, this is an innovative and solid work focus on few-shot time series forecasting task. Serval improvements can be achieved to improve the quality of this paper. These improvements include discussing AutoTimes, discussing experimental settings of using 3 days data to simulate a lack of data, and further explanation on integration experimental details.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mpby017S9O", "reviewer": "Reviewer_WS2i", "summary": "This work introduces a framework called FSTLLM. This framework provides enhanced few-shot time series forecasting performance by integrating LLMs with the STGNN backbone. Specifically, it leverages LLMs for spatial correlation modeling, an STGNN network for spatio-temporal pattern modeling, and a domain knowledge injection module for improved predictions. FSTLLM outperforms state-of-the-art baselines, highlighting superior accuracy and robustness in real-world datasets.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear  Reviewer,\n\nWe sincerely thank the reviewer for the insightful and constructive feedback. Below, we address each point in detail and outline the corresponding changes we will make in the revised manuscript.\n\n**Q1: Missing References — DUET and Time-MoE**\n\nWe appreciate the reviewer’s suggestion and will incorporate both DUET and Time-MoE into our related work discussion:\n\nDUET will be added to Section 2.1 (Classical Neural Network-based Methods), lines 90–93:\n\n>“DUET (Qiu et al., 2025) introduces a framework to tackle multivariate time series forecasting with heterogeneous temporal patterns and complex inter-channel dependencies. It features a Temporal Clustering Module (TCM) for handling temporal heterogeneity, and a Channel Clustering Module (CCM) that applies a novel channel-soft-clustering mechanism in the frequency domain to model inter-channel relationships while mitigating noise.”\n\nTime-MoE will be discussed in Section 2.3 (Large Language Models), lines 126–130:\n\n>“Time-MoE (Shi et al., 2025) presents a scalable architecture for time series forecasting using a sparse mixture-of-experts design to enhance efficiency while maintaining high model capacity. Trained on the extensive Time-300B dataset (over 300 billion time points across nine domains), Time-MoE scales up to 2.4 billion parameters and achieves strong forecasting accuracy. However, its expert routing mechanism is less effective in few-shot scenarios, limiting performance under data-constrained settings.”\n\n**Q2: Clarification of “Candidate Node Embedding”**\n\nThank you for pointing this out. The term refers to the embedding of a specific candidate node in the graph structure. We will revise the corresponding sentence to reader:\n\n>“... the embedding of a candidate node selected from the spatio-temporal graph.”\n\nThis revision will improve clarity and remove ambiguity.\n\n**Q3: Integration Experiment Setup**\n\nWe agree that further clarification is needed. We will revise Section 4.4 (Few-Shot Learning Integration Study), lines 407–410, as follows:\n\n>“To assess the plug-and-play potential of our LLM-enhanced graph construction, we substitute the numerical prediction tokens generated by the STGNN backbone with those from various transformer-based forecasting models. Specifically, we remove both the LLM-enhanced graph construction module and the STGNN backbone, and replace them with external transformer-based methods without modifying or retraining those models. This allows us to evaluate the generality of our integration strategy.”\n\nThis ensures the experiment strictly evaluates modular compatibility in a plug-and-play fashion.\n\n**Q4: Evidence of Data Efficiency**\n\nThank you for highlighting this important point. To evaluate data efficiency, we compare FSTLLM trained on just 3 days of data with baselines trained on 30 days of data, using the Nottingham dataset. The results are summarized below:\n\n| Method            | MAE   | RMSE   | MAPE (%) |\n|-------------------|-------|--------|-----------|\n| FSTLLM (3 days)   | 22.84 | 83.68  | 22.33     |\n| GTS (30 days)     | 23.54 | 90.88  | 23.02     |\n| GPT4TS (30 days)  | 27.50 | 86.88  | 25.70     |\n| PatchTST (30 days)| 33.07 | 95.20  | 30.13     |\n| DLinear (30 days) | 32.82 | 92.64  | 26.63     |\n\nThese results show that FSTLLM with only 3 days of training data outperforms all baselines trained with 10× more data, highlighting the strong data efficiency of our approach. This table will be added to the Appendix in the revised version.\n\nWe appreciate the reviewer’s detailed feedback and believe that the above revisions will substantially improve the clarity and completeness of our submission.", "claims_and_evidence": "The claims in the paper are generally supported by experimental results and case studies.", "methods_and_evaluation_criteria": "Yes. The proposed methods and evaluation criteria are well-suited for few-shot time series forecasting. FSTLLM integrates LLMs for spatial correlation modeling, an STGNN for spatio-temporal pattern modeling, and a domain knowledge injection module for improved predictions. The model is evaluated on real-world datasets (Nottingham Parking and ECL) using standard metrics (MAE, RMSE, MAPE), with a few-shot setup to test its adaptability.", "theoretical_claims": "Yes, the mathematical equations and the claims in the paper appear correct. There are no particular theoretical proofs in the paper.", "experimental_designs_or_analyses": "The experimental design and analyses in FSTLLM are methodologically sound, with well-defined datasets, 12 baselines, and standard evaluation metrics (MAE, RMSE, MAPE). The few-shot setting is realistic, using two real-world datasets (Nottingham Parking & ECL) to test generalizability. The analysis demonstrates consistent performance improvements, supported by ablation studies (assessing model components) and a few-shot integration study, showing FSTLLM’s ability to enhance other forecasting models like GPT4TS and iTransformer.", "relation_to_broader_scientific_literature": "FSTLLM advances spatio-temporal forecasting by leveraging LLMs to enhance spatial correlation modeling, surpassing traditional STGNNs. Unlike existing LLM-based models such as GPT4TS (NeurIPS 2023) and TimeLLM (ICLR 2024), it integrates node-specific domain knowledge for more context-aware predictions, leading to improved forecasting performance. By utilizing LLMs’ few-shot capabilities, FSTLLM enhances forecasting in data-limited settings while also providing reasoning ability, distinguishing it from classical STGNNs like GTS (ICLR 2021) and STEP (KDD 2022). This work introduces a novel approach that combines LLMs with an STGNN backbone rather than relying on a single architecture. Researchers working on other time series tasks like imputation and classification can use this method as a foundation for integrating LLMs with other models.", "essential_references_not_discussed": "Two important work are missing.\n\n[1] Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts, ICLR 2025.\n\n[2] DUET: Dual Clustering Enhanced Multivariate Time Series Forecasting, KDD 2025.", "other_strengths_and_weaknesses": "S1. The problem studied in this work (few-shot time series forecasting) is an important field given real-world applications, where data collection is often limited.\n\nS2. The paper is well-written and easy to follow. The experimental analysis illustrates that the proposed FSTLLM is superior to the existing methods.\n\nW1. Terminology Clarity: The term \"candidate node embedding\" in Section 3.1 (lines 205-206) is unclear. It would help to specify whether it refers to another node in the graph or an alternative embedding technique.\n\nW2. Few-Shot Integration Details: Section 4.4 claims that \"we replace the numerical prediction tokens generated by the STGNN backbone with those produced by alternative transformer-based methods.\" However, implementation details are not clearly specified—particularly whether the LLM-Enhanced Graph Construction Module was considered or omitted, given that GPT4TS does not model multivariate correlations while other transformer-based methods do.\n\nW3. Data Efficiency Evidence: The introduction claims that \"these models typically require large volumes of training data...collecting such data is time-consuming and resource-intensive.\" However, the experimental results primarily focus on forecasting accuracy rather than explicitly demonstrating FSTLLM’s data efficiency. Additional experiments comparing performance with varying data availability could reinforce this claim.", "other_comments_or_suggestions": "Typo in the Introduction section, line 36: Missing comma before \"offer mechanisms to jointly.\"\n\nTypo in the Methods section, line 163: \"The numerical prediction tokens is still suboptimal\" should be \"The numerical prediction tokens are still suboptimal.\"", "questions_for_authors": "I will consider changing my evaluation of the paper based on the author’s response to the following concerns.\n1. Discussing Time-MoE in this paper on a few-shot time series forecasting task.\n2. Clearly explain what \"candidate node embedding\" refers to, as listed in W1.\n3. Explain the implementation in detail in section 4.4, the Few-Shot Learning Integration Study listed in W2.\n4. Discuss and provide experimental results to demonstrate FSTLLM’s data-efficient advantage as listed in W3.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "u1AqM505d3", "reviewer": "Reviewer_KEzm", "summary": "The paper presents FSTLLM, a novel Spatio-Temporal Large Language Model (LLM) framework designed for few-shot time series forecasting. The model effectively integrates domain knowledge through a fine-tuned LLM and a graph-based learning approach to capture spatial-temporal correlations. Experimental results on real-world datasets demonstrate superior performance compared to existing baselines.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Response to Reviewer\n\nWe thank the reviewer for the thoughtful comments. We address each point below and will incorporate the corresponding revisions into the manuscript.\n\n**Q1: Node Description, Pattern Analysis, and Case Study**\n\nWe used ChatGPT-4o to generate both node descriptions and pattern analyses. To ensure reproducibility, we will include the prompt templates in the Appendix. Specifically:\n\n* Node Description Prompt:\n\n>“You are given a feature description from [carpark description link] and user reviews for this carpark at [Google review link]. Write an inductive summary about the carpark based on contents from both sources.”\n\n* Node Pattern Analysis Prompt:\n\n> “I will provide one week of parking lot records for a carpark in England. This record is collected every 15 minutes starting from 12:01 PM on 2016-10-26. Please describe the natural pattern of parking availability, including diurnal and weekly variations, and highlight peak and dip periods observed in the data. The records are: [extracted training data].”\n\nAdditionally, we have conducted a full fine-tuning and inference case study on the ECL dataset. Due to space limitations in this response, the full results and analysis will be included in the Appendix of the revised manuscript.\n\n**Q2: Discussion of \"News2Forecast\" (Wang et al., 2024, NeurIPS)**\n\nWe will include a discussion of this work in Section 2.3 Related Work, and revise the manuscript as follows:\n\n>“News2Forecast (Wang et al., 2024) enhances time series forecasting by integrating social events through LLM-based agents using reflection and reasoning. It fine-tunes a pre-trained LLM to align textual and numerical data, thereby improving forecasting accuracy.”\n\nWhile this work focuses on aligning news events with time series data, it does not account for spatial dependencies across different time series. Furthermore, it may suffer from reduced robustness when training samples are limited or when relevant external textual content from news is unavailable. In contrast, FSTLLM explicitly models spatial-temporal dependencies and is designed to generalize under limited supervision.\n\n**Q3: Rationale for Fine-Tuning vs. Direct Prompting with Powerful LLMs**\n\nWe appreciate this important question. The motivation for fine-tuning instead of relying solely on inference from powerful LLMs is twofold:\n\n* **Modeling Temporal Dynamics:** Off-the-shelf LLMs (e.g., GPT-4o, DeepSeek R1), when used via prompting, do not effectively model the complex temporal dependencies inherent in time series data. Their predictions tend to be shallow, often yielding weighted approximations based on two provided historical input and numerical prediction, rather than learning temporal patterns explicitly.\n\n* **Learning Temporal Representations via Weight Updates:** Fine-tuning enables the model to update its internal weights, allowing it to capture nuanced dynamics and structure within the time series. This results in significantly improved forecasting performance compared to inference-only approaches.\n\nAs evidenced in prior work, fine-tuned models such as GPT4TS and Time-LLM consistently outperform inference-based models like PromptCAST, further justifying our design choice.\n\nPlease let us know if further clarification is needed. We are grateful for the reviewer’s constructive feedback.", "claims_and_evidence": "The claim that \"The model effectively integrates domain knowledge through a fine-tuned LLM and a graph-based learning approach to capture spatio-temporal correlations\" is well-supported by experimental results on real-world datasets. Additionally, the case study effectively demonstrates FSTLLM’s reasoning ability derived from domain knowledge.", "methods_and_evaluation_criteria": "The proposed method primarily applies a fine-tuned LLM to enhance baseline methods such as STGNNs. The evaluation criteria and settings are consistent with existing works. Both the methodology and evaluation framework are appropriate for the problem at hand.", "theoretical_claims": "I have checked equations 1 – 8, and they are correct and align with the submitted code.", "experimental_designs_or_analyses": "The experimental design is sound. Specifically, FSTLLM significantly enhances few-shot forecasting performance of STGNNs while also incorporating reasoning ability, which is crucial for end users. However, some points need to be addressed:\n1. The paper does not clearly explain how FSTLLM crafts its node descriptions and node pattern analyses, as introduced in Section 3.3 (Domain Knowledge Injection Module).\n2. The study presents fine-tuning and inference case study results only on the Nottingham dataset. Including a demonstration on the ECL dataset would provide a more comprehensive understanding of the model's applicability.", "relation_to_broader_scientific_literature": "This work builds upon recent advancements in using LLMs for forecasting tasks, such as GPT4TS and Time-LLM. While these studies demonstrate the potential of LLMs in processing time-series data, they primarily focus on fine-tuning LLMs with numerical input. FSTLLM advances this line of research by incorporating domain knowledge through structured prompt engineering combined with an STGNN backbone, making LLMs more context-aware for time-series forecasting.", "essential_references_not_discussed": "The paper comprehensively covers relevant literature, but I suggest including the following recent work: ‘From News to Forecast: Integrating Event Analysis in LLM-Based Time Series Forecasting with Reflection’ (NeurIPS 2024). This study also fine-tunes an LLM in a textual format to address time-series forecasting challenges.", "other_strengths_and_weaknesses": "Strength: \n1. The paper introduces FSTLLM, a novel framework that effectively integrates LLMs with Spatio-Temporal Graph Neural Networks (STGNNs) to enhance time-series forecasting.\n2. Experimental results demonstrate that FSTLLM consistently outperforms state-of-the-art baselines across multiple forecasting horizons on real-world datasets.\n3. The adaptable architecture of FSTLLM allows integration with existing time-series forecasting models.\n\nWeakness: \n1. As mentioned in the ‘Experimental Designs Or Analyses’ section, it is important to clarify how node descriptions and patterns are derived and to include demonstrations on both the Nottingham and ECL datasets.\n2. A comparison between FSTLLM and the reference paper mentioned in ‘Essential References Not Discussed’ would strengthen the discussion.\n3. The rationale for fine-tuning an LLM remains unclear. Given that all domain knowledge is embedded in the prompt, why not leverage powerful LLMs such as GPT-4o or DeepSeek R1 directly for inference-based forecasting enhancement?", "other_comments_or_suggestions": "1. Page 1, Lines 11–14: \"The fundamental of time series forecasting methodologies\" → Should be \"The fundamentals of time series forecasting methodologies\".\n2. Page 1, Lines 46–47: \"Accurate forecasting requires precise modeling on two dimensions\" → Should be \"Accurate forecasting requires precise modeling of two dimensions\".\n3. Page 4, Lines 206–210: \"these textual data\" → Should be \"this textual data\".", "questions_for_authors": "Please address the three weaknesses listed in the ‘Other Strengths And Weaknesses’ section and all typos listed in ‘Other Comments Or Suggestions’.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The reviewers all consider that the problem studied in this paper is useful, and the proposed method is also novel. The paper also provides convicing experimental results, so I personally recommend and acceptance.
Decision: Accept (poster)