Context-Alignment: Activating and Enhancing LLMs Capabilities in Time Series
Authors: Yuxiao Hu
Qian Li
Dongxiao Zhang
Jinyue Yan
Yuntian Chen
Venue: ICLR 2025 Poster (2025)
Keywords: Time Series
Large Language Models
Context-Alignment
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: LLMs for time series tasks

Abstract: Recently, leveraging pre-trained Large Language Models (LLMs) for time series (TS) tasks has gained increasing attention, which involves activating and enhancing LLMs' capabilities. Many methods aim to activate LLMs' capabilities based on token-level alignment, but overlook LLMs' inherent strength in natural language processing — their deep understanding of linguistic logic and structure rather than superficial embedding processing. We propose Context-Alignment (CA), a new paradigm that aligns TS with a linguistic component in the language environments familiar to LLMs to enable LLMs to contextualize and comprehend TS data, thereby activating their capabilities. Specifically, such context-level alignment comprises structural alignment and logical alignment, which is achieved by Dual-Scale Context-Alignment GNNs (DSCA-GNNs) applied to TS-language multimodal inputs. Structural alignment utilizes dual-scale nodes to describe hierarchical structure in TS-language, enabling LLMs to treat long TS data as a whole linguistic component while preserving intrinsic token features. Logical alignment uses directed edges to guide logical relationships, ensuring coherence in the contextual semantics. Following the DSCA-GNNs framework, we propose an instantiation method of CA, termed Few-Shot prompting Context-Alignment (FSCA), to enhance the capabilities of pre-trained LLMs in handling TS tasks. FSCA can be flexibly and repeatedly integrated into various layers of pre-trained LLMs to improve awareness of logic and structure, thereby enhancing performance. Extensive experiments show the effectiveness of FSCA and the importance of Context-Alignment across tasks, particularly in few-shot and zero-shot forecasting, confirming that Context-Alignment provides powerful prior knowledge on context. The code is open-sourced at https://github.com/tokaka22/ICLR25-FSCA.

Published as a conference paper at ICLR 2025
CONTEXT-ALIGNMENT ACTIVATING AND ENHANC- ING LLM CAPABILITIES IN TIME SERIES
Yuxiao Hu Qian Li Dongxiao Zhang", Jinyue Yan', Yuntian Chen
The Hong Kong Polytechnic University, Hong Kong, China Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo, China *Shanghai Jiao Tong University, Shanghai, China
ZZejjaang Key Laboratory of Industrial Intelligence and Digital Twin, Eastern Institute of Technology. Ningbo, China
0 ucas. ac en, qian1 situsjjuuuuuu... dzhang@eias ac. cn, j-jerry. yanepol edu. hk, ycheneeitech.co edu.
ABSTRACT
Recently, leveraging pre-trained Large Language Models (LLMs) for time ries (TS) tasks has gained increasing attention, which involves activating and enhancing LLMs' capabilities. Many methods aim activate LLMs' capabil- ities based on token-level aligmment, but overlook LLMs' inherent strength natural language processing their deep understanding of linguistic logic and structure rather than superficial embedding processing. We propose Context- Alignment (CA), new paradigm that aligns TS with a linguistic component in the language environments familiar to LLMs to enable LLMs to contextual- ize and comprehend TS data, thereby activating their capabilities. Specifically, such context-level alignment comprises structural alignment and logical align- ment, which achieved by Dual-Scale Context-Alignment GNNs (DSCA-GNNSs) applied to TS- IS-language multimodal inputs. Structural alignment utilizes dual- scale nodes to describe hierarchical structure in IS-language, enabling LLMs to treat long TS data whole linguistic component while preserving intrin- sic token features. Logical alignment uses directed edges to guide logical rela- tionships, ensuring coherence the contextual semantics. Following the DSCA- GNNs framework, we propose an instantiation method of CA, termed Few-Shot prompting Context-Alignment (FSCA), enhance the capabilities of pre-trained LLMs in handling TS tasks. FSCA can be flcxibly and repeatedly integrated into various layers of pre-trained LLMs improve awareness logic and struc- ture, thereby enhancing performance. Extensive experiments show the effective- ness of FSCA and the importance of Context-Alignment across tasks, particu- larly in few-shot and zero-shot forecasting, confirming that Context-Alignmen provides powerful prior knowledge on context. The code open-sourced https://ggtubbcooo/ookkk////C/1s
INTRODUCTION
Time series (TS) tasks are essential in many real-world applications, from weather prediction to navigation optimization. The field has shifted from traditional statistical methods to advanced neu- ral architectures like Recurrent Neural Networks (RNNs), Convolutional Neural Networks (CNNs), and Transformers, improving the handling of complex dependencies. However, challenges in gen- eralizing across diverse datasets and adapting to various TS tasks remain.
Recently, large language models (LLMs) have excelled in various fields, primarily due to their ex- tensive and diverse text corpora training dataset. With rich foundation of multi-domain knowledge,
Alphabetical ordering: The first two authors contributed equally.
Corresponding author

---

Published as a conference paper at ICLR 2025
LLMs achieve impressive generalization in downstream tasks. Thus, there is growing interest in utilizing pre-trained LLMs to solve TS problems. However, distinct differences between the train- ing data of LLMs and TS data have hindered LLMs' full potential in TS applications. To effectively utilize L.Ms in TS tasks, two main issues must be addressed in turn:
1) How to make LLMs understand TS data and activate their capabilities in TS tasks? 2) How to enhance the performance of LLMs on TS tasks?
Regarding the first issue, existing works primarily focus aligning TS token embeddings with lan- guage token embeddings (Jin et al. 2024 Sun et al. token-level alignment can fully leverage the LLMS potential remains questionable. Inspired by re- cent research on LLMs (Ethayarajh) 2019 Nie et al.
2024 Pan et al. 2024). However, whether such
2024 Wang et al. 2023), we reconsider the in- herent advantages of LLMs natural language processing (NLP). We believe the strength of LLMs primarily stems from their deep comprehension of language logic and structure, rather than super- ficial token embedding processing. Clearly, the excessive accumulation of tokens without logical guidance often struggles to effectively convey meaning. Especially TS- TS-language multimodal inputs are lengthy, and lack structure and coherent semanties, greatly challenging LLMs' comprehension. Regarding the second issue, current methods aim to directly enhance LLMs' capabilities in TS tasks through techniques such as TS decomposition (Cao et al./ 2024) and optimizing prompts (Chuang et al., 2024). However, without adequately addressing the first issue, these methods need more terpretability and the improvements remain limited. A natural solution to these issues is to fully leverage the strengths of LLMs transform TS tasks into NLP-like tasks, activating LLMs' capa- bilities first. Then, leveraging NLP techniques further enhances LLMs' performance on TS tasks.
In this paper, we propose Context-Alignment component in the language environment familiar with LLMs. Such context-level alignment lever- ages the LLMs' inherent strength in logic and structure to enable LLMs to contextualize and com- prehend TS data, thereby activating their capabilities. ment and logical alignment to construct a consistent context for TS-language multimodal inputs. We develop a Dual-Scale Context-Alignmen Graph Neural Networks (DSCA-GNNs) framework to achieve both structural and logical alignment. Specifically, structural alignment employs dual-seale nodes to describe hierarchical structure in TS-language, i.e. the structural independence of tokens and the overall structure of modalities. Structural alignment provides LLMs with structural seg- mentation information for lengthy TS language inputs, enabling LLMs to treat long TS data as an individual linguistic component while preserving intrinsic token features. Logical aligument uses directed edges in both scale GNNs to guide the local and global logical relationship between TS data and language prompts, integrating TS within the language environment and ensuring semantic coherence across two modalities. Utilizing the few-shot prompting technique (Brown) 2020), propose Few-Shot prompting based Context-Alignmeen (FSCA) following the DSCA-GNNSs frame- work, which further enhances the LLMs' performance on TS tasks. FSCA can be flexibly and re- peatedly integrated into various layers of pre-trained LLMs to improve awareness of logic and struc- ture. Extensive experiments across various TS tasks demonstrate the effectiveness of our method. Notably, in few-shot and zero-shot forecasting tasks, our approach significantly outperforms others, confirming that the logical and structural alignment provides powerful prior knowledge on context. Ablation studies further validate the importance of Context-Allinneet
new paradigm that aligns TS data with a linguistic
Context-Aligmmeet contains structural align-
In summary, our core contributions can be summarized below:
We emphasize that effectively leveraging LLMs for TS tasks requires first activating their capa- bilities and then enhancing them. Besides, we pinpoint that token-level alignment fails to fully activate pre-trained LLMs due to their neglect of LLMs' inherent strengths, which primarily stem from a deep understanding of logic and structure, rather than superficial token processing.
We are the first to propose Context-Alignment paradigm, which aims to construct a context-level alignment between TS and language, thereby activating LLMs" potential capabilities in TS tasks.
We develop a Dual-Scale Context- Context-Alignment Alignment GNNs framework, which achieves structural and logical alignment through dual-scale nodes and directed edges, thus realizing Context-Alignment Furthermore, by integrating the shot prompting technique, we introduce (FSCA), which en- hances LLMss performance in TS tasks.
2

---

Published as a conference paper at ICLR 2025
Our experiments across multiple datasets and various TS tasks demonstrate that method sur- passes existing techniques, especially in few-shot and zero-shot forecasting tasks. Ablation studies further emphasize the importance of Context- Sintext-Aligmment
language prompt
Prediction Results
Instance De-nom
Output Linear Layer
instance Norm & Patching
xL
Input Embedding
Emboddinng
Multi- Head Attention
Add Layer- Moyer-Norm Norm
Add & Layer- Layer-Norm Norm
Multi- Head Attention
Add Laye Norm
Add & Layer Norm
Fine grained GNN
Feed Forward
Feed Forward
grained GNN
Dual-S Dual-Scale Context Hemilext-Aligmmmett GNNS
Fra-grained GNN
Learnable Interaction
Positional Embeddings
Comntext-Alignmenttt GNNs
xL
Multi Head Attention
Add 8t Layer- Layer-Norm Norm
Add Layer-Norm
Coarse grained GNN
Multi Head Attention
Add & Layer Norm
Add & Layer- Layer-Norm Norm
Coarse grained
Feed Forward
Feed Forward
Loiree-raiint grained GNI
Dual-Scale Eentext-Alignment GNNs
Prompt Embedding (fine-grained)
Embedding (Fine-grained)
Prompt Embedding (Coarse-grained)
TS Embedding (Coarse-grained)
Training
The architecture of our method, where the graph structure demonstrates the prediction task based on FSCA as detailed in Sec.3.3 Dual-Scale Context- Context-Alignment GNNs can be flexibly and repeatedly integrated into pre-trained LLMS at various layers, enhancing L.MMs' awareness of logic and structure and improving performance.
Figure 1:
2 RELATED WORK
2. TIME SERIES TASKS
TS tasks are crucial for various applications, including financial forecasting, weather prediction, and activity recognition, and involve analyzing time-ordered data. Traditional statistical methods like ARIMA (Anderson] 1976) and Prophet (Taylor & Letham 2018) effectively model trends and seasonality. The rise of deep learning introduced CNN-based methods, which use convolutional neural networks to extract features automatically (Bai et al.) 2018 Wu et al., (202)). Additionally, RNNs, such LSTM and GRU, excel sequence prediction by capturing dynamic temporal be- haviors and long-range dependencies (Lai et al. [2018b 8b) Qin et al. (017) (iami -Siam--aammn Namini et al. (018)). More recently, Transformer-basee models have advanced the field by processing sequences in par- allel and applying attention mechanisms to focus
significant temporal aspects, thus improving
performance in complex scenarios (Wen al. 2023 Zhou al. 2002 Wu et al. (202T). However, these methods often struggle with long, intricate sequences and lack the flexibility and generaliz- ability required for diverse real-world TS data across diverse domains.
2.2 LARGE LANGUAGE MODELS FOR TIME SERIES
LLMSs have demonstrated strong capabilities in various fields, becoming a focal point for advanc- ing TS tasks. Recent research highlights their potential in TS tasks. Some methods aim to directly enhance the capabilities of LLMs for TS tasks. For example, [Cao et al. (2024) captures complex interactions among trend, seasonal, and residual components to aid distribution adaptation. CChang et al. 2024) propose a statistical prompting strategy to enhance the performance. However, these methods overlook important step: enable LLMs to understand TS inputs first. Other methods aim to enable LLMs to understand TS data and activate their potential for TS tasks. For example, Jin al. (0024) reprograms input TS into text prototypes and enhances them with prompts. (unn

---

Published as a conference paper at ICLR 2025
al. 2024) aligns TS embeddings space with LLMs embeddings space. Pan et al. (2024) proposes LLM,-LLM, a semantic space- saace-informed prompt learning to align TS embeddings with language to- ken embeddings. However, these methods overlook that LLMs' inherent strength stems from their understanding of the logic and structure, rather than superficial token embedding processing. Merely aligning token embeddings without considering the coherence and consistency of the context fails to leverage this inherent advantage.
3 METHODOLOGY
To address both structural and logical alignment in Context-Alignment, we develop a Dual-Scale Context-Alignment GNNs (DSCA-GNNs) framework. In this framework, dual-scale nodes achieve structural alignment, while directed edges realize logical alignment. Additionally, the specific struc- ture of the graph depends on the language prompts introduced, as different prompts lead varying logic and structures in the TS-language. That is, each prompting method corresponds to a specific ASCAAGNNY framework. this section, we present the corresponding DSCA-GNNs frameworks for both the vanilla prompt and the demonstration examples prompt.
3.1 PRELIMINARIES AND TOKEN EMBEDDING
Time-Series Forecasting. Given past data X E R RxT, where each row represents a TS of T steps across D variables, the goal is to build predictive model F with parameters O and prompt Py to forecast the next T time steps. The forecasting model can be formulated X F(;; Oz).
Time-Series Classification. Given TS data X E RxT the task is to build a classification model C with parameters Oc and prompt Pe assign class label ÿ (1,2, ..., C), where C is the number of classes. The classification is defined as ÿ = C(X; Pc).
Token Embedding. Multivariate TS data is segmented into patches X; using a sliding window of size and stride s. Each patch is embedded into an -ddimensional space compatible with LLMs. The embedded patches are denoted as {e:) where T-pts is the number of patches.
3.2 DEMO: VANILLA
(VCA)
The most straightforward approach to utilizing pre-trained LLMs for TS tasks is to input both TS data and vanilla language prompts into the model directly. For TS forecasting tasks, vanilla prompt like "Predict future sequences using previous data:" can be employed to guide LLMs in completing the prediction. The embeddings of the prompt overall input embeddings of the model can be represented as below:
be represented {21, 22, ................................ 2mh, }. and the
(e1, C2, Cn. 21, 22, ........ Zm] C 2j E R.M.
Due to the verbose and lack of clear structural divisions of TS embeddings {e,) LLMs lack information that TS embedding {e 1 is an integral entity, making it difficult to analyze TS. Fur- thermore, form [ lacks logical guidance. Directly concatenating the TS embeddings and language prompt embeddings loses essential contextual coherence between TS data and the prompt.
Dual-Scale Context-Alignment GNNs of VCA. Context- Alignxt-Aligmment leverages the LLMs' inher- ent strength in logic and structure to enable LLMs to contextualize and comprehend TS data. We delineate clear structures (structural alignment) and establish correct logical guidance (logical align- ment) by Dual-Scale Context-Alignneee GNNs framework to achieve Context- centext-AAlgnnann Alignment. Firstly, using dual-scale nodes, structural alignment aggregates tokens from the same modality into one lin- guistic component while preserving the feature of each token. Finennrained GNN GF treats each token, i.e., each element in the form [ as node. Coarse- grained GNN Ge treats consecutive tokens with the same modality to embed the TS tokens and language tokens into an --diienssonaa space, respectively, which can be formalized
node. Specifically, Ge employs two learnable linear layers, fe and fz,
é = fel, (a(e1,ez) eg..... en);
z == fz(21,1200 Z2. 22.., zm).
Then, form Dis is transformed into form
ē, M.
4

---

Published as a conference paper at ICLR 2025
Each element in the form is regarded node in Gc. Secondly, using directed edges, logical alignment emphasizes the correct semantic association between different components. Clearly. in the prompt "Predict future sequences using previous data:", "previous data:" refers to the lengthy TS data, and requires information from TS data. Therefore, in Gc. we construct directed edge to indicate that the entire TS data serves the upstream information source for the
E ē prompt. In GF, we construct directed edges {Eij : e, + žjl² E {l,..., nt., j E {l...., m}} to convey varying information from each TS token to the prompt token. We also constrain the sum of edge weights from TS tokens to one language token to be 1, i.e. Σi=1 Waj 1, Waj is the edge weight of Esy. implicitly emphasizing that TS should be treated as whole. In Gc. the weights of all directed edges are set to 1. Besides, {wnj] Fi=1 are proportional to the cosine similarity between the embeddings of two nodes.
The updated node embedding matrices for GF and Go denoted as NF and No. respectively, Based on the GNN update strategy, N& and No can be formalized as:
N a (D ALD N.
k [F
where No. denotes the transpose of the pre-updee update node embedding matrix. In GF, NN is shown in form while in Gc. Ne follows form 2] Besides, W₁ E RM XM represents the learnable matrix of the GNN, denotes the nonlinear activation function ReLU, and A A + I is the weighted adjacency matrix with unit matrix added. D₁ is diagonal matrix where DR.i = A.A'ki k.j
Learnable interaction. Macroscopic logical-structue stand TS, but loses key details necessary for TS tasks. In contrast, GF retains detailed information. Thus, we introduce a learnable interaction to transfer macroscopic information from Gc to Gy. The learnable interactions can be represented by the following formula:
information of Ge helps LLMs under-
AN Wo-r NeTo-t.
(4)
where -e-s E (n+mm) is a 0 - 1 assignment matrix, Fiş = means that the j-th node in GF be aggregated into the i-th node in Gc. Wo+s E RMx M is a learnable weight matrix. NE can be updates NO- Nr + AN after aggregating the information from No.
Overview. After structural and logical alignment, ÑF maintains clear structural and coherent mantics information, helping L.MSs to comprehend TS tasks and activating potential capabilities. Both NF and No are input into pre-trained LLMs. The DSCA-GNNs can be flexibly integrated into various layers of pre-trained LLMs, and only the first time apply it need fe and f= to obtain coarse- coarse-grained GNN. The output from the GF branch is used to compute the MSE loss against the ground truth. In this section, we use the vanilla prompt to construct a specific DSCA-GNNs framework, we call it Vanilla Context-Alignmen (VCA).
3.3 FEW-SHOT PROMPTING BASED CONTEXT-ALIGNMERT (FSCA)
The demo VCA in Sec. 3.2 is the simplest and most direct attempt of Context-Alignment Moving forward, we will naturally consider whether Context-Alignment can further enhance the perfor- mance of LLMs on TS tasks by leveraging more advanced prompt techniques from NLP.
In NLP, "how prompting" refers to small set of instances provided to the model to demonstrate a task, enabling the model to perform similar tasks effectively (Brown) 2020). Based on this, we divide the TS embeddings {e,);-, into N parts while preserving their original order. Since the subsequent TS part can be used as ground truth for predictions based on the preceding TS parts, can construct N prediction demonstration examples using N parts of {e:)
1n and language prompt embeddings (z1, 22,... Zm same as demo in Sec. 3.2 The j-th part of {ei is denoted as (ej.1...., )- We arrange the Ia-langaage embeddings in the format as:
(e1.1..... 11,4 21...., Zm, 1, ... 22,2 21. Zms .N. eN.Iy Z1, 21..... =m]] (5)
Dual-Scale Context-Alignment GNNs of FSCA. Due to the verbose and lack of clear structural divisions of the format 5) I.MSs struggle to understand the input. We first construct the coarse- grained GNN Gc. We introduce two learnable linear layers fe and fa to embed the TS tokens and language tokens into an M -idmennsional space, respectively, which can be formalized as
é = fe(e5, e.(e,,,,e........
f2(21, 22. ...... Zm)
5

---

Published as a conference paper at ICLR 2025
Then form is transformed into form
[ē1, (11) Ê2, (2) ēN, 1, ě E j I........
(6)
For clarity in our discussion, we number the in form in fact, == x = =ū,), i,j = I,...,N. The directed edge set of the Go can be represented follows:
{Ec: :c:ee 2
N. l.......) {Ec z(0) ē411/i
13.
In Gc, there are two types of edges, which guide two different logical relationships. The first type, as shown in the first item of signifies that the prompt ( receives TS information from all preceding TS parts. The second type, indicated in the second item of implies that ē,+1 the correct output result of prompt =(0), In Gc, the weights of directed edges are to
Fine-grained GNN Gp treats each token in the form 5 as node. The directed edge set of the G' can be represented follows:
(E Gj + 2( 1, N,J 1, 1.................. 1,
1
m
(8)
U(Er =t - ei+1,s =1
1,8 1,....,........ 1,1,4
m)
The formula means that the directed edges of GF are a decomposition of the directed edges in Gc. Additionally, since LLMs have strong comprehension abilities for language prompts, we can prune the directed edges in GF, transforming form g into form i.e., TS tokens are only connected to the first and last tokens of the prompt, thereby prevent overfitting. We constrain DL "j.
(s) for the first type edges, Di#1 Wi1,,
for the second type edges, where wj. is the edge weight 1h4 proportional to the cosine similarity between node embeddings. Fig. provides schematic of the
of ej.e ø1 (6) (6)
1,+ is the edge weight of 2m Ei-1,a. [wit 1/
J==1 and {w! {wi+1.s
structure. The node embedding update formula is similar to formula
N = I,... U(Ep 2 e ..... N - 1, 1, 1......... 1+11.
E Ej.a
21
43
(9)
Learnable interactions & Overview. Similar to the demo in Sec. 3.2 we introduce an assignment matrix and a learnable weight matrix to achieve learnable interactions between the two scales. The training of FSCA can refer to Sec. 3.2 and Fig.
4 EXPERIMENTS
The proposed Context-Alignment demonstrates robust performance across variety of tasks, de- tailed in Sec. 4.2 (long-term forecasting), Sec. 4.3 (short-term forecasting), Sec. [4.4] (few-shot forecasting), Sec. 4.5 (zero-shot forecasting), and Sec. 4.6 (classifficat)))))) Most experiments lever- age FSCA. Specifically, VCA validates its efficacy using logic guidance and structural division alone in section 4.1 For classification tasks with multiple classes, where GPT-2's length constraints limiting, FSCA is reserved for binary classes, and VCA is applied to multi-class datasets. Context- Alignment significantly boosts training efficiency and cost-effectiveness, especially in few-shot and zero-shot forecasting, by establishing robust a priori structural understanding and logical relation- ships. Full results and dataset description in Appendix C and A.2 respectively.
Baselines. Building on the groundwork of Zhou et al. (2023) and Jín et al. (2024), and mindful of page constraints, we selected representative array of high-performing baselines for extensive evaluation. These encompass Transformer
models such as iTransformer (Liu et al. 2023b),
FEDformer (Zhou al.] (0022). Non-stationary Transformer (Liu et al. 2022). St (Woo) et al. 2022 PatchTST (Nie et al. 2022). alongside notable non- Ton-Transformer methods like Times- Net Wu et al. 2022). and DLinear (Zeng al.) (023). We also included advanced LLM--based
models- GPT4TS (Zhou et al. 2023). Time-LLM (Jin et al., (0024). and SIIP-LLM (Pan et al.) 2024 all utilizing GPT-2 as the standard LLM backbone to ensure model consistency. To ensure
fair comparison, adhere the experimental framework outlined in Zhou et al. (2023) and Wu
ct al. (2022). Detailed evaluations are expanded upon in subsequent sections.

---

Published as a conference paper at ICLR 2025
4.1 DEMO: VANILLA CONTEXT-ALIGNMAN
VCA achieves logical and structural alignment through DSCA-GNNS As demonstrated on the ETT dataset (Table T. VCA substantially out- performs both variants without DSCA-GNNs and other baselines. A performance decline is observed compared to FSCA, which further augments LLMs' TS processing capabilities us- ing demonstration examples prompt. VCA
Table 1: Results for VCA and variants. Bold: hest, Underline: second best.
Long serm Forecusting
Method/Varannt
ETThi
ETTs2
ETTml
ETTm2
GPTITS
0.427
0.354
0,352
0.666
ESCA
0.394
0.316
0.342
0.250
VCA wio DSCA-GNNS
0.435
0.362
0.374
0.271
YCA
0.417
0.385
0.259
without DSCA-GNNS, despite incorporating task description prompts, lacks context-alignment GNNs, resulting in more verbose and semanti-
cally confused inputs for LLM, thus yielding the worst outcomes.
4.2 LONG-TERM FORECASTING
Setups. For long-term forecasting tasks, we validate the efficacy of FSCA across eight prevalent datasets (Wu et al.) 2022): ETTh1, ETTh2, ETTm1, ETTm2, Weather, Electricity, Traffic, and ILL Consistent with GPT4TS (hou et al. 2023). Time-LLM (Jin et al. 2024). and IIIP-LLM (Pan et al., 2024). utilize an input TS length of 512 except for the ILI dataset. Performance is evaluated over four prediction horizons: {24. 36, 48, 60} for ILI and {96, 192, 336, 720} for the other datasets, using Mean Squared Error (MSE) and Mean Absolute Error (MAE) as metrics.
Results. As shown in Table 2 FSCA surpasses all baseline methods in most scenarios. Specifically, it reduces average MSE of 3. 1% over the suboptimal method PSt and outperforms other LLM- based methods Serehodss$$II--LLA ILMMLLLM, Time-LLM, and GPT4TS by 7.3%, 12.2%, and 16.6%. This consistent superiority across diverse datasets highlights the critical role of logical and structural alignment. Furthermore, demonstration examples prompt boost LLMs contextual understanding of TS data.
Table 2: Long-term forecasting tasks, all results are based on different horizons: {24, 36, 48, 60) for ILI and {96, 192, 336, 720 for others. Bold: best, Underline: second best. Full results are provided in Appendix C.1
Methods
PSCA
LLM | Time LLM
GPTTTS [Tansformerr
DLinear
PatcTTT
TimesNe:
FEDferrer
Stationary
ETSformer
Meric
IMS
MAF] MSF MAF
MSF
MAF
MSF
MAR
MSR
MAF
MSR
MAF
MSE
MAF
MSF
MAF]
MSF
MAF| MSF
MAF]
MSF
MAF
TLT
1,380 0.783
1.552
0.226
1,713
0.858
1.925
0.903
2.073
0.941
2.169
1.04)
0.797
139 0.93)
2.547
2.144)2.07
2.077
0.914 2.497
1.004
Weather
0.324 0.262
0.265
0.237
0.269
0.270
0.304
0.287
0.314
ECL
0.159 0.252
0.166
0.262
0.167
0.264
0,167
0.263
0,203
0.298
0,165
0.263
0,161
0.252
0.192
0.295
0.214
0.327
0.193
0.296
0,208 0.323
Traftic
0.386 0.263
0.405
0.286
0.407
0.289
0.414
11294
0,389
0.295
0.433
11295
0.240
0,263
0.020
0.336
0.010
0.376
01624
0.340
0.621 0.396
BTTH:
0.394 0.424
0.418
0.436
0.436
0.435
0.427
0.426
0.451
0.462
0.422
0.437
0.413
0.450
0.458
0-450
0.440
0.460)
0.570
0.537
0.542 0.530
BTTh2
0.316 0.375
0.355
0.399
0.361
0.398
0.354
0.394
0.382
0.414
0.431
0.446
0.330
0.379
0.414
0.127
0.437
0.019
0.526
0.516
0.439 0.452
BTTml
0.342
0.382
0.354
0.384
0.352
0.383
0.370
0.357
0.378
0.251
0.406
0.448
0.481
0.450
BTTm2
0.250
0.314
0.262
0.275
0.334
0.266
0.326
0.333
0.255
0.333
0.305
0.345)
0.306
0.347
0.293
4.3
SHORT-TERM FORECASTING
Table 3: Short-term forecasting on M4, with prediction horizons ranging from [6, 48). The results are weighted averages across all datascts under different sampling intervals. Bold: best, Underline: second best. Full results are provided in Appendix C22
Matsod
FSCA
S"IP-LLM
TimeLLM
GPT4TS
[Trarsfammec]
Diincar
PachTST
N-HITS
N-BEATS
TinceNee
FEDiomaer
Saaiomay
Aoeage
SMARPE
11.828
12,090
12.42
13,639
12.250
12,800
12,780
MASE
1.500
1662
1731
18088
1631
20055
L,63
1.625
1,698
1836
1.775
1.756
OWA
0.800
0857
0913
0.940
0.69
1.851
0.869
0.806
0955
0.49
0.900
Setups. We conduct short-term forecasting experiments on the M4 dataset MMakridakis et al. 2018).
which includes market data across various frequencies, with prediction horizons ranging from 6 to 48. We incorporate N-HiTS (Challu et al. (002) and N-BEATS (Oreshkin et al. 2019) as additional baselines. Performance is quantified using symmetric mean absolute percentage error (SMAPE), mean absolute scaled error (MASE), and the overall weighted average (OWA) as evaluation metrics.
Results. Results in Table 3 indicate that FSCA exhibits competitive performance compared to SOTA methods. FSCA maintains robustness in both long-term and short-term forecasting, at- tributable to the effectiveness of structural and logical alignment across varying sequence lengths.
7

---

Published as a conference paper at ICLR 2025
4.4 FEW-SHOT FORECASTING
Table 4: Few-shot forecasting task 5% training data. Results averaged across different prediction lengths (96, 192, 336, 720). Bold: best, Underline: second best. Full results are provided Appendix
Mothods
PSCA
SPPLLMM
Time-LLM
GPIETS
Tannsorrerr
DLLinzar
TimesNes
|Hiitormer
Stanonnay ETStormer
Monic
IMSG
MAG]
MSC
MABI
MSB
MAD
MAB]
MSE
MAG MSE MAG]
MAD]
MAE
ETTh1
0.575
0.508
0.630
0.550
0.648
0.549
0.681
0.560
1,700
0,730
0,150
0.611 0,695 0,569 0.925 0.647 0.688 0.562) 0.943 0.616 1,199 0,839
bTT22
0.366
0.397
0.380
0.413
0,998
0.426
0,400
0.433
0.485
0,475
0.827
1.615 0,439 0.448 0.463
0.454 0.463 0.454 0,470
0.429 0,809 0.681
ETTml
0.435
0.429
0.455
0.446
0.477
0.451
0,472
0,450 0,784
0.596
0.400
0,417 0.526
0.476 0.717 0.561 0.730 0.592 0,857
0.5%
1.125 0.782
FTTm2
0.284
0.332
0.796
0.342
0.307
0.348
0.908
0,546
0.355
0.388
0.999
0-426) 0.314
0,352 0.344
0.372 0.381
0.404 0.541
0.572
0.534 0.547
Averae
[0.415
0.416
0.445
0.438]
0.458
0.443
00.05 (1447]
0.675
0.542
00,544 0.317)00495 0.493 0.461 0.612 0.50%M 0L558 0.503] 0:663 0.520] 0,914 0.712
Setups. The expressive potential of LLMs often results in a robust performance in few-shot scenar- ios (Brown 2020 Liu et al., 2023a). While current LLM-based methods outperform earlier models like DLinear, PStchTST, and TimesNet, they do not fully exploit LLMs' deep TS comprehension, thus not maximizing their few-shot capabilities. To explore whether Context-Alignment can boost the few-shot efficacy of LLMs, we perform experiments on the ETT datasets, following the protocol established by Jin et al. (2024). We evaluate the performance of FSCA using only 5% of training data, with results from 10% data detailed in Appendix C.3
Results. As shown in Table 4 our method consistently outperforms all baselines. It achieves 6.7% reduction in average MSE compared to the leading further improvements of 9.4% and 10.8% against Time-LLM and GPT4TS. Moreover, FSCA shows a 15.8% performance gain over the SOTA transformer model PatchTST. We attribute these to the integration of prior knowledge in structural division and logic guidance by Context-Alignment. The strong generalizability of these priors, even with limited training data, effectively activates LLMs' latent few-shot capabilities in TS, further boosted by a few demonstration examples.
model, I-IM-LLM. We observe
4.5 ZERO-SHOT FORECASTING
Setups. Despite the inherent zero-shot capabilities of LLMs (Kojima et al. 2022), LLM-based methods struggle to fundamentally understand the structure of TS data and its logical associations with prompts, leading to underperformance compared to Transformer-basee methods like PatchTST. Using the setup from Time-LLM (Jin et al., 2024), we also evaluate the gross-domain efficacy of FSCA on ETT dataset. Specifically, the model is trained on Dataset and then tested on Dataset B without utilizing any training data from Dataset B.
Table 5: sero-shot learning results: the first column A + indicates training on datasct A and testing on dataset B. Bold: best, Underline: second best. Full results are provided in Appendix C.4
Methods
PSCA
SPIP-LLM
Time-LM
GPT4TS
iTraasfomer
Dlisear
PachTST
TimesNet
Maric
MSE
MAE|
| MSE
MAR
MSE
MAR
MSE
MAR
MSE
MAE
|
MSE
MAP
MSR
MAR
MSE
MAR
ETThl - E1Th2
0.313
0.369
| 0,403
0.413
0.384
0,500
0.406
0.422
| 0.437
0.455
| 0.493
0.488
0.380
0.408
0.421
0.431
ETThi - 6T7m2
0.290
0.348
0.325
0.300
0.317
0,370
0.305
0.363
0.360
0.380
] 0.415
0.453
0.14
0.360
0.327
0.361
ETT12 ETDi
0.527
0.507
0.609
0.560
0.63
0.540
0.757
0.578
0.668
0.625
| 0.703
0.574
0.55
0.513
03655
9.611
ETTh2 ETTm2
0.288
0.347
0.327
0.363
0.339
0.373
0.335
0.370
0.335
0.382
[ 0.328
0.386
0.325
0.322
FTTml FTT12
0.430
0.440
0.440
0.433
0.430
0.458
| 0.464
0.475
0.499
ETiml ETTm2
0,264
0.347
0.311
0.343
0.333
0.345
0.303
| 0.335
0.389
0.296
0.334
0.322
0.354
ETTm2 ETTh2
0,343
0,393
0,406
0.429
0.429
0,648
0.435
0,443
0.432
0.447
| 0.455
0.451
0.409
0.425
0.435
0.43
ETTm2 ETTml
0,450
0.463
0.622
0.532
0.588
0.503
0.769
0.567
0.706
0.572
0.649
0.537
0.568
0.492
0.769
0.567
Average
0.357
0.393
0.437
0.431
0.434
0.429
0.472
0.443
0.491
0.461
0.480
0.472
0.412
0.412
0.492
0.451
Table that FSCA substantially outperforms the most competitive baselines. Com- pared to the second-best model, PatchTST, FSCA exhibits 13.3% improvement in performance. Against LLM-based models like S'IP-LLM, Time-LLM, and GPT4TS, FSCA achieves perfor- mance gains of 18.3%, 17.7%, 24.3%. Experiments in both few-shot and zero-shot settings high- light FSCA's exceptional performance under data-scarce conditions. In these scenarios, Context- Alignment paradigm provides a robust contextual prior, enabling accurate logical and structural understanding that enhances the potential of LLMS for cross-domain TS processing.
Results.
8

---

Published as a conference paper at ICLR 2025
4.6 TIME SERIES CLASSIFICATION
Setups. To assess the model's 's capability in learn- ing advanced representations, we conduct compar- ative experiments series classification using the
setup outlined by /Zhou et al. (2023). We select 10 multivariate UEA classification datasets (Bagnall) et al. 2018) from domains including ECG, audio, gesture, spectral recognition, etc. For binary class
873
Rew
Apsker
datasets like FaceDetection, we apply FSCA frame-
Bew 12:
work, adding demonstration example prompt for
GMTE
each category at the beginning of the prediction se- quence. For datasets with multiple classes, such as Handwriting with 26 classes, employ VCA due to GPT-2's input length constraints.
76
Areggg Anouray (S)
Figure 2: For the classification task, the results
show average accuracy across 10 subsets from the UEA dataset. Detailed results in Appendix F.
Results. Fig. 2 presents the average accuracy of various methods on all UEA classification datasets. FSCA achieves an accuracy of 76.4%, surpass- ing all baseline methods and recording 2.4% increase over the next best model, Zhou et al. (023) This performance suggests the effectiveness of FSCA or VCA extends beyond predictive tasks. Ad- ditionally, Context-Alignment shows generality, indicating its potential applicability across various contexts in the field.
4.7 ABLATION STUDY
As shown in Table 6 we conduct an ablation study on the framework design and analyze the ef- fectiveness of FSCA. FSCA" represents the optimal results, highlighted in bold in Table 6 D.4 is optimal for Long-term forecasting, whereas D.3 is best for classification tasks.
Validity of Dual-Scale Context-Alignment GNNs. To evaluate DSCA-GNNs, we con- duct comparative experiments from two per- spectives. In A.1, without Dual-Scale GNNs, we rely solely on demonstration examples as prompts, resulting in higher average MSE com- pared to FSCA*
Table 6: Ablations results (average MSE for four prediction lengths in long- forecasting and accuracy for the classification). Bold: best. In C.*. default insertion positions are the first and last layers; therefore, C.2 and D.3 are identical.
A.2, random initializa- tion of adjacency matrix leads to further perfor- mance decline compared to A. underscoring that incorrect logical information can impair model performance. A comprehensive com- parison across A. A.2, and FSCA" confirms that GNNs guided by appropriate logical frame- works effectively leverage the inherent capabil- ities of LLMs in TS tasks.
Long 4emm Forecasting
Claseriiation
Variant
ETThl
ETTml
FaceDet. Heartbeal
Metric
MSE
Accuracy
Dual-Scale GNNs
A.2 with Random Init
0,463
0.392
64.5
73.1
R.I Course grained Branch]
69.6
78.5
C.I PSCA(2)
0,402
0.357
69.1
78.0
C.2 FSCA(4)
0.396
0.345
70.4
79.5
C.3 FSCA(6)
0.399
0.347
70.1
79.0
C.4 PSCAI8)
0.418
0.362
67.7
77.0
C.5 PSCA(IO)
0.439
0.383
66.4
75.6
C.6 PSCAr12)
0.455
0.396
63.2
72.6
D.I Pesition
Validity of Coarse-Grained Branch. In B.1, omitting this module impairs the model's abil- ity to understand the overall structure and the macro-level logical relationships, resulting in reduced performance compared to FSCA". This decline is due to reliance solely on the fine-grained branch for token- context alignment, which still leaves the inputs verbose and lacking structure.
D.2 Insertion Positio [0. 2]
0.403
0.350
69.5
78.0
Position 4]
70.4
D.4 Insertion Pasition [0, 2, 4]
0.394
0.342
69.7
78.5
D.5 Position 1]
68.7
The Number of L.LM Layers. We conduct ablation studies to assess the impact of varying numbers of GPT-2 layers. C.* indicates that models with 4 and layers perform optimally while adding more layers causes overfitting (Zhou et al. 2023). For computational efficiency, we select 4 layers (C.2).
Insertion Position of Dual-Scale Context-Alignment GNNs. DSCA-GNNs can seamlessly in- tegrate into various layers of LLMs. Firstly, when priors for structural and logical alignment are present at the input, deeper layers of LLMs more fully leverage this prior knowledge, hence, we default to employing this module at the model's 's input, as shown in configuration D.1. Secondly, D.2, D.3, and D.4 involve repeating integration of this module at the mid and output stages of the

---

Published as a conference paper at ICLR 2025
LLMs encoder, enhancing prior guidance and yielding performance improvements. Ablation stud- ies for long-term forecasting and classification tasks reveal that optimal performance results from different insertion points, attributed to domain differences in token features among various LLMs layers (Ethayarajh 2019) and varying demands of different tasks; Thirdly, configuration D.5, which omits GNNs module model's 's input, leads to decreased performance, confirming that incorporating context-alignment at initial stage enhances the overall utilization of pre-trained LLMs.
5 CONCLUSION
In this paper, we point out that effectively utilizing LLMs in TS tasks requires first activating their capabilities, and then enhancing their performance. By rethinking the inherent strength of LLMs, we are the first to propose Context-Alinnmen paradigm, which aims to construct a context-level alignment between TS and language. Unlike previous methods based on token-level alignment, Context-Alignment constructs a consistent context for TS-language multimodal inputs, better har- nessing LLMs' deep understanding of context to activate their potential on TS tasks. We develop DSCA-GNNs to achieve Context- Aolinet-Aligmment. Besides, by integrating the Demonstration Examples Prompt technique, we introduce FSCA, which enhances L.LMs' performance in TS tasks. Exper- iments demonstrate that our method significantly outperforms others, particularly in few-shot and zero-shot forecasting tasks, and ablation studies confirm the importance of Context-Alignment.
ACKNOWLEEDEMMENT
This work was supported by the National Natural Science Foundation of China (Grant No. 621061 16), the China Meteorological Administration (Grant No. QBZ202316), the National Key Research and Development Program of China (Grant No. 0024YFF1500600 ence and Technology Projects of Ningbo (Grant No. 20227236), as well as by the High Performance Computing Centers at Eastern Institute of Technology, Ningbo, and Ningbo Institute of Digital Twin.
and the Major Sci-
REFERENCES
Oliver Anderson. Time-series. 2nd edn., 1976.
Anthony Bagnall, Hoang Anh Dau, Jason Lines, Michael Flynn, James Large, Aaron Bostrom, Paul Southam, and Eamonn Keogh. The uea multivariate time series classification archive, 2018. arXiv preprint arXiv: 01111/111.000.. 2018.
Shaojie Bai, J Zico Kolter, and Vladlen Koltun. empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv: 1803.0121, 2018.
Tom B Brown. Language models are few-shot learners. arXiv preprint arXiv:2005. 14165, 2020.
Defu Cao, Furong Jia, Sercan O Arik, Tomas Pfister, Yixiang Zheng, Wen Ye, and Yan Liu. Tempo: Prompt-based generative pre-trained transformer for time series forecasting. In The Twelfth Inter- national Conference on Learning Representutions, 2024.
Cristian Challu, Kin G Olivares, Boris N Oreshkin, Federico Garza Ramirez, Max Mergenthaler Canseco, and Artur Dubrawski. Nhits: Neural hierarchical interpolation for time series forecast- ing. In Proceedings of the AAAI conference artificial intelligence, volume 37, pp. 6989-6997, 2023.
Si-An Chen, Chun-Liang Li, Nate Yoder, Sercan O Arik, and Tomas Pfister. Tsmixer: An all-mlp architecture for time series forecasting, arXiv preprint arXiv:2303.06053 2023.
Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pp. 785-794, 2016.
Yu-Neng Chuang, Songchen Li, Jiayi Yuan, Guanchu Wang, Kwei-Hemg Lai, Leisheng Yu, Sirui Ding, Chia- Yuam Chang, Qiaoyu Tan, Daochen Zha, et al. Understanding different design choices in training large time series models. arXiv preprint arXiv:2406. 14045, 2024.
10

---

Published as a conference paper at ICLR 2025
Angus Dempster, François Petitjean, and Geoffrey Webb. Rocket: exceptionally fast and accu- rate time series classification using random convolutional kernels. Data Mining and Knowledge Discovery, 34(5):1454-144) 2020.
Kawin Ethayarajh. How contextual are contextualized word representations? comparing the geom- etry of bert, elmo, and gpt-2 embeddings. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNN.P-JCNIIP), pp. 55-65, 2019.
Jean- Yves Franceschi, Aymeric Diculeveut, and Martin Jaggi. Unsupervised scalable representation learning for multivariate time series. Advances in neural information processing systems, 32, 2019.
Shanghua Gao, Teddy Koker, Owen Queen, Thomas Hartvigsen, Theodoros Tsiligkaridis, and Marinka Zitiik. Units: Building a unified time series model. arXiv preprint 1rXiv:2403.0033,, 2024.
Mononito Goswami, Konrad Szafer, Arjun Choudhry, Yifu Cai, Shuo Li, and Artur Dubrawski. Moment. arXiv preprint SAKii::20020088885 2024.
Albert Gu, Karan Goel, and Christopher Ré. Efficiently modeling long sequences with structured state spaces, arXiv preprint arXiv:2/ 2021.
Zhaoyang Huang, Xiaoyu Shi, Chao Zhang. Qiang Wang, Ka Chun Cheung, Hongwei Qin, Jifeng Dai, and Hongsheng Li. Flowformer: A transformer architecture for optical flow. In European conference on computer vision, pp. 668-685. Springer, 2022.
Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Zhang, Xiaoming Shi, Pin- Chen, Yuxuan Liang. Yuan-fang Li, Shirui Pan, et al. Time-Ilm: Time series forecasting by reprogramming large language models. In International Conference Learning Representations, 2024.
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models zero-shot reasoners. Advances in neural information processing systems, 0222199-22211 2022.
Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long-and short-term temporal patterns with deep neural networks. In The 41st international ACM SIGIR conference research & development in information retrieval, pp. 104, 2018a.
Guokun Lai, Wei-Cheng Chang. Yiming Yang, and Hanxiao Liu. Modeling long-and short-term temporal patterns with deep neural networks. In The 41st international ACM SIGIR conference on research & development in information retrieval, pp. 104, 2018b.
Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X Liu, and Schahram Dustdar. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and fore- casting. In International conference learning representations, 2021.
Xin Liu, Daniel McDuff, Geza Kovacs, Isaac Galatzer-Levy, Jacob Sunshine, Jiening Zhan, Ming- Zher Poh, Shun Liao, Paolo Di Achille, and Shwetak Patel. Large language models are few-shot health learners. arXiv preprint arXiv: 15525, 2023a.
Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Non-stationary transformers: Exploring the stationarity in time series forecasting. Advances in Neural Information Processing Systems, 15:9881-9897 2022.
Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers effective for time series forecasting. arXiv preprint arXiv:2310.06625, 2023b.
Spyros Makridakis, Evangelos Spiliotis, and Vassilios Assimakopoulos. The m4 competition: Re- sults, findings, conclusion and way forward. International Journal forecastngg, 34(4):802-88) 2018.
11

---

Published as a conference paper at ICLR 2025
Ercong Nie, Shuzhou Yuan, Bolei Ma, Helmut Schmid. Michael Färber, Frauke Kreuter, and Hin- rich Schütze. Decomposed prompting: Unveiling multilingual linguistic structure knowledge in english-centric large language models. arXiv preprint orXhv:2402.18397 2024.
Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. arXiv preprint arXiv:2211.144 14730, 2022.
BN Oreshkin, Carpov, N Chapados, and Bengio. N-beats: Neural basis expansion analysis for interpretable time series forecasting. arxiv 2019. arXiv preprint WxXiv:1955.1047. 2019.
Zijie Pan, Yushan Jiang, Sahil Garg, Anderson Schneider, Yuriy Nevmyvaka, and Dongjin Song.
s² ip-llm: Semantic space informed prompt learning with Ilm for time series forecasting. In Forty-first International Conference on Machine Learning. 2024.
Yao Qin, Dongjin Song, Haifeng Chen, Wei Cheng, Guofei Jiang, and Garrison Cottrell. A dual-stage attention-based recurrent neural network for time series prediction. arXiv preprint arXiv:1704.02977, 2017.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, llya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Sima Siami-Namini, Neda Tavakoli, and Akbar Siami Namin. A comparison of arima and Istm in forecasting time series. In 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA), pp. 1394-1401, 2018. doi: 10.1 10.109/ICMLL.22018.00022
Chenxi Sun, Hongyan Li, Yaliang Li, and Shenda Hong. Test: Text prototype aligned embedding to activate 11m's s ability for time series. In The Twelfth International Conference on Learning Representations, 2024.
Scan J Taylor and Benjamin Letham. Forecasting at scale. The American Statistician, 72(1):37-45, 2018.
Lean Wang, Lei Li, Damai Dai, Deli Chen, Hao Zhou, Fandong Meng. Jie Zhou, and Xu Sun. Label words are anchors: An information flow perspective for understanding in- in-context learning. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp- 1840-9855, 2023.
Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan, and Liang Sun. Transformers in time series: survey. In Proceedings of the Pherty-Second International Joint Conference on Artificial Intelligence, pp. 6778-6786, 6786,676860 2023.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. Transformers: State-of-the-an natural language processing. In Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, pp. 38 45, 2020.
Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, and Steven Hoi. Etsformer: Exponential smoothing transformers for time-series forecasting, arXiv preprini arXiv:2202.01332) 2022.
Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition trans-
formers with auto-correlation for long-term series forecasting. Advances neural information processing systems, 34:22419-2244
2021.
Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Tem- poral 2d-variation modeling for general time series analysis. arXiv preprint 2..0v:220000/// 2022.
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence, volume 37, pp. 11121-11128, 2023.
Tianping Zhang, Yizhuo Zhang, Wei Cao, Jiang Bian, Xiaohan Yi, Shun Zheng, and Jian Li. Less is more: Fast multivariate time series forecasting with light aampling-oriented oriented mlp structures. arXiv preprint arXiv:2207.00 186, 2022.
12

---

Published as a conference paper at ICLR 2025
Haoyi Zhou, Shanghang Zhang. Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting, In Proceedings of the AAAI conference artificial intelligence, volume 35, pp. 11106-11115, 2021.
Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International conference on machine learning. pp. 22286627286... PMLR, 2022.
Tian Zhou, Peisong Niu, Liang Sun, Rong Jin, et al. One fits all: Power general time series analysis by pretrained Im. Advances in neural information processing systems, 36:43322-43355, 2023.
13

---

Published as a conference paper at ICLR 2025
A EXPERIMENTAL DETAILS
A.1 IMPLEMENTATION
All deep learning networks are implemented in PyTorch and trained on NVIDIA H800 80GB GPUs and GeForce RTX 4090 GPUs. We conduct our experiments using the pre-trained models from Wolf et al. (2020). To ensure fair comparisons, we adhere the experimental configurations outlined in Wu et al. (2022) across all baselines and maintain a uniform evaluation procedure. As discussed in the ablation study in Sec. [4.7] (The Number of LLM Layers), we adopt the first 4 layers of GPT-2. For other LLM-based methods (Zhou et al.] 2023, Jin et al. 2024] Pan et al. 2024), we strictly follow their provided experimental settings or cite their performance if applicable. Most of our proposed method's training settings are based on Zhou et al. (2023): For predictive tasks, we utilize FSCA with N in forms 5 and 6 providing one demonstration example. The Adam optimizer is used with decay rates 3 (0.9, 0.999) and initial learning rates from {10- 4, 5 10 1)= We implement cosine annealing schedule with Timx = 20 and Mmmin 10 8 and set the batch size to 256. Early stopping is configured throughout the training process. MSE loss is employed for long-term forecasting, while SMAPE loss used for short-term predictions. In classification tasks, as described in Sec. 4.6 the FSCA framework is applied to binary class datasets, including demonstration example for each category. For multi-class datasets, such as handwriting recognition with 26 classes, we employ the VCA approach due to the input constraints of LLMs. The RAdam optimizer with initial learning rates from {10-3 10 -3 and batch size of 64 is used. Training also incorporates early stopping and employs cross-entropy loss.
A.2 DATASET DETAILS
For the long-term forecasting task, we utilized eight widely used multivariate datasets (Wu et al, 2022). as detailed in Table 7 These include the Electricity Transformer Temperature (ETT) datasets (Zhou et al. 2021), y. as well as Illness, Weather, Electricity, and Traffic datasets. The ETT dataset comprises EhThi and ETTh2, while the ETTm dataset includes ETTml and ETTm2. Specif- ically, the ETT datasets contain power load data from two power stations at varying resolutions; the Weather dataset features 21 meteorological indicators from Germany; the ILI dataset captures weekly patient counts and influenza- like illness rates; the Electricity dataset consists of hourly con- sumption data from 321 customers; and the Traffic dataset records road occupancy from various sensors on San Francisco highways. Table 7 consolidates the feature details of these datasets, with the ETT dataset being applied in both few-shot and zero-shot learning tasks.
Table 7: Dataset details of long long-term forccasting, wherein both ETTh and ETTm are concurrently utilized for shw-shot and zero-shot learning.
Length
Dimension
Frequency hour
ETTh
17420
ETTm
69680
15 min
Weather
52696
10 min
II.I 966
days
Electricity
26304
321
hour
Traffie
17544
862
hour
For the short-term forecasting task, we employed the M4 benchmark dataset (Makridakis et al.) 2018), which includes 10,000 time series across various domains, from business to economic fore- casting, shown in Table 8 The time series data is categorized into six groups, with sampling rates ranging from annually to hourly.
Table 8: Dataset details of short-term forecasting.
Dataset
Length
Horizon
Frequency
Information
M4 Yearly
23000
6
Yearly
Demographic
Quarterly
8
Quarterly
Finance
M4 Monthly
48000
18
Monthly
Industry
M4 Weekly
359
13
Weekly
Macro
Daily
4227
14
Daily
Macro
M4 Hourly
414
48
Hourly
Other
14

---

Published as a conference paper at ICLR 2025
For the time series classification task, utilized 10 multivariate UEA datasets from Bagnall et al. (2118). Table 9 summarizes the number of classes, series lengths, feature dimensions, and sample sizes for training and testing.
Table 9: Datasct details of time series classification.
Dataset
Train Cases
Test Cases
Dimensions
Length
Classes
EchanslConccarratiinn
261
263
1751
4
FaceDetection
5890
3524
144
62
2
Handwriting
150
850
152
26
Heartbeat
204
205
Japunese Vowels
270
PEMS-SF
267
SellRegulationSCPI
268
293
896
SellRegulationSCP2
200
180
1152
2
SpokenArabieiigiss
6599
2199
93
10
U'WaveGesturel ibruayy
120
320
315
8
A.3 BASELINE DETAILS
iTransformer (Liu et al. 2023b); iTransformer repurposes the Transformer architecture by apply-
ing attention and feed- forward networks inverted dimensions to improve multivariate time series forecasting, achieving tfate-of-the-art performance on real-world datasets.
FEDformer (Zhou et al.) 2022): FEDformer combines Transformer models with seasonal-trend de- composition to capture both global trends and detailed structures in time series data.
Stationary (Liu et al., 2022): : Stationary introduces Non-stationary Transformers, which include Series Stationarization for improved predictability and De-stationary Attention to restore intrinsic non-stationary information, resulting in significant performance improvements and a substantial re- duction in mean squared error compared to mainstream Transformer models.
ETSFormer (Woo et al., 2022): ETSFormer novel Transformer architecture designed specif- ically for time-series forecasting, addressing the limitations of traditional models by incorporating exponential smoothing principles.
PatchTST (Nie et al., p022): PatchTST introduces efficient Transformer-basee architecture for multivariate time series forecasting and self-supervised representation learning, utilizing a segmen- tation approach that divides time series into subseries-level patches. This design enhances local semantic retention, significantly reduces computation and memory usage of attention maps, and al- lows the model to consider longer historical data.
TimesNet (Wu et al. 2022): TimesNet is introduced as a novel approach for time series analysis that focuses on modeling temporal variations by transforming ID time series into 2D tensors, thereby capturing complex intraperiod and interperiod variations.
Dlinear [Zegg et al. )023): Dlinear challenges the efficacy of Transformer- based models for long-
term time series forecasting (LTSF), highlighting their limitations in capturing temporal relation- ships due to the permutation-invarian
nature of self-attention mechanisms.
N-HiTss (Challu et al. 2023): N-HiT's introduces a novel model for long-horizon forecasting that utilizes hierarchical interpolation and multi-rate data sampling techniques to effectively address the challenges of prediction volatility and computational complexity.
N-BEATS (Oreshkin et al. 2019): N-BEATS addresses the univariate time series point forecasting problem using novel deep neural architecture featuring backward and forward residual links with a deep stack of fully connected layers. Importantly, the model's configuration without time-series- specific components suggests that deep learning primitives alone can effectively tackle variety of forecasting challenges while also providing interpretable outputs with minimal accuracy loss. GPT4TS Zhou al., 2023): GPT4TS presents novel approach that leverages pre-trained LLMs for general time series analysis, addressing the challenge of limited training data by utilizing the Frozen Pretrained Transformer (FPT) architecture without altering the self-attention and feedfor- ward layers.
Time-LLM (Jin et al. 2024): Time-LLM a reprogramming framework designed to adapt LLMs for general time series forecasting by aligning time series data with natural language through input transformation and context enrichment techniques. Here, we use GPT-2 (Radford et al. 2019 as the base LLM.
I'IP-LLM (aan et al. 2024]: S'IP-LLM aligns pre-trained LLMs with time series embeddings to enhance forecasting performance. By designing a tokenization module for cross-modallo alignment
15

---

Published as a conference paper at ICLR 2025
and utilizing semantic anchors from pre-trained word embeddings, S'IP-LLM effectively encodes temporal dynamics and retrieves relevant context for time series.
A.4 EVALUATION METRICS
We use Mean Squared Error (MSE) and Mean Absolute Error (MAE) to evaluate the performance of long-term, few-shot, and zero-shot forecasting. For short-term forecasting on the M4 benchmark, following the approach in N-BEATS (Oreshkin et al. 2019), we adopt Symmetric Mean Absolute Percentage Error (SMAPE), Mean Absolute Scaled Error (MASE), and Overall Weighted Average (OWA). For time series classification tasks, as referenced GPT4TS (Zhou et al. 2023). accuracy used the evaluation metric.
1 MSE
Σ(Y,
(Yr 2n
MAE
Σ
Yr
Vnl.
H
SMAPE 200 "
16 Val
100
17
Yn Yyl
MAPE
H
Yx + [VA]
H
h=1
h=1
MASE 1
Σ
Yn Yul
OWA 1
SMAPE
MASE MASE Naiv22
H
j= Y3 Y5
2 EMAAENaive?
4=1
H-
where H denotes the number of data samples, corresponding the forecasting horizon in the periment. s represents the periodicity of the time series. Yn. and Y, refer the h-th ground truth and its corresponding prediction, respectively.
B DUAL-SCALE CONTEXT-ALIGNMER GNNS IN CLASSIFICATION
The construction of Dual-Scale Context-Alignmeet GNNs we introduce the Dual-Scale context-Aligmmant GNNs based on the vanilla prompt and few-shot prompting technique in classification tasks. Due to input length constraints, it is challenging to provide examples for many categories. Therefore, for multi-category classification tasks, we employ VCA approach. For binary classification tasks, category to serve as fixed input examples for both training and testing.
the given prompt. In this section,
FSCA and select example from each
B. VANILLA CONTEXT-ALIGNMEN (VCA)
We only replace the prompt content in Sec. 3.2 with "Predict category (r in total) using previous data:", and the method for constructing the graph remains similar Sec. 3.2 where x denotes the number of categories.
B.2 FEW-SHOT PROMPTING CONTEXT-ALIGNNAN (FSCA)
We replace the prompt content in Sec. 3.2 with "Predict category (a in total) using previous data:", and arrange the input embeddings as the format 10, where every element belongs to an M M-dimensional space.
(e e
(1) (1) .... m y
((1)
(1)
=00 (
(1+1) e (1+1)
(I+1)
(++1)
...
m
(10)
(N) (k)) (z( (1+1) 2m (+11) (,,,...,zz. are token embeddings
..... ) fixed examples during training and testing phases, where is from the k-th categories and k I. (k) is the embedding the correct label for {e) (k) ..... (k)) (e((+1) ..... e(+11) is the TS that needs be classified. In the fine-
prompt. We utilise {e( (k)
(ef .....e,, (k)
grained GNN Gr, each element the form 10is treated as a node. Similar to Sec. Bec.3..aand Sec. [3.] Every element in the form 11 belonggs to
two learnable linear layers fe. fa map form 10 to form 11
16

---

Published as a conference paper at ICLR 2025
an M-dimensional space.
(e0), y yy",...., , "0) ,
(11)
For the coarse-grained GNN Gc, construct directed edges based logical relationships as deseribed in formula [2 The first term indicates that the TS data provides information for the prompt, while the second term implies that ( is the output result of the prompt.
(Ec e(h) < 1,. ... + (Ec *( - y (x) ......... 11.
(12)
The directed edges of Gp are decompositions of the directed edges of Gc, which can be represented by the formula 13
{Ep c + 2j 1i 1, n,j 1,..., ,. .
(k)
(13)
U(Ep 2) + y (k 1, l..... m, 1....,.1. 17.
(1)
Since LLMs have strong comprehension abilities for language modality prompts, directed edges in GF, transforming form 13 into form 14 i.e., TS tokens and label embedding only connected to the first and last tokens of the prompt, thereby prevent overfitting. We constrain Σm= un(*) 1 for the first type edges, w((k) for the second type edges. (un()) )i=1 proportional to the cosine similarity between node embeddings. The updated node embedding
prune the
matrices for GF and Go denoted Np and Nc, respectively, which is similar to formula 3
[EF : e (k) (k:) 1, ,,...,,7, n 1, ...., | [E = (*) I,...,,, .). (14)
Similar to the demo in Sec. 3.2, we introduce an assignment matrix and a learnable weight matrix to achieve learnable interactions between the two scales. Both Np and Ño are input into pra-trained LLMs. The Dual-Scale Context-Alignment GNNs can be flexibly integrated into various layers of pre-trained LLMs as depicted in Fig. | and only the first time apply it needs fe and fz to obtain coarse-gras coarse-grained GNN. The output from the GF branch used to compute the cross entropy loss against the ground truth.
C FULL RESULTS
C.I LONG-TERM FORECASTING FULL RESULTS
Table presents the detailed long- terg-termm forecasting results across four prediction horizons. Com- pared to other models, including LLM-based methods, Transformer-based models, and other high- performing approaches, FSCA demonstrates strong and relatively stable performance across var- ious datasets. It achieves average 3.1% MSE reduction compared to the second-best method, PatchTST, and outperforms LLM-based methods ISLM, Time-LLM, and GPT4TS) by 7.3%, 12.2%, and 16.6%, respectively. We attribute this consistent advantage to the successful Context- Alignment, which effectively guides LLMs' deep understanding of time series data, with demon- stration examples prompt serving key factor in enhancement.
C.2 SHORT-TERM FORECASTING FULL RESULTS
We present the comprehensive short-term forecasting results in Table II] Under different frequency settings, FSCA consistently outperforms most baseline models.
C.3 FEW-SHOT FORECASTING FULL RESULTS
Table presents the detailed results of the few-shot forecasting trained 5% data across dif- ferent prediction lengths. Except for DLinear's 's strong performance on the ETTml dataset, FSCA demonstrates significant enhancements in most scenarios across various datasets, outperforming the second-best model, IIIM-LLM, by 6.7%.
Table 13 presents the detailed results of the few-shot forecasting task using 10% of the training data. FSCA demonstrates superior performance, outperforming the baseline across nearly all settings, with average MSE reduction of 7.8% over the next best model, GPT4TS.
17

---

Published as a conference paper at ICLR 2025
Table 10: Full results of long- term forecasting tasks, all results are based on different prediction horizons: 24,
36, 48, 60 for ILI and 96, 192. 336, 720 for others. Bold: best, Underline: second best.
Methods
ISCA
1P
Time
GPT4TS
|/Transoommer
DLinear
PacCTTT
TimesNet
FEiDoomme
Suzionaay
ETS/omme
MSE
MAE]
MSE
MAE
MSE
MAE]
MAE
MSE
MAE
MSE
MAE
MAE
MSE
MAE]
MSE
MAE
24
1,206
0.728
1,467
0,778
1.622 0,806
2,063 0.58
1,694
0.874
2.215
1,061
1,19
0.754
2.317
0.934
3.226
1.260
2.294
0.545)
2.527
1,020
36
1,251
0.750
1.534
0.843
1.695
0.857
1,868
0.892
2.229
0.983
1.963
0,963
1,400
0.834
1.972
0.920
2,679
1,080
1.825
0.848
2,615 1.007
1.566
0.118
1,608
0.836
1.654
0.863
1,790
0.584
2.382
0.995
2.130
1.004
1,553
0.815
2.038
0.940
2.622
1.078
2,010
0.900
2.359 0.972
1,495
0.835
1,860
0,905
1,979
0.957
1.988
0.913
2.368
1,066
2.027
0.928
2.857
2.178
0,963
2.487
1,380
0.783
1,552
0.826
1.713
0.858
1,925
0.905
2,073
0.941
2.169
1,041
0.791
2.139
0.931
2:447
1,144
2,077
0.914
2.497 1.004
96
11.146
0.196
1.149
0.2000
0.163
0.230
00102
0.212
012253
0.304
0.176
0.237
0,149
0.19%
11.122 0.225
0.217
0.296
0,173
0.223
0.397 0.281
Wencher
192
0.193
0.241
1.195
0.244
0.205
0.245
0.2044
0.248
0.240
0.319
0.220
0.252
0.194
0.241
0.211
0.261
0.276
0.336
0.345
0.285
0.237 0.312
336
00.244
0.279
0.246
0.290
0.257
0.267
0.254
0.286
0.344
0.265
0.319
0.245
0.387
0.380
0.306
0.339
0.350
0.921
0.338
0.208 0.353
0.314
0.333
0.320
0.336
0.332
0.306
0.337
0.374
0.333
0.562
0.365
0.359
0.410
0.352 0.288
Avg
11.224
0.262
1.228
0.265
0.257
0,200
0.257
0.270
01.004
0.335
0.248
0,300
0.225
0.364
0.255
0.287
0.309
0,300
0.288
0.314
0.277 0.334
96
0.128
0.322)
0.138
0.254
0.140
0.236
0.139
0.228
0.347
0.248
0.140
0.239
0.129
0.222
0.168
0.272
0.193
0,308
0.189
0.273
0.187 0.304
192
0.346
0.139
1.153
0.252
0.150
0.249
0,155
0.251
0.305
0.267
0.153
0.249
0.157
0:240
01314
0.289
0:201
0.315
0.182
0.256
0.119 0.315
BCL
336
0..63
0.259
0.169
0.270
0.168
0.267
0.109
0,266
0.17%
0.279
0.169
0.267
0.163
0.259
01300
0.300
0.214
0.329
0.200
0.304
012122 0.329
720
0.199
0.287
0.204
0.293
0.209
0.302
0,206
0.297
0.322
0.398
0.203
0.301
0.197
0.290
0.220
0.320
0,246
0.355
0.222
0.321
0.233 0,345
Ave
01.159
0.252
11106
0.262
0.167
0,264
0,107
0.265
012003
0.298
0.166
0,263
0.161
0.263
[0,392
0.245
0.214
0,327
0,193
0.296
0.208 0.325
96
0.355
0.346
0.379
0.274
0.384
0,278
0,558
0,282
0.367
0.288
0.410
0.282
0.550
0.241
0.593
0.321
0.567
0,366
0.612
0,338
0.607 0.392
Thaliic
192
0.377
0.255
0.307
0.283
0.398
0.256
0.407
0.200
0.37%
0.205
0.423
0.287
0.379
0.356
0.617
0.336
0.604
0.373
0.613
0.540
0.621 0.399
0.387
0.365
0.269
0,412
0.20
0.350
0.294
0:436
0.206
0.609
0.336
0.621
0.363
0.615
0.328
0.622 0.396
720
0:425
0.287
0.440
0.30
0.436
0.303
0,450
0.31.
0,401
0.304
0.466
0.315
0.432
0.286
0.640
0.350
0.626
0.382
0,653
0.355
0.632 0.396
Arg
01336
0.263
0.405
0,286
0.407
0.289
0,414
0.29-
0.389
0.295
0.433
0,295
0,390
0.36.
[1620
0.336
0.610
0,376
0,624
0.340)
0.621 0.296
96
0.349
0.389
11:67
0.395
0.383
0,400
0,376
0.997
0..95
0.425
0.375
0.395
0.570
01.999
01.844
0.401
0.376
0.419
0.513
0.491]
0.4444 0.479
BThh
0.390
0.415
0.402
0.422
0.427
0.431
0.416
0.418
0.427
0.441
0:405
0.416
0.413
0.421
01446
0.429
0.420
0.445
0.554
0.50a
0.538 0.504
0,402
0.432
0.432
0:451
0.430
0,476
0.442
0.433
0.445
0.457
0:439
0.443
0.491
0.469
0:459
0.535
0.574 0:521
730
0.433
0.460
0:472
0.474
0.465
0,459
0.477
0.456
0.537
0.530
0.472
0,490
0:447
0.466
0.521
0.500
0.506
0,507
0,643
0,616
0.562 0.535
Avg
0.394
0.424
(1418
0.486
0.426
0.435
00.47
0.426
0.441
0.462
0:422
0.437
0.413
0.430
(1.458
0.450
0.440
0.460
0.570
0.537
01.542 0.510
96
0.256
0.328
0.284
0.345
0.243
0.348
0.285
0.342
0.204
0.360
0.289
0.353
0.274
0.336
0.340
0.374
0.358
0.397
0,476
0.458
0.340 0.391
B112
0.311
0.372
1.249
0.387
0.356
0.391
0.354
0.589
0.405
0.383
0.418
0.339
0.379
01,422
0.414
0.429
0,439
0,512
0,493
0,430 0.439
336
0.308
0.372
0.268
0.417
0.372
0.408
0,573
0.407
0,405
0.429
0.448
0.465
0.329
0.380
0:452
0.452
0.496
0,557
0,582
0.551
0,485 0.479
720
0,390
0.428
0.419
0.445
0.421
0.446
0,406
0.441
0.443
0.464
0.605
0.551
0,379
0.422
0.462
0.468
0.463
0,474
0.562
0.500
0,500 0.497
Ave
0.316
0.375
11355
0.399
0.361
0.396
0.354
0.394
0.382
0.414
0.451
0,446
0.330
0.379
[1.414
0.427
0.437
0,449
0.525
0.510
0.439 0.452
96
0.262
0.343
0.291
0.365
0.294
0.345
0.292
0.34g]
0.312
0.366
0.299
0.343
0.290
0.342
0.338
0.375
0.379
0.419
0.386
0.398]
0.375 0.395
E77ml
0.304
0.169
0,323
0.368
0.330
0.568
0.352
0.372
0.385
0.935
0.365
0.352
0.374
0.187
0:426
0.441
0.499
0.444
0.408 0.410
336
0,356
0.386
0.361
0.392
0.365
0.392
0,366
0.394
0.379
0.404
0.369
0,386
0,366
0.797
0.410
0.411
0.445
0.459 0,495
0.464 0.435 0.428
720
0.405
0.417
0.410
0.420
0.427
0.431
0.417
0.421
0.441
0.142
0.425
0.421
0,416
0.420
0.478
0.150
0.543
0,490
0,585
0.51e
0.499 0.462
Avg
0.342
0.378
0.346
0.382
0.354
0,364
0.352
0.383
0.370
0.399
0.357
0.378
0.351
0.280
0.400
0.405
0.408
0,452
0,481
0.450|
0.429 0.425
0.164
0.254
0.167
0.257
0.175
0.265
0.173
0.26.2
0.179
0.271
0.167
0.309
0.165
0.255
0.187
0.267
0.203
0.267
0.192
0.274
0.189 0.280
B11m2
0.222
0.296
11227
0.305
0.245
0.316
0.229
0.301
0.342
0.315
0.224
0.303
0.292
0.349
0.309
0.269
0.328
0,280
0.330
0.253 0.319
336
0.269
0.326
0.785
0.346
0.294
0.343
0,286
0.341
0.288
0.344
0.781
0,342
0.274
0.339
0.321
0.351
0.325
0.366
0.354
0.361
0.314 0.357
720
0.346
0.381
11368
0.395
0.389
0.430
0,375
0.401
0.37M
0.397
0.397
0.421
0.302
(1.345
01.00M
0.405
0.421
0.415
0.417
0.413
0.434 0.415
Avg
0.250
0.314
(.362
0.326
0.275
0.334
0.206
0.326
0.372
0.331
0.267
0.333
0.255
0.315
01.201
0.333
0.305
0.349
0.906
0.347
0.203 0.342
Table 11: Full results of short-term time series forecasting on M4, with prediction horizons ranging from [6. 48]. The last three rows weighted averages across all datasets under different sampling intervals. Bold: best, Underline: second best.
Matleds
FSCA
SPI--IM
TimesLLM
GPTITS
[Transfommn
Diincar
PacklES
N-HIS
N-BEATS
TimseNic
FEDifomaz
Statinsee
Quan. TOr.
SMAP::
13.288
13.41
13:550
15.110
13,622
16.965
13,477
13:422
134877
15.97
14:001
14.727
MASE
2.74
3002
3,055
3.565
3095
4.255
3019
3,056
3/036
3.554
3.073
OWA
0.781
0.79
0.805
0.911
0.807
10558
0.792
0.755
0.795
0.918
0.311
0.807
SMAPI
18,037
10.35
10.673
10.99
10.353
12,145
10,300
10.185
10354
10,405
13.100
10.95%
MASE
1.174
1.288
1,253
1,209
1,500
1150
1.252
1.277
1.325
OWA
0.854
0.922
0.950
0.938
0.911
1.106
0.21
0.895
0.936
0.923
0.966
0.051
Mooth
SMAPP
127702
12:955
13.416
13.258
13,079
13,514
12,599
13.059
13,099
13,513
14.403
13.917
MASE
0.447
0.970
1.045
1.003
0.974
10077
0.970
1.013
0.996
1.029
1.147
1.097
OWA
0.897
0.900
0.57
0.911
0.91
0.96
0.955
0.929
0.922
0.957
1.088
0.988
Aog. Unber:
SMARE
61144
4.78
6.709
4711
6.599
6.133
7.148
6.302
MASE
3.202
3.247
3.412
4.116
3,231
4955
3,347
3.054
4.430
4.507
4.054
4.064
OWA
1.007
1.017
1,259
1.012
1L977
1.393
1.438
1.304
SMAPE
11.828
12:021
12.494
12,090
12,142
13,639
12.089
12,055
12.250
12,590
13.100
12,780
MASE
1.500
1.612
1.731
1.808
1.631
2.055
1.623
1.625
1.698
1.836
1.756
OWA
0.830
0.657
0.933
0.940
0.874
1.051
0.859
0.869
0.806
0.955
0.949
0.930
C.4 ZERO-SHOT FORECASTING FULL RESULTS
Table 14 provides detailed results of the zero-shot forecasting task across different prediction lengths. FSCA consistently achieves optimal performance across all settings, with a significant margin. Compared to the second-bes, -bst method, PatchTST, it shows an average improvement of
13.3%. Additionally, FSCA achieves MSE reductions of 18.3%, 17.7%, and 24.3% over other LLM-
18

---

Published as a conference paper at ICLR 2025
Table 12: Full results of sho- learning 5% training data. All results are from four different prediction horizons (96. 192, 336, 7203. A lower MSE indicates better performance. Bold: best, Underline: second best. means that 5% time series is not sufficient to constitute a training set.
Methods
FSCA
|ime-11M
GPTTTS
[Transformer
Di Diiine
PischTST
TimecNet
PRDffomer
Stationayy
FTSSfommer
Metric
MSE
MAE]
MSE
MAE]
MSS
MAE
MSE
MAE] MSE
MAE
MMSE
MAE MSE
MAE MSE
MAE] MSE
MAE
MSE
MAR]
MSE
MAE
96
11,422
0.457
0.500
0.495
0.511
0,406
0,543
0.610
0.547 0,503 0,557 0.519 (.892 0.625)
0,529
0,952
0.650
1,105 03832
192
11.337
0.492
0,000
0.530
0,702
0.547
0,748
0.580
0.908 0.658
0.720
0.604
0.711 0.570
0.940
0.665
0.652
0,563
0,943
0.645
1,221 0.53
336
0,707
0.574
0.761
0.620
0.725
0,603
0,754
0.505
1,475
0.861
0.984
0,727
0,816
0.619
0.945
0.683
0.731
0.504
0,935
0.644
1,179 0.832
720
Aug
0.575
0.508
0.650
0.648
0.549
0.681
0.560
1,000
0.710
0.750
0.611 0.695
0.569
015225
0.647)
0.658
0,943
0.6466
1.189 0.839
11.322
0.352
11365
0.384
0,420
0,576
0.421
0.427
0.442
0,456
02400
0.409
0.420)
0,390
0.424
0,400
0.425
0.67M 0.619
192
11:302
0.409
11375
0:411
0.394
0.424
0,418
0.44]
01433
0.445
0.617
01542
01452
0.445
(1.483
0.464
0.457
034455
0.497
01466
018445 0.697
336
0.397
0.490
0.403
0.421
0.416
0,453
0.408
0.479
0.631
0.555
0.849 1.424
0.464
0.469
0.409
0.479
0.477
0.483
0.507
0:481
0.005 0.727
720
Avg
0.366
0.397
0.380
0.415
0.396
0,426
0,400
0.433
0,475
0.827
0.615
01448)
0:463
0.454
0.463
0.454
0,470
0.489
0,800 0.681
96
0.355
0.285
0.257
0,340
0.412
0,424
0,396
0,405]
0.589
0.510
0.332
0.574
0,399
0.41.
0.006
0.518)
0.628
0,544
0.823
0.587
1,001 0.747
192
0.397
0.405
0.432
0.434
0.448
0,440
0,440
0.436
0,703
0.565
0.358
0.390 0,441
0.436
0.681
0.539
0.666
0,566
0,844
0.591
1,087 0.766
336
0:450
0.440
0.440
0.442
0.452
0.647
0.485
0.459
0.898
0.641
0.402
0.416
0.499
0.167
0.786
0.597
0.807
0.628
0.870
0.603
1,138 0.787
720
0.538
0.486
0.593
0.521
0.585
0.491
0.577
0.499
0.948
0.671
0.511
0.459
0.767
0.587
0.796
0,593
0.822
0,633
0.893
0.611
1.245 0.831
Arg
0.435
0.429)
11455
0.451
00,472
0.450
0.400
0.417 0,520
01.717
0.561
0.730
0,857
0.59%
1,325 0.782
96
11.199
0.374
0.197
0.275
0.205
0.277
0,1999
0.2800
0.365
0.389
0.256
0,336
0.206
(1.388)
(1.20)
0.299
0.229
0,,30
0.258
0.316
01,404 0:485
192
0.250
0.311
0.254
0.920
0.267
0.356
0.255
0.31.
0.310
0.362
0.906
0,373 0.264
0.334
0.311
0.361
0.394
0,361
0.296
0.349
0.479 0.521
336
0.298
0.341
0.315
0.350
0.309
0.347
0.318
0.353
0.373
0.399
0.380
0,423 0.334
0.267
0.338
0.366
0.378
0.427
0.353
0.350
0.552 0.555
0.399
0.403
0:421
0.421
0,460
0.454
0.674
0.583
0.509
0.165
0.523
0.445
0,701 0.627
Avg
0.284
0.382
0.206
0.342
0.307
0,348
0.308
0.346
0.356
0.388
0.399
0,436
0,514
0.352
0.344
0.372
0.381
0.404
0,343
0.372
0.534 0.547
Table 13: Full results of few-shot learning on 10% training data. All results are from four different prediction horizons (96, 192, 336, 7203. A lower MSE indicates better performance. Bold: best, Underline: second best.
Methods
PSCA
|S
Timo-1M
GPT-TS
Tmansformerr
Di Inzinzar
PStcTTTT
TimesNer
|EDDormer
Stationary
ETStormer
Mouic
MMSB
MAE]
MSE
MAB
MSE
MAE
MSE
MAB
MSB
MAE
MMSB
MAB
MSE
MAE
MSE
MAL]
MSE
MAE]
MSE
MAL
MSE
MAE
96
0.449
0.468
0:481
0:474
0,720
0.533
0.455
0.456
0,790
0,566
0,492 0,495
0,516
0,485 0.861 0.628 0.512 0.499
0,918 0.639
1,112 0.806
192
0.491
0.469
0.518
0.491
0.747
0.545
0.570
0.516
0.837
0.609
0.565 0,538
0.598
0.524
0.797 0.593
0.624 0.555
0,915 0.629
1.155 0.823
336
0.549
0.499
0.570
0.793
0.551
0.608
0.535
0.780
0.622
0.657
0.941
0.688 0.691 0.574
0.939
0.641
0.832
720
0.661
0.559
0.584
0.830
0.584
0,725
0.291
1,254
0,986
0.743
0,762
0.610
0.577
0.641 0.728
0.614
0.887
0.645
1,273 0,874
Avg.
0.538
0.494
(1.533
0.529
0.765
0.553
0,500
11.25
0.910
0,800
0.691
1.600
0,633
0,542
0.869
0.625
0.639
0.561
0.915
0.639
1.150 0.834
96
0.287
0.351
0.354
0.400
0.334
0.381
0.331
0.374
0,904
0.435
0.357
0,411
0.355
0,389
0.376
0.409
0.382
0.110
0,389
0.411
0,878 0.619
ET122
192
0.351
0.392
0.401
0.425
0.430
0.438
0.600
0.431
0.470
0,474
0,509
0,519
0.403
0.414
0.490
0.467
01478
0.474
0,473
0.455
0.755 0.606
336
0.386
0.420
0.450
0.449
0.45%
0.40%
01433
0.489
0.671
0.572
0.426
0.441
0.537
0.494
0.504
0.501
0.507
0.430
0,839 0.004
720
0.426
0.447
0.480
0.486
0.485
0.490
0.464
0.593
0.834
0.648
0.477
0.491
0.499
0:477
0:422
1,273 0.874
Avg
0.363
0.403
0.419
0.439
0.434
0.441
0.507
0.421
0.489
0,483
0,605
0.538
0.415
0.451
0:479
0.465)
0.466
0.475
0:462
0.455
0.894 0,713
96
0.371
0.3933
0.388
0
0,412
0.422
0,390
0.404
0,709
0.556
0.352
0,392
0.410
0,419 0.583
0.50
0.578
0.518)
0.761
0.568)
0,911 0,688
192
0.405
0.407
0.422
0.421
0.647
0.438
0.429
0.423
0,717
0,548
0.382
0,412
0.437
0.434 0.630
0.528
0.617
0.546
0,781
0.574
0.955 0,700
376
0.444
0.424
0.456
0.430
0.407
0.465
0.460
0.430
0.735
0.575
0.419
0:434
0.476
0.454 0.725 0.568 0.098
0.775 0.803
0.587
0.901 0.719
720
0.520
0.468
0.554
0.490
0.594
0.521 0.569
0.496
0.752
0.490
0:477
0.681
0.556 0.769
0.569 0,693
0.579 0.844
0.581
Avg
0.435
0.423
0.455
0.435
0.687
0.461 0.464
0.441
0,728
0.565
0.411
0:429
0,501
0.466
0.677
0.537 0.722
0.605
0.797
0.578
0.980 0,714
96
00119
0.270)
0.192
0.274)
0.224
0.296 0,188
0.269
0.245
0.322
0.213
0,303
0.191
0.274
0.212
0.285
0.291 0.299
0.229
0.308)
0.331 0,430
E17m2
192
0.342
0.306
0:246
0.313
0.200
0.317 0.251
0.309
0.274
0.538
0.278
0.545
0.252
0.317 0.270
0.325
01307
0.379
0.291
0.543
0,400 0.464
336
0.286
0.332
0.340
0.312
0.349
0.507
0.346
0.361
0.304
0.338
0.385
0.306
0.323
0.353
0.543
0.559
0.548
0.376
0,409 0.438
0.376
0.386
0.400
0.403
0.434
0.416 0.426
0.467
0.436
0.440 0.433
0.427 0:474 0.449
0.712 0661
0:461
0:488 0.580
Avg
0.374
0.324
0.284
0.332
0.305 0,344 0,293
0.335
0,336
0,373
0,316 0,368 0.296 0,343 0.320 0.353 0.463 0.488 0.332 0.366
0,447 0,457
based methods S'IP-LLM, Time-LLM, and GPT4TS, respectively. This improvement is attributed to FSCA's successful activation of LLMs' structured understanding of TS data, while the design of the Dual-Scale Context-Alignmen GNNs ensures that LLMs grasp the logical relationships in demonstration examples prompt.
C.5 CLASSIFICATION FULL RESULTS
presents the results on 10 multivariate UEA classification datasets. In the time series clas- sification task, we have augmented the baselines with the following methods: XGBoost (Chen &l Guestrin 2016), Rocket (Dempster et al. 2020 LSTNet (Lai et al. 2018a), LSSLL (Gu et al. 2021) LighTTS (Zhang et al.] (022), Pyraformer (Liu et al. 2021), TCN (Franceschi et al. 2019), and Flowformer (Huang et al. 2022.. Compared to classical methods, RNN-based, Transformer based, MLP-based, and other LLM- based approaches, FSCA demonstrates consistently superior perfor- mance. It achieves an average accuracy improvement of 2.4%, 2.7%, and 2.8% over I'IP-LLM, Time-LLM, and GPT4TS, respectively. This demonstrates the versatility of FSCA, showing that
Table 15
19

---

Published as a conference paper at ICLR 2025
Table 14: Full results of Zero-shot learning: the first column A B indicates training on dataset A and testing on dataset B. Bold: best, Underline: second best.
Methods
FSCA
1'
LLM
Time LLM
GPT4TS
(Trnndoomer
DLinear
PatcTST
TimesNet
Metric
MAF
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAR
MSE
MAB
MSE
MAE
0.256
0.323
0.315
0.377
0.324
0.368
0.335
0.374
0.353
0.394
0.367
0,600
0.304
0.350
0.358
0.367
ETThl
192
0,310
0.361
0.402
0.407
0.398
0.396
0,412
0.417
0.437
0.445
0.447
0,460
0.386
0.400
0.427
0.429
336
0.513
0.375
0.453
0.432
0.410
0.425
0.441
0.444
0.422
0.476
0.315
0,508
0.414
0.428
0.449
0.451
ETTh2
0.374
0.419
0.442
0.451
0.403
0.449
0.488
0.452
0.556
0.506
0.665
0,589
0.499
0.443
0.448
0.458
Avg
0.369
0.403
0.417
0.384
0.409
0,406
0.422
0.477
0.455
0.493
0,488
0.300
0.202
0.295
0.242
0,319
0.256
0.320
0.236
0.315
0.247
0.319
0.255
0.357
0.215
0.304
0.239
0.315
ETTh1
192
00.258
0.329
0.286
0.357
0.265
0,353
0.2877
0.342
0.293
0.350
0.338
11.413
0,2775
0.339
0.291
0.342
336
11311
0.360
0.351
0.367
0.357
0.576
0.341
0.374
0.364
0.419
0.425
0.465
0.334
0.373
0.342
0.371
FTTm2
720
0,300
0.407
0.422
0.416
0.429
0.430
0.435
0.422
0.514
0.470
0.640
0.573
0.431
0.434
0.434
0.419
Avg
0.325
0.360
0.317
0.370
0.325
0.363
0.360
0.390
0.415
0.452
0.314
0.360
0.307
0.361
0,475
0.475
0.668
0,567
0.618
0.515
0,732
0.577
0.854
0.606
0.689
0.555
0:485
0.465
0.548
0.601
ETTh2
192
0,520
0.500
0.575
0,526
0.715
0.570
0,758
0.559
0.963
0.615
0.707
01.588
0.505
0.509
0.500
0.610
336
11528
0.512
0.655
0,577
0.636
0.523
0.759
0.578
0.867
0.626
0.710
0,577
0.583
0.511
0.867
0.626
ETTh
720
01586
0.545
0.778
0.568
0.683
0.553
0.781
0.597
0.587
0.654
0.704
0.596
0.628
0.561
0.887
0.648
Avg
0.669
0.560
0.663
0.757
0.578
0.625
0.703
0.574
0.565
0.965
0.621
0.200
0.293
0,505
0.258
0.326
0.253
0.329
0.244
0.390
0.240
0.336
0.236
0.309
0.248
0.324
ETTh2
192
0.225
0.326
0.295
0,344
0.303
0.342
0.299
0.346
0.291
0.356
0.295
113009
0.289
0.345
0.296
0.352
336
0.310
0.358
0.340
0.376
0.356
0.383
0.347
0.376
0.351
0.391
0.345
0.597
0.348
0.379
0.353
0.983
ElTm2
720
0,384
0.409
0.453
0.428
0,460
0.434
0:446
0.429
0.452
0.451
0.432
0.442
0.439
0.427
0.471
0.446
Avg
0.347
0.327
0.362
0.339
0.371
0.335
0.370
0.335
0,382
0.328
0.386
0.325
0.365
0,342
0.376
0,302
0,361
0.358
0.382
0.355
0:401
0.353
0.392
0:371
0.407
0.365
0.415
0.354
0.385
0.377
0.407
ETTml
192
0,354
0.395
0.454
0,444
0.449
0,450
0:443
0.437
0.63
0.458
0.454
0.462
0.447
0:434
0:471
0:453
336
11:49
0.398
0.488
0.452
0.479
0.467
0.469
0.461
0.481
0.485
0.496
01404
0.483
0.463
0.472
0.484
ET
0.467
0.438
0.469
0.478
0:477
0.476
D:466
0.468
0.503
0.483
0.541
0.529
0.474
0.471
0.495
0.482
Avg
0.442
0.439
0.440
0.449
0.431
0.439
0.455
0.458
0.464
0,475
0,439
0.438
0,457
0.454
0.175
0.261
0.299
0.271
0:217
0.294
0:319
0.305
0.221
0.314
0.195
0.271
0.222
0.295
ETTml
192
0.231
0.298
0.272
0,325
0.288
0.335
0:277
0.327
0:227
0.347
0.786
0.359
0.258
0.311
0.288
0.337
356
0.285
0.334
0.303
0.347
0,322
0.355
0,331
0.360
0.354
0.378
0.357
0,406
0.317
0.348
0.341
0.367
ETTm2
720
0.363
0.383
0.436
0.418
0.414
0.409
0:429
0.413
0.426
0.420
0.476
0.476
0,416
0.404
0:436
0.418
Avg
0.264
0.319
0.304
0.347
0,311
0.343
0.313
0.248
0.319
0,363
0.355
0.389
0.296
0.334
0.322
0.354
0.345
0.383
0.416
0.560
0.347
0.333
0.327
0.360
ETTm2
192 0,349
0.393
0.403
0,422
0.439
0.441
0:434
0.437
0438
0.444
0.441
0,456
0,411
0.418
0:434
0:437
336
0.343
0.398
0.434
0.642
0.455
0.457
0,660
0.459
0.459
0.464
0.505
0,500
0.439
0.447
0.460
0.459
ErTb2
720
0,401
0.437
0.462
0.467
0.488
0.479
0.483
0.477
0.485
0.477
0,543
0,534
0,059
0.470
0.485
0.477
Avg
0,343
0.393
0.406
0.429
0:429
0.448
0:435
0.443
0/422
0.447
0:455
0,409
0:425
0,435
0,401
0.524
0.485
0.445
0.747
0.558
0.619
0.564
0.570
0.437
0.747
0.558
ETTm2
192
0.457
0.453
0.609
0.501
0.555
0.464
0.781
0.560
0.685
0.565
0.590
0.506
0,530
0:470
0.781
0.560
336
0.497
0.480
0.585
0,522
0.608
0.538
0,778
0.578
0.792
0.578
0.706
0.567
0.565
0.497
0.778
0.578
ETTml
720
0.563
0.502
0,712
0.579
0.699
0.506
0.769
0.573
0.727
0.579
0.731
0.581
0.696
0.565
0.769
0.573
Avg
0,450
0.463
0.622
0.532
0.588
0.508
0.769
0.567
0.706
0.572
0.649
0,537
0.568
0.492
0.769
0.567
its framework can be effectively applied beyond time-series forecasting tasks to achieve excellent performance in other tasks as well.
Table 15: Full results for the classification task. second best.
indicates the name of *former. Bold: best, Underline:
Mitieds
NGBhord
Realot
1.TNN
LSS
TCN
Tixss
Ro
In
Pyra
iTum:
Suron
FED
FTS
Floze
LighTTS
TixnNN
G1FT-755
Tixv:I1M S7IM-1M FSCA
Falan norationn
65.7
65.2
194
11.
13.3
31.5
11.
12.
11.3
11%
376
15.7
117
'5.1
14.3"
Hesdesting
15.8
08.1
05.8
53.3
27.
524
31.6
75.0
33-5
22.0
13.1
131
55.1
Borrhar
71.3
75.4
77.
13.2
766
36.
721
385
754
75.
73.7
13.3
21.
17.
261
25.1
78.1
77.
38.01
77.5
16.5
06.2
08.1
99.4
9699
96.9
99.2
98.4
05.9
962
98.4
08.6
160
86.5
16.3
86.0
505
96.5
12.2
964
901
88.1
93.7
$9.4
85.7
89.
91.
97.3
20.8
91.3
13.3
$1.5
91.4
142
43.9
53.3
52.8
13.2
55.6
53.0
563
5533
53.3
55.6
59.3
5444
35.0
56.1
30.5
$1.1
57.2
312
38.3
68.1
09.6
1880
91.0
198.0
1660
91.4
001
19.5
Mosiag
00.0
72.3
71.
100
703
710
715
721
125
72.
100
710
730
075
7044
13.
740
753
16.
D VISUALIZATION
Figure presents visualization examples of FSCA prediction results on the ETThI, ETTml, Elec-
tric, and Traffic datasets with an input length of 512 and a prediction length of 96. It can be observed that FSCA achieves good predictive performance across various datasets.
20

---

Published as a conference paper at ICLR 2025
-1
Cooondroch Prodenon
Predectio
400
300
2006
100
(a) ETThl dataset example
(b) ETThl datasct example 2
100
400
300
100
2006
500
400
500
600
(c) ETTml dataset example
(d) ETTml dataset example 2
Producoo
400
(c) Traffic datasct example
(f) Traffic dataset example 2
Drennithoch Produciis
Coundrww Producse
7
/00
200
269
400
306
600
150
206
300
500
600
(g) Electricity dataset example
(h) Electricity dataset example
Figure 3: Sequential visualization examples of prediction results on the ETTh1, ETTm1, Electricity, and Traffic datasets presented, with two examples per dataset. The blue lines represent predic- tions, while the orange lines indicate ground truth. The visualizations start at x-axis position 512.
21

---

Published as a conference paper at ICLR 2025
E SCHEMATIC DIAGRAM OF THE FRAMEWORK
FSCA
Claceification
Comnaggaane
LLM
LLM
Leamablee interaction
4
I
Fin grains GMN
LLM
LLM
LLM
Conabrtt Graph
Impur
Band Paameea
Input
puumping
Figure 4: Schematic diagram of the process of Vanilla Context- Allggmmet (VCA) and Few-Shot Prompting based Context-Alige Alignment (FSCA). The light-colored square sequence represents the fine- grained branch; the dark-colored square sequence represents the coarse-grained branch.
In this section, we describe our method in greater detail and specificity through a diagram and quantitative examples.
The bottom of Fig. features the TS data input. The left subfigure presents the Vanilla Context- Alignment (VCA) process for any TS tasks, where the TS input is tokenized together with the task description prompt to construct the simple graph structure; edges in both coarse-grained and fine- grained graphs are built from the TS input directed toward the task prompt. The middle subfigure illustrates Few-Shot Prompting Context- Context-Aligneen (FSCA) for forecasting tasks, which is detailed Sec. 3.3 showing the division of the TS input into subsequences, with the subsequences posi- tioned later serving the ground truth for earlier ones, thereby forming the edge connections. The right subfigure depicts the classification task using FSCA (detailed in Sec. B.2), differing from fore- casting tasks in that, unable to segment the TS input to create examples, we must extract one sample from the training set for each category as the fixed example.
Below, we would facilitate the understanding of FSCA for tackling forecasting tasks by detailing simple example. This could also provide clearer explanations of Eq Eq|o) The processes VCA and FSCA for classification tasks are similar and would not be reiterated.
Assuming we have inputs processed through patching and token embedding, these include TS em- beddings of length 8 and task description prompt embeddings of length 2 (the prompt is "Predict future sequences using previous data:" in FSCA for forecasting, here the length an example):
Firstly, for the fine fine-grained branch, consider the scenario in which the input TS embed- dings are segmented into subsequences, each comprising four embeddings. Thus, TS1 yub is [e1,1, 01,,,,, e1,4), and 752 sut is [e2,1, (2,,, 62,3, 62,4], where eij indicates j-th embedding in subsequence i. Similarly, Zi,j refers to j-th embedding in the prompt of subsequence i. Here, [21,1, 212] [22,1, 22.2]. Ultimately, Eq instantiated as:
[e1.1, 61,,, G1,3, E1,4; 21,1, 21,2, 62,1; 62,2, 62,3, 62 22,1, 22.2)
Secondly, we need to construct graph structure for this input before enters LLM. The basic logic for constructing the graph is that T sub serves the ground truth for TS! 2ub (The latter subsequence serves as the correct lahel for the former subsequence). Specifically, starting with all elements in TSL yub- construct directed edges the first item of the corresponding task description, 21,1- Subsequently, from the last item of the task description, 21,2, construct directed edges to all elements TS2 Since all TS subsequences are used to predict future sequences, the first token of the last prompt, 22,1, needs establish edge connections with both T and T "ub.
Thirdly, for the coarse-grained branch, it is essential to inform the LLM that a time series should be treated as whole. Thus, 7 yub must be mapped individual node embedding by linear
22

---

Published as a conference paper at ICLR 2025
layer. To align the scales, the prompt embeddings also mapped node embedding. Thus, the coarse-grainet sequences can be denoted as [ē]. _ 62, 2 (instantiation of Eq Additionally,
the graph construction logic consistent with that of the fine-grained branch.
F EXPERIMENTAL ANALYSIS
F.1
RANDOMLY REPLACE THE PRE- TRAINED WEIGHTS LLM
GPTNTS
0.44
GPTNTS
0.52
FSCA
FPP-LMM
0.42
8 II-LLMM
0.50
A635
0.40
6.46
0.44
0.34
0.40
0.32
036
20%
40%
66%
80%
100%
056
20%
40%
66%
80%
100%
Randon iniiaalizsiion ratio
Ranion intialization ratio
(a) ETThl dataset
(b) ETTh2 dataset
Figure 5: Results of random initializing GPT-2 pre-trained weights. The x-axis represents the ratio of GPT-2 pre-trained weights replaced by random initialization, and y-axis shows the Mean Squared Error (MSE) metric values. We conducted this experiment GPT4TS, S'IP-LLM and our FSCA.
As shown in Fig. 5 we randomly initialize GPT-2 pre-trained weights at varying ratios to demon- strate scenarios involving under-trained and untrained conditions. the random initialization ratio increases, LLM's contextual understanding ability decreases, leading to a decline in our model's s performance. When the model's capability is weak, our results are as poor as those of GPT4TS (the most direct method to utilize LLM for TS tasks) and I'I (a token-alignment based method). However, as the I.IM's capability improves, our method significantly outperforms GPT4TS and S'IP-LLM, achieving a lower MSE. These findings demonstrate that our approach more effectively activates the potential pre-trained LLMs for TS tasks.
F.2
SCALE ANALYSIS
We analyze the scaling effect of FSCA in three aspects:
Model Size: Ablation experiments on the number of GPT-2 layers (Table 6 show performance
declines as the layer count increases, consistent with findings in GPT4TS.
Training Data Size: As shown Table 17 using 5%, 10%, 25%, 50%, 75% and full of the training data reveals continuous performance improvement, particularly significant at the 50% data point.
Few-Shot Prompting Examples' amount: Table 16 shows that more examples yield modest gains for short prediction lengths (96) but reduce effectiveness for longer ones (336, 720). This likely results from shorter input lengths per example due to divisions of the TS input, creating mismatch with longer required prediction lengths.
F.3 FULLY TUNING ANALYSIS
GPT4TS has demonstrated that fully tuning LLMs for TS tasks, while straightforward, incurs high computational costs and yields suboptimal results by compromising the inherent generic knowledge of LLMs. Instead, our approach, like other pre-trained --seedded TS methods (e.g.. Time-LLM, TEST, I'MM-,,M) focuses freezing most LLM components efficiently harness their potential. As shown Table I8 we still supplement our results with full tuning, which similarly performed suboptimally.
23

---

Published as a conference paper at ICLR 2025
Table 16: Results of different examples amount for the long-term forecasting task. All results averaged different prediction horizons: (96, 192, 336, 720). Bold: best. Underline: second best.
Examples amount
2
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
96
0.349
0.389
0.343
0.385
0.341
0.382
0.356
0.391
ETTH
0.419
336
0.402
0.432
0.407
0,439
0.414
0.445
0.440
0.456
720
0.435
0.460
0.446
0.471
0.462
0.488
0.485
0.495
Avg
0.394
0.424
0.396
0.428
0.403
0.434
0.421
0.443
96
0.282
0.343
0.277
0.340
0.275
0.34]
0.296
0.352
E3'ml
192
336
0.356
0.386
0.366
0.391
0.370
0.395
0.391
0.408
720
0.405
0.417
0.412
0.425
0.428
0.432
0.451
0.438
Avg
0.342
0.378
0.345
0.383
0.351
0.388
0.370
0.391
Table 17: Results of different training data ratios.
Training daia ralios
ETThi
ETTml
10%
0.538
0.435
25%
0.486
0.411
50%
0.366
75%
0.398
0.350
100%
0.394
0.342
F.4 COMPARISON BETWEEN VCA AND OTHER LLM-BASED METHODS
In this section, foeus comparing VCA with other LLM-based methods. The results show that VCA generally performs second only to FSCA, demonstrating the effectiveness of Context- Alignment, which outperforms Token-Alignment and other LLM-based approaches. Compared to FSCA, the results also validate the enhancing effectiveness of the few-shot prompting technique employed in FSCA. Table Table 20 Table 21 Table 22] Table 23 respectively show the results of long-term forecasting, short-term forecasting, few-shot forecasting, zero-shot forecasting and classification tasks.
F.5 EXPERIMENTAL EFFICIENCY ANALYSIS
We incorporate comparisons of experimental efficiency with other LL.M-based methods, focusing on the number of parameters and execution speed. As shown in Table 24 Our method ranks just behind GPT4TS, which only incorporates linear layers at the input and output stages of LLMs. In contrast, other popular methods require token-alignment to adapt LLMs for time series data, aligning TS data with word embeddings in the vocabulary. Additionally, these methods often include extra operations. For example, Time-LLM repeatedly generates prompts and retrieves corresponding embeddings each iteration, while I'IM-LMM separates TS inputs and performs prompt retrieval.
The computational costs of our FSCA method mainly stem from two aspects. Firstly, the use of dual-scale GNNs introduces two learnable matrices (Eq. 3). which, as shown in the table compar- ing FSCA with the version without dual- scale GNNs, add slight increase computational load. Secondly, the process of constructing coarse-grained inputs necessitates two learnable linear layers to transform fine-grained node embeddings into coarse-gras gaansne ones. The input dimension these layers is dictated by the number of input TS patches, making the primary contributor to increased overhead.
F.6 COMPARISON OF DIFFERENT PROMPT TYPES
We compare our approach with two other types of prompts detailed in Table 25 data domain and input statisties, referencing Time-LLM (Jin et al. 2024). Here are the prompt examples:
Table 18: Fully tuning results of FSCA and GPT4TS.
Method
ETThl
ETTml
GPT4TS
0.427
0.352
GFT4TS/Fally mming)
0.469
0.406
FSCA
PSCA(Fully tuning)
0.457
0.383
24

---

Published as a conference paper at ICLR 2025
Table 19: For long-term term forecasting tasks, comparison between VCA and other LLM-based methods. All results are averaged on different prediction horizons: {24, 36, 48, 60) for ILI and (96. 192, 336, ]20} for others. Bold: best, Underline: second best.
Methods
S'IP-LLM
Time- LLM
GPT4TS
MSE
MSE
MSE
MAE
MAE
ILI
1,380
0.783
1.128
0.799
1.552
0.826
1.713
0.858
1,925
0.903
Weather
0.224
0.262
0.230
0.268
0.228
0.265
0.237
0.269
0.237
0.270
ECL
0.159
0.252
0.163
0.257
0.166
0.262
0.167
0.264
0.167
0.263
0.386
0.263
0.389
0.271
0.408
0.286
0.407
0.414
0.294
RTTh1
0.394
01424
0.417
0.432
0,418
0.436
0.426
0.435
ETTa2
0.316
0.375
0.335
0.382
0.355
0.399
0.361
0.398
0.354
0.394
ETTml
0.342
0.378
0.349
0,380
0.346
0.382
0.354
0.384
0.352
0.383
KTTW2
0.250
0.314
0.259
0,318
0.262
0.326
0.275
0.334
0.266
0.326
Avg 0.431
0.381
0.446
0.388
0.466
0.398
0.492
Table 20: For the short-term time series forecasting, comparison between VCA and other LLM-basee -aseedd methods. Bold: best, Underline: second best.
Methods
FSCA
VCA
SPIP-LLM
Time-LLM
GPT4TS
Average
SMAPE
11.828
11.889
12.021
12,494
12,690
MASE
1,580
1.596
1.612
1,731
1.808
OWA
0.850
0.855
0.857
0.913
0.940
Table 21: For the few-shot learning task on 5% training data, comparison between VCA and other LLM-based methods. results are averaged across four different prediction horizons (96. 192, 336, 720}. Bold: best, Underline: second best.
Methods
FSCA
VCA
IIILLMM
Time-LM
GPT4TS
Metric
MSE
MAE
MAE
MSE
MAE
MAE
ETTbl
0.575
0.508
0.598
0.524
0.650
0.550
0.648
0.549
0.681
0.560
ETTh2
0.366
0.397
0.382
0.405
0,380
0.413
0.398
0.426 0.400
0.433
ETTml
0.435
0.457
0.442
0.455
0.446
0.477
0.451
0.472
0.450
ETTm2
0.284
0.332
0.289
0.340
0.296
0.342
0.307
0.348
0.308
0.346
Avg
0,415
0.416
0.432
0.428
0.445
0.438
0,458
0.443
0.465
0.447
Table 22: For the zero-shot learning results, comparison between VCA and other LLM-based methods. The first column A + B indicates training on dataset A and testing on dataset B. Bold: best, Underline: second best.
Methods
FSCA
VCA
S"IP-LLM
Time-LLM
GPT-TS
Metric
MSE
MAE
MSE
MAE
MSE
MAB
MSE
MAE
MSE
MAE
ETThi ET1b2
0,313
0.369
0.336
0.390
0.403
0.417
0,384
0,409
0,406
0.422
ETTh1 ETTm2
0.290
0.348
0.303
0.362
0.325
0.360
0.317
0.370
0.325
0,363
ETT2 ETTh
0.527
0.507
0.561
0.534
0.669
0.560
0.663
0.540
0.757
0.578
ETTH2 ETTm2
0.288
0.347
0.297
0.361
0.327
0.363
0.339
0.371
0.335
0.370
ETTml > ETT22
0.353
0.398
0.372
0.416
0.442
0.439
0.440
0.449
0.433
0.499
ETTml + ETTm2
0.264
0.319
0.255
0,333
0.301
0.347
0.311
0.343
0.313
0.348
ETTm2 + ETTh2
0.343
0.393
0.352
0:401
0.406
0.429
0.429
0.448
0.435
0.443
ETTm2 ETTml
0.480
0.463
0.514
0.487
0,622
Average
0.303
0.37%
0411
The original prompt is straightforward: "Predict future sequences using previous data."
The data domain prompt, illustrated using the ETTh dataset, is: "[Data domain:] The Electric- ity Transformer Temperature (ETT) is crucial for long-term electric power systems management. ETThl and ETTh2 represent 1-hour level data. Each data point includes the 'oil temperature' and six power load features. [Task:]] Predict future sequences using previous data."
The input statistics prompt "Inpt statistics:] The input features a minimum value of <al>, maximum of <max_ val>, <max_val>, and median of <median > The overall trend is <upward or downward>. [Task:] Predict future sequences using previous data."
Our results show that the data domain prompt performs nearly the same the original prompt. The input statistics prompt slightly enhances performance. However, it requires recalculating statistical features and regenerating corresponding embeddings by the L.LM tokenizer with each iteration,
25

---

Published as a conference paper at ICLR 2025
Table 23: For classification tasks, comparison between VCA and
other LLM-based methods. Bold: best, Underline: second best.
I.-M-hased
Methods
GPT4TS
Time-LLM
IPIP-LLM VCA
FSCA
Conentaionnnnnaattoon
34.2
34.6
35.3
39.2
FuceDetection
67.9
68.5
69.0
70.4
Handwriting
32.7
32.0
33.1
38.4
Heartbeat
77.2
79.5
JapaneseVowehh
98.6
98.1
98.6
98.9
PEMS-SF
87.9
87.2
88.4
91.3
SelfRegulatiioSSCP1
94.2
SellRegulationSCC2
59.4
57.2
58.3
60.5
61.1
SpokenArabicDigits
99.2
99.5
99.0
99.8
UWaveGestureLibbray
88.1
89.3
91.3
Average
74.0
73.7
73.9
76.0
Table 24: Comparisons of experimental efficiency with other LLM-based methods.
Training Params
Training Params Percenluges Training Time for iteration(s) Inference Time for iteration(s)
GPT4TS
Time-LLM
70.85M
46.37
2.894
1.723
S3IP-LLM
56.95M
41.25
FSCA w/o Coarse Branch
12.43M
13.29
0.348
0.155
FSCA w/o Dual-Scale GNNs
35.83M
30.6
0,556
0.322
FSCA
37.02M
31.3
0,587
0.331
significantly slowing down the process: training time for one iteration increased from 0.587 seconds to 1.431 seconds.
Table 25: Comparison of different prompt types.
Variant
ETThl
ETTml
ETTml
ETTm2
PSCA(Original)
0.394
0.316
0.342
0.250
FSCA(Demii))
0.396
0.316
0.343
0.252
FSCA(Statistics)
0.392
0.313
0.346
0.246
F.7 COMPARISON OF DIFFERENT NETWORK TYPES IMPLEMENTING CONTEXT-ALIGMERR
We replace the GCN module in our dual-scale GNN framework with alternative networks, including MLP, CNNs, and self-attention mechanisms, demonstrate that graph structures the optimal choice within the Context-A Context-Alignment paradigm. Table/26coonfrrms that GCN-based methods surpass other network models. The superiority of GNNs stems from their unique ability to model node-edge structures, which allows for more nuanced representation of structural and logical relationships. In framework, dual-scale nodes articulate hierarchical structure, while edges represent logical connections. Consequently, this dual-scale GNN framework enhances the alignment of time series data within contexts that LLMs can understand, thereby leveraging pre-trained LLMs' capabilities for time series tasks. Additionally, we integrate GraphS AGE, another prominent GNN variant, to further confirm the robustness of our framework across different graph networks.
Table 26: Results of implementing Context- Aelixtt-llggmeet paradigm using different network types.
Variant
ETTml
ETTml
ETTm2
FSCA(GON)
0.394
0.316
0.342
0.250
0.397
0.321
0.337
0.247
FSCA(Antention)
0.435
0.347
0.362
0.271
ESCA(MLP))
0.407
0.334
0.349
0.269
ESCA(CNN)
0.411
0.340
0.354
0.262
26

---

Published as a conference paper at ICLR 2025
F.8 COMPARISON WITH TS FOUNDATION MODELS
We include comparative results with pre-trained time series foundation models such as UniTS- ST (Gao al. 2024), MOMENT (Goswami al. 2024), and TSMixer (Chen al. 2023). Table reveal that while UniTS-ST outperforms some LLM-based methods, our approach still shows superior performance. We attribute this advantage our method's effective exploitation of the deep logical and structural understanding inherent in LLMs, which better harnesses their capabilities for TS tasks. This underscores the significant potential of LLMs in TS applications.
28
Table 27: For classification tasks, comparison results with series foundation models.
LLM-based Models
Foundation Models
Methods
GPT4TS
Time-LLM SPIP-LLM FSCA
UniTS-ST
MOMENT
SchanolConceetrattin
34.2
34.6
35,3
39.2
37.6
35.7
FaceDetection
69.2
67.9
68.5
70.4
70.5
63.3
Handwriting
32.7
33.1
38.4
29.7
30.8
Heartbeat
77.2
78
77.5
79.5
80.0
72.2
VapaneseVowela
98.6
98.
97.8
71.6
PEMS-SF
87.9
87.2
88.4
91.3
93.1
89.6
SeltRegulationSCPP
93.2
92.8
91.4
94.2
93.9
84.0
SelfRegulationSCCP
58.3
61.1
47.8
SpolcnArabicDigits
99.2
99.5
99.0
99.8
98.9
98.1
U UWaveGesturel..ie Library
88.1
89.3
88.7
91.3
87.7
90.9
Average
73.9
76.4
75.0
68.4
Table 28: For long-term forecasting tasks, comparison results with time series foundation models.
L.-M-based
Foundation Models
Metheds
FSCA
$ IP.M-LLM
Time LLM
GPT4TS
UniTS ST
MOMENT
TSMixer
Metric
MAE
MAE
MSE
Weather
0.224
0.262
0.228
0.265
0.237
0.269
0.237
0.270
0.216
0.259
0.228
0.270
0.225
0.264
ECL.
0.159
0.252
0.166
0.262
0.167
0.264
0.167
0.263
0.156
0.253
0.165
0.260
0.160
0.257
Traftic
0.386
0.263
0.405
0.286
0.407
0.289
0.414
0.294
0.409
0.278
0.415
0.293
0.408
0.284
ETTh1
0.394
0.424
0.418
0.436
0.426
0.435
0.427
0.426
0.405
0.426
0.418
0.436
0.412
0.428
ETT12
0,316
0.375
0.355
0.399
0.361
0.398
0.354
0.394
0.331
0.387
0.352
0.395
0.355
0.401
ETTml
0.342
0.378
0.346
0.382
0.354
0.384
0.352
0.383
0.337
0.376
0.344
0.379
0.347
0.375
ETTm2
0.250
0.314
0.262
0.326
0.275
0.334
0.266
0.326
0.254
0.315
0.382
0.376
0.267
0.322
0.296
0.324
0.311
0.337
0.318
0.339
0.317
0.337
0.301
0.328
0.329
0.344
0.311
0.333
27


Reviews:
[{"review_id": "U8uCWMe2hZ", "reviewer": "Reviewer_Dkrj", "summary": "This work aims at leveraging and improving LLMs for time series tasks. Specifically, the authors propose context alignment, a technique that utilizes dual-scale GNNs in addition to the basic LLM architectures that helps LLM comprehend time series data. Few-shot prompting techniques in regular LLMs are also used in the time series design. Through various experiments on different time series benchmarks, the authors show the performance advantage of the proposed method over baseline models.", "questions": "N/A", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "This paper is tackling an interesting problem of leveraging and improving pretrained LLMs to do time series tasks. The authors explored a diverse set of benchmarks and baselines and showed a superior performance of the proposed method.", "weaknesses": "This work motivates and explains the advantage of the proposed method by LLMs' deep understanding of linguistic logic and structure rather than superficial embedding processing. However, such explanations lack support from experiments. More analysis can be included on the LLM side. For example, does an inferior LLM basic architecture (either old design or small model sizes) or a badly trained LLM (undertrained or untrained) lead to a bad time series performance?\n\n\nThis work freezes most of the LLM structure and tunes dual-scale context-alignment GNNs between Transformer layers. There seems to be lacking analysis on the effect of fully tuning the attention and feed-forward layers as well, and the subsequent necessity of the dual-scale context-alignment GNNs in that case. \n\n\nOne of the interesting attributes of LLMs is the scaling effect (e.g., on model sizes, training data sizes, few-shot prompting amounts). The scaling aspect of LLMs and the proposed DECA seems to be unknown in the context of time series.\n\n\nThe writing can be improved for clarity, e.g., by providing qualitative examples besides key equations, especially in the few-shot prompting part, and adding an algorithm table.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Vu6HLTHsEA", "reviewer": "Reviewer_2N38", "summary": "This paper aims at addressing the alignment of Time series not just on the token level as done by previous works but at a level that enable LLMs to contextualize and comprehend TS in the same manner as it does for natural language. To this aim, the authors proposed Dual-Scale Context-Alignment GNNs that achieves context level alignment comprising structural alignment and logical alignment thereby activating LLMs’ potential capabilities in time series tasks.", "questions": "- I am not completely sure if averaging the error metrics across different prediction lengths is the best way to report and compare the results.\n- Is there a justification as to why this averaging approach is adopted?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "-  The paper addresses an important question of context alignment which promises better use of LLMs in the time series domain.\n- The idea of using GNNs to introduce context alignment brings novelty to this work.", "weaknesses": "- Since the authors use GNNs for context alignment, a befitting diagram showing the nodes and edges would have made it easier to follow the text.\n- Some typos here and there in the paper.\n- Datasets not described clearly in the experimental setup.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "T7UgAMNYaN", "reviewer": "Reviewer_oFgL", "summary": "This paper introduces Context-Alignment, a novel paradigm for enhancing the capabilities of LLMs in time series tasks. The authors argue that leveraging LLMs' strengths in natural language processing, particularly their understanding of linguistic logic and structure, is key to improving their performance on time series data. They propose a Dual-Scale Context-Alignment Graph Neural Networks (DSCA-GNNs) framework that aligns time series data with linguistic components, enabling structural and logical alignment. This framework is used to develop Demonstration Examples based Context-Alignment (DECA), which integrates seamlessly into pre-trained LLMs to enhance their awareness of logic and structure. Extensive experiments across various time series tasks, including forecasting and classification, demonstrate DECA's effectiveness, especially in few-shot and zero-shot scenarios, highlighting the importance of context alignment in activating and enhancing LLMs' potential in time series applications.", "questions": "I have raised several concerns within the weaknesses. Please address the issues I've mentioned there.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "The paper strengthens the representation of time series data through GNNs, addressing a gap in prior research on applying LLMs to time series tasks. The method shows promising results and could inspire future work in this area.", "weaknesses": "- The paper's incorporation of GNNs is a functional approach, yet it overlooks a comparison of modeling time consumption, which is a critical aspect of efficiency that should be measured against baselines that do not employ this method.\n- Additionally, while the paper presents a generalized embedding technique, further validation across a broader range of time series scenarios is needed to establish its robustness. Testing the method in other contexts, such as time series anomaly detection and imputation, would strengthen the claims of its effectiveness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "44soT9LsfI", "reviewer": "Reviewer_N4CB", "summary": "The authors propose a new method for adapting LLMs for time-series analysis tasks. They test on both time-series generation (forecasting) and time-series classification. Their paper proposes the use of Dual-Scale Context-Alignment GNNs which are used to perform context alignment of the time-series and text tokens/patches.\nThey combine this strategy with example prompts which they are calling (Demonstration Examples Prompt Technique) and show that using DECA and the dual scale context alignment improves the performance of LLM’s compared to existing LLM benchmarks for time-series analysis.", "questions": "1. While your methodology considers the strengths of GNN’s as context aligners, I would like to see the choice of GNN’s to be validated experimentally as opposed to other options\n2. In my opinion the terminology is unnecessarily complicated, and it takes away from the impact of the paper. One key example is “VCA w/o DSCA-GNNs” used in table 1. It seems to me that this is simply just prompting since there are no context aligners used. It may be more clear if you denote VCA as context alignment without DSCA-GNNs and mark in table 1 VCA + DSCA-GNNs. \n 3. What is the difference between “(DECA) DEMONSTRATION EXAMPLES BASED CONTEXT-ALIGNMENT” and few-shot prompting as in NLP. It seems like this term was invented to enhance the complexity of the paper but at the cost of reader understanding, especially if that reader has an NLP background.\n4. How does the prompt effect the context alignment? Are there some prompts that harm context alignment? Since prompting is a key component of this paper, it would be useful to know how their new method performs with different prompt types.\n5. Since DECA involves using examples for context alignment a fair comparison against other baselines would be the VCA with the GNN as a context aligner. I think that should be included beside DECA in the results chart", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Their method of context alignment seems new and interesting\n2. The course and fine-grained context alignments are logical and fit well in combination with patching which is a very popular embedding strategy for time-series data\n3. They several different experiments using different tasks and experiment settings to show their model performs at a high level consistently", "weaknesses": "1. This paper focuses on enhancing LLM’s for time-series analysis and it takes a pretrained GPT model and compares it to other LLM-based approaches. This means this approach is ultimately a pretrained model approach (with the LLM being a pretrained model). The authors need to be comparing to other pretrained models and not just LLM based ones. Since its not being used for text generation-based tasks there’s no reason to limit this evaluation to only LLM based approaches and there are many pretrained models which follow the same experimental method.\n2. The proposed method hinges heavily on the use of GNN’s and while they do run an ablation study, from what I understand they don’t compare other network architectures for performing the context alignment. Given that the GNN is a major part of the proposed implementation, an ablation showing other networks as course and fine-grained context aligners is necessary. For example, Linear layers, CNNs and self-attention could be a good start.\n3. I would like more details on the training, their method does add a layer of complexity in adapting LLM’s for time-series analysis and the amount of added compute for finetuning these models (time and device) should be clearly stated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)