Bridging Time and Linguistics: LLMs as Time Series Analyzer through Symbolization and Segmentation
Authors: Jianyang Qin
Chaoyang Li
Jinhao Cui
Lingzhi Wang
Zhao Liu
Qing Liao
Venue: NeurIPS 2025 poster ()
Keywords: Time series; Large Language Models; Cross-Modal Alignment
Primary area: deep_learning
TL;DR: A framework repurposes LLMs for universal time series analysis by aligning time series with the representation and context of natural language.

Abstract: Recent studies reveal that Large Language Models (LLMs) exhibit strong sequential reasoning capabilities, allowing them to replace specialized time-series models and serve as foundation models for complex time-series analysis. To activate the capabilities of LLMs for time-series tasks, numerous studies have attempted to bridge the gap between time series and linguistics by aligning textual representations with time-series patterns. However, it is a non-trivial endeavor to losslessly capture the infinite time-domain variability using natural language, leading to suboptimal alignment performance. Beyond representation, contextual differences, where semantics in time series are conveyed by consecutive points, unlike in text by individual tokens, are often overlooked by existing methods. To address these, we propose S$^2$TS-LLM, a simple yet effective framework to repurpose LLMs for universal time series analysis through the following two main paradigms: (i) a spectral symbolization paradigm transforms time series into frequency-domain representations characterized by a fixed number of components and prominent amplitudes, which enables a limited set of symbols to effectively abstract key frequency features; (ii) a contextual segmentation paradigm partitions the sequence into blocks based on temporal patterns and reassigns positional encodings accordingly, thereby mitigating the structural mismatch between time series and natural language. Together, these paradigms bootstrap the LLMs' perception of temporal patterns and structures, effectively bridging time series and linguistics. Extensive experiments show that S$^2$TS-LLM can serve as a powerful time series analyzer, outperforming state-of-the-art methods across time series tasks.

Bridging Time and Linguistics: LLMs as Time Series Analyzer through Symbolization and Segmentation
Jianyang Qin', Chaoyang Li Jinhao Cui', Lingzhi Wang', Zhao Liu', Qing Liao 1,2+
Harbin Institute of Technology (Shenzhen), Shenzhen, China
ZPengcheng Laboratory, Shenzhen, China
(22b351005, 22b951022, jasjjaaua@stuu... hit. edu en, [liuzhao0@)@pcl.o ac. en, (wanglingzhi, liaoqing) edu. cn
Abstract
Recent studies reveal that Large Language Models (LLMs) exhibit strong sequen- tial reasoning capabilities, allowing them to replace specialized time-series models and serve foundation models for complex time-series analysis. To activate the capabilities of LLMs for time-series tasks, numerous studies have attempted to bridge the gap between time series and linguistics by aligning textual representa- tions with time-series patterns. However, it is non-trivial endeavor to losslessly capture the infinite time-domain variability using natural language, leading suboptimal alignment performance. Beyond representation, contextual differences, where semantics time series are conveyed by consecutive points, unlike in text by individual tokens, are often overlooked by existing methods. To address these, we propose S-MM. a simple yet effective framework repurpose LLMs for universal time series analysis through the following two main paradigms: (i) a spectral symbolization paradigm transforms time series into frequency-domain representations characterized by fixed number of components and prominent amplitudes, which enables a limited set of symbols to effectively abstract key frequency features; (ii) contextual segmentation paradigm partitions the sequence into blocks based on temporal patterns and reassigns positional encodings accord- ingly, thereby mitigating the structural mismatch between time series and natural language. Together, these paradigms bootstrap the LLMs' perception of temporal patterns and structures, effectively bridging time series and linguistics. Extensive experiments show that S'TS-LLM can serve as a powerful time series analyzer, outperforming state-of-the-arr methods across time series tasks.
1 Introduction
Time series analysis (1] is widely applied across diverse domains such as transportation. energy. climate, finance, and healthcare, supporting variety of tasks such as forecasting and classification. Owing to its practical applications, time series analysis has seen significant progress, giving rise to various specialized models tailored for certain tasks and datasets 2 Recently, Large Language Models (LLMs) [5] exhibit remarkable reasoning performance, generalization ability, and multimodal knowledge, which are gradually replacing specialized models to serve as foundation models applied in various time series analysis tasks (6). However, the inherent disparity between natural language and time series poses significant alignment challenges when leveraging LLMs for time series analysis.
The primary distinction between language and time series arises from their representational nature: natural language composed of discrete tokens, whereas time series are continuously recorded
*Coresponding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

in diverse formats such as univariate or multivariate 77 To bridge this gap, straightforward solution is to tok- enize time series as sequences of dig- its [8) such as representing the value 1.23 as "I", ". "2", "3", which is inefficient in representation and fails to capture temporal dependency in- herent in time series. Alternatively, other approaches 7 transform time series into customized word embed- dings, such as "trend" and "peak", that not only compatible with LLMS but also capable of capturing tempo- ral patterns. Nevertheless, time series typically exhibit complex variations involving numerous patterns (e.g., ris- ing, declining, stabilizing, etc.). As illustrated in Figure (a), remains a non-trivial endeavor to describe such intricate variations with concise lan- guage: the variation within the red circle requires a detailed description of its upward and downward movements, rather than being simply summarized as "relatively stable". To address this, we propose simple yet effective alignment paradigm, Spectral Symbolization, transform time-series patterns into textual embeddings. Techni- cally, we apply the time-frequency transform [10] to efficiently compress time-domain sequences into compact frequency-domain representations. As depicted in the red circle Figure (b), the sparsity nature of the frequency domain enables the selection of a few dominant components to abstract temporal variations, mitigating the challenge of representing complex time-domain series accurately with natural language.
Textual Transform in Time Domain
Textual Transform in Frequency Domain
Flectri Lond
Complex voriations handly
eupress
Magninuk
Donnioatt coptured
10
M dhaakii 16 32 40
0.00
4300
3500
12:0
2000 2400
Hour ofe Day
Frequenccy
Abstraction Frequency Signal
This time series begins with gradual decline from 6.77 around 5.1, followed by brief recovery pods of 6.50. It then undergees sharp drop to 4,4 and enters o long perted of celalivell stuble. low orplinude flactuations between 4.7 and 5.5.
This fruquency in dominated by low frequency components at insicas and with values 45,5, 30.65, and 20.73, and other frequency components are snall.
(a)
(b) Preq--omainn
Figure I: Illustration of the challenge in representation align- ment: time-domain sequences exhibit complex variations that are hard to express concisely, whereas frequency- domain signals are typically sparse and can be captured with a few textual tokens representing dominant components.
Beyond representation differences. the discrepancy in contextual structures [1] between time series and natural language is also significant and cannot be overlooked: each token in language carries semantic meaning [12] (depicted in Figure
Linguistic Context (~>
indictes the contextual links)
Linguettc
Rome
was
HOT
built
day
contrast, a single time point provides limited information, with meaningful patterns emerging only across utive observations [13] (as shown in Figure (b)). Using the toven-level position encoding of standard LLMs, which assigns sequential indices individ- ual time points, would disrupt trend continuity (e.g.. downward trend) across consecutive points, impair- ing the model's awareness of temporal context. Upon this observation, we go beyond previous position processing and propose a Contextual Segmentation paradigm to align the contextual structures between natural language and time series. Specifically, we segment sequence of time series into several blocks based on the semantic meaning between time points. As illustrated in Figure (b). time points within the same block, sharing similar trend semantics, are as- signed the same positional index, allowing the LLMss to model temporal trend continuity.
Takan-Invel
Reditions
consec-
(a) Linguistic Context
Time Series Context (~> indictes the contextual links)
dnamosed
upword
19
15
L2
15
19
27
29
31
30
Pusl finns LLMs
level foons
(b) Time Series Context
Figure 2: Illustration of the challenge in con- text alignment: standard LLMs assign posi- tions sequentially to each token to capture context, whereas time series are better en- coded by reassigning shared positions to cor- related points to preserve trend continuity.
Building on these insights, propose novel frame- work, TS-LMM, that repurposes the LLMs to align Time Series with the representational space and con- textual structures of natural language through spectral Symbolization and contextual Segmentation. Through these modules, we transform the time series into symbolic abstractions and segment-level

---

positions, which are combined with the raw sequence to form more informative input for LLMs. The temporal and textual oulputs of LLMs are subsequently fused produce predictions for downstream tasks. Our comprehensive evaluation demonstrates that S can serve as an effective time se- ries analyzer, providing versatility for diverse time-series analysis tasks, such as long-term/short-eere forecasting, few-shot/zero-sher learning, and classification. The main contributions of this paper are summarized as follows:
We propose T'MM-LLM. framework that repurposes LLMs as foundational time series analyzer by bridging the representational and contextual gaps between time series and natural language.
The proposed spectral symbolization and contextual segmentation paradigms endow time series with frequency-based textual abstractions and language- -inaaee-lkkeeaae structural positions, bootstrapping the LLMs' comprehension of temporal variations and trend continuity.
Extensive experiments on twenty-three datasets show that T'MMLLMM can achieve performance that is comparable to or surpasses state-of-the-ar methods across five major time series tasks.
2
Related Works
Time Series Analysis. Time series analysis, as fundamental task for scientific and industrial applications, has undergone a significant revolution with the advancement of deep learning [14]. Conventional time-series models like ARIMA [5] were built upon statistical characteristics of the data, which are crafted for specific tasks. With the advent of deep learning, neural networks, such as RRNN 21 TCNs [66], MLPs [7., and Transformers [T8] 19 offered task-agnostic solutions by mining the dynamic variations and long-term dependencies of time series. While these models have achieved considerable improvements, they lack generalizability across diverse time-series domains 220 Inspired by the success of pre-training techniques in CV and NLP, recent work has explored pre-trained models for time series [21]. aiming to enhance models' representation and transfer abilities across diverse domains. Although pre-training can improve models' versatility, such models remain restricted to the time-series modality, lacking sufficient data to expand their capacity and to enrich their knowledge 22 Consequently, they fall short in handling complex real-world analytical tasks.
Large Language Models for Time Series Analysis. Compared to pre-trained models, LLMs, which acquire vast knowledge from diverse large-scale datasets, have demonstrated potential universal analyzers applicable not only to CV and NL.P but also to time series tasks 223 Recently, a surge of L.LM-based methods tailored for time series analysis has emerged 24 25 but they inevitably face the challenge cross- modal adaptation--transfeiiin
internal knowledge to align with time series representations [26]. Early attempts addressed this via value-to-digit conversion (23), prompt engineering [27 28], or fine-tuning [22]. yet these simplistic approaches struggle to capture temporal dependencies. Therefore, more recent work focuses aligning textual and temporal semantics. For example, Time-LLM (9] reprograms time series into textual prototypes; CALF [5] reduces the distributional discrepancy between time-series and textual modalities in both the feature and output spaces; S²IP-LLM 7 aligns decomposed components with textual anchors in a joint space; and TEST 29 applies contrastive learning implicitly map text embeddings temporal patterns. However, these methods rely on an unrealistic assumption that a limited set of textual tokens can faithfully represent complex and variable time-domain dynamics. Moreover, they often overlook fundamental differences in contextual structure between language and time series. To this end, this work aims to align time-series and language in the frequency domain, while reconstructing temporal contexts to better resemble textual structures.
3 Methodology
Given a sequence of textual tokens X = [=)))=], LLMs typically encode tokens via tokenizer (i(-) and assign sequential positional indices P {p: |=1 for tokens to generate output tokens
Y 1 follows,
(g1) LLMs (1 ((x:)/)) {
where p: = iive E (1.....1-1),
(1)
where (b-) transforms tokens into d-dimension representations. p; indicates the position of token on; in sequence, allowing LLMs to capture the token's 's contextual dependence. When applying LLMs
3

---

TimeLingudli fuece
Traingg
Protared
Frizon
Fext
Forme
LLM bosed Embodding
Series
(Toctal
Tortal
Spectral Symbolintion
Contectial Segrentar on
Feetion
Nomrikattinn Pasching
Saypertt
Forword
Figure 3: The framework of T'-LLL.. After processing the input time series through ! spec- tral symbolization and @ contextual segmentation, we obtain time-aware textual abstractions and remapped positions. @ These, together with the time series, are embedded, packed with prompts, and fed into the pre-trained LLM. c The LLM outputs are fused and projected to generate predictions.
to time series analysis, the inherent gaps in representation space and contextual structure between time series and natural language hinder the models' capacity for temporal reasoning, To unlock the potential of LLMs for time series tasks, we focus on bridging time series and linguisties by (i) learning textual representations to align with time series patterns and (ii) reassigning the positional indices alleviate the contextual mismatches.
3.1 Overview
As depicted in Figure 3 our framework encompasses four core components: (1) spectral symboliza- tion, (2) contextual segmentation, (3) LLM-based embedding, and (4) time-linguistic fusion. Given a sequence of observations X E R-XxL where X consists of N univariate time series over L time steps. Following conventional LLM-based approaches (22), we initially separate multivariate time series into N univariate series, where the n-th univariate time series is denoted as (n) E TR 1
Each univariate time series is partitioned independently into Q -- J+1 patches, denoted as ( E RQXM M, using a patch size of M and a stride of R. Subsequently, these patches undergo a spectral symbolization to craft a textual abstraction (')) for representing temporal patterns, and a contextual segmentation to reassign positional indices p(a) for capturing temporal context. Upon feedforwarding time series, textual abstractions and positional indices through pre-trained LLM, we fuse the LLM's time-linguistic outputs and project them to derive the predictions (n) for down- stream tasks such as forecasting, classífication, etc., with the overall objective to minimize the errors between ground truth Y and predictions.
3.2 Spectral Symbolization
To bridge the representation gap between time series and linguistics, a common strategy is aligning time series with existing vocabulary via transformations. such as converting time series into word embeddings [24) (23) or prototypes (9] 77). However, these transformations aim to textually represent temporal sequence in a time domain rather than frequency domain, yet they struggle to losslessly describe complex variations of time series. For example, a sequence [66, 45, 24, 32, 27, 33, 16, 8] could be transformed into textual abstraction like "The series starts at 66 and drops to 24 over the first three points. After reaching 24, it briefly fluctuates between 32, 27, and 33, then falls again to 16 and finally 8". While accurate, this lossless abstraction is far longer than the original sequence, reducing the efficiency of LLM learning and limiting practical applicability. In practice, common alternative to simplify temporal dynamics by tokenizing the same sequence merely as "decreasing from 66 to 8". While this achieves efficiency, it overlooks critical local variations (e.g..

---

the intermediate fluctuations 32 - 27 - 33). As such, how to textually abstract time series in a brief yet precise manner mhhnery enhancing LLMs' temporal wwwpeehhnnioo---mmmaie an open issue.
To achieve this, we propose a spectral symbolization paradigm that transforms frequency-domain patterns of time series X (n) E IRQ into symbolic sequences, rather than modeling time-domain features. Technically, time series can be expressed as a superposition of sine waves characterized by various frequencies through Fast Fourier Transform (FFT) (TO),
[a... ... Ja.. FFT
(2)
where { is the g-th patch of time series and fa.- denotes a specific frequency. M is the number of frequencies, equal to half of the dimensions M because of the symmetry of FFT. Compared to time- domain data, frequency- domain signals are typically sparse and dominated by few components, which helps reduce noise caused by complex temporal variations [3]. The compact nature of frequency allows to a finite and fixed set of textual tokens for time-series representation,
[
[fa. fa. A
UI U2 1
where um is the m-th textual token corresponding fa.m. To distinguish time series representation between patches, further identify salient textual tokens from 01, 1 for each patch
(n) to describe its unique patterns. Specifically, we select the textual tokens corresponding to the q
top-k frequencies with the highest amplitudes to form an abstraction of the time series, denoted as T(()) = (ei
(Amp FFT (x(11) )))
(U1, 12,1"" 1k) KE 11 (f4,1,
.g.k)
argTopk RE(1,--11.1)
where Amp(.) represents the calculation of amplitude values. 1. fq... . fq..) are top-k frequency components selected from the q-th data patch. Although these top-k frequencies carry relatively compact information, they are capable of capturing the global patterns of a time effective characterization of the sequence. For instance, given the same sequence [66,45 45, 24, 32, 27, 33, 16, 8] mentioned above, using the top-2 frequency compo- nents can reconstruct a sequence (57, 36, 14, 33, 32, 46, 23, 10] that preserves the original trend of ceserraaiinn-fffuctuue -
series 30 31 32,
supporting
For mapping a frequency fq.x to token tkg one straightforward approach is to assign predefined textual vocabulary (such as "trend" and "peak") to each frequency based on the datasets and tasks. However, this requires labor-intensive manual selection and lacks generalizability across different settings. As an alternative, we choose the first L ASCII symbols (c.g.. "@", "!", or "+") to describe corresponding M frequencies. Although these symbols do not carry explicit semantics, LLMs can use the statistical properties learned during pretraining [33] 34 (35 to interpret them like a foreign language, associating different symbol combinations with distinguishable spectral patterns. Consequently, the mapped top-k textual tokens (U1, U2,- ... 0%k provides concise yet unambiguous cues that enable LLMs distinguish global time-series trends such as periodicity, volatility, or smooth- ness. In this paper, we implement the model using the ASCII symbol mapping scheme, while the implementation details and results of the predefined vocabulary mapping are additionally reported in the Appendix E.6
3.3 Contextual Segmentation
Due to the contextual discrepancy between time series and natural language, the token-level position encoding assigns sequential indices to time-series patches (as shown in [D. which hinders LLMs from capturing temporal trend continuity, such as downward and upward movements. To address this, we propose contextual segmentation paradigm that dynamically partitions a sequence Q patches into several blocks and assigns a positional index to each block. Regarding an input time series
a(i)) we first pass it through a GRU to get hidden embeddings { (()) Q
and
9=1
then calculate their segment score s through fa-distance,
(( (a())) Q
c
(n) = GRU
(=1)
a (i.j) = ||c]") - -c,"" 12-
(5)
9=1
5

---

Intuitively, the score s (i.j) determines whether the patches i and should belong to the same block: if s(i,j) < y, they are grouped together; otherwise, they are separated. After obtaining B segments, we assign sequential positional indices P(*) from to to the resulting segments. Here, we provide an intuitive example to illustrate the difference between our segment-level position and token-level position encoding of standard LLMs: given a time series with 6 patches, conventional token-level encoding assigns sequential indices [1, 2, 3, 4, 4,5, 5,6], whereas our approach may assign grouped indices like [1, 1, 1, 2, 2, 3] to better reflect structural continuity. These segment-level positions are subsequently fed into the LLMs to enhance the model's awareness of temporal context. Notably, segmentation is used solely for reassigning positions and does not alter the input time series itself.
3.4 LLM-based Embedding
Given the time series (n) and learned textual abstraction T(2), we embed them using a simple linear layer (--) and concatenate them with a learnable timestamp parameter T.
E (m)
o X
E ((T(")) @T.
(6)
Then, inspired by success of prompt engineering (36,, we augment the embeddings E and E with task-specific prompts H (e.g.. "Predict fulure sequences using previous data" for forecasting task) as LLMs' input tokens x and Z Regarding that an embedding E. E {x ET) [e.,1. ex,2: ex,Bl B] is divided into B blocks through contextual segmentation and a prompt H =
[hi, hy.... hw] has words, we split the embeddings into slices and insert the prompt the end of each slice as follows,
z
(n) CegxU411"
(n) : h. hw 1 (0,1,.... ,.,,
B [
(7)
The token sequences Z and Z X are processed by a shared backbone LLM with segment-level positional indices p(n), yielding the output representations O and Ofm)) respectively. Following common fine-tuning practices for LLMs [22], we freeze all components of the L.MMs except for Layer Normalization (LN) layers, allowing time series adaptation.
3.5 Time-Linguistie Fusion
As the textual output ((n) represent frequency-domain patterns distinct from the time-domain output (n)
(n) as complementary guide and dynamically integrate it with O via gated residual update, ensuring that predictions remain in the time domain:
(n) we treat O
(n)
(n) O = 0 + (n) a (0 + (N) x
(n)
(8)
10
Here, o()) denotes sigmoid activation that nonlinearly transforms spectral features, serving as gating signal (37 to leverage frequency information in controlling which components of the time- domain signal are preserved or suppressed. The cosine similarity (n)) further modulates the fusion, emphasizing time-domain components that are semantically aligned with frequency-domat patterns. Finally, the fused outputs (r)) are processed through prefix truncation [9), flattening, and linear projection produce the final prediction (n)
4 Experiments
To evaluate the model's effectiveness, we adhere to the experimental settings in 22 and conduct extensive experiments on five mainstream analysis tasks, including long/short-term forecasting, few- shot/zero-shot learning, and time-series classification. As shown in Figure 4 experimental results demonstrate that S'TS-LLMM! achieves performance comparable to surpassing state-of-the-art models across diverse downstream tasks.

---

Baselines. We compare our method against
Long tarm Forecaring
a series of well-known and advanced base- lines, including LLM--ased models IS-
(USE)
SFTS LUM (Ours)
LLM 77. Time-ILMM 99. GPT4TS [22 )
er
5AP-UUM
Transformer-based models (TimeXer [38]
Sterrilanm (SMAPE)
Clussfication
[Accumay]
Time-L
iTransformer [39], PatchTST (40)), MLP-
Tirehhierr
Timeder
based models (Tiee [1]. RLinear
[7]. and CNN-based model (TimesNet
13 Notably, we uniformly use GPT-2 44
as the backbone of all LLM-based models for fair comparison. Besides, broader set of models is further employed for classification comparison. More experimental details shown in Appendix
XoM Zero Leoorhhoiammiiggggg
Few aborl Leaming
(MSE)
(MSE)
Figure 4: Overall Model Comparison.
4.1 Long-term Forecasting
Setups. We perform long-term forecasting on Weather, Electricity (ECL), Traffic, ETTh1, ETTh2, ETTml, and ETTm2 datasets [13]. Following the experimental settings commonly used in LLM- based methods [22]. the input time series length is set to 512, and the forecasting performance is evaluated across four prediction horizons (96, 192, 336, 7203, using Mean Squared Error (MSE) and Mean Absolute Error (MAE) as evaluation metrics.
Results. A summary of the forecasting results is presented in Table [ Due to the expressive power of LLMs, LLM-based models, including our method, S'IP-LLM, Time-LLM, and GPT4TS, generally outperform conventional baselines, highlighting the potential of LLMs in time series reasoning. Notably, while our method shares a similar goal of aligning time series with L.MSs' textual representations
I2IP-LLM and Time-LLM, our method consistently outperforms them. These improvements are mainly attributed to the following factors: (1) the spectral symbolization paradigm is capable of capturing frequency patters,, mitigating the difficulty of representing complex time-domain dynamics with discrete tokens; and (2) we further align contextual structures between time series and language, enhancing the LLMs comprehension of sequential dependencies.
4.2 Short-term Forecasting
which comprises 100,000 time series sampled at various frequencies including yearly, quarterly, monthly, weekly, daily, and hourly, In this case, prediction horizons range from 48, with input lengths set to twice the prediction horizon. For evaluation, we adopt Symmetric Mean Absolute Percentage Error (SMAPE), Mean Absolute Scaled Error (MASE), and Overall Weighted Average (OWA) as metrics.
Setups. We conduct short-term forecasting using the M4 dataset 433
Results. As shown in Table 2 conventional specialized models, such as TimeMixer and PatchTST exhibit more impressive performance over L.LM-based methods in short-term forecasting. A plausible explanation is that short input lengths constrain the statistical inference capabilities of L.LMs. For example, the yearly subset of the M4 dataset provides only 12 time steps of data input, offering in- sufficient long-term dependency; thus, LLMS are prone to overfit local noise rather than capture global statistical patterns. Nevertheless, our S2TS-LLM still achieves competitive suboptimal performance, benefiting from its effective spectral representation and contextual learning capabilities.
Table I: Long-term forecasting results. All the results are averaged from four different prediction horizons (96, 192, 336, 720}. Red denotes the best performance, while Blue indicates the second- best. Detailed results are provided Appendix E.T
Medwoo SirSEI M [0weo)
TheeMoor [1
TheaKe [X
DI
ParnEST [B
Tireshie [T
More
M5AE c
MAE
M5EE
MA22
3150
MAG |
MAB
MAB
MA3
MS5
S4AB
3450
MAEE_
Stalis
0.41
0.56
B426
0.135
0.40
0-37
0.43
0.454
0.40
8446
0845
3.844
0.451
0.40
1TTh2
03376
0.55
0.99
11256
0.864
0.364
13315
0367
0.38
11363
03427
0334
11386
0387
3.400
0.414
0/21
5334
0.300
0.315
0.98
0354
01384
01357
0.985
0.381
0.985
0.80
0.307
0.87
0.41
0414
0.37
0.57
0.400
0.800
0406
B:Txc
0238
03355
0.22
0.336
0275
0.344
01386
0326
0.33
0.233
0.24
0.32
0.355
0.32
0336
0.327
0.51
0335
0.33
0333
Avenge
0.399
0.34
0,34
0.34
0337
0.92
0.308
0.52
7

---

Table 2: Short-term forecasting results. The SMAPE, MASE, OWA results are weighted averaged across all M4 datasets, with prediction horizons ranging from [6, 48). Red denotes the best perfor- mance, while Blue indicates the second-best. Detailed results are provided in Appendix E.2
Medoods
S (Ouns)
SPIP-LLM
Thee
GPT4TS M
TineMixer [1
TineXer
Tronfocns k2
RLineer Dv
PaINTST [0]
Timeehee Dk
SMAPE 1
11,57y
12/001
12,494
12,690
11:723
12.022
12.142
12:363
12:099
12,850
MASE
1810
1612
L.311
1.808
LS99
1621
1.31
16682
L.223
1.856
OWN 1
0.807
0387
0913
8:940
0380
0.576
0.869
0.800
0.399
9995
Table 3: Few-shot learning results on 5% and 10% training data. Detailed results are provided in
Appendix E.3
Mei
MSF
MAG
MAR
MSG
MA.F
MSE
MAFF
MSS
MAKE
MSS.]
MAA....
MAR
MSS
MAE
MAR.
0:500
0.633
0:454
ETTxa
9454
D447
3.460
04777
0.422
0443
0.53
08:9
0.821
0784
0256
0477
0.22
6429
0.17
0:391
0:35
0338
0.26
3.342
0.307
0.365
0.38
0346
1.324
0,337
5.330
0.303
0.35
0.385
0311
0.44
03314
0332
0.54
03722
Areccee
8407
67Th
8585
0.099
0.90
3.50
0.755
6.553
0.900
0.535
3.300
0.96
5.85
0.95
0933
0558
0675
0.37
0603
6.59
0.80
0.76
F7Th
8325
8460
3.19
0.534
D1031
3.196
0-465
2.456
0.551
D484
D161
0460
0.813
0-11
6.01
0:09
8.655
10%
ET1ml
8485
0438
1629
3415
0.64
D441
3230
0,82
2337
0.41
0728
0263
0A89
0.79
(.30)
0469
0877
0337
Avetage
8465
0415
3.64
0,50
0459
0,66
0431
3.52
0
2333
0.40
0605
0.50
0460
0.46
0/66
0.46
0.50
0.496
Table 4: Zero-shot learning results from source dataset to unseen target dataset (e.g., ETThl ETTh2). Detailed results are provided in Appendix E.4
Matuoda
S"TS-LLM (Oues)
IP-LL.
P
F1
CPTATS 15
Tlaskliv
TieeNor
Trandfoone
?
Pac
Misse
14A2 g
3,52 1
36AE
3452.1
34AE
MAA.
MAA
3730
3.20
XAEEL
355E
MAA
MAA
MAE.
0.311
3.317
0.453
0.417
0.384
0.459
0.100
5.422
0.374
0.367
0.340
0.407
8:457
0.455
0.362
0.911
0.562
0.455
0.421
0:431
11.34
3.380
0.325
1.800
03313
0.753
13.23
0.361
01.21
0281
0.381
01.88
0.381
0.393
0335
1.300
01314
13.22
0:381
kmm 6//h1
11.00
3.33
3.89
0.900
1961
03380
1023
5.34
17726
0.35
0.92
0..00
0.84
0825
0309
0.31
11903
0335
1390
0121
STT2 2TT12
0.202
3.300
0337
0.3
0.32
0.71
0.338
0.200
0.331
0388
0.300
0.383
0335
0382
0339
0371
0.32
0365
0334
0.70
KTTvll 7771a=
0.545
0.30
0.90
0:51
ETTx2 117722
0,772
3435
0420
0427
0.46
0638
0.45
01414
0425
0.45
01/23
6.42
0.47
0437
0426
0.45
0.43
0:45
ITTWv - [771ml
0.906
3.60
3.00
0.50
0.91
0.51
0.6
8.57
0.5%
0.6
0710
0.551
8
6.53
0.031
0.01
0.58
0.49
0.50
8.57
4.3 Few-Shot Learning
Setups. Given the remarkable few-shot learning capabilities of LLMs 44 we follow the setup in [22] to assess the few-shot performance of our method. We train the model using limited data (only the first 5% or 10% data of train set) and evaluate it on the whole test set.
Results. The brief 5% and 10% few-shot learning results are illustrated in Table 3 Our S?TS-LLA generally outperforms the baselines on different datasets and settings, with the exception of the ETTml dataset. possible reason that ETTml has finer temporal resolution and more complex variations than other datasets. This increases the difficulty for our method to capture sparse spectral patterns from limited data, leading to suboptimal results. However, our S2TS-LLM realizes average MSE reductions of 4.1 %, 6.7%, and 8.2% over S2IP-LLMM Time-LLM, and GPT4TS, respectively, under the 5% few-shot setting, and reductions of 5.2%, 17.0%, and 4.8% under the 10% few-shot setting. Compared to conventional models such as RLinear and iTransformer. our S*TS-LLM leverages the generalizability of LLMs and yields relative improvements from 6.4% to 34.8% over two settings. These results highlight the effectiveness of our approach in few-shot learning scenarios.
4.4 Zero-Shot Learning
Beyond few-shot lcarning, prior work [8] has shown that LLMs also hold promise as effective zero-shot forecasters. To explore this potential, investigate the zero-shot capabilities of our S'TS-LLM under a cross-domain adaptation setting, where the model is first trained on a source dataset and then directly applied to unseen target dataset.
Setups.
Results. The zero-shot learning results in Table 4 show that our S2MS-LLM remarkably excels over all baselines with 8.3% improvement over second-best RLinear and 13.5% improvement over LLM-bbsed Time-LLM in terms of MSE metric. Experiments in both few-shot and zero-shot settings highlight S-TS-LLMM' exceptional performance under data-scarce conditions. We attribute this to our approach's 's ability to transfer the L.MM's s linguistic knowledge to time-series reasoning by learning robust priors of temporal and contextual patterns.
8

---

4.5 Time Series Classification
Setups. To assess the model's s ability in
76.4
representation learning, we conduct exper- iments on sequence-level classification us-
7600
lime
137
ing the setup outlined by 13 We select 10
Timahee
7346
multivariate datasets from the UEA Time
13D
Nan datamaryy
Series Classification Archive [45 cover-
tharotommer
72.5
ing the spectral recognition, medical diag- nosis, and other tasks. All datasets are pre-
Pocket
23.5
Transformer
processed by following [46), where differ- ent subsets have different sequence lengths.
LSTNcE
13M4
70.9
Results, As shown in Figure our S LLM achieves the best performance with an average accuracy of 76.4%. % surpassing the LLM-based GPT4TS (74.0%), CNN- based TimesNet (73.6%), and classical method Rocket (72.5%). It is worth not- ing that some conventional models achieve competitive performance on forecasting tasks with LLM-based methods, but fail in classification tasks, such PatchTST (67.4%) and RLinear (67.2%). These obser- vations show that the prior linguistic knowl- edge of LLMss can help the time series rep- resentation learning.
IUM
23.1
Timaker
65.0
Xodoest
Average aua (%)
Figure 5: Classification results. The results are averaged from ten datasets of UEA. Detailed results provided in Appendix E.3
Detauff (GPT-2)
Symbol Segnest
Fusion
Vocab
BERT
Pretrahn
Trans
042
4.6 Ablation Study
As depicted in Figure 6 we conduct an ablation study with model variants on ETThl to evaluate the (Q1) components' contribution to the framework, (Q2) scal- ability of the framework, and (Q3) L.M effectiveness time series tasks. To eval- uate Q1, the variants "w/o Symbol" and "w/o Segment" remove the spectral sym- bolization and contextual segmentation modules, respectively, and "w/o Fusion" replaces our fusion strategy with simple addition of the LLM outputs. These three variants impair the knowledge transfer from language to time series and lead to performance drops, confirming the importance of each designed module. Then, we evaluate Q2 from two aspects: first, "w Vocab" replaces ASCII symbols with vocabulary for textual abstracting, but yields similar performance, verifying the scala- bility of our symbol-bard based strategy-it effectively captures spectral patterns while avoiding the need for task-specific vocabulary design; second, "ww BERT" replaces GPT-2 with BERT [4] and still achieves competitive results, demonstrating the framework 's scalability across LLM architectures. Regarding Q3, "w/o Pretrain" (random LLM initialization). "w Att", and "ww Trans" (replace the LLM with single Attention or Transformer layer) all underperform compared to S*TS-LLM, highlighting the value of pre-trained LLMs in time series analysis.
S'TS-LLM
(Q1) Component Contribution
(C2) Scalability
[Q3) LLAb Effectlonness
Figure 6: Ablation Study conducted with three experi- mental repeats. The red dashed line represents the best result of TSTS-LLM. Detailed results are provided in Appendix E.6
4.7 Visual Interpretation
Interpretation of Spectral Symbolization. We provide case study on ETThl to show the effec- tiveness of spectral symbolization in Figure The top 4 subplots depict the progressive refinement of spectral symbolization, as the generated textual abstractions (green dots) transition from (a) scattered, random distribution to (d) a compact distribution that fits the time series space (blue dots). Figure 7 (e) visualizes the cosine similarity between textual abstractions and time series in heatmap form. We observe that most cosine similarity values are close to 1 (indicated by light green), suggesting that the spectral symbolization effectively generates textual abstractions to capture temporal patterns.

---

Interpretation of Contextual Segmenta- tion. Figure presents case study on ETTh2 to illustrate how contextual segmen- tation divides a temporal sequence into dif- ferent blocks. We observe that a time se- ries composed of Q patches generally divided into three main parts: (I) the initial patches are grouped together as the first seg- ment, as they are farthest from the predie- tion sequence and capture long-range de- pendencies; (2) the middle patches seg- mented based on their varying trends and patterns; and (3) the final patches, which are closest to the prediction sequence, are separated into distinct segment model short-range dependencies.
Time Sofies
Trutual Abstractions
(e) Epoch 0
(b) Epoch 10
(c) Epoch
(d) Epoch 40
The distrbutions of testual atsirattions over training epocis
Index samples
Similarity Textual Abstractions and Series
Figure A showcase of spectral symbolization.
The First Segment
The Middle Segments
The Last Segment
5
Conclusion
Value
-2
U
Considering the significant importance of LLMs in complex real-world time series analysis, we empower LLMs' ability perceive temporal variations and contex- tual structures. Technically, we revisit the alignment between natural language and time series representations by operating in the frequency domain rather than the time domain, allowing the use of compact sym- bolic forms to capture frequency semantics. Furthermore, we mitigate contextual discrepancies between time series and language by adapting the LLM's positional indices to reflect segment-level temporal structures. Experimental results show that the proposed S effectively bridges time series and linguistics, enabling LLMs to generalize across diverse time-series tasks while achieving promising performance.
48
96
144
192
336 384 432 480 528 Timestep
240
288
Figure 8: A showcase contextual segmentation, where different segments distinguished by blue and green colors can be grouped into three main parts.
Acknowledgment
This work is supported part by the National Key-Researde and Development Program of China (Grant No. 2020YFB2104003), the Natural Science Foundation of China (Grant Nos. 62402168, U23A20322), and the Natural Science Foundation of Hunan Province (Grant No. 2024116156).
References
[1] S. Wen, X. Yang, T. Zhou, and Sun. Robust time series analysis and applications: An industrial perspective. In Proceedings of ACM SIGKDD Conference Knowledge Discovery and Data Mining, pages 4836-4837, 2022.
[2] G. K. G. Lai, W. C. Chang, Y. M Yang. and H. X. Liu. Modeling long-and and short-term temporal patterns with deep neural networks. In Proceedings of International ACM SIGIR Conference on Research Development in Information Retrieval, pages 95-104, 2018.
[3] H. X. J. Xu, J. M. Wang, and M. S. Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. In Advances in Neural Information Processing Systems, pages 22419-22430, 2021.
[4] A. L. Zeng, M. X. Chen, L. Zhang, and Q. Xu. Are transformers effective for time series forecasting? Proceedings of AAAI Conference on Artificial Intelligence, pages 11 1112-11128. 2023.
10

---

[S] P. Y. Liu, H. Guo, T. Dai, N. Q. Li, G. Bao, X. D. Ren, Y. Jiang, and S. T. Xia. Calf: Aligning Ilms for time series forecasting via cross-modal tin-tuning, In Proceedings of AAAI Conference on Artificial Intelligence, pages 18915-18923, 2025.
[6] H. X. Liu, H. Kamarthi, Z. Y. Zhao, S. Q. Xu, S. Y. Wang, Q. S. Wen, T. Hartvigsen, F. Wang, and B. Prakash. How can time series analysis benefit from multiple modalities? survey and outlook. arXiv preprint arXiv:2503. 1835, 2025.
[7] Z. J. Pan, Y. S. Jiang, S. Garg, A. Schneider, Y. Nevmyvaka, and D. J. Song. s'ip-llm: Semantic space informed prompt learning with Ilm for time series forecasting. In Proceedings of International Conference on Machine Learning, pages 1-19, 2024.
[8] N. Gruver, M. Finzi, S. K. Qiu, and A. G. Wilson. Large language models are zero-shot time series forecasters. In Advances in Neural Information Processing Systems, pages 19622-19635, 2023.
[9] M. Jin, S. Y. Wang, L. T. Ma, Z. X. Chu, J. Y. Zhang, X. M. Shi, Y Chen, X. Liang. Y. Li, S. R. Pan, and Q. S. Wen. Time-llm: Time series forecasting by reprogramming large language models. In Proceedings of International Conference on Learning Representations, pages 1-24, 2024.
[10] H. J. Nussbaumer. The fast Fourier transform. Springer Berlin Heidelberg, 1982.
[11] E. C. Nie, S. Z. Yuan, B. L. Ma, H. Schmid, M. Färber, F. Kreuter, and H. Schütze. Decomposed prompting: Unveiling multilingual linguistic structure knowledge in english-centric large language models. arXiv preprint arXiv:2402. 18397, 2024.
[12] L. A Wang, L. Li, D. M. Dai, D. L. Chen, H. Zhou, F. D. Meng. J. Zhou, and X. Sun. Label words are anchors: An information flow perspective for understanding in-context learning. In Proceedings of Empirical Methods in Natural Language Processing, page 9840-9854 2023.
[13] H. X. Wu, T. G. Hu, Liu, H. Zhou, J. M Wang, and M. Long. Timesnet: Temporal 2d- variation modeling for general time series analysis. In Proceedings of International Conference on Learning Representations, pages 1-23, 2023.
[14] Z. L. Jiang, Z. F. Ning, H. Miao, and L. Wang. Stdnet: A spatio-temporal decomposition neural network for multivariate time series forecasting. Tsinghua Science and Technology, 123-)1123 2024.
[15] S. Vasantha Kumar and L. Vanajakshi. Short-term traffic flow prediction using seasonal arima model with limited input data. European Transport Research Review, 7(3):199, 2015.
[16] Zhou, Z. Q. Ma, X. Wang, Q. S. Wen, L. Sun, T. Yao, W. T. Yin, and R. Jin. Film: Frequency improved legendre memory model for long-term time series forecasting. In Advances in Neural Information Processing Systems, pages 12677-12690, 2022.
[17) 7. Li, Qi, Y. D. Li, and investigation on linear mapping. arXiv preprint arXiv:2305.. 10721, 2023.
Xu. Revisiting long-term time series forecasting: An
[18] T. Zhou, Q. S. Wen Z. Q. Ma, X. Wang, L. Sun, and R. Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In Proceedings of International Conference on Machine Learning, pages 17268-27288 2022.
[19] W. Z. Yue, Y. Liu, X. H. Ying, B. W. Xing, R. H. Guo, and J. Shi. Freeformer: Frequency enhanced transformer for multivariate time series forecasting. arXiv preprint arXiv:2501.13988 2025.
[20] M. Jin, Y. Zhang, W. Chen, K. X. Zhang, Y. X. Liang, B. Yang. J. D. Wang, S. R. Pan, and Q. S. Wen. Position: What can large language models tell us about time series analysis. In Proceedings of International Conference on Machine Learning, pages 1-17, 2024.
[21] C. M. Yeh, X. Dai, H. Y. Chen, Zheng, Y. J. Fan, A. Der, Lai, Z. F. Zhuang, J. P. Wang, L. Wang, and W. Zhang. Toward foundation model for time series data. In Proceedings of ACM International Conference on Information Knowledge Management, pages 2023.
11

---

[22] T. Zhou, P. S. Niu, X. Wang, L. Sun, and R. Jin. One fits all: Power general time series analysis by pretrained Im. In Advances in Neural Information Processing Systems, pages 53322-43355 2023.
[23] C. Chang, W. Y. Wang, W. C. Peng, and T. F. Chen. Llm4ts: Aligning pre-trained Ilms as data-efficient time-series forecasters. ACM Transactions on Intelligent Systems and Technology. 16(3):1-20, 2025.
[24] X. L. Zhang, S. Y. Li, Z. Y. Chen, X. F. Yan, and L. R. Petzold. Improving medical predictions by irregular multimodal electronic health records modeling. In Proceedings of International Conference on Machine Learning, pages 41300-411 1130044311, 2023.
[25] Y. Liu, X. D. Huang G. Qin, J. M. Wang, and M. S. Long. Autotimes: Autoregressive time series forecasters via large language models. In Advances Neural Information Processing Systems, pages 12215-1122180 2024.
[26] X. Liu, Q. X. Xu, H. Miao, S. Yang, L. Z. Zhang, C. Long, Z. Y. Li, and R. Zhao. Timecma: Towards Immeempowered multivariate time series forecasting via cross-modality alignment. In Proceedings of AAAI Conference on Artificial Intelligence, pages 18780-18788, 2025.
[27] Y. X. Hu, Q. Li, D. X. Zhang, J. Y. Yan, and T. Chen. Context-alignment: Activating and enhancing I1m capabilities in time series. In Proceedings of International Conference Learning Representations, pages 1-20, 2025.
[28] D. F. Cao, F. R. Jia, S. O. Arik, T. Pfister, Y. X. Zheng, W. Ye, and Y. Liu. Tempo: Prompt-based generative pre-trained transformer for time series forecasting. In Proceedings of International Conference on Learning Representations, pages 1-33, 2024.
[29] C. X. Sun, H. Y. Li, Y. L. Li, and S. D. Hong. Test: Text prototype aligned embedding to activate I1m's 's ability for time series. In Proceedings of International Conference on Learning Representations, pages 1-25, 2024.
[30] C. Chatfield. The analysis of time series: an introduction. Chapman and hall/CRC, 1981.
[31] R. Liu, E. L. Zippi, H. Pouransari, C. Sandino, J. P Nie, H. Goh, E. Azemi, and A. Moin. Frequency-aware masked autoencoders for multimodal pretraining on biosignals. arXiv preprint arXiv:2402.18397, 2024.
[32] X. Zhang, Z. Y. Zhao, T. Tsiligkaridis, and M. Zitnik. Self-supervised contrastive pre-training for time series via fimeuureyy consistency. In Advances in Neural Information Processing Systems, pages 3988 4003, 2022.
[33] W. J. Su. Do large language models (really) need statistical foundations? arXiv preprint 2505:25005/914: 2025.
[34] G. X. Chen, H. Shi, J. W. Li, H. Gao, X. 7. Ren, Y. M. Chen, X. Jiang, 7. G. Li, W. Y. Liu, and C. Huang. Seplim: Accelerate large language models by compressing one segment into one separator. In Proceedings of International Conference on Machine Learning, pages 1-21, 2025.
[35] F. Q. Jiang, Z. C. Xu, L. Y Niu. Z. Xiang. B. Ramasubramanian, B. Li, and R. Poovendran. Artprompt: Ascii art-based jailbreak attacks against aligned 1lms. In Proceedings of Annual Meeting of the Association for Computational Linguistics, pages 15157-15173, 2024.
[36] H. Xue and F. D. Salim. Promptcast: A new prompt-basee learning paradigm for time series forecasting. IEEE Transactions on Knowledge and Data Engineering, 36(11):6851-666 2023.
[37] M. Beck, K. Pöppel, M. Spanring, A. Auer, O. Prudnikova, M. Kopp, G. Klambauer, J. Brand- stetter, and S. Hochreiter. xlstm: Extended long short-term memory. In Advances in Neural Information Processing Systems, pages 107547-107603, 2024.
[38] Y. X. Wang, H. X. Wu, J. X. Dong, G. Qin, H. R. Zhang, Y. Liu, Y. Z. Qiu, J. M. Wang, and M. S. Long. Timexer: Empowering transformers for time series forecasting with exogenous variables. In Advances in Neural Information Processing Systems, pages 1-30, 2024.
12

---

[39] Y. Liu, T. Hu, H. R. Zhang, H. X. Wu, S. Wang. L. T. Ma, and M. S. Long. itransformer: Inverted transformers are effective for time series forecasting. In Proceedings of International Conference Learning Representations, pages 1-25, 2023.
[40] Y. Q.Q.ee Nie, N. H. Nguyen, P. Sinthong, and J. Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In Proceedings of International Conference on Learning Representations, pages 1-24, 2023.
[41] S. Y. Wang, H. X. Wu, X. M. Shi, T. G. Hu, K. Luo, L. T. Ma, J. Y. Zhang, and J. Zhou. Timemixer: Decomposable multiscale mixing for time series forecasting. In Proceedings of International Conference on Learning Representations, pages 1-27, 2024.
[42] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever. Language models are unsupervised multitask learners. OpenAl blog, 1-8):1-24, 2019.
[43] Spyros Makridakis and Michele Hibon. The m3- mo-competition: results, conclusions and implica- tions. International Journal of Forecasting, 16(4):451-476 2000.
[44] X. Liu, D. McDuff, G. Kovacs, I. Galatzer-Levy, J. Sunshine, J. N. Zhan, M. Poh, S. Liao, P. D. Achille, and S. Patel. Large language models are few-shot health learners. arXiv preprint arXiv:2305. 15525, 2023.
[45] A. Bagnall, H. A. Dau, Lines, M. Flynn, Large, A. G. Bostrom, P. Southam, and E. J. Keogh. The uea multivariate time series classification archive. arXiv preprint 1rXiv:181..00075. 2018.
[46] G. Zerveas, S. Jayaraman, D. Patel, A. Bhamidipaty, and C. Eickhoff. transformer- based framework for multivariate time series representation learning. In Proceedings of ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2114-2124, 2021.
[47] J. Devlin, M. W. Chang, K. Lee,, and K. Toutanova. Bert: pre-training of deep bidirectional transformers for language understanding. CoRR, abs//1810004855, 2018.
[48] Y. Liu, H. X. Wu, J. M Wang. and M. Long. Non-stationary transformers: Exploring the stationarity in time series forecasting. In Advances in Neural Information Processing Systems, pages 1881-9893, 2022.
[49] Z. Huang, X. Y. Shi, C. Zhang, Q. Wang, K. C. Cheung, H. W. Qin, Dai, and H. Li. Flowformer: A transformer architecture for optical flow. In Proceedings of European Conference on Computer Vision, pages 668-685, 2022.
[50] H. Y. Zhou, S. H. Zhang, J. Q. Peng, S. Zhang, J. X. Li, H. Xiong, and W. C. Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of AAAI Conference on Artificial Intelligenee pages 11106-11111 2021.
[51] A. Vaswani, N. Shazeer, and N. Parmar. Attention is all you need. In Advances in Neural Information Processing Systems, pages 6000-6010, 2017.
[52] T. Zhang, Y. Zhang, W. Cao, J. Bian, X. Yi, Zheng, and J. Li. Less is more: Fast multivariate time series forecasting with light compling-orienter mlp structures. arXiv preprint arXiv:2207.0/ 186, 2022.
[53] Franceschi, A. Dieuleveut, and M. Jaggi. Unsupervised scalable representation learning for multivariate time series. In Advances in Neural Information Processing Systems, page 32, 2019.
[54] A. Gu, K. Goel, and C. Ré. Efficiently modeling long sequences with structured state spaces. arXiv preprint arXiv:2111.0036,, 2021.
[55] A. Dempster, F. Petitjean, and G. I. Webb. Rocket: exceptionally fast and accurate time series classification using random convolutional kernels. Data Mining and Knowledge Discovery, 34(5):1454-1495, 2020.
[56] T. Q. Chen and C. Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 785-794, 2016.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and Section | have claimed the contributions made in the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Please refer to the discussion section in Appendix D.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
14

---

Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The framework design has been fully described in the Section and the training details are provided in Appendix B.3.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
15

---

Answer: [Yes]
Justification: Please refer to the dataset details listed in Appendix B.1. The key code can be found in the supplementary materials, and the full code will be published if the paper is accepted.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (kttps://nips.cm cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
7 //ipps.cc/pulli//uu.ddd ss
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please refer Appendix B.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification: Please refer the results in subsection 4.6 and Appendix E.6.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
16

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please refer to Appendix B.3.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics http:::///tturtee httpps/ps///ppee...cc///p//////a
Answer: [Yes]
Justification:
We have reviewed and the reasearch conforms with the NeurIPS Code of
Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Please refer to the discussion section Appendix D.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
17

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pre-trained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper does not pose no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All creators of datasets are properly credited by citations.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
18

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
19

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
Justification: The paper provides a detailed description of the use of GPT-2 backbone model in the main text.
Guidelines:
The answer NA means that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
20

---

A
Notations
The main notations used in this paper are described Table 5
Table 5: A summary of notations.
Symbol
Description
X,
Raw and patched time series
Y.
Y
Ground truth and prediction
Textual Abstractions time series
P
Positional index of the LLMs
f
froquency
textual
Hidden embeddings of GRU
segment score
E
Embeddings time series textuul abstractions
H
Task-spocific prompts
Learnable timestamp parameter
Z
Inputs of the LLMs
0
Outputs of the L.LMs
A
Cosine similarity between time series and textual abustractions
L
Length of input time series
Number of variants
Q
Number of patches
Patch
R
Stride
B
Number of segments
B Implementation
B.1 Dataset Descriptions
We conduct experiments on 23 real-world datasets to evaluate the performance of our proposed S?TS- LLM. These datasets span domains such energy, weather, transportation, finance, and healthcare, and have been widely used as benchmarks for time series analysis tasks, including forecasting and classification. Dataset statistics summarized in Table
Regarding the long-term forecasting task, we utilize well-acknowledgedd multivariate datasets, including: (1) Weather comprises 21 metcorological variables collected at 10-minute intervals by the Weather Station of the Max Planck Biogcochemistry Institute in 2020, (2) Electricity (ECL) provides hourly measurements of electricity consumption from 321 consumers. (3) Traffic contains hourly road occupancy rates collected from 862 sensors installed along the San Francisco Bay Area freeways. (4) Electricity Transformer Temperature (ETT) is key indicator reflecting long-term electric power usage. The dataset provides six power load features and one target variable, the 'oil temperature', collected over two years from two countries in China. It is divided into 4 distinct subsets where ETTh and ETTh2 contain hourly recordings, and ETTm1 and ETTm2 are sampled every 15 minutes. We note that the ETTh1, ETTh2, ETTml, and ETTm2 datasets are additionally employed for few-shot and sero-shot learning.
Regarding the short-term forecasting task. adopt the M4 benchmark for performance evaluation. The M4 dataset consists of 100,000 time series from diverse domains, including business, finance, and economics. These time series data are grouped into six distinct subsets with sampling rates ranging from yearly to hourly.
Regarding the time series classification task, we conduct experiments on the UEA Multivariate Time Series Classification Archive (45), which comprises 30 sub-datasets of time series gathered from diverse fields. Following [3]] we select 10 multivariate subsets from the UEA archive for classification tasks, covering gesture, action, and audio recognition, medical diagnosis, and other practical tasks.
21

---

Table 6: Dataset specifications across different tasks. Notably, the few-shot and zero-shot learning tasks utilize the same datasets as long-term forecasting, but only sample the first 5% or 10% of the data for training.
Tasks
Dataset
Dim
Series Length
Datasel Size
Information (Frequency)
ETTml, ETTm2
7
(96, 192, 336, 720)
(34465, 11521, 11521)
Electricity (15 mins)
ETThi, ETTh2
7
(96, 192, 336, 720)
(8545, 2881, 2881)
Electricity (Hourly)
Long-term Forecasting
Electricity
321
(96, 192, 336, 720)
(18317, 2633, 5261)
Electricity (Hourly)
Traffic
862
(96, 192, 336, 720)
(12185, 1757, 3509)
Trangooration (Hourly)
Weather
21
(96, 192, 336, 720)
(36792, 5271, 10540)
Weather (10 mins)
M4 Yearly
(23000, 0, 23000)
Demographic
M4-Quarterly
(24000, 0. 24000)
Finance
Short-term Forecasting
M4.Monbly
(48000, 0, 48000)
Industry
M4-Weekly
(359, 0, 359))
Macro
M4-Daily
(42277, (224)
Micro
M4-Hourly
48
(414, 0144
Other
CihnnolConneentzationn
3
1751
(261, 263)
Alcohol Industry
FaceDeteccton
144
62
(5890, 0, 35244
Face (250Hz)
Handwriting
3
152
(150, 0, 850)
Handwriting
Heartheat
61
405
(204, 0. 205)
Heart Beat
Classification (UEA)
Japanese
12
29
(270, 0, 370)
Voice
PEMS-SF
963
144
(267, 0, 173)
Trinsportation (Daily)
SellRegulationSCPP
6
896
(268, 0, 293)
Health (2561tz)
SellRegalationSCI22
7
1152
(200, 0, 180)
Health (256Htz)
SpokeeArrabieDigiss
13
93
(6599, 0, 2199)
Voice (1 1025Hz)
UWWveGesuurLLibrary
3
315
(120, 0, 320)
Gesture
B.2 Baselines
We compare our S'TS-LLM against series of well-known and advanced baselines for forecasting tasks, including LLM- models (S'IP-LLM (7], Lime-LLM [9. GPT4TS (22). Transformer- based models (TimeXer (38), iTransformer 33 PatchTST (40), MLP--bsed models (TimeMixer
[1] RLinear 17)), and a CNN-based model (TimesNet 13] The details of the baselines are shown below.
S'IP-LLM 77: S'IP-LLM activates the LLMs for time series tasks by aligning the pre- trained semantic space with time series representation space. Technically, it encodes the large and dense word token embeddings into a small set of semantic anchors and then retrieves the top-k anchors that best match the decomposed time series embeddings, using them as prompts to bootstrap the LLMs' understanding of time series.
Time-LLM 9 Time-ILM introduces reprogramming idea that aligns time series data with textual prototypes. Along with prefix prompts that provide descriptions and statistics of specific datasets, the reprogramming framework achieves cross-modal adaptation of LLMs for time series forecasting without any fine-tuning. For a fair comparison, this paper adopts GPT-2 as the backbone LLM for Time-LLM rather than its default setup (Llama-7B).
GPT4TS leverages pre-trained LLMs for general time series analysis through a simple yet effective architecture called Frozen Pre-trained Transformer (FPT). FPT freezes the self-attention and feedforward layers while fine-tuning the layer normalization and positional embeddings, achieving promising performance across various time series tasks.
GPT4TS 22]:
22

---

TimeMixer proposes an MLP-based architecture designed to capture the multiscale temporal variations. Specifically, it decomposes and mixes the multiscale time series in both fine-to-coarse and coarse-to-fine directions, enabling comprehensive aggregation of microscopic and macroscopie information for time series learning.
TimeMixer AT:
TimeXer [3]: TimeXer leverages the patch-level and variate-level representation ability, as well as the attention mechanism in Transformer, to model interactive dependencies between endogenous and exogenous variables, effectively capturing the causal relationships inherent in multivariate time series for forecasting.
irransformer (39 i'Tranformeer repurposes the canonical Transformer to focus multi- variate dependency modeling by inverting the embedding dimensions the attention and feed-forward layers, achieving strong performance
wallwwolld datasets.
RLinear [17] RLinear shows that a single linear layer, when combined with reversible normalization and channel independence techniques, can achieve efficient yet competitive performance compared to specialized and complex networks.
PatchTST [40] PatchTST introduces the concepts of patching and channel independence by segmenting time series into patches while preserving independence across variates. These designs enable Transformers to effectively capture both local and global dependencies, leading to improved performance.
TimesNet introduces a general time-series analysis approach based on frequency analysis and convolutional operations. It captures both intra-period and inter- period variations by transforming a ID time series into a 2D representation.
TimesNet [3]
Besides, broader set of models is further employed for classification comparison, including Transformer- based models (Non-stationary Transformer [48], FlowFormer [49], Informer [50],
Transformer [51),, MLP-based model (LightTS (52]) CNN-based model (TCN [53]). ). RNN-based
models (LSSL [54), LSTNet [2]), and classical models (Rocket [55], XGBoost (56). The details of
the baselines are as follows.
Non-stationary Transformer 48 Non-stationary Transformer addresses the non-stationarity
problem by employing stationary and de- de-stationary attention modules to restore and re- cover non-stationary information time series, thereby improving the model's ability to distinguish temporal variations for forecasting.
FlowFormer [49]: FlowFormer adopts a Transformer- based architecture inspired by optical flow estimation, utilizing a 4D cost volume to model fine-grained correspondences, which can be adapted for time series classification tasks.
Informer [50] Informer explores the sparsity of the attention mechanism capture long- term temporal dependencies while reducing time and space complexity by identifying dominant features during attention operations.
Transformer [51]: Transformer proposes an encoder-decoder architecture based on attention mechanisms to capture global dependencies sequences, outperforming conventional RNNs/CNNs in speed and accuracy due to parallel processing and global context modeling.
LightTs 52 LighTTS presents an efficient MI.P-based framework that employs two distinct downsampling strategies to extract both short-term and long-term patterns in time series.
TCN [53] TCN introduces a deep neural network based on dilated causal convolutions, enabling efficient modeling of long-range dependencies in multivariate time series. In addition, it is trained in an unsupervised manner using a time-based triplet loss, which encourages similar subseries to have similar embeddings.
L.SLL 54 : LSSL. introduces a structured state space model that reparameterizes state space models (SSMs) using structured normal-plus-low-rand (NPLR) decomposition, enabling recurrent networks to effectively capture long-range dependencies in time series.
LSTNet 2 LSTNet an autoregressive framework that combines a convolutional layer with a recurrent layer to capture short-term local dependencies and long-term periodic patterns with varying scales, enhancing the robustness of time series analysis.
Rocket (55 ROCKET is a classical time series classification method that transforms input sequences using thousands of randomly generated convolutional kernels, achieving ofte-of-thee-ne accuracy with significantly lower computational cost.
23

---

XGBoost [56]: XGBoost builds upon traditional gradient tree boosting by improving sparsity handling and aare-scale scalability, resulting in an efficient and flexible system applicable to a wide range of machine learning tasks, including time series analysis.
B.3 Implementation Details
All deep neural networks implemented in PyTorch and trained Linux server (CPU: Intel(R) Xeon(R) Platinum 8452Y @ 2.00GHz, GPU: NVIDIA H800 GPU 80GB). To ensure fair comparisons, adhere the experimental configurations outlined in [13] maintain uniform evaluation procedure across all baselines. For baselines, we strictly follow their provided experimental settings
or cite their performance if applicable [27 38).
Most of the training settings of our proposed framework are based on [22]. Specifically, we implement the proposed -LLM using the first 4 layers of GPT-2 as the backbone LLM. Regarding the forecasting task, we train our S T'LM-LLM using the Adam optimizer with a learning rate of 0.0001, a decay rate 3 (0.9, 0.999), and a batch size of 256. A cosine annealing schedule with Tmaz = 20 is used to adjust the model optimization. Early stopping is configured throughout the training process with a patience of 10. MSE loss is adopted for long-term forecasting, while SMAPE loss is used for short-term predictions. The top-k frequency in the spectral symbolization module is set to 5, and the segment score threshold y in the contextual segmentation module is set to 0.1. The input time series is split into G = 2 slices for task prompt augmentation. Regarding the classification task, train our S'MS-LLMM using the RAdam optimizer with learning rate selected from {0.0005, 0.005). and a cross-entropy loss is used for model optimization.
and Mmmn =
Regarding the metrics, adopt the Mean Square Error (MSE) and Mean Absolute Error (MAE) for long-term forecasting. For the short-term forecasting, following [9] we adopt the Symmetrie Mean Absolute Percentage Error (SMAPE), Mean Absolute Scaled Error (MASE), and Overall Weighted Average (OWA) as the metrics, where OWA is a special metric used in M4 competition. These metrics defined as follows:
T E(n
Lp
MSE (Yi
81
MAE
Y
VT1.
Ip
k=1
1=1
SMAPE 200 by
Y YI
Y Pi
MAPE
L
Y + Y
Tp
Y
1=1
I
Y Y1
SMAPE
MASE
OWA
MASE
Lp
lp m
1Y, - y
2
SMAPENaNe?
MASENave2
Im1
C More Experiments
C.1 Analysis of Frequency Learning
Since the proposed method extracts spectral information to bootstrap LLMs' time-series comprehen- sion, we further evaluate its frequency-domain learning capacity by examining magnitude sensitivity and trend preservation capability. Regarding magnitude sensitivity, we compute cosine similarity and Pearson correlation between the frequency components of predictions and ground truth. For trend preservation, we measure the dominant frequency overlap to assess how well each model captures underlying trends. Table presents comparative analysis our S*TS-LLM against two representative frequency- based models (FEDformer [18] and FiLM [16). where our S²TS-LLM con- sistently outperforms both baselines, demonstrating its effectiveness in preserving frequency-domain contributions for time-serics modeling.
24

---

Table 7: Comparison of different methods in terms of magnitude sensitivity and trend preservation capability.
Dataset
Frequency Analysis
S'TS-LLM (Ours)
FEDformer
FiLM
Cosine Similarity T
0.926
0.919
0.912
ETThl Magnitude Test
Pearson Correlation f
0.878
0.865
0.857
Trend Test
Dominant Frequency Overlap 1
0.523
0.503
0.507
Cosine Similarity
0.987
0.985
0.982
Magnitude Test
ETTh2
Pearson Correlation f
0.985
0.982
0.979
Trend Test
Dominant Frequency Overlap
0.851
0.844
0.846
0.42
0.43
0.42
0.41
0.41
MSE
MSF
0.41
0.40
0.40
0.39
10
15
20
48
0.39
0.0
0.1
0.2
0.3
0.4
0.5
Parameter
Porameter
Parameter
(a) Number of GPT-2 layers
(b) Number of top-k frequencies
(c) Segment score threshold
Figure 9: Sensitivity analysis of hyperparameters (a) number of GPT-2 layers, (b) number of top-k frequencies, and (c) segment score threshold on the ETThl dataset.
C.2 Hyperparameter Sensitivity
We perform a hyperparameter sensitivity analysis on the three key hyperparameters of the proposed S'TS-LLM: the number of GPT-2 layers, the number of top-k frequencies, and the segment score threshold 7. This analysis conducted on the ETTh1 dataset by averaging results across four prediction horizons (96, 192, 336, 720} under different hyperparameter settings. Figure Secuureppreeennee presents the results averaged over three independent runs for each hyperparameter, where the dark blue lines represent the average results from three runs and the light blue boundaries indicate the error bars.
Number of GPT-2 layers. GPT-2 typically consists of 12 transformer layers. We assess the impact GPT-2 network depth by varying the number of layers from to As shown in Figure Πa), our S2MS-LLM exhibits low sensitivity to the number of layers. This can be attributed to the use minimalist symbolic abstractions, which effectively align time series with natural language and reduce the reliance deep LLM layers for multimodal understanding. While increasing the number of layers can reduce prediction error, deeper models may introduce the overfitting problem, leading unstable results-aas evidenced by the 6-layer variant, which shows the lowest MSE but the highest variance. To trade off performance and computational efficiency, we adopt the first layers of GPT-2 as the backbone LLM our framework.
Number of top-k frequencies. Due the symmetry of the Fast Fourier Transform, the number of top-k frequencies is selected from the candidate set {1. 5, 10, 15, 20, 24), ensuring it does not exceed half the feature dimension of the patched input X (48 dimensions, used in (2)). As depicted in Figure (b), neither too few nor too many top-k frequencies lead satisfactory results: a smaller A fails to capture sufficient frequency patterns, while larger & introduces noisy spectral components. Based on this analysis, we set
achieve the best performance.
Segment score threshold. We select the segment score threshold y from the range [0.0, 0.5). and the sensitivity results are illustrated in Figure/9 (c). When 7 0, each input patch is treated as an individual segment, disabling the contextual segmentation mechanism and resulting in the worst performance. As y increases 0.5, the segmentation tends to form overly long segments even single segment for the entire input, which diminishes the structural granularity the time series and leads to performance degradation. Based on these observations, we set 0.1 to achieve optimal performance.
25

---

13
Predictio GroundParm
Prodiciion
Predittion
Prekiticn Ground Biuh
GroundFurm
Ground Fruth
(2) SITS (OurS)
(b) STIP LUM
(cl Mimmeiiier
Tirmeler
Figure 10: Long-term forecasting cases from ETThl by different models under the 96 settings. Blue lines are the ground truths and orange lines the model predictions.
snace
bnoce
20099
10000
2000e
2000
10039
Gro und
10005
GroundTruth
Prodi stion
20006
Graun diruch
2003
Prodicdan
GroundPacc
(5) LTS (Ours)
(b) STIPILLM
14) Timmelbbar
(20
Figure 11: Short-term forecasting cases from the M4 dataset by different models under the input-36- predict-18 settings. Blue lines are the ground truths and orange lines are the model predictions.
Prodittonn
Prodiction
(Dun))
[h0 SPP-LLM
Figure 12: Few-shot forecasting cases from ETThl 1 by different models under the 9 settings. Blue lines are the ground truths and orange lines are the model predictions.
Preditioo
Pretition Groundhroh
Prod otlin Droundhruh
Cround Trtth
(at STTS (Ours)
(6] S7.P LLM
Figure 13: Zero-shot forecasting cases from ETTh2-ETTh/ by different models under the input-96- predict-96 settings. Blue lines are the ground truths and orange lines the model predictions.
C.3 Forecasting Visualization
To evaluate the forecasting capabilities of the proposed S"TS-LLM, Figures 10 to 13 present vi- sualization examples across long-term forecasting, short-term forecasting, few-shot learning, and zero-shot learning tasks, which are compared against baseline models, including IP [7]. TimeMixer [41). and TimeXer [38]. We observe that the datasets exhibit complex variations that pose significant challenges for prediction. For instance, the ETTh1 example in Figure I0 begins with regular periodic fluctuations, followed by a sequence abrupt drops and spikes. Despite such complexity, the visualizations demonstrate that S*TS-LLM achieves good predictive performance across various tasks. This especially evident few-shot and zero-shot scenarios, where S*TS-LLM successfully captures future trends, whereas the baseline models often struggle.
26

---

Table 8: Training parameters and Training/Inference Cost Comparison
Training Percentage (%)
Training per Step (8)
Time
Type
Model
Training Params
per Batch (s)
Conventional
TimesNet-32
4.77M
100
0.207
0,156
Specialized Model
NimesNet-766
107.63M
100
1.020
0.856
GPT4TS(()-768
1.57M
0.029
0.031
54.25M
0.551
0.941
LLM-based Model
S'IP-LLM(4)-76
49.15M
44
0.562
1.202
STTS-LLM(44)7688 (Ours)
2.71M
0.158
0.189
Prodict Lagtt
Posdict Langh
(a) ETTh1
(b) ETTh2
TLFOOus)
Bileca Pasic
Gred Faios
Aftetios based Fisko
Figure 14: Comparison of different fusion strategy on the ETThl and ETTh2 datasets, where lower
MSE values indicate better performance.
C.4 Computational Cost
The computational cost is a critical factor in assessing the practicality of LLM-based models. To evaluate this, we compare our S²TS-LLM, with a specialized model (TimesNet) and three LLM--bseed models (GPT4TS, Time-LLM, and S'IP-LLM) in terms of training and inference costs. Specifically, TimesNet is tested in two setups with embedding dimensions of 32 and 768, the latter matching the embedding size of GPT-2. Regarding the -LM-based baselines, the default settings of L.M-- based I'IM-LLM, Time-LLM, LLM, and GPT4TS utilize the first 6, 12, and hidden layers of GPT-2, respectively. To ensure a fair comparison, we uniformly configure all LLM-based baselines with layers, consistent with the architecture of our proposed model. The computational cost analysis conducted on the ETTh1 dataset with a batch size of 128, using 80G H800 GPU.
As shown in Table our S achieves competitive efficiency and parameter count compared to baseline methods. Particularly, our S'MS-LLM outperforms TimesNet in training efficiency while delivering superior forecasting accuracy, underscoring the practicality of our approach. Compared to LLM-based methods, S incurs a slightly higher computational cost than GPT4TS, primarily due to the additional overhead introduced by our gross-modal alignment and prompt augmentation. Importantly, while sharing the same objective of cross- cross-modal alignment with S2IP-LLM and Time- LLM, our method achieves superior computational and memory efficiency. This highlights that the sparsity inherent in frequency-domate signals can significantly reduce the complexity of aligning time series with natural language, validating the efficacy of our spectral symbolization paradigm.
C.5 Effectiveness of Time-LLinuistte Fusion
Inspire by the RNNs, our Time- Time-Linguistie Fusion (TLF) adopts gated residual update mechanism (O(m) of") + A (w) @ (O
2 0 0x to fuse textual and time-series outputs. To assess
effectiveness of this design, we compare it against several alternative fusion strategies, including
bilinear fusion (z = x.2 """,,) gated fusion (z ar + (1 o))y), and attention-ban fusion
(Softmax (wWaXaWxx) (xWv)) ). where x 0%x and y of MSE results in Figure 14
Vd
demonstrate that our proposed TLF consistently outperforms these alternatives. This superior performance arises because the sigmoid function o(.) softly modulates frequency information by smoothly mapping spectral features into bounded embedding space (0, 1), rather than discarding them. Furthermore, the residual fusion strategy is particularly well-suited for integrating auxiliary frequency cues with raw numerical features: frequency-doman signals guide LLMs semantically
27

---

Table 9: Performance comparison of different methods across varying lookback lengths.
Lookhack Length
512
336
192
96
48
Dutaset
Metric
MSRJ
MAEE
MSE]
MAEL
MSEL
MAE,
MSE]
MAE]
MSE,
MAR]
S'TS-LLM (Ours)
0.355
0.387
0.365
0.389
0.372
0.391
0.378
0.391
0.398
0.411
ETTh1
S'IP-LLM
0.368
0.403
0.367
0.398
0.368
0.393
0.379
0.394
0.390
0.401
GPT4TS
0.376
0.397
0.378
0.399
0.380
0.401
0.383
0.401
0.385
0.403
SPSS-LLM (Ours)
0.260
0.329
0.281
0.335
0.288
0.338
0.291
0.336
0.310
0.350
ETT12
SPI--ILM
0.284
0.345
0.282
0.343
0.288
0.343
0.293
0.343
0.301
0.344
GPT4TS
0.285
0.342
0.286
0.345
0.294
0.349
0.295
0.349
0.299
0.350
rather than numerically, and residual pathways enable smoother integration without disrupting the structure of the original signal.
C.6 Analysis of Performance Gain
To investigate when our S'S works well and when it fails, we follow prior work [9 19 to evaluate model performance under different lookback lengths {512, 336, 192, 96, 48). The detailed MSE results on the ETThl and ETTh2 datasets are shown in Table 9 We observe that S'STS-LLM outperforms the other two LLM-based methods, S'IP-LLM and GPT4TS, when the lookback length is between 512 and 96. However, as the lookback length decreases to 48, S experiences significant performance drop and underperforms compared to the IPI and GPT4TS. This is consistent with T observation that S'STS-LLM achieves suboptimal results on short-term forecasting tasks. The main reason is that S'STS-LLM uses FFT to extract dominant frequency components, which leads to further information compression when the lookback window is short. As a result, the generated compact textual descriptions may not provide LLMs with sufficient information to bridge the semantic gap between linguistics and time series. Therefore, for long-sequence scenarios. For example, our model shows significant gains in both long- forecasting and zero-shot learning tasks (refer to Tables and 4. where it can capture enough dominant frequency components to distinguish different temporal patterns.
method is better suited
D Discussion
Impact Statement. This work presents step forward in time series forecasting by exploring how time series data can be aligned with the representational space and contextual structures of the LLMs. Our proposed frequency-based aligmment approach offers a compact and efficient alternative for cross-modal adaptation, highlighting the potential of leveraging the frequency domain rather than the time domain for lime-eries-drrvve LLMs. Beyond representation, our study also underscores the value of contextual alignment in adapting LLMs time series tasks. We believe that as the LLMs develop deeper understanding of time series, conventional time series analysis will gradually advance toward more universal form of artificial intelligence. particular, the extensive internal knowledge and reasoning capabilities of LLMs hold great promise for addressing complex time series tasks and improving decision-mate making across critical domains such as finance, healthcare, and environmental monitoring.
Limitation and Future Work. S*TS-LLM demonstrates favorable performance and reasonable efficiency across various time series analysis tasks, including forecasting and classification. However, we observe that the use of frequucyyy transformation may intensify information compression when the input sequence is relatively short, resulting in performance degradation (refer to Table 2 and Table To address this limitation, future work will explore alternative frequency representations to augment frequency information, such as multi-scale fusion of frequency features or contrastive learn- ing across different frequency views. Moreover, this study focuses solely on cross-modal alignment between time series and linguistics, neglecting the visual modality of time series. Incorporating time series, computer vision, and natural language into a unified multi-modal framework is a promising direction. Lastly, given its application-oriente design, we plan to extend S2TS-LLM to broader tasks such as anomaly detection, imputation, and spatio-temporal analysis, while further improving its computational efficiency to enhance practical value.
28

---

Table 10: Full long-tern forecasting results. The prediction horizons are set to (96, 192, 336, 720]. A lower value indicates better performance. Red denotes the best performance, while Blue indicates the second-best.
Methods Metric
5'TS
Time
LLM
TineMiser MSE
TimeNer
iTassfocner
RLiterr
PixhTTS
MSE
MAE
MSE
MAB
MSE
MAE
MSE
MAE
MAE
MSE
MAE
MSE
MAA
MSE
MAE
MSE
MAE
MSE
MAE
96
0.148
0.200
0.149
0.200
0.183
0.210
0.152
0.212
0.163
0.209
0.157
0.205
0.134
0.214
0.192
0.232
0.177
0.218
0.172
0.220
0.308
0.347
0.361
Weathe
336
0.344
0.283
0.346
0.380
0.257
0.267
0.254
0.286
0.251
0.287
0.261
0.200
0.278
0.206
0.292
0.307
0.276
0.207
0.260
0.305
730
0.319
0.358
0.170
0.116
0.329
0.110
0.376
0.312
0.310
0.341
0.340
0.341
0.318
0.349
0.966
0.35
0.354
0.546
0.319
Ang
0.325
0.265
0.365
0.337
0.369
0.270
0.340
0.271
0.258
0.201
0.259
0.257
96
0.353
0.228
0.133
0.234
0.140
0.236
0.139
0.238
0.153
0.247
0.140
0.242
0.148
0.240
0.201
0.251
0.195
0.245
0.168
0.272
192
0.151
0.365
0.250
0.150
0.349
0.251
0.116
0.256
0.256
0.162
0.255
0.353
0.199
0.249
01289
336
0.360
0.100
0.270
0.168
0.367
0.206
0.155
0.277
0.275
0.178
0.215
0.258
0.215
0.300
730
0.315
0.291
0.304
0.293
0.21M
0.312
0.206
0.297
0.325
0.310
0.211
0.306
0.225
0.317
0.257
0.331
0.256
0.337
0.230
0.329
0.364
0.167
0.152
0.178
0.216
0.364
0.379
0.274
0.384
0.278
0.282
0,422
0.285
0.271
0.355
0.258
0.649
0.359
0.462
0.295
0.593
0.321
192
0.389
0.225
0.397
0.382
0.398
0.256
0:400
0.200
0.433
0.20h
0.448
0.282
0.417
0.276
0.601
0.356
0.466
0.206
0.617
0.336
Tratfcc
336
0.395
0.275
0,488
0.43
0.359
0.482
0,336
730
0.466
0:397
0.440
0.501
0.416
0.303
0.450
0:312
0.506
0.315
0.516
0.307
0.467
0.900
0.647
0.357
0.514
0.640
0.350
Avg
0.790
0.378
0.415
0.266
0.407
0.369
0.414
0.294
0.454
0.297
0.466
0.287
0.428
0.253
0.076
0.378
0.081
0.304
0.000
0.366
0.355
0.396
0.383
0.404
0,337
0.397
0,315
0.400
0,405
0.316
0.395
0.414
0.419
0,334
0.402
192
0.199
0.419
0.407
0.422
0.427
0.411
0.416
0.418
0.499
0.421
0.419
0.435
0:441
0.416
0:417
0.434
0.460
0.445
0.416
0.439
336
0.402
0.432
0.432
0.451
0.430
0.436
0:442
0.433
0.454
0.458
0.408
0,448
0.457
0.458
0:479
0.446
0.501
0.866
0.291
0.469
0.463
0.478
0.474
0.465
0.408
0.503
0.500
0,500
Avg
0.404
0.429
0.418
0.436
0.426
0.425
0.427
0.426
0.447
0.410
0.437
0.437
0.454
0.447
0.446
0.434
0.459
0.454
0.45%
0.450
96
0.200
0.329
0.384
0.345
0.299
0.348
0.088
0.342
0.269
0.341
0.286
0.31%
0.207
0.349
0.248
0.316
0.302
0.548
0.940
0:534
192
0.312
0.367
0.356
0.389
0.372
0.369
0.350
0.350
0.386
0.414
916
0.122
0:300
0.168
0.417
0.372
0.408
0.375
0:407
0.366
0.414
0.414
0.425
0.478
0.433
0.415
0.436
0.476
0.411
0.452
0.453
720
0,392
0.450
0.419
0.445
0.421
0.446
0:406
0:441
0,412
0.434
0.408
0.432
0.427
0.445
0.420
0.440
0.431
0.446
0.462
0.468
Aug
0.366
0.385
0.387
96
0.780
0.343
0.791
0.348
0.794
0.345
0.290
0.346
0.330
0.357
0.318
0.356
0.314
0.308
0.355
0.316
0.319
0.367
0.318
0.375
192
0,322
0.368
0.323
0,368
0.330
0.368
0.332
0.372
0,361
0.351
0,362
0.385
0.357
0
0.391
0.392
0.367
0.365
0,374
0.387
0.350
0.392
0.365
0.390
0,407
0.426
0,415
01411
720
0.416
0:419
0.410
0.420
0.427
0.431
0.417
0.411
0.444
0.441
0,452
0.441
0.491
0.459
0.487
0.450
0.454
0.439
0.478
0.450
0.344
0.360
0.346
0.362
0.354
0.364
0.352
0.383
0.381
0.395
0.352
0.397
0.407
0.410
0.414
0.407
0.387
0.400
0.400
0.426
0.189
0.175
0.362
0.175
0.265
0.175
0:267
192
0.225
0.297
0.227
0.308
0.243
0.316
0.229
0.301
0.237
0.259
0.237
0.299
0.250
0.339
0.246
0.304
0.241
0.302
0.249
0.309
ETTa2
316
0.277
0:351
0,288
0.346
0.294
0.343
0.286
0.341
0.288
0.340
0.206
0.318
0.311
0.348
0.907
0.342
0.905
0.345
0
0.351
9.302
0.366
0.398
0.38M
0.401
0.391
0.398
0/412
0.398
0.000
0.803
Agg
0.388
0.318
0.362
0,326
0.375
0.314
0.766
0.336
0.275
0:325
0.214
0.332
0.758
0.332
0.256
0.337
0.281
0.335
0.241
0:513
Averag
0.30
0:329
0.111
0.357
6.318
0.319
0,311
0.312
0.319
0.342
0.314
0.340
0.342
0.347
0:317
0.963
0.354
0.354
0.316
0.962
E Full Results
Here, we provide the full results of each experiment listed in the main text, including oong-term/short- term forecasting, few-shot/zero-shor learning, time series classification, and ablation study.
F.1 Long-term Forecasting
Table 10 presents the detailed long-term forecasting results across four prediction horizons {96, 192. 336, 720) on seven widely used datasets. SP consistently outperforms the high-perforning baselines, achieving an average reduction of 2.9% in MSE and 2.5% in MAE compared to the second-best method, S2IP-LLLM 77 These results validate the effectiveness of our representational and contextual alignment between time series and natural language.
E.2 Short-term Forecasting
Table |I illustrates the detailed short-term forecasting results on six M4 datasets. In this setting,
prediction horizons range from 6 to 48, with input lengths set to twice the horizon. While our
S'TS-LLLM achieves the best performance among L.LM- methods, including 1'IP-LLM [7]. Time-LLM [9, and GPT4TS (22), it performs slightly below the lightweight model TimeMixer [4]. This may be due to the relatively short input sequences (ranging from 12 to 96), which limit the ability of LLMs to effectively model long-range dependencies.
E.3 Fhw-shot Learning
Tables 12 and 13 provide the detailed few-shot forecasting results trained on the first 5% and 10% of the data, respectively, across various prediction lengths. The proposed S'TS-LLM consistently
29

---

Table 11: Full short-term forecasting results. The prediction horizons are in [6, 48], and the last three rows are weighted averaged from all datasets under different sampling intervals. A lower value indicates better performance. Red denotes the best performance, while Blue indicates the second-best
Methods
SPIS-LMM
SPP-LLM
Time-I..M
GPT4TS
TimeMixer
TimeXer
T'ansfommer
RI Ilnincar
Pa
TimesNet
SMAPE
13,306
13:413
13,750
15.110
13.206
13.370
13.652
13.617
13:477
15.378
Your.
MASE
3,007
3.024
3.055
3.565
2.916
3,002
3.095
2.991
3.019
OWA
0.785
0.792
0.805
0.911
0.776
0.787
0.807
0.793
0.792
0.918
Quart
SMAPE
10.242
10.352
10.671
10.597
9,996
10.325
10.353
10.752
10,380
10.465
MASE
1.209
1.228
1.276
1.253
1,166
1.221
1,209
1.313
1,233
1.227
OWA
0
0.922
0,950
0.938
0.825
0.914
0.911
0.967
0.921
0.923
Monilt
SMAPE
12.967
12,995
13.416
13.258
12,605
13.076
13.079
13.338
12,959
13.513
MASE
0.975
0.970
1,045
1.003
0.919
0.977
0.974
1.027
0.970
1.039
OWA
0,957
0,931
0,869
0.912
0.911
0.957
OIDeES.
SMAPE
4.837
4,805
4,973
6.124
4,564
5,054
4,780
4.979
4,952
6.913
MASE
3.310
3.247
3,412
4.116
3.115
3.373
3,231
3.332
3.347
4,507
OWA
1,031
1.017
1,053
1,259
0.982
1,064
1,012
1,049
1.049
1,438
SMAPE
11.979
12,021
12.494
12,690
11.723
12.082
12.142
12.363
12.059
12,880
Ave.
MASE
1.510
1.612
1.731
1,808
1.589
1.621
1.631
1.662
1.623
1.836
OWA
0.862
0.857
0.913
0.940
0.840
0.874
0.869
0.890
0.869
0.958
Table 12: Full few-shot learning results on 5% training data. We use the same protocol as in Table TO
Methods
S-18-11M
SPI-LLL
Time-1IM
GPT4TS
TimeMisser
TimeXer
(Ttansfomme
Rlinner
PachTST
TimenNet
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
0.412
0.467
0.500
0.095
0,518
0.416
0.563
0.558
0,808
0.600
0.501
0.474
0.557
11519
0
0.625
0.711
0.570
0.665
336
0.300
0.577
0.751
0.620
0.725
0.613
0.754
0.595
0.855
0.644
0.954
0.802
1.475
0.561
0.576
0.638
0.816
0.619
0.945
0.683
730
0,580
0.503
0.650
0.550
0.685
0.559
0.695
0.569
0.925
0.647
0:360
0.905
0.375
0.504
0.424
0.418
0.451
0.461
0.445
0.505
0.459
0.452
0.455
0.483
0.44
330
0.419
0.441
0.403
01421
0.616
0.433
01:400
01:31
0.992
0.479
0.755
11000
0.631
0.555
0.538
0,493
0.464
11.09
0.459
0.479
730
0,360
0.314
0.350
0.413
0.305
0.425
0.400
0.517
0.475
0.462
0.443
0.839
0,448
0.463
0.454
0.440
0.43
0.357
0.900
0.423
0.404
0.386
0.963
0.484
0.509
0.462
0.500
0.426
0.425
0.909
0.414
0.606
0.518
192
0.434
0.422
0.432
0.434
0,648
0.400
0.400
0.41%
0,579
0.890
0.334
0,412
0.700
0.505
0.455
0.440
0.441
11:36
0.681
0,539
0.455
0.440
0.400
01442
0.452
0.447
0.485
0.459
0.599
0.510
0.512
0.898
0.641
0.514
0.479
0.499
0.467
0.786
0.597
0.568
0.512
0.593
0.521
0.585
0,586
0.671
0.707
0.565
0.757
0.587
0.716
0.593
Avg
0,424
0.447
0.355
0.946
0.477
0.451
0.472
0.450
0.687
0.353
0.619
0.521
0.788
0,556
0.526
0.477
0.326
1,476
0.717
0.361
96
0.197
0.360
0.197
0.278
0.205
0.277
0.199
0.250
0.199
0.282
0.200
0.267
0.265
0.339
0.193
0.273
0.206
0.288
0.220
0.290
0.257
0.319
0.254
0.312
0.267
0.396
0:256
0.316
0.262
0.300
0.261
0.328
0.310
0.562
0.265
0.319
0.264
0.524
0.11
0.161
0.569
0.336
0.166
730
0.414
0.410
0.421
0.421
0.648
0,632
0.400
0.836
0,502
0.663
0.400
0.445
0.878
0.654
0.461
0.435
0.854
11832
0.509
0.463
0.703
0.358
0.796
0.542
0.507
0.368
01:000
0.346
0.334
0.387
0.330
0.355
0.356
0.588
0.311
0.345
0.314
0.352
0.544
0.372
Average
0.427
0.421
0.445
0.438
0.455
0,444
0.465
0.447
0.005
0,511
0.505
0.509
0.675
0.542
0.496
0.456
0.494
0.461
0.612
0,509
outperforms the baselines on most datasets, with the exception of ETTml. Given the inherent sparsity and variability in few-shot learning scenarios, such performance differences specific datasets within a reasonable and acceptable range. Additionally, the results show that almost all L.MM-based methods outperform conventional specialized models, highlighting the strong few-shot learning capabilities of large language models.
E.4 Zero-shot Learning
Table 14 reports the detailed results of the zero-shot forecasting task across various prediction lengths. S'SS-LLM almost achieves the best performance across all settings, delivering a notable 8.3% improvement over the second-best method, RLinear. Interestingly, RLinear performs surprisingly well, possibly because its simple linear structure mitigates overfitting to the source dataset, enhancing generalization in the zero-shot setting. Different from RLinear, our S?TS-LLM, effectively activates the internal knowledge of LLMs to facilitate cross-domain adaptation from source to target datasets.
E.5
Time Series Classification
Table 15 presents the comparative results on 10 multivariate UEA classification datasets. For the time series classification task, we include diverse set of baselines tailored for time series classification, including classical models (XGBoost [56), Rocket (55)), RNN-based models (L.SSL.
[54], LSTNet (2). Transformer-base models (Transformer [51), Informer [50], Non-stationary
30

---

Table 13: Full few-shot learning results on 10% training data. We use the same protocol as in
Table I0
Mathoos
SPTS-11M
Timp-11 M
TimcMiserr
TimeXer
Tiaasformmr
incar
PaschTST
TimesNet
Metric
MSE
34NE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAX
MSE
MAE
MSE
MAE
96
0.469
0.464
0.481
0.434
0.720
0.553
01458
0.456
0.515
0.477
0.481
0.451
0.700
0.585
0.467
0.442
0.516
0.485
0.661
0.62
0.453
0.412
0.518
0.451
0.747
0.545
0.570
0.451
0,544
0,422
0.598
0.524
0.757
0.593
0.553
0.414
0.604
0.70
0.791
0.713
0.575
0.671
0.558
0.677
0,500
0.81
0.65
730
0.678
0.500
0.711
0.584
0.580
0.584
0.725
0.591
1.004
0.717
0.991
0.697
1.234
0.811
0.539
0.666
0.752
0.610
0.877
0.641
0,543
0.483
0.595
0.529
0.785
0.553
0.500
0.525
0,700
0.568
0.685
0.552
0.10
0.50
0.625
0.522
0.33
01542
0.869
0.625
0:305
0.340
0.354
0.400
0.534
0.181
0.331
0.314
0.318
0.375
0.350
0.365
0.
0.006
0435
0.328
0.362
0.353
0.389
0.375
0.409
0.353
0.301
0.420
0/418
0.03
0.450
0.467
116
0.193
0.430
0.422
0.400
0.449
0.458
0.400
0.435
0.475
0.461
0.495
0.467
0.485
0.516
0.476
0.436
0.641
0.59
0.494
730
0.437
0.446
0.450
0.436
0.485
0
0.419
01408
0.005
0.539
0.054
0.564
0.293
0.535
0.587
0.517
0.477
0.480
0,510
0.491
Ang
0.372
0.419
0.483
0.463
0.443
0.415
01:31
0.679
0.463
0.432
0.421
0.358
0.401
0.412
0.422
0.552
0.556
0.305
0.411
0.419
0.583
0.501
0.436
0.421
0:422
0.421
0:447
0.438
0.429
0.423
0.571
0.491
0.525
0.485
0.217
0.548
0.490
0.426
0.437
0.634
0.690
0.524
336
0.460
0.451
0.456
0.430
0.497
0.465
0:460
0.439
0.580
0.581
0.538
0.483
0.235
0.575
0.464
0.446
0.476
0.454
0.725
0.566
0.530
0.472
0.354
0.050
0.594
01:08
11.8%
11687
0.554
0.586
0.472
0.011
11556
0.769
0.549
Ang
0:465
0.455
0.490
0.677
0.597
0.188
0.372
0.192
0.224
0.224
0.796
0.188
01259
0.189
0.309
0.196
0.279
0:345
0.523
0.181
0.200
0.191
0.274
0:212
0.285
G7Tm2
192
0.348
0.308
0.346
0.315
0.360
0.317
0:251
0.309
0.255
0.308
0.252
0.339
0.254
0.538
0.250
0.303
0.252
0.317
0.220
0.323
0,215
0.338
0.301
0.540
0.31
0,389
01346
0,310
0.351
0.361
0.394
0.313
0.343
0.306
13355
0.323
0.151
0.400
Ang
0.370
0:327
0.284
0.312
0.505
0.344
0.203
0.335
0.901
0.305
0.343
0.316
0.573
0.295
0.330
0.206
0.343
0.330
0.351
Avcrge
0.415
0.415
0.438
0.434
| 0.500
0.490
0:436
01431
0.562
0.478
0.513
0.450
0.616
0.530
0.460
0.436
|
0:461
0.446
0.586
0.495
Table 14: Full zero-shot learning results on ETT datasets. A lower value indicates better performance. Red denotes the best performance, while Blue indicates the second-best
Mathak
8P1S-1IM
SPIP-LI.M
Time1IM
GITIAT
TindMfter
Time
VToendommar
Rliincar
Timechia
Maric
MSR
MAR
MSR
MAR
MSR
MAR
MSR
MAR
MSR
MAF
MSP
MAF
MAR
MSP
MAR
MSS
MAF
MSP
MAR
96
0.279
0.339
0.315
0.377
0.334
0.368
0.355
0.314
0.297
0.346
0.308
0.351
0.253
0.394
0.295
0.342
0.304
0.350
0.355
0.387
0.336
0,412
0.417
0.388
0.112
0.417
0.376
0,391
0.404
0.477
0.314
0.366
0.429
336
0.322
0.402
0.53
0.422
0.410
0.43
0.411
0.444
0.411
0.420
0.499
0.436
0.482
0.476
0.420
0.425
0.414
0.425
0.449
0.451
0.422
0.451
0.422
0.410
0.437
0.555
0.438
0.419
0.455
Aug
0.341
0.367
0.403
0.17
0.364
0.406
0.42
0.374
0.407
0.457
0.401
0.380
0.405
0.421
0.431
96
0.200
0.300
0.242
0.319
0.236
0.330
0.236
0.315
0.223
0.311
0.233
0.313
0.247
0.319
0.215
0.300
0.215
0.304
0.239
0.313
0.357
0.265
0,342
0.278
0.343
0.293
0.335
0.275
0,342
0.312
0.360
0.351
0.367
0.33
0.376
0,341
0.374
0.341
0.364
0.386
0.364
0.419
0.385
0
0.334
0.342
0.371
RTTm2
750
0:394
0.418
0.427
0.416
0:429
0.470
0.451
0.477
0.696
0.434
0.457
0.471
0.554
11.770
0.433
0.470
01431
01:24
0:434
01419
0.767
0.566
0.360
0.361
0.509
0.478
0.668
0.967
0.618
0.515
0.382
0.573
0.476
0.499
0.507
0.421
0.854
0.905
0:455
0.444
0.685
0.465
0.844
0.601
BTT12
193
0.543
0.507
0.575
0.530
0.715
0.570
0.758
0.198
0.765
0.595
0,591
0.522
0.867
11615
11513
1.477
0.505
01500
01500
0.610
0.688
0.997
0.574
0.625
8TTM
720
0.590
0.551
0.773
0.564
0.833
0.553
0.781
0.597
0.985
0.713
0.549
0.541
0.887
11664
0.729
11:017
0.626
11:61
0.387
01615
0.550
0.513
0.699
0.560
0.663
0.540
0.757
0.578
0.726
0.579
0.502
0.530
0.965
0.625
0.576
0.51
0.565
0,513
0.965
0.621
0.204
0.221
0.300
0.253
0.329
0.318
0.244
0,310
0.225
0.324
BTT2
192
0.262
0.332
0.295
0.344
0.315
0.342
0.293
0.346
0.312
0.368
0.293
0.351
0.291
01356
0.288
01347
0.289
0.345
0.296
0.352
336
0.307
0.359
0.340
0.376
0.366
0.363
0.347
0.376
0.351
0.396
0.407
0.422
0.351
0.391
0.380
0.384
0.345
0.379
0.353
0.383
BTTal
0.394
0.419
0.453
0.428
0.40
0.454
0.446
0.429
0.177
0.490
0,462
0.447
0.452
0.452
0.441
0.439
0.427
(.47]
0.446
Aug
0.292
0.350
0.327
0.363
0.339
0.371
0.335
0.370
0.351
0.388
0.350
0.385
0.335
0.582
0.329
0.371
0.525
0.365
0.342
0.376
96
0.327
0.375
0.358
0.382
0.355
0.400
0.353
0.392
0.382
0.406
0.385
0.405
0.371
0.407
0.380
0.383
0.354
0.385
0.375
0.407
836
0.408
0.451
0.488
0.452
0.479
0.467
0.469
0.461
0.585
0.523
0.500
0.520
0.481
04485
0.663
0.453
0.481
0.463
0.471
0.484
8TTh2
0:423
0.409
0.469
0.478
0.477
0.606
0.468
0.495
01.588
0.501
016807
0.059
0.474
01:21
0.009
0.482
Aug
0.369
0.417
0.442
0.459
0.440
0.449
0.433
0.499
0.464
0.466
0.488
0.655
0.458
0.424
0.431
0.430
0.438
0.457
0.454
96
0.177
0.261
0.203
0.299
0.318
0.371
0.317
0.394
0.194
0.375
0.190
0.277
0.219
11:915
1.191
11369
0.195
01221
0.223
0.205
BTTml
0.237
0,272
0.325
0.281
0.277
0.327
0.263
0.269
0.324
0.277
11307
0.255
0.337
335
0.259
0.357
0,303
0.347
0.322
0.355
0,331
0.360
0.323
0.337
0.366
0.354
0.378
0,317
0.344
0.317
0.341
0.367
BTTm2
720
0.368
0.356
0.436
0.418
0.414
0.415
0429
0.413
0.423
0.405
0.435
0.419
0.425
11420
11:15
11399
0.416
01.014
11:36
0.415
0.348
0.296
0.354
0.298
0.357
0.324
0.383
0.334
0.416
0.300
0.401
0.325
0.337
0.378
0.347
0.401
0.334
0.376
0.327
0.367
0.360
0.401
BTTm2
192
0.366
0.399
0.413
0.422
0.439
0.441
0434
0.437
0.417
0.422
0.414
0.420
0.438
01444
11421
1.423
01413
0.415
0.:34
0.437
0.442
0.499
0.448
0.499
0.454
0.430
0.459
ETT22
720
0.419
0,448
0.462
0.467
0.488
0.479
0.485
0.477
0.475
0.479
0.457
0.466
0.485
0.477
0.454
0.460
0.459
0.470
0.485
0.477
0.372
0.408
04006
0.429
0.429
0.488
0435
0.44
0414
0.429
0413
0.428
0.432
0.447
0.417
0.428
0.400
0.425
0.435
0.443
0.468
0.583
0.524
0.488
0.347
0.588
0.500
0.119
0.447
0.558
BTTm3
0:471
0.846
066
0.501
0.555
0.864
0.781
0.560
0.27
0.661
0.677
0.536
0.681
01565
0.519
01467
0.530
0.470
0.241
0.560
336
0.495
0.467
0.565
0.522
0.688
0.558
0.778
0.578
0.572
0.486
0.708
0.588
0.792
0.578
0.531
0.47
0.565
0.497
0.77%
0.578
0,712
0.57M
0.769
0.573
0.635
01600
0.727
0.505
0.085
0.573
Agg
0:406
0.462
0.622
0.592
0.588
0.507
0.769
0.567
0.556
01481
0.710
0.554
0.705
01573
0.521
0.474
0.565
01:00
0.260
0.567
Anarag
0:325
0.401
0.452
0.51
| 00454
0.428
04472
0.441
0.441
0.490
0.454
01433
0.493
0.467
0.400
01413
0-413
0.417
0.493
0.451
(22 and a CNN-based model (TCN 533) Compared with these baselines across various architectures, our S'TS-LLM achieves superior performance, demonstrating its strong versatility and representational capacity.
Transformer 48
FlowFormer [49]. an ML.--ased model (LightTS
31

---

Table 15: Full results for the classification task. indicates the name of *former. A higher value indicates better performance. Red denotes the best performance, while Blue indicates the second-best.
(Dhmn)
STINLIM
11.M
C2T4T5
TCM
Tlackherr
33nce
Ligh17
IThne.
Tim
15
Tross
ES0L
Rockor
30D0cs
66.5
07.0
601
17.0
67.1
673
F01
60.1
08.3
973
60.0
6710
973
66.3
00.7
64.7
01.3
54.
170
171
131
15.
15%
27.
361
546
113
78.
11.
11.
370
16.6
15.4
11.0
PDM5-4
00.7
825
58.5
653
954
580
746
82.3
08.7
113
02.5
12.4
12.
132
925
165
12.5
963
918
170
903
629
12.5
R1A
98.1
922
R08
043
12.5
94.
03.2
364
832
356
267
538
111
533
50.8
533
572
233
52.3
13.
48.3
013
883
854
912
816
125
75.
Armag
735
Table 16: Full MSE results of ablation Study on the ETThi and ETTh2 datasets. A lower value indicates better performance. Red denotes the best performance, while Blue indicates the second-best.
S9TS-LLM
(Q1) Compponent Costribution
(Q2) Scalability
(Q3) LLM Effectiveness
Dataset
(Ours)
wio Symbol
weee Segment
w/o Fission
Vecab
w BERT
w/o Pretrain
Att
W Trans
0.355+0.009
0:361=0:001
0.361+0.001
0.356+0.003
0.351+0:005
0,360=0.001
0.368+0.002
0.388-0.004
0.384+0.003
0:403-0:001
0.403+0.001
0.401+0.000
0.508+0.003
0.306+0.003
0.479+0.001
0.420-0.077
0.411+0.000
ETTh1
336
0,402110.00 004
0.432.00015
0.431110.005
0,41510:003
0,401
0.40410.002
0:45211.0005
0,437-00012
0.43310.0044
730
0.463+0.001
0.477=0:022
0.468++0.003
0.479+0.015
0.464+0.001
0.467=0.006
0.488+0.0007
0.489-0.00
0.4 0.467+0.022
Avg
0.404+0.001
0.11:00:010
0.416+00.002
04413+0.002
0:403+0.002
0,407=0.002
0.43++0.002
0:42:00.009
0.424+0.005
0.260100.00
0:27410:000
0.2751000003
0.26910.005
0:26110:002
0.25710.001
0,27510:0022
0.298.00002
0.291110.00
192
0,312:00004
0.335:0.005
0.338:+0.004
0,319:00004
0.311::0:002
0.313:0.001
0,347::0.004
0.362:00006
0,364:0.001
ETT12
336
0.322.10.001
0.336100.001
0.33910.000
0.321
0.321
0.31910.001
0,360.10.001
0.366.00.00
0.350100.00
720
0.392100002
0.40610:005
0,108.10.006
0,408100005
0.3941000044
0.39410.004
0,4031100004
0.426_00016
0.41910.008
Avg
0.321=:0.002
0.345:0:005
0,347:0.003
0.329=00000
0.321:0:005
0.321=0.001
0.346::00000
0.362=0005
0.356::0:00
E.6 Ablation Study
We conduct ablation study with eight model variants on the ETThl and ETTh2 datasets to address the following questions: (Q1) Do individual components contribute meaningfully to the framework? (Q2) How scalable is the framework? ? and (Q3) Are LLMs effective for time series tasks? Table 16 reports the detailed ablation results using the MSE metric, averaged over three runs.
To answer QI, we design the following three variants of S'TS-LLM, including "w/o Symbol", "woo Segment", and "w/o Fusion":
"w/o Symbol": Removes the spectral symbolization component (Section 3.2), meaning the model does not learn textual abstractions to align time series with natural language in the representation space.
"w/o Segment": Removes the contextual segmentation component (Section 3.3), meaning the model does not learn segment-level positions for aligning temporal structures with natural language contexts.
"w/o Fusion": Replaces our time-linguistic fusion strategy (Section 3.4) with a simple summation of the temporal and textual LLM outputs.
removing either the spectral symbolization or contextual segmentation leads to performance drops, as they disrupt the knowledge transferring from natural language to time series. This highlights the necessity of both components in enabling LLMs to comprehend temporal semantics and structural relationships. Additionally, the inferior performance of "w/o Fusion" variant indicates the effectiieness of our fusion strategy in bridging the representational gap between time series and natural language.
As shown in Table 16
Regarding Q2, we design two variants of S including "w Vocab" and "w BERT", to evaluate the scalability of the symbolic representation and the adaptability of different LLM architectures:
* "w Vocab": Replaces ASCII symbols with vocabulary for textual abstracting. Specifically, we query LLMs to select 24 vocabulary terms that are well-suited for describing frequency characteristics, including trend, daily, harmonic, peak, rhythm, cycle, noise, disturbance, pattern, pulse, beat, fluctuation, seasonality, vibration, waveform, instability, perturbation, anomaly, impulse, oscillation, turbulence, interference, variation, and echo.
32

---

"w BERT": Substitutes GPT-2 with BERT 47 as the backbone LLM. Both GPT-2 and BERT have comparable sizes (12 Transformer layers. 124M vs. 110M parameters). Follow- ing our GPT-2 setup, we utilize the first 4 layers of BERT for time series analysis.
As shown Table T6. the "w Vocab" setting can yield accurate and stable predictions, because vocabulary- representations describe frequency patterns more precisely than ASCII symbols. While the "w Vocab" variant uses general vocabulary terms for frequency descriptions, we believe that customizing the word set for specific datasets could further enhance performance. However, this approach requires prior knowledge and could introduce substantial preprocessing overhead as the vocabulary size grows. Additionally, the "w BERT" variant achieves competitive results relative to the default GPT-2 used in S'MS-LLM, demonstrating the scalability and flexibility of our framework.
Finally, to address Q3, we design three variants by removing the LLM from our framework:
"w/o Pretrain":
Initializes the LLM parameters randomly, without any pre-training.
"w Att":
Replaces the GPT-2 backbone with a single Attention layer.
"w Trans":
Substitutes the GPT-2 backbone with single Transformer layer.
All three variants discard the prior knowledge of pre-trained LLMs, resulting in the poorest perfor- mance among all settings. These results highlight the importance of leveraging LLMs for time series analysis, as their ability to model long-range dependencies and their rich internal knowledge facilitate the effective extraction of periodic patterns and comprehension of multimodal data 20]
33


Reviews:
[{"review_id": "ff3NSRxTjB", "reviewer": "Reviewer_UTxa", "summary": "The authors adapts GPT-2 to time-series tasks through two add-on modules, Spectral Symbolization and Contextual Segmentation. \nSS turn patch into short string of ASCII symbols that stand for the *k* biggest FFT peaks.\nCS use GRU patch embeddings, clustered them and assign unique positional ID. \nThey claim forecasting and classification show improved results compared to prior LLM for TS baselines.", "questions": "1. Does contextual segmentation really beat other long-context tricks (e.g., hierarchical attention, relative positions)? I only see the ablation without it.\n2. Is the simple cosine-similarity gate the best way to fuse the raw-value and spectral streams?\n3. After all, why should casting time-series into a handful of FFT-based ASCII symbols help a frozen GPT-2 transfer its linguistic priors? I don't see even a guess.", "limitations": "I feel there are a handful of situations where SS will cause information loss, though they could perform well on some dataset.\n1. Feed it x^2 or log⁡ x; the DC term plus a handful of low harmonics can’t recover the curvature, which can be easily described by text.\n2. “Noisy but meaningful” beats. Heart-rate variability: the clinical signal sits in tiny shifts around 0.1 Hz, but the big energy spike is the main beat at ≈1 Hz.\n3. Highly tortuous trajectories**.** A pedestrian GPS trace is full of zig-zags; FFT shoves all that into a smear of high-freq energy that says nothing about *where* the person is going.\nSS is very much limited to wave-like time series, but in many real world applications time series.\nM4 dataset is a good reference but it contains none of the above situations. The variability is very limited.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "*Strength*\n\n- The amount of empirical validation seems solid with 23 datasets and decent ablations\n- Contextual segmentation is novel and well motivated. Shared positions solve long flat segment problem.\n- Training budget is friendly. Backbone is frozen, only a few norms move.\n\n*Weakness*\n\n- Spectral symbolisation isn’t new. FreqTST (https://openreview.net/pdf?id=N1cjy5iznY) already tokenised FFT peaks. FreEformer (https://arxiv.org/pdf/2501.13989) works fully in frequency space. Though it's understandable that neither is cited because they are not peer reviewed.\n- Evidence for “spectral has *better* information” is one ETTh1 ablation and a pretty heat-map; no magnitude-sensitive or trend-heavy test at all. I will explain why I suspect this idea in limitation.\n- GPT-2 gets per-set LR sweeps while classic models use canned configs.", "was_revised": false, "final_justification": "", "rebuttal": "## W1:Lack Citations of Similar Idea of FreqTST and FreEformer\n\nA: We appreciate the reviewer's suggestion and will add a more detailed discussion of related frequency-domain works in the revised manuscript. While FreqTST and FreEformer all adopts frequency learning, our approach is conceptually and technically distinct from both:\n\n- FreEformer is a non-LLM-based method that focuses on improving the rank of attention matrices to better handle the sparsity in frequency data for forecasting. In contrast, our S$^2$TS-LLM, a LLM-based framework, aims to bridging the representational gap between time series and language for LLMs. Thus, our method transforms spectral signals to generate textual descriptions, prompting LLMs to capture time-series patterns for downstream tasks.\n\n- FreqTST aims to construct a new token dictionary that can generalizably represent frequency features across datasets to enhance Transformer-based forecasting. To achieve this, it designs a pretraining strategy to vectorize frequency amplitudes into token embeddings. In contrast, our method avoids creating a new dictionary, as LLMs struggle to understand newly introduced token vocabularies. Instead, we leverage the existing token vocabulary of LLMs to represent time series, enabling seamless and efficient adaptation to time-series tasks without additional pretraining. To this end, we propose a frequency-domain tokenization that maps each frequency component to a unique character and dynamically selects dominant frequency components to form distinctive textual descriptions for different time series. Our approach is conceptually akin to keypoint detection in CV, where identifying a few salient features (e.g., car tires) is sufficient to recognize the entire object (e.g., car).\n\n## W2,L1:Potential Information Loss Problem of Spectral Symbolization Module\n\nA: We thank the reviewer for pointing out potential information loss problem of FFT in certain cases, such as curved trajectories. Although our Spectral Symbolization (SS) module is built upon FFT, its design helps mitigate this limitation. First, SS primarily uses FFT to generate textual descriptions of frequency-domain features, which serve as supplementary inputs alongside the original time-domain data for LLM training. Second, the SS module dynamically selects top-k frequency components for encoding, enabling it to capture discriminative patterns through varying combinations of dominant frequencies, even when the signal energy is concentrated in narrow frequency bands. \n\nBelow, we provide an ablation study of short-term forecasting on the M4 datasets to support the effectiveness of our SS module. Although FFT intensifies the information compression problem in short-term forecasting, which may limit the relative contribution of the SS module compared to the other two modules, it still offers valuable frequency patterns that effectively help LLMs understand temporal dynamics.\n\n|||S$^2$TS-LLM(Ours)|w/o Spectral Symbolization|w/o Contextual Segmentation|w/o Fusion|\n|:-:|:-:|:-:|:-:|:-:|:-:|\n|M4-Year.|SMAPE↓|13.306|13.718|13.680|13.537|\n|| MASE↓|3.007|3.092|3.109|3.049|\n||OWA↓|0.785|0.809|0.810|0.789|\n|M4-Quart.|SMAPE↓|10.242|10.303|10.435|10.529|\n||MASE↓|1.209|1.219|1.239|1.254|\n|| OWA↓|0.906|0.912|0.926|0.935|\n\nAdditionally, we further evaluate the magnitude sensitivity and trend preservation capabilities of our method, in comparison with two representative frequency-based models (FEDformer[1] and FiLM[2]). Regarding magnitude sensitivity, we compute cosine similarity and Pearson correlation between the frequency components of predictions and ground truth. For trend preservation, we measure the dominant frequency overlap to assess how well each model captures underlying trends. The results listed below show that our method achieves better performance in both magnitude and trend measures, indicating the effectiveness of our method in frequency learning.\n\n||||S$^2$TS-LLM(Ours)|FEDformer|FiLM|\n|:-:|:-:|:-:|:-:|:-:|:-:|\n|ETTh1|Magtitude Test|Cosine Similarity↑|0.926|0.919|0.912|\n|||Pearson Correlation↑|0.878|0.865|0.857|\n||Trend Test|Dominant Frequency Overlap↑|0.523|0.503|0.507|\n|ETTh2|Magtitude Test|Cosine Similarity↑|0.987|0.985|0.982|\n|||Pearson Correlation↑|0.985|0.982|0.979|\n||Trend Test|Dominant Frequency Overlap↑|0.851|0.844|0.846|\n\nBased on the above results and the strong performance of our method across 23 datasets (as shown in Section 4), we believe that our method retains good generalization ability in real-world applications.\n\n> [1]T. Zhou, et al. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. ICML \n[2]T. Zhou, et al. FiLM: Frequency improved Legendre Memory Model for Long-term Time Series Forecasting. NIPS\n\n## W3:GPT-2 Gets Per-set LR Sweeps While Classic Models Use Canned Configs\n\nA: As described in Appendix B.3, our method follows the standard setup of existing time-series LLM methods [3][4][5] by fine-tuning the LLM with a fixed and small learning rate (e.g., 0.0001) for time-series analysis.\n\n> [3]T. Zhou, et al. One fits all: Power general time series analysis by pretrained lm. NIPS\n[4]M. Jin, et al. Time-llm: Time series forecasting by reprogramming large language models. ICLR\n[5]Z. J. Pan, et al. s2ip-llm: Semantic space informed prompt learning with llm for time series forecasting. ICML\n\n## Q1:Contextual Segmentation vs. Long-Context Tricks\n\nA: We clarify that our segmentation and long-context modeling (the position encoding such as relative positions in RoPE) are two distinct components to address different problems: the former is an LLM-independent module that captures temporal dependencies from time series (as shown in Fig. 3), while the latter is part of the LLM used to model contextual relationships among input tokens. By default, if no inputs' indices are provided, the LLM automatically generates sequential indices (e.g., $P = [0,...,n-1]$) for inputs and pass them into the position encoding module to learn contextual relationships. Our method instead creates grouped indices $P$ and feeds them to the LLM. To illustrate why our segmentation works, we can consider treating each time-series patch as a character, such as $[x_0, x_1, x_2, x_3, x_4, x_5] =$ [\"c\", \"a\", \"t\", \"d\", \"o\", \"g\"]. A standard LLM would assign sequential indices [0,1,2,3,4,5], treating each patch independently. In contrast, our segmentation assigns grouped indices [0,0,0,1,1,1], effectively grouping $x_0$ to $x_2$ as \"cat\" and $x_3$ to $x_5$ as \"dog\". This narrows the structural gap between text and time series, allowing the LLM to better capture trend continuity in time series, without compromising its ability to model long-context dependencies.\n\n## Q2:Effectiveness of Time-Linguistic Fusion\n\nA: To clarify, the proposed time-linguistic fusion (TLF) is designed in a simple yet effective way by combining cosine similarity with a gating mechanism. Specifically, the design of TLF is inspired by the Gated TCN module in GraphWavelet[6], which uses a gating mechanism of the form: $h = g(\\Theta \\star \\mathcal{X} + b) \\odot \\sigma (\\Theta \\star \\mathcal{X} + c)$. The sigmoid function $\\sigma(\\cdot)$ controls the proportion of information from $\\mathcal{X}$ that is propagated to the next layer. In a similar spirit, we apply a sigmoid function to the frequency-domain output, generating a gating signal that controls the retention or suppression of time-series information during fusion. This gating is then complemented by a cosine similarity measure. It refines the fusion process by assigning higher weights to time-domain information that are more closely aligned with frequency-domain semantics.\n\nBelow, we compare the proposed TLF with several other fusion techniques, including bilinear fusion ($z = x^T W y$), gate fusion ($z = \\alpha x + (1-\\alpha)y$), and attention-based fusion. The results show that TLF achieves the best performance, yielding lower prediction errors than the alternatives.\n\n|||TLF(Ours)|Bilinear|Gate|Att|\n|:-:|:-:|:-:|:-:|:-:|:-:|\n|ETTh1|Length|MSE↓, MAE↓|MSE↓, MAE↓|MSE↓, MAE↓|MSE↓, MAE↓|\n||96|0.355, 0.387|0.359, 0.389|0.367, 0.396|0.369, 0.397|\n||192|0.399, 0.419|0.412, 0.423|0.402, 0.422|0.401, 0.421|\n||336|0.402, 0.432|0.439, 0.441|0.423, 0.443|0.446, 0.463|\n||720|0.463, 0.478|0.462, 0.473|0.482, 0.485|0.502, 0.497|\n||AVG|0.405, 0.429|0.418, 0.431|0.418, 0.436|0.430, 0.444|\n|ETTh2|96|0.260, 0.329|0.276, 0.335|0.273, 0.334|0.291, 0.345|\n||192|0.312, 0.363|0.338, 0.375|0.335, 0.377|0.340, 0.379|\n||336|0.322, 0.380|0.336, 0.385|0.333, 0.386|0.348, 0.398|\n||720|0.392, 0.430|0.393, 0.428|0.400, 0.432|0.406, 0.439|\n||AVG|0.322, 0.376|0.336, 0.381|0.335, 0.382|0.346, 0.390|\n\n> [6]Z. Wu, et al. Graph WaveNet for Deep Spatial-Temporal Graph Modeling. IJCAI\n\n## Q3:Lack of Justification for GPT-2's Benefit from ASCII Symbols\n\nA: The effectiveness of ASCII symbols lies in that different combinations of ASCII symbols can reflect different distributional patterns of the time series and can be understood by LLMs. Specifically, we assign an ASCII symbol to each frequency component and encode the top-k components into a textual description of time series. When the resulting text is dominated by symbols representing low-frequency components, it indicates smooth and stable trends of time series; when high-frequency components dominate, it reflects more volatile changes of time series. Although ASCII symbols lacks semantic meaning in natural language, LLMs can, after fine-tuning, learn statistical patterns in ASCII sequences and interpret them as foreign-language tokens representing frequency [7]. As shown in the ablation study in Section 4.6, using ASCII symbols to represent frequency components achieves comparable performance to using natural-language words, demonstrating that ASCII symbols can effectively serve as interpretable tokens for LLMs.\n\n> [7]W. Su, et al. Do Large Language Models (Really) Need Statistical Foundations? arXiv"}, {"review_id": "xcIeMxmUEn", "reviewer": "Reviewer_mbFi", "summary": "This work proposes a model called S2TS-LLM, which repurposes LLMs for time series analysis. The method consists of two major components: (1) a spectral symbolization module that converts time series into the frequency-domain for textual abstraction; (2) a contextual segmentation module that segments time series and groups positional encodings based on the temporal patterns. The representations are later fed into a pre-trained LLM for downstream applications. Authors performed comprehensive experimental evaluation of the proposed method, showing that S2TS-LLM outperforms state-of-the-art methods across multiple standard time series tasks.", "questions": "Can the authors design synthetic experiments or design case studies to understand when will the proposed method have the most significant gains, and when will it fail?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strength\n\n1. Overall, this paper presents a creative methodology and a valid bunch of innovative contributions\n- The spectral symbolization module emphasizes the frequency domain representation, which aligns with previous works [1, 2]\n- The contextual segmentation module makes valid contribution in terms of emphasizing time series shape structures.\n2. Comprehensive ablation experiments, demonstrating the effectiveness of the proposed components\n- Table 6 is very informative.\n3. Decent amount of experimental evaluation and informative interpretation of the results\n4. Informative and insightful visualizations and analysis.\n\n[1] Liu, Ran, Ellen L. Zippi, Hadi Pouransari, Chris Sandino, Jingping Nie, Hanlin Goh, Erdrin Azemi, and Ali Moin. \"Frequency-aware masked autoencoders for multimodal pretraining on biosignals.\" arXiv preprint arXiv:2309.05927 (2023).\n\n[2] Zhang, Xiang, Ziyuan Zhao, Theodoros Tsiligkaridis, and Marinka Zitnik. \"Self-supervised contrastive pre-training for time series via time-frequency consistency.\" Advances in neural information processing systems 35 (2022): 3988-4003.\n\nWeakness\n\n1. The work could use more comprehensive experimental evaluation\n- Can the proposed method be used on anomaly detection experiments, and how it would perform when comparing to other baselines?\n- Why only performing classification experiments on 10 UEA datasets? Can the method perform well on all UEA and UCR datasets?\n- When applying the method on different tasks and datasets, did the authors investigate at which condition, or on which dataset, the proposed method can perform significantly better than other methods? In other words, when would the method provide the biggest performance margin?\n2. The work can include more comprehensive literature review", "was_revised": false, "final_justification": "", "rebuttal": "## W1, Q1: More Experimental evaluation\n\n1. **Anomaly Detection**\n\n    A: We clarify that our S$^2$TS-LLM can be applied on anomaly detection tasks, and the results provided below showing its reasonable performance. However, S$^2$TS-LLM currently falls short of state-of-the-art models such as the LLM-based GPT4TS. One possible reason is that our top-k frequency selection mechanism is designed to capture dominant and generalizable patterns in the time series, rather than rare or subtle deviations. While this focus aligns well with forecasting or classification tasks, it may overlook the sparse and abnormal signals that are essential for anomaly detection. In future work, we plan to explore alternative selection strategies and anomaly-sensitive frequency encodings to better adapt our method to this task.\n\n    |F1-Score ↑ |S$^2$TS-LLM (Ours)|GPT4TS|PatchTST|ETS  |DLinear|\n    |:-:|:-:|:-:|:-:|:-:|:-:|\n    |SMD        |83.93             |86.89 |84.62   |83.13|77.10  |\n    |MSL        |82.16             |82.45 |78.70   |85.03|84.88  |\n    |SWaT       |92.65             |94.23 |85.72   |84.91|87.52  |\n    |PSM        |96.71             |97.13 |96.08   |91.76|93.55  |\n    |Average    |88.61             |90.18 |86.28   |86.21|85.76  |\n\n2. **Experiments on all UEA Datasets**\n\n    A: Following prior work [1][2], we select 10 representative datasets from the UEA time-series classification archive to strike a balance between experimental diversity and computational feasibility, as LLM-based methods are relatively resource-intensive. These 10 datasets span a wide range of application domains, including biomedical signals (e.g., Heartbeat, SelfRegulationSCP1, SelfRegulationSCP2), chemical signals (e.g., EthanolConcentration), human activity (e.g., UWaveGestureLibrary, PEMS-SF), speech and audio (e.g., JapaneseVowels, SpokenArabicDigits), and visual tasks (e.g., FaceDetection, Handwriting). We believe this selection supports both comparability and meaningful evaluation.\n\n3. **More Evaluation on UCR Dataset**\n\n    A: We believe that the results on the UEA datasets provide meaningful insights into the method's potential generalizability to the UCR archive. According to [3], the UEA archive is developed as an extension of the UCR archive, offering more challenging and diverse benchmarks with multivariate and longer time series, in contrast to the mostly univariate and short-sequence nature of UCR datasets.\n\n4. **Study of Model's Performance Gain**\n\n    A: To investigate when our S$^2$STS-LLM works well and when it fails, we follow prior work [4][5] to evaluate model performance under different lookback lengths $\\{512, 336, 192, 96, 48\\}$. The detailed results are as follows.\n    \n    We observe that S$^2$STS-LLM outperforms the other two LLM-based methods, S2IP-LLM and GPT4TS, when the lookback length is between 512 and 96. However, as the lookback length decreases to 48, S$^2$STS-LLM experiences a significant performance drop and underperforms compared to the S2IP-LLM and GPT4TS. This is consistent with our paper's observation that S$^2$STS-LLM achieves suboptimal results on short-term forecasting tasks. The main reason is that S$^2$STS-LLM uses FFT to extract dominant frequency components, which leads to further information compression when the lookback window is short. As a result, the generated compact textual descriptions may not provide LLMs with sufficient prompts to bridge the semantic gap between linguistics and time series.\n    \n    Therefore, our method is better suited for long-sequence scenarios. For example, our model shows significant gains in both long-term forecasting and zero-shot learning tasks, where it can capture enough dominant frequency components to distinguish different temporal patterns.\n\n    |     |Lookback Length   | 512          | 336          | 192          | 96           | 48           |\n    |:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n    |     |     Metric       | MSE↓ , MAE↓  | MSE↓ , MAE↓  | MSE↓ , MAE↓  | MSE↓ , MAE↓  | MSE↓ , MAE↓  |\n    |ETTh1|S$^2$TS-LLM (Ours)| 0.355, 0.387 | 0.365, 0.389 | 0.372, 0.391 | 0.378, 0.391 | 0.398, 0.411 | \n    |     |    S2IP-LLM      | 0.368, 0.403 | 0.367, 0.398 | 0.368, 0.393 | 0.379, 0.394 | 0.390, 0.401 |\n    |     |     GPT4TS       | 0.376, 0.397 | 0.378, 0.399 | 0.380, 0.401 | 0.383, 0.401 | 0.385, 0.403 |\n    |ETTh2|S$^2$TS-LLM (Ours)| 0.260, 0.329 | 0.281, 0.335 | 0.288, 0.338 | 0.291, 0.336 | 0.310, 0.350 |\n    |     |    S2IP-LLM      | 0.284, 0.345 | 0.282, 0.343 | 0.288, 0.343 | 0.293, 0.343 | 0.301, 0.344 |\n    |     |     GPT4TS       | 0.285, 0.342 | 0.286, 0.345 | 0.294, 0.349 | 0.295, 0.349 | 0.299, 0.350 |\n\n> [1] T. Zhou, et al. One fits all: Power general time series analysis by pretrained lm. NIPS, 2023  \n[2] H. X. Wu, et al. Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. ICLR, 2023  \n[3] A. J. Bagnall, et al. The uea multivariate time series classification archive. arXiv, 2018  \n[4] M. Jin, et al. Time-llm: Time series forecasting by reprogramming large language models. ICLR, 2024  \n[5] W. Z. Yue, et al. FreEformer: Frequency Enhanced Transformer for Multivariate Time Series Forecasting. arXiv, 2025\n\n## W2: More Literature Review\n\nA: Thanks for the comments. We will expand the discussion of related work by incorporating [6][7], to provide useful preliminaries on frequency-domain modeling and further demonstrate the potential of frequency-based representations in time-series applications with LLMs.\n\n> [6] R. Liu, et al. Frequency-aware masked autoencoders for multimodal pretraining on biosignals. arXiv 2023.  \n[7] X. Zhang, et al. Self-supervised contrastive pre-training for time series via time-frequency consistency. NIPS, 2022"}, {"review_id": "UJrrsVK1lj", "reviewer": "Reviewer_HLD5", "summary": "This paper aims to enhance the LLM’s perception of structures and patterns of time series data by bridging the modality gap between time series and linguistics. It proposes S2TS-LLM, a framework to repurpose LLM for time series analysis by (1) a spectral symbolization to endow time series with frequency-based textual abstractions (2) a contextual segmentation paradigm to mitigate the structural mismatch between time series and natural language. The proposed method is evaluated on five mainstream analysis tasks following the setting in GPT4TS.", "questions": "1. Both previous time-domain descriptions and Spectral Symbolization can provide information about time series patterns from different perspectives. Is Spectral Symbolization superior and can provide more information about time series variations than time-domain descriptions?\n2. Why can ASCII symbols enhance the comprehension of spectral patterns by LLMs? Is there any evidence or literature to support this assertion? (**Lines 181-183**)\n3. Contextual Segmentation groups the patches into blocks to assign grouped indices, which might destroy the temporal order of patches in positional encoding, thus hinder the LLM’s comprehension of structural continuity embedded in the temporal orders, for example, an upward trend patch and a subsequent downward patch construct a peak shape in time series. How to solve this new challenge brought by the proposed Contextual Segmentation? (**Lines 184-200**)\n4. What is the relation between B segments and Q patches? (**Line 193**)\n5. Since this work focuses on LLM for time series analysis, there have currently been many publications using LLM for time series. More experiments about LLM-based methods should be added to explain why the non-LLM baselines should be reduced. The relevant LLM baselines include TEST, CALF, UniTime, TEMPO, and so on.\n6. Why choose GPT-2 as a backbone? For time-LLM, which uses LLaMA as a backbone, the comparison is not fair. Is there any backbone ablation study?", "limitations": "yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\n1. The paper is well structured and the figures are rich and clear to convey related ideas.\n2. The code is provided, and the experiments are extensive, including multiple task,s and the analysis of the core mechanisms of the model is sufficient.\n3. The paper proposes to construct the textual description of each time series sample in the frequency domain, which is a novel perspective compared with previous works, like Time-LLM.\n\n**Weaknesses**\n\n1. The novelty of the paper is limited, as there have been several works focusing on bridging the modality gap between time series and language by aligning textual and temporal semantics, like Time-LLM, TEST, CALF [1], and S2 IP-LLM. Moreover, the author has not addressed the shortcomings of previous work, and the proposed innovations are not compelling.\n\n2. The motivation and effectiveness of the proposed Spectral Symbolization are not convincingly established. (**Lines 43-61**)\n    * The authors claim that existing time-domain descriptions fail to capture complex patterns in time series data and advocate for frequency-domain representations. While frequency-domain analysis can effectively capture periodic and steady-state features, it falls short in addressing \"temporal localization\" and \"non-stationary changes,\" which are crucial for identifying time series patterns. Consequently, while frequency-domain descriptions may serve as valuable supplements, they are not inherently superior to time-domain analyses.\n    * I am skeptical about the approach's ability to \"losslessly capture the infinite variability of the time domain\" and to \"describe intricate variations with concise language.\" These objectives seem to be the core challenges that the paper aims to tackle, yet the proposed method does not convincingly demonstrate its effectiveness in achieving them.\n\n3. The observation of assigning sequential indices to individual time points lacks literature support and seems to be imaginary. (**Lines 71-84**) The baselines, including Time-LLM, GPT4TS, and S2 IPLLM, all utilize a patching strategy, with positional encoding implemented at the patch level rather than the time point level. Patches can preserve local temporal context and model continuity in temporal trends within the patch. The author should compare their proposed segment-level positional encoding with the previous patch-level positional encoding, instead of time point-based token-level positional encoding.\n\n[1] Peiyuan, Liu, et al., \"CALF: Aligning LLMs for Time Series Forecasting via Cross-modal Fine-Tuning,\" AAAI 2025.", "was_revised": true, "final_justification": "The authors’ thorough rebuttal has addressed most of my earlier concerns. While I still find the novelty to be somewhat limited, the methodology is sound and the empirical evaluation is solid. I've adjusted my score to 4.", "rebuttal": "## W1:Limited Novelty\n\nA: We respectfully clarify that the novelty of our S$^2$TS-LLM lies in two aspects:\n\n- **Frequency-domain Tokenization**: We adapt LLMs to time-series tasks from a frequency-domain perspective, setting it apart from prior time-domain methods like Time-LLM, TEST, CALF, and S$^2$IP-LLM. Based on the insight that top-k frequency components can capture key temporal trends, we tokenize them as textual prompts to help LLMs in distinguish different time-series patterns. This dominant-frequency tokenization is conceptually akin to keypoint detection in CV, where a few salient features (e.g., tires) can represent the whole object (e.g., car). This makes our method be more flexible and efficient for cross-modal alignment than time-domain methods that rely on comparing token discrepancies between text and time series one by one. \n\n- **New Context Alignment**: We go beyond prior work by introducing a contextual alignment, which endows temporal segments with language-like contextual structures. This alignment is done externally by reassigning positional indices of time series, without altering LLM's internal architecture, making our design plug-and-play and more friendly for LLMs' cross-modal adaptation.\n\n## W2,W1,Q1:Motivation, Challenge and Effectiveness of Spectral Symbolization (SS)\n\n1. **Motivation of SS**\n\n    A: Our motivation is to leverage the sparsity of frequency signals to improve the clarity of textual prompts for LLMs. Prior studies [1][2] show that prompts are most effective when they are clear and precise, not necessarily long. Time-domain prompts, while rich in information, often require verbose language to detailedly express trends, local variations, etc., which may overwhelm or confuse LLMs. In contrast, frequency signals are often sparse and quantifiable, with key temporal features concentrated in a few dominant components (as shown in Fig. 1). This sparsity enables the generation of compact and unambiguous prompts that help LLMs distinguish global temporal patterns such as periodicity, volatility, or smoothness.\n\n2. **Not Address the Core Challenge**\n\n    A: We respectfully clarify that our goal is not to \"losslessly capture the infinite variability of the time domain,\" but to appropriately preserve essential patterns that support LLMs' cross-modal adaptation. The core challenge lies in the trade-off between accuracy and efficiency when tokenizing time series as prompts for LLMs. We address this by shifting tokenization from the time domain to the frequency domain.\n\n    To illustrate, consider tokenizing a sequence $[66,45,24,32,27,33,16,8]$:\n\n    - Time-domain Tokenization: When aiming for efficiency, it tends to oversimplify temporal dynamics (e.g., describing a sequence as \"decreasing from 66 to 8\"), which ignores local variations and thus lacks accuracy. When seeking for accuracy, it needs to capture fine-grained changes, which often requires lengthy descriptions and hurts efficiency. For example, a precise description-\"The series starts at 66 and drops to 24 over the first three points. After reaching 24, it briefly rises to 32 and 33, then drops again to 16 and finally 8\"-far exceeds the length of the original sequence.\n\n    - Frequency-domain Tokenization: Through FFT, we reduce the sequence length by half, improving the efficiency due to less information needed to be tokenized. Meanwhile, since using top-2 frequency component can reconstruct a sequence $[57,36,14,33,32,46,23,10]$ with original \"drop→flucuation→drop\" trends, our frequency-domain tokenization can preserve global patterns to ensure accuracy.\n\n    Based on these observation, our method can leverage the sparsity of frequency features to generate compact yet informative prompts, balancing accuracy and efficiency more effectively than time-domain tokenization.\n\n3. **Effectivenss of SS**\n\n    A: While the frequency domain contains less information than the time domain, prior work [3] have shown that frequency features are effective for modeling time-series. These successes support the potential of frequency learning to address LLMs' cross-modal adaptation. Importantly, our method does not rely solely on frequency features. We combine frequency-domain prompts with raw time-domain series, allowing LLMs to benefit from complementary information: the frequency prompts helps LLMs perceive global trends, while raw series allows attention mechanisms to capture local variations. As shown in Section 4 and Appendix Table 7, our method outperforms time-domain methods, like Time-LLM and S$^2$IP-LLM, in both prediction accuracy and computational efficiency, highlighting the benefits of our method.\n\n> [1]Chen.Unleashing the potential of prompt engineering for large language models.arXiv  \n[2]Kusano.Are Longer Prompts Always Better? Prompt Selection in Large Language Models for Recommendation Systems.arXiv  \n[3]Zhou.Fedformer:Frequency enhanced decomposed transformer for long-term series forecasting.ICML\n\n## W3,Q4:Point-level Mismatches Patch-level in Segmentation\n\nA: We believe that the term 'time points' may have unintentionally caused some misunderstanding. We appreciate the reviewer's observation and will revise ambiguous terms like \"time points\" to improve clarity. As stated in lines 147-150 and 188-191, our method, like Time-LLM, first applies a patching strategy to preprocess time series and then performs contextual segmentation at the patch level, not at individual time points. The detailed process is listed below:\n\n- A univariate time series $X \\in \\mathbb{R}^{L}$ is divided into $Q$ non-overlapping patches, resulting in $X \\in \\mathbb{R}^{Q \\times M}$.\n\n- Contextual segmentation is performed over these $Q$ patches. It groups adjacent and semantically similar patches into the same segment for assigning a shared position index, with each segment composed of one or more consecutive patches. For example, GPT-2 assigns sequential indices $[0,1,2,3,4]$ to 5 patches, while our segmentation may group patches 0, 1, 2 into one segment, assigning indices $[0,0,0,1,2]$ to reflect contextual continiuty.\n\n- Segment-level indices are fed into the the LLM's positional encoding, allowing patches within the same segment to share positional embeddings. This helps the attention mechanism fuse related patches, better capturing local trend continuity.\n\n## Q2:Why ASCII Symbol Works?\n\nA: Thank you for the insightful question. ASCII symbols help LLMs understand spectral patterns for two main reasons: 1) they offer a compact and discriminative encoding of frequency components, and 2) LLMs, as statistical models, can learn meaningful patterns from symbolic inputs even without explicit semantics.\n\n- Discriminative representation: By mapping each component to a unique ASCII symbol, dynamically selecting top-k components can form a specific symbolic sequences that differentiate between spectral patterns. For example, sequences dominated by low-frequency symbols reflect smooth trends, while high-frequency symbols indicate volatility.\n\n- Statistical nature of LLMs: LLMs rely on statistical patterns rather than semantic meaning[4]; for example, [5] shows ASCII separators act as structural cues in LLMs. Thus, LLMs can interpret ASCII symbols as frequency-related tokens (like foreign language) via fine-tuning.\n\nOur ablation (Section 4.6) further confirms that ASCII tokens perform comparably to natural language words, validating their utility in frequency-domain prompting.\n\n> [4]Su.Do Large Language Models (Really) Need Statistical Foundations?arXiv  \n[5]Chen.SepLLM:Accelerate Large Language Models by Compressing One Segment into One Separator.ICML\n\n## Q3:Contextual Segmentation may Destroy Temporal Order\n\nA: We respectfully clarify that our segmentation preserves the temporal order of patches. Only adjacent and semantically similar patches are grouped into the same segment, and later patches never get lower position indices than earlier ones, making the sequence in order. This ensures the LLMs perceive the correct sequence structures; for example, a peak (upward followed by downward), won't be mistaken for a valley.\n\n## Q5:More LLM-based Comparison\n\nA: Per your suggestions, we add TEST and CALF as baselines for comparison. Both of them perform cross-modal alignment from the time domain. In contrast, our method achieves better results by using frequency-domain alignment, which guides LLMs through simpler and clearer prompts. Additionally, our contextual segmentation endows time series with language-like structure. Assume that time series can be treated as character-like sequence, $[x_0,x_1,x_2,x_3,x_4,x_5] =$ [\"c\",\"a\",\"t\",\"d\",\"o\",\"g\"]. Our method assigns grouped indices [0,0,0,1,1,1], forming semantic units like \"cat\" and \"dog.\" Compared to LLMs' default sequential indexing [0,1,2,3,4,5], this grouping helps preserve trend continuity, enabling LLMs better modeling of time-series patterns.\n\n|||Weather|ECL|Traffic|ETTh1|ETTh2|ETTm1|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|CALF|MSE↓|0.250|0.175|0.439|0.432|0.349|0.395|\n||MAE↓|0.274|0.265|0.281|0.428|0.382|0.390|\n|TEST|MSE↓|0.229|0.162|0.430|0.414|0.331|0.353|\n||MAE↓|0.271|0.253|0.295|0.431|0.380|0.382|\n|S$^2$TS-LLM(Ours)|MSE↓|0.225|0.164|0.396|0.404|0.321|0.344|\n||MAE↓|0.265|0.256|0.278|0.429|0.376|0.380|\n\n## Q6:Backbone Fairness\n\nA: We follow GPT4TS and S$^2$IP-LLM in using GPT-2 as the backbone. As noted in Appendix (lines 41-42), the Time-LLM results reported are also based on GPT-2, ensuring fair comparison. To verify the robustness of our method across backbones, our ablation (Fig. 6) has included a BERT-based variant. Here, we further compare with Time-LLM using LLaMA, and results show that our method achieves comparable performance with fewer layers (4 vs. 8), verifying its architectural robustness.\n\n|||ETTh1||ETTh2||\n|:-:|:-:|:-:|:-:|:-:|:-:|\n||Length|96|192|96|192|\n|Time-LLM(8)|MSE↓|0.39|0.41|0.30|0.33|\n||MAE↓|0.42|0.43|0.37|0.38| \n|S$^2$TS-LLM(4)|MSE↓|0.39|0.42|0.30|0.35|\n||MAE↓|0.41|0.43|0.36|0.38|"}, {"review_id": "KZ0T8T4HwQ", "reviewer": "Reviewer_Ttyn", "summary": "The paper proposes a framework to repurpose large language models for time series analysis by focusing on the context gaps between natural language and time series data at the representation level. It introduces two core components, including a spectral symbolization that converts time series into sparse frequency-domain representations and then map them to ASCII symbols, and contextual segmentation that reassigns positional indices using GRU to capture temporal continuity of patched inputs. Experiments across multiple tasks demonstrate the effectiveness of the proposed method, including short-term and long-term forecasting, few-shot/zero-shot learning, and classification.", "questions": "1) Could the authors provide the evaluation of related LLM-based time series forecasting methods (mentioned above), to validate the claims in terms of outperforming state-of-the-art baselines?\n\n2) Could the author justify the motivation of using ASCII symbols for frequency component tokenization?\n\n3) In the methodology design, why the frequency-domain output in equation 8 only serves a gating function instead of an actual input to be fused?", "limitations": "Yes", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1. The high-level idea of using Fourier analysis in time series tokenization is somewhat  interesting. The idea of using an auto-regressive model to capture patched input for positional encoding is also well-motivated and interesting.  \n\n2. The proposed method is evaluated across a wide range of tasks, demonstrating certain advantages over the provided baselines.\n\n\nWeaknesses:\n\n1. This paper still lacks comparisons and discussions with several recent and relevant baselines in LLM-based time series analysis, particularly those focused on context modeling, such as TimeCMA [1] and FSCA [2].\n\n2. The ablation study only focuses on long-term time series forecasting, which seems insufficient to demonstrate the effectiveness of the provided component, especially for Q1.  Results of short-term and classification tasks would be needed for a more comprehensive assessment.\n\n3. The authors stated the disadvantages of time-domain tokenization with an example in lines 162-164, which seems not quite clear and precise. It would be better if a frequency-domain example is provided. Moreover, in line 182, the motivation of using ASCII symbols for frequency component tokenization is not clear, more insights will be helpful. \n\n4. In equation 8, it seems that the frequency-domain output *$O^{n}_{T}$* that passes through a sigmoid function only serves as a gating function, in addition to a cosine similarity for weighted fusion. This design is a bit confusing as *$O^{n}_{T}$* is never used as an actual input for fusion.\n\n[1] TimeCMA: Towards LLM-Empowered Multivariate Time Series Forecasting via Cross-Modality Alignment\n\n[2] Context-Alignment: Activating and Enhancing LLMs Capabilities in Time Series", "was_revised": true, "final_justification": "My concerns persist regarding the fact that, after extensive modeling stages for the gated residual component, it is further controlled by a cosine-similarity score, which blurs the clarity of its individual contribution and motivation. The additional results (including the comparison with state-of-the-art method) has reduced some of my initial concerns, and I am less negative toward this paper.", "rebuttal": "## W1, Q1: More Discussion and Comparison of Related LLM-based Methods\n\n1. **Discussion**\n\n    A: We will include more discussions on LLM-based related works in the revised version. Specifically, TimeCMA aims to enhance forecasting performance by leveraging LLM-generated text as an augmentation to time series data. To this end, it retrieves and selects textual information highly relevant to the time series, constructing disentangled and robust cross-modal embeddings for prediction. Unlike TimeCMA, which uses LLMs as external modules, our S$^2$TS-LLM tokenizes time series into textual descriptions, providing prompts to directly help LLMs better capture temporal dynamics during forecasting. FSCA addresses the cross-modal alignment problem in LLMs by representing time series and textual prompts as hierarchical graph nodes. It constructs a directed graph to characterize structural relationships between time series and prompts, and then uses graph convolution to fuse their representations for forecasting. In contrast, our S$^2$TS-LLM does not model the relationship between time series and textual prompts. Instead, we captures internal temporal context by reassigning the position indices of time-series patches, making the time-series representation more akin to linguistic context and thus easier for LLMs to interpret.\n\n2. **Comparison**\n\n    A: In light of your suggestions, we add TimeCMA for further long-term forecasting comparison, as listed below. Although both our method and TimeCMA achieve cross-modal alignment, they differ in direction: TimeCMA adopts an LLMs-for-Time-Series strategy, using LLM-generated text to augment time-series features without adapting LLMs to temporal understanding. In contrast, we adopt a Time-Series-for-LLMs strategy that transforms time series into  language-like representations by tokenizing frequency patterns and learning temporal context. This bridges the gaps between LLMs and time series, leading to improved performance of our method.\n\n    |Method|Metric|Weather|ECL|ETTh1|ETTh2|ETTm1|ETTm2|\n    |:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n    |TimeCMA|MSE↓|0.250|0.174|0.423|0.372|0.380|0.275|\n    ||MAE↓|0.276|0.269|0.431|0.397|0.392|0.323|\n    |S$^2$TS-LLM(Ours)|MSE↓|0.225|0.164|0.404|0.321|0.344|0.258|\n    ||MAE↓|0.265|0.256|0.429|0.376|0.380|0.318|\n\n## W2: More Ablation Study of Short-term Forecasting and Classification Tasks\n\nA: To further evaluate the capability of designed components, we conduct additional ablation study on short-term forecasting and classification tasks as follows. The results consistently show that each module contributes positively across different tasks. Notably, although reduced input length in short-term forecasting limits available frequency information, our method still effectively distinguishes temporal patterns by dynamically selecting top-$k$ components, providing valuable prompts to guide LLMs in forecasting. For classification, contextual segmentation proves most effective, likely because it transforms raw signals into discrete semantic units by reassigning index, making LLMs  easier to capture local patterns and improving performance.\n\n|Short-term Forecasting|Metric|S$^2$TS-LLM(Ours)|w/o Symbolization|w/o Segmentation|w/o Fusion|\n|:-:|:-:|:-:|:-:|:-:|:-:|\n|M4-Year.|SMAPE↓|13.306|13.718|13.680|13.537|\n||MASE↓|3.007|3.092|3.109|3.049|\n||OWA↓|0.785|0.809|0.810|0.789|\n|M4-Quart.| SMAPE↓|10.242|10.303|10.435|10.529|\n||MASE↓|1.209|1.219|1.239|1.254|\n||OWA↓|0.906|0.912|0.926|0.935|\n\n|Classification|Metric|S$^2$TS-LLM (Ours)|w/o Symbolization|w/o Segmentation|w/o Fusion|\n|:-:|:-:|:-:|:-:|:-:|:-:|\n|Heartbeat|Acc↑|79.5|77.6|77.1|78.1|\n|FaceDetection|Acc↑|70.3|68.3|67.9|68.6|\n\n## W3, Q2: Clarify Tokenization Example and Motivation of Using ASCII Symbol\n\n1. **Clarify Tokenization Example**\n\n    A: The example of time-domain tokenization in lines 162-164 aims to convey a message similar to the motivation illustrated in Fig. 1(a), but its simplified expression may lead to ambiguity. To clarify, consider the sequence $[66, 45, 24, 32, 27, 33, 16, 8]$. A time-domain tokenization that captures only the global trend might represent this sequence as \"decreasing from 66 to 8,\" thereby omitting intermediate fluctuations such as 24→32→27→33. To provide a clearer description, more words are required. For example, GPT-3.5 generates: \"The series starts at 66 and drops quickly to 24 over the first three points. After reaching 24, it briefly rises to 32 and 33, then drops again to 16 and finally 8.\" However, such a verbose textual description significantly exceeds the length of the original sequence. Using such long-sequence prompts to train LLMs for time series forecasting would result in significant computational overhead, making it impractical. Therefore, tokenizing a time series for prompting LLMs requires striking a balance between accuracy and efficiency, which is highly challenging.\n\n2. **More Explanation of Frequency-domain Tokenization**\n\n    A: Compared to time-domain tokenization, representing compact frequency-domain information in textual form is more tractable. For the same sequence $[66, 45, 24, 32, 27, 33, 16, 8]$, applying a real-valued FFT yields the frequency-domain components $[251+0j, 30.5-33.4j, 53.0-38.0j, 47,5-17.5j, 15+0j]$, which respectively represents the DC component and the 1/8 to 4/8 frequency bands. By selecting the top 2 components with the largest magnitudes for reconstruction, we obtain the sequence $[56.5, 35.6, 13.8, 33.4, 32.8, 46.2, 22.5, 10.4]$. Although the reconstructed values are not numerically accurate, the sequence successfully preserves the three key trends of the original: an initial decline, a fluctuation, and a subsequent drop. This demonstrates that the temporal pattern of a sequence can be effectively captured by tokenizing only a small number of dominant frequency components. In doing so, frequency-domain tokenization mitigates the limitations of time-domain approaches, which may introduce noise when overly concise or lead to high computational costs when overly verbose.\n\n3. **The Motivation of Using ASCII Symbols**\n\n    A: The motivation for using ASCII symbols to tokenize frequency components lies in the observation that different temporal patterns can be effectively characterized by combinations of salient frequency components. By assigning each frequency component a unique ASCII symbol, we can represent the top-k components of a sequence as a compact symbolic string. For example, combinations dominated by low-frequency symbols typically indicate smooth and stable trends of time series, while those containing more high-frequency symbols suggest volatile or rapidly changing patterns of time series.\n\n    Although ASCII symbols do not carry semantic meaning in natural language, LLMs can still learn their distributional statistics after fine-tuning[1]. In this way, ASCII tokens function similarly to foreign-language or domain-specific tokens, allowing the LLMs to associate certain patterns of symbols with frequency characteristics. As shown in our ablation study (Section 4.6), using ASCII symbols to represent frequency components achieves comparable forecasting performance to using natural-language words, demonstrating their effectiveness as interpretable and learnable tokens for LLMs.\n\n> [1] W. J. Su, et al. Do Large Language Models (Really) Need Statistical Foundations? arXiv, 2025\n\n## W4, Q3: Confusing Design of Time-Linguistic Fusion\n\nA: To clarify, our time-linguistic fusion (TLF) design is inspired by the Gated TCN module in GraphWavelet[2], which uses a gating mechanism of the form:\n$h = g(\\Theta \\star \\mathcal{X} + b) \\odot \\sigma (\\Theta \\star \\mathcal{X} + c)$. The sigmoid function $\\sigma(\\cdot)$ controls the proportion of information from $\\mathcal{X}$ that is propagated to the next layer.\n\nIn a similar spirit, we apply a sigmoid transformation to the frequency-domain output, which acts as a gating signal. This gate determines which parts of the time-series information should be preserved or suppressed during the fusion process. Following the gating, we incorporate cosine similarity to further modulate the fusion weights, emphasizing components of the time-domain signal that are more semantically aligned with the frequency-domain patterns.\n\nWe intentionally do not fuse the raw frequency-domain output directly with the time-domain representation, as the prediction task is inherently defined in the time domain. The frequency-domain output is instead treated as a complementary signal that guides the fusion process without shifting the prediction space. To evaluate the effectiveness of our fusion mechanism, we further conduct comparisons with several alternative fusion strategies, including bilinear fusion ($z = x^T W y$), gated fusion ($z = \\alpha x + (1 - \\alpha)y$), and attention-based fusion. The following experimental results show that our proposed TLF fusion achieves the best performance, consistently yielding lower prediction errors than the alternatives.\n\n|||TLF(ours)|Bilinear|Gate|Att|\n|:-:|:-:|:-:|:-:|:-:|:-:|\n||Length|MSE↓, MAE↓|MSE↓, MAE↓|MSE↓, MAE↓|MSE↓ , MAE↓|\n|ETTh1|96|0.355, 0.387|0.359, 0.389|0.367, 0.396|0.369, 0.397|\n||192|0.399, 0.419|0.412, 0.423|0.402, 0.422|0.401, 0.421|\n||336|0.402, 0.432|0.439, 0.441|0.423, 0.443|0.446, 0.463|\n||720|0.463, 0.478|0.462, 0.473|0.482, 0.485|0.502, 0.497|\n||AVG|0.405, 0.429|0.418, 0.431|0.418, 0.436|0.430, 0.444|\n|ETTh2|96|0.260, 0.329|0.276, 0.335|0.273, 0.334|0.291, 0.345|\n||192|0.312, 0.363|0.338, 0.375|0.335, 0.377|0.340, 0.379|\n||336|0.322, 0.380|0.336, 0.385|0.333, 0.386|0.348, 0.398|\n||720|0.392, 0.430|0.393, 0.428|0.400, 0.432|0.406, 0.439|\n||AVG|0.322, 0.376|0.336, 0.381|0.335, 0.382|0.346, 0.390|\n\n> [2] Z. H. Wu, et al. Graph WaveNet for Deep Spatial-Temporal Graph Modeling. IJCAI, 2019"}]

Author rebuttal:

Meta-review: This paper tries to bridge time series and language with spectral symbolization and contextual segmentation. The idea is creative and the experiments are pretty broad, which most reviewers liked. At the same time, the novelty isn’t huge, since spectral ideas have been tried before, and things like the ASCII symbols and fusion design didn’t fully convince everyone. The rebuttal helps -- the authors added ablations, extra comparisons (FSCA, TimeCMA), and clarified some design choices, which made the work look more solid. While one reviewer stayed skeptical, the others leaned positive after the response. 

Overall, I see this as a decent contribution, not perfect but worth accepting. Given the tough competition this year, I’d mark it as a weak accept.
Decision: Accept (poster)