Are Language Models Actually Useful for Time Series Forecasting?
Authors: Mingtian Tan
Mike A Merrill
Vinayak Gupta
Tim Althoff
Thomas Hartvigsen
Venue: NeurIPS 2024 spotlight (2024)
Keywords: Time Series
Language Models
Time Series Forecasting
Primary area: evaluation
TL;DR: LLM in Time Series Forecasting Task

Abstract: Large language models (LLMs) are being applied to time series forecasting. But are language models actually useful for time series? In a series of ablation studies on three recent and popular LLM-based time series forecasting methods, we find that removing the LLM component or replacing it with a basic attention layer does not degrade forecasting performance---in most cases, the results even improve! We also find that despite their significant computational cost, pretrained LLMs do no better than models trained from scratch, do not represent the sequential dependencies in time series, and do not assist in few-shot settings. Additionally, we explore time series encoders and find that patching and attention structures perform similarly to LLM-based forecasters. All resources needed to reproduce our work are available: https://github.com/BennyTMT/LLMsForTimeSeries.

Are Language Models Actually Useful for Time Series Forecasting?
Mingtian Tan
Mike A. Merrill University of Washington
Vinayak Gupta
University of Virginia wtddgzovirggini. edu
University of Washington
mikeam@cs washington edu
vinayak@cs washington edu
Tim Althoff
Thomas Hartvigsen
University of Washington althoffecs. washington
University of Virginia
hartwiggen@virginiaa.c edu
Abstract
Large language models (LLMs) being applied to time series forecasting. But are language models actually useful for time series? In a series of ablation studies three recent and popular LLM-based time series forecasting methods, find that removing the LLM component or replacing it with a basic attention layer does not degrade forecasting performance-inn most cases, the results even improve! We also find that despite their significant computational cost, pretrained L.LMs do no better than models trained from scratch, do not represent the sequential dependencies in time series, and do not assist in few-shot settings. Additionally, we explore time series encoders and find that patching and attention structures perform similarly to LLM-based based forecasters
Introduction
Time series analysis is critical problem many domains, including disease propagation
forecasting (8) retail sales analysis 3 healthcare 26 17 and finance [31] A great deal of
recent work in time series analysis (constituting repositories with more than 1200 total stars GitHub) has focused on adapting pretrained large language models (LLMs) to classify, forecast, and detect anomalies time series [15] 50 22
44 [6]. These papers posit that language models, being advanced models for sequential dependencies in text, may generalize to the sequential dependencies in time series data. This hypothesis is unsurprising given language models are now pervasive in machine learning research. However, direct connections between language modeling and time series forecasting remain largely undefined. So to what extent is language modeling really beneficial for traditional time series tasks?
32
Our claim is simple but profound: popular LLM-bbsed time series forecasters perform the worse than basic LLM-free ablations, yet require orders of magnitude more compute. Derived from extensive ablations, this reveals worrying trend in contemporary time series forecasting literature. Our goal is not to imply that language models will never be useful for time series. In fact, recent works point many exciting and promising ways that language and time series interact, like
time series reasoning [25] 7 45 (37) social understanding [6] and financial reasoning [36 20).
Rather, we aim to highlight surprising findings that existing methods do very little use the innate reasoning power of pretrained language models on established time series tasks.
We substantiate claim by performing three ablations of three popular and recent LLM--bssed forecasting methods 50
22 using eight standard benchmark datasets from reference methods
resources needed to reproduce our work are available: htap:://github..co//Bmnyyy
AMMForTimeeeriie
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

and another five datasets from MONASH [13. First, we successfully reproduce results from the original publications. Then, we show that replacing language models with simple attention layers. basic transformer blocks, randomly-initialized language models, and even removing the language model entirely, yields comparable or better performance. The same performance was observed on another five datasets that were not studied by the reference methods.
Next, we compare the training and inference speed of these methods against their ablations, showing that these simpler methods reduce training and inference time by up to three orders of magnitude while maintaining comparable performance. Then, to investigate the source of LLM forecaster's performance, we further explore time series encoders. We find that a simple linear model with an encoder composed of patching and attention can achieve forecasting performance similar to that of LLMs. Next, we test whether the sequence modeling capabilities of LLMs transfer to time series by shuffling input time series and find no appreciable change in performance. Finally, we show that L.MMs do not even help forecasting in few-shot settings with 10% of the training data. We discuss the implications of our findings and suggest that time series methods that use large language models are better left to multimodal applications 4 12 38 that require textual reasoning.
The key contributions we make in this paper are as follows:
We propose three straightforward ablation methods for methods that pass time series into LLMs for forecasting. We then ablate three top-tier methods on thirteen standard datasets and find that LLMs fail convincingly improve time series forecasting. However, they significantly increase computational costs in both training and inference.
We study the impact of an LLM's pretraining by re-initializing their weights prior to forecasting. We find that this has no impact on forecasting performance. Additionally, in shuffling input time series, we find no evidence the LLMs successfully transfer sequence modeling abilities from text to time series and no indication they help in few-shot settings.
We find a very simple model, with patching and attention as encoder, can achieve performance similar to LLMs. This suggests massive gap between the benefits LLMs pose and the time series forecasting problem, despite a rapid rush to adopt LLMs.
2 Related Work
Here, we summarize the key works relevant to LLM-based time series models. They can be broadly classified into three sections: (i) time series forecasting using LLMs; (ii) encoders in LLM time series models; and (iii) smaller and efficient neural models for time-series.
Time Series Forecasting Using LLMs. Recently, with the development of Large Language Mod- els (LLMs) IO
29 34 and their demonstrated multi-modal capabilities, more researchers have successfully applied LLMs to time series forecasting tasks [14] 16, 5 4.. Chang et al., [S] used finetuning the transformer module and positional encoding in GPT-2 to align pre-trained LLMs with time series data for forecasting tasks. Zhou et al. [50] proposed similar finetuning method, named AIl"., for time series forecasting with GPT-2. Additionally, Jin et al. (15) introduced a reprogramming method to align L.MM's Word Embedding with time series embeddings, showing good representation of time series data on L.aMA (34). Similarly, CALF [22] and TEST (32) adapted word embeddings to enable LLMs to forecast time series data effectively. In addition to series forecasting models, Liu et al. (23 show that these models can be extended to classifying health-time series, such as heart-rate and daily-footsteps, These models have also been shown to outperform supervised neural models in few-shot settings.
Encoders in LLM Time Series Models. In order for an LLM to learn from text it must first be discretized and encoded as word tokens which are 1 x d vectors (T0 29 34 Similarly, basdd methods for time series learn discrete time series tokens. One method is to segment the time series into overlapping patches, which effectively shortens the time series while retaining its features
15 50 5 28 [1]. Other methods decompose time series into trend, seasonal components, and residual components 4 28 Lastly, Liu et al. [22] feed the multivariate time series using a Transformer to enable different channels to learn the dynamics of other channels. These embedding procedures are followed by linear neural network layer that projects the time series encoding to the same dimensions used by the pre-trained LLM.

---

Dataset
ETTh & ETTh2 ETTml & ETTm2
Traffic
Electricity
Weather
Iilness
Channels Sampling- Rate Timesteps
862
7
Hour 17,420
15 Min.
Hour
Hour
10 Min. 52,696
Week
69,680
17,544
26,304
966
Table I: Statistics for all datasets used in reference methods 50
15
Method
Base Model
Learnable
Positional Embeukiings
Align Word Embeddings
Multimodal
LM Parameters
OneFitsAll 50
GPT-2
Add&Norm
Fine-Tune
Time-LLM 15
LLaMA
None
Freeze
CALF 22
GPT-2
LoRA
Fine-Ture
Table 2: Three popular methods for time series forecasting with Large Language Models. Small and Efficient Neural Forecasters. addition to LLMss, there has been large body of research smaller yet efficient frameworks that outperform their bulky counterparts in time series forecasting [19 47
33 24 2]. For example, Zeng et al. [46] present DLinear, an incredibly simple model that combines decomposition techniques and achieves better forecasting performance than state-of-th tor--t--tth--tttt transformer- based time series architectures at the time, such as Informer [48], FED- former [49]. and Autoformer [40). Furthermore, Xu et al. [3] introduces lightweight model with only 10k parameters, which captures both amplitude and phase information in the time-series to outperform transformer-basee based models.
3 Experimental Setup
We use three state-of-the-art methods for time series forecasting and propose three ablation methods for LLMs: (i) "w/o LLM"; (ii) "LLM2Attn"; (iii) and "LLM2Trsf". To evaluate the effectiveness of LLMs in time series forecasting, test these methods on eight standard datasets.
3.1 Reference Methods for Language Models and Time Series
We experiment with three recent methods for time series forecasting using LLMs. All models were published between December 2023 and May 2024 and are popular, with their GitHub repositories collectively amassing 1,245 stars. These methods are summarized in Table 2 and use either GPT-
2 29 LLaMA
34 base models, with different alignment and fine-tuning strategies.
OneFitsAll [50] OneFitsAll, sometimes called GPT4TS, applies instance norm and patching to the input time series and then feeds it into linear layer to obtain input representation for the language model. The multi-head attention and feed forward layers of the language model are frozen while the positional embeddings and layer norm optimized during training. A final linear layer is used to transform the language model's final hidden states into a prediction.
Time-LLM [15]: In Time-LLM the input time series is tokenized via patching and aligned with a diwddienniooal representation of word embeddings using multi-head attention. The outputs of this alignment, combined with the embeddings of descriptive statistical features, passed to a frozen pre-trained language model. The output representations of the language model are then flattened and passed through linear layer obtain forecast.
CALF [22]: CALF embeds the input time series by treating each channel as a token. One half of the architecture is "textual branch" which uses cross attention align the time series representation with a low dimensional representation of the language model's word embeddings. This representation is then passed through pretrained, frozen language model obtain "textual prediction".. Simultaneously, a "temporal" branch learns a low-rank adapter for a pretrained language model based on the input time series produce "temporal prediction" which used for inference. The model includes additional loss terms that enforce similarity between these representations.
Reproducibility Note. While experimenting with each model, we tried to replicate the conditions of their original papers. We used the original hyper-parameters, runtime environments, and code, including model architectures, training loops, and data-loaders. To ensure a fair comparison, we have included error metrics from the original papers alongside our results wherever possible.
3

---

Projectin Iomeenatinn
Projection
Projection
Projection
Pre-trained LLM
Atteerton
Transformaet
Alggnmmnnt
Encoding
Encoding
Encoding
Instamec
Froszing
C www.
Trainable
Aliguing
Topot. Time Seriios
Inpoi Timm Seriee
Ingont Time Sectiee
Inpat Timm Serime
(a) w/ LLM
(b) w/o LLM
(c) LLM2Attn
(d) LLM2Trsf
Figure 1: Overview of all LLM ablation methods. Figure (a) represents time series forecasting using an LLM as the base model. In some works, the LLM components are frozen [15 14. while in others, they undergo fine-tuning (50 22 4]. Figure (b) shows the model with the LLM components removed, retaining only the remaining structure. Figure (c) replaces the LLM components with single-layer self-attention mechanism. Figure (d) replaces the LLM components with a simple Transformer.
3.2 Proposed Ablations
To isolate the influence of the LLM in an LLM- M-adededd forecaster, we propose three ablations: removing the LLM component replacing it with simple block. Specifically, for each of the three methods we make the following three modifications:
w/o LLM (Figure (b)). We remove the language model entirely, instead passing the input tokens directly to the reference method's final layer.
LLM2Attn Figuue (c)). We replace the language model with a single randomly-inittalized
multi-head attention layer.
LLM2Trsf FFigure I (d)). We replace the language model with single randomly-initialzzed
transformer block.
In the above ablations, keep left parts of the forecasters unchanged (trainable). For example, as shown in Figure I (a), after removing the LLM, the input encodings are passed directly to the output projection. Alternatively, shown in Figure (b) or (c). after replacing the LLM with attention or transformer, they are trained along with the remaining structure of the original method.
3.3 Datasets and Evaluation Metrics
Benchmark Datasets. We evaluate the following real-world datasets: (1) ETT (1]: encompasses seven factors related to electricity transformers across four subsets: ETThl and ETTh2, which have hourly recordings, and ETTml and ETTm2, which have recordings every 15 minutes; (2) Illness [40]: includes the weekly recorded influenza illness among patients from the Centers for Disease Control, which deseribes the ratio of patients seen with influenza-like illness the total number of patients; (3) Weather 400: local climate data from 1,600 U.S. locations, between 2010 and 2013, and each data point consists of 11 climate features; (4) Traffic [40]: is hourly dataset from California transportation department, and consists of road occupancy rates measured on San Francisco Bay area freeways; (5) Electricity 335 contains the hourly electricity consumption of 321 customers from 2012 to 2014. The train-val-test split for ETT datasets is 60%-20%- 20%, and for Illness, Weather, and Electricity datasets is 10%--10%-00 respectively. The statistics for all datasets is given in Table I We highlight that these datasets, with the same splits and size, have been extensively used evaluate time-series forecasting ability of LLM-based and other neural models for time-series data [88 50 4 15
40 49). (6) Exchange Rate [18] collected between 1990 and 2016, it contains daily exchange rates for the currencies of eight countries (Australia, British, Canada, Switzerland, China, Japan, New Zealand and Singapore).
5
46
(7) Covid Deaths [3]] contains daily statistics of COVID-19 deaths 266 countries and states between January and August 2020. (8) Taxi (30 min) [1: contains taxi rides from 1,214 locations in New York City between January 2015 and January 2016. The data is collected every 30 minutes, with an average of 1,478 samples. (9) NNS (Daily) 13] contains daily cash withdrawal data from 111 ATMs in the UK, with each ATM having

---

ETThI-OneFitse
ETTm2 ETTm2-0nneitse OneFitsAll
ECL- Ftell
w/ LLM
LLM W/O LLM LLM2Att
LLM O LLM LLM2Att LLM2Trsf
W/o LLM
LLM2Att
LfM2TTsf
Lf
0.420
0.450
0.310
0.322
0.247
0.260
MAR
ETTEI-CALF'
ETTm2-CALF
ECL-CALF
w/ LLM LLM
W/ LLM LLM LLM2AG0 L.M2T
w/ LLM LLM LLM2AMD LLM2T
LLM2A1G LLM2TTsf
0.426
0.429
0.319
0.322
0.261
0.264
MAR
MAR
MAR
ETTMI-TmeLLM
ETTm2 CTTmm-TTmeLLA TimeLLM
ECL- Time-Lm
w/ LLM
w/ LLM w/O LLM MH
w/ LLM W:O LLM LLM2Attn Lf
w/o LLM
HH
LLM2Attn LLM2TTsf
HH
LLM2Attn LLM2TTsf
0.420
0.435
0.310
0.325
0.256
0.268
MAE
MAR
MAB
Figure 2: In the above examples, only OneFitsAll "w/ LLM" performs better than the ablation methods on ETTh1, but there is substantial overlap in bootstraped confidence intervals. The figures show the comparison of OneFitsAll, CALF, and Time-LLM using LLMs and ablations (i.e., w/o LLM, LLM2Attn, and LLM2Trsf)
ETThI, ETTm2, and Electricity, and the vertical dashed lines represent the results from the original work. Others Figures for MSE and other datasets are available in Figure 5 and Figure 6 in the Appendix.
791 data points. (10) FRED-MD 13: contains 107 monthly macroeconomic indices released by the Federal Reserve Bank since 01/01/1959. was extracted from the FRED-MD database.
Evaluation Metrics and Setup. We report the results in terms of mean absolute error (MAE) and mean squared error (MSE) between predicted and true values of the time-series. Mathematically, given
a test-set with D elements, MAE = Toj EneD [|c - []]] and MSE = DD Etep(G - a)²² where
C; and E denote the true value and predicted value at the i-th index of the time-series respectively.
4 Results
In this section, we provide the details of our comprehensive evaluation of all baseline LLM models for time-series forecasting. Specifically, we ask the following research questions. (RQ1) Do pretrained language models contribute to forecasting performance? (RQ2) Are LL-M-based methods worth the computational cost? (RQ3) Does language model pretraining help performance forecasting tasks? (RQ4) Do LLMS represent sequential dependencies in time series? (RQ5) Do LLMs help with few-shot learning? (RQ6) Where does the performance come from?
4.1 Do pretrained language models contribute to forecasting performance? (RQ1)
Our results show that pretrained LLMs not useful for time series forecasting tasks yet. Overall, as shown in Table 3] across 13 datasets and two metrics, ablations out perform Time-LLM methods in 26/26 cases, CALF in 22/26 cases, and OneFitsAll in 19/26 cases. We averaged results over different predicting lengths, as in (50 [5 (2). Across all prediction lengths (thirteen datasets and four prediction lengths) ablations outperformed Time-LLM. CALF, and OneFitsAll in 35/40 /0, 31/40, 40, and 29/40 cases as measured by MAE, respectively. To ensure fair comparison, we also report results from each method's original paper alongside our replication. For specific results refer to Appendix E To better evaluate the effectiveness of LLMs and ablation methods, we include 95% bootstrapped confidence intervals for each task. In tasks where LLMs performed better, such as OneFitsAll with ETTh1, shown in Figure 2) there is still substantial overlap in the confidence intervals with the ablation method "w/o L.M in MAE. Other datasets results and MSE metrics are shown in Figure 5) and Figure 6 in the Appendix. To summarize, our results on the evaluation above, it is hard to conclude that L.MMs are effective time series forecasting.
4.2 Are LLM-based methods worth the computational cost? (RQ2)
In the previous section, we showed that L.MS do not meaningfully improve performance on time series forecasting tasks. Here, we evaluate the computational intensity of these methods with their
5

---

Model
Time-LLM
w/o LLM
LLMZAtt
LLMETes
From Original Paper
Dataset
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
ETTh1
0,432
0.417
0.419
0.405
0.437
0,422
0,439
0.429
0.423
0.408
ETTh2
0.396
0.360
0.383
0.345
0.389
0.353
0.394
0.359
0.383
0.334
Time-LLM [8)
ETTm1
0.356
0.329
ETTmm
0.315
0.260
0.307
0.252
0.314
0.259
0.310
0.253
0.329
0.250
Illness
2.017
0.924
1,956
0.849
1.435
Weather
0,270
0.243
0.272
0,243
0,254
0.224
0.254
0.226
0.257
0.225
Traffic
0.281
0.421
0.295
0.428
0.276
0.416
0.275
0.416
0.263
0.387
Electricity
0,259
0.164
0.269
0,171
0,260
0,167
0.254
0.161
0,252
0.158
Exchange Rate
0.448
0.422
0.413
0.384
0.432
0.403
0.442
0.422
Covid Deaths
0.189
0.080
Taxi (30 Min)
0,277
0.163
0.286
0.176
0,269
0.157
0.255
0.141
NN5 (Duily)
0.432
0.402
0.425
0.379
0.411
0.364
0.401
0.347
FRED-MD
0.0004
5e-7
0.0002
36-7
0.0046
2.53e-5
0.0008
2.6e-6
# Wins aParameters
12
2
12
6651. 82M.
0.55M
0.55M
0.66M
Model >
CALF
w/o LLM
LLMZAtte
LLM2Trsf
From Original Paper
Dataset
MSE
MAE
MAE
MSE
MSE
ETThi
0.431
0.431
0.428
0.436
0.430
0.428
0,430
0.430
0.428
0.432
ETTh2
0.351
0.382
0.350
0.349
ETTm]
0.391
0.396
0.390
0.397
0.390
0.396
0.390
0.394
0.390
0.395
CALF NN
ETTm2
0.322
0.321
0.281
Ilnness
0,869
1,699
0.861
1.639
0.892
1.748
0.860
1.630
Weather
0.273
0,251
0.277
0.257
0.279
0.258
0.277
0.255
0.274
0.250
Traffic
0,284
0.443
0.278
0.439
0.275
0.271
0.426
0,281
0.439
Electricity
0.266
0.175
0,262
0,174
0.264
0.175
0.261
0.172
0.265
0.175
Exchange
0,388
0.409
0,417
0.367
Covid Deaths
0,084
0.163
0.066
0,115
0.131
0,431
0,066
0.106
Taxi (30 Min)
0.258
0.142
0.264
0.147
0.267
0.150
0.267
0.150
NNS (Daily)
0.403
0.362
0.386
0.336
0.463
0.445
0,415
0.581
FRED-MD
0.0012
2,90-6
0.0011
2,7e-6
0.0015
4.9c-6
0.0017
4.5o-6
# Wins iParameters
11
180.25M
8.17M
10.5M
13.68M
Model Dataset
OneFitsAll
LLM
LLMZAtte
LLMZTst
Original Paper
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
ETTh1
0.420
0.417
0.422
0.417
0.452
0.465
0.525
0.426
0.427
ETTh2
0.388
0.353
0.389
0.356
0.402
0.375
0.397
0.367
0.394
0.354
ETTml
0.377
0.362
0.374
0.358
0.379
0.369
0.379
0.369
0.383
0.351
ETTm2
0,310
0.253
0.309
0,255
0.312
0,256
0,310
0.254
0.326
0.266
OneFitAAl
Illness
0.852
1.871
0.924
1,960
0.829
1.763
0.850
1.830
0.903
1.925
Weather
0.226
0.256
0.236
Traffic
0,273
0.420
0.273
0.439
0.266
0.415
0.256
0.409
0,294
0.414
Electricity
0.169
0.254
0,250
0.162
0.157
0.166
Exchange Rate
0,378
0.357
0.361
0.323
0.376
0,350
0.393
0.387
Covid Deaths
0.057
0.075
0.050
0.073
0.058
0.103
0.078
0.162
Taxi Min)
0,252
0.138
0.259
0,143
0.259
0.145
0.257
0.140
NN5 (Daily)
0.438
0.438
0.422
0.385
0.423
0.390
0,420
0.386
FRED-MD
1.2e-6
0.0002
# Wins AParameters
7
11
3
5
91,36M
9.38M
10.71M
13.54M
Table 3: Forecasting performance of all models Time-LLM, CALF, and OneFitsAll and results from our ablations. All results are averaged across different prediction lengths, though full results are available in Appendix [ Results in Red denote the best-performing model. # Wins refers to the number of times the method performed best, and # Params is the number of model parameters. "L" means the dataset is not included in the original paper.
nominal performance in mind. The language models in our reference methods use hundreds of millions and sometimes billions of parameters to perform time series forecasting. Even when the parameters of the language models are frozen they still contribute to substantial overhead during training and inference. For instance, Time-LLM has 6642 M parameters and takes 3003 minutes to train on the Weather dataset whereas ablation methods have only 0.245 M parameters and take 2.17 minutes on average. Information about training other methods on ETTh1 and Weather datasets are shown in Table 4 the case of inference time, we divide by the maximum batch size to give an estimate of inference time per example. Time-LLM, OneFitsAll, and CALF take, on average, 28.2,

---

Time-LLM-ETT
OueFitsAl-ETTmm2
CALF-ETTm2
0.318
0.324
0,312
0.310
0.320
0,305
100
1.38
2.10
Time- Tine-LLM- LLM-TTrdite
OneFitaAD- -raa--eaffe
CALF- -raffcc
0.300
0.288
MAR
0.290
0.270
0.280
0.280
0.260
*
0.272
0.30
0.45
0.60
30.00
60.00
90.00
Time-LLM-ECL
AlD
CALF-ECL
0.272
0.270
0.264
0,260
0.265
0.256
*
0.250
0.260
0.30
0.45
0.60
12.00
18.00
Inference Time (ms)
Inference Time (ms)
Inference
Time (uus)
w/ LLM
w/o LLM
LLM2Astn
* LLM2TTsf
Figure 3: Ablation methods consume less time for inference while providing better forecasting performance. The figure above shows the inference time and prediction accuracy of Time-LLM, OneFitsAll, and CALF on ETTm2, Traffic, and Electricity datasets, averaged across prediction lengths. For more datasets and MSE metries refer to Fogurg 7and Figure 8 in the Appendix.
Method
Time-LLM (LLaMA)
OneFitsAll (GPT-2)
CALF (GPT-2)
Param (M) Time (min)
Param (M) Time (min)
Param (M) Time (min)
w/ LLM
6652
181
3.28
w/o LLM
0.198
0.27
0.35
LLM2Att
0,202
LLM2Trsf
0.336
0.40
Weather
w/ LLM
6642
3003
86
152
180
12
w/o LLM
0.198
1.91
16
8
2.32
LLMZAtta
0.202
2.22
7
21
10
2.14
LLM2Trsf
0.336
2.38
10
24
13
1.89
Table 4: In time series tasks, LLM (LLaMA and GPT-2) significantly increases training time. The table shows the number of model parameters (in millions) and total training time (in minutes) for three methods predicting over length of 96 ETThl and Weather data. Compared with original method "w/ LLM" are "w/o LLM", "LLM2Attn" and "LLM2Trss".
2.3, and 1.2 times longer than the modified models. Examples can be seen in Figure 3 where the green marks (ablation methods) are typically below the red one (LLM) and are positioned towards the left of the axis, indicating lower computational costs and better forecasting performance. Other datasets and MSE metric refer to Figiure FFnuue iin Appendix, conclusion, the computational intensity of LLMs in time series forecasting tasks does not result in corresponding performance improvement.
4.3
Does language model pretraining help performance on forecasting tasks? (RQ3)
Our evaluation in this section indicates that pretraining with language datasets is unnecessary for time series forecasting, To test whether the knowledge learned during pretraining meaningfully improves forecasting performance we experimented with different combinations pretraining and finetuning CALF's 22 language model time series.
Pretrain Finetune (Pre+FT). This is the original method, wherein pretrained language model is finetuned on time series data. the case of CALF, the base language model is frozen and low rank adapters (LoRA) are learned.
Random Initialization Finetune (woPre+FT)), Does the textual knowledge from pretraining aid time series forecasting? In this method we randomly initialize the weights of the language model (thereby crasing any effect of pretraining) and train the LLM from scratch.
Pretrain + No Finetuning (Pre+woF')). How much does finetuning on time series improve prediction performance? For this baseline we again leave the language model frozen and forgo

---

Methods
Pre+FT
(GPT-2)
woPre+FT
Pre+woFT
woPre+woFT
MAE
MSE
MAE
MSE
MAE
MSR
MAE
MSR
ETTh1
0,4312
0.4313
0.4284
0.4362
0.4267
0,4342
0.4365
0.4474
ETTh2
0.3838
0.3510
0,3839
0.3508
0.3830
0.3514
0.3872
0.3554
ETTml
0.3910
0,3963
0.3933
0.4013
0.3898
0.3954
0.3949
0,4028
ETTm2
0.3230
0.2831
0.3221
0,2852
0.3221
0.2827
0.3224
0.2829
Illness
0.8691
1.6996
0.8523
1.6146
0.8742
1.6640
0.8663
1.6381
Weather
0.2737
0.2510
0.2760
0.2520
0.2771
0,2535
0.2776
0.2582
Traffic
0.2844
0.4438
0.2771
0.4409
0.2820
0,4446
0.2863
0.4483
Electricity
0.2660
0. 0.1758
0.2597
0.1669
0.2635
0.1730
0.2663
0.1784
# Wins:
3
5
0
Table 5: Randomly initializing LLM parameters and training from scratch (woPre) achieved better results than using pretrained (Pre) model. "woFI" and "FT"" refer to whether the LLM parameters are frozen or trainable.
Dataset Input Ablation
ETTh1
Illness
Sf-all.
Sf-half.
Ex-half
Masking
Sf-all.
Sf-half.
Ex-half
Masking
Time-LLM
51.8%
5.6%
79.6%
32.5%
99.05 99.0%
33.6%
34.9%
64.6%
w/o LLM
56.0%
4.5%
89.7%
39.5%
76.5%
20.9%
18.4%
53.0%
LLMZAtta
53.8%
3.3%
92.2%
33,8%
72.7%
20.4%
13.1%
44.6%
LLM2Trsf
50.3%
3.4%
89.2%
34.8%
74.5%
23.0%
14.3%
49.3%
OneFitAll
62.1%
6.1%
16.6%
31.3%
86.2%
30.9%
36.7%
77.5%
w/o LLM
58.6%
6.1%
19.2%
36.1%
68.9%
13.0%
17.3%
43.5%
LLM2Attn
68.5%
9.0%
15.0%
34,4%
108.3%
39.8%
44.2%
74.2%
LLMZTrsf
58.0%
7.8%
12.6%
30.2%
90.8%
27.4%
40.3%
60.6%
CALF
50.5%
9.6%
5.6%
8.5%
113.0%
47.4%
24.4%
22.9%
w/o LLM
56.2%
12.1%
6.1%
10.4%
118.0%
50.4%
45.8%
28.9%
LLM2Attn
51.9%
10.8%
5.8%
7.3%
87.3%
42.4%
35.1%
25.8%
LLMZTrsf
50.3%
8.5%
5.5%
7.0%
102.6%
56.2%
32.6%
26.0%
Table 6: For the input shuffling/masking experiments on ETTh1 (predict length is 96) and Illness (predict length is 24), the impact of shuffling the input the degradation of time series forecasting performance does not change significantly before and after model modifications. Results of other predict lengths refer table 21 in Appendix.
learning LoRAs Results from this model are therefore indicative of the base language model's performance without additional guidance on processing time series.
Random Initialization No Finetuning (woPre+woFT), This baseline is effectively a random projection from the input time series to forecasting prediction and serves as a baseline comparison with the other methods.
Overall, shown in Table 5 across 8 datasets using MAE and MSE metrics, the "Pretraining + Finetune" method performed the best 3 times, while "Random Initialization + Finetune" achieved this 8 times. This indicates that language knowledge offers very limited help for forecasting. However, "Pretrain + No Finetuning" and the baseline "Random Initialization + No Finetuning" performed the best 5 times and 0 times, respectively, suggesting that Language knowledge does not contribute meaningfully during the finetuning process. Detailed results refer to Table 20 in Appendix.
In summary, textual knowledge from pretraining provides very limited aids for time series forecasting.
4.4 Do LLMS represent sequential dependencies in time series? (RQ4)
Most time series forecasting methods that use LLMs finetune the positional encoding to help under- stand the position of time steps in the sequence 4 50 22 5 32] We would expect time series model with good positional representations show a significant drop in predictive performance when the input is shuffled [46]. We applied three types of shuffling to the time series: shuffling the entire sequence randomly ("sf-all"), shuffling only the first half of the sequence ("sf-half"), and swapping the first and second halves of the sequence ("ex-half"). As shown in Table 6) L-M--bsed methods were no more vulnerable to input shuffling than their ablations. This implies that L.MSs do not have unique capabilities for representing sequential dependencies in time series.
8

---

Model
LLaMA
w/o
LLM
LLM2Attn
LLM2Trsf
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
ETTh
0.522
0.555
0.543
0.621
0.547
0.611
0.566
0.656
ETTh2
0.394
0.371
0.432
0.407
0.437
0.416
0.433
0.412
ETTml
0,426
0.404
0.403
0.393
0.428
0.440
0.438
0.457
mTTm2
0.323
0.277
0.319
0.269
0.355
0.316
0.351
0.311
Weather
0.273
0.234
0.271
0.241
0.275
0.241
0.271
0.239
Traffic
0.306
0.429
0.303
0.431
0.302
0.432
0.298
0.432
Electricity 0.270
0.175
0.275
0.175
0.273
0.179
0.274
0.181
Wins:
7
Table 7: In few-shot scenarios (10% dataset), LLaMA (Time-LLM) performs similarly to the ablation methods. LLaMA and "w/o LLM" each outperformed the other 8 times. Note that the results of Time-LLM is from the original paper [1].
Model
GPT-2
W/o
LLM
LLM2Attn
LLM2Trsf
MAE
MSE
MAE
MSE
MAE
MSR
MAE
MSR
ETTh
0.543
0.649
0.542
0.642
0.545
0.646
0.544
0.643
ETTh2
0.433
0.434
0.429
0.427
0.433
0.431
0.432
0.434
ETTml
0,500
0.574
0.499
0.572
0.503
0.581
0.507
0.586
RTTm2
0.339
0.304
0.338
0.303
0.340
0.305
0.340
0.305
Weather
0.286
0.263
0.287
0.265
0.293
0.270
0.286
0.264
Traffic
0.369
0.571
0.337
0.517
0.341
0.518
0.333
0,510
Electricity
0.301
0.220
0.287
0.205
0.292
0.208
0.290
0.206
Wins:
10
2
Table 8: few-shot scenarios (10% dataset), Ablation methods perform much better than GPT-2 (CALF). Without LLMss 12 out of 14 cases showed better performance.
4.5 Do LLMs help with few-shot learning in forecasting? (RQ5)
In this section, our evaluation demonstrates that LLMs are still not meaningfully useful in few-shot learning scenarios.
While our results indicate that LLMs are not useful for time series forecasting, it is nonetheless possible that knowledge encoded in pretrained weights could help performance in few-shot settings where data are scarce. To evaluate whether this is the case we trained models and their ablations on 10% of each dataset. Specifically, we evaluated LLaMA in Time-LLM methods. The results for LLaMA, shown Table 7] compared LLaMA with completely removing the LLM (w/o LLM). There was no difference, with each performing better in 8 cases. We conducted similar experiments with CALF, GPT-2-basce method. Our results in Table indicate that our ablations can perform better than LLMs in few-shot scenarios.
4.6 Where does the performance come from? (RQ6)
In this section, we evaluate common encoding techniques used in LLM time series models. We find that combining patching with one-layer attention is simple and effective choice.
subsection 3.2] we found that simple ablations of LLM-based methods did not decrease performance. To understand why such simple methods work so well we selected some popular techniques used for encoding in
Projection
LLM time series tasks, such patching [50] 15 5 32 22 decom-
Actemtion
position 4 (28). basic transformer block also can used to aid in
encoding 22
Projjection
The specific results, shown in Table 18 in the Appendix, indicate that structure combining patching and attention, named "PAttn", performs bet- ter than most other encoding methods on small datasets (with time stamps less than million) and is even comparable to L.M methods. Its detailed structure, as shown Figure 4 involves applying "instance norm" to the time series, followed by patching and projection. Then, one-layer
Figure 4: PAttn Model.

---

or social attention enables feature learning between patches. For larger datasets, such as Traffic (-15 million) and Electricity (-8 million), a model named "LTrsf," " using the encoder from CALF [22]. performs better. In those methods, finally, time series embedding will be projected with single linear layer to forecast. Details of other encoders Appendix subsection D.3
Overall, patching plays crucial role in encoding. Additionally, basic Attention and Transformer blocks also effectively aid in encoding.
Conclusion
In this paper we showed that despite the recent popularity of LLMs in time series forecasting they do appear to meaningfully improve performance. We experimented with simple ablations, showing that they maintain or improve the performance of the LLM-based counterparts while requiring considerably less compute. Once more, our goal is not to suggest that LLMs have no place time series analysis. To do so would likely prove to be a shortsighted claim. Rather, we suggest that the community should dedicate more focus the exciting tasks could be unlocked by LLMs at the interface of time series and language such as time series reasoning [25 understanding (6.
45
37
Acknowledgements
We thank the University of Virginia's Research Computing team for maintaining the excellent higg--eeformmnce computing resources that allowed us to conduct this research. This research was supported in part by NSF AS-1991386, NSF CAREER IIS-21422 1429944 NIH RO1MH125 179, the Bill & Melinda Gates Foundation (INV-004841), the Office Naval Research #EN00014-21-1-2155 and the National Security Data & Policy Institute, Contracting Activity #2024-24070100011
10

---

References
[1] A. F. Ansari, L. Stella, C. Turkmen, X. Zhang, P. Mercado, H. Shen, O. Shchur, S. S. Ran- gapuram, S. P. Arango, S. Kapoor, J. Zschiegner, D. C. Maddix, H. Wang, M. W. Mahoney, K. Torkkola, A. G. Wilson, M. Schnke-Sch and Y. Wang. Chronos: Learning the language of time series, arXiv preprini arXiv:2403.07811. 2024.
[2] Z. Borsos, R. Marinier, D. Vincent, E. Kharitonov, O. Pietquin, M. Sharifi, D. Roblek, O. Teboul, D. Grangier. M. Tagliasacchi, al. Audiolm: language modeling approach to audio generation. IEEE/ACM Transactions on Audio, Speech, and Language Processing. 2023.
[3] J.-H. Böse, V. Flunkert, J. Gasthaus, T. Januschowski. D. Lange, D. Salinas, S. Schelter, M. Seeger, and Y. Wang. Probabilistic demand forecasting at scale. VLDB, 2017.
[4] D. Cao, F. Jia, S. O. Arik, T. Pfister, Y. Zheng, W. Ye, and Y. Liu. Tempo: Prompt-based
generative pre-trained transformer for time series forecasting. In ICLR, 2024.
[5] C. Chang. W.-C. WenC. and T.-F. Chen. Llm4ts: Two-stage fine-tuning for time-series forecasting with pre-trained llms. arXiv preprint arXiv:2308. 08469, 2023.
[6] J. Cheng and P. Chin. Sociodojo: Building lifelong analytical agents with real-world text and time series. In ICLR, 2024.
[7] W. Chow, L. Gardiner, H. T. Hallgrímsson, M. A. Xu, and S. Y. Ren. Towards time series reasoning with Ilms. arXiv preprint arXiv:2409.111 11376, 2024.
[8] Chung, C. Gulcehre, K. Cho, and Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling, arXiv preprint arXiv: 1 2014.
[9] A. Das, W. Kong, R. Sen, and Y. Zhou. A decoder-only foundation model for time-series forecasting. 2024.
[10] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04800 2018.
[11] Gao, T. Koker, O. Queen, T. Hartvigsen, T. Tsiligkaridis, and M. Zitnik. UniTS: Building a unified time series model. In NeurlPS, 2024.
[12] R. Girdhar, A. El-Nouby, 7. Liu, M. Singh, K. Alwala, A. Joulin, and I. Misra. Imagebind: One embedding space to bind them all. In CVPR, 2023.
[13] R. Godahewa, C. Bergmeir, G. I. Webb, R. J. Hyndman, and P. Montero-Mansoo Monash time series forecasting archive. In NeurIPS, 2021.
[14] N. Gruver, M. Finzi, S. Qiu, and A. G. Wilson. Large language models are zero-shot time series forecasters. In NeurIPS, 2023.
[15] M. Jin, S. Wang, L. Ma, Z. Chu, J. Y. Zhang, X. Shi, P.-Y. Chen, Y. Liang, Y.-F. Li, S. Pan, et al. Time-llm: Time series forecasting by reprogramming large language models. In ICLR, 2024.
[16] M. Jin, Y. Zhang, W. Chen, K. Zhang, Y. Liang, B. Yang. J. Wang, S. Pan, and Q Wen. Position paper: What can large language models tell about time series analysis. In ICML, 2024.
[17] M. Jörke, Sapkota, L. Warkenthien, N. Vainio, Schmiedmayer, E. Brunskill, and J. Landay. Supporting physical activity behavior change with Ilm- conversational agents. arXiv preprint 24v:24055006611, 2024.
[18] G. Lai, W.-C. Chang, Y. Yang, and H. Liu. Modeling long-and short-term temporal patterns with deep neural networks. In SIGIR, 2018
[19] S. Lee, T. Park, and K. Lee. Learning to embed time series patches independently. ICLR, 2024.
[20] Y. Li, B. Luo, Q. Wang, N. Chen, X. Liu, and B. He. A reflective llm-based agent to guide zero-shot cryptocurrency trading. In EMNLP, 2024.
[21] Z. Li, S. Qi, Y. Li, and Z. Xu. Revisiting long-ten term time series forecasting: An investigation on linear mapping. arXiv preprint arXiv:2305. 10t.:2235...02... 2023.
[22] Liu, H. Guo, T. Dai, N. Li, J. Bao, Ren, Y. Jiang, and S.-T. Xia. Calf: Aligning llms for time scries forecasting via cross-modal fine-tuning, arXiv preprint arXiv:2403.07300 2024.
11

---

[23] X. Liu, D. McDuff, G. Kovacs, I. Galatzer-Levy, J. Sunshine. J. Zhan, M.-Z. Poh, S. Liao, Di Achille, and S. Patel. Large language models are few-shot health leamers. arXiv preprint arXiv:2305 15525, 2023.
[24] Y. Liu, T. Hu, H. Zhang, H. Wu, S. Wang, L. Ma, and M. Long. iTransformer: Inverted transformers are effective for time series forecasting. In ICLR, 2024.
[25] M. A. Merrill, M. Tan, V. Gupta, T. Hartvigsen, and TT. Althoff. Language models still struggle to zero-shot reason about time series. In EMNLP, 2024.
[26] M. A. Morid, O. R. L. Sheng, and J. Dunbar. Time series prediction using deep learning methods in healthcare. ACM Trans. Manage. Inf. Syst. 14(1), 2023.
[27] Y. Nie, N. H. Nguyen, P. Sinthong, and J. Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In ICLR, 2023.
[28] Z. Pan, Y. Jiang, S. Garg, A. Schneider, Y. Nevmyvaka, and D. Song. s'ip-Ilm: Semantic space informed prompt learning with Ilm for time series forecasting. In Forty-first International Conference on Machine Learning, 2024.
[29] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
[30] K. Rasul, A. Ashok, A. R. Williams, A. Khorasani, Adamopoulos, R. Bhagwatkar, M. Bilok, H. Ghonia, N. Hassen, A. Schneider, et al. Lag-Ilama: Towards foundation models for time series forecasting.
RO-FoMo: Robustness of Few-shot and Zero-shot Learning Large Foundation Models, 2023.
[31] O. B. Sezer, M. U. Gudelek, and A. M. Ozbayoglu. Financial time series forecasting with deep learning: A systematic literature review: 2005-2001 Applied soft computing, 90, 2020.
[32] C. Sun, Y. Li, H. Li, and S. Hong. Test: Text prototype aligned embedding to activate Ilm's ability for time series. In ICLR, 2024.
[33] S. J. Talukder and G. Gkioxari. Time series modeling at scale: A universal representation across tasks and domains. 2023.
[34] H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, et al. Llama: Open and efficient foundation language models (2023). arXiv preprint 23Xh::2302.13971, 2023.
[35] A. Trindade.
DOI: EleetricityyoaddDaarraaass 12014, UCI Machine Learning Repository, 2015.
[36] S. Wang, T. Ji, L. Wang, Y. Sun, S.-C. Liu, A. Kumar, and C.-T. Lu. Stocktime: A time series specialized large language model architecture for stock price prediction. arXiv preprint DeXiv:240900828) 2024.
[37] X. Wang, M. Feng. J. Qiu, J. Gu, and J. Zhao. From news to forecast: Iterative event reasoning llm-based time series forecasting. Neural Information Processing Systems, 2024.
[38] C. Wimmer and N. Rekabsaz. Leveraging vision- laiguuee models for granular market change prediction. arXiv preprint arXiv:2301 10166, 2023.
[39] G. Woo, C. Liu, A. Kumar, C. Xiong, Savarese, and D. Sahoo. Unified training of universal time series forecasting transformers. 2024.
[40] H. Wu, J. Xu, J. Wang, and M. Long. Autoformer: Decomposition transformers with auto- correlation for long-term series forecasting. NeurlPS, 2021.
[41] H. Wu, T. Hu, Y. Liu, H. Zhou, J. Wang, and M. Long. Timesnet: Temporal 2d-variation model-
ing for general time series analysis. In International Conference Learning Representations, 2023.
[42] Z. Xu, Y. Bian, J. Zhong, X. Wen, and Q. Xu. Beyond trend and periodicity: Guiding time series forecasting with textual cues. arXiv preprini arXiv:2405. 13522, 2024.
[43] Z. Xu, A. Zeng, and Q. Xu. Fits: Modeling time series with 10k parameters. In ICLR, 2024.
[44] H. Xue and D. Salim. Promptcast: A new prompt-based learning paradigm for time series forecasting, IEEE Transactions Knowledge and Data Engineering, 2023.
12

---

[45] W. Ye, Y. Zhang, W. Yang. L. Tang, D. Cao, J. Cai, and Y. Liu. Beyond forecasting: Compositional time series reasoning for end-to-end task execution, 2024. URL https
//arxiv wrrg/abs/2445 04047
[46] A. Zeng, M. Chen, L. Zhang, and Q. Xu. Are transformers effective for time series forecasting?
In AAAI, 2023.
[47] L. Zhao and Y. Shen. Rethinking channel dependence for multivariate time series forecasting: Learning from leading indicators. In ICLR, 2024.
[48] H. Zhou, S. Zhang, Peng. Zhang. J. Li, H. Xiong, and W. Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In AAAI, 2021.
[49] T. Zhou, Z. Ma, Q. Wen, X. Wang, L. Sun, and R. Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In ICML, 2022.
[50] T. Zhou, P. Niu, L. Sun, R. Jin, et al. One fits all: Power general time series analysis by pretrained Im. In NeurIPS, 2023.
13

---

Are Language Models Actually Useful for Time Series Forecasting? (Appendix)
A Limitations
Here, we discuss the limitations of our paper.
1. We evaluate the ability of LLMs using time-series forecasting. However, to get a better picture of how LLMs work with time-series, this ability should be evaluated across other downstream tasks as well, such as time-series classification and question-ansserigg
2. Our evaluation is limited to only time-series datasets, i.e., sequences with even time-intervals. However, there also exists large fraction of data in the form of non-uniform series, such as payment records, online purchases, etc. Understanding the ability LLMs in forecasting non- non-uniform sequences is also necessary to verify claim on the usefulness of LLMs for time-series data.
B Broader Societal Impact
One of the major impacts our study will have is the influx of models that use LLMs for modeling time-series. Our results will help researchers to not simply follow the trend of using LLMs in all applications, but to check their usability in detail. Specifically, these findings will help them determine the LLM component is necessary and if the computational costs are reasonable for the specific setting. In addition to the research community, our findings on the better performance of smaller and simpler models will help develop scalable models that are easy to understand, interpret, and can be deployed cheaply real-world applications. While we agree that majority of our results are experimental and limited to selected datasets, we feel that these results will also help researchers narrow down their search space for better models in time-series forecasting, and not simply neglect the simpler models.
C License
All our contributions will be released under the MIT License.
D Additional Experimental Details
D.1 System Configuration
We train and evaluate each reference method and each architecture modification using the same device. For Time-LLM [5]. applying LlaMA-7B 34] we use NVIDIA A100 GPU with 80GB memory. For other methods [50 122.. applying GPT-2 (29), use NVIDIA RTX A6000 GPU with 48GB memory. Though an analysis of memory footprint is beyond scope of this research, we note that training the baselines in the absence of L.M be done within smaller GPUs well.
D.2 Baseline Hyper-Parameter Details
When reproducing the reference methods, we used the original repository's model structures. In the ablation study, due to the smaller model parameters, adjusted the learning rate or increased the batch size in some cases. All other training details remained identical with the reference methods. The specific training details and parar-erreetters provided in the code and run scripts of the repository Note that the training process and hyper- paramterreeeers for the simple methods can also be accessed via this link.
and
D.3 Details of Encoder Exploration and Simple Methods
To investigate the source LLM method performance, we conducted further research on encoders. We used various encoders to encode time series data, followed by linear layer to project the time
14

---

series embeddings to the forecast. The encoder structure combining patching and attention is shown in Figure The key difference between PAttn and PatchTST [27] the absence of position embedding and the Feed Forward in PatchTST 27, or, more specifically, replacing the Transformer Encoder with a simple single-layer Attention structure. could be simple yet effective method help evaluate the trade-off between cost and performance for newly proposed methods.
In addition, we propose three different neural models (i) "LTrsf":: which performs better on larger datasets, uses CALF's 's 22 encoder without cross-modal attention. This could be potential source of CALF 's performance; (ii) "D-LTrsf"; and (iii) "D-PAttm": both of them decompose the time series into three sub-seue sequences and forecast each using the above two methods respectively. and then linearly combine the results for the final forecast. Across our results in Table [8] we note that even simpler models significantly outperform LLM-based time-series models. In detail, the LLM-based models, all combined we able to appear 33 times as the best and the second-best performer. However, "PAttn" was outperforming them by appearing 34 times as the best and the second-best performer. In addition, the comparison between "PAttn", "LTrsf" and other state-of-the-ard methods shown in Table 19
E
Additional Experiments
Here we present the results of additional experiments that also highlight the ability of LLMs in modeling time-series data.
E.1 Confidence Intervals for Forecasting
Since LLMS and deep learning models in general are probabilistic in nature, their predictions can vary across different runs and different random initializations. Thus, we report the confidence intervals (CIs) for all MAE and MSE predictions made by our baseline models. We report the CIs for MAE prediction by Time-LLM, CALF, OneFitsAll in Tables 9 I1 and 13] for MSE predictions in Tables TO 12] and 14 respectively. Across all results, we note that the range of variation is quite small, and these intervals do not affect our observations. To illustrate the subtle differences more clearly, we present the visualized results in Figure Figure
Confidence Intervals for other Datasets. To evaluate the generality of the ablations in the paper,
introduce five additional datasets that have not been studied by the reference methods (50 [5] 22] The above datasets are used in many time series forecasting studies 30 9
39 [ [6). The prediction
lengths for the "Exchange Rate" "96, 192, 336, 720", as in 46 18 The prediction lengths for
the other four datasets are 30, 48, 56, and 12, respectively, following the settings in Chronos [1] As
shown in Tables 15 16 and 17 the forecasting performance the five new datasets, using the three
methods (50, 15. 22 we referenced, still demonstrates that language models are unnecessary for forecasting tasks.
E.2 Complete Results
Here we provide the results for all methods and datasets that we were unable to add the main paper.
Inference Times. All results regarding "inference time" and forecast performance are shown in Figure and Figure 8 respectively.
Randomized Parameters and Random-Shufiling Inputs. The results of the randomized parame- ters are shown in Table 20, The remaining results for shuffled input are shown in Table 21 22 23 and
24
15

---

Models
Time-I.M
w/o LLM
LLM2AMt
LLMETss
From Paper
Dataset
Window
MAE
MAE
MAE
MAE
96
0.402
(0,399,04005)
0.385
00,383,0.388)
0.400
(0.397.04402)
0.402
(0.399004055
0,392
192
0.421
(0.418,0.424)
0.412
(0.409.0.415)
0.423
(0.420.0.426)
0.426
(0.424.0.129)
0.418
336
0.438
(0.435,0.H0)
0.431
(03133.0.433)
0.475
(0.472.0.178)
0.119
(0.447,0.153)
0.427
720
0.468
(0.465.0:470)
0.452
(0.450,0.455)
0.452
(0.449.04555)
0.450
(0.477,0.483)
0.457
Electriity Traffic Westher Illeoss ETTm2 ETlml ETT12
96
0.346
(0 (0.343,0.3500
0.331
(0.33800333)
0.358
(0.354.0.342))
0.346
(0.342,03350)
0.328
192
0.391
(0. (0.387,0. 3.9)
0.374
(0. (0,30000.335
878
D. 0.384
(0.381.0.38 0.99)
0.390
(0.386.0.395)
0.375
0,414
(0.410,0.418)
0.407
(0. (010
0.406
(0.402.0.410)
0.404
(0.400.0.400 408)
720
0.434
(04429,0:457
0.24
(0:431,0/4288
0.430
(0.42660444)
0.437
(0.45400:441)
0,420
96
0.341
(0.340,0.344 3432)
0.337
(0.336,0.338)
0.333
(0.332.0.335)
0.336
(0.335,03338)
0.334
192
0.369
(0 301.68..225
0.360
(0.359,0.88 361)
0.466
10.364.0.50 367)
0.366
(0.365.0.368)
0.358
336
0.379
(0.378.0.381)
0.379
(0.378,0.381)
0.386
3.377
0.386
(0.385,0.388)
0,384
720
0.419
(0.418,0.428))
0.410
(0. 41
0.422
(0.421 (.423)
0,419
(0.418.0.421)
0.411
96
0.248
(0.247.00.2
0.246
(0.245.0.248)
0.249
(0.248.0.251)
0.249
(0.248,0.251)
0,253
192
0.301
(0.303,0.306)
0.285
(0.283,0.287))
0.293
(0.291.0.295)
0.291
(0.292,0.295)
336
0.329
(0 (0.328.0.33))
0.321
(0.319,0.323)
0.333
10.331 3335)
0.323
00.321.0.323
0.392
720
0.782
(0.380,0.385)
0.377
(0.375,0.379)
0.384
(0.382.0.387)
0.377
(0.376,0.390)
0,379
24
0.807
(0.77200.441)
0.913
(0.879.0.995) 9550
0.848
(0 (844)200884)
0.837
(1.805...870)
0.727
36
0.833
(0.804.0.861)
0.902
(0.878.0.931)
0.846
(0.813.0.882)
0.816
00.1160.877))
0.814
48
1.012
(0.9%6,1.00 1.41))
0.932
(0.9077 (1 958)
0.828
(0 847)
0,805
(0.785.0:842)
0.807
60
0.925
(0.59).0.953)
0,949
(0.920,0.979)
0.873
(0..660.905))
0.862
(0.836.0.893)
0.857
96
0.199
(0. 198.0
0.213
(0:212,0:214)
0.189
10.188 190)
0.189
088.88.
0.201
192
0.261
(0 200.0.0.2200 262)
0.252
(0.251 (..251,
0.231
(0.230.02222)
0.229
(0.229,0.230)
0.234
336
0.279
(0.278,00 2800)
0.288
(1726692899
0.272
(0.
0.273
(1.771 27744)
0.279
0.342
(0.341.0.343)
0.337
(0.336,0.338)
0.327
(0.326.0.328)
0.327
(0.326,0.329)
0.316
96
0.267
(0.267,0
2.267
0.287
(0.287.0.
0.266
(0.266.0 266)
0.269
(0.269, 0.269)
0.218
192
0.271
(0.270.8.271)
0.290
2..200,00290)
0,270
(0.270.0.271)
0,272
(0.272.0.272)
0,247
0,28%
296.0 0:
0.294
0.278
0.278)
0.269
(1.269.02200
720
0.291
(0:291 1,0: (92)
0.312
(0.312,0.312)
0.294
(0.293,0.294)
0.294
(0.294,0.294)
0.248
96
0.233
(0.233,0.235)
0.246
(0.246.0.246)
0.234
(0.233.0.23-1)
0.229
(0.229 2.200)
0.224
192
0.247
(0 22477.0247)
0.257
2.225777
0.250
(0.249.0.250)
0.242
(0.24),0.242)
0.241
336
0.267
(0266.0.267)
0.273
(0.273.0.274)
0.263
(0.263.0.264)
0.257
(0.257.0.258)
0.248
720
0.290
(0.290.0.290)
0.302
(0.302,0.302)
0.296
(0.296.0.22) 296)
0,290
(0.290.02290)
0.298
#Wius
5
13
6
8
Table 9: Confidence Intervals for MAE predictions of TiMe-LLM. The best performing model in
highlighted in Red color text. #Wins refers to the total number of times the method performed best.
Models
Time-LLM
w/o LLM
LLMZAMO
L.M2TTsf
From Puper
Dataset
Window
MSE
MSE
MSR
MSF
96
0.376
(0.371.0.382)
0.360
(0.355.0.364)
0.377
(0.37L..363)
0.382
(0.375,0.387)
0.362
192
0.407
(0.401,0:412)
0.401
(0.397.0.406)
0.408
(0.403.0.414)
0.116
(0.111.0:422)
0.398
336
0.430
(0.423.0.437)
0.431
(0.425,0.437)
0.477
(0.470.0.482)
0.441
(0.437,0.446)
0.430
720
0.457
(0.45000.463)
0.429
(0:422,0.434)
0.429
(0,423.0434)
0.479
(0.473.0.484)
0.442
96
0.286
(0.282.0.22 295)
0.221
(0.264,0.2788
0.281
(0.276,0.288
0.292
(0.284.0.297))
0.268
ETTa2 ETIml ETT12
192
0,361
(0.354.00.34 367)
0.342
(0.336.0.349)
0.355
(0.346.0.363)
0.360
(0.354,0.367)
0,329
336
0.390
(0.384.0.396)
0.379
(0.373.0.334))
0.384
(0.377.0.393)
0.379
(0.371.0.385)
720
0.105
(0.39),0:110)
0.389
(0.382.0.39)
0.395
(0.390.0.100)
0.105
00.397,0.112)
0.372
96
0.291
(0.288.0.
2935)
0.292
(0,289,0.295)
0.289
(0.286,0.29) 292)
0,292
(0.289.0.295)
0,272
192
0.341
(0.339,0.34 3444)
0.331
(0.328.0.334)
0.336
(0.334.0..40) 340)
0.341
(0.338.0.344)
336
0.359
(0:356.0 0. 362)
0.362
(0:359,0.364)
D:373
(0.368.0.36))
0.374
0..77.0.377)
0,352
720
0.433
(0.431.0.4336
0.417
(0,114,0:419)
0.129
(0.425.0.432)
0.129
(0.426,0.133)
0,383
96
0,162
(0 16..0..0.
0.161
(0.159.0.163)
0.163
(0. (0.160.0.166 165)
0.165
(0.163,0.167)
0,161
192
0.235
(0.232.0.22 239)
0.217
(0.214.0.220)
0.223
(0.221.0.227))
0.222
(0.219,0.225))
336
(,280
(0.276.0 282)
0.272
(0.268,0.275)
2.284
(0.28) (02277)
0.230
(0.266.0.77))
0.271
720
0.366
(0. (0.362,0.370)
0.359
(0.356,0.362)
0.367
(0.363.0.371)
0.355
(0.352,0.359)
0,352
1,792
(1 (1.651,1.99) 994)
2.034
(1.872.2.225)
1,923
(1.755,2.073)
1,285
Wealher Iiness
36
1.833
11.701,2.009)
1.923
(1.753,2.056)
805
(1.657.1 967)
18166
(1.707,1.961)
1,404
48
2,269
(2. 159,2.379)
1.916
(1.04.2.024)
L.16
(1.601.1.84)
1.655
(1.552,1.760)
1,523
60
2,177
(2 22.064,2.300 308)
1.953
(1. (1.844,2.076)
L,777
(1.6801.8844)
1.789
(1.637.1.916)
1.531
96
0.155
(0. (03.05.0.157
0.171
(0.168.0.173)
0.147
[0.144.0 149)
0.147
(0.345,0. 150)
0.147
192
0.223
(0.221 20.25)
0.214
(0.212,0.217)
19191
(0.189.0.195
0.191
(0.189.0. 1945
0.189
336
0.251
(0.249.0.23 25))
0.260
(0.258.0.263)
0.212
(0.210.0.245)
0.216
(0.243,0.218)
0,262
0.345
(0.343,0.348)
0.328
(0.325,0.331)
0.319
(0.317.0.522)
0,323
(0.321,0.326)
0,304
Trallc
96
0.392
(0.390.0.35 394)
0.409
(0.407.0.411)
0.395
(0.393.0.397)
0.395
(0.393,0.397)
0.362
192
0,409
(0.4477.0.411
0.177
(0.415,0.420)
D:405
(0.403.0407))
0.407
(0.405.
0.374
336
0.434
(0.45..0.436)
0.426
(0.422.0.428)
0.416
(0.414.0.488)
0.412
(0.410.0.415))
0.385
720
0.451
(0 4544))
0.461
(0.459,0.463)
0.450
(0.448.0442))
0.451
(0.448,0.453)
0,430
Elcoricity
96
0.137
(3.137.0. 138)
0.143
(0.143.0.143)
0.138
10.137.0.138)
0.133
(0.133,0. 134)
0.131
192
0.152
(0.151.0.152)
0.157
(0.157.9.158)
0.154
(0.151.0.155)
0.148
(0.148.0.149)
0,152
336
0.169
(0. 16..0.900000
0.174
(0.173.0.174)
0.169
(0.169.0.170)
0.164
(0.164.0.. 165)
0.160
720
0.200
(0. 199,0.90000 200)
0.211
(0.210.0.21 1)
0.209
(0.208 20.9)
0.201
(0.201 002202
0.192
1'count
5
11
10
7
Table 10: Confidence Intervals for MSE predictions of Time-LLM. The best performing model in
highlighted in Red color text. #Wins refers to the total number of times the method performed best.
16

---

Models
CALF
w/o LLM
LLM2Atta
LLMETTI
From Paper
Dotasel
Window
MAE
MAE
MAE
MAR
E7Th2 ETTb1
96
0.393
(0.391.0. 194)
0.391
00.390,0.392)
0.391
(0.3990.392
0.393
(0.392.03955)
0.389
192
0.426
(0.424,0-427
0.419
(0.418,0-420)
0.424
(0.423.0-25)
0.423
(0:422,0.425)
0.423
0,440
(0.439,0.44)
0.437
(0) (0.43600.433)
0.441
[0.440,0443] 442)
0.439
(0.438.0 440)
720
0.466
(0:465.0:167)
0.467
(0. (0.165.0.467)
0.167
(0.466.0:167)
0.165
(0.161,0.166))
0.467
96
0.336
(0.333,0.33 399)
0.332
(0.329,0.33 335)
0.333
(0.331.0336))
0.333
(0.331.0:336)
0.331
192
0.378
00.376.0.38 381)
0.378
(0.376,0.381)
0.379
(0.376.0.382)
0.379
(0.376.0.382)
0.380
336
0.394
(0 (0:391,0 (B.
0.394
(0.391,0 ,0, 198)
0:393
(0.390.0.366)
0.394
00,391,0 398)
0,394
720
0.428
(0.425,0.4
430)
0.428
(0. (0.426,0.430)
0.427
(0.424.0.430)
0.427
(0.425,0:429)
0.426
ET7n2 ETTal
96
0.350
(0.349,0 3.322)
0,348
(0.346.0.350
0.348
(0.347.0.349)
0.348
(0.347.0.349)
0,349
192
0.376
(0.375.0.376)
0.374
(0.374.0.375)
0.376
(0.375.0.377)
0.374
(0.373,0.375)
0.375
336
0.401
0.399
(0.398.0.400)
0.401
(0.400.0.401)
0.399
(0.398,0.400
0.399
720
0.438
(0.437,0.4399
0,442
(0.44)
0.439
(0.438.0.439)
0.439
(0.438.0.440)
0,438
0.255
00.255.0 0, 255)
0.256
(0. (0.255.0.257)
0.253
(0.252.02254)
0.252
(0.251 2.233)
0.256
192
0.300
(0 29999990 907)
0.299
(0) (9.007.0.33 800)
0.297
(0.295.0-2988
0.297
00.296.0.290
0.297
336
0.34]
(0.340.0.343)
0.310
(0.338,0.341)
0.339
(0.338.0.341)
0.339
(0.338,0.341)
0.339
720
0.395
(0. (0.39).0.3977)
0.395
(0.394,0.397)
0.395
(0.394.0.597)
0.395
(0.393,0.396)
0,393
0.788
(0.742,0:84) 841)
0.800
(0. (0.740,0.852)
0.806
(0.757.0.845)
0.792
(0.745,0:832)
Weather Illoess
36
0.837
(0 798 872)
0,802
(0:771 830
0.892
0.928
(0.895.D.977)
0,000
48
0.890
(0.842,0.99 9393)
0.888
(0.839,0.933))
0.897
(0.869.0.938)
0,859
(0.816.0:007))
0,900
0.962
(0.917,0.999
0.955
(0.917,0.997)
0.975
(0.937.1 (02))
0.861
(0.832,0.904)
0.000
96
0.207
(0.206.0 0.0.6.00...0. 207)
0.212
(0.212.0.213)
0.217
(0.216.0.217)
0.211
(0.210.02211)
0.204
192
0,251
(0 (0005000.33 252)
0.256
(0:255,0 (5.77)
0.257
2522))..288)
0.255
(12254..2255
0,250
336
0.292
(0.291,0
0.296
2977).977))
0.298
(0.297.0.299)
0.296
(0.295.0.297))
0.291
720
0.345
(0.344,00 0.377)
0.347
(0.346,0.349)
0.347
(0.345,0.3488
0.347
(0.346,0.349)
0.352
Trafic
192
96
0,276
0.274
(0.272.0.22 275)
(0.275.0.277))
0.271
0.267
(0.266.0. 0.68)
(0.270,0.273)
0.265
0.268
(0.264.0 2666)
(0.267.0.266
0.265
0.258
(0.265,0.266
(0.257, 0. (.267
0.278
0.266
0.286
(0.285,0.287 (.2)
0.278
0.275
(0.274.0.276)
0.272
(0.271,0:2775
0.281
720
0.301
(0.300,0.302)
0.297
(0.296,0.298)
0.293
(0.292.0.294)
0,291
(0.299,0,291)
0.300
Electrictt
96
0.240
(0.289.0 0 240)
0,238
(0.233,00 (0388,0.2399 239)
0,238
(0.238.0 (.2238
0,236
(0.235.0.256)
0,238
192
0.254
(0.253.0.254)
0.250
(0.219.0.251)
0
(0.250,0.252)
0.218
(0.218,0.219)
336
0.270
(0.270.0.271)
0.265
(0. (2650.2666)
0.266
(0.266.0.267)
0.264
00.263,0.264
0.267
720
0.300
(0. (0.300.0.301)
0.297
(0. (0.296,0.297)
0.302
(0.302.0.303)
0.299
(0.299.0.300)
0,303
#Wins
7
12
Table II: Confidence Intervals for MAE predictions of CALF. The best performing model in highlighted in Red color text. #Wins refers to the total number of times the method performed best.
Models
CALF
LLM
LLMZAMO
LLM2Tes
From Puper
Datasei
Window
MSE
MSE
MSE
CI
MSE
CI
96
0.370
(0.366.0.375)
0.375
(0.372,0.378)
0.368
(0.364.0.37))
0.371
(0.368,0.374)
0.369
192
0.429
(0 (0.426.0.33))
0.427
(3.433.0.433)
0:425
(2.4220.28))
0.428
(0.424.0:431)
336
0.451
(0.428.0.454)
0.577
(0:454,0.46)
0.448
(0.446.0452)
0.449
(0.446,0.451)
0,456
720
0.476
(0:474,0.478)
0.488
(0. (0.456,0.491)
0.471
(0.470,0.473)
0.475
(0.474.0.475)
0.479
ETT22
192
96
0,284
0,353
(0.347.0.333 150)
002299.0.280 288)
0.282
0.354
(0.349.0.359)
(0.277.0.288)
0.354
0.282
(0.348.0.359)
(0.278.0.286)
0.282
0.353
(0.278,0.286) 2826)
(0.349,0.358)
0.279
336
0.361
(0 0...6...00 365)
0.366
(0.362.0.371)
0.360
(0.356.0 364)
0.363
(0.359, 0. 0.367)
0.362
720
0.406
(0.403.00.40 0.49)
0.408
(0.404 00..0.4411
0.404
(0.400.0.408 408)
0.404
(0.400,0.408)
0.404
0.323
(0.319.0.35 329)
0.323
(0.330,0.325)
0.320
(0.318.0.523)
0.321
(0.319.0.324)
0.323
ET7m2 ErIml
192
0.375
(0.374.0.377)
0.374
(0.371.00.376
0.375
(0.373.0.377)
0,372
(0.370.0.374)
0.374
336
0.411
(0.409.0 103)
0.409
(0.107.0.41
0.111
(0.410.0.414)
0.108
(0.106, 0.106,0.110)
0.409
720
0.476
(0.474,0.475)
0.484
(0.482.0.485)
0.477
(0.476.0.479)
0.478
(0.476,0.480)
0.477
96
0.177
(D) 175.0.1 179)
0.178
(0:176.0.1 180)
0.176
(0.174.0.178)
0.175
(0.173.0 176)
0.178
192
0.245
(0 24. 246)
0.244
0.241
(0.289.0.245)
0.242
(0.240.0.2455
0.242
336
0,309
(0. 30..0.3611))
0.308
(0.305.0.310)
0.306
(0.304.0.309)
0.308
00.305.0.310)
0,307
0.402
(0.4000,0.44 4055)
0.401
(0.397.00.404)
0:401
(0.44)
(0.398.0.404)
24
1.460
(1 (.722)
1.544
(1.19.1.719)
1.573
(1.382.1.58)
1,450
(1.267,1.625)
Weather Ilnness
36
1,573
(1 7005)
1.437
(1.307 1.533)
1,999
(1.540,1 868)
1,780
(1.610.1.960)
48
1.784
(1 (1.630.1.90 1.937)
1.710
(1.500.18883 1.88)
1.716
(1.602,1 854)
1.639
(1.528,1882))
60
1.982
(1 (1.7%6,2 2. 128)
1.867
(1.720,2002)
2.004
(1 (37,2.186)
1.652
(1.522,1 .833)
96
0.168
(0. 00.166.0. 169)
0.176
(0.175.0.177)
0.178
(0.177.0.180)
0.173
(0.172.0.175)
0.164
192
0.216
(0.21410.2118)
0.224
(0.223.0.226)
0.225
(0.223.0.266)
0.231
(0.220.0.223)
0,214
0.271
(0.269.0. 273)
0.277
(0.275.0.279)
0.279
(0.277.0.280)
0.276
(0.274.0.278)
720
0.350
(0. 7,0. 355)
0.352
(0.350,0.355)
0.352
(0.549.0 (5.44)..354)
0.353
(0.350.0.354 3555
0.355
96
0,416
(0.413,0.419)
0.410
(0.4007,0 411.,
D.402
(0. (0.399.0.404)
0.39%
(0.393.0 3999)
0.407
Electricity Traficc
192
0.430
(0-429.0.433)
0.428
(0.12250.431)
0.419
(0.416.0-421)
0.116
(0.414.0.411))
336
0.451
(0.449.0.44 0.45)
0.443
(0.441.0.445)
0.435
(0.433.0.437)
0.431
(0.429,0.432)
0.644
720
0.478
(0.476.0.480)
0.476
(0.474,0.479)
0.167
(0.465.0.169)
0.163
(0.161.0.165)
0.477
96
0.147
(0. 146.0.
0.149
(0.14
0.148
(0.146.0.142 149)
0.145
(0.144.0. 147)
0,145
192
0.163
(0. 101,6.0.11 164)
0.162
(0.161.0.163)
0.161
(0.160.0.16]
0.159
(0.158.0. 160)
0,161
336
0.17%
(0. 178,0.177
0.175
(0.175.0.176)
0.175
(0.174.0.776 176)
0.172
(0.171.0.172 1755
0.175
720
0.215
(0.214.0.215)
0.212
(0.211.0.213)
0.217
(0.217.0.218)
0.213
(0.213.0.214)
0,222
AWins
9
Table 12: Confidence Intervals for MSE predictions of CALF. The best performing model in highlighted in Red color text. #Wins refers to the total number of times the method performed best. Note that' '-' means the dataset has not been included in the original paper.
17

---

Models
OneFissAll
w/o LLM
L.LMAAtt
LLM2TT
From Paper
Dataset
Window
MAE
CI
MAE
CI
MAE
MAEE CI
MAE
CI
96
0.389
(0.387.0.391)
0.390
(0.387.0.392)
0,406
(0:404,0.400)
0.412
(0.408,0,414)
0.397
192
0.413
(0.411,0.416)
0.416
(0.413,0.118)
0,441
(0.438,0.443)
0.455
(0.452,0.158)
0.418
336
0.431
(0.428,00413))
0.430
(0.438,0.433)
0.461
(0.458,0-161)
0.460
(0.115
0.433
720
0.449
(0.446,0.451)
0.454
(0.451,0.457)
0.501
(0.498,0.504)
0.570
(0.566,0.575)
0.456
ETTm2 ETTml ETT12
96
0.335
(0.333,0.356)
0.357
(0.335,0
0.511
(0. (0.349,0.355
0.348
(0.346,0.350)
0.342
192
0.380
(0.378.0.38
0.382
(0.380.0.883)
0,395
(0.395.0.396)
0.391
(0.389,0.392)
0.389
336
(0.403,0,,000 (006))
0,410
(1,107
(04414,0118)
0,414
(0,413,0,416)
0.407
720
0.436
(0.435,0.433 0.448)
0.430
(0.428,0.432)
0,448
(0. (0.446,0.450)
0.439
00.437,0.441))
0,441
96
0.340
(0.339.0,34) 3400)
0.342
(0.341,0.343)
0.557
(0. 00.35600.338
0.339
00.338,0.35 (400)
0.346
192
(0.367,03368)
(0.362,0268))
0,266
(0.2660,267)
(0.369,0.3700
0.372
336
0.386
(8.386
0.381
10,381,0,3882
0,389
0,300
0.387
(0.386,0.38 (.3377
0,394
720
0.416
(0.415,0.446)
0.413
(0.412.0.413)
0.427
00.426,0428)
0.421
(0.420,0.422)
0.421
96
1.249
(0.248,0.250
0.248
(0.247,0.248)
0.251
00.251 (2222
0.250
(0.249,00250)
0.262
192
0.291
(0.291,0,22 0.22))
0.286
(0.286.0.287)
0.291
(0
000.200,22222
0.288
(0.287.0 (829)
0,301
336
0.327
(0.326.0.328)
0.523
(0.322.0.324)
0.527
00.3 00.326.06227 (27)
0.523
(0.322,0.324)
0,341
720
0,376
(0.375.0.377)
0,380
(0.379.0338))
0,582
(0.381.0.883)
0,382
(0.381,0.383)
0:401
24
0.823
(0. (0.80))0.841)
0.930
(.945
0,807
00.788.08 (044
0.846
(0.830,0.805)
0.881
Illos
36
0.854
(0.840.0.868)
0.913
(0.901 (.922)
0.616
(0.99,0.830)
0.848
(0.833,08864)
0.892
48
0,855
(0.8400 866)
0.911
(0.897.0.922) 90.92)
0.846
00.8 860)
0.848
(0.0
0.884
(8.867.0.88))
0.942
(0.932.09957)
0.850
(0.836.08863)
0.861
(0.850,9887))
0.957
Trafiie Weather
96
0.188
(0. (8.18)
0.212
(0.212,0.113) 3)
0.193
(0. 10.192..199)
0.188
(0.187 0.. 188)
0,212
192
0.250
(0.2300.231))
0.251
(0:251 (0.252))
0.251
00. 230.0.232
0.233
(0.252.0.234 2334)
0,248
336
(0.272.0.273)
(0.288,0) 2900)
(..273..2774)
(0.274.02775)
0.286
720
0.328
(0.328.0.329)
0.339
(..30)
0.528
0.328
(0.328,0.329)
0.337
96
0.264
0.264
(0.264.0.266 (.26
0.257
00.257,0.257)
0.252
(0.252,0.252)
0,282
192
(0, (0,2688.1 2668)
(0221
(0. (0.26000.26))
(0.245,0.240 2426)
0.290
336
0.273
(0.273.0.273)
0.271
(0.270,0.231)
0.264
(0.264.0.265)
0.255
(0.254,0.255)
0.294
720
0.291
(0.291,0.29) (0.291)
0.289
(0.289.0.23 (.299)
0.284
(0.284.0.2844)
0.274
(0.274,0.274)
0,312
Electriciy
96
0.239
(0.238,0.239)
0.230
(0.229,0.230)
0.224
(0.224.0.224)
0.218
(0.218,0.218)
0.238
192
0.253
(0.252.0.255)
0.242
(0.242.0.242)
0.238
(0.238,0.238)
0.233
(0:233,0.2355
0.251
336
0.266
(0.266.0.267)
0.258
(0.258,0.2588)
0.254
(0. 00.254.0.2244
0.250
(0.250,0.250)
0.266
720
0.293
(0.29330.294))
0.290
(0.290.0.200)
0.285
(0. (0.285,0.2855)
0.283
(0. (0.283,0.2883)
0.297
AWins
Table 13: Confidence Intervals for MAE predictions of OneFitsAll. The best performing model in
highlighted in Red color text. #Wins refers to the total number of times the method performed best.
Models
OneHitsAll
w/o LLM
LLMZAMO
LLM2Trsf
From Puper
Dataset
Windew
MSE
MSE
MSE
MSE
96
0.370
(0.365.0.55 375)
0.371
(0.366.03776)
0.403
(0.397 (.49)
0.397
(0:391.0:4013)
0.376
192
0.412
(0.406.0.417)
0.416
(0:410,0.420)
0.454
(0.448.0.46))
0.482
(0.476,0.489)
0.416
0.448
(04443.0.454)
0.441
(0.434,0.446))
0.483
(9.475.0.490)
0,480
(0.475,0.4487)
0.442
720
0.441
(0:436.0.447)
0.422
(0.437,0.448)
0.522
(0.518,0.527)
0.743
(0.752,07752)
0.477
Illoess ET1m2 ETTml ETT12
96
0,280
(0.278.0. 283)
0.281
(0. (0.282.0.287)
0.304
(0.301.0.307)
0,298
(0.295.0.
0.285
192
0,348
(0.346.0.
0.355
(0.352.0.357)
0.370
(0.367.0.375)
0.363
(0.360.0.366
0.354
336
0.350
(0.377.0.385)
0.388
(0.384.0.390)
0.399
(0.396.0.401)
0.392
(0.389,0.395)
720
0.406
(0.443.00.44
4099
0.400
(0. (0.399,0.4003)
0.431
(0.428.0.4) 0.33)
0.419
(0.416,0.422))
0.406
96
0,300
(0 (0.298.0
0..30...000 301)
0.301
(0.300.0.303)
0.299
(0.297.0 (.2.0.0300)
0.301
(0.303,0.33 3.300,0.300)
0,292
192
0,343
(0
0.338
(0.337,0.339)
0.349
(0.34).0.351)
0.356
(0.355.0.358)
336
0.376
(0.374.0.377)
0.369
(0.368.0.370)
0.383
(0.382.0385)
0.379
(0.378.0.381)
0.366
720
0.43]
(0.430,0.433)
0.425
(0. (0.12240)427)
0.447
(0.4 (0.416.0.149)
0.438
(0.437.0.440)
0.417
96
0.163
(0. 10..6..0.00 164)
0.163
(0.16).0.164)
0.165
(0. 00.164.0.166 166)
0.164
(0.163.0.165)
0.173
192
0.222
(0 200.0.00220 223)
0.221
(02219.0.223
0.223
(0.221 (2-224)
0.219
(0.218.0.220
0.229
336
0.273
(0.272.0.275)
0.275
(0.273,0.226)
0.275
(0.273.0226)
0.272
(0.270.0.275)
0.286
720
0.357
(0.355,0.359)
0.364
(0.363.0.366)
0.362
(0. (0.360.0.34))
0.362
(0 00.360,0.363)
0,378
1,869
(1.780.1 950)
2.119
(2.022.2.215)
(1.711.1 887)
1,929
(1.868.2022)
2,063
36
1.853
1.955
1.929
(1.877.1997))
L.727
(1.660.) 800)
1.801
(1.741,1.859)
1.868
48
1,886
(1.828,1 1.942)
1.883
(1.827,19322)
1.004
(1.74411 866)
1,807
(1.749,1. (.99
1.790
(1 117821997777
1.911
(1.86111955 1.93)
1,724
(1.6721.776)
1.784
(1.736,18841)
Weather
96
0.148
(0. 10.147,00 0, 150)
0.173
(0.172.0.175)
0.150
(0.149.0.152)
0.149
(0.348.0. 150))
0.162
192
0.192
(0.191.0..0 194)
0.216
(0.215.0.218))
0.192
(0.19L.0.194)
0.196
(0.194,0.197)
0.204
336
0,246
(0 204244,00 0 247)
0.263
(0.261.0.264)
0.244
(0.243.0.246)
0.247
(0.246.0.244)
0.254
720
0.320
(0.318.0.321)
0.330
(0.329.0.332)
0.318
(0.316.0.319)
0.322
(0.321,0.324)
0.326
Traliic
192
96
0.412
0.396
(0. 300,33,,.200 398)
(0.410.0.414)
0.430
01422
(0.41190.442))
(3.123.0.432)
D. 0.393
0.106
(0.401.0.40))
10.391.0.395
0.395
0.391
(0.389. 0.393)
(0.393,0.397)
0.407
0.388
0.421
0.437
(0.433.0.433)
0.113
(0.412.0.416)
0.106
(0.105.0.1105)
720
0.455
(0.453.0.457)
0.470
(0. (0.466.00472)
0.451
(0.449.0.453)
0.444
(0.442.0.446)
0,450
Elecrriity
96
0.141
(0. 1011-4,,0 142)
0.137
(0.136,0.137)
D.133
(0.133.0.134)
0.128
(0.128,0) 129))
0,139
192
0.158
(0.157.0.1 155)
0.151
(0.151,0.152)
0.149
(5.149.0.100)
0.145
(0.145.0. 146)
0,153
336
0.172
(0. 172.0. 172)
0.167
(0.167.001668)
0.165
(0.164.0.165 165)
0.160
(0. 10.160..0.00 161)
0.169
720
0.207
(0 207.0.0.0000 208)
0.206
(0.205.0.2006
0.201
(0.201 (0.202)
0.196
(0.195.0. 1966
0.206
#Wins
10
5
7
10
Table 14: Confidence Intervals for MSE predictions of OneFitsAll. The best performing model in
highlighted in Red color text. #Wins refers to the total number of times the method performed best.
18

---

Models
Time-
wio LLM
LLMZAtI
LLMETed
Datanct
Window
MAF
MAR
MAR
MAF
96
0.251
(0.246.02255)
0.209
(0.206,0212)
0.213
(0:20.0:17))
0.230
(0:257.0.342)
Exchange Rate
536
192
0,451
0.348
(0.34000,44)
(0.44600445))
0.423
0.305
(0:417,0.428)
10.301,0.00
0.471
0.330
(0.325.0.354)
(0-4650047))
0.43
0.331
(0.325.0.336)
(0:447.0.458)
0.771
(0.761.07822)
0.719
(0.708,0727)
0.762
(0.752,0771))
0.762
(0.753.0:77)
Covid Deaths
30
0.080
10.018.0.15)
0.080
(0.015.01175)
0.059
(0.00,0.115)
0.055
(0.008.0113))
Tasi (3U Min)
48
0.275
(1.27000220))
0.256
(0.280,0292)
0.269
(0.263.0.255)
0.256
(0.251.0.261)
MNS (Dully)
(0.35%00.500)
0.425
(0.360,0.99))
0.411
(0.341,0.484)
(0.327.0.455)
FRED MD
6.0e
(5.2c 04.7.9
2.0c
(1.4c 04.2.7
4.7c 03
(4.5c 03,4.8
9.0e 04
16.7c 04.1.2
WWins
Models
Time LLM
w/o LLM
LLMZAtt
LLM2Trd
Dataset
Window
MSB
MSE
MSE
MSE
C1
0.123
(0.12000.127)
(0.057.00993
(8.057.0.03)
(0.107.0:113)
Exchange Raie
536
192
0.577
0.224
(1.36910,367)
(0.217.0:230)
0.185
0,541
10.150,0 1909
(0.332,03348)
0.211
0L407
(0.20600:11))
(0.398.0.415)
0.211
0.384
(0.37660)399
(0:205.0:216)
720
1.018
(0.997,1.141)
0,922
(0.89600943)
1.022
(1.00,11043)
0.9%6
(0.975,1.025)
Covid Deatts
30
0.194
(..00.0.667)
0,199
(0.004,0.462)
0.087
(0.009,01199)
0.082
(0.002.0.88))
(0.154.0.16)
0,177
0.157
(0.149,01164)
(0.134.0.148)
NNS (Uaily)
56
0.404
(0,27300548))
0,579
(0,271,05525)
0.365
(0.253,0.599)
0.347
(0:234.0.44))
FRED MD
12
1.4e 06
(7.5c 02.2-00.....00. 06)
2.8e-07
(8.02-08,5.6 07)
2:5e005
(2.4e
2.6e:00 06
(L.Te.06...76.00
WWVins
Table 15: Confidence Intervals for MAE and MSE predictions of TiMe-LLM. The best performing model in highlighted in Red color text. #Wins refers to the total number of times the method performed best. The ablation results in the table above are from datasets that have not been studied by the reference methods [50 13 22
Models
CALF
w/o LLM
LLMEARSS
LLMETes
Datesct
Winlow
MAE
MAE
MAE
MAE
96
0.203
(0.2010.200)
0.207
(0,204,0211)
0.296
(0.203.1129)
0.207
(0.20401:210)
Exchange Rate
192
0.306
(0.3000.312 312)
0.305
(0.300,0311)
0.305
(0 (0.20..0 309)
0.305
(0.195.0 909)
336
0.427
(0.42200.435)
0.466
(0.419.0445)
0.427
(0.41.00435)
0415
(0.405.11444)
720
0.732
(0.7210.745)
0.897
(0.657,07710)
0.751
(0.716,0750)
0.708
(0.695.0.725)
Covid Deaths
30
0.084
174)
0.066
(0.012.0114) 144)
0.131
(0.027.0.24))
0.066
(0.014...11)
Tasi (30 Min)
48
0.258
(0.254.0263)
0,264
(0.299,0268)
0.267
(0.262.0271)
0.267
(0.263.0.22)
MNS (Dally)
(0.34800471)
(0.325,0452))
(0.36700304)
(0.357.0.495)
FRED MD
12
1.3e 03
(1.2e 08.1.4e
1.2e 05
(1.0e 00.1
6
(1 00,1
1.7e 00
(1.6e 03.1
WWins
Midels
L.MMAtt
LLM2Tra
Datasct
Window
MSE
MSF
MSF
MSE
C1
Exchange Rate
336
192
0.350
0.186
(0.139.0.361)
180,0. 1989
0,183
0,345
10.177.0
(0.335,03360)
188)
0.345
0.182
(0.332.0.355)
(0.177.0.183
0.324
0.181
(0.310.0.337)
(0.175.0 186)
720
0:905
(0.906.09965)
0.854
(0.833,0877)
0.943
(0911,0975)
0.878
(0.836.0.500)
Covid Deaths
30
0.163
(0.0050.333)
0.115
(0.003.0227))
0.431
(001001.045)
0.106
(0.008.0.00))
(0.134..112)
0.147
(0.142.0.15))
NN5 (Dally)
0.362
(0.275.0.522)
0,336
(0,236.0445))
0.405
(0.277.0.615)
0.399
(0.272.0:185)
FRED MD
12
2.9e 06
(2.0e 06.4.5 06)
2.76 06
(2.0e 06.3.3 06)
4.9c 06
(3.4e 06,6.8e 06)
4.5e 06
(3.7e 06.5.4e (6)
#Wins
Table 16: Confidence Intervals for MAE and MSE predictions of CALF. The best performing model in highlighted in Red color text. #Wins refers to the total number of times the method performed best. The ablation results in the table above are from datasets that have not been studied by the reference methods (50 15 22
19

---

Models
OnelitsAll
w/o LLM
LLMZAtt
LLMITed
Dataset
Window
MAE
C1
MAE
CI
MAB
MAE
C1
96
0.218
(0.21500.21))
0.204
(0.202.220))
0.215
(0223300215)
0.224
(0:22.00222))
Exchange Rate
536
192
0.461
0.307
(0:455,0.467)
(0.205.03311)
0.422
0,38
(0:44600455)
(0.30.0.312)
0.499
0.339
(0.3540.364)
(0.46300447)
0.344
0.343
(0.34000.47))
(0.450014400)
0,767
(0.756.0.777)
0,727
(0.716,0736)
0.733
(8.722,0747))
0.781
(0.771.0790)
Corid Deatts
30
0,087
(0.019.0112))
0,050
(0,004,01000
0,058
(0.01900.11))
0.078
(0.01500175)
Taxi (30 Min)
48
0.260
(0.25600267)
0,265
(0,259,0.21)
0.264
(0.25800272)
0.263
(0:256.0.66)
NNS (Daily)
0:438
(0.3660557))
0,422
(0.352.049))
0.423
(0.363,0-660)
(0:354.0.87))
FRED-MD
12
7o-04
(5 (5.6e-06777e-04)
2.5e-04
(1 3.-003333-000
6.1-44
(4 (4-6-044711e-44)
1.2-00
0.20-0%1
#Wins
Models
OneliitsAll
w/o LLM
LLMZAtta
LLM2Trd
Datsset
Window
MSE
MSE
MSE
MSE
0:096
(0.093.00000)
0.086
(0.854,00899)
0U892
(0.090,0005)
0.103
(0.10500110))
Exchange Rate
536
0.402
0.182
(1..99...414)
10.177.0::0 186)
0.189
0.392
(0.184,0 194)
(0.352,04400) 4000)
0.233
0.462
(0.444.0-481)
(0:234.0:390)
0.350
0.228
(0:22100:335)
(0.351.01.368)
720
1.085
(1 (1,028,1.004))
0,932
(0,909,0955)
0.985
(0.58.
1.113
(1 (1.9901.146)
Covid Deatts
30
0:075
(0.004.0.11)
0.073
(0.002,01155)
0.103
(0.00300.665)
0.162
(0:009.0.33)
(30 Min)
0,148
(0.139..115)
0.150
(0.141..119)
(0.143,0.159)
(0.141.0.158)
NNS (Dally)
56
0,438
(0,309.05555)
0,585
(0,252.0558)
0.390
(022620)342)
0.385
(0.254.0.515)
FRED MD
12
1.2e 06
(7.4g 07.1 06)
3.8c-07 07
(1 0e-07,7.6e-07
1. 1.5e-06
(8.4e-00,222e-00)
2.4e-05
(2. 1...3-033300
#Wins
Table 17: Confidence Intervals for MAE and MSE predictions of OneFitsAll. The best performing model in highlighted in Red color text. #Wins refers to the total number of times the method performed best.
ET OneFitsAll
ETT12 2TFF
ETTH2 Time LLM
w/ LLM
HH HH
w/ LLM LLM LLM2AG0 LLM2TTsf
w/ LLM
LLM
W:O LLM LLM2Att LLM2Trsf
LLM2Atta LLM2Trsf
0.390 MAR
0.400
0.380
0.383
0.380
0.390
MAR
MAE
ETTml Saaal-OOeFFiAAA
ETTml CALF #TTmI-CALF
ETTml TimeLLM
w/ LLM
w/ LLM w/O LLM LLM2AttD LIM2T
w/ LLM W:O LLM LLM2AHD LLM2T
w/o LLM
LLM2Atta
LLM2T
0.375
0.380
0.392
0.400
0.372
0.376
MAR
MAR
Illness- Atlllt
Illness- CALF
Ilness- TimeLMM
w/ LLM LLM
H+
w/ LLM w/O LLM LLM2AM
w/ LLM WEO LLM LLM2AHD LLM2T
LLM2Atta LLM2Trsf
H+
LLM2TTsf
0.840
0.900
0.850
0.900
0.800
0.880
MAE
MAB
MAE
Weather- Fitsll
Weather- VeLF
Weather TimeLLM
w/ LLM
w/ LLM w/o LLM
W/ LLM W:O LLM LLM2Athn LLM2Trsf
w/o LLM
H
LLM2Attn LLM2TTsf
L.L LLM2TTsf
IN
0.255
0.270
0.272
0.276 MAR
0.255
0.270
MAE
MAE
Traffic Fislll
Traffic CALF
Traffie TimeLLM
w/ LLM
w/ LLM w/o LLM LLM2Att
w/ LLM W:O LLM
w/o LLM
LLM2Atta LLM2TisE
LLM2AHD LLM2T
LLM2Tysf
MN
0.260
0,280
0.272
0.280
0.270
0.285
MAR
MAR
MAR
Figure 5: Ablation studies indicate that when different methods remove the LLM ("w/o LLM"") re- place it with a single-layer attention ("LLM2Attn") or Transformer ("LLM2Trss"', the performance on time series forecasting tasks with MAE metric does not decline and even improves, compared with original methods, such as "GPT-2" or "LLaMA". The vertical dashed line in the figures represents the results from the original paper. Above figures are from 'ETT22', 'ETTml', 'llness', 'Weather', and 'Traffic' datasets.
20

---

ETThi-OnePitsAII
ETTHI-CALE
ETTh Time
LLM
LLM O/O LLM LLM2Attn Lf
LLM W LLM LLM2Ath Lf
W LLM
LLM2Attn
14
LLM2TTsf
0.420
0.480
0.424
0.432
0.400
0.420
ETT12 Oit
ETTH2 CALF
ETT12 TimeLLM
w/ LLM
w/ LLM w/o LLM LLM2Astn LLM2TTsf
w/ LLM WAO LLM LLM2Attn LLM2TTsf
w/o LLM
LLM2Attn
LLM2TTsf
0.350
0.360
0.370
0.344
0.348 MSE
0.345
0.360
MSE
MIS
Fill
ETTm1-CALF
MITmm1-Timell
LLM w/o L.M
LLM w/o L.M LLM2AttD LLM2TTsf
HH
LLM W/O LLM LLM2Attn LLM2Trsf
LLM2Atta LLM2Ttsf
HH
0.360
0.370
0.400
0,410
0.340
0.360
ETTm2--meFiteer
6TTm2-CALF
ET TimmeeLTTLLL.. LiM
w/ LLM
MH
w/ LLM LLM LLM2AM LLM2Trsf
w/ LLM WEO LLM LIM2AMD LLM2Teaf
LLM
LLM2Attn
LLM2Tisf
0.256
0.264
0.279
0.282
0,252
0.258
MSE
XISE
XIE
SCCL-OneFis...e
ECL-CALF
ECL- LLM
LLM
LLM w LLM LLM2Attn LLM2TTsf
LLM W:O LLM LLM2Attn LLM2Trsf
w/o LLM LLM2Attu
LLM2TTsf
0.160
0.168
0.171
0.174
0.160
0.168
Miness FieFitsAl
Mlnessss CALF
Mhess: T.meL.LM
w/ LLM LLM
w/ LLM w/o L.M LLM2Attn LLM2TTsE
w/ LLM W/O LTM LLM2Attn LLM2TTaf
LLM2Atta LLM2TTsf
1.800
2,000
1.600
1.800
1.600
2,000
MSE
MIE
XISE
Weather- Weather-OneFisshl Ait
Weather-CALF
Weather- Weather-TimeLLAA LLM
w/ LLM
LLM w LLM LLM2Atto LLM2TTsf
LLM w/O LLM LLM2Attn LLM2T
w/o LLM
LLM2Atta
LLM2Tisf
0.225
0.240
0.252
0.258
0.225
0.240
Traffic-neFitshl
Traffic-AALF
Traffie-Timel TimeLLM
w/ LLM LLM LLM2Attn LLM2Tisf
w/ LLM LLM LLM2Attn LLM2Trsf
w/ LLM LLM LLM2Attn LLM2T
HM
H+
0.420 MSE
0.440
0.430
0.440
0.400
0.425
MSE
XISE
Figure 6: Ablation studies indicate that when different methods remove the LIM ("w/o LLM"") re- place it with single-layer attention ("LLM2Attn") or Transformer ("LLM2Trss"), the performance on time series forecasting tasks with MSE metric does not decline and even improves, compared with original methods, such as "GPT-2" or "LLaMA", The vertical dashed line in the figures represents the results from the original paper.
21

---

Red: Best performance Blue Second Best.
MST
1,185
1,130
1,099
1.083
2.777
2.987
3,049
3.073
1.056
1,075
1.0%6
1.089
2,730
2,943
3,036
3.109
7.099
7.231
7.205
7,096
0.576
0.605
0.625
0.635
1.765
1.628
1.574
1.543
1.201
1.064
1.031
Seasonal
MAF.
0.799
0,288
0.785
0.789
1.294
1.313
1.324
0,752
0.769
0,778
0.782
1,208
1,285
1.318
1.339
1.897
1.882
1
0.550
0.577
0.594
0.605
0,884
0,844
0.827
0.816
0.891
0,861
0.847
0,838
MSTE
0.753
0.775
0.762
0.720
0.444
0.476
0.469
0.738
0.745
0.752
0,764
0.512
0.335
0.366
0.433
4.524
4,819
5.058
0.297
0,316
0.343
0.384
1.441
1.455
1.469
1.488
0.944
0.966
0.998
Mean?
0
MAR
0.525
0.544
0.548
0.549
0.378
0.422
0.450
0-456
0.512
0.518
0.524
0.53
0.345
0.358
0.377
0.488
1.410
1.444
1.492
0.112
0.323
0.341
0.369
0.771
0.774
0.776
0.780
0.738
0.747
0.762
MSE
0.375
0.405
04439
0.472
0.289
0.385
0.448
0.005
0.299
0.335
0.369
0.455
0.16 161
0.224
0.28
0.397
1.963
2,130
2.368
0.176
0.220
0.265
0.323
0.410
0.423
0.436
0.466
0.340
0. 169
0.203
DI Lincor
MAR
0.399
01416
0.443
0.490
0.418
014665
0.551
0.343
0.365
0.33%
0.421
0.260
0.303
0.341
0-421
0.963
1.024
1.0%
0.257
0.282
0.319
0.362
0.282
0.287
0.29
0.315
0.237
0.249
0.267
0.301
MSE
0.376
0:430
0.440
0.669
0.347
0.341
0
0.300
0.343
0.370
0.431
0.165
0.220
0.265
0.330
1.685
1.739
1.775
1.695
0.352
0.196
0:247
0.316
0.394
0.404
0,412
0.449
0.134
0.166
0.204
D-PAID
Table Comparision between simple methods and original results from reference paper 22 50
MAR
0.387
0.155
U-119
0.455
0,373
0.176
0429
0.346
0.379
0.388
0.418
0,252
0.291
0.319
0.369
0,8
0,853
0.874
0,853
0.195
0.236
0.277
0,350
0,262
0,266
0.269
0.289
0.227
0,257
0.289
MSF
0,583
0
0.447
0.474
0.277
0.345
0,338
0
0.301
0:344
0.300
0.426
0.164
0.224
0.266
0:345
1.767
1.658
0.151
0.193
0.244
0.394
0.414
0.413
0.448
0.134
0.149
0.166
0.204
PAtto
34
MAR
0.387
0.416
0-419
0.457
0.373
0.375
0.255
0.342
0.366
0.383
0.413
0.249
0.292
0.319
0.370
0.794
0,853
0.829
0.191
0,232
0.273
0.323
0.262
0.265
0.270
0.285
0.226
0.257
0.288
MSE
0.571
0428
0.49
0,485
0.551
0.346
0406
0.300
0.527
0.556
0.421
0.172
0.226
0.269
0.44
2,300
2.534
2,490
0.161
0.205
0.253
0.322
0.357
0.377
0,388
0.428
0.130
0.166
0.193
L.7os
16
MAR
0.590
0.418
0.426
0.465
0.376
0.579
0.426
0.351
0.366
0.386
0.411
0.258
0.299
0.323
0.372
0.826
0.927
0,992
0,992
0.204
0.246
0.283
0.332
0.234
0.243
0.250
0.273
0.220
0..39
0.254
0.278
MSE
0.379
0.436
0-447
0.470
0.353
0.350
0.11
0.300
0.335
0.359
0,418
0.168
0,225
0,368
0.344
1.738
1.897
2.049
0.160
0,202
0.320
0.383
0.399
0.413
0.455
0.132
0.154
0.171
0.211
D-L.Ts
MAR
0.392
0.421
0-427
0.579
0,382
0-431
0.347
0.367
0.383
0.414
0.256
0.284
0.522
0.572
0.814
0.895
0.951
0.979
0.203
0,242
0
0.333
0.264
0.275
0.284
0.309
0.226
0.247
0.265
0.299
Time- LLM [3]
MSR
0.362
0. 398
0:430
0.329
0.368
0.372
0.272
0.310
0.352
0.383
16
0.219
0.271
0.352
1.404
1.523
1.531
0.147
0. 18189
0.262
0.35 10394
0.362
0.374
0.385
0.430
0.131
0.152
0. 192
0.
MAR
0.392
0448
0-427
0.457
0.375
0.409
0.40
0.334
0.411
0.253
0.291
0.30 0..922
0.379
727
807
0.857
0.201
0.234
2.2
0.316
242
0:247
0.271
0.288
0.224
248
002998
0
0.55
OneFittll 100
MSE
0.376
0.416
0.422
0.354
0.373
0.406
0.292
0,332
0,366
0.417
0.173
0.229
0.286
0.378
1,808
1.790
1.979
0.162
0,204
0.254
0.326
0.388
0.407
0.412
0.450
0.139
0.153
0. 109
0,206
11
MAR
0.397
04418
0.433
0. 18389
0.407
0.441
0.346
372
0.421
0.262
0.301
0.341
0.401
0.881
888
0.957
0.212
0.248
286
090
0.294
0.312
0.238
0.251
26266
297
0.
MSR
0.369
0.427
0-456
0.479
0.279
0.353
0.362
0.404
0.323
0.374
0.409
0.477
0.178
0.242
0.307
0:307
0.164
0.214
0,299
0.355
0.447
0.430
0.444
0.477
0.145
0.161
0. 175
0.222
CALF 22
MAF.
0.389
0.423
0.436
0,380
18380
0.394
0-426
0.349
0.37 0.325
1.999
0.438
0.256
0.297
0.339
1.399
0.204
0.250
0.291
0.352
0.268
0.278
0.281
0.300
0.238
0.267
303
96
192
336
720
192
355
720
96
192
356
720
96
192
356
720
36
48
60
96
192
336
720
1d god Wins
96
192
356
720
96
192
336
720
1 and gnd Wins
Methods
Matric
BTTh
ETTh2
ETTm]
ETTm2
Illoess
Weather
Triffic
Electicity
22

---

Table 19: Comparison between simple ablations. LLM LLM--ased based methods, and non LLM methods. LLM-basee methods slightly outperform non- LLM methods. The non-LLM results are from the LLM- Time and iTTrasffrme papers. which are computed on the same splits. As expected, the main findings in our paper are unchanged: The LLM based methods are slightly better, but our ablations indicate this isn't ' due to the LLM. Red: Best performance, Blue: Second Best.
FEDGormrr (99
MNE
0.379
0.126
01.45
0.543
0:203
0.266
0.325
0.42
0.376
0.420
0.506
0.358
0.496
0-463
0.193
0.201
0.2
0.246
0.5N7
0.004
0.621
0.636
0.271
0.339
0.403
MAE
0:419
0.441
0.459
0.490
0.287
0.728
0.366
0.415
0.419
0-448
0.507
0.397
0.439
0.487
0-474
0.308
0.315
0.329
0.355
0.366
0.773
0,383
0.392
0.336
0.380
01478
41
MSE
0.338
0.574
01410
0.478
0.187
0.249
0.321
0.384
0436
0.521
0.540
0.402
0.452
0-462
0.168
0.104
0.19
0.220
0.593
0.11
0.629
0.640
0.219
0.280
Timesea
MAE
0,375
0.357
0,41
0.450
0.267
0.309
0.351
0.402
0.429
0.500
0.574
0.422
04668
0.272
0.289
0.300
0.330
0.521
0.336
0.336
0.350
0.261
0.506
0.359
Other State of the art Methods
iTrandormer 24
MSB
0.334
0,377
01:426
0.491
0,1 800
0.250
0,311
0.412
0
0:441
0.457
0.508
0.297
0,380
0.428
0.427
0.1 14148
0,162
0.177
0.225
0.395
0.417
0.433
0.462
0.221
0,223
0.3%
MAB
0.368
0.391
0.420
0459
0.364
0.309
0.348
0.405
0-436
0-491
0.349
0,100
0.32
0.445
0.340
0.53
0.269
0.17
0.268
0.276
0.283
0.302
0.254
0.296
0.447
22]
MSB
0.200
0.332
0.366
0.16
0.165
0.220
0.274
0.370
0413
0:447
0.234
0.329
0.379
0.129
0.157
0.163
0.197
0.360
0.379
0,392
0.432
11194
0.245
0.14
PatchTST
MAB
0.144
0.369
0.392
0.420
0.255
0.292
0.329
0.399
0.421
0:466
0.336
0,380
0.422
0.223
0,240
0:259
0.290
0.219
0:256
0.264
0.286
0.241
0.282
0.334
RLinerr 21
MNE
0.355
0..91
11.424
0.487
0.182
0.245
0.307
0.407
0.386
0-337
0.481
0.208
0.374
0.415
0-120
0.201
0.201
0:215
0,257
0.6-9
0.601
0.609
0.647
0.192
0.240
0.292
MAB
0.576
0.392
01415
0.450
0.265
0.3044
0.342
0.395
0-244
0-770
0.338
0.426
0.410
0.261
0.283
0.
0.331
0.589
1.366
0.369
0L387
0.23
0.271
0.307
0.355
DXLinerr (6
MSG
(12999
0.335
01309
0-225
0.167
0.224
0.381
0.375
0-405
0.43
0-F2
0
0,383
04488
0.605
0.340
0155
0,769
0,303
0410
0.423
0.636
0.466
0
0.265
0.52
MAE
00.443
0.365
0.306
0.421
0.260
0,303
0.343
0.41
0.399
0-16
0.443
0,49
0.553
0,118
0.565
0,551
0.237
0.249
0,267
0,301
0.282
0,287
0.296
0,315
0,282
0.319
0,362
Time LLM 0.
MSB
0.272
0.310
0.352
0,383
0.161
0.219
0.371
0.352
0.362
0.398
0.430
0:442
0.268
0.329
0.368
0.372
0.131
0,152
0.160
0.99
0.362
0.374
0.385
0.430
0.189
0.262
0.304
25
MAE
0.314
0.358
003304
0441
0.253
0.293
0,392
0.379
0.392
0-188
0.627
0-57
0.528
0.409
0-420
0.224
0.241
0:248
0.298
0.248
0:247
0.271
0.288
0.201
0.234
0.279
0.316
L.M Baxed Methodk
(50
MS6
01212
0.332
0.417
12
0.229
296
0.376
0-16
0-77
0.288
0.354
(L573
0.406
0.153
16
0.296
0.398
0.417
0.422
0.450
0.162
2014
0.254
OndFisaAll
MAE
0.3445
0.372
0.421
0.262
0.341
0.997
0-188
0:456
0.342
0.389
0.4077
0:441
0.238
0.251
003366
0.297
0.282
00
0.299
0.312
0.212
0.248
0.2386
0:337
MSE
0.3225
0.374
0.4
0.477
0.178
0.242
303
0.389
0-427
0-779
0.279
0.353
0.362
0-404
0.145
0,161
0.175
0.222
0.407
0.530
0.444
0.477
0.214
0.269
0.355
CALF P2]
MAB
0.349
0.375
0.399
0.638
0.256
0.297
0.339
0
0-23
0.436
0.467
0.531
0.394
0.426
0,258
0.252
0,267
0,303
0.268
0.27%
0
0.500
0,250
0.291
0.552
MSE
0.301
0.344
0.369
0.425
0.164
0.234
0.266
0.345
0.38
0.439
0.447
0.474
0.277
0.345
0.338
0.003
0.334
0.149
0.106
0.204
0.394
0.494
0.413
0.42%
0.193
0.244
0.311
PAJI
18
Simple Methock
MAE
1L542
0.366
0.413
0.340
0.292
0.319
0.387
0441
0-57
0.327
0,395
0.225
0.226
0,239
0:27
0.588
0,262
01265
0.270
01385
11232
0.273
0.573
MSE
00000
0,327
0,356
0.421
0.172
0.2266
0.269
0
0.428
0:405
0.295
0.351
0,3446
0.406
0.30
0,149
0.164
0.99
0.357
8388
0.424
0. 20305
0,253
0:322
LTst
11
MAB
0
0.386
1.3
0.416
0.258
0.295
0.523
0,372
0.390
04188
0.465
0.533
0.376
0.579
0426
0.220
0.239
0.254
0.278
0.231
0.241
0,250
0.273
0.204
0,246
0.283
Melbods
96
192
335
720
96
192
336
720
96
192
336
720
96
192
336
720
96
192
336
720
96
192
336
720
192
336
720
14 Wine
EETTI
PALLE
Wecihhrr
23

---

Methods Metric
Pre+FT (GPT2)
woPre+FF
Pre
woPre+woFT
MAE
MSE
MAE
MSB
MAE
MSE
MAE
MSE
0,3927
0.3695
0.3925
0.3747
0.3920
0.3735
0,3969
0.3798
192
0.4258
0.4290
0.4185
0.4268
0.4178
0.4243
0.4276
0.4345
336
0.4404
0,4510
0.4397
0.4627
0.4316
0.4526
0.4413
0.4656
0.4661
0.4757
0.4629
0.4807
0.4654
0.4804
0.5099
0.3359
0,2841
0.3291
0.2741
0.3340
0.2829
0.3336
0,2831
BTTh2
192
0.3782
0,3532
0,3780
0.3526
0,3758
0.3510
0,3778
0.3547
336
0.3937
0.3611
0.3992
0.3697
0.3946
0.3649
0,4006
0,3667
0.4275
0,4057
0.4295
0.4067
0.4277
0.4366
0.4172
0.3497
0.3228
0.3497
0.3240
0.3483
0.3231
0,3516
0.3243
ET7m2 ET1ml
192
0,3756
0.3751
0.3783
0.3789
0.3751
0.3750
0.3775
0,3801
336
0,4009
0.4108
0.3998
0.4092
0.4014
0,4105
0.4050
0,4152
720
0,4378
0.4765
0.4454
0,4931
0.4343
0.4730
0,4456
0.4916
0.2553
0.1771
0.2529
0.1752
0.2544
0.1771
0.2546
0.1768
192
0,3002
0,2446
0.2986
0.2466
0.2991
0.2451
0,2985
0,2438
336
0,3413
0.3086
0.3375
0.3064
0.3401
0.3079
0.3397
0,3070
720
0.3953
0.4023
0.3995
0.4124
0.3947
0.4005
0.3969
0.4038
24
0.7876
1.4596
0.7862
1.4470
0.8327
1.5333
0.8346
1.5892
Weather IIlness
36
0.8373
1.5726
0.8317
1.5531
0.8308
1 1,5083
0.8438
1.5347
0.8931
1,7503
0,9033
1.7689
60
0.9619
1,9824
0.8612
1.6051
0.9400
T. 1.8644
0.8837
1.6594
96
0.2065
0.1675
0.2092
0.1678
0.2076
0.1677
0.2112
0.1744
192
0.2506
0,2159
0.2526
0,2172
0.2555
0.2165
0.2567
0,2247
0.2923
0,2709
0.2967
0.2734
0.2939
0.2714
0,2949
0,2777
720
0.3454
0,3495
0.3458
0,3494
0.3514
0,3582
0,3476
0.3559
96
0,2737
0.4159
0.2622
0.4103
0,2708
0.4158
0.2751
0.4201
Electricity Traffic
192
0.2764
0.4302
0.2694
0.4287
0.2742
0.4324
0.2776
0.4356
336
0.2863
0.4507
0.2781
0,4456
0.2817
0.4485
0.2868
0.4530
720
0,3010
0,4783
0.2986
0.4792
0.3013
0.4817
0,3056
0.4844
96
0.2397
0,1469
0.2308
0.1377
0.2382
0.1442
0,2430
0.30
0.2538
0.2522
0.2541
336
0.2701
0,1785
0.2676
0.1733
0.2673
0.1768
0.2699
0.1797
720
0,3004
0.2148
0.2897
0987
0.2964
0,2104
0,2980
0.2156
#Wins
17
2
Table 20: Pretraining on language datasets not necessary for time series forecasting tasks, The table shows the performance of using pretraining models versus not using pretraining, as well as the combination of fine-tuning and not fine-tuning L.LMs in time series forecasting.
24

---

Dataset Predict Lengths
ETTh1
Illness
Sf-all.
SF-half.
Ex-half
Masking
SF-all.
SF-half.
Ex-half
Masking
Time-LLM
51.8%
5.6%
79.6%
32.5%
99.0%
33.6%
34.9%
64.6%
w/o LLM
56.0%-
4.5%
89.7%
39.5%
76.5%
20.9%
18.4%
53.0%
96(24)
LLMZAttn
53.8%
3.3%
92.2%
33.8%
72.7%
20.4%
13.1%
44.6%
LLMZTTsf
3.4%
34.8%)
74.5%
23.0%
14.3%
Time-LLM 43.9%
5.9%
72.1%
32.5%
83.9%
43.4%
30.7%
69.9%
Time-LIA LLM
192(36)
w/o LLM
46.8%
4.3%
77.3%
32.9%
77.0%
24.5%
15.3%
56.6%
LLMZAutn
45.0%
4,4%
08.
30.0%
73.8%
25.7%
11.2%
53.3%
LLM2Trsf
44,0%
5.5%
77.1%
29.9%
72.1%
27.0%
6,0%
51.3%
Time-LLM
39.1%
5.9%
67.0/%
24.7%
51.65
26.0%
11 11.2%
48.5%
336(48)
w/o LLM
41.7%
3.9%
71.6%
29.9%
73.6%
27.6%
13.6%
52.9%
LLMZAU 34.1%
1.2%
69.5%
21.1%
85.1%
38.8%
12.8%
57.5%
LLM2Tesf
54.9%
3.7%
75.0%
26.3%
85.2%
42.7%
18.6%
63.6%
Time-LLM
72.6%
31.8%
73.3%
32.1%
73.0%
42.4%
27.5%
76.3%
720(60)
w/o LLM
39.7%:
5.2%
64.3%
29.3%
74.1%
27.7%
14.25
56.6%
LLM2Aut 45.5%
6.0%
66.0%%
26.0%
66.2%%
28.5%
11.9%
51.7%
LLMZTesf
53.3%
63.7%
27.3%
67.7%
30.7%
7.8%
Dataset Predict Lengths
ETTh1 Ex-half
Illness
Sf-all.
SF-half.
Masking
Sf-all.
SF-half.
Ex-half
Masking
CALF
50.5%
9.6%
5.6%
8.5%
113.0%
47.4%
24.4%
22.9%
w/o LLM
12.1%
10.4%,
118.0%
28.9%
96(24)
LLM2Attn
51.9%
10.8%
5.8%
7.3%
87.3%
42.4%
35.1%
25.8%
LLM2Trsf
50.3%
8.3%
5.5%
7.0%
102.6%
56.2%
32.6%
26.0%
CALF (2)
CALF
41.7%
7.8%
3.3%
3.6%
100.9%
45.7%
12.9%
17.1%
192(36)
w/o LLM
50.9%
13.55%
4.5%
6.0%
15.66%
57.0%
28.3%
11.4%
LLM2Attn 45.8%
9.7%
3.6%
5,0%
78.8%
41.8%
10.1 1%
19.9%
LLM2Trsf
42.4%
8.3%
3.4%
4,3%
73.6%
42.5%
7.0%
17.5%
CALF
38.15:
9.0%
1.7%
5.0%
68.9%
43.0%
15.11%
14.9%
336(48)
w/o LLM
47.2%
14.5%
2.3%
8.7%
76.3%
49.0%
18.5%
18.1%
ILMZAU
5.5%
78.9%
54.2%
LLM2Trsf
40.3%
10.0%
1.9%
4.9%
78.9%
52.0%
23.4%
17.7%
CALF
36.5%
10.0%
0.7%
5.5%
63.88%
22.7%
5.4%
15.3%
720(60)
w/o LLM
41.0%
10.25
1.2%
6.2%
69.7%
30.3%
12.1%
19.4%
LLMZAU
36.0%
9.9%
0.7%
5.0%
71.7%
29.2%
12.7%
14.2%
LLM2Trsf
35.2%
9.3%
0.7%
5.1%
90.4%
55.3%
26. 4%
21.9%
Dataset Predict Lengths
ETTh1 Ex-half
Illness
Sf-all.
SE-hulf.
Masking
Sf-all.
SF-half.
Ex-half
Masking
OneFitsAll
62.1%
6.1%
16.6%
31.3%
86.2%
30.9%
36.7%
77.5%
w/o LLM
58.6%
6.1%
19.2%
36.1%
68.9%
13.0%
17.33%
43.5%
96(24)
LLM2Autt
68.5%
9.0%
15.0%
34.4%
108.3%
39.8%
44.2%
74.2%
LLM2Trsf
58.0%
7.8%
12.6%
30.2%
90.8%
27.4%
40.3%
60.6%
OneFitsAl 60
OnelitsAll
52.7%
8.2%
8.8/5
28.5%
52.0%
15.2%
8.5%
44.4%
w/o LLM
47.5%
6.1%
10.6%
31.8%
76.3%
24.0%
18.0%
47.5%
192(36)
LLM2Aut
80.8%
13.4%
6.7%
25.
97.8%
42.0%
36.3%
65.0%
LLM2Trsf
54.7%
12.5%
6.4%
24.1%
82.0%
29.6%
24.4%
63.8%
OneffisAAl
6.1%
31.3%)
79.5%
34.8%
25.11
336(48)
w/o LLM
58.6%
6.1%
19.2%
36.1%%
78.1%%
22.1%
15.7%
49.4%
LLMZAut
68.5%
9.0%
15.0%
34.4%
86.4%)
45.8%
21.7%
69.8%
LLM2Trsf
58.0%
7.8%
12.6%
30.2%
89.2%
35.8%
2..6%%
66.9%
OneFitsAll 40.1%
8.0%
3,5%
25.4%
53.6%
21.8%
9,9%
44.9%
720(60)
w/o LLM
39.4%
7.1%
4.1%
28.2%
74.5%
20.9%
17.1%
45.5%
LLMZAut
25.3%
72.8%
LLM2Trsf
87.2%
20.1%
1.6%
27.1%
92.3%
40.7%
29.3%
76.3%
Table 21: Results for input Sauufling/maskinn
for Time-LLMe CALF, and OneFitsAll methods on
ETTh (predict length are "96, 192, 336 and 720") and Illness (predict length "24, 36, 48 and 60"), the impact of shuffling the input on the degradation of time series forecasting performance does not change significantly before and after model modifications.
25

---

Dataset Input Ablation
ETTh2
Electricity
Sf-all.
Sf-half.
Ex-half
Masking
Sf-all.
Sf-half.
Ex-half
Masking
Time-LLM
27.1%
4.5%
44.6%
99.5%
212.1%
52.2%
323,4%
103.3%
w/o LLM
31.2%
2.9%
52.4%
137.9%
220.9%
47.3%
332.4%
105.5%
LLM2Attn
30.7%
2.9%
46.7%
134.8%
234.7%
58.2%
350.4%
115.3%
LLM2Trsf
24.8%
2.3%
43.4%
115.0%
240.8%
50.8%
362.0%
118.5%
OneFitsAll
33.0%
1.1%
39.7%
112.1%
237.8%
25.2%
382.0%
88.9%
w/o LLM
32.7%
2.8%
38.7%
115.8%
249.7%
43,4%
384.7%
109.2%
LLM2Attn
27.5%
1.0%
36.5%
137.4%
255.2%
49.5%
393.9%
119.2%
LLM2Trsf
29.6%
2.2%
37.2%
130.8%
263.8%
55.1%
406.4%
125.4%
CALF
11.3%
2.0%
9.1%
44.7%
235.1%
108.2%
29.3%
37.6%
w/o LLM
23.1%
4.1%
17.2%
46.9%
239.5%
109.2%
28.6%
39.4%
LLM2A
16.7%
3,4%
14.8%
31.7%
237.9%
116.2%
28.9%
42.2%
LLM2Trsf
18.9%
3.0%
15.3%
43.2%
245.6%
107.0%
29.2%
42.3%
Table 22: For the input shuffling/maskin experiments on ETTh2 and Electricity, the impact of shuffling the input on the degradation of time series forecasting performance does not change significantly before and after model modifications.
Dataset Input Ablation
ETTml
ETTm2
Sf-all.
Sf-half.
Ex-half
Masking
Sf-all.
Sf-half.
Ex-half
Masking
Time-LLM
66.6%
10.1%
107.7%
43.5%
47.0%
5.3%
77.7%
202.9%
w/o LLM
68.7%
12.4%
112.0%
52.6%
49.8%
4.7%
78.7%
199.6%
LLMZAttn
73.5%
13.5%
119.3%
50.7%
49.6%
4.7%
75.9%
196.3%
LLM2Trsf
72.0%
27.6%
117.0%
54.2%
46.4%
3.7%
76.5%
191.9%
OneFitsAll
74.4%
8.0%
123.4%
41.8%
48.0%
3.7%
82,9%
172.9%
w/o LLM
66.7%
10.3%
115.1%
45.0%
48.1%
4.4%
81.9%
190.7%
LLM2Attn
73.9%
11.1%
124.8%
51.6%
45.5%
3.3%
77.8%
183.6%
LLM2Trsf
70.1%
8.3%
126.2%
50.5%
48.2%
4.3%
82.0%
182.9%
CALF
64.3%
24.8%
122.3%
13.7%
23.5%
9.1%%
47.1%
61.7%
w/o LLM
66.7%
26.0%
123.6%
15.9%
27.1%
11.2%
51.9%
58.1%
LLM2Attn
62.5%
23.4%
122.3%
15.6%
25.2%
8.8%
51.5%
57.3%
LLM2Trsf
61.8%
25.6%
122.6%
13.6%
23.7%
8.2%
51.0%
59.3%
Table 23; For the input shuftling/masking experiments on ETTml and ETTm2, the impact of shuffling the input on the degradation of time series forecasting performance does not change significantly before and after model modifications.
Dataset
Weather
Traffic
Input Ablation
Sf-all.
Sf-half.
Ex-half
Masking
Sf-all.
Sf-half.
Ex-half
Masking
Time-ILLM
65.8%
5.5%
85.9%
85.9%
198.1%
56.6%
309.8%
101.3%
w/o LLM
59.2%
4.7%
78.7%
74.6%
196.8%
70.2%
282.1%
78.3%
LLM2Attn
71.8%
8.5%
91.6%
95.3%
212.3%
82.4%
312.7%
92.2%
LLM2Trsf
71.9%
7.1%
94.9%
103.2%
197.5%
64.6%
307.2%
101.1%
OleFitsAll
71.0%
9.8%
102.1%
97.7%
206.4%
11.7%
369.1% 1%
93.5%
w/o LLM
56.4%
2.3%
81.2%
73.5%
238.4%
67.0%
338.6%
85.9%
LLM2Attn
64.9%
5,4%
90.2%
94.8%
222.6% 6%
57.5%
350,5%
95,1%
LLM2TYsf
77.8%
8.3%
101.9%
111.0%
217.8%
35.9%
361.3%
113.3%
CALF
32.6%
4.2%
54.5%
27.2%
201.7%
105.0%
63.1%
27.2%
w/o LLM
28.1%
5.9%
49.4%
13.6%
217.3%
115.4%
67.2%
29.7%
LLM2Attn
26.0%
6.2%
49.2%
17.1%
226.7 7%
122.7%
69. 69.4%
30.7%
LLM2Trs
28.0%
6.4%
54.1%
21.1%
232.8%
120.5%
72.0%
29.6%
Table 24: For the input shuffinggmassiig experiments Weather and Traffic, the impact of shuffling the input on the degradation of time series forecasting performance does not change significantly before and after model modifications.
26

---

Tine LLM-EETTI
CALF-ER ChL
0.440
0.475
0.432
0.430
0.450
0.428
0.425
0.424
160
2.60
2,80
3,00
2.40
2.60
2.80
Time- Lime-LLM-ETTT
OneFits AU--TTTTTT
CALF-ETTh2
0.400
0.388
0.400
0.392
0.392
0.384
0.380
Mm.
OneittsAl-ETTTm
CALE-ETTml
0.380
0.380
U.392
0.370
0.375
0.388
100
200
1,20 OneFitssAU-Ileee
1.30
2.20
2.40 CALF-IUless
2.60
Time-LLM-Illene
0.920
0.920
0.885
0.880
0.880
0.840
0.840
40
60
8.00
8.50
9.00
34.
35.20
Time-LLM-Waathee
Weacher
CALF-Weather
0.270
0.270
0,280
0.260
0.260
0.275
0.30
0.60
0.270
8.00 Inference Time (ms)
Inference Time (ms)
Inference Time (us)
w/ LLM
wyo LLM
LLM2AI5
LLM2Trsf
Figure Ablation methods consume less time for inference while providing better forecasting performance in most cases. The figure above shows the inference time and prediction accuracy of Time-LLLM, OneFitsAll, and CALF on ETTh1, ETTh2, ETTml, Illness, and Weather, Traffic datasets, averaged across prediction lengths. Results of other datasets refer to Figure 3
27

---

Time-LLM-ETT
OueFitsAll-
CALF-ETTHI
0.475
0.432
0,430
0.450
0.428
0,420
01.225
0.424
80
160
2,60
2,50
3.00
2.40
2.60
2.80
Time- LLM-ETTT2
OueFitsAll -I--ETE ETT12
CALF-EITh2
0.358
MSE
0.391
0.400
0.354
0
0.380
80
160
2.60
2.80
2.25
2.40
2.55
Tine. Tine-LLMMEE. L.M.EETTTT
CALF-ETTHE
0.380
MSE
0.375
0.380
*
0.392
0.370
0.375
0.358
100
200
1.20
1.30
2.20
2.10
2
Time--LMM- MM2T
OnePissAD-E ETTTu
CALF CLLF-ETION Tm2
0.318
0.315
MSE
0.324
0.312
0.310
*
0.320
0.3006
100
200
(1,005
1.26
1.32
1.38
1.95
2.10
Inference Time (ms)
Inference Time (me)
Inference Time (ms)
LLM
w/o LLM
LLM2Att1
LfM2TTsf
(a) Inference Time and Performance for ETThI, ETTH2, ETTml, and ETTm2.
Time-I.MM-Ilee Illnes
On-
CALF-Illnes
0.920
0.920
MSB
0.885
0.880
0.880
0.870
0.840
0.840
60
8.00
8.50
9.00
31 10
35.20
Time--LM-
L.M-Leathheee Weather
OneFicsAll Weather
CALF-Weuther Wenther
MSE
0.270
0.270
0.280
0.260
0.275
0.270
0
80
160
0.30
0.45
0.60
4,00
8.00
Time- LrM-Trafic
OneFRtsAl All--Traaee Trafic
CALF-Traffe
0.300
0.288
MSE
0.250
0.270
0.2800
0.280
0,260
*
0.272
0.30
0.45
0.60
30.00
60.00
90.00
Time LLM- ECI.
OneFRs
ECI
0.272
0.270
MSE
0.264
0.260
0.265
0.250
0.260
0.30
0.15
0.60
12,00
18.00
Inference Time (jus)
Inference Time (ms))
Infereace Time (aus)
I.L.
w/o LLM
LLM2Attn
L.MM2Ysf
(b) Inference Time and Performance for Illness, Weather, Traffic, and Electricity.
Figure 8: Ablation methods consume less time for inference while providing better forecasting performance in most cases. The figure above shows the inference time and prediction accuracy of Time-LLM, OneFitsAll, and CALF on all the datasets, averaged across prediction lengths in MSE metric.
28

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: Abstract and the Introduction sections include the the paper's contributions and scope.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Appendix A discusses the limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: The paper does not make any theoretical contributions.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: Section and Appendix Dishows details of the baselines used and their hyper-pate hyper-parameter details respectively.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: have shared the code as a zzip file supplementary material.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Experiments 3.3 and Appendix Dedecribe them.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We have provided the details of confidence intervals for all our results in Appendix E
8. Experiments Compute Resources
Question: For each experiment, does the paper provide sufficient information the puter resources (type of computing workers, memory, time of cxecution) needed to reproduce the experiments?
Answer: [Yes]
Justification: We mention the hardware details in Appendix
29

---

9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics http:p/s/.aaappppp..pppppiicccit
Answer: [Yes]
Justification: Yes, we do follow the Code Of Ethics.
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
Appendix B describes broader impact.
Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA].
Justification: The paper poses no such risks.
12. Licenses for existing assets
Question: Are creators or original owners of assets (e.g., code, data, models), used in the paper properly credited, and the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We provide the license details in Appendix
13. New Assets
Question: Are new assets introduced in the paper well documented, and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We attached the code in supplementary material. All implementation and hyper-parameter details are also reported in the paper.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsoureing or research with human subjects. 15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing research with human subjects
30


Reviews:
[{"review_id": "EgINJ3JRln", "reviewer": "Reviewer_jLi6", "summary": "This paper presents an extensive empirical study on the effect of pre-trained LLMs in time series forecasting tasks. By ablating popular LLM adaptations on widely adopted time series benchmarks, experiments in the paper show LLMs do not benefit from pre-training on text data to gain improvement in forecasting performance. It suggests that the patching and first layer of attention makes most of contribution in forecasting setups.", "questions": "See weakness", "limitations": "Limitations are pointed out in the paper by the authors.", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "This paper is very well written and populated with extensive empirical evidences to support its main arguments. \n1. The experiment setting covers various types of LLM adaptation to forecasting, including simple fine-tuning, PEFT and modality alignment, as well as popular time series forecasting benchmarks, which makes the conclusions fairly solid. \n2. The reasoning behind the 6 research questions are convincing with rigorous ablation studies, and it covers multiple aspects of applicability of LLM in forecasting, from efficacy to efficiency. \n3.  Great reproducibility evidences are provided for an empirical study.", "weaknesses": "A questionable piece of results is the random baseline in Table 5. It is counter-intuitive that, while the `woPre+woFT` variant is completely unfitted according to line 208-210, but its metrics are quite close to other variants in all experiments. \n\nMinor typos/errors:\n1. line 76: One method ~this~ is", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the encouraging feedback and the very positive review! We've fixed the typo and responded to your query about Table 5 below.\n\n**Why does \"woPre+woFT\" perform well in Table 5?**\n\nThis is a great observation! We were initially surprised by this, too. But we’d like to clarify that “woPre+woFT” describes only the LLM in the model, where the LLM’s parameters are randomized (woPre) and then frozen and left untrained (woFT). **The rest of the model is still finetuned.** So the fact that its metrics are similar to the other variants actually backs up our main claim: If the rest of the architecture can achieve similar performance even with a random, frozen LLM injected into the middle, the LLM is likely not driving the forecasting performance.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "I4oBfWaQu9", "reviewer": "Reviewer_UHrk", "summary": "This paper explores the effectiveness of language models in time series forecasting. The authors substantiate their claim by performing three straightforward ablations of three popular and recent LLM-based forecasting methods. After extensive experiments, the authors find that patching and attention structures perform similarly to state-of-the-art LLM-based forecasters.", "questions": "1. Why the author choose ETTh1 and ILI datasets in RQ4? Results on other benchmark datasets are expected.\n2. How does the author implement the \"LTrsf\" model, and what is the difference between \"LTrsf\" and the existing iTransformer model? \n3. Can you compare the results between \"PAttn\" and PatchTST?", "limitations": "yes", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper is well-structured and easy to follow.\n2. The author question existing LLM-based forecasting models by conducting comprehensive ablation studies and provide very interesting and insightful observations.", "weaknesses": "1. Although the paper focuses on the effectiveness of LLM in TSF, existing state-of-the-art forecasting models have been omitted. \n2. The authors conduct ablation studies on the patching and decomposition of LLM-based models to find out where the performance comes from. However, there is a lack of discussion on LLaTA, which embeds the input time series by treating each channel as a token instead of using patching.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your focused and actionable review. We are glad you agree our work provides interesting and insightful observations! As detailed point-by-point below, **we’ve addressed your remaining concerns by running your suggested experiments**.\n\n**W1: Including state-of-the-art forecasting models**\n\nThank you for this suggestion! We’d like to clarify that **each LLM-based method claims to be state-of-the-art,** reporting they outperform non-LLM methods like PatchTST and iTransformer:\n* *“TIME-LLM is a powerful time series learner that outperforms state-of-the-art, specialized forecasting models”* (taken from Time-LLM’s abstract).\n* *“[LLaTA] establishes state-of-the-art performance for both long-term and short-term forecasting tasks”* (taken from LLaTA’s abstract; note that “LLaTA” was renamed to “CALF, which we will update in our final version).\n* *“pre-trained models on natural language or images can lead to a comparable or state-of-the-art performance in all main time series analysis tasks”* (taken from OneFitsAll’s abstract). \n\nSo the merit of our work doesn’t depend on this comparison. Our reproduced MSE and MAE values are also nearly-identical to those of each method’s original experimental results, so we have no reason to believe this comparison won’t also hold. We also agree with you that comparing to non-LLM methods is largely out-of-scope of our contributions.\n\nBut to address your suggestion directly, and in case future readers have a similar question, we have added extended versions of all comparison tables to the Appendix (see pdf uploaded with our rebuttal). These tables include results from the methods compared in the iTransformer paper. As expected, our main findings are unchanged: The LLM-based methods are slightly better than the non-LLM methods, but our ablations indicate this isn’t due to the LLM.\n\n**W2: Including more details on LLaTA’s source of performance**\n\nThanks for this suggestion, we believe you are describing RQ6 where we show that a simple patching method is surprisingly performant (compared to all methods and also other new and simple baselines). While RQ6 focuses on encoders and particularly on the more-popular patching and decomposition, there are still some insights about LLaTA’s encoder. In Section 4.6, we will clarify that “LTrsf” is LLaTA’s encoder without cross-modal attention.\n \n**Q1: Why does RQ4 focus on the ETTh1 and Illness datasets?**\n\nThank you for suggesting we add more datasets to RQ4—We chose ETTh1 and Illness randomly due to compute constraints leading up to the submission.\n\nPer your suggestion, **we ran this experiment, so now RQ4 includes all 8 datasets**. We've previewed our findings below, but could only include \"Sf-all\" and \"Ex-half\" due to OpenReview's space constraints. All results will be added to the Appendix. We also note that **answering RQ4 doesn't depend on the number of datasets** because if LLMs’ strong sequence modeling really drove their forecasting performance, shuffling time series should always drop their performance more than their ablations’.\n\n|                      | **ETTh2** |         | **Electricity** |         | **ETTm1** |         | **ETTm2** |         | **Weather** |         | **Traffic** |         |\n|:--------------:|:---------:|:-------:|:---------------:|:-------:|:---------:|:-------:|:---------:|:-------:|:-----------:|:-------:|:-----------:|:-------:|\n|  |  Sf-all.  | Ex-half |     Sf-all.     | Ex-half |  Sf-all.  | Ex-half |   Sf-all. | Ex-half |   Sf-all.   | Ex-half |   Sf-all.   | Ex-half |\n| Time-LLM       |   27.1%   |  44.6% |     212.1%      | 323.4%  |   66.6%   | 107.7%  |  47.0%    |  77.7%  |    65.8%    |  85.9%  |    198.1%   |  309.8% |\n| w/o LLM        |   31.2%   |  52.4%  |     220.9%      | 332.4%  |   68.7%   | 112.0%  |  49.8%    |  78.7%  |    59.2%    |  78.7%  |    196.8%   |  282.1% |\n| LLM2Attn       |   30.7%   |  46.7%  |     234.7%      | 350.4%  |   73.5%   | 119.3%  |  49.6%    |  75.9%  |    71.8%    |  91.6%  |    212.3%   |  312.7% |\n| LLM2Trsf       |   24.8%   |  43.4%  |     240.8%      | 362.0%  |   72.0%   | 117.0%  |  46.4%    |  76.5%  |    71.9%    |  94.9%  |    197.5%   |  307.2% |\n\n\n**Table**: Subset of results from RQ4 experiments on six more datasets. Due to OpenReview's character constraints, we only show \"Sf-all\" and \"Ex-half\" results for Time-LLM—These experiments are already completed for all methods. These new results agree with those from our original paper.\n\n\n**Q2: How is LTrsf implemented and how does it connect to iTransformer?**\n\nThank you for noting the similarity between LTrsf and iTransformer. The key difference is that \"LTrsf\" leaves each channel independent, ignoring multivariate relationships. LTrsf is implemented as LLaTA's encoder with cross-modal attention removed (so it's only the transformer encoder) followed by one linear layer. We will mention this in RQ6 and clarify that they outperform one another in different cases, so their strengths appear orthogonal.\n\n**Q3 : Can we compare PAttn and PatchTST?**\n\nThanks for suggesting we add PatchTST's results to those of PAttn. We have added this comparison to the pdf uploaded with our rebuttal. We’d like to note that comparing non-LLM forecasting methods is unrelated to our work. Still, we will add this result, per your suggestion, to establish how PAttn compares to others. Please also note that we use PAttn to probe the behavior of LLM-based forecasting methods, not to outperform non-LLM methods. However, it is competitive in many cases.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LKqm2zeXaz", "reviewer": "Reviewer_wccd", "summary": "A recent surge of papers have popularised the usage of pre-trained LLMs for time series forecasting. The paper analyses the claim that LLMs are useful for time series forecasting, by performing a series of ablation studies. Their conclusion is that LLMs bring little to no benefit for the task, and are significantly more costly.", "questions": "See weaknesses", "limitations": "Authors have adequately addressed limitations in appendix.", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. The paper is very well written, the hypothesis is stated clearly and the experiments and results are also clearly stated.\n2. Experimental approach is very sound, experiments are well thought out and very comprehensive.\n3. The paper reveals very significant findings regarding a recent trend in time series forecasting, in which many papers at top conferences have been publishing about. It turns out that the performance of many of these methods is not due to a pre-trained LLM as touted by these papers, but due to other factors such as patching and channel independence is the most important.", "weaknesses": "1. Experiments in section 4.3 are only presented for LLaTa, it should be presented for all the models. Importantly, the OneFitsAll paper showed that pre-trained weights performed much better than random initialized weights. Can the authors comment on this?\n\n2. The experiments in 4.6 lead to a model that is very similar to PatchTST and some other similar patch-based Transformers, which the LLM-based models claim to be better than. Can the authors comment on this?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the positive and encouraging feedback and for your endorsement of our work! We've addressed your questions below.\n\n**W1: Why does RQ3 only consider LLaTa?**\n\nThanks for suggesting we extend RQ3 to include all methods. We completely agree this would strengthen this RQ and our choice was due to computational constraints—**The “woPre+FT” ablation requires training the LLM from scratch**. Training Time-LLM’s 7B base LLM so many times is beyond our immediate resources. But we will take your suggestion and see if we can find a way to run this for the final version. We also omitted OneFitsAll from this experiment because OneFitsAll and LLaTA have similar performance, use the same base LLM, and our aim in this RQ is just on the impact of the LLM’s pretraining. But we will take your suggestion and run this experiment. We strongly believe this finding doesn't hinge on this experiment, but we will include at least one more method (hopefully two).\n\nFor OneFitsAll’s random initialization experiment, we’d like to note that their results were reported in a few-shot setting, unlike ours. Running our experiment again in a few-shot setting is an interesting future experiment, though.\n\t\t\t\n**W2: How does PAttn compare to PatchTST?**\n\nThis is a great question! We agree that they are similar, and we’d like to note that PAttn is designed only to explore the role encoders play in LLM-based forecasters’ performance. As shown in our extended comparisons (see our uploaded pdf), PAttn and PatchTST actually do perform similarly, likely due to their architectural similarity. The key difference is PAttn lacks a position embedding and feedforward layer in the transformer.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Uu3HKxRnq3", "reviewer": "Reviewer_CG3z", "summary": "This paper questions the effectiveness of LLMs in time series forecasting. Through a series of ablation studies on 3 LLM-based forecasting methods, the authors find that removing or replacing the LLM component often improves forecasting results. They conclude that LLMs do not provide significant benefits for time series forecasting tasks, despite their significant computational cost. The paper suggests that simpler models or encoders with basic attention mechanisms perform equally well, challenging the current trend of employing LLMs for time series forecasting.", "questions": "Related to my last point in \"weakness\", can the authors provide more theoretical explanations or hypotheses on why LLMs do not perform well in time series forecasting compared to simpler models?", "limitations": "The authors acknowledge that the study focuses solely on time series forecasting and uses only evenly spaced datasets. Future research should explore LLMs’ effectiveness in other time series tasks and with non-uniform datasets. Additionally, potential negative societal impacts, such as the environmental cost of training large models, should be considered.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper offers an interesting perspective on the use of LLMs in time series forecasting, challenging prevalent assumptions and trends in the field.\n- The empirical analysis seems to be valid, with well-executed ablation studies and comparisons across multiple methods and datasets.\n- The paper is easy to read, with well-organized sections and effective use of figures and tables to present relevant results.\n- The findings may have implications for the research community, potentially redirecting efforts towards more efficient and effective methods for time series forecasting.", "weaknesses": "- The scope is limited to time series forecasting and does not explore other potential applications of LLMs in time series analysis, such as imputation, classification or anomaly detection.\n- The datasets used are all evenly spaced time series, which may not fully represent the variety of real-world time series data, such as those with irregular intervals.\n- The paper could benefit from a more detailed discussion on the potential reasons why LLMs fail to outperform simpler models in this context, providing deeper theoretical insights.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your thoughtful and positive review! We've responded to each of your points below.\n\n**W1: Can our work extend to other uses of LLMs for time series?**\nWe agree this is an exciting, natural direction for our work. While it’s beyond the scope of one paper, we hope our work inspires the community to do exactly this, ultimately taking a closer look at such LLM-based methods broadly.\n\n**W2: Can our work extend to irregularly-sampled time series?**\nThanks for noting our focus on evenly-spaced time series. We completely agree that irregularly-sampled time series are of critical interest and we hope to consider them in future works. For this current work, we believe our hypotheses are best tested by following the prior works’ experimental setups as closely as possible and as LLMs are used for more data types, we hope they are subjected to careful examination!\n\n**W3 and Question: Could we provide deeper discussions of LLMs’ weaknesses and give theoretical insights?**\nThis is a great idea and diving deeper into *why* the LLMs underperform is extremely interesting. But there are many open questions about this type of interpretability and many challenges of proving negative results. While we probe at this “why” question in Sections 4.3-4.6, we hope our work inspires others to ask similar questions and join in developing the theory behind when/why/if LLMs can benefit time series forecasting (and other tasks). As it stands, we believe our work successfully presents tension between the promise of big, multi-modal models and a need to deeply understand the sources of practical performance. \n\nPer your direct question, developing theoretical explanations should be done with care over longer periods of time than a rebuttal. But in the spirit of brainstorming, we provide two speculative lines of attack for future works:\n1. There may be a lack of key, transferrable properties of time series in LLM pre-training data. For example, forecasting often depends on learning periodic trends in time series, which are not a key signal present in natural language.\n2. Long, forecastable sequences of numbers may not be in the pretraining data, driving a need to update an LLM’s weights beyond the point of leveraging its language abilities.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Thank you to all reviewers for your thoughtful feedback—we are thrilled to see such positive reception!

To summarize this feedback, the reviewers emphasize our study's importance, noting we present **"very significant findings regarding a recent trend in time series forecasting"** [wccd], **"very interesting and insightful observations"** [UHrk], and how our **"findings may have implications for the research community, potentially redirecting efforts towards more efficient and effective methods for time series forecasting"** [CG3z]. Reviewers also recognize the quality of our experiments, commending how our paper includes **"extensive empirical evidences to support its main arguments"** [jLi6], how we show **"comprehensive ablation studies"** [UHrk], how our **"experimental approach is very sound, experiments are well thought out and very comprehensive"** [wccd], and how our **"empirical analysis seems to be valid, with well-executed ablation studies and comparisons across multiple methods and datasets"** [CG3z]. All reviewers also appreciated our clarity, noting our paper is **"very well written"** [jLi6, wccd], **"well-structured and easy to follow"** [UHrk], and **"easy to read, with well-organized sections and effective use of figures and tables to present relevant results"** [CG3z].

We’ve responded to each reviewer's questions below, and we'd also like to highlight a few general improvements we’ve made to our work based on your comments:
* Based on suggestions by Reviewer UHrk, we have run our RQ4 experiments to include **six more datasets**. This rounds out RQ4, which now includes all 8 datasets studied in our original submission. We find the same trends as those on the original 2 datasets studied in RQ2.
* Based on suggestions by Reviewers UHrk and wccd, we have **included comparisons to non-LLM methods** to our main results as a new table in the Appendix (shown in the attached pdf). 
* We also ran our main experiments on **five more forecasting datasets** (ExchangeRate, Covid Deaths, Taxi, NN5, and FRED-MD). We will include these new results in the final version of our paper. We find the same trends as those on the original 8 datasets.

Results from each new experiment strengthen our findings even further. So thank you again for these suggestions, they have concretely improved this work!

Meta-review: The paper presents an interesting empirical study by questioning the effectiveness of LLMs in time series forecasting.  The study considers ablations of LLM adaptation to forecasting and popular time series forecasting benchmarks. The finding that pretrained LLMs are not better than models trained from scratch is worth publishing.
Decision: Accept (spotlight)