Time-LLM: Time Series Forecasting by Reprogramming Large Language Models
Authors: Ming Jin
Shiyu Wang
Lintao Ma
Zhixuan Chu
James Y. Zhang
Xiaoming Shi
Pin-Yu Chen
Yuxuan Liang
Yuan-Fang Li
Shirui Pan
Qingsong Wen
Venue: ICLR 2024 poster (2024)
Keywords: time series forecasting
large language models
model reprogramming
Primary area: representation learning for computer vision, audio, language, and other modalities
TL;DR: We present Time-LLM, a reprogramming framework to repurpose LLMs for general time series forecasting with the backbone language models kept intact.

Abstract: Time series forecasting holds significant importance in many real-world dynamic systems and has been extensively studied. Unlike natural language process (NLP) and computer vision (CV), where a single large model can tackle multiple tasks, models for time series forecasting are often specialized, necessitating distinct designs for different tasks and applications. While pre-trained foundation models have made impressive strides in NLP and CV, their development in time series domains has been constrained by data sparsity. Recent studies have revealed that large language models (LLMs) possess robust pattern recognition and reasoning abilities over complex sequences of tokens. However, the challenge remains in effectively aligning the modalities of time series data and natural language to leverage these capabilities. In this work, we present Time-LLM, a reprogramming framework to repurpose LLMs for general time series forecasting with the backbone language models kept intact. We begin by reprogramming the input time series with text prototypes before feeding it into the frozen LLM to align the two modalities. To augment the LLM's ability to reason with time series data, we propose Prompt-as-Prefix (PaP), which enriches the input context and directs the transformation of reprogrammed input patches. The transformed time series patches from the LLM are finally projected to obtain the forecasts. Our comprehensive evaluations demonstrate that \method is a powerful time series learner that outperforms state-of-the-art, specialized forecasting models. Moreover, Time-LLM excels in both few-shot and zero-shot learning scenarios. The code is made available at https://github.com/KimMeen/Time-LLM.

Published as a conference paper at ICLR 2024
TIME-LLM: TIME SERIES FORECASTING BY REPROGRAMMING LARGE LANGUAGE MODELS
Ming Jin'; Shiyu Wang2; Lintao Ma², Zhixuan Chu?, James Y. Zhang', Xiaoming Shi?,
Pin-Yu Yu Chen², Yuxuan Liang", Yuan-Fang Li', Shirui Pan'! Qingsong Wen
Monash University 2 Ant Group IBBM Research *Griffith University Allibbaa Group The Hong Kong University of Science and Technology (Guangzhou)
(ming.jimm yuanfang- 1 ] @monash. edu, pin-yu. mnn-yu-chen@tee wtm.0ibbmccm com
yuxl wwlliang@eutlook com, panegriffith. edu gingeongedu@gmail..cm
fweiming, wsy, aint mlt, chuzhi c2x, janes. 2, peter. sxn/eantgrouup.coo
ABSTRACT
Time series forecasting holds significant importance in many real-world dynamic systems and has been extensively studied. Unlike natural language process (NL.) and computer vision (CV), where single large model can tackle multiple tasks, models for time series forecasting are often specialized, necessitating distinct de- signs for different tasks and applications. While pre-trained foundation models have made impressive strides in NLP and CV, their development in time series domains has been constrained by data sparsity. Recent studies have revealed that large language models (LLMs)) possess robust pattern recognition and reasoning abilities over complex sequences of tokens. However, the challenge remains in effectively aligning the modalities of time series data and natural language to leverage these capabilities. In this work, we present TIME-LLM, reprogram- ming framework to repurpose LLMs for general time series forecasting with the backbone language models kept intact. We begin by reprogramming the input time series with text prototypes before feeding it into the frozen LLM to align the two modalities. To augment the LLM'' ability to reason with time series data, we propose Prompt-as-
(PaP), which enriches the input context and directs the transformation of reprogrammed input patches. The transformed time series patches from the LLM finally projected to obtain the forecasts. Our com- prehensive evaluations demonstrate that TIME-LLM a powerful time series learner that outperforms state-of-the-an thtt---rrt specialized forecasting models. More- over, TIME-LLM excels in both few-shot and zero-shot learning scenarios. The code is made available at https::/gittiiii com/ KimMeen/ Time- LLM
INTRODUCTION
Time series forecasting is critical capability across many real-world dynamic systems (Jin al. 2023a Hu et al.) 2024). with applications ranging from demand planning (Leonard (001)) and inven- tory optimization (Li et al. 2022) to energy load forecasting (Liu et al. 2023a) and climate modeling (Schneider & Dickinson, 1974). Each time series forecasting task typically requires extensive do-
main expertise and mask-specific model designs. This stands in stark contrast to foundation language models like GPT-3 (Brown et al. 22020). GPT-4 (OpenAl 2023), Llama (Touvron et al. (023), inter alia, which can perform well on diverse range of NLP tasks a few-shot or even zero-shot setting.
Pre-trained foundation models, such as large language models (LLMs), have driven rapid progress computer vision (CV) and natural language processing (NLP). While time series modeling has not benefited from the same significant breakthroughs, LLMs' impressive capabilities have inspired their application to time series forecasting (Jin et al.] 2023b; 2024). Several desiderata exist for leveraging L.LMs to advance forecasting techniques: Generalizabiliiy. LLMS have demonstrated remarkable capability for few-shot and zero-shot transfer learning (Brown et al. 2020). This suggests their
"quual Contribution "orresponning Authors

---

Published as a conference paper at ICLR 2024
potential for generalizable forecasting across domains without requiring per-task retraining from scratch. In contrast, current forecasting methods are often rigidly specialized by domain. Data ef- ficiency. By leveraging pre-trained knowledge, LLMs have shown the ability to perform new tasks with only a few examples. This data efficiency could enable forecasting for settings where historical data is limited. In contrast, current methods typically require abundant in-domain data. Reason- ing. LLMs exhibit sophisticated reasoning and pattern recognition capabilities (Mirchandani et al.) 2023 Luo et al. 2023b a). Harnessing these skills could allow making highly precise forecasts by leveraging learned higher-level concepts. Existing non-LLM methods are largely statistical without much innate reasoning. Multimodal knowledge. As LLM architectures and training techniques im- prove, they gain more diverse knowledge across modalities like vision, speech, and text (Ma et al.) 2023). Tapping into this knowledge could enable synergistic forecasting that fuses different data types. Conventional tools lack ways to jointly leverage multiple knowledge bases. Easy optimiza- tion. LLMs are trained once on massive computing and then can be applied to forecasting tasks without learning from scratch. Optimizing existing forecasting models often requires significant architecture search and hyperparameter tuning (Zhou et al. 2023b). In summary, LLMs offer a promising path to make time series forecasting more general, efficient, synergistic, and accessible compared to current specialized modeling paradigms. Thus, adapting these powerful models for time series data can unlock significant untapped potential.
The realization of the above benefits hinges on the effective alignment of the modalities of time series data and natural language. However, this challenging task as LLMs operate on discrete tokens, while time series data is inherently continuous. Furthermore, the knowledge and reasoning capabilities interpret time series patterns are not naturally present within LLMs' pre-training. Therefore, it remains an open challenge to unlock the knowledge within LLMs in activating their ability for general time series forecasting in a way that accurate, data- eeffeeeffiieen, and task-agnostic.
In this work, we propose TIME-LMM. a reprogramming framework to adapt large language models for time series forecasting while keeping the backbone model intact. The core idea to reprogram the input time series into text prototype representations that are more naturally suited to language models' capabilities. To further augment the model's reasoning about time series concepts, we introduce Prompt-as-Prefi (PaP), a novel idea in enriching the input time series with additional context and providing task instructions in the modality of natural language. This provides declarative guidance about desired transformations to apply to the reprogrammed input. The output of the language model is then projected to generate time series forecasts. Our comprehensive evaluation demonstrates that large language models can as effective few-shot and zero-shot time series learners when adopted through this reprogramming approach, outperforming specialized forecasting models. By leveraging LLMs' reasoning capability while keeping the models intact, our work points the way toward multimodal foundation models that can excel on both language and sequential data tasks. Our proposed reprogramming framework offers an extensible paradigm for imbuing large models with new capabilities beyond their original pre-training. Our main contributions in this work can be summarized as follows:
We introduce a novel concept of reprogramming large language models for time series forecast- ing without altering the pre-trained backbone model. In doing so, we show that forecasting can be cast yet another "language" task that can be effectively tackled by an off-the-shelf I.I.M.
We propose new framework, TIME-LLM, which encompasses reprogramming the input time series into text prototype representations that are more natural for the LLM,, and augmenting the input context with declarative prompts (e.g., domain expert knowledge and task instructions) to guide LLM reasoning. Our technique points towards multimodal foundation models excelling in both language and time series.
TIME- LLM consistently exceeds state-of-the-an performance in mainstream forecasting tasks, especially in few-shot and zero-shot scenarios. Moreover, this superior performance is achieved while maintaining excellent model reprogramming efficiency. Thus, our research is a concrete step in unleashing LLMs' untapped potential for time series and perhaps other sequential data.
RELATED WORK
Task-specific Learning. Most time series forecasting models are crafted for specific tasks and domains (e.g., traffic prediction), and trained end-to-end on small-scale data, An illustration is in
2

---

Published as a conference paper at ICLR 2024
(a) Task-Specific Learning
Model Fine-Tuning
(c) Model Reprogramming
Source Madality
Lightweight
Berrartatton]
P Model 0d
Reproggum Spice
Head
Language Model
High Effectiveness
Target Modality
Model d
Contextual Bootstrapping
Broertating
Iroue Rapragrum
(Cha Baation]
pmoptasion]
Pra-
Cross-Modality
Source Data Sample
Target Data Sample
Source Task
Target Task
Frazen
Fine-tune
-..
Pre-training
Figure I: Schematic illustration of reprogramming large language models (LLMs) in comparison of (a) task-specific learning and (b) model one-ttning. Our proposal investigates and demonstrates (c) how to effectively reprogram open-sourced LLMs as powerful time series learners where well- developed time series pre-trained models are not readily available.
Fig. a). For example, ARIMA models are designed for univariate time series forecasting (Box et al. 2015), LSTM networks are tailored for sequence modeling (Hochreiter & Schmidhuber 1997.. and temporal convolutional networks (Bai al.) (0188) and transformers (Wen et al.) 2023) are developed for handling longer temporal dependencies. While achieving good performance on narrow tasks, these models lack versatility and generalizability to diverse time series data.
In-modality Adaptation. Relevant research in CV and NLP has demonstrated the effectiveness of pre-trained models that can be fine-tuned for various downstream tasks without the need for costly
training from scratch (Devlin et al.) 2018: Brown al. 2020 Touvron et al.) 2023). Inspired by these successes, recent studies have focused on the development of time series pre-trained models (TSPTMs). The first step among them involves time series pre-training using different strategies like supervised (Fawaz et al., 2018) or self-supervised learning (Zhang et al., [022bb Deldari et al.) 2022] Zhang et al. 2023). This allows the model to learn representing various input time series. Once pre- trained, it can be fine-tuned on similar domains to learn how to perform specific tasks (Tang et al.) 2022). An example is in Fig. Tjb). The development of TSPTMs leverages the success of pre- training and fine-tuning in NLP and CV but remains limited on smaller scales due to data sparsity.
Cross-modality Adaptation. Building on in-modality adaptation, recent work has further explored transferring knowledge from powerful pre-trained foundations models in NLP and CV to time se- ries modeling, through techniques such as multimodal fine-tuning (Yin et al. 2023) and model reprogramming (Chen) (022). Our approach aligns with this category; however, there is limited pertinent research available on time series. An example is Voice2Series (Yang al. 202T). which adapts an acoustic model (AM) from speech recognition to time series classification by editing time series into a format suitable for the AM. Recently, [Cang et al.) (2023) proposes LLM4TS for time series forecasting using LLMs. It designs a two-stage fine-tuning process on the LLM first supervised pre-training on time series, then task-specific fine-tuning. Zhou et al. (2023a) lever- ages pre-trained language models without altering their self-attention and feedforward layers. This model is fine- nune-tueddde and evaluated on various time series analysis tasks and demonstrates comparable or state-of-the-ar performance by transferring knowledge from natural language pre-training. Dis- tinct from these approach, we neither edit the input time series directly nor fine-tune the backbone LLM. Instead, as illustrated in (nni. TJe), we propose reprogramming time series with the source data modality along with prompting to unleash the potential of LLMs as effective time series machines.
3 METHODOLOGY
We focus reprogramming an embedding- visible language foundation model, such as Llama (Touvron et al. 2023) and GPT-2 (Radford et al. 2019). for general time series forecasting without requiring any fine-tuning of the backbone model. Specif- ically, we consider the following problem: given a sequence of historical observations X E IR x consisting of N different I-dimensional variables across T time steps, we aim reprogram large language model f(:) to understand the input time series and accurately forecast the readings at H future time steps, denoted by Y E N with the overall objective to minimize the mean square errors between the ground truths Y and predictions, i.e., H ΣH I Y, 1/.
Our model architecture is depicted in Fig. 2
Our method encompasses three main components: (1) input transformation, (2) a pre-trained and frozen LLM,, and (3) output projection. Initially, multivariate time series is partitioned into N
3

---

Published as a conference paper at ICLR 2024
Oulput Embeddings
Forecasts
Add & Layer Narm
Output Projection
Feed Forward
Flatten & Linear
Add & Layer Norm
Output Patch Embeddings
Pre-trained LLM (Body)
Multi-Head Attention
Input Embeddings
Output Token Embeddings
Reprogrammed Patch Embeddiings
A
Token Embedder
Pre-trained LLM
Patch Reprogram
Linear
(Embedder)
Tokenization
Patching !
Multi-Head Attention
contaer Bascipriins
Instance Norm
Patch Embedder
Text Prototypes
Input
incruction Information
Deraate: alovrii inoddeiger>
Linear
leput etatetes
Time Series
Pre-trained
Patches
Embeddings
Frozen
Training
Prompt Embeddings
Patch Embeddings
Forward
Backward
Figure 2: The model framework of TIME-LLM. Given an input time series, we first tokenize and embed it via ① patching along with 2 customized embedding layer. ! These patch embeddings are then reprogrammed with condensed text prototypes to align two modalities. To augment the LLM's reasoning ability, 4 additional prompt prefixes are added the input to direct the transfor- mation of input patches.
55) The output patches from the LLM are projected to generate the forecasts.
univariate time series, which are subsequently processed independently (Nie et al. 2023). The i-th series is denoted as ( RIXT which undergoes normalization, patching, and embedding prior to being reprogrammed with learned text prototypes to align the source and target modalities. Then, we augment the LLM's time series reasoning ability by prompting it together with reprogrammed patches to generate output representations, which are projected to the final forecasts ( E <RI
We note that only the parameters of the lightweight input transformation and output projection are updated, while the backbone language model is frozen. In contrast to vision-language and other multimodal language models, which usually fine-tune with paired cross-modality data, TIME-LLM is directly optimized and becomes readily available with only small set of time series and a few training epochs, maintaining high efficiency and imposing fewer resource constraints compared to building large domain-specine models from scratch fine-tuning them. To further reduce mem- ory footprints, various off-the-shelf techniques (e.g., quantization) can be seamlessly integrated for slimming TIME-LLM.
3.1 MODEL STRUCTURE
Input Embedding, Each input channel ( is first individually normalized to have zero mean and unit standard deviation via reversible instance normalization (RevIN) in mitigating the time series distribution shift (Kim et al. 2021). Then, we divide (9) into several consecutive overlapped or non-overlapped patches (Nie et al. 2023) with length Lp: thus the total number of input patches P [ + 2, where denotes the horizontal sliding stride. The underlying motivations two-fold: (1) better preserving local semantic information by aggregating local information into each patch and (2) serving as tokenization to form a compact sequence of input tokens, reducing computational burdens. Given these patches X Px embed them X (i) gPxdm, adopting a simple linear layer as the patch embedder to create dimensions dm.
Patch Reprogramming. Here we reprogram patch embeddings into the source data representation space to align the modalities of time series and natural language to activate the backbone's 's time series understanding and reasoning capabilities. A common practice is learning form of "noise" that, when applied to target input samples, allows the pre-trained source model to produce the desired target outputs without requiring parameter updates. This is technically feasible for bridging data
4

---

Published as a conference paper at ICLR 2024
0.6 Projection N N
Source
Target
+ Preitrained LLM
N
fl
N
Reprogrammed Petch Embeddings 0
hmme
trairrannd
(Bady Head)
Patch
Patch
steedy
Patch Reprogram
Pre-troined LLM
Pre-trained LLM Embedder)
Patch Reprogram
Patch
long
nest value
Suput comtent>
Sinttrcction
(a) Patch Reprogramming
(b) Patch-as- Prefix and Prompt-as-Preth
Figure 3: Illustration of (a) patch reprogramming and (b) Patch-as-Prefis versus Prompt-as- Prefff.
modalities that are identical or similar. Examples include repurposing a vision model to work with cross-domain images (Misra et al. (023) or reprogramming an acoustic model to handle time series data (Yang et al. 2021). In both cases, there are explicit, learnable transformations between the source and target data, allowing for the direct editing of input samples. However, time series can neither be directly edited nor described losslessly in natural language, posing significant challenges to directly bootstrap the LLM for understanding time series without resource-intensive fine-tuning.
To close this gap, we propose reprogramming X using pre-trained word embeddings E RVX in the backbone, where V is the vocabulary size. Nevertheless, there is no prior knowledge indi- cating which source tokens are directly relevant. Thus, simply leveraging E will result large and potentially dense reprogramming space. A simple solution is to maintain a small collection of text prototypes by linearly probing E, denoted as E' E RV''D, where V < V. An illustration is in Fig. 3(a). Text prototypes learn connecting language cues, c.g., "short up" (red lines) and "steady down" (blue lines), which are then combined to represent the local patch information (e.g., "short up then down steadily" for characterizing patch 5) without leaving the space where the language model pre-trained. This approach efficient and allows for the adaptive selection of relevant source information. To realize this, we employ a multi-head cross-attention layer. Specifically, for each head k (1,.... K), define query matrices Q.6 x w key matrices K E'WK, and value matrices v(( = E'WY, where W Rd.. xd and W
k W E Dxd Specifically, D is the hidden dimension of the backbone model, and d= | Then, we have the operation to reprogram time series patches in each attention head defined as:
(1) (
Z ATTENTTINN Q (6) K( V()) = SOFTMAX
(V((
dd
By aggregating each 2 E Pxd in every head, we obtain Z E Pxdm This is then linearly projected to align the hidden dimensions with the backbone model, yielding 0(0) E RR PXD.
Prompt-as-Preff Prefix. Prompting serves as straightfor- ward yet effective approach task-specific activation of LLMs YYn et al.) 2023). However, the direct translation of time series into natural language presents considerable challenges, hindering both the creation of instruction- following datasets and the effective utilization of on-the-
The Electricity Transformer Temoerature (ETT) indicates the electric
tona-term deployment. Each data point consists
of target temperature 6 power ioad features.. Delow is the information about the input time series: [BEGIN DATA]
[Domain]: We usually observe that electricity comsumption peals at noon, with signtficant increase in transformer load ***
fly prompting without performance compromise (Xue & Salim (002).. Recent advancements indicate that other data modalities, such as images, can be seamlessly in- tegrated as the prefixes of prompts. thereby facilitating effective reasoning based on these inputs (Tsiimoukelli et al. 2021). Motivated by these findings, and to render our approach directly applicable to real-world time series, we pose alternative question: can prompts act pre- fixes to enrich the input context and guide the transfor- mation of reprogrammed time series patches? We term this concept Prompt-as- -reefxx (PaP) and observe that it significantly enhances the LLM's adaptability to downstream tasks while comple- menting patch reprogramming (See SSc. 4.5 later).
[Instruction]: Predict
steps given
<J' steps information attached
[Statistics]: input minimum smin_valy maximum of cmoo, valo, a median of cnedin vab. The overall trend is cupward or downwards, The top five lags are vals
DATA]
Figure 4: Prompt example. < and are task-specific configurations and cal- culated input statistics.
5

---

Published as a conference paper at ICLR 2024
An illustration of the two prompting approaches is Fig. 3 b). In Patch-as- -Peeff, language model is prompted to predict subsequent values in a time series, articulated in natural language. This ap- proach encounters certain constraints: (1) language models typically exhibit reduced sensitivity in processing high-precision numerals without the aid of external tools, thereby presenting substantial challenges in accurately addressing practical forecasting tasks over long horizons; (2) intricate, cus- tomized post-processing is required for different language models, given that they are pre-trained on diverse corpora and may employ different tokenization types in generating merals with precision and efficiency. This results in forecasts being represented in disparate natural language formats, such as ['0', '6', '1'] and ['O', :.. '61'], to denote the decimal 0.61.
nu-
Prompt-as- Preff-x,-PPrefx,, on the other hand, tactfully avoids these constraints. In practice, we identify three pivotal components for constructing effective prompts: (1) dataset context, (2) task instruction, and (3) input statistics. A prompt example is in Fig. 4 The dataset context furnishes the LLM with essential background information concerning the imput time series, which often exhibits distinct characteristics across various domains. Task instruction serves as crucial guide for the LLM in the transformation of patch embeddings for specific tasks. We also enrich the input time series with additional crucial statistics, such trends and lags, to facilitate pattern recognition and reasoning.
Output Projection. Upon packing and feedforwarding the prompt and patch embeddings O()) through the frozen LLM as shown in Fig. 2 we discard the prefixal part and obtain the output representations. Following this, we flatten and linear project them to derive the final forecasts (
4
MAIN RESULTS
TIME-LLM consistently outperforms state-of-the-an forecasting methods by large margins across multiple benchmarks and settings, especially in few-shot and zero-shot scenarios. We compared our approach against a broad collection of up-to-date models, including a recent study that fine-tunes language model for time series analysis (Zhou et al. [2023a). To ensure fair comparison, we adhere to the experimental configurations in (Wu et al.
2023) across all baselines with a unified evaluation pipeline We use Llama-7B (Touvron et al. 2023) as the default backbone unless stated otherwise.
Baselines. We compare with the SOTA time series models, and we cite their performance from Zhou et al. 2023a) if applicable. Our baselines include a series of Transformer-based methods: PatchTST (2023). ESTformer (2022). Non- Stationary Transformer (2022). FEDformer (2022). Aut-
oformer (021), Informer (2021). and Reformer (2020). We also select set of recent competi- tive models, including GPT4TS (023a). LLMTTime (2023). DLinear (2023). TimesNet (2023), and LightTS (022a). In short-term forecasting, we further compare our model with N-HiTS (2023b)
and N-BEATS (2020). More details are in inpppeddix A
4.1 LONG-TERM FORECASTING
Setups. We evaluate ETTh1, ETTH2, ETTm1, ETTm2, Weather, Electricity (ECL), Traffic, and ILI, which have been extensively adopted for benchmarking long-term forecasting models (Wul al. @ (023). Details of the implementation and datasets can be found in Appendix B The input time series length T is set as 512, and we use four different prediction horizons / E [96, 192, 336, 720). The evaluation metrics include mean square error (MSE) and mean absolute error (MAE).
Results. Our brief results are shown in Tab. where -LM outperforms all baselines in most cases and significantly so to the majority of them. The comparison with GPT4TS (Zhou et al. 2023a) is particularly noteworthy. GPT4TS is very recent work that involves fine-tuning on the backbone language model. We note average performance gains of 12% and 20% over GPT4TS and TimesNet, respectively. When compared with the SOTA task-specific Transformer model PatchTST, by reprogramming the smallest Llama, TIME-LLM realizes an average MSE reduction of 1.4%. Relative to the other models, e.g.. DLinear, our improvements are also pronounced, exceeding 12%.
4.2 SHORT-TERM FORECASTING
Setups. We choose the M4 benchmark (Makridakis et al. 2018) as the testbed, which contains a collection of marketing data in different sampling frequencies. More details are provided in (Ap pendix B The prediction horizons in this case are relatively small and in [6, 48). The input lengths

---

Published as a conference paper at ICLR 2024
Table 1: Long-term forecasting results. All results are averaged from four different forecasting horizons: H {24, 36,4% 60) for ILI and {96, 192, 336, ]20] for the others. A lower value indicates better performance. Red: the best, Blue: the second best. Our full results are in Appendix D
Methods
TIME-LLM
GPT-TS
Di incar
TimoeNeet
FED
Astufiema
Statimary
ETSfomer
LighTS
Inferma
Refermer
(Ouno
2023
[00
pvair
Matric
MSE
MAE
MSE
MSE
MAE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MSE
MAE
MAE
MSE
MAE
67741
0.408
0.43
0:465
0.435
[.422
0.437
0,413
0.430
[.458
0.450
0.440 0.460 0.406 0.487
0.570 0.537 0.542 0,510 (0:45
0.479 1,010
0.795
1,020
0.805
ETTM2
0.334
0.383
0.381
0.412
[0:4] 0.446
0.330
0.379
[0.414
0:427
[0.437 0.449
0.450
0.459
(.5t6 0.516
0..36 0.452
(0,022
0.543
4,431
1,729
6.736
2.10
ETTml
0.329
0.372
0.368
0.403
[.335
0.378
0:351
0.580
[0.400
0.406
[0.448 0.452
0.558
0.517
[0448 0:456
(0429
0.425 (0:435
0.457
0.961
0.734
[0.79
0.671
ETTw2
[0251
0,313
0.339
[1.267
[0:29]
0.333
(0.305 0.549
0.337 0.571 (0,906 0:347
10:M0 0,542 (a
0.810
1.49
Wanthr
[0225
0.257
(0:25)
0.270
[0.248
0.300
00.325
0.264
[0.259
0.287
(0,30y 0.900
(0.33 0,382 (0.215 0,314 (0:27] 0.534 (0.201 0:312
0.044
0.548
[0w0 0.000
BCL
[0.158
0,252
00,007
0.263
[0.166 0.263
00:001
0.252
[0,192
0,295
(0.214 0.327
00227 0,335 (0.195 0,290
0.200 0,525 (0.229 0,329 0,311
0.997
[0.338 0.422
Troffic
[.358
0.254]
00:14
0.294
[1433
[16620
0.336
(0.500
0.376
(.628 0,379
(o6224
0.340
0.621 0,306 (0:02
0.416
(0744 0.422
[.435
0.801
1,925
0.003
22.15
1.04|
1.443
0.797
2.130
0.33
[2.847
1.146] 3.006 1.161 (2.077
0.04 2.407 1.004
7.382
20000
1.544
4.724
1.445
0
5
0
8 0
0
Table 2: Short-term time series forecasting results on M4. The forecasting horizons are in [6, 48] and the three rows provided are weighted averaged from all datasets under different sampling intervals. A lower value indicates better performance. Red: the best, Blue: the second best. More results are in Appendix D
TIME-LLM
GPT4TS
TimesNei 20233
PatchTST
N-IBIS
N-BEATS
ETSfommer
LightTS
DLinear
FEDformer
Stationary
AAttofomer (3021)
Informer
Reformer
Methods
(Ours)
2025g
2023
202330
3020
2022
20220
P023
2022
2023
2021
20200
Averge
SMAPE
11,983
12,89
12.88
12,059
12.035
12.25
14.718
13.525
13,639
13.16
12,780
12.999
14.086
18.200
MASE
1.595
1,808
1.836
1.623
1665
1.69%
2,408
2.111
2,093
1.775
1.756
1.711
2.718
4,223
OWA
0,859
0.896
1,172
1.051
0.999
1.230
1.775
twice prediction horizons. The evaluation metrics are symmetric mean absolute percentage error (SMAPE), mean absolute scaled error (MSAE), and overall weighted average (OWA).
Results. Our brief results with unified seeds across all methods are in Tab. 2] TIME-LLM consis- tently surpasses all basclines, outperforming GPT4TS by 8.7%. TIME-LLM remains competitive even when compared with the SOTA model, N-HiTS (Challu et al. 2023b). w.r.t. MASE and OWA.
4.3 FEW-SHOT FORECASTING
Setups. LLMs have recently demonstrated remarkable few-shot learning capabilities (Liu et al., 2023b). In this section, we assess whether our reprogrammed L.M retains this ability in forecasting tasks. We adhere to the setups in (Zhou al. 2023a) for fair comparisons, and we evaluate on scenarios with limited training data (i.e., S first 10% training time steps).
Our brief 10% and 5% few-shot learning results are in Tab. 3add Tab. reeppcctivly TIME- LLM remarkably excels over all bascline methods, and we attribute this to the successful knowledge activation in our reprogrammed LLM. Interestingly, both our approach and GPT4TS consistently surpass other competitive baselines, further underscoring the potential prowess of language models as proficient time series machines.
Results.
In the realm of 10% shw- learning, our methodology realizes 5% MSE reduction in comparison to GPT4TS, without necessitating any fine-tuning on the LLM. In relation to recent SOTA models
Table 3: Few-shot learning on 10% training data. We use the same protocol in Tab. All results are averaged from four different forecasting horizons: H E (96, 192, 336, 7203. Our full results in Appendix E
Matheck
TIME-LLM
GPT-TS
Diiisss
PachTST Big
TameaNer
FEDiemas
Autufoemne
Stalinnayy
ETStomer
LighTs
Infenme N0
Reformer 2000
(Ouns)
X02
Maric
MSR
MAF
MSP
MAR]
MSR
MAR
MSP:
MAF
MSR
MAR
MSS
MAF
MSF
MAR
MSH
MAR
MSR
MAF:
MSR
MAR
MSP
MAR
MSR
MAR
ETTAL
[.556
0.522
0.500
0.935
(.691
8.000
0.631
0.542
(1889 0.625 (0,590 0.561
1,200
0.595
(.915
0.699
1.190 0.834
1,335 0377
1.399 0.09
1:349 0883
6,370
0.394
0.075
0.488
0.462
0.39
0.404
0.427
0.464
0.44]
[0.411
0.429
0.501
DL466
[0.677
0.557
(0.722 0.605 0.802
0.625
0.797
0.578
0.990
0.714
0.901
0.705
1.192 0.821
1.426 0.856
6r/ml
|.277
0.333
0,203
0.335
[.316
0.368
0.296
0.343
[1.320
0.353
[0.463
0.488
1.342
0.930
0.332 0.366
0.447 0.487
0.987
0.756
3,370 1.440
3,078 1.587
Wenther
60234
0.273
0.238
0.275
0.241
0.279
0.301
[0.284 0.324
0.300
0.342
(0318
0.323
0.318 0.360 0.259 0.322
0.409
BCL
[0.155
0.270
0.176
0.269
[0.150
0.290
0.080
0.273
[0.323
0.392
(0.346
0.427
0.431
0.478
(0444 0.450
0.6600
0.517 (.44) 0.459
0.891
0.965 0.768
Treffie
[.429
11300
0.440
0.300
(1447
0.313
0.905
[1951
0.538
(0.663
0.425
10:40
0,446
1,455
1.914
0,998
1.248 0.64
1.554 0.11
1.551 0331
CoCont

---

Published as a conference paper at ICLR 2024
Table 4: Few-shot learning 5% training data, We the same protocol in Tab. All results averaged from four different forecasting horizons: E {96, 192, 336, 7203. Our full results are in Appendix E
Methods
(Duns)
LLM
GPEETS M0l
PoschTST H0:
Timesher
2023
FliDfocttrr
Autoformer
Statinaayy H02
E5Stommer A00
LighTTs Aloa
Informer
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAB
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTAL
0.627
0.543
0.681
0.560
[.750
[.604
0.647
10.65
0.562
0.722
0.722
0.595
0.943
0.646
1.88
0.539
1.451 0.033 1.225
0.835
6TTR2
0.882
0.118
0.400
[1,090
0.577]
0.615
0:488
(0,463
0.054
(0:441
0.457
0.470
0.409
0,081
53200 1.208 3,22
1.003
5557
1,422
ET7ml
1.450
0,472
0.400
[1:000
0,417
10:226
0.976
(0.717
0.561
(0.730
0.992
0.790
0,629
(0887
0,598
1.125
0,782
1,125
0.200
1.361
0.191
1.264
0.806
ETTm2
0.274
0,325
0,300
0.346
[0.99M
01420
00:11
0.352
10:344
0.372
(0.88) 0.900
0.388
0,433
0.541
0/312
0.554
0.547
1:415
03811
3,58
1,089
3,551
1.487
Weether
0.260
0,300
0.203
0.901
[0.205
0.905
0,318
(0,309
0.355
0,310
0.353
(0.22 0,330
0.371
(0,305
0.527
[1447
BCL
0.179
0.268
00.178
0.273
[8.176
0.275
(0.881
0.277
[0,402
0.453
(0.265
0.353
0.346
0.404
[06627
0.608
0.500
0.585
(0.878
0.725
1,281
0.929
1,259 0.904
Traffic
0.423
0.298
0:434
0.505
[0.450
0.317
0,415
0.296
[1.667
0.493
(0,676
0:423
0.833
0.502
1,526
0.879
1,359
0,927
1:557
0,705
1.591 0.832
1618 0.851
1"Cout
such PatchTST, DLinear, and TimesNet, our average enhancements surpass 8 %% 12%, and 33% w.r.t. MSE. Analogous trends are discernible in the 5% few-shot learning scenarios, where our average advancement over GPT4TS exceeds 5%. When compared with PatchTST, DLinear, and TimesNet, TIME-LLM manifests a striking average improvement of over 20%
Table 5: Zero-shot learning results. Red: the best, Blue: the sec- ond best. Appendix E shows our detailed results.
4.4 ZERO- SHOT FORECASTING
Setups. Beyond few-shot learning, LLMS hold potential as effective zero-shot reasoners (Kojima et al.] 2022). In this section, we evaluate the zero-shot learning capabilities of the reprogrammed LLM within the framework of cross-domain adapta- tion. Specifically, we examine how well a model performs on dataset & when it is optimized on another dataset A where the model has not encountered any data samples from the dataset & Similar to few-shot learning, we use long-term forecasting protocol and evaluate on various cross-domain scenarios utilizing the ETT datasets.
Metbods
TIME-1
GrTETS
1LMTime
PaccTTT 5024
TimmNNe A0D5
(Ours)
2023
3023
Metric
MSE
MAB
MSC
MAC
MSE
MAE]
MSE
MAE
MSE
MAL
MSE
MAE
ETTAL 6TTh2
0.353
0.387
0.466
0,422
0.0022
0.708
(.449
0.488
0.380
0.405
0.442
ETTM FT7m2
6.273
0,340
0.325
0,363
1,867
0.809
(a.115
0.452
0.314
0.360
0.327
0,361
ETTM2 ETTh1
[0.479
0.474
0.757
0.578
1.961
0.981
|0.70a
0.574
0.965
0.621
ETTA2 - ET7m2
[0272
0.341
0.335
0,370
1.667
0,809
|0.329
0.386
0.325
0.365
(0.342
0.576
8TTml 67772
[6.381
0,412
0.431
0,439
(0,992
0.706
(..54
0.475
0.439
0.433
[0,457 0,654
ETTmL 67Tmm
0.268
0.320 01313
0.348
1.667
0,869
003335
0.388
[12296
0.334
(0.322
FTT=2 - 627722
|0.354
0.400
0.435
0,443
[0,992
0.706
(1:455
0.471
0.401
0.425
(0,435 0,443
ETTm2 577ml [0.414 0.435
[0.44
0.769 0.567
1,935
0.954
0.649
0.537
(0.709 0.567
Results. Our brief results are in Tab. 5 TIME-LLM consistently outperforms the most competitive baselines by a large margin, over 14.2% % w.r.t. the second- in MSE reduction. Considering the few-shot results, we observe that reprogramming an LLM tends to yield significantly better results in data scarcity scenarios. For example, our overall error reductions w.r.t. GPT4TS in 10% few-shot forecasting, 5% few-shot forecasting, and zero-shot forecasting are increasing gradually: 7.7%, 8.4%, and 22% Even when benchmarked against LLMTime, the most recent approach in this field, with the backbone LLM of comparable size (7B), TIME-LLM shows substantial improvement exceeding 75% We attribute this to our approach being better at activating the LLM's knowledge transfer and reasoning capabilities in resource-efficien manner when performing time series tasks.
4.5 MODEL ANALYSIS
Language Model Variants. We compare two representative backbones with varying capacities (A.1-4 in Tab. 6). Our results indicate that the scaling law retain after the LLM reprogramming. We adopt Llama-7B 7B by default in its full capacity, which manifestly outperforms its 1/4 capacity variant (A.2; inclusive of the first 8 Transformer layers) by 14.5%. An average MSE reduction of 14.7% is observed over GPT-2 (A.3), which slightly outperforms its variant GPT-2 (6) (A.4) by 2.7%
Cross-modality Alignment. Our results in Tab. 6 6findicate that ablating either patch reprogramming or Prompt-as-Prefix hurts knowledge transfer in reprogramming the LLM for effective time series forecasting. In the absence of representation alignment (B.1), we observe a notable average perfor- mance degradation of 9.2%, which becomes more pronounced (exceeding 17%) in few-shot tasks. In TIME-LLM, the act of prompting stands as a pivotal element in harnessing the LLM's 's capacity for understanding the inputs and tasks. Ablation of this component (B.2) results in over 8% and 19% degradation in standard and few-shot forecasting tasks, respectively. We find that removing the input statistics (C.1) hurts the most, resulting in an average increase of 10.2% % MSE This is
8

---

Published as a conference paper at ICLR 2024
Table 6: Ablations on ETThl and ETTml in predicting 96 and 192 steps ahead (MSE reported). Red: the best.
Variant
Long serm l'orecasting
Few shot Porecasting
ETT
E1Th1-192
ETTml-96
ETThml-192 192
ETThi-96
ETThi-192
ETTal-96
FTThm1-192
A.1 Liama (Default: 32)
0.362
0.395
0.272
0.310
0.448
0.484
0.346
0.373
0.389
0.412
0.297
0.329
0.567
0.451
0,490
(12)
0.385
0.447
0.290
0
0.571
0.640
0.468
0,512
B.I w/o Patch Reprogramming
0,400
0.412
0.310
0.342
0.498
0.570
0.445
0,487
B.2 wlo Prompe as Prelix
0.298
0.423
0.298
0.339
0.521
0.617
0.432
0
C.I wio Doraset Contest
0.402
0.417
0.208
0.331
0.491
0.538
0.392
0.447
C.2 w/o Task Instruction
0.288
0.420
0.285
0.327
0.476
0.529
0.387
0.439
C.3 wio Statictcal Conteut
0.291
0.419
0.279
0.347
0.433
0.547
0.421
0.461
Table 7: Efficiency analysis of LLM on ET forecasting different steps ahead.
Linggh
IrTTh1-45
IFF
IrFThi-912
Monse
Plraer (ND
(ML3)
Speod) Speod//fier) /ieen]
Pheaer (ND
(Ma3)
Speed//rier) Phraes (3)
Men. (MiB)
Specd/rrten) Parass (MD Menu./Miiit Speedkyricr)
D. (52)
5404.55
32136
0.517
5404.57
33762
0.580
5804.2
57955 131.8
0:632
5404.9
30004
0:697
D.2 LLaaLa (80
975.3
11370
0.184
975.57
12392
D.182
975.92
0.300
976.11
13616
0.317
D.3 LLM
6.39
3678
0.04G
6.42
3812
0.067
6.48
3564
0.083
6:35
4176
0.129
ticipated as external knowledge can be naturally incorporated via prompting to facilitate the learning and inference. Additionally, providing the LLM with clear task instructions and input context (e.g., dataset captioning) is also beneficial (i.e., C.2 and C.1; eliciting over 7.7 % and 9.6%, %, respectively).
Reprogramming Interpretation. We pro- vide case study on ETThl of reprogram- ming 48 time series patches with 100 text prototypes in Fig. 5) The top 4 subplots visualize the optimization of reprogramming space from randomly-initialized (a) to well- optimized (d). We find only small set of pro- totypes (columns) participated in reprogram- ming the input patches (rows) in subplot (e). Also, patches undergo different representations through varying combinations of prototypes. This indicates: (1) text prototypes learn to sum- marize language cues, and select few are highly relevant for representing information in local time series patches, which we visualize by randomly selecting 10 in subplot (I). Our results suggest high relevance to the words that describe time series properties (i.e., word sets I and 2); (2) patches usually have different underlying seman- tics, necessitating different prototypes to represent.
10 Eanen
(N) Spoch
Id soudh
Ie Swid-10
Tedt Autittyee
Teel Protogpe
Ted Pattdage
10 Staullation 10 offereet loanel protonppes Werdien ('ettod besseet 21 DC toomart
Wend
Wher Set
Figure 5: A showcase of patch reprogramming.
Reprogramming Efficiency. Tab. 7 provides an overall efficiency analysis of TIME- with and without the backbone LLM. Our proposed reprogramming network itself (D.3) is lightweight in activating the LLM's ability for time series forecasting (i.e., fewer than 6.6 million trainable parameters; only around 0.2% of the total parameters in Llama-7B), and the overall efficiency of
TIME-LLM is actually capped by the leveraged backbones (c.g.. D.1 and D.2). This is favorable even compared to the parameter-efficient fine-tuning methods (e.g., QLoRA DDttmers el al. 2023 in balaneing task performance and efficiency.
CONCLUSION AND FUTURE WORK
TIME-LLM shows promise in adapting frozen large language models for time series forecasting by reprogramming time series data into text prototypes more natural for LLMs and providing natural language guidance via Prompt-as-Prefif to augment reasoning. Evaluations demonstrate the adapted LLMSs can outperform specialized expert models, indicating their potential as effective time series machines. Our results also provide a novel insight that time series forecasting can be cast as yet an- other "language" task that can be tackled by an off-the-shelf LLM to achieve state-of-the-ar perfor- mance through our Time-LLM framework. Further research should explore optimal reprogramming representations, enrich L.LMs with explicit time series knowledge through continued pre- -raiiiinnnn and build towards multimodal models with joint reasoning across time series, natural language, and other modalities. Furthermore, applying the reprogramming framework to equip LLMs with broader time series analytical abilities or other new capabilities should also be considered.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS AND DISCLOSURE OF FUNDING
The authors extend their deep gratitude to the Intelligent Engine Technology Division of Ant Group for their support in completing this research. We also express special thanks to the Language and Machine Intelligence Department and the Optimization Intelligence Department for their support. Our sincere appreciation directed Jun Zhou, Vice President of the Intelligent Engine Technology Division, and Xingyu Lu, Senior Staff Engineer, for their expert guidance.
This material is based on research partially sponsored by the DARPA Assured Neuro Sym- bolic Learning and Reasoning (ANSR) program under award number FA8750-23-2-1016 and the DARPA Knowledge Management at Scale and Speed (KMASS) program under award number HRO01 12220047. S. Pan was supported in part by the Australian Research Council (ARC) un- der grants FT210100097 and DP240101547, and the CSIRO National Science Foundation (US) AI Research Collaboration Program. Y. Liang was supported by the Granggzou-HKUSTTOR Joint Funding Program (No. 2024A03J0620).
REFERENCES
Shaojie Bai, J Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint
2018.
George EP Box, Gwilym M Jenkins, Gregory C Reinsel, and Greta M Ljung. Time series analysis: forecasting and control. John Wiley & Sons, 2015.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in Neural Information Processing Systems, 33:1877-1901, 2020.
Cristian Challu, Kin G Olivares, Boris N Oreshkin, Federico Garza, Max Mergenthaler, and Artur Dubrawski. N-hits: Neural hierarchical interpolation for time series forecasting. Proceedings of the AAAI Conference on Artificial Intelligence, 2023a.
Cristian Challu, Kin Olivares, Boris N Oreshkin, Federico Garza Ramirez, Max Mergenthaler Canseco, and Artur Dubrawski. Nhits: neural hierarchical interpolation for time series forecast- ing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 6989-6997, 2023b.
Ching Chang, Wen-Chih Peng. and Tien-Fu Chen. Llm4ts: forecasting with pre-trained lims. arXiv preprint 0rXiv:2308.08465 2023.
Two-stage fine-tuning for time-series
Pin-Yu Chen. Model reprogramming: Resource-effiiint cross-domain machine learning. arXiv
preprint arXiv:2202. 10629, 2022.
Shohreh Deldari, Hao Xue, Aaqib Saeed, Jiayuan He, Daniel V Smith, and Flora D Salim. Beyond just vision: A review on self- supervised representation learning on multimodal and temporal data. arXiv preprint 2tXXi::2006.02353, 2022.
Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized 1ls. Advances in Neural Information Processing Systems, 2023.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter the Association for Computational Linguistics: Human Language Technologies, 2018.
Hassan Ismail Fawaz, Germain Forestier, Jonathan Weber, Lhassane Idoumghar, and Pierre-Alas Muller. Transfer learning for time series classification. In IEEE International Conference on Big Data, pp. 1367-1376. IEEE, 2018.
Nate Gruver, Mare Anton Finzi, Shikai Qiu, and Andrew Gordon Wilson. Large language models zero-shot time series forecasters, Advances in Neural Information Processing Systems, 2023.
10

---

Published as a conference paper at ICLR 2024
Julien Herzen, Francesco Lassig, Samuele Giuliano Piazzetta, Thomas Neuer, Leo Tafti, Guillaume Raille, Tomas Van Pottelbergh, Marek Pasieka, Andrzej Skrodzki, Nicolas Huguenin, et al. Darts: User-friendly modern machine learning for time series. The Journal of Machine Learning Re- search, 0:5442-5447, 2022.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):
1735-1780, 1997.
Jiaxi Hu, Yuehong Hu, Wei Chen, Ming Jin, Shirui Pan, Qingsong Wen, and Yuxuan Liang, At- tractor memory for long-term time series forecasting: chaos perspective. arXiv preprint arXiv:2402. 1463, 2024.
Ming Jin, Huan Yee Koh, Qingsong Wen, Daniele Zambon, Cesare Alippi, Geoffrey I Webb, Irwin King, and Shirui Pan. A survey graph neural networks for time series: Forecasting, classifica- tion, imputation, and anomaly detection. arXiv preprint arXiv:2307.03755 2023a.
Ming Jin, Qingsong Wen, Yuxuan Liang, Chaoli Zhang, Siqiao Xue, Xue Wang, James Zhang, Yi Wang, Haifeng Chen, Xiaoli Li, et al. Large models for time series and spatio-temporal data: A survey and outlook. arXiv preprint arXiv:2310. 10/96, 2023b.
Ming Jin, Yifan Zhang, Wei Chen, Kexin Zhang, Yuxuan Liang, Bin Yang, Jindong Wang, Shirui Pan, and Qingsong Wen. Position paper: What can large language models tell about time series analysis. arXiv preprint arXiv:2402.02713. 2024.
Taesung Kim, Jinhee Kim, Yunwon Tae, Cheonbok Park. Jang-Ho Choi, and Jacgul Choo. Re- versible instance normalization for accurate time-series forecasting against distribution shift. In International Conference Learning Representations, 2021.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. International Conference Learning Representations, 2015.
Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. Reformer: The efficient transformer. In International Conference Learning Representations, 2020.
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199 22222199-2221,, 2022.
Michael Leonard. Promotional analysis and forecasting for demand planning: practical time series approach. with exhibits, 2001.
Na Li, Donald M Arnold, Douglas G Down, Rebecca Barty, John Blake, Fei Chiang, Tom Courtney, Marianne Waito, Rick Trifunov, and Nancy M Heddle. From demand forecasting to inventory ordering decisions for red blood cells through integrating machine learning, statistical modeling, and inventory optimization. Transfusion, 62(1):87-99 99, 2022.
Hengbo Liu, Ziqing Ma. Linxiao Yang. Tian Zhou, Rui Xia, Yi Wang, Qingsong Wen, and Liang Sun. Sadi: self-adaptive decomposed interpretable framework for electric load forecasting under extreme events. In IEEE International Conference Acoustics, Speech and Signal Pro- cessing, 2023a.
Xin Liu, Daniel McDuff, Geza Kovacs, Isaac Galatzer-Levy. Jacob Sunshine, Jiening Zhan, Ming- Zher Poh, Shun Liao, Paolo Di Achille, and Shwetak Patel. Large language models are few-shot health learners. arXiv preprint arXiv:2305./5525 2023b.
Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Non-stationary transformers: Exploring the stationarity in time series forecasting. Advances in Neural Information Processing Systems, 35:9881-9883,, 2022.
Linhao Luo, Jiaxin Ju, Bo Xiong, Yuan-Fang Li, Gholamreza Haffari, and Shirui Pan. Chatrule: Mining logical rules with large language models for knowledge graph reasoning. arXiv preprint 009002300 2023a.
11

---

Published as a conference paper at ICLR 2024
Linhao Luo, Yuan-Fang Li, Reza Haf, and Shirui Pan. Reasoning graphs: Faithful and inter- pretable large language model reasoning. In The Twelfth International Conference on Learning Representations, 2023b.
Ziyang Ma, Wen Wu, Zhisheng Zheng, Yiwei Guo, Qian Chen, Shiliang Zhang, and Xie Chen. Leveraging speech ptm, text Ilm, and emotional its for speech emotion recognition. arXiv preprint arXiv:2309. 2023.
Spyros Makridakis and Michele Hibon. The m3-competition: results, conclusions and implications. International journal of forecasting, 16(4))451-476, 2000.
Spyros Makridakis, Evangelos Spiliotis, and Vassilios Assimakopoulos. The m4 competition: Re- sults, findings, conclusion and way forward. International Journal of Forecasting, 34(4):802-808, 2018.
Igor Melnyk, Vijil Chenthamarakshan, Pin-Y Yu Chen, Payel Das, Amit Dhurandhar, Inkit Padhi, and Devleena Das. Reprogramming pretrained language models for antibody sequence infilling. In International Conference Machine Learning, 2023.
Suvir Mirchandani, Fei Xia, Pete Florence, Danny Driess, Montserrat Gonzalez Arenas, Kanishka Rao, Dorsa Sadigh, Andy Zeng, et al. Large language models as general pattern machines. In Proceedings of the 7th Annual Conference Robot Learning, 2023.
Diganta Misra, Agam Goyal, Bharat Runwal, and Pin Yu Chen. Reprogramming under constraints: Revisiting efficient and reliable transferability of lottery tickets. arXiv preprint arXiv:2308. 14969, 2023.
Yuqi Nie, Nam Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. time series is worth 64 words: Long-term forecasting with transformers. In International Conference on Learning Representations, 2023.
OpenAI. Gpt-4 technical report, 2023.
Boris N Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. N-beats: Neural ba- sis expansion analysis for interpretable time series forecasting. In International Conference on Learning Representations, 2020.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, gigh-performance deep learning library. Advances in Neural Information Processing Systems, 32, 2019.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, llya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8);9, 2019.
Stephen H Schneider and Robert Dickinson. Climate modeling. Reviews of Geophysics, 12(3): 447-493, 1974.
Yihong Tang, Ao Qu, Andy HF Chow, William HK Lam, SC Wong, and Wei Ma. Domain adver- sarial spatial-temporal network: a transferable framework for short-term traffic forecasting across cities. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pp. 1905-1915, 2022.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et Llama: Open and efficient foundation language models. arXiv preprint 2377 2023.
Maria Tsimpoukelli, Jacob I. Menick, Serkan Cabi, SM Eslami, Oriol Vinyals, and Felix Hill. Mul- timodal few-shot lcarning with frozen language models. Advances in Neural Information Pro- cessing Systems, 34:200-212, 2021.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances Neural Informa- fion Processing Systems, 30, 2017.
12

---

Published as a conference paper at ICLR 2024
Ria Vinod, Pin- Yu Chen, and Payel Das. Reprogramming language models for molecular represen- tation learning. In Annual Conference on Neural Information Processing Systems, 2020.
Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan, and Liang Sun. Transformers in time series: A survey. In International Joint Conference on Artificial Intelligence, 2023.
Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, and Steven Hoi. Etsformer: Exponential smoothing transformers for time-series forecasting. arXiv preprint arXiv:2202.0138// 2022.
Haixu Wu, Jichui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition trans-
formers with auto-correlation for long-term series forecasting. Advances in Neural Information Processing Systems, 34:22419-22453 2021.
Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang. and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. In International Conference on Learning Representations, 2023.
Hao Xue and Flora D Salim. Prompt-based time series forecasting: A new task and dataset. arXiv preprint arXiv:22100 10.08964, 2022.
Chao-Han Huck Yang, Yun- Yun Tsai, and Pin- Yu Chen. Voice2series: Reprogramming acoustic models for time series classification. In International Conference on Machine Learning, pp- 11808-11819, PMLR, 2021.
Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. A survey on multimodal large language models. arXiv preprint arXiv:2306. 1120006..3440 13549, 2023.
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proceedings of the AAAI conference artificial intelligence, volume 37, pp. 11121-11128, 2023.
Kexin Zhang, Qingsong Wen, Chaoli Zhang, Rongyao Cai, Ming Jin, Yong Liu, James Zhang, Yuxuan Liang, Guansong Pang, Dongjin Song, al. Self-supervised learning for time series analysis: Taxonomy, progress, and prospects. arXiv preprint arXiv:2306. 10125, 2023.
Tianping Zhang, Yizhuo Zhang, Wei Cao, Jiang Bian, Xiaohan Yi, Shun Zheng, and Jian Li. Less is more: Fast multivariate time series forecasting with light sampling-orien orrennte mip structures. arXiv preprint arXiv:2207.01 0 186, 2022a.
Xiang Zhang, Ziyuan Zhao, Theodoros Tsiligkaridis, and Marinka Zitnik. Self-supervised con- trastive pre-training for time series via time- freuueryy consistency. Advances in Neural Informa- fion Processing Systems, 55:3988-4005 2022b.
Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang- Informer: Beyond efficient transformer for long sequence time-series forecasting, In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 11106-11115, 2021.
Tian Zhou, Ziqing Ma. Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International Conference on Machine Learning, pp. 37268-27286 PMLR, 2022.
Tian Zhou, Peisong Niu, Xue Wang, Liang Sun, and Rong Jin. One fits all: Power general time series analysis by pretrained Im. Advances in Neural Information Processing Systems, 36, 2023a.
Yunyi Zhou, Zhixuan Chu, Yijia Ruan, Ge Jin, Yuchen Huang, and Sheng Li. ptse: A multi- model ensemble method for probabilistic time series forecasting. In The 32nd International Joint Conference on Artificial Intelligence, 2023b.
13

---

Published as a conference paper at ICLR 2024
A MORE RELATED WORK
Tpec-ip Learning. We furnish an extension of the related work on task-specific learning, focusing particularly on the most related models to which we made comparisons. Recent works improve Transformer (Vaswani et al. 2017) for time series forecasting by incorporating signal pro- cessing principles like patching, exponential smoothing, decomposition, and frequency analysis. For example, PatchTST (Nie et al. (203) segments time series into patches input tokens to Trans- former. This retains local semantics, reduces computation/memoryy for attention, and allows longer history. It improves long-term forecast accuracy over other Transformer models. It also achieves excellent performance on self-supervised pretraining and transfer learning. ETSffrmmrr (Woo et al., 2022) incorporates exponential smoothing principles into Transformer attention to improve accuracy and efficiency. It uses exponential smoothing attention and frequency attention to replace standard self-attention. FEDformer (Zhou et al., 2022) combines Transformer with seasonal-trend decompo- sition. The decomposition captures the global profile while Transformer captures detailed structures. also uses frequency enhancement for long-term prediction. This provides better performance and efficiency than the standard Transformer. Autoformer (Wu et al., 202T) uses a decomposition ar- chitecture with auto-correlation to enable progressive decomposition capacities for complex series. Auto-correlation is designed based on series periodicity to conduct dependency discovery and rep- resentation aggregation. It outperforms self-attention in efficiency and accuracy.
Although these methods enhance efficiency and accuracy compared to vanilla Transformer, they mostly designed and optimized for narrow prediction tasks within specific domains. These models typically trained end-to-end on small, domain-spectie speciific datasets. While achieving strong perfor- mance on their target tasks, such specialized models sacrifice versatility and generalizability across the diverse range of time series data encountered in the real world. The narrow focus limits their applicability to new datasets and tasks. To advance time series forecasting, there is a need for more flexible, widely applicable models that can adapt new data distributions and tasks without exten- sive retraining. Ideal models would learn robust time series representations that transfer knowledge across domains. Developing such broadly capable forecasting models remains an open challenge. According to our discussions of related previous work, recent studies have begun to explore model versatility through pre-training and architectural innovations. However, further efforts are needed to realize the truly
forecasting systems that we are advancing in this research.
Cross-modality Adaptation. We provide an extended overview of related work in cross-modalin adaptation, with particular focus on recent advancements in model reprogramming for time ries and other data modalities. Model reprogramming is a resource-efficient cross-domain learning approach that involves adapting well-developed, pre- ore-trained model from domain (source) to address tasks in a different domain (target) without the need for model fine-tuning, even when these domains are significantly distinct, as noted by Chen (022). In the context of time series data, Voice2Series (Yang et al.) 2021) adapts an acoustic model from speech recognition for time series classification by transforming the time series fit the model and remapping outputs to new labels. Similarly, LLMTime (Gruver et al. al., (202)) adapts LLMs for zero-shot time series forecasting, focus- ing on the effective tokenization of input time series for the backbone LLM, which then generates forecasts autoregressively. Diverging from these methods, TIME-LLM does not edit the input time series directly. Instead, it proposes reprogramming time series with the source data modality along with prompting to unleash the full potential of LLMs versatile forecasters in standard, few-shot, and zero-shot scenarios. Other notable works in this field, mostly in biology, include R2DL (Vinod let al. 2020) and ReproBert (Melnyk et al. 2023), , which reprogram amino acids using word em- beddings. A key distinction with our patch reprogramming approach is that, unlike the complete set of amino acids, time series patches do not form complete set. Thus, we propose optimizing small set of text prototypes and their mapping to time series patches, rather than directly optimizing large transformation matrix between two complete sets, such as vocabulary and amino acids.
B EXPERIMENTAL DETAILS
B. IMPLEMENTATION
We mainly follow the experimental configurations in (Wu al. 2023) across all baselines within a unified evaluation pipeline in http::/:/gtttttttttt /Timm Series-r.tts Litt-iirrreee for
14

---

Published as a conference paper at ICLR 2024
fair comparisons. We use Llama-7B (Tovrron al. 2023) the default backbone model unless stated otherwise. All our experiments are repeated three times and we report the averaged results. Our model implementation is PyTorch (Paszke et al. 2019) with all experiments conducted on NVIDIA A 100-800 GPUs. Our detailed model configurations are in Appendix B.4, and our code is made available at / tww.tthuub... com/ / K mMeen Time- LLM
Technical Details. We provide additional technical details of TIME-LLM in three aspects: (1) the learning of text prototypes, (2) the calculation of trends and lags in time series for use in prompts, and (3) the implementation of the output projection. To identify a small set of text prototypes E' E I DD from E E IR VxD,, we learn matrix W E RVVV as the intermediary. To describe the overall time series trend in natural language, we calculate the sum of differences between con- secutive time steps. A sum greater than 0 indicates an upward trend, while a lesser sum denotes a downward trend. In addition, we calculate the top-5 lags of the time series, identified by comput- ing the autocorrelation using fast Fourier transformation and selecting the five lags with the highest correlation values. After we pack and feedforward the prompt and patch embeddings ( E R Px through the frozen I.LM,, we discard the prefixal part and obtain the output representations, denoted as O' E RPDx Subsequently, we follow PatchTST (Nie et al. 2023) and flatten O' into 1D tensor with the length x D, which is then linear projected as Y IR
B.2 DATASET DETAILS
Dataset statistics are summarized in Tab. 8 We evaluate the long-term forecasting performance on the eell-established eight different benchmarks, including four ETT datasets (Zhou et al. | (202T) (i.e..
ETTh1, ETTh2, ETTml, and ETTm2), Weather, Electricity, Traffic, and ILI from (Wu et al. 2023).
Furthermore, we evaluate the performance of short-term forecasting on the M4 benchmark (Makri-
dakis et al. 2018) and the quarterly dataset in the M3 benchmark (Makridakis & Hibon 2000).
Table 8: Dataset statistics are from Ww et al. 2023). The dimension indicates the number of time series (i.e., channels), and the dataset size is organized in (training, validation, testing).
Tasks
Dataset
Dim.
Series Length
Dataset Size
Frequency
Domain
ETTml
7
(96, 192, 336, 720)
(34465, 11521, 11521)
15 min
Temperature
Long-term
ETTm2
7
|
(96, 192, 336, 720)
(34465, 11521, 11521)
15 min
Temperature
Forecasting
ETTh1
7
[96, 192, 336, 720)
(8545, 2881, 2881)
hour Temperature
ETTh2
7
(96, 192, 336, 720)
(8545, 2881, 2881)
hour Temperature
Electricity
321
{96,, 192, 336, 720)
(18317. 2633, 5261)
hour Electricity
Traffic
862
(96, 192, 336, 720)
(12185, 1757, 3509)
hour
Transportation
Weather
21
(96, 192, 336, 720)
(36792, 5271, 10540)
10 min
Weather
ILI
7
(24, 36, 48, 60)
(617, 74, 170)
I week
IlIness
M3-Quarterly
1
8
(756, 0, 756)
Quarterly
Multiple
M4- Yearly
6
(23000, 0, 23000)
Yearly
Demographic
M4 -4arttryyyyy
1
8
(24000, 0, 24000)
Quarterly
Finance
Short-term
M4-Monthly
1
18
(48000, 0, 48000)
Monthly
Industry
Forecasting
M4 Weakly
I
13
(359, 0, 359)
Weakly
Macro
M4-Daily
I
I
14
(4227, 0, 4227)
Daily
Micro
M4-Hourly
1
48
(414, 0, 414)
Hourly
Other
The Electricity Transformer Temperature (ETT; An indicator reflective of long-term electric power deployment) benchmark is comprised of two years of data, sourced from two counties in China, and is subdivided into four distinct datasets, each with varying sampling rates: ETThl and ETTh2, which are sampled at a 1-hour level, and ETTml and ETTm2, which are sampled at a 15-minute level. Each entry within the ETT datasets includes six power load features and target variable, termed "oil temperature". The Electricity dataset comprises records of electricity consumptionn from 321 customers, measured at 1-hour sampling rate. The Weather dataset includes one-year records
15

---

Published as a conference paper at ICLR 2024
from 21 meteorological stations located in Germany, with sampling rate of 10 minutes. The Traffic dataset includes data on the occupancy rates the freeway system, recorded from 862 sensors across the State of Califormia, with sampling rate of hour. The influenza-like illness (ILI) dataset contains records of patients experiencing severe influenza with complications.
The M4 benchmark comprises 100K time series, amassed from various domains commonly present in business, financial, and economic forecasting. These time series have been partitioned into six distinctive datasets, each with varying sampling frequencies that range from yearly hourly. The M3-Quarterly dataset comprises 756 quarterly sampled time series in the M3 benchmark. These series are categorized into five different domains: demographic, micro, macro, industry, and finance.
B.3 EVALUATION METRICS
For evaluation metrics, we utilize the mean square error (MSE) and mean absolute error (MAE) for long-term forecasting. In terms of the short-term forecasting on M4 benchmark, we adopt the sym- metric mean absolute percentage error (SMAPE), mean absolute scaled error (MASE), and overall weighted average (OWA) as in N-BEATS (Oreshkin et al. 2020). Note that OWA is specific metric utilized in the M4 competition. The calculations of these metrics are as follows:
1 T
H
MSE =
ΣY - Y h=1
Σ MAE
Y
Yul
H
H
h=1
SMAPE 200
YA YR
MAPE 100
Y
Yal
H
Y + A
Y
Σ
H
Y Ynl +1 Y, y-
SMAPE
MASE MASENavee?
MASE H
OWA 2 SMAPENaive?
h=1
H-
where s is the periodicity of the time series data. H denotes the number of data points (i.c., predicciiiion horizon in our cases). Y, and Y, are the h-th ground truth and prediction where h [1..... .....,,)) H).
B.4 MODEL CONFIGURATIONS
The configurations of our models, relative to varied tasks and datasets, are consolidated in Tab. 9) By default, the Adam optimizer (Kingma & Ba (0155) is employed throughout all experiments. Specifically, the quantity of text prototypes V is held constant at 100 and 1000 for short-term and long-term forecasting tasks, respectively. We utilize the Llama-7B model at full capacity, maintain- ing the backbone model layers at 32 across all tasks the number of time steps present in the original input time series data. Patch dimensions dm rep- resent the hidden dimensions of the embedded time series patches prior reprogramming. Lastly, heads K correlate to the multi-head cross-attention utilized for patch reprogramming. In the four rightmost columns of Tab. 9 we detail the configurations related model training.
standard. The term input length T signifies
Table 9: An overview of the experimental configurations for TIME-LLM. "LTF" and "STF" denote long-term and short-term forecasting, respectively.
Task-Datneet Configuration
Model [lyperparancter
Traiing Process
Test Prototype V
Dackbone Layers
Input Length
7
Pacch Dim. de
Heads K
LR"
Loss
Batch Size
Epochs
LTF--FTTA
1000
32
512
16
8
10
MSE
S0
ITTP- ETTIS
1000
32
512
16
8
10
MSE
50
ETiml
512
10
100
512
LTF Weather
1000
512
16
MSE
100
LTP Elecnisity
1000
32
512
16
8
10
MSE
8
100
LIF Tratfie
1006
32
512
16
8
10"2
MSE
100
10
STF MI-Qunttsly
50
STF M4
100
32
2 H
32
10
SMAPE
32
11 represents the forecasting borizon of the M4 and M3 dutasets.
L.R. micass the intiial learning
16

---

Published as a conference paper at ICLR 2024
C HYPERPARAMETER SENSITIVITY
We conduct hyperparameter sensitivity analysis focusing the four important hyperparameters within TIME-LLM: namely, the number of backbone model layers, the number of text prototypes V/ the time series imput length T, and the number of patch reprogramming cross-attention heads K. The correlated results can be found in Fig. 6, From our analysis, we derive the following observations: (1) There a positive correlation between the number of Transformer layers in the backbone L.M and the performance of TIME-LLLM, affirming that the scaling law is preserved post- LLM reprogramming. (2) Generally, acquiring more text prototypes enhances performance. We hypothesize that a limited number of prototypes V' might induce noise when aggregating language cues, consequently obstructing the efficient learning of highly representative prototypes essential for characterizing the imput time series patches; (3) The input time length T exhibits direct relation with forecasting accuracy, particularly evident when predicting extended horizons. This observation logical and is in congruence with conventional time series models; (4) Increasing the number of attention heads during the reprogramming of input patches proves to he advantageous.
Analysis on Number of Llama Layers
Analysis on Number of Text Prototype
predict-220
predict-720
predict-336
predict-336
predict-192
predict-192
predict 96
predict 96
0.5
MSE
03
500
Liama Layers
Number Prototype
Analysis on Input Length
Analysis on
predict 720
0.4
predict-720
predict-336
predict-336
predict-192
predict-192
prodict 96
predict-96
0.5
0.4
0.4-
96
192
336
512
0.3
Input Langth
Number of Rerogramming Attention Heads
Figure 6: Analysis of hyperparameter sensitivity on ETTh1 dataset.
D LONG-TERM AND SHORT-TERM FORECASTING
D.1 LONG-TERM FORECASTING
By solely reprogramming the smallest Llama model while keeping it intact, TIME-LLM attains SOTA performance 36 out of 40 instances across eight time series benchmarks. This underscores the considerable potential of LLMs robust and reliable time series forecasters. Furthermore, benchmark the proposed method against other well-established baselines in Tab. 11 This compar- ison includes three notable statistical methods (AutoARIMA, AutoTheta, and AutoETS) (Herzen et al. 2022) and two recent time series models, N-HiTS (Challu et al.) (023b) and N-BEATS (Ore- shkin 2020). Remarkably, TIME-LLM secures SOTA performance across all cases, surpassing the second- best results by significant margins of over 22 % and 16 % in terms of MSE and MAE.
17

---

Published as a conference paper at ICLR 2024
Table 10: Full long-term forecasting results. We set the forecasting horizons H E (24, 36, 48, 60) for ILI and {96, 192, 336, }20} for the others. A lower value indicates better performance. Red: the best, Blue: the second best.
Methoda
TIME-LM
GPT4TS
Dlisess
PudTTST
TinesNe:
FEDiormer
Autidoeneer
Stationrry
ETStemee
LighTTS
Informer
Reformer
Metno
MMSB
MAB
MSL
MML]
MSU
MAB
MSE
MAE]
MSE
MSB
MAE:
MSB
MAB
MSSE
MAE:
MSG
MAE
MSB
MAL
96
1.362
0.392
0.376 0.30
0.335
0.399
0.320
0.999
10.84
0,402
0.376
0.419
[1.49
0.459
0.513
0.491
0.404
0.47%
(04044
0.432
0.365
0.713
0.837
0.218
192
0.39%
0.418
0/416
0448
0.455
0.416
0.413
0:421
0.436
0-429
0/420
0.448
0.500
0.482
0.534
0.504
0.53%
0.504
0.475
0.462
1.088
0.792
0.923
0.266
336
11:200
01427
0.442
0.43
0.09
0.443
0.423
0.436
0.891
0.409
0.459
0.465
0.521
0.490
0.598
0.535
1.574
0.521
0.51
0.389
1.107
0.509
1.097
01855
730
[1,412
11457
01477
0.655
0-412
0.400
0.66
0.521
0.500
0,505
0.507
0.536
0.512
0/641
0.615
0.562
0.535
03547
0.533
1.181
0.865
1.257
0189
AvK
[1:008
0.423
0.465
0.455
0.402
0:457
0.113
0.430
0.456
0.450
0.440
0.460
0.496
0.487
0.570
0.537
0.542
0.510
0.491
0.479
1.040
0.795
1,029
0.805
ETTw2 F73mm 67792
192
16
0.268
1.339
01318
01378
00.255
0.194
0.30
0.542
0.288
0.353
0:415
0.224
0.596
0.379
0.540
0.578
1A114
0.558
0/425
0.61
0.957
0.34
(1.56
0.388
0.452
0.512
0.476
01.5%
01699
11:300
0.430
0.391
0.4%
00:997
00.00
0.337
0.900
3.755
5.00
1,525
1.99
2,026
11:12
2.90
1.317
0,368
0.402
0.373
0.407
0.448
0:300
0.62
0,452
0.496
0.487
0.479
0.626
0.559
4.721
1.835
9.323
2.769
0,372
0.430
0.405
0,441
8,655
0,551
0:22
0.652
0,668
0.463
0.474
0.515
0.511
0.563
0.672
3.647
1.625
3,574
1.697
1V6
[1.34
0.353
0.381
0.412
[1431
0.446
0.414
0.427
0:437
0.44P
0.450
0.459
0.516
0:452
00000
0.545
4.451
1.729
5.736
2.191
0.310
0.358
0.352
0,574
0.357
0.400
0.407
0.795
0.609
0.658
0.592
0.352
0.384
0.265
0.381
0.430
0.411
0:445
0.459
0.435
0.435
0.338
1.212
0,89%
0.721
730
0.353
0.411
0411
0.431
0.435
0451
0.416
0400
0.478
0-450
0.543
0.490
0.671
0.561
0.585
0.516
0.409
0.462
0.507
0.302
1.166
0.523
1.102
0.841
0.329
0.372
0,388
0.403
0.357
0.37%
13
0.300
0.400
0.406
0:448
0:452
0.588
0.517
0.481
0.4.6
0.429
0.425
0.435
0.437
0.961
0.734
0.799
0.671
165
[.161
0.253
0.173
0.262
0.167
0.209
0.165
0.255
0.157
0.267
0,203
0.287
0.255
0.333
0.192
0.274
11119
0.290
0,209
0.308
0.365
0.453
0.658
0.619
192
1.219
01299
0.229
00301
11718
0.301
0.220
0.91
0.349
1.399
0.269
0.328
0.261
01340
0.280
0.33
[1255
0:319
0.311
0.382
0.533
0.563
1.07%
0887
336
[.271
0.229
02265
0.811
0.342
0.274
0.329
(321
0.351
0.325
0.366
0.339
0.372
0.384
0.56
0.314
0.357
04442
0:666
1.363
0.587
1,549
0.072
720
1.252
0.277
0.378
0
0.357
0/421
0.362
0.385
0,408
0u403
0.421
0.455
0.43
0.422
0.413
04413
04414
0.413
0.675
0.557
3.379
1.356
2.631
1.242
Avg
[1.251
0,313
00:294
0. 6.
6
0.351
0.755
0.315
(1291
01355
0.105
0 0.549
0.371
0.110
01:44
(1.295
0:343
(0400V
01-16
1:410
0.510
1.479
0.915
fee Eletroctt Weatter
[.147
0.201
0,162
0.17
0.149
0.195
0.172
0.220
0:217
0.296
(.266
0.173
0.107
(0:182
0.342
0:300
0:384
0.689
0.5%
192
P.189
0.234
0.204
0.248
0
0.241
0.29
0.261
0.276
0.336
0.307
0:245
0.237
0.312
0.227
0.257
0.500
0.544
0.752
0,638
326
1,262
0,277
0.254
0.286
0265
0.319
0.245
0,22
0,200
0,306
0.339
0,380
0.359
0.282
0.334
0.577
0.523
0.639
0.586
1.304
01336
Avg
[1.255
0.287
[
(.255
160
0.312
0.656
96
0.332
0.108
0.308
002007
0.307
0.274
0.365
0,312
0.402
192
IU152
0.541
0.151
0.251
0.15% 0.349
0.147
0.346
1.184
0.760
0:201
0.515
0.212
0.354
0.181
0.286
0.109
0.315
01113
0.316
0.306
0.386
0.54
0.45
336
[.160
0.348
0.169
0.266
0.169
0.267
0.16
0.390
0.300
0.214
0.329
0.231
0.358
0.200
0.504
0.212
0.320
0:230
0.315
0.300
0.794
0.350
0.43
730
1.192
0.298
0.206
0.207
0.303
0.301
0.107
0.296
0.250
0.320
0.246
0.355
0.254
0.361
0.223
0.521
0.233
0.345
0.265
0.360
0.373
0.450
0.340
0:420
Arg
[8158
0.253
00116
0.265
[1
11106
0.263
0.161
0.253
0.192
0.299
0,314
0.327
(1.227
02333
0.191
0.299
11.200
0.321
0.222
0.32M
0.311
0,397
0.33%
0.422
96
1.362
01244
0.388
0.282
0.410
0.282
0.360
0.249
0.50%
0.321
0.587
0.566
0.613
0.385
0.613
0.338
[0.607
0.393
(0.615
0.391
(0.739
0.39)
0.792
0.423
192
1.274
0.247
0.407
0.200
0.433
0.287
0.379
0.256
0.617
0.336
0.604
0.373
0666
0.382
0.613
0.540
0.621
0.390
0.601
0.382
0.606
0.370
0.735
0.420
330
1.305
01211
0.412
0.29
0.416
0.246
0.264
0.629
11336
0.621
0.383
0.222
0.337
0.61
0.328
0,222
0.390
0.613
0.386
0.797
0.420
0.242
11:20
220
1.430
0.288
01490
0.512
466
0:315
0.431
0:286
0.640
11350
0.626
0.
0.418
0.681
0.62
0.396
0.686
0.407
0.864
0:472
0.755
0.425
Avg
0.358
0.764
01414
0.284
0
0:395
0.363
0.600
0.3
0.610
0
0.175
0.540
0.396
0.673
0.992
0.764
0.416
0.241
0.43
0.737
2.003
2.317
3:224
5.764
1,677
7.300
1,531
0.854
0.957
0.735
0.928
1006
1.955
5.264
1.564
4.892
1.485
0.801
169
0.7M7
0.931
7,382
2.005
5.137
1.564
4.724
1.445
I"Couut
36
12
0
0
0
D.2 SHORT-TERM FORECASTING
Our complete results on short-term forecasting are presented in Tab. 12 TIME-LLM consistently outperforms the majority of baseline models in most cases. Notably, we surpass GPT4TS by large margin (e.g., 8.7 % overall, 13.4% on M4-Ycarly, and an average of 21.5% on M4-Hourly, M4-Daily, and M4-Weekly), as well as TimesNet (e.g.. 10% overall, 14.1% % on M4-Yearly, and an average of 30.1% % on M4-Hourly, M4-Daily, and M4 M4-Weekly). Compared to the recent state- of-the-art forecasting models, N-HITS and PStchTST, TIME-LLM exhibits comparable or superior performances without any parameter updates on the backbone LLM.
In addition, we conduct a comparative analysis between TIME-I.LM and the top-performing models on the M3-Quarterly dataset, with the findings presented in Tab. 13 We provide additional metrics, namely MRAE and MAPE, alongside the default SMAPE used in the M3 competition. On this dataset, TIME-LLM attains on-par performance compared to TimesNet and PatchTST, outperform- ing GPT4TS by substantial margins, achieving reductions of over 23% 35%, and 26 % in SMAPE, MRAE, and MAPE, respectively.
E FEW-SHOT AND ZERO-SHOT FORECASTING
E. FEW-SHOT FORECASTING
Our full results in few-shot forecasting tasks are detailed in Tab. 14 and Tab. 15 Within the scope of 10% few-shot learning, TIME-LLM secures SOTA performance in 32 out of 35 cases, spanning seven different time series benchmarks. Our approach's 's advantage becomes even more pronounced in the context of 5% few-shot scenarios, achieving SOTA results in 21 out of 32 cases. We attribute this to the successful knowledge activation in our reprogrammed L.I.M
18

---

Published as a conference paper at ICLR 2024
Table 11: Additional comparison with other baselines in long-ter forecasting tasks. We the forecasting horizons E (24, 36, 48, 60) for ILI and (96, 192, 336, 720} for the others. A lower value indicates better performance. Red: the best, Blue: the second best.
Methods
Time-
LLM
N-BEATS
N-RITS
AMRARRIMA
AnnoThes Theta
AuseLTT
Muticc
MSR
MAF
MSF
MAF
MSE
MAF
MSR
MAF
MSB
MAE
MSF:
MAF
96
0.362
0.392
0.496
0.475
0.292
0.407
1.933
0.635
1,256
0.758
1,264
0.756
192
0.398
0.418
0.514
0.504
0.868
0.621
1.18%
0.249
1.881 0,745
0.430
0.64
0.653
0.799
0.442
0.679
1.043
0.705
1.510
0.82
1,405
Aug
0.401
0.636
1.319
0.757
k2 F77ml E222
0.754
0.771
0.461
0.365
0.019
0.560
1,355
1,526
0.522
0.372
0.620
11987
0.638
3.91
1061
0.835
0.334
0.903
0.520
0.664
0.512
1,535
0.678
[.96]
0. 0,505
96
0.272
0.334
0.393
0.412
0.327
0.568
1.091
0.651
1.211
0.704
L.319
0.765
192
01.11
0.85%
11:15
0.427
0.176
0.000
1.119
0.682
1,237
0.734
1.535
0.784
K36
0.382
0.004
01664
0.454
0.407
0.43%
1.125
0.09
1,231
0.73
1.47%
0.782
720
0,383
0.411
0.521
0.485
0.471
0.436
1.243
0.245
1.394
0.001
1.591 0.825
Ang
0.339
0.573
(0.45
0.415
0.412
1.145
0.697
1,208
1.741
1.529
0.790
0.160
0.088
0.22%
01435
0.37
00.455
0.316
0.
0.219
0:224
0.0%
0413
0.00
0.756
0. 595
0.271
0.6h
0.790
11523
1,247
0.467
0.363
10:0
2:51
13877
6,550
0.839
Ang 0.2551
0.014
01975
0.575
7,400
0,489
Fio Elodkiniiy Westher
0.144
0.201
0.351
0.257
0.389
0.234
(1390
0.555
0,317
0.316
0.408
0.261
0.27%
0.321
8:381
1177%
0.557
0.472
0.388
0.308
0.304
0.316
0.340
0.375
0:33
0:50
2389
0.707
0.818
0.536
2,520
0,580
0.225
0.257
0.225
0.300
0.33
0.24
1.000
0.448
0U-77
0.373
1.337
0.405
0.151
0.224
00215
0.327
0 1844
0.215
00:530
0.966
0.655
125
0,000
0,575
192
0.182
0.341
(1.466
D. 0.340
(3581
0.299
0.713
0.561
0.904
0.549
336
0.160
0.348
0.252
0.255
00:05
0,002
0.515
0.797
1U603
0.766
0.577
720
0,192
0.
0.290
0.203
0.685
0.50%
1.003
11,88
0.901
0.528
Aug
0.158
0.20
0.300
0.500
0.797
0.50m
0.755
0.570
0.363
11.008
0.52
0.09%
3,207
1.219
3.254
0,374
1.300
0.775
3,407
1.202
3.509
0.500
1.274
3.971
0.430
3,052
0.765
3,510
1.285
0.727
6.80
1.02%
1.04
4.900
1.339
5,991
1.510
4,309
36
1.404
0.814
6.850
1.300
1.60
1.104
5.079
1.400
5.922
1.539
4.17
1.422
1.523
0.817
6.758
1.376
3.93
1.176
277
1.339
4.637
1.329
3.966
1.30]
60
1.531
0.864
6.908
1999
1.293
5.885
1.276
4.77%
1.345
3.540
1.220
Aug
1,435
0
6.839
1 1.580
3.007
1.171
4.500
1.346
5,232
1.431
4.323
1.317
1'Oment
49
0
0
0
Table 12: Full short-term time series forecasting results. The forecasting horizons are in 16, 48] and the last three rows are weighted averaged from all datasets under different sampling intervals. A lower value indicates better performance. Red: the best, Blue: the second best.
Mithads
TIME-III M.M
GPTT-TS
TimesNet
PascITST
N-HITS
N-REATS
ETS/omer
LighTS
incar FRDlumer
Statinney
Autoformer
Informer
Refomer
Yeary
SMAPE
13.419
15.378
13:477
13:422
13,487
18,029
14.247
16.965
13.717
13,974
14.227
16.169
MASE
3,005
3,565
3.554
3.019
3,056
3,008
4.487
3,109
4.283
3,006
3.078
3.154
3.418
3,800
OANIL
0.789
0
0.918
3.792
0.795
0.795
1.115
0.827
1.058
0.811
0.807
0.822
0
0,973
Quanety
SMAPE
10.110
10.597
10.965
10.38
10.185
10.564
13.376
11.364
12.145
11.1
10.958
11,338
11.300
13,313
MASE
1.178
1,253
1233
1,252
1996
1,328
1,520
1,325
1365
1,401
1.775
OWA
0.889
0.938
0.23
0921
0.895
0.938
1330
1.000
1,106
0.996
0.981
1.01
1.027
1.252
Monthly
SMAPE
12.002
132258
13.513
12.999
13,000
13,089
14.588
14.014
13.514
14,005
13:917
13,958
14.033
20.128
MASE
0.963
1.005
1.099
0.91
1.015
0.958
1368
1.05
1.007
1.147
1.097
1.100
1.141
2,614
0.902
0.951
0.956
1.907
Otbers
SMAPE
4.795
4952
6.599
7.267
15,850
24,460
32,491
MASE
3.178
4.116
4.500
3.347
3.054
4.43
5:340
11.434
4.953
4.041
4.064
3.865
20,960
33.355
OPAIA
1,066
1,259
14438
1.049
0.977
1,393
1.591
3.474
1,487
1,389
1,304
1.157
5,879
8.679
Averia
SMAPE
11.983
12.69
12.88
12.059
12.085
12:35
14.718
13.525
13,699
13.36
12,780
12.909
14186
18,300
MASE
1.995
1663
1.698
2.408
2.111
2,095
1.775
1.756
1.771
4.223
OWA
0,859
0.955
0.862
0.896
1.172
1.051
1.051
0.949
0.930
0.930
1.330
1.775
E.2 ZERO-SHOT FORECASTING
The full results of zero-shot forecasting are summarized in Tab. 16, TIME-LLM remarkably sur- passes the six most competitive time series models in zero-shot adaptation. Overall, we observe over 23.5 % and 12.4% MSE and MAE reductions across all baselines on average. Our improve- ments are consistently significant on those typical cross-domain scenarios (c.g.. ETTh2 - ETTh1 and ETTm2 + ETTml), over 20.8% and 11.3% average w.r.t. MSE and MAE. Significantly, TIME-LLM exhibits superior performance gains in comparison to LLMTime (Gruver et al. 2023), which employs similarly sized backbone L.LM (7B) and is the latest effort in leveraging M.Ms for
19

---

Published as a conference paper at ICLR 2024
Table 13: Additional short-term time series forecasting results on M3 (Quarterly). The forecasting horizon is lower value indicates better performance. Red: the best, Blue: the second best.
Methods
TIME- LLM GPT4TS TimesNet PatcTTST N-HTS N-BEATS DLinear FEDformer
SMAPE
11.121
14,453
10.410
12:380
12,616
18.640
15,028
12:927
MRAE
3.282
5,335
3,310
2.401
4.271
4,612
2.793
3,633
MAPE
0.151
0.203
0.140
0.154
0.168
0.247
0.196
0.174
zero-shot time series forecasting. We attribute this success to our reprogramming framework being better at activating the LLM's knowledge transfer and reasoning capabilities in a resource-efficient manner when performing time series tasks.
F ABLATION STUDY
We additionally compare the model performance under reprogramming and fine-tuning (with Q Dettmers et al. (2023)) protocols. Our results indicate clear performance gain of our approach compared to the QLoA variant (A.5) by 19% in average.
The full ablation results are in Tab. 17
G EFFICIENCY COMPARISON WITH MODEL FINE-TUNING
Setups. We compare the efficiency of model fine-tuning (with QLoRA Dettmers et al. (2023) and our proposed model reprogramming in this section with two different backbones, that is, Llama in 1/4 capacity (first Transformer layers) and full capacity. Here, we adhere to the long-term forecasting protocol on ETTh1 to forecast two different steps (that is, 96 and 336 in this case) ahead. For the evaluation metrics, we report the total number of trainable parameters (in million), GPU memory (in mebibyte), and running time (seconds per iteration).
Results. Our results are given in Tab. 18 We see that model reprogramming remarkably results in better efficiency compared to parameter-efficient fine-tuning (PEFT) with QLoRA on long-range
Table 14: Full few-shot learning results on 10% training data, We use the same protocol as in Tab.
Methodi
TIME-LL
GPT-TS
DLineer
PushTST
TimesNe:
FEDiormer
AAttddemer
Stationary
ETStommer
LighTSS
Infoemer
Reformer
Metno
MMSB:
MAB
MSL
MAE
MSS
MABi
MSE
MAB|
MSE
MAU
MSE
MAL:|
MSB
MAN
MNE:
MAE
MSB
MBB
MSS
MAE
MSB
MAE
MAB
96
1.448
[.4Ml
0.45%
0.466
0.692
0-405
0.316
0-485
(1861 0.628
0.313
0.400
[a618 0.552
0:015
0.639
1.112 0.506
1:208
0.838
1.179
0.793
1.184
0.290
192
1.384
0.483
0.570
0,516
0.565
0.538
0.598
0.534
0.757
01593
0.624
0.55
0.722
0.558
0.915
0.529
1.155
0.823
1.322
0.85%
1.199
0.806
1.255
01850
536
0.589
0.500
0.005
0,535
0.721
0822
0.87
0.500
(191
0.688
0.691
0.544
0.19
0.939
8.646
1.179
0.332
1.347
0.700
1.200
0.81
1.294
0.854
730
0.700
0223
0.966
0.945
0.70
0.60
(187
0.641
0.728
0.64
87711
0.100
0.881
0.685
1.275
0.374
1.554
0.942
1.217
0.825
1.215
0880
AvE
0.556
0.691
0.600
0.633
0.542
0.869
0.628
0.639
0.561
1.150
0.834
1,375
0.877
1.199
0.509
1.349
0.833
g77 67782
0.275
0.357
0.11
0.383
20003
3.837
0.406
0.426
0.537
0.504
3.952
1.526
720
0.427
0.624
0,490
0.510
0,495
0.509
3.842
1.401
0.38
0.479
0.405
1.160
96
0.346
0.388
0.390
0.404
0.902
0.410
0:419
(0.88
0.501
0.574
0.518
0.614
0.766
01 0.566
[1911
0.688
(0:921
0.692
1.162
0:785
1.442 0887
192
1.373
0.416
0.429
0.433
0300
0412
0.437
0.434
0.630
11528
0.617
0.546
0.754
0.592
0.781
0.574
1.955
0.703
0.957
0.701
1.172
0.793
1.444
0.862
336
1.413
11436
0.409
0.439
0419
0.476
0:454
0.725
11558
0,548
0.775
0,35
0,677
0.003
0.587
[1991
0.719
0.996
0.716
1.227
0.905
1.450
0.966
730
1.685
1.416
0.560
0.405
1000
00477
0.681
0.556
0.769
0.549
0.603
0.570
0.800
0.630
0.544
0.581
1.062
0.347
1.007
0.700
1.207
0.797
1.366
0.850
Ave
1.404
07427
0.464
0,441
044
0:400
0.501
0.465
0.677
0.537
0.722
0.605
0.900
0.625
0.797
0.578
0.0000
0.714
0.971
0.705
1.192
0.521
1.426
0.856
ETTm2
1.177
0.261
0.184
0.269
0.213
0.303
0.191
0.274
0.212
0.285
0.291
0.399
0.332
0,454
0.229
0.30%
00331
0.430
00.13
0.688
3.203
1,407
4.195
1.628
192
11241
0211
100
0.278
0.355
0:251
0.317
(1210
0.325
0.307
0.37
(1.99
0.691
0.291
01363
(1:000
0.464
1,008
0.200
5.112
1.389
4,M42
1.601
330
1,274
0.277
0307
0:346
0.338
0.353
0.306
0.351
0.523
0.353
0.543
0.559
2.406
0.376
1,001
0.775
3.355
1.421
3,965
1.585
720
0,417
0.390
0426
0/417
0.436
0.40
0:33
0.427
0.474
0,449
0,712
0.614
1.93
0.461
0.500
0.557
1,096
0.751
3.009
1.543
3,711
1.532
Avg
1.277
0.516
0.706
0.966
0.77%
5.770
fe Elotrvatty Westur
[.16]
0.210
0.224
0.165
0,215
15154
0.255
0.259
0.374
0.335
0.350
0.504
0.548
0.115
0255
0.210
0.257
0.245
0.279
0.332
0.259
0.304
0.552
0.522
0.462
0.561
0.503
0.258
0.299
0.259
0,297
0.305
0.521
0.356
0.386
0.303
0.334
724
0.541
0.715
0.535
730
(.399
00000
0.121
0.150
0.500
04346
0.353
0.346
0.551
0.971
0.967
0.50%
(1500
0.196
0.441
0.405
0.417
0.463
01750
0.55%
0.611
0.500
1.234
0.273
0.558
0.275
0:341
0233
0.242
0:27%
0.279
0.301
0.284
0.504
0.500
0.342
0.315
0.533
0.318
0.360
01280
0.322
0.997
0.495
0.546
0.469
96
8.139
0.241
0.139
0,337
0.150
0.253
0.140
0,238
0.299
0.333
0.231
0,323
0.251
0.348
0.420
3.465
11,59
0.587
00350
0-415
1.250
0.919
0.993
0.784
192
1151
11.44
0.156
0:251
01164
02264
0.160
0.255
0.379
0.261
0.356
11358
0.406
0
0.459
1.630
0.504
0.136
0.448
1.100
0.575
0.91%
0.75%
336
[1169
0.270
0.125
0.278
0.181
0252
0.180
0.276
0.319
0.301
0.360
0.445
0.400
0.474
0.484
0.475
0.682
0.610
0.426
0.85
1.157
0.572
0.025
0.245
720
(1240
0.322
0.233
0.317
0.225
0:211
0.241
0.323
(1:00
11426
0,530
0,555
0715
0.085
0.510
0.521
01757
0.064
0.61
0.597
1.303
0.596
1.004
117500
Avg
1.175
0:210
01.79
0.269
1880
01280
0.100
0.275
00325
11392
0.346
0.427
(1:31
0.473
0.444
016801
0.600
0,677
0.641
01:89
1.195
0.59)
0,905
11200
[.418
0.201
0.414
0.297
0.419
0.206
0.403
0.289
0.719
0416
0.699
18643
1.157
0.636
1.357
1.537
0,815
0.414
0.206
0.426
0.301
0.434
0.305
0.155
0.296
0.748
0.428
0.637
0.416
1.199
0.354
1.207
0.661
1.454
0.765
1.538
0.817
1.421
0:311
0.315
0.436
0.005
1.521
0.512
1,500
11:111
0.429
0.306
0.310
0.447
0.333
0.305
[.951
0.535
1.245
0.684
1.534
1.551
0.821
1"Coust
2
20

---

Published as a conference paper at ICLR 2024
Table 15: Full few-shot learning results 5% training data. We use the same protocol in Tab. 1 '.' means that 5% time series is not sufficient to constitute training set.
Methods
TIME-LEM
GPTSTS
Diinner
PaidUNT
Timeneet
Ausefommrr
Starienayy
LighTSS
| Informer
Reformer
Matric
MSS
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSS
MAE
MSE
MAE
MSE
MAE]
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
0.883
0.464
0.563
0,505
0.547
0.503
0.557
0:119
(0,992
0.625
(1593
0.529
(0.08]
0,570
0.552
0.600
1.169
0.822
LAR33
0.91
1.225
0.112
1,195
0.795
192
8.02
11.40
07768
0.990
0.720
0000
0.11
0.700
0.940
01605
0.561
00725
0.600
0991
0.605
1.211
0883
1.23
0.99
1.249
1.273
0.883
0.768
0.636
0.984
0.727
0.316
0.619
0.624
0.933
0.644
1.17
0.833
1.347
0.87
0.811
Avg.
0.750
0.563
0.645
ETTml F7712
00,336
0.406
0..99
0.442
0.617
0.542
0.456
0,522
0.455
0.496
0.504
0.468
0.497
0.423
0.465
0.678
0.619
2,002
3.554
3.975
1.473
0.405
1.424
0.349
0,464
0.499
0.483
0.486
0.496
0.507
0.905
3,956
1.427
730
[0.322
0.418
0.400
0.433
0.604
0.577
0.827
0.615
[0,490
0.448
0.446
0.454
(0:44]
0.457
0.470
0.489
0.909
0.681
3.206
1.268
3,922
1.653
3.527
1.472
96
0.316
0,212
0.386
0.405
0,332
0.374
0.399
0.414
000005
0,518
0.633
0.544
(0.726
0.578
0.823
0.587
1,031
0.247
1,043
0.733
1.130
0.775
1.234
0.798
192
[1450
01464
0.440
0.435
0.556
0.300
[.441
0.436
0.681
0.539
(1.666
0.566
0.750
0.591
0.344
0.59
1.087
0.266
1,07
0.786
1.150
0.768
1.387
0.839
336
1:450
0.414
0-485
0.459
0.012
0:486
0:409
0.467
0.786
0.597
0.800
0.625
0.831
(:659
0:500
0.603
1.138
0.287
1:147
0.775
1.198
0.800
1.288
0.842
720
1.453
0,471
0.577
0.499
0.511
0:459
0.767
0.587
0.706
0.595
0.822
0:633
0.837
8:655
0:303
0.611
1.245
0,831
1,200
0.799
1.175
0.704
1.247
0.828
Avg
11.255
004144
004922
0,450
0.010
0:417
0.326
0.301
017790
0.990
(1 0.136
010000
0:857
0.595
1.125
1:123
0,105
1.
0.791
1.264
0.8216
0..14
0.261
0.190
0.280
0.236
0.326
0:306
0.288
0.020
0.209
01.29
0:320
00:22
0:322
0:238
0:316
0.404
0.465
0.772
3,
1,545
9.215
0.257
0:315
0.366
0.733
0:64
0:224
0311
0.394
[.201
0:357
0:208
0.349
0.479
0.521
1.317
0.850
3.578
1.475
1.484
316
1.275
04223
0:334
0.207
0:517
0.399
0,502
1415
1.838
0.883
01637
0..274
1.487
Weader
0.172
0.363
00.15
0.250
0.242
0.171
0.224
0.255
0:228
0.309
0.290
0:215
0.252
0.218
0.295
0.250
0.285
0.897
0.405
0.615
199
1.224
0..31
0.229
0.326
0.22%
0.283
0.230
0.379
0.271
1.907
0.266
0.317
0.226
0.315
0.700
0.707
0.204
0.311
0.314
0.37
0.600
0:545
0.445
0.450
336
12522
0.321
0.386
0.121
0.279
0.32
0.294
0,326
0.313
0.50
0.35%
0.391
00.51
0.905
0:351
0.348
0.359
0.908
0.918
0.355
0.649
0.547
0.465
0.459
730
1.506
0.381
0.366
0,379
0.364
0.355
0.354
0.287
0.400
0.385
0.391
0.294
0,38T
0.389
0.452
0.407
0.451
0.461
0.401
0.415
0,570
0.522
0,471
0.468
Avg
[3360
0130
0.701
0,301
0,203
0.306
0.209
0.2011
00.295
0.318
01.300
0.353
0.310
0.355
0.327
0.325
0,335
0,371
0.205
0,545
00.544
0.527
0,447
0.453
Traffe Eledrioot
11147
11242
0.143
0.241
0.150
01251
0.165
0:244
10.115
0 3.5
(1.355
0:323
0.727
0.367
0-484
0.51%
0.607
0.6%8
0.690
0.909
1
0.99
1/414
0.855
192
1.15%
01241
0.159
0.255
0.163
0.253
0.163
0.200
0.315
0.396
0.247
0.343
0.308
0.375
0.500
0.531
0.718
0.688
0.772
0.675
1.298
0:929
1,230
0.919
330
1
1.277
0.179
0.274
0.175
0.278
0.183
0.280
0.340
0.415
0.267
0.355
0.354
0.374
0.578
0.25%
0.607
0.901
0.745
1,302
0542
1,253
0.921
720
1715
11112
0:281
0.325
0:219
0:311
0:233
0.121
0.613
0:318
0:164
0.426
0.006
0.785
1001
11788
1.200
0.871
1.25
0919
1.209
0.921
Ave
0.271
0.176
0.75
0.181
0.277
0.266
0.346
0.404
0.627
0.503
0.800
0.685
0.378
0.725
1.281
0.291
0.245
0.300
0.404
0.286
0.854
1:408
1.643
0.315
0.122
0.313
0.867
0.523
1.602
0.560
2.000
0,305
0.450
0.317
0,418
0.246
(0.33
0.502
1.526
0.339
1.859
0.927
1.557
0.795
1.591
1"Court
forecasting tasks in terms of the total number of trainable parameters, GPU memory overhead, and training speed. Quantitatively, there is an 71.2% % trainable parameter reduction on average over four scenarios, leading 23.1% % smaller memory consumption and 25.3% faster training speed.
H ERROR BARS
All experiments have been conducted three times, and we present the standard deviations of our model and the runner-up model here. The comparisons between our method and the second-best method, PatchTST (Nie al. 2023), on long-term forecasting tasks, are delineated in [Tab. 19, In this table, the average MSE and MAE have been reported across four ETT datasets, complete with standard deviations. Furthermore, Tab. 20 contrasts the effectiveness of our method with that of the becond-best method, N-HiTS (Challu et al., 2023a), employing varying M4 datasets for the comparison.
1 VISUALIZATION
In this part, we visualize the forecasting results of TIME-LLM compared with the state-of-the- art and representative methods (e.g., GPT4TS (Zhou et al. 2023a), PatchTST (Nie et al.) (023). and Autoformer (Wu et al. 2021h) in various scenarios to demonstrate the superior performance of TIME-LLM.
In Fig. 7 and Fig. 8 the long-term (input-96-prediit-99) and short-term (input-36-predict-36) fore- casts of various approaches compared with the ground truth. Here, TIME-LLM showcases forecasting accuracy that is notably superior compared to GPT4TS, PStchTST, and a classical Transformer-baseee method, Autoformer.
We also offer visual comparisons of the forecasting results in both sew- and zero-shot scenarios, as depicted in Fig. 9ndd Fig. 10 We adhere to the long-term (input-96-preiict-906 forecasting setup
21

---

Published as a conference paper at ICLR 2024
Table 16: Full zero-shot learning results on ETT datasets. A lower value indicates better performance. Red: the best, Blue: the second best.
Methods
TIMB-LLM
LLMTime
GPTITS
DLinear
PatcTTST
TimesNet
Autoformer
Metric
MSE
MAE
MSE
MAE
MSE
MAE]
MSR
MAE
MSE
MAE
MSE
MAE]
MSE.
MAE
96
0.279
0.337
0.510
0.576
0.335
0.374
0.347
0.400
0.304
0.350
0,358
0.387
0:469
0.486
192
0.351
0.374
0.523
0,586
0.412
0.477
0.447
0.460
0,386
0.400
0.427
0.429
0.634
0.567
ETTR E77
336
0.388
0.415
0.640
0.441
0.444
0.515
0.505
0444
0.449
0.451
0.588
720
0.391
0.420
2.296
1.034
0.438
0.452
0.665
0.589
0.419
0.44
0.448
0.458
0.570
0.549
Avg
0.353
0.387
0.992
0.708
0.406
0.422
0.493
0.488
0.300
0.405
0.421
0.431
0.582
0.548
96
0.189
0.293
0.646
0.563
0.236
0.315
0.255
0.357
0.215
0.304
0.239
0.313
0.352
0.432
192
0.237
0.312
0.934
0.654
0.287
0.342
0,338
0.413
0.275
0.339
0:291
0.342
0.413
0,460
ETTM -> ETTm2
336
0.291
0.365
1.157
0.728
0.341
0.374
0.425
0.465
0.334
0.373
0.342
0.371
0.465
0.489
720
0.372
0.390
4.730
1.531
0:435
0.422
0.640
0.573
0.431
0.424
0.434
0.419
0.599
0.551
Avg
0.340
1.867
0.869
0.325
0.363
0.415
0.452
0.314
0.360
0.327
0.361
0.457
0.483
96
0.455
0.452
1,130
0.777
0.732
0.577
0.689
0.555
0.485
0.465
0.848
0.601
0.693
0.569
192
0.465
0.461
1.242
0.820
0.758
0.559
0.707
0.568
0.565
0.509
0.860
0.610
0,760
0.601
ETTR2 - ETTRI
336
0.501
0.482
1.328
0.864
0.759
0.578
0,710
0.577
0.581
0.515
0.867
0.626
0.781
0.619
720
0.501
0.502
4.145
1,461
0.781
0.597
0.596
0.628
0.561
0.887
0.648
0.644
Avg
0.479
0.474
1.961
0,981
0.757
0.578
0.703
0.574
0.565
0.513
0.865
0.621
0.757
0.608
96
0.174
0.276
0.646
0,563
0.255
0.329
0.240
0.336
0.226
0.309
0.248
0.324
0,263
0.352
192
0.233
0.315
0.934
0.654
0.293
0.346
0.295
0.369
0.289
0.345
0.296
0.352
0.326
0.389
ETT22 ETTm2
336
0.291
0.337
1.157
0,728
0.347
0.176
0.345
0.397
0.348
0.379
0.353
0,383
0.387
0.426
720
0.392
0.417
4.730
1,531
0.446
0.429
0.432
0.442
0.439
0.427
0.471
0.446
0.487
0.478
Avg
0,272
0.341
1.867
0.869
0.335
0.370
0.328
0.386
0.325
0.365
0.342
0.376
0.366
0.411
96
0.321
0.369
0,510
0.576
0. 353
0.392
0.366
0.415
0.354
0.385
0.377
0.407
0.435
0.470
192
0.389
0.410
0.525
0.586
0.443
0.437
0.454
0.462
0.447
0.434
0.471
0.453
0.495
0.489
ETTml + ETT12
336
0.408
0.433
0.640
0.637
0.469
0.461
0,496
0.494
0.481
0,463
0,472
0.484
0,470
0.472
720
0.406
0.436
2.296
1.034
0
0.466
0.468
0.541
0.529
0.474
0.471
0.495
0.482
0.480
0.485
Avg
0.412
0.992
0.708
0.433
0.439
0.464
0.475
0.439
0.438
0.457
0.454
0.479
96
0.169
0.255
0.646
0.563
0:217
0,294
0,221
0.314
0.195
0.271
0.222
0,295
0.385
0.457
192
0.318
0.934
0.277
0.327
0.359
0.258
0,31
0.288
0.337
0.489
ETTml + E77mm
336
0.290
0.338
1.157
0.728
0.331
0.360
0.357
0.406
0.317
0.348
0.341
0.367
0.476
0.477
720
0.375
0.367
4.730
0.429
0.413
0.476
0.476
0,416
0.436
0,418
0,582
0.535
Avg
0.268
0.320
1.867
0.869
0.313
0.348
0.335
0.389
0.296
0.334
0,322
0.354
0.469
0.484
96
00.298
0.356
0.510
0.576
0.360
0,401
0,333
0.391
0.327
0.367
0.360
0,401
0,353
0.391
192
0.359
0.397
0.523
0.586
0.434
0.437
0.441
0.456
0.411
0,10
0.434
0.437
0.432
0.437
ETTm2 + ETTA2
336
0.367
0,412
0.640
0.637
0.460
0.459
0.505
0.503
0.439
0.447
0.460
0.459
0.452
0.459
720
0.393
0.434
2.296
1,034
0.485
0.77
0.543
0.534
0.459
0,470
0.485
0.477
0,453
0.467
Avg
0.354
0.400
0.992
0.708
0:435
0.443
0.455
0.471
0.409
0.425
0:435
0.443
0.423
0.439
96
0.359
0.397
1.179
0.781
0.747
0.558
0.570
0.490
0.191
0.437
0.747
0.558
0.735
0.576
192
0.390
0.420
1.327
0.846
0.781
0.560
0.590
0.506
0.530
0.470
0.781
0.560
0,753
0.586
E/112 - 377mml
336
0.778
0.567
0.565
0.778
0.593
720
0.487
0.488
3,749
1.408
0.769
0.573
0.731
0.584
0.686
0.565
0.769
0.573
0.782
0.609
Avg.
0.414
0.438
0.984
0.
0.769
0.567
0.649
0.537
0.568
0.492
0.769
0.567
0.758
0.591
Table 17: Full ablations on ETThl and ETTml in predicting 96 and 192 steps ahead (MSE reported).
Variant
Long-term Forecasting
Few-shot Forecasting
ETT111996
ET9h1-199
ETT
ETThm1-192
ET'Th1-966
ETThh--92
ETTm1-96
ETThm1-192
A.1 Llama (Default: 32)
0.362
0,398
0.272
0.310
0.448
0.184
0,346
0.373
A.2 Llama (8)
0.389
0.412
0.297
0.329
0.567
0.632
0.451
0.490
A.3 (12)
0.385
0.419
0.306
0.332
0.548
0.617
0.447
0.500
A.4 GP'-22 (6)
0.394
0:427
0,311
0.342
0.571
0,640
0,468
0.512
A.5 Llama (QLoRA; 32)
0
0.420
0,310
0.338
0.543
0.611
0.578
0.618
B.1 w/o Patch Reprogramming
0.410
0,412
0.310
0.342
0.498
0,570
0.445
0.487
B.2 w/o Prompt-as Prefix
0.398
0.423
0,298
0.339
0.521
0.617
0.432
0.481
Datasei Contect
0.402
0.417
0.298
0.331
0.491
0.538
0.392
0.447
C.2 who Task Instruction
0.388
0.420
0.285
0.327
0.476
0.529
0.387
0.439
C.3 Statisical Context
0.391
0.119
0.279
0,347
0.483
0.547
0.421
0.461
in both cases. TIME-LLM exhibits remarkable superiority in forecasting with limited data- a fact that becomes particularly salient when compared to GPT4TS.
22

---

Published as a conference paper at ICLR 2024
Table 18: Efficiency comparison between model reprogramming and parameter-cfficien fine-tuning (PEFT)
with QLoRA (Dettmers ct al. 2023 on ETTh dataset in forecasting two different steps ahcad.
Length
ETTh1-96
ETTh1-336
Metric
Trainable Param. (M)
Mem. (MiB)
Speed(s/iter)
Trainable Param. (M)
Mem. (MiB)
Speed(s/iter)
QLoRA
12.60
14767
0.237
12.69
15982
0.335
Llama (8)
Reprogram
5.62
11370
0.184
5.71
13188
0.203
QLoRA
50.29
45226
0.697
50.37
49374
0.732
Llama (32)
Reprogram
6,39
32136
0.517
6.48
37988
0.632
Table 19: Standard deviations of our approach and second-best method (PatchTST) on all time series datasets for long-term forecasting.
Model
TIME-LLM
PatchTST (0033
Dataset
MSE
MAE
MSE
MAE
ETTh1
0.408
0.011
0.423 = 0.012
0.413 0.001
0.430 = 0.002
ETTh2
0.334
0.005
0.383 = 0.009
0.002
0.379 + 0.007
ETTml
0.006
0.372 0.007 0.007
0.351 0.006
0.380 0.002
ETTm2
0.251
0.002
0.313 0.003
0,255
0.315
Weather
0.225 0.009
0.257 = 0.008
0,225
0.264
Electricity
0.158 + 0.004
0.252 - 0.007
0.161 0.0001
0.252
Traffic
0.388 + 0.001
0.264 = 0.006
0.390 0.0003
0.263
ILI
1.435 + 0.011
0,801 = 0.008
1..012
0.797
Table 20: Standard deviations of our TIME-LLM and the second-best method (N-HiTS) M4 datasets for
short-term forecasting.
Model
TIME-LLM
N-HiTS
2023a
Dataset
SMAPE
MAPE
OWA
SMAPE
MAPE
OWA
Ycarly
13.419 0.117 3.005 0.011
13.422 0.009
3.056 1 0.017 0.795 0.010
Quarterly
10.110 = 0.107
1.178 0.009
0.889 0.007
10.185 0.107
1.180 + 0.007 0.893 0.001
Monthly
12.980 + 0.102
0.963
+ 0.005
0.903 + 0.001 0.000
13.059
= 0.101
1.013 # 0.007 0.929 + 0.005
Others
4.795 0.117
3.178
1 0.012
1.006 1 0.009
4.711
0.1117 3.054 1. 0.011 0.997 0.012
Averaged
11.983 1 0.011
1.595 0.021
0.859 0.002
12.035 - 0.111
1.625 1 0.012 0.869 0.005
23

---

Published as a conference paper at ICLR 2024
Orundhic
M wh
M
Mn
-10
-20
[4)Time-LLa)
(N)GPT4TS
Pa)aacTTTT
Autoformer
Figure 7: Long-tern forecasting cases from ETThl by different models under the imput- 96-predict- 96 settings. Blue lines are the ground truths and orange lines are the model predictions.
13000
4400
GalTime L.M
(b)GPTITS
(d)
Figure 8: Short-term forecasting from the M4 dataset by different models under the input-36-predice 18 settings.
-03
-00
GO)GTTTS
PatchTTT
(4)
Autoformer
Figure 9: Few-shot forecasting cases from ETTml by different models under the apput-96-rredict. 96 settings. Blue lines are the ground truths and orange lines are the model predictions.
Pattinn
Go)Time-LLM
GG)GTTSS
(4) Autuformer
Figure 10: Zero-shot forecasting cases from ETTh ETThh by different models under the input- 96-predict-96 settings. Blue lines are the ground truths and orange lines are the model predictions.
24


Reviews:
[{"review_id": "8B8VVcLty5", "reviewer": "Reviewer_L3pe", "summary": "This paper proposes the use of pre-trained large language models for time series prediction. As claimed, the main contributions of the paper include introducing a novel concept of reprogramming large language models and augmenting the input context with declarative prompts, such as domain expert knowledge and task instructions, to guide LLM reasoning.", "questions": "More discussions:\n- Can you discuss the connections and differences between LLMs and traditional/existing deep-learning time series models? \n- It would be helpful to define what is a good time series representation expected to be and to provide a more in-depth discussion of why pre-trained LLMs are capable of producing such representations.\n- Regarding cross-domain adaptation in the zero-shot setup, I'm curious about what knowledge from the source domain in time series can be transferred to the target domain to achieve zero-shot prediction.", "limitations": "", "rating": 3, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- This paper provides a summary of the metrics for pre-trained large-language models, including generalizability, data efficiency, reasoning, and multimodal knowledge.\n- The details of the proposed method are presented clearly and are easy to follow.", "weaknesses": "My main concerns include: \n- While LLM is a hot topic in the deep learning community, it is still unconvincing to directly transfer the knowledge of natural language in LLMs to time series tasks. Note that i) text and time series are distinct data modalities, and ii) the pre-trained LLMs are not pre-trained with text-time-series pairs. \n- Furthermore, the first contribution *“introducing a novel concept of reprogramming large language models for time series forecasting without altering the pre-trained backbone model“* is not new, as previous works such as GPT4TS [1] have also explored this reprogramming approach, regardless of which parts of the LLMs are fine-tuned. Additionally, the proposed method requires training the input and output layers for adaptation, which means that it does involve some alteration of the pre-trained backbone model.\n- While using text to aid in time series prediction can be beneficial, as seen in applications such as stock prediction using financial news text mining, it's unclear how the shared declarative for a whole time series dataset can help understand complex temporal behaviors in different windows. Although these prompts may provide domain expert knowledge and task instructions, they do not introduce text information at each time step. Therefore, it remains unclear how these text prompts can benefit the understanding of complex temporal behaviors in time series.\n- Compared to related work such as [1], which has applied pre-trained LLMs to various time series tasks, the experiments in this paper are relatively limited. For instance, only prediction tasks are considered, and even for long-term prediction tasks, only the ETT datasets are included. This narrow scope of experiments limits the evaluation of the proposed approach to other time series tasks and datasets.\n- The link for the source code provided in the paper is empty. I cannot check for more details regarding the experiments.\n\nReference:\n[1] One Fits All: Power General Time Series Analysis by Pretrained LM", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rdaH21r4Q7", "reviewer": "Reviewer_Z9NA", "summary": "The paper proposes an approach to reprogram pre-trained large language models such that they can effectively do forecasting in zero-shot, few-shot, and fully supervised settings. The authors propose two strategies two forecast time-series: (1) by reprogramming patches of time-series by grounding them in text prototypes via cross attention, and (2) by using descriptions of the data, instruction, statistics of time-series as  the prefix. The authors demonstrate promising performance on short and long horizon tasks.", "questions": "I do not have any questions at that would change my opinions regarding the paper. I think that the rigour of the experiments must be improved.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is very well written, clear, with sufficient details to ensure reproducibility. I really liked that desiderata that the authors identified to enable LLMs to produce forecast. \n2. The experiments were well designed with some limitations in rigour which I will discuss in the next section.\n\nI really liked the paper, it was well written, well motivated and performant.", "weaknesses": "Following are some things to improve in the paper. I think in general the experiments can be made more rigorous.\n1. **Baselines**:  I understand that the authors are following the experiment protocol followed by TimesNet, but there are several limitations: (1) Statistical methods such as AutoARIMA, AutoTHETA, AutoETS, Naive and Seasonal Naive, etc. were not compared with. These methods are important and very performant in practice, (2) N-BEATS and N-HITS were only compared during short-horizon forecasting, (3) Recent papers on using LLMs for time-series forecasting were not compared against, for e.g. LLM4TS [1] and PromptCast [2] (4) I am aware that the paper \"LLMs are zero-shot forecasters\" [3] only got recently published, but it would improve the experiments if the authors were able to compare with it. I should emphasize that this is completely optional. \n2. **Datasets:** Increasing the amount of datasets for experimentation will improve the results. The current set of datasets is pretty limited, even for long horizon forecasting datasets, where datasets such as Influenza-like Illnesses, Exchange Rate, Tourism, and Weather etc. (see PatchTST) were missing. For short-horizon datasets, M3 at the very least, and the Monash time-series forecasting archive can be added to improve results. \n\n**References:**\n[1] Chang, Ching, Wen-Chih Peng, and Tien-Fu Chen. \"Llm4ts: Two-stage fine-tuning for time-series forecasting with pre-trained llms.\" arXiv preprint arXiv:2308.08469 (2023).\n\n[2] Xue, Hao, and Flora D. Salim. \"PromptCast: A New Prompt-based Learning Paradigm for Time Series Forecasting.\" (2022).\n\n[3] Gruver, Nate, et al. \"Large Language Models Are Zero-Shot Time Series Forecasters.\" arXiv preprint arXiv:2310.07820 (2023).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5j35okXerl", "reviewer": "Reviewer_MAkL", "summary": "The paper presents Time-LLM, a framework designed to harness Large Language Models (LLMs) for time series forecasting. By converting time series data into text-like prototypes and introducing the Prompt-as-Prefix (PaP) method to supplement this data with additional context, the framework effectively aligns time series data with the modalities of natural language. The empirical results indicate Time-LLM's superiority in comparison to other leading models, particularly in few-shot and zero-shot contexts.", "questions": "1. Does the model operate in an autoregressive manner, predicting only the subsequent step, or does it directly forecast the next $H$ steps?\n2. How is $E'$ derived from $E$?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is articulate and systematically structured, making the motivation and methodology behind the proposed solution evident.\n2. The approach of modality alignment from time series to natural language is both innovative and promising, offering a new perspective for future research.\n3. The empirical evaluation is thorough, encompassing an analysis of different LLM variations, an ablation study, computational efficiency considerations, and model interpretation.", "weaknesses": "**Major**\n1. The choice of datasets for evaluation is restrictive, as the ETT datasets involve similar metrics monitored under different conditions. Their mutual similarities might overinflate the perceived performance of Time-LLM. Inclusion of diverse datasets such as Weather, Electricity, and Traffic, commonly featured in literature, would offer a more holistic assessment. Moreover, there's an emerging consensus that long-term forecasting benchmarks have a preference for univariate models, potentially bypassing the capability of handling cross-variate correlations  ([1], [2]). As Time-LLM also processes each channel separately, this limitation should be discussed.\n2. The mechanism used to produce the next H steps remains unclear, warranting a more detailed explanation.\n\n**Minor**\n1. Figure 3 Ambiguities:\n    1. Fig 3(a): Initially, the patches seem to represent input patches $X_P$. To eliminate any confusion, clearly labeling associated variables like $E, Z$ would be helpful.\n    2. Fig 3(b): The figure raises questions about whether the model outputs only the subsequent step or the next $H$ steps. Additionally, the function of the intermediate layer remains undefined.\n2. Error in Fig 3(b): There seems to be a need for a one-step left shift in the output of Patch-as-Prefix to render it an auto-regressive model.\n\nI would be glad to raising my score if the aforementioned weaknesses are addressed.\n\n[1] Chen, Si-An, et al. \"TSMixer: An All-MLP Architecture for Time Series Forecasting.\" Transactions on Machine Learning Research. 2023\n\n[2] Das, Abhimanyu, et al. \"Long-term Forecasting with TiDE: Time-series Dense Encoder.\" Transactions on Machine Learning Research. 2023", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FB48naOU2d", "reviewer": "Reviewer_cwaB", "summary": "This work casts time series modeling as yet another \"language\" task by addressing the problem of casting continuous-valued data into a discrete representation and leveraging prompts to the language model. Here the authors draw from previous works, including PatchTST, to patch the continuous valued time series data and project them onto word embeddings via an attention mechanism. The new discretized embeddings and a word prompt are fed into a pretrained LLM which predicts future values. This work produces state-of-the-art results on both short-term and long-term predictions on the M4 and electricity transformer temperature (ETT) datasets, respectively. The work also produces the state-of-the-art results on zero and few-shot learning evaluated on ETT.", "questions": "My primary concern is that the evidence provided, the stated intentions, and the claims of this work do not fully line up. I would like to know the primary objective and claims of this work. Depending on that answer I feel there might be more evidence required. \n\na) If this work is meant to show that the model is state-of-the-art across many potential time series tasks then comparisons across other datasets (mentioned above) would be much more convincing. This work only uses ETT and M4.\n\nb) If this work is meant to benchmark this model's capabilities on a single data set then please make that clear. The state-of-the-art claims should only be made with regard to the data sets the model was benchmarked against.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "Instead of creating a new representation of the continous-valued time series data the authors discretized the data by projecting it onto existing word embeddings via an attention mechanism. This approach is exciting in it's use of the word embeddings as the discretization medium rather than using a linear layer in the final calculation of the embeddings. In doing so, this work shows how using not only trained LLMs but also the learned embeddings can transfer to time series problems.\n\nBy building the final embeddings from word embeddings, the authors cast the time-series problem into a \"language\" problem. This allows the authors to further utilize language model features by employing a prompt which is shown to improve time series predictions. One could imagine expanding further on this new property by trying different prompts and seeing if such prompts can illicit adversarial results. Ultimately, this approach may be utilized for new experiments to both probe LLMs and further exploit their capabilities.", "weaknesses": "The authors chose a limited set of time series data to benchmark this model on. Many of the models they compare against have been benchmarked against other data sets: weather, traffic, electricity consumption, and the spread of influenza. This confuses the objective of the paper. If the objective is to show this model gives the state-of-the-art results for the two data sets then that is clear. If the authors wish to claim this model outperforms others across many domains then more evidence is needed. If the authors wish to introduce this model and benchmark it on a single data set to demonstrate its potential capabilities then they provide many examples.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eMBnjIFJfJ", "reviewer": "Reviewer_baDP", "summary": "This paper introduces a pioneering approach, Time-LLM, which harnesses the reprogramming and Prompt-as-Prefix techniques to repurpose large language models for time series forecasting while keeping the backbone LLM intact. The method innovatively bridges input time series to optimized text prototypes, making the time series inputs more digestible for language models. This integration enables large language models to seamlessly tackle time series forecasting. Furthermore, Time-LLM incorporates the Prompt-as-Prefix mechanism to facilitate the LLM's reasoning capabilities over time series. By adding natural language prompts, the approach enriches time series input and presents task directives in a comprehensible language format. In my view, this holds substantial promise for controlled time series analysis across diverse applications. The authors have conducted extensive experiments to demonstrate the effectiveness and efficiency of the proposed reprogramming framework, showing considerable potential in leveraging LLMs for time series tasks.", "questions": "1.\tCould you elucidate the mechanism behind \"Output Projection\" (Sec. 3.1), particularly the aspects related to flattening and the linear projection? A formulaic representation would greatly aid in understanding.\n\n2.\tIn the \"Prompt-as-Prefix\" paragraph, there is a reference to computing the trend and lag in relation to time series, with several potential implementations hinted at. Could the authors detail the methodology used to determine the trend and lag information within the prompts?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1.\tThe paper is articulately composed and well-organized, with most concepts clearly presented. It offers an in-depth exploration of the proposed concepts related to LLM reprogramming and Prompt-as-Prefix.\n\n2.\tThe proposed reprogramming framework is novel and has proven to be highly effective. Instead of directly feeding original time series into LLMs, the innovative approach of converting time series into text prototype representations for language model comprehension stands out. This strategy might set the foundation for a broader method of cross-modality adaptation in LLMs.\n\n3.\tThe augmentation of input context with declarative prompts, such as domain expertise and task guidelines, to steer LLM reasoning is notably promising. It offers significant potential for controlled time series analysis in a variety of applications.\n\n4.\tThe architectural design of Time-LLM is logical and underpinned by clear motivations. Dividing time series into patches and reprogramming each into text prototype representations is a wise choice, which aligns well with the use of natural language prompts to guide LLM in time series reasoning.\n\n5.\tComprehensive experimental results are provided to evaluate the proposed reprogramming framework from various aspects. Indeed, Time-LLM demonstrates promising results, especially under the few-shot and zero-shot protocols, showing considerable sample efficiency and applicability in real-word applications. There are also abundant ablation and other side experiments to study the proposed method from various aspects.", "weaknesses": "1.\tWhile this paper is generally well-written, there are areas that could benefit from further refinement. For instance, Fig. 5 lacks clarity and might benefit from being displayed at a larger scale to improve visibility. In patch reprogramming (Sec. 3.1), it would be beneficial to illustrate how the linear projection aligns the hidden dimensions in Fig. 2. Within the Prompt-as-Prefix discussion, reference is made to the inclusion of statistics within the prompt. Elaborating on the specific content and the calculation methods employed would enhance clarity.\n\n2.\tThe paper's foundational concept hinges on the idea of reprogramming. However, it omits references to several pivotal reprogramming studies from recent years (as listed below). Integrating these pertinent studies in the introductory or related works section could provide a more robust understanding of the concept.\n\n3.\tThere are some writing issues. For instance, there should be uniformity in the references' formatting, with a preference for citing formally published works from conferences or journals over preprints.\n\n[1] Vinod, R., Chen, P. Y., & Das, P. (2020). Reprogramming Language Models for Molecular Representation Learning. In Annual Conference on Neural Information Processing Systems.\n\n[2] Melnyk, I., Chenthamarakshan, V., Chen, P., Das, P., Dhurandhar, A., Padhi, I., & Das, D. (2022). Reprogramming Pretrained Language Models for Antibody Sequence Infilling. International Conference on Machine Learning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)