LLM-TS Integrator: Integrating LLM for Enhanced Time Series Modeling
Authors: Can Chen
Gabriel L. Oliveira
Hossein Sharifi-Noghabi
Tristan Sylvain
Venue: Accepted by TMLR ()

Abstract: Time series~(TS) modeling is essential in dynamic systems like weather prediction and anomaly detection. Recent studies utilize Large Language Models (LLMs) for TS modeling, leveraging their powerful pattern recognition capabilities. These methods primarily position LLMs as the predictive backbone, often omitting the mathematical modeling within traditional TS models, such as periodicity. However, disregarding the potential of LLMs also overlooks their pattern recognition capabilities. To address this gap, we introduce \textit{LLM-TS Integrator}, a novel framework that effectively integrates the capabilities of LLMs into traditional TS modeling. Central to this integration is our \textit{mutual information} module. The core of this \textit{mutual information} module is a traditional TS model enhanced with LLM-derived insights for improved predictive abilities. This enhancement is achieved by maximizing the mutual information between traditional model's TS representations and LLM's textual representation counterparts, bridging the two modalities. Moreover, we recognize that samples vary in importance for two losses: traditional prediction and mutual information maximization. To address this variability, we introduce the \textit{sample reweighting} module to improve information utilization. This module assigns dual weights to each sample: one for prediction loss and another for mutual information loss, dynamically optimizing these weights via bi-level optimization. Our method achieves state-of-the-art or comparable performance across five mainstream TS tasks, including short-term and long-term forecasting, imputation, classification, and anomaly detection. Our code is available at: \url{https://anonymous.4open.science/r/llm_ts_anonymous-F07D/README.MD}

Published in Transactions on Machine Learning Research (04/2025)
LLM-TS Integrator: Integrating LLM for Enhanced Time Series Modeling
Can (Sam) Chen Mila Quebec AT Institute
can. cww.cheemhee www.chen@mmill.guee..
Gabriel L. Oliveira
gabrielleevoss@@maallcoom
RBC Borealis
Hossein Sharifi-Nohhbbi
hossein.sharrffhhoveelaaccccmmm
RBC Borealis
Tristan Sylvain
tristan.sylraan albreaiiaii.n.ooo..sss....ooo
RBC Borealis
Reviewed on OpenReview: https: // operreveeww.we.... net/ forum? id- PVVQ@mjCCy
Abstract
Time series (TS) modeling is essential dynamic systems like weather prediction and anomaly detection. Recent studies utilize Large Language Models (LLMs) for TS modeling, leveraging their powerful pattern recognition capabilities. These methods primarily position LLMs as the predictive backbone, often omitting the mathematical modeling within traditional TS models, such as periodicity. However, disregarding the potential of LLMs also overlooks their pattern recognition capabilities. To address this gap, we introduce LLM-TS Inlegrator, a novel framework that effectively integrates the capabilities LLMs into traditional TS modeling. Central this integration our mutual information module. The core of this mutual information module is a traditional TS model enhanced with LLM-derived insights for improved predictive abilities. This enhancement is achieved by maximizing the mutual information between traditional model's TS representations and LLM's textual representation counterparts, bridging the two modalities. Moreover, we recognize that samples vary in importance for two losses: traditional prediction and mutual information maximization. To address this variability, we introduce the sample reweighting module to improve information utilization. This module assigns dual weights to each sample: one for prediction loss and another for mutual information loss, dynamically optimizing these weights via bi-level optimization. Our method achieves state-of-the-art comparable performance across five mainstream TS tasks, including short-term and long-term forecasting, imputation, classification, and anomaly detection. Our code is available at: https: thub. com/ BorealisAI/LLM-TSS -S-LMM-----MA Htth:/LLLMM----nenrrttooo
1 Introduction
Time series (TS) modeling, as emphasized Hyndman and Athanasopoulos (2018). is crucial a variety of real- world applications. is instrumental forecasting meteorological factors for weather prediction (Wul et al. 2021), imputing missing data in economic TS (Fiiedman 1962) detecting anomalies in industrial monitoring data for maintenance (Gao et al. 2020 and classifying trajectories for action recognition Franceschi et al.] 2019). Given its significant. icactical impact, TS analysis continues to attract substantial
attention [Lm and Zohren 2021 Wen et al. 2022
* Work done during internship RBC Borealis.

---

Published in Transactions on Machine Learning Research (04/2025)
(a) Mutual Information
Inference
Trainable
Template
The M4 Yearly datasel contains yearly collected univariate
content "I, 2, statistics;
value
Backbone: TimesNet
x LLM
Represeenation for time series
Mutual information maximization
Representation for text description
(b) Sample Reweighting
W
Prediction loss
Update
Downstream task
Weighting net
Validiaion data
Bi-level optimization
Figure 1: Illustration of LLM-TS Integralor. Module (a) enhances the traditional TS model (TimesNet) with LLM-derived insights by mutual information maximization. Module (b) optimizes sample importance for both prediction loss and mutual information loss improve information utilization. The LLM is utilized solely during the training phase and is not required during inference.
Recent efforts in TS modeling have increasingly adopted Large Language Models (LLMs) leverage their exceptional pattern recognition capabilities (Jiang al. 2024 Zhou et al. 2023 Jin et al 2024 Sun et al.)
2023 Gruver et al. 2023b Cao et al. 2024). While these innovative approaches validate the potential of LLMs in TS modeling, they primarily position LLMs as the core predictive model. Consequently, they often omit the mathematical modeling tailored specifically for TS models, such as employing the Fourier Transform to capture periodic patterns (Wu et al. 2023
On the other hand, fully disregarding the potential of LLMs also overlooks their powerful pattern recognition capabilities. It is important recognize the balance between leveraging LLMs for their advanced capabilities and utilizing traditional TS models for their mathematical modeling, to enhance the overall performance and accuracy of TS predictions. In response, propose LLM-TS Inlegrulor, novel framework that effectively integrates the capabilities of LLMs into traditional TS modeling.
Central to our framework a mutual information module, as depicted in Figure ! aa. The core of this module a traditional predictive model, which we enhance with insights derived from LLMs to improve its predictive abilities. In this work, we primarily utilize TimesNet (Wu al. 2023) as the traditional predictive model due to its exceptional performance and insight into periodic modeling, and our framework is also applicable to other traditional TS models (see in Section 4.6.3). We achieve this enhancement by maximizing the mutual information (Sun et al., 2020) between the TS representations from traditional models and their textual counterparts from LLMs, thereby bridging these two modalities (a detailed discussion of various LLMs is provided in Section 4.6.4p. Despite its established use, mutual information maximization has not been previously applied the intersection of TS and text domain. With textual descriptions often missing from TS data, we propose gencrating such descriptions via a carefully designed template. This template is enriched with essential background and statistical details pertinent to the TS, thereby enriching the LLM's comprehension of the TS context (a comprehensive discussion of various templates is in Appendix A.5)
Our first module introduccs dual loss framework: traditional prediction and mutual information, and we recognize that the importance of samples dillers between the two losses. For instance, a large prediction loss for a sample highlights its learning potential, emphasizing the need to focus on its prediction loss. This scenario also implics that the model's learning for this sample inadequate and its hidden representation is suboptimal for mutual information computation. Consequently, the sample's contribution to the mutual information calculation should be reduced. To manage this variability, have introduced novel sample
2

---

Published in Transactions on Machine Learning Research (04/2025)
reweighting module powered by simple MLP (multilayer perceptron) network, depicted in Figure [ This module processes the sample prediction loss to produce dual weights for each sample, one for the prediction loss and another for the mutual information loss. These weights are optimized through bi-level optimization, thereby enhancing the efficacy of information utilization.
Our primary contributions are as follows:
We introduce the LLM-TS Inlegrator framework, which consists of mutual information and sample reweighting. The first module enhances traditional TS modeling with capabilities from LLMs through mutual information maximization.
The second module optimizes sample importance for both prediction loss and mutual information loss, which improves information utilization.
Extensive experiments across five mainstream TS tasks seee-sshotttttre and long-term forecasting, imputation, classification, and anomaly deneatiiondmmoosrratt the effectivenoss of our framework. Obtained experi- mental results align across state-of-the-art of traditional time series and various LLMs which demonstrates applicability of LLM-TS Integrator framework regardless of choice of methods.
LLM-TS Integrator framework keeps LLMs frozen and does not require fine-tuning and introduces minimum additional costs compared to traditional time series methods.
2 Preliminaries
NemesNet. In this paper, we mainly choose TimesNet as the traditional predictive model due to its exceptional performance (Wu al., 2023) and also explore other additional traditional models including ETSformer (Woo et al. 2022). Stationary (Liu et al. 2022b and FreTS (Yi et al., 2023) in Section 4.6 Previous studies to modeling temporal variations in 1D time series oflen struggle with complex temporal patterns. TimesNet addresses this challenge by decomposing these complex variations into multiple intra- period and inter-period variations. This is achieved by transforming the 1D time series into series of 2D tensors, each corresponding to different periods. For the time series æ, we derive its representation (x) using the TimesNet model parameterized by where m represents model.
Large Language Models. Language models are trained on extensive collections of natural language sequences, with each sequence consisting of multiple tokens. Notable large language models such as GPT- 3 (Brown et al. 2020) and Llama2 TTovroon et al. 2023) aim to predict the next token based on preceding tokens, demonstrating their capabilitics through improvements in model parameter size and the amount of training data. Each language model uses tokenizer that breaks down an input string into a sequence of recognizable tokens. However, the training current large language models solely focused on natural language, not encompassing time series data. This limitation presents challenges for the direct application of large language models time series analysis.
3 Method
In this section, present the LLM-TS Integrator framework, which effectively integrates the capabilities of LLMs into traditional TS modeling. This framework consists of two modules: mutual information and sample reweighting. The first module enhances a traditional TS model with LLM-derived insights for improved predictive abilities, as explored in Section 3.1 The second module optimizes weights for prediction loss and mutual information loss via bi-level optimization, improving information utilization, as covered in Section 3.2 The overall algorithm shown Algorithm
3.1 Mutual Information
Previous studies (Zhou al. 2023 Jin et al. 2023 have predominantly highlighted the use of Large Language Models (LLMs) as the core predictive model in the TS analysis, often omitting the mathenatical modeling within traditional TS models, such periodicity.
3

---

Published in Transactions on Machine Learning Research (04/2025)
Algorithm LLM-TS Integrator
Input: The TS dataset D, number of training iterations T.
Output: Trained TS model paramcterized by 0'.
1: /* Mutual Information Module */
2: Train a traditional TS model (e.g., TimesNet) parameterized by using D.
Generate text description t for TS sample via designed template.
Derive hidden representations hg)(a) (a:) from the TS model and h (t) from the L.M.M
while <= - do
6: Sample æ, from D, where are the labels.
7:
Optimize a discriminator model Ta to estimate mutual information as per Eq .22
8:
/* Sample le Reweighting Module */
9. Process sample loss to with the weighting net to produce dual weights as per Eq. 8
10: Adopt bi-level optimization to update the weighting net. following Eq.
11: Re-calculate dual weights using the updated weighting net per Eq. (3
12: Calculate the overall loss to update the TS model as per Eq.
13: end while
14: Return the trained TS model parameterized by 0*
In contrast, framework utilizes traditional TS model the predictive backbone, enhanced by the advanced capabilities of LLMs. In this paper, we cmploy TimosNet, as outlined in Section 2 as the traditional predictive model, and we further examine other models in Section 4.6 This hybrid methodology combines the advantages of both traditional TS models and modern LLMs. We achieve this integration via a mutual information module, which maximizes the mutual information between the TS data representations derived from the traditional model and their corresponding textual representations derived from LLMs.
3.1.1 Mutual Information Estimation.
Estimating the mutual information between hidden representations of time series (TS) sample a and its corresponding textual description t is essential, For the TS sample æ, we derive its representation (x) using TimesNet, a model parameterized by O. For the text t, its representation h' h²(t) )) is extracted using a pre-trained LLM, where denotes the language model. In this study, we employ the LLaMA-3b model (Touvron et al,
2023 as our primary LLM, while also evaluating other LLMs as detailed in Section 4.6
We estimate mutual information using the Jensen-Shannon MI estimator (Snn et al.) 2020 Nowozin al. 2016) and additionally explore the MINE estimator (Heelm et al. 2019a detailed in Appendix A. Specifically, let (x, t) represent a sample from the TS set S, and (æ, t) denote a sample from S = S where (x, t) * (z, E). Within this context, S denotes the TS training distribution while the product x S represents pairs of distinct samples within S. Then the lower bound of mutual information can be estimated as:
(00 Eel-sp(-TT - (a), h()),,,
(1)
where Ta denoles the discriminator parameterized by 3 and sp is the softplus function. For the details of T3 we feed the positive and negative examples into a 1-layered fully-comnected network with a hidden size of 64, and then output the dotproduct of the two representations. The mutual information estimation begins by fixing the model parameters 0, followed by training & as the estimator. Specifically, we optimize & via the following:
00
3-
(2)
03
where too denotes the learning rate. Subscquently, we refine the model parameters 0 maximize mutual information, thereby enriching the traditional TS model with insights derived from LLMs. This alternating optimization procedure between model and discriminator is repeated each epoch.
4

---

Published in Transactions on Machine Learning Research (04/2025)
3.1.2 Text Description for Time Series.
In our approach, we assume each time series (TS) sample æ is paired with a corresponding textual description, t. However, textual descriptions are frequently unavailable for many TS datasets. To bridge this gap, we introduce a methodology for generating textual descriptions of TS data. We propose creating textual representations that capture essential background and statistical details of the TS inspired by Jin et al. (2023). This process be systematically implemented using the following carefully designed template:
template
" "(task_ce "(task_descrippp descrripiionn....... The content is: (TS}.
" "Imput statistics: min value {min (TS)), max value {max (TS)}, "median value {median(TS) (TS)}, top 5 lags {compute_ llgg (TS)
3.2 Sample Reweighting
Our mutual information module introduces two distinct loss functions: (1) the original sample prediction loss, (x y), hereafter referred to lo, which corresponds to the prediction loss for a TS sample æ and its label y, and (2) the mutual information maximization loss, denoted as -I (3). We acknowlodge that the significance of samples varies between these two losses. Specifically, large prediction loss to indicates sample's substantial learning potential, thereby justifying higher weight for its prediction loss. Conversely, this suggests that the sample's representation may be suboptimal for mutual information computation, warranting a lower weight w1.
Here, (a, 3) denotes the sample-level loss for each task: For the forecasting task, we use MSE, where y represents the future target values. For the classification task, we use cross-entropy, where y corresponds to a class label. For the imputation task, compute MSE for the masked points that need to be reconstructed. For the anomaly detection task, we adopt a rrconstruction-based approach, measuring MSE between the original and reconstructed time scries.
To address this disparity, we have developed a novel sample reweighting module, described as follows.
3.2.1 Weighting Network.
To automate weight assignment, our module employs a two-layer MLP network parameterized by a, which processes the sample prediction loss to produce pair of weights:
wo(a),wr(a) = (L)a(lo)
(3)
This process involves converting the sample loss lo into a latent code = through hidden layer. The network then outputs dual weights:
(r(a),,7(()) = amoo : 2), (m,
(4)
where mo 0 and my < 0 to ensure negative correlation between wo and wy. The function a(-) denotes the sigmoid activation.
For a batch of N samples, the weight vector wo e N is directly applied to the original prediction loss vector to E RN, resulting in the weighted average loss calculated as mean(wo(a) lo). Similarly, the weight vector (ax) E N not only reflects the overall significance of the mutual information but also each sample's individual contribution to this metric. The mean of these weights mean(wr(a)) represents the overall importance. For mutual information computations, wr(x) is transformed into probability distribution pi of
This adjustment affects the distribution used in mutual information calculations,
necessitating recalculation of mutual information as 1(0, B, a)), with details provided in Appendix A.1 As a result, the overall loss is formulated as:
L(O, ax) mean(wo(a)) 1o) )-)- (eea((wr(a))) (07,),,(((w)))
(5)
5

---

Published in Transactions on Machine Learning Research (04/2025)
3.2.2 Bi-level Optimization.
The ensuing challenge is optimizing the weighting network a. We achieve this by leveraging the supervision signals from a small validation dataset (Chen et al. 2022a). the weighting network is properly optimized, the model trained with these weights expected to show improved performance on the validation dataset in terms of the validation loss (v( M Σj (z,, Yj). where M denotes the size of the validation set. This constitutes bi-level optimization problem (Hospedales et al., 2021
Chen al. 2022a bb. At the inner level,
model training is conducted through gradient descent:
((a)= = -m aL(0,a)
(6)
00
The objective is to ensure that the model performs optimally the validation dataset:
(lv()(())
â = a 7/2
(7)
Both 7 and 7/2 represent the learning rales for the respective optimization steps. Through the minimization of the validation loss, we aim to optimize the weighting network a.
4 Experimental Results
To demonstrate the versatility of our LLM-TS Integrator, conduct extensive experiments across five main tasks: short- and long-term forecasting, imputation, classification, and anomaly detection. To maintain experimental integrity, our methodology adheres to the setup in (Wu et al. 2023). We detail the experimental setting in Appendix A.2
4.0.1 Baselines.
Our evaluation employs comprehensive array of baseline models across several architectural designs (1)
baseb models, specifically TimesNet [Wu et al. 2023 (2) MLP--based models, including LightTS (Zhang
ct al.
2022) and DLincar (Zeng et al. 2023): (3) Transformer based models, such as Reformer (Kitaey
2020); Informer (Zhou et al., 2021 by. Autolormer (Wu et al., 2021), ), FEDformer (Zhou el al. 2022). Nonstationary Transformer (Liu al. 2022b), ETSfformrr (Woo al. 2022), and PatchTST (Nie al.
et al.
2023); (4) LLM-based models, represented by GPT4TS (Zhou et al. 2023). While we assess wide range of models, focus our discussion on the top-performing ones as highlighted in Zhou et al. (023))
Additional comparisons for forccasting tasks include LLM-based models like Time-LLM (Jin et al. 2023 and TEST (Sun et al. 2023 For short-term forecasting, models like N-HiTS (Challu el al. 2023 and
N-BEATS Oreshkin et al. 2020 are included. Anomaly detection tasks are assessed using Anomaly Transformer (Xu et al. 2022 1 and for classification, models such as XGBoost (Chen and Guestrin 2016).
Rocket (Dempster et al. 2020 LSTNet (Lai al., 2018),, LSSL (Gu et al. 2022). Pyraformer (Liu al. 2022a TCN (Franceschi al. 2019 and Flowformer (Huang al.] 2022 are considered. This broad
selection of baselines enables a rigorous comparison across various tasks, highlighting the capabilities of our method.
4.1 Main Results
Figure demonstrates that our LLM-TS Integrator consistently outperforms other methods in various tasks, underscoring its efficacy. We will refer to our method as LLM-TS in the tables for brevity. Unless otherwise indicated, we cite results from TimesNet (Wu et al. 2023). We reproduce TemesNet and GPT4TS (Zhou
et al. 2023 experiments for all tasks. All results are averages from three runs with different seeds. Standard deviations for ablation studics are detailed in Appendix g The best results are highlightod in bold, with the second-best underlined. We also (1) present several showcases of our method Appendix A.4 and (2) discuss the model efficiency in Appendix A.6

---

Published in Transactions on Machine Learning Research (04/2025)
Long-term Forecasting
Imputation
Short-term Forecasting
LLM-TS
TimesNet
GPT4TS
PatchTST
FEDformer
Classification
Anomaly Detection
Figure 2: Model performance across different tasks.
4.2
Short- and Long- Term Forecasting
4.2.1 Setup.
To comprehensively assess our framework's forecasting capabilities, we engage it in both short- and long-term forecasting settings. In the realm of short-term lorecasting, we utilize the M4 dataset (Spyros Makridakis 2018 which aggregates univariate marketing data on a yearly, quarterly, and monthly basis. For long-ten ter-term forocasting, we examine five datasets following (Zhou et al., (023): ETT (Zhou al. 2021a). Electricity (UCI 2015), Traffic (PeMS) 2024), Weather [Wetterstation] 2024 and ILI (CDC 2024). We adhere to the Times Net setting with input. length of 96. For L.L.e methods like GPT4TS and Time-TLM, which use different input lengths, rerun the experiments using their code. For PatchTST, cite the results from (Wang et al. (023aa), as the original PStchTST uses an input length of 512. Due to shorter input lengths in this study compared to the original, the reported performance lower.
4.2.2 Results.
As shown in Tables and our LLM-TS performs exceptionally well in both short- and long-tern settings. It consistently surpasses TimesNet, highlighting the effectivencss of incorporating L.M--derived insights. Furthermore, it generally outperforms other LLM-based methods such GPT4TS, TIME-LLM, and TEST, underscoring the advantages of integrating traditional TS modeling.
Table 1:
Short-term M4 forecasting. The prediction lengths are in [6, 48) and results are obtained by weighting
averages across multiple datasets with varying sampling intervals. Full results are Appendix A.7
Methods
LLM-TS
TimesNet GPTATS TIME-LLM TEST PatchTST N-HITS N-BEATS FEDformer Stationary
Autoformer
SMAPE
11.819
11.908
11.991
11.983
11.927
12.059
11.927
11.851
12.810
12.780
12.909
MASE
1.588
1.612
1.600
1.595
1.613
1.623
1.613
1.599
1.701
1.756
1.771
OWA
0.851
0.860
0.861
0.859
0.861
0.869
0.861
0.855
0.918
0.930
0.939

---

Published in Transactions on Machine Learning Research (04/2025)
Table 2: Long-term forecasting: Averages over lengths: 24, 36, 48, 60 for ILI, and 96, 192, 336, 720 for others. Full
results in Appendix A.8
Methods
LLM-TS
TimesNet
TIME-LLM
DLincar
PatchTST
GPTTTS
FEDiormer
TEST
Stationary
Sformmer
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MAE
MSE
MAE
MSE
Weather
0.257
0.285
0.205
0.290
0.279
0.296
0.265
0.317
0.265
0.285
0.275
0.292
0.309
0.360
0.291
0.315
0.288
0:314
0.271
0.334
ETTh1
0.454
0.451
0.470
0.462
0.474
0.459
0.456
0.452
0.516
0.484
0.473
0.451
0.440
0.460
0.440
0.460
0.570
0.537
0.542
0.510
ETTH2
0.396
0.113
00:133
0.126
0.398
0.415
0.339
0.515
(.391
0.411
0.383
0.410
0.437
0.449
0.414
0.432
0.526
0.516
0.439
0.452
ETTml
0.401
0.109
0.114
0.418
0:137
0.421
0.403
0.407
0.406
0.407
0.108
0.400
0.448
0.452
0.402
0.411
0.481
0.456
0.429 0.425
ETTm2
0.295
0.331
0.294
0.331
0.298
0.342
0.350
0.401
0.290
0.334
0.290
0.335
0.305
0.349
0.323
0.350
0.306
0.347
0.293 0.342
11I1
1.973
0.894
2:266
0.974
2.726
1.098
26166
1.090
2,184
0.906
5.117
1.650
2.847
1.144
3.524
1.232
2,07I
0.914
2.497 1,004
ECL
0.191
0.299
0.196
0.298
0.229
0.315
0.212
0.300
0.216
0.318
0.206
0.285
0.214
0.327
0.237
0.321
0.193
0.296
0.208
0.323
Traficc
0.618
0.333
0.627
0.335
0.606
0.395
0.625
0.383
0.529
0.311
0.561
0.373
0.610
0.376
0.381
0.388
0.624
0.340
0.621
0.396
Average
0.574
0.427
0.618
0.442
0.681
0.468
0.686
0.483
0.600
0.4%6
0.964
0.525
0.701
0.489
0.756
0.491
0.633
0.465
0.662
0.473
4.3
Imputation
4.3.1
Setup.
To assess our method's imputation capabilities, we employ three datasets: ETT (Zoou et al. 2021a). Electricity (UCI 2015), and Weather (Wetterstation 2024), serving as our benchmarks. To simulate various degrees of missing data, we randomly obscure time points at proportions of (12.5%, 25%, 37.5%, 50%} following (Wul
et al.
2023
4.3.2 Results.
Table 3 illustrates that our method achieves performance comparable to GPT4TS and surpasses other basclines, highlighting its effectiveness. We attribute the robust performance of GPT4TS primarily to its backbone feature extractor: the pre-trained language model, which excels at capturing time series patterns, enhancing its imputation proficiency.
Table 3: Imputation task: Randomly masked (12.5% 25%, 37.5%, 50%} of points in 96-length series, averaging results over 4 mask ratios. Full results are Appendix A.9
LLM-TS
TemesNet MSE
GPTNTS MSE
PStchTST
LightTS MSE
DLinear MSE
PEDformer MSE MAE
Stationary MSE
Autoformer MSE MAE
Reformer MSE
Methods
MSE
MAE
MAF
MAE
MSE
MAE
MAF
MAE
MAF
MAE
BTTml
0.025
0.103
0.028
0.100
0.028
0.108
0.047
0.140
0.104
0.218
0.003
0.206
000062
0.177
0.036
0.126
0.0551
0.150
0.055
0.166
ETTm2
0.021
0.087
0.022
0.089
0.023
0.088
0.029
0.302
0.046
0.151
0.096
0.208
0.101
0.215
0.026
0.099
0.029
0.105
0.157
0.280
ETTh1
0.087
0.198
0.090
0.069
0.174
0.115
0.284
0.373
0.201
0.306
0.246
0.094
0.201
0.103
0.214
0.122
0.245
ETT12
0.050
0.148
0.051
0.150
0.050
0.144
0.065
0.163
0.119
0.250
0.142
0.250
0.163
0.279
0.053
0.152
0.055
0.156
0.234
0.352
ECL
0.094
0.211
0.095
0.212
0.091
0.207
0.072
0.183
0.131
0.262
0.132
0.260
0.130
0.259
0.100
0.218
0.101
0.225
0,200
0.313
Weather
0.056
0.031
0.032
0.058
0.031
0.055
0.052
0.110
0.203
0.032
0.039
0.081
0.057
0.038
0.087
Average
0.051
0.134
0.053
0.136
0.049
0.130
0.060
0.144
0.123
0.228
0.119
0.224
0.112
0.229
0.056
0.142
0.061
0.151
0.134
0.240
4.4
Classification
4.4.1
Setup.
We focus on the application of our method to sequence-level time series classification tasks, a crucial test of its ability to learn high-level representations from data. Specifically, we employ 10 diverse multivariate datasots sourced from the UEA Time Series Classification repository (Bagnall et al. 2018). These datascts encompass wide range of real-world applications, including gesture and ict recognition, audio processing, among other practical domains. We reproduce the results of TEST based on their code (Sun et al. 2023
4.4.2 Results.
As depicted in Figure 3 our LLM-TS Integrator achieves superior performance with an average accuracy of 73.4%. As detailed in Appendix A.10 it consistently outperforms other L.L.-based methods across most.
8

---

Published in Transactions on Machine Learning Research (04/2025)
tasks, including GPT4TS and TEST. We attribute this enhanced capability to the traditional TS modeling techniques in our framework, which effectively capture classification characteristics more adeptly than LLMs.
Flowforeer
LUM.TS
73:00
wio
72:70
wio temp are
TimerNet
TimesNet
72:70
Rocket
72:50
Informer
72:10
Tranffommer
71:90
CKA Simbarit Scores
Lt
71:80
Reformer
71.00
Autoformer
71:10
ESTformer
11:00
LSSL
70:90
Pyraformer
70:80
FEDformer
70,70
LightT
70:40
TCN
70.30
0.6
GPT4TS
69.50
DLinear
67.50
XGBoost
TEST
0.5
62
66
72
Anomrlly Detection
Forecasting
Impuration
Chasifiation
Average Accuracy (%)
Tasks
Figure 3: Model comparison in classification.
Figure 4: CKA by Task.
4.5
Anomaly Detection
4.5.1
Setup.
Our study concentrates unsupervised time series anomaly detection, aiming to identify aberrant time points indicative of potential issues. We benchmark method against five established anomaly detection datasets: SMD (Su et al. 2019). MSL (Hundman et al. 2018), SMAP (Hundman et al. 2018). SWaT (Mathur and Tippenhauer 2016). and PSM (Abdulaal et al. 2021). These datasets span a variely of applications, including service monitoring, space and earth exploration, and water treatment processes. For a consistent evaluation framework across all experiments, we employ the classical reconstruction error metric to determine anomalies following Wu al. (2023).
4.5.2 Results.
As indicated in Table 4 our LLM-TS Integrator exhibits superior performance with an average Fi-score of 87% This result underscores the versatility of LLM-TS, demonstrating its capability not only classilying complete sequences, as discussed previously, but also effectively detecting anomalies in time series data.
Table 4: Anomaly detection task. F1-score (as %). the Transformers represents the of *former. Full results are in Appendix A.11
Methods
LLM-TS TimesNet GPT/TS PatehTS. ETS.
FED.
LightTS DLinear Stationary
Auto.
Pyra.
Anomaly:"
In.
Re.
Trans.
SMD
84.69
84.57
84.32
84.62
82.53
77.10
84.72
85.11
$3.04
75.32
79.56
MSI.
81.11
80.34
81.73
78.70
85.03
78.57
78.95
84.88
77.50
79.05
84.86
83.31
64.06
84.40
78.68
SMAP
69.41
09.18
68.86
68.82
69.50
70.76
69.21
69.26
71.09
71.12
71.00
71.18
09.92
70.40
69.70
SWaT
93.2
93.12
92.59
86.72
81.91
93. 19
93.33
87.52
79.88
92.71
91.78
83.10
$1.43
82.80
80.37
97.43
97.27
97.34
96.08
91.76
97.23
97.15
93.55
97.29
62.08
79.40
77.10
76.07
Average
85.17
84.90
84.97
82.79
82.87
84.97
84.23
82.46
82.08
81.26
82.57
80.50
78.83
77.31
76.88
4.6 Ablations
In this section, we first verify the effectiveness our framework by sequentially removing key components: (1) mutual information module and (2) sample reweighting module. Additionally, for mutual information, we explore the impact of removing the template while retaining the raw time series data inputs the LLM. We denote these variants w/o mutual, w/o reweight and w/o template. Our experiments span long-term
9

---

Published in Transactions on Machine Learning Research (04/2025)
Table 5: Results averaged over 4 prediction lengths.
Methods
Ours
w/o
mutual w/o template
w/o reweight
NemesNet
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
Weather
0.257
0,285
0.264 0.290
0.263
0.288
0.264
0.291
0.265
0.290
ETTh1
0.454
0.451
0.467
0.460
0.465
0.460
0.464
0.463
0.470
0.462
ETTh2
0.396
0.113
0.111 0.423
0.401
0.118
0,408
0.119
0.113
0.126
61Tml
0.401
0.409
0.411
0.417
0.406
0.415
0.403
0.411
0.414
0.418
ETTm2
0.295
0.331 0.300
0.335
0.298
0.332
0.292
0.328
0.294
0.331
ILI
1.973
0.894
2.221 0.912
2.173
0.950
2.173
0.947
2.266
0.974
ECL
0.194
0.299
0.199
0.302
0.196
0.301
0.199
0.299
0.198
0.298
Traffic
0.618
0.333
0.624
0.336
0.622
0.335
0.622
0.336
0.627
0.335
forecasting tasks including Weather, ETTh1, ETTml and ILI. As detailed in Table [ the removal of any component leads decrease in performance, confirming the value of each design element. Additionally, we explore the use of the MINE estimator (Hjelm et al. 2019a) instead of the Jensen-Shannon MI estimator in our main paper, with further details provided in Appendix A.12 Lastly, we showcase various case studies to demonstrate the enhancements facilitated by our method in Appendix A.4 and explore template variations in Appendix A.5
4.6.1 Mutual Information.
We further explore the mutual information module from a representation learning perspective, following the findings in Wu al. (2023). They adopt CKA (Centered Kernel Alignment) metric which measures similarity between representations obtained from the first. and last layer of model and they find that forecasting and anomaly detection benefits from high CKA similarity, contrasting with that imputation and classification tasks benefits from lower CKA similarity.
Experiments are conducted using the MSL dataset for the anomaly delection task, the Weather datasel for forecasting, the ETTh1 dataset for imputation, and the PEMS-SF dataset for classification. As depicted in Figure | the removal of components our method results decreased CKA similarity in anomaly detection and forecasting tasks, but an increase imputation and classification tasks. This observation further substantiates the effectiveness of our components.
4.6.2 Sample Reweighting.
Regarding the sample reweighting module, we illustrate the behavior of the learned weighting network in Appendix 1.12 The trend confirms our hypothesis: sample weight wo increases with the prediction loss lo, and weight W/ decreases as lo increases. This pattern validates our sample reweighting module. Further discussion comparing this module to a fixed weight scheme are presented in Appendix A.12
To verify the effectiveness of our method, we conduct ablation studics focusing on (1) traditional time series (TS) models and (2) language models.
4.6.3 Traditional Models.
primary model, framework is applicable to other traditional models. We explored additional traditional models including ETSformer (Woo el al. 2022), Stationary (Liu et al. 2022b), and FreTS (Yi et al. 2023). As shown in Table integrating LLM- TS generally enhances performance across all traditional modols, underscoring the bonofits of our method.
Although utilize TimesNet
4.6.4 Language Models.
In the main paper, the LLaMA-3b model (Touvron al.) 2023) used generate emboddings for the TS language description. We compare with GPT2 (Radford et al. 2019) and BERT (Devlin et al.] 2019)
to assess different embeddings performance. Table reveals that I.LaMA-3b generally outperforms the
10

---

Published in Transactions on Machine Learning Research (04/2025)
alternatives, and all LMs improve results compared to non-LLM approaches, validating the effectiveness of LLMMTS Inlegrator.
Table 6: Ablation results on different traditional models. Full results are Appendix A.12
Methods
ETStormer
|ETS
LLM-TS
Stationary
Stat LLM-TS]
FreT8
Fre'T'S LLM-TS
Metric
MSE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAB
MSE
Weather
0.313
0.382
0.307
0.375
0.282
0.307
0,284
0.309
0.262
0.308
0.255
0.302
0,799
0.684
9.791
0.678
0.667
0,382
0.633
0.572
0.484
0.473
0.478
0.466
ETTml
0.638
0.583
0.555
0.528
0.527
0.477
0.522
0.471
0.415
0.422
0.407
0.415
ILI
3.740
2.722
1.011
2.205
0.985
3.158
Table 7: Ablation results on different LLM embeddings. Full results are in Appendix A.12
Mothods
LLM-TS (LLaMA)
LLaMA w/o template
GPT2
BERT
No LLM
Metric
MSE
MAF
MSF
MAE
MSE
MAR.
MSR
MAE
MSE
MAF
Weather
0.257
0.285
0.263
0.288
0.261
0.287
0.260
0.287
0.261
0,290
ETThl
0.434
0.451
0.465
0.460
0,464
0.438
0.467
0.460
0.467
0.460
mT
0.401
0.409
0.406
0.415
0.406
0.413
0.412
11.1
1.973
0.894
2.173
0.950
2.169
0.906
2.193
0.962
2:221
0.942
5 Related Work
5.1 LLM for TS Modeling.
FPT (Zhou et al. 2023 suggests utilizing pre-trained language models to extract features from time series for improved predictions. TIME-LLM (Jin et al. 2024) and TEST (Sun et al. 2023) adapt LLMs for general time series forecasting by maintaining the original language model structure while reprogramming the input to fit time series data (Zhou et al. 2024). LLMTIME (Gruver et al.) 2023b) interprets time series sequences of numbers, treating forecasting a next-token prediction task akin text processing, applying pre-trained LLMs for this purpose. Given that it is not a state-of-the-art method and primarily targets zero-shot forecasting, it has not been incorporated into our experimental framework. TEMPO (Cao et al.) 2024) utilizes essent.aal inductive biases of the TS task for generative pre-trained transformer models. Gao et al. 2024]; Goswami et al. (2024 also explore time series foundation models by pre-training large models on extensive time series datasets.
5.2 Time Series to Text.
PromptCast (uue and Salim 2023 proposes to transform the numerical input and output into prompts, which enables forecasting in sentence- teaeeett-seee to-seneence manner. Time-LLA (Jin et al. 2024 incorporates background, instruction and statistical information of the time series data via natural language to facilitate time series forccasting in LLM. LLMTIME (Gruver et al.) 2023a) converts time scries data into a string of numbers and predicts future values as if completing a text. AuloTimes (Liu et al. 2024) maps time series into the embedding space of language tokens, enabling autoregressive generation future predictions.
5.3 Mutual Information
The Infomax principle (Linsker 1988 Bell and Sejnowski 1995), applied in the context of neural networks, advocates for maximizing mutual information ion between the inputs and outputs of a network. Traditionally, quantifying mutual information was challenging outside a few specific probability distributions, as discussed in Shwartz-Ziv and Tishby| (2017). This complexity led to the development of various houristics and approximations (Tishby al. 2000). Recently, a breakthrough came with MINE (Belghazi al. 2021). which introduced neural estimator capable of assessing mutual information between two arbitrary quantities
11

---

Published in Transactions on Machine Learning Research (04/2025)
with precision that depends on the capacity of the encoding network. This innovative approach has spearheaded advancements in the field of representation learning (Hjelm et al. 2019b Sylvain et al. 2020 The estimator we utilize is based on the Jensen-Shannol divergence variant of the MINE mutual information estimator.
5.4 Sample Reweighting.
Sample reweighting is commonly used to improve training efficacy (Fang et al. 2023 Wang et al.
2023b Yuan
let al., 2023a Zhang et al. 2023 Yuan et al., 2023b). Traditional approaches (Freund and Schapire 1997;
Sun et aL. 2007) assign larger weights to samples with higher loss values, as these hard samples have greater learning potential. Recent studies (Ren et al. 2018) suggest using a validation set to guide the learning of sample weights, which can enhance model training. Notably, meta-weight-net (Shu et. al. 2019) proposes learning a mapping from sample loss to sample weight. In this work, we adopt an MLP network that takes
sample prediction loss as input and outputs dual weights for prediction loss and mutual information loss.
6 Conclusion and Discussion
In conclusion, the LLM-TS Integrator framework offers promising approach to integrating Large Language Models (LLMs) with traditional TS methods. By encouraging high mutual information between textual and TS data, our method aims to maintain the distinct characteristics of TS while benefiting from the advanced pattern recognition capabilities of LLMs. The introduced sample reweighting module enhances performance by dynamically adjusting the relevance of each sample based on its predictive and informational contributions. Comprehensive empirical evaluations suggest that this framework improves accuracy across various TS tasks, including forecasting, anomaly detection, and classification. However, further rescarch is necessary to confirm these results across more diverse and robust datasets including Monash (Godahewa et al. 2021) and GIFT-Eval (Aksu et al. 2024). and to explore additional LLM features that could further enrich our model. Additionally, it is important to acknowlodge current limitations, such as the need for computational resources and the potential challenges in aligning the two modalities effectively.
References
Ahmed Abdulaal, Zhuanghua Liu, and Tomer Lancewicki. Practical approach to asynchronous multivariate time series anomaly detection and localization. In Feida Zhu, Beng Chin Ooi, and Chunyan Miao, editors, KDD) '21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, Singapore, August 14-18, 2021, pages 2485 2494. ACM, 2021.
doi: 10.1145/3444755833467777 URL
https://doi.. org/ 10.
Taha Aksu, Gerald Woo, Juncheng Liu, Xu Liu, Chenghao Liu, Silvio Savarese, Caiming Xiong, and Doyen Sahoo. GIFT-eval: A benchmark for general time series forecasting model evaluation. In NeurIPS Workshop on Time Series in the Age of Large Models, 2024. URL http:////pppaeeeeeeeeeeeeeerrttr
Anthony Bagnall, Hoang Anh Dau, Jason Lines, Michael Flynn, James Large, Aaron Bostrom, Paul Southam, and Eamonn Keogh. The uea multivariate time series classification archive, 2018. ArXiv preprint, abs/1811.000755, 2018. URL httpss:/arxiv..r. wwwp///g////......///////////aee
Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeswar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and R Devon Hjelm. Mine: Mutual information neural estimation, 2021.
Anthony J Bell and Terrence J Sejnowski. An blind deconvolution. Neural computation, 7(6): 1995.
approach to blind separation and
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranay Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krucger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Ben- jamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, lly Sutskever, and
12

---

Published in Transactions on Machine Learning Research (04/2025)
Dario Amodei. Language models are few-shot learners. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell,
Balcan, and Hsuan-Tien Lin, editors, Advances in Neural Information
Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NewrIPS 2020, December 6-12, 2020, virtual, 2020. URL httpps:/pproceeddngg http://poooedddn...uuuus...... htwpe//pss.ccpppppr///////aaeaee
Defu Cao, Furong Jia, Sercan O Arik, Tomas Pfister, Yixiang Zheng, Wen Ye, and Yan Liu. Tempo: Prompt-based generative pre-trained transformer for time series forecasting, ICLR, 2024.
html CDC. Illness. https: http:///aaaaaaaaaa////s////tt/ta 2024. Online; accessed 10 August 2024.
Cristian Challu, Kin G. Olivares, Boris N. Oreshkin, Federico Garza Ramírez, Max Mergenthaler Canseco, and Artur Dubrawski. NHITS: neural hicrarchical interpolation for time scries forccasting. In Brian Williams, Yiling Chen, and Jennifer Neville, editors, Thirty-Seventh AAAT Conference on Artificial Intelligence, AAAI 2023, Thirty-Fiifth Conference on Innovative Applications of Artificial Intelligence, LAAI 2023, Educational Advances in Artificial Intelligence, EAAI 2023, Washington, DC,
Thirteenth Symposium USA, February 7-14, 2023, pages 10lgg//1..11 10. (a/00//aaa..... v3716. 25854
AAAI Press, 2023. doi: 10.1609/
URL
https://dan .rg.
Can Chen, Xi Chen, Chen Ma, Zixuan Liu, and Xue Liu. Gradient- bi-level optimization for deep 11719
learning: A survey. ArXiv preprint, abs/2207.11 11719, 2022a. URL
Can Chen, Chen Ma, Xi Chen, Sirui Song, Hao Liu, and Xue Liu. Unbiased implicit feedback via bi-level optimization. arXiv preprint arXiv:2206.00147 2022b.
Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Balaji Krishnapuram, Mohak Shah, Alexander J. Smola, Charu C. Aggarwal, Dou Shen, and Rajeev Rastogi, editors, Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13-17, 2016, pages 785 794. ACM, 2016. doi: 00.1145/2939672.2937777 URL https://doi.ors /twgg///... 1et.1111522996922 2939785
Angus Dempster, François Petitjean, and Geoffrey T Webb. Rocket: exceptionally fast and accurate time series classification using random convolutional kernels. Data Mining and Knowledge Discovery, 34(5): 1454 1495, 2020.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio, editors, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, Min- neapolis, Minnesota, 2019. Association for Computational Linguistics. doi: 10.18653/v1/N// 1423. URL attps://aclanthologg hww.illooyy.oggg9999 org/N19- 1423
Tongtong Fang, Nan Lu, Gang Niu, and Masashi Sugiyama. Generalizing importance weighting to A universal solver for distribution shift problems. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurlPS 2023, New Orleans, LA, USA, December 10- 16, 2023, 2023. URL http:// hwtp///appppppppp/ee nips. cc/paper files/ haaeee/pppprr/2//aaatat Abstract- www.ooonneeeeennnooccco
Jean-Yves Franceschi, Aymeric Diculeveut, and Martin Jaggi. Unsupervised scalable representation lcarning for multivariate time series. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelziner, Florence d' d'Alché Buc, Emily B. Fox, and Roman Garnett, editors, Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, Docember 8-14, 2019, Vancouver, BC, Canada, pages 4652 4652-4663, 2019. URL http:s://prcceeeaiii hatl/SScderi22eee2222aaaccaassba
neurips wttpipps.ccppppp//pe/ cc/paper/2019/
13

---

Published in Transactions on Machine Learning Research (04/2025)
Luca Franceschi, Michele Donini, Paolo Frasconi, and Massimiliano Pontil. Forward and reverse gradient- based hyperparameter optimization. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceodings of Machine Learning Research, pages 1165-1173. PMLR, 2017. URL
http://proceedingg mlr. ppppsss//O/fffraccccchh1/3 orrppppeeeeiiiigppppooeeeeerrrri
Yoav Freund and Robert E Schapire. A decision-theoretis generalization of on-line learning and an application to boosting. Journal of computer and system sciences, 1997.
Milton Friedman. The interpolation of time series by related series. Journal of the American Statistical Association, 1962.
Jingkun Gao, Xiaomin Song, Qingsong Wen, Pichao Wang, Liang Sun, and Huan Xu. Robusttad: Robust time series anomaly detection via decomposition and convolutional neural networks. ArXiv preprint, 09545
abs/2002.09545. 2020. URL
Shanghua Gao, Teddy Koker, Owen Queen, Tom Hartvigsen, Theodoros Tsiligkaridis, and Marinka Zitnik. Units: A unified multi-task time series model. Advances in Neural Information Processing Systems, 37: 140589-140631, 2024.
Rakshitha Godahewa, Christoph Bergmeir, Geoffrey I Webb, Rob J Hyndman, and Pablo Montero-Manso. Monash time series forecasting archive. ArXiv preprint, abs/2105.06645 2021. URL https://arxiv.orz
abs/2105.06645
Mononito Goswami, Konrad Szafer, Arjun Choudhry, Yifu Cai, Shuo Li, and Artur Dubrawski. Moment: A family of open time-sories foundation models. ArXiv preprint, 005/2402.03885, 2024. URL https:
wwtrrv..rrgabb//440000 03885
Nate Gruver, Mare Finzi, Shikai Qiu, and Andrew Gordon Wilson Large language models are zero-shot time series forecasters. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Anmual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023a. URL htttp://papers. nips. httpp/pee..lllee/epeeeeeeeeeeeee files/paper/22233 1ee1a228001177111000132226221122 Cowpsttttact-oooooeeee...e.e html
Nate Gruver, Mare Finzi, Shikai Qiu, and Andrew Gordon Wilson. Large language models are zero-shot time series forecasters. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023b. URL http://papers. nips . cc/paper_filessppppeeeeeeeeeeeep /ttpr/eeaaaaeee/22////////////// /hash/ 2a877aa22e2001777761122223322223 oorr html
Albert Gu, Karan Goel, and Christopher Ré. Efficiently modeling long sequences with structured state spaces. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.ner 2022. URL https: hwt.://wwwwee.ooommmmddoaooooooo
R. Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon Karan Grewal, Philip Bachman, Adam Trischler, and Yoshua Bengio. Learning deep representations by mutual information estimation and maximization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.nee net, 2019a. URL https:://openrevien wttp://tt....ooor/ddd/.r/.......
R. Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Philip Bachman, Adam Trischler, and Yoshua Bengio. Learning deep representations by mutual information estimation and maximization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.nel, 2019b. URL
Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey Meta-learning in neural networks: A survey. IEEF transactions on pattern analysis and machine intelligence, 2021.
14

---

Published in Transactions on Machine Learning Research (04/2025)
Zhaoyang Huang, Xiaoyu Shi, Chao Zhang, Qiang Wang, Ka Chun Cheung, Hongwei Qin, Jifeng Dai, and Hongsheng Li. Flowformer: A transformer architecture for optical flow. In European conference on computer vision, pages 668 683. Springer, 2022.
Kyle Hundman, Valentino Constantinou, Christopher Laporte, Tan Colwell, and Tom Soderström. Detecting spacecraft anomalies using istms and nonparametric dynamic thresholding, In Yike Guo and Faisal Farooq, editors, Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2018, London, UK, August 19-23, 2018, pages 187-395. ACM, 2018. doi: 10.1145/3219819 3219845. URL
/219999 32198445
Rob J Hyndman and George Athanasopoulos. Forecasting: principles and practice. OTexts, 2018.
Yushan Jiang, Zijie Pan, Xikun Zhang, Sahil Garg, Anderson Schneider, Yuriy Nevmyvaka, and Dongjin Song. Empowering time series analysis with large language models: A survey. ArXiv preprint, abs hls/2402.031186 2024. URL htttt:///axiiiii org/ 2a..b///4420000 03182
Ming Jin, Shiyu Wang. Lintao Ma, Zhixuan Chu, James Y Zhang, Xiaoming Shi, Pin-Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, et al. Time-llm: Time series forecasting by reprogramming large language models. ArXiv preprint, abs/ 0bs22010.11728, 2023. URL https::/aaxxv..rrg/ 2000///000/01/0222
Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Y. Zhang, Xiaoming Shi, Pin-Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, and Qingsong Wen. Time-LLM: Time series forecasting by reprogramming large language models. In The Twelfth International Conference on Learning Representations, 2024. URL
https://openreviii http:////pppaeeeeeeeeee/e//artaa
Nikita Kitacv, Lukasz Kaiscr, and Anselm Levskaya. Reformer: The efficient transformer. In 8th Intornational Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenRe- veew.net, 2020. URL http:s//opperrrreeeeeeeeffrrrrrn
Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long- and short-term temporal patterns with deep neural networks. In Kevyn Collins-Thonee Thompson Qiaozhu Mei, Brian D. Davison, Yiqun Liu, and Emine Yilmaz, editors, The 41st International ACM SIGIR Conference on Rescarch & Development in Information Retrieval, SIGIR 2018, Ann Arbor, MI, USA, July 08-12, 2018, pages 95 104. ACM, 2018. doi: 10. 001144/3209978321100
URL http::h/t//oo org/10.
Bryan Lim and Stefan Zohren. Time-series forecasting with deep learning: a survey. Philos. Trans. Royal Soc. A, 2021.
Ralph Linsker. Self-organization in perceptual network. Computer, 21:105-117, 1988. URL https
Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X. Liu, and Schahram Dustdar. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. Open- net:/forummdd--EEmf--a Review.net 2022a. URL httttt::////t/rreeee httpp////..pppeeeeeeeeee/eee//ea
Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Non-stationary transformers: Exploring the stationarity in time series forecasting. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Bel- grave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 December 9, 2022, 2022b. URL http://papers wip.. hat1/000455ffcc3340067766aaa222 oaraa/a0000.aa222112222tttttaa:e Abstract- Core html
Yong Liu, Guo Qin, Xiangdong Huang, Jianmin Wang, and Mingsheng Long. Autotimes: Autoregressive time series forccasters via large language models. ArXiv preprint, abs/2402.02370,, 2024. URL https
Aditya Mathur and Nils Ole Tippenhauer. Swat: a water treatment testbed for research and training on TCS security. In CySWATER, 2016.
15

---

Published in Transactions on Machine Learning Research (04/2025)
Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time scries is worth 64 words: Long-term forecasting with transformers. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023, OpenReview net, 2023. URL https://
openreview. wwwp://etpppdddddddeddd
Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. I-gan: Training generative neural sam- plers using variational divergence minimization. In Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Roman Garnett, editors, Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pages 271-279, 2016. URL httpps:/pproceeding newrips cc/papper/211//aaa.. html
Boris N. Oreshkin, Dmiiri Carpov, Nicolas Chapados, and Yoshua Bengio. N-BEATS: neural basis ex- pansion analysis for interpretable time series forecasting. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. URL
PeMS. Traffic.
2024. Online; accessed 10 August 2024.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, el al. Language models are unsupervised multitask learners. OpenAl blog, 2019.
Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. Learning to reweight examples for robust. deep learning. In Jemnifer G. Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pages 1931-430.. PMLR, 2018. URL http: 7/proceedings. laaiggssllll.. mwr. wwt.spee0//eeeee...... html
Jun Shu, Qi Xie, Lixuan Yi, Qian Zhao, Sanping Zhou, Zongben Xu, and Deyu Meng. weth-weight-net: Learning an explicit mapping for sample weighting. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d'Alché-Buee Emily B. Fox, and Roman Garnelt, editors, Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 1917-1928, 2019. URL https:
Ravid Siwartz-Ziv and Naftali Tishby. Opening the black box of deep neural networks via information, 2017.
Spyros Makridakis. M4 dataset, 2018. URL httppp///ttta...oo///////ttttt/t
master/Dataset!
Ya Su, Youjian Zhao, Chenhao NIu, Rong Liu, Wei Sun, and Dan Pei. Robust anomaly delection for multivariate time series through stochastic recurrent neural network. In Ankur Teredesai, Vipin Kumar, Ying Li, Rómer Rosales, Evimaria Terzi, and George Karypis, editors, Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, Anchorage, AK, USA, August 4-8, 2019, pages 2828 2837. ACM, 2019. doi: 0101455//2225003333
URL https:
//doi . org/ro../// 10.
3330672
Chenxi Sun, Yaliang Li, Hongyan Li, and Shenda Hong. Test: Text prototype aligned embedding to activate Ilm's ability for time series. ArXiv preprint, 0t///2308..22// 2023. URL https://aaxiiv http///axxiv..o//bbbbaaa/ 2308
D8241
Fan-Yun Sun, Jordan Hoffmann, Vikas Verma, and Jian Tang. Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. URL https:://peereviiee wttp://rwweeeeeeeeeoooorrerereee
Yanmin Sun, Mohamed S Kamel, Andrew KC Wong, and Yang Wang. Cost-vensitive boosting for classification of imbalanced data. Pattern recognition, 2007.
16

---

Published in Transactions on Machine Learning Research (04/2025)
Tristan Sylvain, Linda Petrini, and R. Devon Hjelm. Locality and compositionality in zero-shot learning. In 8th International Conference on Learning Representations, TCLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. URL attps://openrevieww http://rweeeeeeeetttorrrrradeeee VONKwr
Mingtian Tan, Mike Merrill, Vinayak Gupta, Tim Althoff, and Tom Hartvigsen. Are language models actually useful for time series forecasting? Advances in Neural Information Processing Systems, 37:60162 60191, 2024.
Naftali Tishby, Fernando C. Pereira, and William Bialek. The information bottleneck method, 2000.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine- tuned chat models. ArXiv preprint, abs abs/2337.09288,, 2023. URL /ttpi/ttpss///rvv..ooo/////////
UCL Electricity. https://archive.iee wtteeeeiss....cucueeeeeeee http://aaaaareee..ee...eee/eaa/a ott.e///w//laattaaaaaacccaaaaaaa
2015. Online; accessed 10 August 2024.
Shiyu Wang, Haixu Wu, Xiaoming Shi, Tengge Hu, Huakun Luo, Lintao Ma, James Zhang, and JUN ZHOU. Timemixer: Decomposable multiscale mixing for time series forecasting. In ICLR, 2023a.
Zitai Wang, Qianqian Xu, Zhiyong Yang, Yuan He, Xiaochun Cao, and Qingming Huang, A unified generalization amalysis of re-weighting and logit-adjustment for imbalanced learning. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Sacnko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurlPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023b. URL hnttp://papers.niin niw.. cc/ paper
html
Qingsong Wen, Linxiao Yang, Tian Zhou, and Liang Sun. Robust time series analysis and applications: An industrial perspective. In Aidong Zhang and Huzefa Rangwala, editors, KDD '22: The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington, DC, USA, August 14 18, 2022, pages 4836-4837. ACM, 2022. doi: 15444//3354477.34426222 URL https: http:::://oo.or//1000/ oro//o.. 1145/
Wetterstation. Weather.
mpg. de/ de/wetter/ 2024. Online; accessed 10 August 2024.
Gerald Woo, Chenghao Liu, Doyen Sahoo, Akshat Kumar, and Steven C. H. Hoi. Etsformer: Exponential smoothing transformers for time-series forecasting. ArXiv preprint, abs/2202.01388,, 2022. URL https
//arxiv.org//ass/2202..
Haixu Wu, Jichui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. In Marc' Aurelio Ranzato, Alina Beygekziee, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan, editors, Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pages 22419-22430, 2021. URL https//////// heapiiinnnnnnuuuispeeeeee neurips...... cc/ Bapp:/20hhhaas//ccc0000777eeeee2 http::/aaaaa.cacbbbbbaattttetaaa
Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.nor .ent,, 2023. URL
httpp:://pperrevev..o
?id=ju [03j340g
Jiehui Xu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Anomaly transformer: Time series anomaly detection with association discrepancy. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. URL https://opeereviies neet:/trrweieeeeeee/e/eee net/ forum Lid=L2Q088UUqmm
Hao Xue and Flora D Salim. Promptcast: new prompt- based learning paradigm for time series forecasting. TEEE Transactions Knowledge and Data Engineering, 2023.
17

---

Published in Transactions on Machine Learning Research (04/2025)
Kun Yi, Qi Zhang, Wei Fan, Shoujin Wang, Pengyang Wang, Hui He, Ning An, Defu Lian, Longbing Cao, and Zhendong Niu. Frequency-domain mips are more effective learners in time series forecasting. In Alice Oh, Tris- tan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 16, 2023, 2023. URL http::/papprr..nntt mips. cc/ paper wwwles/paper//ee 2tw///223hhhhhhaaa httaa//aeeeeeeeeeeerrorrrraaaaa. http://///.p/////..////////////. Sateaaa--bbbtaatt-Coooto Confreeect html
Huining Yuan, Hongkun Dou, Xingyu Jiang, and Yue Deng. Task-aware world model learning with meta weighting via. bi-level optimization. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Sacnko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023a. URL http://papers. nips . cc/paper htc/paperrfilespeeeeeeeeeeee http:/pappr//22/////// 119560000133654411866caa03 htt.://eccccccoooooooooo----orr-
Obstract-Conffeee wwwnnerrreee... html
Ye Yuan, Can Chen, Zixuan Liu, Willic Neiswanger, and Xue (Stove) Liu. Importanco-aware co-teaching for offiie model-based optimization. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orlcans, LA, USA, December 10 16, 2023, 2023b. URL http://paperr. nips. cc/paper fwwesllllppppeee http//ppper2////aa/ttte html
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Brian Williams, Yiling Chen, and Jennifer Neville, editors, Thirty-Seventh AAAT Conference on Artificial Intelligence, AAAI 2023, Thirty-Fifth Conference Innovative Applications of Artificial Intelligence, LA 2023, Thirteenth Symposium Educational Advances in Artificial Intelligence, EAAT 2023, Washington, DC, USA, February 7-14, 2023, pages 11121 11128, AAAI Press, 2023. doi: 10.1609/ AAAL 23719.26317. URL attps:///ol org/10 10. 1609/aaati aaai. 23719.26317
Tianping Zhang, Yizhuo Zhang, Wei Cao, Jiang Bian, Xiaohan Yi, Shun Zheng, and Jian Li. Less is morc: Fast multivariate time series forecasting with light sampling-oriented mlp structures. ArXiv preprint, abs/2207.0112 01186, 2022. URL https://arxiv org/abs/////eeeeoooeeg.gogea abs /207 01186
Xiaoying Zhang, Jumpu Chen, Hongning Wang, Hong Xie, Yang Liu, John C. S. Lui, and Hang Li. Uncertainty- aware instance reweighting for off-policy learning. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, nips. cc/paper_ files/ pwtps//ppppe////e/ea// USA, December 10- - 16, 2023, 2023. URI. Conferenceeeccneeeeoeoecceoo hawp:////oabbaaaaaa-oooooeeeeeo html e9 8874c2222750002226666444888888 htt.r///-asabbbo--aaaa-r-------e Abstract-
Hao Zhou, Chengming Hu, Ye Yuan, Yufei Cui, Yili Jin, Can Chen, Haolun Wu, Dun Yuan, Li Jiang, Di Wu, et al. Large language model (Ilm) for telecommunications: A comprchensive survey on principles, key techniques, and opportunities. ArXiv preprint, abs/2405.10825 2024. URL atttps://arxxr org/abs/
10825|
Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang, Informer: Beyond efficient transformer for long sequence time-series forecasting. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, TAAI 2021, The Fleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, pages 11106 11115. AAAI Press, 2021a. URL https://ojs.. aaai org/index php/ /A///AAAA///Aaa http///A//aatiieeeeee////a
Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty- Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, pages 11106-11115. AAAI Press, 2021b. URL https://ojs.... aaai werg/indxx,... hianeerwhhh//A///ttrccll///e////
18

---

Published in Transactions on Machine Learning Research (04/2025)
Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato, editors, International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Procoedings of Machine Learning Research, pages 27268 27286. PMLR, 2022. URL battps://proceedingg. mlr. press/ v162/zhou21) hhmml
presss
Tian Zhou, Peisong Niu, Xue Wang, Liang Sun, and Rong Jin. One fits all: Power general time scries analysis by pretrained LM. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Sacnko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference Neural Information Processing Systems 2023, NewrPPS 2023, New Orleans, LA. USA, December 10- 16, 2023, 2023. URL http::/papeer: nips cc/paper files/papen hatpe//2222//aaaas html
A
Appendix
A.1
Mutual Information Recalculation
Recall that mutual information can calculated using the equation:
100 3) = (1) E
al..all(Ta (()),,
(8)
where Ta signifies the discriminator characterized by parameters 3, and sp denotes the softplus function. Notably, (xx t) symbolizes sample from the dataset S, while (ž, represents a different sample from the dataset $ S.
This formulation presumes uniform distribution of samples. However, we have already computed probabilities pj for each sample, which introduces non-umiform distribution. For a batch of N samples, the expected value is computed as
Es[ (o = > (h))
(9)
Sex (T) = ΣΣ "6 sp() (4)
(10)
1) of Σ. Exe, p
adjusting for the non- non-uniform distribution of sample probabilities. As pf is produced from the weighting network a, we can also write I(0,3) as ((0,,a)).
Here, fd is defined as
A.2 Experimental Settings
Following Shu al. 2019), the weighting network comprises two-layer MLP with hidden size of 100, and set the learning rate 7/2 for this network at 0.001. The learning rate no of the discriminator is set 0.001 at the first epoch and then decreases to 0.0001 for the rest of epochs.
A.3 Standard Error Results
Table g presents the results along with standard errors to underscore the consistency and reliability of our method's 's performance.
19

---

Published in Transactions on Machine Learning Research (04/2025)
LLM-TS
TimesNet
GPT4TS
1.0
1.0
1.0
T
0.5
MY
0.
0.5
0.0
0.0
0.0
-0.5
-0.5
-1.0
GroundTruth
GroundTruth
GroundTruth
100
150
100
150
100
Figure 5: ETTh1
LLM-TS
TimesNet
GPT4TS
Prediction
Prediction
Prediction GroundTruth
3
GroundTruth
GroundTruch
-2
-2
-2
100
200
400
100
300
400
100
300
400
Figure 6: ETTml
20

---

Published in Transactions on Machine Learning Research (04/2025)
Table 8: Ablation results with standard error.
Meshods]
Ours
w/o mutual
w/o reweighs
TemceNet
Metric
MSE
MAE
MSE
MAE
MSB
MAE
MSE
MAE
ETThi Wenther
96
0.166 + 0.001
0.217
0.001
0.168 0.002 0.218 0.003
0.181 = 0.002
0.232 0.001
0.171
0.221
192
0.229 + 0.002
0.269 + 0.002
0.227 + 0.002 0.268 0.002
0.230 + 0.001
0.270 + 0.002
0.235 + 0.001
0.272 + 0.002
336
0.278
1: 0.001
0.302
6.002
0.298 1: 0.002 0.318 0.002
0.283 1 0.002
0.306
1:0661:00 0.001
0.285 1 0.002
0.307 1. 0.001
720
0.351
0.361 0.001
0.361
0.001
0.355
0.365 - 0.001
0.358
96
0.403 + 0.003
0,420
+ 0.002 0.402 = 0.002 0,422 0.001
0.408 + 0.002
0.428
0.
0.411
0.431 0.002
192
0.440 1: 0.005
0.441 0.002 0.450 :: 0.003 0.455 0.003
0.460 + 0.003
0.460 : 0.002
0.463 +: 0.006
0.456 J: 0.003
336
0.471
1: 0.003
0.457
6,002 0.471 + 0.003 0.457 0.002
0.492 + 0.002
0.474
0.002
0.487 1: 0.904
0.466 1: 0.003
720
0.503 + 0.003
0.187
0.002 0,535
0.002 0.507
00418 = 0.003 0.478
0.004
0.517 + 0.002
0.191 0.002
Br7ml
96
0.329 + 0.008
0.371
0.004 0.341 + 0.006
0.377 0.006
0..350 + 0.006
0.387 0. 003
0.3440 + 0.006
0.377 0.004
192
0.380
1. 0.005
0.398
0.002 0.404 1. 0.006 0.413 0.003
0.383 1. 0.006 0.397 1. 0.003
0.406 1. 0.007
0.408 J. 0.002
336
0.418
+: 0.002
0.125
0,002 0,432 = 0.003 0. 0.428 0.001
0.410 + 0.002
01.111 :: 0.002
0.421 + 0.003
0.425 0.002
720
0.476
1: 0.005
0.003
0.468
0.005
0.002
(0.467
0.004
0.448
0.002
00.485
0.006
0.004
24
1.921
+ 1.1.11
0.808
0.019
2.170 + 0.100 0.947 0.023
1.934 + 0.098
0.925
0. 0.020
2.072 + 0.122
0.948 0.015
T1.1
36
2.151
1. 0.035
0,933
0.020 2.083 1 0.089 0.889 0.024
2.505 1 0.103
1.020 1 0.010
2.494 1. 0.072
1.019 1 0.004
48
2.062
+: 0.052
0.892
0.011 2,118 = 0.031 0.959 0.008
2.325 + 0.116
0.948
: 0. 030
2:298 + 0.038
0.964 =: 0.006
1.759
1: 0.124
0.035
2:203
0.105
1.926
0.088
0.896
0.023
2.198 0.040
0.963
A.4 Showcases
To provide a clear comparison among different models, we showcase the forecasting task results on ETThl (96-96) and ETTml (96-336) using three models: LLM-TS, TimesNet, and GPT4TS. As shown in Figures 5 and G our LLM-TS model produces significantly more accurate predictions, demonstrating its effectiveness.
To illustrate the performance improvements achieved by the LLM-TS Integrator framework, we introduce a case study. We created a training set with a weighted sine function:
Σ Wi sin (ftt + PP: + (0,
(11)
where wi = 0.1, w2- = 0.2, wy = 0.3,
0.4; f-
1, P3 2, 4 55: p,
P1 = 3; and c = 0.1 is the noise level. We generated a long sequence of length 000 and then sampled a batch of size 64 with a sequence length of 96 and a prediction length of 336 to train GPT4TS, TimesNet, and LLM-TS on this data for 1, 000 iterations. For testing, we created a test set with frequency f = 20 which is greater than max(f1, fz. f4), and used p = 2.5, w = and e = 0.1. As shown in Figure 7 Figure an Figure g we can know:
1
GPT4TS fails to accurately capture periodic information as it relies solely on a language model without incorporating traditional mathematical modelling.
TimesNet generally captures periodic information due to the use of the FFT mathematical operator, but it still does not perfectly match the ground truth.
LLM-TS captures periodic information and better matches the ground truth by integrating rich language model insights into the traditional TimesNet model.
This case study highlights how the LLM-TS Integrator framework benefits from both inherent properties of traditional TS models and pattern recognition abilities of LLMs, demonstrating the effectiveness of our approach.
A.5 Template Variation
We conducted additional experiments on the ETTh1 dataset for long-term forecasting with GPT2. The original template achieves Mean Squared Error (MSF) of 0.464 and a Mean Absolute Error (MAF) of 0.458.
21

---

Published in Transactions on Machine Learning Research (04/2025)
GPT4TS
Input
1.0
Ground Truth
Prediction
0.5
Values
0.0
-0.5
-1.0
100
200
300
400
Time Steps
Figure 7: GPT4TS on synthetic data
TimesNet
Input
1.0
Ground Truth
Prediction
0.5
Values
0.0
-0.5
-1.0
100
200
300
400
Time Steps
Figure 8: TimesNet on synthetic data
22

---

Published in Transactions on Machine Learning Research (04/2025)
LLM-TS
Input
1.0
Ground Truth
Prediction
0.5
Values
0.0
-0.5
-1.0
100
300
400
Time Steps
Figure 9: LLM-TS synthetic data
We tested variations of the template by changing the original context from 'The Electricity Transformer Temperature is a crucial indicator in electric power long-term deployment." to:
"The temperature of the electricity transformer is vital metric for long-term electric power deployment."
Variation 1:
Variation 2: 'Monitoring the temperature of electricity transformers essential for the long-term deployment of electric power.'
Variation 3:
'The temperature of electricity transformers serves a key indicator in the long-term deployment of electric power.'
Variation No template.
Besides, we also consider the following changes:
w/o Input Statisties: excluding input statistical data from our analysis.
w /o Mean, Max, Median: remove mean, max and median informatino.
w/o Lags: remove lags information.
The performance of these variations is summarized in Table g These results indicate that the performance is quite similar across different variations, supporting the robustness of our approach regardless of minor template modifications. For further details on the template implemennaatinn, refer to our code repository at https://anonymous. wttpr/ouuu.ooo..........mmm www.ssceeeeeeeene.eeeeeeee ooo.aaoosss--7777/tttttstooooooo
A.6 Model Efficiency Analysis
Compared to TimesNet, our LLM-TS inlegralor introduces additional costs due to the mutual information and sample weighting modules. However, after training, the inference cost of our method is the same as TimesNet. We detail the time cost of each component for ETTh1 and ETTml tasks, using batch size of on 32G V100 GPU. As shown in Table IO the training of our method reasonable, given that it achieves the best performance across most tasks.
23

---

Full Results of Short-term Forecasting Published in Transactions on Machine Learning Research (04/2025)
Table 9: Performance across different template variations (standard error)
Template Variation
MSE
MAE
Original Template
0.464 0.002
0.458 + 0.003
Variation
0.460 + 0.003
0.456 + 0.002
Variation
0.465 0.003
0.460 + 0.003
Variation
0.464 0.002
0.459 + 0.002
Variation (No template)
0.466 0.003
0.460 = 0.003
w/o Input Statistics
0.468 + 0.002
0.462 + 0.002
W w/o Mcan/Max Max/Modian
0.465 0.002
0.459 + 0.002
w w/o Lags
0.467 0.002
0.460 + 0.003
It important note that use the pre-trained LLM obtain the text embeddings only once. These embeddings can then be used throughout the training process. For instance, obtaining the embeddings for the ETTh1 dataset using the Ilama-3b model on A100 GPU takes approximately 1 hour. We acknowledge that embedding extraction may become more time-consuming and memory-intensin with significantly larger or multivariate datasets. In such scenarios, strategics such as prioritizing difficult or representative data points could be employed to manage computational costs effectively. After this, the embeddings are utilized in our framework train the model, and the final output of the TimesNet model. This ensures that the inference time of method is identical that of the TimesNet model.
As detailed in the TimesNet paper, backbone model TimesNet is relatively small with 0.067 MB parameters. For comparison, other models have the following sizes: Non-stationary Transformer has 1.884 MB, Autoformer has 1.848 MB, FEDformer has 2.9 MB, LightTS has 0.163 MB, DLinear has 0.296 MB, ETSformer has 1.123 MB, Informer has 1.903 MB, Reformer has 1.157 MB, and Pyraformer has 1.308 MB. The introduced mutual information network consists of only two linear layers of size 64r64 and 64x4096, which is negligible in terms of additional paramctors. Similarly, the introduced MLP network consists of four layers: 100, 100 x 1, 1 x and 1 1, and the number of parameters is also negligible.
Thus, our model remains very small and efficient, with inference time identical to TimesNet (as the mutual information component is only used during training). Given that many TS models are primarily used for inference, our approach offers effective performance gains with minimal additional computational cost.
From a theoretical standpoint, let da denote the dimension the weighting network parameters and de denote the dimension of the prediction network parameters. Denote the computation cost of Eq. (6) c(da. dg). According Section 4 of Franceschi et al. (2017), the time complexity of the bi-level optimization is O(c(da, do)), with the primary computational cost arising from the Jacobian-vector product in Eq. This explains the time complexity of the sample reweighting module.
Table 10: Cost Comparison per step(s).
Methods
Owezll TimesNet Mutual Information Sumple Reweighting
ETTbl
3.177
0.128
0.577
2.474
Weather
5.563
0.436
1.084
4.033
A.7
Table displays the comprehensive results for short- forecasting.
A.8 Full Results of Long- Term Forecasting
Full results for long-term forecasting are presented in Table 12
24

---

Published in Transactions on Machine Learning Research (04/2025)
Table 11: Full results of short-term forecasting.
Melhods
LLM-TS
TinesNet GPTITS TIME-LLM PatcITST N-HITS N-BEATS FEDiomeer Stationary Autoformer
Yearly
SMAPE
13.309
13.512
13.531
13:419
13.477
13.418
13.436
13.717
13.974
MASE
3.021
3.065
3.015
3,0050
3.019
3.045
3.043
3.045
3,078
3.134
OWA
0.789
0.799
0.793
0.789
0.792
0.793
0.794
0.803
0.807
0.822
Quarterly
SMAPE
10.020
10.009
10:177
10:110
10.38
10.202
10.124
10.792
10.958
11.338
MASE
1.102
1,178
1.194
1.178
1,233
1.194
1.109
1.263
1.325
1,365
OW:
0.878
0.867
0.898
0.889
0.921
0.899
0.586
0.956
0.961
1.012
Monchby
SMAPE
12,696
12,783
12.891
12,980
12,959
12.791
12.677
14,260
13.917
13.958
MASE
0.986
0.949
0.956
0.963
0.970
0.969
0.937
1.102
1.067
1.103
OW
0.880
0.889
0.906
1.012
1.002
Ofherr
SMAPE
4.916
4.954
1.900
4.795
4.962
5.061
4.925
4.951
6.302
5.485
MASE
3,310
3.364
3.347
3,236
3,391
3.865
OW A
1,089
1.062
1.029
1,005
1,049
1,040
1.053
1.036
1,304
1.187
Average
SMAPE
11.819
11.908
11.983
12,059
11.851
12,909
MASE
1,888
1.612
1,600
1,595
1,623
1,618
1.599
1.701
1.756
1,771
OW A
0.851
0.860
0.861
0.830
0.869
0.861
0.855
0.918
0.930
0.939
A.9
Full Results of Imputation.
Table
contains the detailed results of our imputation tasks.
A.10
Full Results of Classification
Table
contains the comprehensive results for classification.
A.11
Full Results of Anamoly Detection
Full results for anamoly delection are detailed in Table 15
A.12 Further Ablation Studies
MLP. Given that our template comprises only simple summary statistics of the time series and the original values, we investigated whether the LLM component is essential. To this end, we replaced the LLM with an MLP that accepts the normalized concatenated statistics (mean, max, median, lags) and outputs an embedding of icantical dimension to the LLM. We conducted experiments on ETTh] and ETTml. For ETTml, the MSE increased from 0.401 0.420, and the MAE increased from 0.409 to 0.425. Similarly, for ETTh1, the MSE increased from 0.454 to 0.478, and the MAE increased from 0.451 to 0.463. We also attempted using a single-layer self-attention module instead of the MLP. However, the performance remained suboptimal. On ETTm1, the MSE increased from 0.401 to 0.423 and the MAE from 0.409 to 0.419; similarly, on ETThl, the MSE increased from 0.454 to 0.475 and the MAE from 0.451 to 0.461. These results demonstrate that the LLM component crucial in effectively capturing the mutual information from the template, thereby justifying its use in our approach. Tan et al. (2024) demonstrate that using L.LSs as prediction backbones may not yicld substantial improvements; however, our results highlight the effectiveness of LLM embeddings when integrated as auxiliary information within traditional time-series models.
Mutual Information Estimator. In the main paper, utilize the Jensen-Shannol mutual information (MT) estimator. Additionally, we explore the Mutual Information Neural Estimator (MINE) (Hjelm et al. 2019ah. We evaluate both estimators on two lasks, ETTh1 and ETTml, with results averaged over four prediction lengths. For ETThI, the MSE and MAE using the original Jensen- Sensen-Shannon estimator are 0.454 and 0.451, respectively, compared to 0.460 and 0.457 with MINE. For ETTml, the MSE and MAE are 0.401 and 0.409 with the original estimator, and 0.402 and 0.410 with MINE. These comparisons highlight the robustness of our method across different mutual information estimators.
25

---

Published in Transactions on Machine Learning Research (04/2025)
Table 12: Full results for long-term forecasting. We use prediction length E {96, 192, 336, 720} except for ILII and O E {24, 36, 48, 60) for ILI. A lower MSE indicates better performance.
Methods
LLM-TS
TimesNet
TIME
LLM
DLinear
PatchTST
GPT4TS
FEDiormer
TEST
Stationary
ETSfmmeeer
Metric
MSP:
MAF.
MSR
MAR
MSR
MAR
MSF:
MAR
MSR
MAR]
MSR
MAR
MSP:
MAR
MSR
MAR]
MSF
MAF.
MSR
MAP.
0.166
0.217
0.117
0.224
0.202
0.239
0.196
0.255
0.116
0.227
0.196
0.231
0:217
0.296
0.2214 0.261
0.173
0.223
0.197
0.281
W Weather
192
0.2229
0.269
0.235
0.272
0.245
0.277
0:237
0.29%
0.234
0:265
0.241
0.271
0:276
0.336
0.262 0:298
0.245
0.285
0:237
0:312
0.2278
0.302
0.235
0.272
0.313
0.283
0.335
0.284
0.301
0.380
0.310
0.529
0.521
0.298
0.353
720
0.354
0.351
0.366
0.356
0.369
0.356
0.345
0.381
0.356
0.349
0.367
0.354
0.403
0. 0.428
0.378
0.370
0.414
0.410
0.352
0:288
Ang
00:257
0.285
0.226
0.250
0.277
0.296
0:265
0.317
0.265
0.285
0.275
0.292
0.309
0.360
0:251
0.315
0.288
0.314
0:271
0.334
96
00:103
0.120
00:114
0.431
0.114
0.122
0.386
0.400
0. 06060
0.467
0.009
0.115
0.376
0.119
0.411
0.426
0.513
0.191
0.491
0.119
E77m2 ET7mi ETTM BTTH
0.441
0.463
0.456
0.450
0.437
0.432
0.512
0.477
0.420
0.448
0.475
0.461
0.538
0.504
336
0..171
0.157
0.487
0.466
0.515
0.475
0:481
0.459
0.5:6
0.496
0.503
0.061
0.459
0. 0.165
0.508
0.482
0.588
0.535
0.574
0.521
0.5005
0.487
0.517
0.494
0.519
0.516
0.306
0.507
0.504
0.562 0.535
Ang
00.454
0.451
(0:470
0.462
0..474
0.459
0.456
0.452
0.516
0.484
0.443
0.451
0.440
0. 0.460
0.4476
0.466
0.570
0.537
[0:542
0.510
0.322
0,366
(0.310
0.374
0,330
0.553
0.333
0.387
0.300
0.355
0,298
0.350
0.358
0.397
10.328
0.374
0.476
0,458
00.3:0
0.:911
192
0.400
0.409
0..39
0.410
0,386
0.399
0:477
0.476
0.593
0.405
0.376
0,399
0.429
0.439
0.403
0.418
0.512
0.493
0.430
0.439
0.432
0.135
0.452
0:452
0,060
0.594
0.541
0.227
0:436
0,430
0,
0.496
0.187
0.455
0.551
0.485
0.479
720
0.440
0.442
03.422
0.468
0.442
0.445
0.831
0.657
0.446
0.450
0.428
0.461
D.463
0. 0.477
0.470
0:477
0.562
0.560
0:500
0.497
Ang
0.396
0.113
0:413
0.426
0.396
0.
0.559
0.515
0.391
0:411
0.383
0.410
0.437
0.149
0.414
0,432
0.526
0.516
0.439
0.452
96
0.332
0.371
00.340
0.377
0.399
0.398
0.345
0.372
0.3352
0.374
0.350
0.369
0.379
0.419
003336
0.373
0.386
0.358
0.375
0.338
192
0.396
0.406
0.408
0,412
0.405
0.380
0.389
0.390
0.393
0,387
0.387
0.426
0.441
0.381
0.399
0.459
0.414
0.608
0.410
336
0.4188
0.125
0.
0.424
0.425
0.412
0.425
0:413
0.113
0. 0.421
0.418
0.118
0.407
0.445
0. 0.439
0.411
0.418
0.195
0.464
0:455
0.128
0.478
0.440
0.485
0.461
0,502
0.457
0.474
0.453
0.449
0.477
0, 0.037
0.543
0.490
0.478
0.454
0,516
0.499
0.462
Ang
0.:00
0.409
00:114
0.418
0.137
0. 921
0.403
0.407
106
0.407
0.108
0.400
0:448
0. 0.452
004002
0.411
0.181
0.156
0.429
0.125
0.189
0,206
00.185
0.264
00.193
0.2 0.281
0.199
0.292
0.118
0.270
0,185
0.271
0.200
0.287
00:230
0.307
0.192
0.274
0.189
0.280
192
0.253
0,307
0.252
0.308
0,254
0.315
0.284
0.363
0.255
0.31d
0,250
0.312
0.209
0.328
0.266
0
0.280
0,339
0.253
0.319
336
0.315
0.345
0.333
0.350
0,320
0.355
0.369
0.427
0.309
0.347
0,314
0.351
0.325
0.306
0.340
0.370
0.334
0,061
0.914
0.357
720
0.421
0.408
0.415
0.403
0.426
0.116
0.554
0.522
0.412
0.404
0.110
0.408
0.421
0.115
0.436
0.420
0.417
0.113
0.414
0.413
Ang
0,2295
00:291
0.331
0.312
0.350
0.401
0.220
0
0.290
0.335
0.305
0.349
0.322
0.350
0.341
0.2293
0.:342
24
1.921
0,898
22:072
0.948
2,589
1.054
2.398
1.040
2.229
0.894
5,259
1.089
3.228
1.200
3.3771
1.201
2.294
0.945
2:527
1.020
36
2.151
0.933
22.494
1.019
2,996
1.194
2.646
1.088
2.330
0:025
6.136
1.831
2.679
1.0%0
3.725
1.322
1.825
0.848
2.615
1.007
11.
48
2.062
0,802
2.298
0.964
2.714
1.095
2.614
1.086
2.140
0.894
4.670
1.582
2.622
1.078
3.291
1.207
2.010
0.900
2:359
0.972
L.759
0.853
0.963
2,005
1.050
2,804
1.146
0:912
4,002
1.517
2.857
1.157
2.907
1.136
2.178
0.963
2.487
1.016
Ang
1,923
0,804
2.266
0.974
2,726
1.098
2.616
1.050
2.184
0.906
5.117
1.650
2.847
1.144
3.324
1.202
2.077
0.914
2.497
1.004
0.167
0.271
00.169
0.273
0.2007
0.282
0.197
0.2822
0.190
0:206
0,186
0.273
0.103
0.308
10:218
0.309
0.169
0.278
0.187
0.304
BCL
192
0.178
0,280
0.186
0.288
0,200
0.297
0.196
0.285
0.199
0.304
0,190
0.278
0:201
0.315
0.220
0.311
0.182
0.286
0.199
0.315
0.198
0.302
0.206
0.305
0.224
0.312
0:209
0.301
0.217
0.319
0,204
0.291
0.214
0.329
0.234
0.323
0,200
0.304
0.212
0.329
720
0.233
0,511
0.251
0.327
0.277
0.359
0:215
0.333
0.258
0.352
0.245
0.297
0.325
0.355
0.276
0.351
0.222
0.921
0:253
0.341
Ang
0.199
0.299
0.198
0.298
0.229
0.315
0.212
0.300
0.216
0.318
0.285
0:214
0.327
0.233
0.324
0.193
0.296
0.208
0.323
96
0.587
0.315
0.589
0.313
0.600
0.402
0.650
0.396
0.522
0.347
0.563
0.378
0.587
0.368
0..58
0.310
0.612
0.338
0:607
0.392
Timffe fe
192
0.612
0.326
0:627
0.337
0.586
0.382
0.008
0.370
0.522
0.332
0.540
0.367
0.604
0.373
0.567
0.380
0.613
0.340
0.621
0.309
336
0.634
0,338
0.635
0.341
0.593
0.390
0.605
0.373
0.517
0.334
0.566
0.376
0.621
0.383
0.563
0.389
0.618
0.328
0.622
0.:996
720
0.640
0.351
0.658
0.349
0.636
0.405
0.645
0.304
0.552
0.352
0.567
0.372
0.626
0.382
0.585
0.301
0.653
0.355
0.632
0.396
Ang
0.618
0.333
0.627
0.335
0.006
0.396
0.625
0.383
0.529
0.341
0.561
0.373
0.610
0.376
0.561
0.388
0.624
0.340
0.621
0.399
Average
00.574
0.427
[0:618
0.442|
006681
0.468
0.686
0.483
00.600
0:436
0.964
0.525
0.701
0.
0.480
10.766
0:491
0.633
0.465
[0:662 03473
10 12 and 13 display the learned weighting network applied to various datasets: MSL for anomaly detection, Weather for forecasting, ETTh1 for imputation, and PEMS-SP SF for classification. These visualizations corroborate our hypothesis: the sample weight wo increases with the prediction loss lo, while the weight W/ decrcases as to increasos. This observed pattern supports the efficacy of our reweighting strategy.
Sample Reweighting Illustration. Figures
Static Weighting Scheme. We also explore static weighting scheme as contrast to the dynamic weighting used in our sample reweighting module. This scheme balances the prediction loss and mutual information loss, with a ratio of 0.0 representing pure prediction loss and 1.0 representing pure mutual information loss. As shown in Table [8] the static approach underperforms relative to our dynamic sample weighting module, demonstrating the superior effectiveness of our method.
A.12.1 Comprehensive Results.
The detailed performance of various traditional TS models and LLMs is presented in Table 16 and Table 17
26

---

Published in Transactions on Machine Learning Research (04/2025)
Table 13: Full results for the imputation task. Randomly masked (12.5% 25%, 37.5%, 50%} of points in 96-length series, averaging results over 4 mask ratios.
Methoeks
LLM-TS
TimesNet
GPT4TS MAE
PatchTST MSD MAG
LigheTS
DLinner
FEDiormer
Stationary
Antodormmr
Reformmrr
Mask Ratio
MAE
MSE MAE
MSE MAE
MSE
MSE
MAE
MSE
MAE
MSE
MAE
MAE
12.5%
0.018
0.098
0.023
0.101
00018
0.089
0.041
0.130
0.093
0.206
0.090
0.133
0.005
0.166
0.052
0.119
0.06
0.344
0.042
0.146
2576
0.022
0.097
0.023
0.101
0.003
0.069
6.04
0.135
0.005
0.206
0.080
0.105
0.052
0.166
0.052
0.119
0.06
0.144
0.012
0.146
0.108
0,029
0.000
0.112
0.040
0.143
0.113
0:231
0.103
0.219
0.089
0.191
0.131
0037
0.161
0.182
50%
0.033
0.120
0.035
0.123
0.012
0.131
0.055
0.151
0.134
0.255
0.132
0.268
0.089
0.218
0.047
0.145
0.067
0.174
0.082
0.208
Aug
11,125
0.103
0.028
0.300
00025
0.308
0.047
0.140
0.100
182188
0.083
11,118
00,0022
0.177
0.335
0.126
0.051
0.130
0.055
0.166
0.018
0.079
0.019
0.078
O
0.034
0.127
0.055
0.159
0.068
0.003
0.062
0.008
0.228
25%
0.020
0.085
0.021
0.087
0.021
0.084
0.028
0.009
0.042
0.143
0.085
0.195
0.080
0.195
0.024
0.096
0.026
0.301
0.136
0.202
E73
87.5%
0.1122
0,089
0.025
0.022
0,024
0.050
0.000
0.164
0.001
0.159
0.106
11.222
0.110
0.231
0.027
0.303
0.000
0.308
0.176
0.300
50%
111125
0,09
0,025
0.058
0.059
0.174
0.156
0.277
0.030
0.035
0.119
0.329
Ang
[0.02
0.057
0.022
0.080
0.023
0.068
(0009)
0.102
0.046
0:151
0.096
0.205
0.101
0.215
0.035
0.000
0.020
0.305
0L1667
0.280
12.5%
0.055
0.185
0.084
0.170
0.043
0.141
000000
0.201
0.240
0.345
0.151
0.267
0.070
0.190
0.060
0.163
0.077
0.182
0.074
0,194
ECL BT7N2 E77h1
25%
1.117
0,082
0.150
0,265
0.364
0.106
01213
0,080
0.102
0.227
37.5%
0.209
0,058
0.074
0.182
0.120
0.296
03382
0.215
0,318
0.124
0.258
0,102
0.212
0.100
0.222
0.135
0.261
50%
0.118
0.225
0.136
0:225
0.104
0:2144
0.141
0.248
0.334
0.404
0.207
1.:44
0.1165
0.230
0.133
0.240
0.157
0:248
0.179
0.298
0.087
0.198
0.000
0.190
0.069
1.174
0.115
0.224
0.284
0.375
0.201
0.506
00.117
0.246
0.004
0.201
0.103
0:2/4
0.122
0.245
12.5%
0.0099
0.131
000000
0.041
0.12M
10.057
0.162
0.101
10:2311
0..10
012355
0.212
0.342
0.1:5
(.0144
0.388
0.163
0.289
25%
0.046
0.145
0.048
0.346
00046
1.1
[.06]
0.168
0.115
0.246
0.127
0.247
0.157
0.258
0.049
0.147
0.050
0.148
0.206
0.331
87.5%
0.055
0.154
0.055
0.156
0.053
0.148
0.067
0.166
0.126
0:267
0.158
11,278
0.118
0..304
0.036
0.158
0.000
0.163
0.202
0:370
0.165
0.068
Ang
0.148 0.050
(1:051
0.150
0.050
0.144
(0.065
0,163
0.119
0.250
0.142
0.259
0.116
0.279
0.152 0.053
(.005
0.156
0.204
0.352
12.5%
0.087
0.2033
0,000
0.201
0.080
0. 0.154
[0,055
0.160
0.102
0.223
0.082
0.214
00.107
0.235
0.003
0.210
(.089)
0:210
0.190
0.308
0.207
0/092
0.067
0.200
0.121
0.252
0.120
0.096
0.220
0.312
37.5%
0.095 0:218
0.005
0:213
0.004
0.211
0.076
0.344
0.141
0.273
0.144
0.276
0.136
0.286
0.102
0.220
0.104
0.229
0.903
0.315
50%
0.101 0.220
0.102
0.221
0.101
0.220
0.091
0,208
0.160
0,203
0.175
0.305
0.158
0.284
0.108
0.228
0.113
0:239
0.210
0.319
11.194
0.085
0.207
0.131
00,662
00.180
0.101
0.313
Waatee
12.5%
25%
0.026
0.119
0.048
0.035
0:031
0.025
0.062
0.047
0.027
0.000
0.010
0.054
[0,009
0.001
0.063
0.0:0
0.0:4
0.052
0.111
0:101
0.048
0.039
0.084
0.103
0.041
0.064
0.163
0.107
0.027
0.029
0.051
0.056
0.006
0.000
0.054
0.047
0.001
0.005
0.070
0.082
37.3%
0.052
0,039
0.054
0.061
0.034
0.062
0.005
0.058
0.058
0.121
0.057
0.117
0.107
0.229
0.053
0.052
0.032
0.060
0.040
0.091
30%
0.061
0,003
0.007
0.006
0.008
0.005
0.133
0,006
0.134
0.183
0.312
0.008
0.007
0.067
0.059
Ang
0.080
0.076
0.031
0.030
0.002
0.068
(.060
0.144
0.065
0.117
0.052
0.110
0.090
0.203
0.032
0.050
0.001
0.057
00088
0.087
Table 14: Complete classification task results. *, in the Transformers indicates the name of *former.
Methood
Cloniical XGB
RNN LSINNes
TCN
Transformere Auto. Staticn.
MLP
TimmsNNel
LLM GPT4IS
LLM-TS
Trans.
Rt.
In.
Pyra.
Flow.
TEST
Rahanol
43.7
45.2
39.9
$1.1
28.9
32.7
31.9
31.6
30.8
31.6
32.7
31.2
28.1
33.8
32.6
29.7
30.4
26.2
25.1
$1.9
FunDD
63.3
64.7
61.7
662.7
62.8
67.3
68.6
67.0
65.7
G8.4
68.0
66.0
666.3
67.6
68.0
67.5
68.6
67.8
500.1
68.9
HandW
58.8
27.4
32.5
29-4
32.1
20.1
HeatB
73.2
75.6
77.1
72.7
75.6
76.1
77.1
50.5
75.8
74.6
73.7
79.7
71.2
77.8
75.1
75.1
27.6
72.2
13.7
77.1
JapanV
86.0
96.2
98.1
98.4
96.9
96.7
97.8
98.9
98.4
96.2
98.2
98.4
95.9
98.3
96.2
96.2
97.2
98.4
78.4
98.1
PFMS
75.1
82.1
83.2
87.3
59.5
SCP1
84.6
84.0
90.8
84.6
92.2
90.4
30.1
83.1
81.0
89-4
58.7
80.6
92.5
87.3
89.8
90.4
90.1
$4.0
91.8
SCP2
48.9
53.5
52.8
52.2
55.6
53.9
56.7
55.5
53.5
30.6
57:2
54.4
55.0
56.1
50.5
51.1
57.1
50.0
54.4
57.5
SpokenA
69.6
100.0
100.0
95.6
58.4
97.0
100.0
98.5
100.0
200.0
100.0
98.8
100.0
98.6
82.1
UWive
76.9
91.4
87.8
83.9
88.4
85.6
85.6
85.6
83.4
65.9
87.6
85.3
85.0
86.6
82.1
80.3
85.5
85.6
84.4
86.6
Avg
66.0
72.5
71.8
70.9
70.3
71.9
71.5
72.1
70.8
71.1
72.7
70.7
71.0
13.0
67.5
70.4
72.7
60.5
61.2
73.4
Table 15: Full results for the anomaly detection.
Mechods
SMD
M15L
SMAP
SWi1l
PSM
F1
Metrics
R
X
LAM-TS
88.00
81.54
$4.60
80.04
74.49
$1.11
80.96
56.51
69.41
91.36
95.40
9/3.25
98.44
56.45
97.45
85.17
TimeNeet
87.0 37.93
61.45
86.57
55.
73. 18
80.34
39
56.35
89.18
91.00
96.33
05.12
98.
96.18
97.27
84.00
GPT-IS
87.70
81.19
94.32
82.15
81.32
81.73
90.04
55.75
68.86
92.12
98.06
92.89
98.37
96.33
97.34
84.97
PalcTTT
87.22 27.26
62.14
$4.62
88.32 34
70.96
78.70
90.64
65.46
68.82
91. 91.30
80.94
85.72
36.84
98.47
96.00
82.79
E ETSlomeer
87.44
79.22
53.13
80.13
84.114
85.013
52 22
05.75
69.50
50.002
80.
54.91
599.31
85.22
91.76
82..7
FFITformer
87.00
79.57
70.76
97.31
84.97
LighTS
87.10
78.42
62.55
82 40
75.78
78.05
92.55
55.27
69.21
91 18
94.72
03.53
98.37
95.97
97.15
84.25
DLinsar
83.002
71.52
77.10
$4.34
85.42
84.88
522-02
$5.41
09.25
50.91
565.20
87.52
368
890.26
93.55
82.36
Stationnry
81.21
08.55
89.14
77.50
97.
96.76
97.23
Autolormar
82.85
95.11
77:27
80.92
79.05
50.40
58.62
71.12
89.85
95.81
92.74
99.005
19.10
93.29
84.26
PyraSarmer
85.61
50.61
$3.04
83.51
65.03
84.96
92.54
57.71
71.09
87
57.92
96.00
91.78
71.67
56.02
62.05
82.57
Anemaly Transforme
88.90
62.23
85.49
79.61
87.37
83.31
91.85
58.11
71.18
72.51
97.32
83.10
68:35
94.72
79.60
80.50
Inforuecr
77.23
N1.65
81.77
86.48
84.06
90.11
67.13
09.92
70.29
96.75
81.43
64.27
96..34
77.10
79.88
Reformer
82.58
68.24
75.52
85.51
83.31
$4.40
90.91
67.44
70.40
72.50
96.63
82.80
50.03
96. 38
73.61
77.91
Tranformee
83.36
76.13
79.56
71.57
87.37
78.68
80.57
07.12
69.70
03.
96.53
80.37
02.75
96.06
76.07
76.98
27

---

Published in Transactions on Machine Learning Research (04/2025)
Table 16: Different traditional models. We use prediction length O E O E {24, 36, 48, 60} for others.
{96, 192, 336, 720} for ILI and
Methords
PatehTTST
|Pace/TST
INT
ETS
INT
SStationayy
Stat INT
Phr/T
FieTT INT
Metric
MSE
MAA
MSE
MAE
MSE
MAE
MSE
MAE
MSS
MAE]
MSE
MAE
MSE
MAE]
MSE
MAE
0.117
0.216
(0.172
0.214
0.196
0.282
00:200
0.285
00.178
0.226
0.201
0.187
0.243
0.117
0.235
E774 Wealher
192
0.222
0:258
0:219
0.255
0.282
0.364
0.278
0.361
0:235
0.278
0:238
0.280
0.227
0:274
0:221
0.278
336
0.250
0.298
0.277
0.298
0.344
0.400
0.382
0.3227
0.330
0.312
0.281
0.325
0.277
0.320
720
0.356
0.349
0.335
0.348
0.430
0.472
0.427
0.420
0,387
0.363
0.386
0.383
0.352
0.382
0.344
0.376
Avg
00.258
0.280
0.2257
0.279
0.313
0.382
(0.907
0.375
00,282
0.307
0.284
0.300
0.262
0.306
0.2255
0.302
96
0.9881
0.398
00,382
0. 10.01
0.554
0.536
0.550
0.532
0.534
199
0.325
0.186
0.398
0.412
0.395
0.409
1922
0.421
0.426
01,422
0.428
0.685
0.619
0.690
0.621
0.639
0.560
0.600
0.560
0.454
0.449
01.455
0.451
336
0.464
0.449
01460
0.441
0.809
0.730
0.868
0.728
0.700
0.648
0.780
0.634
0.512
0.483
0.502
0.474
720
0.527
0.500
0.510
0.486
1.083
0.849
1.054
0.830
0.700
0.620
0.701
0.606
0.572
0.547
(1.60)
0.530
Avg
0.448
0.443
01.444
0.442
0.790
0.684
0.7911
0.678
00.607
0.582
0.653
0.572
0.484
0.473
01.478
0.466
96
0.832
0.368
(0.3322
0.972
0.525
0.196
0:424
0.434
[0.117
0.117
0.412
0.110
0.3:0
0.375
00.339
0.371
BTT
192
0.368
0.388
0.367
0.388
0.545
0.538
0.458
0.461
(1.446
0.447
0.445
0.435
0.395
0.408
0.3884
0.359
336
0.397
0:405
01.396
0.405
0.638
0.603
0.537
0.519
0.582
0.507
0.570
0.191
0.431
0.433
01,120
0.423
720
0.457
0.445
(1.460
0.446
0.801
0.696
0.882
0.606
(1661
0.544
0.660
0.546
0.484
0.470
01.484
0.462
Avg
0.339
0.402
(1,380
0.403
0.638
0.583
(0.555
0.528
(0.527
0.477
0.522
0.471
0.415
0.422
01.407
0.415
24
2:229
0.801
2.172
0.856
4.043
1.410
3:607
1.305
2.722
1.024
1.900
0.872
3:226
1:251
33,202
1:213
36
2.330
0.925
2.347
0.978
3,800
1.358
337705
1.315
3.026
1.071
2.790
1.068
3.363
1:250
3.000
1.173
2:140
0.801
1.984
0.869
3.851
1.351
3.714
1.509
2.622
1.032
2.132
0.900
3.456
1:285
3.132
1:213
2.097
0.912
1.770
0.831
3.983
1.349
3.935
1.350
2.520
1.035
1:001
0.901
3.749
1.340
3.208
1:243
Asg
2.184
0.006
2.008
0.884
3.922
1.367
3.744
1.320
2.722
1.041
2.205
0.935
3.440
1.279
3.158
1.211
Table 17: Different LLM embeddings. We use prediction length O E {99,, 336, 720} for ILI and O e {24, 36, 48, 60} for others.
Methods| LLM-TS [LLaMA)
LLaMA w/o texi
GPT2
BERT
No LLM
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
96
0.166
0.217
0.170
0:218
0.168
0.218
0.166
0:217
0.168
0.218
BTPN1 Weather
0:229
0.227
0.267
0.229
339
0:278
0.302
0.205
0.314
0:202
0.310
0.288
0.305
0.208
0.318
720
0.354
0.351
0.300
0.354
0.359
0.354
0.360
0.354
0.361
0.356
Avy.
0:257
0.285
0.263
0.2888
0:261
0:287
0.2260
0.287
0:264
0.250
0.403
0.420
0.426
0.002
0.421
192
0.440
0.111
0.445
0.445
0.412
0.444
0.452
0.450
0.459
0. 0.455
0-471
0.400
0.472
0.487
0.467
0.494
0.457
720
0.503
0.187
0.518
0.496
0.517
0.494
0.497 0.446
0.520
0.535
0.507
Avg.
0.454
0.465
0.460
0.464
0.458
96
0.329
0.371
0.350
0.387
0.338
0.370
0.3300
0.375
0.311
0.377
k77ml
0.380
0.398
0.383
0.404
0.400
333
0.418
0.425
0.123
0.426
0.416
0.123
0.414
0.421
0.452
0.128
720
0/476
0.454
0.470
Avy.
0-:01
0.400
0.406
0.415
0.406
0.413
00.406
0.412
0.411
0.417
1.921
0.896
1.998
0.929
1.917
2.170
0.947
396
2. 2:151
0.933
2.422
0.957
2.333
0.958
2.433
1.0044
2.003
0.889
2.062
2.198
2.209
0.937
2.333
60
1.759
0.853
2.072
0.948
2:077
0.921
2.089
0.926
2:203
0.971
Avg.
1.979
0.894
2.173
0.950
2.
0.996
0.952
2:221
0.942
Table 18: Static Weighting Scheme with Different ratios.
Ratio 0.0
0.2
0.4
0.6
0.8
1.0
Ours
Motrie
MSE MAP| MSE MAE
MSE MAR MSR MAR] MSE MAR MSE MAR
MSR MAE]
EтTH1 0.478 0.468 (0.17
0.459
0,465
0.162
10.170
0.463
(0:733
0.150
0.471
0.463 0.454
0:451
ETTml 0.415 0.417
10.408
0.414
0.405
0.412
|0.406
0.412
(0.417
0.419
0.416 0.419
J0.401
0.409
28

---

Published in Transactions on Machine Learning Research (04/2025)
0.07
0.02510
0.0254
0.025
0.06
0.02505
0.05
0.0252
Weight wo
0.020
Weight W
Weight wo
Weight W
0.04
0.02500
0.0250
0.015
0.03
0.010
0.02495
0.02
0.0248
0.01
0.0246
0.005
0.02490
0.00
-2
-1
1
2
-2
2
Prediction loss lo
Prediction loss
Figure 10: Forecasting.
Figure 11: Anomaly detection
0.027
0.07
0.032
0.05
0.026
0.06
0.030
0.04
0.05
Weight (Jo
0.028
Weight W
Weight wo
0.025
Weight w/
0.03
0.04
0.026
0.024
0.03
0.02
0.024
0.023
0.02
0.01
0.022
0.01
0.022
0.020
0.00
0.00
-2
2
-2
1
2
Prediction loss I.
Prediction loss lo
Figure 12: imputation.
Figure 13: classification
29


Reviews:
[{"review_id": "RB2PW16wen", "reviewer": "Reviewer_QjYM", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n1. The idea of using mutual information criterion to align the two embedding spaces is novel\n2. The empirical results are good across 5 different tasks\n3. Ablation studies provide interesting insights into the methods\n\n**Weaknesses**\n1. A more thorough comparison with baselines and discussion about the differences with TEST would be appreciated\nIt would be great to see how the embeddings aligned within the TEST model (based on contrastive learning) compared to those obtained by the authors. The two methods seem similar and understanding what brings a substantial advantage to the proposed approach would be an interesting contribution.  \nSimilarly, the paper doesn’t use recent (universal) foundation models (Units, MOMENT, etc) that can all be applied to the considered tasks. It would be nice to have them or explain why the authors excluded them.\n2. In A.6 the authors provide the efficiency analysis for inference speed but not for memory footprint.\nI understand that for training, the embeddings from LLM can be obtained beforehand and the training can be done without loading LLM to the memory. However, extracting the embeddings from a large enough dataset can be very time-consuming and may incur a significant memory footprint. This gets even worse when dealing with multivariate datasets. I think a more extended discussion is necessary here. \n3. A recent work [1] showed that LLM-based methods are not very useful in time series analysis: they can often be replaced with standalone models with better or similar accuracy. It will be great to discuss why in this particular case, LLMs seem to bring the advantage. \n\n[1] Tan et al. Are Language Models Actually Useful for Time Series Forecasting? NeurIPS’24.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents a hybrid approach to time series analysis that couples the power of LLMs and that of tailored time series forecasting approaches based on mathematical modeling. In particular, the proposed approach suggests feeding a text prompt with time series data, its description, and some summary statistics to extract a hidden representation of this prompt from LLM. In parallel, a powerful TimesNet model is also fed with the same time series to produce its latent embedding. Both embeddings are then aligned using mutual information-based criterion with the strength of each embedding being determined with a weighting 1-layer fully connected network. The final representation is then used for such downstream tasks as forecasting, classification, anomaly detection, and imputation achieving strong results compared to existing baselines.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. I would greatly appreciate it if the authors addressed the concerns raised above (they are given in the order of priority). I would particularly like to understand how the proposed method (which I found novel and interesting, given that it doesn’t require the LLM during inference) compares to the field's current state, especially compared to foundation models (at least 1). \n2. I think it will be quite useful to modify Figure 1 to show which parts are fine-tuned/frozen during training. Similarly, it will be great to illustrate here that the LLM is not used during inference but during training (or just before) only. Readers may find it appealing for practical applications.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "tuHgGKYrtj", "reviewer": "Reviewer_XMZc", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros:\n\n1.The framework creatively fuses traditional time series models with LLM-derived insights. By maximizing the mutual information between numerical TS representations and their textual descriptions, it leverages the rigorous mathematical modeling of traditional approaches while tapping into the powerful pattern recognition capabilities of LLMs.\n\n\n2.The dual-module approach—combining a mutual information module with a novel sample reweighting mechanism—allows the model to adaptively balance learning from prediction errors and informational alignment. This leads to improved performance across diverse TS tasks such as forecasting, imputation, classification, and anomaly detection.\n\n3.By keeping the LLM component frozen, the framework avoids the computational overhead associated with fine-tuning large language models. This design choice makes the method more accessible and easier to integrate with existing TS models while still harnessing LLM insights.\n\n4.The introduction of a sample reweighting module—where weights for prediction loss and mutual information loss are learned through bi-level optimization—ensures that each sample contributes effectively to the learning process. This adaptive mechanism helps the model focus on samples that need more learning, thereby improving overall model efficiency.\n\nCons:\n1. The combined optimization of mutual information and prediction loss, especially with the bi-level optimization for sample reweighting, adds layers of complexity to training. Tuning the additional hyperparameters and ensuring stable convergence can be challenging. The authors have not provide detailed time complexity analysis both theoretically and empirically.\n\n2.Accurately estimating mutual information between TS and textual representations is nontrivial. While the Jensen-Shannon estimator (and alternatives like MINE) is used, such estimators can be sensitive to the choice of architecture, learning rates, and data distribution. Misestimation might negatively affect the model’s performance.\n\n3.The framework generates textual descriptions for time series data using a pre-defined template. The quality and informativeness of these generated texts are crucial. If the template does not capture the full complexity of the TS data, the resulting LLM-derived features might not be as beneficial.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "It introduces the LLM-TS Integrator framework, which fuses traditional time series (TS) models with the pattern recognition strengths of large language models (LLMs) by maximizing the mutual information between TS representations and their textual descriptions.\n\n• It proposes a new sample reweighting module. This module assigns dual weights—one for the traditional prediction loss and another for the mutual information loss—optimizing sample importance via bi-level optimization, thereby enhancing overall information utilization.\n\n• It validates the framework through extensive experiments across five key TS tasks (short-term and long-term forecasting, imputation, classification, and anomaly detection). The results show that LLM-TS Integrator consistently achieves state-of-the-art or comparable performance compared to both traditional TS methods and recent LLM-based approaches.\n\n• It achieves these improvements while keeping the LLM component frozen (i.e., without fine-tuning), which minimizes additional computational costs and facilitates easy integration with existing TS models.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The authors are required to include detailed time complexity analysis both theoretically and empirically.", "broader_impact_concerns": "n/a", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "mGz2Ozn3Vk", "reviewer": "Reviewer_zRNN", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n- The proposed approach provides consistent improvements to the prediction accuracy compared to the original forecasting model (TimesNet). The method can, in principle, be combined with arbitrary TS models that produce an intermediate embedding, which is a big plus.\n- The paper is well written and easy to follow. The proposed method is presented very clearly and the relevant concepts are introduced, making the paper accessible to readers with different backgrounds.\n- The experimental evaluation includes a large number of ablation studies that investigate the effect of different model components on the final performance.\n\n**Weaknesses:**\n- The empirical evaluation covers a large number of tasks but primarily focuses on a handful of benchmark datasets that have been shown to have serious limitations, as well as omits relevant baselines such as statistical models (see, e.g., Table 4 / Section 3.1.2 in [Hewamalage et al., 2023](https://link.springer.com/article/10.1007/s10618-022-00894-5)). I understand that at this point the authors are unable to re-run all the experiments using different benchmarks, but I would urge them to consider alternatives such as the [Monash repository](https://arxiv.org/abs/2105.06643) or [GIFT-Eval](https://arxiv.org/abs/2410.10393) in the future.\n- The visualizations of forecasts produced by baseline models shown in Figures 7 and 8 look very disturbing. The forecasts are significantly less accurate than, e.g., a seasonal naive forecast, which could indicate some serious problems with the implementation of the baselines.\n- There are several minor points related to the presentation that need to clarified / improved. Please see \"Requested changes\"", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a generic technique for improving the accuracy of time series models by integrating side information from LLMs. The proposed method consists of two components. First, a mutual information (MI) based loss is used to maximize the MI between the embedding of the TS model and the LLM that processes a textual description of the task. Next, a weighting module is used to dynamically balance the contribution of the MI loss and the prediction loss of the TS model. The proposed framework can be combined with any TS model that produces intermediate embeddings of the time series. Experimental evaluation on forecasting, anomaly detection, imputation and classification tasks demonstrates the consistent improvements produced by the LLM-TS framework compared to the original TimesNet model.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please clarify the following aspects in the revised version of the manuscript.\n\n**Critical:**\n\n1. Please include the results for the DHR-ARIMA baseline from [Hewamalage et al., 2023](https://link.springer.com/article/10.1007/s10618-022-00894-5) to the tables showcasing the forecasting performance (Section 4.2)\n2. Why do the ablation experiments in Section 4.6 only consider 4 datasets (Weather, ETTh1, ETTm1, ILI) and omit the 4 datasets (ETTh2, ETTm2, ECL, Traffic) used in previous experiments?\n3. How is the ablation \"w/o template\" performed?\n\n**Nice to have:**\n\n4. Section 3.1.1: How is the discriminator $T_\\beta$ defined?\n5. Section 3.2: Please clarify the meaning of the prediction loss $l_O$ used in different tasks.\n6. Why are none of the related models described in Section 5.2 (e.g., Time-LLM, LLMTIME) are not considered in the evaluation? Just a description in the text would be sufficient.\n7. Given that the template used in this work only contains simple summary statistics of the time series and the original TS values, it would be interesting to investigate whether the LLM component is necessary at all. As an ablation, one could replace the LLM with an MLP that takes an input the appropriately normalized concatenated statistics rrom the template and outputs the embedding.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper tackles a very timely problem which consists of leveraging the expressiveness of time series models with the power of large language models. The major contribution of the paper can be summarized as follows: the same time series is fed to both the time series model and the LLM to generate emdedding based representation which can then be used for multiple downstream tasks. The idea is simple yet effective and the strong experimental results across 5 different time series tasks show that the proposed framework can capture the embedded knowledge within the time series. 

The provided rebuttal was appreciated well by the reviewers and there was a clear consensus among the reviewers for an accept. I would like to ask the authors to consider the reviews and their provided rebuttal into account when preparing the camera ready version of the manuscript.

I would like to congratulate the authors on the nice work and the acceptance.
Decision: Accept (poster)