LangTime: A Language-Guided Unified Model for Time Series Forecasting with Proximal Policy Optimization
Authors: Wenzhe Niu
Zongxia Xie
Yanru Sun
Wei He
Man Xu
Chao Hao
Venue: ICML 2025 poster (2025)
Keywords: Time Series
Large Language Model
Reinforcement Learning
Primary area: deep_learning->sequential_models_time_series

Abstract: Recent research has shown an increasing interest in utilizing pre-trained large language models (LLMs) for a variety of time series applications. However, there are three main challenges when using LLMs as foundational models for time series forecasting: (1) Cross-domain generalization. (2) Cross-modality alignment. (3) Error accumulation in autoregressive frameworks. To address these challenges, we proposed **LangTime**, a **lan**guage-**g**uided unified model for **time** series forecasting that incorporates cross-domain pre-training with reinforcement learning-based fine-tuning. Specifically, LangTime constructs Temporal Comprehension Prompts (TCPs), which include dataset-wise and channel-wise instructions, to facilitate domain adaptation and condense time series into a single token, enabling LLMs to understand better and align temporal data. To improve autoregressive forecasting, we introduce TimePPO, a reinforcement learning-based fine-tuning algorithm. TimePPO mitigates error accumulation by leveraging a multidimensional rewards function tailored for time series and a repeat-based value estimation strategy. Extensive experiments demonstrate that LangTime achieves state-of-the-art cross-domain forecasting performance, while TimePPO fine-tuning effectively enhances the stability and accuracy of autoregressive forecasting.

LangTime: A Language-Guided Unified Model for Time Series Forecasting with Proximal Policy Optimization
Wenzhe Niu
Zongxia Xie
Yanru Sun 1
Wei He
Man Xu
Chao Hao
Abstract
Same Value but Different M Meanings
Different Distribution
Recent research has shown an increasing inter- est in utilizing pre-trained large language mod- els (LLMs) for variety of time series applica- tions. However, there are three main challenges when using LLMS foundational models for time series forecasting: (1) Cross-domale gen- eralization. (2) Cross-modality alignment. (3) Error accumulation in autoregressive frameworks. To address these challenges, we proposed Lang- Time, a aangaagegguidee unified model for time series forecasting that incorporates cross-domain pre-training with reinforcement learning-base -based fine-tuning. Specifically, LangTime constructs Temporal Comprehension Prompts (TCPs), which include dataset-wise and channel-wise instruc- tions, to facilitate domain adaptation and con- dense time series into a single token, enabling LLMS to understand better and align temporal data. To improve autoregressive forecasting, we introduce TimePPO, reinforcement learning- based fine-tuning algorithm. TimePPO mitigates егror accumulation by leveraging multidimen- sional rewards function tailored for time series and a repeat- repeat-based value estimation strategy. Ex- tensive experiments demonstrate that LangTime Time achieves state-of-the-an cross-domain forecast- ing performance, while TimePPO fine-tuning ef- fectively enhances the stability and accuracy of autoregressive forecasting.
0,581.
0.229.
Temperature
0.136,
Electricity 0.50T
0,766,
Consumption 0.136,
Weather
Electricity
(a) Challenge Cross-domain generalization.
Natural I. anguage:
Understand
capital
No Aligmment Needed
LLM
Time Series:
Confused
Difficult to Align
L Language G uided (ours)
The Air Tempesture is 1/WWM Your preditionn.s.
Close Understand
Trained on corpora
Easy Aligm
(b) Challenge 2: Cross-modality alignment.
Bad Prediction
A Auooregressive Model
2
4
Reliable History Observations
Less Reliable Inpur
(c) Challenge 3: Error accumulation.
Figure I. Challenges of Applying LLMs to Time Series. (a) Cross- domain generalization: Different domains possess their unique characteristics and numerical implications. (b) Cross-modality alignment: LLMs struggle directly comprehend unseen time series as they are trained on language data, whereas language guidance can assist in enhancing their understanding. (c) Error accumulation: In autoregressive prediction, using the output from previous step as input is considered unreliable.
1. Introduction
sessment (Baffour et al., 2019), energy sustainability (Yu et al., 2023b), and weather forecasting (Yu et al., 2025; Sun et al., 2021). The rapid development of machine learn- ing has driven significant advances time series forecast- ing(Yu et al., 2023a; Shao et al., 2024; Sun al., 2025). Recently, large language models (LLMMs) (Radford, 2018) have demonstrated remarkable capabilities in capturing se- quential structures and patterns, which is crucial for model- ing -eeedeeenn data in time series forecasting. Given the similarities between time series and natural language in
Time series refers sequences of data points indexed in discrete-time order (Box al., 2015), and they com- mon in real-world applications, such as financial risk as-
Equal contribution 'Tianjin University, Tianjin, China "Meituan, Beijing, China "Xiaohongshu, Beijing, China. Cor- respondence to: Zongxia Xic <caddicxie@ com..
Proceedings of the nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

LangTime: A Language-Cuided Unified Model for Time Series Forecasting with Proximal Policy Optimization
sequence modeling, several approaches have successfully leveraged pre-trained language models for time series fore- casting, yielding promising outcomes (Xue & Salim, 2023; Gruver et al., 2024; Pan et al., 2024; Jia et al., 2024).
dictions, improving long-term forecasting accuracy. Extensive experiments demonstrate that LangTime achieves state-of-the-art performance on widely recog- nized benchmarks and exhibits strong transferability unseen domains. Our code are publicly available at: https: http///gttttbb com wwwiiwwwLagggh ime.
However, as shown in Figure 1, arising from the inherent differences between time series and natural language, apply- ing LLMs time series forecasting presents three primary challenges. Firstly, cross-domain generalization. Unlike natural language with rules, time series exhibit diverse statistical patterns across domains. Moreover, identical values may carry domain- specific meanings, hindering effective multi-domain inte- gration into LLMs. To address this challenge, some meth- ods leverage semantic information embedded in datasets as prompts, enabling LLMs effectively differentiate and adapt to various domains (Liu et al., 2024d). Secondly, cross-modality alignment. Pre-trained on extensive text corpus, LLMs exhibit limited capacity for directly under- standing time series, thereby rendering cross-modal align- ment a formidable challenge (Zhou et al.. 2023). While models concatenate the two modali- ties, the lack of meaningful interaction hinders seamless alignment (Jin et al., 2023).
2. Related Work
structural/semantic
2.1. Large Language Models for Time Series
LLMs have shown considerable promise through pre- training on data spanning various domains (Doddapaneni et al., 2021; Taylor et al., 2022; Zhan et al., 2024). Nonethe- less, foundational models for time series face substantial challenges in aligning multi-domain data due to differences in channel numbers, sampling frequencies, and patterns. Existing works address this issue using tailored strategies. TTM (Ekambaram et al., 2024) employs frequency prefixes, Lag-Llama (Rasul et al., 2023) uses lag features as covari- ates for probabilistic univariate forecasting, UniTime (Liu et al., 2024d) introduces masking and domain instructions to mitigate convergence imbalances, and ROSE (Wang et al., 2024) adopts frrqueenc- bbased masking and reconstruction to unify cross-domain representations.
In addition, error accumulation in autoregressive frame- works. For tasks with varying prediction horizons, autore- gressive models allow a single framework to handle multiple horizons, avoiding the need for separate training protocols (Liu et al., 2024ffe; Yu al., 2024). However, supervised training is limited optimizing a model's 's ability to pre- diet the next step based actual historical observations. As result, it fails to alleviate the adverse effects of error accumulation in autoregressive prediction.
While multi-domain approaches focus on aligning time se- ries data across sources, recent methods explore integrat- ing textual information to further enhance forecasting ca- pabilities, introducing new challenge of cross-modality alignment. Directly concatenating time series and language tokens often lead to modality gap due to structural and se- mantic differences. To address this, CALF (Liu et al., 2024c) applies knowledge distillation, 12IP-LLM (Pan et al., 2024) aligns semantic and time series spaces with tokenization and anchors, and TimeCMA (Liu et al., 2024a) uses prompt- based techniques to extract time series representations.
To address these challenges, we propose LangTime, a language-guided unified model for time series forecasting that incorporates cross-domain pre-training and reinforce- ment learning-basee based fine-tuning. Specifically, we design Temporal Comprehension Prompts (TCPs) integrate se- mantic information and channel details to help LLMs dif- ferentiate time series across various domains. Additionally, we condense the time series data into a single token and in- troduce a reconstruction task to enhance the understanding of the temporal patterns of L.M.. Furthermore, we propose TimePPO, a fine-tuning algorithm based on Proximal Policy Optimization (PPO), which mitigates error accumulation during testing and improves long-term forecasting perfor- mance. Our contributions are summarized as follows:
However, existing methods address these challenges in isola- tion, leaving gap in simultaneously solving multi-domain and cross-modality alignment. LangTime bridges this gap by tackling both challenges concurrently. By facilitating robust cross-modality alignment, LangTime achieves supe- rior performance in time series forecasting across diverse domains.
2.2. Reinforcement Learning from Human Feedback in Large Language Models
Reinforcement Learning from Human Feedback (RLHF) has gained widespread application and demonstrated re-
We propose Tine, an autoregressive model that in- tegrates Temporal Comprchension Prompts to provide domain and -pannfc- information, enabling LLMs to better understand and forecast time series.
markable success in Natural Language Processing (NLP)
tasks. For example, InstructGPT (Ouyang et al., 2022) uses human preference data to train a Reward Model (RM), which is subsequently employed to fine- tune the supervised policy using the Proximal Policy Optimization (PPO) algo-
We introduce TimePPO, novel fine-tuning algorithm that alleviates error accumulation in autoregressive pre-
2

---

TineTime: A Language-Cuided Unified Model for Time Series Forecasting with Proximal Policy Optimization
TS Dependencies
Multi-task Pre-training
TimePPO Fine-tuning
Extraction
Special Token
rrccontruuct + pprediction
Masked Token
م Time Series Rupresentation
Reconstruct
MLP
MLP
MLP
.96
192
Prediction
Temporal Encoder
Pre-trained Large Language Model
Pre-trained LangTime
-EMM
Norm & Embodding
Word Embedding
Projection & Mask
Word Embedding
Word Embedding
Value: V
Roward: Re
Patching
Domain Description
Time Series
Compress Guidance
Prediction Guidance
From diverse domains and lengths
information of given time series is
Time Series Representation
Compress <EMB),
Predict value:
<OUTP.
Eymospo
(GAF(Rg V
Truw Hold
Depredtccton
Temporal Comprehension Prompts(TCPs)
Figure 2. Overview of LangTime: (1) The Temporal Encoder extracts temporal dependencies from time series data of varying lengths and domains, generating unified time series representations. (2) Temporal Comprehension Prompts are constructed by integrating time series representations with demain-specific context, enabling LLMs perform temporal pattern compression. This pre pre-training phase establishes modality alignment by jointly optimizing reconstruction and prediction tasks. (3) During fine-tuning, the TimePPO algorithu is implemented to mitigate error accumulation in autoregressive forecasting. thereby enhancing long-term prediction. The content of the prompt has been simplified in the figure, with the detailed information provided in Section 3.3 and Appendix A.2.
rithm (Schulman et al., 2017). This framework effectively enhances model performance by aligning outputs with hu- man preferences. However, the requirement to train and optimize reward model introduces significant computa- tional overhead, making PPO resource-intensive in practical scenarios.
where N is the number of datasets. For downstream task in dataset j. the model is fine-tuned on D and tested on D The datasets for pre-training, fine-tuning, and testing are pairwise disjoint to ensure generalization.
3.1. Architecture
Despite the success of RLHF in NLP, its application to time series tasks remains underexplored, especially for autore- gressive models. Time series forecasting introduces unique error accumulation challenges that require significant adap- tation of existing RLHF methods. To bridge this gap, we adapt the PPO algorithm the specific demands of time series prediction tasks. We redesign the reward function and value function to better capture the temporal structure and mitigate error propagation, simplifying the algorithm while enhancing its applicability to complex temporal data. These innovations make PPO more efficient and effective for autoregressive time series forecasting, addressing chal- lenges such computational complexity and scalability in long-term predictions.
LangTime includes three novel components: the Temporal Encoder (TE), Temporal Comprehension Prompts (TCPs), and Time Series Proximal Policy Optimization (TmmePPO), as illustrated in Figure To extract meaningful represen- tations, LangTime employs the TE to process continuous time series. To bridge the gap between time series and language, TCPs encode essential contextual information, abling LLMs to effectively differentiate and interpret time series data. The processed time series representations and contextual information are then passed into a pre-trained LLM. To further improve forecasting stability, Lang Time incorporates TimePPO, a reinforcement learning based fine- tuning strategy designed to mitigate error accumulation and enhance multi-step prediction. These components work to- gether to ensure effective alignment between time series and language and enhance long-term forecasting performance.
3. Methodology
Problem Definition. Given the historical observations of multivariate time series X, {xt_ Let) 1 where I. repre- sents the number of lookback time steps and C denotes the number of variates, the goal is to predict the future F time 1 The ground truth of the future
Multi-task Pre-training. To align time series representa- tion with the word embedding space, we utilize two pre- training tasks: reconstruction and prediction. The recon- struction task enhances the model's understanding of time series, while the prediction task leverages the generative capabilities of L.LSs to identify anticipatory dependencies for future forecasting (Cao et al., 2020). We adopt the Hu-
steps Y, { - values is denoted as Y, Fx ) LangTime is pre- trained on multi-source datasets D Prreetrain {(X), Y
3

---

LangTime: A Language-Cuided Unified Model for Time Series Forecasting with Proximal Policy Optimization
ber loss (Huber, 1992) to balance robustness and sensitivity.
Temporal Comprehension Prompts
The overall pre-training loss is formulated as:
The information of the given time series:
pre train a recommettciiiin + (1 a pprediiction
(1)
Period: <TTimestamp>> >,
Dataset: < Dataset Information
where a controls the trade-off between the reconstruction and prediction tasks. Details about the loss function can be found in Appendix A.
Channel: <Channel Information>,
Value: <TTim Series Representation>,
Please compress this series into one word: <|EMB >
Based on the given information, predict next <N> values: 

Input Processing and Adaptive Training. The input se- quence X E R is first divided into non-overlapping patches of length P. For each patch P. E RCP,, we apply a linear transformation to project it into P. E RCXD.
TCPs play crucial role in guiding the LLM's s interpretation of time series data by encoding essential contextual infor- mation. The domain description encodes dataset-specific characteristics, allowing the model to incorporate relevant contextual information. The time placeholder is replaced with time series features extracted by TE to provide direct temporal input. Additionally, compression token enables the condensation of time series information into a single token, enhancing the model's ability to capture key temporal patterns. The prediction guidance directs the forecasting process, ensuring smooth integration between time series data and the LLM's generative structure.
To enable long-term forecasting in an autoregressive manner, our model maintains a fixed output sequence length while allowing the input sequence length to vary, constrained to integer multiples of the patch size. This design exposes the model to diverse input lengths, improving its generaliza- tion ability for autoregressive predictions. To address the imbalance caused by longer input sequences overshadow- ing shorter ones, we propose a progressive training strategy. This approach transitions from short to long sequence data, allowing the model to gradually adapt to the complexities of longer sequences.
All components of the TCPs are fed into the pre-trained L.M,, where the causal attention mechanism enables each token to incorporate information from all preceding tokens. Following (Zhang et al., 2024), we extract the previous to- ken from 
 |> and 
 respectively, serving as the compressed token and prediction token. Ultimately, we employed a projection operation obtain the reconstruction and prediction:
3.2. Temporal Encoder
To align time series with LLMs, we introduce a channel- independent lightweight TE (Nie et al., 2022) to capture temporal dependencies and generate rich time series rep- resentations, which are mapped into the word embedding space using a simple linear layer (Liu et al., 2024b):
X Linear((LL((F)))JeeexXXXXEEE
1)), Y Lineer(LLM(())))idxe(((((((
(3)
== LTEP)))
()
1]). (4)
Time series datasets exhibit varying convergence rates due to differences in their underlying statistical properties (Liu al., 2024d). To address this, we introduce a random mask- ing strategy to enhance the generalization across diverse datasets of L.MSs. Specifically, we use a learnable token Pm. to randomly replace patches in F' at a proportion of rm. thereby obtaining Fn. During training, portions of the time series representation are randomly masked, prompting the model infer missing values and capture deeper temporal structures for the reconstruction task. This strategy not only strengthens the model's robustness but also improves its adaptability to datasets with distinct characteristics.
The compressed token serves as a global summary of the time series and is used for reconstruction, while distinct linear layers handle variable sequence lengths, allowing the model to generalize across different history lengths. Under the guidance of TCPs, LLMs forecast future values by refer- encing both temporal embeddings and the compressed token, ensuring seamless integration of temporal and linguistic rep- resentations. This design enhances alignment between time series and LLMs, allowing LangTime leverage LLMs' strengths in sequential modeling while enhancing forecast- ing performance.
3.3. Temporal Comprehension Prompts
3.4. Time Series Proximal Policy Optimization
To fully leverage the generative capabilities of LLMs, we introduce TCPs specifically designed to bridge the gap be- tween time series and language models. TCPs align time series representations with the structural characteristics of LLMs, facilitating both comprehension and forecasting. A detailed illustration of TCPs is provided in Appendix A.2.
Autoregressive methods often suffer from high variance due to error propagation, which significantly degrades performance in long-term forecasting compared to non- autoregressive methods (Taieb & Atiya, 2015). To address this issue, we propose TimePPO, an extension of the PPO algorithm (Schulman et al., 2017) tailored for time series
4

---

LangTime: A Language-Cuided Unified Model for Time Series Forecasting with Proximal Policy Optimization
Advantage Estimation. To estimate the advantage, em- play Generalized Advantage Estimation (GAE) (Schulman et al., 2015):
Value Function
M MMMM
Repeat the previous octput
Reference Model
Fine-Tuned Model
A. 5. + (T-)64+1 + +... (7A) (7A) 6. + 7VV (IL-1) 100-1)-67 EV (x1),
8T-1,
Timerro
Advantage Estimattion
Rewards Function
where & is introduced to control the sensitivity to estimation errors in the current state's value, thereby mitigating the error accumulation.
M Mm
Freeze
Fine-Tune
Multi-dinacnsioall assessment
TimePPO. Inspired by InstructGPT, we introduce an align- ment tax to prevent performance degradation (Ouyang et al., 2022). The loss function for TimePPO is given as:
MSE perailty
BMSE .5)
Timepro =E min (r(0)A(z,
Figure 3. Overview of TimePPO. Both the Reference Model and the Fine-Tuned Model are initialized with the same parameters, The Value Function uses repeat strategy to estimate expected returns, while the Rewards Function evaluates the accuracy of prediction results through multiple dimensions.
clip(r(0).1 -, (,)_1_,,,))))(x.
+/prrddcttoon
(8)
where r(0) represents the policy ratio, and c regulates the ex- tent of policy updates to ensure stability. The alignment tax term prrddtiiioo penalizes large deviations from the ground truth, ensuring that the model remains aligned with accurate predictions. Further details presented in Appendix A.4.
forecasting. Considering the unique characteristies of time series, we have undertaken a redesign of the Rewards Func- tion and Value Function.
Rewards Function. As shown in Figure 3, the rewards function evaluates the discrepancy between the predicted series De and the ground truth ys at time t from multiple perspectives, Additionally, we introduce an MSE penalty term MSS (E (i")) to constrain the difference between the new policy and the old policy, ensuring stability and safety during policy updates, as defined in Equation (5):
With the aforementioned design, we have successfully en- hanced the PPO algorithm for time series, thereby optimiz- ing the autoregressive prediction model from an entirely novel perspective. The overall process of the TimePPO depicted in Algorithm
Algorithm TimePPO
R(i:)=
Σ Ro Ne)ws SMSE DF)
1: Input: initial policy parameters fn and 0ze--
for l: = 0, 1, 2,... do
(5)
3: Collect set of trajectories Dj f Dc, (1 } by run-
where 7 is hyperparameter controlling the distribution of the reward scores, and R. (ft- y/s) represents individual eval- uation dimension (detailed in Appendix A.3). The weights w, balance the contributions of these dimensions.
ning policy n T(0x) and Tee = (f(ct).
4: Compute rewards R(ú)) and values V(x--1).
5: Compute advantage estimates A.-
6: Update the policy F = T(0)) by maximizing the TimePPO objective in Equation (8).
0,
if
V(x+)
m Σ
tanh -E
)) iif
end for
4. Experiments
(6)
4.1. Training Details
Value Function. We define the value function V(x;) to estimate the expected return from state 2t. As shown in Equation (6), we assume that the model will continue to repeat the output from time t - 1 until the final time m, and then consider all the reward scores obtained as the ex- pected return for predicting the entire sequence. This design ensures that the value function reflects the accumulated rewards, facilitating more accurate return estimation over time.
Datasets. We conduct both pre-training and fine-tuning of LangTime on seven real-world datasets that cover various time series application domains, including ETT (ETTh1, ETTh2, ETTm1, ETTm2), Electricity. Exchange, and Weather. Specifically, we pre-train LangTime across mul- tiple domains and then perform fine-tuning on individual domain using the TimePPO algorithm. We evaluate the performance of LangTime on the test splits of the same
5

---

LangTime: A Lunguege-Guided Unified Model for Time Series Forecasting with Proximal Policy Optimization
Table 1. Forecasting performance comparisons. The input sequence length is set to 96. The predictive lengths are set to 96, 192, 336, 720. Avg is averaged over all predictive lengths. Red: best performance for the entire row. Bold: best performance among models trained
across datasets.
Madele Trained Acrurs Datostts
Madel Frained/Fanr stuned on Each Dalmel
Methoal
LangTime Tumbeys
Un Time!
AutoTimns'
Cat Tomcirr
AutoTamess
S'PPLIM
Tomos11M
GPT4TS
PaccTTT
TimcsNos
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSB
MAB
MSE
MAB
MSE
MAA
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
96
0.323
0,346
0.350
0.385
0.914
0.590
0.319
0.345
0,337
0.374
0.530
0.515
0,359
0.351
0.327
0,361
0.327
0.363
0.329
0.367
0.358
0.375
192
0.372
0.376
0.384
0.401
0.966
0.616
0.368
0.375
0.373
0.393
0.664
0.553
0,383
0.393
0.368
0.381
0.365
0,383
0.367
0.374
0.387
336
0.419
0.403
0.420
0.425
0.935
0.612
0.413
0.402
0.405
0.415
0.836
0.652
0,466
0.414
0.001
0.405
0.400
0,405
0.399
0.410
0.410
0:411
720
0.491
0.473
0.600
0.430
1.207
0,483
0.436
0.439
0.464
0.47%
0.450
Aeg
0.401
0.392
0.407
0.416
0.942
0.612
0.997
0.391
0,305
0.408
0.632
0.634
0.400
0.400
0.390
0,506
0.380
0.308
0.307
0,400
0.400
0.406
96
01184
0.258
0L183
0.264
0.299
0.381
0.188
0.258
0.179
0.264
0.722
0.588
0.195
0.250
0.177
0.262
0.176
0.364
I.175
0.259
0.187
0.267
BTTml
192
0.245
0.300
0.246
0.307
0.364
0.245
0.297
0.246
0.8%%
0.257
0.338
0.305
0.246
0.307
0.241
0.362
0.249
0.309
336
0.308
0.339
0.310
0.346
0.379
0.393
0.301
0.356
0,308
0,366
1.207
0.792
0.317
0.353
0.304
0.342
0.309
0.349
0.305
0.345
0.321
0.351
720
0.110
0.399
0.413
0.405
0.473
0.442
0.402
0.393
0,410
0.406
1.708
0,950
0,419
0.411
0.400
0.397
0410
0,408
0.402
0,400
0.408
0.403
Avg
0.287
0.304
0.285
0.331
0.362
0.385
0.254
0.321
0,286
0.331
1.134
0.751
0.297
0.541
0.242
0.327
0.286
0.332
0.281
0.326
0.29)
0.355
96
0.394
0.395
0.525
0.500
0.417
0.408
0.991
0.363
0.450
0.471
0.381
0.401
0.198
0.410
0.425
0.425
0:385
0.402
0.14
0.419
0.384
0.402
ETTT
192
0.439
0.420
0.544
0.511
0.484
0.444
0.429
0.419
0.505
0.47
0.435
0.434
00451
0.440
0.464
0.446
01422
0.425
0.460
0.445
0.436
0.429
336
0.464
0.442
0.576
0.529
0.529
0.66
0.462
0.460
0.536
0.504
0:480
0.459
0.508
0.471
0.490
0.461
0.467
0.447
0.501
0.466
0.491
0.469
720
0.462
0.449
0.577
0.548
0.549
0.494
0.458
0.445
0.531
0.520
0,499
0.478
01483
0.478
0.505
0.487
0.422
0.466
0.500
0.488
0.521
0.500
Aag
0.440
0.556
0.522
0.444
0.435
0.423
0,513
0.496
0.449
0.443
0,460
0.450
0.473
0.455
0.439
0,435
0.469
0.458
0.450
96
0.301
0.308
0.357
0.301
0.335
0.299
0.336
03300
0,356
0.318
0.352
0,296
0.346
0,306
0.354
0.303
0.354
0.302
0.348
0.340
0.374
677h2
192
0.380
0.389
0.390
0,405
0.410
0.398
0.374
0.383
0,388
0.404
0
0.401
0,386
0.399
0.377
0.398
0.386
0,404
0.385
0.400
0.402
0.414
0.112
0,424
0.436
0.419
0.410
0.418
0/423
0.446
0.441
0,447
0:645
0.435
0,428
0.126
0.452
0.452
720
0.422
0.430
0.435
0.454
0.490
0.444
0.418
0.426
0.433
0.451
0.460
0.459
0.428
0.444
0.431
0.447
0.433
0.452
0.481
0.446
0.462
0.468
Aag
0.379
0.389
0.393
0.319
0.375
0.391
0.388
0.406
0,369
0.409
0.412
0.387
0.414
0.427
0:190
0.279
0.382
0.267
0.181
0.210
0.381
0:306
0,204
0.268
0.184
0.270
01881
0.168
0.272
Eleetreity
192
0.213
0.256
0.276
0.379
0.217
0.291
0.155
0.273
0.249
0.351
0.224
0.200
11207
0.205
0.204
0.246
0.388
0.275
0.145
0.274
0.184
0.289
336
0.234
0.316
0.285
0.385
0.236
0.319
0.198
0.261
0.259
0.357
0.251
0.321
0,219
0.306
0.223
0,308
0.203
0.290
0.204
0.295
0.198
0.300
7.0
0.272
0.357
0.323
0.409
0.272
0.346
0.241
0.320
0.298
0,361
0.318
0.580
0,263
0.341
0.269
0.345
0.243
0,322
0.46
0.304
0.220
0.320
Avg
01230
0,312
0.291
0.389
0.225
0.306
0.201
0.265
0.254
0.361
0,350
0,319
0.223
0.500
0.230
0,302
0.205
0,289
0:305
0.200
0L103
0.395
96
0J086
0.305
0.124
0.354
0131
0.753
0.089
0.201
0.118
0.246
0.087
0.200
00883
0.301
0.087
0.308
0.084
0,301
0.088
0.305
0.107
0.234
Enxcuunge
192
0.175
0.300
0.118
0.338
0.253
0.357
0.175
0.298
0.212
0.332
0.178
0.298
0.178
0.299
0.178
0.302
0.176
0.209
0.176
0.259
0.226
0.344
336
0.329
0,412
0.367
0.443
0,340
0.452
0,329
0.409
0.360
0.438
0.329
0.413
11328
0.415
0.338
0.422
0.343
0.422
0
0.397
0.367
0.448
720
0.844
0.913
0.728
0.31
0.730
0.852
0.690
0.901
0.792
0.854
0.096
0.681
0.803
0.671
0.901
0.961
0.746
Avg
0.361
0,408
0.406
0.441
0.427
0.448
0.361
0.400
0.309
0.435
0.347
0.397
0,361
0,403
0.356
0,408
03352
0.356
0:367
0.404
0.416
0:445
96
0.184
0.303
0.181
0.222
0.219
0.245
0.178
0.202
0,182
0.220
0,188
0.227
0.195
0.235
0.180
0.221
0.183
0.225
0.177
0.218
0.172
0.220
Werker
192
0.216
0.226
0.761
0.310
0.211
0.245
0.226
0.234
0.240
0.260
0.361
0.230
0.362
0218
0.259
0.219
0:261
336
0.275
0.298
0.280
0.209
0.357
0.338
0.269
0.286
0.279
0.823
0.288
0.505
11.255
0.506
0.285
0
0.288
0.302
0.278
0.297
0.280
0.306
720
0.361
0.349
0.356
0.347
0.415
0.385
0.351
0.346
0.354
0.365
0,368
0.354
0.359
0.349
0.362
0.351
0.354
0.348
0,365
0.359
Asg
0.359
0.374
0.261
0.282
0.317
0.319
0.252
0.270
0.260
0.263
0.268
0.253
0.274
0.291
0.263
0,285
0.265
0.285
0.257
0.241
0.259
0.287
1
Count
58
10
45
signifies the use of the official baseline code with cross- dooss- training conducted similarly to our approach. $ denotes supervised fine- tine-tuning using identical data as our method, building upon the conditions specified by f. indicates the adoption of the official baseline code, with adjustments to input sequence length and maximum training epochs for fair comparison with other methods, and other results are sourced from iTransformer(L.i liu et al., 2023).
seven benchmark datasets. The details of these datasets are provided in Appendix B.
The table is divided into two sections by vertical lines. On the left, models are pre-trained across multiple datasets, while on the right, models are either trained separately or fine-tuned for each dataset.
Baselines. We evaluate Lang against state-of-the-art models. (1) LLM-based methods: UniTime (Liu et al., 2024d), AutoTimes (Liu et al., 2024e), SLIP-LLM (Pan et al., 2024), TiMe-LLM (Jin et al., 2023), and GPT4TS (Zhou et al., 2023); (2) Specitic methods: PatchTST (Nie et al., 2022), and TimesNet (Wu et al., 2022). We adopt 0.5-2-005B-Instrueeie
Pre-trained LangTime achieves remarkable improvements over baseline models that are also trained across datasets, achieving the best performance in 58 out of 70 entries. This highlights LangTime's 's ability to generalize effectively across diverse time series distributions. On the right side of the table, results indicate that TimePPO fine-tuned Lang- Time achieves competitive performance, surpassing models trained individually on each dataset in 45 out of 70 entries, establishing new state-of of-the-art results. These findings val- idate LangTimm Time's effectiveness in handling time series data with diverse characteristics. Furthermore, compared to other I.L.M-based forecasting methods, Lang Time demonstrates significant advantages, reinforeing its capability to enhance the alignment between LLMs and time series data.
(Yang et al., 2024) as backbone.
Setup. We adopt pre-training to fine-tuning dataset ratio of D Prre-train -Dine-tuee 8 : 2. To ensure fair compar- ison, all methods maintained fixed look-back window of L = 96 and predicted future values with lengths of F = {96, 192, 336, 720). More implementation details can be found in Appendix B.2.
4.2. Main Results
Comparison with Other Forecasting Methods. Table presents the overall forecasting performance of our model.
Comparison with Other Fine-tuning Algorithms. Table 2 presents comparison between our proposed TimePPO algo-
6

---

LangTime: A Language-Cuided Unified Model for Time Series Forecasting with Proximal Policy Optimization
Table Comparisons forecasting performance among various fine-tuning algorithms. The results are presented as averages over four
forecasting horizons: 96, 192, 336, and 720. Bold: best results. Table 18 shows the full results.
ETTml
ETTm2
ETThl
TTTh2
Electricity
Exchange
Weather
Method
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
LangTime
0.401
0.392
0.287
0.324
0.440
0.427
0.379
0.395
0.230
0.312
0.361
0.403
0.259
0.274
LangTimeser
0.399
0.391
0.285
0.321
0.447
0.423
0.378
0.391
0.211
0.291
0.362
0.404
0.263
0.279
0,397
0.391
0.284
0.321
0.435
0.423
0.375
0.391
0.201
0.285
0.361
0.400
0.252
0.270
AutoTimes
0,942
0.612
0.362
0.383
0.495
0.454
0.392
0.399
0.228
0.306
0.427
0.448
0.317
0.319
AutoTimessFT 0.945
0.613
0,364
0,383
0.491
0.448
0.400
0.403
0.228
0.307
0.424
0.444
0.318
0.319
Auto Timee
0.940
0.610
0.360
0,383
0.485
0.446
0.390
0.399
0.233
0.304
0.425
0.450
0.317
0.318
Table 3. Ablation studies on various components of temporal com-
Table S. Ablation study of different backbones on ETThl and
prehension prompts on ETTh1 and Weather datasets. Results are reported as averages over four forecasting horizons: 96, 192, 336, and 720. Full results are accessible in Table 19.
Weather datasets. Bold: best results.
GPT2
Qwen
Linear
Dataset
MSE
MAE
MSE
MAE
MSE
MAE
ETThl
Weather
LG
TS
DI
CI
96
0.409
0.406
0,388
0.391
0.702
0.567
MSE
MAE
MSE
MAE
192
0.472
0.438
0.442
0.423
0.721
0,580
/
d
0.436
0.436
0.267
0.284
ETThl
336
0,530
0.461
0.479
0.445
0.733
0,592
/
v
d
0.440
0.438
0.269
0,288
720
0,556
0.483
0.482
0.465
0.735
0.611
/
0.442
0.440
0.272
0.292
Avg
0.492
0.447
0.448
0.431
0.723
0.588
0.442
0.443
0.274
0,293
96
0.194
0.233
0.181
0.217
0.223
0.273
192
0.244
0.276
0.236
0,254
0.263
0.310
Table 4. Ablation studies on various dimensions of Rewards Func- tion on ETThl and Weather datasets. Results are reported as averages over four forecasting horizons: 96, 192, 336, and 720.
Weather
336
0.294
0.311
0.294
0.295
0,318
0.343
720
0.366
0.358
0.368
0.341
0.398
0.405
Full results are accessible in Table 20.
Avg
0.275
0.294
0.270
0.277
0.301
0.333
ETTh
Weather
Method
curacy of prediction results from three dimensions in the TimePPO, including Mean Squared Error (MSE), Mean Absolute Error (MAE), and KI. divergence, as detailed in Table 10. We evaluate LangTime's predictions across the three different dimensions and compute the corresponding reward scores, with the results presented in Table 4. The findings indicate that incorporating all dimensions signifi- cantly enhances the performance of pre-trained LangTime Conversely, removing any single dimension leads to de- cline in TimePPO's effectiveness, with MSE exerting the most significant impact compared to others.
MSE
MAB
MSE
MAE
LangTimere
0.436
0.436
0.267
0.284
All dimensions
0.429
0.433
0.259
0.280
TimePPO w/o RMSE
0.435
0.437
0.263
0.281
TimePPO w/o RMAB
0.433
0.438
0,261
0.282
TimePPO w/o RXL
0.435
0.438
0,262
0.282
rithm and Supervised Fine-tuning algorithm (SFT). Across most datasets, models fine-tuned with TimePPO achieve su- perior predictive performance compared to those fine-tuned with SFT. This demonstrates the effectiveness of TimePPO in mitigating cumulative errors and enhancing the stability of autoregressive forecasting models.
Backbones. We selected Qwen2-0.5BB 0.55B-Insttuction (Yang et al., 2024) as the backbone due to its superior instruction- following capability. To validate the framework's 's scalability, we substituted the backbone with GPT-2 (Radford al., 2019). Table 5 demonstrates that while GPT-2 achieves comparable performance on the Weather dataset, it under- performs on ETTh due to weaker instruction compliance. To evaluate LLMs' role in transferring sequential modeling from text to time series, we replaced the backbone with a linear layer (Zeng et al., 2023). As presented in Table 5, this modification resulted in significant performance degrada- tion, conclusively demonstrating that our method effectively enhances LLMS' comprehension of time series patterns.
4.3. Ablation Studies
Temporal Comprehension Prompts. We analyzed the im- pact of TCPs at different levels of detail by segmenting TCPs into the following components: @ Language Guid- ance (LG)); @ Timestamp (TS); 3 Dataset Information (DI); Channel Information (CI). The detailed results in Table show that removing CI and DI leads decline in LangTimmes performance, while the impact of TS is rela- tively minor. Notably, even in the absence of supplementary metadata, using only 1G still yields favorable results, fur- ther confirming the effeectivnnes of our proposed approach in aligning LLMs with time series data.
Adaptability to Different Prompts. The TCPs in our ap- proach comprise two components. The background part incorporates domain and channel descriptions to provide richer linguistic information, enabling L.MSs to develop a
Rewards Function. We comprehensively evaluate the ac-
7

---

LangTime: A Languege-Cuided Unified Model for Time Series Forecasting with Proximal Policy Optimization
Table Impact of language prompt modification on model capa-
0.8
0.40
bility on ETThl and Weather datasets. Bold: best performance.
MSE MAE
MSE
MAE
Original
Instruction
Background
0.35
Dataset
0.6
MSE
MAE
MSE
MAE
MSE
MAE
0.30-
96
0.388
0.391
0,386
0.397
0.387
0.396
0.5
192
0.442
0.423
0.442
0.425
0.441
0.426
ETTh
336
0.479
0.445
0.479
0.445
0.478
0,445
0.3
0.6
1.0
0.20
1.0
720
0.482
0.465
0.488
0.465
0.493
0.468
Avg
0.448
0.431
0.449
0.433
0.450
0.434
(a) ETTh1
(b) Weather
96
0.181
0.217
0,181
0.219
0.184
0.214
Figure 4. Parameter sensitivity analysis on used in Equation (1).
192
0.236
0.254
0.241
0.261
0.237
0.256
0.46
0.40
Pre-trained MAE
Pre-trained MAE
Weather
336
0,294
0.295
0.297
0.300
0.296
0.298
MSE
0.45
MSE
0.35
MSE MAE
720
0.368
0.341
0.371
0.347
0.372
0.348
MAE
Avg
0.270
0.277
0.273
0.282
0.272
0.279
0.14
0.30
Table 7. Comparison of zero-shot performance of pre-trained mod- els. The input sequence length is set to 96 for the Traffic dataset and 48 for the Illness to fit patch size 24. The predictive lengths are set to 24, 36, 48, 60 for Illness, and 96, 192, 336, 720 for Traffic.
0.43
0.25
0.42
0.20
0.01
0.05
0.1
0.3
0.5
0.01
0.05
0.1
0.3
0.5
(a) ETTh1
(b) Weather
Bold: best results.
Figure 5. Parameter sensitivity analysis on I used in Equation (5).
Lang Timepr
UniTime
AutoTimes
Dataset
MSE
MAE
MSE
MAE
MSE
MAE
4.5. Parameter Analysis
96
0.486
0.259
0.550
0.363
0.575
0.376
The Sensitivity of a. Figure 4 presents the impact of on model performance. When only the prediction task (a 0) or only the reconstruction task (a = 1) is applied, the model exhibits suboptimal performance. However, when balanc- ing both tasks (a = 0.4 or a = 0.6), LangTime achieves the best results across both datasets. This highlights the importance of the reconstruction task in guiding the LLM's understanding of time series structures.
192
0.525
0.284
0.536
0.343
0.557
0.353
Traffic
336
0.594
0.298
0.642
0.429
0.697
0.453
720
0.686
0.327
0.675
0.444
0.727
0.397
Avg
0.573
0.292
0.601
0.395
0.639
0.395
24
4.071
1.482
4.221
1.525
5.189
1.572
36
3.962
1.443
4.235
1.496
4.676
1.530
Illness
48
4.006
1.477
4.349
1.515
5.060
1.642
4.053
1.492
4.632
1.559
4.667
1.603
The Sensitivity of Figure 5 illustrates the impact of on TimePPO's rewards function. Since T influences the distribution of reward scores, its effect varies based on the dataset's MSE and MAE convergence values. For ETThI, performance remains relatively stable across different values of T, indicating low sensitivity. In contrast, for Weather, improper selection leads noticeable fluctuations, occasionally causing performance degradation after fine- tuning. Nevertheless, in most cases, TimePPO exhibits robustness with respect to 7 within a certain range.
Avg
4.023
1.474
4.359
1.524
4.898
1.587
more profound understanding of time series background knowledge. The instruction part has two task instructions to guide the model in understanding the time series data and generating predictions. We compared the impact on perfor- mance when these two parts varied separately, as shown in Table 6. However, when their expression changes but the basic meaning remains consistent, modifying the domain description and instructions does not significantly affect the model's performance. The detailed modification of the prompt can be found in Appendix B.3.
The Sensitivity of 3 and f/. Figure 6 illustrates the model's sensitivity to the parameters and 17. Here, 3 controls the MSE penalty term, aiming to regulate the extent of policy updates from the reward score perspective. For 17, ref- erenced the alignment tax design in InstructGPT (Ouyang et al., 2022), enhancing the model's prediction capability and training stability. The sensitivity analysis experiments revealed no significant impact of these parameters, demon- strating the robustness of our proposed method.
4.4. Zero-Shot Transferability Analysis
In this section, we conduct an in-depth analysis of the zero- shot performance of our pre-trained model compared to other cross-domain training models in unseen domains. Ta- ble 7 demonstrates that LangTime consistently outperforms baseline models in most cases. This highlights the effec- tiveness of TCPs in enhancing the model's adaptability to diverse, unseen time series distributions. Additionally, we further analyze LangTime's 's cross-domaie transferability in Appendix C.1.
4.6. Model Analysis
Error Accumulation Analysis. To evaluate how TimePPO improves model resilience against cumulative errors, we
8

---

Time: A Language-Cuided Unified Model for Time Series Forecasting with Proximal Policy Optimization
cth Weater
cmm
3350
40
eenta chesber iightanne Load (ET)
40-
Foecce Be arrertae Westiet ligh densa Look imm
424
0.43
0.432
0.5
(a) |
(b) 17
-00
-46
-28
Figure 6. Parameter sensitivity analysis on & and 17.
(a) Temporal Representations
(b) Compressed Token
Table 8. Comparison of algorithms in mitigating accumulated error on ETThl and Weather datasets. Reporting last-step metrics for long-term autoregressive forecasting (336, 720), with last 96 points
Figure 7. T-SNE visualization of the mean pooling of temporal representations generated by Temporal Encoder and compressed token generated by I.I.M.
for 336 and 48 points for 720.
Pre Pre-Training
SFT
TimePPO
The information given time series:
Datasel
MSE
MAE
MSB
MAE
MSB
MAE
Period: 2020-12-19 05:40:00 to 2020-12-19 21:30:00,
Dalaset: Meteorological indicator data with ten minute sample rale,
336
0,527
0.469
0.526
0.468
0,519
0.464
Channel: Air Pressure,
ETThl
720
0,541
0.513
0.539
0.512
0,537
0.513
Value: [ENC] [ENCCEEEEN
Avg
0.534
0.491
0,533
0.490
0.528
0.489
Please compress this series into one word: 
.
Base on the given information, predict next values:
336
0,364
0.353
0.364
0.353
0,358
0.350
Weather
720
0.467
0.418
0.475
0.418
0.462
0.412
0.054
0,0366
0.3500
adim
onioe
ontos
5mnos
Avg
0.415
0.385
0.419
0.386
0.410
0.381
Figure 8, Attention map visualization for the last token, which is the actual prediction token preceding </OUT|> Special tokens and system prompts are hidden, and softmax values are recalcu- lated over the remaining tokens provide clearer representation.
analyzed prediction metrics at the final step of long-term forecasting (336 and 720). As shown in Table 8, TimePPO demonstrates superior performance in these critical final- step predictions where accumulated errors have greatest im- pact. This improvement stems from TimePPO's unique ap- proach: its value function estimates sequence-wide returns while advantage calculations assess each step's long-term term value relative to these estimates. Unlike methods that focus solely on immediate next-step prediction, these mechanisms optimize the entire sequence's prediction quality. Through this holistic optimization, TimePPO effectively mitigates the cumulative error problem inherent in autoregressive models.
A Attention Map Analysis. To further examine how LLMs interpret and integrate time series representations, we visu- alize the last token attention weights from the final layer of the LLM in LangTime that is ultimately processed through the MI.P to generate predictions, as shown in Figure 8. The results indicate that: ① The last token focuses primarily on the time series representations and the token preceding the compressed token placeholder, which plays a key role in reconstructing the input sequence. @ LLMS also attend to contextual and prediction-relevant information, reinforcing the effectiveness of TCPs in aligning time series with LLMs. These findings confirm that TCPs enhance LLMs' ability to differentiate time series patterns, preserve Nomain--peeiife knowledge, and capture meaningful temporal dependencies.
T-SNE Visualization. In this part, we focus on analyz- ing the three channels of the ETThl and Weather datasets. The ETTh1 dataset emphasizes energy-related features, in- cluding High Useful Load, High Useless Load, and Middle Useful Load, while the Weather dataset comprises features related to the natural environment, such as Air Tempera- ture, Air Pressure, and Potential Temperature. We employ T-SNE visualization techniques to analyze LangTime's per- formance in processing these features, aiming to better un- derstand the model's adaptability across different domains.
5. Conclusion
In this work, we propose LangTime, novel generalized model designed to address the challenges of leveraging multi-domain datasets for improving downstream time se- ries forecasting. By incorporating TCPs, LangTime en- hances the LLM's ability to interpret time series embed- dings and generate accurate predictions. Additionally, we introduce TimePPO, a fine-tuning algorithm specifically designed for time series to effectively mitigate error accu- mulation in long-term forecasting. Extensive experiments demonstrate that LangTime achieves state-of-the-arr perfor- mance on standard benchmarks and exhibits strong zero- shot transferability to unseen domains. Future work will concentrate on broadening LangTime Time's 's applicability to en- compass wider array of time series analysis tasks.
As shown in Figure 7, initially, TE processed all channels uniformly, leading to overlaps in representations between different domains such as High Useful Load and High Use- less Load within the ETTh1 dataset. However, TCPs pro- vided channel guidance to LLMs, enabling them to effec- tively distinguish these features and uncover their seman- tic relationships. Specifically, in the Weather dataset, Air Temperature and Potential Temperature exhibit stronger cor- relation compared to Air Pressure, demonstrating the role of language guidance in uncovering intrinsic dependencies between different channels.

---

LangTime: A Language-Cuided Unified Model for Time Series Forecasting with Proximal Policy Optimization
Acknowledgements
gual language models. arXiv preprini arXiv:2107.00676. 2021.
This work is supported by the National Natural Science Foundation of China under Grant 62376194, and in part by China Scholarship Council Grant 202406250137.
Ekambaram, V., Jati, A., Nguyen, N. H., Dayama, P., Reddy, C., Gifford, W. M., and Kalagnanam, J. Tims: Fast multi-level tiny time mixers for improved zero-shot and few-shot forecasting of multivariate time series, arXiv preprint arXiv:2401.03955 2024.
Impact Statement
The objective of this study is to propel the advancement of large models within the realm of time series analysis. In this work, we have developed a unified time series prediction model grounded in large language models and have imple- mented specific enhancements tailored for time series using the proximal policy optimization algorithm. The optimized model exhibits significant predictive performance across multiple real-world datasets. The algorithm we propose plays a pivotal role in mitigating the impact of cumulative errors in autoregressive time series prediction models. This study provides valuable insights for future related research and offers practical application value for practitioners. Our research is primarily focused scientific exploration and does not present any discernible negative social impacts.
Gruver, N., Finzi, M., Qiu, S., and Wilson, A. G. Large language models are zero-shot time series forecasters. Advances in Neural Information Processing Systems, 36, 2024.
Huber, J. Robust estimation of a location parameter. In Breakthroughs statistics: Methodology and distribu- tion, pp. 492-518. Springer, 1992.
Jia, F., Wang, K., Zheng, Y., Cao, D., and Liu, Y. Gpt4mts: Prompt-based large language model for multimodal time- series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 23343-23351, 2024.
Jin, M., Wang, S., Ma, L., Chu, Z., Zhang, J. Y., Shi, X., Chen, P.-Y,, Liang, Y., Li, Y.-F., Pan, S., et al. Time-llm: Time series forecasting by reprogramming large language models. arXiv preprint arXiv:2310.01728, 2023.
References
Ainslie, J., Lee- J., de Jong, M., Zemlyanskiy, Y., Lebrón, F., and Sanghai, S. Gqa: Training generalized multi-query transformer models from multi-head check- points. arXiv preprint arXiv:2305.13245 2023.
Lai, G., Chang, W.-C., Yang, Y., and Liu, H. Modeling long-and short-term temporal patterns with deep neural networks. In The 41st international ACM SIGIR confer- ence research & development in information retrieval, pp. 95-104, 2018.
Ansari, A. F., Stella, I.., Turkmen, C., Zhang, X., Mercado, P., Shen, H., Shchur, O., Rangapuram, S. S., Arango, S. P., Kapoor, S., et al. Chronos: Learning the language of time series. arXiv preprint arXiv:2403. 15 2024.
Liu, C., Xu, Q., Miao, H., Yang, S., Zhang, L., Long, C., Li, Z., and Zhao, R. Timecma: Towards llm-empowered time series forecasting via cross-modality alignment. arXiv preprint arXiv:2406.01638, 2024a.
Baffour, A. A., Feng, J., and Taylor, E. K. A hybrid artifi- cial neural network-gjr modeling approach to forecasting currency exchange rate volatility. Neurocomputing, 365: 285-301, 2019.
Liu, H., Li, C., Wu, Q., and Lee, J. Visual instruction tun- ing. Advances in neural information processing systems, 36, 2024b.
Box, G. E., Jenkins, G. M., Reinsel, G. C., and Ljung, G. M. Time series analysis: forecasting and control. John Wiley & Sons, 2015.
Liu, P., Guo, H., Dai, T., Li, N., Bao, J., Ren, X., Jiang, Y., and Xia, S. Calf: Aligning lims for time series forecasting via cross-modal fine-tuning. arXiv preprint arXiv:2403.07330 2024c.
Cao. D., Wang, Y., Duan, J.. Zhang, C., Zhu, X., Huang, C., Tong, Y., Xu, B., Bai, J., Tong, J., et al. Spectral tem- poral graph neural network for multivariate time-series forecasting. Advances neural information processing systems, 33:17766-17778 2020.
Liu, X., Hu, J., Li, Y., Diao, S., Liang, Y., Hooi, B., and Zimmermann, R. Unitime: A sanguage-empowere uni- fied model for cross-domain time series forecasting. In Proceedings of the ACM on Web Conference 2024, pp. 4095-4106, 2024d.
Das, A., Kong, W., Sen, R., and Zhou, Y. A decoder- only foundation model for time-series forecasting. In Forty-first International Conference on Machine Learn- ing, 2024.
Liu, Y., Hu, T., Zhang, H., Wu, H., Wang, S., Ma, L., and Long, M. itransformer: Inverted transformers are effective for time series forecasting. arXiv preprint mrXiv2230006612. 2023.
Doddapaneni, S., Ramesh, G., Khapra, M. M., Kunchukut- tan, A., and Kumar, A primer on pretrained multilin-
10

---

LangTime: A Language-Cuided Unified Model for Time Series Forecasting with Proximal Policy Optimization
Liu, Y., Qin, G., Huang, X., Wang. J., and Long, M. Auto- times: Autoregressive time series forecasters via large lan- guage models. arXiv preprint arXiv:2402..02370 2024e.
benchmarking and heterogeneity analysis. IEEE Transac- tions on Knowledge and Data Engineering, 2024.
Sun, Y., Xie, Z.., Chen, Y., Huang, X., and Hu, Q. Solar wind speed prediction with two-dimensional attention mecha- nism. Space Weather, 19():=20208
Liu, Y., Zhang, Li, Li, C., Huang, X., Wang, J., and Long,
M. Timer: Transformers for time series analysis scale. arXiv preprint 4rXiv:2402.02366, 2024f.
2021.
Sun, Y., Xie, Z.. Chen, D., Eldele, E., and Hu, Q. Hi- erarchical classification auxiliary network for time se- ries forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 20743-20751, 2025.
Loshchilov, I. Decoupled weight decay regularization. arXiv preprint arXiv: I. 1.05101, 2017.
Nie. Y., Nguyen, N. H., Sinthong, P., and Kalagnanam, A time series is worth 64 words: Long-tern forecasting with 2022.
Taieb, S. B. and Atiya, A. F. A bias and variance analysis for multistep-aheal time series forecasting. IEEE trans- actions on neural networks and learning systems, 27(1): 62-76, 2015.
transformers. arXiv preprini
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730-27744, 2022.
Taylor, R., Kardas, M., Cucurull, G., Scialom, T., Hartshorn, A., Saravia, E., Poulton, A., Kerkez, V., and Stojnie, R. Galactica: A large language model for science. arXiv preprint arXiv:2211.0008 2022.
Pan, Z., Jiang, Y., Garg, S., Schneider, A., Nevmyvaka, Y., and Song, D. S²ip-IIm: Semantic space informed prompt learning with I1m for time series forecasting. In Forty-first International Conference Machine Learning, 2024.
Wang, Y., Qiu, Y., Chen, P., Zhao, K., Shu, Y., Rao, Z., Pan, L., Yang. B., and Guo, C. Rose: Register assisted gen- eral time series forecasting with decomposed frequency learning. arXiv preprint arXiv:2405.17477, 2024.
Paszke, A., Gross, S., Massa, F,, Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
Wu, H., Hu, T., Liu, Y., Zhou, H., Wang. J., and Long, M. Timesnet: Temporal 2d-variation modeling for general time series analysis. arXiv preprint arXiv:22 10.02/00 2022.
Radford, A. Improving language understanding by genera- tive pre-training 2018
Xue, H. and Salim, F. D. Promptcast: A new prompt- based learning paradigm for time series forecasting, IEEE Transactions Knowledge and Data Engineering, 2023.
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Yang, A., Yang, B., Hui, B., Zheng, B., Yu, B., Zhou, C., Li, C., Li, C., Liu, D., Huang, F., Dong, G., Wei, H., et al. Qwen2 technical report. arXiv preprint arXi::2407.1061 2024.
Rasul, K., Ashok, A., Williams, A. R., Khorasani, A., Adamopoulos, G., Bhagwatkar, R., Biloš, M., Ghonia, H., Hassen, N., Schneider, A., et al. Lag-Ilamar Towards foundation models for time series forecasting. In RO- FoMo: Robustness of Few-shot and Zero-shot Learning in Large Foundation Models, 2023.
Yu, C., Wang, F., Shao, Z., Sun, T., Wu, L., and Xu, Y. Ds- former: A double sampling transformer for multivariate time series long-term prediction. In Proceedings of the 32nd ACM international conference on information and knowledge management, pp. 3062-3072, 2023a.
Schulman, J., Moritz, P., Levine, S., Jordan, M., and Abbeel, P. figh-ddimessooal continuous control using generalized advantage estimation. arXiv preprint urXiv:1506.00 02438, 2015.
Yu, C., Yan, G., Yu, C., Zhang, Y., and Mi, X. multi-factor driven spatiotemporal wind power prediction model based ensemble deep graph attention reinforcement learning networks. Energy. 263: 126034, 2023b.
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. Proximal policy optimization algorithms. arXiv preprint arXiv:
Yu, C.,, Wang, F., Shao, Z., Qian, T., Zhang, Z., Wei, W., and Xu, Y. Ginar: An end-to-end multivariate time se- ries forecasting model suitable for variable missing. In Proceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining, pp. 3989-4000. 2024.
2017.
Shao, Z., Wang, F,, Xu, Y., Wei, W., Yu, C., Zhang, Z., Yao, D., Sun, T., Jin, G., Cao, X., et al. Exploring progress in multivariate time series forecasting: Comprehensive

---

LangTime: A Language-Cuided Unified Model for Time Series Forecasting with Proximal Policy Optimization
Yu, C., Wang, F., Wang, Y., Shao, Z., Sun, T., Yao, D., and Xu, Y. Mgsfformer: A multi- multi-granul granularity spatiotemporal fusion transformer for air quality prediction. Information Fusion, 113:102607, 2025.
Zeng, A., Chen, M., Zhang, L., and Xu, Q. Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence, volume 37, pp. 11111-111111 12128, 2023.
Zhan, J., Dai, Ye, J., Zhou, Y., Zhang, D., Liu, Z., Zhang, X., Yuan, R., Zhang, G., Li, L., al. Anygpt: Unified multimodal IIm with discrete sequence modeling. arXiv preprint ar arXiv:2402. 222.2200..22266,, 2024.
Zhang, C.,, Wu, S., Zhang, H., Xu, T., Gao, Y., Hu, Y., and Chen, E. Notellm: A retrievable large language model for note recommendation. In Companion Proceedings of the ACM Web Conference 2024, pp. 170-179, 2024.
Zhou, H., Zhang. S., Peng, J., Zhang, S., Li. J., Xiong, H., and Zhang, W. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 35, 11106-11115, 2021.
Zhou, T., Niu, P. Sun, L., Jin, R., al. One fits all: Power general time series analysis by pretrained Im. Advances in neural information processing systems, 06:44332-43335 2023.
12

---

LangTime: A Language-Cuided Unified Model for Time Series Forecasting with Proximal Policy Optimization
A. Method Design Details
A.1. Pre- Training Loss Details
During pre-training, we combine two tasks, as shown in Equation (1), For each task, we use the Huber loss, as illustrated in Equation (9) and Equation (10).
(0( - 8)², if x 2 - 6, otherwise.
reconstrnction
(9)
0. (|x 20| 20,
12(0 0)3, 6 (6. 50 4)),
ly y- 5 otherwise.
5
Sprediction
(10)
A.2. Temporal Comprehension Prompts Details
Table 9 provides a description of each channel in the datasets used in TCPs. The full prompts used in TCPs are as follows.
Temporal Comprehension Prompts
<|im starttrt start] >yyyttmm
You are helpful assistant, and your target is to summarize time series and predict the next time series, Note: Value means the actual values of the time series, where each token represents data for <PATCH SIZE> consecutive time points. < </im_end>>>>
<
jim_start]> [nllstar|>>sse user
The information the given time series:
Period: < <Timestamp>>
Dataset: <Dataset Information
Channel: <Channel Information>>
Value: <Time Series Representation>,
Please compress this series into one word: < 
.
Based on the given information, predict next <N> values: < [m_end|
<
<lim_start|> >
<OUT| >
In TCPs, < <Timestamp>> Dataset Information>> and < Chhnnnll Information>> define the domain description, encoding dataset-specific characteristics to help the model incorporate relevant contextual information. <TTme Series Representa- tion>, generated by the Temporal Encoder, represents the processed time series features. < 
 > serves as placeholder for the temporal series embeddings. <N> specifies the number of patches predicted concurrently and 
 represents the placeholder for the predicted outputs.
A.3. Rewards Dimension Details
The Rewards Function we propose assesses the significance of the output across three distinct dimensions. As delineated in Table 10, we independently compute the Mean Squared Error (MSE). Mean Absolute Error (MAE), and KL divergence between the predicted outcomes and the ground truth. For KI. divergence, we first compute the mean and variance based on the two sequences, and then calculate the KL divergence between the two continuous distributions. Subsequently, their reciprocals employed as evaluation metrics. Ultimately, the reward score for the predicted outcomes is determined via Equation (5). The proximity of the predicted result i to the ground truth y, is directly proportional to the reward score attained.
A.4. TimePPO Details
Unlike the discrete action space in NLP, the output of time series prediction models is sequence in continuous space. Therefore, we have redesigned the loss function based on NLP to measure the prediction results of the model from the probability distribution in the continuous space corresponding to the time series.
TimePPO is specifically designed to handle the continuous action space of time series forecasting, where the ratio of predicted mean probabilities computed. This design prevents instability arising from excessive policy updates, thereby
13

---

LangTime: A Language-Cuided Unified Model for Time Series Forecasting with Proximal Policy Optimization
Table Dataset Information and Channel Information used in TCPs. i means column index in dataset Traffic and Electricity. The Traffic and Illness datasets are utilized to evaluate Zero-Shot performance, whereas the remaining datasets are employed for both training and testing. Dataset
Description
Channel Description
High Useful Load
High Useless Load
An hourly-samples (minutely-sampled ETTml and ETTm2) electricity transformer dataset intended for electrical asset monitoring, collected from in province in China.
Middle Useful Load
Middle Useless Load
Low Useful Load
Low Useless Load
Oil Temperature
Air Pressure
Air Temperature
Potential Temperature
Dew Point Temperature
Relative Humidity
Saturation Water Vapor Pressure
Actual Water Vapor Pressure
Water Vapor Pressure Deficit
Specific Humidity
Water Vapor Concentration
Metcorological indicator data with ten minute sample rate.
Weather
Air Density
Wind Velocity
Maximum Wind Velocity
Wind Direction
Precipitation
Duration of Precipitation
Short Wave Downward Radiation
Photosynthetically Active Radiation
Maximum Photosynthetically Active Radiation
Internal Logger Temperature
CO2 Concentration of Ambient Air
Exchange rate of the US dollar to the Australian dollar Exchange rate of the US dollar to the British pound Exchange rate of the US dollar to the Canadian dollar Exchange rate of the US dollar to the Swiss franc Exchange rate of the US dollar the Chinese yuan
Daily exchange rates of the US dollar to eight different currencies ranging from 1990 to 2016.
Exchange
Exchange rate of the US dollar the Japanese yen Exchange rate of the US dollar the New Zealand dollar Exchange rate of the US dollar to the Singapore dollar
Hourly electricity consumption of 321 customers from 2012 2014.
Electricity
Electricity consumption of customer
Hourly road occupancy rates data from 862 detectors across the freeways of the San Francisco Bay area, covering the years 2015 to 2016.
Traffic
Road occupancy rates detected by detector
Weight ILI Rate
Unweight ILI
Weekly influenza- like illness (ILI) patient data from US Centers for Disease Control (2002-2021) showing II.I patient ratio.
Number of ILI patients aged between 0-4 years old Number of ILI patients aged between 5-24 years old Total number of ILI patients across all age groups Number of sentinel providers
Illness
Total patients
14

---

LangTime: A Language-Cuided Unified Model for Time Series Forecasting with Proximal Policy Optimization
Table 10. Dimensions of Rewards Function
Dimension
Calculation Formula
MSE
RMSE 1 2.1- (tte (2))
MAE
RMAE 1 5nh Be-Vil
KL Divergence
RKL.
PP.(P(Q) (2)
enhancing robustness in long-term forecasting.
We consider time series prediction as a continuous action space and calculate the ratio predicted mean probabilities:
1
(-ta- (-la--ia (aa-a()))) p(w))))
x (aly) (2TO²(y) exp
(11)
20²
where ply) represents the mean of sequence y, while (v) denotes the variance of sequence y. Based Equation (11), we employ the probability policy ratio r(0) to quantify the variations between the two policies.
Te TH( (2how ) |Boow)
r r(0) =
(12)
Tha (JL (Jelyne)) Vola
Based on this design, r(0) can accurately reflect the impact on the distribution of prediction results before and after the model parameter updates. Compared with directly calculating the numerical error of prediction results, exhibits stronger robustness.
B. Experimental Details
B.1. Datasets
We perform comprehensive experiments seven extensively employed time series datasets for long-term forecasting. Adhering to the data processing and train-validation- test set split protocol established in TimesNet (Wu et al., 2022), we ensure that the train, validation, and test datasets are rigorously partitioned in chronological order to prevent data leakage. Regarding the long-term forecasting configurations, we set the context length of LangTTmme and the lookback length of other comparative methods to 96, while the forecast length varies among (96, 192, 336, 720). Brief descriptions of the pre-training datasets are as follows: (1) ETT (Zhou al., 2021) includes data for monitoring electricity transformers from July 2016 July 2018, comprising four subsets: ETTml, ETTm2, ETTh1, and ETTh2. (2) Electricity contains hourly power consumption data for 321 clients from 2012 to 2014. (3) Exchange (Lai et al., 2018) records daily exchange rates for eight countries from 1990 to 2016. (4) Weather is recorded every 10 minutes in 2020, featuring 21 meteorological indicators such as temperature, humidity, and precipitation. Additionally, we evaluated the zero-shot performance of Lang Time using two datasets from different domains: (1) Illness includes weekly data on patients with seven influenza- illnesses from 2002 to 2021. (2) Traffic includes data on hourly road occupancy rates, gathered by 862 detectors across the freeways of the San Francisco Bay area from 2015 to 2016. Owing to the constraints of computational resources and given our adoption of channel-independent approach, we partition the number of channels in certain datasets. Table 11 illustrates the number of channels present in each sub-dataset. This strategy facilitates more efficient management of batch size during cross-domain training.
B.2. Experimental Setting
LangTime was implemented using PyTorch (Paszke et al., 2019), and all experiments were executed on 8 NVIDIA A100 80GB GPUs. We employed the AdamW optimizer (Loshchilov, 2017) with an initial learning rate of 1 x 10-1, A cosine annealing schedule with warmup was utilized for learning rate decay during pre-training, with a warmup rate set at 0.05, The Temporal Encoder in LangTime comprises 4 layers utilizing group query attention (Ainslie et al., 2023), with a query head count of 8, and key and value head counts of 2. The dimension of the latent space is set to 512. The patching process employs a patch size of 24, and LangTime predicts patches simultaneously.
15

---

LangTime: A Language-Cuided Unified Model for Time Series Forecasting with Proximal Policy Optimization
Table 11. Summary of datasets used for pre-training.
Dataset Name
Channels Frequency Time Points Channel Split
Application Domain
ETTmI/ETTm
15 mins 57,507
Energy Infrastructure Ohservation
ETTh1/ETTh2
7
1 hour 14,307
Energy Infrastructure Observation
Electricity
321
I hour
26,211
Electricity Consumption
Weather
21
10 mins
52,603
7
Metcorologic Monitoring
Exchange
day
7,207
Foreign Exchange Market
During the warmup phase, we established = 0.7 to facilitate LLMs in comprehending the time series, subsequently reducing a to 0.5 during the learning rate decay phase. The Mask Rate was set at 0.4, and for the pre-trained model with a lookback length of 96, we designated the input lengths as (96, 288, 480, 672}. The batch size is set to 48 in pre-training.
Pre-training.
Customizing the fine-tuning parameters can yield optimal results for different datasets. For instance, in the Weather dataset, configure the initial learning rate to 1 10 10-6, maintain the lookback window at 96, and execute predictions with target lengths of 672, with clip range set to 0.1. The parameter is set to 0.1, while & is set to 0.9.
Fine-tuning.
B.3. Ablation Study Details
In Table 6, compare the impact of modifying the Instruction part and the Background part in TCPs model performance. For the Instruction part, it is modified as follows:
Temporal Comprehension Prompts (Modified)
The details of the provided time series:
Period: <Timestap>>>
Dataset: Dataset Information>,
Channel: <CChannel Information>,
Value: <'Time Series Representation>>
Please summarize this series in single term: 

Using the given details, forecast the upcoming <N> values: 

In Table 6, also made modifications the descriptions of the ETThl and Weather datasets. The specific modification details are shown in Table 12.
Table 12, Details of Modifying the Background Description Prompt for the ETTh1 and Weather Datasets
Prompt
Background of ETThl
Background of Weather
An hourly-sampled electricity transformer dataset intended for electrical asset monitoring, collected from one area in province in China.
Meteorological indicator data with
Original
ten minute sample rate.
An hourly-sampled dataset of electricity transformers designed for monitoring electrical assets.
Data on meteorological indicators sampled every ten minutes.
Modified
C. Supplementary Results
C.1. Cross-Domain Transfer Capability Analysis
Zero-Shot Transferability Analysis. To further investigate the transferability of LangTime across different domains, we pre-train LingTime on the ETThl and E Following the experimental setup of UniTime(Liu et al., 2024d), we evaluate the model on three datasets: ETTh2 (which belongs to the same domain as the source), Electricity (a different domain with certain underlying similarities to the source), and Weather (which represents a completely unrelated domain). The results are summarized in Table 13. Compared to UniTime, which also employs natural language for enhancement, our model achieves
16

---

LangTime: A Language-Cuided Unified Model for Time Series Forecasting with Proximal Policy Optimization
superior performance most datasets. This demonstrates the effectiveness of proposed language-guided approach in enhancing the ability of large language models to understand time series data.
Table 13. Zero-shot transferability comparisons, Models trained and validated under uniform data settings
ETTm2
Weather
Electricity
Method
MSE
MAE
MSE
MAE
MSE
MAE
96
0.199
0.271
0.240
0.279
0.319
0.397
192
0.258
0.310
0.294
0.321
0.339
0.418
LangTime
336
0.322
0.350
0.337
0.346
0.384
0.451
720
0.425
0.409
0.411
0.393
0.467
0.503
Avg
0.301
0.335
0.320
0.335
0.377
0.442
96
0.207
0.284
0.244
0.281
0.432
0.505
192
0.267
0.320
0.293
0.316
0.453
0.525
UniTime
336
0.325
0.357
0.342
0.347
0.459
0.532
720
0.426
0.413
0.414
0.391
0.486
0.552
Avg
0.306
0.343
0.323
0.334
0.458
0.529
Comparison with Time Series Foundation Model. Foundation models for time series are typically pre-trained on large-scale time series datasets and exhibit strong domain transfer capabilities (Ansari et al., 2024; Liu et al., 2024f). To further assess the impact of the language- guided approach on domain transfcrability, we compare LangTime pre-trained on three datasets (Weather, ECI., and Exchange), with TimesFM(Das et al., 2024), which is pre-trained on a large-scale dataset. The results are presented Table 14. To ensure the invisibility of test data, we conduct evaluations on four datasets: ETTh1, ETTh2, ETTml, and ETTm2. Even when pre-trained fewer datasets, Lang' Time still demonstrates competitive performance and achieves favorable results multiple datasets. This further validates the effectiveness of proposed framework.
Table 14. Zero-shot performance comparison of Lang Time and foundation model
ETThl
ETTh2
ETTml
ETTm2
Method
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
96
0.493
0.449
0.331
0.374
0.893
0.590
0.216
0.303
192
0.536
0.475
0.416
0.423
0.866
0.595
0.257
0.324
LimeTime
336
0.570
0.495
0.444
0.447
0.920
0.621
0.342
0.375
720
0.547
0.504
0.474
0.473
0.951
0.653
0.448
0.429
Avg
0.537
0.481
0.416
0.429
0.907
0.615
0.316
0.358
96
0.516
0.429
0.368
0.358
0.687
0.490
0.276
0.300
192
0.533
0.461
0.461
0.434
0.826
0.565
0.416
0.365
TimesFM
336
0.560
0.467
0.507
0.448
0.780
0.569
0.556
0.443
720
1.075
0.652
0.549
0.503
0.862
0.620
0.440
0.434
Avg
0.671
0.502
0.471
0.436
0.789
0.561
0.422
0.386
C.2. TimePPO Fine-Tuning Scalability Analysis
Impact of Fine-Tuning Data Volume. We analyzed the impact of the amount of data used for fine-tuning model performance, as shown in Table 15. Models pre-trained across multiple domains exhibit performance improvements after fine-tuning with target domain datasets, which supports the effectiveness of our proposed method. Furthermore, even with limited data, the models fine-tuned using TimePPO still achieve favorable performance, enhancing their scalability in resource-constrained environments.
17

---

LangTime: A Language-Cuided Unified Model for Time Series Forecasting with Proximal Policy Optimization
Table 15. Performance Impact of Fine-Tuning Data Volume ETThl and Weather Datasets. Representing Data Usage 5%, 10%, 15%,
and 20%
Pre-training
5%
10%
15%
20%
Dataset
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
96
0.388
0.396
0.385
0.397
0.384
0.397
0.383
0.396
0.383
0.396
192
0.435
0,425
0.433
0.426
0.433
0.425
0.431
0.424
0.430
0.424
ETTh1
336
0.469
0.444
0.467
0.444
0.469
0.443
0.467
0.442
0.467
0.443
720
0.466
0,462
0.464
0.459
0.462
0.458
0.460
0.458
0.460
0.459
Avg
0.439
0.432
0.437
0.432
0.437
0.431
0.435
0.430
0.435
0.430
96
0.169
0.205
0.162
0.199
0.162
0.200
0.162
0.202
0.162
0.198
192
0.228
0.262
0.225
0.256
0.225
0.257
0.224
0.256
0.223
0.255
Weather
336
0.278
0.297
0.278
0.297
0.277
0.298
0.275
0.297
0.275
0.298
720
0.360
0.354
0.350
0.345
0.349
0.345
0.347
0.344
0.346
0.345
Avg
0.259
0.280
0,254
0.274
0.253
0.275
0.252
0.275
0.252
0.274
Impact of Fine-tuning Different Components. We compare the effects of freezing certain components during TimePPO fine-tuning, and the results are shown in Table 16. Full-parameter fine-tuning of LangTime achieves the best performance, while fine-tuning only the LLM component yields results close to those of full-parameter fine-tuning. Notably, even when only the TE component is fine-tuned (with less than 3% of the total parameters), the model still achieves performance comparable to fine-tuning larger proportion of parameters. This further demonstrates the adaptability of our method under resource-constrained settings.
Table 16. Effects of Fine-tuning Different Components of LangTime
Pre-training
Full
LLM
TE
Dataset
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
96
0.388
0.396
0.383
0.396
0.383
0.396
0.384
0.396
192
0.435
0.425
0.430
0.424
0.430
0.424
0.431
0.425
ETTh1
336
0.469
0.444
0.467
0.443
0.467
0.443
0.468
0.445
720
0.466
0.462
0.460
0.459
0.460
0.459
0.465
0.461
Avg
0.439
0.432
0.435
0.431
0.435
0.431
0.437
0.432
96
0.169
0.205
0.162
0.199
0.162
0.199
0.163
0.199
192
0.228
0.262
0.225
0.256
0.225
0.256
0.226
0.256
Weather
336
0.278
0.297
0.278
0.298
0.278
0.297
0.278
0.297
720
0.360
0.354
0.350
0.345
0.350
0.345
0.352
0.345
Avg
0.259
0.280
0.254
0.274
0.254
0.274
0.255
0.274
C.3. More Parameter Sensitivity Analysis
We tested the impact of mask rate and single-step prediction length on LanTime during pre-training on the ETTh1 dataset, with results shown Table 17. Among the three output lengths, the model performed best when the output length was 96. This is because shorter output length necessitates steps complete the prediction, leading to greater cumulative enror. On the other hand, while longer single-step prediction length reduces the number of steps, it results in decreased accuracy in single-step predictions.
The mask rate balances two objectives: (1) preventing overfitting in datasets with varying convergence speeds, and (2) enhancing temporal pattern learning through reconstruction tasks. Table 17 shows that extreme mask rates (either too low or too high) degrade performance: low rates fail improve temporal pattern extraction, while high rates impair long-horizon
18

---

Time:ime: A Language-Guided Unified Model for Time Series Forecasting with Proximal Policy Optimization
predictions.
Table 17. Sensitivity analysis of mask rate and output sequence length the ETThl dataset. Bold: best performance for the same output
length, Underline: the best performance for the same mask rate.
0
0.2
0.4
0.6
Mask Rate
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
96
0.392
0.398
0.387
0.398
0.395
0.401
0.409
0.404
192
0.456
0.433
0.442
0,434
0.449
0.432
0.456
0.431
24
336
0,500
0.456
0.483
0.454
0.486
0.444
0.483
0.452
720
0.517
0.481
0.492
0.465
0.487
0.464
0.497
0.466
Avg
0.466
0.442
0.451
0.438
0.455
0.435
0.461
0.438
96
0.384
0.394
0.383
0.396
0.383
0.393
0.383
0.394
192
0.437
0.424
0,436
0.425
0.434
0.423
0.438
0.424
96
336
0.473
0.442
0.470
0.444
0.470
0.443
0.473
0.444
720
0.484
0.462
0.473
0.462
0.475
0.463
0.482
0.466
Avg
0.444
0.431
0.441
0.432
0.440
0.430
0.444
0.432
96
0.394
0.401
0.390
0.399
0.394
0.401
0.390
0.399
192
0,450
0.429
0.445
0.426
0.450
0.429
0.445
0.428
192
336
0.491
0.449
0.484
0.445
0.489
0.447
0.481
0.446
720
0.503
0.468
0.493
0.462
0.499
0.464
0.487
0.465
Avg
0.460
0.437
0.453
0.433
0.458
0.435
0.451
0.435
C.4. Full Results
In this section, we present the experimental results that were not fully displayed in the main text. Specifically, Table 18 provides detailed account of the impact of different fine-tuning algorithms on two models pre-trained across multiple domains. Table 19 and Table 20 display the complete results of the ablation experiments we conducted. Table 21 and Table 22 respectively display the experimental results for and in the parameter sensitivity analysis.
C.5. Cases
In this part, we visualize the forecasting results of LangTime. Figure illustrates the performance comparison between pre-trained Lang Time and LangTime fine-tuned via TimePPO under various settings. Figure 10 visualizes the zero-shot results of the pre-trained LangTime the Traffic dataset. This demonstrates the exceptional performance of our proposed LangTime across domains, achieving impressive results even in unseen domains.
19

---

LangTime: A Language-Cuided Unified Model for Time Series Forecasting with Proximal Policy Optimization
Table 18. Comparisons of forecasting performance among various fine-tuning algorithms.
ETTml
ETTm2
ETThl
ET
Electricity
Exchange
Weather
Method
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
0.323
0.346
0.184
0.258
0.394
0.395
0.301
0.334
0.199
0.277
0.086
0.205
0.184
0.203
192
0.372
0.376
0.245
0.300
0.439
0.420
0.380
0.389
0.213
0.296
0.175
0.300
0.216
0.250
LangTimewr
336
0.419
0.403
0.308
0.339
0.464
0.442
0.412
0.419
0.234
0.316
0.329
0.412
0.275
0.293
720
0.491
0.443
0.410
0.399
0.462
0.449
0.422
0,439
0.272
0.357
0.854
0.696
0.361
0.349
Avg.
0.401
0.392
0.287
0.324
0.440
0.427
0.379
0.395
0.230
0.312
0.361
0.403
0.259
0.274
96
0.324
0,346
0.184
0.257
0.400
0
0.286
0.322
0.186
0.264
0,086
0.205
0.170
0.206
0.366
0.374
0.245
0.296
0.447
0.418
0.384
0.390
0.198
0.286
0.183
0.305
0.221
0.253
LangTimesrr
336
0.417
0.403
0.304
0.337
0.478
0.437
0.416
0.423
0.210
0.293
0.325
0.412
0.284
0.299
720
0.488
0.439
0.406
0.395
0.461
0.448
0.424
0.429
0.249
0.323
0.856
0.695
0.377
0.356
Avg
0.399
0.391
0.285
0.321
0.447
0.423
0.378
0.391
0.211
0.291
0.362
0.404
0.263
0.279
96
0.319
0.348
0.188
0.258
0.391
0.388
0.299
0,336
0.181
0.266
0.089
0.201
0.178
0.202
192
0.368
0.375
0.24S
0.297
0.429
0.419
0.374
0,382
0.185
0.273
0.175
0.298
0.211
0.245
LungTimeninn
336
0.413
0.402
0.301
0.336
0.462
0.440
0.410
0.418
0.198
0.281
0.329
0.409
0.269
0.286
720
0.487
0.439
0.402
0.393
0.458
0.445
0,418
0.426
0.241
0.320
0.852
0.690
0.351
0.346
Avg
0.397
0.391
0.284
0.321
0.435
0.423
0.375
0.391
0.201
0.285
0.361
0.400
0,252
0.270
96
0.914
0.590
0.269
0.331
0.417
0.408
0.301
0.333
0.188
0.267
0.133
0.253
0.219
0.245
192
0.966
0.616
0.326
0.364
0.484
0.444
0.410
0.398
0.217
0.291
0.253
0.357
0.298
0.310
AutoTimeser
336
0.935
0.612
0.379
0.393
0,529
0.468
0.420
0.419
0.236
0.319
0.390
0.452
0.337
0.338
720
0.954
0.630
0.473
0.442
0.549
0.494
0.439
0.444
0.272
0.346
0.931
0.730
0.415
0.383
Avg
0.942
0.612
0.362
0.383
0.495
0.454
0,392
0.399
0.228
0.306
0.427
0.448
0.317
0.319
96
0.916
0.616
0,270
0.329
0.415
0.401
0.301
0.333
0.184
0.261
0.123
0.235
0.220
0.245
192
0.956
0.605
0.326
0.365
0,480
0.438
0.432
0.416
0.217
0.284
0.253
0.357
0.299
0.310
AutoTimesser
336
0.940
0.606
0.383
0.393
0.526
0.461
0.420
0.419
0.238
0.317
0.389
0.452
0.338
0.338
720
0.969
0.626
0.477
0.444
0.544
0.493
0.448
0.444
0.274
0.367
0.932
0.732
0.415
0.383
Avg
0.945
0.613
0.364
0.383
0.491
0.448
0.400
0.403
0.228
0.307
0.424
0.444
0.318
0.319
0.914
0.609
0.257
0.323
0.417
0.403
0.300
0.332
0.208
0.277
0.127
0.252
0.218
0.245
192
0.961
0.598
0.333
0.365
0.476
0.438
0,410
0.406
0.221
0.286
0,256
0.360
0.298
0.310
336
0.938
0.605
0.380
0.403
0.517
0.465
0.416
0.417
0.237
0.308
0.388
0.458
0.337
0.338
720
0.948
0.627
0.471
0.443
0.528
0.479
0.436
0.439
0.267
0.345
0.928
0.727
0.415
0.378
Avg
0.940
0.610
0.360
0.383
0.485
0.446
0.390
0.399
0.233
0.304
0.425
0.450
0.317
0.318
Table 19. Ablation studies on various components of temporal comprehension prompts on ETThl and Weather datasets.
Language Guidance
Dataset Information
Channel Information
ETTh1
Weather
Timestamp
Length
MSE
MAE
MSE
MAE
96
0.384
0.399
0.175
0.209
192
0.432
0.429
0.235
0.265
336
0.471
0.449
0.293
0.312
720
0.459
0.467
0.364
0.349
Avg
0,436
0.436
0,267
0.284
96
0,385
0.402
0.179
0.214
192
0.434
0.429
0.238
0,267
336
0.476
0.451
0.295
0.315
720
0.465
0.471
0.366
0.356
Avg
0.440
0.438
0.269
0.288
96
0.392
0.407
0.180
0.217
192
0.441
0.431
0.240
0.267
336
0.468
0.450
0.299
0.318
720
0,469
0.471
0.369
0.365
Avg
0.442
0.440
0.272
0.292
96
0,393
0.407
0.180
0,220
192
0.440
0.433
0.239
0.269
336
0.470
0.454
0.306
0.320
720
0.467
0.477
0.369
0.365
Avg
0.442
0.443
0.274
0.293
20

---

LingTime: A Language-Cuidel Unified Model for Time Series Forecasting with Proximal Policy Optimization
Table 20. Ablation studies various dimensions of Rewards Function on ETTh1 and Weather datasets.
ETTh1
Weather
Method
MSE
MAE
MSE
MAE
96
0.384
0.399
0.175
0.209
192
0,432
0.429
0.235
0.265
LangTimepr
336
0,471
0.449
0.293
0.312
720
0.459
0.467
0.364
0.349
Avg
0.436
0.436
0.267
0.284
96
0.379
0.399
0.172
0.205
192
0,426
0.427
0.226
0.263
All dimensions
336
0,466
0.448
0.288
0.310
720
0.445
0.458
0.352
0.342
Avg
0.429
0.433
0.259
0.280
96
0.382
0.401
0.175
0.208
192
0.430
0.429
0.229
0.264
TimePPO w/o RuSE
336
0.472
0.452
0.289
0.311
720
0.456
0.465
0.359
0.342
Avg
0.435
0.437
0.263
0.281
0.381
0.401
0.172
0.208
192
0,429
0.430
0.225
0.264
TimePPO w/o RMAE
336
0,470
0.453
0.289
0.312
720
0.453
0.468
0.357
0.345
Avg
0.433
0.438
0.261
0.282
0.382
0.401
0.176
0.207
192
0.430
0.430
0.228
0.264
TimePPO w/o RxL.
336
0.472
0.454
0.289
0.310
720
0.456
0.468
0.357
0.346
Avg
0.435
0.438
0.262
0.282
21

---

LangTime: A Language-Cuidel Unified Model for Time Series Forecasting with Proximal Policy Optimization
Table 21. Full result parameter sensitivity analysis on a
ETThl
Weather
ETThl
Weather
Length
Length
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
96
0.389
0.399
0.165
0.204
96
0.388
0.395
0.182
0.225
192
0.443
0.432
0.254
0.279
192
0.442
0.429
0.234
0.267
336
0.494
0.456
0.329
0.330
0.2
336
0.491
0.452
0.284
0.302
720
0.481
0.467
0.484
0.409
720
0.480
0.466
0.355
0.348
Avg
0.452
0.438
0.308
0.306
Avg
0.450
0.435
0.264
0.285
96
0.394
0.400
0.176
0.220
96
0.399
0.402
0.185
0.229
192
0.445
0,431
0.230
0.264
192
0.449
0.432
0.244
0.276
0.4
336
0.497
0.454
0.281
0.300
0.6
336
0.503
0.454
0.293
0.309
720
0.485
0.467
0.354
0.347
720
0.499
0.468
0.364
0.357
Avg
0.455
0.438
0.260
0.283
Avg
0.462
0.439
0.271
0.293
96
0.406
0.406
0.192
0.235
96
0.768
0.583
0.224
0.276
192
0.459
0.436
0.244
0.277
192
0.757
0.587
0.273
0.311
0.8
336
0.511
0.456
0.292
0.309
336
0.754
0.595
0.315
0.336
720
0.497
0.463
0.363
0.355
720
0.730
0.603
0.379
0.376
Avg
0.468
0.440
0.273
0.294
Avg
0.752
0.592
0.298
0.325
Table 22. Full result of parameter sensitivity analysis on
ETThl
Weather
ETThl
Weather
Length
Langth
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
96
0.384
0.399
0.175
0.209
96
0.377
0.397
0.170
0.212
192
0.432
0.429
0.235
0.265
192
0.424
0.426
0.224
0.259
Lang Timepr
336
0.471
0.449
0.293
0.312
0.01
336
0.465
0.447
0.278
0.299
720
0.459
0.467
0.364
0.349
720
0.443
0.456
0.354
0.349
Avg
0.436
0.436
0.267
0.284
Avg
0.427
0.432
0.256
0.280
96
0.378
0.397
0.224
0.259
96
0.379
0.399
0.171
0.214
192
0.424
0.426
0.224
0.259
192
0.426
0.427
0.224
0.262
0.05
336
0.466
0.448
0.278
0.299
0.1
336
0.466
0.448
0.278
0.302
720
0.445
0.458
0.354
0.349
720
0.445
0.458
0.354
0.352
Avg
0.428
0.432
0.270
0.292
Avg
0.429
0.433
0.257
0.282
96
0.378
0.398
0.173
0.220
96
0.378
0.398
0.175
0.219
192
0.425
0.427
0.221
0.262
192
0.425
0.427
0.223
0.262
0.3
336
0.467
0.449
0.271
0.298
0.5
336
0.467
0.449
0.275
0.300
720
0.448
0.460
0.348
0.348
720
0.449
0.461
0.352
0.350
Avg
0.430
0.433
0.253
0.282
Avg
0.430
0.433
0.256
0.283
22

---

LangTime: A Language-Guided Unified Model for Time Series Forecasting with Proximal Policy Optimization
Ground Truth
Grand Truth Pre-Traned TImePPO
Pre-Trained
TimePPo
(a) 1nput-96-predict-e results ETThl dataset.
(h) 1nput-96-pon -oe----ee---------------- 192 results ETTh1 datasct.
Trainaineed
Ground Truth Trained TimeP90
TimePPO Ground Truth
mm
48-
(c) 1mput-96-predict--e results on Weather dataset.
(d) 1nput-96-predict--1 192 results on Weather dataset.
Figure 9. Long-term forecasting cases for ETTh1 and Weather dataset. Green lines are the ground truths, orange lines are the pre-trained model predictions and blue lines the TimePPO fine-tuned model predictions. The vertical line indicates where the prediction starts.
Ground Truth Pre-Trained
Ground Truth Pre-Trained
(a) 1nput-96-predict-96 results Traffic dataset.
(b) Input-96-predict- 192 results Traffic dataset.
Ground Truth
Ground Truth Pre-Trained
Pre-Trained
(c) Input-96-predict-990 results Traffic dataset.
(d) Input-96-predict-192 results on Traffic dataset.
Figure 10. Zero-Shot forecasting cases for Traffic dataset. Green lines are the ground truths, orange lines are the pre-trained model predictions. The vertical line indicates where the prediction starts.
23


Reviews:
[{"review_id": "hIo8LRRilk", "reviewer": "Reviewer_8SdX", "summary": "The paper introduces LangTime, an approach that builds on top of existing large language models (LLMs) to effectively perform time series forecasting. The paper identifies 3 crucial problems with adapting LLMs for forecasting tasks - cross-domain generalization, cross-modality alignment, and error accumulation in autoregressive frameworks. \n\nLangTime uses a temporal encoder to convert non-overlapping patches of the time series into tokens. These tokens are provided as inputs to the LLM, and the model is additionally trained to reconstruct the time series from the tokens apart from forecasting. For cross-modality alignment, the paper proposes using Temporal Comprehension Prompts (TCP) to provide additional information about the time series domain and specific channel characteristics. To reduce error accumulation, the paper introduces a novel training procedure using Proximal Policy Optimization (TimePPO), similar to RLHF in Natural Language Processing.\n\n## update after rebuttal\nThe authors have addressed my concerns and I am satisfied with the response. Accordingly, I have increased the score from 2 to 3.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "As for Eq.(8), it should actually be: $-||y-\\hat{y}||^2_2$ (refer to `actor_loss_fn` in `ppo_trainer.py`). We will modify the description of hyperparameters and Eq.(6) and Eq.(8) in the next version of the paper. Full results are available https://anonymous.4open.science/r/full-E4EE/README.md.\n> C1: Comparison of LangTime SFT with UniTime SFT\n\nUnlike UniTime and TimeLLM, LangTime uses a more flexible autoregressive structure. SFT focuses on the process of generating the next prediction, which has limited effectiveness on the continuous generation process of autoregressive models. This explains why LangTime performs poorly during SFT.\n> Q1: Cross-Domain Generalization\n\nWe need to clarify that LangTime undergoes joint **pre-training on 7 datasets**, which enables it to understand time series across different domains. When facing unseen domains, we provide description information for each channel in the new dataset via TCP. Leveraging the rich language knowledge of LLM and the understanding of time series across different domains acquired during pre-training, LangTime addresses the cross-domain generalization issue. We demonstrate its generalization capability in Table 5 of the paper. We conducted pre-training on ETTh1 and ETTm1, and evaluated on 3 other datasets.\n\nTab.1: Cross-domain Generalization Ability\n||LangTime||UniTime||\n|-|-|-|-|-\n||MSE|MAE|MSE|MAE|\n|ETTm2|**0.301**|**0.335**|0.306|0.343\n|Weather|**0.320**|0.335|0.323|**0.334**\n|ECL|**0.377**|**0.442**|0.458|0.529\n\n>Q2: How does the TCP used in this paper differ from the language prompts used in UniTime and TimeLLM?\n\nThe prompt-as-prefix approach (UniTime/TimeLLM) cannot fully describe **the connection between the language part and the time series part**, making it difficult for the LLM to understand time series data that it has never seen during the pre-training, leading to challenges in modality alignment. LangTime provides the same domain information as the prompt-as-prefix approach while guiding the LLM to **compress and predict time series through two task instructions**. Modality alignment is achieved through reconstruction and prediction tasks. You can also refer to our response to reviewer **BdAz**'s **Q2** for more details.\n> Q3: Effects of $\\beta$ and $\\eta$\n\nDue to length constraints, please refer to our response to reviewer **5obH**'s **E1**.\n\n>Q4: Comparison of LangTime against foundation model\n\nBecause TimesFM is pre-trained on a large number of datasets, to maintain the invisibility of the test domain, we re-trained LangTime on the Weather, ECL, and Exchange datasets, and conducted zero-shot testing on the ETT dataset. As shown in Tab.2, LangTime achieved better performance on most datasets, demonstrating its effectiveness in cross-domain generalization.\n\nTab.2: Zero-shot performance comparison of LangTime and foundation model\n||LangTime||TimesFM||\n|-|-|-|-|-\n||MSE|MAE|MSE|MAE\n|h1|**0.537**|**0.481**|0.671|0.502\n|h2|**0.416**|**0.429**|0.471|0.436\n|m1|0.907|0.615|**0.789**|**0.561**\n|m2|**0.316**|**0.358**|0.422|0.386\n\n>Q5: Why does TimePPO specifically contribute towards reducing error accumulation? \n\nAutoregressive models have flexible prediction lengths but are significantly affected by accumulated errors because they lack the ability to discern whether the output of the previous step is reliable. TimePPO estimates the return for the entire sequence through the Value Function and evaluates the long-term value of the current step relative to the estimated level using advantages. These designs optimize the prediction of the entire sequence rather than just focusing on the model's ability to predict the next step. Thus, TimePPO helps alleviate the cumulative error issue in autoregressive models. To verify this, we compared the model's metrics of the last step in long-term predictions (336, 720). As shown in Tab.3, TimePPO performs better in the final step predictions most affected by accumulated errors.\n\nTab.3: TimePPO'role in error accumulation\n||ETTh1||Weather||\n|-|-|-|-|-\n||MSE|MAE|MSE|MAE\n|PT|0.534|0.491|0.415|0.385\n|SFT|0.533|0.490|0.419|0.386\n|TimePPO|**0.528**|**0.489**|**0.410**|**0.381**\n\n>Q6: How does the output sequence length affect the error accumulation?\n\nThe output sequence length used in this paper is 96 (the prediction length for each step in autoregression). Since errors occur during each prediction, they accumulate continuously during iterations. If the single output sequence length is small, more steps are needed to predict the same length, making accumulation errors more pronounced and prediction performance relatively poorer. Conversely, although reducing the number of steps, predicting a longer sequence in a single prediction may lead to worse results. To verify this, we compared the impact of different single prediction lengths on final performance.\n\nTab.4: Impact of single prediction length \n|ETTh1|24|48|96|144|192\n|-|-|-|-|-|-\n|MSE|0.461|**0.448**|**0.448**|0.451|0.458\n|MAE|0.438|0.432|**0.431**|0.434|0.435", "claims_and_evidence": "Using TCP (guidance through detailed textual description) helps in improving the cross-modal alignment, as witnessed in Table 3. However, similar approaches are used in TimeLLM and UniTime with respect to using text prompts to describe the domain of the time series. Additionally, UniTime also employs similar reconstruction losses. In my opinion, a comparison of LangTime SFT (supervised fine tuning) results with UniTime SFT and TimeLLM to show quantitative improvements provided by TCP will be useful. The comparison is available in Tables 1 and 2, but the results look mixed with LangTime SFT underperforming in multiple cases. \n\nSimilarly, the effect of TimePPO towards minimizing the error accumulation is not clearly explained. For example, if $\\eta$ is a relatively large value, then isn't the loss essentially the same as in the SFT case? Also, does the predicted output sequence length affect the error accumulation? This has been shown in recent time series foundation models like TimesFM.", "methods_and_evaluation_criteria": "Yes, the paper tests the proposed approach on standard time series forecasting datasets. Additionally, the chosen baselines are valid.", "theoretical_claims": "There are no theoretical claims in the paper.", "experimental_designs_or_analyses": "Yes, I have checked the soundness/validity of experimental designs. Tables 2,3, and 4 showcase the ablation with respect to loss functions (SFT vs PPO), language guidance, and reward functions for PPO, respectively. However, a few more ablations are required to highlight the different contributions in the paper. For example, what are the effects of $\\beta$ and $\\eta$ in equations 5 and 8, respectively?", "relation_to_broader_scientific_literature": "Leveraging LLMs trained on large-scale internet textual data effectively for forecasting is a key problem. The effective adaptation of LLMs, through the approaches shown in the paper, can unlock the ability to obtain accurate forecasts with limited fine tuning. Additionally, this allows for future work in the direction of reasoning about the generated forecasts in natural language.\n\nSpecifically, from my domain knowledge in forecasting, the application of PPO to improve forecasts is novel and interesting. And from the experimental results, such finetuning shows promising results.", "essential_references_not_discussed": "The paper covers multiple SOTA prior works that are related to adapting LLMs for forecasting.", "other_strengths_and_weaknesses": "Strengths:\n1. The paper provides a set of extensive experimental results that showcase the effectiveness of the proposed approach. \n2. The provided ablations are useful in understanding the overall approach. \n3. The qualitative analyses through t-SNE and attention maps provide more interpretability to the proposed approach.\n\nFor Weaknesses, please check the \"Claims and Evidences\" and \"Experimental Designs\" sections.\nOne of the main weaknesses is the delineation of the 3 contributions. The paper highlights cross-domain generalization and cross-modal alignment as challenges, but it is unclear which components address the cross-domain generalization part. The temporal encoders and reconstruction loss help in cross-modal alignment, and the TimePPO objective helps with better forecasting. Also, since the overall setup is trained on a per-dataset basis, why is there a need for cross-domain generalization? Additionally, the experimental results do not showcase cross-domain generalization. \n\nAdditionally, I think the paper might benefit from zero-shot comparisons against other time series foundation models like TimesFM, Chronos, etc.", "other_comments_or_suggestions": "The TimePPO section (Section 3.4) requires more clarity. Some coefficients ($\\beta$, $\\gamma$, $\\lambda$, $\\tau$, $\\eta$) are not clearly explained. \n\nIn Algorithm 1, in line 6, if the objective in Eq 8 has to be maximized, doesn't that imply maximizing $\\|y - \\hat{y}\\|$?\n\nEq 6 is flowing out of the column width.", "questions_for_authors": "1. How is cross-domain generalization addressed/achieved in this paper?\n2. How does the TCP used in this paper differ from the language prompts used in UniTime and TimeLLM?\n3. What are the effects of $\\beta$ and $\\eta$ in equations 5 and 8, respectively?\n4. What is the performance comparison of the pre-trained model against zero-shot time series foundation models?\n5. Why does TimePPO specifically contribute towards reducing error accumulation? Is there any experimental evidence to support this claim?\n6. Can the authors describe the output sequence length used in the paper, and how does that affect the error accumulation?\n\nIf the authors can respond to 1,2,5, and 6, I am willing to raise the score.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yTDdK5FM2F", "reviewer": "Reviewer_BdAz", "summary": "This paper introduces LangTime, a unified framework that leverages large language models (LLMs) for time series forecasting across multiple domains and modalities. The authors identify key challenges when applying LLMs to temporal data—Cross-domain generalization, cross-modality alignment and error accumulation in autoregressive frameworks. To address these, LangTime integrates Temporal Comprehension Prompts (TCPs), which serve as structured inputs to guide LLMs in interpreting time series data by embedding dataset-specific and variable-level context into a compact representation. Furthermore, the paper presents TimePPO, a reinforcement learning-based fine-tuning strategy tailored for time series, which introduces a multi-dimensional reward mechanism and a repeat-based value estimation to improve long-horizon prediction robustness. Through comprehensive empirical evaluation, LangTime demonstrates superior forecasting accuracy and strong generalization to previously unseen datasets. The study concludes by suggesting that future developments will explore extending LangTime’s capabilities to broader time series analysis applications.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for the thorough review and insightful comments. In our response, our model was jointly pre-trained on the ETTh1 and Weather datasets, and experiments involving the TimePPO stage were fine-tuned on individual datasets. We presented the average of the results, and detailed experimental results are available at https://anonymous.4open.science/r/full-E4EE/README.md.\n\n>Q1:Could you provide a detailed explanation of the prompt construction process in your approach, and further discuss the model’s generalizability when faced with slight variations in the prompt formulations?\n\nThe prompts in our approach consist of two parts. The first part includes common domain descriptive information used in existing methods to provide richer linguistic information. The second part has two task instructions to guide the model in understanding the time series data and generating predictions. We compared the impact on performance when these two parts varied separately, as shown in Table 1. However, when their expression changes but the basic meaning remains consistent, modifying the domain description and instructions does not significantly affect the model's performance.\n\nTable 1: Impact of language prompt modification on model capability\n| ETTh1 | MSE | MAE |\n|---|---|---|\n|Original| 0.448  | 0.431  |\n|Instruction Prompt| 0.449  | 0.433  |\n|Data Description Prompt| 0.450  | 0.434  |\n\nIn Table 1, `Instruction Prompt` means changing TCP to the following format:\n\n```\nThe details of the provided time series: \nPeriod: <Timestamp>,\nDataset: <Dataset Information>,\nChannel: <Channel Information>,\nValue: <Time Series Representation>,\nPlease summarize this series in a single term: 
. \nUsing the given details, forecast the upcoming <N> values: 
\n```\n`Data description Prompt` means modifying the descriptive information of the following two datasets:\n\n```json\nETTh1: {\nOriginal: \"An hourly-sampled electricity transformer dataset intended for electrical asset monitoring, collected from one area in a province in China.\"\nModified: \"An hourly-sampled dataset of electricity transformers designed for monitoring electrical assets.\"\n},\nWeather: {\nOriginal: \"Meteorological indicator data with ten minute sample rate.\"\nModified: \"Data on meteorological indicators sampled every ten minutes.\"\n}\n```\n\n>**Q2:** In terms of aligning language with time series data, what are the key innovations of your method compared to existing approaches such as TimeLLM, UniTime, and MetaTST?  \n\n1. **Characteristics of Existing Methods:** Existing approaches like TimeLLM and UniTime integrate linguistic information with time series data merely by concatenation, failing to emphasize **the relationship between the two parts**. This can make it challenging for LLMs, which have not encountered time series data during pre-training, to understand sequential correlations. MetaTST, on the other hand, does not employ LLMs as its backbone architecture, therefore it does not leverage LLMs' comprehension capabilities for time series data. While these methods leverage the comprehensive pre-trained knowledge of LLMs by providing linguistic information, they overlook the powerful instruction-following abilities of LLMs.\n\n2. **Innovations of Our Method:** Our approach introduces diverse domain information via TCP and utilizes two directives to present LLMs with dual tasks: first, compress the presented information to generate a model understanding; then predict future outcomes based on this understanding (compressed token) following instructions. Benefiting from LLMs' robust instruction-following capacity, our language-guided approach aids LLMs in comprehending time series data by integrating reconstruction and prediction training objectives to fully exploit LLM's time series understanding and prediction abilities. Additionally, LangTime employs an autoregressive structure, allowing flexible prediction length. Inspired by RLHF and considering time series data characteristics, we designed a reward function with multi-angle assessment and a repetition strategy-based Value function. Fine-tuned via PPO algorithm, this enhances the model's ability to combat cumulative errors.\n\nTo validate our method's effectiveness in enhancing LLMs' time series understanding, we replaced the LLM with a linear layer. As shown in Table 2, the significant performance decline confirms the necessity of our language-guided strategy.\n\nTable 2: Impact of Removing LLM\n|backbones|Qwen||Linear||\n|-|-|-|-|-\n||MSE|MAE|MSE|MAE\n|ETTh1|**0.448**|**0.431**|0.723|0.588\n|Weather|**0.270**|**0.277**|0.301|0.333", "claims_and_evidence": "The paper lacks a detailed explanation of how the dataset-wise and channel-wise information used in the Temporal Comprehension Prompts (TCPs) is generated. Additionally, it is unclear whether different formulations or descriptions of the data might affect the forecasting performance of LangTime. If the model is sensitive to these variations, how can its performance and generalizability be consistently guaranteed?", "methods_and_evaluation_criteria": "Yes. The methods and evaluation criteria are appropriate.", "theoretical_claims": "I have checked the correctness of theoretical claims.", "experimental_designs_or_analyses": "I have checked the soundness of experimental designs. Deisgn of reconstruction loss can well enhance the alignment between time series and context information.", "relation_to_broader_scientific_literature": "In fact, a growing body of work has explored the integration of language models with time series forecasting, including approaches such as MetaTST, TimeLLM, UniTime, and TimeMMD. It would be beneficial for the paper to further clarify its distinctions and advancements over these existing methods.", "essential_references_not_discussed": "See above.", "other_strengths_and_weaknesses": "Strengths:\n\nLangTime demonstrates state-of-the-art performance on established benchmark datasets and shows strong zero-shot generalization capabilities across previously unseen domains.\n\nWeaknesses:\n\n1）The novelty of the proposed method is limited, as similar concepts and methodologies have been presented in recent works such as TimeLLM, UniTime and MetaTST.\n\n2）The paper does not provide a clear explanation of the methodology used for constructing the prompts. And it also lacks an evaluation of the model's generalizability under slight variations in the language prompt, which is important for assessing its robustness in real-world applications.", "other_comments_or_suggestions": "See above.", "questions_for_authors": "1) Could you provide a detailed explanation of the prompt construction process in your approach, and further discuss the model’s generalizability when faced with slight variations in the prompt formulations?\n\n2) In terms of aligning language with time series data, what are the key innovations of your method compared to existing approaches such as TimeLLM, UniTime, and MetaTST?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vvazZkzbLe", "reviewer": "Reviewer_5obH", "summary": "This paper presents LangTime, a novel language-guided model for time series forecasting that addresses key challenges in leveraging large language models for this task. Specifically, the authors construct Temporal Comprehension Prompts to help LLMs understand domain-specific time series data, along with a new reinforcement learning-based fine-tuning algorithm, TimePPO, designed to mitigate error accumulation in autoregressive models. Extensive experiments on seven time series datasets demonstrate that LangTime achieves state-of-the-art forecasting performance, surpassing previous methods in both cross-domain generalization and autoregressive forecasting stability.\n\n## update after rebuttal\n\nI support for accept", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for the thorough review and insightful comments. In our response, models were jointly pre-trained on the ETTh1 and Weather, and experiments involving the TimePPO stage were fine-tuned on individual datasets. We presented the average of the results, and full results are available at [link](https://anonymous.4open.science/r/full-E4EE/README.md).\n\n>E1: Hyperparameter Explanation\n\nFor Eq. (7), we introduce parameter ξ based on GAE [1] to address error accumulation effects. When prediction steps increase, model errors often exceed those of repeating previous steps, causing underestimated advantages. The coefficient ξ(<1) relaxes this constraint to better reflect relative prediction quality. For the hyperparameters in Eq.(6) and Eq.(8), $\\beta$ controls the MSE penalty term, aiming to regulate the extent of policy updates from the reward score perspective. For $\\eta$, we referenced the alignment tax design in InstructGPT, enhancing the model's prediction capability and training stability. Sensitivity analysis experiments showed no significant impact.\n\nTab.1: sensitivity analysis for $\\beta$\n|||PT|0|0.01|0.03|0.05|0.1|0.3|0.5\n|-|-|-|-|-|-|-|-|-|-\n|ETTh1|MSE|0.447|0.440|0.440|**0.438**|0.438|0.439|0.440|0.440\n||MAE|0.435|0.432|0.432|**0.431**|0.432|0.433|0.433|0.433\n\nTab.2: sensitivity analysis for $\\eta$\n|||PT|0|0.1|0.5|0.7\n|-|-|-|-|-|-|-\n|ETTh1|MSE|0.447|0.439|0.440|**0.438**|0.440\n||MAE|0.435|0.434|0.433|**0.431**|0.432\n\nA detailed parameter discussion will be added in the paper's next version.\n\n[1] Schulman, J. et al. High-dimensional continuous control using generalized advantage estimation. ICLR2016.\n\n>E2: Performance Across Backbones\n\nThe rationale for selecting Qwen and experimental results with other LLMs are detailed in **Q1** of our response to Reviewer **bFpj**, demonstrating adaptability across LLMs and GPT2's limitations.\n\n>W1: Training Efficiency Discussion\n\nFor computational efficiency concerns, please refer to our response **W1** to Reviewer **bFpj**.\n\n>Q1: Difference Between Language Prefixes and Language-Guided Strategy \n\nUsing language as prefixes inadequately describes the cross-modal relationship between language and time series, hindering LLMs' comprehension of unseen time series data during pre-training and causing modality misalignment. LangTime provides equivalent domain information while guiding LLMs through task-specific instructions (compression and prediction) and dual training objectives (reconstruction and prediction) to achieve alignment. For additional details, see our response to **Q2** from Reviewer **BdAz**. To validate LangTime's effectiveness in enhancing LLMs' time series understanding, we replaced the LLM with a linear layer. As shown in Tab.3, the significant performance decline confirms the necessity of our language-guided strategy.\n\nTab.3: Impact of Removing LLM\n|backbones|Qwen||Linear||\n|-|-|-|-|-\n||MSE|MAE|MSE|MAE\n|ETTh1|**0.448**|**0.431**|0.723|0.588\n|Weather|**0.270**|**0.277**|0.301|0.333\n\n>Q2:Could the authors explain why these specific token forms were chosen and what advantages they offer over alternative tokenization strategies?\n\nThe special tokens 
 and 
 are newly added learnable tokens without predefined meanings for LLMs. Their specific forms were chosen primarily for human readability rather than offering inherent advantages in modality alignment. To verify this, we conducted experiments with alternative tokens (
 and 
), which showed no significant performance differences(Tab.4).\n\nTab.4: Impact of Token Replacement\n|ETTh1|MSE|MAE\n|-|-|-\n|Original|0.448|0.431\n|Modified|0.448|0.432\n\n>Q3: How does the choice of mask rate impact the model's ability to extract temporal patterns and maintain prediction accuracy over longer horizons?\n\nThe mask rate balances two objectives: (1) preventing overfitting in datasets with varying convergence speeds, and (2) enhancing temporal pattern learning through reconstruction tasks. Experiments show that extreme mask rates (too low/high) degrade performance: low rates fail to improve temporal pattern extraction, while high rates impair long-horizon predictions. Shorter prediction lengths suffer more from error accumulation(Tab.5).\n\nTab.5: Impact of Mask Rate and Prediction Length on ETTh1\n|Length|24||96||192||\n|-|-|-|-|-|-|-\n|Mask Rate|MSE|MAE|MSE|MAE|MSE|MAE\n|0|0.466|0.442|0.444|0.431|0.460|0.437\n|0.2|0.451|0.438|0.441|0.432|0.453|0.433\n|0.4|0.455|0.435|0.440|0.430|0.458|0.435\n|0.6|0.461|0.438|0.444|0.432|0.451|0.435\n\n>Q4: Does TimePPO fine-tune all parameters of the model, or are specific parts of the model frozen during fine-tuning?\n\nIn our experiments, TimePPO fine-tunes full parameters. However, freezing certain components still achieves similar performance, demonstrating the method's scalability.\n\nTab.6: Effect of Fine-tuning Parameters\n|Fine-tuned Parameters|MSE|MAE\n|-|-|-|\n|pre-trained|0.439|0.432\n|Full|**0.435**|**0.431**\n|LLM(>97%)|**0.435**|**0.431**\n|TE(<3%)|0.437|0.432", "claims_and_evidence": "The claims made in the paper regarding the novel application of LLMs to time series forecasting are well-supported by the presented evidence. In addition, the detailed experimental setup and the availability of the source code of the proposed model enhance the study’s reproducibility.", "methods_and_evaluation_criteria": "The methods presented in the paper are robust and well-suited for time series forecasting tasks, effectively addressing key challenges such as cross-domain generalization, cross-modality alignment, and error accumulation. In addition, the benchmark datasets are carefully selected to cover a wide range of time series characteristics, and the evaluation criteria are well-defined, allowing for a thorough assessment of the model's performance.", "theoretical_claims": "The theoretical contributions of the paper are sound, well-supported, and free from any issues.", "experimental_designs_or_analyses": "The experimental design is comprehensive and well-executed. However, there are several issues that need to be addressed:\n1. The paper introduces TimePPO and provides valuable results; however, the authors need to clarify the choice and tuning of the hyperparameters, such as \\gamma, \\xi in equation (7) and r(\\theta), \\eta in equation (8). More detailed explanations are needed regarding how these parameters are selected and how they impact model performance.\n2. While the authors have demonstrated LangTime’s performance on a number of datasets, why did they not attempt to train LangTime using other large language models to assess its generalizability and adaptability?", "relation_to_broader_scientific_literature": "LangTime situates itself within the growing body of work that applies large language models to time series forecasting. The authors provide a thorough discussion of related work, highlighting existing methods and clearly explaining how their approach advances the field by addressing the unique challenges of multi-domain generalization, cross-modality alignment, and error accumulation in autoregressive predictions.", "essential_references_not_discussed": "The authors have adequately discussed relevant works on large language models for time series forecasting and reinforcement learning within the context of large language models.", "other_strengths_and_weaknesses": "Strengths:\n1. The introduction of Temporal Comprehension Prompts to guide large language models in understanding domain-specific time series data is a novel contribution. It effectively enhances the model's ability to interpret and process time series information.\n2. The proposed TimePPO algorithm, designed to improve autoregressive forecasting by mitigating error accumulation, is an innovative approach. It addresses key challenges in long-term forecasting, ensuring more stable and accurate predictions over extended horizons.\n\nWeaknesses:\n1. While the model shows impressive forecasting accuracy, the complexity of its architecture and training process, particularly with the use of reinforcement learning, may raise concerns regarding scalability and efficiency. A discussion of its training time and computational requirements would be valuable.", "other_comments_or_suggestions": "The authors provide informative figures, but additional explanatory captions or annotations are needed. For example, in Figure 6 (T-SNE visualization), a clearer distinction between the domains being compared and an explanation of why this visualization is important for understanding the model’s capabilities would be helpful. It is recommended that the authors review the manuscript and make the necessary adjustments.", "questions_for_authors": "1. In Section 1 and Figure 1, the paper mentions cross-modality alignment. Could the authors elaborate on the difference between using language as prefixes and the proposed language-guided strategy for aligning modalities? \n2. In Section 3.3, the authors adopt the previous token form 
 and 
 as the compressed token and prediction token. Could the authors explain why these specific token forms were chosen and what advantages they offer over alternative tokenization strategies?\n3. The authors state that the mask rate is set to 0.4. Does this mask rate influence how the model learns and understands time series data? Specifically, how does the choice of mask rate impact the model's ability to extract temporal patterns and maintain prediction accuracy over longer horizons?\n4. Does TimePPO fine-tune all parameters of the model, or are specific parts of the model frozen during fine-tuning?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jJRiBC3Ugq", "reviewer": "Reviewer_bFpj", "summary": "This paper introduces LangTime, which is a language guided unified model for time series forecasting. It integrates LLM with RL based fine-tuning using PPO for time series analysis. Among the designed components, the Temporal Comprehension Prompts (TCPs) align time series data with LLM by embedding data-specific and channel-specific instructions and the time series data is compressed into a single token to facilitate LLM's understanding. The proposed TimePPO addresses error accumulation in AR forecasting and introduces a multi-dimensional reward function for better predictions.  LangTime outperforms existing LLM-based time series models and shows strong. zero-shot transferability", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for the thorough review and insightful comments. In our response, our model was jointly pre-trained on the ETTh1 and Weather datasets, and experiments involving the TimePPO stage were fine-tuned on individual datasets. We presented the average of the results, and detailed experimental results are available at [link](https://anonymous.4open.science/r/full-E4EE/README.md).\n\n>Suggestions: Figure 2 contains too much compressed information, making it difficult to follow. \n\nThank you for your suggestion. We have re-drawn Figure 2 and presented it as Figure 1 in the [link](https://anonymous.4open.science/r/full-E4EE/README.md). In the next version, we will update some images in the paper for clearer expression.\n\n>W1: Do not mention the training efficiency, especially with autoregressive output approach.\n\nOur model achieved the results in the paper with just 1 epoch of pre-training, partially compensating for the slower training speed. Due to the complex procedure of the PPO algorithm, the TimePPO fine-tuning phase is slow. However, our experiments demonstrate that fine-tuning with limited data (Tab.6 in our response to reviewer **5obH**'s **Q4**) and mostly frozen parameters (Tab.2) still yields good performance, enhancing its scalability under resource constraints.\n\nTable 1 Training Speeds of Different Models\n||Ours(pt)|UniTime|AutoTimes|TimeLLM \n|-|-|-|-|-|\n|Speed ms/iter|218|8|89|212\n|Parameter|0.5B|0.13B|7B|7B\n\nTable 2 Impact of Fine-tuning Data Volume on ETTh1\n|data rate|PT|5%|10%|15%|20%\n|-|-|-|-|-|-\n|MSE|0.439|0.437|0.437|0.435|**0.435**\n|MAE|0.432|0.432|0.431|0.430|**0.430**\n\n>W2: The pre-training process is unclear.\n\nWe pre-trained on 7 datasets (ETT, Weather, Exchange, Electricity) and conducted zero-shot experiments on two unseen datasets (Traffic, Illness). All models in Table 5 in paper were trained under the same dataset settings, making the comparison fair.\n\n>Q1: Is there a specific reason for selecting Qwen2-0.5B-Instruction as the backbone LLM? \n\nCompared to GPT2, Qwen is pre-trained on larger datasets, offering better language understanding and stronger instruction-following abilities to better guide LLM in interpreting time series. While the common Llama-7b also has powerful capabilities, its large number of parameters led us to choose the smaller Qwen2-0.5B-Instruction model as the backbone. In Table x, we compared different backbones; GPT2's performance on Weather is close to Qwen, demonstrating our framework's effectiveness, but its performance on ETTh1 reveals its limitations.\n\nTable 3 Performance of Different Backbones\n|backbones|GPT2||Qwen||Linear||\n|-|-|-|-|-|-|-\n||MSE|MAE|MSE|MAE|MSE|MAE\n|ETTh1|0.492|0.447|**0.448**|**0.431**|0.723|0.588\n|Weather|0.275|0.294|**0.270**|**0.277**|0.301|0.333", "claims_and_evidence": "Yes, the claims are clear and convincing.", "methods_and_evaluation_criteria": "Yes, the method and evaluation metrics are aligned.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "The experiments are sound and complete.", "relation_to_broader_scientific_literature": "It involves leveraging LLMs for time series analysis through RL-based pre-training, with the potential to serve as a foundation model for time series forecasting.", "essential_references_not_discussed": "The related work is complete.", "other_strengths_and_weaknesses": "Strength: \n\n1. Integrating time series pre-training with PPO (Reinforcement Learning) is an innovative approach, offering a novel solution for aligning time series data with LLMs. In particular, TimePPO effectively mitigates accumulated errors in autoregressive forecasting.\n2. The experiments are comprehensive and complete, showing the effectiveness of the proposed framework. \n3. Present visualizations of compressed tokens to validate the effectiveness of the approach.\n\nWeakness:\n\n1. Do not mention the training efficiency, especially with autoregressive output approach. \n2. On which datasets did you pre-train LangTime? The pre-training process is unclear to me, making it difficult to assess and potentially unfair to compare with zero-shot performance.", "other_comments_or_suggestions": "Figure 2 contains too much compressed information, making it difficult to follow. Consider splitting it into two figures: one illustrating the overall pipeline and another focusing specifically on the TimePPO algorithm.", "questions_for_authors": "1. Is there a specific reason for selecting Qwen2-0.5B-Instruction as the backbone LLM? I noticed that this choice differs from previous works.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This work proposes a new approach to time series forecasting that uses a pre-trained LLM backbone and addresses three challenges with recent unified time series models. They introduce a training objective based on RLHF and show that it improves their model's zero-shot and fine-tuned performance on standard datasets.

The reviewers lean positively, noting many strengths, especially in thorough experiments with ablations and qualitative analyses and in introducing TimePPO to train forecasting models and TCPs to aid in domain adaptation. The weaknesses discussed mainly include some clarifications that should be incorporated into the camera-ready paper and the datasets are also quite over-studied in the literature at this point. Comparisons with other state-of-the-art methods like MOMENT and UniTS would help and further ablations on the need for the pre-trained LLM vs. simpler architectures would also further strengthen this work.

Overall, the strengths identified by the reviewers outweigh the negatives and I suggest accepting this paper.
Decision: Accept (poster)