Revisiting LLMs as Zero-Shot Time Series Forecasters: Small Noise Can Break Large Models
Authors: Junwoo Park, Hyuck Lee, Dohyun Lee, Daehoon Gwak, Jaegul Choo
Accepted at: acl25

Revisiting LLMs as Zero-Shot Time-Series Forecasters: Small Noise Can Break Large Models

Junwoo Park1,2, Hyuck Lee2, Dohyun Lee1,2,
Daecheon Gwak1, Jaegul Choo1,
1KAIST AI 2KRAFTON
{junwoo.park,aiclaudev,daecheon.gwak,jchoo}@kaist.ac.kr
dlgur0921@krafton.com

Abstract

Large Language Models (LLMs) have shown remarkable performance across diverse tasks without domain-specific training, fueling interest in their potential for time-series forecasting. While LLMs have shown potential in zero-shot forecasting through prompting alone, recent studies suggest that LLMs lack inherent effectiveness in forecasting. Given these conflicting findings, a rigorous validation is essential for drawing reliable conclusions. In this paper, we evaluate the effectiveness of LLMs as zero-shot forecasters compared to state-of-the-art domain-specific models. Our experiments show that LLM-based zero-shot forecasters often struggle to achieve high accuracy due to their sensitivity to noise, underperforming even simple domain-specific models. We have explored solutions to reduce LLMs’ sensitivity to noise in the zero-shot setting, but improving their robustness remains a significant challenge. Our findings suggest that rather than emphasizing zero-shot forecasting, a more promising direction would be to focus on fine-tuning LLMs to better process numerical sequences. Our experimental code is available at https://github.com/junwoopark92/revisiting-LLMs-zeroshot-forecaster

enables LLMs to perform few-shot or zero-shot forecasting. These studies show that LLM-based forecasters consistently perform well across multiple domains while reducing the need for domain-specific training. Notably, LLMTTime (Gruber et al., 2023) demonstrated that LLMs can achieve zero-shot time series forecasting, performing at a level comparable to or even surpassing domain-specific models trained on the full dataset.

However, there remains an ongoing discussion about whether a thorough evaluation was conducted when integrating LLMs into time-series forecasting and assessing their performance. For instance, existing studies have used domain-specific models (Zhou et al., 2021, 2022; Wu et al., 2021) as baselines for comparison with LLMs, but these baseline models may not be sufficiently powerful (Zeng et al., 2023) to provide a fair assessment of LLM performance. Furthermore, a recent comprehensive study (Tan et al., 2024) found that LLMs do not significantly contribute to forecasting performance, undermining the justification for their high computational cost. These concerns discourage the adoption of LLMs for promising applications in time-series forecasting. Therefore, it is crucial to reassess the true performance of LLM-based forecasters. To address this, we raise the following question:

Are LLMs as zero-shot forecasters truly more effective than domain-specific models?

Typically, zero-shot forecasting is useful because it does not require domain-specific training. However, if inference with a zero-shot model takes too much time, the end-to-end process of training and deploying a domain-specific model could be faster, which would undermine the main advantage of using a zero-shot approach. Therefore, to be considered a viable alternative, a zero-shot forecaster must at least offer faster inference than the total time required to train and deploy a domain-specific

1 Introduction

Large Language Models (LLMs) have demonstrated remarkable capabilities in diverse tasks, even without domain-specific training (Kojima et al., 2022; Wei et al., 2022). Building on this success, researchers are increasingly leveraging pre-trained LLMs for time series forecasting (Zhou et al., 2023; Jin et al., 2023; Liu et al., 2024a; Gruber et al., 2023; Requeima et al., 2024; Tang et al., 2025). A common approach involves converting the time series data into a format that LLMs can process, such as through alignment-tuning or by encoding sequences directly within prompts. This
model. While inference speed is essential, accuracy remains equally important in scenarios where real-time prediction is not required. Domain-specific models often rely on high-quality training data, and their performance can be degraded in new domains with limited data. In such cases, pre-trained LLMs may offer better generalization. Therefore, even if an LLM is slower, a higher forecast accuracy can justify its use, making precision a valid criterion for assessing viability.

In summary, for LLMs to be viable zero-shot forecasters, they should meet at least one of two key criteria. (1) they should generate predictions faster than domain-specific models that need to be trained for each new domain, and (2) their forecasting accuracy should be superior to, if not at least comparable to, domain-specific models. Even if only the first criterion is satisfied, meaning the model generates predictions faster than domain-specific models including their training time, it is still suitable for applications where real-time inference is important. Even if only the second criterion is satisfied, meaning the model achieves forecasting accuracy comparable to or better than domain-specific models, it remains useful in cases where performance is critical but immediate inference is not required.

In this paper, we thoroughly evaluate the effectiveness of LLM-based zero-shot forecasters by comparing their inference speed and accuracy against state-of-the-art (SoTA) domain-specific forecasting models. Our experimental results indicate that LLM forecasters struggle to match the forecasting accuracy of domain-specific models. Furthermore, linear models, which can be trained on single input sequence, not only outperformed LLM-based forecasters in forecasting accuracy but also are more cost-efficient. Additionally, our analysis reveals that the low accuracy of LLMs in time-series forecasting is closely tied to the presence of noise commonly found in real-world datasets. Despite exploring various strategies to mitigate this issue in zero-shot settings, our experiments show only marginal improvements, highlighting noise sensitivity as a fundamental limitation of LLM-based forecasters.

2 Experimental Setup

Problem Definition  Time series forecasting is the task of predicting future values based solely on past observations. Formally, given a uniformly sampled \( n \) observations of time series \( s \) in which each observation has \( d \) dimensions, \( s_n(t) = \{x_t, x_{t+1}, \ldots, x_{t+n-1}\}, \quad x \in \mathbb{R}^d \), a forecaster predicts the future sequence \( s_O(t+I) \) based on \( s_I(t) \) where \( I \) and \( O \) denote an input length and output length of sequence, respectively. We define the application of a forecaster across multiple domains as follows: Given the \( N \) datasets \( \{D^1, \ldots, D^N\} \), a forecaster predicts futures based on the most recent input sequence \( s_I^j \) of each \( D^j \). To solve this problem, we introduce three approaches: prompt-based zero-shot forecasting, domain-specific forecasting, and single-shot forecasting.

Prompt-based LLM Forecasting Methods  Prompt-based zero-shot forecasting involves converting \( s_I \) into a text sequence within a prompt and then asking LLMs to continue the sequence up to the desired length \( O \). Existing methods, such as LLMTIME and LLMP (Requeima et al., 2024), provide a foundation for LLM-based forecasting, but exploring additional prompting strategies can offer deeper insights into LLM capabilities. To this end, we designed TS-CoT and TS-InContext prompts (Figure 6), inspired by Chain of Thought (CoT) and In-Context Learning. We use both black-box (GPT-3.5, GPT-4, GPT-4o) and white-box models (LLaMA 2-7B, 3-8B, 2-70B, 3-70B, 3.1-70B).

Domain-Specific Forecasting Models  Gruber et al. (2023) demonstrated that LLMTIME outperforms conventional forecasting methods trained on the entire dataset, despite relying solely on the input sequence for prediction. However, most baseline models (Wu et al., 2021; Zhou et al., 2022) are regarded as ineffective baselines (Zeng et al., 2023) and are relatively outdated compared to recently proposed forecasting models. Therefore, to assess whether utilizing LLMs for forecasting is genuinely meaningful, we conduct comparisons with recent SoTA forecasting models (Wu et al., 2023; Nie et al., 2023; Liu et al., 2024b; Wang et al., 2024). The characteristics of each model are detailed in Appendix A.1.

Domain-Specific Single-shot Linear Models  Additionally, the linear models (Zeng et al., 2023) have been shown to achieve strong performance on widely used benchmarks despite having a small number of parameters. We found that these models can be trained on the same amount of data as LLMTIME receives as input. Thus, we train linear models with only one input sequence and include them as single-shot linear models (details in Appendix B.3).

Evaluation Protocol  To evaluate computational
Figure 1: Multivariate forecasting results on five datasets from the Informer benchmark (\( I=\{96,384\} \) and \( O=\{48,192\} \)). We report the averaged MAE and log-scaled inference time for domain-specific forecasters and LLM-based forecasters using the best-performing LLMTIME prompt (results for other three prompts are provided in Table 5). (Left) Recent domain-specific forecasters achieve lower MAE than LLM-based forecasters in last sample evaluation (Gruver et al., 2023). (Right) Overall, LLM-based forecasters have longer inference times than domain-specific models, despite the latter requiring domain-specific training before inference. Moreover, single-shot linear models (DLinear-S and RLinear-S), trained solely on the input sequence of each domain, achieve significantly shorter inference times and outperform LLM-based forecasters. Appendix B.3 includes a detailed evaluation protocol. Figure 7 shows qualitative results and Figure 12 includes full results with confidence intervals.

cost, we define the total computation cost \( C \) of each approach as the sum of training cost \( C^T \) and inference cost \( C^I \). We assume that domain-specific models require separate training and inference for each dataset \( D^i \), leading to \( C_{\text{Domain}} = \frac{1}{N} \sum_{i=1}^N \left[ C^T(D^i) + C^I(s^i_I) \right] \). For pre-trained LLMs, we assume no additional training cost, so only the inference cost accumulates: \( C_{\text{LLM}} = \frac{1}{N} \sum_{i=1}^N C^I(s^i_I) \). Finally, the single-shot linear model also trained with a input sequence \( s^i \) in each dataset independently before inference, resulting in \( C_{\text{Linear}} = \frac{1}{N} \sum_{i=1}^N \left[ C^T(\{s^i_I\}) + C^I(s^i_I) \right] \). We consider LLM forecasters cost-efficient as zero-shot forecasters only if \( C_{\text{LLM}} < C_{\text{domain}} \) and \( C_{\text{LLM}} < C_{\text{Linear}} \). Appendix B.3 shows details for calculation of the inference time. We measured the mean absolute error (MAE) and the mean squared error (MSE) to evaluate the forecasts.

Benchmark Datasets We evaluate all forecasters using three benchmark datasets: Monash (Godahewa et al., 2021), and Function (Gruver et al., 2023), Informer (Zhou et al., 2021) datasets. The Monash dataset consists of real-world time series from various domains. The Function dataset contains numerical sequences from mathematical functions to test pattern learning and extrapolation. The Informer dataset includes multivariate time-series data for long-sequence forecasting, having larger test set than others. Thus, to reduce LLM costs, we evaluate performance using only the last test sample. Appendix B.1 provide details of the datasets.

3 Comparison with SoTA Domain-Specific Forecasters

Finding 1: SoTA domain-specific models are more accurate and cost-efficient than prompt-based LLM forecasters.

We compare LLM-based forecasting models with SoTA domain-specific models (TimeMixer and iTransformer) in terms of computational efficiency and accuracy. As shown in Figure 1, LLMTIME with LaMA-3.1-70B performs comparably to forecasting models such as Autoformer and FEDformer, as evaluated in their original paper. However, it falls short compared to more recent SoTA models. Notably, LLM forecasters underperform compared to single-shot linear models (DLinear-S and RLinear-S), despite these linear models being trained on just a single input sequence. Moreover, these linear models achieve significantly shorter inference time, even including training time, compared to LLMTIME with LLaMA-2-7B.

Due to space constraints, we provide additional experimental results in Appendix C.2. Appendix C.2.1 presents an analysis of how different decoding settings (e.g., temperature and top-p values) affect forecasting performance across various configurations, as summarized in Tables 6 and 7. Appendix C.2.2 extends the evaluation beyond the last-sample setting to cover the entire temporal horizon in univariate forecasting, as shown in Table 8. Finally, Appendix C.2.4 pro-
<table>
  <tr>
    <th rowspan="2">Models</th>
    <th colspan="2">Clean</th>
    <th colspan="2">Gaussian</th>
    <th colspan="2">Constant</th>
    <th colspan="2">Missing</th>
  </tr>
  <tr>
    <th>MAE</th><th>MSE</th>
    <th>MAE</th><th>MSE</th>
    <th>MAE</th><th>MSE</th>
    <th>MAE</th><th>MSE</th>
  </tr>
  <tr>
    <td>GPT-3.5</td><td>0.8247</td><td>1.4644</td><td>0.9215</td><td>1.6484</td><td>0.9018</td><td>1.4923</td><td>0.9897</td><td>1.7561</td>
  </tr>
  <tr>
    <td>LLAMA2 7B</td><td>0.8875</td><td>1.5366</td><td>0.8831</td><td>1.4087</td><td>0.9380</td><td>1.6075</td><td>0.9482</td><td>1.4286</td>
  </tr>
  <tr>
    <td>LLAMA3 5B</td><td>0.7205</td><td>1.0185</td><td>1.0301</td><td>1.9377</td><td>0.9378</td><td>1.4531</td><td>1.0001</td><td>1.5850</td>
  </tr>
  <tr>
    <td>GPT-4</td><td>0.7196</td><td>1.0028</td><td>0.7776</td><td>1.2476</td><td>0.8773</td><td>1.3353</td><td>0.8404</td><td>1.2749</td>
  </tr>
  <tr>
    <td>LLAMA2 70B</td><td>0.7196</td><td>1.0028</td><td>0.7776</td><td>1.2476</td><td>0.8773</td><td>1.3353</td><td>0.8404</td><td>1.2749</td>
  </tr>
  <tr>
    <td>LLAMA3 70B</td><td>0.7006</td><td>0.9596</td><td>0.8870</td><td>1.5695</td><td>0.8184</td><td>1.1474</td><td>0.8461</td><td>1.1829</td>
  </tr>
  <tr>
    <td>DLinear-S</td><td>0.7074</td><td>0.8890</td><td>0.7063</td><td>0.8896</td><td>0.7120</td><td>0.8997</td><td>0.7112</td><td>0.8910</td>
  </tr>
  <tr>
    <td>RLinear-S</td><td>0.7182</td><td>0.8804</td><td>0.7196</td><td>0.8817</td><td>0.7171</td><td>0.8766</td><td>0.7438</td><td>0.9971</td>
  </tr>
</table>

Table 1: When the Monash dataset contains Gaussian, Constant, and Missing noise, commonly used to evaluate noise robustness (Cheng et al., 2024), LLM-based forecasters experience a performance decline, while Linear-S models perform similarly to clean data.

vides additional baseline comparisons, including results for ARIMA (Box and Jenkins, 1968) and N-BEATS (Oreshkin et al., 2019), alongside the linear models.

3.1 Origin of LLMs Performance Degradation

Finding 2: LLMs are highly sensitive to noise, while linear models remain robust.

Another point of inquiry stems from the fact that LLMs have clearly demonstrated near-perfect performance on various functional datasets and accurate prediction results in the Monash dataset (Gruver et al., 2023). In reproducing these experiments, we discovered that these datasets, unlike Informer datasets, contain very little noise and are clean.

Qualitative Results on the Function Dataset For the function dataset, Figure 2 shows how the performance of LLMs changes when a Gaussian noise is added to the original data. Even a very small amount of noise, relative to the original scale, can lead to significantly larger errors than the original predictions. This finding suggests that LLMs are highly sensitive to noise.

Quantitative Results on the Monash Dataset We also compare LLMs with single-shot linear models using the Monash dataset across various noise types, which are typically addressed in the noise robustness literature. Table 1 shows that LLM-based methods significantly decline in performance compared to their original performance when exposed to three types of noise (details in Appendix B.4). On the other hand, the linear models maintain nearly the same level of performance even under various types of noise. This result aligns with the theoretical findings of Cheng et al. (2024), which suggest that models optimized with L1 and L2 loss exhibit robustness to noise. In addition to the three representative noise types discussed here, further evaluations under additional noise conditions—such as frequency-based noise—are presented in Appendix C.2.3.

<table>
  <tr>
    <th rowspan="2">Models</th>
    <th rowspan="2">Filtering</th>
    <th colspan="2">ETTM2</th>
    <th colspan="2">Exchange Rate</th>
    <th colspan="2">Weather</th>
  </tr>
  <tr>
    <th>MAE</th><th>MSE</th>
    <th>MAE</th><th>MSE</th>
    <th>MAE</th><th>MSE</th>
  </tr>
  <tr>
    <td>GPT-4o</td><td>X</td><td>0.2322</td><td>0.1237</td><td>0.3595</td><td>0.2893</td><td>0.1880</td><td>0.1147</td>
  </tr>
  <tr>
    <td>GPT-4o</td><td>Gaussian</td><td>0.1995</td><td>0.1097</td><td>0.3718</td><td>0.2643</td><td>0.1698</td><td>0.0827</td>
  </tr>
  <tr>
    <td>GPT-4o</td><td>EMA</td><td>0.2501</td><td>0.1519</td><td>0.3615</td><td>0.2882</td><td>0.1808</td><td>0.0972</td>
  </tr>
  <tr>
    <td>LLAMA3.1 70B</td><td>X</td><td>0.2462</td><td>0.1579</td><td>0.4466</td><td>0.3596</td><td>0.1659</td><td>0.0741</td>
  </tr>
  <tr>
    <td>LLAMA3.1 70B</td><td>Gaussian</td><td>0.2659</td><td>0.1868</td><td>0.4323</td><td>0.3579</td><td>0.1735</td><td>0.0981</td>
  </tr>
  <tr>
    <td>LLAMA3.1 70B</td><td>EMA</td><td>0.2431</td><td>0.1661</td><td>0.4323</td><td>0.3579</td><td>0.1735</td><td>0.0981</td>
  </tr>
  <tr>
    <td>DLinear-S</td><td>X</td><td>0.2059</td><td>0.1782</td><td>0.2296</td><td>0.1166</td><td>0.1643</td><td>0.0722</td>
  </tr>
  <tr>
    <td>RLinear-S</td><td>X</td><td>0.1824</td><td>0.0677</td><td>0.2571</td><td>0.1370</td><td>0.1392</td><td>0.0469</td>
  </tr>
</table>

Table 2: When two noise filtering methods (Gaussian and EMA) are applied to LLM-based models on the Informer dataset, performance improves (highlighted in red). Nevertheless, single-shot linear models consistently outperform LLM-based models.

Increasing Input Length to Improve Noise Robustness Because the robustness of conventional time-series models to noise depends on the number of training samples, we investigated whether providing longer input sequences could enhance the resilience of LLMs to noise. We hypothesized that using short input sequences might limit LLMs’ ability to distinguish noise; therefore, we evaluated their performance by gradually increasing the sequence length. As illustrated in Figure 3, where the period in the Fourier series corresponds to an input sequence length, LLMs typically show only slight improvements in performance, whereas linear models gain significantly from longer sequences.

What If Applying Noise Filtering? The straightforward way to deal with noise is to remove it as a pre-processing for input. To observe how they affect performance, we applied two noise filtering techniques—Gaussian and Exponential Moving Average (EMA) filtering—on real-world datasets (details in Appendix B.4). As shown in Table 2, even after applying noise filtering techniques, LLM performance improves only slightly or remains unchanged.

3.2 Enhancing Noise Robustness of LLMs

Finding 3: Additional input samples and noise filtering improve noise robustness of LLMs, but limited.

In this paper, we empirically demonstrate the limitations of LLM-based forecasting in real-world scenarios where noise is prevalent. A potential reason for LLMs’ susceptibility to noise lies in their token-
Figure 2: Performance variations in the Function dataset based on noise levels. LLMTune (GPT-4) performed perfectly on clean data, but even a slight noise (\emph{e.g.}, Gaussian noise with a standard deviation of 0.001) addition within the 0-1 input scale significantly increased MAE.

Figure 3: Increasing the input sequence length in prompts for LLM-based models leads to a slight performance improvement, but the gains remain minimal compared to DLinear-S.

based encoding, which amplifies distortions in representation rather than reflecting actual numerical differences. This distortions hinder their ability to recognize meaningful patterns, making extrapolation more challenging. Given the strong reasoning capabilities of LLMs, we argue that rather than emphasizing zero-shot forecasting, a more promising research direction would be enhancing the capability to process numerical sequences through fine-tuning (Hu et al., 2022; Zha et al., 2022; Dettmers et al., 2023).

5 Limitations

Our study provides a comprehensive evaluation of LLM-based zero-shot forecasting, but there are areas that warrant further exploration. While we identify the sensitivity of LLMs to noise, the exact mechanisms behind this issue, such as the impact of tokenization and encoding, deserve deeper investigation. Our evaluation prioritizes computational cost and accuracy, yet aspects like interpretability, adaptability to diverse domains, and the potential benefits of integrating textual or multimodal data could offer valuable insights for future studies. Additionally, our experiments cover widely used benchmark datasets, but assessing LLM performance across an even broader range of real-world forecasting tasks would further strengthen our understanding of their capabilities and limitations.

Acknowledgments

This work was supported by the Institute for Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (RS-2019-II190075, Artificial Intelligence Graduate School Program at KAIST) and the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. RS-2025-00555621).

References

George EP Box and Gwilym M Jenkins. 1968. Some recent advances in forecasting and control. Journal of the Royal Statistical Society. Series C (Applied Statistics), 17(2):91–109.

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.

Hao Cheng, Qingsong Wen, Yang Liu, and Liang Sun. 2024. Robusttsf: Towards theory and design of robust time series forecasting with anomalies. In Proc. the International Conference on Learning Representations (ICLR).

Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. Qlora: Efficient finetuning of quantized llms. arXiv preprint arXiv:2305.14314.

Rakshitha Godahewa, Christoph Bergmeir, Geoffrey I Webb, Rob J Hyndman, and Pablo Montero-Manso. 2021. Monash time series forecasting archive. arXiv preprint arXiv:2105.06643.

Nate Gruver, Marc Finzi, Shikai Qiu, and Andrew G Wilson. 2023. Large language models are zero-shot time series forecasters. In Proc. the Advances in
Neural Information Processing Systems (NeurIPS), volume 36.

Daehoon Gwak, Junwoo Park, Minho Park, Chaehun Park, Hyunchan Lee, Edward Choi, and Jaegul Choo. 2024. Forecasting future international events: A reliable dataset for text-based event modeling. In Proc. of the Conference on Empirical Methods in Natural Language Processing (EMNLP).

Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2019. The curious case of neural text degeneration. arXiv preprint arXiv:1904.09751.

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen. 2022. Lora: Low-rank adaptation of large language models. In Proc. the International Conference on Learning Representations (ICLR).

Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Y Zhang, Xiaoming Shi, Pin-Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, et al. 2023. Time-llm: Time series forecasting by reprogramming large language models. arXiv preprint arXiv:2310.01728.

Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yu-taka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. In Proc. the Advances in Neural Information Processing Systems (NeurIPS).

Peiyuan Liu, H Guo, T Dai, N Li, J Bao, X Ren, Y Jiang, and ST Xia. 2024a. Calf: Aligning llms for time series forecasting via cross-modal fine-tuning. arXiv preprint arXiv:2403.07300.

Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. 2024b. itransformer: Inverted transformers are effective for time series forecasting. In Proc. the International Conference on Learning Representations (ICLR).

Minh Nguyen, Andrew Baker, Clement Neo, Allen Roush, Andreas Kirsch, and Raviv Shwartz-Ziv. 2024. Turning up the heat: Min-p sampling for creative and coherent llm outputs. arXiv preprint arXiv:2407.01082.

Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. 2023. A time series is worth 64 words: Long-term forecasting with transformers. In Proc. the International Conference on Learning Representations (ICLR).

Boris N Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. 2019. N-beats: Neural basis expansion analysis for interpretable time series forecasting. In Proc. the International Conference on Learning Representations (ICLR).

Junwoo Park, Daehoon Gwak, Jaegul Choo, and Edward Choi. 2024. Self-supervised contrastive learning for long-term forecasting. In Proc. the International Conference on Learning Representations (ICLR).

Junwoo Park, Jungsoo Lee, Youngin Cho, Woncheol Shin, Dongmin Kim, Jaegul Choo, and Edward Choi. 2023. Deep imbalanced time-series forecasting via local discrepancy density. In Proc. of Machine Learning and Knowledge Discovery in Databases: Research Track (ECML/PKDD), pages 139–155. Springer Nature Switzerland.

Xiangyu Peng, Congying Xia, Xinyi Yang, Caiming Xiong, Chien-Sheng Wu, and Chen Xing. 2024. Regenesis: LLMs can grow into reasoning generalists via self-improvement. arXiv preprint arXiv:2410.02108.

James Requeima, John Bronskill, Dami Choi, Richard E Turner, and David Duvenaud. 2024. Llm processes: Numerical predictive distributions conditioned on natural language. In Proc. the Advances in Neural Information Processing Systems (NeurIPS).

Mingtian Tan, Mike A Merrill, Vinayak Gupta, Tim Althoff, and Thomas Hartvigsen. 2024. Are language models actually useful for time series forecasting? In Proc. the Advances in Neural Information Processing Systems (NeurIPS).

Hua Tang, Chong Zhang, Mingyu Jin, Qinkai Yu, Zhenting Wang, Xiaobo Jin, Yongfeng Zhang, and Mengnan Du. 2025. Time series forecasting with llms: Understanding and enhancing model capabilities. ACM SIGKDD Explorations Newsletter.

Huqiang Wang, Jian Peng, Feihu Huang, Jince Wang, Junhui Chen, and Yifei Xiao. 2023. Micn: Multiscale local and global context modeling for long-term series forecasting. In Proc. the International Conference on Learning Representations (ICLR).

Shiyu Wang, Haixu Wu, Xiaoming Shi, Tengge Hu, Huakun Luo, Lintao Ma, James Y Zhang, and Jun Zhou. 2024. Timemixer: Decomposable multiscale mixing for time series forecasting. In Proc. the Advances in Neural Information Processing Systems (NeurIPS).

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837.

Andrew Robert Williams, Arjun Ashok, Étienne Marotte, Valentina Zantedeschi, Jithendaraa Subramanian, Roland Riachi, James Requeima, Alexandre Lacoste, Irina Rish, Nicolas Chapados, et al. 2024. Context is key: A benchmark for forecasting with essential textual information. arXiv preprint arXiv:2410.18959.

Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. 2023. Timesnet: Temporal 2d-variation modeling for general time series analysis. In Proc. the International Conference on Learning Representations (ICLR).
Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. 2021. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. In Proc. the Advances in Neural Information Processing Systems (NeurIPS).

Zhijian Xu, Ailing Zeng, and Qiang Xu. 2024. Fits: Modeling time series with 10k parameters. In Proc. the International Conference on Learning Representations (ICLR).

Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. 2022. STar: Bootstrapping reasoning with reasoning. In Advances in Neural Information Processing Systems.

Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. 2023. Are transformers effective for time series forecasting? In Proc. the AAAI Conference on Artificial Intelligence (AAAI).

Kaiwen Zha, Peng Cao, Yuzhe Yang, and Dina Katabi. 2022. Supervised contrastive regression. arXiv preprint arXiv:2210.01189.

Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. 2021. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proc. the AAAI Conference on Artificial Intelligence (AAAI).

Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. 2022. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In Proc. the International Conference on Machine Learning (ICML).

Tian Zhou, Peisong Niu, Liang Sun, Rong Jin, et al. 2023. One fits all: Power general time series analysis by pretrained Im. Proc. the Advances in Neural Information Processing Systems (NeurIPS), 36:43322–43355.

Zihao Zhou and Rose Yu. 2025. Can llms understand time series anomalies? In Proc. the International Conference on Learning Representations (ICLR).
A  Related Work

A.1  Domain-Specific Models for Time-series Forecasting

Time-series forecasting has evolved from statistical methods to deep learning-based approaches. Informer (Zhou et al., 2021) improved Transformer efficiency with ProbSparse self-attention, reducing computational overhead for long-term forecasting. Early models like LTSF-Linear (Zeng et al., 2023) demonstrated that simple linear regression could outperform complex architectures in structured datasets. PatchTST (Nie et al., 2023) introduced patching techniques inspired by vision models, enabling better locality retention. TimesNet (Wu et al., 2023) leveraged temporal 2D-variation modeling, transforming time series into structured tensors for more efficient feature extraction. iTransformer (Liu et al., 2024b) addressed multivariate dependencies by applying attention mechanisms to variate tokens rather than time steps. TimeMixer (Wang et al., 2024) introduced a fully MLP-based approach, effectively capturing multi-scale variations.

Compared to LLM-based methods, these domain-specific models optimize for numerical representation, long-term dependencies (Park et al., 2024), and computational efficiency (Xu et al., 2024). While LLMs process time-series data as token sequences, they lack specialized multiscale decomposition and structured feature extraction found in these tailored models. Informer and PatchTST improve sparsity and efficiency, whereas iTransformer and TimesNet enhance multivariate correlation modeling. Linear models remain relevant, challenging the necessity of deep learning for structured time-series tasks. The progression from linear models to efficient deep learning architectures highlights the importance of designing task-specific solutions for forecasting. Future research should explore hybrid approaches that combine LLM flexibility with time-series-specific optimizations to further improve performance and efficiency.

A.2  LLM-based Time-series Forecasting

The use of LLMs for time series forecasting has recently gained attention, with two primary approaches emerging in the literature. One approach maps numerical sequences into the token embedding space of LLMs with minimal training, treating time series forecasting as a natural extension of next-token prediction. However, recent studies argue that this method does not effectively leverage the capabilities of LLMs for reasoning about time series. Notably, Tan et al. (2024) raise a crucial question: Are Language Models Actually Useful for Time Series Forecasting?. They challenge the assumption that LLM-based models outperform conventional forecasting methods, demonstrating that removing or replacing the LLM component often improves results.

An alternative approach involves transforming numerical sequences into textual representations, allowing LLMs to interpret time series data through natural language prompts. This paradigm is exemplified by LLMTTime (Gruver et al., 2023), which was the first to introduce a zero-shot forecasting method using LLMs. Subsequently, LLMP (Requeima et al., 2024) introduced modifications to prompt design, incorporating elements from stochastic processes to enhance predictive accuracy.

Recent studies have increasingly emphasized the importance of textual information in forecasting tasks, demonstrating that integrating external knowledge sources can significantly enhance predictive performance. In particular, LLM-based approaches have shifted towards context-aided forecasting, where models utilize domain-specific textual inputs to inform future predictions. For example, Gwak et al. (2024) show that LLMs can successfully predict future events from past news articles, underscoring their ability to capture temporal and contextual signals from language.

Building on this line of research, the CiK benchmark (Williams et al., 2024) provides a systematic framework for evaluating the role of auxiliary textual information in time-series forecasting. It demonstrates that models incorporating relevant domain knowledge significantly outperform purely numerical baselines. Moreover, CiK proposes evaluation protocols to assess how effectively LLMs integrate multimodal context, including both structured and unstructured inputs. These developments suggest that future research should focus not just on zero-shot capabilities of LLMs, but on methods that enable them to synergistically leverage textual and numerical data for more robust forecasting.
<table>
  <tr>
    <th>Dataset</th>
    <th>Domain</th>
    <th>Length</th>
    <th>Channels</th>
    <th>frequency</th>
  </tr>
  <tr>
    <td>ETTm2</td>
    <td>System Monitoring</td>
    <td>69680</td>
    <td>7</td>
    <td>15 minutes</td>
  </tr>
  <tr>
    <td>Electricity</td>
    <td>Energy Consumption</td>
    <td>26304</td>
    <td>321</td>
    <td>1 Hour</td>
  </tr>
  <tr>
    <td>Traffic</td>
    <td>Traffic</td>
    <td>17544</td>
    <td>862</td>
    <td>1 Hour</td>
  </tr>
  <tr>
    <td>Weather</td>
    <td>Weather</td>
    <td>52695</td>
    <td>21</td>
    <td>10 Minutes</td>
  </tr>
  <tr>
    <td>ExchangeRate</td>
    <td>Economic</td>
    <td>7588</td>
    <td>8</td>
    <td>1 Day</td>
  </tr>
  <tr>
    <td>Monash Dataset</td>
    <td>Diverse domains</td>
    <td>[144-900]</td>
    <td>8</td>
    <td>-</td>
  </tr>
  <tr>
    <td>Function Dataset</td>
    <td>Mathematical Equations</td>
    <td>200</td>
    <td>6</td>
    <td>-</td>
  </tr>
</table>

Table 3: Statistics of Informer, Monash, and Function datasets. In the last sample evaluation, we compare the performance of each model by taking the last \( I + O \) length from each dataset. Since LLMs predict each channel independently in multivariate forecasting, the number of sequences that LLMs need to predict is equal to the number of channels.

B Experiment Details

B.1 Dataset Details

To compare LLM forecasters with domain-specific models, we selected three commonly used datasets: Informer (Zhou et al., 2021), Monash (Godahewa et al., 2021), and Function datasets (Gruver et al., 2023). The Informer dataset is a widely used benchmark for evaluating forecasting models, including multivariate time-series data from different domains such as ETTm2, Exchange Rate, Electricity, Traffic and Weather. This dataset mainly focuses on long-sequence forecasting. The Monash consists of public benchmark datasets from the Monash Time Series Forecasting Archive, covering various real-world domains such as energy consumption, traffic, and finance. Among these, we use eight time series from LLMTIme (Gruver et al., 2023). Function dataset contains numerical time series sampled from mathematical functions (\( e.g. \), sine, sigmoid, beat interference) to assess forecasting models’ ability to learn and extrapolate structured numerical patterns.

B.2 Licenses and Terms of Use for Artifacts

We utilize several datasets and pre-trained models, each with specific licensing terms. The Monash Time Series Forecasting (TSF) Repository is licensed under Creative Commons Attribution 4.0 International (CC BY 4.0), allowing research use with proper attribution\footnote{https://huggingface.co/datasets/Monash-University/monash_tsf} The Informer datasets do not explicitly state a license, but users are advised to check the official repository for terms of use\footnote{https://github.com/zhouhaoyi/Informer2020}. The Function dataset similarly lacks stated licensing details, requiring direct consultation with the authors. The study also employs pre-trained models, including GPT-3.5 and GPT-4, which are governed by OpenAI’s proprietary API terms, allowing usage under strict conditions\footnote{https://openai.com/terms}. LLaMA models, provided by Meta AI, are subject to a research-focused license, permitting non-commercial research use\footnote{https://ai.facebook.com/blog/large-language-model-llama-meta-ai}. Users must comply with each artifact’s licensing conditions, particularly regarding research-only restrictions and commercial use limitations.

AI Assistants in Research or Writing We utilized AI assistants, specifically ChatGPT, to support research, writing, and coding. The AI was used for tasks such as drafting text, summarizing information, and refining explanations. This usage is documented to ensure transparency and acknowledgment of AI contributions.

B.3 Details of Comparison with SoTA Domain-Specific Models

Evaluation Protocol A widely used evaluation method for time-series forecasting involves splitting the entire series into training, validation, and test sets while maintaining the chronological order. Specifically, the most recent 20% of the time series is designated as the test set, while the remaining portion is used for training and hyperparameter tuning through validation. Given that time-series forecasting inherently involves predicting future values, it is crucial to split the dataset sequentially to prevent data leakage.

Once the dataset is split into these temporal segments, a sliding window approach is employed, where a fixed-length window moves with a stride through the time series. Each window consists of an input sequence of length \( I \) and an output sequence of length \( O \), which serves as the ground truth for evaluating model predictions. The model is trained by minimizing the loss between its predicted output sequence and the actual values. Table 3 provides statistical information about the datasets used, including their total length and the number of samples obtained after applying the sliding window method.

Last Sample Evaluation While this evaluation method is suitable for domain-specific models, it introduces significant computational and financial costs when applied to LLM-based approaches due to the increased number of API calls and processing
requirements. To mitigate this burden, LLMTime employs a reduced evaluation strategy, selecting only the last sample from the Informer dataset for testing. Consequently, the number of test samples in each domain is proportional to the number of channels in that domain. Since LLMTime does not disclose the exact methodology used for this selection, our evaluation follows a similar approach by selecting the last available sample from the conventional time-series split. While this approach prevents direct comparison with results from the original paper, we confirm that the general trend of the findings remains consistent.

Domain-Specific Model Details In the field of time-series forecasting, numerous models have emerged since the introduction of Informer, each showcasing strong performance with distinct innovations. We concentrate on evaluating the most advanced and high-performing models from recent developments. Our approach is validated against recent seven forecasting baselines, with all models implemented in PyTorch. For the latest forecasting models—including DLinear5, MICN6 (Wang et al., 2023), and PatchTST7 (Nie et al., 2023), TimesNet (Wu et al., 2023), iTransformer (Liu et al., 2024b), TimeMixer8 (Wang et al., 2024)—we utilized the official implementations provided by the original authors rather than developing them independently. For domain-specific models, we conducted five independent training and inference runs, varying the random seed in each run.

LLM-based Model Details For LLM-based models, we performed five inference runs on the same test sample and computed the median of the five predictions to mitigate the impact of stochastic variations due to temperature settings. This median aggregation method is commonly used in LLM-based models to reduce the affect of outlier predictions. As a result, domain-specific models report confidence intervals in the results, while LLM-based models do not due to the fivefold increase in computational cost that would be required. The API model utilizes the API provided by OpenAI 9, while all LLaMA models use pretrained models provided by Hugging Face 10.

Training Strategy for Linear-S Models Linear-

<table>
  <tr>
    <th>Informer Dataset</th>
    <th>d</th>
    <th>I</th>
    <th>O</th>
    <th>K</th>
    <th>I'</th>
    <th>O'</th>
  </tr>
  <tr>
    <td>ETTM2</td>
    <td>7</td>
    <td>384</td>
    <td>192</td>
    <td>1,351</td>
    <td>96</td>
    <td>96</td>
  </tr>
  <tr>
    <td>Exchange Rate</td>
    <td>8</td>
    <td>384</td>
    <td>192</td>
    <td>1,544</td>
    <td>96</td>
    <td>96</td>
  </tr>
  <tr>
    <td>Weather</td>
    <td>21</td>
    <td>384</td>
    <td>192</td>
    <td>4,053</td>
    <td>96</td>
    <td>96</td>
  </tr>
  <tr>
    <td>Electricity</td>
    <td>321</td>
    <td>96</td>
    <td>48</td>
    <td>15,729</td>
    <td>24</td>
    <td>24</td>
  </tr>
  <tr>
    <td>Traffic</td>
    <td>862</td>
    <td>96</td>
    <td>48</td>
    <td>42,238</td>
    <td>24</td>
    <td>24</td>
  </tr>
</table>

Table 4: The number of windows \( K \) for training and validating the single-shot linear models according to the internal prediction length \( O' \) and input length \( I' \) for each dataset with the target input \( I \) and output \( O \) lengths.

S models follow a different training and inference process. These models use a single input sequence only for training. To conduct both training and validation with a single sample, we need to create a windowed dataset from the input sequence, similar to when training a domain-specific model. Given an input sequence length \( I \) and a target prediction length \( O \), we transform this into a smaller input length \( I' \) and output length \( O' \) to construct a windowed dataset. The total number of windows \( K \) is determined as follows:

\[
K = d(I - (I' + O') + 1)
\]

where \( d \) is the number of channels. The linear model employs a channel independence strategy (Nie et al., 2023), where channels are transformed into the batch axis, allowing for independent predictions across channels. As a result, in multivariate forecasting, the number of windows is proportional to the number of channels. In the sliding window framework, if \( K \) is too small, training and validation of the linear model become challenging. Therefore, with \( I, O, \) and \( d \) fixed, we need to adjust \( I' \) and \( O' \) to ensure a sufficient number of windowed data points. If \( O' \) is smaller than \( O \), inference must be performed in an autoregressive manner, which can lead to accumulated errors and degrade performance. Considering these characteristics, we set \( I' \) and \( O' \) as half of \( I \) and \( O' \), respectively. The statistics of each dataset are presented in Table 4.

Total Inference Time Calculation To compare the inference speeds of domain-specific models and LLM-based models, we measure the total time required for a single prediction. For domain-specific models, this total time includes the training time (utilizing both the training and validation sets) and the inference time required to generate a single prediction. For API-based LLM models (e.g.,
GPT-3.5, GPT-4 and GPT-4o), the total inference time includes the duration required to query the API five times and compute the median prediction. Since these queries are processed in asynchronous threading, The runtime for executing five queries is not five times the runtime of a single query. Moreover, reducing the number of queries to a single run results in significantly degraded performance, making this trade-off necessary. For Local-deployed LLM models (e.g., LLaMA-2-70B, LLaMA-3.1-70B), we measured the time taken to perform a batch of five inference queries. Due to the 40GB GPU memory limitation, the 70B model was loaded in 4-bit mode for inference. All local models, except for the API model, were executed in the same computing environment with an A100-40GB GPU.

B.4 Details of Noise Experiments

Noise Types Cheng et al. (2024) categorize three primary types of noise in time series forecasting: Constant, Missing and Gaussian. Constant noise introduce a fixed deviation from the ground-truth value. These noise can be mathematically represented as:

\[
z_A = z + \epsilon,
\]

where \( z_A \) is the observed value with anomaly, \( z \) is the true value, and \( \epsilon \) is a constant perturbation. Missing noise correspond to completely missing values in the dataset. These noise are defined as:

\[
z_A = \epsilon,
\]

where \( \epsilon \) is a constant, often set to zero. Gaussian noise represent deviations that follow a Gaussian distribution. These noise are defined as:

\[
z_A = z + \epsilon, \quad \epsilon \sim \mathcal{N}(0, \sigma^2),
\]

where \( \epsilon \) is drawn from a normal distribution with mean 0 and variance \( \sigma^2 \). We conducted experiments by injecting three types of noise into the Monash dataset to evaluate the robustness of the LLM to each type of noise.

The single-shot linear models exhibit greater robustness to noise compared to LLM-based models due to the statistical properties of their loss functions. The L1 loss function maintains a constant sum of losses under clean and noisy conditions, ensuring that optimization remains unaffected by noise. Theoretical results (Cheng et al., 2024) support our empirical findings. Since LLMs are token-based models and utilized as zero-shot forecasters, achieving noise robustness is more challenging for them.

Beyond the three representative types of noise discussed above, time-series data often involve more complex challenges such as changes in trend and variations in frequency (Park et al., 2023; Zhou and Yu, 2025). To address this, we additionally conducted experiments comparing LLM-based models and domain-specific models under scenarios involving frequency changes as shown in Table 9.

Noise Filtering Methods Gaussian filtering and EMA filtering are both widely used techniques for noise reduction in signal and image processing. Gaussian filtering applies a Gaussian kernel to smooth data by averaging neighboring values with weights determined by a normal distribution, effectively reducing high-frequency noise while preserving important structures. It is commonly used in image processing to blur and remove noise. On the other hand, EMA filtering is a recursive technique that assigns exponentially decreasing weights to past observations, allowing recent data points to have more influence. This makes EMA particularly effective for real-time applications, such as financial analysis and sensor data smoothing, where responsiveness to recent changes is crucial while still maintaining noise reduction.

However, Gaussian filtering can overly smooth time series data, potentially removing important short-term variations and trends. EMA filtering, while responsive to recent changes, may lag behind true values and be sensitive to sudden spikes. Both methods struggle with non-stationary data, where patterns and noise characteristics change over time. Therefore, applying such noise filtering to a time series before inference may degrade the original dynamics and may not necessarily improve performance.

C Additional Results

C.1 Comparison Between Prompt Methods

Based on the findings of Wei et al. (2022) and Kojima et al. (2022), Chain-of-Thought (CoT) prompting plays a crucial role in enhancing the reasoning capabilities of large language models. Wei et al. (2022) demonstrated that explicitly structuring the reasoning process in prompts allows models to perform complex, multi-step reasoning more effectively. Meanwhile, Kojima et al. (2022) showed that even simple prompts like "Let's think step by step" can significantly improve zero-shot reasoning. Building on these insights, when designing a
<table>
  <tr>
    <th>Prompt</th>
    <th>Input Length</th>
    <th colspan="2">ETTm2</th>
    <th colspan="2">Exchange Rate</th>
    <th colspan="2">Weather</th>
  </tr>
  <tr>
    <th></th>
    <th></th>
    <th>MAE</th>
    <th>MSE</th>
    <th>MAE</th>
    <th>MSE</th>
    <th>MAE</th>
    <th>MSE</th>
  </tr>
  <tr>
    <td rowspan="3">LLMTime (b of Figure 4)</td>
    <td>384</td>
    <td>0.2322</td>
    <td>0.1237</td>
    <td>0.3595</td>
    <td>0.2893</td>
    <td>0.1880</td>
    <td>0.1147</td>
  </tr>
  <tr>
    <td>576</td>
    <td>0.2741</td>
    <td>0.2037</td>
    <td>0.3323</td>
    <td>0.2539</td>
    <td>0.1943</td>
    <td>0.1231</td>
  </tr>
  <tr>
    <td>768</td>
    <td>0.2998</td>
    <td>0.2539</td>
    <td>0.3807</td>
    <td>0.4030</td>
    <td>0.2075</td>
    <td>0.1398</td>
  </tr>
  <tr>
    <td rowspan="3">LLMP (b of Figure 5)</td>
    <td>384</td>
    <td>0.2860</td>
    <td>0.1969</td>
    <td>0.3436</td>
    <td>0.2869</td>
    <td>0.2039</td>
    <td>0.1200</td>
  </tr>
  <tr>
    <td>576</td>
    <td>0.3060</td>
    <td>0.2188</td>
    <td>0.3336</td>
    <td>0.2169</td>
    <td>0.1903</td>
    <td>0.1167</td>
  </tr>
  <tr>
    <td>768</td>
    <td>0.2471</td>
    <td>0.1388</td>
    <td>0.4177</td>
    <td>0.3855</td>
    <td>0.1914</td>
    <td>0.1197</td>
  </tr>
  <tr>
    <td rowspan="3">TS-CoT (a of Figure 6)</td>
    <td>384</td>
    <td>0.2823</td>
    <td>0.1799</td>
    <td>0.3647</td>
    <td>0.2358</td>
    <td>0.1958</td>
    <td>0.1451</td>
  </tr>
  <tr>
    <td>576</td>
    <td>0.3121</td>
    <td>0.2075</td>
    <td>0.3483</td>
    <td>0.2384</td>
    <td>0.1831</td>
    <td>0.1666</td>
  </tr>
  <tr>
    <td>768</td>
    <td>0.2941</td>
    <td>0.2075</td>
    <td>0.4083</td>
    <td>0.3284</td>
    <td>0.1831</td>
    <td>0.1666</td>
  </tr>
  <tr>
    <td rowspan="3">TS-InContext (b of Figure 6)</td>
    <td>384 (1-Shot)</td>
    <td>0.2933</td>
    <td>0.1909</td>
    <td>0.3296</td>
    <td>0.2111</td>
    <td>0.1921</td>
    <td>0.1283</td>
  </tr>
  <tr>
    <td>576 (2-Shot)</td>
    <td>0.2678</td>
    <td>0.1830</td>
    <td>0.3288</td>
    <td>0.2044</td>
    <td>0.2212</td>
    <td>0.2227</td>
  </tr>
  <tr>
    <td>768 (3-Shot)</td>
    <td>0.2557</td>
    <td>0.1381</td>
    <td>0.3225</td>
    <td>0.2030</td>
    <td>0.2364</td>
    <td>0.3073</td>
  </tr>
</table>

Table 5: We compared two recently proposed prompt methods (LLMTIme and LLMP) for zero-shot forecasting and two additional prompts (TS-CoT and TS-InContext) using LLM prompting techniques (Chain-of-Thought and In-Context learning), specifically designed for time-series forecasting. LLMTIme demonstrates superior performance compared to other prompts with minimal input. In this experiment, we use GPT-4o as a LLM backbone.

prompt for time-series forecasting, it is essential to incorporate CoT principles. This involves guiding the model through a structured, step-by-step approach to analyzing trends, identifying patterns, and making predictions based on historical data. By explicitly prompting the model to reason through the forecasting process, we design TS-CoT Prompt (Figure 6) to improve the accuracy and reliability of predictions in time-series forecasting.

Building on the insights from Brown et al. (2020), In-Context Learning (ICL) serves as a powerful mechanism for improving downstream tasks when using LLMs (Zelikman et al., 2022; Peng et al., 2024). These studies introduced the concept of few-shot learning, demonstrating that language models can generalize from a limited number of examples without fine-tuning. Zelikman et al. (2022) extended this idea with STaR, enabling self-improvement through iterative reasoning, while Peng et al. (2024) proposed ReGenesis, a framework for continual self-improvement via model-generated feedback. By leveraging these principles, we design TS-InContext prompt (Figure 6), which provides a structured set of in-output sequences within the prompt itself, allowing the model to recognize patterns.

C.2 Model Analysis

C.2.1 LLMs’ Hyperparameter Analysis

Temperature. We adopt the temperature values reported by LLMTIme, which were selected to optimize zero-shot forecasting performance. To assess the sensitivity of LLM performance to this hyperparameter, we conduct an ablation study varying the temperature during inference. Table 6 presents the MAE and MSE scores across three datasets.

On ETTm2 and Weather, the default temperature (1.0) yields the best performance. For Exchange Rate, a lower temperature (0.5) marginally improves the error metrics but still fails to outperform linear baselines. These results indicate that while temperature affects performance to a limited degree, the general trend persists: LLMs consistently underperform compared to domain-specific models across all datasets.

(a) LLMTIme for base models

<table>
  <tr>
    <th>MAE</th>
    <th>MSE</th>
    <th>MAE</th>
    <th>MSE</th>
    <th>MAE</th>
    <th>MSE</th>
  </tr>
  <tr>
    <td>-12</td>
    <td>-13</td>
    <td>-15</td>
    <td>-7</td>
    <td>-11</td>
    <td>-6</td>
    <td>43</td>
    <td>98</td>
    <td>43</td>
    <td>-10</td>
  </tr>
  <tr>
    <td>-11</td>
    <td>-9</td>
    <td>-11</td>
    <td>-12</td>
    <td>-9</td>
    <td>-10</td>
    <td>-12</td>
    <td>-8</td>
    <td>-9</td>
    <td>-13</td>
  </tr>
</table>

Response: -9, -10, -12, -8, -9

(b) LLMTIme for instruction-tuned models

System Prompt
You are a helpful assistant that performs time series predictions. The user will provide a sequence and you will predict the remaining sequence. The sequence is represented by decimal strings separated by commas.

User Prompt
Please predict next sequence following input sequence without producing any additional text. Do not say anything like 'the next terms in the sequence are', just return the numbers. Input Sequence: -12, -13, -15, -7, -11, -6, 43, 98, 43, -10, -11, -12, -9, -10, -12, -8, -9, -13,

Response: -9, -10, -12, -8, -9

Figure 4: Two prompts proposed by LLMTIme for (a) base LLM models (LLaMA2-7B, 70B, and LLaMA3.1-70B) and (b) instruction-tuned LLM models (GPT-3.5, GPT-4, GPT-4o, and LLaMA-Intstruct).

<table>
  <tr>
    <th>Models</th>
    <th>Temp.</th>
    <th colspan="2">ETTm2</th>
    <th colspan="2">Exchange Rate</th>
    <th colspan="2">Weather</th>
  </tr>
  <tr>
    <th></th>
    <th></th>
    <th>MAE</th>
    <th>MSE</th>
    <th>MAE</th>
    <th>MSE</th>
    <th>MAE</th>
    <th>MSE</th>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>0.1</td>
    <td>0.5255</td>
    <td>0.9171</td>
    <td>0.4823</td>
    <td>0.4568</td>
    <td>0.2356</td>
    <td>0.1662</td>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>0.5</td>
    <td>0.3057</td>
    <td>0.2383</td>
    <td>0.3487</td>
    <td>0.2631</td>
    <td>0.2463</td>
    <td>0.1988</td>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>1.0</td>
    <td>0.2322</td>
    <td>0.1237</td>
    <td>0.3595</td>
    <td>0.2893</td>
    <td>0.1880</td>
    <td>0.1147</td>
  </tr>
  <tr>
    <td>DLinear-S</td>
    <td></td>
    <td>0.2039</td>
    <td>0.0782</td>
    <td>0.2296</td>
    <td>0.1166</td>
    <td>0.1643</td>
    <td>0.0522</td>
  </tr>
  <tr>
    <td>RLinear-S</td>
    <td></td>
    <td>0.1824</td>
    <td>0.0677</td>
    <td>0.2571</td>
    <td>0.1370</td>
    <td>0.1392</td>
    <td>0.0469</td>
  </tr>
</table>

Table 6: Forecasting performance by temperatures

Top-p Sampling. Top-p (nucleus) sampling is a widely adopted decoding strategy in modern LLMs, striking a balance between coherence and diversity (Holtzman et al., 2019; Nguyen et al., 2024). It is the default in models such as GPT-4o and LLaMA, and is also used in LLMTIme. We perform an ablation study to evaluate how varying the Top-p parameter impacts zero-shot forecasting.

As shown in Table 7, the default value (0.8) leads to strong performance among the tested configurations. However, consistent with the temperature study, GPT-4o remains inferior to linear models
<table>
  <tr>
    <th colspan="2">(a) LLMP-Independent (Multi-turn)</th>
  </tr>
  <tr>
    <th colspan="2">System Prompt</th>
  </tr>
  <tr>
    <td colspan="2"><b>User Prompt</b><br>0.-12n 1.-13n 2.-15n 3.-7n 4.-11n 5.-6n 6.43n<br>7.98n 8.43n 9.-10n 10.-11n 11.-9n 12.-11n 13.-12n<br>14.-9n 15.-10n 16.-12n 17.-8n 18.-9n 19.-13n 20.,<br>Response: -9</td>
  </tr>
  <tr>
    <td colspan="2"><b>User Prompt</b><br>0.-12n 1.-13n 2.-15n 3.-7n 4.-11n 5.-6n 6.43n<br>7.98n 8.43n 9.-10n 10.-11n 11.-9n 12.-11n 13.-12n<br>14.-9n 15.-10n 16.-12n 17.-8n 18.-9n 19.-13n 21.,<br>Response: -10</td>
  </tr>
  <tr>
    <td colspan="2"><b>User Prompt</b><br>0.-12n 1.-13n 2.-15n 3.-7n 4.-11n 5.-6n 6.43n<br>7.98n 8.43n 9.-10n 10.-11n 11.-9n 12.-11n 13.-12n<br>14.-9n 15.-10n 16.-12n 17.-8n 18.-9n 19.-13n 22.,<br>Response: -12</td>
  </tr>
  <tr>
    <td colspan="2"><b>User Prompt</b><br>0.-12n 1.-13n 2.-15n 3.-7n 4.-11n 5.-6n 6.43n<br>7.98n 8.43n 9.-10n 10.-11n 11.-9n 12.-11n 13.-12n<br>14.-9n 15.-10n 16.-12n 17.-8n 18.-9n 19.-13n 23.,<br>Response: -8</td>
  </tr>
  <tr>
    <td colspan="2"><b>User Prompt</b><br>0.-12n 1.-13n 2.-15n 3.-7n 4.-11n 5.-6n 6.43n<br>7.98n 8.43n 9.-10n 10.-11n 11.-9n 12.-11n 13.-12n<br>14.-9n 15.-10n 16.-12n 17.-8n 18.-9n 19.-13n 24.,<br>Response: -9</td>
  </tr>
</table>

<table>
  <tr>
    <th colspan="2">(b) LLMP-Independent (Single-turn)</th>
  </tr>
  <tr>
    <th colspan="2">System Prompt</th>
  </tr>
  <tr>
    <td colspan="2">You are a helpful assistant that performs time series predictions. The user will provide you with a sequence of ordered pairs (x, y), and you will predict y for pairs where only x is given. Each pair is separated by a newline.</td>
  </tr>
  <tr>
    <td colspan="2"><b>User Prompt</b><br>Please predict the missing values in the y column based on the given x and y data points without producing any additional text.<br>Do not say anything like 'the next terms in the sequence are', just return only the y values as numbers without x values.<br>y0: -12n 1.-13n 2.-15n 3.-7n 4.-11n 5.-6n 6.43n<br>7.98n 8.43n 9.-10n 10.-11n 11.-9n 12.-11n 13.-12n<br>14.-9n 15.-10n 16.-12n 17.-8n 18.-9n 19.-13n 20.,<br>Response: -9, -10, -12, -8, -9</td>
  </tr>
</table>

Figure 5: (a) is the original prompt proposed in LLMP. The (a) prompt requires independently processing O queries to obtain O output values. This significantly increases the computational cost of using LLMs. To address this issue, we designed a single-turn prompt (b) that enables LLMP to be performed more efficiently.

across all datasets, suggesting that decoding hyperparameters do not fundamentally alter the performance gap.

C.2.2 Evaluating Entire Horizon on Three Real-world Datasets

We adopt the last-sample evaluation protocol introduced by prior work (LLMTime), which evaluates forecasting models only on the final prediction window within each test sequence. This approach substantially reduces the computational and financial costs of repeated LLM inference, particularly when relying on commercial APIs. It is important to clarify that the term last sample does not refer to a single data point, but rather to the final forecasting window for each multivariate time series in the test split. Since the LLM-based forecasters operate independently on each time series, the total number of forecast instances evaluated under this setting is 1,219, as detailed in Appendix Table 3.

To provide a more complete evaluation of temporal generalization, we further conduct experiments using a sliding-window evaluation protocol. In this

(a) TS-CoT

<table>
  <tr>
    <th colspan="2">(a) TS-CoT</th>
  </tr>
  <tr>
    <th colspan="2">System Prompt</th>
  </tr>
  <tr>
    <td colspan="2">You are a helpful assistant that performs time series predictions. The user will provide a sequence and you will predict the remaining sequence. The sequence is represented by decimal strings separated by commas.</td>
  </tr>
  <tr>
    <td colspan="2"><b>User Prompt</b><br>Sequence during the input period: -12, -13, -15, -7, -11, -6, 43, 98, 43, -10, -11, -9, -11, -12, -9, -10, -12, -8, -9, -13,<br>Let's think step by step.</td>
  </tr>
  <tr>
    <td colspan="2"><b>Step 1</b> Describe the solution process to make future predictions that reflect the description in up to five sentences.</td>
  </tr>
  <tr>
    <td colspan="2"><b>Step 2</b> Considering the answers to previous steps, please predict next sequence following input sequence without producing any additional text. Do not say anything like 'the next terms in the sequence are', just return the numbers. Input Sequence:-12, -13, -15, -7, -11, -6, 43, 98, 43, -10, -11, -9, -11, -12, -9, -10, -12, -8, -9, -13,</td>
  </tr>
  <tr>
    <td colspan="2">Response:<br><b>Answer 1</b> The sequence provided appears to be non-linear and may contain multiple patterns or regimes. To predict future data points, it's important to identify any apparent periodicity, seasonality, trends, or cycles within the data. The sequence might show different behaviors at different segments, which can include rising trends, falling trends, and various plateaus or periods of relative stability ...<br><b>Answer 2</b> -9, -10, -12, -8, -9</td>
  </tr>
</table>

(b) TS-InContext (3-Shot)

<table>
  <tr>
    <th colspan="2">(b) TS-InContext (3-Shot)</th>
  </tr>
  <tr>
    <th colspan="2">System Prompt</th>
  </tr>
  <tr>
    <td colspan="2">You are a helpful assistant that performs time series predictions. The user will provide a sequence and you will predict the remaining sequence. The sequence is represented by decimal strings separated by commas.</td>
  </tr>
  <tr>
    <td colspan="2"><b>User Prompt</b><br>We give you input and output sequence samples:<br>1. Sequence: -12, -13, -15, -7, -11, <sep>-6, 43, 98, 43, -10<br>2. Sequence: -6, 43, 98, 43, -10, <sep>-11, -9, -11, -12, -9<br>3. Sequence: -11, -9, -11, -12, -9, <sep>-10, -12, -8, -9, -13</td>
  </tr>
  <tr>
    <td colspan="2">Please predict next sequence following input sequence without producing any additional text. Do not say anything like 'the next terms in the sequence are', just return the numbers. Input Sequence: -10, -12, -8, -9, -13, <sep></td>
  </tr>
  <tr>
    <td colspan="2">Response: -9, -10, -12, -8, -9</td>
  </tr>
</table>

Figure 6: (a) TS-CoT uses a step-by-step reasoning approach (CoT) to guide predictions by analyzing patterns before forecasting. (b) TS-InContext (3-Shot) leverages in-context learning by providing examples of input-output pairs, allowing the model to infer patterns directly from demonstrations.
<table>
  <tr>
    <th rowspan="2">Models</th>
    <th rowspan="2">Top-p</th>
    <th colspan="2">ETTm2</th>
    <th colspan="2">Exchange Rate</th>
    <th colspan="2">Weather</th>
  </tr>
  <tr>
    <th>MAE</th>
    <th>MSE</th>
    <th>MAE</th>
    <th>MSE</th>
    <th>MAE</th>
    <th>MSE</th>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>0.1</td>
    <td>0.2816</td>
    <td>0.2032</td>
    <td>0.3697</td>
    <td>0.2930</td>
    <td>0.1902</td>
    <td>0.1170</td>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>0.5</td>
    <td>0.3153</td>
    <td>0.2424</td>
    <td>0.3913</td>
    <td>0.3078</td>
    <td>0.2019</td>
    <td>0.1235</td>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>0.8</td>
    <td>0.3232</td>
    <td>0.1237</td>
    <td>0.3595</td>
    <td>0.2893</td>
    <td>0.1880</td>
    <td>0.1147</td>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>1.0</td>
    <td>0.2825</td>
    <td>0.1996</td>
    <td>0.3562</td>
    <td>0.2926</td>
    <td>0.1897</td>
    <td>0.1268</td>
  </tr>
  <tr>
    <td>DLinear-S</td>
    <td>–</td>
    <td>0.2039</td>
    <td>0.0782</td>
    <td><b>0.2296</b></td>
    <td><b>0.1166</b></td>
    <td>0.1643</td>
    <td>0.0522</td>
  </tr>
  <tr>
    <td>RLinear-S</td>
    <td>–</td>
    <td><b>0.1824</b></td>
    <td><b>0.0677</b></td>
    <td>0.2571</td>
    <td>0.1370</td>
    <td><b>0.1392</b></td>
    <td><b>0.0469</b></td>
  </tr>
</table>

Table 7: Forecasting performance by top-p values

<table>
  <tr>
    <th rowspan="2">Models</th>
    <th colspan="2">ETTm2</th>
    <th colspan="2">Exchange Rate</th>
    <th colspan="2">Weather</th>
  </tr>
  <tr>
    <th>MAE</th>
    <th>MSE</th>
    <th>MAE</th>
    <th>MSE</th>
    <th>MAE</th>
    <th>MSE</th>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>0.3391</td>
    <td>0.2156</td>
    <td>0.3292</td>
    <td>0.1824</td>
    <td>0.0350</td>
    <td>0.0024</td>
  </tr>
  <tr>
    <td>LLaMA3.1-8B-Inst.</td>
    <td>0.3684</td>
    <td>0.2508</td>
    <td>0.3619</td>
    <td>0.2682</td>
    <td>0.0365</td>
    <td>0.0026</td>
  </tr>
  <tr>
    <td>DLinear-S</td>
    <td>0.1455</td>
    <td>0.0425</td>
    <td><b>0.2728</b></td>
    <td><b>0.1105</b></td>
    <td>0.0242</td>
    <td><b>0.0010</b></td>
  </tr>
  <tr>
    <td>RLinear-S</td>
    <td><b>0.1385</b></td>
    <td><b>0.0411</b></td>
    <td>0.2974</td>
    <td>0.1268</td>
    <td><b>0.0235</b></td>
    <td><b>0.0010</b></td>
  </tr>
</table>

Table 8: Forecasting performance on three real-world datasets over the entire test set range.

setup, forecasts are generated across all time steps in the test split by sliding the input window with a stride equal to the prediction length. For consistency and interpretability, we focus our analysis on the main target variable of each dataset (e.g., the oil temperature variable in ETTm2).

As shown in Table 8, the results from this extended evaluation confirm our earlier findings: while LLM-based forecasters such as GPT-4o and LLaMA3.1-8B-Instruct exhibit competitive short-term performance, linear models (e.g., DLinear-S, RLinear-S) continue to outperform them in terms of robustness and accuracy across the entire test horizon. These observations reinforce the limitations of current zero-shot LLM forecasters and underline the temporal stability offered by lightweight, domain-specific architectures.

C.2.3 Evaluating Robustness Against Frequency-Based Noise

<table>
  <tr>
    <th rowspan="2">Models</th>
    <th colspan="2">Clean</th>
    <th colspan="2">Freq-Replace</th>
    <th colspan="2">Freq-Add</th>
  </tr>
  <tr>
    <th>MAE</th>
    <th>MSE</th>
    <th>MAE</th>
    <th>MSE</th>
    <th>MAE</th>
    <th>MSE</th>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>0.6726</td>
    <td>0.9089</td>
    <td>0.9065</td>
    <td>1.5183</td>
    <td>0.7862</td>
    <td>1.2615</td>
  </tr>
  <tr>
    <td>GPT-4</td>
    <td>0.6956</td>
    <td>0.9533</td>
    <td>0.9171</td>
    <td>1.4361</td>
    <td>0.7693</td>
    <td>1.0061</td>
  </tr>
  <tr>
    <td>DLinear-S</td>
    <td>0.7074</td>
    <td>0.8890</td>
    <td><b>0.7059</b></td>
    <td><b>0.8899</b></td>
    <td><b>0.7081</b></td>
    <td><b>0.8902</b></td>
  </tr>
  <tr>
    <td>RLinear-S</td>
    <td>0.7182</td>
    <td>0.8804</td>
    <td>0.7129</td>
    <td><b>0.8733</b></td>
    <td>0.7183</td>
    <td><b>0.8743</b></td>
  </tr>
</table>

Table 9: Forecasting results of each model on the Monash dataset with two periodic noises.

To further investigate the robustness of forecasting models under structured perturbations, we conduct additional experiments using periodic noise. Two variants of noise are considered: (1) Freq-Add, in which a sinusoidal component is added to the original time series, and (2) Freq-Replace, where the original series is replaced entirely by a sinusoidal signal. These settings are designed to emulate scenarios where time-series data are affected by periodic interference unrelated to the true signal.

Experimental results in Table 9 show that LLM-based models, such as GPT-4 and GPT-4o, experience substantial performance degradation in the presence of such periodic distortions. In contrast, linear baselines such as DLinear-S and RLinear-S remain largely unaffected, demonstrating strong resilience to this form of noise. This observation aligns with previous findings, reinforcing that linear models exhibit superior robustness when exposed to structured, non-informative patterns in the input.

C.2.4 Comparison with Additional Baselines

<table>
  <tr>
    <th rowspan="2">Models</th>
    <th colspan="2">ETTm2</th>
    <th colspan="2">Exchange Rate</th>
    <th colspan="2">Weather</th>
  </tr>
  <tr>
    <th>MAE</th>
    <th>MSE</th>
    <th>MAE</th>
    <th>MSE</th>
    <th>MAE</th>
    <th>MSE</th>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>0.2322</td>
    <td>0.1237</td>
    <td>0.3595</td>
    <td>0.2893</td>
    <td>0.1880</td>
    <td>0.1147</td>
  </tr>
  <tr>
    <td>ARIMA</td>
    <td>0.3402</td>
    <td>0.2201</td>
    <td>0.3460</td>
    <td>0.2535</td>
    <td>0.3104</td>
    <td>0.2722</td>
  </tr>
  <tr>
    <td>N-BEATS</td>
    <td>0.2804</td>
    <td>0.1842</td>
    <td>0.3808</td>
    <td>0.3079</td>
    <td>0.2169</td>
    <td>0.1209</td>
  </tr>
  <tr>
    <td>DLinear-S</td>
    <td>0.2039</td>
    <td>0.0782</td>
    <td><b>0.2296</b></td>
    <td><b>0.1166</b></td>
    <td>0.1643</td>
    <td>0.0522</td>
  </tr>
  <tr>
    <td>RLinear-S</td>
    <td><b>0.1824</b></td>
    <td><b>0.0677</b></td>
    <td>0.2571</td>
    <td>0.1370</td>
    <td><b>0.1392</b></td>
    <td><b>0.0469</b></td>
  </tr>
</table>

Table 10: Forecasting results of ARIMA and N-BEATS on three real-world datasets

Statistical and lightweight forecasting models, such as ARIMA (Box and Jenkins, 1968) and N-BEATS (Oreshkin et al., 2019), are widely recognized for their simplicity and computational efficiency. To ensure a comprehensive evaluation, we included these models in our benchmark and report their forecasting performance on three real-world datasets in Table 10. The results show that, while LLM-based zero-shot models (e.g., GPT-4o) can outperform ARIMA and N-BEATS in certain settings, their performance is still surpassed by simpler linear models such as DLinear-S and RLinear-S trained on a single input sequence. These single-shot linear models not only achieve higher forecasting accuracy but also retain strong computational efficiency. We propose these models as new, competitive baselines for zero-shot forecasting with LLMs, providing a more challenging yet realistic reference point for future research.
C.3 Qualitative Results of Forecasting Models

![Qualitative results on Electricity dataset across four channels (58, 118, 178, and 238). The black line represents ground truth, while LLMTIME (yellow) and RLinear-S (purple) show model predictions with corresponding MSE and MAE values. Overall, RLinear-S tends to outperform LLMTIME.](page_146_120_1192_482.png)

Figure 7: Qualitative results on Electricity dataset across four channels (58, 118, 178, and 238). The black line represents ground truth, while LLMTIME (yellow) and RLinear-S (purple) show model predictions with corresponding MSE and MAE values. Overall, RLinear-S tends to outperform LLMTIME.

![Qualitative results on ETTm2 dataset across four channels (0 and 6)](page_146_670_1192_482.png)

Figure 8: Qualitative results on ETTm2 dataset across four channels (0 and 6)
Figure 9: Qualitative results on ExchangeRate dataset across four channels (0 and 2)

Figure 10: Qualitative results on Weather dataset across four channels (0 and 12)

Figure 11: Qualitative results on Traffic dataset across four channels (58, 118, 178, and 238).
C.4 Full Results

(a) ETTm2 Dataset (7 channels)

(b) Exchange Rate Dataset (8 channels)

(c) Weather Dataset (21 channels)

(d) Electricity Dataset (321 channels)

(e) Traffic Dataset (862 channels)

Figure 12: Multivariate forecasting results on five datasets from the Informer benchmark. We report the normalized MAE and log-scaled inference time for domain-specific and LLM-based forecasters with confidence intervals.

Reviewer paper summaries: This paper evaluates various pretrained LLM for zero-shot time-series forecasting tasks, and reveales that LLM-based forecasters perform not as good as domain-specific finetuned forecasters, including linear models. The authors attribute this gap to LLMs' sensitivity to noise in data, and perform experiments to compare the robustness of LLM forecasters with linear models. The paper also shows that mitigating noise for LLM forecasters remains challenging.
Summary of strengths: 1. This paper seamlessly integrate LLM-based models with time-series data for forecasting tasks, using novel methods such as TS-COT and TS-InContext prompting. 

2. The authors conduct a lot of experiments to reveal the effect of noise in LLM-based TS forecasting, which is under-explored in literature. Attributing the performance gap to LLMs' noise-sensitive nature is also a novel perspective in existing evaluations.
Summary of weaknesses: 1. Poor writing: the presentation of the motivation and idea could be greatly improved by writing. For example, In Section 1 (line 067), why LLM forecasters should meet at least one of two key criteria should be discussed in the first place. Additionally, there seems to be "only" before "if" in this paragraph, otherwise the meanings could be vague. 

2. Limited literature review: there are existing LLM-based zero-shot forecasters with comparable performance to finetuned models, such as [1], which should be discussed and compared. 

3. Defective experimental design: there are a few concerns in their experiments. (1) For the prompting strategy, this paper does not experiment with different numbers of few-shot examples, such as 1-shot and 5-shot, which are widely used in in-context learning in related work [2]. (2) the paper uses multiple sampling to alleviate the randomness due to temperature, but does not discuss the effect of different temperature in sampling, as well as the effect of decoding algorithm, which are critical in evaluating LLMs' performance. (3) The comparison between LLM forecasters and linear models seems not fair. The authors use only last example in TS split to evaluate linear models, which could be biased due to overfitting on single data sample. To conduct more convincing evaluation, the authors could test on more data samples. Note that, linear models are intuitively good at narrow slide window due to the nature of first order approximation, which are not trivial for a wider slide window. (4) The noise injected to data could be more sophisticated. Other than noises used in regression tasks, such as Gaussian and constant noise (which are intuitively denoised by linear models), the authors could compare LLM and linear forecasters on nonlinear noise, such as periodic noise. 


[1] Ansari, Abdul Fatir, et al. "Chronos: Learning the language of time series." (2024).

[2] Achiam, Josh, et al. "Gpt-4 technical report." arXiv preprint arXiv:2303.08774 (2023).
Comments, suggestions and typos: This paper could be improved by addressing the concerns mentioned in "Weakness", and I am willing to raise my rating accordingly.
Overall assessment scores: 4