# How Foundational are Foundation Models for Time Series Forecasting?

Nouha Karaouli, Denis Coquenet, Elisa Fromont, Martial Mermillod, Marina Reyboz

► To cite this version:

Nouha Karaouli, Denis Coquenet, Elisa Fromont, Martial Mermillod, Marina Reyboz. How Founda-
tional are Foundation Models for Time Series Forecasting?. BERT2 S 2025 - NeurIPS Workshop on
Recent Advances in Time Series Foundation Models, Dec 2025, San Diego, CA, United States. pp.1-8.
hal-05300066

HAL Id: hal-05300066

https://hal.science/hal-05300066v1

Submitted on 6 Oct 2025

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# How Foundational are Foundation Models for Time Series Forecasting?

Nouha Karaouli¹, Denis Coquenet², Elisa Fromont¹, Martial Mermillod³, and Marina Reyboz⁴

¹Univ. Rennes, CNRS, Inria, IRISA - UMR 6074, F-35000 Rennes, France

²Univ. Rennes, CNRS, IRISA - UMR 6074, F-35000 Rennes, France

³Univ. Grenoble Alpes, Univ. Savoie Mont Blanc, CNRS, LPNC, Grenoble, France

⁴Univ. Grenoble Alpes, CEA, LIST, 38000 Grenoble, France

## Abstract

Foundation Models are designed to serve as versatile embedding machines, with strong zero shot capabilities and superior generalization performance when fine-tuned on diverse downstream tasks. While this is largely true for language and vision foundation models, we argue that the inherent diversity of time series data makes them less suited for building effective foundation models. We demonstrate this using forecasting as our downstream task. We show that the zero-shot capabilities of a time series foundation model are significantly influenced and tied to the specific domains it has been pretrained on. Furthermore, when applied to unseen real-world time series data, fine-tuned foundation models do not consistently yield substantially better results, relative to their increased parameter count and memory footprint, than smaller, dedicated models tailored to the specific forecasting task at hand.

## 1 Introduction

The emergence of Foundation Models (FMs), large-scale pretrained architectures such as BERT [1] in Natural Language Processing (NLP) and Vision Transformers [2] in Computer Vision (CV), has fundamentally transformed artificial intelligence. By leveraging massive and diverse datasets during pretraining, these models exhibit strong generalization abilities, enabling zero-shot and few-shot transfer to a wide range of downstream tasks [3, 4]. This shift has allowed FMs to consistently outperform traditional task-specific models trained from scratch for narrowly defined problems [4].

Inspired by these successes, researchers have recently proposed Time Series Foundation Models (TSFMs), large pretrained models designed to capture general-purpose representations across diverse temporal data [5]. These models aim to transfer knowledge across forecasting tasks by learning temporal patterns at scale, showing promising results in a variety of domains with minimal task-specific tuning.

However, the time series domain poses unique challenges that set it apart from NLP and CV. Time series data often exhibits domain-specific structures such as seasonality, trends, irregular sampling, and high variability across applications, even within the same broad category [6]. Such characteristics introduce distribution shifts that undermine the generalization abilities of TSFMs [7]. In particular, our experiments suggest that TSFMs’ zero-shot performance is highly sensitive to the alignment between the statistical properties of the pretraining and target domains. When this alignment is weak, we observe substantial drops in generalization, even across domains that might appear related.
While TSFMs often benefit from rapid initial convergence, extended fine-tuning can lead to performance degradation, whereas task-specific models trained from scratch typically yield steady accuracy gains under longer training and limited data regimes [8].

Motivated by these challenges, we conduct a thorough empirical evaluation of the univariate forecasting capabilities of TSFMs across diverse tasks. We compare them with traditional models trained from scratch to assess whether TSFMs offer practical advantages when fine-tuned on specific, potentially domain-shifted datasets.

Our main contributions are:

* Evaluating TSFMs in zero-shot mode across both domain-related and domain-shifted forecasting datasets.

* Comparing the fine-tuning capabilities of TSFMs versus traditional models on forecasting tasks to evaluate their adaptability and effectiveness under domain shift and limited data.

* Proposing a new forecasting dataset consisting of daily electricity usage over two years, on which a small dedicated network achieves better results than a fine-tuned TSFM.

## 2 Related Work

Recent TSFMs such as TiReX [9], TimeGPT [10], TimesFM [11], and FEDformer [12] leverage large-scale pretraining to enable strong generalization and transfer across forecasting tasks.

To assess their practical utility, several benchmarking frameworks have emerged. GIFT-eval [13] measures cross-domain generalization using standardized protocols, OpenTS [14] offers a reproducible suite spanning datasets, horizons, and metrics, while Nixtla's Arena [15] provides a comprehensive evaluation across frequencies and domains. [16] have also pointed out that naive baselines (here, a simple auto-regressive model) can achieve competitive performance compared to TSFM on several forecasting tasks.

These efforts report promising performance on public datasets such as Monash [17] and ETT [18]. However, we had to compare the generalization performance of these foundation models on time series ensured to be completely new and not included in these benchmark databases in order to test the challenges faced in deployment.

In contrast, we evaluate TSFMs on a proprietary electricity consumption dataset with realistic and complex domain shifts not seen during pretraining. Our setup introduces explicit distributional changes, enabling a more rigorous assessment of generalization.

Contrary to standard benchmarks that primarily focus on evaluating zero-shot capabilities of TSFMs on public datasets, we further compare these models to conventional ones trained from scratch. This allows us to highlight scenarios where smaller, specialized models achieve comparable performance to large pretrained TSFMs, especially under conditions of data scarcity and nonstationarity.

Through this, we uncover limitations in TSFMs' robustness and provide new insights into their practical effectiveness in real-world forecasting scenarios.

## 3 Methodology

Our evaluation addresses two central questions: (1) Can TSFMs generalize beyond their pretraining distributions? (2) Are they practically competitive with lightweight, specialized alternatives?

We benchmark three leading TSFMs, namely TimesFM [11], TimeGPT [10], and TiReX [9], alongside SAMFormer [19], a compact attention-based model operating over the channel dimension. Unlike the other models, SAMFormer is trained from scratch in our experiments.

**Synthetic benchmarks.** We construct four datasets that reflect recurring structures in TSFM pretraining, while ensuring zero data overlap.

* D1 and D2 are composed of harmonically aligned sine waves with full observability, probing the models' ability to recognize and extrapolate clean periodic signals.
* D3 and D4 consist of randomly sampled, non-harmonic sine waves, forming complex, partially observable cycles. These challenge the models to generalize from incomplete patterns.

All synthetic sequences contain 2,688 time steps (8 weeks sampled at 30-minute intervals).

**Real-world evaluation.** We test TSFMs on *Elec_Consumption*, a proprietary small dataset capturing daily electricity usage of a single home over two years (2023–2024). Unlike the generic, population-level datasets typically used during TSFM pretraining, this dataset reflects individual consumption behavior, introducing a clear distribution shift. This setting allows us to rigorously evaluate whether pretrained models retain strong forecasting performance when faced with personalized, unseen patterns, a crucial requirement for real-world deployment in user-specific applications.

All datasets used in this work are made publicly available for reproducibility<fnref n="1" />.

**Fine-tuning experiments.** We fine-tune TimesFM on *Elec_Consumption* and compare it to SAM-Former trained from scratch. This setup quantifies the trade-off between the computational overhead of fine-tuning large pretrained models and the efficiency of smaller models tailored to specific domains.

Together, these evaluations dissect the *one-size-fits-all* [20, 21] promise of TSFMs, distinguishing their theoretical representational capacity (via synthetic benchmarks) from their practical effectiveness in real-world deployment. We report Mean Absolute Error (MAE) as the primary metric.

# 4 Results

We begin our experimental evaluation by testing all models in zero-shot mode on both synthetic and real-world datasets, Appendix A provides additional details on our models querying setup. Tables 1 and 2 report results on synthetic data using a fixed context length of 512 across three forecast horizons. Table 3 presents results on the *Elec_Consumption* dataset.

<table><caption>Table 1: Zero-shot MAE on D1 and D2 for various forecasting horizons and models. Lower is better.</caption><thead><tr><th rowspan="2">Datasets</th><th rowspan="2">Models</th><th colspan="3">D1</th><th colspan="3">D2</th></tr><tr><th>TimeGPT</th><th>TiReX</th><th>TimesFM</th><th>TimeGPT</th><th>TiReX</th><th>TimesFM</th></tr></thead><tbody><tr><td rowspan="3">Horizons</td><td><strong>128</strong></td><td>0.89</td><td><strong>0.11</strong></td><td>0.13</td><td>0.80</td><td>0.29</td><td><strong>0.15</strong></td></tr><tr><td><strong>256</strong></td><td>1.08</td><td><strong>0.21</strong></td><td>0.22</td><td>1.25</td><td>0.72</td><td><strong>0.35</strong></td></tr><tr><td><strong>512</strong></td><td>1.09</td><td>0.37</td><td><strong>0.34</strong></td><td>1.57</td><td>1.11</td><td><strong>0.72</strong></td></tr></tbody></table>

<table><caption>Table 2: Zero-shot MAE on D3 and D4 for various forecasting horizons and models. Lower is better.</caption><thead><tr><th rowspan="2">Datasets</th><th rowspan="2">Models</th><th colspan="3">D3</th><th colspan="3">D4</th></tr><tr><th>TimeGPT</th><th>TiReX</th><th>TimesFM</th><th>TimeGPT</th><th>TiReX</th><th>TimesFM</th></tr></thead><tbody><tr><td rowspan="3">Horizons</td><td><strong>128</strong></td><td>1.86</td><td><strong>1.1</strong></td><td>1.13</td><td>1.3</td><td><strong>0.78</strong></td><td>0.89</td></tr><tr><td><strong>256</strong></td><td>1.43</td><td><strong>0.95</strong></td><td>0.98</td><td>1.63</td><td><strong>1.6</strong></td><td>1.62</td></tr><tr><td><strong>512</strong></td><td><strong>2.29</strong></td><td>3.3</td><td>3.5</td><td><strong>2.31</strong></td><td>2.8</td><td>2.98</td></tr></tbody></table>

Among all five experiments, TiReX and TimesFM consistently perform best, particularly on D1 and D2, which exhibit simple and periodic sinusoidal patterns, highlighting their ability to capture repetitive temporal structures. In contrast, forecasting on D3 and D4, involving irregular and composite sinusoidal signals, is more challenging. Despite this, foundation models still generalize reasonably well, likely due to pretraining on structurally similar synthetic patterns. However, on the real-world *Elec_consumption* dataset, even with careful tuning of context and horizon lengths, the models struggle to accurately forecast the future values. This shows the limits of the generalization abilities of current state-of-the-art TSFMs for a real-case forecasting scenario.

<footnote n="1">¹https://doi.org/10.5281/zenodo.17199246</footnote>
<table><caption>Table 3: Zero-shot MAE on *Elec_Consumption* for varying context-horizon pairs and models. Lower is better.</caption><thead><tr><th rowspan="2">Models</th><th colspan="5">Context - Horizon</th></tr><tr><th>15-7</th><th>30-7</th><th>60-30</th><th>128-128</th><th>365-365</th></tr></thead><tbody><tr><td>TimeGPT</td><td>6.60</td><td>6.52</td><td>5.60</td><td>6.91</td><td>6.44</td></tr><tr><td>TiReX</td><td>6.94</td><td><strong>5.71</strong></td><td>4.61</td><td><strong>3.78</strong></td><td>5.90</td></tr><tr><td>TimesFM</td><td><strong>5.07</strong></td><td>5.83</td><td><strong>4.08</strong></td><td>4.63</td><td><strong>5.30</strong></td></tr></tbody></table>

This performance contrast is clearly illustrated in Figure 1. On dataset D1, the TSFMs demonstrate strong generalization, achieving MAE scores of 0.95, 0.46, and 0.71 on TimesGPT, TimesFM, and TiReX, respectively. In contrast, on Elec_consumption, the forecasts deviate more noticeably from the expected values.

<figure><img src="image_2.png" /><figcaption>Figure 1: Forecasting results using different models. Panel (a) shows results on D1, while panel (b) presents results on Elec_Consumption.</figcaption></figure>

While zero-shot results show that TSFMs perform well on target distributions that resemble their pretraining data, their ability to adapt to small, domain-specific datasets produces high errors and low prediction ability. To investigate this, we compare fine-tuned TimesFM with SAMFormer trained from scratch on our Elec_consumption dataset. This evaluation tests whether TSFMs' learned representations and inductive biases confer advantages for personalization.
Fine-tuning and training from scratch were performed using Adam with a learning rate $10^{-4}$, weight decay 0.01, and batch size 64. The choice of LR follows the default fine-tuning configuration used in the public TimesFM examples, ensuring consistency with recommended practice for this foundation model. Data were standardized and framed with a sliding window (context = 128, horizon = 128). TimesFM was fine-tuned from a fixed pre-trained checkpoint, excluding any significant source of randomness. In contrast, SAMFormer was trained from scratch, introducing natural variability in the results due to the random weight initialization. To make the evaluation robust, we computed the mean and standard deviation over 5 runs with different random seeds. Models were trained for up to 100 epochs with early stopping (patience = 10). Experiments were conducted on an NVIDIA Tesla V100 GPU. Results are shown in Table 4.

<table><caption>Table 4: MAE for TimesFM and SAMFormer with a context window of 128 and a forecast horizon of 128.</caption><thead><tr><td>Models</td><td>MAE</td></tr></thead><tbody><tr><td>TimesFM</td><td>4.49 ± 0.00</td></tr><tr><td>SAMFormer</td><td>4.28 ± 0.05</td></tr></tbody></table>

As one can note, the results show that SAMFormer, trained entirely from scratch with fewer than 50K parameters, ultimately outperforms TimesFM on the target forecasting task. While TimesFM benefits from large-scale pretraining and contains over 500 million parameters, SAMFormer achieves superior accuracy while remaining extremely lightweight and efficient to train on consumer-grade GPUs. This contrast highlights a key point: massive pretrained models do not always guarantee superior downstream performance, particularly in settings where data distributions differ from the pretraining corpus or where the target domain exhibits specific structural regularities that a smaller model can exploit more effectively. Moreover, SAMFormer’s compact size reduces both training time and inference cost, making it well-suited for rapid experimentation and deployment in resource-constrained environments. These findings illustrate that carefully designed, domain-adapted models can deliver competitive or even superior performance compared to large foundation models, while offering substantial advantages in efficiency, accessibility, and environmental sustainability.

# 5 Conclusion

While TSFMs show strong zero-shot performance on synthetic and structurally similar data, their generalization ability is tightly coupled with the distribution seen during pretraining. In real-world settings involving domain shifts and limited data, a lightweight model like SAMFormer, with only 49.5K parameters and no large-scale pretraining, can still achieve better results when trained from scratch. This suggests that the “one-size-fits-all” promise of TSFMs may not hold in practice, especially under resource constraints or personalization requirements. Our findings advocate for a more nuanced deployment strategy: leveraging TSFMs when pretraining-task similarity is high, and favoring lightweight, specialized models when personalization, efficiency, or domain mismatch is critical.

# References

[1] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In *Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)*, pages 4171–4186. Association for Computational Linguistics, 2019.

[2] Alexey Dosovitskiy, Lucas Beyer, and Alexander Kolesnikov et al. An image is worth 16x16 words: Transformers for image recognition at scale. *arXiv preprint arXiv:2010.11929*, 2020.

[3] Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, and Jared Kaplan et al. Language models are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, editors, *Advances in Neural Information Processing Systems*, pages 1877–1901. Curran Associates, Inc., 2020.
[4] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In *Proceedings of the 38th International Conference on Machine Learning*, volume 139 of PMLR, pages 8748–8763, San Francisco, CA, USA, 2021. PMLR. Equal contribution. OpenAI.

[5] Y. Liang et al. Foundation models for time series analysis: A tutorial and survey. In *Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining*, pages 6555–6565, Barcelona, Spain, August 2024. ACM.

[6] Jiexia Ye, Weiqi Zhang, Ke Yi, Yongzi Yu, Ziyue Li, Jia Li, and Fugee Tsung. A survey of time series foundation models: Generalizing time series representation with large language model. *arXiv preprint arXiv:2405.02358*, 2024.

[7] Zhe Li, Xiangfei Qiu, Peng Chen, Yihang Wang, Hanyin Cheng, Yang Shu, Jilin Hu, Chenjuan Guo, Aoying Zhou, Qingsong Wen, et al. Foundts: Comprehensive and unified benchmarking of foundation models for time series forecasting. 2024.

[8] Lifan Zhao, Yanyan Shen, Zhaoyang Liu, Xue Wang, and Jiaji Deng. Less is more: Unlocking specialization of time series foundation models via structured pruning, 2025.

[9] Andreas Auer, Patrick Podest, Daniel Klotz, Sebastian Böck, Günter Klambauer, and Sepp Hochreiter. Tirex: Zero-shot forecasting across long and short horizons with enhanced in-context learning. *arXiv preprint arXiv:2505.23719*, 2025.

[10] Azul Garza, Cristian Challu, and Max Mergenthaler-Canseco. Timegpt-1. *arXiv preprint arXiv:2310.03589*, 2024. Version 3, 27 May 2024.

[11] Abhimanyu Das, Weihao Kong, Rajat Sen, and Yichen Zhou. A decoder-only foundation model for time-series forecasting. In *Proceedings of the 41st International Conference on Machine Learning (ICML)*, volume 235 of *Proceedings of Machine Learning Research*, pages 4599–4623. PMLR, 2024.

[12] Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting. In *Proceedings of the 39th International Conference on Machine Learning*, pages 27268–27286, 2022.

[13] Taha Aksu et al. GIFT-Eval: A Benchmark for General Time Series Forecasting Model Evaluation. arXiv, November 2024.

[14] DecisionIntelligence. OpenTS – A Comprehensive and Fair Benchmark for Time Series Analytics. https://decisionintelligence.github.io/OpenTS/algorithms/methods/index.html, October 2024. Accessed 25 October 2024.

[15] Nixtla. Foundation time series arena – benchmarking foundation models for time series. https://github.com/nixtla/experiments/tree/main/foundation-time-series-arena, 2024. GitHub repository.

[16] Zongzhe Xu, Ritvik Gupta, Wenduo Cheng, Alexander Shen, Junhong Shen, Ameet Talwalkar, and Mikhail Khodak. Specialized foundation models struggle to beat supervised baselines. In *The Thirteenth International Conference on Learning Representations*, 2025.

[17] Rakshitha Godahewa, Christoph Bergmeir, Geoffrey I Webb, Rob J Hyndman, and Pablo Montero-Manso. Monash time series forecasting archive. *arXiv preprint arXiv:2105.06643*, 2021.

[18] Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In *Proceedings of the AAAI Conference on Artificial Intelligence*, volume 35, pages 11106–11115. AAAI Press, 2021.

[19] Romain Ilbert, Ambroise Odonnat, Vasilii Feofanov, Aladin Virmaux, Giuseppe Paolo, Themis Palpanas, and Ievgen Redko. Samformer: Unlocking the potential of transformers in time series forecasting with sharpness-aware minimization and channel-wise attention. *arXiv preprint arXiv:2402.10198*, 2024.
[20] Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models. *arXiv preprint arXiv:2108.07258*, 2021.

[21] Peiwen Yuan, Yueqi Zhang, Shaoxiong Feng, Yiwei Li, Xinglin Wang, Jiayi Shi, Chuyi Tan, Boyuan Pan, Yao Hu, and Kan Li. Beyond one-size-fits-all: Tailored benchmarks for efficient evaluation. *arXiv preprint arXiv:2502.13576*, 2025.