Title: The Language of Time: a Language Model Perspective on Time Series Foundation Models
Primary area: learning on time series and dynamical systems
Keywords: Time Series Foundation Models, Patch-based Representation Learning, Distributional Motif Tokens

Abstract: Large language models have established a successful paradigm of training foundation models on massive datasets, extending this approach to multiple domains. Time series foundation models extend this paradigm, demonstrating exceptional cross-domain transfer and prediction capabilities in both industrial and academic scenarios. This creates a paradox: while time series from different domains reflect distinct dynamical systems that should limit transferability, empirical results demonstrate remarkable cross-domain performance.
To resolve this paradox, this paper investigates the representation learning mechanisms and generalization capabilities of patch-based time series foundation models from both theoretical and experimental perspectives. We demonstrate that these models fundamentally extend language model representations from deterministic vectors to probabilistic distributions, enabling effective cross-domain transfer. Our analysis shows that time series patches can be quantized into discrete vocabularies with statistical properties similar to natural language.
This theoretical framework explains how time series models inherit the robust representation and transfer abilities of large language models, accounting for their superior performance in temporal tasks. Our work provides a rigorous theoretical foundation for understanding, evaluating, and improving the safety and reliability of large-scale time series foundation models for time series analysis.

TL;DR: This paper proposes a framework named 'The Language of Time,' demonstrating that  patch-based time series foundation models are essentially a generalization of the distributional tokens language models.