This Time is Different: An Observability Perspective on Time Series
  Foundation Models
Authors: name: Ben Cohen, name: Emaad Khwaja, name: Youssef Doubli, name: Salahidine Lemaachi, name: Chris Lettieri, name: Charles Masson, name: Hugo Miccinilli, name: Elise Ramé, name: Qiqi Ren, name: Afshin Rostamizadeh, name: Jean Ogier du Terrail, name: Anna-Monica Toon, name: Kan Wang, name: Stephan Xie, name: David Asker, name: Ameet Talwalkar, name: Othmane Abou-Amal
Organization: Datadog

We introduce Toto, a time series forecasting foundation model with 151
million parameters. Toto uses a modern decoder-only architecture coupled with
architectural innovations designed to account for specific challenges found in
multivariate observability time series data. Toto's pre-training corpus is a
mixture of observability data, open datasets, and synthetic data, and is
4-10times larger than those of leading time series foundation models.
Additionally, we introduce BOOM, a large-scale benchmark consisting of 350
million observations across 2,807 real-world time series. For both Toto and
BOOM, we source observability data exclusively from Datadog's own telemetry and
internal observability metrics. Extensive evaluations demonstrate that Toto
achieves state-of-the-art performance on both BOOM and on established general
purpose time series forecasting benchmarks. Toto's model weights, inference
code, and evaluation scripts, as well as BOOM's data and evaluation code, are
all available as open source under the Apache 2.0 License available at
https://huggingface.co/Datadog/Toto-Open-Base-1.0 and
https://github.com/DataDog/toto.