This Time is Different: An Observability Perspective on Time Series Foundation Models
Authors: Ben Cohen
Emaad Khwaja
Youssef Doubli
Salahidine Lemaachi
Chris Lettieri
Charles Masson
Hugo Miccinilli
Elise Ramé
Qiqi Ren
Afshin Rostamizadeh
Jean Ogier du Terrail
Anna-Monica Toon
Kan Wang
Stephan Xie
Zongzhe Xu
Viktoriya Zhukova
David Asker
Ameet Talwalkar
Othmane Abou-Amal
Venue: NeurIPS 2025 poster ()
Keywords: time series
foundation models
observability
benchmark dataset
Primary area: deep_learning
TL;DR: We release a new open-weights time series foundation model (TSFM) with SOTA performance on all evaluated benchmarks, as well as a new public observability benchmark, which contains 350 million observations across 2,807 real-world productions series.

Abstract: We introduce Toto, a time series forecasting foundation model with 151 million parameters. Toto uses a modern decoder-only architecture coupled with architectural innovations designed to account for specific challenges found in multivariate observability time series data. Toto's pre-training corpus is a mixture of observability data, open datasets, and synthetic data, and is 4-10$\times$ larger than those of leading time series foundation models. Additionally, we introduce BOOM, a large-scale  benchmark consisting of 350 million observations across 2,807 real-world time series.  For both Toto and BOOM, we source observability data exclusively from our own telemetry and internal observability metrics. Extensive evaluations demonstrate that Toto achieves state-of-the-art performance on both BOOM and on established general  purpose time series forecasting benchmarks. Toto's model weights, inference code, and evaluation scripts, as well as BOOM's data and evaluation code, are all available as open source under the Apache 2.0 License.

This Time is Different: An Observability Perspective on Time Series Foundation Models
Ben Cohen* Emaad Khwaja*!
Youssef Doubli' Salahidine Lemaachi Chris Lettieri't Charles Masson
Hugo Miccinillit Elise Ramét Qiqi Ren t Afshin Rostamizadeh Jean Ogier du Terrail Anna-Monica Toon : Kan Wang Stephan Xie Zongzhe Xul Viktoriya Zhukova 14 David Asker Ameet Talwalkart Othmane Abou-Amal {ben. cohen, emaad, ameet, othmane Jlamaangg@dattadohq..ccmmm
Abstract
We introduce TOTO, a time series forecasting foundation model with 151 mil- lion parameters. TOTO uses modern decoder-only architecture coupled with architectural innovations designed to account for specific challenges found multivariate observability time series data. TOTO's pre-training corpus is a mix- ture of observability data, open datasets, and synthetic data, and is 4-10x larger than those of leading time series foundation models. Additionally, we introduce BOOM, large-scale benchmark consisting of 350 million observations across 2,807 real-world time series. For both ToTO and BOOM, we source observability data exclusively from Datadog's own telemetry and internal observability metrics. Extensive evaluations demonstrate that TOTO achieves state-of-the-art perfor- mance on both BOOM and on established general purpose time series forecasting benchmarks. ToTo's model weights [https: htt///uuginggg ingface co/Datadog/ innfeaaeeee......nneeeennnneee
inference code. and evaluation scripts (https://wwx lattps://github com wwtpa///.ccoo/aaaa/o//oootoooooo as well as BOOM's data and evaluation code (https: / www https::/p///... http://:///ggggggaaaeeeee co/ are available as open source under the Apache 2.0 License.
Introduction
Observability is the practice of collecting and analyzing data generated by distributed computer systems to detect, diagnose, and swiftly resolve performance and reliability issues [4]. major component of observability is monitoring time series metrics; observability tools generate massive and diverse sets of metrics that reflect system' operational health over time. These metrics encom- pass wide variety of indicators suchh as memory usage, CPU load, disk I/O, network throughput, hit counts, error rates, and latency-that each exhibit distinct behavioral patterns, and collectively represent important but under-studied subset of general time series data.
Accurately modeling observability metrics is essential for critical tasks like anomaly detection 5 (e.g., identifying spikes in error rates) and predictive forecasting [6] (e.g., anticipating resource exhaustion scaling needs). Observability data present challenges for traditional forecasting meth- ods due to diversity, high-dimensionality. and complex distributional characteristics (Section 4.3) Moreover, real-world observability systems routinely generate millions billions of distinct time
"Joint First Author, listed alphabetically
*Core Contributor, listed alphabetically
*Datadog Al Research
Carnegic Mellon University, AT and SX contributed to this work their Datadog capacities.
39th Conference on Neural Information Processing Systems (NeurIPS 2025).

---

+ Toto
Zero-Shot Forecasts
Pretraining Data
Model Architecture
OBSERVABLITY
CAUSAL NORMALIZATION
INRRASTRUCTURE METRICS
APPLICATION ION METRICS
TRANSF ORMER DECODER (Proportional Factorized Attention)
OTHER DATASETS
PROBABLISTIC OUTPUT [Student:
HYBRID LOSS
ONL Robust
BOOM Results (Rank
GIFT-Eval Results (Rank 4)
Chronos
Principal Comporent
No1ra1
4
Toto
2
GIPT-EvaL)
Component
Figure 1: @ TOTO is zero-shot time series forecasting model trained on a mixture observability data, open datasets, and synthetic data. To predict, context time series points are passed through patch embedding, processed via proportional factorized variate-ting time attention layers, and projected learned Student-T Mixture model. We sample from this distribution produce prediction forecast. Note that ToTO's novel architectural components are highlighted in purple. i 2D PCA projections of statistical features (described in Section 4.3) of GIFT-Eval [ LSF 22, and BOOM highlight a clear distinction in the underlying time series characteristies of BOOM relative to general-purpose time series benchmarks. C ( Toto is the top performing model BOOM, the GIFT-Eval public leaderhoard and LSS (sce Table
probabilistic output via
series 17/9), rendering fine-tuning or supervised training of complex models per time series infeasi- ble. These operational challenges suggest a compelling use for zero-shot time series foundation models (FMs). However, we find that existing FMs [10 12 trained for ceneral-purpose forecasting struggle to generalize observability data; see Fig. [C and Section [
In this work, we focus on the unique challenges of modeling observability data, while accounting for the constraints of production settings. Our main contributions
follows:
1. ToTO (Time Series Optimized Transformer for Observability). We develop novel open- weights time series forecasting foundation model, with a focus on zero-shot capabilities. ToTO modern decoder-only architecture coupled with architectural innovations to account for the specific challenges found in observability time series data: a novel per-variate patch- based causal scaling address highly nonstationary sequences; proportional time-variate factor- ized attention to judiciously attend across large number of covariates; and a Student-T mixture prediction head optimized via robust composite loss fit complex and highly skewed distribu- tions. TOTO's pretraining corpus contains 4- 10x more unique data points than those other time series FMs 11144) using mix of domain-specifee observability time series data, multi-domain public datasets, and synthetic data.
BOOM (Benchmark of Observability Metrics). We introduce an open-source benchmark specif- ically for observability time series. BOOM includes large-scale, novel dataset with 350 million observations across 2,807 distinct multivariate time series, approximately twice the size of the general- purpose GIFT-Eval benchmark (3). Unlike existing multi-domain benchmarks, BOOM is comprised entirely of woal-world observability data. It captures diverse time series dynamics and challenging behaviors across several subdomains (Fig. 4 thus providing uniquely realistic and comprehensive evaluation environment for forecasting models.

---

In our evaluations against leading foundation models and traditional time series forecasting base- lines, ToTo achieves state-of-the-art performance on both general-purpost and observability- oriented time series forecasting benchmarks. On BOOM, TOTO achieves 12% improvement in terms of CRPS compared to the next best method (see Section [5). TOTO also achieves the top position by a wide margin on two standard general-purpose time series benchmarks-Gile GIFT-Eval and Long Sequence Forecasting (LSF) implying our observability-focusee design also pays divi- dends in other time series domains i ISI We additionally perform ablations to motivate TOTO's architecture design (Fig. 3B), and conduct data analyses BOOM to illustrate the unique aspects of observability data that pose significant modeling challenges (Section
For both ToTo and BOOM, we source observability data exclusively from our own telemetry and internal observability metrics. We provide Toro's 's model weights, inference code, and evaluation scripts, as well as BOOM evaluation code and data, under permissive (Apache 2.0) license.
Related Work
Supervised models. Current observability systems typically rely classical modeling approaches such as Wolt-Winters [5). tree-based methods [16). or (S)ARIMA [17] for forecasting and anomaly detection [8] These approaches require individual training for each dataset, impeding scalability in large real-world systems. While neural models have been shown to surpass these classical models in some settings [99] they are generally larger, more complex, and still require training for each datset; they are thus operationally infeasible in practice.
Time series foundation models. By pre-training large multi-domain datasets, several time se- ries foundation models [1+4 25-25) have achieved impressive zero-shot prediction capabilities on general purpose time series benchmarks, eliminating the need for domain-specific training or finetuning. This approach is especially promising for observability workloads, as a single model can be deployed and horizontally scaled to provide low-latency and relatively low-cost zero-shot inference. Our evaluations indicate that ToTo outperforms existing time series foundation models by wide margin both public forecasting benchmarks and on BOOM (Section
Time series benchmarks. Traditional benchmarks include Monash [26]. LSF (2, M3
[2 : and
M4 (28). These benchmarks are either commonly used for pretraining FMs (Monash) [1] 13 or
are limited in capacity measuring the impact of modern methods (LSF, M3, M4) 229 Aksu
et al. D. Qiu et al. [30], and Ansari et al. [1] all recently introduced multi-domain benchmarks that are better suited in scale and complexity to evaluate general-purpen
time series foundation
models. GIFT-Eval [ in particular is orders of magnitude larger than LSF (see Tarl details); introduces standardized evaluation protocols to facilitate fair comparisons across models; and is unique in containing large decontaminated pretraining dataset for rigorous measurement of zero- shot capabilities without test data leakage.
Our evaluations show that ToTO achieves state-the-an art- performance on LSF and GIFT-Eval. More- over, BOOM adopts the evaluation protocols introduced in GIFT-Eval, though it is unique in its scale (it has approximately twice as many time series points as GIFT-Eval) and its domain of focus (it is derived entirely from real-world observability data).
Challenges of observability data. Several works have explored observability time series data and the prospect of applying the foundation model paradigm to it. Joosen al. [3]] analyze serverless function logs from Huawei Cloud, highlighting challenges such as values spanning nine orders of magnitude, heavy-tailed distributions, and multi-scale seasonality. Toner et al. [2] find that existing time series foundation models perform poorly on small dataset of cloud data. Woo et al. B3 pretrain models on CPU and memory metries, while Palaskar et 34 propose an observability- specific architecture trained in a full-shot setting on four small-scale observability datasets. Both BOOM and TOTO's pretraining corpus are more diverse and most cases orders of magnitude larger than the datasets considered in prior studies. TOTO is also the first abservability-specille foundation model that also performs competitively on broad benchmarks like GIFT-Eval and LSF
3

---

X
Next-patch predictions
twbeniding O/P 0)
SMILU
AMSNorm
Patch peg.
Variate block
(Px0)
Variate
Patchez (4 (IXL)P-P) P))
Pi
RNSNorm
Transformer Decader
Stack xF
Muttiartonee time series (Mt
SwiCLU
Manatts
Patch Embedking
...
AMSNorm
Push
Cauaal instance Norm
Time-wisn
steps
Time Nilk RePC
AMSNorm
Inputs
Figure 2: Overview of the TOTO architecture, highlighting our novel components in bold. A Multivariate input time series of steps scaled using causal patch-based instance normalization, transformed into patch emheddings, and passed through decnder-only transformer stack. The transformed features unemhedded passed through a Student-T mixture model (optimized via composite robust loss) which generates probabilistic next-patch predictions. ① The patch emhedding takes as input a time series of M variates by time steps. It divides the time dimension into patches of size P' = 64 and projects these linearly into an 768. This results in output of size M x transformer decoder. C The transformer stack features proportional factorized attention. It contains identical segment(s), with N 11 time-wis wise transformer blocks followed by one variate-win wise block.
embedding space of latent dimension
D which is fed
3
TOTO
We next outline key architectural components of ToTo, and summarize its large-scale pre-training corpus. See Appendix A for extended details and full hyperparameters.
3.1 Model architecture
Transformer models for time series forecasting have variously used encoder-decoder 2 15,,
encoder-only [13 19 33), and decoder-only architectures [12 21.. ToTo uses decoder-only chitecture (trained on a next-patch prediction task), as it has been shown to scale well with respect to training efficiency when provided with sufficient data 36 37.. We use non-overlapping patch embeddings 19 38 39], with a patch size P = 64, to project input time series of context length L = 4096 points to embeddings of size 64 x D per variate, where D 768 is the embedding dimen- sion for our final model (Fig.2B). We also utilize techniques demonstrated to yield performance and efficiency improvements in contemporary transformer literature, including pre-normalization (40), RMSNorm [1]. SwiGLU feed-forward layers and RoPE 43 with XPOS 44 for improved extrapolation.
We further develop four specialized components purpose-built for handling multivariate observ- ability time series data. Fig. 3 presents an ablation study that demonstrates the impact of these components, and we next highlight the motivations and intuitions behind each of them.
Patch-based causal instance normalization to handle highly nonstationary data. To improve generalization across varying input scales, instance normalization is commonly applied prior to embedding time series data (for example, RevIN [5)).. However, computing normalization statistics from the entire series would leak information from future time steps. This violates the causality of next patch prediction training and results in poor performance (see ablation in Appendix [E). Das et al. 12) normalize the entire series according to the statistics of the first patch. Such an approach preserves causality, but can be ineffective for highly nonstationary data with statistics that vary significantly over time, as the case with observability data (see Section 4.3)
To resolve these issues, we propose novel per-patch normalization approach, where scaling factors for each patch are computed exclusively from the current patch and past data. For timestep we

---

A
Foundation Model Training Corpora
B
Ablation Results
Rixture
Scrabng
of points
12
HET (
23
Luae
Tonwoft
FuctorAced
Figure 3: @ A comparison of the number unique time series points within the pretraining corpora of different time series foundation models. The scale of ToTo's 's training corpus is 4x that of TimesFM 1.0, 5x that of Time-MoE, 6.5x that Moirai, and over x that of Chronos. ①1 Ablation results demonstrate the impact of four of ToTo's 's architectural components motivated by unique properties of observability time series data. Results report the change (relative to the full ToTo model) in negative log likelihood on held-out observability pretraining data when systematically disabling one component a time. See Appendix for details.
define the causal mean رúg and causal variance & as:
WiT
Ei= (x, mb) 0.1,
V E W.
E1=1 W/
where r; represents the input value and w: the corresponding weight at timestep i. We set the weight to for padding positions and for all other positions. We add a minimum value of 0.1 to the causal standard deviation, in order to limit the amount of scaling applied to any particular value and avoid numerical overflow. Timesteps within each patch share the normalization values determined by the final timestep of that patch. As computing causal statistics for every subsequence would have suboptimal Onn)) complexity in the sequence dimension, we instead use Welford's s online algorithm [6). method that provides numerically stable variance calculations in (n)) time. We gain further efficiency with a vectorized adaptation of the algorithm, allowing for GPU parallelism.
This normalization approach preserves causality and is more adaptive than a fixed per-variate scaling factor. However, in practice, we still find training instability in the presence of extreme nonstation- arity. To address this, we relax our requirement of strict causality and introduce a simple clipping mechanism using variate-level statistics. We constrain s, within a range defined by a minimum value, constant exponent K, and the full-variate variance s: max(0.1, x 10-") < 6t < s x 10", (we set K 10 in practice). At inference we compute statistics based solely on the historical context.
Thus, our final approach predominantly preserves causality while substantially enhancing forecast- ing performance, particularly for highly nonstationary series. Additional technical and implementa- tion details are provided in Appendix I
Proportional factorized attention to judiciously capture variate interactions. We design TOTO to natively handle multivariate forecasting by analyzing relationships the time dimension ("time- wise" interactions) and the variate dimension ("variate-wise interactions). While prior works that do not utilize variate-wise relationships (such as PatchTST [99 and TimesFM (12)) can still achieve competitive performance multivariate datasets, other studies (e.g. Woo et al. (I3) have shown benefits from including variate- wise attention in ablations. However, observability metrics are often gigh-cardinality, multivariate time series, and full attention schema simultaneously attending to both the time and variate dimensions (also in Woo et al. 133 can be computationally costly.
Drawing from our experience that time relationships are often more important than cross-variate relationships, we propose a relaxation of factorized attention. Factorized attention alternates atten- tion operations in the time and variate dimensions, allowing for time and variate mixing with lower algorithmic complexity (47)50). While prior works typically apply both variate-wise and time-wiss wise attention in each layer, our design provides more granular control over the relative proportion of time-wise and variate-wise interactions. Specifically, each transformer block has attention along only a single axis, and we can change the ratio of time-wise to variate-wise transformer blocks as a hyperparameter (as illustrated in Figure 2C). ToTo uses an 11:1 ratio (11 time-wise transformer blocks followed by single variate-wiss transformer block), which we found via hyperparameter optimization (see Appendix[A.66
In Section D.4 we observe that TOTO scales significantly better with increasing variates when com- pared to other TSFMs, consistently achieving the lowest CUDA time in its forward pass pass when the number variates is greater than 30.
5

---

Student-T mixture model (SMM) head to model heavy-tailed observability time series. Pro- ducing probabilistic outputs a critical feature of time series models in several domains, including observability [51 In order to produce probabilistic forecasts across the wide range of output distributions present in observability data, we employ a method based on Gaussian mixture models (GMMs), which can approximate any density fitsction [54]. We found that fitting GMMs in the pres- ence of the extreme outliers and high skew found in observability data (see Fig.5) leads to numerical instability in training, so we instead utilize a SMM of K distributions, which robustly generalizes GMMs (55), and has shown promise for modeling heavy-tailed financial time series [56, [57]. The mixture distribution used by Woo al. [1] contains Student-T component alongside other distri- butions, in contrast to our model which solely uses Student-T components. In a contemporaneous work, Yao et al. [58) also explored time series foundation models with Student-T T mixture output. A complete mathematical formulation of our SMM, including equations and parameterizations, is provided in Appendix A.3]
Composite robust loss to stabilize training dynamics. Mixture models optimized via maximum likelihood are known to suffer from singularities [59] and cluster collapse [60]. We use composite loss formulation that we find, in practice, mitigates these effects. During training, we optimize a next-patch prediction task, where the model's objective is to predict the distribution of values in the next patch given all previous patches. Our training combines the standard negative log- ligeliillhood loss. LNLL, and general robust loss, (Romust(ar.)) [61] (see Appendix A.5). The robust loss provides a unified framework that allows for smoothly interpolating between several common robust loss functions [0), using parameters
| 2] and 8 > O (see Fig. 7. After hyperparameter optimization, we found the Cauchy loss (a = 0) performed best in our setting:
(Robuss(0.8) (2 22) = LCauchy (Fz, &t, 6) = log (0 +1)
with - 0.1. While the NLL loss utilizes the full probabilistic output of the model, the robust loss operates point-wise and measures the prediction error between the predicted SMM mean and the ground truth data point. The final combined loss used for training Toto is: C = LNLL ANLL
+ (1 \NLL) LRobust, where ANLL [0, 1] is ratio tuned simultancously with the robust loss hyperparameters, with optimal value ANLL = 0.57. In summary, we find the composite loss critical for avoiding degenerate solutions and the robustness of the point-wise loss necessary for mitigating the effects of outliers that are common in observability data (see Scction|.3) Further details, including explicit definitions of each loss component, are provided in Appendix A.5
3.2 Training data
We trained ToTO with a dataset of approximately 2.36 trillion time series points, of which 1.59 trillion are non-repeated and non-synthetic. This is significantly larger than the pretraining corpora of existing time series foundation models (Fg..3A). Critically, 43% of our training mixture contains anonymous observability metrics from the Datadog platform. This data excludes any customer data and is sourced solely from the platform's is own monitoring of internal systems. It consists only of numerical time series data, and does not include any identifiable metadata. However, much of this data is sparse, noisy, or too granular or high in cardinality to be useful in its raw form. To curate high-quality dataset, we sample queries based on quality and relevance signals from dashboards, monitor alerts, and notebooks built by domain experts using the platform.
Alongside the Datadog data, we include public time series datasets, in particular, the GIFT-Fxal Pretrain ① and Chronos [1] collections. Importantly, we remove the subset of the Chronos datasets that overlap with the GIFT-Eval benchmark in order to avoid any leakage from the test data. We also find that adding synthetic data improves model generalization and performance. For more details on the preparation of public, synthetic, and Datadog data, please see Appendix B
4 BOOM
We next introduce BOOM, a large-scale, open-source evaluation framework specifically designed to capture the unique forecasting challenges posed by modern observability workloads.

---

A
Domains in BOOM
Example Series
45%
Inf tructuuu
Application
Application
afaetructure
Datatoon
34.0h
Database
29.5%
Metworkking
Secur by
15c
18. 8%
1959
Figure 4: @ BOOM consists of data from various domains represented within the Datadog platform. @ Ex- ample series from three of the domains. From left to right, these series represent: sum of failed requests backend API, grouped by error type and source (Application); CPU limits on multi-tenant service deployed on a Kubernetes cluster, grouped by tenant (Infrastructure); and sum of command executions on Redis cache, grouped by command (Database).
4.1 Dataset
Like the ToTO pretraining data, the observability data comprising BOOM is also sourced from the Datadog platform. However, to ensure robust evaluation setting and prevent contamination, train- ing data for ToTO (see Section 3.2) originates exclusively from production environments, while BOOM'S evaluation data is drawn from separate staging environment. Because isolation between production and staging data is guaranteed at the platform level, we can be certain there is no leakage. Like the training data, the BOOM data excludes any customer data and is sourced solely from the platform's 's own monitoring of internal systems.
BOOM'S dataset consists of approximately 350 million data points across 2,807 metric queries gen- erated through the monitoring of real-world distributed systems (Fig. [8. These series vary signifi- cantly in sampling frequency, temporal length, and cardinality, capturing realistic operational condi- tions (Table I. Despite having fewer unique series compared to GIFT-Eval (2,807 vs. 144K), BOOM features substantially more total data points (350M vs. 158M) and significantly higher dimension- ality, with a median of 60 variates per series compared to GIFT-Eval's predominantly univariate or low-cardinality multivariate series. Both BOOM and GIFT-Eval are significantly larger and more diverse than the older LSF benchmark. For accessibility in light of BOOM'S large scale, we also provide (and evaluate on) a smaller uniformly sampled subset ("BOOMLET"), which we describe in Appendix
4.2 Domain taxonomy
In order to highlight the diverse characteristics BOOM'S data, we assign each time series one or more labels, based on its query string, according to taxonomy of key domains within the scope of observ- ability monitoring. The initial labeling is conducted by prompting an I.LM, which is then followed by human verification. Application Usage covers application interactions and user activity (c.g., request rates, API calls); Infrastructure includes system-level metrics (e.g., CPU usage, memory consumption); Database focuses on database operations and efficiency (e.g. query latency); Net- working encompasses network monitoring signals (e.g. throughput, latency); and Security relates to authentication, intrusion detection, and compliance checks (e.g. log-in attempts, code vulnera- bilities). We show the prevalence of each class within BOOM and provide illustrative examples in Fig. 4
Details of the taxonomy labeling procedure provided in Appendix C
4.3 Statistical characteristics
Observability time series are noted for having challenging distributional properties, including spar- sity, spikes, and noisiness 71]; anomaly contamination [72] (which can manifest as skew and/or kurtosis); and for their high-cardinality multivariate nature, with complex relationships among vari- ates [73]. They can also exhibit extreme dynamic range [3 for example, metrics recording disk usage in bytes can range across many orders of magnitude. To illustrate these properties, we examine six relevant statistics for each variate in BOOM, Gval-Eval, and LSF (compared in B):
The first-lag autocorrelation function (ACF) measures short-term temporal dependence, with a small value indicating noisy local oscillations. Note the large lower tail in the BOOM distribution.
7

---

Staristic
Entraay
Spots
4.0
5.1
Denesty
LSP
Figure 5: Distributional comparison of statistical features computed on normalized time series from the BOOM GIFT-Eval, and LSF benchmark datasets. The broader and shifted distributions in the BOOM series reflect the increased diversity, irregularity, and nonstationarity characteristic of observability data.
The ARCH-LM statistic 774 detects autoregressive conditional heteroskedasticity. Lower values suggest time-varye vare-iarying volatillity; the BоOM distribution is bimodal with a large peak near zero.
Spectral entropy
73,, which quantifies the unpredictability of series in the frequency domain, is also higher on average in observability series. This indicates less periodicity and greater irregularity.
The KPSS statistic 77 test of nonstationarity, and assumes larger values in the BOOM observ- ability data, suggesting more frequent deviations from a deterministic trend.
"Flat spots" (76) measures the longest constant subsequence with a time series, normalized by series length. This metric is higher in BOOM, indicative of sparse metrics within observability data.
Skew reveals heavier-tailed, asymmetric distributions in observability data, often reflecting bursty behavior and rare but extreme events.
Overall, we find that the more extreme values of each of these statistics the BOOM data, reinforces the distinct and challenging nature of observability times series (see also Fig. (B).
4.4 Evaluation protocol
To evaluate models on BOOM we closely follow the evaluation methodology proposed by Aksu al. [ for GIFl-Eval, including its standardized prediction lengths, strides, and crann/vlidationn/est splits. As in GIFT-Eval, our key accuracy metries are Mean Absolute Scaled Error (MASE) [77] and approximate Continuous Ranked Probability Score (CRPS) 778 79, both normalized by the Seasonal Naive forecast. And like GIFT-Eval, we also compute an average rank (where rank computed as the mean rank with respect to CRPS across all prediction tasks).
We deviate from the GIFT-Eval protocol in two key respects in order to handle the presence of metrics with mostly constant values and infrequent spikes, which can cause problems in computing traditional metrics. First, we employ shifted geometric mean for aggregating MASE and CRPS across tasks, which is stable in scenarios where constant subsequences would otherwise cause a standard geometric mean to collapse 80 81]. Second, we use separate evaluation approach for small subset of the time series in BOOM where extreme cases of constant subsequences still cause numerical instability for www.---rrratttommaaaai metrics. See Appendix C for details.
Dataset Series
# Varintes
# Points (M)
Interval
Series Length Mod
Varlates Per Series
Pred. Length
Min
Max
Min
Max
Min
Med
Max
Min
Max
BOOM
2,807
32,887
350
10 sec.
day
101
13,631
16.384
1
60
100
48
900
BOOMLET
1,627
sec.
5,231
16.384
16,384
GIFT-Eval
144,246
147.688
158
10 soc.
yr.
19
1,043
140,256
1
21
6
900
LSF
370
17,420
39,500
69,680
7
720
Table I: Key statistics the time series benchmarks used for evaluation in this study. BOOM, introduced
in this paper, contains approximately twice as many total time series points as GIFT-Eval (350M vs. 158M)
and vastly more than LSF. BOOM, consists of multivariate series of much higher average cardinality than the
other benchmarks, with median of 60 variates per series only for GIFT-Eval and 7 for LSF. Conversely, BOOM contains narrower range intervals compared to GIFT-Eval, ranging from 10 seconds to day, vs. 10 seconds to year for GIFT-Eval. This reflects the generally shorter time scales relevant to the observability domain. BOOMLET, smaller subset of BOOM that we provide for easier evaluation, exhibits similar statistics.
8

---

5
Experiments
We compare against a comprehensive set of methods, including zero-shot foundation models ('Zero Shot') ) neural models trained on the target data ('Full Shot'), and classical supervised approaches ( ("Baselines)) )
We evaluate ToTO on three benchmarks: BOOM, GIFT-Eval, and LSF (see Table
BOOM. We evaluate TOTO's zero-shot forecasting performance alongside other foundation models, I14 23 [4], as well as full-shot statistical baselines. We do not evaluate full-shot deep learning models on BOOM; as discussed in Section [ these models are often impractical in real-word ob- servability systems at scale. Similarly, we were unable to evaluate TabPFN [82] (a recent zero-shot model with strong performance on GIFT-Eval) on BOOM, as its open- source implementation' 's lack of support for batched inference made impractically slow to run. Details of the inference settings
and evaluation procedures for all models are described in Appendix C.2 As shown in Table 2
Toto consistently outperforms other models, achieving 13.1% % and 12.4% lower MASE and CRPS, respectively, than the next best (Moiraigase. and a significantly lower rank (2.351 vs. 4.278). Similar qualitative results hold on the BOOMLET subset (Appendix I.I.II.
Shot
Burface
Datreet
Metric
Toto
Mideaigus
TimsFMy
Chropogaca
Tiner
Tane-MrEque
VaiouTS
AstoAARMMA
AutoETS
Aute- Thets
DLineu
DeeAR
RooM
CRPS
0.375
0/118
(.469
0.451
0.899
0.43
0.613
0.756
1.755
1.08
Rank
2.369
1378
52443
5.576
0.900
9.843
10.989
9.680
11:071
12.472
0.779
0.793
0,90
CRPS
0.519
0,703
0.865
0.500
0,917
Rank
1.300
5,344
4.44
6411
11.344
11.011
13.278
13.778
14.156
14,522
6,44
7,822
Table 2: BOOM results. Performance of TOTO, other zero-shot models, and baselines. MASE and CRPS are
normalized by the Seasonal Naive forecast and aggregated across tasks using shifted geometrie mean. Rank is the mean rank across tasks with respect CRPS. For model families with multiple sizes (Moirai, Chronos) we show the hest-performing variant. ToTO significantly outperforms other methods on all metries. We observe similar qualitative trends on the BOOMLET results. Additional results, including all model sizes evaluated as Key: Best results, Second-best results.
well as categorical breakdowns, are available in Appendix D.I
GIFT-Eval. We evaluate TOTO's zero-shot performance on general-purpose time series forecasting via the GIFT-Eval benchmark M. ToTO achieves the top performance among all reported mod- els, with an average ranking score of 5,495 as of May 2025. It achieves strong results both in point forecasting, with a MASE of 0.673, and probabilistic forecasting, with a CRPS of 0.437. De- tails of our evaluation settings for GIFT-Eval are provided in Appendix D.2 Notably, ToTO is the pop-perfrrming method in spite of the fact that several competing models have known partial data leakage with the benchmark, as discussed by Aksu et al.
Zaco Shot
Full Short
Barcliass
Metric
Toro
Moirthare
TimesPMaa
Chromessestor
TabPEN-TS S TEMPO
TTM-R2
PatchTST
TFT
Auto-ARIMA
Auto-ETS ETS
Auto- Auto-Thelu
MASE
0.673
0.725
0.762
0.964
L088
0.978
CRPS
0.477
0.185
0,496
0,770
Kank
5.495
10.300
8.412
10
10.258
11.629
21,608
25.134
24.134
Table 3: GIFT-Eval results. TOTO's performance vs. the top Zero Shot, Full Shot, and Baseline models re-
ported on the lcaderboard (3. MASE and CRPS are normalized by the Scasonal Naive forecast and aggregated across tasks using geometric mean. Rank is the mean rank across tasks with respect to CRPS. For multi-size models (Moirai, Chronos) we show the best-performing variant. ToTO achieves top performance on GIFT-Eval Rank and MASE, and second-best performance CRPS. Key: Best results, Second-best results.
LSF. We evaluate ToTO on the widely-used Long Sequence Forecasting (LSF) benchmark Despite known limitations regarding dataset diversity and saturation by supervised methods [ 29 ToTo achieves
of-e---f-hearr results in zero-shot evaluations, attaining the best performance on out of 12 reported metrics when compared against other zero-shot methods, and the lowest average MAE and MSE (see Table 4). Furthermore, while our focus this work is on the zero-shot setting, we explored the efficacy of fine-tuning TOTO on the training splits of LSF and report the results in Table 15 We find that ToTo achieves state-of-the-are results in full-shot evaluations, also attaining the best performance on 8 out of 12 reported metrics, and the lowest average MAE and MSE of all methods. These results underscore ToTo's strong generalization capabilities and its effectiveness when fine-tuned on specialized datasets, making it a versatile choice for wide range of time series forecasting tasks. See Appendix D33 for full results and additional experimental details.

---

Dataset
Metric
Toto
Moiraisual
Moiraigas
Moiraiy
VisionTS
Time-MoEgue
Time-MoEAang
Time-MoEquoe
ETTh
MAEL
0.438
0.414
0.426
MSE
0.435
0.400
0.434
0,510
0,390
0.400
0.394
0.412
ETTh2
MAE
0.363
0.379
0.382
0.376
0.375
0.415
0.399
MSE
0.340
0.341
0.345
0.354
0,333
0.366
0.405
0.371
ETTml
MAE
0.378
0.409
0.388
0.389
0.372
0,455
0,405
0.391
MSE.
0.396
0.448
0.381
0.390
0.374
0.394
0.376
0.356
ETTm2
MAE
0.303
0.341
0.321
0.320
0.321
0.365
0,361
0.344
MSE
0.267
0.300
0.272
0.276
0,282
0.317
0.316
0.288
Electricity
MAE
0.243
0.320
0.274
0.273
0.294
MSE
0.161
0.233
0.188
0.188
0,207
Weather
MAE
0.245
0.267
0.261
0.275
0.292
0.297
0,300
0.288
MSE
0.224
0.242
0.238
0.259
0.269
0.270
0.256
Mean
MAE
0.324
0.357
0.344
0.350
0.345
MSE
0.304
0.327
0,310
0.330
0.309
Best Count
3
Table 4: LSF results Zcro-Shot comparison of models on the LSF benchmark. Non- TOTO values are repro- duced from published tables. Key: Best results, Second-best results. "Best Count" row reports the number of times each model attains the best result for given datasct-metric pair.
6 Conclusion
This work reframes time series forecasting through the lens of observability-a domain marked by scale, complexity, and real-world urgency. We presented ToTo, foundation model purpose-built to forecast multivariate observability metrics with zero-shot accuracy, and introduced BOOM, the first benchmark to capture the messy, high-cardinality reality of production telemetry data at scale. ToTo advances the frontier in zero-shot time series forecasting and sets new state-of-the-an results on BOOM, GIFT-Eval, and LSF. A limitation of ToTO (and other TSFMs) is the assumption of fixed- interval times series. Currently, we address missing points with heuristic imputations; natively han- dling missing data may provide better solution. Additionally, TOTO does not directly incorporate calendar-based features-an interesting direction for future work. Finally, a study of performance extreme prediction lengths would be insightful for certain applications. By open-sourcing both model and benchmark, we hope to accelerate research to answer these and other open questions, contribute the community, and to draw attention an important real- woaldworld application.
Acknowledgements
Our work is made possible by the efforts of numerous teams at Datadog and beyond. Special thanks and acknowledgment to:
Aaron Taa, Alexis Lê-Quôc, Amine Naouas, Antoine Gaillard, Ara Pulido, August Marsalis, Ben Donohue, Ben Hinthorne Benedetto Buratti, Bharath Vontimitta, Bill Birkholz, Brendan Rhoads, Charuprabha Gaur, Damián Vicino, Dan Haggerty, Dengke Liu, Dimitrios Gklezakos, Dominique West, Emilie Xu, Erica Hale, Gerald Woo, Jake Femminineo, Jake Hooker, Janine Kromhout, Jared Ledvina, Jared Schifrien, Jeremy Garcia, Jeromy Carriere, Jesse Mack, Jessica Cordonnier, Joe Jones, Johan Andersen, Kathy Nguyen, Kevin Beach, Luca Pizzamiglio, Madison Moore, Mar- ion Chan-Renout Max Livingston, Maxim Brown, Maxime Visonneau, Mayeul Blanzat, Michael Hoang, Mikhail Khodak, Mononito Goswami, Nick Sollecito, Olivier Pomel, Phil Sarin, Quentin François, Quentin Gendre, Raya Wakil, Rikki Endsley, Roashan Ayene, Rob Boll, Romoli Bakshi, Sajid Mehmood, Sean O' Connor,, Steven Zhou, Vyom Shah, and Zakaria Fikrat.
References
[1] Taha Aksu, Gerald Woo, Juncheng Liu, Xu Liu, Chenghao Liu, Silvio Savarese, Caiming Xiong, and Doyen Sahoo. Gift-eval: A benchmark for general time series forecasting model evaluation, 2024. URL httthtttt///aiiii /ttpvv/axvvvvvv/ (arg/abs/2444 10393
[2] Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. 2021. URL https: //
10

---

openreview.nee http:/nne/fooooo hianrorr.e..et/ooooum////////11
[3] GIFT Eval Hugging Face Space by Salesforce.
URL https: hwwgingggange....c/// co/
spaces/Saleefforre@trF-P
[4] C. Majors, L. Fong-Jones, and G. Miranda. Observability Engineering. O'Reilly Media, 2022. ISBN 9781492076414, URL http:://bo::::ss www.ookkss.ogg...ooo http://ec/ooooostooa/AAAAAAAAA.
[5] Ze Li, Qian Cheng, Ken Hsieh, Yingnong Dang, Peng Huang, Pankaj Singh, Xinsheng Yang, Qingwei Lin, Youjiang Wu, Sebastien Levy, and Murali Chintalapati. Gandalf: an intelligent, end-to-end analytics service for safe deployment in cloud-scale infrastructure. In Proceedings of the 17th Usenix Conference on Networked Systems Design and Implementation, NSDI'20, page 389-402, USA, 2020. USENIX Association. ISBN 978 97999333331.7.
[6] Emily Chang. Introducing metric forecasts for predictive monitoring in Datadog, December 2017. URL WtRL:htts::// htt.ttttt//ww..ccom http:///wwwwaaaadddoooo cww/blog////occocccccccccc.ooo.o com/blog/fooeccaseeoooooooooo/co woweeaaattsdtttaoooo.ooo
[7] How we scaled our new
Prometheus TSDB Grafana Mimir to
bil-
lion active series.
URL
https grafana com / blog 2022 04 08
how-se-scaled-our pre.eaaeeeeerrreeeeerrepepeeeeee pww.ethuus-
[8] How Cloudflare runs Prometheus at scale, March 2023. URL http:::/bbo...clouuet
[9] Observability at Scale: How the Right Stack Can Help Recurly. URL https://recurly.
[10] Xu Liu, Juncheng Liu, Gerald Woo, Taha Aksu, Yuxuan Liang, Roger Zimmermann, Chenghao Liu, Silvio Savarese, Caiming Xiong, and Doyen Sahoo. Moirai-moe: Empowering time series foundation models with sparse mixture of experts. arXiv preprint 21X1v:2410.10469, 2024.
URL https://arxiv org/ abs///////21..../ 10469
[11] Abdul Fatir Ansari, Lorenzo Stella, Caner Turkmen, Xiyuan Zhang, Pedro Mercado, Huibin Shen, Oleksandr Shchur, Syama Sundar Rangapuram, Sebastian Pineda Arango, Shubham Kapoor, Jasper Zschiegner, Danielle C. Maddix. Hao Wang, Michael W. Mahoney, Kari Torkkola, Andrew Gordon Wilson, Michael Bohlke-Schneiderr and Yuyang Wang. Chronos: Learning the language of time series, 2024. URL https ptattps:///rxxt //aaxivv..vi/////a wbwg//as//03.0000 07815
[12] Abhimanyu Das, Weihao Kong, Rajat Sen, and Yichen Zhou. A decoder-only foundation model for time-series forecasting. In Forty-firsi International Conference on Machine Learn- ing, 2024. URL/https://oppeerrviie htt.er/e...eeoooorrddnnaaaaaaaaa
[13] Gerald Woo, Chenghao Liu, Akshat Kumar, Caiming Xiong, Silvio Savarese, and Doyen Sa- hoo. Unified training of universal time series forecasting transformers. In Forty-first Interna- tional Conference on Machine Learning, 2024. URL httpas///openrevvie net//frumm
id= Yd8eHMY1wz
[14] Xiaoming Shi, Shiyu Wang, Yugi Nie, Dianqi Li, Zhou Ye, Qingsong Wen, and Ming Jin. Time-moe: Billion-scale time series foundation models with mixture of experts. In Interna- tional Conference on Learning Representations (ICLR), 2025. URL iot Spotlight Presentation.
[15] Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wan Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. 2020. htRpsstts:://et
wwwanticscholar.......e.cac.aa.e orgg Cotg/oorpssI..... :oo 229156882
[16] Sudipto Guha, Nina Mishra, Gourav Roy, and Okke Schrijvers. Robust random cut forest based anomaly detection on streams. In Maria Florina Balcan and Kilian Q. Weinberger, ed- itors, Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, pages 2712-2721 New York, New York, USA, 20-22 Jun 2016. PMLR. httpst:s:: /ttpr///eeeeee..../////// mlr. press/v48/ http//ghh1..... html
[17] George E. P. Box and Gwilym M. Jenkins. Time Series Analysis: Forecasting and Control. Holden-Day, San Francisco, 1970.
11

---

[18] Datadog. Anomaly Monitor.
URL httpss://doce waw.adoggh.cco//ooottrrrrrr
anomaly/
[19] Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time se-
ries is worth 64 words: Long-term forecasting with transformers. 2023. URL https:
/ //openreview nef/fooruriddd/oooooooooo
[20] Azul Garza and Max Mergenthaler-Cansee Timegpt-1, 2023.
[21] Kashif Arjun Ashok, Andrew Robert Williams, Arian Khorasani, George Adamopoulos, Rishika Bhagwatkar, Marin Biloš, Hena Ghonia, Nadhir Hassen, Anderson Schneider. Sahil Garg, Alexandre Drouin, Nicolas Chapados, Yuriy Nevmyvaka, and Irina Rish. Lag-llama: Towards foundation models for time series forecasting. In (O-FFoMoRRbuuttnee of Few-shot and Zero-shot Learning in Large Foundation Models, 2023. URL https: //te:////oeeeeeeeeeee pet//oorun?id=-yuu22Z2
[22] Nate Gruver, Mare Anton Finzi, Shikai Qiu, and Andrew Gordon Wilson. Large language models are zero-shot time series forecasters. In Thirty- Conference on Neural Informa- tion Processing Systems, 2023, htttss:/ wwwprrreeeeieeeeefffeeeeeeee www./eeenormiie.. .ett:forrummddmmmm888881222 d=m.2ideiimmmeee111111
[23] Yong Liu, Haoran Zhang, Chenyu Li, Xiangdong Huang, Jianmin Wang, and Mingsheng Long. Timer: generative pre-trained transformers are large time series models. In Proceedings of the 4 st International Conference on Machine Learning, ICML'24. JMLR.org. 2024.
[24] Mouxiang Chen, Lefei Shen, Zhuo Li, Xiaoyun Joy Wang, Jianling Sun, and Chenghao Liu. Visionts: Visual masked autoencoders are free-lunch zero-shot time series forecasters. arXiv preprint 111vv:2408.17253, 2024. doi: 10.44550/arxi..2408..777
[25] Mononito Goswami, Konrad Szafer, Arjun Choudhry, Yifu Cai, Shuo Li, and Artur Dubrawski. Moment: a family of open time-series foundation models. In Proceedings of the 41 Interna- tional Conference on Machine Learning, ICML'22. JMLR.org, 2024.
[26] Rakshitha Godahewa, Christoph Bergmeir, Geoffrey Webb, Rob J Hyndman, and Pablo Montero-Manso Monash time series forecasting archive. Neural Information Processing Systems Track on Datasets and Benchmarks, 2021.
[27] Spyros Makridakis and Michèle Hibon. The m3-competition: results, conclusions and impli- cations. International Journal of Forecasting, 16(4):451-47,, 2000. ISSN 0169-2070. doi: http:://w.ooy//.....///660000000 com/ coop://cccccc//rtcccc////////000
URL Ihttps:/ :/pps://wwwwt /twp:/cccieceeieretttt... The M3- Competition.
[28] Spyros Makridakis, Evangelos Spiliotis, and Vassilios Assimakopoulos. The m4 competition: 100.000 time series and 61 forecasting methods. International Journal of Forecasting, 36(1): 54-74, 2020. ISSN 0169-2070. doi: http://ww.ccctteeeeeoe//eeeaaaea
URL
M4
Competition.
[29] Zongzhe Xu, Ritvik Gupta, Wenduo Cheng, Alexander Shen, Junhong Shen, Ameet Talwalkar, and Mikhail Khodak. Specialized foundation models struggle to beat supervised baselines. arXiv preprint 21X07:241..02766, 2024.
[30] Xiangfei Qiu, Jilin Hu, Lekui Zhou, Xingjian Wu, Junyang Du, Buang Zhang, Chenjuan Guo, Aoying Zhou, Christian S. Jensen, Zhenli Sheng, and Bin Yang. Tib: Towards comprehensive and fair benchmarking of time series forecasting methods. Proc. VLDB Endow, 17(9):2333- 2377, 2024.
[31] Artjom Joosen, Ahmed Hassan, Martin Asenov, Rajkarn Singh, Luke Darlow, Jianfeng Wang, and Adam Barker. How does function? characterizing long-term trends in production serverless workloads. In Proceedings of the 2023 ACM Symposium on Cloud Computing, SoCC '23, page 443-458, ACM, October 2023. doi: 10. 10.1145/3620678. 36247883. URL h/dx: doi www.oog/////.
3624783
12

---

[32] William Toner, Thomas L. Lee, Artjom Joosen, Rajkarn Singh, and Martin Asenov. Per- formance of zero-shot time series foundation models on cloud data, 2025. URL https:
//arxiv.org/ 2502. 12944
[33] Gerald Woo, Chenghao Liu, Akshat Kumar, and Doyen Sahoo. Pushing the limits of pre- training for time series forecasting in the cloudops domain, 2023. URL https: //arxiv.
org/abs/2310. 23063000633333
[34] Santosh Palaskar, Vijay Ekambaram, Arindam Jati, Neelamadhav Gantayat, Avirup Saha, Seema Nagar, Nam H. Nguyen, Pankaj Dayama, Renuka Sindhgatta, Pratceti Mohapatra, Harshit Kumar, Jayant Kalagnanam, Nandyala Hemachandra, and Narayan Rangaraj. Au- tomixer for improved multivariate time-series forecasting on business and it observability data, 2310. 20280
2023. BRLLhttps:
[35] Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting. 2024. URL
httts:://pperreviev. ott.://www.....eoooomoooooooe.e.
[36] Alec Radford and Karthik Narasimhan. Improving language understanding by generative pre- training. 2018. URL
http://ppp..snaanicccccccollaaaa httpcchollaorrr///ooooo Drgg/orpusII::9000 49313245
[37] Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. 2019. URL httpss///e /ppi.
semanticacholar.c pwtianttieccolaa...oo///pppppp/p org/Corppooo: :1::00pp
[38] Man-BBatiste Cordonnier, Andreas Loukas, and Martin Jaggi. On the relationship between self-attention and convolutional layers. In 8th International Conference on Learning Rep- resentations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020.
URL httpts:://ppeereeiieeeeee wwwperewwwwwwweeeeeeeeeeeeeeeeee waeee//ooummiidelllCiii...
[39] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition scale. In International Conference on Learning Representations, 2021, URL https://openrevier www.earrrrreee.eteoooummmdooood
[40] Ruibin Xiong, Yunchang Yang, Di He, Kai Zheng, Shuxin Zheng Huishuai Zhang, Yanyan Lan, Liwei Wang, and Tie- Yan Liu. On layer normalization in the transformer architecture, 2020. /RRL/ttts://opeeeeeee httttp/se/.ppeereerveeeeeeee//ra wwt.://ooormmmdddB18aaaaaa
[41] Biao Zhang and Rico Sennrich. Root Mean Square Layer Normalization. In Advances in Neural Information Processing Systems 32, Vancouver, Canada, 2019. URL https:
/openreviev. http:///efeereeccee////r/-saee
[42] Noam Shazeer. Glu variants improve transformer, 2020. URL htttss:///aa::: http:://./r.voooo///.///o arg/abs/ abs/
[43] Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568: 127063, 2024. ISSN 0925-2312. doi: https::/doi.orr/10..n 1011///0..0000....... new.// 202. URL https: //www.
sciencedirect.co cww.coo/ssssooo http:///w...aaaaaaaaaa...///////
[44] Yutao Sun, Li Dong, Barun Patra, Shuming Ma, Shaohan Huang, Alon Benhaim, Vishrav Chaudhary, Xia Song, and Furu Wei. A length-extrapolatable transformer. In ACL 2023, December 2022. URL httt::://///w/
http:///wwwmmmcootttttt www./e/oseeeeccccooooooo wwtp///wwwn/eeeeeeerrr////aaaaaa
a- e-length-eztrapollttee oatttppppolaaabl--ttaasffrrrrrr transformer/
[45] Taesung Kim, Jinhee Kim, Yunwon Tae, Cheonbok Park, Jang-Ho Choi, and Jaegul Choo. Re- versible instance normalization for accurate time-series forecasting against distribution shift. In International Conference Learning Representations, 2022. URL https: /ttpps///openrevierr net/forum?id=cCoae (tt.foorum?dd=ccBkkk0000000
[46] B. P. Welford. Note on method for calculating corrected sums of squares and products. Technometrics, 4(3):419-420, 1962.
13

---

[47] Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension de- pendency for multivariate time series forecasting, In The Eleventh International Confer- ence on Learning Representations, 2023. URL VSVLM2j9eie
[48] Roshan M Rao, Jason Liu, Robert Verkuil, Joshua Meier, John Canny, Pieter Abbeel, Tom Sercu, and Alexander Rives. Msa transformer. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Pro- ceedings of Machine Learning Research, pages
PMLR, 18-24 Jul 2021. URL.
httpp://proceedinggs mlr
[49] Anurag Arnab, Mostafa Dehghani, Georg Heigold, Chen Sun, Mario Lučić, and Cordelia Schmid. Vivit: A video vision transformer. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 6816-6826, 2021. doi: 10.110//CCV4499222221..000
[50] Shanghua Gao, Teddy Koker, Owen Queen, Thomas Hartvigsen, Theodoros Tsiligkaridis, and Marinka Zitnik. UniTS: A unified multi-task time series model. NeurlPS, 2024. URL https:
hwtp:/rgggbbbbbbeeeeeeooeee 2aa.b//2443.0000 00131
[51] Lingxue Zhu and Nikolay Laptev. Deep and confident prediction for time series at uber. In 2017 IEEE international conference on data mining workshops (ICDMW), pages 103-110. IEEE, 2017.
[52] Cheryl Lee, Tianyi Yang, Zhuangbin Chen, Yuxin Su, and Michael R Lyu. Maat: Perfor- mance metric anomaly anticipation for cloud services with conditional diffusion. In 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 116 128. IEEE, 2023.
[53] Haitian Hang, Xiu Tang, Jianling Sun, Lingfeng Bao, David Lo, and Haoye Wang. Robust auto-scaling with probabilistic workload forecasting for cloud databases. In 2024 IEEE 40th International Conference on Data Engineering (ICDE), pages 4016-4029. IEEE, 2024.
[54] Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. http://www.deepleeriinnooooo org
MIT Press, 2016.
[55] D. Peel and G.J. McLachlan. Robust mixture modelling using the t distribution. Statistics and Computing, 10(4):339-348, 2000.
[56] Mika Meitz, Daniel A. Preve, and Pentti Saikkonen. A mixture autoregressive model based on student's ' t-distribution. Communications in Statistics Theory and Methods, 52:499 - 515, 2018. URL
http::/pp...aaata..coooaooooorrp 73311544
[57] C. S. WONG, W. S. CHAN, and P. L. KAM. student -mixture autoregressive model with applications to heavy-tailed financial data. Biometrika, 96(3):511-76,, 2009. ISSN 00063444, 14643510. http:///: http://wwwwjt.orrro/oo/oo jstor. atg/staable/2777886)
[58] Qingren Yao, Chao-Har Huck Yang, Renhe Jiang, Yuxuan Liang, Ming Jin, and Shirui Pan. Towards neural scaling laws for time series foundation models, 2025. https:t::: //arxi/.x..v
org/abs/2410.. 02360
[59] Christopher M. Bishop. Mixture models and em. In Pattern Recognition and Machine Learn-
ing. pages 423-49994 Springer. New York, 1 edition, 2006. ISBN 978-0387310732. URL
https:://inde http:///iinn..iinggerre.. wttp://oooo////////////0//
[60] Jason Eisner. Dynamics of optimizing gaussian mixture models. https::/ cs js.juu.
edu/ htaaa//jaoontutorialss///// www.o/MMtttiiiiiiiiiiioiiii
htmll Accessed: 2025-04-09
[61] Jonathan T. Barron. A more general robust loss function. CoRR, abs/1701.03077 2017. URI.
http:///rxiv..rrtt wtt..///aaaa...../00/0110
[62] Michael J. Black and P. Anandan. The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. Computer Vision and Image Understanding, 63(1):75-104, 1996. ISSN 1077-3142, doi: https://doi.org/10.. 1006/ 1996. 0006.
URI. https: www.
14

---

[63] Donald Geman and Stuart Geman. Bayesian image analysis. In E. Bienenstock, F. Fogelman Soulié, and G. Weisbuch, editors, Disordered Systems and Biological Organization, pages 301-319, Berlin, Heidelberg, 1986. Springer Berlin Heidelberg. ISBN 978-3-642-82657-33
[64] G. Aubert, M. Barlaud, L. Blanc-Feraud, and P. Charbonnier. A deterministic algorithm for edge-preserving computed imaging using Legendre transform. In 12th IAPR International Conference Patlern Recognition, 1994, volume 1, pages 188-191 vol.3, Los Alamitos, CA, USA, October 1994, IEEE Computer Society, doi: 10. 109/CPR. 1994.577154. URLhttpp:: //oo__Leecapppeeroooeeee.y.o/g/p 11t.:0..1199.....1 1999994.771599
[65] Deqing Sun, Stefan Roth, and Michael J. Black. Secrets of optical flow estimation and their principles. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 2432-2433 2010. doi:
[66] John E. Dennis Jr. and Roy E. Welsch and. Techniques for nonlinear least squares and robust regression. Communications in Statistics Simulation and Computation, 7(4):345- 359, 1978. doi: 10. 00.10000/350100118888111111 03610917808812088)
URL attps://dde org/ 10. 1080/
[67] Yvan G. Leclerc. Constructing simple stable descriptions for image partitioning. International Journal of Computer Vision, 3(1):73-102, 1989. ISSN 1573-1405, doi: 10.1007/BF000054839, URL https: URLhttps: //oi: 10. 1007/BF000058839
[68] Pierre Charbonnier, Laure Blanc-Féraud, Gilles Aubert, and Michel Barlaud. Two determin- istic half-quadratic regularization algorithms for computed imaging, In Proceedings 1994 International Conference on Image Processing, Austin, Texas, USA, November 13-16, 1994, pages 168-172. IEEE Computer Society. 1994. doi: 10. 10.1109/ICIP. 1994. 413553. URL https://doi. org/10. 11w.1/119I/IIPP 113554
[69] Peter J. Huber. Robust estimation of a location parameter. Annals of Mathematical Statistics, 35:492-518, 1964. URL http:tttsss/ppp..emmttccccc www.aaarooooCo/oorr/raaaa 121252793
[70] Zhengyou Zhang. Parameter estimation techniques: a tutorial with application to conic fitting. Image and Vision Computing, 15(1):59-76, 1997. ISSN 0262-8856. doi: https://doi.c org//
URL. https: : //www. sciencedirect. com/ science/
#riicee/pp1//0222255560011
[71] Qian Cheng, Doyen Sahoo, Amrita Saha, Wenzhuo Yang, Chenghao Liu, Gerald Woo, Man- preet Singh, Silvio Saverese, and Steven C. H. Hoi. Ai for it operations (aiops) on cloud platforms: Reviews, opportunities and challenges, 2023. URL https::t///are org/ /bbs/ 2364.04661
[72] Xudong Mou, Rui Wang, Bo Li, Tianyu Wo, Jie Sun, Hui Wang. and Xudong Liu. Roca: Robust contrastive one-class time series anomaly detection with contaminated data, 2025. URL
https://arxiv.org/aaa abs/00003... 18385
[73] Santosh Palaskar, Vijay Ekambaram, Arindam Jati, Neelamadhav Gantayat, Avirup Saha, Seema Nagar, Nam H. Nguyen, Pankaj Dayama, Renuka Sindhgatta, Prateeti Mohapatra, Harshit Kumar, Jayant Kalagnanam, Nandyala Hemachandra, and Narayan Rangaraj. Au- tomixer for improved multivariate time-series forecasting on business and it observability data. Proceedings of the AAAI Conference on Artificial Intelligence, 3821)) Mar. 2024. doi: (0.1660/aaai.v882)) v33.21.30336.6666 URL hhttps://o// aaai org/ index. php/AAAI/ article/view/300 30336
[74] Robert F. Engle. Autoregressive Conditional Heteroscedasticity with Estimates of the Variance of United Kingdom Inflation. Econometrica, 50(4):987, July 1982 ISSN 00129682. doi: 10.2307/1912773. URL http://ww...oo.rrrr/aaaall//////
[75] Denis Kwiatkowski, Peter C. B. Phillips, Peter Schmidt, and Yongcheol Shin. Testing the null hypothesis stationarity against the alternative of unit root: How sure are we that economic time series have a unit root? Journal of Econometrics, 54(1):159-178, October 1992. ISSN 0304-4076. doi: 10. 10010016/00004444076/99999000000 coom cttl://cceeeeerrrrreeeee///////0
URL https: / www. sciencedirect..
15

---

[76] Rob Hyndman, Yanfei Kang, Pablo Montero-Manso, Mitchell O' Hara-Wild, Thiyanga Ta- lagala, Earo Wang. and Yangzhuoran Yang. tsfeatures: Time Series Feature Extraction, 2024. URL https: //g.. robjhyndman com/tsfeatures: http:///gthub..oom//tttttttttttt
Python port, version 0.4.5,
[77] R. Hyndman and A. B. Koehler. Another look at measures of forecast accuracy. International Journal of Forecasting, 22, 2006.
[78] Tilmann Gneiting and Adrian Raftery. Strietly proper scoring rules, prediction, and es- timation. Journal of the American Statistical Association, 102:359-378, 03 2007. doi: 10. 101199/0162155000000
[79] Youngsuk Park, Danielle Maddix, François-Xavier Aubet, Kelvin Kan, Jan Gasthaus, and Yuyang Wang. Learning quantile functions without quantile crossing for distribution-free time series forecasting. In Gustau Hamps-Valls,, Francisco J. R. Ruiz, and Isabel Valera, editors, Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, vol- ume 151 of Proceedings of Machine Learning Research, pages 8127-8150. PMLR, 28-30 Mar 2022. URL htttttttt///ccceeeeeee
mlr press/ / v151/park22a. html
[80] Sri Devi Ravana and Alistair Moffat. Exploring evaluation metrics: Gmap versus map. In Proceedings the 31st Annual International ACM SIGIR Conference Research and Devel- opment in Information Retrieval, SIGIR '08, page 687-688, New York, NY, USA, 2008. As- sociation for Computing Machinery. ISBN 978160558 1644. doi: 10. N0.11455/39003341399452 URL htt.ttttt///:i....oo org/ 1145/1399334 1390452
[81] Benjamin Beach, Robert Burlacu, Andreas Bärmann, Lukas Hager, and Robert Hildebrand. Enhancements of discretization approaches for non-convex mixed-integer quadratically con- strained quadratic programming: part ii. Computational Optimization and Applications, 87(3):893-934, Apr 2024. ISSN 1573-2894 doi: 10. 1007/s10669--044-05554, URL https://doi.org:
1007 000007/11589-02--100 1007//00099-00240055--
[82] Shi Bin Hoo, Samuel Müller, David Salinas, and Frank Hutter. The tabular foundation model tabpfn outperforms specialized time series forecasting models based on simple features. arXiv preprint arXiv:2501. 020044555.0 2025, doi: 10.48550/arXi.2500 2002455
[83] Bias correction in weighted variance. 2012. Accessed: 2025-04-09.
stttss..t.s..... siiiaatssstaccccccangeeeee com.aaaaaeecccaaaaaaaaaaa/aaeaae a/61641
[84] Romain Ilbert, Ambroise Odonnat, Vasilii Feofanov, Aladin Virmaux, Giuseppe Paolo, Themis Palpanas, and levgen Redko. SAMformer: Unlocking the potential of transformers in time series forecasting with sharpness-aware minimization and channel-wise attention. In Forty- first International Conference on Machine Learning, 2024. URL https:// httpp:///pperrevier openreview. hett/forum71d-8kLLL..B0
[85] David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. Deepar: Proba- bilistic forecasting with autoregressive recurrent networks. International Journal of Fore- casting, 36:1181-1199 2020. ISSN 0169-2070. doi:
2019. 00w.07.000. 001. URL https: : www. sciencedirect. com www.inceeeaaiceie.ccoo
$0169207019301888
[86] Abhimanyu Das, Weihao Kong, Andrew Leach, Shaan K Mathur, Rajat Sen, and Rose Yu. Long-term forecasting with tiDE: Time-series dense encoder. Transactions on Machine Learn- ing Research, 2023. ISSN 2835-8856. URL https::/openteee opwnreeieeeeeeeeee// wwt./ee//fooooo ftr/formm id=
pCbC3aQB5W]
[87] Alexander Alexandrov, Konstantinos Benidis, Michael
Valentin Flunkert, Jan Gasthaus, Tim Januschowski, Danielle C. Maddix, Syama Rangapuram, David Salinas, Jasper Schulz, Lorenzo Stella, Ali Caner Türkmen, and Yuyang Wang. Gluon'TS: Probabilistic and Neural Time Series Modeling in Python. Journal of Machine Learning Research, 21(116):
1-6, 2020.
html
[88] Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: next-generation hyperparameter optimization framework. CoRR, abs/1907. 10902, 2019.
URL. http: http:/tpp//.aaa..o../aa///////// 10902
16

---

[89] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International
Conference on Learning Representations, 2019. URL https://openrevieww net/forum?
1d=Bkg6RiCqy7
[90]
Shengding Hu, Yuge Tu, Xu Han, Chaoqun He, Ganqu Cui, Xiang Long, Zhi Zheng, Yewei Fang, Yuxiang Huang. Weilin Zhao, Xinrong Zhang, Zheng Leng Thai, Kaihuo Zhang, Chongyi Wang, Yuan Yao, Chenyang Zhao, Jie Zhou, Jie Cai, Zhongwu Zhai, Ning Ding, Chao Jia, Guoyang Zeng, Dahai Li, Zhiyuan Liu, and Maosong Sun. Minicpm: Unveil- ing the potential of small language models with scalable training strategies, 2024. URL
https://arxiv.crm org/ abs/2404.0633
[91] Benjamin Lefaudeux, Francisco Massa, Diana Liskovich, Wenhan Xiong, Vittorio Caggiano, Sean Naren, Min Xu, Jieru Hu, Marta Tintore, Susan Zhang, Patrick Labatut, Daniel Haziza, Luca Wehrstedt, Jeremy Reizenstein, and Grigory Sizov. xformers: A modular and hack- able transformer modelling library. http:://ggttooocccaaffaeeeeeree 2022.
[92] Jay Shah, Ganesh Bikshandi, Ying Zhang, Vijay Thakkar, Pradeep Ramani, and Tri Dao. Flashattention-3: Fast and accurate attention with asynchrony and low-precision, 2024. URL.
https://arxiv.. org/ abs/ 2407 08608
[93] Karl Pearson and. Liii. lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 2(11); 559-572, 1901. doi: 10. 10..1800/447 11880//17774401944677220 URL stttps://do: http://ooo...////...// 1080/
14786440109462720
[94] Philip J. Fleming and John J. Wallace. How not to lie with statistics: the correct way to summarize benchmark results. Commun. ACM, 29:218-221, 1986. URL /ttpp://appt
semanticscholar: org/ (CorposII::1173300
[95] Ming Jin, Shiyu Wang. Lintao Ma, Zhixuan Chu, James Y. Zhang, Xiaoming Shi, Pin- Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, and Qingsong Wen. Time-Ilm: Time series forecasting by reprogramming large language models, 2024. URL hnttps://arxiv. /rrg/ 0ts//3300001228
[96] Tian Zhou, PeiSong Niu, Xue Wang. Liang Sun, and Rong Jin. One fits power general time series analysis by pretrained Im, 2023. https:/:::///ntt //t::i///axiiii/a org/ abs 2a0s./232...2 11939
[97] Musleh Alharthi and Ausif Mahmood. xlstmtime Long-term time series forecasting with xlstm, 2024. URL
10240
[98] Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. In International Conference on Learning Representations, 2023.
17

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims in the abstract and introduction are (1) introduction of a new time-series foundation model, Toto, (2) introduction new benchmark focused on ob- servability data, BOOM, (3) SOTA performance of ToTO compared to other established time- seme-series baselines the BOOM, GIFT-Eval, and LSF benchmarks. These are directly sup- ported by the exposition and empirical results in the Section Section 4 and Section respectively.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss limitations of the study in the Section 6
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The au- thors should reflect on how these assumptions might be violated practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the ap- proach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-tex system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to ad- dress problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed penalize honesty concerning limitations.
3. Theory assumptions and proofs
18

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: N/A
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theo- rems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be comple- mented by formal proofs provided appendix or supplemental material. Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclu- sions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide full details used for evaluation in Appendix C.2 Appendix D.2 and Appendix D.3
Guidelines:
The answer NA means that the paper does not include experiments.
the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate the research performed.
While NeurIPS does not require releasing code, the conference does require all sub- missions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results way produce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case thors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
19

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: Toro model weights and inference code, as well as, BOOM data and evalua- tion code are released under the Apache 2.0 license.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
//nips. cct.ccccppppoo pte/puulic/ae wwtlcccgggse//eeee Setes/Cooeuubbiii watteeSummsssiooooo ssionsolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: (Ys]]
Justification: We describe all hyperparameters used during training and inference. We describe the training dataset at high level, but do release is proprietary. The procedure for choosing hyperparamters is described in detail (see Appendix (A.6).
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should presented in the core the paper to level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropri- ate information about the statistical significance of the experiments?
Answer:
[No]
Justification: We adopt the same procedure as other time-series foundation model studies when evaluating ToTO and baselines the BOOM and GiftEval benchmarks, which do not include computing error bars. Running several independent trials of each experiment (i.e. retraining the foundation model for each trial) compute empirical standard deviation would add significantly to the cost of the study. Instead, we report results over large bench- marks, with metrics that measure the quality of both distributional and point estimates of the models.
20

---

Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should prefer- ably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out of range (c.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We mention the computing hardware used for study in Appendix A.6
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't t make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: Data release All released BOOM data contains only numerical time series data and categorical labels and contains no PII.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethies.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
21

---

Answer: [Yes]
Justification: Yes, there is a impact statement included in Appendix
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact spe- cific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitiga- tion strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: As described in our broader impact statement, we believe time series models (and de-identified time series data) pose relatively low risk.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by re- quiring that users adhere usage guidelines restrictions access the model implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All relevant licenses for open-source code used by our project are included in our open-source release.
Guidelines:
The answer NA means that the paper does not existing assets.
22

---

The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name the license (c.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the pack- age should be provided. For popular datasets, shiperswithhode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the li- cense of dataset.
For existing datasets that re- re-packaged, both the original license and the license of the derived asset (if has changed) should provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documenta- tion provided alongside the assets?
Answer: [Yes]
Justification: A new open-weight, model and evaluation benchmark are released, each with their own documentation, including Model Card and Data Card, respectively.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the pa- per include the full text instructions given participants and sereenshots, applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: N/A
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contri- bution of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, cura- tion, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: N/A
23

---

Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research
with human subjects.
Depending on the country in which research conducted, IRB approval (or equiva-
lent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: We describe the use of LLM to conduct initial application taxonomy labeling of each time series in the BOOM benchmark in Appendix 4.2 Each label was then verified by human domain expert.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non standard components.
Please refer to our LLM policy for what should or should not be described.
24

---

A Model architecture details
A.1 Input/output scaling
To ensure numerical stability, we compute the çi and š, using an efficient vectorized implemen- tation (Listing [ of Welford's online algorithm [46), incorporating Bessel's correction provide an unbiased estimator of variance, as described in Option A of [83]. We stabilize training against extreme outliers by incorporating weak information from the global statistics.
def caaptee_ccusalsstttttt
data: torch. Tensor, weights: torch. Tensor, minimum_acall float,
-> [torch Tenor,, torch. Tensor]:
# Compute causal means at each time step reighted_data weights data cem_ttights torch. cumsum cumsun (weights, dim=-1) cum_values torch cumsum (veighted_ dim=-1)
10
denominator cum cum_veighte clamp (1..0)
11
causal_means cum_values / denominator
For Welford's algorithm, delta using the difference between the current value and the previous running mean.
need compute the correction term
14
15
16
shifted_means torch. tctos_ shifted_means ..... 1:] causal_ means [eans..... :-1] delta data shifted_mean means
18
19
20
# Compute m_2, the second moment accumulator for Welford's algorithm.
increment delta (data causal_means) weights m_2 torch. cumsum (increment, dim=-1)
23
24
25
# Compute the variance using Bessel's 's correction. causal_variance = m_2 / torch clamp (denominator 1.0, min-1.0) causal_scale torch. sqrt (cwasaallaaranneeeee
27
iinimum_scales
28
29
return causal_meane means, lausal_scals
Listing 1: Vectorized PyTorch implementation of Welford's algorithm for computing causal statistics
In our ablation study (Section we find that causal scaling leads dramatic performance im- provements over naive global scaling.
A.2 Attention mechanism
To address the unique challenges of time series data, and particularly adapt transformer architec- tures for multivariate time-series forecasting, several works have implemented modifications to the attention mechanism. These strategies have included:
Concatenating variates along the time dimension and computing full self-attention between every variate/time location, as in the "any-variate attention" used by Woo et al. 13] This capture every possible variate and time interaction, but it is costly in terms of compu- tation and memory usage.
Assuming variate independence, and computing attention only in the time dimension as in Nie et al. [19]. Shi et al. [14]. This is efficient, but throws away all information about variate-wist wise interactions.
Computing attention only in the variate dimension, and using a feed-forward network in the time dimension 84 35)
Computing "factorized attention." where each transformer block contains a separate variate and time attention computation 47 This allows both variate and time mixing, and is
25

---

more efficient than full aross-attention. However, it doubles the effective depth of the network.
In Section 3.1] we propose a novel approach that allows for both variate and time interactions, while reducing the computational cost and improving overall scalability.
A.2.1 Complexity analysis
After the patchwise embedding layer, we have inputs of shape X E RBXMxXXD, where B is the batch dimension, is the number variates per batch item, fo is time steps divided by patch width, and D is the model embedding dimension.
Tise-wise attention. We parallelize along the time dimension by reshaping the input tensor from 4 dimensions to 3:
E BXXMXXXD Xime E (B f xD
This allows for attention to be calculated independently parallel per variate, giving a complexity of:
O(M 1 D)
In the time-wise attention blocks, causal masking and rotary positional embeddings 43 with XPOS [44) in order to autoregressively model time-dependent features.
Variate-wise attention. input tensor:
We similarly parallelize along the variate dimension by reshaping the
X E RBXXMXXXX
X variate (Bx
We calculate attention in parallel for each time step, with complexity:
of
- x2
M x D
In the variate-wise blocks, full bidirectional attention (without causal masking) in order to preserve permutation invariance of the covariates, with a block-diagonal ID mask to ensure that only related variates attend each other. This masking allows to pack multiple independent multivariate time series into the same batch, in order to improve training efficiency and reduce the amount of padding.
Computational complexity. Each transformer block in our model contains time-wise attention layers and 1 variate-wist layer. The complexity for full self-attention over N + 1 layers, where interactions can occur across all variates and sequence positions, would be complexity:
o (N ((N+1))M*
L P
x
x D
(1)
This reflects the quadratic dependence on both the sequence length & and the variate dimension M. with linear dependence on the embedding dimension D. However, by utilizing factorized attention. we can reduce the computational complexity of the attention calculation to:
(
1.
L D
N x M x
2 x P
L
D P
M
Nx P
+ M
26

---

We demonstrate that factorized variate-wise attention is asymptotically smaller in computational complexity than full self-attention (see Equation and Equation 22. When comparing model with full self-attention, we can assume N and D are fixed. Therefore:
L P
L M²
<0 <0
COMMx
(
1 2)
M
+
Max
P
which reduces to:
o
L M P) <
M
Thus, by factorizing attention into time-wise and variate-wise components, the computational com- plexity is reduced, especially for large numbers of variates M or long sequences f. making it more scalable than full self-attention.
A.3 Probabilistic prediction
Practitioners who rely time series forecasting typically prefer probabilistic predictions. A com- mon practice neural time series models is to use an output layer where the model regresses the parameters of probability distribution. This allows for prediction intervals be computed using Monte Carlo sampling (see Appendix A.4
85].
Common choices for an output layer are Normal 85 and Student-T 86 21,
which can improve robustness to outliers. Moirai [T3] allows for more flexible residual distributions by proposing a novel mixture model incorporating weighted combination of Gaussian, Student-T, Log-Normal, and Negative-Binomial outputs.
However, real-world time series can often have complex distributions that are challenging to fit, with outliers, heavy tails, extreme skew, and multimodality. In order to accommodate these scenarios, introduce an even more flexible output likelihood in Section 3.1 based on a SMM [55].
TOTO makes predictions using mixture of K Student-T distributions (where K is hyperparame- ter) for each time step, as well as a learned weighting. Formally, the SMM is defined by:
p(x)
mk T
ph, Tk, Vk)
where TReK are nonnegative mixing coefficients which sum to I for the kth Student's t-distribution Ty with Vy degrees of freedom, mean μæx and scale Tg. T(x μ. o, v) is defined as:
1 - Tx 10)
T(x (,T,2) I (g) (1z)d/2
(4)
where r(.) is the gamma function.
In ablation study (Appendix D. we find that the SMM improves both point prediction and probabilistic forecasting accuracy when compared with a single Student-T distribution as used in TiDE (86), Lag-Llama (21), and implementations of DeepAR [85. PatchTST (19), iTransformer 422. and others in the popular open-source GluonTS library [87.
The parameters of this mixture model are computed from the flattened features he E R² produced by the transformer backbone for each time step where D the model's embedding dimension. Using a set of linear projections with weight matrices W E RKxD and bias vectors e k we derive all K mixture components simultaneously. For each time step f, the parameters are computed
27

---

MULL MUFL HULL HUFL
tULL LUFL
OT
2016-07-11
2016-07-13
2016-07-15
2016-07-17
2016-07-19
2016-07-21
Figure 6: Example of ToTO 96-step zero-shot forecasts on the ETThl dataset, showing multivariate prob- abilistic predictions. Solid lines represent ground truth, dashed lines represent median point forecasts, and shaded regions represent 95% prediction intervals.
as:
14 = 2 + max(softplus(W..t + bu),
(5)
H4, W W + by
(6)
T; max(softplus(Whh, b)).c)
7. Wal,, + b
where each equation produces vector in I&K containing the parameters for all mixture components at time The individual component parameters Vt,k, Pt,k, Tr,k; and 7e,k (the mixture logits) are small positive constant, and softplus(x)
the kth elements of these vectors. The parameter log(1 + e"). The use of softplus and ensure that the scale t remains positive. Similarly, we add ensure that each component of mixture has well-defined first and second moments (mean and variance).
the constraint
The mixture weights ı are computed using by applying softmax to the logits:
Ti,k = sfftmax
(9)
Ej= ac.s
example distribution median and 95th percentile is illustrated in Fig.
A.4 Forecasting
When performing inference, we draw u (for some user specified integer 0) samples from the mixture distribution at each timestamp, then feed each sample back into the decoder for the next prediction, resulting in n identically and independently sampled time-series. This allows us produce prediction intervals any quantile, limited only by the number of samples. Our exact sampling procedure for several tasks is detailed in Section
A.5
Loss function
ToTO learns the conditional distribution [1s), where X₃ represents the i-th patch contain- ing multiple time steps.
The LNLL optimizes probabilistic predictions and is defined as:
K
(x,(x,,,TT)) = log (X.)) = - log
Te, T(xe Pe. (e
(10)
28

---

0
2=2
Loke
2
-20
-15
-10
10
Figure 7: Visualization of generalized robust loss for different values of a, with d fixed at 1. Changing à scales the horizontal axis.
where p(r,) x ) is the probability density of the ground truth Ia under the model's predicted mix- ture distribution conditioned on all previous patches. The parameters Te,ks Me, Tr,k, and V4,6 the mixture weights and Student-T parameters computed by the model for time step t.
For ground truth value in patch i + 1 and the mean prediction e, loss defined below [61]:
Elp(ar X1.)). the robust
[ - 21)
2 a
log (5((x; 2z) + 1 exp (- - -)
(11)
a = - oc
la-21 (1م 1- + a/2
otherwise
Here, CRobust(a,8) serves a point prediction error measure, where S 2 is shape parameter that controls the robustness to outlier observations (Fig. 7 and > 0 is a scale parameter that determines the size of the parabolic portion of the loss curve. This loss component directly penal- izes point prediction accuracy, and we conjecture this may help steer the mixture model away from degenerate solutions of the type described in (60) In ablation study, we find that adding the bust loss component significantly improves point forecasting accuracy without hurting probabilistic predictions (Section D
is applied to each timestep in the target patch X and the total loss is aggregated across all timesteps during training. By combining the probabilistic LNLL loss with the robust point- prediction loss, we achieve both accurate distribution modeling and stable convergence, especially in domains with highly heterogeneous data characteristics. The hyperparameter ANLL controls the balance between these two loss components and is tuned empirically.
A.6 Hyperparameter optimization
To determine the optimal architecture and training configuration for Toto, we conducted an extensive hyperparameter sweep using Optuna [88) a Bayesian optimization framework. We employed the Tree-structured Parzen Estimator (TPE) algorithm to efficiently explore the high-dimensional search space.
Our optimization objective was to minimize the validation mean absolute error (MAE) on multi- step forecasting tasks on a random validation split of the observability portion of the pretraining data. We train the model using the AdamW optimizer [89] with WSD learning rate scheduler [90]. We performed this sweep for 133 iterations over ranges described in Table 5 each for 50,000 training steps.
29

---

Category
Values Ranges
Patch Size
(16, 32, 64)
Variate-wise Attention Frequency
Every (3, 4, 6, 12] layers
Variate-wise Layer First
[True, False]
T Components
[8, 16, 24, 32]
Loss Function
ANLL E [0.05,
Robust Loss Params
a E (-xx- -2,0,055 1.0), 8 E [0.1,3.0]
Warmup Steps
[0, 10,000]
Stable Ratio
[11,99]
Learning Rate
[10-$,5 10-3]
Weight Decay
[10-3,10
Synthetic Data Proportion
0..0,
Shuffling Type
[Normally Distributed, Adjacent, Random, None]
Normally Distributed Shuffling Standard Deviation
[115, 5000]
Shuffling Frequency
[0.0.033
Table 5: Summary of hyperparameter search space. "Stable Ratio defines the proportion of steps that are stable after the warmup phase of the WSD learning rate schedule.
The resulting hyperparameter configuration described in Toblanoo the best multistep (average of 96 and 192) MAE on the Datadog validation set.
Hyperparameter
Value
Embedding Dimension
768
MLP Dimension
3072
Layers
12
# Heads
12
Variates
32
Spacewise Layer Cadence
12
Patch Size
64
7 Mixture Model Components
24
Annealing Schedule
WSD
Optimizer
Adam
(81. (2)
(0.9579,
0.9581)
Weight Decay
0.0014
Initial Learning Rate
0.0005
Warmup Steps
6784
Stable Steps
112,255
Decay Steps
15,962
Batch Size
128
Total Train Steps
135,001
CRobust
0,0000
CRobost 6
0.1010
ANLL
0.5755
10
Table 6: Hyperparameters for Toto
In Section E. we perform an ablation study on the impact of various model components. We optimize speed and memory usage utilizing fused kernel implementations and memory efficient attention operations via xformers [91], (with the Ftteniott kernel 92
We ran all experiments, including hyperparameter tuning, final model training, and benchmark eval- uation on a GPU cluster consisting A.00s and H100s.
30

---

B
Training data preprocessing
B.1 Observability dataset
Observability metrics are retrieved from large-scale time series database using specialized query language supporting filters. group-bys, time aggregation, and various transformations and postpro- cessing functions (Fig. 88. We consider groups returned from the same query to be related variates in multivariate time series. After we retrieve the query results, we discard the query strings and group identifiers, keeping only the raw numeric data. As described in Section we source metrics defined by user-genn ser--geneatted queries. This excludes any customer data and is sourced solely from the internal users and telemetry.
1. Metric name
2. Filter(s)
sum:redis, stats. www.pacce.htttteeeeppeeeeeeeeeee app:"> by (ciuster_nane) 60)
Aggregation
Interval (seconde)
Aggregation
Group by key(s)
function
Space
aggragation
aggregation
Figure 8: Example metric query in the Datadog platform. The metric name (1) determines which metric is being queried. The filter clause (2) limits which contexts are queried, in this case restricting the query to apps in the prod environment. The space aggregation (3) indicates that the sum of the metric value should be returned for each unique value the group-by key(s), aggregated across other keys. time aggregation (4) indicates that metric values should be aggregated to the average for each 60-second interval. The query results will be a multivariate time series with 1-minute time steps, and with separate individual variates for each unique value of cluster name.
B.2 Public datasets
We train on a public dataset corpus, which exposes the model to diverse time se- ries behaviors across different domains and sampling frequencies. Our pre-training dataset incorporates a diverse collection of time series from the GIFT-Eval Pretrain col- lection (26) and non-overlapping Chronos datasets [Н]. These datasets include ercot, exchange_res r_ate, weatherbench_daiie dominick, mexico_ cit__citttttkteee...eeeee bikes, ushcn_ daily, and
beatheerbenchhhourlr weatheerbench_monthl.e
B.3 Synthetic data
We supplement our training with synthetic data to further improve model performance. Our syn- thetic dataset consists of procedurally generated time series using approach similar to TimesFM 12], as well as kernel_synth_ 1m from the Chronos dataset Synthetic data constitutes ap- proximately 33% of our training dataset.
We generate synthetic time series through the composition of components such as piecewise lin- ear trends, ARMA processes, sinusoidal seasonal patterns, and various residual distributions. Our procedural generation randomly combines multiple processes per variate to introduce diverse pat- terns. The generation includes creating base series with transformations, clipping extreme values, and rescaling to specified ranges.
These synthetic datasets help the model learn robust representations by providing examples with specific characteristics that might be underrepresented in real-world data.
31

---

Froquencies Distribttion
Variotes longth: Distribution
Cardinality Distribution
32.75
0,4N
100
Figure 9: Representative figure showing the metadata breakdown by variate in the dataset: (left) sampling frequency distribution bar heights show the number of variates with each frequency, while the percentages show the fraction of overall observations, (middle) series length distribution, and (right) number of variates distribution.
B.4 Preprocessing
To prepare the raw time series for training, we apply padding and masking techniques to align the ries lengths, making them divisible by the patch stride. This involves adding necessary left-padding both the time series data and the ID mask, ensuring compatibility with the model's requirements.
Next, various data augmentations are employed to enhance the dataset's robustness. We introduce random time offsets to prevent memorization caused by having series always align the same way with the patch grid. After concatenating the Datadog and public datasets for training, we also im- plement variate shuffling strategy to maintain diversity and representation. Specifically, domly combine variates from either Datadog, open source datasets (GIFT-Eval pretrain and Chronos datasets), and/or synthetic data with probability of 14%, thus creating new, diverse combinations of data points. We shuffle series with adjacent indices (batched by 32 variates), favoring data points that were closer together in the original datasets. This approach improves the model's ability to generalize across different types of data effectively.
C BOOM
C.1
Domain taxonomy
BOOM data is collected using the same query language as described in Section B.1 Each met- ric query yields collection of time series-one per unique attribute combination resulting in multivariate time series with attributes serving as variates. The distribution of aggregation metrics collected is described in Table 7 In Table 8, we categorize the series into major groups based on the domain described within the query. As part of the labeling process, a large language model was used pre-fill labels from the metric names, which were then manually reviewed by human annotators to ensure consistency and accuracy.
Metric Type
Description
Proportion (%)
Gauge
Last measurement reported within intervals Number of event occurrences per second
65.7
Rate
26.8
Distribution
Aggregated statistical summaries across sources (e.g., average, percentiles)
5.3
Count
Total number of event occurrences within intervals
2.2
Table Taxonomy of metric types in the benchmark dataset with their relative proportions.
Each time series in the BOOM undergoes standardized preprocessing pipeline designed to ad- dress the noise, irregularities, and high cardinality typical observability data. First, missing inter- vals, common in telemetry due to irregular metric emission, are filled using metric-aware strategies: count-based metrics are zero-filled under the assumption that missing values reflect inactivity, while real-valued metrics (e.g., rates gauges) are linearly interpolated. Following imputation, series are sliced into fixed-length windows of up to 16,384 points. Unlike the training set, which uses ran- dom offsets and padding augment diversity, the benchmark slices extracted without offset or
32

---

System Domain
Description
Proportion (%)
Application Usage
Covers application interactions and user activity (e.g. request rates, API calls)
41.3
Infrastructure
System-level metrics (e.g., CPU usage, memory consumption)
34.4
Database
Focuses on database efficiency (e.g.. query lateney)
29.3
Networking
Encompasses network behavior, including handwidth usage latency
10.0
Security
Relates authentication, intrusion attempls, compliance checks
0.3
Table 8: TTxooomyy of system domains in the benchmark dataset with their relative proportions. A single time series can helong to multiple domains;
(a) Zero-inflated
(b) Highly dynamic
(c) Seasonal
zoom
Trending
(e) Stochastic
(f) Highly skewed
-1
(g) High SNR
(h) Irregular
(i) Complex
Mm0
skedle
dynamic seasonal
5809
10090
15899
8
5009
18900
15800
0
5800
10908
15998
Time (s)
Figure 10: Representative examples from the BOOM, illustrating the unique temporal patterns associated ob- servability data.
padding to ensure quality and comparability. To avoid data leakage in the provided validation split, only one slice is selected per metric, randomly sampled and similar for all the groups. For metrics with more than 100 variates (groups), we randomly subsample 100 to cap input dimensionality. Se- ries are then normalized using the first 90% split of points that will be used for the context windows. We further filter out variates exhibiting abnormal scale change in the test split (final 10% of points) while having constant values in the context window, as these result in degenerate cases that are im- possible to forecast meaningfully. This preprocessing results high-quality, variable-length slices that preserve the structural diversity and challenges of real-world observability time series.
The final benchmark comprises 350 million points across 2,807 metric queries. These series vary widely in sampling frequency, temporal length, and number of variates. Figure illustrates the distribution of series frequencies (left), lengths (middle), and cardinalities (right).
C.2 Evaluation protocol
C.2.1 Prediction terms and evaluation windows
Following the GIFT-Eval protocol, we assign a fixed prediction horizon to each time series based on its intrinsic sampling frequency. The specific mapping from frequency default horizon length is provided in Table This default value defines the short-term prediction task.
To define medium- and long-term prediction tasks, scale the short-term horizon by factors of 10 and 15, respectively. These extended horizons are only applied when they entirely within the
33

---

10.0
ROOM Giflbval
5.0
2.3
0.0
-2.54
-7.5
-2.0
9.8
2.0 Princtpal
4.8
6.8
8.8
Figure 11: 2D Principal Component Analysis 98 projection of normalized statistical features computed from the Datadog Benchmark and GIFT-Eval datascts. The clear separation between the two distributions highlights significant shift in underlying time series characteristics.
Frequency
Monthly (M) Weekly (W)
Daily (D)
Hourly (H)
Minutely (T)
Secondly (S)
Prediction Length (steps)
12
8
30
48
48
60
Table 9: Mapping from time series frequency to default short-term prediction length.
test window, defined as the final 10% of the time series. This strategy enables evaluation across increasingly challenging forecasting ranges.
For each prediction term, series are evaluated using rolling, non-overlapping window scheme, where each window has length equal to the corresponding prediction horizon. This ensures full coverage of the test split while avoiding overlapping forecasts.
The evaluation is hierarchical: if a series qualifies for long-term forecasting (i.c., has enough test points accommodate the longest horizon), it is also evaluated under the medium- and short-term settings. In total, this procedure yields 7,416 evaluation instances across the 2,807 time series in the benchmark. Each instance corresponds the average performance over multiple rolling windows within a series for a specific prediction term.
C.2.2 Evaluation metrics
Following GIFT-Eval, our two main metrics forecasting accuracy are Mean Absolute Scaled Error (MASE) and Continuous Ranked Probability Score (CRPS).
MASE point-forecasting score, is defined
MAEmodel
MASE
(12)
MAE seusonal naive in-sample
with the in in-sample Seasonal Naive MAE being defined as
MAE emsimnal naive in sample
Σ
Y, Y-m
(13)
T m
where T' is the length of the training split of the series, is the value of the series at time and m is the seasonal period (typically defined based upon lookup table according to the time series frequency).
CRPS 78 is scoring rule for probabilistic forecasts, defined with respect to an observation y
CRPS(D. = (Fo(z) - H(x - y))
(14)
34

---

where FD is the cumulative distribution function of the forecast distribution D and H is the Heav- iside step function. We follow the standard practice of taking the mean weighted quantile loss as discrete approximation, as described by Park et al. 779
As in GIFT-Eval, both MASE and CRPS further normalized by the performance of the Seasonal Naive forecast on the test split.
C.2.3 Inference procedures
To evaluate the comparison models BOOM, closely follow the evaluation methodology used in the GIFT-Eval implementation, For models not included in GIFT-Eval, we rely on their official implementations and recommended evaluation procedures. All foundation models are evaluated using a unified context length of 2048. This choice is informed by preliminary experiments showing that shorter context length (512) leads to general degradation in performance across models. Therefore, we opt for relatively large context window (2048) to preserve forecast quality, while ensuring feasibility available hardware.
To evaluate the zero-shot performance of other foundation models on BOOM, we follow the sampling procedures outlined in their respective manuscripts. For Chronos, generate 20 samples and use the median prediction as the point forecast. For Moirai, we generate 100 samples, again taking the median, and set the patch size to "auto". For ToTO generate 256 samples and take the median as the point forecast. TimesFM produces only point forecasts of the mean, which we use directly. In all cases, we compute CRPS with respect to the probabilistic samples and MASE with respect to the point forecast. Since Ti and Chronos support only univariate forecasting, we evaluate each variate independently. In contrast, both Moirai and ToTO support joint prediction over groups of related variates.
For the three statistical hyperparameter settings from the statsforecast package, with one exception: for AutoARIMA, we reduce maxa and maxp from 2 to I due to frequent numerical instability when d D 2. Following GIFT-Eval, we set the maximum input length for all statistical models to 1000.
AutoTheta, and AutoETS we use the default
C.2.4 Aggregation of results
Common practice for aggregating normalized benchmarking statistics is to use the geometric mean 194. We adopt this approach, with a slight caveat: due to the presence of constant subsequences in observability time series, it's possible for models to achieve zero error on handful of series in BOOM. As zeros cause geometric means to collapse, we instead use the shifted geometric mean, which is stable in such scenarios
[80 81) for aggregating MASE and CRPS. Specifically, for metric m across a set of test instances N, the shifted geometric mean FiShiiledGeoom is defined as
MShiitedeeom
exp
Σ PEN
log(mn + +6) +
(15)
[
where = 1c-5 is small stabilizing constant.
Evaluating forecasts on observability data using these standard evaluation metrics introduces several numerical instabilities which must also handle:
NaN values:
The benehmark includes small number of zero-inflated series, which can result in invalid CRPS values due to division-by-zero errors. To mitigate this, we apply a generic postprocess- ing strategy where NaN and infinite values are imputed using the mean CRPS across the remaining series.
Extremely low naive errors: Certain flat series exhibit extremely low even null in-sample MAE, leading to instability when normalizing by the seasonal naive MAE, with potentially exploding nor- malized values. Rather than discarding these special cases which are informative for evaluating how models handle anomalous flat behavior we isolate them into a separate data split. This split evaluated using simple MAE instead of MASE and non-normalized CRPS. To define this subset, we apply objective criterion: we include all series where either the MAE for the Seasonal Naive forecast on the test set or the in-sample Seasonal Naive MAE are zero. If any prediction window of any variate within query satisfies the above conditions, assign the entire query to the second,
35

---

Metrie
Moirabaua
Moiralaae
Moirah.rg
TimesFMan
Chronos-olltuu Chrono-Bolts.al
Timer
Time-Mof:
VisionTS
MAE
0.001
0.014
0.00
CRPS
0.025
0.009
0.003
0.005
0.091
0.022
0.019
0.005
0,005
0.009
Table 10: Performance of Toto and other zero-shot models on the BOOM dataset using the subset of near-
constant series.
low-variability set, where metrics are computed without normalization. For evaluation, results in this subset are not normalized and thus are aggregated using simple arithmetic mean.
The outcomes for this subset are reported Table I0 All the zero-shot models evaluated seem to handle these cases trivially; they all have extremely low MASE. Notably, ToTo's CRPS is slightly elevated relative to the other models. We observe that TOTO seems to produce overly wide prediction intervals in least some of these flat series, and conjecture that this may be due to the frequent presence of large anomalies in its pretraining data. This is an interesting avenue for further study, as this conservatism may in fact be desirable in downstream anomaly detection use cases where overconfident predictions can lead to false positive detections.
Taxonomy breakdowns. For the BOOM, we evaluate model performance across stratified groups defined by the dataset's taxonomy categories. For term (Table [2]. metric type (Table [3). and domain [Ta) we report the shifted geometric mean of the evaluation metric within each group. These results are discussed in detail Section D.I
C.3 BOOMLET
To facilitate research in settings where it is computationally infeasible to evaluate on the full BOOM benchmark, we construct a smaller, representative subset denoted as BOOMLET. This subset is created via uniform sampling over metric queries in BOOM, while additionally prioritizing queries with a relatively large number of variates to ensure sufficient training signal. This selection strategy preserves the distributional properties across key taxonomy dimensions, while also maintaining practical volume of training data per query. BOOMLET comprises 32 metric queries, encompassing 1,627 variates and approximately 23 million observation points.
D Results
D.1 BOOM
we present qualitative comparisons across three representative forecasting scenarios to highlight the behavioral differences between ToTo, Chronos, and Moirai. In the first example @. features a highly stochastic signal interwoven with complex seasonal components. While Moirai and Chronos models tend to overfit short-term fuccuattion--rssulting resulting in jagged forecasts and unstable confidence inter- vals- ToTO effectively identifies and extrapolates the latent scasonal structure, yiclding smoother, more co- herent trajectories and uncertainty bands that reflect deeper structural understanding of the series dynamics. Example K the target signal exhibits high dynamism with rapidly oscillating structure sustained amplitude modulations- posinng a challenge for long-range temporal modeling. While both Moirai Chronos models progressively lose phase alignment and dampen their amplitude estimates, Toto consistently maintains sharp, temporally aligned forecasts with well-calibrated uncertainty, accurately tracking the intricate periodic structure far into the forecast horizon. Finally, example @. the target series characterized by sparse, bursty impulses with high variance across events. Here, although ToTo mean prediction does not always precisely capture individual peaks, its predictive distribution faithfully mirrors the underlying spikiness the series, in stark contrast to Chronos, which collapses to an overconfident flat trajectory.
In Fig. 12
Table 11 reports the results for all versions and sizes of the zero-shot models.
CRPS
8.778
(1.43
0136
D44)
0.205
8451
03.34
01825
$375
1100
Hank
3,80
5921
533
MASE
8317
0138
6774
670
D761
B311
0321
0341
060
8746
0788
0338
8945
Hook
1.30
S211
5:00
636
67%
K311
11.50
11.29
1::
13509
6105
1.00
14113
Table 11: BOOM results. Full results across all models evaluated from Table Key: Best results.
Sccond-best results,
36

---

To better understand the capabilities and limitations of different forecasting models, conduct disaggregated evaluation across four major characteristics that describe time series in the BOOM dataset. This analysis enables us to probe how models respond to structural diversily real-world time series data.
Across all three categorical axes, the ToTO consistently achieves the lowest CRPS, with strong margins over all baselines.
Real Term
Metri
Teln
Mairalaa
Mainitaa
Mairalur
TiresFMa
Cheasen-lllle Hellow
Tleer
Tire-Mollew
Molli ory
VidueTS
Nakee
Eong
MASK
CRPS
0.434
0.188
11325
11482
3473
0.791
0.491
(1:W
[1811
0.522
M8155
0.528
0.510
11306
0.661
0.724
13380
0.094
0.556
1,300
1.500
Median
MASE
CRPS
0.186
0331
0.176
0.59
0.400
0.700
0.175
0.899
0.583
0.505
00382
0.00
0.611
0.25
0.99
0,00
0.656
1.000
Shor
CHPP
10399
0.70
0.389
1179
0.56
07385
01997
0.51
0,50
(1540
1.000
Table 12: Performance comparison of TOTO and other zero-shot models across different prediction ferms.
MASE and CRPS are normalized by the Seasonal Naive forecast and aggregated across tasks using the shifted geometric mean. Key: Best results, Second-best results.
1ype
Aatric
Teto
Mtdrabeatt
Matralaus
Mairakup
lime
Cheeno Hello
Chrmus-Sillee
Tierr
Timm-Milaa.
Vake15
Natve
Claar
M6ASE
CRPS
1.317
08667
0.814
0.130
0.755
0.355
0.77
6313
0.5m
0.919
0:m
0,53
0.580
0460
6963
0882
0851
0.68
1,055
0.00
1.220
1,500
Durshaliv
MASS
CRPS ;
11.58
1L382
144
[034)
1122
1152
0/44
0.73
0.68
0.48
0482
975M
0333
6446
0661
0J993
0171
0694
R177
0498
0874
1004
1.00
1800
CMPS
0.382
0.071
044
0/196
0.16
0400
046
0661
0694
0303
0.77
1.00
CMPS
11369
0/422
0351
0448
01463
Table 13: Performance comparison of TOTO and other zero-shot models across different metric types, MASE
and CRPS normalized by the Scasonal Naive forecast and aggregated across tasks using the shifted geomet- ric Key: Best results, Second-best results.
Domals
Matric
Tele
Meimique
Aberaker
Timee Mas
Chron-Hiae Hill
Chnon-Hiehue
Timer
Valerts
Natve
Applialie sage
MASE:
CRPS
0.639
0.440
03:17
02222
0.721
0.732
6.76
6718
0331
0.635
096
0.61
0381
1.03
(00)
Dasakor
MASE
CMPS
0,63
0.362
0.51
6449
0414
0.758
0.343
6444
6.66
0.44
0.761
0/61
0.57
0.89
0.16
9.166
0668
683
0.53
0M4
1.077
1000
MASE
8.58
0367
0850
0.30
6679
0668
0/6
0738
0791
680
036
(00
CHPS
0,391
0.46
0,46
0462
0.73
0310
0.06
Nowweloo
MASE
CRPS
0.65
0,400
0.35
0.484
0.766
0.73
6.66
0.06
0.79
A00
0.75
0371
085
0721
039
0533
6774
1065
100
Swaely
MASE
CRPS
0.882
0.476
0344
0.55
0.20
2.908
00335
0.53
6717
0/344
0.50
0.64
0.70
0376
0.28
0504
0.758
Table 14: Performance comparison of TOTO and other zero-shot models across different metric domains.
MASE and CRPS are normalized by the Scasonal Naive forecast and aggregated across tasks using the shifted geometric mean. Key: Best results, Second- results.
D.1.1 BOOMLET
We present results on the BOOMLET subset in Table 11
D.2 GIFT-Eval
To provide a comprehensive evaluation of Toto's forecasting capabilities, we benchmarked our model on the Gval-Eval benchmark (26). Gval is collection of diverse time series datasets that covers a wide range of domains and characteristics, including:
Various frequencies (hourly, daily, weekly, monthly, yearly)
Different domains (energy. traffic, retail, economics, etc.)
Varying series lengths and forecasting horizons
Single and multiple seasonality patterns
The benchmark evaluates models using multiple metrics, with particular emphasis on:
Mean Absolute Scaled Error (MASE): Measures point forecast accuracy relative to naive forecast
Continuous Ranked Probability Score (CRPS): Evaluates the quality of probabilistic forecasts
Overall Rank: Aggregates performance across all datasets
37

---

Figure 12: Example of 336-step zero-shot comparative forecasts on the Boom, showing multivariate probabilis- tie predictions. Solid lines represent ground truth, dashed lines represent median point forecasts, and shaded regions represent 95% prediction intervals.
For models other than ToTO, report the published numbers from the public leaderboard B For ToTo, the same inference settings described in Section C.2.3] with modification: GIFT- Eval does impose maximum context length, we select length of 4096 (the native context length used in ToTo's pretraining, as described in Table 6).
D.3 LSF
In addition primary evaluations, we also assess the model's performance the Long Sequence Forecast- ing (I.FF) benchmark Mitasets-ETThi. ETTh2, ETTml, ETTm2, Electricity, and Weather 22. As noted hy
38

---

Aksu et al. [.] these datasets are limited in size and diversity, and recent findings [29] suggest that strong super- vised baselines already perform the upper bound such benchmarks. This may indicate saturation point where further gains from foundation models are difficult observe, rather fundamental limitation of the models themselves. Nevertheless, as remains a widely used legacy benchmark in the literature, we report zero-shot results of ToTO on it to maintain consistency with established practices in the field.
Furthermore we leverage its small scale and constrained use-cases to examine ToTo's capacity to transfer to new datasets and specialized domains by conducting fine-tuning experiments on the training splits of its datasets.
Following standard practice for the LSF henchmark, report normalized Mean Absolute Error (MAE) and Mean Squared Error (MSE), in order to able to compare performance across different datasets. evaluate using forecast lengths of 96, 192, 336, and 720 time steps. Predictions are generated using sliding windows with a stride of 1. For the Electricity dataset, however, we use a stride equal to the prediction length to reduce computational resource requirements. The results then averaged. We compare ToTO's performance with results reported by recent -tatc-of-the-art time series foundation models, including Moirai [3]. VisionTS 24
TimesFM [2], Time-MoF [14) TimeLLM (95), GPT4TS (96), xLSe (7] and other models evaluated
in Woo et al. 1 and Das ct al. 12 We display zcro-shot and full-shot ToTO results in Table 4 and Table 15
respectively. We also provide additional per prediction length results in Table and Table 17
Table shows that ToTo consistently delivers the best overall performance across all datasets, achieving the lowest average MAE and MSE, and oulperforming other zero-shot baselines on 8 out of 12 evaluation metrics. Its performance especially strong ETTm2, Electricity, and Weather, where continues to excel even zero-shot scenarios.
39

---

Times reports saluee for MAE ETThi, ETTh2
0.400
0.400
0449
0.55
0.50
0.305
0.33
0.24
0.550
0.509
0.00
0.55
0
0.617
0.247
0.725
0.054
0.571
0.965
0.358
083655
0,292
0.510
0.555
0
0:432
0.436
0.515
0.599
0-481
0.305
0221
0.117
0.345
0.309
0.574
0:500
0:54
0.55
0661
13
1220
01320
0.271
U1:00
"Best Count" row reports the number of times each model attains the best result for a given dataset lataset-mettic metr-mettii pair.
0. 0.502
0.529
0. 08
0.942
0,41
0.30
05:15
02:59
0.:01
02:41
0.655
064
00407
00301
0.00
0.30%
0.31m
00-20
0.259
0,360
0.355
0:550
0.458
0.007
0.41
0.33
0.305
0795
0.30
0259
0,75%
0335
0.41%
0.54
0:407
0.385
0.312
0.20
0.17%
0.7%
0.555
0.35%
0.325
Table 15: Full Full-Shu Shot companson of models on the LSF benchmark, with TOrO ' Zero Zero-Shot Shot result in the first data lumnn
0-428
0:408
0. 1.595
0:346
0.30
0:250
0.557
0.255
0.22
0,335 0.389
Times
0,136
0.41
0.405
0.353
0.334
0.334
01:30 0,550
2
Tima- uoo
0.804
0.375
0.366
01361
0.322
1L312
1L234
0.23 0.34
2
0.10
0.37
0.366
0.346
0LYS
n.71
0.275 0.236
0.000
0,305
0,38
01356
0330
0.240
01150
01282
0.227
0,331
02550
GPT4TS
0.423
0.394
0.554
0.33
0.35
0.200
0.251
0.344
0.30
8
ETTml. Em after fine- -tnning
Key: Best results, Second results.
0-623
0.408
0,334
0.111
0.41
0.20
0.55
0:551
0.225
0.333
0.284
0.000
0:115
0.303
535
1L291
0.234
11 152
0.233
1.314
8
0-313
0:555
140
0.300
0.24
0.245
0
MAR
MISE
MAR
MAS
MAR
MAR
MAST
MAR
MANE
MEA
MISSE
Best Coust
ETTh
Weathrr
Mean
40

---

Furthermore, Table [3] shows that even when starting from strong SOTA baseline, Toro's performance im- proves with fine-tuning. showing can achieve full-shot SOTA results and adapt domains with limited data. This highlights ToTo's robustness and versatility as foundation model for wide range time-series forecasting tasks.
Full-shot results on LSF benchmarks We conduct fine-tur tuning experiments on Toto following similar proce- dure delineated by [98] and [13]. The full-shot results for each dataset, comparing fine-tuned and zero-shot performance, are reported in Table T5
Results Our experimental results demonstrate that when finetuned, denoted as TOTÖPT, achieves state- of datasets in the LSF benchmark--sccificalll ETTm2, Electricity, and
of-the-art performance Weather where it outperforms all other models on both MAE MSE metries. Additionally, TOTO,r achieves the best MAE score on ETTml and ETTh2, although it does not lead on MSE for those datasets. Compared to its shot counterpart, TOTO consistently improves both MAE and MSE metries across most datasets, with particularly notable gains in ETTml (MAE: 0.378 0.357, MSE: 0.396 0.349) and ETTm2 (MAE: 0.303 0,291, MSE: 0.267 0.244). Overall, TOTOpT ranks first 8 of 12 metric-dataset pairs, outperforming other models, including both zero-shot and full-shot baselines. Notably, also delivers the best overall performance on the benchmark, achieving the lowest average MAE (0.314) and MSE (0.284). These results underscore the effectiveness of fine-tuning in enhancing Toto's 's predictive performance, estab- lishing TOTOFT the SOTA model the LSF benchmark. In addition. this demonstrates that Toto is robust foundation model, adaptable a wide range of downstream datasets, including those from entirely new domains, making it a versatile choice for time-serics forecasting tasks.
whete
closer examination of the results reveals that while Totoy; achieves state-of -hht---rtt performance on most datascts, the effectiveness of fine-tuning varies across them. Fine-tuning proves especially beneficial on ETTml, ETTm2, and Weather, where it significantly enhances model predictions. In contrast, the improve- ments ETThl relatively small size these datasets. Moreover, even though fine-tuning generally improves performance over the original ToTo model, TOTOpr does not outperform other full-shot models on ETTh1.
modest, and for ETTh2, fine-tuning yields notable gains--otentiilll due to the
Additional details on zero-shot and full-shot results per prediction length are displayed in Table
41

---

Zero Shor
Dutaset
Prediction Length
Metric
Toto
Mairalsua
Moiralaaw
Moiraliarz
VisionTS
TIME-MoE
TIME-
TIME-MoEun
MSE
0.382
0.375
0.384
0.280
0.353
0.357
0.350
0.349
192
MAE
0408
0.419
0.429
0.432
0.410
0.411
0.412
0.413
ETINI
MSE
0.25
0.399
0.425
11441)
0.392
0.384
0. 1.588
0.395
336
MAE
0.422
0.429
0.450
0.476
0.421
0.434
0440
0.653
MSE
0-57
0.412
0.456
0.514
0.407
0.411
0411
0.447
720
MAE
0.480
0.464
0.473
0.568
0.511
0.477
0.455
0,962
MSE
0.472
0.413
0.470
0.705
0.406
0.419
0.427
0.457
96
MAE
0.310
0.334
0.327
0.325
0.325
0.359
0.354
0.352
MSE
0.773
0.281
0.217
11267
0.271
0.315
0.302
0.202
192
MAE
0.356
0.373
0.374
0.367
0.386
0.385
0,379
MSE
0.130
0.340
0. 340
0.347
0.325
0.351
0. 0.364
0.347
336
MAE
0.387
0.393
0.295
0.381
0.415
MSE
0.374
0.362
0.371
0.377
0,345
0.391
0.417
0,806
MAE
MAE
MSF
0.320
0.385
0410
0-426
0.177
MSE
0.485
0,490
0.434
0.432
0.416
0.504
0.475
0,069
MSE
0.173
0.205
0.195
0.189
0,228
0.201
0.197
0.198
192
MAE
0.280
0.118
0.303
11300
0.305
0.354
0.522
0.512
ETTm2
MSE
0.232
0,261
0.247
11247
0.202
0.258
0.250
0,235
336
MAF
0.120
0.355
0.333
0.3
0. 72A
0.375
0.335
0. 348
MSE
0.290
0.119
0.791
0.205
0.201
0.724
0.337
0.201
729
MAE
0.375
0.110
0.377
0.286
0,370
0.464
0.661
0,425
MSE
0.372
0.415
0.355
0.372
0.343
0.488
0.430
0.427
96
MAE
0.213
0.299
0.248
0.242
0.266
MSE
0.129
0.205
0.158
1.252
0.177
192
MAE
0.229
0.310
0.263
0.50
0,277
Flectricity
MSE
0.145
0,220
0.174
11
0.158
336
MAE
0.247
0.323
0.278
0.216
0.296
MSE
MAE
0.2N2
MAE
MSE
0.157
0.265
0.245
0.260
0.312
MSE
0.310
0,320
0.321
0.265
0.330
0.418
0.415
0,905
Best Count
29
Table 16: Shot Comparison of different models with ToTO on the LSF benchmark datasets for each prediction length. Non-ToTO values are reproduced from published tables. Key: Best results, Second-bess bbstt results. "Best Count" row reports the number of times each model attains the best result for given metric.
42

---

0.40
0.716
0-118
0.40
0.49
0.50
0.397
0.350
0.49
0.80
0.411
0.46
0.11
0.3%
0.441
0-46
0.45
0.40
0.145
0.208
0.728
0.200
0.10
0415
0.41
0.16
0.315
0.30
0.30
0.1
0.26
0.56
0217
0.2%
0.300
0.39
0.410
Table 17: Full- Shll-ShotCCa Comparison of different models with TOTO on the LSF benchmark datasets for cach prediction length, with TOTO's os Zero-Shot result in the first data column. Key: Best results, Second-best results. "Best Count" row reports the number of times each model attains the best result for a given metric.
0.39%
0164
061
01000
0.778
0.69
0.00
0,305
0.68
1000
0.538
1.30
0:014
01:50
0.39
0.000
0.500
1:94
0.56
0.545
0.39
0.96
0.20
0.35
0.355
0338
12
0.39
0.30
0.57
0.39
0.400
11:00
01:2
0.29
0.48
0516
0.19
0.30
033
0.26
042
85N
0005
083
0322
035
0:30
0.90
0251
0424
038
13354
025
0351
0.30
035
0.99
0.00
022
035
0.55
0.2
035
0.30
0.404
0.00
0.001
0.51
0.30
0.55
0.505
0.40
0:30
016
0.51
0.30
0.00
0.40
0:000
0.36
0.20
0500
0355
0.20
0.55
0.3%
0.5
0.38
0.30
0.33
0.300
06623
0.52
01124
0.56
0,50
0821
0.50
0.35
01066
1002
033
1.04
01235
0.08
0.55
0209
655
11996
0.00
0.314
0252
10:2
1:20
0.14
023
0357
0.50
0.50
0.5
020
0,55
0.22
0.413
0.41
0.501
0.688
0.33
0.30
0.40
03235
02655
0.43
03300
0359
0:305
0.01
0390
0220
11854
030
12211
03:1
0.00
0:002
0.23
0.50
0.09
0.05
0211
025
0.14
0127
0.5
0300
0.23
01412
01306
01:29
0.40
04491
0.500
0137%
0134
0-14
0/42
0:55
0.462
017.5
0.350
0.307
0.77
0410
(450)
0.300
11300
04005
0406
0.20
0.18
0.300
0.50
6230
00:72
0.20
0.306
02300
01405
01316
0-116
0.250
0-467
0.40
0,30
0,307
01400
0130
0:455
0.423
11700
0.25
01.59
0.37
0426
0.30
01200
0311
0.005
0412
0:30
0.25
0.062
0239
117700
0.25
0214
0.23
0.006
0.238
0.305
11.08
0-116
0.22
0.465
01441
03233
0.22
0.55
0323
0.53
0,590
03335
0220
030
0329
0.35
0211
01316
0.h64
0.259
1211
0350
0.361
02211
0333
0.00
0.29
0188
0.087
0.00
0.241
0331
0.395
0:024
0.035
0.45
0.305
0.00
0:57
0354
0:236
0.500
0.415
Fall
6.365
0.231
0.41
0.388
0-050
0.35
0.24
0.50
0411
0:300
0.53
0.256
0.541
8:301
0.38
0.007
0.45
0.30
0:221
0.33
0.04
0.255
0.305
0:154
0.307
0.502
0.311
0-000
0.300
6-4M
0.402
0.335
022N
0.373
0.182
0.433
0.457
0.735
0.250
0.305
0.32
0.455
0.4M
0.160
0.355
0.525
0.341
0.435
0.451
0.30
0:49
0.30
0.365
0.345
0.775
03385
0.412
0.500
03433
0.410
0.340
632%
0.51
9451
0.80
0.155
0.286
0.150
0.30
0.455
0.435
0.50
0.228
0.108
0,63
0.40
0.305
0.51
0.305
0.208
0.247
11.3%
0.30
11:00
11-03
0.541
01211
11300
0.50
0::30
0.990
11301
0,22
0.35
1.30
11:34
12255
033
0302
01392
0.3
01.41
0.358
0.
922
11142
01
0.36
0.36
03300
01.58
0111
0.40
0.688
01497
0333
01305
0.39
03523
014
0.00
11506
0222
03322
0353
0122
1117
0300
1122
035
0401
035
0228
0251
0.55
0.206
0.06
0212
0.00
0.386
0254
0.3022
0.961
115
0-47
0.40
0:487
0:402
0:334
0.308
03311
0.304
0.43
0:772
0.334
0.223
0350
130
0:11
0.30
010
8.38
021
0:30
0:55
0352
0:2
030
0152
100
9M
0.30
147
110
0.5
300
Tolers
1.3344
944
X02
0:48
09435
0:440
109
0.222
08325
1-278
0:312
0:40
0:318
0.311
0:27
0:228
0.34
0:48
0.39
6112
0:300
0.3
0.34
0.224
0.41
22
0.1
16
0.57
0,353
02
Jenn Sten
0.311
0.3
0.57
0.440
04/2
0.310
0223
0.356
6.30
0.7%
0.50
0.375
0.31
0.300
0330
0.71
0255
635
0222
122
0355
0.72
0212
0.45
0247
0.00
0.70
7.70
0.40
200
0.25
0
MAR
MAR
MAB
1858
MAA
MAB
3858
MAR
1858
MAEE
3858
MAR
385
MAR
M48
MAB
355
MAR
MAB
355E
MAEE
3888
MAR
362
1838
MAAE
385
96
192
720
192
730
192
720
192
336
%
336
%6
356
ETTal
Elenniis
Weatber
Coun
43

---

D.4 Computational Efficiency
We have conducted an empirical study evaluate the computational efficiency of recent time series foundation models, which we present in Table [8] All experiments
performed single NVIDIA A100 (40 GB) GPU using synthetic multivariate time series. Each model was profiled under identical settings, length 2,048 and a prediction horizon 480 for variable number of variates. We include comparisons against both multivariate (Moirai-Ban Base) and univariate models (Time-MoE- Mo0-50ME, Chronos Eironos-Bolt--ie Base. and TimesFM- 500M).
context
Table 18: Model Benchmark Results
# Variates
Model
Wall Time (ms)
CUDA Time (ms)
Peak Memory (MB)
Total FLOPs (GFLOPs)
TOTO
652.8 = 12.6
37.0 0.0
721.0 + 0.0
121.6 0.0
TOTO cache)
585.0 7.9
45.8
733.6
Moirai
148.1 2.4
19.1 + 1.7
489.8 0.0
167.6 0.0
Time-MoE
1569.0 = 150.1
3.1.2
5341.2 0.0
3449.9 0.0
Chronos
831.6 25.2
9.07
925.5+00.0
2162.5
TimesFM
579.5 44.9
1.1
2023.6
5188.1 0.0
TOTO
627.2 = 10,4
43.8
9.4.910000
364.7 0,0
TOTO (no eache)
628.2 55.5
7 5.5
971.9
2635.5
204.7
4..4
1221.6
Time-Ma MoE
1924.7 +: 22.6
9.5
+503.33
10349.8 0.0
Chronos
925.8 = 22.1
179.7
1135.4 0.0
6487.5 0.0
Timesf'M
698.6 18.1
280.3
2200.9
15564.3
TOTO
687.5 = 72.6
50.7 29
1148.1 0.0
6178
TOTO (no KV cache)
627.1 + 55.4
99.9 7.7
1209.3 0.0
4425.8 0.0
Moirai
303.1 23,6
186.4 1.5
2.42.0
1302.2
50
Time-MoE
3239.9 252.8
1572.0
2.730.3
17249.6 0.0
Chronos
1029.6 14.8
266.5 + 18.2
1335.4 0.5
10812.5 + 0.0
TimesFM
662.0 9.2
441 11.6
2383.2 0.0
25940.5 : 0.0
TOTO
12.4
850.9
TOTO (no KV cache)
669.8 = 55,9
126.5
1447,4 0.0
6196.1 # 0.0
Moirai
454.8 = 16.1
3.6.0
4763. 0.0
2148.0 1 0.0
70
Time-MoB:
3464.1 168.3
30394.6 423.5
37882.8
Chronos
1193.4 44.0
3 10.7
1543.3
15137.6 +: 0.0
TimesFM
842.0 + 15.2
606.6 7.7
2555.2 0.0
36316.6 + 0.0
TOTO
712.1 48.6
66.9
1375.9
1094.0
TOTO cache)
714.8 91.3
156.8
1685.5 0.0
7966.5 0.0
Moirai
667.2
562.1 1.0
7577.0 0.0
3179.4 4 0.0
Time-MoEE
8985.5 729.0
3315.5 157.5
36581.7 2532.6
36010.0 1572.7
1209.9
427.9
1750.2
TimesFM
1124.7 36.1
786.3 8.5
2727.2 = 0.0
46692.8 + 0.0
TOTO
708.3 1::::::00 8.8
89.1 0.8
2223.2 123:22000
1823.3 = 0.0
TOTO (no cache)
714.6 31.5
5.2
2406.4
13277.4
Moirai
1830.3 122.1
1488.5
20193.6
7387.2 0.0
Time-MoB
11903.6 677.6
+
56227.9 3128.8
Chronos
1435.2
670.4
2375.2
32437.6
TimesFM
1699.8
1239.3
3256.4 0.0
77821.4 0.0
TOTO
710.2 16.1
100,9 4.7
2757.5
2431. 0.0
TOTO (no KV cache)
747.7 + 71.8
340.7 +
2998.2 0.0
17703.2 + 0.0
Moirai
3097.1
2316.1
35487.6
0.0
200
Time-MoE
15668.0 152.2
7828.7 3.0
30352.4
86490.8 0.0
Chronos
1799.4 23.9
890.3 7.0
2897.4 0.0
43250,2 0.0
TimesFM
2284.4
1665.3 3.7
3698.9 1
103761.8
TOTO
775.2 # 15.4
132.8 3.6
3821.7 = 0.0
3646.7 + 0.0
TOTO (no KV cache)
805.1 = 25.3
505.8 +: 5.3
4187.8 0.0
26554.9 + 0.0
Moirai
OOM 16590.2 254.8
0OM
0OM
0OM 108459.7
300
Time-MoE
9698.9
31517.0 0.0
Chronos
2583.3 + 23.6
1290.2 4.5
3935,1 0.0
64875.2 0.0
Timest'M
3302.4 +: 22.0
2445.4 5.9
4569.9 0.0
155642.8 0.0
To isolate the fundamental computational cost of each architecture, we evaluate simple forward passes generat- ing single output sample. This configuration reflects the core operation dominating training and fine-tuning, where repeated forward evaluations occur scale, and thus provides standardized measurement training efficiency. Many models, including ToTo, increase the number samples at inference time to boost predictive accuracy; this analysis does not address such test-time scaling.
Toro demonstrates consistently strong computational efficiency and scalability compared to other models, with the lowest FLOPs across all variate counts. It achieves the lowest second-lowest wall times across all variate counts, maintaining stable performance even the input dimensionality increases from 10 300 variates. Starting from 150 variates onward, TOTO outperforms Moirai all metries, including wall time and CUDA time; the gap grows significantly with more variates. Even against univariate competitors, where cross-variat
44

---

interaction is ignored, ToTo provides better performance, demonstrating its highly optimized architecture and efficient memory We note that ToTo KV eaching. while other models do appear to implement this. For additional transparency, we also share ToTo statistics with caching turned off and note that general trend remains the same.
E
Ablations
evaluate contribution various architectural components of the TOTO model by systematically dis- abling one component at a time and measuring the relative performance degradation. The full Toto model serves as the control, and each variant's performance is presented relative to this baseline in Table [9 All models in the ablation study, including the control, were trained for 75,000 steps (a subset of the full-length training of the ToTo base model).
Model
Best NLL Loss (% increase) +
Control
0.0%
No Variate-wise Attention
1.6%
No Robust Loss
11.1%
No Student-T Mixture
27.2%
No Causal Scaling
27.3%
Table 19: Relative change in NLL on held-out observabiiiy pretraining data when removing key design features of the ToTO architecture.
To compare performance between the different arms of the experiment, we look at NLL loss on a held-our validation split of the observability portion of the pretraining data. This summarizes the output distribution and gives single performance metric to compare both point forecasting and probabilistic forecasting. For each model, we pick the checkpoint with lowest NI.I throughout the training run (evaluating on the validation set every 5,000 steps).
The results reveal that removing key modeling elements significantly impacts performance. Disabling Causal Scaling leads to the largest degradation, with an increase of 27.3% in NLL when we replace the causal scaler with naive global scaler. Replacing the Student-T mixture model with a single Student-T output causes similar NLL increase of 27.2%. Interestingly, removing the robust loss component and optimizing NLL alone actually leads worse overall NLL, with 11.1% increase: we speculate this is because the robust loss stabilizes the training, as discussed in Section 3. Finally, removing the variate-witg wise attention (i.c. making all the attention layers tine-wise while holding the parameter count constant) leads to more modest increase in NLL of 1.6%.
Impact statement
In developing ToTo, we followed structured approach to ensure responsible development, focusing on iden- tifying, assessing. and mitigating potential risks associated with the of model. Given that ToTO specif- ically generates time series forccasts, the potential harms considerably lower compared to language, image, or other more -prrpoee models. Our primary focus was ensuring the accuracy and reliability the forecasts generated by ToTo, which are crucial for maintaining and optimizing infrastructure and application performance.
The BOOM benchmark provides numerical time series generated from observability metrics, which we view valuable resource the broader time series research community. Each series has associate high-level application label and no other metadata and contains PII.
45


Reviews:
[{"review_id": "zqs3UNgitM", "reviewer": "Reviewer_cYnS", "summary": "This manuscript introduces Toto, a 151‑million‑parameter, decoder‑only transformer tailored for multivariate time‑series forecasting, with a focus on observability data (e.g., telemetry). Toto is trained on a large, diverse corpus—4–10× larger than prior time‑series models—combining proprietary observability data, public datasets, and synthetic time‑series. To benchmark Toto, the authors construct BOOM, a comprehensive dataset composed of 2,807 real-world time series and totaling 350 million observations. They evaluate Toto against both BOOM and established benchmarks, demonstrating state‑of‑the‑art performance in forecasting accuracy.", "questions": "- Please explain the data split in BOOM, given the heterogeneity of the data how the split was handled? And does the split happen at data point level or query level?\n\n- How do you ensure the quality of the labels generated from the LLM and verified by humans is consistent? Please elaborate on the labeling approach, and if any inter-annotator agreement (for multiple humans) has been deployed.", "limitations": "Yes.", "rating": 5, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "The paper bridges the gap between foundation models and time‑series data by leveraging scale and architectural tuning. \nToto captures nuanced temporal patterns present in observability workloads by being trained on a massive mixed corpus - something that smaller, domain‑limited models often miss. The decoder‑only transformer design, alongside task‑specific innovations, shows a specific adaptation approach in the model design instead of a generic application of NLP architectures to time series. The construction of BOOM dataset is a major contribution. BOOM is a benchmark based on real-world data (not synthetic) on a scale rarely seen in forecasting research. By using over 2,800 traces and rigorous evaluation, the authors offer both transparency and a meaningful metric for future models. Authors also report comparisons with other models on three datasets (including BOOM) and share zero-shot and full-shot results. Toto consistently  beats other models and generalizes well to existing forecasting datasets, suggesting robustness and broader applicability.", "was_revised": true, "final_justification": "Authors responded to my questions, and my score is now final.", "rebuttal": "Thank you for the insightful comments. Below are the answers to your questions:\n\n> Please explain the data split in BOOM, given the heterogeneity of the data how the split was handled? And does the split happen at data point level or query level?\n\nWe describe the evaluation setting for BOOM, including details of the train/test split, in Appendix C.2 in the supplemental materials. To summarize, we adopt the standard practice within the forecasting literature of splitting each time series along the time dimension. Following the same setting used by GIFT-Eval \\[1\\], we designate the first 90% of each series as train and the last 10% as test. We do this split per query, meaning that each variate within a multivariate time series has the train/test split at the same timestamp.\n\nAdditional information regarding data preprocessing and quality controls for BOOM is provided in Appendix C.1.\n\n> How do you ensure the quality of the labels generated from the LLM and verified by humans is consistent? Please elaborate on the labeling approach, and if any inter-annotator agreement (for multiple humans) has been deployed.\n\nLLM labeling was used to identify the subdomains (“Application”, “Network”, etc.) of the time series in the BOOM benchmark. This labeling does not impact the main results in Table 2, but rather only affects the taxonomy-level statistics provided in Figure 4 and the breakdowns in Tables 8 and 14 in the appendices. To clarify, the labeling followed a two-step protocol:\n\n1. LLM Pre-labeling: system domains were initially labeled by a large language model  \n2. Human Review: The labels were then manually reviewed by multiple experts with domain knowledge. Each expert received a disjoint subset of the dataset to review. The experts consulted each other when they felt a particular example was ambiguous.\n\n### **Please do let us know if there are any remaining concerns that prevent you from increasing your score.**\n\n\\[1\\] Gift-eval: A benchmark for general time series forecasting model evaluation. arXiv preprint arXiv:2410.10393, 2024"}, {"review_id": "PjAbAfxkNE", "reviewer": "Reviewer_pfs4", "summary": "This paper presents TOTO, a decoder-only foundation model tailored for time series forecasting in the domain of observability metrics. The authors also introduce BOOM, a large-scale benchmark consisting of 350 million observations across 2,807 real-world multivariate time series collected from production monitoring systems. TOTO incorporates several architecture innovations—including causal patch-based normalization, proportional factorized attention, and a Student-T mixture model head—specifically designed to address the challenges posed by observability data such as high dimensionality, nonstationarity, and heavy-tailed distributions.", "questions": "see weaknesses", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n- The paper is clearly written and logically structured, making it easy to follow the motivation, methodology, and experimental results. \n- Substantial contributions: The work introduces two major contributions to the time series forecasting community\n- Strong empirical results: TOTO achieves state-of-the-art performance on three benchmarks under zero-shot settings, and demonstrates competitive performance even in full-shot scenarios.\n- Reproducibility and openness: The authors provide open access to the model weights, training and inference code, as well as the BOOM datasets.\n\nWeaknesses\n- Potential overgeneralization in scope: While the paper presents strong contributions in the domain of observability data, the title and framing suggest broader applicability to general time series forecasting.Since both the model design and benchmark (BOOM) are heavily tailored to observability workloads, it may be somewhat overstated to position this work as a general-purpose time series foundation model without more extensive validation beyond this domain.\n- Although the authors state that there is no data leakage between the TOTO pretraining corpus and the BOOM benchmark due to environment isolation, both datasets are sourced from the same commercial observability platform. This raises concerns about distributional similarity. Since TOTO is pre-trained on large-scale observability data while competing foundation models are generally trained on more diverse or generic time series corpora, the performance advantage on BOOM may partially reflect domain alignment rather than purely architectural superiority.", "was_revised": true, "final_justification": "I appreciate the clarifications and additional empirical evidence provided regarding the generalization capability of the Toto model beyond the observability domain. The results on GIFT-Eval and LSF, as well as the efforts to address potential data distribution concerns, have addressed my main concerns. I have decided to increase my score.", "rebuttal": "Thank you for your thoughtful comments!\n\n> Potential overgeneralization in scope: While the paper presents strong contributions in the domain of observability data, the title and framing suggest broader applicability to general time series forecasting. Since both the model design and benchmark (BOOM) are heavily tailored to observability workloads, it may be somewhat overstated to position this work as a general-purpose time series foundation model without more extensive validation beyond this domain.\n\nYou correctly point out that the two main contributions of the paper, Toto and BOOM, are motivated by applications within the observability domain. And indeed, the architectural choices of Toto were designed with performance on observability data in mind. \n\nHowever, we *do* find strong empirical evidence that Toto generalizes well to other domains. To demonstrate this, in addition to BOOM, we also provide evaluation results in Section 5 on GIFT-Eval \\[1\\] and LSF \\[2\\]. These are both widely accepted, general-purpose, multi-domain benchmarks that are not primarily focused on observability. They include many time series examples from non-observability domains, including finance, weather, energy, natural science, sales, and transportation. **We feel that Toto’s state-of-the-art performance on these benchmarks clearly demonstrates Toto’s applicability outside of observability.** \n\n> Although the authors state that there is no data leakage between the TOTO pretraining corpus and the BOOM benchmark due to environment isolation, both datasets are sourced from the same commercial observability platform. This raises concerns about distributional similarity. Since TOTO is pre-trained on large-scale observability data while competing foundation models are generally trained on more diverse or generic time series corpora, the performance advantage on BOOM may partially reflect domain alignment rather than purely architectural superiority.\n\nWhile we did put significant effort into ensuring no data overlap between our training and test data by sourcing them from isolated environments, we acknowledge that there likely is some distributional similarity. That being said, as mentioned above, Toto also achieves state-of-the-art performance on GIFT-Eval and LSF, two general-purpose benchmarks containing time series primarily from domains outside of observability. As we show in Section 4.3 (“Statistical Characteristics”), these benchmarks are distributionally dissimilar from BOOM, suggesting that Toto’s strong performance is not solely due to in-domain evaluation.\n\n### **Please do let us know if there are any remaining concerns that prevent you from increasing your score.**\n\n\\[1\\] Gift-eval: A benchmark for general time series forecasting model evaluation. arXiv preprint arXiv:2410.10393, 2024  \n\\[2\\] Informer: Beyond efficient transformer for long sequence time-series forecasting. AAAI, 2024"}, {"review_id": "gWsakEMGEW", "reviewer": "Reviewer_WAXQ", "summary": "This paper proposes a pre-trained model for time series forecasting in the observability domain. It further introduces BOOM, a large-scale benchmark of 350 million observations derived from observability time series, exceeding the scale of the current largest zero-shot benchmark, GIFT-Eval.", "questions": "1. What is the lookback window size used for the LSF benchmark?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. This work is valuable to the community by focusing on foundation models for observability time series and further contributes by open-sourcing the constructed dataset to support reproducibility and future research.  \n2. Extensive experiments conducted on the LSF, BOOM, and GIFT-Eval benchmarks demonstrate the strong generalization ability of TOTO across short-, medium-, and long-term forecasting horizons.\n\nWeaknesses:\n1. The technical novelty of this work appears to be limited. First, several proposed components—such as Proportional Factorized Attention and the Student-T Mixture Model—have already been employed in prior foundation models. For instance, introducing a fixed ratio to balance variate-wise and time-wise MHA within a Transformer decoder represents only incremental innovation. Similarly, the Student-T Mixture Model closely resembles the Student’s t-distribution used in Moirai [1], and the method does not present clear distinctions from existing probabilistic forecasting approaches. Second, these components seem broadly applicable to general time series data, and it remains very unclear how they are specifically tailored to the unique characteristics of observability time series.\n2. The ablation experiments and analyses of individual components are overly simplistic. Merely removing them is insufficient to fully demonstrate the critical role each plays in the overall architecture.\n3. A comprehensive efficiency comparison with existing foundation models such as Moirai and Time-MoE is lacking.\n\n[1] Woo G, Liu C, Kumar A, et al. Unified training of universal time series forecasting transformers. ICML, 2024.", "was_revised": true, "final_justification": "The concerns regarding certain aspects of novelty, ablation study, and efficiency have been addressed.", "rebuttal": "We thank the reviewer for their comments and provide clarifying discussion as well as new empirical support.\n\n## Novelty & Specialization to Observability\nFirst, there were concerns raised around the novelty of Toto’s main components as well as their specific applicability to observability time series. We respectfully contend that our use of the **Student-T Mixture Model (SMM)** and the **Hybrid NLL/Robust Loss** for time series foundation models is novel; furthermore, the **Causal Patch-based Instance Normalization** algorithm is entirely novel. And while factorized attention has prior art, our specific **Proportional Factorized Attention** design offers meaningful computational efficiency gains tailored to high-cardinality observability data. We’ve organized our discussion per component to clarify what we feel makes them both novel and important for observability.\n\n### Student-T Mixture Model (SMM)\n\n**Novelty and comparison to Moirai:** While other authors have used mixture models (e.g. Moirai), **to our knowledge Toto is the first time series foundation model to use a Student-T Mixture Model head.** This is a nontrivial improvement and generalization of the mixture model used in Moirai. As stated in their paper, Moirai uses a fixed mixture of four different distributions [1]. The Moirai authors suggest that these four individual components were chosen heuristically to match specific common cases within time series datasets.\n\nBy contrast, our model utilizes a mixture consisting of ***only*** Student-T distributions, where the number of components, $k$, is a hyperparameter that we optimize ($k=24$ in our selected model). This is fundamentally a more general and less heuristic approach. We derive our approach from first principles: mixtures of certain distributions (including Gaussians and Student-T’s) are universal approximators for ***any probability density function*** when using a sufficiently large number of mixture components [2, 3]. We choose the SMM (rather than GMM for example) based on its superior performance with heavy-tailed distributions [4].\n\nIn contrast to Moirai and other prior work, we also adopt a **novel loss formulation** which leads to significantly better convergence when training with our mixture model head. Alongside NLL, we jointly optimize the SMM with Robust Loss [4] to concentrate the mean towards accurate point predictions.\n\n**Significance for observability:** As we show in Section 4.3, observability time series have challenging and diverse characteristics. The SMM’s expressive power is more important the further away we get from simple, well-behaved distributions. We use SMM rather than a GMM since we find that its heavy tails make it significantly more robust to outliers during training. Finally, the Robust Loss was chosen (rather than L1 or L2 loss) precisely because the presence of outliers in the training data makes these less-robust losses unstable.\n\n### Causal patch-based instance normalization\n\n**Novelty:**  To improve generalization across varying input scales, instance normalization is commonly applied prior to embedding time series data. However, computing normalization statistics from the entire series would leak information from future time steps; it violates the model’s autoregressive causality and results in poor performance from overfitting. To the best of our knowledge, this is a completely novel approach, and it yields highly non-trivial improvements, as demonstrated in our ablation. \n\n**Significance for observability:**  Preserving causality is especially critical in data which is highly nonstationary, like observability data (due to frequent regime changes and anomalies). The more the distribution shifts over time, the more future information is leaked by global statistics, which hurts generalization performance. Our ablation (Figure 3B) on observability data shows that causal patch-based normalization is the single most impactful architectural improvement of Toto.\n\n### Proportional factorized attention\n\n**Novelty**: You correctly point out that factorized attention is not novel, and we do not make such a claim. While we agree that introducing an asymmetrical ratio of layers is a relatively simple idea, it nevertheless represents a meaningful improvement in computational efficiency of our model. Our asymmetric design is motivated by the observation that channel-independent models such as TimesFM [5] and Chronos [6] still achieve competitive performance on multivariate benchmarks and, therefore, we expect diminishing returns on FLOPs spent on variate-wise attention operations.\n\n**Significance for observability:** Optimizing the variate-mixing of our model is particularly important for observability data, which typically has higher cardinality than other datasets. For example, our BOOM benchmark has a median number of variates of 60 (vs. 1 for GIFT-Eval and LSF).\n\nWe will revise the discussion in our Related Works and Model Architecture sections  to make all of these points clearer.\n\n### Contribution of BOOM\n\nBeyond the technical contributions described above, we would like to re-emphasize the significant contribution of BOOM, the first large-scale, comprehensive forecasting benchmark within the observability domain. In terms of absolute scale, it contains nearly twice as many points as GIFT-Eval; and as we show in Section 4.3, its distributional characteristics are significantly different from those of GIFT-Eval and LSF.\n\n## Other feedback / new results\n\n> The ablation experiments and analyses of individual components are overly simplistic. Merely removing them is insufficient to fully demonstrate the critical role each plays in the overall architecture.\n\nWe chose a “leave-one-out” ablation design as this allows us to isolate each novel architecture component, while saving significant compute resources vs. a full grid search. This methodology is commonly used in recent foundation model literature; for example, similar leave-one-out ablations are used by Moirai [1], Time-MoE [7], UniTS [8], PowerPM [9], and others.\n\nFurthermore, we describe our hyperparameter selection procedure, which included an extensive sweep, in Appendix A.6 in the supplemental materials.\n\n> A comprehensive efficiency comparison with existing foundation models such as Moirai and Time-MoE is lacking.\n\nThank you for this feedback, we have conducted an empirical study of inference cost to help address this. We profile several models on synthetic data using a single A100-40GB, with a context length of 2,048 and a prediction length of 480, across 10, 50, 150, and 300 variates. We include comparisons against both multivariate (**Moirai-Base**) and univariate (**Time-MoE-50M**, **Chronos-Bolt-Base**, **TimesFM-500M**) models.\n\nToto consistently has the lowest FLOPs across all variate counts. Starting from 150 variates onward, Toto outperforms Moirai on all metrics; the gap grows significantly with more variates. Even against univariate competitors, where cross-variate interaction is ignored, Toto provides better performance, demonstrating its highly optimized architecture and efficient memory use. We note that Toto uses KV caching, while other models do not appear to implement this. For additional transparency, we also share Toto statistics with KV caching turned off and note that the general trend remains the same.\n\nThe results below show mean and std. dev. across 10 replications for each number of variates. (**best,** *second-best*)\n\n|# Variates| Model | Wall Time (ms) | CUDA Time (ms) | Peak Memory (MB) | Total GFLOPs |\n|-----|:---:|:---:|:---:|:---:|:---:|\n|**10**|**Toto**|652.8±12.6|*37.0±0.0*|*721.0±0.0*|**121.6±0.0**|\n||**Toto (no kvcache)**|585.0±7.9|45.8±4.1|733.6±0.1|885.2±0.0|\n||**Moirai**|**148.1±2.4**|**19.1±1.7**|**489.8±0.0**|*167.6±0.0*|\n||**TimeMoE**|1569.0±150.1|361.2±4.1|5341.2±0.0|3449.9±0.0|\n||**Chronos**|831.6±25.2|93.7±0.0|925.5±0.0|2162.5±0.0|\n||**TimesFM**|*579.5±44.9*|109.6±0.1|2023.6±0.0|5188.1±0.0|\n|**150**|**Toto**|**708.3±8.8**|**89.8±0.8**|**2223.2±0.0**|**1823.3±0.0**|\n||**Toto (no kvcache)**|*714.6±31.5*|*257.5±5.2*|2406.4±0.0|13277.4±0.0|\n||**Moirai**|1830.3±122.1|1488.5±0.7|20193.6±0.0|*7387.2±0.0*|\n||**TimeMoE**|11903.6±677.6|5109.4±271.8|33639.9±2419.7|56227.9±3128.8|\n||**Chronos**|1435.2±25.1|670.4±4.3|*2375.2±0.0*|32437.6±0.0|\n||**TimesFM**|1699.8±7.7|1239.3±5.1|3256.4±0.0|77821.4±0.0|\n|**300**|**Toto**|**775.2±15.4**|**132.8±3.6**|**3821.7±0.0**|**3646.7±0.0**|\n||**Toto (no kvcache)**|*805.1±25.3*|*505.8±5.3*|4187.8±0.0|*26554.9±0.0*|\n||**Moirai**|OOM|OOM|OOM|OOM|\n||**TimeMoE**|16590.2±254.8|9698.9±0.2|31517.0±0.0|108459.7±0.0|\n||**Chronos**|2583.3±23.6|1290.2±4.5|*3935.1±0.0*|64875.2±0.0|\n||**TimesFM**|3302.4±22.0|2445.4±5.9|4569.9±0.0|155642.8±0.0|\n\n>What is the lookback window size used for the LSF benchmark?\n\nWe used a fixed lookback window size of 2,048. This is similar to Moirai, which uses lookback windows of 2,000 or 4,000 depending on the dataset.\n\n### **Please do let us know if there are any remaining concerns that prevent you from increasing your score.**\n\n[1] Unified Training of Universal Time Series Forecasting Transformers. ICML 2024  \n[2] Mixture models and em. In Pattern Recognition and Machine Learning, pages 423–495. Springer, New York, 1 edition, 2006  \n[3] Adaptive Mixture of Student-T Distributions As a Flexible Candidate Distribution for Efficient Simulation. Journal of Statistical Software, Jan. 2009  \n[4] Robust mixture modelling using the t distribution. Statistics and Computing, 10(4):339–348, 2000.  \n[5] A general and adaptive robust loss function. CVPR 2019  \n[6] A decoder-only foundation model for time-series forecasting. ICML 2024  \n[7] Chronos: Learning the language of time series. TMLR 2024  \n[8] Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts. ICLR 2025  \n[9] UniTS: A Unified Multi-Task Time Series Model. NeurIPS 2024.  \n[10] PowerPM: Foundation Model for Power Systems. NeurIPS 2024."}, {"review_id": "Jelar3Xxo6", "reviewer": "Reviewer_RNqe", "summary": "The paper proposes a time series forecasting foundation model (TOTO) to account for the challenges of observability time series data, including Patch-based causal instance normalization, Student-T mixture model head. At the same time, the paper proposes a large-scale benchmark (Boom) consisting of 350 million observations. Extensive experiments show that ToTo achieves good prediction results on Boom and GIFT-Eval benchmarks.", "questions": "1. The UNiTS model also utilizes Attention for variable correlation modeling. Please explain the differences compared to your approach. MoIRAI also employs a Student-T Mixture for probability modeling, but the Student-T Mixture can only fit specific distributions. Please clarify how this differs from TOTO.\n2. TOTO incorporates variable modeling during pre-training. Since different datasets have different variables:\n    a) How do you ensure that the number of variables in each batch of time-series data remains the same?\n    b) Could varying numbers of variables across batches lead to difficulties in model pre-training?\n3. In Table 2, I believe it is necessary to evaluate smaller models to verify TOTO’s predictive performance. Simply comparing it with zero-shot Foundation Models is unfair because other Foundation Models were pre-trained on very limited observational time-series data, whereas TOTO used a much larger amount.\n4. Table 3 lacks comparisons with other Foundation Models such as VisionTS and Time-MOE.\n5. There is a lack of analytical experiments, ablation studies, and hyperparameter sensitivity analysis for the model.\n6. The main experiments on Foundation Models are all zero-shot evaluations. It would be beneficial to fine-tune TOTO and observe the effects under different levels of fine-tuning.", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The paper proposes a time series forecasting foundation model for real-world observation data.\n2. The paper proposes a large-scale time series benchmark consisting of a large amount of real-world observation data.\n3. ToTo achieves good prediction results on Boom and GIFT-Eval benchmarks.\n\nWeaknesses:\n1. The primary module proposed in the method section has been used in other Foundation Models, such as variable association modeling and Student-T mixture model Head.\n2. The experiment is imperfect, lacking model analysis experiments, ablation experiments, and parameter sensitivity analysis.", "was_revised": true, "final_justification": "The author provided clear responses to all my queries, covering TOTO's advantages over other base models and fine-tuning improvements. Overall, I recommend increasing the score.", "rebuttal": "Thank you for your review and thoughtful questions.\n\n> The primary module proposed in the method section has been used in other Foundation Models, such as variable association modeling and Student-T mixture model Head.\n\n> MoIRAI also employs a Student-T Mixture for probability modeling, but the Student-T Mixture can only fit specific distributions. Please clarify how this differs from TOTO.\n\n>The UNiTS model also utilizes Attention for variable correlation modeling. Please explain the differences compared to your approach.\n\nWhile we agree there are similarities between Toto’s architecture and other recent works, let us attempt to clarify which contributions we feel are meaningful and novel:\n\n**Student-T Mixture Model (SMM)**  \nWhile other authors have used mixture models (e.g. Moirai), to our knowledge **Toto is the first time series foundation model to use a Student-T Mixture Model head**. \n\nWe cite the prior art from Moirai, and took it into consideration when designing our mixture head; however the two approaches differ in crucial and meaningful ways. Moirai does *not* use a Student-T mixture model. As stated in their paper, Moirai uses a fixed mixture of four different distributions [1]. The Moirai authors suggest that these four individual components were chosen heuristically to match specific common cases within time series datasets.\n\nBy contrast, Toto's mixture consists of ***only*** Student-T distributions, where the number of components, $k$, is a hyperparameter that we optimize ($k$=24 in our selected model). This is fundamentally a more general and less heuristic approach. We derive it from first principles: mixtures of certain distributions (including Gaussians and Student-T’s) are universal approximators for ***any probability density function*** when using a sufficiently large number of mixture components [2, 3]. We choose the SMM (rather than GMM for example) based on its superior performance with heavy-tailed distributions [4], which frequently appear in the observability domain.\n\nWe also adopt a novel loss formulation which leads to significantly better convergence when training with our mixture model head. Alongside NLL, we jointly optimize the SMM with Robust Loss [4] to concentrate the mean towards strong point predictions.\n\nBased on this feedback, we will revise the discussion in our Related Works section to make these distinctions clearer.\n\n**Causal patch-based instance normalization**  \nThis is a novel form of normalization which dramatically improves performance in our ablation study (Figure 3B). It does this by preserving the causality of our autoregressive decoder-style model, which is critical for highly nonstationary time series (where future information leakage can cause severe overfitting).\n\n**Proportional factorized attention**  \nIn Section 3.1, we cite previous demonstrations of factorized attention [5-7]. Our novel contribution introduces an asymmetrical ratio of layers, with the ratio of timewise to variate-wise attention operations as a tunable hyperparameter. While this is a relatively simple idea, it represents a meaningful improvement in computational efficiency. Our design is motivated by the observation that channel-independent models such as TimesFM [8] and Chronos [9] still achieve competitive performance on multivariate benchmarks and, therefore, we expect diminishing returns on FLOPs spent on variate-wise attention operations.\n\nWe will add a citation and discussion of UNiTS to make this distinction clearer.\n\n>The experiment is imperfect, lacking model analysis experiments, ablation experiments, and parameter sensitivity analysis.\n\nWe do perform an ablation experiment to test the contribution of major architectural choices. The results are reported in Figure 3, and we provide methodological details in Appendix E in the supplemental materials.\n\nWe describe our hyperparameter selection procedure, which included an extensive sweep (see Table 5), in Appendix A.6 in the supplemental materials. Unfortunately, a full sensitivity analysis is prohibitively costly given the hyperparameter search space. We can, however, add to the paper a parallel coordinate plot or other data visualization for the configurations evaluated during our hyperparameter sweep.\n\n>TOTO incorporates variable modeling during pre-training. Since different datasets have different variables: a) How do you ensure that the number of variables in each batch of time-series data remains the same? b) Could varying numbers of variables across batches lead to difficulties in model pre-training?\n\nThanks for the question; it highlights an implementation detail which we mention briefly in Appendix A.2.1 but currently do not discuss in the main body. In pretraining, we use a fixed batch dimension of 32 variates and use document masking to pack multiple unrelated multivariate series in a single batch item. For series with more than 32 variates, we split these into multiple batch items.\n\nAt inference time, because the variate-wise attention layers are permutation invariant, we are able to scale up to as many variates as will fit into memory without needing any explicit extrapolation techniques. For example, in GIFT-Eval there are series with up to 321 variates, while BOOM contains series with up to 100 variates.\n\n> In Table 2, I believe it is necessary to evaluate smaller models to verify TOTO’s predictive performance. Simply comparing it with zero-shot Foundation Models is unfair because other Foundation Models were pre-trained on very limited observational time-series data, whereas TOTO used a much larger amount.\n\nWe do compare against deep-learning-based full-shot methods on GIFT-Eval, which is a multi-domain benchmark that is not primarily focused on observability data. We find that Toto’s zero-shot forecasts outperform full shot and small baseline models even when these competing models have been explicitly trained on the training splits of the datasets in GIFT-Eval (see Table 3).\n\nTo provide further empirical support, we have added evaluations on a subset of BOOM (what we call “BOOMLET” in the paper) using DLinear and DeepAR. These are two smaller full-shot models commonly used as baselines in the field.  Interestingly, DLinear does outperform some other foundation models, including Chronos and Time-MoE. However, Toto still achieves the best overall performance by a wide margin on BOOMLET (we will update Table 2 accordingly):\n\n|model|MASE|CRPS|Rank|\n|:----|:----|:----|:----|\n|*Toto*|*0.617*|*0.519*|*1.300*|\n|TimesFM|0.685|0.603|4.844|\n|Moirai|0.767|0.621|4.911|\n|*DLinear*|*0.823*|*0.641*|*6.044*|\n|Chronos|0.711|0.637|6.411|\n|*DeepAR*|*0.883*|*0.697*|*7.822*|\n\nWe followed the same training setup proposed by the GIFT-Eval authors [10] for full-shot evaluation with hyperparameter tuning.\n\n> Table 3 lacks comparisons with other Foundation Models such as VisionTS and Time-MOE.\n\nTable 3 is a reproduction of the GIFT-Eval leaderboard which includes our results. In the interest of saving space, we only printed the top several models in each category (zero-shot, full-shot, and baseline). With respect to VisionTS and Time-MoE:\n\n- VisionTS is not competitive with the top models on GIFT-Eval, with MASE of 0.775, CRPS of 0.638, and Rank of 20.464 as of the time of our submission.\n\n- Time-MoE has not submitted results to the GIFT-Eval leaderboard, and thus was not included in Table 3 (which as mentioned is a reproduction of the public leaderboard; we did not rerun models other than Toto).\n\nWe are happy to add a supplemental table with the complete GIFT-Eval leaderboard if the reviewer believes it to be necessary.\n\n>  The main experiments on Foundation Models are all zero-shot evaluations. It would be beneficial to fine-tune TOTO and observe the effects under different levels of fine-tuning.\n\nWe did perform a fine-tuning experiment using the training portions of the LSF datasets, which we discuss in Section 5. The procedure and results are outlined in Appendix D.3, and summarized in Table 16. Our results show that finetuning results in an improvement Toto’s zero-shot performance, and achieves state-of-the-art performance over other full-shot methods. For convenience, we reproduce an excerpt of Table 16 showing the Toto's finetuned performance on the LSF datasets against the next-best full-shot model (TimeLLM):\n\n|Dataset|Metric|Toto (Zero-shot)|Toto (Fintuned)|TimeLLM (full-shot)|\n|:---:|:---:|:----|:----|:----|\n|ETTh1|MAE|0.413|**0.409**|0.423|\n||MSE|0.435|0.415|**0.408**|\n|ETTh2|MAE|**0.363**|**0.363**|0.383|\n||MSE|0.340|**0.339**|0.334|\n|ETTm1|MAE|0.378|**0.357**|0.372|\n||MSE|0.396|0.349|**0.329**|\n|ETTm2|MAE|0.303|**0.291**|0.313|\n||MSE|0.267|**0.244**|0.251|\n|Electricity|MAE|0.242|**0.233**|0.252|\n||MSE|0.158|**0.150**|0.158|\n|Weather|MAE|0.245|**0.233**|0.257|\n||MSE|0.224|**0.206**|0.225|\n|Avg.|MAE|0.324|**0.314**|0.333|\n||MSE|0.303|**0.284**|0.284|\n\n### **Please do let us know if there are any remaining concerns that prevent you from increasing your score.**\n\n[1] Unified Training of Universal Time Series Forecasting Transformers. ICML  2024  \n[2] Mixture models and em. In Pattern Recognition and Machine Learning, pages 423–495. Springer, New York, 1 edition, 2006  \n[3] Adaptive Mixture of Student-T Distributions As a Flexible Candidate Distribution for Efficient Simulation. Journal of Statistical Software, Jan. 2009  \n[4] Robust mixture modelling using the t distribution. Statistics and Computing, 10(4):339–348, 2000.  \n[5] A general and adaptive robust loss function. CVPR 2019  \n[6] Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. ICLR 2023  \n[7] Msa transformer. ICML 2021  \n[8] Vivit: A video vision transformer. ICCV 2021  \n[9] A decoder-only foundation model for time-series forecasting. ICML 2024.  \n[10] Chronos: Learning the language of time series. TMLR 2024  \n[11] Gift-eval: A benchmark for general time series forecasting model evaluation. arXiv:2410.10393."}]

Author rebuttal:

Meta-review: This paper introduces TOTO, a foundation model for time series forecasting, addressing challenges in observational data . It also presents BOOM, a large-scale benchmark with 350 million observations. Experiments demonstrate strong performance on BOOM and GIFT-Eval. 

The authors provided clear and thorough responses, highlighting TOTO’s advantages and fine-tuning improvements. Overall, this is a solid contribution, and I recommend acceptance.
Decision: Accept (poster)