Beyond Accuracy: Are Time Series Foundation Models Well-Calibrated?
Authors: Coen Adler
Yuxin Chang
Samar Abdi
Felix Draxler
Padhraic Smyth
Venue: ICLR 2026 Poster (2026)
Keywords: Time Series
Foundation Models
Calibration
Confidence
Primary area: foundation or frontier models, including LLMs
TL;DR: We evaluate model calibration of time series foundation models and find that they are generally well-calibrated.

Abstract: The recent development of foundation models for time series data has generated considerable interest in using such models across a variety of applications. Although foundation  models achieve state-of-the-art predictive performance, their calibration properties remain relatively underexplored, despite the fact that calibration can be critical for many practical applications. In this paper, we investigate the calibration-related properties of five recent time series foundation models and two competitive baselines. We perform a series of systematic evaluations assessing model calibration (i.e., over- or under-confidence), effects of varying prediction heads, and calibration under long-term autoregressive forecasting. We find that time series foundation models are consistently better calibrated than baseline models and tend not to be either systematically over- or under-confident, in contrast to the overconfidence often seen in other deep learning models.

Published as a conference paper at ICLR 2026
BEYOND ACCURACY: ARE TIME SERIES FOUNDATION MODELS WELL-CALIBRATED!
Coen Adler', Yuxin Chang', Felix Draxler' Samar Abdi', Padhraic Smyth 1.2
*Department of Computer Science DDepartment of Statisties University of California, Irvine
Google, Irvine
{ctadler, yuxinc20, fdraxler, smyth)@ucii edu - samaradbi@google. com
ABSTRACT
The recent development of foundation models for time series data has generated considerable interest in using such models across a variety of applications. Al- though foundation models achieve state-of-the-art predictive performance, their calibration properties remain relatively underexplored, despite the fact that cali- bration can be critical for many practical applications. In this paper, we investigate the calibration-related properties of five recent time series foundation models and two competitive baselines. We perform series of systematic evaluations assess- ing model calibration (i.e., over- or confideenf, effects of varying predic- tion heads, and calibration under long-term autoregressive forecasting, We find that time series foundation models are consistently better calibrated than base- line models and tend not to be either systematically over- under-confident, in contrast to the overconfidence often seen in other deep learning models.
INTRODUCTION
Prediction and modeling of time series data is ubiquitous in data analysis, with applications across a broad range fields including climate science (Mudelsee) 2014), energy forecasting (Deb al.]
2017), healthcare (Crabtree et al. (990), consumer behavior modeling (Goel et al.) 2010), and fi- nancial forecasting (Tsay] 2005). Traditional statistical approaches such as linear autoregressive (AR) models and associated variants
well-established in the field (Hamilton, 1994 Hyndman & Athanasopoulos 2018); they have been supplemented recent years by a variety of machine learning approaches using richer model representations, including deep learning time series models such as N-BEATS (Oreshkin et al. 2020) and Informer (Zhou et al. (202).
A more recent trend is the emergence of time series foundation models (TSFMs) (Nie et al. 2023 Liang et al. 2024). Unlike traditional statistical and machine learning approaches where models fitted to time series from single source, TSFMs general-purpose models trained broad range of time series from various domains and are capable of zero-shot or few-shot forecasting on any time series in principle (Ye al., 2024 Liang al., 2024). This is appealing to practitioners in that only a single global model is required rather than retraining a new model for every time series (Bommasani et al., 2021 Benidis al. 2022).
With this increase of interest in TSFMs, it becomes important to understand and characterize the calibration properties of such models. TSFMs, in general, produce conditional distributions over potential future values, rather than just single point forecasts (e.g., the expected value of the time series at a future time). This distributional information is important and useful in many applications. In particular, it can be essential for decision decision-making (Gneiting et al. [2007] Petropoulos al. (022), for example in downstream tasks such as anomaly detection (con 2018 and in critical domains such healthcare (Farah et al. 2014, Marlin et al. 2012). A natural question in this context is: how well calibrated are TSFMs in terms of their conditional distributions, i.e., how well do the predicted probabilities match the observed data?
While the calibration propertics of classification and regression models have received considerable attention in machine learning in recent years (e.g., Guo et al. (2017); ( et al. (2019): Chung al.

---

Published as a conference paper at ICLR 2026
2021 ), the calibration properties of TSFMs remain relatively underexplored. To address this gap, we perform an empirical study of TSFMs and baselines with respect to calibration celibrationspeccffe metrics. Our systematic evaluation includes five state-of sfate-of-the-art the-art TSFMs and two well-established baseline methods in terms of their zero-shot forecasts across six univariate time series datasets with different temporal granularities. We measure calibration properties through comprehensive set of metrics, such as Probabilistic Calibration Error (Dheur & Taieb (2023] Kuleshov et al. 2018 Chung et al.) 2021. in the context of variety of different aspects of conditional uncertainty model predictions.
In general, TSFMs produce conditional distributions using two approaches: some models predict the parameters of conditional density models (Rasul al. 2023 Woo et al. 2024 Ansari et al., 2024a), while others predict sets of conditional quantiles (Das et al. [2024] Auer et al. [025]] Wang et al. 2025). To fully understand the calibration properties of TSFMs, we isolate the impact of different prediction heads by training individual quantile and distribution heads for each of the TSFM base architectures and then evaluate the effect of prediction heads on calibration performance.
Additionally, TSFMs vary in their approaches for long-term forecasting. For long-term forecast- ing beyond the forecast horizon (the prediction length single forward-pass), many TSFMs rely on autoregressive forecasting (Das et al.] 2024]] Ansari et al., 2024b; [ohen et al. 2025). Prior works (Auer et al., 2025 Wang et al., 2025) have shown that using autoregressive forecasting can have negative effects model performance due to the reinitialization of the probabilistic forecast. Solutions include multi-patch forecasting (Woo et al., 2024) where models can predict sequential patches at a time, and stochastic autoregression which propagates probabilistic information to sub- sequent forecasts as described in Section In this context, it is important to understand how lim- ited forecast horizons can affect calibration long-term forecasts. To investigate this, we compare TSFM calibration on long-term forecasting across varying forecast horizon lengths and autoregres- sive implementations.
The primary contributions of our work are as follows:
We conduct a first-of-its -fn----d---- systematic calibration study with five state-of of-t-earrr- founda- tion models and two well-established baselines across six datasets from different domain
We use three calibration-specifi metrics quantify the overall calibration error and over- and under-confidence prediction models; we analyze the calibration effects of various distributional and quantile prediction heads; and we investigate the effect of different long- range forecasting methods on calibration.
We find that TSFMs are better calibrated than baseline models, tend to be neither system- atically over- nor under-confident, are generally insensitive to the form of distributional prediction head, and consistently competitive in both prediction and calibration.
2 BACKGROUND AND RELATED WORK
2.1 PROBABILISTIC TIME SERIES FORECASTING
Given a context of T observations V1.r for time series, we evaluate the L-length forecasting per- formance on the prediction VT+1-T+L | F1.r. We denote the median prediction as i at each t e [T 1, ..., T + L} for point estimates, and predicted quantiles g to assess the uncertainty and calibration. The predicted quantiles are either produced directly by multi-quantile prediction head for each of the I. future points, or can be computed from the predicted conditional density.
TSFMs can differ slightly in their approaches to data aggregation and functionality, but typically they tokenize time series patches of a predetermined length (Nie et al., 2023). We denote the forecast horizon H as the number of future time steps model can forecast in single forward- pass. A simple approach when the desired forecast length L is greater than the model's forecast horizon H is to use autoregressive (AR) forecasting based point estimates, where the model will incorporate the previous forecasts (r+IIT Ht into the observed context. The model then uses this extended or shifted context y1 41+HH0 H h to forecast the next set of patches ÿr + 1-7-2H1 recursively until the desired forecast fr+11T+L is reached (Ansari et al. 2024b Cohen et al. 2025). Methods to
!Code
available
https://0o http:://ggttbbb.cco/////tt//e/// /eyondeeodddooooo Accuracy TSFM
Ca Calibration
2

---

Published as a conference paper at ICLR 2026
Calibration Error
Callbration Eror
(WVQL)
10y
(PCE)
0x 10"
10"1
10-1
6x 10""
x 10"
6 101
4 10
6 x 10"
Accuracy Error (MASE)
Accuracy Error (MASE)
Chronos-
TimesFM
N-BEATS
Figure 1: Weighted Quantile Loss (WQL) calibration metric can be highly correlated with point accuracy error (MASE). Results shown for WQL (left) and PCE (right) plotted against MASE on the Glucose dataset used in our experiments. Background markers results for each model and each individual time series; larger centroids are medians across time series. On this dataset, WQL incorrectly identifies ARIMA as the best calibrated model.
reduce the reliance on AR forecasting include extending the number of simultaneous output patches in multi-patch forecasting (Woo al., 2024), increasing output patch length (Das al. 2024).
and incorporating missing- data masks to extend the context into the future (Auer et al. 2025). We
empirically compare these approaches for long-term forecasting in Section
In TSFMs, patched time series are passed into model-specific backbone. which translates the em- beddings into latent space as an input to the final projection block. These probabilistic predictions can then be directly predicted as quantiles through multi-headed projection block model can use the latent activations to output the parameters to a distribution where the full densities can be re- covered. Some prediction heads can better represent different types of data, for instance, a negative binomial would be well-suited for count data, while a Student's 's f distribution would be appropriate for a general continuous-vallin butions, as well as quantile heads, have also been proposed (Woo et al.) 2024, Das et al., [024] Cohen et al. 2025) to allow for additional flexibility in terms of how TSFMs can model conditional distributions. We evaluate these different projection blocks across a range of domains to identify certain heads are better calibrated for specific datasets in Section 4.3
time series. Furthermore, parameterizations of mixture of distri-
2.2 RELATED WORK ON CALIBRATION
Predictive calibration has long been recognized as a topic of importance in statistics and machine learning (Brier 1950: Gneiting et al. 2007). In recent years, number of different studies have investigated the calibration of deep learning models, and have shown that such models are often systematically susceptible to overconfidence, for example in image classification with deep models (Guo et al. 2017]] Ye et al.] 2023 Pinto et al. (022) and in multiple anesttoo--nnswerng tasks for large language models (Jiang et al. 202 Mielke al. 2022 Xiong al. 2024).
In the evaluation of time series forecasting models, calibration is also an important compo- nent (Makridakis al., [2022] Aksu et al. 2224 Das et al., (0024). One of the most commonly used calibration metrics is the Continuous Ranked Probability Score (CRPS), which directly compares the cumulative distribution function (CDF) of model predictions the observed value's CDF (Hers- bach 2000). Since computing CRPS can be intractable and some TSFMs only produce quantiles rather than full conditional densities, the Weighted Quantile Loss (WQL) is also often used as a discrete surrogate for CRPS (Woo et al. 2024 Aksu al., 2024), defined as the pinball (or quantile) loss scaled by the absolute sum of the true values. Another common calibration metric is Mean Scaled Interval Score (MSIS) (Makridakis et al.) 2020), which takes the mean difference in upper and lower bound predictions and adds an error penalty when the true value lies outside the bounds.
Using these metrics, prior works have claimed that TSFMs are better calibrated than baseline meth- ods like ARIMA and N- N-BEATS (Aksu et al. 2024 Ansari et al. 2024a Auer et al., 2025). However, Chung et al. (2021) proved that the metrics above (CRPS, WOL, MSIS) measure combination of both probabilistic calibration and sharpness (Gneiting et al. 2007), rather than calibration alone. This combination can result in imbalance in evaluation that can skew towards prioritizing predic-
3

---

Published as a conference paper at ICLR 2026
tive sharpness (Chung et al. 2021) Figure shows concrete example of how PCE and WQL can differ in practice: WQL can be highly correlated with Mean Absolute Scaled Error (MASE), while PCE directly measures calibration. In the next section, discuss calibration-focusee metries that address this issue by focusing solely on measuring calibration.
3
EXPERIMENTAL SETUP
3.1
METRICS
The first calibration metric we use is Probabilistic Calibration Error (PCE) (Kuleshov et al. 2018
Dheur Taieb 2023):
T+L Σ QEQ t=
1 Q
PCE
1ly S 02
(1)
where q E Q 00.., 0.9} is the set of quantiles that we average over in our experiments. Intuitively, PCE measures the differences between empirical and predicted CDFs with lower values indicating better-calibrated models. PCE is case where predicted quantiles are always all above or below the observed value yt-
by 0 and noper-bounded by 0.5 for the
However, well-calibrated models not sufficient to produce useful forecasts: model could for example always predict the marginal distribution, independent of the inputs. In this context, a met- ric that specifically captures sharpness (the concentration of the predictive distributions) is also important in an overall evaluation of calibration (Gneiting et al.) [2007] Kuleshov et al.) (018). A simple surrogate for sharpness is the width of predicted confidence interval, e.g., where an 80%- confidence interval is the interval between the symmetric 9oo 10% and Qiggh 90% quantile predictions. We refer to this as Scaled Interval Width (SIW) where s the confidence associated 80% in the preceding example):
with the interval (i.e., Qhigh qlw
T+L Hhih Ht Thss y n
SIW, 1
Σ
L
T=
Models with lower SIW values (i.e., tighter prediction intervals) tions, while models with larger SIW values (i.e., wider-spread predictions) are less confident.
confident in their predic-
We are also interested in whether model is systematically miscalibrated in direction other. To quantify this, we define the metric Centered Calibration Error (CCE) that compares the amount of observed data in a predicted interval with the associated confidence st
T+L Σ I=
CCE =
Σ SE9
1
10 Ppn S
y1 S
y/1
Miah
S
A model's over- or under-confidence can be identified by combining the direction of CCE and its SIW. Positive CCE values indicate there is more observed data outside the predicted interval than expected by the confidence level; together with a low SIW value, we can infer that a model is overconfident. On the other hand, negative CCE values and larger SIW values imply that the model under-confident. For both CCE and SIW, we average over & E (0.2, 0.4, 0.6, 0.8) our analyses.
Finally, to assess the point accuracy of TSFMs (independently from any calibration error) we use Mean Absolute Scaled Error (MASE) for the predicted median, scaling the Mean Absolute Error (MAE) by naive predictor (Hyndman & Athanasopoulos 2018):
1 T+L 0.5 yel
MASE
T, T+L lye v--1
- 11=
Note that for multi-step predictions (L > 1) that the MAE of the naive predictor in the denominator is in effect using information "from the future," with the result that the overall MASE may take values greater than even for useful models. The absolute values of MASE are not important, but the relative MASE values across different predictors ý are what will focus on.
4

---

Published as a conference paper at ICLR 2026
3.2 MODELS
Base Models We evaluate calibration properties of five TSFMs in terms of their zero-shot fore-
casts:
Chronos- Bolt (Ansari et al. 2024a b). TimesFM (Das et al. 2024), Moirai 2.0 (Woo et al,
2024
Taha Aksu et al. 2025). TiRex (Auer al.) 2025), and YingLong (Wang al.] 2025). As
baselines, we use ARIMA (Hyndman & Khandakar 2008) and N-BEAT' (Oreshkin et al. 2020)
to represent well-known parametric and neural time-series prediction alternatives TSFMs. All selected TSFMs have been pretrained on various large pretraining datasets and use a quantile pre- diction head. Additionally. they all use transformers, except for TiRex which uses xLSTM (Beck) et al. 2024) backbone. TimesFM, Moirai 2.0, and TiRex use a decoder-only architecture with attention mechanism, while Chronos- Prronos-Boott uses the encoder-decoder backbone from the T5 model (Raffel et al. (020) and YingLong uses a bi-directional encoder-onke architecture.
casua
Autoregressive Methods Autoregressive (AR) forecasting often necessary for models with limited forecast horizon for long-term forecasting. Each model implements AR forecasting slightly differently. Chronos-Bolt and TimesFM use naive point-based AR, where only the mean or me- dian predictions are autoregressively added to the context. This method while simple is known to significantly harm probabilistic forecasting due to the re-initialization of the context each itera- tion (Auer et al. 2025). Improvements to the naive method typically require forecasting each time step multiple times with different AR contexts to propagate probabilistic information across fore- cast iterations: Moirai 2.0 implements a branching approach which autoregressively forecasts using a separate context for each quantile. The Toto TSFM (Cohen et al.] 2025) uses a trajectory approach which produces (usually >> IQ) independent AR forecasts trajectories, where each trajectory is produced similarly to the point-based approach. Instead of adding the mean or median forecast to the context, adds random sample from the predicted distribution at each time step. We explain these methods in more detail in Appendix [A.3 These AR methods trade off robustness for com- putational efficiency, where the TimesFM and Chronos-Boo approach only forecasts each time step once, while the branching (Moirai 2.0) and trajectory approach (Toto) require [Q] and n forecasts per time step respectively. In addition to comparing these AR methods, we evaluate how larger or smaller forecast horizons affect calibration on long-term AR forecasting.
3.3 DATASETS
We selected evaluation datasets representing a range of tasks differing in time-step granularity, sea- sonality, and forecasting difficulty across a variety of domains. To the best of our knowledge, these datasets were not used in the training of the foundation models (except the M5 data for the Moirai 2.0 and YingLong models). Each dataset is split into train and test set where the non- non-TSFMs, ARIMA and N-BEATS, are trained independently toARIMA and grid search respectively. All models are evaluated on the held-out test set. Table summarizes the dataset statisties. Our results each dataset aggregated over multiple settings (see Section 3.I), prediction steps, and all possible context- preeextttreiitiion combinations in the test set for each time series, i.c., effectively generating many more time series forecasts than the number of time series (see the last column in Table T). Additional details are provided in Appendix A.2
the train set, where hyperparameters are chosen by Au-
Table Datasets used in calibration experiments.
# Time Series
Granularity
Time Steps per Series
# Total Forecasts
Reviews
239
Hourly
13,000
360,490
Shopping (M5)
70
Daily
1,912
62,160
Glucose
16
Mins
1,686
13,840
Heart-Rate
6
Second
744
1,800
Crime
5
Daily
6574
4,110
Patents
83
Monthly
408
10,956
To evaluate the effect of different prediction heads for the TSFMs, trained prediction heads using a large diverse dataset that is comparable to those used in TSFM pretraining. Specifically, we se- lected TSMixup from Ansari al. (2024a) it is independent of the six datasets use to evaluate
5

---

Published as a conference paper at ICLR 2026
the models. This allows the trained projection blocks to be an equivalent plug-in replacement to the default quantile projection heads of the selected models.
4 EXPERIMENTS
Based on the models, datasets, and metrics outlined above, found that foundation models exhibit competitive and often better point-forecasting performance compared to baselines, with the TSFMs often having lower MASE than N-BEATS and ARIMA (see Figure 2 The Glucose and Patents datasets have significantly higher MASE numbers than the other datasets-thi may be due to the fact that for each of these datasets there appears to be little significant linear dependence beyond or 2 lags (see partial autocorrelation plots in Figure 6 in Appendix B.4).
TSFMs also exhibit significantly better calibration performance than baseline models. We discuss the main results on calibration in detail below. In the Appendix, we also include additional figures and findings, where we discuss the empirical correlation of WQL, MSIS, and MASE, along with details on the calibration of tail probabilities and additional prediction heads and datasets.
4.1 ARE FOUNDATION MODELS WELL-CALIBRATEDS
Previous works have indicated that TSFMs are well calibrated using CRPS, WQL, and MSIS (Asu et al. 2024, Auer et al. 2025). but as discussed in Section 3.1 and in Chung et al. (002I). these metries can be biased towards sharpness and accuracy rather than reflecting calibration per In our experiments, we evaluate directly if TSFMs are well-calibrated, focusing on the PCE metric, and we find that, yes, TSFMs BEATS and ARIMA: see top plot in Figure S For a general sense of scale, PCE values below 0.05 or 5% eπror relative to the quantiles can loosely be considered be much better calibrated than values larger than say 0.15, in the context of the range of PCE being between O and 0.5. The TSFMs are general close to below 5% in PCE error, except for the relatively difficult Patents dataset where all methods (TSFMs and baselines) have high calibration error. No single TSFM significantly dominates in calibration performance over the others.
generally better calibrated than the baseline models of N-
A natural additional question is how calibration performance is affected by how far in time model prediction is from the context. Naturally, as both TSFMs and baselines predict further into the future, both the point accuracy (MASH) and calibration error (PCE) degrade (see Figures ( and in the Appendix). However, calibration overall remains relatively stable with TSFMs achieving PCE values close to below 5% even 64 time-steps out (e.g., for Reviews, M5, and Crime data), unlike the baseline models which have a consistently high PCE over all prediction lengths.
As control experiment, we also evaluated the calibration performance of TSFMs two synthetic datasets, one with pure IID noise 3/2 = e and the other a noisy first-order linear process yt = 0.9. For these datasets the question of interest is
ow--1 + (1 aee,, with e (O, 1) and whether TSFMs might overfit (for either calibration or point prediction) relative to a baseline like ARIMA (which in principle can be perfect on these datasets). As shown in the lower right two plots in Figure i in the Appendix, the TSFMs do not overfit (they perform well on both MASE and PCE). ARIMA does well MASE but is not as well-calibrated (PCE) (nor is N-BEATS)
the TSFMs.
In summary, while past works have shown that TSFMs perform well on WQL and CRPS metrics al. (2025) as we discussed earlier (see Figure and further in Appendix B.2), WQL alone is not necessarily reliable indicator of calibration performance. Our results provide direct confirmation that TSFMs tend to be well-calibrated.
(c.g.. Aksu et al. (2024); Ansari et al. (2024)) Auer Auer
4.2
ARE FOUNDATION MODELS SYSTEMATICALLS BIASED IN MISCALIBRATION?
For image and text modalities, deep models are well-known to often be overconfident (Ye et al. 2023 Xiong et al.) 2024 Kapoor et al. 2024). In the context of time series, it is natural to ask if TSFMs similarly exhibit systematic biases, either towards over- or under-confidence.
To answer this, evaluated the CCE metric across datasets and models. The lower plot in Figure shows the directionality of model confidence. Except for the Patents dataset, where all models are overconfident, we find that TSFMs tend not to be systematically over- under-confident. In

---

Published as a conference paper at ICLR 2026
1.5
Accuracy Error
(MASE)
20
0.0
Reviews
Shopping (MS)
Patents
Chronos-Bolt
TimesFM
TiRex
YingLong
N-BEATS
ARIMA
Figure 2: TSFMs offer competitive point forecasting performance compared the baseline models, often having better accuracy than the baselines. y-axis: Mean Absolute Scaled Error (MASE) measuring point accuracy error of the median prediction (lower is better), Glucose and Patents use their own y-axis scale (on the right). The error bars are computed as the Standard Error of the Mean (SEM) across timeseries.
0.15
0.2
(PCE)
Calibration
0.10
0.00
Reviews
Shopping (M5)
Crime
Heart-Rate
Glucose
Patents
Confidince (CCE)
0.2
Over /Under
-0.4
Reviews
Shopping (MS)
Heart-Rate
Patents
Chronos-Bolt
Moirai
TimesFM
TiRex
YingLong
N-BEATS
ARIMA
Figure 3: TSFMs are better calibrated than the baseline models and are not systematically over- or under-confident.. Top: Probabilistic Calibration Error (PCE) across datasets and models using the default quantile projection block. (Patents PCE uses its own y-axis scale (on the right)). Lower PCE values are better. Bottom: Centered Calibration Error (CCE) evaluating systematic overconfidence (positive) or under-confidence (negative).
contrast, N-BEATS and ARIMA are consistently under-confident As shown in Figure 12 in the Ap- pendix, the interval width (SIW) and centered calibration (CCE) tend to have negative correlation: models with wider confidence intervals tend to have a smaller CCE or are under-confident.
It is noteworthy that TSFMs not systematically overconfident, in the way that foundation models from the text and image domains tend to be. This can likely be explained by the fact that TSFMs are being directly trained with calibration-aware loss (i.e., trained to minimize WQL), while text and image models are trained to minimize reconstruction or classification error.
4.3 How Do PREDICTION HEADS AFFECT CALIBRATION?
major design choice of TSFMs is selecting a suitable prediction head. Multiple approaches have been presented in the literature, including quantile prediction heads (the default choice for all TSFMs in our experiments), parametric distribution heads (c.g.. Lag-Llama (Rasul et al. (003)) with Stu- dent's (), categorical distribution heads (e.g., Chronos (Ansari et al. 2024a)), and semi-parametric

---

Published as a conference paper at ICLR 2026
Reviews
Shopping (M5)
Crime
0.10
Calibration Eroor (PCE)
0.08
0.15
0.15
0.06
0.10
0.10
0.05
0.02
0.05
0.2
0.0
-0.2
0.0
-0.2
0.0
Heart-Rate
Glucose
Patents
Calibration Ertor (PCE)
0.10
0.25
0.08
0.20
0.06
0.06
0.15
0.04
0.01
0.10
-0.2
-0.1
0.2
0.11
0.0
0.2
0.0
Over- Over-/Under-Ce Ferr/nnndr-Cooddddee Confidence (CCE)
Over- Seee-/Under-Cone Confidence (CCE)
Over-/ Snee/Unnderrnnnie Confidence (CCE)
Chronos-Bolt
Moirai 2.0
TimesFM
TiRex
YingLong
Quantiles
Gaussian
Student's
X
Mixture Distribution
Figure 4: Gaussian prediction heads are under-confident and are in most cases outperformed by the other prediction heads. Calibration properties of various prediction heads with Centered Calibration Error (CCE) on the z-axis and Probabilistic Calibration Error (PCE) on the y-axis. Large positive CCE values indicate overconfident predictions, while large negative CCE values show under-confidence. Background markers are the results for each head (shape) with each backbone model (color), while the larger gray centroids are the mean results across models.
mixture heads such as Toto (Cohen al. 2025) and Moirai 1.0 (Woo al. 2024) which use a variety of components such as Gaussian, Student's log-normal, and negative-binomial distributions.
important question is whether the form of a prediction head for a model affects its calibration performance? To address this question, we used the latent information produced by each pretrained TSFM backbone and trained different prediction heads (using TSMixup) for each, specifically: a Gaussian distribution, Student's t distribution, and mixture distribution containing a Gaussian, Student's s f, log-normal, and a Laplace distribution. We also retrained a separate quantile projection head on TSMixup
control to compare against and found that the trained projection heads cor- rectly learn the latent information the original foundation model with the trained quantile heads having equivalent forecasting performance the original quantile head.
We then evaluated all combinations of models and prediction heads using the PCE and CCE metrics. As shown in Figure the quantile, Student's and mixture distribution heads have very similar calibration error across all datasets, indicating that there is no sigmificant advantage for any of the three over the others. On the other hand, the Gaussian distribution's calibration results are significantly worse than the other heads. In particular, the Gaussian heads are consistently under- confident with CCE scores (r-axis of Figure 4 always lower than the other heads. This under- confidence also translated to consistently higher calibration error. We speculate that the limited expressiveness of Gaussian distribution results poor calibration while the more expressive distributions are better calibrated without suffering from overfitting issues.
4.4 How Do AR METHODS AFFECT CALIBRATION IN LONG- MENG-TERM FORECASTING?
is well known that autoregressive (AR) models can deteriorate when making long-term predictions when the desired forecast length L is much longer than the model's forecast horizon H (Auer et al.) 2025). Different approaches to AR forecasting have been proposed, with different tradeoffs in terms of how errors accumulate and how much computational effort involved. An important question from calibration perspective is how do the different AR forecasting methods affect calibration
8

---

Published as a conference paper at ICLR 2026
Reviews
Shopping (M5)
Crime
Glucose
Calbrrtion Error (PCE)
0.08
0.06
0.10
0.2
0.04
0.10
0,00
0.1
0.02
32
64
128
16
64
128
16
32
64
128
30
64
128
Horizon Length
Horizon Length
Horizon Length
Honzzo Length
Chronos-Bolt
Branch
Mairal 2.0
Branch
TimesFM
Branch
TiRex
non-AR
Chronos- Belt
Trajectory
Mairai 2.0
Trajectory
TIMesFM
Trajectory
Yingl ong
non-AR
Figure 5: For long-term AR forecasting, increasing the horizon length and using the trajectory AR method produces better calibrated forecasts. The figure compares Probabilistic Calibration Error (PCE) for long-term forecasting using autoregression on the y-axis with AR prediction horizon on the aaiis.. The color of the line depicts the model used and the line style indicates the AR method. The pink and blue dashed horizontal lines are the reference PCE for TiRex and Yinglong which capable of long-term forecasting without the use of AR.
performance? To investigate this, we evaluated how calibration properties of TSFMs vary as function of horizon length H and AR method. We focused primarily on the Chronos- Bolt, TimesFM, and Moirai 2.0 models in these experiments given that TiRex and YingLong model architectures allow for long-term forecasting natively without requiring AR.
For both the trajectory and branching AR methods, the predictions from models with shorter forecast horizon H have poorer calibration on fixed forecast length L.. This is more pronounced in the branching method, where forecasts with horizon lengths of 16 have notably worse PCE than with 64 or 128 as shown in Figure [ The trajectory AR approach has generally lower PCE than the branching method when comparing with the same forecast horizon. The explanation for this becomes more clear when viewing CCE from Figure in in the Appendix with respect to these meth- ods. All autoregressive TSFMs are consistently overconfident in long-term forecasting. While both methods show that shorter forecast horizons results in more confident forecasts, the CCE val- ues for the branching method are often greater than 0.15 for horizon lengths of 16 and 32. This overconfidence reduces steeply as the horizon length increases thus reducing overall calibration er- ror. Although the trajectory method can be more computationally expensive than the branching approach, our results indicate that the trajectory method is better calibrated for long-term forecast- ing. The non-AR approaches of TiRex and YingLong are significantly more efficient than both AR methods, and they tend to be better calibrated and were not significantly over- or under-confident. Future work in long-term forecasting should prioritize models with longer forecast horizons and alternatives to AR-based approaches.
CONCLUSION
Understanding the calibration properties of TSFMs closes gap in the literature, which has primarily focused date on point accuracy. Calibration crucial, however, to accurately quantify the inherent uncertainties associated with time-series forecasting.
In this paper, we evaluated the calibration properties of current leading TSFMs and found that they are consistently well-calibrated relative to the N-BEATS and ARIMA baselines. Unlike the base- lines, which are under-confident,, the TSFMs show no signs of systematic over- or under-confidence in short-term forecasting. When replacing the projection heads of the TSFMs, the Gaussian pre- diction head results in consistently under-confident forecasts across the different model backbones, while other distribution and quantile heads are all well-calibrated without significant differences. For long-term forecasting using autoregression, having larger prediction horizons and using the trajec- tory AR approach over the branching method decrease calibration error and reduces overconfidence.
A limitation of our study is that our evaluations only considered TSFMs for zero-shot univariate time series and calibration relative to fixed set of quantiles q {0.1..... 0.9}. As such, worth- while extensions of our work could be to investigate calibration in the context of fine-tuning, to extend multivariate data, and to examine higher-resolution quantiles. Another important practical

---

Published as a conference paper at ICLR 2026
direction for future investigation is how distribution shift and non-stationarity may affect the calibra- tion performance of both TSFMs and baselines, given the well-known sensitivity the calibration performance of deep classification models to distribution shift (Snock et al. 2019).
ACKNOWLEDGMENTS
This work was supported in part by the Hasso Plattner Institute (HPI) Research Center in Machine Learning and Data Science at the University of California, Irvine, by a Google faculty award, by the National Science Foundation under award NSF RISE-2425932. and by the National Institutes of Health under awards IROICA297869001 and R01-LM013344. FD acknowledges funding from the National Science Foundation (NSF) through NSF CAREER Award IIS-2047418. IIS-2007719, the NSF LEAP Center, and the Chan Zuckerberg Initiative.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
A loss framework for calibrated anomaly detection. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa- masa-Bianchi, and Roman Garnett (eds.), Advances in Neu- ral Information Processing Systems 31: Anmual Conference on Neural Information Processing Systems 2018, NeurlPS 2018, December 3-8, 2018, Montréal, Canada, pp. 1494-1504, 2018.
Taha Aksu, Gerald Woo, Juncheng Liu, Xu Liu, Chenghao Liu, Silvio Savarese, Caiming Xiong, and Doyen Sahoo. Gift-eval: benchmark for general time series forecasting model evaluation. ArXiv preprint, abs/2410. 103993, 2024. httpp: wttp://...rr//////21...... 2210010993
Abdul Fatir Ansari, Lorenzo Stella, Caner Turkmen, Xiyuan Zhang, Pedro Mercado, Huibin Shen, Oleksandr Shchur, Syama Sundar Rangapuram, Sebastian Pineda Arango, Shubham Kapoor, et al. Chronos: Learning the language time series. Transactions on Machine Learning. 2024a.
Abdul Fatir Ansari, Caner Turkmen, Oleksandr Shchur, and Lorenzo Stella. and accurate zero-shot forecasting with Chronos-Bolt and AutoGluon, 2024b. URL
Fast
https://aws www.aa com/bl wwwg//aaciinneeeeeeeeeeeeeeeeeee Seeechinee-earnnt Learreranninn ing and lonndacccuaate
wwwpeecatttiiiwwth--hhe-----ee wwwth-hhhhhossss -ates-ooltaaaaaa autogluon//
Andreas Auer, Patrick Podest, Daniel Klotz, Sebastian Bück,, Günter Klambauer, and Sepp Hochre- iter. Tirex: Zero-shot forecasting across long and short horizons with enhanced in-context learning. ArXiv preprint, abs/2505.23711 2025. URL
23719
Maximilian Beck, Korbinian Pöppel, Markus Spanring, Andreas Auer, Oleksandra Prudnikova, Michael Kopp. Günter Klambauer, Johannes Brandstetter, and Sepp Hochreiter. xLSTM: Ex- tended long short-term memory. ArXiv preprint, abs/2405.04517, 2024. URL https:// wrx.v.org/abs////oooo..o.o. 2405.
Konstantinos Benidis, Syama Sundar Rangapuram, Valentin Flunkert, Yuyang Wang. Danielle Mad- dix, Caner Turkmen, Jan Gasthaus, Michael Schnke-Schneider, David Salinas, Lorenzo Stella, et al. Deep learning for time series forecasting: Tutorial and literature survey. ACM Computing Surveys, 55(6):1-36, 2022.
Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the op- portunities and risks of foundation models. ArXiv preprint, abs/2108.07258, 2021. URL
hittps://ariis http://ggg/bb//o/ooo
Glenn W Brier. Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1):1-3, 1950.
Peter Cho, Juseong Kim, Brinnae Bent, and Jessilyn Dunn. Big ideas lab glycemic variability and wearable device data. (version 1.1. 1). PhysioNer, 2023.
Youngseog Chung, Willie Neiswanger, Ian Char, and Jeff Schneider. Beyond pinball loss: Quantile methods for calibrated uncertainty quantification. Advances in Neural Information Processing Systems, 14:10971-10998 2021.
Ben Cohen, Emaad Khwaja, Youssef Doubli, Salahidine Lemaachi, Chris Lettieri, Charles Masson, Hugo Miccinilli, Elise Ramé, Qiqi Ren, Afshin Rostamizadeh, et al. This time is different: An observability perspective on time series foundation models. ArXiv preprint, abs/2505.14766, 2025. URL https://arxii..ttt
abs hbtp://aa.//52.....// 255..146656.6
Benjamin F Crabtree, Subhash C Ray, Priscilla M Schmidt, Patrick O'Connor, and David D Schmidt. The individual over time: time series applications in health care research. Journal of Clinical Epidemiology, 4313):241-260, 1990.
Abhimanyu Das, Weihao Kong, Rajat Sen, and Yichen Zhou. A decoder-only foundation model for time-series forecasting. In Proceedings of the 41st International Conference on Machine Learning, 2024.
11

---

Published as a conference paper at ICLR 2026
Chirag Deb, Fan Zhang, Junjing Yang. Siew Eang Lee, and Kwok Wei Shah. A Areiiew time series forecasting techniques for building energy consumption. Renewable and Sustainable Energy Reviews, 74:902-955 -202229242222 2017.
Victor Dheur and Souhaib Ben Taieb. A large-scale study of probabilistic calibration in neural network regression. In International Conference Machine Learning, pp. 7813-7836. PMLR, 2023.
Marian Farah, Paul Birrell, Stefano Conti, and Daniela De Angelis. Bayesian emulation and cali- bration of a dynamic epidemic model for a/hInl influenza. Journal of the American Statistical Association, (096508):1398-11 141111 2014.
Tilmann Gneiting and Adrian E Raftery. Strietly proper scoring rules, prediction, and estimation. Journal of the American statistical Association, 102(477):359-378 2007.
Tilmann Gneiting, Fadoua Balabdaoui, and Adrian Raftery. Probabilistic forecasts, calibration and sharpness. Journal of the Royal Statistical Society Series B: Statistical Methodology, 69(2):
243-268, 2007.
Sharad Goel, Jake M Hofman, Sébastien Lahaie, David M Pennock, and Duncan J Watts. Predicting consumer behavior with Web search. Proceedings of the National Academy of Sciences, (41(; 17486-17490) 2010.
Xiao Gu, Yu Liu, Zaineb Mohsin, Jonathan Bedford, Anshul Thakur, Peter Watkinson, Lei Clifton, Tingting Zhu, and David Clifton. Are time series foundation models ready for vital sign forecast- ing in healthcare? In Proceedings of the 4th Machine Learning for Health Symposium, volume 259 of Proceedings of Machine Learning Research, pp. 401-419. PMLR, 2025.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Con- ference on Machine Learning, ICML 2017, Sydney, NSW. Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pp. 1321-1330. PMLR, 2017.
James D Hamilton. Time Series Analysis. Princeton University Press, 1994.
Hans Hersbach. Decomposition of the continuous ranked probability score for ensemble prediction systems. Weather and Forecasting, 15(5):559-577) 2000.
Yupeng Hou, Jiacheng Li. Zhankui He, An Yan, Xiusi Chen, and Julian McAuley, Bridging lan- guage and items for retrieval and recommendation. ArXiv preprint, abs/2403.03952, 2024. URL https:://axxiv org/ abs 2403. 220..//33323222222222 03952
Rob Hyndman and Yeasmin Khandakar. Automatic time series forecasting: the forecast package for R. Journal of Statistical Software, 27:1-22, 2008
Robin John Hyndman and George Athanasopoulos. Forecasting: Principles and Practice. OTexts, Australia, 2nd edition, 2018.
Zhengbao Jiang, Jun Araki, Haibo Ding, and Graham Neubig. How can we know when language models know? on the calibration of language models for question answering, Transactions of the Association for Computational Linguistics, 9:962-977, 2021.
Sanyam Kapoor, Nate Gruver, Manley Roberts, Arka Pal, Samuel Dooley, Micah Goldblum, and Andrew Wilson. Calibration-tuning:: Teaching large language models to know what they don't know. In Proceedings of the 1st Workshop on Uncertainty-Aware NLP (UncertaiNLP 2024), pp- 1-14, 2024.
Volodymyr Kuleshov, Nathan Fenner, and Stefano Ermon. Accurate uncertainties for deep learning using calibrated regression. In Jennifer G. Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference Machine Learning, ICML 2018. Stockholmsmässan. Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pp. 2801- 2809. PMLR, 2018.
12

---

Published as a conference paper at ICLR 2026
Jiacheng Li, Jingbo Shang, and Julian McAuley. UCTopic: Unsupervised contrastive learning for phrase representations and topic mining, ArXiv preprint, abs/2202. 13469, 2022. URL https:
//txxii.org/////aa.a.aa 2202. 13469
Yuxuan Liang, Haomin Wen, Yuqi Nie, Yushan Jiang, Ming Jin, Dongjin Song, Shirui Pan, and Qingsong Wen. Foundation models for time series analysis: A tutorial and survey. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 6555- 6565, 2024.
Spyros Makridakis, Evangelos Spiliotis, and Vassilios Assimakopoulos. The m4 competition: 100,000 time series and 61 forecasting methods. International Journal of Forecasting, 36(1): 54 74, 2020.
Spyros Makridakis, Evangelos Spiliotis, and Vassilios Assimakopoulos. The M5 competition: Back- ground, organization, and implementation. International Journal of Forecasting, 38(4):3325- 1336, 2022.
Alan Marco, Michael Carley, Steven Jackson, and Amanda Myers. The USPTO historical patent data files two centuries of innovation, 2015.
Benjamin M. Marlin, David C. Kale, Robinder G. Khemani, and Randall C. Wetzel. Unsupervised pattern discovery in electronic health care data using probabilistic clustering models. In Proceed- ings of the 2nd ACM SIGHIT International Health Informatics Symposium, IHI '12 pp. 389-398, New York, NY, USA, 2012. Association for Computing Machinery.
Sabrina Mielke, Arthur Szlam, Emily Dinan, and Y-Lan Boureau. Reducing conversational agents' overconfidence through linguistic calibration. Transactions of the Association for Computational Linguistics, 10:857-877 872, 2022.
Manfred Mudelsee. Climate Time Series Analysis: Classical Statistical and Bootstrap Methods, volume 51. Springer, 2014.
New York City Police Department. NYPD Complaint Data Historic, 2025.
URL
https://@ta..cityyoeevvvyxzx wsskubliccuccccccc Safety/NYPD-Comg aestorrcc/geea-155//bboooe data
Data
Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. time series is worth 64 words: Long-term forecasting with transformers. In The Eleventh International Conference
Learning Representations, 2023.
Boris N. Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. N-BEATS: neural ba- sis expansion analysis for interpretable time series forecasting. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenRe- view.net, 2020.
C.-K Peng, Joseph Mietus, Yanhui Liu, Gurucharan Khalsa, Pamela S Douglas, Herbert Benson, and Ary I. Goldberger. Exaggerated heart rate oscillations during two meditation techniques. International Journal of Cardiology, 1012):001-177, 1999.
Fotios Petropoulos, Daniele Apiletti, Vassilios Assimakopoulos, Mohamed Zied Babai, Devon K Barrow, Souhaib Ben Taieb, Christoph Bergmeir, Ricardo Bessa, Jakub Bijak, John Boylan, et al. Forecasting: theory and practice. International Journal of Forecasting, 38(3)7705-871, 2022.
Francesco Pinto, Philip HS Torr, and Puneet K. Dokania. An impartial take the CNN vs trans- former robustness contest. In European Conference on Computer Vision, pp. 466-480. Springer, 2022.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 11( 2020.
13

---

Published as a conference paper at ICLR 2026
Kashif Rasul, Arjun Ashok, Andrew Robert Williams, Hena Ghonia, Rishika Bhagwatkar, Arian Khorasani, Mohammad Javad Darvishi Bayazi, George Adamopoulos, Roland Riachi, Nadhir Hassen, et al. Lag- -lamm: Towards foundation models for probabilisticc time series forecasting. ArXiv preprint, abs/2310.08221 2023. URL https: wwwp:l/wwwwwa/aa////eeo 231.200210000028888
Jasper Snoek, Yaniv Ovadia, Emily Fertig, Balaji Lakshminarayanan, Sebastian Nowozin, D. Scul- ley, Joshua V. Dillon, Jie Ren, and Zachary Nado. Can you trust your model's uncertainty? eval- uating predictive uncertainty under dataset shift. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' Alché-Bue, Emily B. Fox, and Roman Garnett (eds.), Advances in Neu- ral Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 13969-13980, 2019.
Hao Song, Tom Diethe, Meelis Kull, and Peter A. Flach. Distribution calibration for regression. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pp. 5897-5906. PMLR, 2019.
Ibrahim Taha Aksu, Juncheng Liu, Chenghao Liu, Doyen Sahoo, Junnan Li, Caiming Xiong, and Silvio Savarese. Introducing Moirai 2.0, 2025. URL https: h/tp::://wwww.. www.sslreesfoeooooo com/
blog/ 0hog/moirrii2-90
Ruey S Tsay. Analysis of Financial Time Series. John Wiley & Sons, 2005.
Xue Wang. Tian Zhou, Jinyang Gao, Bolin Ding, and Jingren Zhou. Output scaling: YingLong- delayed chain of thought in a large pretrained time series forecasting model. ArXiv preprint, abs/2506.11029, 2025. httpppps//aa....oo//////////////
Gerald Woo, Chenghao Liu, Akshat Kumar, Caiming Xiong, Silvio Savarese, and Doyen Sahoo. Unified training of universal time series forecasting transformers. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. JMLR.org. 2024.
Miao Xiong, Zhiyuan Hu, Xinyang Lu, YIFEI LI, Jie Fu, Junxian He, and Bryan Hooi. Can LLMs express their uncertainty? an empirical evaluation of confidence elicitation in LLMs. In The Twelfth International Conference on Learning Representations, 2024.
Jiexia Ye, Weiqi Zhang, Ke Yi, Yongzi Yu, Ziyue Li, Jia Li, and Fugee Tsung. A survey of time series foundation models: Generalizing time series representation with large language model. ArXiv preprint, abs/2405.0233 .22355 2024. URL httpt::ts///att /ttp://aaivvvv/aa org/ abs 2405. 02358
Wenqian Ye, Yunsheng Ma. Xu Cao, and Kun Tang. Mitigating transformer overconfidence via Lipschitz regularization. Uncertainty in Artificial Intelligence, pp. 2422-2432. PMLR, 2023.
Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 11106-11115, 2021.
14

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 LLM USAGE STATEMENT
LLMs used by the authors tool to assist in producing code for experiments. LLMs were not used to aid in the writing of the paper or during research ideation.
A.2 ADDITIONAL DETAILS ON EXPERIMENTAL SETUP
We report the default experiment parameters used in the main paper results in the following table, where the train/test split size varies by dataset, with train sets containing approximately 700 time steps per time series:
Table 2: Experimental setup details per dataset.
Prediction
Context Size (7)
Seasonality
Train/test split
Length (L)
(ARIMA only)
Reviews
64
512
24
2020-01-31 23:00
Shopping (M5)
64
512
7
2015-04-23
Glucose
64
512
1
2020-02-16 12:43
Heart-Rate
64
128
1
2000-01-01 00:04:59
Crime
64
512
7
2015-01-01
Patents
64
128
1
2004-01-01
Prediction length is fixed at 64 time steps across all datasets for short-term forecasting experiments while long-term forecasting experiments used a desired forecast length of 256 time steps.
Baseline models (N-BEATS and ARIMA) use the training data (dates before train/test split column) for parameter selection and training. The train/test split was determined based on the minimal size of the train set for training N- N-BEATS and ARIMA models, limited by time series length. Models forecast the first 64 time steps the test set using the end of the training set as the context. The forecast date and non-AR context are then shifted by the stride, forecasting an additional 64 time- steps. The models forecast repeatedly until the end of the time series. For a stride d E (1,4,8) dependent on time series length, the first forecast predicts 11-11742/11.7 while the second forecast is shifted by the stride to ÿT+ +1+11+d:: L+ ( ddd
Seasonality is hyperparameter for the ARIMA models, used for initialization. We limit the max lag (context) for the ARIMA model to 64 for all datasets. We train an ARIMA model with AutoARIMA on the train and then use the selected hyperparameters to train the models for evaluation. We train a new ARIMA model each time we shift the context.
For N-BEATS we perform grid search a variety of hyperparameters (see Table using the training data and select the model with the lowest WQL on the same data. Quantile loss outperforms normal distribution loss on all datasets. When forecasting with distribution loss in N-BEATS, use the default 100 samples. We train a single N-BEATS model each dataset, training across multiple time series and reuse the same learned parameters each time we shift the context.
Table 3: N-BEATS hyperparameter grid search space.
Hyperparameter
Search Space
Epochs
{100, 1000} (0.001, 0.00013 {-1, 2} ((1,1,1), (3.3.3) {10, 100)
Learning Rate
Early Stop Patience Steps
Number of Blocks
Validation Check Steps
Loss Function
(Normal Distribution Loss, Quantile Loss)
We describe the AR methods in depth in the next section and used n = 100 as the number of trajectories for the trajectory AR method. When sampling from the quantile heads in the trajec-
15

---

Published as a conference paper at ICLR 2026
tory method, we linearly interpolate between the quantiles convert the quantile heads into to continuous distribution.
A.3 AUTOREGRESSIVE METHODS
The naive AR method used by TimesFM and Chronos-Bov adds only the median or mean forecasts to the context. For example if the desired forecast length is L 256 and the model's forecast horizon is H = 64, the model will generate a probabilistic forecast 1:741:4464 given the observed context уı.r. To make the next 64-step forecast, it adds the mean median forecast, derived from 91+1-4+61- to the original context. The result is a shifted context 963:T+61 containing a mix of observed values and forecasted values. The model can then use the shifted context to produce the forecasts for 5T++5:T++28. This process is repeated until all 256 probabilistic forecasts have been completed, and the quantiles can be obtained directly from the probabilistic forecasts.
The branching approach from Moirai 2.0 is different in that it requires the model to forecast the same time steps multiple times. The first 64 time steps are normal, relying only on the observed contexts and producing [Q] quantile forecasts o #T+1:7-64 where q E Q. The context is duplicated or branched into [Q] new extended contexts, where each context is extended by the forecasts at specific quantile. The next 64 steps 4/.4 forecasted using all IQ] contexts, which requires running the model [Q] times independently, once for each of the |g] contexts. Because the model has been run |Q] times and each time the model produces [Q] quantiles, there will be a total of IQ forecasts for each time step. Rather than increasing the number of contexts exponentially from [QI to 1Q12 the IQ quantile forecasts at each time step are aggregated back down to set of [Q] quantile predictions. This is done by taking the Q quantile of the [Q|2 forecasts (for each time step). These aggregated quantiles become the returned quantile forecasts and are used to extend and shift the corresponding [Q] contexts.
trajectory- AR method where the model generates n independent AR forecasts or trajectories. Each trajectory is produced similarly to the naive method where single point forecast (at each time step in H) is added to the context for AR. However, instead of using the mean or median, the model samples from the predicted density and uses the samples as the point forecast (one sample for each time step). The model will create n independent trajectories, each with separate contexts. The final quantile forecasts are produced by taking the quantiles of the n trajectories at each time step independently.
The Toto TSFM (Cohen et al. 2025
A.4 MODELS
We primarily selected models with diverse backbones that were sfute-of-the-ae on the GIFT-EVAL benchmark as of the time of writing (Aksu et al. 2024) and pre-trained on time series datasets.
TimesFM TimesFM (Das et al. 2024 uses decoder-only stacked transformer architecture, and provides probabilistic predictions pre-trained fixed quantile heads. The training objec- tive combines mean squared error (MSE) and quantile loss [yndmma & Athanasopoulos (2018); Gneiting al. (2007). The model is trained with larger output patches than inputs, and thus able to make joint predictions of forecast quantiles over lengths H < 128. We use the times fm-2.44--0 www0000000pyttrre version for experiments.
TTaha Aksu et al. 2025 is a recent update on Moirai 1.0 (Woo et al., 2024) transitioning from an mecoer--bssed transformer that uses mixture distribution projection head to a decoder-only backbone with quantile head. Moirai 2.0 is implemented using AR with multi-patch forecasting support. In our experiments, we used a patch size of 16 and set the model to jointly predict 4 patches at time for max prediction horizon of H = 64. We use the moirai-2 R---malll version for our experiments.
Moirai 2.0 Moirai 2.0
2 TTieeFFM checkpoint Hugging Face: http:://higgingffeecc//////
fm- 2.
500m pytooch
Moirai 2.0 checkpoint on Hugging Face:
sma
16

---

Published as a conference paper at ICLR 2026
Chronos-Bolt Chronos-Bou Bolt (Ansari al.) 2024b) is a foundation model with backbone from the T5 family of encoder-decoder language models Raffel et al. (2020). Real-valued time se- ries is tokenized into fixed vocabularies via scaling and quantization. Unlike its predecessor Chronos (Ansari et al.] 2024a) which used a categorical prediction head forecasting the tokenized patches within
vocabulary, Chronos-Bolt directly predicts quantiles using an output patch length of 64. We use the chronos -bolt base version for our experiments,
TiRex TiRex (Auer et al., 2025) deviates from most TSFMs by using an xLSTM (Beck et al, 2024) decoder-only backbone. They apply contiguous patch masking during pretraining to allow for long-term forecasting without the need for AR. For long-term forecasts at inference time, TiRex pads and shifts the context that the model does not need condition on previous forecasts. Their implementation allows for multi-patch accelerated roll-out by using the forecasts at multiple patches in single forward pass, rather than only using single patch before padding the context. We use the base TiRex version for our experiments and set the accelerated roll-out to 4 patches with patch size of 32. F]
YingLong YingLong (Wang al. 2025) employs bi-directional U-net encoder-only transformer backbone. This enables the model to use delayed chain-of-thoughe reasoning by forecasting beyond the desired forecast length. The desired forecast then condition on both the observed context and the future delayed chain-of-thur of-thoof- for more accurate predictions. We use the YingLong Cm version for our experiments and use an extended forecast horizon of 2048.
N-BEATS N-BEATS (Oreshkin et al. 2020 is deep neural architecture that has been designed for the purposes of time series predictions. Similar to TimesFM, N-BEATS jointly forecasts the entire prediction horizon in a single forward pass.
ARIMA We use Nixtla's Statssorecast implementation of AutoARIMA (Hyndman & Khandakar 2008) to automatically select the optimal ARIMA parameters for each time series on the training set. The model is then refit on all earlier data before each forecast on the evaluation set. The ARIMA implementation
Kalman filters recursively predict the mean and variance. Quantiles are computed by fitting normal distribution to the forecasts and using the inverse CDF (PPF) with the appropriate z-scores.
A.5 PREDICTION HEADS
To evaluate the impact the prediction head has on calibration, we replaced the default quantile pro- jection heads of each model with a fine-tuned version of each head we tested. To train each head, we cached the embeddings outputted by each model's backbones and used them as the input each of the heads. Specifically, we trained an independent quantile, Gaussian, Student's 's t. and mixture distribution head. For the mixture head we replicate the Moirai 1.0 (Woo al., 2024) prediction head using Gaussian, Student's 's f, log-normal, and replace Negative- distribution due to issues related to model convergence during training.
Negative-Binomia Binomial with the Laplace
A.6 DATASETS
We evaluate models on three human behavior datasets: (i) Reviews dataset consisting of hourly counts of Amazon product reviews (Hou et al. 2024) and Google Places reviews (Li et al. 2022). (ii) a modified Shopping (M5) dataset (Makridakis al. 2022 consisting of the daily number of products being sold at different locations, and (iii) an NYC Crime report dataset (New York City
Police Department 2025) aggregating daily crime occurrences.
For the Reviews ensemble dataset, we aggregated Amazon Hou et al. 2024 and Google Li et al. 2022) reviews by product and location category and sampled 239 the most abundant categories
'Chronos-Bolt checkpoint on Hugging Face: htteoss/hhgggigggaeee.e/////////
bolt bost-base!
"TiRex checkpoint on Hugging Face: http://s///ghgggaaaasso/////////
"Yingl checkpoint on Hugging Face:
httpss//huggingfaee..o//////
110ml
17

---

Published as a conference paper at ICLR 2026
from the ensemble. We binned the event data count data an hourly granularity and trimmed the dataset to the most active time range from 2020-01-01 08:00:00 to 2021-06-25 23:00.
We aggregated the daily Shopping (M5) Makridakis al. (2022 dataset reduce sparsity by binning each product by their product department and store ID, totaling 70 time series.
We aggregated the NYC Crime reports dataset New York City Police Department (2025) by number of daily reports and cutoff the dataset to only include reports between the years 2006 and 2023. We split the dataset into time series based borough.
Many datasets used in training TSFMs are related to human behavior and natural phenomena that exhibit periodic trends (e.g., 24-hour effects). As noted in Gu et al. (2025), the strong predictive accuracy of foundation models on many of the datasets used in machine learning evaluations does necessarily translate into high predictive accuracy in applications such as vital sign forecasting in healthcare. To analyze model performance on datasets that differ in this respect from the pre- training data, we further included datasets for Glucose Level (Cho et al., 2023) and Heart Rate prediction (Peng et al. 1999).
The Glucose Clucose/Chocoo et al. (2023) dataset measures interstitial glucose concentration of 16 subjects over the course of T0 days. We used the Dexcom G6 dataset measuring interstitial glucose concentration (mg/dI.) every minutes.
The meditative Heart-Rate dataset [eng et al. (1999) records heart-rate of 14 volunteers during a 10-minute metronomic breathing meditation session, where it is recorded as relative time since the start of the meditation session. However, some foundation models (TimesFM) require and use timestamps for forecasting so we mapped the Heart-Rate dataset to start at 2000-01-01 00:00:00 and last 10 minutes.
To evaluate models on coarser time granularities, we also use the Patents dataset (Marco et al. 2015) which counts the number of US patents filed per month from 1981 to 2014. We removed sparse time series, aggregated by patent field and category, and filtered out timeseries with a minimum value of less than 100.
Additional dataset statistics can be found in the partial autocorrelation plots in Figure 6 evaluate the AR experiments the Heart-Rate and Patents dataset they did not contain enough data to accurately assess long-term forecasting calibration.
We did not
single time series with length 4367 and the train/test split at 1440. The IID noise time series was generated using yt = and the noisy first-order linear process 1/t 03/ 1 + (1 ar)e where a 0.9 and is drawn from the standard normal distribution (00 1).
Both synthetic noise datasets were generated
18

---

Published as a conference paper at ICLR 2026
Reviews
Shopping (M5)
Crime
Correlation
-1.0
-1.0
-1.0
Heart-Rate
Glucose
Patents
Correlation
Lag
Lag
Figure 6: Partial Autocorrelation Plots: Glucose and Patents datasets have limited number of lags with significant partial aufocorrelation while having large correlation at the first lag. The naive model is well suited for these two datasets which would explain the poor point forecasting performance (MASE). The specific time-series from each dataset used to generate the plots were selected based having approximately median MASE scores in that dataset.
19

---

Published as a conference paper at ICLR 2026
B ADDITIONAL RESULTS AND FIGURES
B.1 BASE FINDINGS
We include additional figures and findings for each of the experiments below. Table shows the short-term calibration and accuracy scores of each model across all of the datasets.
In Figure we show that across time series in the same dataset, the model point accuracy, MASE, are not correlated with model calibration. However, across datasets as in Figure 9 we see strong correlation between TSFM, MASE, and PCE with a correlation coefficient 0.90. This relationship does not always hold. For example, despite the TSFMs having poor point accuracy the Glucose dataset, worse than naive predictor with MASE greater than 1.0, they are still well calibrated with PCE less than 0.05. We note that for multi-step predictions (L > 1) that the MAE of the naive predictor the denominator of MASE is effect using information "from the future," " with the result that the overall MASE may take values greater than even for useful models. In Figures and as the TSFMs predict further into the future, not surprisingly the forecasts become both less accurate and less calibrated. We exceptions in Reviews where the TSFMs remain equally well calibrated up to a forecast length of 64. However, unlike the TSFMs, the baseline models have consistently high PCE over all prediction lengths.
The TSFMs are not consistently over- nor under-confident (see Figures and 12 However among the TSFMs, YingLong more confident with tighter intervals compared the other TSFMs, with some indication of overconfidence on Heart-Rar Rate and Patents datasets.
Reviews
Shopping (M5)
Crime
2xx
00
11P Box
.PP
Acorney
7 D0
6: 00
10P
Smer
10
50
12
24 Horizee (Prediction) Leugth
Hor am (Prodction) Length
Harion (Predktton) Longth
Mekra
Magglog
Figure 7: TSFM point accuracy gets increasingly worse the further out the model forecasts Mean Absolute Scaled Error (MASE) as a function of prediction length across the six datasets. The dashed horizontal line is the MASE the naive predictor (y) = 91-1) as reference.
20

---

Published as a conference paper at ICLR 2026
Table 4: Calibration evaluation results. Best results are highlighted in bold, and second best results are underlined.
Foundation Models
Baselines
Chronos-Boll Moirai
TimesFM
TiRex
YingLong
N-BEATS ARIMA
Reviews
0.717
0.748
0.720
0.710
0.823
1.416
1,211
Shopping (M5)
0.785
0.727
0.753
0.743
0.835
0.791
0.854
Glucose
6.362
5.986
6,051
5.841
6.041
7.326
5.958
Heart-Rate
1.119
1.118
1.104
1.096
1.257
1.335
1.120
MASE
Crime
0.741
0.759
0.759
0.740
0.761
0.767
0.829
Patents
11.896
10.639
12.193
12.510
21.297
22.814
14.534
IID Noise
0.706
0.705
0.707
0.708
0.709
0,926
0.704
Linear Process
2.491
2.417
2.210
2.325
2.322
2.790
2.166
Reviews
0.015
0.013
0.012
0.012
0.014
0.061
0.176
Shopping (M5)
0.033
0.045
0.036
0.031
0.052
0.108
0.131
Glucose
0.036
0.033
0.050
0.031
0.042
0.092
0.141
Heart-Ratc
0.063
0.051
0.052
0.048
0.077
0.121
0.204
PCE
Crime
0.023
0.035
0.023
0.021
0.014
0.103
0.168
Patents
0.176
0.154
0.206
0.192
0.259
0.242
0.247
IID Noise
0.009
0.008
0.004
0.012
0.010
0.072
0.122
Linear Process
0.033
0.015
0.008
0.022
0.021
0.138
0.122
Reviews
-0.012
0.007
0.004
-0.013
-0.024
-0.040
-0.351
Shopping (M5)
0.002
0.000
0.005
-0.026
0.051
-0.203
-0,259
Glucose
-0.011
-0.050
-..08
-0.027
0.034
-0.172
00,800
Heart-Rate
-0.061
-0.008
0.001
-0.037
0.105
-0,243
-0.111
CCE
Crime
-0.030
-0,020
-0.011
-0,041
0.015
-0.207
-0.337
Patents
0.028
0.046
0.089
0.069
0.201
0.322
0.031
ITD Noise
0.007
-0.008
0.007
-0.012
0.016
0.111
-0.244
Linear Process
0.006
-0.020
0.002
0.044
0.256
-0.244
Reviews
0.198
0.204
0.180
0.195
0.248
0.573
1.138
Shopping (M5)
0.240
0.231
0.241
0.260
0.233
0.538
0.644
Glucose
0.996
0.972
1.001
0.911
0.761
1.727
1.967
Heart-Rate
0.716
0.605
0.585
0.653
0.496
1.416
2.174
SIW
Crime
0.278
0.276
0.261
0.290
0.246
0.553
0.852
Patents
0.108
0.096
0.049
0.092
IID Noise
0,991
1.035
0.999
1.051
0.977
1.118
2,026
Linear Process
1.107
1.164
0.985
1.110
0.930
0.629
1.942
Reviews
56.732
$9.070
56.929
56.141
65.569
98.865
128.830
Shopping (M5)
59,558
55.129
57.137
56.454
63.402
56.402
63.618
Glucose
61.234
57.926
58.157
56.564
58.398
64.942
56.257
WQL
Heart-Rate
19.410
19.332
18.967
19.011
21.812
22.679
27.851
Crime
39.293
40.129
40.198
39.426
40.409
37.809
47.707
Patents
22.549
20.958
24.341
24.435
45.050
47.238
25.886
21

---

Published as a conference paper at ICLR 2026
Shopping (MS)
ta (PCE)
1011
1077
24 Ponents
Heart-Fore
Giucose
Callerne kne (92))
10-1
5 1011
6x 1011
2xx 1011
ur1
56
26
30
o0 (Prokston)) Longt
Marion (Prelction) Lengie
Horize (Prediction) Length
Figure 8: TSFM calibration gets increasingly worse the further the model forecasts Proba- bilistic Calibration Error (PCE) as a function of prediction length across the six datasets. PCE values below the dashed horizontal line (y 0.05) can be considered to be well-calibrated.
Calibration Error (PCE)
10-1
10
10
10'
Accuracy Error (MASE)
Chronos Bolt
Moirai 2.0
TimesFM
TiRex
YingLong
Reviews
Shopping (M5)
Crime
Heart-Rate
Glucose
Patents
Figure 9: Aggregated across each dataset TSFMs have correlation between point accuracy (MASE) and calibration (PCE), Probabilistic calibration error (PCE) versus point-forecast accu- racy (MASE) averaged over all time series in each dataset.
22

---

Published as a conference paper at ICLR 2026
Linear Process
|ID Noise
3 10"
10P
Accuracy Error (MASE)
9 10-1
100
8 10
10P
7.x 10
1.0
12
24
36
48
64
1.4
12
24
36
48
64
Linear Process
Noise
Calibration (PCE)
10-1
10-2
10"2
24
24
Linear Process
IID Noise
0.3
0.2
Over /Under- Confidence (CCE)
0.2
0.1
0.0
0.0
0.1
0.1
0.2
144
12
24
36
48
64
1.4
12
24
36
48
64
Linear Process
IID Noise
2.0
2.00
Interval Width (SIW)
1.5
1.50
1.00
24
36
48
64
1.4
12
24
48
Horizon (Prediction) Length
Horizon (Prediction) Length
Chronos- Bolt
Moirai 2.0
TimesFM
TiRex
Yinglong
N-BEATS
ARIMA
Figure 10: TSFMs are well calibrated on synthetic datasets while ARIMA is under-confident predicting higher variance (SIW). Plots calibration and accuracy metries as a function horizon length for synthetic datasets.
23

---

Published as a conference paper at ICLR 2026
Reviews
Shopping (M5)
Crime
Heart- Rate
Calorrtion Ener (PCE)
toms
10m4
10-1-
10
10-1
10
204% la 30
Glucose
Potents
HDD Noise
Linear Process
Cálbvret in Erer (MCE)
1gm4
10-1
10-9
10
10
10!
10
10-1
22
28
Accurscs
Accuracy
Acourccy Enior (MMSE)
Accuracy Enror (MASE)
Chronos Bo
M.
TimsFM
TiRcx
YindLor
N-BEATS
ARIMA
Figure 11: TSEMs are better calibrated than the baselines. Within each dataset, there is no correlation between MASE and PCE. Probabilistic calibration error (PCE) versus point-forecass accuracy (MASE) across the six datasets. Each dot represents model performance on an individual time series; larger centroids being the average over all time series a dataset. PCE values below 0.05 (dotted horizontal line) well-calibrated while MASE values less than 1.0 (dotted horizontal line) are better than the naive predictor. MASE values greater than 1.0 can still be a useful model due to the naive model using information "from the future."
Reviews
Shopping (M5)
Crime
0.2
Over- Under Confidence (CCE)
0.0
-0.2
0.2
-0.2
-0.3
-0.3
-0.4
10P
10a
10
10
10a
Heart-Rate
Glucose
Patents
0.2
0.1
01.4
Over- Feer/Under Confidence (CCE)
0.0
01.2
00.1
0.2
-0.22
0.2
-0.4
0.4
170g
10a
6
10
10a
10
10
10-1
10"
Interval Width (5IW)
Interval Width (5IW)
Interval Width (SIW)
Chronos Holt
Moirai 2.0
TimesFM
TiRex
YingLong
N-BEATS
ARIMA
Figure 12: TSFMs tend to be neither over/anner-cooffdent overall. Models with larger SIW, wider confidence intervals, were more under-confident. Centered Calibration Error (CCE) versus Scaled Interval Width (SIW) across the six datasets. Each dot represents model performance on an individual time series; larger centroids being the average over all time series in a dataset.
24

---

Published as a conference paper at ICLR 2026
Reviews
Shopping (M5)
Crime
Calibration Error (PCE)
0.10
0.08
0.15
0.15
0.06
0.10
0.10
0.05
0.77
0.79
Heart-Rate
Glucose
Patents
Calibration Error (PCE)
0.10
0.25
0.08
0.08
0.20
0.06
0.06
0.01
0.04
0.10
1.20
1.25
6.2
2.00
2.50
Accuracy Error (MASE)
Accuracy Error (MASE)
Accuracy Error (MASE)
Chronos-Bolt
Moirai 2.0
TimesFM
TiRex
Yinglong
Quantiles
Gaussian
Student's
Moxtre Distribution
Figure 13: All prediction heads tend to have very similar accuracy error with the Gaussian head occasionally having slightly worse error. Calibration and accuracy properties of various prediction heads with Accuracy Error (MASH) on the z-axis and Probabilistic Calibration Error (PCE) on the y-axis. Background markers are the results for each head (shape) with each backbone model (color), while the larger gray centroids are the mean results across models.
25

---

Published as a conference paper at ICLR 2026
B.2 COMPARISON WITH ADDITIONAL METRICS
As mentioned in the main paper, WQL, MSIS, and CRPS are common metrics for evaluating model calibration. Weighted Quantile Loss (WQL) is approximation of Continuous Ranked Probability Score (CRPS) defined as the pinball (or quantile) loss Pq scaled by the absolute sum of the true values:
(2- (1 9) (02 yE)
if 12 2 V1 if gf < Vt
(M, 01)
(5)
2 9. (yt- v.)
1 T+L
WQL Σ EET-1 lVel
Σ
Σ
Py (2t 02)
(6)
1=T+1
However, as described in Chung et al. (2021), CRPS and WQL,, measure combination of prob-
abilistic calibration and sharpness Gneiting et al. (2007). This combination leads to an imbalance often skewing prioritize predictive sharpness Chung al. (202I).
Mean Scaled Interval Score (MSIS) is scaled version of Mean Interval Score (MIS) which is the mean difference upper and lower bound prediction penalized with the error when the true value lies outside the bounds:
T+L Σ 1=
MSIS MAE
(Ui - Di)
(Li - y1)) y < Li]
(we. - 11) > U;]
MSIS has the same limitations being a measure of interval size with penalty term for observed values outside the interval Gneiting al. (2007): (Hyddman & Athanasopoulos (2018): Gneiting & Raftery (2007). We find that these metrics were highly correlated to MASE in Figure 15 Therefore, when using these metrics to evaluate calibration, their values result in a measure of sharpness and accuracy that diverge from measurement of calibration (see Figure 14h. For example, if we evaluate calibration using WQL or MSIS, we would incorrectly conclude that ARIMA is equally or better calibrated than the best foundation models the Glucose dataset as in Figure T and 14
26

---

Published as a conference paper at ICLR 2026
Calibration Error (PCE)
0:20
0.15
0.2
0.10
0.1
0.05
0.00
0.0
Reviews
Shopping (M5)
Crime
Heart-Rate
Glucose
Patents
Calibration Error (WQL)
Reviews
Shopping (M5)
Crime
Heart: Rate
Glucose
Patents
Calibration (MSIS)
3010
200
100
Reviews
Shopping (M5)
Crime
Hate
Glucose
Patents
1.5
Accuracy Error (MASE)
1.0
0.5
10
0.0
Reviews
Shopping (M5)
Crime
Heart: Rate
Glucose
Patents
Chronos-Bal Bolt
Moirai 2.0
TimesF M
TiRex
YingLong
N-BEATS
ARIMA
Figure 14: WQL and MSIS incorrectly imply that N-BEATS and ARIMA are well calibrated on some datasets, while PCE indicates they are always poorly calibrated. Top: Probabilis- tic Calibration Error (PCE) across datasets and models using the default quantile prediction head, Patents PCE uses the right y-axis scale. Upper Middle: Weighted Quantile Loss (WQL) measuring combination of sharpness and calibration. Lower Middle: Mean Scaled Interval Score (MSIS) where Glucose and Patents use right y-axis scale. Bottom: Mean Absolute Scaled Error (MASE) measuring point accuracy of median prediction, Glucose and Patents use the right y-axis scale.
27

---

Published as a conference paper at ICLR 2026
10
10"
10-11
10-1-
10
10'
10'
10'
Shopping (MS)
12P
10a
10'
Accuraey
10-1
10-1
10-1
6x
10
10"
10
10m2
Crime
106
107
10"
Accracy Eree (MASE)
6.x 10-1
xxx 10-1
6xx 10-
1x 16-1
3x -0
10-10-
6x:10
$x.10
30
2 107
4 300
Heart-Rate
Accunacy
2x 20-1
3xx 1x 10-1
6 60-00
4.x
5:x.10f Glucose
Azzuray Eror (MASE)
10P
8xx 103
6xx 10P
10-3
6.x 10-1"
2.x 101
3 10 4
10
101
Patents
107
10"
Accurrry
10
10'
10
10:1-
Calbration Eneor (WQQL)
Calliaziion (MSS5))
Calbbattoon Erro (PCE)
Chronos Bolt
Moral 2.0
TimeFFM
Rox
YingLong
BEATS
ARIMA
Figure 15: WOL and MSIS often highly correlated with MASE across time series within a dataset WQL (left), MSIS (middle), and PCE (right) compared to MASE where each dot is the performance of individual time series.
28

---

Published as a conference paper at ICLR 2026
B.3 TAIL FORECASTING
In downstream tasks such Anomaly Detection, calibration the tail ends of probabilistic predic- tions is more important than at the body predictive distribution. We evaluate the tailed calibra- tion with modified PCE and CCE that only considers the 0.1 and 0.9 quantile predictions rather than averaging over all the quantiles or confidence intervals. The Tailed PCE (TPCE) and Tailed CCE (TCCE) values are not directly comparable their aggregate values their scales could be slightly offset. Models that are in general overconfident with positive CCEs tend to be have an ex- aggerated overconfidence and miscalibration the tails. The opposite true with under-confident models, having improved calibration at the tails while their under-confidence CCE is capped and reduced by being the tail. In Figure T6 we find that YingLong which overconfident with overly tight confidence intervals, are relatively poorer at the tails of the distribution compared to the other models, while ARIMA sees relative improvement. For the TSFM models that are neither over nor under-confident,, their calibration looks to be improved with very minor calibration error. Promising future work should evaluate tailed calibration with quantiles further on the tail than 0.1 or 0.9 such as 0.001 0.999.
Calibration Error (PCE)
0.15
0.2
0.10
0.00
Reviews
Shopping (M5)
Crime
Heart-Ratt
Glucose
Patents
0.3
Tailed Calibration
Error (TPCE)
0.08
0.06
0.04
0.00
Reviews
Shopping (M5)
Crime
Heart-Rate
Glucose
Patents
Confidance (CCE)
Over: Under
0.0
-0.2
0.1
Reviews
Shopping (M5)
Crime
Heart: Rate
Glucase
Patents
0.6
Tailed Over /Under- Confidence (TCCE)
0.0
0.1
Reviews
Shopping (M5)
Crime
Heart- Rate
Glucose
Patents
Chronos-Ban Bolt
Moirai 2.0
TimesF M
TiRex
YingLong
N-BEATS
ARIMA
Figure 16: Tailed Calibration Error (TPCE) is generally smaller than Calibration Error (PCE) while Tailed Poer//nnde-Confiieen (TCCE) grows for overconfident models and reduces for under-confident models. Comparison of model calibration the entire probabilistic distribution versus calibration the tail-ends the distribution (0.1 and 0.9).
29

---

Published as a conference paper at ICLR 2026
B.4 ADDITIONAL FINDINGS FOR AR FORECASTING
In this section, we elaborate on two key issues that arise when choosing horizon length in autore- gressive time series forecasting models (TSFMs). Compounding errors degrade accuracy, and block- wise independence introduces bias in the input for future predictions.
1. Autoregressive errors compound. When forecasting multiple time points autoregressively, each predicted value is fed back as input for predicting subsequent points. Even small errors in early predictions can accumulate over the horizon, leading to progressively degraded forecasts. This is exacerbated when using smaller horizons as the error can accumulate faster.
2. Independent block modeling introduces bias. TSFMs model a block of H future time points as independent given past context:
i+1--1
pl D I
II p(ay(res).
While convenient for training, this ignores any intra-block dependencies in longer horizons. During autoregressive rollouts, the next block of predictions is conditioned on samples from the previous block. Because the sampled block does not preserve the true joint dependen- cies, the model sees biased context, resulting in a shift in the marginal distribution of later time points. This bias larger for longer blocks H (since then there are fewer biased inputs) and vanishes as H - h (where each time point is predicted independently in an autoregressive manner).
Reviews
Shopping (M5)
Over /Under Confidence (CCE)
0.15
0.2
0.10
0.05
0.1
0.0
0.05
16
128
32
64
128
Crime
Glucose
Over-/U Unnder Confidenca (CCE)
0.3
0.1
0.0
0.0
16
32
64
128
32
64
12B
Step Size (Horizon Length)
Step Size (Horizon Langth)
Chronos- Bolt
Branch
Moirai 2.0
Branch
TimesFM Branch
TiRex YingLong
non-AR
Chronos-
Bolt
Trajectory
Moirai 2.0
Trajectory
Tim TimesFM Trajectory
non-AR
Figure 17: TSFMs are consistently overconfident on long-term forecasting. The magnitude of the over confidence decreases with longer horizon lengths and when using the trajectory AR method. The figure compares Centered Calibration Error (CCE) for long-term forecasting using autoregression on the y-axis with AR prediction horizon on the a-axis. The color of the line depicts the model used and the line style indicates the AR method. The pink and blue dashed horizontal lines are the CCE for TiRex and Yinglong without using AR.
B.5 ADDITIONAL MIXTURE DISTRIBUTION HEADS
In addition to the four prediction heads included the main paper, Quantiles, Gaussian, Student's and Moirai 1.0 mixture distribution, we include experiments with two additional mixture heads:
30

---

Published as a conference paper at ICLR 2026
mixture of Gaussian distributions and mixture of Student's t distributions. Each mixture head con- tains five of their respective distributions.
Reviews
Shopping (M5)
Crime
Calbbration Error (PCE)
0,100
0.15
0.075
0.15
0.10
0.050
0.10
0.05
0.05
-0.2
0.11
0.0
0.4
0.2
0.0
-0.2
0.0
Heart-Rate
Glucose
Patents
Calibration Enror (PCE)
0.10
0.25
0.08
0.08
0.20
0.06
0.06
0.15
0.04
0.04
0.10
-..2
Over- Over-/Under-CCe Linder-Confidenc (CCE)
Over-/Under- Confeieeennneeeee (CCE)
Over- Operr/Under-Conideene (CCE)
Chronos-Bod Bolt
Moirai 2.0
TimesFM
TiRex
YingLong
Quantiles
+
Student's t
*
Student's 1 Mixture
Gaussian
X3
Moirai Mixture
Gaussian Mixture
Figure 18: Both mixtures of Gaussian and Student's t distribution heads maintain a similarly low calibration error the other non-Gaussian heads from the main paper. Calibration prop- erties of various prediction heads with Centered Calibration Error (CCE) on the æ-axis and Prob- abilistic Calibration Error (PCE) on the y-axis. Large positive CCE values indicate overconfident predictions, while large negative CCE values show under-confidence. Background markers are the results for each head (shape) with each backbone model (color), while the larger gray centroids the mean results across models.
B.6 ADDITIONAL. RESULTS ON GIFT-EVAL
To further supplement our experiments, we evaluated the TSFMs on subset of the GIFT-EVAL dataset (Aksu et al.) 2024). Due the GIFT-EVAL dataset containing heterogeneous data with inconsistent time series lengths and unbalanced sub-datasets, we filtered out time series with lengths less than 768 points (512 context and 256 prediction length) and sampled max of 1024 time series from each sub-dataset. In Figure 19 we show that the calibration results on this subset of the GIFT- EVAL dataset align with the results from the datasets included in the main paper.
31

---

Published as a conference paper at ICLR 2026
Accuracy (MASE)
20
GIFT-Eval
Reviews
Shopping (M5)
Crime
Heart- Rate
Glucose
Patents
0.201
0.15
0.2
Calibration
(PCE)
0.30
0.05
0.00
0.0
GIFT-Eval
Reviews
Shopping (M5)
Crime
Heart-Rate
Patents
Confidence (CCE)
0.2
Over- Under-
0.00
-0.2
GIF T-Eval
Reviews
Shopping (M5)
Crime
Heart-Rate
Glucose
Patents
Chronos-Bolt
Moirai
TimesFM
TiRex
YingLong
N-BEATS
ARIMA
The TSFMs have significantly reduced calibration error compared to the baseline methods on the GIFT-EVAL dataset similar to the other datasets. Top: Mean Absolute Scaled Error (MASE) measuring point accuracy eπrоr of the median prediction (lower better). Glucose and Patents use their own y-axis scale (on the right). Middle: Probabilistic Calibration Error (PCE) across datasets and models using the default quantile projection block. (Patents PCE uses its own y-axis scale (on the right)). Lower PCE values are better. Bottom: Centered Calibration Error (CCE) evaluating systematic overconfidence (positive) or under-confidence (negative).
Figure 19:
32


Reviews:
[{"review_id": "ncYjGo0GDi", "reviewer": "Reviewer_c1Kq", "summary": "The paper performs an empirical study on whether time series foundation models are calibrated. They select 5 TSFMs, Chronos-bolt, Moirai 2.0, YingLong, TimesFM and Tirex, as well as N-BEATS and ARIMA as baselines. Then, paper evaluates these models on 6 datasets, focusing on calibration metrics, specifically using Probabilistic Calibration Error, Scaled Interval Width (to evaluate sharpness), and Centered Calibration Error. Experiments show that TSFMs have better calibration than baselines, and a few other insights are presented.", "questions": "See weaknesses section", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The paper explores an understudied aspect of time series foundation models. It investigates a wide range of recent models across 6 different datasets. Ideas are well presented and the relevant experiments have been performed.", "weaknesses": "* Given that the field has been mostly motivated by accuracy metrics, it would be good to give more motivation in the introduction about why researchers should be interested in calibration.\n* It would be good to perform a larger scale investigation, for example computing these metrics for GIFT-eval, to get a more comprehensive understanding.\n* It would be good to add in mixture of gaussians and mixture of student-ts for Fig 4, given that Toto has been mentioned several times in the paper, and Toto uses mixture of student-t.\n* Section 4.4 is quite confusing, I am unclear why forecasts are more calibrated as horizon length increases. Fig 8 in the appendix also seems to contain contradictory findings. Also, why are tirex and yinglong straight lines?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "C2Jy5FIRKp", "reviewer": "Reviewer_NKRH", "summary": "This paper presents the first systematic empirical study examining the calibration properties of time series foundation models (TSFMs). Using appropriate metrics and datasets, the authors first evaluate the forecasting calibration levels of TSFMs, finding that they outperform traditional statistical models and exhibit better calibration. Furthermore, the paper investigates the effect of prediction head design on calibration, revealing that Gaussian heads tend to be under-confident in most cases and are outperformed by alternative head types. The study also explores the impact of forecasting horizon length on calibration, showing that for long-term autoregressive forecasting, increasing the horizon length and using the trajectory AR method leads to better-calibrated forecasts.", "questions": "1. Do inherent data characteristics such as randomness and autocorrelation influence the calibration evaluation of TSFMs?\n\n2. The paper finds that TSFMs are not systematically over- or under-confident, which contrasts with observations in image and text domains. What is the possible reason for this difference?\n\n3. A low calibration error does not correspond to a low forecasting error, as seen in the Heart Beat and Patents datasets. What causes this discrepancy between the metrics?\n\n4. What could be the underlying reason that all autoregressive TSFMs are consistently overconfident in long-term forecasting? Whether are there measures could be taken during training to mitigate this bias?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Appropriate experiment setting: The work identifies the shortcomings of commonly used metrics (CRPS, WQL, MSIS) that conflate calibration and sharpness. By prioritizing PCE and CCE, the paper provides a clearer measure of true probabilistic calibration.\n\n2. Novel insights: The study offers meaningful analysis and insights in each experiment. In particular, it systematically investigates the calibration properties of TSFMs, and explores how factors such as prediction head design and forecasting horizon influence calibration—topics that have not been examined in prior work\n\n3. Clear organization: The paper is well-structured, with logically arranged sections and a coherent flow, making it easy for readers to follow the results and key takeaways.", "weaknesses": "1. Section 4.3 and 4.4 lack clear motivation for investigating the influential factors affecting calibration. Why did the authors choose to focus on the impact of prediction head design and forecasting horizon, rather than other factors such as context length or data characteristics ?\n\n2. The investigation of the calibration properties of TSFMs remains at the validation stage and does not further explore the underlying reasons why TSFMs tend to be well-calibrated, whereas traditional statistical models are often overconfident. A deeper analysis regarding the mechanisms behind this difference would make the work more insightful.\n\n3. While the finding of general insensitivity to the prediction head form is valuable, the scope of prediction head selection is limited. Recently, more novel designs like flow-matching prediction head [1] could also offer new insights into calibration behavior.\n\n[1] Liu Y, Qin G, Shi Z, et al. Sundial: A family of highly capable time series foundation models. arXiv preprint arXiv:2502.00816, 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0h00JmjBuh", "reviewer": "Reviewer_dAcF", "summary": "The paper targets an underexplored but crucial aspect of time-series foundation models (TS FMs): calibration. It conducts a systematic evaluation of five recent TS FMs against two strong baselines, examining (i) overall calibration and confidence (over/under), (ii) the effect of different prediction heads, and (iii) calibration behavior under long-horizon autoregressive forecasting.", "questions": "1. For different prediction heads, it looks like they are similar in terms of calibration error except Gaussian prediction heads. But I wonder how the forecasting performance changes with different prediction heads. Because I feel if the calibration errors are similar, then we should choose the one with better forecasting performance when training a time series foundation model.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The authors investigate the calibration properties in time series foundation models. The topic is of interest in the community.\n2. The experiments are comprehensive and confirm that time series foudnation models are well-calibrated.", "weaknesses": "1. Although it is an interesting topic and well motivated, but the manualscript does not provide technicial contributions. The main finding is that time series foundation models are well-calibrated compared with traditional and machine learning / deep learning models. It would be better that if we have some contributions on how to further improve the calibration ability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Bb789zDzTH", "reviewer": "Reviewer_cuh6", "summary": "This paper investigates the calibration of time series foundation models. The study analyzes five foundation models and two non-pretrained baseline models, evaluating their calibration performance, the impact of different prediction heads, and calibration under long-term autoregressive forecasting. The findings reveal that time series foundation models consistently demonstrate superior calibration compared to baseline models, exhibiting neither systematic over-confidence nor under-confidence.", "questions": "- In Equation 3, it appears that a curly brace is missing to enclose all the content after the $\\sum_{s \\in S}$.\n- In Section 4.3, the authors concluded that the calibration performance of the Gaussian distribution is inferior to that of other distributions. Could you provide a more in-depth explanation for this phenomenon? Generally, errors are widely assumed to follow a Gaussian distribution. But why is the calibration performance of the Gaussian distribution the poorest?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is well written and easy to follow.\n- The problem is interesting: calibration is an important problem, yet not commonly discussed by the community.\n- The evaluation metrics set in the paper are comprehensive, and a series of very interesting conclusions have been drawn.", "weaknesses": "- This is a highly experimental paper with substantial experimental content. But it lacks a deep analysis. For example, in Section 4.4, the author's experimental results indicate that \"the predictions with a shorter forecast horizon $p$ have poorer calibration\", yet no thorough analysis is provided. It would be interesting to formally know how the \"horizon length\" affects the prediction performance and why.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)