There are no Champions in Supervised Long-Term Time Series Forecasting
Authors: Lorenzo Brigato
Rafael Morand
Knut Joar Strømmen
Maria Panagiotou
Markus Schmidt
Stavroula Mougiakakou
Venue: Accepted by TMLR ()

Abstract: Recent advances in long-term time series forecasting have introduced numerous complex supervised prediction models that consistently outperform previously published architectures.
However, this rapid progression raises concerns regarding inconsistent benchmarking and reporting practices, which may undermine the reliability of these comparisons. In this study, we first perform a broad, thorough, and reproducible evaluation of the top-performing supervised models on the most popular benchmark and additional baselines representing the most active architecture families. This extensive evaluation assesses eight models on 14 datasets, encompassing $\sim$5,000 trained networks for the hyperparameter (HP) searches. Then, through a comprehensive analysis, we find that slight changes to experimental setups or current evaluation metrics drastically shift the common belief that newly published results are advancing the state of the art.
Our findings emphasize the need to shift focus away from pursuing ever-more complex models, towards enhancing benchmarking practices through rigorous and standardized evaluations that enable more substantiated claims, including reproducible HP setups and statistical testing. We offer recommendations for future research.

Published in Transactions on Machine Learning Research (01/2026)
There are no Champions in Supervised Lerg-Term Time Series Forecasting
Lorenzo Brigato', Rafael Morand 1,.3+ Knut Strømmen
Maria Panagiotou Markus Schmidt', Stavroula Mougiakakou'
ARTORG Center, University of Bern," Graduate School for Cellular and Biomedical Sciences, University of Bern
Center for Experimental Neurology, Departmend of Neurology, Bern University Hospital
Reviewed on OpenReview: https: openreview. net/ forum? <d=001JuBp
Abstract
Recent advances in long-term time series forecasting have introduced numerous complex supervised prediction models that consistently outperform previously published architectures. However, this rapid progression raises concerns regarding inconsistent benchmarking and reporting practices, which may undermine the reliability of these comparisons. In this study, we first perform broad, thorough, and reproducible evaluation of the top-performing supervised models on the most popular benchmark and additional baselines representing the most active architecture families. This extensive evaluation assesses eight models on 14 datasets, encompassing ~5,000 trained networks for the hyperparamcter (HP) searches. Then, through comprehensive analysis, we find that slight changes to experimental setups current evaluation metrics drastically shift the common belief that newly published results advancing the state of the art. Our findings emphasize the need to shift focus away from pursuing ever-more complex models, towards enhancing benchmarking practices through rigorous and standardized evaluations that enable more substantiated claims, including reproducible HP setups and statistical testing. We offer recommendations for future research.
1 Introduction
Long-lerm time series forecasting (LTSF) is critical across var- ious domains, including energy management (Weron, 2014), financial planning (Sezer et al., 2020), and environmental modeling (Soni et al., 2024). Accurately predicting future values time series data enables better decision-making and resource allocation. LTSF remains challenging due to the complex temporal dynarmics, including trends, seasonality, ir- regular fluctuations, and significant variability across datasets (Qiu et al., 2024; Shao et al., 2024).
ETTh2
Weather
ETTh1
Electricity
100%
Australia Rainfail
0%
Exchange
Motor Imagery
BeijingAir
Recent advances in deep learning have improved LTSF capa- bilities, and the field is currently witnessing an exponential surge in publication rates (Kim et al., 2024). Popular research directions within the field include supervised LTSF, which involves training and testing models on ITD data from histor- ical time series (Wang al., 2024a; Liu al., 2024a; Wang et al., 2025), in contrast to the pre-training and zero-shot fine-tuning paradigms introduedd by foundation ion models (Cao et al., 2024; Woo et al., 2024). From an architectural
Benzene Contentration
KDDCup2018
Pedestrian Counts
TDBrain
DLinear
TimeMixer
Transformer
PatchTST
TimeXer
XLSTMTime
ModernTCN
SMamba
Figure 1: There is no champion. The rela- tive MSE averaged over all forccast horizons re- veals that no model dominates on all datasets.
"Equal contribution. Fmails: (name). (1astnane] Bunibe.ch

---

Published in Transactions on Machine Learning Research (01/2026)
perspective, transformer models have been adapted to time series forecasting with innovative modifications, such as univariate patching (Nie et al., 2023) and attention mechanisms tailored to exploit inter-variate dependencies (Liu et al., 2024a; Wang et al., 2024c). In addition, models leveraging multiscale signal mixing (Wang et al., 2024a), Fourier-based 2D decomposition (Wu et al., 2023), state-space modeling (Wang et al., 2025), 1D convolution (Luo Wang, 2024), and novel recurrent processing (Beck et al., 2024; Alharthi & Mahmood, 2024) have expanded the field.
However, we claim that the field is facing significant challenges regarding fair benchmarking, transparent reporting, and guidelines for model selection. Although this problem is not uniquely encountered in LTSF (Eriksson et al., 2025; Herrmann et al., 2024), we highlight the specific challenges in LTSF to promote discussions towards improvements in the field, as it was done in other disciplines (Bechler-Speicher et al., 2025; McIntosh et al., 2025; Sarfraz et al., 2024; Wu Keogh, 2023). In this work, we focus on supervised LTSF and therefore exclude foundation models, as their evaluation requires distinct experimental protocols involving largo-scale pro-training with potential data leakage concerns (Aksu et al., 2024). We observe inconsistencies in test. selups across different benchmarks, biased comparisons, and challenges with reproducibility that hinder fair performance assessment in the field. Moreover, marginal performance gains in recent literature cast doubt on the practical value of increasingly complex model architectures (Zeng ct al., 2023). To support our claim, we conduct comprehensive, rigorous, and reproducible evaluation of the top-performing models on the most widely used benchmark, encompassing five models and three additional baselines representing the most popular neural architectures over 14 datasets (~5,000 trained networks for the HP searches). Our results reveal that no single model consistently outperforms all the baselines (Fig. 1), directly challenging the prevailing narrative of new architectures consistently surpassing competing models across all domains (Liu et al., 2024;; Nic et al., 2023; Wang et al., 2024a;e; Wu et al., 2023; Alharthi & Mahmood, 2024). The findings of our work emphasize the need to shift focus away from pursuing ever-more complex models and towards enhancing benchmarking practices through rigorous and standardized evaluation methods. We analyze the potential reasons behind this phenomenon and propose recommendations to help the field progress. To foster http://tttt......//tttt/t/
reproducibility, our code is available at tttps://githan com comm/ALHlLaa/NNoChaaps The contributions of our paper are as follows:
We question the narrative of consistently dominated supervised LTSF benchmarks (Soc. 2), and support our claim with results obtained through a compreleensive and reproducible experimental setup (Sec. 3).
We challenge previous guidelines for model selection based on dataset characteristics (Sec. 3) and highlight the need for further research this direction (Sec. 5).
We investigate potential causes behind overstated claims by carefully analyzing our experimental setup and those from prior work in the literature (Sec. 4), and offer recommendations to help prevent similar issues in future work (Sec. 5).
2 Field overview
We provide an overview of recent advancements in LTSF, focusing on current benchmarks (See. 2.1) and emerging champions (Sec. 2.2). Due to space limitations, additional related work on recent time series forecasting models and paradigms is included in Appendix A.
2.1 Benchmarks and their recommendations for dataset-guided model selection
TSLib (Wang et al., 2024b) compares 12 deep lcarning models across five tasks: classification, imputation, anomaly detection, and long//hhort-Lemn forecasting. For long-term forecasting, nine dalasets from four domains are used. Results are presented for two settings: unified hyperparameters (HPs) and an HP search per model, but details on paramcters, context length, forecast horizon, or the scarch process are missing. The evaluation metric the mean squared error (MSE) averaged across datasets. The authors claim that their results clearly demonstrate the superior forecasting capabilities of transformer models, particularly
2

---

Published in Transactions on Machine Learning Research (01/2026)
iTransformer and PatchTST, despite arguably marginal improvements over MLP-based models, such N-Beats (Oreshkin et al., 2020). They further emphasize continued exploration of temporal-token methods.
TFB (Qiu et al., 2024) evaluates 22 statistical, classical machine learning, and deep learning methods using 25 multivariate and 8,068 univariate datasets. Based on their results, the authors claim that linear models outperform deep learning methods datasets with increased trends and distribution shifts. Conversely, transformers excel in datasets with marked patterns (e.g,, seasonality). Notably, PatchTST and DLinear (Zeng et al., 2023) consistently perform well across datasets, exhibiting major weaknesses.
BasicTS+ (Shao et al., 2024) incorporates 28 forecasting models, including 17 short-term forecasting (STF) and 11 LTSF models, across 14 widely used datasets. STF models encompass prior-graph-basee latent-graph-baesed, and non-graph-based meth while LTSF models consist of Transormmr-baseer and Suear-layer-based architectures. Models are implemented following publicly available architectures and HPs, with further tuning of paramcters like learning rate and batch size via grid search ensure performance is at least as good as reported in the original paper. Upon analyzing the results, the authors argue that dataset characteristies play major role in determining model performance. They claim that transformer models excel on datasets with clear, stable patterns, whereas simpler models like DLinear perform comparably on datasets without such patterns. The authors emphasize the need to address data distribution drift and unclear patterns instead of focusing solely on increasing model complexity. They suggest that this may indicate potential overfitting to commonly used datasets like ETT*, Electricity, Weather, and Exchange, which risks creating a misleading impression of progress. They conclude that careful dataset selection and curation are essential to advance the field.
GIFT-Eval (Aksu al., 2024) assesses five statistical forecasting models, cight supervised deep learning models, and four foundation models 21 LTSF datasets and 55 STF datasets. The implementations of the supervised deep learning models adhere to the original works. This benchmark enables simple assessments of model performances across several dataset characteristics domain, and forecast horizon). The authors found that PatchTST offered the most reliable results across all characteristics, while foundation models showed inconsistent performance, suggesting that these models are still in an early and relatively underperforming stage compared to well-tuned supervised approaches, a finding also corroborated by Xu et al. (2025).
aeaivaaiiate/ultivvriin
sampling frequency,
2.2 Emergent LTSF champions
Recent models have made a leap in LTSF perfor- mance (Liu al., 2024a; Nie al., 2023; Wang et al., 2024açe; Wu et al., 2023; Zeng et al., 2023). A popular benchmark for supervised LTSF is TSLib (Wang et al., 2024b), which, at the time of acceptance, has accumulated over 11.2k stars and 1.8k forks on GitHub, reffect- ing its wide adoption in the field. A series of new models 2024 has reportedly dominated the field. The current leaderboard in TSlib includes five models, originally published in top-tier machine learning conferences. We sum- marize the striking win percentages from their original works in Tab. and briefly describe each model in the following box.
Table 1: Model win rates in previous works. Winners in LTSF for forecast. horizons TE (96, 192, 336, 7203. The win rates (%) are according to each T without averaging.
TimeMixer reported unified parameters (A) and HP search (B) results. avg. over T and ETT datasets.
Model
Win % (MSE)
Win % (MAE)
DLinear (Zeng al., 2023)
50.0
16.7
PatchTST (Nie sl., 2023)
87.5
59.1
A) 93.8
A) 100
TimeMixer (Wang et al., 2024n)
B) 81.2
B) 81.2
Traauformer (Liu el al., 2024a)
33.3
47.2
71.4*
85.71
TimeXer (Wang et al., 2024e)
85.7
60.7
TSLib Long- Term Time Series Forecasting Leaderboard
We present top TSLib models under fixed and searched look-back settings, ordered by publication year: DLinear Zeng
AAAT-23: Linear madel introduced challenge transformers in early benchmarks.
PatchTST Nie et al. ICLR-23: Transformer with univariate patching of time series for tokenisation. TimeMixer Wang et al. JCLR-24: MLP-mixer that introducvd past-decomposable and future-predictor mixing.
iTransformer- Liu et al. (CLR-24: Transformer variant that attends across variates instead of patches.
TimeXer- Wang et al. NewrIPS-22 24: Transformer with dual attention for interactions on patches and variates.
3

---

Published in Transactions on Machine Learning Research (01/2026)
Table 2: Results. Mean values averaged over prediction lengths. Datasets span the energy, economy, transport, health, and environment domains. Best and sccond-best highlightod.
Model
DLinear
PatchTST
Trannforeer
TimeMixer
Timexer
S-Macnha
SLSTMTine
ModemnTCN
Metrie
MSE
MAE
MSE
MAE
MSB
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ETTM
0.474
0.477
0.414
0.482
0.424
0.443
0.422
0.443
0.425
0.44
0.467
0.466
0.535
0.532
ETTol
0.403
0.422
0.3914
0.431
0.408
0-437
0.432
0.449
0.412
0.445
0.409
0.434
0.434
0.449
0.414
0.436
ETT22
0.485
0.474
0.379
0.41
0.378
0.405
0.374
0.404
0.377
0.406
0.383
0.407
0.803
0.647
0.405
0.423
ETTm2
0.151
0.261
0.156
0.269
0.166
0.276
0.162
0.27
0.168
0.277
0.168
0.279
0.17
0.278
0.166
0.276
Electricity
0.162
0.259
0.161
0.261
0.262
0.156
0.254
0.17
0.268
0.160
0.275
Weather
0.244
0.
0.298
0.225
0.263
0.238
0.279
0.231
0.271
0.224
0.264
0.237
0.277
0.229
0.268
0.241
0.282
Exchange
0.389
0.434
0.368
0.409
0.379
0.416
0.55
0.476
0.374
0.412
0.426
0:438
0.600
0.761
Motorlmagery
4.592
1.183
3.775
1.006
1,62
0.083
3,800
1.011
3.649
0.915
0.745
0.244
6.622
1.362
3.088
0.769
TDBrain
1.151
0.802
0.882
0.722
0.978
0.722
0.981
0.722
0.97
0.719
0.972
0.72
1:211
0.750
0.984
0.725
BoijingAir
0.583
0.472
0.678
0.46
0.58
0.464
0.092
0.471
0.582
0.462
0.567
0.432
0.508
0.43
0.081
0.461
BeuzeneCaareatrntion
0.008
0.02
0.011
0.012
0.012
0.053
0.008
0.022
0.012
0.037
0.01
0.061
0.01
0.028
0.15
0.265
AustralinReicfall
0.838
0.751
0.855
0.757
0.849
0.754
0.853
0.735
0.847
0.753
0.845
0.753
0.827
0.743
0.872
0.766
KDDCup2018
0.997
0.03
1.086
0.617
1,088
0.618
1.035
0.631
1.045
0.61
1.085
0.686
1.051
0.628
1.068
0.636
PodestrianCounts
0.298
0.289
0.291
0.293
0.295
0.285
0.292
0.284
0.311
0.307
0.202
0.278
0,282
0,259
0.466
0.408
Average
0.77
0.484
0.691
0.438
0.541
0.116
0.712
0.162
0. 0.083
0. 0.153
0.484
0.407
0.971
0.519
0.714
0.19
Ranks
4.43
4.5
3.5
3.93
4.29
4.5
4.07
4.14
4.07
4.14
4.07
Beyond these leading models, additional neural architectures have recently gained attention within the community for their focus on efficient computation. Namely, these approaches include state-space Mamba models (Gu & Dao, 2024), recurrent models via the extended LSTM (xLSTM) architecture (Beck et al., 2024), and fully convolutional networks (Luo & Wang, 2024). We select three models that represent these architecture familics, were explicitly adapted or designed for LTSF, and gained rapid interest within the community (measured in terms of citations). Specifically, we include S-Mamba (Wang et al., 2025), xLSTMTime (Alharthi & Mahmood, 2024), and ModernITN (Luo & Wang, 2024).
3 Who is the real champion?
Motivating confirmatory research in LTSF. As seen in the previous section, recent papers often suggest. that newly proposed architectures outperform others across almost all tested datasots. However, the variability in results for the same algorithms and reliance on prior studies with different experimental selups raise questions about their consistent superiority. This concern goes beyond the field of LTSF and is well supported by a growing body of work criticizing the reliability and generalizability of empirical results in machine learning (Bechler-Speicher et al., 2025; Eriksson et al., 2025; Jordan et al., 2024; Sarfraz el al., 2024; Snoek et al., 2018; Stine, 2006; Wu & Keogh, 2023). In response, the community has increasingly recognized the need for more confirmatory research (Herrmann et al., 2024), i.e., empirical evaluations conducted by researchers without vested interest in a particular method, aiming to ensure fairness, minimize bias, and reduce overly optimistic conclusions. Motivated by the exponential surge in publication rates in time series forecasting (Kim et al., 2024) and the community quest for more neutral benchmarking (Herrmann et al., 2024), we selected the previously introduced five top-performing models from TSLib (Wang et al., 2024b), S-Mamba, xLSTMTime, and ModeruTCN to perform a comprehensive empirical evaluation across 14 datasets from various domains. For completeness, a comparison including classical statistical methods is presented Appendix H.
Datasets. We evaluate models on 14 datasets from five domains: Energy, Economy, Transport, Health, and Environment. They vary substantially in sample size, sampling frequency, number of variates, and various statistics for time series dynamics (Appendix B). The selection aims to minimize bias by preventing any model from gaining an unfair advantage due to specific dataset characteristics.
HP search. We searched HPs aligned with those described in TimeMixer (Wang et al., 2024a). Specifically, we optimized input lengths, learning rates, and the number of encoder layers. We used the Optuna framework (Akiba et al., 2019), employing the default TPEsampler for HP sampling and the Sucoessireefallvnngnunee the trial scheduler. We employed Adam with an exponentially decaying scheduler. For xLSTMTime,
4

---

Published in Transactions on Machine Learning Research (01/2026)
GIFT-Eval
TSLib
BasicTS+
TFB
1,000
Nature
Energy
Variates
100
Econ/Fin
Transport
Healthcare
Environment All
10
102
10*
10
102
10f
10
10a
104
100
102
10a
Samples per day
Samples per day
Samples per day
Samples per day
Figure 2: Potential lack in dataset diversity The benchmarks do not span wide range of frequencies and number of variates across domains.
we switched the loss function from MSE (default in TSLib) to MAE, following previous implementations (Alharthi & Mahmood, 2024; Kraus al., 2024), as we indeed observed worse performance using MSE preliminary experiments. The optimized HPs were used to train and evaluate the final model across three random seeds to ensure robust results. For the rest of the model HPs, default to the configurations provided in TSLib, We refer the reader to Appendix C for more detailed description.
Finding: There is no champion. We follow the TSlib benchmark and use MSE and mean absolute error (MAE) to evaluate model performance. Tab. 2 presents the MSE and MAE for each dataset, averaged over the most common forecast horizons (Liu et al., 2024a; Nie et al., 2023; Wang et al., 2024c), revealing results that differ substantially from recent papers where proposed algorithms often dominate. Instcad, our findings indicate no definitive best-performing model across all datasets and forecast. horizons (Tab. 12 and Tab. 13, Appendix H). To assess reliability, compared our results with the best-reported outcomes from the original studies on three common datasets ETT*, Electricity, and Weather and found that our HP search performed similarly or better, confirming the proper implementation and tuning of our baselines (Tab. 9, Appendix D). To present a comprehensive view that highlights both the optimal outcomes and the realistic performance variability, we report the minimum values (best MSE/MAE) and the averages. We further analyze He-scarch run-to-run variability in Appendix E.
Recommendations for dataset-guided model selection do not hold. We analyze whether our results align with the guidelines proposed by BasicTS+ (Shao et al., 2024) and TFB (Qiu et al., 2024) (Sec. 2.1), which aim to address the challenge of selecting appropriate models for specific datasets in LTSF.
Finding:
We revisit the example in (Shao et al., 2024) and assess the performance of a linear model (DLinear) versus a transformer model (PatchTST) on data with clear and unclear patterns, respectively. We use Ezchange as the dataset with an unclear pattern and Pedes- trianCounts as the dataset with a clear pat- tern (Fig. 6, Appendix B.2). PatchTST out- performs DLinear on both occasions (Tab. 12, Appendix H), contradicting the previous rec- ommendations that lincar models should be preferred when the data lacks clear patterns (Shao et al., 2024). Next, to assess guidelines regarding univariate and multivariate data, we compare PatchTST (univariate) against iTrrnsformer (multivariate) all datasets. We use the explained variance from principal component analysis as proxy for inter-variate similarity (Appendix B.1). Neither model performs increasingly better depending on the inter-variate similarity as ranks fluctuate across the spectrum (Fig. 3), contradicting the guideline that multivariate models should be preferred if the data has strong inter-variate similarities (Qiu et al., 2024; Shao et al., 2024).
2nd
PatchTST
MSE ())
Transforme
Rank
25%
50%
75%
100%
25% 50% 75%100%
Explained variance PC1 (%)
Explained varlance PC1 (%)
Figure Uni- vs. multivariate PatchTST and iTrans- former perform comparably in terms of explained variance.
Furthermore, GIFT-Eval claims to provide insights into the strengths of different models across domains, frequencies, prediction lengths, and the number of variates. To investigate this, we compiled all datasets
5

---

Published in Transactions on Machine Learning Research (01/2026)
Motaohaugera
W/ Molochmaggry
W/o Mokorlanery
S-Mamba
300
300
Thurker
TimeXar
PAAATST
17.58
10.58
400
Whrndlmmer
PTchTTT
Pat
TinsMiscr
Dlutrer
-LSTMM
5.10
2.56
8:88
0.71
0.97
0.77
0.30
0.55
0.46
Count
300
Comnt
200
MalenTTN
MiderrTCN
ITressooraer
0226
0.64
0.55
0.72 2: 0.31
0.21
2.0
Sttamao
Silaziee
W/o Muturfmugery
Win
MSR
PateITST
46.78
0.45
0.14
ThneXer
21.45
TineMistr
0
0.0
0.5
1.0
1.5
2.0
S-Maniis
2.70
0.45
MSE
MSB
3.70
0.51
0:37
ModeraTNN
0.08
0.63
0.15
Figure 4: Model rankings are highly sensitive to dataset and horizon selection. We assess the robustness of rankings across 5,000 experimental configurations, each using random subset of datasets and forecast horizons. Including MolorImagery, the only dataset with clear model gaps (Fig. 8, Appendix H), favors S-Mamba, while excluding it yields close performance across models. This highlights the brittleness of current benchmarks, where small changes in datascts or forecast horizons can casily shift which model appears as a champion. Best and second-best are highlighted.
included in the LTSF benchmarks and analyzed their distribution by sampling froquency, number of variates, and domain, revealing substantial imbalance (Fig. 2). For instance, GIFT-Eval represents the Transport domain with two univariate datasets, while BasicTS+ and TFB include several multivariate transport datasets, which is more consistent with the graph-structured nature of such systems (Shao et al., 2024; Qiu et al., 2024). Morcover, no benchmark includes high-frequency datasets (e.g., EEG or wearable-sensor data), excluding many real-world applications. Hence, we speculate that LTSF benchmarks lack sufficient. dataset variety to enable systematic, dataset-guided model evaluation, ultimately hampering progress toward understanding when and why particular models succeed in LTSF.
4 Why are they all champions?
4.1 Impact of selective inclusion of datasets and forecast horizons
Model rankings are highly sensitive to dataset and horizon selection. We systematically analyze how the selective exclusion of datasets and forecast horizons in the experimental settings may affect overall rankings. Let D (DI..... DM} } be the collection of datasets and = (Ti...., ., Th} the forecast horizons. We sample K- 5,000 experimental configurations, cach defined by uniformly drawn subsets of datascts and horizons: SD S D, |Sp] k, k M), and Sy CT, |Srl E, e~ 1~111, When all datasets are included, S-Mamba seems to clearly be the best model, obtaining a win percentage of 556 and the lowest average MSE of 0.49 over the distribution (Fig. 4, left). However, after repeating the analysis without Motorlmagery, i.c., the only dataset with a clear performance gap among baselines (Fig. 8, Appendix H), the distributions overlap (Fig. 4, center), implying equivalency of all models. Since win percentages and average MSEs are similar across models, minor changes in datascts and forecast horizons can shift which model appears best. supporting our view regarding the brittleness of current model superiority claims.
Prior work employed inconsistent dataset and horizon selection. We observe that subsets of the full benchmark were occasionally used, which may be justified, c.g., for lack of datasct size. In iTransformer (Liu et al., 2024a), the authors averaged the performance over the ETT datasets, which they justified by their intrinsic similarity. However, this increased the percentage of wins of their proposed method from 33.3% to 71.4%. BasicTS+ (Shao et al., 2024) focuses solely on a forecast horizon of 336, whereas TFB (Qiu al., 2024) bases the analysis of the impact of different data characteristies on horizon of 96, despite reporting performance for all four forecast horizons.

---

Published in Transactions on Machine Learning Research (01/2026)
Table 3: HP search sensitivity. We report the MSE of DLinear for Weather at prediction length 96 when HP tuning is/is not performed, both in our and previous papers, along with the relative performance improvement expressed in % (when possible). =_" indicates a missing analysis.
DLinear (MSE)
(Zeng al., 2023) (Nie sl, 2023) (Wu 2023) (Láu al,, 2024a) (Wang sl., 2024a) (Wang al, 2024c)
Ours
Unified HP
0.196
0.196
0.195
0.196
0.198
HP tuming
0.176
0.176
0.176
0.168
Rel. Improv.
+9.7%
+15.1%
4.2 Impact of selective inclusion of baseline models
Model exclusion reshape leaderboards. While unsurprising, we stress that excluding the top model from benchmark may automatically crown the second-best as champion. For instance, removing S-Mamba from Tab. 2 would champion i'Transforme as it scores the second-best average metrics.
Prior work overlooked SOTA models in the analyses. We notice the lack of inclusion of baselines like N-Beats in the publications of recent "champions", although it is a top-3 method in (Wang et al., 2024b) (Appendix G). In addition, we identified cases where the best-performing model was excluded from discussions without justification. For example, (Qin et al., 2024; Shao et al., 2024) claim recent transformers underperform compared to earlier methods, but their experiments show the most recent LTSF transformer al the time (PatchTST) outperformed competitors, contradicting this claim. Moreover, (Shao et al., 2024) claims linear models are better for LTSF on datasets with unclear patterns or distribution shifts. However, the claim is based on experiments that excluded PStc thereby weakening the strength of their conclusion.
4.3 Impact of hyperparameters
HP search raises absolute performance. The impact of HP tuning on the benchmarking performance of models is becoming increasingly evident (Brigato et al., 2021; McElfresh et al., 2024). We investigate whether this is also the case in LTSF via a proof-of-concept example. We report the evaluation in terms of MSE for DLinear on the Weather dataset at forecast horizon 96 (Tab. 3). HP tuning brings a relative performance boost of 155% in our setup and an ~10% in (Wang et al., 2024b). Similarly, building on the HP search details in (Wang et al., 2024c), we found comparable performance between TimeMixer, iTTranformer, and PatchTST, unlike the original work, where TimeMixer consistently ranks first. This underscores how close the actual performance of models is, making outcomes and conclusions sensitive to slight variations in HP search.
Prior work put models at disadvantage through unified HP setup. In TSLib, models are usually based on the implementation of the original publications. However, in (Wu et al., 2023), for a fair comparison, they changed the input embeddings and the final projections to be the same for all models. Specifically, the sequence length was set to 96 for all models by default. This is critical since DLinear (Zeng et al., 2023), after a broad ablation study, explicitly states that short input sequences (< 336) lead to underfitting.
4.4 Influence of visualizations
Visualizations may bias perceived rankings. Visualizations are a strong tool to convey a message. In Fig. 5, we investigate the impact of scales visualize results. We observe that using an absolute scale in radar plots exaggerates differences between models that are not perceived in the relative scale with uniform axes. Conversely, it can obscure substantial differences, as in the example of MotorImagery and (Fig. 5, right, b and e).
Prior work employed misleading visualization practices. We identified two examples that used radar plots with absolute scales to visualize the model performances between models (Liu et al., 2024a) and across dataset characteristics (Qiu et al., 2024), respoctively. These choices can create mislcading impression of the models' actual performance and lead to false conclusions. Note that other plots may also create biased impressions through axis scaling or selective metric representation.
7

---

Published in Transactions on Machine Learning Research (01/2026)
Absolute scale
Relative scale
b
b
best (MSE)
100%
worst
0%
g
k
k
h
DLinear
iTransformer XLSTMTime
TimeMixer ModernTCN
TimeXer
PatchTST
SMamba
a) Exchange, b) Motorimagery, <) ETT, d) Electricity, e) BenzeneConcentraation
f) AustralidRainfall, g) KDDCUp2018, h) Weather, i) BeijingAir, j) PedestrianCounts, k) TDBrain
Figure 5: Bias in visualizations. The plots show the same results (MSE) represented at two scales. The relative scale makes performance differences between models appear more subtle.
4.5 Statistical evidence for model superiority
Analysis. We illustrate, through proof-of- concept example, how the lack of robust sta- tistical testing can lead to false claims regard- ing models' superiority. First, we introduce recommended non-parametric statistical tests (Demšar, 2006). Second, we upgrade the ex- isting iTransformer emulate current model- design proposals and rigorously evaluate it in sctup (sec Sec. 3 for dctails).
Table 4: iPatch questionable champion. Although iPatch scores the best average rank and the third-best. MSE/MAB averaged over all datasets, it does not statisti- cally differ from all the other baselines under Friedman test. Best and second-best are highlighted.
Model
MSE
MAE
Rank (MSR)
Rank (MAR)
DLinear
0.770
0.484
4.93
5.00
PatchTST
0.691
0.458
4.07
4.50
Statistical tests. (Demšar, 2006) studied var- ious statistical tests for comparing classifiers from both thcoretical and empirical perspec- lives. The study recommended a set of simple, reliable, and robust tests for such comparisons. In particular, the sign test (Salzberg, 1997) com- pares two classifiers over multiple datasets, and the Friedman test compares various classifiers over multiple datasets (see Appendix F).
Traanformer
0.547
0.416
5.07
5.29
TimeMixer
0.712
0.462
4.64
4.79
TimeXer
0.683
01453
4.57
4.71
iPatch
0.580
0.422
4.00
4.07
S-Mambba
0.484
0.407
4.64
4.57
xLSTMTime
0.971
0.519
5.64
5.21
ModernTCN
0.714
01.990
7.43
6.86
iPatch: A proof-of-concept model. We briefly introduce the iPatch model as a hybrid architecture that integrates design principles PatchTST into iTransformer, emulating common model design trends observed in recent research. Let the input series be BxxxxL where B is the batch size, C. is the number of variates, and L is the lookback length. Firstly, in iPatch, unlike iTransformer, the sequence into N cycles of length P (L= N P), transforming it from B x C x L to B (C -N) P to later enable in the layer the temporal attention as in PatchTST. Each cycle is subsequently embedded to dm, resulting in z E (CC-NN)) Secondly, we enhance the attention module as a sequence of two attentions over variates and cycles. First, the input is reshaped to (B. NN) x x dm to isolate the C variates for each cycle and consequently to apply attention over C' similarly to iTransformer. Next, the output of the variate attention is reshaped (B- C) x N x dn isolate temporal cycles and apply the second attention mechanism similarly to PSt Since the iTransformer MLPs operating on univariate data were
reshape the input by splitting
8

---

Published in Transactions on Machine Learning Research (01/2026)
Table 5: Limited gains from increased model complexity. Efficiency-weightee performance comparison (5) relative to DLinear (t is better). Despite higher complexity, most models fail to outperform DLinear across multiple performance-weige weighted efficiency metries with all datasets (A) and without MotorImagery (B). TP indicates throughput. Best and second-best are highlighted.
DLincar
PatchTST
iTransfommr
ThneMixer
Timexer
S-Mumba
xLSTMTime
ModerTTNN
E/n, DLinear MSS, 0) ct?
A
B
A
B
A
11
A
B
A
B
A.
I1
A
B
A
B
(FLOPs)
1.00
1.00
0.88
0.82
1.21
0.90
0.80
0.65
0.78
0.72
1.42
0.92
0.71
0.81
0.75
0.63
4= (Aparane)
1.00
1.00
0.98
0.92
1277
0.91
0.95
0.89
0.96
0.89
1.39
0.91
0.70
0.79
0.71
0.62
4 {TP, pemory) (laii))
1.00
1.00
0.86
1.25
0.92
0.83
0.79
0.93
0.86
1.18
0.77
0.75
0.77
0.61
4- (TP, nenory] (tee)
1.00
1.00
0.92
0.87
1.26
0.93
0.88
0.83
0.96
0.89
1.29
0.83
0.68
0.76
0.78
0.66
hypothesized to capture time series properties like amplitude and periodicity (Liu et al., 2024a), we reshape the series to (B N) x C' x dm before applying the MLP and finally map it back to B x (C' .N) x dm for the next transformer layer. The linear decoder is applied channel-wise to predict T steps from B x C x (dm .N..
Statistical testing substantiates performance gains from targeted architectural adjustments. Initially, we evaluate the performance of iPatch following either average MSE/MAE (Wang et al., 2024b) or ranks. Tab. 4 shows that iPatch achieves the best average rank and the third-best average MSE and MAE. Complete results for iPatch available in Tab. 14, Appendix H. In line with these outcomes and common practices in the field of LTSF, iPatch may "almost" seem the best-performing model. However, performing the Friedman test, we observe that no model differs from the others considering MAE (x² = 9.33, = 0.31). Given the lack of overall differences, we conduct a focused comparison between iPatch and iTransformer using the sign test (Salzberg, 1997), since iPatch builds directly upon iTransformer. Despite their similar architectures, iPatch wins in terms of MSE/MAE on 11 out of 14 datasets, yielding a statistically significant p-value of 0.05 under the sign test. (Appendix F), suggesting that targeted architectural modifications can lead to performance improvements.
Prior work neglected statistical testing. The TSLib benchmark (Wang et al., 2024b) employs averaging for presenting aggregated results. However, averages are susceptible to outliers (Demšar, 2006). A classifier's strong performance on one dataset can mask weaknesses elsewhere, so we prioritize consistent performance across problems, making dataset averaging unsuitable for evaluation (Sec. 4). BasicTS+ (Shao el al., 2024) and TFB (Qiu et al., 2024) focus on the number of wins achieved by each model but do not incorporate any statistical testing, making conclusions less reliable and harder to communicate in a concise manner.
4.6 Trade-off between model performance and efficiency
Analysis. It is also essential to consider other factors that contribute a model's superiority, such as the trade-offs between performance (e.g., MSE) and elficiency (e.g., training speed or memory consumption) introduced by architectural modifications.
Efficincy-woighheed error metric. Drawing inspiration from nenral-architectur--sacchh literature (Tan et al., 2019), we define a composite metric that summarizes prediction quality and efficiency relative to a baseline model. Let m denote a candidate model and b our baseline, with e(-) representing our prediction error metric and + = (ox()))
K being sct of efficicncy metrics. The officiency-weighted error metric & is formulated as: E(m,b, b, ()) <[m]
e(b) II dx(b) sew
@u(m] where Sy E {-1,+1} controls the metric-specific ratio directionality Nloer/highee is better) and w encodes the relative importance of efficinnc. Thus, the weighted product formulation of & ensures models' error ratios against the baseline are scaled by efficiency ratios weighted by the exponent w. When w I, the efficiency ratio scales linearly with the error ratio, while if w 0, the efficiency ratio has zero relevance. Models with & >>>>> outperform the baseline, while & < indicate unfavorable trade-offs. The baseline always scores a value of one, i.e., when m = b, & = 1. Furthermore, h is (b) < 1, ensuring that the efficiency term always penalizes all other models unless
chosen such that 0 < du (m)
compensated by accuracy gains.
Performanco-offccinncy rankings show lack of improvements. For this analysis, we consider DLinear the baseline, being the most efficient model across all metrics. Furthermore, we set w 0.07 that.
9

---

Published in Transactions on Machine Learning Research (01/2026)
10% reduction in any efficiency metric corresponds approximately to 1% loss in the error-metric ratio dx(b) lies in the range [0.3, 1], as 20.07 x 0.1x + 0.9 in this interval. We select FLOPs and #params for theoretical complexity, and throughput (TP) with memory usage for practical hardware efficiency. In Tab. 5, we report that, with all datasets included (column A), all models except i'Transfommer and S-Mamba underperform DLincar (5 1). These two models are primarily leading the performance-cficieng leaderboard for their superior results on MolorImagery. Indeed, when the Molorlmagery dalaset, is excluded (see Sec. 4.1), even iTrrasformer and S-Mamba no longer achieve the best trade-offs. These results suggest that the additional architectural complexity does not currently lead to meaningful benefits. This pattern is clearly captured by Sarciency-weighten error metrics such as &, which summarize trade-offs between accuracy and efficiency across all datasets and horizons. Further analyses and details on efficiency metric estimation are provided in Appendix I.
<(6) when the efficiency ratio [a(m) <[m]
Prior work lacks consensus on performance- trade-offs. Although there are examples that perform certain trade-off analyses (Liu et al., 2024a; Wang al., 2024a; Shao et al., 2024; Wang et al., 2024c), their interpretations appear to be limited by selected subsets of datasets, models, and efficiency metrics, lacking holistic perspective offered by metrics aggregaled across all dalasets and horizons, thereby conveying incomplete conclusions.
5
How can the field establish real champions?
Summary of Recommendations
Improving
Results should be reported across all datasets and forecast horizons (Sec. 4.1).
benchmarking practices
Results from the best-performing models should always be included (Sec. 4.2).
Rigorously tumed HP configurations must be used for all models (Sec. 1.3).
Third-party evaluations should be encouraged to strengthen reliability.
Reducing unsubstantiated claims
Visualizations should not exaggerate minor differences (Sec. 4.4). Statistical tests should be used when comparing models (Sec. 4.5).
Incressing datasel diversity and revising guidelines for model selection
Benchmarks should include datasets that reflect real-workd diversity.
Benchmarks should define forecast horizons informed by dataset characteristics. Methodologies relating model-dataset characteristics should be further explored. Performance-effcciency trade-offs should be tackled systematically (Sec. 4.6).
Concluding the previous chapters, we provide recommendations that can be tackled by the community. To ensure contimued progress in LTSF, the field must address persistent shortcomings in benchmarking, evaluation methodology, and guidelines for model selection. We outline guidelincs aimed at improving transparcncy, rigor, and the practical relevance of LTSF research.
Improving benchmarking practices. Benchmarks should provide rigorously tuned HP configurations for all models, idcally supported by integrated HP optimization tools. Benchmark users must report performance consistently across all supported datasets, forecast horizons, and context lengths. Even minor deviations in experimental setup can dramatically shift performance rankings (Sec. 4), underscoring the need for transparency and standardization. Additionally, the field would benefit from objective, independent. evaluations, in which test sets are withheld and assessed by third parties, e.g., as originally practiced for ImageNet.
Reducing unsubstantiated claims. Researchers should adopt robust statistical testing to supplement performance rankings and mitigate unreliable claims, as exemplified in Sec. 4.5. Visualizations must be designed with care to avoid distorting perceived differencees. For instance, scale choices can casily exaggerate or obscure performance gaps as highlighted in Sec. 4.4.
Increasing dataset diversity and revising guidelines for model selection. To develop effective model selection guidelines, the benchmarks should include datasets to cover a large spectrum of charactoristics. Potential starting points are the UTSD database (Liu al., 2024b) and the LOTSA database (Woo al., 2024), both databases encompass wide range of datasets with diverse characteristics. In addition
10

---

Published in Transactions on Machine Learning Research (01/2026)
to providing the data, a crucial step is to define meaningful forecast horizons based on intrinsic dataset characteristies an issue exemplified by the arbitrary performance on datasets such as Erchange (Hewamalage et al., 2023). Then, future studies should focus on datasets where performance varies significantly among SOTA models. As illustrated in Fig. 1, only two clearly distinguishable performance patterns, highlighting the need for further investigation into what dataset. characteristics drive such differences. In this context, we particularly value dedicated studies examining more broadly which architectures succeed or fail under varying conditions, following the style of recent work (Chen et al., 2025). From benchmarking perspective, instead, the field should conduct comprehensive evaluations across datasets with diverse characteristics and consistently compare a broad range of model architectures, ensuring that the best-performing architecture for each category, such as linear, MLP, and transformer models, is clearly reported. Additionally, practical trade-offs between model performance and efficiency should be assessed by systematically analyzing how architectural changes impact computational cost and memory usage (Sec. 4.6). Composite metrics such as & can unify performance and efficiency into a single score. To support such evaluations, benchmarks should provide standardized functionalities for consistent. and detailed comparisons.
Beenzeneonncentratinn and Motorlmagery- cxhibit
6 Discussion and limitations
Objective and scope of our evaluation. While experimental design enables a broad and reproducible evaluation of recent supervised LTSF models, it also carries inherent limitations. We focused on a represcntative subset of recent, high-impact models belonging to the most popular families, i.e., transformers, MLPs, state- space, convolutional, and recurrent models, to capture current evaluation practices in the field. Our results may be sensitive to specific dataset choices, hyperparameter scarch ranges, and implementation details, which remain open challenges for reproducible supervised LTSF. However, our goal was not to establish exhaustive benchmarks or definitive rankings but to show that recent advancements often yield only marginal improvements when evaluated under consistent and controlled settings with experimental variance dominating over architectural advancements. By emphasizing recent models, we intentionally highlight the present challenges of the field particularly the difficulty of reliably assessing progress across comparable experimental conditions. Moreover, while certain models may excel in narrow, context-specific scenarios (e-g., S-Mamba in Motorlmagery), such isolated successes do not translate into universal applicability, further supporting our argument against the "champion" narrative.
Setting of fixed "long-term" forecast horizons. A limitation of our study lies in the use of fixed forecast horizons across datasets from different domains which can render the notion of "long-term" arbitrary and detached from domain-specific constraints. Although not necessarily optimal, the chosen forecast horizons reflect current practice and enable comparability with past work. In particular, we adopt the horizons used in TSLib, which are consistent with those in other recent benchmarks, such as BasicTS+ and TFB, further aligning with the ranges reported in the original publications. Identifying truly meaningful forecast horizons remains an open challenge, with recent efforts aiming to define horizons in a more data-informed manner (Aksu et al., 2024), complicated by the unclcar distinction between short- and long-term forccasting. Our findings may also be applicable to shorter horizons, although this requires empirical testing.
Exclusion of foundation models. While recent trends in time series research increasingly explore the development of foundation models (Shi et al., 2024; Yao et al., 2025), including multimodal large language models (LLMs) (Jin et al., 2024), as SOTA time series forecasters, we purposely excluded their evaluation from our work. Practically, substantial differences in terms of benchmarking compared to supervised models hold, including factors such as potential data leakage and the considerable computational cost of pre-training (Aksu et al,, 2024). Dedicated benchmarks are more suitable for critically evaluating and moving forward this parallel line of research in LISF (Aksu et al., 2024). In this regard, the claims following our evaluations do not directly apply to this set of models, given the lack of empirical evidence. However, we argue that incorporating them would be unlikely to alter the conclusions of our work considering recent st studies questioning their actual effectiveness (Xu et al., 2025; Aksu et al., 2024; Tan et al., 2024; Bergmeir, 2024; Karaouli et al., 2025; Zhao et al., 2025). Specifically, in GvFl-Eval, the best-performing foundation model, MOIRAI (Woo et al., 2024), did not outperform PST on medium and long-term forecasts, even without any HP optimization appliod to the latter. This result highlights that these models are still in an early and relatively underperforming
11

---

Published in Transactions on Machine Learning Research (01/2026)
stage compared to well-tuned supervised baselines, finding also corroborated by (Xu et al., 2025). Therefore, while we concur that future work should revisit this question as the field progresses, some of the insights derived from our study may prove valuable for future benchmarking efforts involving foundation models (e.g., rigorous statistical testing). To acknowledge the growing interest in this direction, we include a bricf overviow of recent developments in foundation models for time series Appendix A.3, as well as another description of LLM-based approaches Appendix A.4.
7 Conclusions
In this work, we critically evaluate supervised LTSF research and put forward proposal to address persistent issues in the field. Importantly, our aim is not to criticize prior work in this longstanding and recently revitalized domain but to provide a constructive analysis that supports both our own work and future rescarch in the field, including its translation into domain-specific applications. Through an extensive and reproducible evaluation of eight models across 14 datasets, we demonstrated that claims of consistent, performance improvements in newly published models often rely specific experimental setups and evaluation methods. Our findings question the idea of universal advancements, revealing that no single model consistently excels across our experiments. We identified issues in the supervised LTSF domain, such as non-standardized evaluation frameworks, biased comparisons, and limited reproducibility that hinder fair assessment and delay real progress. To address these challenges, we propose a set of actionable recommendations: adopt standardized evaluation protocols, prioritize benchmarking robustness over architectural complexity, and deepen the analysis of how dataset characteristies influence model performance.
Acknowledgments. We would like to thank the anonymous reviewers for their constructive feedback, which has improved our manuscript. This work was partly supported by the European Commission and the Swiss Confederation State Secretariat for Education, Research and Innovation (SERI) within the project 101057730 Mobile Artificial Intelligence Solution for Diabetes Adaptive Care (MELISSA) and by the Stiftung Sanitas within the framework of the Sanitas Diabetes Technologie 2.0 project.
References
Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: A next- generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2019.
Taha Aksu, Gerald Woo, Juncheng Liu, Xu Liu, Chenghao Liu, Silvio Savarese, Caiming Xiong, and Doyen Sahoo, Gift-eval: benchmark for general time series forecasting model evaluation. arXiv preprint arXiv:2410.1053 2024.
Musleh Alharthi and Ausif Mahmood. xlstmtime: Long-term time series forecasting with xlstm. AI, 5(3): 1482-1495, 2024.
Abdul Fatir Ansari, Lorenzo Stella, Ali Caner Turkmen, Xiyuan Zhang, Pedro Mercado, Huibin Shen, Oleksandr Shchur, Syama Sundar Rangapuram, Sebastian Pineda Arango, Shubham Kapoor, Jasper Zschiegner, Danielle C. Maddix, Hao Wang, Michael W. Mahoney, Kari Torkkola, Andrew Gordon Wilson, Michael Schnie-Schneider, and Bernie Wang. Chronos: Learning the language of time series. Transactions on Machine Learning Research, 2024. ISSN 2835-8856.
Maya Bechler-Speicher, Ben Finkelshtcin, Fabrizio Frasca, Luis Müller, Jan Tönshoff, Antoine Sirandin, Viktor Zaverkin, Michael M. Bronstein, Mathias Niepert, Bryan Perozzi, Mikhail Galkin, and Christopher Morris. Position: Graph Learning Will Lose Relevance Due To Poor Benchmarks, February 2025. URL http://arxiv. http://arxiv.orr/abb// org/abs/ 2502. 14546. arXiv:2502.1453 [cs).
Maximilian Bock, Korbinian Pöppel, Markus Spanring, Andreas Auer, Oleksandra Prudníkova, Michael Kopp, Günter Klambauer, Johannes Brandstetter, and Sepp Hochreiter. xistm: Extended long short-term memory. Advances in Newral Information Processing Systems, 2024.
12

---

Published in Transactions on Machine Learning Research (01/2026)
Christoph Bergmeir. Lims and foundational models: Not (yet) good as hoped. Foresight: The International Journal of Applied Forecosting, 73:33 -38, 2024.
George E. P. Box and David A. Pierce. Distribution of residual autocorrelations in autoregresirr-iintegaate moving average time scries models. Journal of the American Statistical Association, 65(332))1500-15223 1970.
Leo Breiman. Random forests. Machine learning, 45:5-32 2001. doi: 10.1023/A:10109330
Lorenzo Brigato, Björn Barz, Luca Tocchi, and Joachim Denzler. Tune it don't use it: Benchmarking data-efficient image classification. In Proceedings of the IEEE/CVF International Conference Computer Vision (ICCV) Workshops, pp. 1071-1080, October 2021.
Defu Cao, Furong Jia, Sercan O Arik, Tomas Pfister, Yixiang Zheng, Wen Ye, and Yan Liu. Tempo: Prompt-based generative pre-trained transformer for time series forecasting, In The Twelfth International Conference on Learning Representations, 2024.
Cristian Challu, Kin G Olivares, Boris N Oreshkin, Federico Garza Ramirez, Max Mergenthaler Canseco, and Artur Dubrawski. Nhits: Neural hierarchical interpolation for time series lorecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, pp.
2023.
Ching Chang, Wei-Yao Wang, Wen-Chih Peng, and Tien-Fu Chen. Llm4ts: Aligning pre-trained 1lms data-efficient time-series forecasters. ACM Trans. Intell. Syst. Technol. February 2025. ISSN 2157-6904. doi: 10.1145/371922 URL https://doi. org/ 11. 1441///77/77
Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd aem sigkdd international conference on knowledge discovery and data mining, pp. 785 194,,94,7 2016.
Yu Chen, Nathalia Céspedes, and Payam Barnaghi. A closer look at transformers for time series forecasting: Understanding why they work and where they struggle. In Forty-second International Conference Machine Learning, 2025.
Abhimanyu Das, Weihao Kong, Andrew Leach, Shaan K Mathur, Rajat Sen, and Rose Yu. Long-term forecasting with tide: Time-series dense encoder. Transactions on Machine Learning Research, 2023. ISSN 2835-8856.
Janez Demšar. Statistical comparisons of classifiers over multiple data sets. The Journal of Machine learning research, 2:130, 2006.
Vijay Ekambaram, Arindam Jati, Pankaj Dayama, Sumanta Mukherjee, Nam H Nguyen, Wesley M. Gifford, Chandra Reddy, and Jayant Kalagnanam. Tiny time mixers (ttms): Fast pro-trained models for enhanced zero/few-shot forecasting of multivariate time series. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
Maria Eriksson, Erasmo Purificato, Arman Noroozian, Joao Vinagre, Guillaume Chaslot, Emilia Gomez, and David Fernandez-Llorca. Can We Trust AI Benchmarks? An Interdisciplinary Review of Current Issucs AI Evaluation, February 2025. URL http:://axxv.oorg//a
06559. arXiv:2502.06559 [cs).
Jerome H Friedman. Greedy function approximation: gradient boosting machine. Annals of statistics, pp.
1189-1232, 2001.
Milton Friedman. The use of ranks to avoid the assumption of normality implicit in the analysis of variance. Journal of the american statistical association, 6755770 1937.
Milton Friedman. comparison of alternative tests of significance for the problem of m rankings. The annols of mathematical statistics, 11(1):86-92 92, 1940.
Azul Garza and Max Mergenthaler-Canseer Timegpt-1. arXiv preprint ar xrXi:22310.09550 2023.
13

---

Published in Transactions on Machine Learning Research (01/2026)
Nate Gruver, Marc Finzi, Shikai Qiu, and Andrew G Wilson. Large language models are zero-shot time series forecasters. Advonces in Neural Informalion Processing Systems, 36, 2024.
Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. In First conference on language modeling, 2024.
Moritz Herrmann, F Julian D Lange, Katharina Eggensperger, Giuseppe Casalicchio, Marcel Wever, Matthias Feurer, David Rügamer, Eyke Hüllermeier, Anne-Taure Boulesteix, and Bernd Bischl. Position: Why we must rethink empirical research in machinc learning. In International Conference on Machine Learning, pp- 18228 18247. PMLR, 2024.
Hansika Hewamalage, Klaus Ackermann, and Christoph Bergmeir. Forecast evaluation for data scientists: common pit.flls and best practices. Data Mining and Knowledge Discovery, (7(2)8788- 832, March 2023. ISSN 1573-756X. doi: 10.1077/310115-022--0944.4 URL https: /top:://doi..o http:///ii..rg/..110///1 1007 1ttp:/10..000///1011110000000330
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Compulation, 9(8):1735 1780,
1997. doi: 10.1162 10162///co.19//0 19....9...9.9..... 1995.98....755
Rob Hyndman, Anne B Koehler, J Keith Ord, and Ralph D Snyder. Forecasting with exponential smoothing: the state space approach. Springer Science & Business Media, 2008
Ming Jin, Yifan Zhang, Wei Chen, Kexin Zhang, Yuxuan Liang, Bin Yang, Jindong Wang, Shirui Pan, and Qingsong Wen. Position: What Can Large Language Models Tell Us about Time Series Analysis. In Proceedings of the 41st International Conference on Machine Learwing, pp. 22260 22276. PMLR, July 2024.
URL attps://proceediine mlr h.l..press/v233. jin24i html. ISSN: 2640-3498.
Scott M Jordan, Adam White, Bruno Castro Da Silva, Martha White, and Philip S Thomas. Position: Benchmarking is limited in reinforcement learning research. In International Conference on Machine Learning, pp. 22551 22569. PMLR, 2024.
Nouha Karaouli, Denis Coquenet, Elisa Fromont, Martial Mermillod, and Marina Reyboz. How foundational are foundation models for time series forecasting? arXiv preprint arXi::25100007 2025.
Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tic- Tic-Yan Liu. Lightgbm: A highly efficient gradient boosting decision tree. Advances in neural information processing systems, 30, 2017.
Jongseon Kim, Hyungjoon Kim, HyunGi Kim, Dongjun Lee, and Sungroh Yoon. A comprehensive survey of time series forecasting: Architectural diversity and open challenges. arXiv preprint arXivv2411.05579,, 2024.
Lambert H Koopmans. The spectrol analysis of time series. Elsevier, 1995. doi: https://doo.org/10..01.
Maurice Kraus, Felix Divo, Devendra Singh Dhami, and Kristian Kersting, xlstm-mixer Multivariate time scrics forecasting by mixing via scalar memories. arXiv preprint arXiv:2440. 16928, 2024.
Shiyang Li, Xiaoyong Jin, Yao Xuan, Xiyou Zhou, Wenhu Chen, Yu-Xiang Wang, and Xifeng Yan. Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
Zhe Li, Shiyi Qi, Yiduo Li, and Zenglin Xu. Revisiting Long-term Time Series Forccasting: An Investigation on Linear Mapping, May 2023. URL tttp:://arxivv www.ooogg///......oooooooooo. 10721. arXiv:2305.10721 [cs].
Minhao Liu, Ailing Zeng, Muxi Chen, Zhijian Xu, Qiuxia LAI, Lingna Ma, and Qiang Xu. Scinet: Time series modeling and forccasting with sample convolution and interaction. In S. Koyejo, S. Mohamod, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. Curran Associates, Inc., 2022a.
14

---

Published in Transactions on Machine Learning Research (01/2026)
Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X. Liu, and Schahram Dustdar. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. In International Conference on Learning Representations, 2022b. URL
Yong Liu, Haixu Wu, Jianmin Wang, and Mingsheng Long. Non-stationary transformers: Exploring the stationarity in time series forecasting. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 9881 9893. Curran Associates, Inc., 2022c.
Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting. In The Twelfth International Conference on Learning Represenlations, 2024a.
Yong Liu, Haoran Zhang, Chenyu Li, Xiangdong Huang, Jianmin Wang, and Mingsheng Long. Timer: Generative pre-trained transformers are large time series models. In Forty-first International Conference Machine Learning, 2024b. URL https: //tp:///eppeeeeeeeeeemee wwwpreeieww.ee/ffrreeoeeeeeeeeee htte:/formmmdd=DRRYBO
Donghao Luo and Xue Wang. Modernten: A modern pure convolution structure for general time series analysis. In The twelfth international conference on learning representations, pp. 1-43, 2024.
Duncan McElfresh, Sujay Khandagale, Jonathan Valverde, Vishak Prasad C, Ganesh Ramakrishman, Micah Goldblum, and Colin White, When do neural nets outperform boosted trees on tabular data? Advances in Neural Information Processing Systems, 36, 2024.
Timothy R McIntosh, Teo Susnjak, Nalin Arachchilage, Tong Liu, Dan Xu, Paul Watters, and Malka N Halgamuge. Inadequacies of large language model benchmarks the era of generative artificial intelligence. IEEE Transactions on Artificial Intelligence, 2025.
Yuqi Nie, Nam Н Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In The Eleventh International Conference on Learning Representations, 2023.
Boris N. Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. N-beats: Neural basis expansion analysis for interpretable time series forecasting. In International Conference on Learning Representations, 2020. URL sttps://openreeviee www.:n/eoooommieereecccttoc
Xiangfei Qiu, Jilin Hu, Lekui Zhou, Xingjian Wu, Junyang Du, Buang Zhang, Chenjuan Guo, Aoying Zhou, Christian S. Jensen, Zhenli Sheng, and Bin Yang. Tib: Towards comprehensive and fair benchmarking of time series forecasting methods. Proc. VLDB Endow., 17(9):2363 2377, May 2024. ISSN 2150-8097. doi: 10.14778/ 10.14778/36658444.0 ///33366644446668880
URL https://doi: orweii..rg/1... 11447773665844 3665863.
Kashif Rasul, Arjun Ashok, Andrew Robert Williams, Arian Khorasani, George Adamopoulos, Rishika Bhagwatkar, Marin Biloš, Hena Ghonia, Nadhir Hassen, Anderson Schneider, Sahil Garg, Alexandre Drouin, Nicolas Chapados, Yuriy Nevmyvaka, and Trina Rish. Tag-Ilama: Towards foundation models for time series forecasting. In Ro-FFoMM:toouusteess of Few-shot and Zero-shot Learning in Large Foundation Models, 2023.
Steven L Salzberg. On comparing classifiers: Pitfalls to avoid and recommended approach. Data mining and knowledge discovery, 1:317 328, 1997.
M. Saquib Sarfraz, Mei-Yen Chen, Lukas Layer, Kunyu Peng, and Marios Koulakis. Position: Quo vadis, unsupervised time series anomaly detection? In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp- 43461 43476. PMLR, 21 27 Jul 2024. URL
mlr, www.ppresssvveee /veas////////// htt////2/////f//e// sarfraz24a html.
Omer Berat Sezer, Mehmet Ugur Gudelek, and Ahmet Murat Ozbayoglu. Financial time series forecasting with deep learning systematic literature review: 2005-2019. Applied Soft Computing, 90:106181, 2020. ISSN 1568-4946. doi:
20000000000000000111111111 URL https://www.ccienedd
15

---

Published in Transactions on Machine Learning Research (01/2026)
Zezhi Shao, Fei Wang, Yongiun Xu, Wei Wei, Chengqing Yu, Zhao Zhang, Di Yao, Tho Sun, Guangyin Jin, Xin Cao, et al. Exploring progress in multivariate time series lorecasting: Comprehensive benchmarking and heterogeneity analysis. IEEE Transactions on Knowledge and Data Engineering, 2024.
Jingzhe Shi, Qinwei Ma, Huan Ma, and Lei Li. Scaling law for time series forecasting. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomezak, and Cheng Zhang (eds.), Advances in Neural Information Processing Systems 38: Annual Conference Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Camada, December 10 15, 2024, 2024.
Jasper Snoek, Alex Wiltschko, and Ali Rahimi. Winner's curse? on pace, progress, and empirical rigor. 2018.
Rajat Soni, Mohit Nathani, and Rajiv Mishra. Comprehensive evaluation of deep Itsf models for forecasting of air quality index. In 2024 IEEE International Conference on Big Data (BigData), pp. 4410 4419, 2024. doi: 10.1109 1..1109/BiiDat522200 Biggatt623233... wwwpa/2232222222105222555
RA Stine. Comment: Classifier technology and the illusion of progress. Statistical science, (1):27 29, 2006.
Chenxi Sun, Hongyan Li, Yaliang Li, and Shenda Hong. Test: Text prototype aligned embedding to activate LLM's ability for time series. In The Twelfth International Conference on Learning Representations, 2024.
Mingtian Tan, Mike A. Merrill, Vinayak Gupta, Tim Althoff, and Thomas Hartvigsen. Are Language Models Actually Useful for Time Series Forecasting? Advances in Newral Information Processing Systems, 37:60162-60199 December 2024. URL https: httpp:://prooeedinss.eei/// neurips papppps.ppr.prpssppppppp.r.r ctppppereppeelleeeeee_peeepaepep
files/ wwwess/aaeee///e/eeeeeea http///pppr//////////
html.
Mingxing Tan, Bo Chen, Ruoming Pang, Vijay Vasudevan, Mark Sandler, Andrew Howard, and Quoc V Le. Mnasnet: Platform-aware neural architecture search for mobile. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognilion, pp. 2820-2823 2019.
Hiro Y. Toda and Peter C. B. Phillips. Vector autoregressions and causality. Ecmometrica, 61(6):1367-139) 1993. doi: 10.2307 10.2307/2051647 /2911677.
Shiyu Wang, Haixu Wi, Xiaoming Shi, Tengge Hu, Huakun Luo, Lintao Ma, James Y. Zhang, and JUN ZHOU. Timemixer: Decomposable multiscale mixing for time series forecasting, In The Twelfth International Conference on Learning Representations, 2024a.
Yuxuan Wang, Haixu Wu, Jiaxiang Dong, Yong Liu, Mingsheng Long, and Jianmin Wang. Deep Time Series Models: A Comprehensive Survey and Benchmark, July 2024b. URL http://arxiv http//bbs/20...1//2 1rXiv:2407.13278 [cs].
Yuxuan Wang, Haixu Wu, Jiaxiang Dong, Guo Qin, Haoran Zhang, Yong Liu, Yunzhong Qiu, Jianmin Wang, and Mingsheng Long. Timexer: Empowering transformers for time series forecasting with exogenous variables. In The Thirty-eighth Anneual Conference on Neural Information Processing Systems, 2024c.
Zihan Wang, Fanheng Kong, Shi Feng, Ming Wang, Xiaocui Yang, Han Zhao, Daling Wang, and Yifei Zhang. Is mamba effective for time series forecasting? Neurocomputing, 619:129178, 2025.
Rafal Weron Electricity price forecasting: A review of the toite--o--hheartt with look into the fu- ture. International Journal of Forecasting, 1044);1030-1081 2014. ISSN 0169-2070. doi: https: http://:001000//.ooooras22200000 pi11/801692001440000...
URL https://ww..sciencddreett wtw.///ccc.coom/ccncn//eeeeeeeee article/
Gerald Woo, Chenghao Liu, Akshat Kumar, Caiming Xiong, Silvio Savarese, and Doyen Sahoo. Unified training of universal time series forecasting transformers. In forty-first International Conference on Machine Learning, 2024.
Haixu Wu, Jichui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with auto-corrclation for long-term series forecasting. Advances in neural information processing systems, 34: 22419 22112,,,, 2021.
16

---

Published in Transactions on Machine Learning Research (01/2026)
Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. In The Fleventh International Conference on Learning Representations, 2023.
Renjie Wu and Famonn J. Keogh. Current. Time Series Anomaly Detection Benchmarks are Flawed and Creating the Illusion of Progress. IEEE Transactions on Knowledge and Data Engineering, 35(3): 2421 2429, March 2023. ISSN 1558-2191. doi: 10.100/TKDDE2202111112222 URL https: http:://ieeepplore. ieee. org/ document/9637291.
Zongzhe Xu, Ritvik Gupla, Wenduo Cheng, Alexander Shen, Junhong Shen, Ameet Talwalkar, and Mikhail Khodak. Specialized foundation models struggle beat supervised baselines. In The Thirteenth Interna- tional Conference Learning Representations, ICLR 2025, Singapore, April 24-28, 2025, 2025.
Hao Xue and Flora D Salim. Promptcast: new prompt-based learning paradigm for time series forccasting. IEEE Transaclions on Knowledge and Dala Engineering, 2023.
Qingren Yao, Chao-Han Huck Yang, Renhe Jiang, Yuxuan Liang, Ming Jin, and Shirui Pan. Towards neural scaling laws for time series foundation models. In The Thirteenth International Conference on Learning Representations, 2025.
Ailing Zeng. Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In The Seirth--Sventh AAAI Conference on Artificial Intelligence (AAA1-23). AAAI Press, 2023. ISBN 978-1-57735-880-0 doi: 10.1609 10009/aaaivv33 waw....3...266333 URL https: http:://ooooooo....o.ooo// 1609/ v3719, v3719.26317.
Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension dependency for mul- tivariate time series forecasting. In The eleventh international conference on learning representations, 2023.
Lifan Zhao, Yanyan Shen, Zhaoyang Liu, Xue Wang, and Jiaji Deng. Less is more: Unlocking specialization of time series foundation models via structured pruning. In The Thirty- winth Annual Conference Neural Information Processing Systems, 2025. URL https: //penreviee
Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond elficient transformer for long sequence time-series lorecasting. In Proceedings of the AAAI conference artificial intelligence, volume 35, pp. 11106-1111 2021.
Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International conference on machine learning, pp. 27268 27286. PMLR, 2022.
17

---

Published in Transactions on Machine Learning Research (01/2026)
A Related work
A.1 Classical approaches
Traditional statistical methods, such AutoRegressive Integrated Moving Average (Box & Pierce, 1970), Vector Autoregression (Toda & Phillips, 1993), Exponential Smoothing (Hyndman et al., 2008), and Spectral Analysis (Koopmans, 1995) were widely used in TS forecasting. Progressively, machine learning models such as XGBoost (Chen & Guestrin, 2016), Random Forest (Breiman, 2001), Gradient Boosting Regression Trees (Friedman, 2001), and LightGGBM (Ke et al., 2017) have shown improvements the forecast due to their ability to handle non-linear patterns.
A.2 Deep learning models
Deep learning models have advanced TS forecasting, starting with Recurrent Neural Networks (RNNs), specifically designed to model sequential data. In particular, advanced variants such as RNNs with Long Short-Term Memory units, widely adopted within the TS community, have seen significantly increased usage (Hochreiter & Schmidhuber, 1997). Additionally, MLP-based modcls, such as DLinear (Zeng et al., 2023), N-BEATS (Oreshkin et al., 2020), and N-Hits (Challu et al., 2023) use MLP to learn the coefficients that produce both backcast and forecast outputs from their structure.
Originally from Natural Language Processing (NLP), the Transformer architecture is increasingly adapted for time series forecasting, often with modified attention layers to caplure temporal dependencies, as seen in Sec. 2 and other prior works, which we describe in the following, Informer (Zhou et al., 2021) and Pyaformer (Liu et al., 2022b) are transformer-basee models that modify the attention mechanism. Informer designs a ProbSparse self-attention mechanism to replace the standard self-attention. Pyaformer, on the other hand, presents pyramidal attention module, where the inter-scale tree structure captures features at different resolutions, and the intra-scale neighboring connections model the temporal dependencies across different ranges. Wu et al. (Wu et al., 2021) introduced the Autoformer with an Auto- Into-Correlation mechanism to capture the series-wise temporal dependencies based on the learned periods. Following, FEDformer (Zhou et al., 2022) utilizes a mixture-of-expert framework to improve seasonal-trend decomposition and integrates Fourier and Wavelet-enhanced blocks to capture key structures in the TS. (Zhang & Yan, 2023) presented Crossformer, a transformer- based model utilizing cross-dimension dependency for multivariate TS forecasting. Another recent approach is TimesNet (Wu et al., 2023), which is univariate 2D CNN that segments 1D time scries according to Fourier decomposition. The segments then stacked to build 2D scries. This enables the convolutions to simultaneously look at the local structure of the signal at 4 and 4-r simultaneously, where I' denotes a dominant signal period.
A.3 Foundation Models
There is a growing interest in foundation models designed explicitly for TS tasks (Shi et al., 2024; Yao et al., 2025). Tiny Time Mixers (Ekambaram et al., 2024) introduce compact model for multivariate TS forecasting. Timer-XI, is a foundation model for unified time series forecasting, supporting univariate and multivariate data by extending next-token prediction for causal generation (Liu et al., 2024b). The model introduced a universal TimeAt mechanism to capture fine-grained intra- and inter-series dependencies. MOIRAI (Woo et al., 2024) addresses challenges like cross- prossffreueency learning and varied distributional properties in large-scale data, achieving competitive zero-shot forecasting performance. TimeGPT-1 (Garza & Mergenthaler- Gorgenthaler-Canscoo, 2023) and Lag-LLama (Rasul al., 2023), utilizing decoder-only transformer architectures and achieving strong zero-shot generalization. Chronos (Ansari et al,, 2024) trains transformer- based models on discrete tokens processed from TS data, demonstrating superior performance on diverse datasets.
A.4 Large Language Models
The success of Large Language Models (LLMs) like BERT and GPT in NLP has inspired rescarchers apply these models to TS tasks. As outlined in (Jin et al., 2024), LLMs may serve in three roles (R): as Enhancers (R1), which incorporate domain-specifie external knowledge while relying on specialized models
18

---

Published in Transactions on Machine Learning Research (01/2026)
Table 6: Dataset statistics. Refer to Appendix B.1 for a detailed description of the statistics.
Domain
Datwart
Timeecops
# Channels
Sharmon
Spectaal Fetr. Sample Fiirr
Statimarity
Compleityy
Expll
Suorow
Eusagy
ETTL
17420
7
0.775
0.669
0.769
-5.900
0.497
0.544
Wiug as (302
Eneegy
ETTm
60630
0.799
0.548
0:430
-14985
0.485
0.546
Wang (2023)
Energy
ETTh2
17420
(2023)
Eusegy
BTTm2
606900
7
0.817
0.527
0.319
-0.964
0.425
0.431
Wang ai al. (302-0)
Euegy
Elettiiiiy
26331
821
0.516
0.887
0.714
-8.45
0.673
01547
Wang (2024))
Eurtzenmen
Weather
(2023)
Econonic
Exchange
7388
0.805
0.347
0.066
1,002
0.529
0.618
Wang et (2020))
Health
Moloclnuggrry
113-000
61
0.719
0.519
0.326
+3.153
0.763
01305
Lin el (2024))
Health
TDRrssn
2221 212
-3.167
(2024))
Earzzenmens
Begingkhr
077184
0.493
0.686
0.981
13,253
0.165
0.583
Liu et (2024b)
Eurzronmen:
ConerrtConeetstaattt oncessration
2042880
8
0.789
0.201
1.908
3.114
0.049
0.534
Liu et al. (2024))
Rnvimmmes
AustrslicRsifall
3846408
-31.734
(2024))
Rnvinommes
KDDCop2008
2042364
0.009
0.410
-0.379
0.530
1.000
(202-0)
Transport
Codesttianoonts
0112346
0.687
0.522
0.412
4,350
0.630
1,000
Liu et al. (2024h)
for prodiction; models; or Agents (R3), which orchestrate workflows involving external tools and models. One significant approach involves transforming numerical TS data into natural language prompts to leverage pre- pre-trained language models without modifications. Prompt (Xue & Salim, 2023) and (Gruver et al., 2024) present this mothod, demonstrating generalization in zero-shot settings and often outperforming traditional numerical models. However, recent work cast doubts on the actual significance of LLMs as base forecasters (Tan el. al., 2024). Moving few-shot training strategies, TEST (Sun et al., 2024) adapts TS data for pre-trained LLMs by tokenizing the data, and aligning the embedding space, particularly in few-shot and generalization scenarios. Several frameworks focus enhancing TS forecasting through specialized fine-tuning strategies such LLM4TS (Chang et al., 2025) and TEMPO (Cao et al., 2024).
Forecasters (R2), which replace expert models entirely and cast LLMs directly predictive
B Datasets
We include popular set of datasets (ETT*, Electricity, Weather, Exchange) and set of larger datasets (Mo- lorlmagery, TDBrain, BeijingAir, BenzeneConcenlrulinn, Australia Rainfull, KDDCup2018, PedestrianCounls) representing subset of the Unified Time Series Dataset (UTSD) (Liu et al., 2024b).
This section provides a summary of descriptive statistics about the employed datascts, example of two datasets with clear and unclear patterns, respectively, and datasel-specific preprocessing steps.
B.1 Dataset statistics
In Tab. 6, provide comprehensive description of the datasets employed in this work and their corresponding statistics. Next, we describe in detail the methodology to derive such dataset statistics.
Time steps and channels. We counted the number of time steps and number of channels.
Shannon entropy and spectral entropy. Shannon entropy quantifies the average level of uncertainty or information content associated with the outcomes in a discrete variable X. Spectral entropy, a related concept, applies this measure to the frequency domain, using the normalized power spectral density as the probability distribution. Entropy is caleulated
H(X) = --=
((x) logp(a)
zEX
where X is the set of all possible outcomes, and p(x) the probability of outcome x.
Sample entropy. Sample entropy statistical measure used to quantify the complexity or regularity of a time series. Unlike Shannon entropy, which evaluates uncertainty in discrete probability distributions, sample
19

---

Published in Transactions on Machine Learning Research (01/2026)
entropy assesses the likelihood that similar patterns in the time scries remain similar at the next time step. It is defined as the negative natural logarithm of the conditional probability that two sequences of length that match within a tolerance r will still match when extended to m + 1. A lower sample entropy indicates more regularity or prodictability the series, while a higher value suggests greater randomness or complexity. Sample entr is calculated as
A
SampEn(m, r, N)
where is the embedding dimension, r is the tolerance, N is the total length of the time series, A is the number of matching pairs of length m + 1, and B is the number of matching pairs of length
Stationarity. In LTSF, stationarity is an important characteristic of time scries, where the statistical characteristics, such as mean and variance, remain constant over time. We used the Augmented Dickey-Fuller (ADF) test to assess stationarity. This test evaluates the null hypothesis that a unit root is present and confirms stationarity if y < 0 and the result is statistically significant. We report y since it scales with stationarity.
Complexity. In time series analysis, complexity refers to the irregularity umpredctabilityy in the data. We quantify complexity by using Higuchi's ' fractal dimension. A higher fractal dimension indicates greater complexity, while a lower value suggests more regularity or predictability in the data. Higuchi's fractal dimension is calculated as
log
Σ Di=1
D lim k-0
log k
where x; is a point, & is the time scale, and the taken over different segments of the time series.
Inter-variate similarity. As proxy for vater-variate similarity, we provide the explained variance of the first principal component (PC1) obtained through principal component analysis (PCA). PC1 represents the direction of maximum variance in the data, capturing the dominant shared variation among the variables. The explained variance of PC1 quantifies the proportion of the total variance that is accounted for by this component. A higher explained variance indicates stronger similarity and shared dynamics among the variables, while lower value suggests more independent behavior.
B.2 Clear vs. unclear patterns
In Sec. 3, we assessed the performance of a linear model versus a transformer model on datasets with clear and unclear patterns, respectively. We use the same dataset as BasicTS+ (Shao et al., 2024) with an unclear pattern (Erchange) and replace their previously used PEMS with a clear pattern by PedestrianCounts (Fig. 6). The plots of the time series highlight contrasting charactoristics: Erchange displays scemingly random trends, whereas PedestrianCounl exhibits evident cyclic behavior. To further emphasize this distinction, we provide butterfly plots, which reveal pronounced periodic patterns in InnestrianConnte and irregular, stochastic-like trends in Exchange. Additionally, the power spectrum analysis underscores this contrast, showing a dominant peak for Pedestrian Counts and an absence of such peaks for Ezchange.
B.3 Preprocessing
We followed the preprocessing steps from TSlib (Wang et al., 2024b). Furthermore, we added functionality import data from UTSD curated by (Liu al., 2024b). Since the UTSD datasets are magnitudes larger, modified the respective dataloader return sequences at stride length S 100 to accelerate the training.
C HP search details
HP search. To ensure fair comparison, we performed an extensive HP search for all models on all the datasets. Specifically, we searched for an input length between 96 and 512, model size dm from 16 to 512, learning rate ranging from 10-5 to 0.1, and encoder layers between and 5. The specific ranges are presented
20

---

Published in Transactions on Machine Learning Research (01/2026)
PedestrianCounts
6000
6000
0.4
Amplitude
Amplitude
4000
4000
Power
0.2
2000
2000
0
0.0
120
240
360
12
24
0.0
0.2
0.4
Time (samples)
Time (samples)
Frequency (1/samples)
Exchange
0.8
0.3
1.0
Amplitude
WM mym
Amplitude
Power
0.2
0.7
0.8
0.1
0.6
0.6
0.0
480
960
1440
96
0.0
0.2
0.4
Time (samples)
Time (samples)
Frequency (1/samples)
Figure 6: Clear vs. unclear patterns (top vs bottom) in two datasets.
in Tab. 7. TimeMixer was limited to maximum of 3 layers and a model size of 128 due to increasingly high memory demands (> 49GB of VRAM on an RIX A6000 GPU if dm > 128 and L = 720). However, this is unlikely to affect performance, as the original HP search for all datasets in this study yielded results within these limits. Additionally, we limited the search space for dm in ModernTCN to a region closely centered around its default value for LTSF (dm = 64). We used the Optuna framework (Akiba et al., 2019) with a budget of 40 trials to optimize the HPs. We employed the default TPEsampler for HP sampling and applied the Successi www.aiiiiaaaaiiiiiavaaaaa prune unpromising trials. The search was conducted with a batch size of 8, a maximum of 15 epochs, and early stopping with a patience of 3 epochs. All models were optimized with the Adam optimizer and an exponentially decaying scheduler following the default TSLib configuration. The optimal HPs, determined by the minimum validation loss the trials, were used train and evaluate the final model across three random seeds ensure robust results. We set the dimensions of the fully connected layers dy equal to dm- The patch length, a parameter used in PStchTST and TimeXer (and by extension also iPatch), was set based the characteristics of the datasets (Tab. 8). As visualized in Fig. 6, there are datasets with dominant periodic behavior. We set the patch length P x argmax(FFT pattern. Unsurprisingly, the patch length resulted in span of one day in all datascts with a pattern. In the remaining cases, we set the patch length manually (Eachange, MolorImagery, TDBruin). We aligned with the idea of NimesNet, which introduced series segmentation based on dominant frequencies to enhance performance (Wu al., 2023). Moreover, the original works concluded that variations in patch length have minimal effects (Nie al., 2023; Wang al., 2024c). For the rest of the model HPs, we default to the configurations provided in TSL.b.
configured with minimum of three epochs and reduction factor of two to
wherever observed such natural
Table 7: Searched HPs values.
Hyperparameter
TimeMixer
ModernTNN
Other Models
Input Length
(96, 192, 336, 720) {10-a, 10 4, 10 (0-0,10-²)
{96, 192, 336, 720)
{96, 192, 336, 720} {10-2, 10 10 10-²)
Learning Rate
{10-9, 9, 10 10-2 100²) [1, 2, 3, 4]
Layers dm
{1, 2, 3} (16, 32, 64, 128)
{1. 2, 3, 4)
[32, 64, 128, 256)
(16, 32, 64, 128, 256, 512)
21

---

Published in Transactions on Machine Learning Research (01/2026)
Table 8: Patch lengths. Employed patch lengths for PatchTST, TimeXer, and iPatch models.
Motor MotorTaager Tnuggory
Benzzne Cone.
Cone.
AustraliaRain
KDDCup2018
Palestrian
ETTR
ETTu*
Electricity
Weather
Eschange
TDBrain
BeijingAh
Datasct
Patch Length
24
96
24
24
96
96
48
24
24
24
24
24
Implementation reliability
To assess the reliability of our implementation, we compare our results (best and average MSE) against the original works on popular datasets. Despite not. being directly comparable for slight differences in setups, we align with previous values (Tab. 9).
Table 9: Implementation reliability. To assess the reliability of our implementation, we compare results (best and average MSE) against the original works popular datasets. Despite not being directly comparable due to slight differences in experimental setups, we align with previous values.
Model
Dataset
Original
Ours Min
Ours Mean
ETT
0.370
0.378
0.378
DLinear
Electricity
0.166
0.162
0.162
Weather
0.246
0.244
0.244
ETT*
0.338
0.332
0.336
PStchTST
Electricity
0.159
0.163
0.164
Weather
0.225
0.224
0.225
ETT*
0.383
0.342
0.344
iTransformer
Electricity
0.178
0.162
0.166
Weather
0.258
0.237
0.239
ETT*
0.333
0.342
0.349
TimeMixer
Electricity
0.156
0.154
0.156
Weather
0.222
0.225
0.231
ETT*
0.363
0.341
0.346
TimeXer
Electricity
0.171
0.165
0.170
Weather
0.241
0.222
0.224
ETT*
0.333
0.374
0.379
Modern TCN
Electricity
0.156
0.268
0.275
Weather
0.224
0.233
0.241
ETT*
0.339
0.460
0.484
xLSTMTime
Electricity
0.157
0.160
0.163
Weather
0.221
0.226
0.229
E
0.380
0.353
0.357
S-Mambb
Electricity
0.170
0.164
0.166
Weather
0.251
0.230
0.237
E Stability of HP search
In this section, we perform a small proof-of-concee experiment to show the reliability of our HP search. For subset of datasets and models, we performed two indopendent HP runs and consoquent training three models with the found optimal HPs analyze the stability of the performed HP search. We focused on forecast horizon 96. In Fig. 7, plot the MSE of the two searches over two opposing axes, meaning that.
22

---

Published in Transactions on Machine Learning Research (01/2026)
DLincar
Dlineat
2.5
PuteirST
2.5
PRidITT
Tronsfoueer
U'onndormeer
TimeXer
TimeXer
2.0
2.0
MSE
1.5
MSE
1.5
1.0
Motoclmagery
1.0
Motorkmaggry
TDBmin
TDOrain
ReijingAir
Reijing,
E/TTh
0.5
ETTml
ETTml
F/TT12
Tuu2
ETTT2
Wistther
Winther
0.0
Eachange
0.0
Exchunge
0.0
0.5
1.0
1.5
2.0
0.0
0.5
1.0
1.5
2.0
MSE
MSF
Figure 7: HP search variability. Comparison among two independent HP search results in terms of MSE for forecast horizon 96. The average of the final three models shows minimal variability except. for a few cases (left), while the best model is even more stable (right).
Dataset
7.5
DenzeneConcentran
Motorlmagery
MSE (L)
5.0
TDBrain
BeijingAir
2.5
ETTh1
ETTTl
0.0
ETTh2
ETTm2
DLinear
ModernTNN
PStchTST
SMamba
TineMixer
TimeXer
iTransformer
ALSTMTimme
Weather
Electricity
Exchange
Model
PedestrisnCounts
KDDCup2018
AustraliaRainaal
Figure 8: Violin plot for datasets and horizons. We provide alternative view of model performance datasets and prediction horizons. Each point represents the average MSE obtained for given forecast horizon over three independent seeds with the found HP configuration. It is clearly visible that the difference in average scores solely depends on the MotorImagery dataset since baseline scores are comparable over the rest the dataset-horizon setups.
points on the diagonal indicate very stable scarch that lcads to the same final result. To provide an even more comprehensive analysis, we show the average of the three final models (left) and also the best model out of the final three (right). The averages show few cases with slight variability, namely PatchTST on BejingAör and TimeXer on RenseneConcentrution, but overall, the experiment proves a reliable and stable HP search across independent runs. In the case of best results (Fig. 7, right), the variability is even less noticeable.
23

---

Published in Transactions on Machine Learning Research (01/2026)
F Statistical tests
F.1
Friedman test
The Friedman test (Friedman, 1937; 1940) is a non-parametrie statistical method designed as an alternative to repeated-measuree ANOVA. enables the comparison of multiple algorithms across multiple datasets when the assumptions of parametric tests may not. hold. The test works by ranking the algorithms on each dataset separately, with the best-performing algorithm assigned a rank of 1, the second-best rank of 2, and forth. In cases of ties, average ranks are assigned across the tied algorithms.
Let rf denote the rank of the j-th algorithm out of k algorithms on the i-th dataset out of N datasets. The Friedman test evaluates the average ranks of the algorithms, calculated as R Ei=17 r/. Under the mull hypothesis, which assumes that all algorithms equivalent in performance and thus their ranks R should be approximately equal, the Friedman statistic is given by:
XF = 12N
Σ
A(Ks + 1)² R2)
k(k 1)
4
For sufficiently large values of N and k, as rule of thumb N 10 and k 5 (Demsar, 2006), this statistic follows x² distribution with k - degrees of freedom. Note our experimental setup aligns with these conditions.
The Friedman lest, though less powerful than parametric repealed-measee ANOVA when its assumptions are met, more robust in handling violations of these assumptions, with (Friedman, 1940) observing largely consistent rosults between the two tests across 56 independent problems.
F.2
Sign test
The sign test (Salzberg, 1997) is non-parametric statistical method commonly used to compare the performance of two algorithms across multiple datasets. It operates by evaluating the number of datasets which each algorithm outperforms the other, assuming that the outcomes are independent and identically distributed. Contrary to popular belief, counting only significant wins and losses actually makes the tests less reliable, as it imposes an arbitrary threshold of p < 0.05 to determine meaningfulness (Demsar, 2006).
Under the null hypothesis, it assumed that the two algorithms are equivalent in their performance, and thus, cach algorithm has an equal probability (0.5) of outperforming the other on given dataset. This leads to the number of wins for either algorithm following a binomial distribution with parameters N (the total number of datasets) and p 0.5. The mull hypothesis is rejected if the observed number of wins for algorithm is significantly different from N indicating that one algorithm systematically outperforms the other.
For small sample sizes, as in our case, with a total number of datasets being equal to 14, critical values can be determined directly from the cumulative distribution function of the binomial distribution. For larger sample sizes, the central limit theorem allows for an approximation using the normal distribution. Specifically, the number of wins under the null hypothesis
be approximated by normal distribution with mean VN/2. /2. In cases where there are ties in performance, these ties are
/ N/2 and standard deviation treated as supporting evidence for the null hypothesis. To account for this, ties are split evenly between the two algorithms. If the number of tied datasets is odd, one tie is disregarded to ensure that only whole numbers are assigned to each algorithm.
G Baseline comparisons
We provide an overview of included basclincs for the top-performing LTSF models (Tab. 10). We observe that previous models were replaced by recent ones as the field progressed. We highlight that TimeMixer was not included as baseline in TimeXer, although it available at the time.
24

---

Published in Transactions on Machine Learning Research (01/2026)
Table 10: Included baseline models from top-performing models in long-term forecasting.
x: included; introduced
Crossformer Zhang & Yan (2023)
TimeMixer Wang et al. (2024a)
iThansformer Liu al. (2024a)
TimeXer Wang et al. (2024c)
FEDformer Zhou et al. (2022)
Pyraformer Lin al. (2022b)
Autoformer Wu et al. (2021)
Stationary Liu et al (2022c)
DLinear Zeng et al. (2023)
PatchTST Nie et al. (2023)
Informer Zhon et al, (2021)
TimesNet Wi: al. (2023)
Log Log Trans Li et al. (2019)
SCINet Liu al. (2022a)
Rlinear Li et al. (2023)
TIDE Das et al. (2023)
others
Model
DLinear Zeng et al. (2023)
X
X X
PatchTST et (2023)
X
0
x
x
TimeMixer Wang et al. (2024a)
iTransformer Liu et al. (2024a)
X
X
X
X
X
X
X
X
X
X
TimeXer Wang et al. (2024c)
X
Table 11: Efficiency metrics. Average efficiency metrics for all optimized models over datasets and prediction horizons (1,000 iterations, batch size of 1). Best and second-best are highlighted.
Metric
DLincar
PatchTST
iTransformer
TimeMixer
TimeXor
S-Mamba
xLSTMTiie
ModernTCN
Train TP (29)
1663.58
281.78
324.49
92.82
162.29
68.57
187.70
97.30
Train menory (MB)
150.86
477.50
170.13
380.41
243.88
456.86
179.42
1086.83
Test TP (00)
4483.45
879.69
1038.87
311.98
529.83
246.29
577.69
242.11
Test memory (MB)
143.66
440.95
155.21
197.89
176.21
158.36
167.00
717.08
FLOPs (G)
0.02
0.69
0.20
15.16
4.36
0.13
0.11
4.21
#params (M)
0.37
2.37
1.54
2.47
3.76
2.40
2.15
74.33
H Full results
Tab. 12 and Tab. 13 shows the full results from our extensive experiments. We present the MSE and MAE for all forecast horizons T E {96, 192, 336, 720} and their average, respectively. To provide a comprehensive view, we show the average (Tab. 12) and the best (Tab. 13) values over three random seeds. Tab. 14 presents the corresponding full results for iPatch.
For completeness, also included two simple baselines to contextualize model performance in the challenging task of long-term forecasting. We included ARIMA as classic statistical forecasting model (Box & Pierce, 1970) and Last Observation Carried Forward (LOCF), which predict all future time steps by repeating the last observed value from the input sequence (Hewamalage et al., 2023). We observe that, on average, the more recent machine learning models perform substantially better than the simple baselines (Tab. 12, Tab. 13). However, ARIMA and LOCF are among the best-performing models on Exchange. This not surprising, given that stock market. data lacks obvious periodicities and was previously shown to be best predicted by simple baselines (Hewamalage al., 2023). This further supports broader message: model is consistently best, and performance can vary widely depending on the dataset.
Efficiency comparisons
Efficiency metrics setup. The efficiency metrics were computed by evaluating model performance across 1,000 iterations using synthetic input and target data shaped according to the specified sequence length and prediction horizon stemming from the optimal experimental setup found during the HP search. During each iteration, the model was executed in either training inference mode with batch size of one, depending
25

---

Published in Transactions on Machine Learning Research (01/2026)
the configuration, and both the throughput (TP) and peak GPU memory (memory) usage recorded. The TP was quantified as the number of processed sequences per second, calculated by dividing the total number of sequences by the elapsed time. We run the above analyses on a machine equipped with an RTX 4090 NVIDIA GPU and an AMD EPYC 7742 64-Core Processor (128 threads) CPU. Additionally, we computed the number of Bloating-point operations (FLOPs) and total trainable parameters (#params), offering insight into the thcoretical computational complexity of evaluated models. We subsequently scaled all efficiency metrics for interpretability: parameters to millions, FLOPs to gigaflops, and memory usage megabytes (MB).
Efficiency metrics results. In Tab. 11, we provide an overview of the average efficiency metrics for all models over datasets and prediction horizons. Note that for each dataset-horizon combination, we compute the metrics for the configuration found with the HP search. Unsurprisingly, DLinear consistently demonstrates superior efficiency across all metrics, achieving the highest training and inference throughput, lowest memory usage, minimal FLOPs, and the smallest paramcter count. iTransformor is the second most efficient model in all metrics, except. for FLOPs (xLSTMTime). showing an advantageous trade-off between throughput, and memory across the more complex models. TimeMixer and TimeXer lag particularly behind in terms of efficiency against DLincar. Specifically, DLincar attains training throughput of =1,600 sequenccs/s, making it roughly 10x faster than TimeXer and 20x faster than TimeMixer. In terms of training memory, DLinear requires on average 150.86 MB, which about 2.5x less than TixeMixer and 1.6x less than TimeXer.
26

---

Published in Transactions on Machine Learning Research (01/2026)
Table 12: Full results. Mean values for all prediction lengths. Best and second-best are highlightod.
Dataset
Model
PasebTTIT
ITraneformer
TimeXer
8Mamba
ModerwTeN
ARIMA
CLoF
Metric
MSB
MAE
MSB
MAB
MSB
MAE
MSB
MAB
MBB
MAB
MBB
MAB
MSE
MAE
MSB
MAB
MSE
MAB
MSB
MAB
86
0.37
0.384
0.38
0.405
0.081
0.
0.400
0.37
0.397
0.376
0.4
0.386
0.406
0.435
0.367
0:474
0.472
0.964
0.508
1.295
0.713
102
0.423
0.437
0.418
0.428
0.413
0.408
0.411
0.435
0.417
0.431
0.44
0:400
0.493
0.500
0.488
0.967
0.016
1.325
0:793
896
0.582 0.643
0.492
0.488
0.485
0.442
0.435
0.442
0.428
0.434
0.480
0.472
0.400
0.486
0.524
0.501
0.974
0.08
1.38
0.746
0.061
0.485
0.468
0.488
0.563
0.756
Avg
0.474
0.477
0.414
0.482
0.424
0.448
0.429
0.443
0.425
0.44
0.467
0.466
0.68
0.G15
0.082
0.01
0.900
0.021
1.521
0.787
86
0:341
0.374
0.326
0.374
0:337
0.387
0.341
0.387
0.348
0.396
0.335
0.382
0.344
0.386
0.3:2
0.389
0.818
0.638
1.463
0.777
102
0.403
0.407
0.302
0.412
0.403
0.426
0.397
0.427
0.401
0.428
0.402
0.419
0.425
0.426
0.08
1.517
0.805
S96
0.412
0.495
0.406
0.445
0.428
0.402
0.406
0.481
0.428
0.461
0.412
0.435
0.452
0.486
0.421
0.444
0.988
0.714
1.377
0.581
0.485
0.867
Ave
0.408
0.422
0.304
0.491
0.408
0.487
0.482
0.449
0,412
0.445
0.400
0.484
0:484
0.449
0:414
0.486
0.944
0.096
1.351
0.52
06
0.304
0.366
0.288
0.341
0.903
0 353
0.287
0.341
0.266
0.349
0.208
0.351
0.546
0.582
0.322
0:371
0.366
0.388
0.432
0.429
102
0.425
0.448
0.30
0.893
0.384
0.386
0.406
0.906
0.879
0.906
0.652
0.686
0.401
0:408
0.442
0.584
0:478
BTTM
336
0.815
0.467
0.408
0.428
0.403
0.425
0.414
0.429
0.402
0.427
0:417
0.425
0.015
0.874
0.40
0.484
0.528
0.487
0.507
0.511
0.7
0.599
0:48
0.420
0.439
0.467 0:405
0.324
0.533
Avg
0.485
0.474
0.379
0.41
0:078
0 0.405
0.3774
0.404
0:377
0.406
0.383
0.407
0.903
0.647
0.493
0:480
0:06
0.539
0. 0.481
06
0.108
0.221
0.109
0.224
0.118
0.332
0.11
0.225
0.114
0.237
0.118
0.284
0.100
0.223
0.111
0.227
0.102
0.206
0.202
0.31
192
0.184
0.259
0.146
0.202
0.166
0.269
0.161
0.265
0:146
0.264
0.144
0.261
0.387
BTTm2
303
0.142
0.271
0.168
0.281
0.183
0.29
0.179
0.285
0.176
0.267
0.181
0.291
0.183
0.29
0.188
0.293
0.258
0.544
0.27
0.561
0.203
0.335
0:32
0.338
0.233
0.344
0.324
0.324
0.389
0.335
0.401
Avg
0.1.1
0.201
0.150
0.
0.268
0.106
0.276
001102
0.27
0.166
0.277
0.168
0.279
0.078 0.11
0.166
0.976
0.25
0:34
0.261
0.359
96
0.185
0.28
0.18
0.227
0.185
0.28
0.131
0.228
0.187
0.24
0.182
0.28
0.18
0.225
0.268
0. 0.866
1.506
0.802
1.588
0.946
0.147
0.241
0.142
0.160
0.246
1:457
0.951
Electricity
301
0.148
0:282
0.172
0.272
0.17
0.200
0.165
0.205
0.160
0.261
0.173
0.27
0.171
0.267
0:275
0.377
0.954 1.824
1.628
0.961
0.903
0.304
0.305
0.267
0.299
0.903
0.3
0.39
0.305
0.975
Avg
0.162
0.250
0.164
0.261
0.165
0.302
0.158
0.254
0.17
0.268
0.166
0.263
0.168
0.267
0.276
0.876
2.058
0.961
1:012
0.008
86
0.166
0.228
0.145
0.185
0.164
0.216
0.137
0.211
0:147
0.199
0.156
0.91
0.18
0.195
0:172
0.236
0.213
0. 0.253
0.258
0.254
0.247
0:207
0.247
0.262
0.292
Weather
336
0.263
0.331
0.243
0.282
0.36
0.3
0.255
0.289
0.245
0.266
0.261
0.301
0,243
0. 0.281
0:248
0.29
0.32
0.31
0.376
0.338
0.381
0.377
0.312
0.33
0.399
0.343
0.013
0.338
0.31
0.396
0.344
0.348
0.349
0.306
0.465
Ave
0.244
0.298
0.225
0.208
0.288
0.270
0.281
0.271
0.224
0.264
0.287
0.277
0.229
0.268
0.241
0. 0.262
0.501
0.321
0.352
0.310
86
0.102
0.23
0.085
0.203
0.092
0. 0.214
0.084
0.901
0.086
0.905
0. 0.086
0.906
0.149
0.28
0.088
0. 0.906
0.092
0. 1.8
0.081
0.186
102
0.118
0.20
0.197
0.317
0.184
0.507
0.76
0.626
0.181
0.184
0.906
0.686
0.96
0.916
0.108
0.289
0.167
0.260
Tirhhange
336
0.63.
0.561
0.338
0.421
0:354
0:431
0.384
0.119
0.359
0.133
0.454
0.196
0.088
0.645
0.4
0.457
0.316
0.400
0.300
0.398
720
0.678
0.680
0.849
0.698
0.884
0.71
0.078
0.78
0.868
0.081
0.742
1.481
0.016
3.850
1.146
0.914
0.087
0.81
0.070
Ave
0.869
0.454
0.368
0.409
0.379
0.416
0.68
0.479
0.874
0.412
0.426
0.438
0.009
0.6
0.701
0. 0.681
0.946
0.396
0.841
0.90
06
2.734
0.918
2.049
0.781
0.472
0. 1.165
2:044
0.754
3:27
0.344
0.108
0.127
6.583
1 1:286
1:728
0.581
4.702
1.100
2:192
0.97
102
4.560
1.152
0.272
3.604
0.048
3.161
0.224
0.121
7.679
1.878
0.748
1.378
3.480
1.04
Mutor
336
5,965
1.528
4.815
1.005
2,373
0. 4.437
4.995
1.119
5.177
1.175
0.864
0.261
7.25
1.465
4.042
0.922
7.04
1.482
5.08
1.211
Imagery
720
5,101
1.841
4.905
1.207
3.08
0.00
4.08
1.224
4.988
1:268
1.726
0.486
6.128
1.37
3.701
0.874
6.047
1.280
5.805
1.341
Avg
4.892
1.189
3,775
1.009
1,092
0 0.583
3,860
1.011
3.040
0.915
0.748
0.244
6.022
1.362
3.088
0.789
6.818
1.921
4.018
1.116
96
0.769
0.65
0.664
0.606
0.078
0. 0.566
0.678
0.696
0.057
0.591
0.67
0.506
0.87
0. 0.667
0.601
0. 0.604
0.563
0.676
1.106
0.764
192
0.669
0.007
0.689
0.824
0.827
0.064
0.789
0.072
0.780
0.820
TDBrain
331
1.286
0.848
1.008
0.754
1.003
0.740
1.037
0.75
1.05
0.748
3.042
0.744
1.317
0.836
1.056
0.751
1.207
0.800
1.487
0.595
720
1.647
0.004
1,863
0.88
1.866
0.876
1.858
0.877
1.948
0.872
1.840
0.874
L.622
0.946
1.844
0.871
1.496
0.92
1.752
1.007
Avg
1.153
0.802
0.982
0.725
0.978
0.722
0.981
0.725
0.97
0.719
0.972
0.72
1.211
0.799
0.984
0.725
1.348
0.780
1.406
0,873
96
0.629
0.441
0.522
0:43
0.64
0.441
0.651
0.452
0.026
0.450
0.662
0.442
0.687
0.41
0.527
0. 0.485
0.666
0.504
1.10
0.598
0.869
0.640
BeiionAAr
306
0.391
0.473
0.593
0.465
0.395
0.488
0.589
0.487
0.588
0.465
0.584
0.461
0.372
0.435
0.588
0. 0.463
0.756
0.548
1.077
0.668
0.643
0.508
0.488
0.615
0.488
0.495
0.629
0.567
0.453
0.008
0.452
0.461
0.788
0.571
0.659
Avg
0.083
0.472
0.678
01:88
0.58
014484
0.099
0.471
0.582
0.462
0.567
0.452
0.500
0.48
0.581
0.463
0.737
0.541
1.35
0.633
96
0.007
0.016
0.01
0.041
0.01
0.068
0.006
0.010
0.000
0.041
0.007
0.057
0.008
0.016
0.281
0.99
0.081
0.704
1.254
0.502
Honzone
0.157
0.9
Concen-
336
0.008
0.019
0.009
0.042
0.013
0.058
0.008
0.024
0.011
0.032
0.011
0.053
0.008
0.024
0:131
0.258
1.053
0.929
1:281
0.47
tration
720
0.011
0.025
0.016
0.058
0.015
0.058
0.013
0.037
0.016
0.038
0.014
0.048
0.015
0.041
0.1
0.356
1.104
0.950
1:281
0.871
Ave
0.008
0.02
0.011
0.042
0.012
0.063
0.008
0.022
0.012
0.037
0.01
0.044
0.01
0.028
0.16
0.266
1.02
0.816
1,208
0.870
96
0.808
0.75
0.815
0.752
0.800
0.727
0.815
0.729
0.807
0.728
0.808
0.728
0.768
0.716
0.84
0.745
1.011
1.002
1.578
1.058
Austzalia
0.758
0.851
0.755
0.847
0.847
0.762
1.115
Rainfall
336
0.83
0.76
0.861
0.761
0.863
0. 0.763
0.808
0.755
0.861
0.752
0.863
0.761
0.841
0.754
0.888
0.737
1.976
1.128
1.945
1.118
720
0.858
0.704
0.877
0.772
0.874
0.77
0.876
0.771
0.871
0.760
0.878
0.769
0.853
0.701
0.503
0.78
2.011
1.141
1.070
1.182
Avg
0.838
0.751
0.865
0.757
0.849
0.764
0.853
0.755
0.847
0.758
0.846
0.753
0.827
0.743
0.872
0.766
1.909
1.121
1.938
1.112
96
1.086
0.627
1.169
0.652
1.186
0.658
1:12
0.63
1.145
0.636
1.188
0.654
1.136
0.633
1.158
0.851
1.152
0.881
1.508
0.782
KDDCup
0.641
1.034
0.774
2018
336
1.021
0.649
1.092
0.658
1.065
0 0.653
1.611
0.631
1.01
0.638
1.068
0.65
1.014
0.026
1.022
0.026
1.04
0.654
2.097
0.782
720
0.001
0.819
0.99
0.684
1.008
0.086
0.050
0.610
0.99
0.630
0.008
0.682
0.979
0.028
1.002
0.084
1.088
0.081
1.492
0.821
Avg
0.997
0.93
1.088
0.647
1.088
0.048
1.0:3
0.631
1.045
0.64
1.086
0.646
1.051
0.822
1.008
0.038
1.087
0.904
1.627
0.79
96
0.239
0.255
0.22
0.254
0.226
0.346
0.225
0.248
0.210
0.245
0.216
0.295
0:311
0.222
0.456
0. 0.408
1.008 2.656
1.995
0.064
Pedestrion
0.374
0.955
0.255
0.339
0.377
0.24
0.38
Couats
386
0.807
0.296
0.305
0.803
0.312
0.306
0.297
0.281
0.289
0.287
0.364
0.291
0.201
0.305
0.451
0.4
2.663
1.123
3.065
0.088
720
0.881
0.352
0.364
0.841
0.386
0.984
0.880
0.338
0.80
0.927
0.99
0.982
0.970
0.81
0.541
0.440
3.011
1.208
2.238
1.062
Avg
0.298
0.289
0.291
(1.293
0,290
0.
0.280
0.299
0.284
0.311
0.307
0.299
0.275
0.282
0.259
0:488
0.408
2.764
1.133
2,052
0.99
Avg
Avs
0.77
0.454
0.601
0.455
0.647
0. 1.41
0.713
0.462
0.688
0.458
0.484
0.407
0.971
0.619
0.714
0.40
1.419
0.726
1.502
0.740
Raak
Ava
4.79
4.80
8.64
4.07
4.6
4.04
4:21
4.29
4:21
4.29
4.21
4.29
6.67
5.14
0.04
6.14
8.43
8.50
27

---

Published in Transactions on Machine Learning Research (01/2026)
Table 13: Full results. Best values for all prediction lengths. Best and second-best are highlighted.
Datoset
Model Metric
DLineur
PatchTST
P'rnarformer
TimeMMssr
TimeXer
BMambu
xLSTMcTime
MederaTTN
ARIMA
CLOF
MAB
MBG
MAR
MBG
MAR
MAR
MAR
MBR
MAR
MAR
MSR
MAR
MER
MAR
MaK
MAR
96
0,599
0.405
0.379
0,308
0.371
0.598
0.405
0,452
0.450
0.467
0.944
0.508
1.204
0.719
192
0.423
0:437
0:412
0.428
0.411
0.437
0.404
0.420
0.407
0.423
0.428
0.439
0.449
0.451
0.502
0.480
0.907
0.516
1.325
0.733
836
0.548
0.582
0.417
0.481
0.438
0.441
0.484
0.438
0.422
0.481
0.47
0.40
0.47
0.491
0.522
0.490
0.074
0.03
1.33
0.740
720
0,330
0.543
0.423
0.454
0.480
0.480
0.493
0.494
0.463
0.493
0.036
0.542
0.803
0.578
0.098
0.588
0.975
0.661
1.335
0.750
Avg
0:472
0.376
0.408
0.028
0:423
0:441
0.124
0.439
0.42
0.438
0.45
0.481
0.487 0.488
0.522
0. 505
0.9655
0.521
1.021
0.737
0.341
0.374
0.371
0.339
0.38
0.339
0.339
0.331
0.378
0.818
102
0.402
0.407
0.361
0.41
0:401
0.424
0.300
0.413
0.808
0.422
0.395
0.410
0.387
0.432
0.414
0.421
0.912
0.68
1.617
0.806
336
0:409
0.482
0.402
0.443
0.410
0.447
0.487
0.473
0.425
0.457
0.411
0.438
0.448
0.452
0.421
0.443
0.083
0.714
1.677
0.891
720
0.457
0.47
0:479
0.494
0.406
0.482
0.477
0.491
0.465
0.489
0.485
01 0.4915
0.038
0.506
0.467
0.478
1.085
0.752
1.647
0.887
Ave
0.402
0.431
0.801
0.429
0.408
0.453
0,416
0.459
0.405
0.44
0.405
0.452
0.425
0.441
0.411
0.492
0.044
0.090
1.651
0.82
0.301
0.306
0:281
0.339
0.296
0.349
0.984
0.841
0:295
0.346
0:297
0.35
0.533
0.54
0.32
0.300
0.386
0.432
0.422
192
0.428
0.448
0.360
0.89
0.379
0.397
0.384
0.404
0.862
0.505
0.375
0.300
0.816
0.5
0.380
0.401
0.408
0.442
0.684
0.478
336
0.513
0.487
0.403
0.424
0.398
0.423
0.397
0.42
0.398
0.422
0.436
0.427
0.59
0.550
0.432
0:45
0.528
0.487
0.807
0.511
720
0.600
0.009
0:456
0.471
0.443
1.429
0.914
0.441
0.400
0.570
0.524
0.694
0.519
Ave
0.484
0.474
0.374
0.406
0.377
0.403
0. 0.368
0.401
0.375
0.404
0.88
0.405
0.767
0.628
0.4
0.42
0.48 0.455
0.839
0.451
06
0.108
0.291
0.100
0.228
0:118
0.382
0.100
0.224
0.113
0.227
0.117
0.288
0.1.07
0.22
0.111
0.227
0.192
0.295
0.202
0.31
192
0.184
0.247
0.198
0.265
0.147
0.201
0.139
0.253
0.15
0.294
0.140
0.294
0.138
0.248
0.148
0.20
0.224
0.524
0.295
0.387
BTTm2
886
0.166
0.271
0.166
0.279
0.182
0.350
0.155
0.283
0.177
0.255
0.176
0.258
0.18
0.258
0.181
0.359
0.258
0.348
0.27
0.361
720
0.201
0.904
0.208
0.528
0.289
0.338
0.222
0.389 0.324
0.401
Ave
0.2.5
0.261
0.155
0. 0.208
00.105
0.275
0.16
0.269
0.165
00.274
0.165
0.277
0.165
0.273
0.164
0.275
0.34 0.25
0.261
0.352
06
0.188
0:28
0.18
0.227
0.134
0.220
0.18
0.227
0.187
0.239
0.120
0.228
0.18
0.224
0.249
0.350
1.300
0.802
1.688
0.046
102
0.147
0.243
0.146
0.241
0.149
0.246
0.146
0.148
0.247
0.156
0.251
0.149
0.242
0.254
0.361
1.437
0.9
1.595
0.951
Electricity
886
0.168
0.261
0.171
0.271
0.168
0.28
0.168
0.262
0.10
0.257
0.100
0.268
0.168
0.266
0.269
0.378
1.824
0.064
1.618
0.001
720
0.304
0.302
5.624
0.975
Ave
0.361
0.256
0.163
0.261
0.162
0.258
0.154
0.252
0.165
0.253
0.164
0.251
0.16
0.255
0.258
0.372
2.053
0.951
1.632
0.958
86
0.168
0.226
0.145
0.195
0.163
0.214
0.149
0.301
0:145
0.197
0.134
0.208
0.149
0.385
0.17
222
0.212
0. 0.252
0.259
0.251
102
0.216
0.275
0.196
0.246
0.207
0.358
0.194
0.188
0.239
0.202
0.349
0.197
0.244
0.201
0.353
0.235
0.394
0.309
0.292
Weather
886
0.202
0.916
0.241
0.281
0.268
0.200
0.26
0.285
0.248
0.252
0.281
0.294
0.241
0.279
0.247
0.288
0.32
0.54
0.378
0.558
0.326
0.32
0.309
0.438
0.594
Ave
0.264
0.298
0.228
0.262
0.287
0.278
0.225
0.264
0.232
0.202
0.23
0.272
0.226
0.206
0.233
0.277
0.301
0.321
0.352
0.310
66
0.081
0.202
0.084
0.201
0.089
0.21
0. 0.082
0.2
0.085
0.204
0.085
0.206
0.142
0.28
0.085
203
0.082
0. 188
0.081
0.186
102
0.165
0.284
0.318
0.181
0.308
0.171
0.297
0.16
0.302
0.175
0.299
0.431
0.507
0.174
2.0
0.168
0.289
0.167
0.350
Tichhange
336
0.500
0.424
0.528
0.418
0.546
0.427
0,360
0.437
0.35
0.425
0.417
0.475
0.658
0.625
0.375
0.44
0.310
0.400
0.506
0.508
0.514
0.87
0.092
0.848
0.687
0.007
0.704
0.865
0.705
0.930
0.734
1.857
0.591
2.258
0.814
0.687
0.81
0.670
Avw
0.265
0.37
0.302
0.405
0.304
0.409
0.36
0.400
0.87
0.409
0.409
0.428
0.646
0.570
0.728
0.516
0.345
0.306
0.341
0.50
66
2.728
0.011
3.028
0.779
0.438
0.1
1, .805
0.721
1:15
0. 0.306
0.168
0.124
6.415
1.232
1.659
0. 0.522
4.702
1.100
2.192
0.87
102
4.561
1.16
0.984
0.709
8.463
3.11
0.557
0.110
7.382
1.358
2.868
0.484
1.378
1.04
Motor
Imagory
336
5.956
1.397
4.777
1.088
9.071
0.345
1.021
1.105
5.061
1.146
0.086
0.282
6.927
1.399
4.014
0. 0.116
7:04
1 1.452
5.08
1:211
720
5.098
1.341
4.38G
1.2
2.700
4.013
1.22
4.084
1.264
1.005
0.428
5.006
1.330
3.07
0.508
5.047
1.280
5.805
1.341
Avg
4.583
1.182
3.745
1.496
0.338
8,813
0.909
3.670
0.893
0.070
0.225
0.432
1.325
5.054
0.702
5.818
1.521
4.028
1.315
96
0.709
0.05
11.092
0.595
0.07
0.307
0.854
0.092
0.657
0.501
0.002
0.592
0.861
0.632
0.691
0.603
0.865
0.675
1.108
0.764
0.609
1.013
0.738
TDlrann
a86
1.286
0.946
1.047
0.747
1:044
0.745
1.049
0.747
1.087
0.743
1.038
0.742
1.307
0.632
1.055
0.7 0.761
1.207
0.800
1.467
0.505
720
1.544
0.963
1.301
0.879
1.366
0.876
1.854
0.870
1.844
0.871
1.947
0.873
1.602
0.037
1.844
0.871
0.02 1.405
1.782
1.007
Ave
1.15
0.801
0.976
0.722
0.974
0.721
0,974
0.72
0.960
0.717
0.967
0.717
0.794 1.198
0.983
0.724
0.788 1.145
1.404
0.873
96
0.328
0.44
0.518
0.328
0.515
0.433
0.526
0.443
0.522
0.43
0.535
0.437
0.528
0.408
0.527
0.435
0.655
0.500
1.36
0.598
0.063
0.568
0.576
0.563
0.567
0.74
0.541
BallnggAr
a86
0.501
0.473
0.50
0.466
0.508
0.407
0.688
0.460
0.086
0.464
0.682
0.40
0.500
0.488
0.687
0.468
0.730
0.548
1.077
0.628
720
0.541
0.507
0.523
0-186
0.613
0.187
0.039
0.193
0.635
0.191
0.547
0.445
0.605
0.118
0.638
0. 198
0.788
0.571
1.149
0.653
0.589
0.471
0.576
0.468
0.572
0:461
0.66
0.467
0.677
0.459
0.650
0.4
0.004
0.427
0.38
0.737
0.541
1.15
0.691
96
0.007
0.014
0.009
0.039
0-009
0.0.8
0.006
0.013
0.009
0.00 0.038
0.007
0.033
0.008
0.013
0.279
0.330 0.388
0.931
0.764
1.261
0.862
Bonsone
0.000
0.013
0.024
0.008
0.035
0.006
0.010
0.01
0.033
0.041
0.017
0.052
1.3
0.51 1.008
1.345
0.9
Concen-
3336
0.008
0.017
0.009
0.04
0.013
0.055
0.007
0.02
0.013
0.03
0.03
0.058
0.008
0.011
0.127
252
1.043
0.829
1.281
0.87
trasion
720
0.011
0.026
0.015
0.05
0.014
0.053
0.012
0.026
0.014
0. 0.024
0.013
0.06
0.014
0.028
0.13
252
1.104
0.856
1.281
0.871
0.008
0.038
0.008
0.011
0.052
0.041
0.009
0.017
0.147
0.262
1.02
0.816
1:293
0.870
66
0.905
0:739
0.814
0.731
0.908
0.727
0.81
0.738
0.805
0.726
0.805
0.725
0.784
0.715
0.538
0.740
1.911
1 1.082
1.878
1.050
Australta
192
0.839
0.78
0.555
0.786
0.849
0.754
0.802
0.754
0.847
0.752
0.845
0.751
0.825
0.742
0.867
0.762
1.978
1.122
1.949
1.335
Raisfall
336
0.85
0:759
0.864
0.763
0.962
0.763
0.865
0.761
0.839
0.761
0.763
0.81
0.753
0.883
777
1.976
1.128
1.941
1.118
720
0.807
0. 764
0.772 0.877
0.874
0.77
0.874
0.77
0.871
0.768
0.873
0.769
0.852
0.761
0.692
0.778
1.141 2.011
1.079
1.182
0.756
0.763
0.86
0.840
0.752
0.87
1.121 1.909
1.112
96
1.082
0.627
1.14
0.047
1.188
0.054
1.113
0.627
1.111
0.645
1.185
0.633
1.128
0.028
1.141
0.85
1.152
0.601
1.805
0.752
KDDCup
192
0.978
0.026
1.08
0.645
1.087
0.648
1.044
0.638
1.0134
0.628
1.084
0.645
1.072
0.626
1.088
0.638
1.059
0.558
1.434
0.774
336
1.006
0.641
1.085
0.654
1.059
0.65
1.008
0.980
0.63
1.034
0.635
0.621
1.021
0.621
1.04
0.654
2.097
0.763
720
0.869
0.010
0.988
0.681
1.007
0.686
0
0.906
0.617
0.975
0.620
0.00
0.632
0.977
0.632
1
0.684
1.088
0. 0.681
1.492
0.821
0.037
0.611
1,03
1.03
1.052
1.087
0.78
96
0.238
0.257
0.218
0.252
0.22
0.2411
0.219
0.233
0.215
0.241
0.212
0.227
0.236
0.222
0.435
01:411
2.635
1.093
1.595
0.954
Pedestsian
192
8.265
0.272
0.255
0.273
0.253
0.255
0.253
0.365
0.332
0.331
0.253
0.258
0.244
0.24
0.415
0.378
2.644
1.1
2.029
0.961
336
0.307
0.205
0.303
0.301
0.300
0.303
0.295
0.283
0.284
0.252
0.302
0.391
0.291
0.265
0.451
0.300
2.603
1,123
2.065
0.958
Counts
730
0.391
0:302
0.378
0.333
0-383
0.3332
0.375
0.324
0.358
0.323
0.385
0..3
0.172
0.307
0.54
08:38
3.031
1.209
2.338
1.052
0.289
0.281
0.386
0.395
0.285
0.977
0.455
2.744
0.90
Avg
Avg
0.759
0.376
0.586
0.455
0.529
0.41
0.099
0.452
0.673
0.448
0.474
0.403
0.945
0.508
0.708
00:488
3.439
0.722
1.392
0.748
Rusk
Avs
4.04
4.71
3.50
4.29
4.6
8
9.58
3.79
4.29
6.14
4.45
4.48
5.45
4.50
0.57
0.50
8.5
8.6
8.05
8.93
28

---

Published in Transactions on Machine Learning Research (01/2026)
Table 14: Full results iPatch. We present TSLib (left) and UTSD datasets (right). See Tab. 4 for average MSE, MAE, and Rank.
Model
iPatch
Model
iPatch
Datnset
Metric
MAE
MSE
Dataset
Metric
MAE
MSE
Statistic
Mean
Min
Mean
Min
Statistic
Mean
Min
Mean
Mín
96
0.4074
0.3995
0.3877
0.379
96
0.1433
0.1287
0.4238
0.3541
192
0.4241
0.4221
0.415
0.413
192
0.2633
0.2079
1.0616
0.7352
ETTh1
336
0.4301
0.4284
0.4241
0.4198
MotorImagery
336
0.3999
0.3458
2.183
1,8659
720
0.4635
0.4599
0.45
0.4451
720
1,1959
1.1879
4.8701
4.8506
Average
0.4313
0.4275
0.4192
0.4142
Average
0.5006
0.4676
2.1346
1.9515
0.383
0.3726
0.3229
0.3115
96
0.5985
0.5909
0.6747
0.6592
192
0.418
0.4171
0.3749
0.3719
192
0.6704
0.665
0,8383
0.824
ETTml
336
0.4478
0.446
0.4289
0.4259
TDBrain
336
0.7483
0.7423
1.0502
1.036
720
0.4963
0.4919
0.4937
0.4862
720
0.871
0.8704
1.3459
1.3451
Average
0.4363
0.4319
0.4051
0.3989
Average
0.722
0.7171
0.9773
0.9661
0.3445
0.3423
0.2935
0.2902
96
0.43445
0,4302
0,5334
0.5231
192
0.3953
0.3931
0.3759
0.3711
192
0.4559
0.455
0.569
0.5643
ETTh2
336
0.4337
0.4323
0.4226
0.4205
BeijingAir
336
0.4665
0.4658
0.5962
0.5941
720
0.4443
0.4419
0.4197
0.4172
720
0.4721
0.4685
0.5909
0.5843
Average
0.4045
0.4024
0.3779
0.3747
Average
0.4573
0.4549
0.5724
0.5665
0.2282
0.2281
0.1145
0.1134
96
0.0197
0.0192
0.0059
0.0059
0.2595
0.2593
0.146
0.1457
192
0.0247
0.0244
0.0112
0.0111
ETTm2
336
0.2865
0.285
0.18
0.1786
BenzeneConcenttaatin
336
0.0188
0.0166
0.0077
0.0076
720
0.3181
0.3148
0.214
0.2099
720
0.0315
0.0302
0.0152
0.0149
Average
0.2731
0.2718
0.1636
0.1619
Average
0.0237
0.0226
0.01
0.0098
0.2315
0.2304
0.1333
0.133
96
0.7284
0.7274
0.8115
0.8093
192
0.2471
0.2456
0.1511
0.15
0.7533
0.7524
0.85
0.8484
Electricity
0.2808
0.2782
0.1798
0.1769
Australia Rainfall
336
0.7623
0.7608
0.8624
0.8583
720
0.3196
0.3159
0.2295
0,2268
720
0.7691
0.7683
0.8719
0.8702
Average
0.2698
0.2675
0.1734
0.1717
Average
0.7533
0.7522
0.8489
0.8466
96
0.204
0.2031
0.1532
0.1525
96
0.6556
0.6539
1.1853
1.1828
192
0.2501
0.2494
0.2032
0.2016
192
0.6455
0.6454
1.0951
1.0948
Weather
336
0.2879
0.2878
0.2516
0.2511
KDDCup2018
336
0.6397
0.6366
1.0313
L.0267
720
0.3348
0.3337
0.3202
0.3192
720
0.6504
0.6478
1.0057
0.9874
Avcrage
0.2692
0.2685
0.232
0,2311
Average
0.6478
0.6459
1.0793
1.0729
96
0.21 46
0.2114
0.0934
0.0906
96
0.2316
0.2309
0.2139
0.213
192
0.3127
0.3067
0.1901
0.1853
192
0.2628
0.2498
0.2564
0.2475
Exchange
336
0.4260
0.4219
0.3461
0.3397
PodestrianCount
336
0.2797
0.274
0.2939
0.2894
720
0.8205
0.8104
1.1578
1.1318
720
0.3196
0.318
0.3726
0.372
Average
0.4435
0.4376
0.4469
0.4369
Average
0.2734
0.2682
0.2842
0.2805
29


Reviews:
[{"review_id": "F0nSlBfOPN", "reviewer": "Reviewer_w6it", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper takes the position that, in long-term time series forecasting, improvements reported in multiple “SOTA” papers are merely incremental (i.e., not statistically significant) and are partly an artifact of heterogeneous experimental setups. The authors argue that “SOTA” claims can flip when one changes datasets, metrics, or hyperparameter search spaces. To support this with evidence, the authors pick 5 models from the TSLib leaderboard (plus a hybrid model they propose) to illustrate this behavior. The core message is that, instead of chasing ever more complex architectures (e.g., moving from TimeMixer to TimeMixer++), the community should standardize benchmarking, hyperparameter reporting, and statistical testing.\n\n### Strength:\n The topic is very important and has already been mentioned in several works/workshops, which emphasizes its relevance.\n\n### Weakness:\n Although the paper mentions evaluation issues (though not exhaustively), it omits the 2024–2025 generation of foundation time-series models (Chronos, TimesFM, Moirai, TiRex) that were explicitly built to address cross-domain/zero-shot evaluation and that are now routinely reported on newer benchmarks such as GIFT-Eval. As a result, the central claim “there are no champions” is only demonstrated for a restricted, older slice of the model space. This goes beyond foundation models: completely newer model classes that target the claimed missing efficiency analysis, like xLSTMs (e.g. xLSTM-Mixer) or SSMs (e.g. Mamba), are also absent.", "claims_and_evidence_yn": "No", "claims_explanation": "The empirical study is careful within its chosen scope, but that scope is too narrow for the paper’s headline conclusion. GIFT-Eval (which is not mentioned at all) was created specifically to evaluate supervised and foundation TS models in various scenarios, and it now comes with baselines for classical, deep, and FM-style models. Excluding it weakens a paper as one of the claims is that current evaluation is insufficient.\n\nLikewise, the paper does not evaluate any of the major TS foundation models that, by 2025, are the default. Chronos/Chronos-2, TimesFM, Moirai, and TiRex. All of these report on GIFT-Eval (partly also on TSLib) or very similar broad suites and show strong zero-shot or few-shot behavior. While I see the point of not including all of them, including at least all relevant architecture families that are not necessarily more complex (convolutions, xLSTMs, SSMs) is highly recommended to strengthen the claim of the paper.\n\nLeaving them out makes the conclusion \"\"slight experimental changes overturn SOTA\" less convincing, because we don’t know whether that also holds for the very models that were designed to be robust and efficient.\n\nFinally, the paper’s story \"\"SOTA is unstable\"\" is quite close to the line of argument in “Accuracy Law for the Future of Deep Time Series Forecasting” by Wang et al., 2025, which also explains why progress looks saturated and provides a statistical view over many forecasters. While this work is concurrent, I strongly suggest to include/discuss this work explicitly.", "audience_yn": "Yes", "audience_explanation": "TMLR has been publishing a lot of time-series papers recently in various scnearios, and a paper that tries to clean up evaluation is relevant if it engages with the currently used FM-era setups and adds all major model families or, at minimum, provides stronger justification for a restricted selection. In that case, I think this would be important work. Right now, it speaks more to “2023-style LTSF papers.” Lastly, the inclusion of GIFT-Eval would make it clearly useful to today’s audience.", "requested_changes": "### Critical:\n\nAdd an evaluation on GIFT-Eval (or explain convincingly why it’s not applicable). This benchmark was introduced to evaluate general/foundation TS models across heterogeneous domains, which is exactly the problem the paper claims to address. Showing whether the “no champions” phenomenon persists on GIFT-Eval would make the conclusions much stronger.\n\nBroaden the architectural coverage. Right now the selection seems skewed and omits whole families that are still active in TS (SSM-based forecasters, conv-style models, xLSTMs).\n\n### Optional:\n\nInclude representative foundation models e.g. Chronos, TimesFM , Moirai, or TiRex. Even one or two of these, evaluated under the same perturbations the authors use, would already tell us whether the observed ranking volatility is a general phenomenon or an artifact of older supervised models.\n\nClarify novelty vs. “Accuracy Law for the Future of Deep Time Series Forecasting.” Explicitly state what this paper adds. Right now they read as different takes on the same concern.\n\nRelease code / HP search spaces or reference a repo, in line with the paper’s own reproducibility message.", "broader_impact_concerns": "No major ethical/societal concerns. This is an evaluation/benchmarking paper.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "U1A3RWPR30", "reviewer": "Reviewer_ndsu", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper investigates current time series forecasting models and datasets on their significance concerning obtained results in previous literature. The authors find most models to perform comparably across tasks while the underlying papers claimed improvements in performance. Further, they outline issues with currently used evaluation metrics and experiment procedures. They support their observations with evaluations on TSlib datasets for long term forecasting. \nIn addition the authors outline current issues with visualization and data driven selection of models, methods currently used in time series forecasting research. \n\nFrom these observations the authors propose multiple positions in order to align time series data research better with the goals of increasing model performance.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The claims made by the authors are mostly supported by evidence. As for the experimental results claims made in the paper can be observed in experimental observations highlighting the current issues of long term time series forecasting. Throughout the paper the authors provide results highlighting the empirical issues observed in time series forecasting as well as explanations to common pitfalls or issues with current models. Furthermore, the evaluation of multiple current models provides clear intuition on how these results are affecting research directions and potential model evaluation. \n\nHowever, the positions are at times only supported by empirical evidence and no proposed direction is given in the paper. Furthermore, the experimental results do not contain some current models which would be useful to strengthen the claim of no model being the \"champion\" in long term time series forecasting. Furthermore, some of the positions feel disconnected from each other.", "audience_yn": "Yes", "audience_explanation": "As the paper outlines current issues with time series data forecasting it would be interesting to the TMLR audience. Especially the comparison of different current models outlines the missing advantage of certain models as soon as they are evaluated against each other. Furthermore, arising issues and shortcomings with current evaluation metrics are highlighted in the paper, allowing for a more concise analysis of current models by interested readers. \n\nWith similar observations made in other domains and areas of machine learning research I find this paper to be of interest to TMLR readers not only in time series forecasting. \n\nHowever, a clear message is lacking in the current state of the paper (see requested changes). Therefore, to make the paper more interesting to the audience I would suggest the changes proposed below in the next section.", "requested_changes": "C1: The paper currently reads like a position track paper from one of the major conferences such as NeurIPS or ICML. To fit TMLR properly I would suggest to rework the paper to fit more the style of a TMLR submission instead. \n\nC2: Going along with C1 I feel like the alternative view section is disconnected from the remaining paper. As the paper is not a position paper (or at least not clearly labeled as such) the section does not fit to other sections in the paper. Furthermore, I'm not convinced of the LLM hypothesis from section 6 but as I'm not an expert on time series data I'm willing to accept the hypothesis given suitable evidence. \n\nC3: The paper starts with clearly defined positions in section 1. However, these positions are only partly explored. For example the authors provide guidance on existing datasets not being a good choice for model evaluation. But they don't provide additional insights on how to improve on existing datasets. Similar things occur with the evaluation metrics criticized by the authors. I would like to see a more in depth discussion of these concerns. \n\nC4: The experimental evaluation is only in parts sufficient. As outlined by the authors less complex models such as DLinear perform similar to more complex models. However, there is no clear reasoning on why this is happening or how more complex models fail to perform better (except MAE values from the experiments). Further, there are many more models for long term time series forecasting which are not evaluated at all in the current paper. Since the positions refer to the field as a whole I would like to see some additional models to strengthen the experimental findings. \n\nMoreover, the dataset selection consists of a variety of datasets but is limited to toy datasets. As application to real world data may differ it would be interesting to see additional results outlining performance on real world datasets. \n\nC5: Overall, I'm missing a clear direction the paper wants to propose. The authors showcase current limitations in time series data research. Nonetheless, a clear future direction and research suggestions are lacking in the current paper. Since the paper reads like a position paper I would expect additional suggestions and proposals in the paper.", "broader_impact_concerns": "There are no broader impact concerns from this work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "mRAblN73DY", "reviewer": "Reviewer_SQ1T", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper argues that recent gains reported for long-term time-series forecasting (LTSF) models may stem less from model innovations and more from evaluation choices. This includes dataset selection, horizon, hyperparameters, and baseline inclusion. The authors re-evaluate several popular models and conclude that there is no single consistent winner. Additionally, they advocate for rigorous, standardized evaluation for future research.\n\n**Strengths:**\n\nI agree with the author’s position that many recent studies have minor differences in experimental settings. This raises the concern if the performance benefits are actually *only* due to the modeling novelty as these research papers generally claim. Standardization of the experiments is an important aspect of the scientific investigation, and this work is an attempt in that direction for LTSF.\n\n**Weakness:**\n\nI’m concerned that the best performing models in the main results (table 12 and 13) are significantly worse than what the other papers report. For example, I compared the results from this work with the results in [1] (that they used to define the hyperparameter search), and note that the performance of top-performing model is much lower in this study. This discrepancy coupled with the authors *not* publishing their code, makes it less convincing and difficult to assess the fairness of *this* work. Since all the claims, discussion and future recommendations, i.e. the contributions of this work are heavily based on these results, I think this is a major limitation.\n\n**References:**\n1. Wang, S., Wu, H., Shi, X., Hu, T., Luo, H., Ma, L., ... & ZHOU, J. TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting. In *The Twelfth International Conference on Learning Representations*.", "claims_and_evidence_yn": "No", "claims_explanation": "The claims are not well justified given the reasons mentioned in weakness section.", "audience_yn": "Yes", "audience_explanation": "This is an interesting position paper. It can potentially motivate future research work to include rigorous standardized benchmarking results.", "requested_changes": "(critical): Can the authors clarify why the top-performing models are significantly weaker in comparison to the ones reported in existing works?\n\n(strengthen): The paper can benefit from the analysis on synthetic datasets with known patterns.", "broader_impact_concerns": "Broader Impact Statement is not included in the paper.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)