ModernTCN Revisited: A Critical Look at the Experimental Setup in General Time Series Analysis
Authors: Önder Akacik
Mark Hoogendoorn
Venue: Accepted by TMLR ()

Abstract: While numerous time series models claim state-of-the-art performance, their evaluation often relies on flawed experimental setups, leading to questionable conclusions. This study provides a critical re-evaluation of this landscape, using ModernTCN as a case study. We conduct a rigorous and extended benchmark, correcting methodological issues related to data loading, validation, and evaluation methods, and show that performance claims are sensitive to these details. Additionally, we find that ModernTCN overlooks a line of research in global convolutional models, and our comparison reveals that despite claims of an enlarged effective receptive field (ERF), it falls short of these methods. More than a critique, we introduce an architectural innovation: by embedding irregularly sampled data with a continuous kernel convolution and processing it with the ModernTCN backbone, we achieve new state-of-the-art performance on the challenging PhysioNet 2019 dataset. This work not only provides a robust reassessment of ModernTCN but also serves as an audit of the commonly used general time series analysis experimental setup, which includes tasks such as forecasting, imputation, classification, and anomaly detection.

Published in Transactions on Machine Learning Research (06/2025)
ModernTCN Revisited: A Critical Look at the Experimental Setup in General Time Series Analysis
Önder Akaçık Vrije Universiteit Amsterdam
o. o.ekacik@vu.tt
Mark Hoogendoorn Vrije Universiteit Amsterdam
m.hoogendoom @vr.oo
Reviewed on OpenReview: https: openreviee.. net/ forum? id= R20kkdWmVZ
Abstract
While numerous time series models claim state-of-tho-art performance, their evaluation often relies on flawed experimental setups, leading to questionable conclusions. This study provides a critical re-evaluation of this landscape, using ModernICN as a case study, We conduct a rigorous and extended benchmark, correcting methodological issues related to data loading, validation, and evaluation methods, and show that performance claims to these details. Additionally, convolutional models, and our comparison reveals that despite claims of an enlarged effective receptive field (ERF), it falls short of these methods. More than a critique, we introduce an architectural innovation: by embedding irregularly sampled data with a continuous kernel convolution and processing it with the ModernTCN backbone, we achieve new state-of-the- art performance on the challenging PhysioNet 2019 dataset. This work not only provides robust reasscssment of ModernTCN but also serves as an audit of the commonly used general time series analysis experimental setup, which includes tasks such forecasting, imputation, classification, and anomaly detection
sensitive find that Modern TCN overlooks line of research in global
1 Introduction
Time series analysis is a fundamental problem with broad applications across various domains, including weather forecasting (Bi et al. 2023). anomaly detection in spacecraft monitoring (Su et al. 2019b). medical symptom classification (Kiyasseh et al. 2021 and missing data imputation (Luo et al. 2018).
ModernTCN is a general time series analysis model designed enhance performance across five key time series lasks: long-term forecasting, short-term forecasting, classification, imputation, and anomaly detection. It modernizes traditional Temporal Convolutional Networks (TCNs) Bai al. (2018) by enlarging the effective receptive field (ERF), drawing inspiration from computer vision advances Liu et al. 2022a Ding 2022a] and Transformer architectures (Vaswani et al. 2017). The model also incorporates techniques transformer-bosed time series models, such as patching and variable independence approaches Nie
ct al.
from
et al.
2023
The concept of a general time series analysis model, seen in ModernTCN, builds on the experimental setup introduced by TimesNet (Wu et al. 2023). However, this setup has several flaws. For example, the setup often accolerates testing by splitting data into batches and discarding the last incomplete batch of the test set, which can lead to unfair comparisons and misleading results (Qiu et al., 2024). In anomaly detection, this setup employs an evaluation method prior to scoring, which can lead to an overestimation of performance, as demonstrated by Kim et al.] (2022). Additionally, for short-term forecasting and classification, the test set is used for validation, leading to overly optimistic results and an unfair comparison with earlier baselines
The official repository this work available httpp:://gttuub..cm//aeeeesssss htt.:///...cccoooeaaaaaaiii////a ModernTCN

---

Published in Transactions on Machine Learning Research (06/2025)
that adhere to stricter evaluation protocols. Alongside ModemnTN,, many papers, such as GPT4TS Zhou
et al. 2023), UniTS (Gao et al. 2024), and TimeMixer++ (Wang et al. 2025 follow this general time series analysis setup.
The primary purpose of this study is to validate this setup using ModernTCN as a means. Furthermore, we extend the benchmarks and experiments to rectify the limitations of the original setup, thereby improving the robustness. Additionally, we address the oversight of global convolutional- based models (Romero et al, 2021b; Gu et al., 2021; Romero et al., 2021a Knigge et al.] 2023) in the original study. By harnessing the synergy between global convolutional method (Romero et al. 2021b and ModernICN, we enhance the model performance on irregularly sampled data.
The contributions of this study are:
Auditing the general time series analysis experimental setup using ModernTCN case study, identifying and correcting critical flaws in data handling and evaluation protocols.
Innovating ModernICN by integrating it with global convolutional method, resulting in a superior model for irregularly sampled data and achieving statc-of-tho-ae performance on the PhysioNet 2019 dataset (Reyna et al. 2019
Validating Modern TCNNss claims of an extended effective receptive field by visualizing and comparing its ERF against global convolutional methods.
2
Related Work
General time series models like ModernTCN, TimesNet and UniTS aim to provide comprehensive solutions across various time series tasks. TimesNet introduces a unified framework for time series analysis by trans- forming ID time series into 2D tensors, allowing for the modeling of complex temporal variations using 2D kernels (Wu et al. 2023). UniTS employs task tokenization to integrate predictive and generative tasks into a single framework, using modified transformer block to caplure universal time series representations. enabling transferability across diverse datasets and tasks (Gao et al. 2024p. All these models utilize an experimental setup that originates from TimesNet, similar to ModernTCN.
Recent studies are addressing the faulty setups in long-term forecasting and anomaly detection, which are part of the general time series analysis setup. The 'Drop Last Trick,' as highlighted by Qîu et al. (2024). involves discarding the last batch if it contains fewer instances than the batch size, which can lead to misleading results. Kim et al. (022) have shown the misleading nature of point adjustment in anomaly detection. Additionally, Liu & Paparrizos (2024) proposes comprehensive benchmark addressing the known but often ignored issucs in the domain of time series anomaly detection.
Transformers have shown strong performance in time series forecasting, as demonstrated by models like PStchTST (Nie et al.) 2023). However, MLP-based models have gained popularity by questioning the ne- cessity of transformers, offering similar performance with greater efficiency, as seen in DLincar (Zeng et al.
2023
Convolution-basel models, on the other hand, strike balance by being more efficient. than trans-
formers with subquadratic time complexity and more expressive than MLPs Luo & Wang
2024
Wang
et al. 2022
In addition to TimesNet and ModernTCN, convolutional models like TCN (Bai et al. 2018 and MTCN
(Wang al., 2 offer distinct approaches to time scries analysis. TCN, as described by Bai al. (2018).
employs causal convolutions lo ensure no information leakage from future to past, allowing it to handle sequences of any length and map them to output sequences of the same length. This architecture is designed to combine simplicity with autoregressive prediction and long memory, making it effective for sequence modeling, On the other hand, MICN goes beyond causal convolution by proposing a multi-scale convolution structure that combines local features and global correlations in time series, enhancing its ability to capture complex temporal patterns.
In addition the tpsk-sp models, significant recent development is the emergence of Time Series Foundation Models (TSFMs), which pre-trained on massive data to achieve zero-shot. forecasting capa-
2

---

Published in Transactions on Machine Learning Research (06/2025)
bilitics (Gao et al. 2024 Das al. 2024 Ansari et al. 2024 Woo et al 2024 Liu et al. 2024 Auer et al,
2025). These models offer enhanced generalizability and simplified application, often leveraging in-context learning from their extensive pre-training.
Lastly, MCN is inspired by computer vision techniques to enlarge the ERF, but overlooks a line of research involving models with global convolution capabilities, such as CKConv Romero et al. (2021b), FlexConv Romero al. (2021a), CCNN Knigge et al. (2023), and S4 Gu et al. (2021). These models achieve substantially larger ERFs through continuous kernel formulations, enabling arbitrarily large memory horizons while maintaining efficiency.
3 Methodology
3.1
ModernTCN
ConvFFNI Groups:M
ConvFFN2 Groups:D
input
DWConv
output
Figure Diagram of the ModernTCN block. Adapted from Luo & Wang 2024
2024) introduces convolutional approach with large kernel sizes inspired by advances in computer vision [Liu al. 2022c Ding et al. 2022b Liu al. 2022b).. It leverages modern convolution techniques, incorporating depthwise and pointwise convolution layers organized similarly to Transformer blocks (Vaswani et al. 2017 The depthwise convolution (DWConv) layer is responsible for learning temporal information among tokens on per-channel basis, akin to the self-attention module in Transformers. A large kernel used in DWConv to enhance the effective receptive field, allowing the model to capture long-term dependencies more effectively. The ConvFFN module consists of two pointwise convolution layers that adopt an inverted bottleneck structure. This module applied twice in the architecture, first along feature dimensions and then between variables, to learn new feature representations.
ModerTTN (Luo & Wang
The pipeline of ModeruTCN is as follows:
1. Input: The process begins with the input time series Xin E RML, where M is the mumber of variables and L is the input length.
Patchify Embeddings: The input transformed into higher-dimensional space by patching a variabll-independent manner, resulting in Xemb E MxDxN where D is the feature dimension and N is the number of patches.
3. Backbone: As shown in Figure each Modern block consists of:
DWConv Layer: Processes temporal information independently for cach variable and cach feature, maintaining dimensions M x D.
ConvFFN Module: This module consists of two pointwise convolution layers and adopts an inverted bottleneck structure. It applied twice:
ConvFFN1: Processes each variable independently with -roups-M,, learning new feature representations independently for each variable. Until this point, the entire pipeline main- tains variable independence.
ConvFFN2: Processes across variables with groups=D, capturing dependencies bet ween different variables. This is the first component in the pipeline that allows cross-variable information mixing.
These blocks are stacked and organized in residual manner as shown in Figure
4. Flatten and Prediction Head:
3

---

Published in Transactions on Machine Learning Research (06/2025)
The output is reshaped and passed through prediction head to produce the final time series output X E RRMX where T is the prediction length.
Forecasting:
Imputation and Anomaly Detection: The prediction head produces output ee RMX.
Classification: The representation is flattened to and passed through a projection layer with a SoftMax activation to yield the classification result X E RIxCI, where Cls is the number of classes.
5. RevIN: For all tasks except classification, Stationary Technique RevIN Kim et al. 2021] is applied. It normalizes the imput time series per variable with zero mean and unit standard deviation before patching and embedding. After the forward process, the mean and deviation are added back to the final prediction per variable.
This architecture loverages the decoupling of temporal, feature, and variable dimensions to improve both performance and efficiency in time series analysis, while the enlarged kernel size in DWCony enhances its ability to capture long-term dependencics.
3.2 Tasks and Datasets
Dataset descriptions. The dataset size is organized in (train, validation, test). The column length refers to prediction length for forecasting lasks, input length for imputation, series length for classification, and sliding window length for anomaly detection. Partially adapted from Wu et al. 2023
Table 1:
Thsslo
Dataset
Length
Fintaeet Sien
Information (Frequency)
Reectiiity (15 mine)
ETTE:, ETTH
[15.
J51,, 7200
(654b, 2003, 2061)
Electriitty (bourly)
Longster
Fleetricity
[16 192,
336, 7003
[19917, 2033, 5061)
Blectriity (hourly]
lineswcingg
Traffie
[16.
189,
336, F70)
(12185, 1757, 3564)
Traneporbatin (hourly;
Wanlber
21
[96
192,
38G, 7200
(91792, 1271, 10640)
Wanthn (T mital
Eachinge
[M5.
P90)
(G120,
(dally
Yoarly
(230000 2090000
Lemographic
Short-wera
Quiiteely
(91000, 940000
Pisance
forsracting
Indietry
original
MisWislly
1860)
Maueu
Mi-Dally
14
(4997, 0, (937)
Miem
XIG Hourly
43
(414, Uy 414)
Othes
Retemdeed
B77h
ET'mi
24:
(54685, 13525, 015223
18546, 2881)
ESectrictt (15 mine)
(lourls)
ETTmi, ETTs2
06
(84666, 11:31, 115231
Exctriclly (16 mine
Empetattion
ETTb. bTTT2
[6946, 2881, 2861)
Isectincty (15
Eluctaieity
Exctricity mite)
(36792, 10540)
mine)
Ri Milamnnlznreeeeeeiin
1781
(861,
Alroboi Indosty
J5001)
(250lle)
Hunderiling
102
Hanlwalling
Heartbest
405
(204, 0, 205)
Hear best
Clucedcation
ConncscVenccle
12
201
(2%) My 3T00
Voice
urigind
PEMS--S SF
968
144
(267. 0, L78)
Trutspctaiion (duil))
Salfiiegilatoon FF
1906
(269. 2003)
Hiaittcare (250l1s)
SelfeegilatinnSCP
1159
(800,
Heelthoaee (95ilHs)
(1102311)
DWNueSistur Labeary
(13%,
Goouus
Speech Commands MPCC
161
[46485, N246, 52401
Retmoww
Spowth Commandds ls Raw
16000
(201883, 5915, 5216)
Voliei 1c000liig
PlyesnNet
30(+5]
72
[J8225, 00/00 6036)
Heuithrane
Tinjotaris
[80-183)
(1200,
1436)
Handscttnn
SMD
100
(566624, 14:08t, 700 620)
Suryer muchine
detection
Auaudl
SMAP
MSL
1001
(J000000
(44855), 11664, 737299
Spucecraft
Spacscraft
1001
(100883, 20430, W7+63)
Seever machine
Eetenced
1SB-AD-U
ASB-AD-M
[2-48]]
[t curstod from. 17 didiorat datuads, Sox Apoendia detaile
is curated from 23 ditierent dattantts See Apoendin A.2 or detaila
Time scries analysis encompasses variety of tasks, each designed to extract different insights from sequential data. This study focuses on five primary tasks: long-terın forecasting, short-term forecasting, imputation, classification, and anomaly detection. An overview of all the datasets used is given in Table 0
Long-term forecasting involves predicting future values over an extended time horizon, often requiring the model to capture complex temporal dependencies. ModerTTNN (Luo & Wang 2024) uses datasets such ETT (Zhou et al. 2021), Electricity (UCT), Traffic (PeM), Weather (Wet), Exchange [Lai et al.] 2018).
4

---

Published in Transactions on Machine Learning Research (06/2025)
and ILI CDC covering real-world applications from different domains. In these multivariate forecasting tasks, the model predicts future values for all variables in the time series simultaneously.
Short-term forecasting focuses on predicting values over a shorter time frame, typically requiring the model to capture immediate trends and patterns. ModeraTCN uses the M4 dataset (Makridakis) 2018)) which contains univariate time series with different frequencies from different domains. We extend the experiments to ETT (Zhou al., 2021 by adapting it for short term multivariate forecasting.
Imputation aims to fill in missing values in time series, which is crucial for handling incomplete data and ensuring data quality. Modern (Luo & Wang 2024) uses ETT (Zhou et al. 2021 Electricity (UCT). and Weather (Wet) datasets for this task.
Classification is sequence-level that verifies the model's capacity in high-level representation learning. ModernTNN (Luo & Wang] 2024g uses 10 multivariate datasets from the UEA Time Series Classification Archive (Bagnall et al.) 2018). We extend to Speoch Commands (Warden 2018 and PhysioNet (Reyna et al. 2019), following the experimental setup from CKConv (Romero al., 2021bp. Speech Commands enables us to compare ModernICN with other convolution- based models that has global receptive fields. PhysioNet's irregular sampling
and high proportion of missing values present a challenging classification task. Additionally, we extend the evaluation to variable length time series classification using Character Trajectories dataset (Bagnall et al. 2018).
Anomaly detection seeks to identify unusual patterns or outliers in a time series, which essential for detecting abnormal events and potential issues. ModernTCN (Luo & Wang 2024 evaluates anomaly detection performance on SMD (Su et al. 2019a), MSL (Hundman et al. 2018), SMAP (Hndmman et al. 2018), SWaT (Mathur & Tippenhauer 2016)) and PSM (Abdulaal et al., 2021), covering service monitoring, space & earth exploration, and water treatment applications. We extend the evaluation to TSB-AD (Liu & Paparrizos 2024), comprehensive benchmark for anomaly detection consisting of 1070 high-quality time scries from a diverse collection of 40 datasets (sce Table 19 for details).
3.3 Experimental Setup
For the reproduction of the experiments from ModernITN, the optimal settings specified in the paper (Loo & Wang 2024 and the source code are used. For the extended experiments we conducted grid search over the relevant hyperparameters for each setup (See Appendix A.1 for details). For the best-performing hyperparamcters, we reported the mcan and standard deviation over 5 runs.
Upon investigating the source code, we identified issues affecting the five tasks used in Luo & Wang (0244) We fixed the errors and rerun the experiment for long-ters termterm forecasting and imputation tasks. For short-term forccasting, classification and anomaly detection, we extended the experiments to new datasots instcad.
3.3.1 Issues in the Original Setup
Drop Last Trick. As pointed out by Qiu al. (2024 many implementations of existing methods often employ "Drop Last Trick' during the testing phase NNee et al. 2022 Wang et al.) 2022 Zhou et al., 2022 2021). ), which involves discarding the last batch if it contains fewer instances than the batch sizc. This approach may lead to unfair comparisons by excluding part of the test data. Upon reviewing the source code, we found this trick is employed for long-term forecasting and imputation tasks. Given that most datasets in the long-term forecasting experiments use a batch size of 512, this could result in discarding a significant amount of the test set (See Appendix 25 for further analysis). Therefore, we additionally experiment without employing this trick to assess its impact on the results and ensure fair comparison (Table 2 and Table
Data Leakage from Test to Validation. As detailed in Table I the M4 dataset (Makridakis 2018) for short-term forecasting and the UEA dataset (Bagnall et al. 2018 for classification are provided with only train and test splits. Upon reviewing the source code, we identify that the test set is used for validation. This practice introduces an information leakage as the model gains indirect exposure to the test data, leading to
Thh official source code for ModeroTCN can found http:://sttuul.cc//auhhhhhaeeeet
5

---

Published in Transactions on Machine Learning Research (06/2025)
overestimation of its generalization capabilities. Consequently, the reproduction results for these datascts are excluded from the main results section and are instead presented in Appendix F.1 and Appendix E.2
Point Adjustment in Anomaly Detection. Reconstruucion-baseed anomaly detection identifies anomalies when reconstruction errors between input sequences and their model-generated reconstructions exceed a threshold. The threshold is calculated by taking a specific percentile of the reconstruction error scores from the training data, ensuring that defined percentage of the data is considered normal. But ModernITN calculates the threshold using both training and test sets, introducing information leakage. We reproduce the experiments using thresholds calculated from training data only. More importantly, the evaluation employs 'point adjustment' (PA), which considers an entire anomaly segment correctly detected if just one point within it is flagged a protocol that Kim et al. (2022) demonstrated can severely overestimate performance. To illustrate this overestimation, we implemented a naive baseline that periodically flags every 100th time point as anomalous, which should perform well under PA despite having no actual anomaly detection capability. 100 is set as the period in the naive approach because the sliding window size is 100.
3.3.2 Extended Setups
Extending Short-term Forecasting to ETT. Inspired by the appendices in Luo & Wang (2024), we evaluate short-term multi-variate lorecasting on the ETT dataset (Zhou et al. 2021J. While MCN (Luo & WWang] 2024) keeps the input sequence length constant at 2x the prediction length, we explored the input sequence length as a hyperparameter, experimenting with 2x, 3x, and 4x the prediction length. We consider three prediction lengths: 6, 12, and 18 time points, selecting the best-performing imput length for reporting. We did hyperparameter search for the kernel size and employed the optimal hyperparameters from the corresponding long-term forecasting experiments for the rest. For comparison, we decided to include three top-performing models of different kinds from QQu et al. (024): TemesNet connolution-based (Wu et al. 2023 PatchTST (transformer- baass (Nie et al. 2023). and DLinear (MLP-based) (Zeng et al. 2023
Extending Classification to Speech Commands. For the classification task, we employ the Specch Commands dataset. (Warden) 2018). This dataset has two versions: Speech Commands Raw and Speech Commands MFCC. The Speech Commands Raw version consists of 105,809 one-second audio recordings of 35 spoken words sampled at 16kHz. Following the preprocessing steps of Kidger et al. (2020), we extract 34,975 recordings from ten spoken words to construct a balanced classification problem. The Speech Commands MFCC utilizes mel-frequency cepstrum coefficients extracted from the raw data, resulting in time series of length 161 and 20 channels. This dataset is frequently used by convolution- based models with global roceptive fields (Romero et al. 2021b a. Knigge ct al., 2023 Gu ct al. 2021).
Extending to Variable Length Time Series Classification. The CharacterTrajectoie dataset is part of the UEA time series classification archive (Bagnall et al. (018). It consists of 2,858 time series of varying lengths, cach with 3 channels representing the x, positions and the pen tip force while writing a Latin alphabet. character in a single stroke. The goal is to classify which of the 20 different characters was written using the time series data. The length of the time series varies between 60 and 182. Unlike Romero et al. (2021b), we retained the original test set and created a validation split by taking 15% of the training set. This approach ensures comparability with Lee & Shin (2025). Additionally, we reran the CKConv (Romero et al. 2021b) experiments with this new setup using its official codebase
Extending to Irregularly Sampled Data. The PhysioNet 2019 challenge on sepsis prediction provides an irregularly sampled, partially observed dataset with 40,335 time series of variable lengths, capturing ICU patient data (Reyna et al. 2019). Each series includes 5 static features, like age, and 34 dynamic features, such as respiration rate, with only 10.3% of values observed. Following Kidger et al.] (2020) : Romero et al. (2021b), we analyze the first 72 hours of patient data to predict sepsis development. over their entire slay, which can last up to a month. This setup tests the model's ability to handle sparse and irregular data, creating robust test bed for ModernTCN, which claims statc-of-the-art performance task. Similar to ModernTN,, we ablate the vross-variable component to assess its impact on handling missing values in a multivariate setting. Most importantly, we introduce an innovation in our reproducibility
the imputation
The official CKConv codebase can be found at
6

---

Published in Transactions on Machine Learning Research (06/2025)
study by combining global convolutional method Roomero et al. 2021bb with ModernTCN, detailed in Table 10
Extending Anomaly Detection to TSB-AD. Extending our anomaly detection experiments to the TSB-AD datasets and benchmark (Liu & Paparrizos, 2024) is beneficial due to its comprehensive nature, encompassing over 40 datasets and wide array of baselines. The benchmark includes 40 time-series anomaly detection algorithms categorized into statistical, neural network- beasor,,aasd,,,,, and foundation model-based methods. This extension addresses critical issues in the field, such as flawed datasets and biased evaluation measures, by employing VUS-PR. (Volume Under the Surface of Frecision-Recall VUS-PR is threshold-indeeendemi summarizing performance across all possible anomaly score thresholds, thus providing a holistic and stable evaluation. It is more robust to temporal misalignments and less biased towards random or noisy scores compared to traditional meastres like PA-F1 and AUC-ROC. By incorporating a tolerance buffer around anomaly boundaries, VUS-PR enhances the relevance of anomaly scoring, making it a superior choice for ovaluating time-scrics anomaly detection models.
more reliable evaluation metric.
4 Experimental Results
4.1 Results Reproducing the Original Paper
4.1.1
Long- Term Forecasting
The 'Drop Last Trick" significantly impacts datasets like ETTh and Exchange, where a substantial percentage of the test set is discarded, as in Table 2 This leads to notable changes MSE/MAF, particularly for these datasets, as a high percentage of data is dropped. Additionally, the 1LI dataset exhibits significant changes in MSE/MAE, likely due to its small size (Table D For further analysis, please refer to the Appendix D
All results in Table 3 are presented after the 'Drop Last Trick" is corrected. PatchTST Nie et al. 2023
TimesNet (Wu et al. 2023), and DLincar (Zeng et al. 2023) are the best-performing methods based on transformer, convolution, and MLP architectures, respectively, as identified by Qiu et. al. (2024). We include p and TCN (Bai et al.
2018 as extra convolution based baselines. Additionally, we include results for several Time Scries Foundation Models (TSFMs) in a zero-shot sotting
MICN (Wang et al.
(aao et al.
2024
Das et al. 2024 Ansari et al., 2024 Woo et al., 2024 Liu et al. 2024 Auer al.) 2025
The ranks in Table are calculated independently for task-specific models and foundational models. While PStchTST is the top-performing task-specific model, Modern remains highly competitive by frequently ranking second and outperforming other convolutional baselines. Notably, TiRex not only leads the founda- tional models but also rivals the best task-specific models with superior MAE in most cases and competitive MSE even though only the Electricity dataset from our benchmark was part of its pre-training data (Auer ct al. 2025
Table 2: Long-lerm forecasting reproduced results of ModeraTNN. Reported, Rerun and Fixed refer to original paper results, results before and after the 'Drop Last Trick" fixed respectively. Dropped % shows percentage of test data points excluded when using the trick. A MSE/MAE shows performance change when the trick is fixed and it is calculated as "A Fixed Rerun'. Results are averaged from 4 different prediction lengths: (24, 36, 48, 60) for II.I and {96, 192, 336, 720) for others.
ETTM
BTTh2
ETTml
STT=2
Electriity
Weather
TieTTo
Exclango
IE.1
Mesult Type
MSI
MAN
MAX
MAX
MBB
KAB
MBK
MAE
MAE
MSS
MA
Mis
MAI
Reported
1.410
0.001
0.585
6.555
0.355
0.310
0.139
0.23 2.221
0.307
0.254
0.305
6.365
0.797
Ebeed
0.415
0.429
0.347
0.394
6.586
0.388
0.254
0.318
0.383
0,200
0.225
0,307
0.430
0.280
0.343
0.392
2.000
0,812
Dropped
11.14
1..0%
13.0%%
11.05
4,99%
233%
2533
2335
6.20%
0,88%
166%
1.6%
0.005
0.403
14.4%%
14,0%%
5.4%
5.80%
-0033
10000
10.001
0000
40.00
10000
10.00
1000
0002
10.21
10.00
100001
4.1.2 Imputation
The results presented in Table demonstrate that due to the smaller batch size used in the imputation task, as referenced in Table 25 the percentage of dropped points from the test set is minimal. This minimal drop percentage explains why the results remained unchanged after the "Drop Last Trick" was corrected.
7

---

Published in Transactions on Machine Learning Research (06/2025)
Table 3: Long-term forecasting task results after the "Drop Last Trick" is fixed. All results averaged from 4 different prediction lengths: (24, 36, 48, 60) for ILI and {96, 192, 336, 720} for others. Lower MSE/MAE
indicates better performance. Bold indicates best performance, and underline indicates recond-best perfor- mance. Ranks are calculated independently for task-specific models and foundational models.
PTTM
ETT
Flecriiity
Weather
Truffie
Euchang
11.1
Model
1st/2nd
Meer:
VAAR
XISt:
XEAR
Mese
MAE
MAE
ViSR
STArt
MIsI
MAY
MIST:
MENE
SESP:
MAR
MIS
MAA
Moderaron
1713
0:290
0:09
0:3:3
0:395
0:305
0-397
0.254
0.315
6.254
0.29
D7307
0:413
0.313
0.392
2007
(C9299
PaCeTTo
1973
0:411
0:425
0.247
0.280
0.349
07351
52235
0.315
0.103
0:231
0.223
0:202
0-405
0-283
0.372
0.297
1.770
0.389
Dlansse
0:450
0.435
0:478
0.877
6:29
0.289
0.250
0.434
0:205
0:330
0:207
07:00
0:371
0.335
0.303
02258
0:357
01175
0:280
1.235
02500
00313
0.310
023900
23385
1070
TCN
070
1003
0.787
5.398
1000
E,350
0.340
2,435
1,37
0.353
0.445
0.407
0:422
1
0.305
1.757
1.015
4.330
M
0:381
01:37
0:011
0.000
0.310
03305
0.316
0.155
0.30
00:322
0.399
01301
0.00
0.410
130
UnitS
170
00638
0.491
0.408
0.418
0.715
0.383
6.82
0,585
0.422
0.658
0,201
0.333
0.728
0.382
0.424
0.645
4.000
1.450
MOFTAT
0.720
11:00
0.515
02307
0.300
0.305
0.00
0.200
0.305
00381
0.223
0.30
0.715
Tiner
0.00
0.30
00100
01000
0750
02222
0.205
0100
20200
0.000
0.300
0.00
00383
0.416
0:378
0.305
0.305
0.300
0.186
0.328
0.24Y
0.013
0.203
0.821
In our comparison of ModeruTCN with TimesNet, PatchTST, DLinear, and MICN, ModernTCN clearly outperforms all other methods (Table 5). TimesNet ranks second, which aligns with the fact that both ModernTCN and TimesNet are designed for general time series analysis, whereas PatchTST, DLincar, and MICN are specifically developed for lime series forecasting. Additionally, PatchTST and DLinear are variable-independent models, while this task is inherently multivariate. The ability to model cross- variable interactions may facilitate the imputation of missing values by leveraging information from existing variables at the same time point, thereby modeling inter-variate dependencics effectively.
Table 4: Imputation task reproduced results of ModernITN. Reported, Rerun and Fixed refer to original paper results, results before and after the 'Drop Last Trick' is fixed respectively. Dropped % shows percentage of test data points excluded when using the trick. A MSE/MAE shows performance change when the trick is fixed and it is calculated as 'A= = Fixed Rerun". Results are averaged from 4 different mask ratios 12.55% 25%, 37. 37.5%, 50%).
ET
ETTh2
ET
ETTm2
Electricity
Weather
Result Type
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
Reported
0.050
0.150
0.042
0.131
0.020
0.093
0.019
0.082
0.073
0.187
0.027
0.044
Rerun
0.050
0.150
0.042
0.131
0.021
0.094
0.020
0.083
0.073
0.186
0.027
0.044
Fixed
0.050
0.150
0.042
0.131
0.021
0.094
0.020
0.083
0.073
0.186
0.027
0.044
Dropped %
0.03%
0.03%
0.03%
0.03%
0.01%
0.01%
0.01%
0.01%
0.25%
0.25%
0.11%
0.11%
A MSE MAE
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
Table 5: Imputation task results after the 'Drop Last Trick' is fixed. We randomly mask {12.5% 25%, 37.5%, 50%} time points in length-96 time series. The results are averaged from 4 different mask ratios. A lower MSE or MAE indicates better performance. Bold indicates best performance, and underline indicates socond-best performance.
ETThl
ETTh2
ETTml
ETTm2
Electricity
Weather
Model
MSE
MAE
MSE
MAF
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
Modern
0.050
0.150
0.042
0.131
0.021
0.094
0.020
0,083
0.073
0.186
0.027
0.044
TimesNet
0.089
0.199
0.051
0.149
0.027
0.107
0.022
0.090
0.094
0.211
0.031
0.056
PatchTST
0.1
0.238
0.066
0.164
0.045
0.133
0.028
0.098
0.091
0.209
0.033
0.057
DLinear
0.201
0.306
0.142
0.259
0.093
0.206
0.096
0.208
0.132
0.260
0.052
0.110
MICN
0.125
0.250
0.205
0.307
0.070
0.182
0.141
0.219
0.119
0.217
0.056
0.128
4.1.3 Anomaly Detection
As shown in Table [6 the rerun results generally align with those reported in the original paper. When climinating data leakage by calculating thresholds using only training data, we observe a slight perfor- mance degradation. Most notably, our naive baseline-white simply flags every 100th timepoint as anoma- lous out-oouurroomms ModernTCN across most datasets, achieving the highest average F1 score. This finding
8

---

Published in Transactions on Machine Learning Research (06/2025)
raises significant questions about the effectiveness of sophisticated models for anomaly detection when eval- uated using point adjustment protocols, supporting the critique by Kim et al. (2022) that such evaluation methods can severely overestimate model performance.
Table 6: Anomaly Detection lask. P: precision, R: recall, Fl: F1-score (all values in %). Reported: scores from Luo & Wang 2024). Rerun: naively reproduced results. No Leakage: results calculating the thresholds using only training data. Naive: baseline approach. Bold: best score between valid models (No Leakage and Naive). Avg.: average F1 score across all 5 datasets.
Model
SMD
MSL
SMAP
SWTT
PSM
Avg-
Reported
81.86
83.85
85.81
K3.94
85.33 74.94
84.32
03:17
57.70
91.83
905.08
48.09
100.3
W7:23
8h:022
Rerun
87.43
84.44
89.50
81.61
90.82
55.93
95.77
50.28
92.94
94.57
84.96
Lealuage
79.42
93.76
81.00
92.31
93.16
98.62
94.58
96.56
Naien
79.86
91.40
85.24
90.14
80.78
85.19
93.27
94.44
98.87
96.93
94.95
97.32
94.55
95.91
4.2 Results Beyond the Original Paper
4.2.1
Short-Tern Forecasting ETT
Referring to the results presented in Table 7 ModernTCN exhibits competitive performance, particularly in the ETTml dataset, where it achieves the best results for very short-term forecasting. This aligns with findings from Yang et al. (2024), which demonstrated ModernTCN's effectiveness for ultra-short-terns multi- step prediction tasks. However, in ETThl dataset, PatchTST demonstrates superior performance.
Table 7: Short-Term Forecasting Results ETT (ETThl and ETTm1). 6x1h, 12xlh, and 18x1h refer to steps ahead forecasting for ETTh1, while 6x15m, 12x15m, and 18x15m refer to steps ahead forecasting for ETTml. Results are averaged over 5 runs, and both mean and standard deviation are reported.
de10m
12citm
14eifon
0xth
12clh
1dcth
Moded
MAA
VSR
WAR
XBC
MAY
MISK
MFT
MBF
WAR
VAA
B.12Y - 1100
0:201
0101
0.23
1181
8100
0.00
0:39
BIGT
8138 680
0.339
0.61
0.00
0101
8.36 8.05
0.29 - 0197
B512 6.60
0.251
1.00
0.321
0.35% à 0.66
0.353
0.02
0.248
0.301
1 0.00
0.21:
+000
0.51
6.00
KOXS
022-0000
8.50=6000
2.N 2005
0.20
= 0000
A.T 6..00
0,841
2004
2.16
0004
KJR6 0:00
0.313 10000
0.50
6.00
6XKI
0.001
0.34Y 6.000
0,24 2000
0.33 = 0100
6.00
0.31 2 2002
0.00 0.00
2000
4.2.2 Classification on Speech Commands
ModernTCN exhibits limited parameter efficiency, particularly when handling many input variables. Its design maintains a separate feature space for cach variable throughout the network, which inflates the model size as the number of variables increases. For instance, in the Speech Commands MFCC dataset with 20 variables and a feature dimension of 32, this results in over million parameters. Tn contrast, CCNN (Knigge) et al. 2023) projects all variables into a shared model space at the input stage, resulting in significantly greater parameter efficiency.
This inefficiency becomes more apparent when comparing across datasets. On the Speech Commands Raw dataset, which contains only single variable, ModernITN's selected model size of 512 results in a much smaller model with approximately 500K parameters. Despite the challenge posed by the long input sequence length (16,000), ModernTCN performs better than CKConv this setting. This is primarily due two factors: the aggressive downsampling of the raw signal during embedding and the use of large model dimen- sion (512), which enhances the model's expressiveness. However, even in this favorable setting, Modern still underperforms compared to more advanced global convolutional baselines.
We further analyze ModernTCN's ability to model long-range dependencies through effective receptive field (ERF) visualizations, following the methodology of Kim et al. (2023) and Ding et al. (2022a), as adapted by Luo & Wang (2024). We sample 50 sequences from the Speech Commands MFCC validation set and compute the relative gradient contributions of each input timepoint to the center of the feature map after fixed number of blocks. The resulting visualizations are shown in Figure where lighter regions indicate stronger influence.
9

---

Published in Transactions on Machine Learning Research (06/2025)
Global models like CCNN(4,140) achieve a much larger ERF with significantly fewer parameters (200K vs. 4M) due to their global kernel design. This enables them to better capture long-range dependencies and contributes to their superior classification performance, as reported in Table Additional implementation details and extended analysis for ERFs are available in Appendix
(a) CCNN after 1 block
(b) CCNN after 3 blocks
t.
(c) Modern''CNN (kernel size 13) after block
(d) Modern'T'N (kernel size 13) after blocks
t.
(e) Modern'I'NN (kernel size 71) after 1 block
(f) ModernIIN (kernel size 71) after 3 blocks
Comparison of effective receptive fields (ERF) for different models. Each row shows different model configuration, while columns show the ERF after block (left) and 3 blocks (right)..
Figure 2:
Table 8: Classification Results Speech Commands Accuracy
Model
Size
Speech Commands MFCC Speech Commands Raw
CKCNN-Scg
Romero al.
0.953
84 [Gu et al.
2021
300K
0.940
0.983
Flex T
Romera et
2021a
375K
0.977
0.917
CCNN
Kmiggo
200K
0.950
0.983
CCNN (6,380)
Kiggee et AL
2023
2M
0.980
0.984
ModernICN
4M/300k
4.2.3 Variable Length Time Series Classification
Both ModernTCN and CKConv demonstrate strong performance in the variable length time series classifi- cation task (Table g outperforming the best baselines from Lee & Shin 20025). The high scores achieved by these models can attributed to the nature of the task. In this context, convolutional models like Mod- ernTCN and CKConv benefit from their ability to extract features through template matching as they slide over the time dimension. CKConv, in particular, excels in this task, likely due to the continuous nature of the handwriting trajectories. By representing the 1D convolutional kernel continuous function, CKConv effectively captures the underlying patterns in the data, leading to its exceptional performance.
Table Results of Variable Length Time Series Classífication (Accuracy and MAP) over runs, with values representing the mean and standard deviation. Modern and CKConv were adapted, while the other baselines selected from (Lee & Shin 2025) the best pariants/models for both metrics.
Model
Accuracy (%)
MAP (%)
Modern CKConv
96.02 0.9
94.42
99.07 0.3
99.00 + 0.3
Refined TIP
78.11 + 1.18
83.35 0.83
InceptionTime (IP)
82.81 0.40
09.28 0.45
4.2.4 Irregularly Sampled Time Series Classification
Sepsis prediction is a critical task, as early detection significantly improves patient outcomes. The PhysioNet Sepsis Prediction dataset (Reyna al., (2019) presents challenging test bed due to its irregularly sampled nature, which results in missing values. CKConv demonstrates strong performance on this task, surpassing models specifically designed for irregularly sampled data. (Kidger al. 2020
10

---

Published in Transactions on Machine Learning Research (06/2025)
The ability of ModernTCN to handle missing values is evidenced by its superior performance in the impu- tation tasks within this study. This capability extends to irregularly sampled data, shown in Table TO Additionally, ModernI'CN's proficiency in mixing information between variables through its cross-varie variallariable component (ConvFFN2) is crucial, as its removal leads to a performance drop. This is indicated as "Mod- ernTCN Variable Independent," where the model becomes variable-independent until the fully connected projection layer at the end.
Both Mo and CKConv exhibit strong performance data with missing values. To harness their synergistic potential, we combined them by employing CKConv to embed the raw signals within Mod- enT,CN, while retaining the Modern pipeline for subsoquent processing. This approach, denoted as 'CKConv + ModernTCN, outperforms all other baselines. The improvement is statistically significant in both comparisons against CKConv and ModernTCN alone (Wilcoxon signed-rank test, one-sided p = 0.031 for both). This combination leverages two key properties of CKConv: (1) the continuous nature of the kernel, which inherently handles irregularly sampled data, and (2) parameter efficiency, as the parameter size of the continuous kernel is decoupled from the input sequence length. This innovative combination could serve as promising starting point for future studies, particularly in testing on irregularly sampled datasets, including those with very long sequence lengths.
Table 10: Comparison of Model Performance on the PhysioNet Dataset (AUC + Std Dev)
Model
AUC = Std Dev
CKConv
0.883 + 0.011
TCN
0.895 0.007
ModernIICN Variable Indepeudent
0,875 1 0.004
CKConv + ModernTCN
0.904 = 0.004
4.2.5 Anomaly Detection
In the context of anomaly detection, ModernICN's performance varies significantly between univariate and multivariate time series datasets. On the univariate TSB-AD-U dataset (Figure 3(a)). ModernICN ranks among the lowest-performing methods, placing third to last. This result aligns with the findings of Liu & Paparrizos (2024), which suggest that statistical methods often provide more effective and robust solutions for univariate time-series anomaly detection. Conversely, on the multivariate TSB-AD-M dataset (Figure 3(b), ModernTNN emerges the best pes--performin method. This success can be attributed to the extensive engineering and design considerations within Modern TCN that specifically address the complexities of multivariate time series data.
5 Discussion
This study provides a critical re-evaluation of MCN (Luo & Wang 2024), using it as a case study to audit the general time series analysis experimental setup. Our rigorous and extended benchmark revealed that performance claims are sensitive to methodological details in data handling and evaluation. After correcting flaws like the "Drop Last Trick' in long-term forecasting, we found that ModernTTN, while competitive, is often outperformed by other leading models like PatchTST. Similarly, our anomaly detection experiments showed that its performance varies drastically; it lags behind simpler statistical methods on univariate data but excels in multivariate contexts. This highlights the importance of its architectural components, such as the cross-variable FFN2 module, which our ablation study confirmed crucial for handling missing data.
Furthermore, addressed MCdernTCN's oversight of a significant line of research in global convolutional models. Our analysis of effective receptive fields (ERFs) revealed that despite its claims of an enlarged ERF, it achieves a smaller receptive field compared to continuous kernel approaches like CCNN (Knigge et al. 2023 and at a much higher parameter cost. This finding motivated our main architectural innovation: integrating ModernTNN with a global continuous kernel for embedding irregularly sampled data. This hybrid model achieves new state-of-thh-art performance on the PhysioNet 2019 datasct. This innovative combination opens new avenues for future research, particularly in exploring its application to other irregularly sampled datasets and those with extended sequence lengths.
11

---

Published in Transactions on Machine Learning Research (06/2025)
1.0
WUS.PR
0.5
0.0
Sub-PCA
KShapeAD
POLY
Series2Graph
MOMENT (FT)
MOMENT (2S)
KMeansAD
USAD
1N-K
MatrixProfile
SAND
CNN
LSTMAD
SR
TimesFM
IForest
OmniAnomaly
Lag Llama
Chronos
TimesNet
AutoEncoder
TranAD
FITS
Sub-LOF
OFA
Sub-MCD
HBb-HBOS
Sub- Sub-OCSVM
Sub Obb-IForest
Donut
ModernTCN
LOF
tromalytransformer
(a) Accuracy evaluation on univariate TSB-AD-U.
VUS-PR
ModernTNN
CNN
OmniAnomaly
PCA
LSTMAD
USAD
AutoEncoder
KMeansAD
CBLOF
MCD
OCSVM
Donut
RobustPCA
FITS
OFA
EIF
IForest
HBOS
TimesNet
KNN
TranAD
LOF
Anomaly/ransformer
(b) Accuracy evaluation multivariate AD--AD-M.
Figure 3: Accuracy evaluation on univariate and multivariate time series anomaly detection. In the boxplot, the mean value is marked by a dashed line and the median by a solid line, Adapted from (Liu & Paparrizos 2024).
Overall, this work highlights the importance of carefully considering experimental design choices in time series analysis. Some widely used practices- such as the "Drop Last Trick" in forecasting, using the test set for validation in classification and short-term forecasting, and point adjustment in anomaly detection may unintentionally bias results. We encourage the community to carefully review these aspects and consider adopting benchmarking frameworks like the one we present here. Doing so can help ensure more robust, reliable, and comparable evaluations, ultimately fostering stronger progress in time series modeling.
5.1 Contact with the authors
The authors were contacted via email to request the code for ERF visualization and to discuss the flaws in the experimental setup. Unfortunately, no response was received.
Acknowledgments
All experiments presented in this paper were the Dutch national supercomputer Snellius. We thank David W. Romero for his insightful comment on the OpenReview page of ModernTCN, which highlighted the overlooked research line of long convolutional models there and further motivated this study.
12

---

Published in Transactions on Machine Learning Research (06/2025)
References
Illness.
lops. http://aps.a///ttaaeccccclllll/1 /ttp://ffaattt_deta1711/101
Traffic. http://pems.dot.ca. ca..dott.a..oo......
Tao. httpps//www.pmee..ooo
Electricity, http://///////taaaaa///////////a
Weather.
Ahmed Abdulaal, Zhuanghua Liu, and Tomer Lancewicki. Practical approach to asynchronous multivariate time series anomaly detection and localization. KDD, 2021.
C. C. Aggarwal. Oullier Analysis. Springer International Publishing, 2 edition, 2017.
S. Ahmad, A. Lavin, S. Purdy, and Z. Agha. Unsupervised real-time anomaly detection for streaming data. Newrocomputing, 262:134-147, 2017.
Abdul Fatir Ansari, Lorenzo Stella, Caner Turkmen, et al. Chronos: Learning the Language of Time Series. Transactions on Machine Learning Research, 2024.
J. Audibert, P. Michiardi, F. Guyard, S. Marti, and M. A. Zuluaga. Usad: Unsupervised anomaly detection on multivariate time series. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery dala mining, pp. 3395 3404, 2020.
Andreas Auer, Patrick Podest, Daniel Klotz, Sebastian Böck, Günter Klambauer, and Sepp Hochreiter. TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In- nn-Coneext Learning. ArXiu, 2505.23719 2025.
M. Bachlin, M. Plotnik, D. Roggen, I. Maidan, J. M. Hausdorff, N. Giladi, and G. Troster. Wearable assistant for parkinson's 'ss discase patients with the freezing of gait symptom. IEEE Transactions on Information Technology in Biomedicine, 14(2):436-446, 2009.
Anthony Bagnall, Jason Lines, Aaron Bostrom, James Large, and Eamonn Keogh. The uea multivariate time series classification archive. arXiv preprint arXim:1811.00073 2018.
Shaojic Bai, J. Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. CoRR, abs/1803.01271, 2018. URL http://arxiv.orrt .rg. /bb///
1803.00 01271
Kaiseng Bi, Lingxi Xie, Hengheng Zhang, Xin Chen, Xiaolao Gu, and Qi Tian. Accurate medium-range global weather forecasting with 3d neural networks. Nature, 619(7970):5333 538, 2023.
P. Boniol and T. Palpanas. Series2graph: graph-based subsequence anomaly detection for time series. Proceedings of the VLDB Endowment, 13(12):1821 1834, 2020.
P. Boniol, J. Paparrizos, T. Palpanas, and M. J. Franklin. Sand: streaming subsequence anomaly detection. Proceedings of the VLDB Endowment, 14110)11717-1729, 2021.
M. M. Breunig, H.-P. Kriegel, R. T. Ng, and J. Sander. LOF: identifying density-basse local outliers. In ACM SIGMOD Record, volume 29, pp. 93-104, May 2000.
Abhimanyu Das, Weihao Kong, Rajat Sen, et al. A decoder-only foundation model for time-series forecasting. In ICML, 2024.
Xiaohan Ding, Xiangyu Zhang, Jungong Han, and Guiguang Ding. Scaling up your kernels to 31x31: Revisiting large kernel design in enns. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pallern Recognition, pp. 11963 11975, 2022a.
13

---

Published in Transactions on Machine Learning Research (06/2025)
Xiaohan Ding, Xiangyu Zhang, Jungong Han, and Guiguang Ding. Scaling up your kernels to 31x31: Revisiting large kernel design in cnns. In Proceedings of the IEER/CVF Conference on Compuler Vision and Pattern Recognition, pp. 11963 11975, 2022b.
P. Filonov, A. Lavrentyev, and A. Vorontsov. Multivariate industrial time series with cyber-attack simulation: Fault detection using an lstm-based predictive data model. arXiv preprint arXiv:162..00 2.6666666 2016.
P. Fleith. Controlled anomalies time series (cats) dataset, Sep 2023. Accessed: 2023-09-01.
Shanghua Gao, Teddy Koker, Owen Queen, Thomas Hartvigsen, Theodoros Tsiligkaridis, and Marinka Zitnik. Units: Building unified time series model. Aduances Newral Information Processing Systems, 2024.
L. Goldberger, L. A. Amaral, L. Glass, J. M. Hausdorff, P. C. Ivanov, R. G. Mark, J. E. Mietus, G. B. Moody, C.-K. Peng, and E. Stanley. Physiobank, physiotoolkit, and physionet: components of new research resource for complex physiologic signals. circulation, 101 101 (23):e215 e220, 2000.
M. Goldstein and A. Dengel. Histogram-based outlier score (hbos): A fast unsupervised anomaly detection algorithm. In KT-2012: poster and demo brack, volume 1, pp. 59 59-63, 2012.
M. Goswami, K. Szafer, A. Choudhry, Y. Cai, S. Li, and A. Dubrawski. Moment: A family of open lime-series foundation models. In International Conference on Machine Learning, 2024.
S. D. Greenwald. Improved detection and classification of arrhythmias in noise-corrupled electrocardiograns using contextual information. 1990. Thesis, Massachusetts Institute of Technology, Accepted: 2005-10- 07T20:45:222.
Albert Gu, Karan Goel, and Christopher Ré. Efficiently modeling long sequences with structured state spaces, Xix preprint (Xx::2111000000 2021.
S. Hariri, M. C. Kind, and R. J. Brunner. Extended isolation forest. IEEE transactions on knowledge and dala engineering, 34)::14479 1489, 2019.
Z. He, X. Xu, and S. Deng. Discovering cluster-based local outliers. Pattern recognition letters, 24(9-10): 1641-1650, 2003.
Kyle Hundman, Valentino Constantinou, Christopher Laporte, Ian Colwell, and Tom Soderström. Detecting spacecraft anomalies using lstms and nonparametric dynamic thresholding. KDD, 2018
V. Jacob, F. Song, A. Stiegler, B. Rad, Y. Diao, and N. Tatbul. Exathlon: A benchmark for explainable anomaly detection over time series. Proceedings of the VLDB Endowmenl, 14(11):26) 2626, 2021.
E. Keogh, J. Lin, S.-H. Lee, and H. V. Herle. Finding the most unusual time series subsequence: algorithms and applications. Knowledge and Information Systems, 11:1-22 2007.
Patrick Kidger, James Morrill, James Foster, and Terry Lyons. Neural controlled differential equations for irregular time series. arXiv preprint arXix:2005.08920 2020.
Bum Jun Kim, Hyeyeon Choi, Hyeonah Jang, Dong Gu Lee, Wonseok Jeong, and Sang Woo Kim. Dead pixel test using effective receptive field. Pattern Recognition. Letters, 167:149-1566 2023.
Siwon Kim, Kukjin Choi, Hyun-Soo Choi, Byunghan Lee, and Sungroh Yoon. Towards rigorous evaluation of time-series anomaly detection. In Proceedings of the 36th AAAT Conference Artificial Intelligence (AAAI), pp. 7194 7201, 2022.
Taesung Kim, Jinhee Kim, Yunwon Tae, Cheonbok Park, Jang-Ho Choi, and Jaegul Choo. Reversible instance normalization for accurate time-series forecasting against distribution shift. In International Conference Learning Representations, 2021.
Dani Kiyassch, Tingting Zhu, and David A. Clifton. Clocs: Contrastive learning of cardiac signals across space, time, and patients. In International Conference Machine Learning, pp. 5606 5615. PMLR, 2021.
14

---

Published in Transactions on Machine Learning Research (06/2025)
D. M. Knigge, D. W. Romero, A. Gu, E. Gavves, E. J. Bekkers, J. M. Tomezak, and J. J. Sonke. Mod- elling long range dependencies in nd: From task-specific to a general purpose enn. or preprinl arXiv:231.11540, 2023.
Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long-and short-term temporal patterns with deep neural networks. In SIGIR, 2018.
K.-H. Lai, D. Zha, J. Xu, Y. Zhao, G. Wang, and X. Hu. Revisiting time series outlier detection: Defini- tions and benchmarks. In Thirby-fitth conference on newral information processing systems datasets and benchmarks track (round 1), 2021.
N. Laptev, S. Amizadeh, and Y. Billawala. S5 a labeled anomaly detection dataset, version 1.0(16m), Mar 2015.
Hyeonsu Lee and Dongmin Shin. Beyond information distortion: Imaging variable-length time series data for classification. Sensors, 25(3):621, 2025, doi: 10.3390 s25030621. URL https:///ol: http:///oooorr//.../// htgg/10.3900 s25030621 This article belongs to the Topic AI and Data-Driver Advancements Industry 4.0.
Z. Li, H. Ma, and Y. Mei. A unifying method for outlier and change detection from data streams based on local polynomial fitting. In Z.-H. Zhou, H. Li, and Q. Yang (eds.), Advances in Knowledge Discovery and Data Mining, Lecture Notes in Computer Science, pp. 150-164 Berlin, Heidelberg, 2007. Springer.
Z. Li, Y. Zhao, N. Botta, C. Ionescu, and X. Hu. Copod: copula-based outlier detection. In 2020 IEEE international conference on dola mining (ICDM), pp. 1118 1123, 2020.
Zhe Li, Xiangfei Qiu, Peng Chen, Yihang Wang, Hanyin Cheng, Yang Shu, Jilin Hu, Chenjuan Guo, Aoying Zhou, Christian S. Jensen, and Bin Yang. Tsfm-bench: A comprehensive and unified benchmarking of foundation models for time series forecasting. pp. 17, August 2025. doi: 00.144//7111896337777444 URL Attps://dois 10. 11tp///0..11///1186...ooo
F. T. Liu, K. M. Ting, and Z.-H. Zhou. Isolation forest. In 2008 eighth ieee international conference data mining, pp. 413 422, 2008.
Qinghua Liu and John Paparrizos. The elephant in the room: Towards reliable time-series anomaly detection benchmark. In NeurIPS 2024 D&B, The Ohio State University, 2024.
Shiwei Liu, Tianlong Chen, Xiaohan Chen, Xuxi Chen, Qiao Xiao, Boqian Wu, Mykola Pechenizkiy, Decebal Mocanu, and Zhangyang Wang. More convnets in the 2020s: Scaling up kernels beyond 51x51 using sparsity. Xiviv preprint 03X66:2007.03620, 2022a.
Shiwei Liu, Tianlong Chen, Xiaohan Chen, Xuxi Chen, Qiao Xiao, Boqian Wu, Mykola Pechenizkiy, Decebal Mocanu, and Zhangyang Wang. More convnets in the 2020s: Scaling up kernels beyond 51x51 using sparsity. arXiv preprint arXfc:2207.08620, 2022b.
Yong Liu, Haoran Zhang, Chenyu Li, et al. Timer: Transformers for Time Series Analysis at Scale. In ICLR, 2024.
Zhuang Liu, Hanzi Mao, Chao- Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11976-11986, 2022c.
Donghao Luo and Xue Wang. Modernten: A modern pure convolution structure for general time series analysis. International Conference on Learning Representations (ICLR), 2024.
Youghong Luo, Xiangrui Cai, Ying Zhang, Jun Xu, and Yuan Xiaojie. Multivariate time series imputa- tion with generative adversarial networks. In Advances in Neural Information Processing Systems 31 (NeurIPS), 2018.
Spyros Makridakis. Dataset: 2018.
M4 dataset.
15

---

Published in Transactions on Machine Learning Research (06/2025)
P. Malhotra, L. Vig, G. Shroff, P. Agarwal, et al. Long short term memory networks for anomaly detection in time series. In Esann, volume 2015, pp. 89, 2015.
Aditya Mathur and Nils Ole Tippenhauer. Swat: water treatment testbed for research and training on ics security. In CySWATER, 2016.
S. Moritz, F. Rehbach, S. Chandrasekaran, M. Rebolledo, and T. Bartz-Beielstein. Gecco industrial challenge 2018 dataset: water quality dataset for the 'internet of things: Online anomaly detection for drinking water compityccompttiiion the genetic and evolutionary computation conference 2018, kyoto, japan, 2018. Kyoto, Japan.
M. Munir, S. A. Siddiqui, A. Dengel, and S. Ahmed. Deepant: deep learning approach for unsupervised anomaly detection in time series. Leee Access, 7:1991-2005 2018.
Y. Nie, N. Nguyen, P. Sinthong, and J. Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. International Conference on Learning Representations (ICLR), 2023.
Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. ar preprint arXiv:2211.14730, 2022.
Boris N Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua. Bengio. N-beats: Neural basis expansion
analysis for interpretable time series forecasting, In International Conference Learning Representations, 2020.
R. Paffenroth, K. Kay, and L. Servi. Robust pea for anomaly detection in cyber networks. arXiv preprint wxXiv1881.0157%, 2018.
J. Paparrizos and L. Gravano. k-shape: Efficient and accurate clustering of time series. Proceedings of the 2015 ACM SIGMOD international conference on management of data, pp. 1855-1870, 2015.
J. Paparrizos and L. Gravano. Fast and accurate time-series clustering. ACM Transactions on Dalabase Systems (TODS), 42(2):1-49, 49, 2017.
Xianglei Qiu, Jilin Hu, Lekui Zhou, Xingjian Wu, Junyang Du, Buang Zhang, Chenjuan Guo, Aoying Zhou, Christian S. Jensen, Zhenli Sheng, and Bin Yang. Tib: Towards comprehensive and fair benchmarking of time scries forocasting mothods. 2024.
S. Ramaswany, R. Rastogi, and K. Shim. Efficient algorithms for mining outliers from large data sets. In Proceedings of the 2000 ACM SIGMOD international conference on Management of data, pp. 427-438, 2000.
K. Rasul, A. Ashok, A. R. Williams, A. Khorasani, G. Adamopoulos, R. Bhagwatkar, M. Biloš, H. Ghonia, N. V. Hassen, A. Schneider, et al. Lag-Ilama: Towards foundation models for time series forecasting. arXiv preprint XrX::22310.082
2023.
H. Ren, B. Xu, Y. Wang, C. Yi, C. Huang, X. Kou, T. Xing, M. Yang, J. Tong, and Q. Zhang. Time-series anomaly detection service at microsoft. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery data mining, pp. 3009-3017, 2019.
Matthew A Reyna, Chris Josef, Salman Seyedi, Russell Jeter, Supreeth P Shashikumar, M Brandon Westover, Ashish Sharma, Shamim Nemati, and Gari D Clifford. Early prediction of sepsis from clinical data: the alysionet/coomputing in cardiology challenge 2019. In 2019 Computing in Cardiology (CinC), 2019.
D. Roggen, A. Calatroni, M. Rossi, T. Holleczek, K. Förster, G. Tröster, P. Lukowicz, D. Bannach, G. Pirkl, and A. Ferscha. Collecting complex activity datasets in highly rich networked sensor environments. In 2010 Seventh international conference networked sensing systems (INSS), pp.
IEEE, 2010.
David W. Romero, R. J. Bruintjes, J. M. Tomezak, E. Bekkers, M. Hoogendoorn, and J. C. van Gemert. Flexconv: Contimuous kernel convolutions with differentiable sizes. arXiv preprint or 2rXiw:211000080 2021a.
16

---

Published in Transactions on Machine Learning Research (06/2025)
David W Romero, Anna Kuzina, Elias J Bekkers, J.M. Tomczak, and M. Hoogendoorn. Ckconv: Continuous kernel convolution for sequential data. arXiv preprint or wrXic2212200011 2021b.
P. J. Rousseeuw and K. V. Driessen. fast algorithm for the minimum covariance determinant estimator. Technometrics, 41 (3):212-223, 1999.
M. Sakurada and T. Yairi. Anomaly detection using autoencoders with nonlinear dimensionality reduction. In Proceedings of the MLSDA 2014 2nd workshop machine learning for sensory dala analysis, pp. 11, 2014.
B. Schölkopf, R. C. Williamson, A. Smola, J. Shawe Taylor, and J. Platt. Support vector method for novelty detection. In Aduances in neural information processing systems, 12, 1999.
1. Sharafaldin, A. H. Lashkari, and A. A. Ghorbani. Toward generating new intrusion detection dataset and intrusion traffic characterization. In ICISSp, volume 1, pp. 108-116, 2018.
H. Si, C. Pei, H. Cui, J. Yang, Y. Sun, S. Zhang, J. Li, H. Zhang, J. Han, and D. Pei. Timescricsbench: An industrial Mouutrial-grade lal-grade benchmark for time series anomaly detection models. arXie preprint urXiv:2402.1000 10802, 2024.
Ya Su, Y. Zhao, Chenhao Niu, Rong Liu, W. Sun, and Dan Pei. Robust anomaly detection for multivariate time series through stochastic recurrent neural network. KDD, 2019a.
Ya Su, Youjian Zhao, Chenhao Niu, Rong Liu, Wei Sun, and Dan Pei. Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery Dala Mining (KDD); pp. 2828 2837, 2019b. doi: 10.1145/ 10.1145/3292500.3333 3330067
M. Thill, W. Konen, and T. Bäck. Markusthill/mgab. The mackey-glass anomaly benchmark, Apr 2020.
L. Tran, Fan, and C. Shahabi. Distanco-basod outlier detection in data streams. Proceedings of the VLDB Endowment, 9(12):1089-11 1100, 2016.
S. Tuli, G. Casale, and N. R. Jennings. Tranad: deep transformer networks for anomaly detection in multivariate time series data. Proceedings of the VLDB Endowment, 15()):10011121, 2022.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, pp. 5998 6008, 2017.
A. von Birgelen and O. Niggemann. Anomaly detection and localization for cyber-physical production systems with self-organizing maps. IMPROVE-Innoud Modelling Approaches for Production Systems to Raise Validatable Efficiency: Intelligent Methods for the Factory of the Future, pp. 55-71, 2018.
Huiqiang Wang, Jian Peng, Feihu Huang, Jince Wang, Junhui Chen, and Yifei Xiao. Mien: Multi-scale local and global context modeling for long-term series forecasting. In ICLR, 2022.
Shiyu Wang, Jiawei Li, Xiaoming Shi, Zhou Ye, Baichuan Mo, Wenze Lin, Ju Shengtong, Zhixuan Chu, and Ming Jin. Timemixer++: A general time series pattern machine for universal predictive analysis. International Conference on Learning Representations (ICLR), 2025.
Pete Warden. Speech commands: A dataset for limited-vocabulary speech recognition. arXio preprint AXXi::184..0200 2018.
Gerald Woo, Chenghao Liu, Akshat Kumar, et al. Unified training of universal time series forecasting transformers. In ICML, 2024.
Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. Xiv preprint 2xXi:2220002166 2023.
17

---

Published in Transactions on Machine Learning Research (06/2025)
R. Wu and E. J. Keogh. Current time series anomaly detection benchmarks are flawed and are creating the illusion of progress. IEEE transactions knowledge and dala engineering, 95(3):2421-2422) 2021.
H. Xu, W. Chen, N. Zhao, Z. Li, J. Bu, Z. Li, Liu, Y. Zhao, D. Pei, Y. Feng, al. Unsupervised anomaly detection via variational auto-encoder for seasonal kpis in web applications. In Proceedings of the 2018 world wide web conference, pp. 187-196, 2018.
J. Xu, H. Wu, J. Wang, and M. Long. Anomaly transformer: Time series anomaly detection with association discrepancy. In International Conference on Learning Representations, 2021.
Z. Xu, A. Zeng, and Q. Xu. Fits: Modeling time series with 10k parameters. In The Twelfth International Conference on Learning Represenlations, 2023.
T. Yairi, Y. Kato, and K. Hori. Fault detection by mining association rules from house-keeping data. In proceedings of the 6th International Symposium on Artificial Intelligence, Robolics and Aulomalion in Space, volume 18, pp. 21, 2001.
Mao Yang, Xiangyu Li, Zifen Han, Yong Sun, and Xiqiang Chang, Ultra-short-temn multi-step prediction of wind power based modernten and multi-task learning. In The 9th International Conference Power and Renewable Eneryy, 2024.
C.-C. M. Yeh, Zhu, L. Ulanova, N. Begum, Ding, H. A. Dau, D. F. Silva, A. Mueen, and Keogh. Matrix profile i: all pairs similarity joins for time series: unifying view that includes motifs, discords and shapelets. In 2016 TEEE 16th international conference data mining (ICDM), pp. 1317-1322, 2016.
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? AAAI, 2023.
S. Zhang, Z. Zhong, D. Li, Q. Fan, Y. Sun, M. Zhu, Y. Zhang, D. Pei, J. Sun, and Y. Liu. Efficient kpi anomaly detection through transfer learning for large-scale web services. IEEE Journal on Selected Areas in Communications, 2448):2440-244 2022.
Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang, Informer: Beyond efficient transformer for long sequence time-series forecasting. AAAI, 2021.
Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang. Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. International Conference Machine Learning (ICML), 2022.
Tian Zhou, Peisong Nuu, Liang Sun, and Rong Jin. One fits all: Power general time series analysis by pretrained Im. Advances in Neural Information Processing Systems, 2023.
18

---

Published in Transactions on Machine Learning Research (06/2025)
Appendix
Contents
Introduction
1
2
Related Work
2
3
Methodology
3
Modern TCN
3
3.2
Tasks and Datasels
4
3.3
Experimental Setup
5
Experimental Results
7
Results Reproducing the Original Paper
7
4.2
Results Beyond the Original Paper
9
Discussion
11
5.1
Contact with the authors
12
Appendix
19
A Experimental details
21
Hyperparameters on the Extended Setups
21
Anomaly Detection Datascts of TSB- U-AD and TSB-M-AD
22
TSB-DD Baselines
23
Reproducibility and Bascline Sources
23
B Visualizing the ERF
24
C
Full Results
25
C.1 ModernTCN Long- Term Forecasting
25
C.2 ModernTTN Imputation
25
C.3 Results on PhysioNet 2019 and Statistical Significance
25
C.4
TiRex Full Results with Varying Input Lengths
26
D Drop Last Trick
26
E
Excluded Results
27
M4 Dataset Results
27
E.2 EAA Dataset Results
28
19

---

Published in Transactions on Machine Learning Research (06/2025)
F Short- Term Forecasting on ETT
28
20

---

Published in Transactions on Machine Learning Research (06/2025)
A Experimental details
A.1 Hyperparameters on the Extended Setups
For the original experiments, which aimed at reproducing results, we utilized the optimal settings reported in the ModernTCN paper. For the extended experiments, we conducted grid search to determine the optimal hyperparameters. The grid for this search was constructed based on the optimal parameters of corresponding tasks as described the ModernTCN paper. The specific details of the grid scarch and the selected parameters for the extended selups provided in the tables below. Evaluations were performed the validation set for 10 epochs, and the hyperparameters yielding the best validation
selected.
Table 11: Hyperparameters for Speech Commands (MFCC)
Parameter
Options Considered
Selected Value
Patch Size, Stride
1,1; 8,4; 16,8 32, 64, 128
1,1
Dimension
32
Kernel Size
13, 31, 51, 71
13
Learning Rate Scheduling
exponcntial (typel), stop (type2), delayed exponential (type3) 2, 3,
type3
Number of Modern'T Blocks
3
Table 12: Hyperparameters for Speech Commands (Raw)
Parameter
Options Considered 1,1; 16,8; 32,16; 64,32 64, 128, 256, 512
Selected Value
Patch Size, Stride
32,16
Dimension
512
Kernel Size
13, 31, 51, 71 exponential (typel), step (type2), delayed exponential (type3) 3,
71
Learning Rate Scheduling
type3
Number Modern'ICN Blocks
3
Table 13: Hyperparameters for PhysioNet
Parameter
Options Considered
Selected Value
Patch Size, Stride
1,1; 8,4; 16,8 16, 32, 64, 128
1,1
Dimension
32
Kernel Size
13, 31, 51, 71
31
Learning Rate Scheduling
exponential (typel), step (type2), delayed exponential (type3) 3,
type3
Number ModernTCN Blocks
Table 14: Hyperparameters for Character Trajectories
Parameter
Options Considered
Selected Value
Patch Size, Stride
1,1; 8,4; 16,8 16, 32, 64, 128 13, 31, 51, 71
1,1
Dimension
32
Kernel Size
13
Learning Rate Schoduling
exponential (typel), step (type2), delayed exponential (type3) 3,
type3
Number ModernTCN Blocks
Table 15: Hyperparameters for TSB-AD Anomaly Detection (Multivariate)
Parameter
Options Considered
Selected Value
Window Size
32, 96, 192 1u-3, 1e-4, 1e-5
96
Learning Rate
0.001
Keruel Size
13, 31, 51, 71
13
21

---

Published in Transactions on Machine Learning Research (06/2025)
Table 16: Hyperparameters for TSB-AD Anomaly Detection (Univariate)
Parameter
Options Considered
Selected Value
Window Size
32, 96, 192 1e-3, 1e-4, 1e-5
Learning Rate
0.001
Kernel Size
13, 31, 51, 71
Table 17: Hyperparameters for Short Term Forccasting ETTh1
Parameter
Options Considered
Selected Value
Input Sequence Length (Pred Len 6)
6x1, 6x2, 6x3
6x3
Kernel Size (Pred Len 6)
13
13
Imput Sequence Length (Pred Len 12)
12x1, 12x2, 12x3
12x3
Kernel Size (Pred Len
13
Imput Sequence Length (Pred Len 18)
18x1, 19x2, 18x3
18x3
Kernel (Pred 18)
13,
Table 18: Hyperparameters for Short Term Forecasting on ETTml
Parameter
Options Considered
Selected Value
Tnput Sequence Length (Pred Len 6)
6x1, 6x2, 6x3
6x3
Kernel Size (Pred Len 6)
7, 13
13
Input Sequence Length (Pred Len 12)
12x1, 12x2, 12x3
12x3
Kernel Size (Pred Len 12)
13, 31
13
Input Sequence Length (Pred Len 18)
18x1, 19x2, 18x3
18x3
Kernel Size (Pred Len 18)
7, 13, 31
13
A.2 Anomaly Detection Datasets of TSB-U-AD and TSB-M-AD
We extend the evaluation to TSB-AD (Liu Paparrizos 2024 a comprehensive benchmark for anomaly detection. This benchmark consists of 1070 time series from a diverse collection of 40 datasets, including univariate and 13 multivariate datasets. For detailed characteristics, Table I9
22

---

Published in Transactions on Machine Learning Research (06/2025)
Table 19: Summary characteristics 40 datasets included in TSB-AD. 1. in the 2nd column indicates this
dataset is transformed from the multivariate dataset. The 'Category' column indicates whether the datasets feature point anomalies (P) or sequence anomalies (Seq). Adapted from Liu & Paparrizos 2024
Ave
Anomals
Aug Anomaly
Anomall
Cacogiry
5000.7
YAHl
1560.2
5.5
2.5
0.95%
PArSin
101S
(02-)
727922
25.6
68.7
1.5%6
Sen
MGAAI 11
a]
2020
10
07717 8
9.7
20.0
0.
WSn (annnng 0E m0 2766
210
111
17488.5
5.1
25.4
0.655
Sin
380
93330 3
14.7
64.0
4.156
1501
750
Ben
Daphei
2001
38775.0
361.3
5.9%5
Sii
CATSS-2 (1))
200000.0
19.0
778.0
4.355
SWiit (1)
1at -in---aame miaaue 2018
115919.0
27.0
1876.0
12 850
San
2001
09700.0 5
127.5
144.5
18
1501
10000.0
MITDB
051260.0
MSL
$492.0
SMAP (1
7700
1.2
210.1
2.555
SMD (U) STEAE 2011
28
24207.7
2.4
173.7
2.055
SVDB LevetemAnn
70501
20
17:38000
36.4
292.5
5.650
San
OPP (U)
2010
16344.8
1.4
653.4
5.476
TSD-ADOM
GHI
Fllnne al.
2015
19
180001
9.8
1005
Seq
Duploce
95776.0
Sen
16 16220.3
0.355
245
17426. 75
254.3
SMD
25-88..4 36
8.0
112.8
5.895
Sin
SWi
2016
207457 DO
16.5
1050.
12 756
2001
FSM
(1i00iin 11 m1
TA35 25
217824.0
72.0
555
12 255
PacSSon
1.3
196-3
Seg
1.8
111.7
Seg
GBCCO
Mories
195521
51.3
MITD:N
296153.5
15.2
2.755
SVDB
222.5
08.5
268.2
4.8%5
Sen
LEDU
2000
2
100000.2
105.3
131.4
15 536
Nen
CATS. (eae 2625)
17
24.000.0
11.5
811.
5.795
Sen
TAD
TAX
10000.0 3
745.2
1.3
ParSng
A.3 TSB-AD Baselines
We compare Modern against wide range of baselines from the TSB-AD benchmark (Liu & Paparrizos 2024), which are categorized into three groups:
Statistical Methods: MCD Rousseeuw & Driessen 1999 OCSVM Schölkopp et al. 1999 LOF
[Breunig et al. 2000), KNN (Ramaswany et al. 2000), KMeansAD Yairi et al. 2001). CBLOF
(He et al. 2003). POLY (Li et al.] 2007). IForest [Liu et al.) 2008). HBOS (Goldstein & Dengel
2012), KShapeAD (Paparrizos & Gravano 2015 2017: Boniol et al. 2021). MatrixProfile (Ych et al.
2016), PCA [Aggarwal] (017)) RobustPCA (Palfenroth et al. 2018). ETF (Hariri et al.] 2019), ; SR
(Ren al. 2019). COPOD (Li al. 2020 Series2Graph (Boniol & Palpanas 2020). and SAND
Boniol et al. 2021
Neural Network-based Methods: AutoEncodor (Sakurada & Yairi 2014 LSTMAD Malhotra
ett al. 2015), Donut (Xu et al. 2018), CNN (Munir et al.] 2018). OmniAnonalyy
(Su et al.) 2019a
USAD (Audibert et al.] 2020). moomalyTransfrrmm (Xu al, 2021), TranAD
Tuli
2022
TimesNet (Wu al. 2023) and FITS (Xu al. 2023).
Foundation Model-based Methods: OFA (GPT4TS) (hhou et al. 2023 Lag-Llama Rasul
et al.
2023
Chronos (Ansari et al. 2024 TimesFM (Das et al. 2024), and MOMENT (Goswami
al.
2024
A.4 Reproducibility and Baseline Sources
To reproduce the results presented in this paper, one can refer to the original repository Some of the baseline results are produced in their official codebases; one can refer to the sources listed in Table 20
4TiRex was for 4 different input lengths, and the hest results are reported the main content. For detailed results,
A Appendix C.4
23

---

Published in Transactions on Machine Learning Research (06/2025)
Table 20: Overview of Baseline Score Calculation Sources
Tosk
Beseline Type / Source
Citation
Code Repository
Long-term Forecasting
Task SpeciSpecific Baselines
Qia et a
2024
TFB
Long-lerm Forecasting
Foundational Models (exel. Tirex)
Beach
Long-term Forecasting
Tirex
Auer e$
22025
Irex
(Subnodule in REB LE-MMoeraTCCN
Inpulation
Baselines
Series- Libeary
thorm-term Forecasting
Baselines
WT e4 04.
202.9
Ine Scres Library
Speech Commande Clasedication
Baselines
KLony
Flex CONN
S4
Variable Length Time Series
Baselines
Leu & Shin 2225
CRCoRV
Physiones 19
Baselines
Romero el al
272
CRooRy
Anouady Detection
Baselines
Paparriae
5B-AD
(Submodule RE- MNddeeerrTTCN)
B Visualizing the ERF
Formally, let [(n x m x 1) be the input time series, where n is the number of samples, m is the mumber of variables, and is the input sequence length. Let F(n x m x d x I') be the final output feature map, wo desire to measure the contributions of every time point on to the central points of every channel on F, i.e., F //..///2, which can be simply implemented via taking the derivatives of F...///2 mechanism. Concretely, we sum up the central points, take the derivatives the input as the time contribution scores and remove the negative parts (denoted by P). Then aggregate the entries across all the examples and the input variables, and take the logarithm for better visualization. Formally, the aggregated contribution score matrix A(I) is given by
I with the auto-grad
O t 2jj
EE (1..112)) ,0)
P max
(1)
A = loggu
Paja + 1).
(2)
Then we respectively rescale A of each model to [0, 1] via dividing the maximum entry for the comparability across models.
(a) MCNernTCN
(b) Our Method
Figure 4: ERF visualizations from ModernTCN and our method for the same setup.
Following the mentioned methodology (from Kim et al. 2023 and Ding et al. 2022a as adapted in Luo & Wang (2024), we explored the ERF visualizations for the ETThl forecasting task with an input sequence length of 336. The ModernTCN approach claims to adapt these methodologies for 1D sequences; however, the specifies of their adaptation are not detailed, nor is the code available their official repository. To establish a reference point, we visualized the same model under the same sctup. As shown in Figure 4 the ERF visualization from ModernTCN exhibits more colored areas compared to method. Such differences might be attributed to the color scheme used. In our visualizations, corresponding to values between [0.1], whereas MCdernTCN does not. Additionally, in our plots, not all black areas represent zero gradient contribution; rather, they are very close to zero relative to the maximun gradient contribution due to normalization. Upon closer inspection, the black regions near the center in our visualization reveal a slight red hue, indicating a minimal gradient contribution.
clearly indicate the range of colors
Nevertheless, as depicted in Figure 2 the global convolution--basee method demonstrates a significantly greater ability to enlarge the ERF compared to ModernTCN.
24

---

Published in Transactions on Machine Learning Research (06/2025)
C
Full Results
C.1 ModernTCN Long- Lerm-Term Forecasting
Table 21: Long-terin forecasting results. 'Reported': scores from Modern paper. 'Rerun:: our repro- duced results. "N.D.L.": No Drop Last, results with test loader not dropping incomplete last batches. 'Len" indicates prediction length in time steps (96, 192, 336, and 720 for most datascts; 24, 36, 48, and 60 for ILI). Lower MSE/MAL indicates better performance. Values are multiplied by 1000 for brevity.
MANE
MS5T
MAAF
VIS
M/V
MISE
MAX
XSSI
M4AT
3/A/T
MISE
XISR
MANE
MFAR:
35%
2055
232
1332
132
155
143
3333
135
125
1351
TIV
41.
320
-95
222
253
239
2243
-256
1200
5598
720
653
3843
193
817
-311
34.1
01.8
0331
0.88
436
144
1720
Nores
190
356-
61.4
3118
0773
0231
290
032
200
140
2458
110
0.77
247
-67
710
1833
100
1303
463
304
413
3.6
300
385
303
050
585
117
210
011
100
202
122
210
235 311
107
2007
151
2110
27 1110
110
1000
018
335
432
033 8
370 8-3
11c
353
8:3
203
073 81
338
174
360 8.2
845 81
584
688
284
337 8
380
3008
538
055
1949
C.2 ModernTCN Imputation
Table 22: Imputation task results with varying percentages of missing data (12.5,, 25%, 37.5%, 50%). 'Original': reproduced results. "N.D.L.": No Drop Last, results with test loader not dropping incomplete last batches. 'Miss %" indicates percenlage of missing data. Lower MSE/MAE indicates better performance. All settings from original paper's repository. Values are multiplied by 1000 for brevity.
ECL
Weather
ETTh1
ETTA2
ETTml
ETTm2
N.D.L. Original Model
MSE
MSE
MSE
MSE
12.5
170
38
16
25.0
89
181
41
44
143
40
127
18
19
37.5
196
54
157
43
134
22
84
199
68
175
48
142
27
106
12.5
59
170 2
24 :
40 =
34 #
126 1
38
122
53 1
76 +
25.0
181 +
23
11
41
143
127
0
89
37.5
81 +
196
+
27
44
54 #
157 +
43
134
2
22
96 +
21 +
85 $
C.3 Results on PhysioNet 2019 and Statistical Significance
This section provides a detailed breakdown of the model performance on the PhysioNet 2019 dataset, in- cluding the results from five separate runs for each model configuration. Table 23 presents the individual AUC scores, along with the mean and standard deviation, offering a comprehensive view of model stability and performance.
Table 23: Detailed AUC scores for models on the PhysioNet 2019 dataset over 5 runs.
Model
Run 1
Run 2
Run 3
Run
Run 5
Mean
Std Dev
CKConv
0.868
0.888
0.884
0.897
0.876
0.883
0.011
Modern TCN
0.891
0.896
0.886
0.901
0.902
0.895
0.007
ModernTCN VarIndep
0.880
0.869
0.875
0.873
0.876
0.875
0.004
CKConv + ModernTCN
0.901
0.902
0.901
0.908
0.908
0.904
0.004
The statistical significance of the performance improvement of the 'CKConv ModernTNN- model over both 'CKCony' and 'ModerITNN was evaluated using one-sided Wilcoxon signed-rank test. This non- parametric test is suitable for comparing two related samples. In both comparisons, the alternative hypothesis
25

---

Published in Transactions on Machine Learning Research (06/2025)
was that 'CKConv MCNernICN has greater mean AUC score. The test yielded p-value of 0.031 for both comparisons, indicating that the observed improvements are stat statistically significant.
The Python code suippet below, using the 'SciPy' library, was used to perform the statistical tests. from scipy- stats import wilcoxon
# AUC scores for CKConv, ModernICN and CKConv + ModernTCN over 5 runs ckconv_scored [0.868, 0. 8888,, O. 0.884, 0 0.897, 0.876] modernten_scoree scores [0.891, 0.896, 0.886, 0.901, 0.902]
m_ p_ckconv_scores [0.901] 0.902, 0.901, 0.908, 0.908]
One-sided Wilcoxon test: H1: M+CKConv > ModernICN stat_m, P_ p_value_m = wilcoxon (m_ akconvyscores, modernten_scoree, alternative= 'reateerrrrrrrr))) pfint((""CCoo (f"CCoooooMMMderaTT vs ModernTCN: p prraluee(p_value_n::
# One-sided Wilcoxon test: H1: M+CKConv > CKConv
stat_c, P_ value_c wilcoxon (m ju_ckconv_
ckconv_scores, scores, sccovvsscores, alternative= 'gretterr""r)
print (f CCCCen-MMMder vs CKConv: p-value=(p_ callue_c:.3f)+
C.4 TiRex Full Results with Varying Input Lengths
TiRex was run with four different imput lengths for each prediction length to identify the optimal configu- ration. The best-performing results are presented in the main long-tew forecasting table. The full results, detailing performance across all tested imput lengths, provided below to show the sensitivity of the model to this perpparrreeeer.
Table 24: Performance with varying imput (IL) and prediction (PL) lengths long-term forecasting datasets (PL: 96, 192, 336, 720; IL: 336, 720, 1440, 2048). Note that the Illness dataset was evaluated with different prediction and input lengths (PL: 24, 36, 48, 60; IL: 36, 48, 60, 120), and these results are not included in this table.
Clontiguration
RETBE
KTAI
ETT52
XEAK
MIDK
KTAE
KENE
SEAU
SEST:
Eiecericity
KISEE
Weather
KTAV
X.AE
XISII
Eachhange
KTAL:
AESE
Illnnes
XEXIC
730(60) 2300 (1) 102,060 (0)223)
720 (46)
0.382
0.377
0.258
0.353
0.329
0.345
0.121
0.192
0.350
0.344
0.063
0.200
1.445
TL. 1440 (00)
0:377
0.375
0.955
0.221
0.382
0.359
0.181
0.344
0.190
0.300
0.151
0.197
0.350
0.217
0.086
0:313
4.067
1,200
Th 2045 (1200)
0.377
0.377
0.274
0.815
0.304
0.323
0.174
0.239
0.121
0.205
0.151
0.183
0.878
0.235
0.007
0.213
1-493
0.744
335 (30)
0.407
0.413
00.77
10:00
0:200
00:022
01143
0.223
0.201
03412
00.500
0/202
0.312
5.085
L:21
1440 (60)
0.095
0.405
0.347
0.354
0.375
0.217
0.140
0.235
0.222
0.005
0.317
0.316
0.385
4.505
1.476
ITc 2016 (1266
0.124
0.405
0.333
0.358
0.360
B-354
0.235
0.243
0.141
0.926
0.196
0.227
0.882
0.246
0.208
0-315
1.767
0.624
LL: 335 (50)
0.485
0.430
0.815
0,350
0.455
0.383
0.515
0.335
0.154
0.240
0.252
0.201
0.446
0.201
0.362
0.441
5.247
L,500
720 (48)
0.425
0.48L
0.205
0.426
0.410
0.402
L.TLT
0.351
0.168
IL. 2046 (126)
0.452
0.432
0.949
0.876
0.306
0.876
0.365
0.515
0.150
0.245
0.246
0.265
0.418
0.264
0.357
0.465
1.814
0.545
[E: 1335 (36)
02302
0.465
0.385
0.815
11216
11:127
0.00%
07:50
0.8
01365
11331
0:556
0.200
1,023
0.700
7.305
1.775
720 (48)
0.478
0.459
0.388
0.5000
0.422
0.3844
0.386
0.202
0.300
0.300
0.058
0.277
1.173
1..04
5.718
1.816
0.686
0.462
0.564
0.452
0.408
0.969
0.201
0.313
0.313
1.090
0.770
0.870
Drop Last Trick
The 'Drop Last Trick" refers to discarding the last batch if it contains fewer instances than the batch size during evaluation. Table 25 examines how this affects model performance across different datasets. Our analysis reveals that significant changes in MSE and MAE primarily occur when more than 5% of the test set is dropped, seen with ETTh1, ETTh2, Exchange (720), and ILI datasets. In most cases, performance metrics increase (positive A MSE/MAE values) when all data points are included, indicating worse performance as lower values are better for these metrics. This suggests that models may have been implicitly tuned for the smaller (est sets. While most datasels show performance degradation when including all data points, cases like Exchange (336) show improvement, which aligns with the fact that dropping data points can lead to either better or worse performance depending on the characteristics of the excluded points. This indicates that this faulty setup significantly affects the results, especially when high percentage of the test set is dropped due to high batch size and/or low test set size.
26

---

Published in Transactions on Machine Learning Research (06/2025)
Table 25: Analysis the 'Drop Last 'rick'' impact across datasets. For each dataset, we show test set size,
batch size, excluded data points, and performance differences. Numbers in parentheses indicate prediction length for forecasting tasks. A MSE/MAE shows the difference (No Drop Last Rerun), with positive values indicating performance degradation when all data points are included.
Dataset
That Set Size
Batch
Dropped Points
Dropped Points (%)
Loug-ttera Forectecing
ETTml
1,521
512
257
2 2323%%
+0.001
0.000
Long-term Forecasting
ETTm2
11.521
512
257
2. 2.23%
-0.001
40.001
Long-term Forernsting
ETTh1
2,881
512
11.10%
10,015
10.008
Long-term Forecueting
ETTT22
2,881
512
321
11.10%.
+9.025
+0.015
Long-term Forecnsting
Flectricity
5,261
32
13
0.25%
+4.004
+00.003
Loug-term Forecarting
Traffic
Long-term Forecasting
Weather (96)
10.540
256
44
0.42%
-0.002
-0.001
Long termi Porecasting
Weather (192)
10,540
250
44
0. 0.42%
-0.001
-0.001
Long-term
Weather (336)
10.540
512
300
2.85%
10,007
100001
Long-term Forecneting
Weather (720)
10,540
512
300
2.85%
+40003
+0.001
Loug-tert term- Forecarting
Exchange (96)
1,422
128
14
0. 0.98%
0.000
+0.001
Long-terat Forecneting
Exchange (192)
1,422
128
14
0. 0.98
+0.001
0.000
Long Forecasting
Exchange (336)
Long-term Foremnstnng
Exchange (720)
1,422
512
398
27. 90%
14.156
0.000
Long-term
Forecasting
ILI
170
32
10
5.98%
+0.510
+0.095
Inputtation
Imputation
ETTm2
11,521
16
0.01%
0.000
0.000
Imputation
2,881
16
0.03%
0.000
0.000
Imputation
ETTh2
2,881
0.000
Imputation
Electricity
5.261
13
0.255
0.000
0.000
Imputation
Weather
10,540
10
12
0.11%
0.000
0.000
E
Excluded Results
The results presented this section were excluded from our main analysis due to methodological concerns regarding validation procedures. As detailed in the Experimental Setup section, both the M4 dataset for short-term forecasting and the UEA dataset for classification are provided with only train and test splits, with no designated validation set. Upon reviewing the source code, we identified that the test set was used for validation in both cases, introducing data leakage as the model gains indirect exposure to the test data during training through early stopping and hyperparameter selection.
A methodologically sound approach would involve creating a validation split from the training data. For instance, N-BEATS (Oreshkin et al. 2020) addresses this issue with M4 by first creating a validation split from the training set, finding optimal training settings on that validation set, and then training on the full training set using these optimal settings. However, we chose not to implement. this approach for several reasons:
First, many of these datasets are already small for deep learning approaches. For example, M4- Weekly (359 samples), M4-Hourly (414 samples), and several UEA classification datasets like CthhanooConcentration (261 samples) and Heartbeat (204 samples) have limited training samples to create a validation split from.
Second, implementing a proper validation approach like N-BEATS (Oreshkin al. 2020) for MCN would yield worse results (at best the same results) than the reported, the training is full training set in both cases. On top of that, those reported results are already worse than the reported results in N-BEATS Oreshkin et al. 2020
Rather than allocating resources to rerun these experiments, we extended our study to new datasets such as multivariate short-term forecasting on ETT, time series classification on Speech Commands, and the PhysioNet 2019 sepsis challenge. These extensions allowed for more methodologically sound comparisons and a more comprehensive evaluation of MCdernTCN's capabilities across diverse time series tasks.
The tables below present the reproduced results for completeness, but emphasize that these should be interpreted with caution due to the methodological concerns outlined above.
E.1 M4 Dataset Results
27

---

Published in Transactions on Machine Learning Research (06/2025)
Table 26: Short-term forecasting task. Results are weighted averaged from several datasets under different sample intervals. Lower metrics indicate better performance. Reported refers to the scores reported in the ModernICN paper. Rerun refers to the scores obtained by rerunning the model with the settings specified in the source code.
Yearly
Monthly
SMAPK
MASE
SMAPE
MASE
SMAPE
MASE.
5MAFF
MASE
SMAPE
OWA
Reported
0.878
0:917
4.715
3.107
Reron
18.251
2.957
0.777
10.001
1.170
0.881
12,598
0.920
0.968
4.835
3.176
1.000
11.782
1.561
0.841
E.2 UEA Dataset Results
Table 27: Classification task. Accuracy metric is used. Reported refers to the scores reported in the Mod- emTTCN paper. Rerun refers to the scores obtained by rerunning the model with the settings specified in
the source code. Datasets: ErhanolConcentration (EC), FaceDetection (FD), Handwriting (HW), Heart-
beat (HB), JapaneseVowels (JV), PEMS-SF (PS), SelfRegulationSCPI (SR1), SelfRegulationSCP2 (SR2), SpokenArabicDigits (SAD), UWaveGestureLibbaary (UWL).
EC
FD
HW
HIB
JV
PS
SR1
SR2
SAD
UWL
Averaged
Reported 0.363
0.708
0.306
0.772
0.988
0,891
0.934
0.603
0.987
0.867
0.742
Rerun 0.319
0.687
0.284
0.771
0.981
0.832
0.928
0.617
0.981
0.859
0.726
F Short-Term Forecasting on ETT
4030
660
-00
(a) TCN
(b) PatchTST
(c) DLinear
(d) TimesNet
Figure 5: Short-term forecasting ETTml with prediction length 12 and input length 48.
28

---

Published in Transactions on Machine Learning Research (06/2025)
-3.4
-04
-04
(a) Modern TCN
(b) PStchTST
(c) DLinear
(d) TimesNet
Figure 6: Short-term forecasting on ETTh1 with prediction length 18 and input length 72.
29


Reviews:
[{"review_id": "yUGgz8sMVV", "reviewer": "Reviewer_Yx9H", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "> Strength:\n\n+ 1. The original experimental settings were carefully examined during the reproduction process, and modified comparative experiments were conducted.\n\n+ 2. The ERF visualization tool code of ModernTCN was provided to the community (but it does not seem to come from the author), which promoted the further development of related research.\n\n> Weakness:\n\n+ 1. This work is unsuitable for TMLR. Even as a reproducibility study, its scope is too narrow, focusing solely on ModernTCN code with limited community impact.\n\n+ 2. Innovation is limited. As a reproducibility paper, its novelty lies only in experimental extensions and deeper analysis, not in new methodologies. The identified issues, like the \"Drop Last Trick,\" have been previously noted in benchmarking papers.\n\n+ 3. It offers no effective suggestions for improvement. For instance, when ModernTCN underperforms on tasks suited to convolutional methods with global receptive fields (e.g., Speech Commands classification) and shows low parameter efficiency, the authors provide no better alternatives.\n\n+ 4. Despite detecting some methodological issues, there's still room for improvement in experimental setups for certain tasks. The use of \"point adjustment\" protocols in anomaly detection tasks is one such area where refinement is needed.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper systematically reproduces and expands ModernTCN, verifies its performance on a variety of time series tasks, and finds its advantages and disadvantages in different scenarios. By evaluating ModernTCN on more datasets and tasks, and providing an effective visualization implementation of the receptive field (ERF), it helps to understand the characteristics of the model.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "No", "audience_explanation": "", "requested_changes": "+ 1. Add more evaluations and discussions of TCN-Based baselines.", "broader_impact_concerns": "Not applicable", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ixlm41duwC", "reviewer": "Reviewer_NpRp", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strenths:\n1. The paper thoroughly reproduces the ModernTCN experiments and provides additional evaluations on new tasks and datasets.\n2. It expands the original paper’s scope by including comparisons against additional convolution-based models with global receptive fields (e.g., CKConv, S4, CCNN), providing a broader context.\n3. Highlights significant issues such as the “Drop Last Trick,” data leakage, and anomaly detection evaluation flaws, which are often overlooked but impactful.\n4. Clearly demonstrates the role of the cross-variable component (ConvFFN2) in handling missing values on irregularly sampled data.\n\nWeaknesses:\n1. ModernTCN’s performance significantly deteriorates when methodological issues such as the “Drop Last Trick” are addressed (Section 4.1.1), suggesting sensitivity to experimental setup, which is not analyzed thoroughly\n2. Compared to convolution-based models with global receptive fields (e.g., CCNN and S4), ModernTCN uses considerably more parameters yet achieves lower performance on certain classification tasks, notably Speech Commands (Section 4.2.2, Table 7).\n3. Despite claims of improved receptive fields, visualization results show ModernTCN’s ERF is relatively limited compared to CCNN, questioning the effectiveness of the model’s large kernels\n4. The anomaly detection protocol (“point adjustment”) severely overestimates model effectiveness, as a naive baseline outperforms ModernTCN (Table 4).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper is a reproducibility study of the ModernTCN model, a recently proposed convolutional network architecture designed for general time series analysis. ModernTCN aims to improve Temporal Convolutional Networks (TCNs) by using large kernel sizes to increase the effective receptive field (ERF), enabling the capture of long-term dependencies. The authors of this study validate ModernTCN’s claims regarding performance across multiple tasks such as forecasting, imputation, classification, and anomaly detection. They also extend the evaluation to additional datasets (Speech Commands and PhysioNet), compare ModernTCN against other convolutional architectures (CKConv, FlexConv, CCNN, and S4), and investigate experimental methodologies and components like the cross-variable module through ablation studies. Overall, the paper finds that ModernTCN achieves competitive performance but identifies several limitations related to experimental sensitivity, parameter efficiency, ERF effectiveness, and anomaly detection evaluation.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Clearly report results both with and without the tricks (Section 4.1.1). A detailed analysis of why the model’s performance deteriorates significantly without it would be helpful.\n2. Provide clear details on anomaly detection threshold selection and discuss the implication of “point adjustment” evaluation (Section 4.1.3). Additionally, explicitly compare the performance against straightforward baseline models to validate the effectiveness of ModernTCN.\n3.  Include and discuss effective receptive field visualization comparisons explicitly (as shown in Figure 2). Clarify the reason behind the observed discrepancy between the claimed large ERF and the visualized results.\n4. Add explicit discussions on ModernTCN’s parameter count relative to competing convolutional models and the implications on computational resources and training time (Table 7)\n5. Clarify and justify kernel size selection for classification tasks (Tables 6 / 8). Explain why performance does not follow clear trends across kernel sizes, and provide guidelines or recommendations for kernel size choice based on experimental results.\n6. Clearly document library and framework versions to ensure reproducibility is robust to minor implementation variations noted in the rerun results (Section 4.1).\n7. Provide a brief analysis or insight into why ModernTCN underperforms convolutional models with global receptive fields on datasets such as Speech Commands, despite employing large kernels (Table 7).\n8. In the ablation study on ConvFFN2 (Section 4.2.3), clarify explicitly how and why the cross-variable dependencies improve model handling of missing values and irregular sampling\n\nWhile I enjoy the topic to systematically reproduce a series of methods, I feel it requires more analysis on the pros / cons and why it succeeds or fails in certain cases.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "kK9hgX6rDT", "reviewer": "Reviewer_aWxW", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strengths\n## Reproducibility Commitment:\nThe authors provide a comprehensive re-run study of ModernTCN with detailed experimental setups and code disclosures in the Appendix. This commitment to reproducible research is commendable and crucial given the current reproducibility challenges in machine learning.\n\n## Identification of Experimental Pitfalls:\nThe paper exposes several key issues in experimental design, such as the \"Drop Last Trick\", using test data for validation, and tuning thresholds on the test data. These insights are valuable for the community and offer actionable guidelines for improving experimental rigor.\n\n## Thorough Empirical Evaluation:\nThe authors conduct extensive experiments across multiple time series tasks (e.g., long-term forecasting, imputation, anomaly detection, and classification) and include advanced analyses such as ERF visualizations. These methodologies enhance our understanding of the underlying model behavior.\n\n\n# Weaknesses  \n\n## Narrow Focus on ModernTCN:\nWhile revisiting studies are essential, the paper primarily focuses on the ModernTCN architecture. Although ModernTCN currently represents state-of-the-art performance in certain tasks, its heuristic nature and lack of theoretical guarantees (unlike approaches based on, e.g., Rough Path theory [1, 2]) may limit the long-term impact of the findings. By concentrating almost exclusively on one neural network structure, the paper's relevance might diminish as more advanced models emerge.\n\n[1] T. Lyons, et al. Differential equations driven by rough paths. 2007.  \n[2] T. Lyons and A. D. McLeod. Signature methods in machine learning. arXiv, 2022.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This study rigorously reproduces ModernTCN’s results across various time series tasks, extends the evaluation to additional datasets—including irregular and variable-length data—and identifies key methodological flaws, and offering new insights via effective receptive field (ERF) visualizations.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I would suggest the followings to make the paper more robust and broadly applicable, thereby enhancing its contribution to the field of time series analysis.\n\n## (Critical) Broaden the Scope and Refocus the Presentation:\nThe paper should be repositioned as a revisiting study of time series analysis in general, rather than solely focusing on ModernTCN. By broadening the perspective, the study would offer insights into common pitfalls—such as the \"Drop Last Trick\", data leakage, and threshold tuning—that can occur even in top-tier papers. The current inclusion of baselines like Transformer-based PatchTST indicates that a broader narrative is feasible, and rewriting the paper to address a wider range of time series models would significantly increase its general interest and impact.\n\n## (Less Critical) Statistical Reporting:\nAlthough the experimental evaluation is extensive, incorporating error bars or statistical significance measures for the results would further strengthen the claims. For example, reporting confidence intervals or standard deviations for performance metrics such as MSE or MAE would add rigor. Given the breadth of experiments, we acknowledge that this may be challenging, but even a subset of statistically robust comparisons would enhance the credibility of the findings.\n\n## (Least Critical) Extend the Study to Variable-Length Time Series:\nThe current study largely focuses on fixed-length time series. Evaluating additional TCN variants that are designed to handle variable-length inputs (e.g., [3-6]), as well as benchmarking against other architectures such as state-space models and Transformers, would provide a more comprehensive overview of the field. This extension is essential to determine whether the issues identified—like the sensitivity to experimental design—are unique to ModernTCN or represent broader challenges across different time series modeling approaches.\n\n[3] E. J. Keogh and C. A. Ratanamahatana. Exact indexing of dynamic time warping. Knowledge and Information Systems, 2002.  \n[4] M. Schneider, et al. An end-to-end machine learning approach with explanation for time series with varying lengths. Neural Comput. Appl., 2024.  \n[5] A. Sawada, et al. Convolutional neural networks for time-dependent classification of variable-length time series. 2022 International Joint Conference on Neural Networks (IJCNN), 2022.  \n[6]  H. Lee and D. Shin. Beyond information distortion: Imaging variable-length time series data for classification. Sensors, 2025.", "broader_impact_concerns": "No significant ethical or broader impact concerns are identified in this work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper provides a well-executed reproducibility study of ModernTCN, offering significant methodological improvements and addressing several important issues in time series analysis. While the authors have made improvements in the breadth of their study, further revision is needed to strengthen the generalizability and impact of their findings.

Strengths:
- The paper demonstrates strong reproducibility efforts and provides valuable insights on ModernTCN, including performance analysis across various tasks and datasets.
- It identifies and addresses methodological issues like the "Drop Last Trick" and the use of test data for validation, which contribute to the broader understanding of time series analysis models.
- The addition of relevant baselines and the extension of evaluations to new tasks make the study more comprehensive.

Weaknesses:
- The focus on ModernTCN limits the paper's long-term impact, and the framing of the paper still heavily revolves around this model. A more general presentation would enhance the paper’s relevance.
- While the paper addresses methodological flaws, it lacks clear suggestions for improving ModernTCN’s performance on tasks where it underperforms compared to models with global receptive fields (e.g., Speech Commands).
- There is room for improvement in experimental setups, particularly for tasks like anomaly detection and classification with varying input lengths.

Revision Requests:
- Broaden the scope of the paper to include more time series models and not focus exclusively on ModernTCN, which has been addressed by the authors
- Include more detailed analysis and justification for certain methodological choices, such as kernel size selection and parameter tuning.
- Provide statistical significance for the experimental results and include error bars where possible to strengthen the claims made.
Decision: Accept (poster)