## 5 Conclusions  
In this study, we compared six anomaly detection methods, three of which were classical machine-learning methods and three of which were based on deep learning. We conducted extensive experiments on the UCR Anomaly Archive benchmark dataset, which we annotated with the types of anomalies present. We compared the methods on both a dataset level and an anomaly-type level to address two main questions: Does the potential superior performance of deep-learning methods justify the sacrifice of the intrinsic interpretability of classical methods? What are the similarities and differences between the analyzed methods when detecting different anomaly types? Our experiments showed that the classical machine-learning methods MDI and MERLIN outperform the deep-learning methods. The third classical method, RRCF, was unable to detect a substantial number of anomalies, but it improved when using sequence-based statistical features instead of raw data points. Among the deep learning methods, the Autoencoder model detected the most anomalies and was also the simplest model in this group.  
While we present our experimental results in this work, a deeper theoretical analysis of the reasons and mechanisms behind these results is left for future research. Regarding the second question about the similarities and differences in detecting certain anomaly types, we found that all subsequence-based methods detect the \"steep increase\" anomaly but not the \"smoothed increase\", while the opposite is true for the method that uses point-wise features. However, these classes are too small to produce a significant result. Although MDI and MERLIN had the best results in this comparison, they detected a diverse range of anomaly types. Together, they detected most of the anomalies, i.e., they detected 11 out of 16 anomaly types. However, the anomaly types \"unusual pattern\", \"time shift\", \"reversed\" and \"flat\" could not be reliably detected by any of the analyzed models. A more theoretical analysis of these results will be conducted in a subsequent study.  
**Author Contributions:** Conceptualization, F.R., J.D. and J.N.; Data curation, F.R.; Formal analysis, F.R.; Investigation, F.R.; Methodology, F.R.; Project administration, J.N.; Software, F.R.; Supervision, J.D. and J.N.; Validation, F.R.; Visualization, F.R.; Writing--original draft, F.R.; Writing--review and editing, F.R., J.D. and J.N. All authors have read and agreed to the published version of the manuscript.  
**Funding:** This research received no external funding.  
**Institutional Review Board Statement:** Not applicable.  
**Informed Consent Statement:** Not applicable.  
**Data Availability Statement:** The UCR Anomaly Archive dataset [38] that has been used in this work is available at [[https://www.cs.ucr.edu/~eamonn/time_series_data_2018/](https://www.cs.ucr.edu/~eamonn/time_series_data_2018/)]([https://www.cs.ucr.edu/~eamonn/time_series_data_2018/](https://www.cs.ucr.edu/~eamonn/time_series_data_2018/)) (accessed on 18 January 2023). A python repository which contains our benchmark source code and the anomaly type annotations to reproduce the results can be found at: [[https://gitlab.com/dlr-dw/is-it-worth-it-benchmark](https://gitlab.com/dlr-dw/is-it-worth-it-benchmark)]([https://gitlab.com/dlr-dw/is-it-worth-it-benchmark](https://gitlab.com/dlr-dw/is-it-worth-it-benchmark)) (accessed on 18 January 2023).  
**Conflicts of Interest:** The authors declare no conflict of interest.