## 3 Results
### Performance Analysis by Method  
We evaluate the performance of six anomaly-detection methods using three metrics: macro-averaged AUC ROC, F1 score and UCR score, as well as the average runtime for a single time series. The results are visualisd in Figure 4. Of the methods compared, MDI achieves the highest AUC ROC and UCR scores, while MERLIN performs better in terms of F1 score. Among the deep-learning methods, GANF has the highest scores across all three metrics. F1- and UCR scores are 4% lower compared to the best-performing classical method, and the AUC ROC of 0.66 is the second best. AE scores are higher than TranAD for the AUC ROC and UCR Score, while TranAD shows a slightly higher F1 score. RRCF performs poorly, failing to detect a notable amount of anomalies in the test set and demonstrating the lowest F1 and UCR scores. The numerical results are shown in Table 2. The scores in Table 2 are generally low across all methods, likely due to the test set time series producing low or zero scores. We discuss the implications of various combinations of high and low scores, as well as their potential causes, in Section 4. MDI and MERLIN, being deterministic methods, are not subject to sampling effects and therefore have a standard deviation of 0.0 among the six repetitions of the experiment. \begin{table}
\begin{tabular}{l l l l l l l l} \hline \hline
**Class** & **Method** & \multicolumn{2}{c}{**AUC ROC**} & \multicolumn{2}{c}{**F1 Score**} & \multicolumn{2}{c}{**UCR Score**} & \multicolumn{2}{c}{**Runtime (s)**} \\ \hline \multirow{3}{*}{Classical ML} & MDI & **0.66 \(\pm\) 0.0** & & 0.25 \(\pm\) 0.0 & & **0.47 \(\pm\) 0.0** & & **74** \\  & MERLIN & 0.51 \(\pm\) 0.0 & 0.58 \(\pm\) 0.0006 & **0.27 \(\pm\) 0.0** & **0.20 \(\pm\) 0.004** & 0.44 \(\pm\) 0.0 & **0.31 \(\pm\) 0.0033** & 291 \\  & RRCF & 0.56 \(\pm\) 0.0019 & & 0.07 \(\pm\) 0.011 & & 0.03 \(\pm\) 0.0004 & & 162 \\ \hline \multirow{3}{*}{Deep Learning} & AE & 0.58 \(\pm\) 0.01 & & 0.16 \(\pm\) 0.013 & & 0.28 \(\pm\) 0.025 & & 149 \\  & TranAD & 0.56 \(\pm\) 0.003 & **0.59 \(\pm\) 0.002** & 0.18 \(\pm\) 0.003 & 0.19 \(\pm\) 0.009 & 0.16 \(\pm\) 0.003 & 0.29 \(\pm\) 0.007 & 109 \\  & GANF & 0.63 \(\pm\) 0.009 & & 0.23 \(\pm\) 0.021 & & 0.43 \(\pm\) 0.03 & & 96 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Performance comparison of six anomaly detection methods on macro-averaged AUC ROC, F1 score, UCR score and runtime for a single time series. Results are grouped by model class (classical ML and deep learning) and presented as mean \(\pm\) standard deviation over six repetitions. The value in each second column denotes the mean aggregated by method class. Figure 4: (**a**) Macro-averaged performance metrics for each method. The error bars indicate the standard deviation caused by random effects over six repetitions of the experiment. (**b**) Average runtime per time series. For the deep-learning methods, training time is included. The error bars indicate the standard deviation over six repetitions of the experiment. In terms of average runtime for a single time series from the UCR anomaly Archive, MDI performs best, with a runtime of 74 s. TranAD is about 22 s slower on average (96 s) and GANF has an average runtime of 109 s. AE has a runtime that is twice as long as that of MDI, while RRCF (162 s) has a slightly longer runtime, but both fall below 200 s. MERLIN has the worst runtime of 291 s, which is almost four times that of MDI. It is worth noting that these runtimes may be influenced by the specific implementations used. MDI is implemented in C++ with a Python interface, while the other methods are purely implemented in Python. For the deep learning methods AE, GANF and TranAD, the training time is included in the reported runtime.