# Online Detection and Fuzzy Clustering of Anomalies in Non-Stationary Time Series +
## 4 Detection Results
### Alternative Approaches  
We compare the performance of NARX-SCAPA with two popular, existing alternatives from the R packages Forecast and HDoutlier. The Forecast package, based on the model estimation method, is a commonly used tool for analysing large numbers of univariate time series in real time [34]. It uses the stoutliers method to model univariate non-stationary time series via Multiple Seasonal-Trend decomposition using Loess (MSTL) and Friedman's super smoother [35]. Possible anomalies are identified within the two most extreme quartiles of the residual series. The HDoutlier method allows probabilistic labelling of anomalies [36]. The method partitions data into different exemplars according to the length of the series. It then computes nearest-neighbour distances between exemplars and fits the cumulative distribution function and labels data points in any exemplar that is far from the others as anomalies. Tables 2 and 3 numerically compare the performance of these alternatives to NARX-SCAPA on the simulated data, both applied directly to the raw data and to the detrended residuals. We see that Forecast was more suitable for the residual series, and HDoutlier performed similarly in both cases. However, neither can achieve the same accuracy as NARX-SCAPA for detecting collective anomalies. HDoutlier achieved high accuracy for point anomalies, but this was coupled with an extremely high false positive rate. Figures A1 and A2 in Appendix A provide a fuller visualisation and discussion of the results of implementing Forecast and HDoutlier approaches on the original BT network traffic data. We see that the stoutliers algorithm was overly sensitive in handling the residuals, and HDoutliers was more conservative. It is worth noting that, for the BT traffic data, the probability of anomalies is low, yet their intensity is high. Therefore, Forecast does not work well with a fixed strength of seasonality for the decomposition. Meanwhile, collective anomalies of the split Type remain a challenge for HDoutliers as it either treats the outer data points as anomalies or labels entire regions as anomalies with a high false positive rate. Overall, compared to the performance of Forecast and HDoutlier, the proposed NARX-SCAPA model has achieved the best anomaly detection results with minimal false positives and false negatives (see Table 3). \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline Anomaly Type & 1 & 2 & 3 & 4 & 5 & 6 & 7 & 8 & 9 & 10 \\ \hline Forecast—Obs & 29.0 & 1.9 & 10.9 & 3.2 & 24.9 & 1.2 & 3.0 & 16.6 & 8.0 & 9.8 \\ Forecast—Res & 50.7 & 75.3 & 48.9 & 53.1 & 76.7 & 50.5 & 29.7 & 57.2 & 8.7 & 32.2 \\ \hline HDOutlier—Obs & 67.0 & 76.8 & 66.8 & 68.3 & 75.2 & 67.3 & 81.9 & 91.7 & 69.1 & 79.6 \\ HDOutlier—Res & 64.9 & 75.0 & 65.0 & 66.7 & 75.3 & 67.0 & 82.7 & 90.9 & 68.0 & 78.7 \\ \hline Anomaly Type & 11 & 12 & 13 & 14 & 15 & 16 & 17 & 18 & 19 & 20 \\ \hline Forecast—Obs & 41.1 & 0.0 & 14.1 & 2.6 & 27.6 & 5.5 & 30.0 & 60.0 & 0.0 & 0.0 \\ Forecast—Res & 5.5 & 7.1 & 13.9 & 14.6 & 66.3 & 63.5 & 70.0 & 100 & 80.0 & 100 \\ \hline HDOutlier—Obs & 58.7 & 58.3 & 65.6 & 69.1 & 92.3 & 90.4 & 100 & 100 & 90.0 & 100 \\ HDOutlier—Res & 60.7 & 62.3 & 68.9 & 68.1 & 91.3 & 86.7 & 100 & 100 & 90.0 & 100 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Accuracy of Forecast and HDOutlier methods on simulated BT traffic time series. We report performance on raw observations (Obs) and NARX residuals (Res). All accuracies are percentages. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Performance** & \multirow{2}{*}{**NARX-SCAPA**} & \multicolumn{2}{c}{**Forecast**} & \multicolumn{2}{c}{**HDOutlier**} \\
**Rate** & & **Observation** & **Residual** & **Observation** & **Residual** \\ \hline True negative & 98.0\% & 98.5\% & 99.6\% & 54.8\% & 54.9\% \\ True positive & 83.1\% & 10.3\% & 38.9\% & 72.8\% & 73.2\% \\ False negative & 16.9\% & 89.7\% & 61.1\% & 27.2\% & 26.8\% \\ False positive & 2.0\% & 1.5\% & 0.4\% & 45.2\% & 45.1\% \\ \hline \hline \end{tabular}
\end{table}