# Anomaly Detection in Financial Time Series by Principal Component Analysis and Neural Networks +
## 5 Model Evaluation: Main Results
### Numerical Results against Benchmark Models  
To assess the performance, the suggested approach was compared numerically with well-known machine learning algorithms for anomaly detection, that is, isolation forest (IF), local outlier factor (LOF), density-based clustering of applications with noise (DBSCAN), K-nearest neighbors (KNN), and support vector machine (SVM), all reviewed in Appendix B. In addition to these state-of-the-art techniques, we considered the recent anomaly detection technique proposed by [PERSON] et al. (2014), referred to as sig-IF, which combines a feature extraction step through a path signature computation with IF. Tables 9-11 summarize the performance on the training and test sets in Section 3 of each model for both the contaminated time series identification and the anomaly localization steps. The following paragraphs provide similar conclusions, which are drawn from the results of the identification and localization steps (see Tables 9 and 11). For the unsupervised learning methods, there was not a proper learning step and even though Tables 9 and 11 show the scores of the training and test sets, these scores should be seen as the ones obtained by testing the models on the independent data sets. For the IF, LOF, and DBSCAN models, the performance across the sets was not stable, as a significant difference could be observed between the scores on the training and test sets. The poor performance of the unsupervised learning algorithms could be explained by the difficulty in estimating the contamination rate for the IF and the LOF algorithms. For DBSCAN, the poor performance was rather due to the high dimensionality of the data. As for the sig-IF approach, its performance in this specific case was not as overwhelming as when used to detect pump-and-dump operations ([PERSON] et al., 2014). This difference in performance was driven by the fact that in our case, signatures were computed on the stock price path only since no \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline
**Algorithm** & **IF** & **LOF** & **DBSCAN** & **KNN** & **SVM** & **sig-IF** & **PCA NN** \\ \hline Exec. Time & 0.6915 & 0.2015 & 0.1275 & 1.725 & 4.572 & 2.776 & 0.003523 \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Execution time in seconds for identification of contaminated time series step. \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Data Set** & **Accuracy** & **Precision** & **Recall** & \(\mathbf{F_{1}}\)**-Score** \\ \hline Train set & 85.23\% (6.0\%) & 85.23\% (6.0\%) & 85.87\% (5.7\%) & 85.36\% (6.0\%) \\ Test set & 85.17\% (7.1\%) & 85.17\% (7.1\%) & 86.04\% (6.8\%) & 85.30\% (7.1\%) \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Mean (standard deviation) of performance metrics of suggested model over multiple runs for localization step of non-extreme anomalies. \begin{table}
\begin{tabular}{l c c c c} \hline \hline  & **Train** & \multicolumn{2}{c}{**Test**} \\ \hline
**Model** & **Accuracy** & \(\mathbf{F_{1}}\)**-Score** & **Accuracy** & \(\mathbf{F_{1}}\)**-Score** \\ \hline IF & 42.31\% & 42.31\% & 69.64\% & 7.022\% \\ LOF & 59.95\% & 59.95\% & 90.00\% & 62.41\% \\ DBSCAN & 50.00\% & 66.67\% & 16.65\% & 28.53\% \\ sig-IF & 49.48\% & 49.23\% & 72.32\% & 15.19\% \\ \hline KNN & 94.68\% & 94.39\% & 64.82\% & 26.69\% \\ SVM & 81.96\% & 82.33\% & 44.39\% & 27.44\% \\ PCA NN & 90.97\% & 90.31\% & 88.58\% & 71.30\% \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Performance evaluation of unsupervised (**upper part**) and supervised (**lower part**) models for contaminated time series identification step. additional information was available to describe the price path we were analyzing. For the pump/dump detections, instead, the signatures were computed on a set of variables' paths, including the price path and additional variables path, which was helpful in the identification of the pumps/dump attempts. Regarding the supervised approaches, although KNN outperformed the suggested approach on the training set, there was a non-negligible decrease in these scores on the test set.