# Semi-Supervised Time Series Anomaly Detection Based on Statistics and Deep Learning
## 4 Performance Evaluation and Comparison  
The NAB (Numenta anomaly benchmark) [40] is used for evaluating the prediction performance of the proposed Tri-CAD framework. NAB contains many labeled time series datasets, which are either artificial or real-world time series. The datasets have entries containing timestamps and values. The timestamp is of the format \"yyyy-mm-dd hh.mm.ss\" representing the year, month, day, hour, minute, and second. The difference of adjacent timestamps is fixed; however, such a difference varies for various datasets. An entry is labeled as anomalous if its timestamp is specified explicitly in a separate file. As Tri-CAD is for semi-supervised anomaly detection, it does not rely on labels to train anomaly detection models. The labels are used only for the purpose of performance evaluation. This paper applies three real-world time series for performance evaluation. The applied time series are (a) \"NAB NYC taxi\", the data of taxi passenger counts in the New York city; (b) \"NAB CPU Utilization\", the CPU utilization data from Amazon Web Services (AWS); and (c) \"NAB Machine Temperature\", the data from a temperature sensor attached to a machine component. The performance measurements are precision, recall, and \(F_{1}\)-score of the prediction, whose definitions are shown in (12)-(14), respectively. In the definitions, \(TP\) (true positive) represents the number of anomalies that are truly (or correctly) detected as anomalies,_FP_ (false positive) represents the number of normal data that are falsely (or incorrectly) detected as anomalies, _TN_ (true negative) represents the number of normal data that are truly detected as normal data, and _FN_ (false negative) represents the number of anomalies that are falsely detected as normal data. \[\textit{precision}=\textit{TP}/(\textit{TP}+\textit{FP}) \tag{12}\]  
\[\textit{recall}=\textit{TP}/(\textit{TP}+\textit{FN}) \tag{13}\]  
\[\textit{F}_{1}\text{-score}=2\times\text{precision}/(\textit{precision}+ \textit{recall}) \tag{14}\]  
The NAB time series have labels given on the basis of anomaly windows [40]. Specifically, the entire time series is divided into time windows of a fixed window size (_fvs_). Note that different NAB datasets have different window sizes. If one or more anomalies appear in a time window, the time window is labeled as an anomaly window; otherwise, it is labeled as a normal window. Note that a labeled anomaly corresponds to an anomaly window whose center is the labeled anomaly. Figures 7-14 show the performance evaluation results of Tri-CAD using the three NAB datasets with the deviation threshold set as 3.89, 8, and 8.35, respectively. The performance evaluation is also conducted for different training dataset sizes and different WT sliding time window sizes. Specifically, the NYC taxi dataset and the CPU utilization dataset are assumed to have training datasets with a small size of 500 or a large size of 1000 (i.e., \(D\) = 500 or 1000). The machine temperature dataset is assumed to have training datasets with a small size of 1000 or a large size of 2000 (i.e., \(D\) = 1000 or 2000). Moreover, the machine temperature dataset is also assumed to have WT sliding window sizes of a small size of 30 or a large size of 60 (i.e., _vs_ = 30 or 60). In Figures 7-14, the anomaly windows are marked with pink strips, whereas detected anomalies are marked with red dots. If anomalies are claimed to be detected in an anomaly window by the proposed framework, then _TP_ is increased by one. On the contrary, if anomalies are claimed to be detected in a normal window, then _FP_ is increased by one. Note that all anomalies in a time window altogether increase a count by one. Similarly, if no anomaly is claimed to be detected in an anomaly window, then _FN_ is increased by one. After _TP_, _FP_, and _FN_ are derived, the precision, recall, and \(F_{1}\)-score can be evaluated accordingly.