# Time Series Anomaly Detection for KPIs Based on Correlation Analysis and HMM
## 4 Experiment
### Construction of Correlation Matrix  
The correlation score of the KPI residual sequence is derived after abnormal KPI screening, and then the correlation matrix is constructed to realize alarm clustering. In this experiment, we divided the KPIs screened in the previous section into two groups for correlation calculation and correlation matrix construction. The specific KPIs of each group of experiments are shown in Tables 6 and 7. In Tables 6 and 7, the KPI numbers in the first column show each of the KPI indexes in the experiment. The meanings of \"kpi_name\" and \"cmdbi\" in the second and third columns are the same as those in the second and third columns in Table 1. For each group of KPIs, the F1-score (see Formula (14)-(16)) is used to evaluate the effectiveness of R-SWFCC in this paper as well as in those of Pearson [22] and Coflux [2]. Table 8 describes FP (false positive) and FN (false negative) in the correlation analysis.  
In Table 8, if the final correlation between \(X\) and \(Y\) is higher than the threshold, then \(X\)-\(Y\). Otherwise, \(X\)-\(Y\). We use the F1-score to evaluate the effectiveness of our method, as well as those of Coflux and Pearson. The calculation of the F1-score is shown in Equations (14)-(16):  
\[Precision=\frac{TP}{TP+FP} \tag{14}\]  
\[Recall=\frac{TP}{TP+FN} \tag{15}\]  
\[F1-score=\frac{2\times Precision\times Recall}{Precision+Recall} \tag{16}\]  
Table 9 shows the best F1-scores of the three methods employed on two datasets. In these two datasets, the performance of proposed method is better than that of the other two methods. In the first set, the highest F1-score is 0.9162, which is derived from the method in this paper, and in the second set, the highest F1-score is 0.9020, again derived from the method in this paper.  
Figure 7 shows the times required for the correlation analyses of SWFCC, Coflux, and Pearson for different lengths of KPIs. It can be seen from the figure that the calculation speed of SWFCC is in the mid-range, and the calculation time increases linearly with the increase in KPI data length. The Pearson method is faster and the Coflux method is slower.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline  & **FP/FN** & **Ground Truth** & **Output** \\ \hline \multirow{2}{*}{Existence} & FP & \(X\sim Y\) & \(X\sim Y\) \\  & FN & \(X\sim Y\) & \(X\sim Y\) \\ \hline \hline \end{tabular}
\end{table}
Table 8: Descriptions of FP and FN in correlation.  
Figure 7: Calculation time of three of the methods under different KPI lengths. In the figure, the calculation speed of the SWFCC method proposed in this paper is close to that of Pearson, and the calculation speed of the Coflux method is slower.  
A thermal diagram is used to show the specific correlation, as in Figure 8. A thermal diagram can show experimental results more clearly and intuitively. The matrix in Figure 8a is a heat map of the correlation matrix calculated by the first set of experimental KPIs in Table 6, and that in Figure 8b is a heat map of the correlation matrix calculated by the second set of experimental KPIs in Table 7. The value at \((x,y)\) in the graph represents the absolute value of correlation between KPI \(x\) and KPI \(y\). The correlation between each KPI can be more clearly shown using a thermal diagram. The higher the correlation between two KPIs, the deeper the blue in the corresponding thermal diagram; conversely, the lighter yellow in the corresponding thermal diagram means a lower correlation. It can be seen from the figure that, in the first group of experiments, the correlations between KPI 1, KPI 2, KPI 3, KPI 4, KPI 5, KPI 6, KPI 7, KPI 8, KPI 9, and KPI 10 are high, shown by the deep blue rectangle. In the second group of experiments, the correlations between KPI 5, KPI 6, KPI 7, KPI 8, KPI 9, and KPI 10 are high, again presented by a deep blue rectangle. Therefore, we selected KPIs with exceptions and high correlations in this period for the next round of anomaly detection.