# Time Series Anomaly Detection for KPIs Based on Correlation Analysis and HMM
## 3 Method
### KPI Alarm Clustering Based on Correlation Analysis
#### 3.4.2 Correlation Matrix Construction and Alarm Clustering  
This paper proposes a correlation analysis method called R-SWFCC, and constructs a correlation matrix to realize alarm clustering between KPIs. In this paper, we set the correlation score of the KPIs as _CCscore_, and the _CCscore_ comprehensively considers the correlation value, offset, and influence sequence. Firstly, whether sequence \(Y\) affects sequence \(X\) is determined. If not, a _CCscore_ = 0 is set directly; that is, the sequence of correlation influences is incorrect. If the temporal order is correct, the correlation scores will continue to be calculated, and the calculation formula is shown in Equation (8):  
\[CCscore=\left\{\begin{array}{c}0\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \,if\ X\to Y\\ W_{1}*|SWFCC|+W_{2}*\frac{1}{|s|+1},if\ Y\to X\ or\ X\leftrightarrow Y\end{array}\right. \tag{8}\]  
Here, \(SWFCC\) represents the correlation calculation results after the sliding window is added on the basis of Coflux. \(W_{1}\) and \(W_{2}\) in Equation (8) are the two weights corresponding to the correlation value and the shift value, respectively, obtained via the correlation calculation with the sliding window. After the analysis and test, they are set as \(W_{1}=0.8\) and \(W_{2}=0.2\). Shift value \(|s|\) is inversely proportional to correlation, that is, the higher of shift value, the lower the similarity between two sequences. Therefore, the shift value is used in the form of reciprocal calculation in Equation (8). Given that \(CCscore\in[0,1]\) and the denominator in the fraction cannot be zero, the influence of additional values on the overall results needs to be minimized. After comprehensively analyzing the above problems, this paper adds 1 to the offset, and then takes its reciprocal to prevent the denominator from being zero. Therefore, the specific form of the offset is set as \(\frac{1}{s+1}\).  
Through Equation (8), a correlation between two KPIs can be calculated, and a correlation matrix can be formed. Since there may be some abnormal KPIs with very low correlation in the correlation matrix, said correlation matrix can be classified and clustered through alarm clustering for all abnormal KPIs in a known period. Alarm clustering is used to classify and merge similar, associated, or identical KPIs through correlation analysis. Alarm clustering can reduce the impact of some irrelevant alarm KPIs. After the correlation calculation, a correlation matrix \(M\), with high correlation among multiple sets of KPIs, can be obtained according to the threshold of alarm clustering, and this threshold can be obtained through experimental analysis. There are multiple sets of KPIs in the correlation matrix, and the correlation between KPIs in each group is high, while the correlation between groups is low. Aiming at the abnormal KPIs in a certain period, KPIs with high correlation are selected, and their correlation is determined by the final correlation matrix. The calculation formulas for the matrix and each element in the matrix are shown in Equations (9) and (10):  
\[M=\left(\begin{array}{ccc}m_{11}&\cdots&m_{1N}\\ \vdots&\ddots&\vdots\\ m_{N1}&\cdots&m_{NN}\end{array}\right) \tag{9}\]  
\[m_{ij}=CCscore_{ij},i\in(1,N),j\in(1,N) \tag{10}\]  
There are \(N\) different KPIs in the matrix, and \(m_{ij}\in[0,1]\) represents the similarity of the transition probability between \(i\)th and \(j\)th KPIs.