# Time Series Anomaly Detection for KPIs Based on Correlation Analysis and HMM
## 1 Introduction  
KPI (Key Performance Indicator) anomaly detection is a low-level core technology in intelligent operation and maintenance. It is mainly aimed at current events. By analyzing the KPI curve, the abnormal behaviors of KPIs (sudden increase, sudden drop, and jitter) imply that some potential faults have occurred in related applications, such as increased access latency, network failure, or sharp decreases in access users [1]. Due to the huge complexity of the system, KPIs of the monitoring system are numerous and various. When the system fails, the efficiency of manually searching for abnormal KPIs is extremely low, and consumes a lot of manpower and material resources. The manual analysis of system failure will also cause many misjudgments and produce certain economic losses. In the field of intelligent operation and maintenance, KPI anomaly detection for a multi-index system is difficult. In KPIs of the system, there may be a temporal correlation between two indicators; that is, when a KPI is abnormal, it will cause similar fluctuations in the trends of other KPIs in a short time. Due to the presence of numerous monitoring indicators and complex structures in the distributed system, fluctuations will continue to spread to more KPIs. This causes more KPI anomalies in the entire system, makinganomaly detection and root cause analysis very complicated. The correlation analysis can give rise to multiple similar abnormal KPIs. For example, a CPU utilization rate increase in the system will cause a class of CPU-related KPIs (such as CPU utilization rate of a single machine, CPU idle state time, overall CPU load, etc.) to produce anomalies. Without using correlation analysis, it is very difficult for operation and maintenance personnel to analyze and infer the correlation between large numbers of complex KPI anomalies. When faults occur in large-scale distributed systems, many redundant alerts will arise that are relevant but distract operation and maintenance personnel. Correlation analysis can narrow the scope of the analysis of important abnormal KPIs and preclude the interference of redundant alerts; other KPIs unrelated to the fault can also be excluded. At the same time, correlation analysis can reduce the misstatement of abnormal KPIs after anomaly detection, and improve the accuracy of anomaly detection results. At present, most anomaly detection methods based on correlation analysis use baseline methods such as Pearson or Granger to calculate raw KPIs. However, these baseline methods can only obtain the correlation value. They do not take into account important factors such as the shift value between KPIs along the time axis and the order of influence, which often causes the misjudgment of abnormal KPIs, reducing the accuracy of the correlation analysis. Meanwhile, the volatility of raw KPIs will affect correlation analysis results at abnormal points between indicators [2]. At present, most of the time series anomaly detection methods for KPIs are concentrated in traditional machine learning and deep learning. The processing speed and accuracy of traditional machine learning and deep learning for KPI time series data are ideal. Some anomaly detection methods based on supervised learning [3; 4] can perform fast and accurate anomaly detection by relying on a large number of types of anomaly-labeled data, but they are not suitable for the actual operation and maintenance environment, which contains fewer anomalies. The new unsupervised and semi-supervised learning anomaly detection methods [5; 6; 7; 8] can better adapt to the actual operation and maintenance environment. Some models use RNN and LSTM to analyze time series data, but RNN and LSTM have problems, such as error accumulation and the need for a lot of training memory, which leads to some false positives and false negatives in anomaly detection. Most existing anomaly detection methods are for large-scale anomaly detection in all the KPI timing data in the system.