# Time Series Anomaly Detection for KPIs Based on Correlation Analysis and HMM
## 4 Experiment
### Abnormal Screening of KPIs Based on 1D-CNN-TCN  
#### 4.1.1 Data Set  
The data were provided by the 2021 International AIOps Challenge. The number of anomalies in the real system environment is very small, and there is a lack of abnormal samples available to evaluate the anomaly detection method. Therefore, to evaluate the relevant model in detail, the competition data are injected into the real environment of large commercial banks by replaying the real fault type and injecting it in batches. This paper uses the first and second batches of data in the pre-match stage. The first batch of data has no anomalies, and the second batch contains abnormal KPIs that inject faults. In this experiment, the KPI data that do not contain anomalies for two days are used as the input of the 1D-CNN-TCN model, and the corresponding prediction data are obtained as the output. The residuals of the abnormal KPI and prediction data injected in the next two days are calculated to obtain the corresponding residual sequence, and this residual sequence is screened for abnormal KPIs. This experiment selects some KPIs from the metric.csv file provided by the competition for illustration, with the KPI names shown in Table 1. In Table 1, the \"system\" section shows that the experimental data are derived from the b system in the AIOps competition. \"kpi_name\" shows the specific name of the KPI used in the experiment. \"cmbdid\" shows the object of the KPI index, namely, the service node. The 2021 AIOps competition data were generated based on a simulation of large commercial banks, and include the metric data file, the trace data file, the log data file, and the specific description of the injected fault. The KPI data used in this paper are from the metric file in the competition data set. In the metric file, the time stamp, service node name, KPI name, and the specific values of each KPI under different timestamps are given. An example of the format of the monitoring index data is shown in Table 2. In the fault label file, the specific fault time, duration, fault category, fault content, fault service node, and specific root cause index of the injected fault are described in detail. Due to the massive content of the fault label file, the relevant information on fault time, duration, service node, and specific root cause index of the fault label of the two periods addressed in this paper are selected for display. These details can be seen in Table 3. \begin{table}
\begin{tabular}{c c c} \hline \hline
**System** & **kpi_Name** & **cmbdid** \\ \hline system-b & OSlinux-OSLINux NETWORK NETWORK TCP-FIN-WAIT & Tomcat02 \\ system-b & OSlinux-OSlinux\_NETWORK NETWORK TCP-CLOSE-WAIT & Tomcat02 \\ system-b & OSlinux-CPU\_CPU-0\_SingleCpuUtil & MG02 \\ system-b & OSlinux-CPU\_CPU\_SingleCpuil & MG02 \\ system-b & OSlinux-CPU\_CPU-1\_SingleCpuil & MG02 \\ system-b & OSlinux-CPU\_CPU-1\_SingleCpuUtil & MG02 \\ system-b & OSlinux-CPU\_CPU, CPUUUserTime & MG02 \\ system-b & OSlinux-CPU\_CPUCPUULoad & MG02 \\ system-b & OSlinux-CPU\_CPU\_CPUUidelutil & MG02 \\ system-b & OSlinux-CPU\_CPU\_CPUCPUUil & MG02 \\ system-b & OSlinux-QC\_ENinux\_LOCALDISK LOCALDISK\_sdb\_DSKPCrentBusy & Tomcat03 \\ system-b & OSlinux-OCADISK LOCALDISK\_ACADISK-sdb\_DSKPCrentBusy & Tomcat03 \\ system-b & OSlinux-OSlinux\_LOCALDISK LOCALDISK\_sdb\_DSKWTPs & Tomcat03 \\ system-b & OSlinux-OSlinux\_LOCALDISK\_LOCALDISK-sdb\_DSKReadWrite & Tomcat03 \\ system-b & OSlinux-OSlinux\_LOCALDISK\_LOCALDISK-sdb\_DSKWrite & Tomcat03 \\ system-b & OSlinux-OSlinux\_LOCALDISK\_LOCALDISK-sdb\_DSKBytes & Tomcat03 \\ system-b & OSlinux-CPU\_CPU\_CPUUidelutil & Tomcat03 \\ system-b & OSlinux-QC\_ENinux\_MEMORY\_MEMORY\_MEMPerc & Tomcat03 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Information about KPIs used in experiments. “System” represents the system wherein each KPI is located, “kpi_name” shows the KPI name, and “cmbdid” shows the service node whereat each KPI is located.