# Time Series Anomaly Detection for KPIs Based on Correlation Analysis and HMM
## 4 Experiment
### Abnormal Screening of KPIs Based on 1D-CNN-TCN
#### 4.1.3 Experimental Comparison  
(1) Ablation experiment  
In this paper, 1D-CNN-TCN is compared with models using only 1D CNN and only TCN for ablation experiments. The evaluation results using the 2021 AIOPs Challenge data show that 1D-CNN-TCN had a better effect than 1D CNN and TCN. We used RMSE, MAE, and MAPE to evaluate these three methods. The performance comparison of the three prediction methods used on this dataset is shown in Table 5, and the best results are marked in bold.  
In Table 5, we can see that the 1D-CNN-TCN prediction method achieves the best performance among the three methods. 1D CNN alone cannot effectively capture the dependencies between time series, and only TCN can derive all the time series features, but its range is too large. Therefore, 1D-CNN-TCN first uses 1D CNN to obtain the local features of the input data, and then it uses TCN to obtain the time-dependent features, meaning the prediction results will be more accurate.  
1D CNN is commonly used in 2D image data processing, and its time complexity is expressed as \(O(k\cdot n\cdot d^{2})\), where \(k\) is the convolution kernel size, \(n\) is the sequence length, and \(d\) is the dimension [35]. Since 1D CNN cannot effectively obtain temporal dependencies, its time series prediction results are not ideal. In contrast, TCN combines one-dimensional full convolution and causal convolution, and also uses residual network and dilated convolution, so its time complexity will be higher. However, TCN can derive the long-term temporal dependencies of time series data, and there is no problem of error accumulation. Its performance is significantly improved compared with LSTM and 1D CNN. In addition, this paper adds a layer of 1D CNN to obtain local features based on TCN, which can reduce the analysis of some secondary features and improve the efficiency of TCN processing. At the same time, in this paper, 1D-CNN-TCN achieves the fastest convergence rate compared with 1D CNN, which can make up for the problem of its high time complexity.  
Figure 6: MAE under different kernel sizes.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline  & **RMSE** & **MAE** & **MAPE** \\ \hline
1D CNN only & 3.8277 & 3.0511 & 12.1083 \\ TCN only & 3.0243 & 2.2045 & 8.2456 \\ A novel LSTM [33] & 3.5280 & 2.5353 & 10.5672 \\ CNN-LSTM [34] & 2.5629 & 2.6272 & 8.1959 \\
1D-CNN-TCN & 2.2100 & 2.2004 & 7.9307 \\ \hline \hline \end{tabular}
\end{table}
Table 5: The proposed 1D-CNN-TCN prediction model is compared with related models. The table shows the evaluation results of the 1D-CNN-TCN, 1D CNN, and TCN ablation experiments using RMSE, MAE, and MAPE, and the evaluation results of 1D-CNN-TCN compared with a novel LSTM [33] and CNN-LSTM [34] using RMSE, MAE, and MAPE.  
Results and Analysis  
We also compare 1D-CNN-TCN with novel LSTM [34] and CNN-LSTM [35] models, and the results are shown in Table 5. The novel LSTM consists of an LSTM network and a fully connected layer. The ability of LSTM to obtain longer time dependence is poor, so its experimental results are poor. CNN-LSTM uses LSTM instead of the TCN used in the 1D-CNN-TCN method. From the experiments, we can see that the TCN achieves better results for longer time series data prediction, while 1D CNN can capture local features, which is beneficial to the prediction effect of 1D-CNN-TCN.  
Through experiments, we found that when only using CNN to predict, the time is the shortest, but the effectivity of CNN is far less than that of the novel LSTM and other time series prediction methods. The running time of 1D-CNN-TCN in this paper mainly depends on the running time of TCN. In general, the running time of 1D-CNN-TCN is longer than that of the novel LSTM, but there is no significant difference. This is because LSTM requires much training memory to increase the running time. However, the convergence rate of 1D-CNN-TCN in this paper is the fastest, and the ideal prediction effect can be achieved with fewer iterations.