# Unsupervised Deep Anomaly Detection for Industrial Multivariate Time Series Data
## 3 Materials and Methods
### Inference
### Visualizing Anomaly Results

Regarding the effectiveness of anomaly detection, taking the SMD 1-1 sensor dataset as an example, we illustrate the fluctuation of real values for variables 1 and 12 on the test set and their corresponding anomaly scores at each timestamp. Figure 4 indicates that the model performs well on the testing set, effectively expressing normal data. However, for anomalous data, the model struggles to represent them accurately, leading to higher anomaly scores. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \(\alpha\) & \(\beta\) & \(\mathbf{P}\) & \(\mathbf{R}\) & \(\boldsymbol{AUC}\) & \(\boldsymbol{F1}\) \\ \hline
0.1 & 0.9 & 0.6968 & 0.8295 & 0.9105 & 0.7574 \\
0.2 & 0.8 & 0.7660 & 0.8295 & 0.9117 & 0.7966 \\
0.3 & 0.7 & 0.7880 & 0.8295 & 0.9121 & 0.8082 \\
0.4 & 0.6 & 0.7791 & 0.6541 & 0.8248 & 0.7111 \\
0.5 & 0.5 & 0.7981 & 0.6541 & 0.8251 & 0.7190 \\
0.6 & 0.4 & 0.8233 & 0.6541 & 0.8254 & 0.7290 \\
0.7 & 0.3 & 0.8585 & 0.6541 & 0.8257 & 0.7425 \\
0.8 & 0.2 & 0.8758 & 0.6541 & 0.8259 & 0.7489 \\
0.9 & 0.1 & 0.8938 & 0.6541 & 0.8261 & 0.7554 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Anomaly detection results on the WADI dataset based on different sensitivities. Figure 4: Visualization of the AT-DCAEP model on the SMD dataset. To vividly showcase the effectiveness of our proposed anomaly detection method, we also visualize its performance on the MBA dataset. In Figure 5, subplots 1-4 and 5-8, respectively, display the real time series data, reconstruction scores, prediction scores, and anomaly scores for variables 1 and 2 in the MBA dataset. In Figure 6, subplots 1-4 present the real time series data, reconstruction scores, prediction scores, and anomaly scores. The blue area represents true anomaly regions, while the pink area indicates predicted anomaly regions. Anomalous behavior in one variable in multivariate time series data may trigger a series of events, leading to anomalies in other variables. The AT-DCAEP accurately localizes anomalous behavior to specific variables. As observed in the figure, our method accurately identifies most anomalous states in variables 1 and 2. Additionally, we find that combining reconstruction scores and prediction scores effectively enhances anomaly detection performance. Through the visual presentation of anomaly detection results, the effectiveness of our method in multivariate time series data anomaly detection is clearly demonstrated, making a significant contribution to the advancement of the field. Figure 5: Variable-wise visualization on MBA dataset. Figure 6: Comprehensive visualization on MBA dataset. ## 5 Analysis

### Ablation Study

To investigate the relative importance of each component in the model, we conducted ablation experiments, observing the change in model performance after removing each major component. The evaluation was based on \(AUC\) and \(F1\) scores. Specifically, we considered the following scenarios:

* \(AT-DCAEP_{w/o\_Pre}\): removal of the prediction network, retaining only the characterization network for computing reconstruction error;
* \(AT-DCAEP_{w/o\_LSTM}\): removal of the LSTM component from the prediction network;
* \(AT-DCAEP_{w/o\_LAT}\): removal of the external attention component from the prediction network;
* \(AT-DCAEP_{w/o\_LDFF}\): removal of low-dimensional feature fusion, i.e., no fusion of low-dimensional features extracted by the characterization network in the prediction network;
* \(AT-DCAEP_{w/o\_MAT}\): removal of the multi-head attention module for low-dimensional feature extraction in the characterization network. The experimental results are summarized in Table 5, indicating a corresponding performance decrease in \(F1\) scores when different components are removed. Specifically, the AT-DCAEP model without the prediction model experiences an average decrease of 5% in \(F1\) scores.