# Unsupervised Deep Anomaly Detection for Industrial Multivariate Time Series Data
## 3 Materials and Methods
### Inference
This decrease is more pronounced in the WADI and MSDS datasets, with reductions of 8.5% and 12.7%, respectively, emphasizing the effectiveness and necessity of our composite model for anomaly detection in multivariate time series data. Compared with the original AT-DCAEP model, removing the external attention component (\(AT-DCAEP_{w/o\_LAT}\) ) results in a 7.6% decrease in the \(F1\) score for the WADI dataset and a 19% decrease for the MBA dataset, highlighting the critical role of external attention in the model. Similarly, removing the multi-head attention for low-dimensional feature extraction significantly reduces performance on most datasets, indicating the effectiveness of multi-head attention in capturing the relevance of low-dimensional features. ### Overhead Analysis

The computational cost of the model is also a crucial evaluation criterion. An excellent algorithm for large-scale data should efficiently detect anomalies. To compare the computational cost and performance of these benchmark algorithms with our proposed method, we examined the time each model spent training for one epoch on different \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multicolumn{2}{c}{**WADI**} & \multicolumn{2}{c}{**MSDS**} & \multicolumn{2}{c}{**MBA**} \\ \cline{2-7}  & \(AUC\) & \(F1\) & \(AUC\) & \(F1\) & \(AUC\) & \(F1\) \\ \hline \(AT-DCAEP_{w/o\_Pre}\) & 0.8257 & 0.7393 & 0.5163 & 0.8433 & 0.9545 & 0.9183 \\ \(AT-DCAEP_{w/o\_LSTM}\) & 0.8257 & 0.7425 & 0.9577 & 0.9613 & 0.9978 & 0.9957 \\ \(AT-DCAEP_{w/o\_LAT}\) & 0.8259 & 0.7467 & 0.9591 & 0.9630 & 0.8383 & 0.8066 \\ \(AT-DCAEP_{w/o\_LDFF}\) & 0.9120 & 0.8063 & 0.9577 & 0.9613 & 0.9979 & 0.9959 \\ \(AT-DCAEP_{w/o\_MAT}\) & 0.9105 & 0.7574 & 0.7461 & 0.9106 & 0.9708 & 0.9461 \\ \(AT-DCAEP\) & 0.9121 & 0.8082 & 0.9609 & 0.9658 & 0.9980 & 0.9961 \\ \hline \multirow{2}{*}{**Method**} & \multicolumn{2}{c}{**SMAP**} & \multicolumn{2}{c}{**SMD**} & \multicolumn{2}{c}{**SWaT**} \\ \cline{2-7}  & \(AUC\) & \(F1\) & \(AUC\) & \(F1\) & \(AUC\) & \(F1\) \\ \hline \(AT-DCAEP_{w/o\_Pre}\) & 0.9933 & 0.9355 & 0.9982 & 0.9945 & 0.8425 & 0.8061 \\ \(AT-DCAEP_{w/o\_LSTM}\) & 0.9939 & 0.9408 & 0.9983 & 0.9956 & 0.8424 & 0.8054 \\ \(AT-DCAEP_{w/o\_LAT}\) & 0.9938 & 0.9396 & 0.9983 & 0.9956 & 0.8451 & 0.8035 \\ \(AT-DCAEP_{w/o\_LDFF}\) & 0.9938 & 0.9402 & 0.9983 & 0.9956 & 0.8423 & 0.8047 \\ \(AT-DCAEP_{w/o\_MAT}\) & 0.9938 & 0.9402 & 0.9982 & 0.9942 & 0.8423 & 0.8047 \\ \(AT-DCAEP\) & 0.9940 & 0.9420 & 0.9987 & 0.9960 & 0.8383 & 0.8066 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Ablation study of AT-DCAEP and its ablated versions with \(AUC\) and \(F1\) scores. datasets. Additionally, we calculated the average training time and average \(F1\) score over the six datasets. Figure 7 illustrates the computational cost and average \(F1\) score for each algorithm. Although our proposed method has a slightly higher average computational cost than TranAD, it is significantly lower than that of other methods. Moreover, our method achieves the highest average \(F1\) score, with an 8.79% improvement compared to TranAD. ### Sensitivity Analysis

Figure 8 analyzes the sensitivity to training set size, showing the average \(F1\) scores and _AUC_ scores on different datasets as the training data ratio varies. It also presents the change in training time as the training data ratio ranges from 20% to 100%. From the figure, it is evident that our proposed AT-DCAEP method achieves a significant performance improvement when the training dataset ratio reaches 60%. This strongly validates the excellence of our method in scenarios with small datasets. As the training data size increases, predictive performance shows an upward trend, accompanied by an increase in training time. Additionally, we systematically tested the sensitivity of our proposed method to different window sizes, considering window sizes of [4, 8, 12, 16, 20, 24].