# Unsupervised Deep Anomaly Detection for Industrial Multivariate Time Series Data
## 3 Materials and Methods
### Inference
### Evaluation Metrics

We employ Precision (\(P\)), Recall (\(R\)), Receiver Operating Characteristic/Area Under the Curve (\(ROC\)/\(AUC\)), and \(F1\) score (\(F1\)) to assess the performance of anomaly detection:

\[P=\frac{TP}{TP+FP},\ \ R=\frac{TP}{TP+FN},\ \ F1=2\cdot\frac{P\cdot R}{P+R}, \tag{21}\]

where \(TP\) represents true positives, \(FP\) is false positives, and \(FN\) is false negatives. \(P\) and \(R\) signify precision and recall, respectively. A higher \(F1\) score indicates better performance. We partition 20% of the original testing dataset as the validation set and the remaining 80% as the testing set. After completing all model training, we measured the \(AUC\) and \(F1\) scores on the test dataset. ### Results

We conducted comparative experiments with various baseline methods, including DAGMM [21], MAD-GAN [22], OmniAnomaly [23], CAE-M [24], MSCRED [25], USAD [26], \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Dataset** & **Train** & **Test** & **Dimensions** & **Anomalies (\%)** \\ \hline MSDS & 146,430 & 146,430 & 10 & 5.37 \\ SMD & 708,405 & 708,420 & 38 & 4.16 \\ WADI & 1,048,571 & 172,801 & 123 & 5.99 \\ SWaT & 496,800 & 449,919 & 51 & 11.98 \\ SMAP & 135,183 & 427,617 & 25 & 13.13 \\ MBA & 100,000 & 100,000 & 2 & 0.14 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Benchmarked datasets. GDN [27], TranAD [28], MERLIN [45], and MTAD-GAT [46]. We evaluated these methods on six different datasets, measuring their performance through comparisons of precision, recall, \(AUC\), and \(F1\) scores. For a direct comparison with the benchmarks proposed by TranAD, we adopted their evaluation methodology. As shown in Table 2, on average, the AT-DCAEP model achieved an average \(F1\) score of 0.9191 across the six datasets. While it performed slightly poorer on the SWaT dataset compared to other methods, it exhibited significantly superior \(F1\) scores on the remaining five datasets. Due to the longer sequence length and complex data patterns in the WADI dataset, all models showed relatively inferior performance on this dataset. However, our proposed approach demonstrated remarkable effectiveness on the WADI dataset compared to other methods. Specifically, the AT-DCAEP outperformed state-of-the-art baseline models with a 5.68% increase in the average \(F1\) score across the six datasets. These results indicate the significant advantages of the AT-DCAEP in multivariate time series anomaly detection tasks. ### Hyperparameter Experiments

To explore the impact of the number of stacked convolutional layers (denoted as N) on the model's performance, we experimented with different values of N. By comparing the model's \(AUC\) and \(F1\) scores when N = 2 and N = 3, we observed that stacking more convolutional layers did not necessarily lead to better performance. The specific results are presented in Figure 3. In the case of N = 3, both \(AUC\) and \(F1\) scores showed a slight decrease across the six datasets. This suggests that increasing the number of convolutional layers does not always significantly improve model performance and may even have a negative impact.