## 4 Experiments
### Results and Analysis  
We compared the performance of MTAD-GAN with six popular anomaly detection methods, including: **PCA:** The method is based on Principal Component Analysis [50]; **RawSignal**: Raw Signal is a trivial model that reconstructs any signal to 0, so that the error is the same as the normalized signals; **UAE:** The method is based on Unimodal Autoencoder [51]; **LSTM-ED:** The method is based on Long Short Term Memory neural network and Encoder-Decoder [52]; **AE:** The method is based on Autoencoder [53]; **TenED:** The method is based on Temporal Convolutional Network and Encoder-Decoder [54].  
Figure 6 shows the performance comparison of this method with PCA, RawSignal, UAE, LSTM-ED, AE and TcnED on SMAP, SMD, DAMADICS, SKAB, MSL and J10031 datasets based on different scoring functions. The experimental results show that the choice of detection model and scoring function has different effects on anomaly detection. Compared with other algorithms, the performance of MTAD-GAN is improved by 0.01\(\sim\)0.4, while ADGS integrates multiple loss items to better identify abnormal situations. Compared with other scoring functions, the model performance is improved by 0.16\(\sim\)0.4.  
Table 5 shows the prediction results based on the ADGS scoring function in the J10031 dataset. It can be seen that the accuracy of MTAD-GAN is improved by 2.6% compared with TcnED algorithm, indicating that the proposed model has good performance. To compare and analyze the differences between the methods in this paper and TcnED, Table 6 shows the accuracy on SMAP, SMD, DAMADICS, SKAB, MSL, SWAT and WADI datasets. It can be seen that the accuracy of MTAD-GAN algorithm is improved by 6.42\(\sim\)15.42% compared with TcnED algorithm.  
To verify the model's ability for detecting abnormal data, the SWAT, WADI, MSL and J10031 datasets are evaluated for abnormal scores. It can be seen from Figure 7 that there is a significant difference between normal and abnormal distribution in the scores, and the abnormal score is higher than the normal score with the average score difference between 0.01 and 9.21, which proves that the MTAD-GAN model can reflect the characteristic distribution of data and give more accurate abnormal scores.  
We introduce a scoring network to analyze the sensitivity of different weight parameters of MTAD-GAN and optimize the model. The parameters in Equation (22) are divided into two groups, namely \((\varepsilon,a)\) and \((\lambda_{GS},\lambda_{a})\). Figure 8 shows the AUCROC indexes used to measure the sensitivity of the two groups of parameters on the MSL and J10031 data sets respectively. It can be seen that different data sets have different sensitivity to parameters. On MSL dataset, when \(a<20\) and \(\lambda_{a}>4\), the correlation of parameter is strong. On the J10031 dataset, when \(a<25\) and \(\lambda_{a}>12\), the model performance is relatively stable.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Dataset** & **TcnED** & **MTAD-GAN** \\ \hline SMAP & 76.64 & 88.46 \\ SMD & 80.46 & 86.88 \\ DAMADICS & 80.28 & 87.46 \\ SKAB & 78.52 & 86.59 \\ MSL & 80.21 & 88.62 \\ KDD99 & 85.72 & 94.57 \\ SWAT & 89.44 & 97.15 \\ WADI & 88.35 & 98.54 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Performance comparison of the accuracy of different datasets.  
Figure 6: AUCROC comparison of each model method.