# TGAN-AD: Transformer-Based GAN for Anomaly Detection of Time Series Data
## 6 Experiment
### Performance and Analysis  
To answer **Q1**, the proposed model was used in an experiment with four baseline methods on three publicly available real-world datasets with ground truth labeled anomalies. Their performance on the metrics was recorded. The goal of anomaly detection is to detect anomalies as completely as possible, and we place more emphasis on the performance of the model on the recall metric when conducting experiments and model evaluation. Therefore, while ensuring higher F1 scores, this paper primarily uses recall as a performance measure for the model. Table 2 shows the performance of anomaly detection for three datasets by five methods, including six baseline methods. Bolded items in the table indicate the highest values of metrics in each dataset. * For the SWaT dataset, TGAN-AD has excellent performance in _PRAUC_, _Recall_, and _F1-Score_. The _AUC_ and _Precision_ are second only to FNN. However, in conditions where _Recall_ reaches an impressive 99%, the comprehensive indicator _F1-Score_ has a significant improvement compared to other methods. It scored 10.6% higher than the second best baseline method. Apart from the _AUC_ of 0.896, the other four metrics all reached more than 90%, of which the _Recall_ reached 99.0%, which was higher than the MAD-GAN by 8.1%. * For the WADI dataset, most of the metrics are the best among all methods. This dataset, as we know, is unbalanced and has higher dimensionality than other datasets. However, TGAN-AD performs as well on this dataset as on any other dataset. In addition, the anomaly detection performance of TGAN-AD is significantly higher than that of FNN, LSTM, MAD-GAN, and GDN, all of which are deep learning methods. * For the KDDCUP99 dataset, TGAN-AD has excellent performance in all five metrics. All five metric values of TGAN-AD reached the highest among all methods. The _F1-Score_ is 7% higher than the second best baseline method. Except for the recall rate of 87.5, the other four values all reached 90%. Moreover, _PRAUC_, _Recall_, and _F1-Score_ maintain the highest levels across all datasets. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Datasets** & **Methods** & **AUC** & **PRAUC** & **Precision** & **Recall** & **F1** \\ \hline \multirow{8}{*}{SWaT} & PCA & 0.771 & 0.716 & 57.1 & 80.0 & 0.667 \\  & Random Forests & 0.808 & 0.792 & 57.1 & 96.6 & 0.718 \\  & LSTM & 0.866 & 0.640 & 41.1 & 80.8 & 0.545 \\  & FNN & **0.969** & 0.963 & **96.2** & 75.8 & 0.847 \\  & MAD-GAN & 0.750 & 0.917 & 19.6 & 90.9 & 0.323 \\  & GDN & 0.898 & 0.696 & 94.9 & 64.2 & 0.766 \\  & TGAN-AD & 0.896 & **0.992** & 91.8 & **99.0** & **0.953** \\ \hline \multirow{8}{*}{WADI} & PCA & 0.772 & 0.930 & **92.0** & 60.5 & 0.730 \\  & Random Forests & 0.808 & 0.792 & 57.1 & 96.6 & 0.718 \\  & LSTM & 0.842 & 0.497 & 72.4 & 27.9 & 0.404 \\  & FNN & 0.824 & 0.395 & 53.3 & 94.1 & 0.681 \\  & MAD-GAN & 0.506 & 0.526 & 13.6 & 72.7 & 0.229 \\  & GDN & 0.866 & 0.581 & 86.5 & 35.1 & 0.507 \\  & TGAN-AD & **0.924** & **0.985** & 88.1 & **96.7** & **0.922** \\ \hline \multirow{8}{*}{KDDCUP99} & PCA & 0.740 & 0.870 & 92.3 & 46.2 & 0.615 \\  & Random Forests & 0.595 & 0.808 & 88.9 & 18.6 & 0.308 \\ \cline{1-1}  & LSTM & 0.292 & 0.192 & 68.9 & 81.9 & 0.749 \\ \cline{1-1}  & FNN & 0.800 & 0.808 & 93.3 & 66.7 & 0.778 \\ \cline{1-1}  & MAD-GAN & 0.881 & 0.552 & 96.3 & 71.1 & 0.818 \\ \cline{1-1}  & GDN & 0.669 & 0.224 & 85.9 & 85.2 & 0.855 \\ \cline{1-1}  & TGAN-AD & **0.959** & **0.971** & **98.0** & **87.5** & **0.925** \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Anomaly detection metrics for different datasets. The given result fully demonstrates the usage of Transformer which can well represent the data of Generator \(G\) and Discriminator \(D\) and paves the most direct way to the optimal representation of the testing sample which can filter the anomaly data from normal data. We implement our method and its variants on an NVIDIA Tesla T4 graphics card. The models are trained for up to 50 epochs and use early stopping with patience of 10.