# Unsupervised Transformer-Based Anomaly Detection in ECG Signals
## 4 Results and Discussion
### MIT-BIH Arrhythmia Dataset Results  
Examples of abnormal and normal heartbeats, together with their corresponding predictions, are shown in Figure 8. Normal heartbeats appear to be rather accurately predicted by the model, as shown in Figure 8b. There is little doubt that the model has picked up on the basic morphological behavior of a normal cardiac cycle pattern. The model can adjust the output to the input as much as the normal latent space allows, while continuously aiming to minimize prediction loss. The model attempts to forecast the input using only the normal properties of the cardiac cycle so abnormal heartbeats are mapped onto a normal heartbeat latent space. As expected, this results in lower prediction quality and higher prediction loss for abnormal heartbeats, as shown in Figure 8a. As shown in Table 5, we trained multiple transformer-encoder networks to determine the best number of attention-feedforward blocks (1 or 2), embedding size (512 or 1024 neurons), and number of attention heads (16 or 32). As can be seen in Table 5 the transformer-encoder anomaly detection results for the MIT-BIH t \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Model** & **S/U** & **Accuracy** & **Recall** & **Precision** & **F1-Score** \\ \hline Hierarchical [41] & U & 95.5\% & 94.6\% & 95.8\% & 94.6\% \\ Spectral [41] & U & 95.8\% & 95.1\% & 94.7\% & 94.7\% \\ VRAE + Wasserstein [41] & U & 95.1\% & 94.6\% & 94.6\% & 94.6\% \\ VRAE + k-Means [41] & U & 95.9\% & 95.3\% & 95.4\% & 95.2\% \\ VAE [42] & U + S & 96.8\% & — & — & 95.7\% \\ VAE [43] & S & 95.2\% & 92.5\% & 98.4\% & 95.4\% \\ AE-Without-Attention [43] & S & 97\% & 95.5\% & 98.8\% & 97.1\% \\ CAT-AE [43] & S & 97.2\% & 95.6\% & 99.2\% & 97.4\% \\ LSTM AE [44] & U & 97.93\% & — & — & — \\ This work & U & 99\% & 98.1\% & 100\% & 99\% \\ \hline \hline \end{tabular}  
For evidence, symbols (S,U) denote the supervised and unsupervised learning, respectively.  
\end{table} 
Table 4: Results of the ECG5000 test dataset. Figure 8: (**a**) Anomalous heartbeat (blue) with corresponding predicted heartbeat (orange); (**b**) normal heartbeat (blue) with corresponding predicted heartbeat (orange). models were close together. However, the best model configurations contained two blocks, 64 embedding sizes, and 32 attention heads. Our best-performing transformer-encoder model achieved an accuracy of 89.5%, F1 score of 92.3%, recall of 98.2%, and precision of 87.1%. Figure 9 presents the ROC curve of the best model on the testing set with an AUC score of 93% which means that the model is performing relatively well in distinguishing between anomaly and normal signals. The calculated threshold value was 0.12. Comparing our results with other existing deep learning anomaly detection models that used the MIT-BIH dataset is shown in Table 6. In [26], the authors used stacked LSTM and applied unsupervised training, since no anomaly classes are needed for training the model. The dataset splitting was 80% training and 20% testing. Therefore, we also compare our model with theirs because they followed the same training procedure and data splitting. Our model achieved an F1 of 92.3%, recall of 98.2% and precision of 87.1% while they reported an F1 of 81%, recall of 87% and precision of 82% which indicates that our model outperforms the unsupervised LSTM attempts. Furthermore, in [45] the authors used a novel hybrid architecture consisting of LSTM cells and multi-layer perceptrons (MLP), and the dataset splitting was 70% training and 30% testing. They achieved 87% F1 and 75% sensitivity in a supervised manner, which is also less than our result, while the higher accuracy may be attributed to the fact that the authors have used supervised learning. Finally, the authors in [42] used unsupervised varia