# Unsupervised Transformer-Based Anomaly Detection in ECG Signals
## 5 Conclusions  
We have introduced a robust and effective unsupervised transformer anomaly detection model in time series data. The suggested model was utilized to detect anomalies in human heartbeat time series signals, such as premature ventricular contraction (PVC), supraventricular premature (SP), and other ECG anomalies. A transformer encoder network and linear dense decoder networks comprise the model architecture. The ECG time series anomaly detection approach is based on a sequence prediction method that has two stages: model training, in which the model learns the normal data distribution, and anomaly detection, in which the ECG time series anomaly score is computed to identify abnormalities. We adapt the anomaly detection method into an unsupervised framework by using a pre-set threshold of mean plus two times the standard deviation of training loss. We have shown how the transformer encoder can be leveraged for ECG time series anomaly detection. Our results for both the ECG5000 and MIT-BIH Arrhythmia datasets have shown that a transformer encoder is a viable substitute for this task. The transformer encoder was tested against several state-of- the art deep learning models. We have demonstrated that our model outperformed the compared models in accuracy, F1-score, recall, and precision. For the ECG5000 heartbeat dataset, we have achieved a 99% accuracy, a 99% F1 score, a 100% precision, a 98.1% recall, and a 99% AUC score, demonstrating an excellent ability to determine abnormalities in ECG heartbeat signals. Moreover, the model revealed respectable weighted F1 score of 92.3%, accuracy of 89.5%, AUC score of 93%, recall of 98.2%, and precision of 87.1% on the MIT-BIH Arrhythmia dataset. In addition, the proposed model is more robust than other deep learning models in recent ECG analyses using MIT-BIH Arrhythmia dataset. Though this work has focused on comparing the model with deep learning anomaly detection approaches, we aim in future work to expand our investigation on other anomaly detection techniques of time series data. Conceptualization, [PERSON] ([PERSON]) and [PERSON] ([PERSON]); methodology, [PERSON] ([PERSON]); software, [PERSON] ([PERSON]); validation, [PERSON] ([PERSON]) and [PERSON] ([PERSON]); formal analysis, [PERSON] ([PERSON]); investigation, [PERSON] ([PERSON]); writing--original draft preparation, [PERSON] ([PERSON]); writing--review and editing, [PERSON] ([PERSON]); visualization, [PERSON] ([PERSON]); supervision, [PERSON] ([PERSON]); project administration, [PERSON] ([PERSON]) All authors have read and agreed to the published version of the manuscript. This research received no external funding. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Model** & **S/U** & \begin{tabular}{c} **Dataset** \\ **Spliting** \\ \end{tabular} & **F1** & **Accuracy** & \begin{tabular}{c} **Recall** \\ **(Sensitivity)** \\ \end{tabular} & **Precision** \\ \hline Stacked LSTM [26] & U & \begin{tabular}{c} 80\% training, \\ 20\% testing \\ \end{tabular} & 81\% & - & 87\% & 82\% \\ (LSTM) with (MLP) [45] & S & \begin{tabular}{c} 70\% training, \\ 30\% testing \\ \end{tabular} & 87\% & 95\% & 75\% & - \\ VAE [42] & U & \
\begin{tabular}{c} AAMI Dataset \\ splitting \\ \end{tabular} & 76.55\% & 87.77\% & - & - \\ This work & U &
\begin{tabular}{c} 80\% training, \\ 20\% testing \\ \end{tabular} & 92.3\% & 89.5\% & 98.2\% & 87.1\% \\ \hline \hline \end{tabular}  
* For evidence, symbols (S,U) denote the supervised and unsupervised learning, respectively.  
\end{table} 
Table 6: Results of the MIT-BIH Arrhythmia test dataset.