# Deep Learning-Based ECG Arrhythmia Classification: A Systematic Review
## 3 Results
### DL Methodology
#### 3.3.1 Model
Furthermore, the improved RNN, i.e., the long short-term memory (LSTM), gains higher popularity than the conventional RNN because of its higher capability to analyze time series. Specifically, the LSTM has three gate structures to control the output, input, and forget information flow in stored memory cells [57]. Compared to the RNN, the LSTM could deal with longer signal sequences as it selectively acquires useful information from historical inputs. In [58], a 6-layer LSTM is developed to automatically identify PVC beats based on ECG sequences. Furthermore, bidirectional LSTM (BiLSTM) is a special type of LSTM consisting of two LSTMs that go through the input sequence along the temporal direction forwardly and reversely, respectively [32]. Hence, it could capture both the causal and noncausal time dependency information of signals to pursue potential better classification performance. In [59], the BiLSTM model is used for ECG classification based on the extracted ECG wave statistics along the temporal dimension, including RR interval, QR interval, ST segment starting point, and amplitudes of Q- and R-waves. In [60], a 2 DBiLSTM is used for AF detection based on the spectrogram of ECG signals, where the input features are the frequency components at each time instance. In [61], the BiLSTM taking the sequence of RR intervals as input, is proposed for AF detection. To summarize, the input sequences for RNNs can be the raw ECG sequences, time-varying wave statistics, and time-frequency representation of ECG. * Transformer  
The attention mechanism gained more popularity in recent DL research communities as it is capable of learning how to assign higher learning weights to significant features [62]. The transformer is an encoder-decoder structure that consists of only attention mechanisms and fully connected layers [63]. It was originally designed for natural language processing (NLP) but has been extended to other applications since it could achieve better performance than RNN/LSTM [64]. In [65], the encoder part of the transformer is used for heartbeat classification with ECG signals where the heart beat sequences are considered as input. In addition, RR intervals are concatenated with the features extracted by the attention module for final classification. In [66], a random forest model is first used to select 22 important features, such as RR interval median and P-wave correlation coefficients. Then the encoder of the transformer is exploited to extract features directly from ECG signals. The combination of the hand-crafted features and the features automatically extracted by the transformer is used for ECG classification. A waveform transformer is proposed in [67] in which the input ECG segments are first projected to a 1D vector through a multi-layer perceptron. Then the embedded segments, together with positional embedding and learnable class embedding, are taken as the input for the transformer encoder. The extracted features from the transformer are combined with 22 static features together for the final ECG classification. The transformer was developed in 2017, and its application to ECG signal is still in its early stage; however, more results with the transformer are expected in the future. * Hybrid DL model  
Many selected studies consider integrating multiple DL models into one DL network for ECG arrhythmia classification. For example, in [68], it combines the CNN and the RNN to form an encoder-decoder structure for heartbeat classification. CNN is used for feature extraction, and RNNs are used to translate the extracted features to their corresponding categories. More examples of the combination of CNN with LSTM and BiLSTM can be seen in [69; 70; 71], where CNNs are stacked in front of LSTM/BiLSTM modules for feature extraction. In [63], 1D CNN is first used to extract the features from ECG sequences. Then the CNN features are added with the positional encoding to further serve as the input for a transformer to finally detect the ECG arrhythmia.