# A Deep Neural Network Ensemble Classifier with Focal Loss for Automatic Arrhythmia Classification

Authors: Han Wu; Senhao Zhang; Benkun Bao; Jiuqiang Li; Yingying Zhang; Donghai Qiu; Hongbo Yang
Journal: Journal of Healthcare Engineering
 (2022)

---

## Abstract

Automated electrocardiogram classification techniques play an important role in assisting physicians in diagnosing arrhythmia. Among these, the automatic classification of single-lead heartbeats has received wider attention due to the urgent need for portable ECG monitoring devices. Although many heartbeat classification studies performed well in intrapatient assessment, they do not perform as well in interpatient assessment. In particular, for supraventricular ectopic heartbeats (S), most models do not classify them well. To solve these challenges, this article provides an automated arrhythmia classification algorithm. There are three key components of the algorithm. First, a new heartbeat segmentation method is used, which improves the algorithm’s capacity to classify S substantially. Second, to overcome the problems created by data imbalance, a combination of traditional sampling and focal loss is applied. Finally, using the interpatient evaluation paradigm, a deep convolutional neural network ensemble classifier is built to perform classification validation. The experimental results show that the overall accuracy of the method is 91.89%, the sensitivity is 85.37%, the positive productivity is 59.51%, and the specificity is 93.15%. In particular, for the supraventricular ectopic heartbeat(s), the method achieved a sensitivity of 80.23%, a positivity of 49.40%, and a specificity of 96.85%, exceeding most existing studies. Even without any manually extracted features or heartbeat preprocessing, the technique achieved high classification performance in the interpatient assessment paradigm.


---

## Body

## 1. Introduction

An electrocardiogram is a sequence that records the electrical activity of the heart [1]. With the increase in the number of heart diseases [2] and the rapid development of computer technology such as deep learning, there is a growing interest in how to use computer technology to aid in the automatic diagnosis of heart diseases [3, 4]. The automated diagnosis of arrhythmias, one of the most common cardiac diseases, has been a popular area of research in computer-aided diagnosis [5]. It consists of two main challenges. The first is the dataset’s high imbalance, with normal beats taking up the majority of the ECG signal. The normal beats (N) account for more than 90% of all beats in the MIT-BIH arrhythmia dataset. Second, the training and testing sets used in the interpatient evaluation paradigm are from different populations, which is more in line with our actual requirements for automatic arrhythmia diagnosis, but the prediction difficulty is greatly increased due to individual variability of different populations.The existing arrhythmia classification algorithms include two main types, depending on the input data. The one is arrhythmia classification based on manual feature extraction, and the other is arrhythmia classification based on automatic feature extraction [5].In the method of manual feature extraction, the manually extracted features [3, 6, 7, 8, 9, 10, 11, 12, 13, 14] mainly include RR interval, short-time Fourier transform, morphological features, empirical modal decomposition, higher-order statistics, and entropy metric. Then machine learning classifiers are adopted to classify the extracted features, including the weighted linear discriminator [15, 16, 17, 18], support vector machine [4, 19, 20, 21, 22, 23], multilayer perceptron [24], and convolutional neural network [25, 26, 27, 28]. Chazal et al. [29] extracted five groups of features, including R-R, HOS, wavelet, morphology, and LBP, from ECG signals and used these features to classify ECG signals by linear classifier. H. Shi et al. [30] extracted six groups of features, including RR interval, morphology, statistics, higher-order statistics, wavelet transform, and wavelet packet entropy, then used a hierarchical XGBoost classifier for classification. Dias et al. [31] manually extracted the RR interval, morphology features, and higher-order features of ECG signals and used an LD classifier to classify arrhythmia. Yang et al. [32] constructed a hybrid kernel-based extreme learning machine to compare different combinations of feature inputs, which yielded the best classification results when ten randomly selected combinations of feature inputs were used in the input. However, these methods rely excessively on manual feature selection, which increases the complexity of the computational process and the time required to extract features upfront.With the rapid growth of deep learning, there is a need to use deep learning for automatic heartbeat classification by simply inputting the raw ECG data and then enabling the deep learning algorithm to learn the features for us and provide the final classification result [33, 34, 35]. Garcia et al. [36] explored a vector ECG-based deep convolutional neural network-based arrhythmia classification method to classify three beats, N, SVEB(S), and VEB(V). Takalo-Mattila et al. [37, 38] used a high-pass filter, band-stop filter, and low-pass filter to remove noise from the ECG signal, then sliced the ECG signal according to the location of the marked R peaks in the database, and finally used a convolutional neural network to classify the four beats of N, S, V, and F. Li et al. [39] used equal time (5 s) slicing of the raw ECG, the discrete wavelet transform for noise removal, and a deep residual convolutional neural network for classification. However, the sensitivity of S was often low in these studies, making it challenging to apply in real-life situations. Sellami et al. [40] used a robust deep convolutional network to classify arrhythmia. The authors created a batch-weighted loss function to alleviate the data imbalance problem and used three different heartbeat input patterns for experimental comparison. In the classification model, S had a high sensitivity, whereas N had a low sensitivity when compared to other algorithms. Niu et al. [41] used the SBCX method to process the input beats, effectively removing the effect of baseline drift noise. At the same time, the authors combined the processed heartbeat signal and RR interval features together as the input data for classification. The classification effect was good, but too much preprocessing was carried out, and the classification process was not automated enough.Based on these problems, this article proposes a new heartbeat segmentation method and constructs a deep neural network ensemble classifier with focal loss. It performs effective arrhythmia classification without any preprocessing using only raw heartbeat data.We present the ECG dataset used and discuss the details of the implementation of the heartbeat classification algorithm in this article in the next section. In Section3, we compare the algorithm suggested in this article to some existing algorithms and conduct ablation studies on the algorithm’s primary components. We conclude this article in Section 4.


## 2. Methods

The overall structure diagram of the proposed classification system is shown in Figure1. The publicly available MIT-BIH arrhythmia dataset is used as the input heartbeat in this article. After heartbeat sampling, the input heartbeat data are segmented using a special heartbeat segmentation method and used as the training set. The deep convolutional neural network ensemble classifier’s training process uses focal loss as the loss function, and the classifier is voted to obtain the final classification results. The method proposed in this paper is validated under the interpatient assessment paradigm.Figure 1
Overall system structure diagram.


### 2.1. ECG Database

The MIT-BIH arrhythmia database, which contains 48 30-minute long records from 47 patients, is used for the raw data. The dataset includes detailed annotations from cardiologists containing the type of heartbeat for each beat and the location of eachR-peak peak [42].For comparison, the ECG dataset is partitioned into DS1 and DS2, as described in [29]. In the interpatient evaluation paradigm, a modified limb lead II (MLII) is used to go as the input signal for the model, using DS1 for training and DS2 for experimental testing. As shown in Table 1, the heartbeat types are classified into five categories according to the American Association for the Advancement of Medical Devices (AAMI). Four records with rhythm are excluded, as suggested by AAMI. The specific division is shown in Table 2.Table 1
AAMI classification of heartbeat types.
ClassSymbolMembersNormalNNormal beat left bundle branch block beat right bundle branch block beat atrial escape beat nodal (junctional) escape beatSupraventricular ectopic beatSVEB (S)Atrial premature beat aberrated atrial premature beat nodal (junctional) premature beat supraventricular premature beatVentricular ectopic beatVEB (V)Premature ventricular contraction ventricular escape beatFusion beatFFusion of ventricular and normal beatUnknown beatQPaced beat fusion of paced and normal beat unclassifiable beatTable 2
Using interpatient division of training set DS1 and test set DS2.
DatasetPatient numbers in the MIT-BIH arrhythmia databaseDS1101, 106, 108, 109, 112, 114, 115, 116, 118, 119, 122, 124, 201, 203, 205, 207, 208, 209, 215, 220, 223, 230DS2100, 103, 105, 111, 113, 117, 121, 123, 200, 202, 210, 212, 213, 214, 219, 221, 222, 228, 231, 232, 233, 234


### 2.2. Segmentation of ECG Signals

Following the database division, each heartbeat record is segmented based on the R-peak annotation location provided by the dataset. Table3 shows the segmentation lengths of the different methods and the sensitivity of the S in the final classification results. It can be seen that most methods have a heartbeat segmentation length of 300 or less. Due to the high degree of similarity between S and N, and the small proportion of S in the overall heartbeats, it is easy to cause overfitting of S during the training process, resulting in low sensitivity of S. Under the interpatient assessment paradigm, Garcia (2017) and Takalo-Mattila (2018) had a classification sensitivity of just around 60% for S [36, 37]. Jinghao Niu (2020) and Haojie Zhang (2021) had higher sensitivity of S (77.35% and 88.24%), respectively, but their training data not only contained the original signal of the heartbeat but also included the manually extracted R-R interval features [39, 41]. After removing the R-R interval features, Haojie Zhang (2021) and Jinghao Niu (2020) both had extremely low sensitivity (38.7% and 8.06%).Table 3
Heartbeat segmentation length and S sensitivity for different methods.
WorkLength of heartbeatSen (%) of SManually extracted featuresGarcia (2017)27061.96NoTakalo-mattila (2018)13062.49NoJinghao Niu (2020)25677.35/38.7Yes/NoHaojie Zhang (2021)25688.24/8.06Yes/NoIn this article, we want the model to be able to classify heartbeats automatically using only the raw heartbeat signal, with no data preprocessing or manual feature input. So, we provide a new heartbeat segmentation strategy. Figure2 shows the difference between the traditional single heartbeat segmentation approach and the heartbeat segmentation approach in this article. Figure 2(a) shows the traditional single heartbeat segmentation method [41]. Figure 2(b) shows the segmentation approach in this article. The length of the heartbeat window in this article is 508, consisting of 250 samples before and 257 samples after the R-peak. Figure 2(c) shows a comparison of N and S in the two segmentation methods. Figure 2(c)(i) and Figure 2(c)(ii) show the N segmented by the conventional 256 sampling points and the 508 sampling points segmentation of this article, respectively, while Figure 2(c)(iii) and Figure 2(c)(iv) show the S segmented by the conventional 256 sampling points and the 508 sampling points segmentation of this article, respectively. Comparing Figure 2(c)(i) and Figure 2(c)(iii), it can be seen that the morphology of N and S is highly similar in the traditional segmentation method, which makes it difficult for the classifier to distinguish N and S. Comparing Figure 2(c)(ii) and Figure 2(c)(iv), it can be seen that the segmentation method in this article increases the window length of the heartbeat, which is conducive to extracting more neighbourhood features from the original signal. The morphological distinctions between N and S are bigger when applying the segmentation method in this research, making it easier for the classifier to distinguish between the two.Figure 2
Diagram of the heartbeat segmentation strategy. (a) The traditional heartbeat segmentation strategy. (b) The heartbeat segmentation strategy proposed in this article. (c)(i) N by the traditional heartbeat segmentation strategy. c(ii) N by the segmentation strategy in this article. c(iii) S by the traditional heartbeat segmentation strategy. c(iv) S by the segmentation strategy in this article.
(a)(b)(c)Table4 shows the proportions of each class of heartbeats after segmentation. To make the classification more automated, this article does not apply any filtering to the segmented heartbeat signals and only inputs the raw heartbeat data for classification. The proportion of heartbeats following segmentation is severely unbalanced, as shown in Table 4. To prevent overfitting in the training, the effect of data imbalance needs to be further weakened.Table 4
Proportion of each class of heartbeats.
SegmentNSVFQDS14582494337854148DS244215183632193887


### 2.3. Overall Structure of the Algorithm and Heartbeat Sampling

Figure3 shows the general structure of the method. Based on the interpatient paradigm, the MIT-BIH arrhythmia database is classified into DS1 and DS2. Then, for N, random sampling is used, and incremental sampling is used for the remaining data classes [43]. The neural network is trained using the obtained samples from DS1. Six focal loss deep neural network classifiers are trained in total, and the final arrhythmia classification result is determined by voting on them.Figure 3
Overall structure of the classification algorithm.Random sampling and incremental sampling in DS1 mitigate the effects of data imbalance. Unlike previous research [37, 39, 40, 41, 44], this publication raises the sampling size for class S to 7544, for class V to 4592, and for class N to be randomly sampled with a size of 11188 and the sizes of F and Q remain unchanged,.There are two main reasons why such a unique approach to data sampling is chosen. Firstly, we want the classifier to focus more on the classification ofS, so S accounts for the largest proportion of abnormal heartbeats in the training set. Secondly, the final result is a vote for each focal loss deep neural network classifier. In the training set of each classifier, only N is a random sample from DS1; the rest of the samples are the same. According to the theory of ensemble learning [45, 46], when the base classifiers have the same classification performance, the higher the independence of each base classifier, the better the overall classification performance. As a result, the highest percentage of N (47.11%) is found in each training set.


### 2.4. Focal Loss Function

After random sampling and incremental sampling of DS1, the impact of data imbalance is mitigated. To further reduce the problem of overfitting, the focal loss function is adopted.The focal loss was proposed by He et al. [47] to address the problem of difficult imbalance in classification among data in dense object detection. The amount of data containing objects in object detection is much smaller than the amount of data without objects, and the difficulty of classifying data without objects is low, which has a very small improvement effect on the model. Focal loss reduces the weight of easily classifiable samples, allowing the model to focus more on the hard-to-classify samples.(1)CE=−logPt,(2)Lf=−at1−PtylogPt.In (1) and (2), Pt represents the probability of the predicted value. Equation (1) is the traditional cross-entropy loss function, where the closer Pt is to 1, the smaller the loss value (CE) is, thus achieving the training purpose by making the total loss value decrease in the training of the model. Equation (2) is the local loss function, compared to the traditional cross-entropy loss function addedat and1−Pty. The proportion of loss value weights for different samples can be adjusted by at, which helps us to alleviate the problems caused by data imbalance. For 1−Pty, the proportion is smaller when Pt is closer to 1. Therefore, we can make the model pay more attention to data with smaller Pt values, thus increasing the model’s attention to the hard-to-classify samples. In this study, such a focus is very important. We can adjust the size of y to make the model more focused on the distinction between N and S.In this article, we choosey = 2.35 and use different weighting ratios for different heartbeat categories. The specific weight ratio is N : S : V : F : Q = 1.6 : 1.8 : 0.8 : 1.0 : 1.0.


### 2.5. Structure of the Deep Residual Convolutional Neural Network

Figure4 shows the classification model structure for each base classifier. The target heartbeats and their category labels comprise the input data.Figure 4
Network structure of the base classifier.The network consists of 9 convolutional layers, each with a convolutional kernel size of 17. The first five convolutional layers include 20 convolutional kernels, while the final four convolutional layers contain 40 convolutional kernels. A batch normalization [48] process is added after each convolutional layer to speed up the training process. The use of “Tanh” after the first convolutional layer and “ReLU” after the other convolutional layers allows the model to better adapt to the data. To prevent overfitting during the deep convolutional neural network training, a 40% dropout is added after 3–9 convolutional layers [49]. To enable the network to be trained at a deeper level, the residual structure proposed by Kaiming He et al. [50] is used. The addition of jump connections helps to weaken the problem of depth information loss. After the last jump connection, batch normalization and “ReLU” processing are again performed. Finally, our base classifier results are obtained after global average pooling and a fully connected layer.


## 2.1. ECG Database

The MIT-BIH arrhythmia database, which contains 48 30-minute long records from 47 patients, is used for the raw data. The dataset includes detailed annotations from cardiologists containing the type of heartbeat for each beat and the location of eachR-peak peak [42].For comparison, the ECG dataset is partitioned into DS1 and DS2, as described in [29]. In the interpatient evaluation paradigm, a modified limb lead II (MLII) is used to go as the input signal for the model, using DS1 for training and DS2 for experimental testing. As shown in Table 1, the heartbeat types are classified into five categories according to the American Association for the Advancement of Medical Devices (AAMI). Four records with rhythm are excluded, as suggested by AAMI. The specific division is shown in Table 2.Table 1
AAMI classification of heartbeat types.
ClassSymbolMembersNormalNNormal beat left bundle branch block beat right bundle branch block beat atrial escape beat nodal (junctional) escape beatSupraventricular ectopic beatSVEB (S)Atrial premature beat aberrated atrial premature beat nodal (junctional) premature beat supraventricular premature beatVentricular ectopic beatVEB (V)Premature ventricular contraction ventricular escape beatFusion beatFFusion of ventricular and normal beatUnknown beatQPaced beat fusion of paced and normal beat unclassifiable beatTable 2
Using interpatient division of training set DS1 and test set DS2.
DatasetPatient numbers in the MIT-BIH arrhythmia databaseDS1101, 106, 108, 109, 112, 114, 115, 116, 118, 119, 122, 124, 201, 203, 205, 207, 208, 209, 215, 220, 223, 230DS2100, 103, 105, 111, 113, 117, 121, 123, 200, 202, 210, 212, 213, 214, 219, 221, 222, 228, 231, 232, 233, 234


## 2.2. Segmentation of ECG Signals

Following the database division, each heartbeat record is segmented based on the R-peak annotation location provided by the dataset. Table3 shows the segmentation lengths of the different methods and the sensitivity of the S in the final classification results. It can be seen that most methods have a heartbeat segmentation length of 300 or less. Due to the high degree of similarity between S and N, and the small proportion of S in the overall heartbeats, it is easy to cause overfitting of S during the training process, resulting in low sensitivity of S. Under the interpatient assessment paradigm, Garcia (2017) and Takalo-Mattila (2018) had a classification sensitivity of just around 60% for S [36, 37]. Jinghao Niu (2020) and Haojie Zhang (2021) had higher sensitivity of S (77.35% and 88.24%), respectively, but their training data not only contained the original signal of the heartbeat but also included the manually extracted R-R interval features [39, 41]. After removing the R-R interval features, Haojie Zhang (2021) and Jinghao Niu (2020) both had extremely low sensitivity (38.7% and 8.06%).Table 3
Heartbeat segmentation length and S sensitivity for different methods.
WorkLength of heartbeatSen (%) of SManually extracted featuresGarcia (2017)27061.96NoTakalo-mattila (2018)13062.49NoJinghao Niu (2020)25677.35/38.7Yes/NoHaojie Zhang (2021)25688.24/8.06Yes/NoIn this article, we want the model to be able to classify heartbeats automatically using only the raw heartbeat signal, with no data preprocessing or manual feature input. So, we provide a new heartbeat segmentation strategy. Figure2 shows the difference between the traditional single heartbeat segmentation approach and the heartbeat segmentation approach in this article. Figure 2(a) shows the traditional single heartbeat segmentation method [41]. Figure 2(b) shows the segmentation approach in this article. The length of the heartbeat window in this article is 508, consisting of 250 samples before and 257 samples after the R-peak. Figure 2(c) shows a comparison of N and S in the two segmentation methods. Figure 2(c)(i) and Figure 2(c)(ii) show the N segmented by the conventional 256 sampling points and the 508 sampling points segmentation of this article, respectively, while Figure 2(c)(iii) and Figure 2(c)(iv) show the S segmented by the conventional 256 sampling points and the 508 sampling points segmentation of this article, respectively. Comparing Figure 2(c)(i) and Figure 2(c)(iii), it can be seen that the morphology of N and S is highly similar in the traditional segmentation method, which makes it difficult for the classifier to distinguish N and S. Comparing Figure 2(c)(ii) and Figure 2(c)(iv), it can be seen that the segmentation method in this article increases the window length of the heartbeat, which is conducive to extracting more neighbourhood features from the original signal. The morphological distinctions between N and S are bigger when applying the segmentation method in this research, making it easier for the classifier to distinguish between the two.Figure 2
Diagram of the heartbeat segmentation strategy. (a) The traditional heartbeat segmentation strategy. (b) The heartbeat segmentation strategy proposed in this article. (c)(i) N by the traditional heartbeat segmentation strategy. c(ii) N by the segmentation strategy in this article. c(iii) S by the traditional heartbeat segmentation strategy. c(iv) S by the segmentation strategy in this article.
(a)(b)(c)Table4 shows the proportions of each class of heartbeats after segmentation. To make the classification more automated, this article does not apply any filtering to the segmented heartbeat signals and only inputs the raw heartbeat data for classification. The proportion of heartbeats following segmentation is severely unbalanced, as shown in Table 4. To prevent overfitting in the training, the effect of data imbalance needs to be further weakened.Table 4
Proportion of each class of heartbeats.
SegmentNSVFQDS14582494337854148DS244215183632193887


## 2.3. Overall Structure of the Algorithm and Heartbeat Sampling

Figure3 shows the general structure of the method. Based on the interpatient paradigm, the MIT-BIH arrhythmia database is classified into DS1 and DS2. Then, for N, random sampling is used, and incremental sampling is used for the remaining data classes [43]. The neural network is trained using the obtained samples from DS1. Six focal loss deep neural network classifiers are trained in total, and the final arrhythmia classification result is determined by voting on them.Figure 3
Overall structure of the classification algorithm.Random sampling and incremental sampling in DS1 mitigate the effects of data imbalance. Unlike previous research [37, 39, 40, 41, 44], this publication raises the sampling size for class S to 7544, for class V to 4592, and for class N to be randomly sampled with a size of 11188 and the sizes of F and Q remain unchanged,.There are two main reasons why such a unique approach to data sampling is chosen. Firstly, we want the classifier to focus more on the classification ofS, so S accounts for the largest proportion of abnormal heartbeats in the training set. Secondly, the final result is a vote for each focal loss deep neural network classifier. In the training set of each classifier, only N is a random sample from DS1; the rest of the samples are the same. According to the theory of ensemble learning [45, 46], when the base classifiers have the same classification performance, the higher the independence of each base classifier, the better the overall classification performance. As a result, the highest percentage of N (47.11%) is found in each training set.


## 2.4. Focal Loss Function

After random sampling and incremental sampling of DS1, the impact of data imbalance is mitigated. To further reduce the problem of overfitting, the focal loss function is adopted.The focal loss was proposed by He et al. [47] to address the problem of difficult imbalance in classification among data in dense object detection. The amount of data containing objects in object detection is much smaller than the amount of data without objects, and the difficulty of classifying data without objects is low, which has a very small improvement effect on the model. Focal loss reduces the weight of easily classifiable samples, allowing the model to focus more on the hard-to-classify samples.(1)CE=−logPt,(2)Lf=−at1−PtylogPt.In (1) and (2), Pt represents the probability of the predicted value. Equation (1) is the traditional cross-entropy loss function, where the closer Pt is to 1, the smaller the loss value (CE) is, thus achieving the training purpose by making the total loss value decrease in the training of the model. Equation (2) is the local loss function, compared to the traditional cross-entropy loss function addedat and1−Pty. The proportion of loss value weights for different samples can be adjusted by at, which helps us to alleviate the problems caused by data imbalance. For 1−Pty, the proportion is smaller when Pt is closer to 1. Therefore, we can make the model pay more attention to data with smaller Pt values, thus increasing the model’s attention to the hard-to-classify samples. In this study, such a focus is very important. We can adjust the size of y to make the model more focused on the distinction between N and S.In this article, we choosey = 2.35 and use different weighting ratios for different heartbeat categories. The specific weight ratio is N : S : V : F : Q = 1.6 : 1.8 : 0.8 : 1.0 : 1.0.


## 2.5. Structure of the Deep Residual Convolutional Neural Network

Figure4 shows the classification model structure for each base classifier. The target heartbeats and their category labels comprise the input data.Figure 4
Network structure of the base classifier.The network consists of 9 convolutional layers, each with a convolutional kernel size of 17. The first five convolutional layers include 20 convolutional kernels, while the final four convolutional layers contain 40 convolutional kernels. A batch normalization [48] process is added after each convolutional layer to speed up the training process. The use of “Tanh” after the first convolutional layer and “ReLU” after the other convolutional layers allows the model to better adapt to the data. To prevent overfitting during the deep convolutional neural network training, a 40% dropout is added after 3–9 convolutional layers [49]. To enable the network to be trained at a deeper level, the residual structure proposed by Kaiming He et al. [50] is used. The addition of jump connections helps to weaken the problem of depth information loss. After the last jump connection, batch normalization and “ReLU” processing are again performed. Finally, our base classifier results are obtained after global average pooling and a fully connected layer.


## 3. Results and Discussion

### 3.1. Performance Metrics

To evaluate the performance of the model, the confusion matrix of the model is given, and four statistical performance metrics are adopted to evaluate our method as a whole according to the guidelines provided by AAMI [29], namely, accuracy (Acc), sensitivity (Sen), positive productivity (+P), and specificity (Spe). The calculation of each metric is defined in the following equations:(3)Acc%=TP+TNTP+FP+TN+FN×100%,(4)Sen%=TPTP+FN×100%,(5)+P%=TPTP+FP×100%,(6)Spe%=TNTN+FP×100%.When calculating the overall model performance metric, TP is defined as the number of correctly classified abnormal heartbeats, TN is defined as the number of normally classified normal heartbeats, FP indicates the number of normal heartbeats classified as abnormal, and FN indicates the number of abnormal signals classified as normal.


### 3.2. Experimental Results and Discussion

The model performance for the six base classifiers and the final ensemble classifier is shown in Table5. The overall accuracy (Acc) of the base classifiers is all above 83%, with the majority exceeding 88%. Most of the sensitivities (Sen) are around 85%. The final classification result is obtained after voting for the six base classifiers.Table 5
Performance metrics of six base classifiers and ensemble model
WorkAcc (%)+P (%)Sen (%)Spe (%)#188.7149.0377.6690.68#288.1847.6683.6189.37#388.6048.8188.3689.78#486.9244.5885.6487.91#589.8052.3684.9691.23#683.5438.1484.9283.91Average87.6346.7684.1988.81Result91.8959.5185.3793.15As shown in Table5, the final ensemble classification results are significantly better than the average metrics of all six classifiers. Accuracy (Acc) increases by 4.26% (87.63% to 91.89%), positive productivity (+P) increases by 12.75% (46.76% to 59.51%), sensitivity increases by 1.18% (84.19% to 85.37%), and specificity (Spe) increases by 4.34% (88.81% to 93.15%). The accuracy (Acc) of the 6th base classifier is low in Table 5. There are two reasons for this. First, neural network training has a chance. Second, in the focal loss function in this article, S has a high weight, which may cause the classifier to overfocus on S during the training process. The confusion matrix of the final classification results is given in Table 6.Table 6
Confusion matrix of the ensemble classifier.
Predicted labelNSVFQTrue labelN41186142753510670S31714734240V133682929890F3101415490Q30400Table7 shows the sensitivity (Sen), positive productivity (+P), and specificity (Spe) for classes N, S, and V and compares them with six classical arrhythmia classification algorithms. Table 8 shows the overall performance of the model in terms of accuracy (Acc), positive productivity (+P), sensitivity (Sen), and specificity (Spe) and compares it with the classical algorithms.Table 7
Classification performance results of our method and 6 advanced methods.
WorkAccClass (N)Class (S)Class (V)Sen (%)+P (%)Spe (%)Sen (%)+P (%)Spe (%)Sen (%)+P (%)Spe (%)DeChazal (2004)85.8899.1686.8694.0038.5375.9495.3581.5977.7498.78Garcia (2017)92.3893.9997.9582.5561.9652.9697.8987.3459.4495.91Takalo-mattila (2018)89.9191.8997.0076.8362.4955.8698.1189.2350.8594.02Sellami (2019)88.3488.5298.891.382.0430.4492.892.0572.1397.54Jinghao Niu (2020)95.8798.2897.3978.6977.3573.2998.9285.0891.7599.47Yuanlu Li (2021)88.9994.5493.3380.835.2265.8898.8388.3579.8694.92Our methods91.8993.1598.1886.0080.2349.4096.8590.9983.0998.72Table 8
Overall model performance metrics for our method and 6 advanced methods.
Acc (%)+P (%)Sen (%)Spe (%)DeChazal (2004)85.8842.2192.8586.86Garcia (2017)92.3859.3681.7393.99Takalo-mattila (2018)89.9152.8676.1891.89Sellami (2019)88.3448.2590.9088.51Jinghao Niu (2020)95.8784.5578.1898.28Yuanlu Li (2021)88.9956.8252.1094.75Our methods91.8959.5185.3793.15In Tables7 and 8, the model input data for Garcia (2017), Takalo-Mattila (2018), Sellami (2019), and Yuanlu Li (2021) are all raw heartbeat signals without the inclusion of manually extracted heartbeat features [36, 37, 39, 40].Among them, the accuracy of Garcia (2017) is slightly higher than the method proposed in this article (92.38% to 91.89%), and the sensitivity of N is also slightly higher than that of the method proposed in this article (93.99% to 93.15%). However, the sensitivity of S in this article is much higher than that of Garcia (2017) (80.23% to 61.96%). The sensitivity of V in this article is slightly higher than that of Garcia (2017) (90.99% to 87.34%) and the productivity (+P) of V is much higher than that of Garcia (2017) (83.09% to 59.44%). Of interest is that Garcia (2017) only achieved 3 classifications, rounding off F andQ in the training process, whereas this article is the result of 5 classifications, which is much more difficult. The recognition of abnormal heartbeats is especially significant in classifying arrhythmia, which is related to the model’s effectiveness in real-world applications. Reflecting on the overall metrics of the model, in Table 8, the sensitivity (Sen) of the algorithm in this article is higher than it (85.37% to 81.73%). Thus, although the method proposed in this article is slightly lower in terms of overall accuracy (Acc), the model is more effective in practical applications [36].Takalo-Mattila (2018) used a convolutional neural network with three convolutional layers to classify the heartbeat signal. Although the adoption of a smaller network structure to perform classification is advantageous in terms of reducing training time, it has a significantly lower classification effect than the approach in this study. Despite the model structure being more complex, our input heartbeats do not need to be denoised. This also reduces the overall time and better reflects the model’s automation [37].The accuracy of the proposed algorithm is higher than that of Sellami (2019) (91.89% to 89.91%) and the sensitivity of N is higher than that of Sellami (2019) (93.15% to 88.52%). The sensitivity of S in this article is slightly lower than that of Sellami (2019) (80.23% to 82.04%), but the productivity (+P) is higher than it (49.40% to 30.44%). The sensitivity of V in this article is slightly lower than that of Sellami (2019) (90.99% to 92.05%), but the productivity (+P) is higher (83.09% to 72.13%). Sellami (2019) has a high classification performance for all types of heartbeats. However, it has a slightly lower sensitivity to N of only 88.52% compared to other methods. The algorithm in this article has comparable classification results for S and V but has a higher sensitivity to N(93.15%) [40].Yuanlu Li (2021) desired a more automated classification algorithm and therefore used an equal-length segmentation method that did not require R-peak location. However, the sensitivity of S in their classification results is too low, at only 35.22%. This causes its classification model to almost fail in identifying S. In Table8, the overall sensitivity of their model is the lowest at just 52.10%, making it difficult to apply in real-life situations [39].De Chazal (2004) and Jinghao Niu (2020) used manually extracted features in their input data. De Chazal (2004) extracted many domain-specific features from the two-lead ECG signal to construct the classifier [29]. Due to the time-consuming nature of manual feature extraction and the low sensitivity of S, it has been difficult to meet the current demand for real-time classification of arrhythmia. Jinghao Niu (2020) used input data as a combination of raw heartbeat and RR interval features. To improve the classification performance of the classifier, Jinghao Niu (2020) used SBCX for the input heartbeats [41]. The accuracy of this article’s method is lower than that of Jinghao Niu (2020) (91.89 to 95.87%), and the sensitivity of N is lower than that of Jinghao Niu (2020) (93.15% to 98.28%). However, the sensitivity of S is higher (80.23% to 77.35%) and the sensitivity of V is higher than that (90.99% to 85.08%) for the algorithm in this article. We compared our data input with that of Jinghao Niu (2020). The input data taken in this article is 508 points of raw heartbeats and is not filtered or preprocessed in any way. The input data taken in Jinghao Niu (2020) is a combination of 256 heartbeats and RR interval features and is preprocessed using SBCX. Therefore, the method proposed in this article is more automated.


### 3.3. Ablation Studies

We conduct ablation studies on the different components of the framework. In Table9, signal-508 represents the heartbeat segmentation method proposed in this article. Signal-256 represents the conventional 256-sample heartbeat segmentation method [41, 44]. Focal loss represents the loss function used in this article. Ensemble represents the model ensemble module.Table 9
Ablation studies on our proposed model.
WorkAcc (%)Class (N)Class (S)Class (V)Class (F)Sen (%)+P (%)Spe (%)Sen (%)+P (%)Spe (%)Sen (%)+P (%)Spe (%)Sen (%)+P (%)Spe (%)Signal-256 + focal loss + Ensemble89.5293.6395.3963.2516.3414.6096.3385.1577.3298.275.413.1798.70Signal-508 + ensemble88.3493.6297.6681.8061.6040.7996.5789.0084.9498.914.901.6797.74Signal-508 + focal loss88.1889.3797.9284.6171.6233.8794.6390.7169.8197.2811.863.6997.56Signal-508 + focal loss + ensemble91.8993.1598.1886.0080.2349.4096.8590.9983.0998.7212.634.0597.65Replacing the heartbeat segmentation method with the traditional 256-sample heartbeat segmentation method, the sensitivity of S is greatly reduced (80.23% to 16.34%), and the ability to classify S is almost lost. This fact strongly suggests that the heartbeat segmentation approach adopted in this article helps to improve the model’s ability to classify heartbeats. The medical diagnosis of arrhythmia is not only based on a single heartbeat but is determined by combining multiple consecutive heartbeats. For example, S will have a shorter RR interval relative to N. When the heartbeat segmentation approach in this article is used, it helps to make it easier for the classifier to obtain distinct features to distinguish between N and S, preventing the overfitting of S during training. By replacing the focal loss with the traditional cross-entropy loss function, the sensitivity of S will decrease (80.23% to 61.60%). The focal loss function helps to make the model focus more on the hard-to-discriminate categories during training, and S, as the generally less sensitive type among the classification results of the various methods, is more likely to receive attention. After removing the ensemble module, the sensitivity of N decreased by 3.78% (93.15% to 89.37%), and that of S decreased by 8.61% (80.23% to 71.62%). In this research, six basic classifiers are trained for voting, resulting in a more robust classification model with increased performance. The comparisons in Table9 demonstrate the effectiveness of our strategy.


## 3.1. Performance Metrics

To evaluate the performance of the model, the confusion matrix of the model is given, and four statistical performance metrics are adopted to evaluate our method as a whole according to the guidelines provided by AAMI [29], namely, accuracy (Acc), sensitivity (Sen), positive productivity (+P), and specificity (Spe). The calculation of each metric is defined in the following equations:(3)Acc%=TP+TNTP+FP+TN+FN×100%,(4)Sen%=TPTP+FN×100%,(5)+P%=TPTP+FP×100%,(6)Spe%=TNTN+FP×100%.When calculating the overall model performance metric, TP is defined as the number of correctly classified abnormal heartbeats, TN is defined as the number of normally classified normal heartbeats, FP indicates the number of normal heartbeats classified as abnormal, and FN indicates the number of abnormal signals classified as normal.


## 3.2. Experimental Results and Discussion

The model performance for the six base classifiers and the final ensemble classifier is shown in Table5. The overall accuracy (Acc) of the base classifiers is all above 83%, with the majority exceeding 88%. Most of the sensitivities (Sen) are around 85%. The final classification result is obtained after voting for the six base classifiers.Table 5
Performance metrics of six base classifiers and ensemble model
WorkAcc (%)+P (%)Sen (%)Spe (%)#188.7149.0377.6690.68#288.1847.6683.6189.37#388.6048.8188.3689.78#486.9244.5885.6487.91#589.8052.3684.9691.23#683.5438.1484.9283.91Average87.6346.7684.1988.81Result91.8959.5185.3793.15As shown in Table5, the final ensemble classification results are significantly better than the average metrics of all six classifiers. Accuracy (Acc) increases by 4.26% (87.63% to 91.89%), positive productivity (+P) increases by 12.75% (46.76% to 59.51%), sensitivity increases by 1.18% (84.19% to 85.37%), and specificity (Spe) increases by 4.34% (88.81% to 93.15%). The accuracy (Acc) of the 6th base classifier is low in Table 5. There are two reasons for this. First, neural network training has a chance. Second, in the focal loss function in this article, S has a high weight, which may cause the classifier to overfocus on S during the training process. The confusion matrix of the final classification results is given in Table 6.Table 6
Confusion matrix of the ensemble classifier.
Predicted labelNSVFQTrue labelN41186142753510670S31714734240V133682929890F3101415490Q30400Table7 shows the sensitivity (Sen), positive productivity (+P), and specificity (Spe) for classes N, S, and V and compares them with six classical arrhythmia classification algorithms. Table 8 shows the overall performance of the model in terms of accuracy (Acc), positive productivity (+P), sensitivity (Sen), and specificity (Spe) and compares it with the classical algorithms.Table 7
Classification performance results of our method and 6 advanced methods.
WorkAccClass (N)Class (S)Class (V)Sen (%)+P (%)Spe (%)Sen (%)+P (%)Spe (%)Sen (%)+P (%)Spe (%)DeChazal (2004)85.8899.1686.8694.0038.5375.9495.3581.5977.7498.78Garcia (2017)92.3893.9997.9582.5561.9652.9697.8987.3459.4495.91Takalo-mattila (2018)89.9191.8997.0076.8362.4955.8698.1189.2350.8594.02Sellami (2019)88.3488.5298.891.382.0430.4492.892.0572.1397.54Jinghao Niu (2020)95.8798.2897.3978.6977.3573.2998.9285.0891.7599.47Yuanlu Li (2021)88.9994.5493.3380.835.2265.8898.8388.3579.8694.92Our methods91.8993.1598.1886.0080.2349.4096.8590.9983.0998.72Table 8
Overall model performance metrics for our method and 6 advanced methods.
Acc (%)+P (%)Sen (%)Spe (%)DeChazal (2004)85.8842.2192.8586.86Garcia (2017)92.3859.3681.7393.99Takalo-mattila (2018)89.9152.8676.1891.89Sellami (2019)88.3448.2590.9088.51Jinghao Niu (2020)95.8784.5578.1898.28Yuanlu Li (2021)88.9956.8252.1094.75Our methods91.8959.5185.3793.15In Tables7 and 8, the model input data for Garcia (2017), Takalo-Mattila (2018), Sellami (2019), and Yuanlu Li (2021) are all raw heartbeat signals without the inclusion of manually extracted heartbeat features [36, 37, 39, 40].Among them, the accuracy of Garcia (2017) is slightly higher than the method proposed in this article (92.38% to 91.89%), and the sensitivity of N is also slightly higher than that of the method proposed in this article (93.99% to 93.15%). However, the sensitivity of S in this article is much higher than that of Garcia (2017) (80.23% to 61.96%). The sensitivity of V in this article is slightly higher than that of Garcia (2017) (90.99% to 87.34%) and the productivity (+P) of V is much higher than that of Garcia (2017) (83.09% to 59.44%). Of interest is that Garcia (2017) only achieved 3 classifications, rounding off F andQ in the training process, whereas this article is the result of 5 classifications, which is much more difficult. The recognition of abnormal heartbeats is especially significant in classifying arrhythmia, which is related to the model’s effectiveness in real-world applications. Reflecting on the overall metrics of the model, in Table 8, the sensitivity (Sen) of the algorithm in this article is higher than it (85.37% to 81.73%). Thus, although the method proposed in this article is slightly lower in terms of overall accuracy (Acc), the model is more effective in practical applications [36].Takalo-Mattila (2018) used a convolutional neural network with three convolutional layers to classify the heartbeat signal. Although the adoption of a smaller network structure to perform classification is advantageous in terms of reducing training time, it has a significantly lower classification effect than the approach in this study. Despite the model structure being more complex, our input heartbeats do not need to be denoised. This also reduces the overall time and better reflects the model’s automation [37].The accuracy of the proposed algorithm is higher than that of Sellami (2019) (91.89% to 89.91%) and the sensitivity of N is higher than that of Sellami (2019) (93.15% to 88.52%). The sensitivity of S in this article is slightly lower than that of Sellami (2019) (80.23% to 82.04%), but the productivity (+P) is higher than it (49.40% to 30.44%). The sensitivity of V in this article is slightly lower than that of Sellami (2019) (90.99% to 92.05%), but the productivity (+P) is higher (83.09% to 72.13%). Sellami (2019) has a high classification performance for all types of heartbeats. However, it has a slightly lower sensitivity to N of only 88.52% compared to other methods. The algorithm in this article has comparable classification results for S and V but has a higher sensitivity to N(93.15%) [40].Yuanlu Li (2021) desired a more automated classification algorithm and therefore used an equal-length segmentation method that did not require R-peak location. However, the sensitivity of S in their classification results is too low, at only 35.22%. This causes its classification model to almost fail in identifying S. In Table8, the overall sensitivity of their model is the lowest at just 52.10%, making it difficult to apply in real-life situations [39].De Chazal (2004) and Jinghao Niu (2020) used manually extracted features in their input data. De Chazal (2004) extracted many domain-specific features from the two-lead ECG signal to construct the classifier [29]. Due to the time-consuming nature of manual feature extraction and the low sensitivity of S, it has been difficult to meet the current demand for real-time classification of arrhythmia. Jinghao Niu (2020) used input data as a combination of raw heartbeat and RR interval features. To improve the classification performance of the classifier, Jinghao Niu (2020) used SBCX for the input heartbeats [41]. The accuracy of this article’s method is lower than that of Jinghao Niu (2020) (91.89 to 95.87%), and the sensitivity of N is lower than that of Jinghao Niu (2020) (93.15% to 98.28%). However, the sensitivity of S is higher (80.23% to 77.35%) and the sensitivity of V is higher than that (90.99% to 85.08%) for the algorithm in this article. We compared our data input with that of Jinghao Niu (2020). The input data taken in this article is 508 points of raw heartbeats and is not filtered or preprocessed in any way. The input data taken in Jinghao Niu (2020) is a combination of 256 heartbeats and RR interval features and is preprocessed using SBCX. Therefore, the method proposed in this article is more automated.


## 3.3. Ablation Studies

We conduct ablation studies on the different components of the framework. In Table9, signal-508 represents the heartbeat segmentation method proposed in this article. Signal-256 represents the conventional 256-sample heartbeat segmentation method [41, 44]. Focal loss represents the loss function used in this article. Ensemble represents the model ensemble module.Table 9
Ablation studies on our proposed model.
WorkAcc (%)Class (N)Class (S)Class (V)Class (F)Sen (%)+P (%)Spe (%)Sen (%)+P (%)Spe (%)Sen (%)+P (%)Spe (%)Sen (%)+P (%)Spe (%)Signal-256 + focal loss + Ensemble89.5293.6395.3963.2516.3414.6096.3385.1577.3298.275.413.1798.70Signal-508 + ensemble88.3493.6297.6681.8061.6040.7996.5789.0084.9498.914.901.6797.74Signal-508 + focal loss88.1889.3797.9284.6171.6233.8794.6390.7169.8197.2811.863.6997.56Signal-508 + focal loss + ensemble91.8993.1598.1886.0080.2349.4096.8590.9983.0998.7212.634.0597.65Replacing the heartbeat segmentation method with the traditional 256-sample heartbeat segmentation method, the sensitivity of S is greatly reduced (80.23% to 16.34%), and the ability to classify S is almost lost. This fact strongly suggests that the heartbeat segmentation approach adopted in this article helps to improve the model’s ability to classify heartbeats. The medical diagnosis of arrhythmia is not only based on a single heartbeat but is determined by combining multiple consecutive heartbeats. For example, S will have a shorter RR interval relative to N. When the heartbeat segmentation approach in this article is used, it helps to make it easier for the classifier to obtain distinct features to distinguish between N and S, preventing the overfitting of S during training. By replacing the focal loss with the traditional cross-entropy loss function, the sensitivity of S will decrease (80.23% to 61.60%). The focal loss function helps to make the model focus more on the hard-to-discriminate categories during training, and S, as the generally less sensitive type among the classification results of the various methods, is more likely to receive attention. After removing the ensemble module, the sensitivity of N decreased by 3.78% (93.15% to 89.37%), and that of S decreased by 8.61% (80.23% to 71.62%). In this research, six basic classifiers are trained for voting, resulting in a more robust classification model with increased performance. The comparisons in Table9 demonstrate the effectiveness of our strategy.


## 4. Conclusion

This article proposes a brand-new system for classifying arrhythmias. In order to achieve a more automatic classification effect, we use the original heartbeat signal as the data input in this article. Three key characteristics define the classification model put forth in this article. First, we employ a novel heartbeat segmentation strategy to assist the model in automatically extracting more features. Second, in order to aid in model training, we use the focal loss as the loss function. Finally, an ensemble classifier is used to produce more reliable classification results. According to the analysis of the experimental data, increasing the input heartbeat window length enhances the model’s classification performance, particularly in terms of its sensitivity to S. In the meantime, the ensemble training strategy used aids in reducing the issue of training overfitting brought on by data imbalance.The classification method proposed in this article is still plagued by low classification performance forF and Q. This is due to the fact that the F and Q samples in the MIT-BIH dataset are far too minimal. As a result, we intend to collect some additional arrhythmia data to help us better train the classification model in future work.



---

*Source: 9370517-2022-09-09.xml*