dataesr/scientific-paragraphs-categorization huggingface dataset:
Teaching these algorithms new use cases featuring high levels of noise or artifacts requires adapting their parameters or developing new ones, whereas deep learning (DL) architectures can be tailored through the adaptation of their training databases  [7]. Compared with classical algorithms, DL architectures require large amounts of annotated data and high computing resources, which are not available in all circumstances. In the last decade, DL architectures have successfully been explored across many sub-fields following early successes in computer vision and pattern recognition  [8, 9]. Many studies confirmed that DL architectures outperform classical algorithms for signal analysis, image recognition, and object/pattern detection  [10]. Many medical applications have benefited from the development and progress in computing, data availability, and DL to address a large variety of complex challenges  [11].
Van  and Podmasteryev reviewed in detail four MLbased algorithms  [65] , including artificial neural network (ANN)  [66] , k-nearest neighbor (k-NN) algorithm [67], k-means  [68] , and SVM  [69]. Performances of investigated ML algorithms were shown to depend on the quality of their training datasets, which was an essential drawback. Few approaches were designed to work with multiple lead ECG recordings. Śmigiel et al. used 6 classical detectors and a k-means clustering method after processing a 12-lead ECG signal  [70]. Chen et al. used the PT algorithm to detect R-peaks in each lead of the ECG recordings of the LUDB database  [71] [72] [73]. The main disadvantage of classical algorithms is that they depend on many empirical parameters (e. g. , self-adaptive threshold values) and lack robustness, particularly with lowquality, noisy, or pathological signals, changes in sensors or different QRS morphologies.
DL-Based Approaches Several DL architectures have been proposed for different tasks related to computer vision and pattern recognition  [8, 9] , including ECG analysis  [2, 4]. A DL architecture is an ML algorithm based on a neural network (NN). It works by learning a correlation between input features using a large amount of data. The larger the database, the better the DL architecture can learn by optimizing its weights and perform the targeted task by taking the most suitable decision. Commonly used DL architectures to detect R-peaks are: convolutional neural network (CNN)  [74] , recurrent neural network (RNN)  [75] , long short-term memory (LSTM)  [76] , and gated recurrent unit (GRU)  [77]. CNN-based solutions represent the most used techniques for ECG analysis, and particularly for feature extraction tasks  [2].
Šarlija et al. reported a 1D CNN-based architecture exceeding 99% in recall and precision on unseen data from the MIT-BIH arrhythmia database  [78]. Oudkerk-Pool et al. reported 92. 6% recall, 91% precision, and 91. 8% F1-score on 100 recordings from the PhysioNet/CinC challenge 2017 using a fully convolutional dilated NN  [20, 79]. Tison et al. used a modified CNN (U-Net) architecture for automating the classification of six types of ECG segments (P-wave, PR segment, QRS complex, ST segment, T-wave, and TP segment)  [80]. Using the MIT-BIH arrhythmia database and a modified U-Net architecture, Oh et al. reported a high recall of 98. 76% but a low precision of 29. 55%  [81]. Jimenez-Perez et al. used a U-Net architecture to analyze dual lead ECG recordings from the QTDB database and achieved a recall of 99.
94%  [82] [83] [84]. Some novel DL-based methods combine variant CNN architectures with other DL models. Liu et al. achieved high accuracy by using three variants of bidirectional long short-term memory (BLSTM) architectures: with attention model, combined with U-Net, and combined with U-Net++  [85]. Peimankar and Puthusserypady demonstrated a recall of 97. 95% and a precision of 95. 68% on 105 recordings from QTDB by combining a CNN architecture with a BLSTM model to detect onset, peak, and offset of different heartbeat waveforms (P-waves, QRS complexes, and T-waves)  [86]. Vijayarangan et al. proposed a fused CNN-ResNet, called RPnet, an architecture combining the 1D U-Net model with inception and residual modules, to extract R-peaks from noisy ECG  [87]. This approach contributed to alleviate the vanishing gradient issues of DL architectures.
RPnet outperformed three classical algorithms (Hamilton and Tompkins  [31] ,  Christov [35] , and stationary wavelet transform  [50] ), with an F 1-score of 98. 37%, on the second CPSC database  [24]. Duraj et al. showed high performance on LUDB by incorporating the residual and squeezeexcitation blocks into the 1D U-Net architecture for extracting segments, such as P-waves, QRS complexes, and T-waves, regardless of the lead  [88]. Gabbouj et al. proposed a 1D selforganized operational neural network (ONN), evaluated on second CPSC, and achieved recall of 99. 79%, precision of 98. 42%, and F1-score of 99. 10%  [24, 89]. However, during the training phase, it featured a high complexity in the number of multiply-accumulate operations and the number of parameters compared with 1D CNN architectures.
Many DL architectures are computationally expensive and hence not well-adapted to embedded devices. Furthermore, CNN and their variants, such as U-Net, are the most common DL architectures for R-peak detection. Yet, many researchers stated that the best accuracy was achieved by combining different DL architectures, at a cost of increasing complexity  [90]. Single ECG lead DL architectures already outperform classical algorithms, but few use multiple leads. Increasingly available 12-lead ECG databases may increase the precision and robustness of DL-based approaches.
[12]. To validate detected R-peaks in a single ECG lead from the MIT-BIH arrhythmia and third CPSC databases, Zahid et al. carried out a verification model based on a timing criterion, considering the following hypothesis: if the predicted R-peak locations of two beats fall within 300 ms, one is an F P  [25, 93]. In multi-lead ECG signals, Han et al. introduced a post-processing step based on an adaptive dynamic threshold strategy and electrophysiology knowledge, after applying a linear ensemble method averaging the outputs of two DL models  [94]. They used CNN and LSTM models, both based on the U-Net architecture. A pre-processing step consisted of removing the wandering baseline and high frequency noise based on discrete wavelet transforms and a third order Butterworth band-pass filter. They reported recall and precision, ranging from 99.
45% to 100%, on QTDB, LUDB, and CCDD databases  [95] [96] [97]. Their strategy effectively reduced FN and FP; however, it did not correctly detect some arrhythmic QRS complexes, such as left bundle branch blocks. Cascading several steps of variable complexities may require extensive computational power, and make the application of hybrid approaches difficult in embedded devices. Furthermore, introducing thresholds, heuristics, and rules in post-processing steps may reduce their generalization capacity to other data and reduce their robustness, particularly in the case of pathological QRS morphologies.
            (2) Comparing the Kors quasi-orthogonal transformation with three other VCG transformations (inverse Dower, Kors regression, and Frank's orthogonal lead system), Kors et al. reported the best results with the Kors regression transformation  [100]. Figure  2  illustrates the Kors regression transformation in a recording from the INCART database. In the figures below, all signals were scaled between -1 and 1. All annotated R-peaks were highlighted using two yellow bars delineating a five-sample segment.
Experimental Corpora In the context of ECG analyses, not all available databases provide expert-validated annotations of R-peaks locations. Furthermore, public databases are often built using single or dual lead ECG recordings, such as MIT-BIH arrhythmia  [16, 17] , QTDB  [83, 84] , and third CPSC  [25]  databases. PTB Diagnostic ECG [104], PTB-XL  [105, 106] , INCART  [21, 107] , first CPSC  [23, 108] , SPH  [109] , LUDB  [72, 73] , CSE  [110] , CCDD  [95] [96] [97] , and Georgia [111] feature 12-lead ECG recordings. However, the number of publicly available databases having both 12-lead ECG recordings and expert-validated annotations is limited. To the best of our knowledge, CCDD, LUDB, and INCART are the only publicly available databases with the 12 conventional leads (I, II, III, AVR, AVL, AVF, and V1 to V6) and expert-validated R-peak annotations.
1. Chinese Cardiovascular Disease Database (CCDD): is composed of 193,690 12-lead ECG datasets collected and annotated by 2 cardiologists  [95] [96] [97]. Only 943 complete annotated recordings are publicly available. The positions of the QRS complexes, onsets, peaks, offsets for P and T-waves were provided. Among the 943 recordings, the first 251 recordings are usually selected by researchers in their experiments, since the corresponding annotations were made following the same protocol  [94]. Based on the set segment size and stride, 753 windows with a total of 3973 R-peaks were obtained and used in our experiments. To appropriately evaluate the proposed approach and conduct an objective comparison with other methods, few adaptations of the annotations of QRS complexes were required. In the first and/or last cycles of the recordings, a few QRS complexes were not annotated.
We discarded them, considering only annotated cycles. All samples located inside a QRS interval were annotated, while the proposed approach required only one R-peak location. A kmeans algorithm was hence applied to extract one precise R-peak location per QRS interval after setting the number of R-peaks. Figure  4  illustrates a 12-lead ECG recording example of the CCDD database. In the first and last cycles of the recordings, some R-peaks were not annotated. We discarded them, considering annotated cycles only during the evaluation of the proposed approach. Figure  5  illustrates a 12-lead ECG recording example of LUDB.
4. CCDD-Extra: is a set of 103 recordings pseudo-randomly selected from the 692 (943 -251) remaining recordings of the CCDD database. Annotations of these recordings followed the same protocol as CCDD (i. e. extracting the positions of R-peaks with the k-means algorithm). Based on the set analysis window size and stride, 309 windows with a total of 1616 R-peaks were used. Figure  7  illustrates a 12-lead ECG recording example of the CCDD-Extra database. CCDD-Extra was used to evaluate the performance of our DL architecture in a cross-database validation protocol. Table  1  summarizes the features of the four databases used in our experiments.
Experimental Protocol After converting the ECG recordings into 3D VCG, a set of thorough experiments were conducted to evaluate and validate the proposed approach. They followed the experimental protocols described below: 1. Leave-one-out cross-validation protocol: was applied to the CCDB, LUDB, and IN-CART databases independently. One recording was extracted for the evaluation phase, while the remaining recordings of the same database were used for the training phase. For instance, in the case of the INCART database, 74 recordings were used to train our DL architecture, and the 75th remaining one was used for evaluating its performance. Then, an average of the 75 different evaluation experiments was computed.
2. Cross-database validation protocol: was carried out by evaluating the proposed approach on a database different from those used in the training phase to demonstrate its generalization capacity  [112, 113]. In our experiments, all 526 recordings (251 + 200 + 75) of CCDB, LUDB, and INCART databases were used to train the DL architecture, while the 103 recordings of CCDD-Extra were used for its evaluation. A total of 34,078 windows and 181,711 R-peaks were used during the training phase. For both protocols, the same hyperparameters, summarized in Table  2 , were selected. For stochastic optimization, an Adam optimizer was used with an initial learning rate of 0. 001 and a batch size of 64. Weights were randomly initialized with the Xavier uniform distribution. Training was stopped early only when no more improvement in the crossentropy loss value was recorded for at least 10 epochs (i.
e. patience of 10). Python version 3. 10 was used to implement the proposed approach, and Keras version 2. 9 was employed for training the DL architecture. Our experiments were carried out on a physical computer equipped with 2. 4 GHz Intel® Core™ i9-12900F having 32 GB of RAM and a single NVIDIA® GeForce RTX™ 3060 with 12 GB of RAM. Google Colaboratory servers were also used (Tesla T4 GPU with 16 GB of RAM). The training phase was processed using CUDA kernels, while the testing phase was implemented with a single CPU. Training on all data from CCDD, LUDB, and INCART took 7274 s on the computer (resp. 4297 s on Google Colaboratory servers). As an example, testing the proposed approach on a ~13 s recording (6484 samples at 500 Hz) of CCDD-Extra took 1. 15 s on the computer (0. 555 s on Google Colaboratory servers) to detect 13 R-peaks.
00781) is higher in the cross-database validation protocol compared to the loss values obtained in the leave-one-out cross-validation protocol (Figure  8 ). To validate the 3D VCG approach, the performances of individual lead-by-lead trainings of the DL architecture were assessed, first using representative ECG leads (I, II, and V1 to V6), and second, using individual 3D VCG vectors (X, Y, and Z). Multi-lead trainings were performed using all 8 ECG leads (I, II, and V1 to V6) as input and the 3 best leads from the first experiment.
Qualitative Results Figures 9-12 illustrate the R-peaks detected using the proposed approach on recordings from CCDD, LUDB, INCART, and CCDD-Extra, respectively. Detected R-peaks are marked with red triangles. Visual inspection of the qualitative results revealed that the proposed approach for detecting R-peaks is reliable and robust. First and/or last cardiac cycles were sometimes not annotated in some ECG recordings and were discarded from the analysis, as can be seen in Figures  9  and 10. Figure  11  demonstrates that the proposed approach succeeded in detecting the R-peak located between 8 s and 9 s, despite a different QRS morphology compared with its predecessors and successors. In Figure  12 , the proposed approach detected all R-peaks from an ECG recording of the CCDD-Extra database, never seen during the training phase.
This reproducible result, along with quantitative results described below, confirmed that the proposed approach has the capacity to be generalized. To illustrate some failed detections, a recording from the INCART database shows noice-induced FP in Figure  13. Another source of FP is pacemaker patients (LUDB). However, these errors were limited as shown in the quantitative analysis. Missed detections (FN) are mostly related to pathological QRS morphologies that are less often represented in the training datasets. They were mostly seen in CCDD and INCART databases.
Quantitative Results on CCDD, LUDB, INCART, and CCDD-Extra Databases In the leave-one-out cross-validation protocol, the proposed approach produced results with recall ranging from 99. 39% to 99. 84% and precision ranging from 99. 88% to 100. 00% (Table  3 ). In the cross-database validation protocol, the proposed approach produced results with recall of 99. 41% and precision of 99. 89% (Table  4 ). The DL architecture was trained on 3D VCG data and could also be trained on a different number of leads, extracted or reconstructed from the 12-lead ECG recordings. Four different experiments were carried out, using the leave-one-out cross-validation protocol and the INCART database. Results are summarized in Table  5. In the first and second experiments, the DL architecture was trained on one lead/vector at a time.
31% to 99. 14% vs. 99. 88% to 100. 00%). Results from PT were also compared with the results of the proposed approach in the cross-database validation protocol (Table  7 ). This is more fitting than the previous comparison as in both cases, the algorithms used to analyze the data were not previously exposed to them. The PT algorithm outperformed the proposed approach in recall in all leads except V6 (99. 84% vs. 99. 41%). However, its precision was inferior, regardless of the chosen lead (98. 42% vs. 99. 89%). The two best leads for precision were leads II and V4 (98. 96%), exhibiting 16 FP vs. 2 for the proposed approach. In the literature, few R-peak detection methods were evaluated using the same databases. We selected recent methods to compare their performances with ours  [71, 94, 115].
Discussion This work proposes a DL-based approach to detect R-peaks. The novelty of the proposed approach consists in using reconstructed 3D vectorcardiogram (VCG) leads  [100]  as input instead of single or multiple ECG lead(s). A 3D VCG is obtained by a linear combination of multiple ECG leads. It reduces the computational complexity (i. e. memory and time requirements) both for training and inference compared to using all available leads. Using a 3D VCG as input during the training phase of a DL architecture has the advantage that both strong temporal correlation features and fine morphological features are extracted. Indeed, a 3D VCG provides a calibrated maximal QRS vector within a specific timing. In contrast, when using multiple ECG leads, each scalar lead has its own sensitivity and timing for the R-peak maximum, which could bias data and subsequently results  [116].
The proposed approach could have applications in automated measurements of clinical features  [119]. For instance, heart rates and heart rate variability measurements were computed based on R-peak detections, using the leave-one-out cross-validation protocol in the INCART database  [120]. The obtained values are almost identical to values obtained from ground truth annotations (see Table  11 ). This is further evidence of the clinical validity of the proposed approach. Table  11. Heart rates and heart rate variability measurements derived from R-peak detections using the leave-one-out cross-validation protocol in INCART: AVG-HR: average heart rate; SD-HR: standard deviation of heart rates; RMSSD (ms): square root of the mean sum of squares of differences between adjacent RR intervals. The proposed approach was thoroughly evaluated on multiple 12-lead ECG databases in leave-one-out cross-validation and cross-database validation protocols.
The crossdatabase validation protocol, exposing the proposed approach to data never seen in the training phase, demonstrated a tiny drop in performance: recall dropped from 99. 66% to 99. 41% and precision from 99. 96% to 99. 89%. This shows the robustness of our approach, acquired from training on a large number of ECG cycles (181,711). However, the databases used for training were unbalanced. The INCART database features a larger number of ECG cycles (175,907) than LUDB (1831) or CCDD (3973); it also has less patients: 75 vs. 200 (LUDB) or 251 (CCDD). More balanced ECG databases may further improve the results and robustness of the trained model. It could also provide more insights to fine-tune the model hyperparameters. The CCDD-Extra database used in the cross-database validation protocol was not completely independent from the CCDD database used for training.
This can be explained by the insufficient statistical representations of such QRS morphologies in the training databases. Zahid et al. identified this behavior and used a data augmentation strategy, generating additional extrasystole segments to address issues related to limited training examples  [93]. False detections (FP), although extremely rare, usually happened in situations of noise, artifacts, or pacemaker pulses. They could be dealt with in a post-processing step, which could also reduce the number of FN. However, avoiding any post-processing is an advantage, especially for applications requiring a low latency (e. g. , CMR). Our hypothesis was to use a 3D VCG reconstruction scheme to avoid the arbitrary decision of selecting which lead(s) to use. A 3D VCG can be reconstructed from many ECG acquisition schemes and provides increased robustness to a large variety of changes, such as errors in positioning electrodes, artifacts, acquisition protocols, and devices.
Another hypothesis was to define a single QRS label (or timing) representative of the onset of ventricular systole. We used the expert-validated annotations from lead II as the gold standard. This choice may have had a negative impact on the accuracy of the PT algorithm in other ECG leads. It could also have reduced the performance of the DL architecture after training on other ECG leads. Finally, it may explain why 3D VCG vectors X and Y have the highest accuracy. To enhance the performance of our DL model, a refined timing of R-peaks could be modeled on a lead-by-lead basis. To demonstrate both robustness and generalization capacity of the proposed approach in detecting premature ventricular contraction or supra-ventricular premature beats, we evaluated its performances on the testing set of the MIT-BIH arrhythmia database (DS2) after training it using lead II data from INCART only and compared them with other state-of-the-art R-peak detection methods (Table  12 ).
The DL architecture used in this work, a modified U-Net, has the potential to run with low latency  [93]. It could be implemented on a powerful computer or on a GPUcapable edge device, and run in quasi real-time. Since it does not require CPU intensive pre-processing nor post-processing steps, it could be applicable in a number of use cases. Furthermore, adapting the training database, and thus the trained model, could provide similar accuracy in various use cases. One example of use cases is CMR, where the ECG signal suffers from a magneto-hydrodynamic artifact  [121, 122]. This artifact could be added to the training datasets. Other use cases featuring high noise could also be addressed by adding similar noise features to the training datasets. A performance evaluation of the impact of adding synthetic noise using a signal to noise ratio metric could be developed to assess the robustness of the proposed approach  [123].
Conclusions and Further Work In this article, we propose a novel approach for detecting R-peaks based on training a DL architecture with 3D VCG data, without pre-processing nor post-processing steps. R-peak detection was approached as a segmentation task using the U-Net architecture which was fed with normalized 3D VCG segments of durations ~4 s (2048 samples at 500 Hz). The proposed approach can be performed in ECG recordings of any practical durations (e. g. , from tens of minutes down to a few seconds) and is robust to low-quality, noisy, or pathological signals acquired from different sensors. It is also robust to various QRS morphologies. Using the proposed approach, high performances were achieved in leave-one-out cross-validation and cross-database validation protocols. Furthermore, false detections (FP) and missed beats (FN) were significantly reduced compared to recent state-of-theart methods or the PT algorithm.
Lobachevsky University Electrocardiography Database (LUDB): is composed of 200 12-lead ECG recordings with a duration of 10 s, and a sampling rate of 500 Hz, recorded using a Schiller Cardiovit AT-101 cardiograph (Schiller AG, Baar, Switzerland) and released on the PhysioNet Website in 2020. It features a large variety of QRS morphologies, from healthy volunteers to patients with different cardiovascular diseases, some of them with pacemakers. All boundaries and peaks of P-waves, QRS complexes and T-waves were manually annotated by cardiologists for each of 12 leads independently. We considered the annotations of lead II in our experiments. Based on the set segment size and stride, 400 windows with a total of 1831 R-peaks were used.