## 3 Tools for R-Peak Validation  
Accurate validation of detected R-peaks with reference annotations is a crucial step for developing a reliable detection algorithm. However, the literature usually does not provide detailed descriptions on the contextual variables for the employed validation methodologies, making it difficult to replicate the exact conditions for testing and conducting a fair comparison of peak detection results. To address this issue, standard systems and benchmarks were established by ANSI/AAMI to ensure transparency and consistent testing conditions. Various tools have been established to facilitate the R-peak validation process, many of which are part of the waveform database (WFDB) software package, version 0.10.0 available on PhysioNet [23]. Below, we discuss these tools and their testing mechanisms. **Beat-by-beat (bxb):** The bxb tool is a utility comparator of the WFDB software package available on PhysioNet [23]. This tool complies with the EC57-2012 standards for ECG R-peak validation [71; 72]. The testing mechanism of bxb starts with taking a predicted sample file generated by a custom peak detection algorithm and creates corresponding annotations using the writing annotation (WRANN) function. After running a peak detection algorithm on an ECG signal, a list of predicted R-peak locations, typically represented as time instances or sample numbers, is obtained. These predictions are formatted into a text file, with each line containing the sample location for the detected R-peak. WRANN reads this input file and generates a corresponding predicted annotation file in the WFDB format, similar to the provided reference art files with clinical annotation or beat types, aligning and labeling the predicted sample or time location with a user-defined symbol [73]. This symbol can be any letter and does not need to follow the AAMI beat notations, as the focus is solely on detection validation not classification, meaning only the predicted time locations are required. This predicted annotation file can then be used with WFDB validation tools to quantitatively assess the performance of the detection algorithm in comparison with reference annotations using a comparison window size of 0.15 s or 150 ms [74]. To handle non-beats or segments of the ECG without real annotated R-peaks, bxb uses a shutdown and resume function. It pauses the comparison process during noisy segments or annotated non-beats and resumes when those segments end, ensuring that noise and non-beat annotations do not affect validation statistics [75]. The bxb program generates various performance reports, including summaries of annotation discrepancies and statistics on RR interval errors, false positives, and missed beats. It also offers options to generate condensed reports, verbose output for mismatches, and labeled error files, allowing developers to thoroughly evaluate their detection algorithms. **Run-by-run (rxr):** The rxr tool is another WFDB comparator that aligns and compares annotation files from reference and predicted samples within a 0.15 s window [76]. This tool was primarily designed to compare RR intervals but can be used to evaluate the accuracy of automated algorithms for detecting R-waves. It mainly reports statistics such as mean error and standard deviation. For segments other than actual beats, it uses a similar concept of shutdown and resume like bxb. **Measurement-by-measurement (mxm):** The mxm tool is another validation tool of WFDB, used to validate heart rate measurements from multichannel ECG. It starts comparison after the first 5 min of a given recording to allow the signal to stabilize and minimize the impact of initial artifacts and noise [77]. [PERSON] et al. applied a similar concept with bxb to validate atrial fibrillation [27]. For each test measurement, mxm calculates the error relative to the closest reference measurement, reporting normalized or unnormalized root mean square (RMS) errors.