## 4 Discussion
This discrepancy arises because, in the case of a single dataset for both training and testing, samples from the same patients are mixed in both the training and testing sets. Consequently, the algorithms tend to memorize the data, resulting in inflated classification metrics, failing to demonstrate the model's ability to generalize to new, unseen data, which is crucial for assessing the model's true performance. This effect is particularly crucial to consider when using cross-validation. Most studies classifying OSA episodes have not mentioned or specified implementing a patient hold-out strategy, which is essential to avoid potential bias in cross-validated tests [85] (see Table 6). In order to provide a more realistic assessment of the model performance, it is imperative to use an external validation framework, which is advocated by international guidelines such as the TRIPOD statement [76]. In this study, distinct datasets have been employed with variations in recording times, patient demographics, and acquisition systems for training and testing, offering a less biased representation of performance in conditions more similar to clinical practice. Indeed, as previously published in an earlier work [40], when the same ML models are tested with 10-fold cross-validation, Ac, Se, and Sp are, on average, up to 15% higher than those obtained through external validation regardless of the feature set used to train the models. The datasets used in this study (Apnea-ECG, MIT-BIH, and UCD-DB) were collected in diverse clinical contexts, including healthy individuals and patients with cardiac conditions such as arrhythmias. While these datasets differ in demographics and technical characteristics such as age ranges, gender representation, and equipment specifications, our HRV-based methodology is robust to these variations. Unlike methods reliant on raw ECG signals, which are sensitive to factors like electrode placement and sampling frequency, HRV analysis ensures greater adaptability across different acquisition systems. However, the limited demographic diversity highlights the need for future studies with more representative datasets to further validate and generalize the findings. The analysis of false positives and false negatives may provide valuable information. False positives are often linked to subtle signal artifacts or transient HRV fluctuations that, despite noise filtering, may affect the classifier's belief. This suggests that refining artifact detection criteria, particularly for borderline cases such as bradycardia or transient autonomic changes, could improve accuracy [15]. Conversely, false negatives might indicate an underrepresentation of certain apnea patterns or demographic subgroups in the training data. If this is the case, expanding the diversity of datasets and incorporating augmentation techniques could help the model generalize better [76]. These considerations point to the importance of balancing preprocessing rigor with representativeness in training. When comparing our findings with those of [PERSON] et al. [22], who also validated an ML model based on RQA features extracted from HRV using a three-fold cross-validation on a single database (the Physionet Apnea-ECG database), our results align with theirs, demonstrating more moderate performance compared to inflated results from cross-validation alone (Table 6). This reinforces the robustness of our model under more strict evaluation conditions. Other works not solely based on RQA but also concerning OSA detection using HRV, such as [PERSON] et al. [86] and [PERSON] et al. [87], reported similarly moderate results (around 80-85%) when using external databases with segments of one-minute length of ECG. This consistency underscores the importance of rigorous validation practices. Moreover, the present study has faced common limitations related to data scarcity and the computational demands of the proposed models, especially for the DL ones. These challenges are common in the field, requiring larger and more diverse datasets.