## 4 Discussion
### Statistical Results  
Following the descriptive statistical analysis (Figure 4, Table 5--Chapter Results/Descriptive Statistics), a comparison was conducted between the results obtained from standard 10-fold cross-validation (10 estimates) and 10-times repeated 10-fold cross-validation (100 estimates) across various performance metrics, including accuracy, precision, recall, F1, and ROC-AUC. The findings indicate that, on average, the results obtained from 100 estimates (folds) were slightly worse than those from the standard 10-fold CV (10 estimates). This discrepancy may be attributed to the effect of random variation, which, while introducing greater variability, can also lead to a more robust and generalized performance estimate. Single 10-fold CV relies on a specific partition of the data, which may inadvertently favor the model due to random splits that include \"easier\" test examples. Repeated 10-fold CV averages over multiple partitions, reducing the influence of chance and providing a more stable estimate of model performance. This aligns with findings that repeated cross-validation decreases variance in performance estimates and better approximates the true generalization error [18]. Another possible explanation is that repeated cross-validation introduces variability in the composition of test sets, which may expose the model to more challenging instances or less favorable class distributions. [PERSON] & [PERSON] (2003) emphasize that repeated data splits can inadvertently create test sets with different underlying distributions compared to a single split, potentially leading to lower performance metrics while simultaneously enhancing the robustness and realism of the evaluation [19]. Furthermore, as observed in Figure 4, the ROC-AUC metric consistently outperforms other evaluation metrics across all ML models, with an approximate 10% improvement. For instance, CatBoost achieves accuracy, precision, recall, and F1 scores around 0.80, while its ROC-AUC reaches 0.87. This pattern is evident across all models, except for SVM, where the difference between ROC-AUC and other metrics is less pronounced. The reason can be that metrics like accuracy, precision, and F1 depend on a fixed decision threshold (often 0.5), while ROC-AUC evaluates the model's ability to rank positive instances above negatives across all possible thresholds. A model with well-separated class probabilities (e.g., CatBoost, XGBoost) may have suboptimal performance at the default threshold but strong ranking capability, leading to a higher ROC-AUC. This aligns with [PERSON] (2006), who emphasizes that AUC measures \"the probability that a classifier will rank a randomly chosen positive instance higher than a negative one\", independent of threshold choice [34]. One more reason can be that SVM might be less \"probabilistic\". Standard SVMs output uncalibrated decision values (distances to the hyperplane) rather than true probabilities. Without post-processing (e.g., Platt scaling), these scores lack the spread and calibration needed to exploit threshold variations, narrowing the gap between ROC-AUC and threshold-based metrics. [PERSON] & [PERSON] (2005) show that SVMs require explicit calibration to produce reliable probabilities, and uncalibrated scores often cluster near the decision boundary, limiting their threshold adaptability [44]. This explains why SVM's ROC-AUC and threshold metrics may align more closely than probabilistic models like CatBoost or logistic regression. Further analysis of the [PERSON] test and the paired 5 \(\times\) 2 cross-validation \(t\)-test reveals that their conservative nature--manifesting in fewer statistically significant differences compared to other methods--stems from their inherent methodological properties. [PERSON]'s test focuses on discordant pairs. [PERSON]'s test evaluates differences in model performance by analyzing only the disagreements between two models (i.e., instances where one model is correct and the other is wrong).