## 3 Results
### Power Analysis Results  
While power analysis is not inherently required for comparing machine learning (ML) models--largely due to the violations of independence in cross-validation and resampling--it can still serve as a valuable tool to gain deeper insights into the adequacy of sample sizes for robust model evaluation. By quantifying the variability in sample size requirements across different performance metrics and model pairs, power analysis provides a structured framework for addressing two critical challenges in ML research: determining the optimal dataset size for reliable comparisons and understanding the limitations of statistical power under practical constraints.  
Pairwise effect sizes and corresponding power analyses revealed that substantially larger sample sizes would be desirable to attain adequate statistical power than initially ten estimates anticipated, as shown in Table 4, which presents the pairwise median calculated sample sizes across all metrics for referent models. If we take averages for a particular metric into consideration, variations are even more extreme. For example, the average proposed sample size for the accuracy metric for certain pairs (RF/LGBM and RF/ANN) was over five thousand when all model pairs were considered, while the calculated mean sample size for accuracy was 189 (including all models). Comparable disparities were observed across all other metrics, with certain model pairs necessitating only modest sample sizes (for example, LR pairs) while others demanded exceedingly large ones. The exact median calculated sample sizes for the referent metrics (including all models) were as follows: accuracy (189), precision (38), recall (43), F1-score (268), and ROC AUC (19). These results underscore the variability in sample size requirements depending on the performance metric under consideration.  
To simplify the research process, the sample size was standardized to 100 for all subsequent analyses. This choice was informed primarily by two considerations. First, the median required sample sizes for most metrics clustered around or below 100, making it a suitable upper bound for ensuring sufficient power in a broad range of comparisons. Second, using a uniform sample size streamlined the design of robust comparative tests without incurring prohibitive computational overhead.  
Due to the differing requirements of statistical tests, two complementary evaluation schemes were finalized: one offering 10 performance estimates per model (via \(5\times 2\) cross-validation and single-run 10-fold cross-validation) and another producing 100 estimates per model (using ten repeated 10-fold cross-validation and corrected random resampling with 100 splits). The [PERSON] and Wilcoxon tests were similarly adapted to operate on 100 estimates (and for 10 as well), thereby facilitating more reliable multiple-model comparisons.  
Figure 3: Best loss (\(-\)F1 scores) for ML models during hyperparameter tuning”.  
By balancing established (10-split) and enhanced (100-split) procedures, it is anticipated that a more comprehensive and robust assessment of model performance will be obtained. Adopting 100 estimates supports the substantial variability in required sample sizes observed in the power analysis while offering a feasible pathway to improved statistical rigor.