## 5 Results
### Evaluation of Conformal Prediction Methods  
This section examines SCP applied to the CatBoost classifier, along with various calibration strategies. Table 5 and Figures 12-15 summarize the main findings for marginal coverage, average coverage gap, minority error contribution, and the single prediction set ratio. Together, these results reveal how often the conformal procedure yields valid and precise predictions, and how misclassification risk is distributed across the minority (admission to ICU) class. Figure 12 shows that marginal coverage for the CP converges near or slightly above 0.95, indicating that the true label is included in the model's prediction set roughly 95-96% of the time. Uncalibrated CatBoost achieves the highest average coverage (0.9615 \(\pm\) 0.02), narrowly surpassing Platt Scaling and Beta/Spline calibrations (around 0.9599-0.9608). Isotonic Regression and Venn-Abers trail slightly at 0.9578. Statistical tests (\(\chi^{2}_{\text{Friedman}}=80.5488\), \(p=6.4421\times 10^{-16}\)) confirm significant differences among the calibration methods. Post-hoc comparisons suggest that although Uncalibrated CatBoost has a minor coverage edge, several pairwise differences (e.g., uncalibrated vs. Beta or Spline) are subtle. Clinically, these marginal coverage levels imply that across repeated samples, the prediction sets produced by CP include the correct label at or beyond the intended 95% frequency--a reassuring result for risk-sensitive healthcare environments. Average coverage gap (Figure 13) measures how much coverage can fluctuate relative to the nominal 95% target across different test instances. Uncalibrated CatBoost achieves the smallest gap (18.66 \(\pm\) 7.61), with Beta, Platt Scaling, and Spline following closely (all around 19). Isotonic Regression and Venn-Abers lie marginally higher (approximately 19.48 and 19.68, respectively). The Friedman test (\(\chi^{2}_{\text{Friedman}}=19.8589,\ p=1.3283\times 10^{-3}\)) again indicates statistically significant variation. For the examined use case, a lower coverage  
Figure 12: Friedman-Nemenyi significance test results regarding Marginal Coverage of Split conformal prediction and calibration methods on the CatBoost model across 200 bootstrapping iterations. \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Calibration Method** & **Marginal Coverage** & **Average Coverage Gap** & **Minority Error Contribution (\%)** & **Single Prediction Set Ratio (\%)** \\ \hline Beta Calibration & 0.9599 (\(\pm\)0.02) & 19.0294 (\(\pm\)7.99) & 93.4913 (\(\pm\)12.11) & 90.6204 (\(\pm\)10.02) \\ Isotonic Regression & 0.9578 (\(\pm\)0.02) & 19.4813 (\(\pm\)8.27) & 90.8993 (\(\pm\)14.13) & 91.1105 (\(\pm\)13.55) \\ Platt Scaling & 0.9608 (\(\pm\)0.02) & 19.1753 (\(\pm\)8.11) & 95.3790 (\(\pm\)10.44) & 98.2572 (\(\pm\)10.87) \\ Spline Calibration & 0.9599 (\(\pm\)0.02) & 19.3481 (\(\pm\)7.73) & 94.3152 (\(\pm\)11.21) & 90.8709 (\(\pm\)9.62) \\ Uncalibrated CatBoost & **0.9615 (\(\pm\)0.02)** & **18.6658 (\(\pm\)7.61)** & 95.5935 (\(\pm\)10.37) & 89.7549 (\(\pm\)10.80) \\ Venn-Abers & 0.9578 (\(\pm\)0.02) & 19.6807 (\(\pm\)8.29) & 92.0942 (\(\pm\)14.88) & 89.8766 (\(\pm\)12.73) \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Average Conformal Prediction Results for Split Conformal with CatBoost Model Across 200 Bootstrap Samples. gap means the conformal intervals (or sets) remain more consistently valid across different patients. Large gaps can signal that certain subpopulations--for example, older patients or those with atypical symptoms--might be over- or under-covered. Although Uncalibrated CatBoost exhibits the tightest overall coverage gap, the differences here are modest (roughly 1% across methods). Hospitals with large, diverse patient populations might still opt for a calibration method if it yields other benefits (e.g., improved minority error rates) without inflating coverage gap too severely. Figure 14 presents the minority error contribution: the proportion of total misclassifications arising from the positive (ICU admission) class.