## 5 Results
### Evaluation of Calibration Approaches (RQ2)
For example, an overly confident model might assign probabilities close to 1.0 for patients who ultimately do not get readmitted, incurring large penalization. If healthcare decisions hinge on probability thresholds to, say, intensify monitoring or allocate ICU beds, a method with a low LogLoss is valuable because it avoids severe misclassifications. Hence, Platt or Beta might be more attractive if one seeks a stable, precise reflection of readmission likelihood without excessively skewing the probability scale. While LogLoss focuses on penalizing incorrect high-confidence assignments, ECE and MCE capture how close predicted probabilities are to empirical event frequencies. This distinction is crucial in domains like sepsis management, where calibrating risk estimates for threshold-based interventions can directly affect patient outcomes. ECE measures the  
Figure 8: Friedman-Nemenyi significance test results regarding log loss of calibration methods on the CatBoost model across 200 bootstrapping iterations. Figure 7: Friedman-Nemenyi significance test results regarding AUPRC of calibration methods on the CatBoost model across 200 bootstrapping iterations. average gap between predicted probability and the true proportion of positives. A low ECE means that if the model predicts, for instance, a 40% chance of readmission, then roughly 40% of those patients do indeed return. MCE captures the largest such deviation across all probability bins, highlighting worst-case miscalibrations that could be critical in a high-stakes clinical workflow. Results show that Isotonic Regression achieves the lowest ECE (0.0318 \(\pm\) 0.01) and lowest MCE (0.4663 \(\pm\) 0.24). Beta and Platt Scaling do yield moderate improvements compared to the uncalibrated model, but they cannot match Isotonic Regression in terms of minimizing average and worst-case calibration error. The Friedman tests for ECE (\(\chi^{2}_{\text{[PERSON]}}\approx 236\)) and MCE (\(\chi^{2}_{\text{[PERSON]}}\approx 148\)) both yield \(p\)-values far below 0.01, indicating statistically significant differences among methods. Pairwise Nemenyi tests highlight that Isotonic Regression's ECE and MCE are significantly lower than those of Venn-Abers, the uncalibrated baseline, and sometimes Spline. in certain probability bins, creating large penalty spikes. This means the model is, on average, well-calibrated but can mispredict certain individual cases rather sharply. In sum, each calibration method presents a distinct trade-off. Isotonic Regression dominates in calibration error metrics (ECE, MCE), ensuring a tight alignment of predicted versus observed risk. However, it incurs higher LogLoss and occasionally lowers AUROC or AUPRC. Platt Scaling and Beta Calibration preserve near-top discrimination (AUROC, AUPRC) while also achieving consistently low Log Loss, indicating more balanced updates to probability scores. Spline calibration is moderately effective across all metrics, offering flexible piecewise corrections but does not stand out as best in any one category. Venn-Abers provides interval predictions with theoretical validity guarantees--a unique advantage in safety-critical settings--but has higher calibration errors. From a healthcare standpoint, the choice of calibration hinges on balancing the need for accurate high-risk identification (AUROC, AUPRC) with the demand for trustworthy probability statements (LogLoss, ECE, MCE). Figure 11's reliability plots illustrate these findings. The uncalibrated model slightly overestimates risk in mid- to high-probability bins. Platt Scaling, Beta, and Spline each temper this overconfidence, more closely aligning with the diagonal \"perfect calibration\" line. Although Isotonic stands out for particularly low calibration errors overall, its stepwise adjustments sometimes cause spikier changes in probability assignment, visible as flatter regions on the reliability curve. For contexts where stable, smoothly varying scores are preferred, such discontinuities might be less desirable despite the strong ECE performance.