## 2 Materials and Methods
### Clinically Oriented Evaluation Protocol  
The clinically oriented evaluation protocol is designed to bridge the gap between high statistical performance and the nuanced requirements of patient-centered healthcare. In this protocol, two composite metrics are defined that integrate standard performance measures with clinical priorities. These metrics are central to the clinically oriented evaluation framework and are used to both select and validate models in a manner that aligns with clinical decision making. The first composite metric is the overall model score, which is defined as  
\[\text{Score}(m_{\theta})=\alpha_{1}\cdot\text{AUC}(m_{\theta})+\alpha_{2} \cdot\text{Sensitivity}(m_{\theta})+\alpha_{3}\cdot\text{Specificity}(m_{\theta}), \tag{20}\]  
where:  
* \(\text{AUC}(m_{\theta})\) is the Area Under the Receiver Operating Characteristic (ROC) Curve, computed by \[\text{AUC}=\int_{0}^{1}\text{TPR}(FPR)\,dFPR,\] (21) with the True-Positive Rate (TPR) defined as \[\text{TPR}=\frac{TP}{TP+FN},\] (22) and the False-Positive Rate (FPR) given by \[\text{FPR}=1-\frac{TN}{TN+FP}.\] (23)
* Sensitivity(\(m_{\theta}\)), which is equivalent to the TPR, quantifies the model's ability to correctly detect positive cases. * Specificity(\(m_{\theta}\)) is defined as \[\text{Specificity}=\frac{TN}{TN+FP},\] (24) and reflects the model's capacity to correctly identify negative cases. * The weighting coefficients \(\alpha_{1}\), \(\alpha_{2}\), and \(\alpha_{3}\) are selected based on clinical priorities; for instance, in many healthcare applications, a higher weight is assigned to sensitivity to minimize the risk of missing a critical diagnosis. For example, in cancer screening tasks such as mammography for breast cancer detection, prioritizing sensitivity is important to ensure that cases of malignancy are not missed, as a false negative could result in delayed treatment and adverse outcomes [21]. Similarly, in diabetic retinopathy screening, high sensitivity is often emphasized to minimize the risk of overlooking patients who require urgent ophthalmologic intervention [22]. These examples underscore why, in many healthcare applications, the weighting of sensitivity over other metrics is not only justified but necessary for patient safety. Equation (20) is important in ML for healthcare because it integrates global discriminative performance (via AUC) with more clinically meaningful sensitivities and specificities. This combination ensures that a high overall score is achieved only when the model both discriminates well across classes and meets the clinical requirements for minimizing false negatives and false positives. The second metric, referred to as the _clinical utility_ score, is defined as  
\[\text{Clinical Utility}=\omega_{1}\cdot\text{PPV}+\omega_{2}\cdot\text{ NPV}+\omega_{3}\cdot\text{F1}, \tag{25}\]  
where the components are defined as follows:  
\[\text{PPV}=\frac{TP}{TP+FP}, \tag{26}\]  
\[\text{NPV}=\frac{TN}{TN+FN}, \tag{27}\]  
\[F1=2\cdot\frac{\text{PPV}\cdot\text{TPR}}{\text{PPV}+\text{TPR}}. \tag{28}\]  
Here, TPR is as defined in Equation (22). Equation (25) is pivotal in healthcare ML because it consolidates three key aspects of clinical performance. By integrating PPV, NPV, and the F1 score, the equation directly correlates the algorithm's predictions with outcomes that are of clinical importance. This is particularly critical in scenarios where the consequences of misclassification are high, and there is a need to balance the trade-offs between over-diagnosis and under-diagnosis. Together, Equations (20) and (25) form a dual-layered evaluation framework. The first layer (Equation (20)) assesses the inherent discriminative power of the model while accounting for statistical robustness. The second layer (Equation (25)) translates statistical performance into clinically actionable insights by emphasizing the predictive values and balanced accuracy.