## 3 Overview of the U-Smile Method  
The U-smile method [6; 7] is a novel approach to evaluate whether a new variable added to a baseline model for binary classification improves its prediction separately for both outcome classes (Figure 1). It was designed to address the limitations of traditional evaluation metrics, which often produce a single global measure of model performance and do not provide a separate assessment for both outcome classes, as well as lack an interpretable graphical summary of results. Additionally, the U-smile method combines a quantitative and visual evaluation of the usefulness of a new predictor. The U-smile method analyzes prediction errors and does not influence model training. The U-smile method compares two nested models: a reference model, built using a baseline set of variables, and a new model that includes an additional candidate variable. For each individual, a prediction is made under both models, and the difference between the predicted probability and the true outcome, called the residual, is calculated. These residuals reflect prediction errors and form the basis for assessing whether the new variable led to better or worse predictions. An improved prediction is defined as a smaller prediction error (residual) under the new model compared to the reference model; a worsened prediction is defined as a larger prediction error. The effect of the new variable at the individual level is visualized through a prediction improvement-worsening (PIW) plot with reference probabilities on the x-axis and new probabilities on the y-axis. Then, through cross-tabulating the outcome class (non-event or event) with the direction of prediction change (better or worse), each individual falls into one of the four subclasses of the PIW matrix:  
* Non-events with improved prediction,
* Non-events with worsened prediction,
* Events with worsened prediction,
* Events with improved prediction. To quantify prediction changes, the U-smile method uses three families of coefficients:
* BA coefficients express average absolute changes in prediction between new and reference models,
* RB coefficients express relative changes in prediction between new and reference models relative to the prediction error of the reference model,
* I coefficients express the proportions of individuals with prediction changes. * These coefficients are calculated at three levels of generality:Figure 1: A step-by-step guide to the U-smile method. Superscripts \(+\) and \(-\) indicate better or worse prediction in the new model compared to the reference model, respectively. Subscripts \(0\) and \(1\) refer to the non-event and event outcome classes, respectively. BA coefficients: average absolute changes in prediction between new and reference models; RB coefficients: relative changes in prediction between new and reference models (relative to the prediction error of the reference model); I coefficients: proportions of the prediction changes in each class. The U-smile method evaluates the added predictive value of a candidate variable by comparing residuals (prediction errors) from two nested models: a reference model and a new model that includes the candidate variable. In **Step 1**, these changes are cross-tabulated with the outcome class (non-event or event) to create a prediction improvement-worsening (PIW) matrix, dividing individuals into four subclasses: non-events with improved prediction (\(0^{+}\)), non-events with worsened prediction (\(0^{-}\)), events with worsened prediction (\(1^{-}\)), and events with improved prediction (\(1^{+}\)). **Step 2** represents the three-level approach of the U-smile method, i.e., subclass-specific, class-specific, and overall. At level \(1\), for each subclass, the magnitude of prediction change is quantified as BA and RB coefficients, and the number of individuals in quantified as I coefficients. At level \(2\), for each class, the net coefficients are defined as differences of the subclass-specific coefficients of level \(1\), i.e., improvement coefficient less worsening coefficient.