# Predicting the Survival of Primary Biliary Cholangitis Patients
## 3 Materials and Methods
### Evaluation  
During this phase, several metrics were used to evaluate the performance of the different DMTs developed at the previous stage, as well as to determine the quality and reliability of the results. Since this study fits into a binary classification scope, the applied metrics were derived from the Confusion Matrix, which is a predictive classification table that contains the number of True Positives (TP), True Negatives (TN), False Positives (FP), and False Negatives (FN). The evaluation metrics extracted from the Confusion Matrix were Accuracy, Precision, Sensitivity, and Specificity. In addition, the Area Under the ROC (Receiver Operating Characteristic) Curve (AUC) was also used to assess the performance of the models. The metrics chosen are the most commonly used when addressing this type of problems; specifically, the sensitivity measure is the most important in domains, where FNs have a high cost, such as medical diagnosis. Accuracy, mathematically defined according to Equation (4), represents the ratio between the instances that the model was able to correctly classify and all the classified  
Figure 6: (**a**) Weight by Information Gain Ratio. (**b**) Feature weight analysis. instances. In this study, this metric refers to the number of PBC patients' status that were correctly classified either as surviving or dying out of all the patients [23]:  
\[Accuracy=\frac{TP+TN}{TP+FP+FN+TN} \tag{4}\]  
Precision, expressed by Equation (5), measures a classifier's exactness and is the ratio of the positive instances that were correctly classified by the model among all the positive instances classified by the model, i.e., it provides information on how many positive values were actually correctly predicted [23]. This metric evaluates the ability of the model to identify PBC patients who will not survive, i.e., the fraction of PBC patients who are classified as non-surviving and who have actually died. \[Precision=\frac{TP}{TP+FP} \tag{5}\]  
Sensitivity, also known as Recall, is calculated through Equation (6) and measures a classifier's completeness. It is associated with the ratio between the positive instances correctly predicted by the model and all the actual positive instances [23]. In the context of this study, this metric evaluates the ability of the model to correctly identify non-surviving PBC patients, i.e., the fraction of PBC patients classified as dying, labelled as 1, who actually died among all PBC patients who did not survive, described as 1 in the dataset. Sensitivity is therefore a good measure to evaluate models in domains where there is a high cost associated with FN, as it is with medical diagnosis, where it is harmful to predict that a PBC patient will not die when, in fact, he/she will die (FN). Thus, the closer the sensitivity is to 100% the better because a higher FN value means that the survival of some patients was incorrectly predicted, leading physicians to neglect intensive treatments or therapeutics that could save the patient's life. Therefore, and because of the critical nature of this problem, FN must be avoided at all costs. \[Sensitivity=\frac{TP}{TP+FN} \tag{6}\]  
Specificity, mathematically described by Equation (7), measures the ratio of negative instances correctly predicted by the model and all the negative instances, i.e, it provides information on how many negative values were actually correctly predicted [23]. In this study, the metric informs the proportion of surviving PBC patients that the model was able to classify correctly, evaluating the ability of the model to identify the PBC patients who will survive, i.e., the patients labelled as 0. A higher FP value means that the non-survival of patients was incorrectly predicted.