# Machine Learning for Evaluating Hospital Mobility: An Italian Case Study
## 3 Materials and Methods
### Data Measurements  
In our study, we developed a predictive model to classify hospitals in 2021 into three mobility categories: LOW, MEDIUM, and HIGH, based on the distance patients traveled. Evaluating the model's effectiveness is crucial for understanding its potential to support clinical decisions and patient management strategies. The model's performance is evaluated using several crucial metrics to ensure accurate and reliable predictions:  
AUC-ROC (area under the curve-receiver operating characteristics): This metric assesses the discriminative ability of the model across the outcome classes, either individually \begin{table}
\begin{tabular}{c c c c} \hline
**MOBILITY LEVEL** & **Counts** & **\% of Total** & **Cumulative \%** \\ \hline
**HIGH** & 17 & 13.4\% & 13.4\% \\
**LOW** & 81 & 63.8\% & 77.2\% \\
**MEDIUM** & 29 & 22.8\% & 100.0\% \\ \hline \end{tabular}
\end{table} 
Table 4: Frequencies of target variable mobility level. or in an aggregated manner through macro or micro averaging. A value of AUC-ROC equal to 1 indicates a perfect classification, while a value of 0.5 indicates a random classification. \[AUC_{macro}=\frac{1}{n}\sum_{i=1}^{n}AUC_{i} \tag{3}\]  
where \(n\) is the total number of classes and \(AUC_{i}\) is the \(AUC\) calculated for class \(i\). For a specific class \(i\) in a multiclass context, where class \(i\) is considered positive and all other classes are combined as negative, the \(AUC\), denoted as \(AUC_{i}\), is calculated through the following integral:  
\[AUC_{i}=\int_{0}^{1}TPR_{i}[FPR_{i}^{-1}(u)]du \tag{4}\]  
Here, _TPRi_ and _FPRi_ are, respectively, the true positive rate and false positive rate for class \(ii\), calculated across varying decision thresholds \(u\). This integral covers all possible decision thresholds, providing a comprehensive measure of predictive accuracy for class \(i\). Accuracy: Accuracy measures the percentage of correctly classified cases relative to the total number of cases. It is a general metric that provides an overview of the overall effectiveness of the model. \[Accuracy=\frac{\sum_{i=1}^{n}True\;Positives_{i}+\sum_{i=1}^{n}True\;Negatives _{i}}{\sum_{i=1}^{n}Total\;Population_{i}} \tag{5}\]  
Precision: Precision measures the proportion of true positive cases correctly identified relative to all cases identified as positive by the model. It quantifies the accuracy of positive predictions made by the model. \[Precision_{i}=\frac{TruePositives_{i}}{True\;Positives_{i}+\sum_{j=1,j\
eq i}^{ n}False\;Positives_{ji}} \tag{6}\]  
where \(False\;Positives_{ji}\) is the number of times that class \(j\) was incorrectly predicted as class \(i\). Recall (sensitivity): Recall measures the proportion of true positive cases correctly identified relative to all actually positive cases. It quantifies the model's ability to capture all positive cases, minimizing false negatives. \[Recall_{i}=\frac{TruePositives_{i}}{True\;Positives_{i}+\sum_{j=1,j\
eq i}^{ n}False\;Negatives_{ij}} \tag{7}\]  
where \(False\;Negatives_{ij}\) is the number of times that class ii was incorrectly not predicted as class \(i\). F1 score: The F1 score is a harmonic mean of precision and recall. This metric is useful when a balance between precision and recall is desired, and a single measure is sought to evaluate the model's performance. \[F1\;Score_{i}=2\times\frac{Precision_{i}\times Recall_{i}}{Precision_{i}+Recall _{i}} \tag{8}\]  
The overall F1 score can be calculated by averaging the F1 scores for each class. Matthews correlation coefficient (\(MCC\)): \(MCC\  
\(C\) is the sum of all elements on the diagonal of the confusion matrix (correct predictions). \(S\) is the total sum of all elements in the confusion matrix. \(P_{k}\) is the sum of all elements in column k of the confusion matrix (predictions for class \(k\)). \(T_{k}\) is the sum of all elements in row k of the confusion matrix (actual values for class \(k\)).