# Machine Learning-Based Model for Emergency Department Disposition at a Public Hospital
## 4 Results  
The well-known performance evaluation metrics, which are accuracy, precision, recall, F1 score, and AUC scores, were used in this study. The performance evaluation metrics assess the effectiveness of each classifier on datasets when test data is given separately for classifier-based ML-based models. Higher values of performance metrics indicate a more successful classification. Performance evaluation metrics utilized True Positive (TP), False Positive (FP), False Negative (FN), and True Negative (TN) outcomes in the classification task. These metrics are calculated as follows:  
\[accuracy=\frac{TP+TN}{TP+TN+FP+FN} \tag{1}\]\[precision=\frac{TP}{TP+FP} \tag{2}\]  
\[recall=\frac{TP}{TP+FN} \tag{3}\]  
\[F1\ score=\frac{2\times Precision\times Recall}{Precision+Recall} \tag{4}\]  
\[AUC=\frac{1}{2}\left(\frac{TP}{TP+FN}+\frac{TN}{TN+FP}\right) \tag{5}\]  
The accuracy, precision, recall, F1 score, and AUC scores of our models are shown in Table 2. The AUC values used in our study for the seven models are also presented in Figure 2. RF was the model with the best performance according to all evaluation benchmarks. Considering all these benchmarks, the XGBoost model ranked second and CatBoost ranked third. The KNN, LR, and AdaBoost models had similar results. Additionally, there were certain occasions when these models were superior to each other in different metrics. For instance, while the LR model performed better than the AdaBoost model in the evaluation made according to the accuracy (acc) benchmark, the Adaboost model performed better than LR when evaluated according to the F-score benchmark. Considering that both true positive results (cases in which patients who really need to be hospitalized are hospitalized) and false positive results (cases in which patients who do not need to be hospitalized are hospitalized) are important, the F1-score benchmark, which was evaluated following these cases, was significant. To present details about the performance of the ML models, we introduced confusion matrices for LR, CatBoost, AdaBoost, XGBoost, LightGBM, RF, and KNN models, which are also given in Supplementary Figures S1-S8, respectively, in the Supplementary File. Figure 2: ROC curves for all models. The ranking in Table 3 presents the order in which the features determine the class label according to the Chi-squared method. The impact of class label determination, presented in Table 3, highlights the significance of each feature in contributing to the accurate determination of the class label. The variables PCT, prothrombin time, INR, and hospital arrival type in our models were not statistically significant. However, all of the other variables were statistically significant (Table 3). \begin{table}
\begin{tabular}{c c c c c c} \hline
**ML Model** & **Accuracy** & **Precision** & **Recall** & **F1 Score** & **Auc Score** \\ \hline CatBoost & 0.8306 & 0.8208 & 0.8306 & 0.8104 & 0.8928 \\ AdaBoost & 0.7626 & 0.6818 & 0.7192 & 0.7002 & 0.7680 \\ XGBost & 0.8460 & 0.8402 & 0.8462 & 0.8300 & 0.9234 \\ LightGBM & 0.8108 & 0.7942 & 0.8106 & 0.7832 & 0.8798 \\ RF & 0.9348 & 0.934 & 0.9332 & 0.9318 & 0.9584 \\ KNN & 0.7686 & 0.7432 & 0.7704 & 0.7504 & 0.8290 \\ LR & 0.7636 & 0.7212 & 0.7640 & 0.6954 & 0.6944 \\ \hline \end{tabular}
\end{table} 
Table 2: Performance metrics of the ML models.