# Machine Learning for Early Outcome Prediction in Septic Patients in the Emergency Department
## 4 Discussion  
In this study, we confirmed that interpretable and scalable machine learning models may outperform published qSOFA, APACHE II, and SOFA scores in identifying patients at highest risk of mortality. We demonstrated that random forest achieved the highest discriminative performance compared to standard statistical techniques such as balanced and unbalanced logistic regression. Our study population was composed of elderly and frail patients (median age 77 [IQR 69; 70; 71; 72; 73; 74; 75; 76; 77; 78; 79]), with a median age in non-survivors of 83 [IQR 75; 76; 77; 78; 79]) and several baseline comorbidities. Given the frailty of this population, mortality was high, reaching 15.3% in the whole population, similarly to mortality reported in other studies on septic patients admitted to the ED, resulting in higher sepsis mortality compared to other hospital populations [10; 31]. This is due to the specific characteristics of the ED patient population, a \begin{table}
\begin{tabular}{l c c c c c c c c c c} \hline \hline
**Model** & **F1** & **F1** & **Precision** & **Precision** & **ACC** & **ACC** & **Recall** & **Recall** & **AUC** & **AUC** \\  & **Test** & **Train** & **Test** & **Train** & **Test** & **Train** & **Test** & **Train** & **Test** & **Train** \\ \hline Dummy\_strat & 0.810 & 0.853 & 0.833 & 0.844 & 0.688 & 0.747 & 0.787 & 0.861 & 0.524 & 0.443 \\ Dummy\_strat * & 0.858 & 0.834 & 0.847 & 0.842 & 0.758 & 0.721 & 0.870 & 0.825 & 0.414 & 0.506 \\ LR & 0.913 & 0.930 & 0.861 & 0.869 & 0.844 & 0.872 & 0.972 & 1.000 & 0.813 & 0.845 \\ LR\({}^{*}\) & 0.913 & 0.932 & 0.861 & 0.872 & 0.844 & 0.875 & 0.972 & 1.000 & 0.826 & 0.843 \\ LR\_balanced & 0.856 & 0.932 & 0.925 & 0.943 & 0.773 & 0.886 & 0.796 & 0.921 & 0.811 & 0.881 \\ LR\_balanced * & 0.866 & 0.930 & 0.935 & 0.943 & 0.789 & 0.882 & 0.806 & 0.917 & 0.818 & 0.893 \\ RF & 0.847 & 0.866 & 0.943 & 0.970 & 0.766 & 0.795 & 0.769 & 0.782 & 0.863 & 0.923 \\ RF * & 0.837 & 0.881 & 0.932 & 0.971 & 0.750 & 0.815 & 0.759 & 0.806 & 0.834 & 0.914 \\ SOFA & 0.739 & 0.771 & 0.895 & 0.899 & 0.625 & 0.660 & 0.630 & 0.675 & 0.712 & 0.698 \\ APACHE II & 0.775 & 0.749 & 0.892 & 0.891 & 0.664 & 0.633 & 0.685 & 0.647 & 0.664 & 0.655 \\ qSOFA & 0.783 & 0.764 & 0.913 & 0.912 & 0.680 & 0.656 & 0.685 & 0.706 & 0.658 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Performance of random forest, balanced and unbalanced logistic regression, and simple classifier as the baseline, in training and test set. Figure 2: Feature importance for random forest (**left**) and odds ratio for balanced (**middle**) and unbalanced (**right**) logistic regression model. mixed group that includes very frail patients from long-term care facilities with a poor physiological reserve and baseline prognosis. In contrast, the cohort of septic hospitalized patients often includes younger patients and surgical patients, which are characterized by lower baseline frailty [10]. The Acute Physiology and Chronic Health Evaluation II (APACHE II) and the Sequential Organ Assessment Score are among the most diffuse predictive scores for mortality, and were used as a reference in this population [18; 32]. The qSOFA, an early reduced version of the SOFA score proposed by the Surviving Sepsis Campaign for screening purposes, was also included as a comparison in the model. The SOFA score achieved an intermediate discriminative performance (AUC= 0.712), which was slightly higher than the performance of the APACHE II score (AUC = 0.647). The qSOFA score had an AUC of 0.706. qSOFA, SOFA, and APACHE II were superior to the dummy classifier (a simple model which classifies patients according to chance, and included as a reference only). In contrast, they performed worse than logistic regression, balanced logistic regression, and random forest. Random forest may thus represent a simple solution to approaching data in sepsis ED patients, and outperforms standard statistical scores.