# Machine Learning for Early Outcome Prediction in Septic Patients in the Emergency Department
## 2 Materials and Methods  
We conducted an electronic health record study. We extracted de-identified data from hospital electronic health records of all patients admitted to a tertiary clinical center, over 18 months. Study received ethical approval from Humanitas Research Hospital independent Ethical Committee. We identified septic patients according to Angus criteria and Agency for Healthcare Research and Quality (AHRQ) criteria, two tools frequently adopted for studies using EHR and administrative data [20; 21; 22; 23]. Accordingly, International Classification of Diseases, Ninth Edition, Clinical Modification (ICD-9-CM) codes of patients admitted from the ED were extracted from EHR data, and screened for Angus or AHRQ criteria [21; 22; 23]. In these criteria, there are two ways for a hospitalization to become selected by Angus. The patient is selected when (a) explicit ICD-9 codes for severe sepsis or septic shock are found or (b) there is an ICD-9 diagnosis code for infection. Detailed AHRQ criteria according to ICD9 codes are available on the AHRQ website [23]. Full EHR data of patients identified through these criteria were subsequently re-evaluated in depth by a doctor according to Sepsis Criteria before inclusion in the study [24]. Criteria are reported in Supplemental Material File S1. All patients admitted to the ED were treated according to Surviving Sepsis Campaign guidelines [25]. The site and type of underlying infection were determined by clinical signs and symptoms, medical history, laboratory exams, and biological cultures. The primary study goal was to assess whether an interpretable and scalable machine learning model can predict mortality in septic patients in the ED better than SOFA, qSOFA, and APACHE II scores, and to compare the model to logistic regression and a dummy classifier. To evaluate the models produced and to assess their performance, we chose to describe and analyze five different parameters: precision, recall, accuracy, receiver operating characteristic (ROC) curve with area under the curve (AUC), and F1-score. Precision is defined as the number of true septic patients divided by the total number of positive predictions; recall quantifies how many true positives were retrieved by our models. F1-score represents a harmonic mean of precision and recall. Accuracy is the number of correctly predicted data across both positive and negative classes, while ROC and its AUC are measures of the ability of a classifier to distinguish between classes. The secondary objective was the descriptive analysis of epidemiological and microbiological characteristics of the study population, specifically on covariates that were available in the EHR (a list of all the covariates can be found in the Supplemental Materials Table S1). **Statistical analysis**  
Categorical data are described as frequency and percentage. Continuous data are expressed as the median and interquartile range (IQR). We tested the association between independent variable and outcome at univariate analysis, using the chi-squaredtest for categorical variables or the Wilcoxon rank-sum test as a non-parametric test for continuous variables. Missing data were explored for visual patterns of missingness as reported in Supplemental Material Figure S1, and considered missing at random (MAR). To account for missing data, we used a chained random forest to perform imputation of missing data using the missRanger package [26], S, which basically imputes each variable using predictions from a random forest that includes all other variables as covariates, and iterating this process multiple times until the process stops improving. Results of the imputation process are reported in Supplemental Material Figure S2. A baseline reference model was created using a dummy classifier, which is a simple uniform classifier commonly employed as a baseline in ML models [27].