12 Early Mortality Risk Assessment Modeling by Random Forest Analysis of Patients Presenting to a Tertiary Medical Center Emergency Department

Study ObjectivesIdentifying patients at higher risk for in-hospital mortality can inform shared decision making and Goals of Care (GOC) discussions. Traditionally, logistic regression (LR) has been used to develop mortality risk models, such as the sequential organ failure assessment (SOFA) and acute physiology and chronic health evaluation (APACHE) scores. The development of machine learning (ML) methods of analysis can yield higher area under receiver operating cure when compared to LR models and has been shown to generate predictive results much earlier in hospitalization than feasible previously. Unfortunately, some current risk assessment scores require lab values which are not frequently collected in the emergency department (ED), use truncated data sets for feasibility, or require serial comparisons which are not available within the first several hours of evaluation. The widespread use of electronic health record (EHR) systems allows for integrated multivariate ML models which can provide earlier predictions of mortality while decisions are still being made during a patient’s initial care. The goal of this study is to develop an all-cause mortality prediction model, derived from commonly collected ED data, which can assess mortality risk early during an ED encounter.MethodsData was obtained for all patients age 18 and older admitted to Wake Forest Baptist Medical Center, now Atrium Health Wake Forest Baptist, from April 1, 2016 through March 31, 2020. A total of 77,119 patients met inclusion criteria, of these 52.8% were male and 47.2% were female. Initial vital signs including heart rate, respiratory rate, systolic blood pressure, diastolic blood pressure, mean arterial pressure, pulse oximetry, weight, and body mass index were electronically retrieved for all patients as well as all components of a comprehensive metabolic panel, and a complete blood count not including cell differentials. Random forest (RF) analysis, cross validation, and area under the curve analysis was applied to a training sample of 70% of the data set and with the remaining 30% being used as a test data set. Missing predictor data were imputed using RF imputation. Missing indicator variables were included in data models, as the presence of missing data can independently correlate with mortality. The Rapid Emergency Medicine Score (REMS) was used as a comparator for performance analysis since it contains similar variables and does not include values derived from blood gas analysis, as utilized by SOFA and APACHE.ResultsA full RF model of 68 variables, using 34 measured variables and 34 missing indicator variables yielded an AUC of 0.892 (0.876, 0.908) and a reduced model of 18 variables, using 12 measured variables and 6 missing indicator variables, yielded an AUC of 0.857 (0.838, 0.876) Table 1. Both the full and reduced RF models outperformed REMS in this dataset which yielded an AUC of 0.500 (0.457, 0.543).ConclusionsOur RF model utilizing commonly used vital sign and laboratory data obtained on initial ED assessment is predictive of in-hospital mortality and was superior to REMS for this data set. Notably REMS was validated specifically in patients admitted to intensive care units which may partially account for the difference in performance between data models.No, authors do not have interests to disclose Study ObjectivesIdentifying patients at higher risk for in-hospital mortality can inform shared decision making and Goals of Care (GOC) discussions. Traditionally, logistic regression (LR) has been used to develop mortality risk models, such as the sequential organ failure assessment (SOFA) and acute physiology and chronic health evaluation (APACHE) scores. The development of machine learning (ML) methods of analysis can yield higher area under receiver operating cure when compared to LR models and has been shown to generate predictive results much earlier in hospitalization than feasible previously. Unfortunately, some current risk assessment scores require lab values which are not frequently collected in the emergency department (ED), use truncated data sets for feasibility, or require serial comparisons which are not available within the first several hours of evaluation. The widespread use of electronic health record (EHR) systems allows for integrated multivariate ML models which can provide earlier predictions of mortality while decisions are still being made during a patient’s initial care. The goal of this study is to develop an all-cause mortality prediction model, derived from commonly collected ED data, which can assess mortality risk early during an ED encounter. Identifying patients at higher risk for in-hospital mortality can inform shared decision making and Goals of Care (GOC) discussions. Traditionally, logistic regression (LR) has been used to develop mortality risk models, such as the sequential organ failure assessment (SOFA) and acute physiology and chronic health evaluation (APACHE) scores. The development of machine learning (ML) methods of analysis can yield higher area under receiver operating cure when compared to LR models and has been shown to generate predictive results much earlier in hospitalization than feasible previously. Unfortunately, some current risk assessment scores require lab values which are not frequently collected in the emergency department (ED), use truncated data sets for feasibility, or require serial comparisons which are not available within the first several hours of evaluation. The widespread use of electronic health record (EHR) systems allows for integrated multivariate ML models which can provide earlier predictions of mortality while decisions are still being made during a patient’s initial care. The goal of this study is to develop an all-cause mortality prediction model, derived from commonly collected ED data, which can assess mortality risk early during an ED encounter. MethodsData was obtained for all patients age 18 and older admitted to Wake Forest Baptist Medical Center, now Atrium Health Wake Forest Baptist, from April 1, 2016 through March 31, 2020. A total of 77,119 patients met inclusion criteria, of these 52.8% were male and 47.2% were female. Initial vital signs including heart rate, respiratory rate, systolic blood pressure, diastolic blood pressure, mean arterial pressure, pulse oximetry, weight, and body mass index were electronically retrieved for all patients as well as all components of a comprehensive metabolic panel, and a complete blood count not including cell differentials. Random forest (RF) analysis, cross validation, and area under the curve analysis was applied to a training sample of 70% of the data set and with the remaining 30% being used as a test data set. Missing predictor data were imputed using RF imputation. Missing indicator variables were included in data models, as the presence of missing data can independently correlate with mortality. The Rapid Emergency Medicine Score (REMS) was used as a comparator for performance analysis since it contains similar variables and does not include values derived from blood gas analysis, as utilized by SOFA and APACHE. Data was obtained for all patients age 18 and older admitted to Wake Forest Baptist Medical Center, now Atrium Health Wake Forest Baptist, from April 1, 2016 through March 31, 2020. A total of 77,119 patients met inclusion criteria, of these 52.8% were male and 47.2% were female. Initial vital signs including heart rate, respiratory rate, systolic blood pressure, diastolic blood pressure, mean arterial pressure, pulse oximetry, weight, and body mass index were electronically retrieved for all patients as well as all components of a comprehensive metabolic panel, and a complete blood count not including cell differentials. Random forest (RF) analysis, cross validation, and area under the curve analysis was applied to a training sample of 70% of the data set and with the remaining 30% being used as a test data set. Missing predictor data were imputed using RF imputation. Missing indicator variables were included in data models, as the presence of missing data can independently correlate with mortality. The Rapid Emergency Medicine Score (REMS) was used as a comparator for performance analysis since it contains similar variables and does not include values derived from blood gas analysis, as utilized by SOFA and APACHE. ResultsA full RF model of 68 variables, using 34 measured variables and 34 missing indicator variables yielded an AUC of 0.892 (0.876, 0.908) and a reduced model of 18 variables, using 12 measured variables and 6 missing indicator variables, yielded an AUC of 0.857 (0.838, 0.876) Table 1. Both the full and reduced RF models outperformed REMS in this dataset which yielded an AUC of 0.500 (0.457, 0.543). A full RF model of 68 variables, using 34 measured variables and 34 missing indicator variables yielded an AUC of 0.892 (0.876, 0.908) and a reduced model of 18 variables, using 12 measured variables and 6 missing indicator variables, yielded an AUC of 0.857 (0.838, 0.876) Table 1. Both the full and reduced RF models outperformed REMS in this dataset which yielded an AUC of 0.500 (0.457, 0.543). ConclusionsOur RF model utilizing commonly used vital sign and laboratory data obtained on initial ED assessment is predictive of in-hospital mortality and was superior to REMS for this data set. Notably REMS was validated specifically in patients admitted to intensive care units which may partially account for the difference in performance between data models.No, authors do not have interests to disclose Our RF model utilizing commonly used vital sign and laboratory data obtained on initial ED assessment is predictive of in-hospital mortality and was superior to REMS for this data set. Notably REMS was validated specifically in patients admitted to intensive care units which may partially account for the difference in performance between data models.