Real-time Machine Learning Alerts to Prevent Escalation of Care: A Pragmatic Clinical Trial
Authors: Levin Matthew A. , Kia Arash , Timsina Prem , Cheng Fu-yuan , Nguyen Kim-Anh-Nhi , Kohli-Seth Roopa , Lin Hung-Mo , Ouyang Yuxia , Freeman Robert , Reich David L. , Ahmed Sanam , Zhao Shan , Gupta Rohit , Mazumdar Madhu , Klang Eyal 
Published: December 2022

Importance

Automated machine learning algorithms have been shown to outperform older methods in predicting clinical deterioration requiring escalation of care, but rigorous prospective data on their real-world efficacy are limited.

Objective

We hypothesized that real-time deterioration prediction alerts sent directly to front-line providers would reduce escalations.

Design

Single-center prospective pragmatic clinical trial conducted from July 2019 to March 2020. The trial was terminated early due to the COVID-19 pandemic. Patients were followed for 30 days post-discharge.

Setting

Academic tertiary care medical center located in New York City.

Participants

All adult patients admitted to any of four medical-surgical units were included. Assignment to intervention or control arms was determined by initial unit admission.

Intervention

Real-time alerts stratified according to predicted likelihood of clinical deterioration sent to the nursing/primary team or directly to the rapid response team. Clinical care and interventions were at the discretion of the providers. For the control units, alerts were generated but not sent.

Main Outcomes

The primary outcome was the incidence of escalation of care. Secondary outcomes included orders placed for cardiovascular support, in-hospital and 30-day mortality. Ad-hoc outcomes included time to ICU escalation and time to discharge.

Results

2,780 patients were enrolled, 1,506 in the intervention group and the 1,274 in the control group. Average age was 66.2 years and 1,446 (52%) of participants were female. There was no difference in escalation between the groups, relative risk(RR) 1.22(95% Confidence Interval[CI] (0.97,1.54),p=0.10). Patients in the intervention group were more likely to receive cardiovascular support orders RR 1.35(95% CI (1.10,1.66),p=0.022). Median time to escalation with alerts was 50.6 [21.6-103] versus 58.6 [25.4-115] hours (difference -5.70;95% CI (-10.00,-2.00),p<0.001). The hazard ratio for likelihood of ICU escalation within 12 hours of an alert was 3.36 (95% CI (1.38,8.21),p=0.01) and for faster hospital discharge was 1.10 (95% CI (1.01,1.19),p=0.02). Combined in-hospital and 30-day-mortality was lower in the intervention group, RR 0.72 (95% CI (0.54,0.94),p=0.01).

Conclusions and Relevance

Preliminary evidence suggests that real-time machine learning alerts do not reduce the incidence of escalation but are effective in reducing time to escalation, hospital length of stay and mortality.

Trial Registration

ClinicalTrials.gov,NCT04026555,https://clinicaltrials.gov

Key Points

Question

Can real-time machine learning generated alerts predicting clinical deterioration, sent directly to front-line providers, reduce escalations in care?

Findings

In this pragmatic clinical trial that included 2780 adults, that was terminated early due to the COVID-19 pandemic, the incidence of escalation among patients who received alerts was 11.2% versus 9.7% among patients who did not, a non-significant difference. Combined in-hospital and 30-day mortality in the alerts group was 6.9% versus 9.4% in the group with no alerts, a significant difference.

Meaning

Preliminary evidence suggests that real-time machine learning generated alerts do not reduce the incidence of escalation but may reduce mortality.