Machine Learning Sepsis Alert Notification Using Clinical Data

Study Overview
=================
Brief Summary
-----------------
Machine learning is a powerful method to create clinical decision support (CDS) tools, when training labels reflect the desired alert behavior. In our Phase I work for this project, we developed HindSight, an encoding software that was designed to examine discharged patients' electronic health records (EHRs), identify clinicians' sepsis treatment decisions and patient outcomes, and pass those labeled outcomes and treatment decisions to an online algorithm for retraining of our machine-learning-based CDS tool for real-time sepsis alert notification, InSight. HindSight improved the performance of InSight sepsis alerts in retrospective work. In this study, we propose to assess the clinical utility of HindSight by conducting a multicenter prospective randomized controlled trial (RCT) for more accurate sepsis alerts.

Detailed Description
-----------------
We will evaluate the performance of HindSight in a randomized controlled trial (RCT). HindSight is a novel encoding software designed to optimize alerts for sepsis alert notification. HindSight identifies clinicians' sepsis-related decisions in the electronic health records of former patients and passes those events to InSight, thus supplying InSight with labeled examples of true positive sepsis cases for retraining. In our retrospective work, we have shown that HindSight enables InSight to adapt to site-specific deviations of real-world clinical deployment by successfully reducing false and irrelevant alarms, without human supervision. The goal of this project is to demonstrate that the retrospective success of HindSight can be successfully translated to live clinical environments. In our Phase I work, HindSight achieved an area under the receiver-operating characteristic (AUROC) of 0.899, 0.831 and 0.877 for clinician sepsis evaluation, treatment, and onset, respectively. By using an online learning algorithm to incorporate HindSight-labeled data into the InSight predictor, we showed that the online-trained InSight can adapt to the HindSight-labeled data and outperform both baseline and periodically re-trained versions of InSight (p < 0.05). In Aim 1, we will prospectively validate HindSight's performance on real-time patient data streams in three diverse hospitals non-interventionally. In Aim 2, we will evaluate the effect of the tool in a prospective, interventional RCT. HindSight will first be evaluated by live deployment at four academic and community hospitals, during which time it will not provide alerts of future sepsis onset. Following any necessary algorithm optimization arising from live hospital validation, we will perform an RCT to evaluate reductions in false alerts from InSight trained on HindSight sepsis labels (experimental arm), compared to InSight trained on gold standard Sepsis-3 labels (control arm).

Official Title
-----------------
Using Clinical Treatment Data in a Machine Learning Approach for Sepsis Detection

Conditions
-----------------
Sepsis, Severe Sepsis, Septic Shock

Intervention / Treatment
-----------------
* Other: HindSight
* Other: InSight


Participation Criteria
=================
Eligibility Criteria
-----------------
Inclusion Criteria: During the study period, all patients over the age of 18 presenting to the emergency department or admitted to an inpatient unit at the participating facilities will automatically be enrolled in the study, until the enrollment target for the study is met Exclusion Criteria: Patients under the age of 18 Prisoners

Ages Eligible for Study
-----------------
Minimum Age: 18 Years

Sexes Eligible for Study
-----------------
All

Accepts Healthy Volunteers
-----------------
Accepts Healthy Volunteers

Study Plan
=================
How is the study designed?
-----------------

Design Details

Primary Purpose: Other
Allocation: Randomized
Intervention Model: Parallel Assignment
Masking: Triple


Arms and Interventions

| Participant Group/Arm | Intervention/Treatment |
| --- | --- |
| Experimental: Experimental<br>The experimental arm will involve patients monitored by HindSight. | Other: HindSight<br>* HindSight will examine the dynamic trends of clinical measurements taken from a patient's EHR and analyzes correlations between vital signs to alert for the onset of sepsis.This machine learning based tool is optimized by encoder and utilizes periodic retraining to improve its performance over time.<br>* Other names: HindSight-Clinical decision support (CDS) system for sepsis alert notification;|
| Active Comparator: Control<br>The control arm will involve patients monitored by InSight. | Other: InSight<br>* Compared to the ability of the InSight software's recognition of sepsis onset to HindSight's performance. The study determines if the HindSight software has equivalent or better performance than the InSight software.<br>|

What is the study measuring?
-----------------
Primary Outcome Measures

| Outcome Measure | Measure Description | Time Frame |
| --- | --- | --- |
| Rate of reduction in false alerts | The primary outcome measure of interest will be false alert reduction. Successful completion of Aim 1 will be demonstrated by a positive predictive value (PPV) in a live clinical setting for which the lower bound of the 95% confidence interval meets or exceeds the benchmark from prior retrospective studies. Meeting the retrospective PPV benchmark indicates that prospective CDS quality reflects retrospective CDS quality, and is sufficiently high to reduce alarm fatigue and improve clinical utility. Success of Aim 2 is contingent upon achieving a 15% relative reduction in false alerts when comparing between the two treatment arms (p < 0.05; Fisher's Exact Test). | Through study completion, human subjects involvement will occur for an average of eight months | 

 Terms related to the study
=================
Keywords Provided by Centre Hospitalier Valida
-----------------
Sepsis, Machine learning algorithm, Clinical decision support