Clinical Validation of Machine Learning Triage of Chest Radiographs

Study Overview
=================
Brief Summary
-----------------
Artificial intelligence and machine learning have the potential to transform the practice of radiology, but real-world application of machine learning algorithms in clinical settings has been limited. An area in which machine learning could be applied to radiology is through the prioritization of unread studies in a radiologist's worklist. This project proposes a framework for integration and clinical validation of a machine learning algorithm that can accurately distinguish between normal and abnormal chest radiographs. Machine learning triage will be compared with traditional methods of study triage in a prospective controlled clinical trial. The investigators hypothesize that machine learning classification and prioritization of studies will result in quicker interpretation of abnormal studies. This has the potential to reduce time to initiation of appropriate clinical management in patients with critical findings. This project aims to provide a thoughtful and reproducible framework for bringing machine learning into clinical practice, potentially benefiting other areas of radiology and medicine more broadly.

Official Title
-----------------
Clinical Validation of Machine Learning Triage of Chest Radiographs

Conditions
-----------------
Chest--Diseases

Intervention / Treatment
-----------------
* Other: Traditional workflow triage
* Other: Machine learning workflow triage
* Other: Random workflow triage


Participation Criteria
=================
Eligibility Criteria
-----------------
Inclusion Criteria: Radiologist at Stanford Hospital and Clinics Exclusion Criteria: None

Ages Eligible for Study
-----------------
Minimum Age: 18 Years

Sexes Eligible for Study
-----------------
All

Accepts Healthy Volunteers
-----------------
Accepts Healthy Volunteers

Study Plan
=================
How is the study designed?
-----------------

Design Details

Primary Purpose: Diagnostic
Allocation: Randomized
Intervention Model: Crossover Assignment
Interventional Model Description: Radiologists will triage chest radiographs using traditional, machine learning, and random methods.
Masking: Single


Arms and Interventions

| Participant Group/Arm | Intervention/Treatment |
| --- | --- |
| Active Comparator: Traditional workflow triage<br>Radiologists follow standard triage of chest radiographs. | Other: Traditional workflow triage<br>* Workflow triage is based on order location, STAT designation, and first-in-first-out status.<br>Other: Machine learning workflow triage<br>* Workflow triage is based on the machine learning model's confidence of abnormality.<br>Other: Random workflow triage<br>* Workflow triage is based on random order.<br>|
| Active Comparator: Machine learning workflow triage<br>Radiologists follow machine learning triage of chest radiographs. | Other: Traditional workflow triage<br>* Workflow triage is based on order location, STAT designation, and first-in-first-out status.<br>Other: Machine learning workflow triage<br>* Workflow triage is based on the machine learning model's confidence of abnormality.<br>Other: Random workflow triage<br>* Workflow triage is based on random order.<br>|
| Sham Comparator: Random workflow triage<br>Radiologists follow randomly ordered triage of chest radiographs. | Other: Traditional workflow triage<br>* Workflow triage is based on order location, STAT designation, and first-in-first-out status.<br>Other: Machine learning workflow triage<br>* Workflow triage is based on the machine learning model's confidence of abnormality.<br>Other: Random workflow triage<br>* Workflow triage is based on random order.<br>|

What is the study measuring?
-----------------
Primary Outcome Measures

| Outcome Measure | Measure Description | Time Frame |
| --- | --- | --- |
| Turnaround time | Time from completion of radiograph to time that radiologist issues an assessment via preliminary or final report | up to 1 hour |