Evaluation of Use of Diagnostic AI for Lung Cancer in Practice

Study Overview
=================
Brief Summary
-----------------
This study investigates ways of improving radiologists performance of the classification of CT-scans as cancerous or non-cancerous. Participants interact with an AI to classify CT-scans under three different conditions.

Detailed Description
-----------------
The three conditions are as follows: probabilistic classification, where the radiologist diagnoses scans using an AI cancer likelihood score; classification plus detection, where the radiologist see detecting lung nodules in addition to the AI's probabilistic classification score before making her own examination of the CT-scan; and classification with delayed detection, where the radiologist identifies regions of interest independently of the AI and then sees the AI's detected ROIs.

Official Title
-----------------
Evaluation of Use of Diagnostic AI for Lung Cancer in Practice

Conditions
-----------------
Lung Cancer

Intervention / Treatment
-----------------
* Behavioral: AI-human interaction


Participation Criteria
=================
Eligibility Criteria
-----------------
Inclusion Criteria: The participant performs radiology screenings professionally Exclusion Criteria: -
Sexes Eligible for Study
-----------------
All

Accepts Healthy Volunteers
-----------------
Accepts Healthy Volunteers

Study Plan
=================
How is the study designed?
-----------------

Design Details

Primary Purpose: Diagnostic
Allocation: Randomized
Intervention Model: Crossover Assignment
Masking: Single


Arms and Interventions

| Participant Group/Arm | Intervention/Treatment |
| --- | --- |
| Experimental: Probabilistic Classification<br>Radiologists see a score from 1-100 that represents the AI's prediction of whether the CT-scan comes from a patient with cancer or not before beginning their analysis of the scan. | Behavioral: AI-human interaction<br>* Exploring what kinds of AI-human interaction improve radiologists detection accuracy.<br>|
| Experimental: Classification Plus Detection<br>Radiologists see a score from 1-100 that represents the AI's prediction of whether the CT-scan comes from a patient with cancer or not before beginning their analysis of the scan. They also see ROIs identified by the AI that represent lung nodules. | Behavioral: AI-human interaction<br>* Exploring what kinds of AI-human interaction improve radiologists detection accuracy.<br>|
| Experimental: Classification With Delayed Detection<br>Radiologists see a score from 1-100 that represents the AI's prediction of whether the CT-scan comes from a patient with cancer or not before beginning their analysis of the scan. After identifying their own ROIs, the radiologist then can see ROIs identified by the AI that represent lung nodules before making final decisions. | Behavioral: AI-human interaction<br>* Exploring what kinds of AI-human interaction improve radiologists detection accuracy.<br>|

What is the study measuring?
-----------------
Primary Outcome Measures

| Outcome Measure | Measure Description | Time Frame |
| --- | --- | --- |
| Classification accuracy | This compares radiologists' classifications with the ground truth in the tested cases. | up to 4 months after initiation of evaluation of the test set | 

Secondary Outcome Measures

| Outcome Measure | Measure Description | Time Frame |
| --- | --- | --- |
| detection concordance | Evaluation of concordance between radiologists in the tested cases in detection of lung nodules > 4 mm | up to 4 months after initiation of evaluation of the test set |