# Presentation: 918885

## Clinical Description – Part I
William Sacks, PhD, MD--ODE

**Clinical ****Description**** – ****Part I**
**William Sacks, PhD, MD--ODE**

**R2 Technology, Inc.**
** ****ImageChecker CT CAD Software System*****                  *****P030012**
**Radiological Devices Advisory Panel**
**February 3, 2004**

- Sacks

- 2/3/04

**	****  ****	**** ****Outline**
** ****Character of device**** **
** ****Clinical utility**
** ****Instructions for use**** ****	**** **
** ****Issues new to this device**

- Sacks

- 2/3/04

**	****  ****Character of device**
** ****For chest **_**CT**_** scans**
** ****For CTs done **_**for any indication**_
** ****Detects **_**solid**_** lung nodules**
**  ****E.g., not ground glass opacities**** (****GGOs)**** ****	****                         **
** ****Nodules between 4-30 mm**
** ****Computer-Aided **_**Detection**_** (CAD) **

- Sacks

- 2/3/04

## Computer-Aided Detection(CAD)/Diagnosis(CADx) 

**Computer-Aided Detection(CAD)/Diagnosis(CADx)**** **

**CAD                    CADx**

| Detection | Discrimination |
| --- | --- |
| Scans entire image | Scans only portion selected by user |
| Marks images | Gives level of suspicion |

- Sacks

- 2/3/04

** ****	****	**
**	****     ****Clinical Utility**
** ****Many nodules missed in clinical ****	****practice: **
** ****other pathology distracts**
** ****hundreds of images per exam**
** ****CAD intended to reduce missed nodules**

- Sacks

- 2/3/04

**	****  **
**	****  ****Instructions for Use**
** ****Reader reviews films unaided first **
** ****CAD marks candidate nodules**
** ****Reader looks again in vicinity of marks ****	**** **
** ****If CAD fails to mark a nodule judged actionable****	****on unaided review, reader should **_***retain***_** ****	****initial judgment**

- Sacks

- 2/3/04

**Issues new to this device**
** ****CAD target**
** ****Definition of “truth”**
** ****Unit of analysis ****	**** **
** ****Endpoints**

- Sacks

- 2/3/04

**CAD TARGET**
**Not malignant nodule**
**but**
**Actionable nodule**

- Sacks

- 2/3/04

**DEFINITION OF “TRUTH”**
**Not biopsy (tissue histology)**
**but**
**Expert panel**

- Sacks

- 2/3/04

**  ****UNIT OF ANALYSIS**
**Statistical unit = person**** **
**further broken down into**
**Lung quadrant**

- Sacks

- 2/3/04

**	****	****  ****ENDPOINTS **
_**Sensitivity**_** and **_**specificity**_** (or **_**FPR**_**) of action recommendation**** **
**and/or**

**MRMC ROC**

- Sacks

- 2/3/04

**CLINICAL STUDY**
**  ****3 ****expert radiologists on panel to ****	****	****   ****	****  determine “truth” for each nodule**
**  ****15**** other radiologists with range of experience,****	****   called “the readers” **
**  ****90**** subjects (****360**** lung quadrants)****	**
**  ****15 readers used ****100-point scale**** for****	**** ****	****  ****	****   confidence-in-actionability of each case ****	**** **

- Sacks

- 2/3/04

**	****   **
**	****CLINICAL STUDY**
**	****	****Presentation by **
**Nicholas Petrick, Ph.D.****	**** **

- Sacks

- 2/3/04

## Clinical Conclusions – Part II
William Sacks, PhD, MD--ODE

**Clinical ****Conclusions**** – ****Part II**
**William Sacks, PhD, MD--ODE**

**R2 Technology, Inc.**
** ****ImageChecker CT CAD Software System*****                  *****P030012**
**Radiological Devices Advisory Panel**
**February 3, 2004**

- Sacks

- 2/3/04

**What does gain in A****z**  **of 0.02 ****(95% CI: 0.01, 0.04)**** ****mean clinically?**

- Sacks

- 2/3/04

**When CAD is used according to instruction to retain all judgments of actionability even if unmarked by CAD...**

- Sacks

- 2/3/04

**... user always maintains or increases ****sensitivity****, and always maintains or increases FPR, as well.**

- Sacks

- 2/3/04

**  ****Gain of 0.2 in A****Z  **** ****	****	****understates****	****	**** ****	****relative gain in ****	****sensitivity.**

- Sacks

- 2/3/04

*  ** **                                                                                                                                                      *

**FPR**

**0**

**1**

**0**

**1**

**TPR**

**Unaided**

**Aided**

**Δ****Se  =****Δ****FNR**

**Δ****FPR**

**FNR**U

**Se****U**

**Chance line (guessing)**

**ROC**

- Sacks

- 2/3/04

*  ** **                                                                                                                                                      *

**FPR**

**0**

**1**

**0**

**1**

**TPR**

**Unaided**

**Aided**

**Chance line (guessing)**

**Loss of Se only possible if instruction not followed**

**Realistic**

**ROC**

- Sacks

- 2/3/04

**So any stat sig improvement in A****z**** means an even greater relative gain in Se, without “undue” increase in FPR.**

- Sacks

- 2/3/04

**Can we infer from improved user performance in clinical study ...**

- Sacks

- 2/3/04

**... that user will improve performance with CAD in  clinical practice?**

- Sacks

- 2/3/04

**For example, in actual clinical practice, the ****unaided**** A****z**** could be lowered by failure to read first as one would normally read (i.e., with adequate vigilance).**

- Sacks

- 2/3/04

**If this were to happen, then the ****aided**** A****z**** could also be lower than current practice (in the complete absence of a CAD).**

- Sacks

- 2/3/04

**Two rules, if followed by user, will prevent that from happening:**
** ****Always**** read unaided first, ****	****and as carefully as if you ****	****had no CAD.**
** ****Never ****back off from unaided ****	****judgment of actionability of a ****	****nodule if CAD fails to mark it.**

- Sacks

- 2/3/04

**	****	****	****  ****Summary**
** ****Data show stat sig improvement of ****A****z**** of ****	****aided over unaided readings with CAD**
** ****If “****Always****” rule followed then aided curve****	**** will be higher than current practice**
** ****If “****Never****” rule followed, then gain in A****z**** ****	**** implies even greater relative ****	****  ****	**** ****	**** improvement in ****Se**** of aided readings over****	**** current practice, without “undue” ****	****  ****	****	**** increase in ****FPR**

- Sacks

- 2/3/04