# Presentation: 540619

## The Lung Image Database Consortium (LIDC):

- Fundamental Issues for the Creation of a Resource for the Image Processing Research Community

## Exhibit Learning Objectives:

- Learn about the LIDC’s goals and methods for creating:
- a publicly available database,
- for the development, training, and evaluation of Computer-Aided Diagnosis (CAD) methods,
- for lung cancer detection and diagnosis using helical CT.

## Learning Objectives:

- Learn about challenges in interpreting CT image data sets for the detection and diagnosis of lung cancer

## Learning Objectives:

- Learn about the intricacies of establishing spatial “truth” for lesion location and boundary.

## The Challenge: Lung Cancer

-  Cancer of the lung and bronchus is the leading fatal malignancy in the United States.
- Five-year survival is low, but treatment of early-stage disease improves chances of survival considerably.

## The Challenge: Lung Cancer

-  Given:
- Promising results from recent studies involving the use of helical computed tomography (CT) for the early detection of lung cancer
- As well as rapid developments in Multi-detector CT (MDCT) technology which provide for the possibility of the detection of smaller lung nodules and offers a potentially effective tool for earlier detection.
-  There has been an increased interest in computer-aided diagnosis (CAD) techniques applied to CT imaging for lung cancer to assist radiologists’ with their decision-making.

## The NCI Response:  Forming the LIDC

- To stimulate research in the area of CAD, the National Cancer Institute (NCI) formed a consortium of institutions to develop the standards and consensus necessary for constructing an image database resource of thoracic helical CT images.

## Motivation

-   The development of CAD methods by the imaging research community would be facilitated and enhanced through access to a repository of CT image data

## Motivation

-   The development of CAD methods by the imaging research community would be facilitated and enhanced through access to a repository of CT image data
- (1)  It would provide data to researchers without access to clinical images** **

## Motivation

**  **The development of CAD methods by the imaging research community would be facilitated and enhanced through access to a repository of CT image data
- (1)  It would provide data to researchers without access to clinical images
- (2)  It would also allow for meaningful comparisons of different CAD methods

## The LIDC

- This consortium - called the Lung Image Database Consortium (LIDC) - seeks to establish standard formats and processes by which to manage lung images and the related technical and clinical data that will be used by researchers to develop, train and evaluate CAD algorithms for lung cancer detection and diagnosis.

## Member Institutions

-   Five institutions were selected to form the Lung Image Database Consortium (LIDC)

## Member Institutions

**Cornell University**
**	**
**UCLA****	**
**	**
**University of Chicago****	**
**	**
**University of Iowa****	**
**	**
**University of Michigan****	****	**
**	**

## Steering Committee

**Cornell University****	****David Yankelevitz**
**	****Anthony P. Reeves**
**UCLA****	****Michael F. McNitt-Gray**
**	****Denise R. Aberle**
**University of Chicago****	****Samuel G. Armato III**
**	****Heber MacMahon**
**University of Iowa****	****Geoffrey McLennan**
**	****Eric A. Hoffman**
**University of Michigan****	****Charles R. Meyer**
**	****Ella Kazerooni**
**NCI****	****Laurence P. Clarke**
**	****Barbara Y. Croft**

## Contributing Participants

**Claudia Henschke, Cornell**
**David Gur, U. of Pittsburgh**
**Robert Wagner, FDA**
**Nicholas Petrick, FDA**
**Lori Dodd, NCI**
**Ed Staab, NCI**
**Daniel Sullivan, NCI**
**Houston Baker, NCI**
**Carey Floyd, Duke**
**Aliya Husain, U. of Chicago**
**Matthew Brown, UCLA**
**Christopher Piker, U. of Iowa**
**Peyton Bland, U. of Michigan**
**Andinet Asmamaw, Cornell**

**Richie Pais, UCLA**
**Antoni Chan, Cornell**
**Gary Laderach, U. of Michigan**
**Junfeng Guo, U. of Iowa**
**Charles Metz, U. of Chicago**
**Roger Engelmann, U. of Chicago**
**Adam Starkey, U. of Chicago**
**Jim Sayre, UCLA**
**Mike Fishbein, UCLA**
**Andy Flint, U. of Michigan**
**Barry DeYoung, U. of Iowa**
**Brian Mullan, U. of Iowa**
**Madeline Vazquez, Cornell**

## Mission

- The mission of the LIDC is the sharing of lung images, especially low-dose helical CT scans of adults screened for lung cancer, and related technical and clinical data for the development and testing of computer-aided detection and diagnosis technology

## Principal Goals

- To establish standard formats and processes for managing thoracic CT scans and related technical and clinical data for use in the development and testing of computer-aided diagnostic algorithms.

## Principal Goals

- To establish standard formats and processes for managing thoracic CT scans and related technical and clinical data for use in the development and testing of computer-aided diagnostic algorithms.
- To develop an image database as a web-accessible international research resource for the development, training, and evaluation of computer-aided diagnostic (CAD) methods for lung cancer detection and diagnosis using helical CT.

## The Database

- The database will contain:
-   a collection of CT scan images
-   a searchable relational database

## Fundamental Issues for the LIDC

## LIDC Challenge #1 - Define a Nodule

- Though at first this seems trivial, the LIDC had significant discussion about what to include and what not to include as a nodule
- A Nodule is part of a spectrum of focal abnormalities.
- This spectrum includes scars, cancers, benign lesions, calcified lesions, etc.

## What is a Nodule?

**“****nodule: any pulmonary or pleural lesion represented in a radiograph by a sharply defined, discrete, nearly circular opacity 2-30 mm in diameter”**
**from the Fleischner Society's Glossary of Terms for Thoracic Radiology (**_**AJR**_** 1984)**

## Slide 23

## What is a Nodule?

**from the Fleishner Society's Glossary of Terms for CT of the Lungs (**_**Radiology**_** 1996)**

## Slide 25

## Is this a Nodule?

## Slide 27

## Slide 28

## Slide 29

## Slide 30

## Slide 31

## Slide 32

## Slide 33

## Slide 34

## Slide 35

## Slide 36

## Slide 37

## NOTE: This is the slice which was shown earlier

## Slide 39

## Slide 40

## Slide 41

## Slide 42

## Slide 43

## Slide 44

## Slide 45

## Slide 46

## Slide 47

## What is a Nodule?

**  ****a spectrum of abnormalities**

- Scar

- Spiculated Nodule

- Calcified Nodule

## LIDC Challenge #1 - Define a Nodule

- LIDC Response is to develop a Nodule Visual Library using:
- Cases that ARE in Nodule portion of spectrum
- Cases that are OUTSIDE Nodule portion of spectrum
- Classification by Thoracic Radiologists
- In Development Now
- Expected Completion Feb 2004.

## Truth Assessment

- Investigators will require “truth” information

**  **

## Truth Assessment

- Investigators will require “truth” information
- location of nodules

**  **

## Truth Assessment

- Investigators will require “truth” information
- location of nodules
- spatial extent of nodules

**  **

## Truth Assessment

- Investigators will require “truth” information
- location of nodules
- spatial extent of nodules
- Spatial “Truth” will be estimated by “Radiologic Truth”

**  **

## LIDC Challenge #2Define the Boundary of a Nodule

- Though it seems that the boundary of a nodule should be easy to define, we (and others) have found that there is considerable inter-reader variability in defining the boundary of a nodule.
- This is difficult enough with a solid nodule, but even more difficult with spiculated nodules, ground glass nodules or non-solid nodules.

## SPICULATED NODULE

- Instructions to Thoracic Radiologists were
-  “Draw the Boundary of the Nodule”

## SPICULATED NODULE

- Expert Number 1 Contour

## SPICULATED NODULE

- Expert Number 2 Contour

## SPICULATED NODULE

- Comparison of Contours

- Five radiologists using 3 drawing methods:
- One manual 3-panel (3D) drawing method
- Two different semiautomatic 3D methods

- Reader and Method Variability in Drawing Boundaries

## Case 5, Slice 19

## Radiologist 1 - Method 1

## Radiologist 1 - Method 2

## Radiologist 1 - Method 3

## Radiologist 2 - Method 1

## Radiologist 2 - Method 3

## Radiologist 3 - Method 1

## Radiologist 3 - Method 2

## Radiologist 3 - Method 3

## Radiologist 4 - Method 1

## Radiologist 4 - Method 2

## Radiologist 4 - Method 3

## Radiologist 5 - Method 1

## Radiologist 5 - Method 3

## For each voxel, sum the number of occurrences (across reader and method combinations) that it was included as part of the nodule
Create a probabilistic map of nodule voxels
Higher probability voxels are shown as brighter; lower probability are darker
Can use apply a threshold and show only voxels > some prob. Value if desired.

- Create a probabilistic map of nodule voxels
- Higher probability voxels are shown as brighter; lower probability are darker
- Can use apply a threshold and show only voxels > some prob. Value if desired.

- Create a Probabilistic Description of Nodule Boundary

- Probabilistic Description of Boundary

- Apply Threshold if Desired

## LIDC Challenge #2Define the Boundary of a Nodule

- Do we need to reconcile these Boundaries?
- LIDC’s answer is no.
- LIDC Approach will be to:
- Come to a consistent definition of the desired boundary (include just solid portion? non-solid portion?)
- Assess reader variability of contours
- Construct a probabilistic description of boundaries to capture reader variability

## LIDC Challenge #3-  Data Collection Process

- Recent research has demonstrated that Single reads are not sufficient – At least two and perhaps four readers may be required.
- Not practical to do joint readings across five institutions
- LIDC Will NOT do a forced consensus read.

## LIDC Challenge #3-  Data Collection Process

- Will do a Two-Staged Process:
- Perform independent (Blinded) readings of cases by multiple radiologists
- Compile readings and redistribute composite readings
- Perform a Second, Unblinded read by same set of radiologists
- Each reader can see readings of every other reader.
- No forced consensus
- Capture probabilistic detection (e.g. a nodule can be identified by 3 of 4 readers) and probabilistic contours.

## LIDC Process Model 2.4 October 2003

- Overview
- Prerequisites
- major data collection steps, and
- data collected at each step.

## Prerequisites

- Actions

- Data Collected

- IRB approvals

- Participants Scanned as part of  Study/Clinical Program

- LIDC Activities

## Prerequisites

**Radiologist Review Process (described next)**

- Identified lesions for each condition: Each reader, blinded and unblinded read
-  Location
-  Outline
-  Label

- Definition of Nodules to be included in Db
- Agreement on Marking /Contouring process

- LIDC Activities

- Actions

- Data Collected

- Reader 1

- Reader 2

- Reader 3

- Reader 4

## Blinded Reads – Each Reader Reads Independently

-           (Blinded to Results of Other Readers)

## Reader 1

- Blinded Read for Reader 1 – Marks Only One Nodule

## Reader 2

- Blinded Read for Reader 2 – Marks Two Nodules
- (Note:  One nodule is same as Reader 1)

## Reader 3

- Blinded Read for Reader 3 – Marks Two Nodules
- (Note: Again, One nodule is same as for Reader 1)

## Reader 4

- Blinded Read for Reader 4 – Did Not Mark Any Nodules

## UnBlinded Reads – Readings in Which Readers Are

- Shown Results of Other Readers

- Each Reader Marks Nodules After Being Shown Results From Other Readers’ Blinded Reads
- (Each Reader Decides to Include or Ignore).

## Reader 1

- Unblinded Read for Reader 1 – Now Marks Two Nodules
- 		(Originally only marked one)

## Reader 2

- Unblinded Read for Reader 2 – Still Marks Two Nodules
- 			(No Change)

## Reader 3

- Unblinded Read for Reader 3 – Now Marks Three Nodules
- 		(Originally only marked two)

## Reader 4

- Unblinded Read for Reader 4 – Now Marks Three Nodules
- 		(Originally did not mark any)

## 4 Markings

- 2 Markings

- 2 Markings

- Composite on Unblinded Reads for All Four Readers

## Database Implementation

- TASKS COMPLETED (see reports on website):
- Specification of Inclusion Criteria:
- CT scanning technical parameters
- Patient inclusion criteria
- Process Model for Data collection
- Determination of Spatial "truth" Using Blinded and Unblinded reads
- Development of Boundary Drawing/Contouring Tools

## Database Implementation

- TASKS ONGOING (expected completion date)
-  Definition of Nodule - Nodule Visual Library  (Feb 04)
-  Evaluation of Boundary Variability  (Feb 04):
- Inter-Reader Variability
- Boundary Drawing Tool Variability

## Implementation Timeline

- _Task_						_Date Expected_
- Specify Complete Data Model			 Jan 04
- Specify LIDC internal workflow			 Jan 04
- 	 Data passing, Performing reviews
- Initial implementation, testing workflow   Jan/Feb 04
- Database Implementation- Start  		  Jan 04
- Database Implementation- Completion    Mar/Apr  04
- Implement Public Interface to Database  Apr/May  04
- PUBLIC ACCESS TO CASES – EXPECTED  MAY/JUN 04

## Publications/Presentations

- LIDC Overview manuscript
- In Preparation, submission in 1st Quarter 04
- Assessment Methodologies manuscript
- In Preparation, submission in 1st Quarter 04
- Special Session _[SPIE Medical Imaging ](http://spie.org/Conferences/Programs/04/mi/specialevents/)_
- Sunday evening Feb 15, 2004

## To learn more about the LIDC:

- Return for CME Category 1 Credit:
- Monday – Thursday
- 12:15 pm to 1:15 pm
- At these times, LIDC members will be here to describe the efforts of the consortium, this exhibit and any other questions you might have

## Return to Exhibit Home Page

- _[Click Here](file:///LIDC_Ed_Exhibit_Home.ppt)_