# Presentation: 599041

## AVT: Algorithm Validation Toolkit

***Phase II: Pilot Implementation***
- Bob Schwanke
- Siemens Corporate Research
**March, 2009**

## What’s in a Name?

**A****lgorithm ****V****alidation ****T****oolkit?**
**A****nnotation ****V****ariation ****T****oolkit!**
- Many sources of variation in tumor measurement
- Different equipment
- Different field strength / X-ray dosage
- Different readers
- Same reader, different day
- Reposition patient
- Breathing
- Oh, and by the way, ***different algorithm***
*AVT helps create ****annotations**** and statistically analyze ****variations***

## Purposes of AVT

**Support annotation variability analysis**
**Integrated statistics review and drill-down visualization**
**Open source toolkit**
- XIP imaging application toolkit
- XIPHost: DICOM WG 23 Application Hosting
- caGRID connectivity
- AIM/XML for **A**nnotations and **I**mage **M**arkup
- R Statistics package
**Batch processing facility for large scale algorithmic experiments**

## Objectives of Phase II

**Use case model as roadmap for future**
**Deliver toolkit framework**
**Example applications and database**
**caGRID client connectivity**
**Demonstrate generality and modifiability**
**End-to-end example study**
**Open source**
**SIIM ’09 hands-on delivery**

## Data Flow Alternate Text

**The diagram shows AVT’s major components, the connectors that tie them together, their connection to the grid, and the data flows between them.**
**Etc.**

## Data Flow

## Status: Use Case Model

**Detailed Business Use Case Model for annotation variability analysis**
**Hierarchically Organized**
**Fifty cases and sub-cases**
**Reviewed by**
-  Imaging Telecon audience
- About 10 subject matter experts in detail
**Defines roadmap for future development**
**Demonstrates generality of the approach**

## April Status: Toolkit Framework 

**Applications and database now integrated by XIPHost Framework**
**caGRID client**
**Local database for experiments**
**Infrastructure**
**Open source (except DB engine*)**
**XIP**
**XIPHost**
**DICOM standard**
**AIM for storing and retrieving annotations**
**R statistics package**
**Java Swing user interfaces**
*** IBM DB2 Express-C database engine**
- Free
- Powerful XML support
- Open source alternatives have very limited XML indexing capability

## April Status: applications and database

**Applications launch from XIPHost via WG23 protocol**
**Database**
**Stores and Queries DICOM, AIM, and other XML**
**Integrated with XIPHost**
**Also direct interface for application query/retrieve/store**
**Image Annotation Tool **
**receives DICOM images, returns AIM annotations**
- Segmentations: AIM references DICOM segmentation objects
- Textual annotations structured in AIM/XML, proper VCDE lexicon
**Example: segment and characterize lung lesions**
- Automatic segmentation: off-the-shelf from XIP

## April Status: applications and database

**Measurement Variability Tool**
**Reads AIM annotations, performs calculations**
**Reads and displays DICOM images to visualize volumes**
**Example: Volume difference, overlap, surface distance, etc.**
**Mean, SD, and *****user-defined***** statistics calculations (modifiability)**
**R statistics package**
- XIP components written in R
- On-the-fly scripting
**Outlier identification**
**Algorithm Execution Tool (Batch facility): not started**

## Next Step: End-to-end demonstration

**First release to friendly first users in April**
**Shadow two real projects**
**FDA/CDRH (Petrick) and RadPharm**
- Reader study
- Thoracic phantom lesions
**UCLA (McNitt-Gray)**
- Glioblastoma treatment study
**Feature freeze for SIIM 2009**
**Current project ends in August**

## Future

**Use Case Model provides roadmap for future development**
**Prioritization by needs of first users**
**Applying for grants to support further development**
- QIN grants
- Economic Stimulus grants (we’re shovel-ready)
- ACR/ACRIN discussions
- Other?