**caBIG Vocabulary and CDE Workspace Development Project**

# caTIES: Modular Tools for EVS-based Concept Annotation of Pathology Reports

**SCOPE DOCUMENT -- Initial Draft**

**Authors: Rebecca Crowley, MD, MS and Kevin Mitchell, MS**

The goal of this project is to develop a general purpose Text
Information Extraction System (caTIES) that can be used by caBIG
adopters to automate the process of coding, storing and retrieving data
from free-text Pathology Reports. The data extracted from these clinical
reports will directly populate caBIG data structures. The kinds of data
we will extract include (but are not limited to) all of the following:

1.  Body parts present in specimen

2.  Surgical procedure performed

3.  Diagnoses

4.  Findings (present and absent)

5.  Values of important attributes (specific to particular
    Organ/Procedure/Diagnosis sets) including:

-   TNM classification

-   Tumor grading (Gleason Score, Nottingham Score, etc)

-   Degree of differentiation

-   Presence of absence of lymphatic metastases

-   Presence or absence of marginal involvement

-   Degree of involvement

-   Invasion of adjacent structures

-   Results of immunohistochemical stains and molecular markers reported
    in free text

To accomplish this task, we propose to build on the existing UPCI
GATE-based pipeline for concept annotation of Pathology Reports that has
been developed for the Shared Pathology Informatics Network (SPIN). The
fundamental differences between our proposed caBIG project and SPIN are
four-fold:

1.  We will use the NCI Metathesaurus and NCI Thesaurus as our source of
    concepts for concept coding

2.  We will use the NCI Thesaurus to assist in the classification of
    documents

3.  We will populate the caDSR with CDEs derived from CAP protocols to
    direct our information extraction system

4.  These CDEs will act as the target of extracted data.

Consequently, the information extracted will conform to metadata
standards, and can therefore be accessed across the caBIG.

Our existing system is built on GATE - an open source Java framework for
language engineering -- (<http://gate.ac.uk/>). Our code is already open
source under terms of the GNU license, and available at:
<http://spin.nci.nih.gov/content>[.]{.underline} We expect to be able to
automate coding of more than half of the items above during the first
year, and to automate coding of the remaining items during Y2 and Y3.
The development of an automated method for coding pathology report text
would provide a critical link in connecting genomics and proteomics to
clinical phenotype.

The short-term, mid-term, and long-term goals for this three-year
project are as follows:

A)  ***[Within the first three months]{.underline}* -- (a) deploy our
    existing software to one or two adopter sites by modifying our
    existing pipeline (to utilize the NCI Metathesaurus) and by
    providing it as a Java Webstart application that can be downloaded
    from our server, already pre-configured, and used on the desktop
    and (b) lay the groundwork for a caBIG Information Extraction System
    by populating the caDSR with common data elements derived from the
    CAP protocols.**

A critical component of success in caBIG will be rapid deployment of
caBIG compliant applications. Our existing application already uses
several standards (UMLS -- Metathesaurus; SPIN XML Schema). In the first
three months, we will create a caBIG compliant application leveraging
existing work to provide a quick deployment for EVS-based concept coding
of Pathology Reports. The purpose of this rapid deployment is to quickly
empower one or two adopter sites with the ability to easily, and
automatically concept-code Pathology Reports and make them available as
XML documents which can be fed into whatever architecture is ultimately
agreed upon by the caBIG community.

In parallel, with deployment efforts, we will lay the groundwork for
developing caTIES. The most critical early deliverable will be a set of
CDEs for the caDSR which are based on pathology report attributes
described in the College of American Pathologists (CAP) protocols for
Surgical Pathology.

The taggers we have already developed include: (A) a GATE UMLS-tagger
that accesses the UMLS KSS via Java RMI, (B) GATE UMLS-tagger which
accesses a local UMLS installation, (C) GATE MetaMap tagger which
accesses a local UMLS installation. We anticipate that we can quickly
convert Tagger A from a UMLS-Metathesaurus concept tagger to an
NCI-Metathesaurus concept-tagger, by simply replacing our RMI calls.

We will also modify our existing GATE resources so that they can be
deployed using Java Webstart (java.sun.com/products/javawebstart/). This
will allow adopters to download the pre-configured GATE pipeline with a
single click over the WWW. Users can choose to run the pre-configured
pipeline saved locally, or keep their application seamlessly updated to
our newest version.

B)  ***[Within the first six months]{.underline}* -- design, develop and
    deploy a simple query interface to be used to search the resulting
    archive of coded documents. Develop an underlying data model to
    organize extracted information in formats that facilitates rapid and
    accurate query response.**

The power of information from clinical reports requires the ability to
search and retrieve documents using complex concept-based queries over
large volumes of reports across multiple institutions. The mid-term goal
of this project will be to develop a simple interface (which we will
later expand) for searching archives of coded pathology reports. At the
minimum, we expect to allow users to construct queries that permit
aggregation of simple demographic information (age, gender), diagnosis,
procedure, body-part, TNM Classification, and positive/negative Meta
concepts.

A simple query might request all reports from pediatric populations for
brain biopsies of any kind of astrocytoma, without necrosis:

> *Return all reports where (AGE = 0-18) and (DIAGNOSIS= CL024452:
> Astrocytic Tumor OR children of that Thesaurus concept) and
> (PROCEDURE= C0740294: Biopsy brain) and (NEGATED-FINDING= C0027540:
> Necrosis).*

We will develop a Java based GUI for querying the report archive that
allows both free-text entry (which will then be converted to NCI
Metathesaurus concepts) and tree-based browsing of concepts using the
NCI Thesaurus, using standard CaBIO APIs. Additionally, we eventually
anticipate using the metadata in the caDSR to help users quickly select
relevant attribute value pairs (see Goals A and C).

We will develop backend support for the query language and document
browser user interface. Initially we will organize data locally in data
warehouses capable of rapid and accurate query support. These initial
data repositories will provide a fully-functioning capability. We will
synchronize our data representation model with ongoing efforts in the
caBIG architecture. Major architectural decisions (for example whether
caBIG will be a computational grid or a data grid) are necessary before
we can connect our query interface to a caBIG wide archive of Pathology
Reports. Consequently, in the first 6 months, we will focus on
delivering the following:

(1) A well-designed, front-end for concept- based retrieval from local
    archives of concept coded pathology reports.

(2) A sufficient data warehouse representation of underlying extracted
    information to facilitate rapid and accurate query response. This
    warehouse will utilize existing caBIG capabilities when possible.

(3) An ongoing contribution to the conceptual operations plan for caBIG
    based on our requirements analysis and initial system development
    experience.

We will use standard methods of system and interface development
including use cases (developed with researchers), storyboards, rapid
prototyping with user feedback, and usability testing to maximize the
usability of the interface for querying reports. Once the architecture
for caBIG has been established -- we will appropriately modify our query
mechanism to conform to the accepted architecture.

C)  ***[Within the first three years]{.underline}* -- develop more
    advanced methods for EVS-based Information Retrieval to improve
    performance of our existing report coder in automating the coding of
    data described above.**

We also propose to significantly improve our GATE pipeline during the
caBIG pilot project. We believe that we can use and expand existing EVS
resources to drive the Information Extraction aspects of our caTIES.
Specifically we expect to:

1.  Develop a Jess-based document classification system that uses
    content from NCI Thesaurus and caDSR to classify pathology reports
    and set agenda of goals for information extraction.

One of the most difficult aspects of Information Extraction is knowing
what to extract. As described in previous sections, we propose to use
the EVS resources to assist us in both classifying the type of pathology
report (a combination of Body Part, Surgical Procedure, and Diagnosis)
and to set an agenda of goals for extraction. We will use the NCI
Thesaurus for assisting us with document classification, and the
CAP-derived CDEs in the caDSR for determining what to extract.

An important aspect of this work will involve working with caBIG and NCI
to (1) add a select set of properties in the NCI Thesaurus that will be
necessary for document classification (mainly to express relationships
between surgical procedure and body parts) and (2) to evolve the set of
CDEs that encapsulate the CAP protocols for surgical pathology reports,
from which we will use Data Element concepts to drive Information
Extraction. CAP-derived CDEs will also be used on the Query Interface
side to simplify construction of user queries.

2.  Develop general methods for information extraction of simple
    attribute value pairs.

A successful general method for extracting values for the attributes we
set as extraction goals, will require retraining of existing POS taggers
and parsers, so that they are optimized for the linguistic structure of
clinical narrative in this domain. We will focus initially on CDEs that
will require only simple NLP methods such as regular expressions,
negation detection, etc. However, during the first three years of the
pilot project -- we propose to develop both a POS tagger and a partial
parser to be used in our GATE pipeline. Existing parsers will be
retrained using a combination of SPECIALIST lexicon and disambiguation
by expert review.