# Presentation: 003396

## HelpfulMED: Creating a Knowledge Portal for Medicine

**Hsinchun Chen, Ph.D.**
- McClelland Professor of MIS
- Director, Artificial Intelligence Lab
- University of Arizona
- Member, Arizona Cancer Center
- Senior Research Scientist, NCSA

**NSF DLI-2 All-Project Meeting**
**Stratford-upon-Avon, UK, June 12-13, 2000**** **

**Acknowledgments:**
**NSF, NLM, NCI, NCSA**
_**(DLI-2 Project: “High-Performance Digital Library Classification Systems: From Information Retrieval to Knowledge Management”)**_

## Heterogeneous

- Medical
- Literature Databases
- and the Internet

- Medical
- Professionals
- & Users

**TOXLINE**

**CancerLit**

**EMIC**

**Hazardous**
**Substances**
**Databank**

**MEDLINE**

- Current Information
- Interfaces

- The Medical Information Gap

## How can linguistic parsing and statistical analysis techniques help extract medical terminology and the relationships between terms?
How can medical and general ontologies help improve extraction of medical terminology?
How can linguistic parsing, statistical analysis, and ontologies be incorporated in customizable retrieval interfaces?

- How can _linguistic parsing_ and _statistical analysis_ techniques help extract medical terminology and the relationships between terms?
- How can _medical and general ontologies_ help improve extraction of medical terminology?
- How can linguistic parsing, statistical analysis, and ontologies be incorporated in _customizable retrieval interfaces_?

## Previous Work: Linguistic Parsing and Statistical Analysis

## Noun compounds are widely used across sub-language domains to describe concepts concisely
Unlike keyword searching, contextual information is available
Relationship between a noun compound and the head noun is a strict conceptual specification.
“breast” and “cancer” vs. “breast cancer”
“treatment” and “cancer” vs. “treatment of cancer”
Proper nouns can be captured

(Anick and Vaithyanathan, 1997)

- Unlike keyword searching, contextual information is available
- Relationship between a noun compound and the head noun is a strict conceptual specification.
- “breast” and “cancer” vs. “_breast cancer_”
- “treatment” and “cancer” vs. “_treatment of cancer_”
- _Proper nouns_ can be captured
- (Anick and Vaithyanathan, 1997)

- Benefits of Natural Language Processing

## Appropriate level of analysis: Extraction of grammatically correct noun phrases from free text

Used in other domains, noun phrasing has been shown to improve the accuracy of information retrieval (Girardi, 1993; Devanbu et al., 1991; Doszkocs, 1983)

Cooper and Miller (‘98) used noun phrasing to map user queries to MeSH with good results

- Used in other domains, noun phrasing has been shown to improve the _accuracy of information retrieval_ (Girardi, 1993; Devanbu et al., 1991; Doszkocs, 1983)
- Cooper and Miller (‘98) used noun phrasing to _map user queries to MeSH_ with good results

- Natural Language Processing: Noun Phrasing

## NSF Digital Library Initiative I & II Research
Developed to improve document representation and to allow users to enter queries in natural language 

- Developed to improve document representation and to allow users to enter queries in natural language

- Arizona Noun Phraser

## Tokenizer
Takes raw text and generates word tokens 			(conforms to UPenn Treebank word tokenization rules)
Separates punctuation and symbols from text without affecting content
Part of Speech (POS) Tagger
Based on the Brill Tagger 
Two-pass parser, assigns parts of speech to each word
Uses both lexical and contextual disambiguation in POS assignment
Lexicons: Brown Corpus, Wall Street Journal, Specialist Lexicon
Phrase Generation
Simple Finite State Automata (FSA) of noun phrasing rules
Breaks sentences and clauses into grammatically correct noun phrases

- Takes raw text and generates word tokens 			(conforms to UPenn Treebank word tokenization rules)
- _Separates punctuation and symbols_ from text without affecting content
- Part of Speech (POS) Tagger
- Based on the Brill Tagger
- Two-pass parser, _assigns parts of speech_ to each word
- Uses both lexical and contextual disambiguation in POS assignment
- Lexicons: _Brown Corpus, Wall Street Journal, Specialist Lexicon_
- Phrase Generation
- Simple Finite State Automata (FSA) of _noun phrasing rules_
- Breaks sentences and clauses into grammatically correct noun phrases

- Arizona Noun Phraser: Three Modules

## Results of Testing (Tolle & Chen, 1999)
	The Arizona Noun Phraser is better than or comparable to other techniques (MIT’s Chopper and LingSoft’s NPtool)

Improvement with Specialist Lexicon
	The addition of the Specialist Lexicon to the other non-medical lexicons slightly improved the Arizona Noun Phraser’s ability to properly identify medical terminology

- 	The Arizona Noun Phraser is better than or comparable to other techniques (_MIT’s Chopper_ and _LingSoft’s NPtool_)
- Improvement with _Specialist Lexicon_
- 	The addition of the Specialist Lexicon to the other non-medical lexicons slightly improved the Arizona Noun Phraser’s ability to properly identify medical terminology

- Arizona Noun Phraser

## Statistical Analysis Techniques:
Based on document term co-occurrence analysis, weights between concepts establish the strength of the association
Four steps: Document Analysis, Concept Extraction, Phrase Analysis , Co-occurrence Analysis
Systems: 
Bio-Sciences: Worm Community System (5K, Biosys Collection, 1995), FlyBase experiment (10K, 1994)
DLI: INSPEC collection for Computer Science & Engineering (1M, 1998)
Medicine: Toxline Collection (1M, 1996), National Cancer Institute’s CancerLit Collection (1M, 1998) and National Library of Medicine’s Medline Collection (10M, 2000)
Other: Geographical Information Systems, Law Enforcement
Results: 
Alleviate cognitive overload, improve search recall

- Based on document _term co-occurrence analysis_, weights between concepts establish the strength of the association
- Four steps: Document Analysis, Concept Extraction, Phrase Analysis , Co-occurrence Analysis
- Systems:
- Bio-Sciences: Worm Community System (5K, _Biosys Collection_, 1995), FlyBase experiment (10K, 1994)
- DLI: INSPEC collection for Computer Science & Engineering (1M, 1998)
- Medicine: _Toxline Collection_ (1M, 1996), National Cancer Institute’s _CancerLit Collection_ (1M, 1998) and National Library of Medicine’s _Medline Collection_ (10M, 2000)
- Other: Geographical Information Systems, Law Enforcement
- Results:
- Alleviate cognitive overload, improve search recall

- Creating Knowledge Sources: Concept Space  (Automatic Thesaurus)

## The computation generated Cancer Space, which consists of 1.3M cancer terms and 52.6M cancer relationships.
The approach: Object-Oriented Hierarchical Automatic Yellowpage (OOHAY) -- the reverse of YAHOO!
Prototype system available for web access at: ai20.bpa.arizona.edu/cgi-bin/cancerlit/cn
Experiments for 10M Medline abstracts and 50M Web pages under way

- The computation generated Cancer Space, which consists of _1.3M cancer terms_ and 52.6M cancer relationships.
- The approach: _Object-Oriented Hierarchical Automatic Yellowpage (OOHAY)_ -- the reverse of YAHOO!
- Prototype system available for web access at: ai20.bpa.arizona.edu/cgi-bin/cancerlit/cn
- Experiments for _10M Medline abstracts_ and _50M Web pages_ under way

- Supercomputing to Generate Largest Cancer Thesaurus

## “NCSA capability computing helps generate largest cyber map  for cancer fighters...”  

- The Arizona team, used NCSA’s 128-processor Origin2000 for over 20,000 CPU-hours.
- _Cancer Map_ used 1M CancerLit abstracts to generate 21,000 cancer topics in a 5-layer hierarchy of 1,180 cancer maps.
- The research is part of the Arizona OOHAY project funded by _NSF Digital Library Initiative_ 2 program.
- Techniques: computational linguistics and neural network text mining

- High-Performance Computing for Cyber Mapping

## Medical Concept Mapping:Incorporating Ontologies (WordNet and UMLS) 

## Princeton, George A. Miller (psychology dept.)
95,600 different word forms, 57,000 nouns
grouped in synsets, uses word senses
used to extract textual contexts (Stairmand, 1997), text retrieval (Voorhees, 1998), information filtering (Mock & Vermuri, 1997) 
available online: http://www.cogsci.princeton.edu/~wn/

- 95,600 different word forms, _57,000 nouns_
- grouped in synsets, uses word senses
- used to extract textual contexts (Stairmand, 1997), text retrieval (Voorhees, 1998), information filtering (Mock & Vermuri, 1997)
- available online: http://www.cogsci.princeton.edu/~wn/

- Incorporating Knowledge Sources: WordNet Ontology

## Slide 15

## Incorporating Knowledge Sources: UMLS Ontology

-  _Unified Medical Language System_ (UMLS) by the National Library of Medicine (_Alexa McCray_)
- 1986 - 1988: defining the user needs and the different components
- 1989-1991: development of the different components: _Metathesaurus_, _Semantic Net_, _Specialist Lexicon_
- 1992 - present: updating & expanding the components, development of applications
-  available online: http://umlsks.nlm.nih.gov/

## 730,000 concepts, 1.5 M concept names
60+ vocabulary sources integrated
15 different languages
organization by concept, for each concept there are different string representations

- _60+ vocabulary sources_ integrated
- _15 different languages_
- organization by concept, for each concept there are different string representations

- UMLS Metathesaurus (2000 edition)

## UMLS Metathesaurus (2000 edition)

## 134 semantic types and 54 semantic relations
metathesaurus concepts  semantic net
relations between types, not between concepts

- metathesaurus concepts  semantic net
- _relations between types_, not between concepts

- UMLS Semantic Net (2000 edition)

## UMLS Semantic Net (2000 edition)

## A general English lexicon that includes many biomedical terms
130,000+ entries
each entry contains syntactic, morphological and orthographic information
no different entries for homonyms

- _130,000+ entries_
- each entry contains syntactic, morphological and orthographic information
- no different entries for homonyms

- UMLS Specialist Lexicon (2000 edition)

## UMLS Specialist Lexicon (2000 edition)

## Ontology-Enhanced Concept Mapping: 

- Design and Components

## WordNet
Return synonyms if there is only one word sense for the term 
E.g. “cancer” has 4 different senses,  one of them is:
Cancer,  Cancer the Crab,  fourth sign of the Zodiac
UMLS Methathesaurus
find the underlying concept of a term and retrieve all synonyms belonging to this concept
E.g. term = tumor  concept = neoplasm
synonyms: 
Neoplasm of unspecified nature NOS |  tumor <1> |  Unspecified neoplasms |   New growth |  [M]Neoplasms NOS  |  Neoplasia |  Tumour  |  Neoplastic growth |   NG - Neoplastic growth |   NG - New growth | 800 NEOPLASMS,  NOS |  
filtering of the synonyms (personalizable for each user): filter out the terms
tumor <1> | [M]Neoplasms NOS | NG - Neoplastic growth |  NG - New growth |  800 NEOPLASMS,  NOS |

- Return synonyms if there is only _one word sense_ for the term
- E.g. “cancer” has 4 different senses,  one of them is:
- Cancer,  Cancer the Crab,  fourth sign of the Zodiac
- UMLS Methathesaurus
- find the underlying concept of a term and retrieve _all synonyms_ belonging to this concept
- E.g. term = tumor  concept = neoplasm
**synonyms: **
- Neoplasm of unspecified nature NOS |  tumor <1> |  Unspecified neoplasms |   New growth |  [M]Neoplasms NOS  |  Neoplasia |  Tumour  |  Neoplastic growth |   NG - Neoplastic growth |   NG - New growth | 800 NEOPLASMS,  NOS |
**filtering of the synonyms (personalizable for each user):** filter out the terms
- tumor <1> | [M]Neoplasms NOS | NG - Neoplastic growth |  NG - New growth |  800 NEOPLASMS,  NOS |

- Synonyms

## Retrieve related concepts for all search terms from Concept Space
Limit related concepts based on Deep Semantic Parsing 
(by means of the UMLS Semantic Net)

- _Limit related concepts_ based on Deep Semantic Parsing
- (by means of the UMLS Semantic Net)

- _Deep Semantic Parsing - Algorithm_
-  Step 1: establish the semantic context for each original query (_find the semantic types and relations_ of the search terms)
-  Step 2: for each related concept, find if it _fits the established context_
-  Step 3: reorder the final list based on the weights of the terms (relevance weights from CancerSpace)
- _ _Step 4: select the best terms (highest weights) from the reordered list

- Related Concepts

## Are lymph nodes and stromal cells related to each other?

## Medical Concept Mapping:

- User Validation

## Input: 
30 actual cancer related user-queries
Input Method:
Original Queries
Cleaned Queries
Term Input

- Input:
- 30 actual cancer related user-queries
- Input Method:
- Original Queries
- Cleaned Queries
- Term Input

- Golden Standards:
- by _Medical Librarians_
- by _Cancer Researchers_
- Recall and Precision:
- based on the Golden Standards

-   Study 1: _Incorporating Synonyms_
-   Study 2: _Incorporating Related Concepts_

## Original Query: “What causes fibroids and what would cause them to enlarge rapidly (patient asked Dr. B and she didn’t know)” 

Cleaned Query: “What causes fibroids and what would cause them to enlarge rapidly?”

Term input: “fibroids”

- Cleaned Query: “*What causes fibroids and what would cause them to enlarge rapidly?”*
- Term input: “*fibroids”*

- Example of a Query

## Golden Standards

## Adding Metathesaurus synonyms doubled Recall without sacrificing Precision.

- Adding Metathesaurus synonyms _doubled Recall without sacrificing Precision_.
- WordNet had no influence.

## Adding Synonyms did not improve Recall, but it lowered Precision.

## Adding Concept Space terms increased Recall.

- Adding Concept Space terms _increased Recall_.
- Precision did not suffer when Semantic Net was used for filtering.

## Adding Concept Space had no effect on Recall or Precision.

- Adding Concept Space had _no effect on Recall or Precision_.

## There was no difference in performance for Original and Cleaned Natural Language Queries
Medical Librarians:
provided large Golden Standards
14% of the terms could  be extracted from the query
adding synonyms and related concepts doubled recall, without affecting precision
Cancer Researchers:
provided very small Golden Standards
22% of the terms could be extracted from the query
adding other terms did not increase recall, but lowered precision

- _Medical Librarians_:
- provided large Golden Standards
- 14% of the terms could  be extracted from the query
- _adding synonyms and related concepts doubled recall, without affecting precision_
- _Cancer Researchers_:
- provided very small Golden Standards
- 22% of the terms could be extracted from the query
- _adding other terms did not increase recall, but lowered precision_

- Conclusions of the User Studies

## System Developments:HelpfulMED

## HelpfulMED  on the Web

- Target users:  _Medical librarians, medical professionals, advanced patients_
- _One Site, One World_
- Medical information is abundant on the Internet
- No Web-based service currently allows users to search all high-quality medical information sources from one site

## Search among high-quality medical webpages, updated monthly  (350K, to be expanded to 1-2M webpages)
Search all major evidence-based medicine databases simultaneously
Use Cancer Space (thesaurus) to find more appropriate search terms (1.3M terms)
Use Cancer Map to browse categories of cancer journal literature (21K topics)

- Search _all major evidence-based medicine databases_ simultaneously
- Use _Cancer Space (thesaurus)_ to find more appropriate search terms (1.3M terms)
- Use _Cancer Map_ to browse categories of cancer journal literature (21K topics)

- HelpfulMED  Functionalities

## Spider technology navigates WWW and collects URLs monthly

- _UMLS filter_ and _Noun Phraser_ technologies ensure quality of medical content
- Web pages meeting threshold level of medical phrase content are collected and stored in database
- Index of medical phrases enables efficient search of collection
- Search engine permits Boolean queries and emphasizes exact phrase matching

- Medical Webpages

## 5 databases (to be expanded to 12) including:
full-text textbook (Merck Manual of Diagnosis and Therapy)
guidelines and protocols for clinical diagnosis and practice (National Guidelines Clearinghouse, NCI’s PDQ database)
abstracts to journal literature (CancerLit database, Americal College of Physicians’ journals)

Useful for medical professionals and advanced consumers of medical information

- full-text textbook (_*Merck Manual of Diagnosis and Therapy*_)
- guidelines and protocols for clinical diagnosis and practice (_*National Guidelines Clearinghouse*_, NCI’s _*PDQ*_ database)
- abstracts to journal literature (_*CancerLit*_ database, _*Americal College of Physicians’ journals*_)
- Useful for medical professionals and advanced consumers of medical information

- Evidence-based Medicine Databases

## Suggests highly related noun phrases, author names, and NLM Medical Subject Headings

- Phrases automatically transferred to “Search Medical Webpages” for retrieval of relevant documents
- Contains _1.3 M unique terms_, 52.6 M relationships
- Document database includes _830,634 CancerLit abstracts_

- HelpfulMED Cancer Space

## Multi-layered graphical display of important cancer concepts supports browsing of cancer literature

- Document server retrieves relevant documents
- Presents _21,000 topics_ of documents in 1180 maps organized in 5 layers

- HelpfulMED Cancer Map

## HelpfulMED Web site

- http://ai.bpa.arizona.edu/HelpfulMED

## HelpfulMED  Search of Medical Websites

## HelpfulMED  search of Evidence-based Databases

## Consulting HelpfulMED Cancer Space (Thesaurus)

## Browsing HelpfulMED Cancer Map

## For Project Information:

**For Project Information:**
**http://ai.bpa.arizona.edu**
**(hchen@bpa.arizona.edu)**
**For HelpfulMED Demos:**
**http://ai.bpa.arizona.edu/HelpfulMED**