Contents
Information Extraction and Sentence
Classification applied onClinical Trial MEDLINE Abstracts

1. Backgrounds
2. Methods① Information Extraction
by manually written regular expressions② Sentence Classification

Nara Institute of Science and Technology
Kazuo Hara and Yuji Matsumotoby automatically produced patterns by BACT

3. Experiments
4. Future Work
1

Backgrounds

2

Targets of Information Extraction• Needs in the medical society• 「Compared Treatment」– Practice of Evidence Based Medicine• MEDLINE– The US National Library of Medicine's bibliographic
database including pharmaceutical domain– The most popular information source for finding evidenceof new therapy• Our Research Goal:– The aim of clinical trial is to investigate the efficacy
and safety of the new treatment comparing with current
therapy.

• 「Endpoint」– To what the new treatment shows greatness?

• 「Patient population」– To whom the new treatment shows greatness?

– To construct the information extraction (IE) system fromMEDLINE clinical trial abstracts
3

Example of the Targets• INPUT– 「We compared drug X and drug Y by investigating bloodpressure and survival time in patients with high bloodpressure. 」• OUTPUT– 「Compared Treatment」（two targets） → drug X, drug Y– 「Endpoint」（two targets） → blood pressure, survival time
– 「Patient Population」（one target） → patients with high bloodpressure※ Take notice that each clinical trail has sometimes morethan one IE target.
54Research Goal
• To construct the information extraction (IE)system from MEDLINE clinical trial abstracts【 a MEDLINE abstract 】•TITLE: Peginterferon Alfa-2a plus ribavirin
versus interferon alfa-2a plus ribavirin forchronic hepatitis C in HIV-coinfected persons.
•BACKGROUND: Chronic hepatitis C virus(HCV) infection is a cause of major … interferonplus ribavirin for the treatment of chronic hepatitis
C in persons coinfected with HIV.
•METHODS: A total of 66 subjects wererandomly assigned to receive … either a virologic
response or histologic improvement.
•RESULTS: Treatment with peginterferon andribavirin was associated with a significantly
higher rate of sustained virologic response thanwas treatment with interferon and ribavirin. ・・・【 IE result 】(1) Compared Treatment：・peginterferon alfa-2a plus
ribavirin・interferon alfa-2a plus ribavirin(2) Endpoint：・sustained virologic response(3) Patient Population：
・persons coinfected with HIV
61

The preliminary process

Contents

INPUT sentence: from a clinical trial MEDLINE abstract“We conducted a multi-center, randomized trial comparing peginterferon plus
ribavirin with interferon plus ribavirin for the treatment of chronic hepatitis C
in persons co-infected with HIV.”

1. Backgrounds
2. Methods① Information Extraction

NP chunking: by YamCha (Kudo and Matsumoto, 2001)

by manually written regular expressions“[We] conducted [a multi-center, randomized trial] comparing[peginterferon plus ribavirin] with [interferon plus ribavirin] for [thetreatment] of [chronic hepatitis C] in [persons] co-infected with [HIV].”② Sentence Classification
by automatically produced patterns by BACT

3. Experiments
4. Future Work

NP tagging: manually by domain specific knowledge

7“[We] conducted [STUDY] comparing [DRUG] with [DRUG]
for [THERAPY] of [DISEASE] in [PATIENT] co-infectedwith [DISEASE].”Tag definition【Tag】【covered concept】
DISEASE: disease, symptom, virusDRUG: drug, chemical compound
STUDY: clinical trial
THERAPY: treatment, regimen
PATIENT: participants in the trial
TARGET: endpoints
SCHEDULE: time schedule of the trial
VALUE: value of TARGETNUMBER: numeral expression

Task definition【example】
chronic hepatitis Cinterferon
clinical trial
antiviral treatment
HBeAg-positive patients
efficacy and safety
an additional 24 weeks
significantly higher rates
20 percent
9NP tagging: manually by domain specific knowledge“[We] conducted [STUDY] comparing [DRUG] with [DRUG] for[THERAPY] of [DISEASE] in [PATIENT] co-infected with
[DISEASE].”
Result:② Sentence
Classification: by BACT (aCompared Treatment: +1 （Yes）

Boosting Algorithm forClassification of Trees; Kudoand Matsumoto, 2004)

Endpoint: -1 （No）
Patient Population: +1 （Yes）
Result:① Information
Extraction:
Pattern matching byregular expressions

Compared Treatment : “peginterferon plusribavirin”
Compared Treatment : “interferon plus ribavirin”
Endpoint : （none）
Patient Population : “persons co-infected with

① Information Extraction by
regular expressions

ContentsNP tagging: manually by domain specific knowledge1. Backgrounds
2. Methods“[We] conducted [STUDY] comparing [DRUG] with [DRUG] for[THERAPY] of [DISEASE] in [PATIENT] co-infected with [DISEASE].”① Information Extraction: by manually written regular expressions① Information Extraction「Compared Treatment」： “compar＊” [DRUG] “with” [DRUG]

by manually written regular expressions② Sentence Classification「Patient Population」： [PATIENT] ＊ with [DISEASE]

by automatically produced patterns by BACT

Result:

3. Experiments
4. Future Work

Compared Treatment : “peginterferon plus ribavirin”
Compared Treatment : “interferon plus ribavirin”
Endpoint : （none）
11Patient Population : “persons co-infected with HIV”

122

② Sentence Classification

Example of Sentence Classification“[We] conducted [STUDY] comparing [DRUG] with [DRUG] for[THERAPY] of [DISEASE] in [PATIENT] co-infected with
[DISEASE].”NP tagging: manually by domain specific knowledge“[We] conducted [STUDY] comparing [DRUG] with [DRUG] for[THERAPY] of [DISEASE] in [PATIENT] co-infected with
[DISEASE].”② Sentence Classification: by BACT (Kudo andMatsumoto, 2004) , that automatically produce optimal patterns
by machine learning.
Result:Compared Treatment: +1 （Yes）
Endpoint: -1 （No）
Patient Population: +1 （Yes）※ The accuracy of IE may improve if we apply ① Information Extraction 13
to “yes” sentences only.

constructedpatterns(default)“We”“STUDY”“DRUG”“with”“with DRUG”“for”“THERAPY”“for THERAPY”“of”“DISEASE”“of DISEASE”“in”“PATIENT”“in PATIENT”“with DISEASE”
Total weight
yes or no

ComparedTreatment-0.079
0.051
0.013
0.045
0.008
-0.003–0.014-0.0060.002
0.000-0.009-0.013
0.007
–
0.006
0.035
+ 1 (yes)

Endpoint

Patient
Population-0.2100.105
0.081-0.0030.037-0.050-0.006-0.001
–
–
0.034
–
0.000
0.070
–
0.018
0.074
+ 1 (yes)-0.141
0.016
0.065
0.009
-0.002
–
–
–
–
–
–
–
0.012
-0.0280.000
0.005-0.065-1 (no)※ BACT calculates
the weight for eachpatterns by learningfrom training data.
（red numerals areminus weight）

14

Interpretation of patterns by BACT
automatically constructed patterns by
BACT that include “DRUG”

Contents

ComparedPatient
Endpoint
Treatment
Population“PATIENT received DRUG”

0.048

–

–

“DRUG”

0.046

–

–

“TARGET of DRUG”–0.035–“DRUG, DRUG”

0.013

–

–

“received DRUG”

0.01

0.023

–

“of DRUG”

0.006

0.012

–

“with DRUG”-0.004–-0.026“to DRUG”-0.013–-0.012“in DRUG”-0.019––※ This table shows that not all NPs tagged by “DRUG” correspond toIE target of “Compared Treatment”.

1. Backgrounds
2. Methods① Information Extraction
by manually written regular expressions② Sentence Classification
by automatically produced patterns by BACT

3. Experiments
4. Future Work1516

DataResults• Precision, Recall（5-fold cross validation）

We downloaded the 50 most recent abstracts ofclinical trials from the MEDLINE database:
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi
on October 2004.

① InformationExtraction② Sentence
Classification （dep）② Sentence
Classification （Ngram）

Query:「"hepatitis"[MeSH Terms] AND hasabstract[text] AND
Randomized Controlled Trial[ptyp]」※ To simplify the experiment, abstracts were selectedComparedTreatment
precision recall

precision

recallprecision

recall

84.8%

64.0%

77.0%

52.0%

76.2%

82.0%

86.8%

78.5%

84.7%

72.2%

75.2%

71.4%

82.6%

71.7%

85.7%

73.2%

81.5%

81.5%

Endpoint

Patient Population• dep is outperformed by Ngram with respect to “Patient Population”.
We can guess the reason here: parse errors occurred in many of thedependency trees caused by PP attachment ambiguity (“PATIENTwith DISEASE”).

from the medical area of hepatitis.
17183ContentsFuture Work

1. Backgrounds
2. Methods• Automatic NP Tagging– In this experiment, manually tagged by domain
specific knowledge.

① Information Extraction• Bigger Corpus

by manually written regular expressions– In this experiment, only 50 abstracts.

② Sentence Classification• Apply 「① Information Extraction」 to theresult from 「② Sentence Classification」

by automatically produced patterns by BACT

3. Experiments
4. Future Work– How Sentence Classification contribute to the
Information Extraction?
19204