-

VOOGO: A SEARCH TFOR CLINICAL DATA Qing Zeng -Treitler Treitler, Doug Redd VA Salt Lake City 
INTRODUCTION INTRODUCTION 

 I i f Increasing amount of d i li data in cli

ni cal i repositories 

 VINCI (30 million patients and 1.8 billion notes) 

 Increasing need to direct access and analyze clinical Increasing need to direct access and analyze clinical  Simple queries often result in high false positives and 

negatives 
VOOGO FEATURES FEATURES 

 T d d Text and structured d h ata search  Multiple views of results 

 Summary Summary  Patient  Document 

 Query suggestion 
EVALUATION #1 EVALUATION #1 

 300 fnotes 

from post- traumatic stress di d (PTSD) 

disor

der (PTSD) 

 300 notes from diabetes mellitus (DM) patients  12 topics (e g suicide ideation or diet/exercise topics (e.g. suicide ideation, or diet/exercise

) 

 Clinician review 

 0: Irrelevant, y ,1: Possibl

y relevant, and 2:Defintiel

yrelevant
t t

EVALUATION #1 EVALUATION #1 

 19 iquer es  3 query expansion methods  Baseline method: sum of TFIDF Baseline method: sum of TFIDF  4 performance measures 

 Precision (p p ositive predictive value) Recall (sensitivity)  F-measure (the harmonic mean of precision and recall)  P10 ( 10 k d d ) P10 (top 10 ranked documents ) 
RESULTS RESULTS 

07 0.5 0.6 0.7 

Baseline 

T i P 

0.2 0.3 0.4 Synonym 

Topic model Semrap 

Treating Poss

as Relevant 

0 

0.1 

Precision Recall F-measure 

Precision Recall F-m

Baseline Baseline 0.62 0.23 0.3

Synonym 0.46 0.44 0.3Top ic model 0.46 0.47 0.4

SemRep 0.55 0.31 0.3
RESULTS

0 

RESULTS 

05 

0.35 

0.4 

0.45 

0.5 

T ti P 

0.2 

0.25 

0.3 Baseline 

Synonym 

Trea

Topic model Semrap 

ting Poss

as Irrelevant 

0 

0.05 

0.1 

0.15 

Precision Recall F-measure 

Precision Recall F-me

B li Baseline 0 48 . 0 25 . 0 29 . Synonym 0.34 0.47 0.35 Topic model Topic model SemRep 

0 35 . 

0.43 

0 48 . 

0.33 

0 38 . 0.34 
y y p

RESULTS RESULTS 

0.5 0.6 0.7 0.3 

Top 10 0.4

Precision 

(P@10) 

Baseline Synonym Topic Model Predicate 

0 

0.1 0.2 

Treating Possible as Treating Possible as 

yes No 

Baseline Synonym Topic Mode

Treating Possible as yes 0.61 0.57 0.6

Treating Possible as No 0.52 0.43 0.4
EVALUATION #2 EVALUATION #2 

 19 iquer es  3 query expansion methods and one ensemble metho Baseline method: sum of TFIDF Baseline method: sum of TFIDF  4 performance measures 

 MAP ( Mean avera

g p ) e 

precision)

 P10 
t

MEAN AVERAGE MEAN AVERAGE PRECISION PRECISION 

0 48 0.46 0.47 0.48 0.44 0.45 

AP 

0 41 0.42 0.43

M 

0.39 

0.4 

0.41 

B li Baseline Synonym T i M d l TopicModel P di i Predication 
t

P10 

0 66 0.64 0.65 0.66 0.61 0.62 0.63 

ge P(10) 

0.58 0.59 

0.6

Averag 

0.55 0.56 0.57 

B li Baseline Synonym T i M d l TopicModel P di i Predication 
EVALUATION #3 EVALUATION #3 

 50 i (TREC) 50 queries (TREC)  100k documents  Expansion methods Expansion methods  Concept indexing and negation filtering  Baseline method: Lucene Baseline method: Lucene  4 performance measures 

 infAP (Inferred Average Precision)  infNDCG (Inferred Normalized Discounted Cumulated Gain)  R-prec (R-precision)  P10 
RESULTS

04 0.5 0.6 

Lucene baseline 

0.2 0.3 0.4 

Median 

0 

0.1 

infAP infNDCG P@10 

Ensemble + Concept+ Negation Boosting 

RESULTS

infAP infNDCG P@10

infAP infNDCG P@10

Lucene baseline 0.152 0.388 0.421 

Median 0.170 0.424 0.470 

Ensemble + Concept + 

Negation Boosting 

0.200 0.445 0.479 
FUTURE STEPS FUTURE STEPS 

 M ore structured 

fi eld s 

 Complex text criteria  Temporal search support Temporal search support  Concept indexing  Intelligent Visual analytics Intelligent Visual analytics 
ACKNOWLEDGEMENT ACKNOWLEDGEMENT 

Thi k i f d d b VA This work is 

fun

d

ed 

by VA g

HIR 08-374 and VINCI HIR-
CONTACT US CONTACT US 

 Q @ h d Q.t.zeng

@uta

h.e

du 

 Doug.redd@utah.edu