Interactive Visual Text Analytics
for Decision Making

Shixia Liu
Microsoft Research Asia

1

IBM China Research Laboratory

Text is Everywhere We use documents as primary information artifact
in our lives
 Our access to documents has growntremendously in recent years due to networking
infrastructure– WWW– Digital libraries–…

2IBM China Research Laboratory

Big Question What can information visualization provide to help
users in understanding and gathering information fromtext and document collections?

3

IBM China Research Laboratory

Outline
 Example tasks in text analytics
 Visually analyzing textual information– Dynamic Word Cloud– Topic-based Visual Text Summarization– TextFlow: Towards Better Understanding of Evolving
Topics in Text

 Text Visualization Perspectives

4IBM China Research Laboratory

Text Analytics: Our Understanding
How can I find information buried inside the piles of text?

Information finding5IBM China Research Laboratory

Text Analytics: Our Understanding
What is in my text?

What’s inside theNHTSA Data:

What are the major
causes of injuries• 450,000+
documents• 70,000+ patient
emergency room records

What did mycustomers say aboutmy hotels• 3000+ customerposted reviews

Information Understanding: Text Summarization
6

IBM China Research Laboratory

Text Analytics: Our Understanding
What is in my text?

Which hotel features do How customers’
my customers
sentiment havelike/dislikechanged toward myhotels• 3000+ customer reviews• 3000+ customerposted reviews

How do customers
feel about my new
product launch• thousands of eopinion postings

Insight Discovery: Sentiment Analysis
7

IBM China Research Laboratory

Text Analytics: Our Understanding
What is in my text?
What are thecorrelations oftire problems andhighway death inthe NHTSA Data:• 450,000+documents

What are thecorrelations ofpatient gender andthe cause of injury• 70,000+ patient
emergency room
records

Compare thecustomers’
attitude toward
our product withtheirs for ourcompetitors• thousands of eopinion postings

Decision Making and Problem
Solving: Text Analysis++8IBM China Research Laboratory

Major Challenges
 Huge amounts of complex information– Understanding the meanings of free text isjust hard– Performing analysis on top of that is harder Different people want different things– No one-size-fits-all solutions

 People may not know what they want
– “Tell me something I don’t know”– “I will tell you when I see it”

Machines are *not* just smart enough.
9

IBM China Research Laboratory

Outline
 Example tasks in text analytics

 Visually analyzing textual information– Dynamic Word Cloud– Topic-based Visual Text Summarization– TextFlow: Towards Better Understanding of Evolving
Topics in Text

 Text Visualization Perspectives

10IBM China Research Laboratory

Cui et al. IEEE CG&A 2011

Dynamic Word Cloud Word clouds for content overview– Aesthetic issues– Inadequate for temporal patterns

 Standard time chart: trend– Inadequate for correlations

11

IBM China Research Laboratory

Our Solution
 A evolution trend chart + word clouds– Measure the evolution– Ensure the semantic coherence between clouds

12

IBM China Research Laboratory

Evolution Measurement
 Conditional entropy: measure the amount ofinformation contained by Xi but not by Xj

13

IBM China Research Laboratory

Word Cloud Layout
 Geometry meshes to ensure the semantic coherence– Semantically related words stay together– The same word in different clouds stay at the similar place

14

IBM China Research Laboratory

Example: CG&A Abstracts

1989
2004

1994

1999

2009
1,984 abstracts IEEE Computer
Graphics and Applications (CG&A)
from 1981 to 200915

IBM China Research Laboratory

Comparison with Wordle

13,828 news articles
16IBM China Research Laboratory

Outline
 Example tasks in text analytics

 Visually analyzing textual information– Dynamic Word Cloud– Topic-based Visual Text Summarization– TextFlow: Towards Better Understanding of Evolving
Topics in Text

 Text Visualization Perspectives

17

IBM China Research Laboratory

Liu et al. CIKM 09

Demo: Topic-based Visual Text Summarization
Y axis encodes topicsignificance(3) Height encodes
the number of emails
in the topic at thistime(1) A layer
represents a topic

X axis encodes time

~10,000 emails in 2008

(2) Topickeywords

18

IBM China Research Laboratory

Demo19

IBM China Research Laboratory

Visual Text Summarization: Key Challenges
 How to summarize massive, time-varying text corpora– Huge amounts of complex information– Accuracy + extensibility– Time-varying– Effectiveness How to visually explain text summarization results– Which visual metaphors to use– How to switch between visualizations

 How to allow users to provide feedback or articulate
their needs– Incorrect summarization results or varied user needs

20IBM China Research Laboratory

TIARA Overview
Text collection

Text Preprocessing
Text content + meta data
Text Summarization
Summarization results
Data Transformation

Transformed results
Visualization
User
Interaction
21

IBM China Research Laboratory

TIARA Technical Focus
Text collection

Text Preprocessing
Text content + meta data
Text Summarization
Summarization results
Data Transformation

Transformed results
Visualization
User
Interaction
22IBM China Research Laboratory

Adopted Text Summarization: LDA-based Topic Analysis
 Latent Dirichlet Allocation(LDA) model [Blei et al. 03]– Statistical analysis that
requires no extra knowledge
about the language or theworld (high portability)

kth document inthe collection{…, P(Ti | Dk), …}
A set of topic
probabilities– Keyword-based description ofthematic structure of a text
collection (high compaction
rate for scalability){T1, …Ti,… TN }– A finer grained model thanclustering{W1, …, Wj, …, WM}• One document  multiple
topics  finer-grainedsummarization

A set of topic A set of topics
keywordsA set of word probabilities{…, P(Wj | Ti), …}

23

IBM China Research Laboratory

LDA Data Transformation
kth document inthe collection{…, P(Ti | Dk), …}

A set of topicprobabilities
A set of topics{T1, …Ti,… TN }A set of keywords
Rank the topics to present
most interesting ones first

Select keyword sub-sets fortime-based summary{…} t-1, {…, Wj, …}t, {…} t+1,

{W1, …, Wj, …, WM}

A set of word probabilities{…, P(Wj | Ti), …}
24

IBM China Research Laboratory[Song et al. CIKM 09]Topic Ranking by User Interests
Domain-dependent
activeness metric Rank topics by “strength”

rank (Tk )  f  (Tk ) ,  (Tk ) ,  (Tk ) N ˆ (T )  NMkm 1
Mm 1

m m, ktopic
variance

topic
coveragem Rank topics by “distinctiveness”

v~kT Lv~krank (Tk )  l (Tk )  ~T ~v Dv
kgraph degree
matrixk (T ) ˆ   (T )Nmm, kkm 1
Mkgraph Laplacian

Mm 1

Nm

doc-topicdistribution

Tˆˆˆfor each Tk, vk  1,k , 2,k ,..., M ,kv~k is normalized vk
25

IBM China Research Laboratory

Topic Ranking by User Interests: Experiments Goal– Measure which metric produces more “important” topics

 Data sets
– Email• 8326 email messages• 958,069 word tokens– News• 34,690 documents• 11,491,246 word tokens

 Method– Users indicate the importance of top-K ranked topics• Very important• Somewhat important• Unimportant

26

IBM China Research Laboratory

Topic Ranking by User Interests: Results Email data (by F1 measure)strength
distinctiveness News data (by F1 measure)strength
distinctiveness27

IBM China Research Laboratory

TIARA Technical Focus
Text collection

Text Preprocessing
Text content + meta data
Text Summarization
Summarization results
Data Transformation

Transformed results
Visualization
UserInteraction
28

IBM China Research Laboratory

Visualizing Text: Existing Work Visualize text at a highlevel (e.g., documents)– Issues• Inadequate information

 Visualize text at a low
level (e.g., words andphrases)– Issues• Missing a big picture• Scalability

 Few on explaining
advanced text analysis
results

29

IBM China Research Laboratory[Liu et al. CIKM 09]

Our Approach Create visual metaphors toexplain text summarization
results
 Use interaction to let people– Examine data from multiple
aspects

 Support both top-down andbottom-up analysis– Forest  tree  leaf
– Leaf  tree  forest Design principle– Augment common visual
metaphors to convey complex
summarization results
30IBM China Research Laboratory

Visual Text Summary Metaphor
Data to be visualized:
1. Topics: {T1, …Ti,… TN } and their probabilities
2. For each Ti ,Topic keywords by time: … {…,wik, …, }t, …
and their probabilities over time
3. For each Ti , Topic strength: {..., Si(t), …} over time

Visual encoding: Augmented stacked graph31

IBM China Research Laboratory

Enhanced Stacked Graph: Key Steps
1. Computing geometry of layers

2. Layer coloring
3. Layer ordering
4. Layer labeling32IBM China Research Laboratory

Enhanced Stacked Graph: Key Steps
1. Computing geometry of layers

2. Layer coloring

Byron_Infovis08

3. Layer ordering
4. Layer labeling33

IBM China Research Laboratory

Enhanced Stacked Graph: Layer Ordering Goals– Minimize distortion– Maximize usable space– Ensure semantic coherence

unorderedordered34

IBM China Research Laboratory

Enhanced Stacked Graph: Layer Ordering (cont’d) Our approach– Optimization-based approach to• Minimize volatility (curvature) oftopics• Maximize geometriccomplementariness of adjacent
topics• Ensure semantic proximity of topics

unorderedfor topics
Ti and Tj

F ( p'ij (t )); p'ij (t )  pi (t )  p j (t )ordered35

IBM China Research LaboratoryEnhanced Stacked Graph: Layer Ordering (cont’d)Observed topic
36

IBM China Research Laboratory

Enhanced Stacked Graph: Layer Ordering (cont’d)Same topic

37

IBM China Research Laboratory

Enhanced Stacked Graph: Layer Ordering (cont’d)Same topic

Alternative solution:
Interactive reordering38IBM China Research Laboratory

Enhanced Stacked Graph: Layer Labeling Goals– Temporal proximity– Informativeness39IBM China Research Laboratory

Enhanced Stacked Graph: Layer Labeling (cont’d) Our approach [Liu et al. CIKM09]– Constraint-based space allocation– Particle-based layout [Luboschik et al. 08] + wordle

s2s1t-tt+2tT

40

IBM China Research Laboratory

Enhanced Stacked Graph: Layer Labeling (cont’d)41

IBM China Research Laboratory

Top-down and Bottom-up Analysis
VisualSummary

Top-down
navigation

Bottom-up
navigation

Summary of selected emails

Iterative analysis
42

IBM China Research Laboratory

Interacting with Visual Summary

Selected keyword “cotable” and relevant email snippets
43

IBM China Research Laboratory

Interacting with Visual Summary“people” involved and theirrelationships

44

IBM China Research Laboratory

TIARA Preliminary Evaluation Goal
– What kind of tasks can TIARA help users accomplish?
– What factors impact the use of TIARA?

 Data set– Emails (~10,000)

 12 Participants: 6 familiar with the email owner’s work
 Tasks– Task 1: Analyze emails between two people– Task 2: Answer questions about specific projects– Task 3: Answer questions characterizing the email owner’s work45

IBM China Research Laboratory

TIARA Preliminary Evaluation
 Method– Objective measures• answer completion rate, answer error rate, and answer time

– Subjective measures• usefulness, usability, and system satisfaction– Compared TIARA and Themail for Task 146

IBM China Research Laboratory

Sample Evaluation Questions

47

IBM China Research Laboratory

TIARA Evaluation Results
 Type of tasks impacted the effectiveness of TIARA– TIARA helped users complete complex analytic tasks faster– Visual summary too coarse for extracting details (e.g.,names)
 User’s knowledge impacted the effectiveness of TIARA– TIARA helped knowledgeable users more48

IBM China Research Laboratory

Application Example: Healthcare Visualize unstructured data (text) to facilitate analysis– Cause of injury– Reason for visit– Diagnosis

 Handle multiple fields of text data and show their correlation
 Leverage structured data to help better illustrate textinformation– Gender + Cause of injury

49

IBM China Research Laboratory

Correlation between Structured and Text Fields

50

IBM China Research Laboratory

Correlation between Two Text Fields

Correlation between two fields,
51
diagnosis and reason for visit

IBM China Research Laboratory

Outline
 Example tasks in text analytics

 Visually analyzing textual information– Dynamic Word Cloud– Topic-based Visual Text Summarization– TextFlow: Towards Better Understanding ofEvolving Topics in Text
 Text Visualization Perspectives

52

IBM China Research Laboratory

TextFlow: Towards Better Understanding of Evolving
Topics in Text
Cui et al. Infovis 11

 Problems– Understanding topic evolution in large text collections is important• Keep abreast of hot, new, and intertwining topics• Gain insight into the latent topics

 Challenges– Model topic merging/splitting patterns– Visually convey the topic merging/splitting patterns in an intuitive way– Facilitate analytical reasoning Solutions– Leverage Hierarchical Dirichlet Processes to model topic merging/splitting– Augment the familiar visual metaphor, the river flow, to convey thecomplex analytic results– Interact with the topic from global structure to local salient features53

IBM China Research Laboratory

Related Work
 Little work has focused on studying topic merging andsplitting patterns
 It has barely been touched by using visual analysis
techniques to interactively analyze complex topic
evolution from multiple perspectives

54

IBM China Research Laboratory

TextFlow Overview

55

IBM China Research Laboratory

Topic Data and Relationship Extraction

56

IBM China Research Laboratory

Merging/Splitting Relationships
 Merging input from time t-1 to t: Splitting output from time t-1 to t:

57

IBM China Research Laboratory

Critical Event Extraction
Domain-dependent
activeness metric Critical events– Birth, death, merge, and split

 Score of a merging event

Neighborhood
setEntropy score

 Score of a splitting event

58

IBM China Research Laboratory

Keyword Correlation Discovery
 Extract “noun phrases,” “verb phrases”, and “namedentities” in each document, and count co-occurrencesamong them59IBM China Research Laboratory

Visualization Design60

IBM China Research Laboratory

Topic Evolution as Flow

Number ofdocuments

61

IBM China Research Laboratory

Critical Event as Glyph

62

IBM China Research Laboratory

Keyword Correlation as Thread
Alternatives

Keyword thread63

IBM China Research Laboratory

Layout Algorithm A three-level
Directed Acyclic
Graph (DAG)

64

IBM China Research Laboratory

Interactive Exploration

65IBM China Research Laboratory

Application Example – VisWeek Pulications

geometry/rendering
flow/vector

volume/rendering
structure/layout

document/temporal

exploration/analytics

933 research papers
66IBM China Research Laboratory

Application Example - VisWeek Pulications

67

IBM China Research Laboratory

Application Example – Bing News

events happening in Egypt
protest in Egypt

budget/spending

Obama government’s actions

republican/campaign

68IBM China Research Laboratory

Application Example – Bing News

Comparing timelines extracted from different topics: (a) timeline
extracted from topic B, which mainly describes protest events
happening in Egypt; (b) timeline extracted from topic C, which
focuses on the Obama’s government’s actions on events in Egypt.

69

IBM China Research Laboratory

Application Example – Bing News70

IBM China Research Laboratory

Video71

IBM China Research Laboratory

Outline
 Example tasks in text analytics

 Visually analyzing textual information– Dynamic Word Cloud– Topic-based Visual Text Summarization– TextFlow: Towards Better Understanding of Evolving Topics inText

 Text Visualization Perspectives

72

IBM China Research Laboratory

Future Text Visualization Topics
 Interactive, incremental text analytics
Topic editing

TIARA LDA latent text semantic David models edt keywords
summarization topic
Topic split
bibframe spotfire framemaker bibtex tableau visualization
spotfire tableau
visualization

bibframe framemakerbibtex Multi-level visual text summarization (keywords + sentences) Multi-faceted text analytics (e.g., summarization + sentimentalanalysis) Multimedia document summarization (text + image + video) Interactive, visual social media analysis
73

IBM China Research Laboratory

Acknowledgements
Nan Cao, Yingcai Wu, Weiwei Cui, Prof. Huamin Qu (HKUST)
Dr. Michelle X Zhou (IBM Almaden Research Center)74

IBM China Research Laboratory

Thanks a lot for your attention!

75