Information Visualization:
Connecting the Abstract and Visual Worlds
Shixia Liu (刘世霞)Internet Graphics Group,
Microsoft Research Asia

Outline• Definition• Criteria• History• InfoVis 1.0• InfoVis 2.0• InfoVis 3.0• Opportunities and challenges

Outline• Definition• Criteria• History• InfoVis 1.0• InfoVis 2.0• InfoVis 3.0• Opportunities and challenges

What is Information Visualization• Use graphical representations of abstract information to facilitate– Explanation– Information understanding– Decision making–…

• Translation from the abstract language to the visual/intuitive
languageWhy InfoVis – Basic Problem
We live in a new ecology: how
do you make sense of it

People are 5 times fasterwith the visual aid(slide “borrowed” from PARC User Interface Research Group)Why InfoVis - Web Ecologies

10000000
1000000

Servers

100000
10000
1000
100
10

1 new server every 2 seconds
7.5 new pages per second

1Aug-92 Feb-93 Aug-93 Feb-94 Aug-94 Feb-95 Aug-95 Feb-96 Aug-96 Feb-97 Aug-97 Feb-98 Aug-98
Source: World Wide Web Consortium, Mark Gray, Netcraft Server Survey(slide “borrowed” from PARC User Interface Research Group)Why InfoVis - Scientific Journals
Journals/person increases 10X every 50 years
1000000
100000
10000

Journals

1000

Journals/People x106

100
10
1
0.1

0.01
1750

Darwin
1800

1850

Year

1900

V. Bush

You

1950

2010

Why InfoVis - Innate Human
Capacity
1000000

100000
10000

1000
100

10
1
0.1
0.01
1750

Darwin
1800

1850

1900

V. Bush

You

1950

2010

Bad Visualization?

Good Visualization?

Outline• Definition• Criteria• History• InfoVis 1.0• InfoVis 2.0• InfoVis 3.0• Opportunities and challenges

Criteria: Trustfulness (信)Criteria: Effectiveness (达)Criteria: Elegance (雅)TextFlow

Whisper

Outline• Definition• Objective• History• InfoVis 1.0• InfoVis 2.0• InfoVis 3.0• Opportunities and challenges

History of Information Visualization• InfoVis1.0 (Bronze Age： 青铜时代)– Hand-made drawing (Info Graphics)• InfoVis 2.0 (Silver Age： 白银时代)– Information display• InfoVis 3.0 (Golden Age： 黄金时代)– Visual analytics

Outline• Definition• Objective• History• InfoVis 1.0• InfoVis 2.0• InfoVis 3.0• Opportunities and challenges

InfoVis 1.0: Overview
and gaininsight

Data
Human-made

PictureWe look atthat picture

Ah HA!!

Information Visualization

Goals• Making invisible visible• Communication• Understanding• Gaining insightExampleOutline• Definition• Objective• History• InfoVis 1.0• InfoVis 2.0• InfoVis 3.0• Opportunities and challenges

InfoVis 2.0: Overview
Raw Data
and gaininsight

Transformation
Data Tables

Abstraction

Visualization

We interact withthe visualization

Ah HA!!

Information Visualization

Basic Types of Visual Encodings• “Retinal” properties– spatial position (e.g., x-y axes)– size– shape– color– orientation– texture• “Gestalt” properties– connectivity– grouping (e.g., enclosure)• Animation– view transitions– animated elements

Later in the semester we’ll
examine guidelines onwhen to apply these
mappings…

Pre-attentive Processing• < 200 - 250ms qualifies as pre-attentive– eye movements take at least 200ms– yet certain processing can be done very quickly, implying low-levelprocessing in parallel• If a decision takes a fixed amount of time regardless of the numberof distractors, it is considered to be preattentive.

Accuracy Ranking of Quantitative Perceptual Tasks
Estimated; only pairwise comparisons have been validated(Mackinlay 88 from Cleveland & McGill)Major Topics• List• Chart, timeline• Tree• Tree visualization• Graph• Graph visualization• Multidimensional• Multidimensional dataviz

List Visualization

Tree Visualization• Node-link diagram• Space-filling

Tree Visualization: Hyperbolic Tree
• Problems– Large hierarchies are everywhere– Scalability• Basic idea
– Focus + context (fisheye)– Lay out the hierarchy on a hyperbolic plane and map this plane onto a
circular display region

Hyperbolic Tree: Demo

Tree Visualization: Treemap

Voronoi Treemap• Treemap Problems– Aspect ratios uncontrolled leads to lots of skinny boxes that clutter– Hard to understand the hierarchical structure• Solution– Voronoi treemap

Post Office: What is the area of service?
pi : site points
q : free pointe : Voronoi edgev : Voronoi vertex
v
q

pi

e

Demo

Graph Visualization• Node-link diagram• Matrix• HybridClutter Reduction in Node-Link Diagram• U.S. immigration graph with 1790 nodes and 9798 edges

Clutter Reduction Techniques
Clutter reduction techniques
sampling
filteringAppearancechange point sizechange opacity
clustering
point/line displacement
topological distortion

Spatialdistortion

space-filling
pixel-plotting

dimensional reordering

Temporalanimation

Clustering – Edge Bundling• Visual clutter is a serious problem in huge graph visualization• Solution– Geometry-based edge bundlingInspiration• Road-map style graphs– Cut straight lines into segments– Provide different levels of details– Familiar to everyone(straight line graph)(road-map style graph)

Method

Control Mesh

Example: Artificial Data
128 links, arranged in a circle, roughly from a sourceExample: Airlines
Northwest Airlines in the U.S.
2101 links, 235 nodes

VideoHybrid Graph Visualization• Problem– Huge amounts of rich context social media– These data involve multiple social relations and content context• Challenges– Illustrate both global and local graph structures– Offer the multiscale and cross-scale exploration– ScalabilityHybrid Graph Visualization• Traditional solutions– Force-directed layout– Matrix visualization• New solution– Hybrid methodHybrid Method: NodeTrix• A hybrid visualization for analyzing social networks– Node-link diagrams: the global structure of a network– Adjacency matrices: better support the analysis of communities

NodeTrix

VideoOutline• Definition• Objective• History• InfoVis 1.0• InfoVis 2.0• InfoVis 3.0• Opportunities and challenges

InfoVis 3.0: Motivation• Data/Information is everywhere– Structured + unstructured• Data mining is important, however, it is far from perfectVisual Analytics in a Nutshell
Research and develop visual analytics methods and systems
to facilitate individual or collective human decision making
User interaction

Mapping

Data

Visualization
Resultvisualization

Analytic
technologies

Analytics
enhancement

Data Mining &Machine Learning

Human
Knowledge

Visual Analytics @MSRA• Visual text analytics• Visual social analytics• Visual search log analytics

Topic-based Text Visualization
TIARA

TextFlow
FacetAltas
InfoVis’ 10

ACM TIST ’ 12, KDD’10,
TKDE’ 12

InfoVis’ 11

Hierarchical topic evolution

KDD’13

ViSizer
TVCG’13

Visual User Behavior Analytics
Whisper
InfoVis’12
Breaking news on Twitter
CHI’12

StoryFlow
Infovis’13

OpinionSeer
InfoVis’ 10
Issue competitor

VAST’13

Search Log Analytics
RankExplorer

Taxonomy Construction
InfoVis’12

KDD’12Given a Large Document Collection

Big Question• What can information visualization provide to help users inunderstanding and gathering information from text and document
collections?

Visual Text Analytics: Pipeline
Combine advanced text analytics and interactive visualization
to help end users in the decision making process

Interaction

Topic Mining
Classification

Data

Text Mining

Visualization

Interpretation

Our Focus: Data
Interaction

Topic Mining
Classification

Data

Text Mining

Visualization

Interpretation

Streaming
Heterogeneous

HomogeneousOur Focus: Data
Interaction

Topic Mining
Classification

Data

Text Mining

Visualization

Interpretation

Streaming
Heterogeneous

HomogeneousDemo
Y axis encodes topic
significance(3) Doc #(1) A layer
represents a topic(2) Topic
keywords

X axis encodes time

~10,000 emails in 2008

DemoKey Challenges• Summarize text corpora– Huge amounts of complex information– Time-varying• Visually explain summarization results– Consistent visualization• Provide feedback or articulate their needs– Imperfect summarization results or varied user needs

TIARA Overview
Data collection

Text Preprocessing
Text content + meta data
Text Summarization
Summarization results
Data Transformation
Transformed results
Visualization
User
Interaction

TIARA Overview
Data collection

Text Preprocessing
Text content + meta data
Text Summarization
Summarization results
Data Transformation
Transformed results
Visualization
User
Interaction

Text Summarization• Latent Dirichlet
Allocation (LDA)
model [Blei et al. 03]

kth document in
the collection{…, P(Ti | Dk), …}
A set of topic
probabilities{T1, …Ti,… TN }– High portability– High compaction rate
for scalability– A finer grainedmodel

A set of topics

A set of keywords{W 1, …, W j, …, WM}
A set of word probabilities{…, P(W j | Ti), …}

LDA Data Transformation
kth document in
the collection{…, P(Ti | Dk), …}
A set of topics

Rank the topics to present
most interesting ones first
Select keyword sub-sets fortime-based summary{…} t-1, {…, W j, …}t, {…} t+1,

A set of topic
probabilities{T1, …Ti,… TN }A set of keywords{W 1, …, W j, …, WM}
A set of word probabilities{…, P(W j | Ti), …}

[Song et al. CIKM 09]

Topic Ranking by User Interests

Domain-dependent
activeness metric• Rank topics by “strength”
rank (Tk )  f  (Tk ) ,  (Tk ) ,  (Tk ) N ˆ (T ) 
 N
M

k

m 1
M

m 1

m m, ktopic
coverage

topic
variance (T ) 

M
m 1

k

m

• Rank topics by “distinctiveness”
v~kT Lv~krank (Tk )  l (Tk )  ~ T ~v Dv
kgraph degree
matrixkgraph LaplacianN m ˆm, k   (Tk )



M
m 1

Nm

doc-topic
distribution



T
ˆ
ˆ
ˆ
for each Tk, vk  1,k ,  2,k ,...,  M ,kv~k is normalized vk

Experiments• Goal– Measure which metric produces more “important”
topics• Data sets
– Email: 8326 email messages– News: 34,690 documents• Method– Users indicate the importance of top-K ranked topics• Very important, somewhat important, Unimportant

Results• Email data (by F1 measure)• News data (by F1 measure)TIARA Overview
Data collection

Text Preprocessing
Text content + meta data
Text Summarization
Summarization results
Data Transformation
Transformed results
Visualization
User
Interaction

Visualizing Text: Existing Work• Visualize text at ahigh level• Visualize text at alow level• Few on explaining
advanced text
analysis results

Visual Text Summary Metaphor
Data to be visualized:

1. Topics: {T1, …Ti,… TN } and their probabilities
2. For each Ti ,Topic keywords by time: … {…,wik, …, }t, …
and their probabilities over time
3. For each Ti , Topic strength: {..., Si(t), …} over time
Visual encoding:Augmented stacked graphEnhanced Stacked Graph: Key Steps• Computing geometry of layers• Layer coloring• Layer ordering• Layer labelingEnhanced Stacked Graph: Key Steps• Computing geometry of layers• Layer coloring• Layer ordering• Layer labeling

Byron_Infovis08

Layer Ordering• Goals– Minimize distortion– Maximize usable space– Ensure semantic coherence

unorderedorderedLayer Ordering - Comparison

Observed topic

Layer Ordering - Comparison

Same topic

Layer Ordering - Comparison

Same topic

Enhanced Stacked Graph: Layer Labeling• Goals– Temporal proximity– InformativenessEnhanced Stacked Graph: Layer Labeling (cont’d)• Our approach [Liu et al. CIKM09]– Constraint-based space allocation– Particle-based layout [Luboschik et al. 08] + wordle
s2

s1

t-

t

t+

2t

T

Enhanced Stacked Graph: Layer Labeling (cont’d)Interacting with Visual Summary“people” involved and theirrelationships

Application Example: Healthcare• Visualize text to facilitate analysis– Cause of injury– Reason for visit– Diagnosis• Multiple fields of text data and their correlation• Leverage structured data to help better illustrate text information– Gender + Cause of injury

Correlation between Structured and Text Fields

Correlation between Text Fields

Correlation between two fields,
diagnosis and reason for visitTextFlow - Problems
Understanding topic evolution inlarge text collections is important– Keep abreast of hot, new, and intertwining topics– Gain insight into the latent topics

Application Example

Application Example

DemoExample – VisWeek Publications
geometry/rendering
flow/vector

volume/rendering
structure/layout
exploration/analytics

document/temporal

Example – VisWeek Publications

Challenges• Model– Topic merging/splitting patterns• Visually convey– Topic merging/splitting patterns in an intuitive way• Facilitate– Analytical reasoning

Our Solution• Leverage hierarchical Dirichlet processes– To model topic merging/splitting• Augment familiar visual metaphors (rivers)– To convey the complex analytic results• Support interactions at different levels– From global structure to local salient features

TextFlow Overview

Topic Data and Relationship Extraction• Incremental Hierarchical Dirichlet Processes– Online learning of the topics in text
– Automatically detect the topic numbers– Extract the merge/splitting relationships• Based on document topic change• Online compute the merging/splitting probabilities

Splitting RelationshipTime tTime t+1
Predict

RefineMerging RelationshipTime tTime t+1
Predict

RefineCritical Event Extraction• Types of critical events– Birth, death, merge, and split• Scoring the merging/splitting event– Number of the branches– Entropy of the branching probabilities

Keyword Correlation Discovery• Extract– noun phrases, verb phrases, and named entities in each document• Count– Co-occurrences among them

• Be used to illustrate “why”

Topic Evolution as Flow

Critical Event as Glyph• Emerge, dissolve, split, and merge

Keyword Correlation as Thread Alternatives

Keyword Correlation as Thread Intertwine to indicate co-occurrences

Visualization Design - Consistency
Critical event
Topic

KeywordLayout Algorithm• Three-level directed acyclic graph (DAG)

Adapt to Big Data - Challenges• Generate evolving multi-branch trees as well as to model theirevolution patterns over time
– Fitness– Smoothness• Handle huge document collection efficientlyOur Solution• Bayesian online filtering framework– Fitness: adopt the fast Bayesian rose tree method (our work in KDD’12)– Smoothness: build a constraint tree

• Efficiency: build a constraint tree and treat it as a tree index

Example

66,528 news articles

Dave CulterOur Focus: Data
Interaction

Topic Mining
Classification

Data

Text Mining

Visualization

Interpretation

Streaming
Heterogeneous

HomogeneousRoseRiver: How Hierarchical Topics Evolve inLarge Text Corpora• Dilemma of topic numbers when handling large collections

Too general

Too clutter(X-axis represents time. Each colored stripe represents a topic.)Our Solution: Evolving Topic Trees• Interpretable topic results– Real-world text corpora are naturally organized as trees• Intuitive navigation– From a global overview to local details

Visualization Challenges• Effective representation– Topic nodes– Topic relationships• Progressive navigation– Obtain a full picture– Quickly focus on the information of interest– Preserve the mental-map

Case Study: Prism Scandal• 69,867 news articles and 568,225 tweets• June 3, 2013 to February 9, 2014
• Grouped into 36 topic trees by week
– Tree depth: 3 – 9– Total nodes: 19 – 165– Nodes at first level: 3 – 20

Visual Encoding: topic in the tree
: topic size

:similar document pairs: relevanceSystem Overview

Choose/refine topics
based on userinterests orinteractions
Preserve the mental
mapOur Focus: Data
Interaction

Topic Mining
Classification

Data

Text Mining

Visualization

Interpretation

Streaming
Heterogeneous

HomogeneousMotivation• Relevant topics is heavily discussed in multiple sources• Checking one source may take a part for the whole• Gather separate pieces of
information about these
topics scattered in different
sources and reconstruct thefull pictureImage Panorama

TopicPanorama
Microblog

Blog

The Topic Graph of a Microblog

The Topic Graph of a BlogNewsThe Topic Graph of News

TopicPanorama Overview

Graph Matching: Why• A straightforward method– merged data + a topic graph construction method• Deficiency– may fail to model the diversity across different corpora

Graph Matching: Our Solution• Objective– Finding correspondence among multiple topic graphs• Base– Graph-edit-distance-based matching method• Minimize the cost of all pairwise graph matchings– s.t. all node mapping relationships are transitiveGraph Matching: MetagraphIncremental Algorithm• Allow users to interactively modify the graph matching result• Develop an incremental graph matching algorithm based on theincremental Hungarian algorithm

Visualization• Graph matching as densitybased graph visualization• Topic hierarchy as stackedtree• Coupling graph visualization
with stacked tree

Layout Algorithm

Basic principle: the common parts-> the layout areaof each area; the distinctive parts -> the corpus arealayout of theclusternodesof the andselectedtree levelandcomputelayoutof thecommondistinctive
parts
and
Layoutofrepresentativenodes
a new Voronoi
computetessellation
the corresponding
based on theVoronoi
new layout
tessellation
resultVisual User Behavior Analytics: Key FocusesMultifaceted
Behavior Analytics

Design advanced techniques to explore
multifaceted behaviors and their relationships

Implicit Behavior
Analytics

Design model-driven, interactive visual analytics
methods to understand implicit behaviors

Explicit Behavior
Analytics

Design effective, fundamental techniques andmethodologies to visualize explicit behaviors

“Big Picture”: Our Key FocusesMultifaceted
Behavior Analytics

Implicit Behavior
Analytics

Explicit Behavior
Analytics

Step 3

Step 2

Step 1

“Big Picture”: Our Key FocusesMultifaceted
Behavior Analytics

Implicit Behavior
Analytics

Explicit Behavior
Analytics

Step 3

Step 2

Step 1

“Big Picture”: Our Key FocusesMultifaceted
Behavior Analytics

Implicit Behavior
Analytics

StoryFlow
Explicit
Behavior
Analytics
LoyalTracker

Step 3

Step 2

Step 1

“Big Picture”: Our Key FocusesMultifaceted
Behavior Analytics

Implicit Behavior
Analytics

StoryFlow
LoyalTracker

Explicit Behavior
Analytics

Step 3

Step 2

Step 1

Storytelling

A Good StorytellerWho, When, and WhereStories Are Complicated• The dynamic relationships of characters• The complex structure of scenes

Storyline VisualizationtimeStoryline Visualization

One character
T-Rex

Dinosaurs
Human

time

Storyline Visualization

Five characters in the same scene
Dinosaurs
Human

time

Storyline Visualization

Dinosaurs
Human

time

Storyline Visualization

timeThe Lord of the Rings Trilogy

The Lord of the Rings Trilogy

The Lord of the Rings Trilogy

AttackAragorn

The Lord of the Rings Trilogy

Mount DoomThe Lord of the Rings Trilogy

The Lord of the Rings Trilogy

Boromir

The Lord of the Rings Trilogy

StoryFlow (InfoVis 2013)• Real-time Hierarchy

 Level-of-detail renderingFirst debate
Voting

VP debate

Second debate

Third debate

Objective

Crossings
WigglesWhite Space

155Strategy
Importancedecrease

Crossings
Wiggle number
Wiggles
Wigglesdistance
White space156

Quantitative Analysis• Intel i7-2600 CPU (3.4GHz), 8GB memory• Comparison with TM’s method
Data

Time(s)

Crossings

Wiggles#Entity#Frame

Ours

TM

Ours

TM

Ours

TM

Star Wars

14

50

0.16

129.79

48

93

82

133

Inception

8

71

0.16

149.67

23

99

88

162

Matrix

14

42

0.16

172.47

14

43

54

94

MID

79

523

0.60

>10^5

1267

1871

831

874

Quantitative Analysis• Intel i7-2600 CPU (3.4GHz), 8GB memory• Comparison with TM’s method
Data

Time(s)

Crossings

Wiggles#Entity#Frame

Ours

TM

Ours

TM

Ours

TM

Star Wars

14

50

0.16

129.79

48

93

82

133

Inception

8

71

0.16

149.67

23

99

88

162

Matrix

14

42

0.16

172.47

14

43

54

94

MID

79

523

0.60

>10^5

1267

1871

831

874

Jurassic Park

159

Inception

160

Inception

161

King Lear

162

Overall Patterns (1/2)• Three user groups focus mainly on Election–Grassroots focus on Economy and switched frequently– Political figures are more focused– Media occasionally switched
DefenseElection

EconomyWelfareHorse Race

Grassroots

Media

Political Figures

TimelineOverall Patterns (2/2)• Five significant peaks on Election

GrassrootsMedia– Third debate on foreign affairs

Political Figures

DefenseElectionFirst debate

VP debate

Second debate

Third debate

VotingEconomyWelfareHorse Race

TimelineSignificant transitions (1/3)• Transition from Election to Defense
DefenseElectionFirst debate

VP debate

Second debate

Third debate

VotingEconomyWelfareHorse Race

Timeline

GrassrootsMedia

Forward

Political Figures

benghazi‘doesn’t obama owe us answers (on #benghazi)?
americaforwardSignificant transitions (2/3)• Transition from Election to Economy
DefenseElectionFirst debate

VP debate

Second debate

Third debate

VotingEconomyWelfareHorse Race

Timeline

Grassroots
Media
Political Figures
Sensatatlot
teaparty
gop

think Romney is tough on china? ask the workers of#sensata about that as they train their Chinese
replacements

Significant transitions (3/3)• Transition from Election to Welfare
DefenseElectionFirst debate

VP debate

Second debate

Third debate

VotingEconomyWelfareHorse Race

Timeline

Grassroots
Media
Political Figures
sandy

Issue-attention cycle
fema

“Big Picture”: Our Key FocusesMultifaceted
Behavior Analytics

OpinionFlow
SocialFlow

Implicit Behavior
Analytics

Explicit Behavior
Analytics

Step 3

Step 2

Step 1

Implicit Behavior Analytics: OpinionFlow• OpinionFlow: visual analysis and exploration of implicit opinion
propagation on social media
Visual Analysis of Opinion Diffusion

IEEE VAST 2014

Opinion on Social Media• Forrester 2009: 78%
consumers trust peer
recommendations on social
media• Facebook impact on purchase47%
24%

Analyzing opinion diffusion is importantMajor Challenges

Key Focus of OpinionFlow
Visually trace and analyze opinion diffusion on social media in largescale events

System Overview• Data analysis + diffusion modeling + interactive visualization

Questions• Question A: overview of topics and opinions– What are the major opinions in a large-scale event on social media?
– How are they changing over time?

• Question B: diffusion of opinions– What different roles do individual users play in opinion diffusion?

– How does the attention transition behavior of users relate to opinion
diffusion?

Outline• Definition• Objective• History• InfoVis 1.0• InfoVis 2.0• InfoVis 3.0• Opportunities and challenges

Massive Data - Golden Mountain

Big Data’s Hot CousinObama’s big data plans• The Department of Energy is getting in on the big data frenzy, too,
investing $25 million to develop Scalable Data Management,
Analysis and Visualization Institute, which aims to develop
techniques for visualizing the incredible amounts of data generated
by the department’s team of supercomputers.

Obama’s big data plans• A $2 million award for a research training group in big data that will
support training for undergraduate and graduate students andpostdoctoral fellows using novel statistical, graphical, andvisualization techniques to study complex data.

Challenges• Big data availability• Data quality is not high• Collaboration with domain• Multiple skills• Different evaluation systems• Scalability is a big problem

Acknowledgements• HKUST– Conglei Shi, Panpan Xu, Prof. Huamin Qu• Georgia Tech
– Mengdie Hu, Prof. John Stasko• Tsinghua University– Jianfei Chen, Prof. Jun Zhu• MSRA colleagues and interns– Weiwei Cui, Yingcai Wu, Yangqiu Song, Furu Wei, Xin Tong, Xiting Wang,
Mengchen Liu, Kai Yan,…

Thanks a lot for your attention!

• Homepage: http://research.microsoft.com/en-us/um/people/shliu/• Email: shixia.liu@microsoft.com