Systems of Thought
Pat Hanrahan
Stanford University
EuroVis 2009 Keynote

Systems of Thought:
Representations (Symbols)
andRules to Manipulate Them

Page 1

Systems of Thought
Language

Systems of Thought
Language
Logic (Boole’s The Laws of Thought)

Page 2Systems of Thought
Language
Logic (Boole’s The Laws of Thought)
Mathematics

Systems of Thought
Language
Logic (Boole’s The Laws of Thought)
Mathematics
Probabilistic reasoning and statistics

Page 3

Systems of Thought
Language
Logic
Mathematics
Probabilistic reasoning and statistics
Computation

Computational Thinking
Jeanette Wing et al.

Collection of ideas and techniques
 Algorithmic thinking
 Programming
 Systems building
 …

Page 4

Systems of Thought
Language
Logic
Mathematics
Probabilistic reasoning and statistics
Computation
Visualization and visual thinking

Systems of Thought
Language
Logic
Mathematics
Probabilistic reasoning and statistics
Computation
Visualization and visual thinking

Which one is best? (Hint: None)
What are the advantages of each?

Page 5

Let’s Solve a Problem:

Number Scrabble
Herb Simon

Number Scrabble
Goal: Pick three numbers that sum to 15

A:

B:Page 6

Number Scrabble
Goal: Pick three numbers that sum to 15

A:

B:Number Scrabble
Goal: Pick three numbers that sum to 15

A:

B:

Page 7

Number Scrabble
Goal: Pick three numbers that sum to 15

A:

B:

Number Scrabble
Goal: Pick three numbers that sum to 15

A:

B:

Page 8

Number Scrabble
Goal: Pick three numbers that sum to 15

A:

B:

Number Scrabble
Goal: Pick three numbers that sum to 15

A:

B:?

Page 9Tic-Tac-Toe

Tic-Tac-Toe

X

Page 10

Tic-Tac-Toe

XO

Tic-Tac-Toe

X

X

O

Page 11

Tic-Tac-Toe

X

O

X

O

X

O

Tic-Tac-Toe

X

XOPage 12

Tic-Tac-Toe

XOXXO?

Tic-Tac-Toe

XOXX
OO

Page 13

Problem Isomorphs

Problem Isomorph

438951276

Magic Square: All rows, columns, diagonals sum to 15

Page 14

Switching to a Visual Representation

438

95

12

76

Switching to a Visual Representation

43895

12

76

Page 15

Switching to a Visual Representation

438

95

12

7

6

Switching to a Visual Representation

43895

1276

Page 16

Switching to a Visual Representation

43895

1276?

Switching to a Visual Representation

438951276Page 17

The Representation Effect

On Being in the Right Space

Page 18

Azimuthal Equidistance

Figure 3.4, Flattening the Earth, Snyder

Mercator Projection

Figure 1.35, Flattening the Earth, Snyder

Page 19

On Being the Right Size
“The most obvious differences between different animals
are differences of size, but for some reason zoologists have
paid singularly little attention to them. In a large textbook ofzoology before me I find no indication that the eagle islarger than the sparrow, or the hippopotamus bigger thanthe hare, though some grudging admissions are made in thecase of the mouse and the whale. But yet it is easy to showthat the hare could not be as large as a hippopotamus, or awhale as small as a herring. For every type of animal there
is a most convenient size, and a large change in size
inevitably carries with it a change of form”
J. B. S. Haldane

On Being in the Right Space“The most obvious differences between different
visualizations are differences of space, but for some reason
visualization scientists have paid singularly little attention
to them. In a large textbook of visualization before me I findno indication that the log-log space is different than the loglinear space, or that the Mercator projection is different than
the azimuthal equidistant projection, though some grudging
admissions are made in the case of the parallel andperspective projections. But yet it is easy to show that
distances are difficult to estimate under perspective, or thatdata obeying a power law is easy to see in a log-log plot. For
every type of visualization there is a most convenient space,and a change into the right space inevitably makes
relationships clearer.”
P. Hanrahan

Page 20

The Value of Visualization

The Value of Visualization
The representation effect: Human performance varies
enormously (10-100:1) with different representations

The right representation
 Faster solution
 Fewer errors
 Better comprehension and memory …

But, the R. R. depends on the problem/question/task

Page 21

The Julia Set

Julia Set

Julia and Mandelbrot Sets

Mandelbrot Set

Julia Set

Page 22

Key Questions
1.

What is the problem you are trying to solve?

2.How do you think about the problem? What are thesemantic objects and their relationships?

3.

What visual representations are already used? How
does the visualization represent those objects andsupport inference?

How Visual Representations
Support Reasoning?

Page 23

“Why is a Picture(Sometimes) Worth
10,000 Words”
Larkin and Simon,Cognitive Science, 1987

Why?
Reduce memory load Working memory is limited
 Store information in the diagram

Reduce search time Pre-attentive (constant-time) search process
 Spatially-indexed patterns store the “facts”

Allow perceptual inference Map inference to pattern finding

Page 24

“Number Representations”
Norman and Zhang

Number Representations
Counting – Tallying

Adding – Roman numerals

XXIII + XII = XXXIIIII = XXXVMultiplication – Arabic number systems

Page 25

Long-Hand Multiplication

34x 7268
238
2448
From “Introduction to Information Visualization,”
Card, Schneiderman, Mackinlay

Zhang and Norman, The Representations of Numbers,Cognition, 57, 271-295, 1996

Page 26

Distributed Cognition
External (E) vs. Internal (I) process
Roman

Arabic1.

Separate power & base

IE2.Get base value

EI3.

Multiply base values

II

4.

Get power values

IE

5.

Add power values

IE

6.

Combine base & power

IE

7.

Add results

IE

Arabic more efficient than Roman

Contemplate
Why do computers use binary representations ofnumbers?

Page 27

Contemplate
Why do computers use binary representations ofnumbers?

Who don’t people use binary representations ofnumbers? For example, we use base 10.

What about the abacus?

Abstraction in Computer Science
Abstract data type
Choose the interface
Different possible representations of the data
Running times of key operations depend on
representation
Choose the appropriate implementation for theproblem

Page 28

Notation as a Tool of Thought
K. Iverson’s
1979 ACM Turing Award Address

Notation as a Tool for Thought“The thesis of the present paper is that the advantages ofexecutabiity and universality found in programming
languages can be effectively combined, in a single coherent
language, with the advantages offered by mathematicalnotation”
K. Iverson

Page 29

Arithmetic and Algebra in APL (k)>k=5> til k01234> 1 + 2 * til k13579> +/ 1+2*til k // 1 + 3 + 5 + 7 + 925> k*k25

Visual Proofs

Algebra1+3+5+7+9=52

Page 30

Program Transformations as Proofs+/ (1 + 2 * til k)// odd numbers+/ (1 + (til k) + (til k))// def of multiplication+/ (1 + (til k) + (reverse til k))// addition associative// 0 1 2 + 2 1 0 = 2 2 2+/ (1 + k # (k-1))+/ k#k// k = k-1+1k*k// k*k = +/ k#k// 3*4 = +/ 4 4 4

K. Iverson, Arithmetic, 1991

The Incredible Convenience ofMathematica Image Processing
Theodore Gray

Page 31

Key Questions
1.

What is the problem you are trying to solve?

2.

How do you think about the problem? What are thesemantic objects and their relationships?

3.

What visual representations are already used? How
does the visualization represent those objects andsupport reasoning about them?

4.

How can the manipulation of the representation beembodied in the interaction?

Multiple Representations

Page 32

MIT Intro to Comp. Prob. Solving
Lecture 3:
 There exists a constant, , such that  r2 is thearea of a circle Archimedes believed that 223/71 <  < 22/7
 The Bible later asserted that  = 3
 Which is right?

Solution: Buffon-Laplace

simulation

Python Solution
A = 0
for i in range(N): x = Uniform()
 y = Uniform()
 if x*x + y*y < 1:


A += 1p = A/N
return 4*p

Page 33

John Guttag

Probability of Hitting the Dartboard

Archimedes Traps Pi

Page 34

Archimedes Traps Pi
4 sided polygon

Archimedes Traps Pi
4 sided polygon

Page 35

Archimedes Traps Pi
96 sided polygon

Mathematics of Monte CarloCentral limit theorem for Bernoulli process

Page 36

Key Questions
1.

What is the problem you are trying to solve?

2.

How do you think about the problem? What are thesemantic objects and their relationships?

3.

What visual representations are already used? How
does the visualization represent those objects andsupport reasoning about them?

4.

How can the manipulation of the representation beembodied in the interaction?

5.

How can visualization be coupled with othersystems of thought?

Final Thoughts / Bigger Questions
If you can see it, it’s a visualization!
Simple is complex; abstracting the essence is hard

Language and visualization?
What if the problem is not well-defined?
What about aesthetics and style?

Page 37

Thank you

Questions?

Page 38