Pixels Everywhere
Media Tech and How it Changed the World
Pat Hanrahan
Department of Computer Science
Stanford University
Computing Research that Changed the World:
Reflections and Perspectives
Washington, DC
March 25, 2009

Transforming Media
From Analog to DigitalFrom Analog to Digital
Traditional media
 Desktop publishing and printing
 Digital audio, music and radio Digital photography
 Digital video, HDTV and movies

Timelines

PrintBezier curve (1962) for outlines
Bravo WYSIWYG text editor (1974)
First laser printer (1969)
ResearchProducts1960

1970

1980

1990

2000

2010

Adobe Postscript (1982)
Apple Laserprinter (1985)
Aldus Pagemaker (1985)

AudioReed-Solomon codes (1960)
FM Synthesis (1973)
Soundstream restorationof Caruso recordings (1975)
MP3 standard (1991)
ResearchProducts
1960

1970

1980

Optical CD (1982)

1990

2000

2010

Apple iPod (2001)

Internet Radio (1990s)

Photography
CCD imager (1969)

First digital paint system (1974)

CMOS imager (1993)
Discrete-cosine transform
ResearchProducts
1985

1990

1995

2000

2005

2010

Adobe Photoshop 1.0 (1990)
Nikon D1 digital SLR (1999)
flickr.com photosharing (2004)

TV and Movies
NHK demo of analog HDTV (1969)
Motion-compensated image compression (-1993)
NSF STC Computer graphics and vis (1991-2002))
First feature-length computer-generated movie (1995)
ResearchProducts
1985

1990

1995

ATSC standard ratified (1998)
TIVO time-shifted video (1999)

2000

2005

2010

Analog broadcast ends
(June 12, 2009)
3D digital projectors

Media & Technology Interdependent|Size| of media determines when it was transformed
Media poses science & technology problems
 StorageCD, DVD, Flash Networking
Gigabyte networks, internet caching ProcessingGPUs, signal and media processors

Invention of New MediaGames
Multimedia computers and media servers
Networked graphics (flash) and the WWW
Sharing music (iTunes), photos (flickr, phototourism),
videos (youtube)
Virtual worlds (Google Earth, Second Life, WoW)
Electronic books (Amazon Kindle)

Research Trends

Research Trends
Supercomputers on a chip
Reinventing photography and cameras
Building planetary-scale virtual worlds
New interaction devices

Modern GPU Architecture: 240 Cores
Host
Input Assembler

Setup & Rasterize

SP

SP

SP

TF

SP

SP

TF

SP

SP

SP

TF

SP

Pixel Thread Issue
SP

SP

SP

TF

SP

SP

TF

L1

L1

L1

L1

L1

L1

L1

L1

L1

L1TF

SP

SP

Geom Thread Issue

TF

SP

SP

SP

TF

SP

SP

SPTF

SP

SP

SPThread Scheduler

Vertex Thread Issue

TF

SPSPSPSPSP

L2L2L2L2L2L2L2L2Framebuffer

Framebuffer

Framebuffer

Framebuffer

Framebuffer

Framebuffer

Framebuffer

FramebufferChallenges
Architectures that support 1000s of cores
Programming environments for 1000s of cores
Applications beyond graphics and media
One of the most pressing current problems in computersciencefolding@home

Hypersonic vehicle

From Glass to Digital Lenses

From Glass to Digital Lenses

Focus in Software

conventional photograph, conventional photograph, light field, main lens at f / 4,main lens at f / 4main lens at f / 22
after all-focus algorithm[Agarwala 2004]

Light Field Microsope

sensoreyepieceintermediate
image plane

objectivespecimen

Virtual Worlds

Virtual LA (UCLA)

Rome Reborn (Virginia)

Meru (Stanford)
IBM Meeting in Second Life

Challenges
Planetary distributed object system (Web 10.0?)
 Real-time response Scalable (100B objects)
 Robust and secure

Scalable simulation
 Simulating physics across a world Simulating evolving eco/social system

Laboratory for studying social scienceOcarina by Smule on the iPhone

Image courtesy of Ge Wang, Stanford Music Department

Multi-touch Display

Thinsight, Hodges et al. Microsoft