The Demand for Many Cores:
Tera-scale Usage Models

Dr. Yimin Zhang
Research ManagerIntel China Research Center

Agenda• Introduction to Tera-scale• Tera-scale Usage Models• Enabling future applications• Deeper look – Visual Media Research inChina© 2008 Intel Corporation. 無断での引用、転載を禁じます。
2Multi-core is now mainstream

Over 6 MU
Quad-core
In 2007

100%

Multi-core Top to Bottom

SINGLE-CORE*
75%
MarketMarketCrossover
Crossovertomulticore
to multi-corein
in2006
2006

50%

25%

MULTI-CORE
0%

Q3’05

Q4’05

Q1’06

Q2’06

Q3’06

Q4’06

Question: How do we continue adding cores, and why?

© 2008 Intel Corporation. 無断での引用、転載を禁じます。
3Transition from more Clock to more Cores

Xeon™ Processor
Itanium® Processors

4.01

PerformanceFrequency (GHz)

Pentium®-4 Processors
Pentium®-II/III Processors
Pentium® Processors

3.01Intel386/486™ Processors
Pentium®-M Processor
2.01

1.01

Multi-core performance:
P ~ Area
Gap driving us
to Multi-coreSingle Stream
PerformanceP ~ √Area

Time

20 09

20 06

20 04

20 01

19 98

19 95

19 93

19 90

19 87

19 84

19 82

0.01

Transistor Count (area)• Frequency limited by leakage and power (P = CV2f)• Transistor counts continue to increase with Moore’s Law• Multi-core adds performance for less power© 2008 Intel Corporation. 無断での引用、転載を禁じます。
4

Intel Tera-scale Research
Scaling multi-core to bring tera-scale performance the mainstream

Rich Visual Computing
Turning Data into
Understanding

Performance

TIPS

Emerging

GIPS

3D &Video
MIPS

KIPS

Printer
PrinterX42
X42

Tera-scaleMultiMedia
Text

Kilobytes

Sensing& Perception

10s-100s of coresMulti-coreSingle-core

HealthMegabytes

Gigabytes

Dataset Size

Terabytes© 2008 Intel Corporation. 無断での引用、転載を禁じます。
5

Agenda• Introduction to Tera-scale• Tera-scale Usage Models• Enabling future applications• Deeper look – Visual Media Research inChina© 2008 Intel Corporation. 無断での引用、転載を禁じます。
6Emerging Performance-Driven Appsy Based on modeling or simulating the real world
y Make technology more immersive and human-likey Algorithms are highly parallel in naturey Real-time results essential
for user-interaction

MODEL-BASED APPS
R = Recognition
M = Mining
S = Synthesis

Security
Biometrics

CancerR Detection

Body
Tracking

FacialAnimation

RayInteractive
Tracing
VirtualMedia
FinancialWorlds
Indexing
Predictions
Web searchS

DataWarehousing

MIn 2004, these observations led us to explore tera-scale© 2008 Intel Corporation. 無断での引用、転載を禁じます。
7

A Top-down Approach
MODEL-BASED
Security
Biometrics

CancerR Detection

MediaIndexing

Body
Tracking

FacialAnimation

VirtualWorlds

RayTracing
Financial
Predictions

Web search

Optimization
Methods

Data
Warehousing

M

VectorTraining
Monte
CarloS

Geometric
Structures
Collision
Detection

MATHEMATICAL Classifiers
MODELS & METHODSNUMERICAL METHODS& DATA STRUCTURES

Tera-scale processors will be
designed to addressapplication and user needs© 2008 Intel Corporation. 無断での引用、転載を禁じます。
8

Example: Visual Computing (part 1)
Computational Perception: Expanding human capabilities
through improved machine understanding of our world

Location-basedServices

Medical Imaging

New Interfaces

Smart cars

Video Mining© 2008 Intel Corporation. 無断での引用、転載を禁じます。
9

Personal Robotics

Face tracking

Example: Visual Computing (part 2)
Physics-based realism: graphics that can look, act and react
like real-world objects.

Ray Tracing

Modeling thephysics of light.
Used in movies
today for more
photorealisticgraphics.

Virtual Materials

Sophisticatedcollisions

ExpressiveFaces

Fluids physics© 2008 Intel Corporation. 無断での引用、転載を禁じます。
10

Breakable objects

More Compute Æ Better Experiencey Perceptual accuracy & graphical realism scale with computey Algorithms ideal for array of general purpose CPUsy Shared algorithms… i.e. ray-tracing for lighting aids physics
collision detection and path selection AI

Today: Second Life Fluid
1X

Now

Tomorrow: Particle Fluid

Future: Natural Looking Fluid

Effects Physics

10+ years

10X Compute© 2008 Intel Corporation. 無断での引用、転載を禁じます。
111000X ComputeProgress in Real-Time Ray Tracing• Approaching real-time functionality for ray-traced graphics• Possible due to increased parallel computing and software innovations

IDF 2004

Early 2007

Fall 2007

50 Intel® Xeon™
Processors

Yorkfield(45nm Quadcore)Dual-X5365(total: 8 cores)

4 Frames per Second
640x480~90 frames per Second
768x768~90 frames per Second

1280x720

See demo in the technology showcase (BU026)© 2008 Intel Corporation. 無断での引用、転載を禁じます。
12Research shows Applications Scale Well

Parallel Speedup

644832160016324864

Cores

Production Fluid
Production Face
Production Cloth
Game Fluid
Game Rigid Body
Game ClothMarching Cubes
Sports Video Analysis
Video Cast Indexing
Home Video Editing
Text Indexing
Ray Tracing
Foreground Estimation
Human Body Tracker
Portifolio Management
Geometric MeanGraphics Rendering – Physical Simulation -- Vision – Data Mining -- Analytics© 2008 Intel Corporation. 無断での引用、転載を禁じます。
13

Virtual Worlds: “Connected” Visual Computing
Users Create

Users Collaborate & PlayWorld of Warcraft Avatar

Eiffel Tower inGoogle Earth

Virtual Teamroom
Scenario Play

Users Explore and Learn

Users Enhance the Actual World

Machinima Interactive Movies

Qwaq Treefort
Virtual Room

Visualizing RealWorld Information -Dust storm in Morocco

West Nile Virus
Visualization

CVC apps will transform the Internet from 2D to 3D…but require LOTS of compute horsepower© 2008 Intel Corporation. 無断での引用、転載を禁じます。
14

Tera-scale will enable CVC apps
Communication

Gesture track

Spatial Audio

3D interfaces

ContentWorldSimulation

Audio/VisualEffects
Input

Audio

Physics

Physics

Content

High quality
content fromuser-friendly tools

Gesturerecognition

Ray Tracing

Graphics
Display

WorldSimRay
Tracing

Virus Security
Check

Operating System

All elements processed on a common platform in parallel.
Tera-scale chips could provide this.

© 2008 Intel Corporation. 無断での引用、転載を禁じます。
15

Processing Will Span Client/Server
Tera-scale Serveror Compute Cloud
Get Input
World Simulationz Collision Physics
z NPC
z Script Execution
z Simulation
Send Requested Update

User Takes Action on
the “World”
World”

Client (moving to TS)
User Inputs

Collects Changes
From All Users

Audio/Visual Effectsz Animationz Spatial Audioz Smoke, Crowds, Fluids

Resolves All ObjectBehaviors andInteractions

Rendering

Generates A New
PerPer-User Model

Display Updates

Observations:
MoreAlways
Server Connected
Compute• Significant client/server compute every cycle• Many aspects best computed on client• Extensive use of MIPS, FLOPS, threads• Partitioning depends on client capability, connectivity© 2008 Intel Corporation. 無断での引用、転載を禁じます。
16MoreClientComputeMobile CVC includes Augmented Reality
Visual computing + mobility + sensors will mix the virtual and real
Tera-scaleComputerGet Input
World Simulationz Collision Physics

MobileBroadband

z NPC
z Script Execution
z Simulation

MobileInternetDevice
withCamera,
Sensors

Send Requested Update• Mobile broadband
will connect future,
more capable MIDs totera-scale computeresources• This will allow us toaugment our view ofreality© 2008 Intel Corporation. 無断での引用、転載を禁じます。
17

Printer
Printer X42
X42

Agenda• Introduction to Tera-scale• Tera-scale Usage Models• Enabling future applications• Deeper look – Visual Media Research inChina© 2008 Intel Corporation. 無断での引用、転載を禁じます。
18

HW/SW interdependence• Scaling multi-core will be challenging• Parallel programming is a major shift for mainstream software

Hardware must be built to fit needs of SW
Software is developed to exploit existing HWProcessor
Path-Finding

Tech Readiness

Debug

Development

SoftwarePlanning and Incubation
OS and Dev Tools

Beta 1

Beta 2

Application

RC
Beta

Both take ~5 yrs how do we avoid a ~10yr transition?

© 2008 Intel Corporation. 無断での引用、転載を禁じます。
19

Joint Hardware & Software R&D
Prototype
Threaded
Application

NewApplication
Ideas

PrototypeMulti-core
Architecture

NewArchitecture
Ideas

SW HW+SW HW
TESTRefineSoftwareDesigns

RefineHardwareDesigns
RESEARCH
TOOLS

Cycle-accurate
Simulators

FPGAEmulators

SiliconPrototypes

HW/SW co-development & emulation critical© 2008 Intel Corporation. 無断での引用、転載を禁じます。
20

Example: Memory Bandwidth
App Research Findings
Memory Bandwidth

256 KB SRAM per core
4X C4 bump density
3200 thru-silicon vias

10000.0
Ray Tracing
Scenarios
1000.0GB/s

3D Memory Stacking R&D

8080-tile processor with Cu bumps“Polaris”

Physics
Simulation

100.0

Denser thanC4 pitchMemory“Freya”
C4 pitch

10.0

Thru-Silicon
ViaPackage
Package

1.01234

Fidelity

5(Best)Memory access to match the compute power

Assumes 16 MB L2$• Visual Computing application research shows atremendous increase in memory bandwidth requirements• In parallel, we are developing new memory options© 2008 Intel Corporation. 無断での引用、転載を禁じます。
21

Many efforts to enable many cores
Joint HW/SW R&D
program to enable
Intel products
3-7+ in futureIntel Tera-scale
Research

Academic Research

UPCRCsEnablingParallel
Software

SoftwareProducts
Wide array of leading
multi-core SW
development tools &info available today

Free Software
Tools

Multi-core
Educationzzzzz

Academic researchseeking disruptive
innovations 7-10+
years out

TBB Open Sourcedz
STM-Enabled Compiler on
Whatif.intel.com
zParallel Benchmarks at
Princeton’s PARSEC site

Multi-core Education Program- 400+ Universities
25,000+ students
2008 Goal: Double thisIntel® Academic Community
Threading for Multi-core SW
community
Multi-core books

Must work closely with customers, andindustry and academic partners© 2008 Intel Corporation. 無断での引用、転載を禁じます。
22

Tera-scale Programming Research
Transactional Memory
Lock-free Parallel
Memory Management

Main threadACSpeculativethread

Spawn

B

Ct: C/C++ for Throughput
ComputingMaking it easier to program of awide array of high-throughput
applications.

Speculative Multi-threading

Threading serial code segments
at the hardware level

11000000

2200
3300000000
4455
6600
77

11 22 44 55
3366
77

Commitspeculative
results

1/3 of Tera-scale research is software enabling© 2008 Intel Corporation. 無断での引用、転載を禁じます。
23

Princeton Application Repository for Shared memory Computers
blackscholes*

Standard Financial analytics benchmark.

bodytrack*

Build body model from video input.

facesim*physical modeling, face animation.

fluidanimate*smoothed particle hydrodynamics

freqmine*frequent item set mining

Swaptions*financial Monte Carlo code

ferretServer for image similarity search

dedupEnterprise Storage

streamcluster

streaming clustering of multidimensional data

vips

Image processing system

x264

H.264 (MPEG-4) video encoding

cannealVLSI placement program using simulated annealing

Open Source Parallel
App Benchmarks
From Kai Li’s and J. P.
Singh’s groups atPrinceton* Benchmarks
provided by Intel

http://parsec.cs.princeton.edu/© 2008 Intel Corporation. 無断での引用、転載を禁じます。
24

March 2008: New “Universal Parallel
Computing Research Centers”
y $20 million committed by Intel and Microsoft
y Two University centers: Berkeley and Illinois

Prof. Wen-Mei Hwu
Professor David Patterson
UCB UPCRC Director

Prof. Marc SnirUIUC UPCRC Co-Directors

Catalyze breakthrough research to help makeparallel computing mainstream in 7-10+ years© 2008 Intel Corporation. 無断での引用、転載を禁じます。
25

Agenda• Introduction to Tera-scale• Tera-scale Usage Models• Enabling future applications• Deeper look – Visual Media Research inChina© 2008 Intel Corporation. 無断での引用、転載を禁じます。
26

Video MiningMarket Trends#1 Explosion of digital content• Over 400M of digital photos shot daily(IDC)• 11% of U.S PCs have >10K photos(Tabblo)#2 Consumers want smart filters
based on their preferences• Locating photos is cumbersome
today• Text search uses file & folder names• Time & hassle to rename & tag image
files#3 Image recognition is slow

• > 20 seconds / photo / iteration(Pentium M)© 2008 Intel Corporation. 無断での引用、転載を禁じます。
27

Video Mining

Online Video Services

Fee-BasedContentGenerallyFree
Content withAds

Professional and User-Generated
Multimedia content growing rapidly© 2008 Intel Corporation. 無断での引用、転載を禁じます。
28

Video Mining

Key technology for media mining© 2008 Intel Corporation. 無断での引用、転載を禁じます。
29

Video Mining

Person recognition: Face recognition in photo• Highly accurate multi-view face detection + FIGHT feature + LDA• 90% accuracy in 1000 personal photos from 24 people

Best accuracy reported© 2008 Intel Corporation. 無断での引用、転載を禁じます。
30

Video Mining

Person recognition: Cast Indexing/facerecognition in video
Faceclustering

Face query

Retrieve relevant videos

z Multi-modality (face/speaker), Hybrid (supervised + unsupervised)z Promising results: 90% in news video, 60-80% in movie and home video© 2008 Intel Corporation. 無断での引用、転載を禁じます。
31

Video Mining

Person recognition: Humandetection/tracking
y Human detection/tracking
High detection accuracy: precision 92.38%, recall 88.82%
Tracking: 75%, some ability to handle merge and occlusion

1.81 fps
8 objects, obj-size: 15x55, 100 particles/obj© 2008 Intel Corporation. 無断での引用、転載を禁じます。
32

Video Mining

Generic Concept Detection in TRECVIDTRECVIDy Yearly international workshop sponsored by NIST for evaluation ofresearch in content-based retrieval of digital video.
y High level feature extraction/concept detection (joint with Tsinghua)© 2008 Intel Corporation. 無断での引用、転載を禁じます。
33

Video Mining

Concept Detection Results

z Fully automaticz State-of-the art accuracy© 2008 Intel Corporation. 無断での引用、転載を禁じます。
34

Concept

Precision

Animal

0.6054

Building

0.5002

Car

0.626

Crowd

0.7068Dog

0.2778Food

0.6656

Mountain0.5498Outdoor

0.9464TV-Screen

0.6066

Sky

0.7934Sports

0.7916Vegetation

0.4424

Walk/Running0.4522

Waterscape0.6075

Person0.9745

Video Mining

Future media applications
Organized byperson/face
Photo Album

Recognition

Character

Face
PoseGesture
Expression

TV & Movie

Home video

Classification

Summarization

Composition

Retrieval

Scene

Object

Event

Audio

Vegetation
WaterscapeView typeOut-door

Logo
VehicleAnimalBuilding

Walking
RunningStanding
Sitting

SpeechLaughSilentMusic

Script-based
authoringMulti-core/
Many-coreplatform

Photo/video
Photo/video
Organizer
Organizer

Search byperson/place/event/example image

Coarse to finecustomization

VideoVideo
Editing
Editing
VideoVideo
Editing
Editing© 2008 Intel Corporation. 無断での引用、転載を禁じます。
35

MediaMedia
SearchSearchVideo Mining

Sample usage model: Automatic
personalized music video generation© 2008 Intel Corporation. 無断での引用、転載を禁じます。
36

Video Mining

Sample usage model: Script Based Authoring
Structured categories

Selected storyboard

Input script

Who: friends in school
Where: on stage
What: dancing together
View: wide shot
Who: myself
Where: playground
What: playing ball
View: medium shot
Who: daddy and me
Where: park
What: swimming
View: closeup shot© 2008 Intel Corporation. 無断での引用、転載を禁じます。
37

Tera-Scale Computing ResearchTera-Scale Applications Research© 2008 Intel Corporation. 無断での引用、転載を禁じます。
38

Video Mining

Profiling of Video Mining System
S.Others

S.Others

P.Others

IIMG

BCH

OF-LK

S.Others

P.Others

P.Others

SIFT

OF-BM
SIFT

VB-FD
HOUGH

GCUT

ASM-FARANSACSports Video Analysis

Video Cast Indexing

Salient object detection

Parallelize codes accounting for more than
99% of total execution time© 2008 Intel Corporation. 無断での引用、転載を禁じます。
39

Video Mining

Parallelization:
Coarse-Grain vs. Fine-Grain
Granularity

CoarseFine

Parallelization

Between frames

Tiling within frame

Memory BW
Requirements

HighLow

Programmability

Easy

Difficultz Future system may need to support bothz Parallelization is not as easy as it looks (even for coarse-grain)© 2008 Intel Corporation. 無断での引用、転載を禁じます。
40

Video Mining

Parallel Scaling Performance--- Fine-grain parallelization

70
61.7
60

61.1

57.93
53.30

50
Parallel Speedup

46.72

46.57
45.09
38.4

40

28.40

30
20.8

18.67

20
10064
32
16
842
164
32
16
842
164
32
16
8
42
164
32
16842
164
32
1684
2
164
32
16
8
4
2
164
32
16
8
4
2
164
32
16
8
4
2
164
32
16
8
4
2
164
32
16
8
4
2
164
32
16
8
4
2
1BCH

OF-LK

SIFTsports

HOUGH

RANSAC

VB-FD

IIMG

ASM-FA

SIFT-castGCUTOF-BMy Most algorithms scale very well up to 64 cores in simulation- Many useful feedbacks on multi-core architecturey The full applications achieve 47x, 37x, and 53x speedup on 64 cores© 2008 Intel Corporation. 無断での引用、転載を禁じます。
41

Summaryy Future CPU performance increases will be primarily
achieved through multi-core parallelism

y Intel Tera-scale research aims to enable a wide range ofcompelling, compute intensive applications including
visual computing.

y Intel is driving research as well as industry and academic
collaboration to solve HW and SW challenges.

y The Intel China Research Center is developing advanced,
highly parallel computer vision algorithms which could
enable compelling multi-media search, editing and facialrecognition applications.

© 2008 Intel Corporation. 無断での引用、転載を禁じます。
42