# Presentation: 657652

## Rapid Paths to  Petaflops Computing and Beyond

- Thomas Sterling
- California Institute of Technology
- NASA Jet Propulsion Laboratory
- March 10, 2000

- 4th Workshop on Distributed Supercomputers

## Predictions and Insights

- Punch cards are for ever
- Video monitors are a fad
- Apple is dead
- SGI is alive
- Everyone will want an IBM 360 on a chip
- Parallel programmers won’t tolerate message passing
- Linux will last maybe two years at most
- You can’t make money selling a magazine (or books) about free software
- You can’t make money selling CDs of software you can download off the net for free
- Java, you can’t be serious

- 4th WDSC     Thomas Sterling/Caltech

- March 10, 2000

## Prediction Paradigms

- Crystal  ball reflection
- seeing your own reflection
- Road Kill
- staring in to the headlights
- Yellow brick road
- don’t look behind the curtain
- Ballistic trajectory
- e.g. Moore’s Law
- Surfs Up
- catch the wave
- Chapter 11
- give up

- 4th WDSC     Thomas Sterling/Caltech

- March 10, 2000

## Algorithms Provide the Abstract Medium to Match the Computing Requirements of Applications to the Attributes of the System Architecture to Deliver Sustained Performance

- APPLICATIONS

- ALGORITHMS

- SYSTEM  ARCHITECTURE

- TECHNOLOGY

- PERFORMANCE

- Algorithmic Solutions Driven by and Respond to Assumptions about Underlying System Architecture Properties

- 4th WDSC     Thomas Sterling/Caltech

- March 10, 2000

## Points of Inflection Computing

**Heroic Era (1950)**
- technology: vacuum tubes, mercury delay lines, pulse transformers
- architecture: accumulator based
- model: von-Neumann, sequential instruction execution
- examples: Whirlwind, EDSAC
**Mainframe (1960)**
- technology: transistors, core memory, disk drives
- architecture: register bank based
- model: reentrant concurrent processes
- examples: IBM 7042, 7090, PDP-1
**Scientific Computer(1970)**
- technology: earliest SSI logic gate modules
- architecture: virtual memory
- model: parallel processing
- examples: CDC 6600, Goodyear STARAN

- 4th WDSC     Thomas Sterling/Caltech

- March 10, 2000

## Points of Inflection in the History of Computing

**Supercomputers (1980)**
- technology: ECL, semiconductor integration, RAM
- architecture: pipelined
- model: vector
- example: Cray-1
**Massively Parallel Processing (1990)**
- technology: VLSI, microprocessor,
- architecture: MIMD
- model: Communicating Sequential Processes, Message passing
- examples: TMC CM-5, Intel Paragon
**? (2000)**
- trans-teraflops epoch

- 4th WDSC     Thomas Sterling/Caltech

- March 10, 2000

## Challenges

- Bandwidth/throughput
- PBps
- Parallelism
- 100 million-way
- Latency
- time of flight, cycle time disparity
- thousand to a million-way
- Overhead
- Data set size/Memory capacity
- Pbyte down to a Terabyte (or less, severely domain specific)
- Resource and task management
- O/S, languages, compilers, runtime nucleus
- Power, complexity, cost, and reliability

- 4th WDSC     Thomas Sterling/Caltech

- March 10, 2000

## CMOS Technology Projections

- 2001
- logic: 0.15 um, 38 Mtr, 1.4 GHz
- memory: 1.7 Gbits, 1.18 access
- 2005
- logic: 0.10 um, 250 Mtr, 2.0 GHz
- memory: 17.2 Gbits, 1.45 access
- 2008
- logic: 0.07 um, 500 Mtr, 2.5 GHz
- memory: 68.7 Gbits, 1.63 access
- 2011
- logic: 0.05 um, 1300 Mtr, 3.0 GHz
- memory: 275 Gbits, 1.85 access

- 4th WDSC     Thomas Sterling/Caltech

- March 10, 2000

## Opportunity Drivers

- Technology
- optical communications
- optical storage
- merged logic and DRAM
- very high speed logic
- Architecture
- System On a Chip
- Multithreaded
- Processor in Memory (PIM)
- Dynamic adaptive resource management
- Open Source Software Framework
- for ISV target stability
- low entry level cost of new systems

- 4th WDSC     Thomas Sterling/Caltech

- March 10, 2000

## Technologies:  Data Vortex Network 

-  Optical Communication Technology
-  640 Gbps per fiber
-  Time/Wave Division Multiplexing
-  Pure Optical Data Switching
-  1 Nanosecond Packet Switching

- 4th WDSC     Thomas Sterling/Caltech

- March 10, 2000

## Technologies:  Holographic Storage

- 4th WDSC     Thomas Sterling/Caltech

- March 10, 2000

## Technologies:  Superconductor RSFQ the world’s fastest digital logic

- 4th WDSC     Thomas Sterling/Caltech

- March 10, 2000

## Architecture Opportunities for Petaflops in the 21st Century

- PC Clusters
- e.g. Beowulf
- Constellations of SMPs or DSMs
- Multithreaded Architecture
- e.g. Tera MTA
- Processor in Memory
- e.g. IRAM, DIVA, Blue Gene
- Dynamic Adaptive
- e.g. HTMT

- 4th WDSC     Thomas Sterling/Caltech

- March 10, 2000

## Percolation and Parcels

- Resources to perform available tasks, not wait for future work
- Pre-instantiated tasks register precedent constraints
- Distinguish between high utilization (high cost) and high availability (low cost)
- The right tool for the right job:
- numeric intensive calculations performed on high throughput engines
- data manipulations performed on high bandwidth storage
- Pro-active prestaging of task state at compute site
- Task results returned through independent write back and synchronization

- 4th WDSC     Thomas Sterling/Caltech

- March 10, 2000

## The Beowulf Deltalooking forward

- 6 years
- Clock rate: X 4
- flops (per chip): X 50 (2-4 proc/chip, 4-8 way ILP/proc)
- #processors: 32
- Networking: X 32  (32 - 64 Gbps)
- Memory:  X 10 (4 Gbytes)
- Disk:  X 100
- price-performance:  X  50
- system performance: 50 Tflops

- 4th WDSC     Thomas Sterling/Caltech

- March 10, 2000

## Slide 16

- 4th WDSC     Thomas Sterling/Caltech

- March 10, 2000