# Presentation: 373033

## The Evolution of Computing Architectures

**Doug Carmean**
**Sr. Principal Architect**
**January 15, 2006**

**Notes:**
Acknowledge the Willamette team: Architects, designers, and validators that conceived, built and
Validated the processor.

## The Evolution of Computing Architectures

- Back to the Future

**Notes:**
Acknowledge the Willamette team: Architects, designers, and validators that conceived, built and
Validated the processor.

## The Evolution of Computing Architectures

- Back to the Future

**Notes:**
Acknowledge the Willamette team: Architects, designers, and validators that conceived, built and
Validated the processor.

## The Evolution of Computing Architectures

- Back to the Future

- Software Really Sucks

**Notes:**
Acknowledge the Willamette team: Architects, designers, and validators that conceived, built and
Validated the processor.

## The Evolution of Computing Architectures

- Back to the Future

- Software Really Sucks...and it is Going to Get Worse

**Notes:**
Acknowledge the Willamette team: Architects, designers, and validators that conceived, built and
Validated the processor.

## The Evolution of Computing Architectures

**Doug Carmean**
**Sr. Principal Architect**
**January 15, 2006**

**Notes:**
Acknowledge the Willamette team: Architects, designers, and validators that conceived, built and
Validated the processor.

## Caveats to this talk

**Wild speculation**
**Personal opinions**
**This is not a product description, nor a commitment to build any product**

## Agenda

**The Motivation for Change**
**Implications**
**Future Architectures**
**Summary**

## Where we’ve been with CPUs

## Where that takes us

**Where that takes us**

***Practical***
***Limit***
***Of ***
***Power***

***Too Hot,***
***Too Big,***
***Too Noisy,***
***$$***

## What Were We Thinking?

**Need to improve traditional performance**
**Start working on threaded performance**
**The P6 family is out of gas**
**Performance ~ Frequency**
**Frequency Sells**
**We can pipeline **_***anything***_

***Improve Performance: Push IPC, Push Frequency, ***
***Invent the new uArch for a family of processors,***
***Start down the SMT/CMP path***

## Reality Sets in....

## Process Scaling

**Power = ****1****/****2**** C V****2**** F**
**On each silicon process step (every 2 yrs)**
**Capacitance decreases: 0.7x**
**Supply voltage decreases: 0.9x**
**Frequency increases: 1.4x**
**Area improves: 0.5x **
**Power: 0.7 * 0.9****2**** * 1.4 = 0.8x **
***for the same number of transistors***
**2x transistors => 0.8 * 2 = 1.6x power**

***Power is increasing at the rate of 1.6x every 2 years***

## Process Scaling

**Define a process capability**
**Theoretical peak performance scales by  # of transistors and frequency**
**On each process, 2x transistors, 1.4x frequency => 2.8x capability**
**Limitations to process scaling**
**Transistors don’t work below 0.7V**
**Wires and leakage power scale follow sane trends.**

***Capability increases at 2.8x; Power at 1.6x every 2 years***

## Process Scaling

**Process Scaling**

***Native characteristics ***
***of a process improvement***

## 3 phases of silicon scaling

**Phase 1: not power limited**
**1 process step => 2x transistors * 1.4x freq = 2.8x *****capacity/process***
***CPUs were completely area limited before 2005***
**Phase 2: start hitting the 100W pain threshold**
**Lower the voltage as we move forward a process step**
**1.6x power is offset by dropping voltage by 15% every process step**
**Frequency scales with voltage**
**0.85****3**** = 1/1.6**
**1 process step => 2x transistors * (1.4 *0.85) freq = *****2.4x capacity/process***
***Today is the transition from area limited to power limited***
**Phase 3: voltage reduced to Vmin; reduce rate of frequency scaling**
**1 process step => 2x transistors * (1.4 / 1.6) freq = 1.75x capacity/process**
***CPUs will be completely power limited after 2013***

***We architect differently when***
***power limited vs. area limited***

## Area Limited -> Power Limited Transition

*Power*

*Perf*

## Power Limited Era

**Power Limited Era**

***Area***
***Limited***

***Transition:***
***Area->Power Limited***

***Limit of Voltage***
***Scaling***

*Power*

*Perf*

## Architecting for Power Limited Silicon

**Out-of-order cores try to extract parallelism using (power consuming) hardware**
**Sometimes there is obvious parallelism that software can identify**
**When we are no longer area limited, we can dedicate some portion of the silicon that only works only when running obviously parallel code**
**This new portion of the silicon can be more power efficient because the hardware does not need to extract the parallelism **

## Optimizing Architecture for Power Efficiency

**General purpose processors, optimized for single threaded workloads, spend a significant amount of power extracting parallelism**
**Out-of-order**
**Register renaming**
**dynamic schedulers**
**reorder buffers, etc.**
**Latency tolerance structures**
**Trace cache**
**Instruction queues**
**Addressing caches (PTE, Segment descriptor)**
**Byproducts of pushing frequency**
**Long pipelines**
**Bypass intensive structures**
**Complexity**

## Optimizing Architecture for Power Efficiency

**Increase power efficiency with explicit parallelism:**
** ****VLIW & Vectors:**
**Good when there is tight synchronization**
**Bad when data being processed follows different paths**
**Threads**
**Bad when there is tight synchronization**
**Fine when data being processed follows different paths**
**Push each technique to a practical point**
**4 elements vectors often makes sense**
**Graphics (XYZW, RGBA) can be efficiently processed as vectors**
**Efficiency starts to degrade as vector length increases**
**Today: 4-wide SIMD; Tomorrow: 16-wide SIMD**

## Consider the comparison

***Traditional Processor***

***D$***

***I$***

- Out of Order
- 2 Threads

***TC$***

***ROB***

***MOB***

***Rename***

***Schedule***

***Inst Queue***

***Bypass***

## Consider the comparison

**Consider the comparison**

## Potential Architecture Choices

***140mm^2***
***90W***
***1.0x ST***
***64 GFLOPS ***

## What is this thing?

**What is this thing?**

*** ******Chip Multi Processors***
*** ******10 cores on a die***
*** ******SMP like scaling***
*** ******Threaded Processors***
*** ******4 threads per core***
*** ******SMT like scaling***
*** ******Vector machine***
*** ******16-wide vector***
*** ******Vector tradeoffs***
*** ******VLIW Instructions***
*** ******VLIW tradeoffs***

***Write good code for that!***

## It Gets Better....

## We’ll concede the following

**Caches with decent behavior**
**They are coherent**
**Relatively low latency (~10 clock L2)**
**Reasonable communication latency**
**<20 clock locks, semaphores**
**Primitives for thread synchronization**
**Good bandwidth**
**1TB/s on die aggregate bandwidth**
**>50GB/s off die bandwidth**

## But we want:

**Dynamic cache partitioning**
**Arbitrary data replication**
**Data swapping between partitions**
**Heuristic based configuration changes**
**Will likely come with non-uniform access latencies**

## Is that a good idea?

**Is Hardware managed dynamic partitioning the right answer?**
**Should we abdicate to software partitioning?**
**What about the middle ground with software hints?**

## We’ll Likely Want the Following:

**We’ll Likely Want the Following:**

*** ******Chip Multi Processors***
*** ******Threaded Processors***
*** ******Vector machine***
*** ******VLIW Instructions***
*** ******Single Thread Goodness***

## Architecture Trajectory

**Year**

*Threads*

**1**

**10**

**2003**

**2005**

**2007**

**2009**

**2011**

**2013**

***Multi-threading***

***Multi-core***

***Lots-o-Core Era***

***SMT***

***Small Number***
***Of Traditional***
***Cores***

***Arrays of***
***Throughput ***
***Cores***

**100**

## Or said another way,

**Year**

**1**

**10**

**2003**

**2005**

**2007**

**2009**

**2011**

**2013**

***Medieval Times***

***Renaissance Period***

***Industrial ***
***Age***

***SMT***

**100**

*Threads*

***Small Number***
***Of Traditional***
***Cores***

***Arrays of***
***Throughput ***
***Cores***

## Summary

**Transitioned from fundamentally area limited to fundamentally power limited**
**Architectures will change radically**
**Hardware will rely on software for extracting parallelism**
**Desperately need help from programming languages to exploit hardware potential**

## Slide 34