# Presentation: 111856

## Planned Machines: BluePlanet

- SOS7 March 5, 2003
- Brent Gorda
- bgorda@lbl.gov
- Future Technologies Group
- Lawrence Berkeley National Lab
- http://www.nersc.gov/research/ftg

## NERSC

**National Energy Research Scientific Computing Center**

**~2000 Users in ~400 projects**

**Serves all disciplines of the DOE Office of Science**

** ****Focus on large-scale computing**

## The Divergence Problem

- The requirements of high performance computing for science and engineering, and the requirements of the commercial market are diverging.
- The commercial cluster of SMP approach is no longer sufficient to provide the highest level of performance
- Lack of memory bandwidth - high memory latency
- Lack of interconnect bandwidth - high interconnect latency
- The commodity building block was the microprocessor but is now the entire server (SMP)!
- U.S. computer industry is driven by commercial applications
- The decision for NERSC-3 E can be seen as an indication of the divergence problem: Power 4 had a low SSP number – scaling problem

## Cooperative Development – NERSC/ANL/IBM Workshop

- Workshops: Sept 2002 – defining the Blue Planet architecture
- 		  Nov. 2002 – IBM gathered input for Power 6
- White Paper: "Creating Science-Driven Computer Architecture: A New Path to Scientific Leadership,“
- http://www.nersc.gov/news/blueplanet.html

**Cooperative Development – NERSC/ANL/IBM Workshop**

## What is unique...

- ... in the design process:
- Awareness and sensitivity to vendor’s mainstream products
- Get back “inside the box”
- ... in the collaboration:
- Application teams drive the design process
- ...in the structure and function of the system:
- Single core cpu design – not a previously planned product
- Additional Federation stage – 4x scale up in links
- Virtual Vector Architecture – ViVA

## Applications to Drive the Design of New Architectures

- Combustion Simulation and Adaptive Methods
- Computational Astrophysics
- Nanoscience
- Climate Modeling
- Accelerator Modeling
- Lattice Quantum Chromodynamics
- Quantum Monte Carlo Calculations of Nuclei
- High Energy / Elementary Particle Physics
- Biochemical and Biosystems Simulations
- Advanced Simulations of Plasma Microturbulence
- Computational Environmental Molecular Science
- Application’s needs vary widely, however most utilize MPI

## Interaction based on Scientific Apps

|  | AMR | Coupled Climate | Astrophysics |  | Nanoscience |  |
| --- | --- | --- | --- | --- | --- | --- |
|  |  |  | MADCAP | Cactus | FLAPW | LSMS |
| Sensitive to global bisection | X | X | X |  | X |  |
| Sensitive to processor to memory latency | X | X |  |  | X |  |
| Sensitive to network latency | X | X | X | X | X |  |
| Sensitive to point to point communications | X | X |  |  |  | X |
| Sensitive to OS interference in frequent barriers |  |  |  | X | X |  |
| Benefits from deep CPU pipelining | X | X | X | X | X | X |
| Benefits from Large SMP nodes | X |  |  |  |  |  |

## Slide courtesy of Peter Ungaro, IBM

**Slide courtesy of Peter Ungaro, IBM**

## What prior experience guided this choice? 

- Power 4 memory bandwidth does not support 32 CPUs, and Power 4 Memory Latency is only 29% longer than Power 3.

## The majority of applications achieve low percentage of peak
Adding cpu’s w/o adding bandwidth makes no sense to HPC applications

The majority of applications are data starved: memory and interconnect

Many of the applications are regular and may be able to take advantage of ViVa

- Adding cpu’s w/o adding bandwidth makes no sense to HPC applications
- The majority of applications are data starved: memory and interconnect
- Many of the applications are regular and may be able to take advantage of ViVa

- What prior experience guided this choice?

## Other than your own machine, for your needs what are the best and worst machines?

- Difficult to push the envelope as a production facility
- Balance would appear to be the most important technical feature for a general purpose system
- Stability is the most important production feature
- Both involve more than just hardware
- The best system is one that lives up to (sometimes slightly reduced) expectations