# Presentation: 412022

## Peta-FLOPS Programming 

*Larry Snyder*
- University of Washington, Seattle

## Truth in Advertising 

- ZPL will be a witness to most of my remarks
- ZPL ...
- Version 1.0 released in ’97
- ZPL Programmer’s Guide published (MIT) in ’99
- Demonstrates portability, performance, performance model; *time*(ZPL) < *time*(hand-code in message passing)
- Global model: auto ||, comm, synch; Ironman Layer
*Advanced* ZPL (A-ZPL)
- Language upgrade: domains/grids/regions, sparse, hierarchical, remap, “local” leverage
- Sophisticated parallel and scalar optimizations
- A-ZPL will be released open source shortly

- http://www.cs.washington.edu/research/zpl/

## Take Home Message ...

- The programming model is critical element ...
- Desiderata  for a successful model
- Global view of computation – it’s not shared memory
- Performance model – programmers must know what they are doing to write good programs
- Model must always deliver on its “performance contract”
- Abstractions must limit unnecessary constraints – let the compiler find the parallelism, perform the communication, exploit the locality, identify the points of synchronization, flow-control, etc.

## Sample Code

**program Jacobi;**
**config var n : integer = 512; **
**         ****eps : float = 0.00001;**
**region     R = [1..n, 1..n]; **
**        ****BigR = [0..n+1,0..n+1];**
**direction  N = [-1, 0];  S = [ 1, 0];**
**           ****E = [ 0, 1];  W = [ 0,-1];**
**var     Temp : [R] float;**
**           ****A : [BigR] float;**
**         ****err : float;**
**procedure Jacobi();**
**	****   ****[R] begin**
**	****[BigR] A := 0.0;**
**[S of R] A := 1.0;**
**	****	****	****repeat**
**	****	****	****  ****Temp := (A@N + A@E + A@S + A@W)/4.0;**
**	****	****	****  ****err  := max<< abs(Temp - A);**
**	****	****	****  ****A    := Temp;**
**	****	****	****until**** err < eps;**
**	****	****  ****end;**
**end;**

- Compute new averages
- Find the largest error
- Update array
- ... until convergence

## Features of Example

- The Jacobi iteration is intuitive: the code (color) corresponds 1-1 to the abstract ideas
- ZPL’s performance model indicates (cues in syntax) where communication will be placed
- A@N implies point-to-point communication inserted
- max<< implies (combining) tree communication
- A := Temp implies no communication used
- 	(See Chap. 8, *ZPL Programmer’s Guide* for details)
*Programmers can write quality code and know it!*

## Burton’s Questions ...

- Will MPI predominance persist? For a while...
- 	PVM/MPI are parallel assembly languages, and like 40 years ago, programmers will adopt high-level languages when value is clear

## Burton’s Questions ...

- Will MPI predominance persist? For a while...
- Are New Parallelism Paradigms Probable? Not automatically, but they might be ...
- 	Since it’s not automatic, any new paradigm should have it’s portability demonstrated on all contemporary parallel platforms

## Burton’s Questions ...

- Will MPI predominance persist? For a while...
- Are New Parallelism Paradigms Portable? Not automatically, but they might be ...
- How can compilers promote pFLOPS pgming
- Discover and/or package parallelism: Absolutely
- Discover and/or package locality: Absolutely
- Raise/lower the level of abstraction: Absolutely
- 	A compilers success depends on the language ... ZPL’s compiler does these but only because the language helps critically

## Burton’s Questions ...

- Will MPI predominance persist? For a while...
- Are New Parallelism Paradigms Portable? Not automatically, but they might be ...
- How can compilers promote pFLOPS pgming
- Discover and/or package parallelism: Absolutely
- Discover and/or package locality: Absolutely
- Raise/lower the level of abstraction: Absolutely
- Could architecture help prevent programming pain? Yes, but ... it’s better for the arch to do basic operations (ops, 1-side comm, etc.) well