# Presentation: 475633

## Field-Programmable Gate Array Research Speeds HPC “up to 100X”

- Field-Programmable Gate Array Research Speeds HPC “up to 100X”

- Olaf O. Storaasli
**Future Technologies Group****Computer Science and Mathematics Division**

## Explore FPGAs for future ORNL HPC

**Contents**

**Background: Why FPGAs?**
**ORNL success: FPGA systems, tools and up to 100X speedup **
**Partners:                  Research Lab,          , SRC,**

**Virtex4 FPGA blades “accelerate mission-critical applications > 100X.”**

**Steve Scott, CTO HPCWire 24/3/2006**

- “*After exhaustive analysis, Cray concluded that, although multi-core commodity processors will deliver some improvement, exploiting parallelism through a variety of processor technologies using scalar, vector, multithreading and **hardware accelerators** **(e.g., **FPGAs** or ClearSpeed co-processors) creates the ****greatest opportunity**** for application acceleration*.”

- ORNL benefit: Exceed petaflops and reduce power

**Why HPC vendors offer FPGAs**

- ,

- ,

## What’s an FPGA?Your “custom chip”

**FPGA Logic slice**

- What’s an FPGA?Your “custom chip”

**Xilinx Virtex4 FPGA: 25K slices (miniCPUs)**
**Logic array: user-tailored to application**
**On-chip RAM, multipliers and PowerPCs**
**Gigabit transceivers/DSP blocks => FastIO/precision**
**100–1000 operations/clock cycle**

## Why FPGAs? 

**0**

**100**

**200**

**300**

**Computation**
**(GOPS)**

**Memory Bandwidth**

**(GB/sec)**

**IO Bandwidth**
**(Gbps)**

**Pentium**

**Virtex-4**

- FPGA
- Virtex4

- Pentium

- Why FPGAs?

**Performance****—optimal silicon use (maximize parallel ops/cycle)**
**Rapid**** ****growth****—cells, speed, I/O **
**Power****—1/10th CPUs**
**Flexible****—tailor to application**

**1000**

**800**

**600**

**400**

**200**

**0**

**0**

**100**

**200**

**300**

**400**

**500**

**600**

**700**

**2002**

**2004**

**2006**

**2008**

**Thousands**

**Logic Cells**

**MHz**

**Clock speed (MHz)**

## ORNL FPGA hardware/tools

- Cray XD1

- ORNL FPGA hardware/tools

**SRC-6 (Carte), Digilent (Viva, VHDL), Nallatech (Viva)**
**Cray XD1 (MitrionC, VHDL): ****6 FPGAs + 144 Opterons**
**SGI RASC-Altix/Virtex4s (MitrionC)**
**CHiMPS (Bee2 => Cray XD1 => DRC => XT4)**

**RASC**

***sgi***

**Notes:**
We were the first to install Mitrion-C on our XD1 with 6 FPGAs
And are porting applications

## Ported HPC code spectral transform shallow water model (STSWM) to FPGAs

- Find parallelism: 80% FFTs

- More GF/$ GF/Watt

- Goal

- Profile

- Model faster

- Ported HPC code spectral transform shallow water model (STSWM) to FPGAs

**HLL compiler**
**CHiMPS, Mitrion**
- (FPGA Tools Inside)

**FPGA**
**speedup**

**HLL  developer**
**profiles**

## Exploring programming options

**Viva: Graphical icons****—****3-dimensional**

**MitrionC: Text/flow****—****1-dimensional**

- Exploring programming options

**Gauss matrix solver**

**Compiler, simulator, and debugger**

**+ Carte/SRC, CHiMPS-VHDL/Xilinx ,**

**Notes:**
Viva and Mitrion-c are Widely Different approaches to code FPGAs

## 37X* LU decomposition speedup10X for matrix equation solver

- *FPGA vs 2.2 GHz Opteron

***First mixed-precision LU and solver for FPGAs***

**Benefits:**
**High performance**** of LP arithmetic**
**High precision**** accuracy**
**Speedup increases**** with matrix size**
**(as LU dominates calculations)**

| Design | Double FP | Single FP | S10e5 |
| --- | --- | --- | --- |
| PE Amount | 8 | 16 | 32 |
| Max size | 128 | 256 | 256 |
| Achievable
frequency | 120 MHz | 150 MHz | 150 MHz |
| Slices | 27,005 (57%) | 14,792 (59%) | 14,730 (62%) |
| BRAMs | 68 (29%) | 129 (55%) | 65 (28%) |
| MULT18X18 | 128 (55%) | 64 (27%) | 32 (13%) |

- 0

- 200

- 400

- 600

- 800

- 1000

- Execution time (us)

- 64

- 96

- 128

- Matrix size

- 87

- 57

- 149

- 218

- 133

- 404

- 865

- 443

- 258

- S10e5
- Single
- Double

- 0

- 20

- 30

- Speedup

- double

- single

- S10e5

- 10.9

- 7.7

- 21.3

- 9.7

- 10.3

- 36.6

- LU
- Solver

- Design data type

- 10

- 40

- 37X* LU decomposition speedup10X for matrix equation solver

## 100X* DNA sequence speedupBacillus anthracis human DNA comparison

**FPGA Speedup**

- 8 hrs => 5 min

- *Virtex-4 FPGA vs 2.2 GHz Opteron on Cray XD1

- 120

- 100

- 80

- 60

- 40

- 20

- 0

- 26

- 28

- 30

- 32

- 34

- 36

- 38

- 40

**Genome sequence**

- 8K w/align
- 16K w/align
- 8K w/o align
- 16K w/o align

- 100X* DNA sequence speedup*Bacillus anthracis* human DNA comparison

- 24#

- # 24= Sequence AE17024

## FPGA speedup growswith query size

## Summary

- This research is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725.

**ORNL FPGA research:**
**Increasing HPC relevence****	**
**FPGA systems: Cray, SRC, Nallatech, Digilent, SGI**
**Compilers: Mitrion-C, Carte, Viva, DSPlogic, CHiMPS **
**Speedup: ****10X**** eqn soln, ****100X**** DNA sequencing **
**Partners: Xilinx, UT, Mitrion, Cray, SGI**
**Next: Explore DRC, more FPGAs and CHiMPS**

## Contact

-    Storaasli_ReconfigHPC_SC07

- Olaf Storaasli
- Future Technologies GroupComputer Science and Mathematics Division
- olaf@ornl.gov
**G****o****o****g****l****e**** ****Olaf ORNL**