# Presentation: 933417

## General-Purpose GPU Computing

- By Aaron Collier

## GT200 GPU Architecture

- implements Tesla unified graphics and computing architecture which is generally referred to as Scalable Processor Array (SPA) design
- 240 Scalar/Streaming Processors (SPs) are organized into 10 Texture/Processor Clusters (TPCs) with each TPC containing 3 multithreaded Streaming Multiprocessors (SMs) or Streaming Processor Arrays (SPAs) and each SM is composed of a group of 8 SPs with 16 KB of shared local memory
- SM is based on a Single-Instruction, Multiple-Thread (SIMT) architecture and each thread controller unit manages local thread execution
- each SM also includes a 64-bit double-precision FPU for a total of 30 double-precision FPUs since the SPs are only capable of performing single-precision arithmetic
- each SP is a generalized scalar processor (fully pipelined, single-issue, in-order processing element) with 3 execution units (two ALUs and one single-precision FPU)

## GT200 GPU Architecture

- Streaming Multiprocessor

- Texture/Processor Cluster

## GT200 GPU Architecture

- Scalable Processor Array

## GT200 GPU Performance

- SP can perform one single-precision fused multiply-add (MAD) operation per clock cycle
- SM also contains two Special Function Units (SFUs) which each include four multiply (MUL) units that can perform one single-precision multiply operation per clock cycle
- SP can dual-issue MAD and MUL operations in conjunction with SFU which means it is theoretically capable of performing 3 floating point operations per clock cycle
- since the core SM functional units operate at a frequency of 1,296 MHz we have that the peak theoretical single-precision floating point performance of the GT200 GPU is approximately **933 GFLOPS**
- each double-precision FPU can perform one MAD operation per clock cycle
- given there are only 30 SMs we also have that the peak theoretical double-precision floating point performance of the GT200 GPU is approximately **78 GFLOPS**

## x86 CPU Performance

- current x86 CPUs utilize Single Instruction, Multiple Data (SIMD) instructions to operate on a set of packed data values simultaneously
- Streaming SIMD Extension (SSE) instructions (consist of SSE to SSE5) are an extension to the basic IA-32 SIMD instruction set and operate on multiple data elements packed into 128-bit registers
- each 128-bit register can store either 4 single-precision or 2 double-precision floating point values for processing so an SSE-enabled CPU could potentially perform 4 SP or 2 DP operations per clock cycle
- given an Intel Core 2 Extreme Quad-Core (QX9775) CPU which operates at 3.2 GHz we have that the aggregate peak theoretical floating point performance is approximately **52 GFLOPS** SP and                    **26 GFLOPS** DP
- future processors will implement an advanced version of the SSE specification called Advanced Vector Extensions (AVX) which will include 256-bit data registers

## Performance ComparisonComputational Capabilities

- regarding raw floating point performance the NVIDIA GT200 GPU is about **18x** and **3x** faster for SP and DP computations respectively than the Intel QX9775 CPU
- for GT200 GPU probably more realistic to assume for general computing tasks each SP is capable of performing only one single-precision floating point operation per clock cycle which would result in a peak performance of about **311 GLOPS**

## Performance ComparisonMemory Bandwidth

- GeForce GTX 280 has 512-bit GDDR3 memory interface that operates at a clock frequency of 1,107 MHz (2,214 MHz effective) which results in a peak theoretical sustained memory bandwidth of approximately **142 GB/s**
- Intel X48 Express chipset supports dual-channel quad-pumped 64-bit DDR3 memory interface that operates at a frequency of 400 MHz (1,600 MHz effective) which results in a peak aggregate memory bandwidth of about **26 GB/s** (**13 GB/s** per channel)
- GeForce GTX 280 has approximately **5.5x** the available local memory bandwidth of the Intel X48 Express chipset

## Performance ComparisonComputational Efficiency

- GeForce GTX 280 is rated for a maximum power consumption of 236W which results in peak efficiency ratings of **4 GFLOPS/W** SP and             **0.33 GFLOPS/W** DP
- Intel QX9775 CPU has maximum power consumption of 150W which results in peak efficiency ratings of  **0.35 GFLOPS/W** SP and            **0.17 GFLOPS/W** DP
- concerning computational efficiency the GeForce GTX 280 is about **11x** and **2x** more energy efficient for SP and DP computations respectively than the Intel QX9775 CPU

## Performance ComparisonComputational Cost

- NVIDIA GT200 GPU in the form of the GeForce GTX 280 product with a current market price of $500 achieves **$0.54/GFLOP** SP and             **$6.41/GFLOP** DP
- Intel QX9775 CPU with a current market price of $1,550 achieves **$29.81/GFLOP** SP and         **$59.62/GFLOP** DP
- regarding relative cost effectiveness the GeForce GTX 280 is about **55x** and **9x** more economical for both SP and DP computations respectively than the Intel QX9775 CPU

## GPGPU Programming Overview

- two methods for programming GPUs for HPC applications are OpenGL/GLSL and CUDA
- OpenGL/GLSL is a standard API for GPU programming meant for developing graphics applications and consequently requires an understanding of domain-specific terminology and the state machine concept
- NVIDIA’s Compute Unified Device Architecture (CUDA) programming interface provides a set of extensions to the C language and a parallel programming model abstraction meant to achieve transparent scaling
- CUDA abstractions provide fine-grained data parallelism (cooperative) and thread parallelism nested within coarse-grained data parallelism (independent) and task parallelism

## GPGPU Programming Overview

- GPUs based on the SPA architecture using the stream processing paradigm are ideally suited to data-parallel computations with relatively high arithmetic intensity
- x16 PCI Express v2.0 port is only capable of a peak theoretical bandwidth of **16 GB/s** (**8 GB/s** each direction) so data transfers between host and device should be minimized
- within the context of GPGPU computing data-parallel processing involves mapping each data element to a separate thread
- additional CUDA-specific information:
- logically related threads (meaning threads executing a common computational kernel) are grouped into blocks that are distributed among available SMs
- thread blocks are next subdivided by the multiprocessor SIMT unit into warps which ideally consist of 32 parallel threads
- each thread is then mapped to a single scalar processor core

## GPGPU Applications

- Q-Chem which is a general purpose code for quantum mechanical calculations of molecular properties reported a **4x** speed up using an early generation CUDA-enabled GPU                                        **Accelerating Resolution-of-the-Identity Second-Order Mller-Plesset Quantum Chemistry Calculations** **with Graphical Processing Units**                                                                                                                               Vogt, L., Olivares-Amaya, R., Kermes, S., Shao, Y., Amador-Bedolla, C., and Aspuru-Guzik, A.*                                                             J. Phys. Chem. A*, 112, 10, 2049 - 2057, 2008,  10.1021/jp0776762
- HOOMD (Highly Optimized Object-oriented Molecular Dynamics) which performs general purpose molecular dynamics simulations attained a **15x** speed up using CUDA-enabled GPUs                                                               **General purpose molecular dynamics simulations fully implemented on graphics processing units**Joshua A. Anderson, Chris D. Lorenz, and Alex Travesset                                                                                                              *J. Comp. Phys.*, 227, 10, 5342 - 5359, 2008,  10.1016/j.jcp.2008.01.047
- Code using GPUs to calculate two-electron repulsion integrals over Gaussian basis functions showed a **130x** speed up                                            **Quantum Chemistry on Graphical Processing Units. 1. Strategies for Two-Electron Integral Evaluation**                                                                                                                                                      Ufimtsev, I.S. and Martinez, T.J.*                                                                                                                                                   J. Chem. Theory Comput.*, 4, 2, 222 - 231, 2008,  10.1021/ct700268q

## Final Remarks

- moving toward hybrid system architectures with specialized co-processors used to accelerate computations
- IBM’s BlueGene/L platform essentially merges a standard PowerPC core with an optimized DSP to implement a dual-FPU architecture called Double Hummer
- IBM’s Roadrunner system uses IBM PowerXCell 8i (CBE) CPUs as co-processors for commodity AMD Opteron 2210 processors
- AMD’s recent Torrenza initiative is focused on providing support for the direct integration of specialized co-processing elements into AMD-based platforms
- GPGPU computing is growing in popularity due to the commoditization of SPA-based architectures (meaning general-purpose GPUs) by both AMD and NVIDIA

## References

- Bell, Brandon.  “NVIDIA GeForce GTX 280/GTX 260 Performance
-    Preview.”  *FiringSquad*.  16 June 2008.  2 July 2008
-    <http://www.firingsquad.com/hardware
-    nvidia_geforce_gtx_280_260_performance/default.asp>.
- Chester, Edward.  “NVIDIA Geforce GTX280.”  *TrustedReviews*.  25 June
-    2008.  3 July 2008  <http://www.trustedreviews.com/graphics/review
-    2008/06/25/nVidia-GeForce-GTX-280/p1>.
- Shimpi, Anand et al.  “NVIDIA’s 1.4 Billion Transistor GPU: GT200 Arrives as
-    the Geforce GTX 280 and 260.”  *AnandTech*.  16 June 2008.  3 July 2008
-    <http://www.anandtech.com/video/showdoc.aspx?i=3334&p=1>.
- Smalley, Tim.  “GT200: NVIDIA GeForce GTX 280 Analysis.”  *bit-tech.net*.
-    24 June 2008.  2 July 2008  <http://www.bit-tech.net/hardware
-    2008/06/24/nvidia-geforce-gtx-280-architecture-review/1>.

## Questions