Programming Massively Parallel Processors: A Hands-on Approach
Publisher: Morgan Kaufmann
Published: Dec 28, 2012
Number of pages: 514
ISBN-13: 9780124159921
Table of contents: title: Preface
pagenum: xiii; title: Acknowledgements
pagenum: xix; title: Dedication
pagenum: xxi; label: Chapter 1
title: Introduction
pagenum: 1; label: 1.1
title: Heterogeneous Parallel Computing
pagenum: 2; label: 1.2
title: Architecture of a Modern GPU
pagenum: 8; label: 1.3
title: Why More Speed or Parallelism?
pagenum: 10; label: 1.4
title: Speeding Up Real Applications
pagenum: 12; label: 1.5
title: Parallel Programming Languages and Models
pagenum: 14; label: 1.6
title: Overarching Goals
pagenum: 16; label: 1.7
title: Organization of the Book
pagenum: 17; title: References
pagenum: 21; label: Chapter 2
title: History of GPU Computing
pagenum: 23; label: 2.1
title: Evolution of Graphics Pipelines
pagenum: 23; title: The Era of Fixed-Function Graphics Pipelines
pagenum: 24; title: Evolution of Programmable Real-Time Graphics
pagenum: 28; title: Unified Graphics and Computing Processors
pagenum: 31; label: 2.2
title: GPGPU: An Intermediate Step
pagenum: 32; label: 2.3
title: GPU Computing
pagenum: 34; title: Scalable GPUs
pagenum: 35; title: Recent Developments
pagenum: 36; title: Future Trends
pagenum: 37; title: References and Further Reading
pagenum: 37; label: Chapter 3
title: Introduction to Data Parallelism and CUDA C
pagenum: 41; label: 3.1
title: Data Parallelism
pagenum: 42; label: 3.2
title: CUDA Program Structure
pagenum: 43; label: 3.3
title: A Vector Addition Kernel
pagenum: 45; label: 3.4
title: Device Global Memory and Data Transfer
pagenum: 48; label: 3.5
title: Kernel Functions and Threading
pagenum: 53; label: 3.6
title: Summary
pagenum: 58; title: Function Declarations
pagenum: 59; title: Kernel Launch
pagenum: 59; title: Predefined Variables
pagenum: 60; title: Runtime API
pagenum: 60; label: 3.7
title: Exercises
pagenum: 60; title: References
pagenum: 62; label: Chapter 4
title: Data-Parallel Execution Model
pagenum: 63; label: 4.1
title: CUDA Thread Organization
pagenum: 64; label: 4.2
title: Mapping Threads to Multidimensional Data
pagenum: 68; label: 4.3
title: Matrix-Matrix Multiplication—A More Complex Kernel
pagenum: 74; label: 4.4
title: Synchronization and Transparent Scalability
pagenum: 81; label: 4.5
title: Assigning Resources to Blocks
pagenum: 83; label: 4.6
title: Querying Device Properties
pagenum: 85; label: 4.7
title: Thread Scheduling and Latency Tolerance
pagenum: 87; label: 4.8
title: Summary
pagenum: 91; label: 4.9
title: Exercises
pagenum: 91; label: Chapter 5
title: CUDA Memories
pagenum: 95; label: 5.1
title: Importance of Memory Access Efficiency
pagenum: 96; label: 5.2
title: CUDA Device Memory Types
pagenum: 97; label: 5.3
title: A Strategy for Reducing Global Memory Traffic
pagenum: 105; label: 5.4
title: A Tiled Matrix-Matrix Multiplication Kernel
pagenum: 109; label: 5.5
title: Memory as a Limiting Factor to Parallelism
pagenum: 115; label: 5.6
title: Summary
pagenum: 118; label: 5.7
title: Exercises
pagenum: 119; label: Chapter 6
title: Performance Considerations
pagenum: 123; label: 6.1
title: Warps and Thread Execution
pagenum: 124; label: 6.2
title: Global Memory Bandwidth
pagenum: 132; label: 6.3
title: Dynamic Partitioning of Execution Resources
pagenum: 141; label: 6.4
title: Instruction Mix and Thread Granularity
pagenum: 143; label: 6.5
title: Summary
pagenum: 145; label: 6.6
title: Exercises
pagenum: 145; title: References
pagenum: 149; label: Chapter 7
title: Floating-Point Considerations
pagenum: 151; label: 7.1
title: Floating-Point Format
pagenum: 152; title: Normalized Representation of \(M\)
pagenum: 152; title: Excess Encoding of \(E\)
pagenum: 153; label: 7.2
title: Representable Numbers
pagenum: 155; label: 7.3
title: Special Bit Patterns and Precision in IEEE Format
pagenum: 160; label: 7.4
title: Arithmetic Accuracy and Rounding
pagenum: 161; label: 7.5
title: Algorithm Considerations
pagenum: 162; label: 7.6
title: Numerical Stability
pagenum: 164; label: 7.7
title: Summary
pagenum: 169; label: 7.8
title: Exercises
pagenum: 170; title: References
pagenum: 171; label: Chapter 8
title: Parallel Patterns: Convolution
pagenum: 173; label: 8.1
title: Background
pagenum: 174; label: 8.2
title: 1D Parallel Convolution—A Basic Algorithm
pagenum: 179; label: 8.3
title: Constant Memory and Caching
pagenum: 181; label: 8.4
title: Tiled 1D Convolution with Halo Elements
pagenum: 185; label: 8.5
title: A Simpler Tiled 1D Convolution—General Caching
pagenum: 192; label: 8.6
title: Summary
pagenum: 193; label: 8.7
title: Exercises
pagenum: 194; label: Chapter 9
title: Parallel Patterns: Prefix Sum
pagenum: 197; label: 9.1
title: Background
pagenum: 198; label: 9.2
title: A Simple Parallel Scan
pagenum: 200; label: 9.3
title: Work Efficiency Considerations
pagenum: 204; label: 9.4
title: A Work-Efficient Parallel Scan
pagenum: 205; label: 9.5
title: Parallel Scan for Arbitrary-Length Inputs
pagenum: 210; label: 9.6
title: Summary
pagenum: 214; label: 9.7
title: Exercises
pagenum: 215; title: Reference
pagenum: 216; label: Chapter 10
title: Parallel Patterns: Sparse Matrix-Vector Multiplication
pagenum: 217; label: 10.1
title: Background
pagenum: 218; label: 10.2
title: Parallel SpMV Using CSR
pagenum: 222; label: 10.3
title: Padding and Transposition
pagenum: 224; label: 10.4
title: Using Hybrid to Control Padding
pagenum: 226; label: 10.5
title: Sorting and Partitioning for Regularization
pagenum: 230; label: 10.6
title: Summary
pagenum: 232; label: 10.7
title: Exercises
pagenum: 233; title: References
pagenum: 234; label: Chapter 11
title: Application Case Study: Advanced MRI Reconstruction
pagenum: 235; label: 11.1
title: Application Background
pagenum: 236; label: 11.2
title: Iterative Reconstruction
pagenum: 239; label: 11.3
title: Computing FHD
pagenum: 241; title: Step 1: Determine the Kernel Parallelism Structure
pagenum: 243; title: Step 2: Getting Around the Memory Bandwidth Limitation
pagenum: 249; title: Step 3: Using Hardware Trigonometry Functions
pagenum: 255; title: Step 4: Experimental Performance Tuning
pagenum: 259; label: 11.4
title: Final Evaluation
pagenum: 260; label: 11.5
title: Exercises
pagenum: 262; title: References
pagenum: 264; label: Chapter 12
title: Application Case Study: Molecular Visualization and Analysis
pagenum: 265; label: 12.1
title: Application Background
pagenum: 266; label: 12.2
title: A Simple Kernel Implementation
pagenum: 268; label: 12.3
title: Thread Granularity Adjustment
pagenum: 272; label: 12.4
title: Memory Coalescing
pagenum: 274; label: 12.5
title: Summary
pagenum: 277; label: 12.6
title: Exercises
pagenum: 279; title: References
pagenum: 279; label: Chapter 13
title: Parallel Programming and Computational Thinking
pagenum: 281; label: 13.1
title: Goals of Parallel Computing
pagenum: 282; label: 13.2
title: Problem Decomposition
pagenum: 283; label: 13.3
title: Algorithm Selection
pagenum: 287; label: 13.4
title: Computational Thinking
pagenum: 293; label: 13.5
title: Summary
pagenum: 294; label: 13.6
title: Exercises
pagenum: 294; title: References
pagenum: 295; label: Chapter 14
title: An Introduction to OpenCL
pagenum: 297; label: 14.1
title: Background
pagenum: 297; label: 14.2
title: Data Parallelism Model
pagenum: 299; label: 14.3
title: Device Architecture
pagenum: 301; label: 14.4
title: Kernel Functions
pagenum: 303; label: 14.5
title: Device Management and Kernel Launch
pagenum: 304; label: 14.6
title: Electrostatic Potential Map in OpenCL
pagenum: 307; label: 14.7
title: Summary
pagenum: 311; label: 14.8
title: Exercises
pagenum: 312; title: References
pagenum: 313; label: Chapter 15
title: Parallel Programming with OpenACC
pagenum: 315; label: 15.1
title: OpenACC Versus CUDA C
pagenum: 315; label: 15.2
title: Execution Model
pagenum: 318; label: 15.3
title: Memory Model
pagenum: 319; label: 15.4
title: Basic OpenACC Programs
pagenum: 320; title: Parallel Construct
pagenum: 320; title: Loop Construct
pagenum: 322; title: Kernels Construct
pagenum: 327; title: Data Management
pagenum: 331; title: Asynchronous Computation and Data Transfer
pagenum: 335; label: 15.5
title: Future Directions of OpenACC
pagenum: 336; label: 15.6
title: Exercises
pagenum: 337; label: Chapter 16
title: Thrust: A Productivity-Oriented Library for CUDA
pagenum: 339; label: 16.1
title: Background
pagenum: 339; label: 16.2
title: Motivation
pagenum: 342; label: 16.3
title: Basic Thrust Features
pagenum: 343; title: Iterators and Memory Space
pagenum: 344; title: Interoperability
pagenum: 345; label: 16.4
title: Generic Programming
pagenum: 347; label: 16.5
title: Benefits of Abstraction
pagenum: 349; label: 16.6
title: Programmer Productivity
pagenum: 349; title: Robustness
pagenum: 350; title: Real-World Performance
pagenum: 350; label: 16.7
title: Best Practices
pagenum: 352; title: Fusion
pagenum: 353; title: Structure of Arrays
pagenum: 354; title: Implicit Ranges
pagenum: 356; label: 16.8
title: Exercises
pagenum: 357; title: References
pagenum: 358; label: Chapter 17
title: CUDA Fortran
pagenum: 359; label: 17.1
title: CUDA Fortran and CUDA C Differences
pagenum: 360; label: 17.2
title: A First CUDA Fortran Program
pagenum: 361; label: 17.3
title: Multidimensional Array in CUDA Fortran
pagenum: 363; label: 17.4
title: Overloading Host/Device Routines With Generic Interfaces
pagenum: 364; label: 17.5
title: Calling CUDA C Via Iso_C_Binding
pagenum: 367; label: 17.6
title: Kernel Loop Directives and Reduction Operations
pagenum: 369; label: 17.7
title: Dynamic Shared Memory
pagenum: 370; label: 17.8
title: Asynchronous Data Transfers
pagenum: 371; label: 17.9
title: Compilation and Profiling
pagenum: 377; label: 17.10
title: Calling Thrust from CUDA Fortran
pagenum: 378; label: 17.11
title: Exercises
pagenum: 382; label: Chapter 18
title: An Introduction to C++ AMP
pagenum: 383; label: 18.1
title: Core C++ AMP Features
pagenum: 384; label: 18.2
title: Details of the C++ AMP Execution Model
pagenum: 391; title: Explicit and Implicit Data Copies
pagenum: 391; title: Asynchronous Operation
pagenum: 393; title: Section Summary
pagenum: 395; label: 18.3
title: Managing Accelerators
pagenum: 395; label: 18.4
title: Tiled Execution
pagenum: 398; label: 18.5
title: C++ AMP Graphics Features
pagenum: 401; label: 18.6
title: Summary
pagenum: 405; label: 18.7
title: Exercises
pagenum: 405; label: Chapter 19
title: Programming a Heterogeneous Computing Cluster
pagenum: 407; label: 19.1
title: Background
pagenum: 408; label: 19.2
title: A Running Example
pagenum: 408; label: 19.3
title: MPI Basics
pagenum: 410; label: 19.4
title: MPI Point-to-Point Communication Types
pagenum: 414; label: 19.5
title: Overlapping Computation and Communication
pagenum: 421; label: 19.6
title: MPI Collective Communication
pagenum: 431; label: 19.7
title: Summary
pagenum: 432; label: 19.8
title: Exercises
pagenum: 433; title: Reference; label: Chapter 20
title: CUDA Dynamic Parallelism
pagenum: 435; label: 20.1
title: Background
pagenum: 436; label: 20.2
title: Dynamic Parallelism Overview
pagenum: 438; label: 20.3
title: Important Details
pagenum: 439; title: Launch Environment Configuration
pagenum: 439; title: API Errors and Launch Failures
pagenum: 439; title: Events
pagenum: 439; title: Streams
pagenum: 440; title: Synchronization Scope
pagenum: 441; label: 20.4
title: Memory Visibility
pagenum: 442; title: Global Memory
pagenum: 442; title: Zero-Copy Memory
pagenum: 442; title: Constant Memory
pagenum: 442; title: Texture Memory
pagenum: 443; label: 20.5
title: A Simple Example
pagenum: 444; label: 20.6
title: Runtime Limitations
pagenum: 446; title: Memory Footprint
pagenum: 446; title: Nesting Depth
pagenum: 448; title: Memory Allocation and Lifetime
pagenum: 448; title: ECC Errors
pagenum: 449; title: Streams
pagenum: 449; title: Events
pagenum: 449; title: Launch Pool
pagenum: 449; label: 20.7
title: A More Complex Example
pagenum: 449; title: Linear Bezier Curves
pagenum: 450; title: Quadratic Bezier Curves
pagenum: 450; title: Bezier Curve Calculation (Predynamic Parallelism)
pagenum: 450; title: Bezier Curve Calculation (with Dynamic Parallelism)
pagenum: 453; label: 20.8
title: Summary
pagenum: 456; title: Reference
pagenum: 457; label: Chapter 21
title: Conclusion and Future Outlook
pagenum: 459; label: 21.1
title: Goals Revisited
pagenum: 459; label: 21.2
title: Memory Model Evolution
pagenum: 461; label: 21.3
title: Kernel Execution Control Evolution
pagenum: 464; label: 21.4
title: Core Performance
pagenum: 467; label: 21.5
title: Programming Environment
pagenum: 467; label: 21.6
title: Future Outlook
pagenum: 468; title: References
pagenum: 469; title: Appendix A: Matrix Multiplication Host-Only Version Source Code
pagenum: 471; title: Appendix B: GPU Compute Capabilities
pagenum: 481; title: Index
pagenum: 487