Performance and Power Optimization of GPU Architectures for General-purpose Computing

vii CHAPTER 1: INTRODUCTION . 1 1.1 Power and Energy Concern in GPU Computing Systems . 2 1.2 Performance Concern in GPU Computing Systems . 2 1.3 Contributions of This Dissertation . 3 1.4 Outline of Dissertation . 5 CHAPTER 2: BACKGROUND AND RELATED WORK . 7 2.1 GPU Architecture and Terminology. 7 2.1.1 Single Instruction Multiple Thread (SIMT) Architecture . 8 2.1.2 Warp Scheduling . 9 2.1.3 Memory Hierarchy . 10 2.2 Power Consumption in CMOS Circuit. 11 2.2.1 Power Gating Technique . 12 2.2.2 Dynamic Frequency Scaling . 13 2.3 Proportional-Integral-Derivative Controlling Algorithm . 14 2.4 Linear Programming and Simplex Method . 16 2.5 Related Work . 20 2.5.1 Dynamic Workload Management on CPU/GPU . 20 2.5.2 Power Optimization on CPU/GPU . 21 2.5.2.1 To Reduce Static Leakage Power . 21 2.5.2.2 To Reduce Dynamic Power . 22 CHAPTER 3: RUNTIME POWER GATING IN CACHES OF GPUS FOR LEAKAGE ENERGY SAVINGS . 24 3.1 Motivation . 24 3.2 Proposed Microarchitectural Technique for Leakage Reduction on Caches Using Power Gating . 26 ii 3.2.1 Power Gating L1 Caches of GPU . 27 3.2.2 Power Gating L2 Caches of GPU . 28 3.2.3 Analysis of Latency Hiding . 30 3.2.3.1 Hiding Transition Latency Between off and active in L1 . 30 3.2.3.2 Hiding Transition Latency Between sleep and active in L1 . 31 3.2.3.3 Hiding Transition Latency Between sleep and active in L2 . 32 3.3 Experimental Setup and Results . 32 3.3.1 Modeling of Power Gated SRAM Array . 32 3.3.2 Benchmarks and Simulator Setup . 33 3.3.3 Evaluation and Analysis . 37 3.4 Conclusion . 39 CHAPTER 4: LOAD BALANCING ON THE RUNTIME FOR GPUS . 41 4.1 Motivation . 41 4.2 Methodology and Implementation . 43 4.2.1 Overview of Dynamic Workload Balancing . 44 4.2.2 Microarchitecture Design. 45 4.2.3 Dynamic Workload Balancing Algorithm . 48 4.2.3.1 Memory Request Handling . 49 4.2.3.2 Synchronization and Shared Memory Access . 50 4.3 Experimental Results and Analysis . 52 4.3.1 Performance Improvement Result . 53 4.3.2 Choosing Optimal Values for threshold and fraction . 55 4.3.3 Result Analysis . 56 4.3.3.1 Warp-level Parallelism . 57 4.3.3.2 Average Memory Access Time . 58 4.3.4 Overhead Analysis . 59 4.4 Conclusion . 61 CHAPTER 5: DYNAMIC FREQUENCY SCALING ON GPUS . 63 5.1 PIDDFS Dynamic Frequency Scaling Using PID Controlling Theory . 64 5.1.1 Motivation . 64 5.1.2 PIDDFS Methodology . 66 5.1.2.1 SMX Frequency Controller . 67 5.1.2.2 Memory Channel Frequency Controller . 68 5.1.3 Implementation of PIDDFS . 68 5.1.3.1 Prototype of Code Implementation. 68 5.1.3.2 Fixing the Activation Period . 69 5.1.3.3 Tuning the PID Parameters . 71 5.2 LPDFS Dynamic Frequency Scaling Using Linear Programming . 71 5.2.1 Motivation . 72 iii 5.2.2 Methodology of LPDFS . 73 5.2.3 Implementation of LPDFS . 76 5.3 Simulation and Evaluation . 78 5.3.1 Evaluation of Memory Stall Reduction via PIDDFS . 79 5.3.1.1 Result . 79 5.3.1.2 Analysis . 80 5.3.2 Evaluation of Performance Improvement . 82 5.3.2.1 Result . 83 5.3.2.2 Analysis . 83 5.3.3 Evaluation of Power Saving . 84 5.3.3.1 Result . 84 5.3.3.2 Analysis . 85 5.4 Conclusion . 86 CHAPTER 6: CONCLUSIONS AND FUTURE DIRECTIONS . 88 REFERENCES . 90 ABOUT THE AUTHOR . END PAGE