The high amount of memory requests from massive threads may easily cause cache contention and cache-miss-related resource congestion on GPUs. This paper proposes a simple yet effective performance model to estimate the impact of cache contention and resource congestion as a function of the number of warps/thread blocks (TBs) to bypass the cache. Then we design a hardware-based dynamic warp/thread-block level GPU cache bypassing scheme, which achieves 1.68x speedup on average on a set of memoryintensive benchmarks over the baseline. Compared to prior works, our scheme achieves 21.6% performance improvement over SWLbest [29] and 11.9% over CBWT-best [4] on average.
Dai et al. propose a mechanism to bypass cache based on a model of a cache miss rate REF .
A model-driven approach to warp/thread-block level GPU cache bypassing
DAC '16
Computer Science