In Search of Lost Bandwidth: Extensive Reordering of DRAM Accesses on FPGA For efficient acceleration on FPGA, it is essential for external memory to match the throughput of the processing pipelines. However, the usable DRAM bandwidth decreases significantly if the access pattern causes frequent row conflicts. Memory controllers reorder DRAM commands to minimize row conflicts; however, generalpurpose controllers must also minimize latency, which limits the depth of the internal queues over which reordering can occur. For latencyinsensitive applications with irregular access pattern, nonblocking caches that support thousands of inflight misses (missoptimized memory systems) improve bandwidth utilization by reusing the same memory response to serve as many incoming requests as possible. However, they do not improve the irregularity of the access pattern sent to the memory, meaning that row conflicts will still be an issue. Sending out bursts instead of single memory requests makes the access pattern more sequential; however, realistic implementations trade high throughput for some unnecessary data in the bursts, leading to bandwidth wastage that cancels out part of the gains from regularization. In this paper, we present an alternative approach to extend the scope of DRAM row conflict minimization beyond the possibilities of general purpose DRAM controllers. We use the thousands of future memory requests that spontaneously accumulate inside the missoptimized memory system to implement an efficient largescale reordering mechanism. By reordering single requests instead of sending bursts, we regularize the memory access pattern in a way that increases bandwidth utilization without incurring in any data wastage. Our solution outperforms the baseline missoptimized memory system by up to 81% and has better worst, average, and best performance than DynaBurst across 15 benchmarks and 30 architectures.

To Update or Not To Update?: BandwidthEfficient Intelligent Replacement Policies for DRAM Caches This paper investigates intelligent replacement policies for improving the hitrate of gigascale DRAM caches. Cache replacement policies are commonly used to improve the hitrate of onchip caches. The most effective replacement policies often require the cache to track and update perline reuse state to inform their decision. A fundamental challenge on DRAM caches, however, is that stateful policies would require significant bandwidth to maintain perline DRAM cache state. As such, DRAM cache replacement policies have primarily been stateless policies, such as alwaysinstall or probabilistic bypass. Unfortunately, we find that stateless policies are often too coarsegrain and become ineffective at the size and associativity of DRAM caches. Ideally, we want a replacement policy that can obtain the hitrate benefits of stateful replacement policies, but keep the bandwidthefficiency of stateless policies. We perform our study on a DRAM cache design similar to the one in Knights Landing, and find that tracking perline reuse state can enable an effective replacement policy that can mitigate the common thrashing patterns seen in gigascale caches. We propose a stateful replacementbypass policy called RRIP AgeOnBypass (RRIPAOB), that tracks reuse state for highreuse lines, protects such lines by bypassing other lines, and Ages the state On cache Bypass. Unfortunately, such a stateful technique requires significant bandwidth to update state. To this end, we propose Efficient Tracking of Reuse (ETR). ETR makes state tracking efficient by accurately tracking the state of only one line from a region, and using the state of that line to guide the replacement decisions for other lines in that region. ETR reduces the bandwidth for tracking replacement state by 70%, and makes stateful policies practical for DRAM caches. Our evaluations with a 2GB DRAM cache, show that our RRIPAOB and ETR techniques provide 18% speedup while needing u003c1KB SRAM.