Haakon Dybdahl Architectural Techniques to Improve Cache Utilization

The cache is a memory area where recently accessed data is stored for fast access. The size of the cache has grown rapidly during the last decades to compensate for the increasingly slower main memory. The state-of-the-art memory hierarchy has multiple levels of cache with the last-level being the largest and slowest. The area of the chip dedicated to the last-level cache is substantial, but the performance of the cache is quite low. Typically 50% of the data in the cache will not be accessed before the data is removed. This thesis takes three approaches to improve the utilization of the cache. The first approach extends the cache functionality to include write-backs for destructiveread DRAM. Destructive-read DRAM is faster than conventional DRAM, but the content is lost on read operations. The contribution is two schemes for using the cache as a write-back buffer instead of using a dedicated write-back buffer. In the second approach several different schemes are proposed that improve control of which data that are kept in the last-level cache. The first scheme bypasses memory accesses that are transient. A heuristic classifies accesses as transient or non-transient based on a run-time analysis of the instructions that load the data. This heuristic is more precise than earlier proposed schemes intended for first-level caches. In the second scheme the references to each cache block are counted. This information is used in an extension to the conventional least-recently-used (LRU) replacement policy so cache blocks that are accessed frequently are protected from eviction by cache block which are less frequently accessed. The scheme is fairly simple as it requires no additional information from the processor and only extends the cache block with an additional counter per block. The scheme is efficient in a chip multiprocessor (CMP) architecture with shared last-level cache since it can stop a single processor from filling the cache with unnecessary blocks. This protects needed cache blocks for the other processors. In the third approach the last-level cache is dynamically partitioned in a CMP. This is evaluated for a conventional shared cache architecture and for a novel non-uniform cache architecture (NUCA) with concurrent private and shared last-level cache. Novel mechanisms for determining the best partition sizes for each processor are described and used in these schemes. These schemes outperform conventional partitioning by an LRU replacement policy which partitions the cache more arbitrary. A mechanism is proposed for monitoring performance of an alternative cache scheme in order to activate it only when it improves performance. The tags for a limited number of sets in the cache are duplicated and the alternative scheme is run on these tags. This mechanism can stabilize schemes that are not robust and hence increase the overall performance. The different schemes are simulated by an extended version of SimpleScalar. The SPEC2000 benchmark suite is used as workload. The performance gains of the new schemes are shown relative to conventional architectures, state-of-the-art techniques and compared to upper-bound non-implementable approaches.