exploring code cache eviction granularities in dynamic optimization systems
Dynamic optimization systems store optimized or translatedcode in a software-managed code cache in order tomaximize reuse of transformed code. Code caches storesuperblocks that are not fixed in size, may contain linksto other superblocks, and carry a high replacement overhead.These additional constraints reduce the effectivenessof conventional hardware-based cache management policies.In this paper, we explore code cache managementpolicies that evict large blocks of code from the code cache,thus avoiding the bookkeeping overhead of managing singlecache blocks. Through a combined simulation and analyticalstudy of cache management overheads, we show thatemploying a medium-grained FIFO eviction policy resultsin an effective balance of cache management complexityand cache miss rates. Under high cache pressure the choiceof medium granularity translates into a significant reductionin overall execution time versus both coarse and finegranularities.
exploration code cache cache dynamic optimization systems management order reuse size constraint effect policy paper simulation timing software hardware caches