# Late-Stage Optimization of Modern ILP Processor Cores via FPGA Simulation
###### Abstract  
Late-stage (post-RTL implementation) optimization is important in achieving target performance for realistic processor design. However, several challenges remain for modern out-of-order ILP (instruction-level-parallelism) processors, such as simulation speed, flexibility, and complexity problems. This paper restudy FPGA simulation as an effective performance simulation method and proposes FPGA-enhanced design flow as an effective method to address these problems. It features a late-stage aware RTL design that parameterizes various potential design options induced from early-stage optimization. This flow enables the feasibility of late-stage design space exploration. To resolve the performance accuracy of the FPGA system for peripheral designs, reference models are introduced. With an example implementation of out-of-order core running up to 80 MHz, the experimental results show that the proposed method is practical and allows the fine-grain optimization of the processor core to be more effective. FPGA; performance; simulation  
for late-stage optimization. However, software simulators are typically inaccurate and operate with abstract, incomplete, or missing components. A cycle-accurate simulator can be implemented similarly as RTL, but doing so requires additional work, such as in RTL implementation, and reduces speed. The FPGA simulator [6; 7; 8; 9; 10; 11; 12; 13; 14; 15] is an appealing means of late-stage optimization due to its advantages, such as speed and low cost. The FPGA system is already mature for simple in-order processors. Achieving target speed [16] is difficult for modern complex out-of-order processors due to issues in capacity and structural complexity. A previous study [17] indicated a speed problem, whereas others highlighted such issues as extended setup process, which results in high simulation turnaround time; significant effort to design the change in RTL for each experimental machine, and accuracy for simulated peripheral designs. FPGA technology has significantly improved recently, and we show that it can be used as a method for the practical performance simulation of the late-stage optimization of modern out-of-order cores. Furthermore, we introduce reference models to solve the accuracy problem of peripheral designs on FPGA system. To demonstrate, we use a typical out-of-order core to develop a corresponding FPGA simulation system for late-stage optimization. It implements a complete computation platform equipped with a modern complex processor core into a single FPGA chip. It can achieve up to 80 MHz processor frequency. An efficient microarchitecture exploration flow can be formed, and FPGA simulation is used as a new performance test iteration for rapid optimization flow at the late stage. For the processor core, coarse-grain optimization can be done at the early stage using software simulation, whereas fine-grain optimization can be performed at the late stage using FPGA simulation. It features a late-stage aware RTL design, which parameterizes various late-stage design options derived from early-stage optimization. This flow minimizes the RTL change effort for late-stage optimization. Using widely used SPEC CPU benchmarks as an example, we show the effectiveness of FPGA simulation for late-stage optimization. The rest of the paper is organized as follows. Section 2 summaries the FPGA-enhanced design methodology. Section 3 introduces the FPGA implementation. Section 4 provides the evaluation results and case studies. Section 5 describes the related works, and Section 6 concludes this paper. In contrast to previous works, we believe this paper makes the following unique contributions:  
(1) We identify the potential benefits of late-stage optimization, which is rarely studied, and use the proposed FPGA-enhanced design flow to accomplish it for fine-grain microarchitecture optimization. (2) We introduce an FPGA system for demonstration, which ports a modern processor core with extensive optimizations at 80 MHz frequency.