BOA: Targeting Multi-Gigahertz with Binary Translation

This paper presents BOA (Binary-translation Optimized Architecture), a processor designed to achieve high frequency by using software dynamic binary translation. Processors for software binary translation are very conducive to high frequency because they can assume a simple hardware design. Binary translation eliminates the binary compatibility problem faced by other processors, while dynamic recompilation enables re-optimization of critical program code sections and eliminates the need for dynamic scheduling hardware. In this work we examine the implications of binary translation on processor architecture and software translation and how we support a very high frequency PowerPC implementation via dynamic binary translation. 1.0 Introduction The design of processors with clock speeds of 1 GHz or more has been a topic of considerable research in both industry and academia. Binary translation presents an interesting alternative for processor design as it enables good performance on simple processor designs. Processors for binary translation achieve maximum performance by enabling high frequency processors while still exploiting available parallelism in the code. The effect of both of these optimizations is to minimize overall execution time [1]: Execution Time Ζ (Number Of Instructions) Cycles Instruction Seconds Cycle One method of minimizing execution time is instruction set simplicity which can enable high frequency design. Many approaches can be taken to achieve high frequency while maintaining compatibility with existing code. Three popular possible methods include: Hardware Cracking: Hardware cracking of complex instructions into simpler "micro" operations is frequently used (e.g., Intel processors such as Pentium Pro, Pentium II, and Pentium III). Nair and Hopkins' DIF processor [2] also uses a hardware approach, to crack and schedule simple PowerPC operations into groups for high-speed execution on a VLIW (Very-Long Instruction Word) processor. Cracked code can be stored in the cache, or cracking can occur on the fly (in the pipeline), but either way, it consumes time and transistors. This limits the achievable frequency. Microcode Emulation: Microcode can also be used to maintain compatibility, with each instruction of the original architecture going to a fixed microcode routine for emulation on the new (high frequency) architecture. However, if the new instruction set architecture does not closely resemble the original architecture, the total number of instructions executed is much higher than on a direct implementation of the original machine. In addition, there is no opportunity to overlap the execution of instructions from the original machine, thus preventing the exploitation of any intrinsic parallelism. Binary Translation: A third alternative is dynamic binary translation via software as exemplified by FX! 32 [3] or DAISY [4]. Since the translation is done by software, no extra transistors are needed for implementation -the same resources used to execute the application code are used by the translation software. Assuming that the high frequency processor for a binary translation machine is designed with the original in mind, the high frequency primitive operations can efficiently execute instructions for the original machine. Unlike microcode, the high frequency primitives from several original instructions can be overlapped so as to exploit parallelism. Another advantage of dynamic binary translation over hardware cracking and microcode is adaptability -the code generated can be adapted to efficiently handle special cases discernible only at runtime. For example if fixed parameters are always passed to a subroutine, the dynamic translation software can generate code which checks for these fixed parameters, and then jump to a highly optimized translation of the subroutine if these parameters are encountered3. Similarly, translations can be optimized to varying degrees depending upon the execution frequency of the code section. 1 3 Our current implementation does not support this optimization. 2 IBM Server Division, Burlington, VT, USA 1 IBM T. J. Watson Research Center, Yorktown Heights, NY, USA In looking forward to future high performance microprocessors, we have adopted the dynamic binary translation approach as it promises a desirable combination of (1) high frequency design, (2) greater degrees of parallelism, and (3) low hardware cost. In this paper, we describe the application of dynamic binary translation to the design of BOA (Binary-translation Optimized Architecture) a high-frequency EPIC-style implementation of the PowerPC architecture. The remainder of the paper describes our approach in designing a high frequency PowerPC compatible microprocessor through dynamic binary translation. Section 2 describes how the processor builds groups of PowerPC instructions and the translation process into the BOA architecture. Section 3 describes the BOA instruction set architecture and details the high frequency implementation. Section 4 gives experimental microarchitectural performance results, and Sections 5 and 6 conclude and describe future work. 2.0 Binary Translation Strategy Our overall strategy of binary translation is to translate a PowerPC instruction flow into a set of BOA instruction groups which are then stored in memory and executed natively. The system must start the process by interpreting the PowerPC stream of instructions and make intelligent decisions on when and how to translate these instructions into BOA groups. This process is described in Section 2.1. Once groups have been formed, the translation process continues by scheduling the BOA groups for performance. These optimizations are covered in Section 2.2 where again, intelligent tradeoffs must be made between code optimization, code reuse, and software optimization costs. Code execution is extremely fast when the processor is able to use BOA groups in memory. These optimized groups are generally much larger than a basic block and can contain many branches. The last instruction in each BOA group is a direct branch to the next group stored in memory. As long as a translation exists for the address pointed to by the final branch in a group, the new groups is fetched and executed. If the code does not exist, the machine re-enters interpretation mode to slowly execute PowerPC instructions and possibly build more BOA groups. Likewise, if a branch is encountered within the BOA group that is mispredicted, native execution must be halted, and a branch is taken back to the interpreter. It is therefore critical that BOA groups not only be as long as possible (to give ample opportunity for code optimization), but also that they be as high quality as possible (to avoid calls back to the interpreter). 2.1 Group Formation BOA instruction groups are formed along a single path after interpreting the entry point of a PowerPC instruction sequence 15 times. In the PowerPC interpretation process, we decided to only follow single paths of execution as to support very high-frequency hardware (described in Section 3). This single path approach is somewhat similar to that used by DIF [2], except that DIF requires special hardware to form groups and does not interpret PowerPC groups. DAISY [4, 5] by contrast forms groups from operations along multiple paths, and thus requires somewhat more time-consuming group formation and scheduling heuristics. During BOA’s interpretation phase, statistics are kept on the number of times each conditional branch is executed as well as on the total number of times it is taken, thus allowing a dynamic assessment of the probability the branch is taken. Similar information is also kept about the targets of register branches. As noted, once the group entry has been seen 15 times, the code starting at the entry point is assembled into a PowerPC group, and translated into BOA instruction groups for efficient execution on the underlying hardware. At each conditional branch point, the most likely path is followed. For efficient execution, it is best if the translated path always falls through at conditional branch instructions. In some cases, this requires that the sense of the branch be changed. As each conditional branch is reached during the translation, the probability of reaching this point from the start of the group decreases. When the probability goes below a threshold value, the group is terminated. For example, if a particular branch goes less than 12 times in its more likely direction, we might decide to terminate the group. Later in the paper, we refer this as “bias-12”, i.e. the branch must go in one direction at least 80% (12 of 15) of the time. Since we are applying this strategy in a runtime environment, quick group formation is essential, even if it can produce sub-optimal results. For example, consider the following PowerPC code: