November 10, 2020 was in many ways a landmark event in the microprocessor industry: Apple unveiled its new Mac Mini, the main feature of which was the new M1 chip, developed in-house. It is not an exaggeration to say that this processor is a landmark achievement for the ARM ecosystem: finally an ARM architecture chip whose performance surpassed x86 architecture chips from competitors such as Intel, a niche that had been dominated for decades. But the main interest for us is not the M1 processor itself, but the Rosetta 2 binary translation technology. This allows the user to run legacy x86 software that has not been migrated to the ARM architecture. Apple has a lot of experience in developing binary translation solutions and is a recognized leader in this area. The first version of the Rosetta binary translator appeared in 2006 were it aided Apple in the transition from PowerPC to x86 architecture. Although this time platforms were different from those of 2006, it was obvious that all the experience that Apple engineers had accumulated over the years, was not lost, but used to develop the next version - Rosetta 2.  We were keen to compare this new solution from Apple, a similar product Huawei ExaGear (with its lineage from Eltechs ExaGear) developed by our team. At the same time, we evaluated the performance of binary translation from x86 to Arm provided by Microsoft (part of MS Windows 10 for Arm devices) on the Huawei MateBook E laptop. At present, these are the only other x86 to Arm binary translation solution that we are aware of on the open market. Since all solutions were originally created for different operating systems (Huawei ExaGear for Linux, Apple Rosetta 2 for MacOS, and Microsoft binary translator for MS Windows), we had to find an appropriate comparison method, as it was impossible to execute them under the same conditions. We chose a “translation efficiency” metric, which is the ratio between the native version of the binary and a version of the binary for the guest architecture executed using the binary translator. For our tests the native target platform is Arm and the guest platform is x86. In other words, we compared how many percent of the native indicator the benchmark execution achieves in binary translation mode. For Geekbench in which a higher score is better, this ratio is: translated score divided by native score. For Spec in which a lower execution time is better, this ratio is: the native execution time divided by the translated execution time. The same metric was used by experts from the Anandtech website, who published a review article about the Apple M1, where performance figures were also given for Rosetta 2. All measurements are for a single thread, we are primarily interested in the performance of the translated code. That is, what percentage of performance is lost compared to native code. So, let us start by comparing Huawei ExaGear with Apple Rosetta 2 on Apple MacBook Pro (M1). The Rosetta 2 tests run in the native MacOS BigSur 11.1, the ExaGear tests run in a virtual machine with Linux kernel 5.4.1. Geekbench 5.4.1 (points, higher is better ): Bench   name ARM64 MacOS Rosetta 2 (x86) Rosetta 2 efficiency ARM64 Linux(VM) Exagear (x86) Exagear efficiency AES-XTS 2769 1720 62.1% 2703 1823 67.4% Text   Compression 1502 1319 87.8% 1438 1349 93.8% Image   Compression 1356 1056 77.9% 1335 1230 92.1% Navigation 1716 1678 97.8% 1717 1605 93.5% HTML5 1642 1066 64.9% 1717 1302 75.8% SQLite 1400 1000 71.4% 1328 1229 92.6% PDF   Rendering 1600 1324 82.8% 1738 1464 84.2% Text   Rendering 1778 1290 72.6% 1726 1480 85.8% Clang 1661 1244 74.9% 1738 1335 76.8% Camera 1612 1278 79.3% 1464 1148 78.4% N-Body   Physics 1789 1503 84.0% 1830 1616 88.3% Rigid   Body Physics 1772 1352 76.3% 1690 1176 69.6% Gaussian   Blur 1407 1251 88.9% 1435 1297 90.4% Face Detection 2215 1500 67.7% 2216 1632 73.7% Horizon   Detection 1964 1268 64.6% 1879 1386 73.8% Image   Inpainting 3214 2893 90.0% 3345 2903 86.8% HDR 2486 2250 90.5% 2761 2690 97.4% Ray   Tracing 2553 1970 77.2% 2055 1992 96.9% Structure   from Motion 1406 1068 76.0% 1507 1150 76.3% Speech   Recognition 1601 1371 85.6% 1485 1355 91.3% Machine   Learning 1243 713 57.4% 1229 727 59.2% GeoMean   76.9%   82.4% As we can see, Huawei ExaGear has a better average performance of 82.4% in comparison to Rosetta 2’s 76.9%, and ExaGear loses only in 4 tests out of 21: Navigation, Camera, Rigid Body Physics, and Image Inpainting.  Here we need to make a small, but quite interesting digression. A closer examination of the M1 processor reveals that despite the fact that it was officially released only in November, meaning that the internal design and instruction set was finalized long before, likely in 2019. Despite this it contains support for the Arm V8.7 architecture extension, not published until Autumn 2020. A significant portion of this extension aims to simplify and improve the performance of some operations common to binary translation for an x86 guest architecture. That is, Apple was developing a processor with an extension that was not official at the time. Moreover, a close look at earlier extensions reveals that ArmV8.5 and ArmV8.4 also included operations to support binary translation. This suggests that Apple has been working in close cooperation with Arm for quite some time, pursuing hardware support for their binary translation solution. Apple Rosetta 2 leverages all these features and therefore has a definite advantage over Huawei ExaGear, which does not exploit these extensions.  We believe this is why Apple Rosetta 2 performs better in these four benchmarks. But nevertheless, Huawei ExaGear shows better results in other tests, despite being at somewhat of a disadvantage for not using these advanced architecture features. It is also worth noting that the performance figures in native Arm-mode for MacOS and Linux are generally quite close, which confirms the general correctness of our approach of comparing the performance of binary translators.  Next, let us compare performance on SpecCPU2006 and SpecCPU2017. The execution time of each subtest is measured in seconds. Benchmarks written in Fortran were excluded. SpecCPU2006 (in seconds, lower is better) Compiler: clang 11.0 -O3 –flto INT tests ARM64 MacOS Rosetta 2 (x86) Rosetta 2 efficiency ARM64 Linux(VM) ExaGear (x86) ExaGear efficiency 400.perlbench 157 218 72.0% 145 166 87.3% 401.bzip2 248 326 76.1% 247 282 87.6% 429.mcf 106 118 89.8% 129 123 104.9% 445.gobmk 178 198 89.9% 183 193 94.8% 456.hmmer 159 170 93.5% 164 142 115.5% 458.sjeng 246 300 82.0% 253 281 90.0% 462.libquantum 94 107 87.9% 101 128 78.9% 464.h264ref 203 328 61.9% 201 271 74.2% 471.omnetpp 146 179 81.6% 173 193 89.6% 473.astar 184 203 90.6% 196 201 97.5% 483.xalancbmk 82 104 78.8% 96 112 85.7% GeoMean  INT   81.7%   90.8%        FP  tests       433.milc 95 130 73.1% 98 127 77.2% 444.namd 139 172 80.8% 139 164 84.8% 447.dealII 108 117 92.3% 115 151 76.2% 450.soplex 91 104 87.5% 95 107 88.8% 453.povray 60 78 76.9% 52 65 80.0% 470.lbm 113 119 95.0% 114 99 115.2% 482.sphinx3 194 207 93.7% 195 215 90.7% GeoMean  FP   85.2%   86.7% SpecCPU2017 (in seconds, lower is better) Compiler: clang 11.0 -O3 –flto INT   tests ARM64 MacOS Rosetta 2 (x86) Rosetta 2 efficiency ARM64 Linux(VM) ExaGear (x86) ExaGear efficiency 500.perlbench_r 216 282 76.6% 210 241 87.1% 502.gcc_r    125 164 76.2% 125 153 81.7% 505.mcf_r 202 235 86.0% 217 231 93.9% 520.omnetpp_r 277 360 76.9% 295 324 91.0% 523.xalancbmk_r 166 204 81.4% 189 197 95.9% 525.x264_r 154 176 87.5% 161 189 85.2% 531.deepsjeng_r 175 221 79.2% 190 197 96.4% 541.leela_r 282 294 95.9% 285 277 102.9% 557.xz_r 275 319 86.2% 307 335 91.6% GeoMean   INT   82.7%   91.6%        FP   tests       508.namd_r 111 134 82.8% 111 133 83.5% 510.parest_r 308 331 93.1% 304 336 90.5% 511.povray_r 211 320 65.9% 196 253 77.5% 519.lbm_r 121 153 79.1% 127 142 89.4% 526.blender_r 149 179 83.2% 163 175 93.1% 538.imagick_r 210 345 60.9% 227 271 83.8% 544.nab_r 156 186 83.9% 150 174 86.2% GeoMean   FP   77.7%   86.1% As we can see, the results from SpecCPU2006 and SpecCPU2017 follow that of Geekbench: on average, Huawei ExaGear beats Apple Rosetta 2, although in a few subtests Rosetta 2 is able to outperform ExaGear.  Next, we will compare the performance of Huawei ExaGear against the binary translator from Microsoft using a Huawei MateBook E. ExaGear runs in WSL environment, and Microsoft’s translator runs natively under Windows 10. When running ExaGear within a WSL environment, there is an issue in that the timers we require for profiling are not available. This prevented use of full code optimization by ExaGear. We estimate that this diminishes performance by around 10-20% depending on the benchmark. Geekbench 5.4.1 (points, higher is better): Bench   name ARM64 Windows MS BT (x86) MS BT efficiency ARM64 WSL ExaGear (x86) ExaGear efficiency AES-XTS 872 437 50.1% 892 437 49.0% Text   Compression 514 381 74.1% 518 451 87.1% Image   Compression 577 328 56.8% 606 433 71.5% Navigation 402 393 97.8% 522 502 96.2% HTML5 517 224 43.3% 522 371 71.1% SQLite 534 240 44.9% 565 412 72.9% PDF   Rendering 515 253 49.1% 574 462 80.5% Text   Rendering 530 264 49.8% 544 430 79.0% Clang 485 187 38.6% 601 405 67.4% Camera 437 221 50.6% 479 308 64.3% N-Body   Physics 390 253 64.9% 390 317 81.3% Rigid   Body Physics 634 299 47.2% 717 485 67.6% Gaussian   Blur 279 217 77.8% 282 238 84.4% Face   Detection 598 301 50.3% 657 387 58.9% Horizon   Detection 484 235 48.6% 542 331 61.1% Image   Inpainting 580 419 72.2% 775 504 65.0% HDR 637 481 75.5% 1035 843 81.4% Ray   Tracing 758 296 39.1% 762 513 67.3% Structure   from Motion 379 203 53.6% 457 289 63.2% Speech   Recognition 346 283 81.8% 355 312 87.9% Machine   Learning 252 108 42.9% 240 128 53.3% GeoMean   55.6%   70.9% SpecCPU2017 (in seconds, lower is better) Compiler: clang 11.0 -O3 -flto INT   tests ARM64 Windows MS BT (x86) MS BT efficiency ARM64 WSL ExaGear (x86) ExaGear efficiency 500.perlbench_r 824 1526 54.0% 800 1042 76.8% 502.gcc_r 716 1119 64.0% 690 820 84.1% 505.mcf_r 781 1069 73.1% 792 992 79.8% 520.omnetpp_r 1280 2025 63.2% 1186 1414 83.9% 525.x264_r 460 771 59.7% 481 638 75.4% 531.deepsjeng_r 507 711 71.3% 524 629 83.3% 541.leela_r 564 897 62.9% 547 652 83.9% 557.xz_r 733 938 78.1% 733 827 88.6% GeoMean  INT   65.4%   81.9%        FP   tests       508.namd_r 423 698 60.6% 429 619 69.3% 510.parest_r 1006 1274 79.0% 970 1179 82.3% 511.povray_r 808 1540 52.5% 761 1027 74.1% 519.lbm_r 428 767 55.8% 429 549 78.1% 526.blender_r 474 688 68.9% 484 716 67.6% 538.imagick_r 560 1075 52.1% 566 847 66.8% 544.nab_r 522 1050 49.7% 517 617 83.8% GeoMean  FP   59.0%   74.3% Unfortunately, we were unable to build SpecCPU2006 for MS Windows. On the other hand, SpecCPU2017 built successfully and results were able to be collected. As seen when comparing against Rosetta 2, the results of SpecCPU2006 and SpecCPU2017 are broadly similar. Conclusion Apple’s engineers have not only produced an outstanding and game changing processor. Furthermore, they have equipped this with a performance x86 binary translator. Nevertheless, our tests show that Huawei ExaGear confidently outperforms Apple’s Rosetta 2. Microsoft’s solution is shown to be inferior to both ExaGear and although we can’t directly compare, also to Rosetta 2. This is expected given Microsoft’s lack of expertise and experience in developing binary translators.