## 4 Experiment and Results
### Numerical Experimental Results  
#### 4.2.1 Parallel Efficiency  
In order to test the acceleration effect under different computations, this paper selects the grid size of 50 million, 100 million, and 200 million; the maximum number of MPEs is 2048, and the maximum total number of cores is 133,120. The acceleration effects of only the MPEs and the MPEs plus CPEs mode are counted separately. Figure 8 shows the speedup and parallel efficiency for only MPEs operations at three problem sizes. Figure 9 shows the percentage of communication time spent on only MPEs for the three problem sizes. As can be seen in Figure 8, both the speedup ratio and the efficiency decrease as the number of computational cores increases. This is because, for the same problem size, as the number of computational cores increases, the number of computational grids on each core decreases. For one computational core, if we assume the size of the computation area to be \(NX\times NY\), the number of grid points that the current computation core needs to send data to the adjacent computation core is \(NX\times NY-(NX-2)\times(NY-2)\), and the ratio of the grid points that need to send data to the total grid point of the calculated core is R, where  
\[R=\frac{NX\times NY-(NX-2)\times(NY-2)}{NX\times NY}=\frac{2(NX+NY+2)}{NX\times NY}. \tag{3}\]  
It can be seen from Equation (3) that, as the number of computational cores increases, both NX and NY decrease, and the ratio R increases. That is, the percentage of communication time in total time increases (Figure 9). Therefore, parallel efficiency will gradually decline. Similarly, for the same number of computational cores, as the problem size increases, the amount of computational grid allocated to each computational core increases, and the percentage of communication time as a percentage of total time decreases (Figure 9); therefore, the acceleration ratio and parallelism efficiency will increase (Figure 8). When the problem size is 200 million, the parallel efficiency is over 94%. As the problem size is 200 million, only MPEs have the highest parallel efficiency. Therefore, this paper uses the 200 million problem size to test the acceleration effect of the CPEs, and tests up to 133,120 cores. Based on a single CG (1 MPE and 64 CPEs, a total of 65 cores), the number of CGs is increased gradually. The acceleration ratio and parallel efficiency are shown in Table 2 and in Figure 10a. The Mega Lattice Site Updates Per Second (MLUPS) is shown in Figure 10b. As the number of CGs increases, the parallel efficiency and MLUPS decrease, and the speedup and parallel efficiency also decrease compared with the only MPE mode. This is mainly attributed to the startup and shutdown of the CG operation thread group and the communication between the MPE and CPEs. However, when the number of CGs is 2048 (133,120 cores total), the efficiency can still reach more than 72%, and our simulation performs at 285,600 MLUPS on 2048 CGs, corresponding to 54.8% of the effective memory bandwidth performance. Figure 8: For only management processing elements (MPEs) at three problem sizes: (**a**) Acceleration ratio; (**b**) Parallel efficiency. Figure 9: Communication time as a percentage of total time for only MPEs at three problem sizes. \begin{table}
\begin{tabular}{c c c c c c c c c c c c c} \hline \hline
**CGs** & **1** & **2** & **4** & **8** & **16** & **32** & **64** & **128** & **256** & **512** & **1024** & **2048** \\ \hline Cores & 65 & 130 & 260 & 520 & 1040 & 2080 & 4160 & 8320 & 16640 & 33280 & 66560 & 133120 \\ Theoretical & 1 & 2 & 4 & 8 & 16 & 32 & 64 & 128 & 256 & 512 & 1024 & 2048 \\ acceleration ratio & 1 & 1.9983096 & 3.9549608 & 7.84065056 & 15.6233262 & 30.9049656 & 60.5670289 & 117.7244635 & 228.170942 & 440.4674223 & 841.5104023 & 1492.812774 \\ Parallel efficiency (\%) & 100 & 99.915482 & 98.874021 & 98.0081321 & 97.645789 & 96.5780176 & 94.6359827 & 91.9722371 & 89.1292742 & 86.02879342 & 82.17875022 & 72.89124874 \\ \hline \hline \end{tabular}
\end{table}