## 3 Three-Dimensional FDTD Parallel Algorithm on the CUDA Platform
### FDTD Calculation Domain Decomposition Scheme  
During the time-marching of an FDTD, the recursion of the electric and magnetic fields requires using the magnetic and electric fields from adjacent subdomains, thus necessitating data exchange at the boundaries between adjacent subdomains. The computational region of an FDTD can be divided into multiple subdomains in one, two, or three dimensions. The subdomain partitioning strategy affects the communication between adjacent  
Figure 2: Internal structure diagram of CUDA memory.  
processes, further influencing the computational efficiency. Generally, the principle of minimizing the communication volume between subdomains should be followed when partitioning subdomains. For the FDTD algorithm, reducing the cross-sectional area at the subdomain boundary is necessary, ensuring the least amount of communication between the subdomains.  
Based on the actual computational requirements and hardware resources, this paper adopts a two-dimensional (along the x- and y-directions) subdomain partitioning scheme for the computational domain. In this case, the data of the yoz cross-section are continuously distributed in the memory, while those of the xoz cross-section are discontinuous. Thus, it is efficient to exchange data between adjacent processes of the yoz cross-section, but the not for the data of the xoz cross-section. When exchanging the data of the xoz cross-section, by creating a temporary array and establishing the mapping relationship between the xoz cross-section and the temporary array, then the communication efficiency can be improved by exchanging the temporary arrays between adjacent processes. Figure 3 shows a schematic diagram of the computational domain cut into four subdomains along the x- and y-directions.  
Load balancing is also an important indicator for measuring the quality of region partitioning. Ensuring that each GPU is assigned a comparable computational task is necessary when dividing the computational domain. Otherwise, it will increase the synchronization waiting time and reduce the computational efficiency. To calculate the electric (magnetic) field of the cells at the subdomain boundary, it is necessary to know the magnetic (electric) field of the adjacent cells in the adjoining subdomain. Therefore, the current subdomain needs to receive data from the adjacent subdomain. We achieve data exchange between adjacent processes by setting up data buffers. In each direction, one data buffer is set for the first and last subdomains, while two data buffers are required for the middle subdomains. Therefore, while evenly distributing the computational tasks among the various computational subdomains, it is also necessary to ensure that the computational tasks assigned to the middle subdomains are less than those assigned to the first and last subdomains to provide load balancing and improve the computational efficiency. Assuming that the calculation area is divided solely in the x-direction, with Nx representing the number of grids in this direction, if the number of parallel processes is n, then the average number of grids allocated to each process in the x-direction is px = int (Nx/n), where 'int' denotes the integer part of the division. The remaining grids are rx = mod (Nx, n), where'mod' signifies the modulo operation. Since the main process is typically involved in more I/O operations, this article proposes allocating the remaining grids to processes with higher process numbers. For instance, when Nx = 206 and n = 4, the number of grids handled by processes 0, 1, 2, and 3 are Nx0 = 51, Nx1 = 51, Nx2 = 52, and Nx3 = 52, respectively. By adopting this approach, we can ensure a relatively balanced workload among the processes, eliminate the waiting time between them, and enhance the computational efficiency.  
Figure 3: FDTD calculation region division scheme.