## 3 Three-Dimensional FDTD Parallel Algorithm on the CUDA Platform
### Data Transmission Scheme Based on NCCL
#### 3.2.2 GPUs Communicate Directly Based on NCCL
Magnetic field iteration. Repeat steps (a-d) until the time step iteration is completed. The flow chart of the FDTD algorithm based on clustered GPUs is shown in Figure 6.  
The above flow chart demonstrates a computing mode where the CPU dominates and the GPU collaborates. The program's flowchart visually depicts the key steps of this process. 1. The system starts computing tasks by creating CPU processes that match the number of available GPUs using OpenMPI technology. At this stage, the main process of the CPU plays a crucial role, and is responsible for initializing all the variable information required throughout the entire computation process. This variable information is essential for the subsequent calculations, as it ensures the accuracy and consistency of the calculations. 2. The CPU main process performs load balancing and partitioning of the computing domain based on the number of processes. This step aims to optimize the allocation of computing resources, ensuring that each CPU process can handle an appropriate amount of data, thereby improving the overall computing efficiency. A segmented computing domain is allocated to each CPU process to prepare for the subsequent  
Figure 6: The flow chart of the FDTD algorithm based on clustered GPUs. computing work. The model data for each CPU process are efficiently copied to the corresponding GPU global memory. 3. The GPUs fully utilize their powerful computing power by mapping each Yee cell onto a thread and carrying out the computing work in parallel. This calculation mode not only greatly improves the calculation speed, but also ensures the accuracy of the calculation. At each time step, the interaction of electric field data and magnetic field data between the GPUs is achieved through the NCCL technology. The NCCL technology provides an efficient and reliable solution for data communication between GPUs, ensuring the accuracy and consistency of data. 4. Finally, the collection of the calculation results is the responsibility of OpenMPI. OpenMPI allows for direct access to each GPU storage space, enabling the efficient collection of the computation results. From the above flowchart, it can be seen that the CPU only participates in task allocation and data collection, and only the GPUs participate in the time iteration process. Compared with the traditional mode of forwarding data through the CPU, the NCCL can directly transmit data between multiple GPU devices without relying on the CPU for data transfer. This method not only greatly reduces the delays from data transmission, but also optimizes the process according to the communication topology structure, fully utilizes the hardware resources of the NVIDIA GPUs, and improves the communication efficiency and computing performance. Therefore, the NCCL plays an important role in multi-GPU computing environments, especially in scenarios that require efficient and large-scale parallel computing, greatly improving the overall computing and communication performance. This paper uses speedup and parallel efficiency to evaluate the parallel effect of the FDTD algorithm of distributed cluster GPUs. The speedup is defined as  
\[Speedup=t_{CPU}/t_{GPU} \tag{3}\]  
where \(t_{CPU}\) is the CPU calculation time and \(t_{GPU}\) is the GPU calculation time. Parallel efficiency is defined as  
\[Efficiency=t_{GPU}/(N_{GPU}\cdot t_{GPU})\times 100\% \tag{4}\]  
where \(t_{GPU1}\) is the calculation time of a single GPU and \(N_{GPU}\) is the number of GPUs used.