## 4 Simulation Results and Discussion  
In the simulations, we set a 5000 m \(\times\) 5000 m target area and a 1500 m \(\times\) 1500 m no-fly zone. \(M=10\) buoys with a data volume of 10 Mbits and a maximum transmit power \(P_{\max}=24\) dBm were randomly distributed in the target area. The horizontal coordinates of the OBS were \((0,0)\) m. The initial horizontal coordinates of the UAVs with the flight height \(H=100\) m were \((0,0)\) m, \((0,2500)\) m, and \((2500,0)\) m, respectively. The maximum flight velocity and angle of the UAVs were \(V_{\max}=50\) m/s and \(\psi_{u,k}\in[0,2\pi]\), respectively. The transmit power of each UAV was \(P_{u,0,k}=0.1\) W. The minimum safety distance between the UAVs was \(d_{\min}=50\) m. The time slot duration was \(\delta=1\) s. For the MAHPPO algorithm, we set the length of the episode \(Ep_{\max}=18,000\), \(K^{\max}=250\). The actor network and the critic network both had three hidden layers with \((256,128,64)\) neurons. The learning rates of the actor network and the critic network were \(beta_{a}=0.0001\) and \(beta_{b}=0.0003\), respectively. The discount factor was \(\gamma=0.99\). The experience replay buffer size \(D\), mini-batch size \(D^{\prime}\), and sample reuse time \(N\) were set to 1024, 256, and 8, respectively. The hyperparameters were \(\Delta=0.2\) and \(\iota=0.01\). Moreover, our simulation was based on Python 3.8 with the Pytorch package. Other simulation parameters are shown in Table 2.  
Furthermore, we compared the proposed SU-MAHPPO algorithm with three baseline algorithms, which are listed as follows: \begin{table}
\begin{tabular}{l l} \hline
**Parameter** & **Value** \\ \hline environment parameter, \(a\), \(b\) & \(9.61,0.16\) \\ excessive path loss, \(\xi_{\text{LoS}}\), \(\xi_{\text{NLLoS}}\) & \(1,20\) \\ channel parameter, \(\mu_{u,\iota,k}\) & \(1\) \\ wavelength, \(\lambda\) & \(0.15\) m \\ noise power, \(\sigma^{2}\) & \(-104\) dBm \\ SNR threshold, \(\overline{\xi}_{\text{co}}\), \(\overline{\xi}_{\text{off}}\) & \(8\) dB, \(2\) dB \\ energy threshold, \(\overline{E}_{\text{u}}\), \(\overline{E}_{\text{m}}\) & \(150\) kJ, \(1.25\) J \\ tip speed of the rotor blade, \(U_{\text{tip}}\) & \(120\) \\ fuselage drag ratio, \(d_{0}\) & \(0.6\) \\ air density, \(\rho\) & \(1.225\) \\ rotor solidity, \(l\) & \(0.05\) \\ blade angular velocity, \(f\) & \(0.503\) \\ blade profile power and induced power, \(P_{o}\), \(P_{i}\) & \(79.86\) W, \(0.99\) W \\ LoS and NLoS link factor, \(v^{\text{LoS}}\), \(v^{\text{NLoS}}\) & \(2,2\) \\ \hline \end{tabular}
\end{table} 
Table 2: Simulation parameters. * SU-MAPDQN algorithm: This algorithm refers to using the multi-agent PDQN (MAPDQN) algorithm to replace the MAHPPPO algorithm in the algorithm proposed in our paper. * SU-MAPPO algorithm: This algorithm refers to using the multi-agent PPO (MAPPO) algorithm to replace the MAHPPPO algorithm in the algorithm proposed in this paper. Furthermore, the continuous action space of the UAVs is discretized in this algorithm. The flight angle of UAV \(u\) at time slot \(k\) is expressed in four directions: up, down, left, and right, i.e., \(\psi_{u,k}\in\{0,\pi/2,\pi,3\pi/2\}\). The flight velocity of the UAV at time slot \(k\) is expressed as \(V_{u,k}\in\{0,10,20,30,40,50\}\) m/s. * MAHPPPO algorithm: In this algorithm, the association relationships of the buoy-UAV and UAV-OBS are optimized. Hence, the action space of each agent of this algorithm is denoted as \(A_{u,k}=\{\mathcal{I},\psi_{u,k},V_{u,k},P_{u,k}\}\). The state space is denoted as \(\mathcal{S}_{k}=\left\{\kappa_{k},\kappa_{k}^{\mathrm{a}},c_{k}^{\mathrm{buoy} },E_{k}^{\mathrm{buoy}},x_{k},y_{k},c_{k}^{\mathrm{uav}},E_{k}^{\mathrm{uav}},B_{k}\right\}\), where \(\kappa_{k}^{\mathrm{a}}\) represents the discrete actions of all agents. The MAHPPPO algorithm is shown in Algorithm 5. ```
1:\(/*\) In Algorithm 1 */
2: Perform Lines 1-6 of Algorithm 1. 3: Obtain \(\kappa_{k}^{\mathrm{a}}\) according to the discrete actions of \(A_{k}\).