## 3 Joint Optimization Algorithm Design
### Joint Optimization Algorithm
In lines 11-24 of Algorithm 3, the UAV whose action does not satisfy the constraints is transformed to \(\chi^{\text{new}}_{u,k}=0\), and its number is formed into a new set \(\kappa^{\prime}_{k}\). In lines 9-10, we update \(\mathcal{U}^{\text{co}}\) by \(\mathcal{U}^{\text{co}}\cap\kappa^{\prime}_{k}\). Then, we perform Algorithm 2 to judge \(\mathcal{U}^{\text{co}}\). Some UAVs in \(\kappa^{\prime}_{k}\) may not satisfy the constraints of both data collection and data offloading after two judgments. Hence, in lines 32-34 of Algorithm 2, we set \(\chi^{*}{}_{u,k}=2\) if \(\chi_{u,k}=0\) and \(\chi^{\text{new}}_{k}=0\). In lines 12-13, for UAVs that have only experienced the judgment of Algorithm 2 once, but did not satisfy the constraints, we perform Algorithm 3 to make the second judgment. In lines 22-24 of Algorithm 3, we set \(\chi^{*}{}_{u,k}=2\) if \(\chi_{u,k}=1\) and \(\chi^{\text{new}}_{k}=1\). Hence, there may be some UAVs that neither collect, nor offload data in a certain time slot. After the above three judgments, the association relationship of all UAVs can be determined. The above procedure is looped until done is True. ```
1:Input \(g_{\text{of}}\), \(V_{u,k},\forall u\in\mathcal{U}\), \(\mathcal{U}^{\text{of}}\), \(\chi_{k}^{\text{new}}\), \(\kappa_{k}\), \(P_{u,0,k},\forall u\in\mathcal{U}\), and current channel gain \(h_{u,0,k},\forall u\in\mathcal{U}\). 2:repeat
3: Obtain the SNR of UAV-OBS \(g_{u,0,k}\) according to \(P_{u,0,k}\). 4: Obtain the maximum transmission rate of UAV \(R_{\max u,0,k}\) according to \(B\). 5: Obtain the energy consumption of UAVs \(E_{u}^{\text{total}}\) according to \(V_{u},k_{r}\). 6:if\(g_{u,0,k}\), \(E_{u}^{\text{total}}\), and \(R_{\max u,0,k}\) satisfy C4, C6, and C15, respectively then
7: Set \(\chi_{k}^{\text{new}}=1\), and \(\kappa_{u,k}=0\). 8:else
9: Set \(\chi_{k}^{\text{new}}=0\)
10:endif
11:if\(R_{\max u,0,k}\) does not satisfy C15, and the data of all buoys are fully collected then
12:if\(g_{u,0,k}\) and \(E_{u}^{\text{total}}\) satisfy C4 and C6 then
13: Set \(\chi_{k}^{\text{new}}=1\) and \(\kappa_{u,k}=0\). 14:else
15: Set \(\chi_{k}^{\text{new}}=0\). 16:endif
17:endif
18:\(u=u+1\)
19:until\(u=U^{\text{of}}\)
20: Let \(\kappa_{k}^{\prime}\) indicate the set of UAVs that are first transformed. 21:if\(\chi_{u,k}=0\) and \(\chi_{k}^{\text{new}}=0\)then
22: Set \(\chi^{*}_{u,k}=2\) and \(\kappa_{k}=-1\). 23: Add the number of UAVs \(u\) to \(\kappa_{k}^{\prime}\). 24:endif
25:return\(\chi_{u,k}^{\text{new}},\kappa_{k},\kappa_{k}^{\prime}\). ```  
**Algorithm 3** UAV-OBS association algorithm (UAA). ```
1:The initial positions of UAVs \(q_{u,1},\forall u\in\mathcal{U}\), the positions of buoys \(x_{m},y_{m},\forall m\in\mathcal{M}\), and the position of OSB \(x_{0},y_{0}\). 2:\(A,P,V,\psi\). 3:\(/*\) In Algorithm 1 */
4:for episode = 1 to \(Ep_{\max}\)do
5:while done is not True do
6: Perform _Lines_ 6-9 of Algorithm 1. 7: Obtain \(\chi_{k}\) according to \(A_{u,k}\). 8: Obtain initial \(\mathcal{U}^{\text{co}}\) and \(\mathcal{U}^{\text{of}}\) according to \(\chi_{k}\). 9: Let \(\chi_{k}^{\text{new}}=\chi_{k}\) and \(\kappa_{u,k}=-1,\forall u\in\mathcal{U}\). 10: Update \(\chi_{k}^{\text{new}}\), \(\kappa_{k}\), and \(\kappa_{k}^{\prime}\) by performing Algorithm 3. 11:if\(\kappa_{k}^{\prime}\) is not null or \(\chi_{u,k}^{\text{new}}=0\) exists in \(\chi_{k}^{\text{new}}\)then
12:\(\mathcal{U}^{\text{co}}\leftarrow\mathcal{U}^{\text{co}}\cap\kappa_{k}^{\prime}\). 13: Update \(\chi_{u,k}^{\text{new}}\), \(\kappa_{k}\), and \(\kappa_{k}^{\prime}\) by performing Algorithm 2. 14:if\(\kappa_{k}^{\prime}\) is not null then
15:\(\mathcal{U}^{\text{of}}\leftarrow\kappa_{k}^{\prime}\). 16: Update \(\chi_{u,k}^{\text{new}}\), \(\kappa_{k}\), and \(\kappa_{k}^{\prime}\) by performing Algorithm 3. 17:endif
18:endif
19: Perform _Lines_ 10-26 of Algorithm 1. 20:endwhile
21:endfor
```  
**Algorithm 4** Joint optimization algorithm based on SAA, UAA, and MAHPPO (SU-MAHPPO).