## 2 Materials and Methods
### MADDPG Solves UAV Path Planning and Task Offloading Problem
UAV Communication Mechanism  
When using the MADDPG reinforcement learning algorithm to jointly optimize task offloading and UAV path, since there is no communication function between UAVs, each UAV can only make a joint decision on task offloading and UAV path planning based on its observed situation. Therefore, for a UAV, it is not aware of the situation of other UAVs covering mobile user devices. This may result in the problem of multiple UAVs covering the same mobile device or some user devices not being serviced by a UAV. To address this issue, we propose a joint optimization algorithm for task offloading and path planning that incorporates communication mechanisms for UAVs. The structure of the UAV communication mechanism is as shown in Figure 2, which specifically includes four parts: observation encoding, estimation inference, message sending, and decision-making. The observation encoding part uses the self-attention method to encode the local observations of the UAV because the self-attention method is permutation-invariant and order-invariant, which is beneficial for model expansion. Moreover, due to the weighted sum mechanism, multi-objective information can be encoded into a single feature. Scaled dot-product self-attention is used to transform the local observation \(O_{i}\) into keywords \(K_{i}\), queries \(Q_{i}\), and values \(V_{i}\) by three different neural networks. The calculation formula is as shown in (32). \[E_{i}=\text{softmax}(\frac{Q_{i}K_{i}^{T}}{\sqrt{d_{k}}})\odot V_{i} \tag{32}\]  
The encoded feature \(E_{i}\) of UAV \(i\) and the location information of other UAVs (e.g., \(\phi_{j}\), \(\phi_{k}\), \(\phi_{l}\)) are taken as inputs to the estimation and inference part, where \(\zeta_{i,j}\) represents the information observed by UAV \(j\) as inferred by UAV \(i\). Combining the information observed by UAV \(i\) itself with the user device information perceived by drone \(i\) to be observed by other drones,because the structures of the two sets of observation information are both one-dimensional vectors, the way of combination is to concatenate the two sets of one-dimensional vectors. That is, after the combination, the size of the observation space of drone \(i\) is expanded to the sum of the lengths of the two sets of vectors. The combined observation information can help drones better assess the situation of the mobile devices covered by other drones. Next, \(g^{*}_{i,j}\) is obtained by taking \(E_{i}\) and \(\zeta_{i,j}\) as the input of the target inference subpart, which can be represented as \(g^{*}_{i,j}=GI(E_{i},\zeta_{i,j})\). \(g^{*}_{i,j}\) is an m-dimensional vector, where \(m\) represents the number of user devices, and the vector represents the probabilities of UAV i inferring that each mobile user device is covered by UAV \(i\). The update of the observation estimation subnetwork parameters is achieved through the binary cross-entropy loss function, and the specific calculation is as shown in Formula (33). \[L^{\text{OE}}=-\frac{1}{N}{\sum_{i}{\sum_{j\
eq i}{\sum_{q}{\left[{c_{j,q}} \cdot log(c^{*}_{i,j,q})+(1-c_{j,q})\cdot log(1-c^{*}_{i,j,q})\right]}}}} \tag{33}\]  
Here, \(c^{*}_{i,j}\) is the distance relationship between all user devices and UAV \(j\) inferred by UAV \(i\). \(c_{j,q}\) represents the real coverage relationship between agent \(j\) and device \(q\). If agent \(j\) covers device \(q\) or UAV \(j\) is the closest agent to device \(q\), then \(c_{j,q}=1\); otherwise, \(c_{j,q}=0\). Similarly, the target inference subnetwork parameters are also updated by the binary cross-entropy loss function, as shown in Formula (33). Here, \(g^{*}_{i,j,q}\), represe nts the probability that UAV \(i\) predicts UAV \(j\) chooses target \(q\), and \(g_{j,q}\) is the actual situation in which UAV \(j\) covers the user device. When \(g_{j,q}=0\), there is only the second term in Formula (34); at this time, the closer the sub-network inference \(g^{*}_{i,j,q}\) is to 0, i.e., the closer it is to the true value, and the smaller the loss value.