## 4 Methodology
### Solution
When the learning rate of actor network \(\alpha_{a}\) is 0.00001 and the learning rate of critic network \(\alpha_{c}\) is 0.00002, the lower learning rate will lead to a slower neural network update rate, resulting in slower convergence. Therefore, when the learning rate of actor network \(\alpha_{a}\) and the learning rate of critic \(\alpha_{c}\) are 0.0001 and 0.0002, respectively, the performance is optimal. In Figure 4, we compare the effects of different discount factors \(\gamma\) on the convergence performance of the proposed algorithm. The results show that when the discount factor is 0.9, the performance of the trained algorithm is optimal. Because circumstances vary widely from time to time, data over the entire period are not fully representative of long-term behavior. A larger \(\gamma\) means that the neural network regards the data collected in the whole period as long-term data, resulting in poor generalization ability in different time periods. Therefore, an appropriate value of \(\gamma\) will improve the final performance of our trained strategy, and we set the discount factor \(\gamma\) to 0.9 in the following experiment. Figure 4: Convergence performance with different discount factors. Figure 3: Convergence performance of the DCMCE algorithm under different learning rates. The larger the clipping function factor \(\epsilon\) is, the greater the difference between the two output strategies. As shown in Figure 5, when the clipping function factor \(\epsilon\) is 0.2, both excellent exploration ability and stability are considered. ### Baseline

The simulation results are provided below, and the proposed algorithm is contrasted with three baselines to assess the performance and efficiency of the algorithm. * Random-offload (RO); some tasks are randomly selected within the UAV communication range to offload. * Local-offload (LO); all computing tasks are computed locally. Trust region policy optimization (TRPO): The output of TRPO is continuous action. Here, we output a continuous value for each user according to the method [9]. Like DCMCE, we select the top O users to associate with the UAV. \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter** & **Value** \\ \hline \(K\) & 50 \\ \(L\), \(H\), \(W\) & 100 m \\ \(P_{\epsilon}\) & 0.1 W \\ \(N\) & 80 \\ \(t_{fly}\) & 1 s \\ \(\bar{B}_{0}\) & 100 MHz \\ \(P_{NLOS}\) & 20 dB \\ \(E_{b}\) & 500 kJ \\ \(f_{i}(n)\) & 0.1–0.2 GHz \\ \(\theta^{max}\) & \(\pi/4\) \\ \(M_{\text{UAV}}\) & 9.65 kg \\ \(Z\) & 2500 \\ \(v_{\text{max}}\) & 30 m/s \\ \(a_{0}\) & \(-50\) dBm \\ \(P_{up,max}\) & 0.2 W \\ \(\rho\) & 1000 c/be/bit \\ \(T\) & 320 s \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Experimental parameters. Figure 5: Convergence performance with different clipping function factors. ### Experimental Results

In Figure 6, the optimal UAV trajectory for a cycle is shown, with \(M=20\) MDs randomly distributed throughout the target area. Each time slot includes a sample of a UAV's flight path. We choose some of the key points to stay represented as red stars for a better display of flight conditions. The larger the red star, the longer the UAV stays. The location of the UAV will be suitable for real-time requests. The UAV always flies directly to requests, making the system computationally efficient and spending more time in areas with high user density. Figure 7 depicts the speed distribution of the UAV in a cycle. The UAV flies faster at the beginning. It may be that the UAV quickly approaches the requested area and spends most of the time providing services for the request area to save energy. When there is no request in the current area, the UAV quickly flies to other request areas to provide services. As shown in Figure 8, the convergence performance of all algorithms is verified. It can be seen from Figure 8 that the DCMCE in this paper tends to be stable after the first 200 iterations and can achieve relatively good computational efficiency.