## 4 Methodology
### Solution
The convergence rate of TRPO is lower than that of DCMCE, and the convergence results fluctuate greatly. This is because the offloading decision is a discrete action, but TRPO does not handle

Figure 6: The trajectories of the UAV. Figure 7: The speed of the UAV. discrete actions well. Our algorithm processes the two actions separately to achieve better results. As shown in Figure 9, the average computational efficiency of the LO scheme will fluctuate as the number of users increases, because the average computational efficiency of the LO scheme will change with the computing power of different users. The computational efficiency of the RO scheme is increasing. When the number of users increases, the distribution of users is dense. Although the association of users is blind, the computational efficiency increases. The TRPO algorithm first rises and then falls with the increase of users. The TRPO algorithm performs well when the action-space dimension is low. However, the action-space dimension increases due to the increase in users, and the continuous actions output by the algorithm cannot properly evaluate the user association decision. As an excellent reinforcement learning algorithm, DCMCE has better performance than other algorithms. This is because our scheme can find the optimal user association strategy, and combines the exploration mechanism of reinforcement learning, thereby obtaining the optimal control strategy. With an increase in devices, DCMCE will not show obvious performance degradation. Figure 8: The convergence performance of algorithms. Figure 9: Computational efficiency of schemes versus MDs. +++ ==ERROR: No output for this page==
## 6 Conclusions

To solve the problem of heavy load on a fixed-base station, we studied a computation offloading problem in a UAV-assisted MEC system in which a flying UAV provides services to MDs. We tended to optimize user associations and the UAV trajectory to maximize the computing efficiency of all MDs. To tackle the computation offloading problem, we propose a DCMCE algorithm based on DRL. Simulation results show that the proposed DCMCE algorithm performs better than the baseline algorithm. We will consider expanding the DCMCE algorithm to multi-UAV scenarios in subsequent work. Conceptualization, H.Q. and Z.Z. ; methodology, H.Q. and Z.Z. ; validation, H.Q. and Z.Z. ; formal analysis, H.Q. and Z.Z. ; writing--original draft, Z.Z. ; writing--review and editing, H.Q. and Z.Z. ; visualization, Z.Z. All authors have read and agreed to the published version of the manuscript. This research has been sponsored by Natural Science Foundation of Hunan Province (2022 JJ30755). Not applicable. The authors declare no conflict of interest. ++

* [PERSON] et al. (2021) [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON] Cooperative Data Sensing and Computation Offloading in UAV-assisted Crowdsensing with Multi-agent Deep Reinforcement Learning. _IEEE Trans. Netw. Sci. Eng._**2021**, \(9\), 3197-3211. [CrossRef]
* [PERSON] and [PERSON] (2019) [PERSON]; [PERSON] Energy-efficient mobile crowdsensing by unmanned vehicles: A sequential deep reinforcement learning approach. _IEEE Internet Things_**2019**, \(7\), 6312-6324. [CrossRef]
* [PERSON] et al. (2017) [PERSON]; [PERSON]; [PERSON] Mobile cloud computing with a UAV-mounted cloudlet: Optimal bit allocation for communication and computation. _IET Commun._**2017**, _11_, 969-974. [CrossRef]
* [PERSON] et al. (2018) [PERSON]; [PERSON]; [PERSON]; [PERSON] Computation Rate Maximization in UAV-Enabled Wireless-Powered Mobile-Edge Computing Systems. _IEEE J. Sel. Areas Commun._**2018**, _36_, 1927-1941. [CrossRef]
* [PERSON] et al. (2017) [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON] Proximal Policy Optimization Algorithms. _arXiv_**2017**, arXiv:1707.06347.