# Energy Saving Strategy of UAV in MEC Based on Deep Reinforcement Learning
###### Abstract  
Unmanned aerial vehicles (UAVs) have the characteristics of portability, safety, and strong adaptability. In the case of a maritime disaster, they can be used for personnel search and rescue, real-time monitoring, and disaster assessment. However, the power, computing power, and other resources of UAVs are often limited. Therefore, this paper combines a UAV and mobile edge computing (MEC), and designs a deep reinforcement learning-based online task offloading (DOTO) algorithm. The algorithm can obtain an online offloading strategy that maximizes the residual energy of the UAV by jointly optimizing the UAV's time and communication resources. The DOTO algorithm adopts time division multiple access (TDMA) to offload and schedule the UAV computing task, integrates wireless power transfer (WPT) to supply power to the UAV, calculates the residual energy corresponding to the offloading action through the convex optimization method, and uses an adaptive \(K\) method to reduce the computational complexity of the algorithm. The simulation results show that the DOTO algorithm proposed in this paper for the energy-saving goal of maximizing the residual energy of UAVs in MEC can provide the UAV with an online task offloading strategy that is superior to other traditional benchmark schemes. In particular, when an individual UAV exits the system due to insufficient power or failure, or a new UAV is connected to the system, it can perform timely and automatic adjustment without manual participation, and has good stability and adaptability. U 202  
or binary offloading methods [11]. Partial offloading [12] means that the computing data can be split into two parts: one part is executed locally and the other part is offloaded to the edge server for execution. Binary offloading [13] is suitable for the case where the computing task cannot be split. The whole task is either executed locally or offloaded to the edge server for execution. On the basis of reducing the energy consumption of a UAV itself, by incorporating radio frequency (RF)-based [14] wireless power transfer (WPT) [15] into MEC, it can continuously provide energy for the UAV. WPT uses an energy transmitter to wirelessly broadcast energy to the UAV [16; 17]. The UAV can use the energy for computing or offloading, and the residual energy can also be converted into equipment power, thereby extending the working time of the equipment. In the scenario of multiple UAVs, this is generally achieved through the joint optimization of the UAV computing task offloading decision, offloading time and other resources, which is transformed into a mixed integer programming (MIP) problem for a solution [18; 19; 20; 21; 22; 23]. The algorithms used mainly include the coordinate descent method, heuristic algorithm, convex optimization method and convex relaxation technique. However, in some cases, the computational complexity of these algorithms may increase rapidly, which may affect the decision-making effect. Moreover, traditional offline offloading strategies are not suitable for fast fading channels. Therefore, they cannot meet the requirements of ultra-low delay. Deep reinforcement learning [24; 25] is suitable for dealing with variable state space and high-dimensional data, and has a strong fitting effect and learning ability. It can obtain the online offloading strategy according to the actual wireless channel environment changes. The online algorithm is practical because the decision of each time block is made under the condition of a random channel and unknown computing data [26]. Therefore, this paper proposes a deep reinforcement learning-based online task offloading (DOTO) algorithm that can prolong the working time of UAVs. The main contributions of this paper are as follows:  
1. The MEC system that we consider consists of a multi-antenna AP with an edge server placed on the ship and multiple UAVs with a single antenna. The AP can provide wireless energy for the UAV and can also be used to receive UAV computing tasks.