# Energy Saving Strategy of UAV in MEC Based on Deep Reinforcement Learning
## 2 Related Work  
MEC has the characteristics of strong real-time performance, ultra-high bandwidth and ultra-low delay [27]. It provides a solution to the problem of the limited computing and battery life of wireless devices such as UAVs. Thus, it is a very promising technology. By jointly optimizing communication and computing resources, the computing power and battery life of wireless devices can be effectively improved [18]. In the literature [19], scholars have jointly optimized communication resources, computing task offloading, CPU and other resources. The problem is solved by using the distributed convex optimization method and alternating direction multiplier method. In the literature [20], scholars use the coordinate descent method (CD) in a non-gradient optimization algorithm. The CD algorithm will fix the dimensions of other variables when looking for the optimal solution, and only search in the direction of one variable. It is a simple but very efficient algorithm. However, the disadvantage is that if there is a strong correlation between multiple variables, the search will be very slow, and the corresponding search results will be poor. In the literature [21], scholars proposed a heuristic algorithm with low complexity and adopted convex optimization technology for the resource allocation problem. However, in the process of offloading strategy optimization, with the increase in wireless devices, the search space grows exponentially, and the convergence time of the algorithm decreases significantly. In the literature [22], a heuristic search algorithm based on convex relaxation is studied, which makes it continuous from 0 to 1 by relaxing integer variables. In the literature [23], a positive semi-definite relaxation method is employed by adopting quadratic constrained quadratic optimization. However, the method based on convex relaxation needs many iterations to find the qualified local optimal solution. The search time is long, and it is not suitable for fast fading channels requiring ultra-low delay. In the wireless fading environment, the optimal offloading decision of the wirelessly powered MEC system will be affected [28]. Moreover, the traditional offline offloading decision is no longer applicable in practical scenarios, because it cannot make timely and correct uninstallation decisions. TDMA can divide time into small blocks wherein wireless channels do not interfere with each other, and, combined with MEC, can well overcome the effects of wireless fading [29]. In the literature [30], based on TDMA, scholars solved the problem of minimizing the total energy consumption of MEC by jointly optimizing the energy transmission beam, CPU frequency and computing task offload ratio of edge servers. In the literature [31], a two-stage algorithm and a three-stage alternative algorithm are proposed for solving the computation rate maximization problems in a UAV-enabled MEC wireless-powered system. At the same time, in order to continuously provide energy for wireless devices, RF-based WPT technology can be integrated into the MEC. WPT technology is simple, safe, highly adaptable to the environment, very convenient to use and widely used. In the literature [8], scholars studied the problem of extending the battery life of wireless devices through energy harvesting and resource allocation in the WPT-MEC system. Deep reinforcement learning can update the offloading strategy in real time according to wireless channel changes [32; 33]. An offloading strategy based on deep Q-network (DQN) was proposed in [34] to optimize computation, but DQN is not suitable for the case of too many wireless devices. In the literature [35], a cooperative multi-agent deep reinforcement learning framework is investigated. By jointly designing the trajectories, computation task allocation and communication resource management of UAVs, the sum of execution delay and energy consumption is reduced.