# Dynamic Computation Offloading with Deep Reinforcement Learning in Edge Network  
[PERSON]  
1 School of Information Engineering, China University of Geosciences (Beijing), Beijing 100083, China 1, [PERSON]  
1 School of Information Engineering, China University of Geosciences (Beijing), Beijing 100083, China 1, [PERSON]  
1 School of Information Engineering, China University of Geosciences (Beijing), Beijing 100083, China 1, [PERSON]  
1 School of Information Engineering, China University of Geosciences (Beijing), Beijing 100083, China 1,2,*  
algorithm is proposed to achieve energy-optimal computation offloading [8]. To minimize the overall offloading time, the authors designed a delay-dependent, priority-aware, task-offloading strategy in [9], which assigns priority to each task according to its deadline. These papers concentrate on partial offloading strategies or the joint optimization of offloading decisions and resource allocation [10; 11; 12; 13; 14]. However, most of the related research only focuses on task offloading to tackle the problems of service response latency and the limited energy consumption of IoT devices, ignoring the fact that the users' service requests are highly dynamic in practice. Edge servers may not configure the services required by the task, which will constrain the task offloading strategy. Service migration from the cloud to the network's edge is introduced in [15] to resolve the conflict between the diversity of service requests and the limited number of services on the edge server. However, the communication time and energy cost regarding communication between cloud servers and edge servers are high. As a result, the frequent migration of services from the cloud to edge servers will lead to an increased user response latency and increased energy consumption by IoT devices. One possible solution to this problem is to migrate services between edge servers, bringing available services closer to the user devices, and thereby reducing the service migration's extra cost [16]. To achieve the best balance between QoS and migration costs, and bring services closer to users, when and where [17] to migrate services must be determined. These works aim to achieve low-latency service migration and seamless computation in a dynamic traffic environment in [18; 19; 20; 21]. An autonomous bandwidth allocation approach is proposed to support low-latency service migration [22]. The authors used an algorithm based on decision theory to find an optimal service migration path [23]. Most of the existing works concentrated on finding an optimal algorithm to lower the delay in task offloading or service migration. However, the decision time of the algorithm itself is ignored. For delay-sensitive applications and the unstable network environments, the complexity and robustness of decision-making algorithms are key to fast decision-making [24]. In general, the problem of task-offloading and service migration can be expressed as a Markov Decision Process (MDP) [25]. The use of reinforcement learning algorithms to find the optimal MDP strategy [26] is a great choice. Among the various reinforcement learning algorithms, the Q-learning algorithm has the advantage of allowing for fast decision-making [27]. Nevertheless, for large sets of states and actions, the traditional Q-learning algorithm cannot solve large-scale MDP problems. Deep learning can use Deep Neural Networks (DNNs) to learn very complex functions and deal with high-dimensional data samples. By combining q-learning with deep learning, a Deep Q-leaning Network (DQN) reinforcement learning model is proposed, which inspired our work [28]. Collaboration among edge servers, such as task offloading and service migration, essentially improves the QoS and reduces energy consumption in the EC paradigm. Most of the related works consider the two collaboration strategies separately, and do not efficiently combine the two strategies.