# A Dynamic Service Placement Based on Deep Reinforcement Learning in Mobile Edge Computing
## 2 Related Work  
The concept of mobile edge computing is introduced to extend the cloud paradigm, which enables a new breed of services and applications. It provides a service environment closer to both users and IoT devices by deploying several mobile edge servers. Service placement is a well-investigated problem in mobile edge computing that advocates for providing service offering at the users' side [2, 3]. Various works have studied different aspects of this problem. A subset of existing work in this area relates to improving the utilities and reducing the operational cost. [PERSON] et al. [4] propose a dynamic storage-stable service placement strategy by using the Lyapunov optimization method to maximize the system utility while striking a balance between the overhead and stability. [PERSON] et al. [5] focus on the problem of service placement by considering a heterogeneous mobile edge computing system, and they propose a deterministic approximation algorithm to maximize the total revenue. [PERSON] et al. [6] propose an efficient decentralized algorithm by exploiting the graph coloring on the small cell network for performing collaborative service placement in order to optimize the utility of operators. [PERSON] et al. [7] investigate the collaborative service placement problem in mobile edge computing by proposing an efficient decentralized algorithm based on the matching theory. They try to minimize the traffic load to realize a high utilization on computing and radio resources. [PERSON] et al. [8] focus on the layer-aware service placement and request scheduling problem, and they design an iterative greedy algorithm by formulating it into an optimization problem with approximate submodularity. In addition, quite a few works have been carried out on optimizing the quality-of-service (QoS). [PERSON] et al. [9] tackled it by proposing a trust-oriented IoT service placement method for smart cities in edge computing, and they try to optimize the execution performance with privacy preservation. [PERSON] et al. [10] formulate the load distribution and placement problem as integer nonlinear programming, and they try to minimize the potential violation to improve the QoS by using the genetic algorithm. [PERSON] et al. [11] propose a runtime system that effectively deploys user-facing services in a cloud-edge continuum to ensure the QoS by jointly considering communication, contention, and load condition. However, these works ignore the coupling relationship between service performance and operational cost caused by users' erratic movements. In response to the challenge on users' mobilities across multiple timescales, some works are based on service migration in mobile edge computing. There are a few works that assume that the user mobility follows a Markovian process and apply the Markov Decision Process (MDP) technique. [PERSON] et al. [12] formulate the service migration problem with minimum cost as an MDP and propose a new algorithm for computing the optimal solution that is significantly faster than traditional methods based on standard value or policy iteration. [PERSON] et al. [13] jointly optimize the network selection and service placement to improve the QoS by considering switching and communication delay, and they propose to dynamically place and migrate the services according to the mobility of users by introducing an iteration-based algorithm. [PERSON] et al. [14] study the mobile edge service performance optimization problem by applying the Lyapunov optimization. They design an approximation algorithm based on Markov approximation under long-term cost budget constraint. Since the characteristics of mobile users are moving without a priori knowledge, some researchers introduce deep reinforcement learning. [PERSON] et al. [15] propose a novel service migration method based on state adaptation and deep reinforcement learning to overcome network failures, and they use the satisfiability modulo theory to solve the candidate space of migration policies. [PERSON] et al.