# Virtual Machine Migration Strategy Based on Multi-Agent Deep Reinforcement Learning
## 1 Introduction  
With the development of science and technology, there are many applications that require high computing power and are extremely sensitive to delay, which are far beyond the service capacity that mobile devices can provide. In addition, due to the limitations of the development of battery technology, the power consumption of running these applications is unbearable for mobile devices. These problems have become an opportunity for the development of Mobile Cloud Computing (MCC) technology, through the integration of cloud computing capabilities into the mobile network, so that mobile users can access and use cloud computing services [1]. The Mobile Capabilities Augmentation using Cloud Computing (MCACC) framework proposed in reference [2] divides mobile applications into a group of services. The framework is deployed in the application layer, and each service is executed locally or remotely on the cloud based on dynamic unloading decisions, so that mobile applications are not limited by mobile device resources. However, because mobile cloud computing uses a centralized model, there is often a very long distance between the server and mobile users, which will lead to high network delay. Additionally, in the longer path, the probability of data being attacked on the way will be greatly increased. The European Telecommunications Standards Institute (ETSI) introduced the concept of Mobile Edge Computing (MEC) in 2014. According to the white paper released by ETSI for the first time, mobile edge computing is defined as: \"Mobile edge computing can provide IT service environment and cloud computing capabilities within the range of Ratio Access Network (RAN) near mobile devices [3]\". Mobile edge computing releasesthe mobile device from the heavy computing work. By transferring the computing task on the mobile device to the MEC server, which is closer from the space [4], it can reduce the user-perceived delay, reduce the network congestion probability and prolong the battery life of the mobile device. Although MEC greatly improves the computing power of mobile users, the unpredictable user mobility in wireless networks also brings new challenges. Reducing latency and smoothing the user experience as users move cannot be solved by simply pushing cloud capabilities to the edge of the network. In order to ensure the business continuity of users between different edges, it is necessary to adopt an efficient mobility management scheme at the network edge. Software Defined Network (SDN) [5] is an emerging technology, which provides a simplified method to dynamically control multiple simple switches through a single controller program. In the fog computing architecture based on SDN [6], routing logic and intelligent management logic are deployed on the SDN controller, which greatly simplifies the operation and management of the network. However, the placement of dynamic service profiles is not simple. On the one hand, the user-perceived delay is determined by both the communication delay and the computing delay [7,8]. Therefore, if each user's service profile is placed on the nearest MEC node, some MEC nodes may be overloaded, resulting in increased computing latency. On the other hand, when the MEC computing node is overloaded or unable to calculate, the computing services need to be migrated to the Virtual Machines (VMs) of other nodes for computing. In order to improve Quality of Service (QoS) such as TCP throughput, in Reference [9], the author proposed a VM migration method based on expected TCP throughput, which migrates VMs from crowded nodes to another node on the mobile edge; this method is a VM migration method considering edge node congestion without considering user mobility in MEC environment. Moreover, during the migration, you also need to ensure that the resources (CPU, network bandwidth, etc.) of the migration operating system will not be unnecessarily disrupted by contention for active services [10].