# Adaptive BBU Migration Based on Deep Q-Learning for Cloud Radio Access Network
## 4 Proposed Dynamic Migration System  
Since each physical compute node's operating state must be monitored and migration requirements for the resource state must be defined to decide whether to activate the dynamic migration. The suggested system established a dual threshold technique for the migration process. The higher threshold is specified in this work to mitigate the issue of the physical server increasing energy consumption due to the high load. To save energy, set a lower threshold to move the virtual machine (VM) on the compute nodes when the server utilization falls below it and shut down or sleep the physical server. Because physical servers' resource use is unstable, the CPU or memory utilization at a given moment may result in needless migration because it is either too high or too low, wasting system energy. The parameter definitions suggested in the paper are displayed in Table 1. \begin{table}
\begin{tabular}{l l} \hline \hline
**Terminology and Parameters** & **Meaning** \\ \hline Over\_load\_Host & Overload compute node \\ \hline C\_N\_i[CPU/Mem] util & The utilization of the resources; CPU or memory for a compute node \\ \hline D\({}_{CPU}\)/D\({}_{Mem}\) & The distinction between the upper threshold and overflow resource utilization \\ \hline C\_N\_i\_pre[CPU/Mem]util & The prediction of resource utilization; CPU or memory for a compute node \\ \hline VM \({}_{k\_CPU/Mem}\) & CPU or memory utilization of the VMk \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Important terminology and their meaning. Because deep reinforcement learning (DQN) can learn and optimize control techniques in non-linear situations, it was selected for resource consumption prediction in this research. DRL's salient characteristics that render it appropriate for this system are as follows:  
1. Non-linear decision making: DQN can handle complex, dynamic resource utilization patterns. 2. Policy optimization: Through training, DQN learns optimal migration strategies based on past experience and rewards. 3. Continuous improvement: Unlike static methods like the Kalman filter, DQN improves as it receives feedback. The chosen architecture was based on **deep Q-networks (DQNs)**. This model uses a neural network to approximate the value of taking certain actions (i.e., migrating resources) in different states of resource utilization. The system continuously adapts by learning how actions influence future states and rewards. Since server resource utilization is immediate, predictive technology can effectively avert unnecessary virtual resource migrations. A predictive migration trigger strategy, driven by a DQN algorithm, is used to evaluate if server resource usage exceeds or falls below a predefined threshold. The iCanCloud simulation tool was employed to generate data on CPU usage, memory consumption, energy usage, migration time, and migration count in a cloud setting. This dataset was subsequently used to train a deep reinforcement learning model designed to enhance the resource migration strategy. These data points served as input features for training the DQN model, derived from the iCanCloud simulation, and included the following:  
* State variables: CPU and memory utilization, current load on BBUs, and system energy consumption. \[\mathrm{s_{t}}=\left[\mathrm{CPU\ Utilization_{i},\ Memory\ Utilization_{i},\ Energy\ Consumption_{i},\ VM_{i}}\right]\forall\mathrm{i}\in\mathrm{hosts} \tag{6}\]  
* Action space: Decisions on whether to migrate VMs or not, and the selection of target physical machines for migration. \[\mathrm{a_{t}}\in\left\{\mathrm{Migrate\ VM\text{$\rightarrow$h_{j}$, Keep\ VM\ on\ h_{i}$}\right\} \tag{7}\]
* Reward function: The reward was based on optimizing resource utilization and min. imizing energy consumption, and the number of migrations. The reward is positive for resource behaviors that reduce energy consumption.