# Adaptive BBU Migration Based on Deep Q-Learning for Cloud Radio Access Network
## 4 Proposed Dynamic Migration System
\[\mathrm{r_{t}}=\lambda_{1}\times\left(1-\frac{CPU\ Load}{Threshold}\right)- \lambda_{2}\times Migration\ Cost-\lambda_{3}\times Energy\ Consumption \tag{8}\]  
where \(\lambda 1,\lambda 2,\lambda 3\) (lambda_1, lambda_2, lambda_3) are weighting factors determining whom to give importance. In our case, we have carried out a simulation, mostly optimizing the energy consumption. **Example of Reward Calculation**  
* Case 1: Balanced load without migration  
\(\mathrm{\SIUnitSymbolicircap{}}\)  
CPU usage: 50% (below threshold) \begin{table}
\begin{tabular}{l l} \hline
**Terminology and Parameters** & **Meaning** \\ \hline VM\({}_{k\text{-}\left[\mathrm{\SIUnitSymbolicap{}}\right]}\) & Difference between VMk\_[CPU/Mem] \\  & and DCPU/DMem \\ \hline VM\_to\_migrate & Selected VM to be migrated from overload host \\ \hline \end{tabular}
\end{table} 
Table 1: _Cont._ \begin{tabular}{c c} \(\bigcirc\) & Migration cost: 0 \\ \(\bigcirc\) & Energy consumption: 0 \\  & \\  & Case 2: High load and migration needed \\ \(\bigcirc\) & CPU usage: 90\% (above threshold) \\ \(\bigcirc\) & Migration cost: 0.1 \\ \(\bigcirc\) & Energy consumption: 0.05 \\  & \\  & \(R_{t}=\lambda_{1}\times\left(1-\frac{90}{70}\right)-0.1\lambda_{2}-0.05\; \lambda_{3}\) \\  & \(R_{t}=-0.29\lambda_{1}-0.1\lambda_{2}-0.05\;\lambda_{3}\) \\ \end{tabular}   
A negative reward pushes the agent to take corrective actions (e.g., migrate a VM). The deep Q-network (DQN) model, a widely used DRL architecture was chosen. The model's layers were structured as follows:  
* Input layer: takes the current state of CPU and memory utilization, energy consumption, and migration metrics. * Hidden layers: fully connected layers with ReLU activation functions to handle the non-linearity in the data. * Output layer: produces the Q-values corresponding to the possible migration actions, where higher Q-values indicate better decisions. This research proposes a migration technique that combines prediction and resource utilization to evaluate the relationship. Figure 3 depicts a thorough procedure and Algorithm 1 as a trigger flow. The proposed trigger method begins by polling each compute node's load utilization for monitoring purposes, using CPU and memory usage as criteria for initiating migration. It then evaluates the relationship between utilization levels and the threshold. The prediction model is used to continuously determine the relationship between load utilization and the threshold if any parameters exceed the threshold range. A compute node is identified as overloaded if the predicted value surpasses the upper threshold, in which case it calculates \(\text{D}_{\text{CPU}}/\text{D}_{\text{Mem}}\), representing the difference between the overload resource utilization and the upper threshold. Otherwise, the method employs a prediction model to initiate the underload decision process, recognizing the node as underloaded if the expected value is below the lower threshold. To avoid performance degradation, the VM selected for migration is chosen from an overloaded or underloaded compute node following the VM selection strategy. According to the proposed method, overload may occur in three scenarios: In case 0, both memory and CPU utilization exceed the upper threshold. In cases 1 and 2, only utilization of one of the resources, memory or CPU, respectively, surpasses the upper threshold. For energy efficiency, dynamic migration is triggered, and the underloaded node is powered down if any utilization parameters fall below the lower threshold. If all parameters remain within the threshold range, the system is considered to be in a normal state, and the next phase of the decision-making process begins. The strategy then selects the VM with the shortest migration time from the identified subset of VMs, prioritizing VMs with the fewest previous migrations. Algorithm 2 illustrates the proposed VM selection process. Figure 3: The flowchart of the proposed migration procedure.