# Adaptive BBU Migration Based on Deep Q-Learning for Cloud Radio Access Network
## 4 Proposed Dynamic Migration System
```
Initialize DQN_Model with trained weights  Set threshold (upper and lower) for resource utilization (e.g., CPU_threshold, Memory_threshold)  While the simulation is running: # Step 1: Collect environment state  For each host in the iCanCloud environment:  current_CPU_utilization = host.getCPUUtilization()  current_Memory_utilization = host.getMemoryUtilization()  current_Energy_consumption = host.getEnergyConsumption() # Step 2: Define the state (combine CPU, Memory, and Energy into a state vector)  state = [current_CPU_utilization, current_Memory_utilization, current_Energy_consumption] # Step 3: Feed state to DQN model to obtain action = DQN_Model.predict(state) # Step 4: Action decision  If action == 'MIGRATE_VM':  # Perform migration  target_host = selectBestHostForMigration() vm_to_migrate = selectVMToMigrate(host) migrate(vm_to_migrate, target_host)  log(\"Migration performed from\", host, \"to\", target_host)  Else if action == 'DO_NOTHING':  # No migration needed, continue monitoring log(\"No migration required for\", host)  # Step 5: Monitor environment and obtain feedback for reward  post_migration_CPU_utilization = host.getCPUUtilization()  post_migration_Memory_utilization = host.getMemoryUtilization()  post_migration_Energy_consumption = host.getEnergyConsumption()  reward = computeReward(post_migration_CPU_utilization, post_migration_Energy_consumption)  # Step 6: Train DQN with the reward (optional for continued training)  DQN_Model.update(state, action, reward)  # Continue to the next host or iteration
```  
**Algorithm 1**DQN migration method with dual threshold pseudo-code  
M\({}_{\text{k-DCPU}}\) and VM\({}_{\text{K-DMem}}\) show the difference between D's (D\({}_{\text{CPU}}\) or D\({}_{\text{Mem}}\)) and VM's CPU and memory use. Only one virtual machine (VM) needs to be moved in order to obtain the server's resource usage rate below the upper threshold if the difference is larger than or equal to 0. This VM is retained within the VM_list subset. Within this subset, VMs that meet the minimum migration count are ordered by memory usage, and the VM with the smallest memory usage is selected to be migrated. If VM_list is empty, the strategy calculates \(f\) from Algorithm 1, whether VM_list is empty for each VM on the overloaded host, and then selects the VM that will most effectively reduce server load. ```
Input:Over_load_Host, DCPU and DMem Output:VM_to_migrate Step1 # Calculate the utilization load for eachVM in Over_load_Host do case0: VMk-DCPU = VMk_CPU-DCPU,VMk-DMem = VMk_Mem-DMem case1: VMk-DCPU = VMk_CPU-DCPU case2: VMk-DMem = VMk_Mem-DMem Step2# Check the VMk_DCPU and VMk_DMem for each VMs to be migrated if (case 0:VM k-DCPU >= 0 and VM k-DMem >= 0; case 1:VM k-DCPU >= 0; case 2:VM k-DMem >= 0) Step3# If the difference is greater than or equal to zero sort the VMlist_count according to minimum memory usage and then select the smallest one to be migrated VM_list \(\leftarrow\)VMk sort VM_list by memory VM_to_migrate = VM_list[0] Step4# Or else, calculate the utilization weight of the three cases as follows: case0:\(e1=\frac{D_{CPU}}{D_{CPU}+D_{Mem}}\)\(e2=\frac{D_{Mon}}{D_{CPU}+D_{Mon}}\) case1:\(e1=1\), \(e2=1\) case2:\(e1=0\), \(e2=1\) \(fk=w1\cdot\)VMk-DCPU + \(w2\cdot\)VMk-DMem VMlist_f \(\leftarrow\)VMk, \(fk\) Step5# Sort the list of VMs according to f sort VMlist_f by \(fk\) VM_to_migrate = VMlist_f [0] return VM_to_migrate
```  
**Algorithm 2**VM Migration selection strategy pseudo-code  
The VM selection is not required when the compute node initiates migration due to an excessively low load because all of the virtual machines within the node are immediately relocated, and the node will be shut down to conserve energy.