# Dynamic Computation Offloading with Deep Reinforcement Learning in Edge Network
## 4 Algorithm
### Dynamic Computation Offloading Strategy  
Task offloading and service migration are promising strategies to meet the stringent requirements regarding the delay in user requests and the energy consumption of IoT devices in edge networks. There are two main causes of the task offloading strategy:  
* The workload on the local edge server is heavy, which significantly increases the task processing delay, leading to QoS reductions. * The corresponding service for the task request is not configured on the local edge server. In both cases, tasks need to be offloaded to other edge servers and configured with the related services. In the service migration strategy, if the local edge server does not have the relevant service, it is necessary to migrate the service from the cloud or other edge servers configured by the service to the local edge server to process the task. As shown in Figure 2, the processing of a single task can be split into three parts: the transmission process, the queuing process, and the computing process. The transmission processes in the two abovementioned strategies are the offloading and migration process, respectively. We can also see that the tasks can only be processed at the local edge server or the other edge servers that are configured with the related service, whether offloading or migration tasks. These two schemes limit the processing location of the tasks to certain network servers and do not fully exploit the potential of collaboration between edge servers. In our strategy, the task-processing location is extended to any server in the edge network. More specifically, when an edge server receives a task request, if the local workload is heavy or the corresponding service is not configured, the task will be offloaded to another intermediate node with sufficient computing resources. It is worth noting that the intermediate node may not be configured with the corresponding service. Therefore, it is necessary to migrate the related service from the neighboring nodes to the intermediate node. It is worth mentioning that this migration process is executed in parallel with the offloading process. As Figure 3 shows, at time slot \(T_{3}\), \(F_{1}\) receives a task request from a user. The heavy workload of \(F_{1}\) means that the task has a long queuing time, and no related service is configured at \(F_{1}\). Therefore, the task is offloaded to the intermediate node \(F_{2}\), which has sufficient computing resources. Meanwhile, the service corresponding to the task migrates from \(F_{3}\) to \(F_{2}\). It can be observed that the queuing time is eliminated by its execution on \(F_{2}\) because it is idle. Moreover, since the location of \(F_{2}\) is between \(F_{1}\) and \(F_{3}\), the transmission distances for both offloading and migration are correspondingly reduced compared to the offloading and migration process between \(F_{1}\) and \(F_{3}\) in Figure 2. Thus, the transmission delay is reduced for both. Furthermore, since both transmission processes are executed in parallel, there is some overlap between the offloading delay and migration delay. It is worth noting that the intermediate node mentioned above is not a necessary geographic intermediate. Our strategy is based on edge servers'  
Figure 3: Task processing for dynamic computation offloading strategy. Figure 2: Task processing for migration and offloading strategies. if a node is far from the local node, we can still target it as the processing node for the task because it has sufficient computing resources, and can more efficiently process the task. Based on the above description, we can classify our strategy's transmission process in the following three cases:  
* Service migration process: the total transmission delay is the migration delay \(d_{cm}\). * Task offloading process: the total transmission delay is the offloading delay \(d_{to}\). * Parallel service migration process and task offloading process: the total transmission delay is \(max(d_{cm},d_{to})\).