# Dynamic Computation Offloading with Deep Reinforcement Learning in Edge Network
## 5 Experiments
### Experimental Settings  
In system model simulation settings, to illustrate the performance of the proposed strategy, we set an area of \(500\times 500\) m\({}^{2}\), which includes 50 randomly deployed edge servers. The time slot value was random in 20-30 ms, and the IoT device issue \(k\) task requests at each time slot. For all simulation parameters, we simplified some parameters, but kept key parameters directly involved in our algorithm's decision-making process. The key parameters are described in Table 1.  
In the DQN algorithm, a deep neural network structure will significantly affect the efficiency of the algorithm. Our work shows that a simple four-layer network architecture, with one input layer, two hidden layers and one output layer, can achieve a better convergence performance. There were 100 and 60 hidden neurons, respectively, in these two hidden layers. The hidden layer used Relu as the activation function, and the sigmoid activation function was employed in the output layer to output the optimal action. The DQN parameters were configured as follows. The learning rate \(\alpha\) equalled 0.01, the discount parameter \(\gamma\) was set to 0.9, and the training batch size and the size of experience replay memory were set to 100 and 200, respectively.  
To illustrate and validate the effectiveness of our proposed DCOS algorithm, we focus on the following two metrics.  
* The time delay for task-offloading, service migration, and task-processing.
* The energy consumption for task-offloading, service migration and task-processing.  
\begin{table}
\begin{tabular}{c c l} \hline
**Parameters** & **Simulation Value** & **Description** \\ \hline \(F\_cap\) & 2 GHz & The processing capability of edge server \\ \hline \(T_{i}\) & random in [20; 30] ms & The \(i\)-th time slot \\ \hline \(n\) & 50 & The number of edge servers \\ \hline \(k\) & 30 & The max number of task requests at each time slot \\ \hline \(m\) & 50 & The number of service classes \\ \hline \(H\_bytes\) & random in [0.2, 5] MB & The capacity of service \\ \hline \(C\_cir\) & uniform in [2; 12] cycles/bit & The number of CPU circles required by the task \\ \hline \(C\_bytes\) & random in [0.5, 5] MB & The task capacity \\ \hline \(B\) & 500 MHz & The channel bandwidth \\ \hline \(N_{noise}\) & \(-100\) dBm & The background noise \\ \hline \(\omega_{1}\) & 0.8 & Weight of delay in total cost \(\mathcal{C}\) \\ \hline \(\omega_{2}\) & 0.2 & Weight of energy consumption in total cost \(\mathcal{C}\) \\ \hline \(\epsilon_{amp}\) & 0.1 nJ/(bit \(\times\) m\({}^{2}\)) & The energy consumption constant of the transmit amplifier \\ \hline \(e_{dec}\) & 50 nj/bit & The energy consumption constant of the transmission and receiver electronics \\ \hline \end{tabular}
\end{table}
Table 1: Simulation parameters.  
To show the superiority of the proposed DCOS algorithm in terms of its delay and energy metrics, we compare it with two baseline schemes, as follows:  
* Deep Deterministic Policy Gradient (DDPG) [33]: Task scheduling is only based on the task-offloading strategy, and the optimal offloading target node is output by the DQN network.
* Extensive Service Migration Model (ESM) [39]: The task is processed on the local node. If the node has not configured the related service, the system model performs the service migration according to the optimal policy related to the migration costs.