# Intelligent Computation Offloading Based on Digital Twin-Enabled 6G Industrial IoT
## 5 Experiments and Discussion  
In this section, we carry out extensive experiments to simulate the proposed DT-enabled DDPG algorithm. These experiments are used to evaluate the learning process and network performance of the proposed computation offloading scheme in a 6G IIoT scenario. We use Python 3.8 and TensorFlow 2.4 to evaluate our proposed algorithm. We consider a network with a maximum of 12 edge servers. The physical devices are connected to the edge servers through access points and the number of IIoT devices is between 40 and 50. The process by which IIoT devices generate computing tasks follows a Poisson distribution. The arrival rate of tasks on each access point is between 1 and 5 per second. We consider Rayleigh fading channels. Three fully connected hidden layers with [256 256 512] neurons are deployed for DDPG, and its output layer is activated by the tanh function. The batch size is 32 and the total training episode is 2000. All the experimental results have been performed by extracting 100 data points on a PC with 2.81 GHz CPUs, 8.00 GB RAM, and Windows 10 OS. The key parameters employed in the simulations are summarized in Table 3, unless specified. We first tested the learning relationships of the DDPG algorithm with different parameter settings to get the best rewards. Figure 4 describes the influence of the proposed algorithm on the convergence performance at different learning rates. The proposed algorithm achieves poor performance after convergence when the learning rate is 0.01. The reason is that the greater learning rate will cause both the actor network and the critic network to adopt large updating steps. The algorithm converges faster and can obtain better cumulative rewards when the learning rate is 0.001. When the learning rate is very small, e.g., 0.0001 and 0.00001, the algorithm produces large fluctuations in the convergence process. The reason for this is that the lower learning rate makes the network update more slowly and therefore requires more training steps to obtain better results. In subsequent experiments, we set the learning rate to 0.001. Next, we evaluate the effect of the discount factor on the system rewards. Figure 5 describes the influence of different discount factors on the convergence performance of the proposed algorithm. The discount factor is used to adjust the weight of the effect of long-term and short-term rewards on the system. The larger the \(\gamma\) value, the more the agent considers long-term training factors, and the higher the training difficulty. The smaller the \(\gamma\) value, the more the agent focuses on short-term benefits, and the lower the training difficulty. From the figure, we can see that the trained rewards value has the best performance when the discount factor is 0.01 (Figure 5a) and 0.1 (Figure 5b), respectively. The reason is that the environment of different periods is quite different, so the generalization ability of different periods is poor. The discount factor is not a universal hyperparameter, and its optimal value may be affected by many factors such as task, environment, and algorithm. In different scenarios, the discount factor needs to be adjusted and optimized to maximize the performance of the algorithm. In the following experiment, our communication scenario is set as a discount factor of 0.01. \begin{table}
\begin{tabular}{l l} \hline \hline
**Parameters** & **Value** \\ \hline Maximum number of edge servers & 12 \\ Number of physical devices & 40–50 \\ CPU cycle frequency of IIoT & (0, 1] GHz \\ CPU cycle frequency of edge server & (1, 2] GHz \\ Task computation requirement & [100, 200] M \\ Bandwidth & 20 MHz \\ Gaussian white noise power & \(10^{-11}\) mW \\ Batch size & 32 \\ Episodes & 2000 \\ Sample rate & 0.1 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Key parameters in experiments. For performance comparison, we enumerate two benchmark schemes: (1) the local processing scheme and (2) the nearest edge server selection scheme. 1.