## 5 Experiment Results and Analysis
value of different reward intervals, no matter which interval it is, the DTe-DQN is better than the DQN, reflecting the advantages mentioned earlier. Figures 10 and 11 are the iterative curves of the average ping-pong rate and the average failure rate of each fading interval. First of all, with the increase of fading standard deviation, there is a higher probability of ping-pong, reflecting higher optimization difficulty. In the 3-4 dB interval, the ordinary DQN method has declined to a worse local optimal solution. Even so, the DTe-DQN still has better performance. As for the handover failure rate, the higher the fading degree is, the smaller the improvement effect of the two methods could achieve. It can be derived that in the higher fading range, the optimization mainly focuses on reducing high ping-pong rate, whose benefit is far greater than reducing the failure rate, and this phenomenon is obvious in high fading interval. Figure 10: Average handover ping-pong rate iterative curves for each fading interval. Figure 9: Average reward iterative curves for each fading interval. We contribute this to the joint optimization reward and the generalization ability of the Q-network. This paper designs reward function with the aim of maximizing effective handover rate. Thus, the overall reward has been increased, even though there is little improvement in handover failure rate of high fading interval. Furthermore, the strong generalization ability of the Q-network enables digging out better parameter decision strategy on maximizing reward. Figure 12 shows the iterative results of effective handover rate in different fading intervals. It can be seen that for 3-4 dB interval, the effective handover rate has been improved by about 16%. This is mainly because if the handover parameters were not optimized, the high signal fading would lead to a higher ping-pong probability. So its optimization space is significantly larger than 1-2 dB interval, about 7%. In addition, the DTe-DQN method also has a higher effective handover rate than the DQN most of the time. This figure validates the advantage of the DQN design based on signal fading that in each fading interval, both proposed algorithms have improved effective handover rate and achieved convergence. In other words, the proposed algorithms take advantage of the signal fading to improve handover performance, which would have degraded the handover performance due to fluctuation in time and space domain. Figures 13 and 14 summarize the handover ping-pong rate and failure rate that can be achieved in various cases. It can be seen that in relatively low fading interval, the handover failure rate and ping-pong rate have both been reduced at the same time. In high fading interval, however, the failure rate becomes harder to decrease, but the ping-pong rate is also decreased, making up for the shortcoming brought by failure rate, which can be validated from the comparison of the DQN and the DTe-DQN in 3-4 dB fading interval. This is due to the joint optimization design as discussed earlier that the handover failure and ping-pong rate are optimized for the overall effective handover rate. Figure 15 summarizes the achievable effective handover rate in various cases. Whether it is average effective handover rate or in each fading interval, the DTe-DQN has 2-3% better performance than the DQN. In addition, the DTe-DQN performs 7-16% better than random handover parameter strategy, especially in high fading interval. It can be seen that the DTe-DQN proposed in this article can not only comprehensively improve the handover performance, but also outperforms the ordinary DQN in terms of high fading interval with a higher difficulty of optimization. We contribute it to the additional exploration mechanism of the DTe-DQN. The results of achievable handover performance reveals that with the aid of digital twin, the DTe-DQN achieves more effective interaction, which accelerates and stabilizes the DQN algorithm iteration. Figure 11: Average handover failure rate iterative curves for each fading interval.