## 4 Proposed Algorithm
### Network Slicing Resource Allocation Algorithm Based on LSTM-D3 QN with DC
throughput, when the number of users in the system is 700 and 750, the system throughput is almost equal, but both are smaller than the throughput when the number of users is 650. Figure 13a shows that when the number of users is 650 and 700, the QoE of eMBB service can reach 1.0, but when the total number of users is 750, the QoE of eMBB can only converge to about 0.9. From the QoE of URLLC service, due to the role of DC, even if the total number of users of the system reaches 750, its QoE can reach more than 0.97. As can be observed from Figures 10-13, the number of users that the system can accommodate is between 700 and 750 under the condition that the QoE of the users can meet the requirements. Figure 11: The comparison of the number of dual-connected users under different numbers of users. Figure 10: The comparison of system utility under different numbers of users. #### 5.2.4 Simulation Results and Analysis with Different Algorithm Parameters

Figure 14 shows the effect of different learning rates on the performance of the algorithm. As can be observed from the figure, under different learning rates, our algorithm can converge to an optimal value. However, when the learning rate is different, the convergence speed of the curve is different. When the learning rate is 0.1, the convergence rate is the slowest, because the optimization gradient is very large in this case, and it is easy to skip the optimal solution. When the learning rate is 0.0001, the convergence speed is also very slow, because the gradient is too small, resulting in a slow optimization speed. When the learning rate is 0.001, the optimization gradient will not be too large, and the learning speed will not be very slow, so it has the fastest convergence rate. As can be observed from Figure 15, different batch sizes have little effect on the performance of our proposed algorithm, and even the optimization curves overlap. This is because more optimal actions

Figure 3:0 The comparison of system throughput under different numbers of users. Figure 3:1 The comparison of QoEs under different numbers of users. (**a**) The comparison of QoE of eMBB; (**b**) the comparison of QoE of URLLC. appear in our environment due to dual connections, so, in most cases, the same training result will be obtained even if the batch size of each training is different. Figure 3:2 Comparison of algorithm performance with different learning rates. Figure 3:3 Comparison of algorithm performance with different batch sizes. ## 6 Contribution Discussion

In Section 3, we propose a heterogeneous network scenario with two services, and introduce the establishment of the system model in detail, including signal-to-noise ratio, downlink transmission rate, user experience quality and additional energy consumption caused by dual connectivity users. We then use these optimization metrics to form our optimization problem, a weighted sum of system throughput and QoE, minus the weighting of additional energy consumption. Finally, we show that the optimization objective is an NP-Hard problem, which leads to the reason why we use deep reinforcement learning methods to solve our optimization problem. This section corresponds to our first part of the contribution. In the first part of Section 4, we introduce the basic formulations of DQN, Double DQN and Dueling DQN. At the same time, their advantages and disadvantages are introduced, and the role of long short-term memory network is briefly introduced, paving the way for proposing our resource allocation algorithm. In the second part, we introduce how the network slice resource allocation algorithm based on LSTM-D3 QN is derived, list the Q-value formula of LSTM-D3 QN, and set the state, action and reward for the proposed algorithm. Finally, we introduce the algorithm execution process in detail, and draw a flowchart and pseudocode for our algorithm to facilitate readers' understanding. This section corresponds to the second part of the contribution.