## 4 Proposed Algorithm
### Network Slicing Resource Allocation Algorithm Based on LSTM-D3 QN with DC
Meanwhile, according to the \(\epsilon\)-greedy strategy, the action that maximizes the current Q value in a certain state is selected. After a certain number of iterations, the parameters of the target Q network are updated by copying the current Q network parameters to the target Q network. After a predetermined number of iterations, LSTM-D3 QN is able to obtain the best action based on the best policy for a given state. Therefore, the optimal resource allocation scheme can be obtained. In order to facilitate readers to understand our process more clearly, the algorithm flowchart is shown in Figure 2. In addition, the pseudocode of the proposed algorithm is shown in Algorithm 1. ```
1:Initialize the environment parameters, reply memory D, the current network parameters \(\theta\), target action-value function \(Q^{L}\) parameters \(\zeta\), \(\kappa\); C, \(\gamma\). 2:Choose random action \(a_{0}\) to allocation bandwidth for users
3:Dual Connectivity:
4:Users \(\leftarrow\) the bandwidth resources
5:The bandwidth resources are not enough
6:The user opens dual connectivity
7:The \(pkt\) are calculated, and used as the current state \(S=s\) of the last iteration;
8:Repeat
9:For iteration = 1, \(T\)do
10:Choose an action according to the policy of LSTM-D3 QN
11:For slot = 1, \(I\)do
12:Execution scheduling
13:Execution dual connectivity
14:Endfor
15:The throughput is calculated according to Equation (5)
16:The QoE is calculated by Equation (6)
17:Calculate the utility based on Equation (7)
18:Calculate the reward
19:The \(pkt\) are calculated, and used as the state of this iteration
20:The agent input \((s,a,s^{\prime},r)\) into the LSTM-D3 QN

### Time Complexity Analysis

The time complexity of the training phase needs to consider the time complexity of training the Q network and the number of times of training the Q network. In the process of training the Q network, the connection weights between every two adjacent layers of neurons need to be updated. We set the number of layers of the Q network to be \(n_{l}\), the number of neurons in the \(i\)th layer to be \(l_{i}\), and the number of iterations in each training to be \(T_{train}\), then the time complexity \(C_{once}\) of training a Q network once can be calculated as

\[C_{once}=T_{train}K\bigg{(}\sum_{i=1}^{n_{l}-1}l_{i}l_{i+1}\bigg{)} \tag{24}\]

Figure 2: The algorithm flowchart. We denote the total number of iterations in the algorithm as \(T_{ite}\), and the number of steps in each iteration as \(T_{step}\), then the number of times to train the Q network is \(T_{ite}T_{step}\), so the time complexity of the proposed algorithm training phase can be calculated as

\[C_{train}=T_{ite}T_{step}T_{train}K\Bigg{(}\sum_{i=1}^{n_{l}-1}l_{i}l_{i+1} \Bigg{)} \tag{25}\]

The time complexity of the online training phase of the deep reinforcement learning algorithm is high, but after the Q network is trained, the Q network does not need to be updated in the running phase, and the time complexity is low, which can meet the requirements of online decision-making time under real-time network conditions. Since the algorithms we compared in the simulation are all deep reinforcement learning algorithms and are set the same parameters, they are roughly the same in terms of algorithm complexity. ## 5 Simulation Results and Discussion

### Simulation Parameters

In this section, we conduct an extensive evaluation of the proposed algorithm and the role of DC in resource allocation. In the simulation scenario, with a MBS as the center, three SBSs are distributed around, and the positions of the MBS and the SBS are fixed. Assuming that the MBS is located at the origin of the Cartesian coordinate system, the three SBSs are located at (80, 0), (\(-\)80, 80) and (\(-\)80, 80) coordinates, respectively. The weights of the optimization target throughput, QoE of both services and extra system energy consumption are set as \(\alpha=0.005\), \(\beta=[3,3]\), \(\eta=0.02\), respectively.