## 4 Proposed Algorithm
### Network Slicing Resource Allocation Algorithm Based on LSTM-D3 QN with DC  
In this subsection, we introduce the LSTM-D3 QN slicing resource allocation algorithm. In D3 QN, to find the Q value, we need to input the state \(s\) to the fully connected layer, which is processed by two fully connected layers to form the feature tensor. With the introduction of the LSTM neural network, we only need to input the state \(s\) into one layer of LSTM network to form the feature tensor, and then input the feature tensor to the fully connected layer of the value function and the fully connected layer of the dominance function in the dueling network to obtain the state value and the dominance value. It can be considered that the state of the input has changed from \(s\) to \(s^{L}\). Thus, the current Q-value updated formula is expressed as  
\[Q_{u}^{L}\Big{(}s^{L},a;\theta,\xi,\kappa\Big{)}=V\Big{(}s^{L};\theta,\kappa \Big{)}+\bigg{(}A\Big{(}s^{L},a;\theta,\xi\Big{)}-\frac{1}{|A|}\sum_{a^{ \prime}}A\Big{(}s^{L},a^{\prime};\theta,\xi\Big{)}\bigg{)} \tag{20}\]  
Thus, our final Q-value updated formula can be denoted as  
\[Q^{L}\Big{(}s^{L},a;\theta,\xi,\kappa\Big{)}=Q_{u}\Big{(}s^{L},a;\theta,\xi, \kappa\Big{)}+\varepsilon\Big{(}\text{TargetQ}-Q_{u}\Big{(}s^{L},a;\theta, \xi,\kappa\Big{)}\Big{)}. \tag{21}\]  
We define a tuple \((s,a,r,s^{\prime})\), where \(s\) represents the state, \(a\) represents the action taken by the algorithm, and \(r\) is defined as the reward obtained by the agent interacting with the environment after taking action \(a\). In a certain time slot, the agent observes a certain state \(s\), and takes action \(a\). After executing this action, the environment state transitions to \(s^{\prime}\). Since the number of successful packet transmissions reflects the QoE, as well as the throughput in general, we define the state \(s\) as  
\[s=\frac{pkt-y_{mean}}{std} \tag{22}\]  
where \(y_{mean}\) represents the average value of the number of transmitted packets and \(std\) represents standard deviation of packet transmission. We define the action as the size of the bandwidth that each BS allocates to each service, denoted as  
\[a=[(w_{0,0},\ w_{0,1}),\ (w_{1,0},\ w_{1,1}),\cdots,\ (w_{k,0},\ w_{k,1})] \tag{23}\]  
With regard to the reward, we set the standard of throughput and QoE for eMBB and URLLC services as 450 Mb/s, 0.97 and 0.94, respectively. Since we aim to improve the throughput and QoE of the services, the reward is given based on satisfying the three metrics. After initializing the parameters of the LSTM-D3 QN algorithm, we need to randomly select an action \(a_{0}\), and the BS obtains a resource allocation scheme according to action \(a_{0}\). Then, the agent calculates the \(pkt\) of the two services as the observation value and inputs it into the Q network to form the initial state \(s\) according to Equation (22). In each iteration, the BS selects an action to obtain the corresponding bandwidth resource size of each service. After the user receives the bandwidth resource, if the bandwidth obtained by the user in this time slot is not enough to handle the task, the user will open a second connection at this time. Then, it can calculate each rate, QoE of services, extra energy consumption and the total utility of the system according to the Equations (5)-(8). Finally, the agent calculates the \(pkt\) of the two services again as the next state \(s^{\prime}\), and input \((s,a,s^{\prime},r)\) into LSTM-D3 QN for training. For the update of network parameters, the agent interacts with the environment to obtain information and stores the information in the memory pool as transfer samples. When the number of samples in the memory pool is large enough, a certain amount of data is randomly selected from the sample pool for training. Then, LSTM-D3 QN uses the evaluation Q network to calculate the state value and advantage value to obtain the current Q value and uses the target Q network to obtain the target Q value.