## 5 Numerical Results  
### Simulation Setting  
We perform a Python-based system-level simulation, and the parameters are summarized in Table 2. In this work, three kinds of slices with different downlink and uplink traffic ratios are considered, and the slice parameters are summarized in Table 3. Each slice uses a round-robin algorithm for resource scheduling. In each episode of 10 steps, the number of UEs in each slice is set to increase linearly from 30 to 120 to simulate the load increase. More specifically, 10 UEs are randomly assigned to three slices in each step to simulate a random increase in the number of UEs. Within each episode, the weights of all slices remain the same. In different episodes, the weights of six parts (three slices with downlink and uplink) are increased in turn, while the other five parts share the remaining weights equally. For example, the weight of S1 DL is set to 0.35, and the weights of the other five parts are set to 0.13. The size of each packet is set to 12 kbits in the simulation, as the commonly used Maximum Transmission Unit (MTU) is usually set to 1500, that is, 1500 bytes (12 kbits). The arrival of packets satisfies the Poisson distribution. It is assumed that all UEs have the same traffic model. The throughput requirement for each UE is set to be \(1.05\times R_{m}^{d}\) instead of a common-used full buffer. To ensure that all UEs experience both good channel quality and bad channel quality, UEs are set to move along a straight line. Once a UE reaches the boundary of the given area, it moves in a random direction that does not go outside the boundary. The path loss model of an urban macro scenario is adopted, referring to [2]. The training computer has an Intel core i7-13700K CPU and 32 GB of memory.  
We compare the proposed QGJRA-SUL framework with several baselines, which are listed as follows. Note that the NR bandwidth of all baselines without SUL is set to be 100 MHz + 10 MHz for fairness. In other words, the proposed framework contains 100 MHz for NR and 10 MHz for SUL. Other frameworks labeled as \"-nonSUL\" contains 100 MHz for NR, and another 10 MHz for NR.  
* QGJRA-SUL, \(\beta=0\): The weight value \(\beta\) in the penalty term \((-\beta\cdot\varsigma)\) in the reward function (11) is set to 0, which means that the load balancing mechanism is removed.
* QGJRA-nonSUL: QGJRA without the SUL band is tested for comparison.
* DNSIE-nonSUL [30]: A heuristic algorithm based on a fixed threshold realizes resource sharing among different slices.
* Static-NR-nonSUL: The whole QGJRA framework is static with a preset action.
* Optimal: We try almost every possible action to derive the optimal result for QGJRA-SUL.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Parameters** & **Value** & **Parameters** & **Value** \\ \hline \(\tau\) & 2 ms & \(W^{nr}\) & 100 M (250 RB) \\ \(\mu\) & 1 & \(W^{sul}\) & 10 M (25 RB) \\ \(\mathrm{N}^{UL}+\mathrm{N}^{uL}\) & 56 & \(p_{lL}\) & 43 dBm 1  \\ \(T\) & 100 ms & \(p^{ul}\) & 23 dBm \\ NR frequency & 3.5 GHz & \(N_{0}\) & \(-174\) dBm/Hz \\ SUL frequency & 1.8 GHz & BS radius & 350 m \\ \hline \hline \end{tabular}
\end{table}
Table 2: The parameters of the simulation environment.  
\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline  & **S1 DL** & **S1 UL** & **S2 DL** & **S2 UL** & **S3 DL** & **S3 UL** \\ \hline Packet Size (bits) & 12 k & 12 k & 12 k & 12 k & 12 k \\ QoS (Mbps) & 1 & 1 & 2 & 1 & 1 & 2 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Slice Parameters.