# User Pairing for Delay-Limited NOMA-Based Satellite Networks with Deep Reinforcement Learning
## 5 Results  
In this section, simulation results are provided to characterize the effects of users' specific delay QoS requirements on the power allocation scheme, user selection strategy, and system performance. Without loss of generality, we assumed \(T_{f}B=1\), the carrier frequency as 4 GHz, and radius \(R=125\) km [5; 13]. Moreover, we set the number of users as 150, the fading severities, location information, and delay requirements of these users were randomly generated within [ILS, AS, FHS], [0, 1R], and [0.5, 10], respectively, to show the various channel conditions, locations, and application scenarios of different satellite users. The delay QoS exponent of the delay-sensitive user, i.e., User \(c\), was set as \(\theta_{c}=9.38\), and the label (ILS/AS) denotes the link-shadowing severity of User \(t\)/User \(c\) in this paper. We first conducted numerical simulations to show the impact of shadowing, \(\gamma\), and \(d_{c}\) on the power allocation coefficient \(a_{c}^{p}\), as illustrated in Figure 3. From this figure, we can clearly see that, when User \(c\) experiences a lighter shadowing, a higher \(\gamma\), or a closer location information \(d_{c}\), a larger \(a_{c}^{p}\) is needed to ensure that the performance achieved with the NOMA scheme is not less than that achieved with the TDMA scheme, which is consistent with the analytical result given in (11). In the following simulations, \(a_{c}^{p}\) was set to meet the condition of \(C_{c}^{N}(\theta_{c})=C_{c}^{\mathrm{T}}(\theta_{c})\) without other descriptions. Moreover, it can be observed that the analytical results were all consistent with the Monte Carlo simulations. Then, simulations were conducted to illustrate the capacity of User \(t\) achieved with the NOMA scheme and TDMA scheme versus delay requirement \(\theta_{t}\), shown in Figure 4), from which we can clearly observe that the capacity curves all degrade with increasing \(\theta_{t}\). This is an expected result because a larger \(\theta_{t}\) means a smaller tolerated delay outage and a lower supported constant arrival rate. Moreover, we find that the superiority of the NOMA scheme gradually decreases with increasing delay limitation \(\theta_{t}\), i.e., when \(\theta_{t}\geq 10^{0.4}\), the capacity gap between NOMA and TDMA curves almost disappears. The superiority of the NOMA scheme, for the case \(\theta_{t}<10^{0.4}\), is significantly upgraded for a larger \(\gamma\), a lighter fading severity of User \(t\), or a smaller \(d_{t}\). This is because any of these factors corresponds to a more favourable condition. This phenomenon suggests that, in addition to the shadowing, \(d_{t}\), and \(\gamma\), \(\theta_{t}\) must be taken into account to form a flexible NOMA user group and ensure the superiority of NOMA-based satellite networks. Finally, the DQN algorithm was adopted to select one from many users to be User \(t\) and pair them with User \(c\) to form a NOMA user group. Specially, since the assumption that \(Q_{c}<Q_{t}\) must be satisfied, only users with ILS/AS severity were viewed as candidates. Figure 4: Effective capacity of User \(t\) for two access schemes versus \(\theta_{t}\) with various \(\gamma\), \(d_{t}\), and fading severities, when \(d_{c}\in[0.6R,R]\) and \(a_{t}^{p}=1-a_{c}^{p}\). Figure 3: Effective capacity of User \(c\) achieved with TDMA and NOMA schemes versus \(\alpha_{c}^{p}\) under various system parameters. Meanwhile, \(\alpha_{t}^{p}=1-\alpha_{c}^{p}\) varied with the location and fading severity of User \(c\) as well as the transmission average SNR \(\gamma\), as shown in Figure 3.  
The convergences of the proposed DQN algorithm with different learning rates are shown in Figure 5, from which we find that a smaller value of learning rate leads to a faster convergence, since a smaller learning rate means a lower newly acquired cost is accepted to adjust the evaluated \(Q_{\pi}(s_{l},a_{l})\). Thus, \(\tilde{\alpha}=0.01\) was set in our algorithm.