# User Pairing for Delay-Limited NOMA-Based Satellite Networks with Deep Reinforcement Learning
## 1 Introduction
Under these conditions, the concept of effective capacity, which was proposed in [10] as an effective performance metric to show the maximum constant arrival rate with a given delay QoS constraint, has been introduced in various satellite communication scenarios to show the adverse impact of delay QoS limitations on system performance [11; 12; 13; 14], such as the authors in [11], who proposed an algorithm to schedule users in different time slots while guaranteeing users' delay QoS requirements in a satellite-terrestrial backhaul network. In cognitive satellite-terrestrial networks, the effective capacity was introduced to guarantee the delay requirement of a primary user [12], whose extension to study effective energy efficiency of the same networks was studied in [13]. Moreover, work [14] studied the achieved effective capacity of a NOMA-based satellite system with delay adhering to users' service requirements. Although these aforementioned works have shown the negative impact of delay QoS requirements on OMA-/NOMA-based satellite networks, how to select users in a NOMA based system, i.e., whether it is effective to only select users with big channel differences, to form a NOMA pair has not been investigated. It is worth noting that, in addition to free space loss (FSL), antenna gain, fading severity, and location information in a beam spot can also influence the link budget of a satellite user, all of which, combined with users' various delay QoS requirements, make the user grouping in a NOMA-based system nontrivial, especially in satellite networks, which are highly applied in military and civilian fields. To solve this challenge, a supervised learning algorithm, with which solutions can be obtained without model-oriented analysis and design, as an effective solution for resource management has been widely used in several prior works, such as work [15], which proposed a genetic algorithm (GA)-improved support vector machine scheme to effectively pair users for NOMA-based satellite networks. A fully connected deep neural network-assisted approach was studied in [16; 17] to facilitate efficient beam hopping and design beam illumination pattern in multibeam satellite systems, respectively. The work in [18] proposed an accurate forecasting method by using deep neural networks for LEO satellite links. Notably, supervised learning, such as the algorithms used in [15; 16; 17; 18], needs to learn characteristics from input data and desired output data, while a reinforcement learning (RL) algorithm, which is model-free and data-driven, has been extensively adopted in various wireless networks with different objectives. For example, based on \(Q\)-learning, an algorithm for jointly optimizing user pairing and power allocation was proposed in [19] to maximize the total sum rate of a satellite random access system. Considering large-scale low-earth orbit constellations, the work in [20] developed a low-complexity successive deep Q-learning algorithm for optimal satellite handover. The authors in [21] proposed a \(Q\)-learning NOMA-based random access scheme for time slot and channel allocation in satellite-terrestrial relay networks. In [22], the authors adopted a graph neural network and RL algorithms in a hybrid satellite-terrestrial network to optimize UAV trajectory and maximize the number of served users. In [23; 24], the authors conducted resource management in a relay-aided network with the help of distributionally robust deep RL (DRL) and enhanced DRL algorithms, respectively. Motivated by these observations, for the work herein, we leaned upon a DRL algorithm to pair users and provide services with various delay QoS requirements for future NOMA-based satellite networks (since this paper's aim was to pair users in delay-limited NOMA-based satellite networks with a DRL algorithm, while the impacts of low-density parity check codes [25] in NOMA-based satellite networks will be our follow-up research.).