# User Pairing for Delay-Limited NOMA-Based Satellite Networks with Deep Reinforcement Learning
## 1 Introduction  
Due to the inherent nature of providing vast coverage and economic service, satellite communication has the ability to effectively supplement terrestrial networks during disasters and in rural and deserts areas; thus, it has been considered as an important component for next-generation wireless networks [1]. However, the dramatically increased demand for data access can result in even bigger challenges, including massive connectivity, limited power/spectral resources, and various quality of service (QoS) requirements, in future satellite networks. Recently, non-orthogonal multiple access (NOMA) schemes, including power domain NOMA [2] and code domain NOMA [3], featuring multiple access, high resource utilization efficiency, and user fairness, has become a promising solution to alleviate these challenges faced by future satellite networks. Of these two schemes, power domain NOMA (or simply NOMA for short) scheme, which has the ability to harmoniously integrate with orthogonal multiple access (OMA) techniques in existing satellite architectures, is the main motivation and focus of this article. In a NOMA-based satellite network, a satellite/multiple users can simultaneously communicate with multiple users/a satellite in downlink/uplink transmissions by superposing various signals with different power levels in the same time/spectral block. To date, many works have investigated the performance enhancement of various NOMA-based satellite networks, such as the improved outage probability in integrated satellite terrestrial relay networks with perfect successive interference cancellation (SIC) [4], augmented ergodic capacity in uplink satellite communications [5], and increased network utility in the satellite-based internet of things [6]. An extension of work [4] to an imperfect SIC scenario with Alamouti space-time block coding was studied in [7]. Moreover, some works studied resource management of the NOMA-based satellite network from the perspective of increasing system resource efficiency, e.g., aiming to maximize long-term age of information, the authors in [8] utilized a ListNet algorithm and a particle swarm optimization algorithm to obtain an optimized power allocation solution in a satellite-based internet of things scenario. Similarly, the work in [9] proposed a joint subchannel assignment and power allocation algorithm to further optimize the sum rate of a secondary network in cognitive satellite-unmanned aerial vehicle-terrestrial networks. Although NOMA-based satellite networks can enhance spectrum/power utilization efficiency and a system's performance, we must note that these Shannon performance enhancements were achieved by selecting users with distinctive differences in channel gains to form a NOMA pair, which is only suitable for a system with delay-insensitive applications. However, with rising technological developments in wireless communications, new satellite applications with diverse delay QoS requirements have occurred to facilitate our daily life and provide more efficient service, such as applications on smart grids, environmental monitoring and forecasting, navigation, smart cities, and telemedicine. Among these applications, telemedicine and smart grids are identical delay-critical scenarios, while environmental monitoring is a typical delay-tolerant scenario. Thus, Shannon capacity, which fails to take users' diverse delay QoS requirements into consideration, is no longer suitable to use in future satellite networks to characterize the performance of real-time and delay-sensitive applications/scenarios, and it is of paramount importance to study the achievable performance of satellite networks under heterogenous delay QoS requirements.