# BeiDou Short-Message Satellite Resource Allocation Algorithm Based on Deep Reinforcement Learning
###### Abstract  
The comprehensively completed BDS-3 short-message communication system, known as the short-message satellite communication system (SMSCS), will be widely used in traditional blind communication areas in the future. However, short-message processing resources for short-message satellites are relatively scarce. To improve the resource utilization of satellite systems and ensure the service quality of the short-message terminal is adequate, it is necessary to allocate and schedule short-message satellite processing resources in a multi-satellite coverage area. In order to solve the above problems, a short-message satellite resource allocation algorithm based on deep reinforcement learning (DRL-SRA) is proposed. First of all, using the characteristics of the SMSCS, a multi-objective joint optimization satellite resource allocation model is established to reduce short-message terminal path transmission loss, and achieve satellite load balancing and an adequate quality of service. Then, the number of input data dimensions is reduced using the region division strategy and a feature extraction network. The continuous spatial state is parameterized with a deep reinforcement learning algorithm based on the deep deterministic policy gradient (DDPG) framework. The simulation results show that the proposed algorithm can reduce the transmission loss of the short-message terminal path, improve the quality of service, and increase the resource utilization efficiency of the short-message satellite system while ensuring an appropriate satellite load balance. BeiDou short-message; deep reinforcement learning; resource allocation; multi-objective optimization 23 June 2021 20 July 2021 22 July 2021 22 July 2021 2390 22021 2390 22021 22 July 2021 2390 22021 2390 22021 2390 22021 2021 2021 2021 2021 2021 2021 2021 2021 20  
request delay by up to hundreds of milliseconds. However, to determine how to reasonably allocate the global short-message processing resources of the BDS3-SMC, improve the resource utilization rate, and ensure the service quality of the short-message terminal, further study is required. The existing satellite resource allocation (SRA) algorithms can be divided into traditional algorithms and artificial intelligence algorithms. Research involving the use of traditional optimization algorithms in satellite resource allocation is quite advanced. Developed algorithms include the genetic algorithm (GA) [10], simulated annealing algorithm (SA) [11], non-dominated sorting genetic algorithm (NSGA) [12], random geometry [13], and game theory [14; 15]. [PERSON] et al. [16] established the satellite system power allocation optimization problem and used Lagrangian duality theory to optimize the total system capacity. Similarly, [PERSON] et al. [17] applied Lagrangian theory to Karush-Kuhn-Tucker (KKT) conditions. [PERSON] et al. [18] achieved multi-objective joint optimization of the energy efficiency (EE) and spectral efficiency (SE) of multi-beam satellites. At the same time, it was proven that the resource allocation problem in a multi-objective constraint scenario is an NP-hard problem. Therefore, heuristic algorithms such as the GA, SA, and NSGA can be widely used in satellite resource allocation scenarios. [PERSON] et al. [19] proposed a multi-objective optimization strategy to minimize the power consumption of user terminals and satellites by using a meta-heuristic algorithm to reach a Pareto optimal solution. However, the calculation delay of the algorithm is long, and it is difficult to meet the requirements of real-time processing on the satellite using this method. Based on the above problems, [PERSON] et al. [20] designed a continuous convex approximation algorithm to solve the multi-objective optimization problem of power distribution for energy-sensing of multi-beam satellites. This algorithm has a fast convergence speed and can be used for the dynamic allocation of satellite resources. By combining the particle swarm optimization algorithm and the Lyapunov optimization framework, [PERSON] et al.