# BeiDou Short-Message Satellite Resource Allocation Algorithm Based on Deep Reinforcement Learning
###### Abstract
[21] solved the joint network stability and resource allocation optimization problems of high-throughput satellites (HTSs). [PERSON] et al. [22] achieved joint optimization of wireless information resource allocation and power transmission of multi-beam solar satellites through particle swarm optimization (SPO), the improved harmony search algorithm (IHSA), and the monkey algorithm (MA), and analyzed SPO, ISHA, and MA algorithms. The results showed that the IHSA algorithm can maximize power transmission without affecting information transmission. However, the above work [20; 21; 22] did not consider the transmission power consumption and the quality of service of the task initiator. There has been little research on short-message satellite resource allocation. [PERSON] et al. [23] proposed a task-oriented satellite network resource allocation algorithm (SAGA) based on the GA. [PERSON] et al. [24] combined this with the BDS3-SMC to form a short-message transmission mechanism and solved the problem of short-message satellite resource allocation by improving the Hungarian algorithm. However, for scenarios with a large number of terminals, the applicability of the algorithm is poor. With the development of artificial intelligence technology, deep reinforcement learning (DRL) has made a substantial breakthrough in many tasks that need to interpret high-dimensional raw input data and implement sequential decision-making control [25]. Researchers have proven the effectiveness of DRL in many fields. Preliminary applications of DRL include resource allocation in the Internet of Things [26], heterogeneous cellular networks [27], and 5 GHetNet uplink/downlink [28]; dynamic beam-hopping of satellite broadband systems [29]; and edge computing in the Internet of Things [30]. DRL has frequently been used in research work to optimize satellite resources. In fact, the SRA problem can be modeled as an interaction between the satellite system and the user terminal service, where the best solution to the problem is equivalent to the maximal cumulative reward that the agent (satellite system or user terminal) can get from the environment. In terms of multi-agent environments, DRL has been considered a solution for cognitive radio networks [31]. [PERSON] et al. [32] proposed a reinforcement learning algorithm based on a deep neural network to solve the multi-objective optimization problem of resource allocation in cogntive satellite communication. [PERSON] et al. [29; 33; 34] used DRL to make dynamic decisions for hopping beams in multi-beam satellite systems and next-generation broadband satellite systems, which have a lower level of complexity than traditional algorithms. He also proposed a resource allocation framework for multi-beam satellite systems based on DRL. In contrast, [PERSON] et al. [35] proposed a dynamic satellite power allocation method based on DRL to minimize system power consumption. [PERSON] et al. [36] proposed an optimization method to balance energy, power consumption, and efficiency in heterogeneous computing systems through reinforcement learning, and carry out hardware simulation experiments based on FGPA. The results show that reinforcement learning can greatly reduce system energy consumption without affecting hardware performance. [PERSON] et al. [37] proposed a multi-objective optimization algorithm based on deep reinforcement learning (DRL-MOP), which achieves multi-objective joint optimization of the satellite spectrum efficiency and improvements in energy efficiency and the service satisfaction index. Compared with the traditional GA and SA algorithms, it has been verified that the DRL-MOP algorithm has the characteristics of fast convergence and low complexity. [PERSON] et al. [38] proposed a software-defined satellite-terrestrial network (STN), which can be used to coordinate a satellite cache and computing resources, and can be combined with the DQN algorithm to optimize the cache and computing resources jointly.