# BeiDou Short-Message Satellite Resource Allocation Algorithm Based on Deep Reinforcement Learning
## 4 Simulation and Performance Analysis
### Simulation Parameter Setting  
The SMSCS uses MEO satellites with an orbital altitude of 21,528 km and an orbital inclination of 55\({}^{\circ}\). It is distributed in the Walker 24/3/1 constellation, which contains a total of 24 satellites, of which 14 provide GSMC. The scene was built with STK satellite simulation software. Considering future practical application scenarios, 1000 short-message terminals were randomly placed in the Asia-Pacific region at 55\({}^{\circ}\) N-10\({}^{\circ}\) S and 75\({}^{\circ}\) W-135\({}^{\circ}\) E (the red area in Figure 7). In the other region, 100,400 short-message terminals were randomly placed. The short-message terminals in the two areas can randomly choose whether or not to send short-message requests in each snapshot. The size of short-messages transmitted by the short-message terminals obeys a normal distribution with an expectation of \(\frac{C_{\max}}{2}\) and a variance of 1. The network design consists of two parts. The feature extraction network includes two convolution layers (Conv) and three FC layers (FC), which were used to extract the features of the graph tensor. The specific parameters are shown in Table 1.  
The four DDPG networks have the same network structure, in which there are three hidden layers. The loss function adopts the Relu function, and the number of neurons in each layer were set to 16, 32, 64, and 128, respectively. First of all, the deep neural networks with different structures were run to analyze the training efficiency, as shown in Figure 8. The abscissa represents the number of iterations, and the ordinate represents the average loss of the network after continuous learning. The average loss was obtained by exponential moving average (EMA) smoothing, and the smoothed curve better reveals the changing trends of the data. The results show that deep neural networks with different neurons converge after 35,000 iterations of training. However, the structure of 64 neurons in each layer can ensure the minimum network parameters are met to ensure the best convergence performance is achieved. Therefore, we set the number of neurons in the hidden layers to 64, which can also be used as the training structure for the next part of the network performance evaluation. At the same time, we analyze the scalability of DRL-SRA when the number of short message terminals is 1200, 1500, 1800, and 2400, and the impact on the performance of the DRL-SRA algorithm. As shown in Figure 9, it can be found that when the number of short message terminals takes different values, the algorithms can all converge. Moreover, the smaller the number of short message terminals, the faster the convergence speed of the algorithm, but the approximate convergence state can be obtained in the end, indicating that the algorithm has good scalability. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Layer** & **Input** & **Kernel** & **Activation** & **Output** \\ \hline Conv1 & \(\alpha\times\alpha\times 26\) & \(1\times 32\times 26\),4 & Relu & \(\alpha\times 32\times 4\) \\ Conv2 & \(\alpha\times 32\times 4\) & \(\alpha\times 16\times 4\),8 & Relu & \(\alpha\times 16\times 8\) \\ FC1 & \(\alpha\times 16\times 8\) & NA & Relu & 1024 \\ FC2 & 1024 & NA & Relu & 256 \\ FC3 & 256 & NA & Relu & \(14\times v_{\text{max}}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The parameters of feature extraction network. Figure 7: Schematic diagram of the short-message terminal distribution. Finally, when the number of terminals is 1500, we compare the performance of the feature extraction network with the PCA dimensionality reduction model and the random forest feature extraction model based on information gain. As shown in Figure 10, the comparison results show that the algorithm using the feature extraction network has a better convergence effect during training. Figure 8: Convergence rate under different numbers of hidden layer units. Figure 10: Convergence of DRL-SRA algorithm under different feature extraction algorithms.