# BeiDou Short-Message Satellite Resource Allocation Algorithm Based on Deep Reinforcement Learning
## 2 System Model
### Terminal Satisfaction Model  
In the previous section, the short-message tasks requested by \(m\) short-message terminals were recorded as the set \(\textbf{Task}=\left\{task_{j1},task_{j2},\cdots,task_{j\psi}\right\}\). We define the short-message task \(task_{i}\) as a triple:  
\[task_{i}\triangleq\left\langle c_{i},size_{i}^{t},\tau_{i}\right\rangle \tag{12}\]  
where \(c_{i}\) is the content of the short-message to be transmitted by \(task_{i}\). \(size_{i}^{t}\) is the size of the transmission task \(task_{i}\) (in bytes). \(\tau_{i}\) is the acceptable processing delay for \(task_{i}\).  
The satisfaction of the short-message terminal depends on the processing speed of the short-message satellite in response to the short-message terminal task request. The factors affecting the processing speed include the short-message transmission delay \(t_{t,i}\), the processing delay \(t_{p,i}\), and the task queuing delay \(t_{q,i}\). Assuming that \(SAT_{j}\) responds to the task request of \(ST_{i}\), Equation (13) shows that the communication rate is \(r_{i,j}\), the size of the transmission task \(task_{i}\) is \(s_{i}\), and the transmission delay is:  
\[t_{i,j}=\frac{size_{i}^{t}}{r_{i,j}}. \tag{13}\]  
The processing delay is:  
\[t_{p,i}=\frac{size_{i}^{t}}{f_{j}} \tag{14}\]  
where \(f_{j}\) is the computing power of the short-message satellite, i.e., the number of bytes processed per unit of time.  
The task queuing delay is:  
\[t_{q,i}=t_{i}^{end}-t_{i}^{start}-t_{p,i}. \tag{15}\]  
Equations (13)-(15) show that the total execution delay for \(task_{i}\) is:  
\[t_{i}^{\Sigma}=t_{t,i}+t_{p,i}+t_{q,i}. \tag{16}\]  
If \(ST_{i}\) does not request a short-message task, \(t_{i}^{\Sigma}=0\).  
Thus, the service satisfaction index for the short-message terminal set \(\mathbf{ST}\) is:  
\[SI=\sum_{i=1}^{m}SI_{i}=\sum_{i=1}^{m}\text{sgn}\left(t_{i}^{\Sigma}-\tau_{i}\right) \tag{17}\]  
where \(SI_{i}\) is the service satisfaction index of \(ST_{i}\).  
The main problem considered in this article is determining how to improve the resource utilization of the SMSCS, meet the quality-of-service requirements of the short-message terminal, and reduce the energy loss of the short-message terminal at the same time. In summary, the objective function of optimization under the snapshot \(t\) is:  
\[U(t)=\alpha_{1}\frac{L_{\min}}{L_{total}(t)}+\alpha_{2}\frac{LI(t)}{L_{\max}}+ \alpha_{3}\frac{SI(t)}{SI_{\max}} \tag{18}\]  
where \(\alpha_{1}\), \(\alpha_{2}\), and \(\alpha_{3}\) are weight values, and \(\alpha_{1}+\alpha_{2}+\alpha_{3}=1\). \(L_{\min}\) represents the minimum value of the overall path transmission loss of the short-message under the snapshot \(t\). \(LI_{\max}\) represents the maximum value of the load balance index. \(SI_{\max}\) represents the maximum value of the terminal satisfaction index. The optimization objectives can be expressed as follows:  
\[\max_{L_{total}(t),LI(t),SI(t)}U(t) \tag{19}\]  
\[\text{C1}:\ \sum_{j=1}^{m}e_{ij}\leq 1,\ \forall i\in\{1,2,\cdots,n\} \tag{20}\]  
\[\text{C2}:\ \sum_{i=1}^{\psi}s_{j,i}\leq\zeta\text{C}_{\max},\ \forall j\in\{1,2,\cdots,m\} \tag{21}\]  
\[\text{C3}:P_{i}\geq l_{ij}e_{ji},\ \forall j\in\{1,2,\cdots,m\},\ \forall i\in\{1,2,\cdots,n\} \tag{22}\]  
\[\text{C4}:s_{i}\leq C_{\max},\ \forall i\in\{1,2,\cdots,n\} \tag{23}\]  
C1 is used to ensure that the short-message task of each short-message terminal in the communication system is answered by, at most, one short-message satellite. C2 is used to ensure that the number of short-message tasks answered by each short-message satellite does not exceed the maximum capacity it can handle. C3 is used to ensure that the energy consumed by the terminal transmission is not greater than the maximal amount of energy contained in the short-message terminal. C4 is used to ensure that the message size transmitted by the terminal does not exceed the maximal length specified by the system.