# A DRL-Based Satellite Service Allocation Method in LEO Satellite Networks
## 3 Problem Statement
\begin{table}
\begin{tabular}{c c} \hline
**Variable** & **Description** \\ \hline \(\mathcal{N}=\{1,2,...,n,...N\}\) & Set of LEO satellites with available resources \\ \(\mathcal{K}=\{1,2,...,k,...K\}\) & Set of users with resource requirements \\ \(\mathcal{D}=\{B^{UE},G^{UE},C^{UE},O^{UE},F^{UE}\}\) & Multi-dimensional resource requirements for \\  & user services \\ \(\mathcal{V}=\{B^{Sat},G^{Sat},C^{Sat},C^{Sat},C^{Sat},F^{Sat}\}\) & Remaining available resources on \\  & LEO satellites \\ \(Loc_{k}^{UE}=(x_{k}^{UE},y_{k}^{UE},z_{k}^{UE})\) & The three-dimensional coordinates of users \\  & with service requests \\ \(Loc_{n}^{Sat}=(x_{n}^{Sat},y_{n}^{Sat},y_{n}^{Sat})\) & The three-dimensional coordinates of \\  & LEO satellites \\ \(d\) & Orbital altitude of LEO satellites \\ \(\alpha_{n,k}\) & The minimum communication elevation angle \\  & between \(UE_{k}\) and \(Sat_{n}\) \\ \(\varphi\) & Indicator for the existence of services on \\ \(l_{n}^{t}=<Loc_{n}^{Sat},V_{n},\varphi>\) & A struct used to describe the state of a \\  & single satellite \\ \(\varepsilon_{k}=<Loc_{k}^{UE},D_{n}>\) & Mathematical model of a UE \\  & The satellite assigned to the \(UE_{t}\) in timeslot \(t\); if \\ \(\pi_{t}\) & unassigned, it is indicated as 0 \\ \hline \end{tabular}
\end{table} 
Table 1: Variable description. \[S=\{s_{1},s_{2},...,s_{t},...,s_{N}\}, \tag{20}\]  
\[s_{t}=<e_{t},I_{N}>. \tag{21}\]  
Each network state \(s_{t}\) can be jointly determined by the network state from the previous moment and the chosen action \(A\). In our study, we set \(A\) as the satellite number \(n\) to which the UE is currently connected in the current state. \[A=\{\pi_{1},\pi_{2},...,\pi_{t},...\pi_{n}\}. \tag{22}\]  
After the execution of action \(\pi_{t}\) in each timeslot \(t\), the state \(S\) is updated based on the amount of resources occupied by the service. This involves updating the resource allocation of the satellite to which the service has been assigned. When a service cannot be allocated to any satellite, we consider the service as incomplete. In such cases, it will move on to the next timeslot, and there is no need to update the satellite resource status. When the MDP concludes and when all services have been allocated, the reward calculation is based on the overall network resource allocation results. Typically, in the SSA problem, evaluation metrics for resource allocation results include resource utilization, the service allocation rate, and others. Our ultimate goal is to allocate suitable resources for as many services as possible while achieving a high level of resource utilization. Therefore, we define the reward as follows:  
\[R=\lambda\times R^{UE}(A)+(1-\lambda)R^{Sat}(A). \tag{23}\]  
where \(R^{UE}\) represents the UE service allocation rate, and \(R^{Sat}\) represents the satellite resource utilization rate. \(\lambda\) is a predefined constant used to control the weighting coefficients of the two metrics. The calculation methods for \(R^{UE}\) and \(R^{Sat}\) can be expressed as follows:  
\[R^{UE}(A)=\frac{1}{K}\sum_{l=1}^{K}\pi_{t}, \tag{24}\]  
Figure 4: MDP schematic diagram. \[R^{Sat}(A)=avg\Bigg{\{}\ M\in\mathcal{D},Z\in\mathcal{V}|\frac{\sum_{k\in\{k|\pi_{k} \
eq 0\}}M_{k}^{UE}}{\sum_{n\in\mathcal{N}}Z_{n}^{Sat}}\Bigg{\}}. \tag{25}\]  
where \(M\) and \(Z\) refer to any element within the sets \(\mathcal{D}=\{B^{UE},G^{UE},C^{UE},O^{UE},F^{UE}\}\) and \(\mathcal{V}=\{B^{Sat},G^{Sat},C^{Sat},O^{Sat},F^{Sat}\}\), respectively. A higher \(R^{UE}\) value indicates that a greater number of user service requests have been satisfied. Furthermore, a higher \(R^{Sat}\) value signifies that satellite resources are being utilized more effectively.