# Caching-Aware Intelligent Handover Strategy for LEO Satellite Networks
## 3 Caching-Aware Intelligent Handover Strategy
### Intelligent Handover Strategy with Multiple Attributes  
The handover decisions will affect the resource utilization of the connected satellite resources, so the handover decisions at each time slot will also be affected by the decisions at the previous time slots; thus, the process can be modeled as the Markov decision process. In order to achieve a dynamic and continuous handover decision that takes into account multiple factors in a continuous environmental state, we propose an intelligent handover strategy based on the DRL. The strategy can intelligently decide when will the handover be activated and select the optimal satellite based on the resource utilization information of the candidate satellite. It fully takes into account the joint-effect of multiple attributes which include the remaining service time, the number of idle channels, and the remaining caching capacity. Among these attributes, two attributes are the state of the candidate satellite (the remaining service time and the number of idle channels), and another attribute is the state of the serving satellite (the remaining caching capacity). Moreover, the handover strategy can make continuous decisions for the dynamic LEO network. The network structure is shown in Figure 7, and the detailed explanation for the DQN training process can be found in Reference [26]. We need to train this decision network in advance and use the parameter-stabilized DRL network to output the decision actions. During the actual handover process, the satellite simply feeds the environment information from the mobile terminal periodically into the DRL network, and the corresponding handover decision will be obtained. The specific design of an intelligent handover decision network based on DRL, which is shown in Figure 7, can be denoted as a tuple (\(S\), \(A\), \(P\), \(R\)). \(S\) denotes the state space of the LEO system. \(A\) is the handover action space. \(P\) is the space of the state transition probability. \(S\) denotes the reward of state and action. **State space (S)**: the state space is derived from the environment of the LEO satellite communication system. There are four environment state quantities in this paper: the satellite label \(\Omega\), the number of remaining idle channels \(\Theta\), the remaining cache capacity \(C\), and the remaining service time T. Firstly, the satellite label is used to express which satellite is currently connected to the user, and which satellites are candidates. The number of remaining idle channels, remaining cache capacity, and the remaining service time are the factors to be taken into account for handover. And each state consists of the corresponding state of the serving satellite and the handover candidate satellites. The state at time slot \(p\) can be defined as  
Figure 7: The intelligent handover network based on DQN. \[s_{p}=\{\Omega_{p},\Theta_{p},C_{p},\mathrm{T}_{p}\}. \tag{14}\]  
\(\Omega_{p}\) is the set of satellite label, which can be expressed as \([\Omega_{p}^{*},~{}\Omega_{p}^{1},~{}\Omega_{p}^{2},~{}\ldots,~{}\Omega_{p}^{K }]\). \(\Omega_{p}^{*}\) is defined as \([\Omega_{1,p^{\prime}}^{*},~{}\Omega_{2,p^{\prime}}^{*},~{}\ldots,~{}\Omega_{ i,p}^{*}]\). The upper corner of the status variable symbol is used to refer to the serving satellite or the candidate satellite. \"*\" represents the relevant status of the serving satellite which connected to the user. The lower corner \"\(i\),\(p^{\prime}\)\" indicates the user number \(i\) and the time slot \(p\). So, \(\Omega_{i,p}^{*}\) is used to express the serving satellite label of user \(i\). \(\Omega_{i,p}^{*}\in\{1,~{}2,~{}3,~{}\ldots,M\}\), \(M\) is the total number of the LEO satellites. \(\Omega_{p}^{K}\) is defined as \([\Omega_{1,p^{\prime}}^{K},~{}\Omega_{2,p^{\prime}}^{K},~{}\ldots,~{}\Omega_{ i,p}^{K}]\). The upper corner \"\(k\)\" indicates the relevant status of the candidate satellite \(k\). For example, the maximum number of visible candidate satellites for the user in this paper is 3, then the upper corner is 1 or 2 or 3.