## 3 The Proposed Algorithm
### SARSA for Solving the MDP Model
#### 3.3.3 The Handoff Call Event  
If the policy could ensure an eligible channel for the handoff call of its adjacent cell, the rejection probability of the handoff call was effectively reduced. We assumed that there were 5 channels (\(ch1\)-\(ch5\)) in the LEO satellite network and the minimal reuse distance was 3. The served calls and the channel status are shown in Figure 1(a). When the call on channel 1 of cell 25 handed over, the following three options were available for handling the handoff call event: (1) As shown in Figure 1(b), without a reallocation, the call on channel 1 of cell 25 departed. The status of channel 1 of cell 25 was transformed from 1 to 0 and then the channel was released. According to the relative definitions and the channel status in \(I(26)\), \(\tilde{A}(26)\) was none. Thus, adjacent cell 26 had no eligible channel. As a result, the handoff call was rejected. (2) As shown in Figure 1(c), channel 1 (occupied by the departure call) was reallocated to the call on channel 3 and then channel 3 was released. According to the current channel status in \(I(26)\), \(\tilde{A}(26)\) was none. Thus, adjacent cell 26 still had no eligible channel. As a result, the handoff call was rejected. (3) As shown in Figure 1(d), channel 1 (occupied by the departure call) was reallocated to the call on channel 4 and then channel 4 was released. According to the current channel status in \(I(26)\), \(\tilde{A}(26)=\{ch4\}\). Thus, adjacent cell 26 had an eligible channel \(ch4\). As a result, the handoff call was successfully accepted. If the policy selected option 3, as shown in Figure 1(d), the policy ensured an eligible channel for the handoff call and reduced the rejection possibility of the handoff call.  
When a handoff call event occurred (\(e_{t}\)= \(e_{n,m,k,t}^{handoff}\)), the current state was \(s_{t}=(X_{t},e_{n,m,k,t}^{handoff})\). The departure call event was handled first and then the related new call event was handled. The currently performed action was adjusted by the action for the new call event in the adjacent cell. The policy for the handoff call event was expressed by Equation (12) and Equation (13), respectively.  
\[\pi\Big{(}s_{t}=\Big{(}X_{t_{t}},e_{n,m,k,t}^{handoff}\Big{)}\Big{)}=\pi\Big{(}s_{t }=\Big{(}X_{t},e_{n,k,t}^{end}\Big{)}\Big{)}=\text{argmin}Q(s_{t},a) \tag{12}\]  
\[\pi\Big{(}s_{t}=\Big{(}X_{t},e_{n,m,k,t}^{handoff}\Big{)}\Big{)}=\pi\big{(}s_{t +1}=\big{(}X_{t+1},e_{m,t+1}^{new}\big{)}\big{)}=\text{argmax}Q(s_{t+1},a) \tag{13}\]  
where the first action for the departure call event is expressed by \(\mathfrak{a}_{t}\), corresponding with the minimal state-action value of \(Q(s_{t},a)\) and the second action for the new call event in the adjacent cell is expressed by \(a_{t+1}\), corresponding with the maximal state-action value of \(Q(s_{t+1},a)\). The maximum of \(Q(s_{t+1},a_{t+1})\) and \(Q(s_{t},a_{t})\) was selected to update the current Q-value. The channel occupied by the departure call event was reallocated to the call on the channel represented by the action with the maximum Q-value. The policy for the handoff call event is shown in Algorithm 4.