## 3 Proposed Approach
State prediction:  
Update particle distances using UE velocity \(v_{j}\) and random noise:  
\[d_{k}^{(t+1)}=d_{k}^{(t)}+v_{j}\cdot\Delta t+\xi_{k}, \tag{7}\]  
where  
\(v_{j}\): Velocity of UE jj. \(\Delta t\): Time step. \(\xi_{k}\sim\mathcal{N}\big{(}0,\sigma^{2}\big{)}\) is Gaussian noise. Weight update:  
Update particle weights based on the likelihood of the measured _RSSI_:  
\[w_{k}^{(t+1)}=\frac{\exp\left(-\frac{\left(RSSI_{\text{measured}}-RSSI_{k}^{(t+ 1)}\right)^{2}}{2\sigma^{2}}\right)}{\sum_{j=1}^{K}\,\exp\left(-\frac{\left( RSSI_{\text{measured}}-RSSI_{j}^{(t+1)}\right)^{2}}{2\sigma^{2}}\right)}. \tag{8}\]  
where the following apply:  
\(w_{k}^{(t+1)}\): Weight of particle \(k\) at time \(t+1\). \(RSSI_{\text{measured}}\): Observed signal strength. Resampling:  
Resample particles based on their weights to concentrate around high-likelihood regions. Distance Estimation:  
The estimated distance is the weighted average:  
Figure 2:0  
where the following apply:  
\(\hat{d}_{i,j}(t)\): Estimated distance between UE \(j\) and satellite \(i\) at time \(t\). \(w_{k}^{(t)}\): Weight of particle \(k\) at time \(t\). \(d_{k}^{(t)}\): Distance of particle \(k\) at time \(t\). Based on the estimated distance \(\hat{d}_{i,j}(t)\) and location data, the handover decision is made as follows:  
Hysteresis condition:  
Figure 2:1  
Time-to-trigger condition:  
Figure 2:2  
Reinforcement learning Framework, elements of reinforcement learning  
State Space \(S(t)\), includes the following:  
Figure 2:3  
where the following apply:  
\(\hat{d}_{i,j}(t)\): Estimated distance. \(RSSI_{i,j}(t)\): Observed signal strength. \(p_{j}(t)\): UE position vector. \(v_{j}(t)\): UE velocity. Action Space \(A(t)\), Actions involve adjusting hysteresis and \(TTT\):  
Figure 2:4  
where the following apply:  
_H_: Hysteresis threshold. _TTT_: Time-to-trigger. Reward Function \(R(t)\): Combines throughput, handover failures, and ping-pong effects:  
\[ \begin{split} R(t)&=w1\cdot Throughput(t)-w2\cdot Handover \text{Failures}(t)\\ &-w3\cdot PingPongRate(t)\end{split}  \tag{14}\]  
where the following apply:  
_w1, w2, w3_: The weights for each performance metric. Objective: Maximize long-term cumulative reward:  
\[G(t)=\sum_{k=0}^{\infty}\gamma^{k}R(t+k) \tag{15}\]  
where \(\gamma\) is the discount factor. The policy is updated using a policy gradient method:  
\[\theta\leftarrow\theta+\alpha\
abla\theta E[G]] \tag{16}\]  
where the following apply:  
_\(\theta\)_: Policy parameters. _\(\alpha\)_: Learning rate. _G_: Discounted cumulative reward. _\(\gamma\)_: Discount factor. In our proposed LBH-PRL approach, we adopted a policy gradient method to dynamically optimize handover decisions. Unlike value-based methods such as Q-learning, which rely on discrete Q-values for action selection, policy gradient methods directly learn an optimal policy by adjusting the probability distribution of actions based on observed rewards. This approach is particularly well-suited for NTN environments, where handover decisions require continuous parameter tuning, such as dynamically adjusting hysteresis and \(TTT\). To ensure stable convergence and robustness, we designed a reward-shaping mechanism that penalizes unnecessary handovers while prioritizing service continuity. The accumulated reward considers multiple factors, including throughput, handover failure rate, and ping-pong rate. This enables the RL model to optimize handover strategies by learning a probability distribution over actions rather than selecting the action with the highest Q-value, making it more adaptive to dynamic NTN conditions. To enhance training efficiency and policy stability, we fine-tune key hyper parameters. The learning rate is set to 0.01, ensuring a balance between convergence speed and stability, while the discount factor is set to 0.95 to prioritize long-term optimization of handover decisions. Additionally, our approach employs stochastic policy updates with policy gradients, where action selection probabilities are continuously refined through gradient updates.