## 3 Network Handover Algorithm Based on Location Prediction
### Seamless Network Handover  
Q-learning is a reinforcement learning algorithm based on value iteration and independent of the environment state. It relies on the interaction between the agent and the environment to iteratively learn an optimal strategy and make the best decision. We apply this algorithm to network handover. Due to its low complexity and computational simplicity, it significantly simplifies the network handover implementation process. The algorithm identifies the optimal action for each corresponding state by continuing to explore the action-state space until convergence. In this paper, the central controller is considered an agent that performs the assignment of network access points. Based on the network state \(S=(s_{1},s_{2},s_{3})\), the greedy strategy selects the optimal action from set \(A=(a_{1},a_{2},a_{3})\) with probability \(1-\varepsilon\), derived from feedback on state transition and reward. Another action is randomly performed with probability \(\varepsilon\). The detailed parameters are defined as follows:  
State: \(S=(s_{1},s_{2},s_{3})\),where \(s_{1}\) represents the current service network, either VLC or Wi-Fi; \(s_{2}\) indicates the VLC coverage area to which the user will move in the future; and \(s_{3}\) indicates the current quality of service, represented by 0 or 1, depending on whether the SINR is below or above the threshold (10 dB). Action: \(A=(a_{1},a_{2},a_{3})\),where \(a_{1}\) represents handover to the best VLC, \(a_{2}\) represents handover to Wi-Fi, and \(a_{3}\) represents no handover. Reward function: The reward function combines the key elements of handover success, user satisfaction with the data rate, and handover cost. \[R=\mathrm{I}*[\zeta*\beta+(1-\delta)(1-\beta)] \tag{20}\]  
The parameters are defined as follows:  
\(\mathrm{I}\in\{0,1\}\) is the handover success identifier. If the handover is successful, then \(\mathrm{I}=1\); if it fails, \(\mathrm{I}=0\). \(\zeta=\frac{r_{start}}{r_{previous}}\) represents user satisfaction, which indicates the data rate ratio after handover compared to before handover. The higher the value of \(\zeta\), the more significant the improvement in the data rate the user experiences after handover, leading to greater satisfaction with the network service. \(\delta\) represents the handover cost. HHO involves only handover between the APs of the same technology and has a low latency of approximately 200 ms. VHO requires cross-technology negotiation and resource reconfiguration. It incurs a higher latency, approximately 500 ms [26]. Based on the measured data and the literature [26], the handover cost is set to \(\delta_{NHO}=0\), \(\delta_{HHO}=0.3\), and \(\delta_{VHO}=0.7\), respectively, to quantify the difference in resource overhead for different handover types. \(\beta\in\{0,1\}\) is a tradeoff between the data rate and the number of handovers. When \(\beta=1\), the reward function simplifies to \(R=\mathrm{I}*\zeta\). At this point, the reward favors actions that achieve the maximum data rate, with less consideration given to the number of handovers required. In this case, the algorithm will prioritize pursuing a high data rate, even if it may result in more handover actions. When \(\beta=0\), the reward function becomes \(R=1*(1-\delta)\). At this point, the algorithm prioritizes minimizing the number of handovers. It may sacrifice some data rate to reduce handovers. Since handover introduces data overhead in addition to information transmission, it can degrade the system performance. Therefore, this paper introduces a sequence number (SN) state synchronization mechanism and an acknowledgment number (Ack) mechanism within the transmission control protocol [27] during network handover to ensure seamless handover and uninterrupted data transmission, as depicted in Figure 4. The specific implementation process is as follows:  
1. The central controller periodically collects the user's location and network status parameters. 2.