# A Hybrid Handover Scheme for Vehicular VLC/RF Communication Networks
## 3 Hybrid Handover Scheme
### The MDP Setup  
The state space for the VTM access state in the vehicular VLC/RF communication networks is defined as follows:  
\[\Omega=\{(s,s^{\prime},h,b,w),s,s^{\prime}\in\mathcal{S},h\in\mathcal{H},b\in \mathcal{B},w\in\mathcal{W}\} \tag{1}\]  
where \(s\) and \(s^{\prime}\) represent the availability of the host VLC AP and the next VLC AP, respectively. An ON-OFF model is adopted to represent the intermittent VLC links, where ON and OFF represent available VLC links and unavailable links, respectively. Therefore, \(\mathcal{S}=\{\text{On},\text{Off}\}\) indicates the accessibility of VLC APs. The RF AP is assumed to always be available, thus not specified in the state space [14]. \(h\in\mathcal{H}=\{1,2\}\) is a location sign, with \(h=2\) indicating the VMT located in the overlapping area of two adjacent VLC APs as shown in Figure 1 and 1 indicating the non-overlapping regions. \(b\in\mathcal{B}=\{0,1,\cdots,B\}\) is the packet number in the queue, with \(B\) the network buffer size. \(w\in\mathcal{W}=\{0,1\}\) represents the transmission mode, where \(0\) means the VTM is connected to the RF AP and \(1\) is the VLC AP. State transitions occur because of environmental variations, including packet arrival and departure, and changing optical channel. Figure 2 shows the MDP state transitions considering \(6\) states denoted as \((s,s^{\prime},h)\) for a certain queue length \(b\). Obviously, the VTM is located in the overlapping areas of two adjacent VLC APs for states \((\text{On},\text{On},2)\) and \((\text{Off},\text{On},2)\). The arrival process of the packets is modeled as a Poisson process with arriving rate \(\lambda\) (packets/s). The service processes of the RF APs and VLC APs are also Poisson, with service rate \(\mu_{0}\) (packets/s) and \(\mu_{1}\) (packets/s), respectively [27]. Let \(\gamma_{1}\) (s\({}^{-1}\)) and \(\beta_{1}\) (s\({}^{-1}\)) be the rates of the host VLC AP changing from available to unavailable and the inverse based on the ON-OFF model. Statistically, \(1/\gamma_{1}\) and \(1/\beta_{1}\) are the average durations of the host VLC AP staying ON and OFF, respectively. Similarly, \(\gamma_{-1}\) (s\({}^{-1}\)) and \(\beta_{-1}\) (s\({}^{-1}\)) represent the changing rate of the next VLC AP, from available to unavailable and the inverse. The availability of the host VLC AP and the next VLC AP is assumed to be independent. A Poisson process with parameter \(\alpha_{21}\) is used to describe the changes in the VTM from the VLC APs' overlapping areas to the non-overlapping area. The reverse process is described by a Poisson process with parameter \(\alpha_{12}\). These two changing rates depend  
Figure 2: Illustration of states of the Markov process. on the vehicle velocity and the coverage area of the VLC APs. Since the overlapping area of VLC APs is commonly smaller than the non-overlapping area, \(\alpha_{21}\geq\alpha_{12}\). The HHO execution process takes place in the overlapping area denoted by \(h=2\). Normal HHO is assumed to have been executed when the VTM drives into the non-overlapping area from the overlapping area if the target VLC APs are accessible. Once the HHO operation is complete, the original target VLC AP becomes the host VLC AP, and the new next VLC AP is evidently not accessible. In the continuous Markov chain, it is assumed that two events can never occur simultaneously [11; 28]. The action space for the MDP is \(\mathcal{A}=\{-1,0,1\}\), where \(0\) indicates the wireless access to RF APs, and \(1\) and \(-1\) indicated the connection to the host VLC AP and the next VLC AP, respectively. Let \(p(i)\) denote the action taken at the beginning of the state \(i\). A policy, denoted by \(\mathbf{p}=(\cdots,p(i),\cdots)\), refers to the actions for all states. The handover decision is taken by the action \(p(i)\) and the current transmission mode \(w\). For instance, assume both the host and the next VLC APs are available \((s,s^{\prime},h)=(\text{On},\text{On},2)\) and the VTM is connected to the host VLC AP (\(w=1\)) at the beginning.