# A Hybrid Handover Scheme for Vehicular VLC/RF Communication Networks
## 3 Hybrid Handover Scheme
### The MDP Setup
While the state is in transit to \((s,s^{\prime},h)=(\text{Off},\text{On},2)\), \(p(i)\) can be \(\{1,0,-1\}\), where \(1\) means the VTM is waiting for the link recovery of the host VLC AP, and \(-1\) and \(0\) represent the early HHO and VHO processes, respectively. Since early HHO can be executed for \(h=2\) only, the action \(p(i)\) can be taken from \(\{1,0,(-1)(h-1)\}\) if it transits to a new state. We aim to find the optimal policy \(\mathbf{p}^{*}\) to maintain a high speed transmission rate with efficient handover cost. Let \(\text{Pr}_{i\to i^{\prime}}\{(\text{On},\text{Off},1),p(i)\}\) denote the transition probability from state \(i\) to \(i^{\prime}\) with action \(p(i)\). The transition probability for \(i=(s,s^{\prime},h)=(\text{On},\text{Off},1)\) is given by  
\[\text{Pr}_{i\to i^{\prime}}\{(\text{On},\text{Off},1),p(i)\}= \begin{cases} \frac{\lambda}{\lambda+\gamma_{1}+\hat{\alpha}_{12}+\mu_{(\text{On},\text{Off },1)}}&i^{\prime}=(\text{On},\text{Off},1,b+1,|p(i)|)\\ \frac{\gamma_{1}}{\lambda+\gamma_{1}+\hat{\alpha}_{12}+\mu_{(\text{On},\text{ Off},1)}}&i^{\prime}=(\text{Off},\text{Off},1,b,|p(i)|)\\ \frac{\hat{\alpha}_{12}}{\lambda+\gamma_{1}+\hat{\alpha}_{12}+\mu_{(\text{On}, \text{Off},1)}}&i^{\prime}=(\text{On},\text{On},2,b,|p(i)|)\\ \frac{\mu_{(\text{On},\text{Off},1)}}{\lambda+\gamma_{1}+\hat{\alpha}_{12}+ \mu_{(\text{On},\text{Off},1)}}&i^{\prime}=(\text{On},\text{Off},1,b-1,|p(i)|) \end{cases} , \tag{2}\]  
where \(\mu_{(\text{On},\text{Off},1)}=(1-|p(i)|)\mu_{0}+\frac{1}{2}p(i)(1+p(i))\mu_{1}\). \(\lambda+\gamma_{1}+\hat{\alpha}_{12}+\mu_{(\text{On},\text{Off},1)}\) is the sum of the transition rates, where \(\hat{\alpha}_{12}=[1-\frac{1}{2}|p(i)|(1-p(i))]\mu_{12}\). \(\hat{\alpha}_{12}\) is formulated for states \((s,s^{\prime},1)\) to avoid the action of accessing to the next VLC AP, since it is not available. The transition probability for \((s,s^{\prime},h)=(\text{On},\text{Off},2)\) is given by  
\[\text{Pr}_{i\to i^{\prime}}\{(\text{On},\text{Off},2),p(i)\}= \begin{cases} \frac{\lambda}{\lambda+\gamma_{1}+\hat{\beta}_{-1}+\hat{\alpha}_{21}+\mu_{( \text{On},\text{Off},2)}}&i^{\prime}=(\text{On},\text{Off},2,b+1,|p(i)|)\\ \frac{\lambda+\gamma_{1}+\hat{\beta}_{-1}+\hat{\alpha}_{21}+\mu_{(\text{On}, \text{Off},2)}}{\lambda+\gamma_{1}+\hat{\beta}_{-1}+\hat{\alpha}_{21}+\mu_{( \text{On},\text{Off},2)}}&i^{\prime}=(\text{Off},\text{Off},2,b,|p(i)|)\\ \frac{\mu_{(\text{On},\text{Off},2)}}{\lambda+\gamma_{1}+\hat{\beta}_{-1}+ \hat{\alpha}_{21}+\mu_{(\text{On},\text{Off},2)}}&i^{\prime}=(\text{On}, \text{Off},2,b-1,|p(i)|)\\ \frac{\mu_{(\text{On},\text{Off},2)}}{\lambda+\gamma_{1}+\hat{\beta}_{-1}+ \hat{\alpha}_{21}+\mu_{(\text{On},\text{Off},2)}}&i^{\prime}=(\text{On},\text{ Off},2,b-1,|p(i)|)\end{cases} , \tag{3}\]  
where \(\mu_{(\text{On},\text{Off},2)}=(1-|p(i)|)\mu_{0}+\frac{1}{2}p(i)(1+p(i))\mu_{1}\). The transition probability for \((s,s^{\prime},h)=(\text{On},\text{On},2)\) is written as  
\[\text{Pr}_{i\to i^{\prime}}\{(\text{On},\text{On},2),p(i)\}= \begin{cases} \frac{\lambda}{\lambda+\gamma_{1}+\gamma_{-1}+\hat{\alpha}_{21}+\mu_{(\text{ On},\text{On},2)}}&i^{\prime}=(\text{On},\text{On},2,b+1,|p(i)|)\\ \frac{\gamma_{1}}{\lambda+\gamma_{1}+\gamma_{-1}+\hat{\alpha}_{21}+\mu_{(\text{ On},\text{On},2)}}&i^{\prime}=(\text{Off},\text{On},2,b,|p(i)|)\\ \frac{\gamma_{-1}}{\lambda+\gamma_{1}+\gamma_{-1}+\hat{\alpha}_{21}+\mu_{(\text{ On},\text{On},2)}}&i^{\prime}=(\text{On},\text{Off},1,b,|p(i)|)\\ \frac{\mu_{21}}{\lambda+\gamma_{1}+\gamma_{-1}+\hat{\alpha}_{21}+\mu_{(\text{ On},\text{On},2)}}&i^{\prime}=(\text{On},\text{Off},1,b,|p(i)|)\\ \frac{\mu_{(\text{On},\text{On},2)}}{\lambda+\gamma_{1}+\gamma_{-1}+\hat{\alpha}_{ 21}+\mu_{(\text{On},\text{On},2)}}&i^{\prime}=(\text{On},\text{On},2,b-1,|p(i)|) \end{cases} , \tag{4}\]  
where \(\mu_{(\text{On},\text{On},2)}=(1-|p(i)|)\mu_{0}+|p(i)|\mu_{1}\). The transition rate for \((s,s^{\prime},h)=(\text{Off},\text{On},2)\) is formulated as  
\[\Pr_{i\to i^{\prime}}\{(\text{Off},\text{On},2),p(i)\}=