## 4 Reinforcement Learning Algorithms for HO
### Reinforcement Learning  
RL means to inflate the rewards by taking number of actions in environment. This learning involves performing those actions which maximize these rewards. This type of learning behaves same as natural learning where agent must learn by himself through hit and trial mechanism for maximum reward [79]. Supervised, unsupervised, and semi-supervised are the classifications of ML. RL (semi-supervised) is different from supervised and unsupervised learning. In supervised learning, there are set of instruction for each action and the objective is to map the input correspondent output and learn the rules from labelled data. Regressive and classification model are used in this category depends upon whether the value is continuous or discrete. While in case of unsupervised learning, agent must discover the hidden structure for unlabeled data [80]. Unsupervised learning is vice versa to supervised learning and can be applied typically when data are insufficient and is not labelled. But in case of RL, the agent has initial and end points and to reach its destination, agent must find best possible actions by manipulating the environment. After reaching the final solution, agent receive the rewards but if he fails to reach, he does not get any reward, so agent must learn environment to receive maximum rewards [81]. In RL, the problem formulation is done using markov decision process (MDP) and the solution can be \begin{table}
\begin{tabular}{c c c} \hline \hline
**Schemes** & **Advantages** & **Disadvantages** \\ \hline Hard HO & Efficient user of spectrum & Short interruption of service \\  & No data overhead & Sensitive to link transfer time \\  & & (may result in dropped call) \\ \hline Seamless HO & Reliable (no service interruption) & Inefficient use of \\  & & spectrum Data overhead \\ \hline Soft HO & Highly reliable & Data overhead \\  & No loss of QoS in HO & Inefficient use of spectrum \\ \hline Predictive Rerouting & Minimized HO latency & Signaling over-head \\  & & Possible data overhead \\ \hline Static GC & Reserved channels for HO & Possible under-utilization \\  & & of spectrum \\ \hline Dynamic GC & Reserved channels for HO & Signaling and \\  & & computational overhead \\  & More efficient use of spectrum & \\ \hline Queuing Schemes & Easy to Implement (FIFO) & Degradation of channel \\  & Queue reorder according to & disregarded (FIFO Queue) \\  & degradation of channel & Signaling and \\  & & computational overhead \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Advantages and Disadvantages of HO Schemes. policy or model base and can be model free i.e., Q-learning, SARSA. In this technique, the agent interacts with environment and generate policy based on rewards and at the end, systems is trained and delivers improved performance [82; 83]. \[Q_{t+1}(s,a)\leftarrow(1-\alpha)Q_{t}(s,a)+\alpha[R_{t+1}+\lambda\frac{max}{ \alpha^{\prime}eA}Q_{t}(s^{\prime},a^{\prime})] \tag{1}\]  
In Equation (1), sum of old \(Q\)-value and learned \(Q\)-value provides an updated \(Q\)-value. Where \(Q_{t}(s,a)\) is a old \(Q\)-value, \(Q_{t+1}\)\((s^{\prime},a^{\prime})\) is an updated \(Q\)-value, and \(R_{t+1}\)\(+\lambda\frac{max}{\alpha^{\prime}eA}\)\(Q_{t}\)\((s^{\prime},a^{\prime})\) is a learned value. Moreover \(\alpha\) is a learning rate that can be set between 0 to 1, where 0 means never updated and 1 means quickly learning. While \(\lambda\) is a discount factor that also can be set between 0 to 1, where 0 means considering current rewards and 1 means striving for a long-term high reward. The intermediary scheme in which labeled and unlabeled data are exploited for the training is known as semi-supervised learning [84]. In deep learning (DL), the rules are established using neurons operation that approximate at the complex function. In mobile communication, DL has significant importance to tackle with complex nonconvex challenges and high computational problem [85].