## 4 Reinforcement Learning Algorithms for HO
### Reinforcement Learning
As neural network is used for feature extraction and learning phase so this algorithm can be used in number of scenarios i.e., non-linear model enhancement, continuously varying mobile environment evaluation, degree of overfitting and complexity reduction, and reconstruction error of the data minimization [86]. DRL is the revolutionary and emerging tool in many fields of sciences particularly in mobile communications for efficiently deliver the solution of various challenges [87]. Deep convolution neural network (DNN) intends to learn the characteristics of the channel and forecast the appropriate modulation coding scheme (MCS). For intelligent decision without human mediation the multiple layers employed to build an artificial neural network. For improving parameters of the network, the artificial intelligence, machine/deep learning techniques are the best approach as we supposed fewer physical intervention and advanced computational constraints [88]. Nowadays advances networks such as HetNets, IoT, and unmanned aerial vehicle (UAV) networks reshaped to autonomous, adhoc, and decentralized form in which mobile users, UAVs and IoT devices take decisions by theirself i.e., cell association, power control, data rat etc. In these scenarios, the problems sculpted by MDP has worth to make decisions accordingly and number of algorithms and learning techniques helped to solve MDP [89]. Computational complexity of advanced and large networks turns out to be very difficult. In this regard, DRL delivers some required benefits such as decision making independently, improves the learning speed with large state and action spaces, learn and develop network understanding about the communication and environment, sophisticated network optimizations, data offloading, interference management and cyber physical attacks modeled. DRL-based joint resource management functions for 5G HetNets, multi-objective DRL based resource management, flexible resource management design, DRL-based load balancing for 5g HetNets need to be investigated under the 5G context [90]. Figure 3 shows categories of HO optimization techniques using machine learning tools. For the prediction analysis the artificial intelligence needs more matureness in channel modeling. The main issues are i.e., high dimensional search due to huge antenna, transmitted and received signal relationship, learning faster combination of transmitting and receiving beam, convergence in training the AI model. The advance techniques of AI/ML/DL energize the 5G and beyond 5G wireless networks to support emerging use cases introduces in the real world. However, despite of advancement the open research issues and future directions still need to be address. In practical implementation the efficiency of the training procedure needs matureness such as getting faster convergence with the best possible parameters of the learning algorithms. To acquire data from extensive measurement operations there is a still gap for real experiments results from dense urban propagation areas, high speed moving nodes on terrestrial areas and dynamically changing environments to justify the precision of the learning algorithms [91]. In hierarchical net works, architecture design, control of the communication parameters of network entities, computation proficiency, centralized or distributed control performance, and accuracy requirements still need to be explored using AI/ML/DL aspect [92]. Advance algorithms and techniques for cyber attack during the operation is also an open challenge in this field such as reliable communication of unmanned ariel systems, session hijacking, man-in-the middle attack etc. RL has two main features (i) trial and error search (ii) delayed rewards [93]. Following the guidelines in [94], Figure 4 shows both RL, and deep Q-learning schema. Model is used to anticipate the nature of an environment. The ML designs which is using both planning and models are model based mechanism. If the model of environment is absent, then learning can be done through trial and error methods.