# Survey on Q-Learning-Based Position-Aware Routing Protocols in Flying Ad Hoc Networks
## 5 Comparison of QL-Based Position-Aware Routing Protocols
Most routing protocols consider the IEEE 802.11g/n default MAC standard to define the physical layer standard. To achieve a balance between exploration and exploitation in the QL model, some algorithms consider the exploration strategy based on UCB, \(\in\)-greedy strategy, one-hop PTS, and two-hop PTS and integrate the metaheuristic algorithm (SAO) with the QL model. Network Simulator 3 (NS-3), MATLAB, and WSNet are popular tools used to simulate QL-based position-aware routing protocols. The path selection strategy, local minimum avoidance, adaptability in the hello interval, link stability, localization error consideration, and scalability features are summarized in Table 7 (in the continued table). All routing protocols utilize a single-path strategy to discover the multihop routing path to reduce the number of broadcasts in energy- and bandwidth-constrained FANETs. They avoid the local minimum problem in the position-based forwarding technique by assigning the minimum reward to the corresponding relay nodes that have no further nodes to forward the data packets toward the destination. The SAIQL avoids the routing loop problem by saving the visited UAVs in the end-to-end path, and none of the forwarding UAVs select the relay UAV that has been selected previously in the end-to-end path. QTAR optimizes the hello interval in FANETs by adaptively selecting its value according to the minimum link duration found within a one-hop neighbor. This technique adaptively sets the hello interval and controls the control overhead. We consider the stability of the selected links to be higher in QMR, QTAR, and PARRoT. This is because both QMR and QTAR consider the PTS and residual energy levels in the reward function when selecting the relay node. PARRoT introduces a mobility prediction mechanism by calculating the link duration and cohesion metric according to the relative trajectory knowledge of neighboring UAVs. Thus, it selects a more stable path by considering high-mobility FANETs. Only QGEO, RFLQGEO, and QLCLRP consider the node localization error when selecting the relay UAV. We consider a low scalability for QLMF because it updates two different Q-values in the learning process and send them to the fuzzy logic controller for decision making. We consider high scalability for QGEO, RFLQGEO, QMR, and PARRoT because they maintain only single Q-value to update the quality of neighbor links. We consider medium scalability for QTAR because it utilizes two-hop neighbor information to make a better optimal decision; however, keeping two-hop information at each UAV produces higher overhead. Similarly, we consider medium scalability for SAIQL because it utilizes the SAO-based metaheuristic algorithm to update the Q-values for each neighbor link, which is an iterative process. The end-to-end delay optimization, UAV energy efficiency, control overhead, PDR, and adaptive learning features are summarized in Table 7 (in the continued table). We observed that QGEO, RFLQGEO, and QLCLRP consider only the MAC delay to select relay node links. Additionally, they consider the progress of the transmission distance toward the destination node to select the relay link. Among them, QMR, Q-FANET, and QTAR jointly considers the MAC, queuing, and transmission delays to select the relay node. QMR, QTAR, SAIQL, and QLMF consider the UAV residual energy level to select the relay node as part of their objective in the reward function. Thus, QMR, QTAR, SAIQL, and QLMF produce balance in energy consumption, which prolongs the lifetime of FANETs. QTAR provides a higher control overhead than others because it uses two-hop neighbor information. We can consider medium control overhead size for QLMF because it requires storing two different Q-values. Similarly, we consider medium control overhead for PARRoT and SAIQL because they learn the optimal policy by sharing data packets with their neighbors. QMR and QTAR give higher PDR compared to others because they choose more stable links by considering PTS and link residual energy.