# Survey on Q-Learning-Based Position-Aware Routing Protocols in Flying Ad Hoc Networks
### Related Studies
Deterministic progress-based routing protocols have several relay node-selection strategies, including greedy forwarding, compass forwarding, and most forwarding [37]. Multipath strategies include restricted direction flooding, random directional flooding, and classic flooding of data packets [37]. According to the aforementioned study, considering the dynamism in network topology in the 3D space, inter-UAV collision, high overhead, and delay, position-based routing protocols are attracting the interest of researchers. However, position-based routing protocols encounter several challenges in FANETs, such as maintaining the link quality [39], controlling the hello interval to predict up-to-date topology [21], localization errors, blind paths, the presence of routing loops, and energy holes [23]. Additionally, to prolong the lifetime of a FANET, it is necessary to achieve a proper load balance in terms of energy and delay while determining the optimal routing path [22]. Tracing the shortest routing path may be initially beneficial, but it cannot be an optimal routing path as it depletes the energy of a few selected UAVs, and the shortest paths can be extremely congested by traffic over time [23]. It also creates energy holes in FANETs because selecting the shortest path always drains the energy of a few selected UAVs. Greedy forwarding cannot ensure optimal performance in terms of energy consumption, delay, and link quality, as it always seeks progress in the transmission distance toward the destination. Additionally, owing to the selection of relay nodes at the edge of the transmission range of the source node, greedy forwarding encounters blind path and link-breakage problems. The compass and most forward techniques have higher possibilities of trapping in routing loops and local minimum [37]. The term local minimum (routing holes) in position-based routing is defined as the selected relay UAV with no further neighbors to relay toward the target destination node. Flooding techniques in multipath forwarding produce excessive overhead, high MAC layer contention, high bandwidth, and energy consumption. UAVs in a swarm exchange hello packets to update their position coordinates and residual energy. The low hello-interval provides better positioning accuracy but simultaneously increases the control overhead cost [21]. Consequently, an adaptive strategy is required to determine the optimal hello interval in FANETs to optimize the control overhead cost and predict the updated network topology. To address the above challenges in position-based forwarding, researchers have proposed intelligent decision-making algorithms in dynamic FANETs utilizing the QL technology incorporated with position-based routing protocols. QL-based routing protocols can perform multi-objective optimization; i.e., delay and energy are minimized by leveraging the PMDP for predicting the dynamic topology with the aid of a topology controller. Recently, a significant amount of research has been conducted to enhance the performance of position-based forwarding techniques by integrating them with QL. The QL model can be trained to identify a link that is trapped in the local minimum in position-based forwarding by providing a minimum reward to the relay nodes for taking a bad action. Additionally, in [22; 24], the designed QL-model allocates minimum reward to the relay UAVs that do not send the acknowledgment to the corresponding source UAV by considering the failure state of the selected relay nodes. In FANETs, the UAV failure state might happen owing to the hardware failure, the depletion of UAV energy, and the environmental dynamism encountered by the UAVs (external obstacles). Thus, a robust self-healing topology controller is required to reestablish the swam topology without creating any partition in topology, while performing the mission in a complex dynamic environment [11; 17].