# Survey on Q-Learning-Based Position-Aware Routing Protocols in Flying Ad Hoc Networks
## 5 Comparison of QL-Based Position-Aware Routing Protocols  
In this section, the QL-based position-aware routing protocols discussed in Section 4 are discussed in terms of their objectives and innovative features. Additionally, they are qualitatively compared. The objective and innovative features of the QL-based position-aware routing protocols are summarized in Table 6. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Protocol** & **Objective** & **Innovative Features** \\ \hline \multirow{3}{*}{QGEO [39]} & Utilizes MAC delay, link error, and localization error & \multirow{3}{*}{Introduces the concept of PTS and PTT in QL-based position-aware single path forwarding.} \\  & in QL to determine the link quality and avoid the local minimum problem. & \\ \hline \multirow{3}{*}{
\begin{tabular}{c} RFLQGEO [50] \\ \end{tabular} } & Utilizes MAC delay, link error, localization error, and distance progress toward destination to determine the forwarding node. & \multirow{3}{*}{Introduces inverse reinforcement learning by exchanging hello packets to estimate the link Q-value as a link quality metric.} \\ \cline{2-3}  & Design a cross-layer adaptive QL-based routing & & \\ \cline{2-3}  & protocol to minimize delay, link error, and localization error. & & \\ \hline \multirow{3}{*}{QMR [22]} & Jointly minimizes the delay and energy by designing a multi-objective reward function in adaptive QL. & \multirow{3}{*}{Introduces the exploration and exploitation strategy in QL-based routing decisions according to the degree of mobility with neighbor UAVs.} \\ \cline{2-3}  & Utilizes QL and fuzzy logic to select the optimal path in FANETs by jointly considering two path-related parameters (hop count, packet delivery ratio) and three link parameters (transmission rate, energy state, and flight direction). & \multirow{3}{*}{Designs an integrated framework using fuzzy logic and QL to select the optimal forwarding path.} \\ \cline{2-3}  & Designs an adaptive QL model to minimize jitter, end-to-end delay, and latency in highly dynamic conditions in FANETs with local minimum avoidance. & \multirow{3}{*}{Q-values are updated more precisely by considering the weighted reward for the finite number of last episodes and the SINR level of the link.} \\ \cline{2-3}  & Designs a multi-objective reward function to optimize delay, energy consumption, to avoid routing loop, and blind path challenges in FANETs. & Introduces PTS and PTT up to two-hop neighbor links to extend the local view of each UAV for early convergence. \\ \hline \multirow{3}{*}{PARRoT [29]} & Designs an adaptive QL model by exchanging hello packets with neighboring nodes to select more stable paths in FANETs. & \multirow{3}{*}{Introduces relative trajectory knowledge (link duration, and cohesion metric) to adaptively train the QL model.} \\ \cline{1-1} \cline{2-3}  & Designs an adaptive QL model to select the energy and delay optimal relay node in FANETs while avoiding routing loop. & \multirow{3}{*}{Introduces SAO to control the balance between exploration and exploitation rate in FANETs.} \\ \cline{1-1} \cline{2-3}  & & \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Objective and innovative features of QL-based position-aware routing protocols. The mobility model, localization technique, neighbor information, MAC protocol type, exploration strategy, and simulation tool are presented in Table 7. Most of the routing protocols consider generic mobility models such as Gaussian Markov and RWP mobility models for simulating the designed routing protocols. Only PARRoT considers realistic mobility models for UAV swarm missions by adjusting UAV locations according to the mobility of the ground vehicle to provide better network coverage. All of them utilize GPS to detect locations in the global frame. All routing protocols except QTAR utilize one-hop neighbor information to select relay nodes. By utilizing two-hop neighbor information, QTAR extends the local view of each agent and avoids link breakage and blind-path challenges in FANETs.