# Reinforcement Learning-Based Routing Protocols in Flying Ad Hoc Networks (FANET): A Review
## 7 Discussion
\begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Scheme**} & \multirow{2}{*}{**Knowledge**} & \multirow{2}{*}{**Neighbor Information**} & \multicolumn{2}{c}{**Route**} & \multicolumn{2}{c}{**Adaptive**} & \multirow{2}{*}{**Routing Loop**} & \multirow{2}{*}{**Energy**} & \multirow{2}{*}{**Scalability**} & \multicolumn{2}{c}{**Bilind Path**} \\  & & & **Discovery** & & & & **Hello Broadcast** & & **Balancing** & **Problem** \\  & & & & & & & **Interval** & & & \\ \hline DQN-VR [67] & Local and global & \(\times\) & & Hello & \(\times\) & \(\times\) & ✓ & High & \(\times\) \\ \hline QTA [68] & Local & Single and two-hop neighbors & Hello & \(\checkmark\) & ✓ & ✓ & High & \(\checkmark\) \\ \hline TGNCFN [69] & Local & Single-hop neighbors & Hello & \(\times\) & ✓ & \(\times\) & Medium & \(\times\) \\ \hline QMR [70] & Local & Single-hop neighbors & Hello & \(\checkmark\) & ✓ & ✓ & High & \(\checkmark\) \\ \hline QGe [71] & Local & Single-hop neighbors & Hello & \(\checkmark\) & ✓ & \(\times\) & Low & \(\times\) \\ \hline QGRP [72] & Global & Single-hop neighbors & Discovery and ACK packets & \(\checkmark\) & ✓ & \(\times\) & Low & \(\times\) \\ \hline QLGR [73] & Local & Single-hop neighbors & Hello & \(\checkmark\) & ✓ & \(\times\) & High & \(\checkmark\) \\ \hline FRQ-routing-SA [74] & Local & Single-hop neighbors & \(\times\) & \(\times\) & ✓ & \(\checkmark\) & Low & \(\times\) \\ \hline Q-GANET [5] & Local & Single-hop neighbors & Hello & \(\times\) & ✓ & \(\times\) & Low & \(\checkmark\) \\ \hline PPMAC+RLSRP [76] & Local & Single-hop neighbors & Hello & \(\times\) & \(\times\) & \(\times\) & Low & \(\times\) \\ \hline PARRtoT [77] & Local & Single-hop neighbors & Chirp & \(\times\) & ✓ & \(\times\) & Low & \(\checkmark\) \\ \hline QFL-MOR [78] & Local and global & Single-hop neighbors & Hello & \(\times\) & \(\times\) & ✓ & Low & \(\times\) \\ \hline
3 DQ [79] & Local & Single-hop neighbors & Unknown & \(\times\) & \(\times\) & \(\times\) & Low & \(\times\) \\ \hline ICRA [80] & Local & Single-hop neighbors & Hello & \(\times\) & \(\times\) & ✓ & High & \(\times\) \\ \hline TARAQ [81] & Local & Single-hop neighbors & Hello & \(\checkmark\) & ✓ & ✓ & Low & \(\checkmark\) \\ \hline \hline \end{tabular}
\end{table} 
Table 23: Comparison of RL-based routing methods. Table 26 compares RL-based routing methods in terms of different learning components in learning algorithms. According to this table, we can find that Q-learning is \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Scheme**} & \multirow{2}{*}{**Simulation Tools**} & \multirow{2}{*}{**Mobility Model**} & \multirow{2}{*}{**Localization Service**} & \multirow{2}{*}{**Simulation**} \\  & & & & & & **Environment** \\ \hline DQN-VR [67] & MATLAB and Python & Unknown & Unknown & Unknown & 3D \\ \hline QTAR [68] & MATLAB & 3D Gauss–Markov & GPS & 3D \\ \hline TQNGPSR [69] & Python and SimPy & Aircraft model & Unknown & 2D \\ \hline QMR [70] & WSNet & Random Waypoint & GPS & 2D \\ \hline QGeo [71] & NS3 & Gauss–Markov & GPS & 2D \\ \hline QSRP [72] & OPNET & Random Waypoint & Unknown & 3D \\ \hline QLGR [73] & NS3 & Gauss–Markov & Unknown & 2D \\ \hline FEQ-routing-SA [74] & Unknown & Unknown & x & Unknown \\ \hline Q-FANET [75] & WSNet & Random Waypoint & GPS & 2D \\ \hline PPMAC+RLSRP [76] & MATLAB and NS2 & Random Waypoint & GPS & 3D \\ \hline PARRoT [77] & OMNeT++ &
\begin{tabular}{c} Random Waypoint, \\ distributed dispersion \\ detection, and dynamic \\ cluster hovering \\ \end{tabular} & Unknown & 3D \\ \hline QFL-MOR [78] & Unknown & Unknown & Unknown & 2D \\ \hline
3 DQ [79] & Unknown & Gauss–Markov & GNSS & 2D \\ \hline ICRA [80] & OPNET & Gauss–Markov & GPS & 2D \\ \hline TARRAQ [81] & Monte Carlo & 3D Random Waypoint & GPS & 3D \\ \hline \hline \end{tabular}
\end{table} 
Table 24: Comparison of RL-based routing protocols in terms of simulation environment and tools.