# Reinforcement Learning-Based Routing Protocols in Flying Ad Hoc Networks (FANET): A Review
## 8 Challenges and Open Issues  
Despite progress in designing the RL-based routing methods for flying ad hoc networks, this subject still deals with various challenges and open issues, which should be addressed. In this section, the most important challenges are presented in this field. * **Mobility models:** Most RL-based routing protocols use RWP and GM mobility models to simulate the movement of drones in the network. However, these models cannot simulate the actual movement of drones. Therefore, the simulation results may not guarantee the performance of the routing protocols in real conditions. As a result, in future research directions, researchers must consider realistic mobility models such as [82; 83; 84; 85; 86] to simulate the movement of UAVs in the network because they are close to real mobility models and can evaluate the performance of the routing protocols under realistic scenarios. * **Simulation environment:** In many RL-based routing protocols, researchers have implemented drones in a two-dimensional environment. However, this implementation is incompatible with the three-dimensional environment of FANETs and affects the performance of these methods. Therefore, researchers must close the simulation environment to the real conditions to evaluate the performance of these protocols more accurately. As a result, deploying the routing methods in a 3D environment and considering all FANET needs and restrictions are subjects that must be studied in future research directions. * **Simulation tool:** Most RL-based routing protocols are simulated using simulators such as WSNet, NS3, and MATLAB to evaluate their performance in a virtual environment. They require low cost. However, these tools cannot accurately simulate a real environment, and the simulation results do not usually match the real environment. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Scheme** & **Unicast** & **Multicast** & **Broadcast** & **Geocast** \\ \hline DQN-VR [67] & ✓ & × & × & × \\ \hline QTAR [68] & × & × & ✓ & × \\ \hline TQNGPSR [69] & ✓ & × & × & × \\ \hline QMR [70] & × & × & ✓ & × \\ \hline QGeo [71] & × & × & ✓ & × \\ \hline QSRP [72] & × & × & ✓ & × \\ \hline QLGR [73] & × & × & ✓ & × \\ \hline FEQ-routing-SA [74] & ✓ & × & × & × \\ \hline Q-FANET [75] & × & × & ✓ & × \\ \hline PPMAC+RLSRP [76] & × & × & ✓ & × \\ \hline PARRoT [77] & × & × & ✓ & × \\ \hline QFL-MOR [78] & × & × & ✓ & × \\ \hline
3 DQ [79] & ✓ & × & × & × \\ \hline ICRA [80] & ✓ & × & ✓ & × \\ \hline TARRAQ [81] & × & × & ✓ & × \\ \hline \hline \end{tabular}
\end{table} 
Table 32: Comparison of RL-based routing methods based on the data dissemination process. Therefore, these methods must be implemented in real environments to analyze their performance in real conditions. However, this is extremely expensive. * **Localization:** Most RL-based routing methods must obtain location information using a positioning system. Therefore, their performance is dependent on a positioning system. If the positioning system is not accurate and cannot properly measure the position of drones, the network performance will be weakened. GPS is the most common positioning system used in RL-based routing protocols. The accuracy of this positioning system depends on environmental conditions. For example, if UAVs are in an open environment with a good climate, GPS can predict the position of drones with proper accuracy. In contrast, in indoor areas such as tunnels or in inappropriate weather conditions, GPS signals are not properly received. This has a negative effect on network performance. Focusing on free infrastructure localization methods and calculating the position of nodes without the need for GPS is a good solution, which can be considered by researchers in the future. * **Efficiency:** Reinforcement learning algorithms are useful for designing RL-based routing methods in FANETs when they can solve a serious problem in this area.