# Q-Learning Empowered Highly Dynamic and Latency-Aware Routing Algorithm for Flying Ad-Hoc Networks
## 1 Introduction  
In recent years, UAV clusters have become increasingly popular in harsh environments due to their low costs and fast response times. However, these applications face the challenge of high-quality data transmission. Flying ad-hoc networks (FANETs [1]), which are derived from traditional MANETs in Figure 1, offer advantages such as easy deployment, high mobility, self-organization, and decentralization, which make them well-suited for UAV cluster applications. Node mobility in FANETs is typically greater than that in VANETs and MANETs [2]. The node's speed in FANETs is highly variable, which can range from zero during aerial coverage to full flight in missions. These nodes are characterized by the ability to move randomly in three dimensions with the help of rotatory wings, which can rotate independentlyon three axes (roll, tilt, and yaw). In contrast, in MANETs, mobile nodes typically have low mobility (e.g., people walking at 6 km/h) and limited speed variation. In VANETs, nodes are vehicles that travel on the street with moderate speed variation (about 100 km/h on highways and 50 km/h on city roads) in a two-dimensional moving plane (horizontal plane). The random wandering movement model is more suitable for MANETs when the direction and speed of nodes are chosen randomly. When nodes move on streets or highways, the street random wandering model or Manhattan mobility model can be chosen to simulate VANETs. Table 1 shows comparisons among MANETs, VANETs, and FANETs. Due to the three-dimensional environment in UAV ad-hoc networks, the network topology of FANETs exhibits highly dynamic characteristics, which poses significant challenges to the design of communication protocols. Therefore, it is essential to consider different mobility models to address these challenges [3]. By simulating the movement of UAVs in the network under various scenarios, mobility models provide valuable insights into the behavior of the network. These models enable researchers to design and evaluate protocols that are robust and efficient under different network conditions. Thus, considering different mobility models is crucial in designing protocols that can effectively cope with the dynamic nature of FANETs. Table 2 shows several basic mobility models and mission scenarios. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Characteristics** & **MANETs** & **VANETs** & **FANETs** \\ \hline Mobility & Low(2D) & Low(2D) & FW-Med(3D) \\  & & & RW-High(3D) \\ \hline Speed & Low(6 km/h) & Med-High(20–100 km/h) & FW-High(100 km/h) \\  & & & RW-Med(50 km/m) \\ \hline Mobility Model & Random & Manhattan & FW-Paparazzi \\  & & & RW-RWP \\ \hline Topology Variation & Low & Med & FW-Med \\  & & & RW-High \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Comparison among MANETs, VANETs, and FANETs. Figure 1: MANETs, VANETs, and FANETs. The wireless routing protocol plays a critical role in ensuring stable transmissions of data packets in the communication of FANETs. Each node simultaneously behaves as a source and a router. As the transmission path of data packets is typically composed of multi-hop routes, developing efficient and stable routing algorithms can significantly enhance network performance. However, the highly dynamic characteristics of the network topology resulting from the rapid movement of aircraft nodes in FANETs, as well as their random entries and exits from the network, can increase the interruption rate of transmission links and decrease the network quality of service (QoS). Consequently, to improve the QoS of moving ad-hoc networks, routing protocols must be comprehensively designed to exploit the unique advantages of FANETs. Most of the existing traditional ad-hoc routing protocols are capable of meeting the basic requirements if topology changes are relatively fixed.