# Q-Learning Empowered Highly Dynamic and Latency-Aware Routing Algorithm for Flying Ad-Hoc Networks
## 2 Related Works  
Most of the ad-hoc routing protocols are developed for MANETs, whereas the FANETs protocol can be obtained by directly modifying MANETs routing protocols. References [4; 5; 6; 7; 8] have detailed analyses of current trends, challenges, and future prospects of routing protocols in FANETs. Generally speaking, traditional ad-hoc routing protocols fall into the following categories:  
(1) Reactive routing protocols  
In reactive routing protocols, routing information is created on demand, and the route discovery process executes when a transmission requirement is established. The main benefit of this approach is low costs in the case of low traffic, but it takes a long time to re-establish a new route in the case of failed routing packets. The typical protocols have Dynamic Source Routing (DSR) [9] and ad-hoc on Demand Distance Vector (AODV) [10]. The route request and reply processes are round trips, where all intermediate nodes of the route store the routing information in a particular format. Such routing protocols minimize overhead. The main disadvantage is that it takes a long time to find a route, causing congestion in the network. (2) Proactive routing protocol  
In proactive routing protocols, nodes periodically update and share routing tables; therefore, available routing information exists between each pair of nodes in the network. Typical routing protocols in this category mainly include Destination Sequenced Distance Vector (DSDV) [11]. This algorithm is a simple and loop-free table-driven routing algorithm. Each node maintains the IP address of the next hop, and the hop counts to all possible destinations. Its disadvantage is that updating the node's routing table involves a lot of overhead, which is unsuitable for highly dynamic UAV network topologies. Optimized Link State Routing (OLSR) [12] is another typical proactive protocol. In this routing algorithm, each node obtains network topology information by using topology control (T.C.) information, named hello information. The original OSLR does not consider link quality, which may lead to suboptimal routing. Directional Optimized Link State Routing (DOLSR) [13] modifies OLSR by using directional antennas to minimize the number of multi-hop relays. Specifically, the UAV tests the distance of each packet to the destination. If the distance exceeds half of the maximum distance, which can be achieved through the use of directional antennas, the node adopts the DOLSR mechanism. On the other hand, if the distance is less than half of the maximum distance, the original OLSR of the omnidirectional antenna is used. This method reduces end-to-end delays; however, the large overhead generated by DOLSR is unsuitable for the rapidly changing UAV network. (3) Hybrid routing protocol  
The hybrid routing protocol combines proactive and reactive routing protocols, which allows the protocol to adjust routing modes according to real-time network conditions. Initially, a proactive protocol determines the route; meanwhile, a reactive routing protocol starts when broken routes are identified or a large number of network topologies change. Zone Routing Protocol (ZRP) [14] is a typical hybrid routing protocol containing the concept of a zone using the reactive method, which reduces the processing time and overhead of the route discovery mechanism. However, ZRP cannot easily maintain node and link information in highly dynamic UAV networks. The Temporarily Ordered Routing Algorithm (TORA) [15] is a routing protocol used in multi-hop networks where the router maintains only the information of adjacent nodes. However, it limits the dissemination ofinformation in highly dynamic networks and weakens the efficiency of the protocol itself in UAV networks. (4) AI-based routing protocol  
[PERSON] et.al. [16] reviewed some AI applications in the overall domain of FANETs. The routing protocol based on machine learning focuses more on learning the whole network state than the traditional protocol; therefore, it is more suitable for dynamic FANETs.