# Reinforcement Learning-Based Routing Protocols in Flying Ad Hoc Networks (FANET): A Review
###### Abstract  
In recent years, flying ad hoc networks have attracted the attention of many researchers in industry and universities due to easy deployment, proper operational costs, and diverse applications. Designing an efficient routing protocol is challenging due to unique characteristics of these networks such as very fast motion of nodes, frequent changes of topology, and low density. Routing protocols determine how to provide communications between drones in a wireless ad hoc network. Today, reinforcement learning (RL) provides powerful solutions to solve the existing problems in the routing protocols, and designs autonomous, adaptive, and self-learning routing protocols. The main purpose of these routing protocols is to ensure a stable routing solution with low delay and minimum energy consumption. In this paper, the reinforcement learning-based routing methods in FANET are surveyed and studied. Initially, reinforcement learning, the Markov decision process (MDP), and reinforcement learning algorithms are briefly described. Then, flying ad hoc networks, various types of drones, and their applications, are introduced. Furthermore, the routing process and its challenges are briefly explained in FANET. Then, a classification of reinforcement learning-based routing protocols is suggested for the flying ad hoc networks. This classification categorizes routing protocols based on the learning algorithm, the routing algorithm, and the data dissemination process. Finally, we present the existing opportunities and challenges in this field to provide a detailed and accurate view for researchers to be aware of the future research directions in order to improve the existing reinforcement learning-based routing algorithms. f 2022 10 3017 10.3390/math10163017 10.3390/  
unique features such as the very fast movement of nodes, the frequent changes in topology, and low-density network. In Figure 1, a flying ad hoc network is shown. This network has different applications in military and civilian areas. Military applications include public protection, search and rescue, reconnaissance, border monitoring, independent tracking, fire fighting, internal security, wind estimation, remote sensing, traffic monitoring, and relaying networks [3]. Furthermore, drones have many commercial applications in civilian areas, such as film-making, agricultural monitoring, Internet shipping, transportation, and architecture and infrastructure monitoring [4; 5]. Given specific features of FANETs, such as high dynamic topology, the rapid motion of nodes, and frequent link failure, designing an efficient routing protocol is a very important research challenge in these networks. In general, in the routing process, we answer the question \"_How do UAVs send data packets from source to destination?_\". In this process, if there is no direct connection between the source and destination, the data packets must be transferred by intermediate nodes (that play the router role) to the destination [6; 7]. According to this definition, the routing path represents a series of hops (i.e., intermediate nodes) that relay data packets. A routing protocol is responsible for building such a route between the source and destination. Additionally, these protocols should manage the link failure by finding appropriate alternative routes. In FANETs, there are many challenges that can affect the routing process. For example, routing protocols must solve problems related to intermittent links, frequent changes in topology, network partitioning, and node movement [8; 9]. Furthermore, they must consider some constraints such as energy, computing power, and delay. In addition, these protocols must be free-loop, self-repairing, and scalable. Note that UAVs move in a three-dimensional space, which affects the quality of communication links [10; 11]. This is a major challenge when designing routing protocol in FANETs. In recent years, many researchers try to improve the performance of routing protocols in FANETs.