# A Survey on Applications of Reinforcement Learning in Flying Ad-Hoc Networks
###### Abstract  
Flying ad-hoc networks (FANET) are one of the most important branches of wireless ad-hoc networks, consisting of multiple unmanned air vehicles (UAVs) performing assigned tasks and communicating with each other. Nowadays FANETs are being used for commercial and civilian applications such as handling traffic congestion, remote data collection, remote sensing, network relaying, and delivering products. However, there are some major challenges, such as adaptive routing protocols, flight trajectory selection, energy limitations, charging, and autonomous deployment that need to be addressed in FANETs. Several researchers have been working for the last few years to resolve these problems. The main obstacles are the high mobility and unpredictable changes in the topology of FANETs. Hence, many researchers have introduced reinforcement learning (RL) algorithms in FANETs to overcome these shortcomings. In this study, we comprehensively surveyed and qualitatively compared the applications of RL in different scenarios of FANETs such as routing protocol, flight trajectory selection, relaying, and charging. We also discuss open research issues that can provide researchers with clear and direct insights for further research.  
flying Ad-hoc network; reinforcement learning; routing protocol; flight trajectory; unmanned air vehicles +
Footnote †: journal: _Electronic Notes_  
definition  
Routing protocol: Routing in FANETs is also a challenge owing to the high mobility and power constraints of the UAVs. Many routing protocols have been designed for ad-hoc networks but FANET requires a highly dynamic routing protocol to cope with the dynamic changes in the FANET topology [12].
* Ensuring QoS: There are also some quality of service (QoS) related challenges that should be addressed such as ensuring low latency, determining the trajectory path to provide service, synchronization among UAVs, and protection against jamming attacks.  
Many researchers have been working for the last few years to overcome these challenges. They have been using different techniques related to artificial intelligence (AI) so that the network can autonomously and adaptively learn and overcome the challenge by itself. Reinforcement learning (RL) is one of the most important algorithms that has a significant contribution to the development of AI [13; 14; 15]. RL is popular for its trial-and-optimize scheme. RL consists of an agent and an environment in which the agent explores the environment by taking actions and reaches an optimal policy for the system [16]. However, to achieve the optimal policy, the agent must know the entire system, which makes the RL time-consuming and unsuitable for large networks. With the development of computational capability achieved by the GPU, this problem can be addressed by integrating deep neural networks (DNNs) into RL, namely deep reinforcement learning (DRL) [15; 17].  
Currently, there is no survey discussing the applications of RL in FANETs. This motivates us to deliver the survey with the fundamentals of RL and DRL and a comprehensive literature review on the applications of RL and DRL to address the challenges in FANETs. The major issues include routing protocol, selecting flight trajectory, charging UAVs, anti-jamming, and ensuring the QoS of FANETs.