# Reinforcement Learning-Based Routing Protocols in Flying Ad Hoc Networks (FANET): A Review
## 2 Related Works  
Our reviews show that there are few review papers which survey routing issues, specifically RL-based routing methods in FANETs. We studied some related papers in this field, as follows:  
In [20], authors studied artificial intelligence-based routing protocols in flying ad hoc networks. In this survey, the applications of artificial intelligence (AI) are studied in different areas of FANETs. This paper pursues two goals: (1) investigating the features of FANET, UAV technology, networking protocols, and UAV swarms; (2) studying the routing protocols designed in these networks by emphasizing the AI application in this area. In [20], authors suggested the classification of routing methods in FANETs. It includes centralized and distributed routing, deterministic and probabilistic routing, and static and dynamic routing. Then, they categorized dynamic routing methods into five classes: position-based (geographic, location-based, and opportunistic), proactive, reactive, hybrid, and AI-based (topology predictive and self-adaptive learning-based). Then, different routing methods have been investigated based on this categorization, and the challenges and issues in this area have been expressed. It is a comprehensive and useful review paper that is recommended to researchers in this field. However, they do not emphasize the AI techniques and their structures in these methods. In [21], the authors presented reinforcement learning (RL) and deep reinforcement learning (DRL), and studied their applications in FANETs. They claim that this paper is the first review paper on RL applications in FANETs. In general, they focused on the RL applications in five important areas, including routing protocols, flight trajectory, protection against jamming, relaying, and charging UAVs. Then, they studied RL-based methods in the five areas and expressed their advantages and disadvantages. However, this paper does not review the details of the RL algorithms used in these methods. In [22], the authors studied and evaluated the Q-learning-based position-aware routing protocols in FANETs. Initially, they introduced flying ad hoc networks and their features, and described all mobility models available for FANETs by focusing on their applications. Then, they introduced a Q-learning algorithm and its application for designing routing protocols in FANETs. Next, Q-learning-based routing protocols were investigated and their advantages and disadvantages were expressed. Finally, these methods were compared with each other in terms of key features, performance, and implementation. However, the most important disadvantage of this paper is that it focuses only on a Q-learning algorithm and ignores other reinforcement learning algorithms. In [23], the authors examined machine learning (ML) and artificial intelligence (AI) applications for UAV networks. They studied various communication issues, from the physics layer, channel modeling, and resource management, to flight trajectory and caching; in particular, they emphasized security and safety issues, and provided solutions based on learning techniques in this area. However, this paper does not focus on ML-based routing protocols in FANETs. In [24], the authors presented a comprehensive review to examine issues related to FANETs, including their features, architecture, and communication. Then, the authors examined various mobility models such as random-based, time-based, path-based, group-based, and topology-based mobility models for these networks. Finally, in [24], a detailed classification of routing protocols was provided in flying ad hoc networks. In this classification, routing methods were divided into various groups, including delay-tolerant network (deterministic, stochastic, social network), position-based (single-path and multi-path), heterogeneous, energy-aware, swarm-based, cluster-based (probabilistic and deterministic), topology-based (static, hybrid, reactive, proactive), and secure routing protocols.