# Survey on Q-Learning-Based Position-Aware Routing Protocols in Flying Ad Hoc Networks  
[PERSON]  
1 Department of Computer Engineering, Chosun University, 309 Pilmun-daero, Dong-gu, Gwangju 61452, Korea; [EMAIL_ADDRESS]  
[PERSON]  
Correspondence: [EMAIL_ADDRESS]; Tel. : +82-62-230-603211109911  
This multi-UAV collaborative network, also known as flying ad hoc networks (FANETs), has several key challenges such as a dynamic time-varying topology owing to the mobility of UAVs in 3D space, node density representing the trade-off between sensing coverage and quality of service (QoS) in communication [8], limited transmission range, and limited onboard energy. Although UAV swarms have a wide range of applications, they should generally involve cooperative coordination to achieve mission objectives. Simultaneously, they should consider all constraint resources to optimize the overall network performance to ensure the desired signal-to-interference-plus-noise ratio (SINR) in the aerial links. Therefore, proper collaboration among UAVs is necessary to achieve a balance between mission performance, such as maximizing the coverage to the mission area or ground users (GUs), and communication performance to achieve a desirable connectivity rate with minimal delay in the aerial links. To achieve coverage efficiency, UAVs may frequently fly away from each other's transmission ranges, which causes frequent link breakages. Frequent link breakages during data transmission may increase the number of retransmissions that consume more energy, increase the number of collisions in the medium access control (MAC) layer, and consume unnecessary bandwidth of wireless links. To maintain a better SINR, UAVs should not frequently fly away from each other's transmission ranges. Simultaneously, UAVs should maintain a certain separation distance to avoid physical collision and maximize sensing coverage toward the ground terminal. Therefore, UAVs in the swarm should maintain a strong neighbor relationship and optimal node density to maintain a stable topology for each time interval via proper topology control by following the three principles of flocking: separation, cohesion, and alignment [9]. The separation rule aids UAVs in maintaining a particular separation distance to avoid inter-UAV collisions and maximize coverage. The cohesion rules define the mobility of each UAV in the swarm to be attracted to the average centroid of the neighboring UAV positions. This aids the UAV swarm in staying close to each other to avoid frequent link breakages. Finally, the alignment rule requires that each UAV adopt a velocity direction according to its neighboring UAVs. Similarly, in complex environments, UAVs can sense the presence of external obstacles and generate external obstacle avoidance rules. Each flocking rule generates one motion component, and the sum of these motion components determines the optimal mobility of UAVs [10]. The adaptive adjustment of the weight of each rule can generate a smoother and more energy-efficient trajectory for UAVs [11]. Topology control in a FANET is a mechanism of coordinating and optimizing the mobility of UAVs (position, velocity, and acceleration) according to the transmission range of each UAV, which can produce a network with optimal transmission power and node density. Topology control reduces energy consumption, improves connectivity, and facilitates wider coverage with desirable throughput by controlling UAV mobility and transmission power [12]. Compared with centralized control, the distributed control of the UAV swarm enhances stability and routing efficiency by maintaining neighbor intimacy utilizing one- or two-hop neighbor information. Centralized control provides less scalability and causes the possibility of a single point of failure. It also consumes a high bandwidth because each UAV must transmit its mobility information to a central node to update the topology.