# Deep-Reinforcement-Learning-Based Intelligent Routing Strategy for FANETs
## 1 Introduction  
The next generation mobile communication technology (6G) aims to realize the full connection between terrestrial communication networks and nonterrestrial communication networks, so as to achieve seamless global coverage and allow network signals to reach any remote area [1, 2]. The wireless communication technology assisted by unmanned aerial vehicles (UAVs) including blimps, balloons, and fixed-wing and rotary-wing UAVs is getting more and more attention from scholars [3, 4, 5, 6], and is expected to become an indispensable part of 6G [7, 8, 9, 10, 11]. Compared with terrestrial networks, flying ad hoc networks (FANETs) have more flexible nodes and communication links, so the latter can give play to their asymmetric advantages compared with the former to achieve auxiliary data transmission or emergency communication. FANETs based on UAVs are capable of improving the coverage of ground wireless service as well as providing reliable communications for ground devices [12, 13]. Routing technology aims to ensure that the control and business data in the network can be delivered to the destination quickly, stably, and completely [14, 15, 16, 17, 18]. FANETs differ from traditional ground communication networks in terms of connectivity, mobility, applications areas, etc. In this case, the routing algorithms specifically proposed for fixed topology characteristics of ground networks may not apply well to FANETs, as the topology of the latter is in a process of continuous change. Meanwhile, this situation makes location-based algorithms become the main development direction of FANETs' routing technology [19]. A lot of research efforts have been devoted into routing technology in FANETs, for which some effective and robust algorithms have been presented [20, 21, 22, 23, 24, 25, 26, 27, 28]. In [20], a bidirectionalQ-learning routing strategy was proposed on the basis of the Ad Hoc On-Demand Distance Vector Routing (AODV) method, which could obviously accelerate the convergence speed of the Q-learning process. The particle swarm optimization was adopted in [21] to address the suboptimal choice problem of greedy forwarding. Relying on the cooperation between ground equipment and UAV nodes, [PERSON] et al. [22] proposed a method that integrated fuzzy logic and a depth-first search to construct a Q-value routing table with well convergence performance. In [23], a routing strategy dominated by upper UAV nodes participating in multilayer FANETs was proposed, which was based on information such as node location and network connectivity. [PERSON] et al. [24] introduced LEO satellites into FANETs and adopted Q-learning to adapt routing decision, so as to improve the efficiency of the network providing services to the ground in a 6G scenario. An improved Optimized Link State Routing (OLSR) protocol based on dynamic topology was presented in [25], which could adaptively adjust the sending period of HELLO packets according to the dynamics of the network topology. In [26], the distance factor and link lifetime factor were mainly considered, and the routing problem was effectively solved by linear programming. Taking into account the deviation angle, node spacing, and link lifetime factors, the authors in [27] designed an optimal virtual routing model to select the neighboring relay node. [PERSON] et al. [28] utilized the ant colony optimization approach for calculating the route in UAV-assisted massive machine-type communications networks to reduce the energy consumption and prolong the network lifetime. The UAV nodes in FANETs are symmetrical, and each node is responsible for data collection and packet forwarding, which also limits the application of central routing algorithms in FANETs.