# Survey on Q-Learning-Based Position-Aware Routing Protocols in Flying Ad Hoc Networks
## 2 Flying Ad Hoc Networks
LAP UAVs can be classified as fixed-wing (small aircraft), rotary-wing (single rotor or multi-rotor), and small tethered balloons. Among them, rotary-wing, particularly multi-rotor UAVs, are mostly used in surveillance and ABS applications as they support vertical take-off and landing, hovering, and provide better formation stability. In addition, multi-rotor UAVs fly in confined areas. Typically, each UAV has four major modules: flight control, energy management, computation, and communication modules. The flight-control module is responsible for controlling the mobility of UAVs in a 3D space. The mobility information of UAVs in a 3D space can be described by six degrees of freedom (DoFs): surge, heave, sway, pitch, yaw,  
Figure 1: FANET and its different components. and roll. The formation controller obtains the optimal mobility control policy by adjusting these six DoFs according to the neighbor UAV mobility (position, velocity, and acceleration) and targets or the GU distribution/mobility within the mission area. It determines the optimal 3D positions and linear/angular velocity for each UAV to achieve both the mission and communication performance. The term surge defines the UAV's forward and backward movement on the X-axis, the sway defines the left and right movement on the Y-axis, and the heave defines up and down movement on the Z-axis. These three terms are used to calculate the 3D Cartesian coordinates of the UAVs, and the rate of change of these 3D coordinates defines their linear velocity. Similarly, the term roll defines side-to-side tilting on the X-axis, the pitch defines forward and backward tilting on the Y-axis, and the yaw defines left and right turns on the Z-axis. These three terms (roll, pitch, and yaw) define the attitude angles and angular velocities. The energy management module produces continuous power for UAVs to generate thrust using UAV rotors and propulsion energy to continue flying in the air. It also supplies energy to perform the computation of different algorithms to execute the mission and communicate with other neighboring UAVs, remote BSs, or GUs with the aid of a communication module. Frequently, the propulsion energy of an UAV is significantly higher than the communication energy. The propulsion power consumption is proportional to the UAV trajectory [41]. Thus, in UAV missions, the joint optimization of UAV trajectory and UAV communication provides better energy efficiency [41]. The computation modules execute algorithms to process the collected data using onboard sensors such as cameras, LiDAR, sound navigation and ranging (SONAR), and inertial measurement units (IMUs). Communication modules consist of wireless radio antennas and wireless interfaces. There are various communication links in FANETs depending on the mission planning and control methods. These communication links can be classified as air-to-ground (A2G) and air-to-air (A2A) links. Generally, A2G links include UAV to BS (U2 BS) and UAV to GU (U2 GU). Similarly, the A2A links are UAV to UAV (U2U), UAV to HAPs (U2H), and UAV to low earth orbit (LEO) satellites (U2S). UAVs can directly communicate with satellites [42], particularly with GPS, to localize themselves in global coordinates. LAP UAVs communicate with the BS using U2 BS downlinks. U2 BS and U2 GU links are mostly Wi-Fi links, and they have low costs in terms of delay, transmission power, latency, and path loss in line-of-sight (LoS) scenarios. However, the quality of the A2G links significantly degrades in the no-line-of-sight (NLoS) scenarios, particularly in the urban environment. However, depending on the signal quality and mission design, LAP UAVs can also utilize U2H or U2S uplinks to communicate with the BS as a redundant path [43]. A2A links are mostly considered free-space paths and are dominated by LoS links [1; 15]. However, owing to the high mobility and energy limitations of UAVs, these paths encounter high dynamism, Doppler effects, and link breakage.