## 3 FANET Routing Algorithms
### Reinforcement-Learning-Based Routing Algorithms
In addition, the researchers propose a full-echo Q-routing algorithm with optimized control of the adaptive learning rate in order to select unstable links, minimize the total energy consumption of the network, and minimize packet loss. Experimental results show that the algorithm is able to adapt to dynamic changes in the network and does not require manual re-initialization after network topology changes. [PERSON] et al. [98] propose a UAV self-organizing network algorithm, called FAN-RL, for 5G using reinforcement learning. This algorithm feeds UAVs with higher residual energy and stability as environmental states into a policy network to determine optimal routing, extend network lifetime, and reduce energy consumption and the number of broken links. To reduce energy consumption, extend network lifetime, and minimize the number of broken links, this algorithm utilizes a policy network that considers higher residual energy and stability of UAVs as environmental states for optimal routing. FAN-RL uses a modified Q-routing model embedded in a central controller to select the best possible routes based on the UAV self-organizing network's higher remaining charge and stability, resulting in improved QoS and network lifetime. Although effective in reducing energy consumption and extending network lifetime, FAN-RL only considers the impact of network density and stability on the time the network provides service, the amount of UAV power, and the number of failed links, and does not consider other factors such as link jitter. [PERSON] et al. [99] proposed an algorithm for delay-sensitive FANETs called XiA that utilizes Q-learning and considers the SAGIN's FANET characteristics, designing a bonus function that factors in channel conditions and equipment configurations to minimize the time required to forward data to the access point via multiple relay UAVs, including those in restricted or affected communication areas. If the UAVs are unable to reach the access point, they send the data to the LEO satellite, which then forwards the data to the access point. This algorithm improves the latency by 82% compared to the traditional Dijkstra shortest-circuit algorithm and the FAN-RL method proposed by [PERSON] et al. [98] in 2020. Although consideration of satellite networks is added in the paper, attention to ground nodes in SAGIN is still missing. [PERSON] et al. [100] proposed a Q-learning-based adaptive UAV-assisted geographic routing algorithm with concerns about path selection errors in the routing mechanism, reduced convergence speed due to routing table maintenance, and the inability to use it for a long time after convergence. The algorithm adds consideration of ground network nodes. As the research progresses, more researchers are thinking beyond Q-learning based on value functions, and some of them have innovatively proposed the use of multi-intelligent reinforcement learning to solve the routing problem in complex FANETs. In contrast, multi-intelligent reinforcement learning can improve the efficiency and performance of system routing through techniques such as parallelization. [PERSON] et al. [PERSON] et al. (2017) proposed a training routing strategy using an actor-critic neural network. This algorithm state space includes FANET local information and global information based on the mean field. The complexity is greatly reduced compared to the high complexity of centralized routing algorithms facing large-scale networks. Similarly, [PERSON] et al. [PERSON] et al. (2017) continue to adapt multi-intelligent reinforcement learning to the complex and variable FANET environment by using LSTMs instead of the fully connected layers of actor-critic networks.