## 3 FANET Routing Algorithms
### Reinforcement-Learning-Based Routing Algorithms
The algorithm starts with an LSTM-based behavioral network that performs an output operation based on the current state, followed by an intelligent body that operates, receives a reward and evolves to a new state, and finally updates the parameters of the judging network using a loss function and updates the parameters of the behavioral network using a gradient ascent algorithm. The algorithm has advantages in terms of transmission delay and transmission rate. Unfortunately, the method was only tested against traditional routing algorithms and not against other reinforcement-learning-based routing algorithms. [PERSON] et al. [PERSON] et al. (2017) also proposed a hybrid PG-based routing control algorithm. This algorithm uses a centralized training platform to guide each UAV node through policy updates, enabling the UAV nodes to collaboratively learn. In addition, the algorithm innovatively introduces a counterfactual baseline scheme to improve the convergence speed of the algorithm. In the training phase, joint values are generated by feeding the environment state and joint behavior into the neural network, after which each decentralized intelligence is trained based on the joint values, and in the execution phase, each intelligence will make behavioral choices based on its local observations. The algorithm greatly improves the collaboration between UAV nodes and effectively balances network congestion and group distribution while maintaining high throughput capabilities. Multi-intelligent reinforcement learning is also employed by [PERSON] et al. [PERSON] et al. (2017) for routing optimization. However, their algorithm is distinct in that it uses the packet loss rate and routing overhead of the routing protocol as the optimization objective. Each node is considered an intelligent agent, and its neighboring nodes are evaluated based on local information. The value function takes into account link quality, residual energy, and queue length, among other factors, to minimize the probability of routing vulnerabilities. Additionally, the protocol utilizes global rewards for collaborative data transmission. Researchers have abstracted UAVs as nodes and FANETs as topological networks, converted the FANET routing problem into a graph problem, and then used graph neural networks (GNNs) in deep learning to optimize it. For example, [PERSON] et al. [PERSON] et al. (2017) proposed a GNN-based joint optimization framework for solving the joint flight relay localization and routing optimization problem in UAV and IoT networks. The approach uses reinforcement learning algorithms to optimize the locations of all UAVs via relay GNNs and location GNNs and select optimal relay paths to improve network throughput, reduce latency, and increase energy efficiency. Experimental results show that the method has significant advantages over existing methods in solving joint optimization problems. FANET routing problems often involve large-scale nodes and edges, and the combination of GNNks and RL can handle these large-scale problems quickly and efficiently. This study provides an innovative combination of GNN and reinforcement learning, which also provides a new way of thinking about joint routing optimization under SAGIN. In SAGIN, the satellite, aerial, and ground-based networks each possess distinct network characteristics, requiring different routing algorithms to accommodate them [PERSON] et al. (2017). To address the frequent changes in network topology, it is advantageous to integrate reinforcement learning, which offers a long-term vision for sequential problems. Additionally, it is essential to consider the utilization of Markov decision processes to model the comprehensive network routing problem in SAGIN. In addition, the proposed routing algorithm should provide guarantees across multiple metrics, such as transmission delay and packet loss rate.