# Dynamic Routings in Satellite Networks: An Overview
## 2 Related Works
Deploying specific analytical models during the initial development process can consider satellite altitude, the number of satellites in the constellation, orbital planes, and other critical network factors. Finally, the analytical model is significant for other large and complex satellite networks. Satellite communications offer lower latency and faster transmissions due to the growth of LEO satellite networks. However, the time-varying topology of the entire network makes the routing problem of the LEO satellite network challenging. Inspired by machine learning approaches to network routing problems, several researchers have studied the LEO satellite routing problem in reinforcement learning [16]. However, in current research, multi-step learning has rarely been studied to speed up training and improve learning efficiency. Therefore, the work in [17] proposed a routing scheme based on deep reinforcement learning for satellite routing (DRL-SR). This scheme aims to assign the corresponding destinations to all users' routing requests in a DTN-based satellite network. The scheme focuses on two factors: delay and link capacity. The authors suggested a non-deterministic polynomial-time hard (NP-hard) pure binary integer programming optimization approach. The authors reconstructed the model using the Markov decision process (MDP) to improve the learning rate and quality. Based on this model, the authors proposed a multi-depth Q-network (DQN)-based RL method. The solution uses multi-step learning to route user requests efficiently through the agent. In addition, the authors utilize greedy strategies to allow the agent to explore new surroundings and make numerous routing decisions for user requests. In numerical experiments, the authors evaluated the proposed scheme's performance through network delay with varying numbers of satellites and users. The simulation results show that the authors' proposed DRL-SR learning algorithm outperforms the shortest path algorithm. Current maritime communications rely mainly on satellites with weak transmission resources and lesser performance than modern terrestrial wireless networks. To meet heterogeneous business requirements, the work in [18] proposed a SAGIN framework. However, due to heterogeneity, self-organization, and time unpredictability, designing and optimizing the SAGIN framework is challenging. The primary difficulty is designing an effective routing system that can handle a highly dynamic network architecture. Recent advances in artificial intelligence have generated various routing algorithms for wireless communications, such as deep learning (DL)-assisted routing algorithms for balancing traffic in SAGINs [19]. However, in such routing algorithms, the network topology is assumed to be static, and routing decisions are made based on the state of the nodes in the global network. Therefore, to handle highly dynamic network topologies, the authors designed a deep reinforcement learning (DRL)-assisted routing algorithm for ad hoc aeronautical networking (AANET). The algorithm relies entirely on local information and can achieve near-optimal end-to-end delays. To meet the needs of heterogeneous services and to accommodate the dynamic nature of SAGINs, the authors further propose a DL-assisted multi-objective routing algorithm. The algorithm utilizes a quasi-predictable network topology and operates in a distributed manner. The author proposed a DL-assisted routing algorithm that minimizes end-to-end delay to aid comprehension. For single destination routing, each snapshot of the network topology can calculate link delays based on each node's coordinates and delay model. The approach trains a single-objective deep neural network (SO-DNN) to embed network topology information. During algorithm training, all nodes' queuing delays are constant. Then, the SO-DNN calculates the minimal delay between each source-destination pair using the shortest path technique. The authors also utilized local information to solve the multi-target routing problem for multi-target routing.