# DRL-Based Load-Balancing Routing Scheme for 6G Space-Air-Ground Integrated Networks
## 6 Conclusions  
We proposed a Markov decision-process model for route generation and introduced a deep-reinforcement-learning-based LEO satellite-routing algorithm called DQN-LLRA in this paper. In the DQN-LLRA scheme, each routing node is controlled by a DQN agent, which can quickly select the best next hop for the current satellite node. The algorithm only relies on the state information of the surrounding nodes of the current satellite node  
Figure 14: Path average queue utilization rates of different algorithms under various test flows. and obtains the optimal decision model through iterative training. Compared with the traditional routing algorithm through numerical simulation, our algorithm can reduce the maximum queue utilization and the average queue utilization of the routing path, reduce the occurrence of congestion to a certain extent, and realize the satellite network routing under load balancing subject to the condition of ensuring that the delay cost increases only a little. This paper also found that when the number of satellite nodes increased to thousands, the routing path exceeded 20 hops, and then the convergence speed of the DQN-LLRA algorithm decreased, and the accuracy of the decision model also decreased. Therefore, using more effective neural network structures such as Graph Neural Networks (GNN) for network feature extraction is an important direction for future research. Although the proposed load-balancing routing algorithm for LEO satellites based on Deep Q-Network shows promising performance improvements compared to traditional routing algorithms, there are still some limitations and areas for further research. Firstly, the proposed algorithm is evaluated through simulations, and it remains unclear how well it would perform in real-world scenarios. Further experiments and measurements in actual LEO satellite networks are necessary to validate its effectiveness. Secondly, the proposed algorithm considers delay, bandwidth, and queue utilization of the surrounding satellite nodes as factors for selecting the best routing results. Other factors, such as link reliability, congestion level, and energy efficiency, may also be considered in future work. Lastly, while deep reinforcement learning has shown great potential in improving network routing, it is a computationally expensive approach that requires significant training time and resources. Hence, developing more efficient and scalable algorithms that can maintain good performance with a lower computational overhead is also an important direction for future research. The proposed DQN-LLRA algorithm is a novel approach for load-balancing routing in LEO satellite networks. However, further research is needed to address the abovementioned limitations and achieve more robust and efficient routing. Conceptualization, F.D. and J.S. ; methodology, F.D. and Y.Z. ; software, J.S. ; validation, F.D., J.S. and Y.Z. ; formal analysis, Y.Z. ; investigation, F.D. ; resources, F.D. and Y.Z. ; data curation, J.S. ; writing--original draft preparation, J.S. and Y.W. ; writing--review and editing, J.S. and F.D. ; visualization, J.S. ; supervision, F.D. ; project administration, F.D. and Y.Z. ; funding acquisition, Y.Z. and T.H. All authors have read and agreed to the published version of the manuscript. This paper was supported in part by the National Key Research and Development Program of China under Grant 2020 YFB1807500 and in part by the 54 th Research Institute of China Electronics Technology Group Corporation Cooperative Project under Grant SKX212010039. The data presented in this study are available on request from the corresponding author. These authors would like to thank the China Electronics Technology Group for providing some orbital statistics. The authors declare no conflicts of interest. The following abbreviations are used in this manuscript: