## 1 Introduction  
As the powerful supplement of terrestrial networks, the satellite networks are playing an increasingly significant role in the next generation global communication system [1]. Satellite networks inherently offer many advantages, such as worldwide coverage and better multicast ability. Compared with geostationary earth orbit (GEO) and medium earth orbit (MEO), low earth orbit (LEO) satellites system has a shorter delay, low propagation, and globally seamless coverage [2; 3]. However, since LEO satellite networks are usually composed of tens or hundreds of satellites, its routing problem is more complicated than terrestrial network mainly due to its features, such as dynamic link states and unbalanced traffic load caused by arbitrary flow arrival and communication hot spots [4]. Therefore, merely applying terrestrial routing algorithm on satellite networks is impracticable. Applied to transmit service data in the LEO satellite networks efficiently, an effective routing strategy is essential. Previously, many proposed routing strategies for LEO satellite networks focused on how to minimize end-to-end propagation delay. With the explosive growth of satellite applications, however, there are two defects shown in traditional satellite routing strategy: the packet drop rate at network layer becomes abnormally high, and the cumulative queuing delay during transmission gets non-negligibly large. An ant colony optimization-based routing strategy is proposed in [5] for LEO networks, it can adjust the routing path when the network topology changes, but requires a long convergence time. Load balance is an effective strategy to avoid congestion [6; 7; 8]. To guarantee a better distribution of traffic among satellites, an explicit load balancing (ELB) scheme is proposed in [9]. In ELB, the total load of the satellite is monitored. When it becomes congested, the satellite will send a notification to its neighbors and request them to decrease the sending rate. However, a part of link congestion still cannot be prevented because the ELB did not consider individual queues. To adjust the route dynamically, [PERSON] [10] set up traffic lights for each direction and proposed a public waiting queue strategy to reduce packet drop rate in [9]. However, the traffic lights routing (TLR) required global state information to calculate the best routes for each source and destination pair periodically. It may cost a lot to collect global state information due to propagation delay between satellites. Therefore, the route calculated cannot be absolutely real-time. Considering the link state may frequently change in the partial area, a state-aware and load-balanced routing (SALB) model for dynamic LEO satellite networks is proposed in [11]. However, SPT algorithm has a lack of observation of satellite networks topology. An extreme learning machine (ELM)-based distributed routing strategy is proposed in [12] for LEO networks; it can forecast the traffic at each satellite node. The simulation results show that the method in [12] has lower end-to-end delay and packet loss rate than [5]. However, the mobile agents are needed to make the routing decision in [12], which makes it difficult to track changes in network status in a timely manner. To cope with routing packets efficiently and dynamically, a method to observe LEO satellite networks topology is necessary. Deep reinforcement learning (DRL) is valuable for establishing network models [13; 14]. The authors in [15] proposed a Reinforcement Learning (RL) agent that can form an optimal policy through a series of trial-and-error processes with its environment. The RL-based routing in [16] is adaptive to link changes by choosing a next-hop node based on various local states. In [17], Deep Learning (DL) was guaranteed to be efficient for characterizing the appropriate input and output pattern in a highly dynamic network.