## 2 LEO Satellite Networks Model
### Two-Hops State Aware Updating
Therefore, when the transmission rate increases, the link state during the routing path may change from the idle state to the congestion state, so that the end-to-end delay of the data packet increases. Since the TLR and ELB do not consider

Figure 4: The average end-to-end delay with different transmission rates. Figure 5: The average end-to-end delay with different number of flows. the impact of short-term light traffic fluctuations on routing computation, the short-term light traffic load will lead the TLR and the ELB to choose this link. Therefore, the packets might be sent to the short-term light traffic load nodes and increase the congestion level, which brings a longer average end-to-end delay. Another reason is that the DRL-THSA alternates path according to route state within two-hops, which avoids more queuing delay. The routing strategy is determined by the DDQN model, which is per-training. The more accurate estimation of the average traffic rate, the more accurate the state entered into the DDQN, which provides a more optimal route than TLR and ELB. In Figure 5, the average end-to-end delay of DRL-THSA increases with the increasing of flows. Since the congestion of node is tried to be avoided, the packets will be transmitted on another routing path, which increases the average end-to-end delay. #### 4.2.2 Packet Drop Rate

The performance of DRL-THSA is evaluated by the total packets drop rate. The total packets are recorded to obtain the drop rate. Figures 6 and 7 show the total packet drop rate with different transmission rates and different number of flows. It should be noted that the links between satellites are assumed as error-free. Thus, the packets are dropped when the queue buffer of the satellite is not enough. In other words, the satellite has been congested. Furthermore, when the time to live (TTL) of a packet decreases to zero, this packet is dropped. It can be seen that the packet drop rate of DRL-THSA is lower than that of the ELB, TLR and ELMDR. According to the routing mechanism, ELMDR has the highest packets drop rate. The reason is that the pass back routing information of ELMDR under high traffic load may be outdated. At the same time, the packets drop rate of ELB is higher than that of the TLR. The reason is that the congestion at the current hop is not considered in the ELB. Therefore, packets might be dropped before sending. In DRL-THSA, the DDQN considers the node state within two-hops and routes the packets through the optimal path. Thus, the DRL-THSA can be seen as a dynamic optimal routing strategy which can avoid the congestion before it occurs. The TLR only considers the node state within one hop, and the congestion might be occurring at the next hop when the next hop cannot find a suitable node to route the packets. In Figure 7, the packets drop rate increases with the increasing of the number of flows. The reason is that since more packets are routed to the sub-optimal next hop, the TTL of packets is more likely to decrease to zero. Figure 6: The packet drop rate with different number of rates. #### 4.2.3 System Throughput

Figures 8 and 9 show that the DRL-THSA has the highest throughput among the three routing strategies. It should be noted that the traffic load has been balanced among all the satellites in DRL-THSA strategy, resulting in higher throughput than that of ELB, TLR and ELMDR. Figure 8: The total throughput with different sending rates. Figure 7: The packet drop rate with different number of flows. Figure 9: The total throughput with different number of flows. #### 4.2.4 Traffic Distribution Index

The traffic distribution index in [10] is used to investigate how well the traffic is distributed over the entire constellation, which can be expressed as

\[\textit{Index}=\frac{{\left(\sum\limits_{i=1}^{n}{{x_{i}}}\right)}^{2}}{{n\sum \limits_{i=1}^{n}{{x_{i}^{2}}}}} \tag{18}\]

where \(n\) is the number of ISLs and \(x_{i}\) represents the actual number of packets that traversed the \(i^{th}\) ISL.