## 5 Algorithm
### E2E Operation and DGA Architecture
Overall, it must be emphasised that 3 DQR provides stable performance regardless of the network load, thus addressing one of the critical issues of DRL-based approaches, i.e., poor performance and unpredictable behaviour in the previously unseen states. 3 DQR via mechanisms of MPNN and GAP (cf. Section 5.3) can overcome this problem by first extracting the key features out of the network graph and then appropriately scaling the node features according to their importance, i.e., the risk of causing congestion in the network. The second optimisation target for 3 DQR is to prioritiise the most advantageous QoS classes (i.e., with the highest QF, cf. Table 2). Figures 8 and 9 show the QoS class rejection rate for each algorithm and traffic load and gains compared to the baseline H-SP routing. First, 3 DQR enables prioritisation of the most important 5 QI class 75, reducing the drop rate in almost every case up to 4.5% in the congested scenario; however, this occurs at the cost of class 4 (total increase up to 4.5%). QoS class 4, however, consumes the most bandwidth across the classes--it is prone to rejection due to occasional congestions (e.g., due to topological reasons, as it can be observed for s0-c and \(N_{flows}=2000\)). Moreover, the

Figure 7: Rejected flows per test setups s0–s1. drop rate for the remaining classes is significantly lower and increases with the number of flow requests. The second major goal of 3 DQR is to load balance the traffic to improve resource utilisation, especially ISLs and FLs. Figure 10 shows the distributions of \(std(u_{ij})\) obtained from the environment during a single episode (aggregate of all segments). Figure 8: Rejected flows rate for tests per QoS class. Figure 9: Change of flow rejection rate per QoS class compared to baseline H-SP routing. First, 3 DQR enables a decrease in the mean \(std(u_{ij})\), which corresponds to better flow allocation decisions during the episodes (cf. Table 3). For the sparse network, the 3 DQR enables up to 52% better distribution of traffic compared to H-SP routing (s0-c). It must be noted that for dense network deployment (s1-c), the gains are consistently lower, but the impact is, respectively, lower. This is caused by the much larger average degree of the network, which leads to evaluating by M-QR multiple shortest paths with a large portion of common links. A potential solution would be to search for the shortest node-disjoint paths to minimise this phenomenon and improve the performance. 3 DQR-U, on the other hand, provides consistent gains oscillating around 4-6% for sparse comparable results to H-SP routing for dense networks. A similar trend can be observed for the classical DNN-based DDQN, which indicated the issues of the model to converge in a vast state space. Finally, as shown in Figure 10, 3 DQR features lower values of 25 th and 75 th percentiles for both sparse and dense network scenarios. In the optimal case, the Inter-Quartile Range of \(std(u_{ij})\) and mean value equal 0. While this is unachievable in real-life networks, 3 DQR features \(std(u_{ij})\) promising values (s0-c). ### Transfer

Here, an attempt to verify the ability of 3 DQR to generalise and its reasoning capabilities during the operation in previously unseen topologies is performed. To this end, selected were the best-performance models trained in the T1 and T2 topologies (cf. Section 6.1) and the agents were transferred to over 30 different topologies comprising 16-80 NTN nodes, 32-80 TN nodes, and an average node degree ranging from three to six (sparse to dense network). Also, T1 and T2 topologies were included in the tests to check how well the models native to the environment behave, compared to the transferred ones. Each test consisted of 1500 flow allocation requests with the same configuration described above. The performance comparison is shown in Figure 11.