## 5 Algorithm
### E2E Operation and DGA Architecture
Section 6.1);
* Transfer capabilities--comprising performance tests of DGAs trained in one topology and operating in previously unseen topologies with different topological properties (cf. Section 6.2);
* Aggregation impact--verifying the impact of interval of topology aggregation by GIMMF on 3 DQR performance under low traffic load (cf. Section 6.3). 3 DQR was tested in over 30 different topologies comprising both sparse and dense networks (T1-T32) with TN nodes located within western and central Europe. The training and performance evaluation (s0-x, s1-x) were conducted in reference topologies comprised of two TN graphs (32 nodes--50% relays, 25% gateways, and 25% gNodeBs) with average node degrees equal to four (T1) and seven (T2); and (ii) two NTN graphs--subsets of

Figure 4.6 Complexity comparison of 3 DQR, H-SP, and SP routing for different average node degrees. Starlink satellite constellations containing 32 nodes (T1) and 24 nodes (T2), both with an average node degree equal to four (fixed due to limited number of optical interfaces onboard LEOs). To verify transfer capabilities, 20 previously unseen topologies were used, each comprising 16-80 TN and NTN nodes with average node degrees of 3-6. The TN-NTN connections are calculated dynamically based on LEOs proximity to the TN gateway and the visibility criterion (elevation \(\geq\) 30\({}^{\circ}\)). Domain-level links and FLs have a capacity of 100 and 200 kilounits, respectively, \(\delta=10^{-8}\), and delay is calculated based on the spatial distance between nodes. For UE-LEO links, the worst-case delay of 13 ms (the largest value at 600 km orbit as indicated by the 3 GPP [48]) and PER of \(10^{-7}\) were adopted. For UE-TN, the Radio Access Network (RAN) delay of 2 ms and \(\delta=10^{-8}\) were assumed. Finally, the adopted QoS classes of arriving flows correspond to expected NTN-based services (cf. Table 2). The arriving flows have a duration of 0.5 to 20 min (generated randomly with uniform distribution) and an equal rate per-QoS class (25% each). In each test, an equal number of flows per domain was generated, i.e., 1/3 inter-domain flows and 2/3 intra-domain flows (equal portions for TN and NTN segments). To set up the DRL models, tests were conducted to fine-tune the network architectures and hyperparameters (e.g., \(k\), \(\tau\), \(\gamma\)) to maximise the models' performance. As a result, the following configuration was derived for DQRs/M-QR and DGAs: \(k=8\), \(\gamma=0.85\), \(\tau=10^{-2}\), \(\epsilon_{min}=0.01\), \(\epsilon_{decay}=0.995\), \(N=16\), \(|t|=4\), and a learning rate of \(10^{-4}\). The DGA Q-networks comprise six passes in the MPNN, GAP with MLP (64 hidden neurons and ReLu activation) as a gate function computing attention scores, and one-neuron LL for the compression. The DDQN architecture (s0-b, s1-b) comprises four LLs: 64, 32, 16, and 1 output neurons. GIMMF aggregates the topology for a 5 s window and provides the topological prediction for another 5 s (10 s into the future). In the 3 DQR case, the input size equals the number of node and edge features, while in the DDQN case, the input size equals the number of node and edge features multiplied by the number of graph links. \begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Test ID**} & \multirow{2}{*}{**Topology**} & \multicolumn{2}{c}{**Algorithm**} & \multirow{2}{*}{**Scope**} \\  & & **(TN, NTN, TN-NTN)** & \\ \hline s0-a & & H-SP & \\ s0-b & & DDQN-SP-SP & \\ s0-c & & 3 DQR & \\ s0-d & & 3 DQR-U & \\ \hline s1-a & & H-SP & \\ s1-b & & DDQN-SP-SP & \\ s1-c & & 3 DQR & \\ s1-d & & 3 DQR-U & \\ \hline s2-a & & H-SP & \\ s2-b & T1-T23 & 3 DQR (s0-c) & Transfer \\ s2-c & & 3 DQR (s1-c) & \\ \hline s3 & T1 & 3 DQR & Topology aggregation interval 1–40 s \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Conducted test scenarios. Transfer capabilities are evaluated using models trained in s0-c and s1-c tests. ### Performance

To evaluate the performance, the impact of the algorithms on network availability and load distribution is assessed.