## 3 Related Work
A two-hops state aware Double Deep Q-Network (DDQN)-based routing strategy for LEO networks called DRL-THSA [34] assumes that a node needs only link-state information of neighbouring nodes up to two hops away for the prediction of the next hop. The approach shows solid performance regarding the E2E delay, throughput, packet drop rate, and traffic distribution. A DRL-based satellite routing method solving multiple path requests at once to minimise latency is proposed in [35]. The centralised DRL, however, does not offer high performance, especially in wide-scale network deployments in which components are spatially distant due to network observability issues. Therefore, to optimise complex and distributed networks, such as NTN and 3D networks, a common approach is to use Multi-Agent Deep Reinforcement Learning (MADRL) methods featuring flat or hierarchical structures. A two-stage MADRL framework has been introduced for mobile edge computing [11], which employs high-level agents for the optimisation of transmitting power and duration of wireless transfer power, and low-level agents to improve offloading decisions of wireless devices, time allocation for offloading, and used computing resources. A distributed routing algorithm based on the Transformer-MIX architecture and MADRL, which optimises E2E latency and load balancing, has been proposed [9]. In the concept, the LEO satellites can make individual next-hop decisions for the packets in real-time while participating in the centralised training scheme focusing on load balancing. A MADRL approach, which uses agents enhanced with self-attention mechanisms to first better understand the semantics of network state and then optimise routing in LEO networks, has been proposed [10]. The method enables lowering E2E latency and improves average throughput compared to other distributed routing methods. Another emerging trend in AI-driven routing is using GNNs for more efficient network state representation and predictions to fuel AI reasoning and performance. A method using GNNs is proposed to handle routing strategies devoted to finding the shortest path and maximising the minimum allocated bandwidth for improved load distribution across network links in [36]. Another approach uses GNNs to predict packet delay distribution and loss and uses these predictions for routing optimisation and network planning [37]. A Graph-Aware Deep Learning model is proposed [38] to maintain precise network measurements along with current positions and destinations of forwarding tasks. The Contact Graph Routing algorithm is proposed with a mix of reinforcement and Bayesian learning for delay-tolerant networks in [39]. A topology-aware GNN framework for link prediction highlighting the trade-offs between the algorithm precision and speed, which can also be used for routing, is introduced in [40]. Another GNN-based routing method for terrestrial optical networks combining Message Passing Neural Network (MPNN) and DRL is proposed in [41]. The method yields good performance and generalises well on unseen topologies. A graph-based MADRL approach called GraphPR has been proposed [12] to optimise routing in satellite networks with partial observability. The method employs Graph Attention Networks to transform satellite states (e.g., location, queue, neighbour states), which are later processed by MADRL to make routing decisions. Due to the increasing popularity of GNNs, a benchmark system aimed at accelerating the development of quality GNNs has been developed and described [42]. Clearly, 3D networks will comprise multi-tier, multi-dimensional and heterogeneous network architectures with different capabilities and properties that will complexify QoS-driven communication [43; 44]. The most popular State of the Art (SotA) approach is the adoption of distributed architectures with extensions implementing E2E functionality via cross-domain coordination.