## 1 Introduction  
One of the key targets of the 6 th Generation (6G) mobile network is ubiquitous coverage and broadband network access worldwide [1]. While extending the terrestrial infrastructure in remote locations is often infeasible due to financial, physical, or legal reasons, network omnipresence is expected to be achieved via the integration of Terrestrial Networks (TNs) and Non-Terrestrial Networks (NTNs) [2, 3, 4], including both aerial and satellite systems. Nowadays, carrier-grade aerial networks are largely limited and correspond mainly to High Altitude Platform Systems (HAPSs) or ad hoc networks based on Unmanned Aerial Vehicles (UAVs) (set up dynamically during natural disasters, mass events, etc.). Currently, the primary interest lies in the integration of TNs with already operating satellite systems with a focus on Low Earth Orbit (LEO) systems due to acceptable latency (30-50 ms) [5]. The LEO networks, however, introduce several challenges caused by the fast mobility of infrastructure causing frequent topology changes [6] and severe capacity constraints--bothInter-Satellite Links (ISLs) and Feeder Links (FLs). To maintain service continuity and economise NTN resources, it is necessary to adopt intelligent Traffic Engineering (TE) for Quality of Service (QoS) routing, End-to-End (E2E) cross-layer operations, and network-wide load balancing. The latter is critical to mitigating the impact of the disappearing links and related distribution of rerouted traffic across multiple links (not necessarily the closest ones as in shortest path routing). In TNs, Deep Reinforcement Learning (DRL) is a common candidate method for TE due to high performance and adaptability [7; 8]. Its application in NTNs is, however, problematic due to the strong dependence of the learned policies on the problem setup. Time-variant environments complicate the exploitation of traditional \"deep\" architectures and degrade DRL's models convergence. Moreover, DRL faces observability and scalability issues, and its usage in distributed and decentralised environments is still being heavily researched [9; 10; 11; 12]. Software Defined Networking (SDN) is often perceived as a vital concept to introduce flexibility into the network and facilitate cross-layer integration [13]. The logically centralised software-driven control, support for TE, and capabilities to leverage contextual information are essential to optimise networking in the evolving NTN segment. The usage of SDN in carrier-grade networks is, however, still limited due to scalability, interoperability [14], and SDN Controller (SDNC) placement issues [15]. While distributed architectures address the above partially, they complicate E2E TE and E2E QoS management and require SDNCs coordination. In this paper, based on the authors' previous work [16], proposed is a novel TE approach called 3D QoS-Aware Routing (3 DQR) for optimising routing and path allocation in future mobile Three Dimension (3D) networks. The key contributions of this paper are as follows:  
* Considering the heterogenous and distributed character of emerging 6 th Generation (6G) networks, introduced is a TE framework based on distributed (multi-controller) hierarchical SDN and Multi-Agent Deep Reinforcement Learning (MADRL), which enables flexible global and per-segment optimisation of 3D networks (i.e., comprising terrestrial, aerial, and satellite segments). The proposed framework improves load distribution and flow acceptance rate while considering QoS requirements of individual flows and traffic priorities. Moreover, 3 DQR tries to minimise the number of broken paths and path reconfigurations to reduce session disruptions and SDN Control Plane (CP) overhead to improve SDN scalability. * A hierarchical MADRL routing and path allocation strategy is developed, which involves intelligent DRL-Graph Neural Network (GNN) TE agents leveraging message passing and attention mechanisms as well as network topology predictions to improve the reasoning of agents and adaptability to frequently changing 3D network topology.