## 3 System Model and Problem Formulation
### DRL-SFCR Training Algorithm  
In the experiments in this paper, GDRL-SFCR adopts a centralized training architecture, where the training nodes are located at the logical center node where the centralized network controller is located. This is mainly because the GNN needs to integrate network topology, node state, and link load information to generate a global graph representation, and a centralized architecture facilitates real-time access to network data. In addition, centralized training ensures that all nodes execute the same policy version and avoids decision conflicts caused by unsynchronized policies in distributed training. The training data are obtained from the full network topology at a fixed period.  
This paper uses PPO [50] for training, which employs an actor-critic structure. The actor is the policy network \(\pi_{\omega}\), and the critic is the value network \(\pi_{\varphi}\) used to predict the value of state st. In this paper, we design the critic as an MLP, which inputs the computed state to obtain \(v_{\varphi}(s_{t})\). \(MLP_{\pi}\) and \(MLP_{\varphi}\) have the same structure, with two d1 dimensions and a tanh-activated hidden layer, but with different parameter \(\varphi\) and input dimensions. The routing agent is trained as shown in Algorithm 1 with \(max\_episode\) training iterations, during which the DRL agent first initializes the environment and obtains the node embeddings of the current network using the GNN, followed by using the MLP to obtain the features of the current node and its surrounding neighboring nodes and calculating their inner product as the input state \(S_{t}\). With this information as input, the actor network generates actions \(a_{t}\). The environment computes the reward and the next state \(S_{t+1}\) based on the current action a1, obtains this information, saves it in the cache queue, uses MSE to compute the loss, and uses the Adam optimizer to optimize the network parameters. The policy is validated in a new environment every five iterations during the training period.  
Although GDRL-SFCR performs well in end-to-end routing decision optimization, its computational cost still needs to be considered. (a) GNN message passing overhead: the time complexity of node embedding updates is \(O(N^{2})\), where N is the number of network nodes. (b) The agent needs to interact with the environment a great deal in order to converge on the routing decision policy.  
Considering the traffic problem that may be caused by the large-scale features of the nodes in the LEO-MSN network, the solution proposed in this paper involves a series of optimizations, which include the following. (a) Reducing the dimensionality of the nodes' features: in this paper, the GNN adopts 32-dimensional node features and combines them with 8-bit quantization. The single node feature packet is reduced from 512 KB to 256 KB. (b) DRL model lightweighting: the DRL strategy network fuses GNN-MLP features, and it is trained with 16-bit mixed precision with gradient sparsification (threshold \(\theta=1\times\) 10\({}^{-5}\)). The single model update is compressed to 154 KB, and the update frequency is adaptively adjusted to two times per hour. (c) The feature data are transmitted in bulk using the Polar Satellite with a 12-times-over-head window every day (the top of which is not exceeded by the top of the window), and the feature data are transmitted in bulk using the Polar Satellite over the top window batch transmission (8 min each time). Therefore, the total global daily communication amount is 6000 \(\times\) (256 KB \(\times\) 4 + 154 KB \(\times\) 12) = 16.43 GB, and the bandwidth requirement is 16.43 \(\times\) 8 \(\div\) (24 \(\times\) 3600) \(\approx\) 1.55 Mbps. This value is significantly lower than the earth station uplink capacity (>10 Mbps), proving that the scheme is feasible.