###### Abstract
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Categories** & **Advantage** & **Shortcoming** & **Delay** \\ \hline \multirow{4}{*}{LEO} & low launch costs, low & \multirow{4}{*}{short coverage duration and large satellite constellation size} & \multirow{4}{*}{10–100 ms} \\  & communication latency, and & & \\  & high-resolution & & \\  & observation capabilities & & \\  & fixed propagation delay and & & \\  & wider coverage area compared & & \\  & to LEO & & \\  & high operational stability and & & \\  & extensive coverage capability & & \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Characteristics of satellites in different orbits (LEO, MEO, GEO). Figure 1: DV-DVTR schematic diagram. The concept of virtual nodes was first proposed by [PERSON] and [PERSON] [6], as shown in Figure 2. This strategy uses the concept of the logical positions of satellites to form a globally covered network, where each logical node is associated with the closest satellite in terms of physical proximity. In recent research, [PERSON] [7] proposed the Temporal Netgrid Model (TNM) to optimize the virtual node strategy, replacing the concept of virtual nodes with virtual grids, where each grid is serviced by its corresponding satellite. Thus, satellite positions can be represented by grid coordinates. Some scholars have proposed distributed satellite routing algorithms based on reinforcement learning, which hold significant value. [PERSON] [8] used Q-routing to determine data packet forwarding routes in LEO satellite networks. [PERSON] [9] employed deep reinforcement learning, replacing Q-tables with neural networks to store Q-values. These methods require global information for training, leading to high loads. [PERSON] [10] proposed a reinforcement learning-based dynamic distributed routing scheme, treating each satellite node as an agent for training and routing. However, this method had relatively slow convergence. [PERSON] [11] proposed a routing strategy for LEO satellite networks based on state information and load conditions within two hops of the satellite. [PERSON] [12] also proposed a reinforcement learning method, considering factors such as satellite signal-to-noise ratio, transmission rate, and delay, to design a reward function for global optimal routing. [PERSON] [13] proposed an adaptive routing algorithm based on multipath transmission and Q-Learning. This algorithm utilized an improved breadth-first search algorithm to obtain multiple shortest paths and incorporated Q-Learning to optimize path selection in dynamic network environments. Simulation experiments demonstrated that the proposed algorithm effectively enhanced the throughput of inter-satellite networks, reduced transmission latency and packet loss rate, and adaptively selected optimal paths even in the absence of global topology information. Although significant progress has been made in the research on routing algorithms for inter-satellite networks, several challenges remain in practical applications. For instance, as the number of satellites increases and the network scale expands, further optimization of routing decisions in highly dynamic environments, reduction in computational complexity, and effective handling of bursty traffic and link failures are still critical issues that require in-depth investigation. In this paper, we propose the Multi-Agent Reinforcement Learning-Based Joint Routing (MARL-JR) algorithm, which integrates centralized ground-station training to optimize initial routing performance for satellite networks with distributed in-orbit training to enable real-time link-state awareness and rapid routing decisions. The main contributions of this study are summarized as follows:  
Figure 2: Virtual node strategy. * We propose a joint reinforcement learning method named MARL-JR; it has lower end-to-end latency and more balanced network load. * We propose Q-table Initialization, which allows the routing algorithm to have a better performance during the initial deployment phase of the satellite network.