###### Abstract  
This paper addresses the problem of real-time onboard routing for dynamic low earth orbit (LEO) satellite networks. It is difficult to apply general routing algorithms to dynamic LEO networks due to the frequent changes in satellite topology caused by the disconnection between moving satellites. Deep reinforcement learning (DRL) models trained by various dynamic networks can be considered. However, since the inference process with the DRL model requires too long a computation time due to multiple convolutional layer operations, it is not practical to apply to a real-time on-board computer (OBC) with limited computing resources. To solve the problem, this paper proposes a practical co-design method with heterogeneous processors to parallelize and accelerate a part of the multiple convolutional layer operations on a field-programmable gate array (FPGA). The proposed method was tested with a real heterogeneous processor-based OBC and showed that the proposed method was about 3.10 times faster than the conventional method while achieving the same routing results. onvolutional neural network (CNN); deep reinforcement learning (DRL); FPGA; low earth orbit (LEO) satellites; algorithm parallelization +
Footnote †: journal: Accepted: 6 June 2024  
but also included satellite link disconnection. The previous works described focus on LEO satellite routing algorithms, but they did not consider parallelization. Our proposed method is different in that it incorporates the previous topics and uses parallelization to accelerate the algorithm. As discussed previously, LEO satellite networks require fast and accurate routing techniques that account for frequent changes in satellite topology, making it difficult to replicate conventional ground-based methods. In a real LEO satellite network, satellite positions change in real time, and there are link disconnects. In addition, in the space environment, each satellite needs to actively and adaptively schedule and map routes without relying on ground stations. Routing algorithms based on deep reinforcement learning can be a solution to these problems because they are highly adaptive to the real-time changing environment. However, most deep reinforcement learning-based algorithms are computationally intensive; so, it is necessary to reduce the computation time before it can be used for real-time LEO satellite network routing. An additional problem is the constraints of the LEO satellite network environment. In the space environment, the computing resources used on the ground cannot be used, as such, due to the temporary alteration of bits by cosmic radiation (SEU), permanent damage to semiconductors (SEL), extreme environmental changes, and thermal differences due to the presence or absence of sunlight. Considering these problems, this paper assumes that the computing resources on a satellite are an on-board environment consisting of a central processing unit (CPU) and a field-programmable gate array (FPGA) instead of conventional ground resources. FPGAs are being studied in a variety of fields, because their reconfigurable architecture allows for development customized to user needs, offers the hardware advantage of parallel processing, and is resistant to heat and power consumption [16; 17; 18; 19]. However, FPGAs also have a disadvantage in that the performance may be inferior to a CPU if the naive routing technique used in conventional computing resources is applied; so, careful adjustment is required [20]. In this paper, we propose a parallelized reinforcement learning-based routing algorithm for accurate and fast routing in a real-time LEO satellite network environment. The parallelism is achieved through a co-design of a CPU and an FPGA. The contributions of this paper are as follows. * A practical co-design method with heterogeneous processors to parallelize and accelerate the Dueling-DQN-based routing algorithm was proposed to solve the problem of real-time onboard routing for dynamic LEO satellite networks.