###### Abstract  
In the communication-based train control (CBTC) system, traditional modes such as LTE or WLAN in train-to-train (T2T) communication face the problem of a complex and costly deployment of base stations and ground core networks. Therefore, the multi-hop ad hoc network, which has the characteristics of being relatively flexible and cheap, is considered for CBTC. However, because of the high mobility of the train, it is likely to move out of the communication range of wayside nodes. Moreover, some wayside nodes are heavily congested, resulting in long packet queuing delays that cannot meet the transmission requirements. To solve these problems, in this paper, we investigate the next-hop relay selection problem in multi-hop ad hoc networks to minimize transmission time, enhance the network throughput, and ensure the channel quality. In addition, we propose a multiagent dueling deep Q learning (DQN) algorithm to optimize the delay and throughput of the entire link by selecting the next-hop relay node. The simulation results show that, compared with the existing routing algorithms, it has obvious improvement in the aspects of delay, throughput, and packet loss rate. communication-based train control (CBTC); train-to-train(T2T) communication; relay selection; multiagent dueling DQN +
Footnote †: journal: Sensors  
1
Footnote 1: email: [EMAIL_ADDRESS]. and base station are complex, which makes network construction and maintenance cost high. Therefore, new technologies such as reconfigurable intelligent surfaces [8; 9] and wireless ad hoc networks have been proposed to improve T2T communication. The wireless ad hoc network is a novel approach to improve the performance of CBTC systems. In the wireless ad hoc network, the packages are sent to the wayside node from the on-board node. Similar to the vehicular ad hoc network (VANET), the role of wayside nodes is to assist packet transmission [10]. Therefore, the packages are transmitted through the transmission network formed by the wayside nodes hop-by-hop and finally to the running train, so that the transmission link is more stable. Furthermore, the deployment of wireless ad hoc network nodes is less costly, and it does not have a fixed topology or require any fixed infrastructure to communicate, allowing it to be deployed more flexibly and configured quickly. The relay selection strategy plays an important role in wireless ad hoc networks to reduce transmission delay, improve throughput, and decrease packet loss. However, since wireless ad hoc networks are rarely used in CBTC systems, no suitable routing strategy has been proposed. Therefore, the existing strategies from VANETs and mobile ad hoc networks (MANETs) should be considered and improved to adapt to the CBTC system. In VANET and MANET relay selection strategies, the routing algorithms are generally divided into two types: proactive routing and reactive routing [11]. Traditional proactive routing approaches, such as optimized link-state routing (OLSR) [12] and destination sequenced distance vector (DSDV) [13], require significant overhead for node exploration and maintenance of routing tables, which is not feasible for CBTC systems with strict requirements for low latency. At the same time, although the traditional reactive routing greedy perimeter stateless routing (GPSR) can directly select the next-hop relay, it cannot fully consider various factors such as node congestion and channel quality. In the CBTC system, the transmission channel can be affected by the high mobility of the train, causing shadow fading and multipath fading, which leads to a sudden change in transmission link. Therefore, the relay selection strategy must be able to make decisions according to the varying channel state. On the basis of past training experience, the learning-based routing algorithm can make real-time decisions well by observing current channel state. Meanwhile, in deep reinforcement learning (DRL), the optimization problem of multiple factors can be transformed to maximize cumulative rewards [14].