## 2 Related Work
### Position-Based Routing Algorithms with Reinforcement Learning
The former is for the exchange of information such as the position, speed, and direction of travel between vehicles. The latter is used for vehicle nodes to collect traffic data and transmit it to the traffic control center (TCC), so that the TCC can gain a macroscopic understanding of the entire vehicle network environment, analyze the current number of neighbor vehicle nodes and the transmission delay of each node, and substitute this decision data into Q-learning, before using the Q-value to select better next-hop neighbor nodes for each vehicle node. The advantage of this approach is that it can better adapt to the high mobility and topology variability of the VANET network environment, whereas the disadvantage is that it does not refer to the future trajectory of the vehicle. 3. In the QTAR [15] network architecture, Q-learning is used for vehicle-to-vehicle (V2V) and RSU-to-RSU (R2R) networks. Decision elements, such as connection reliability and EED, are added to the Q-learning operation, and then, next-hop neighbor nodes are selected and packets are forwarded. The advantage of this approach is that the Q-learning planning method can improve throughput and PDR, but the disadvantage is that the directionality of the nodes is not considered, resulting in an unsatisfactory overall transmission performance when used in a real-world environment. 4. The QGrid [16] divides the entire map environment into equal grids, and uses Q-learning in advance. It uses the number of vehicles in the area as a parameter, and calculates the Q-table in advance to determine the transfer direction between grids, which is a macrotransfer consideration. QGrid_G uses the greedy method of packet transmission, which preferentially selects the neighbor node closest to the destination. QGrid_M uses the two-hop Markov prediction [26; 27; 28] method to predict the grid that the packet will pass through in the future, and preferentially selects the neighbor node with a higher probability of entering the next-hop grid as the next-hop node. The above two methods are microscopic transfer considerations. The advantage of these is that the transmission of packets is determined by considering the different macro- and microlevel aspects at the same time. Considering the number of vehicle nodes in each grid, the packets are preferentially transmitted to the grid with more vehicle nodes so that the packet loss rate is reduced. The disadvantage is that the vehicle density is only considered, and the influence of the overall movement direction between grids on the overall environment is not considered. The advanced QGrid is a routing protocol improvement made by QGrid, which is for vehicles with relatively regular and fixed trajectories, such as bus nodes, in the network environment. The vehicle node carries the packet to the destination grid and then continues the next-hop packet forwarding. The advantage of this is that it considers the reference vehicle trajectory, as well as the next-hop node selection, based on the trajectory information. The disadvantage is that even if the vehicle node has neighbor nodes, it still needs to continue to carry the packet until it reaches the grid destination where the node is located, resulting in a longer packet transmission delay time. We have discussed several routing algorithms that analyze vehicle trajectories and formulate designs based on them. Table 1 is a comparison of related studies. It can be found that most of the V2X routing methods lack the analysis of real vehicle trajectory data, even if they use reinforcement learning. They also lack the analysis of the source-destination direction, as well as a software-defined routing platform and two-hop next-grid routing.