# An Integrated DQN and RF Packet Routing Framework for the V2X Network
###### Abstract
The algorithm emphasizes the establishment of routing paths for traffic and vehicle density predictions to increase the packet forwarding probability, reduce the transmission delay, and minimize the packet-carrying instances. Both VRDRT steps are implemented using DRL. The first step explores the optimal paths from the current path, and the second searches for potential routing paths. A RSU collects and stores traffic information from all vehicles in each road segment. It uses DRL to predict road traffic conditions and calculate delays and destinations. However, the algorithm does not account for the latest packet forwarding paths or the reliability of inter-vehicle links. This study adopted the SDN architecture to improve the performance of the AI training phase and to allow real-time optimal decision-making. The proposed algorithm is divided into offline and real-time phases. The offline phase includes several algorithms that correct the GPS trajectory, determine the actual vehicle movement trajectories, add information on vehicle arrival, and estimate the average vehicle speed on the road to rectify historical trajectory data. Next, we proposed the first routing strategy of the proposed Integrated DQN and Random Forest (RF) [19; 20] Packet Routing Framework (IDRF), known as IDRF_DQN. The strategy exclusively utilizes the DQN for optimal relay node selection and addresses the excessive storage consumption and searching delays in Q-learning caused by numerous action choices. IDRF_DQN solely relies on the DQN to select the vehicle with the highest delay weight as the packet relay node and does not consider the movement directions of the target vehicle or the newly joined vehicles with lower end-to-end path delays during packet transmission. Thus, it may lack the ability to determine the shortest end-to-end delay path, or the packet may fail to meet the target vehicle based on the real-time VANET environment. Conversely, if the routing scheme solely relies on RF to select an intersection for packet forwarding without considering which relay node is optimal, some vehicles may experience long delays when carrying packets. Furthermore, if the communication time between vehicles is too short, incomplete packet transmission may occur. To address the abovementioned issues, we designed a second routing strategy within the IDRF framework, IDRF_DQN_RF. The algorithm utilizes a combination of the DQN and RF approaches. The delay weights of relay nodes computed by the DQN and the intersection transfer probabilities calculated by RF are recalculated as their transfer delay weights to determine a new relay path, addressing the problem of exclusively relaying to the vehicle with the highest delay weight determined by the DQN. Furthermore, we proposed an algorithm to estimate vehicle link stability, determine the feasibility of packet transmission with neighboring vehicles, and calculate the minimum road segment delay. This approach prevents relays from carrying a packet for a long time or incomplete packet transmission due to inadequate vehicle communication time. However, the IDRF_DQN_RF approach still does not account for the vehicles not included in the training process. In this study, we defined vehicles that are not incorporated in DQN training as \"real-time new vehicles\". The packet transmission path generated by new vehicles may be better. Therefore, in this study, during the real-time phase, we calculated the end-to-end delay and vehicle link reliability based on vehicle information. In combination with offline RF and DQN models, we established a comprehensive IDFR routing strategy to attain the shortest end-to-end delay based on the optimal packet transmission path for real-time vehicles based on the current VANET environment.