# An Integrated DQN and RF Packet Routing Framework for the V2X Network
## 6 Conclusions and Future Work  
Due to the highly dynamic characteristics of VANET networks, the IDRF based on the SDN architecture has been adopted to incorporate historical vehicle trajectories into training and account for the addition of vehicles to the forwarding path in real time. Several algorithms have been proposed for the offline phase, i.e., GPS trajectory correction by the vehicle trajectory continuity algorithm, the determination of the actual vehicle movement trajectory, the addition of arrival information at the intersection, and an estimation of the average vehicle speed on the road segment. These algorithms rectify historical trajectories and employ DQN to determine the optimal relay node and RF to determine the intersection probabilities. Additionally, this study introduces the estimation of vehicle arrival times at intersections, the vehicle link connection stability, and the delay time at the road segment based on real-time information collected by vehicles in the real-time phase. This information is organized into road segment tables and intersection tables for each road segment and intersection. DQN and RF are then recalculated using these tables, considering the addition of new vehicles in real time to estimate the delay of new vehicles and make relay node decisions for new packet forwarding paths. In simulations using the historical GPS trajectories of 10,357 taxis traveling within Beijing city, the IDRF is shown to consistently outperform other algorithms. Compared to TDRL-RP and VRDRT, the IDRF shows performance improvements for the average packet delivery ratio, end-to-end delay, and overhead ratio with different numbers of pairs, and the transmission ranges are at least 3.56%, 12.73%, and 5.14%, and 6.06%, 11.84%, and 7.08%, respectively. This is due to its ability to adapt to real-time changes in VANET networks, which leads to an improved average packet delivery ratio, reduced average end-to-end delay, and a lower average overhead ratio for both sparse and congested periods. Thus, the IDRF addresses the limitations of traditional routing protocols such as TDRL-RP and VRDRT associated with the handling of real-time VANET networks.  
In the future, the following research issues will be addressed: First, how can DQN parameters such as reward, learning rate \(\alpha\), and discount factor \(\gamma\) be adapted to real-time VANET environments? Second, with the progress of advanced artificial intelligence models, the IDRF will be improved to transition from the current RF to the Extremely Randomized Trees model [37] and from the current DQN to the Dueling DQN [38; 39] or Double DQN [40]. The training times and simulation results of the improved IDRF will be compared to those of the original IDRF to validate this improvement.  
**Author Contributions:** Conceptualization, I-C.C.; Formal analysis, C.K. and I-C.C.; Funding acquisition, C.-E.Y. and I-C.C.; Investigation, C.-E.Y.; Methodology, Y.-S.J., C.K. and I-C.C.; Project administration, I-C.C.; Resources, C.-E.Y.; Software, Y.-S.J., Y.-H.H. and Y.-C.C.; Supervision, I-C.C.; Validation, C.-E.Y.; Visualization, Y.-S.J., Y.-H.H. and Y.-C.C.; Writing--original draft, Y.-S.J.; Writing--review and editing, C.-E.Y., Y.-H.H., Y.-C.C. and I-C.C. All authors have read and agreed to the published version of the manuscript.  
**Funding:** This research was funded by the National Science and Technology Council, Taiwan, under grant number NSTC 112-2221-E-018-009. The APC was funded by NSTC 112-2221-E-018-009.  
**Data Availability Statement:** Data are contained within this article.  
**Conflicts of Interest:** The authors declare no conflicts of interest.