# An Integrated DQN and RF Packet Routing Framework for the V2X Network
## 2 Related Work
### Learning-Based VANET Routing Research for Packet Forwarding  
This study focused on learning-based VANET routing research for packet forwarding. Here, we present the six works most relevant to our proposed IDRF approach. Since signal propagation can be hindered by tall urban buildings, causing rapid signal decay, the intersection-based traffic-aware routing protocol with fuzzy Q-learning (ITAR-FQ) was proposed [12]. The framework comprises two main components: the real-time traffic aware process and road evaluation (RTAP-RE) and the routing decision process with fuzzy Q-learning (RDP-FQ). The RDP-FQ indicates the traffic information from adjacent roads at intersections under the operation of the RTAP-RE without utilizing actual vehicle trajectory data, neglecting end-to-end delay issues and real-time updates regarding optimal routing paths upon the entry of new vehicles. In QAGR [13], a global message-forwarding routing scheme that categorizes the environment into aerial UAVs and ground vehicles is introduced. UAVs collect global road traffic information, and global routing paths are calculated using the Fuzzy Logic and DFS algorithms. Subsequently, these paths are relayed to ground request vehicles. Vehicles maintain a fixed-size Q-table and converge toward the reward function proposed by the author. A search of the Q-table filtered by global routing paths allows route requests to be relayed to the optimal nodes. However, the proposed approach does not incorporate the actual vehicle trajectory data, thereby neglecting the end-to-end delay issue, the inter-vehicle link reliability, and the real-time update of optimal routing paths upon the entry of new vehicles. In [14] the PFQ-AODV was proposed. This is based on ad hoc on-demand distance vector routing (AODV) and integrates Fuzzy Logic to constrain Q-learning and determine the optimal routing path. The aim of this method is to address the routing selection challenges caused by the VANET environment. The available bandwidth (BWF), node mobility (MF), and link quality (LQF) are considered in Fuzzy Logic. However, this process does not incorporate the actual vehicle trajectory data, inter-vehicle link reliability, vehicle traveling direction, or real-time updates of optimal routing paths upon the entry of new vehicles. In [15], a software-defined trust-based DRL framework, TDRL-RP, is proposed. Due to the high mobility of vehicles and the scarcity of infrastructure in VANETs, communication can be affected by malicious node attacks, causing a decline in the overall network performance. TDRL-RP integrates DRL into an SDN infrastructure, aiming to learn the optimal trusted routing path in the VANET and enhance the overall network scalability, adaptability, and self-learning capabilities. DRL is deployed using SDN, and the addition of the CNN in DRL results in faster convergence than in the original neural network. Regarding forwarding strategies, a neighbor behavior trust evaluation module is introduced. However, issues related to new vehicles, end-to-end delays, or vehicle traveling directions have not been addressed. In VRDRT [18], the incorporation of DRL into VANET is introduced. The high mobility of vehicles leads to rapid changes in communication links and network topology, making it crucial to address issues related to longer transmission delays and stability. Traffic prediction and road vehicle density are emphasized during the routing path establishment process. The objective is to enhance the packet forwarding probability, reduce transmission delays, and optimize packet-carrying scenarios. The VRDRT is divided into two steps, both implemented using DRL. However, the issues of new-added vehicles, connection reliability, and the adequacy of packet forwarding in regions with higher vehicle density are not considered. To reduce the state space size and accelerate the Q-learning convergence speed, IV2 XQ [23] was developed. This contains an intersection-based V2X routing protocol using Q-learning and sets the intersection as the state.