# An Integrated DQN and RF Packet Routing Framework for the V2X Network
## 3 Design of the IDRF System
### Real-Time Phase
The total \(\mathit{Transfer\ Time}\) from when \(V_{1}\) and \(V_{5}\) meet until their connection is broken is \(36\) s + \(36\) s = \(72\) s. The minimum arrival time of \(V_{1}\) and \(V_{5}\) at intersection \(I_{5}\) is \(\min\{28.8\) s, \(31.5\) s\(\}=28.8\) s. Finally, the modified \(\mathit{Transfer\ Time}\) required to finish the packet transmission within road segment \(S_{1}\) is \(28.8\) s \(-12\) s = \(16.8\) s.  
Figure 21: Diagram of the estimated vehicle connection stability for \(v_{1}\). After creating a road segment table based on the estimated vehicle link stability, the prediction unit will forecast the next intersection that each vehicle moves towards. Based on the associated RSU, intersection tables are compiled with fields recording the arrival time at the next intersection, the next intersection, the predicted intersection after the next, and the predicted arrival time to the intersection after the next. We can calculate the average speed for each road segment by utilizing the average vehicle speeds from different periods. Based on the distance between the next intersection \(I_{i}\) and the predicted intersection after the next \(I_{j}\), we can estimate the time taken for a vehicle to travel between intersections \(I_{i}\) to \(I_{j}\) and record it as the expected arrival time to the intersection after the next field (Figure 23). Finally, we propose two cases to compute the minimum packet propagation delay of a road segment. First, based on the vehicle connection stability on the road segment table, if the link connection time between any two vehicles on a road segment is greater than or equal to the theoretical packet transmission time and any vehicle is within the communication range of its peers, the packet can be propagated from the originating intersection through vehicles within the road segment to the next intersection. In this case, the packet propagation delay on the road segment (\(D_{S}\)) equals the smallest number of vehicles required to forward the packet on the road segment (\(N\)) multiplied by the theoretical packet transmission time (\(P_{T}\)), which is formulated as \(D_{S}=N\times P_{T}\). Second,  
Figure 23: The addition of the predicted intersection after the next and the expected arrival time fields to the intersection after the next into intersection tables. \[d=\sqrt{{D_{xij}}^{2}+{D_{yij}}^{2}}\leq r \tag{7}\]0 The road segment table for \(v_{1}\) after adding the vehicle connection stability (the dotted lines represent the vehicles with zero _Transfer Time_). if the relay vehicle finds no neighbors within its communication range, the extra packet propagation delay \(M_{T}\) must be added to the _Encounter Time_ required for the relay vehicle to meet the next hop vehicle. The total packet propagation delay on the road segment equals the sum of \(N\times P_{T}\) and \(M_{T}\), which is formulated as \(D_{S}=N\times P_{T}+M_{T}\). We can obtain vehicle transmission paths with sufficient historical data through training units using DRL (Figure 17). However, in scenarios involving real-time vehicles with insufficient data, the information from these vehicles can be incorporated into the decision-making process of the DQN model using the method proposed above for computing the minimum packet propagation delay for a road segment. Consequently, the analysis unit of the RSU can determine whether a new transmission path can be formed, as illustrated in \[d=\sqrt{{D_{xij}}^{2}+{D_{yij}}^{2}}\leq r \tag{7}\]1  
2.3 Real-Time DQN Routing Decision for the Packet Transmission Path and the Exception Handling Mechanism by the Integration Unit  
After the DQN decides to choose the relay node \(v_{relay}\), the SDN CN transmits the \(v_{relay}\) information to the RSU located at the intersection. Then, the RSU notifies \(v_{carry}\) that its packet needs to be forwarded to \(v_{relay}\).