# An Integrated DQN and RF Packet Routing Framework for the V2X Network
## 3 Design of the IDRF System
### Real-Time Phase
#### 3.2.4 Integration of DQN and RF Using the Real-Time Decision Unit
As illustrated in Figure 27a, assuming that both the source and destination vehicles have been trained, DQN is used to determine the forwarding path with the shortest end-to-end delay from the source to the destination. In this scenario, \(v_{4}\) and \(v_{10}\) represent real-time new vehicles, while the remaining vehicles are relay nodes trained by DQN. Hence, it is necessary to determine whether a new packet forwarding path can be formed by \(v_{4}\) and \(v_{10}\). The estimated connection stability of the inter-vehicle connections can be used to determine whether a new link can be established between two vehicles. It is assumed that \(v_{10}\) cannot form a new link, but \(v_{4}\) can connect to \(S\) and \(V_{7}\), as illustrated in Figure 27b. After adding \(v_{4}\) to the original DQN path, three new packet forwarding paths can be established: \(S\to V_{3}\to V_{6}\to V_{9}\to D\), \(S\to V_{4}\to V_{7}\to V_{11}\to D\), and \(S\to V_{4}\to V_{7}\to V_{12}\to D\). Since \(v_{4}\) is a newly introduced real-time vehicle, it lacks the DQN-trained weight. The reward between \(v_{4}\) and its neighbor vehicles is calculated using Equation (5) and is used as the weight of the corresponding hop, as presented in Table 2. First, we calculate the total delay weights for the three paths without considering intersection transfer probabilities. Since path 1 has the highest total delay weight, it is selected as the packet forwarding route, as illustrated in Table 3. Further, packets at Hop 1, Hop 2, and Hop 4 in Figure 27b are transferred to the relay vehicles on road segments. Intersection transfer probabilities calculated by RF are not used at these hops. However, because packets at Hop 3 are transferred to the relay vehicles at intersections, new total delay weights must be calculated for these three packet forwarding paths while considering the intersection transfer probabilities. It is assumed that the intersection transfer probabilities for vehicles at Hop 3 are those shown in Table 4. First, the new delay weight \(H_{w}\) is calculated by multiplying the original DQN delay weight \(D_{w}\) by the RF intersection transfer probability \(R_{w}\) (Equation (9)). The results are presented in Table 5. Next, the IDRF is used to calculate the new total delay weights of these three packet forwarding paths (Table 6). Since Path 3 has the highest total delay weight, the IDRF selects it as the new packet forwarding path. This example demonstrates that without considering the real-time vehicle \(v_{4}\) and intersection transfer probabilities calculated by RF, Path 1 is chosen as the packet forwarding path instead of Path 3, as it has the highest total delay weight, which means that it has the shortest end-to-end delay. \[H_{w}=D_{w}\times R_{w} \tag{9}\] \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Link** & \begin{tabular}{c} **Hop 1 Delay/DQN** \\ **Delay Weight** \\ \end{tabular} & **Link** & \begin{tabular}{c} **Hop 1 Delay/DQN** \\ **Delay Weight** \\ \end{tabular} \\ \hline \(S\to V_{1}\) & 1.40 s/0.0702 & \(V_{1}\to V_{5}\) & 0.70 s/0.5388 \\ \hline \(S\to V_{2}\) & 0.40 s/0.2463 & \(V_{2}\to V_{5}\) & 1.70 s/0.2218 \\ \hline \(S\to V_{3}\) & 0.17 s/0.5789 & \(V_{3}\to V_{6}\) & 2.60 s/0.1450 \\ \hline \(S\to V_{4}\) & 0.94 s/0.1046 & \(V_{4}\to V_{7}\) & 4.00 s/0.0944 \\ \hline \multirow{2}{*}{**Link**} & \
\begin{tabular}{c} **Hop 3 DQN Delay** \\ **Weight** \\ \end{tabular} & **Link** &
\begin{tabular}{c} **Hop 4 DQN Delay** \\ **Weight** \\ \end{tabular} \\ \hline \(V_{5}\to V_{8}\) & 0.0491 & \(V_{9}\to D\) & 0.1420 \\ \hline \(V_{6}\to V_{9}\) & 0.4270 & \(V_{11}\to D\) & 0.4806 \\ \hline \(V_{7}\to V_{11}\) & 0.1723 & \(V_{12}\to D\) & 0.4615 \\ \hline \(V_{7}\to V_{12}\) & 0.3516 & & \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The hop delays and DQN delay weights \(D_{w}\) for each hop.