## 3 SD-QGrid Routing Platform and Algorithms
### SD-QGrid Online Routing Decision Process
5 If dst_grid = false // calculate the two-hop weight. for\(t=\) (1, 2, 3) for\(s=\) (1, 2, 3) if \(TimeDiff^{X}_{X\to 1,t}\) and \(TimeDiff^{X}_{1,t\to 1,t,s}>0\): \(V^{X}_{X\to 1,t,s}=\) [ \(\frac{1}{Distance(X,D)}\times\frac{Qvalue^{D}_{t,s}}{TimeDiff^{X}_{X\to 1,t,s}}\times\frac{Qvalue^{D}_{t,s}}{TimeDiff^{X}_{X\to 1,t,s}}\) ] if \(Grid_{1,t,s}\) is the grid where the destination node is located: \(V^{X}_{X\to 1,t-1,s}=V^{X}_{X\to 1,t-1,s}\) *100 \(TwoHopVal_{X}=Max(V^{X}_{X\to 1,t-1,s})\) // the maximum weight among all two-hop next grid weight \(V^{X}_{X\to 1,t-1,s}\). Return \(OneHopVal_{X}\), \(TwoHopVal_{X}\) ;



## 4 Performance Evaluation

### Simulation Environment and Parameters

In this section, we use real vehicle trajectories to evaluate the performance of our proposed SD-QGrid method, and compare it with three other methods. We use Python for reinforcement learning calculations and NS-3 [32] for a network simulation experiment. The MAC protocol in NS-3 is IEEE 802.11p and the radio propagation model is the Log Distance Propagation Loss Model in this simulation. We capture the 4000-by-6000-m Taipei City area as the simulated environment, as shown in Figure 5, and use the bus and vehicle nodes within the range as the packet-forwarding nodes. Then, we add fixed node pairs as source and destination nodes on random roads in the map. The distance between the fixed node pairs is not less than 4000 m, and fixed nodes cannot be used as transit nodes for packet routing. The experimental parameters are shown in Table 8, where the number of fixed nodes is numPair, the vehicle node broadcasts a HELLO packet after a beacon time, and the fixed node generates a message packet after a message time. The parameters of the simulation experiment are the transmission range, TTL, numPair, and message time. \begin{table}
\begin{tabular}{p{113.8 pt} p{113.8 pt}} \hline \hline
**Parameters** & **Parameter Value or Range** \\ \hline \(\alpha\) & 0.8 \\ \(\gamma\) & [0.3, 0.9] \\ \(m\) & 0.1 \\ w & 3 \\ j & 1 \\ Reward & 0, 100 \\ Experimental map range & 4000 m \(\times\) 6000 m \\ Experiment time & 3000 s \\ MAC protocol & IEEE 802.11 p \\ Radio propagation model & Log Distance Propagation Loss Model \\ Buffer size & 10 MB \\ Bandwidth & 11 Mbps \\ Transmission range (m) & 400, 450, 500, 550, 600 \\ Grid size & 1000 m \\ TTL (s) & 10, 20, 30, 40, 50 \\ numPair & 50, 100, 150, 200, 250 \\ Beacon time (s) & 1 \\ Message time (s) & 1, 5, 10, 15, 20 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Simulation parameters. Figure 13: One-Two-Hop Value algorithm. We compare the following three routing algorithms with the SD-QGrid method:

* QGrid_G_[16]: A grid-based routing protocol proposed by QGrid [16]; its intergrid routing uses the greedy method to find the nearest next-hop neighbor node to the destination. * Advanced QGrid [16]: QGrid improves the routing protocol for vehicle nodes with relatively fixed trajectories, such as buses. If the vehicle node passes through the grid where the packet destination is located in the future, it will continue to carry the packet until it enters the grid, and then proceed with next-hop routing selection to transfer the packet. * GPSR [33]: A position-based routing protocol that continuously forwards packets from the nearest neighbor to the destination until it reaches the destination. We use the different transmission ranges, TTLs, numPairs, and message times, as shown in Table 8, as the horizontal axis, and use the following indicators to compare the performance of various routing algorithms. These figures are drawn with a 95% confidence interval. The number of seeds used in NS-3 is five. 1. Delivery ratio: the ratio of the successful arrival of message packets to the total number of message packets generated. 2. Average end-to-end delay: how long it takes, on average, for a message packet to travel from the source to the destination. 3. Overhead: the ratio of the total number of forwarded message packets to the total number of originally sent message packets.