## 3 SD-QGrid Routing Platform and Algorithms
### SD-QGrid Online Routing Decision Process
Figure 16: Simulation comparison among different routing protocols vs. different numPairs. our proposed SD-QGrid method pays some extra overhead, but still achieves the highest transmission success rate, and also has the greatest degree of end-to-end delay reduction. In conclusion, compared with QGrid, our method increases the average delivery ratio by 16.47%, reduces the end-to-end delay by 28.65%, and pays a 1.42% overhead, which is shown in Table 12. Compared with advQGrid, the average delivery ratio increases by 10.77%, the end-to-end delay decreases by 35.08%, and the overhead is 2.31%. ## 5 Conclusions and Future Directions

In this study, we address the shortcomings of the traditional V2X grid-based routing algorithm using reinforcement learning, propose the SD-QGrid routing platform, deploy the SDN CN for centralized control, and consider the real vehicle movement trajectory in different time periods. Additionally, regarding the directionality from source to destination, we propose the SD-QGrid offline Q-learning training process to use Q-learning to calculate the Q-table of the eight neighbor grids. This paper further proposes the online routing decision process with the two-hop trajectory-based routing (THTR) packet-routing algorithm, which selects the best next-hop node to forward packets in an intragrid mode. Finally, we use the real vehicle trajectory data in Taipei to conduct simulation experiments in order to draw the data graph of four horizontal-axis parameters, which proves that compared with QGrid and advQGrid, our proposed SD-QGrid method can improve the performance with less than 2% overhead. The transmission delivery ratio is more than 10%, and the end-to-end delay is reduced by more than 25%. This research is only limited to consider types of vehicles with accurate trajectories, relatively fixed arrival times, and travel paths, such as buses. Hence, we plan to design a new routing architecture to handle packet routing for vehicles without historical trajectory information in the future. We will use a CNN (Convolutional Neural Network) and LSTM

Figure 17: Simulation comparison among different routing protocols vs. different message times. \begin{table}
\begin{tabular}{l c c c} \hline \hline  & **Average Delivery** & **Average End-to-End** & **Average Transmission** \\  & **Ratio** & **Delay** & **Overhead** \\ \hline QGrid & +16.47\% & \(-\)28.65\% & +1.42\% \\ advQGrid & +10.77\% & \(-\)35.08\% & +2.31\% \\ \hline \hline \end{tabular}
\end{table} 
Table 12: Performance improvement of SD-QGrid compared to routing protocols with message time. (Long Short-Term Memory) to establish a trajectory prediction model. This system will include a packet-forwarding decision model based on Deep Q-Learning. Through the predicted vehicle trajectory and the packet-forwarding decision model, this system can select a better next-hop node to forward packets. Conceptualization, C.-P.Y. and I.-C.C. ; formal analysis, C.-P.Y. and I.-C.C. ; methodology, I.-C.C. ; resources, C.-E.Y. ; supervision, I.-C.C. ; writing--original draft, C.-P.Y. ; writing--review and editing, C.-E.Y. and I.-C.C. All authors have read and agreed to the published version of the manuscript. This research was funded by the Ministry of Science and Technology, Taiwan, grant number MOST 110-2221-E-018-002. The APC was funded by MOST 110-2221-E-018-002. Not applicable. The study did not report any data. The authors declare no conflict of interest. ## References

* [PERSON] et al. (2017) [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON] Vehicle-to-Everything (v2x) Services Supported by LTE-Based Systems and 5G. _IEEE Commun. Stand. Mag._**2017**, \(1\), 70-76. [CrossRef]
* [PERSON] et al. (2019) [PERSON]; [PERSON]; [PERSON] Novel Self-Adaptive Routing Service Algorithm for Application in VANET. _Appl. Intell._**2019**, _49_, 1866-1879. [CrossRef]
* [PERSON] et al. (2013) [PERSON]; [PERSON]; [PERSON] VANET: Challenges, Issues and Solutions.