## 3 SD-QGrid Routing Platform and Algorithms  
### SD-QGrid Network Architecture  
First, we define the grid in the SD-QGrid network architecture. As shown in Figure 4, we calculate the length and width of the map range of this experiment; we know the latitude and longitude of the four vertices of the map; we take the vertex P in the upper left corner as the reference point; we calculate the distance between the point P and the vertex in the upper right corner to obtain the length L of the outgoing map; and we obtain the width W of the map by calculating the distance between the point P and the vertex in the lower left corner. After determining the length L and width W of the experimental map, we can then give the grid length d to mark m \(\times\) n grids in this map. The grid number range is (0... m \(-\) 1, 0... n \(-\) 1).  
\begin{table}
\begin{tabular}{p{42.7 pt} p{42.7 pt} p{42.7 pt} p{42.7 pt} p{42.7 pt} p{42.7 pt} p{42.7 pt} p{42.7 pt} p{42.7 pt}} \hline \hline
**Protocol** & **Analyzes** & **Considers** & **Considers** & **Considers** & **Uses Rein-** & **Simulation** & **Considers** & **Designs** & **Software-** \\
**Proced** & **Real Vehicle** & **the Directionality of the Vehicle** & **the Directionality of the Vehicle** & **the Directionality of Vehicle** & **the Number of Vehicles** & **Learning for Routing** & **Using Real** & **Next-Grid** & **Defined Routing** \\
**Data** & **Data** & **Trajectory** & **Des Destination** & **in Each Area** & **Doe Decisions** & **Trajectories** & **No** & **No** & **No** \\ \hline ITAR-FQ & No & No & No & Yes & Yes & No & No & No \\ ADOFEL & No & Yes & No & Yes & Yes & No & No & No \\ QTAR & No & No & No & Yes & Yes & No & No & No \\  & & & & & Yes (only one & & & \\ QGrid\_G & No & No & No & Yes & Q-table, four neighbor grids) & Yes & No & No \\ AdvQGrid & Yes & No & No & Yes & Q-table, four neighbor grids) & Yes & No & No \\  & & & & & Yes (different & & \\ SD-QGrid & Yes & Yes & Yes & Yes & to different time periods, eight neighbor grids) & Yes & Yes & Yes \\ \hline \hline \end{tabular}
\end{table}
Table 1: Comparison of related studies.  
The entire SD-QGrid has an SDN Control Node (CN), which is responsible for initializing the grid structure of the entire map by defining the grid size, cutting the experimental map, and calculating the grid where all vehicle nodes are located. In addition, each grid needs to be equipped with an RSU, which can communicate with the CN through wired lines, at each intersection to collect vehicle trajectory information of all vehicles located at road segments connected to this intersection. Each vehicle on a road segment assumes to own the communication module and the GPS receiver such that it can periodically broadcast its instant information, including the vehicle ID, velocity, GPS position, moving direction, etc., to the first reached RSU through the HELLO messages. In this way, the loss of vehicle information caused by building obstacles in the grid can be reduced [29].  
The SD-QGrid is proposed to contain two operation processes. The first one is the offline Q-learning training process and the second one is the online routing decision process. Details of these two processes are explained below.  
1. The offline Q-learning training process: Using the vehicle trajectory and grid-related information, the offline Q-learning training process is responsible for training the Q-learning model to generate Q-tables of each grid at the SDN CN when the whole SD-QGrid system initializes or a new vehicle enters this V2X environment. The SD-QGrid may periodically perform this process to update Q-tables.
2. The online routing decision process: Whenever a vehicle node carrying a packet intends to transmit this packet to the destination node, the SD-QGrid executes the online routing decision process to select the best neighbor node as the relay. Consequently, the packet will efficiently reach its destination at the end of the routing process.