## 7 Discussion
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Scheme** & **RL Algorithm** & **Agent** & **State Space** & **Action Space** & **Reward Function** & **Learning Rate** & **Discount Factor** \\ \hline RSAR [65] & Q-learning & Each vehicle & The set of all nodes in the network & Transferring a beacon packet & 1, If the next hop vehicle is a single-hop neighbor of the destination; 0, Otherwise & Related to link head- reliability \\ \hline [PERSON] et al. [66] & Q-learning & Each vehicle & The set of all single-hop neighbors & Selecting the best new-hop node among single hop & 1, If the next hop vehicle is a single-hop neighbor of RSU; Otherwise & Fixed & Fixed \\ \hline RHR [67] & Q-learning & Unknown & Vehicles & Receiving different kinds of packets related to the current next hop & Unknown & Fixed & Fixed \\ \hline SeScR [68] & DDPG & Each vehicle & The location, velocity, and orientation of the vehicle & The decision for packet routing & Related to SDN controller & Fixed & Fixed \\ \hline IRQ [69] & Q-learning & Central server & All intersections in the network & Road sections connected to the current intersection & Related to the density of vehicles, connection time, and delay & Fixed and the distance to the destination \\ \hline QFHR [70] & Q-learning & Each packet & All intersections in the network & Four road sections connected to each intersection & Related to the density of vehicles, connection quality, the congestion level, the road traveling time & Fixed & Related to the road sections and the distance to the destination \\ \hline \hline \end{tabular}
\end{table} 
Table 8: _Cont._Additionally, Table 11 compares various routing methods based on their need to control messages. These control messages are exchanged between nodes to evaluate communication links and share spatial and velocity information of vehicles with other nodes in the network. However, control messages increase communication overhead, bandwidth consumption and other resources, network congestion, and delay in the routing process. Therefore, many researchers are trying to lower control messages exchanged between nodes. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Scheme** & **Positioning System** & **Digital Map** \\ \hline IV2 XQ [54] & ✓ & ✓ \\ \hline QAGR [55] & ✓ & x \\ \hline CEPF [56] & ✓ & ✓ \\ \hline PFQ-AODV [57] & x & x \\ \hline QGrid [58] & ✓ & x \\ \hline RRPV [59] & ✓ & ✓ \\ \hline RRFV [59] & ✓ & ✓ \\ \hline QTAR [60] & ✓ & x \\ \hline Q-LRR [61] & ✗ & x \\ \hline ECTS [62] & ✓ & ✓ \\ \hline RLRC [63] & ✓ & ✓ \\ \hline GLS [64] & ✓ & ✓ \\ \hline RSAR [65] & ✓ & x \\ \hline [PERSON] et al. [66] & ✓ & ✓ \\ \hline RHR [67] & ✓ & ✓ \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Comparison of various routing methods in terms of positioning systems. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Scheme** & **V2V** & **V2I** & **V2U** \\ \hline IV2 XQ [54] & ✓ & ✓ & x \\ \hline QAGR [55] & ✓ & x & ✓ \\ \hline CEPF [56] & ✓ & x & x \\ \hline PFQ-AODV [57] & ✓ & x & x \\ \hline QGrid [58] & ✓ & x & x \\ \hline RRPV [59] & ✓ & x & x \\ \hline QTAR [60] & ✓ & ✓ & x \\ \hline Q-LBR [61] & ✓ & x & ✓ \\ \hline ECTS [62] & ✓ & ✓ & x \\ \hline RLRC [63] & ✓ & x & x \\ \hline GLS [64] & ✓ & x & x \\ \hline RSAR [65] & ✓ & ✓ & x \\ \hline RHR [67] & ✓ & x & x \\ \hline ReScR [68] & ✓ & ✓ & x \\ \hline IRQ [69] & ✓ & ✓ & x \\ \hline QFHR [70] & ✓ & ✓ & x \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Comparison of the routing methods in terms of various communication types. on the network and control these broadcast messages. For example, IV2 XQ and QGrid methods do not use any control messages to design the Q-Learning-based routing model. These schemes only broadcast beacon messages between vehicles to find the best route in each road segment. As a result, these methods have a suitable bandwidth consumption, low delay, and low communication overhead. Moreover, RHR has presented an adaptive broadcast technique to manage bandwidth consumption.