## 2 Related Works
\begin{table}
\begin{tabular}{p{56.9 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline \hline
**Review Paper** & **Publication Year** & **Classification Year** & **Application** & **Network** \\ \hline \multirow{4}{*}{[13]} & \multirow{4}{*}{2021} & \multirow{4}{*}{Based on the routing algorithm, leading hybrid, zone-based, geographical, topology-based, hierarchical, secure, and DTN} & \multirow{4}{*}{Routing protocols} & \multirow{4}{*}{VANET} \\ \cline{1-1} \cline{5-5}  & & & & \\ \cline{1-1} \cline{5-5}  & & & & \\ \cline{1-1} \cline{5-5}  & & & & \\ \cline{1-1} \cline{5-5}  & & & & \\ \cline{1-1} \cline{5-5}  & & & & \\ \cline{1-1} \cline{5-5}  & & & & \\ \cline{1-1} \cline{5-5}  & & & & \\ \cline{1-1} \cline{5-5}  & & & & \\ \hline \multirow{4}{*}{[15]} & \multirow{4}{*}{2021} & \multirow{4}{*}{Based on three AI techniques, including machine learning methods, deep learning, and swarm intelligence} & \multirow{4}{*}{Routing, security, resource and access technologies, and mobility management} & \multirow{4}{*}{VANET} \\ \cline{1-1} \cline{5-5}  & & & & \\ \cline{1-1} \cline{5-5}  & & & & \\ \cline{1-1} \cline{5-5}  & & & & \\ \hline \multirow{4}{*}{[16]} & \multirow{4}{*}{2021} & \multirow{4}{*}{Based on seven applications} & \multirow{4}{*}{Routing, scheduling, resource allocation, dynamic spectrum access, energy, mobility, and edge caching} & IoT systems \\ \cline{1-1} \cline{5-5}  & & & & \\ \cline{1-1} \cline{5-5}  & & & & \\ \hline \multirow{4}{*}{[17]} & \multirow{4}{*}{2019} & \multirow{4}{*}{Resource allocation, caching and data offloading} & Streaming applications and mission-critical applications & \multirow{4}{*}{VANET} \\ \cline{1-1} \cline{5-5}  & & & & \\ \cline{1-1} \cline{5-5}  & & & & \\ \cline{1-1} \cline{5-5}  & & & & \\ \hline \multirow{4}{*}{[18]} & \multirow{4}{*}{2022} & \multirow{4}{*}{Based on the learning algorithm, the routing algorithm, and the data dissemination process} & \multirow{4}{*}{Routing protocols} & \multirow{4}{*}{FANET} \\ \cline{1-1} \cline{5-5}  & & & & \\ \cline{1-1} \cline{5-5}  & & & & \\ \cline{1-1} \cline{5-5}  & & & & \\ \hline \multirow{4}{*}{Our survey} & \multirow{4}{*}{2022} & \multirow{4}{*}{Based on learning framework, learning model, learning algorithm, learning process, and routing algorithm} & \multirow{4}{*}{Routing protocols} & \multirow{4}{*}{VANET} \\ \cline{1-1} \cline{5-5}  & & & & \\ \cline{1-1} \cline{5-5}  & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Some review articles related to RL applications in VANETs. In [17], authors have examined multi-agent reinforcement learning (MARL) techniques to solve various problems in VANET. In this paper, various research works are focused on resource allocation, caching and data offloading in VANET. Also, the authors have explained how to use MARL techniques in streaming applications and mission-critical applications. Finally, they have presented the challenges related to these systems in VANETs. However, this paper does not focus on the MARL applications for designing routing protocols in VANETs. In [18], the authors have investigated how to apply reinforcement learning (RL) to build routing approaches in flying ad hoc networks (FANETs). For this purpose, they explained these networks, their constraints, main components, especially drones, and applications in different fields and specified the routing challenges in these networks in detail. Finally, a classification of routing approaches was presented. It includes three main fields, namely learning algorithm, routing algorithm and data dissemination process. According to the presented classification, the latest RL-based routing approaches in FANET have been reviewed. Overall, our studies show that few review papers are presented in the field of RL applications for designing routing schemes in VANETs. Thus, we focus on the RI-based routing protocols in VANETs and review their learning structure.