## 2 Literature Review
### Review of Similar Works
\begin{table}
\begin{tabular}{p{113.8 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline \hline
**Routing Protocol** & **Objective** & **RL Technique** & **Control Technique** & **Drawback** \\ \hline \hline Q-Routing [19] & Learns the optimal paths to minimizes the packet delivery delay. & Q-learning & Distributed & i. Requires Q-value freshness. ii. Sensitivity to parameter setting. iii. Slow convergence to optimal routing paths. \\ \hline RL-based constrained flooding [20] & Optimizes the cost of constrained flooding (delivery delay, hop count). & Q-learning & Distributed & Degradation in packet delivery delay when compared with direct routing. \\ \hline AdaR [24] & Maximizes network lifetime taking into consideration & & & i. No explicit definition of the network lifetime. ii. High computation complexity. \\ \hline RL-based routing [25] & Optimizes the energy, link reliability, and the number of paths crossing a node. & & & \\ \hline Energy-aware selfishness RL-based consumption. [25] & Minimizes the energy consumption. & Q-learning & Distributed & The selfishness and energy functions were not provided. \\ \hline RLGR [26] & Improved the network lifetime by learning the optimal routing paths with factors such as hop count and node residual energy. & Q-learning & Distributed & Slow convergence to the optimal routing paths. \\ \hline Q-PR [28] & Maintains the trade-off between network lifetime and retransmissions while increasing the packet delivery ratio. & & i. The message’s importance is not balanced with the energy cost of using a constant a discount factor of one. & i. The selection of the next forwarder requires the requisites of neighbors. iii. Non-refinement of the estimation of the residual energy of the sensor nodes. \\ \hline RL-based balancing energy routing [29] & Balancing the trade-off of minimizing energy consumption and maximizing the network lifetime by selecting routing paths based on the energy consumption of paths and residual energy of nodes. & & The network lifetime is the time when the first node depletes its energy source, however, sensing is still possible unless the node is the sink. \\ \hline E-FROMS [30] & Balancing the energy consumption in multiple sinks by learning the optimal routing tree that minimizes the energy-based reward. & & & The state space and action space overhead are high and very high respectively. \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Comparison of the related works. \begin{table}
\begin{tabular}{p{113.8 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline
**Routing Protocol** & **Objective** & **RL Technique** & **Control Technique** & **Drawback** \\ \hline RL-based routing interacting with WSN with moving vehicles [32] & Learn the routing paths between sensor nodes and moving sinks taking into consideration of hop count and energy signal strength to maximize the network lifetime. & Q-learning & Distributed to control packets. \\ \hline OPT-EQ-Routing [33] & Optimizes the network lifetime while minimizing the control overhead by balancing the routing load among the sensor nodes taking into consideration the sensor nodes’ current residual energies. & Optimizes the network minimizing the network while ensuring the packet delivery delay by learning the optimal routing path taking into consideration the residual energy of the next forwarder, the ratio of packets between the packet sender to the selected forwarder, and link delay. & Optimizes the network’s lifetime and packet delay by learning the routing paths using the node residual energy and hop counts to the sink in the reward function.