# Q-RPL: Q-Learning-Based Routing Protocol for Advanced Metering Infrastructure in Smart Grids
## 3 RPL Parent Selection Background
\begin{table}
\begin{tabular}{p{42.7 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline \hline
**Work** & **Key Contribution** & **Methodology** & **Performance Metrics** & **Limitations** & **Main Results** \\ \hline \multirow{6}{*}{[40]} & RI-RPL employs Q-learning to optimize IoT routing by addressing dynamic conditions and instant negative impacts of path selection. & Utilizes Q-learning to adapt to network changes, improving parent selection with real-time data. & Measures success delivery ratio, latency, energy, throughput, and data loss, noting major improvements. & Uniform weight in the reward function could impact decision accuracy; stability may reduce responsiveness after convergence. & Enhances service delivery, reduces delays, and boosts may reduce responsiveness after convergence. & \\ \hline \multirow{6}{*}{[32]} & Tabu-RPL employs Tabu Search to dynamically optimize routing in IoT, reducing network overhead. & Uses metaheuristic search based on ETX and residual energy to adapt routing to network conditions. & Evaluates network overhead, energy, PDR, and delay, showing significant reductions and improvements. & Details on overhead reduction and adaptability in diverse networks are not fully explained. & Achieves a 30\% reduction in overhead, significantly enhancing energy efficiency and PDR. \\ \hline \multirow{6}{*}{[33]} & EE-trickle modifies the trickle timer to boost energy efficiency and RPL. & Optimizes listening and transmission energy use while maintaining performance. & Achieves notable reductions in energy per node and enhancements in PDR through simulations and testbeds. & May overlooked challenges in dynamic environments, especially in link quality management. & Demonstrated efficiency and better PDR compared to the standard trickle method. & Demonstrated efficiency and better PDR compared to the standard trickle method. & Demonstrated efficiency and better PDR compared to the standard trickle method. & PDR compared to the standard trickle standard trickle method. & PDR compared to the standard trickle standard trickle method. & \\ \hline \multirow{6}{*}{[41]} & QFS-RPL combines Q-learning and FSR to optimize routing for mobile nodes in RPL. & Applies Q-learning and FSR for dynamic network changes and mobile node management. & Boosts PDR, latency, throughput, and control overhead, while improving energy efficiency in mobile settings. & Perform similarly to standard RPL in static conditions and lacks extensive testing in dynamic environments. & Improves mobile performance by effectively managing mobility and load, but does not exceed standard RPL in static conditions. & Improves mobile performance by effectively managing mobility and load, but does not exceed standard RPL in static conditions. & Improves mobile performance by effectively managing mobility and load, but does not exceed standard RPL in static conditions. \\ \hline \multirow{6}{*}{[42]} & ACTOR employs a UCB-based RL strategy for dynamic power management in RPL, thoroughput in dense networks. & Uses UCB to dynamically adjust transmission power, optimizing power levels for better performance. & Significantly improves end-to-end delay, packet delivery, and energy efficiency, decisions, which may affect adaptability. & Depends mainly on ETX for routing speeding network topology with fewer parent switches. & Enhances throughput and energy efficiency, stabilizes network topology with fewer parent switches. \\ \hline \hline \end{tabular}
\end{table} 
Table 1: _Cont._commonly used in RPL are the Objective Function Zero (OF0) [22] and the Minimum Rank with Hysteresis Objective Function (MRHOF) [23]. OF0 primarily uses hop count as its metric. It selects the parent node that offers the shortest path to the root, simplifying the routing process but potentially overlooking other critical metrics like link quality or reliability. MRHOF is more sophisticated and can consider multiple metrics, primarily focusing on link quality and node stability.