# Q-RPL: Q-Learning-Based Routing Protocol for Advanced Metering Infrastructure in Smart Grids
## 2 Related Work
\begin{table}
\begin{tabular}{p{42.7 pt} p{56.9 pt} p{56.9 pt} p{56.9 pt} p{56.9 pt} p{56.9 pt}} \hline
**Work** & **Key Contribution** & **Methodology** & **Performance Metrics** & **Limitations** & **Main Results** \\ \hline \multirow{6}{*}{[27]} & WRF-RPL integrates energy and parent counts for balance routing, enhancing network life and PDR. & Applies a weighted random forward next-hop selection. & Evaluates network life, PDR, control overhead, and energy under high traffic. & Lacks link quality metrics, risking suboptimal paths; parent count may not reflect actual load. & Increases network life and PDR, risks uneven load distribution and inefficiencies. \\ \hline \multirow{6}{*}{[28]} & RMP-RPL uses node mobility, connectivity, and ETX for reliable multi-parent selection. & Dynamic multi-path selection based on mobility, connectivity, and ETX. & Evaluates PDR, end-to-end delay, and overhead, showing inductive improvements. & May increase congestion due to multiple simultaneous transmissions and is untested across variable traffic loads. & Enhances PDR and reduces delays, suitable for mission-critical applications with scalability concerns. \\ \hline \multirow{6}{*}{[31]} & FL-HELR-OF uses fuzzy logic in a cross-layer architecture to dynamically select parents using hop count, energy, latency, and RSSI. & Fuzzy logic integrates multiple metrics into a cohesive routing decision framework. & Assesses PDR, latency, energy, overhead, and hop count, showing major improvements. & Requires complex setup and tuning, with potential scalability challenges. & Exceeds standard RPL in performance, improving reliability and efficiency in diverse IoT setups. \\ \hline \multirow{6}{*}{[34]} & RPL+ uses Random Forest to refine parent selection by analyzing and weights routing. & Employs Random Forest for analyzing ETX, MAC losses, channel utilization, and throughput to optimize routing. & Evaluates PDR and end-to-end delay, ending improvements in network changes. & Static weights in decision-making may limit adaptability to network changes. & Shows significant PDR improvements, enhancing reliability and efficiency compared to standard RPL. \\ \hline \multirow{6}{*}{[18]} & ML-RPL uses CatBoost GBDT to enhance routing decisions via comprehensive metrics. & Model trained on smart meter data deployments, and predicts optimal routes based on probability. & Improvements in PDR and end-to-end delay, ending improvements in delay demonstrate enhanced network efficiency. & Depends on training data quality, affecting adaptability in new scenarios. & Outperforms standard RPL and RPL+, especially in dynamic conditions. \\ \hline \multirow{6}{*}{[19]} & GNB-RPL employs Gaussian Naive Bayes to optimize routing in smart grids for enhanced scalability. & Applies Gaussian Naive Bayes based on smart meter data from Montreal for routing decisions. & Shows marked improvements in packet delivery and reduced delays across varied loads. & Performance variability due to Naive Bayes’ and simplistic assumptions about feature independence. & Enhances scalability and reduces training data needs, suitable for dynamic, large networks. \\ \hline \multirow{6}{*}{[20]} & Analyzes supervised ML methods in RPL, focusing on scenario-dependent performance. & Assesses Catboost and Naive Bayes in varied scenarios for routing effectiveness. & Uses AUC to evaluate ML model predictions for routing success. & Effectiveness decreases in scenarios different to the training environment; requires frequent retraining. & Recommends exploring RL for adaptable, scenario-independent routing in dynamic settings. \\ \hline \multirow{6}{*}{[39]} & Employs Q-learning in an RL-based protocol to optimize parent node selection in WSNs. & Uses Q-learning to dynamically choose the best parent based on network data and performance metrics. & Shows improvements in PDR, delay, and energy consumption over linear methods. & Needs more detail on reward function and concerns about scalability with periodic messages. & Surpasses traditional linear weighted methods, enhancing responsiveness and efficiency in dynamic WSNs. \\ \hline \end{tabular}
\end{table}