# Q-RPL: Q-Learning-Based Routing Protocol for Advanced Metering Infrastructure in Smart Grids
## 2 Related Work
The author of [28] proposed the RMP-RPL algorithm with a view to improving RPL reliability for critical applications. This paper proposes a new ranking calculation and parent selection method for RPL. In RMP-RPL, each node forwards its data packet to \(n\) number of ranked nodes based on three routing metrics: node mobility, alternative parent connectivity, and ETX. Despite the good results shown in this work in terms of PDR, sending packets to multiple nodes simultaneously can significantly increase the overall network traffic, and lead to severe congestion, especially in networks with limited bandwidth or a high number of nodes. This is a concern since the proposed method was not tested for different traffic loads. The use of fuzzy logic to improve RPL has been considered by some authors in previous studies such as [29; 30; 31]. The authors of the most recent work [31] introduce a modified version of RPL named FL-HELR-OF. They propose a cross-layer architecture and a fuzzy logic system that integrates four input metrics: hop count, energy consumption, latency, and RSSI. The best route to reach DODAG's root is chosen based on the output of the fuzzy system, while the new protocol outperforms the standard RPL implementations for different network sizes that range from 10 to 100 nodes, the use of logic fuzzy encloses complex configuration and tuning of membership functions and rule sets. The problem of routing overhead in RPL is addressed in [32]. The authors propose an adaptive routing algorithm, named Tabu-RPL, that dynamically adjusts data dissemination paths based on network conditions and device capabilities. The core of the proposal is the integration into RPL of the Tabu Search algorithm, which is a metaheuristic search method used to solve mathematical optimization problems. Tabu-RPL achieves a 30% reduction in network overhead according to the results presented in the paper. It is worth mentioning that the design basically focuses on optimizing the parent selection considering metrics such as ETX and residual energy, however, the paper does not clearly delineate how this approach translates into reduced network overhead. Additionally, the design suggests that the algorithm considers only dynamic adjustments during the search process phase since there is no information about how the algorithm continues to look for better neighbors after the stopping condition is reached. This aspect can potentially affect the adaptability of the protocol in more changing environments. A further study [33] addresses the issue of energy efficiency in RPL. The authors propose a modification to the traditional trickle timer algorithm, introducing an enhanced version named EE-trickle. The primary aim is to reduce energy consumption and improve the PDR of the network. The standard trickle algorithm, which controls the timing and dissemination of control messages in RPL can lead to high energy consumption due to extended listening periods and frequent transmissions. The EE-trickle algorithm addresses this by optimizing how listening and transmission intervals are managed, thereby reducing unnecessary energy expenditure. The performance evaluations demonstrate that EE-trickle significantly lowers energy usage per node and better PDR is achieved in the network compared to when the standard approach is used, while the results are encouraging in terms of power consumption and PDR, solely managing control messages in this manner may overlook other critical factors such as link quality variability that can impact network reliability and efficiency. A recent trend in improving RPL has been to use Machine Learning to enhance routing decisions. For example, in [34], RPL+ introduces a refined parent selection strategy to choose the best forwarding node when two or more candidates have the same ranking in the RPL destination-oriented routing tree. Central to RPL+ is the utilization of a Random Forest (RF) algorithm [35] for analyzing the significance of different routing metrics.