# Q-RPL: Q-Learning-Based Routing Protocol for Advanced Metering Infrastructure in Smart Grids
## 5 Performance Evaluation
### Barcelona Scenario  
We extended our testing to an AMI deployment in Barcelona, Spain, to further validate the efficacy of our routing protocol in diverse real-world scenarios. Distinct from the Montreal scenario, the Barcelona deployment, as noted in Section 5.1, consists of 355 smart meters. Moreover, the distribution of the smart meters is more centered around the collector. Figure 8a displays the PDR in the Barcelona scenario for traffic load 1. Q-RPL is observed to outperform the other routing variants across the three applications considered. Most notably, Q-RPL achieves a 10% higher PDR than MRHOF in the MR application, and an 8% greater PDR compared to RPL+ and RI-RPL in the PQ application. Additionally, Q-RPL surpasses ML-RPL, its nearest competitor in terms of performance, by margins of 5%, 4%, and 5% for MR, AE, and PQ traffic, respectively. Figure 8: Performance metrics in the Barcelona scenario under traffic load 1. The packet delay for this experiment is shown in Figure 7(b). The figure clearly illustrates the superior performance of Q-RPL over the other routing protocols across all traffic applications in terms of median delay and consistency. Particularly notable is its performance in the MR traffic scenario, where it maintains a median delay of 286 ms, significantly lower than MRHOF; RI-RPL, RPL+, and ML-RPL. This trend is consistent across AE and PQ traffic, where Q-RPL again demonstrates the lowest median delays (177 ms and 192 ms, respectively) and minimal variability. RI-RPL, RPL+, and ML-RPL show improvements in median delay and consistency over MRHOF but still do not match the efficiency and reliability exhibited by Q-RPL. In terms of CF, which is depicted in Figure 7(c), we can see the same pattern as in the Montreal scenario. For MR traffic, all the routing protocols are close in performance, but for AE traffic the difference in favor of Q-RPL is more notable. In this case, Q-RPL's CF stands at 94%, outperforming RI-RPL, ML-RPL, RPL+, and MRHOF by margins of 7%, 9%, 10%, and 12%, respectively. For PQ traffic, both Q-RPL and ML-RPL lead the performance with a CF of 96%, indicating their superior ability over MRHOF, RPL+, and RI-RPL in meeting stringent time constraints in packet delivery. Following our approach in the Montreal scenario, we further assessed the robustness of our routing protocols in the Barcelona deployment under increased traffic conditions. As illustrated in Figure 8(a), the PDR in the Barcelona scenario with traffic load 2 again underscores the efficacy of Q-RPL. Q-RPL consistently outperforms the other routing variants across all application categories. For MR traffic, Q-RPL's advantage over MRHOF peaks at a significant 10%, as for traffic load 1, while it holds a steady lead of 4% to 5% over ML-RPL throughout all the applications categories. Compared to RPL+, Q-RPL achieves a consistent performance edge of 6-7%, and compared to RI-RPL, it shows an advantage of 6%, 4%, and 7% in the MR, AE, and PQ applications, respectively. Figure 9: Performance metrics in the Barcelona scenario under traffic load 2. The end-to-end delay in the Barcelona scenario with traffic load 2 is shown in Figure (b)b. It can be noticed from the figure that Q-RPL maintains a lower median delay, evident in its performance across the MR, AE, and PQ applications with median values of 246 ms, 199 ms, and 196 ms, respectively. In contrast, MRHOF shows a substantially higher median delay, particularly in the AE application, where it peaks at 2039 ms. RPL+, ML-RPL, and RI-RPL record intermediate median delay values, positioned between Q-RPL and MRHOF across all traffic applications. These findings not only highlight the efficiency of Q-RPL but also reinforce its reliability under more challenging traffic conditions. The enhanced performance in terms of end-to-end delay of our routing proposal is reflected in the CF metric, as depicted in Figure (c)c. Q-RPL exhibits a CF of 100%, 94%, and 98% for MR, AE, and PQ traffic, respectively.