# Q-RPL: Q-Learning-Based Routing Protocol for Advanced Metering Infrastructure in Smart Grids
## 5 Performance Evaluation
### Simulation Settings  
The simulations are carried out using the discrete network simulator OMNeT++ [47]. OMNeT++ is one of the most widely used and powerful simulation tools for network modeling [48]. The simulator works under an Eclipse-based IDE, and it can run basically on all platforms where a C++ compiler is available (Linux, Mac OS/X, Windows). It is free for academia and open source, which allows the reuse and modification of its modules. In addition, several simulation frameworks have been created to extend the functionality of the simulator to specific areas. One of those frameworks is INET, which we have used along with OMNeT++ since it includes all the protocol layers that we need to create our simulation environment. In comparison to other network simulator alternatives like ns-3 [49], RPL implementations available in OMNeT++ are more mature and updated than those available in ns-3 [50; 51; 52]. Lastly, while in [53], there is a mention of an implementation of IEEE 802.15.4g/Wi-SUN for the ns-3 simulator, to the best of our knowledge, this implementation is not available. All the previous reasons make OMNeT++ a suitable simulator to ensure an accurate and comprehensive evaluation of our proposed solution.  
The first scenario in which our proposal is tested consists of a deployment of 200 smart meters and one collector in the city of Montreal, while the second scenario in the city of Barcelona has 355 smart meters and also one collector. This larger scenario allows us to test the scalability and efficiency of Q-RPL in a more demanding environment, where the increased number of smart meters and potential network congestion present additional challenges. The scenarios are depicted in Figure 4(a),b. The channel characteristics, physical-MAC layer, and learning parameters are shown in Table 3.  
The traffic load is varied in each scenario according to Table 4. The applications that we have considered are typical in Smart Grids: Meter Reading (MR), Alarm Events (AEs), and Power Quality (PQ). MR refers to the usage information that smart meters collect and must report periodically to utilities. AEs is the second application taken into consideration. Alarms can happen at any time and are sent randomly during the simulation time by a percentage of smart meters. AEs can include events such as measurement failure, system restart, system memory full, configuration errors, etc. The other application considered is PQ. Examples of PQ events include leading/lagging power measurements, imbalance in energy flow, voltage fluctuations, harmonics, and voltage sags and swells. For traffic load 1, MR is transmitted for each smart meter every hour, while AEs and PQ are transmitted by 25% of the smart meters in the scenario. In contrast, traffic load 2, while still including the same applications, shows a variation in the sending frequency for MR, which is now every 30 min. Moreover, the percentage of meters involved in transmitting alarm events and power quality data is increased to 50% under traffic load 2, as opposed to 25% in traffic load 1.  
Figure 5: Urban scenarios.