## 4 Performance Evaluation
### Energy Saving, Benchmarking and Metrics  
We evaluated the performance of the proposed QL-based cell switching algorithm in respect of live BS. The learning rate \(\lambda_{r}\) was set to 0.3 and the discount factor \(\phi\) was 0.9 [37]. The energy efficiency performance of the proposed QL assisted approach was compared to various cell switching approaches, namely _NS_, _Greedy_ and _ES_. In no-switching case, the SCs are always kept on, while the SCs are always switched off regardless of the available capacity of the MC in Greedy approach. ES, on the other hand, goes through all possible switching options to find the best option which reduces the total energy consumption of the network without exceeding the capacity of the MC. Figure 6 demonstrates the power consumption of all the approaches.  
As expected, the Greedy approach outperforms all other methods, as it does not consider the availability of the MC. Thus, its superiority in terms of power consumption comes at the expense of exceeding the MC capacity, which in turn decreases the QoS. The ES approach finds the best trade-off between power consumption reduction and the capacity of MC and as expected the proposed QL assisted approach converges to ES. In other words, the QL-assisted proposed method reduces the power consumption of the network without degrading the QoS. The ECRs of all approaches included in the HetNet are shown in Figure 7a, while Figure 7b presents the gains of NS, Greedy and ES approaches. As shown in Figures 6 and 7, the number of SCs is the most contributing factor for saving power and energy. As shown in Figure 7a, the total energy consumption of the network increases almost linearly with rising number of SCs. Therefore, it is expected that the ECR increases with the growth in number of components. Thus, maintaining CIOs are significant without dropping out capacity for QoS demands. The results in Figure 7b show that the energy saving increases with increasing number of SCs but only to some extent (when the number of SCs is 4). The reason is that the contribution of the SCs on the total energy consumption becomes more significant when their quantity increases, and thus the ECR improves by switching off the SCs. On the other hand, the energy consumption gain starts decreasing once the number of SCs exceed some certain quantity, which is 4 in our model. The reason behind this is the capacity of MC becomes insufficient to accommodate more users; hence, there is no more room to switch off additional SCs. In other words, the MC reaches its limit in terms of capacity, i.e. the number of SCs that can be switched off also reaches the limit. Consequently, the network cannot save more energy even with additional SCs, and the relative gain starts decreasing, as the total energy consumption of the network increases. Finally, CO\({}_{2}\) emissions are directly proportional to ECR, which keeps on increasing with the ratio being increased, as shown in Figure 8. On comparing all methods discussed in this proposal, the overall HetNet gain in terms of energy consumption coupled with CO\({}_{2}\) emissions between NS and Greedy approach is approximately 45.63%, between NS and ES is approximately 35.60% and between NS and QL is approximately 31.83%. For the proposed statistics methods, we can conclude that our robust framework can save considerable amount of energy and subsequent carbon emissions.  
Figure 6: Power consumption of various approaches when the number of SCs is 9. Episodes refer to the instances in the network.