## 4 Performance Analysis and Evaluation
### Convergence and Computational Complexity of the EESA-RLC Algorithm  
The algorithm converges to an optimal solution after adequate exploration and exploitation of state-actions pairs. The algorithm's convergence period is a function of some parameter values which include action selection strategy, discount factor and learning rate. We simulate the Q-learning and SARSA RL algorithms to examine the convergence of the algorithms over \(E_{psd}=5000\) episodes and evaluate their performance as shown in Figure 8. The result indicates that both the learning algorithms converged to the optimal solution but after different numbers of episodes. The SARSA learning algorithm converged to the optimal value after \(E_{psd}=3020\) episodes and achieved a maximum average expected cumulative reward value of \(r_{Av}^{t}=0.52\) which is much higher than that of the Q-learning algorithm. On the other hand, the Q-learning algorithm converged to an optimal solution at \(E_{psd}=2020\) which is much lower than SARSA and achieved a maximum expected cumulative reward value of \(r_{Av}^{t}=0.44\). This suggests that the Q-learning algorithm converges to optimal solution in a relatively short learning period because of its reliance on an action selection strategy rather than cluster exploration to update its estimated optimal policy. In contrast, cluster explorations while updating the Q-value slows the convergence of the SARSA algorithm due to the extension of its learning period, but this of course yields a better expected accumulative reward. It can be concluded that learning period has a significant effect on the convergence of the algorithms, increasing as the number of episodes increases which in turn decreases the learning rate and therefore, the algorithm converges slower. Figure 8: Expected cumulative rewards and algorithms convergence. Optimal clustering can be achieved through many techniques. While some techniques employ computationally infeasible approaches such as exhaustive search techniques which try all possible options for efficient clustering and select the best option among them, while others employ less computationally complex approaches based on heuristic techniques such as hierarchical, k-means, c-means and fuzzy clustering, _etc._ For example, the GWSA approach performs a high number of iterations to merge the nearest clusters in the nodes' proximity matrix till an optimal solution is achieved [17]. However, its overall computational complexity \(O(N^{2}\:logN)\) is extremely high and increases proportionally with increase in size of the number of cognitive radio sensor nodes which make it impractical to implement in a large scale network. Distributed Group Wise Spectrum Aware (DGWSA) offers relatively low complexity as shown in Figure 9[18]. This approach merges multiple neighboring cluster pairs at every iteration instead of using a proximity matrix and its complexity largely depends on the size of neighboring clusters instead of network size. Therefore, we compare the computational complexity of our algorithm with that of the GWSA and DGWSA algorithms by increasing the size of the cognitive radio sensor nodes as shown in Figure 9. The result shows that the GWSA algorithm offers high computational complexity, which increases with size of the network, while our algorithm and the DGWSA one achieve much lower complexities that are largely influenced by the number of neighboring clusters. We employed Q-learning to evaluate the performance of our algorithm in terms of learning and adaptability to a dynamic environment in achieving an optimal solution. Our choice for Q-learning is due to its faster convergece which requires a shorter learning period.