# Q-Learning Based Fair and Efficient Coexistence of LTE in Unlicensed Band
## 4 Performance Evaluation
This occurs because the LTE-U BS tries to explore many different states in search of achieving the highest reward. Figure 5: LTE-U BS with DC only method. (**a**) Channel selection vs number of steps; (**b**) Dutycycle selection vs number of steps. Figure 6: LTE-U BS with COT method. (**a**) Channel selection vs number of steps; (**b**) Dutycycle selection vs number of steps. As the learning continues, the LTE-U BS discovers the channel and duty-cycle pattern that can deliver the highest fairness and effective coexistence with a Wi-Fi network, increasing the rewards received. As indicated in Figure 8, after approximately 240 iterations, LTE-BS determines the configurations that can direct the system toward a fair and optimal coexistence. By contrast, the DC-only scheme converges must faster than the other schemes because it has only one state. The convergence is directly proportional to the available \(\ (no.of\ state)^{no.of\ actions}\). After the system has already learned any changes in the environment, the configuration requires much fewer steps (approximately less than 20 iterations) to converge. Figure 9 shows the fairness index achieved in the proposed network scenario with the DC-only, COT, and proposed schemes. The simulation results indicate that the proposed scheme attains the highest fairness index in a dynamic network environment when compared to the other two schemes. This is because the proposed scheme attempts to maintain its duty cycle close to the target value, offering a proportional duty cycle configuration according to the load of each coexisting network. However, the COT-based scheme attempts to maintain its target value toward the idle time of the operating channel. In the COT-based scheme, the LTE-U continuously suffers from both airtime and throughput unfairness as the number of users increases. The DC-based scheme achieves the highest fairness index in a constant environment because it only operates in the single best channel. Although  
Figure 8: Q-value convergence. Figure 7: LTE-U BS with proposed method. (**a**) Channel selection vs number of steps; (**b**) Dutycycle selection vs number of steps. the DC-only mechanism learns to adapt its duty cycle according to a changing environment, as shown in Figure 5b, the fairness of the network degrades because there is always the possibility of another channel delivering a higher fairness to the network. Figure 10 shows the network efficiency achieved in a network scenario when applying the DC-only, COT, and proposed schemes. The maximum network efficiency occurs under a scenario in which LTE-U uses a single dedicated channel among the available channels (one channel used by LTE-U and three other channels used by Wi-Fi) without having to share with Wi-Fi. The simulation results in Figure 10 indicate that the proposed scheme achieves the highest overall network performance compared to the COT-based and DC-only schemes. This is because our proposed scheme provides more airtime access opportunity to LTE-U than the COT-based approach (higher priority given to Wi-Fi), allowing it to operate in the most uncongested channel. Because the DC-only approach operates in the single best channel for both fairness and efficiency, it demonstrates the highest performance in a constant environment. However, with a change in environment, its performance is significantly reduced because it lacks a switching feature to operate on the most uncongested channel in the network.