# Q-Learning Based Fair and Efficient Coexistence of LTE in Unlicensed Band
## 1 Introduction  
An exponential increase in demand for wireless multimedia data and the limited nature of a licensed spectrum for cellular networks have inspired the use of unlicensed bands for long term evolution (LTE) in internet of things (IoT). In an unlicensed spectrum of below 6 GHz, a large part of spectrum, approximately 600 MHz [1], is globally available for various purposes. LTE uses this unlicensed band to offload multimedia traffic through either downlink-only or both downlink and uplink approaches in IoT systems. Although, the use of LTE in an unlicensed band enhances the capacity and achieves a seamless user experience, a few issues in allowing different networks to operate in a mutually shared spectrum need to be considered. One important issue is the coordination and management of an interference among the different coexisting technologies [2; 3]. A Wi-Fi system utilizes the carrier sense multiple access (CSMA) protocol to coexist harmoniously with various other Wi-Fi systems, whereas LTE uses continuous traffic generation with the smallest time gaps even in the absence of data traffic. Considering these operating characteristics in both systems, Wi-Fi seems to have a minimal opportunity to use the channel compared with LTE under a coexistence scenario, resulting in a performance degradation for Wi-Fi [4]. In the literature, some solutions such as license assisted access (LAA) [5] and LTE-U [6] have been proposed for IoT systems. LAA is based on the cellular industry's standard body, the Third-Generation Partnership Project (3 GPP), and its coexistence processes follows the same path used in Wi-Fi, i.e., listen before talk (LBT). By contrast, LTE-U isbuilt on proprietary technology established by the LTE-U forum, which takes a completely different approach. A small group of companies such as Ericsson, Qualcomm, Verizon, and Samsung have been developing LTE-U through a closed process. LTE-U applies a carrier ON/OFF switch which duty cycles the LTE transmissions. It switches to ON mode to transmit the signal for a certain amount of time and then it switches to OFF mode so that Wi-Fi can access the channel. Most of the proposed fair coexistence mechanisms for LTE-U largely suffer from a loss of spectrum resource efficiency, while maintaining the airtime and throughput fairness with Wi-Fi [7; 8; 9; 10]. The experiment conducted by CableLabs [7], shown in Figure 1, which demonstrated the fairness conceived by LTE-U over a proportionate airtime, has a disproportionately negative impact on the Wi-Fi performance, i.e., a 50% duty cycle of LTE causes a Wi-Fi performance decrease of 70%. As a result, LTE-U can be ON for only 35% of the cycle to maintain 50% Wi-Fi throughput. Similar to the authors in [8], our initial simulation results demonstrate that a change in duty cycle affects both LTE-U and Wi-Fi under a coexistence scenario. We can see that, to achieve a fair coexistence, the maximum achievable throughput of the network is significantly reduced (18%) and vice versa, as shown in Figure 2. This occurs because of the unbalanced physical and link layer parameters between these two technologies. In the case of Wi-Fi, most of the channel resources are wasted during contention, thereby requiring significant radio resources to maintain equal throughput as compared to LTE-U. Because both technologies have an equal right to use the unlicensed band, Wi-Fi can be regarded as the least efficient technology compared to the schedule-based LTE-U technology in terms of spectrum utilization [11; 12]. Hence, enabling a fairness measure among Wi-Fi and LTE-U may lead to an underutilization of the wireless spectrum resources compared to the exploitation of the entire unlicensed band. Similarly, in the context of increasing the network efficiency, the studies described in [13; 14; 15; 16; 17] have contributed to an increase in network efficiency by simply scarifying the fairness measure under a coexistence scenario.