# Q-Learning Based Fair and Efficient Coexistence of LTE in Unlicensed Band
## 2 Related Studies  
In recent years, extensive research has been conducted to assure fair coexistence between LTE-U and Wi-Fi networks. Some countries have prohibited a continuous signal transmission and bind limits on the maximal duration of a transmission burst in an unlicensed spectrum, and as a result, carrier sense adaptive transmission (CSAT) [18; 19] was introduced during the early deployment of LTE in the unlicensed band. CSAT permits an LTE-U network to share an unlicensed channel with a Wi-Fi network through time division multiplexing. To implement CSAT, the existing almost-blank subframes (ABS) framework (i.e., defined in 3 GPP Release 10 of LTE) was initially considered in [20]. The ABS reserves a group of LTE subframes, during which the macro UEs are partly muted (data, control, or reference symbols), allowing the UEs in pico-base stations (BSs) to be assisted with a lower interference [21]. An LTE silence period occurs during these gaps, allowing the Wi-Fi system to access the channel. Wi-Fi uses these gaps for transmission and must end its transmission whenever the communication is resumed by LTE-U. However, synchronization signals and control information are still present in ABS, which may influence the Wi-Fi transmissions and carrier sensing, as discussed in [22]. To overcome this issue, the LTE-U forum [2] adopted the carrier aggregation feature of MAC channel element activation and deactivation, which is compatible with the Rel. 10/11/12 LTE PHY/MACstandards. Many studies based on LTE-U have since been proposed. [PERSON] et al. [9] introduced the LTE time division duplex (TDD) configuration with different numbers of uplink and downlink frames for LTE-U. However, their configuration consists of numerous uplink slots, making it inefficient under a real scenario. Under such a real scenario, more downlink frames are expected than uplink frames in any network. [PERSON] and [PERSON] in [19] proposed a duty-cycle mechanism for LTE-U that selects the suitable probability to access the channel and transmission duration. This ensures proportional fairness among LTE-U and Wi-Fi nodes. [PERSON] et al. [21] showed that, without a proper ABS assignment, the throughput of a Wi-Fi network under the coexistence of an LTE-U network can be seriously degraded, and similar concerns were reported in [22]. Likewise, in their technical report, the 3 GPP Workgroup listed LBT [22; 23; 24; 25] for LTE LAA for nations such as Japan and European countries. The application of LBT potentially enhances the coexistence, such as with Wi-Fi, through a clear channel assessment. Some analysis and performance tests have been reported toward the fairness under such a scenario [26; 27]. However, a duty-cycle based approach still utilizes resources more tightly and with no modifications of the LTE standard [19].  
In the context of increased network efficiency, approaches have been proposed [13; 14; 15] to increase the LTE-U efficiency through the selection of most of the idle channels. Similarly, the authors in [28] proposed a multichannel coexistence approach to increase the spectrum efficiency through which LTE-U uses all available channels simultaneously. However, the realization of such an approach requires a vast change in the preexisting system hardware.