# A Multiarmed Bandit Approach for LTE-U/Wi-Fi Coexistence in a Multicell Scenario
###### Abstract  
Recent studies and literature reviews have shown promising results for 3 GPP system solutions in unlicensed bands when coexisting with Wi-Fi, either by using the duty cycle (DC) approach or licensed-assisted access (LAA). However, it is widely known that general performance in these coexistence scenarios is dependent on traffic and how the duty cycle is adjusted. Most DC solutions configure their parameters statically, which can result in performance losses when the scenario experiences changes on the offered data. In our previous works, we demonstrated that reinforcement learning (RL) techniques can be used to adjust DC parameters. We showed that a Q-learning (QL) solution that adapts the LTE DC ratio to the transmitted data rate can maximize the Wi-Fi/LTE-Unlicensed (LTE-U) aggregated throughput. In this paper, we extend our previous solution by implementing a simpler and more efficient algorithm based on multiarmed bandit (MAB) theory. We evaluate its performance and compare it with the previous one in different traffic scenarios. The results demonstrate that our new solution offers improved balance in throughput, providing similar results for LTE and Wi-Fi, while still showing a substantial system gain. Moreover, in one of the scenarios, our solution outperforms the previous approach by 6% in system throughput. In terms of user throughput, it achieves more than 100% gain for the users at the 10 th percentile of performance, while the old solution only achieves a 10% gain. W1-Fi; LTE-U; coexistence; reinforcement learning; multiarmed bandit; Q-learning 2023  
2023/12/2023/  
that in mind, mobile operators and standard organizations proposed the use of unlicensed spectra, since unlicensed bands are free from usage costs in terms of purchasing a license of operation. However, implementing solutions tailored to unlicensed bands requires careful attention to ensure compliance with the regulations of each target region. In order to operate in unlicensed spectrum, certain areas require medium access mechanisms, such as the Carrier Sensing Multiple Access with Collision Avoidance (CSMA/CA), while other regions only need the implementation of a particular mechanism that limits the time a technology can take to control the transmitting medium [2]. The 3 GPP proposed three solutions to address these two requirements: (i) LTE-Licensed Assisted Access (LTE-LAA) [3]; (ii) the LTE-Unlicensed (LTE-U) [4], also called LTE-Duty Cycle (LTE-DC); and, (iii) New Radio Unlicensed (NR-U), with either LTE dual-connectivity mode or stand-alone operation [5]. In our previous works [6; 7; 8; 9; 10], we have demonstrated how the first two solutions work and how they interact with Wi-Fi, which is currently the most successful technology utilizing the unlicensed spectrum. We also proposed a centralized Q-Learning (QL) solution to increase system throughput in a coexistence multicell scenario with LTE-U and Wi-Fi [10]. The solution provided gain in terms of system and user throughput, and we have identified further improvements for this upcoming work. In this paper, we extend our previous work by implementing a simpler yet efficient algorithm that achieves higher gain than our previous QL solution and offers improved balance by providing similar throughput for both Wi-Fi and Long Term Evolution (LTE). Our new solution applies an Multi-Armed Bandit (MAB) approach to provide a coexisting framework between Wi-Fi and LTE-U within a multicell environment. We compare the two solutions and demonstrate their performance in each case, indicating which one performs better or worse. We also compare their implementations, highlighting the advantages of our new solution. This paper is structured as follows. Section 2 presents the related works, followed by Section 3, which presents the evaluation scenario and preliminary results. The problems involved with coexistence in unlicensed bands are introduced in Section 4, and Section 5 presents our proposed MAB solutions.