# A Multiarmed Bandit Approach for LTE-U/Wi-Fi Coexistence in a Multicell Scenario
## 2 Related Works
However, the rest of this section explores the scope of this paper, which includes the use of LTE-U and two Reinforcement Learning (RL) algorithms: QL and MAB. The authors in [8] propose a framework to maximize the system transmission rate for a controlled interference scenario defined by 3 GPP, using the most widespread RL algorithm, Q-Learning. This framework considers the offered data rate to each user and maximizes the system throughput by dynamically choosing the DC parameter. The authors of [26] use QL in a multichannel scenario, with the algorithm dynamically selecting the least populated channel and the DC parameter to improve system performance. In [27], the authors propose a QL algorithm that takes into account the LTE and Wi-Fi queue-length processes as a metric to reflect traffic load. The papers in [28; 29] also employ dynamic DC adjustments with QL similar to the objective of fairness in coexistence, with differences only in the parameters and metrics used. Additionally, recent studies have also explored the use of Deep Q-Learning (Deep QL) as a means to enhance throughput and promote fairness in coexistence scenarios, such as [30; 31]. However, modeling QL algorithms requires the definition of several state-action pairs in the target environment. As the number of pairs increases, the algorithm's complexity also increases, limiting the solution's flexibility. Thus, modeling a complex and very dynamic environment, such as multicellular wireless scenarios, can be a problem that makes Q-Learning unfeasible as a solution if the requirements for convergence time are too strict. An alternative is to replace Q-Learning with MAB. One of the first papers to use the MAB theorem to solve wireless communications problems was [32]. The authors investigate solutions for cognitive radio networks in highly dynamic environments, in which the user seeks to take advantage of available frequency bands in a radio spectrum with multiple bands. The authors model the availability of each channel as a Markov chain, and two scenarios are proposed to measure performance: with and without knowledge of the Markov chain. The first models the problem as a competitive multiuser bandit, while the second uses an optimal symmetric strategy to maximize the total user throughput. In [33; 34], the authors propose an MAB for the coexistence of LTE-U and Wi-Fi in the Citizens Broadband Radio Service (CBRS) band at 3.5 GHz. The algorithm provides significant gain by tuning power control and the DC to reduce interference. An MAB algorithm is proposed in [35] for the coexistence of Machine to Machine (M2M) systems with Wi-Fi. The authors model the MAB algorithm to take advantage of white spaces in Wi-Fi transmission, aiming at improving the transmission rate of M2M devices. In this paper, we present a flexible strategy for adaptive DC selection in a multicellular environment using MAB. Our key contributions are as follows:  
* A simple, adaptive DC selection strategy with better performance and enhanced fairness compared with the Q-Learning-based strategy proposed in [10];
* A flexible solution for the coexistence that does not require the definition of state-action pairs, allowing a higher number of available DC values to choose from;
* An adaptive solution that combines low complexity and lower number of parameters to configure;
* We show that in environments where the rate changes dynamically, the sequential decision-making approach can result in a base model that is applicable for a wide range of reinforcement learning algorithms, whether they are simple or complex;
* A key comparison of reinforcement learning strategies applied to the adaptive multicell data rate environment for coexistence. To the best of our knowledge, this kind of comparison has not yet been done using the approach presented in this work.