# Q-Learning Based Fair and Efficient Coexistence of LTE in Unlicensed Band
## 3 LTE-U Coexistence Mechanism
### Deployment Environment  
We consider the deployment scenario shown in Figure 3, in which LTE-U BS consists of multiple Wi-Fi access points (APs), which operate simultaneously in the unlicensed channel. LTE-U BS and each Wi-Fi AP consists of \(M\) number of LTE-U user equipment (UE) and \(N\) Wi-Fi stations (STAs) respectively, which are arbitrarily distributed within the coverage area of the cell. TDD LTE is taken into consideration, and it is assumed that LTE-U BS and UEs are synchronized with each other for the entire duration. In the 5-GHz band, a total of \(K\) unlicensed channels is accessible, with one AP and a set of \(N_{k}\) STAs active in each unlicensed channel \(k\). The LTE-U system communicates over the LTE air interface. The BS informs the UEs regarding the use of a channel for transmission over the licensed band. There are \(B\) resource blocks (RBs) accessible for transmission during each transmission time frame (TTF), where \(B\) is the bandwidth of an unlicensed channel. The total aggregated throughput served by LTE-U BS in channel \(k\) during a single TTF can thus be mathematically assessed as follows:  
\[R_{k}^{LTE-U}=\sum_{m_{k}=1}^{M_{k}}\frac{B}{M_{k}}\ *SINR_{m_{k}}*(1-\Theta_{k}) \tag{1}\]  
where \(M_{k}\) represents the total number of LTE-U UEs assisted by the LTE-U BS exploiting the supplemental downlink capacity in channel \(k\), \(SINR_{m_{k}}\) is the signal-to-noise and interference ratio perceived by the \(m\)th UE when downlink data are conveyed on the \(k\)th channel, and \(\Theta\) is the portion of time related with the idle periods imposed by the LTE-U strategy on the \(k\)th channel. In addition, \(SINR_{m}\) delivers the spectral efficiency, which relies on the propagation environments between the \(m\)th UE and the LTE-U BS and the interference produced by other cells using the \(k\)th channel.  
In a Wi-Fi network, all STAs contend for channel access by means of a carrier sensing scheme called a distributed coordination function (DCF) protocol. In a DCF, the system throughput of the Wi-Fi network is determined based on the number of contending Wi-Fi STAs. Let \(P_{k}^{tr}\) represent the probability of at least one transmission signal being present in a time slot, and \(P_{k}^{s}\) denote the probability of a successful transmission on a channel, which can be mathematically formulated as  
\[P_{k}^{tr}=1-(1-\tau_{k})^{N_{k}} \tag{2}\]  
\[P_{k}^{s}\ =\ \frac{N_{k}\tau_{k}(1-\tau_{k})^{N_{k}-1}}{P_{k}^{tr}} \tag{3}\]  
where \(\tau_{k}\) is the transmission probability for each STA in channel \(k\), and \(N^{k}\) is the number of competing Wi-Fi STAs in channel k. According to [PERSON] [29], the Wi-Fi network throughput \(S_{k}^{Wi/fi}\) can be formulated as  
\[S_{k}^{Wi/fi}(N_{k})=\frac{P_{k}^{tr}P_{k}^{s}E[P]}{\Big{(}1-P_{k}^{tr}\Big{)} T_{k}^{\alpha}+P_{k}^{tr}P_{k}^{s}T_{k}^{s}+P_{k}^{tr}\Big{(}1-P_{k}^{s} \Big{)}T_{k}^{\alpha}} \tag{4}\]  
where \(T_{k}^{s}\) is the average duration of a channel detected as busy owing to a successful transmission, \(T_{k}^{c}\) is the average duration of a channel detected as busy owing to a collision, \(E[P]\) is the average packet size, and \(T_{k}^{\alpha}\) is the empty time slot duration.  
In LTE-U networks, Wi-Fi has an opportunity to access the channel only when LTE-U is OFF. Thus, the throughput achieved by Wi-Fi in the shared spectrum when coexisting with LTE-U can be expressed as follows:  
\[R_{Wi-Fi}^{k}=\ S_{k}^{Wi/fi}(N_{k})\times\Theta_{k} \tag{5}\]