# Coexistence Scheme for Uncoordinated LTE and WiFi Networks Using Experience Replay Based Q-Learning
## 2 Related Work
### Coexistence in Coordinated LTE and WiFi Networks  
In [10], [PERSON] et al. proposed a coexistence scheme that uses blank LTE subframes to give transmission opportunities to WiFi. Simulation results in this work show that the sequence and number of the blank subframes have a significant impact on the performance of coexistence solutions. In [11], a coexistence mechanism is proposed to guarantee a fair coexistence scheme between WiFi and LTE-U. The authors proposed a mechanism that is used to adjust the LTE duty-cycle time fraction based on the traffic status of a co-located WiFi and the available licensed spectrum resource of the LTE-U.  
In [12; 13; 14], Q-learning based CSAT mechanism that adapts LTE duty-cycle ON-OFF time ratio to the transmitted data rate is proposed. Their solution aims to maximize the WiFi and LTE-U aggregated throughput while maintaining fairness. Similarly, the authors in [15] propose a Q-learning based LTE-U and WiFi coexistence algorithm in multi-channel scenarios. By taking the idea of alternately transferring data in LTE-U and WiFi, the algorithm takes into account both the fairness and the performance of the system and optimizes the duty cycle.  
In our previous work [16], Q-learning based LTE-LAA and WiFi coexistence algorithms are proposed. Q-learning is used in these algorithms to implement an autonomous selection of optimum parameter combinations that can ensure fair and efficient coexistence between co-located LTE-LAA and WiFi networks. Similarly, the authors in [17; 18; 19] propose a machine learning based coexistence solutions to select optimal parameters that lead to the best coexistence performance.  
To ensure fair coexistence of LTE and WiFi, authors in [20] proposes a contention window (CW) size adaption algorithm-based channel access strategy. Similarly, the authors in [21] propose a mechanism for adaptively adjusting the back-off window size of WiFi and LTE duty-cycle time fraction based on the traffic status of a co-located WiFi and the available licensed spectrum resource of the LTE-U, while ensuring fair coexistence between the technologies. In [22], a reinforcement learning technique is implemented on the LTE-LAA to control its contention window size adaptively. This coexistence solution is constructed for coordinated LTE and WiFi networks. In other words, a cooperative learning method is constructed under the assumption that information across multiple systems can be exchanged.  
The studies are described in this section assume that co-located LTE eNB and WiFi Access Point (AP) can exchange the exact traffic requirements between each other. In practice, the two systems do not have a dedicated common control channel that may be utilized to share traffic status. As a result, the suggested coexistence schemes necessitate a system architectural change on the two technologies to create a new channel for traffic status reporting. In other words, modifications are required in both legacy LTE eNB and WiFi AP sides and these additional system requirements make implementing coexistence schemes difficult.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Notation** & **Description** & **Notation** & **Description** \\ \hline \(S\) & State space & \(a,s,r,s^{s}\) & Action, state, reward, new state \\ \hline \(A\) & Action space & \(T_{D}\) & Fixed traffic load duration \\ \hline \(f_{f}\) & Fairness factor & \(K\) & Number of replays \\ \hline \hline \end{tabular}
\end{table}
Table 1: _Cont._