# Coexistence Scheme for Uncoordinated LTE and WiFi Networks Using Experience Replay Based Q-Learning
## 2 Related Work
### Coexistence in Uncoordinated LTE and WiFi Networks  
In this section, we discuss coexistence mechanisms that consider uncoordinated LTE and WiFi networks. In uncoordinated LTE and WiFi networks, there is no cooperative channel used to exchange the traffic status between the technologies. Hence, additional features that enable to sense the WiFi traffic have to be included in the co-located technologies. In [23; 24], number of WiFi APs are determined to estimate the WiFi traffic. The work in [23] proposes a method for discriminating between one and two WiFi APs by employing an auto-correlation function on the WiFi preamble and setting appropriate detection thresholds to estimate the number of active WiFi APs. Similarly, an energy detection-based approach is used to distinguish between one and two WiFi APs in [24]. The work in [25] offers an ML-based strategy for determining numerous WiFi APs using energy levels measured. This work also proposes an LTE-U/WiFi coexistence scheme that utilizes the number of APs detected by the system. Generally, the proposed approach of determining the number of co-located WiFi APs is a less complex approach to get a rough estimate of the WiFi traffic as compared to installing a full WiFi receiver at the LTE eNB side to decode the WiFi packets. The problem in using the number of active WiFi APs to estimate Wifi traffic is the fact that each active WiFi AP can have a varied traffic load. Hence, counting the number of active APs can lead to wrong coexistence decisions. In our previous work [26], the CNN model that can identify the duration of each transmitted frame from each co-located technology was proposed and validated. The validation was carried out utilizing commercially available LTE and WiFi hardware. Similarly, CNN based models are used to perform identification of WiFi transmissions from other co-located transmissions of other technologies in [27; 28]. Our CNN based technology classification proposed in [26] is used to implement coexistence schemes between private LTE and WiFi in [29; 30]. These coexistence schemes use Channel Occupancy Time (COT) to estimate the WiFi traffic and select optimal transmission time of LTE and WiFi. However, COT is not a good indicator of the WiFi traffic load as it depends on the packet size and the number of active users [8]. Hence, COT based coexistence decisions are not efficient in variable packet size and active nodes. The authors in [31] propose a coexistence solution that uses reinforcement learning to estimate WiFi traffic demand. In this work, the LTE eNB uses reinforcement learning to learn and predict future WiFi traffic demands. Similarly, the authors in [32] offer a Q-learning-based methodology to estimate WiFi traffic characteristics. The suggested solution provides a decision-making framework that uses carrier detection at the LTE eNB to determine WiFi idle time. The primary goal of this effort is to maximize unlicensed LTE exploitation of idle spectrum resources. In general, these systems estimate WiFi traffic demand by continuously monitoring the average number of total idle slots, the average number of successfully sent WiFi packets, and the average number of collisions, all of which can indirectly indicate WiFi traffic demands. However, the proposed solutions only consider unsaturated WiFi networks. In practice, the co-located WiFi network can be either saturated or unsaturated and the metrics found in the DRL-based WiFi traffic estimation, such as the number of successfully transmitted WiFi packets, differ in saturated and unsaturated WiFi traffic [33]. In [22], reinforcement learning techniques are used to tune the contention window size for both LTE-LAA and WiFi nodes. This coexistence solution is constructed for uncoordinated LTE-LAA and WiFi networks. Reinforcement learning based solutions are implemented in both LTE and WiFi nodes.