# Coexistence Scheme for Uncoordinated LTE and WiFi Networks Using Experience Replay Based Q-Learning
## 6 Performance Evaluation
### Fairness and Throughput Performance
For example, the probability of getting an aggregated throughput of 40 Mbps or less is 46.4% and 35.6% for the case or rule-based and Q-learning-based coexistence schemes solutions. However, the probability of getting this range of aggregated throughput is only 24.8% for the case of ER Q-learning and REQ-learning-based solutions. In general, the graph shows that in terms of aggregated throughput, the ER Q-learning and REQ-learning-based solutions outperform the other proposed alternative solutions. Figure 4(a) shows the average fairness values for the rule-based, traditional Q-learning-based, ER Q-learning-based, and RER Q-learning-based solutions. The bar graph shows that the ER Q-learning-based and RER Q-learning-based coexistence schemes lead to a better coexistence in terms of the obtained average fairness. The average fairness is computed by taking an average of the fairness values obtained in each fixed load duration in the entire simulation period. The fairness value in each fixed traffic load duration is computed using Equation (3). Figure 4(b) shows a histogram of the fairness values obtained in each fixed traffic load duration \(T_{D}\) during the simulation period. From the histogram, it can be observed that the ER Q-learning and RER Q-learning-based solutions perform better than the other coexistence solutions in terms of fairness between the technologies. Figure 4: Comparison of aggregated throughput for the proposed solutions (**a**) average aggregated throughput (**b**) CDF of aggregated throughput. For the fairness performance evaluation, each LTE UE or WiFi station generates random data rates, which are randomly picked between {50 kbps, 500 kbps, 1 Mbps, 2 Mbps, 4 Mbps}. Likewise the throughput performance evaluation, each randomly generated data rate remains constant for \(T_{D}\) seconds where \(T_{D}\) is randomly picked from 5-15 s. This is done for a simulation time of 300 s and it is repeated for 100 iterations. The coexistence solutions are used to select the best actions while the offered traffic changes dynamically. The fairness performance of an action selected based on the optimal policy is evaluated by computing its fairness using Equation (5). For validation purposes, we also determine the rank of the each selected action (interms of fairness) as compared to the other possible actions. For each fixed load traffic of LTE and WiFi networks, the selected action, the obtained and target throughput, \(T_{D}\), and the fairness value are stored for comparison purposes. Following this, all the other possible actions are also executed sequentially by the proposed algorithm. Figure 5: Comparison of fairness for the proposed solutions (**a**) average fairness (**b**) histogram of fairness values obtained in each fixed load duration in the entire simulation period. manual selection and the fairness value is stored for every action. This is repeated for every fixed traffic generated during all the iterations of the simulation period. Finally, the rank of the action selected by the policy is determined based on its fairness as compared to the other possible actions which are selected manually. Table 4 shows the distribution of rank of actions taken based on the traditional Q-learning, ER Q-learning, and REQ-learning schemes in 300 s simulation with dynamic traffic. The table only presents the comparison of the reinforcement learning based solutions as the rule-based solution always leads to the best action despite the fact that it requires multiple trials until it reaches the best action. The table shows that the traditional optimal policy of the Q-learning based solution can only get the best action at a probability of 66.7%. This low performance occurs due to the dynamic nature of the environment. The ER Q-learning and REQ-learning based solutions on the other hand have better performance leading to the best action at a probability of 86.3% and 89.4%, respectively.