# Coexistence Scheme for Uncoordinated LTE and WiFi Networks Using Experience Replay Based Q-Learning
## 6 Performance Evaluation
### Fairness and Throughput Performance  
In this section, we present the performance of the schemes presented in Section 5 in terms of the fairness and aggregated throughput obtained in our simulations. The aggregated throughput is used as a performance evaluation metric as it reflects the spectrum  
Figure 3: Convergence of Q-learning, ER Q-learning, and RER Q-learning based schemes. efficiency of the coexistence schemes. The corresponding LTE-U and WiFi throughput values, on the other hand, are used to assess the fairness of the coexistence schemes. During the performance evaluation of the coexistence solutions, each LTE UE or WiFi station generates random data rates, which are randomly picked between [50 kbps, 500 kbps, 1 Mbps, 2 Mbps, 4 Mbps]. The UDP data rate of UE/station is updated after a certain duration \(T_{D}\). The value of \(T_{D}\) is randomly picked from 5-15 s, leading to 20 to 60 possible fixed load cases in a total duration of 300 s simulation time. To achieve statistical regularity, this 300 s of simulation time is repeated for 100 iterations and, fairness and aggregated throughput are computed for each fixed load. The aggregated throughput in each fixed load duration is the sum of WiFi obtained throughput and LTE obtained throughput whereas the corresponding fairness value is given by:  
\[f_{\text{ev}}=1-\big{|}\frac{WiFi_{\text{Tar}}-WiFi_{\text{Ob}}}{LTE_{\text{ Ref}}}-\frac{LTE_{\text{Tar}}-LTE_{\text{Ob}}}{LTE_{\text{Ref}}}\big{|}. \tag{5}\]  
where \(LTE_{\text{Ref}}\) and \(WiFi_{\text{Ref}}\) are the reference throughput values which are the maximum possible aggregated throughput values that can be attained when the LTE and WiFi networks operate in standalone mode. \(WiFi_{\text{Tar}}\) is the target throughput of the WiFi network and it is equal to the offered data rate of the WiFi network as far as the offered data rate of the WiFi network does not exceed the \(WiFi_{\text{Ref}}\). On the other hand, the target throughput is set equal to \(WiFi_{\text{Ref}}\) if the offered data rate of the WiFi network is greater than the \(WiFi_{\text{Ref}}\). The target throughput of the LTE network (\(LTE_{\text{Tar}}\)) is also set in a similar way based on the offered data rate and reference throughput (\(LTE_{\text{Ref}}\)) of the LTE network. \(WiFi_{\text{Ob}}\) and \(LTE_{\text{Ob}}\) are the obtained throughput values of the WiFi and LTE networks, respectively. Equation (5) is used to evaluate the fairness of an action taken at a given state. This fairness value considers the obtained and target throughput values for the LTE and WiFi networks [14; 38]. The fairness evaluation equation is set in such a way that the fairness of a given action is high if the values of \(\frac{WiFi_{\text{Tar}}-WiFi_{\text{Ob}}}{LTE_{\text{Ref}}}\) and \(\frac{LTE_{\text{Tar}}-LTE_{\text{Ob}}}{LTE_{\text{Ref}}}\) are close. Otherwise, if these two values have a higher gap, it indicates that one network is getting higher spectrum share and hence the fairness of a given action decreases. In other words, the fairness of an action increases if both technologies are able to achieve close fraction \(X\), where \(X\) = (_target throughput \(-\) obtained throughput)/(reference throughput) for each technology._  
Figure 4a shows the aggregated throughput of rule-based, traditional Q-learning based, ER Q-learning based, and RE Q-learning based solutions. The bar graph shows that the ER Q-learning-based and REQ-learning-based coexistence schemes lead to a better coexistence in terms of the obtained aggregated throughput. On the other hand, the rule-based and traditional Q-learning-based solutions have relatively lower performance. Figure 4b shows the Cumulative Distribution Function (CDF) of aggregated throughput during the simulation. The CDF shows the distribution of aggregated throughput values obtained in each fixed traffic load duration \(T_{D}\) during the simulation period.