# Q-Learning Based Fair and Efficient Coexistence of LTE in Unlicensed Band
## 4 Performance Evaluation  
In this study, we developed an LTE-U system level simulation platform using MATLAB. The system simulation statistics were obtained by acquiring the mean values over random user positions. A set of unlicensed channels and APs were deployed in each channel with a variable number of STAs. The LTE-U BS is permitted to switch over unlicensed carriers and notify the LTE-U UE to complete the communication procedure in the corresponding channel through a licensed carrier. The other simulation parameters for LTE-U and Wi-Fi are set as shown in Table 1.  
For an evaluation, the proposed algorithm is compared with a duty-cycle based method and a COT-based channel selection approach. In the duty-cycle based approach, which is similar to those described in [8; 9], and [19], LTE-U BS selects the single best channel and applies the Q-learning approach for a dynamic duty cycle selection. Using the COT-based channel selection approach [16; 42], the LTE-U BS adjusts its ON time, according to the channel occupancy measurement of coexisting Wi-Fi users, i.e., the LTE-U ON time is proportional to the measured idle time in the channel. LTE-U BS switches the channel with the highest occupancy time. In this study, the LTE-U BS is facilitated using the Q learning approach [31; 44] and thus the BS is capable of adjusting its available actions according to the change in environment. In this way, LTE-U BS can modify the Q-matrix and acquire new best duty-cycle patterns and achieve its goal. Figures 5-7 show the simulation results of the joint LTE-U DCS and ACS mechanisms for DC only, COT, and the proposed mechanism. Here, the number of active STAs of each channel changes every 400 steps. Ten STAs of each channel move to other channels, maintaining the same number of total STAs in the network. In Figure 5a, we can see that the DC-only scheme uses a single channel throughout the simulation. However, in Figure 6a and Figure 7a, we can see that COT and the proposed scheme switch to best channel based on the changes in the number of STAs in the channel. Likewise, Figure 5b, \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter** & **Value** \\ \hline
**Common Parameters:** & \\ Number Of channel & 4 \\ Simulation Time & 1200 ms \\ Bandwidth & 20 MHz \\ Spectrum & 5 GHz \\ Traffic Model & Full Buffer \\ Transmission Scheme & OFDM \\ \hline
**LTE-U Parameters:** & \\ LTE-U BS & 1 \\ UE Number & 10 \\ Frame Duration & 10 ms \\ Duty Cycle & 0.2/0.4/0.6/0.8 \\ Transmit Power & 15 dBm \\ Terminal Noise Figure & 9 dB \\ PL Model & 32.8 + 20*log10(f) + 16.9*log10(d) (ITU InH model [43]) \\ Discount Factor \(\beta\) & 0 \\ Learning Factor \(\alpha\) & 0.3 \\ T\({}_{\text{init}}\) & 0.5 \\ \hline
**Wi-Fi Parameters:** & \\ Wi-Fi AP & 4 \\ STA Number & 10/20/30/40 \\ Wi-Fi MAC Protocol & DCF \\ Time Slot & 50 us \\ CW & 32–256 \\ SIFS & 28 us \\ DIFS & 128 us \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Simulation parameters. Figure 6b, and Figure 7b, show the LTE-U BS duty-cycle pattern against the number of steps. In Figure 5b, the DC-only scheme adaptably maintains its muting period according to the change in the number of users in the operating channel. However, in Figures 6b and 7b, we can see different trends in the LTE-U BS duty cycle against the number of steps, the reason being that the COT and the proposed scheme both have easier channel switching features. They switch to the most optimal channel and maintain the highest duty cycle pattern according to the target value. Figure 8 shows the convergence of the Q-learning process by the LTE-U BS. The horizontal axis indicates the number of steps, and the vertical axis is the aggregated sum of values of the Q matrix, i.e., the Q-value. When the Q matrix converges, the LTE-U BS has learned the present configurations and can execute the optimal duty-cycle pattern in any channel. We can see that the sum of the Q-matrix decreases during the start of the learning process.