# Solution for Interference in Hotspot Scenarios Applying Q-Learning on FFR-Based ICIC Techniques
## 10 Proposed Solution: Background, Implementation, and Simulation Results
### Evaluation Scenario  
The scenarios are similar to that of Section 9. There are three eNBs equally distanced by 1000 m, and the users are allocated in two manners. The first group of users is randomly allocated in the entire scenario, using a uniform distribution. The second group is allocated on hotspots with a 100 m radius, and there are five hotspots with fixed locations, as illustrated in Figure 15. This positioning aims at modeling different situations: close and far from a eNB and between multiple eNBs.  
```
1 Initializefor\(s\in S,a\in A\)do
2 Initialize Q-table with all \(Q(s,a)\) equal to 0.
3
4 end for
5 Estimate initial state \(s\).
6 Learning:
7 Loop
8 Generate random number \(r\in U(0,1)\)
9 if\(r<\epsilon\)then
10 Randomly choose action \(a\in A\) ;
11 else
12 Choose action \(a\in A\) according to Equation 3;
13
14 end if
15 Execute action \(a\);
16 Receive immediate reward \(r\), \(\text{SINR}_{\text{media}}\);
17 Observe next state \(s^{\prime}\in S\);
18 Update Q-table according to Equation 5;
19\(s=s^{\prime}\)
20 endloop
```  
**Algorithm 1**Pseudo-code of the proposed \(Q\)-_learning_ algorithm.  
As described in Section 2, this paper uses the term hotspot to describe regions with high user density. It does not mean a new AP. Hence, hotspot users do not have a dedicated AP. All users are served by the closest eNB.  
Table 9 presents the difference between the scenarios used to evaluate the QL algorithm. **Scenario 1** has 60 users distributed uniformly and 10 users on each hotspot. A total of 52 RBs are allocated for the common sub-band, and 16 RBs are allocated to the private sub-band. **Scenario 2** has the double of users on each hotspot and 2/3 of the uniform-distributed users when compared to Scenario 1. Hence, there are 40 users distributed uniformly and 20 users on each hotspot, which results in a higher number of users in the system. A total of 28 RBs are allocated for the common sub-band, and 24 RBs are allocated to the private sub-band. Therefore, the available bandwidth for users allocated in the common sub-band is significantly smaller. Other simulation parameters are summarized in Table 10.  
A hotspot (HS) begins to transmit and receive data every 10 s to evaluate the algorithm's ability to adapt. From 0 to 10 s, only the uniformly distributed users are active. Between 10 and 20 s, one of the hotspots is also transmitting/receiving data. From 30 s, another hotspot becomes active, and so forth. The change on the scenario during simulation is presented in Table 11. Before discussing the results, the following section details how data are collected on the ns-3 simulator to enable the QL calculations.  
\begin{table}
\begin{tabular}{l l} \hline \hline
**Parameter** & **Value** \\ \hline Bandwidth (RBs) & 100 \\ \hline UE distribution & Uniform / Hotspots \\ \hline Number of UEs & 110 / 140 \\ \hline Distance between eNBs (m) & 1000 \\ \hline Scheduling algorithm & _Proportional Fair_ \\ \hline Simulation duration & 60,000 subframes \\ \hline Channel model & Friis model \\ \hline Error model & MIESM \\ \hline UE mobility & No mobility \\ \hline Traffic model & Non-GBR TCP-based \\  & Video (Buffered Stream) \\ \hline \hline \end{tabular}
\end{table}
Table 10: Simulation parameters for the QL algorithm evaluation.  
\begin{table}
\begin{tabular}{l} \hline \hline
**Scenario 1** \\ \hline
60 users uniformly distributed \\ \hline
10 users on each hotspot \\ \hline BandwidthDistribution: 52/16 \\ \hline
**Scenario 2** \\ \hline
40 users uniformly distributed \\ \hline
20 users on each hotspot \\ \hline BandwidthDistribution: 28/24 \\ \hline \end{tabular}
\end{table}
Table 9: Evaluations scenarios for proposed solution.