# A Reinforcement Learning Based Intercell Interference Coordination in LTE Networks
## 3 Problem Formulation  
Intercell interference is caused by multiple sources transmitting signals with the same subcarrier and received by a receiver. A user receives signals from the serving cell and neighboring cells but at different power levels due to the pathloss. The interference is measured by the Signal to interference plus noise ratio (SINR).  
SINR for a user \(UE_{j}\) located on cell \(j\) can be expressed by [29]:  
\[SINR_{UE_{j}}=\frac{p_{j}r_{j}^{-\alpha}}{N_{0}B+\sum_{k=1,k\
eq j}^{k}p_{k}r_ {k}^{-\alpha}} \tag{1}\]  
where \(\alpha\) is the path-loss exponent, \(p_{j}\) is the transmit power of the serving enodeB \(j\), \(n\) is the number of neighboring enodeBs, \(p_{k}\) is the transmit power from the neighboring enodeBs, \(r_{j}\) is the distance of the user \(UE\) to the serving station, \(r_{k}\) is the distance of the user \(UE\) to each of the neighboring stations, and \(N_{0}B\) is the background noise with \(N_{0}\) the thermal noise and B the system bandwidth.  
Assuming a severe interference scenario, the background noise \(N_{0}B\) can be ignored [29]. The expression of SINR can then be rewritten as:  
\[SINR_{UE}=\frac{p_{j}r_{j}^{-\alpha}}{\sum_{k=1,k\
eq j}^{k}p_{k}r_{k}^{- \alpha}} \tag{2}\]  
The expression of SINR shows that it depends on the distance between the user \(UE\) and the transmitter. It clearly appears that SINR for cell edge users will be low as channel gain for neighboring cells will be greater than transmit power of serving cell. In the case where each cell is transmitting with the same power level, cell edge users will experience low SINR as the distance between serving enodeB and some neighboring enodeBs are approximatively the same.  
To ensure good quality signal received by a user, interference has to be minimized. Minimizing interference is the same thing as maximizing SINR value. This becomes an optimization problem.  
From the expression of the SINR, it can be seen that it can be increased either by decreasing transmit power from neighboring cells or by increasing transmit power from serving cell. This can be done using power control by coordination between enodeBs through X2 interface.  
The distance of a user from its serving enodeB and neighboring enodeBs also cannot be changed but transmit power from serving enodeB and neighboring enodeBs can be assigned different values to increase SINR value. Given that there are multiple neighboring enodeBs, it becomes mathematically difficult to get a solution of finding transmit power of serving enodeB and neighboring enodeBs when targeting a desired SINR value.  
Heuristic methods are well suited for this kind of optimization problem. They use computational intelligence to find solution to optimization problems. Some heuristic methods that use computational intelligence are genetic algorithm (GA) [32], ant colony optimization (ACO) [33], swarm optimization (PSO) [34], simulated annealing (SA) [35], reinforcement learning (RL) [13], and decision tree [36].  
A comparison of some CI approaches in [15] shows that reinforcement learning and swarm optimization have low computational requirements, medium memory requirements, high flexibility, and are optimal, thus are the best choices to consider for our optimization problem. Since LTE networks offer online services to users to users, these approaches cannot be used on live LTE networks because of the computation time they will require which will certainly lead to disasters. Reinforcement learning however offers the advantage that it can learn the optimal solutions with data coming from real networks through exploration process by simulating the real network [13] and can then be use online using exploitation process. We thus chose reinforcement learning instead of swarm intelligence.