# A Reinforcement Learning Based Intercell Interference Coordination in LTE Networks
## 5 The Proposed ICIC Scheme
### Resource Allocation Step
```
1 Initialize user's HoL vector Initialize user's transmission and retransmission buffers Set packet inter arrival time Set number of RBs to be allocated Set size of policies population Set generations number Set crossover rate Set mutation rate for eachTTIdo
2 foreach celldo
3 Generate initial population of policies with selected users in the user's selection step for eachgenerationdo
4 Select chromosomes to be crossed Cross selected chromosomes Select chromosomes to be mutated Mutate selected chromosomes Evaluate population of policies according to the fitness function Select chromosomes for the next generation population
5 end for Choose the best chromosomes according to the fitness function Allocate RBs to users in best policy for eachuserdo
6 Discard packets that have reached maximum delay allowed Discard packet that have reached maximum retransmission number Update user's HoL Update user's transmission buffer Update user's retransmission buffer
7 end for
8 end for
9 end for
```
**Algorithm 2**Resource allocation algorithm
```
1 for\(t\gets 1\)to\(Simulation\_time\)do
2 Packets_discarded\(\leftarrow\) 0
3 Total_packets\(\leftarrow\) 0
4 for\(p\gets 1\)to\(Population\_size\)do
5 for\(r\gets 1\)to\(Resource\_blocks\_number\)do
6 chromosome\([p][r]\gets random\) user in Selected_users
7
8 end for
9
10 end for
11 for\(g\gets 1\)to\(Generations\_number\)do
12\(Number\_of\_chromosomes\_to\_be\_crossed\leftarrow\) crossover_rate\(*Population\_size\)
13 for\(n\gets 1\)to\(Number\_of\_chromosomes\_to\_be\_crossed\)do
14\(n_{c}\leftarrow\) random number between 1 and \(Population\_size\) Add chromosome\([n_{c}]\) to chromosomes to be crossed
15
16 end for
17 for\(n\gets 1\)to\(Number\_of\_chromosomes\_to\_be\_crossed/2\)do
18 Cross chromosome\([n]\) with chromosome\([Number\_of\_chromosomes\_to\_be\_crossed/2+n]\)
19 end for
20 Add new \(Number\_of\_chromosomes\_to\_be\_crossed\) children obtained to population \(Number\_of\_chromosomes\_to\_be\_nutate\getsmutation\_rate*Population\_size\)for\(n\gets 1\)to\(Number\_of\_chromosomes\_to\_be\_mutate\)do
21\(n_{m}\leftarrow\) random number between 1 and \(Population\_size\) Add chromosome\([n_{m}]\) to chromosomes to be mutate
22 end for
23 for\(n\gets 1\)to\(Number\_of\_chromosomes\_to\_be\_mutate\)do
24 Mutate chromosome\([n]\)
25 end for
26 Add new \(Number\_of\_chromosomes\_to\_be\_mutate\) mutated chromosomes obtained to population for\(p\gets 1\)to\(Population\_size\)do
27\(sum\_HoL[p]\gets 0\)for\(r\gets 1\)to\(Resource\_Blocs\_number\)do
28\(sum\_HoL[p]\gets sum\_HoL[p]+Head\_of\_Line\_delay[chromosome[p][r]]\)
29 end for
30
31 end for
32 Sort chromosome according to \(sum\_HoL\) descending
33 Keep chromosome\([1]\) to chromosome\([Population\_size]\) for next generation population
34 end for
35 for\(r\gets 1\)to\(R\)do
36\(Resource\_Bloc[r]\gets chromosome[1][r]\)
37 end for
38 Update buffers and Head of Line delay of users
39 end for
```
**Algorithm 3**Resource allocation algorithm details
```
1 for\(r\gets 1\)toResource_Blocks_numberdo
2\(Head\_of\_Line\_delay[chromosome[1][r]\gets HoL[chromosome[1][r]-1\)
3 end for
4 for\(u\leftarrow 1\)toNumber_of_user_in_the_celldo
5\(HoL[u]\gets Head\_of\_Line\_delay[u]+1\)
6 end for
7 for\(u\leftarrow 1\)toNumber_of_user_in_the_celldo
8 while\(Head\_of\_Line\_delay[u]>delay\_threshold\)do
9\(Transmission\_Buffer[u]\gets Transmission\_Buffer[u]-1\)
10\(Head\_of\_Line\_delay[u]\gets Head\_of\_Line\_delay[u]-1\)
11\(Packets\_discarded\leftarrow Packets\_discarded+1\)Total_packets\(\leftarrow\)Total_packets\(+1\)
12 end for
13 while\(BSR[u]>buffer\_size\)do
14\(Transmission\_Buffer[u]\gets Transmission\_Buffer[u]-1\)
15\(Head\_of\_Line\_delay[u]\gets Head\_of\_Line\_delay[u]-1\)
16\(Packets\_discarded\leftarrow Packets\_discarded+1\)
17\(Total\_packets\gets Total\_packets+1\)
18 end for
19\(Packet\_loss\leftarrow\frac{Packet\_discarded[u]}{Total\_packets[u]}\)
20 end for
21 for\(r\gets 1\)to\(R\)do
22\(number\_of\_harq\_process\gets 0\)ifRetransmission_Buffer[chromosome[1][r]]>0\)then
23 if\(number\_of\_harq\_process<8\)then
24\(Retransmission\_Buffer[chromosome[1][r]]=Retransmission\_Buffer[chromosome[1][r]]-1\)
25\(number\_of\_harq\_process=number\_of\_harq\_process\)
26 else
27\(Transmission\_Buffer[chromosome[1][r]]=Transmission\_Buffer[chromosome[1][r]]-1\)
28 end for
29 else
30\(Transmission\_Buffer[chromosome[1][r]]=Transmission\_Buffer[chromosome[1][r]]-1\)
31\(Head\_of\_Line\_delay[chromosome[1][r]]=Head\_of\_Line\_delay[chromosome[1][r]]-interarrival\_time\_of\_packets\)
32 end for
33\(Total\_Packets=Total\_Packets+1\)
34 end for
```
**Algorithm 4** Buffers and HoL update