# Self-Learning Power Control in Wireless Sensor Networks
## 5 Simulations
### Results
#### 5.2.1 Single Agent WSN  
The agent is found only in Q-state equal to zero that expresses the condition with null retransmissions and CCA reattempts. During this state, Figure 7 shows the normalized Q-value convergence of the transmitter at 2 m distance from the receiver for the four lowest transmission power levels, using an inter-arrival time of 25 ms. The separation among the power levels is fixed and represents the upper-bounds for each level. Such separation is the outcome of the reward function, linearly decreasing as the power level increases, as per Equation (9). The prevailing transmit power is the lowest, equal to \(-35\) dBm. In Figures 8 and 9, we analyse the PRR, latency and energy consumption per bit of the WSN in two different conditions, where \(d\) is equal to 2 m and 4 m, respectively, changing the average inter-arrival time value, \(\mu\), of the Poisson distribution for the traffic generation. Each curve of the homogeneous case shows the global performance, as the average among all the nodes. The results of the QL-TPC case are plotted with marks, where one mark represents the value of the transmitter's \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Phase** & \(\epsilon\) & \(\alpha\) & \(\gamma\) & **Time (s)** \\ \hline \multirow{4}{*}{Learning} & 1 & 0.9 & 0.8 & \(t<600\) \\  & 0.7 & 0.9 & 0.8 & \(600\leq t<1200\) \\  & 0.3 & 0.9 & 0.8 & \(1200\leq t<1800\) \\  & 0.1 & 0.9 & 0.8 & \(1800\leq t<2400\) \\ \hline \multirow{4}{*}{Convergence} & 0.1 & 0.1 & 0.8 & \(2400\leq t<3000\) \\  & 0.1 & 0.01 & 0.8 & \(3000\leq t<3600\) \\  & 0.1 & 0.001 & 0.8 & \(3600\leq t<4200\) \\ \hline Testing & 0 & 0.0001 & 0.8 & \(4200\leq t\leq 6000\) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Scheduling of \(\alpha\), \(\epsilon\) and \(\gamma\) over time. performance parameter, corresponding to the average transmission power used as per Equation (13). The simulations run for 500 s in the case of HG and 6000 s in the other case of QL-TPC, as stated in Table 3. When \(d\) is set to 2, the average PRR is 100% in HG, independently of \(\mu\) and transmission power used, since there is no interference and the receiver is in the communication range of the transmitter, regardless of power. In this simplest case, the QL-TPC is also able to obtain 100% of PRR, choosing the lowest average transmission power equal to \(-35\) dBm. When the average inter-arrival time is set to 25 ms, the selected transmission power is slightly higher at \(-34.40\) dBm. Although the average PRR is constant for all the transmission power levels, the average latency in Figure 7(a) decreases when the transmission power changes from \(-35\) dBm to \(-25\) dBm with constant value of \(\mu\). Moreover, the average latency raises at a fixed transmission power level and at increasing inter-arrival times. This is due to the higher average waiting time of the packets in the queue. The latency obtained by the agent using QL-TPC is better than in the homogeneous case for the selected transmission power by a factor that varies between 0.1 and 0.2 ms. Nevertheless, it is higher than the minimum latency of the homogeneous network. For the purpose of our protocol, it is not relevant because the focus is only on the maximization of PRR. The third parameter that we analyse is the energy per bit (Figure 7(b)). The node consumes less energy if the transmitter uses the lowest power level. Indeed, the current used by a device is proportional to the transmission power, as per Equation (12). The values of energy per bit in QL-TPC at the selected transmission power levels correspond to the minimum values in the homogeneous case. Therefore, the device learns to save energy using QL-TPC. Figure 8: Performance in WSN with 2 nodes during the testing period (2 m node distance): average latency (**a**); and energy consumption per bit (**b**). Figure 7: Normalized Q-value of the single agent in a network of two nodes (2 m node distance) and average inter-arrival time of 25 ms, where the trend is focused on the lowest transmission power levels.