## 3 Results  
As shown in Figure 8b, the node location in the static topology model is immovable, while it was observed that nodes were constantly moving random distances in random directions in the dynamic  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Parameter** & **Value** & **Unit** \\ \hline Field dimension & \(1000\times 100\) & m\({}^{2}\) \\ Number of sensor (N) & 110 & sensors \\ Initial Energy & 0.1 & Joule \\ _EEFS_ & \(10^{-12}\) & Joule \\ _EEMP_ & \(0.0013\times 10^{-13}\) & Joule \\ _Eelcc_ & \(50\times 10^{-9}\) & Joule \\ _EDA_ & \(50\times 10^{-9}\) & Joule \\ _Etrans_ & \(50\times 10^{-9}\) & Joule \\ _ERX_ & \(50\times 10^{-9}\) & Joule \\ Message length (\(L\)) & 4000 & Bits \\ Round (\(r\)) & 2000 & - \\ \(\alpha\) & 0.1 & - \\ \(\beta\) & 0.2 & - \\ \(\gamma\) & 0.3 & - \\ \(\varepsilon\) & 0.99 & - \\ \(a\) & 1,2,3.... 10 & - \\ Threshold(T) & 8 & dB \\ _mrandom_ & 0.1 & m \\ \hline \hline \end{tabular}
\end{table}
Table 2: Simulation parameters of the proposed LEACH protocol.  
Figure 7: Q-LEACH: Target matrix from original topology and Q-table probability.  
topology model, as shown in Figure 8a. The original node location is marked in red, and, while round is operated 3 times, the nodes move to each direction marked as a green arrow.  
AS shown in Figure 9, we calculate energy loss, number of alive nodes and data throughput as evaluation index of LEACHs. D-LEACH, Q-LEACH and F-LEACH which sets CH in their own way, perform better than LEACH, which is characterized as random setting of CH.  
In the static topology model, he performance of D-LEACH and F-LEACH were similar, but in the case of moving nodes in the dynamic topology model, the performance of F-LEACH is significantly lowered compared to D-LEACH. In particular, the significant reduction in the life time of the node in F_LEACH at the beginning increases energy consumption and decreases throughput significantly.  
Figure 8: The node location (**a**) dynamic topology model, (**b**) static topology model.  
Figure 9: LEACH results on energy loss, number of alive nodes, throughput. (**a–c**) is the result from dynamic topology model and (**d–f**) is the result from static topology model.  
As a result of Q-LEACH & F-LEACH and existing LEACH (LEACH, D-LEACH) on the aspects of energy loss, number of alive nodes and throughput, Q-LEACH showed the best performance both when a node was immobile/mobile. Q-LEACH on the static node model shows constantly better performance; on the other hand, at the beginning of operation, Q-LEACH and D-LEACH in the dynamic node model shows a similar tendency; however, over time, it can be found that the performance of Q-LEACH has been greatly improved by the reinforced learning effect.  
Figure 10 shows the comparison of various LEACH protocols, and the percentage of the life time of node each system compared to original LEACH is marked [56; 57].  
As in this study, Q-table is applicable since the table is only 4,000,000 (1000 (\(x\) grid) \(\times\) 1000 (\(y\) grid) \(\times\) 4 (up, down, left, right)) but in the most of application such as invade game the overall calculation is determined by the number of pixels. As related studies on comparison of RL algorithms shows that performance of RL algorithms are showing environment dependence [58]. The environment of this study has the same structure as the OpenAI gym environment [59], and this allows developers to easily and quickly implement the environment by sharing the environment. As can be seen in the pseudocode of Q-LEACH, we tried to improve the performance optimized for the situation by customizing the logic in the openAI environment. In addition, we presented as experimental results that the Q-table showed better performance than other RL methods. The Q-network is applied to a vast number of cases situation, and the Q-table is more suitable for this study because it shows better results in finite situations. As mentioned above, performance improvement is observed in Q-LEACH compared to other LEACH methods.