# Research on High-Efficiency Routing Protocols for HWSNs Based on Deep Reinforcement Learning
## 5 Algorithm Simulation and Result Analysis
### Simulation Experiment of Average Remaining Energy of Nodes in HWSNs  
As presented in Figure 8, after running for the same number of rounds, a comparison of the average remaining energy of the nodes among the six algorithms, including HDQN and five other algorithms, is conducted. This comparison focuses on how the average remaining energy of the heterogeneous nodes changes with the number of rounds. The maximum number of rounds is set at 1600. At round 1006, the average remaining energy of nodes running the NCOGA algorithm drops to 0 mJ. For the algorithms 2L-HMGEAR, DEEC, EEHCHR, SEP, and HDQN, the rounds at which the average remaining energy of nodes first drops to 0 mJ are 1305, 1431, 1458, 1713, and 1844, respectively. The experimental results indicate that the HDQN algorithm, by employing deep reinforcement learning to select information transmission paths, significantly enhances the efficiency of energy utilization in heterogeneous nodes. This leads to a more balanced energy consumption across nodes. Consequently, the HDQN algorithm maintains a higher average remaining energy of nodes over the same number of rounds compared to the other algorithms.  
The simulation experiment data presented in Table 3 compare the average remaining energy of the nodes in HWSNs using different algorithms. As the number of operational rounds increases, a higher average remaining energy of heterogeneous nodes signifies the better optimization of energy efficiency, allowing the network to perform tasks such as information collection, transmission, and processing more efficiently over an extended period without interruptions due to energy depletion. From Table 3, it is observed that after 500 rounds, the SEP algorithm demonstrates the best performance in terms of the average remaining energy of its heterogeneous nodes, with the proposed HDQN algorithm coming in second. However, as the number of rounds progresses, the advantages of the HDQN algorithm become increasingly apparent. After 600 rounds, the HDQN algorithm surpasses the other five algorithms, including SEP, in terms of the average remaining energy of its nodes.  
\begin{table}
\begin{tabular}{l c c c c c} \hline \hline \multirow{3}{*}{**Algorithm**} & **The 1 st Node’s** & **The 25 th Node’s** & **The 50 th Node’s** & **The 75 th Node’s** & **The 100 th Node’s** \\  & **Energy** & **Energy** & **Energy** & **Energy** \\  & **Consumption** & **Consumption** & **Consumption** & **Consumption** \\  & **Rounds** & **Rounds** & **Rounds** & **Rounds** \\ \hline EEHCHR & 183 & 398 & 644 & 823 & 1458 \\
2L-HMGEAR & 195 & 437 & 601 & 754 & 1305 \\ NCOGA & 111 & 319 & 455 & 696 & 1006 \\ DEEC & 187 & 357 & 506 & 653 & 1431 \\ SEP & 198 & 513 & 731 & 929 & 1713 \\ HDQN & 179 & 557 & 810 & 1057 & 1844 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Simulation experiment data on number of surviving nodes in HWSNsThis improvement can be attributed to the HDQN algorithm's unique approach of utilizing the real-time network status sensed by the Agent and employing decision-making for information transmission paths using an improved DQN algorithm. This mechanism effectively reduces energy consumption in heterogeneous nodes and enhances the network's sustained operation capability. Although the HDQN algorithm's average remaining energy was slightly behind the SEP algorithm by 1.98% in the early stages (before 600 rounds), it began to lead the other five algorithms from the 600 th round onwards. By this point, the HDQN algorithm showed improvements of 0.78%, 4.04%, 13.50%, and 15.37% over the 2L-HMGEAR, DEEC, EEHCHR, and SEP algorithms, respectively. This indicates that the HDQN algorithm continuously enhances the energy efficiency of HWSNs, with its optimization effects becoming more pronounced in the later stages of network operation. In summary, the HDQN algorithm's superior performance, especially during the later stages of the simulation, highlights its effectiveness in extending the operational lifetime and reliability of HWSNs through improved energy management and utilization.