# Research on High-Efficiency Routing Protocols for HWSNs Based on Deep Reinforcement Learning
## 6 Conclusions  
This paper tackles the challenge of enhancing the overall energy efficiency in HWSNs through the exploitation of energy disparities among nodes. We introduce a DQN-based information transmission path selection algorithm, termed HDQN, which leverages the energy heterogeneity of the network and environmental characteristics to make real-time decisions on information transmission paths, thus improving the overall network efficiency. The HDQN algorithm employs the DQN framework to manage the intricate action space typical of network environments, focusing on optimizing energy efficiency. Our design of the reward function incorporates elements such as node distance, remaining node energy, and the number of relay nodes to balance the involvement of various heterogeneous nodes in the information transmission process and minimize overall energy consumption in the WSNs. The loss function within the DQN network utilizes mean squared error to compare the Q-value of the action taken in the current state with the anticipated cumulative reward, enabling a more precise estimation of cumulative rewards and enhancing decision-making strategies. Furthermore, we implement an improved packet header structure as input for network state information, facilitating the selection of appropriate data transmission optimization routing strategies. In particular, this packet header information is broadcasted to neighboring cluster head nodes, which subsequently propagate it to designated relay nodes until reaching the BS. This approach substantially boosts the overall energy efficiency of data transmission in HWSNs. Through simulation experiments comparing the average remaining node energy, number of surviving nodes, amount of information transmitted, total network remaining energy, and network robustness, our HDQN algorithm is evaluated against five other algorithms--EEHCHR, 2L-HMGEAR, NCOGA, DEEC, and SEP. The  
\begin{table}
\begin{tabular}{l c c c} \hline \hline  & \multicolumn{3}{c}{**The Number of Times Energy is Transferred Using Free Space**} \\ \cline{2-4}
**Algorithm** & **For 100 Nodes** & **For 200 Nodes** & **For 300 Nodes** \\ \hline EEHCHR & 72,360 & 156,848 & 257,446 \\
2L-HMGEAR & 57,564 & 123,420 & 210,091 \\ NCOGA & 63,475 & 93,090 & 238,809 \\ DEEC & 50,216 & 100,253 & 162,343 \\ SEP & 72,402 & 153,372 & 253,011 \\ HDQN & 83,503 & 169,091 & 260,918 \\ \hline \hline \end{tabular}
\end{table}
Table 7: HWSNs free-space energy transmission frequency simulation experiment data.  
Figure 12: Simulation of transmission counts with free-space energy model in HWSNs.  
results substantiate that the HDQN algorithm markedly advances energy efficiency and prolongs the network's lifetime in complex HWSNs environments.  
Methodology, Y.S. and X.H.; investigation, W.Z.; writing--original draft preparation, Y.S.; writing--review and editing, Z.L.; supervision, Z.L., Y.S., Z.L., X.H. and W.Z.; conceptualization, K.L. All authors have read and agreed to the published version of the manuscript.  
This work was supported in part by the Key Science and Technology Plan Project of Zigong City (Zigong Medical Big Data and Artificial Intelligence Research Institute) (2022 ZCYGY16), by the Teaching Reform Research Project of the Sichuan University of Science and Engineering (JG-24025), by the Sichuan University of Science and Engineering Graduate Education and Teaching Reform Research Project (JG202403), by the Third Phase of the Ministry of Education's Supply and Demand Docking Employment Education Project (2023122021680) in 2024, by the Second Batch of the Ministry of Education's Industry University Cooperation Collaborative Education Project (202102123021), by the Opening Project of the Key Laboratory of Higher Education of Sichuan Province for Enterprise Informationization and the Internet of Things (2022 WYY02), and by the Talent Introduction Project of the Sichuan University of Science and Engineering under Grant (2020 RC22).  
Data are contained within the article.  
The authors declare no conflicts of interest.