# Research on High-Efficiency Routing Protocols for HWSNs Based on Deep Reinforcement Learning
## 1 Introduction
### Related Work  
Research on HWSNs has yielded significant results, focusing primarily on energy efficiency optimization, load balancing, Quality of Service (QoS) assurance, routing protocol design, and energy-efficient communication strategies. The progress of HWSNs has been profoundly impacted by the demand for energy efficiency, especially in large-scale deployments where sensor nodes are mainly battery-powered. The heterogeneity of these networks, in terms of both node capabilities and mobility, presents distinctive challenges that require innovative solutions to optimize energy consumption and guarantee network robustness. This section examines the recent literature that tackles these challenges through novel routing protocols, deployment strategies, and energy management techniques. Regarding energy-efficient routing protocols, [PERSON] et al. introduced the NMSFRA routing protocol, designed for HWSNs characterized by mobile nodes. Distinct from traditional homogeneous WSN routing protocols, NMSFRA aimed to balance energy consumption by addressing the uneven cluster distribution and unstable network connections resulting from node mobility. This protocol effectively accommodated the dynamic nature of heterogeneous nodes, thereby enhancing overall energy efficiency and network stability [1]. Another significant contribution in this field was the energy-efficient cooperative routing scheme (EERH) proposed in reference [2]. The EERH enhanced data transmission efficiency in HWSNs by enabling different WSNs to share routing paths and nodes for event message forwarding. The routing strategy in the EERH was dynamically adjusted based on the remaining energy of the underlying sensors and their neighboring sensors, reducing transmission energy consumption and extending the network lifespan [2]. In terms of deployment optimization, reference [3] investigated the deployment of heterogeneous fusion centers (FCs) in WSNs, modeling the optimal placement of access points (APs) and FCs as an optimization problem with the objective of minimizing total wireless communication power consumption. The study considered both static and mobile WSN scenarios, emphasizing the need for adaptive deployment strategies in heterogeneous environments. In terms of clustering and energy management, the significance of efficient clustering techniques in prolonging the lifespan of HWSNs was emphasized in references [4; 5]. Reference [4] proposed a distributed energy-based epoch-clustering method combined with a multi-parameter weighted scalarization function to optimize cluster head selection. This approach introduced a novel weight calculation strategy using the analytical hierarchy process (AHP) and a two-phase analytical algorithm, enhancing cluster head selection efficiency. Similarly, reference [5] presented the cluster routing protocol for heterogeneous network (CPHN), which selected cluster heads based on both initial and remaining energy levels to maximize energy efficiency. This protocol ensured prolonged network operation by prioritizing nodes with higher energy reserves. In terms of performance analysis and enhancement, reference [6] critically assessed various fixed heterogeneous clustering algorithms, evaluating their performance in terms of network lifespan and throughput in mobile node environments. The proposed cluster head-restricted energy-efficient protocol for WSNs (CREW) modified channel selection thresholds in two-layer HWSNs to improve network survival time, overcoming the limitations observed in traditional fixed clustering algorithms. In addition, reference [7] addressed specific constraints in heterogeneous 5G WSNs, focusing on reducing costs and energy consumption and ensuring reliable data transmission. The proposed routing strategy aimed to alleviate challenges such as excessive energy consumption in void areas, packet loss, and the over-consumption of energy, enhancing network stability.