## 1 Introduction  
Wireless Sensor Networks (WSNs) are wireless network systems composed of several small autonomous devices called sensor nodes distributed in space according to specific requirements. The function of the sensor node is to transmit the data to the base station or the destination node through sensing and collecting the ambient information, such as sound vibration, pressure, temperature, light intensity, and so on. With the development of wireless communication technology and the progress made over time, WSNs have been applied more and more widely in the field of information, involving many important fields such as environmental monitoring, urban management, industrial and agricultural automation, intelligent transportation, and military [1]. In recent years, WSNs have successfully replaced wired networks and been adopted in the industrial field due to simple deployment, low maintenance cost, and high flexibility [2]. However, the characteristics of wireless communication determine that interference and conflict inevitably exist in the process of data transmission, and data packets may be lost or delayed beyond their expected deadline [3]. Due to the importance of timing, packets produced in industrial environments often have strict deadlines. In order to achieve the reliability of data transmission, a feasible method is to use the Medium Access Control (MAC) protocol based on Time Division Multiple Access (TDMA) to eliminate the interference in the network. Furthermore, it can improve the probability of transmitting the packet to the destination node before the deadline. TDMA is widely used in wireless network communication because it is easy to implement and avoids data collision. The application of WSNs in the industrial environment should ensure that strict timing requirements are met and improve the reliability and real-time performance of message forwarding between sensor nodes [4]. Therefore, how to improve the reliability and real-time performance of data transmission in WSNs has become an important research topic in WSNs. The scheduling algorithm is one of the main methods and key technologies to improve wireless network reliability and real-time performance. Effective scheduling algorithms can realize the improvement of network technology and make it meet the requirements of WSNs environment with strict deadlines. Traditional scheduling algorithms are mostly heuristic scheduling algorithms, such as Earliest Deadline First (EDF) [5] or BSSA algorithm [6]. As the WSNs data transmission scheduling problem is proven to be an NP hard problem [7; 8], in recent years, researchers have turned their attention to introducing Machine Learning (ML) methods into WSNs. Many new algorithms were proposed in combination with ML methods in multiple aspects of WSNs [9]. In the design and research of WSNs, the research on functional requirements can be summarized into the following aspects according to the research direction of WSNs and the association among all directions: energy sensing and real-time routing, node clustering and data aggregation, event detection and query processing, localization and object positioning, and media access control protocol. Routing protocol and energy perception Research on routing protocols in WSNs is a hot field to solve quality-of-service (QoS)-related problems. Therefore, routing protocols must consider various challenges, such as energy consumption, fault tolerance, scalability, and data coverage. Traditionally, routing problems in WSNs can be abstracted as graph \(G=(V,E)\), where \(V\) represents the set of all nodes in the network and \(E\) represents the bidirectional communication edge connecting nodes. The routing problem can be defined as the process of finding the least-cost path from the source vertex to the destination vertex through the model graph \(G\). Reinforcement learning is used to propose a routing protocol based on the gradient to learn and find routes that exhaust node energy in a balanced way [10].