## 2 Related Works  
In recent years, in order to design energy-efficient routing protocols, researchers have combined various techniques such as clustering algorithms and machine learning algorithms, using ring routing structures, employing grid structures, and using mobile sinks. These methods have a large number of contributions to improve the network lifetime, but sometimes it is inevitable to sacrifice some network metrics such as packet delivery rate, end-to-end delay, throughput or network overhead. Therefore, when improving the network lifetime, we should not neglect the improvement of other network metrics so that the network has a strong comprehensive working capability and can adapt to the data collection in the case of no maintenance and management. Routing protocols based on the clustering approach usually assume that the sensor nodes can send data directly to the sink instead of using multi-hop data transmission. In this approach, the cluster head in each cluster needs to collect the data from the member nodes and send them to the sink. In [9], the authors propose a clustering and routing algorithm for fast-changing (FC-CRAs) large-scale wireless sensor networks. In FC-CRAs, clusters are constructed based on the cluster radius, which dynamically adapts to changes in node energy and distribution. The FC-CRAs algorithm improves nodes' survival time, increases network throughput, and saves network energy. In [10], the authors propose a novel distributed two-hop cluster routing protocol, where each sensor node obtains information about neighboring nodes within a two-hop range, the two-hop cluster is formed in a completely distributed way and the total transmission distance of inter-cluster communication is reduced. In [11], the authors exhibit an energy-saving clustering protocol based on adaptive Voronoi dividing, which effectively balances the energy consumption between cluster heads and cluster member nodes. The major drawbacks of these protocols are imbalanced energy consumption due to the frequent communication of the cluster heads and difficulty in practical applications in real life because of the limited single-hop communication range of sensors. In recent years, machine learning-based routing protocols have been widely used in WSNs. Standard geographic routing only considers the distance to the sink of neighboring nodes to select the next hop and does not make more favorable decisions based on the status information of the surrounding nodes; such a path selection scheme is often not optimal at present. Literature [12] combines deep learning to study an efficient packet routing strategy, each node can learn to develop a good routing strategy, which effectively reduces the network's energy consumption and improves the data transmission efficiency. In [13], the authors propose a multi-sink tactical mobile sensor network routing solution based on distributed multi-intelligent deep reinforcement learning (MADRL), which outperforms benchmark algorithms in terms of packet delivery rate (PDR), packet delivery time and energy efficiency. However, these machine learning algorithms need to obtain the status information of the surrounding nodes many times, which also increases the network overhead at the same time. In [14], the authors design a routing protocol with an inverse reinforcement learning concept in order to solve the problem of high network overhead caused by machine learning-based routing and the transmission environment of changing information due to node mobility; the network overhead and delay are reduced to some extent. However, the algorithm does not sufficiently reduce the energy consumption of the network. Saving energy consumed by sensor nodes in the network has always been a very important challenge in WSN, and the ring routing structure has been proven to be effective in saving energy.