## 1 Introduction  
During recent years, Underwater Acoustic Sensor Networks (UASNs) have gained much attention for the potential to explore and monitor the underwater environment [1], UASN is one of the fundamental techniques of the Internet of Underwater Things (IoUT), which was developed from the concept of terrestrial Internet of Things (IoT) [2]. Large-scale UASN enables the extension of IoT to ocean applications, considered to be a promising solution for exploring the oceans [3]. One of the key problems for these applications is how to collect and forward the sensed data from the source node to the sink node [4]. Owing to the unique features of the underwater acoustic environment, routing in UASNs confronts crucial challenges, such as signal propagation delay, limited bandwidth, and low energy efficiency [5]. Scaling up or down the network size according to the actual demand in the underwater environment is usually necessary, while maintaining the reliability of the network. Adaptive formation of the network is taken into consideration in this article, enabling nodes to independently join or leave the network. Moreover, the propagation delay has a significant impact on energy consumption, resulting in node failure due to fast energy depletion. Hence, centralized topology should avoid the failure of a single node, which could make the overall network crash. Moreover, due to the serious loss of signals and multipath effect, the packet loss rate of the acoustic channel leads to the unreliable transmission of data packets [6]. In addition, the energy of battery is constrained, and the battery is expensive to recharge or replace owing to the harsh underwater environment. To make things worse, the communication energy cost is greater than radio communications. In this context, to overcome energy constraints in UASNs, energy efficiency should also raise great attention [7]. Therefore, a scalable and energy-efficient routing protocol urgently needs to be designed. Recently, numerous routing techniques for UASNs have been proposed. To improve the network scalability, [PERSON] et al. [8] proposed a Self-Organizing and Scalable Routing Protocol. The proposed protocol makes use of a multi-hop communication method to send sensed data to the sink node, and each node creates its own routing table by utilizing control packet broadcasts during the startup and neighbor discovery phases. While the protocol makes the communication efficient and the network scalable, the energy consumption for control packets is relatively high. To decrease energy consumption, [PERSON] et al. [9] proposed the hop-by-hop vector-based forwarding (HHVBF) routing protocol to reduce energy consumption and network latency, in which the forwarders are selected within the virtual pipeline. The performance of HHVBF is highly related to the pipeline radius, and thereby short or large radius will result in the collisions and low delivery ratio, respectively. [PERSON] et al. [10] introduces another calculation called compelling Energy Resourceful Routing utilizing cost work. This calculation chooses the course that fulfills the nature of the administration in vitality stable, postpone requirements and throughput, hub in a flexible, and reduces power usage which drags out the lifetime of the system. Moreover, intelligent algorithms are utilized by routing protocols in UASNs. These routing protocols design the reward function, taking remaining energy into account. For example, [PERSON] et al. [11] presented a Q-learning-based routing protocol for energy-efficient and lifetime-extended underwater sensor networks (QELAR). The QELAR applied the Q-Learning approach to the underwater sensor network. Routing decisions were made using a reward function that took residual energy into account. However, the link quality is ignored, and propagation delay is not seriously considered, which may result in unreliable transmission when the distances between nodes are far from each other.