## 1 Introduction
Furthermore, routing algorithms that rely on instant rewards may become stuck in local optima instead of discovering global optima. In addition to the scalability and energy efficiency, the reliability of the network is also critical to UASN. Opportunistic routing (OR) has been adopted by UASN to improve the reliability in wireless networks [12]. The opportunistic forwarding method can reduce packet loss while avoiding retransmission [13]. Nonetheless, the majority of existing OR protocols lack an alternation mechanism for sorting the priorities of relay set nodes, resulting in the frequent participation of dominating nodes in forwarding and nonuniform energy consumption over the network. [PERSON] et al. [14] proposed the GEDAR, which is a geographic and opportunistic routing protocol. Each node in the protocol greedily chooses the forwarding node with the highest expected packet advance (EPA), and the EPA is proportional to the distance between the nodes. The authors additionally designed a recovery mode for the void node based on the depth adjustment to address void routing. However, the greedy criterion and depth adjustments consume a lot of energy in packet forwarding while energy is extremely important for acoustic signal propagation in UASNs. Moreover, many researchers propose routing protocols to overcome the constraints of the undersea environment, which are based on routing strategies [15]. To some extent, the forwarding algorithm of protocols can be utilized to reduce latency and energy consumption, while the key problem remains unresolved. In terms of the direction of data forwarding, the traditional underwater sensor network usually works in the way that data packets are sent upward from the bottom to the ocean surface, namely the land data center [11]. However, the vast majority of monitoring data originates from the deep sea, hence long paths caused by the traditional architecture result in more energy consumption and transmission delay. [PERSON] et al. [16] shows that the major source of energy consumption is bulk data and long transmission distances, particularly in the underwater environment. Based on these facts, an on-site architecture is taken into consideration in UASNs, which deploys data centers under the sea. Fortunately, recent studies have demonstrated the viability of locating servers underwater [17]. With the on-site architecture, energy consumption and transmission delay can be significantly reduced. The acoustic channel in the deep sea is less affected by seasonal changes and transmission quality is much better, when comparing with the sea surface. Moreover, the cost of deploying and maintaining large-scale servers will be significantly lowered [18]. Therefore, to enhance the performance of opportunistic routing and intelligent routing algorithms, based on the on-site architecture, we propose a scalable and energy-efficient routing protocol (SROA) that applies the Q-Learning technique to the OR paradigm in UASNs. The SROA protocol is a clustered-based protocol with four phases and finds the optimal routing paths to achieve scalable and energy-efficient transmission. Three main contributions of this paper are summarized in the following. 1. We apply a novel on-site architecture to the proposed protocol, locating the data center near the data source on the seabed. The on-site architecture can effectively minimize the number of forwarding hops in routing by shortening the distance between the source and sink nodes, lowering the hop count in routing and enhancing transmission reliability. 2. We group the network into a number of clusters by an unsupervised learning algorithm. Besides, to improve the reliability of the network and to avoid the failure of a single-point, a mechanism for the selection of the cluster head and potential cluster head is designed, which both takes the residual energy and location of the nodes into account. 3.