# Distributed Learning for Dynamic Channel Access in Underwater Sensor Networks
## 1 Introduction
To implement the energy-efficient operation of UASNs with three-dimensional topology, the amount of redundancy in a fountain-code-based transmission as well as the density of data sinks were optimized in [29; 30]. In [31; 32; 33; 34; 35; 36; 37], routing techniques and relay transmission protocols were proposed to improve the energy efficiency for multi-hop relay UASNs. Especially, the authors of [31] analyzed the energy consumptions of acoustic communication modems in various states (i.e., transmit, receive, and idle), and an energy-efficient routing algorithm was proposed by optimizing the hop length based on this analysis. In addition, in [32], a pressure-gauge-information-based routing protocol, which exploited periodic beaconing to build directional trails toward the surface and featured greedy opportunistic directional forwarding for packet delivery, was developed. More details on underwater routing protocols can be found in the survey papers [38; 39]. In the conventional wireless sensor networks (WSNs), one of the important issues is to the manner of accessing or assigning channels [40; 41; 42]. In [43], a multichannel protocol was proposed for WSNs, assuming that each IoT device was equipped with a single transceiver and that the MAC layer packet size was very small. In [44], a novel tree-based multichannel protocol based on a distance-aware interference model was presented and demonstrated through both simulations and real experiments for WSNs. The authors of [45] developed a channel allocation algorithm that could reduce the overhead of multichannel interference measurement by exploiting the power spectral density of the transmitter in low-power WSNs. However, all algorithms in [43; 44; 45] were centralized, and therefore were not suitable for UASNs because of the difficulty of maintaining a central entity that could coordinate IoT devices in underwater environments. Many channel allocation techniques for UASNs have been studied extensively, and a number of optimal and suboptimal solutions have been presented [46; 47; 48; 49; 50; 51; 52]. In [46], a channel sharing technique that takes advantage of long delays of underwater channel was presented, and it resulted in an improved spectrum efficiency compared to the conventional spectrum reuse scheme. The authors of [47] proposed a heuristic channel allocation method that provides performance improvements in terms of minimum capacity and fairness. In [48], a channel selection scheme for cognitive UASNs was investigated to increase fairness and maximize the minimum capacity based on user location information. In addition, the authors of [49] studied a joint channel and power allocation algorithm for cognitive UASNs that aim at providing efficient spectrum utilization while avoiding harmful interference to other UASNs. In [50], the problem of minimization of total collision-free transmission time was demonstrated as NP-hard, and therefore the authors proposed a suboptimal algorithm which could work much better than the conventional scheduling algorithms. In [51], a joint channel and power allocation algorithm was developed to maximize the network capacity. This algorithm could be operated in a distributed manner and had no overhead when compared to the conventional MAC protocols. In [52], a receiver-initiated spectrum management system was presented wherein receivers replaced the role of transmitters in conventional protocols as the initializers of the handshake process. By collecting the local sensing results from the neighboring transmitters, the receivers could assign vacant spectrum resources and optimal transmit powers. Recently, with the development of deep reinforcement learning (DRL) algorithms, dynamic DRL-based resource management problems were investigated in UASNs [53; 54]. In [53], an agent node which uses DRL-based MAC protocol learns underwater environment and occupy the spare time slots to achieve minimum collision when coexisting with a time division multiple access based node and a slotted ALOHA-based node.