# Distributed Learning for Dynamic Channel Access in Underwater Sensor Networks
## 2 System Model  
We consider UASNs that consist of a data sink and a set of underwater sensors \(U=\{1,2,...,|U|\}\), as illustrated in Figure 1, where \(|\cdot|\) denotes the cardinality of a set.  
In the UASN, each underwater sensor collects a variety of information with respect to the underwater conditions, such as oceanographic data, ocean sampling data, environmental monitoring data, etc. It then sends the collected data to a data sink via an underwater acoustic link. The data sink gathers the data transmitted from the sensors and sends the aggregated data to a surface buoy through high-speed wired communication (e.g., wired fiber optical communication). Finally, the surface buoy, which is equipped with a radio frequency transceiver, transfers the data to a control center located on land. In this entire process of data transmission from the underwater sensors to a control sensor, this 'work focuses on the data transmission from the underwater sensors to the data sink via acoustic links.  
The time domain of the underwater acoustic link is divided into time slots, and each time slot is utilized for the transmission of one packet. At the beginning of every time slot, each underwater sensor must choose one channel from among the shared acoustic channels, denoted by \(K=\{1,2,...,|K|\}\), for packet transmission. It is assumed that the underwater sensors always have packets to transmit.  
Among a variety of MAC protocols, we consider a random access protocol, which is the representative MAC protocol adopted in UASNs. The transmission of underwater sensor \(u\) on channel \(k\) can be successful if and only if sensor \(u\) alone occupies channel \(k\) (i.e., there is no collision) and there is no channel error on channel \(k\) due to bad channel quality in a given time slot. After the data  
Figure 1: Illustration of underwater acoustic sensor networks (UASNs).  
transmission, sensor \(u\) receives feedback information from the data sink, which indicates whether the transmitted data have been delivered successfully to the data sink or not. In this work, we consider a two-bit feedback scenario. Let \(o_{u}\left(t\right)\) be the feedback information of sensor \(u\) at time slot \(t\), which is defined as follows,  
\[o_{u}(t)=\begin{cases}0,&\text{if transmission has succeeded,}\\ 1,&\text{if channel error has occurred,}\\ 2,&\text{if collision has occurred.}\end{cases} \tag{1}\]  
More specifically, if a packet has been delivered successfully, then \(o_{u}\left(t\right)=0\). On the other hand, if the transmission has failed owing to channel error due to bad channel quality without collision, then, \(o_{u}\left(t\right)=1\). Otherwise, \(o_{u}\left(t\right)=2\), i.e., the transmission has failed due to collision. We differentiate three types of feedback because, when packet collision occurs (i.e., \(o_{u}\left(t\right)=2\)), it will be impossible to know at the data sink whether a channel error has occurred or not, regardless of the occurrence of an actual channel error. In this case, by feeding back this fact to the underwater sensor, the estimation of CEP is not updated, which will be explained in a next section.