###### Abstract  
Message conflicts caused by large propagation delays severely affect the performance of Underwater Acoustic Networks (UWANs). It is necessary to design an efficient transmission scheduling algorithm to improve the network performance. Therefore, we propose a Deep Reinforcement Learning (DRL) based Time-Domain Interference Alignment (TDIA) scheduling algorithm (called DRLSA-IA). The main objective of DRLSA-IA is to increase network throughput and reduce collisions. In DRLSA-IA, underwater nodes are regarded as agents of DRL. Nodes intelligently learn the scheduling by continuously interacting with the environment. Therefore, DRLSA-IA is suitable for the highly dynamic underwater environment. Moreover, we design a TDIA-based reward mechanism to improve the network throughput. With the TDIA-based reward mechanism, DRLSA-IA can achieve parallel transmissions and effectively reduce conflicts. Unlike other TDIA-based algorithms that require enumeration of the state space, nodes merely feed the current state to obtain the transmission decision. DRLSA-IA solves the problem of computational expense. Simulation results show that DRLSA-IA can greatly improve the network performance, especially in terms of throughput, packet delivery ratio and fairness under different network settings. Overall, DRLSA-IA can effectively improve network performance and is suitable for ever-changing underwater environments. underwater acoustic networks; medium access control protocol; deep reinforcement learning; time-domain interference alignment +
Footnote †: journal: Marine Science and Engineering  
propagation delays between nodes do not change. However, the location of underwater nodes is constantly changing due to internal waves and other factors. The propagation delays between nodes vary with the location of the nodes. Consequently, the proposed underwater MAC protocol needs to fit well in the highly dynamic underwater environment. Time-domain Interference Alignment (TDIA)-based underwater MAC protocols have attracted wide attention due to their excellent performance [12; 13]. These protocols regard large propagation delays of UWANs as an advantage and achieve parallel transmissions using large propagation delays. The protocols usually adopt enumeration or greedy methods to obtain optimal scheduling [14]. However, these protocols are computationally expensive in general due to the massive state space. For example, in an underwater node collaboration network with \(N-\)nodes, the complexity of the enumerated decision space is \(N^{3}\), and the state space is even larger. The protocols are not feasible in terms of run-time and computation when the number of nodes slightly increases. Therefore, we need to design a scheduling algorithm that does not need to enumerate the decision space and state space. To address the above issues, we design a Deep Reinforcement Learning (DRL)-based TDIA scheduling algorithm for UWANs (named DRLSA-IA). We regard the underwater nodes as the DRL agents. In particular, nodes learn transmission scheduling as they continuously interact with the environment. Therefore, DRLSA-IA adapts to the highly dynamic underwater environment. Furthermore, instead of enumerating the state space, nodes merely feed the observed state to the trained algorithm to obtain transmission decisions. Hence, DRLSA-IA does not face computational restriction. To improve the network throughput, we design a novel TDIA-based reward mechanism. With the reward mechanism, the DRLSA-IA can align interference in time slots that have collided or interfered. In this way, the number of conflicting and interfered time slots can be effectively reduced. Additionally, nodes can use the free time slots to transmit messages and receive expected messages. Parallel transmission of messages also can be implemented. Therefore, DRLSA-IA can effectively improve the network throughput and reduce collisions. The main contributions of this paper are as follows:  
* We propose a deep reinforcement learning based TDIA scheduling algorithm (called DRLSA-IA).