# AUV-Aided Optical--Acoustic Hybrid Data Collection Based on Deep Reinforcement Learning
## 2 Related Works  
In recent years, multi-modal communication has become a research topic to improve network performance and optimize data transmission in various marine application scenarios. Commonly adopted multi-modal technologies include acoustic multi-modal communication and acoustic-optical hybrid communication [13; 22; 23; 24]. Among them, the acoustic multi-modal communication is constructed by a set of UAC modems working on different frequency bands [13]. In [22], the authors proposed a multi-modal underwater routing protocol based on the reinforcement learning technique. In this protocol, the reliability and delay of data transmission are optimized by UAC modems in multiple frequency bands. To explore the advantages of UAC and UOC during data transmission, [PERSON] et al. proposed an acoustic-optical multi-modal routing scheme based on packet size and link adaptation, which reduces packet loss and end-to-end delay [23]. However, the challenge of unbalanced energy consumption still exists in the multi-hop underwater networks. The AUV-assisted data collection can mitigate the energy consumption unbalance that occurs in multi-hop routing. [PERSON] et al. [18] explored the characteristics of underwater acoustic and optical communication in AUV-assisted data collection and showed that hybrid acoustic-optical data collection outperforms the one with a single acoustic modem in terms of both throughput and energy consumption. To cope with the impact of the harsh underwater environment, [PERSON] et al. [25] maximize the network throughput by capturing the dynamic characteristics of the channel and the mobility of the AUV. [PERSON] et al. [17] proposed a mobile data collection method for the heterogeneous sensor network using multi-hop acoustic communication to build an intra-cluster network where CHs collect large-scale data and upload them to a mobile receiver via optical communication. Although the aforementioned schemes improve the efficiency of data collection, these ignore the difference in the importance of data and the decay of freshness over time. To handle the aforementioned issues, [PERSON] et al. [16] proposed a greedy adaptive navigation algorithm to guide AUV for data collection, which considers the characteristics of data decay, but it is only applicable to sparse networks due to the unavoidable long paths. To deliver emergency data faster, [PERSON] et al. [26] proposed a hybrid data collection scheme, in which the urgent data are routed using a multi-hop scheme, while the delay-insensitive data are collected by AUVs. [PERSON] et al. [27] studied a hierarchical data collection problem using AUVs to optimize the information quality of the collected data while considering the importance and timeliness of the events. More recently, researchers have proposed the concept of AoI to model the timeliness of data while considering the quality of experience (QoE) [28]. [PERSON] et al. [29] provided an optimization algorithm to ensure the freshness of the collected data. [PERSON] et al. [5] used a vocational queuing model to improve the data reliability and peak AoI of the data. Then, the communication link is established using the UAC when the AUV arrives near the node. [PERSON] et al. [30] proposed a framework to optimize the trajectories of AUV and minimize the normalized weighted sum of the average AoI. [PERSON] et al. [31] studied the AUV transmission scheduling policy by considering both the age and the importance of the message. Although the aforementioned approaches have promoted the study of underwater data collection, only a single communication technology was considered for the data collection. Moreover, none of them addressed the issue of leveraging multiple data types and multiple communication technologies to improve data freshness and energy efficiency.