# AUV-Aided Optical--Acoustic Hybrid Data Collection Based on Deep Reinforcement Learning
## 5 Proposed DRL-Based Multi-Modal Data Collection Scheme
### Multi-Modal Steering Angle Optimization Algorithm  
In the multi-modal data collection network, since the communication radius can reduce the navigation time of AUVs, we propose an MSAO algorithm to adjust the AUV heading under the maximum steering angle constraint. In MSAO, the steering angle of the AUV is calculated based on its position, the navigation target and the communication options. As shown in Figure 2, the yellow triangle indicates the position \(p_{a,t}\) of the AUV at time slot \(t\), the blue pentagram indicates the CHs that need to perform the data collection operation, the outer circle \(C_{ac}\) and inner circle \(C_{op}\) indicate the communication range of UAC and UOC, respectively. Let \(c_{i}\) be the AUV's current target CH and \(c_{j}\) be the next target CH, the \(p_{r,m}\) indicates the target hover point when the AUV selects communication option \(m=\{ac,op\}\), the \(\psi_{m,t}\) indicates the angle of the AUV toward the target hover point \(p_{r,m}\) at time slot \(t\). The goal is to obtain the point \(p_{r,m}\) such that the \(\|p_{a,t}-p_{r,m}\|+\|c_{j}-p_{r,m}\|\) distance is shortest within the communication range \(C_{m}\) of the communication options \(m\). This problem is a classical pilgrimage problem in ancient castles, and hence, an approximate solution of \(p_{r,m}=(x_{r_{i}},y_{r_{i}})\) can be obtained by the following equation [45]. \[-\varsigma_{1}\sqrt{1-y^{2}}+\varsigma_{2}y=0, \tag{22}\]  
where \(\varsigma_{1}=\frac{x_{i}-x_{j}}{d_{m}d_{ij}}-\frac{x_{i}-x_{d,t}}{d_{m}d_{i,t}}\), \(\varsigma_{2}=\frac{y_{i}-y_{a,t}}{d_{m}d_{i,t}}-\frac{y_{i}-y_{j}}{d_{m}d_{ij}}\), \(y=\frac{y_{i}-y_{r_{i}}}{d_{m}}\), \(d_{ai,t}\) and \(d_{ij}\) denote the distance of the AUV from the target at time slot \(t\) and the distance of the current target CH \(c_{i}\) from the next target CH \(c_{j}\), respectively. Then, the steering angle of the AUV at time slot \(t\) can be expressed as \[\Psi_{m,t}=\left\{ \begin{array}{l}\min(\psi_{m,t}-\psi_{t},\psi_{\max}),\psi_{m,t }\geq\psi_{t}\\ \max(\psi_{m,t}-\psi_{t},-\psi_{\max}),\psi_{m,t}<\psi_{t}\end{array} \right., \tag{23}\]  
where \(\psi_{\max}\) is the maximum steering angle allowed by the AUV. Then, depending on the target location and the communication option, the steering angle of the AUV can be adjusted in the following two cases. * Case 1: The AUV is not through the region \(C_{m}\) from the current position \(p_{a,t}\) to the next target collection point \(c_{j}\); i.e., the distance \(d_{seg_{i}}\) from point \(c_{i}\) to the segment \(\overline{p_{a,t}c_{j}}\) is greater than the UAC radius. As shown in Figure 1(a), after determining the communication option, the points \(p_{r,ac}\) (or \(p_{r,\rho p}\)) are obtained in circle \(C_{ac}\) (or \(C_{op}\)) to minimize the length of the AUV trajectory. For example, when the CH \(c_{i}\), \(c_{j}\) and acoustic modem are selected, the AUV hover position \(p_{r_{i},ac}=(x_{r_{i}},y_{r_{i}})\) for data collection and the steering angle \(\Psi_{ac,t}\) can be calculated by (22) and (23), respectively. Similarly, when \(m=op\) holds, the data collection hover point \(p_{r_{i},op}\) and the steering angle \(\Psi_{op,t}\) can be obtained using the same approach. * Case 2: The trajectory of the AUV from the current coordinate \(p_{a,t}\) to the next target CH \(c_{j}\) sails through the communication region \(C_{m}\) of \(c_{i}\). If the AUV crosses the UAC area \(C_{ac}\) without crossing the communication area \(C_{op}\), \(d_{seg_{i}}\) becomes shorter than \(d_{ac}\) but greater than \(d_{op}\). As shown in Figure 1(b), the data collection hover point of the AUV is the vertical foot \(p_{r_{i},ac}\) from \(c_{i}\) to segment \(\overline{p_{a,t}c_{j}}\) if UAC is selected as the communication option. Then, the steering angle of the AUV can be obtained by (23). If the selected communication option is UOC, the data collection point and steering angle are calculated following the method in Case 1.