## 3 Proposed Methodology
### Modified Ant Colony Optimization  
Among the various techniques discussed in the literature, ant colony optimization (ACO) is widely utilized to determine the sequential order of paths based on the total path length [20]. The AUV in question can alter its course without needing to gather information from the precise locations of the sensor nodes thanks to the successful transmission area that exists between the AUV and the SNs. Therefore, to establish the path order, we initially implemented a modified version of ACO to determine the sequence in which the AUV should visit the sensor nodes, relying on the exact coordinates of the SNs. The modified ACO (MACO) proposed in this work starts with the initial pheromone matrix, and it can be calculated as follows [21]:  
\[\tau_{0}=\frac{10\times Q}{n\times\left(\frac{1}{n}\sum_{i=1}^{n}\sum_{i=1}^{n}d _{i,j}\right)} \tag{2}\]  
where \(n\) represents the number of sensors and \(Q\) represents constant pheromone updating. In addition, the AUV journey starts from the mount point on the surface. We assume that the AUV's departure point from the ship is in the middle of the area to be surveyed. Hence, the first node represents the ship that the AUV starts its tour from this node or point. The second node is selected randomly as an integer within [2, \(n\)]. Then, the remaining nodes are selected consequently in a systematic way as follows [22]:  
1. Select the current node \(cn\) (i.e., second node), which is the last node in the tour. 2. Calculate transition probabilities \(P\) by using \[TP_{cn,j}=\left\{ \begin{array}{ll}\left(\tau_{cn,i}\right)^{\alpha(\text{I}) }\times\left(\eta_{cn,j}\right)^{\beta(\text{I})}&\text{$j\in unvisited\ nodes$}\\ 0&\text{otherwise}\end{array} \right.\] (3)  
\[P=\frac{TP}{\sum TP} \tag{4}\]  
where \(TP_{cn,j}\) and \(P\) represents the transition value and transition probability from the \(cn\) to the \(j\)th node, where \(j\) = 1, 2, 3, \(\ldots\), \(n\). The probabilities between the \(cn\) and the other remaining nodes are calculated according to unvisited nodes. \(\tau_{cnj}\) is a learning factor that denotes the value of the pheromone whenever there is a possibility of moving the AUV from the \(cn\) to the \(j\)th node. The pheromone value gives importance to a particular path by drawing on previous experiences gained during the previous search process. Figure 1: An illustration of the AUV trajectory to collect information from SNs. Consequently, the pheromone level creates an indirect connection between ants and their reinforcement learning expressed in Equation (8). On the other hand, the heuristic factor \(\eta\) is a fixed quantity that remains constant throughout the AUV tour and is independent of the pheromone parameters. It indicates the willingness or attractiveness of the AUV to travel from the \(cn\) to the \(jth\) node. Mathematically, the heuristic value can be represented by the inverse distance value (i.e., \(\eta_{cnj}=\frac{1}{\alpha_{cnj}}\)). \(\alpha(It)\) and \(\beta(It)\) represent the pheromone exponential weight and the heuristic exponential weight. The calculations of these weights are given by Equation (5). \[\alpha(It)=\beta(It)=13.217\times e^{-0.02\times It} \tag{5}\]  
Equation (5) possesses identical mathematical structures that are conceptually divergent. \(\alpha(It)\) regulates the pheromone weighting, affecting the reinforcement process, whereas \(\beta(It)\) regulates the heuristic weighting, affecting the utilization of past knowledge. The Roulette Wheel selection method is used to pick the next sensor node with respect to the derived probability of visiting the selected node in the future. This process continues until the current tour \(T\) is constructed. In addition to the aforementioned tour construction strategy, a new strategy is proposed to extend the exploration ability and overcome the stagnation problem of the algorithm in the search space.