## 2 Materials and Methods
### Classical Algorithms  
This study used five classical algorithms--artificial potential field (APF), A*, a genetic algorithm (GA), particle swarm optimization (PSO), and Rapidly Exploring Random Tree-Star (RRT*)--which are well established in the literature and widely used for AUV path planning. Each algorithm generates a computed path from the starting point to the target location in the form of coordinates based on a known obstacle map. The generated path is then processed for smoothing and trajectory determination to ensure the feasibility of the vehicle. One of the classical approaches explored in this study is the artificial potential field (APF) algorithm, which relies on an attractive force directed toward the target and a repulsive force generated by an obstacle. The well-known issue of local minima is mitigated by limiting the range of the repulsive field, thereby preventing distant objects from exerting unnecessary influence. Additionally, the total force exerted on the vehicle is supplemented by a random component to facilitate escape from potential trap regions and to reduce oscillatory behavior in narrow passages. The APF technique is integrated into a path planning framework, where subsequent waypoints are derived from the net force of attraction and repulsion. The pseudocode for APF is presented in Algorithm 4. ```
1:Initialize the critic networks \(Q_{\theta_{1}},Q_{\theta_{2}}\) and the actor network \(\mu_{\phi}\) with random parameters
2:Initialize the target networks \(\theta^{\prime}_{1}\leftarrow\theta_{1}\), \(\theta^{\prime}_{2}\leftarrow\theta_{2}\), \(\phi^{\prime}\leftarrow\phi\)
3:Initialize the replay buffer \(\mathcal{D}\)
4:for each time step \(t=1\) to \(N\)do
5:Select action \[a_{t}\ =\ \mu_{\phi}(s_{t})\ +\ \epsilon\quad\text{where }\epsilon\sim \mathcal{N}(0,\sigma)\] \(\triangleright\) Add Gaussian noise for exploration
6:Execute action \(a_{t}\) in the environment and observe the reward \(r_{t}\) and the next state \(s_{t+1}\)
7:Store transition \((s_{t},a_{t},r_{t},s_{t+1})\) in buffer \(\mathcal{D}\)
8:Sample mini-batch \(\{(s_{i},a_{i},r_{i}\,s^{\prime}_{i})\}_{i=1}^{M}\) from \(\mathcal{D}\)\(\triangleright\) Batch size \(M\)
9:Compute the target actions for smoothing (TD3 trick): \[a^{\prime}_{i}=\mu_{\phi^{\prime}}(s^{\prime}_{i})\ +\ \text{clip}\left( \mathcal{N}(0,\tilde{\sigma}),\ -c,c\right)\] \(\triangleright\) Clipped noise for policy smoothing
10:Compute the target values \[y_{i}=r_{i}\ +\ \gamma\min_{j=1,2}Q_{\theta^{\prime}_{j}}\big{(}s^{\prime}_{i},a^{ \prime}_{i}\big{)}\] \(\triangleright\) Use both critics and take the minimum
11:Update the critic networks \(Q_{\theta_{j}}\) by minimizing \[\mathcal{L}_{Q}(\theta_{j})=\frac{1}{M}\sum_{i=1}^{M}\big{(}Q_{\theta_{j}}(s_{ i},a_{i})-y_{i}\big{)}^{2}\] \(\triangleright\) The mean squared error with relation to target \(y_{i}\)
12:if\(t\ \text{mod}\ d=0\)then\(\triangleright\) Delay actor/target updates for stability (TD3 trick)
13:Update the actor network \(\mu_{\phi}\) by maximizing \(Q_{\theta_{1}}\): \[\mathcal{L}_{\mu}(\phi)\ =\ -\frac{1}{M}\sum_{i=1}^{M}Q_{\theta_{1}}\big{(}s_{i},\ \mu_{\phi}(s_{i})\big{)}\] \(\triangleright\) Deterministic policy gradient
14:Soft-update the target networks: \[\theta^{\prime}_{j}\leftarrow\tau\,\theta_{j}+(1-\tau)\theta^{\prime}_{j}, \quad\phi^{\prime}\leftarrow\tau\,\phi+(1-\tau)\phi^{\prime}\] \(\triangleright\) Slowly track learned parameters
15:endif
16:endfor
```  
**Algorithm 2** Twin Delayed Deep Deterministic Policy Gradient (TD3) with comments  
Another classical algorithm used in this study is the A\({}^{\star}\) approach described in Algorithm 5. Its operation involves discretizing the area into cells and computing the total cost \(F(n)\), which is the sum of the distance traveled from the start \(G(n)\) and a heuristic estimate of the distance to the goal \(H(n)\). While A\({}^{\star}\) efficiently determines a path considering static obstacles, the computational load increases substantially with larger maps or numerous obstacles.