## 2 Materials and Methods
### Classical Algorithms
Therefore, the approach used in this paper employs bicubic interpolation to downscale the map and applies morphological operations (dilation and closing) to artificially enlarge the obstacles, thereby preventing the algorithm from finding collision-prone paths on the reduced image. After planning the route in this simplified environment, the calculated path coordinates are then upscaled to the original map size. This process significantly reduces the number of operations required while maintaining a high level of accuracy in the final path. ```
1:Initialize \(K\) critic networks \(Q_{\theta_{k}},Q_{\theta_{k}},\ldots,Q_{\theta_{k}}\) and the actor \(\mu_{\phi}\) randomly
2:Initialize the target networks \(\theta_{k}^{\prime}\leftarrow\theta_{k}\) for each \(k=1,\ldots,K\), and \(\phi^{\prime}\leftarrow\phi\)
3:Initialize the replay buffer \(\mathcal{D}\)
4:for each time step \(t=1\) to \(N\)do
5:Select action \[a_{t}=\mu_{\phi}(s_{t})\ +\ \epsilon,\quad\epsilon\sim\mathcal{N}(0,\sigma)\] \(\triangleright\) Deterministic policy with exploration noise
6:Execute action \(a_{t}\) and observe the reward \(r_{t}\) and the next state \(s_{t+1}\)
7:Store \((s_{t},a_{t},r_{t},s_{t+1})\) in \(\mathcal{D}\)
8:Sample mini-batch \(\{(s_{i},a_{t},r_{i},s_{i}^{\prime})\}_{i=1}^{M}\) from \(\mathcal{D}\)
9:Compute the target action \[a_{i}^{\prime}=\mu_{\phi^{\prime}}(s_{i}^{\prime})+\text{clip}\big{(}\mathcal{ N}(0,\sigma),-c,c\big{)}\] \(\triangleright\) A similar smoothing approach to that for TD3
10:Compute \(K\) target quantiles: \[y_{i,k}=r_{i}+\gamma\ Q_{\theta_{k}^{\prime}}\big{(}s_{i}^{\prime},a_{i}^{ \prime}\big{)}\quad\text{for }k=1,\ldots,K\] \(\triangleright\) The distributional Bellman target
11:Sort \(\{y_{i,k}\}\) and select the lowest \(M\) quantiles \(\triangleright\) Truncation to reduce overestimation
12:Update each critic \(Q_{\theta_{k}}\) via quantile Huber loss: \[\mathcal{L}_{Q}(\theta_{k})=\frac{1}{M}\sum_{i=1}^{M}\rho_{\tau_{k}}\big{(}y_{ i,k}-Q_{\theta_{k}}(s_{i},a_{i})\big{)}\] \(\triangleright\) Distributional \(Q\)-learning objective
13:if\(t\bmod d=0\)then\(\triangleright\) Delayed update for the actor and the target
14:Update the actor \(\mu_{\phi}\) by maximizing the mean of the lowest quantiles: \[a^{\prime}_{i}=\mu_{\phi^{\prime}}(s^{\prime}_{i})\ +\ \text{clip}\left( \mathcal{N}(0,\tilde{\sigma}),\ -c,c\right)\]0 \(\triangleright\) Averaging to handle uncertainty
15:Soft-update targets: \(\theta_{k}^{\prime}\leftarrow\tau\theta_{k}+(1-\tau)\theta_{k}^{\prime}\), \(\phi^{\prime}\leftarrow\tau\phi+(1-\tau)\phi^{\prime}\)
16:endif
17:endfor
```  
**Algorithm 3** Truncated Quantile Critics (TQCs) with comments  
The genetic algorithm (GA) employed in this study follows the approach illustrated in Algorithm 6 and is directly inspired by the natural evolution process, in which only the best-adapted organisms survive. The procedure begins with generating a random population of potential solutions (paths), each represented by a set of waypoints. A high penalty factor is incorporated into the objective function for any waypoint that intersects with an obstacle's coordinates. In each generation, the least suitable individuals (those exhibiting the highest path cost) are eliminated, and the most promising candidates are chosen for replication. New offspring result from the crossover operation, in which complementary segments of the parents' genotypes are combined. Additionally, the mutation operation introduces random modifications into the newly generated offspring, preventing premature convergence to suboptimal solutions. The algorithm then evaluates all members of the current generation according to the objective function, keeping those with the lowest path cost. This iterative selection process continues until the maximum number of generations is reached. In the approach used in this study, two intermediate waypoints are calculated between the start and target points.