## 5 Simulation Results and Analysis
As shown in Figure 5a, in several cases, the algorithm encountered the local minimum problem, preventing the determination of a collision-free path. Despite introducing a random repulsive force vector that activated when the vehicle became trapped in a local minimum, the extensive transverse shape of the obstacles made it impossible to generate a feasible, collision-free trajectory. Considering the feasibility-constrained distance metric, the best result was achieved by the classical PSO algorithm (242 m) and the RL-based TQC algorithm (244.7 m). The longest path among the RL algorithms was recorded for TD3 with the HER buffer (472.6 m). Among the classical algorithms, the longest value for this metric was observed for the RRT* algorithm (366.8 m). For TD3 with the HER buffer, the high value of the feasibility-constrained distance metric can be attributed to visible circulations around the location of the goal, as shown in Figure 5i. In several cases, the vehicle failed to reach the goal due to the centrifugal force resulting from its relatively high speed, combined with \begin{table}
\begin{tabular}{l c c c c c c c c} \hline \hline
**Algorithm** & **Avg. Succ.** & **Feas. Constr.** & **Total Avg.** & **Reward** & **Avg. Std. Dev.** & **Avg. Comp.** & **Avg. Direct** & **Feas. Constr.** \\  & **Rate** & **Dist. [m]** & **Speed [m/s]** & & **of Hdg. Diff. [*]** & **Time [s]** & **Path \%** & **Time [s]** \\ \hline APF & \(0.63\pm 0.49\) & \(318.8\pm 140.2\) & \(1.39\pm 0.80\) & - & 19.929 & \(0.05\pm 0.02\) & \(86.3\pm 14.2\) & \(350.4\pm 121.7\) \\ A* & \(0.91\pm 0.29\) & \(288.8\pm 107.9\) & \(1.12\pm 0.47\) & - & 15.458 & \(1.70\pm 1.06\) & \(97.0\pm 9.0\) & \(316.4\pm 144.0\) \\ GA & \(1.00\pm 0.00\) & \(263.6\pm 105.4\) & \(1.61\pm 0.54\) & - & 7.075 & \(0.55\pm 0.09\) & \(100.0\pm 0.0\) & \(160.0\pm 45.5\) \\ PSO & \(1.00\pm 0.00\) & \(242.0\pm 110.0\) & \(1.65\pm 0.50\) & - & 6.274 & \(0.65\pm 0.10\) & \(100.0\pm 0.0\) & \(143.8\pm 38.7\) \\ RRT* & \(0.98\pm 0.14\) & \(366.8\pm 222.5\) & \(0.97\pm 0.27\) & - & 12.33 & \(0.04\pm 0.17\) & \(99.0\pm 7.8\) & \(418.9\pm 283.9\) \\ SAC & \(0.91\pm 0.29\) & \(269.4\pm 100.3\) & \(1.87\pm 0.55\) & \(-10.74\pm 5.20\) & 8.426 & \(0.46\pm 0.17\) & \(97.0\pm 8.3\) & \(144.7\pm 60.2\) \\ SAC-HER & \(0.91\pm 0.29\) & \(2852.2\pm 110.2\) & \(1.89\pm 0.56\) & \(-8.93\pm 4.60\) & 8.608 & \(0.49\pm 0.19\) & \(96.9\pm 9.1\) & \(152.0\pm 72.0\) \\ TD3 & \(0.85\pm 0.36\) & \(283.6\pm 111.1\) & \(1.88\pm 0.49\) & \(-10.57\pm 5.10\) & 8.636 & \(0.43\pm 0.15\) & \(97.0\pm 7.5\) & \(153.1\pm 51.2\) \\ TD3-HER & \(0.86\pm 0.35\) & \(472.6\pm 210.0\) & \(1.96\pm 0.52\) & \(-10.85\pm 5.25\) & 10.366 & \(0.71\pm 0.27\) & \(96.6\pm 9.8\) & \(245.1\pm 100.5\) \\ TQC & \(0.99\pm 0.10\) & \(244.7\pm 95.6\) & \(1.88\pm 0.50\) & \(-8.52\pm 4.30\) & 8.183 & \(0.46\pm 0.16\) & \(99.9\pm 0.3\) & \(133.3\pm 45.1\) \\ TQC-HER & \(0.93\pm 0.26\) & \(324.9\pm 135.7\) & \(1.89\pm 0.49\) & \(-9.10\pm 4.60\) & 8.94292 & \(0.50\pm 0.21\) & \(98.3\pm 5.6\) & \(172.2\pm 70.6\) \\ \hline \hline \end{tabular}
\end{table} 
Table 5: A comparison of the average results (\(\pm\sigma\)) for the analyzed algorithms. the vehicle's limited ability to perform tight turns imposed by its propulsion constraints and the circling problem mentioned earlier in this section. The poor performance of the TD3-HER algorithm was further amplified by the additional 25% penalty applied due to its relatively low average success rate. The increased value of the feasibility-constrained distance metric observed for the RRT* primarily results from the inherent sampling-based nature of the algorithm, which often produces suboptimal node distributions. Specifically, the randomized node placement leads to locally dense clusters that inadequately capture the global connectivity, resulting in excessively winding and indirect paths toward the goal (Figure 5e). Figure 5: Trajectories computed by classical path-finding algorithms—(**a–e**)—and reinforcement learning models—(**f–k**). It is worth noting that despite its significantly worse avg.