## 5 Simulation Experiments and Analysis
### Path Planning Scenarios
#### 5.2.3 Simulation Scenarios That Contain More Odd-Shaped Obstacles  
On the basis of the experimental scenario setting in Section 5.2.2, the experimental scenario is complicated again to verify the robustness of the OWS Q-learning algorithm. In the more complex scenario, 12 irregularly shaped obstacles were set, and the environment size was still 38 \(\times\) 38. The coordinates of the start and target points of each UGV are shown in Table 2.  
In the more complex scenario, the path planning simulation results of the SARSA algorithm, Q-learning algorithm, speedy Q-learning algorithm, and the OWS Q-learning algorithm are shown in Figure 9.  
From the path planning results in the more complex scenario, the SARSA algorithm only plans the route from the starting point to the target point for UGV2. The Q-learning algorithm only maps out the route from the starting point to the target point for UGV2. Compared with the simulation results in the simple scenario and the complex scenario, we find that with the increase in complexity of the experimental scenario, the effectiveness of the SARSA algorithm and Q-learning algorithm in solving multi-UGV path planning problems is getting worse and worse. This is because the SARSA algorithm is conservative in action selection and the Q-learning algorithm converges slowly, so it is difficult for the SARSA algorithm and the Q-learning algorithm to achieve the goal of planning a path for each UGV under the same round of training. Only the speedy Q-learning algorithm and the OWS Q-learning algorithm mapped out the route from the start point to the target point for the three UGVs. The specific experimental results are shown in Table 5.  
From the experimental results in Table 5, compared with the speedy Q-learning algorithm, the total path length planned by the OWS Q-learning algorithm for UGVs is shorter. Compared with the three comparison algorithms, the OWS Q-learning algorithm takes the shortest time under the same training episodes.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Index** & **Number** & **SARSA** & **Q-Learning** & **Speedy Q-Learning** & **OWS Q-Learning** \\ \hline \multirow{3}{*}{Shortest path steps} & UGV1 & 0 & 285 & 41 & 32 \\  & UGV2 & 178 & 166 & 35 & 35 \\  & UGV3 & 195 & 0 & 38 & 37 \\  & UGV1 & 0 & 298.61 & 50.36 & 42.60 \\ \multirow{2}{*}{Path length (unit)} & UGV2 & 197.45 & 183.31 & 42.53 & 44.36 \\  & UGV3 & 209.94 & 0 & 49.43 & 48.43 \\ \multirow{2}{*}{Calculation time (s)} & — & 12,143 & 11,255 & 5453 & 3577 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Comparison of simulation results of four algorithms containing only non-convex obstacles.  
The simulation results in the above three experimental scenarios show that compared with the SARSA algorithm, Q-learning algorithm, and speedy Q-learning algorithm, under the same training episodes, the OWS Q-learning algorithm has the shortest calculation time and the shortest planned path length, and the best effect of solving the multi-UGV path planning problem, which verifies the stability and robustness of the OWS Q-learning algorithm.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Index** & **Number** & **SARSA** & **Q-Learning** & **Speedy Q-Learning** & **OWS Q-Learning** \\ \hline \multirow{4}{*}{Shortest path steps} & UGV1 & 0 & 0 & 35 & 37 \\  & UGV2 & 301 & 162 & 35 & 37 \\  & UGV3 & 0 & 0 & 53 & 35 \\  & UGV1 & 0 & 0 & 45.60 & 48.01 \\ \multirow{4}{*}{Path length (unit)} & UGV2 & 319.53 & 174.89 & 44.77 & 45.53 \\  & UGV3 & 0 & 0 & 61.77 & 45.18 \\ \cline{1-1}  & — & 6789 & 9364 & 4149 & 3110 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Comparison of simulation results of four algorithms containing more odd-shaped obstacles.  
Figure 9: Path planning that contains more odd-shaped obstacles. (**a**) SARSA algorithm path planning, (**b**) Q-learning algorithm path planning. (**c**) Speedy Q-learning algorithm path planning. (**d**) OWS Q-learning algorithm path planning.