# Rough-Terrain Path Planning Based on Deep Reinforcement Learning
## 4 Simulation Experiment
### Simulation Results and Analysis  
In order to verify the path and energy consumption balance effect, the terrain parameters were all set to 0 and the speed was set to 45 km/h at a constant speed for training at first. The variations in the reward with the training rounds of Map 1, Map 2, and Map 3 are shown in Figure 4(a),c,e. A total of 200 sets of starting and ending points were randomly selected for testing, and the reward curves of the test rounds are shown in Figure 4(b),d,f. Figure 4: Map of simulation experiment. Figure 5: A-DQN algorithm reward change curve (light color: original rewards; dark color: rewards smoothed by moving average). The simulation results of the A-DQN were compared with the RRT, A* algorithm, and Dijkstra algorithm. The numerical statistics of the results of 200 tests under different maps and different algorithms are shown in Table 2. The cost in the Dijkstra algorithm is computed as the sum of the three-dimensional distance and the energy consumption from the starting point to the current point, with the energy consumption calculated according to Equation (11). In the A* algorithm, the heuristic function calculates distance based on the intersection line between the vertical plane formed by the current point and the goal point and the terrain surface. The energy consumption along this path is also determined using Equation (11). Since the energy consumption of a path is as important as the distance traveled, a direct comparison is made by summing the two, with paths with smaller sums indicating better global properties. From the results, it can be seen that the A-DQN is superfast and the least time-consuming to plan, with an average time of about 0.23 s in the Map 1 scenario. Also, its path quality is comparable to that of the Dijkstra and A* algorithms. Although the Dijkstra algorithm path quality is high, it takes more time because of the large-scale search. The A* algorithm takes less time than the Dijkstra algorithm, as it explores through the designed heuristic function. Overall, the A-DQN maintains both the decision time and path quality. Taking Map 1 as an example, three sets of typical test results were selected to show the comparison of the paths planned using the A* algorithm, RRT algorithm, and global path planner in the Dijkstra. The paths are shown in Figures 6-8. The quantitative results are shown in Tables 3-5. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{4}{c}{**Test 2**} \\ \hline
**Arithmetic** & **Distance (m)** & **Energy (u)** & **Sum** & **Time (s)** \\ \hline A-DQN & 4518.2799 & 15,580.0503 & \(2.0098\times 10^{4}\) & 0.23 \\ [PERSON] & 4868.6638 & 15,319.1458 & \(2.0188\times 10^{4}\) & 85.72 \\ A* & 4868.6638 & 15,319.1458 & \(2.0188\times 10^{4}\) & 42.67 \\ RRT & 6050.9644 & 30,981.1138 & \(3.7032\times 10^{4}\) & 158.12 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparison of test 2 path results. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{4}{c}{**Test 3**} \\ \hline
**Arithmetic** & **Distance (m)** & **Energy (u)** & **Sum** & **Time (s)** \\ \hline A-DQN & 4293.9799 & 10,076.9074 & \(1.4371\times 10^{4}\) & 0.22 \\ Dijkstra & 4434.5732 & 10,014.551 & \(1.4449\times 10^{4}\) & 63.60 \\ A* & 4434.5732 & 10,014.551 & \(1.4449\times 10^{4}\) & 29.19 \\ RRT & 4426.41 & 21,454.6406 & \(2.5881\times 10^{4}\) & 199.36 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Comparison of test 3 path results. Figure 6: Test 1 path diagram: (**a**) A-DQN (red), Dijkstra (blue) 3D view of the planning path. (**b**) A* (yellow), RRT (black) 3D view of the planning path. (**c**) A-DQN (red), Dijkstra (blue) 2D view of the planning path. (**d**) A* (yellow), RRT (black) 2D view of the planning path. From the quantitative results of the randomized test, as shown in Tables 3-5, it can be seen that that the paths planned by the A* algorithm and Dijkstra algorithm are very close to each other and, in some scenarios, plan the same paths; the RRT algorithm path length and energy consumption are relatively large.