# Rough-Terrain Path Planning Based on Deep Reinforcement Learning
## 4 Simulation Experiment
### Simulation Results and Analysis
The A-DQN algorithm in this paper is very close to the path quality of the Dijkstra and A* algorithms and, in most cases, prefers to choose the routes with lower gradients and fewer uphill slopes to save energy. As shown in Table 4, the A* algorithm resulted in a path length of 4868.66 m and energy consumption of 15,319.1458 u. The path planned by A* is the same as that of Dijkstra. The energy consumption of the paths planned by both is minimal. This is because the global path planner from Dijkstra constructs a cost map, allowing it to plan paths with lower total energy consumption, as shown in Table 5, with the lowest energy consumption at 10,014.551 u. However, the Dijkstra algorithm is very time-consuming due to its large search range. The A* algorithm has a much shorter planning time compared to the Dijkstra algorithm. As shown in Table 5, A* takes about half as long as Dijkstra. This is due to the fact that A* performs a heuristic search. However, the heuristic function in the rugged-terrain design for A* is complex. The RRT algorithm can cover a large area in high-dimensional space, but its randomness leads to its poor performance in rugged scenes, as seen in Table 3, with a path length of 5255.3713 m and an energy consumption of 25,210.7808 u, both at their largest. It cannot be directly used in the planning of undulating paths and usually consumes more resources. Figure 7: Test 2 path diagram: (**a**) A-DQN (red), Dijkstra (blue) 3D view of the planning path. (**b**) A* (yellow), RRT (black) 3D view of the planning path. (**c**) A-DQN (red), Dijkstra (blue) 2D view of the planning path. (**d**) A* (yellow), RRT (black) 2D view of the planning path. The paths planned by the proposed A-DQN algorithm closely match those of the Dijkstra algorithm, and in some cases, it performs better at balancing the path length and energy consumption. Once trained, the neural network can quickly plan paths through inference, avoiding extensive searching, and improves the efficiency of path planning. To verify the ability of the terrain recognition, a 5 \(\times\) 5 range of muddy ground was set in Map 1, and the muddy ground range of 10 \(\times\) 10 was set in Map 2 and Map 3, as shown in Figure 9. After setting up the muddy-terrain region, the agent underwent 8000 episodes of training. A 200-round test was conducted, and the training convergence curve and test results are shown in Figure 10.  
Figure 8: Test 3 path diagram: (**a**) A-DQN (red), Dijkstra (blue) 3D view of the planning path. (**b**) A* (yellow), RRT (black) 3D view of the planning path. (**c**) A-DQN (red), Dijkstra (blue) 2D view of the planning path. (**d**) A* (yellow), RRT (black) 2D view of the planning path. Figure 9: Map with 10 \(\times\) 10 range of muddy ground. Taking Map 1 as an example, the start point and end point for testing were randomly selected. Several sets of representative starting and finishing points were compared with the training results in the original map, and the results are shown in Figure 11. As can be seen from Figure 11, the \(5\times 5\) mud field was set in the valley terrain. The comparison between Figure 11, shows that the agent's driving path effectively avoided this \(5\times 5\) mud range and learned to go around the mud, which indicates that the A-DQN algorithm effectively recognized the terrain. In addition, the agent also chose the route with less uphill in the bypassed driving path. Thus, it is evident that the A-DQN algorithm effectively selects energy-efficient paths and demonstrates reliable terrain recognition, which proves the rationality and effectiveness of the algorithm. Figure 10: A-DQN algorithm reward change curve after muddy-ground setting (light color: original rewards; dark color: rewards smoothed by moving average).