# Rough-Terrain Path Planning Based on Deep Reinforcement Learning
## 1 Introduction
And such traditional algorithms are very time-consuming in large-scale exploration, especially when the map is very large. Deep reinforcement learning (DRL) is an artificial intelligence method that combines the perceptual capabilities of deep learning and the decision-making capabilities of reinforcement learning. The trained neural network can make decisions based on the system situation and guide the unmanned vehicle in choosing a better path according to the strategy. With the development of deep reinforcement learning algorithms, more and more scholars have begun to use them to solve the problem of path planning. For example, [PERSON] et al. proposed a DRL-based navigation method that has been used to navigate robots on flat surfaces with dynamic obstacles for navigation [26]. However, this method only considers flat terrain and does not apply to rugged terrain. For rugged terrain, [PERSON] and [PERSON] proposed a localized path-planning methodology using a deep reinforcement learning approach for vehicle safety planning [27]. [PERSON] et al. proposed a learning-based planning method for predicting the passability of mobile ground robot roads on unstructured terrain [28]. [PERSON] et al. proposed a maximum entropy-based nonlinear inverse reinforcement learning (IRL) framework to more accurately deal with corner situations in the environment, such as staircases, slopes, and underpasses [29]. [PERSON] et al. used deep reinforcement learning (DRL) to develop and train a DRL network that uses raw perceptual data from the robot's on-board sensors to determine a set of local navigational operations to be performed by the mobile robot, solving the problem of navigating the robot in a cluttered environment with unknown rugged terrain [30]. However, these methods focus on obstacle avoidance and safety, do not consider energy consumption, and are not applicable to global path planning for large off-road terrains. One of the main challenges in path planning for non-flat terrain lies in the accurate modeling of complex terrain features and the design of effective heuristic functions to guide the search process. Since traditional heuristic algorithms have a limited ability to represent intricate environmental features, it is often difficult to handle non-flat terrain. In this paper, the passability, tradeoff distance, and energy consumption are taken into account. We address this limitation by utilizing the powerful inference capabilities of DRL to perform path planning on rugged terrain. In this paper, a rough-terrain path-planning algorithm called A-DQN is proposed. Compared with traditional search-based global-path-planning algorithms such as A*, the advantages of A-DQN are that it eliminates the need for heuristic function design on complex terrain, avoids overestimating or underestimating costs, and utilizes the network's reasoning ability, resulting in much shorter decision times than those of traditional search-based global-path-planning algorithms. In addition, during the training period, we used a hybrid training strategy to accelerate the training process. Our main contributions are as follows:  
* We consider an unmanned vehicle as an agent and construct a model including states, actions, rewards, and strategies;
* We construct an energy consumption model about the slope and consider the energy consumption and the constraints of different classified terrains through a reward function;
* We used a hybrid exploration strategy combining A* and stochastic exploration to accelerate the training process. The effectiveness of the algorithm was verified by simulation analysis. The rest of the sections of our paper are organized as follows. In the second section, the energy consumption model suitable for the agent in this paper is constructed according to the force analysis of the vehicle on the slope. The third section introduces the theory of the DQN algorithm and then introduces our A-DQN algorithm and its applicability on rough terrain for the research problem in this paper.