###### Abstract  
The autonomous walking of an underground load-haul-dump (LHD) machine is a current research hotspot. The route of an underground LHD machine is generally definite, and most research is based on the logic of positioning-decision control. Based on a reinforcement learning algorithm, a new autonomous walking training algorithm, Traditional Control Based DQN (TCB-DQN), combining the methods of traditional reflective navigation and reinforcement learning deep q-networks (DQN), is proposed. Compared with the logic of location-decision control, TCB-DQN does not require accurate positioning, but only determines how to reach the endpoint by sensing the distance from the endpoint. Through experimental verification, after using the TCB-DQN algorithm for training in a simple tunnel, the LHD machine could achieve a walking effect similar to that of a human driver's manual operation, while after training in a more complex tunnel, the TCB-DQN algorithm could reach the endpoint smoothly. virtual simulation; reinforcement learning; LHD machine control +
Footnote †: journal: Applied sciences  
vehicle can be divided into two degrees of freedom [6], three degrees of freedom [7], four degrees of freedom [8], and multiple degrees of freedom [9; 10]. Theoretically, four degrees of freedom can meet the needs of driving force distribution control and autonomous driving control. [PERSON] [11], of Beijing University of science and technology, optimized a model based on the classical articulated vehicle kinematics model [12] proposed by [PERSON] et al., and regarded the articulated vehicle as a rigid body, ignoring the tire slip, and established a set of dynamic models suitable for a four-wheel independent drive articulated vehicle. The research on LHD machine autonomous walking mainly focuses on the research of control algorithms, such as PID (proportion integral differential), LQR (linear quadratic regulator), SMC (sliding mode control), MPC (model predictive control), etc. [PERSON] [13] et al., based on the SMC [14] algorithm and inspired by the research of SSV (skid-steer vehicles) [15; 16; 17; 18] based on FLC (fuzzy logic control), integrated FLC [19; 20] with SMC to solve the jitter problem often encountered in traditional sliding module control. [PERSON] [21], based on the PID algorithm and the reactive navigation technology of \"along the wall\", designed an autonomous navigation bivariate PID controller to realize the autonomous navigation and driving of a LHD machine. [PERSON] [22; 23] and others established a lateral control algorithm based on MPC control for a complete kinematic model of an articulated vehicle. Although the vehicle cannot reach the speed of manual operation, it can stably realize the autonomous walking of the LHD machine. [PERSON] [24] applied the QPSO (quantum-behaved particle swarm optimization) algorithm to the selection of LQR control weighting matrix parameters and proposed the LQR-QPSO algorithm, which greatly shortens the time for correcting the body attitude of an underground LHD machine in case of abnormal conditions. Benefiting from the rapid development of computer performance in recent years, machine learning has been widely used in the field of vehicle control [25; 26; 27]. [PERSON] [28; 29] used a feedforward neural network to track the trajectory of intelligent vehicles and achieved good experimental results. [PERSON] et al. [30; 31; 32; 33] proposed to use of the reinforcement learning method to adjust PID control parameters in real-time, and applied this to the autonomous walking of an articulated vehicle. Experiments showed that, under the support of reinforcement learning, a PID controller can effectively reduce vibration. Although these control algorithms can achieve good results, this depends on the artificially given route, and the use of a reinforcement learning algorithm will enable the LHD to explore a route by itself; so as to realize the real sense of autonomous walking.