## 4 Experimental Results and Analysis
### Experimental Results and Analysis of TCB-DQN
#### 4.2.2 Experimental Results and Analysis of TCB-DQN Robustness Verification  
The training results are shown in Figure 16. It can be seen from the figure that when the training reached about 250 times, the score fluctuated around 20 points, indicating that the LHD machine had learned how to reach the middle stage point through the first intersection. When it reached about 1800 times, the LHD machine had successfully learned how to pass the second intersection. When it reached about 2000 times, the score of the LHD machine tended to converge.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Episode**} & \multicolumn{4}{c}{**Operation Steps**} & \multirow{2}{*}{**Score**} \\ \cline{2-2} \cline{4-5}  & **Step 1** & **Step 2** & **Step 3** & **Step 4** \\ \hline Manual & \(\rightarrow\) & \(\rightarrow\) & \(\rightarrow\) & / & / \\
1 & \(\rightarrow\) & \(\rightarrow\) & \(\rightarrow\) & / & 16 \\
2 & \(\rightarrow\) & \(\rightarrow\) & \(\rightarrow\) & / & 16 \\
3 & \(\rightarrow\) & \(\rightarrow\) & \(\rightarrow\) & / & 16 \\
4 & \(\rightarrow\) & \(\rightarrow\) & \(\rightarrow\) & / & 16 \\
5 & \(\rightarrow\) & \(\rightarrow\) & \(\rightarrow\) & / & 16 \\
6 & \(\rightarrow\) & \(\rightarrow\) & \(\leftarrow\) & \(\rightarrow\) & 15 \\
7 & \(\rightarrow\) & \(\rightarrow\) & \(\rightarrow\) & / & 16 \\
8 & \(\rightarrow\) & \(\rightarrow\) & \(\rightarrow\) & / & 16 \\
9 & \(\rightarrow\) & \(\rightarrow\) & \(\rightarrow\) & / & 16 \\
10 & \(\rightarrow\) & \(\rightarrow\) & \(\rightarrow\) & / & 16 \\ \hline \hline \end{tabular}
\end{table}
Table 7: The actions and scores of the LHD machine in the final stage of turning based on TCB-DQN.  
Figure 15: The TCB-DQN training results.  
Figure 17 shows the changes in the LHD machine's running track during the whole training process. At first, the LHD machine explored in the direction of the first intersection. After exploration, it was found that only turning right can reach the middle stage point. Then, after passing through the straight tunnel using the control strategy, the LHD machine explored the direction of the second intersection. Due to the large curvature of the second intersection, the LHD machine explored the turning mode many times. Finally, it could determine a route to keep the score stable. At the same time, the DQN algorithm combined with the conditions of the Equation (10). was used to train the LHD machine, but 1200 training iterations could not make the LHD machine successfully reach the second intersection, which shows that TCB-DQN could effectively reduce the ineffective exploration action of the DQN algorithm, and the training efficiency was significantly better than the DQN algorithm. It could still realize relatively stable autonomous walking of the LHD machine under complex conditions. The algorithm has strong robustness.  
Figure 16: TCB-DQN training results under complex tunnel conditions.  
Figure 17: Different stages of TCB-DQN training, the blue line indicates that the LHD machine is controlled by the traditional control algorithm, and the yellow line indicates that the LHD is controlled by the DQN algorithm. (**a**) LHD machine makes a right turn exploration at the first turning point; (**b**) LHD machine makes a left turn exploration at the first turning point; (**c**) LHD machine makes a right turn exploration at the second turning point; (**d**) LHD machine makes a left turn exploration at the second turning point; (**e**) LHD machine turned late and hit the tunnel; (**f**) LHD machine explore a method for turning; (**g**) although the LHD machine reached the endpoint successfully, the turn at the second turning point was not smooth enough; (**h**) LHD machine final training track.