# Deep Q-Network Algorithm-Based Cyclic Air Braking Strategy for Heavy-Haul Trains
## 3 DQN Control Algorithm
### DQN Algorithm Model
#### 3.2.4 Action Value Function Design
Figure 7 shows that the DQN agent in the experiment learned a safe and feasible control strategy to ensure the safe operation of heavy-haul trains. It can be seen that after training, the algorithm is able to control the train speed by applying the air brakes before the train reaches the maximum speed of 80 km/h under the premise of a slope speed of 68 km/h, and by releasing the air brakes before the speed drops below the minimum release speed of 40 km/h after braking. This is done to ensure the train can maintain safe operation until it exits the long downhill section. #### 4.2.3 Comparison of Algorithm Performance

To prove the superiority of the DQN algorithm in dealing with the cyclic braking of a heavy-haul train running on a long and steep downhill section, the optimization results with the above settings were compared with those of the Q-learning algorithm [10]. The key parameters of the Q-learning algorithm were as follows: the learning rate \(\lambda\) was 0.05, the

Figure 6: Output of brake command during braking: (**a**) output of air braking; (**b**) output of electric braking. Figure 7: Control Strategy for Running Speed of The Heavy-Haul Train. maximum number of iterations \(M\) was 100,000, the discount rate \(\gamma\) was 0.95, the acquisition probability \(\varepsilon\) was 0.1, and the state transition interval \(\Delta t\) was 10. It can be seen from Table 4 that compared with the results of the Q-learning algorithm from [10], the proposed algorithm based on DQN is superior in terms of braking distance, braking efficiency and operation efficiency during the cyclic braking of a heavy-haul train running on a long and steep downhill section. The safe operation of the train is, thus, effectively ensured. From Table 4, it can be observed that under the same conditions and with the same hyperparameter settings, the DQN algorithm, which is focused on air braking distance, actual running time and the train's average speed, shows superiority in braking distance and running efficiency when heavy-haul trains perform cyclic air braking on long downhill sections. Our findings reveal that the DQN-based intelligent control strategy decreased the air braking distance by 2.1% and enhanced the overall average speed by over 7%. These results unequivocally demonstrate the efficacy and superiority of the DQN-based intelligent control strategy. Our experimental results indicate that the DQN algorithm proposed in this paper effectively ensures the safety of train operation. ## 5 Conclusions

In this article, we explore the optimal working condition (braking and braking release) transition during cyclic braking when a heavy-haul train is running on a long and steep downhill section. Aiming at obtaining the shortest air brake distance and the highest operating efficiency, various constraints of the actual operation are considered at the same time, including factors such as the air-refilling time of the auxiliary reservoir, operating speed and operation action switch. The main conclusions are as follows:

(1) To achieve the optimization of multiple objects, a mathematical model is established for a heavy-haul train running on a long and steep downhill section. An intelligent cyclic braking system design based on the DQN algorithm is introduced for the train to adapt to a variety of complex operation environments and line conditions. To improve the convergence speed of the algorithm, the priority experience replay mechanism is used instead of ordinary experience replay. By prioritizing experiences, the agent can choose experiences with a higher priority for learning. In this way, the agent can more quickly learn important experiences that are more conducive to the rapid convergence of the algorithm. As a result, it improves the performance of the control algorithm. (2) To verify the performance of the proposed DQN algorithm, comparative simulations were carried out and tested with different parameters.