# Deep Q-Network Algorithm-Based Cyclic Air Braking Strategy for Heavy-Haul Trains
## 2 Model Construction for Heavy-Haul Trains  
### Dynamic Model  
During the operation of a heavy-haul train, factors such as track gradient, train composition and on-board mass cause the train to be subjected to diverse forces. In this study, the interaction forces between the cars are not taken into account when calculating the additional resistance. Therefore, the forces imposed on the train during operation mainly include locomotive traction force, braking force (including electric braking and pneumatic braking), basic running resistance and additional resistance. Essentially, a heavy-haul train is a distributed power network system consisting of multiple locomotives and freight cars. According to the Newtonian principle of dynamics, the mathematical expression of each train model can be defined as follows [17]:  
\[M\dot{v}=F-U_{1}-U_{2}-F_{R}, \tag{1}\]  
Usually, the running resistance \(F_{R}\) encountered by a heavy-haul train during braking on a long and steep downhill section is mainly composed of basic resistance \(M_{R}\) and additional resistance \(L_{R}\). These resistances depend on the operating speed of the heavy-haul train as well as its physical characteristics [18]. The running resistance is calculated as follows:  
\[F_{R}=M_{R}+L_{R}, \tag{2}\]  
According to previous research, the calculation formula of the basic resistance for a heavy-haul train is as follows [19]:  
\[M_{R}=M(\varphi_{1}+\varphi_{2}v+\varphi_{3}v^{2}), \tag{3}\]The additional resistance is determined by the section force \(g_{R}\), the curvature resistance \(c_{R}\) and the tunnel resistance \(t_{R}\)[19], as shown in Equation (4). The specific calculations of these factors [20] are presented in Equation (5):  
\[L_{R}=g_{R}+c_{R}+t_{R}, \tag{4}\]  
\[ \begin{cases}g_{R}=Mg\sin(\arctan\frac{i}{1000})\\ c_{R}=600/R\\ t_{R}=0.00013L_{s}\end{cases} , \tag{5}\]  
For heavy-haul trains, the control inputs for the locomotives include the traction force and braking force, whereas the input for freight trains includes only the braking force. There are mainly two types of brake equipment for heavy-haul trains: one is rheostatic brake equipment and the other is pneumatic braking equipment. Rheostatic brake equipment, also known as the regenerative brake, can feed back energy to other locomotives to provide power. A pneumatic braking system achieves braking force by reducing the air pressure in the train's air braking pipes [21]. The traction force \(F\) of a heavy-haul train depends on the relative output ratio \(h^{tr}\) of the maximum traction force \(u^{tr}_{max}(v)\), while the electric braking force \(U_{1}\) is determined by the maximum electric braking force \(u^{d}_{max}(v)\) and its relative output ratio \(h^{d}\). Therefore, the traction force and electric braking force of a train can be calculated according to Equation (6):  
\[ \begin{cases}F=F^{tr}(u^{tr}_{max}(v),h^{tr})\\ U_{1}=F^{d}(u^{tr}_{max}(v),h^{d})\end{cases} , \tag{6}\]  
where \(u^{tr}_{max}(v)\) and \(u^{d}_{max}(v)\) are the piecewise functions of the train's running speed [22]. In addition, it is impossible for each locomotive to output electric braking force and traction force at the same time [23]; thus, the relative output ratio of \(h^{d}\) and \(h^{tr}\) is \(h^{d}\times h^{tr}=0\). The air braking system of a heavy-haul train is its main braking force and the key to ensuring the safety of train operation. Figure 1 is a diagram showing the structural composition of the air braking system of a heavy-haul train [24]. According to Reference [25], \(U_{2}\) in Equation (1) can be calculated based on Equation (7):  
\[U_{2}=\theta_{b}\times\varphi_{b}\times\beta_{s}\times 1000, \tag{7}\]  
where \(\theta_{b}\) and \(\varphi_{b}\) are intimately associated with the train's physical characteristics, such as the brake lever and transmission efficiency of locomotives and freight cars, while the service braking coefficient \(\beta_{s}\) is assigned a value based on the rated pressure \(p\) of the air pipes and the corresponding air pressure drop \(\Delta p\).