# Real-Time Adjustment Method for Metro Systems with Train Delays Based on Improved Q-Learning
## 3 Model Construction
### Constraint Conditions  
1. Train Headway Constraint  
In metro operations with train delays, trains operating at blocks necessitate a specific distance between them to ensure safety. As a result, there are minimum constraints on the train headways between two adjacent trains, as expressed in Equations (2) and (3).  
\[\overline{G}_{k-1,i}^{arr}-\overline{G}_{k,i}^{arr}\geq U_{\min},\forall k\in K,i\in I \tag{2}\]  
\[\overline{G}_{k-1,i}^{dep}-\overline{G}_{k,i}^{dep}\geq U_{\min},\forall k\in K,i\in I \tag{3}\]
2. Block Running Time Constraint  
According to the analysis in Section 2, the train's running time within a block can be divided into several levels, referred to as block running time levels, denoted by \(\mu_{k,i}^{m}\). In this model, Boolean variables, denoted by \(\lambda_{k,i}^{m}\), are introduced to represent the chosen running time level for the current block. The constraints on the train's running time in each block can be expressed using Equations (4) and (5), where Equation (4) states that only one block running time level can be selected, and Equation (5) limits the train's running time within a block to the specified time corresponding to a particular block running time level.  
\[\overline{G}_{k-1,i}^{arr}-\overline{G}_{k,i}^{dep}=\sum_{m\in M}\lambda_{k,i} ^{m}\mu_{k,i}^{m},\forall k\in K,i\in I \tag{4}\]  
\[\sum_{m\in M}\lambda_{k,i}^{m}=1 \tag{5}\]  
When it comes to train delays, the train operation mode is downgraded to a fixed-block system under emergency situations. According to Figure 4, if the distance between two trains is within one block, the following train must come to a complete stop. If the distance is within two blocks, the train operates at a reduced speed using the slowest running level. If the distance is three blocks or more, the train operates normally. Therefore, Equation (5) can be further expressed as Equation (6). The block in which train k is located can be represented as \(B_{k}\), which can be represented by station, such as station \(i\) to \(i+1\).  
\[\begin{cases}\lambda_{k,i}^{m}=0,\forall m\in M\\ \lambda_{k,i}^{1}=1,\sum\limits_{\forall m>1}\lambda_{k,i}^{m}=0&\begin{array} []{l}B_{k-1}-B_{k}\leq 1\\ B_{k-1}-B_{k}\leq 2\end{array},\forall k\in K\\ \sum\limits_{m\in M}\lambda_{k,i}^{m}=1&\begin{array}{l}B_{k-1}-B_{k}>2 \end{array},\forall k\in K\end{cases} \tag{6}\]
3. Station Dwelling Time Constraint  
The minimum station dwelling time constraint for trains is defined by Equation (7), where \(S_{\text{min}}\) represents the minimum station dwelling time. It should be noted that the constraint is applicable only to physical stations. This time is determined based on factors such as the train door opening and closing times, and passenger boarding and alighting times, as expressed in Equation (8). Figure 5 provides a visual representation of the construction of the minimum station dwelling time. In metro systems, the door opening and closing time of the train are usually constant values. The passenger boarding time, on the other hand, is determined by the passenger demand on the platforms. If the number of passengers exceeds 90% of the station's capacity, the passenger boarding time is increased accordingly [48]. Otherwise, it remains at the minimum passenger boarding time, as depicted in Equation (9).  
\[\overline{G}_{k,i}^{dep}-\overline{G}_{k,i}^{arr}\geq S_{\text{min}},\forall k \in K,i\in I^{\prime} \tag{7}\]  
\[S_{\text{min}}=t_{\text{open}}+t_{\text{close}}+t_{\text{on}}+t_{\text{off}} \tag{8}\]  
\[t_{\text{on}}=\begin{cases}t_{\text{min}}+t_{\text{add}}\times(n_{\text{aboard }}\div n_{\text{max}})&if(n_{\text{aboard}}\div n_{\text{max}})\geq 90\%\\ t_{\text{min}}&if(n_{\text{aboard}}\div n_{\text{max}})<90\%\end{cases} \tag{9}\]  
4. Turnaround Time Constraint  
The turnaround time for trains at terminal stations is defined in Equation (10).  
\[\overline{G}_{k,|I|+1}^{arr}-\overline{G}_{k,|I|}^{dep}\geq T_{\text{turn}}^{ \text{min}},\forall k\in K \tag{10}\]