# Real-Time Adjustment Method for Metro Systems with Train Delays Based on Improved Q-Learning
## 1 Introduction
### Literature Review
Manual adjustment methods rely on the experience and manual intervention of dispatchers when dealing with train delays. However, these methods are constrained by the formulation and adaptability of manual rules, which makes it difficult to cope with the complexity and variability of actual operational scenarios. Optimization model methods can simulate train operations through complex algorithms, but they often struggle to accurately capture various complex factors in real-world scenarios. Simulation methods attempt to predict via simulating the interactions between trains, but they face computational complexity challenges when dealing with large-scale networked operations, making it difficult to rapidly solve problems with high real-time requirements. Heuristic algorithms are prone to getting stuck in local optima and often require significant computation time. 2. Reinforcement learning has gained considerable attention. Through interactive learning between intelligent agents and the environment, reinforcement learning gradually optimizes decision-making strategies, exhibiting adaptability and intelligence. In the field of metro systems, utilizing reinforcement learning to directly construct solution strategies can better adapt to the complexity and variability of actual operational scenarios. \begin{table}
\begin{tabular}{l l c c} \hline \hline
**Work** & **Problem Setting** & **Training Instances** & **State Definition** \\ \hline [PERSON] et al. [37] & Multiple train delays & Single instance & Train locations, current time and track availability \\ [PERSON] et al. [38] & Multiple train delays & Single instance & Track availability around the train to be controlled \\ [PERSON] et al. [39] & Multiple train delays & Single instance & Delay time, location, and local \\ [PERSON] et al. [40] & Multiple train delays & Single instance & track availability of the current train \\ [PERSON] et al. [41] & Multiple train delays & Single instance & Departure time, whether to stop at stations, and \\  & & & running direction of the current train \\  & & & Actual arrival time and \\  & & & departure time of trains \\ [PERSON] et al. [42] & Single train delays & Multiple instances & Arrival and departure time of \\  & & & trains entering junction \\ [PERSON] et al. [43] & Single train delays & Multiple instances & Planned/actual arrival time and departure time \\ [PERSON] et al. [44] & Single train delays & Multiple instances & Actual arrival time and the number of \\  & & & passengers onboard \\ [PERSON] et al. [45] & Single train delays & Multiple instances & The speed, position, and \\  & & & current driving status of the train \\ This paper & Multiple train delays & Multiple instances & Train delays \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Brief summary of the existing learning-based methods in the TTR problem. Currently, most reinforcement learning methods focus on cases where only a single train is delayed, neglecting scenarios where multiple trains experience simultaneous delays, failing to consider the interactions between trains. Furthermore, existing methods often involve complex state representations, which can lead to inefficient problem-solving processes and challenges in ensuring real-time accuracy and prediction.