# Real-Time Adjustment Method for Metro Systems with Train Delays Based on Improved Q-Learning
## 6 Conclusions  
In this study, an optimization model is established, and an improved Q-learning algorithm is proposed to address the TTR problem. The Shenzhen Metro is used as a case study, and the results demonstrate that satisfactory solutions can be achieved within a short time. The proposed approach effectively meets the real-time timetable adjustment requirements, showcasing its ability to quickly reschedule train timetables in response to actual metro operation delays. The main conclusions of this study are as follows:  
1. The improved Q-learning algorithm exhibited a stable convergence and rapid computation speed. Compared to no adjustment, manual adjustment, and FIFO methods, it  
Figure 13: Two scenarios of train delays in the case of Shenzhen Metro Line 1. (**a**) Scenario I: the 2 nd train experienced a delay of 300 s at the 19 th physical station. (**b**) Scenario II: the 9 th train experienced a delay of 300 s at the 18 th physical station, and the 11 th train experienced a delay of 200 s at the 16 th physical station.  
achieved average reductions of approximately 39.09%, 50.16%, and 71.82% in total train delay, respectively. In comparison to the traditional Q-learning algorithm, the average computation time decreased by around 55.89%, and the average iteration count decreased by approximately 65.22%.
2. The optimized method resulted in a more evenly rescheduled train timetable and aimed to minimize the total delay time caused by train delays. It effectively coordinated the detention of subsequent trains and the adjustment of section running times, ensuring the smooth operation of subsequent trains.
3. The transferability of the algorithm proposed in this paper is verified through the case study of Shenzhen Metro Line 1. This method demonstrates real-time adjustment capability and promotes the utilization of pre-trained Q-value table in various scenarios.  
In our future research, we will conduct experimental analyses on different metro systems to enhance the robustness and generalizability of the proposed algorithm. Additionally, we will explore the incorporation of more complex state variables to further improve the accuracy of our model. To address the challenge of dealing with large-scale state variables, we will consider utilizing deep reinforcement learning algorithms, such as deep Q-networks (DQN).  
The following supporting information can be downloaded at [[https://www.mdpi.com/article/10.3390/app14041552/s1](https://www.mdpi.com/article/10.3390/app14041552/s1)]([https://www.mdpi.com/article/10.3390/app14041552/s1](https://www.mdpi.com/article/10.3390/app14041552/s1)).  
The authors confirm contribution to the paper as follows: study conception and design, data collection, analysis and interpretation of results, draft manuscript preparation: Y.H.; study conception and design, analysis and interpretation of results, funding acquisition: W.L. and Q.L. All authors have read and agreed to the published version of the manuscript.  
This paper is supported by the National Social Science Fund of China (20 BGL301).  
Not applicable.  
The data presented in this study are available in the article and Supplementary Materials.  
The authors declare no conflicts of interest.