# Real-Time Adjustment Method for Metro Systems with Train Delays Based on Improved Q-Learning
## 5 Case Study
### Results Analysis  
The evaluation of the algorithm's performance is depicted in Figure 11a,b, which illustrate the total rewards achieved after 400 and 500 iterations, respectively. In the first scenario, convergence is observed after approximately 200 iterations, while in the second scenario, convergence is achieved after around 300 iterations. These results demonstrate the algorithm's ability to rapidly converge and maintain high stability once convergence is reached. Such characteristics highlight the algorithm's robustness and fast convergence properties.  
The study compares the total delay values obtained from the proposed algorithm with those of the FIFO algorithm, as depicted in Figure 12a,b. In the first scenario, after approximately 200 iterations, the total delay value obtained from the proposed algorithm is notably lower than that of the FIFO algorithm. Similarly, in the second delay scenario, the proposed algorithm outperforms the FIFO algorithm in less than 250 iterations, again leading to lower total delay values. The limitations of the FIFO algorithm become apparent when faced with multiple delayed trains. In contrast, the proposed algorithm's ability to  
Figure 11: The total reward value when delays occur in the two scenarios of Shenzhen Metro Line 14. (**a**) Scenario I: the second train experienced a delay of 200 s at the sixth physical station. (**b**) Scenario II: the second train experienced a delay of 200 s at the sixth physical station, and the fourth train experienced a delay of 200 s at the ninth physical station.  
dynamically adjust the train timetable and consider operational constraints allows it to effectively minimize delays and achieve superior performance.  
To further analyze the advantages of the algorithm proposed in this paper, a comparative analysis was conducted among five approaches: no adjustments, manual adjustment, FIFO, traditional Q-learning, and the improved Q-learning proposed in this study, as shown in Table 4. In the first delay scenario, the number of affected trains for each approach was as follows: 5, 5, 3, 2, and 2, respectively. Regarding the total delay time, the proposed algorithm reduces it from 885 to 256, approximately 71% compared to no adjustments, from 400 to 256, approximately 36% compared to the FIFO algorithm, and from 510 to 256, approximately 49.8% compared to the manual adjustment method. In the second scenario, the number of affected trains for each approach was as follows: 10, 11, 6, 4, and 4, respectively. Regarding the total delay time, the reductions are approximately 72.57% (1750 to 480) compared to no adjustments, approximately 42.17% (830 to 480) compared to the FIFO algorithm, and approximately 50.52% (970 to 480) compared to the manual adjustment method.  
Figure 12: The total delay values in the two scenarios on Shenzhen Metro Line 14 when delays occur using Q-learning (QL) and FIFO strategies. (**a**) Scenario I: the second train experienced a delay of 200 s at the sixth physical station. (**b**) Scenario II: the second train experienced a delay of 200 s at the sixth physical station, and the fourth train experienced a delay of 200 s at the ninth physical station.  
Furthermore, compared to the traditional Q-learning algorithm utilizing the \(\varepsilon\)-greedy strategy (\(\varepsilon\) = 0.6), the improved algorithm demonstrates notable improvements. In both delay scenarios, the average computation time decreases by approximately 56% and 55.77%, while the average convergence iterations decrease by approximately 62.93% and 67.5%, respectively.