## 4 Case Study
### Evaluation of Multi-Train Control Effect  
To test and verify the reliability and performance of the DRLDA algorithm, different cases for simulations were made. The training process of DDPG-RS and DQN-RS ([PERSON], Zhou and Fujita [57]) was compared with the proposed method in an ordinary scenario. According to the relationship complexities among states, actions and evaluation indexes, Neural Network structures of different depths can be employed. Since the states and actions are multi-dimensional for the problem of train energy-saving control, the Q networks, actor networks and critic networks are composed of several fully connected hidden layers, with some neurons for the fully connected hidden layers, respectively. The output layer of the actor networks and critic networks have only one neuron in continuous action space ([PERSON] et al. [58]; [PERSON] et al. [59]; [PERSON] et al. [60]). The related parameters for training are shown in Table 7. If the termination condition described in the DRLDA strategy were triggered during the training, the current single training would be stopped. The train would be reset to the initial state to start a new training session. The levels were discretized into eleven different control levels [100%, \(-\)80%, \(-\)60%, \(-\)40%, \(-\)20%, 0%, 20%, 40%, 60%, 80%, 100%]. The higher the number of discretions, the more difficult the training. After adding noise in the exploration process, three different algorithms were selected as the closest control levels. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Index** & **Xinjiekou/Pinganli** & **Pinganli/Xisi** & **Xisi/Lingjinghutong** \\  & **Station** & **Station** & **Station** \\ \hline Actual average passenger waiting time [s] & 105.00 & 102.00 & 93.00 \\ Calculated average passenger waiting time [s] & 103.00 & 101.80 & 96.00 \\ Actual energy consumption [kW\(\cdot\)h] & 14.67 & 14.28 & 13.64 \\ Calculated energy consumption [kW\(\cdot\)h] & 13.85 & 13.94 & 12.88 \\ Energy efficiency [\%] & 5.58 & 2.38 & 5.57 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Comparison of optimization results. Figure 19: Train parking interface. As shown in Figure 20, the average reward value of the variety process curve is shown for the training of three different algorithms. Figure 20 demonstrates that the average reward values of three different algorithms tend to be stable after rising. The DRLDA algorithm more easily learns the action space, so the training process is more stable. The average episode reward value is stable at about \(-135\), with a small amplitude of vibration. The average reward value per 200 episodes for DRLDA is generally higher than the reward values for DDPG-RS and DQN-RS in Figure 20. This implies that DA-DRL learns better than these two algorithms. With the increase in training times, the average episode reward values of the DDPG-RS and DQN-RS algorithms have been rising in vibration. The values of these two algorithms began to rise slowly after reaching \(-160\) and \(-200\), respectively. The results, compared with the computer resources demanded by the above three and other alternative approaches, are shown in Table 8. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Parameter** & **DRLDA** & **DDPG-RS** & **DQN-RS** \\ \hline State transition step & 100 & 100 & 100 \\ The batch size \(N\) of transfer samples from replay memory & 32 & 32 & 32 \\ Single training step & 20 & 20 & 20 \\ Actor network learning radio & 0.0001 & 0.001 & – \\ Critic network learning radio & 0.0001 & 0.001 & 0.001 \\ Soft update parameter & 0.0001 & 0.001 & 0.001 \\ Sample data & 125 & 125 & 125 \\ Replay memory capacity & 12,000 & 10,000 & 10,000 \\ Discount factor & 0.99 & 0.8 & 0.9 \\ Line length & 3069 & 3069 & 3069 \\ Other parameters & – & See Ref. [57] & See Ref. [57] \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Training parameters. Figure 20: Training process of three different algorithms.