## 4 Experiment and Results
### Training Results  
Using peak hour traffic data (4576 pcu/h) and the collected real traffic flow vehicle turn ratios, 10 sets of test traffic flows were randomly generated. The validity of the proposed signal control scheme was confirmed by assessing the average CO\({}_{2}\) emissions and vehicle waiting times throughout the simulation period. Figure 8 shows a comparison of average vehicle CO\({}_{2}\) emissions and average waiting times. Min-Max normalization was applied to both metrics, and the results are presented as a heatmap. Each ring in the heatmap represents the optimization effectiveness of a signal control scheme, with lighter colors indicating better optimization results. Figure 8a shows the results for average vehicle waiting time. Compared to traditional signal control methods and actual signal timing, the DRL-based signal control methods provided better optimization for average vehicle waiting time at intersections. Among these, the Rainbow DQN_AM-CO2 Reduction scheme performed the best, only slightly outperformed by the Rainbow DQN_AM_Time scheme in the second round of testing. Figure 8b displays the optimization effects on average vehicle CO\({}_{2}\) emissions. Similar to the results in the left figure, deep reinforcement learning control schemes generally outperformed traditional signal control algorithms across all test rounds. Furthermore, the Rainbow DQN_AM-CO2 Reduction scheme consistently showed the best CO\({}_{2}\) reduction effects across all 10 test rounds. The results indicate that the two schemes using the Rainbow DQN_AM algorithm outperform the deep reinforcement learning schemes using D3 QN and DQN. Figure 7: The average reward of the training process. A statistical significance analysis was conducted on the results for two indicators: average vehicle waiting time and average CO\({}_{2}\) emissions across different models. The Rainbow DQN_AM-CO2 Reduction algorithm represents the optimized result, while the others serve as control groups. Data from 10 sets of traffic flow were used to perform an analysis of variance (ANOVA) to compare the indicator values from the control groups and the Rainbow DQN_AM-CO2 Reduction algorithm. The results, shown in Table 7, demonstrate that there are significant differences between the control group models and the proposed model in both average waiting time and average CO\({}_{2}\) emissions, with \(p\)-values less than 0.05. This indicates that the optimization effects of the model are statistically significant and representative. To analyze the optimization degree of different schemes and the model's robustness under real traffic conditions, box plots of the average vehicle waiting time and average CO\({}_{2}\) emissions are shown in Figure 9. Figure 9 shows that the average waiting time for vehicles for the Rainbow DQN_AM-CO2 Reduction signal control scheme is 19.30 s. This represents reductions in 7.48%, 14.50%, and 27.58% compared with the Rainbow DQN_AM-Time scheme, the D3 QN-CO2 Reduction scheme, and the actuated signal timing scheme, respectively. Compared to the actual timing scheme, the reduction is as high as 57.95%. As shown in Figure 9, the average CO\({}_{2}\) emissions per vehicle for the Rainbow DQN_AM-CO2 Reduction scheme are only 190.33 g, which is 4.37%, 6.40%, and 7.34% lower than the Rainbow DQN_AM-Time scheme, the D3 QN-CO2 Reduction scheme, and the actuated signal timing scheme, respectively. In a scenario where electric vehicles account for 29% of the traffic, the average CO\({}_{2}\) emissions are optimized to a lesser extent than the \begin{table}
\begin{tabular}{c c c} \hline \hline
**Control Method** & **Average Waiting Time** & **Average CO\({}_{2}\) Emissions** \\  & _p-Value_ & _p-Value_ \\ \hline Actual signal timing & 0.002 & 0.000 \\ Webster signal timing & 0.000 & 0.000 \\ Actuated signal timing & 0.000 & 0.000 \\ Rainbow DQN\_AM-Time & 0.003 & 0.000 \\ DQN-CO2 Reduction & 0.000 & 0.000 \\ D3 QN-CO2 Reduction & 0.000 & 0.000 \\ \hline \hline \end{tabular}
\end{table}