# A Novel Deep Reinforcement Learning Approach to Traffic Signal Control with Connected Vehicles +
## 4 Simulation Results
### Comparison with Baselines
at each time step; (4) average queue time, which is calculated by dividing the total waiting time of vehicles caused by a queue by the total number of lanes at each time step and offers a similar perspective as average waiting time, but with a focus on traffic congestion; (5) average vehicle density, which is the average number of vehicles in each lane at each time step and represents the density of vehicles approaching the intersection. The performance comparisons of our proposed method with the baseline methods is shown in Figure 10. These metrics were obtained through 100 repetitions of statistical testing using the same vehicle initialization file. Each data point represents the traffic state at a single time step of the simulations and the median values are indicated by the white labels in the middle of the boxes. As can be seen from this figure, the wide distribution of the average vehicle speed and the average vehicle waiting time is due to the stochastic nature of the traffic flow, while the lower median values indicate that our DRL-based controller can acquire the signal control policies that may prioritize the overall traffic performance over the performance of individual vehicles, which is also verified in Figure 13 that will be discussed later. However, there is room for further improvement of the control policies by tuning the learning algorithm, such as by adding a maximum constraint for the vehicle's waiting time, which will be explored in future research. The performance of the proposed DRL-based control method is also demonstrated in Figure 11 through the five common traffic mobility metrics considered, each with its  
Figure 10: Performance comparisons with baselines. average value and confidence interval in a time series. Comparing Figure 11a-e, it can be seen that the curves have similar trends, implying that these metrics are interconnected. For example, longer queues lead to longer average queue time, slower average vehicle speed, and slower vehicle discharge. Specifically, Figure 11a shows that the proposed method had a much shorter average queue length than the three baselines. The curve of our proposed method increased at the beginning and then decreased sharply at the end, whereas the other methods did not follow this trend, demonstrating the effectiveness of the proposed controller in recovering from saturation and clearing the intersection quickly when the traffic volume decreases. By contrast, the other methods could not fully recover from congestion within the simulation time. Regarding the average queue time in Figure 11b, our proposed DRL method resulted in smaller queue times than the baseline DRL method while there seems to have been no obvious improvement over the fixed-timing and actuated methods. This is due to the acquired asymmetric policy, which sacrificed traffic flow in the north and south approaches to maximize the overall performance. If we continue to examine Figure 11c-e, we can see that our developed DRL method can lead to slightly better results (e.g., higher average vehicle speed, lower average vehicle density, and shorter average vehicle waiting time) than the three baseline methods under the simulation scenario considered in this work. Figure 11: Simulation comparisons with baselines. To better understand the reason behind the wide distribution of the mobility metrics simulated by the proposed method, the lane-wise simulation data of our proposed DRL method is plotted in Figures 12 and 13. The comparison of the average queue length and queue time of each lane reveals that the lanes in the north and south approaches have similar curves with lower values than the west and east lanes. This suggests that our DRL-based controller has adopted an asymmetric traffic flow policy, even though the traffic volumes are balanced, to optimize the performance of the entire intersection. Figure 12 further confirms that the significant deviations observed in the green boxes of Figure 10 are primarily due to the asymmetric traffic control policy.