# Intelligent Traffic Signal Phase Distribution System Using Deep Q-Network
## 4 Performance Evaluation  
The performance experiment conducted on the proposed intelligent traffic signal control system was conducted at nine intersections having a 4-way structure, as shown in Figure 8. Vehicles can move in two directions on each road. One can turn left, and the other can continue straight or turn right. The duration of the green signal was set to 30 s. This experiment was conducted to simulate urban mobility (SUMO). To evaluate the performance of the proposed algorithm, the model classifies the reward parameters into two categories: the combination of throughput and queue length, and the combination of throughput and standard deviation of the queue length. In addition, the performance is evaluated by comparing fixed signal control as a baseline, which is a traditional traffic signal control technique, and comparing two other RL techniques [8; 16]. One comparison algorithm is referred to as QT-CDQN [8]. QT-CDQN considers the average queue length as a reward parameter to propose an order-allocation technique, such as the proposed algorithm. The other comparison algorithm is referred to as CRL which also uses an order-allocation technique [16]. CRL considers the difference between the sum of queue length of time t and t + 1 as a reward parameter  
The vehicle distribution data used in the experiment were generated using SUMO. Based on the real data of the Seoul area [17], the SUMO simulator was used to generate vehicle data suitable for our experimental environment. The vehicle route data are set to a repetition rate, period, and arrival rate. Then, based on this, the SUMO simulator determines the time and route of the vehicle entering the road. Accordingly, the vehicle is input into the road network according to a preset start time and route. Figure 9 shows the distribution of the number of vehicles entering an intersection on weekdays. The distribution shows a large amount of traffic during rush hour. The experiments were conducted using vehicle distribution. One step is represented as one hour. Figure 8: Structure of the experiment environment: nine intersections. In Figure 10, the performance in terms of average queue length is compared. Step 1 to 5 refer to relatively light traffic load, and step 6 to 10 refer to relatively heavy traffic load. In the case of heavy traffic load, tp + qlstd has an average queue length about 20% lower than QT-CDQN and about 48% lower than CRL. Moreover, tp + qlstd is about 30% lower than tp + avgql. This shows that tp + qlstd appropriately distributes traffic signals in consideration of the standard deviation of the queue length in the intersection. Figure 11 shows the comparison of the average wait times. For heavy traffic load, QT-CDQN has a similar wait time on average with that of tp + qlstd. On the other hand, tp + qlstd has about 20% shorter wait time than that of CRL. As a reward parameter, QT-CDQN applied the average queue length, CRL applied on the sum of queue lengths, and tp + qlstd applied the standard deviation of the queue length. This shows that the same factor, the queue length, was applied, but the performance varies depending on how it is applied. Figure 10: Performance comparison: average queue length. Figure 9: Traffic distribution used in the experiment. Figure 12 and 13 show the performance comparison using the standard deviation of performance indicators. In the case of the standard deviation of the queue length, it was the lowest when the standard deviation of the queue length and throughput were considered as the reward parameters. Considering the standard deviation of the queue length, it is interpreted that the queue lengths of the lanes within the intersection are evenly adjusted by efficiently distributing the green signals. By contrast, in QT-CDQN, the standard deviation of the queue length was approximately 35% higher on average than the standard deviation of the combination of throughput and queue length. In addition, in CRL, the standard deviation of the queue length was also about 16% higher on average than that of tp + qlstd.