# Traffic Signal Time Optimization Based on Deep Q-Network
## 4 Experimental Studies
### Experimental Results  
The experimental results were evaluated, where the demand fluctuation follows a Poisson distribution. For the experimental results, the performance of the middle of the nine intersections was used. To test the performance of the proposed algorithm, we compared its results with those of other algorithms, namely, TFP-CTSC [21] and QT-CDQN [13]. TFP-CTSC considers the waiting time as a reward parameter, whereas QT-CDQN considers the queue length as a reward parameter. Both techniques under comparison also considered the surrounding intersections. Figure 9 shows the rate of green signal waste of the middle of the nine intersections under the experimental environment. For the Poisson distribution, whereas TFP-CTSC and QT-CDQN wasted approximately 22% to 23% of the green signals, the proposed algorithm wasted only approximately 13% of them. The proposed algorithm wasted approximately 40% less green signal time than those of the other algorithms. This is because, although both TFP-CTSC and QT-CDQN were distributed in order of green time according to the situation, the time was fixed. However, the proposed algorithm efficiently adjusted the green signal time to suit the situation. The performance results were the average queue length, standard deviation of the queue length, average waiting time, standard deviation of the waiting time, and capacity. Figure 10 shows the change in the capacity of the intersection over time. As shown in the figure, the proposed algorithm handled approximately 40% more vehicles than those handled by TFP-CTSC and 30% more vehicles than those handled by QT-CDQN. The proposed algorithm handled more vehicles than those handled by the two other algorithms because it considers capacity as a reward parameter. In addition, the green signal time was efficiently used as the proposed algorithm considered the circumstances in the surrounding intersections. The efficient use of green signals also increased the capacity of the intersections. Figure 8: Structure of the environment for nine interconnected intersections. Figure 11a shows the performance results of the vehicle waiting time according to the traffic load, and Figure 11b shows the results of the standard deviation of the waiting time. The performances of the waiting times of the TFP-CTSC and the proposed algorithm are lower than that of the QT-CDQN when the traffic load is less than 70%. This is because the TFP-CTSC considers the waiting time as a reward parameter. However, as the traffic load increases, the performance of the proposed algorithm is better than that of the two other algorithms. This is because each lane was given an appropriate green signal time, considering the standard deviation of the waiting times of the vehicles waiting at the intersection. The standard deviation of the waiting time of the proposed algorithm is approximately 35% shorter than that of TFP-CTSC and approximately 60% shorter than that of QT-CDQN. The proposed algorithm performs well with time-related values because it considers the standard deviation of the waiting time as a reward parameter. This indicates that the appropriate green time was distributed because the standard deviation of the waiting time of each lane in the intersection was considered. Figure 10: Change in the capacity of the intersection over time. Figure 9: The rate of green time wasted. Figure 12a shows the performance results for the queue length according to the traffic load, and Figure 12b shows the standard deviation of the queue length. The queue length is measured on the basis of the number of vehicles waiting at the intersection. In Figure 12a, the proposed algorithm has a queue length that is approximately 50% shorter than that of TFP-CTSC and approximately 45% shorter than that of QT-CDQN on average. Although QT-CDQN considers the queue length as a reward parameter, wasted green signal times occurred because the green time of the traffic signals was fixed. Figure 11: Performance results: (**a**) average waiting time and (**b**) its standard deviation.