# Double Deep Q-Network with a Dual-Agent for Traffic Signal Control
## 5 Results and Discussion
### Experimental Results
Furthermore, comparing to control algorithm with binary action, our control algorithm has a more stable and effective control ability in most cases, especially in high traffic productions. This poor performance is caused by traffic congestion. The Figure 10 (described hereafter) illustrates the sum of vehicles existing in an intersection per second. As shown in the results under the high traffic production (4320 veh/h), the vehicle number in the phase switch algorithm continues to increase, resulting in traffic congestion. On the contrary, the proposed algorithm ensures sustainable vehicle number and preserves unimpeded traffic flow. Figure 9: Comparison performances of different traffic signal control algorithms: (**a**) average waiting time; (**b**) the number of stops; (**c**) the number of arrived vehicles. For testing whether the intersection suffers from congestions, we compare the sum of vehicles that exist in an intersection for every second with different algorithms too. It is noted that these control methods are also trained in the traffic environment with 3600 veh/h traffic production (i.e., ratio \(r=1\)). Under the same traffic production rate in Table 2, the vehicle productions are set to be 2520 veh/h, 3600 veh/h and 4320 veh/h (i.e., ratio \(r=\)0.7, 1.0, and 1.2 separately) for each comparison, which contains 50 test experiments. Figure 10 plots the test curve of each control method, where the solid shows the average existing vehicles per test episode and the shade shows its standard deviation. Since the traffic volume is hard to be saturated under the low traffic production, there are no significant differences among these methods (see Figure 10a). In Figure 10b,c, due to the non-stationary distribution, the phase switch (binary number) method is failed to maintain sustainable vehicle number. As expected, the performance of DQN algorithm is limited by traffic volume of left-turn, particularly in the high traffic production. On the other hand, the proposed algorithm shows the best and the most robust control ability as its test curve keeps stable and has small standard deviation. Further, to verify the feasibility and practicality of the DDQN with dual-agent algorithm, we make a comparison (the same setting as above experiment) in improving the efficiency of the intersection between our algorithm and actuated traffic lights based on time gaps. Its mechanism is to prolong a traffic phase whenever a continuous traffic stream is detected and changes to the next phase after detecting a sufficient time gap between successive vehicles. In this paper, the maximum time gap (i.e., sufficient time gap) is set to be 2 s, which is found to be the optimal value in our simulation environment. To be fair, minimum phase time in the actuated traffic light and yellow light time are  
Figure 10: The number of vehicles existing in the intersection per second: (a) vehicle production 2520 veh/h; (b) vehicle production 3600 veh/h; (c) vehicle production 4320 veh/h. set to be 6 s that is the same as a DDQN with dual-agent algorithm. The comparison results (the ratio \(r\) from 0.5 to 1.2) are shown in Figure 11. The results indicate our algorithm is more efficient to resolve traffic congestion in most cases except that traffic productions is 1800, 2160, 2520, and 2880 veh/h, respectively. There are two possible reasons for this performance. The first is that the neural network is trained in the traffic system with 3600 veh/h traffic production. The second is that the proposed algorithm uses a longer time interval (\(\tau_{p}\), \(\tau_{a}\)) than actuated traffic light. To further prove our viewpoints, we re-train the DRL agent in two different setting. The first is to set the ratio \(r=0.7\) (i.e., 3600 \(\times\) 0.7 = 2520 veh/h traffic production), \(\tau_{p}=6\) s, and \(\tau_{a}=6\) s (referred as \(r=0.7\)\(\tau_{p}=6\)\(\tau_{a}=6\)), the second is to set the ratio \(r=0.7\), \(\tau_{p}=3\) s, and \(\tau_{a}=3\) s (referred as \(r=0.7\)\(\tau_{p}=3\)\(\tau_{a}=3\)).