## 4 Results and Evaluation  
### Performance of DRL, ATSC, and MP-Based Models  
The cumulative negative reward (cumulative waiting time) and delay time were used to evaluate the effectiveness of the DNN model. The simulated results are shown in Figure 10 below. Through the learning process, the cumulative negative reward approached zero, which means that the total waiting time of all vehicles decreased, and the agent learned gradually. Although the reward line fluctuated, its overall trend was upward. At the beginning of training, this curve oscillated with a large amplitude but then gradually decreased. It can be seen that after the training process, the cumulative waiting time of all vehicles decreased by around 70% compared to that before training. The model results show that the cumulative reward was increased from around \(-\)35,000 s, \(-\)28,000 s, and \(-\)20,000 s to around \(-\)9600 s, \(-\)11,200 s, and \(-\)8000 s, respectively, for the cases of platoons consisting of 3, 2, and 1 vehicles. The cumulative waiting time of all vehicles passing through the intersection decreased, so the cumulative delay time also decreased, as expressed in part (b) of Figure 10. This result proved that the model was efficient, and it was implemented in the next steps. In the early episodes, the epsilon rate (in Equation (10)) was high, and the agent did not fully explore the action and state space. Therefore, the total cumulative time of this model was larger than in the ATSC- and MP-based scenarios. As the model continued to learn, the epsilon was gradually reduced, and the agent concentrated on exploiting the optimal policy from the previous episodes for higher rewards. And the models also gradually converged when the reward curve was unchanged. Finally, the total cumulative  
Figure 10: Cumulative negative reward and delay time of DRL, ATSC, and MP-based scenarios with different platoon sizes. waiting time of the DRL-based scenario was 50% and 65% better than that of the MP- and ATST-based scenarios for platoons with three vehicles. However, as the number of episodes increased, the total wait time in the ATSC- and MP-based scenarios did not change significantly because the intersection traffic management in the two scenarios was based on the pre-defined algorithm without \"learning\". These two scenarios have the same characteristics: (a) 4 phases to separate the queue length in all movements; (b) the assumption of queue length with unbounded capacities on all links; (c) the traffic controller without a fixed cycle and cyclic phases. The cumulative waiting time of the MP- and ATSC-based scenarios fluctuated around 25,000 and 32,000 s. The performance difference can be explained through the intersection traffic controller. The algorithm in the MP-based scenario prevented any queue length from growing indefinitely and created a stable intersection with a suitable queue length. The MP-based scenario had better performance than ATST because it gave priority to stabilizing the queue length. This algorithm reduces the pressure at the intersection reasonably, whereas the traffic controller in the ATSC-based scenario manages the intersection based on the sensor's vehicle detection. It prolongs the green time for the high-traffic directions but also increases the queue length in other directions. Therefore, the green time duration in the ATSC method was large than that in the MP method. Finally, it achieves worse results, but the difference between the ATSC- and MP-based scenarios is not large (20%). Another important finding was the influence of traffic flow rates on these models. Vehicles arrived at the intersection independently when traffic flow was low (platoon_size = 1). When CVs arrived, the signal agent quickly changed phase to ensure the minimum waiting time; and for RVs, the traffic signal did nothing. Since all vehicles arrived individually, the agent handled the vehicles separately.