## 1 Introduction
Other papers also expressed that overly large platoons affect traffic stability, making it difficult to manage platoons (merging and splitting) [27]. The research results also emphasize that the platoon size does not need to be too large, as this value is a trade-off between capacity and traffic stability. In addition to the management of vehicle connections, intersection management policies also greatly affect traffic results. In the last decade, many major cities have changed their intersection management technologies to achieve better traffic efficiency. Instead of using traditional management methods (e.g., fixed time or stop signs), they have applied modern techniques, including adaptive traffic signals or intelligent traffic lights, combining cameras to determine traffic parameters to provide optimal scenarios. An actuated traffic signal controller (ATSC) used its algorithm, sensors, and Boolean logic to create a dynamic signal cycle. When traffic conditions changed, the ATSC automatically adjusted the active phase duration and cycle. The performance of signal light control depended on the optimization technique applied. Over the decades, there has been a great deal of research on signal light controllers based on techniques such as evolutionary algorithms [28; 29], max pressure (MP) [30; 31], self-organization [32], the Sydney coordinated adaptive traffic system (SCATS) [33], the split cycle offset optimization technique (SCOOT) [34] and adaptive control software like (ASC-Lite) [35]. However, most of the algorithms used in signal light controllers consider the same aspects for all modes. Each mode in the traffic volume was treated equally. In addition, these algorithms also did not take full advantage of modern traffic data sources. Therefore, they were difficult to apply to intersections with high complexity. Along with the development of science and technology, the input data of the deep reinforcement learning (DRL) model was also increasing. Deep neural networks (DNN) have been used to solve this complex problem to enhance model efficiency. Besides the traditional traffic management at intersections, reinforcement learning (RL) has also been used to manage adaptive traffic signal lights. In the past, due to technical limitations, models often had a small-sized state and used linear functions to approximate Q values. However, the complexity of the model was not fully demonstrated because of the limited input information. With the development of algorithms, DNN has been used to solve traffic problems such as autonomous vehicle training, signal optimization, traffic prediction, smart routing, etc. DNN was used to optimize signal lights and minimize waiting time [36; 37; 38]. In these models, the operation of traffic lights based on the traffic situation was defined in terms of actions, states, and rewards. Researchers used DRL with proximal policy optimization (PPO) in a different approach to train autonomous vehicles and improve traffic performance [39; 40]. Taking advantage of communication and CAV technology, [PERSON] et al. [41] proposed a hybrid reinforcement learn ing (HRL) framework to control traffic at signalized intersections. To improve the driver's behavior and optimize the signal lights, [PERSON] et al. [42] developed a deep deterministic policy gradient (DDPG) algorithm-based car-following model for CAVs. The results after the training process included optimizing the traffic signals and adjusting the vehicle's trajectory behavior to ensure the minimum waiting time. In addition, some authors have also optimized signal lights with platooning vehicles in some papers [43; 44; 45]. Research results show that combining reinforcement learning with information sharing (V2V and V2I) improved the traffic model very effectively. The results after training depend on the set of random seeds and other parameters. The intersection management performance of the DRL-based approach was higher, but the input data requirements were more complex.