# A Novel Deep Reinforcement Learning Approach to Traffic Signal Control with Connected Vehicles +
## 5 Conclusions  
This paper presents a novel DRL-based traffic signal controller for a typical four-way intersection. Different from the existing data-driven methods, this work takes advantage of CAE to capture traffic states into compressed representations. With a reduced dimensionality of the input traffic states, the proposed DRL-based controller enables more flexible design of the action space and increased responsiveness to dynamic traffic conditions. The simulation results demonstrate the effectiveness and performance of the proposed method through comparisons with three baseline methods across five commonly used performance metrics. The proposed DRL-based controller also exhibits more consistent training results compared to existing DRL methods. To further validate the control policy learned by the proposed DRL algorithm, the traffic flows with different SPaT plans are analyzed. In addition, the proposed DRL-based controller is tested for robustness against varying traffic volumes and compared with controllers retrained for specific traffic conditions. The results indicate that the proposed DRL agent is capable of handling unseen traffic scenarios effectively. The proposed method has a limitation in that it incurs a high training cost due to the expanded action space. Additionally, it has only been tested on a single four-phase intersection with 100% CV penetration rate. Future work will aim to extend the proposed method  
Figure 16: Robustness analysis by comparing the performance of DRL-based controller. to more complex scenarios and scale it up for corridor and network-level signal control with joint optimization of signal timing. This will present a greater challenge, as the traffic state and action space dimensions will increase exponentially. One potential solution to this challenge is to utilize the multi-agent reinforcement learning approach, which addresses the control problem of multiple autonomous, interactive agents in a common environment by distributing the global control to multiple local RL control agents [52]. The sharing of information among intersections can help individual signal controllers to learn and work together to optimize the overall performance of the traffic network. Additionally, we aim to expand the scenario to include varying traffic flows and heterogeneous vehicles, to further test the robustness and scalability of the data-driven methods. By incorporating these challenges, we hope to develop a data-driven approach that can learn a practical and truly optimal signal control strategy for real-world traffic systems. Furthermore, the proposed method uses the position and speed information of CVs at the end of a control cycle to construct the traffic state matrix as input to the DRL controller, ignoring the temporal information. Utilizing recurrent neural networks, such as long short-term memory (LSTM), has the potential to capture the complex dynamics within the temporal information of input data. Integrating an LSTM-autoencoder into the encoder-decoder network architecture can help it learn a representation for time series sequence data, enabling the DRL controller to make more accurate traffic state estimations and improve control strategies. These are promising avenues for future research. Conceptualization, Y.S. ([PERSON]), C.W., Y.S. ([PERSON]) and J.Y. ; Funding acquisition, Z.W. ; Investigation, Y.S. ([PERSON]); Methodology, Y.S. ([PERSON]); Project administration, Z.W. and T.J.L. ; Supervision, Z.W. ; Validation, Y.S. ([PERSON]); Writing--original draft, Y.S. ([PERSON]); Writing--review and editing, Z.W., T.J.L., C.W., Y.S. ([PERSON]) and J.Y. All authors have read and agreed to the published version of the manuscript.