## 7 Conclusions  
The core part of this paper is an overview of the recent applications of DRL in ATSC. Although the application of DRL in ATSC is a relatively new field, the papers reviewed show promising results compared to the traditional ATSC algorithms and ATSC algorithms that are based on RL methodologies. During the last five years, it is possible to notice an increase in the number of intersections controlled by the use of various multi-agent frameworks based on DRL. Transfer learning, multi-agent frameworks, and complex DNN model structures based on the Actor-Critic architecture are the most prominent approaches for improving the effectiveness of the DRL in the holistic traffic signal control. The development of the advanced DRL frameworks for large-scale ATSC is proof that the scientific community has recognized this approach as a promising direction for future traffic signal control. Furthermore, this study presents recent advancements in the state, action, and reward modeling, which is currently a very open area for research. Additionally, this study provides an assessment of the issues concerning the relationship between the learning convergence, DNN model complexity, data availability, and scalability for large intersection networks. Feasible solutions to those issues are key for a possible real-world application of those complex traffic signal control frameworks. Standardized image-like state representation is found to be important for interoperability and flexible scalability between the various ATSC approaches that are based on DRL. The availability of the larger traffic data spectrum generated by the various sensors and upcoming CAVs will create information-dense traffic data environments. Data from those environments must be obtained in a centralized fashion, pre-processed, and formatted up to a higher level of image-like traffic state representation. Each application of DRL in ATSC can directly use those image-like data structures or additionally process them in order to be digested by the used DNN model. This paper provides several existing DTSE approaches suitable for initial high-level pre-processing of the raw traffic data. The legislative and technical framework for gathering and sharing mentioned pre-processed traffic data must be in line with the Open Traffic Data concept in order to enable network-wide, inter-operable, and multi-agent ATSC based on information dense state representation. Future improvements in DRL frameworks for ATSC are expected to be based on the integration of Deep MARL approaches with multi-objective reward function and transfer learning to address the problem regarding the multi-intersection signal control. Mentioned complex DRL frameworks are expected to be optimized by the multi-objective evolutionary algorithms that can potentially enable integrated control between multi-intersection traffic signal control and traffic control methods at nearby motorway (e.g., ramp metering, variable speed limit control, etc.). The DOTD as the conceptual approach is expected to provide network-wide tensor-based data structures that will integrate fused traffic data, weather data, and road condition data in order to proved robust and more comprehensive inputs for advanced DRL frameworks applied in ATSC.  
This research has received funding from the European Union's Horizon 2020 research and innovation program under grant agreement No. 857592.  
This research was part of a project Twinning Open Data Operational that has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No. 857592.  
The authors declare no conflict of interest.