# An Overview of Reinforcement Learning Methods for Variable Speed Limit Control
## 5 Conclusions  
The identified open areas of application of VSL based on RL methods are as follows. From the traffic engineering perspective, the concept of establishing a multi-agent RL-VSL system based on the cooperation of agents that could simultaneously position and determine the length of the VSL area as well as calculate the speed limit jointly has to be considered in the future studies. These three decisions could be calculated simultaneously by three different agents within the VSL. Such dynamic positioning of the VSL area on the urban motorway depending on the current spatial-temporal characteristics of the traffic flow, as well as integrating several connected VSLs on a longer motorway segment could achieve smoother speed transition. Therefore, enabling more effective traffic flow control, limiting the need for more restrictive actions such as lower speed limits directly in front of congestion which negatively impact the free-flow upstream of the controlled section. From the algorithmic side, an open area is visible in MARL where decentralized approaches have to be further investigated since centralized approaches are prone to the curse of dimensionality.  
Until now, all analyses are based on discretized time sampling of traffic states and discrete actions used for the RL algorithm. Real-time traffic management is one of the areas of interest given that in the real environment of the motorway, the position and speed of the vehicle changes as a continuous variable. The application of continuous variables to describe agent actions and states would allow smoother speed control of an individual vehicle, which is interesting for testing simulation models that support a CAV and AV environment. This will trigger-off the usage of the policy iteration algorithm for the case of continuous-valued action and RNN architecture like LSTM and GRU, which are able to take spatial-temporal behavior of the traffic as input into the learning process for a more accurate DRL-VSL control. The weather condition has a significant impact on traffic dynamics and safety, and it should be examined in-depth as a state variable in the RL-VSL control as well.  
The conceptualization of the study was done by [PERSON], [PERSON], and [PERSON], who also did the funding acquisition. The writing of the original draft and preparation of the paper was done by [PERSON], and [PERSON] All authors contributed to the writing review and final editing. The supervision was done by [PERSON] Visualizations were done by [PERSON] All authors have read and agreed to the published version of the manuscript.  
This work has been partly supported by the University of Zagreb and Faculty of Transport and Traffic Sciences under the grants \"Investigation of the impact of autonomous vehicles on urban traffic flow characteristics\" and \"Innovative models and control strategies for intelligent mobility\", and by the European Regional Development Fund under the grant KK.01.1.1.01.009 (DATACROSS).  
The authors thank the company PTV Group for providing a research license of the simulator VISSIM. This research has also been carried out within the activities of the Centre of Research Excellence for Data Science and Cooperative Systems supported by the Ministry of Science and Education of the Republic of Croatia.  
The authors declare no conflict of interest. The funding institutions had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.  
The following abbreviations are used in this manuscript:  
ANN Artificial Neural Network AV Autonomous Vehicles