# An Overview of Reinforcement Learning Methods for Variable Speed Limit Control
## 4 Discussion
Since the proposed risk model is taking into account the characteristics of vehicle deceleration trajectories, potentially a more reliable evaluation will be achieved through microscopic simulations taking microscopic dynamic parameters of each individual vehicle in the calculation. The weather condition as a state variable for QL-VSL conceptually could be the way for further research on risk control on motorways since the weather has a significant impact on traffic dynamics and safety. In [34], results showed that there could be more than one optimal traffic equilibrium according to different control objectives (safety and flow maximization) due to separately implemented and tested objectives criteria. It will be interesting to have both included as a multi-objective optimization problem or multi-agent MDP modeled VSL problem where each agent seeks to find a balance between its local goal while at the same time not deviating from the global goal. In that case, it could be possible to have only one unique equilibrium regarding the optimal control policy. In [21], a continuous offline learning function was proposed in RL to enhance the robustness of the control strategy regarding the overspeed issue. The results showed that with continuous learning, the QL-VSL control performed reasonably well under lower driver compliance situations regarding reduction in \(TTT\). This is desirable since, in a real environment on motorways, the compliance rate can vary a lot and needs to be considered in the calculation. Similarly, in [18], an analysis of the robustness of action selection strategies (control policy) by adding noise to the measured traffic parameters (speed and density) was performed. Control policy behaves well for noise up to 10% otherwise learned control policy can no longer be used to appropriately select speed limits. This indicates that in a real environment, the placement of sensors has to be carefully chosen to be able to reconstruct missing data if one or more sensors break down. Regarding the hierarchical MARL approach in [19] for solving multi-agent RM and VSL problem, the action is communicated to the second-highest ranked agent. As shown, the result of this communication considering hierarchical MARL, the state-action space of the second agent grows by a factor equal to the number of actions available to the first agent. In contrast, as proposed in [49]the WL-VSL strategy is based on the idea that there is no need for any global controller analyzing the behavior of all agents. Similarly to DQL-VSL [34], the WL-VSL does not require additional communication between agents. The agents are incrementally setting their so-called \(W\)-values, using only local information (same in QL). An agent will be aware of its competition indirectly by the interference they cause. As stated in [34], the distributed system can deploy the controllers flexibly along the motorway, and there is no concern about the breakdown of the central traffic controller. Due to the low number of available research (until now [34; 49]) considering developing and testing the pure multi-agent RL-VSL systems, further investigation is needed in those directions.