# A Deep Reinforcement Learning Approach for Efficient, Safe and Conformtable Driving +
## 2 Related Work  
ML techniques have been widely used in autonomous and automated vehicles to address a wide range of applications requiring a decision-making process [13; 14; 15]. Below, we discuss the works that are most relevant to ours, as they have focused on ACC and addressed the drawbacks that affect traditional ACC approaches. In this context, among the various possible ML approaches, (D)RL has been often applied, because of its effectiveness in dealing with control problems in dynamic and partially observable environments. Further, the survey in [15] validates the effectiveness of (deep) reinforcement learning models in vehicle longitudinal control systems. (D)RL models provide an edge over their counterparts, which suffer from the lower variety of scenarios recorded in complex road environments. The reward function of a (D)RL framework is a critical component in the DRL agent's learning process, as it allows the agent to assess its performance in different states and learn the optimal strategy. In particular, ref. [16] shows that the usage of sparse rewards can result in instability and policy convergence to non-optimal solutions. We remark that, in contrast to earlier work [17], we have defined a continuous reward function in our model in order to address the aforementioned issues and successfully determine the ideal trade-off between efficiency, safety, and driving comfort. The study in [7] presents one of the earliest research works leveraging DRL for CACC. The framework proposed therein leverages the information acquired from the RADAR sensor and through V2V communications to preserve the time headway with the leading vehicle. The solution, however, limits the action space with discrete values where the vehicle can accelerate/decelerate only with predetermined values, which may result in oscillating behavior of the vehicle control and may not accurately represent the vehicle's response in real-world conditions. Further, ref. [7] considers the headway as the only safety objective of the DRL, thus neglecting comfort and stability. This is a also major drawback of traditional ACC solutions, which do not account for road conditions that can lead to a dangerous situation while controlling the vehicle acceleration. These issues are partially addressed by the DRL-based framework proposed in [6], which deals with a continuous action space and reward function, accounting for multiple criteria concerning time-to-collision, headway, and jerk. The major shortcoming of the framework in [6] resides in the representation of the vehicle dynamics or the lack thereof. Without considering vehicle dynamics, the framework's vehicle response is inconsistent with the vehicle's actual performance. In addition, the representation of the RADARsensor outputs is generated using the vehicle positions, as opposed to simulating a realistic RADAR sensor. Other works proposing multi-objective functions to deal with safety and comfort can be found in [18; 19; 20]. Even though vehicle stability under different road conditions is an essential factor in passenger safety, none of them consider it an objective. In the context of comfort, ref. [21] presents a comprehensive empirical analysis of the factors affecting the comfort of the passenger when designing an automated driving vehicle. In our study, we evaluate the influence of situational driving conditions, such as potential collisions, on passenger comfort. A work that on the contrary focuses solely on stability is the scheme introduced in [22]. The solution therein controls the torque vectoring of an electric vehicle to improve vehicle stability. Ref. [22] use a discrete action space to determine the optimal torque vectoring ratio by exploiting yaw rate and steering angle as states. Finally, ref. [23] presents a performance comparison between the Model Predictive Control (MPC) and DRL for ACC. This study highlights that, being a model-based system, the MPC requires online optimization, hence exacting a high toll in terms of computing resource consumption for real-time applications.