# Design of a Reinforcement Learning-Based Lane Keeping Planning Agent for Automated Vehicles
## 1 Introduction
A better concept is proposed in [14], which uses the well-known Deep Q-network algorithm for training the neural network to predict the appropriate yaw acceleration for a lane-changing maneuver. Most of the longitudinal control problems are complex and show firm nonlinear behavior, which makes deep learning-based solutions suitable for this type of control task because these methods support adaptation and demonstrate the capability of handle nonlinear control problems. The authors of [15] combine fuzzy logic with reinforcement learning and create a hybrid method for the longitudinal control of a vehicle. The RL method is a Q estimator network, and for fuzzy inference they use Takagi-Sugeno. The reward function only uses the distance between the ego and the front vehicle, which is not able to provide a solution that considers safety and efficiency aspects in the decision-making. Consequently, in such control problems, a multi-objective reward scheme is required for appropriate control. In [16], a Policy Gradient-based agent is proposed, which uses a multi-objective reward function to train the agent for the control task of Cooperative Adaptive Cruise Control. In the reward function, the time distance between the ego and the front vehicle is used along with the derivative of this quantity. The actions, in this case, are simple brake, throttle, and pass commands. The trained network is tested in simulation and the results show that it can keep the required time distance accurately, but the control signal always oscillates, which is uncomfortable for the passengers. This behavior can be corrected by sanctioning this exact operation during training by the reward function, but another approach is the use of RNN networks. The authors in [17] propose a Parametrized Batch Actor-Critic algorithm that utilizes both the policy and value-based model-free RL techniques for a longitudinal control problem. In this case, the real-word testsshow that the multi-objective reward system mitigates the oscillation, which results in great comfort for passengers. In [18], the authors introduce a Deep Q-Learning (DQN)-based method that is dedicated to controlling the braking system for avoiding collisions autonomously. Along with the original memory buffer of DQN, they use a special memory where only experiences of critical events are stored. These types of events are rare, so without this memory, these events become underrepresented, which deteriorates the reliability of the agent. The trained model is tested in situations where accidents have to be avoided with pedestrians. The model reaches excellent performance, and it is also tested according to the restrictions of the Euro NCAP test protocol, and the model passes this test too. The methods above utilize reinforcement learning's unique features such as online learning; however, it also has some disadvantages like the sample inefficiency. To mitigate these issues, several authors propose methods that combine reinforcement and supervised learning approaches, where supervised learning accelerates the training process, but thanks to reinforcement learning, the possibility of adaption is maintained. For such concepts see in [19; 20]. Autonomous driving requires both lateral and longitudinal control to operate in a simultaneous manner to perform the dynamic driving task on a proper performance level [21]. A method that combines supervised learning and reinforcement learning is presented in [22], where the authors use the exact idea for formulating their own algorithm to solve the lateral and longitudinal control task simultaneously, which control task is formulated as keeping the vehicle on the road. In this particular algorithm, DQN is combined with SL, where the experiences are gathered from professional drivers to accelerate training. The DQN's experience memory is also filtered to eliminate poor experiences. In [23], the authors compare the DQN and the Deep Deterministic Actor-Critic (DDAC) algorithm to show how important the usage of continuous actions in such control problems.