## 1 Introduction  
Autonomous driving is increasingly being valued due to its potential in reshaping and enhancing safety in mobility, as well as in efficiency [1]. In urban environments, self-driving vehicles are less prone to traffic accidents so that the traffic network can be more fluid. In agricultural areas, reducing the required labor is a decent compensation for the decline in agricultural labor force and it makes it possible to complete agricultural tasks during certain hours that are not suitable for humans to work. Despite various technologies such as localization, perception and planning are required in realizing the full potential of autonomous driving; ensuring that the vehicle can stabilize at a set point or dynamically track reference signals and trajectories is a fundamental technology requirement [1, 2, 3]. Path following has been widely studied as an alternative to trajectory tracking in various types of vehicles due to its looseness with respect to time constraints [2]. In trajectory-tracking problems, the reference trajectory defines when and where the vehicle is supposed to be in the state space, while the objective of path following is to keep the vehicle as close as possible and moving along a predefined geometry path with no preassigned time information [2, 4, 5]. Pure pursuit is one of the earliest proposed strategies for following a path [6, 7]. The straightforward nature of this strategy has made it a popular choice in many applications where real-time control is essential. Assuming that the reference path has no curvature and the vehicle is moving at a constant speed, the pure pursuit controller is based on fitting a circle through the vehicle's current configuration, which in the case of a car is the rear wheel position, to a point on the path ahead of the vehicle by a so-called look-ahead distance \(L\)[1; 8]. However, the controller does not take the case where the distance from the current configuration to the reference path is greater than \(L\) into consideration. Among the feedback control-based approaches, an approach where the reference path is parameterized as a continuous function and the rear wheel position is used as the regulated variable to minimize the cross-track error between the rear wheel and the path, while also ensuring the stability of the vehicle heading, has been proposed in [9]. Notwithstanding the local exponential convergence ensured by the control law, knowledge of the reference path's curvature is required, thus necessitating that the parameterized function be twice continuously differentiable. Another feedback-based control law that minimizes the cross-track error of the front wheel position relative to a reference path that is discretized from the base trajectory to smooth waypoints has been proposed in [10]. This approach, which utilizes a nonlinear feedback function of the cross-track error to ensure local exponential convergence to the reference path, provides effective control for lower speeds, but necessitates some variations to be used for reverse driving. The vehicle robot using this controller for steering, namely Stanley, won the 2005 DARPA Grand Challenge. The aforementioned approaches [6; 9; 10] are suitable as baselines for reference and comparison, since they achieve satisfactory performance with a small number of adjustable parameters, moderate model accuracy and uncertainty requirements. In recent years, RL has achieved significant accomplishments in many fields, such as gaming and robotics, which have brought it great attention and widespread recognition [11; 12]. RL is a machine learning approach that enables an agent to learn optimal decisions through interactions with the environment. Since the use of deep neural networks to approximate the action-value function [13], which is also known as the Q-function, RL has been capable of dealing with high-dimensional state spaces.