###### Abstract
Bionic heuristic-based methods include genetic, particle swarm, neural network, and ant colony algorithms. However, existing path planning algorithms, such as A*, APF, and DWA still have certain limitations. The DWA algorithm, for instance, is only applicable to omnidirectional mobile autonomous vehicles as it requires the vehicle to perform in-place rotation for obstacle avoidance and path replanning. The APF algorithm faces challenges of unreachable targets and local minima. Meanwhile, the A* algorithm may result in unsmooth planning paths with excessive turns. The artificial potential field method is a path planning method with a simple principle, concise mathematical model, smooth path, and high computational efficiency and is commonly used in local obstacle avoidance movement. The artificial potential field method is favored because it requires less external information, a faster computation speed, and smoother planning paths, but there are problems such as insufficient planning ability in complex environments, and local minima and unreachable goals. Moreover, all three types of path planning algorithms rely on pre-established maps, which limits autonomous vehicles in exploring unknown environments. The rapid development of deep learning, driven by the continuous advancements in computer hardware, has found extensive applications in various fields, including autonomous vehicles and computer vision. DRL, a combination of deep learning and reinforcement learning techniques, has gained significant attention [12]. In this approach, an intelligent agent interacts with an unknown environment (DRL), taking actions based on feedback from reward signals to achieve predefined goals [13]. The concept of reinforcement learning dates back to the early 20 th century [14]. In 1992, [PERSON] introduced the Q-Learning algorithm, which marked a significant milestone in the development of reinforcement learning algorithms. The Q-Learning algorithm utilizes a Q-table to store the expected reward values for different actions in each state, enabling the agent to make optimal decisions in various states [15]. [PERSON] and [PERSON] further improved upon Q-Learning by introducing the Sarsa algorithm, an on-policy Q-Learning algorithm that performed better in learning control problems. However, as the complexity of the problems in terms of states and actions increases, maintaining a Q-table becomes increasingly challenging, leading to the problem of dimensional catastrophe. To address this issue, in 2013, [PERSON] et al. introduced the Deep Q-Network (DQN) algorithm, which combined the Q-Learning algorithm with convolutional neural networks. The DQN algorithm overcomes the problem of dimensional catastrophe by utilizing end-to-end mapping and simulating the Q-table model through neural networks. This breakthrough marked a new era in reinforcement learning [16]. In 2015, [PERSON] proposed the distributed DQN, which utilized a distributed approach to handle neural networks and experience replay buffers, thereby further improving learning efficiency [17]. [PERSON] et al. introduced the Deep Double Q-Network (DDQN) to address the overestimation problem in the DQN algorithm. By utilizing two structurally identical networks, one for action selection and the other for Q-value estimation, the DDQN effectively mitigates the issue of overestimation that arises with single networks [18]. [PERSON] et al. proposed an enhanced version of the DQN called Dueling DQN. This architecture incorporates a convolutional layer to extract input states, followed by a fully connected layer that splits the output into two parts: the value function and the action advantage function. This separation enables better learning of the value function [19]. To enhance the stability of the algorithm during training, [PERSON] et al. suggested calculating the average value of previously learned Q-values [20]. This approach reduces the variance of the approximation error, further improving algorithm stability. [PERSON] et al.