## 4 Conclusions  
At present, traditional path planning algorithms for autonomous vehicles heavily rely on high-precision maps to generate routes. However, in obstacle-free and expansive environments, the mapping results may be inadequate, leading to suboptimal or even non-feasible driving paths. Additionally, conventional DQN DRL algorithms tend to converge slowly or fail to converge during training. To address these challenges, this article presents the APF-D3 QNPER algorithm along with an improved network structure and an appropriate reward function. The algorithm is evaluated through simulation experiments and further validated by implementing it in a real autonomous vehicle system to assess its performance in unknown environments. The results demonstrate the algorithm's superiority and its potential for practical applications. The contributions and innovations of this paper include the following:  
1. To enhance the convergence ability of the traditional DRL algorithm, this study introduces a modified approach for assigning rewards to experience samples during the experience replay phase. Specifically, the TD error and a weighted sum are employed as sampling probabilities for prioritized experience sampling. This prioritization method improves the overall convergence performance of the algorithm. 2. By integrating an action output method based on the APF algorithm with the D3 QNPER algorithm, this study achieves improved convergence speed, as well as enhanced obstacle avoidance and generalization capabilities for autonomous vehicles operating in complex dynamic environments. 3. Adding an LSTM network to the state feature extraction network and inputting state data between several frames before and after into the LSTM network simultaneously further improves its adaptability in unknown environments and enhances its spatiotemporal sensitivity to the environment. This paper validates the effectiveness of the APF-D3 QNPER algorithm through simulation experiments. Currently, the action output of the algorithm is discrete with multiple values. In future research, the consideration of continuous actions can be explored to achieve smoother driving paths. For unmanned vehicles or small unmanned mobile robots, the limited movement output will affect their maneuvering performance seriously. Continuous action can make the path smoother, reduce the jitter generated by the movement and improve the comfort level, further improve the obstacle avoidance ability, and passing performance, and protect the safety of the vehicle carrying people or objects. The D3 QNPRE algorithm is an improved algorithm based on the DQN, which outputs a limited Q-value, and therefore cannot output continuous action; so, in the subsequent improvement process, the combination of the policy function and the value function should be considered for the deep reinforcement learning algorithm to solve it. Additionally, conducting experiments in real-world environments can provide further insights and validation. In the future, the APF-D3 QNPRE algorithm can be applied to multiple domains in real-world scenarios. In the field of autonomous vehicles, it can help autonomous vehicles with more accurate path planning and decisions, enabling them to drive safely in different traffic situations, avoid accidents, and choose the best route more efficiently. In the field of drones, they can be used to plan the flight paths of drones so that they can accomplish tasks in complex environments, such as search and rescue, agricultural inspections, and cargo transportation. In the fields of logistics and warehousing, it can be used to optimize the transportation path of goods and the operation process inside the warehouse to improve efficiency and reduce costs. In the field of intelligent robots, it helps robots to perform tasks in various environments, such as cleaning, food delivery, medical assistance, etc., while avoiding obstacles and interacting with human users.