# Dynamic Path Planning for Vehicles Based on Causal State-Masking Deep Reinforcement Learning
## 2 Related Work
### RL-Based Path-Planning Methods  
With the rapid development of deep learning and RL, RL-based path-planning methods have become a research hotspot because of their adaptability and online learning capabilities in dynamic environments. Many researchers have explored how to leverage RL to enhance the effectiveness of path planning. [PERSON] et al. (2019) proposed a complete coverage path-planning algorithm based on Q-learning, which achieved 100% coverage by introducing global environmental information and optimizing path strategies near obstacles while significantly reducing path redundancy. For unplugged ground platforms, [PERSON] et al. (2019)improved the Q-learning algorithm to enhance path-planning flexibility and convergence speed, making it suitable for autonomous driving functions. Similarly, [PERSON] et al. [30] combined deep Q-Learning with path-smoothing techniques, demonstrating their model's effectiveness in complex environments.  
[PERSON] et al. [31] proposed an APF path-planning strategy based on the partially observable Markov decision process (POMDP) model, which integrates sensor information to predict obstacle positions, enabling real-time path planning for uncrewed aerial vehicles in complex environments. In further research [32], the path generation method based on the deep Markov model (RL-PG) could ensure path-tracking safety through a motion fine-tuning module, exhibiting a high success rate. In research on drone path planning [33], the authors combined the A\({}^{\star}\) algorithm with velocity obstacles in an RL approach, improving planning efficiency in dynamic environments. In addition, [PERSON] et al. [34] designed a reward function strategy to encourage agents to choose the shortest paths with fewer turns, thereby optimizing the number of turns and total distance.  
For time-constrained path planning in complex environments, [PERSON] et al. [35] integrated prioritized experience replay and improved exploration mechanisms in an off-policy RL method, enhancing sampling efficiency and rescue target achievement rates. [PERSON] et al. [36] introduced the PIOPGRL algorithm based on the pigeon-inspired optimization algorithm, fusing it with RL to optimize spacecraft path planning under multiple attitude constraints. Finally, [PERSON] et al. [37] proposed a path obstacle-avoidance control method based on the deep deterministic policy gradient that maps perception to control through deep neural networks, improving obstacle-avoidance performance.  
RL methods are prone to capturing task-irrelevant noise and features in high-dimensional state spaces, leading to unstable policies and decreased generalization capabilities. Moreover, these methods typically require large amounts of training data and computational resources, resulting in lengthy training processes that hinder real-time applications. Additionally, many RL algorithms lack effective causal inference mechanisms when facing environmental disturbances and dynamic changes, affecting the model's convergence in complex environments and its ability to be generalized to new environments.