# Dynamic Path Planning for Vehicles Based on Causal State-Masking Deep Reinforcement Learning
## 5 Comparison of Simulation Results
### Generalization Analyses  
In this path-planning simulation, we designed a dynamic path-planning environment to evaluate the generalization capabilities and path-planning performance of three RL algorithms: DQN, TD3, and CSM-TD3. The entire experiment was divided into two phases: the first phase involved a static obstacle environment, while the second phase featured a dynamic obstacle environment. This design allows us to progressively increase the difficulty of path planning from simple to complex scenarios. In the static obstacle environment, the agent only needs to contend with the challenges posed by a static setting, providing a performance baseline for each algorithm under fundamental conditions. In contrast, the dynamic obstacle environment introduces continuously moving obstacles, significantly increasing the complexity of path planning and enabling the assessment of each algorithm's adaptability and decision-making responsiveness in the face of dynamic changes. Through this phased approach, we can observe how the agent transitions from handling static obstacles to confronting more complex dynamic environments. The environment is modeled as a two-dimensional plane with dimensions set to \([-5,5]\times[-5,5]\), where the agent's starting point is at \((0,0)\) and the target points are located at three positions: \((0,4)\), \((-3,4)\), and \((4,4)\). This design maintains consistent initial conditions for each algorithm across different phases, ensuring comparability in their performances. The diversity of target points further tests the agent's flexibility and adaptability in addressing various path requirements. This design, transitioning from the same starting point to different endpoints, increases the difficulty of path planning incrementally in each phase. As a result, the agent must demonstrate higher generalization and policy optimization capabilities to effectively navigate the challenges presented. In Figures 7-12, the black points represent the starting point, gray points represent obstacles, dashed circles represent the moving trajectory of obstacles, and purple points represent the goal point. The arrow indicates the direction of movement of the obstacle  
In the static obstacle scenarios, as illustrated in Figures 7-9, the path-planning results of the DQN exhibit significant deviations and instability. When faced with static obstacles, the DQN frequently demonstrates unnecessary detours and backtracking, particularly in densely packed obstacle areas, leading to chaotic path selections filled with superfluous turns. This performance issue stems from the DQN's heavy reliance on discrete Q-value estimations, which results in inaccurate assessments of the optimal path and an inability to generate effective continuous actions. This directly manifests in its paths, showing excessive backtracking and redundant movements, greatly increasing the complexity and overall cost of the planned routes. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**DRL Algorithm** & **DQN** & **TD3** & **CSM-TD3** \\ \hline Mean average return & \(-781.6090\) & \(-508.2607\) & \(-318.9694\) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Mean average returns obtained from different DRL algorithms. Figure 8: Comparison of the path-planning diagrams of the DQN, TD3, and the proposed GER-TD3 when the obstacle is stationary and the endpoint is \((-4,3)\). Figure 7: Comparison of the path-planning diagrams of the DQN, TD3, and the proposed GER-TD3 when the obstacle is stationary and the endpoint is \((0,4)\). Figure 9: Comparison of the path-planning diagrams of the DQN, TD3, and proposed GER-TD3 when the obstacle is stationary and the endpoint is \((4,4)\). Figure 11: Comparison of the path-planning diagrams of the DQN, TD3, and the proposed GER-TD3 when the obstacle is moving and the endpoint is \((-4,3)\). Figure 12: Comparison of the path-planning diagrams of the DQN, TD3, and the proposed GER-TD3 when the obstacle is moving and the endpoint is \((4,4)\).