# Dynamic Path Planning for Vehicles Based on Causal State-Masking Deep Reinforcement Learning
## 3 Problem Formulation
### Dynamic Path-Planning Model  
In dynamic path planning, accurately describing the vehicle's dynamic characteristics is crucial for achieving precise control and path optimization. We model the vehicle as a planar rigid-body dynamic model moving within a two-dimensional plane, as representedin Figure 1. The dynamic equations of the vehicle in the body coordinate system are as follows:  
\[ \begin{cases}M(\dot{v}_{x}-v_{y}\omega)=F_{x}\\ M(\dot{v}_{y}+v_{x}\omega)=F_{y}\\ I_{\gamma}\dot{\omega}=M_{z}\end{cases}  \tag{1}\]  
where \(M\) is the mass of the vehicle; \(I_{\gamma}\) is the moment of inertia of the vehicle about the centroid perpendicular to the plane; \(\dot{v}_{x}\) and \(\dot{v}_{y}\) are the longitudinal and lateral accelerations, respectively; \(\dot{\omega}\) is the angular acceleration of the vehicle; and \(F_{x}\), \(F_{y}\), and \(M_{z}\) are the longitudinal force, lateral force, and yaw moment of the vehicle, respectively. To represent the vehicle's motion state in the global coordinate system, we establish a kinematic model describing the position and heading-angle changes in that system:  
\[ \begin{cases}\dot{x}=v_{x}\cos\psi-v_{y}\sin\psi\\ \dot{y}=v_{x}\sin\psi+v_{y}\cos\psi\\ \dot{\psi}=\omega\end{cases}  \tag{2}\]  
where \(\dot{x}\) and \(\dot{y}\) are the longitudinal and lateral velocities of the vehicle in the global coordinate system, respectively, and \(\psi\) is the vehicle's heading angle. By integrating the vehicle's dynamic and kinematic Equations (1) and (2), respectively, we can comprehensively consider the vehicle's motion in its coordinate system and the global coordinate system, as well as the coupling effects of rotational motion on lateral and longitudinal movements. In the design of dynamic path planning for vehicles, the state variables are defined as  
\[\mathbf{s}_{v}= \begin{bmatrix}x&y&\psi&v_{x}&v_{y}&\omega\end{bmatrix} , \tag{3}\]  
and the control variables are defined as  
\[\mathbf{u}= \begin{bmatrix}F_{x}&F_{y}&M_{z}\end{bmatrix} . \tag{4}\]  
Based on the established vehicle dynamics model, we describe the dynamic path-planning problem as follows: given the vehicle's initial state \(\mathbf{s}_{0}\) and the target position \(\mathbf{s}_{g}=[x_{g},y_{g}]\), a control policy \(\pi_{\theta}\) that enables the vehicle to move from the starting position to the target position while satisfying dynamic and environmental constraints needs to be learned. During motion, the vehicle must avoid dynamic obstacles in the environment while optimizing path length and smoothness to achieve efficient and safe path planning. To formalize the dynamic path-planning problem as an optimization problem, we aim to minimize the cumulative cost from the start point to the target point. The specific optimization objective function is defined as  
\[\min_{\pi_{\theta}}J=\sum_{t=0}^{T}\Delta t\big{[}\alpha C_{\text{pat}}(t)+ \beta C_{\text{smo}}(t)+\lambda C_{\text{col}}(t)\big{]} \tag{5}\]  
where \(\Delta t\) is the control time step; the coefficients \(\alpha\), \(\beta\), and \(\lambda\) are weight factors used to balance the importance of each cost component; \(C_{\text{pat}}(t)\) represents the path length cost, encouraging the vehicle to choose shorter paths and reducing the travel distance and time; \(C_{\text{smo}}(t)\) denotes the path smoothness cost, encouraging the vehicle to maintain a smoother motion process and reducing motion shocks; and \(C_{\text{col}}(t)\) signifies the collision risk cost, penalizing the vehicle's proximity to obstacles to ensure the safety of the path.