# Dynamic Path Planning for Vehicles Based on Causal State-Masking Deep Reinforcement Learning
## 3 Problem Formulation
### Causal Relationship Model  
In complex dynamic environments, the state information perceived by the vehicle may contain a large amount of noise and many redundant features. These task-irrelevant features can lead to decreased model performance and weakened generalization capabilities. To address this, we introduce an SEM to identify key state features that have a direct causal impact on vehicle decision making and propose the concept of a state mask. In the causal graph model, we use a directed acyclic graph (DAG) to represent the causal relationships between variables [39], as represented in Figure 3. Nodes represent states, actions, and rewards, while directed edges represent causal influences. We assume the following causal relationships:  
1. Relationships between state features. \(s_{i}\to s_{i+1}\), indicating that feature \(s_{i}\) has a direct causal influence on feature \(s_{i+1}\). 2. The direct influence of state features on control inputs. \(s_{i}\to a_{i}\), indicating that feature \(s_{i}\) has a direct causal influence on control input \(a_{i}\). 3. The influence of control inputs on the next state features. \(a_{i}\to s_{i+1}\), indicating that control input \(a_{i}\) influences the state feature \(s_{i+1}\) at the next time step. 4. States and control inputs jointly determine rewards. \(\mathbf{s}\), \(\mathbf{a}\to r\). Based on the above assumptions, we can define an SEM for each variable to provide a quantitative description of causal relationships:  
\[ \begin{cases}s_{i}=f_{i}(\mathbf{Pa}(s_{i}),n_{i}),&i=1,2,\ldots,n\\ a_{l}=\pi_{\theta}(\$),&l=1,2,\ldots,m\\ r=r(\mathbf{s},\mathbf{a})\end{cases}  \tag{15}\]  
where \(\mathbf{Pa}(s_{i})\) denotes the set of parent nodes of \(s_{i}\), i.e., the variables that directly influence \(s_{i}\); \(n_{i}\) represents independent noise terms, that is, unmodeled external influences \(n_{i}\sim P(n_{i})\); and \(\$\) represents the set of state features after being masked. To identify the state features that have a direct causal influence on the control inputs \(\mathbf{a}\), we partition the set of state features \(\mathbf{s}\) into causal and non-causal feature sets. The causal feature set \(\mathbf{s}^{C}\) represents the features that directly influence the control inputs. The non-causal feature set \(\mathbf{s}^{N}\) represents the features that do not have a direct causal relationship with the control inputs, that is, \(\mathbf{s}=\mathbf{s}^{C}\cup\mathbf{s}^{N}\). In the causal graph, only the features in \(\mathbf{s}^{C}\) point to the control inputs \(\mathbf{a}\). To leverage causal features in DRL, we introduce a state mask vector \(\mathbf{m}=[m_{1},m_{2},\ldots,m_{n}]^{T}\), where \(m_{i}\in 0,1\), which is used to indicate whether each feature is causal:  
\[m_{i}= \begin{cases}1&s_{i}\in\mathbf{s}^{C}\\ 0&s_{i}\in\mathbf{s}^{N}\end{cases}  \tag{16}\]  
After applying the state mask, the masked state is represented as \(\$=\$\odot\mathbf{m}\). In this manner, the policy function \(\pi_{\theta}\) makes decisions solely based on the causal features \(\mathbf{s}^{C}\), filtering out noise features irrelevant to decision making and reducing the impact of spurious correlations on the model. To ensure model consistency under intervention  
Figure 3: Causal graph model of the DAG. conditions, we introduce a causal consistency constraint. Specifically, when intervening on a causal feature \(s_{i}\in\mathbf{s}^{C}\) by setting it to a specific value \(\text{do}(s_{i}=\hat{s}_{i})\), the conditional distribution of the control input should satisfy  
\[P(\mathbf{a}\mid\text{do}(s_{i}=\hat{s}_{i}),\mathbf{s}^{\backslash s_{i}})=P( \mathbf{a}\mid s_{i}=\hat{s}_{i},\mathbf{s}^{\backslash s_{i}}) \tag{17}\]  
where \(\mathbf{s}^{\backslash s_{i}}\) denotes the set of state features excluding \(s_{i}\). This consistency constraint helps the policy network learn the true causal relationships, preventing it from being misled by spurious correlations.