# Root Cause Attribution of Delivery Risks via Causal Discovery with Reinforcement Learning
## 2 Methodology  
_The Proposed Method_  
Building on the gaps identified in existing works, this section introduces a novel framework that integrates causal discovery with reinforcement learning to address delivery risks in supply chains. The proposed method consists of two main components: (1) causal discovery through reinforcement learning and (2) causal strength quantification using information entropy. The framework is designed to uncover the root causes of delivery risks by identifying \begin{table}
\begin{tabular}{l l l} \hline \hline
**Article** & **Existing Methods** & **Results** \\ \hline
[13] & Optimized Production Processes & Mitigated delays at the production stage \\ \hline
[14] & Optimizing delivery strategies & Reduced delivery delays post-risk materialization \\ \hline
[15] & Enhanced delivery strategies & Improved on-time delivery rate by optimizing routes \\ \hline
[17] & Machine Learning (Random Forest) & Improved accuracy in predicting delivery delays \\ \hline
[19] & Support Vector Machines & Identified key delay patterns in historical data \\ \hline
[21] & Clustering with Attention & Higher predictive accuracy for identifying delayed shipments \\ \hline
[23] & Optimization Algorithms & Adjusted inventory levels and routing plans to mitigate delays \\ \hline
[24] & Real-Time Adjustment (IoT) & Enabled dynamic responses to real-time risks \\ \hline
[28] & IoT-enabled Supply Chain Management & Improved response time to detected delivery risks \\ \hline
[30] & IoT and Predictive Analytics & Detected anomalies and delayed shipments using sensor data \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Summary of methods and results on delivery risk prediction. the underlying causal relationships between supply chain variables. We adopt the data-generating model proposed by [31; 32], where each variable \(x_{i}\) corresponds to a node \(i\) in a \(d\)-node directed acyclic graph (DAG) \(\mathcal{G}\). The observed value of \(x_{i}\) is modeled as a function of its parent variables in the graph, plus an independent additive noise \(n_{i}\). Specifically,  
\[x_{i}:=f_{i}(\mathbf{x_{pa(i)}})+n_{i},\quad i=1,2,\ldots,d, \tag{1}\]  
where \(f_{i}(\mathbf{x_{pa(i)}})\) represents the set of parent variables \(x_{i}\) that have directed edges toward \(x_{i}\), and the noise terms \(n_{i}\) are assumed to be jointly independent. We also assume causal minimality, meaning that each function \(f_{i}\) is non-constant in any of its arguments, as explained by [32]. As shown in Figure 1, our model consists of two components: causal discovery via reinforcement learning and the calculation of inverse information entropy (IIE) causal strength [33]. The model processes the observed dataset \(X=\{x_{1},x_{2},\ldots,x_{i}\}\), where \(x_{i}\) represents the dimensions of the input observations and outputs a causal structure comprising a causal graph \(G\) and causal strengths. Figure 2 illustrates the causal structure simulated from the observed data \(O=\{G,S\}\), where \(S\) denotes the strength of the causal relationships in graph \(G\). Our encoder-decoder approach builds upon the work of [10]. The model employs an encoder-decoder framework to generate a directed graph. The encoder, following the architecture introduced by [34], consists of six identical layers, each with two sublayers. The first sublayer implements a multi-headed self-attention mechanism, while the second is a fully connected feedforward network with positional encoding. These sublayers are connected using residual connections, as described by [35]. The output of each sublayer is normalized as follows:  
\[\text{Layer Norm}(x+\text{Sublayer}(x)), \tag{2}\]  
Figure 1: Proposed causal discovery approach with reinforcement learning for delivery risk. where \(\text{Sublayer}(x)\) represents the function applied in the sublayer. To maintain consistency, all sublayer and embedding outputs in the model have the same dimension, \(d_{\text{model}}=512\).