# Root Cause Attribution of Delivery Risks via Causal Discovery with Reinforcement Learning
## 2 Methodology
For decoding, we use a single-layer decoder, defined as:  
\[g(W_{1},W_{2},u)=u^{\top}\text{tanh}(W_{1}\text{enc}_{i}+W_{2}\text{enc}_{j}), \tag{3}\]  
where \(W_{1},W_{2}\in\mathbb{R}^{d_{h}\times d_{n}}\), and \(u\in\mathbb{R}^{d_{h}\times 1}\) are trainable parameters. Here, \(d_{h}\) represents the number of hidden layers in the decoder, and \(d_{n}\) is the dimension of the encoder output. In this equation, the decoder processes the transformed representation \(z=\text{Sublayer}(x)\) and attempts to map it to the observed relationships in the data. The decoder uses the features learned by the sublayer to predict the strength and direction of causal relationships. Thus, the sublayer extracts meaningful features from the input, and the decoder uses those features to estimate the causal relationships. The sublayer function \(\text{Sublayer}(x)\) acts as a feature extractor, filtering out noise and producing a transformed representation \(z\), which retains the most relevant information from the input \(x\). This transformation is crucial because the input \(x\) often contains high-dimensional, redundant data. The sublayer extracts key features that the decoder then uses to reconstruct the causal graph. To generate the adjacency matrix, each element is passed through a sigmoid function and then sampled based on a Bernoulli distribution with probability \(\sigma(g)\):  
\[M\sim\text{Ber}(\sigma(g)). \tag{4}\]  
Here, the Tanh (hyperbolic tangent) function in Equation (3) and the sigmoid function in Equation (4) are combined to balance transformation values and interpretability. The Tanh function maps input values to a range of \([-1,1]\), ensuring that both positive and negative influences are captured symmetrically, which is important when dealing with data involving variations in both directions. However, for outputs like probabilities or causal relationship strengths, values need to be constrained within the range of \([0,1]\). This is where the sigmoid function comes into play. The sigmoid function compresses the Tanh-generated values into this range, providing a smooth and interpretable output that represents probabilities or relative strengths. By using Tanh to handle a wide range of input values and sigmoid to transform these into a usable probability space, our model ensures that the results are both balanced and interpretable for causal discovery. Figure 2: Causal structure. To prevent self-loops, the diagonal elements \((i,i)\) of the adjacency matrix are set to zero. By iteratively inputting the encoder outputs of all variables, a complete directed graph's adjacency matrix is obtained. The scoring function uses the Bayesian Information Criterion (BIC), which is decomposable and allows for adjusting the penalty term. The BIC score for a graph \(\mathcal{G}\) is given by:  
\[S_{\text{BIC}}(\mathcal{G})=-2\log p(X;\hat{L};\mathcal{G})+d_{L}\log m, \tag{5}\]  
where \(\hat{L}\) is the maximum likelihood estimate, \(d_{L}\) is the number of parameters in \(L\), and \(m\) is the size of the dataset \(X\). To ensure the generated graph is a directed acyclic graph (DAG), the score function includes both a reward and a penalty term, incorporating two acyclic constraints:  
\[\text{reward}=-[S_{\text{BIC}}(\mathcal{G})+\lambda_{1}I(\mathcal{G}\
otin \text{DAGs})+\lambda_{2}h(A)], \tag{6}\]  
where \(I(\cdot)\) is an indicator function, \(\lambda_{1},\lambda_{2}\geq 0\) are hyperparameters, \(A\in\{0,1\}^{d\times d}\), and \(h(A)\) is a function introduced by [36], which is non-negative and small for cyclic graphs. The binary adjacency matrix of a directed graph \(\mathcal{G}\) is acyclic if and only if:  
\[h(A)=\text{trace}\Big{(}e^{A}\Big{)}-d=0, \tag{7}\]  
where \(e^{A}\) is the matrix exponential of \(A\). Larger values of \(\lambda_{1}\) and \(\lambda_{2}\) increase the likelihood that a high-reward graph is acyclic.