# LogEDL: Log Anomaly Detection via Evidential Deep Learning
## 4 Methodology
### LogEDL
#### 4.5.5 Evidential Uncertainty Loss  
To train the LogEDL model, we follow the MLM paradigm to capture patterns in normal log sequences. The neural network architecture of LogEDL is very similar to that of a classical neural network. The only difference is that we replace the softmax layer with an activation layer, such as ReLU, to ensure non-negative outputs, which serve as the evidence vector for predicting the Dirichlet distribution.  
Given a sample \(k_{i}\), let \(f\ (k_{i}\ |\Theta)\) represent the evidence vector \(e_{i}\) predicted by the network for classification, where \(\Theta\) denotes the network parameters. The corresponding Dirichlet distribution has parameters \(\alpha_{i}=f\ (k_{i}\ |\Theta)+1\). Once these distribution parameters are computed, their mean, \(k_{i}/\mathcal{S}\), can be used as an estimate of the class probabilities. Let the ground truth \(y_{i}\) be the one-hot vector encoding the observation \(k_{i}\), with \(y_{ij}=1\) and \(y_{ij}=0\) for all \(i\
eq j\). Here, \(\alpha_{i}\) is the parameter of the Dirichlet density for the predictor. Assuming that class probability follows a prior Dirichlet distribution, the evidential mean squared error loss (EMSE) to be minimized for learning eventually reduces to the following form.  
\[\mathcal{L}_{EMSE}^{i}(\Theta) =\int\|y_{i}-\text{p}_{i}\|_{2}^{2}\frac{1}{B(\alpha_{i})}\prod _{i=1}^{K}p_{ij}^{\alpha_{ij-1}}d\text{p}_{i} \tag{7}\] \[=\sum_{j=1}^{K}\mathbb{E}[y_{ij}^{2}-2y_{ij}p_{ij}+p_{ij}^{2}]\] \[=\sum_{j=1}^{K}(y_{ij}^{2}-2y_{ij}\mathbb{E}[p_{ij}]+\mathbb{E}[ p_{ij}^{2}])\] \[=\sum_{j=1}^{K}(y_{ij}-\alpha_{ij}/\mathcal{S}_{i})^{2}+\frac{ \alpha_{ij}(\mathcal{S}_{i}-\alpha_{ij})}{\mathcal{S}_{i}^{2}(\mathcal{S}_{i} +1)}\] \[=\sum_{j=1}^{K}(y_{ij}-p_{ij})^{2}+\frac{p_{ij}(1-p_{ij})}{( \mathcal{S}_{i}+1)}\]By decomposing the first and second moments, the loss function aims to achieve a joint objective of minimizing both the prediction error and the variance of the Dirichlet distribution generated by the neural network for each sample in the training set. In this way, it prioritizes data fitting over variance estimation. Given that the patterns of normal log sequences and anomalous log sequences are different, we expect that once LogEDL can correctly predict the masked log keys, it will be able to distinguish between normal and anomalous log sequences.  
We can obtain the negative log marginal likelihood by integrating out the class probability; the evidential negative log-likelihood loss (ENL) takes  
\[\begin{split}\mathcal{L}_{ENL}^{i}(\Theta)&=-\log( \int\prod_{j=1}^{K}p_{ij}^{y_{ij}}\frac{1}{B(\alpha_{i})}\prod_{i=1}^{K}p_{ij} ^{\alpha_{ij-1}}d\text{p}_{i})\\ &=\sum_{j=1}^{K}y_{ij}(\psi(S_{i})-\psi(\alpha_{ij}))\end{split} \tag{8}\]  
The same approach can be applied also to the cross-entropy loss, the evidential cross-entropy loss (ECE) reads  
\[\begin{split}\mathcal{L}_{ECE}^{i}(\Theta)&=\int \left[\sum_{j=1}^{K}-y_{ij}\log\left(p_{ij}\right)\right]\frac{1}{B(\alpha_{i} )}\prod_{i=1}^{K}p_{ij}^{\alpha_{ij-1}}d\text{p}_{i}\\ &=\sum_{j=1}^{K}y_{ij}(\psi(S_{i})-\psi(\alpha_{ij}))\end{split} \tag{9}\]