# Explainability of Predictive Process Monitoring Results: Can You See My Data Issues?
## 4 Experiments
### Experimental Setup
#### 4.1.4 Dimension 4: XAI  
Explainability experiments are divided into two sets, i.e., experiments depending on intrinsic explainability and experiments depending on post hoc XAI methods. _Intrinsic explainability_ is achievable through querying mechanisms provided by the predictive model for retrieving features that influence prediction generation. We compared the explainability of predictive models to the results generated by each applied XAI method. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Event Log** & **Prefix Length** & **Training Bucket Size** & **Testing Bucket Size** & **\# Features** \\ \hline \multirow{4}{*}{Sepsis2} & 1 & 620 & 156 & 99 \\ \cline{2-5}  & 6 & 614 & 154 & 240 \\ \cline{2-5}  & 11 & 468 & 124 & 408 \\ \hline \multirow{3}{*}{Traffic\_Fines} & 1 & 103,652 & 25,923 & 201 \\ \cline{2-5}  & 6 & 8736 & 1965 & 901 \\ \hline \multirow{3}{*}{BPIC2017\_Refused} & 1 & 25,130 & 6283 & 120 \\ \cline{2-5}  & 6 & 25,118 & 6283 & 1143 \\ \cline{1-1} \cline{2-5}  & 11 & 24,952 & 6283 & 4104 \\ \cline{1-1} \cline{2-5}  & 16 & 24,589 & 6261 & 7214 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: \(Cont.\)To choose which XAI methods to apply, we considered criteria obtained from the explainability taxonomy available in [28]. Considering the explanations scale, in order to be able to explain the reasoning process of an ML model over the whole training event log, a global XAI method is needed. As intrinsic explainability is already satisfied by querying the used ML models themselves, post hoc XAI methods are necessary to validate the important features revealed by the models themselves. The chosen XAI methods have to be explaining by showing the relevance of features to the resulting predictions. Outcomes and explanations of the aforementioned methods are most comparable to the resulting set of important features when the ML model is queried. Furthermore, it is necessary to choose model-agnostic methods in order for the results to be applicable to predictions of both the LR and XGBoost models. To this end, we used Permutation Feature Importance and SHAP (the global form). Moreover, we analysed LR coefficients and features important to XGBoost, as this provides a means to query them for important features. To check execution stability, we ran experiments with different settings and combinations of settings from the taxonomy in Figure 3 twice. Stability checks following this definition are expensive due to computational costs, which are affected by the number of choices used in the experiments for each dimension. As a result, it was not possible to run the full experimental settings combinations more than twice in the context of stability checks. However, in this study we are more concerned with a comprehensive experimental framework including several experimental dimensions. The focus of stability checks in this dimension is on the stability of the explanations being generated by the model itself. Furthermore, we compared explanations of two global XAI methods over two runs of the experiments. In this study, we adopt a data-oriented focus and follow the data with analyses, starting from the event log and ending with the different explanations generated. A comparison of XAI methods with an in-depth analysis of their strengths along with their weaknesses constitutes a research gap that should be addressed; however, for this study these aspects are out of scope. Figure 4 provides a summary of this section. For each of the selected event logs, we ran several experiments that follow the taxonomy depicted in Figure 3 and constitute steps in the following order. 1. Conduct data analysis of the event log features using three different techniques, namely, pandas profiling, correlation analysis, and mutual information analysis. 2. Preprocess the event log using _single-aggregation_ and _prefix-index_ configurations, which in turn produce several versions of the same event log. 3.