# Explainability of Predictive Process Monitoring Results: Can You See My Data Issues?
## 5 Results
### Effect of the applied preprocessing configurations
#### 5.2.2 Observations of Global XAI Methods Results  
In this subsection, an analysis of Permutation Feature Importance (PFI) and SHAP results is presented. For each XAI method, we search for indications of multicollinearity and (dis)conformance with important feature sets indicated by the employed predictive model. **Permutation Feature Importance (PFI). **  
The basic idea of PFI is to measure the average between the prediction error before and after permuting the values of a feature [4]. Feature values permutation or shuffling aims to estimate the increase in prediction error as an indicator of the feature importance. PFI was executed twice to query LR and XGBoost models trained over the event logs preprocessed with single-aggregation and prefix-index configuration. Each execution run included ten shuffling iterations. The mean importance of each feature was computed. In detail, PFI execution led to the following observations. **Observation 12:**_In single-aggregated event logs, the results of the two runs are consistent with respect to feature sets and the importance scores of these features. However, the resulting feature sets are affected by multicollinearity between the features. In prefix-indexed event logs, the two runs are consistent in all event logs._  
An exception in prefix-indexed event logs is present in the four logs derived from  
_BPIC2017_Refused_, with prefix lengths of [1; 6; 11; 16]. In the latter event logs, the dissimilarity between the feature sets across the two runs increases with increasing length of the prefixes. This observation can be attributed to the effect of the increased dimensionality in the event logs with longer prefixes. In prefix-indexed event logs, weights or scores of important features change with increasing prefix length. **Observation 13:**_(Dis)similarity between feature sets that results in the first run of PFI, on one hand, and LR coefficients or XGBoost feature sets on the other, is independent of the preprocessing configuration. In nearly all cases, dissimilarity can be observed._  
After comparing the results of the first PFI run with LR coefficients and XGBoost feature sets (based on the total gain criterion), complete dissimilarity can be observed between the PFI feature sets and LR coefficients regardless of the preprocessing configuration. Moreover, when compared to XGBoost feature sets, event logs showing no similarity in a single-aggregated form showed more similarity in certain features in prefix-indexed form with increasing length of the prefixes. Examples of this observation include the  
Figure 9: XGBoost features importance (gain) for prefix length (16) _BPIC2017_Refused_. _Observation 14: Multicollinearity between PFI feature sets of all single-aggregated event logs is at high levels, and increases with longer prefixes in prefix-indexed event logs._  
Multicollinearity implies features with very high or complete correlations amongst each other. SHapley Additive exPlanations (SHAP).SHAP is an explanation method that belongs to the class of feature additive attribution methods [5]. These methods use a linear explanation model to compute the contribution of each feature to a change in the prediction outcome with respect to a baseline prediction. Afterwards, a summation of the contributions of all features approximates the prediction of the original model. To maintain comparability of the global XAI methods used in our experiments, we constructed a SHAP explainer model on training event logs independently of another SHAP explainer model constructed from relevant testing event logs. The observations in this section are obtained based on the training SHAP explainer model. _Observation 15: While comparing the two execution runs, the results do not depend on the preprocessing configuration; rather, they differ depending on the respective predictive model for which explanations are generated._  
Figure 10: PFI according to LR trained over _BPIC2017_Refused_ (preprocessed with prefix index configuration).