## 5 Evaluation
### Examples of Model Visualisations  
Figure 3 shows an example of the global-level visualisations obtained for the BPLC2015 event log at prefix length 10. Several insights regarding the behaviour of the model trained and tested using this event log can be drawn from this visualisation. A first quick comparison of the two graphs clearly reveals that the features determining the positive label (label 1)  
Figure 2: Predictive performance of RIPPER against ML-based models. differ substantially from the ones determining the negative label (label 0). It appears that, for the positive label, there are basically three groups of features determining the positive outcome: (i) a first cluster involving the co-occurrence of the length of the eighth event to be high (Time8=Long) and the activity label of the seventh event (Activity7) to be enter send date retrieving missing data; (ii) a second cluster involving the occurrence of specific values for the activity label of the seventh event and the case-level attribute parts; and (iii) a third cluster involving a larger combination of values of several features. Regarding the latter, the occurrence of the activity enter send date acknowledgement in the second position in a case appears to be a discriminant on the label: when this happens, the outcome is in fact more likely to be positive. There is also a fourth cluster of rule antecedents involving the features SUMleges and requestComplete, which are both features derived from case-level attributes. The negative label global-level model visualisation shows only one cluster involving a high number of features. Considering the number of rules in which the features are involved, i.e., the size of the nodes in the graph, the strongest predictors of the negative outcome appear to be the non-occurrence of the activity enter send date acknowledgement as the second event in cases for which the case-level attribute requestComplete evaluates to True. Figure 4 shows an example of the global-level visualisations obtained for the BPIC 2017 event log. Given the nature of the prediction for this log, it should not surprise that most features appearing in these visualisation derive from event-level attributes (purple nodes). These in fact often capture the characteristics of the loan request, e.g., the loan amount requested (OfferedAmount) or the amount of the monthly instalments (MonthlyCost). More in detail, for the negative label, it can be noted that there are two clusters of rule antecedents: the first cluster (left hand side) involves requests rejected characterised by a medium amount requested, while the second one (right hand side) applies to loans for which the amount requested is high. For instance, high amount loan requests appear to be rejected when the number of instalments (NumberTerms) is medium and the monthly instalments (MonthlyCost) are not large. Figure 5 and 6 show examples of the local-level visualisations obtained for traces where the predicted outcome matched the actual one. In both examples, it is possible to clearly identify the rules that have been activated to support the prediction of the positive label. Specifically, Figure 6 shows a case in which the positive label is predicted. Based on the global level explanation, this case has a predictive positive label because it has the case-level features defining the fourth cluster discussed above, i.e., derived from case attributes SUMleges and requestComplete (which activate Rule 1). However, the local-level explanation also shows many additional rules that are activated and that may help explaining the positive label. An analysis of these rules can be used by a domain expert decision-maker to better characterise the particular case at hand, deciding for instance whether the positive level predicted should be trusted or not in a specific decision-making context.