## 5 Evaluation
### Datasets, Model Parameters, and Hyper-Parameters  
We consider six event logs publicly available at [https://data.4 tu.nl/](https://data.4 tu.nl/) (accessed on 27 April 2022) published by the Business Process Intelligence Challenge in 2011, 2015 (Note that there are five different instances of this event log, with data from different municipalities. We consider the instance numbered as 1, 2, and 3), 2012 and 2017. The BPIC 2011 event log is about a diagnosis and treatment process in a Dutch academic hospital. The BPIC 2015 event logs are from a process of managing building permit requests at different Dutch municipalities. The BPIC 2012 and BPIC 2017 event logs are from a process of managing loan requests at a Dutch financial institution. These logs have been chosen because they contain an outcome label and have been used by previous research on outcome-based process predictive monitoring [10]. Table 1 shows the descriptive statistics of these event logs. Note that event logs differ widely for number of activities, number of attributes, and label class proportion.  
In the BPIC 2011 and BPIC 2015 event logs, given a linear temporal logic constraint predicated on the order and occurrence of activities, the outcome label is set to 1 for cases satisfying the constraint and to 0 for other cases. In the BPIC 2015 event logs, the constraint is that activity'send confirmation receipt' is eventually followed by the activity'retrieve missing data'. In the BPIC11 event log, the constraint is that either the activity 'ca-19.9 tummormarker' or 'ca-125 mbv meia' occur in a trace. In the BPIC 2012 and BPIC 2017 event logs, the outcome label captures whether a loan request is eventually accepted or not.  
The value of hyper-parameters has a great influence on the performance of a model. To improve the performance of the model, we tune the hyper-parameters with their range using Bayesian Optimisaion. These hyper-parameters and search spaces are listed in Table 2.  
\begin{table}
\begin{tabular}{l c c c c c c c c} \hline  & & &
\begin{tabular}{c} **\# Categorical** \\ **Case** \\ \end{tabular} & **\# Continuous** & **\# Categorical** & **\# Continuous** \\ \multicolumn{1}{c}{\multirow{2}{*}{**\# Activities**}} & \multirow{2}{*}{**\# Cases**} & **\# Variants** & **Case** & **Case** & **Event** & **Event** \\  & & & & **Attributes** & **Attributes** & **Attributes** & **Attributes** & **Attributes** \\ \hline BPIC 2011 & 677 & 1143 & 981 & 4 & 6 & 5 & 0 & 683 & 460 \\ BPIC 2015\_1 & 289 & 1199 & 1100 & 8 & 1 & 3 & 0 & 910 & 289 \\ BPIC 2015\_2 & 304 & 832 & 828 & 8 & 1 & 3 & 0 & 670 & 162 \\ BPIC 2015\_3 & 277 & 1409 & 1349 & 8 & 1 & 3 & 0 & 1122 & 287 \\ BPIC 2012 & 36 & 13,087 & 4366 & 1 & 1 & 2 & 0 & 10.844 & 2243 \\ BPIC 2017 & 26 & 31,509 & 4047 & 1 & 5 & 2 & 0 & 14.281 & 17.228 \\ \hline \end{tabular}
\end{table}
Table 1: Descriptive statistics of event logs used for evaluation.  
We consider prefixes between 5 and 40 with a gap of 5 for all event logs. For example, if prefix length is equal to 5, then RF, XGB, and RIPPER are trained and tested on a dataset containing all the prefixes of length 5 in the event log, that is, partial traces considering the first 5 events of traces with at least 5 events. To cope with sampling bias, we consider five-fold cross-validation. The proportion of train and test set sampling in every replication of the experiment is 70% to 30%. The code to reproduce the experiments is publicly available at [[https://github.com/eekfkskgus/Rule_based_classification_for_interpretability](https://github.com/eekfkskgus/Rule_based_classification_for_interpretability)]([https://github.com/eekfkskgus/Rule_based_classification_for_interpretability](https://github.com/eekfkskgus/Rule_based_classification_for_interpretability)) (accessed on 27 April 2022).