# Explainability of Predictive Process Monitoring Results: Can You See My Data Issues?
## 4 Experiments
### Experimental Setup
#### 4.1.1 Dimension 1: Data  
The experiments were carried on three real-life event logs, which are publicly available from the 4 TU Centre for Research Data [22]. One challenge in predictive process monitoring research is the limited availability of real-life event logs. Therefore, we depended on event logs available in Business Process Intelligence Challenges (BPIC) published by the 4 TU Centre for Research Data [22]. We discarded logs that did not include a business process and those which did not include both static and dynamic attributes [2]. After filtering these, we obtained eight event logs, from which we selected three. The three basic event logs [22] used are as follows:  
* **Sepsis:** This event log refers to the healthcare domain and reports cases of sepsis as a life threatening condition. * **Traffic Fines:** This event log is a governmental one extracted from an Italian information system for managing road traffic fines. Figure 3: Experimental Taxonomy. * **BPIC2017:** This event log records instances of loan application processes in a Dutch financial institution. These vary with respect to the referenced domain (healthcare, government, and banking), the number of cases (i.e., process instances), and the number of events per case. The chosen event logs further vary in the number of static and dynamic attributes, the number of categorical attributes, and as a result, the number of categorical levels available in each categorical attribute. We believe that these three logs represent a comprehensive basis for our experiments with respect to the five event logs that were left out. Choices in the data dimension are mainly impacted by the issues that need to be studied in **RQ1**. The focus of our study is to analyze explainability of outcome predictions, i.e., to explain predictions of how a running process instance will terminate. Outcome predictions answer a specific question driven by the labels of the process instances, which are stored either in the training part of the event log or in its relevant testing event log. Outcome prediction is usually a binary classification problem [2]. In order to use the three chosen event logs in our experiments, we applied the same labelling functions defined in [2] to classify each process instance into one of two classes, i.e., a binary classification task. Regarding _Sepsis_, patients were classified according to  
* whether a patient returned to the emergency room within 28 days of discharge in _Sepsis1_, and
* whether a patient was admitted to intensive care in _Sepsis2_. Process instances in _Traffic_fines_ are binary classified according to whether the fine was repaid in full or was sent for credit collection. Furthermore, instances from the _BPIC2017_ event log are classified by whether the loan application was accepted or refused, resulting in _BPIC2017_Accepted_ and _BPIC2017_Refused_ event logs. As a result of applying these labelling functions, the number of considered event logs increased from three to five. A predictive model built for this classification task was then used to predict the labels we created in this step. We chose event logs with different positive class ratios varying between <1% and 47% to study the effects of imbalanced event logs on explanations of different granularity. Table 4 shows basic statistics of the event logs used in our experiments. All event logs were cleaned and transformed according to the rules suggested by the framework presented in [2]. For example, traces were cut before the event used in the labelling step. Moreover, the event logs were enriched with derived attributes, e.g., the number of concurrently running cases at the time of executing the current event. Other examples of derived attributes include the position of an event in the case and the weekday, hour, and month, which were extracted based on the timestamp of each event. In order to avoid a dimensionality explosion after the encoding step, for each categorical attribute, category levels that appeared more than ten times were the only ones kept.