## 5 Evaluation  
We conducted a large number of experiments to prove that our proposed framework can accurately detect redundant activity labels in event logs. Overall, two groups of experiments were performed. The first group of experiments compared the performance of our framework with the existing state-of-the-art method to detect redundant activity labels. The second group of experiments further analysed the effectiveness of our framework. To evaluate the performance of our framework, we apply the same evaluation matrix as found in [11], which is the standard f-score metric. Detection results fall into one of the following four categories: (1) true-positive (TP), where positive outcomes are correctly detected (the detected redundant activity label is actual redundant), (2) false-positive (FP), where negative outcomes are detected as positive (the detected redundant activity label is actually not redundant), (3) true-negative (TN), where negative outcomes are correctly detected (the detected non-redundant activity label is actually not redundant), and (4) false-negative (FN), where negative outcomes are falsely detected (the actual redundantactivity label is detected as non-redundant). Based on these four indicators, we calculate the following evaluation scores. Precision defines how many positive classes are detected correctly out of all positive detection. \[Precision=\frac{TP}{TP+FP} \tag{7}\]  
Recall indicates how many from all positive classes are detected correctly. \[Recall=\frac{TP}{TP+FN} \tag{8}\]  
The f-score is the harmonic mean of precision and recall. \[F\text{-}score=\frac{2\times Precision\times Recall}{Precision+Recall} \tag{9}\]  
Several publicly available datasets were utilised in the experiments. In total, our evaluation was based on four publicly available event logs:  
* Hospital Billing log ([[https://doi.org/10.4121/uuid:76c46b83-c930-4798-a1c9-4](https://doi.org/10.4121/uuid:76c46b83-c930-4798-a1c9-4) be9]([https://doi.org/10.4121/uuid:76c46b83-c930-4798-a1c9-4](https://doi.org/10.4121/uuid:76c46b83-c930-4798-a1c9-4) be9) 4 dfeb741 (accessed 7 April 2022)): An event log records processes related to billing medical services provided by a Dutch hospital. * Sepsis log ([[https://doi.org/10.4121/uuid:915d2](https://doi.org/10.4121/uuid:915d2) bfb-7e84-49 ad-a286-dc35f063a460]([https://doi.org/10.4121/uuid:915d2](https://doi.org/10.4121/uuid:915d2) bfb-7e84-49 ad-a286-dc35f063a460) (accessed 7 April 2022)): An event log records treatment processes of sepsis patients from a Dutch hospital. * Helpdesk ([[https://doi.org/10.4121/uuid:0c60](https://doi.org/10.4121/uuid:0c60) edf1-6f83-4e75-9367-4c63b3e9d5 bb]([https://doi.org/10.4121/uuid:0c60](https://doi.org/10.4121/uuid:0c60) edf1-6f83-4e75-9367-4c63b3e9d5 bb) (accessed 7 April 2022)): An event log contains the ticketing management process of the help desk in a software company in Italy. * BPI Challenge 2012 ([[https://doi.org/10.4121/uuid:3926](https://doi.org/10.4121/uuid:3926) db30-f712-4394-aebc-759760]([https://doi.org/10.4121/uuid:3926](https://doi.org/10.4121/uuid:3926) db30-f712-4394-aebc-759760) 70e91f (accessed 7 April 2022)): An event log of a loan application process in a Dutch financial institute. The details of all used event logs are presented in Table 1. Following the data preparation methods in [11], we randomly selected a certain amount of activity labels and randomly varied the percentage (i.e., 1% to 30%) of its events to simulate activity labels with both low-occurrence and high-occurrence frequency. In total, seven different settings were used on four event logs. For each setting, 5 rounds were performed, and the average results are reported. It is worth mentioning that the Sepsis event log also contains different variants of discharging a patient, which are \"_Release C_\", \"_Release D_\", and \"_Release E_\". They are regarded as redundant [11; 17]. Thus, the ground truth not only contains activity labels that we manually changed, but also consists of any pair of these three activity labels.