# A Goal-Oriented Evaluation Methodology for Privacy-Preserving Process Mining
## 4 Case Study
### Results Evaluation
(_Q2.1.2._) As seen from the results, DP-Decision Trees provide an effective trade-off among interpretability, privacy preservation, and computational efficiency. However, understringent privacy constraints, performance declines as a result of imprecise split decisions introduced by noise. Training maintains a high level of computational efficiency, even when operating at lower \(e\) values. The behavior of the model also exemplifies the bias-variance trade-off within the framework of differential privacy, necessitating careful management during parameter selection to optimize performance. (Q2.1.3.) We have visualized the inertia values with respect to the epsilon parameter in the DP-k-Means model results. A lower inertia value indicated that the event log instances were closer to their respective cluster centroids, suggesting more compact and well-defined clusters. That is desirable as it means the clusters are tight and compact. On the other hand, the results indicated that a higher degree of trace uniqueness within the event logs is associated with a more pronounced decline in the model runtime performance. (Q2.2.1.) Our _LaplacianDP_ findings revealed that different hyperparameter configurations with the data characteristics had different effects on algorithm behavior. Especially for the Road Traffic Fine event log data, the parameter configurations from the original publication had unfeasible runtimes within the _PRIPEL_ framework. Since the smaller \(e\) parameter introduces more privacy, precision reduces due to this fact. It indicates that the discovered model started capturing noisy events so overfitting occurred. Generalization also decreased in parallel. In addition to these, for the runtime performances, increasing the \(e\) parameter reduces the runtime in inverse proportion. In the _PRIPEL_ original publication [25], the authors considered only the Sepsis event log for their experiments. (Q2.2.2.) In the _SaCoFa_ model experiments, various hyperparameter configurations, in conjunction with the underlying data characteristics, exerted distinct influences on algorithmic behavior. Especially for the Road Traffic Fines event log data, the parameter configurations (from original publication) had unfeasible runtimes within the _PRIPEL_ framework, so we discarded it from our experiments. Thus, the data instance count is the main issue to consider for this model's contextual information-carrying nature. We concluded that the _PRIPEL_ framework is unfeasible to use with event logs that have a high count of events. (Q2.2.3.) As validated with the _TraVaS_ PPPM model quality results, the effectiveness of this model is considerably influenced by the privacy parameters. It may struggle with event logs that contain many infrequent trace variants. While _TraVaS_ had runtimes that are efficient for event logs that have a high count of instances, its quality is affected by the high trace uniqueness characteristics. In order to extend and support our fine-grained analysis results, we validated and discussed the result and time utility qualities of each PPPM model under their respective categories by using different statistical analysis techniques. To align with our subgoals, we applied statistical tests for each model within its category to gain a further comprehensive and competitive perspective. To evaluate the practical equivalence of alternative models relative to a predefined baseline, we employed the Two One-Sided Tests (TOST) procedure, a statistical framework specifically designed to formally assess equivalence within a defined tolerance margin [37]. In contrast to conventional hypothesis testing, which seeks to identify statistically significant differences, the TOST procedure explicitly evaluates whether a model's performance is statistically equivalent to that of a baseline, within a user-specified range of practical relevance known as the equivalence margin (\(\triangle\)).