Why should I trust your explanation? An evaluation approach for XAI methods applied to predictive process monitoring results
Creators: name: Elkhawaga, Ghada
name: Elzeki, Omar M.
name: Abu-Elkheir, Mervat
name: Reichert, Manfred
Publisher: name: Universität Ulm
Year: 2024
Subjects: subject: Explainability
subject: Functionally grounded evaluation
subject: Global explainability methods
subject: Predictive process monitoring
subject: Proccess mining
subject: XAI
subject: DDC 004 / Data processing &amp; computer science

description: As a use case of process mining, predictive process monitoring (PPM) aims to provide information on the future course of running business process instances. A large number of available PPM approaches adopt predictive models based on machine learning (ML). With the improved efficiency and accuracy of ML models usually being coupled with increasing complexity, their understandability becomes compromised. Having the user at the center of attention, various eXplainable artificial intelligence (XAI) methods emerged to provide users with explanations of the reasoning process of an ML model. Though there is a growing interest in applying XAI methods to PPM results, various proposals have been made to evaluate explanations according to different criteria. In this article, we propose an approach to quantitatively evaluate XAI methods concerning their ability to reflect the facts learned from the underlying stores of business-related data, i.e., event logs. Our approach includes procedures to extract features that are crucial for generating predictions. Moreover, it computes ratios that have proven to be useful in differentiating XAI methods. We conduct experiments that produce useful insights into the effects of the various choices made through a PPM workflow. We can show that underlying data and model issues can be highlighted using the applied XAI methods. Furthermore, we could penalize and reward XAI methods for achieving certain levels of consistency with the facts learned about the underlying data. Our approach has been applied to different real-life event logs using different configurations of the PPM workflow. Impact Statement—As ML models are used to generate predictions for running business process instances, the outcomes of these models should be justifiable to users. To achieve this, explanation methods are applied on top of ML models. However, explanations need to be evaluated concerning the valuable information they convey about the predictive model and their ability to encode underlying data facts. In other words, an explainability method should be evaluated concerning its ability to match model inputs to its outputs. Our approach provides a means to evaluate and compare explainability methods concerned with the global explainability of the entire reasoning process of an ML model. Based on experimental settings, where each step of the PPM workflow is changeable, we could study the ability of our approach to evaluate different combinations of data, preprocessing configurations, modeling, and explanation methods. This approach allows an understanding of which PPM workflow configurations increase the ability of an explanation method to make the prediction process transparent to users.