# XAI Systems Evaluation: A Review of Human and Computer-Centred Methods
## 6 Discussion
Therefore, continuous monitoring after deployment of XAI solutions is very important to ensure that the quality and validity of explanations evolve as expected across time and system usage. For evaluation methods in general, it is common to set a baseline scenario, used as a point of comparison to others with different conditions. The goal is to understand if these different conditions lead to a different outcome from the baseline case. The choiceof the baseline scenario can greatly impact the meaning of the results. The majority of approaches reviewed adopt a \"no explanation\" baseline, where the participants do not have access to the model explanations [31; 45]. In another case, one of the experiments described in [PERSON] et al. [46] asked users to complete a particular task before even consulting the model. The usage of different baselines adds another level of complexity to the discussion of experimental results from the same or different works, as there is the need analyse and compare them in a fair setting. As the reviewed works do not provide a clear view of what experimental settings are vital for a reliable and comparable comparison between them, we believe it should be a high priority for future works to clearly define the evaluation settings considered, while taking into account already available knowledge from other works within the same subject. This would enable more transparent and comparable research works for both publications' authors and readers. Despite the controlled nature of evaluation experiments, there might still exist certain human behavioural differences between scenarios that are not reflected on the evaluation results. In a study by [PERSON] et al. [75] about people's behaviour when presented with explanations, the results showed that the inclusion of explanations increased the participants' willingness to comply with a request, even if those explanations did not contain relevant information. [PERSON] et al. [7] denominated these explanations as \"placebic explanations\" and argued that placebo and real explanations could lead to a similar level of user trust in the system. The authors also stated that works on explainability techniques might benefit from using placebic explanations as a baseline scenario, since it would eliminate any influence the presence of explanations might have on the user. Another study by [PERSON] et al. [76] evaluated two commonly used techniques for evaluating XAI systems. One of the experiments conducted tested the usage of \"proxy tasks\", a simplified version of a real task for a particular system. The results support the hypothesis that the outcomes from proxy tasks may not be a good indicator for the performance of the XAI system in a realistic setting. Therefore, it is important to ensure that the metrics and measures used to evaluate experimental results are as representative as possible of the several aspects of the XAI system undergoing evaluation. One could argue that the limitations of existing evaluation methods could be a consequence of the nature of XAI techniques themselves. [PERSON] and [PERSON] [5] raised concerns regarding this same topic. Although efforts have been yield to approximate ML systems to their users through explainability, there is still a clear disconnection between technical XAI approaches and their real effectiveness in supporting users objectives for their particular use case. Although it is not considered on the proposed taxonomy, the kind of end-users and their respective needs from a XAI system is an important aspect that should be taken into account during the development and evaluation process. Users can differ not only on their role when interacting with the system, but also in their domain knowledge, expertise with ML systems or even in which environment will they use the XAI system. Furthermore, research also shows that issues arise when the assumptions underlying XAI technical approaches are detached from people's cognitive processes.