# XAI Systems Evaluation: A Review of Human and Computer-Centred Methods
## 4 Human-Centred Evaluation Methods
### Explanation Usefulness and Satisfaction  
The ultimate goal of any system is to satisfy a particular user need. Therefore, in a XAI system, it is also important to evaluate how useful explanations are to the user and how satisfied the user is with them. A large portion of literature follows a qualitative evaluation of user Satisfaction of explanations, using questionnaires and interviews [29; 41; 42]. One possibility to consider is to conduct expert case studies. Compared to lay people, experts have more knowledge on the subject around the system and can provide a more complete and in-depth opinion [43; 44].  
Some authors study the explanation Usefulness, which helps understand whether explanations are valuable to the user or not. For example, [PERSON] et al. [45] studied the Satisfaction level of expert translators when using either an intelligible or a non-intelligible version of the same system. The results show that the added explanations do not necessarily lead to a significant change in user experience. Participants stated that the intelligibility was only valuable when the information provided was beneficial to the translation process, or when it added information that the expert was not immediately aware of.  
Besides the Usefulness of explanations, it is also important to consider the amount of information those explanations present to the user. [PERSON] et al. [46] conducted a series of experiments focused on evaluating the effect of presenting different information to the user on the ability to fulfil a particular task. In a few cases, although users had access to either more information about the model or about its prediction process, they also were less able to identify and correct models' mistakes, seemingly due to information overload. [PERSON] and [PERSON] [29] also refer this information overload as an important problem to consider when developing systems for real-world applications, since those scenarios are usually more complex. On another work, [PERSON] et al. [41] evaluated a recommender system according to some explainability goals. One of those was efficiency, which can be defined as how good an explanation is in reducing the decision time. The authors concluded that explanations helped users decide more quickly, but they do not guarantee there would be no implications on the decision quality or the user Satisfaction.  
Despite the interpretability benefits of explanations, they also bring costs to the user. In the case that the explanations are too extensive, the user might need more time to process the information presented. [PERSON] et al. [47] investigated this aspect by conducting two studies on low-cost decision support systems: one on the comprehensibility and perceived cost of explanations and the other on the user desire for explanations. The results show that most users are interested in having a \"sufficient\" transparency level, while access to too much information leads to a viewing cost that outweighs the benefit of the explanations.  
[PERSON] et al. [42] studied the effectiveness of different types of explanations. On the one hand, the users had \"why\" explanations, which showed why the system behaved in a certain way. On the other hand, the users had access to \"why not\" explanations, which showed why the system did not behave in a certain way. Results showed that the former contributes to a better understanding and stronger feeling of trust on the model when compared to the latter. This strategy could be used when researchers want to compare case-based explanations versus counterfactual explanations, which correspond to the \"why\" and \"why not\" scenarios.  
Thus, Understandability and Usefulness are mutually complementary explanation's attributes. The latter enables us to assess an explanation's real benefit towards users' needs. Ultimately, the Usefulness of explanations may translate into gains in users' Task Performance, another metric we now describe.