# XAI Systems Evaluation: A Review of Human and Computer-Centred Methods
## 4 Human-Centred Evaluation Methods  
Human-Centred Evaluation methods follow a human-in-the-loop approach, which involves the interaction of individuals with the system to test one or more of its properties. The main goal is to provide insights on the expected end-users' perspective of the system, by highlighting strengths and weaknesses of the XAI method and its application, as well as possible opportunities for improvement. Although literature has a high diversity of techniques, there are some that are more widely used than others. Firstly, Think-Aloud is a method in which participants can freely express any thought regarding the task at hand, which might not be directly related to XAI explanations. A similar, yet more focused in explanations methods, is User Self-Explanation where participants expose their thoughts in a more structured manner, particularly towards explaining their reasoning in particular actions. \begin{table}
\begin{tabular}{p{142.3 pt} p{142.3 pt}} \hline \hline
**Category** & **Sub-Category** \\ \hline
**Trust**: a variable factor shaped by user interaction across time and usage, which affects how comfortable the user is when using the XAI system. User perception influences its beliefs on the XAI system outputs. & **User’s Perceived System Competence**: depicts the user position on how capable an ML system is when solving a particular task. \\ \hline
**User’s Compilance with the System**: focused on understanding if the user would rely on the system’s decision or not to act upon a task. \\ \hline
**Evolution of User Trust**: represents how a user’s Trust can vary across time and usage of a particular ML system. \\ \hline
**Explanation Usefulness and Satisfaction**: two inherently connected aspects relevant to assess user experience. The same explanation can imply different levels of Usefulness, depending on the information revealed, and can also lead to a different level of user Satisfaction. & **User’s Perceived Understandability**: depicts the user understanding of system’s underlying functions. \\ \hline
**Understanding**: the ability to outline the relation between the input and output of a particular system with respect to its parameters. It is usually defined as a user’s mental model of the system and its underlying functions. & **User Prediction of Model Capture**: focused on understanding if the user is correctly able to identify the scenarios where the system fails a particular task. \\ \hline
**Performance**: the performance of ML systems usually depends not only on the models but also on their respective users. Evaluating the performance of both agents and their interaction is essential to assess the expected performance on real scenarios. \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Human-centred evaluation methods: categories and sub-categories. \begin{table}
\begin{tabular}{p{142.3 pt} p{142.3 pt}} \hline \hline
**Category** & **Sub-Category** \\ \hline
**Interpretability**: implies that the explanation should be understandable to humans, being important to manage the social interaction of explainability. & **Broadness**: describes how generally applicable is an explanation. \\ \hline
**Simplicity**: implies that the explanation is presented in a simple and compact form. & **Simplicity**: implies that the explanation is presented in a simple and compact form. \\ \hline
**Fidelity**: implies that the explanations should accurately describe model behaviour in the entire feature space, being important to assist in verifying other model desiderata or discover new insights of explainability. & **Completeness**: implies that the explanation describes the entire dynamic of the ML model. \\ \hline
**Soundness**: describes how correct and truthful is an explanation. & **Soundness**: describes how correct and truthful is an explanation. \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Computer-centred evaluation methods: categories and sub-categories.