# XAI Systems Evaluation: A Review of Human and Computer-Centred Methods
## 5 Computer-Centred Evaluation Methods  
Human-centred methods are more commonly applied to evaluate XAI systems than Computer-centred, partially due to the complexity associated with judging certain properties such as Trust or Understandability from a non-human point of view. Nevertheless, Human-centred approaches also have identifiable drawbacks. In particular, [PERSON] [8] indicates that the human bias towards simpler explanations can result in more persuasive explanations as opposed to transparent systems [3]. As such, explanation evaluation methods disconnected from the human user must also be considered. [PERSON] and [PERSON] [9] categorize these methods as functionally grounded evaluations, suggesting that they are better employed after human validation of the model or system, with the exceptions of when these are not yet mature enough or human experimentation is unethical.  
Two commonly found properties used to evaluate this type of explanations are their **Interpretability** and **Fidelity**. Interpretability can be described as the ability to explain in understandable terms to a human [9], while Fidelity indicates how accurately a model's behaviour is described by an explanation [6]. These properties can be further subdivided into more specific characteristics. More specifically, within the Interpretability category, it is also possible to identify the **Broadness** sub-property, which measures how generally applicable an explanation is [10] as well as the **Simplicity** (also known as Parsimony) and **Clarity** characteristics, with the former indicating if an explanation is presented in a compact or not too complex form and the latter the unambiguity associated with an explanation [6]. As for the Fidelity property, we can sub-divide it into the **Soundness** and **Completeness** categories. The first informs on how truthful an explanation is to the task model, while the last indicates if an explanation provides sufficient information to compute the output for a given input [6]. According to [1], Completeness is a measure of how many input features that affect the decision process are captured in an explanation.  
These properties are, at times, also accompanied by other ones aimed at evaluating the XAI system as well, such as the Sensitivity to input perturbation and model parameter randomization. [PERSON] et al. [49] did precisely this, evaluating the LIME, anchors, SHAP, LORE, ILIME and MAPLE Interpretability frameworks, albeit with different methods and metrics. More specifically, they based their work on the three axioms proposed by [PERSON] [50]. These pertain to the **Identity** (identical instances must have identical explanations), **Stability** (instances belonging to the same class must have comparable explanations) and **Separability** (dissimilar instances must have dissimilar explanations) of the XAI system. It should be noted that these proprieties could be considered ramifications of the Soundness sub-category. Thus, we considered that including a third hierarchical level in our proposed taxonomy would bring an unnecessary level of complexity that could compromise its practical use.  
Given this, we considered that the best approach to structure the Computer-centred evaluation methods would be to follow the categorization already proposed by [PERSON] et al. [10]. Sections 5.1 and 5.2 describe each evaluation category considered in the proposed taxonomy for the Computer-centred methods, namely Interpretability and Fidelity, respectively. These descriptions were based on explainability concepts presented in previous works [6; 9; 10] and adapted to our taxonomy vision. The most relevant evaluation methods for each category are briefly presented and discussed, a summary of these methods and respective categorization also presented in Section 5.3.