## 5 Evaluation Measures
### Usability
#### 5.2.2 Utility and Suitability  
Usability engineering emphasizes the teleological nature of human behavior, where people use systems not only for enjoyment but also as tools to achieve instrumental goal. This instrumental perspective is reflected in the theoretical constructs of pragmatic quality [123] and perceived utility [124]. Pragmatic quality is defined as the perceived ability of a system to assist users in completing tasks and achieving their so-called do-goals or action-oriented objectives [123]. Related to this, perceived utility describes the user's subjective assessment of how useful a product, system, or service is in helping them achieve their specific goals. This perception is influenced not only by the actual functions of the product but also by the user's expectations, needs, goals, individual experiences, and the context of use. Pragmatic quality [123] and perceived utility can be evaluated by different but similar constructs, such as the explanation's helpfulness, usefulness, personal relevance, and actionability. Helpfulness as a theoretical construct refers to the degree to which explanations provided by AI systems are perceived as valuable, informative, and supportive by users in aiding their decision-making or understanding of the system's outputs. In the literature, evaluating helpfulness is often based on self-reports, where users rate to what extent explanations are tailored to specific tasks [42; 44; 48; 50; 125; 126; 127; 128]. In our sample, various studies [35; 42; 44; 60; 61] have evaluated the helpfulness or usefulness of AI systems' explanations. For instance, [PERSON] et al. [59] used a 5-point Likert scale to measure the perceived helpfulness of explanations for both correct system classifications and misclassifications. Similarly, [PERSON] et al. [42] evaluated helpfulness by asking participants to rate the statement _\"This AI helped me assess the percent fat content\"_, on a 5-point Likert scale. They used this rating as an indicator of the usefulness of the explanations provided. Concerning causality, [PERSON] et al. [44] measured helpfulness in the questionnaire via the item _\"I found the explanations helped me to understand causality\"_. [PERSON] et al. [36] evaluated perceived utility by asking participants to rate the usefulness of the explanation element on 5-point Likert scale items. Similarly, [PERSON] et al. [60] evaluated the perceived usefulness quantitatively in a post-task survey, where participants indicated whether they found the model's assistance helpful. [PERSON] et al. [33] evaluated utility using the cost-benefit ratio, including the item _\"Do you feel the effort you put into adjusting the computer was worth the result?\"_ in their post-task survey. In contrast, [PERSON] et al. [61] used a free-form questionnaire in their user study about pipeline-generated visual explanations, where participants were asked to write about their views on usefulness. Studying how data scientists use XAI tools, [PERSON] et al. [30] also asked the participants whether the explanations were useful and, if so, how they would use them in their typical work. [PERSON] et al. [31] asked the participants of the user study to report their subjective perception of tutorial usefulness as reported in the exit surveys. Another indicator of usefulness is actionability, which means the ability of users to apply the explanations within their specific context. [PERSON] et al. [35] suggest measuring actionability through the item _\"The explanation is actionable, that is, it helps me know how to use the [software, algorithm, tool]\"_. This concept emphasizes that a useful explanation should provide information that guides users toward reaching their goals. Closely related to this is the concept of \"easy-to-use\", which also assesses how well the explanation facilitates the process of making use of explanations.