## 5 Evaluation Measures
### Usability
#### 5.2.3 Task Performance and Cognitive Workload  
Satisfaction is not the only goal in usability engineering; improving task performance is equally important. According to ISO 9241-11, task performance is defined by two main components: effectiveness and efficiency. Similarly, [PERSON] and [PERSON] (2018) define performance in the context of XAI as the degree of success with which the human-AI system effectively and successfully conducts the task for which the technology is designed. With regard to human-AI collaboration, performance can refer to different aspects. It can relate to the technical system, such as being measured by the model's accuracy in making correct decisions or predictions. Additionally, it can also refer to the user's performance in utilizing the system's output. However, the performance of the system often cannot be separated from the performance of the user, and vice versa. In such cases, task performance refers to the joint effectiveness of both the AI and the user working together. Effectiveness refers to the accuracy and completeness with which users perform tasks and achieve their goals when interacting with a system. The specific operationalization of this construct depends on the nature and structure of the task or goal. Most commonly, effectiveness is measured by the success rate in completing tasks or sub-tasks, often quantified by the number of successfully accomplished trials within a specific time period (2018). For example, in game-based applications like chess, task effectiveness can be assessed using metrics such as winning percentage and percentile ranking of player moves (2018). In the context of decision-support systems, effectiveness can be measured by how much the decision-making process is improved [132]. Analogous effectiveness in recommender systems can be measured by the percentage of cases in which the user finds a suitable item. In cases where the goal is to explain the system behavior, effectiveness can be measured by the percentage of accurate user predictions of the system's output, typically evaluated through metrics like the number of hits, errors, and false alarms [60], [130] (as outlined in previous sections, this approach also uses a metric to measure understandability indirectly). In our sample, various studies evaluate effectiveness in the context of explainability. Theoretically, [PERSON] et al. [15] define the effectiveness of explanations as helping users make good decisions. On an empirical level, various studies explore how well different explanation methods help users reach their goals in different contexts. [PERSON] et al. [49], [PERSON] et al. [57], [PERSON] [37], [PERSON] et al. [65], [PERSON] et al. [68], and [PERSON] et al. [69] address effectiveness in the context of decision- and prediction-support systems, but they use different operationalization approaches. [PERSON] [49] employs an objective measure, evaluating effectiveness by counting the number of times a correct decision is made. Similarly, [PERSON] et al. [65], [PERSON] et al. [68], [PERSON] et al. [69], and [PERSON] et al. [61] measured the effect of explanations on the user's accuracy of AI-assisted prediction/classification tasks. In contrast, [PERSON] et al. [57], [PERSON] [37], and [PERSON] et al. [63] focus on softer, more subjective, aspects of effectiveness, such as whether users make _good decisions_[57], _better decisions_[37], or _improved decision-making_[63]. To assess these factors, they use subjective measures based on self-reports, asking participants to rate statements like _\"The app helps me make better medical choices\"_[57], _\"The explanation provided contains sufficient information to make an informed decision\"_[37], and _\"This explanation-component improves my decision-making process\"_[63]. [PERSON] et al. [41] use a similar approach to measure effectiveness in the case of explainable music recommender systems, using the item _\"The recommender helped me find good songs for my playlist\"_ to operationalize the construct.