## 5 Evaluation Measures
### Usability
#### 5.2.3 Task Performance and Cognitive Workload
Evaluating different kinds of explanations supporting humans in classification tasks, [PERSON] et al. [53] and [PERSON] et al. [31] quantified effectiveness via the percentage of instances correctly labeled by participants. [PERSON] et al. [76] also addressed a classification task in which physicians diagnosed cancer with the help of AI. They operationalized effectiveness as participants' task performance accuracy. This was measured under different conditions (with and without providing explanations) by asking participants _\"What is your final judgment on the tumor grade?\"_  
In his work, [PERSON] [34] evaluates the effectiveness of explanations by focusing on understandability. Since the primary design goal was to improve users' understanding, he uses measures of understandability as a proxy for effectiveness, studying how well the explanation approach enables the users to understand the AI decisions and to assess whether those decisions are appropriate. In a similar approach, [PERSON] et al. [43] assess the effectiveness of different explanation concepts by comparing how well participants understand the algorithm and how much they trust it, with each group receiving access to a specific explanation interface, along with a control group. [PERSON] et al. [40] conceptualize effectiveness as a multi-dimensional construct to evaluate the XAI framework LIME from a user's perspective. According to [PERSON] et al. [40], effectiveness is associated with the complete and accurate completion of tasks, how well users achieve their goals using the system, and the effective mitigation of potential negative consequences if the task is not completed correctly. _Efficiency_ refers to the resources required to achieve specific goals or complete a specific task [64]. It is usually measured by the time a user needs to successfully complete a task, using a timer/stopwatch or a log of time stamps representing when the user starts and finishes the task [110]. This is also reflected in our sample where efficiency was typically measured by response time [51; 59], reaction time [65], annotation time [53], second-per-task completion [68], interaction time [33], time spent using the tool [43], or faster decision-making, [15]. An objective measure was also used by [PERSON] et al. [66], evaluating the explanation interface concept using a fictive multi-round game. They assessed efficiency by the number of moves participants needed to solve the task. Similarly, [PERSON] et al. [33] also counted the number of interactions with the system. In contrast, [PERSON] et al. [58] used a subjective measure in the context of explainable recommender systems. They evaluate efficiency by asking participants to rate the statement \"_This explanation helps me determine more quickly how well the recommendations match my interests_\". _Mental or cognitive workload_ is another important theoretical construct for measuring efficiency in terms of the mental effort required to perform the task and the mental resources, such as attention, memory, and cognitive abilities, demanded to reach the goal [133]. A high cognitive workload is linked to stress, increased mental activity, and information-processing behavior. This is reflected in physiological activities, such as changes in heart rate, brainwave activity, skin conductance, pupil size, and saccadic movements. Hence, measuring these changes using EEG, GSR, or eye-tracking systems is often used as an objective operationalization of the theoretical construct [134; 135]. Another approach to measuring users' cognitive load is to capture the log reading time in memorizing explanations [133]. Another approach is to use subjective measures for operationalization by collecting self-reported data, where individuals assess their own perceived cognitive workload [121]. The NASA Task Load Index (NASA-TLX) is a widely used subjective rating scale for measuring the user's perceived workload during interaction task performance or in post-task surveys [136].