###### Abstract  
Recent advances in technology have propelled Artificial Intelligence (AI) into a crucial role in everyday life, enhancing human performance through sophisticated models and algorithms. However, the focus on predictive accuracy has often resulted in opaque black-box models that lack transparency in decision-making. To address this issue, significant efforts have been made to develop explainable AI (XAI) systems that make outcomes comprehensible to users. Various approaches, including new concepts, models, and user interfaces, aim to improve explainability, build user trust, enhance satisfaction, and increase task performance. Evaluation research has emerged to define and measure the quality of these explanations, differentiating between formal evaluation methods and empirical approaches that utilize techniques from psychology and human-computer interaction. Despite the importance of empirical studies, evaluations remain underutilized, with literature reviews indicating a lack of rigorous evaluations from the user perspective. This review aims to guide researchers and practitioners in conducting effective empirical user-centered evaluations by analyzing several studies; categorizing their objectives, scope, and evaluation metrics; and offering an orientation map for research design and metric measurement. AI transparency; explainable artificial intelligence (XAI); XAI evaluation procedure; user-centered evaluation +
Footnote †: journal: applied sciences  
research methods, scales, and probing concepts from psychology and human-computer interaction [8; 14; 15; 16]. Empirical studies evaluating explanations in AI are labor-intensive and require careful planning to ensure that they are rigorous and valid [3]. For example, in their literature review, [PERSON] et al. [17] noted that only 5% of studies conducted an empirical evaluation. Similarly, the literature review by [PERSON] et al. [18] found that 32% of studies conducted no evaluation, 59% conducted only an informal user study, and only 9% conducted thorough evaluations with well-defined metrics. In the most recent literature review, [PERSON] et al. [19] highlighted that 33% of the research was evaluated using anecdotal evidence, 58% applied quantitative evaluation, 22% evaluated human subjects in a user study, and 23% of the research was evaluated using domain experts, i.e., application-grounded evaluation. The immaturity of this subject area is also reflected in the lack of literature reviews on empirical evaluation studies. While various surveys, such as [5; 6; 8; 10; 15; 17; 18], provide a sound overview of formal evaluations, empirical methodologies, and procedures are only marginally discussed. Without a clear, systematic understanding of the different goals, methods, and procedures used in evaluating explanation systems, it is difficult to establish best practices, standards, and benchmarks. The goal of this exploratory scoping review [20] is to inform and sensitize researchers and practitioners on how to conduct empirical evaluations effectively and rigorously. To achieve this, we analyzed the most relevant papers on XAI evaluation studies from prominent academic databases. The papers we analyzed demonstrate that the scope of explainable AI systems is quite broad, both in terms of domains (ranging from supporting engineers in debugging AI systems to helping consumers understand why certain product recommendations are made) and explanation objectives (ranging from enhancing understanding over improving task performance towards fostering trust). Through this review, we aimed to identify common patterns and recognize the essential elements that need to be considered when planning and conducting empirical evaluations of explanation systems. In our analysis, we categorized the objectives, scope, and evaluation metrics used in the studies. We also categorized the procedures and evaluation methods that were applied.