The LATEC evaluation scheme, while robust in its quantitative analysis of XAI methods, can be significantly enhanced by integrating human-centric assessments that focus on interpretability and usefulness. To achieve this, several strategies can be employed:
User Studies: Conducting user studies involving domain experts and end-users can provide qualitative insights into how well different XAI methods communicate their reasoning. Participants can evaluate the interpretability of saliency maps and other outputs based on their understanding and the context of the application. This feedback can be systematically collected and analyzed to inform the evaluation metrics.
Cognitive Load Assessment: Incorporating measures of cognitive load can help assess how easily users can understand and utilize the explanations provided by different XAI methods. Techniques such as think-aloud protocols or eye-tracking can be employed to gauge how users interact with the explanations and where they encounter difficulties.
Task-Based Evaluation: Implementing task-based evaluations where users are asked to perform specific tasks using the explanations generated by various XAI methods can provide insights into the practical usefulness of these methods. Metrics such as task completion time, accuracy, and user satisfaction can be collected to evaluate the effectiveness of the explanations in real-world scenarios.
Diversity of User Perspectives: It is essential to consider a diverse range of users, including those with varying levels of expertise and different backgrounds. This diversity can help ensure that the evaluation captures a broad spectrum of interpretability and usefulness, making the findings more generalizable.
Integration of Qualitative Metrics: Alongside the quantitative metrics already established in LATEC, qualitative metrics such as user satisfaction scores, perceived clarity, and trust in the model can be integrated. These metrics can be derived from surveys or interviews conducted with users after they interact with the XAI outputs.
By incorporating these human-centric assessments, the LATEC evaluation scheme can provide a more holistic view of XAI methods, ensuring that they are not only technically sound but also practically useful and interpretable for end-users.