The current set of XAI evaluation metrics, while extensive, is not without its biases and limitations. Here are some key issues and potential solutions:
Selection Bias: Many studies rely on a limited set of metrics to evaluate XAI methods, which can lead to selection bias. This bias occurs when the chosen metrics do not comprehensively represent the criteria of faithfulness, robustness, and complexity. To address this, researchers should adopt a broader range of metrics that capture diverse perspectives on each criterion. The LATEC framework's approach of utilizing 20 distinct metrics is a step in the right direction, but continuous expansion and refinement of this metric set are necessary.
Metric Sensitivity: Different metrics can exhibit varying sensitivities to the same XAI method, leading to inconsistent evaluations. For instance, a method may perform well on one metric but poorly on another, creating confusion about its overall effectiveness. To mitigate this, a meta-evaluation approach can be employed, where the performance of each metric is analyzed across multiple XAI methods and datasets. This analysis can help identify which metrics are more reliable and consistent, allowing practitioners to prioritize those in their evaluations.
Lack of Ground Truth: The absence of a clear "ground truth" for XAI explanations complicates the evaluation process. Metrics often rely on approximations of faithfulness and robustness, which may not align with human interpretations of explainability. To address this limitation, researchers can explore the use of human-in-the-loop evaluations, where human judgments are incorporated into the assessment process. This can help validate the effectiveness of the metrics and ensure they align with user expectations.
Overfitting to Metrics: There is a risk that XAI methods may be optimized to perform well on specific metrics rather than providing genuinely interpretable explanations. This overfitting can lead to misleading evaluations. To counteract this, it is essential to adopt an "aggregate-then-rank" approach, as suggested in the LATEC framework. By focusing on rankings rather than raw scores, the evaluation can reduce the influence of outlier metrics and provide a more balanced view of method performance.
Contextual Relevance: The effectiveness of XAI methods can vary significantly depending on the context in which they are applied. Metrics that work well in one domain may not be suitable in another. To enhance the applicability of evaluation metrics, researchers should consider developing context-specific metrics that account for the unique characteristics and requirements of different application areas.
By addressing these biases and limitations, the evaluation of XAI methods can become more comprehensive and unbiased, ultimately leading to more reliable and interpretable AI systems.