Evaluating Input Feature Explanations through a Unified Diagnostic Evaluation Framework (2025)

Abstract: Explaining the decision-making process of machine learning models is crucial for ensuring their reliability and transparency for end users. One popular explanation form highlights key input features, such as i) tokens (e.g., Shapley Values and Integrated Gradients), ii) interactions between tokens (e.g., Bivariate Shapley and Attention-based methods), or iii) interactions between spans of the input (e.g., Louvain Span Interactions). However, these explanation types have only been studied in isolation, making it difficult to judge their respective applicability. To bridge this gap, we develop a unified framework that facilitates an automated and direct comparison between highlight and interactive explanations comprised of four diagnostic properties. We conduct an extensive analysis across these three types of input feature explanations – each utilizing three different explanation techniques–across two datasets and two models, and reveal that each explanation has distinct strengths across the different diagnostic properties. Nevertheless, interactive span explanations outperform other types of input feature explanations across most diagnostic properties. Despite being relatively understudied, our analysis underscores the need for further research to improve methods generating these explanation types. Additionally, integrating them with other explanation types that perform better in certain characteristics could further enhance their overall effectiveness.

## introduction input feature explanations reveal how a model makes decisions based on a specific input. among these, the most widely used explanation type is token explanations (tokenex), which for natural language understanding tasks provide importance scores for the tokens of the input, using methods such as shapley values (lundberg and 1 our code can be found at https://github.com/copenlu/aunified-framework-for-input-feature-explanations a man playing guitar on stage. a man playing banjo on the floor. ## a unified automated evaluation framework for highlight explanations according to the established taxonomy of evaluation approaches for explainability methods, evaluations can be functionally grounded, humangrounded, or application-grounded @xcite . our work focuses on automated functionally-grounded evaluation, which serves as a foundation for the other evaluation types as it enables the rapid, scalable comparison across explainability methods, essential for the growing number of new techniques. to this end, we present a unified framework comprised of four widely employed diagnostic properties: faithfulness, agreement with human annotations, simulatability, and complexity. this section formally introduces them and outlines the extensions that allow for their application across different input feature explanation types. ## results and discussion we now present the results of our unified evaluation framework ( §2) illustrated in fig. 2 . they include explanation methods of types spanintex, tokenintex, and tokenex ( §3.2), two models ( §3.3), two datasets ( §3.1), and three base explanation techniques per explanation type ( §3.2). for simulatability, we select the results of i sym , as this form avoids repeating the input text and increasing the input size substantially. for agreement with human annotations, we select the token-level results as they are present for all explanation types. §f lists detailed results per property. ## conclusion we introduced a unified evaluation framework for input feature attribution analysis to guide the principled selection of the most suitable explainability technique in practical applications. our analysis outlines the diverse strengths and trade-offs among tokenex, tokenintex, and spanintex. our findings particularly underscore spanintex's superior performance in sufficiency, agreement with human inductive biases, its enhancement of simulatability, and complexity, compared to tokenex and tokenintex. future efforts should focus on developing combined methods that enhance all explanation properties.