In Natural Language Processing, feature-additive explanation methods quantify the independent contribution of each input token towards a model's decision. By computing the rank correlation between attention weights and the scores produced by a small sample of these methods, previous analyses have sought to either invalidate or support the role of attention-based explanations as a faithful and plausible measure of salience. To investigate what measures of rank correlation can reliably conclude, we comprehensively compare feature-additive methods, including attention-based explanations, across several neural architectures and tasks. In most cases, we find that none of our chosen methods agree. Therefore, we argue that rank correlation is largely uninformative and does not measure the quality of feature-additive methods. Additionally, the range of conclusions a practitioner may draw from a single explainability algorithm are limited .
 By computing the rank correlation between attention weights and feature-additive explanation methods, previous analyses either invalidate or support the role of attention-based explanations as a faithful and plausible measure of salience. To investigate whether this approach is appropriate, we compare LIME, Integrated Gradients, DeepLIFT, Grad-SHAP, Deep-SHAP, and attention-based explanations, applied to two neural architectures trained on single- and pair-sequence language tasks. In most cases, we find that none of our chosen methods agree. Based on our empirical observations and theoretical objections, we conclude that rank correlation does not measure the quality of feature-additive methods. Practitioners should instead use the numerous and rigorous diagnostic methods proposed by the community .
D
In Natural Language Processing, feature-additive explanation methods quantify the independent contribution of each input token towards a model's decision.
R
the scores produced by a small sample of these
feature-additive explanation
D
have sought to
R
what measures of rank correlation can reliably conclude, we comprehensively compare feature-additive methods, including
whether this approach is appropriate, we compare LIME, Integrated Gradients, DeepLIFT, Grad-SHAP, Deep-SHAP, and
R
across several neural architectures and
applied to two neural architectures trained on single- and pair-sequence language
R
Therefore, we argue
Based on our empirical observations and theoretical objections, we conclude
D
is largely uninformative and
R
Additionally, the range of conclusions a practitioner may draw from a single explainability algorithm are limited
Practitioners should instead use the numerous and rigorous diagnostic methods proposed by the community