We present a novel feature attribution method for explaining text classifiers , and analyze it in the context of hate speech detection . Although feature attribution models usually provide a single importance score for each token , we instead provide two complementary and theoreticallygrounded scores -necessity and sufficiencyresulting in more informative explanations . We propose a transparent method that calculates these values by generating explicit perturbations of the input text , allowing the importance scores themselves to be explainable . We employ our method to explain the predictions of different hate speech detection models on the same set of curated examples from a test suite , and show that different values of necessity and sufficiency for identity terms correspond to different kinds of false positive errors , exposing sources of classifier bias against marginalized groups . Explainability in AI ( XAI ) is critical in reaching various objectives during a system 's development and deployment , including debugging the system , ensuring its fairness , safety and security , and understanding and appealing its decisions by end-users ( Vaughan and Wallach , 2021 ; Luo et al . , 2021 ) . A popular class of local explanation techniques is feature attribution methods , where the aim is to provide scores for each feature according to how important that feature is for the classifier decision for a given input . From an intuitive perspective , one issue with feature attribution scores is that it is not always clear how to interpret the assigned importance in operational terms . Specifically , saying that a feature is ' important ' might translate to two different predictions . The first interpretation is that if an important feature value is changed , then the prediction will change . The second interpretation is that , as long as the feature remains , the prediction will not change . The former interpretation corresponds to the necessity of the feature value , while the latter corresponds to its sufficiency . To further illustrate the difference between necessity and sufficiency , we take an example from hate speech detection . Consider the utterance " I hate women " . For a perfect model , the token ' women ' should have low sufficiency for a positive prediction , since merely the mention of this identity group should not trigger a hateful prediction . However , this token should have fairly high necessity , since a target identity is required for an abusive utterance to count as hate speech ( e.g. , " I hate oranges " should not be classified as hate speech ) . In this paper , we develop a method to estimate the necessity and sufficiency of each word in the input , as explanations for a binary text classifier 's decisions . Model-agnostic feature attribution methods like ours often perturb the input to be explained , obtain the predictions of the model for the perturbed instances , and aggregate the results to make conclusions about which input features are more influential on the model decision . When applying these methods to textual data , it is common to either drop the chosen tokens , or replace them with the mask token for those models that have been trained by fine-tuning a masked language model such as BERT ( Devlin et al . , 2019 ) . However , deleting tokens raises the possibility that a large portion of the perturbed examples are not fluent , and lie well outside the data manifold . Replacing some tokens with the mask token partially remedies this issue , however it raises others . Firstly , the explanation method ceases to be truly model-agnostic . Secondly , a masked sentence is in-distribution for the pre-trained model but out-of-distribution for the fine-tuned model , because the learned manifolds deviate from those formed during pre-training in the fine-tuning step . To avoid these problems we use a generative model to replace tokens with most probable n-grams . Generating perturbations in this way ensures that the perturbed instances are close to the true data manifold . It also provides an additional layer of transparency to the user , so they can decide whether to trust the explanation by checking how reasonable the perturbed examples seem . Although supervised discriminative models rely fundamentally on correlations within the dataset , different models might rely on different correlations more or less depending on model architecture and biases , training methods , and other idiosyncrasies . To capture the distinction between correlations in the data and the direct causes of the prediction , we turn to the notion of interventions from causal inference ( Pearl , 2009 ) . Previous work employing causal definitions of necessity and sufficiency for XAI have assumed tabular data with binary or numerical features . The situation in NLP is much more complex , since each feature is a word in context , and we have no concept of ' flipping ' or ' increasing ' feature values ( as in binary data and numerical data , respectively ) . Instead , our method generates perturbations of the input text that have high probability of being fluent while minimizing the probability that the generated text will also be a direct cause of the prediction we aim to explain . As our application domain we choose hate speech detection , a prominent NLP task with significant social outcomes ( Fortuna and Nunes , 2018 ; Kiritchenko et al . , 2021 ) . It has been shown that contemporary hate speech classifiers tend to learn spurious correlations , including those between identity terms and the positive ( hate ) class , which can result in further discrimination of already marginalized groups ( Dixon et al . , 2018 ; Park et al . , 2018 ; Garg et al . , 2019 ) . We apply our explainability metrics to test classifiers ' fairness towards identity-based groups ( e.g. , women , Muslims ) . We show how necessity and sufficiency metrics calculated for identity terms over hateful sentences can explain the classifier 's behaviour on non-hateful statements , highlighting classifiers ' tendencies to over-rely on the presence of identity terms or to ignore the characteristics of the object of abuse ( e.g. , protected identity groups vs. non-human entities ) . The contributions of this work are as follows : • We present the first methodology for calculating necessity and sufficiency metrics for text data as a feature attribution method . Arguably , this dual explainability measure is more informative and allows for deeper insights into a model 's inner workings than traditional single metrics . • We use a generative model for producing input perturbations to avoid the out-of-distribution prediction issues that emerge with token deletion and masking techniques . • To evaluate the new methodology , we apply it to the task of explaining hate speech classification , and demonstrate that it can detect and explain biases in hate speech classifiers . We make the implementation code freely available to researchers to facilitate further advancement of explainability techniques for NLP.1 This work is a step towards more informative and transparent feature attribution metrics for explaining text classifiers . We argue that standard token importance metrics can be ambiguous in terms of what ' importance ' means . Instead , we adapt the theoretically-grounded concepts of necessity and sufficiency to explain text classifiers . Besides being more informative , the process of generating these two metrics is intuitive and can be explained to lay people in terms of " how much the perturbations in input change the output of the classifier " . Moreover , the input perturbations can be presented to the users , leading to a transparent and understandable explainability framework . Considering the complexities of perturbing textual features , we introduced a practical implementation to compute the necessity and sufficiency of the input tokens . Taking hate speech detection as an example application , we showed that sufficiency and necessity can be used to explain the expected differences between a classifier that is intended to detect identity-based hate speech and those trained for detecting general abuse . We also leveraged these metrics to explain the observed over-sensitivity and under-sensitivity to mentions of target groups , issues that are tightly related to fairness in hate speech detection . While the current work focused on binary hate speech detection for English-language social media posts , in future work , we will explore the effectiveness of these metrics in generating explanations for other applications and languages . We will also explore how the new metrics can improve the debugging of the models or communicating the model 's decisionmaking process to the end-users .

Challenge: Feature attribution methods are used for local explanation but provided scores are not always clear on how to interpret the assigned importance in operational terms.
Approach: They propose a feature attribution method for text classifiers which computes necessity and sufficiency metrics coupled with a generative model-based explicit text perturbation method.
Outcome: Experiments with different hate speech detection models towards identity-based groups show that the proposed method can explain the classifier's behaviour on non-hateful statements.