Hate and Toxic Speech Detection in the Context of Covid-19 Pandemic using XAI : Ongoing Applied Research (2020)

Abstract: As social distancing, self-quarantines, and travel restrictions have shifted a lot of pandemic conversations to social media so does the spread of hate speech. While recent machine learning solutions for automated hate and offensive speech identification are available on Twitter, there are issues with their interpretability. We propose a novel use of learned feature importance which improves upon the performance of prior state-of-the-art text classification techniques, while producing more easily interpretable decisions. We also discuss both technical and practical challenges that remain for this task.

## introduction in the day and age of social media, a person's thoughts and feelings can enter the public discourse at the click of a mouse or tap of a screen. with billions of individuals active on social media, the task of finding reviewing and classifying hate speech online quickly grows to a scale not achievable without the use of machine learning. additionally, the definition of hate-speech can be broad and include many nuances, but in general hate speech is defined as communication which disparages or incites violence towards an individual or group based on that person or groups' cultural/ethnic background, gender or sexual orientation. @xcite . in the context of covid-19, the united nations has released guidelines on covid-19 related hatespeech guidance on covid-19 related hate speech cautioning that member states and social media companies that with the rise of covid-19 cases there has also been an increase of hate speech. the un warns that such communication could be used for scapegoating, stereotyping, racist and xenophobic purposes. ## explainability and text classification in the same vein of our research, others have leveraged explainability derived with integrated gradients @xcite and subject matter experts to create priors for use in text classification. in this research, they showed a decrease in undesired model bias and an increase in model performance when using scarce data @xcite . overall, our method appears to have similar results and lessens the impacts of specific key terms to the the overall model prediction. one of the more commonly utilized explainability methods, shap provides a framework within the feature contrubutions to a a model's output can be derived by borrowing aultman-shapely values from cooperative game theory (lundberg and lee, 2017). while there are several "explainer" implementations included with shap, gradint explainer allowed us to leverage our entire training dataset as the background dataset which allows our global average term values described in our experimental design to represent all terms in the training corpus. shap's gradient explainer builds off of integrated gradients and leverages what are called expected gradients. this feature attribution method takes the integral from integrated gradients and reformulates it as an expectation usable in calculating the shapely values. the resulting attributions sum to the difference between the expected and current model output. however, this method does assume independence of the input features, so it would violate this assumption if we were to leverage any sequence models in classification. ## results and application to covid 19 tweets we found that the enhanced predictions predominately help to correct false positive classifications and shift predictions towards the negative class. we hypothesize this is due to the diversity of language and relatively neutral feature importance most terms have on the negative class. the relative neutrality in both global and local feature importance scores can be seen in figure 2 , and this results in a higher overall average for the aggregation of the term difference multiplier. when we applied our model to covid-19 tweets we found similar results as described above. quite often, we found that tweets providing information about specific ethnic groups or migrants were labeled as hateful or toxic towards immigrants by our baseline model and then correctly labeled as not hateful or offensive speech when we applied the term difference multiplier. an example of this exact scenario is provided in figure 2 . ## conclusion and future work here we have experimented with a novel method to leverage the global feature importance from a model's training dataset to reinforce or even penalize new predictions when their local feature importance varies from this learned global value. this novel algorithm marries the field of xai and nlp in a manner which allows prior knowledge obtained in model training to impact present predictions. overall, we believe this technique is especially applicable in scenarios like covid-19 where little to no pre-existing labeled data are available. by training this method on a similar corpus it can be used to detract from incorrect predictions made due to a few highly influential terms in covid-19 datasets. at present due to this method's ability to decrease false positives, we believe one application of this research is increasing the efficiency of systems monitoring for hateful and toxic communication. however, this research is ongoing. we intend to explore further scenarios such as altering the equation used in our term difference multiplier and the datasets used since the global feature importance can greatly influence the multiplier combined with our model's original predicted probabilities.