Leveraging World Knowledge in Implicit Hate Speech Detection (2022)

Abstract: While much attention has been paid to identifying explicit hate speech, implicit hateful expressions that are disguised in coded or indirect language are pervasive and remain a major challenge for existing hate speech detection systems. This paper presents the first attempt to apply Entity Linking (EL) techniques to both explicit and implicit hate speech detection, where we show that such real world knowledge about entity mentions in a text does help models better detect hate speech, and the benefit of adding it into the model is more pronounced when explicit entity triggers (e.g., rally, KKK) are present. We also discuss cases where real world knowledge does not add value to hate speech detection, which provides more insights into understanding and modeling the subtleties of hate speech.

## introduction hate speech on social media facilitates the spread of violence in the real world. for this reason, the detection of hatred content online increasingly gains importance. however, most work in hate speech detection has focused on explicit or overt hate speech, failing to capture the implicit hateful messages in coded or indirect language (e.g., sarcasm or metaphor) that disparage a protected group or individual, or to convey prejudicial and harmful views about them @xcite . examples (1) and (2) from @xcite show the two types of hate speech, explicit vs. implicit: as shown in (1) and ( 2 ), explicit hate speech is direct and uses specific keywords while implicit hate speech does not contain explicit hateful lexicon or phrases and often uses coded or indirect languages to disguise the malicious intent @xcite . modeling implicit sentiment in hate speech is still in its infancy, and the capacity to acquire background knowledge enhances the correct detection of hate speech by machines @xcite . to be able to understand the implied statement of hate speech, machine learning systems need extratextual information that provides world knowledge associated with natural language concepts. for example, it would be impossible for a reader who does not know what happened in charlottesville to understand the implicit hateful message in (2). the reader wouldn't be able to understand the implied message "larger white supremacist events will happen" without knowing that charlottesville is a metonym for a white supremacist rally that took place in charlottesville, virginia in august, 2017. conversely, background knowledge that reasons about entity mentions in a text could add value to the detection of implicit hate speech. incorporating such knowledge ideally should make it easier for the learning model to detect hate speech where it is not apparent from the text. with this motivation, this study applies entity linking (el) to identify entities in tweets, link them to an external knowledge base (kb; wikipedia in this study), and acquire their wikipedia descriptions that would be encoded with sentence bert @xcite for representation. our proposed model incorporates such knowledge representation into identifying both explicit and implicit hate speech in investigating the effectiveness of real world knowledge. overall, this study makes the following contributions: (i) to the best of our knowledge, this work is the first attempt to leverage el techniques in tackling the problem of implicit hate speech detection. (ii) to evaluate the effectiveness of real world knowledge in both explicit and implicit hate speech detection, where we investigate how incorporating wikipedia descriptions of linked entities into the model affects performance. ## related work identifying hate speech has been a topic of immense interest in recent years, and a number of studies have approached this problem in different ways. early work on hate speech detection has focused on explicitly abusive text using keyword-based methods that rely on lexical features @xcite , while more recent studies have highlighted the linguistic nuance and diversity of the implicit hate expressions, which includes stereotypes @xcite , indirect sarcasm, humor, and metaphor @xcite that cannot be captured by keyword-based systems. implicit hate expressions are no less harmful than explicit ones and make up a large portion of false negatives errors @xcite . systems that rely on explicit hateful lexicon or phrases are unable to capture underlying hateful intent like humans. up until now, predicting implicit hate or abuse remains a major challenge for machine systems. existing solutions for identifying implicit cases of hate speech involve taking context into account. for example, @xcite included original news articles as the context of the hateful comments. other studies have built datasets with "implicit" labels or annotations @xcite @xcite . this is crucial not only for evaluation but also for training, as systems that are not trained on implicit hate would not go beyond explicit features and are thus far from being applicable in the real world as a moderation tool. recently, an emerging line of research has started to explore the idea of incorporating real world knowledge in a related task, sarcasm detection, but not for hate speech detection task. this line of research @xcite hypothesizes that infusing real world knowledge such as commonsense knowledge in sarcasm detection ideally should make the learning model easier to detect sarcasm where it is not apparent from the text. @xcite proposed a novel architecture to integrate knowledge into learning model. for knowledge representation, they applied the pre-trained comet model (commonsense transformers, @xcite ) to generate relevant commonsense knowledge from sarcastic instances and use it as input to the proposed model in investigating how commonsense knowledge influences performance. similarly, chowdhury and chaturvedi (2021) leveraged comet to infuse commonsense knowledge in their graph convolution-based model, in which a graph is formed with edges between the input sentence and comet sequences. the node representations of the graph are then passed through a fully-connected neural network to generate the output. the results of the effectiveness of commonsense knowledge are still inconclusive. @xcite found that integrating commonsense knowledge information contributes to sarcasm detection, yet it only plays a supporting role as models using only knowledge information do not perform satisfactorily. interestingly, chowdhury and chaturvedi (2021) found an opposite result on the role of commonsense knowledge in sarcasm detection, showing that comet infused model performs at par with the baseline. in many cases, the model is more reliant on the input sentence and less on the comet sequences for making the prediction. their in-depth error analysis shows that commonsense is most effective in identifying sarcasm with polarity contrast but fails to explain non-sarcastic samples or other types of sarcasm effectively. the study suggests that exploring the utility of other forms of external knowledge such as factual world knowledge for sarcasm detection would be a promising line of inquiry. inspired by previous research, the present study assumes that real world knowledge would be beneficial to implicit hate speech detection in that such knowledge allows a fully understanding of the background knowledge required for deciphering the hateful intent of the text. ## analysis this section investigates whether the model is reliant on wikipedia descriptions while making decisions. we leverage lime (locally interpretable model-agnostic explanations) algorithms @xcite to explain our model's predictions on both explicit and implicit hate statements through random examples picked from our dataset. ## conclusion this paper has proposed the idea of integrating real world knowledge into the task of hate speech detection. experimental results show that real world knowledge is helpful, especially in cases where entity triggers (e.g., rally, kkk) are present in the tweet. however, our analysis also shows that this knowledge fails to predict implicit hate speech types as wikipedia knowledge does not add value and sometimes even hurts the classification, suggesting that a more sophisticated model that enables understanding beyond knowledge about the involved entities is required for implicit hate speech type classification. to mitigate these challenges, works on model architecture that enable information flow between the representations of the tweet and wikipedia knowledge is a reasonable next step. additionally, exploring the possibility of combining different kinds of external knowledge, for example, combining commonsense knowledge (chowdhury and chaturvedi, 2021) in modeling implicit hate speech would also be a promising line of inquiry. to further understand the subtleties of hate speech, deciphering models for coded language or indirect language (e.g., metaphor, irony) in hate speech expression would be beneficial.