Why Is It Hate Speech? Masked Rationale Prediction for Explainable Hate Speech Detection (2022)

Abstract: In a hate speech detection model, we should consider two critical aspects in addition to detection performance–bias and explainability. Hate speech cannot be identified based solely on the presence of specific words; the model should be able to reason like humans and be explainable. To improve the performance concerning the two aspects, we propose Masked Rationale Prediction (MRP) as an intermediate task. MRP is a task to predict the masked human rationales–snippets of a sentence that are grounds for human judgment–by referring to surrounding tokens combined with their unmasked rationales. As the model learns its reasoning ability based on rationales by MRP, it performs hate speech detection robustly in terms of bias and explainability. The proposed method generally achieves state-of-the-art performance in various metrics, demonstrating its effectiveness for hate speech detection. Warning: This paper contains samples that may be upsetting.

## introduction with the recent development of social media and online communities, hate speech, one of the critical social problems, can spread easily. the spread of hate strengthens discrimination and prejudice against the target social groups and can violate their human rights. moreover, online hatred extends offline and causes real-world crimes. therefore, properly regulating online hate speech is important to address many social problems related to aversion. in addition to the detection performance, two essential considerations are involved in implementing a hate speech detection model-bias and explainability. hate speech should not be judged by any specific word but by the context in which the word is used. even if any word generally considered vicious does not exist in a text, the text can be hate speech. a specific expression does not always imply hatred either (e.g., e.g., 'nigger') @xcite figure 1 : examples for the two methods to get the final ground truths. example input sentences are represented with the class and human rationale labels. in this figure, hatexplain uses the same ground truth about both normal and hateful sentences for the loss. however, our method could determine the two classes with the ground truths. @xcite . however, the presence of this word can cause a model to make a biased detection of hate speech. this erroneous judgment may inadvertently strengthen the discrimination against the target group of the expression @xcite . in this respect, the model's bias toward specific expressions should be excluded. the expressions that can cause biased judgment should be interpreted in context. it means it is vital for the hate speech detection models to have the ability to make judgments based on context, as humans do. therefore, the model should be explainable to humans so that the rationale behind a result is explained @xcite . here, the rationale is a piece of a sentence as justification for the model's prediction about the sentence, as defined by related research @xcite . to the best of our knowledge, hatexplain @xcite is the first hate speech detection benchmark dataset that considers both these aspects. they proposed a method that utilizes rationales as attention ground truths to complement the performance of the two elements. however, when most tokens are annotated as the human rationale in a hateful sentence, the rationale's information figure 2 : framework of the proposed method. we finetune a pre-trained bert through two training stages-masked rationale prediction (mrp) and then hate speech detection. in mrp, the partially masked rationale label is inputted as the rationale embeddings by being added into the input embeddings of bert. the model predicts each masked rationale per token. the model for hate speech detection is initialized by the updated parameters during mrp. in this paper, we present a method to implement a hate speech detection model much more effectively by using the human rationale of hate for finetuning a pre-trained language model. to achieve this, we propose masked rationale prediction (mrp) as an intermediate task before finetuning the model on hate speech detection. mrp trains a model to predict the human rationale label of each token by referring to the context of the input sentence. the model takes the human rationale information of some input tokens among the sentence along with the corresponding tokens as input. it then predicts the rationale of the remaining tokens on which the rationale is masked. we embed the rationales to provide the human rationales as input per token. the masking process of the partial rationales is implemented while creating rationale embeddings; some rationale embedding vectors are replaced with zero vectors. mrp allows the model to make judgments per token about its masked rationale by considering surrounding tokens with an unmasked rationale. with this, the model learns a human-like reasoning process to get context-dependent abusiveness of tokens. the model parameters trained on mrp become the initial parameter values for hate speech detection in the following training stage. in this way, based on the way of human reasoning for hate, the model can get improved abilities in terms of bias and explainability in detecting hate speech. we experimented with bert @xcite as the pre-trained model. consequently, our models finetuned in the proposed way-bert-mrp and bert-rp-achieve state-of-the-art performance overall on all three types of 11 metrics of hatexplain benchmark-performance-based, biasbased, and explainability-based @xcite . and the two models, especially bert-mrp, also show the best results in qualitative evaluation of explicit and implicit hate speech detection. the main contributions of this paper are: ## related works hate speech detection with the advance of deep learning, hate speech detection studies have utilized neural networks @xcite , and word embedding methods @xcite . more recently, transformer-based @xcite ) models have shown remarkable results. in hate speech detection, bert has been adopted for various studies as hate speech detection can be considered a classification task. @xcite compared a bert-based model with recurrent neural networks (rnns)-based models and showed the bert-based model outperforms other models. furthermore, some studies have considered the model's bias and explainability. @xcite improved accuracy and reduced unintended bias by adopting multi-task learning that predicts toxicity of text and target group labels as additional information. @xcite utilized rationales of the dataset as additional information for finetuning bert to deal with the bias and explainability. to improve performance in terms of the two considerations, we propose a more effective finetuning approach based on bert and the rationales by adopting the pre-training framework. pre-finetuning on an intermediate task recently, finetuning a pre-trained model on a downstream task has become the norm @xcite . however, it cannot be guaranteed that the model finetuned with a small dataset compared to its size will be sufficiently welladjusted for the target downstream task @xcite . pre-finetuning is a technique to train the model on a task before the target task @xcite . this can help the model learn the data patterns or reduce the tuning time so that it converges quickly to better fit the target task. according to @xcite , the more closely the intermediate task is related to the target task, the better the effect of pre-finetuning. and inference tasks involving the reasoning process show a remarkable improvement in the target task performance. we adopt this method to train a pre-trained language model through two stages for hate speech detection. as the intermediate task, we propose mrp, which guides the model to infer the human rationale of each token based on surrounding tokens. explainable nlp and rationale explaining the rationale of the result of an ai model is necessary for it to be explainable to humans @xcite . some natural language processing (nlp) studies define rationale as snippets of an input text on which the model's prediction is supported @xcite . @xcite implemented a generator that generates words considered rationales and used them as input of an encoder for sentiment classification. @xcite mapped the human rationales into the model attention values to solve the low-resource problem by learning a domain-invariant token representation. for hate speech detection, hatexplain employs the human rationales as ground truth attention to concentrate on aggressive tokens. unlike existing approaches, we utilize the masked human rationale label embeddings as input. they become the useful additional information of each token. ## method hate speech detection can be described as a text classification problem. following the problem setting of hatexplain, we define the problem as a three-class classification involving three categories-'hate speech,' 'offensive,' or 'normal'. we finetune a pre-trained bert on hate speech detection. note that other transformer encoder-based models can be used instead. before finetuning the model on this task, we pre-finetune it on an intermediate task. we propose masked rationale prediction (mrp) as the intermediate task. our method is described in figure 2 . ## conclusion this paper presents a method to implement a hate speech detection model considering bias and explainability. we adopt a framework to finetune a pre-trained language model in two stages. as the intermediate task, we propose masked rationale prediction (mrp), which predicts masked rationales for some tokens with the additional rationale information of the remaining surrounding tokens. with this, the model learns to identify abusiveness for each token and the human reasoning process based on context. the trained model by mrp is finetuned again on hate speech detection. as a result, across quantitative and qualitative evaluations, the proposed model shows state-ofthe-art performance in bias and explainability, as well as the detection result. and the examples demonstrate its robustness in detecting hate speech, whether explicit or implicit, based on its superior explainability. meanwhile, we experimented with only bert as the pre-trained model to compare our method with base models. but any other transformer encoder-based model can be easily applied, which can be taken as future work. model example sentence and rationale label human if you ' re an army that is republican you are not welcome here we hate you hs bert-mrp if you ' re an army that is republican you are not welcome here we hate you hs bert-rp if you ' re an army that is republican you are not welcome here we hate you no bert-hx if you ' re an army that is republican you are not welcome here we hate you no human the german regime is more scared of women like this than than are of any moslem terrorist hs bert-mrp the german regime is more scared of women like this than than are of any mo ##sle ##m terrorist hs bert-rp the german regime is more scared of women like this than than are of any mo ##sle ##m terrorist hs bert-hx the german regime is more scared of women like this than than are of any mo ##sle ##m terrorist no human i poisoned sergei skripal he was a faggot anyway hs bert-mrp i poisoned sergei sk ##rip ##al he was a fa ##gg ##ot anyway hs bert-rp i poisoned sergei sk ##rip ##al he was a fa ##gg ##ot anyway hs bert-hx i poisoned sergei sk ##rip ##al he was a fa ##gg ##ot anyway of @xmath0: the highlighted words of the human rationale and the rationale of the models with detection results in 'hate speech' sentences.