Reducing Unintended Identity Bias in R ussian Hate Speech Detection (2020)

Abstract: Toxicity has become a grave problem for many online communities, and has been growing across many languages, including Russian. Hate speech creates an environment of intimidation, discrimination, and may even incite some real-world violence. Both researchers and social platforms have been focused on developing models to detect toxicity in online communication for a while now. A common problem of these models is the presence of bias towards some words (e.g. woman, black, jew or женщина, черный, еврей) that are not toxic, but serve as triggers for the classifier due to model caveats. In this paper, we describe our efforts towards classifying hate speech in Russian, and propose simple techniques of reducing unintended bias, such as generating training data with language models using terms and words related to protected identities as context and applying word dropout to such words.

## introduction with the ever-growing popularity of social media, there is an immense amount of user-generated online content (e.g. as of may 2019, approximately 30,000 hours worth of videos are uploaded to youtube every hour 1 ). in particular, there has been an exponential increase in user-generated texts such as comments, blog posts, status updates, messages, forum threads, etc. the low entry threshold and relative anonymity of the internet have resulted not only in the exchange of information and content but also in the rise of trolling, hate speech, and overall toxicity 2 . harassment is a pervasive issue for most online communities. a pew survey conducted in 2014 3 1 https://vk.cc/aanmr4 2 https://vk.cc/aanmzn 3 https://vk.cc/aann6p explicit policies against hate speech can be considered an industry standard foot_0 across social platforms, including platforms popular among russianspeaking users (e.g. vk, the largest social network in russia and the cis foot_1 ). the study of hate speech, in online communication in particular, has been gaining traction in russia for a while now due to it being a prevalent issue long before the internet @xcite . the number of competitions and workshops (e.g. hasoc at fire-2019; trac 2020; hateval and offenseval at semeval-2019) on the topic of hate speech and toxic language detection reflect the scale of the situation. social platforms utilize a wide variety of models to detect or classify hate speech. however, the majority of existing models operate with a bias in their predictions. they tend to classify comments mentioning certain commonly harassed identities (e.g. containing words such as woman, black, jew or женщина, черный, еврей) as toxic, while the comment itself may lack any actual toxicity. identity terms of frequently targeted social groups have higher toxicity scores since they are found more often in abusive and toxic comments than terms related to other social groups. if the data used to train a machine learning model is skewed towards these words, the resulting model is likely to adopt this bias foot_2 . inappropriately high toxicity scores of terms related to specific social groups can potentially negate the benefits of using machine learning models to fight the spread of hate speech. this motivated us to work towards reducing these biases. in 66 this paper, our main goal is to reduce the false toxicity scores of non-toxic comments that include identity terms empirically known to introduce model bias. ## results & conclusion the results are provided in we showed that, for our dataset and for the benchmark from @xcite , adding an extra task of predicting the class of a protected identity can indeed improve the quality of toxicity classification in terms of reducing unintended bias. moreover, we observed that simple techniques such as regularizing the input and extending the training data with external language models can help reduce unintended model bias on protected identities even further. for the @xcite ) benchmark, we did not see much improvement in our metrics. this can be attributed to language differences, as the benchmark contains abusive words both in russian and ukrainian. we also observed that the proposed models achieved competitive results across all three datasets when evaluated with f1 score. the best performing model (attn + identity d/o + lm data + multitask setup) achieved an f1 score of 0.86 on the (smetanin, 2020) benchmark, which is 93% of the reported sota performance of a much larger model fine-tuned from a bert-like architecture.