Detecting Hate Speech in T urkish Print Media: A Corpus and A Hybrid Approach with Target-oriented Linguistic Knowledge (2024)

Abstract: The use of hate speech targeting ethnicity, nationalities, religious identities, and specific groups has been on the rise in the news media. However, most existing automatic hate speech detection models focus on identifying hate speech, often neglecting the target group-specific language that is common in news articles. To address this problem, we first compile a hate speech dataset, TurkishHatePrintCorpus, derived from Turkish news articles and annotate it specifically for the language related to the targeted group. We then introduce the HateTargetBERT model, which integrates the target-centric linguistic features extracted in this study into the BERT model, and demonstrate its effectiveness in detecting hate speech while allowing the model’s classification decision to be explained. We have made the dataset and source code publicly available at url{https://github.com/boun-tabi/HateTargetBERT-TR}.

## introduction hate speech, typically characterized by defamatory statements targeted at specific groups based on ethnicity, nationality, religion, color, gender, sexual orientation, among other characteristics @xcite , presents unique challenges in media discourse. contrary to the expectation of objectivity in news and print media, hate speech is surprisingly prevalent @xcite . this study explores this phenomenon, broadening the scope to include discriminatory speech, which, while not explicitly hateful, still fosters discrimination. despite regulatory efforts, such speech persists in media, often masked by subtle linguistic tactics. for example, distortion involves making unfair generalizations, as seen in headlines like "greeks deliberately target refugees on sinking boat." similarly, symbolization uses identity traits to convey messages, evident in phrases like "will a muslim represent us at eurovision?" these methods not only spread hate speech but also magnify its damaging effects, highlighting the need for vigilant monitoring and action. with the significant advances in pre-trained large language models and the transformer architecture in natural language processing, researchers have developed various architectures based on bert @xcite that have achieved successful results in the area of hate speech detection @xcite @xcite @xcite . although lexical and linguistic features have been used in different model architectures @xcite @xcite , the integration of target-oriented linguistic features into the bert model has not yet been studied. there are open data sets on certain aspects of hate speech in different languages and especially in social media @xcite @xcite . however, resources for languages like turkish are scarce @xcite . recent studies have addressed this issue by compiling turkish tweets from "hate domains" on specific topics like politics, religion, and vaccination where hate speech might emerge @xcite @xcite . concurrently, bert-based models are being developed for hate speech detection @xcite . previous work has also focused on hate speech in turkish news articles and proposed a hybrid model for hate speech detection by integrating linguistic features into @xcite . however, the linguistic features used in this study rely on general morpho-syntactic properties of turkish, neglecting the crucial aspect of the target groups of hate speech. in this study, we compile a dataset of hate speech derived from turkish print news and annotate it specifically for language related to the targeted group. we then introduce the hatetarget-bert model, which integrates the target-centric linguistic features extracted in this study into the bert model, and demonstrate its effectiveness in detecting hate speech while allowing the model's classification decision to be explained. the main contributions of this paper can be summarized as follows: (i) we develop hate-targetbert, a model that couples bert with hate speech target-oriented linguistic features extracted from hate speech content in the news articles and enables the generation of an explanation for the model's classification decision. (ii) we release turkishhateprintcorpus, a humanannotated hate speech dataset derived from turkish print media and make the dataset, our model, and its source code publicly available foot_0 . ## methodology we develop hatetargetbert, a model that couples bert with target-oriented linguistic features specifically designed for hate speech detection. as illustrated in figure 1 , the model architecture consists of a bert model followed by fully connected network (fcn) layers. these layers not only take the last hidden representation of the [cls] token, which is typically used as a sentence embedding, but also incorporate the extracted linguistic features as input. to prevent overfitting, we incorporate dropout layers @xcite between the fcn layers. ## results as shown in we also conducted a user study using the qualtrics online survey tool foot_7 to demonstrate the effectiveness of the hatetargetbert model. for this purpose, we randomly selected ten articles from the test set that were predicted to contain hateful content and asked participants to rank the linguistic features shown in the articles based on their helpfulness in understanding the model's prediction of hatefulness. each article was rated on a 5-point likert scale (strongly @xmath0, somewhat @xmath1, neither agree nor @xmath2, somewhat @xmath3, strongly @xmath4. from the user study and its english translation where "rum sevici" (greek-loving) is highlighted as a post-target feature of degree 1. article excerpt ... bazı önemli milliyetçi şahsiyetler kişisel menfaatlerine hizmet edilmedigi degerlendirmeleriyle gidip bir kez daha akıncı veya benzeri teslimiyetçi ve rum sevici bir başka adaya, sırf inat olsun diye oy vererek göreve getirecekler ... ## related work automated detection of hate speech has been extensively studied over the years due to its positive impact on society. many studies have proposed methods to identify hateful content across different platforms in order to assist in content moderation. previous work utilized traditional machine learning models and neural networks that leveraged features such as tf-idf, word vectors @xcite , n-grams @xcite , and lexical features @xcite @xcite . however, more recent models have developed various architectures based on bert, resulting in significant performance improvements @xcite @xcite @xcite . although hate speech is a topic that has attracted a lot of attention, there is a lack of resources for languages like turkish @xcite . recently, several datasets have been compiled from turkish tweets @xcite @xcite . concurrently, bertbased models are being developed to detect hateful content in these tweets @xcite . previous work has focused on hate speech in turkish news articles and proposed a hybrid model for hate speech detection by integrating linguistic features into bert @xcite . however, the linguistic features used in this study rely on general morpho-syntactic properties of turkish, neglecting the crucial aspect of the target groups of hate speech. in our work, we address this challenge by building a model that combines target-centric linguistic features with bert. this approach achieves high performance while also providing explainability, which is particularly im-portant when dealing with longer contexts such as news articles. ## conclusion we introduced turkishhateprintcorpus, a manually annotated hate speech dataset, compiled from turkish newspaper articles and categorized for target groups. in addition, we developed a model, hatetargetbert, combining bert with targetoriented linguistic features. the results demonstrate that integrating target-oriented linguistic knowledge into a transformer model is an effective strategy for hate speech detection and for the explanation of the model's classification decision.