How Does the Hate Speech Corpus Concern Sociolinguistic Discussions? A Case Study on K orean Online News Comments (2021)

Abstract: Social consensus has been established on the severity of online hate speech since it not only causes mental harm to the target, but also gives displeasure to the people who read it. For Korean, the definition and scope of hate speech have been discussed widely in researches, but such considerations were hardly extended to the construction of hate speech corpus. Therefore, we create a Korean online hate speech dataset with concrete annotation guideline to see how real world toxic expressions concern sociolinguistic discussions. This inductive observation reveals that hate speech in online news comments is mainly composed of social bias and toxicity. Furthermore, we check how the final corpus corresponds with the definition and scope of hate speech, and confirm that the overall procedure and outcome is in concurrence with the sociolinguistic discussions.

## introduction hate speech is an issue that has permeated deeply into our daily life @xcite . hate speech is often explicitly stated along with insulting expressions, and some of them are perceived as hateful or offensive just by incorporating social bias. accordingly, public figures or underrepresented groups suffer from tremendous mental damage, while some experience depression or end their lives. regarding the hate speech in online spaces, discussions are divided into definition and detection @xcite point of view. definition mainly concerns "what the hate speech is" in a deductive manner, while detection attacks the issue with more an inductive methodology. for korean, the hate speech study has mainly been conducted in the sociolinguistics community regarding the definition, and these include the discussion on the appropriateness of expression "hate speech" itself @xcite , its scope @xcite , and the legal issues around discrimination and insult @xcite . in hong et al. @xcite , hate speech is defined as "an expression that discriminates/hates or instigates discrimination/hostility/violence for some social minority individual/group". to back up these studies, further discussions on the underrepresentedness of each group have also actively taken place @xcite . however, aside from the importance of such discussions, there is a gap between the theoretical definition of hate speech and real hateful expressions that appear in our lives @xcite . from a detection perspective, the following questions are mainly discussed which are not easy to answer in a definition point of view: "should a certain expression be regarded as hateful even if a majority of people do not feel offensive for the same sentence?", "what if for the pre-existing terms that a small group of people insists on its harm?", and "how about the toxic expressions that head the criminals?". if there is a consensus on these issues, collecting data to develop a model for hate speech detection would be more clear. most previous approaches on korean online hate speech detection have been keyword-based that regards glossaries on profanity terms foot_0 @xcite . it is also challenging to find cases of constructing a corpus referring to preceding researches in other cultural regions @xcite @xcite . therefore, to understand how hate speech is represented in the korean online expressions and how the inductive analysis corresponds with the concurrent discussions, we should investigate the attributes of hate speech and construct a corpus in advance. in light of this, we study on a hate speech corpus construction scheme that reflects the characteristics of the korean expressions. recent works on hate speech corpus have considered social bias as one of the hate components @xcite as the hypothesis that bias and hate are closely related @xcite but hardly labeled it. though @xcite performed the most decent work on italian, we wanted to give more fine-grained analysis on the social bias and stereotype. referring to the prior works, we create an annotation guideline for korean entertainment news articles comments where hate speech issues have been prevalent in recent years @xcite , and construct annotated corpus through crowd-sourcing. specifically, we describe bias and toxicity as two main attributes of hate speech and label each of them with three-fold categories. throughout the paper, we present the annotation guideline built upon the observation of the comments and corpus construction scheme based on crowd-sourcing. then, we introduce the corpus characteristics and check whether our procedure and result are adequately accepted within the preceding hate speech studies. the contribution of our study to the field is as follows: ## annotation the annotation guideline described above was primarily constructed through the analysis of the observed 1,000 comments. however, to help the annotators refer to it in tagging the large-scale corpus, utilization of the crowd-sourcing platform was inevitable. the three factors considered in this process are: 1) whether the platform has a sufficient number of potential workers, 2) whether our guideline can be well taken into account in the annotation process, and 3) if the annotation can be performed by the workers that exhibit an expected ethical standard. based on these, we adopted deep-natural ai 17 that accommodates a variety of participants, allows pilot study for the selection of the annotators, and supports the system for checking whether the feedback to the annotators is reflected in the resubmission. labeling was performed for each attribute through the pilot study and crowd-sourcing, and the decision was made with majority voting. for this, the tagging of three participants were guaranteed for each instance. additional adjudication was conducted in cases where all three annotators tagged differently or the answers were significantly divided (e.g., when there was no choice in the middle area for the tagging over toxicity). ## corpus the final dataset comes from a total of 10,000 instances, namely those exploited in making up the guideline, the instances reviewed and accepted through pilot studies, and the rest 8,000, crowdsourced through the annotation of the selected participants. in this process, 659 cases that did not reach the final agreement or was omitted by the participants were dropped. ## discussion our guideline aims at making the blurry boundary between hate speech and freedom of speech more explicit in order to attack the real-world problem. in this section, we extend how this categorization process can connect with "which expressions are sociolinguistically defined as hate speech". we refer to @xcite , @xcite , and @xcite , where each mainly concerns the definition of hate speech, its target and scope, and the legal issues regarding freedom of speech. ## conclusion throughout this study, we investigated which factors result in hate speech in an inductive way. the hate speech found in korean online news comments contains either social bias, toxicity, or both. we built a guideline upon the findings and constructed a dataset to train a model that automatically detects them. furthermore, we refer to the previous discussions on hate speech treated in sociolinguistics and journalism, to see how our approach is related to them and what the distinguished points scrutinized in our approach are. as a follow-up study, we verified how effective this corpus is as input data for real-problem-solving machine learning model @xcite , and will check whether its detection performance is affected by (maybe biased) pre-trained language models. also, we will investigate how the construction scheme of our corpus can be leveraged in other domain such as depressive online text detection @xcite . besides, we expect that by the release of this corpus, korean hate speech research is to be diversified and that real online space might be detoxified.