APEACH : Attacking Pejorative Expressions with Analysis on Crowd-Generated Hate Speech Evaluation Datasets (2022)

Abstract: In hate speech detection, developing training and evaluation datasets across various domains is the critical issue. Whereas, major approaches crawl social media texts and hire crowd-workers to annotate the data. Following this convention often restricts the scope of pejorative expressions to a single domain lacking generalization. Sometimes domain overlap between training corpus and evaluation set overestimate the prediction performance when pretraining language models on low-data language. To alleviate these problems in Korean, we propose APEACH that asks unspecified users to generate hate speech examples followed by minimal post-labeling. We find that APEACH can collect useful datasets that are less sensitive to the lexical overlaps between the pretraining corpus and the evaluation set, thereby properly measuring the model performance.

## introduction detecting toxic or pejorative expressions has been a crucial issue in various online communities. in particular, flaming or trolling in online communities is regarded as hostile behavior that can disrupt the public order and cause mental harm to individuals and groups. attempts to define and detect hate speech from a natural language processing (nlp) perspective have called for timely works, and notable approaches have been suggested so far. in specific, @xcite primarily attacked the judgment on hate speech for twitter text, and @xcite further investigated the offensiveness of social media texts beyond binary detection. recently, @xcite have suggested how demographic matters in hate speech analysis, for the corpora of five different languages, discerning the multilingual tendency. creating a hate speech foot_0 dataset generally involves annotating short documents such as web text, and the context where the hate expressions are from may or may not be given in the process. however, annotating on existing web text has several limitations that deter the dataset's reliability. first, the corpus may incorporate the potential risk of license and personally identifiable information issues that come from the characteristics of online materials. next, if the text is crawled from a restricted scope of domain, the topics of examples may not be diverse, which can result in the evaluation that focuses only a part of social issues (e.g., gender). moreover, in view of model training, using a specific domain of web text that might have been used as a pretraining corpus of public language models, may intervene the fair competition between models and mislead the evaluation. such tendency is more apparent in non-english hate speech detection tasks where the training and evaluation of language models rely upon only a small number of benchmarks. for instance, in korean, the usual performance checking of pretrained language models (plms) adopts beep! @xcite , a currently available hand-labeled hate speech dataset in korean where the domain of raw text is celebrity news comment. people found out that the plms trained upon news comments perform better than others which base on news or wikipedia, but it throws a question to the generalizability and fairness of the evaluation. how can we address such limitation of annotation-based web text corpus construction? though collecting a variety of data as in multigenre natural language inference (multinli) through this process, we can create datasets that are less vulnerable to probable bias in domain and style of the text. also, creating datasets using this scheme can help prevent license and privacy issues. @xcite ) might be the most intuitive solution, it requires economic and human resources and still does not resolve the issue of corpus overlap. in this regard, we hypothesized it a reasonable approach to let anonymous paid workers generate toxic expressions, where the expressions are generated from scratch with a minimum guideline. at a glance, simply opening a web page for text collecting and encouraging user participation seemed to work. however, we noted that neither the data quality nor the open license of the output could be guaranteed in those processes. thus, we established a crowd-driven hate speech generation scheme using a moderator to ensure the privacy of hate speech authors and obtain a qualitychecked corpus at the same time. in specific, we adopt crowd-sourcing platform (as a moderator) and workers for the paid writing, provided with the prompts to guide the generation, to achieve diverse hate speech and prevent participants' disgrace. for the facilitation of text generation and collection process in our research, we devise 'system', an environment that interacts with the crowd (who provides the data) and the task manager (who collects the data), which is composed of i) building a hate speech pseudo-classifier and deploying the model, ii) collecting user-generated data and feedback, and iii) post-labeling of the task managers (figure 1 ). followingly, we obtain apeach, which denotes the collecting scheme and the resulting dataset at the same time. it contains about 3k instances for korean hate speech detection evaluation; the corpus is well balanced in sentence length and topics, also being aligned with the mod-els trained with the existing hate speech dataset (beep!). most of all, by comparing the model performances of our dataset using publicly available plms where the pretraining corpus overlap with beep!, we prove that apeach is less vulnerable to misleading results that might come from the corpus-level similarity with the pretraining corpus. our contribution to this field is as follows: ## system our system consists of three processes: 1) building pseudo-classifier, 2) collecting users' data & feedback, and 3) post-labeling. first two are to be described in this section. ## dataset using the above system, we construct an evaluation dataset with the user-generated data that is inspected with a model prediction. ## experiment we exploit our corpus to evaluate the hate speech detection models trained with a widely used korean hate speech benchmark, beep! @xcite . in specific, we compare apeach (ours) and beep! dev set as an evaluation corpus, to check the generalizability and performance tendency using each set. ## conclusion in this work, we introduce a crowd-driven generation scheme in constructing an evaluation set for hate speech detection, distinct from the existing corpus construction schemes based on crawling and annotation. after a managed human text generation that ensures both the participants' anonymity and the reliability of the corpus, we report a thorough analysis of the created data, accompanied by a comparison with the prior work in korean. the resulting corpus, apeach, displays the potential of adopting crowd-driven generation in hate speech dataset construction, achieving generalizability and topic variety. though there is headroom for the scalability of the corpus, we believe that the proposed scheme can be utilized to make up the evaluation set and training set of domain-agnostic hate speech detection.