D e T ex D : A Benchmark Dataset for Delicate Text Detection (2023)

Abstract: Over the past few years, much research has been conducted to identify and regulate toxic language. However, few studies have addressed a broader range of sensitive texts that are not necessarily overtly toxic. In this paper, we introduce and define a new category of sensitive text called “delicate text.” We provide the taxonomy of delicate text and present a detailed annotation scheme. We annotate DeTexD, the first benchmark dataset for delicate text detection. The significance of the difference in the definitions is highlighted by the relative performance deltas between models trained each definitions and corpora and evaluated on the other. We make publicly available the DeTexD Benchmark dataset, annotation guidelines, and baseline model for delicate text detection.

## introduction the prevalence of user-generated toxic language on online social networks has motivated many to develop automatic methods of detecting such content @xcite , @xcite , @xcite , @xcite , @xcite , @xcite , @xcite , @xcite . these efforts towards moderating toxic language have gained even more momentum as large language models, which have the potential to generate harmful content, have become more mainstream @xcite , @xcite , @xcite , @xcite . much of this work has been constrained to texts that are toxic or otherwise overtly harmful; however, there are many other sensitive texts where interaction with other users or virtual agents may be triggering or offensive. while some studies @xcite , @xcite , @xcite have addressed specific sensitive areas (e.g., insults, geopolitics, or illegal activity), to our knowledge, this is the first study that comprehensively analyzes sensitive content in general. in this study, we target a broader set of sensitive texts that we call "delicate texts," an umbrella term covering toxic language as well as lower-severity sensitive texts, with a focus on sensitive texts ( in this study, we introduce the task of delicate text detection. we present a comprehensive definition of delicate texts and a dataset of 1,023 labeled delicate texts @xcite . we share our data collection, annotation, and quality control methods along with the detailed annotation schema. we describe the development of our baseline delicate text detection model. finally, we demonstrate the difference between delicate text detection and existing content moderation methods by testing our model against toxic language benchmark datasets and testing popular content moderation models against our detexd dataset. ## related works several studies have investigated the use of various nlp methods to detect inappropriate content; most of these works targeted toxic and offensive language. some focused on developing more robust models to detect hateful content @xcite , @xcite , @xcite , while others focused on building better and less biased datasets @xcite , @xcite , @xcite , @xcite , @xcite , @xcite , @xcite , @xcite , @xcite , @xcite . with respect to dataset creation, @xcite created ethos, a binary and multi-labeled dataset of hate speech, along with a detailed annotation protocol. their dataset covers various hate speech categories (including race, gender, religion, nationality, sexual orientation and disability), as well as target and whether the texts incited violence. they also examined the quality of their data using both binary and multi-label classification. in another study, @xcite created hatexplain, a hate speech dataset that reflects annotators' rationale for their labeling task. their data went through a three-step annotation process in which a text was first classified as "offensive," "hate," or "normal;" next, the target of the hate was identified as "individual" or "generalized." last, the annotators were asked to highlight parts of the text that justified their annotation decisions. they reported that models that used annotators' rationale in the training data performed slightly better than those without human rationale. most studies have targeted hate speech; however, some studies have addressed a more general concept: inappropriate content. while most of these works used the term 'inappropriate content' to refer to hate speech, they also included sensitive topics. for instance, @xcite focused on identifying inappropriate content; they defined inappropriate content as impolite and disrespectful posts that offend certain groups, are related to illegal activities, or induce violence. they developed a deep learning-based model to identify inappropriate content in detecting query completion suggestions and user conversation texts. in another study, @xcite focused on detecting sensitive content in user interactions with voice services. they targeted profanity, insult, geopolitical topics, explicit sexual and anatomical content, weapons, war, explicit graphical violence, race, religion, and gender. they focused on binary classification of sensitive content. in @xcite semeval 2019 task 5 dataset is described, a specific case of hate speech against immigrants and women in spanish and english twitter messages. they provide both: a main binary subtask for detecting the presence of hate speech, and a finer-grained one for identifying features such as hate attitude or target. during this competition, over 100 models were submitted. we evaluate our baseline model for delicate text detection on their main dataset. ## experiments in this section, we share results from a series of experiments. first, we show the potential to create a delicate text detection system which is suitable for practical usage. for this purpose, we developed and evaluated a baseline delicate text detection model. next, we demonstrate the originality of this task by evaluating the performance of our model on toxic language datasets and evaluating toxic language detection models on detexd. since toxic language detection and delicate text detection are two distinct tasks, detexd does not perform well on toxic language benchmarks and other content moderation methods that target mainly toxic language do not perform well on the detexd benchmark dataset. ## conclusions we introduced a new type of sensitive language called "delicate text," an umbrella term covering not only toxic language but also sensitive language with a priority focus on the latter. we annotated the detexd benchmark dataset for delicate text detection. the significance of the difference in the definitions is highlighted by the relative performance deltas between models trained each definitions and corpora and evaluated on the other. we make our annotation guidelines, annotated dataset, and baseline model publicly available.