Data Augmentation with Dual Training for Offensive Span Detection (2022)

Abstract: Recognizing offensive text is an important requirement for every content management system, especially for social networks. While the majority of the prior work formulate this problem as text classification, i.e., if a text excerpt is offensive or not, in this work we propose a novel model for offensive span detection (OSD), whose goal is to identify the spans responsible for the offensive tone of the text. One of the challenges to train a model for this novel setting is the lack of enough training data. To address this limitation, in this work we propose a novel method in which the large-scale pre-trained language model GPT-2 is employed to generate synthetic training data for OSD. In particular, we propose to train the GPT-2 model in a dual-training setting using the REINFORCE algorithm to generate in-domain, natural and diverse training samples. Extensive experiments on the benchmark dataset for OSD reveal the effectiveness of the proposed method.

## introduction it's no secret that social networks are growing in popularity. however, growth in popularity also brings some challenges, including the toxicity associated with the content posted by users. it may take different forms in social media, including insults, mockery, threats, discrimination, or swearing. the presence of offensive text in social networks can have a detrimental effect on their users, making it desirable to identify and remove them from the text. since this is an important requirement, the task of offensive language detection has been extensively studied in nlp community @xcite @xcite @xcite @xcite most existing works, however, only classify a text snippet as offensive or not, failing to provide further information on which specific words and phrases in the text contribute the most to its offensive tone. if the text snippet is lengthy, the moderators will need this information to decide how to proceed with the offenses flagged. as such, in this work, we fill this gap by proposing a novel model for the task of offensive span detection (osd). as an example, in the given text "this live streamer clearly has no brain; he is such a tool!", the phrase "has no brain" and the slang word "tool" are two offensive spans responsible for the toxicity of the text. one of the barriers to this task is the lack of labeled data. inspired by the recent advances in the application of pre-trained language models to augment training data for low-resources tasks @xcite @xcite @xcite , we propose to employ the gpt-2 model to overcome the data scarcity of osd. to address this limitation, we propose a novel model in which the osd training data are augmented with the synthetic samples generated by a transformer-based language model. in particular, the original labeled samples of osd, with special markers before and after each offensive span, are employed to fine-tune the parameters of the gpt-2 model to generate sentences containing offensive spans. moreover, in order to increase the quality of the generated samples, we propose to explicitly encourage the gpt-2 model to generate diverse sentences while keeping them similar to the original training samples. also, the model is encouraged to generate sentences that will result in improvement of the performance of the osd task. to fulfill these objectives, in a dual training setting, the reinforce algorithm @xcite is exploited to train the gpt-2 model. we evaluate the proposed model on a recently released dataset for offensive span detection. our extensive experiments show the effectiveness of the proposed model by outperforming the strong baselines. ## experiments in order to evaluate the effectiveness of the proposed model, called gaosd (generation-based augmentation for offensive span detection), in our experiments, we use the dataset of semeval 2021 task 5 @xcite . this dataset contains annotations for 10,000 posts (comments) obtained from the archive of civil comment platform (a platform for community to share comments about various civility issues). we use the official splits with 7939/690/2000 documents in train/development/test sets. for each document, the word indices of offensive spans are provided. in our experiments, we create the bio labels using the provided word indices of the offensive spans. in our experiments, we use the bert base to encode data; 2 layers for feed-forward neural networks with 250 hidden dimensions. the trade-off parameters β and γ are set to 0.1 and 0.05, respectively. the learning rate is set to 0.3 for the adam optimizer and the batch size of 64 is employed during training. to evaluate the performance, we use the official evaluation metrics for the semeval 2021 task 5 @xcite . we compare the performance of gaosd with the following baselines: (1) bilstm+crf: the glove embedded document is encoded by bil-stm and the labels are predicted by a crf layer; analysis: to study the contribution of the proposed techniques, we conduct an ablation study on the development set of the semeval 2021 task 5 dataset. specifically, we ablate the quality improvement component which ensures the usefulness and diversity of the generated samples. in particular, we study the performance of the model when the usefulness reward (ur -), the diversity reward (dr -), or both of them (udr -) are ablated. also, we study the performance of the model when no dual training is employed (dt -). specifically, we first pre-train the base model on the available original data. next, we fix the parameters of the base model and we use it to compute the usefulness reward. the results are shown in finally, in order to provide more insight into the quality of the generated data, we provide some randomly selected text generated by the fine-tuned gpt-2 model. the results are shown in ## related work prior works related to this task can be categorized into two groups: (i) toxicity detection: these works aim to classify a piece of text as toxic or nontoxic @xcite @xcite @xcite . the main limitation of these works is that they cannot recognize the spans in the text that are responsible for the toxicity of the text. (ii) opinion word extraction: in this group of prior works, models perform a sequence labeling task to identify the spans in the text that convey the sentiment @xcite @xcite @xcite @xcite . the major limitation of all these models is that they require the existence of the target opinion (i.e., the word or phrase that the text has a sentiment polarity toward it). ## conclusion in this work, we propose a novel method for augmenting data for offensive span detection tasks. specifically, we employ the pre-trained language model gpt-2 to be fine-tuned on the available training samples for osd. the fine-tuned model is able to generate in-domain texts with special tokens in-dicating the offensive spans in them. moreover, to improve the quality of the generated documents, we propose a novel dual training setting in which the feedback from the osd model is employed to guide the gpt-2 model to generate more impact synthetic data. together with a reward for encouraging the diversity of the generated data, the proposed method is effective to augment the training data for osd, resulting in the state-of-the-art performance on the recent benchmark datasets.