Transfer Learning and Prediction Consistency for Detecting Offensive Spans of Text (2022)

Abstract: Toxic span detection is the task of recognizing offensive spans in a text snippet. Although there has been prior work on classifying text snippets as offensive or not, the task of recognizing spans responsible for the toxicity of a text is not explored yet. In this work, we introduce a novel multi-task framework for toxic span detection in which the model seeks to simultaneously predict offensive words and opinion phrases to leverage their inter-dependencies and improve the performance. Moreover, we introduce a novel regularization mechanism to encourage the consistency of the model predictions across similar inputs for toxic span detection. Our extensive experiments demonstrate the effectiveness of the proposed model compared to strong baselines.

## introduction with the proliferation of social networks, the amount of textual data posted online is also everincreasing. this growth comes with some challenges too. one of the issues associated with social networks is the level of toxicity expressed in posts or comments shared online. the toxic/offensive languages in social networks can be realized in different forms such as insults, mockeries, threats, discrimination, or swearing. due to their detrimental effect on users of social networks, it is desirable to identify and remove offensive text from these networks. since this is an important requirement, the task of offensive language detection has been extensively studied in nlp community @xcite @xcite @xcite . however, most of the existing works are limited to classifying a text snippet as offensive or not. in other words, these models fail to provide further information about what specific phrases in the text snippet contribute the most to the offensive tone of the text. this information is necessary for the moderators to decide further actions for the posts/comments flagged as offensive, especially if the text snippet is long. as such, in this work, we fill this gap by proposing a novel model for the task of offensive span detection (osd). as an example, in the given text "this livestreamer clearly has no brain; he is such a tool!", the phrase "has no brain" and the slang word "tool" are two offensive spans responsible for the toxicity of the text. one of the barriers for this task is data scarcity. to address this limitation, we propose a novel model trained in multi-task setting in which the model is trained on two tasks: (1) offensive phrase detection whose goal is to detect word(s) contributing to the toxicity of the text, (2) opinion word extraction which is supposed to assist the main model to pinpoint word(s) conveying subjectivity. note that the second task could help the model restrict its prediction to more likely words. as the available resources for offensive span detection do not provide any annotation for opinion words in the text, in this work, we propose to employ transfer learning to fulfill the training on the second task (i.e., opinion word extraction). in particular, a separate model is pre-trained on sentiment polarity prediction on a sentiment analysis corpus. afterward, the pretrained model is exerted to provide supervision for the task of opinion word extraction. in addition to the proposed multi-task setting, we also introduce a novel regularization loss in which the model is encouraged to make consistent predictions on similar inputs. concretely, in this work, we propose to compute the similarity between samples in a mini-batch with respect to two criteria: (i) word representations (ii) prediction of offensive words. during training, the samples that have the highest similarity are encouraged to have less discrepancy with each other. in order to fulfill this goal, for the first time, we propose to employ optimal transport to compute the consistency loss between samples. we evaluate the proposed model on a recently released dataset for offensive span detection. our extensive experiments show the effectiveness of the proposed model by outperforming the strong baselines. ## experiments in order to evaluate the effectiveness of the proposed model, called tposd (transfer learning and prediction consistency for offensive span detection), in our experiments, we use the dataset of semeval 2021 task 5 (john pavlopoulos and laugier, 2021). we use the official splits with 7939/690/2000 documents in train/development/test sets. also, to pre-train the sa model for the sentiment analysis task to be used for auxiliary training, we employ the amazon-2 dataset @xcite . in our model we use the (fixed) bert base to encode data; 250 dimensions for the hidden states of lstm and 2 layers for feed-forward neural networks with 250 hidden dimensions. the trade-off parameters α, β and γ are set to 0.1, 0.1, and 0.05, respectively. the learning rate is set to 0.3 for the adam optimizer and the batch size of 64 is employed during training. analysis: to study the contribution of the proposed techniques, we conduct an ablation study on the development set of the semeval 2021 task 5 dataset. specifically, we ablate the auxiliary task (owe -), the regularization in the auxiliary task (auxreg -), i.e., l reg , the consistency loss (cons -), i.e., l cons . also, we study the performance of the model when the wasserstein distance is computed regardless of the document representations (cons -sem ) or model predictions (cons -pred ). the results are shown in in the proposed approach, to simultaneously train the model on osd and owe, as the existing training data for osd does not provide gold labels for owe, we resort to transfer-learning, in which a pre-trained sentiment-analysis model is employed to supervise the main model on owe task. however, one natural question is that why transfer-learning is the optimal approach to train the model on owe? to answer this question, in this section, we propose a baseline, in which a model pre-trained on owe is employed to automatically annotate the existing osd training data with opinion words for owe task. in particular, we first train a sequence-tagger consisting of a bilstm encoder followed by a feed-forward layer on the available owe dataset. specifically, in our experiments, we use the combinations of the four benchmark datasets presented by @xcite as the training data to pre-train the owe model. note that the original datasets by @xcite provide opinion words with respect to a given target mention. however, in the pre-training of the owe model, we aim to train the model to detect all opinion words in the input text. as such, in our experiments, we combine opinion words of all samples of the same sentence in the dataset. finally, the pre-trained owe model is employed to annotate the opinion words in the osd dataset, i.e., semeval 2021 task 5 @xcite . the auare the gold labels for offensive span detection (osd), provided in the semeval dataset, and opinion word extraction (owe) tasks, provided by the pre-trained owe model, for i-th word. the overall loss to train the model jointly on both tasks is then defined by: we call this baseline pre-train owe and its performance on the test set of the semeval dataset is reported in ## related work prior works related to this task can be categorized into two groups: (i) toxicity detection: these works aim to classify a piece of text as toxic or nontoxic @xcite @xcite @xcite . the main limitation of these works is that they cannot recognize the spans in the text that are responsible for the toxicity of the text. (ii) opinion word extraction: in this group, models perform a sequence labeling task to identify the spans in the text that convey the sentiment @xcite @xcite @xcite @xcite . the major limitation of all these models is that they require the existence of the target opinion (i.e., the word or phrase that the text has a sentiment polarity toward it). ## conclusion in this work, we proposed a novel model for offensive span detection. to train the model, in a novel framework, we propose to exploit the interaction with the related task of opinion word extraction. specifically, in a multi-task learning setting, we train the model for offensive and opinion word extraction. also, we introduce a novel regularization loss based on optimal transport which encourages the consistency of the model prediction on similar documents. our experiments on the available benchmark dataset show the effectiveness of the proposed model and outperform strong baselines.