Cisco at S em E val-2021 Task 5: What’s Toxic?: Leveraging Transformers for Multiple Toxic Span Extraction from Online Comments (2021)

Abstract: Social network platforms are generally used to share positive, constructive, and insightful content. However, in recent times, people often get exposed to objectionable content like threat, identity attacks, hate speech, insults, obscene texts, offensive remarks or bullying. Existing work on toxic speech detection focuses on binary classification or on differentiating toxic speech among a small set of categories. This paper describes the system proposed by team Cisco for SemEval-2021 Task 5: Toxic Spans Detection, the first shared task focusing on detecting the spans in the text that attribute to its toxicity, in English language. We approach this problem primarily in two ways: a sequence tagging approach and a dependency parsing approach. In our sequence tagging approach we tag each token in a sentence under a particular tagging scheme. Our best performing architecture in this approach also proved to be our best performing architecture overall with an F1 score of 0.6922, thereby placing us 7th on the final evaluation phase leaderboard. We also explore a dependency parsing approach where we extract spans from the input sentence under the supervision of target span boundaries and rank our spans using a biaffine model. Finally, we also provide a detailed analysis of our results and model performance in our paper.

## introduction it only takes one toxic comment to sour an online discussion. the threat of abuse and harassment online leads many people to stop expressing themselves and give up on seeking different opinions. toxic content is ubiquitous in social media platforms like twitter, facebook, reddit, the increase of which is a major cultural threat and has already lead to a crime against minorities @xcite . toxic text in online social media varies depending on targeted groups (e.g. women, lgbt, gay, african, immigrants) or the context (e.g. protrump discussion or the metoo movement). toxic text online has often been broadly classified by researchers into different categories like hate, offense, hostility, aggression, identity attacks, and cyberbullying. though the use of various terms for equivalent tasks makes them incomparable at times @xcite , toxic speech or spans in this particular task, semeval-2021 task 5 @xcite , has been considered as a super-set of all the above sub-types. while a lot of models have claimed to achieve state-of-the-art results on various datasets, it has been observed that most models fail to generalize @xcite . the models tend to classify comments as toxic that have a reference to certain commonly-attacked entities (e.g. gay, black, muslim, immigrants) without the comment having any intention to be toxic @xcite . a large vocabulary of certain trigger terms leads to a biased prediction by the models @xcite . thus, it has become increasingly important in recent times to determine parts of the text that attribute to the toxic nature of the sentence, for both automated and semi-automated content moderation on social media platforms, primarily for the purpose of helping human moderators deal with lengthy comments and also provide them attributions for better explainability on the toxic nature of the post. this in turn would aid in better handling of unintended bias in toxic text classification. semeval-2021 task 5: toxic spans detection focuses on exactly this problem of detecting toxic spans from sentences already classified as toxic on a post-level. in this paper, we approach the problem of multiple non-contiguous toxic span extraction from texts both as a sequence tagging task and as a standard span extraction task resembling the generic approach and architecture adopted for single-span reading comprehension (rc) task. for our sequence tagging approach, we predict for each token, whether it is a part of the span. for our second approach, we predict and compute a couple of scores for each token, corresponding to whether that token is the start or end of the span. in addition to this, we deploy a biaffine model to score start and end indices, thus adopting the methodology for multiple non-contiguous span extraction. ## literature previous work on automated toxic text detection, and its various sub-types, focuses on developing classifiers that can flag toxic content with a high degree of accuracy on datasets curated from various social media platforms in english @xcite @xcite , other foreign languages @xcite @xcite @xcite including code-switched text @xcite and multilingual text @xcite . this topic has also evidenced a number of workshops @xcite and competitions @xcite @xcite . recent work shows transformer based architectures like bert @xcite have been performing well on the task of offensive language classification @xcite @xcite . transformer based architectures have also produced state-of-the-art performance on sequence tagging tasks like named entity recognition (ner) @xcite @xcite span extraction @xcite and qa tasks @xcite @xcite . multiple span extraction from texts has been explored both as a sequence tagging task @xcite and as span extraction as in rc tasks @xcite . very recently hatexplain @xcite ) proposed a benchmark dataset for explainable hate speech detection using the concept of rationales. attempts have also been made to handle identity bias in toxic text classification @xcite and also to make robust toxic text classifiers which help adversaries not bypass toxic filters @xcite . ## methodology for our sequence tagging approach, we explore two tagging schemes. first, the well known bio tagging scheme, where b indicates the first token of an output span, i indicates the subsequent tokens and o denotes the tokens that are not part of the output span. additionally, we also try a simpler io tagging scheme, where words which are part of a span are tagged as i or o otherwise. formally, given an input sentence @xmath0,...,x n ), of length n,and a tagging scheme with |s| tags (|s| = 3 for bio and |s| = 2 for io), for each of n tokens the probability for the tag of the i-th token is our other approach is based on the standard single-span extraction architecture widely used for rc tasks. with this approach, we extract toxic spans from sentences under the supervision of target span boundaries, but with an added biaffine model for scoring the multiple toxic spans instead of simply taking top k spans based on the start and end probabilities, thus giving our model a global view of the input. the main advantage of this approach is that the extractive search space can be reduced linearly with the sentence length, which is far less than the sequence tagging method. given an input sentence @xmath1,...,x n ), of length n, we predict a target list @xmath2,...,t m ) where the number of targets is m and each target t i is annotated with its start position s i , its end position e i and the class that span belongs to (only one in our case, toxic). however, to adapt to the problem of extracting multiple spans from the sentence, instead of taking the top k spans based on the start and end probabilities, we apply a biaffine model @xcite to score all the spans with the constraint s @xmath3. post this we rank all the spans in descending order and choose every span as long it does not clash with higher-ranked spans. ## dataset the dataset provided to us by the organizers of the workshop consisted of a random subset of 10,000 posts from the publicly available civil comments dataset, from a set of 30,000 posts originally annotated as toxic (or severely toxic) on post-level annotations, manually annotated by 3 crowd-raters per post for toxic spans. the final character offsets were obtained by retaining the offsets with a probability of more than 50%, computed as a fraction of raters who annotated the character offsets as toxic. basic statistics about the dataset can be found in train dev test 1 7897 687 1650 2-4 1617 153 174 >=5 784 63 26 if predicted span i.e s t a is empty for a post t then we set f t 1 (a,g) = 1 if the gold truth i.e s t g is also empty, else if s t g is empty and s t a is not empty then we set f t 1 (a,g) = 0. for our sequence tagging approach we employ the commonly used bilstm-crf architecture @xcite used predominately in many sequence tagging problems, but with added contextual word embeddings for each word using transformer and character-based word embeddings. we experiment with a total of 5 transformer architectures, namely bert @xcite , xlnet @xcite , roberta @xcite , albert @xcite and spanbert @xcite . for all of the above mentioned transformer architectures, the large variant of the transformer was used except albert for which we use its xlarge-v2 variant. first, the tokenized word input is passed through the transformer architecture and the output of the last 4 encoder layers is concatenated to obtain the final contextualized word embedding e t for each word in the sentence. additionally, we also pass each character in a word through a character-level bilstm network, to obtain character-based word embeddings for the word e c as used by @xcite . finally, both these word embeddings, e t and e c , for each word are concatenated and passed through a bilstm layer followed by a crf layer to obtain the best probable tag for each word in the sentence. ## experimental setup data was originally provided to us in the form of sentences and the corresponding character offsets for the toxic spans of the sentence. before converting the character offsets to our required format for our respective approaches, we apply some basic text pre-processing to all our sentences. first, we normalize all the sentences by converting all white-space characters to spaces. second, we split all punctuation characters from both sides of a word and also break abbreviated words. these pre-processing steps help improve the f 1 score of both our approaches as shown in we mention the major hyperparameters of our best-performing systems experimental setting for our dependency parsing approach and span extraction approach in tables 3 and 4 respectively. parameter value bilstm size 256 bilstm layer 1 bilstm dropout 0 transformer size 1024 transformer encoder layers last 4 char bilstm hidden size 25 char bilstm layers 1 optimiser adam learning rate [1e-3,1.56e-4] we train all our sequence tagging models with 2 http://huggingface.co/ stochastic gradient descent in batched mode with a batch size of 8. in the training phase, we keep all layers in our model, including all the transformer layers trainable. we start training our model at a learning rate of 0.01, with a minimum threshold limit of 0.0001, and half the learning rate after every 4 consecutive epochs of no improvement in the f 1 score of the development set. we train our model to a maximum of 100 epochs or 4 consecutive epochs of no improvement at our minimum learning rate. we train our our model for dependency parsing approach with adam optimizer in batched mode with a batch size of 32 and a learning rate of 0.0001 for a maximum of 40,000 steps. with this approach too, we keep all layers trainable in the training phase except the bert transformer layers. pretrained bert and fasttext embeddings were just used to extract context-dependent and independent embeddings respectively and bert was not finetuned in the training phase. the training was performed on 1 nvidia titan x gpu. our code is available on github foot_0 . ## results in our best performing architecture proved to be the sequence tagging system with xlnet trans-former trained with io tagging scheme. additionally, in ## conclusion in this paper, we present our approach to semeval-2021 task 5: toxic spans detection. our best submission gave us an f 1 score of 0.6922, placing us 7 th on the evaluation phase leaderboard. future work includes independently incorporating both post level and sentence level context for determining the toxicity of a word, and also collating a dataset with toxic spans comprising of a healthy mixture of simple cuss words (which can always be attributed as toxic independant of the context) and words for which the toxicity of the word depends on the context in which it appears, thereby making better systems towards contextual toxic span detection.