G olden W ind at S em E val-2021 Task 5: Orthrus - An Ensemble Approach to Identify Toxicity (2021)

Abstract: Many new developments to detect and mitigate toxicity are currently being evaluated. We are particularly interested in the correlation between toxicity and the emotions expressed in online posts. While toxicity may be disguised by amending the wording of posts, emotions will not. Therefore, we describe here an ensemble method to identify toxicity and classify the emotions expressed on a corpus of annotated posts published by Task 5 of SemEval 2021–our analysis shows that the majority of such posts express anger, sadness and fear. Our method to identify toxicity combines a lexicon-based approach, which on its own achieves an F1 score of 61.07%, with a supervised learning approach, which on its own achieves an F1 score of 60%. When both methods are combined, the ensemble achieves an F1 score of 66.37%.

## introduction healthy conversations are only possible when people feel safe from abuse and do not resort to using violent language. regrettably, violent and inflammatory language is becoming increasingly common online. indeed, the rhetoric of violence recently employed on social media has persuaded platforms, such as twitter, to create new policies to prevent the use of threatening language @xcite . a jargon word, cyberbullying, has been coined lately to refer to the use of electronic communication to send or post messages of an intimidating or threatening nature @xcite . along with cyberbullying, other forms of verbal abuse employed on social media, such as online harassment and hate speech, are now being collectively referred to as toxicity in language @xcite . we are interested in developing algorithms to recognise toxicity and measure its impact on the sentiment expressed. most of the data available to investigate toxicity classify whole comments or documents @xcite , and do not identify "spans"-that is, the precise word sequences that make a text toxic. given how important such spans are for the implementation of semiautomated moderation, we have participated on task 5 (toxic spans detection) of the international workshop on semantic evaluation (semeval) 2021 @xcite . thus far, we have concentrated on the combination of two approaches: a lexicon-based approach and a supervised learning approach to identify toxic spans. although the identification of toxic spans in online posts can be aided by a suitable lexicon of toxic words, such words can easily be concealed through minor changes-for instance, "fck urself" is a toxic span that would evade detection based on basic lists of profane words. however, emotions are harder to conceal. hence, we are interested in using opinion mining to uncover the emotions expressed in text. emotions may be able to identify toxicity, regardless of wordings and spellings. thus, we dedicate part of this study to measure the correlation between toxicity and emotions. the remainder of this paper is organised as follows: section 2 reviews the related work. section 3 describes the datasets that we used for our experimentation. section 4 is dedicated to explain our algorithm for the identification of toxic spans. section 5 presents our results and, finally, section 6 offers our conclusions. ## background the existing literature on toxicity focuses on two main aspects: the compilation and annotation of corpora for research purposes @xcite ; and the automatic detection of different types of toxic text. among the different types of toxic text under scrutiny, we may include hate speech @xcite @xcite , online harassment @xcite , racism @xcite , sexism @xcite , abusive language @xcite and cyberbullying @xcite . at present, the detection of toxicity is largely based on state-of-the-art natural language processing techniques, typically involving machine learning. the main drawback of such techniques resides in the limited generalisation potential of trained machine learning models @xcite . to overcome this weakness, we have integrated into our research the use of a lexicon-based approach, where toxic language is identified with the help of a dictionary of words associated with toxic text @xcite . combining lexicons with machine learning approaches has already been evaluated by other researchers, remarkably pamungkas and patti (pamungkas and patti, 2019), though they employed a lexicon originally built for the italian language, and then translated it into other languages, whereas we focus on english from the start. various other lexicons, handcrafted by domain experts who specialise on the identification of toxicity have been published too-for example, textgain's profanity and offensive words lexicon (de smedt et al., 2020)-but many of them are not available for free. in an attempt to mitigate toxicity and promote work on this area, the research community has released a number of annotated datasets for investigating different forms of toxicity @xcite @xcite . however, they all follow different labelling conventions. consequently, they cannot be analysed using a uniform method. overall, toxicity detection and classification lacks a consistently labelled standard dataset for comparative evaluation @xcite . therefore, the data provided by task 5 (toxic spans detection) of semeval 2021 is very well regarded @xcite . ## experimental setup task 5 of semeval 2021 uses posts from the publicly available civil comments dataset (tensor-flow, 2021). such a dataset comprises annotations indicating which entire posts are toxic, but it does not label particular toxic spans within the posts. the civil comments platform @xcite , which is where the posts come from, is a commenting plugin for independent news websites. all the comments were created between 2015 and 2017, and they appeared on approximately 50 english language websites across the world. when civil comments shut down in 2017, the comments became publicly available in an open archive for future research @xcite . to build the dataset, semeval retained only posts that were found toxic-or severely toxic-by at least half of the annotators involved in borkan, et al.'s annotation @xcite . this comprises 30k toxic posts, approximately, out of the original 1.2m. then, a random subset of 10k posts from these 30k toxic posts were chosen for toxic spans annotation @xcite . ## system overview although machine learning technology is being widely employed to detect toxic text automatically, the use of a lexicon to identify and prevent toxicity in social media still constitutes a valuable approach. indeed, the number of lexicons specialised on the detection of profanity, offensive speech and toxicity in general has grown steadily in recent times @xcite . lexicons are not susceptible to algorithmic bias @xcite , and are not limited to the domain and scope of the training data, which has previously raised a number of ethical concerns, given how much training data is historically associated with particular communities @xcite . hence, we employ a lexicon as our first step in the detection of toxic spans. originally, our lexicon was made, specifically, for task 5 of semeval 2021, as we compiled it by extracting all the toxic words available in the training and trial datasets for task 5-we considered a word as a toxic word if it was included in a toxic span identified by the annotators. upon compiling all the toxic words available in the training and trial datasets (1,287 words), we proceeded to extend our lexicon with words listed in other lexicons. while there are many freelyavailable lexicons of toxic words, we favoured those that maintained the accuracy of the detection of toxic spans achieved by our lexicon. ## results approach f1 score orthrus lexicon 61.07% orthrus lexicon + spacy model (union) 61.53% orthrus lexicon + spacy model (intersection) 66.37% as shown in we are interested in the identification of emotions expressed in text, because concealing emotions may be harder than disguising toxicity. for example, the post "uh, no, he's a belligerent buffoon (and a traitor)", which is post 1,928 of the training dataset of task 5 of semeval 2021, lacks any recognisable toxic features, such as insults or swear words. hence, it is classified as non-toxic by any of the lexicons highlighted in bold font in if we included emotion information in our analysis, we could immediately detect the negative tone of the post mentioned above. indeed, the probability of such a post to communicate anger is 1.0, according to text2emotion, a python package to extract emotions from text (python software foundation, 2021). the expression of anger is so evident in this case that the post can be marked as a candidate to be considered toxic. using text2emotion, we assigned each post in the test dataset a probability associated with each of the emotions reported in figure 1 . the values shown in figure 1 represent the addition of the probabilities of each emotion to occur in each of the posts of the test dataset. clearly, fear, sadness and anger-the three emotions combined together-are more likely to occur than happiness and surprisethe two emotions combined together-which may characterise the toxicity of the dataset. ## conclusions in this paper, we have described the creation of a lexicon of toxic words and a supervised learning approach to identify toxicity in online posts. our lexicon, along with the supervised learning approach, achieved an f1 score of 66.37% on task 5 of semeval 2021. we have also explored the relationship between emotions and toxicity. although our study is still in progress, preliminary results indicate that there exists a correlation between emotions such as sadness and fear and toxicity.