S em E val-2021 Task 5: Toxic Spans Detection (2021)

Abstract: The Toxic Spans Detection task of SemEval-2021 required participants to predict the spans of toxic posts that were responsible for the toxic label of the posts. The task could be addressed as supervised sequence labeling, using training data with gold toxic spans provided by the organisers. It could also be treated as rationale extraction, using classifiers trained on potentially larger external datasets of posts manually annotated as toxic or not, without toxic span annotations. For the supervised sequence labeling approach and evaluation purposes, posts previously labeled as toxic were crowd-annotated for toxic spans. Participants submitted their predicted spans for a held-out test set and were scored using character-based F1. This overview summarises the work of the 36 teams that provided system descriptions.

## introduction discussions online often host toxic posts, meaning posts that are rude, disrespectful, or unreasonable; and which can make users want to leave the conversation @xcite . current toxicity detection systems classify whole posts as toxic or not @xcite @xcite , often to assist human moderators, who may be required to review only posts classified as toxic, when reviewing all posts is infeasible. in such cases, human moderators could be assisted even more by automatically highlighting spans of the posts that made the system classify the posts as toxic. this would allow the moderators to more quickly identify objectionable parts of the posts, especially in long posts, and more easily approve or reject the decisions of the toxicity detection systems. as a first step along this direction, task 5 of semeval 2021 provided the participants with posts previously rated to be toxic, and required them to identify toxic spans, i.e., spans that were responsible for the toxicity of the posts, when identifying such spans was possible. note that a post may include no toxic span and still be marked as toxic. on the other hand, a non toxic post may comprise spans that are considered toxic in other toxic posts. we provided a dataset of english posts with gold annotations of toxic spans, and evaluated participating systems on a held-out test subset using character-based f1. the task could be addressed as supervised sequence labeling, training on the provided posts with gold toxic spans. it could also be treated as rationale extraction @xcite , using classifiers trained on larger external datasets of posts manually annotated as toxic or not, without toxic span annotations. there were almost 500 individual participants, and 36 out of the 92 teams that were formed submitted reports and results that we survey here. most teams adopted the supervised sequence labeling approach. hence, there is still scope for further work on the rationale extraction approach. we also discuss other possible improvements in the definition and data of the task. ## competition dataset creation during 2015, when many publications were closing down comment sections due to moderation burdens, a start up named civil comments launched @xcite . using a system of peer-based review and flagging, they hoped to crowd source the moderation responsibility. when this effort shut down in 2017 @xcite , they cited the financial constraints of the competitive publishing industry and the challenges of attaining the necessary scale. the founders of civil comments, in collaboration with researchers from google jigsaw, undertook an effort to open source the collection of more than two million comments that had been collected. after filtering the comments to remove personally identifiable information, a revised version of the annotation system of @xcite was used on the appen crowd rating platform to label the comments using a number of attributes including 'toxicity', 'obscene', 'threat @xcite . the complete dataset, partitioned into training, development, and test sets, was featured in a kaggle competition, 1 with additional material, including individual rater decisions, published @xcite after the close of the competition. civil comments contains about 30k comments marked as toxic by a majority of at least three crowd raters. toxic comments are rare, especially in fora that are not anonymous and where people have expectations that moderators will be watching and taking action. we undertook an effort to reannotate this subset of comments at the span level, using the following instructions: for this task you will be viewing comments that a majority of annotators have already judged as toxic. we would like to know what parts of the comments are responsible for this. extract the toxic word sequences (spans) of the comment below, by highlighting each such span and then clicking the right button. if the comment is not toxic or if the whole comment should have been annotated, check the appropriate box and do not highlight any span. and a custom javascript based template, 2 which allowed selection and tagging of comment spans 1 www.kaggle.com/c/jigsaw-unintendedbias-in-toxicity-classification 2 github.com/ipavlopoulos/toxic_spans ## task description the objective of this task is the detection of the spans that make a post toxic, when detecting such spans is possible. systems had to extract a list of toxic spans, or an empty list, per post. a toxic span was defined to be a sequence of words that attribute to the post's toxicity. although we defined the task at the word level, gold labels were provided at the character level counting from zero (see offsets post {} i will stand by my statement that liberalism is what smart looks like to stupid people. the letter posted yesterday states the real problem clearly enough. "it doesn't matter if it was protected by the first amendment." i have legal news flash for the university of oregon. because the university is public and receives government tax dollars it is indeed bound to abide by the us constitution with all of it's protection included. which should tell everyone right off the signatories are anti-american, and perhaps even anti-america in their twisted pc nonsense. {} fits the despicable piece of trash like a glove. {} no he shot someone so he shouldn't be free. he should be locked up like he is for the next decade. you holla like its no big deal but what if it was your little brother who was shot? then your dumb ass wouldnrlt be screaming free ace g or whatever gay nickname he has. that is the lamest gangster name ever. do you think that name scares people? ace clown is what i will now call him. {55, 56, 57, 58} time to give the opposition weapons and overthrow this jerk. {31, . . . , 39} you are a paid liberal hack of stupidity shore living the liberal lie every day {9, . . . , 13} youre an idiot. {7, 8, . . . , 11, 14, 15, . . . , 18} ## participation overview we received 479 individual participation requests, 92 team formations, and 1,449 submissions. 91 teams submitted valid predictions (1,385 valid submissions in total) and were scored; out of these, only 36 submitted system descriptions. ## evaluation this section focuses on the evaluation framework of the task. first, the official measure that was used to evaluate the participating systems is described. then, we discuss baseline models that were selected as benchmarks for comparison reasons. finally, the results are presented. ## analysis and discussion overall the organisers were happy to see the degree of involvement in this shared task, and the resulting diversity of approaches to this problem. we include some of our observations regarding the administration of the evaluation and what we have learned from the results. ## conclusions we provided 10,629 posts that were annotated for toxic spans and we defined the task of toxic span detection. the task was popular, attracting almost 500 individual participants. eventually 91 teams were formed, out of which 36 submitted a description report. this overview described the approaches of these 36 teams and discussed their results. pre-trained transformers, fine-tuned by viewing the task as a sequence labelling one, performed well and solutions that combined these models within an ensemble were highly-rated. the performance of these models increases further with the help of pre-trained word embeddings or by using multiple transformer layers to embed words. long toxic spans were more likely contextdependent and less frequent in the dataset compared to single-word spans, which made their detection a challenge. the winners included in their ensemble an approach that performed better on long spans, but we note that the problem of detecting long uncommon toxic spans is far from solved. of particular interest were approaches that employed rationale extraction mechanisms, which do not require any training data annotated at the span level. they performed much worse than sequence labeling approaches, but this is a promising direction that was considered by only a few participants. future similar competitions could benefit from tracks that separate supervised from unsupervised solutions. the development of datasets created with the help of crowd annotators should focus on addressing ambiguity, bias, inconsistencies, and misannotations. this could be accomplished by adding more annotators per post. future competitions could also require participants to both classify posts as toxic or not, and detect toxic spans only when posts are classified as toxic, instead of providing the participants only with posts already classified as toxic. finally, future competitions could require participants to distinguish toxic posts of different kinds (e.g., insult, threat, profanity, along with supporting spans), which are sometimes easier to define compared to the more general umbrella toxicity term we (and others) have used.