UPB at S em E val-2021 Task 5: Virtual Adversarial Training for Toxic Spans Detection (2021)

Abstract: The real-world impact of polarization and toxicity in the online sphere marked the end of 2020 and the beginning of this year in a negative way. Semeval-2021, Task 5 - Toxic Spans Detection is based on a novel annotation of a subset of the Jigsaw Unintended Bias dataset and is the first language toxicity detection task dedicated to identifying the toxicity-level spans. For this task, participants had to automatically detect character spans in short comments that render the message as toxic. Our model considers applying Virtual Adversarial Training in a semi-supervised setting during the fine-tuning process of several Transformer-based models (i.e., BERT and RoBERTa), in combination with Conditional Random Fields. Our approach leads to performance improvements and more robust models, enabling us to achieve an F1-score of 65.73% in the official submission and an F1-score of 66.13% after further tuning during post-evaluation.

## introduction nowadays, online engagement in social activities is at its highest levels. the lockdowns during the 2020 covid-19 pandemic increased the overall time spent online. in germany for instance, @xcite observed that 71% of considered subjects increased their online media consumption during this period. unfortunately, online toxicity is present in a large part of the social and news media platforms. as such, automated early detection is necessary since toxic behavior is often contagious and leads to a spillover effect @xcite . recently, a significant effort was put into the detection of toxic and offensive language @xcite @xcite , but the challenging nature of these problems leaves several avenues unexplored. in addition, most shared tasks focus on the distinction between toxic/non-toxic @xcite @xcite or offensive/non-offensive posts in various languages @xcite @xcite @xcite . the semeval-2021 task 5, namely toxic spans detection @xcite , tackles the problem of identifying the exact portion of the document that gives it toxicity. the provided dataset is a subset of the jigsaw unintended bias in toxicity classification dataset foot_0 , with annotated spans that represent toxicity from a document. in this paper, we describe our participation in the aforementioned toxic spans detection task using several transformer-based models @xcite , including bert @xcite and roberta @xcite , with a conditional random field (crf) @xcite layer on top to identify spans that include toxic language. we introduce virtual adversarial training (vat) @xcite in our training pipeline to increase the robustness of our models. furthermore, we enhance part of our models with character embeddings based on the jigsaw unintended bias dataset to improve their performance. finally, we compare the proposed models and analyze the impact of various hyperparameters on their performance. the rest of the paper is structured as follows. the next section introduces a review of methods related to toxic language detection, sequence labeling, and adversarial training @xcite . the third section discusses the employed models, as well as the vat procedure. results are presented in the fourth section, followed by discussions, conclusions, and an outline of possible future works. sequence labeling. predicting the type for each token from a document rather than providing a label for the whole sequence is a task often associated with named entity recognition @xcite , but can be performed in other natural language processing pipelines, including part-ofspeech tagging @xcite and chunking @xcite . a common practice in sequence tagging models @xcite @xcite is to use a crf as a final decoding layer. ## results the evaluation metric for the toxic spans detection task was an adapted version of the f1-score @xcite that takes into account the size of the overlap between prediction spans and golden labels. results for all developed models with the aforementioned hyperparameters (i.e., γ = 0.5, = 2, η = 0.1, and two power iterations) are presented in figure 2 introduces the influence of the perturbation magnitude on the overall performance of three models. the impact of in the adversarial training effectiveness is significant, but it is also highly dependant on the used model and can only be determined experimentally. our models performed well on the detection task, learning not only common toxic expressions like "moron", "stupid", "pathetic troll", "disgusting", "hang-em high", but also obfuscated expressions like "f*cking nasty" and "b*tchy". nonetheless, the models fail to detect more obscured words like "you don't know s***" or "kill this f'n w*ore on site". all models have the tendency to over-predict toxicity by adding words to the toxic expressionfor example, "what a pile of shit" was automatically labeled as "what a pile of shit". the character-level embeddings boosted the performance of the baseline lstm-crf-vat model but did not improve any bert model since it leads to detecting longer spans as toxic (see ## discussions and error analysis in this section, we analyze the bert-toxic-crf-vat performance versus the golden label values from the competition test set. the precision and recall for our best model are 65.74% and 85.54%, respectively, which are indicative of a tendency to over-predict toxic spans. as we mentioned in section 3.1, even though almost all documents in the test set had a high toxicity score in the original jigsaw dataset, many had no annotated toxic spans. indeed, there were 295 records where our model detected a toxic span and none were labeled in the test set. a fool and our money are soon to be parted. a fool and our money are soon to be parted. ## conclusions and future work in this paper, several transformer-based models (i.e., bert and roberta) were tested together with virtual adversarial training to increase their robustness for identifying toxic spans from textual information. our experiments argue that applying vat increases performance and that domainspecific models have higher performance when compared to larger general models. in terms of future work, we plan to experiment with self-supervised adversarial training @xcite to improve the robustness of our models. as we noticed in this dataset too, online users find clever ways to hide offensive and toxic expressions. adversarial training can be effectively employed to detect these attempts and a study of its impact on offensive and hate speech classifiers is worth pursuing as follow-up leads.