UIT - ISE - NLP at S em E val-2021 Task 5: Toxic Spans Detection with B i LSTM - CRF and T oxic BERT Comment Classification (2021)

Abstract: We present our works on SemEval-2021 Task 5 about Toxic Spans Detection. This task aims to build a model for identifying toxic words in whole posts. We use the BiLSTM-CRF model combining with ToxicBERT Classification to train the detection model for identifying toxic words in posts. Our model achieves 62.23% by F1-score on the Toxic Spans Detection task.

## introduction detecting toxic posts on social network sites is a crucial task for social media moderators in order to keep a clean and friendly space for online discussion. to identify whether a comment or post is toxic or not, social network administrators often read the whole comment or post. however, with a large number of lengthy posts, the administrators need assistance to locate toxic words in each post to decide whether a post is toxic or non-toxic instead of reading the whole post. the semeval-2021 task 5 @xcite provides a valuable dataset called toxic spans detection dataset in order to train the model for detecting toxic words in lengthy posts. based on the dataset from the shared task, we implement the machine learning model for detecting toxic words posts. our model includes: the bilstm-crf model @xcite for detecting the toxic spans in the post, and the toxi-cbert (hanu and unitary team, 2020) for classifying whether the post is toxic or not. before training the model, we pre-process texts in posts and encode them by the glove word embedding @xcite . our model achieves 62.23% on the test set provided by the task organizers. ## related works many corpora are constructed for toxic speech detection problems. they consist of flat label and hierarchical label datasets. the flat label datasets only classify one label for each comment in the dataset (e.g., hate, offensive, clean), while hierarchical datasets can classify multiple aspects of the comment (e.g., hate about racism, hate about sexual oriented, hate about religion, and hate about disability). for flat label, we present several datasets including the two datasets which are provided by @xcite and @xcite in english, the dataset of @xcite in arabic, and the dataset of by @xcite in indonesian. for the hierarchical label, we introduce the dataset constructed by @xcite in english, the dataset provided by @xcite in portuguese, and the co-nan dataset by @xcite , which is the multilingual corpus (constructed in italian, english, and french). in addition, many of shared tasks about hate speech and abusive languages are organized, such as the semeval-2019 task 5 (multilingual) @xcite , the semeval-2019 task 6 (english) @xcite , the poleval 2019 shared task 6 (polish) @xcite , the ger-meval 2018 (germany) @xcite , evalita 2019 (italian) @xcite , toxic comment classification challenge foot_0 , and vlsp 2019 shared task (vietnamese) @xcite . besides, sota approaches like deep learning @xcite and transformers models @xcite are applied in the hate speech detection and toxic posts classification. however, these models only classify based on the whole posts or documents. for the toxic spans detection task, we adapt the mechanism from sequence tagging @xcite and name entities recognition @xcite for detecting toxic words from posts. ## dataset according to besides, figure 2 illustrates the number of toxic words in spans per post for the test set. spans containing single words account for the highest percentage (70.35%) in the test set, and are higher than in the training set, while the multiple-word spans are few. also, the empty spans in the test set are higher than the training set, and the longest post in the test set contains only seven words. ## conclusion we use the bilstm-crf and toxicbert models for detecting toxic words in the posts. our model achieves 62.23% by f1-score from the competition. from the error analysis, we found that our model predicts well just for single-word spans and empty spans. in further researches, we improve the performance of the detection model by applying the attention mechanism and using the character-level representation combining with word-level representation. character-level models like charbert @xcite is a potential approach to increase the performance of toxic spans detection tasks. no. posts gold span prediction span: bilstm-crf prediction span: bilstm-crf+ toxicbert 1 indeed, people the world over, all know that president trump is a loser! [] [66, 67, 68, 69, 70] [] 2 idiot "miner" in the photo isn't even wearing a dust mask. msha needs to inspect this operation for more worker safety violations. [0, 1, 2, 3, 4] [-1, 0, 1, 2, 3] [-1, 0, 1, 2, 3] 3 people who are antiimmigration are weak, lying, racist whiners. [49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62] [] [] 4 uh-no, keep voting for failed liberal idiocy that guarantees results ala detroit, chicago, etc. you'll wish your body had only some crap rather than gangbanger gunfire. @xcite 5 what is he going to do about those toxic mercury florescent bulbs bush and gore pushed on the stupid american public? @xcite