V i HOS : Hate Speech Spans Detection for V ietnamese (2023)

Abstract: The rise in hateful and offensive language directed at other users is one of the adverse side effects of the increased use of social networking platforms. This could make it difficult for human moderators to review tagged comments filtered by classification systems. To help address this issue, we present the ViHOS (Vietnamese Hate and Offensive Spans) dataset, the first human-annotated corpus containing 26k spans on 11k comments. We also provide definitions of hateful and offensive spans in Vietnamese comments as well as detailed annotation guidelines. Besides, we conduct experiments with various state-of-the-art models. Specifically, XLM-R_Large achieved the best F1-scores in Single span detection and All spans detection, while PhoBERT_Large obtained the highest in Multiple spans detection. Finally, our error analysis demonstrates the difficulties in detecting specific types of spans in our data for future research. Our dataset is released on GitHub.

## introduction social networking sites have been widely used all over the world. here, users can easily share their thoughts, connect with others, or earn money by selling items, creating content, and so on. since these sites are universally accepted, many extreme users misuse comment functions to abuse other individuals or parties with hate and offensive language. consequently, it has been proved that these types of speech could harm other users' health @xcite . sometimes these behaviors can be considered cyberbullying, cyber threats, or online harassment. however, current studies are mainly about classifying comments as a whole with binary labels 1 https://github.com/phusroyal/vihos figure 1 : an example of the aid of spans detection for traditional foul language tagging system can provide additional insightful knowledge about tagged comments for human moderators. @xcite or multiple labeling schemes of abusive behaviors @xcite @xcite . these efforts are made to aid human moderators, who need to review a massive number of online tagged comments that violate their community standards. however, a system that can highlight the spans that make a comment hateful or offensive can be more advantageous to human moderators who frequently deal with long and tedious comments and prefer explanations over a system-generated unexplained tag per comment. furthermore, in some cases, using highlighted spans and moderators' context knowledge, they can take some actions to stop cyberbullying or online harassment. nevertheless, there is only a study on toxic spans, semeval-2021 task 5: toxic spans detection @xcite . on the other hand, in a study of @xcite , hate and offensive spans worked as a rationale to support models in classifying the whole comments. in vietnamese, the resources about hate and offensive language are limited, namely vihsd @xcite , hsd-vlsp @xcite , and uit-victsd @xcite . indeed, there is no study about hate and offensive spans in vietnamese. this motivated us to (i) develop a new task of extracting hate and offensive spans from vietnamese social media texts that conceivably impact research and downstream applications and (ii) provide the natural language processing (nlp) research community with a new dataset for recognizing hate and offensive spans in vietnamese social media texts. our two main contributions are summarized: ## related work to the best of our knowledge, much of the research in the field of hate speech detection has been conducted in english due to the abundance of corpora and the robust pre-trained models. many benchmark datasets for hate and offensive speech in other languages have also been published in recent years, including arabic @xcite , dutch @xcite , and french @xcite . novel models are introduced to improve the efficiency of hate and offensive speech detection. initial approaches were based on typical machine learning and deep neural networks with word embeddings. transformer-based models such as bert @xcite , bertology @xcite , and bert-based transfer learning @xcite have recently been used to detect hate and offensiveness that achieved competitive results in major semeval shared tasks such as semeval-2020 task 12 @xcite , and semeval-2021 task 5 @xcite . however, research in vietnamese is still limited in terms of the dataset and experimental methods. only a few outstanding research exist, such as vihsd @xcite , hsd-vlsp @xcite , and uit-victsd @xcite . for the topic of detecting foul spans, there are only a few case studies in english that are closely related, namely the semeval-2021 task 5: toxic spans detection dataset @xcite and the hatexplain dataset @xcite . the toxic spans, defined in the semeval-2021 task 5 dataset, are the sequences of words that make a text toxic. there are a total of 10,629 posts in this dataset, which stems from the civil comments dataset @xcite . another dataset with hate and offensive spans at the word level is hat-explain. the hatexplain contains 20,148 gab and twitter posts. each post is manually classified into one of three labels: hateful, offensive, and normal. in this study, we focus on vietnamese to close the gap and develop the first vietnamese hate and offensive spans detecting benchmark. ## conclusion and future work we presented vihos, a new vietnamese dataset for evaluating hate and offensive spans detection models. vihos includes 26,467 humanannotated spans on 11,056 comments. in addition, state-of-the-art models are conducted as the first baseline models, including bilstm-crf and pre-trained language models such as xlm-r base , xlm-r large , phobert base , and phobert large . as a result, the xlm-r large model achieves the best performance, with an f1-score of 0.7770. furthermore, we discover that the performance when detecting multiple spans is better than the performance in detecting single spans in vietnamese hate and offensive spans detection. our dataset is available publicly at the github link 9 . despite the study's many promising contributions, the proposed research work still has several potential concerns, especially since the performance is still modest, and incorrect predictions could harm users' reputations if they rely heavily on our method. we intend to expand the dataset size and diversity of hate and offensive context for vietnamese in the future to address this shortcoming. furthermore, pre-and post-processing techniques will be used to standardize social networking texts @xcite and deal with complex cases (as discussed in subsection 4.5) to improve model performance @xcite @xcite , particularly for vietnamese pre-trained language models.