nlp BD patriots at BLP -2023 Task 1: Two-Step Classification for Violence Inciting Text Detection in B angla - Leveraging Back-Translation and Multilinguality (2023)

Abstract: In this paper, we discuss the nlpBDpatriots entry to the shared task on Violence Inciting Text Detection (VITD) organized as part of the first workshop on Bangla Language Processing (BLP) co-located with EMNLP. The aim of this task is to identify and classify the violent threats, that provoke further unlawful violent acts. Our best-performing approach for the task is two-step classification using back translation and multilinguality which ranked 6^{th} out of 27 teams with a macro F1 score of 0.74.

## introduction in an era dominated by social media platforms such as facebook, instagram, and tiktok, billions of individuals have found themselves connected like never before, enabling them to swiftly share their thoughts and viewpoints. the growth of social networks provides people all over the world with unprecedented levels of connectedness and enriched communication. however, social media posts often abound with comments containing varying degrees of violence, whether expressed overtly or covertly @xcite . to combat this worrisome trend, social media platforms established community guidelines and standards that users are expected to adhere to. 1,2 .violations of these rules may result in the removal of offensive content or even the suspension of user accounts. given the vast amount of user-generated content on these platforms, manually scrutinizing and filtering potential violence is a very challenging task. this moderation approach is limited by moderators' capacity to keep pace, comprehend evolving slang and language nuances, and navigate the complexity of multilingual content @xcite . to address this issue, several social media platforms turn to ai and nlp models capable of detecting inappropriate content across a range of categories such as aggression and violence, hate speech, and general offensive language @xcite . the shared task on violence inciting text detection (vitd) @xcite aims to categorize and discern various forms of communal violence, aiming to shed light on mitigating this complex phenomenon for the bangla speakers. for this task, we carry out various experiments presented in this paper. we employ various models and data augmentation techniques for violent text identification in bangla. ## related work violence identification in bangla several works have been done on building datasets similar to this task and training models on those data. such datasets include the works of @xcite , which mostly gather data by social media mining. however, most of the datasets are comparatively small in size. one of the larger datasets is prepared by @xcite , which consists of 30,000 user comments from youtube and facebook, annotated using crowdsourcing. while most works focus primarily on the datasets, they also present some experimental analysis. @xcite evaluates transformer-based models like m-bert, xlm-roberta, indicbert, and muril. xlm-roberta excels with ample training and muril performs well in joint training, while m-bert and indicbert show proficiency in zero-shot scenarios. however, the most notable work here is done by @xcite who introduces banglahatebert, a re-trained bert model for abusive language detection in bangla. it is trained on a large-scale bangla offensive, abusive, and hateful corpus. the authors collect and annotate a balanced bangla hate speech dataset and use it to pretrain banglabert. the proposed model, banglahatebert, outperforms other bert models and cnn-based models in detecting hate speech on benchmark datasets. related shared tasks @xcite organized offenseval, a series of shared tasks identifying and categorizing offensive language in tweets organized at semeval 2019 and 2020. at offenseval, participants trained a variety of models ranging from machine learning to deep learning approaches. while bert and other transformed dominated the leaderboard in 2020, systems' performance in 2019 was more varied with traditional ml classifiers and ensemblebased approaches achieving competition performance along with deep learning approaches. another shared task, mex-a3t track at iberlef 2019 @xcite , focused on author profiling and aggressiveness detection in mexican spanish tweets. additionally, @xcite presents an overview of the hasoc track at fire 2021 for hate speech and offensive content detection in english, hindi, and marathi, where the highest accuracy is achieved on the marathi dataset. ## dataset the vitd shared task @xcite provides the participants with a bangla dataset including 2700 instances for training and 1330 instances for development. the blind test set contains 2016 instances. the dataset @xcite ) has been annotated using three labels: non-violence, direct-violence, and passive-violence. this threeclass annotated dataset differs from similar datasets where a binary annotation is used @xcite . the data distribution per label is shown in ## conclusion in this paper we described the nlpbdpatriots approach to the vitd shared task. we evaluated various models on the data provided by the shared task organizers, namely statistical machine learning models, transformer-based models, few shot prompting, and some customization with transformer-based models with multilinguality, back translation, and two-step classification. we show that the two-step classification procedure with multilinguality and back translation is the most successful approach achieving a macro f1 score of 0.74. our two-step approach towards solving the problem presented for this shared task shows promising results. however, the relatively small size of the dataset made it difficult for the other pre-trained models to learn informative features that would help them perform classification. also, the dataset contains three imbalanced labels making it easy for the models to overfit. our approach with data augmentation and two-step classification generates good results, but it is still below one of the three baseline results announced by the organizers prior to the start of the competition.