S core_ I s A ll_ Y ou_ N eed at BLP -2023 Task 1: A Hierarchical Classification Approach to Detect Violence Inciting Text using Transformers (2023)

Abstract: Violence-inciting text detection has become critical due to its significance in social media monitoring, online security, and the prevention of violent content. Developing an automatic text classification model for identifying violence in languages with limited resources, like Bangla, poses significant challenges due to the scarcity of resources and complex morphological structures. This work presents a transformer-based method that can classify Bangla texts into three violence classes: direct, passive, and non-violence. We leveraged transformer models, including BanglaBERT, XLM-R, and m-BERT, to develop a hierarchical classification model for the downstream task. In the first step, the BanglaBERT is employed to identify the presence of violence in the text. In the next step, the model classifies stem texts that incite violence as either direct or passive. The developed system scored 72.37 and ranked 14th among the participants.

## introduction social media and the internet have become crucial components of daily interactions. they can quickly spread information to millions of people. thus, identifying and categorizing aggressive texts on social media is paramount in maintaining online safety, fostering positive digital interactions, and preventing dissemination of harmful or offensive content @xcite . real-world problems like relational anger or even violence are significant problems since threats and insults made online can occasionally result in actual hurt. to keep us secure and calm, we must address this issue because it can make an impact in the short term and also in the long term on the victims @xcite . different regions' governments try to prevent violations and ensure the safety of the nation's citizens due to social media @xcite . the blp shared task 1, violence inciting text detection (vitd), was launched to address this problem @xcite . this work presents us with the challenge of devising effective methods to identify diverse types of violent content within the text. the primary objective is to detect and avoid violence from internet remarks. the data used for this task was gathered from youtube comments on violent incidents that have taken place in the bengal region (bangladesh and west bengal) over the past ten years. we have tried to solve this problem of violence-inciting text detection with two significant contributions. ## related work detecting violence-inciting text has become increasingly crucial in natural language processing. numerous studies have already focused on identifying hate speech and aggression on social media comments @xcite . mustakim et al. (2022) employed classify emotions in tamil text xlm-r model obtained the highest macro f1-score of 0.33. riza and charibaldi (2021) detected emotions in twitter text using the lstm and achieved an accuracy of 73.15% with both word2vec and fasttext embeddings. this corpus was used in the da-vincis (ta et al., 2022) for detecting aggressive and violent incidents on spanish social media. to train users' tweets on their text embeddings from previously learned transformer models, they employed a multi-task learning network and achieved the best f 1 of 74.80%. plaza-del-arco et al. (2021) applied the transformer-based model to identify hate speech in spanish tweets. sharif et al. (2020) proposed a machine learning-based model that classifies bangla texts into non-suspicious and suspicious categories. this work attained the highest accuracy (84.57%) for the sgd classifier with tf-idf features. sharif and hoque (2020) developed a corpus containing 2000 texts and used several machine learning techniques (such as lr, nb, svm, knn, and dt) to classify the suspicious bangla texts, where lr gained the best performance (accu-@xmath0. hossain et al. (2022) proposed a dataset (mute) containing 4158 memes with bangla captions for identifying hateful memes, and they obtained the maximum f 1 -score of 0.672 with the vgg16+bangla-bert model. sharif et al. ( 2022 ) introduced a bangla aggressive text dataset (m-bad). using a transformer-based technique (bangla-bert), they achieved top scores of 92% in identifying aggressive texts. a recent study @xcite introduced a bangla aggressive text dataset (bad). using a weighted ensemble of m-bert, distil-bert, bangla-bert, and xlm-r, they achieved top scores of 93.43% (coarse-grained) and 93.11% (fine-grained) in identifying and categorizing aggressive bangla texts. as far as we are concerned, the research has yet to be conducted on identifying and classifying violence-inciting texts in bangla. this work exploited a transformer-based model to detect violence-inciting texts and classify them into direct, passive, and non-violence targets. ## task and dataset descriptions the task organizer developed a benchmark corpus for the shared task 1 @xcite . to perform the violence-inciting text classification, this task developed a dataset called violence inciting text detection (vitd) corpus foot_0 consisting of 6046 texts and 20199 unique words. this task focuses on classifying bangla texts inciting violence into three categories: direct (dvio), passive (pvio), and non-violence (nvio). the definition of each class is illustrated in the following: the vitd dataset @xcite was divided into training (2700 texts), validation (1330 texts), and test sets (2016 texts) for training and evaluation purposes. ## methodology this work exploited three pre-trained transformerbased models, xlm-r, banglabert, and m-bert, for classifying violence inciting text in bangla. specifically, we have used the 'xlm-roberta-base' @xcite , 'csebuetnlp/banglabert' @xcite and 'bert-base-multilingual-cased' @xcite from huggingface transformers foot_1 library and fine-tuned on the dataset. figure 1 illustrates the schematic process of the proposed system. ## results the assessment of the models' performance relies on the macro f1-score (mf1) as a primary metric. in addition, we incorporated precision (p) and recall (r) metrics for analysis. the evaluation encompassed banglabert, xlm-r, and mbert models in single-step multiclass classification. among these models, the banglabert achieved the highest macro f 1 (mf1) score, reaching 56.45. banglabert emerged as the top-performing model in the hierarchical framework, surpassing all others with an impressive mf1 score of 72.37. additionally, it is worth highlighting that the results in the hierarchical approach demonstrated a remarkable improvement of almost 28% over the single-step method. ## conclusion this paper developed a transformer-based model to address the task of identifying and classifying violence-inciting texts in bangla. the experimental investigation demonstrated that the banglabert model outperformed the other transformer models (xlm-r and mbert) by obtaining the highest macro f 1 -score (0.72 ). we plan to investigate the task with the advanced transformer-based model (such as gpt). additionally, we aim to explore various ensemble techniques of transformers to enhance the model's performance.