UFAL - ULD at BLP -2023 Task 1: Violence Detection in B angla Text (2023)

Abstract: In this paper, we present UFAL-ULD team’s system, desinged as a part of the BLP Shared Task 1: Violence Inciting Text Detection (VITD). This task aims to classify text, with a particular challenge of identifying incitement to violence into Direct, Indirect or Non-violence levels. We experimented with several pre-trained sequence classification models, including XLM-RoBERTa, BanglaBERT, Bangla BERT Base, and Multilingual BERT. Our best-performing model was based on the XLM-RoBERTa-base architecture, which outperformed the baseline models. Our system was ranked 20th among the 27 teams that participated in the task.

## introduction the rapid proliferation of social media platforms has revolutionized the way we communicate, share information, and engage with diverse communities online. however, with this newfound connectivity and freedom of expression, we have also witnessed a troubling trend -the weaponization of social media for the incitement of violence. the bengal region, comprising bangladesh and west bengal, india, has not remained untouched by this unsettling phenomenon. online platforms, once hailed as vehicles for progress and connection, are now grappling with the disturbing spread of violenceinciting language, leading to communal discord, destruction, and loss of life. in this digital age, where the boundaries between the virtual and the real world blur, it becomes imperative to address the multifaceted manifestations of communal violence, particularly in regions like bengal. the violence inciting text detection (vitd) shared task emerges as a beacon of hope and a clarion call for the natural language processing (nlp) community to confront this pressing issue head-on. the vitd shared task centres on the precise categorization and discernment of violence-inciting text within social media comments, echoing the broader challenge of understanding the dark underbelly of online discourse. the violence we seek to detect and categorize transcends mere words on a screen; it has the potential to manifest as explicit threats, divisive propaganda, and derogatory language that can irreparably harm individuals and communities. this paper discusses our team's system, built as a part of the blp shared task 1: violence inciting text detection (vitd) @xcite . in this work, we experimented with several pretrained sequence classification models with the provided data only where we contributed to data augmentation, sampling stratgies, fine-tuning and hyper-parameter tuning to optimize the performance of these models. 1 our system was ranked 20 th among the 27 teams that participated in the task. ## related work the proliferation of hate-speech, verbal threats, aggression, cyberbullying, trolling, abuse, offensive and misogny content are experiencing rapid growth on social media. a considerable number of researchers have been actively involved in investigating the automated detection of offensive and hate speech content as well as many shared tasks were organising (waseem and hovy, 2016; @xcite @xcite . however, there is considerably less research on violence detection specifically. a few works are as follows: cano basave et al. (2013) present the violence detection model (vdm), a probabilistic framework for identifying violent content and extracting violence-related topics from social media without requiring labeled data. vdm uses word prior knowledge derived from relative entropy to cap-ture word violence indicators, outperforming information gain methods in topic identification and violence classification. chang et al. (2018) address the detection of aggression and loss in social media, particularly among gang-involved youth. their system incorporates contextual representations and domain-specific resources, improving the convolutional neural network's performance for detecting aggression and loss. jahan et al. (2022) introduce banglahatebert, a retrained bert model for abusive language detection in bangla. it outperforms generic pre-trained models on various datasets and includes a 15k bangla hate speech dataset for research use. zandam et al. (2023) explore the expression of threatening themes in the hausa language on social media, developing a classification system using machine learning algorithms. xgboost achieves the highest accuracy of 72% in classifying threatening content. abercrombie et al. (2023) conduct a systematic review of resources for automated identification of online gender-based violence (gbv), highlighting limitations in existing datasets, such as a lack of theoretical grounding and stakeholder input. the study recommends future resources grounded in sociological expertise and involving gbv experts and those with lived gbv experience. ## dataset the vitd shared task 1 dataset @xcite was provided by the task organisers. individual samples in the dataset are labeled as direct violence, indirect violence, and non-violence, which are represented numerically by 2, 1 and 0 respectively (see @xcite for further details). the dataset is divided into training, development and test sets, consisting of 2,700, 1,330 and 2,016 samples respectively. 2 4 experiments this section discusses an extensive account of the system we designed for the vitd and sentiment analysis of bangla social media posts tasks. our strategy encompasses several stages, such as data preprocessing, model choice, hyperparameter adjustment, and advanced methods, all aimed at attaining commendable outcomes. ## results the macro-f1 metric has been used for evaluation measure in the blp shared task 1 @xcite , with comparisons made against the ground truth labels. this metric signifies the comprehensive effectiveness of our system in accurately categorizing text that incites violence into the specified classifications: direct violence, passive violence, and non-violence. the macro f1 score is a resilient measurement that considers precision and recall across all categories, making it particularly suitable for tasks with imbalanced class distributions. our system achieved a macro f1 score of 69.01 on the test set (see ## conclusion in this shared task on violence inciting text detection (vitd), we have presented our system's approach and results, emphasizing the significance of addressing the challenging problem of identifying and categorizing violence-inciting text in the bangla language. our system, equipped with a comprehensive set of natural language processing techniques, achieved a competitive macro f1 score of 69.009 on the test set. our system was ranked 20 th among the 27 teams that participated in the task. we remain committed to further refining our system and exploring innovative approaches to contribute to the ongoing efforts in violence detection and prevention.