L ow R esource NLU at BLP -2023 Task 1 & 2: Enhancing Sentiment Classification and Violence Incitement Detection in B angla Through Aggregated Language Models (2023)

Abstract: Violence incitement detection and sentiment analysis hold significant importance in the field of natural language processing. However, in the case of the Bangla language, there are unique challenges due to its low-resource nature. In this paper, we address these challenges by presenting an innovative approach that leverages aggregated BERT models for two tasks at the BLP workshop in EMNLP 2023, specifically tailored for Bangla. Task 1 focuses on violence-inciting text detection, while task 2 centers on sentiment analysis. Our approach combines fine-tuning with textual entailment (utilizing BanglaBERT), Masked Language Model (MLM) training (making use of BanglaBERT), and the use of standalone Multilingual BERT. This comprehensive framework significantly enhances the accuracy of sentiment classification and violence incitement detection in Bangla text. Our method achieved the 11th rank in task 1 with an F1-score of 73.47 and the 4th rank in task 2 with an F1-score of 71.73. This paper provides a detailed system description along with an analysis of the impact of each component of our framework.

## introduction natural language processing (nlp) has witnessed remarkable advancements in recent years, transforming the way we interact with and understand textual data @xcite . from chatbots and machine translation to information retrieval and sentiment analysis, nlp has become an indispensable tool for extracting meaning from the vast sea of human-generated text @xcite . among the diverse array of nlp tasks, sentiment analysis, and violence incitement detection stand out as pivotal areas with far-reaching implications for societal well-being and communication @xcite . ## dataset for task 1, participants are presented with a bangla dataset comprising youtube comments related to the top 9 violent incidents that have occurred in the bengal region (comprising bangladesh and west bengal) over the past decade, with comments up to 600 words long @xcite . the training set (2700 samples) comprises approximately 15% direct vio-lence, 34% passive violence, and 51% non-violent instances. in the development set (1330 samples), a similar distribution is observed: 15% direct violence, 31% passive violence, and 54% nonviolence. for task 2, the given dataset combines two primary sources: the multiplatform bangla sentiment (mubase) @xcite and @xcite datasets. thus, this dataset includes public comments on news and videos across 13 domains, and multiplatform content such as tweets and facebook posts, all manually annotated for sentiment polarity as shown in ## system description in our methodology, we aggregate three bertbased language models in order to tackle both classification tasks. the proposed methodology of our system is as shown in figure 1 . model a: in this model configuration, we incorporate two heads within the banglabertlarge @xcite framework. our choice of incorporating two heads in the banglabert-large architecture, one for masked language modeling (mlm) and the other for classification, is driven by a thoughtful rationale and strong motivation. firstly, this dual-headed approach enables us to retain the invaluable language understanding capabilities embedded in pre-banglabert-large mlm head bert multilingual banglabert-base classification head loss function (mlm loss) loss function (focal loss) finetuning (shared banglabert-large parameters) classification head 1 classification head 2 loss function (c.c.e.) loss function (c.c.e.) finetuning (shared banglabert-base parameters) classification head loss function (c.c.e.) finetuned bert (multilingual) aggregation xnli data our dataset model a model b model c figure 1: proposed methodology for our system. our methodology combines three bert-based models capitalizing on the strength of each of them. model b: it is multilingual bert (mbert) @xcite , a versatile architecture designed to handle multiple languages @xcite @xcite , including bangla. leveraging mbert's rich multilingual knowledge, our methodology gains valuable linguistic insights, enhancing our understanding of bangla text. model c: this model introduces a multi-head training strategy, simultaneously addressing two distinct yet interrelated tasks. the first head within banglabert-base focuses on the xnli dataset @xcite , specifically targeting the task of textual entailment. this choice is motivated by the rationale of knowledge fusion, aiming to merge insights and linguistic patterns from both textual entailment and classification domains. xnli has languages like hindi, urdu, and swahili whose dialects and cultural nuances are similar to bangla. we hypothesize that this helps the model to acquire better parameters. by sharing parameters across heads, the model seeks to develop a deeper and more comprehensive understanding of bangla language nuances. the second head is dedicated to our data, which is centered around a classification problem. this dual-task approach not only boosts efficiency but also contributes to achieving our primary objective: solving the classification problem. the inclusion of the textual entailment task acts as an auxiliary training signal, facilitating the acquisition of versatile and adaptable language representations. this, in turn, aids in achieving superior performance in our core clas-sification task, making model c a powerful and efficient component of our methodology. for all models a, b, and c, we made trials with focal loss and cross-entropy loss and used the loss function which gave the optimal performance. we also made trials with banglabert-large and banglabert-base and selected the most optimal framework as shown in figure 1 . all models have objective function as classification. aggregation: our aggregation technique employs a multi-step process to effectively combine predictions from multiple models. initially, we extract individual predictions from each model using the argmax function @xcite , selecting the class with the highest confidence score for each model. subsequently, to consolidate these individual predictions, we apply another argmax operation, this time on the maximum logit values obtained from each model. this step ensures that we capture the most confident prediction across all models. if two labels have equal highest probabilities, we select the majority sample class. ## results performance on the task 1 and task 2 were evaluated on the basis of macro and micro f1-score respectively. our team ranks 11th in task 1 with f1-score of 73.47. similarly, our team ranks 4th in task 2 with macro f1-score of 71.72. ## conclusion in conclusion, our methodology presents a detailed and novel approach to addressing the challenges of sentiment analysis and violence detection in bangla text. by aggregating insights from three different language models, we achieve a high performance in both tasks. through a detailed ablation analysis, we have analyzed the impact of each component, demonstrating the efficiency of our proposed approach. while our primary focus lies in sentiment analysis and violence detection, the consistently high performance across both tasks underscores the potential versatility of our method in various other text analysis applications in bangla. in the future, more research can be done on bias mitigation, ensuring responsible and equitable deployment of our framework in a real-world context.