RSM - NLP at BLP -2023 Task 2: B angla Sentiment Analysis using Weighted and Majority Voted Fine-Tuned Transformers (2023)

Abstract: This paper describes our approach to submissions made at Shared Task 2 at BLP Workshop - Sentiment Analysis of Bangla Social Media Posts. Sentiment Analysis is an action research area in the digital age. With the rapid and constant growth of online social media sites and services and the increasing amount of textual data, the application of automatic Sentiment Analysis is on the rise. However, most of the research in this domain is based on the English language. Despite being the world’s sixth most widely spoken language, little work has been done in Bangla. This task aims to promote work on Bangla Sentiment Analysis while identifying the polarity of social media content by determining whether the sentiment expressed in the text is Positive, Negative, or Neutral. Our approach consists of experimenting and finetuning various multilingual and pre-trained BERT-based models on our downstream tasks and using a Majority Voting and Weighted ensemble model that outperforms individual baseline model scores. Our system scored 0.711 for the multiclass classification task and scored 10th place among the participants on the leaderboard for the shared task. Our code is available at https://github.com/ptnv-s/RSM-NLP-BLP-Task2 .

## introduction in the era of a high influx of social media platforms, blogs, and online reviews, sentiment analysis has become the need of the hour. also known as opinion mining, sentiment analysis is a computational linguistic task that is aimed at determining whether a text contains a positive, negative, or neutral sentiment behind it @xcite text label positive neutral negative sentiment analysis has diverse uses, including preventing adolescent suicide by detecting cyberbullying and mitigating unjust actions that target specific communities through hate speech detection, among numerous other applications @xcite ). approximately 284.3 million people worldwide speak bangla as their primary language. individuals speaking bangla increasingly engage in social media platforms like instagram, facebook, reddit, and twitter and express opinions on microblogging platforms, commenting on news portals and online shopping. however, analyzing vast volumes of rapidly generated data in the digital age is a very tedious job to do. this is where sentiment analysis can be applied @xcite . most sentiment analysis research predominantly focuses on english, leaving bangla sentiment analysis in its nascent stages. recently, some works have addressed this issue. however, none of these studies have fully embraced the different perspectives of bangla. to address this problem, we present our contributions to shared task 2 at blp workshop -sentiment analysis of bangla social media posts. this task aims to detect the polarity associated with a given social media text. this multiclass classification task involves determining whether the sentiment expressed in the text is positive, negative, or neutral. for this problem statement, we have conducted various experiments using multi-lingual berts @xcite @xcite and various pre-trained transformers @xcite by finetuning them on downstream tasks. we also apply majority voting and weighted ensembling on the top-k models to show how these methods affect the models' performance and how an ensemble of these models performs better than the individual baselines. ## system overview we conducted extensive experiments for the given task involving bangla sentiment analysis. we finetuned various multilingual and pre-trained transformer architectures, including bert (kenton and toutanova, 2019), distillbert @xcite , roberta @xcite , and various pre-trained bert models @xcite on our downstream task of polarity classification. we shortlist the top-k model based on the performance metrics and ensemble the predictions using majority voted and weighted ensemble. ## experiments & results the dataset used for the task is organized in 3 columns, with id, text, and label. it has also been partitioned into a train set with 35266 samples, a dev set with 3935 samples, and a dev-test set with 3427 samples. the distribution in the training set is shown in figure 1 . the preprocessing pipeline before model training included padding, tokenizing, and truncating text data to ensure uniformity and manage lengthy inputs. we used the adamw optimizer, a learning rate of 2x10 -5 and a batch size of 32 over 32 epochs was chosen to strike a balance between convergence speed and stability with a maximum sequence length of 512 tokens used with huggingface autotokenizer to tokenize the data. we evaluated models using four metrics: accuracy, precision, recall, and f1-score. f1-score is we did an ensemble of both types (majority-voted and weighted) with the top 3 ( banglabert @xcite , banglishbert, hf-pt-bert-1 @xcite ), top 5 (hf-pt-bert-2, ban-glishbert, hf-pt-bert-1 @xcite , banglabert(base), hf-pt-bert-3 @xcite ) and lastly using all the models. as in ## conclusion in this work, we benchmarked various multilingual and pre-trained bert-based models -roberta @xcite , distillbert @xcite , banglabert @xcite , banglishbert @xcite and various pre-trained bert models @xcite for bangla sentiment analysis @xcite @xcite while identifying the polarity of social media content by determining whether the sentiment expressed in the text is positive, negative, or neutral as our downstream tasks and using a ma-jority voting and weighted ensemble model that outperforms individual baseline model scores. our system achieved a micro f1-score of 0.711 for the multiclass classification task and scored 10th among the participants on the leaderboard for the shared task.