B angla NLP at BLP -2023 Task 2: Benchmarking different Transformer Models for Sentiment Analysis of B angla Social Media Posts (2023)

Abstract: Bangla is the 7th most widely spoken language globally, with a staggering 234 million native speakers primarily hailing from India and Bangladesh. This morphologically rich language boasts a rich literary tradition, encompassing diverse dialects and language-specific challenges. Despite its linguistic richness and history, Bangla remains categorized as a low-resource language within the natural language processing (NLP) and speech community. This paper presents our submission to Task 2 (Sentiment Analysis of Bangla Social Media Posts) of the BLP Workshop. We experimented with various Transformer-based architectures to solve this task. Our quantitative results show that transfer learning really helps in better learning of the models in this low-resource language scenario. This becomes evident when we further finetuned a model that had already been finetuned on Twitter data for sentiment analysis task and that finetuned model performed the best among all other models. We also performed a detailed error analysis where we found some instances where ground truth labels need to be looked at. We obtained a micro-F1 of 67.02% on the test set and our performance in this shared task is ranked at 21 in the leaderboard.

## introduction sentiment analysis is the task of determining the attitude or opinion expressed in a piece of text. typically sentiment categories are of three types: positive, negative, and neutral. in today's increasingly interconnected world, where digital communication abounds, sentiment analysis has emerged as a vital component of natural language processing (nlp) and computational linguistics. it enables us to gauge public sentiment on diverse topics, monitor social media trends, and make datadriven decisions in various domains, including marketing, customer service, and politics. social media provides an interesting platform to study sentiment analysis. people have diverse opinions regarding any topic and they express them accordingly. mining sentiments from them often become very critical due to the trending social media lingo. the use of slang, informal language, and emojis in social media posts can further complicate the task of sentiment analysis. the scarcity of resources and research initiatives dedicated to bangla sentiment analysis can be attributed to several factors. firstly, bangla is considered a lowresource language within the nlp and speech community, primarily due to limited and scattered research efforts undertaken by individual researchers or small teams. secondly, the development of robust deep learning models pre-trained on monolingual bengali data is not that widely available like we have numerous models pre-trained on english data. ## related works early work on sentiment analysis in bangla relied on lexicon-based and rule-based methods like in @xcite . lexiconbased methods use a dictionary of sentiment words to identify the sentiment of a text. rule-based methods use a set of rules to identify sentiment words and phrases. however, with the advancement of deep learning models, these approaches were outperformed by them because they are more capable of understanding the contextual meaning of the sentence and they do not require handcrafted rules or a set of lexicons to identify the sentiment present in text segment. bhowmick and jana (2021) performed sentiment analysis using bert and xlm-roberta on three datasets -prothom alo (islam et al., 2020), youtube-b @xcite and book-b @xcite . @xcite introduced a large-scale bangla dataset for sentiment analysis from book reviews. islam et al. (2023) introduced a multi-domain bangla sentiment analysis dataset across 30 different domains. ## system description this section describes our system which is developed to classify sentiment present in bangla social media posts. this section starts with the shared task description, followed by the description of the dataset released by the shared task organizers, then our proposed architecture which produced our team's standing on the leaderboard, and finally the results achieved and observations made. all the codes and datasets used for performing the experiments are available in https://github.com/ saumajit/banglanlp/tree/main/task_2 . ## conclusion we have provided an overview of how some of the promising approaches using transformer-based models perform with bengali text data. we have also pointed out a few flaws in the annotation quality of the data, which if corrected, may lead to better performance of the models. we find that a transfer learning-based approach with a multilingual model works best in such a low-resource scenario when there are not too many models available that are pre-trained on a huge corpus of monolingual data. an interesting future research direction seems to be the application of recently released large language models (llms) in the nlp space and see how they perform with a low-resource language like bengali.