M1437 at BLP -2023 Task 2: Harnessing B angla Text for Sentiment Analysis: A Transformer-based Approach (2023)

Abstract: Analyzing public sentiment on social media is helpful in understanding the public’s emotions about any given topic. While numerous studies have been conducted in this field, there has been limited research on Bangla social media data. Team M1437 from George Mason University participated in the Sentiment Analysis shared task of the Bangla Language Processing (BLP) Workshop at EMNLP-2023. The team fine-tuned various BERT-based Transformer architectures to solve the task. This article shows that BanglaBERT_{large} , a language model pre-trained on Bangla text, outperformed other BERT-based models. This model achieved an F1 score of 73.15% and top position in the development phase, was further tuned with external training data, and achieved an F1 score of 70.36% in the evaluation phase, securing the fourteenth place on the leaderboard. The F1 score on the test set, when BanglaBERT_{large} was trained without external training data, was 71.54%.

## introduction social networking platforms have emerged as avenues where people share their thoughts and feelings on diverse subjects such as entertainment, politics, and education @xcite . natural language processing (nlp) can effectively evaluate the sentiment of a text @xcite and explore the information discussed in social networking platforms. however, most research in this field has focused on english as the primary language; many other languages (e.g., bangla) have remained largely unexplored @xcite . despite being the seventh most commonly spoken language worldwide, as well as the sixth in terms of native speakers @xcite , bangla is regarded as a low-resource language @xcite . the inaugural bangla language processing (blp) @xcite sought to address sentiment analysis of bangla social media posts. within the scope of this workshop's sentiment analysis shared task, two datasets were utilized: the multiplatform bangla sentiment (mubase) @xcite dataset, which features tweets and facebook posts paired with their corresponding sentiment polarity, and the sentiment on noisy bangla texts (sentnob) @xcite dataset, which consists of user comments on news articles and social media videos in various domains, such as education, politics, etc. this paper presents our solution to sentiment analysis in bangla on the workshop datasets. our experiments with various bidirectional encoder representations from transformers (bert)based models @xcite indicated that banglabert @xcite , a bert language model that is pretrained on more than 27 gb of bangla data is effective for classifying bangla text sentiment. this system achieved an f1 score of 73.15% during the development phase. to further improve performance, we supplemented the training set with the cognisenti dataset @xcite containing facebook posts and tweets authored by bangla speakers. this updated system achieved the best f1 score of 70.36% on the test set, securing the fourteenth place on the evaluation leaderboard. without training data from cognisenti dataset, the f1 score was 71.54%. our code is publicly available on github foot_0 . ## related work extensive research has been carried out regarding sentiment analysis in languages with abundant resources, such as english. traditional sentiment analysis approaches on resource-abundant languages relied heavily on syntactic parsing @xcite . the advent of transformerbased architectures @xcite , such as bert @xcite , greatly improved the state-of-the-art @xcite on sentiment classification @xcite . low-resource languages have traditionally lagged behind these advancements. in recent years, however, the nlp community has turned its attention to low-resource languages like bangla. sentiment analysis for low-resource languages became one of the tasks to receive attention. the availability of high-quality datasets, such as aspectbased sentiment analysis (absa) of bangla text @xcite dataset, has supported sentiment analysis in bangla. example approaches to sentiment analysis on bangla primarily utilized long short-term memory (lstm) units (hochreiter and schmidhuber, 1997; tripto and ali, 2018; rezaul karim et al., 2020). the sentnob dataset (islam et al., 2021) was introduced in 2021, which consists of noisy bangla texts. islam et al. showed that combining lexical features resulted in better performance than neural models for sentnob. hasan et al. developed the cognisenti dataset (hasan et al., 2020), which leverages transformer models like xlm-roberta (conneau et al., 2020) to predict sentiment polarity in bangla text, with promising results. in a recent comparative study of various bangla sentiment classification datasets using different transformer-based architectures, xlm-roberta outperformed all models @xcite . these results demonstrate the growing potential of transformer-based architectures to improve language processing even in low-resource languages such as bangla. banglabert @xcite is a language model based on bert, pretrained on a large dataset of 27. ## results and discussions during the development phase, our system attains the top position in the leaderboard, which is evaluated using the dev-test split. during the evaluation phase, our model ranks as the fourteenth-best model evaluated using the test split, as illustrated in ## conclusion team m1437 had the privilege of participating in the bangla sentiment analysis challenge during the inaugural blp workshop at emnlp-2023. for this task, we prefer the banglabert large as our language model due to its exceptional pre-trained proficiency in the bangla language. during development, our system ranked first on the leaderboard. although we achieved a comparable f1 score during the evaluation phase, we remain committed to exploring a range of large language models (llms) to improve the true positive rates for longer input sequences.