UFAL - ULD at BLP -2023 Task 2 Sentiment Classification in B angla Text (2023)

Abstract: In this paper, we present the UFAL-ULD team’s system for the BLP Shared Task 2: Sentiment Analysis of Bangla Social Media Posts. The Task 2 involves classifying text into Positive, Negative, or Neutral sentiments. As a part of this task, we conducted a series of experiments with several pre-trained sequence classification models – XLM-RoBERTa, BanglaBERT, Bangla BERT Base and Multilingual BERT. Among these, our best-performing model was based on the XLM-RoBERTa-base architecture, which outperforms baseline models. Our system was ranked 19th among the 30 teams that participated in the task.

## introduction sentiment analysis, the task of determining the sentiment expressed in textual data, is a critical component of natural language processing (nlp). it plays a vital role in understanding public opinion, social media trends, and user sentiments in various languages. in the context of the bangla language, sentiment analysis poses unique challenges due to the language's specific characteristics such as complex morphology, making it an intriguing area of research. sentiment analysis classification in bangla remains less explored in comparison to english for reasons ranging from the non-availability of the datasets to lack of development of models or shared tasks @xcite @xcite . this paper presents our system developed for the blp shared task 2 @xcite . our objective is to provide a comprehensive description of our approach and results. we aim to contribute valuable insights and techniques to the field of bangla sentiment analysis. our team conducted a series of experiments utilising several pretrained sequence classification models where we contributed to fine-tuning and hyper-parameter tun-ing to optimize the performance of these models. 1 we further employed focal loss to counter class imbalance in the data. the resulting system placed 19 th out of 30 submissions in the shared task. ## related works several works on bangla sentiment were released prior to the present shared task challenge. ali et al. (2020) introduces "banglasenti", a dataset of 61,582 bangla words for sentiment analysis, originally developed for social media, with potential uses in emotion detection and opinion mining. this dataset's polarity categorizations are crucial for understanding sentence sentiment. @xcite provides "banglabook," a dataset of 158,065 bangla book reviews categorized as positive, negative, or neutral. pre-trained models outperform manually crafted features, underscoring the need for more training resources in bangla sentiment analysis, and an error analysis reveals common classification mistakes in underresourced languages. @xcite focuses on sentiment analysis in twitter for indian languages, including bangla, hindi, and tamil. the paper presents the first sentiment analysis attempt for these languages and ranks participating teams based on accuracy, achieving a maximum accuracy of 55.67% for hindi. @xcite explores sentiment analysis in bangla using pretrained transformer models, achieving state-of-theart performance with a maximum accuracy of 95% for a two-class sentiment classification task, setting a benchmark for bangla sentiment analysis. ## dataset the data for the blp shared task 2 (hasan et al., 2023a) is curated from two primary sources @xcite : multiplatform bangla sentiment (mubase) and sentnob datasets. these datasets collectively contribute diverse textual data for sentiment analysis in the bangla language. each item of the data is annotated as having a positive, neutral or negative sentiment. automating such annotation is the subject of blp shared task 2. the mubase dataset offers a substantial multiplatform collection. it includes a range of textual data, such as tweets and facebook posts, each meticulously annotated with their respective sentiment polarity labels. the sentnob dataset comprises public comments extracted from various social media platforms, specifically associated with news and video content. this dataset encompasses a broad spectrum of 13 distinct domains, spanning subjects such as politics, education, and agriculture. the datasets were separated into sections for training, development, development-test and test (for final evaluation), consisting of 35,266, 3,934, 3,426 and 6,707 comments respectively. 2 4 experiments this section provides a detailed description of our system's design, methodology, and the steps taken to achieve competitive results in the blp shared task 2. ## results the official evaluation metric for the blp shared task 2 is micro-f1 @xcite . this metric is commonly used in multi-class classification tasks, including sentiment analysis. it combines precision and recall to provide an overall assessment of the system's performance across all sentiment classes. the performance of our sentiment analysis system in the blp shared task 2 is a testament to the effectiveness of our approach. we achieved a micro-f1 score of 0.6768 (see ## conclusion in this paper, we detailed the approach and methodologies we used for the blp shared task 2: sentiment analysis of bangla social media posts. our system demonstrated a strong performance, achieving a micro-f1 score of 0.6768, signifying its proficiency in classifying bangla social media posts into positive, negative, and neutral sentiments. our team's system was ranked 19 th among the 30 th teams that participated in the task. through data preprocessing, model selection, hyperparameter tuning, and the incorporation of advanced techniques such as focal loss, we optimized our system to excel in sentiment analysis tasks for the bangla language. our results substantially outperformed baseline models, underscoring the effectiveness of our strategies. our work contributes to the advancement of sentiment analysis in bangla social media, enabling a deeper understanding of user sentiment and trends in the bangla-speaking community. we believe that our system's success marks a significant step toward improving sentiment analysis in underrepresented languages, and we look forward to further advancements in this field.